BioPose Recorder

A live split bench in the experiences gallery: camera + 33-point body stick on the left, live bio traces on the right. Record writes both streams into one take. Review plays the points. Camera pixels never leave the tab.

Open it from cloud.pieeg.com (or the self-hosted dashboard) → V → BioPose Recorder. Registry id: biopose-recorder.

What it is (and is not)

Bio is the connected board — face, scalp, palm, demo, anything useEEG is already writing. Camera-only takes are valid: pose records, bio stays empty.

Live loop

1. Enable camera — facingMode: "user", ideal 960×720, no audio. Video is mirrored (scaleX(-1)).
2. Pose Landmarker — MediaPipe Tasks Vision, lite float16 model, VIDEO mode, one pose. GPU delegate first, CPU fallback. WASM from jsDelivr @mediapipe/tasks-vision@0.10.34.
3. Detect — requestAnimationFrame gated to 20 Hz (POSE_HZ). detectForVideo(video, performance.now()).
4. Pack - image landmarks, 33 × (x, y, z), normalised 0–1. World landmarks are not stored; they only bump score to 1 when present (0.8 if image-only, 0 if absent).
5. Draw - body stick (torso, arms, legs). Face mesh is omitted on purpose.
6. Bio traces - the live ring buffer, same amplitude scale as the dashboard. Not the recorded take until you stop.
7. Blinks - useBlink on the live stream (default channels 0/1). Each confirmed blink writes a blink mark while recording. The detector warms up ~0.8 s. Threshold, duration gate, and refractory are on that page. This is frontal EOG, not eyelid tracking.

If a pose tick arrives more than 2.4 × the 20 Hz budget late while recording, that frame is counted as dropped and not written.

Two clocks

The two streams are not resampled onto each other at capture.

On Stop, every timestamp is subtracted from t0 (first recorded event). Export times start at 0. The streams still have different rates and jitter.

bio:  t = frame.t          →  ~board rate (often 250 Hz)
pose: t = Date.now()/1000  →  ~20 Hz, browser wall clock

That skew is real. Do not treat a pose frame and a bio sample with the same exported t as simultaneous in the lab sense. They share a zero, not a hardware sync pulse.

How review lines them up

Review has one playhead. It does not interpolate pose.

So the body hold-last-samples for up to 150 ms, then goes blank. Click the traces to seek on the bio clock. The scrubber and marks sit on the same relative seconds.

Take format biopose-take:v1

Capture stays in typed-array chunks so a long session does not allocate a JS object per sample.

On stop, chunks flatten into one contiguous take:

{
  "format": "biopose-take:v1",
  "created_at": "2026-09-17T12:00:00.000Z",
  "sample_rate": 250,
  "channels": 8,
  "duration_sec": 12.4,
  "caveats": [
    "Monocular pose is a vision estimate, not mocap.",
    "Bio and camera clocks are independent; review uses last-sample hold.",
    "Camera pixels are not stored. Import plays points, not video.",
    "Occlusion, clothing, and framing drop pose frames.",
    "Blink marks are frontal EOG detections (default Fp1/Fp2), not eyelid tracking."
  ],
  "events": [
    { "t": 1.04, "text": "blink" },
    { "t": 3.21, "text": "reach" }
  ],
  "bio": { "t": [0, 0.004, ...], "ch": [/* frame-major, channels packed */] },
  "pose": { "t": [0.02, 0.07, ...], "packed": [/* stride 101 per frame */] }
}

Pose packing (one frame)

Image space, not metres. x, y in [0, 1] of the camera frame (mirrored on draw). z is MediaPipe’s image-relative depth, not world coordinates.

Bio packing

bio.ch[frame * channels + ch] — same µV values the dashboard already shows. sample_rate is whatever getSampleRate() reports at stop (default 250 if unset). It is metadata, not a resampling step.

Export

JSON is assembled as Blob parts in 2000-number slices. There is no JSON.stringify of the whole session. Precision: bio t 6 decimals, bio values 2, pose t 6, packed pose 5.

Filename: biopose_YYYYMMDD_HHMMSS.json.

Import accepts that object only (format must be biopose-take:v1, lengths must match channels and stride 101). Older takes without the blink caveat still parse.

Events

events is a time-sorted list of { t, text }. Format is unchanged (biopose-take:v1). Two writers share it:

Blink ingest edge-detects state.current.count at 40 Hz (25 ms poll). A count jump of n writes n marks at the same wall t. Marks fire only while Record is on; the detector keeps running in live so it is already warmed. Review can still add or delete any event, including blink.

How a blink is confirmed: average frontal p2p, median+MAD baseline, hysteresis, 60-600 ms duration, 300 ms refractory. Full contract: useBlink.

The live Blink meter is warming / ready / take count. Review counts text === "blink". Jaw, forehead EMG, and head motion also trip frontal p2p: a blink mark is an EOG detection, not an eyelid class.

Controls

A new Record resets the writer. The previous take stays on screen until then.

Limits

• Laptop / phone camera, one body, lite model, ~20 Hz.
• Occlusion, clothing, tight framing, and a late rAF drop pose frames (Dropped meter).
• No magnetometer, no mocap suit, no pixel archive, no class labels.
• Clocks are independent; 150 ms hold (useBlink)is a viewer convenience, not alignment.
• Score is a presence hint, not a quality metric you should threshold science on.
• Blink marks use default channels 0/1. Wrong montage, or a board without a frontal pair, still writes false detections or none.

Related