Optical Physiological Bench

A quantitative optical recording and signal-analysis bench in the experiences gallery. It treats an RGB webcam as a timestamped optical instrument, preserves unmodified spatial RGB series alongside derived physiological estimates, and records concurrent PiEEG bio channels on a monotonic timebase.

Open it from cloud.pieeg.com (or the self-hosted dashboard) -> V -> Optical Physiological Bench. Registry id: optical-bench. Tag: Lab / Optical.

Three-tier measurement taxonomy

The bench separates direct sensor readouts from model-derived estimates and experimental spatial/interval metrics in both the UI (TierBadge) and exported files (SCIENTIFIC_CAVEATS):

What it is (and is not)

Live loop

1. Camera acquisition - facingMode: "user", ideal 640x480 at 30 Hz, no audio. Track settings (width, height, reportedFrameRate, exposureMode, whiteBalanceMode) are inspected via MediaStreamTrack.getSettings() and stored in session metadata.
2. Anatomical ROI tracking - MediaPipe FaceLandmarker (@mediapipe/tasks-vision@0.10.34, float16 model, VIDEO mode, GPU delegate with CPU fallback). If landmark detection is offline or no face is present, the sampler falls back to static center-framed polygons and marks facePresent: false.
3. Offscreen polygon sampling - OpticalFrameSampler draws each video frame into a 192x144 offscreen canvas (willReadFrequently: true) and computes exact point-in-polygon spatial averages (r, g, b, spatialStd, clippedRatio, chromR, chromG, pixelCount) across 5 ROIs and R × C forehead grid cells.
4. Uniform temporal resampling - Browser frame delivery is irregular. resampleRgbUniform linearly interpolates raw (camT, R, G, B) onto a uniform 30 Hz grid (UNIFORM_ANALYSIS_HZ = 30) before digital filtering or spectral estimation.
5. Decoupled analysis cadence - Frame extraction runs up to ~30 Hz; full multi-algorithm rPPG, spatial grid coherence, and slow dynamics run every 5th frame (~6 Hz) inside OpticalSignalEngine (rolling buffer of 1800 frames, ~60 s) so canvas rendering stays smooth.
6. Concurrent bio + blink capture - While recording, subscribeEegSamples writes incoming PiEEG channel frames (BIO_CHUNK = 4096) and confirmed frontal EOG blinks from useBlink append event markers { t, text: "eog_blink", source: "blink" }.

Five-panel workbench layout

Tracked ROIs & forehead virtual sensor grid

Anatomical ROIs

Forehead bilinear virtual sensor grid

The forehead polygon is partitioned into R × C quadrilateral cells (2x2 = 4, 3x4 = 12 default, or 4x6 = 24 virtual sensors) labeled F11 through Frc. Cell vertices are computed via piecewise bilinear interpolation across the 6 forehead anchors so the sensor grid tracks head translation, roll, and forehead curvature.

Multi-ROI SNR-weighted fusion

When SNR-Weighted Multi-ROI Fusion (combineRois, default true) is enabled and multiple skin ROIs are active, each ROI's resampled RGB series is weighted by its current spectral SNR before running the primary and comparison rPPG pipelines:

w_k = max(0.2, 10^(clamp(SNR_k, -5, 8) / 20))

Signal processing & rPPG algorithms (Module A)

All three rPPG algorithms run simultaneously on the exact same resampled RGB window so algorithm performance can be compared without confounding frame or ROI differences.

Detrending & zero-phase bandpass filtering

1. Detrending (detrendSignal):
   • moving_average (default): subtracts a centered local moving average with half-window about fs / (2 * 0.75 * lowCutHz).
   • linear: ordinary least-squares linear trend subtraction.
   • none: mean-centering only.
2. Zero-phase bandpass (bandpassZeroPhase): 2nd-order biquad bandpass filter (lowCutHz default 0.75 Hz / 45 BPM, highCutHz default 3.5 Hz / 210 BPM) applied forward and backward (filtfilt-style) to eliminate phase lag and preserve systolic peak timing.
3. Scaling: Filtered waveforms are scaled by 1000 into parts per thousand (ppt).

Spectral PSD & De Haan SNR

computePulseSpectrumAndSnr applies a Hann window and evaluates a discrete Fourier transform across [0.5 Hz, 4.2 Hz] at 0.025 Hz (1.5 BPM) spacing, followed by 3-point parabolic interpolation around the passband peak bin for sub-bin frequency resolution.

Spectral SNR (De Haan & Jeanne, 2013) is computed in dB as the power ratio of the fundamental (± 0.12 Hz around f0) plus second harmonic (± 0.15 Hz around 2 f0) to all remaining in-band noise power:

SNR_dB = 10 * log10( (P(f0 ± 0.12 Hz) + P(2 f0 ± 0.15 Hz)) / (P_passband - P_signal) )

Confidence is mapped linearly from SNR_dB in [-3, +8] dB onto [0, 1].

Spatial optical pulsation mapping (Module B)

Each forehead cell Frc is resampled to 30 Hz and analyzed with the active primary algorithm. Clicking any cell in Panel 3 selects that virtual sensor (selectedCellIndex) and plots its regional waveform in Panel 2.

Summary spatial metrics reported in Panel 3:

• Pairwise correlation matrix: N_cells × N_cells Pearson correlation across all virtual sensors.
• Mean spatial coherence: Narrowband magnitude-squared coherence with the reference pulse evaluated across [f0 - 0.08, f0, f0 + 0.08] Hz, averaged over all cells.
• Amplitude uniformity: 1 - std(amp) / mean(amp) clamped to [0, 1].

Slow optical dynamics & lighting quality gates (Module C)

Over the configurable slowWindowSec (15, 30 default, or 45 s), computeSlowOpticalDynamics measures baseline-relative percentage shifts relative to the first 1.5 s of the window:

• Smoothed RGB drift (deltaRPct, deltaGPct, deltaBPct): 2.5 s moving-average trend of dC / C0 * 100%.
• Chromaticity trend (chromDiffTrend): Smoothed slow drift in normalized (G - R)/(R + G + B).
• Low-frequency oscillation (lowFreqOscillation): Zero-phase bandpass in 0.05 - 0.18 Hz (Mayer-wave / vasomotor band) on dG%.
• Interval A vs. Interval B comparison: Splits the slow window into first half (A) and second half (B) and reports dR, dG, dB (in DN), dChromG (%), and ambient shift (%).

Ambient lighting, auto-exposure (AE), and auto-white-balance (AWB) alerts

Slow optical trends are easily confounded by room lighting or camera ISP adjustments. The bench flags lightingShiftDetected: true when any of three gates trip:

Gated pulse-derived metrics (Module D)

computePulseDerivedMetrics detects systolic peaks on the primary filtered waveform (minPeakHeight = mean + 0.25 * std, refractory 0.33 s / max ~180 BPM) and applies three acceptance gates to each inter-beat interval (IBI):

1. SNR gate: primary.snrDb >= config.snrThresholdDb (default 1.5 dB). If false, all peaks are marked rejectReason: "low_snr" and variability metrics read Gated.
2. Physiological bounds: 330 ms <= ibiMs <= 1600 ms (37.5 - 181.8 BPM); otherwise rejectReason: "out_of_bounds".
3. Successive step gate: |ibiMs - prevAcceptedIbi| / prevAcceptedIbi <= 0.28 (28% max beat-to-beat step); otherwise rejectReason: "step_change".

Two clocks & technical validation strip

Optical frames and PiEEG bio samples are captured on independent clocks and are not resampled onto each other during recording:

On recording start, t0Sec is latched and all exported timestamps (camT, procT, bioT, events[].t) are stored as seconds elapsed since t0Sec. Because board transport buffering introduces a variable offset between physical EEG sampling and browser packet arrival, camT and bioT share a common session zero rather than a hardware trigger pulse.

Header technical validation strip

The top bar continuously reports 7 acquisition health indicators over the latest 120 frames:

• FPS: Measured frame rate (>= 18 Hz green).
• dt / Jitter: Mean inter-frame interval ± standard deviation in ms (jitter < 15 ms green).
• Proc Latency: Mean (procT - camT) * 1000 in ms (below 20 ms green).
• Continuity: Percentage of consecutive frames with dt < 65 ms (>= 90% green).
• ROI Track: Percentage of frames with facePresent === true (>= 85% green).
• Drops: Estimated dropped frames when 0.085 s < dt < 2.0 s.
• AE / AWB / Light: Stable vs. SHIFT DETECTED.

Reference validation datasets & offline reprocessing

Built-in synthetic reference datasets

Panel 4 includes three deterministic 30 Hz synthetic datasets (generateValidationDatasetFrames) so algorithms, spectral SNR, spatial maps, and lighting shift detectors can be verified reproducibly without a camera:

Offline reprocessing (reprocessSessionTake)

Because every recorded sample row preserves unmodified roiRgb (forehead, left_cheek, right_cheek, glabella, ambient_ref) and gridRgb (3 × R × C floats), clicking Reprocess Buffer / Take in Panel 4 reconstructs the raw optical frame sequence and re-runs OpticalSignalEngine with any updated algorithm, filter band, detrending mode, ROI selection, or SNR gate. Derived columns (posWave, chromWave, greenWave, bpm, snrDb, confidence, instantBpm, slowDelta*) are updated in place before CSV/JSON export without altering the raw RGB measurements.

Take format optical-bench-take:v1

JSON export (session_YYYYMMDD_HHMMSS.json)

{
  "format": "optical-bench-take:v1",
  "metadata": {
    "sessionName": "forehead_rppg_run01",
    "subjectId": "S01",
    "lightingCondition": "indoor_diffuse_led",
    "notes": "",
    "createdAtIso": "2026-10-17T12:00:00.000Z",
    "algorithmVersion": "rppg-core:1.0.0",
    "cameraSettings": {
      "width": 640,
      "height": 480,
      "reportedFrameRate": 30,
      "deviceLabel": "USB Camera",
      "exposureMode": "continuous",
      "whiteBalanceMode": "continuous"
    },
    "processingConfig": {
      "primaryAlgorithm": "POS",
      "compareAllAlgorithms": true,
      "windowSec": 10,
      "lowCutHz": 0.75,
      "highCutHz": 3.5,
      "detrendMode": "moving_average",
      "snrThresholdDb": 1.5,
      "gridRows": 3,
      "gridCols": 4,
      "activeRois": ["forehead", "left_cheek", "right_cheek"],
      "combineRois": true,
      "slowWindowSec": 30
    }
  },
  "durationSec": 30.0,
  "gridRows": 3,
  "gridCols": 4,
  "caveats": [
    "[Tier 1 - Direct Optical] ...",
    "[Tier 2 - Algorithm Estimate] ...",
    "[Tier 3 - Experimental Metric] ..."
  ],
  "events": [
    { "t": 4.12, "text": "eog_blink", "source": "blink" },
    { "t": 10.0, "text": "breath_hold", "source": "manual" }
  ],
  "samples": [
    {
      "camT": 0.0,
      "procT": 0.002,
      "latencyMs": 2.1,
      "facePresent": true,
      "trackingConfidence": 0.94,
      "motionSpeed": 0.012,
      "qualityFlags": {
        "facePresent": true,
        "trackingConfidence": 0.94,
        "motionArtifact": false,
        "illuminationShift": false,
        "saturationClipped": false,
        "snrPass": true,
        "overallValid": true
      },
      "roiRgb": {
        "forehead": { "r": 184.2, "g": 137.8, "b": 111.5, "clipped": 0 },
        "left_cheek": { "r": 188.1, "g": 140.2, "b": 113.0, "clipped": 0 },
        "right_cheek": { "r": 182.0, "g": 136.1, "b": 110.4, "clipped": 0 },
        "glabella": { "r": 186.0, "g": 139.0, "b": 112.1, "clipped": 0 },
        "ambient_ref": { "r": 90.0, "g": 90.0, "b": 90.0, "clipped": 0 }
      },
      "gridRgb": [184.1, 137.9, 111.4, "...3 x gridRows x gridCols values"],
      "posWave": 0.412,
      "chromWave": 0.389,
      "greenWave": 0.351,
      "bpm": 66.0,
      "snrDb": 5.42,
      "confidence": 0.765,
      "instantBpm": 65.8,
      "slowDeltaRPct": 0.02,
      "slowDeltaGPct": -0.04,
      "slowDeltaBPct": 0.01
    }
  ],
  "bio": {
    "sampleRate": 250,
    "channels": 8,
    "t": [0.0, 0.004, "..."],
    "ch": ["...frame-major packed µV samples"]
  }
}

CSV export (session_YYYYMMDD_HHMMSS.csv)

Each row corresponds to one optical frame and contains 36 fixed columns followed by 3 × R × C forehead virtual sensor columns (F11_R, F11_G, F11_B, ..., Frc_B):

• Timing & tracking: cam_t_sec, proc_t_sec, latency_ms, face_present, tracking_conf, motion_speed
• Quality gates: overall_valid, motion_artifact, illumination_shift, saturation_clipped, snr_pass
• Raw ROI RGB (T1): forehead_R/G/B, left_cheek_R/G/B, right_cheek_R/G/B, glabella_R/G/B, ambient_ref_R/G/B
• Derived rPPG (T2): rppg_POS_ppt, rppg_CHROM_ppt, rppg_GREEN_ppt, pulse_bpm, instant_bpm, snr_db, confidence
• Slow drift (T3): slow_delta_R_pct, slow_delta_G_pct, slow_delta_B_pct
• Virtual sensor grid (T1): F11_R, F11_G, F11_B ... Frc_R, Frc_G, Frc_B

Controls

Limits

• Standard 8-bit RGB webcams are subject to sensor shot noise, rolling-shutter compression, and browser-managed auto-exposure / auto-white-balance unless locked in OS camera drivers.
• Head motion (motionSpeed > 0.25), facial occlusion, and uneven shadows degrade pulse SNR; POS and CHROM reduce specular reflection artifacts relative to GREEN but do not eliminate large motion transients.
• Spatial forehead maps reflect relative optical AC/DC modulation across skin patches, not calibrated depth-resolved perfusion or cortical hemodynamics.
• Optical inter-beat intervals (sdnnOptMs, rmssdOptMs) include vascular pulse-transit dispersion and peak-interpolation jitter at 30 Hz; they are gated when snrDb < snrThresholdDb and must not be equated with clinical ECG HRV.
• Optical and PiEEG bio streams run on independent clocks without hardware trigger synchronization.

Related