Experiences Gallery

The gallery on cloud.pieeg.com (opens in a new tab) (and the self-hosted dashboard) is a card-based launcher for immersive EEG experiences. Every entry is lazy-loaded from a single registry — adding one does not grow the initial dashboard bundle.

Open the gallery with V, or from the dashboard header. Search and filter by category (VR, AR, XR/MR, BCI, ML, EOG, Face/EMG, Audio, Games, Lab, Meditation). Featured cards currently highlight octoCaptcha, Neural Flight, PiEEG Virtual Lab, Room Buddy, Mind Reader, and Spoon Bend.

How to run them

Every experience must explain itself in the UI: a How it works overlay on first open, a header button to reopen it, and honest sensor limits. Do not bury that only in a README.

Featured

Liveness & identity

Learn

XR, VR & AR

Audio

Games

BCI & machine learning

See also Detectors for the shared useFocus / useRelax / useBlink / useBandPowers hooks many of these sit on.

Avatar, face & EMG

Aura VR (palm EMG + IMU)

These need Aura VR. Orientation comes from the IMU; EMG is for grip / pen-down — not wrist pose.

Lab & recording

Integrations (need PiEEG Server)

These send commands or OSC and will not work in the 100% cloud tab. Use self-hosted PiEEG Server, or Local Bridge where OSC is involved.

Creating an Experience

Cloud path: cloud/src/experiences/. Self-hosted path: dashboard/src/experiences/. Same contract.

import { useRef, useEffect } from "react";
import type { ExperienceProps } from "../registry";
import { useFocus, useRelax, useBlink } from "../../hooks/detectors";
 
export default function MyGame({ eegData, onExit }: ExperienceProps) {
  const { state: focus } = useFocus(eegData);
  const { state: relax } = useRelax(eegData);
  const { state: blink } = useBlink(eegData);
  const canvasRef = useRef<HTMLCanvasElement>(null);
 
  useEffect(() => {
    let raf: number;
    function loop() {
      const f = focus.current.focus;       // 0–1
      const r = relax.current.relaxation;  // 0–1
      const blinkCount = blink.current.count; // edge-detect vs last count
      // --- your game logic here ---
      raf = requestAnimationFrame(loop);
    }
    raf = requestAnimationFrame(loop);
    return () => cancelAnimationFrame(raf);
  }, []);
 
  return (
    <div style={{ position: "fixed", inset: 0, background: "#000" }}>
      <canvas ref={canvasRef} />
      <button onClick={onExit} style={{ position: "absolute", top: 12, left: 12 }}>
        ← Exit
      </button>
    </div>
  );
}

const MyGameExperience = lazy(() => import("./my-game/MyGame"));
 
// Add to EXPERIENCES array:
{
  id: "my-game",
  name: "My Game",
  description: "One-sentence summary.",
  tag: "Focus",
  gradient: ["#ec4899", "#8b5cf6"],
  component: MyGameExperience,
  author: "Your Name",
}

The gallery picks it up automatically. Each experience is code-split — no impact on initial load.

ExperienceProps

Set requiresServer: true or requiresAura: true on the registry entry so the gallery can badge the card.

Shared detector hooks (useBandPowers, useBlink, useFocus, useRelax) are documented in Detectors.

Advanced: Eye Track (EOG gaze estimation)

Eye Track bypasses the detector hooks and reads the ring buffer directly — custom signal processing, multi-phase calibration, and a trained model.

How it works

1. EOG extraction — Horizontal gaze ≈ Fp2 − Fp1 (differential), Vertical ≈ (Fp1 + Fp2) / 2 (common-mode). The corneal-retinal dipole (~0.4–1.0 mV) shifts with gaze angle.
2. 5-point calibration — Fixate center, up, down, left, right for 2.5 s each. Mean EOG features per target.
3. Polynomial ridge regression — Features [1, h, v, h², h·v, v²] fit with λ = 0.01 (Gaussian elimination). Captures nonlinear response at extreme angles.
4. Online adaptive learning — New (EOG, target) pairs ~4 Hz; model refits every 12 samples. Pause/resume learning.
5. Persistence — Model + samples in localStorage; load on the next session.
6. Algorithm editor — Edit the gaze function live in a code panel.

Key code patterns

// Direct ring-buffer read (no hooks)
function readEOGFeatures(eeg: EEGData, windowSamples: number) {
  const fp1 = eeg.buffers.current[0]; // Fp1
  const fp2 = eeg.buffers.current[1]; // Fp2
  // Slide backwards from writeIndex
  for (let i = 0; i < windowSamples; i++) {
    const idx = (writeIndex - windowSamples + i + bufferSize) % bufferSize;
    sumH += fp2[idx] - fp1[idx];       // horizontal
    sumV += (fp1[idx] + fp2[idx]) * 0.5; // vertical
  }
  return { hEOG: sumH / windowSamples, vEOG: sumV / windowSamples };
}
 
// Polynomial feature expansion
const feat = [1, h, v, h*h, h*v, v*v]; // 6 features
let x = 0;
for (let i = 0; i < feat.length; i++) x += feat[i] * model.weightsX[i];

Related