You put on the band, open the app, and study. StudyTwin reads your body, calculates your state, and adapts your session — automatically.
ML inference at the edge. Sync in the cloud. Adaptation in the browser. Pre-analysis in the GPU cluster.
Three validated biomarkers of cognitive load, captured in real time. The Grove GSR sensor outputs a signal conditioned by an LM324 op-amp. MAX30102 extracts inter-beat intervals via 880nm infrared PPG for HRV. Blink rate runs in-browser via MediaPipe Face Mesh — zero extra hardware.
The ESP32 WROOM-32 runs a 1D-CNN trained on Edge Impulse, performing 4-class cognitive state inference entirely on-device at 47ms latency. Signal preprocessing — EMA smoothing (alpha = 0.28), baseline drift correction, RMSSD extraction — runs in Arduino C++ firmware.
Firebase Realtime Database receives the ESP32 JSON payload and immediately pushes it to all subscribed browser clients via onSnapshot(). Mean propagation latency: 154ms. Firebase Auth handles per-user isolation. Firestore stores session history for longitudinal analysis.
The dashboard fuses the three biosignals into a final CLI score, drives the adaptive Pomodoro engine, renders the 3D Digital Twin, and generates session insights. All adaptation logic runs client-side — it functions even if Firebase degrades.
Before each session, TRIBE v2 — Meta's brain-predictive foundation model trained on 1,115 hours of fMRI data from 720 subjects — analyses your study material and predicts which brain circuits it will demand. Content Demand Scores for three regions (executive, language, visual) are written to Firebase session metadata. The adaptive engine uses these scores to set personalised session parameters before you put on the headband.
When your brain works hard, your sympathetic nervous system activates and your skin conducts electricity differently. The Grove GSR sensor on your wrist detects this in real time — a reliable, well-studied marker of cognitive load.
The variation between heartbeats is deeply meaningful. High variability means your body is calm. When HRV drops, your nervous system is under load. MAX30102 measures this continuously via 880nm infrared PPG.
Normal: 15-20 blinks per minute. Deep focus or fatigue drops this below 10. Your webcam detects this via MediaPipe Face Mesh — no hardware, no clip, nothing to attach to your face.
Cognitive Load Index
Each signal is normalised to 0-100, then weighted according to predictive power validated against NASA-TLX ground truth. Smoothed with EMA (alpha = 0.28) to prevent jitter.
Your Cognitive Load runs 0-100. StudyTwin maps it to four states and responds differently to each.
Your body is relaxed and your mind has headroom. A great window for complex, new material.
Engaged and handling load well. You're in the optimal learning zone — no changes needed.
Load is climbing. Still functional, but retention is dipping. Time to ease up slightly.
Working memory saturated. Continuing now costs more than it gives. A break restores capacity.
Loaded by complex logic, multi-step problem solving, and abstract reasoning. Demands high working memory.
Loaded by dense reading, vocabulary acquisition, and verbal reasoning. Saturated by technical prose.
Loaded by complex animations, spatial reasoning, and diagram-heavy content. Saturated by high-motion video.