PUZZLE #185

Gesture-Controlled Computer Interface

A real-time, touchless computer vision control interface built in Python that translates hand gestures captured via a local webcam into operating system mouse and media control commands. Powered by OpenCV, MediaPipe 21 hand landmarks, EMA cursor smoothing, and multi-frame temporal voting.

Landmarks 21 / 21
FPS 59.8
Latency 1.4 ms
Stable Frames 4 / 7
Computer Vision Feed
60.0 FPS
OPENCV + MEDIAPIPE POINTING
TEMPORAL STABILITY GATE 85%
Gesture Preset Triggers (Click to Test):
RIGHT HAND
Interactive Desktop Playground
Status: [IDLE]
● VIRTUAL DESKTOP RESOLUTION: 1920 × 1080
Pinch Click Target

Move index finger here and pinch thumb & index to trigger a single left click.

Sustained Pinch Drag-and-Drop

Sustain pinch for >0.25s to engage mouse down, drag chip into drop target!

📄 Project_Specs.pdf
Two-Finger Scrollable Document

Section 1: Real-Time Hand Tracking

Using Google MediaPipe Hands, our Python engine tracks 21 skeletal landmarks with 3D normalization.

Section 2: Multi-Frame Temporal Quorum

Rather than making fragile decisions on a single frame, a sliding window of 7 frames requires 4 consistent detections before triggering an action.

Section 3: Exponential Smoothing

EMA filtering completely removes sensor tremor and sub-pixel hand shakes.

Section 4: Click and Drag Mechanics

A sustained pinch transitions the mouse state into mouse down and streams smoothed coordinates.

System Volume Control

Make the "Shaka" gesture (Thumb & Pinky) and raise/lower hand to adjust master audio!

🔊
65%
[19:58:00.1] SYSTEM INITIALIZED Loaded gestures_config.json (v1.0.0)
[19:58:00.2] LANDMARK TRACKER READY Tracking 21 skeletal joints (0..20)
Architectural Clue Implementation
Track landmarks across multiple frames: Decisions are never made from a single transient frame. A sliding FIFO buffer aggregates candidate detections and requires $\ge 4$ consensus frames before state transitions.
Temporal Buffer Inspector (7-Frame Window)
Quorum: 5/7
F1
POINT
F2
POINT
F3
POINT
F4
POINT
F5
POINT
F6
NOISE
F7
POINT
Vector Metric Ratio Threshold State
Thumb-Index Distance 0.041 < 0.055 PINCH
Thumb-Middle Distance 0.182 < 0.055 OPEN
Index-Middle Spread 0.038 < 0.065 ALIGNED
Thumb-Pinky Span 0.312 > 0.280 SPREAD
Local JSON Configuration
Smoothing Alpha (α) 0.35
Min Stable Frames (Quorum) 4 frames
Debounce Cooldown 350 ms
{ "version": "1.0.0", "camera": { "width": 1280, "height": 720, "fps": 30 }, "tracking": { "smoothing_alpha": 0.35, "deadband": 2.0 }, "stability": { "history_buffer": 7, "min_stable_frames": 4 }, "actions": { "pointing": true, "pinch": true, "drag": true } }