Build a webcam air instrument with hand tracking in one HTML file

A Redditor used Claude on Fable 5.2 for about two hours to build a browser instrument with 24 synthesised voices that you play by drawing and plucking strings in the air in front of a webcam.

Done withClaude· Nov 21, 2026

What they did

The poster wanted to try mixing AI coding with motion tracking and music. Over one long session with Claude on Fable 5.2, they described the instrument, tested each version and reported back what was broken or missing. Claude wrote a single HTML file with no build step or backend. It uses MediaPipe hand tracking in the browser: you pinch with both hands and pull apart to leave a string in the air, then sweep a fingertip across it to play. Every sound is made live with the Web Audio API (Karplus-Strong for plucked strings, oscillators and FM for the rest), with a beat sequencer, a looper, chord-following, and video, MP3, WAV and stem export. The code and a live demo are on GitHub Pages.

How it went

The poster reports about two hours from start to finish, 83 requests and about 31 million tokens, nearly all of them cached re-reads, with about 286,000 tokens of output. It ran inside a Claude Max plan, so there was no separate bill. They say gesture detection took the most rounds: telling a pluck from a hold or a pull needed a dead zone against tracking jitter. One tester in the comments found it sometimes too touchy and sometimes slow to respond.

Worth knowing

Commenters in both threads pointed to an earlier Instagram reel of a near identical instrument and said the idea was copied; the poster had not credited it, though that one was not open source. It needs Chrome or Edge on a computer, good light and hands 40 to 80 cm from the camera. The post names the model but not the harness.

Try it

With Claude· 6 parts to fill in

Build a musical instrument I can play in the air with my hands in front of a webcam, as one self-contained HTML file with no build step or backend. Use [MediaPipe or another in-browser hand tracker] and keep the camera feed on my machine. Let me create a string by [a gesture, e.g. pinching with both hands and pulling apart] and play it by [a gesture, e.g. sweeping a fingertip across it], with a dead zone so tracking jitter does not trigger notes. Synthesise every sound in Web Audio with no samples, starting with [list of instruments] in [scales]. Add [looper, beats, recording or export features I want]. Build it in small steps, and after each step tell me exactly what to test and wait for my feedback before moving on.

Square brackets are the parts to fill in. The prompt is ours; the story is theirs.

Where they said it

Related