Turn your webcam into an instrument
Tunevate is a free, zero-install browser studio that reads your hands through the webcam and turns them into music in real time — chords, filter sweeps, drum patterns and multitrack loops, with the camera feed never leaving your device.
Every page in one place
The live studio holds the whole dashboard. The pages below are focused guides for each instrument and the studio tools.
What Tunevate is
Most ways of making music on a computer put a wall between the idea and the sound: a MIDI keyboard, a DAW, gigabytes of sample libraries, plug-in licensing. Tunevate removes the wall. You open a page, allow the camera, and your hands are the controller.
A machine-vision model (Google's MediaPipe HandLandmarker) tracks 21 points on each hand, 60 times a second, entirely inside your browser. A small music-theory engine maps those hand shapes onto diatonic chords, inversions and octaves for any of the 12 keys, so you land on musical notes rather than random ones. The Web Audio engine (Tone.js) turns the result into sound with no perceptible latency — sampled acoustic pianos and guitars, subtractive synth voices, a 31-slot drum kit, and a looper that snaps every take to a shared clock.
Everything runs client-side. No account, no upload, no server round-trip for the audio or the video. The only thing that ever leaves your device is anonymous usage telemetry, and you can read exactly what that covers on the privacy page.
How to play — a two-minute guide
Two hands, a handful of gestures. This is the whole control scheme.
Pick the chord
Raise 1–5 fingers to choose the scale degree in your selected key (1 = I / tonic, up to 5 = V / dominant). Tilt your wrist to flip between major and minor.
Shape the voicing
Raise fingers to choose the voicing: 1 = root position, 2 = 1st inversion, 3 = add a 7th, 4 = extended colour. Extend your thumb to drop the whole chord an octave.
Perform and record
Tilt your right wrist to sweep the lowpass filter; move your hand up and down for volume. Arm Record, wait for the count-in, and layer loops against the shared clock.
How it was built
Tunevate started as a question: how much of a music studio can run with nothing but a webcam and a browser tab? The first prototype did one thing — count fingers and play a triad. It grew from there in deliberate phases.
- Hand tracking. Wire up MediaPipe, classify left/right, turn raw landmarks into stable finger counts and wrist tilt without jitter.
- Music theory. A pure, unit-tested engine that maps gestures to diatonic chords and voicings across all 12 keys, so a beginner can't play a wrong note.
- Audio engine. A Tone.js layer with sampled instruments, synth voices, an EQ chain and a limiter — swap voices mid-performance without a gap.
- Transport & looper. One shared clock, a metronome, quantised multitrack recording, and a
.tvatesession format you can export and reload. - Drums. A 31-slot kit addressed by 5-bit finger combinations and a 16-step sequencer locked to the transport.
- The dashboard. An arrangement canvas, a layers mixer, and a UI that collapses down to work on a phone.
It is built with Next.js, React and TypeScript in an npm-workspace monorepo — each capability (hand tracking, music theory, audio, transport, looper, drums, session) is its own package with its own tests.