GestureSynth Play
Play Gesture Synth online.
Use your left hand for chords, then shape volume, octave, and synth tone with your right hand.
PC recommended. Use a desktop or laptop for reliable camera tracking and audio.
Play Gesture Synth
Turn a webcam view into a playable synth.
How Gesture Synth works
Gesture Synth is a browser hand-gesture instrument built around two-hand camera tracking. Choose Start playing to enable audio and camera access, then keep both hands visible while the browser converts recognized hand shapes into harmony and synth movement. The left hand decides where you are in the scale; the right hand decides how that harmony is voiced and expressed.
The experience is intentionally different from a piano roll or on-screen keyboard. You do not click a separate control for every note. Instead, a small gesture vocabulary stays active while you move, so changing a chord can feel closer to conducting or shaping an instrument in space.
Gesture controls at a glance
Start with degree selection and Quality 1. Add inversions, sevenths, octave changes, volume, and filter only after the basic chord change is stable. The full chord guide explains every I-VII gesture and right-hand quality in detail.
A five-minute first practice session
- Start the play experience, allow camera access, and wait until both hands are recognized.
- Hold one left finger and one right finger until you hear a stable I chord.
- Move the left hand through I-IV-V-I while keeping the right-hand quality unchanged.
- Repeat the loop slowly until transitions stop flickering between neighboring gestures.
- On the next pass, move the right hand higher and lower for volume, then add a small wrist tilt for filter movement.
This separates recognition practice from musical expression. If all controls change at the same time, it is difficult to tell whether a missed chord came from the left-hand degree, the right-hand quality, or a tracking problem. A short repeatable loop makes each source easier to diagnose.
Camera and browser requirements
Use a current desktop browser with camera permission and Web Audio support. Place the camera where it can see both hands without clipping fingertips, and prefer soft front lighting over a bright window behind you. A busy background matters less than clear hand edges and enough separation between the two hands.
Camera permission is requested only after you start. Gesture Synth does not request microphone access. Hand landmarks are processed in the browser with MediaPipe; the application does not upload or store your camera video. See the webcam privacy guide for the data path and the browser compatibility guide for setup notes.
Troubleshooting common problems
A hand keeps disappearing
Move slightly farther from the camera, keep the wrist and fingertips in frame, and separate overlapping hands before changing gestures.
The chord changes by itself
Hold each shape briefly. Small landmark movement near a finger threshold can look like a new gesture if you move too quickly.
Camera permission was denied
Use the browser site-permission control to allow the camera, confirm the intended webcam is selected, then reload the play page.
Tracking works but there is no sound
Confirm browser audio is enabled and show valid gestures with both hands. A closed right-hand fist intentionally produces silence.
For a longer diagnostic checklist, use the hand-tracking troubleshooting guide.
How the controls work together
A complete musical state combines several decisions. The left hand establishes the scale degree and major/minor context; the right hand supplies the voicing and expressive motion. That means a chord can change even when only one visible detail moves. Turning the left wrist can change harmonic context, extending the right thumb can move the voicing down an octave, and raising the right hand can change volume without changing the chord root.
When learning, deliberately freeze the controls you are not practicing. To work on degrees, keep the right hand on Quality 1 at a comfortable height. To work on voicings, hold degree I with the left hand. To practice expression, keep both finger patterns stable and move only right-hand height or wrist tilt. This isolation method turns a complicated two-hand instrument into a sequence of smaller repeatable exercises.
Once those layers are reliable separately, reconnect them with a short progression before attempting a full song. The goal is not maximum movement; it is a gesture vocabulary that stays readable to both the camera and the player. Clear transitions usually sound more musical than fast but ambiguous motion.