How to use Gesture Synth

Use a desktop or laptop in a well-lit space. Place the camera so both hands fit comfortably in frame, and keep your hands separated enough for MediaPipe to identify them. Gesture Synth asks for camera access only after you choose Start playing; it never requests your microphone.

1. Open the play page and choose Start playing.
2. Allow camera access, then wait for the hand model and game HUD.
3. Keep your left hand on the left side of the camera view and your right hand on the right.
4. Begin with one clear gesture per hand before adding movement.

Your left hand chooses the scale degree. One through five extended fingers select I through V. VI uses index and pinky; VII adds the thumb. Turn the left wrist to move between major and minor harmony.

No command

Closed fist. Releases the current chord after a short stabilization delay and acts as a silent rest.

I / i

Index finger. Major mode shows I, minor mode shows i.

II / ii

Index and middle fingers. Selects the second scale degree.

III / iii

Index, middle, and ring fingers. Selects the third scale degree.

IV / iv

Four raised fingers with the thumb tucked. Selects the fourth scale degree.

V / v

Five extended fingers. Selects the fifth scale degree.

VI / vi

Index and pinky only. This is a special combination, not a finger-count gesture.

VII / vii

Thumb, index, and pinky. This special combination selects the leading-tone degree.

Your right hand chooses chord quality. One to four raised fingers select the available voicing; a closed fist silences the chord. Keep the shape steady briefly so the stabilizer can commit the gesture.

No command

Closed fist. Selects no chord quality and silences the current chord after a short stabilization delay.

Quality 1

One raised finger. Major mode plays Major, minor mode plays Minor.

Quality 2

Two raised fingers. Plays the first inversion version of the current chord.

Quality 3

Three raised fingers. Major mode plays Major 7th, minor mode plays Minor 7th.

Quality 4

Four raised fingers. Major mode plays Dominant 7th, minor mode plays Diminished 7th.

Add the right thumb to a quality gesture to play that chord one octave lower. Move the right hand higher in the frame for more volume, and tilt the right wrist to sweep the synth filter.

Quality 1 (-8ve)

Thumb plus index finger. Plays the low-octave Major or Minor version.

Quality 2 (-8ve)

Thumb, index, and middle fingers. Plays the low-octave first inversion.

Quality 3 (-8ve)

Thumb, index, middle, and ring fingers. Plays the low-octave seventh chord.

Quality 4 (-8ve)

Five extended fingers. Plays the low-octave Dominant or Diminished seventh.

Leave the default key in place and show one right-hand finger for the simplest quality. With your left hand, move slowly through I, IV, V, and back to I. Once each change is stable, use right-hand height to fade the loop in and out, then add a gentle filter sweep.

When that feels reliable, read the Gesture Synth chord guide for inversions, seventh chords, and a G major example.