A Python-based hand gesture recognition app that allows you to control your system (keyboard presses) using simple finger gestures via webcam. It uses MediaPipe for real-time hand tracking and PyAutoGUI for simulating keypresses.
- Move hand right →
RightArrow - Move hand left →
LeftArrow - Move hand up →
UpArrow - Move hand down →
DownArrow
These gestures simulate keyboard arrow keys in any active application (like car racing games, character movement, or presentations).
Install dependencies with:
pip install opencv-python mediapipe pyautoguiPython 3.8+ is recommended.
gesture_control.py– Main script that runs the gesture control- No other files required; runs directly using webcam (Used DroidCam app for camera)
- Connect your webcam.
- Run the script:
python gesture_control.py- Hold your hand in the center for auto-calibration.
- Move hand up/down/left/right to control.
- Press c to recalibrate anytime.
- Press q to quit.
- Detects hand landmarks using MediaPipe Hands.
- Takes the wrist point (landmark 0) as the reference.
- Calibrates a neutral center position.
- Calculates hand displacement (dx, dy) from center:
- If movement crosses a dead zone threshold → triggers a direction.
- Ensures gestures don’t repeat too fast using a small delay.
- Sends corresponding arrow key presses with PyAutoGUI.
- Calibration System → Press c to reset neutral position.
- Dead Zone → Ignores small jittery movements.
- Gesture Delay → Prevents multiple keypress spam.
- Works in real time with ~30fps.
| Hand Movement | Keys Triggered |
|---|---|
| Move Right | → Right Arrow |
| Move Left | ← Left Arrow |
| Mode Up | ↑ Up Arrow |
| Mode Down | ↓ Down Arrow |
- Good lighting = better tracking.
- Keep only one hand visible.
- Works on Windows, macOS, Linux.
- Can be easily extended (e.g., WASD keys, custom actions).
Built with ❤️ by Pragya using Python, MediaPipe, OpenCV, PyAutoGUI.