A real-time Spider-Man mirror that runs entirely in the browser. Enable your camera and the suit follows your head, upper body, arms, hands, wrists, and fingers.
- Full-body Spider-Man 3D model rendered with Three.js
- Real-time face, pose, hand, and finger tracking with MediaPipe
- Mirrored selfie-camera alignment for natural movement
- Gesture command dial with a three-second photo countdown
- Film-frame photo export including the rendered suit
- English by default, with Simplified Chinese, Japanese, Korean, and Spanish
- On-device processing: camera frames and tracking data are not uploaded
Requirements: a current desktop browser with WebGL and camera support, plus Node.js 18 or newer.
npm install
npm run devCamera access requires localhost or HTTPS. Grant camera permission when prompted, then keep your face and upper body visible while tracking initializes.
npm run build
npm run previewThe browser build is written to dist/client. The build also prepares the server files used by the bundled hosting adapter.
npm run test:sites- React + Vite
- Three.js and Three-VRM
- MediaPipe Tasks Vision
- Kalidokit
- Improve fine-grained hand control, including independent finger joints, wrist rotation, pinch distance, and natural hand-open/hand-close transitions
- Add adaptive smoothing and motion prediction to reduce jitter without making fast movements feel delayed
- Add a short calibration flow for different body proportions, camera positions, handedness, and movement ranges
- Improve hand identity stability so the left and right hands do not swap when they cross
- Improve tracking through partial occlusion, fast motion, hands near the face, and hands leaving or re-entering the frame
- Use landmark confidence to blend unreliable joints back to a stable neutral pose instead of letting the model snap or deform
- Recognize the classic Spider-Man web-shooting gesture and trigger a matching suit pose and web animation
- Add fist, open-palm, pointing, thumbs-up, peace-sign, pinch, grab, and release gestures
- Add two-hand gestures, such as aiming with both hands, crossing arms, and synchronized web shooting
- Support short gesture sequences and held gestures while preventing accidental activation
- Let users choose left-handed, right-handed, or two-handed gesture mappings
- Provide clear visual feedback for gesture recognition, hold progress, cooldowns, and failed recognition
- Add automatic quality profiles for desktop, mobile, and lower-powered devices
- Reduce end-to-end camera-to-render latency and keep tracking stable when the frame rate drops
- Add a lightweight tracking diagnostics panel for FPS, inference time, landmark confidence, and lost-tracking events
- Improve face, shoulder, elbow, and torso alignment during turning, leaning, crouching, and close-up framing
- Handle camera resizing, orientation changes, and mirrored coordinates consistently across browsers
- Add an interactive onboarding guide that teaches camera placement, lighting, framing, and supported gestures
- Add an in-app gesture reference with localized names and animated demonstrations
- Provide recovery guidance when camera permission is denied or the user moves out of frame
- Add reduced-motion, high-contrast, and keyboard-accessible interface options
- Complete localization coverage for Simplified Chinese, Japanese, Korean, and Spanish
- Add automated tests for gesture recognition thresholds, handedness, coordinate mapping, and tracking-loss recovery
- Test the experience across Chrome, Edge, Safari, Firefox, iOS, and Android camera implementations
- Add graceful fallback messaging for browsers without WebGL, WebAssembly, or camera support
- Document performance targets and create a repeatable real-device QA checklist
Camera video is processed locally in the browser and is not sent to a server by this app.
Spider-Man and related characters are properties of their respective owners. The bundled character model and visual assets are provided for prototype and demonstration use; review the original asset terms before any commercial distribution.