An arcade browser prototype for practising Singapore Sign Language (SgSL).
Sentence-shaped speech clouds drift down towards your crystal, and you clear
each one by performing its signs in order before it lands. Recognition is
real rather than mocked: the webcam feed goes through MediaPipe hand tracking
into a small trained classifier (public/models/model_signs.json), so a sign
only counts when you actually hold it.
Prototype. Sentence examples and sign order are placeholder content and should be reviewed by an SgSL expert before any release.
You need Node.js and npm — install with nvm.
git clone https://github.com/junwxn/sign-game-starter.git
cd sign-game-starter
npm i
npm run dev| Script | What it does |
|---|---|
npm run dev |
Start the dev server |
npm run build |
Production build |
npm run preview |
Serve the production build |
npm run lint |
ESLint |
npm run format |
Prettier |
Camera play needs localhost or HTTPS and camera permission. The in-app
Camera check scene verifies the webcam, the hand tracker and the sign model
before gameplay starts. No camera? Pick Keyboard Demo Mode in the run setup
menu, or append ?input=keyboard to the URL.
- Sentence run — the main single-player mode; also quick match against a simulated opponent, local versus, and a Sign Library with per-sign and sentence-quest pages.
- Difficulty sets sentence length — easy builds 2-sign foundations, normal connects 3 signs, hard practises full 4+ sign sentences.
- Progress and settings persist in
localStorage; reset from Settings.
The trained model covers 11 labels, and the word and sentence banks are
filtered to them: coffee, eat, finish, go, good, home, morning,
now, toilet, want, where.
TanStack Start · TypeScript · React · Tailwind CSS · MediaPipe Tasks Vision
| Path | Contents |
|---|---|
src/routes/ |
TanStack Start file routes; index.tsx holds the scene machine |
src/components/game/ |
Every game scene and UI piece |
src/game/ |
Sign and sentence data, run engines, save state |
src/cv/ |
Hand tracking, feature extraction, streaming verifier, model loader |
src/recognizer/ |
Recognizer contract, live webcam and keyboard-mock adapters |
src/config.ts |
All game-feel and CV tunables |
public/models/ |
The trained sign model |
src/cv/signfeat.js is numerically parity-locked with the Python trainer -
don't edit or port it.
- Sign reference GIFs come from the NTU SgSL Sign Bank; they are linked, not vendored, and are not licensed for reuse. See THIRD_PARTY_ASSETS.md.
- This repo is connected to Lovable, so pushed commits sync back into the editor — avoid rewriting published history. See AGENTS.md.