A real-time Punjabi-to-English translation app for Android. A Punjabi speaker talks near the phone; you read the English translation on screen at conversational cadence.
Everything runs on-device — no cloud APIs, no data leaves the phone.
- Tap Start to begin listening
- Say "Roxy" to activate translation
- Punjabi speech is transcribed and translated to English in real time
- Say "Neeko" to return to standby
- The app keeps running with the screen off or locked
Mic (16kHz PCM) → Silero VAD → IndicConformer-pa STT → NLLB-200 Translation → Display
- Speech-to-Text: IndicConformer-pa (int8 quantized, 142 MB) — monolingual Punjabi, always outputs Gurmukhi script
- Translation: NLLB-200 distilled-600M with KV-cache decoder — Gurmukhi Punjabi (
pan_Guru) to English - Voice Activity Detection: Silero VAD (629 KB, bundled in APK)
- Wake Word: Vosk on-device speech recognition for "Roxy"/"Neeko" commands
- Runtime: sherpa-onnx for STT inference, ONNX Runtime Mobile for translation
- Android device (arm64-v8a), minSdk 26
- ~1.5 GB free storage for model files
- ~1.5 GB RAM during active translation
# Clone
git clone https://github.com/johnathanneals-dev/RoxyTranslate.git
cd RoxyTranslate
# sherpa-onnx AAR (not tracked in git)
# Download from https://github.com/k2-fsa/sherpa-onnx/releases
# Place at app/libs/sherpa-onnx-*.aar
# Build
./gradlew assembleDebug
# Install
adb install -r app/build/outputs/apk/debug/app-debug.apkModels are imported in-app (Settings → Import model files). No internet permission — pick files from device storage. Filenames must match exactly:
| File | Role |
|---|---|
model_int8.onnx |
IndicConformer STT |
tokens.txt |
STT tokens |
tokenizer.json |
NLLB tokenizer |
encoder_model_int8.onnx |
NLLB encoder |
decoder_model_quantized.onnx |
NLLB decoder |
decoder_with_past_model_quantized.onnx |
NLLB decoder (KV cache) |
Developer fallback (adb):
adb push model_int8.onnx /sdcard/
adb push tokens.txt /sdcard/
adb shell run-as com.johnathanneals.roxytranslate mkdir -p files/indicconformer_pa files/nllb
adb shell run-as com.johnathanneals.roxytranslate cp /sdcard/model_int8.onnx files/indicconformer_pa/
adb shell run-as com.johnathanneals.roxytranslate cp /sdcard/tokens.txt files/indicconformer_pa/
# …same pattern for nllb/* filesModel sources:
- IndicConformer: ai4bharat/indicconformer_stt_pa (CTC branch, int8 quantized)
- NLLB-200: Xenova/nllb-200-distilled-600M (split decoder format for ONNX Runtime). NLLB weights are CC-BY-NC-4.0 (non-commercial).
Settings → Microphone gain:
| Preset | Gain | Use |
|---|---|---|
| Close | 2× | Arms-length |
| Desk | 4× | Typical indoor |
| ~10 ft | 8× | Quiet room |
| Outdoor ~15 ft | 12× | Outdoor quiet (start here for your work env) |
Also: Android mic source (Voice recognition / Raw mic / Unprocessed) and optional hardware AGC (usually leave off for distant speech). Gain applies the next time translation activates (say “Roxy”).
audio/ Mic capture, wake word (Vosk), gain prefs
pipeline/ STT+VAD manager, translation manager, model import, clause boundary
stt/ IndicConformer STT engine
translation/ NLLB-200 engine + BPE tokenizer
service/ Foreground service, state machine
data/ Room database (sessions + transcripts)
export/ Session log file export
Key design decisions:
- Conversation mode (default) — ~2.8s max speech chunks + short silence so live talk produces frequent lines (keep-up priority)
- Dual UI — large English + Gurmukhi source; source never waits on NLLB
- MT queue decoupled from STT — English backfill is FIFO and does not block new source lines
- No-drop audio backlog — merges under load instead of discarding speech
- KV-cache NLLB — encoder + decoder-init + decoder-with-past
- No INTERNET permission — models imported via Storage Access Framework
- Not true streaming ASR — offline Conformer still needs a chunk boundary; rapid monologues lag
- Fast native Punjabi with few pauses is the hardest case; Conversation mode is the best tradeoff on this stack
- English quality can be choppier in Conversation mode (short chunks vs SOV clauses)
- Overlapping multi-speaker audio is not diarized
- Proper noun confusion; model re-import after uninstall
- High software gain can clip if the speaker is close
Apache License 2.0 — see LICENSE.
Note: The NLLB-200 model is licensed under CC-BY-NC-4.0 (non-commercial use).