gigastt

Embeddable on-device Russian speech-to-text — one Rust binary, no cloud, MIT-clean weights.

English | Русский

---

gigastt turns any machine into a private Russian speech-recognition server — or embeds the same engine into a Rust app or an Android binary. It runs the open GigaAM v3 model fully on-device via ONNX Runtime: no cloud, no API keys.

At a glance

Private, on-device	Embeddable + streaming	Accurate Russian	Tiny & real-time
No cloud, no keys — after the one-time model download the runtime is 100% local. MIT engine on MIT weights, commercial-ready.	One static binary, a C-ABI FFI for mobile, or the gigastt-core crate — with incremental WebSocket partials, no Python.	Most accurate on 3 of 4 Russian domains: far-field 4.08%, phone 18.50%, YouTube 10.91%; statistical tie on clean read.	~225 MB INT8 model, RTF ~0.10 (~10× real-time on CPU), 0.94 s cold-start.

WER clean 3.55% / far-field 4.08% / phone 18.50% / YouTube 10.91% · held-out CV 2.63% (beats Vosk+FW) · FLEURS 5.26% (FW 3.84 leads) · RuLS 4.21% (beats Vosk+FW) · SOVA device: Vosk ahead · ToneWebinars: FW 8.33 leads (gigastt 13.0) · RTF ~0.10 · Model ~225 MB INT8 · Cold-start 0.94 s · RAM ~46 MB resident (~277 MB ps) · ~66 MB pool-2 (~510 MB ps) · Streaming first partial ~0.78 s

GigaAM v3 rnnt head, INT8, Apple M1 CPU, 1000 samples/domain (FLEURS n=775), failures = 100% WER, 95% bootstrap CIs. Every competitor is measured like-for-like through the same harness, manifests, and normalization.

How it compares

WER (%) on four Russian domains, lower is better — plus every axis that decides a deployment. gigastt is the rnnt head, INT8.

Engine	Clean	Far-field	Phone	YouTube	RTF	Disk	Peak RAM	Cold-start	Streaming	Punct.
gigastt (GigaAM v3 rnnt)	3.55	4.08	18.50	10.91	0.10	~225 MB	~46 MB · ~66 MB pool-2	0.94 s	Yes — incremental WS	Yes
Vosk 0.54 (Zipformer2)	2.97	6.29	22.74	17.24	~0.03	966 MB	560 MB	1.16 s	Yes (server)	Add-on
T-one (beam + LM)	6.61	14.62	21.73	23.23	0.065	138 MB + 5.5 GB LM	—	—	Yes (300 ms)	No
T-one (greedy, no LM)	7.85	17.22	22.37	26.54	0.065	138 MB	672 MB	1.87 s	Yes (300 ms)	No
whisper.cpp (Large v3)	15.26	17.91	32.73	22.61	0.36–0.77	2.9 GB	—	—	No	Yes
faster-whisper (Large v3)	15.53	17.34	24.93	15.45	>1.0	2.9 GB	2619 MB	8.2 s	No	Yes
faster-whisper-turbo	14.45	18.30	26.58	15.45	>1.0	1.6 GB	2154 MB	6.8 s	No	Yes

Conditions: Apple M1, CPU EP, INT8/greedy, 1000 samples/domain (clean read 992; turbo = 300-sample slice), 95% bootstrap CIs. Clean read 3.55 (2.9–4.2) overlaps Vosk 0.54 2.97 (2.4–3.6) — a statistical tie; far-field / phone / YouTube wins are CI-separated. RTF > 1.0 = slower than real-time on CPU. gigastt RAM is the resident footprint (dirty + compressed pages, macOS footprint) after warm decodes on Apple M1 Pro (INT8): ~46 MB at --pool-size 1, ~66 MB at the default --pool-size 2; ps RSS reads ~277 / ~510 MB because it counts the shared memory-mapped model, whose clean pages the OS reclaims under pressure. "—" = not measured. Full methodology and caveats: Benchmarks.

Raspberry Pi / edge: performance on Pi hardware is not yet measured — no RTF, RAM, or cold-start claims for edge devices; status and protocol: Benchmarks § Edge / Raspberry Pi and the edge roadmap.

Streaming: the Whisper engines are offline-only — no partials while you speak. gigastt streams genuine incremental WebSocket partials (~0.78 s to first partial on CPU) from one self-contained binary with no Python; Vosk-server and T-one (300 ms chunks) also stream. So streaming is gigastt's clear win over the Whisper family; over Vosk / T-one the edge is packaging — incremental partials plus a C-ABI FFI in a single binary — not lower latency.

Punctuation & casing: gigastt outputs readable Russian out of the box — native on the e2e_rnnt head, or via a small bundled RuPunct + ITN pass on the default rnnt head (--punctuation / --itn, auto-downloaded). That matches the Whisper engines (punctuated natively) and beats the Russian specialists — Vosk needs a separate recasepunc add-on (a model of its own, comparable in size to the recognizer) and T-one emits none.

Scope & honest caveats

Where rivals win, and when not to reach for gigastt:

• Clean read is a tie, not a win — gigastt 3.55% (2.9–4.2) vs Vosk 0.54 2.97% (2.4–3.6); the CIs overlap and Vosk's point estimate is slightly ahead.
• Russian-first, narrowly multilingual — the default rnnt / e2e_rnnt heads are Russian-only; the opt-in ml_ctc / ml_ctc_large heads add just ru/en/kk/ky/uz. For real breadth use Vosk (20+ languages) or whisper.cpp / faster-whisper / sherpa-onnx (~99). gigastt is a specialist.
• Not the speed leader — Vosk (RTF ~0.03) and T-one (~0.06) are faster; gigastt (~0.10) is comfortably real-time, not the fastest.
• RAM is tiny but easy to misread — resident footprint is ~46 MB at --pool-size 1 / ~66 MB at the default pool 2, the lightest in the table (Vosk 0.54 is 560 MB, T-one greedy 672 MB; an extra pool slot costs only ~20 MB resident). But ps / Activity Monitor shows ~277 / ~510 MB because RSS counts the shared memory-mapped model; the OS reclaims those clean pages under pressure, so the resident figure is what you actually need to budget.
• Streaming is buffered/chunked over an offline RNN-T, not a natively streaming acoustic model; ~0.78 s TTFP is not a lowest-latency claim.
• Training-data overlap — GigaAM v3 is trained heavily on Golos; Golos / OpenSTT numbers are best-case in-distribution upper bounds. Held-out public sets (CV / FLEURS / RuLS / SOVA / Podlodka / ToneWebinars) give a second column — see Benchmarks.

Install

# Homebrew (macOS arm64 / Linux x86_64)
brew tap ekhodzitsky/gigastt https://github.com/ekhodzitsky/gigastt && brew install gigastt

# crates.io — needs protoc on PATH (brew install protobuf / apt install protobuf-compiler)
cargo install gigastt

# Prebuilt image from GHCR (CPU, multi-arch amd64+arm64; CUDA variant: :cuda)
docker pull ghcr.io/ekhodzitsky/gigastt:latest

# Or build your own image (CUDA: Dockerfile.cuda; bake the model with --build-arg GIGASTT_BAKE_MODEL=1)
docker build -t gigastt . && docker run -p 9876:9876 gigastt

Embedding instead of serving? npm install gigastt (Node.js) · pip install gigastt (Python on PyPI) · Swift / Kotlin bindings in progress — all wrap the same engine, model side-loaded: In-process quickstarts.

The GigaAM v3 INT8 model (~225 MB) downloads on first run (lean bundle from GitHub Releases). Runtime is INT8 only — no FP32 download or inference path.

Building also fetches a prebuilt onnxruntime over the network (ort's default download-binaries); the on-device / no-cloud guarantee covers runtime inference, not the build. See Architecture for air-gapped builds.

Quickstart

$ gigastt transcribe recording.wav
Привет, как дела?

# Batch-process a whole folder (txt + json per file, 2 workers):
$ gigastt transcribe-batch samples/ out/

# Or watch a folder and transcribe files as they are dropped in:
$ gigastt watch inbox/ out/ --move-to inbox/done/

# Or run the server — WebSocket + REST + SSE on one port (loopback only):
$ gigastt serve
# WebSocket  ws://127.0.0.1:9876/v1/ws
# REST       http://127.0.0.1:9876/v1/transcribe
# OpenAI     http://127.0.0.1:9876/v1/audio/transcriptions

Capabilities

Capability	Support
Heads	rnnt (34-token char, default — lowest WER) · e2e_rnnt (1025-token BPE, punctuation / casing / ITN baked in) · ml_ctc / ml_ctc_large (GigaAM Multilingual charwise CTC, 220M / 600M, 71-token multilingual char — ru/en/kk/ky/uz)
Post-processing	optional punctuation, casing & Russian ITN — native on e2e_rnnt, or a bundled RuPunct + ITN pass on rnnt (auto-downloaded; --punctuation / --itn), overridable per request (?punctuation= / ?itn= / ?vad=)
Delivery	static binary · C-ABI FFI cdylib (Android / mobile) · gigastt-core crate (no server deps)
Execution providers	CPU (any platform) · CoreML EP (macOS ARM64) · CUDA 12+ (Linux x86_64) · NNAPI (Android) · ANE (--features ane, macOS ARM64 — encoder ≈15.6× on the Neural Engine, warm e2e ≈10× over the CPU build, WER ≈1.11% vs ort; file-mode only) · Candle/Metal (--features candle, experimental — output byte-for-byte identical to ort)
Streaming	incremental WebSocket partials · REST + SSE for files · OpenAI-compatible /v1/audio/transcriptions · single port 9876
Audio in	WAV · M4A/AAC · MP3 · OGG/Vorbis · OGG/Opus (.opus) · WebM/Opus (browser MediaRecorder) · FLAC (auto mono mix for multi-channel)
Stereo telephony recordings	Optional channel-speaker mode (--stereo-speakers CLI / channels=split REST) labels the left/right channels as speaker_0 and speaker_1
Speaker diarization	WeSpeaker ResNet34 embeddings + polyvoice clustering, compiled in by default (speaker model fetched by gigastt download, --skip-diarization to opt out) — offline files opt in per request (?diarization=true, exclusive with channels=split), live sessions via WS Configure; words & segments gain speaker labels
Async jobs	Long-file / batch transcription queue via /v1/jobs (opt-in with --enable-jobs): submit, poll, cancel, SSE progress, retry, and TTL eviction
Client SDKs	Typed WebSocket clients for protocol v1.0 with reconnect honoring retry_after_ms: Go (sdks/go) · TypeScript @gigastt/client (sdks/js)
Export	JSON · TXT · SRT · VTT · Markdown — per-word timings + confidence, or segment-level (?segments=true JSON, ### [mm:ss] Markdown)
Server hardening	loopback-only by default · origin allowlist · per-IP rate limiting · graceful drain · Prometheus /metrics on a separate port · loopback-only model hot-reload (POST /v1/admin/reload)

Documentation

Guide	Contents
Docs index	Full map of guides under docs/
Workbook	Scenario-driven recipes (EN + RU): install → transcribe → stream → deploy
API	WebSocket protocol, REST + SSE, jobs, admin reload, error codes, client examples
Benchmarks	WER / RTF / footprint vs 6 engines across 4 Russian domains, with caveats
Architecture	Pipeline, crates, hardware acceleration, INT8 quantization, project layout
Android / FFI	Embedding via the C-ABI on Android
CLI · Deployment · Security · Troubleshooting	Reference & ops

Requirements

Rust 1.88+, protoc on PATH (build-time only — the quantizer crate regenerates ONNX types). macOS 14+ (Apple Silicon, CoreML) or Linux x86_64 (optional NVIDIA CUDA 12+). ~250–400 MB disk for the lean INT8 install + binary (optional punct/VAD side models extra), ~66 MB resident RAM at the default --pool-size 2 (~46 MB single-session; ps RSS reads ~510 / ~277 MB because it counts the shared memory-mapped model). The gigastt-core crate has no server dependencies — embed it directly: gigastt-core = "2.17".

License

MIT — see LICENSE.

Benchmark data under benchmark/ is not MIT: OpenSTT (openstt_*, CC BY-NC 4.0) and Golos (golos_*, Sber Public License) transcripts keep their non-commercial licenses. See NOTICE and benchmark/DATA_LICENSE.

Acknowledgments

• GigaAM by SberDevices — the speech recognition model
• onnx-asr by @istupakov — ONNX export & reference
• ONNX Runtime · ort — inference engine & Rust bindings