Voice Assistant

Hybrid voice assistant. Supports local and API models.

For local mode hardware requirements: GPU with 8GB VRAM

Installation

uv sync                 # base dependencies
uv sync --extra cuda    # + CUDA 12 runtime libs for GPU STT (see below)

Python 3.11, package manager uv.

Running test utilities

test-wakeword — wake word configuration

Listens to the microphone via openWakeWord, prints Wake word! on detection. Useful for tuning --threshold / --gain without loading VAD/STT.

uv run test-wakeword
uv run test-wakeword --debug-scores          # print per-model scores
uv run test-wakeword --list-devices          # pick a microphone

test-stt — full audio pipeline

Waits for wake word → plays earcon → records speech → stops recording after a silence pause (5 sec by default) → transcribes via faster-whisper → prints text to stdout. Loops; --once for a single command.

uv run test-stt
uv run test-stt --once
uv run test-stt --silence-sec 2 --language ru
uv run test-stt --no-earcon                  # skip confirmation sound

Transcribed text goes to stdout, all status/diagnostics — to stderr.

test-tts — audio pipeline with TTS echo

Same flow as test-stt, but after transcription it speaks the text back via Piper TTS (echo). Useful for testing the full voice pipeline.

uv run test-tts
uv run test-tts --once
uv run test-tts --tts-model ru_RU-irina-medium   # change voice
uv run test-tts --tts-speaker 0                  # multi-speaker models
uv run test-tts --no-tts                         # equivalent to test-stt

Voice configuration: TTS_MODEL=ru_RU-irina-medium in .env or --tts-model. The model is looked up in TTS_MODEL_PATH (default .local/piper) by prefix — e.g. ru_RU-irina-medium will find ru_RU-irina-medium.onnx. Download voices: huggingface.co/rhasspy/piper-voices.

NLU evals

The main graph-eval datasets are split by language: evals/datasets/command_graph_ru.yaml and evals/datasets/command_graph_en.yaml. They validate the full multi-turn dialog: slot filling, continuation, policy, and permit issuance. For diagnosing individual draft extraction, use nlu_drafts_ru.yaml and nlu_drafts_en.yaml.

uv run eval-nlu --language ru
uv run eval-nlu --language en --min-composite 0.85
uv run eval-nlu --language all --repeat 3
uv run eval-nlu-graph --language ru
uv run eval-nlu-graph --language all --repeat 3

Each run saves a full JSON report to evals/reports/. This directory is gitignored; YAML datasets and their schema are versioned golden data.

Observability

Local JSONL logging is always on and writes to logs/voice-assistant.jsonl by default.

• LOGFIRE_TOKEN enables trace export to Logfire.
• Langfuse is enabled only when LANGFUSE_PUBLIC_KEY, LANGFUSE_SECRET_KEY, and LANGFUSE_BASE_URL are all set.
• Langfuse credentials do not require LOGFIRE_TOKEN: Langfuse will receive the same OpenTelemetry traces independently.
• With an incomplete set of Langfuse variables or a backend error, the app continues to work and retains local logs/reports.

When running evals with Langfuse enabled, Datasets, Dataset Runs, links to case traces, and evaluator scores are created without re-running the model.

STT on GPU (CUDA)

faster-whisper runs on top of ctranslate2, which is built against CUDA 12. If you don't have the CUDA 12 toolkit installed, install runtime libs from pip wheels:

uv sync --extra cuda

This installs nvidia-cublas-cu12, nvidia-cuda-runtime-cu12, nvidia-cudnn-cu12 into the venv. Their shared libraries are loaded automatically before importing ctranslate2 (see src/voice_assistant/cuda_bootstrap.py) — no manual configuration needed.

On a system with incompatible CUDA (e.g. CUDA 13) or without a GPU, test-stt will still work: on CUDA initialization failure it automatically falls back to CPU (--stt-device cpu, compute_type=int8).

Local models

Model weights are stored in .local/ (gitignored):

• openwakeword/ — melspectrogram, embedding model, wake word models (hey_jarvis), silero_vad.onnx. Downloaded automatically by openWakeWord on the first run of test-wakeword / test-stt.
• whisper small (~480 MB) is downloaded to the HF cache on the first run of test-stt.
• piper/ — voice models for TTS (.onnx + .onnx.json). Download manually from Piper Voices.

Audio module structure

src/voice_assistant/
├── core/
│   ├── config.py                 # env configuration (.env)
│   ├── device.py                 # audio device selection/listing
│   └── paths.py                  # paths to .local/ models
├── modules/voice/
│   ├── model.py                  # AudioSegment, Transcript
│   ├── stt.py                    # STT protocol
│   ├── tts.py                    # TTS protocol
│   └── adapters/
│       ├── wake_word.py          # WakeWordDetector (openWakeWord)
│       ├── vad_silero.py         # VadSilero (ONNX, no torch)
│       ├── earcon_player.py      # EarconPlayer (numpy beep)
│       ├── audio_recorder.py     # AudioRecorder
│       ├── stt_faster_whisper.py # FasterWhisperSTT
│       └── tts_piper.py          # PiperTTS (local TTS)
└── entrypoints/
    ├── wakeword_cli.py           # test-wakeword
    ├── stt_cli.py                # test-stt (state machine)
    └── tts_cli.py                # test-tts (STT + TTS echo)