Jarvice

Like Jarvis from Iron Man, but... spicier.

Jarvice is an open-source voice bridge that lets you talk to your OpenClaw bot using your microphone and speakers. It runs entirely on your local machine — no cloud TTS/STT needed. Speak into your mic, your AI agent answers back with a cute girl voice.

🎤 Mic → faster-whisper (GPU) → text
    → OpenClaw Gateway (HTTP API) → Bot response
        → Piper TTS (CPU) → 🔊 Speaker

Features

Local Speech-to-Text — faster-whisper running on your NVIDIA GPU for near-instant transcription. Handles accents really well (French, Japanese, etc.)
Multilingual Text-to-Speech — Edge TTS (default, free, 70+ languages with auto language detection) or Piper (local, CPU, English only)
OpenClaw integration — talks to any OpenClaw agent (GPT, Claude, Ollama, etc.) via the gateway HTTP API
Wake word detection — optional always-listening mode with openwakeword ("Hey Jarvice!")
Push-to-talk — hold a key to talk, release to send (default mode)
Sentence streaming — starts speaking the response before it's fully generated (low latency!)
Interrupt support — speak over the bot to cut it off mid-sentence
Meta Quest VR ready — architecture designed for future wireless VR voice control
100% local & free — no API costs for voice, everything runs on your machine

How It Works (For Beginners)

Jarvice connects three things together:

Your microphone — you speak into it
Your GPU — it converts your speech to text (using AI model called Whisper)
OpenClaw — an agent framework that manages AI chatbots. It takes the text, sends it to an AI model (like GPT, Claude, or a local model), gets a response, and Jarvice converts that response back to speech using your CPU

Think of it like this:

You speak → [Your GPU converts speech to text] → [OpenClaw sends text to AI] → [AI responds] → [Your CPU converts text to speech] → You hear the response

OpenClaw is the brain that manages everything. It can connect to:

OpenAI Codex (GPT-4o, GPT-5.4) — uses your ChatGPT Plus subscription, no extra cost
Anthropic Claude — via API key
Ollama (Llama, Qwen, etc.) — fully local, runs on your GPU
OpenRouter — access to hundreds of models

Jarvice is just the "mouth and ears" — OpenClaw is the "brain".

Prerequisites

Hardware Requirements

Component	Minimum	Recommended	What it's for
GPU	NVIDIA GPU, 4GB+ VRAM	RTX 3060+ / 8GB+ VRAM	Running Whisper (speech-to-text)
CPU	4 cores	8+ cores	Running Piper TTS (text-to-speech)
RAM	8 GB	16 GB+	General processing
Microphone	Any USB/built-in mic	Dedicated mic (Blue Snowball, etc.)	Voice input
Speakers	Any speakers/headphones	—	Hearing the response
OS	Windows 10/11 or Linux	Windows 11, Ubuntu 22.04+	—

No NVIDIA GPU? You can still use Jarvice with WHISPER_DEVICE=cpu in your .env file. It will be slower (~5-10x) but still works. AMD GPU support is experimental via ROCm.

Step 0: Install the Basics

You need Python and Node.js installed on your machine before anything else.

Install Python (3.10 or newer)

Go to https://www.python.org/downloads/
Download Python 3.11 or 3.12 (either works)
IMPORTANT: During installation, check the box that says "Add Python to PATH"
Click "Install Now"
Verify it works — open a terminal (PowerShell on Windows, Terminal on Mac/Linux):
```
python --version
# Should print: Python 3.11.x or similar
```

Windows tip: If python doesn't work, try python3 or py instead. If none work, Python wasn't added to PATH — reinstall with the PATH checkbox checked.

Install Node.js (18 or newer)

Node.js is needed for OpenClaw (the bot framework).

Go to https://nodejs.org/
Download the LTS version (green button)
Install with default settings

Verify it works:

node --version
# Should print: v18.x.x or newer

npm --version
# Should print: 9.x.x or newer

Install Git

Go to https://git-scm.com/downloads
Download and install for your OS
Use default settings during installation

Verify:

git --version
# Should print: git version 2.x.x

Step 1: NVIDIA GPU & CUDA

Jarvice uses your NVIDIA GPU to convert speech to text. This is much faster than using your CPU.

Check if you have an NVIDIA GPU

# Windows (PowerShell)
nvidia-smi

# Linux
nvidia-smi

You should see something like:

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 550.xx       Driver Version: 550.xx       CUDA Version: 12.x    |
| GPU  Name                    ...
| NVIDIA GeForce RTX 3090      ...
+-----------------------------------------------------------------------------+

If this command fails:

"command not found": You need to install NVIDIA drivers
No NVIDIA GPU listed: You have an AMD/Intel GPU — see the "No NVIDIA GPU?" note above

Install NVIDIA Drivers (if needed)

Go to https://www.nvidia.com/Download/index.aspx
Select your GPU model and OS
Download and install the Game Ready or Studio driver
Restart your computer
Run nvidia-smi again to verify

CUDA toolkit: You do NOT need to install CUDA separately. The Python packages (ctranslate2, faster-whisper) bundle their own CUDA libraries automatically.

Step 2: Install OpenClaw

OpenClaw is the agent framework that manages your AI bot. Jarvice talks to OpenClaw, and OpenClaw talks to the AI model.

# Install OpenClaw globally
npm install -g openclaw

# Verify it installed
openclaw --version
# Should print something like: 2026.3.28

Permission errors on Linux/Mac? Try sudo npm install -g openclaw or use nvm to manage Node.js.

Run the OpenClaw Setup Wizard

openclaw configure

This interactive wizard will:

Ask you to choose a mode (pick "local" for running on your own PC)
Set up the gateway (the local server that Jarvice connects to)
Ask about model providers (where the AI brain lives — see next step)

Follow the prompts. When in doubt, accept the defaults.

Verify OpenClaw is working

# Check model status
openclaw models status

# Open the TUI (terminal chat interface) to test
openclaw tui

Type a message in the TUI. If the bot responds, OpenClaw is working. Press Ctrl+C to exit.

Step 3: Set Up a Model Provider

OpenClaw needs at least one AI model to work. Here are your options, from easiest to most advanced:

Option A: OpenAI Codex (Easiest — uses your ChatGPT Plus subscription)

If you have a ChatGPT Plus ($20/mo) subscription, this is the easiest option. It uses your existing subscription — no extra API costs.

openclaw models auth login --provider openai-codex

This opens a browser window. Log in with your OpenAI/ChatGPT account. Done!

Set it as your default model:

openclaw models set openai-codex/gpt-5.4

Option B: Anthropic Claude (API key required)

Get an API key from https://console.anthropic.com/

Run:

openclaw models auth login --provider anthropic
# Choose "API key" when prompted
# Paste your key

Set as default:

openclaw models set anthropic/claude-sonnet-4-6

Option C: Claude CLI (Reuse your Claude Code subscription)

If you already use Claude Code (Anthropic's CLI tool), you can reuse its login:

openclaw models auth login --provider anthropic
# Choose "Claude CLI" when prompted

This routes through your existing Claude Code authentication. No extra API key needed.

Option D: Ollama (100% Local — Free, but needs a good GPU)

If you want everything running locally with zero cloud dependencies:

Install Ollama from https://ollama.ai

Pull a model:

ollama pull qwen3:8b        # 8B model, needs ~6GB VRAM
ollama pull qwen3.5:27b     # 27B model, needs ~18GB VRAM (better quality)
ollama pull llama3.3:70b    # 70B model, needs ~42GB VRAM (best quality)

Ollama models are auto-detected by OpenClaw. Set as default:
```
openclaw models set ollama/qwen3:8b
```

Which model size? Bigger = smarter but needs more VRAM. Check your VRAM with nvidia-smi. A rough guide: 8B models need ~6GB, 27B need ~18GB, 70B need ~42GB.

Verify your model works

openclaw models status
# You should see your model listed with "Auth: yes" or "Local: yes"

openclaw tui
# Type "hello" and you should get a response

Step 4: Enable the HTTP API

Jarvice communicates with OpenClaw through an HTTP API. You need to enable it.

Where is the config file?

OS	Path
Windows	`C:\Users\YourName\.openclaw\openclaw.json`
Linux	`~/.openclaw/openclaw.json`
Mac	`~/.openclaw/openclaw.json`

Edit the config

Open the file in any text editor (VS Code, Notepad, nano, vim, etc.).

Find the "gateway" section. It looks something like this:

"gateway": {
    "port": 18789,
    "mode": "local",
    "bind": "loopback",
    ...
}

Add the "http" block inside the "gateway" section:

"gateway": {
    "port": 18789,
    "mode": "local",
    "bind": "loopback",
    ...
    "http": {
      "endpoints": {
        "chatCompletions": {
          "enabled": true
        }
      }
    }
}

Be careful with JSON commas! Every item except the last one in a JSON object needs a comma after it. If you get a JSON parse error when starting OpenClaw, you probably have a missing or extra comma.

Find your gateway token

In the same config file, look for:

"auth": {
    "mode": "token",
    "token": "some-long-string-here"
}

Copy that token — you'll need it when configuring Jarvice.

Restart OpenClaw

After editing the config, restart OpenClaw for changes to take effect:

# If running in TUI, press Ctrl+C to stop, then:
openclaw tui

Verify the HTTP API works

# Replace YOUR_TOKEN with your actual gateway token
curl -X POST http://127.0.0.1:18789/v1/chat/completions \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"model":"openclaw/default","messages":[{"role":"user","content":"hello"}]}'

If you get a JSON response with the bot's reply, the API is working!

Windows PowerShell note: curl in PowerShell is actually Invoke-WebRequest, which has different syntax. Use curl.exe instead, or install Git Bash and run the command there.

Step 5: Discord Bot (Optional)

If you want your OpenClaw agent to also work as a Discord bot (so you can chat with it in Discord AND talk to it with Jarvice):

Create the Discord bot

Go to Discord Developer Portal
Click "New Application" → give it a name (e.g., "Grimoire") → click Create
Go to Bot in the left sidebar
Click "Reset Token" → copy the token (save it somewhere safe!)
Scroll down to Privileged Gateway Intents:
- Enable Message Content Intent (required for the bot to read messages)
- Enable Server Members Intent (optional but useful)
Click Save Changes

Invite the bot to your Discord server

Go to OAuth2 → URL Generator in the left sidebar
Under Scopes, check: bot
Under Bot Permissions, check:
- Send Messages
- Read Message History
- Embed Links (optional, for rich responses)
- Attach Files (optional)
Copy the generated URL at the bottom
Open that URL in your browser → select your server → click Authorize

Get your Discord User ID

You need your Discord user ID so the bot only responds to you:

Open Discord → go to Settings → Advanced → enable Developer Mode
Right-click your own username anywhere in Discord
Click "Copy User ID"
It's a long number like 185125313955627008

Add Discord to OpenClaw config

Edit ~/.openclaw/openclaw.json and add this in the "channels" section:

"channels": {
    "discord": {
        "enabled": true,
        "token": "YOUR_BOT_TOKEN_FROM_STEP_1",
        "groupPolicy": "allowlist",
        "streaming": "off",
        "allowFrom": ["YOUR_DISCORD_USER_ID"],
        "guilds": { "*": {} }
    }
}

Replace:

YOUR_BOT_TOKEN_FROM_STEP_1 — the bot token you copied earlier
YOUR_DISCORD_USER_ID — the user ID you copied

Security: "allowFrom" restricts who can talk to the bot. Only the user IDs listed here can send commands. This prevents random people from using your bot/API credits.

Restart OpenClaw

openclaw tui

Your bot should appear online in Discord within a few seconds. Send it a message to test!

Installation

Now that OpenClaw is set up, let's install Jarvice.

1. Clone the repository

git clone https://github.com/youruser/jarvice.git
cd jarvice

2. Create a Python virtual environment

A virtual environment keeps Jarvice's packages isolated from your system Python. This prevents version conflicts.

# Create the virtual environment
python -m venv .venv

3. Activate the virtual environment

You need to do this every time you open a new terminal to run Jarvice.

# Windows (PowerShell)
.venv\Scripts\Activate.ps1

# Windows (Command Prompt)
.venv\Scripts\activate.bat

# Windows (Git Bash / MSYS2)
source .venv/Scripts/activate

# Linux / Mac
source .venv/bin/activate

You should see (.venv) at the beginning of your terminal prompt. This means the virtual environment is active.

PowerShell error "cannot be loaded because running scripts is disabled"? Run this first:
Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser

4. Install dependencies

pip install -r requirements.txt

This will download and install all required packages. It may take a few minutes (especially PyTorch, which is ~300MB, and NVIDIA CUDA libraries, which are ~500MB).

CUDA errors? If you get errors about CUDA during installation, make sure your NVIDIA drivers are up to date (see Step 1).

5. Install Jarvice as a package

pip install -e .

This registers Jarvice as a Python package so you can run it with python -m jarvice. The -e flag means "editable" — if you modify the source code, changes take effect immediately without reinstalling.

What does pip install -e . do? It tells Python where to find the jarvice module. Without this, you'll get No module named jarvice when trying to run it.

6. Download voice models

python scripts/download_models.py

This downloads the Piper voice model (~60MB). The Whisper speech-to-text model (~3GB) will be downloaded automatically the first time you run Jarvice.

7. Create your config file

# Windows
copy .env.example .env

# Linux / Mac
cp .env.example .env

8. Edit the config

Open .env in a text editor and set your gateway token:

OPENCLAW_GATEWAY_TOKEN=paste_your_gateway_token_here

That's the token from Step 4. Everything else can stay as default for now.

Configuration

All configuration is done in the .env file. Here's what each option does:

OpenClaw Connection

Variable	Default	Description
`OPENCLAW_GATEWAY_URL`	`http://127.0.0.1:18789`	URL of your OpenClaw gateway. Don't change this unless you moved OpenClaw to another machine.
`OPENCLAW_GATEWAY_TOKEN`	(empty)	Required. Your gateway auth token from `openclaw.json`.
`OPENCLAW_MODEL`	`openclaw/default`	Which model to use. `openclaw/default` uses whatever model OpenClaw is configured with.
`OPENCLAW_SESSION_USER`	`jarvice-voice`	Session identifier. Changing this starts a fresh conversation.

Input Mode

Variable	Default	Description
`INPUT_MODE`	`push_to_talk`	How Jarvice listens. Options: `push_to_talk`, `wake_word`, `continuous`
`PUSH_TO_TALK_KEY`	`space`	Key to hold when talking (push-to-talk mode)
`SILENCE_TIMEOUT_MS`	`1500`	How long to wait after you stop talking before sending (milliseconds). Lower = faster but may cut you off.

Speech-to-Text (Whisper)

Variable	Default	Description
`WHISPER_MODEL`	`large-v3`	Whisper model size. Options below.
`WHISPER_DEVICE`	`cuda`	`cuda` for GPU (fast), `cpu` for CPU (slow but works without NVIDIA GPU)
`WHISPER_COMPUTE_TYPE`	`float16`	Precision. `float16` is fastest on modern GPUs. Use `int8` to save VRAM.
`WHISPER_LANGUAGE`	(empty)	Language code (`en`, `fr`, `ja`, etc.) or empty for auto-detect. Auto-detect handles accents and code-switching well.

Whisper Model Sizes

Model	VRAM	Speed	Quality	Best for
`tiny`	~1 GB	Very fast	Low	Testing, potato GPUs
`base`	~1 GB	Fast	OK	Low VRAM budget
`small`	~2 GB	Fast	Good	GTX 1060/1070
`medium`	~5 GB	Medium	Great	RTX 2060/3060
`large-v3`	~10 GB	Slower	Best	RTX 3070+ (recommended)
`distil-large-v3`	~6 GB	Fast	Near-best	Good balance of speed/quality

Tip: Start with large-v3 if you have 8GB+ VRAM. It handles accents, background noise, and multiple languages much better than smaller models.

Text-to-Speech

Variable	Default	Description
`TTS_ENGINE`	`edge`	TTS engine: `edge` (multilingual, cloud, free) or `piper` (English only, local)
`EDGE_VOICE`	`en-GB-RyanNeural`	Default Edge TTS voice. Auto-switches for other languages.
`EDGE_RATE`	`+0%`	Speech rate adjustment (e.g. `+10%` for faster, `-10%` for slower)
`EDGE_PITCH`	`+0Hz`	Pitch adjustment
`PIPER_MODEL`	`en_GB-alan-medium`	Piper voice model (only used when `TTS_ENGINE=piper`)
`PIPER_SPEAKER_ID`	`0`	Speaker ID for multi-speaker Piper models

Edge TTS Auto-Language Detection

When using Edge TTS, Jarvice automatically detects the language of the AI's response and switches to the appropriate native voice:

Language	Voice	Trigger
English	`en-GB-RyanNeural`	Default
Japanese	`ja-JP-KeitaNeural`	Hiragana, Katakana, Kanji detected
French	`fr-FR-HenriNeural`	French text detected (via langdetect)
Chinese	`zh-CN-YunxiNeural`	Chinese characters detected
Korean	`ko-KR-InJoonNeural`	Hangul detected
Spanish	`es-ES-AlvaroNeural`	Spanish text detected
German	`de-DE-ConradNeural`	German text detected
Russian	`ru-RU-DmitryNeural`	Cyrillic detected
Arabic	`ar-SA-HamedNeural`	Arabic script detected
And more...	Auto	See `LANG_VOICE_MAP` in `src/jarvice/tts/edge.py`

This means you can say "translate to Japanese: hello" and the Japanese part of the response will be spoken by a native Japanese voice, while the English explanation uses your default British voice.

Wake Word

Variable	Default	Description
`WAKE_WORD_ENABLED`	`false`	Set to `true` to use wake word mode.
`WAKE_WORDS`	`hey jarvice`	Comma-separated list of wake words.

Audio Devices

Variable	Default	Description
`AUDIO_DEVICE_INPUT`	`default`	Microphone. Use `default` or a device index number.
`AUDIO_DEVICE_OUTPUT`	`default`	Speakers. Use `default` or a device index number.

Finding your audio device index

python -c "import sounddevice; print(sounddevice.query_devices())"

This prints a numbered list of all audio devices. Look for your mic (input) and speakers (output). The > marker shows the default input, < shows the default output.

Example:

>  1 Microphone (Blue Snowball), MME (1 in, 0 out)     ← default mic (index 1)
<  4 Speakers (Creative Stage Air V2), MME (0 in, 2 out)  ← default speakers (index 4)
   7 Headphones (Oculus Virtual Audio), MME (0 in, 2 out)

To use a specific device, set it in .env:

AUDIO_DEVICE_INPUT=1
AUDIO_DEVICE_OUTPUT=4

Usage

Quick Start

You need two terminal windows: one for OpenClaw, one for Jarvice.

Terminal 1 — Start OpenClaw:

openclaw tui

Leave this running. This is the AI brain.

Terminal 2 — Start Jarvice:

cd path/to/jarvice
.venv\Scripts\activate      # Windows
# or: source .venv/bin/activate  # Linux/Mac

python -m jarvice

You should see:

Jarvice v0.1.0
Like Jarvis, but spicier.

  Gateway:  http://127.0.0.1:18789
  Model:    openclaw/default
  STT:      faster-whisper (large-v3) on cuda
  TTS:      Piper (en_US-amy-medium)
  Mode:     push_to_talk

Ready. Speak now.

Input Modes

Mode	How to use	Best for
Push-to-talk (default)	Hold `Space` to talk, release when done	Desktop use, noisy environments
Wake word	Say "Hey Jarvice" to activate, then speak naturally	Hands-free, quiet rooms
Continuous	Always listening, auto-detects when you speak	Quick commands, dedicated mic setup

Tips for Best Results

Speak clearly but naturally. You don't need to enunciate like a robot.
Wait for the beep/log before speaking (in push-to-talk mode).
Keep your mic close — within arm's reach for best recognition.
Background noise is handled well by Whisper, but a quiet room helps.
Accented English works great — Whisper was trained on many accents (French, Japanese, Indian, etc.)
You can interrupt — if the bot is talking and you start speaking, it will stop and listen to you.

Troubleshooting

"No module named jarvice"

You forgot to install the package. Run:

pip install -e .

Make sure you're in the jarvice/ directory and your virtual environment is activated (you should see (.venv) in your prompt).

"OPENCLAW_GATEWAY_TOKEN not set"

You forgot to set the token in your .env file. Open .env and add your gateway token:

OPENCLAW_GATEWAY_TOKEN=your_token_here

Find the token in ~/.openclaw/openclaw.json → gateway.auth.token.

"Connection refused" / "Cannot connect to gateway"

OpenClaw isn't running. Start it in another terminal:

openclaw tui

"CUDA out of memory"

Your GPU doesn't have enough VRAM for the selected Whisper model. Try a smaller model:

WHISPER_MODEL=medium        # needs ~5GB instead of ~10GB
# or
WHISPER_MODEL=small         # needs ~2GB
# or
WHISPER_MODEL=distil-large-v3  # nearly as good as large-v3, needs ~6GB

"Library cublas64_12.dll is not found" / "RuntimeError: Library ... cannot be loaded"

This means the NVIDIA CUDA libraries are installed but your system can't find them. Jarvice handles this automatically, but if you still see this error:

Make sure you installed the CUDA pip packages:

pip install nvidia-cublas-cu12 nvidia-cudnn-cu12 nvidia-cuda-nvrtc-cu12

If it still fails, the DLLs need to be on your system PATH. Find where they are:
```
python -c "import nvidia.cublas; print(nvidia.cublas.__path__)"
```
Then add the bin subdirectory of that path to your system PATH environment variable.
Nuclear option — install the full CUDA Toolkit from NVIDIA:
- Download from: https://developer.nvidia.com/cuda-downloads
- Install with default settings
- This adds all CUDA DLLs to your system PATH permanently
Give up on GPU — use CPU mode instead (slower but works):
```
WHISPER_DEVICE=cpu
WHISPER_COMPUTE_TYPE=int8
```

"No CUDA-capable device detected"

Make sure you have an NVIDIA GPU
Update your NVIDIA drivers: https://www.nvidia.com/Download/index.aspx
If you don't have an NVIDIA GPU, use CPU mode:
```
WHISPER_DEVICE=cpu
WHISPER_COMPUTE_TYPE=int8
```

Mic not working / "PortAudio error"

Check your mic is plugged in and working (test in Windows Sound Settings)

List available devices:

python -c "import sounddevice; print(sounddevice.query_devices())"

Set the correct device in .env:

AUDIO_DEVICE_INPUT=1   # replace with your mic's index number

No sound output

Check your speakers/headphones are connected and volume is up

Test Piper directly:

python -c "
from src.jarvice.tts.piper import PiperTTS
from pathlib import Path
import sounddevice as sd
tts = PiperTTS(models_dir=Path('models/piper'))
tts.load()
audio = tts.synthesize_sync('Testing, one two three.')
sd.play(audio, samplerate=tts.sample_rate)
sd.wait()
"

"Piper model not found"

Run the model downloader:

python scripts/download_models.py

Or download manually from Piper Voices and place the .onnx + .onnx.json files in models/piper/.

Bot responds but doesn't say anything useful

This is an OpenClaw/model issue, not a Jarvice issue. Test in the TUI:

openclaw tui

If the bot gives weird responses there too, check your model configuration with openclaw models status.

Whisper transcribes incorrectly

Try auto-detect language (set WHISPER_LANGUAGE= empty in .env)
Use a larger model (large-v3 instead of small or medium)
Speak closer to your microphone
Reduce background noise

High latency / slow responses

The latency budget is roughly:

Silence detection: ~1.5s (configurable via SILENCE_TIMEOUT_MS)
Whisper transcription: ~0.3-0.5s (GPU), ~3-5s (CPU)
AI model response: depends on model/provider
TTS synthesis: ~0.1-0.3s

To reduce latency:

Lower SILENCE_TIMEOUT_MS to 700 (may cut off speech early)
Use distil-large-v3 instead of large-v3 (2x faster, similar quality)
Use a faster model provider (Codex/Claude are fast, local Ollama varies)

Changing the Voice

Edge TTS Voices (default, recommended)

Edge TTS is the default TTS engine. It's free, cloud-based, and supports 70+ languages with automatic language detection.

To list all available voices:

edge-tts --list-voices

Some good options for the default English voice:

Voice	Gender	Accent	Style
`en-GB-RyanNeural`	Male	British	Jarvis-like (default)
`en-GB-SoniaNeural`	Female	British	Warm, professional
`en-US-GuyNeural`	Male	American	Clear, neutral
`en-US-JennyNeural`	Female	American	Friendly
`en-US-AriaNeural`	Female	American	Expressive
`en-AU-WilliamNeural`	Male	Australian	Distinctive

Change the default voice in .env:

EDGE_VOICE=en-US-GuyNeural

You can also adjust speed and pitch:

EDGE_RATE=+10%      # faster
EDGE_PITCH=-2Hz     # slightly deeper

Non-English languages are handled automatically — when the AI responds in Japanese, French, etc., Jarvice switches to a native voice for that language. You can customize which voice is used for each language by editing LANG_VOICE_MAP in src/jarvice/tts/edge.py.

Piper TTS Voices (local fallback)

If you prefer fully local TTS (no internet needed), switch to Piper:

TTS_ENGINE=piper

Note: Piper voices only speak one language each. Non-English text will sound wrong.

Browse Piper voices: Piper Voice Samples

Voice	Gender	Accent
`en_GB-alan-medium`	Male	British (Jarvis-like)
`en_US-amy-medium`	Female	American
`en_US-lessac-medium`	Female	American (neutral)

To use a Piper voice:

Download the .onnx and .onnx.json files from Piper Voices
Place them in models/piper/

Update .env:

TTS_ENGINE=piper
PIPER_MODEL=en_GB-alan-medium

Restart Jarvice

Architecture

┌─────────────────────────────────────────────────────┐
│                      Jarvice                         │
│                                                      │
│  ┌──────────┐   ┌────────────┐   ┌───────────────┐ │
│  │ 🎤 Mic    │──▶│ Silero VAD │──▶│ faster-whisper│ │
│  │ capture   │   │ (CPU)      │   │ (GPU/CUDA)    │ │
│  │ 16kHz     │   │ detects    │   │ speech→text   │ │
│  │ mono      │   │ speech     │   │               │ │
│  └──────────┘   └────────────┘   └──────┬────────┘ │
│                                          │ text     │
│  ┌──────────┐   ┌────────────┐   ┌──────▼────────┐ │
│  │ 🔊 Speaker│◀──│ Piper TTS  │◀──│   OpenClaw    │ │
│  │ playback  │   │ (CPU/ONNX) │   │   Gateway     │ │
│  │           │   │ text→speech│   │   HTTP API    │ │
│  └──────────┘   └────────────┘   └───────────────┘ │
│                                                      │
│  ┌────────────────────────────────────────────────┐ │
│  │ 🥽 Meta Quest VR (future)                       │ │
│  │ WiFi audio streaming ◀──▶ PC pipeline           │ │
│  └────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────┘

Pipeline Flow

Mic Capture — records audio at 16kHz mono via sounddevice
Voice Activity Detection (VAD) — Silero VAD detects when you start/stop speaking
Speech-to-Text — faster-whisper on GPU transcribes the audio to text
Gateway — sends text to OpenClaw via HTTP API, receives streaming response
Text-to-Speech — Edge TTS (or Piper) synthesizes each sentence as it arrives, auto-switching voice for detected languages
Playback — plays audio through speakers, supports interruption

Project Structure

jarvice/
├── README.md                    # this file
├── LICENSE                      # MIT license
├── requirements.txt             # Python dependencies
├── .env.example                 # example config (copy to .env)
├── .gitignore
├── pyproject.toml               # Python packaging config
├── config/
│   └── default.yaml             # pipeline tuning parameters
├── scripts/
│   └── download_models.py       # model downloader
├── src/
│   └── jarvice/
│       ├── __init__.py
│       ├── __main__.py          # entry: python -m jarvice
│       ├── app.py               # main orchestrator
│       ├── config.py            # settings from .env (pydantic)
│       ├── audio/
│       │   ├── capture.py       # mic input (sounddevice)
│       │   ├── playback.py      # speaker output
│       │   └── vad.py           # voice activity detection (silero)
│       ├── stt/
│       │   └── whisper.py       # faster-whisper GPU transcription
│       ├── tts/
│       │   └── piper.py         # piper TTS synthesis
│       ├── gateway/
│       │   ├── http_client.py   # OpenClaw HTTP API (v1)
│       │   └── ws_client.py     # WebSocket protocol (future v2)
│       ├── wake/
│       │   └── detector.py      # wake word detection
│       └── pipeline/
│           └── voice_loop.py    # main listen→transcribe→send→speak loop
├── models/                      # downloaded models (git-ignored)
│   ├── whisper/                 # faster-whisper model cache
│   └── piper/                   # piper voice .onnx files
└── tests/

Future: Meta Quest VR Integration

The architecture is designed to support Meta Quest VR as a wireless voice client:

Quest captures mic audio → streams raw PCM over WiFi to your PC
PC runs the full pipeline (STT → OpenClaw → TTS) — Quest doesn't need GPU power
Synthesized audio streams back to the Quest headset
Protocol: WebSocket with binary audio frames
Latency target: <500ms over local WiFi

This means you could walk around in VR and talk to your AI assistant naturally. The PC does all the heavy lifting.

This feature is planned for a future release. The gateway/ws_client.py module is a placeholder for this integration.

Contributing

Contributions are welcome! Here's how:

Fork the repo
Create a feature branch: git checkout -b my-feature
Make your changes
Test locally: python -m jarvice
Submit a PR

Ideas for contributions

More TTS backends (Coqui, Bark, StyleTTS2)
Web UI for configuration
Meta Quest VR client
Discord voice channel integration
Better wake word training
Multi-language support
Conversation history viewer

License

MIT License — see LICENSE.

Credits

OpenClaw — the agent framework that powers the AI brain
faster-whisper — GPU-accelerated speech recognition by SYSTRAN
Piper — lightning-fast local text-to-speech by Rhasspy
Silero VAD — voice activity detection
openWakeWord — open-source wake word detection
sounddevice — PortAudio bindings for Python

Made with love and questionable taste. Jarvice — because Jarvis was taken.

Name		Name	Last commit message	Last commit date
config		config
scripts		scripts
src/jarvice		src/jarvice
tests		tests
.env.example		.env.example
.gitignore		.gitignore
LICENSE		LICENSE
README.md		README.md
pyproject.toml		pyproject.toml
requirements.txt		requirements.txt

Latest commit

History

Folders and files

Repository files navigation

Jarvice

Table of Contents

Features

How It Works (For Beginners)

Prerequisites

Hardware Requirements

Step 0: Install the Basics

Install Python (3.10 or newer)

Install Node.js (18 or newer)

Install Git

Step 1: NVIDIA GPU & CUDA

Check if you have an NVIDIA GPU

Install NVIDIA Drivers (if needed)

Step 2: Install OpenClaw

Run the OpenClaw Setup Wizard

Verify OpenClaw is working

Step 3: Set Up a Model Provider

Option A: OpenAI Codex (Easiest — uses your ChatGPT Plus subscription)

Option B: Anthropic Claude (API key required)

Option C: Claude CLI (Reuse your Claude Code subscription)

Option D: Ollama (100% Local — Free, but needs a good GPU)

Verify your model works

Step 4: Enable the HTTP API

Where is the config file?

Edit the config

Find your gateway token

Restart OpenClaw

Verify the HTTP API works

Step 5: Discord Bot (Optional)

Create the Discord bot

Invite the bot to your Discord server

Get your Discord User ID

Add Discord to OpenClaw config

Restart OpenClaw

Installation

1. Clone the repository

2. Create a Python virtual environment

3. Activate the virtual environment

4. Install dependencies

5. Install Jarvice as a package

6. Download voice models

7. Create your config file

8. Edit the config

Configuration

OpenClaw Connection

Input Mode

Speech-to-Text (Whisper)

Whisper Model Sizes

Text-to-Speech

Edge TTS Auto-Language Detection

Wake Word

Audio Devices

Finding your audio device index

Usage

Quick Start

Input Modes

Tips for Best Results

Troubleshooting

"No module named jarvice"

"OPENCLAW_GATEWAY_TOKEN not set"

"Connection refused" / "Cannot connect to gateway"

"CUDA out of memory"

"Library cublas64_12.dll is not found" / "RuntimeError: Library ... cannot be loaded"

"No CUDA-capable device detected"

Mic not working / "PortAudio error"

No sound output

"Piper model not found"

Bot responds but doesn't say anything useful

Whisper transcribes incorrectly

High latency / slow responses

Changing the Voice

Edge TTS Voices (default, recommended)

Piper TTS Voices (local fallback)

Architecture

Pipeline Flow

Project Structure