Rokid Claude

Voice-control Claude Code on your home Mac from Rokid Glasses — watch it work, live, on a monochrome-green HUD.

hero.mp4

Speak a task; watch Claude Code do it on the green HUD — permission confirmed by a gesture. · 中文 README

What it is

Rokid Claude turns a pair of Rokid Glasses (or any Android phone) into a remote control for Claude Code running on your Mac at home. You speak a request; the glasses send it to a small relay on your Mac, which runs Claude Code and streams the agent's progress — thoughts, tool calls, results — back to the green heads-up display in real time. Use it over USB on your desk, or over an ngrok tunnel from anywhere.

Demo

Voice model switch — say "switch model", then swipe to pick a model:

switchmodel.mp4

Runtime language switch — say "switch language"; the whole HUD flips between Chinese and English:

switchlanguage.mp4

Architecture

┌─────────────┐   audio(PCM)/prompt    ┌──────────────────────┐   spawn   ┌──────────────┐
│  Android    │ ─────────────────────► │  Relay (Node/TS)     │ ────────► │  claude -p   │
│  client     │   WebSocket (ws/wss)   │  :8787               │ stream-   │  (Claude     │
│  (glasses/  │ ◄───────────────────── │  whisper.cpp STT     │  json     │   Code CLI)  │
│   phone)    │   events / usage /     │  RunStore + replay   │ ◄──────── │              │
│  green HUD  │   permission prompts   │  PreToolUse hook     │           └──────────────┘
└─────────────┘                        └──────────────────────┘

Speech is transcribed locally with whisper.cpp; the relay speaks a small JSON protocol over WebSocket so the core is client-agnostic. See ARCHITECTURE.md for details.

Features

• 🎙️ Voice → Claude Code, streamed to a green HUD (tap to talk, tap to interrupt)
• ✅ On-glasses permission confirmation for risky tool calls (gesture verdict)
• 📊 Statusline: current model + session cost/tokens
• 🗣️ Voice model switch (opus / sonnet / haiku) via a picker
• 🌍 Bilingual (Chinese / English): set via lang, or switch at runtime by voice ("切换语言 / switch language")
• 📷 On-device QR provisioning — scan a Wi-Fi QR to join a network, or a config QR to set serverUrl/token (no cable, no typing)
• 🛟 Offline self-rescue — when disconnected, single-tap opens the QR scanner
• 🖥️ Mirror the live session on your Mac (local web client)
• 🌐 Remote control from anywhere via an ngrok tunnel (token-authed)
• 🔋 Screen-off power saving; editable command dictionary

Requirements

A hobby project with a deliberately niche setup. Before you start, you'll need:

• A Mac (macOS). The relay, Claude Code, and the helper *.command scripts all run on macOS. (The relay is plain Node/TS and could be adapted to Linux, but the scripts and setup assume a Mac.)
• Claude Code installed and logged in on that Mac — a paid Claude subscription or API access.
• Rokid Glasses (or any Android phone) as the client.
• A USB data cable for the one-time setup. Installing the APK, enabling USB debugging, and provisioning the first network each need a wired connection once — a charge-only cable won't work. After that, wireless adb and QR provisioning keep it cable-free.
• Node.js ≥ 18, whisper.cpp + the ggml-small model, and ngrok (for remote/out-of-home use).

Quick Start

git clone https://github.com/williamlzz/Rokid_Claude && cd Rokid_Claude
cd relay && npm install

Then install the client and configure it:

1. Sideload the APK from the latest Release (or build it: cd android && ./gradlew :app:assembleDebug).
2. cp config.example.json config.json, set serverUrl, and push it to the device.
3. Run ./start.command (USB) and open Rokid Claude on the device.

Full local and remote instructions: docs/SETUP.md.

Development

Run the tests:

cd relay && npm test                              # relay (vitest)
cd android && ./gradlew :app:testDebugUnitTest    # Android (JUnit)

Building the Android app on macOS needs JDK 17 (the system default may be older — Gradle then fails with a cryptic "Could not resolve com.android.tools.build:gradle") plus the Android SDK:

export JAVA_HOME=/opt/homebrew/opt/openjdk@17/libexec/openjdk.jdk/Contents/Home
export ANDROID_HOME="$HOME/Library/Android/sdk"

Security

The relay runs Claude Code on your Mac, so its access token equals remote code execution on your machine. Keep relay/.remote.env and your real config.json out of git (both are gitignored), never expose the relay publicly without a token, and rely on the on-glasses permission confirmation as a second line of defense.

License

MIT — see LICENSE.

Acknowledgements

Built on Claude Code, whisper.cpp, and ngrok. Architecture inspired by the lark-coding-agent-bridge and clawsses projects (studied, not copied).