Skip to content

vetcoders/codescribe

BranchesTags

Repository files navigation

⌜ CodeScribe ⌟

Version License: FSL-1.1-ALv2 CI Landing

Native macOS tray dictation and assistive voice overlay with local Whisper live preview, optional cloud final transcript paths, and quality tooling.

Overview

CodeScribe is a native macOS menu-bar application that captures audio through global hotkeys, shows live local transcription while you speak, and pastes or routes the final result into the focused application. The shipped product in this repo is a tray app with three explicit surfaces: onboarding, settings, and overlays.

Local Whisper is the low-latency path. Cloud STT is optional and currently used as a post-capture transcript backend, not as live cloud preview. AI formatting and assistive mode use OpenAI Responses API (/v1/responses) by default, configured in Onboarding, Settings, or ~/.codescribe/.env.

flowchart TB
    classDef default fill:#fff,stroke:#333,stroke-width:1px;
    classDef box fill:#fafafa,stroke:#666,stroke-width:1px,stroke-dasharray: 0;

    subgraph APP[CodeScribe Runtime]
        direction LR
        TRAY[Tray + Hotkeys]
        ONB[Onboarding]
        SET[Settings Window]
        OVL[Dictation Overlay]
        CHAT[Assistive Voice Overlay]
    end

    subgraph CORE[Core Pipeline]
        direction TB
        REC[Streaming Recorder]
        POST[Stream Postprocess]
        STT[Whisper / Cloud Final Pass]
        LLM[Responses API Formatting / Assistive]
        QL[Quality Loop]
    end

    subgraph TOOLS[CLI Suite]
        QCLI[qube-report]
        LCLI[qube-daemon]
    end

    TRAY --> ONB
    TRAY --> SET
    TRAY --> OVL
    TRAY --> CHAT
    TRAY --> CORE
    SET --> CORE
    OVL --> CORE
    CHAT --> CORE
    REC --> POST --> STT
    STT --> LLM
    QL -.-> STT
    CORE -.-> TOOLS

    class APP,CORE,TOOLS box
Loading

Current runtime truth: live overlay preview is local Whisper. Cloud STT is configurable in Settings, but in the current build it is still a post-capture path rather than live cloud preview.

Status: current source version is 0.12.2 (see Cargo.toml) and ships as a native macOS tray/settings/overlay app with local live preview, tiered settings (settings.json + Keychain + optional .env), and quality-loop tooling.

See: docs/WHISPER_LIVE.md | docs/BACKLOG.md | docs/ARCHITECTURE.md

OpenAI Provider

CodeScribe uses OpenAI Responses API (/v1/responses) by default for AI formatting and assistive mode.

Default Setup

Put your OpenAI API key in Onboarding or Settings. CodeScribe stores it in macOS Keychain and applies it to both AI modes:

# ~/.codescribe/.env

# Shared defaults
LLM_ENDPOINT=https://api.openai.com/v1/responses
LLM_MODEL=gpt-4.1
LLM_API_KEY=sk-proj-xxx

# Formatting mode / cleanup pass
LLM_FORMATTING_ENDPOINT=https://api.openai.com/v1/responses
LLM_FORMATTING_MODEL=gpt-4.1
LLM_FORMATTING_API_KEY=sk-proj-xxx

# Assistive mode / agent chat
LLM_ASSISTIVE_ENDPOINT=https://api.openai.com/v1/responses
LLM_ASSISTIVE_MODEL=gpt-5.5
LLM_ASSISTIVE_API_KEY=sk-proj-xxx

Note: All requests use previous_response_id for conversation chaining. Context persists across transcriptions.

MCP Extension Path

CodeScribe can load custom MCP servers from ~/.codescribe/mcp.json. That keeps the free product useful with user-owned tools today, while leaving room for first-party Pro integrations such as AICX and Loctree later.

Features

  • Pure Rust Implementation — Native macOS app built entirely in Rust with candle-core + Metal GPU
  • Two DMG variants — The standard DMG embeds Silero VAD + MiniLM support assets and resolves Whisper from cache/download. The _full DMG embeds Silero + MiniLM + Whisper for users who prefer one larger download.
  • Whisper Live — Streaming transcription happens during recording (chunks + overlap), so stop() is near-instant
  • Stream postprocess — semantic gating + cleanup of live chunks before final output
  • IPC Server — Stable runtime interface for GUI/clients
  • Quality Loop + Report — Automated quality scoring and batch reports
  • CLI Suite — codescribe, qube-report, qube-daemon (renamed from codescribe-quality / codescribe-loop in 0.8.1)
  • Metal GPU Acceleration — Hardware-accelerated inference on Apple Silicon
  • System Tray App — Minimal menu-bar presence with animated status glyphs
  • Global Hotkeys — Hold Fn (default) or double‑tap Option to record
  • OpenAI Responses by default — Formatting uses gpt-4.1; Assistive uses gpt-5.5
  • Custom MCP Servers — Add your own MCP tools through ~/.codescribe/mcp.json
  • AI Formatting — Optional post-processing via Responses API
  • Slug Filenames — Transcripts named with first 3 words for easy identification

Tech Stack

Component Technology Purpose
Language Rust 2024 Edition Native performance
ML Framework candle-core + candle-transformers Whisper inference
GPU Acceleration Metal (Apple Silicon) Hardware-accelerated STT
System Tray tray-icon + muda + tao Menu bar app
Hotkeys CGEventTap (core-graphics) Global key detection
Audio cpal + hound + symphonia Recording & format support
HTTP Client reqwest LLM API calls
API Format openai-harmony Responses API support
Security cap-std Path safety hardening
Embeddings candle-transformers (MiniLM) Local semantic gating

Installation

Prerequisites

  • macOS 14+ (Sonoma or later)
  • Apple Silicon (M1, M2, M3, or later)
  • Rust Toolchain (1.85+ with edition 2024 support)

Install from Source

# Clone the repository
git clone https://github.com/VetCoders/CodeScribe.git
cd CodeScribe

# Install the hook runner once (required for local commit/push gates)
pipx install pre-commit

# Install CLI + repo-local git hooks
make install

# Verify installation
codescribe --version

Install via Release DMG

Tagged builds publish DMGs through GitHub Releases:

  1. Open Releases
  2. Download CodeScribe_<version>.dmg for the standard build, or CodeScribe_<version>_full.dmg for the larger build with embedded Whisper.
  3. Drag CodeScribe.app into Applications

Current truth: source install is the guaranteed path inside this repo until a current v0.12.x GitHub Release exists. Public release DMGs must be Developer ID signed and notarized; the release workflow is wired to fail if the required Apple signing/notary secrets are missing.

Build Options

make build              # Debug build
make release            # Release build with embedded Silero + embedder; Whisper from cache/download
make install            # Install CLI + repo-local git hooks
make install-app        # Build + install macOS .app (auto-downloads models if missing)
make install-no-embed   # Install without optional embedding + repo-local git hooks

Quick Start

# Start tray app
codescribe

# Open/create config file
make config
# or: codescribe --config

# Verbose logging
codescribe -v

# CLI transcription
codescribe transcribe audio.wav
codescribe transcribe -l pl audio.m4a
codescribe transcribe -f audio.mp3  # with AI formatting

Default Hotkeys (macOS)

  • Dictation: hold your configured modifier (default: Hold Fn/Globe) → release to send + paste
  • Formatting: Double‑tap Left Option → hands‑free recording + AI formatting (auto‑paste ON)
  • Assistive (Agent): Double‑tap Right Option → voice‑chat overlay + agent response (auto‑paste OFF)

Hotkeys are configured in Settings → Modes & Shortcuts. Double‑tap modes auto‑send an utterance when you pause, and stop on the next double‑tap.

Settings & Secrets

  • GUI settings: ~/Library/Application Support/CodeScribe/settings.json
  • API keys: macOS Keychain (com.vetcoders.codescribe)
  • Power‑user overrides: ~/.codescribe/.env

How It Works

flowchart TD
    A[Hotkey Press] --> B{Mode?}
    B -->|Hold Fn| C[Start Recording]
    B -->|Double Option| C
    C --> D[Recording]
  D -->|live chunks| E[Whisper STT (streaming)]
    D -->|Release / Toggle| F[Stop]
    F --> G[Finalize last chunk]
    G --> H{AI Enabled?}
    H -->|Yes| I[LLM Formatting]
    H -->|No| J[Raw Transcript]
    I --> K[Paste to Active App]
    J --> K

    E -.- E1[Metal GPU • runtime model]
    I -.- I1[Responses API • previous_response_id]
Loading

Transcription Pipeline

Live transcription is now modeled as:

  • committed utterances already safe to keep
  • one active preview tail for the current utterance
  • corrections that rewrite only that active tail

That means streaming partials are appended session-wide, but partial-pass fixes only backspace inside the current tail instead of overwriting earlier committed text. Final utterances keep their timestamp/segment metadata through the event pipeline, while overlays/chat bubbles still receive only backspace-encoded TranscriptDelta payloads.

Recording Modes

Mode Trigger Description
Dictation Hold Fn/Globe (default) Fast transcript (AI optional), auto‑paste
Formatting Double‑tap Left Option AI formatting pass, auto‑paste
Assistive (Agent) Double‑tap Right Option Agent chat with optional selection context

See docs/BACKLOG.md for detailed mode descriptions and future enhancements (VAD, Overlay).

Configuration

GUI settings live in settings.json, secrets in Keychain, and power‑user overrides in ~/.codescribe/.env.

# Open config helper (creates ~/.codescribe/.env if missing)
make config

Environment Variables

# STT (Speech-to-Text)
WHISPER_LANGUAGE=pl                  # pl | en | de | fr (no auto!)
# CODESCRIBE_MODEL_PATH=             # Override runtime Whisper model lookup

# Hotkeys behavior
# Per-mode bindings live in Settings -> Modes & Shortcuts (settings.json)
HOLD_EXCLUSIVE=1                     # ignore extra modifiers during hold
HOLD_START_DELAY_MS=800              # Delay before recording starts
DOUBLE_TAP_INTERVAL_MS=200           # Toggle gesture timing
TOGGLE_SILENCE_SEC=5.0               # Auto-send after silence in toggle modes

# AI Formatting
AI_FORMATTING_ENABLED=1              # 1=format via LLM, 0=raw transcript

# OpenAI Responses provider (shared defaults)
LLM_ENDPOINT=https://api.openai.com/v1/responses
LLM_MODEL=gpt-4.1
LLM_API_KEY=sk-proj-xxx

# Mode-specific overrides (optional)
# LLM_FORMATTING_{ENDPOINT,MODEL,API_KEY}=
# LLM_ASSISTIVE_{ENDPOINT,MODEL,API_KEY}=

# History
HISTORY_ENABLED=1                    # Save transcripts
DUMP_AUDIO_LOGS=0                    # 1=save .wav paired with .txt

# Audio
BEEP_ON_START=1
SOUND_VOLUME=0.5
# AUDIO_INPUT_DEVICE=                # Specific device name

# Logging
LOG_LEVEL=INFO                       # TRACE | DEBUG | INFO | WARN | ERROR

See .env.example for complete reference.

CLI Reference

codescribe (Tray App)

Main application — runs as menu bar app with global hotkeys.

codescribe [OPTIONS]

Options:
  -v, --verbose      Enable verbose logging
  --config           Create/edit config file
  --version          Show version
  -h, --help         Show help

codescribe transcribe

CLI transcription without tray app.

codescribe transcribe FILE [OPTIONS]

Arguments:
  FILE               Audio file (WAV, MP3, M4A)

Options:
  -l, --language     Language hint (pl, en, de, fr)
  -f, --format       Apply AI formatting
  -m, --model        Model name (if using external)
  --llm              LLM model for formatting
  -h, --help         Show help

Model

CodeScribe uses whisper-large-v3-turbo-mlx-q8:

  • 4-layer turbo architecture (vs 32 layers in full model)
  • Q8 quantization (~894MB weights)
  • ~10x faster than whisper-large-v3
  • Metal GPU acceleration

Runtime Whisper (Current)

The standard build embeds Silero VAD and MiniLM semantic support assets, then resolves Whisper at runtime from the locations below. The full release DMG embeds the same support assets plus Whisper by building with CODESCRIBE_EMBED_WHISPER=1.

  1. CODESCRIBE_MODEL_PATH environment variable
  2. ~/.codescribe/models/whisper-large-v3-turbo-mlx-q8/
  3. ./models/whisper-large-v3-turbo-mlx-q8/
  4. Hugging Face cache snapshots for LibraxisAI/whisper-large-v3-turbo-mlx-q8

CODESCRIBE_NO_EMBED=1 remains a development/recovery path and disables optional embedded support assets too; it is not the public standard DMG mode.

Model files required:

  • config.json
  • weights.safetensors
  • tokenizer.json
  • mel_filters.npz

Architecture

CodeScribe/
├── core/                      # Portable pipeline, STT, config, quality
├── app/                       # macOS app shell
│   ├── controller/            # Recording/transcription orchestration
│   ├── os/                    # Hotkeys, permissions, clipboard
│   └── ui/
│       ├── settings/          # Persistent settings window
│       ├── onboarding/        # First-run flow
│       ├── overlay/           # Dictation overlay
│       ├── voice_chat/        # Assistive overlay
│       └── tray/              # Menu bar UI
├── bin/                       # CLI entry points
├── tests/                     # Integration + E2E tests
└── docs/                      # Product + technical docs

Development

# Clone and setup
git clone https://github.com/VetCoders/CodeScribe.git
cd CodeScribe

# Development build with explicit runtime Whisper fallback
CODESCRIBE_MODEL_PATH=./models/whisper-large-v3-turbo-mlx-q8 cargo run

# Quality checks
make lint           # clippy + fmt check
make test           # Unit + integration tests
make check          # Full quality gate

# Formatting
make format         # cargo fmt

Makefile Targets

make build            # Debug build
make release          # Release build with embedded support assets; Whisper from cache/download
make install          # Install CLI
make install-no-embed # Install without optional embedding
make release-dmgs     # Build both signed + notarized release DMGs
make config           # Edit ~/.codescribe/.env
make start            # Start as daemon
make stop             # Stop running instance
make logs             # View logs
make lint             # Clippy + format check
make test             # Run tests
make check            # Full quality gate
make download-model   # Download Whisper model

Code Quality

Tool Purpose Config
Clippy Linting -D warnings
rustfmt Formatting Rust 2024 edition
cargo test Testing Unit + E2E

Permissions

CodeScribe requires macOS permissions for:

  • Microphone — Audio recording
  • Accessibility — Global hotkey detection
  • Input Monitoring — Keyboard event capture

Grant permissions in System Settings > Privacy & Security when prompted.

Current Focus

  • Keep the VAD auto-stop path honest and fully integrated before presenting it as the default hands-off mode.
  • Preserve the explicit split between onboarding, settings, dictation overlay, and assistive overlay.
  • Ship the macOS distribution path cleanly: bundle, sign, and notarize the DMG story.

See docs/BACKLOG.md for the working backlog, docs/PUBLIC_RELEASE_CHECKLIST.md for the public launch gate, and docs/ARCHITECTURE_VISION.md for longer-range ideas that are not part of the current shipped surface.

License

CodeScribe is licensed under the Functional Source License 1.1, ALv2 Future License (FSL-1.1-ALv2).

This is a Fair Source / source-available license while the current FSL terms apply. You may read, fork, build, and modify the source for permitted purposes including personal use, education, research, and professional services. Competing Use is not permitted: do not make CodeScribe available as a commercial product or service that substitutes for CodeScribe.

Each released version automatically converts to Apache-2.0 two years after the date we make that version available. See LICENSE and https://fsl.software for the full terms.

Made with (งಠ_ಠ)ง by the ⌜ VetCoders ⌟ 𝖙𝖊𝖆𝖒 (c) 2024-2026 Maciej & Monika + Klaudiusz (AI) + Junie (AI)

About

CodeScribe is a dictation-driven orchestration agent for non-IDE software work. Operators guide development through spoken intent, prompts, agents, repo maps, transcripts, reports, commits, and verification gates — without manually navigating or editing code in an IDE.

vetcoders.github.io/codescribe/

Topics

macos rust speech-to-text dictation whisper voice-agent

Resources

Readme

License

View license

Code of conduct

Code of conduct

Contributing

Contributing

Security policy

Security policy
Activity
Custom properties

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases 1

v0.8.0 Latest
Mar 9, 2026

Packages

 
 
 

Contributors

Languages