privateer-speak

Voice for Pi coding agents, both directions. /speak on, and answers are read aloud as they are written — sentence by sentence, so the reply starts while Pi is still typing it, with code blocks, tables and URLs stripped. Press alt+t and talk; the mic stops when you stop talking, and the transcript lands in the composer for you to review before Enter sends it.

• Speech works out of the box with your OS's local voice (say on macOS, espeak-ng on Linux, System.Speech on Windows). Nothing leaves the machine.
• Any OpenAI-compatible endpoint: TTS via POST {baseUrl}/audio/speech, STT via POST {baseUrl}/audio/transcriptions — OpenAI, Tinfoil, a local Kokoro/whisper.cpp/LocalAI server, or a proxy, from one config block.
• Hands-free when you want it: /talk loop on reopens the mic after each answer, so you can keep your hands off the keyboard and your eyes off the screen.
• Pluggable: hosts register their own providers (e.g. privateer-agent registers its account's confidential-compute TTS and STT) through tiny structural interfaces.

Install

Add privateer-speak to your Pi packages, or load it from an extension factory:

import { makePiSpeakExtension } from "privateer-speak";
factories.push(makePiSpeakExtension());

Use

/speak on            start speaking answers
/speak off           stop
/speak stop          cut off the current utterance
/speak test          say a sample line now
/speak provider      list providers (→ marks the active one)
/speak provider <id> pick one deliberately ("auto" un-picks)
/speak voice         show / list voices for the active provider
/speak voice <name>  pick a voice (stored per provider)
/speak rate <n>      speaking pace, 0.5–3× normal (stored per provider)
/speak length <n>    characters spoken per answer, or "full" for all of it
/speak stream on|off speak sentences as they arrive vs once the turn ends
/speak announce on|off  say what tools are running ("Running npm.")

alt+t                push to talk — press to listen, press again to send
/talk                the same thing, typed
/talk loop on|off    conversation mode: reopen the mic after each answer
/talk silence <ms>   silence that ends an utterance (default 1200, "off" disables)
/talk vocab <words>  vocabulary the transcriber should expect
/talk provider [id]  list / pick the transcription provider
/talk send on|off    submit transcripts immediately vs prefill the composer
/talk lang [code]    language hint for the engine ("auto" clears)
/talk key <binding>  rebind push-to-talk ("none" unbinds; /reload applies it)

While recording, the footer shows a live level meter; enter sends early, escape discards. In conversation mode, touching the keyboard while Pi is talking stops it — you have the floor back.

Speech is off by default and only ever runs in interactive UI sessions — headless daemons never speak. The microphone opens only on something you did: /talk, the push-to-talk key, or a conversation turn you switched on. There is no wake word. Capture is hard-capped at input.maxSeconds (default 60), audio is held in memory and never written to disk, and discarding drops it unheard. Transcripts prefill the composer so you read what was heard before it sends — send on and loop on are the deliberate hands-free opt-ins, and a mis-heard transcript is still just a prompt: tool calls stay behind the same permission gate as anything you type.

Recording needs a capture tool on PATH: sox anywhere, or arecord/parecord/ffmpeg on Linux, ffmpeg on macOS.

Conversation mode

/speak on
/talk loop on

Now it goes back and forth on its own: you speak, the mic closes when you stop, the transcript sends, the answer is spoken as it is written, and the mic reopens. Tool cues ("Running npm.") come on with it, because with your eyes off the screen a silent two-minute tool run is indistinguishable from a crash.

Three things end a turn without ceremony: say nothing and the mic closes by itself, press escape to discard what you just said, or type — which stops the voice mid-sentence and leaves you at a normal composer. /talk loop off ends the mode.

Config

~/.pi/speak.json (hosts can relocate it):

{
  "enabled": true,
  "provider": "auto",
  "maxChars": 400,
  "stream": true,
  "announce": false,
  "providers": {
    "local": { "voice": "Samantha", "rate": 1.3 },
    "openai-compatible": {
      "baseUrl": "https://api.openai.com/v1",
      "apiKeyEnv": "OPENAI_API_KEY",
      "model": "tts-1",
      "sttModel": "whisper-1",
      "voice": "alloy",
      "rate": 1.2
    }
  },
  "input": {
    "provider": "auto",
    "language": "en",
    "autoSend": false,
    "conversation": false,
    "maxSeconds": 60,
    "silenceMs": 1200,
    "threshold": 0.02,
    "shortcut": "alt+t",
    "hints": "privateer-speak, Pi, espeak-ng, whisper.cpp, Tinfoil"
  }
}

apiKeyEnv names an environment variable so the key stays out of the file; a literal apiKey also works. Voice and rate are stored per provider because neither ports across engines. maxChars: 0 speaks answers in full. silenceMs is how long you have to stop talking before the utterance is considered finished — lower it if you speak in short bursts, raise it if you think mid-sentence, and raise threshold in a loud room. hints is passed to the transcriber as a vocabulary bias, which is what keeps project nouns and command names from coming back as prose.

Provider resolution

1. Your explicit /speak provider pick, if it's available.
2. The most recently registered provider whose preferWhen() is true (how a host makes "signed in → use the account's TTS" automatic without stomping your pick).
3. The first available provider — the local engine, so an unconfigured install talks.

Registering a provider

import { registerSpeechProvider } from "privateer-speak";

registerSpeechProvider(
  {
    id: "my-tts",
    label: "My TTS",
    privacy: "standard", // "local" | "confidential" | "standard" — shown in the picker
    defaultVoice: "ada",
    voices: () => ["ada", "bo"],
    available: () => haveCredentials(),
    fetchSpeech: async (text, { voice, rate, signal }) => ({
      data: await synthesize(text, voice, rate, signal), // Uint8Array
      format: "mp3",
    }),
  },
  { preferWhen: () => haveCredentials() },
);

Providers implement either fetchSpeech (return bytes; pi-speak plays them) or speak (own the utterance; return a stoppable handle — how the local engines work). rate is a multiple of the engine's normal pace; map it to whatever dial the engine actually has.

A streaming answer arrives as several short utterances rather than one long one, so fetchSpeech is called repeatedly and its signal matters: pi-speak synthesizes up to two clips ahead of the voice and aborts the rest the moment the user interrupts.

Transcription is the mirror image, in its own registry with the same resolution rules:

import { registerTranscriptionProvider } from "privateer-speak";

registerTranscriptionProvider(
  {
    id: "my-stt",
    label: "My STT",
    privacy: "standard",
    available: () => haveCredentials(),
    transcribe: async (audio, { language, prompt, signal }) =>
      await recognize(audio.data, audio.format, language, prompt, signal), // → string
  },
  { preferWhen: () => haveCredentials() },
);

prompt is the vocabulary hint (project name plus input.hints). Whisper-style engines take it as preceding context and bias decoding toward it; engines without the concept can ignore it. audio is 16 kHz mono WAV.

License

MIT