Voice assistant: hands-free barge-in (WI-174 v2) - #682
Closed
thedancingdeveloper wants to merge 1 commit into
Closed
thedancingdeveloper wants to merge 1 commit into
thedancingdeveloper wants to merge 1 commit into
Conversation
Lets the speaker talk over a playing reply: with interrupt_response on, the
device stops speaking and listens the moment you start.
- voiceTurn: `speechDetected()` — halts the reply and re-opens the mic, gated
on interrupt_response and an actually-playing reply, so a stray onset outside
playback cannot disturb the loop; the halted reply's own speechFinished is
then ignored. Unit tests for off (default), on, and outside-playback.
- voiceVad: a pure `OnsetDetector` (leaky accumulator over frame RMS — a
sustained ~500 ms onset fires once, a click or the AEC residual does not) +
`frameRms`, unit-tested off frame energies; a thin `startBargeInDetection`
Web Audio shell (echo-cancelled getUserMedia → AnalyserNode, 50 ms frames)
that a WASM Silero VAD can replace behind the same seam later.
- Assistant.tsx: runs the echo-cancelled capture only during `speaking` (a
createEffect on voiceState), torn down the moment playback ends — the second
mic is never held longer than the reply it listens over. Off by default
(imperfect WebView AEC can false-trigger); config vogt.assistant.voice.
{interrupt_response,vad_threshold,vad_onset_ms}.
Tests: full web unit suite green (931); a real-browser Playwright round trip
proves the halt + reopen. Docs: ENGINE.md §6 + USER_GUIDE §2.5.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BojRRh8yXBBqwemk6WkqVo
thedancingdeveloper
force-pushed
the
feat/voice-handsfree
branch
from
September 10, 2026 02:36
9208f5b to
c389532
Compare
thedancingdeveloper
force-pushed
the
feat/voice-handsfree-v2
branch
from
September 10, 2026 02:36
d659c51 to
62b123c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #680 (base
feat/voice-handsfree). Adds opt-in barge-in to hands-free mode: withinterrupt_responseon, the speaker can talk over a playing reply — the device stops speaking and listens the moment they start.What
voiceTurn.ts—speechDetected(): halts the reply and re-opens the mic, gated oninterrupt_responseand a reply actually playing (a stray onset outside playback is a no-op); the halted reply's ownspeechFinishedis then ignored.voiceVad.ts(new) — a pureOnsetDetector(leaky accumulator over frame RMS: a sustained ~500 ms onset fires once; a click or the echo-cancellation residual does not) +frameRms, unit-tested off frame energies.startBargeInDetectionis a thin Web Audio shell (echo-cancelledgetUserMedia→AnalyserNode, 50 ms frames) that a WASM Silero VAD can replace behind the same seam later — not adopted here to avoid the onnxruntime bundle/model weight.Assistant.tsx— runs the echo-cancelled capture only duringspeaking(acreateEffectonvoiceState), torn down the moment playback ends, so the second mic is never held longer than the reply it listens over.Off by default — the WebView's AEC residual is imperfect and a false trigger cuts a reply short; half-duplex is the safe default. Config:
vogt.assistant.voice.{interrupt_response,vad_threshold,vad_onset_ms}.Tests
cancel) and re-opens the mic mid-playback.Notes
mainonce Voice assistant: tap-to-talk + hands-free conversation mode (WI-173, WI-174 v1) #680 lands.docs/local/VOICE_HANDSFREE_DESIGN.md.🤖 Generated with Claude Code
https://claude.ai/code/session_01BojRRh8yXBBqwemk6WkqVo