Skip to content

feat(voice): set the runtime up on a click, and show that it is listening - #107

Merged
adityak74 merged 2 commits into
mainfrom
feat/voice-auto-setup
Aug 24, 2026
Merged

feat(voice): set the runtime up on a click, and show that it is listening#107
adityak74 merged 2 commits into
mainfrom
feat/voice-auto-setup

Conversation

@adityak74

Copy link
Copy Markdown
Contributor

Voice input told you to run python -m pip install "qwen-asr[vllm]==0.0.6" and
then qwen-asr-serve. On macOS arm64 that command cannot succeed: vLLM's
dependency tree has no matching wheel, so the microphone showed a command that
was guaranteed to fail. This makes setup automatic and picks the backend from
what pip can actually resolve.

What changed

A microphone click now starts local setup through the readiness request that
already existed, at the same time as the browser asks for permission. Neither
waits on the other, because a permission prompt hidden behind a model download
is a prompt nobody sees.

Setup creates a marked virtual environment under the platform's local data
directory, pins qwen-asr[vllm]==0.0.6, and runs every command as an argument
vector with no shell. If pip cannot resolve the vLLM extra, it recreates its own
environment with the plain qwen-asr==0.0.6 and starts an embedded
Transformers-backed server instead. The backend follows the resolver rather than
an OS list, so a machine that can install vLLM gets it with no code change.

The page reports the real create, install, download, load, and ready stages. It
invents no percentage. No command reaches the browser.

ZORP_VOICE_AUTOSTART=0 disables every install and spawn path. In that
compatibility mode the status API still carries the old operator start command,
and the browser still never renders shell text.

The page says it is listening

While the microphone is open the composer draws a live level meter from the
stream, because a click that starts a minutes-long setup with no visible sign of
the microphone reads as a broken button.

The meter reads amplitude and nothing else. It keeps no sample, copies the audio
nowhere, and holds nothing after it stops. A browser with no Web Audio, or one
where the audio context throws, gets an inert meter rather than a failed
recording, and it is built from DOM nodes rather than assembled markup like
everything else on that page. It is decoration and is hidden from assistive
technology; the status line is what a screen reader is told.

What did not change

The loopback boundary. The endpoint still passes the written-form and resolution
checks, the client still gets a resolver for one host and port, redirects are
still off, proxy discovery is still off, and the tests still count connections to
loopback canaries rather than checking for an error. A spawned server binds only
to a validated loopback target. There is still no cloud ASR provider and no
fallback.

Setup refuses to run as root, and it refuses to reuse an environment it did not
mark as its own. HTTPS and path-prefixed endpoints still need an
operator-managed proxy; automatic setup cannot bind those and does not pretend
to.

No model tool can start a runtime, start setup, or start a recording. A
transcript is still untrusted, editable composer text that grants no tool,
changes no approval, bypasses no denylist, and is never sent for you.

Verification

Codex ran out of usage credits partway through this work, so the last of it and
all of the verification below was done by hand. Every command was run and every
number is quoted from that run.

  • cargo fmt --all --check: clean.
  • cargo test -p zorp-voice: 26 passed across six binaries, 0 failed.
  • cargo test -p zorp-web --features voice: 20 binaries, all ok, no failures.
  • cargo test --workspace: 77 binaries, all ok, no failures.
  • From web/: npm run check clean, npm test 273 passed and 0 failed,
    npm run build fine.

The voice feature is not covered by CI, which is why it is in that list.

One gap worth naming rather than hiding: these runs happened on a developer
machine with no sandbox restrictions, so socket-bound tests really did bind. I
have not confirmed how the suite behaves under a sandbox that denies
TcpListener::bind. Automatic setup itself was exercised through its tests and
not by a real end-to-end install on a clean machine.

https://claude.ai/code/session_01KGPVQ8wUG7h36zashWYCp4

…ning

Voice input printed `pip install "qwen-asr[vllm]==0.0.6"` and asked the
person to run it. On macOS arm64 that install cannot succeed, because
vLLM's dependency tree has no matching wheel, so the microphone showed a
command guaranteed to fail.

A microphone click now starts setup through the readiness request that
already existed, at the same time as the browser asks for permission.
Neither waits on the other, because a permission prompt hidden behind a
model download is a prompt nobody sees. Setup creates a marked virtual
environment under the local data directory, pins the runtime, and runs
every command as an argument vector with no shell. When pip cannot
resolve the vLLM extra it recreates its own environment with the plain
package and starts an embedded Transformers server instead, so the
backend follows the resolver and not an OS list. The page reports real
stages and invents no percentage, and no command reaches the browser.
`ZORP_VOICE_AUTOSTART=0` turns every install and spawn path off.

While the microphone is open the composer now draws a live level meter
from the stream, because a click that starts a minutes-long setup with
no visible sign of the microphone reads as a broken button. The meter
reads amplitude only. It keeps no sample, copies the audio nowhere, and
holds nothing after it stops. A browser with no Web Audio, or one where
the audio context throws, gets an inert meter rather than a failed
recording. It is decoration and hidden from assistive technology; the
status line is what a screen reader is told.

The loopback boundary is untouched: written-form and resolution checks,
a resolver pinned to one host and port, no redirects, no proxy
discovery, and tests that count connections to canaries. A spawned
server binds only to a validated loopback target. Setup refuses to run
as root and refuses to reuse an environment it did not mark. There is
still no cloud provider and no fallback, no model tool can start a
runtime or a recording, and a transcript is still untrusted composer
text that is never sent for you.

Claude-Session: https://claude.ai/code/session_01KGPVQ8wUG7h36zashWYCp4
The dtype test runs the embedded server's audio helpers under the host
python3, and that probe imports numpy because what normalize promises is
a numpy dtype maximum. Setup installs numpy into its own environment, so
the host does not need it, and CI does not have it. Both runners failed
on a missing module rather than on anything about the code.

It now checks for numpy first and says on stderr when it is skipping,
the same shape zorp-mcp uses for a missing npx. The assertions run on
machines that have numpy, which does not include CI, so the comment says
so and names the syntax test that covers this file everywhere.

Claude-Session: https://claude.ai/code/session_01KGPVQ8wUG7h36zashWYCp4
@adityak74
adityak74 force-pushed the feat/voice-auto-setup branch from b72da06 to 1e562d0 Compare August 24, 2026 21:44
@adityak74
adityak74 merged commit 74bb667 into main Aug 24, 2026
7 checks passed
@adityak74
adityak74 deleted the feat/voice-auto-setup branch August 24, 2026 21:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant