Skip to content

Latest commit

 

History

History
135 lines (115 loc) · 7.77 KB

File metadata and controls

135 lines (115 loc) · 7.77 KB

Agent first-contact record

Date: 2026-07-14

This record combines the reproducible CLI-only rehearsal with the physical device results. Individual unverified paths remain explicitly marked.

Clean-environment rehearsal

The release candidate was built, then exercised with an empty temporary configuration directory. Starting from limb --help, every command and nested subcommand help path parsed successfully. The rehearsal found one real CLI problem: commands with an optional device followed by a required positional argument conflicted with --last. Those positionals are now resolved by the command layer, so both forms are accepted:

limb blit DEVICE pixels.rgb565 --rect 0,0,16,16
limb --last blit pixels.rgb565 --rect 0,0,16,16

The first mDNS implementation also assumed one default network interface. On a host with both LAN and tunnel interfaces this could miss a valid limb announcement. Discovery now joins and queries _limb._tcp.local on every active multicast-capable IPv4 interface, merges replies, and still permits an explicit --interface override. Physical testing found that point-to-point VPN interfaces must be excluded: concurrent mDNS sockets on those tunnels could consume replies intended for the LAN socket. The default scan now finds the device without an interface hint.

Protocol rehearsal

The same public liblimb client used by the CLI passed a loopback consistency test covering control JSON Lines, unsolicited gate events, authentication, single-use stream tickets, binary frame binding, raw audio output, leases, and release. This verifies the host implementation without weakening the physical gate model.

Physical-device results

The default scan found a v0.2 device on the LAN. An existing token was imported through the private terminal prompt, authenticated, and persisted without putting the secret in an argument or repository file. Authenticated describe, auth list, minimum-RTT clock estimation, gate status, kv list, and reflex status all succeeded.

The output path also passed on hardware: a 16x16 RGB565 rectangle was sent to the screen, and 0.2 seconds of raw 16-kHz mono S16LE silence was sent through the speaker path. Both operations exercised session-scoped leases, one-use stream tickets, binary binding, and release. This completes the practical scan-to-import-to-blit milestone. A fresh physical pair gesture remains unverified because the device already had the supplied trust anchor.

Speech findings and fixes

The user twice opened the gate with the physical button. limb listen observed the authenticated button gate event and successfully bound an audio_in stream, but the device sent no AUDIO_IN_PCM frames. A simultaneous raw capture remained exactly zero bytes while gate status still reported the gate open. Review found that the SDK reports the total number of interleaved stereo samples, while the capture hook treated it as a count of mono frames. Every 512-byte microphone frame was consequently calculated as 1024 bytes and dropped by the bounded capture slot. The hook now extracts one 256-sample microphone channel, preserves the DMA-point timestamp, and fits the expected 512-byte slot. The corrected image was built, linked with the event wrapper, flashed, and verified. A physical retest then captured and recognized a spoken question end to end. During that retest, an STT event could cancel a partially read device frame and make the next PCM bytes look like a frame length. The stream client now retains received bytes across cancellation; a split-frame regression test covers that exact interleaving.

The first successful final result also exposed a lifecycle gap: the CLI kept the physical gate open until its hardware deadline even though the utterance was complete. listen now sends the protocol's early gate_close immediately after its first final result, so the indicator turns off, the command exits, and a later local gesture starts a fresh cycle. One watch process subsequently completed two consecutive physical cycles: both recognized and played a response, both extinguished the indicator after final, and the second local gesture opened a fresh gate as intended.

Repeated watch attempts then revealed that an earlier watch and its child listener were still alive. Multiple listeners raced for the protocol's single audio_in stream, so each newly bound connection correctly replaced the old one and surfaced as host-side connection resets. The CLI now holds per-device process locks for both watch ownership and actual input capture, reports the existing PID instead of opening a competing stream, treats a reset as clean only when gate_status confirms closure, and reaps child processes on Ctrl-C. The duplicate processes were removed. A later apparent network disappearance was reproduced by opening a generic serial monitor whose default RTS/DTR initialization controlled the board's RESET/BOOT lines; closing the monitor restored network service. It was not evidence of a Wi-Fi reconnect failure. Ctrl-C then stopped the validated watch without leaving a child listener; a lock regression test independently covers duplicate ownership and release.

The standalone ListenAI STT integration test successfully connected, uploaded 16-kHz mono S16LE audio, ended the session, and received a clean finish. The legacy TTS PCM WebSocket connected, but the service rejected its old default voice with error=2 (parameter error). The current v2 endpoint instead returns a short-lived HTTPS stream URL. The provider now sends integer status values, requests 16-kHz PCM with a current voice, securely follows that URL, and handles fixed-length, chunked, or connection-delimited HTTP bodies. Old official stream_data configurations migrate in memory without rewriting the user's registry. A live TTS-to-STT test recognized the synthesized PCM.

The first successful spoken playback exposed regular gaps. Device review found that the adapter inserted silence whenever its outer queue was empty for 20 ms, even though the SDK still held up to 120 ms of real audio and already supplied silence on a genuine underrun. Removing the duplicate insertion and pacing the host against an absolute clock with a 200 ms prebuffer eliminated the regular gaps, but repeated watch cycles still exposed lighter stutter. A three-second raw sine wave played smoothly without increasing the device's dropped-block counter, isolating the remainder to the TTS path. The HTTPS service returned PCM in highly fragmented chunks, including pieces smaller than 100 bytes; the host had forwarded every transport chunk as its own device frame. It now coalesces those boundaries into 3,200-byte (100 ms) PCM frames and sends only one shorter tail frame. Live PCM continuity and TTS-to-STT checks passed, and the user confirmed that physical playback was now very smooth.

Daemon reception follow-up

The first-contact implementation depended on a host process already holding a device control connection when the physical gate opened. watch also created one listener per turn. That made an ASR or stream failure indistinguishable from losing the next receive window and encouraged multiple listeners to race for one input stream.

The current path replaces that polling lifecycle with an authenticated gate beacon and the per-user limbd broker. A paired device announces a new gate epoch over UDP, buffers its opening PCM, and accepts one ticket-bound input stream through gate_accept. The daemon keeps no idle device TCP connection, owns one local consumer per device, and treats provider or stream failures as belonging only to the current epoch. limb listen --follow, limb watch, and the OpenClaw channel now keep one local subscription across turns. Host tests cover beacon authentication, exclusive subscriptions, token hot reload, and failure recovery; both supported board builds cover the shared firmware path.