Voice agent announces the tool call instead of running it

[YellowSnnowmann]

Summary

Asking the realtime voice agent to summarise emails returns no summary. The final answer delivered to chat is the agent announcing that it is about to fetch the inbox.

Problem

What happened. Asked the voice agent "summarize my emails". The final response delivered to the chat thread was:

"Let me pull up your inbox now — if it takes a moment I'll drop the summary in your chat."

No inbox summary ever arrived, in chat or spoken.

What was expected. The inbox summary.

Why it happens. That sentence is the VOICE_DIRECTIVE's own example, echoed almost verbatim:

"Sure, let me pull up your inbox — if it takes a moment I'll drop the summary in your chat."

The directive instructs the model to speak before acting:

"Before you use a tool or delegate … first say one short spoken sentence telling the user what you are doing … then proceed."

But a reply carrying only text and no tool call is the end of a turn. A model that follows the instruction emits the sentence, its turn ends there, and the harness delivers that sentence as the final answer. The Gmail call never runs. The more obediently the model complies, the more reliably the turn fails.

Impact. Any voice request that needs a tool — email, calendar, files, web — can silently return a promise instead of an answer. The failure is invisible: the user is told the result is coming, so nothing reads as broken until they notice it never arrived.

Steps to reproduce.

1. Start a realtime voice chat on the Human tab.
2. Ask something that requires a tool ("summarize my emails").
3. Observe the chat thread: the delivered answer is the announcement, not the summary.

Environment. Desktop dev build, macOS, realtime voice path (src/openhuman/voice/realtime_harness.rs).

Solution

Remove the preface clause and instruct the opposite: call the tool and answer from the result.

The clause existed for a latency reason that no longer holds. It was there to get audio to the caller early, back when the relay's spoken filler was inaudible until the turn closed — the provider segments on sentence boundaries, and the filler was ellipsis-terminated (tinyhumansai/backend#1275). The relay now speaks a terminated sentence ~700ms in, and does so without depending on the model choosing to speak first.

The replacement states the reason as well as the rule, since a bare prohibition invites the model to hedge back into the same behaviour.

Acceptance criteria

• Repro gone — a tool-backed voice request ("summarize my emails") returns the actual result, not an announcement of intent.
• Regression safety — a test pins the directive's contract: it must forbid announcing, and must not carry the phrasing that caused this.
• No silent regression path — the acknowledgement the caller hears no longer depends on the model choosing to speak, so removing the preface does not reintroduce dead air.
• Diff coverage ≥ 80% — the fix PR meets the changed-lines coverage gate (Vitest + cargo-llvm-cov, enforced by .github/workflows/ci-lite.yml).

Related

• Replace current voice chat with ElevenLabs Voice Agents #5399 — realtime ElevenLabs voice agent (parent)
• tinyhumansai/backend#1275 — the relay filler that makes the preface unnecessary
• fix(voice): make the deadline line a neutral acknowledgement #5549 — the deadline line, same spoken-copy surface