Skip to content

Repository files navigation

Voice Agent Runtime

tests release

RU. Рантайм голосового агента: политика диалога таблицей переходов, перебивание без доигрывания фразы, бюджет задержки по этапам, ёмкость линий. Подробности о production-системах, из которых это выросло: 94spec.github.io.

A chat agent that answers slowly is annoying. A voice agent that answers slowly is hung up on. This repository is the part of a voice agent that has nothing to do with the model: where the conversation is allowed to go, what happens when the customer interrupts, how long each stage of the pipeline is allowed to take, how many calls the thing can hold at once, and what a reply must satisfy before it is allowed to be spoken.

Nothing here calls a model, telephony, speech recognition or synthesis: replies come from a scripted responder, so a scenario runs identically on every machine and in CI.

Try it

python -m pip install -e .

voiceagent lint-policy                    # stages, transitions, handoff rules
voiceagent simulate                       # every scenario, with a readable trace
voiceagent simulate --scenario barge-in
voiceagent budget --turns 2000            # where the latency actually goes
voiceagent capacity --lines 10            # can ten lines hold the load?
voiceagent capacity --find-lines --calls-per-hour 60 --max-rejection-rate 0.01
voiceagent guardrails --reply "Смотрите на https://example.invalid"

Running from a checkout without installing works too: PYTHONPATH=src python -m voiceagent …

Barge-in is the rule that shapes everything

When the customer starts speaking, the agent stops mid-word — and does not resume. Repeating the interrupted sentence is the single most irritating thing a voice agent does, because the person interrupted precisely to avoid hearing it. What was already spoken counts as said; the rest is dropped, and the next reply answers the interruption.

$ voiceagent simulate --scenario barge-in
    [ greeting → discovery] клиент: Да.
                            агент: Добрый день, звоню по вашей заявке. Удобно говорить?
  ! [discovery → answer   ] клиент: Стоп, только сроки скажите
                            агент: Программа рассчитана на девять месяцев, занятия в записи. Подскажу, как записаться?

The runtime records how much of the reply was audible before the interruption, so a trace shows what the customer actually heard rather than what was generated.

The policy is a table, not a paragraph

An agent that decides its next stage inside the prompt cannot be tested: the same sentence moves the conversation differently on two runs. Here the stages and the moves between them are declared, and the model only chooses what to say within a stage.

   greeting (max 2 turns) → discovery, handoff, closed
  discovery (max 4 turns) → answer, next_step, handoff, closed
     answer (max 5 turns) → answer, discovery, next_step, handoff, closed
  next_step (max 3 turns) → confirm, answer, handoff, closed
    confirm (max 2 turns) → closed, handoff

A move the policy does not allow is not a judgement call — the call goes to a human rather than somewhere undeclared. So does a stage that stops advancing, a customer who asks for a person, a complaint, a refund question, and a second "that is not in what I was given" in a row. Every one of those has a test.

Latency is measured per stage, not per model call

"The model is slow" is the intuitive diagnosis and usually the wrong one. The reply often starts late because endpointing waited for a pause that never came, or because the retrieved context grew and the model had more to read. So every stage declares a budget and the simulator reports where the time went:

$ voiceagent budget --turns 2000
turns simulated: 2000   target: 10000 ms per turn (3000 ms to the first reply)

  p50    2632 ms
  p90    3455 ms
  p95    3719 ms   within target
  p99    4394 ms

where the time goes:
   47.5%  p95   2147 ms  model
   27.8%  p95   1277 ms  endpointing
   14.0%  p95    633 ms  speech_synthesis
    6.0%  p95    286 ms  retrieval
    4.9%  p95    228 ms  transport

biggest consumer: model
turns within target: 100.0%

Targets are the ones this design is built for: 3 seconds to the first reply, 10 seconds per turn. --skip endpointing prices a change before anyone builds it — the report says what removing a stage would actually buy at p95, which is where the caller feels it, rather than on average, where it hides.

Capacity is a number, and it is not the line count

Ten lines is not ten calls per hour and it is not unlimited. The number that matters is how often an incoming call finds every line busy, because the caller does not know that nine others were answered.

$ voiceagent capacity --find-lines --calls-per-hour 60 --mean-call-seconds 240
11 line(s) keep rejections at or under 1.0%
  measured rejection rate: 0.55%
  utilisation: 40.8%
  peak concurrent: 11

A discrete-event simulation over synthetic arrivals rather than a closed-form formula, so the assumptions are visible in the code instead of hidden inside an expression that quietly assumes exponential everything.

A reply has to be speakable

Text written for a screen fails out loud in ways a transcript hides. Every reply is checked before it can be spoken, and a violation is not a logged warning — the reply is rejected and the runtime falls back to a handoff, because the alternative is the customer hearing it.

Rejected: links and email addresses, bullet lists and markup, characters with no sound, numbers long enough to become a digit stream, more than two sentences, anything over the character limit, a reply that ends without a question or a stated next step, and an answer that uses knowledge the retrieved fragment does not support.

Accepted and counted separately: the honest "that is not in what I was given". That is the behaviour the policy wants — but two in a row mean the agent cannot help, and the call belongs to a person.

Layout

src/voiceagent/policy.py      stages, allowed moves, handoff rules, validation
src/voiceagent/runtime.py     the turn loop: barge-in, moves, refusals, trace
src/voiceagent/guardrails.py  what a reply must satisfy to be spoken
src/voiceagent/budget.py      per-stage latency simulation and percentiles
src/voiceagent/capacity.py    concurrent lines, rejection rate, lines needed
src/voiceagent/scripted.py    the deterministic responder and the scenarios
docs/adr/                     why the design is what it is
tests/                        73 dependency-free tests

Related

License

MIT

About

Dialogue runtime for a voice agent: policy as a transition table, barge-in that never resumes, per-stage latency budget at p95, line capacity, speakable-reply guardrails. Dependency-free, no vendor calls.

Topics

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages