After-hours voice agent

A voice agent that answers a small business's calls outside working hours in Indian languages, handles a narrow set of things well, and queues everything else for a human callback the next morning.

Runs entirely on one machine. No paid APIs, no cloud inference, no per-call cost.

This repository currently contains the build scaffold, not the finished product. The specs, tests and orchestrator are written. Running the loop builds the implementation.

Quick start

# Windows
powershell -ExecutionPolicy Bypass -File scripts\bootstrap.ps1

# macOS / Linux
./scripts/bootstrap.sh

That installs dependencies, initialises git, and starts the unattended build. It does not stop to ask you anything. Expect a few hours.

What the loop does

for each of 9 steps:
    implement  ->  test  ->  fix  ->  test  ->  ...  (up to 4 fixes)
                              |
                              +-- still failing? escalate to Opus with
                                  permission to rewrite from scratch

after every 3 steps (33%):
    full-suite deep debug  ->  fix  ->  full suite  ->  ...  (up to 3)

Three properties make this trustworthy rather than a wrapper around hope:

1. State lives in orchestrator/tasks.yaml, not in a context window. Kill the loop, restart it, it resumes.
2. pytest decides whether a step passed, not the model. tests/ is written before the code and is reverted automatically if the agent edits it.
3. Failures are fed back verbatim. Real tracebacks go into the fix prompt.

Monitoring

python orchestrator/run_loop.py --status     # progress table
python orchestrator/run_loop.py --dry-run    # plan, with checkpoint markers
tail -f logs/orchestrator/loop.log           # live

Every Claude pass and every pytest run is written to logs/orchestrator/. When the loop finishes it writes BUILD_REPORT.md.

If a step gets stuck

The loop halts rather than building on broken foundations. BUILD_REPORT.md names the step and includes the failing output. To retry one step after a manual look:

python orchestrator/run_loop.py --only 4
python orchestrator/run_loop.py --from 5     # then continue

What gets built

Step	Module	What it does
1	config, logging, languages	Settings, business profiles, 11-language registry
2	stt	faster-whisper transcription with code-switching
3	tts	MMS-TTS synthesis, per-language voices
4	llm	Ollama intent classification and replies
5	dialogue	Business hours, flows, escalation, advice guard
6	session	VAD, turn-taking, model lifecycle
7	server	FastAPI + WebSocket + browser widget
8	store	SQLite calls, transcripts, callback queue
9	digest, cli	Morning report, voiceagent command

Honest limitations

• Browser widget, not a phone number. Real PSTN calls need a telephony provider, which costs money. session.py accepts audio frames from any source, so a bridge can be added later without touching the pipeline.
• One call at a time. 6GB of VRAM does not stretch further. A second caller is rejected with a clear message rather than crashing the first.
• No order backend. order_status captures the order ID and promises a callback. There is an OrderLookup protocol to plug into later.
• No tax advice, deliberately. The agent books an appointment instead. Wrong compliance advice is a real liability for a CA firm.
• Odia STT is weak. Whisper does not support it; TTS works fine.

Hardware target

Windows 11, RTX 3050 (6GB VRAM), 16GB RAM. LLM on GPU via Ollama, Whisper and TTS on CPU. Roughly 2-4 seconds per turn, which is fine for an after-hours line.