Subtext (家常) — AI Voice Agent for Elderly Care & Caregiver Insights

Subtext (家常) is an autonomous, multi-agent AI voice system built for AI Tinkerers x Tencent Cloud Hackathon: Agent Development Challenge 2026.

It holds short, unhurried daily Mandarin voice conversations with an elderly parent over Telegram, infers what she is not saying against her own 14-day baseline, and delivers clear, actionable insights to her adult child.

🎬 Demo Video: Watch on YouTube
📊 Live Audit Trace: View interactive trace dashboard

Important

Hackathon Prototype Notice: This project was created specifically for the AI Tinkerers x Tencent Cloud Hackathon: Agent Development Challenge 2026. It is a functional proof-of-concept demonstrating multi-agent voice inference, baseline tracking, and deterministic safety hooks — it is NOT production-ready software and must NOT be used for live clinical, medical, or emergency healthcare management.

---

Table of Contents

• The Insight: Why Voice Subtext Matters
• How It Works: Multi-Agent Architecture
• Role Specifications & Cost Architecture
• Core Design & Safety Principles
• System Architecture & Data Flow
• Getting Started & Runbook
• Live Evidence & Artefacts
• Honest Limitations
• Roadmap: Deliberately Not Built
• Safety Layer Evaluation & Benchmarks
• Development Log & AI Engineering Story

---

The Insight: Why Voice Subtext Matters

Ask an elderly Chinese mother how she is and she will say 我没事 ("I'm fine"). She is not lying; she is protecting her adult child from worry, and protecting herself from feeling like a burden. The critical information a caregiver needs is almost never in what she explicitly says.

It is hidden in what she avoids: an appointment mentioned without a date, a daily cooking habit that quietly stops, or asking the exact same question three times in one week. Standard LLM summarisers read the transcript literally and conclude "she is fine".

Subtext reads the transcript for evasion, scores subtle shifts against her own 14-day behavioral baseline, and sends her caregiver the single most important action worth taking today.

The product is not a chatbot for lonely elders. It is an instrument for noticing, pointed at a person who will not tell you when something is wrong.

---

How It Works: Multi-Agent Architecture

                    PLANNER  ── decides whether and when to reach out today
                       │        (once daily, reads across days)
                       ▼
  MOTHER ◀── voice ── COMPANION ── executes the plan, every draft via CRITIC
   (Telegram)          │
       │  voice        ▼
       └──────────▶ LISTENER ──▶ signals ──▶ DIGEST ──▶ CAREGIVER (Telegram)
                       │                                     │
                    MEMORY (SQLite)  ◀── relays, corrections, calendar ──┘

---

Role Specifications & Cost Architecture

Subtext splits governance across specialized LLM roles using a tiered reasoning architecture:

Role	Does	Model	Temp	Timeout	Runs
Planner	Decides whether to talk today, when, and what for	deepseek-reasoner	0.0	120s	once a day
Listener	Reads for evasion; every signal quotes her verbatim	deepseek-reasoner	0.0	120s	once a day
Companion	Two sentences per turn, never interrogates	deepseek-chat	0.7	30s	per turn
Critic	Reviews every outbound message before she hears it	deepseek-chat	0.0	30s	per draft
Intent	Classifies what the caregiver just said	deepseek-chat	0.0	30s	per message
Inferred ask	Decides what to ask him for, unprompted	deepseek-chat	0.0	30s	per turn
Digest	≤120 words to her son, ending in exactly one action	—	—	—	once a day

The reasoning tier runs twice a day; the fast tier runs per turn. That is the cost architecture, and the table is here so it can be checked rather than taken on trust. Measured: about USD 0.0017 per voice round trip, roughly USD 0.35 per month per elder at the spec's 8-turn daily conversation.

Temperature 0.7 on the Companion alone is deliberate — warmth is the one place variance helps. The two 120s timeouts are measured, not guessed: the Listener was raised after a live run exceeded 30s, and the Planner matches it because it is the same model on a larger input. Every role has its own model constant and its own environment override, so retiering one cannot move another — the Critic in particular should never change by accident.

Tools: Google Calendar (read and write — the agent puts the reminder on his calendar itself), DashScope ASR/TTS, Telegram.

---

Core Design & Safety Principles

Two strikes, then a human takes over. A thread probed twice and still open is not probed a third time — it escalates to her son. A goal carried twice is not carried again. The same rule appears independently in both loops, enforced in code rather than in a prompt, and when the model disagrees the plan records what it wanted.

A safety component that cannot run must not approve. The Critic returns unavailable — distinct from reject — when it has no key, cannot reach the model, or gets a malformed answer. Nothing is sent. She hears silence rather than something unreviewed.

---

System Architecture & Data Flow

• Zero third-party Python dependencies. DeepSeek, DashScope, Telegram and Google Calendar are all hand-rolled over stdlib urllib. pytest is the only dev dependency; ffmpeg is a system tool.
• Roles behind one seam (invoke_role), so a role can be swapped without touching its callers.
• Transports behind an ABC. WhatsApp and WeChat are deliberate stubs that raise — those adapters are blocked on business verification, not engineering.
• Two test tiers. Default pytest is mocked, free and deterministic, with an autouse guard that fails any test reaching a vendor host. RUN_LIVE_ALL=1 runs everything against real keys.
• A daily cycle on the poll loop. The Listener, Planner, post_review, digest and calendar write are once-daily work triggered by the transport's idle tick — not by her speaking, because the silence alert has to fire when she does not.

---

Getting Started & Runbook

Full run-book, every command executed: docs/DEMO.md

brew install ffmpeg
python3 scripts/seed_history.py     # 10 synthetic days
python3 -m pytest                   # 373 passed, 30 skipped
python3 -u subtext/main.py          # -u matters; see DEMO.md

---

Live Evidence & Artefacts

docs/artefacts/ — real plans from live model calls:

• two consecutive days, where day two visibly inherits an unmet goal from day one
• a plan where code overruled the model on the probe cap, with the model's proposal kept verbatim
• a plan where both caps fire at once

---

Honest Limitations

• The Critic's stated reason is not always the reason. The verdict is reliable; the explanation attached to it is not. It receives only the draft, yet a live rejection cited "the mother's mention of being unwell" on a turn where she had mentioned no such thing. That reason is what lands in critic_review and in the trace, so the audit trail can be corrupted while the safety decision itself is correct. Read verdicts as evidence; read reasons as commentary.
• The caregiver's side is not persisted. His messages are classified, dispatched, replied to and discarded — no utterance rows, no record of the intent or its confidence. What he asked and what he was told cannot be reconstructed from the database, only from his phone.
• nightly.py --force does not force a fresh plan. It bypasses the daily-run marker, but write-once is enforced inside build_plan, which returns the cached plan for that date.
• ASR corrupts Malay loanwords. lontong → 隆冬, nasi → 那西, laksa → 拉沙. Context biasing is rejected by the endpoint (reproducible HTTP 400), so the transcript cannot be fixed from here. The Listener is told which tokens are recognised corruptions and not to read them as confusion — measured at 3-of-3 false concerning ratings before that note and 0-of-3 after — but the words in the transcript are still wrong, and a term outside the list is still read literally.
• Corrections are recorded, not learned from. When the caregiver corrects an inference, the agent updates that record and writes a correction row — but nothing reads those rows back into its reasoning. Reporting the correction rate is the honest v1; an agent that adjusts its own inference from them is a research problem, not a week's work.
• Temperature 0 is not reproducible on deepseek-reasoner; plans are written once per day rather than regenerated.
• speech_rate is not honoured — the TTS HTTP path exposes no rate parameter.
• Audio and plans accumulate; no purge schedule.
• One family. relay has no person_id.

---

Roadmap: Deliberately Not Built

Medication reminders, voice cloning of the caregiver, unlimited engagement — and the reasoning for each: docs/ROADMAP.md. Naming a capability you declined is a stronger statement than listing features you shipped.

---

Does the Safety Layer Degrade the Product?

Asked as a measurement rather than a hope. 70 short Mandarin utterances through the Critic's deterministic rules — 50 of them raw Companion output, generated live, because sentences we wrote ourselves had passed every previous test. 13 refused, 11 of them wrongly: every false refusal a two-sentence reply with exactly one question, refused for asking two.

The rate is not the finding; the shape is. 18 of 25 transcripts never tripped it and five tripped it every time — the family turns and the concealment turns, burden_avoidance and minimisation among them. 16% across the corpus was effectively 100% on the turns the product exists for, and each refusal spent one of three attempts, so those turns went silent. After the fix: 0 of 70, with genuine double questions still refused.

Full method, provenance, every refusal and all 50 drafts: docs/artefacts/local-rules-measurement.md

---

How This Was Built

Three coding agents, ten review rounds, a written handoff between each (docs/build-log.md). The recurring defect classes — code that exists and nothing calls, defaults that manufacture a reassuring answer, tests whose assertions contradict their own names — are catalogued there with the fixes.

Synthetic fixture data only. No real transcript or chat id is in this repository.