feat(chat-routes): persist assistant parts incrementally as a turn streams

[drewstone]

What

The assistant row is written once today, after the stream drains. A viewer arriving mid-run reads an empty transcript unless the streaming gateway's hot event buffer still holds the turn. This persists the assistant row incrementally as the turn streams, in both lanes.

Why this and not "raise the buffer TTL"

The gateway's hot buffer is one Redis sorted set per session, EXPIRE refreshed on every push. Lingering keys = arrival_rate x TTL, so memory is linear in the TTL:

TTL	lingering keys @10k concurrent	tool-heavy memory
60 s	10,020	73 GB
5 min	50,100	365 GB
60 min	601,200	3.66 TB
24 h	14.4M	88 TB

A 12x TTL bump buys 12x memory and zero extra coverage for a viewer who opens the tab tomorrow. The hot buffer must stay a short reconnect window; durable storage must be the history tier — which only works if storage actually holds the in-flight turn. That is this PR.

Design

• /stream draftAssistantParts — finalizeAssistantParts MINUS the dangling-tool terminalizer. A mid-stream snapshot through the finalizer would rewrite every in-flight tool to state.status:'error' + metadata.terminalized, so a reader would see phantom tool failures the final write silently reverses. Terminalization stays a completion-time decision. Pending asks likewise stay pending.
• /chat-routes/draft-persistence — coalescing writer: time floor (default 2 s), dirty gate (content events only — heartbeats and lifecycle envelopes never arm a write), single-flight (an in-flight write suppresses the next trigger rather than queueing), best-effort (a store outage degrades freshness, never the stream), redaction parity on draft text AND every draft text part, draft-only tool-output cap (32 KiB) with size-based interval backoff.
• Both lanes — createChatTurnRoutes (incrementalPersistence, on by default when the store can patch rows) and runDetachedTurn (persist, opt-in; transfers row ownership and returns messageId).
• Idempotency reuses the turn's OWN identity, no new state — the row id derives from deriveExecutionId (interactive) / turnId (detached). A re-entered turn looks that id up through the store's ordinary listMessages and patches the row a crashed attempt started. The final write is insert-or-patch under the same id, producing the row today's single write produces; an empty turn retracts the draft.
• resolveChatTurn gains hasRunningTurn so a draft row cannot break retry dedup. Trailing assistant rows are walked past only while the thread's turn is still running in the turn buffer — so a retry converges, and a user genuinely repeating a message still gets a new turn.
• /chat-store gains updateMessage/deleteMessage and honors a caller-assigned row id.

Why 2 s

Measured on a real tool-heavy run: 517 stream events over ~90 s (~5.7 events/s). A 2 s floor plus the dirty gate turns 517 candidate writes into <=45, leaving the row at most 2 s stale — an order of magnitude below the time it takes a viewer to open a tab and render. Fleet arithmetic at 10k concurrent 60 s runs: <=30 updates/run x 167 run-starts/s = ~334 row-updates/s across per-tenant shards. Product-tunable via incrementalPersistence: { intervalMs, backoffBytes, maxDraftToolOutputBytes }.

Proof — a real sandbox turn, real model, real SQLite

tests/chat-routes/incremental-persistence.live.test.ts (skipped without LIVE_SANDBOX=1 + creds) against a real box on sandbox.tangle.tools, a real 26.5 s agent turn with two real bash tool calls, driven through the real createChatTurnRoutes into a real better-sqlite3 chat store, with an independent poller reading the DURABLE row every 100 ms:

t+16412ms  chars=  4  parts=2  tools=[]
t+18730ms  chars=428 parts=3  tools=[running]
t+22760ms  chars=428 parts=6  tools=[completed,running]
t+24871ms  chars=428 parts=8  tools=[completed,completed]
final:     chars=469 parts=9  inputTokens=24910
           rowId=assistant:agent-app-live:2a949e3068f8974fd49cdddd80458ef4:0

The row grew from 4 chars to 469 while the turn ran, in-flight tools persisted as running (never terminalized), exactly one assistant row survived, and the settled row byte-matched producer.assistantParts() — the only thing today's persistAssistantMessage writes.

tests/chat-routes/incremental-persistence.test.ts — 10 tests over the real drizzle/SQLite store (no store fake): mid-run growth queried from inside the stream, byte-identical final row vs. the single-write path, coalescing (200 token events -> 1 append + <=3 patches), empty-turn retraction, draft redaction parity, interactive-lane crash convergence, detached-lane crash convergence, the genuine-repeat non-regression, and the additive guarantee.

Every test proven capable of failing

Reverted	Failure
draft projection -> finalizer	expected 'error' to be 'running'
incremental persistence off	waitUntil timed out: assistant row after first text
turn-identity walk-back	expected [2 rows] to have a length of 1 (duplicate user row)
row adoption on re-entry	expected 'Checking ' to be 'Checking the lease. Found 2.'
draft scalar redaction	expected 'SSN 123-45-6789 on file' to be 'SSN [redacted] on file'
draft parts redaction	expected '[{"type":"text","text":"SSN 123-45-67…' not to contain '123-45-6789'
write coalescing	expected 182 to be less than or equal to 3
empty-turn retraction	expected [1 row] to have a length of +0
detached-lane draft notify	waitUntil timed out: partial row from the crashed detached attempt

Gates

pnpm typecheck clean - pnpm test 194 files, 3161 passed, 1 skipped (the live test, correctly skipped without creds) - pnpm build clean - agent-docs --check fresh.

Additive

A store without updateMessage keeps today's exact single-write behavior. An explicit opt-in against such a store throws at route construction rather than silently degrading every turn.

[tangletools]

✅ Auto-approved drewstone PR — 5de90df8

This PR was opened by the trusted drewstone account.
The full PR reviewer audit still runs separately and will publish findings if it detects issues.

_{tangletools · auto-approval · reason: drewstone_author · 2026-07-25T00:40:12Z}

[tangletools]

✅ Auto-approved drewstone PR — eb51c186

This PR was opened by the trusted drewstone account.
The full PR reviewer audit still runs separately and will publish findings if it detects issues.

_{tangletools · auto-approval · reason: drewstone_author · 2026-07-25T00:47:46Z}