perf(authority): reuse owned state during SQLite and archive replay

[LIHUA919]

Refs #4224 and shared-authority RFC L6 / D2. SQLite historical reads repeatedly copy, sort and encode unchanged projection subtrees; receipt lookup also materializes complete projections it discards. This PR makes retained replay cheaper without changing persisted proof bytes, and fixes archive consumers being able to corrupt the decoder's next replay basis by editing a returned projection. It does not activate a default or claim D2 qualification.

Result and ownership

• One privately owned TS AuthorityStateReplay serves SQLite historical proofs and provider-neutral archive recovery. Changed paths are copied; unchanged subtrees reuse canonical encodings. Private state never escapes. Weak object caches and bounded string buffers live only within the read/audit.
• Every accessed commit still validates its exact v0 SHA-256 bytes, lineage, events and original receipts. Empty non-root deltas reuse their proven predecessor state digest; roots still derive their own proof. Receipt lookup avoids constructing unused snapshots; scan rows are independent objects.
• Archive callers can edit returned rows without poisoning subsequent delta replay or the terminal seal. The regression failed before the fix and passes afterward. Failed delta batches do not advance private state; sparse protocol arrays now fail validation instead of skipping absent elements.
• The separate digest-window encoder and intermediate receipt projection collection are removed. The existing state-log owner handles reconstruction; there is no new capability, provider, storage format, checkpoint interval, authority policy or default switch. Reverting the optimization needs no migration.

Matched evidence

Linux x86_64 (16 vCPUs), Node 22.22.3 / SQLite 3.51.3; baseline 96ce9efb3 versus runtime source 758db221e (final head adds documentation only). Five workloads ran sequentially before/after SQLite, followed by five candidate File arms. Each read has 20 samples; writes use the last 100 commits; five cold-process samples include imports, not cold OS cache. All 15 arms verify complete projections/Todo metadata, original receipts and reopened state. These are bounded storage observations, not full CLI/Turn or formal capacity results.

Workload / commits	SQLite receipt p95 ms, before → after	SQLite scan100 p95 ms, before → after
Mixed 20 KiB / 128	67.69 → 16.04	184.69 → 71.37
Mixed 20 KiB / 512	75.70 → 17.18	187.22 → 67.61
464 Todos + 64 leases / 128	786.01 → 91.52	1964.03 → 623.49
Changing 1 MiB / 128	456.97 → 288.76	1190.10 → 1114.94
Fixed 64 KiB / 128	62.96 → 9.57	170.34 → 56.36

Many-field receipt/scan costs fall 88%/68%, but still miss the 50/250 ms targets on this host. The changing-1-MiB scan improves only about 6%; returning 100 large snapshots remains costly. Its post-fill RSS rises from approximately 118 to 160 MiB, including fixture/verification allocations; this is not evidence of stable long-run memory. A contended macOS paired run is excluded from timing claims, while its correctness evidence remains retained.

SQLite has lower write/restart cost than File in all five measured shapes. File has cheaper warm receipts and the many-field warm head; SQLite now wins ordinary/many-field scans, while File wins the changing-1-MiB scan. This supports SQLite as the short-term implementation target for the next qualified new-Goal default, rather than an unconditional read-performance claim. The bilingual RFC and provider reference contain the full provider comparison, reproducible command and dependencies. Both providers require Node >=22.22.3; SQLite uses built-in node:sqlite with actual driver admission, no extra database package or silent fallback.

Validation and limits

• Node 22.22.3: 344/344 SQLite/state-log/runtime/migration/capacity tests; 354/354 File/common/archive/crash/migration tests; 305/305 real isolated PostgreSQL 16 store/archive tests; zero skips. 5/5 real Python→TS SQLite CLI tests.
• Independent Linux run after measurements: 33/33 native replay, bounded SQLite history and File/SQLite archive tests; zero skips.
• TypeScript project and explicit state-log/archive-test/runner typechecks pass. Exact-scope quality receipt cqr_5ddb7b3d0e0a1668d2e1 is valid for final scope 5ddb7b3d0e0a1668d2e1f25b558c5af4afdff35c2ed99e48f8bdcbab2f434930 (10 files, zero blockers, one disclosed performance warning; safe-fix allowed but no extra fix pass applied). Risk-based premerge passes: 3 direct checks, 5 catalog canaries, 8 risk-profile smokes and the public-boundary check; zero failures/manual holds.
• Historical formal 1-MiB/100k failures on e99a83b63 remain recorded (receipt 269.03 >50 ms; scan100 801.81 >250 ms). This PR does not relabel them. Reference-runtime 10k/100k/headroom reruns, sustained RSS, consumer lag, large-history recovery/platform coverage and >=10-day elapsed soak remain D2 qualification work.
• D3 integrated Goal operations/migration recovery and L9 new-Goal onboarding/settings/install entrypoints remain release dependencies. Reuse merged feat(authority): add reviewed File and SQLite provider cutover #5173, fix(turn): supervise managed Host process lifetimes in TypeScript #5144 and refactor(state): retire legacy Todo events and isolate supervisor history #5054 instead of counting them as new tasks. No live Goal was promoted or rewritten by these experiments.

No frontend companion is needed: provider APIs and projection shapes are unchanged; the affected real CLI bridge and real storage backends are covered. The future-facing refactor is applied at the existing TS state-log boundary and removes duplicate reconstruction work. Runtime merge remains a maintainer decision after exact-head review/CI.

[huangruiteng]

Author validation for a75e168. Verdict: READY FOR MAINTAINER REVIEW; NOT D2 DEFAULT QUALIFICATION.

The relevant product outcome is cheaper retained-history recovery without stranding long-running callers or changing the data they recover. The shared TS replay owner preserves exact v0 proofs, receipts, metadata and provider selection. Returned archive projections are now detached; the prior consumer-mutation failure was reproduced before fixing it. Rejected delta batches cannot advance the replay frontier; sparse protocol arrays now fail closed. No domain-specific policy, implicit permissions, new default or frontend contract is introduced. File and real PostgreSQL coverage exercise the shared changed boundary.

Validation: 344 SQLite/state-log/runtime/migration/capacity tests, 354 File/common/archive/crash/migration tests, 305 real isolated PostgreSQL 16 tests, 5 Python-to-TS SQLite CLI tests, and 33 Linux native replay/archive tests passed; zero skips. Fifteen matched Linux storage arms preserve full records and original receipts. Typechecks and risk-based premerge passed (3 direct checks, 5 catalog canaries, 8 risk-profile smokes, public-boundary scan; no failures/manual holds). Exact quality scope 5ddb7b3d0e0a1668d2e1f25b558c5af4afdff35c2ed99e48f8bdcbab2f434930 is valid: 10 files, zero blockers, one performance warning; safe-fix allowed, no additional safe-fix pass applied.

The warning is substantive: large-state receipt/scan targets remain unmet on the comparison host, and 1-MiB post-fill RSS grows from roughly 118 to 160 MiB including fixture allocations. Bounded speedups do not prove stable memory, 10k/100k capacity or ten elapsed days. Existing failed formal evidence and D2/D3/L9 dependencies remain explicit in the updated RFC/reference. No live Goal was migrated and no runtime self-merge is performed. CI for this new remote head is separate from the completed local checks.

[huangruiteng]

审查 exact head：a75e16849d3f046f061dce8bb462a9a5ad943f52。结论：未发现阻塞问题，APPROVE。本次从不可变 merge base d23f1c87d05dce02ed6368ff91ff8b2d0c163213 重做整个 PR 的检查，没有沿用先前 head 的结论；没有执行合并。

动机

按 #4224 的既有评审框架 与共享 Authority RFC §7.2，本 PR 是可独立交付的重放成本修复，不是 D2 默认资格认证。目标是减少历史 receipt/scan 对完整状态的重复复制与编码，同时保留全部历史 projection、原始 receipts、events 和原有 v0 digest。

归档部分也解决了真实缺口：我用同一份四记录归档在 base/head 执行，旧版调用方修改第一条返回 projection 后，第二条报 archive transaction belongs to another goal；新版逐条修改返回对象仍能读完四条并通过 terminal seal。普通归档 bytes、完整记录、restore 后原始 receipt 及下一次独立 CAS 保持一致。

改动思路

复用既有 TypeScript state-log owner，而不是新增另一套状态决策源。AuthorityStateReplay 私有持有 canonical 状态，只复制变化路径，复用不可变子树的精确 JSON 编码；仍对完整原始 v0 字节计算 state/commit proof，不是新的 Merkle 格式。对象 cache 是 weak keys，长字符串 buffer 单项 2 MiB、总量 4 MiB，生命周期仅本次 read/audit。

SQLite receipt 验证完整覆盖窗口，却不构造不会返回的 projection；scan 和 provider-neutral archive 返回独立对象。合法输入与存储格式保持兼容；归档对象隔离、失败 delta 不推进 frontier、稀疏 protocol array 拒绝是明确披露并验证的通用修复，不能笼统宣称全部变化只在 SQLite 开启时发生。

具体改动

• authority_state_log.ts:197 AuthorityStateReplay：canonical 输入隔离、整批 delta 原子发布、path-copy 和精确编码复用；decodeAuthorityStateDelta:316 用逐项验证拒绝 sparse operation/path/insert holes，合法 sparse state value 仍保持 JSON null 语义。
• sqlite_authority_store.ts:250 verifyCommitRow / :306 verifiedRange：保留 sealed/predecessor、cursor/parent/state/完整 commit 校验。仅非根空 delta 可复用已证明 predecessor digest；root、events、receipts 不跳过证明。receipt/scan/audit 使用同一 replay，移除中间完整窗口对象；writer、schema2 和 64 间隔不变。
• authority_archive_read.ts:86 archiveRecords：decoder 保持私有 replay，yield detached projection；现有 reviewed-copy、Goal identity、顺序、EOF/seal、验证后 restore 边界保留。实际 File/SQLite/PostgreSQL 恢复路径已验证。
• 三组测试覆盖 snapshot/input ownership、失败批次 frontier、Unicode/numeric/__proto__ 字节语义、cache 大项与淘汰、空根/空 delta、损坏拒绝以及消费者修改归档记录。
• 152 行显式 disposable provider comparison runner 复用既有生产规模 fixture/latency helper；校验完整 projection/receipt/reopen 与 source/runner 身份。三个文档更新 File 成本模型、matched experiment、SQLite 未来资格方向和仍未满足的 D2/D3/L9，不改变 release 默认。

对主干的风险

最大风险是编码缓存/可变别名导致 proof 或历史字段丢失。因此我在真实 File/SQLite 上做了双向兼容：同一 base-written 数据由两个 revision 读取，同一 head-written 数据也由两个 revision 读取；每组 48 个完整观察一致，包含 66 次提交、checkpoint 65、原始 receipt、7/66 分页、默认 File/显式 SQLite、重开、完整诊断及重叠非法条件的拒绝顺序。六个真实 SQLite 损坏观察同样保持完整诊断一致；故意让生产 scan 丢掉 Todo metadata 会触发独立全量 fixture 断言，恢复后通过，不是用实现当前输出生成 expected。

本次 exact-head 本地验证：421 个 SQLite/state/codec/admission/migration/provider 测试、677 个 shared/File/NoKV/archive/transaction 测试、305 个隔离真实 PostgreSQL 16.15 集成测试、5 个实际 Python→TS SQLite CLI 测试全部通过且无 skip；control-plane typecheck、新 runner/测试显式严格编译、docs governance、13 个 RFC index 测试通过。premerge 执行 3 个直接检查及 5 catalog + 8 risk-profile + 1 public-boundary 检查，均通过；10 个变更文件 boundary scan、diff 和 DCO 检查通过。

首次并行启动 shared suite 曾有一条未改动的 Python succession 子进程在 90 秒上限超时。保留首次失败记录后，同一测试在不可变 base 和 head 隔离运行均通过，原十模块组按原 timeout 串行重跑 677/677 通过；没有改代码或提高超时来抹掉失败。按 review packet wait_for_ci=false，未查询或等待远端 CI；本次结论基于上述本地证据，不拿远端 CI 颜色代替 PR 的因果判断。

独立性能对照使用相同 runner SHA、真实 SQLite 3.51.3 / Node 22.22.3、各 128 commits / 20 samples，base/head 顺序执行：

workload	historical receipt p95	scan 100 p95
full	270.7 → 37.1 ms	705.3 → 231.1 ms
changing-1m	459.1 → 113.8 ms	1046.4 → 466.3 ms

这是共享主机上的有界复核，不是稳定容量证明或作者 Linux 15-arm 实验的独立复现；cold child 包含 module loading，RSS 包含 fixture/验证分配，不是 steady state。尤其 changing-1m 的 receipt/scan 仍分别超过 50/250 ms，正式 10k/100k 大状态失败不能由这里的小规模结果改标为 pass。未提高任何冻结阈值，未测试或改变活动 Goal/fence/provider。

我的整体评价

这是有正收益、边界完整且可逆的阶段修复：复用同一 typed owner，删除重复重放/中间物化，修复实际归档 ownership 缺陷，保留原始 digest 与历史读回；不是仅添加测量字段或重复 smoke。相关 future-facing pass 已在共享 replay owner 与 SQLite window 边界应用；暂不增加更广泛缓存框架或额外协议。没有新增 CLI/前端/Lark 配置路径，因此不需要伴随新的配置编辑器。

剩余最大证据缺口明确留给既有 #4224 D2 owner：原容量 profiles、many-field/1 MiB budgets、steady-state RSS、恢复/lag/OS matrix、十个 elapsed days，以及 D3/L9 的集成切换/新 Goal 默认资格。文件未来默认方向不等于现在激活；File 仍是当前无 selector 默认。相关 #5169 的新幂等语义与 #5175 batching/retry 是独立未合入工作，不能借用其新契约来证明本 PR。当前代码/文档均没有冒充这些验收已完成，因此这些后续资格项不应阻止这个有界修复的批准。建议维护者合并此 exact head；本 review 本身不授予生产切换或自合并权限。

English verdict: APPROVE - head a75e168; no blocking findings. Owned replay preserves full original v0 proofs/receipts and fixes the independently reproduced archive consumer-alias regression. Verified 421 SQLite/shared-state, 677 common/File/NoKV/archive, 305 real PostgreSQL, 5 CLI tests and risk-based premerge; bounded base/head performance is not D2/default qualification.