perf(authority): copy journal JSON without repeated primitive allocation

[LIHUA919]

Historical SQLite scans were allocating repeated immutable strings while materializing complete projections. This PR reuses the strict JSON codec for independent mutable containers and removes the extra canonical JSON stringify/parse step in SQLite scan reconstruction. Persisted canonical encoding, revisions, original receipts, strict validation and complete metadata remain intact.

The copy-only object traversal also avoids per-field entry tuples. It fills an object without a prototype, then restores the ordinary prototype; special keys and inherited setter names remain data. Focused fixtures cover sparse arrays, scalar edges, nested isolation and complete checkpoint history. Existing test setup is corrected for ambiguous bootstrap receipts, quota binding and the original execution ACK; runtime authority rules are not weakened.

The shared-authority RFC now distinguishes merging an improvement, a recoverable opt-in developer cohort, and selecting the release default. Proposed absolute latency budgets are engineering targets; matched current-release performance and consumer impact guide the tradeoff. Data loss, incorrect decisions, changed original receipts and unrecoverable migration remain blockers. Frozen report failures are preserved.

Matched local evidence

Pinned base 2bd9b32693085e9d9e33cd68cda641a8e546bf17, candidate 4a944ca19de494797152453a0c3cc42336f6311c; macOS arm64, Node 24.21.0 / SQLite 3.53.4, unchanged runner, clean source, sequential arms, 20 warm samples each. Small projections were repeated in reverse order. Values are scan-100 p95 milliseconds:

Complete projection / commits	Base	Candidate
Mixed ~20 KiB / 512	24.37 / 22.13	26.27 / 29.98
464 Todos, 64 leases, ~220 KiB / 128	205.20	243.79
Changing 1 MiB / 128	359.57	227.08

This improves the large-payload scan by about 37%, with a 2–8 ms small-projection increase and a 39 ms many-field increase. It is not a universal speedup. Current-state reads and write policy are unchanged; archive/export and outbox scans retain the tradeoff for observation. Every arm independently checked complete projections, events, original receipts, reopened history, exact historical retry and conflicting intent rejection. Final store bytes matched within each paired workload. These are bounded local observations, not population estimates or sustained-memory qualification.

Validation and limits

Current review head: e95f560117dea9793527c6691f857bc4ac26ca3a, integrated with main 88132051b0d302a4b752ba3be59302a9d799b642. The four production owner files are unchanged from the measured candidate 4a944ca19; the main integration adopts the current quota settlement tests. Timing above is historical matched evidence, not a newly measured soak on the integrated head.

• 741 real File/SQLite, codec/replay/scan, archive/migration and process-CAS tests passed. Complete metadata, original receipts, corruption, interruption and retry controls remain covered.
• 319 tests passed against a disposable real PostgreSQL 16.15 server: 315 authority-store and 4 archive tests. Suites ran sequentially; the server and data were removed.
• 54 focused Python tests and 2 real CLI quota/settlement tests passed.
• TypeScript typecheck, configured mypy, Ruff, semantic advisory/drift smoke and diff hygiene passed.
• Risk-based premerge passed on this head: 10 catalog canaries, 8 risk-profile smokes, public/private boundary and direct checks. Strict change-quality receipt cqr_dd942fd2ea52c5e2a508 verifies the exact diff; no unresolved quality blockers.
• One browser fixture test still fails because its expected four stages lag the six-stage fixture. The same failure was independently reproduced on pinned main 88132051b; existing #5418 owns that repair. This is a disclosed failure, not a passed test. Fresh remote CI and exact-head independent review remain pending; other old UI CI failures are not waived by this diagnosis.
• The earlier real source CLI rehearsal completed 100/1,000 commits with SQLite WAL/FULL, stable source and verified cleanup. Its formal ledger remains incomplete (25 missing rows); it does not establish sustained qualification.

Historical formal evidence is separate: the author reported 8 passed / 6 failed / 10 missing at d767b06f1, then 14 passed / 0 failed / 10 missing at 02d3dee83. Those reports do not certify this candidate. No ten-day/100k, cross-platform or release-default completion is claimed. High-load exploratory timing runs were excluded; raw local evidence remains outside Git.

No frontend, Lark or public CLI schema changes are needed: this changes internal materialization while retaining complete state and consumer values, verified through real providers and CLI readback. The bounded future-facing pass reuses the existing codec owner rather than adding another validator or storage format.

[mergify]

This pull request has merge conflicts with main and cannot be merged
until they are resolved. Please rebase or merge the base branch, @LIHUA919.

Choose the remote for the base repository, not an out-of-date fork.
For a fork clone, first inspect git remote -v; upstream must point
to https://github.com/loopx-project/loopx.git. If it is absent, add it
with git remote add upstream https://github.com/loopx-project/loopx.git.
Then run:

git fetch upstream
git rebase upstream/main
# Resolve each conflict, git add the resolved files, then git rebase --continue.
git push --force-with-lease origin HEAD

For a same-repository clone whose origin points to
https://github.com/loopx-project/loopx.git, use origin instead of
upstream for fetch/rebase. If you prefer merging the base, use
git merge <base-remote>/main and push normally.

Keep the DCO Signed-off-by trailer on every commit when you rebase.
https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[mergify]

This pull request has merge conflicts with main and cannot be merged
until they are resolved. Please rebase or merge the base branch, @LIHUA919.

Choose the remote for the base repository, not an out-of-date fork.
For a fork clone, first inspect git remote -v; upstream must point
to https://github.com/loopx-project/loopx.git. If it is absent, add it
with git remote add upstream https://github.com/loopx-project/loopx.git.
Then run:

git fetch upstream
git rebase upstream/main
# Resolve each conflict, git add the resolved files, then git rebase --continue.
git push --force-with-lease origin HEAD

For a same-repository clone whose origin points to
https://github.com/loopx-project/loopx.git, use origin instead of
upstream for fetch/rebase. If you prefer merging the base, use
git merge <base-remote>/main and push normally.

Keep the DCO Signed-off-by trailer on every commit when you rebase.
https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[Duang777]

Commit 106b923a57c60d411dba9276c37d1f927e0023bc also fixes three failures currently reproducible on main@996bcc027 and blocking #5340. I applied only that signed commit to an isolated current-main worktree; the affected tests passed (3 passed in 6.74s).

Could you split this test-only commit into a small standalone PR? That would let the baseline repair land without waiting for this draft performance change and its main synchronization.

[Duang777]

I extracted the test-only repair into #5344 to unblock the current main baseline while preserving Lihua as the commit author. No performance changes from this draft were copied.

[mergify]

This pull request has merge conflicts with main and cannot be merged
until they are resolved. Please rebase or merge the base branch, @LIHUA919.

Choose the remote for the base repository, not an out-of-date fork.
For a fork clone, first inspect git remote -v; upstream must point
to https://github.com/loopx-project/loopx.git. If it is absent, add it
with git remote add upstream https://github.com/loopx-project/loopx.git.
Then run:

git fetch upstream
git rebase upstream/main
# Resolve each conflict, git add the resolved files, then git rebase --continue.
git push --force-with-lease origin HEAD

For a same-repository clone whose origin points to
https://github.com/loopx-project/loopx.git, use origin instead of
upstream for fetch/rebase. If you prefer merging the base, use
git merge <base-remote>/main and push normally.

Keep the DCO Signed-off-by trailer on every commit when you rebase.
https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[mergify]

This pull request has merge conflicts with main and cannot be merged
until they are resolved. Please rebase or merge the base branch, @LIHUA919.

Choose the remote for the base repository, not an out-of-date fork.
For a fork clone, first inspect git remote -v; upstream must point
to https://github.com/loopx-project/loopx.git. If it is absent, add it
with git remote add upstream https://github.com/loopx-project/loopx.git.
Then run:

git fetch upstream
git rebase upstream/main
# Resolve each conflict, git add the resolved files, then git rebase --continue.
git push --force-with-lease origin HEAD

For a same-repository clone whose origin points to
https://github.com/loopx-project/loopx.git, use origin instead of
upstream for fetch/rebase. If you prefer merging the base, use
git merge <base-remote>/main and push normally.

Keep the DCO Signed-off-by trailer on every commit when you rebase.
https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[mergify]

This pull request has merge conflicts with main and cannot be merged
until they are resolved. Please rebase or merge the base branch, @LIHUA919.

Choose the remote for the base repository, not an out-of-date fork.
For a fork clone, first inspect git remote -v; upstream must point
to https://github.com/loopx-project/loopx.git. If it is absent, add it
with git remote add upstream https://github.com/loopx-project/loopx.git.
Then run:

git fetch upstream
git rebase upstream/main
# Resolve each conflict, git add the resolved files, then git rebase --continue.
git push --force-with-lease origin HEAD

For a same-repository clone whose origin points to
https://github.com/loopx-project/loopx.git, use origin instead of
upstream for fetch/rebase. If you prefer merging the base, use
git merge <base-remote>/main and push normally.

Keep the DCO Signed-off-by trailer on every commit when you rebase.
https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[huangruiteng]

Validation and merge follow-up for exact head e95f560117dea9793527c6691f857bc4ac26ca3a (main baseline 88132051b0d302a4b752ba3be59302a9d799b642).

Resolved the merge conflict by retaining the current main settlement tests: historical settlement replay must not mint a new scheduler operation after an intervening Turn. The production change remains confined to the strict JSON codec, replay snapshots, journal page copies and SQLite historical reconstruction; no public schema, provider default, persisted encoding or authority policy changes.

Passed on the integrated code:

• 741 real File/SQLite, replay/scan, process-CAS, archive and migration tests;
• 319 real PostgreSQL 16.15 tests in an isolated disposable server;
• 54 focused Python tests and 2 production CLI settlement tests;
• TypeScript typecheck, configured mypy, Ruff, semantic advisory/drift, public boundary and diff checks;
• risk-based premerge (10 catalog + 8 risk-profile checks), with no skips, failures or manual holds in the selected checks;
• strict exact-diff change-quality receipt cqr_dd942fd2ea52c5e2a508 verified valid.

Disclosed failure: test_browser_operation_fixture_needs_no_test_framework expects four fixture stages while the current fixture has six. It fails identically on pinned main; #5418 already owns the repair. Fresh remote CI remains pending. Older browser/build failures are not classified as resolved solely by that diagnosis.

The historical matched scan results remain a tradeoff: ~37% improvement for changing 1 MiB projections, a 2–8 ms increase for small projections, and ~39 ms increase for the many-field projection. The four measured production files are unchanged by this integration. These timings are not a fresh long-running measurement and do not close SQLite admission/default selection.

The bounded refactor pass reuses the existing strict codec and removes redundant encode/parse work without adding a storage format or parallel validator. Complete metadata, original receipts, reopened history, exact retries and conflicting intents remain covered across providers. No frontend/Lark companion is needed for an internal materialization change with unchanged consumer values.

Independent review of this exact head is being requested. This record is validation evidence, not an approval verdict or authorization to bypass remaining review/CI gates.

[huangruiteng]

APPROVE — e95f560117dea9793527c6691f857bc4ac26ca3a。未发现阻塞性问题；批准的是有界 JSON 复制/扫描改进，不是长期规模、安装推广或默认 provider 的资格认证。

动机

历史扫描需要为调用者隔离可变容器，但反复序列化、解析和 structuredClone 大字符串会增加成本。这个 PR 的有效结果是降低部分真实扫描路径的复制成本，同时完整保留历史状态、事件、原始 receipt、分页和重试含义。 owning acceptance 是 shared-goal-authority-state-provider RFC 的 local persistence 边界；一次合并不能替代其持续运行和迁移验收。此次评审覆盖全部 15 文件，包括四个生产文件、八个测试文件及三份 RFC/ledger 文档。

改动思路

在已有 authority_store_codec.ts 复用严格 JSON 遍历，而非添加独立优化 framework 或第二个 Python 决策 owner。canonical 路径仍按 Unicode code point 排序并生成相同持久化 bytes；copy 路径只隔离容器、保留枚举顺序，字符串等不可变值不额外复制。SQLite 读取历史 projection 使用 replay snapshot，摘要、链验证、lookahead 和 CAS 没有被绕过。RFC 文档明确区分有界合并、可回退 opt-in cohort 和 release default，并保留旧报告的失败、缺测和 frozen workload 身份。

具体改动

关键代码讲解

authority_store_codec.ts:49–56 暴露两个用途不同、验证 owner 相同的入口：

export function canonicalAuthorityJson(value: unknown, stack = new Set<object>()): unknown {
  return cloneAuthorityJson(value, stack, true);
}
export function copyAuthorityJson(value: unknown): unknown {
  return cloneAuthorityJson(value, new Set<object>(), false);
}

canonical 用于持久化证明，copy 用于调用者拥有的快照，不能互换。私有 boolean 是遍历模式，不是新增共享状态分类。对象先写入 null-prototype 容器再恢复普通 prototype，避免 __proto__、constructor 和 inherited setter 被当成行为；数组逐 own key 定义数据，保留 holes。有限数值、plain object 和 cycle 检查沿用同一 owner。AuthorityStateReplay.snapshot():219 返回隔离副本，后续 apply 不能污染旧 snapshot。AuthorityJournalScan.page():37 仍先验证连续 cursor、重复 operation 和 head lineage，再复制完整交易行，不丢 events/receipts/metadata。SQLite scanCommitted():578 在同一事务内证明 retained range，再把每个 replay state 给调用者；File、NoKV、PostgreSQL 的共享 page 路径也必须验证，不能把这当成只影响 SQLite 的 helper。

测试伴随改动没有改变生产超时语义：outbox fixture 只在 RPC response ambiguous 时，取得既有 maintenance lock 后读取同一 operation 和 request digest的完成 receipt；missing/pending/other operation/digest/manifest 损坏均拒绝，semantic rejection 不被吞掉，且不重发不确定写。quota fixture 去除 goal-wide gate 的不合法 agent claim，原 gate 范围断言仍保留；coverage 环境清理仅用于临时复制的语义测试子进程。

独立验证从公开 commitAuthority → reopen/loadAuthority → readReceipt → scanCommitted 入口走真实 File/SQLite：70 次提交跨越 checkpoint，核对特殊键、Unicode、事件和 receipt；历史同 intent 重试返回原 receipt，stale CAS 拒绝；修改返回快照后，其他行、再次读取和持久化 head 不变。相同 harness 在 base 88132051… 和当前 head 执行，三个 SQLite profile 的 fixture hash 及完整观察 hash 均一致。再故意把 replay snapshot 变成陈旧缓存，真实 SQLite 扫描的独立 oracle 在 cursor 64 抓到返回 ordinal 63；未变异 head 通过。这不是只验证新 helper 自己的输出。

对主干的风险

最强风险是别名污染或复制快了却少带历史字段；另一风险是把短期提速当成完整 provider 资格。前者有真实存储、重开、原 receipt、corrupt lineage/forged digest、并发及分页覆盖。后者必须保持边界：matched 70-commit/8 warm-sample SQLite service-time p50，小对象约 4.38→3.63 ms、多字段约 51.78→40.38 ms、变化的 1 MiB projection 约 214.49→73.00 ms；这是一次有界本地测量，不是 p95 长期 certificate，也未测整机持续资源增长。File 顺序复跑约 1.43→0.97 ms，但 tail 会波动，不能宣传全路径恒定加速。作者较大 workload 的小对象/多字段回退仍保留，不能用本次结果抹掉。没有改变同步持久化设置、用户 provider 选择或正式 qualification thresholds 的历史结果。

本地当前 head 通过：350 项 codec/replay/SQLite 聚焦测试、371 项 File/共享合同/archive 测试、54 项相关 Python 测试、control-plane TypeScript typecheck、changed-path Ruff、语义 advisory 和 full-tree vocabulary smoke。另在隔离真实 PostgreSQL 16 跑 315 项 integration，并在 immutable base 跑相同 suite 315 项；四项跨 File/SQLite/PostgreSQL archive 读回也通过，无跳过。隔离数据库已停止，未使用或修改活动 Goal。初版独立 probe 误把 provider 的原 receipt 期待为 coordination replayed，及误调用 File 不存在的 SQLite audit 方法，查明公开合同后纠正 oracle并重跑；没有修改产品来迁就验证。未查询、轮询或等待 GitHub CI；未重新跑全树全部 suite，也不声称 release-default 或十天验收通过。

生产范围只有 +38/-7；没有新状态、schema、CLI、权限、fallback authority 或安装激活。调用者仍走已有 AuthorityStore 和 typed coordination owner，Python 仅有测试调整。Frontend/Lark 不需新交互或设置，因为扫描的 public result/persistence contract 和 provider activation 都未改变。可回滚到原复制实现，不需数据迁移；持久化编码与旧 receipt 解码不能随优化删除。

我的整体评价

这是一份同一 change reason 下的有界优化和行为保持重构，风险覆盖与实现成本相称，可以批准。未来向重构 pass 已应用于共享 JSON traversal；额外把各存储封装成统一复杂 framework 无实际需要。最强反对理由是局部收益不能覆盖所有 workload、尚无长期资源证明，因此批准范围只到当前可逆扫描改进，原 RFC qualification 继续由既有 owner 负责。后续默认选择和更广推广不能复用这条 APPROVE 作为完成证据。没有合并，也没有修改已安装行为。

English verdict: APPROVE — HEAD e95f560. No blocking finding in this bounded copy/scan change. Exact-head real File/SQLite readback and real PostgreSQL integration passed, with matched base/head and mutation-sensitive evidence. This does not qualify sustained scale, universal speedup, or a release-default provider.