feat(authority): add transaction-bound runtime shadow capture - #3870
Conversation
huangruiteng
left a comment
huangruiteng
left a comment
There was a problem hiding this comment.
动机
这个 PR 承接 #3818 的 one-way local authority shadow，但修正上一阶段无法把候选侧 receipt 精确绑定到某一次源写事务的问题。旧 observation 是 post-commit snapshot：它能证明“某个时刻观察到了这份状态”，却不能证明 receipt 对应哪一个 primary transaction，也可能包含并发写者的结果。当前增量引入按 todos / leases primary-lock partition 划分的 durable outbox，让 writer 在原有锁内先写 prepared、primary write 成功后写 committed marker，再由有界 drain 以 entry id 作为 candidate store operation_id 提交。设计目标合理：legacy local 仍是唯一 primary authority，candidate 不参与决策，parity 明确保持 not_evaluated；本 PR 只交付 plumbing、operator surface 与 crash-window 语义，不提前宣称 writer wiring 或 parity 完成。
改动思路
数据流分为三段。第一段是纯 projection/canonicalization：Python 与 TypeScript 共享字段集合、排序和摘要规则，并拒绝 float，避免跨运行时制造 digest 分歧。第二段是 source-side outbox：每个 Goal、每个 partition 使用 gap-free seq，prepared 文件绑定计划写入的 source bytes/event，committed marker 表明 primary write 已返回；prepared-only 只有在 primary lock 可确认空闲时才根据 on-disk source 解析为 committed_proven_by_readback、abandoned 或 unproved，后者会在同一锁内 reseed。第三段是 candidate drain：per-goal drain lock 串行化处理，按 partition/seq 把每个 entry 转成一个 FileAuthorityStore transaction，CAS 冲突重试，lost response 通过 receipt 对账，cursor 持久化后删除 outbox entry，并做 candidate head readback。
正向路径为 primary lock 内 prepared → primary write → committed → drain → operation_id=entry_id 的 candidate commit → cursor → 删除 entry → readback。负向路径覆盖 primary write 未落盘、marker 丢失、provider ambiguity、revision conflict、held primary/drain lock、batch/time budget 与 malformed result。功能是 opt-in plumbing：PR 自述也确认尚未接入 writers；现有 writer 继续走 #3818 observation，candidate 永远不回写 local，也不提供调度/lease/Todo 决策权。
具体改动
local_authority_shadow_projection.py：定义 Todo/lease compact fields、canonical JSON、partition/head digest；拒绝 float 与非法 lease identity。local_authority_shadow_outbox.py：实现 durable JSON、entry identity、prepared/committed 文件、cursor、seed/reseed、prepared-only source resolution、Todo capture 与 backlog status。local_authority_shadow_outbox.ts：实现 TypeScript lease writer 侧的 binding decode、lease partition snapshot、跨语言一致的 entry identity，以及两阶段 capture；effect_runtime_io.ts增加带 file/directory fsync 的 durable writer。local_authority_shadow.ts：新增 closed-shape commit/read 协议；composeLocalAuthorityShadowHead()按 partition 折叠 v1 head；commitLocalAuthorityShadowEntry()通过 CAS 与 receipt reconciliation 实现幂等提交；read API 返回 bounded transaction scan 和 comparison digest。handlers/tsconfig 完成注册。local_authority_shadow_adapter.py：贯通 source projection、candidate effect boundary、有界 drain、primary/drain lock、cursor、readback、status 和 evidence v1；goal_todo_projection.py抽出可在 writer 锁内对给定文本运行的 projection。- CLI、help、mypy/Windows CI 列表与 Python/TypeScript/E2E 测试补齐 operator 和维护入口。
关键代码讲解
TodoPartitionCapture.prepare()/committed()与beginLeaseOutboxEntry()定义 source transaction 的两阶段边界：prepared 绑定计划 bytes，marker 只在 primary write 返回后产生，且失败不阻塞 primary authority。_PartitionDrainer负责 sequence processing、prepared-only 恢复、candidate commit、cursor write、entry retirement 与 readback，是 crash safety 的核心状态机。commitLocalAuthorityShadowEntry()用 entry id 作为 operation id；applied、ambiguous、operation-id conflict 与 revision conflict 被区分，只有 receipt 的 partition/seq/digest 与请求一致才接受 replay。composeLocalAuthorityShadowHead()只更新一个 partition；abandoned/unproved事务保持审计序列但不改 comparison fields。capture_evidence()生成 writer-facing typed facts，继续把source_candidate_compared=false、parity_verdict=not_evaluated和primary_authority=legacy_local固定在未验证状态。
对主干的风险
当前 exact head 存在两个需要修复的 blocker。
-
cursor 已提交后的 entry 清理仍有不可恢复 crash window。
_PartitionDrainer._commit()正确地先写 drain cursor 再调用remove_entry_files()，但后者依次删除 prepared、再删除 committed。若进程在两个 unlink 之间崩溃，磁盘上会留下 committed-only marker。重启后list_entries()在读取 cursor 之前就把任何 orphan marker 判为outbox_file_invalid，因此 drain/status 永久停止，虽然 cursor 已经证明该 seq 已结算。我在 exact head 上构造cursor.last_seq=1、删除 seq 1 prepared、保留 committed，稳定得到outbox_file_invalid committed marker without prepared entry。最小修复应让扫描/清理依据 durable cursor 容忍并回收seq <= cursor.last_seq的退休残留，并新增在两次删除之间注入崩溃后可恢复的测试；单纯交换删除顺序只会把故障形态变成 prepared-only，并可能触发错误的 source resolution/reseed。 -
durable_source_outbox在没有 outbox transaction 时错误为 true。capture_evidence()当前使用capture.failure is None，所以shadow_disabled或partition_unchanged的CaptureOutcome(entry_id=None, failure=None)会输出outcome=no_transaction、source_transaction_correlated=false，却同时输出durable_source_outbox=true。我直接调用 exact-head 函数已复现两种情况。该字段作为“measured fact”应至少与实际记录成功绑定（例如capture.recorded），并给 disabled/unchanged 增加明确断言，否则下游会把零写路径误认为已有 durable outbox 证据。
此外，outbox 文件边界目前只校验 filename/schema/seq/entry_id，没有重新验证 prepared record 的 goal_id、partition、source_root_digest 或根据记录字段重算 entry_id。我构造位于 goal-a/todos 目录、内容却声明 goal-other/leases 且 identity 不匹配的 prepared 文件，list_entries() 仍接受它。虽然这需要本地文件损坏/篡改，不是本 PR 的首要 blocker，但既然 commit receipt 声称 source transaction correlation，建议 closed-shape 校验这些绑定，至少在进入 candidate commit 前 fail closed。
独立验证：Python shadow/outbox/drain/CLI/runtime 聚焦测试 33 passed；TypeScript 新 suite 8 passed；configured mypy（两个新 pure modules）、typecheck、Ruff 与 diff-check 通过。全 control-plane TypeScript 运行中相关测试均通过，但本地缺少 PostgreSQL integration 环境时该单项无法加载 pg；GitHub required checks 全绿。上述两个 blocker 都不被现有测试捕获。
我的整体评价
方向与分层是成熟的：source lock 内 capture、candidate effect boundary、receipt reconciliation、typed evidence 和 operator surface 的职责清楚，而且明确不偷渡 parity/authority claim。但这是约 4.5k 行、直接处理 durable transaction 与 crash recovery 的高风险基础设施，完成门槛必须以实际 crash-window closure 和证据真实性为准。当前一个删除中断即可把 outbox 置为永久 invalid，且 no-transaction 路径会夸大 durable evidence，因此整 PR 结论为 REQUEST_CHANGES。修复上述两点并补回归后，我会在新 exact head 重新检查；prepared-record identity closed-shape 也建议在同轮加固。
English verdict: REQUEST_CHANGES at exact head 50afe7cba7d06718e7a6a630692fffc98c129ae8. A crash after cursor persistence but between deleting the prepared and committed files leaves an orphan marker that permanently fails list_entries(), and no-transaction outcomes incorrectly report durable_source_outbox=true. Focused Python (33) and TypeScript (8) tests, typecheck, mypy, Ruff, diff-check, and GitHub checks pass, but they do not cover these reproduced failures.
|
The crash-window design here is well thought through. The cursor-before-delete ordering in Three points (fine to defer any of them to a later increment):
Minor: Python Looking forward to the writer-wiring increment on top of this — the plumbing looks ready to carry it. |
|
@huangruiteng Round 2 on exact head
Validation on this head: focused suites 55 passed; full pytest 5186 passed, 12 skipped, the same 12 environment-only failures as |
huangruiteng
left a comment
huangruiteng
left a comment
There was a problem hiding this comment.
动机
这个 PR 承接已经合入的 #3818，把“事后观察到一份 local authority snapshot”推进为“候选侧 receipt 能绑定到某一次源写事务”的 Stage 2C 基础设施。原有 observation 只能证明某个时刻看到的状态，无法证明 receipt 对应哪个 Todo/lease 写入，也可能把并发写者的结果混入同一快照。当前方案按 todos / leases 两个 primary-lock partition 建 durable outbox：源写入先产生 prepared，primary write 返回后再产生 committed marker；有界 drain 以 entry id 作为 candidate transaction operation_id，同时继续明确 legacy local 是唯一 primary authority，source_candidate_compared=false、parity_verdict=not_evaluated。这个问题和边界都合理，而且本轮确实修复了上次评审指出的 cursor 已落盘后清理中断、no-transaction 证据夸大，以及 prepared record identity 未闭合三个问题。
改动思路
整体链路分为四层。第一层是 Python projection/canonicalization，统一 Todo/lease 的 compact shape、排序与 digest，并拒绝 float，避免 Python/TypeScript 跨运行时摘要漂移。第二层是 source outbox：TodoPartitionCapture 与 beginLeaseOutboxEntry() 建两阶段 prepared/committed 文件，entry identity 绑定 Goal、partition、seq 与 source reference。第三层是 candidate drain：_PartitionDrainer 在 per-goal drain lock 下处理 prepared-only recovery、CAS/replay reconciliation、cursor-before-delete、candidate readback 和 bounded batch。第四层是 operator/effect surface：TypeScript 注册 commit_entry/read handler，Python CLI 提供 authority-shadow drain|status，同时把新模块加入 mypy、tsconfig 和 CI 测试清单。
正向路径是：源锁内 prepared → primary write → committed → drain → candidate commit → cursor → 删除 entry → candidate readback。负向路径包括 prepared-only source readback、revision conflict、ambiguous receipt、held lock、source-root mismatch、invalid entry 和 cursor 已覆盖但文件尚未删完的 residue。新 head 的 residue 修复方向正确：seq <= cursor.last_seq 的文件先按退休残留回收，不会重新提交，也不会复用 seq；cursor 以上的 orphan marker 仍然 fail closed。
具体改动
local_authority_shadow_projection.py定义 canonical value、Todo/lease partition projection 与 head/partition digest。local_authority_shadow_outbox.py实现 durable JSON、entry identity、prepared/committed lifecycle、cursor、seed/reseed、source resolution 与 backlog summary；本轮新增record_source_ref()、prepared closed-shape 校验、retired_residue()/reclaim_retired_residue()和跨运行时一致的 root digest。local_authority_shadow_outbox.ts提供 lease entry 的两阶段写入及跨语言 identity；effect_runtime_io.ts新增 temp-file + fsync + rename + directory fsync 的 durable writer。local_authority_shadow.ts实现 v1 partition fold、operation-id replay reconciliation、no-op resolution 和 bounded read；effect_runtime_handlers.ts注册 commit/read handler。local_authority_shadow_adapter.py负责 registry/binding 解码、primary/drain lock、prepared-only recovery、candidate effect 调用、cursor/entry retirement、status、readback 与 evidence v1。authority_shadow.py、cli.py、lazy command registry 和 help surface 暴露drain|status；goal_todo_projection.py提供在 writer 锁内对给定 state text 做 projection 的纯入口。- 三组 Python 测试、TypeScript parity/outbox 测试，以及 mypy/tsconfig/Windows CI 配置覆盖新增表面。本轮新增的 crash-between-unlinks、cursor-before-any-unlink、foreign-root、tampered prepared record 和 disabled/unchanged evidence 测试能够证明前次 blocker 已关闭。
关键代码讲解
TodoPartitionCapture.prepare()/committed()与beginLeaseOutboxEntry()定义 source transaction 的两阶段边界；失败被转成 typed capture evidence，不能反向阻塞 primary authority。_PartitionDrainer.run()先回收 cursor 已覆盖的 residue，再按 seq 处理 live entries；_commit()在 candidate commit 前验证 runtime-root lineage，并在 cursor 持久化后才删除 entry。commitLocalAuthorityShadowEntry()使用 entry id 作为 operation id，将 applied、replay、ambiguous、operation-id conflict 与 revision conflict 分开；只有 receipt 的 partition/seq/digest 与请求一致才接受 replay。_load_prepared_record()现在同时校验目录 Goal/partition、writer runtime、source kind、root digest 形状和重算后的 entry id，关闭了上次指出的 prepared-record identity 缺口。outbox_summary()是 CLIstatus、drain prelude 和 drain settlement 的共同 backlog 入口；它必须把 malformed cursor/entry 转成 typed invalid 状态，不能在构造 summary 时再次抛出。
对主干的风险
当前 exact head 仍有一个新的阻塞问题：outbox_summary() 捕获 list_entries() / read_cursor() 的 OutboxError 后，会在 try 外无条件调用 retired_residue(directory)；而后者再次调用 read_cursor()。因此，只要 drain-cursor.json 存在 unsupported schema 或其他 cursor 读取错误，原本已经被折叠为 invalid="outbox_file_invalid" 的异常会再次逸出。这个入口同时被 local_authority_shadow_status()、drain 的 _outbox_is_idle(...) 预检和 _count_backlog() 使用，所以 operator status 不能返回 typed backlog，drain 甚至会在自身 try/except OutboxError 之前失败。我用一个只有 schema_version="bad" 的 cursor 直接调用 exact-head outbox_summary()，稳定得到未捕获的 OutboxError(outbox_file_invalid, "drain cursor schema is unsupported")；现有测试没有 malformed-cursor coverage。
最小修复是让 residue 计数复用同一次受保护的 cursor/index 读取，或把 retired_residue() 也放进同一个 error boundary，并确保 invalid cursor 时返回 retired_residue=0、保留 typed invalid，而不是二次读取并抛出。需要补一条 malformed/unsupported cursor 回归，分别断言 status 返回 typed invalid，drain 返回 stopped/outbox_file_invalid，且不会调用 candidate store 或删除任何 entry。
范围方面还保留一个需要 owner 明确接受的风险：约 4.9k 行生产 plumbing 中，TodoPartitionCapture 和 beginLeaseOutboxEntry() 在当前 head 没有生产 writer caller；当前可运行表面主要是 status/drain 和 candidate handlers。PR 已主动声明 writer wiring 在下一增量，因此这里不能把“测试可构造 entry”当成已交付 source transaction 行为。若继续保持分阶段 PR，请在合入前给出明确的 coverage-only/staged-boundary 接受记录；否则应把至少一个真实 writer capture 路径放进同一可验证增量。
独立验证：新增 Python outbox/drain/CLI suites 29 passed；TypeScript outbox suite 8 passed；control-plane typecheck 通过；两个新 strict mypy 模块通过；git diff --check 通过；远端 SonarCloud 成功。未独立复跑作者报告的 full pytest 5186 / full TS 471 / full-public canary。新 blocker 是我在 exact head 上额外构造的负向路径，不在现有绿色检查覆盖内。
我的整体评价
架构分层、primary/candidate authority 边界、receipt reconciliation 与本轮对旧 findings 的修复都做得扎实；尤其 cursor-covered residue 的恢复顺序和 no-transaction 证据真实性已经达到可审计要求。但这是 crash-recovery/operator 路径，malformed cursor 恰好属于它必须稳定 fail closed 的核心场景。当前 summary 的二次读取会让 status/drain 绕过 typed outcome，因此结论仍为 REQUEST_CHANGES。修复这一处并补 status/drain 级负向测试后，可以在新 exact head 快速复核；同时请明确记录 staged coverage-only 边界或补真实 writer caller。
English verdict: REQUEST_CHANGES at exact head 87d7165850c6d9019edb5493e29111c7e9b5c4c6. The previous residue, evidence-truthfulness, and prepared-record binding findings are fixed, but outbox_summary() catches an invalid cursor and then re-reads it through retired_residue() outside the error boundary, so status and drain preflight raise instead of returning a typed outbox_file_invalid outcome. Focused Python (29) and TypeScript (8) suites, typecheck, strict mypy, diff-check, and the remote SonarCloud check pass; malformed-cursor coverage is missing.
|
@huangruiteng Rebased onto
This PR's outbox closes both windows for whichever lineage is Stage 2C's: the prepared entry is written inside the lock the writer already holds, the committed marker after the primary write returns, and the bounded drain turns each entry into exactly one shadow transaction with
The docs PR #3909 records this as question 14 with option 1 as the proposed answer. The fixes you asked for on |
87d7165 to
a16f746
Compare
huangruiteng
left a comment
huangruiteng
left a comment
There was a problem hiding this comment.
动机
这个 PR 继续 Stage 2C 的 local authority shadow 工作，目标是把 #3818 的 post-commit snapshot 提升为与一次 primary 写事务绑定的候选侧证据。旧路径只能说明“某个时刻观察到了这份状态”，无法证明 receipt 对应哪一次 Todo/lease 写入，并可能把并发写者的结果采入同一 snapshot。当前设计按 todos / leases 两个 primary-lock partition 建 durable outbox：锁内先写 prepared，primary write 返回后写 committed marker，再由有界 drain 以 entry id 作为 candidate transaction 的 operation_id 提交。边界声明是清楚的：legacy local 仍是唯一 primary authority，candidate 不参与决策，source_candidate_compared=false、parity_verdict=not_evaluated；writer wiring、verify/reset 与 retention 修复都不在本 PR。
改动思路
链路分为四层。Python projection 层固定 Todo/lease compact fields、Unicode/key 排序、canonical JSON 与 digest，并拒绝 float，避免跨语言摘要漂移。source outbox 层由 TodoPartitionCapture 和 beginLeaseOutboxEntry() 生成 prepared/committed 文件，entry identity 绑定 Goal、partition、seq 和 source reference。candidate drain 层由 _PartitionDrainer 在 per-goal drain lock 下顺序处理 committed 与 prepared-only entry，做 source readback、CAS/replay reconciliation、cursor-before-delete、residue 回收和 candidate head readback。operator/effect 层注册 commit_entry/read handler，并增加 loopx authority-shadow drain|status。
正向路径是 primary lock 内 prepared → primary write → committed → candidate commit → cursor → 删除 entry → readback。负向路径包括 prepared-only 的 committed/abandoned/unproved 分类、held primary/drain lock、revision conflict、ambiguous receipt、foreign root、cursor 已覆盖但 entry 未删净、malformed entry/cursor。当前 head 对上一轮 residue、no-transaction evidence 和 prepared identity finding 的修复方向正确：cursor watermark 以下只回收不重投，capture.recorded 才能使 durable/correlated 为真，prepared record 会重算 entry identity 并校验目录和 root lineage。
具体改动
local_authority_shadow_projection.py定义 canonicalization、Todo/lease partition projection 与 head/partition digest；goal_todo_projection.py增加基于调用方 state text 的纯 projection 入口。local_authority_shadow_outbox.py实现 durable JSON、seq/entry identity、prepared/committed lifecycle、cursor、seed/reseed、prepared-only source resolution、retired residue 与 backlog summary；local_authority_shadow_outbox.ts提供 lease 侧等价 capture 和跨语言 identity/root digest。local_authority_shadow.ts增加 v1 partition fold、transaction receipt/event、operation-id replay/ambiguous reconciliation、bounded read；effect_runtime_handlers.ts与effect_runtime_io.ts分别注册 handler、增加 fsync + rename durable write。local_authority_shadow_adapter.py贯通 registry/binding、source projection、primary/drain lock、bounded drain、candidate effect 调用、cursor/retirement、status/readback 与 evidence v1。authority_shadow.py、cli.py、lazy exports 和 help surface 暴露drain|status；pyproject、tsconfig 和 Windows workflow 纳入严格类型检查及测试。三组 Python tests 与 TypeScript suite 覆盖 normal delivery、replay、prepared-only recovery、residue、identity/root mismatch 和 CLI readback。
关键代码讲解
TodoPartitionCapture.prepare()/committed()与beginLeaseOutboxEntry()定义 source transaction 两阶段边界；捕获失败被降为 typed evidence，不能改变 primary write verdict。_PartitionDrainer.run()先按 cursor 回收 settled residue，再按 seq 处理 live entry；_commit()校验 root lineage、调用 candidate、持久化 cursor 后才删除 entry。commitLocalAuthorityShadowEntry()以 entry id 作为 operation id，将 applied、replayed、ambiguous、operation-id conflict 与 revision conflict 分开；receipt 必须匹配 partition/seq/digest 才能证明 replay。composeLocalAuthorityShadowHead()只折叠一个 partition；abandoned/unproved保留审计 transaction，但不修改 compared fields。outbox_summary()是status、drain preflight 和 drain settlement 的共同入口，因此它必须把所有持久化文件错误稳定投影为 typed invalid，而不能在生成 summary 时再次抛异常。
对主干的风险
当前 exact head 仍有两个 blocker。
-
malformed cursor/JSON 仍绕过 typed failure contract。
outbox_summary()虽捕获了list_entries()/read_cursor()的OutboxError，但在try外又无条件调用retired_residue(directory)，后者重新读取同一 cursor。放入schema_version="bad"的 cursor 后，summary、local_authority_shadow_status()和 drain preflight 都直接抛OutboxError(outbox_file_invalid)，而不是 status 返回invalid、drain 返回stopped/outbox_file_invalid。进一步地，_load_json()不包装JSONDecodeError/I/O 错误，合法 schema 但last_seq="oops"又会从int(...)抛 rawValueError。我在本 exact head 上分别复现了这三种逸出；候选 store 调用数和 entry 删除数均为 0，cursor 保留。最小修复应让 cursor/index 只在一个受保护边界内解析一次，closed-shape 校验 cursor 字段与类型，把 JSON/decode/I/O/type 错误统一映射为outbox_file_invalid，并新增 summary/status/drain 级测试，断言无 candidate call、无删除。 -
scope fit 与 proportionality 仍未被当前可运行产品路径证明。 exact-head diff 是 19 files、
+4907/-4，其中 production/config+3427/-4、tests+1480；但仓库内除 tests 外没有任何 caller 调用TodoPartitionCapture或beginLeaseOutboxEntry()。当前实际可达表面只有 operatorstatus|drain和 candidate handlers，生产 writer 不会生成 entry，所以这 3.4k 行 durable state/CLI/protocol 尚未交付它声称解决的 source-transaction 行为。与此同时 main 已有coordination.runtime_shadow的另一条 candidate lineage，作者也在本 PR 评论中明确提出两种 retarget/保留方案并等待 owner 选择；尚无 owner 接受 staged coverage-only boundary 或确定 lineage 的记录。根据当前 review contract，测试能够手工构造 entry 不能替代 active call site。最小修复是先由 owner 明确选择唯一 lineage，并把至少一个真实 writer capture 路径纳入同一个可验证增量；若确需 staged merge，则需要 owner 对 coverage-only 边界和后续删除/迁移成本作显式接受。
独立验证：新增 Python outbox/drain/CLI suites 29 passed；TypeScript outbox suite 8 passed，全 control-plane 500 passed / 1 PostgreSQL integration skipped；control-plane typecheck、两个新 strict-mypy 模块、Ruff 与 git diff --check 通过。exact-diff standard premerge 通过：4/4 direct checks、18/18 catalog/risk/boundary checks、0 warnings/holds，public boundary clean。远端 pytest/build/Sonar/Windows/dependency-review 通过；Sign-off 在复核时仍 queued。未独立复跑作者报告的 full pytest。
我的整体评价
架构分层、legacy-local authority 语义、receipt reconciliation，以及本轮对旧 findings 的修复都很扎实；residue watermark 与 evidence truthfulness 已明显改善。但 crash-recovery/operator contract 仍会被最基本的 malformed cursor 绕过，且整个 outbox writer 半边在生产中没有 caller，同时与现有 runtime shadow 的最终 lineage 尚未决策。正确性和绿色测试不足以证明当前 4.9k 行增量已经具备可合入价值，因此 exact head a16f746bfeaa60d83468da27c5ae6b3c28c0eb13 的整体结论仍为 REQUEST_CHANGES。请先关闭 typed invalid 路径，并解决或显式接受 staged scope/lineage；新 head 到位后再做快速复核。
English verdict: REQUEST_CHANGES at exact head a16f746bfeaa60d83468da27c5ae6b3c28c0eb13. Malformed cursor/JSON state still escapes outbox_summary(), status, and drain preflight instead of producing typed outbox_file_invalid results; additionally, the 3.4k production/config-line outbox has no production writer caller and its lineage relative to coordination.runtime_shadow is still awaiting owner choice. Focused Python (29), focused TypeScript (8), full control-plane TypeScript (500/1 skipped), typecheck, strict mypy, Ruff, diff-check, exact-diff premerge (4 direct + 18 canary/boundary checks), and remote required checks otherwise pass.
|
@huangruiteng Round 3 is on exact head The blocker reproduced as described: with a On the malformed cursor I kept fail-closed on purpose and wrote the policy into the PR body: once residue has been unlinked the cursor is the only record of the retired watermark, so rebuilding it from surviving entry files can re-mint a seq below the real watermark; a rebuild that reconciles against candidate receipts belongs to the verify increment (@now-ing's point 2). Point 3 was closed in round 2 (both runtimes digest the abspath, no symlink resolution); point 4 is cosmetic since digests exclude timestamps, and I would normalize both sides to milliseconds in the wiring increment rather than touch record shapes here. On the staged boundary: I added the explicit coverage-only record to the PR body rather than wiring a writer in this round, because the drain target of that wiring depends on which shadow lineage Stage 2C keeps (question 14 in #3909, and my earlier comment above). If you would rather see one real writer capture path in this PR regardless, say so and I will add the Validation on |
|
@huangruiteng Your round-4 review was recorded against
On blocker 2 I agree with the standard (tests constructing entries are not an active call site) and that I cannot self-certify a staged boundary. I do not think it is fixable from my side in this PR, for a technical reason beyond the lineage question: the drain and the #3818 observation share one candidate store ( A. Keep the #3818 lineage: I deliver that todos-partition slice on top of this PR (writers capture inside their lock, inline bounded drain, observation retired for todos) as the next commit here, so this PR merges with an active call site. My recommendation is B if you intend the runtime shadow to be the Stage 2C lineage (question 14 in #3909), otherwise A. Validation on |
huangruiteng
left a comment
huangruiteng
left a comment
There was a problem hiding this comment.
详细中文评审
评审对象：exact head 90cb457ea1f66443b373380c85e8e897ca6997bb
动机
这个 PR 试图把 #3818 的 post-commit authority shadow 观察升级为与一次 primary 写事务绑定的 durable outbox。旧观察只能说明“某时读到了这份状态”，崩溃时可能漏采，并发 writer 也可能让 receipt 关联到错误快照；新设计按 todos / leases partition 在 primary lock 内写 prepared、primary write 成功后写 committed，再由 bounded drain 以 entry id 作为 candidate transaction 的 operation_id 提交。它明确保持 legacy local 为唯一 primary authority，candidate 仅用于证据，且尚不声称 parity。
本轮 head 已完整修复上一轮 malformed-cursor blocker：Python 和 TypeScript 都把 cursor 绑定到 drain writer 产出的 closed shape，JSON、字段类型、partition、entry id、digest 与 I/O 错误均 fail closed。作者同时明确承认 TodoPartitionCapture / beginLeaseOutboxEntry() 仍没有生产 writer caller，并给出 A/B/C 三种后续方向；这说明剩余问题已不再是局部正确性，而是该 5.2k 行增量是否已经选定产品 lineage 并交付可达行为。
改动思路
实现分为四层。projection 层固定 Todo/lease compact fields、canonical JSON 与 digest；source outbox 层实现 prepared/committed 文件、seq、entry identity、cursor、seed/reseed 与 prepared-only recovery；candidate drain 层在 per-goal lock 下按序提交、做 replay/CAS reconciliation、先写 cursor 再删 entry，并回收 cursor 以下 residue；operator/effect 层注册 commit_entry / read，并暴露 loopx authority-shadow drain|status。
正向路径为：writer lock 内 prepared → primary write → committed → bounded drain → candidate commit/readback → durable cursor → entry retirement。负向路径覆盖 held lock、prepared-only readback、revision/operation conflict、ambiguous receipt、foreign runtime root、orphan marker、cursor-covered residue 与 malformed cursor。当前 cursor 修复在这两条运行时上是一致的：Python read_cursor() 验证完整 shape，TypeScript nextSeq() 不再忽略非法 last_seq 后从残留文件继续 mint sequence。
但 source transaction 的正向路径仍只在测试中被手工构造。仓库 production 搜索没有发现 TodoPartitionCapture 或 beginLeaseOutboxEntry() 的 writer call site；可运行入口只有对“已存在 outbox”的 status|drain 和 candidate handler。与此同时 #3909 的 question 14 已把 runtime-shadow lineage 作为拟议收敛方向，而本 PR 仍写入 #3818 的 local_authority_shadow store。作者也明确说明若选择该方向必须 retarget drain 并一次性迁移 todos partition writers，不能安全地只接一个 writer。
具体改动
local_authority_shadow_projection.py定义跨运行时 canonicalization、Todo/lease partition projection 与 head digest；goal_todo_projection.py增加基于待写文本的纯 projection。local_authority_shadow_outbox.py实现 durable entry/cursor、prepared/committed lifecycle、source resolution、residue recovery、seed/reseed 和 operator summary；本轮新增 closed-shape cursor binding、单次受保护 summary 读取以及 JSON/I/O typed errors。local_authority_shadow_outbox.ts提供 lease capture 和 byte-compatible identity；本轮让nextSeq()对非法 cursor fail closed。local_authority_shadow_adapter.py负责 binding、primary/drain locks、bounded drain、candidate effect 调用、cursor/retirement、status/readback 与 evidence v1。local_authority_shadow.ts实现 v1 partition fold、transaction receipt/event、operation-id replay reconciliation 与 bounded read；effect runtime 注册commit_entry/read。authority_shadow.py、CLI registry/help surface 暴露drain|status；mypy、tsconfig 与 Windows workflow 纳入新模块和测试。- 三组 Python tests 与 TypeScript suite 覆盖 normal/replay、prepared-only、residue、identity/root mismatch、malformed cursor、CLI readback 和跨语言 identity。
关键代码讲解
TodoPartitionCapture.prepare()/committed()与beginLeaseOutboxEntry()定义 source transaction 的两阶段边界；禁用时不创建任何状态，capture 失败也只记录 typed evidence，不反向改变 primary verdict。问题是这两个入口目前没有 production writer caller。_PartitionDrainer.run()先回收 cursor watermark 已覆盖的 residue，再按 seq 处理 live entry；prepared-only resolution 只在 primary lock 可获得时执行，无法证明时追加 reseed。_PartitionDrainer._commit()校验 runtime-root lineage，以 entry id 调 candidate handler，只有 typed settled outcome 才先写 cursor、后删 entry；这一顺序把“store committed、cursor 未写”和“cursor 已写、文件未删”两个 crash window 分别折叠为 replay 与 residue recovery。commitLocalAuthorityShadowEntry()将 applied/replayed/ambiguous/operation conflict/revision conflict 分开，receipt 只有在 partition、seq、operation id 与 digest 全匹配时才证明 replay。read_cursor()/cursorHasClosedShape()是本轮修复核心：非法 schema、partition、last_seq、entry id、digest、时间字段或不可读文件均阻止 sequence minting；summary/status/drain 将其稳定投影为outbox_file_invalid。
对主干的风险
上一轮 correctness blocker 已关闭，当前仍有一个合入 blocker：scope fit、change proportionality 与唯一 lineage 尚未成立。 exact diff 为 19 files、+5247/-4，其中 production/config 约 +3569/-4；但 source outbox capture 没有 active writer call site，当前用户只能查看或 drain 测试/人工生成的 entry，因此还没有交付“把 primary write 变成 transaction-bound candidate evidence”这一原始 outcome。测试可达不能替代 shipped behavior。
该问题也不能用接入一个 writer 的小补丁规避。作者已经说明，当前 store 同时接受 #3818 v0 observation head 与本 PR v1 partition head；若只接一个 writer，其余 observation writer 会覆盖 partition watermark，形成无 parity 含义的 hybrid lineage。#3909 question 14 又拟议选择 coordination.runtime_shadow 作为 Stage 2C 唯一 lineage，而本 PR 的 candidate target 仍是另一套 coordination.local_authority_shadow。在 owner 尚未明确批准 staged coverage-only merge及其迁移/删除成本、且目标 lineage 未落到实现前，批准会先固化第二套 durable schema、CLI 和 recovery surface，再要求后续 retarget 或删除。
最小可接受修复是二选一：按 owner 最终选择把 drain/capture 收敛到唯一 runtime-shadow lineage，并一次性接入一个不会产生 hybrid head 的完整 partition writer slice；或由 owner 明确接受本 PR 作为 coverage-only staged merge，同时记录后续迁移/删除成本和不可形成双 lineage 的约束。作者自身的 PR 评论明确表示无法自证后一种接受，因此当前证据不足。
独立验证：Python outbox/drain/CLI focused suites 43 passed；TypeScript outbox suite 9 passed；control-plane tsc --noEmit、focused Ruff、strict mypy 与 git diff --check 均通过；远端 9 个 required checks 全绿。malformed/unsupported cursor、text/bool/zero sequence、partition/entry/digest shape 和 I/O failures 均有回归，上一轮 typed-failure finding 已关闭。未验证真实 writer capture、promotion parity 或 production drain outcome，因为 exact head 中不存在对应 caller。
我的整体评价
outbox 的 crash-window设计、closed-shape binding、primary/candidate authority 分离、receipt reconciliation 和跨运行时测试都很扎实，本轮也确实修完了所有已知局部 correctness 问题。但当前 PR 的大部分机制仍是未接线的生产 plumbing，而且其 candidate lineage 与正在形成的 runtime-shadow 方向尚未收敛；这正是 review contract 要求 scope-fit 与 proportionality 独立于“代码正确、CI 全绿”判断的场景。结论：REQUEST_CHANGES。请先取得明确的 staged-boundary owner decision，或把唯一 lineage 与完整 partition writer call sites 纳入同一可验证增量。
English verdict: REQUEST_CHANGES on exact head 90cb457ea1f66443b373380c85e8e897ca6997bb. The malformed-cursor blocker is fully fixed and focused validation is green (Python 43, TypeScript 9, control-plane tsc, Ruff, strict mypy, diff check; all remote checks pass), but the 3.5k-line production/config outbox still has no production writer capture call site and targets a local-shadow lineage that has not been reconciled with the proposed runtime-shadow Stage 2C lineage. Obtain explicit owner acceptance of a coverage-only staged merge, or converge on one lineage and wire a complete partition slice before approval.
New post-#3909 finding — REQUEST_CHANGES remains[P1] The v1 outbox projection now conflicts with the merged Stage 2C completeness contract. On exact head That shape is no longer compatible with current This strengthens the existing lineage/scope blocker: merging this head would establish both a second durable lineage and a lossy schema that cannot qualify for PostgreSQL or NoKV promotion without migration. The smallest coherent fix is to make the outbox a transport/capture mechanism for the existing runtime-shadow transaction contract: carry its versioned complete canonical projection/manifest, drain into This does not block independent PostgreSQL provider work ( Validation: reviewed the exact head against current |
90cb457 to
6ca95d0
Compare
huangruiteng
changed the title
|
Updated the existing PR branch directly at head 6ca95d0. This rebases the original six signed commits onto current main, retains the original author history, and adds three signed commits for production writer binding, coverage, and RFC alignment. The current scope uses the single coordination.runtime_shadow file-v0 lineage, preserves complete Todo and lease records, keeps legacy observation compatibility explicit, and leaves event-log-only Todo capture as a typed promotion hold. Local validation and the exact change-quality receipt are recorded in the refreshed PR summary; GitHub checks are now running. |
|
CI note: the Python workflow is red only at the repository-wide mypy step (81 errors in existing control-plane files); the current main run is also red, including the same type-check baseline. This PR’s scoped Python tests (138), focused Ruff, TypeScript typecheck, full TypeScript control-plane suite, maintainability ratchet, and LoopX premerge canary all pass. No changed runtime-shadow writer file appears in the CI mypy error list. Main baseline: https://github.com/huangruiteng/loopx/actions/runs/33895057706 ; PR job: https://github.com/huangruiteng/loopx/actions/runs/33896090462/job/101098957155 . |
huangruiteng
left a comment
huangruiteng
left a comment
There was a problem hiding this comment.
动机
这个 PR 试图解决 authority shadow 之前最关键的可信度缺口：候选 authority store 虽然已经有读写协议，但生产 Todo/lease writer 并没有在自己的权威写事务旁边留下可恢复、可核对的 shadow capture，因此观察结果无法证明它与实际 primary state 属于同一次写入。exact head 6ca95d0e656638341257ed2577ede8fcb8ef796b 把目标收敛为 transaction-bound runtime shadow：Todo Markdown 与 task lease 的生产写路径在原锁内生成 prepared/committed outbox 记录，锁外再有界投递到统一的 runtime-shadow candidate store；默认关闭时维持原行为，event-log-only completion 则明确报告未绑定而不是冒充完成。这个方向能关闭上一轮“只有 plumbing、没有 caller”和“双 lineage”的主要问题。
改动思路
整体架构是 primary-first 的两阶段旁路：writer 在 primary lock 内调用 capture begin，根据即将落盘的 canonical projection 写 durable prepared；primary 写成功后追加 committed marker；离开 primary lock 后，adapter 在独立 drain lock 下按 partition/sequence 读取 outbox，通过 effect runtime 的 coordination.runtime_shadow.commit_entry 提交到 <runtime_root>/authority-shadow/file-v0，再写 drain cursor 并清理已结算文件。Todo 和 lease 各自保持 partition 边界，operation id 绑定 goal、partition、sequence 与 source ref，以便 crash replay 和幂等提交。CLI 的 authority-shadow status/drain 提供 operator readback，Python/TypeScript codec、projection 与 RFC 一起改到统一 schema。
正向路径上，todo add/update/claim/complete/supersede/archive 以及 lease acquire/renew/transfer/release/fence-close 都已接到真实 writer；实测这些路径会生成并结算 runtime-shadow evidence。负向路径上，默认关闭不会创建 outbox/store；primary 写失败不会写 committed；candidate 调用失败保留 backlog；prepared-only 会在 primary lock 可证明空闲后按权威源解析。这个设计总体上避免让 shadow 故障改变 primary verdict。
具体改动
local_authority_shadow_outbox.py/.ts新增 durable prepared/committed entry、sequence、cursor、retired residue、prepared-only resolution 与幂等 replay 协议。runtime_shadow_writer_adapter.py和local_authority_shadow_adapter.py负责生产 writer capture、bounded drain、candidate readback，以及 Todo/lease source readers。local_authority_shadow.ts、effect runtime handlers/IO 将 commit/read 操作统一到coordination.runtime_shadow.*与loopx_coordination_runtime_shadow_projection_v0，legacy observation 只留兼容读取。todos.py、Todo projection、task lease acquire/lifecycle 及其 Python adapters 把真实生产写路径接到 capture；event-log-only 路径显式skip("event_log_writer_not_bound")。- CLI/help、双语 RFC、workflow、mypy/tsconfig 和 Python/TypeScript tests 随之更新。整个 exact-head diff 为 29 个文件、约 5.5k 新增行，属于跨 writer、runtime、operator surface 的大改动。
关键代码讲解
begin_todo_runtime_shadow_capture在 Todo primary lock 内创建与待写 Markdown bytes 绑定的 prepared entry；settle_todo_runtime_shadow_capture在锁外做有界 drain，因此不会把 runtime/store I/O 带进 primary 临界区。captureLeaseWrite/attachRuntimeShadowCapture把 TypeScript lease lifecycle 的各个落盘分支绑定到同一个 outbox 协议，beginLeaseOutboxEntry只在 primary lease write 成功后 commit。commitLocalAuthorityShadowEntry使用 entry id 作为 operation id，把一个 partition projection 事务性写入 candidate store，并返回 cursor/provider revision 供 outbox settlement。_PartitionDrainer先解析 prepared-only/committed 状态，再提交、写 cursor、清理文件；cursor-before-unlink 的顺序支持 crash 后把残留识别成 retired residue。outbox_summary是 status 与 drain prelude 的共同边界；它本应把损坏 cursor/entry 转成invalid=outbox_file_invalid，不能让本地 corruption 逃逸成未分类异常。
对主干的风险
有一个仍然可复现的 P1 correctness blocker：outbox_summary() 捕获 list_entries() / read_cursor() 的 OutboxError 后，又在 try 外调用 retired_residue(directory)；后者会通过 _retired_watermark() 再读同一份损坏的 drain-cursor.json。因此 malformed cursor 不会稳定停在 typed boundary，反而从 status/drain 的汇总路径再次抛出。反事实复现是在空 runtime root 建立 Todo partition，写入 {"schema_version":"bad"} 的 drain-cursor.json，直接调用 outbox_summary(root, goal)；当前 exact head 得到未捕获的 OutboxError: outbox_file_invalid，而不是包含 invalid 的 summary。这正是本分支较早的 67154d4bd 已修复、随后 a1a51e021 又回退的行为；对应 malformed-cursor 回归测试也在后一个 commit 中被删除。
最小修复是一次性在同一个 protected read 内读取 cursor 与目录 index，使用该快照同时计算 live entries 和 retired residue，任何 OutboxError/JSON decode 都只生成 typed invalid，不要在 catch 后重读 cursor；同时恢复 status、drain 与 capture 三条 malformed JSON/schema cursor 的回归测试。required pytest check 当前也为红（日志落在 mypy 的主干范围错误，未指向这两个新 shadow 模块），所以合入前还需让 required check 变绿或用可验证证据说明并重新运行。
独立验证方面，我在该 exact head 跑过四个 shadow Python suites，33 passed；control-plane TypeScript typecheck 通过，两个 TS suites 共 45 passed；git diff --check 通过。现有 focused suites 没抓到上述回归，因为 malformed cursor cases 已从当前 head 删除。
我的整体评价
上一轮的核心架构 blocker 已经得到实质修复：production callers、default-off isolation、统一 runtime-shadow lineage、Todo canonical receipt 和 lease lifecycle capture 都比旧 head 完整很多，设计方向值得保留。但 operator status/drain 面对损坏 cursor 时必须 fail closed 且保持 typed；当前 exact head 在这个基础恢复边界上发生明确回归，并且 required CI 仍是红色。因此本轮结论仍是 REQUEST_CHANGES。修复单次 protected read、恢复 malformed cursor 回归覆盖并让 required checks 通过后，我愿意按新 exact head 重新评审。
English verdict: REQUEST_CHANGES for exact head 6ca95d0e656638341257ed2577ede8fcb8ef796b. Production writer capture and runtime-shadow lineage are now substantially complete, but a malformed drain cursor escapes the typed status/drain boundary because outbox_summary() re-reads it via retired_residue() after catching the first error. Restore one protected snapshot read plus malformed-cursor regression coverage, and rerun the required failing check. Independent validation: 33 focused Python tests and 45 TypeScript tests passed; control-plane typecheck and diff check passed.
|
The Python CI baseline is now repaired on this branch rather than waived. What changed:
Validation:
The broad local Python run reached 5,622 passing tests and 72.30% coverage. Its remaining five failures are isolated local-machine/global-Git-configuration effects outside these typing changes; GitHub CI provides the clean environment authority for the updated head. |
Stage 2C second half, first increment: the durable, per-partition outbox and the bounded drain that turns each committed entry into exactly one candidate transaction. Nothing is wired into a writer yet; this PR only adds the plumbing, its operator surface, and the tests that pin its crash-window semantics. - local_authority_shadow_projection.py: pure compact/canonical projection rules shared by capture, drain, and the later parity verify (floats are rejected so Python and TypeScript digests cannot disagree). - local_authority_shadow_outbox.py: two-phase prepared/committed entries per lock partition (todos, leases) under <runtime_root>/authority-shadow/outbox/<goal>/, gap-free seq minting across the drain cursor, entry identity bound to the primary bytes, prepared-only resolution (committed_proven_by_readback / abandoned / unproved), and seed/reseed entries. The projector is injected, so the module stays stdlib-only and mypy strict. - local_authority_shadow_outbox.ts: beginLeaseOutboxEntry for the TypeScript lease writers, byte-compatible entry identity, durableWriteJson in effect_runtime_io.ts. - local_authority_shadow.ts: coordination.local_authority_shadow.commit_entry (folds one partition into a loopx_local_authority_shadow_projection_v1 head, operation_id = entry_id, receipt bound to the source transaction, replay only on matching partition digest, no-op transactions for abandoned and unproved resolutions) and coordination.local_authority_shadow.read (head, comparison digest, bounded scan page). - local_authority_shadow_adapter.py: drain_local_authority_shadow_outbox (per-goal drain lock, bounded batch, prepared-only resolution only under a free primary lock, cursor-before-delete ordering, readback verification), local_authority_shadow_status, read_local_authority_shadow, evidence v1 builders with measured flags only. - loopx authority-shadow drain|status CLI (help surface, lazy command registry), mypy strict list, Windows test list. - control_plane/todos/goal_todo_projection.py: the todo list projection (filtered summaries, Markdown/event merge, goal_todo_summaries, project_goal_todo_items) moves out of loopx/todos.py so the capture can project the bytes about to be written from text instead of re-reading the file; loopx/todos.py shrinks and stays within its maintainability ceiling. The observation path from the first half is unchanged and still used by every writer; evidence v0 flags stay false until the writers are wired in the next increment. No parity claim is made anywhere. Signed-off-by: wchwawa <wch19961116@gmail.com>
No behavior change; every shadow, drain, and CLI suite passes unchanged. - Reuse authorityUnicodeCompare instead of nested-ternary sort comparators in local_authority_shadow.ts and local_authority_shadow_outbox.ts. - Split readLocalAuthorityShadow into readResultBase, loadedReadResult, scanTransactionView, and appendScanPage. - Split list_entries into _index_entry_files, _load_prepared_record, and _load_committed_record; split resolve_prepared_only_entry into one resolver per source kind with a shared _lease_matches. - Split drain_local_authority_shadow_outbox into _drain_prelude, _outbox_is_idle, _drain_partitions, _settle_drain_outcome, and _count_backlog. - Split the authority-shadow Markdown renderer into per-action helpers. - Tests: one assertion per line instead of composite `and` assertions; the canonical-digest test compares two differently ordered objects instead of the same expression twice. Signed-off-by: wchwawa <wch19961116@gmail.com>
…and settlement helpers SonarCloud flagged commitLocalAuthorityShadowEntry at cognitive complexity 18. attemptCommitEntry performs one load-compose-commit round and settleCommitOutcome maps the store result to a final or retryable outcome; the exported function keeps its contract and retry count. No behavior change. Signed-off-by: wchwawa <wch19961116@gmail.com>
… recorded outbox facts Review findings on 50afe7c, both reproduced on that head: 1. A crash after the drain cursor was written but between unlinking the prepared and the committed file left a committed-only marker that list_entries() rejected as corruption, wedging drain and status for good. The cursor is written before any unlink, so every entry file at or below cursor.last_seq is settled residue: retired_residue() names it, list_entries() no longer lists or rejects it, reclaim_retired_residue() unlinks it under the drain lock at the start of every partition pass, and status reports it as retired_residue instead of invalid. A committed marker above the cursor is still corruption and still fails closed. Swapping the unlink order alone would only have turned the window into a prepared-only entry and risked a wrong source resolution or reseed. 2. capture_evidence() derived durable_source_outbox from "no failure", so a disabled or unchanged capture reported no_transaction together with durable_source_outbox=true. Both durable_source_outbox and source_transaction_correlated are now capture.recorded: true only when a prepared/committed entry was actually written for this write. Hardening from the same review: a prepared record must bind its directory (goal id and partition), carry a well-formed source_root_digest, name a known writer runtime and source kind, and recompute to its own entry id from its source reference (bytes digest, event id, or seed digest); anything else fails closed before it can reach the candidate. The drain also refuses an entry recorded for a different runtime root (source_root_mismatch). Both runtimes now hash the dot-normalized absolute root without resolving symlinks so the digest agrees across the effect-runtime boundary. Tests cover the crash between the two unlinks, the crash before any unlink (reclaimed without a store call), an orphan marker above the cursor, the foreign-root entry, tampered prepared records, the watermark semantics, and the honest evidence flags for disabled and unchanged captures. Signed-off-by: wchwawa <wch19961116@gmail.com>
outbox_summary() caught the OutboxError raised by list_entries() or read_cursor() and then re-read the cursor through retired_residue() outside that boundary, so an unsupported or unparseable drain-cursor.json escaped as an exception from `authority-shadow status`, from the drain's idle preflight, and from its backlog count instead of surfacing as the typed `outbox_file_invalid` the summary had already folded it into. The summary now performs one protected read of the cursor and the directory index and derives every count from it: a malformed cursor or entry reports `invalid=outbox_file_invalid` with zero counts and `retired_residue=0`, and nothing is re-read outside the boundary. `_load_json` types JSON and text decode errors as `outbox_file_invalid` as well, so a corrupt entry file is the same typed fact rather than a raw JSONDecodeError. No self-heal: the cursor stays exactly as found. Drain returns `stopped/outbox_file_invalid` without calling the candidate store, unlinking a file, or rewriting the cursor; status reports the typed partition fact and exits 1; a capture against the broken cursor records `outbox_prepare_failed` instead of raising into the primary write. Regressions cover all three surfaces plus the unparseable-entry case. Signed-off-by: wchwawa <wch19961116@gmail.com>
Signed-off-by: huangruiteng <huangrt01@163.com>
Signed-off-by: huangruiteng <huangrt01@163.com>
Signed-off-by: huangruiteng <huangrt01@163.com>
Signed-off-by: huangruiteng <huangrt01@163.com>
Signed-off-by: huangruiteng <huangrt01@163.com>
Signed-off-by: huangruiteng <huangrt01@163.com>
huangruiteng
left a comment
huangruiteng
left a comment
There was a problem hiding this comment.
详细中文评审
评审对象：exact head 91c1383fa09a645f422752c4b0038d448ace8f5a
动机
这个 PR 要把已有的 authority-shadow 事后快照提升为与真实 primary 写事务绑定、可恢复且可核对的 runtime-shadow evidence。Todo Markdown 与 task lease writer 在各自权威锁内写 prepared/committed outbox，锁外再有界投递到统一 candidate store；默认关闭时保持原路径，candidate 也不参与 Todo、lease 或调度决策。此前“只有 plumbing、没有生产 caller”和双 lineage 的主要问题已经解决。
改动思路
实现采用 primary-first 两阶段旁路：writer 先把即将落盘的 canonical projection 记录为 prepared，primary write 成功后追加 committed；drainer 在独立锁内按 partition/sequence 提交，以 entry id 做 operation id，成功后先写 cursor、再清理 entry。正向链路覆盖 Todo/lease 的生产写入、candidate commit/readback 与 operator status/drain；负向链路覆盖 prepared-only、candidate ambiguity、revision conflict、held lock、retired residue、foreign root 和 malformed cursor。legacy local 始终是唯一 authority，shadow 故障不能反向改变 primary verdict。
具体改动
- projection/outbox 模块固定跨语言 canonical shape、digest、sequence、prepared/committed 文件和 cursor。
- runtime-shadow writer adapter 接入 Todo add/update/claim/complete/supersede/archive 与 lease acquire/renew/transfer/release/fence-close；event-log-only completion 明确报告未绑定。
- drainer/effect runtime 统一使用
coordination.runtime_shadow.*，处理 CAS、receipt reconciliation、cursor-before-unlink 和 bounded readback。 - CLI
authority-shadow status|drain、双语 RFC、mypy/tsconfig/workflow 与 Python/TypeScript 测试同步更新。 - 本 head 新增的 17 个 kernel 文件改动仅为类型收窄与局部表达式改写，未改变业务分支；但它也没有触碰当前剩余 correctness blocker。
关键代码讲解
begin_todo_runtime_shadow_capture/settle_todo_runtime_shadow_capture把 Todo capture 分置于 primary lock 内外，避免 candidate I/O 扩大权威临界区。captureLeaseWrite/attachRuntimeShadowCapture将 lease 生命周期各落盘分支绑定到同一 outbox 协议。_PartitionDrainer._commit()校验 runtime-root lineage，并以 entry id 幂等提交；settled 后才持久化 cursor、退休文件。commitLocalAuthorityShadowEntry()区分 applied、replayed、ambiguous、operation conflict 与 revision conflict，只有完整 receipt 匹配才接受 replay。outbox_summary()是 status 与 drain preflight 的共同边界，因此必须把损坏的本地状态稳定折叠为 typed invalid，不能在 catch 后再次抛出。
对主干的风险
当前仍有一个可复现的 P1 blocker：outbox_summary() 捕获 list_entries() / read_cursor() 的 OutboxError 后，会在 try 外无条件调用 retired_residue(directory)；后者再次读取同一份 cursor。给 Todo partition 写入 {"schema_version":"bad"} 的 drain-cursor.json 后，exact head 会抛出 OutboxError(outbox_file_invalid, "drain cursor schema is unsupported")，而不是返回带 invalid="outbox_file_invalid" 的 summary。该 summary 同时服务 operator status 与 drain preflight，所以损坏 cursor 会绕过 typed fail-closed outcome。focused suites 仍未包含 malformed-cursor regression。
最小修复应在一次受保护读取中解析 cursor/index 并复用结果计算 live entries 和 residue，或至少将 residue 读取纳入同一 error boundary；补 summary/status/drain 级测试，断言 typed invalid、无 candidate call、无 entry 删除。
此外 required pytest 仍失败：test_monitor_poll_writeback_allows_when_only_registry_root_is_fenced 抛出 LocalCoordinationAuthorityUnavailable。本 head 的类型修复没有关闭该 required-check failure。独立复核：四组 shadow Python suites 33 passed，git diff --check 通过；GitHub build、Windows、dependency review、DCO 通过，但 required pytest 和非阻塞 Sonar workflow 失败。
我的整体评价
生产 caller、统一 runtime-shadow lineage、default-off isolation、transaction identity 与 crash replay 的整体方向已经成熟，前几轮大部分架构问题都已关闭。但 malformed cursor 仍能让 status/drain 绕过 typed recovery boundary，且 required pytest 是红色；这两点都属于合入前必须闭合的运行时证据。结论为 REQUEST_CHANGES。修复单次受保护 summary 读取、恢复 malformed-cursor 回归并让 required pytest 通过后，可以在新 exact head 快速复核。
English verdict: REQUEST_CHANGES at exact head 91c1383fa09a645f422752c4b0038d448ace8f5a. A malformed drain cursor is caught once and then re-read by retired_residue() outside the protected boundary, so status/drain raise instead of returning typed outbox_file_invalid; the required pytest check also still fails on split-root monitor-poll writeback. Focused shadow Python suites pass (33), diff-check passes, and build/Windows/dependency/DCO checks are green, but malformed-cursor coverage is missing.
91c1383 to
edc1682
Compare
|
Rebased this branch onto current The failure was a split-root authority bug in monitor Todo writeback: mutations used the caller's effective
Validation after rebase:
New head: |
huangruiteng
left a comment
huangruiteng
left a comment
There was a problem hiding this comment.
Final review on exact head edc168277163297f6666b18e47453ae08d7e6254: APPROVED.
The earlier requested changes were bound to superseded heads and are now closed:
- malformed cursor/status/drain handling is covered by the restored typed fail-closed path and regression suite;
- production Todo/lease writers target the single complete
coordination.runtime_shadowlineage; - monitor Todo resolution and quota fallback reads now use the same effective
runtime_rootas mutation and writer-fence checks; - all 21 configured mypy files pass without narrowing the CI scope or adding blanket ignores.
Remote checks on this exact head are green: Python full job (including fast tests), Windows, TypeScript qualification, Ruff, mypy, CLI output, DCO, dependency review, both builds, and SonarCloud. Local LoopX premerge passed 18/18 with zero holds; exact-scope receipt cqr_a6344a8a0e0c4c7480a4 is valid.
Architecture judgment: this is a large runtime change, but it now delivers the coherent Stage 2C increment rather than disconnected plumbing: primary-first transaction-bound evidence, one complete candidate lineage, bounded/idempotent drain, default-off isolation, and fail-closed authority transition behavior. No unresolved merge blocker remains.
3 participants
Current scope (2026-09-05, head 6ca95d0)
This update preserves the original contributors outbox and crash-window work, rebases it onto current main, and completes the next increment: production Todo and task-lease writers now capture transaction-bound runtime-shadow entries. The candidate is the existing coordination.runtime_shadow file-v0 AuthorityStore lineage; no second candidate store is introduced. Todo records reuse the complete canonical Todo read contract and lease projections retain the complete versioned record, so this PR does not create a field-dropping migration boundary.
Default-off behavior remains zero-effect. The prior coordination.authority_shadow observation API and store remain available only for compatibility; new outbox delivery uses coordination.runtime_shadow.commit_entry and coordination.runtime_shadow.outbox_read. Event-log-only Todo completion is intentionally reported as event_log_writer_not_bound until that multi-event writer owns its prepare/commit boundary, so it remains a promotion hold rather than false parity evidence.
The production writer coverage now includes Todo add, update and claim, complete, supersede, and archive, plus lease acquire, renew, transfer, release, user-gate auto-acquire, and terminal fence-close release. Writer orchestration was extracted from the legacy adapter to keep the maintainability ratchet green. The RFC English and Chinese implementation plan now reflect this current boundary.
Validation on the current main basis: TypeScript control-plane typecheck passed; full TypeScript control-plane suite passed 556 with one optional live PostgreSQL integration skipped; 138 focused Python tests passed; focused Ruff, diff-check, maintainability ratchet, public boundary scan, catalog canaries, and risk-profile smokes passed. Exact LoopX change-quality receipt: cqr_bf66c53dd411ca484b3d.
Summary
FileAuthorityStoretransaction whoseoperation_idis the entry id, so a receipt names the primary transaction it records instead of a post-commit snapshot that may include other writers.false. This PR adds the plumbing, its operator surface (loopx authority-shadow drain|status), and the tests that pin its crash-window semantics.source_candidate_compared=falseandparity_verdict=not_evaluated; the verify step is the next increment.@huangruiteng This PR is stacked on #3818 (base branch
codex/local-authority-shadow-product-path) and consumes itseffective_runtime_rootandgoal_todo_projectionmodule; retarget tomainonce #3818 merges. Increment order for the second half: this plumbing, then writer wiring plus seed and evidence v1, then verify with a typed parity verdict, then the recovery matrix and reset.Issue Or Task
docs/architecture/rfcs/shared-goal-authority-state-provider-v0.md(section 11, Stage 2C second half). The RFC text is not changed by this PR; it changes when the writers are wired.stage-2c-parity-outbox-plumbingWhat it adds
local_authority_shadow_projection.py: pure compact and canonical projection rules shared by capture, drain, and the later parity verify; floats are rejected so Python and TypeScript digests cannot disagree.local_authority_shadow_outbox.py: two-phaseprepared/committedentries per lock partition (todos,leases) under<runtime_root>/authority-shadow/outbox/<goal>/, gap-freeseqminting across the drain cursor, entry identity bound to the primary bytes, prepared-only resolution (committed_proven_by_readback/abandoned/unproved), seed and reseed entries. The projector is injected, so the module stays stdlib-only and mypy strict.local_authority_shadow_outbox.ts:beginLeaseOutboxEntryfor the TypeScript lease writers with byte-compatible entry identity;durableWriteJsonineffect_runtime_io.ts.local_authority_shadow.ts:coordination.local_authority_shadow.commit_entryfolds one partition into aloopx_local_authority_shadow_projection_v1head (a v0 head is a valid starting point), replays only when the existing receipt carries the same partition digest, and records abandoned or unproved resolutions as no-op transactions so the sequence stays auditable;coordination.local_authority_shadow.readreturns the head, its comparison digest, and a bounded scan page.local_authority_shadow_adapter.py:drain_local_authority_shadow_outbox(per-goal drain lock, bounded batch, prepared-only resolution only under a free primary lock, cursor-before-delete ordering, readback verification),local_authority_shadow_status,read_local_authority_shadow, evidence v1 builders with measured flags only.loopx authority-shadow drain|status, help surface, lazy command registry, mypy strict list, Windows test list.Crash windows pinned by tests: prepared entry without a marker resolves from the source bytes (
committed_proven_by_readbackorabandoned); an unexplained gap produces anunprovedno-op plus areseed_after_crash_gapsnapshot under the primary lock; store committed but cursor missing replays; cursor written but entry files not (fully) removed is retired residue reclaimed under the drain lock without a store call; a committed marker above the cursor is corruption and fails closed; an entry recorded for another runtime root fails closed (source_root_mismatch); a held drain lock defers; a bounded batch reportspending_after; an invalid store result stops in order; TypeScript-written lease entries are compacted at drain.Not in this PR: writer wiring, seed on enable,
verify,reset, rollback accounting, and the file-store growth fix (still quadratic;statusonly reportsstore_bytesandretention_pressure).Round 2 (exact head
a16f746bf, rebased ontomainafter #3818 merged)Both review findings on
50afe7cbareproduced on that head before the fix.retired_residue(directory)names every entry file at or belowdrain-cursor.last_seq;list_entries()skips it,reclaim_retired_residue()unlinks it under the drain lock at the start of each partition pass,statusreportsretired_residue, andDrainResult.reclaimed_residuecounts it. A committed marker above the cursor staysoutbox_file_invalid. The unlink order is unchanged on purpose: swapping it would turn the window into a prepared-only entry and invite a wrong source resolution or reseed.durable_source_outboxandsource_transaction_correlatedare bothcapture.recorded; a disabled or unchanged capture now saysno_transactionwith both flagsfalse.source_root_digest, name a known writer runtime and source kind, and recompute to its own entry id from its source reference (bytes digest,event:<id>, orseed:<partition digest>); otherwiselist_entries()fails closed. The drain additionally refuses an entry whosesource_root_digestis not this runtime root (source_root_mismatch), and both runtimes now hash the dot-normalized absolute root without resolving symlinks so the digest agrees across the effect-runtime boundary.commit_entrycalls), orphan marker above the cursor (still corruption), foreign-root entry (typed stop, entry retained), tampered prepared records (goal, partition, source digest, root digest, writer runtime, source kind), watermark semantics, and the honest flags for disabled and unchanged captures.Rebase note: #3818 merged into
mainon 2026-09-03, so this PR now targetsmaindirectly; the only rebase conflict was thetsconfig.control-plane.jsoninclude list, resolved as the union. On the rebased head the same validation holds: focused shadow, migration, ratchet, and the new legacy-writer-fence suites 54 passed; fullpytest -n 45318 passed, 12 skipped, with the same environment-only failures asmainhere (the git-hook, doctor runtime-readiness, and pydantic/fastmcp MCP tests, plus the read-only SSH transport test #3903 added to that family); TypeScript control plane 500 passed / 1 PostgreSQL integration skipped; typecheck, Ruff, configured mypy, diff-check clean; premerge againstorigin/main: 4/4 direct checks, 18/18 catalog canaries, boundary clean, 0 holds. A combined tree ofmain+ #3869 + this branch also runs green (typecheck, mypy 21 files, 64 focused tests, the full ladder at pass 9 / unverified 3 / pending 9, TypeScript 500/1).Round 3 (exact head
ba9122f3f)Fixes the blocker found on
87d716585:outbox_summary()caught theOutboxErrorfromlist_entries()/read_cursor()and then re-read the cursor throughretired_residue()outside that boundary, so an unsupported or unparseabledrain-cursor.jsonescaped as an exception fromauthority-shadow status, from the drain's idle preflight, and from its backlog count.invalid=outbox_file_invalidwith zero counts andretired_residue=0; nothing is re-read outside the boundary._load_jsontypes JSON and text decode errors asoutbox_file_invalid, so a corrupt entry file or cursor is the same typed fact instead of a rawJSONDecodeErrorreachingstatus, the drain, or a capture.next_seq/list_entries/retired_residueraise the typed error; drain returnsstopped/outbox_file_invalidwith zero candidate-store calls, no unlink, and a byte-identical cursor; status reports the typed partition fact withok=false(CLI exit 1); a capture against the broken cursor recordsoutbox_prepare_failedinstead of raising into the primary write; an unparseable entry file is typed too.Malformed-cursor policy (deliberate, no self-heal): the cursor is the only record of the retired watermark once residue has been unlinked, so rebuilding it from the surviving entry files can re-mint a seq below the real watermark. Repair is an operator action; a
reset/rebuild command that reconciles against candidate receipts belongs to the verify increment, not to this plumbing.Staged boundary (coverage-only, for the record): at this head
TodoPartitionCaptureandbeginLeaseOutboxEntry()have no production caller; the runnable surface isauthority-shadow status|drainand the candidatecommit_entry/readhandlers, and the existing writers keep using the #3818 observation. Tests construct entries through the same two-phase API the writers will call. Writer wiring is the next increment and its drain target follows the lineage decision recorded as question 14 in #3909.Round 4 (exact head
90cb457ea)Closes what remained of the malformed-cursor blocker after round 3, and the cross-runtime hole found while closing it.
read_cursor()binds the cursor to the closed shapewrite_cursor()produces for its partition: schema,partitionequal to the directory name, integerlast_seq >= 1(not a bool),last_entry_idmatching the entry id pattern, optional text fields,updated_at. Anything else isoutbox_file_invalid, so a valid schema withlast_seq: "oops"no longer reachesint()and escapes as a rawValueErrorfrom the summary, status, the drain preflight, ornext_seq().OSError) are typedoutbox_file_invalidat the two primitives every reader goes through (_load_json, the directory listing), alongside the JSON/text decode errors typed in round 3.nextSeq()read the cursor leniently and would have minted a seq from the surviving files alone wheneverlast_seqwas not an integer, which lets the next drain retire that entry unread as cursor-covered residue. It now applies the same closed-shape binding; the lease capture fails closed withoutbox_prepare_failedand writes nothing.read_cursor/next_seq/list_entries/retired_residuewith the files left untouched; the closed shape accepted; unparseable and unreadable entry files typed; drain and status typed on the reviewer's exactlast_seq: "oops"cursor with zero store calls and no unlink; the TypeScript capture fails closed on four cursor shapes and accepts the drain's own.Validation
Round 4 on
90cb457ea: focused shadow suites 57 passed; TypeScript outbox suite 9 passed, full control plane 501 passed / 1 PostgreSQL integration skipped; control-plane typecheck, Ruff, configured mypy (17 files),git diff --checkclean; premerge againstorigin/mainok; the CLI on the reviewer's exact cursor exits 1 from bothstatus(typedinvalid) anddrain(stopped/outbox_file_invalid).Round 3 on
ba9122f3f: focused shadow suites 46 passed (outbox, drain, CLI E2E, config, runtime); fullpytest -n 25384 passed / 12 skipped with the same 10 environment-only failures asmainhere (the nine git-hook tests and the doctor runtime-readiness test); Ruff, configured mypy (17 files),git diff --checkclean; premerge againstorigin/mainok; the CLI on a goal with an unparseable cursor exits 1 from bothstatus(typedinvalid) anddrain(stopped/outbox_file_invalid). No TypeScript file changed this round.tests/control_plane/test_local_authority_shadow_outbox.py(8) andtests/control_plane/test_local_authority_shadow_drain.py(10), plus a CLI E2E row forauthority-shadow status|drainthat proves a status probe mints no store identitypytest -n 4: 5186 passed, 12 skipped, with the same 12 environment-only failures (git-hook install, doctor runtime readiness, pydantic/fastmcp MCP tests) that fail identically onmainin this environmenttests/control_plane_ts/local_authority_shadow_outbox.test.ts(8, including Python/TypeScript entry-identity and head-digest parity); TypeScript control plane 471 passed, 1 PostgreSQL integration skipped;npm run typecheck:control-planecleanloopx canary premerge --from-git-diff --git-diff-base <#3818 head>on this head: standard tier, 4/4 direct checks, 18/18 catalog canaries with 0 failures and 0 warnings,full-publicrisk profile ok,premerge-public-boundaryclean, 0 manual holdsPremerge comment fields: changed_surfaces = coordination adapter and two new pure modules, TypeScript shadow handlers and outbox module,
effect_runtime_io.ts, effect runtime handler registration,loopx/cli.pyandcli_commands(newauthority-shadowverb),help_surface.py,goal_todo_projection.py(addsproject_goal_todo_items), mypy list, Windows test list, tsconfig include list; direct_checks = 4/4; catalog_canaries = 18/18; risk_profile_smokes = full-public ok; public_private_boundary = clean; failures_or_skips = none in the gate; manual_holds = none; merge_decision = ready for review after #3818; no writer behavior changes.Type of Change
LoopX Area
Technical Direction
Shared Goal Authority and cross-host coordination
Target base branch:
main(feat(authority): add local post-commit observation capture #3818 merged 2026-09-03)Direction tracker or promotion unit: Stage 2C second half, parity plumbing increment 1
Boundary Checklist
.loopx/,.codex/goals/, liveACTIVE_GOAL_STATE.md, credentials, private benchmark traces, verifier output, raw agent sessions, internal document links, or local machine paths.Signed-off-bytrailer (git commit -s).