fix(authority): recover concurrent commands and verify current claim execution - #4886
Conversation
… execution Signed-off-by: huangruiteng <14976749+huangruiteng@users.noreply.github.com>
… providers Signed-off-by: huangruiteng <14976749+huangruiteng@users.noreply.github.com>
…roof Signed-off-by: huangruiteng <14976749+huangruiteng@users.noreply.github.com>
huangruiteng
left a comment
There was a problem hiding this comment.
Approval conclusion (author-owned PR; GitHub blocks formal self-approval)
动机
这个 PR 修的是两个彼此关联、但过去分散在各 command caller 里的 authority 缺口。
第一,command receipt 与 authority head 是两次独立读取:旧流程先读一次 receipt,看到 missing 后再读 head。如果另一个进程恰好在这两个读取之间提交了同一个 operation_id,当前调用者会拿着已经包含该 effect 的 head 继续做“新请求”判断,可能错误返回 duplicate/stale/conflict,而不是恢复已经 durable 的结果。
第二,receipt 只证明历史 acquisition decision,不证明这个 execution 现在还有效。尤其 atomic Todo claim + lease acquire 的历史结果,在 lease 已 renew、release、transfer、expire,或 Todo 被 archive/done/exclude、claim 转移、agent 注销、handoff mode/acceptance contract 改变后,不能继续携带旧 lease 作为执行授权。
本次变更实现了完整的有用切片:同一个操作在观察窗口内完成时能稳定恢复,而所有 lease-bearing success 都重新经过当前 canonical state 的执行证明。它没有把范围扩成 provider migration、writer retry 或新的 authority model。
改动思路
设计把三种不同权威保持分离:
CoordinationCommandReceipt仍只拥有 operation identity、历史 payload 解码和 post-commit recovery;新增observe()统一执行“load head 后再读 receipt”。第二次 receipt 只要存在就优先返回,只有确认 missing 才把刚才加载的 head交给 domain admission。- 各 command 原有 planner、request digest、receipt schema、projection commit 和 CAS 仍是各自 owner;
observe()不做 transaction,也不重试 write。观察后再发生的竞争继续由 provider CAS 和已有 post-commit receipt recovery 收口。 - 新的
currentLeaseAcquisitionProof()复用原 standalone acquire 里已经存在的 present-tense proof,并让 atomic claim/acquire 走同一规则。它检查 current handoff mode、Todo eligibility、owner、idempotency key、epoch、version、expiry 和 Goal acceptance;renew 可以返回同一 execution 的新版本,而 release/transfer/expiry 等必须拒绝旧 receipt。
我也检查了 latest origin/main 上的集成结果。这个 exact head 可以无冲突叠加到 2e94e625ecb22adab8ae2c9db900b685480ebd4e;与近期 promotion recovery 的边界一致:receipt/plan 是 identity carrier,不自动成为当前 execution authority。
具体改动
command_receipt.ts新增 typedCommandObservation和observe(),把重复的 receipt/head ordering 规则集中到一个 owner。- Todo create/update/claim/terminal/archive/monitor、lease acquire/lifecycle、team plan、Goal acceptance configure/verify 都改用这条 observation path;dry-run archive 仍只读当前 head,不消费 durable operation identity。
lease_acquisition_proof.ts提取完整 current proof;standalone acquire 与 atomic claim/acquire 共用,atomic claim 额外要求hard_lease。- receipt 保持 immutable:renew 后返回 current lease,但原 acquisition receipt 的 provider revision/payload 不被改写。
- 新 conformance scenarios 在 File、SQLite、NoKV provider 上覆盖 mid-read concurrent commit、lost ACK、renew/release/transfer/expiry、eligibility/mode/acceptance 改变,以及 current head unavailable。
关键代码讲解
CoordinationCommandReceipt.observe:先loadAuthority(),后readReceipt()。这不是“head 和 receipt 的假原子读”;它只规定第二次 receipt observation 的优先级。若 receipt 仍 missing,caller 使用已经加载的 head;若随后才发生竞争,commit CAS 仍会拒绝/恢复,避免第二次 write。currentLeaseAcquisitionProof:先解码历史 lease identity,再从完整 canonical projection 取得 current lease/Todo/mode。只有同 owner/key/epoch、current version 不回退且 active,Todo/agent/mode/acceptance 全部允许时才返回 lease;否则 failure 中不泄露 usable old lease。executeCanonicalTaskLeaseAcquire:删除本地重复 proof,保留原 normalization、request hash、planner 与 commit 结构。每个 applied/no_change/replayed/recovered result 都必须经过 current proof。executeCoordinationTodoClaim:plain historical claim 继续只做 source/acceptance checks;带lease_request的任何 replay、recovery 或新 commit 都强制 present-tense proof,从而避免“claim history 等于 execution grant”。
对主干的风险
未发现 blocking finding。最强反例是:receipt 完全合法,但在返回前另一 actor 已 release/transfer 该 lease,或 acceptance 已重绑。新增 proof 会从当前 full head 观察到变化并返回 typed failure;它不会修改历史 receipt,也不会自动重试 write。
我执行的验证:
- latest-main integration merge:干净,无冲突;
git diff --check通过。 npm run typecheck:control-plane:通过。- File / SQLite / NoKV 上按
command observation|claim proof筛选的 provider-neutral exact-head matrix:全部通过。 - command receipt unit contract:第二次 read、read failure、confirmed absence、late CAS recovery、no write retry 全部通过。
uv run --extra test python -m pytest -q tests/control_plane/test_canonical_claim_execution_proof.py:4 passed,走 shipped Python CLI bridge。
第一次直接从 Node 启动大矩阵时,有若干 succession/consumer tests 因 subprocess 取到系统旧 Python、不支持 dataclass(slots=...) 而失败;这不是代码断言失败。按仓库 source-checkout 规则改由 uv run --extra test 固定 CPython 3.13.13 后,受影响的目标路径通过。
语义与 CI 对齐
这是对既有 receipt / current lease / provider CAS vocabulary 的复用,不新增第二套 state authority。默认行为的变化已经在 RFC、canonical-lease-renew.md 和具名 conformance 中披露:并发同操作从错误冲突改为恢复;历史 lease-bearing replay 从“返回旧结果”改为“必须证明当前执行”。错误文案保持 goal/provider-neutral,所有 obligation 都是 typed branch,不是 prose guidance。packet 明确 wait_for_ci=false,因此本结论基于本地 exact-head evidence,没有轮询或借用远端 CI。
我的整体评价
这是必要且比例合适的 control-plane correctness 修复。生产代码没有新增 persisted state、provider 或 CLI,而是把已经存在的两个规则放回正确 owner:receipt observation 归 command receipt,current execution validity 归共享 lease proof;各 domain planner 与 CAS 继续独立。测试体量较大,但覆盖的是跨 provider、跨 command 的真实复用面,不是一次性 output fixture。
剩余风险是本轮没有重新跑完整 PostgreSQL conformance/长 soak;本 PR 没改 provider implementation,File/SQLite/NoKV 的真实 journal/CAS 和 Python shipped entrypoint 已覆盖变化本身。由于它改变 control-plane/runtime authority behavior,仍必须由 maintainer 决定合并,我没有执行 merge。
English verdict: APPROVE — the exact head correctly recovers concurrent identical commands and prevents historical receipts from granting retired execution authority; no blocking finding remains.
huangruiteng
left a comment
There was a problem hiding this comment.
Approval conclusion (author-owned PR; GitHub blocks formal self-approval)
动机
这个 PR 修的是两个彼此关联、但过去分散在各 command caller 里的 authority 缺口。
第一,command receipt 与 authority head 是两次独立读取:旧流程先读一次 receipt,看到 missing 后再读 head。如果另一个进程恰好在这两个读取之间提交了同一个 operation_id,当前调用者会拿着已经包含该 effect 的 head 继续做“新请求”判断,可能错误返回 duplicate/stale/conflict,而不是恢复已经 durable 的结果。
第二,receipt 只证明历史 acquisition decision,不证明这个 execution 现在还有效。尤其 atomic Todo claim + lease acquire 的历史结果,在 lease 已 renew、release、transfer、expire,或 Todo 被 archive/done/exclude、claim 转移、agent 注销、handoff mode/acceptance contract 改变后,不能继续携带旧 lease 作为执行授权。
本次变更实现了完整的有用切片:同一个操作在观察窗口内完成时能稳定恢复,而所有 lease-bearing success 都重新经过当前 canonical state 的执行证明。它没有把范围扩成 provider migration、writer retry 或新的 authority model。
改动思路
设计把三种不同权威保持分离:
CoordinationCommandReceipt仍只拥有 operation identity、历史 payload 解码和 post-commit recovery;新增observe()统一执行“load head 后再读 receipt”。第二次 receipt 只要存在就优先返回,只有确认 missing 才把刚才加载的 head交给 domain admission。- 各 command 原有 planner、request digest、receipt schema、projection commit 和 CAS 仍是各自 owner;
observe()不做 transaction,也不重试 write。观察后再发生的竞争继续由 provider CAS 和已有 post-commit receipt recovery 收口。 - 新的
currentLeaseAcquisitionProof()复用原 standalone acquire 里已经存在的 present-tense proof,并让 atomic claim/acquire 走同一规则。它检查 current handoff mode、Todo eligibility、owner、idempotency key、epoch、version、expiry 和 Goal acceptance;renew 可以返回同一 execution 的新版本,而 release/transfer/expiry 等必须拒绝旧 receipt。
我也检查了 latest origin/main 上的集成结果。这个 exact head 可以无冲突叠加到 2e94e625ecb22adab8ae2c9db900b685480ebd4e;与近期 promotion recovery 的边界一致:receipt/plan 是 identity carrier,不自动成为当前 execution authority。
具体改动
command_receipt.ts新增 typedCommandObservation和observe(),把重复的 receipt/head ordering 规则集中到一个 owner。- Todo create/update/claim/terminal/archive/monitor、lease acquire/lifecycle、team plan、Goal acceptance configure/verify 都改用这条 observation path;dry-run archive 仍只读当前 head,不消费 durable operation identity。
lease_acquisition_proof.ts提取完整 current proof;standalone acquire 与 atomic claim/acquire 共用,atomic claim 额外要求hard_lease。- receipt 保持 immutable:renew 后返回 current lease,但原 acquisition receipt 的 provider revision/payload 不被改写。
- 新 conformance scenarios 在 File、SQLite、NoKV provider 上覆盖 mid-read concurrent commit、lost ACK、renew/release/transfer/expiry、eligibility/mode/acceptance 改变,以及 current head unavailable。
关键代码讲解
CoordinationCommandReceipt.observe:先loadAuthority(),后readReceipt()。这不是“head 和 receipt 的假原子读”;它只规定第二次 receipt observation 的优先级。若 receipt 仍 missing,caller 使用已经加载的 head;若随后才发生竞争,commit CAS 仍会拒绝/恢复,避免第二次 write。currentLeaseAcquisitionProof:先解码历史 lease identity,再从完整 canonical projection 取得 current lease/Todo/mode。只有同 owner/key/epoch、current version 不回退且 active,Todo/agent/mode/acceptance 全部允许时才返回 lease;否则 failure 中不泄露 usable old lease。executeCanonicalTaskLeaseAcquire:删除本地重复 proof,保留原 normalization、request hash、planner 与 commit 结构。每个 applied/no_change/replayed/recovered result 都必须经过 current proof。executeCoordinationTodoClaim:plain historical claim 继续只做 source/acceptance checks;带lease_request的任何 replay、recovery 或新 commit 都强制 present-tense proof,从而避免“claim history 等于 execution grant”。
对主干的风险
未发现 blocking finding。最强反例是:receipt 完全合法,但在返回前另一 actor 已 release/transfer 该 lease,或 acceptance 已重绑。新增 proof 会从当前 full head 观察到变化并返回 typed failure;它不会修改历史 receipt,也不会自动重试 write。
我执行的验证:
- latest-main integration merge:干净,无冲突;
git diff --check通过。 npm run typecheck:control-plane:通过。- File / SQLite / NoKV 上按
command observation|claim proof筛选的 provider-neutral exact-head matrix:全部通过。 - command receipt unit contract:第二次 read、read failure、confirmed absence、late CAS recovery、no write retry 全部通过。
uv run --extra test python -m pytest -q tests/control_plane/test_canonical_claim_execution_proof.py:4 passed,走 shipped Python CLI bridge。
第一次直接从 Node 启动大矩阵时,有若干 succession/consumer tests 因 subprocess 取到系统旧 Python、不支持 dataclass(slots=...) 而失败;这不是代码断言失败。按仓库 source-checkout 规则改由 uv run --extra test 固定 CPython 3.13.13 后,受影响的目标路径通过。
语义与 CI 对齐
这是对既有 receipt / current lease / provider CAS vocabulary 的复用,不新增第二套 state authority。默认行为的变化已经在 RFC、canonical-lease-renew.md 和具名 conformance 中披露:并发同操作从错误冲突改为恢复;历史 lease-bearing replay 从“返回旧结果”改为“必须证明当前执行”。错误文案保持 goal/provider-neutral,所有 obligation 都是 typed branch,不是 prose guidance。packet 明确 wait_for_ci=false,因此本结论基于本地 exact-head evidence,没有轮询或借用远端 CI。
我的整体评价
这是必要且比例合适的 control-plane correctness 修复。生产代码没有新增 persisted state、provider 或 CLI,而是把已经存在的两个规则放回正确 owner:receipt observation 归 command receipt,current execution validity 归共享 lease proof;各 domain planner 与 CAS 继续独立。测试体量较大,但覆盖的是跨 provider、跨 command 的真实复用面,不是一次性 output fixture。
剩余风险是本轮没有重新跑完整 PostgreSQL conformance/长 soak;本 PR 没改 provider implementation,File/SQLite/NoKV 的真实 journal/CAS 和 Python shipped entrypoint 已覆盖变化本身。由于它改变 control-plane/runtime authority behavior,仍必须由 maintainer 决定合并,我没有执行 merge。
English verdict: APPROVE - at 2decb65, the exact head correctly recovers concurrent identical commands and prevents historical receipts from granting retired execution authority; no blocking finding remains, and the validated provider-neutral, Python CLI, typecheck, diff-check, and latest-main integration evidence passed.
Goal And Delivered Outcome
Related to #4574 and shared-authority L2/L3; base:
main.A command could read an absent receipt, observe a head already changed by a peer using the same operation id, then fail duplicate-Todo or stale-revision admission instead of recovering that operation. Separately, atomic claim/acquire replay could return its historical active lease after execution had expired or been released. Standalone acquire already checked current proof.
Commands now recheck receipts after loading the decision head, before interpreting new state. Standalone and atomic claim acquisition share current execution proof: renewal returns the current lease with the immutable original receipt; retired/ineligible execution is rejected; unavailable current authority is ambiguous and recoverable with the same operation id.
Scope And Continuation
Complete within this concurrency/current-proof slice:
Provider defaults, whole-Goal promotion/export/rollback, SQLite D2/soak (#4224), executor external-effect fences and Python retirement remain separate roadmap work. #4870 is merged; reviewed promotion/recovery #4879 remains open. This PR neither promotes an active Goal nor claims that its command rehearsal qualifies promotion. The conditional 5–8-package default-migration estimate remains unchanged. Reverting this slice changes no stored schema or provider selection.
Validation
2decb65d507799cc19ba38662de79afc11d2e7a0finishedsynthetic,public_fixture,authorized_private_read_onlyCoverage includes deterministic late-receipt races, stale CAS, loss of acknowledgment, changed intent, no-op receipts, renewal, expiry, joint transfer, registration/exclusion/archive changes, acceptance rebinding, unavailable current head and source-witness drift. Current proof is a point-in-time observation; later external effects still require their existing execution fences.
Frontend / Visual Evidence
UI impact:
none. No settings, parameters, new screen or wire schema is introduced. Existing Chat Todo actions still delegate to the unchanged Python command adapters and TS owners; the public CLI validates success/error readback. Packaged frontend rendering is not changed or separately qualified.Shared-authority RFC fixture impact
Type / Area / Boundary
Bug fix, bounded TS refactoring, tests and documentation; control plane / Shared Goal Authority.
All three commits have DCO sign-offs. No private state, raw logs, credentials, internal links or local paths are included. Maintainer review and merge are required.