Skip to content

fix(turn): record quota CAS failures as failed settlement - #6449

Merged
huangruiteng merged 1 commit into
loopx-project:mainfrom
ikunikun1145:codex/turn-cas-failure-5885
Oct 15, 2026
Merged

huangruiteng merged 1 commit into
loopx-project:mainfrom
ikunikun1145:codex/turn-cas-failure-5885

Conversation

@ikunikun1145

@ikunikun1145 ikunikun1145 commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Goal And Delivered Outcome

Implemented Behavior

Criterion Disposition Symbol / path Regression coverage
Preserve the typed diagnostic for a definitive quota run-index CAS rejection implemented QuotaRunIndexConflict in loopx/control_plane/quota/spend_commit.py test_spend_quota_slot_rejects_reusing_a_stale_status_snapshot
Persist failed settlement with completed validation/writeback receipts implemented spend_effect in loopx/control_plane/turn_driver/executor.py test_quota_run_index_conflict_is_audited_and_retry_keeps_completed_work
Keep non-CAS provider errors on the existing recovery path implemented Handle only the existing index_digest_conflict diagnostic test_permission_denial_during_spend_preserves_writeback_and_resumes

Validation

  • Tested revision: 52bfcf6
  • Input classes: synthetic, public_fixture
Check Result Evidence / limitation
Focused tests passed tests/test_loopx_turn_executor.py and tests/control_plane/test_quota_spend_commit_runtime.py: 91 passed.
Static checks passed Ruff, changed-file Python compilation, semantic-vocabulary drift smoke, and diff checks passed.
Regression behavior passed The regression records quota_spend_failed; retry keeps host and writeback call counts at one.
Pre-merge gate passed Current-base premerge selected and executed 13 checks with zero failures or warnings.
  • Full repository pytest is still running on the prior revision bcb8d80; it has not produced a terminal result. It is not counted as validation of this tested revision.
  • Hosted workflows are awaiting maintainer approval and have not run. A code-owner review is also required.

Scope And Compatibility

  • The change completes the linked quota-settlement defect. Ambiguous provider outcomes remain on the existing recovery path.
  • No UI behavior changes.
  • Every commit includes a DCO Signed-off-by trailer.

Signed-off-by: ikun <ikunikun1145@gmail.com>
(cherry picked from commit bcb8d80)
@ikunikun1145
ikunikun1145 force-pushed the codex/turn-cas-failure-5885 branch from defdbe5 to 52bfcf6 Compare October 14, 2026 19:46

@loopx-agent loopx-agent left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewer: model_agent; model=gpt-6.1-sol; provider=OpenAI; reasoning_effort=xhigh; declaration_source=runtime_reported

动机

评审 head 52bfcf643a9289e1f2c9acbcc7e9c41c2707d320(base 3c681f1132fe719e01f75e5a41bfd854dfa9e346)。并行运行任务的操作者,已经完成并验证工作,却在最后记账时遇到另一个任务更新账本。工作已被下游使用,日志却仍显示进行中且没有可归属的失败回执,操作者需手工核对并担心重试会重复工作或扣额。两个任务先后记账,后一个使用旧快照而被拒绝;原来错误直接退出并留下进行中日志,现在保留完成阶段并明确记录结算失败,刷新状态后可用原身份重试。真实本地账本拒绝写入后,当前版本留下可审计的失败状态、归属和回执;显式重试只完成剩余记账,Host 工作与写回各一次。本 PR 不自动重试、不解决所有并发冲突,也不声称已复现 Windows 原始 dsh 环境或证明长期吞吐收益。

改动思路

账本的 CAS 是写入前的快照比较:只有预期摘要仍等于当前账本,新的记账才可提交。这里复用原有权威:TypeScript 继续决定账本 CAS 与结算状态;Python 只把确切的未提交错误转换为既有失败输入。只修复原有 Turn 记账尾部,复用原回执、重试和 callback adapter,不新增自动调度或第二套状态机。 没有按错误文本猜测结果,也没有新增自动重试循环。既有回执的修复/重放先于新写入 CAS,因此当前精确的摘要冲突可确认本次未追加;无法确定是否提交的 provider 错误仍交给原恢复流程。

具体改动

关键代码讲解

  1. QuotaRunIndexConflict(quota/spend_commit.py:29)继承 ValueError,保留原直接调用者的异常兼容性,只携带既有 index_digest_conflict 原因。
  2. _quota_spend_commit_result(同文件 :92)分别处理 runtime exception 与结构化 conflict 两条返回路径,仅在确切 diagnostic/reason code 匹配时抛该类型;其他冲突、拒绝和未知错误不扩大归类。
  3. _typed_settlement_stage(turn_driver/executor.py:1105)在 spend adapter 捕获该类型,提交 ok=false/appended=false 给原 TypeScript settlement reducer。原 owner 产生 quota_spend_rejected、quota_spend_failed、失败阶段、归属及此前已完成回执,最后持久化失败日志。
  4. invoke_turn_effect(turn_driver/settlement.py:84)由原私有 helper 暴露给 adapter,复用原有关键字/位置 stable-ref 与零参 callback 兼容。已做有界前向重构检查,避免复制 signature 反射,也未建立第二套 Python 状态决策。

全 diff 为五路径 +101/-7:三处 runtime 与两组回归测试,一份 DCO commit。依据 原问题 #5885:auditable_finalization_failure 和 safe_remaining_stage_retry 在当前 head implemented;建议中的 automatic_bounded_retry 是替代方向,当前 out_of_scope。本次完成其明确失败与显式安全重试的最小闭环,不宣称有自动执行者。

对主干的风险

最需要避免的是把“可能已经提交但丢回复”的异常误认成未提交,再重复扣额。当前代码只捕获确切类型,真实 TS CAS 判定发生在追加之前;PermissionError 等不确定失败继续保持 prepared 并要求原 resolver 读回。当前两组 Python 91 passed / 0 skipped,相同 base 原 90 passed / 0 skipped,包含 provider 不确定错误、原 stable-ref、失败重放和恢复边界。

独立实际账本+Turn 对照:先真实追加一个竞争操作,再用旧摘要结算。base 抛 ValueError、日志 in_progress、无回执;head 返回 failed/quota_spend_failed,回执含 Goal/Agent/Todo lineage 及 validation、durable_writeback。拒绝后索引未变,普通重放没有效果;刷新摘要后显式 retry_failed 成功,Host=1、writeback=1、spend 尝试=2,账本总计两行(竞争操作一行、本 Turn 一行),成功后的重放也不重复追加。这里 quota transaction、文件和 Turn journal 是真实本地实现;Host、验证、写回及初始 status 输入是合成 fixture,未冒充 Windows/dsh 原场景。

Ruff、diff check、DCO、先 advisory 后完整 semantic 与 exact-base 风险 canary(5 direct + 13 selected,零失败/hold)均单独核验;本轮不获取或等待 hosted CI。没有新增前端入口或配置,此修复接入现有 turn run-once spend closure;源码组合验证不代表已验证安装升级或原 Windows 宿主。新增默认仅为确切 CAS 失败可见,普通成功/未显式 retry 的执行边界保留。

语义与 CI 对齐

复用既有 quota transaction 的 reason 与 Turn failure vocabulary;新异常仅是本地 Python 传输类型。advisory 对三个源码没有发现支持的 vocabulary carrier,仍人工追踪了异常与动态原因;全量语义检查通过。未知 provider 错误仍需原回执恢复,失败日志和恢复建议都不授予额外执行、配额或合并权限。

我的整体评价

APPROVE,goal_achieved,限定于 #5885 的明确结算失败和显式剩余阶段恢复。 long_horizon 为 improved:完成的工作在冲突后仍可归属、审计并重试;user_experience 为 improved:调用者得到明确的失败和恢复边界。以上是本地实证,不外推长期吞吐或原 Windows 环境。没有发现当前五路径的新增阻塞,机制与问题成比例;需原生 Windows 或长期并发结论时补相应证据。当前评审不授予合并或部署权限。

English verdict: APPROVE — 52bfcf643a9289e1f2c9acbcc7e9c41c2707d320 records definitive quota CAS rejection as auditable failed settlement and safely retries only remaining accounting. Independent real local quota/Turn base-head evidence and 91 focused tests pass; native Windows/dsh and hosted CI were not verified.

@huangruiteng
huangruiteng merged commit d56a963 into loopx-project:main Oct 15, 2026
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

turn run-once: quota_spend CAS failure leaves turn permanently in_progress (completed work unauditable)

3 participants