Skip to content

fix(release): bound vision closeout and runtime responses under load - #5025

Merged
huangruiteng merged 3 commits into
mainfrom
codex/release-vision-closeout-budget
Sep 25, 2026
Merged

huangruiteng merged 3 commits into
mainfrom
codex/release-vision-closeout-budget

Conversation

@huangruiteng

@huangruiteng huangruiteng commented Sep 25, 2026 •

Copy link
Copy Markdown
Collaborator

Problem and result

The actual-default release portfolio failed one of two required-vision closeout repeats when its 32-call ceiling expired before durable writeback and one-spend settlement. Three isolated real-model diagnostics completed in 16, 25, and 32 calls. The full closeout ceiling is now 40; the narrower single-action qualifier remains at 7, and semantic acceptance, evidence, and settlement rules are unchanged.

The bilingual testing guide now reflects the current 21-scenario/42-attempt portfolio and the 40-call full-closeout limit. The portfolio also exposes bounded, public-safe per-repeat diagnostics (call count/limit and allowlisted error-code counts), so an individual failed repeat can be identified without storing commands, prompts, provider responses, or private paths. The original failed release receipt remains failed. A fresh two-repeat portfolio must qualify the final release commit.

During goal-scoped premerge validation, the monitor writeback smoke twice hit the shared Effect runtime's 5-second response budget on different typed projections (todo.succession.project and scheduler.monitor_target.select); the same smoke passed alone. The shared default response budget is now 10 seconds, while explicit caller budgets and the no-replay-after-send rule remain unchanged. This can delay reporting a genuinely unresponsive runtime by up to five seconds. The complete goal-scoped canary, including the previously failing smoke in sequence, now passes.

Validation

  • 112 runtime integration, restart, required-vision, and actual-default portfolio tests passed, including response ambiguity and final-spend-at-limit negative cases.
  • Ruff on all six changed Python paths and repository-configured mypy (23 source files) passed.
  • Public/private boundary scan: all seven changed paths, zero violations.
  • Goal-scoped premerge canary: 14 selected checks (including the public-boundary lane), five direct checks, zero failures or manual holds. The monitor writeback smoke passed in the original canary sequence (70.6 seconds).
  • Exact committed diff quality receipt: cqr_f3fa587935105c1397b6, verified valid.

The additional eight model calls are worst-case paid headroom only for full closeout; the portfolio still requires two independent real repeats. The related refactor pass kept one shared transport timeout instead of duplicating per-caller budgets or adding replay logic.

Signed-off-by: huangruiteng <14976749+huangruiteng@users.noreply.github.com>
…diagnostics

Signed-off-by: huangruiteng <14976749+huangruiteng@users.noreply.github.com>
Signed-off-by: huangruiteng <14976749+huangruiteng@users.noreply.github.com>

@huangruiteng huangruiteng left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approval conclusion (author-owned PR; GitHub blocks formal self-approval)

PR #5025,精确 head 785cf0615ee81c3646c60c954a5faa0e45ff5e3a。无阻断性发现。

动机

v1.2.0 的原始完整资格验收有 21 个场景,其中 required-vision 的一次真实重复在旧 32 次工具调用上限前未完成写回与结算;Goal 级预合并验证又两次在不同的 TS 投影上遇到默认 5 秒响应超时。旧版因此既不能通过发布门槛,也会在负载下中断正常的监控写回。这个 PR 修复两个已复现的有界等待点,并让失败具体落到哪一次工具重复变得可读;整版发布仍要在合并后的最终 main 单独验收。

改动思路

保留当前三个既有所有者:真实模型闭环的调用预算由 required-vision actor 持有,场景通过与否由现有 portfolio oracle 判断,TypeScript 负责投影和效果而 Python bridge 只负责通信等待。三次独立真实闭环分别用了 16、25、32 次调用,最后一次刚好触及旧上限,因此仅将完整闭环提高到 40 次,窄范围单动作仍为 7 次。两个不同投影都在同一 5 秒默认预算下失败,故在共享传输处改成 10 秒,没有给每个调用者复制规则,也没有添加发送后的自动重试。

具体改动

loopx/control_plane/effect_runtime.py 为 request/result 统一使用 10 秒默认响应预算;显式传入的超时继续优先,失去响应后的不确定状态仍向调用者抛出。loopx/control_plane/testing/replan_vision_closeout_behavior.py 将完整闭环的工具上限改为 40,同时保持“证据、修订、checkpoint、一次 spend、下一轮读回”验收链。loopx/control_plane/testing/actual_default_model_behavior_portfolio.py 从现有 actor receipt 派生每次重复的通过状态、调用数/上限与受限错误码统计,拒绝输出原始命令、路径和 provider 内容;这只是诊断投影,不参与通过判定。三个测试文件分别覆盖传输默认值与不重放、预算边缘的正反例、诊断脱敏;英文和中文开发指南同步写明 40/7 次及当前 21 场景、42 次尝试。

关键代码讲解

  • effect_runtime_request() / effect_runtime_result() 共享 DEFAULT_REQUEST_TIMEOUT_SECONDS。改变的是等待窗口,EffectRuntimeResponseAmbiguous 的发送后禁止重放分支没有变化。
  • REQUIRED_VISION_CLOSEOUT_MAX_CALLS 只供完整 required-vision actor 使用,独立于七次窄项;test_budget_boundary_still_requires_the_final_spend() 证明第 40 次若没有最终结算仍失败。
  • _tool_repeat_diagnostic() 与 _scenario_result() 在已有每次 actor receipt 上汇总有限个、格式受控的字段;非工具场景不新增该字段,真正验收仍由 _receipt_alignment() 与既有 hard invariants 完成。

对主干的风险

主要代价是完整闭环最多多 8 次付费工具调用,真正失联的默认 runtime 请求最多晚 5 秒报错。更大的风险是预算变大后掩盖未结算的假成功,或超时后重复执行效果:负例仍要求第 40 次缺 spend 判失败,传输集成测试仍验证发送后响应不明不重放。旧 main 的真实组合收据保留为 20/21 且失败;新 head 用相同 21 场景、每项两次的真实 Doubao 门槛得到 21/21、全部对照通过。最终 Goal canary 为 14 项选中检查、5 项直接检查,零失败、零人工 hold;112 项相关测试、Ruff、mypy、七文件公开边界扫描与精确质量收据也通过。Goal 的 PR 审阅设置不等待 CI,所以没有用 PR CI 代替这些本地证据。

语义与 CI 对齐

改变的是现有响应等待和发布资格预算,没有新增 Goal 权限或语义状态。英文、中文指南直接替换旧 32 次和已过期的场景数量;历史失败收据没有被重标为通过。正式 release 仍必须针对合并后的干净提交重跑八项精确提交准入。

我的整体评价

对长期工作进展,真实闭环与监控写回从已证实的中断恢复为可完成路径;对使用体验,失败诊断更具体,但接受上述有界时间和费用代价。两个预算都留在原所有者,新增诊断是派生数据,七文件改动与问题影响相称。正反路径、真实模型、实际 runtime 序列和隐私边界均有精确 head 证据,支持 APPROVE;合并授权来自用户明确的 bypass 指示,仍以 exact-head readiness 为准。

English verdict: APPROVE - exact head 785cf06; real two-repeat portfolio passes 21/21 with all contrasts, goal canary 14/14 and focused/static/boundary/quality checks pass; final exact-main release qualification remains mandatory.

@huangruiteng
huangruiteng merged commit 5a4617e into main Sep 25, 2026
24 of 25 checks passed
@huangruiteng
huangruiteng deleted the codex/release-vision-closeout-budget branch September 25, 2026 01:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant