fix(release): bound vision closeout and runtime responses under load - #5025
Conversation
Signed-off-by: huangruiteng <14976749+huangruiteng@users.noreply.github.com>
…diagnostics Signed-off-by: huangruiteng <14976749+huangruiteng@users.noreply.github.com>
Signed-off-by: huangruiteng <14976749+huangruiteng@users.noreply.github.com>
huangruiteng
left a comment
There was a problem hiding this comment.
Approval conclusion (author-owned PR; GitHub blocks formal self-approval)
PR #5025,精确 head 785cf0615ee81c3646c60c954a5faa0e45ff5e3a。无阻断性发现。
动机
v1.2.0 的原始完整资格验收有 21 个场景,其中 required-vision 的一次真实重复在旧 32 次工具调用上限前未完成写回与结算;Goal 级预合并验证又两次在不同的 TS 投影上遇到默认 5 秒响应超时。旧版因此既不能通过发布门槛,也会在负载下中断正常的监控写回。这个 PR 修复两个已复现的有界等待点,并让失败具体落到哪一次工具重复变得可读;整版发布仍要在合并后的最终 main 单独验收。
改动思路
保留当前三个既有所有者:真实模型闭环的调用预算由 required-vision actor 持有,场景通过与否由现有 portfolio oracle 判断,TypeScript 负责投影和效果而 Python bridge 只负责通信等待。三次独立真实闭环分别用了 16、25、32 次调用,最后一次刚好触及旧上限,因此仅将完整闭环提高到 40 次,窄范围单动作仍为 7 次。两个不同投影都在同一 5 秒默认预算下失败,故在共享传输处改成 10 秒,没有给每个调用者复制规则,也没有添加发送后的自动重试。
具体改动
loopx/control_plane/effect_runtime.py 为 request/result 统一使用 10 秒默认响应预算;显式传入的超时继续优先,失去响应后的不确定状态仍向调用者抛出。loopx/control_plane/testing/replan_vision_closeout_behavior.py 将完整闭环的工具上限改为 40,同时保持“证据、修订、checkpoint、一次 spend、下一轮读回”验收链。loopx/control_plane/testing/actual_default_model_behavior_portfolio.py 从现有 actor receipt 派生每次重复的通过状态、调用数/上限与受限错误码统计,拒绝输出原始命令、路径和 provider 内容;这只是诊断投影,不参与通过判定。三个测试文件分别覆盖传输默认值与不重放、预算边缘的正反例、诊断脱敏;英文和中文开发指南同步写明 40/7 次及当前 21 场景、42 次尝试。
关键代码讲解
effect_runtime_request()/effect_runtime_result()共享DEFAULT_REQUEST_TIMEOUT_SECONDS。改变的是等待窗口,EffectRuntimeResponseAmbiguous的发送后禁止重放分支没有变化。REQUIRED_VISION_CLOSEOUT_MAX_CALLS只供完整 required-vision actor 使用,独立于七次窄项;test_budget_boundary_still_requires_the_final_spend()证明第 40 次若没有最终结算仍失败。_tool_repeat_diagnostic()与_scenario_result()在已有每次 actor receipt 上汇总有限个、格式受控的字段;非工具场景不新增该字段,真正验收仍由_receipt_alignment()与既有 hard invariants 完成。
对主干的风险
主要代价是完整闭环最多多 8 次付费工具调用,真正失联的默认 runtime 请求最多晚 5 秒报错。更大的风险是预算变大后掩盖未结算的假成功,或超时后重复执行效果:负例仍要求第 40 次缺 spend 判失败,传输集成测试仍验证发送后响应不明不重放。旧 main 的真实组合收据保留为 20/21 且失败;新 head 用相同 21 场景、每项两次的真实 Doubao 门槛得到 21/21、全部对照通过。最终 Goal canary 为 14 项选中检查、5 项直接检查,零失败、零人工 hold;112 项相关测试、Ruff、mypy、七文件公开边界扫描与精确质量收据也通过。Goal 的 PR 审阅设置不等待 CI,所以没有用 PR CI 代替这些本地证据。
语义与 CI 对齐
改变的是现有响应等待和发布资格预算,没有新增 Goal 权限或语义状态。英文、中文指南直接替换旧 32 次和已过期的场景数量;历史失败收据没有被重标为通过。正式 release 仍必须针对合并后的干净提交重跑八项精确提交准入。
我的整体评价
对长期工作进展,真实闭环与监控写回从已证实的中断恢复为可完成路径;对使用体验,失败诊断更具体,但接受上述有界时间和费用代价。两个预算都留在原所有者,新增诊断是派生数据,七文件改动与问题影响相称。正反路径、真实模型、实际 runtime 序列和隐私边界均有精确 head 证据,支持 APPROVE;合并授权来自用户明确的 bypass 指示,仍以 exact-head readiness 为准。
English verdict: APPROVE - exact head 785cf06; real two-repeat portfolio passes 21/21 with all contrasts, goal canary 14/14 and focused/static/boundary/quality checks pass; final exact-main release qualification remains mandatory.
Problem and result
The actual-default release portfolio failed one of two required-vision closeout repeats when its 32-call ceiling expired before durable writeback and one-spend settlement. Three isolated real-model diagnostics completed in 16, 25, and 32 calls. The full closeout ceiling is now 40; the narrower single-action qualifier remains at 7, and semantic acceptance, evidence, and settlement rules are unchanged.
The bilingual testing guide now reflects the current 21-scenario/42-attempt portfolio and the 40-call full-closeout limit. The portfolio also exposes bounded, public-safe per-repeat diagnostics (call count/limit and allowlisted error-code counts), so an individual failed repeat can be identified without storing commands, prompts, provider responses, or private paths. The original failed release receipt remains failed. A fresh two-repeat portfolio must qualify the final release commit.
During goal-scoped premerge validation, the monitor writeback smoke twice hit the shared Effect runtime's 5-second response budget on different typed projections (
todo.succession.projectandscheduler.monitor_target.select); the same smoke passed alone. The shared default response budget is now 10 seconds, while explicit caller budgets and the no-replay-after-send rule remain unchanged. This can delay reporting a genuinely unresponsive runtime by up to five seconds. The complete goal-scoped canary, including the previously failing smoke in sequence, now passes.Validation
cqr_f3fa587935105c1397b6, verified valid.The additional eight model calls are worst-case paid headroom only for full closeout; the portfolio still requires two independent real repeats. The related refactor pass kept one shared transport timeout instead of duplicating per-caller budgets or adding replay logic.