test(managed): assert the operator managed flow through one entry - #4507
Conversation
The managed contract is currently proven in pieces: the executor binding, the default flow, the steward channel and the segment's single-executor refusal each have their own coverage. What no artifact asserts is the operator-facing flow itself -- that one resolved credential drives the planned executor readback and the channel readback to the *same* mode, executor and profile, and that the delivery fails closed instead of launching when it cannot run. Add one public-safe smoke that walks that flow on an injected runtime and an injected segment runner: refuse an unconfigured delivery with its typed reason, project executor/kind/profile/availability/credential-source from one stored credential, refuse an unsupported profile, and refuse a second concurrent executor on one binding. Signed-off-by: huangruiteng <14976749+huangruiteng@users.noreply.github.com>
huangruiteng
left a comment
There was a problem hiding this comment.
Approval conclusion (author-owned PR; GitHub blocks formal self-approval)
审阅的精确 head:6c9c728e88ced9cec45355e478ac7b09c6625b0f
动机
managed 模式的契约目前是「分片证明」的:执行器回读有自己的测试,通道绑定有自己的测试,单执行器拒答在 tests/test_chat_dsh_adapter.py,续跑消费者在 tests/test_loopx_turn_managed_step.py。缺的是把操作者流程本身串起来的那一条断言——同一份解析出的凭据,是否让「计划中的 Turn 回读」和「管家通道回读」指向同一个执行器、同一个 profile,以及在跑不了的时候是否真的按类型拒答而不是换个东西启动。
这个缺口不是理论问题:如果两个回读各自漂移,四个分片套件会全部继续绿。
改动思路
不加生产代码,只加一个 public-safe smoke,用注入的托管运行时探测和段执行器走完整条流程(不发起真实 provider 调用、不写仓库状态、不花配额),断言五件事:
- 未配置凭据时按
operator_credential_unconfigured拒答; - 存下凭据后托管宿主可用,并投影
deepseek-v4-flash@high; - 两个回读对执行器、
executor_kind、profile、可用性一致,且通道点明operator_credential_source=machine_store与显式的session_mode_source; - provider 已知会拒绝的推理强度按
invalid_reasoning_effortfail-closed; - 同一绑定上的第二个并发执行器按
managed_host_chat_segment_in_flight被拒。
关键取舍:不把它塞进任何一个既有套件。这个性质是跨表面的,放进单表面文件会把跨表面不变量寄养在别人的所有权里;也不新增 fixture 或 helper——断言直接打生产函数。
具体改动
新增 examples/loopx-managed-turn-operator-flow-smoke.py(+181 行,唯一改动文件)。结构:一个临时 runtime root → operator_provider_environ 解析出唯一环境 → managed_executor_binding / manager_channel_binding / managed_execution_profile 三处回读互相对照 → DshChatAdapter 用一个阻塞 runner 占住单执行器槽位后验证第二次 start_turn 被拒。失败时抛 SystemExit 并打印出问题的载荷,所以坏掉的是哪个字段在失败输出里直接可见。
无生产模块、fixture、文档页或打包产物改动,因此没有运行时、API、schema 或用户可见变化,也不触碰 control-plane 的行数天花板。
对主干的风险
主要风险是变成同义反复:因为运行时与执行器都是注入的,如果断言只是复述注入值就会永远绿。这一点由「跨回读比较」本身挡住——通道的 execution_profile 是和同一次运行里 managed_executor_binding 的返回值比较的;拒答断言比较的是生产面产出的 typed code,而不是 smoke 自己给的字符串。写作过程中它确实先失败过一次(未绑定通道报的是 session_mode_source=unbound,而不是字段缺失),这也是「断言非空洞」的现场证据。
其余风险有限:纯测试文件,最坏情况是掩盖将来的回读漂移,不会改变运行时行为;删除即回滚。未覆盖的是真实 provider 握手与 HTTP 路由——本 smoke 不替代真实通道验证。仓库 main 目前是继承性的红灯(repo 级静态检查),因此本 head 的覆盖证据是本机 pre-merge 门禁加本 smoke。
验证:python3 examples/loopx-managed-turn-operator-flow-smoke.py → ok(5 项检查);loopx canary premerge --from-git-diff --git-diff-base origin/main --tier standard → passed,self_merge_allowed: true。
我的整体评价
正向、比例合适,建议合并。
它补的是真实的验证空洞而不是代码量:managed 模式的判断标准恰恰是「回读之间是否自洽」和「跑不了时是否拒答」,而这两点此前没有任何单一artifact 保证。选型也守住了边界——不加 fixture、不改生产、把跨表面不变量放在跨表面的文件里,并且用注入而不是真实调用换取可重复性。
去重判断我核过:rg 扫出的 5 个既有 managed smoke 各自只断言一个面,本文件断言的是两面之间的一致性加两条拒答路径,不构成重复;同日作者的批次扫描只有这一个 managed 相关 PR,也不是刷量。
无阻断性发现。
审阅的精确 head:6c9c728e88ced9cec45355e478ac7b09c6625b0f
English verdict: APPROVE - head 6c9c728 adds one public-safe smoke that walks the operator-facing managed flow through a single entry on an injected runtime and an injected segment runner: an unconfigured delivery refuses with operator_credential_unconfigured, a stored credential makes the managed host available and projects deepseek-v4-flash@high, the channel readback agrees with the planned Turn readback on executor, kind, profile and availability while naming its credential and session-mode sources, an unsupported reasoning effort fails closed, and a second concurrent executor is refused with managed_host_chat_segment_in_flight. It is test-only (+181, one file), adds no fixture or helper, asserts the cross-surface agreement that none of the five existing per-surface managed smokes own, and the same-author batch scan shows no PR farming. No blocking finding; residual risk is that the provider call and the HTTP route are substituted rather than live. Validation: smoke ok, and loopx canary premerge --from-git-diff --git-diff-base origin/main --tier standard passed with self_merge_allowed: true; GitHub CI is not used as evidence on this head because main is currently red on inherited static checks.
Motivation
The managed contract is proven in pieces today: the executor binding, the
managed default flow, the steward channel and the segment's single-executor
refusal each have their own coverage. Nothing asserts the operator-facing
flow itself — that one resolved credential drives the planned executor readback
and the channel readback to the same mode, executor and profile, and that the
delivery fails closed instead of launching when it cannot run.
Change
One public-safe smoke,
examples/loopx-managed-turn-operator-flow-smoke.py,which walks the flow on an injected runtime and an injected segment runner (no
live provider call, no repository write, no quota spend):
operator_credential_unconfiguredrather than launching on something else;
deepseek-v4-flash@highin the planned Turn readback;executor_kind=managed, the same execution profile,available=true,operator_credential_source=machine_store, and an explicit session-modesource (
unboundwhen no Session is bound) instead of an absent field;invalid_reasoning_effort;managed_host_chat_segment_in_flight.Validation
python3 examples/loopx-managed-turn-operator-flow-smoke.py-> okloopx canary premerge --from-git-diff --git-diff-base origin/main --tier standard-> passed,self_merge_allowed: true