Skip to content

test(managed): assert the operator managed flow through one entry - #4507

Merged
huangruiteng merged 1 commit into
mainfrom
codex/managed-steward-product-lane
Sep 16, 2026
Merged

huangruiteng merged 1 commit into
mainfrom
codex/managed-steward-product-lane

Conversation

@huangruiteng

Copy link
Copy Markdown
Collaborator

Motivation

The managed contract is proven in pieces today: the executor binding, the
managed default flow, the steward channel and the segment's single-executor
refusal each have their own coverage. Nothing asserts the operator-facing
flow
itself — that one resolved credential drives the planned executor readback
and the channel readback to the same mode, executor and profile, and that the
delivery fails closed instead of launching when it cannot run.

Change

One public-safe smoke, examples/loopx-managed-turn-operator-flow-smoke.py,
which walks the flow on an injected runtime and an injected segment runner (no
live provider call, no repository write, no quota spend):

  1. an unconfigured delivery refuses with operator_credential_unconfigured
    rather than launching on something else;
  2. one stored credential makes the managed executor available and projects
    deepseek-v4-flash@high in the planned Turn readback;
  3. the steward channel readback resolves the same executor, reports
    executor_kind=managed, the same execution profile, available=true,
    operator_credential_source=machine_store, and an explicit session-mode
    source (unbound when no Session is bound) instead of an absent field;
  4. a profile the provider is known to reject fails closed with
    invalid_reasoning_effort;
  5. a second concurrent turn on one managed binding is refused with
    managed_host_chat_segment_in_flight.

Validation

  • python3 examples/loopx-managed-turn-operator-flow-smoke.py -> ok
  • loopx canary premerge --from-git-diff --git-diff-base origin/main --tier standard -> passed, self_merge_allowed: true

The managed contract is currently proven in pieces: the executor binding, the
default flow, the steward channel and the segment's single-executor refusal each
have their own coverage. What no artifact asserts is the operator-facing flow
itself -- that one resolved credential drives the planned executor readback and
the channel readback to the *same* mode, executor and profile, and that the
delivery fails closed instead of launching when it cannot run.

Add one public-safe smoke that walks that flow on an injected runtime and an
injected segment runner: refuse an unconfigured delivery with its typed reason,
project executor/kind/profile/availability/credential-source from one stored
credential, refuse an unsupported profile, and refuse a second concurrent
executor on one binding.

Signed-off-by: huangruiteng <14976749+huangruiteng@users.noreply.github.com>

@huangruiteng huangruiteng left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approval conclusion (author-owned PR; GitHub blocks formal self-approval)

审阅的精确 head:6c9c728e88ced9cec45355e478ac7b09c6625b0f

动机

managed 模式的契约目前是「分片证明」的:执行器回读有自己的测试,通道绑定有自己的测试,单执行器拒答在 tests/test_chat_dsh_adapter.py,续跑消费者在 tests/test_loopx_turn_managed_step.py。缺的是把操作者流程本身串起来的那一条断言——同一份解析出的凭据,是否让「计划中的 Turn 回读」和「管家通道回读」指向同一个执行器、同一个 profile,以及在跑不了的时候是否真的按类型拒答而不是换个东西启动。

这个缺口不是理论问题:如果两个回读各自漂移,四个分片套件会全部继续绿。

改动思路

不加生产代码,只加一个 public-safe smoke,用注入的托管运行时探测和段执行器走完整条流程(不发起真实 provider 调用、不写仓库状态、不花配额),断言五件事:

  1. 未配置凭据时按 operator_credential_unconfigured 拒答;
  2. 存下凭据后托管宿主可用,并投影 deepseek-v4-flash@high;
  3. 两个回读对执行器、executor_kind、profile、可用性一致,且通道点明 operator_credential_source=machine_store 与显式的 session_mode_source;
  4. provider 已知会拒绝的推理强度按 invalid_reasoning_effort fail-closed;
  5. 同一绑定上的第二个并发执行器按 managed_host_chat_segment_in_flight 被拒。

关键取舍:不把它塞进任何一个既有套件。这个性质是跨表面的,放进单表面文件会把跨表面不变量寄养在别人的所有权里;也不新增 fixture 或 helper——断言直接打生产函数。

具体改动

新增 examples/loopx-managed-turn-operator-flow-smoke.py(+181 行,唯一改动文件)。结构:一个临时 runtime root → operator_provider_environ 解析出唯一环境 → managed_executor_binding / manager_channel_binding / managed_execution_profile 三处回读互相对照 → DshChatAdapter 用一个阻塞 runner 占住单执行器槽位后验证第二次 start_turn 被拒。失败时抛 SystemExit 并打印出问题的载荷,所以坏掉的是哪个字段在失败输出里直接可见。

无生产模块、fixture、文档页或打包产物改动,因此没有运行时、API、schema 或用户可见变化,也不触碰 control-plane 的行数天花板。

对主干的风险

主要风险是变成同义反复:因为运行时与执行器都是注入的,如果断言只是复述注入值就会永远绿。这一点由「跨回读比较」本身挡住——通道的 execution_profile 是和同一次运行里 managed_executor_binding 的返回值比较的;拒答断言比较的是生产面产出的 typed code,而不是 smoke 自己给的字符串。写作过程中它确实先失败过一次(未绑定通道报的是 session_mode_source=unbound,而不是字段缺失),这也是「断言非空洞」的现场证据。

其余风险有限:纯测试文件,最坏情况是掩盖将来的回读漂移,不会改变运行时行为;删除即回滚。未覆盖的是真实 provider 握手与 HTTP 路由——本 smoke 不替代真实通道验证。仓库 main 目前是继承性的红灯(repo 级静态检查),因此本 head 的覆盖证据是本机 pre-merge 门禁加本 smoke。

验证:python3 examples/loopx-managed-turn-operator-flow-smoke.py → ok(5 项检查);loopx canary premerge --from-git-diff --git-diff-base origin/main --tier standard → passed,self_merge_allowed: true。

我的整体评价

正向、比例合适,建议合并。

它补的是真实的验证空洞而不是代码量:managed 模式的判断标准恰恰是「回读之间是否自洽」和「跑不了时是否拒答」,而这两点此前没有任何单一artifact 保证。选型也守住了边界——不加 fixture、不改生产、把跨表面不变量放在跨表面的文件里,并且用注入而不是真实调用换取可重复性。

去重判断我核过:rg 扫出的 5 个既有 managed smoke 各自只断言一个面,本文件断言的是两面之间的一致性加两条拒答路径,不构成重复;同日作者的批次扫描只有这一个 managed 相关 PR,也不是刷量。

无阻断性发现。

审阅的精确 head:6c9c728e88ced9cec45355e478ac7b09c6625b0f

English verdict: APPROVE - head 6c9c728 adds one public-safe smoke that walks the operator-facing managed flow through a single entry on an injected runtime and an injected segment runner: an unconfigured delivery refuses with operator_credential_unconfigured, a stored credential makes the managed host available and projects deepseek-v4-flash@high, the channel readback agrees with the planned Turn readback on executor, kind, profile and availability while naming its credential and session-mode sources, an unsupported reasoning effort fails closed, and a second concurrent executor is refused with managed_host_chat_segment_in_flight. It is test-only (+181, one file), adds no fixture or helper, asserts the cross-surface agreement that none of the five existing per-surface managed smokes own, and the same-author batch scan shows no PR farming. No blocking finding; residual risk is that the provider call and the HTTP route are substituted rather than live. Validation: smoke ok, and loopx canary premerge --from-git-diff --git-diff-base origin/main --tier standard passed with self_merge_allowed: true; GitHub CI is not used as evidence on this head because main is currently red on inherited static checks.

@huangruiteng
huangruiteng merged commit dd9a95c into main Sep 16, 2026
15 of 17 checks passed
@huangruiteng
huangruiteng deleted the codex/managed-steward-product-lane branch September 16, 2026 07:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant