Skip to content

feat(benchmark): add reviewed LHTB generic CLI heartbeat runner - #4504

Merged
huangruiteng merged 4 commits into
loopx-project:mainfrom
shangzh0:codex/lhtb-heartbeat-review-fixes
Sep 16, 2026
Merged

huangruiteng merged 4 commits into
loopx-project:mainfrom
shangzh0:codex/lhtb-heartbeat-review-fixes

Conversation

@shangzh0

Copy link
Copy Markdown
Contributor

Summary

  • add a reproducible LHTB 46-task treatment under benchmark/LHTB
  • run LoopX generic_cli heartbeats through one fresh codex exec --json per wake, without codex exec resume
  • persist continuation through the task workspace and trial-local LoopX registry, with replan_after_completed_todos=3
  • preserve LHTB task-defined internet and verifier policies while staging Codex, Python, Node, and LoopX without task-container downloads
  • make the external scheduler worker honor the producer-owned scheduler_hint.unchanged_poll.local_scheduler=stop directive before parsing omitted cadence detail
  • reuse the existing swe-marathon offline Codex adapter instead of maintaining an LHTB copy

This supersedes #4464 and starts from current upstream main at f4ed58de9e44ff58e1a3e46731c520309e486e2f.

Owner review addressed

The blocking and non-blocking findings from #4464 are included in this revision:

  1. Complete stop vocabulary without duplicating it. The worker no longer owns a TERMINAL_ACTIONS literal set. It derives terminality from the producer's unchanged_poll.local_scheduler=stop directive, so both stop_until_explicit_resume and return_to_owner_until_material_change terminate without requiring cold_path_detail.
  2. Durable public smoke coverage. The public external-worker smoke now removes cold-path detail, directly asserts terminal, after_limit, and unchanged_limit, and runs both producer stop actions end to end with no sleep and exit code 0.
  3. Focused unit decision coverage. The unit test is parameterized over both stop actions and asserts the decision shape before checking run_worker state and exit behavior.
  4. Semantic preflight. LHTB preflight imports the real worker and executes both stop packets instead of comparing source line positions.
  5. One offline Codex adapter owner. benchmark/LHTB/agents/codex_offline.py is removed; the runner explicitly imports benchmark/swe-marathon/agents/codex_offline.py, and preflight plus a real Harbor import probe verify that resolution.

Validation

  • python3 -m pytest -q tests/test_external_scheduler_worker.py (6 passed)
  • python3 examples/external-scheduler-worker-smoke.py (12 checks passed)
  • ruff check benchmark/LHTB scripts/external_scheduler_worker.py tests/test_external_scheduler_worker.py examples/external-scheduler-worker-smoke.py
  • python3 -m compileall -q benchmark/LHTB scripts/external_scheduler_worker.py tests/test_external_scheduler_worker.py examples/external-scheduler-worker-smoke.py
  • bash -n benchmark/LHTB/run.sh benchmark/LHTB/scripts/build_verifier_images.sh
  • real Harbor import probe confirmed LoopxHeartbeatCodex inherits the shared swe-marathon CodexOffline
  • benchmark/LHTB/run.sh preflight against the LHTB checkout: 46/46 tasks, 22 offline/24 online, 44 shared/2 separate verifier tasks, both scheduler stop actions, shared adapter, fresh-exec contract, Web Search disabled, and replan threshold readback all passed
  • loopx canary premerge --from-git-diff --git-diff-base origin/main --tier standard: direct checks passed; 10/10 catalog canaries passed; 8/8 risk-profile smokes passed; public-boundary scan passed; 0 failures; expected benchmark_sensitive manual-review hold retained

Boundaries

  • no benchmark run, trajectory, reward, verifier output, credential, or local path is included
  • no LHTB task definition, verifier scoring, network declaration, model setting, or LoopX default is changed
  • setup and preflight do not launch the full benchmark
  • this PR is intentionally left for maintainer review and will not be self-merged

Future-facing scope check

The related bounded refactor was applied: terminality now has one producer-owned directive instead of a second consumer action vocabulary, and offline Codex staging has one existing adapter owner. A wider scheduler schema migration or benchmark framework extraction was intentionally deferred because neither is required for this runner or the reported stop-loop defect.

Signed-off-by: shangzh0 <97216392+shangzh0@users.noreply.github.com>
Signed-off-by: shangzh0 <97216392+shangzh0@users.noreply.github.com>
Signed-off-by: shangzh0 <97216392+shangzh0@users.noreply.github.com>
Signed-off-by: shangzh0 <97216392+shangzh0@users.noreply.github.com>
@shangzh0

Copy link
Copy Markdown
Contributor Author

动机

复审 exact head 82ec4fd6e307d682f282b96170ad8f31a7a0ecea,并把它临时合并到当前 main@75fcd5556c3f8dd438365ba7f16fea7c955955ba 后验证。#4464 的三项 owner feedback 已实质解决:两个 producer stop action 都按 unchanged_poll.local_scheduler=stop 停止,公共 smoke 与单测覆盖了缺失 cold-path detail 的 terminal packet,LHTB 也复用了 swe-marathon 的离线 Codex adapter。下面是当前 whole-PR 仍存在的问题,按严重度排序。

改动思路

这条路径是 run.sh -> Harbor agent -> external_scheduler_worker -> run_capped_process(wake_once) -> fresh codex exec。LoopX registry 是控制状态权威,Harbor 是 phase/timeout/verifier 权威。最关键的不变量是:Harbor 认为 agent phase 返回之前,当前 wake 及其 Codex 后代必须已经退出;否则 verifier 与 agent 会并发访问同一工作区,最终分数不再对应一个冻结的提交状态。网络层的对应不变量是:preflight 验证的 Docker network 与 Compose 实际挂载的 network 必须是同一个,并且既有 Harbor patch 必须经过语义验证,不能只凭一个 marker 字符串视为兼容。

具体改动

P1(阻断)外层 scheduler timeout 不能保证 fresh wake/Codex 已终止

codex_loopx_heartbeat.py:531-565 用 GNU timeout 限制 worker 为 5080 秒,并把 rc=124 转成成功;但 worker 在 external_scheduler_worker.py:265-271 通过 run_capped_process 启动 wake。后者在 POSIX 上使用 start_new_session=True,因此 wake 脱离 worker 的进程组。

当 worker 已运行一段时间后再启动一个最长 4800 秒的 wake,5080 秒总 deadline 可能先到:外层 timeout 杀掉 worker,独立 session 中的 wake/Codex 仍可继续;shell 随后把 124 改成 0,Harbor 会进入 verifier。exact-head 最小实验证实了这一点:用 1 秒 GNU timeout 包住调用 run_capped_process([python, sleep], timeout_seconds=60) 的父进程,父进程返回 124 后,记录下来的子 Python PID 仍存在。这个结果也符合 review contract 的强制反例:“parent exit does not prove descendants drained”。

这会允许 agent 在 verifier 或下一 phase 开始后继续改工作区,并且 Harbor 的 tmux-descendant freeze 无法可靠捕获已经 reparent 的独立 session,属于评分完整性问题。最小修复是让 worker 自己拥有总 deadline,把每次 wake timeout 截断为剩余预算,并在任何 SIGTERM/异常路径中 drain/kill 当前 wake 的完整进程树;在确认后代退出前不能把外层 124 当作正常完成。回归测试应在临近总 deadline 时启动长 wake,并断言 worker 返回前其后代 PID 已消失。

P2 可配置 network 名称没有传入 Compose

.env.example:24 和 run.sh:45-102 允许设置 LHTB_MODELONLY_NETWORK,并创建/检查该名称;但两份 Compose overlay 仍硬编码 lhtb-modelonly(docker-compose-modelonly.yaml:4-8、docker-compose-modelonly-plus.yaml:5-9)。因此非默认名称会通过当前 network preflight,却在 trial 启动时挂载错误/不存在的 network。要么把 network 名称模板化并对最终 docker compose config 做断言,要么删除该可配置项并 fail closed 要求固定名称。

P2 Harbor patch 的兼容性只检查 marker,可能接受旧版或半安装状态

run.sh:86-91 只要 docker.py 出现 LHTB_MODELONLY_NET 就完全跳过 patch 安装;preflight.py:254-256 也只检查同一字符串。旧实验 patch、只改了 docker.py 但没装 __init__.py 常量/Compose 文件的半安装状态、或语义不同的实现都会被判定为 ready,实际网络与 PR 声明可能不同。建议为 patch 加版本化 sentinel/结构化 readback,并让 preflight import 实际常量、验证两份 overlay 存在,再检查最终 Compose 的 main networks;仅 marker 存在不能作为行为证据。

P3 smoke 可选择 separate-verifier 任务,但不会准备其本地镜像

run.sh:108-112 接受任意 46-task 名称作为 smoke;而 verifier image 构建只在 MODE=full 执行(run.sh:176-179)。在干净机器上 smoke langchain-version-migration 或 nbody-accel-iterative 会缺少 manifest 指定的 lhtb-local/*-verifier 镜像。应在 smoke 选中这两题时按需构建,或在 preflight 明确 fail closed 并给出准备命令。

对主干的风险

P1 会改变被评分工作区的时间边界,可能导致 verifier 与未终止 agent 并发,是当前不能合并的 blocker。两个 P2 会让同一份公开配置在不同宿主机状态下得到不同网络行为,破坏 runner 的可复现性;P3 影响干净环境的 smoke 入口。默认关闭方面,Harbor patch 仍由 LHTB_MODELONLY_NET opt-in,未发现 feature-off 路径被静默改变;authority semantics 方面,generic_cli、fresh exec 和 Harbor verifier 的职责命名与实现一致。scheduler 的 terminal directive 复用 producer vocabulary,语义与 CI 对齐,没有再引入第二套 stop action 列表。

验证结果:当前 main 临时合并无冲突;tests/test_external_scheduler_worker.py 6/6、公共 external-worker smoke 12/12、scheduler interaction/execution-context 227/227、terminal/quota tests 27/27、Ruff、compileall、bash -n 全部通过。GitHub 当前没有远程 CI check;本轮未运行真实模型的全量 LHTB 46-task trial。

我的整体评价

方向和 #4464 review 的修订是正确的,terminal packet 修复可以保留,LHTB 适配器的职责划分也基本清晰。但 exact head 尚不能视为评测安全:先修复并测试 descendant drain,再处理 network/patch 的真实 readback,才有足够证据证明“agent phase 已结束”和“preflight 所验证的网络就是实际运行网络”。

English verdict: REQUEST_CHANGES — exact head 82ec4fd6e307d682f282b96170ad8f31a7a0ecea. Blocking P1: the outer scheduler timeout can return success while a fresh wake/Codex process created in a separate session is still alive, allowing workspace mutation during verification. P2: configurable model-only network names are not propagated into the hard-coded Compose overlays. P2: Harbor patch compatibility is accepted from a marker string rather than semantic/readback validation. P3: smoke runs for either separate-verifier task do not prepare their required local verifier image. Focused tests pass after merging with current main, but there is no remote CI and no full model-backed LHTB run for this exact head.

@huangruiteng huangruiteng left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

动机

这个 PR 独立替代已经关闭的 #4464,做两件事:一是给 LHTB 46 任务补一个可复现的 generic_cli 心跳 treatment,二是修掉上一轮 review 在 scripts/external_scheduler_worker.py 上指出的真实缺陷。缺陷本身不难理解:worker 原来自己维护 TERMINAL_ACTIONS = {stop_until_explicit_resume},于是 producer 的另一种停止动作(peer 协调阻塞时的 return_to_owner_until_material_change)对 worker 不算终止;同时停止包会省略 cold_path_detail,而 worker 却先解析 cadence 再判断终止,于是"已经宣布结束的循环"要么继续被唤醒、要么直接报错。对跑长任务的操作者来说,这意味着配额被无意义地烧掉,而且每轮都是没有合法收尾的空转。

改动思路

作者没有往那个字面量集合里再加一个动作名——那只会把"谁定义停止"这件事留在消费侧,而是改成消费 producer 自己的指令:scheduler_hint.unchanged_poll.local_scheduler == stop 时立刻返回终止决策,cold_path_detail 不再参与判断。这一点我对着 producer 侧核过:loopx/control_plane/scheduler/scheduler_hint.py:249-256 的共享 stop-hint 构造器,以及 1257-1267 的 goal 停止/配额暂停路径,都会发出同一个指令,所以两个停止动作都真正终止。benchmark 侧则是新建 benchmark/LHTB/:一次唤醒一个全新的 codex exec --json(不使用 resume),续跑状态放在任务工作区和 trial 本地 registry;Codex/Python/Node/LoopX 都预先 staged,任务容器不下载;并且按上一轮要求复用 benchmark/swe-marathon/agents/codex_offline.py,不再自己留一份副本。

具体改动

  • scripts/external_scheduler_worker.py:删掉消费侧的动作集合,改成 producer 指令判定;停止包不再需要冷路径 cadence 细节。
  • benchmark/LHTB/:run.sh(环境、网络、preflight、Harbor 启动)、configs/heartbeat-generic-cli.yaml(46 任务模板)、agents/codex_loopx_heartbeat.py(Harbor 生命周期与 Goal 建立)、runtime/wake_once.py(每轮唯一 Turn + 全新 codex exec)、scripts/(preflight、render_config、summarize_results、build_verifier_images)、harbor_patch/(可选 model-only 网络补丁)、verifier-images/(两个任务自带 verifier 镜像)。
  • 验证面:tests/test_external_scheduler_worker.py 与 examples/external-scheduler-worker-smoke.py 被扩展,而不是新加一份并行的 smoke。

关键代码讲解

  • scripts/external_scheduler_worker.py:45 LOCAL_SCHEDULER_STOP_DIRECTIVE 与 :97 的短路返回:停止判定改为读 producer 指令,且在任何 cadence 解析之前返回 TickDecision(terminal=True, after_limit="stop_tick_loop")。我用独立探针喂了两个 producer 停止动作,带指令时都返回 terminal=True。
  • producer 侧对照:loopx/control_plane/scheduler/scheduler_hint.py:249-256(共享 stop-hint)与 1257-1267(goal_stopped/quota_paused)都发出 unchanged_poll.local_scheduler: stop,所以上一轮那个"只覆盖一个停止动作"的问题在这条链上是真的关掉了,不是靠再补一个字面量。
  • benchmark/LHTB/run.sh:49-51,136:PYTHONPATH="$CODE_DIR/agents:$SHARED_CODEX_AGENT_DIR:...",并在文件缺失时直接 die;scripts/preflight.py:159 再校验同一条路径。head 上 benchmark/LHTB/agents/ 只剩 codex_loopx_heartbeat.py,之前那份重复的 codex_offline.py 确实没有了。
  • benchmark/LHTB/runtime/wake_once.py:每个心跳唤醒一次全新 codex exec,续跑靠任务工作区与 trial 本地 registry,而不是 codex exec resume,这与 treatment 声明一致。

对主干的风险

共享面的改动只有 35 行,且没有引入新的 CLI 选项、协议字段或持久化状态。我跑到的证据:pytest -q tests/test_external_scheduler_worker.py 6 项通过;examples/external-scheduler-worker-smoke.py 12 项检查通过(含 test_end_to_end_terminal_stops_immediately,无需 sleep 即退出);独立探针还确认了反例——把指令从停止动作里拿掉时,worker 会以原有的 cold_path_detail ... missing 报错 fail closed,而不是静默继续轮询,这条耦合是安全的。公开边界上也干净:.env.example 用的是 /absolute/path/to/... 占位和 RFC 5737 的 192.0.2.x 文档网段,runs/、reports/、.env 都在 .gitignore 里保留 .gitkeep,没有凭据、原始轨迹或本地绝对路径进入提交。

一条非阻塞 P3 与一条残余风险:

  1. verifier 基础镜像只按可变 tag 固定(benchmark/LHTB/verifier-images/nbody/Dockerfile:1、.../langchain/Dockerfile:1)。两个镜像都继承自第三方个人命名空间的时间 tag(zli12321/lhtb-*:20260615),仓库与 preflight 都没有记录 digest。判定任务是否通过的是 verifier 环境,上游重新打 tag 就能在不改本仓库一行代码的情况下改变它。建议把解析出的 digest 写进 README 并在 scripts/preflight.py 里断言(或直接按 digest 固定),这样重建要么可复现、要么直接失败。
  2. 终止语义现在完全依赖 producer 发出指令。这是有意的单一权威,且已由扩展测试覆盖两个已发出的停止动作;但若将来新增第三种停止动作而忘了带指令,worker 会走 fail-closed 报错路径(不会静默空转)。保持"每个 producer 停止路径都带指令"的断言即可。

另外,本环境没有 LHTB checkout、Harbor 与任务容器,因此这个 treatment 的端到端运行没有复现,这一项在我的结论里保持 unverified;harness 自身的 preflight 会在缺前置条件时快速失败。

English verdict: APPROVE — exact head 82ec4fd6e307d682f282b96170ad8f31a7a0ecea of #4504. The shared-code change is 35 lines and it removes a duplicate authority instead of extending a literal: terminality now comes from the producer's scheduler_hint.unchanged_poll.local_scheduler = stop, and I verified on the head that both producer stop paths emit that directive (loopx/control_plane/scheduler/scheduler_hint.py:249-256 and 1257-1267), so the earlier review's "only one of two stop actions is terminal" blocker is genuinely closed. An independent probe confirms both stop actions return terminal=True with the directive and fail closed with the pre-existing cold-path error without it, so the coupling is safe rather than silent. pytest -q tests/test_external_scheduler_worker.py passes 6 tests and examples/external-scheduler-worker-smoke.py passes 12 checks, including an end-to-end terminal stop with no sleep. The superseded revision's duplicated benchmark/LHTB/agents/codex_offline.py copy is gone: only codex_loopx_heartbeat.py remains, and reuse is guarded by run.sh:49-51,136 (PYTHONPATH plus fail-fast) and scripts/preflight.py:159. The public boundary is clean (placeholder /absolute/path/to and RFC 5737 192.0.2.x values only; runs/, reports/ and .env are gitignored). One non-blocking P3: the two verifier images inherit third-party date tags with no recorded digest, so the scoring-relevant verifier base can move without a repository change — record the digests in the README and assert them in preflight. Residual risk: terminality now depends entirely on the producer emitting the directive, which the extended tests cover for both emitted stop actions. No product runtime, quota, todo, authority or persistence behaviour changes, and no live LHTB task run was executed here.

@huangruiteng huangruiteng left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

动机

这个 PR 独立替代已经关闭的 #4464,做两件事:一是给 LHTB 46 任务补一个可复现的 generic_cli 心跳 treatment,二是修掉上一轮 review 在 scripts/external_scheduler_worker.py 上指出的真实缺陷。缺陷本身不难理解:worker 原来自己维护 TERMINAL_ACTIONS = {stop_until_explicit_resume},于是 producer 的另一种停止动作(peer 协调阻塞时的 return_to_owner_until_material_change)对 worker 不算终止;同时停止包会省略 cold_path_detail,而 worker 却先解析 cadence 再判断终止,于是"已经宣布结束的循环"要么继续被唤醒、要么直接报错。对跑长任务的操作者来说,这意味着配额被无意义地烧掉,而且每轮都是没有合法收尾的空转。

改动思路

作者没有往那个字面量集合里再加一个动作名——那只会把"谁定义停止"这件事留在消费侧,而是改成消费 producer 自己的指令:scheduler_hint.unchanged_poll.local_scheduler == stop 时立刻返回终止决策,cold_path_detail 不再参与判断。这一点我对着 producer 侧核过:loopx/control_plane/scheduler/scheduler_hint.py:249-256 的共享 stop-hint 构造器,以及 1257-1267 的 goal 停止/配额暂停路径,都会发出同一个指令,所以两个停止动作都真正终止。benchmark 侧则是新建 benchmark/LHTB/:一次唤醒一个全新的 codex exec --json(不使用 resume),续跑状态放在任务工作区和 trial 本地 registry;Codex/Python/Node/LoopX 都预先 staged,任务容器不下载;并且按上一轮要求复用 benchmark/swe-marathon/agents/codex_offline.py,不再自己留一份副本。

具体改动

  • scripts/external_scheduler_worker.py:删掉消费侧的动作集合,改成 producer 指令判定;停止包不再需要冷路径 cadence 细节。
  • benchmark/LHTB/:run.sh(环境、网络、preflight、Harbor 启动)、configs/heartbeat-generic-cli.yaml(46 任务模板)、agents/codex_loopx_heartbeat.py(Harbor 生命周期与 Goal 建立)、runtime/wake_once.py(每轮唯一 Turn + 全新 codex exec)、scripts/(preflight、render_config、summarize_results、build_verifier_images)、harbor_patch/(可选 model-only 网络补丁)、verifier-images/(两个任务自带 verifier 镜像)。
  • 验证面:tests/test_external_scheduler_worker.py 与 examples/external-scheduler-worker-smoke.py 被扩展,而不是新加一份并行的 smoke。

关键代码讲解

  • scripts/external_scheduler_worker.py:45 LOCAL_SCHEDULER_STOP_DIRECTIVE 与 :97 的短路返回:停止判定改为读 producer 指令,且在任何 cadence 解析之前返回 TickDecision(terminal=True, after_limit="stop_tick_loop")。我用独立探针喂了两个 producer 停止动作,带指令时都返回 terminal=True。
  • producer 侧对照:loopx/control_plane/scheduler/scheduler_hint.py:249-256(共享 stop-hint)与 1257-1267(goal_stopped/quota_paused)都发出 unchanged_poll.local_scheduler: stop,所以上一轮那个"只覆盖一个停止动作"的问题在这条链上是真的关掉了,不是靠再补一个字面量。
  • benchmark/LHTB/run.sh:49-51,136:PYTHONPATH="$CODE_DIR/agents:$SHARED_CODEX_AGENT_DIR:...",并在文件缺失时直接 die;scripts/preflight.py:159 再校验同一条路径。head 上 benchmark/LHTB/agents/ 只剩 codex_loopx_heartbeat.py,之前那份重复的 codex_offline.py 确实没有了。
  • benchmark/LHTB/runtime/wake_once.py:每个心跳唤醒一次全新 codex exec,续跑靠任务工作区与 trial 本地 registry,而不是 codex exec resume,这与 treatment 声明一致。

对主干的风险

共享面的改动只有 35 行,且没有引入新的 CLI 选项、协议字段或持久化状态。我跑到的证据:pytest -q tests/test_external_scheduler_worker.py 6 项通过;examples/external-scheduler-worker-smoke.py 12 项检查通过(含 test_end_to_end_terminal_stops_immediately,无需 sleep 即退出);独立探针还确认了反例——把指令从停止动作里拿掉时,worker 会以原有的 cold_path_detail ... missing 报错 fail closed,而不是静默继续轮询,这条耦合是安全的。公开边界上也干净:.env.example 用的是 /absolute/path/to/... 占位和 RFC 5737 的 192.0.2.x 文档网段,runs/、reports/、.env 都在 .gitignore 里保留 .gitkeep,没有凭据、原始轨迹或本地绝对路径进入提交。

一条非阻塞 P3 与一条残余风险:

  1. verifier 基础镜像只按可变 tag 固定(benchmark/LHTB/verifier-images/nbody/Dockerfile:1、.../langchain/Dockerfile:1)。两个镜像都继承自第三方个人命名空间的时间 tag(zli12321/lhtb-*:20260615),仓库与 preflight 都没有记录 digest。判定任务是否通过的是 verifier 环境,上游重新打 tag 就能在不改本仓库一行代码的情况下改变它。建议把解析出的 digest 写进 README 并在 scripts/preflight.py 里断言(或直接按 digest 固定),这样重建要么可复现、要么直接失败。
  2. 终止语义现在完全依赖 producer 发出指令。这是有意的单一权威,且已由扩展测试覆盖两个已发出的停止动作;但若将来新增第三种停止动作而忘了带指令,worker 会走 fail-closed 报错路径(不会静默空转)。保持"每个 producer 停止路径都带指令"的断言即可。

另外,本环境没有 LHTB checkout、Harbor 与任务容器,因此这个 treatment 的端到端运行没有复现,这一项在我的结论里保持 unverified;harness 自身的 preflight 会在缺前置条件时快速失败。

English verdict: APPROVE — exact head 82ec4fd6e307d682f282b96170ad8f31a7a0ecea of #4504. The shared-code change is 35 lines and it removes a duplicate authority instead of extending a literal: terminality now comes from the producer's scheduler_hint.unchanged_poll.local_scheduler = stop, and I verified on the head that both producer stop paths emit that directive (loopx/control_plane/scheduler/scheduler_hint.py:249-256 and 1257-1267), so the earlier review's "only one of two stop actions is terminal" blocker is genuinely closed. An independent probe confirms both stop actions return terminal=True with the directive and fail closed with the pre-existing cold-path error without it, so the coupling is safe rather than silent. pytest -q tests/test_external_scheduler_worker.py passes 6 tests and examples/external-scheduler-worker-smoke.py passes 12 checks, including an end-to-end terminal stop with no sleep. The superseded revision's duplicated benchmark/LHTB/agents/codex_offline.py copy is gone: only codex_loopx_heartbeat.py remains, and reuse is guarded by run.sh:49-51,136 (PYTHONPATH plus fail-fast) and scripts/preflight.py:159. The public boundary is clean (placeholder /absolute/path/to and RFC 5737 192.0.2.x values only; runs/, reports/ and .env are gitignored). One non-blocking P3: the two verifier images inherit third-party date tags with no recorded digest, so the scoring-relevant verifier base can move without a repository change — record the digests in the README and assert them in preflight. Residual risk: terminality now depends entirely on the producer emitting the directive, which the extended tests cover for both emitted stop actions. No product runtime, quota, todo, authority or persistence behaviour changes, and no live LHTB task run was executed here.

我的整体评价

整体 APPROVE。这个 PR 的价值不在体量,而在于把"谁定义停止"收敛回 producer:worker 删掉自己那份动作字面量、改为消费 unchanged_poll.local_scheduler=stop,我在 head 上核到 producer 的两条停止路径都发出该指令,所以上一轮 review 的阻塞项是真的关掉,而不是绕过去;指令缺失时 worker 会 fail closed 报错,而不是静默继续空转。benchmark 侧是本 PR 的主交付,位置正确(研究工件区、opt-in、不参与 CI 执行),并按上一轮要求删掉重复的 offline adapter,改用 PYTHONPATH 复用加 preflight 校验。公开边界干净,.env.example 只有占位符与 RFC 5737 文档网段,运行产物目录都被 gitignore。唯一 P3 是 verifier 基础镜像只固定可变 tag、没有 digest 记录,而 verifier 环境决定任务是否通过,建议按 digest 固定或在 preflight 断言;本环境没有 LHTB checkout 与 Harbor,treatment 端到端运行保持 unverified。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants