Skip to content

test(heartbeat): right-size the shared prompt JSON envelope budget - #4255

Merged
huangruiteng merged 1 commit into
mainfrom
codex/heartbeat-envelope-budget
Sep 12, 2026
Merged

huangruiteng merged 1 commit into
mainfrom
codex/heartbeat-envelope-budget

Conversation

@huangruiteng

Copy link
Copy Markdown
Collaborator

Summary

  • Raise only the compact heartbeat JSON qualification envelope from 3,600 to 4,800 characters. Keep the thin body cap at 2,500, native Goal body at 4,000, structural counts and real CLI ceilings unchanged.
  • Cover representative agent/scope-bound automation payloads, plus exact-limit/one-over-limit and independent body-overflow negatives.
  • Clarify why envelope size is not prompt text size, token usage, execution quota or permission to grow static instructions.

Issue Or Task

Owner-requested assessment of recent heartbeat growth. #4201 increased the thin body allowance from 1,900 to 2,500 for the shared host contract, including safety and retry-stable Turn setup. The older JSON envelope ceiling remained 3,600. Current unbound / agent-bound / scoped fixtures measure 3,825 / 4,180 / 4,362 characters; scoped normalized body is only 1,894. A 4,800 envelope leaves approximately 10% headroom for the scoped fixture. Increasing the body cap again is not justified by these measurements.

Validation

  • Tested revision: e1581af (same final diff tested before commit)
  • Run state: finished
  • Input classes: synthetic, public_fixture
Check kind Result Public-safe evidence / limitation
unit passed pytest -q tests/control_plane/test_heartbeat_prompt_support.py: 28 passed; envelope 4,800 accepted / 4,801 rejected, unchanged body caps independently enforced.
real_entrypoint passed pytest -q tests/control_plane/test_cli_output_budget.py tests/control_plane/test_cli_output_differential.py: 67 passed, including real CLI output/mode qualification.
real_entrypoint passed pytest -q tests/control_plane/test_host_bootstrap_lifecycle.py tests/control_plane/test_goal_prompt_dispatch.py tests/control_plane/test_automation_prompt_upgrade.py: 61 passed.
regression_parity passed python examples/control_plane/hot-path-interface-budget-smoke.py and python examples/control_plane/heartbeat-prompt-smoke.py passed. Old envelope reproduces 3,825 > 3,600; candidate admits representative scoped payloads while overflow negatives remain enforced.
static passed git diff --check; explicit path classification and public/private scan.
  • Coverage and gaps: 156 tests plus two smokes. No production code, prompt text, scheduler, quota, saved automation or installation changes; therefore no live model runs or active Goal mutations are needed. No frontend companion change is needed: this updates a qualification ceiling, not an exposed configuration. Arbitrarily long custom paths/scopes are not guaranteed by the compact fixture budget; emitted CLI qualification remains separate. Hosted CI is pending.

Type of Change

  • Documentation update
  • Test update

LoopX Area

  • Build, packaging, installer, or CI
  • Host or runtime integration

Technical Direction

  • Core control-plane hardening
  • Target base branch: main
  • Direction tracker or promotion unit: heartbeat interface qualification; no promotion claim
  • Future-facing pass: extend the existing smoke to bound agent/scope envelopes rather than introducing another budget system or changing runtime serialization.

Shared-authority RFC fixture impact

N/A: no state, provider, projection authority or migration changes.

Boundary Checklist

  • No private state, credentials, raw traces, internal links or local machine paths in the diff or this body.
  • No duplicate benchmark work.
  • Scoped to heartbeat qualification.
  • DCO sign-off included.

Signed-off-by: huangruiteng <huangrt01@163.com>

@huangruiteng huangruiteng left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approval conclusion (author-owned PR; GitHub blocks formal self-review)

复审绑定 exact head:e1581afe49bc9233db37eb81a11324f542616e95。结论:APPROVE。

动机

这个 PR 修的是 qualification budget 的层次错位:thin task body 已经允许到 2,500 字符,但独立的 outer heartbeat_prompt_json fixture ceiling 仍停在 3,600;当前包含完整 shared-host safety、Turn 与 agent/scope binding 的代表性 envelope 已达到 3,825–4,362,因此 canary 会在 body 仍合法时阻断无关分支。

这里不应该删除必要的 hot-path 语义,也不应该顺手放宽内部 body 预算。把唯一的 outer envelope ceiling 调整到能容纳当前结构、同时保留精确 overflow 负例,是更小且可验证的修复。

改动思路

改动只触及现有 interface-budget contract、现有 hot-path smoke 和现有 heartbeat support test:将 heartbeat_prompt_json.max_json_chars 从 3,600 调到 4,800,并新增 unbound、agent-bound、scoped 三类代表性 payload、4,800/4,801 边界以及 body 独立超限的测试。

这不会改变生产 prompt builder、scheduler cadence、quota、token budget、permission 或保存的 automation。outer JSON ceiling 也没有替代 budget.py 中 thin 2,500 / native 4,000 的 body 限制;结构深度、top-level/nested counts 与其他 surface budgets 全部保持不变。

具体改动

真实 base fa57253a7888a33eb98b3b6d49d69726f54e5034 到 head 为 3 个非生产文件、+70/-2。

关键内容讲解

  • contract 文档明确区分 outer JSON qualification、inner task body、token/quota 和 authority,避免 4,800 被误读成可随意扩写 prompt 的预算。
  • hot-path smoke 复用同一个 canonical assert_surface,并遍历 unbound、agent-bound、scoped payload;没有新增第二套 budget system。
  • pytest 精确证明 4,800 接受、4,801 拒绝,同时单独把 thin/native body 推到 limit+1,确保扩大 envelope 不会掩盖 body overflow。

我运行了 156 个 focused pytest tests、hot-path-interface-budget-smoke.py、heartbeat-prompt-smoke.py、Ruff 与 git diff --check,全部通过。base/head 对照使用同一 payload:base 在 3,825 > 3,600 失败,head 在 3,825 <= 4,800 通过,因果链是直接的。当前 scoped fixture 为 4,362,仍低于新上限。

对主干的风险

主要风险是把提高 ceiling 当成今后继续堆 metadata 的许可。4,800 对当前最宽的 scoped fixture 只留下约 10% headroom,后续增长仍应优先压缩语义、复用字段,并在同一 measured contract 中解释;不能因为本次通过就取消结构/body 预算。

当前 exact-head required rollup 仍为红,但 head 与 exact base 的 mypy 都只报告相同的 7 个 decision_scope.py no-any-return 错误;四个 hosted shards、Stage2c、DCO、build 和 dependency checks 均通过。它不是本 PR 引入,但这个 approval 也不等于忽略 merge gate,基线修复后仍需重跑 required rollup。

Future-facing pass:这是既有 canonical smoke 的窄扩展,覆盖真实 production builder 输出,且没有重复 smoke family 或 runtime config knob。当前无需再抽象。

我的整体评价

这个调整把预算放在正确的一层,并用 exact boundary 与独立 body negative 防住“顺手放宽一切”的语义回归。它解决了已复现的 baseline canary failure,范围小、无运行时副作用、验证具有反事实性,因此我批准这个 exact head。

English verdict: APPROVE on exact head e1581afe49bc9233db37eb81a11324f542616e95. This test/contract-only PR raises only the outer heartbeat JSON qualification ceiling to 4,800, keeps the 2,500/4,000 inner body and all structural budgets unchanged, and proves 4,800 accepted, 4,801 rejected, plus independent body overflow rejection. All 156 focused tests, both heartbeat/interface smokes, Ruff, and diff check passed; the exact base fails the same 3,825-character fixture under 3,600 while this head passes it. The remaining red rollup is the same seven baseline decision_scope.py mypy errors and is not branch-attributable, though merge readiness must still be rechecked once that shared baseline is repaired.

@huangruiteng huangruiteng left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approval conclusion (author-owned PR; GitHub blocks formal self-approval)

复审绑定 exact head:e1581afe49bc9233db37eb81a11324f542616e95。结论:APPROVE。

动机

这个 PR 修的是 qualification budget 的层次错位:thin task body 已经允许到 2,500 字符,但独立的 outer heartbeat_prompt_json fixture ceiling 仍停在 3,600;当前包含完整 shared-host safety、Turn 与 agent/scope binding 的代表性 envelope 已达到 3,825–4,362,因此 canary 会在 body 仍合法时阻断无关分支。

这里不应该删除必要的 hot-path 语义,也不应该顺手放宽内部 body 预算。把唯一的 outer envelope ceiling 调整到能容纳当前结构、同时保留精确 overflow 负例,是更小且可验证的修复。

改动思路

改动只触及现有 interface-budget contract、现有 hot-path smoke 和现有 heartbeat support test:将 heartbeat_prompt_json.max_json_chars 从 3,600 调到 4,800,并新增 unbound、agent-bound、scoped 三类代表性 payload、4,800/4,801 边界以及 body 独立超限的测试。

这不会改变生产 prompt builder、scheduler cadence、quota、token budget、permission 或保存的 automation。outer JSON ceiling 也没有替代 budget.py 中 thin 2,500 / native 4,000 的 body 限制;结构深度、top-level/nested counts 与其他 surface budgets 全部保持不变。

具体改动

真实 base fa57253a7888a33eb98b3b6d49d69726f54e5034 到 head 为 3 个非生产文件、+70/-2。

关键内容讲解

  • contract 文档明确区分 outer JSON qualification、inner task body、token/quota 和 authority,避免 4,800 被误读成可随意扩写 prompt 的预算。
  • hot-path smoke 复用同一个 canonical assert_surface,并遍历 unbound、agent-bound、scoped payload;没有新增第二套 budget system。
  • pytest 精确证明 4,800 接受、4,801 拒绝,同时单独把 thin/native body 推到 limit+1,确保扩大 envelope 不会掩盖 body overflow。

我运行了 156 个 focused pytest tests、hot-path-interface-budget-smoke.py、heartbeat-prompt-smoke.py、Ruff 与 git diff --check,全部通过。base/head 对照使用同一 payload:base 在 3,825 > 3,600 失败,head 在 3,825 <= 4,800 通过,因果链是直接的。当前 scoped fixture 为 4,362,仍低于新上限。

对主干的风险

主要风险是把提高 ceiling 当成今后继续堆 metadata 的许可。4,800 对当前最宽的 scoped fixture 只留下约 10% headroom,后续增长仍应优先压缩语义、复用字段,并在同一 measured contract 中解释;不能因为本次通过就取消结构/body 预算。

当前 exact-head required rollup 仍为红,但 head 与 exact base 的 mypy 都只报告相同的 7 个 decision_scope.py no-any-return 错误;四个 hosted shards、Stage2c、DCO、build 和 dependency checks 均通过。它不是本 PR 引入,但这个 approval 也不等于忽略 merge gate,基线修复后仍需重跑 required rollup。

Future-facing pass:这是既有 canonical smoke 的窄扩展,覆盖真实 production builder 输出,且没有重复 smoke family 或 runtime config knob。当前无需再抽象。

我的整体评价

这个调整把预算放在正确的一层,并用 exact boundary 与独立 body negative 防住“顺手放宽一切”的语义回归。它解决了已复现的 baseline canary failure,范围小、无运行时副作用、验证具有反事实性,因此我批准这个 exact head。

English verdict: APPROVE on exact head e1581afe49bc9233db37eb81a11324f542616e95. This test/contract-only PR raises only the outer heartbeat JSON qualification ceiling to 4,800, keeps the 2,500/4,000 inner body and all structural budgets unchanged, and proves 4,800 accepted, 4,801 rejected, plus independent body overflow rejection. All 156 focused tests, both heartbeat/interface smokes, Ruff, and diff check passed; the exact base fails the same 3,825-character fixture under 3,600 while this head passes it. The remaining red rollup is the same seven baseline decision_scope.py mypy errors and is not branch-attributable, though merge readiness must still be rechecked once that shared baseline is repaired.

@huangruiteng
huangruiteng merged commit 82752b3 into main Sep 12, 2026
19 of 22 checks passed
@huangruiteng
huangruiteng deleted the codex/heartbeat-envelope-budget branch September 12, 2026 02:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant