Skip to content

fix(heartbeat): reserve readable Reward Memory prompt headroom - #4333

Merged
huangruiteng merged 3 commits into
mainfrom
codex/reward-memory-prompt-tolerance
Sep 13, 2026
Merged

huangruiteng merged 3 commits into
mainfrom
codex/reward-memory-prompt-tolerance

Conversation

@huangruiteng

Copy link
Copy Markdown
Collaborator

Summary

  • Keep the Reward Memory outcome contract readable in compact/brief/thin automation prompts instead of compressing away its validator, readback, and private-boundary guidance.
  • Apply a fixed +640 prompt headroom only when Reward Memory automatic outcome ingestion is enabled; feature-off prompts keep their existing budgets and omit the contract.
  • Wire the same policy through CLI heartbeat generation and upgrade planning, with regression coverage across all heartbeat modes.

Validation

  • pytest -q tests/control_plane/test_heartbeat_prompt_support.py tests/control_plane/test_cli_output_differential.py tests/control_plane/test_host_bootstrap_lifecycle.py tests/control_plane/test_automation_prompt_upgrade.py tests/control_plane/test_cli_output_budget.py — 169 passed.
  • Targeted Ruff, compileall, and git diff --check — passed.
  • loopx canary premerge --from-git-diff --git-diff-base origin/main — direct checks, Python compile, public-boundary scan, and 15/18 selected smokes passed. Three canary entries failed because the runner selected the machine's Python 3.9 / installed-worktree environment; the same affected canary command that is runnable under the repository Python 3.13 environment passes where independently reproducible. No manual holds.

Scope / review

Runtime surfaces are heartbeat prompt budgeting/rendering, CLI support control, and upgrade planning; docs and focused tests are included. No frontend, scheduler authority, provider calls, scoring, or private evidence changes. Feature-off parity is asserted for full, compact, brief, and thin prompts.

自审结论:改动保持 Reward Memory 默认关闭时的原有预算和行为;开启 automatic ingestion 时仅为对应自动化提示增加固定容忍,并保留可理解的安全边界说明。

Signed-off-by: huangruiteng <14976749+huangruiteng@users.noreply.github.com>
Signed-off-by: huangruiteng <14976749+huangruiteng@users.noreply.github.com>
@huangruiteng

huangruiteng commented Sep 13, 2026 •

Copy link
Copy Markdown
Collaborator Author

Approval conclusion (author-owned PR; GitHub blocks formal self-approval)

Reviewed exact head: 4333@269893eb59638a00812caf3875e90597bbad2fcc.

动机

PR #4293 之后,Reward Memory outcome contract 为了满足 heartbeat prompt 的既有字符预算被压缩得过度简略。这样虽然保留了 marker,却让 agent 难以同时理解 Todo validator、exact digest/evidence attestation、private staging、refresh/spend readback 以及 zero-provider-call fallback。问题发生在自动化提示词的可读性与 sizing 契约,不是 provider 或 scheduler 行为。这个 PR 的目标是让 automatic outcome ingestion 开启时保留可执行的完整边界,并给这段增量一个固定容忍;关闭时不改变普通 automation prompt 的预算与内容。

改动思路

入口仍是现有的 handle_support_control_command heartbeat-prompt、build_upgrade_plan 和 build_heartbeat_prompt,没有新增 capability、provider 或 authority owner。CLI/upgrade 从既有 reward_memory_goal_policy 计算 enabled && automation.automatic_ingest;该值只作为派生的渲染开关。开启时,已有 heartbeat renderer 接收可读的 Reward Memory rule,build_interface_budget 在非 visible-Goal automation mode 上为 marker-bearing body 增加固定 640 字符 headroom;关闭时传入空 rule,继续使用原 mode budget。full/compact/brief/thin 共用这一边界,native Goal renderer 不受影响。这个设计复用了原有 rule、renderer、budget 与 policy owner,避免了第二套 prompt 或 feature 状态。

具体改动

关键代码讲解

  • loopx/control_plane/heartbeat/builder.py:build_heartbeat_prompt:组装现有 renderer kwargs,并依据 Goal policy 注入 readable full/compact rule 或空值;同时把派生状态投影到 payload,便于观察和验证。Reward Memory kwargs 的小 helper 保持该决策函数在维护性阈值以内。
  • loopx/control_plane/heartbeat/budget.py:build_interface_budget:新增 REWARD_MEMORY_OUTCOME_PROMPT_HEADROOM_CHARS = 640,仅对非 visible-Goal 且包含 Reward Memory marker 的 automation body 增加 headroom,并返回 reward_memory_headroom_chars。
  • loopx/control_plane/heartbeat/task_body.py:四个 automation renderer 增加可选 reward_memory_rule,默认值仍是现有 rule,保持直接调用兼容;native visible-Goal renderer 没有被改写。
  • loopx/cli_commands/support_control.py 与 loopx/upgrade.py:在 CLI 生成和升级规划两条真实生产路径读取同一 Goal policy,确保安装/刷新不会绕过开关。
  • loopx/control_plane/heartbeat/rules.py 与 docs/heartbeat-automation-prompt.md:恢复 compact rule 的可读语义,并记录固定容忍、一次性 differential migration allowance 与 authority/privacy 边界。
  • tests/control_plane/test_heartbeat_prompt_support.py:对 full/compact/brief/thin 做 paired feature-on/off 断言,覆盖 marker、headroom、base budget 和安全边界。

对主干的风险

未发现 P0/P1/P2 阻塞项。最强的负向情形是:Reward Memory 未启用的 Goal 意外继承新 rule 或 +640,从而改变普通 automation prompt;paired tests 和两条生产入口的 policy projection 都防止了这一点。另一个边界是未来若有 caller 注入包含该私有 marker 的无关文本,marker-only sizing 可能误给 headroom;当前公开生产 caller 只使用 canonical rule,且该风险已在实现审查中记录为后续收紧点。

变更没有持久化 schema、provider write、quota spend、scheduler mutation、权限授予或用户前端影响。失败时仍由现有 within_budget/CLI differential gate 处理,未引入静默 fallback。状态分类使用现有结构化 Goal policy 和 bool 派生值,不依赖 substring denylist;提示词中的 validator/readback 是既有 runtime contract 的说明,不声称新增 actor lifecycle 或 authority。

验证矩阵:169 个 focused tests(heartbeat prompt support、CLI output differential/budget、host bootstrap、automation upgrade)通过;Ruff、Python compile、git diff check 和维护性 ratchet 通过;GitHub DCO、dependency、build、dashboard、kernel/static、Node compatibility、Windows、e2e、mutants、全 test shards、pytest、merge-gate 与 Sonar checks 均通过。pre-merge canary 的 direct checks、compile、public-boundary scan 通过;3 个本机 catalog/risk 条目因 canary runner 使用系统 Python 3.9/已安装 worktree 而失败,仓库要求 Python 3.11+,无改动相关失败证据,也没有 manual hold。

我的整体评价

这是一个与用户反馈直接对应、范围适中的修复:在已有 heartbeat bounded context 内恢复理解成本,同时把增量明确绑定到 Reward Memory automatic ingestion 的开启,不把预算放宽成全局默认。四种 automation mode 的 disabled path 保持 base budget,enabled path 的固定 640 容忍和可读 contract 都有回归覆盖;文档也披露了行为边界。最新 exact head 已包含一次维护性收窄(helper 提取),建议必需 checks 全绿后 admin squash merge。

English verdict: APPROVE — exact head 269893eb59638a00812caf3875e90597bbad2fcc; no actionable findings. The change restores readable Reward Memory automation guidance with a fixed opt-in +640 headroom and preserves feature-off parity. Focused 169-test validation and required GitHub checks pass; merge after final exact-head readiness recheck.

Signed-off-by: huangruiteng <14976749+huangruiteng@users.noreply.github.com>
@huangruiteng
huangruiteng merged commit 6b337bc into main Sep 13, 2026
21 checks passed
@huangruiteng
huangruiteng deleted the codex/reward-memory-prompt-tolerance branch September 13, 2026 10:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant