Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
4c8a714
fix(heartbeat): share live dispatch across automation and goal hosts
huangruiteng Sep 10, 2026
ca2e5e2
test(heartbeat): add opt-in release native Goal regression
huangruiteng Sep 10, 2026
bea6b4e
test(hosts): add Claude release and response-loss coverage
huangruiteng Sep 10, 2026
5605159
fix(hosts): separate Todo acceptance from Turn settlement
huangruiteng Sep 10, 2026
9d7e3f9
fix(heartbeat): restore host boundaries and isolate release environments
Green-hats Sep 11, 2026
ddcd3a0
fix(heartbeat): share live dispatch across automation and goal hosts
huangruiteng Sep 10, 2026
bd9de58
test(heartbeat): add opt-in release native Goal regression
huangruiteng Sep 10, 2026
4be0d71
test(hosts): add Claude release and response-loss coverage
huangruiteng Sep 10, 2026
d40bb27
fix(hosts): separate Todo acceptance from Turn settlement
huangruiteng Sep 10, 2026
47aa6e1
fix(heartbeat): preserve static safety without stale execution policy
huangruiteng Sep 11, 2026
258d380
fix(qualification): isolate host configuration and child environments
huangruiteng Sep 11, 2026
5deda25
test(hosts): preserve conditional skill routing in activation checks
Green-hats Sep 11, 2026
f7a6c26
Merge updated host contract and reconcile qualification regressions
Green-hats Sep 11, 2026
ad31b80
fix(heartbeat): share live dispatch across automation and goal hosts
huangruiteng Sep 10, 2026
24c2ec1
test(heartbeat): add opt-in release native Goal regression
huangruiteng Sep 10, 2026
8f05fc4
test(hosts): add Claude release and response-loss coverage
huangruiteng Sep 10, 2026
520c06f
fix(hosts): separate Todo acceptance from Turn settlement
huangruiteng Sep 10, 2026
263c794
fix(heartbeat): preserve static safety without stale execution policy
huangruiteng Sep 11, 2026
e14d922
fix(qualification): isolate host configuration and child environments
huangruiteng Sep 11, 2026
4324180
test(hosts): absorb contributor qualification regressions from #4216
huangruiteng Sep 11, 2026
8015862
feat(hosts): load current Goal rules and reconcile prompts during upg…
huangruiteng Sep 11, 2026
55c0e0e
test(cli): attribute one-time static safety output growth
huangruiteng Sep 11, 2026
94b67cf
test(cli): cover brief and compact safety transition variants
huangruiteng Sep 11, 2026
08f597a
fix(update): reconcile through the upgraded pipx interpreter
huangruiteng Sep 11, 2026
7219dce
Merge current #4201 into contributor follow-up and verify live bootst…
huangruiteng Sep 11, 2026
688abec
Merge pull request #4216 from Green-hats/niu/pr4201-fixes
huangruiteng Sep 11, 2026
b572baf
fix(hosts): qualify brief output and automatic prompt adoption
huangruiteng Sep 11, 2026
e085144
Merge remote-tracking branch 'origin/main' into codex/native-goal-pro…
huangruiteng Sep 11, 2026
f1d227e
fix(heartbeat): preserve brief authority within the App budget
huangruiteng Sep 11, 2026
1beb54e
test(hosts): qualify prompt decisions with release-only model probes
huangruiteng Sep 11, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
119 changes: 119 additions & 0 deletions docs/development/testing-and-quality.md
Original file line number Diff line number Diff line change
Expand Up @@ -737,6 +737,125 @@ Onboarding 输入来自正式 guided packet builder;provider 调用前只替
门禁错误、写入或 quota 消耗。Human gate 的优先级是显式规则:等待用户时没有
executable work 属于预期状态,不能误判为 projection gap。

## Release-only native Goal regression / 仅发布前的原生 Goal 回归

`scripts/qualify-native-goal-release.py` exercises the real Codex CLI app-server
Goal lifecycle on a disposable ledger project with two dependent Todos. It
reuses the shipped native Goal transport and current prompt, then checks an
independent acceptance oracle, completed Todos, unique bound spends, durable
writeback readback, and terminal no-follow-up quota. This is not a benchmark
score or evidence of universal model reliability.

The Codex release arm requests the shipped bootstrap, not an injected private
work recipe. Deterministic regressions execute the saved CLI loader, change its
registry inputs, and prove fresh loading, non-recursion, preserved explicit
policy and removed-agent rejection. Claude's stdio regression loads
`host_prompt` through the real MCP transport and verifies the same bound Goal.
These tests are free of model calls; passing them is not a live model pass.

Upgrade regression uses real temporary SQLite/TOML stores, a second connection,
writer-lock contention, injected mirror failure, journal recovery and stale or
custom-input rejection. New-runtime reconciliation is exercised through the
real CLI, while package installation is substituted in that focused test.
Running-App deployment additionally needs a selected owner-authorized canary
and delayed readback; synthetic SQLite tests alone do not qualify App caches.
The output differential permits a bounded one-time transition to the exact
static-safety marker, not permanent growth allowances or relaxed quota budgets.

```bash
# No model invocation, no token cost; explicit skipped result, exit 0.
python3 scripts/qualify-native-goal-release.py
# Release operator opt-in only; explicit isolated API profile (Responses API).
# Supply LOOPX_CODEX_QUALIFICATION_API_KEY securely in this process, plus:
export LOOPX_CODEX_QUALIFICATION_MODEL='<selected-model>'
export LOOPX_CODEX_QUALIFICATION_BASE_URL='https://example.com/v1'
python3 scripts/qualify-native-goal-release.py --release-live
```

Do not add the live command to default pytest, PR CI, per-diff canaries, or
ordinary developer iteration. The deterministic runner-policy tests may run
there; they never opt into real model execution. Missing CLI, native Goals or
the explicit Codex API profile returns `skipped` and exit 0, not a claimed live pass.
Once qualification is attempted, failed acceptance, incomplete settlement,
blocked/unfinished Goals and deadline expiry fail with exit 1. The default
deadline is 1,200 seconds; this is a wall-clock ceiling, not a token budget.

仅 release 前显式开启,避免默认消耗开发者 token。CI/本机环境不支持时跳过且不阻塞,
但保留 `skipped` 标记;真实执行后失败不能冒充环境跳过。使用操作者显式选择的 API
模型、地址与密钥,不导入日常 Codex 配置、登录或会话,不修改活跃 Goal/automation。
任务、registry、runtime 与 Git worktree
在一次性目录内;沙箱允许该目录及本地 TS worker 所需的网络能力,
这不是网络隔离,任务不授权外部操作。回归脚本不采集或上传原始对话/工具日志,
公开结果仅包含状态、计数和错误类别;Codex 会话仅留在一次性隔离目录内。
两个 runner 均从允许列表创建环境并隔离 HOME、配置和缓存;不透传其他 token、
认证 socket、shell 启动变量或原始 ARK_API_KEY。Codex 工具 shell 从空环境注入必要
运行变量,不继承 host API key。Claude host 仅接收所选 provider 的映射密钥;这不是
对同用户进程或 Claude Bash 的凭据隔离沙箱,不能把真实业务秘密加入测试任务。

### Claude Code and release coverage / Claude Code 与发布覆盖

For focused thin/brief prompt-decision regression, use
`python3 scripts/qualify-host-prompt-release.py --release-live` only during
explicit release qualification. It defaults to no calls; missing credentials
report `skipped`, not a live pass. With securely injected `ARK_API_KEY`, it uses
Doubao evolving for two independent repetitions of quiet-work, notifying-wait,
quiet-wait and required-vision-replan cases in each mode. Expected decisions
remain outside model input. All attempts must pass; no answer correction or
retry-until-pass is used. Ordinary pytest only checks the probe and negative
oracles with scripted responses, without provider calls.

This is a synthetic decision-level probe using current generated prompts,
not proof of tool execution, host scheduling, upgrade delivery or full-Goal
completion. Keep the Codex/Claude live Goal arms and real CLI/MCP/SQLite tests
as separate evidence. Only hashes and pass/fail receipts are emitted, not raw
prompts/responses. Model transport failures fail qualification rather than
becoming environment skips.

仅发布前显式执行,普通 CI 不调用模型。检查静默不等于空转、等待不能擅自执行、
vision replan 未关闭时不能提前结束 Goal;这不是完整 Claude/Codex 行为验收的替代。

```bash
# No provider call by default. Explicit release opt-in uses ARK_API_KEY from the environment.
python3 scripts/qualify-claude-goal-release.py --release-live
```

This arm uses the same ledger specification, independent oracle and durable
settlement readback as the Codex arm. It launches actual Claude Code with the
project's shipped `loop.md` and LoopX stdio MCP server, using
`doubao-seed-evolving` through Ark's Anthropic-compatible API. It does not
inherit another Anthropic account, install into the user's Claude configuration,
or retain host sessions. The subprocess timeout also cleans its process group
on POSIX. Allowed local development tools are not a security sandbox; the
synthetic task authorizes no external side effects.

**A headless work-loop pass is not a `/loop` timer pass.** The release report
explicitly returns `scheduler_qualification=not_run_headless`; interactive
native wakeup, cancel/resume and process-restart behavior need their own host
qualification. Do not turn repeated `claude -p` invocations into a substitute
scheduler and claim host lifecycle coverage.

Before calling a changed host surface release-qualified, distinguish:

| Boundary | Required evidence |
| --- | --- |
| Work and terminal closeout | Final candidate, actual host, independent artifact checks, completed Todos and terminal quota; code delivery alone is insufficient. |
| Idempotency and failure | Real committed lifecycle/writeback/spend followed by lost-response injection and same-intent retries; one final spend. Failed declared validation must not complete or spend. |
| Authority and transport | Actual MCP initialization/tool invocation and mismatched-agent rejection; existing claim/lease and validation suites remain required. |
| Host lifecycle | Native scheduler wakeup/cancellation/resume on supported versions, reported separately from headless execution. |
| Upgrade and isolation | Exact managed-wrapper recognition, preview/apply revision checks, preserved scheduler state, explicit skips, no default model calls or leaked test processes. |

普通 CI 只跑确定性规则、真实 CLI/MCP 和故障注入;模型执行仍仅 release 前显式启用。
环境缺失可 skip 且退出成功,但最终版本没有完整的真实 host 结果时,不得写成
“产品级发布验证通过”。单次成功也不是模型可靠性或长程调度 soak 的证明。

The real delivery regression covers ordinary Todo acceptance before internal
writeback/spend, existing-successor linking, and receipt-backed terminal closure.
Its task specification describes only the deliverable; the external oracle also
checks LoopX accounting. Passing non-delivery fixtures does not qualify delivery.
The same delivery class must pass failed-validation rejection and committed
response-loss recovery without duplicate spending or premature terminal closure.
Do not relabel delivery work or weaken the independent oracle to pass a host test.

## Exact Release Commit Gate / 精确发布 Commit 门

The final release gate does not rerun tests through a second orchestration
Expand Down
117 changes: 112 additions & 5 deletions docs/heartbeat-automation-prompt.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,118 @@ transport-neutral goal prompt and lets the goal runtime own inner iteration;
see the host integration protocol instead of adapting this recurring
automation contract.

### Native Goal bootstrap and live execution instructions

Brief automation now uses the same fully qualified notification/execution rule
as thin, including `heartbeat_recommendation.agent_must_attempt` and
`execution_obligation.must_attempt_work`. Brief no longer embeds a second
static refresh/spend sequence: after validated work it follows the current
`interaction_contract.cli_channel.settlement_plan.ordered_steps`, or current
`next_cli_actions` when there is no plan. Generator command fields remain for
compatibility, not as a stale fallback. Todo acceptance alone is not Turn
settlement or terminal vision closure. The brief budget remains 3,500 characters.

Brief 与 thin 共用完整执行义务路径;这次有意移除 brief 固定结算配方,而不是
删除结算义务。真实 App preflight、registry scope、完整 guard 和静态安全规则
均保留;结算顺序与身份以本轮动态 contract 为准,vision replan 不由历史成功清账。

New supported host activations use `heartbeat-prompt --bootstrap`: a saved
loader requests the installed rules rather than freezing a long execution body.
The loader and automation bootstrap share rendering and successful-response
checks. Registry-derived state is resolved at load time; explicitly supplied
policy remains bound. The inner command does not request another bootstrap.
Claude Code loads its inner body through the bound MCP `host_prompt` tool.
TraeX's separate capability projection remains separate, not embedded by this
loader. See [prompt upgrade lifecycle](reference/automation-prompt-upgrades.md)
for automatic exact-managed adoption during `update --apply`, including the
qualified running-App SQLite/TOML adapter and conflict recovery boundary.

#### Static semantics retained across hosts

Thinning removes duplicated recipes, not authority boundaries. The shared
runtime body keeps repository rules, credential/private-material protection,
explicit authorization for destructive Git/production actions, and exception
routes (`loopx-project` for lifecycle/registry, `loopx-self-repair` for drift).
These routes are conditional, not mandatory skill calls on every iteration.
Ordinary Claude MCP iterations still use MCP; the CLI route is not a second
accounting path.

| Semantic | Owner after thinning |
| --- | --- |
| Privacy, repository rules, dangerous-action authority | Shared static safety rule; a trusted host is not blanket permission |
| Lifecycle or projection repair | Conditional static repair route; repair does not bypass gates |
| Selection, claims, vision replan, exact settlement identities/order | Current successful interaction contract, not saved command recipes |
| Blocked path vs whole Goal | Gate only the affected path; continue independent admitted work; only terminal no-follow-up completes the Goal |
| Git branch/worktree/PR policy | User and repository rules; no generic `No project branches` restriction |
| Prompt authoring/maintenance advice | Documentation, not per-iteration executor instructions |

For heartbeat shells, assign `LOOPX_TURN` in a separate statement before the
guard, in the same shell invocation. A command-prefix assignment does not make
the variable available to argument expansion in Bash/zsh. Reuse the same value
on retries. Native Goal entry remains host-specific and does not inherit this
heartbeat bootstrap.

Thin's ceiling is 2,500 characters (previously 1,900), and compact's is 6,500
(previously 6,200): the additional room covers shared safety and an executable
Turn/guard block rather than omitting identities or static obligations.

The automation lifecycle is the reference for shared execution, not a wrapper
around native Goal behavior. Thin automation and Codex CLI/SSH, TraeX and Ark
Managed Agent Goal bodies share quota dispatch: selection/re-entry, admitted work
and validation, then the current writeback/settlement instructions. They do not
share scheduler ownership, host completion, or blocked/resume rules.

Native Goal bodies share a compact bootstrap.
Generate it with the host's existing profile (for example `heartbeat-prompt
--runtime-profile codex_cli --goal-id <GOAL_ID> --agent-id <AGENT_ID>`).
The persistent body binds the Goal/Agent and quota entrypoint; each work iteration
reads the current complete, successful quota JSON. The **inner execution
instructions remain in `interaction_contract`**, including selection/re-entry,
admitted work, and exact `cli_channel.settlement_plan.ordered_steps`.

Native Goal bodies no longer embed a second static accountable refresh/spend
template. Those command fields remain available in the generator response for
compatibility/inspection, but are not a fallback for the live settlement plan.
When no ordered settlement plan applies, consume the current `next_cli_actions`,
including any required re-entry; do not substitute a saved generator command.
Execute selection/re-entry before work and writeback/spend only after the
corresponding validated work; a projected accounting command is not evidence
that work happened. Preserve the plan's identity and flags, and follow readback
or recovery after an ambiguous write instead of retrying a guessed command.
Failed or incomplete contract reads permit neither work nor spend.

An unbound Codex CLI or Ark Goal with selected Todo/replan work now receives a
quota re-entry template with `--turn-instance-id`. Fill it with one public-safe
unique work-iteration id and reuse that id on retries. The next packet supplies
the same ordered settlement machinery used by automation, with `visible-goal`
attribution. SSH Goal continues to use its existing `--begin-turn` path. This
fixes the previous unbound native refresh/spend projection: those commands could
not satisfy the existing settlement identity guard. It does not turn CLI/Ark
Goals into App heartbeat receipts or move scheduler ownership into LoopX.

The bootstrap retains work-sizing guidance and the distinction between progress
and Goal completion. A new Todo is not a new host Goal; quiet/blocked states are
not terminal no-follow-up. Codex alone retains its native blocked/resume rule.
User/repository authority still applies; a trusted host is not blanket permission.
This changes newly generated native Goal bodies and thin automation dispatch,
not active host Goal objectives or benchmark prompts already pinned to a run.
An installed runtime supplies updated dynamic contracts on later reads; upgrading
it does not retroactively remove old text from an existing Goal.

Claude Code's MCP-backed `loop.md` follows the same work-sizing rule and current
quota contract, without a fixed one-segment limit or empty-Todo-list completion
shortcut. Its `complete_task` tool already owns the ordered writeback/spend
transaction: callers must not perform a second accounting sequence through CLI.
Partial work is not Todo completion. Native `/loop` remains Claude's scheduler;
only the current Goal's wakeup may be cancelled after terminal no-follow-up.
These changes apply when `loop.md` is regenerated, not by editing active user files.
The MCP tool now exposes `successor_todo_ids`, reusing CLI/TS completion semantics
to link known follow-up without creating another Todo. Ordinary acceptance and
Turn settlement are distinct: the adapter validates/completes work before its
writeback/spend; only terminal closeout requires the full receipt chain. This
removes the former delivery-class circular prerequisite, not validation or
accounting. See the [release test guide](development/testing-and-quality.md#claude-code-and-release-coverage--claude-code-与发布覆盖).

For Codex App, the generated quota command carries the compact explicit runtime
profile `--runtime-profile codex_app_heartbeat` (generated commands use the
equivalent compact alias `--codex-app`). The prompt does not restate the
Expand Down Expand Up @@ -285,11 +397,6 @@ Replace the placeholders before installing the automation:
```text
Advance the goal described in <ACTIVE_GOAL_STATE_PATH>.

Generic LoopX lifecycle. Keep project-specific branching out of the
automation prompt. Put local policy in registry, active-state sections, adapter
output, quota should-run.goal_boundary, or boundary rules; if a lifecycle rule
is needed, update loopx heartbeat-prompt so all projects inherit it.

Before spending delivery compute, first make the LoopX CLI reachable in
this automation shell, then run the quota guard:

Expand Down
4 changes: 4 additions & 0 deletions docs/product/release-readiness.md
Original file line number Diff line number Diff line change
Expand Up @@ -144,6 +144,10 @@ Before moving `stable`, maintainers should:
- bump `loopx.__version__` and `pyproject.toml` together when user-visible
release behavior changes;
- create or verify the matching Git tag, for example `v0.1.3`;
- for host Goal/prompt changes, explicitly run the
[release-only native Goal regression](../development/testing-and-quality.md#release-only-native-goal-regression--仅发布前的原生-goal-回归)
in a supported Codex environment; record an unavailable environment as
`skipped`, not a live pass. Never enable paid model execution in default PR CI;
- fast-forward `stable` to that tagged commit after the release canary passes;
- confirm `release.json`, `loopx doctor`, and `loopx update check` report the
same package version and tag;
Expand Down
Loading