Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
73 changes: 73 additions & 0 deletions docs/development/testing-and-quality.md
Original file line number Diff line number Diff line change
Expand Up @@ -817,6 +817,8 @@ vision replan 未关闭时不能提前结束 Goal;这不是完整 Claude/Codex
```bash
# No provider call by default. Explicit release opt-in uses ARK_API_KEY from the environment.
python3 scripts/qualify-claude-goal-release.py --release-live
# A completed inventory stage must not hide unfinished integrity acceptance.
python3 scripts/qualify-claude-goal-release.py --release-live --scenario replan
```

This arm uses the same ledger specification, independent oracle and durable
Expand Down Expand Up @@ -856,6 +858,77 @@ The same delivery class must pass failed-validation rejection and committed
response-loss recovery without duplicate spending or premature terminal closure.
Do not relabel delivery work or weaken the independent oracle to pass a host test.

The `replan` scenario starts from a real, settled filename-inventory Todo and its
valid `vision_closed` stage decision, not a fabricated missing writeback. The
business specification still requires file sizes, checksums and a read-only
integrity verifier. It requires an explicit successor vision/path decision,
completed concrete successor work and terminal readback. The independent oracle
checks actual hashes and sizes, then changes, removes and adds files in disposable
copies; a verifier that silently regenerates its evidence fails. The original
inventory must retain exactly one spend. This complements the finite delivery
scenario: a model that simply closes every vision cannot pass both.
The oracle does not require the literal final disposition `replan`: after the
new successor has actually delivered, `no_followup` + `stop` is a valid scoped
decision. It must still pass independent artifact, new successor, durable receipt
and fresh terminal checks; `vision_closed` + `stop` is not Goal closure.

`replan` 场景从真实完成并结算、具有有效 `vision_closed` 判断的“文件名清单”阶段启动,
但完整验收仍缺少大小、校验和与只读校验器。测试要求后继 vision/显式路径调整、
具体后继交付及最终终态;独立验收在一次性副本里篡改、删除、新增文件,拒绝通过
自动重建清单掩盖错误。每个 Todo 必须恰好结算一次。后继真实交付后,最终路径可为
`no_followup` + `stop`,不强求字面值 `replan`;`vision_closed` + `stop` 仍不是 Goal
完成。两种场景都仅 release 前显式运行,普通 CI 不调用模型。

### MCP vision authoring and recovery / MCP vision 写入与恢复

The MCP guard projects `interaction_contract.mcp_channel`. For admitted normal
Todo delivery, it replaces the raw CLI writeback/spend instructions with
`complete_task` ownership; those are alternate transports, not two obligations.
Replan-only and blocked lanes preserve their live CLI actions and binding.
Vision field and total limits come from the same TS validator, not copied prompt
constants. Quota admission, permission and workspace facts are unchanged.

MCP 的普通 Todo 交付不再同时要求模型执行 CLI 记账和 MCP 结算两套流程;独立 replan
仍使用动态 CLI 契约。vision 字段与预算直接来自 TS 校验器,不要求模型猜格式或翻测试。

`complete_task` accepts either `agent_vision` (the existing bounded
`goal_vision_replan_contract_v0` JSON packet) or `vision_unchanged_reason`.
An unchanged decision needs a persisted valid baseline. The TypeScript host
plan forwards that authored decision to its ordinary writeback; v1 requests
fail closed against old runtimes instead of silently dropping the fields.
Syntax and vision-budget preflight reuse the TS validator before lifecycle writes;
baseline-dependent checks still run at writeback. Oversized authoring is a
correctable input failure, not a terminal Goal failure. If an older/interrupted
host already completed the Todo but failed writeback, retry `complete_task` with
the same completion intent and a corrected uncommitted vision. Checkpoint-only
recovery is not a substitute for unfinished settlement.

If a previously completed MCP Todo omitted its decision, call
`review_task_vision(todo_id, agent_id, agent_vision=...)` with that same Todo.
It uses the original host Turn and the same writeback command constructor,
delegating to the existing typed checkpoint recovery. It neither repeats Todo
completion nor spends again. Exact replay is idempotent; a conflicting committed
decision or a later superseding vision is rejected. It does not change Next
Action or erase other work, gates, or permissions. A genuinely new replan follows
the current interaction contract under a fresh admitted binding, not an edit to
an already committed decision. Claude Todo-less replan now projects that identity
re-entry before any refresh/spend instructions; ordinary MCP Todo delivery is
unchanged.

Todo acceptance, settled accounting, checkpoint satisfaction and Goal termination
are separate facts. `vision_closed` closes a stage and still requires a successor
vision for an active Goal. `no_followup` is an authored scoped closure assertion,
not a substitute for evidence; remaining acceptance gaps or gates still prevent
terminal quota. Kernel validation does not independently prove arbitrary prose
true, so behavior qualification must also inspect the delivered artifacts.

MCP 可随完成操作携带 vision 判断,也可用 `review_task_vision` 在原 Turn 补齐遗漏。
复用 TS 的既有恢复规则,不新增结算引擎、不重扣额度;已提交的判断不能偷偷改写。
格式和预算预检在 Todo 完成前拒绝非法输入;若旧宿主已部分完成,则修正未提交的
vision 并重试原 `complete_task`,不能用仅补 checkpoint 的操作替代未完成结算。
“checkpoint 满足”不等于“Goal 完成”,`vision_closed` 只结束阶段,真实缺口仍须规划。
外层任务只描述业务验收,LoopX 协议由宿主内层指令和工具承接。

## Exact Release Commit Gate / 精确发布 Commit 门

The final release gate does not rerun tests through a second orchestration
Expand Down
11 changes: 10 additions & 1 deletion loopx/claude_goal_mode/scripts/goalmode_cmd.py
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,16 @@ def loop_execution_content(goal_id, agent_id) -> str:
"Honor claim/lease and user/repository authority; claim only when required.\n"
"Run real acceptance checks before `complete_task`; supply truthful evidence\n"
f"and the bound agent_id=\"{agent_id}\". Complete only finished Todos, not partial work.\n"
"That MCP operation owns writeback/spend; do not repeat its accounting via CLI.\n"
"That MCP operation owns writeback/spend: use it INSTEAD OF the raw CLI sequence.\n"
"Use interaction_contract.mcp_channel for tool ownership and vision input limits.\n"
"At material delivery, compare the Goal's vision/acceptance with actual evidence.\n"
"Pass the resulting agent_vision or a justified vision_unchanged_reason to\n"
"complete_task. If omitted, use review_task_vision on that completed Todo to\n"
"repair its missing checkpoint without another spend. This is not a Goal-stop\n"
"shortcut: open acceptance needs replan; vision_closed closes a stage and needs\n"
"a successor vision; no_followup requires evidence of no remaining scoped work.\n"
"For new replan work not covered by these tools, use the exact live\n"
"interaction_contract CLI actions, preserving its binding and settlement order.\n"
"Link already planned follow-up via successor_todo_ids; next_agent_todo creates\n"
"new work, not a reference to an existing id. Do not duplicate the current plan.\n"
"After a lost response, read back or retry the same completion intent; do not\n"
Expand Down
32 changes: 30 additions & 2 deletions loopx/control_plane/goals/vision_checkpoint.ts
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ const GOAL_PATH_DELTA_SCHEMA_VERSION = "goal_path_delta_v0";
const GOAL_VISION_BUDGET_ERROR = "vision_budget_exceeded";
// Direction and evidence-linked path changes share one bounded packet.
const GOAL_VISION_TOTAL_LIMIT = 1_800;
const GOAL_VISION_ADVANCEMENT_POLICIES = ["as_needed", "repeat_until_closed"] as const;
const VISION_UNCHANGED_REASON_LIMIT = 240;
const VISION_BUDGET_SUGGESTION_LIMIT = 96;

Expand Down Expand Up @@ -59,6 +60,33 @@ const GOAL_PATH_DELTA_LIST_LIMITS = {
unresolved_questions: [2, 140],
evidence_refs: [4, 140],
} as const;

/** Authoring hints share the validator's limits; they grant no transition authority. */
export function visionAuthoringContract(): JsonObject {
return {
schema_version: GOAL_VISION_REPLAN_SCHEMA_VERSION,
fields: {state: "lifecycle token", vision_patch: {...GOAL_VISION_FIELD_LIMITS}},
common_states: ["vision_patch_proposed", "vision_closed", "no_followup"],
advancement_policies: [...GOAL_VISION_ADVANCEMENT_POLICIES],
minimal_example: {schema_version: GOAL_VISION_REPLAN_SCHEMA_VERSION, state: "vision_patch_proposed", vision_patch: {
vision_summary: "Scoped outcome", acceptance_summary: "Verified evidence and remaining gap",
}},
authoring_hint: "Fields are optional, not a checklist. Keep the whole decision compact; total includes path_delta. Do not copy the delivery evidence report into every field.",
total_text_limit: GOAL_VISION_TOTAL_LIMIT,
unchanged_reason_limit: VISION_UNCHANGED_REASON_LIMIT,
path_delta: {
schema_version: GOAL_PATH_DELTA_SCHEMA_VERSION,
outcomes: [...GOAL_PATH_DELTA_OUTCOMES],
required: ["outcome", "prior_assumption", "observed_reality"],
require_any: ["retained", "changed", "stopped"],
scalar_limits: {...GOAL_PATH_DELTA_SCALAR_LIMITS},
list_limits: Object.fromEntries(Object.entries(GOAL_PATH_DELTA_LIST_LIMITS).map(
([field, [maxItems, maxChars]]) => [field, {item_type: "string", max_items: maxItems, max_item_chars: maxChars}],
)),
},
rule: "Compare acceptance with evidence. vision_closed closes a stage, not the Goal; no_followup requires no remaining scoped work. A changed mainline needs path_delta; respect the live replan contract.",
};
}
// Bounded typed fallback declarations survive prepare unchanged so the
// declared direction cannot disappear behind later read-model compaction.
const VISION_FALLBACK_DECLARATION_ENTRY_LIMIT = 4;
Expand Down Expand Up @@ -299,7 +327,7 @@ function normalizeGoalVisionState(value: unknown): string {

function normalizeAdvancementPolicy(value: unknown): string {
const candidate = compactText(value).toLowerCase().replaceAll("-", "_");
if (candidate !== "as_needed" && candidate !== "repeat_until_closed") {
if (!GOAL_VISION_ADVANCEMENT_POLICIES.some(policy => policy === candidate)) {
throw new EffectRuntimeRequestError(
"agent_vision.advancement_policy must be one of: as_needed, repeat_until_closed",
);
Expand Down Expand Up @@ -624,7 +652,7 @@ function deliveryBoundary(value: unknown): DeliveryBoundary {
throw new EffectRuntimeRequestError("delivery_boundary is unsupported");
}

function normalizeVisionUnchangedReason(value: unknown): string | null {
export function normalizeVisionUnchangedReason(value: unknown): string | null {
const unchanged = compactText(value);
if (!unchanged) return null;
validatePublicSafeText("vision_unchanged_reason", unchanged);
Expand Down
62 changes: 61 additions & 1 deletion loopx/control_plane/host_adapter_settlement.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,17 @@
from __future__ import annotations

import json
import tempfile
from collections.abc import Mapping
from dataclasses import dataclass
from contextlib import contextmanager
from dataclasses import dataclass, replace
from pathlib import Path
from enum import StrEnum
from typing import Any, Protocol

from .effect_program import SettlementIdentity
from .effect_runtime import EffectRuntimeRejected, effect_runtime_result
from .goals.vision_checkpoint import prepare_vision_refresh


HOST_ADAPTER_SETTLEMENT_SCHEMA_VERSION = "host_adapter_todo_settlement_v0"
Expand Down Expand Up @@ -55,6 +59,8 @@ class HostTodoSettlementRequest:
execution_mode: str
completion_args: tuple[str, ...]
no_follow_up: bool = False
vision_path: str | None = None
vision_unchanged_reason: str | None = None


class HostCliRunner(Protocol):
Expand Down Expand Up @@ -95,9 +101,63 @@ def _request_payload(
}
if provider_outcomes is not None:
payload["provider_outcomes"] = provider_outcomes
if request.vision_path or request.vision_unchanged_reason or phase == "vision_refresh":
payload.update(
schema_version="loopx_host_todo_completion_transaction_v1",
vision_path=request.vision_path,
vision_unchanged_reason=request.vision_unchanged_reason,
)
return payload


@contextmanager
def host_vision_request(request: HostTodoSettlementRequest, vision: dict | None, unchanged: str):
"""Materialize authored JSON for the existing CLI codec, never as public state."""
if vision is not None and unchanged:
raise ValueError("choose a vision patch or an unchanged reason, not both")
if vision is not None and not isinstance(vision, dict):
raise ValueError("agent_vision must be a JSON object")
if vision is None:
yield replace(request, vision_unchanged_reason=unchanged or None)
return
# Reject malformed, misbound or oversized authoring before lifecycle writes.
# This is syntax/budget preflight only: refresh-state still validates against
# the real baseline and current replan/settlement state at writeback time.
prepare_vision_refresh(vision, goal_id=request.goal_id, agent_id=request.agent_id,
existing_agent_vision=None, merge_patch=False, require_path_delta_for_durable_change=False)
with tempfile.TemporaryDirectory(prefix="loopx-host-vision-") as directory:
path = Path(directory) / "vision.json"
path.write_text(json.dumps(vision, ensure_ascii=False, allow_nan=False), encoding="utf-8")
yield replace(request, vision_path=str(path))


def refresh_host_todo_vision(request: HostTodoSettlementRequest, *, run_cli: HostCliRunner) -> str:
"""Repair the original checkpoint; no lifecycle operation, new Turn or spend."""
plan = _runtime_reduction(_request_payload(request, phase="vision_refresh"), phase="vision_refresh")
_runtime_identity(plan.get("identity"))
args = plan.get("args")
if not isinstance(args, list) or any(not isinstance(arg, str) for arg in args):
raise RuntimeError("TypeScript host vision command shape mismatch")
return run_cli(args)


def project_host_interaction(output: str) -> str:
"""Keep admission facts; let the typed host lens choose the transport instructions."""
try:
packet = json.loads(output)
except ValueError:
return output
if not isinstance(packet, dict):
return output
projection = _runtime_reduction({
"schema_version": "loopx_host_todo_completion_transaction_v1",
"phase": "project_guard", "packet": packet,
}, phase="project_guard")
if not isinstance(projection.get("packet"), dict):
raise RuntimeError("TypeScript host interaction projection shape mismatch")
return json.dumps(projection["packet"], ensure_ascii=False)


def _runtime_reduction(params: Mapping[str, Any], *, phase: str) -> dict[str, Any]:
try:
value = effect_runtime_result(_RUNTIME_METHOD, params)
Expand Down
14 changes: 9 additions & 5 deletions loopx/control_plane/quota/spend_sources.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,9 @@
from typing import Any

from ..scheduler.execution_context import (
NATIVE_GOAL_RUNTIME_PROFILES,
SchedulerExecutionContextResolution,
SchedulerRuntimeProfile,
VISIBLE_GOAL_SETTLEMENT_RUNTIME_PROFILES,
scheduler_runtime_profile_for_execution_context,
)
from ..todos.contract import normalize_todo_id, normalize_todo_replan_obligation_id
Expand Down Expand Up @@ -41,12 +41,12 @@ def quota_spend_source_for_execution_context(
value: Mapping[str, Any] | SchedulerExecutionContextResolution | None,
) -> str:
profile = scheduler_runtime_profile_for_execution_context(value)
if profile in NATIVE_GOAL_RUNTIME_PROFILES:
if profile in VISIBLE_GOAL_SETTLEMENT_RUNTIME_PROFILES:
return VISIBLE_GOAL_SLOT_SPEND_SOURCE
return DEFAULT_SLOT_SPEND_SOURCE


def visible_goal_turn_reentry_action(
def host_goal_turn_reentry_action(
payload: Mapping[str, Any],
settlement_plan: Mapping[str, Any] | None,
scheduler_execution_context: (
Expand All @@ -62,12 +62,16 @@ def visible_goal_turn_reentry_action(
selected = selected_value if isinstance(selected_value, Mapping) else {}
replan_value = payload.get("replan_action_packet")
replan = replan_value if isinstance(replan_value, Mapping) else {}
replan_obligation_id = normalize_todo_replan_obligation_id(
replan.get("obligation_id")
)
has_settlement_binding = bool(
normalize_todo_id(selected.get("todo_id"))
or normalize_todo_replan_obligation_id(replan.get("obligation_id"))
or replan_obligation_id
)
requires_turn_reentry = profile in VISIBLE_GOAL_SETTLEMENT_RUNTIME_PROFILES
if (
profile in NATIVE_GOAL_RUNTIME_PROFILES
requires_turn_reentry
and has_settlement_binding
and settlement_plan is None
and turn_instance_id is None
Expand Down
11 changes: 11 additions & 0 deletions loopx/control_plane/scheduler/execution_context.py
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,17 @@ class GoalRuntimeContinuationDisposition(str, Enum):
}
)

# Interactive hosts in this set can carry one explicit Turn identity through
# accountable writeback and visible-Goal quota settlement. Claude's ordinary
# Todo delivery remains MCP-owned; this shared contract also permits its
# Todo-less replan re-entry to finish once a Turn has been supplied.
VISIBLE_GOAL_SETTLEMENT_RUNTIME_PROFILES = frozenset(
{
*NATIVE_GOAL_RUNTIME_PROFILES,
SchedulerRuntimeProfile.CLAUDE_CODE_VISIBLE,
}
)

GUIDED_START_TURN_RUNTIME_PROFILES = frozenset(
{
SchedulerRuntimeProfile.CODEX_APP_HEARTBEAT,
Expand Down
35 changes: 35 additions & 0 deletions loopx/control_plane/turn_driver/host_interaction.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
import type { JsonObject } from "../effect_program.ts";
import { visionAuthoringContract } from "../goals/vision_checkpoint.ts";

function object(value: unknown): JsonObject | null {
return value !== null && typeof value === "object" && !Array.isArray(value)
? value as JsonObject : null;
}

/** A transport lens, never a replacement for quota admission or vision rules. */
export function projectMcpInteraction(guard: JsonObject): JsonObject {
const contract = object(guard.interaction_contract);
if (guard.ok !== true || !contract) return guard;
const selected = object(guard.selected_todo);
const ownsDelivery = guard.normal_delivery_allowed === true &&
typeof selected?.todo_id === "string";
const cli = object(contract.cli_channel);
// The raw CLI sequence is the implementation of complete_task, not a second
// obligation. Replan-only and blocked lanes retain their actual CLI actions.
let channel = cli;
if (ownsDelivery && cli) {
const {settlement_plan: _plan, next_cli_actions: _actions, ...facts} = cli;
channel = {...facts, next_cli_actions: [], executor: "mcp_complete_task"};
}
return {...guard, interaction_contract: {...contract,
...(channel ? {cli_channel: channel} : {}),
mcp_channel: {
schema_version: "host_mcp_interaction_v0",
delivery_executor: "complete_task",
delivery_todo_id: ownsDelivery ? selected!.todo_id : null,
rule: "For verified Todo delivery, call complete_task INSTEAD OF manual refresh/spend. It owns lifecycle, writeback and accounting. For independent replan, follow the live CLI binding/actions; do not replay an old completion.",
vision_authoring: visionAuthoringContract(),
checkpoint_recovery_tool: "review_task_vision",
},
}};
}
Loading
Loading