Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .claude-plugin/plugin.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "loop-engineer",
"version": "0.6.1",
"version": "0.7.0",
"description": "Design, launch, verify, repair, and improve agent loops. A Claude-Code-native architect+operator for long-running, verifiable, self-improving agentic-coding systems.",
"author": { "name": "Sollan Systems", "url": "https://github.com/SollanSystems" },
"homepage": "https://github.com/SollanSystems/loop-engineer",
Expand Down
55 changes: 54 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,60 @@ All notable changes to `loop-engineer` are documented here.
`WORKFLOW.md` and `README.md` are reworded to describe the mechanism; the 0.3.4
history is left intact.

## Unreleased
## 0.7.0 — 2026-07-08

**ST2 — the portable standard.** The on-disk contract is now a documented,
versioned, tool-agnostic standard, not an implicit format one validator happens
to enforce. `reference/repo-os-contract.md` is promoted to the normative spec:
a stability note (§0 — `$id` majors of the form `loop-engineer/<artifact>@<major>`,
strictly additive within a major, breaking changes ship as a new major side by
side), an artifact/schema table across all 7 published schemas with required
keys read verbatim from `schemas/*.schema.json` (§11), the lifecycle vocabulary
and terminal-file-iff rule (§12), the repair-record vs rollout-record two-shape
clarification (§13), and a conformance checklist (§14, items A1–E1) any harness
can satisfy to claim it "emits a Loop-Engineer-conformant contract v1."

### Added
- **`doctor` lifecycle line** — `validate_contract` (and so `loop doctor`)
reports `lifecycle: planned | running | terminated:<State> | unknown`,
derived from `state.json` and the terminal file. Additive reporting only —
never an issue source — so an operator sees *why* no terminal file is
expected on an in-flight loop instead of being pushed to fabricate one.
DG-3 regression tests pin both directions in both validation modes: a
null-terminal loop without `terminal_state.json` is conformant; a non-null
`terminal_state` without the file still fails.
- **Round-trip template regression** (`scripts/test_template_roundtrip.py`) —
every `templates/*` artifact, filled with schema-valid values, passes
`validate_contract` with zero issues in both validation modes, for both an
in-flight and a terminated scaffold. The DG-class template↔validator↔schema
drift cannot silently return.
- **Runnable conformance checklist** (`scripts/test_conformance.py`) — executes
checklist items A1–E1 in CI against the flagship example
(`examples/coverage-repair`) and a fresh template scaffold, including
additive-key tolerance (D2) and lifecycle honesty (E1). A doc-parity test
pins every checklist ID to the normative doc so the checklist and its
documentation cannot drift apart.
- **README "A versioned, conformance-checkable standard"** — a pointer
subsection linking the promoted normative doc.

### Fixed (external-review patch set, PRs #27–#30)
- **doctor evidence and surface fixes (#27)** — an empty-evidence `Succeeded`
now fails validation in both modes (G1 cross-check); ledger validation is
scoped to the canonical rollout/receipt files instead of force-validating
foreign `.loop/*.jsonl`, and fails closed on corrupt UTF-8; the fallback YAML
parser no longer strips `#` inside quoted strings; missing verify scripts and
dangling task file targets are surfaced as issues.
- **atomic terminate (#28)** — `loop.emit.terminate` writes the terminal record
exactly once, atomically; a second call raises `EmitError` instead of
silently overwriting the loop's end record.
- **strict-by-install gates (#29)** — the GitHub Action and the pre-commit hook
install the `[schemas]` extra so consumer repos gate in real JSON-Schema
mode, not the structural fallback; the Action's PR comment is sticky and its
score parsing robust.
- **inspector scores execution evidence (#30)** — `loop inspect` credits
verification gates on execution evidence rather than keyword presence, so a
keyword-stuffed contract can no longer buy a "strong" scorecard while the
gate-backed flagship example keeps its score.

**B1 — the writer API.** `loop.emit` lets a foreign runtime (LangGraph, a plain
script, any orchestrator) record an evidence-backed loop contract without
Expand Down
14 changes: 11 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
[![CI](https://github.com/SollanSystems/loop-engineer/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/SollanSystems/loop-engineer/actions/workflows/ci.yml)
[![Python 3.10–3.12](https://img.shields.io/badge/python-3.10%E2%80%933.12-blue)](https://www.python.org/downloads/)
[![License: MIT](https://img.shields.io/badge/license-MIT-green)](LICENSE)
[![Release](https://img.shields.io/badge/release-0.6.1-blue)](https://github.com/SollanSystems/loop-engineer/tags)
[![Release](https://img.shields.io/badge/release-0.7.0-blue)](https://github.com/SollanSystems/loop-engineer/tags)

Long-running agents commit **false completion**. After context compaction they
forget what "done" meant, optimize to the visible test, patch in circles, and
Expand Down Expand Up @@ -238,6 +238,14 @@ maps to a success criterion, its verifier passes, and evidence is recorded.

See `reference/repo-os-contract.md` for the canonical artifact schemas.

### A versioned, conformance-checkable standard

The on-disk contract is a **documented, versioned, tool-agnostic standard** — conformance is
defined by the published `schemas/*.schema.json` (`$id` `loop-engineer/<artifact>@<major>`,
additive within a major), and a runnable **conformance checklist** (A1–E1) lets any harness claim
*"emits a Loop-Engineer-conformant contract v1."* See
[`reference/repo-os-contract.md`](reference/repo-os-contract.md) §0 / §11 / §14.

---

## Install
Expand Down Expand Up @@ -389,8 +397,8 @@ license, and README differentiation.

## Status

- Version: `0.6.1`
- Release tag: `v0.6.1` (PyPI publish trigger; plugin tags through 0.6.0 used `loop-engineer--v<version>`)
- Version: `0.7.0`
- Release tag: `v0.7.0` (PyPI publish trigger; plugin tags through 0.6.0 used `loop-engineer--v<version>`)
- License: MIT
- Primary interface: Claude Code plugin
- Portable core: Python CLI + JSON schemas
Expand Down
34 changes: 34 additions & 0 deletions loop/contract.py
Original file line number Diff line number Diff line change
Expand Up @@ -485,6 +485,37 @@ def _validate_optional_records(paths: LoopPaths, mode: str, issues: list[dict])
return checked


def _derive_lifecycle(state: Any, terminal: Any, terminal_exists: bool) -> str:
"""Report which lifecycle band a loop is in — additive reporting only, never
an issue source. Total and pure: never raises.

Per the ratified rule:
1. state parsed with a non-null terminal_state, OR terminal_state.json
present → ``terminated:<X>`` where X is the terminal file's ``state``
(dict + string) if available, else state.json's ``terminal_state`` if a
string, else ``unknown``.
2. state parsed and iteration_id is 0 / "0" → ``planned``.
3. state parsed → ``running``.
4. else → ``unknown``.
"""
state_is_dict = isinstance(state, dict)
terminal_state_val = state.get("terminal_state") if state_is_dict else None
if (state_is_dict and terminal_state_val is not None) or terminal_exists:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Reject stale terminal files for in-flight state

When state.json still has terminal_state: null but a stale .loop/terminal_state.json remains, this branch treats the contract as terminated and validate_contract() never emits an issue, so the report is ok: true/lifecycle: terminated:<state>. That violates the new terminal-file-iff/B1 rule in this commit, which requires exactly the in-flight arm (terminal_state null and no terminal file) or the terminated arm (non-null terminal state plus terminal file), and lets non-conformant contracts pass doctor.

Useful? React with 👍 / 👎.

if isinstance(terminal, dict) and isinstance(terminal.get("state"), str):
resolved = terminal["state"]
elif isinstance(terminal_state_val, str):
resolved = terminal_state_val
else:
resolved = "unknown"
return f"terminated:{resolved}"
if state_is_dict:
iteration_id = state.get("iteration_id")
if iteration_id == 0 or iteration_id == "0":
return "planned"
return "running"
return "unknown"


def validate_contract(target: str | Path) -> dict[str, Any]:
paths = resolve_loop_paths(target)
issues: list[dict] = []
Expand Down Expand Up @@ -530,11 +561,14 @@ def validate_contract(target: str | Path) -> dict[str, Any]:
schema_id for key, schema_id in _RECORD_SCHEMA_IDS if key in records_checked
]

lifecycle = _derive_lifecycle(state, terminal, paths.terminal.exists())

return {
"ok": not issues,
"paths": paths.to_json(),
"validation_mode": mode,
"schemas_checked": schemas_checked,
"lifecycle": lifecycle,
"issues": issues,
}

Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ build-backend = "hatchling.build"

[project]
name = "loop-engineer"
version = "0.6.1"
version = "0.7.0"
description = "Portable Loop Contract Core: validate and inspect repo-native operating contracts for agent loops."
readme = "README.md"
requires-python = ">=3.10"
Expand Down
158 changes: 158 additions & 0 deletions reference/repo-os-contract.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,36 @@ in `loop-patterns.md`; the safety/terminal semantics live in `safety-and-approva

---

## 0. The contract is a versioned, tool-agnostic standard

This document is the **normative standard** for the repo-OS contract. It is not a
description of one tool's private file format: it is a **portable, tool-agnostic on-disk
standard**. Any surface that can read a repo, run a shell command, and write files can emit or
consume it — Loop Engineer is the *reference implementation*, not the only permitted producer.

- **Conformance is defined by the published JSON Schemas** in `schemas/*.schema.json`, not by
any one validator's source code. Every schema-bearing artifact carries a `schema` key, and
every schema an `$id`, of the form **`loop-engineer/<artifact>@<major>`**
(e.g. `loop-engineer/state@1`). The major integer in that identifier is the version an
external emitter targets.
- **Within a major, changes are strictly additive and optional.** Every artifact schema sets
`"additionalProperties": true`, so a validator for major *N* accepts any artifact whose
required keys and types match major *N* and **ignores unknown keys** — a newer emitter's
extra fields never reject a valid v1 artifact. Adding an optional key, or a new optional
file, does not bump the major.
- **Breaking changes get a new major and a new `$id`.** Removing or renaming a required key,
changing a type, or tightening an enum ships as `loop-engineer/<artifact>@2` with a new
`$id`. Both majors may be published and validated **side by side**.
- **Stability tiers.** The artifact table (§11) records each artifact's tier. For v1:
**manifest / state / tasks / terminal are `stable`**; **receipt / repair-record /
rollout-record are `provisional`** (the newest surfaces, whose additive shape may still be
refined within `@1`).

A third-party harness whose output satisfies the §14 conformance checklist may claim it
**"emits a Loop-Engineer-conformant contract v1."**

---

## 1. The full repo-OS tree

```
Expand Down Expand Up @@ -381,6 +411,134 @@ architecture decision record that `[[loop-architect]]` emits.

---

## 11. Artifact & schema reference

Every schema-bearing artifact in the contract, its on-disk location, the schema that defines it,
its embedded `$id`, its **required keys** (read verbatim from `schemas/*.schema.json` — an
emitter MUST supply all of them), its lifecycle role, and its stability tier (§0). Required keys
are the floor; `additionalProperties: true` means an artifact may carry more.

| Artifact | Contract path | Schema file | `$id` | Required keys | Lifecycle role | Tier |
|---|---|---|---|---|---|---|
| manifest | `.loop/manifest.yaml` | `schemas/manifest.schema.json` | `loop-engineer/manifest@1` | `schema`, `loop`, `policies`, `terminal_states` | The explicit, machine-readable operating contract for one loop (§10). | **stable** |
| state | `.loop/state.json` | `schemas/state.schema.json` | `loop-engineer/state@1` | `schema`, `iteration_id`, `state`, `plan_version`, `budget_remaining` | The live FSM cursor — the source of machine truth for resume (§7). | **stable** |
| tasks | `TASKS.json` *(workspace root)* | `schemas/tasks.schema.json` | `loop-engineer/tasks@1` | `schema`, `tasks`; each task: `id`, `title`, `status`, `criterion_ref`, `verify`, `depends_on`, `attempts`, `evidence` | The machine-readable task queue (§5). | **stable** |
| terminal | `.loop/terminal_state.json` | `schemas/terminal.schema.json` | `loop-engineer/terminal@1` | `schema`, `state`, `criteria_met`, `evidence`, `false_completion` | The single end record, written once at loop end (§8). | **stable** |
| receipt | `.loop/receipts/*.jsonl` | `schemas/receipt.schema.json` | `loop-engineer/receipt@1` | `schema`, `iteration_id`, `role`, `model`, `outcome` | Append-one-per-line dispatch/cost trail (role vs model, cost-per-success). | *provisional* |
| repair-record | `.loop/repair/<iteration_id>.json` | `schemas/repair-record.schema.json` | `loop-engineer/repair@1` | `schema`, `iteration_id`, `attempt`, `failure_mode`, `hypothesis`, `repair_action`, `verification_before`, `verification_after`, `remaining_delta`, `productive` | One bounded repair pass (diagnosis shape); the canonical repair-productivity input (§13). | *provisional* |
| rollout-record | `.loop/rollout.jsonl` | `schemas/rollout-record.schema.json` | `loop-engineer/rollout@1` | `id`, `parent`, `verdict`, `score`, `score_delta`, `coherent_with_prior_winner`, `productive` | One candidate adjudication in a rollout / genetic-hardening ledger (§13). | *provisional* |

The rollout-record's required set is the only one that does **not** require a `schema` envelope
key (the ledger writer today emits bare records); the schema permits one via
`additionalProperties`, but does not demand it. `doctor` validates receipts and repair/rollout
records **only when the files are present** (§14 C1–C3): an in-flight loop that has not yet
produced a trail still conforms.

---

## 12. Lifecycle vocabulary

The 7 terminal states (§8) are the **frozen** set of ways a loop *ends*. Before it ends, a loop
also holds non-terminal lifecycle values while it is *scaffolded but not started* or *running*.
These non-terminal values are **not** terminal states and never appear in the 7-member
`terminal_state` enum. Two rules make an in-flight loop a first-class, conformant state.

### 12.1 The terminal-file-iff rule

`terminal_state.json` is required **iff** `state.json`'s `terminal_state` is non-null.

- While `state.json` reports `terminal_state: null`, the **absence** of `.loop/terminal_state.json`
is **conformant** — the loop is in-flight, not failing validation. (`validate_contract` gates
the terminal-file read on `state.terminal_state`; a null with no file is treated as an
in-flight loop, not a `missing_file` issue.)
- A non-null `terminal_state` **without** the terminal file is a `missing_file` failure.

**Why the iff, not "always require a terminal file":** a gate that demands a terminal record from
a live loop pushes an operator to *write a terminal state onto a loop that has not terminated* —
a fabricated end record. That is exactly the false completion this contract exists to prevent.
The iff rule removes the incentive: an honest in-flight loop is green without inventing an ending.

### 12.2 The `doctor` lifecycle line

`doctor` (`validate_contract`) adds a `lifecycle` field to its report so an operator sees *why*
no terminal file is expected. It is derived (total and pure — never an issue source) as:

1. **`terminated:<X>`** — if `state.json` parsed with a non-null `terminal_state`, **or**
`.loop/terminal_state.json` exists. `<X>` is the terminal file's `state` value when the file
parses to a dict with a string `state`; else `state.json`'s `terminal_state` when that is a
string; else `unknown`.
2. **`planned`** — else, if `state.json` parsed and its `iteration_id` is `0` (or `"0"`):
scaffolded, not yet run.
3. **`running`** — else, if `state.json` parsed: executing.
4. **`unknown`** — else (no parseable `state.json`).

`planned`, `running`, and `unknown` are lifecycle-report values only; none is a terminal state,
and no terminal state ever surfaces as one of them. The `terminated:<X>` form is the only overlap
point, and there `<X>` is always drawn from the frozen 7 (or `unknown`).

---

## 13. Two distinct record shapes — repair-record vs rollout-record

The repair-record and the rollout-record are **different artifacts** that share only a
`productive` boolean; they must not be conflated (this section exists so no one conflates them
again). They differ in shape, location, and what `productive` measures:

| | repair-record (`loop-engineer/repair@1`) | rollout-record (`loop-engineer/rollout@1`) |
|---|---|---|
| **Shape** | **Diagnosis** of one bounded repair pass | **Ledger** entry adjudicating one rollout candidate |
| **Location** | `.loop/repair/<iteration_id>.json` (one JSON object per file) | `.loop/rollout.jsonl` (append one JSON object per line) |
| **Key fields** | `failure_mode`, `hypothesis`, `repair_action`, `verification_before`, `verification_after`, `remaining_delta`, `productive` | `id`, `parent`, `verdict`, `score`, `score_delta`, `coherent_with_prior_winner`, `productive` |
| **`productive` means** | repair-productivity: `verification_after.score > verification_before.score` | rollout-productivity: `score_delta` is not null and `> 0` |
| **Feeds** | the repair-productivity metric / baseline (`loop-repair`) | the flywheel's candidate-hardening view (`loop-flywheel`) |

The repair-record is the diagnosis shape the repair skill prescribes and the eval structural
invariant pins; the rollout-record is genome/candidate bookkeeping. Publishing them as two `$id`s
resolves the historic "two 7-field shapes both called *the* repair record" ambiguity.

---

## 14. Conformance checklist

A harness that satisfies **every** item below may claim it **"emits a Loop-Engineer-conformant
contract v1."** Each item is a third-party-checkable statement against the published schemas.
Items **C1–C3 are checked-when-present** — an in-flight loop that has not yet emitted a receipt,
repair, or rollout trail still conforms. `scripts/test_conformance.py` executes this checklist in
CI against the flagship example ([`examples/coverage-repair`](../examples/coverage-repair)) and a
fresh template scaffold, so a drift between this doc, the schemas, and the shipped scaffold cannot
land silently.

**A. Artifacts present & well-formed**
- **A1** — `.loop/manifest.yaml` validates against `loop-engineer/manifest@1` (including the
canonical 7 `terminal_states`, verbatim and in order).
- **A2** — `.loop/state.json` validates against `loop-engineer/state@1`.
- **A3** — `TASKS.json` validates against `loop-engineer/tasks@1`; no duplicate task ids; no task
marked `done` without `evidence`.
- **A4** — `RUNLOG.md` is present.

**B. Lifecycle honesty**
- **B1** — Exactly one of: (`state.terminal_state` is null **and** no `terminal_state.json`) **or**
(`terminal_state` is one of the canonical 7 **and** `terminal_state.json` is present and valid).
- **B2** — `terminal_state.json`, when present, validates against `loop-engineer/terminal@1` with a
`criteria_met` object, an `evidence` list, and an explicit `false_completion` boolean; a
`Succeeded` terminal additionally has `false_completion=false`, at least one true criterion, and
non-empty `evidence`.

**C. Evidentiary trail (checked when present)**
- **C1** — every `.loop/receipts/*.jsonl` line validates against `loop-engineer/receipt@1`.
- **C2** — every `.loop/repair/*.json` validates against `loop-engineer/repair@1`.
- **C3** — `.loop/rollout.jsonl`, when present, validates against `loop-engineer/rollout@1`.

**D. Versioning**
- **D1** — every artifact's `schema` key names a published, current-major schema `$id`.
- **D2** — unknown keys are tolerated (additive fields never reject a v1 artifact).

**E. Lifecycle report**
- **E1** — `doctor` reports a `lifecycle` value consistent with B1: `terminated:<state>` iff the
terminal pair is present and valid; `planned` / `running` otherwise (§12.2).

---

Sources: "Designing a Loop Engineer Skill for Frontier Agent Workflows" (2026), synthesizing
Anthropic guidance on long-running agent harnesses (anthropic.com, 2025), OpenAI Agents/Codex guidance, Google
Conductor, and arXiv PreFlect (2602.07187), SWE-Marathon (2606.07682), Web Agents
Expand Down
Loading
Loading