Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .github/CODEOWNERS
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# Changes to what the gate checks require human review (ADR 0002, decision 6).
# schemas/ is included beyond the ADR's literal list: the contract schemas
# define what doctor accepts, so they are the same enforcement surface.
# Everything else in this repo stays autonomous.
/loop/ @SollanSystems
/schemas/ @SollanSystems
/action.yml @SollanSystems
/.github/workflows/ @SollanSystems
/.github/CODEOWNERS @SollanSystems
109 changes: 109 additions & 0 deletions .github/workflows/attest.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
name: attest

on:
push:
branches: [main]

permissions:
contents: read

jobs:
verdict:
# Push-to-default-branch only, by ADR 0002 decision 5: attesting on a PR
# would mint a signed verdict under the repository's identity before review.
runs-on: ubuntu-latest
permissions:
contents: read
id-token: write
attestations: write
steps:
- uses: actions/checkout@v7

- uses: actions/setup-python@v7
with:
python-version: "3.12"

- name: Install seed dependencies
run: python -m pip install --upgrade pip pyyaml jsonschema

- name: Seed a terminated, chained workspace
id: seed
# Through the REAL writer path — the runner's dispatch + auto-terminal —
# never a hand-written terminal (doctor flags those as desynced) and
# never a tracked example (no examples/* contract ships an events.db, so
# the attest step would skip and this job would pass having attested
# nothing: unfalsifiable, the exact false-completion shape this project
# refuses). Two dispatches: the first executes the one task and binds
# its evidence into the chain, the second fires the auto-terminal.
run: |
workspace="${RUNNER_TEMP}/verdict-ws"
head="$(python -B - "$workspace" <<'PY'
import json
import pathlib
import sys

sys.path.insert(0, ".")
from loop import emit
from loop.contract import doctor_report
from loop.events import SQLiteEventStore
from loop.runner import dispatch_once

ws = pathlib.Path(sys.argv[1])
emit.open_contract(ws)
task = {"id": "T-1", "title": "T-1", "status": "pending",
"criterion_ref": "T-1", "verify": "./scripts/verify-fast.sh",
"depends_on": [], "attempts": 0, "evidence": None}
(ws / "TASKS.json").write_text(
json.dumps({"schema": "loop-engineer/tasks@1", "tasks": [task]}),
encoding="utf-8")
script = ws / "scripts" / "verify-fast.sh"
script.parent.mkdir(parents=True, exist_ok=True)
script.write_text("#!/bin/sh\nexit 0\n", encoding="utf-8")
script.chmod(0o755)
store = SQLiteEventStore(ws / ".loop" / "events.db")
store.append("run-1", "contract_opened", {"workspace": ws.name}, actor="ci")
for state in ("plan", "critique-plan", "queue-tasks", "execute-task"):
store.append("run-1", "iteration_appended",
{"iteration_id": 0, "outcome": "replanned", "state": state},
actor="ci")
dispatch_once(ws)
dispatch_once(ws)
report = doctor_report(ws)
if not report["ok"]:
raise SystemExit(
f"seeded workspace is not doctor-clean: "
f"{[issue['code'] for issue in report['issues']]}")
head = (((report["event_store"].get("chain") or {}).get("head") or {})
.get("event_hash")) or ""
if not head:
raise SystemExit("seeded workspace has no chain head")
print(head)
PY
)"
echo "head=$head" >> "$GITHUB_OUTPUT"
echo "workspace=$workspace" >> "$GITHUB_OUTPUT"

- id: gate
uses: ./
with:
path: ${{ steps.seed.outputs.workspace }}
attest: "true"

- name: assert an attestation was actually created
# Without this the job passes when the attest step skips, which is the
# failure mode this whole task exists to avoid.
env:
SEEDED_HEAD: ${{ steps.seed.outputs.head }}
OBSERVED_HEAD: ${{ steps.gate.outputs.chain-head }}
ATTESTATION: ${{ steps.gate.outputs.attestation-url }}
run: |
if [ -z "$ATTESTATION" ]; then
echo "::error::attest was requested but no attestation URL was produced"
exit 1
fi
if [ "$OBSERVED_HEAD" != "$SEEDED_HEAD" ]; then
echo "::error::gate observed head '$OBSERVED_HEAD', seed produced '$SEEDED_HEAD'"
exit 1
fi
echo "chain head: $OBSERVED_HEAD" >> "$GITHUB_STEP_SUMMARY"
echo "attestation: $ATTESTATION" >> "$GITHUB_STEP_SUMMARY"
32 changes: 32 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,38 @@ All notable changes to `loop-engineer` are documented here.
`WORKFLOW.md` and `README.md` are reworded to describe the mechanism; the 0.3.4
history is left intact.

## Unreleased

**A verdict you can hand to a signer (slice 4a of tamper-evident provenance).**
The kernel gains `loop verdict <workspace>`: a pure projection of a finished
run — doctor verdict, chain head, terminal outcome, and the chain-bound
evidence digests that pass the strict verified-evidence bar — into one
canonical `loop-engineer/verdict@1` predicate body (`schemas/verdict.schema.json`,
normative in `reference/repo-os-contract.md` §23). Digests, enums, and issue
codes only; `run_id` is the single operator-controlled string; the field set is
an allowlist held by test. The kernel never signs, never builds an in-toto
Statement, and never reads an environment variable — `scripts/test_verdict_purity.py`
makes each boundary mechanical.

The composite action gains an opt-in `attest` input (default false): it writes
the predicate to the runner temp dir and hands it to `actions/attest` with
`subject-name: loop-chain-head` / `subject-digest: sha256:<chain-head>`,
exposing `attestation-url`/`attestation-id` outputs; a legible permission
precheck replaces the raw OIDC 403, and an empty chain head skips with a
warning rather than shipping a malformed subject. `.github/workflows/attest.yml`
mints a real attestation on every push to main over a workspace seeded through
the runner's own dispatch + auto-terminal path, and fails loud if no
attestation URL is produced or the observed head differs from the seeded one.

What this does not buy: the signature attests context — repo, workflow,
trigger, time — never correctness, so a signed verdict over a weakened gate is
just a signed weakened gate. An agent with ordinary merge rights can loosen
`loop/**`/`schemas/**`/`action.yml`/the workflow and then mint a perfectly
genuine attestation for the result — the control is code-owner review on those
paths, which is in force only once the repository ruleset requires it — and an
unattested chain rewrite is detected at best one run late. Verification (`--compare`, anchor auto-resolution, signer-trust policy)
is slice 4b and does not ship here.

## 0.11.0 — 2026-07-26

**Verifier identity, and evidence that is load-bearing.** Two slices of the
Expand Down
56 changes: 55 additions & 1 deletion action.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
name: "loop-engineer gate"
description: "Proof-of-done gate for agent-loop contracts: hard-fails on doctor, scores with inspect (warn-only by default)."
description: "Proof-of-done gate for agent-loop contracts: hard-fails on doctor, scores with inspect (warn-only by default), and can keylessly attest the run's verdict@1 predicate (opt-in)."
branding:
icon: "check-circle"
color: "green"
Expand Down Expand Up @@ -32,11 +32,26 @@ inputs:
detection — the gate then only records the head for a later comparison.
required: false
default: ""
attest:
description: >-
Emit a loop-engineer/verdict@1 predicate and attest it keylessly via
GitHub OIDC. Requires the CALLING JOB to declare `id-token: write` and
`attestations: write` — a composite action cannot declare its own
permissions. Default false. The predicate is PUBLIC and permanent for
public repositories.
required: false
default: "false"

outputs:
chain-head:
description: "Chain head event_hash observed by this gate run ('' when the store has no chained events)."
value: ${{ steps.chain-head.outputs.chain-head }}
attestation-url:
description: "URL of the attestation created by this run ('' when attest is false)."
value: ${{ steps.attest.outputs.attestation-url }}
attestation-id:
description: "ID of the attestation created by this run ('' when attest is false)."
value: ${{ steps.attest.outputs.attestation-id }}

runs:
using: "composite"
Expand Down Expand Up @@ -106,6 +121,45 @@ runs:
open(sys.argv[3], "a").write(f"chain-head={value}\n")
PY

- name: verdict predicate
id: verdict
if: ${{ inputs.attest == 'true' }}
shell: bash
env:
LOOP_PATH: "${{ inputs.path }}"
run: |
# Fail with a legible message rather than a raw OIDC 403 from the attest
# step: a composite action cannot declare permissions, so the CALLING
# job must. ACTIONS_ID_TOKEN_REQUEST_URL is present only when the job
# declared id-token: write.
if [ -z "${ACTIONS_ID_TOKEN_REQUEST_URL:-}" ]; then
echo "::error::attest: true requires the calling job to declare" \
"'permissions: { id-token: write, attestations: write }'." \
"A composite action cannot declare its own permissions."
exit 1
fi
loop verdict "$LOOP_PATH" > "${RUNNER_TEMP}/verdict.json"
echo "predicate-path=${RUNNER_TEMP}/verdict.json" >> "$GITHUB_OUTPUT"

- name: attest verdict
id: attest
if: ${{ inputs.attest == 'true' && steps.chain-head.outputs.chain-head != '' }}
uses: actions/attest@v4
with:
subject-name: loop-chain-head
subject-digest: sha256:${{ steps.chain-head.outputs.chain-head }}
predicate-type: urn:loop-engineer:verdict:1
predicate-path: ${{ steps.verdict.outputs.predicate-path }}
push-to-registry: false
# Only effective when push-to-registry is true, but pinned false
# explicitly per ADR 0002 open item 5: no storage record, ever.
create-storage-record: false

- name: attest skipped (no chained events)
if: ${{ inputs.attest == 'true' && steps.chain-head.outputs.chain-head == '' }}
shell: bash
run: echo "::warning::attest requested but the store has no chained events; nothing to attest."

- name: loop inspect (scorecard)
shell: bash
env:
Expand Down
79 changes: 61 additions & 18 deletions docs/superpowers/plans/2026-07-25-slice4a-verdict-emission.md
Original file line number Diff line number Diff line change
Expand Up @@ -311,10 +311,10 @@ import pytest

def _scaffold_terminal(tmp_path, name, state="Succeeded", policy="all_required"):
"""A doctor-clean scaffold advanced to a terminal record."""
from loop.contract import scaffold_contract # existing scaffold entry point
from loop.scaffold import scaffold

target = tmp_path / name
scaffold_contract(target)
scaffold(target)
terminal = {
"schema": "loop-engineer/terminal@1",
"state": state,
Expand Down Expand Up @@ -357,11 +357,11 @@ def test_issue_codes_are_sorted_deduplicated_and_carry_no_detail(tmp_path):


def test_build_verdict_refuses_a_workspace_with_no_terminal_record(tmp_path):
from loop.contract import scaffold_contract
from loop.scaffold import scaffold
from loop.verdict import VerdictError, build_verdict

target = tmp_path / "no-terminal"
scaffold_contract(target)
scaffold(target)

with pytest.raises(VerdictError, match="no terminal record"):
build_verdict(target)
Expand All @@ -379,7 +379,7 @@ def test_build_verdict_refuses_a_nonexistent_target(tmp_path):
Run: `uv run --with pyyaml --with jsonschema --with pytest python3 -B -m pytest -q -p no:cacheprovider scripts/test_verdict.py -v`
Expected: FAIL with `ImportError: cannot import name 'build_verdict'`

> If `scaffold_contract` is not the exported scaffold name at HEAD, run `uv run --with pyyaml python3 -B -c "import loop.contract as c; print([n for n in dir(c) if 'scaffold' in n])"` and use the real one in the helper. Do not invent an API.
> Verified at HEAD: the scaffold entry point is `loop.scaffold.scaffold(target)` (`loop/scaffold.py:104`). There is no `scaffold_contract`. `LoopPaths` also exposes `.terminal` directly (`loop/paths.py:24`), so prefer `paths.terminal` over `paths.loop_dir / "terminal_state.json"`.

- [ ] **Step 3: Implement the projection**

Expand Down Expand Up @@ -1033,10 +1033,19 @@ git commit -m "feat(action): opt-in keyless attestation of the verdict predicate
- Create: `.github/workflows/attest.yml`

**Interfaces:**
- Consumes: the composite action from Task 6.
- Produces: a real attestation on every push to the default branch, over a **tracked** `examples/*` contract.
- Consumes: the composite action from Task 6; the existing `scripts/ci_anchor_probe.py`.
- Produces: a real attestation on every push to the default branch, over a **seeded chained workspace**.

Never point this at the live gitignored `.loop/` — CI runs on a fresh checkout where it does not exist.
**Pre-flight correction.** An earlier draft pointed this job at `examples/flaky-test-triage`. That is
wrong: **no tracked `examples/*` contract ships an `events.db`** (verified — event stores are runtime
artifacts and `.loop/` is gitignored). With no store the chain head is empty, Task 6 Step 4's guard
skips the attest step, and the job goes green having attested nothing — an unfalsifiable CI job, which
is the exact false-completion shape this project exists to refuse.

Reuse the pattern the repo already proved. `ci.yml`'s `chain anchor (live end-to-end)` job seeds a
chained workspace via `python -B scripts/ci_anchor_probe.py "$workspace"`, which prints the resulting
head; its own comment states the rationale verbatim — *"action-dogfood gates a store-free example, so
the anchor surface has no live cover there."* Never point this at the live gitignored `.loop/`.

- [ ] **Step 1: Write the workflow**

Expand All @@ -1060,25 +1069,59 @@ jobs:
id-token: write
attestations: write
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7

- uses: actions/setup-python@v7
with:
python-version: "3.12"

- name: Install probe dependencies
run: python -m pip install --upgrade pip pyyaml jsonschema

- name: Seed a chained workspace
id: seed
# Same probe the chain-anchor job uses. A store-free example would make
# the attest step skip and this job unfalsifiable.
run: |
workspace="${RUNNER_TEMP}/verdict-ws"
head="$(python -B scripts/ci_anchor_probe.py "$workspace")"
echo "head=$head" >> "$GITHUB_OUTPUT"
echo "workspace=$workspace" >> "$GITHUB_OUTPUT"

- id: gate
uses: ./
with:
path: examples/flaky-test-triage
path: ${{ steps.seed.outputs.workspace }}
attest: "true"
- name: record

- name: assert an attestation was actually created
# Without this the job passes when the attest step skips, which is the
# failure mode this whole task exists to avoid.
env:
SEEDED_HEAD: ${{ steps.seed.outputs.head }}
OBSERVED_HEAD: ${{ steps.gate.outputs.chain-head }}
ATTESTATION: ${{ steps.gate.outputs.attestation-url }}
run: |
echo "chain head: ${{ steps.gate.outputs.chain-head }}" >> "$GITHUB_STEP_SUMMARY"
echo "attestation: ${{ steps.gate.outputs.attestation-url }}" >> "$GITHUB_STEP_SUMMARY"
if [ -z "$ATTESTATION" ]; then
echo "::error::attest was requested but no attestation URL was produced"
exit 1
fi
if [ "$OBSERVED_HEAD" != "$SEEDED_HEAD" ]; then
echo "::error::gate observed head '$OBSERVED_HEAD', seed produced '$SEEDED_HEAD'"
exit 1
fi
echo "chain head: $OBSERVED_HEAD" >> "$GITHUB_STEP_SUMMARY"
echo "attestation: $ATTESTATION" >> "$GITHUB_STEP_SUMMARY"
```

- [ ] **Step 2: Confirm the checkout action major matches the repo's other workflows**
- [ ] **Step 2: Confirm the action majors match the repo's other workflows**

```bash
grep -rn "actions/checkout@" .github/workflows/
grep -rn "actions/checkout@\|actions/setup-python@" .github/workflows/
```

Use whatever major `ci.yml` uses. A mismatched pin is a dependabot PR waiting to happen.
Pre-flight observed `actions/checkout@v7` and `actions/setup-python@v7` in `ci.yml`; confirm before
committing. A mismatched pin is a dependabot PR waiting to happen.

- [ ] **Step 3: Commit**

Expand Down Expand Up @@ -1227,6 +1270,6 @@ Carried from ADR 0002. None blocks starting; each blocks the task that touches i

1. **Subject digest algorithm** (Task 6). The chain head is not a hash of retrievable bytes, so the `sha256` DigestSet key licenses a false inference. A namespaced key is correct in in-toto terms but `actions/attest`'s `subject-digest` may accept only `sha256:`. Resolve by experiment. If the input constrains us, §23 must carry the disambiguation instead.
2. **`create-storage-record` / `push-to-registry` defaults** (Task 6). Pass both explicitly so the two-permission claim is true by construction.
3. **`scaffold_contract` name** (Task 2 Step 2). Confirm the real scaffold entry point before writing the test helper.
4. **`event_store` key names** (Task 2 Step 5). Confirm `run_id`, `chain.head.event_hash`, `chain.head.sequence`, `chain.unchained_prefix` against real doctor output.
3. ~~`scaffold_contract` name~~ — **RESOLVED in pre-flight.** The entry point is `loop.scaffold.scaffold`; the plan now uses it.
4. **`event_store` key names** (Task 2 Step 5). Pre-flight confirmed the store-absent shape is exactly `{"present": false}` and that `doctor_report` also returns `paths` (absolute filesystem paths — deliberately excluded from the predicate) and `requested_mode`. `validation_mode` is the single token `"jsonschema"`, so Task 5's no-whitespace assertion is safe. The populated `chain.head.*` key names still need confirming against a seeded store.
5. **The chain-bound evidence map producer** (Task 3 Step 4). Reuse `loop/contract.py`'s, never re-derive.
Loading
Loading