feat: verdict@1 emission — loop verdict, opt-in keyless attestation, mechanical ADR 0002 boundary (slice 4a) - #106
Conversation
Three defects found by scanning the plan against the real repo before dispatching any implementer: - The Task 2 test helper called loop.contract.scaffold_contract, which does not exist. The scaffold entry point is loop.scaffold.scaffold (loop/scaffold.py:104). - Task 7 attested examples/flaky-test-triage, but no tracked examples/* contract ships an events.db — event stores are runtime artifacts and .loop/ is gitignored. An empty chain head makes Task 6 Step 4 skip the attest step, so the job would pass having attested nothing. It now seeds a chained workspace with the existing scripts/ci_anchor_probe.py (the pattern ci.yml chain-anchor already uses) and fails when no attestation URL is produced or the observed head differs from the seeded one. - Recorded the verified doctor_report shape: it also returns paths (absolute, and deliberately excluded from the predicate) and requested_mode; validation_mode is the single token jsonschema, so the no-whitespace assertion in Task 5 is safe.
Task 1 of Slice 4a (ADR 0002). Adds schemas/verdict.schema.json, the loop/verdict.py skeleton (VERDICT_SCHEMA_ID, PREDICATE_TYPE, VerdictError, _load_verdict_schema), and three tests. The kernel builds a predicate body; it never signs one and never constructs an in-toto Statement. Every digest field pairs a 64-hex pattern with maxLength 64, because jsonschema pattern matching uses re.search semantics and would otherwise accept a trailing newline. Authored via the Claudex lane (gpt-5.6-terra/medium, session 019f9c74), verified outside the worker: scope 3/3, full suite 1308 passed / 18 skipped against a 1305/18 baseline, independent claude-sonnet-5 review PASS. Receipt cx_s4a_t1_verdict_schema_a1.
Task 2 of Slice 4a (ADR 0002). Adds build_verdict(target, *, mode=None), projecting a workspace's doctor report, event-chain head, and terminal record into a verdict@1 predicate body. Evidence stays empty until Task 3. Fail-closed on the terminal record: a false_completion that is absent or not a bool raises VerdictError rather than projecting False. The predicate is signed and written to a public transparency log, so a missing safety flag is unprojectable, not reassuring. isinstance(x, bool) is deliberate -- isinstance(True, int) is True, so a truthiness check would admit 0 and 1. RuntimeError joins the resolution guard, matching the convention at evidence.py:192,295 and verifier.py:76-77. It is tested by monkeypatching the binding loop.verdict actually calls, not through a filesystem symlink loop: resolve_loop_paths resolves non-strictly and never raises for a loop, and strict resolution yields OSError on this interpreter. Authored via the Claudex lane (gpt-5.6-terra, sessions 019f9ea1 / 019f9eaa / 019f9f7c / 019f9f87). Verified outside the worker: scope clean, suite 1320 passed / 18 skipped against a 1314 baseline, and five mutation probes killed on a green baseline -- stubbed schema loader, nulled chain head, emptied issue codes, reverted fail-open false_completion, and RuntimeError dropped from the except tuple. Independent claude-sonnet-5 review PASS on all eight criteria. Receipt cx_s4a_t2b_verdict_correction_a2.
… invariants Closes the three low follow-ups left open by Task 2. Split the resolution guard from the doctor_report guard. ValidationModeError subclasses RuntimeError, so a single guard reported an invalid mode= as 'cannot resolve a loop workspace' -- a misleading frame for an argument error. Both paths still raise VerdictError, so the fail-closed posture is unchanged; only the diagnostic improves. Add None to the non-boolean false_completion cases. JSON null round-trips to None, so the key is present and only the isinstance check stands between it and a signed false claim -- this is the exact value the original bool(None) is False defect turned on, and it was proven by inspection but untested. Assert PREDICATE_TYPE is derived from VERDICT_SCHEMA_ID. ADR 0002 chose a URN matching the schema $id; a URN cannot idiomatically carry '/' or '@', so the mapping is a transliteration and nothing stopped the two from drifting apart. Governor-authored and gated mechanically, not lane-delegated: full suite 1323 passed / 18 skipped against a 1320 baseline, and three mutation probes killed -- re-merging the guards fails the invalid-mode test, drifting PREDICATE_TYPE fails the derivation test, and reverting to fail-open fails all three parametrized cases including None.
… bar The digest projected for each evidence entry is the sha256 of the evidence record FILE BYTES - the value the event chain committed - never the record's sha256 field, which hashes the cited artifact instead (the plan sketch lifted a 'digest' field evidence@1 does not have). Only entries that clear loop.contract._strict_evidence_failure project; an unreadable event store projects an empty list fail-closed rather than laundering into the store-less degradation; output is de-duplicated and canonically sorted. Claudex lane s4a-t3-evidence-digests, gpt-5.6-terra/medium attempt 1 accepted: scope 2/2, targeted 26 exact, extras 1331/18 exact (+8), pyyaml-only 1236/113 (+8), purity clean, mutation probes M1-M5 all KILLED, fresh claude-sonnet-5 review PASS (0 blockers). Receipt cx_s4a_t3_evidence_digests_a1. Claude-Session: https://claude.ai/code/session_01JK76jSm45nHcoRoP1SdxXF
…ed digest Closes the two substantive review minors from the T3 acceptance. The strict bar validates its own read of an evidence record; _evidence_digests then read the file a second time, leaving a window where the projected digest could diverge from the digest the chain committed. The projection now drops any entry whose second-read digest is not exactly the chain's one-element binding (store-less workspaces keep the documented degradation). Also adds the populated-evidence schema validation test the suite lacked (importorskip'd so the structural-fallback leg stays honest). Governor-authored, not lane-dispatched: the review itself prescribed the hardening and the change is eleven lines. Held to the follow-up evidence standard: targeted 28, extras 1333/18 (+2), pyyaml-only 1237/114 (+1 pass +1 skip), probes M6 (check removed -> TOCTOU test fails) and M7 (extra key leaked -> shape + schema tests fail) both KILLED, tree restored intact. Claude-Session: https://claude.ai/code/session_01JK76jSm45nHcoRoP1SdxXF
python3 -m loop verdict [--mode basic|strict|release] <workspace> prints canonical_json(build_verdict(...)) to stdout and exits 0; every projection failure - VerdictError AND canonical_json's ChainHashError - degrades to one typed 'verdict: ...' stderr line and exit 2. Registered at all five CLI surfaces (_COMMANDS, _READ_COMMANDS, _USAGE, the --mode extraction set, _HELP with the signer-boundary description and content summary). Claudex lane s4a-t4-verdict-cli, gpt-5.6-terra, attempt 2 accepted after one verifier-directed repair: attempt 1 passed every deterministic gate but the fresh review caught the dispatch catching only VerdictError while a NaN-bearing terminal record (json.loads accepts NaN) crashes canonical_json with a raw ChainHashError traceback - governor-reproduced, then repaired with a regression test plus a valid --mode success-path test proving flag threading. Gates: targeted 11 exact, extras 1344/18 exact, pyyaml-only 1248/114, probes M1-M6 all KILLED. Receipts cx_s4a_t4_verdict_cli_a1 (repair_requested) / _a2 (accepted). Claude-Session: https://claude.ai/code/session_01JK76jSm45nHcoRoP1SdxXF
…al file The T4 re-review advisory (R-101) claimed a raw UnicodeDecodeError escapes _terminal_record; independent reproduction REFUTED the reachability - the doctor_report wrapper converts it first (its except tuple carries ValueError, and UnicodeDecodeError is one), so a guard widen there would be dead code and was not kept. What is worth pinning is the invariant itself: build_verdict on an invalid-UTF-8 terminal raises VerdictError at whichever site converts, and the new test holds under projection reorder. Teeth proven: narrowing the doctor wrapper's ValueError makes it fail (M8 KILLED, tree restored). The diagnosis surfaced the REAL pre-existing gap, out of 4a scope: doctor_report itself raises raw UnicodeDecodeError on an undecodable terminal file, so 'loop doctor' tracebacks (exit 1) instead of reporting a typed issue - recorded for an issue at PR time. Test-only change; loop/ byte-identical. Targeted 29, extras 1345/18 (+1), pyyaml-only 1249/114 (+1). Claude-Session: https://claude.ai/code/session_01JK76jSm45nHcoRoP1SdxXF
Six purity tests so the kernel/signer boundary cannot decay into an intention: no signing-stack token anywhere under loop/, no environment read, verdict.py imports only stdlib + loop.*, the emitted predicate holds the exact field allowlist at every level, carries no free text but run_id (whitespace as the prose proxy), and validates against its own schema. Governor-authored plan-verbatim (Task 5 ships its full test text in the plan; zero judgment surface), with one deviation: the schema-validation test uses pytest.importorskip, not bare __import__ - the plan's version ERRORS instead of skipping in the structural-fallback environment (plan defect five). Teeth proven per the plan's own probe step, extended: P1 os.environ read in verdict.py, P2 extra returned field, P3 prose string leak - all KILLED, tree restored byte-identical. Targeted 6; extras 1351/18 (+6); pyyaml-only 1254/115 (+5 passed, +1 honest skip). Claude-Session: https://claude.ai/code/session_01JK76jSm45nHcoRoP1SdxXF
New attest input (default false): the gate emits loop verdict's predicate body to RUNNER_TEMP and hands it to actions/attest@v4 with subject-name loop-chain-head + subject-digest sha256:<chain-head>, exposing attestation-url and attestation-id outputs. A legible permission precheck fails before the OIDC 403 would (ACTIONS_ID_TOKEN_REQUEST_URL is present only when the calling job declared id-token: write - a composite action cannot declare its own permissions), an empty chain head skips the attest step with a warning instead of shipping a malformed sha256: subject, and a doctor failure never reaches the attest steps at all. ADR 0002 open items 1 and 5 resolved against the LIVE actions/attest surface (gh api fetch of its action.yml, not memory or docs): subject-digest requires exactly the algorithm:hex form; subject-name is required alongside it; predicate-type + predicate-path are current; v4 is the current major; create-storage-record EXISTS (default true, effective only with push-to-registry true) and is pinned false explicitly. Governor-authored: plan-specified YAML whose one judgment surface WAS the live input verification, which a sandboxed worker cannot perform; the runtime experiment is Task 7's fail-loud CI job. YAML parse + structural lint green; step order chain-head -> verdict -> attest -> skipped-warning preserved. Claude-Session: https://claude.ai/code/session_01JK76jSm45nHcoRoP1SdxXF
… to main The job seeds its subject through the REAL writer path - contract + one task + event ramp, then two dispatch_once calls (execute-and-bind, then the runner's auto-terminal) - because the two shapes the plan considered do not survive contact: no tracked examples/* ships an events.db (empty head -> the attest step skips -> an unfalsifiable green job), and scripts/ci_anchor_probe.py seeds a chained-but-unterminated workspace where loop verdict refuses (no terminal record) - and hand-terminating it leaves doctor dirty (state_field_mismatch + desynced_terminal_window), which the composite doctor hard-gate would fail. The runner path was proven locally: doctor ok with zero issues, terminal Succeeded under all_required_verified_evidence, and the attested predicate carries a real chain-bound evidence digest. The job fails loud when no attestation URL is produced or the gate-observed head differs from the seeded head. Push-to-main only (ADR 0002 decision 5); job-scoped id-token/attestations permissions; checkout/setup-python pinned v7 matching ci.yml. Cannot execute before merge - first post-merge run is the live experiment; a failure there is a follow-up PR, not a revert (attest defaults false everywhere else). Claude-Session: https://claude.ai/code/session_01JK76jSm45nHcoRoP1SdxXF
reference/repo-os-contract.md gains an appended section 23 (no new reference/ file - the structural pin holds the list at eight): the predicate shape with a machine-pinned conformance vector, field-by-field semantics, the refusal/ degradation table (terminal required; store-less projects honestly; unreadable store projects empty evidence - an errored check fails, never skips), the urn:loop-engineer:verdict:1 identity and why it names no vendor host, and the subject-seam disambiguation ADR 0002 open item 1 required once actions/attest constrained the digest key: the chain head is a SHA-256 over a synthesized event preimage, not retrievable bytes - a consumer must never fetch-rehash- compare. The honest-limits block carries the ADR's standing limits in substance: context not correctness, the worker-can-edit-the-verifier path with CODEOWNERS as the control, fabricated-history indistinguishability, the one-run detection latency, and attestation-as-decoration until 4b consumes it. CHANGELOG Unreleased entry in the same voice; the phrase tamper-proof appears nowhere. Gates: self_eval 13/13, frontmatter 9/9, extras 1351/18, pyyaml-only 1254/115 - docs-only, counts unchanged. Claude-Session: https://claude.ai/code/session_01JK76jSm45nHcoRoP1SdxXF
CODEOWNERS covers loop/, action.yml, and .github/workflows/ - the three paths that define what the gate checks - per ADR 0002 decision 6: this is the only control for the largest standing limit (a worker with merge rights can loosen the gate, then mint a genuine attestation for it). The file alone enforces nothing until the operator flips the main-protection ruleset to require code-owner review; until then decision 6 is documented, not in force, and the PR body records that state. Claude-Session: https://claude.ai/code/session_01JK76jSm45nHcoRoP1SdxXF
R-001 (p0): the CHANGELOG and section-23 limits language claimed code-owner review as the operative control while the live main-protection ruleset does not yet require it (verified: require_code_owner_review false, approvals 0). Both now state the control is in force only once the ruleset requires code-owner review - a CODEOWNERS file the ruleset does not enforce is documentation, not a control. The PR body carries the operator action. R-002 (high): /schemas/ added to CODEOWNERS and to the loosen-the-gate path list in both docs - the contract schemas define what doctor accepts as ok and what the strict bar accepts as verified evidence, so they are the same enforcement surface as loop/ (an extension beyond ADR decision 6's literal three paths, consistent with its threat model). R-003 (low): the action's marketplace description now names the opt-in attestation capability. R-004 (seed-logic triplication) recorded as a non-blocking follow-up candidate in the PR body. Claude-Session: https://claude.ai/code/session_01JK76jSm45nHcoRoP1SdxXF
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 67ee119980
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| "unchained_prefix": int(chain.get("unchained_prefix") or 0), | ||
| }, | ||
| "terminal": { | ||
| "state": terminal.get("state"), |
There was a problem hiding this comment.
Reject terminal states outside the verdict schema
When terminal_state.json omits state or contains an invalid value while retaining a Boolean false_completion, doctor_report() records an issue but this projection still emits null or the invalid string and loop verdict exits 0. The emitted document then violates verdict.schema.json, whose terminal.state is a required enum, so schema-validating consumers cannot process the purported verdict@1; validate this field and return VerdictError (or otherwise ensure the projection remains schema-conformant).
Useful? React with 👍 / 👎.
…a missing store (#108) The limits-that-become-tests list said the verb refuses on 'a workspace with no event store or terminal record'. The shipped semantics - per the plan's binding predicate shape, the nullable chain.head schema, Task 4/5's tests on the store-free example, and the action's own empty-head skip guard - refuse only on a missing or invalid terminal record; a store-less workspace projects honestly with a null chain head. Adjudicated during the slice 4a whole-branch review (PR #106 body, point 3); section 23 already documents the shipped behavior, this aligns the ADR sentence with it. Claude-Session: https://claude.ai/code/session_01JK76jSm45nHcoRoP1SdxXF
…traceback (#109) Closes #107. _read_json catches UnicodeDecodeError and reports it as the existing invalid_encoding issue - the same rule _validate_jsonl already applies to ledgers - so loop doctor on a terminal_state.json holding invalid UTF-8 returns ok:false with a typed issue instead of tracebacking (exit 1). read_manifest gains the matching guard, failing safe to {} exactly like its malformed-YAML branch. loop/verdict.py's _terminal_record widens its read guard to include UnicodeDecodeError: that path became REACHABLE the moment doctor_report stopped raising, and the site-agnostic typed-contract test planted in 3e26e91 flips to depending on it - probe-proven (reverting the widen fails exactly that test). TDD red->green; probes P1/P2/P3 (each site reverted) all KILLED, tree restored; extras 1353/18 (+2), pyyaml-only 1256/115 (+2), zero regressions. Touches loop/ - a CODEOWNERS gate-defining path; the ruleset half of ADR 0002 decision 6 is not yet in force (PR #106 body point 2), and this lands under the repo's standing autonomous-merge procedure with the full evidence bar. Claude-Session: https://claude.ai/code/session_01JK76jSm45nHcoRoP1SdxXF
Collects slice 4a (#106) and slice 4b (#110, #111, #112): loop verdict, opt-in keyless attestation, --compare, replay-based chain ancestry, anchor@1, signer trust, and the head-bearing subject. Neither slice shipped on its own; both sat in Unreleased and go out under one number. Beyond the version surfaces, this cut carries corrections a mechanical bump would not have made. The changelog said the predicate is handed to actions/attest as a subject-path. It is handed as predicate-path, alongside a SEPARATE subject file, and §23 says outright that the predicate bytes are deliberately not the subject — doctor validation_mode and tool.version live inside the predicate, so the same run projects different bytes in different environments. A reader following that sentence would hash the wrong file and fail every verification. "the three attestations minted before this release": the count was right, the scope word was wrong. attest.yml shipped inside this same window, so nothing predates the release; the three carrying the retired subject form are the pushes through c493804. Re-scoped to match what attest.yml and reference §24 already said. ADR 0002 decision 6 is withdrawn, in a new dated amendment. It described code-owner review as in force once the ruleset required it. The ruleset cannot require it: one collaborator, no self-approval, bypass_actors empty, current_user_can_bypass never, and no classic protection underneath. Flipping it would invert the intent rather than merely fail — maintainer-authored pull requests become unmergeable while bot-authored ones become gated, and agent work here lands under the maintainer's account. CODEOWNERS is demoted in place to what it honestly is, a record of the gate-defining surface, across the ADR, the changelog, reference §23 and §24, and the file's own header. test_docs_version.py's changelog check now ADDS 0.12.0 and keeps 0.11.0, and restores 0.10.0 — the previous cut replaced its predecessor's assertion instead of adding to it, leaving that heading with no cover. Baselines, live checkout: 1577 passed / 16 skipped with pyyaml+jsonschema+pytest, 1478 / 115 with pyyaml+pytest. Both legs +7 against pre-cut, which is the CLI guard's new tests plus the retired version-bump pin offset by the new code-owner pin. CI additionally installs hypothesis, which moves the number.
Slice 4a —
verdict@1emission (tamper-evident provenance 4/5, part a)Implements ADR 0002 decisions 1–3 and the emission half of the program: the
kernel projects a finished run into a
loop-engineer/verdict@1predicate body;the CI lane signs it. Slice 4b (
--compare, anchor auto-resolution,signer-trust policy) is explicitly out of scope — no bundle parsing, no
verification, no
--signanywhere in this diff.What ships
loop/verdict.py—build_verdict(): doctor verdict + chain head +terminal outcome + the chain-bound evidence digests that pass the strict
verified-evidence bar (
_strict_evidence_failurereused, never restated).Fail-closed store semantics: store-less projects honestly (
chain.head: null, the bar's documented degradation); an unreadable store projectsevidence: []— an errored check fails, it never skips. Typed refusals formissing/corrupt terminal records; a second-read digest that is not exactly
the chain's one-element binding is dropped (TOCTOU).
evidence[].digestsemantics correction — the plan's sketch liftedrecord.get("digest"), a field evidence@1 does not have. The digest is thesha256 of the evidence record file bytes — the value the chain committed
(ADR 0002 decision 1) — proven equal to
bound_artifact_digests(...)[entry][0]by a real-fixture test and deliberately unequal to
record["sha256"](thecited artifact's hash).
loop verdictverb — canonical JSON to stdout, exit 0; every projectionfailure (
VerdictErrorandcanonical_json'sChainHashError) degrades toone typed
verdict:stderr line, exit 2.scripts/test_verdict{,_cli,_purity}.py— 46 tests across projection,CLI, and the mechanical ADR boundary (no signing token, no env read, stdlib
imports only, field allowlist, no free text but
run_id, schemaconformance). 14 mutation probes were run across the slice; every one
KILLED by a named test.
action.yml— opt-inattestinput (default false) →actions/attest@v4withsubject-name: loop-chain-head,subject-digest: sha256:<chain-head>,predicate-type: urn:loop-engineer:verdict:1;push-to-registryandcreate-storage-recordboth pinned false; a legible permission precheck replaces the raw OIDC 403;
an empty chain head skips with a warning. Input names were read from the
live actions/attest definition (
gh api .../contents/action.yml), notfrom memory or docs.
.github/workflows/attest.yml— push-to-main only (ADR decision 5).Seeds its subject through the runner's real dispatch + auto-terminal path
and fails loud when no attestation URL is produced or the observed head
differs from the seeded head.
reference/repo-os-contract.md§23 — normative predicate spec with amachine-pinned conformance vector, the subject-seam disambiguation (the
chain head is a hash of a synthesized event preimage, not retrievable
bytes — never fetch-rehash-compare), and the honest limits.
.github/CODEOWNERS—loop/,schemas/,action.yml,.github/workflows/(ADR decision 6;schemas/is a deliberate extensionbeyond the ADR's literal three paths — the contract schemas define what
doctor accepts, so they are the same enforcement surface).
gh api .../codeowners/errors?ref=feat/verdict-emission→[].Verification
Every task went through deterministic gates + fresh independent review before
landing. Final branch state (fresh tmpfs worktree): extras 1351 passed / 18
skipped, pyyaml-only 1254 / 115 (vs 1305/18 and 1228/113 at the branch
point — +46/+2 honest skips, zero regressions),
self_eval13/13,validate_frontmatter9/9. Lane receipts:cx_s4a_t1_verdict_schema_a1,cx_s4a_t2_*(4 attempts),cx_s4a_t3_evidence_digests_a1,cx_s4a_t4_verdict_cli_a1/_a2.Things reviewers must know
.github/workflows/attest.ymlcannot run before merge (push: branches: [main]). The first post-merge run is the live experiment forthe attest wiring; if it fails, the fix is a follow-up PR, not a revert —
the
attestinput defaults to false everywhere else.main protectionruleset still requires 0 approvals and does not requirecode-owner review, so CODEOWNERS currently enforces nothing. Operator
action: Settings → Rules →
main protection→ require review from CodeOwners, approvals ≥ 1. Until then decision 6 is documented, not in force.
loop verdictrefuses on "a workspace with no event store or terminalrecord". Shipped behavior — per the plan's binding shape, the nullable
schema, and the action's own empty-head skip guard — refuses only on a
missing/invalid terminal; a store-less workspace projects honestly with
chain.head: null. §23 documents the shipped semantics; the ADR sentenceshould be corrected in a docs follow-up.
refuting a review advisory):
doctor_reportraises a rawUnicodeDecodeErroron a terminal file with invalid UTF-8 bytes, soloop doctortracebacks (exit 1) instead of reporting a typed issue. Same familyas the fix(doctor): close review findings F1/F5/F6/F2/F7 — evidence, ledgers, YAML, verify surface, file targets #27–fix(inspect): score on execution evidence — close review findings M2/M3 #30 strict-UTF-8 work. Needs its own issue + fix.
and recorded in the run ledger): the evidence digest field name; the seed
workspace for the attest job (probe seeds an unterminated workspace; a
hand-terminate leaves doctor dirty — the runner path is the only
doctor-clean seed); the purity schema test erroring instead of skipping in
the fallback leg; and three from the earlier pre-flight.
assembled branch; auto-merge would outrun a post-PR review). Verdict
FIX-FIRST; all findings applied in-branch in the final commit: the
operative-control wording (p0),
/schemas/in CODEOWNERS (high), theaction description (low). One non-blocking follow-up candidate recorded:
the terminated-workspace seed logic now exists in three hand-written
copies (workflow, test fixtures, ci_anchor_probe variant) — consider a
shared pytest-covered
scripts/ci_verdict_seed.py.release-cut PR.
https://claude.ai/code/session_01JK76jSm45nHcoRoP1SdxXF