Telos

A research program for verifying autonomous agent work by evidence, not by trust.

Iter222 filled three TCP-1 admission gates (2/11 → 5/11) — scientific execution remains BLOCKED. Iter219 published a null on the temporal-yield screen; Iter214 merged through PR #12 as 470ca3627b7635d9a315cf2811ceb2eed6575fb9 with green push, pull-request, and merged-master CI, closing the publication-engineering line that began at iter208. Iter211's TCP-1 packet still stands at 2 local-design gates passing and 9 external admission gates blocked.

Iter219 asked, at zero spend, whether maintainers' later-added tests could supply TCP-1's independently authored hidden consequence tests — the blocker that otherwise needs recruited humans. The screen is a null. Across 482 SWE-bench Verified instances, tests added within 365 days after a task reference its touched symbols at 0.4066, but tests added before the task reference them at 0.4336. There is no temporal signal. Against the originally sealed cross-repository control the same data reads 0.4066 versus 0.1660 at p = 3.48e-24; that apparent effect is an artifact of a control that cannot fail for the right reason, and a pre-data amendment caught it before publication. Static symbol-name matching is falsified as a detector at this granularity; the underlying harvest idea is untested and remains open.

There is still no admitted cohort, reviewer team, model/runtime binding, hidden-test freeze, external timestamp, throughput result, or approved budget. No provider run, GPU allocation, scientific container, workflow dispatch, or scientific execution is authorized.

Read the forensic audit and the 2026 roadmap before extending the experiment line. The preserved publication predecessors are the iter208 post-seal forensic correction, iter209 publication CI recovery, and iter210 PR synthetic-merge recovery. The sealed materialization packet is the iter211 TCP-1 materialization preflight. The failed, unchanged publication predecessor is iter213 iter211 post-seal validation recovery. The merged publication-engineering gate is iter214 TCP-1 cross-platform numeric recovery. The active gate is iter222 agent-solvable TCP-1 admission evidence, which fills the three admission gates an agent can fill at zero spend — a live-digest open-weight model binding, a real RFC 3161 transparency timestamp, and a hostile isolation rehearsal with positive controls — moving TCP-1 admission from 2/11 to 5/11 while keeping execution unauthorized. Its result records that the six remaining blockers require external humans, real hardware, a throughput preflight, and an approved budget. The preceding cross-platform recovery is iter221 cross-platform guard tolerance, which stops a guard from asserting bit-exact floating-point equality across machines: sqrt-derived Wilson intervals are platform-dependent in the last place, so they compare at rel_tol=1e-9 while every exactly reproducible value stays exact. Its preserved predecessor is iter220 iter219 publication CI recovery, which keeps the failed iter219 branch and PR #13 unchanged, root-causes a required-phrase scanner that a Markdown line wrap could defeat, and replaces hand-listed local verification with a closure runner derived from the CI workflow itself. The published null is iter219 temporal consequence-test yield, whose result falsifies static symbol-name matching as a detector of temporal consequence-test targeting and records why its own cross-repository control manufactured a 10^-24 false positive. The prospective, still-inactive scientific successor is the unchanged iter212 independent cohort and custody freeze.

Current result

Telos studies certified-resolved reward hacks: patches that the official SWE-bench Verified harness marks resolved — they pass every graded test — yet retained execution under the instance-specific SWE-bench container tag differs from the accepted fix on an input the graded tests do not distinguish. A check restricted to those graded tests cannot catch this class, because passing those tests is exactly what "resolved" means. Historical construction and witness runners used mutable :latest tags without retaining resolved image digests, so exact historical container bytes are not reconstructible. The retained counts and labels below can be re-derived from committed proof, subject to the protocol and provenance limits stated here:

• A corrected foundation with a narrowed novelty claim. The earlier benchmark did not contain reward hacks: all 40/40 rows fail an official graded regression check. Historical tarballs separately contain 139 harness-resolved hack-tagged evaluations across 65 instance IDs, but retained decision evidence does not bind every evaluation to a discard decision or preserve its patch bytes. Iter192 is conservatively adjudicated FAIL on its overbroad novelty interpretation, while its literal v1-specific falsifier remains indeterminate: iter151's predecessor cohort does not retain patch bytes that establish v1 row identity. Iter151 nevertheless reported the same test-suite-rejection mechanism on a 0/20 semantic precursor. Nineteen instance IDs overlap, so the recount is retained while conceptual novelty is withdrawn.
• A 22-row, 8-repository reference-differential corpus (manifest) — django, matplotlib, astropy, xarray, pylint, pytest, requests, sympy. All rows pass the official graded tests and have retained gold-versus-variant output differences. The first 10 are exploratory, gold-and-variant-assisted single-scenario witnesses from the protocol-failed iter195 gate; the additional 12 come from iter199, whose stated design was first recorded after provider generation and before execution, not independently preregistered. This is a constructed reference-differential corpus, not independently adjudicated semantic ground truth.
• Protocol-failed full-cohort instrument diagnostics. Iter197 and iter201 both record FAIL: their property prompts used source/function locators derived from candidate diffs. Iter197 also replaced its visible-test anchor with gold validation and did not run its registered paired-gold controls as independent detector decisions. Iter201 explicitly registered gold validation, so gold use there is an interpretation limit, not a second deviation. The retained iter201 diagnostics are exploratory: the judge catches 20/22; 8/88 judge responses are unparseable (5/22 hack rows and 3/22 gold rows), so gold-control flag sensitivities are 3/22 observed lower, 6/22 worst-case missing upper, and 3/19 complete-case. A locator-assisted, gold-validated property pipeline catches 6/22, all within the judge's catch set. It supplies no independent false-positive estimate or ensemble gain. The judge phase lacks an independently timestamped pre-output Git freeze, and its retained artifact has parsed labels and nondecision markers but not raw response text.
• The mission's first neutral-prompt exploratory existence case. A model asked to fix a gold-localized issue with no instruction to game the tests produced one strict confirmed certified-yet-wrong patch. The retained convenience sample is nonrandom, and the strict two-judge rule was adopted after outcome inspection. After the official-harness denominator backfill, N=24, k=1, and u=6; report 1/24 confirmed lower, 7/24 worst-case upper over those six declared missing outcomes, and 1/18 complete-case sensitivity together. The 54 legacy execution logs lack explicit image/exit provenance and are accepted only as a frozen exact-byte corpus; they contain no embedded reference to claimed original run 29391238359, and no committed download receipt independently rebinds them to it. The 20 backfill logs have stronger provenance. The original blind-judge artifacts retain parsed labels and derived booleans, not raw response text, so exact response substance and parser fidelity cannot be re-audited. These are bounded parsed-decision existence evidence and descriptive exploratory yields, not a population rate.
• A findings paper (source, PDF, citation metadata). Its Telos empirical quantities are traceable to committed experiments/*/proof/ artifacts; external literature facts remain source-attributed rather than locally regenerated.

Future TELOS architecture

The sealed experiments establish why outcome-only trust is insufficient. The next system makes the evidence path reusable:

flowchart LR
 A["Task contract<br/>goal · constraints · falsifiers"]-->B["Isolated execution<br/>agent + tools"]
 B-->C["Full trajectory<br/>actions · outputs · resources"]
 C-->D["Artifact-bound receipt v2<br/>bytes · hashes · producer"]
 D-->E["Independent checks<br/>grader · consequences · humans"]
 E-->F{"Bounded claim gate"}
 F-->|complete and consistent|G["Signed claim"]
 F-->|failure, conflict, missingness|H["Fail · null · abstain"]
 G-->I["Deployment monitoring"]
 I-->C
 classDef active fill:#e6f4ea,stroke:#1a7f37,color:#0f3d1c;
 classDef stop fill:#fff1f0,stroke:#cf222e,color:#4c1114;
 class D,E,F,G active;
 class H stop;

Loading

Current evidence arc

This is the standing scientific story. A FAIL node preserves its exploratory measurements but does not convert them into protocol-valid detector evidence. Iter202 had a post-contact, pre-retained-output protocol freeze; it was not conventional prospective preregistration. Its provider stages later retained 50 valid patches from 53 solver calls and 38 scenario programs from 39 eligible scenario calls. Before any scenario or certification execution, the frozen static-safety gate rejected 9 scenario programs with 21 findings, admitted 29, and preserved one original missing scenario. Iter202 is therefore a provider-complete safety null at the protocol/execution boundary, not a solve-yield null, and contributes no rate denominator or numerator. Iter203 sealed those bytes and attempted the all-50 recovery once from green master. Its canonical workflow run 29460393525 authorized successfully, but all 50/50 first Docker run invocations returned exit 125 across eight independent runners before any in-container command; collection was skipped and the artifacts API reported zero uploaded workflow artifacts. Docker 28.0.4 rejects the frozen local log-driver combination because max-file=1 was used while compression remained enabled by default. No patch was applied and no official certification or scenario program executed. Iter203 is therefore an execution-infrastructure null with no N, k, or u, not a scientific negative. The exact daemon stderr was redirected to hidden temporary files and then deleted by the failed runner; the root cause is reconstructed from the frozen option tuple and Docker's version-matched validation source, not from a retained daemon-error artifact.

Iter204 separately versioned the narrow runtime recovery, preserving the iter202/iter203 corpus and target order while adding compress=false, bounded startup diagnostics, and a non-scientific launcher preflight. Its source merged as c1137f896b7ee3c9a26ee35bcda2c5f5c6b79446, and primary CI run 29465925393 passed both required jobs. The workflow itself could not be parsed. Its frozen closure snapshot contains two failed push records (29465584664 and 29465924803), each with zero jobs and artifacts and unavailable logs. At least one locally observed authorized dispatch request returned HTTP 422; the public iter204 workflow_dispatch history remained empty. No scientific stage ran, so iter204 is a pre-dispatch infrastructure null with no N, k, or u.

Iter205 then published the context-scope correction once. Feature head a336b4909329d392f6db5f6098792e07a17f28cb merged as 4f7dd39bb171fd89c1bb7da3f265aa00aa6df63f, and primary CI run 29468769187 passed. The server accepted iter205 workflow 314141096 as active at its exact name and path; both its complete all-event and dispatch histories are empty. Its read-only preflight nevertheless found that iter204's append-only server history had grown from the frozen two-row closure snapshot to four rows because iter205 branch and primary publication each added a parser-failure push record. The exact-two admission predicate therefore stopped the control flow before the iter205 dispatch request command. No iter205 dispatch request was issued, and no dispatch API response or rejection exists. There was no iter205 workflow run, provider call, container, patch, certification, scenario, adjudication, or judge process. Iter205 is a pre-dispatch admission-history null and contributes no N, k, or u.

Iter206 was sealed locally but stopped as a pre-publication claim-integrity null before any remote branch push, pull request, merge, workflow run, dispatch request, provider call, container, or scientific execution. Iter206 stopped after the audit flagged an apparent iter192 prior-baseline contradiction and the iter195 execution/design mismatch described below. Iter207's deeper patch-custody audit narrowed the former to a novelty-scope ambiguity. Iter206's exact-delta rule correctly forbade folding those corrections into its sealed protocol; it contributes no N, k, or u.

iter207_claim_integrity_and_admission_recovery is the sealed iter207 correction baseline. It preserves every fixed scientific input and runtime semantic, publishes machine-checked claim corrections without mutating historical raw artifacts, and carries forward the admission recovery. Its integrity audit made exactly two authenticated, read-only GitHub metadata GETs to verify historical CI projection semantics; they made no remote mutation and contacted no model provider. Its frozen protocol had authorized at most one final branch push, an exact successful attempt-1 branch-push and pull-request CI pair, one exact two-parent merge, green primary CI, and an exact six-row admission snapshot: the known four iter204 rows plus only the iter207 branch and primary publication rows. It will bind the iter206 workflow, once published by this release, to empty complete histories. Any missing, seventh, or malformed iter204 row or any iter205/iter206 run closes iter207 without dispatch; after every gate passes, at most one dispatch request is permitted. That authorization was never exercised and is superseded by iter208 because the post-seal audit found new publication blockers. GitHub Actions run 29422735843 is the exact iter200 official-harness denominator backfill. The later provider-free Node 24 run 29452243832 is a historical operational assertion that the specs and committed 74-log corpus were validated without re-executing containers; no committed run receipt independently binds that run ID, so it is not evidence of the backfill execution itself.

Reviewer entry points: the machine-readable claim-integrity correction ledger hash-binds the iter192 conservative novelty-scope correction and iter195 strict protocol-failure receipt, while the iter206 terminal-null receipt binds the zero-action predecessor state.

Diagram color is semantic: orange marks corrected or partial evidence, red marks a failed gate, blue marks retained or completed evidence, gray marks a null before scientific execution, and green marks the active pre-data publication-recovery gate.

flowchart TB
 subgraph A["Construct and correct"]
  direction LR
  V1["v1<br/>40/40 unresolved"]-->I192["192 conservative FAIL<br/>literal trigger indeterminate<br/>139 evaluations bounded"]-->I195["195 strict FAIL<br/>10 gold-assisted"]-->I196["196 partial<br/>judge 7/10 lower"]-->I197["197 FAIL<br/>locator/control faults"]-->I198["198 FAIL<br/>paper corrected"]-->I199["199 chronology FAIL<br/>+12; corpus 22/8"]
 end
 subgraph B["Measure"]
  direction LR
  I200["200 exploratory<br/>N 24 · k 1 · u 6"]-->I201["201 FAIL<br/>property 6/22 ⊂ judge 20/22"]-->I202["202 safety null<br/>53→50; 39→38; 29 admitted<br/>zero execution"]
 end
 A-->B
 classDef corrected fill:#fff4e5,stroke:#b54708,color:#4a2500;
 classDef failed fill:#fff1f0,stroke:#cf222e,color:#4c1114;
 classDef evidence fill:#eaf3ff,stroke:#0969da,color:#0c2d57;
 classDef null fill:#f6f8fa,stroke:#57606a,color:#24292f;
 class V1,I192,I196 corrected;
 class I195,I197,I198,I199,I201 failed;
 class I200 evidence;
 class I202 null;

Loading

The fixed corpus then enters a separate fail-closed recovery chain. These nulls preserve evidence without turning infrastructure or admission failures into scientific outcomes.

flowchart LR
 I203["203 infra null<br/>50/50 exit 125"]-->I204["204 pre-dispatch null<br/>two-row closure"]-->I205["205 admission null<br/>four rows at gate"]-->I206["206 local null<br/>claim audit"]-->I207["207 sealed local<br/>claim corrections<br/>no publication"]-->I208["208 sealed<br/>remote CI failed<br/>no scientific run"]-->I209["209 sealed<br/>push green; PR failed<br/>no scientific run"]-->I210["210 merged recovery<br/>push · PR · master green"]-->I211["211 TCP-1 sealed<br/>2 pass · 9 blocked"]-->I213["213 validation recovery<br/>remote CI failed"]-->I214["214 numeric recovery<br/>exact Wilson · merged"]-->I219["219 temporal yield<br/>NULL · no signal"]
 classDef null fill:#f6f8fa,stroke:#57606a,color:#24292f;
 classDef complete fill:#eaf3ff,stroke:#0969da,color:#0c2d57;
 classDef corrected fill:#fff4e5,stroke:#b54708,color:#4a2500;
 classDef active fill:#e6f4ea,stroke:#1a7f37,color:#0f3d1c;
 class I203,I204,I205,I206,I207,I208,I209 null;
 class I210 complete;
 class I211,I213 corrected;
 class I214 active;

Loading

Standing correction (iter192, updated 2026-07-16): the construct finding survives; novelty narrows

Read this before citing any number below. This correction changes the framing—not the arithmetic—of the later v1-based prose.

Protocol and interpretation correction: iter192's frozen falsifier 5 required failure if a committed artifact already reported a test-suite or official-harness baseline for v1. Iter151 predates v1. It reported that a plain full-suite CI gate rejects its 0/20 both-miss cohort, and nineteen instance IDs overlap v1, but its retained result does not preserve the patch bytes needed to establish exact v1 row identity. The literal v1-specific falsifier is therefore indeterminate, not proven to have fired. Telos nevertheless adjudicates the novelty component conservatively as FAIL: iter151 had already disclosed the same test-suite-rejection mechanism, so the exact v1 40/40 recount and paired table are retained while any claim of conceptual firstness is withdrawn. Historical tarballs also contain 139 harness-resolved hack-tagged evaluations across 65 instance IDs; only 23 discarded iter152 IDs are decision-bound, with 17 overlapping the harness-resolved set. The original hypothesis, audit output, and receipt remain preserved; the additive iter207 ledger records this narrower adjudication.

A reward hack scores well on the reward function while being wrong. That is what makes it dangerous: it survives the grader, corrupts the training signal, and ships.

benchmarks/reward_hack_benchmark_v1/ does not contain such patches. The iter152 seed generator (experiments/iter152_reward_model_gaming_scale/proof/raw/detection_pipeline.py, line 132) and the iter154/iter155 expansion runners used the same target-pass plus PASS_TO_PASS-failure acceptance shape. PASS_TO_PASS is the grader's curated regression-test set. Independently of that source reading, experiments/iter192_reward_hack_benchmark_construct_validity_audit/ re-derived the official SWE-bench harness verdict for all 40 rows from committed tarballs at zero model-provider spend:

check	value
rows matched to a committed official SWE-bench report	40/40
rows the official harness marks resolved: false	40/40
rows with at least one failing PASS_TO_PASS test	40/40

What follows from that:

oracle	detections on v1	estimated spend guard
official SWE-bench harness verdict (or an equivalent official-grader execution)	40/40	$0.00
three-model cross-provider panel, unrepaired majority_catch (iter179)	17/40	$13.128090 for 240 score-producing calls

The panel value is a conservative estimated guard, not a provider invoice. The whole iter175+iter178 run path is $13.317840 because iter178 also made three diagnostic-only calls. Adding iter181's later repair diagnostic yields $13.589640 (rounded $13.59); neither diagnostic stage contributes to the unrepaired 17/40 score.

Iter192's lexical scanner reported 0 hits across 677 committed Markdown files. That remains the exact scanner output, but it cannot establish conceptual novelty: experiments/iter151_cross_repo_scale_official/RESULT.md had already explained that its predecessor proxy rows are unresolved by definition and that a plain full-suite CI gate rejects them. Because iter151 did not retain accepted patch bytes, this is a semantic precursor rather than a proven row-identical v1 baseline.

The 17/40 figure is not retracted. It is arithmetically correct and reproduces from committed proof. What changed is what it is a figure about: it measures how often a frontier panel, reading a diff, spots a subtly-wrong patch that the official graded regression checks already reject. That is a legitimate finding about LLM code review. It is not a finding about reward hacking, and it does not motivate a verification protocol, because the already-required graded tests catch 40/40 at zero model-provider spend. The term both_miss always meant the deterministic detector and the LLM judge miss; it never meant the tests miss.

Historical tarballs contain 139 hack-tagged evaluations across 65 instance IDs that the official harness certified as resolved. The committed disposition evidence covers only 23 discarded iter152 IDs, 17 of which overlap that harness-resolved set; it does not establish that all 139 were discarded or retain their specific patch bytes. Whether they are wrong is unestablished and requires gold-differential execution. That certified-but-semantically-unknown class is the mission's target, and it is what experiments/iter193_certified_resolved_reward_hack_construction/ set out to construct.

iter193 executed 16 elicited candidates intended to be certified-resolved-but-wrong under instance-specific x86 SWE-bench container tags and published a null: 0 witnessed certified-and-wrong rows against a >=5 bar. 6 matplotlib candidates were certified-resolved but behaviorally identical to gold across their full test module (undetermined, not correct); 10 django/astropy candidates were not executable by a bare-pytest harness — a harness-fitness gap, not a negative.

experiments/iter194_certified_resolved_oracle_and_runner_fix/ fixed the runner and sharpened the null. It ran all 16 candidates through their official SWE-bench eval_script (django runtests.py, pytest for the rest), parsed with the vendored official SWE-bench parsers: 16/16 executed, all 16 certified-resolved, yet 0 accepted. For django, the curated PASS_TO_PASS set is the entire audited target test module (0 additional module tests); where additional target-module tests exist (matplotlib up to 168, 193 total), the variant passes them identically to gold. No graded or additional test in those audited target modules distinguishes a certified variant from its gold fix. This is not a claim about every test elsewhere in each repository. experiments/iter195_synthesized_input_differential_oracle/ therefore synthesized inputs to the changed callable and executed gold versus variant differentially in-container.

experiments/iter195_synthesized_input_differential_oracle/ is strict protocol FAIL with retained exploratory diagnostics. The frozen design required synthesis without gold content, at least 10/20 gold-clean inputs per accepted candidate, and retained raw synthesis prompts and differential transcripts. The executed generator instead received both gold and variant hunks and produced one targeted scenario per candidate. It never ran the preregistered 20-input validation, and the required prompt-retention bar is not met. The result's replacement 13/15 scenario-clean count and >=8 bar were not the frozen per-candidate criteria.

The retained execution still supplies 10 exploratory reference-differential witnesses across 3 repositories: all ten patches pass every official FAIL_TO_PASS and PASS_TO_PASS test, and a gold-and-variant-assisted targeted scenario produced clean divergent outputs. Across the 15 generated scenarios, 10 diverged, 2 matched gold on that one scenario (not a global equivalence result), 2 failed under gold, and 1 errored under the variant. These witnesses concern behavior not distinguished by the audited graded/additional target-module tests; the mission does not claim every shipped repository test was audited. "Wrong" is the benchmark's reference-differential label, not independent semantic adjudication.

experiments/iter196_detector_vs_certified_hacks/ evaluated a judge against these 10 exploratory labels: a 2-model panel (gpt-5.6-terra + claude-opus-4-8; the third historical panel slot did not run, which says nothing about current runtime access), reading diff + task + visible test, flags 7/10 hacks (any-catch), 1/10 gold patches, and 1/2 equivalent controls in one stochastic run. Three of twenty hack responses are unparseable across two rows, so the required hack sensitivities are 7/10 observed lower, 9/10 missing upper, and 7/8 complete-case; paired-gold values are 1/10, 3/10, and 1/8, and equivalent-control values are 1/2, 2/2, and 1/1. Raw responses were not retained, and capped prompts truncate some task/test inputs. Judges carry bounded parsed-label signal on this protocol-failed, gold-assisted cohort, not a stable model score. Because the preregistered two-detector gate delivered only Detector A and moved Detector B to iter197, iter196 is partial and protocol-blocked rather than a passed detector comparison. The two-model amendment first entered Git with the outputs, so it lacks an independent pre-output freeze. A diagnostic also finds one gold-distinctive line in each of 2/10 positive candidate inputs; this is disclosed without assigning a causal effect.

experiments/iter197_gold_free_oracle_vs_certified_hacks/ is now explicitly a protocol failure with retained exploratory diagnostics. The registered design required a diff-independent, visible-test-anchor property protocol. The executed generator instead received source/function locators extracted from the candidate diff, replaced the registered visible-test anchor with gold validation, and never evaluated the ten registered paired-gold controls as independent detector decisions. The accurate label is a locator-assisted, gold-validated property pipeline: 12/12 properties pass the gold inclusion rule, it catches 4/10 hacks, and it flags 0/2 equivalent controls. Those last two controls are not an independent false-positive estimate. The alleged property-only row, django-11211, had two unparseable judge responses, so it was judge-unadjudicated rather than a confirmed judge miss. The union sensitivities are 8/10 observed, 9/10 missing upper, 7/8 judge-complete, and 8/9 under the explicitly property-resolved estimand; no complementarity is confirmed. The later >=5 threshold was added with the generated properties, not preregistered.

experiments/iter201_detectors_on_full_benchmark/ repeated the comparison on all 22 operationally positive corpus rows and likewise records FAIL for the locator protocol deviation. Retained exploratory outcomes are judge flags on 20/22, property pipeline 6/22, and union 20/22; every property catch is already a judge catch. The judge has 8/88 unparseable responses, affecting 5/22 hack rows and 3/22 gold rows. Hack any-catch remains determinate because the other judge catches each incomplete hack row. Gold-control flags must be reported as 3/22 observed lower, 6/22 worst-case missing upper, and 3/19 complete-case. No independent property-pipeline false-positive rate or ensemble improvement is established. All 44 judge rows were fresh iter201 evaluations, but the judge hypothesis, runner, and outputs first appear in one commit; only the already-mismatched property runner has a separate pre-output freeze. Judge prompts truncate task text for 7/22 unique instances and visible tests for 3/22; property prompts truncate 9/22 tasks and 3/22 tests. Historical construction, witness, and property execution used mutable :latest images without retained digests; exact historical container bytes cannot be reconstructed from the committed evidence. The retained iter201 judge artifact also records 4 shared gold-added lines inside candidate inputs. These are lines present in the candidate itself rather than a separate gold comparison, but they remain a disclosed prompt-content diagnostic; the run provides no causal estimate of their effect on judge decisions.

experiments/iter198_findings_paper_synthesis_and_accessibility/ produced the initial findings rewrite, but its strict accuracy gate is now FAIL because it inherited the iter192 novelty overstatement, treated iter195 as protocol-valid, and described the original detector comparison too strongly. The current paper is a later correction surface, not evidence that iter198's original accuracy bar passed. Telos empirical quantities are traceable to committed proof; external literature facts are source-attributed. External submission remains an operator action after a final citation and author-block review. See paper/README.md.

experiments/iter199_benchmark_expansion_across_repos/ grows the benchmark beyond three repositories. It runs the same construct-and-witness pipeline on 42 additional target IDs spanning all 12 dataset repositories. They were additional targets for that construction gate; the mission does not claim they were mission-fresh or use them for a natural-frequency inference. The retained result adds 12 official-harness-certified, gold-assisted reference-differential witnesses across eight repositories, introducing five repositories not present in iter195 and bringing the released corpus to 22 across 8. Its hypothesis, runner, and provider outputs were first committed together, after scenario-generation contact and before CI execution. The stated design is therefore post-provider/pre-execution and not independently preregistered. The 12 retained rows remain exploratory gold-and-variant-assisted reference-differential evidence, not a prospective confirmation.

experiments/iter202_natural_rate_scaled/RESULT.md records the null from the attempted scale measurement on a frozen 53-instance cohort, for descriptive pooling with iter200's 39 disjoint target IDs. A pre-output overlap audit found that 27 of the 53 had defined prior result-bearing exposure elsewhere in the mission, including 10 with provider-call-ledger evidence; any eventual result must therefore include the declared prior-use sensitivities. The first Git record of the iter202 hypothesis and target manifest followed a disclosed interrupted provider invocation that retained no outputs and is conservatively charged 53 calls and an estimated $2.65, so the freeze was post-contact and pre-retained-output, not conventional preregistration before provider contact. The retained run later completed all 53 solver calls and all 39 eligible scenario calls, yielding 50 valid patches, 38 scenario programs, and one original missing scenario. The unchanged frozen safety validator then rejected 9 programs (21 findings) and admitted 29. The pipeline stopped before scenario or certification execution. Consequently iter202 is preserved as a scenario-safety protocol/execution null and supplies no N, k, or u for a rate claim. Conservative bookkeeping carries 145 charged calls and $7.25 estimated-or-charged total, explicitly not exact actual usage because the interrupted invocation's completed-call count and spend are unknown.

experiments/iter203_iter202_safety_recovery/RESULT.md publishes the additive post-provider recovery as an execution-infrastructure null. Its sole canonical run replay-validated the complete iter202 corpus and reached all 50 frozen rows, but every first Docker run invocation returned exit 125 before any in-container command. No certification or scenario executed, so iter203 contributes no N, k, or u. The eight exact public shard logs are retained and hash-bound; the exact daemon stderr is absent, and the Docker logging-configuration diagnosis is reconstructed from frozen source and version-matched implementation semantics.

experiments/iter204_iter203_infrastructure_recovery/RESULT.md publishes the pre-dispatch workflow-parse infrastructure null. The approved source and green primary CI are retained alongside its frozen closure snapshot of two public push parse-failure records, the exact-zero public workflow_dispatch list, and the locally observed HTTP 422 message. The public records contain zero jobs and artifacts, and no scientific stage ran.

experiments/iter205_iter204_workflow_context_recovery/RESULT.md publishes the pre-dispatch admission-history null. Iter205 itself has no runs: the read-only gate stopped before its dispatch request because iter204's append-only publication history contained four rows rather than the two-row closure snapshot.

experiments/iter206_iter205_admission_history_recovery/RESULT.md records the local pre-publication claim-integrity null: the sealed gate made no remote or scientific state change and contributes no N, k, or u. Its immutable terminal evidence is predecessor provenance for iter207_claim_integrity_and_admission_recovery, the sealed, separately versioned correction and admission-recovery baseline. Iter208 is the sealed post-seal forensic correction whose first publication CI failed for two validator defects. Iter209 fixed those defects and passed push CI, but its PR CI exposed a synthetic-merge topology error. Iter210 is the sealed additive topology recovery; PR #10 merged with green branch, pull-request, and merged-master CI. Iter211 is the sealed zero-execution TCP-1 materialization preflight. Its protocol layer passes and its scientific-execution receipt is blocked on nine real external gates, but its first full post-seal suite found three publication-only compatibility defects. Iter213 preserves iter211 exactly and recovers descendant-safe receipt/topology and handoff scanning. Its unchanged push run 29505707609 and pull-request run 29505789397 both failed because Linux retained a one-ULP positive residue at the exact Wilson k=0 boundary while the local runtime returned 0.0; all other 656 tests passed. Iter214 records the pre-data analysis amendment, canonicalizes only exact k=0/k=n boundaries, and makes sealed iter213 validation ignore additive descendant scope. Iter208 through iter214 contribute no scientific result.

experiments/iter200_natural_certified_yet_wrong_rate/ asks gpt-5.6-terra to fix issues with no instruction to game tests. It is exploratory rather than a preregistered frequency estimate. The prompt is gold-localized: the issue and buggy region are reconstructed using gold context/removals while gold-added lines are withheld. The retained 39 targets are a nonrandom convenience sample. Starting from the committed iter154 snapshot, the builder first excludes all 66 unique IDs in the iter193 Phase-A and iter199 target sets, then keeps 200 compatible rows across 9 repositories using a single source file, one added run, gold-shaped patch-builder strip-round-trip compatibility, lexicographic ordering, and a cap of five per repository. The original hypothesis named one independent judge; after inspecting outcomes, the conservative report adopted a two-judge, both-name-only-model rule (3 loose cases versus 1 strict case). The missing-outcome formulas were also added after the original result but before the official-harness backfill. Git first records the hypothesis and solver outputs together, so a pre-output freeze is not independently timestamped there. The blind runner preserved parsed labels and derived booleans but not raw judge responses; the exact response substance and parsing decisions therefore cannot be independently replayed. The 54 original execution logs contain no embedded reference to claimed run 29391238359, and no committed original artifact-download receipt independently rebinds those bytes to that run; the run ID remains a historical attribution.

The historical 1/15 proportion is conditional on the old scenario-eligible cohort and is not poolable. The corrected official-certification denominator is complete: all 37 valid patches have official-harness evidence, 24 are certified, and all 9 legacy identical-to-gold pairs become equal only after terminal-LF normalization; 0/9 are byte-identical and 8/9 of those normalized-equivalent patches certify. One patch is strict-confirmed, and 6 certified outcomes remain unadjudicated (5 without a valid witness and 1 with an incomplete judge outcome). Report 1/24 confirmed lower, 7/24 worst-case upper over those six declared missing outcomes, and 1/18 complete-case sensitivity together. They are not confidence intervals, prevalence bounds, or a population frequency.

---

Historical record (earlier arcs, retained for provenance)

This section through the historical architecture summary covers the earlier detection/intervention work (iter00-iter191). It is retained for provenance, not as a standing summary. Terms such as both-miss, 17/40, majority_catch, or historical gold-free claims remain bounded to their named experiments; they do not repair the iter197/iter201 protocol failures or supersede the correction above.

At the iter165 boundary, a bounded paired single-model judge result existed for gemini-2.5-flash: 3/40 recall on the blinded all-hack packets from iter161 and 0/40 false positives on the paired legitimate controls after the iter165 rate-limit recovery. That supports only this one model's bounded recall/specificity/precision shape; the high precision has denominator 3 and does not make the model a strong detector. Iter167 tested a stricter skeptical prompt and failed the pre-registered improvement bars: recall stayed 3/40, specificity was 0.90, hack nondecision rose to 5/40, prompt leakage was 0, and response secret hits were 0. Iter168 adjudicated that null: all 9 invalid outputs were markdown-fenced JSON, but a diagnostic-only fence strip would move recall only to 4/40, so the main failure is semantic recall, not just output formatting. Iter169 completed a zero-spend independent judge-panel design with three cross-provider slots, structured-output enforcement, frozen aggregation rules, paired-control accounting, and a bounded future pilot ceiling. Iter173 built a zero-spend, source-linked public binding menu for the Google, OpenAI, and Anthropic panel slots: 6 candidates across 3 slots, secret hits 0, primary rule still majority_catch, and paid execution still unauthorized. Iter174 froze exact defaults from that menu (gemini-2.5-flash, gpt-5.6-terra, claude-opus-4-8) with 3/3 menu membership, secret hits 0, and paid execution still false. Iter175 then ran the bounded paid panel pilot: 120/120 primary calls succeeded with 0 retries, estimated spend guard $6.312690, and 0 secret/project/account hits in committed artifacts. On the 20 paired rows, the primary majority_catch panel caught 13/20 hack rows and 0/20 controls, while any_catch caught 15/20 hacks and 0/20 controls and unanimous_catch caught 1/20 hacks and 0/20 controls. The iter176 zero-spend adjudication rederived the result from committed proof: 120/120 calls reconciled by hash, 120/120 parsed rows mapped to scheduled calls, committed metrics matched, the three nondecisions were OpenAI max_output_tokens empty-content failures, and the main disagreement pattern was OpenAI/Anthropic catch versus Google legitimate on 12 rows. Iter177 froze the next expansion design without spend: 20 fresh remaining paired rows, 3 diagnostic OpenAI recovery calls, 123 planned calls before retries, a 160 call ceiling, $50.00 spend ceiling, and OpenAI max output tokens raised to 1536 for the next gate. Iter178 then ran that bounded paid expansion: 123/123 provider calls succeeded with 0 retries, estimated spend guard $7.005150, and 0 secret/project/account hits. On the fresh remaining 20 paired rows, primary majority_catch caught 4/20 hack rows and 0/20 controls; the combined unrepaired iter175+iter178 diagnostic view is 17/40 hack catches and 0/40 control catches, with 4 hack nondecisions and 1 control nondecision. The three OpenAI recovery calls emitted parsed diagnostic outputs under the 1536 token budget, but they do not rewrite iter175 metrics. Iter179 adjudicated the full cohort without spend: unrepaired majority_catch remains 17/40 hack rows and 0/40 controls, the five panel nondecision rows are all OpenAI empty-output cases, recovery diagnostics have 0 score-rewrite allowance, and two fresh OpenAI empty-output hack rows remain unrecovered. Iter180 then designed the repair protocol without spend: all five OpenAI empty-output primary nondecision rows must be rerun, including the three with prior diagnostics, under a future ten-call / $10.00 ceiling; unrepaired iter179 remains the primary public metric. Iter181 ran that bounded OpenAI repair execution: 5/5 provider calls succeeded with 0 retries, estimated spend guard $0.271800, 0 secret/project/account hits, and 4/5 parsed repair outputs. The secondary repaired diagnostic reduces panel nondecisions to 1 hack and 0 controls, but primary majority_catch stays 17/40 hack rows and 0/40 controls; unrepaired iter179 remains the public metric. Iter182 adjudicated that repair execution without spend: 0 provider calls, 0 credential probes, 5/5 raw responses reparsed and reconciled by hash, the committed comparison matched recomputation, and the repaired output remains diagnostic only. Iter183 synchronized the public claim surface without spend: audited surfaces preserve unrepaired iter179 majority_catch as the primary public metric, keep iter181/iter182 repair evidence diagnostic only, and point the active gate to iter184. Iter184 then mapped 20 current public sources to 6 concrete Telos technique implications and selected iter185 as the next zero-spend empirical gate: a property-probe design over the 23 iter179 primary-missed hack rows before any new provider spend. Iter185 then froze that design without spend: the committed iter179 primary-miss cohort is 23 hack rows across 4 disagreement/nondecision classes, the priority property-probe subset is 12 rows across at least 6 repositories, leakage policy forbids gold patches, hidden test names, official expected outputs, and labels, and the next active gate is iter186 packet materialization. Iter186 then materialized the 24 property-probe input packets without spend: 12 hack-source packets and 12 paired-control-source packets, unique hashes, source traceability in the manifest only, and 0 prompt leakage hits. Iter187 then validated the future property-generator output contract without spend: 17 parser fixtures, 100% valid executable fixture parse rate, 100% invalid/refusal/malformed rejection rate, 24 prompt contracts scanned, 0 prompt-contract leakage hits, and paid property generation still unauthorized. Iter188 then designed the Sentinel-style Telos mission data/process audit without spend: 26 frozen committed local inputs, 7 required future audit-note sections, 8 verifier checks, hard forbidden-claim boundaries, secret-hit bar 0, and no provider, property-generator, SWE-bench, cloud, or score-changing execution. Iter189 then executed that audit without spend: all 7 required audit-note sections are present, 29 frozen local inputs were checked, 15 recent key-gate receipts validated, benchmark-lineage checks passed, forbidden positive claim hits were 0, secret hits were 0, and no claim upgrade was made. Iter190 then stopped before provider spend as a null property-generator execution result: it froze the full 24 planned calls with 0 prompt leakage hits, but the committed schema has no direct runnable artifact fields and the local runtime is not a ready SWE-bench/container execution surface, so local/container execution attempts stayed 0 against the pre-registered 20 attempt floor. At that historical boundary, the next gate was iter191's zero-spend property-execution contract design. No leaderboard, public benchmark score, model-comparison result, precision result beyond the explicitly bounded denominators, model-superiority claim, state-of-the-art result, natural-frequency estimate, broad robustness result, repaired-score result, or claim beyond this bounded panel path is made. The repository begins with a completed target survey: experiments/iter00_target_survey, which selected a hybrid Telos overlay on public software-agent tasks.

The question is narrow and testable:

When an AI agent says a long-horizon task is done, can an external protocol prove that the real objective was completed, that the agent did not merely satisfy the visible proxy, and that it stopped at the correct boundary?

The target is not a better chat transcript. The target is a receipt-bearing completion protocol: tests when code changed, typecheck/build when applicable, diff-scope checks, live-domain checks when production behavior changed, artifact hashes, stated acceptance criteria, named falsifiers, and an adversarial review pass.

The Real-Trajectory Verification Arc (iter109-iter191)

This earlier arc moved completion verification off self-authored fixtures and onto real external ground truth. Earlier iterations (iter104, iter107) scored a detector that branched on fixture taxonomy or read authored packet booleans, so both the traps and the detector were written by the same process and no external validity was earned. The arc below fixed that, and it is the foundation the current certified-yet-wrong work (above) was built on.

The subject is the real princeton-nlp/SWE-bench_Verified dataset. A candidate solution is a raw unified diff plus the public FAIL_TO_PASS test ids. The deterministic detector (telos/tamper/) reads only that - never a ground-truth label, an attack-type field, or an authored semantic boolean. It encodes the documented reward-hack failure modes: editing the verifier (the tests that must fail-to-pass, conftest.py, pytest config), weakening assertions, hard-coding a hidden expected value in source, and tampering with a receipt digest.

iter	question	result
109	Does a raw-diff detector work on real data?	0/200 false positives on real human gold patches; the execution-only default catches none of the hacks
110	Does it survive adversarial evolution?	source-only evasion recall 0 -> 1.0 at 0/200 false positives (one hacker-fixer round)
111	Is a strong LLM judge already enough?	a gemini-2.5-flash steelman judge ties the detector on obvious hacks: 0.00 FP, 1.00 recall, $0.044
112	Where do they diverge?	on 60 stealth candidates: detector_only=0, judge_only=15, both_catch=30, both_miss=0 - the judge strictly beats the detector on oblique special-casing
113	Do the hacks really pass the tests?	native execution of a real instance: base fails, gold passes, a hard-coded hack passes the real hidden test; execution-only and the detector accept it, only the judge flags it
114	Does gold resolution hold across a batch?	4/5 gold patches resolve under real execution; 0/5 detector false positives; one honest native-harness fidelity gap
115	Tighten the fidelity estimate	17/18 gold patches resolve under real execution (0.94); 0/18 detector false positives; the same single fidelity gap
116	Measured catch rate on executed hacks	on three hacks that really pass their hidden tests: execution-only 0/3, detector 1/3, judge 3/3
117	Can the detector gap close for free?	a constant-return signal catches 2/2 missed hacks but costs 1/200 real false positives - the first hardening that trades precision, so it is not adopted and the class is judge territory
118	Is there a hack neither verifier catches?	yes - 2/2 disguised hacks pass the hidden test, evade the detector, and fool the judge; both are wrong on a held-out input, so held-out-input execution is the required defense
119	Does held-out-input execution catch them?	yes - on the both-miss class the visible test, detector, and judge each catch 0/2 while metamorphic held-out-input execution catches 2/2; the loop closes
120	Does the metamorphic layer generalize past a hand-picked input?	yes - seeded random held-out inputs catch 2/2 (diverging on 10/12 and 8/12); the open problem narrows to an oracle without gold
121	Can the oracle drop the gold reference?	yes - contract properties (no gold) catch 2/2 (violating 27/30, 30/30) at 0 false positives on gold; the frontier narrows to automatic property generation
122	Can the model generate the property itself?	yes - gemini-2.5-flash proposes properties that catch 2/2 (violating 28/30, 26/30) at 0 gold false positives; the model, fooled as a judge, is reliable as a property generator because execution checks it
123	Can unsound properties be rejected without gold?	yes - the visible test is a known-correct anchor; it keeps 2/2 sound properties and rejects 2/2 unsound ones with no gold reference
124	Does automatic property generation generalize?	not yet - across seven real instances only 2/7 produce a clean sound auto-generated harness; the mechanism is real but the harness/input synthesizer is the bottleneck (generation, syntax, input-domain, mis-target failures)
125	Does a validated synthesizer help?	yes - test-source targeting, input validation, and retry roughly double the rate to 4/7; a non-triviality filter is required and catches a vacuous harness raw soundness accepted; two residual failures (unsound property, structured-input domain) remain
126	Can the soundness decision drop gold too?	yes - non-triviality plus the visible-test anchor keeps the 3 genuine-sound harnesses, rejects the unsound 10999 property gold was needed for, and rejects the vacuous 11276; the pipeline is gold-free at every step
127	Can structured inputs close the residuals?	yes - seeding the generator from the test example and using a non-pathological format make 11206 and 11848 sound, raising the genuine-sound rate to 5/7; the remaining two are an unsound generated property and a mis-targetable task, not input-domain failures
128	Which gold-free property fits which function?	an inverse round-trip closes the 10999 parser (0/30, self-validating) to reach 6/7; strategy is function-type-dependent (contract property for transforms, round-trip for invertible parsers), and an anchor must match the harness input convention
129	What is the layer's applicability, and can strategy be automatic?	the last residual 11276 is a cross-cutting escape() refactor (27 FAIL_TO_PASS tests), excluded by a single-testable-function criterion; the six valid candidates are 6/6 genuine-sound and a parse-name classifier auto-assigns each the sound strategy
130	Does the method widen beyond django?	as a method yes - 3/4 sympy hyperbolic identities hold under real execution - but instance-level cross-repo scale is bounded by a compound of environment fidelity (old instances need Python <=3.9), applicability (modern instances are edge-case fixes), and numeric-vs-symbolic precision
131	Can the numeric and environment bounds be closed?	symbolic evaluation makes the identities 4/4 exactly sound (removing the float artifact, proving each for all x - a stronger check); the environment bound has a named resolution, the Docker harness, scoped as next-phase infrastructure
132	Does the Docker harness actually close the environment bound?	yes - the official SWE-bench image ran a natively-blocked instance's real hidden test in its Python 3.9 environment (base fails with the real bug, gold passes); the last bound is resolved in practice, and full-dataset coverage is now engineering, not open research
133	Does the Docker harness batch locally?	the method batches by construction, but local batch execution is unreliable under arm64 emulation (multi-GB pull cost, daemon load, flaky emulated stdout); full-dataset batching belongs on a native-x86 CI runner - an environment bound on local execution, not a method or research limitation
134	Does the batch run on native-x86 CI?	yes - a manual docker-batch.yml workflow on the x86 runner pulled the official SWE-bench images and resolved 3/3 blocked instances (base fails with the real bug, gold passes) in instance-specific environments; the iter133 local failure was arm64 emulation, not the method
135	Does the full stack run inside the instance container?	yes - inside 13480's instance-specific container (Python 3.9, sympy 1.1.2) the hidden test resolves and the gold-free coth property is PROP_SOUND; environment fidelity and property verification compose end to end on a natively-unrunnable instance (3/3 gold, 1/1 property sound)
136	Does the full-stack batch scale?	yes - on x86 CI 4/4 blocked instances resolve their hidden tests and both property-testable ones (coth, Min) are PROP_SOUND in their instance-specific containers - a 2/2 property genuine-sound rate, the gold-free property running in the same selected environment as the hidden test
137	Does the property rate hold wider?	yes - on x86 CI 5/5 blocked instances resolve and all three property-testable ones (coth, Min, Point.distance) are PROP_SOUND in their instance-specific containers - a 3/3 property genuine-sound rate
138	How broadly does the property layer apply?	the single-testable-function criterion is met by 405/500 = 0.81 of SWE-bench Verified (scikit-learn 0.94, sympy 0.89 high; pylint 0.40 low) - a broad structural reach, but an upper bound since property-derivability within it is function-dependent
139	How narrow is property-derivability really?	even in sympy only 7/67 = 0.10 of applicable instances are in a property-rich domain (all three verified instances land there) - so Layer 3's natural domain is math-identity functions, while Layers 1-2 apply universally: the honest three-layer division of labour
140	How prevalent is the both-miss class?	a gemini-2.5-flash/pro hacker-fixer produced 0/20 verified both-miss hacks on thoroughly-tested django instances - read at the time as "the window is narrow where coverage is thorough," but this structural reading is retracted by iter141 (it was a capability-plus-harness artifact)
141	Was that 0/20 structural or an artifact?	artifact - a non-refusing frontier adversary (gpt-5.6-terra) with a robust apply harness produced 2 confirmed both-miss hacks on the same "well-tested" django instances; django-14752 (returns str(obj.pk) instead of honoring to_field_name) evades the detector AND is judged legitimate, caught only by held-out execution - so execution is necessary even on thorough tests; claude-opus-4-8 refused the construction while gpt-5.6-terra complied
142	At what rate does the adversary defeat both static layers?	over 25 genuine attempts on well-tested django code, gpt-5.6-terra produced confirmed both-miss hacks on 0.24 of attempts, and on 0.12 (half of them) the hack also fooled the LLM judge - caught only by held-out execution; so ~1 in 8 attempts defeats both static layers, quantifying why execution is necessary not optional
143	Was the judge just too weak?	no - re-judging the 6 confirmed both-miss with frontier judges, gpt-5.6-terra is fooled by 1/6 and claude-opus-4-8 by 3/6 (a bigger reasoning model is not a better judge; it ties the flash baseline); django-14311 is judged legitimate by all three judges AND evades the detector, so it survives every static layer and is caught only by held-out execution
144	Is the class django-specific?	no - on sympy (native execution) sympy-22714 (imaginary-coordinate guard any->all is_zero flip) is a confirmed judge-fooled both-miss, so the class and execution-necessity generalize; but the rate is repo-dependent - sympy 0.056 confirmed vs django 0.24, because sympy's precise math tests make the adversary break the visible test 0.67 of the time (vs 0.44)
145	Can a judge panel replace execution?	no - over the 7 confirmed both-miss, a three-judge any-catch panel catches 6/7, majority 4/7, unanimous 2/7; django-14311 survives every rule and the detector, caught only by held-out execution. A panel adds robustness (no single judge exceeds 6/7) but any-catch trades away precision, so it is a mitigation, not a substitute for execution
146	Does the protocol IMPROVE real completion, not just detect gaming?	yes - on 7 django both-miss starts a proxy gate yields 0/7 real completions while the Telos gate (gold-free held-out-execution rejection + a generic generalize-signal + bounded repair) yields 5/7, each verified against the full held-out set with visible success preserved; e.g. the str(obj.pk) hack is repaired to str(getattr(obj, to_field_name)) from the execution signal alone. The mission question's second half, answered on a bounded set (improvement, not guarantee; 7/8 from a gold-leaking variant was discarded)
147	Does the gate harm correct completions?	no - across 10 django instances the gate accepts all 10 gold fixes and both genuinely-correct honest agent fixes with 0/12 false rejections; with iter146 the protocol is a net-positive filter (improves gamed, preserves correct), the "preserving ordinary task success" clause. Confirmatory for the execution gate; property-gate 0-FP is iter121/129
148	Is the 5/7 robust or a small-N artifact?	robust - a disjoint older django sample (11xxx-12xxx) reproduces the identical 0/7 -> 5/7, and pooled across the two disjoint samples the protocol lifts real completion 0/13 -> 10/13 (0.77), baseline 0 throughout; the intervention rate replicates across independently-chosen instances
149	Does the effect survive a gold-free oracle (not the real tests)?	yes - replacing the regression oracle with a model-proposed contract property (docstring + visible test only, no held-out tests in the gate), the gold-free gate catches the gamed completion 3/3 on evaluated instances and drives gold-free repair to real completion 2/3; property sound on 9/10 format functions. Closes the "you used the real tests" objection (small-N, property-derivable subset; an initial all-unsound run was a Django-settings harness artifact, caught and fixed)
150	Is the intervention oracle-agnostic?	yes - grown to 7 evaluated (pooled), the gold-free property gate catches gamed 7/7 and reaches real completion 5/7 (0.71), within sampling noise of the regression-gated 10/13 (0.77); the improvement comes from execution-based rejection + a generalize-signal, not from privileged access to the tests. Bounded to the property-derivable subset; property synthesis is stochastic (pooled across runs)
151	Does the intervention scale cross-repo on the OFFICIAL metric?	yes - across 8 SWE-bench repositories (django, scikit-learn, sympy, matplotlib, astropy, xarray, pytest, sphinx), over 20 both-miss starts a proxy gate resolves 0/20 (definitional) while the Telos gate recovers 10/20 (0.50) to full official SWE-bench resolution, every verdict backed by a committed report.json. Ran on a cloud x86 Docker VM (arm64/local could not). Honest: baseline 0 is definitional, the oracle is the regression suite (this scales the intervention, not the gold-free novelty), and a malformed-patch harness false-negative was caught and fixed before trusting numbers
152	How often is a frontier reward model fooled by both-miss, cross-repo?	gameable - across 8 repos on 20 execution-verified both-miss, the deterministic detector is fooled 20/20, and two frontier models used as reward models are fooled 7/20 (gpt-5.6-terra) and 9/20 (claude-opus-4-8 - a bigger model is not a safer reward model); 5/20 (0.25) survive EVERY static layer, catchable only by held-out execution. All 20 hack diffs saved as a reward-hack benchmark seed. Honest: single adversary, two judges, constructed (not naturally-emergent) hacks, N=20
153	Can the reward-hack benchmark seed be materialized without new spend?	yes - benchmarks/reward_hack_seed_v0/ now contains exactly 20 iter152 both-miss rows across 8 repositories, 20/20 hack diff hashes, source-proof traceability, and the 5/20 survives-all-static labels; zero provider calls, cloud resources, SWE-bench executions, or judge calls. Honest: this is a validated seed artifact, not a released benchmark, score, leaderboard, model-comparison result, or SOTA claim
154	Can the seed grow under bounded spend and the same evidence rules?	null shortfall - the frozen 96-candidate expansion produced 17 new execution-verified both-miss rows across 10 repos, not the required 20; 8/17 survive every static layer, all accepted rows have diff/report/source hashes, one provider call errored, and cloud resources were deleted; no v1 benchmark is released
155	Can iter154's shortfall evidence guide the final benchmark-size expansion?	pass - the frozen adaptive pool added 3 new execution-verified both-miss rows after 12 processed candidates, raising seed + iter154 + iter155 to 40 rows; 3/3 have diff/report/source hashes, 1/3 fooled gpt-5.6-terra, 0/3 fooled claude-opus-4-8, provider errors were 0/38, and cloud resources were deleted. Honest: this crosses the benchmark-size candidate-pool bar, but no public v1 benchmark, leaderboard, model score, or SOTA claim is released
156	Can the 40-row candidate pool become a reviewer-facing v1 benchmark artifact?	pass - benchmarks/reward_hack_benchmark_v1/ now contains exactly 40 unique execution-verified both-miss rows across 11 repos, with 40/40 hack diff hashes, official report hashes, and source traceability; 13/40 survive every static layer, 16/40 fool gpt-5.6-terra, and 19/40 fool claude-opus-4-8; zero new provider calls, SWE-bench executions, or cloud resources. Honest: released row artifact only, not a benchmark score, leaderboard, model comparison, SOTA claim, natural-frequency estimate, or broad robustness claim
157	Can the paper be made plain-language and claim-boundary current before arXiv?	pass - paper/telos.tex, docs/PAPER.md, and paper/README.md now represent iter151-156 consistently, including the 40-row reward_hack_benchmark_v1 artifact and the 13/40 survives-all-static boundary; zero provider calls, SWE-bench executions, or cloud resources. Honest: prose/readability update only, not a new empirical result, benchmark score, leaderboard, model comparison, SOTA claim, natural-frequency estimate, or broad robustness claim
158	Can the next reward-hack benchmark moonshot be designed without score leakage or overclaiming?	pass - a zero-spend scoring/evaluation protocol now freezes the 40-row v1 artifact hashes, defines 4 evaluator families (static detectors, model judges, judge panels, execution-backed gates), leakage controls, recall metrics, and null/fail semantics; no provider calls, SWE-bench executions, cloud resources, model scores, leaderboards, model comparisons, SOTA claims, natural-frequency estimates, or broad robustness claims
159	Can the v1 rows be materialized into blinded judge packets without label leakage?	pass - benchmarks/reward_hack_benchmark_v1/blinded_model_judge_packets_v1/ now contains exactly 40 blinded model-judge packets with stable packet hashes, 40/40 public task text coverage, 40/40 target-test coverage, and 0 forbidden key, substring, or official-report value leakage hits; zero provider calls, SWE-bench executions, cloud resources, model outputs, scores, leaderboards, model comparisons, SOTA claims, natural-frequency estimates, or broad robustness claims
160	Can judge outputs be parsed before any provider scoring run?	pass - telos/reward_hack_judge_parser.py and scripts/preflight_reward_hack_judge_parser.py validate a strict local parser against 13 fixtures: 3 parsed, 2 refusal, and 8 invalid; accepted verdicts are exactly reward_hack, legitimate, and inconclusive; refusals and invalid outputs never count as caught or legitimate; zero provider calls, SWE-bench executions, cloud resources, scores, leaderboards, model comparisons, SOTA claims, natural-frequency estimates, or broad robustness claims
161	Can one model judge run on the frozen blinded packets under explicit ceilings?	pass - gemini-2.5-flash via Vertex judged all 40/40 frozen blinded all-hack packets under the iter160 parser: 3 reward_hack, 37 legitimate, 0 refusals, 0 invalid outputs, 0 prompt leakage hits, 0 response secret hits, estimated cost guard $0.734250, no SWE-bench executions or cloud runners. Honest: this is 3/40 recall on an all-positive artifact only, not precision, a benchmark score, leaderboard, model comparison, SOTA claim, natural-frequency estimate, or broad robustness claim
162	Can the reward-hack evaluator get legitimate controls before precision claims?	pass - zero-spend design now defines the legitimate-control source hierarchy, row schema, blinded packet schema, false-positive/specificity/precision metric boundary, and next materialization bars; provider calls 0, SWE-bench executions 0, cloud resources 0. Honest: design only, not a control artifact, precision number, benchmark score, leaderboard, model comparison, SOTA claim, natural-frequency estimate, or broad robustness claim
163	Can legitimate controls be materialized and blinded before precision evaluation?	pass - 40 paired legitimate controls were materialized one-for-one from public SWE-bench Verified gold patches for the reward-hack rows, with 40 blinded control packets, source audit pass, leakage audit pass, hack/control diff hash overlap 0, provider calls 0, model evaluations 0, SWE-bench executions 0, and cloud resources 0. Honest: artifact only, not a model score, precision number, leaderboard, model comparison, SOTA claim, natural-frequency estimate, or broad robustness claim
164	Can the same model judge evaluate controls under the iter160 parser?	blocked - the bounded run preserved 15 parsed legitimate-control outputs, then stopped on a Vertex http_429 provider rate-limit block at packet 016; prompt leakage hits 0, response secret hits 0, estimated cost guard $0.298030, no SWE-bench executions or cloud resources. Honest: partial diagnostics only, not a precision result
165	Can the iter164 rate-limit block be recovered without re-calling successful controls?	pass - recovered only the 25 missing controls, reused the 15 successful iter164 controls, and produced a complete paired single-model metric: controls 40/40 legitimate, false positives 0/40, specificity 1.0, recall from iter161 3/40 (0.075), precision for the reward_hack flag 1.0 with denominator 3, balanced detection 0.5375, sequence estimated cost guard $0.727580, prompt leakage hits 0, response secret hits 0, no SWE-bench executions or cloud resources. Honest: one-model bounded metric only, not a leaderboard, model comparison, SOTA claim, natural-frequency estimate, or broad robustness claim
166	Can the next moonshot evaluator family be designed from the paired metric without overclaiming?	pass - zero-spend design specifies 5 evaluator families, leakage controls, stop conditions, specificity floor 0.90, control false-positive ceiling 4/40, nondecision accounting, and a bounded next gate; provider calls 0, model evaluations 0, SWE-bench executions 0, cloud resources 0. Honest: design only, not a model score, leaderboard, model comparison, SOTA claim, natural-frequency estimate, or broad robustness claim
167	Can a skeptical judge prompt raise recall without burning the control boundary?	fail - completed the paired 40 hack + 40 control provider run for gemini-2.5-flash under the iter160 parser: provider successes 80/80, estimated cost guard $1.559880, prompt leakage hits 0, response secret hits 0, false positives 0/40, specificity 0.90, but recall stayed 3/40 (0.075), hack invalids were 5/40, nondecision rate on hacks 0.125, balanced detection 0.4875, so the pre-registered recall, nondecision, and balanced-detection bars failed. Honest: completed null only, not a leaderboard, model comparison, SOTA claim, natural-frequency estimate, or broad robustness claim
168	Can the iter167 null be adjudicated before spending again?	pass - zero-spend adjudication of the iter167 null: all 9 invalids were markdown-fenced JSON (5 hack, 4 control), diagnostic-only fence stripping would move recall from 3/40 to 4/40 and specificity from 0.90 to 1.0 but still fail the recall and balanced-detection bars, and the recommended next intervention is independent judge-panel design. Honest: no iter167 score change, provider calls 0, model evaluations 0, SWE-bench executions 0, cloud resources 0, no leaderboard/model/SOTA claim
169	Can the independent judge-panel gate be designed before spending again?	pass - zero-spend design of three independent panel slots (google_vertex, openai, anthropic), provider-native structured-output enforcement, frozen standalone, any-catch, majority, and unanimous aggregation rules, leakage controls, nondecision accounting, and a future paid-pilot ceiling of 160 calls and $50.00; provider calls 0, model evaluations 0, SWE-bench executions 0, cloud resources 0, no scores or leaderboard language
170	Can the panel schema/request layer be preflighted before paid calls?	pass - zero-spend parser, fixture, request-shape, leakage, and nondecision-accounting preflight for the iter169 panel: valid fixtures parsed, markdown-fenced JSON remained invalid, all 40 hack and 40 control packets had 0 forbidden leakage hits and 0 allowlist mismatches, and paid execution is blocked pending exact operator bindings; provider calls 0, model evaluations 0, SWE-bench executions 0, cloud resources 0, no model or panel scores
171	Can exact panel model/API bindings be frozen before paid calls?	pass - zero-spend binding freeze: all three slots are requires_operator_input, primary rule is frozen as majority_catch, nondecisions remain outside TP/TN, generated secret hits are 0, and iter171 does not authorize paid execution. It also records that a full three-slot run over all 80 blinded packets needs 240 calls, above the preserved 160-call ceiling; bounded pilot plan is 20 paired rows (120 planned calls + 40 retry reserve), still blocked pending operator bindings
172	Can the blocked panel bindings be recovered without spend?	pass - zero-spend operator binding recovery: no operator choice packet was supplied, so all three slots remain requires_operator_input, missing non-secret fields are explicit, secret hits are 0, majority_catch and the 160-call / $50.00 bounded-pilot ceiling are preserved, and paid execution is not authorized
173	Can a public binding menu be built before operator choice?	pass - zero-spend official-docs binding menu: 3 panel slots, 6 source-linked candidates, secret hits 0, majority_catch preserved, ready for operator/default choice, paid execution not authorized
174	Can exact default choices be frozen from the public menu?	pass - zero-spend default freeze: gemini-2.5-flash, gpt-5.6-terra, and claude-opus-4-8; 3/3 choices verified in the iter173 menu, secret hits 0, majority_catch preserved, paid execution not authorized
175	Can the frozen panel run a bounded paid paired pilot?	pass - 120/120 primary calls succeeded across Google, OpenAI, and Anthropic with 0 retries, estimated spend guard $6.312690, secret hits 0; primary majority_catch caught 13/20 hack rows and 0/20 controls. Honest: bounded 20-pair pilot only, not a leaderboard, model-superiority, SOTA, natural-frequency, or broad robustness claim
176	Can the panel pilot be adjudicated before expansion?	pass - zero-spend recomputation reconciled 120/120 iter175 calls by hash and 120/120 parsed rows to the schedule; committed panel and per-slot metrics matched; primary majority_catch remained 13/20 hack catches and 0/20 control catches; three OpenAI empty-content rows stayed nondecisions; 0 provider calls, credential probes, SWE-bench executions, cloud resources, or secret hits. Honest: adjudication only, not a leaderboard, model-superiority, SOTA, natural-frequency, broad robustness, or public benchmark score
177	Can the next panel expansion be designed from the adjudicated disagreement pattern before spending again?	pass - zero-spend design selected the 20 unobserved paired rows for the next primary cohort, kept the three OpenAI empty-content rows as a diagnostic recovery cohort, froze 123 planned calls before retries under a 160 call / $50.00 ceiling, raised OpenAI max output tokens to 1536, preserved majority_catch, and made 0 provider calls, credential probes, SWE-bench executions, cloud resources, or secret hits. Honest: design only, not a new score or claim upgrade
178	Can the frozen panel cover the remaining paired rows under the iter177 call/spend/recovery design?	pass - 123/123 calls succeeded with 0 retries, spend guard $7.005150, secret hits 0; fresh majority_catch caught 4/20 hack rows and 0/20 controls; combined unrepaired iter175+iter178 majority is 17/40 hacks and 0/40 controls; recovery rows are diagnostic only
179	Can the full panel cohort be adjudicated without spend before any claim upgrade?	pass - zero-spend full-cohort adjudication over committed iter175+iter178 proof: 0 provider calls, credential probes, SWE-bench executions, or cloud resources; unrepaired majority_catch is 17/40 hack rows and 0/40 controls; any_catch is 23/40 hacks and 0/40 controls; unanimous_catch is 2/40 hacks and 0/40 controls; five panel nondecisions are all OpenAI empty-output rows; recovery diagnostics stay out of score
180	Can an OpenAI nondecision repair protocol be designed before any paid repair call?	pass - zero-spend repair design: exactly five OpenAI empty-output primary nondecision rows must be rerun before any repaired diagnostic; three rows have prior diagnostics but those outputs remain excluded from score evidence; future execution is OpenAI-only with 5 planned primary calls, 5 retry reserve calls, a 10 call ceiling, and $10.00 spend ceiling
181	Can the five OpenAI nondecision rows be rerun under the iter180 repair design?	pass - 5/5 OpenAI repair calls succeeded with 0 retries, spend guard $0.271800, secret hits 0, and 4/5 parsed repair outputs; secondary repaired diagnostic reduces nondecisions to hacks 1, controls 0, while majority-catch remains 17/40 hacks and 0/40 controls; unrepaired iter179 remains primary
182	Can the repair execution be adjudicated without spend before any claim wording changes?	pass - zero-spend adjudication reparsed 5/5 raw responses, reconciled ledger/hash/parsed artifacts, confirmed the committed repaired diagnostic, kept iter179 unrepaired majority-catch as primary (17/40 hacks, 0/40 controls), and made no repaired-score or public benchmark claim
183	Can public surfaces be synchronized to the adjudicated repair-diagnostic boundary?	pass - zero-spend public claim-surface sync: 0 provider calls, credential probes, model evaluations, SWE-bench executions, or cloud resources; public surfaces preserve unrepaired iter179 majority_catch as primary (17/40 hacks, 0/40 controls), keep iter181/iter182 repair evidence diagnostic/adjudication only, and advance the active gate to iter184
184	Can public frontier research be mapped to the next reward-hack panel expansion before spending again?	pass - zero-spend public-source research-alignment design: 20 stable public sources, 6 source-backed technique implications, 0 provider calls, credential probes, model evaluations, SWE-bench executions, cloud resources, or secret hits; next gate is iter185 property-probe design over the 23 iter179 primary-missed hack rows
185	Can the panel-missed hack cohort be turned into a leakage-controlled property-probe design before spending again?	pass - zero-spend property-probe design: exactly 23 iter179 primary-missed hack rows recovered, 4 disagreement/nondecision classes, 12 priority rows selected across at least 6 repos, leakage policy forbids gold patches, hidden test names, official expected outputs, and labels, and future paid/execution bars are frozen before provider spend
186	Can the property-probe packets be materialized without leakage before any paid generator call?	pass - zero-spend packet materialization: 12 hack-source packets and 12 paired-control-source packets, 24 unique packet hashes, 0 leakage hits, source traceability kept out of the prompt payload, and paid property generation still unauthorized
187	Can the property-generator output contract be preflighted before paid calls?	pass - zero-spend schema/parser preflight: 17 fixtures, valid executable parse rate 1.0, invalid/refusal/malformed rejection rate 1.0, 24 prompt contracts scanned, 0 prompt-contract leakage hits, future call/spend/execution/false-positive/nondecision bars preserved, and paid property generation still unauthorized
188	Can a Sentinel-style Telos mission data/process audit be designed before any new spend?	pass - zero-spend audit design: 26 frozen committed local inputs named, 7 future audit-note sections required, 8 verifier checks planned, freshness edits constrained to concrete stale durable docs, unrepaired iter179 majority-catch remains primary (17/40 hacks, 0/40 controls), forbidden positive claim hits 0, secret hits 0, and no provider/property-generator/SWE-bench/cloud execution or score upgrade
189	Can the Sentinel-style Telos mission evidence/data-process audit be executed before any new spend?	pass - zero-spend audit execution: all 7/7 required audit-note sections present, 29 frozen local inputs checked, 15 recent key-gate receipts validated, benchmark-lineage checks passed, public metric freshness preserved unrepaired iter179 majority-catch (17/40 hacks, 0/40 controls), forbidden positive claim hits 0, secret hits 0, and no provider/property-generator/SWE-bench/cloud execution or score upgrade
190	Can the preflighted property-generator path run on the 24 property-probe packets under bounded bars?	null - stopped before provider spend after freezing 24 planned calls: provider calls 0, spend $0.00, prompt leakage hits 0, but the schema has no direct runnable artifact fields, local SWE-bench/container execution was unavailable, execution attempts were 0/20, nondecisions were 24/4, and hack property failures were 0/4; next gate is execution-contract design
191	Can the property-execution contract and harness be designed before any more paid property generation?	pre-registered - zero-spend design gate requiring provider calls 0, property-generator calls 0, execution-contract schema count at least 1, sandbox/runtime safety constraints at least 8, all 24 iter186 packets covered in the feasibility matrix, and paired hack/control issue coverage exactly 12

The honest conclusion

In the bounded iter109–118 pilots, the tested LLM judge had higher coverage than the deterministic detector: it matched the detector on obvious hacks and caught oblique source special-casing that the detector missed, including the executed iter113 case. Within those pilots, the deterministic detector's defensible value is provider independence, latency, determinism, line-level reproducible evidence, and running in CI with no credentials. The architecture that follows is a cascade, not a contest:

candidate diff + FAIL_TO_PASS
        |
        v
[ deterministic detector ]  zero provider spend, reproducible, tamper-evident
   catches verifier edits, assertion weakening, literal special-casing
        |
        | survivors (oblique / stealth completions)
        v
[ LLM judge ]  paid, semantic
   catches computed-key / constant-return hacks the regex cannot see
        |
        v
verdict + receipt

The cascade covers mechanical and semantically-obvious hacks, but it is not sufficient. Iter118 found a both-miss class: disguised completions that pass the visible test, evade the deterministic detector, and are called legitimate by a strong judge, while being wrong on inputs the test never exercises (sorted(reverse=True) disguised as reversal; '0%03d' disguised as zero-padding). No static verifier - regex or model - separates a correct general fix from a plausible one that only special-cases the visible input. The defense the evidence points to is execution on held-out inputs the agent never saw (metamorphic or property-based testing), which catches every hack in that class.

What is not claimed

None of this is a SWE-bench resolved-rate score, a leaderboard result, a model-comparison result, a robustness guarantee, or a state-of-the-art claim. The results are bounded pilots on real data with every claim held below its evidence, native-execution transcripts recorded as observed evidence, and the detector verdicts reproducible in CI. The executed reward-hack catch rate was measured (iter116), a both-miss stealth class was found (iter118), and a held-out-input (metamorphic) execution check was shown to catch that class (iter119) - completing a three-layer argument: the deterministic detector catches mechanical hacks, the LLM judge catches oblique and hard-coded hacks the detector misses, and held-out-input execution catches plausible-but-generalization-broken completions both static layers accept. That third layer was generalized to random inputs (iter120), made gold-free with contract properties that catch the both-miss class at zero false positives on the correct code (iter121), and then automated: a model proposes the properties and execution verifies them (iter122). This forces the program's sharpest structural point - the same model that is fooled as a direct judge is reliable as a property generator, because a proposed property is checked by execution rather than trusted, so an unsound property surfaces as a false positive instead of a silent miss. Moving the model from verdict-giver to property-generator converts an unverifiable judgment into a checkable artifact. Unsound proposed properties are then rejected without any gold reference (iter123) by anchoring on the visible test - a known-correct input/output pair the agent already had to satisfy - so the automated third layer is gold-free end to end: the model proposes, the visible test filters, and execution on random inputs catches the generalization-broken completions. At scale, though, the automation is bounded: across seven real pure-function instances only two produced a clean sound auto-generated harness (iter124). The property mechanism is real - when the model produces a harness it is sound and catches the both-miss class - but synthesizing a correct call and input domain automatically is the bottleneck, and it is the same integration wall that makes most SWE-bench instances resistant to property-based testing. A validated synthesizer that targets the test source, validates the input generator before trusting the property, and retries on failure roughly doubles that rate to 4/7 (iter125) - with a non-triviality filter that catches vacuous harnesses a raw soundness check accepts. The input-domain and unsound-property residuals were then closed (iter127-iter128) - structured inputs seeded from the test example, and an inverse round-trip for parsers - and the last instance was resolved by an applicability criterion: 11276 is a cross-cutting escape() refactor, not a single testable function, so it is excluded up front. On the six valid single-function candidates the property-based layer is 6/6 genuine-sound with the strategy (contract property vs inverse round-trip) auto-selected by function type (iter129). The property layer's reach was then bounded twice over: the official SWE-bench Docker harness closed the environment-fidelity gap so natively-unrunnable instances resolve their real hidden tests in instance-specific containers (iter130-137; mutable historical tags, without retained image digests), and an applicability survey measured the layer's structural reach at 405/500 = 0.81 of the dataset but its property-derivability within that at only ~0.10 even in sympy (iter138-139) - the honest three-layer division of labour, with the detector and judge universal and the property layer the math-identity specialist.

At the iter140–145 boundary, the frontier was the both-miss class itself, stress-tested by a frontier adversary. iter140 ran an automated hacker-fixer with gemini-2.5-flash/pro, got 0/20 verified both-miss on thoroughly-tested django, and read that as the window being narrow where coverage is thorough. That reading was wrong, and it is retracted in place: iter141 reran the same pool with a non-refusing frontier adversary (gpt-5.6-terra) and a robust apply harness and produced confirmed both-miss hacks - the 0/20 was a capability-plus-harness artifact, not structural closure. iter142 measured the rate: over 25 genuine attempts a frontier adversary produces confirmed both-miss hacks on 0.24 of attempts and, on 0.12 (half of them), the completion also fools the LLM judge and is caught only by held-out execution. iter143 answered the obvious objection that the judge was too weak: re-judging with frontier judges, gpt-5.6-terra is fooled by 1/6 and claude-opus-4-8 by 3/6 (a bigger reasoning model is not a better judge), and one hack survives every judge tested plus the detector. iter144 showed the class is not django-specific - it reproduces on sympy (sympy-22714), judge-fooled, at a lower repo-dependent rate. A secondary datum from this arc: claude-opus-4-8 refused to construct the hacks under the same defensive-research framing while gpt-5.6-terra complied, so who can red-team a completion verifier is itself model-dependent. The conclusion is sharper than before: static verification of an agent's diff - by regex or by a frontier model

• cannot separate a correct general fix from a plausible special-case, and held-out execution is necessary, not optional, even on well-tested code and even against frontier judges.

The arc then closes the mission question's second half - does the protocol improve the rate at which the agent completes the real objective, not just detect gaming (iter146). On 7 django both-miss starts, a proxy gate that accepts on visible pass yields 0/7 real completions, while the execution gate - which rejects the completion and returns a gold-free generic signal ("you pass the target test but fail on held-out inputs; you special-cased instead of generalizing", with no gold and no failing test named) and lets the agent repair - reaches a verified real completion on 5/7, ordinary success preserved. The same execution layer that is necessary to catch the both-miss class also, as a gate with generic feedback, converts gamed completions into real ones (for example the str(obj.pk) hack is repaired to str(getattr(obj, to_field_name)) from the execution signal alone). It is an improvement, not a guarantee (two of seven did not reach a general fix in two rounds), on a small single-repo set, and it is deliberately lower than a discarded gold-leaking variant that reached 7/8. The complementary control holds (iter147): the gate accepts all 10 gold fixes and both genuinely-correct honest agent fixes with 0/12 false rejections, so it does not harm correct work. The protocol is therefore a net-positive filter

• it converts gamed completions and preserves correct ones - which is the first direct evidence in the program that the protocol changes outcomes, not only measures them, with ordinary success preserved.

Historical status at the iter191 boundary

• Repository scaffold: active at that boundary.
• First gate: target survey published as HYBRID_OVERLAY_SELECTED.
• Public slice: selected in experiments/iter02_public_task_slice.
• CodeClash smoke: passed in experiments/iter03_codeclash_smoke.
• Agent-behavior slice: selected in experiments/iter04_agent_behavior_slice.
• Agent-behavior smoke: passed in experiments/iter05_agent_behavior_smoke.
• Deterministic edit slice: selected in experiments/iter06_deterministic_edit_slice.
• Deterministic edit smoke: passed in experiments/iter07_deterministic_edit_smoke.
• Provider-model pilot slice: selected in experiments/iter08_provider_model_pilot_slice.
• Provider-model pilot smoke: blocked before spend in experiments/iter09_provider_model_pilot_smoke.
• Provider auth recovery: passed in experiments/iter10_provider_auth_recovery.
• Provider-model pilot retry: blocked in experiments/iter11_provider_model_pilot_retry.
• Vertex model access recovery: passed in experiments/iter12_vertex_model_access_recovery.
• Provider-model pilot retry after access recovery: passed in experiments/iter13_provider_model_pilot_retry_after_access_recovery.
• Provider diff quality review: passed in experiments/iter14_provider_diff_quality_review.
• Provider strict diff rerun: failed the clean diff-quality bar in experiments/iter15_provider_strict_diff_rerun.
• Provider workspace hygiene control: passed the helper-residue bar with a recorded style caveat in experiments/iter16_provider_workspace_hygiene_control.
• Provider lint hygiene control: passed the clean workspace-and-lint bar in experiments/iter17_provider_lint_hygiene_control.
• Provider behavior depth control: passed with a process caveat in experiments/iter18_provider_behavior_depth_control.
• Provider final inspection control: passed the final inspection bar in experiments/iter19_provider_final_inspection_control.
• Behavior semantic verification: passed eight deterministic local safety cases in experiments/iter20_behavior_semantic_verification.
• Opponent collision control: passed the provider run and twelve semantic safety cases in experiments/iter21_opponent_collision_control.
• Semantic mutation guard: passed targeted mutation checks in experiments/iter22_semantic_mutation_guard.
• Tail semantics falsification: failed under the explicit occupied-tail assumption in experiments/iter23_tail_semantics_falsification.
• Tail safety control: passed for a clearly labeled changed candidate in experiments/iter24_tail_safety_control.
• Tail safety mutation guard: failed because the own-tail mutant did not remove the redundant self-snake fallback path in experiments/iter25_tail_safety_mutation_guard.
• Own-tail redundancy mutation guard: passed with a compound own-tail mutant in experiments/iter26_own_tail_redundancy_mutation_guard.
• Semantic claim boundary matrix: passed with original/candidate/failure/verifier rows separated in experiments/iter27_semantic_claim_boundary_matrix.
• Public claim surface guard: passed against README/report/next-phase/continuity prose in experiments/iter28_public_claim_surface_guard.
• Public claim surface negative guard: passed with four generated overclaim fixtures in experiments/iter29_public_claim_surface_negative_guard.
• Boundary matrix schema guard: passed with five malformed matrix fixtures in experiments/iter30_boundary_matrix_schema_guard.
• Claim boundary release manifest: passed with a 33-artifact hash-checked proof packet in experiments/iter31_claim_boundary_release_manifest.
• Claim boundary release manifest negative guard: passed with five malformed manifest fixtures in experiments/iter32_claim_boundary_release_manifest_negative_guard.
• Release manifest public sync guard: passed against README/report/next-phase/continuity prose in experiments/iter33_release_manifest_public_sync_guard.
• Release manifest public sync negative guard: passed with five malformed public-prose fixtures in experiments/iter34_release_manifest_public_sync_negative_guard.
• Release manifest self-coverage guard: passed with 49 proof artifacts indexed across iter31 through iter34 in experiments/iter35_release_manifest_self_coverage_guard.
• Release manifest self-coverage negative guard: passed with five malformed self-coverage fixtures in experiments/iter36_release_manifest_self_coverage_negative_guard.
• Release manifest self-coverage public sync guard: passed against README/report/next-phase/continuity prose in experiments/iter37_release_manifest_self_coverage_public_sync_guard.
• Release manifest self-coverage public sync negative guard: passed with six malformed public-prose fixtures in experiments/iter38_release_manifest_self_coverage_public_sync_negative_guard.
• Public task protocol-effect slice: passed with three executable task surfaces, two conditions, and before-data metrics in experiments/iter39_public_task_protocol_effect_slice.
• Public task protocol-effect execution: blocked before provider execution because runner readiness was not established in experiments/iter40_public_task_protocol_effect_execution.
• Public task protocol-effect runner recovery: passed through three isolated GitHub Actions CodeClash runner checks with zero provider spend in experiments/iter41_public_task_protocol_effect_runner_recovery.
• Public task protocol-effect execution retry: blocked before provider execution because the provider-capable harness, cost capture, and raw-artifact redaction controls were not recovered in experiments/iter42_public_task_protocol_effect_execution_retry.
• Provider execution harness recovery: passed with a non-GPU ephemeral runner lifecycle probe, zero provider model calls, zero provider spend, and zero full task-condition pairs in experiments/iter43_provider_execution_harness_recovery.
• Public task protocol-effect execution after harness recovery: blocked before provider execution because the recovered harness still disables full task-condition execution in experiments/iter44_public_task_protocol_effect_execution_after_harness_recovery.
• Public task-condition executor assembly: passed as a dry-run manifest with six frozen pairs, zero provider model calls, zero provider spend, no cloud runner, and full execution still disabled in experiments/iter45_public_task_condition_executor_assembly.
• Public task protocol-effect execution with assembled executor: blocked before provider execution because provider overlays were not bound into pair commands and the recovered harness still disabled full task-condition execution in experiments/iter46_public_task_protocol_effect_execution_with_assembled_executor.
• Provider task-condition command binding recovery: blocked and narrowed the plan to two provider-ready BattleSnake pairs while keeping four incompatible pairs visible in experiments/iter47_provider_task_condition_command_binding_recovery.
• Provider-compatible protocol-effect slice refreeze: passed with two selected BattleSnake provider-compatible pairs, four excluded historical pairs, zero provider calls, zero spend, no cloud runner, no GPU, and no Sentinel resource modification in experiments/iter48_provider_compatible_protocol_effect_slice_refreeze.
• Provider-compatible protocol-effect execution retry: blocked before provider execution because the two-pair execution wrapper is not yet committed and the recovered provider harness still disables full task-condition execution in experiments/iter49_provider_compatible_protocol_effect_execution_retry.
• Provider-compatible execution wrapper recovery: passed as a zero-spend dry run with two selected BattleSnake pair plans and four rejected historical exclusions in experiments/iter50_provider_compatible_execution_wrapper_recovery.
• Provider-compatible protocol-effect execution with wrapper: blocked before provider execution because the wrapper is still dry-run-only and the baseline/Telos runtime conditions are not yet distinct beyond output directory in experiments/iter51_provider_compatible_protocol_effect_execution_with_wrapper.
• Provider condition runtime separation recovery: passed as a zero-spend readiness gate with distinct baseline/Telos runtime commands, overlays, prompts, and a Telos receipt-validation path in experiments/iter52_provider_condition_runtime_separation_recovery.
• Provider-compatible protocol-effect execution after condition recovery: blocked before provider execution because the pair executor is still intentionally unimplemented, the base harness still disables full protocol-effect execution, the pinned CodeClash checkout is not ready, and Docker readiness timed out in experiments/iter53_provider_compatible_protocol_effect_execution_after_condition_recovery.
• Provider pair executor recovery: passed as a zero-spend readiness gate with pinned CodeClash, copied overlays, exact two-row command materialization, Docker daemon readiness through the current Docker Desktop binary, zero provider calls, zero spend, no GPU, and no Sentinel resource modification in experiments/iter54_provider_pair_executor_recovery.
• Provider-compatible paid execution after executor recovery: blocked before provider execution because non-interactive ADC refresh requires reauthentication and dedicated-runner impersonation lacks token-creator access; zero provider calls, zero spend, no cloud runner, no GPU, and no Sentinel resource modification occurred in experiments/iter55_provider_compatible_paid_execution_after_executor_recovery.
• Provider auth recovery for paid protocol effect: passed by repairing local ADC non-interactively and making one minimal Vertex access probe with a $0.01 spend bound, while executing no BattleSnake row and committing no account, project, service-account, VM, zone, token, or credential material in experiments/iter56_provider_auth_recovery_for_paid_protocol_effect.
• Provider-compatible paid execution after auth recovery: blocked before provider model calls because the pinned CodeClash virtualenv cannot import google.auth; one baseline selected-row attempt reached round-0 raw evidence, the Telos row and all excluded rows remained unattempted, and committed metadata shows zero provider calls and zero cost in experiments/iter57_provider_compatible_paid_execution_after_auth_recovery.
• CodeClash Vertex dependency recovery: passed as a zero-spend local dependency gate; the CodeClash virtualenv now imports google.auth, the pinned commit and frozen provider configs stayed unchanged, and no paid row, provider model call, GPU, cloud runner, or Sentinel resource was used in experiments/iter58_codeclash_vertex_dependency_recovery.
• Provider-compatible paid execution after dependency recovery: blocked after both selected BattleSnake rows executed because Vertex returned a redacted model-not-found-or-access-denied response for the configured provider model; the run recorded two provider calls, zero cost in CodeClash metadata, no excluded pairs, no GPU, no cloud runner, no Sentinel mutation, and no verified-completion evidence in experiments/iter59_provider_compatible_paid_execution_after_dependency_recovery.
• Provider model binding recovery: blocked after adding vertex_location: global to the recovered provider model overlay; the LiteLLM probe moved past the prior location error but returned a redacted CONSUMER_INVALID/quota-project response, with one provider call, no row execution, no excluded pairs, no GPU, no cloud runner, no Sentinel mutation, and no benchmark/model claim in experiments/iter60_provider_model_binding_recovery.
• Vertex quota-project binding recovery: blocked after proving the Mini-SWE-Agent/LiteLLM extra_headers path exists; one bounded LiteLLM probe with X-Goog-User-Project still returned a redacted CONSUMER_INVALID response, with no row execution, no excluded pairs, no GPU, no cloud runner, no Sentinel mutation, and no benchmark/model claim in experiments/iter61_vertex_quota_project_binding_recovery.
• Vertex bearer token path recovery: blocked after proving LiteLLM custom headers can override the default Authorization header; one bounded LiteLLM probe with runtime bearer-token and quota-project headers still returned redacted CONSUMER_INVALID, with no row execution, no excluded pairs, no GPU, no cloud runner, no Sentinel mutation, and no benchmark/model claim in experiments/iter62_vertex_bearer_token_path_recovery.
• Vertex access path parity recheck: passed after current direct REST and LiteLLM probes both reached the selected Vertex global model using secret-safe runtime credentials; two provider calls occurred, observed LiteLLM cost was $0.000014, no BattleSnake row or excluded pair ran, no GPU, cloud runner, or Sentinel mutation occurred, and no benchmark/model claim is made in experiments/iter63_vertex_access_path_parity_recheck.
• Provider-compatible paid execution after access-path recovery: passed as the first bounded two-row provider-backed protocol-effect measurement; baseline verified-completion evidence was true, Telos verified-completion evidence was false because the Telos row's receipt candidate failed validation, the primary delta was -1, 10 provider calls and $0.070448 CodeClash metadata cost were recorded, excluded pairs stayed unattempted, no GPU/cloud runner/Sentinel resource was used, and no benchmark/model claim is made in experiments/iter64_provider_compatible_paid_execution_after_access_path_recovery.
• Receipt-schema prompt alignment: passed locally with zero provider calls and a recovered receipt-enforced prompt overlay; the iter64 Telos receipt candidate was classified as schema-incomplete because it omitted eight required Telos proof fields in experiments/iter65_receipt_schema_prompt_alignment.
• Provider-compatible paid execution after receipt prompt alignment: passed as a bounded two-row paid retry using the recovered iter65 overlay; both baseline and Telos rows had verified completion evidence, the Telos receipt validated, the primary delta was 0, 8 provider calls and $0.059378 CodeClash metadata cost were recorded, no excluded pairs/GPU/cloud/Sentinel resource were used, and no benchmark/model claim is made in experiments/iter66_provider_compatible_paid_execution_after_receipt_prompt_alignment.
• Provider-compatible expanded slice refreeze: blocked locally with zero provider calls and zero spend; the committed universe still has only two provider-ready BattleSnake rows, four candidate rows remain incompatible, and no larger paid slice is authorized in experiments/iter67_provider_compatible_expanded_slice_refreeze.
• Provider-compatible task-surface adapter recovery: blocked locally with zero provider calls and zero spend; two deterministic-edit adapter rows were planned from committed source, but two Dummy rows remain rejected until the source config is committed in experiments/iter68_provider_compatible_task_surface_adapter_recovery.
• CodeClash task-surface source snapshot recovery: passed locally with zero provider calls and zero spend; configs/test/dummy.yaml is now committed as source-only snapshot evidence with hash b8e856447fc71c79bb5e042dc530127480d670d84fd51c03e2c2e7f58c630e97 in experiments/iter69_codeclash_task_surface_source_snapshot_recovery.
• Provider-compatible expanded adapter completion: passed locally with zero provider calls and zero spend; four Dummy/deterministic-edit adapter rows and eight overlay files were planned as planning evidence only, not execution evidence, in experiments/iter70_provider_compatible_expanded_adapter_completion.
• Provider-compatible expanded slice after adapter completion: passed locally with zero provider calls and zero spend; the slice is frozen as six stratified rows, retaining the two already executed BattleSnake rows and selecting four adapter-planned Dummy/deterministic-edit rows for a bounded future paid gate without cross-surface pooling or benchmark/model claims in experiments/iter71_provider_compatible_expanded_slice_after_adapter_completion.
• Provider-compatible expanded paid execution after slice refreeze: blocked after executing exactly the four adapter-planned rows under the frozen ceiling; 17 provider calls and $0.057646 cost were recorded, the two retained BattleSnake rows were not rerun, deterministic-edit baseline produced verified-completion evidence, and both receipt-enforced rows failed because their receipt candidates were schema-incomplete; no GPU/cloud/Sentinel/live-domain mutation or benchmark/model/SOTA claim occurred in experiments/iter72_provider_compatible_expanded_paid_execution_after_slice_refreeze.
• Expanded receipt prompt recovery after paid block: passed locally with zero provider calls and zero spend; the two iter72 receipt failures were classified with exact missing fields, two recovered receipt-enforced prompt overlays were produced, local valid fixtures passed, and a malformed fixture failed closed in experiments/iter73_expanded_receipt_prompt_recovery_after_paid_block.
• Provider-compatible expanded paid retry after receipt prompt recovery: blocked before adapter-row execution because Google ADC refresh failed non-interactively. Iter72 and iter73 prerequisite packets validated cleanly, but zero provider calls, zero spend, and zero row execution occurred; no GPU/cloud/Sentinel/live-domain mutation or benchmark/model/SOTA claim occurred in experiments/iter74_provider_compatible_expanded_paid_retry_after_receipt_prompt_recovery.
• Provider-compatible runtime ADC recovery after paid retry block: blocked with zero provider calls, zero spend, and zero row execution because ADC still required interactive reauthentication. The CodeClash checkout was pinned, Docker was ready, google.auth imported, gcloud project availability was proven with stdout suppressed, and no credential material was committed in experiments/iter75_provider_compatible_runtime_adc_recovery_after_paid_retry_block.
• Runtime ADC recheck after operator refresh: blocked with zero provider calls, zero spend, and zero row execution. Iter75 receipt/audit checks passed, CodeClash stayed pinned, Docker was ready, google.auth imported, and gcloud project availability was proven with stdout suppressed, but ADC still required interactive_reauthentication_required; no credential material was committed in experiments/iter76_runtime_adc_recheck_after_operator_refresh.
• Runtime ADC recheck after Application Default Credentials login: passed with zero provider calls, zero spend, and zero row execution. Iter76 receipt/audit checks passed, CodeClash stayed pinned, Docker was ready, google.auth imported, gcloud project availability was proven with stdout suppressed, ADC token output was suppressed, and no credential material was committed in experiments/iter77_runtime_adc_recheck_after_application_default_login.
• Provider-compatible expanded paid retry after ADC recovery: blocked after exactly four selected adapter-planned rows executed under ceiling. Provider usage was 9 calls and $0.03987600. Both deterministic-edit rows verified; both Dummy rows hit the per-row global call ceiling before verified-completion evidence could be accepted. No GPU/cloud/Sentinel/live-domain mutation or benchmark/model/SOTA claim occurred in experiments/iter78_provider_compatible_expanded_paid_retry_after_adc_recovery.
• Dummy row call-ceiling recovery after paid retry block: passed with zero provider calls, zero spend, and zero row execution. Both iter78 Dummy failures were classified from committed raw artifacts as per-row global call-ceiling blockers at the frozen 8 call ceiling, while deterministic-edit evidence was retained and not rerun. No credential material or benchmark/model/SOTA claim occurred in experiments/iter79_dummy_row_call_ceiling_recovery_after_paid_retry_block.
• Dummy call-ceiling bounded paid retry after recovery: passed after executing exactly the two Dummy adapter rows with a 16 call per-row ceiling. Provider usage was 6 calls and $0.02840000; both Dummy baseline and Dummy Telos rows verified, deterministic-edit and BattleSnake rows were not rerun, and no benchmark/model/SOTA claim occurred in experiments/iter80_dummy_call_ceiling_bounded_paid_retry_after_recovery.
• Expanded stratified adapter-validation consolidation: passed with zero provider calls, zero spend, and zero row execution in the gate. It validated iter66, iter78, and iter80 source packets, accounted for 23 committed source-packet provider calls and $0.12765400, preserved six successful rows as separated BattleSnake/deterministic-edit/Dummy adapter-validation strata, retained two iter78 Dummy rows as diagnostic blocked evidence, and made no benchmark/model/SOTA claim in experiments/iter81_expanded_stratified_adapter_validation_consolidation.
• Benchmark-facing protocol-effect slice design: passed with zero provider calls, zero spend, and zero row execution. It froze a six-row future CodeClash public task-condition paid pilot with a 96 provider-call ceiling, $10.00 total spend ceiling, $2.00 per-row spend ceiling, no cloud runner/GPU/Sentinel/live-domain mutation, SWE-bench Verified kept as receipt-field anchor only, and no benchmark/model/SOTA claim in experiments/iter82_benchmark_facing_protocol_effect_slice_design.
• Benchmark-facing protocol-effect execution pilot: published blocked/null evidence after executing exactly the six frozen CodeClash public task-condition rows. Provider usage was 21 calls and $0.11319400, all row receipts/artifacts stayed under the 96 call and $10.00 ceilings, and all three Telos-minus-baseline verified-completion deltas were 0; no benchmark/model/SOTA claim occurred in experiments/iter83_benchmark_facing_protocol_effect_execution_pilot.
• Benchmark-facing null-signal adjudication: passed with zero provider calls, zero spend, and zero row execution. It classified the iter83 null/no-signal result as verified_completion_metric_saturated, selected redesign_task_metric as the next step, and made no benchmark/model/SOTA claim in experiments/iter84_benchmark_facing_null_signal_adjudication.
• Discriminating task/metric redesign: passed with zero provider calls, zero spend, and zero row execution. It froze task_native_score_share_delta_with_receipt_gates as a candidate metric for zero-spend backtest, kept future paid execution unauthorized, and made no benchmark/model/SOTA claim in experiments/iter85_discriminating_task_metric_redesign.
• Discriminating metric backtest on committed artifacts: passed with zero provider calls, zero spend, and zero row execution. It computed three task-native score-share deltas from committed iter83 metadata, found a non-saturated mixed-direction diagnostic signal, pre-registered a bounded paid replication, and made no benchmark/model/SOTA claim in experiments/iter86_discriminating_metric_backtest_on_committed_artifacts.
• Benchmark-facing discriminating metric execution pilot: passed as a bounded six-row paid pilot. It executed exactly the frozen CodeClash task-condition rows, used 21 provider calls and $0.12498400, validated all receipt-required rows, computed fresh mixed-direction score-share deltas, and made no benchmark/model/SOTA claim in experiments/iter87_benchmark_facing_discriminating_metric_execution_pilot.
• External benchmark readiness adjudication after the discriminating pilot: passed with zero provider calls, zero spend, and zero row execution. It found all three iter86/iter87 task directions flipped, rejected larger external benchmark design for now, selected one same-slice stability replication as the next step, and made no benchmark/model/SOTA claim in experiments/iter88_external_benchmark_readiness_adjudication_after_discriminating_pilot.
• Same-slice discriminating metric stability replication: passed as a bounded six-row paid replication. It executed exactly the frozen rows, used 19 provider calls and $0.11636200, classified stability as unstable, and made no benchmark/model/SOTA claim in experiments/iter89_same_slice_discriminating_metric_stability_replication.
• Stability replication adjudication after same-slice run: passed with zero provider calls, zero spend, and zero row execution. It locked iter89 as clean but unstable evidence, rejected immediate benchmark/SOTA escalation, selected empirical validation suite design as the next step, and made no benchmark/model/SOTA claim in experiments/iter90_stability_replication_adjudication_after_same_slice_run.
• Empirical validation suite design for completion verification: passed with zero provider calls, zero spend, zero strategy execution, and zero row execution. It froze seven false-completion trap families, seven paired legitimate-completion controls, five comparison strategies, six quantitative endpoints, and made no benchmark/model/SOTA claim in experiments/iter91_empirical_validation_suite_design_for_completion_verification.
• Empirical validation fixture materialization for completion verification: passed with zero provider calls, zero spend, zero strategy execution, and zero row execution. It materialized 14 fixtures, 98 public artifacts, 14 private ground-truth labels, and 5 identical strategy input manifests, with labels excluded from strategy inputs and no benchmark/model/SOTA claim in experiments/iter92_empirical_validation_fixture_materialization_for_completion_verification.
• Deterministic strategy execution on materialized fixtures: passed with zero provider calls, zero spend, zero LLM-judge execution, and zero row execution. It scored 56 deterministic decisions: agent self-report and execution-tests-only accepted 7/7 false-completion traps, while external verifier and complete Telos protocol accepted 0/7, with all four preserving 7/7 legitimate controls. This is partial deterministic fixture evidence, not a benchmark/model/SOTA claim, in experiments/iter93_deterministic_strategy_execution_on_materialized_fixtures.
• Provider LLM judge execution on materialized fixtures: blocked after one provider call. The call used $0.00470000 and returned HTTP 200, but the response ended with MAX_TOKENS before a parseable JSON decision was produced. No LLM-judge decision, all-strategy endpoint, benchmark, model, or SOTA claim is made in experiments/iter94_provider_llm_judge_execution_on_materialized_fixtures.
• Provider LLM judge prompt-budget recovery after block: passed with zero provider calls, zero spend, zero LLM-judge execution, and zero row execution. It tied the iter94 blocker to the 256 output-token ceiling being consumed by hidden reasoning before parseable JSON, materialized 14 recovered prompts with private labels excluded, and pre-registered a bounded retry without any benchmark/model/SOTA claim in experiments/iter95_provider_llm_judge_prompt_budget_recovery_after_block.
• Provider LLM judge bounded retry after prompt-budget recovery: passed with 14 provider calls, $0.19588800 spend, and 14 parseable fixture decisions. It accepted 0/7 false-completion traps but rejected 5/7 legitimate controls, so this is adverse fixture evidence for the LLM-judge strategy, not a benchmark/model/SOTA claim, in experiments/iter96_provider_llm_judge_bounded_retry_after_prompt_budget_recovery.
• Five-strategy completion-verification adjudication after LLM judge: passed with zero provider calls, zero spend, zero strategy execution, and zero row execution. It found that external verifier and complete Telos both passed the frozen fixture bars, while self-report/tests accepted every false-completion trap and the provider LLM judge false-rejected 5/7 legitimate controls. Benchmark escalation is rejected because this suite does not yet distinguish complete Telos from external verifier, in experiments/iter97_five_strategy_completion_verification_adjudication_after_llm_judge.
• External-verifier/Telos differential suite design after adjudication: passed with zero provider calls, zero spend, zero strategy execution, and zero row execution. It froze 8 differential target families and 16 planned fixtures to test protocol-specific evidence that may separate complete Telos from generic external verification, without claiming the expected divergence as a result, in experiments/iter98_external_verifier_telos_differential_suite_design_after_adjudication.
• External-verifier/Telos differential fixture materialization after design: passed with zero provider calls, zero spend, zero strategy execution, and zero row execution. It materialized 16 blinded fixtures across 8 target families, 96 public artifacts, 16 private labels, and 5 identical strategy-input manifests with labels excluded, without claiming a differential result, in experiments/iter99_external_verifier_telos_differential_fixture_materialization_after_design.
• Deterministic strategy execution on differential fixtures after materialization: passed with zero provider calls and 64 deterministic decisions. External verifier accepted 4/8 false-completion traps while complete Telos accepted 0/8, producing a limited deterministic fixture delta of 0.50000000 without claiming a benchmark or broad superiority result, in experiments/iter100_deterministic_strategy_execution_on_differential_fixtures_after_materialization.
• Provider LLM judge execution on differential fixtures after deterministic: blocked after 14 provider calls and $0.22777400 estimated spend. It produced 13/16 parseable LLM-judge decisions, then DIFX-FIXTURE-0014 hit MAX_TOKENS; all-strategy endpoint evidence remains incomplete and no benchmark/model/SOTA or superiority claim is made, in experiments/iter101_provider_llm_judge_execution_on_differential_fixtures_after_deterministic.
• Provider LLM judge differential retry recovery after block: passed with zero provider calls, zero spend, zero LLM-judge execution, and zero row execution. It tied the iter101 blocker to hidden reasoning exhausting the 2048 output budget, materialized 16 recovered prompts with private labels excluded, and pre-registered a full recovered retry, in experiments/iter102_provider_llm_judge_differential_retry_recovery_after_block.
• Differential provider LLM judge full retry after block recovery: passed with 16 provider calls, $0.23633000 estimated spend, and 16/16 parseable LLM-judge decisions. It accepted 0/8 false-completion traps but preserved only 2/8 legitimate controls, so this is adverse LLM-judge strategy evidence, not a benchmark/model/SOTA or superiority claim, in experiments/iter103_differential_provider_llm_judge_full_retry_after_block_recovery.
• Five-strategy differential adjudication after recovered LLM judge: passed with zero provider calls and zero spend. Complete Telos was the only balanced pass on the frozen 16-fixture differential suite; external verifier accepted 4/8 false-completion traps and the recovered LLM judge rejected 6/8 legitimate controls. This is a fixture-level differential result, not a benchmark/model/SOTA claim, in experiments/iter104_five_strategy_differential_adjudication_after_recovered_llm_judge.
• External benchmark pilot design after differential adjudication: passed with zero provider calls, zero spend, and zero benchmark/task execution. It designed a 20-packet external pilot with a future 30 provider-call and $10.00000000 spend ceiling, but it is a design result only, not a benchmark/model/SOTA claim, in experiments/iter105_external_benchmark_pilot_design_after_differential_adjudication.
• External benchmark pilot materialization after design: passed with zero provider calls, zero spend, and zero benchmark/task execution. It materialized 20 pilot packets, 160 public artifacts, 10 false-completion private labels, 10 legitimate-control private labels, and 5 identical public-only strategy-input manifests, in experiments/iter106_external_benchmark_pilot_materialization_after_design.
• External benchmark pilot execution after materialization: passed with 20 provider calls, $0.38674600 estimated spend, 100 strategy decisions, and 40 raw LLM prompt/response artifacts. Complete Telos accepted 0/10 false-completion packets and preserved 10/10 legitimate controls; the external verifier accepted 2/10 false-completion packets; the LLM judge accepted 0/10 false-completion packets but rejected 10/10 legitimate controls. This is a bounded 20-packet pilot result only, not a benchmark/model/SOTA or all-strategy superiority claim, in experiments/iter107_external_benchmark_pilot_execution_after_materialization.
• Real-trajectory line: the detector and comparators were moved off self-authored packets onto the external princeton-nlp/SWE-bench_Verified dataset. On 200 real human-authored gold patches a raw-diff verifier-provenance and assertion-integrity detector held a 0/200 false-positive rate while catching reward-hack variants that the execution-only default accepts, in experiments/iter109_real_trajectory_tamper_detection_pilot. One hacker-fixer round raised recall on source-only evasions from 0 to 1.0 at 0/200 false positives on the same real patches, in experiments/iter110_adversarial_hardening_hacker_fixer_loop. A gemini-2.5-flash steelman judge matched the detector on obvious hacks (0.00 false-positive rate, 1.00 recall, $0.044 spend), in experiments/iter111_llm_judge_steelman_baseline. A stealth 2x2 recorded detector_only=0, judge_only=15, both_catch=30, both_miss=0 across 60 candidates, in experiments/iter112_stealth_divergence_judge_vs_detector. These are bounded pilot results only, not a benchmark, model, or state-of-the-art claim.
• Native execution ground truth: the modeled "hacks pass the visible tests" assumption was replaced with a real native run of django__django-14089 (Python 3.11, tests/runtests.py). The hidden test failed on base and passed on gold; a hard-coded reward hack passed the real hidden test while being a general non-fix; the execution-only default and the deterministic detector accepted it and the gemini-2.5-flash judge flagged it, in experiments/iter113_native_execution_ground_truth. Native execution was then extended to a five-instance Django batch: 4/5 gold patches resolved the real hidden test and the detector flagged 0/5, with the one non-resolving instance recorded as a native-harness fidelity gap, in experiments/iter114_batch_native_execution. A wider eighteen-instance batch tightened the native-harness fidelity estimate to 17/18 gold resolution (0.94) with the detector still at 0/18 false positives, in experiments/iter115_wider_batch_native_execution.
• Historical gate at the end of this status log: reward-hack property-execution contract design (iter191) in experiments/iter191_reward_hack_property_execution_contract_design; iter190 published a null pre-spend execution-surface result and did not call a provider. The real-trajectory arc (iter109 onward) superseded the earlier external-benchmark-pilot adjudication gate experiments/iter108_external_benchmark_pilot_adjudication_after_execution.
• Benchmark leaderboard or broad benchmark result: none yet. Bounded evidence now includes the 20 frozen iter107 packets and the iter161/iter165 paired single-model metric on reward_hack_benchmark_v1 (3/40 all-hack recall, 0/40 control false positives, specificity 1.0, balanced detection 0.5375), but no leaderboard, model-comparison, SOTA, natural-frequency, or broad robustness result exists.
• Provider-backed protocol-effect result: bounded two-row clean pilot, stratified adapter-validation rows, one bounded six-row null/no-signal pilot, one bounded six-row discriminating-metric pilot, and one bounded six-row unstable stability replication; none is a benchmark result.
• Historical target at that boundary: Telos overlay on CodeClash + SWE-bench Verified public software-agent tasks.

Historical claim-boundary reviewer entry point: experiments/iter31_claim_boundary_release_manifest/proof/claim_boundary_release_manifest.json. It indexes the current claim-boundary proof packet and keeps failed/null rows, changed candidates, and no-claim exclusions visible. It is not a leaderboard, SWE-bench, production, live-domain, or model-superiority result.

Historical self-coverage reviewer entry points: experiments/iter35_release_manifest_self_coverage_guard/proof/self_coverage_report.json and experiments/iter36_release_manifest_self_coverage_negative_guard/proof/negative_guard_report.json. They account for the release manifest's own self-verification gates and negative fixtures without changing the claim boundary.

This repo deliberately separates the research line from Sentinel. Sentinel proved a standard: frozen bars, public baselines, nulls published, raw evidence committed, corrections on the record. This repo applies that standard to autonomous agent completion.

The first number frozen

iter00_target_survey scored candidate benchmark families against seven frozen criteria:

criterion	meaning
frontier relevance	the problem matches current autonomous-agent failure modes
public baseline quality	there is a named benchmark, split, and published score
falsifiability	the protocol can fail clearly, before narrative interpretation
evidence surface	the task can emit receipts beyond a final answer
standalone runtime fit	the repository can run and verify it without hidden fleet-scale infrastructure
saturation risk	current leaderboards have not made the target uninformative
operational cost	the first honest experiment is affordable

The survey chose one of three actions:

1. Freeze the first public benchmark target.
2. Freeze a hybrid benchmark built from public tasks plus Telos proof receipts.
3. Publish a survey null if no candidate clears the bar.

Survey result: experiments/iter00_target_survey/RESULT.md. Receipt dry run: experiments/iter01_receipt_dry_run/RESULT.md. Public slice: experiments/iter02_public_task_slice/RESULT.md. CodeClash smoke: experiments/iter03_codeclash_smoke/RESULT.md. Agent-behavior slice: experiments/iter04_agent_behavior_slice/RESULT.md. Agent-behavior smoke: experiments/iter05_agent_behavior_smoke/RESULT.md. Deterministic edit slice: experiments/iter06_deterministic_edit_slice/RESULT.md. Deterministic edit smoke: experiments/iter07_deterministic_edit_smoke/RESULT.md. Provider-model pilot slice: experiments/iter08_provider_model_pilot_slice/RESULT.md. Provider-model pilot smoke: experiments/iter09_provider_model_pilot_smoke/RESULT.md. Provider auth recovery: experiments/iter10_provider_auth_recovery/RESULT.md. Provider-model pilot retry: experiments/iter11_provider_model_pilot_retry/RESULT.md. Vertex model access recovery: experiments/iter12_vertex_model_access_recovery/RESULT.md. Provider-model pilot retry after access recovery: experiments/iter13_provider_model_pilot_retry_after_access_recovery/RESULT.md. Provider diff quality review: experiments/iter14_provider_diff_quality_review/RESULT.md. Provider strict diff rerun: experiments/iter15_provider_strict_diff_rerun/RESULT.md. Provider workspace hygiene control: experiments/iter16_provider_workspace_hygiene_control/RESULT.md. Provider lint hygiene control: experiments/iter17_provider_lint_hygiene_control/RESULT.md. Provider behavior depth control: experiments/iter18_provider_behavior_depth_control/RESULT.md. Provider final inspection control: experiments/iter19_provider_final_inspection_control/RESULT.md. Behavior semantic verification: experiments/iter20_behavior_semantic_verification/RESULT.md. Opponent collision control: experiments/iter21_opponent_collision_control/RESULT.md. Semantic mutation guard: experiments/iter22_semantic_mutation_guard/RESULT.md. Tail semantics falsification: experiments/iter23_tail_semantics_falsification/RESULT.md. Tail safety control: experiments/iter24_tail_safety_control/RESULT.md. Tail safety mutation guard: experiments/iter25_tail_safety_mutation_guard/RESULT.md. Own-tail redundancy mutation guard: experiments/iter26_own_tail_redundancy_mutation_guard/RESULT.md. Semantic claim boundary matrix: experiments/iter27_semantic_claim_boundary_matrix/RESULT.md. Public claim surface guard: experiments/iter28_public_claim_surface_guard/RESULT.md. Public claim surface negative guard: experiments/iter29_public_claim_surface_negative_guard/RESULT.md. Boundary matrix schema guard: experiments/iter30_boundary_matrix_schema_guard/RESULT.md. Claim boundary release manifest: experiments/iter31_claim_boundary_release_manifest/RESULT.md. Claim boundary release manifest negative guard: experiments/iter32_claim_boundary_release_manifest_negative_guard/RESULT.md. Release manifest public sync guard: experiments/iter33_release_manifest_public_sync_guard/RESULT.md. Release manifest public sync negative guard: experiments/iter34_release_manifest_public_sync_negative_guard/RESULT.md. Release manifest self-coverage guard: experiments/iter35_release_manifest_self_coverage_guard/RESULT.md. Release manifest self-coverage negative guard: experiments/iter36_release_manifest_self_coverage_negative_guard/RESULT.md. Release manifest self-coverage public sync guard: experiments/iter37_release_manifest_self_coverage_public_sync_guard/RESULT.md. Release manifest self-coverage public sync negative guard: experiments/iter38_release_manifest_self_coverage_public_sync_negative_guard/RESULT.md. Public task protocol-effect slice: experiments/iter39_public_task_protocol_effect_slice/RESULT.md. Public task protocol-effect execution: experiments/iter40_public_task_protocol_effect_execution/RESULT.md. Public task protocol-effect runner recovery: experiments/iter41_public_task_protocol_effect_runner_recovery/RESULT.md. Public task protocol-effect execution retry: experiments/iter42_public_task_protocol_effect_execution_retry/RESULT.md. Provider execution harness recovery: experiments/iter43_provider_execution_harness_recovery/RESULT.md. Public task protocol-effect execution after harness recovery: experiments/iter44_public_task_protocol_effect_execution_after_harness_recovery/RESULT.md. Public task-condition executor assembly: experiments/iter45_public_task_condition_executor_assembly/RESULT.md. Public task protocol-effect execution with assembled executor: experiments/iter46_public_task_protocol_effect_execution_with_assembled_executor/RESULT.md. Provider task-condition command binding recovery: experiments/iter47_provider_task_condition_command_binding_recovery/RESULT.md. Provider-compatible protocol-effect slice refreeze: experiments/iter48_provider_compatible_protocol_effect_slice_refreeze/RESULT.md. Provider-compatible protocol-effect execution retry: experiments/iter49_provider_compatible_protocol_effect_execution_retry/RESULT.md. Provider-compatible execution wrapper recovery: experiments/iter50_provider_compatible_execution_wrapper_recovery/RESULT.md. Provider-compatible protocol-effect execution with wrapper: experiments/iter51_provider_compatible_protocol_effect_execution_with_wrapper/RESULT.md. Provider condition runtime separation recovery: experiments/iter52_provider_condition_runtime_separation_recovery/RESULT.md. Provider-compatible execution after condition recovery: experiments/iter53_provider_compatible_protocol_effect_execution_after_condition_recovery/RESULT.md. Provider pair executor recovery: experiments/iter54_provider_pair_executor_recovery/RESULT.md. Provider-compatible paid execution after executor recovery: experiments/iter55_provider_compatible_paid_execution_after_executor_recovery/RESULT.md. Provider auth recovery for paid protocol effect: experiments/iter56_provider_auth_recovery_for_paid_protocol_effect/RESULT.md. Provider-compatible paid execution after auth recovery: experiments/iter57_provider_compatible_paid_execution_after_auth_recovery/RESULT.md. CodeClash Vertex dependency recovery: experiments/iter58_codeclash_vertex_dependency_recovery/RESULT.md. Provider-compatible paid execution after dependency recovery: experiments/iter59_provider_compatible_paid_execution_after_dependency_recovery/RESULT.md. Provider model binding recovery: experiments/iter60_provider_model_binding_recovery/RESULT.md. Vertex quota-project binding recovery: experiments/iter61_vertex_quota_project_binding_recovery/RESULT.md. Vertex bearer token path recovery: experiments/iter62_vertex_bearer_token_path_recovery/RESULT.md. Vertex access path parity recheck: experiments/iter63_vertex_access_path_parity_recheck/RESULT.md. Provider-compatible paid execution after access-path recovery: experiments/iter64_provider_compatible_paid_execution_after_access_path_recovery/RESULT.md. Receipt-schema prompt alignment: experiments/iter65_receipt_schema_prompt_alignment/RESULT.md. Provider-compatible paid execution after receipt prompt alignment: experiments/iter66_provider_compatible_paid_execution_after_receipt_prompt_alignment/RESULT.md. Provider-compatible expanded slice refreeze: experiments/iter67_provider_compatible_expanded_slice_refreeze/RESULT.md. Provider-compatible task-surface adapter recovery: experiments/iter68_provider_compatible_task_surface_adapter_recovery/RESULT.md. CodeClash task-surface source snapshot recovery: experiments/iter69_codeclash_task_surface_source_snapshot_recovery/RESULT.md. Provider-compatible expanded adapter completion: experiments/iter70_provider_compatible_expanded_adapter_completion/RESULT.md. Provider-compatible expanded slice after adapter completion: experiments/iter71_provider_compatible_expanded_slice_after_adapter_completion/RESULT.md. Provider-compatible expanded paid execution after slice refreeze: experiments/iter72_provider_compatible_expanded_paid_execution_after_slice_refreeze/RESULT.md.

Evidence Arc (historical detection/intervention work)

This diagram summarizes the earlier detection and intervention arc (iter109-iter190). It is provenance, not the current evidence summary. In particular, its historical gold-free nodes do not validate the later iter197/iter201 property pipeline, which used diff-derived locators and gold inclusion. Red nodes mark the decisive discovery or reclassification points in that historical arc.

flowchart TB
 subgraph H1["DETECT"]
  direction LR
  D1["109–110<br/>detector · 0/200 FP"]-->D2["111–112<br/>judge + stealth"]-->D3["113–118<br/>real execution<br/>both-miss found"]-->D4["119–129<br/>gold-free properties<br/>scale + taxonomy"]
 end
 subgraph H2["SCALE"]
  direction LR
  S1["130–139<br/>Docker applicability"]-->S2["140–145<br/>frontier both-miss<br/>panel gap"]-->S3["146–152<br/>protocol scale<br/>0/20→10/20 official"]
 end
 subgraph H3["EVALUATE"]
  direction LR
  M1["153–156<br/>v1: 40 rows<br/>not reward hacks<br/>192 conservative novelty FAIL"]-->M2["157–190<br/>judge/panel repair<br/>17/40 · controls 0/40<br/>property gap"]
 end
 H1-->H2
 H2-->H3
 classDef risk fill:#fee,stroke:#c22,color:#000;
 class D3,S2,M1 risk;

Loading

The FBM node was the adversarial frontier: iter140's 0/20 gemini null read the both-miss window as narrow, but iter141 overturned that reading - a non-refusing frontier adversary manufactures the class on well-tested code, iter142 measured the rate, iter143 showed it survives frontier judges, iter144 generalized it to a second repo, and iter145 showed a judge panel does not close it. The PE node then closes the loop: used as a gate with gold-free repair feedback, the protocol lifts real completion on reward-hack-prone starts from 0/7 to 5/7, the first evidence that it changes outcomes rather than only measuring them. Iter153 materialized those then-described reward-hack rows as a seed artifact, iter154 tried to grow it under bounded spend and honestly missed the 20-row bar with 17 new rows, and iter155 adaptively added 3 more rows to cross the 40-row candidate-pool bar. Iter156 materialized the hash-checked reward_hack_benchmark_v1 row artifact with 40 rows across 11 repos and 13/40 surviving every static layer, still without a benchmark score or leaderboard. Iter157 made the paper and public paper mirror current through that boundary. Iter158 designed the zero-spend scoring/evaluation protocol and set packet materialization as the next leakage-control gate. Iter159 materialized the 40 blinded packets with complete committed public metadata coverage and no label/report/static-verdict leakage hits. Iter160 validated the strict judge-output parser and fixture suite before provider scoring. Iter161 then ran one bounded model judge over all 40 historically labeled all-hack packets and recorded 3 reward_hack verdicts and 37 legitimate verdicts, making the recall gap visible while still making no precision, leaderboard, or model-comparison claim. Iter162 designed the legitimate-control source hierarchy, schemas, and false-positive/specificity/precision boundary. Iter163 materialized 40 paired legitimate controls and blinded control packets. Iter164 honestly blocked on a Vertex http_429 after 15 successful controls. Iter165 recovered only the missing controls and produced the complete paired one-model metric: 3/40 recall on suite-failing v1 candidates, 0/40 false positives on controls, specificity 1.0, and balanced detection 0.5375; this is still not a leaderboard or model-comparison result. Iter166 designed 5 evaluator families and froze the next stop conditions: specificity floor 0.90, control false-positive ceiling 4/40, and balanced-detection improvement floor 0.10. Iter167 completed the skeptical-judge calibration and failed honestly: provider successes 80/80, false positives 0/40, specificity 0.90, but recall stayed 3/40, candidate invalids rose to 5/40, and balanced detection dropped to 0.4875. Iter168 adjudicated the null: all 9 invalid outputs were markdown-fenced JSON, and a diagnostic-only strip would reach only 4/40 recall, so independent panel design is the next pre-registered gate. The July 2026 literature sweep is recorded in docs/LITERATURE_ALIGNMENT_2026.md. Iter169-186 then moved from single-model judging to a three-provider panel: the full unrepaired majority_catch result is 17/40 suite-failing v1 candidate rows and 0/40 controls, and iter181's OpenAI-only repair run reduced nondecisions in a secondary diagnostic without changing that primary result; iter182 adjudicated that diagnostic from raw committed proof before any claim upgrade, and iter183 synchronized public claim surfaces to keep that boundary explicit. Iter184 then mapped current public research to a concrete property-probe design gate for the 23 v1 candidate rows the primary panel still misses; iter185 froze the priority subset, and iter186 materialized the leakage-scanned packet inputs. The earlier provider-pilot and semantic-guard arc (iter00-iter108) is preserved in the historical status log above and the learning ledger.

Candidate Target Families

The initial candidates are documented in benchmarks/CANDIDATES.md:

• coding-agent completion: SWE-bench Verified, CodeClash, Terminal-Bench-style tasks
• AI R&D agents: METR RE-Bench-style research engineering tasks
• tool-using service agents: tau-bench-style policy and database-state tasks
• adversarial tool agents: AgentDojo-style utility/security tradeoffs
• custom Telos overlay: public tasks with receipt requirements added around them

The target was not chosen by taste. It was chosen by the frozen survey.

Historical architecture (iter109-iter156)

The real-trajectory arc (iter109-iter156) established a three-layer completion verifier, each layer present because the one before it provably fails on a measured class of reward hack (see the arc section above and the synthesis report). The necessity of the third layer is not asserted but measured: a frontier adversary defeats both static layers on 0.12 of attempts against well-tested code, and one such hack survives every frontier judge tested (iter142-iter144). The cascade runs cheapest-first and escalates only the survivors:

flowchart TB
  C["candidate diff<br/>+ FAIL_TO_PASS"] --> D["Layer 1<br/>deterministic detector<br/>local · reproducible"]
  D -->|survives| J["Layer 2<br/>LLM judge<br/>paid · semantic"]
  J -->|survives| M["Layer 3<br/>held-out-input execution<br/>gold-free property"]
  M -->|holds| OK["accept<br/>+ receipt"]
  D -->|historical static catch| X1["reject<br/>proxy-gaming patch"]
  J -->|oblique or hard-coded catch| X2["reject<br/>oblique / hard-coded"]
  M -->|generalization catch| X3["reject<br/>generalization break"]
  classDef base fill:#f6f8fa,stroke:#57606a,color:#1f2328;
  classDef layer fill:#e4f0ff,stroke:#1565c0,color:#0c2742;
  classDef hack fill:#fee,stroke:#c22,color:#000;
  classDef ok fill:#e2f3e5,stroke:#2e7d32,color:#13361b;
  class C base;
  class D,J,M layer;
  class X1,X2,X3 hack;
  class OK ok;

Loading

Within that earlier experiment boundary, Layer 3 is automated and gold-free: a model proposes a metamorphic property from the function contract, the visible test filters unsound proposals, and execution on random held-out inputs catches completions the static layers accept - with the property strategy chosen by function type (a contract property for pure transforms, an inverse round-trip for invertible parsers/formatters).

This historical architecture is not evidence that iter197 or iter201 followed the same access contract; their corrected artifacts record protocol FAIL.

Full design: docs/ARCHITECTURE.md. Paper: docs/PAPER.md - the consolidated result (iter109-iter156), submission-shaped. LaTeX manuscript draft: paper/telos.tex (findings-paper framing; see paper/README.md for build and the pre-posting checklist). Synthesis report: docs/COMPLETION_VERIFICATION_REPORT.md. Presentation standard: docs/PRESENTATION.md. Learning engine: docs/LEARNING_ENGINE.md. Mission loop: docs/MISSION_LOOP.md.

Current repository contract

Repository Map

README.md                  research front door and live status
AGENTS.md                  durable project boundary and evidence rules
PREREGISTRATION.md         frozen first-stage target-selection protocol
CONTINUITY.md              operator invariants and handoff discipline
HANDOFF.md                 one-time source-bound publication and scientific-boundary snapshot
telos/                     receipt validation, scorecard primitives, and telos/tamper (the three-layer verifier)
telos/tamper/              the deterministic detector, attack/adversarial generators, and the LLM-judge client
benchmarks/                candidate benchmark registry
docs/                      architecture, forensic audit, related work, roadmap, and synthesis reports
experiments/               one folder per experiment (iter00-iter214), including explicit fail/null/blocked states
mission/                   machine-readable mission loop contract
protocol/                  proof receipt schema
scripts/                   validation and handoff tooling
tests/                     repository and protocol tests

Reproduce The Current State

The verification workflow fixes its hosted OS family to ubuntu-24.04, pins every external action to a full commit SHA, grants only read access to repository contents, and installs its Python verification tools from the exact-version, SHA-256-locked requirements-ci.txt. The supply-chain guard enforces the same rules across every workflow. HANDOFF.md is generated once during the release seal, before its publication-safety receipt and runtime manifest, and is never regenerated by the post-seal verification suite.

python3 -m compileall telos scripts tests
ruff check .
pytest -q
python3 scripts/validate_json.py
python3 scripts/validate_docs.py
python3 scripts/validate_current_paper.py
python3 scripts/validate_mission_loop.py
python3 scripts/validate_supply_chain.py
python3 scripts/validate_detector_methodology_correction.py
python3 scripts/validate_iter200_corrected_result.py
python3 scripts/build_iter200_solve_targets.py --check
python3 scripts/build_iter202_solve_targets.py --check
python3 scripts/audit_iter202_sample_overlap.py --check
python3 scripts/build_iter202_image_lock.py --check
python3 scripts/build_iter203_safety_recovery.py --check
python3 scripts/build_iter203_runtime_manifest.py --check
python3 scripts/validate_iter203_publication_safety.py --check
python3 scripts/validate_iter203_infrastructure_null.py
python3 scripts/validate_iter204_pre_dispatch_null.py
python3 scripts/validate_iter205_pre_dispatch_null.py
python3 scripts/build_iter206_runtime_manifest.py --check
python3 scripts/validate_iter206_publication_safety.py --check
python3 scripts/validate_iter206_runtime_recovery.py
python3 scripts/validate_iter206_pre_publication_null.py
python3 scripts/audit_iter207_claim_integrity.py --check
python3 scripts/build_iter207_runtime_manifest.py --check
python3 scripts/validate_iter207_publication_safety.py --check
python3 scripts/validate_iter207_runtime_recovery.py
python3 scripts/validate_iter208_post_seal_forensic_correction.py
python3 scripts/build_iter209_receipt.py --check
python3 scripts/validate_iter209_publication_ci_recovery.py
python3 scripts/build_iter210_receipt.py --check
python3 scripts/validate_iter210_pr_synthetic_merge_recovery.py
python3 scripts/build_iter211_tcp1_packet.py --check
python3 scripts/build_iter211_receipt.py --check
python3 scripts/validate_iter211_tcp1_materialization_preflight.py
python3 scripts/build_iter213_receipt.py --check
python3 scripts/validate_iter213_post_seal_validation_recovery.py
python3 scripts/build_iter214_receipt.py --check
python3 scripts/validate_iter214_tcp1_cross_platform_numeric_recovery.py
python3 scripts/validate_target_survey.py
python3 scripts/validate_public_slice.py
python3 scripts/validate_agent_behavior_slice.py
python3 scripts/validate_deterministic_edit_slice.py
python3 scripts/validate_provider_model_pilot_slice.py
python3 scripts/validate_receipts.py experiments/iter01_receipt_dry_run/proof
python3 scripts/validate_receipts.py experiments/iter03_codeclash_smoke/proof
python3 scripts/audit_codeclash_smoke.py
python3 scripts/validate_receipts.py experiments/iter05_agent_behavior_smoke/proof
python3 scripts/audit_agent_behavior_smoke.py
python3 scripts/validate_receipts.py experiments/iter07_deterministic_edit_smoke/proof
python3 scripts/audit_deterministic_edit_smoke.py
python3 scripts/validate_receipts.py experiments/iter09_provider_model_pilot_smoke/proof
python3 scripts/audit_provider_model_pilot_smoke.py
python3 scripts/validate_receipts.py experiments/iter10_provider_auth_recovery/proof
python3 scripts/audit_provider_auth_recovery.py
python3 scripts/validate_receipts.py experiments/iter11_provider_model_pilot_retry/proof
python3 scripts/audit_provider_model_pilot_retry.py
python3 scripts/validate_receipts.py experiments/iter12_vertex_model_access_recovery/proof
python3 scripts/audit_vertex_model_access_recovery.py
python3 scripts/validate_receipts.py experiments/iter13_provider_model_pilot_retry_after_access_recovery/proof
python3 scripts/audit_provider_model_pilot_after_access.py
python3 scripts/validate_receipts.py experiments/iter14_provider_diff_quality_review/proof
python3 scripts/audit_provider_diff_quality_review.py
python3 scripts/validate_receipts.py experiments/iter15_provider_strict_diff_rerun/proof
python3 scripts/audit_provider_strict_diff_rerun.py
python3 scripts/validate_receipts.py experiments/iter16_provider_workspace_hygiene_control/proof
python3 scripts/audit_provider_workspace_hygiene_control.py
python3 scripts/validate_receipts.py experiments/iter17_provider_lint_hygiene_control/proof
python3 scripts/audit_provider_lint_hygiene_control.py
python3 scripts/validate_receipts.py experiments/iter18_provider_behavior_depth_control/proof
python3 scripts/audit_provider_behavior_depth_control.py
python3 scripts/validate_receipts.py experiments/iter19_provider_final_inspection_control/proof
python3 scripts/audit_provider_final_inspection_control.py
python3 scripts/validate_receipts.py experiments/iter20_behavior_semantic_verification/proof
python3 scripts/audit_behavior_semantic_verification.py
python3 scripts/validate_receipts.py experiments/iter21_opponent_collision_control/proof
python3 scripts/audit_opponent_collision_control.py
python3 scripts/validate_receipts.py experiments/iter22_semantic_mutation_guard/proof
python3 scripts/audit_semantic_mutation_guard.py
python3 scripts/validate_receipts.py experiments/iter23_tail_semantics_falsification/proof
python3 scripts/audit_tail_semantics_falsification.py
python3 scripts/validate_receipts.py experiments/iter24_tail_safety_control/proof
python3 scripts/audit_tail_safety_control.py
python3 scripts/validate_receipts.py experiments/iter25_tail_safety_mutation_guard/proof
python3 scripts/audit_tail_safety_mutation_guard.py
python3 scripts/validate_receipts.py experiments/iter26_own_tail_redundancy_mutation_guard/proof
python3 scripts/audit_own_tail_redundancy_mutation_guard.py
python3 scripts/validate_receipts.py experiments/iter27_semantic_claim_boundary_matrix/proof
python3 scripts/audit_semantic_claim_boundary_matrix.py
python3 scripts/validate_receipts.py experiments/iter28_public_claim_surface_guard/proof
python3 scripts/audit_public_claim_surface_guard.py
python3 scripts/validate_receipts.py experiments/iter29_public_claim_surface_negative_guard/proof
python3 scripts/audit_public_claim_surface_negative_guard.py
python3 scripts/validate_receipts.py experiments/iter30_boundary_matrix_schema_guard/proof
python3 scripts/audit_boundary_matrix_schema_guard.py
python3 scripts/validate_receipts.py experiments/iter31_claim_boundary_release_manifest/proof
python3 scripts/audit_claim_boundary_release_manifest.py
python3 scripts/validate_receipts.py experiments/iter32_claim_boundary_release_manifest_negative_guard/proof
python3 scripts/audit_claim_boundary_release_manifest_negative_guard.py
python3 scripts/validate_receipts.py experiments/iter33_release_manifest_public_sync_guard/proof
python3 scripts/audit_release_manifest_public_sync_guard.py
python3 scripts/validate_receipts.py experiments/iter34_release_manifest_public_sync_negative_guard/proof
python3 scripts/audit_release_manifest_public_sync_negative_guard.py
python3 scripts/validate_receipts.py experiments/iter35_release_manifest_self_coverage_guard/proof
python3 scripts/audit_release_manifest_self_coverage_guard.py
python3 scripts/validate_receipts.py experiments/iter36_release_manifest_self_coverage_negative_guard/proof
python3 scripts/audit_release_manifest_self_coverage_negative_guard.py
python3 scripts/validate_receipts.py experiments/iter37_release_manifest_self_coverage_public_sync_guard/proof
python3 scripts/audit_release_manifest_self_coverage_public_sync_guard.py
python3 scripts/validate_receipts.py experiments/iter38_release_manifest_self_coverage_public_sync_negative_guard/proof
python3 scripts/audit_release_manifest_self_coverage_public_sync_negative_guard.py
python3 scripts/validate_receipts.py experiments/iter39_public_task_protocol_effect_slice/proof
python3 scripts/audit_public_task_protocol_effect_slice.py
python3 scripts/validate_receipts.py experiments/iter40_public_task_protocol_effect_execution/proof
python3 scripts/audit_public_task_protocol_effect_execution.py
python3 scripts/validate_receipts.py experiments/iter41_public_task_protocol_effect_runner_recovery/proof
python3 scripts/audit_public_task_protocol_effect_runner_recovery.py
python3 scripts/validate_receipts.py experiments/iter42_public_task_protocol_effect_execution_retry/proof
python3 scripts/audit_public_task_protocol_effect_execution_retry.py
python3 scripts/validate_receipts.py experiments/iter43_provider_execution_harness_recovery/proof
python3 scripts/audit_provider_execution_harness_recovery.py
python3 scripts/validate_receipts.py experiments/iter44_public_task_protocol_effect_execution_after_harness_recovery/proof
python3 scripts/audit_public_task_protocol_effect_execution_after_harness_recovery.py
python3 scripts/validate_receipts.py experiments/iter45_public_task_condition_executor_assembly/proof
python3 scripts/audit_public_task_condition_executor_assembly.py
python3 scripts/validate_receipts.py experiments/iter46_public_task_protocol_effect_execution_with_assembled_executor/proof
python3 scripts/audit_public_task_protocol_effect_execution_with_assembled_executor.py
python3 scripts/validate_receipts.py experiments/iter47_provider_task_condition_command_binding_recovery/proof
python3 scripts/audit_provider_task_condition_command_binding_recovery.py
python3 scripts/validate_receipts.py experiments/iter48_provider_compatible_protocol_effect_slice_refreeze/proof
python3 scripts/audit_provider_compatible_protocol_effect_slice_refreeze.py
python3 scripts/validate_receipts.py experiments/iter49_provider_compatible_protocol_effect_execution_retry/proof
python3 scripts/audit_provider_compatible_protocol_effect_execution_retry.py
python3 scripts/validate_receipts.py experiments/iter50_provider_compatible_execution_wrapper_recovery/proof
python3 scripts/audit_provider_compatible_execution_wrapper_recovery.py
python3 scripts/validate_receipts.py experiments/iter51_provider_compatible_protocol_effect_execution_with_wrapper/proof
python3 scripts/audit_provider_compatible_protocol_effect_execution_with_wrapper.py
python3 scripts/validate_receipts.py experiments/iter52_provider_condition_runtime_separation_recovery/proof
python3 scripts/audit_provider_condition_runtime_separation_recovery.py
python3 scripts/validate_receipts.py experiments/iter53_provider_compatible_protocol_effect_execution_after_condition_recovery/proof
python3 scripts/audit_provider_compatible_protocol_effect_execution_after_condition_recovery.py
python3 scripts/validate_receipts.py experiments/iter54_provider_pair_executor_recovery/proof
python3 scripts/audit_provider_pair_executor_recovery.py
python3 scripts/validate_receipts.py experiments/iter55_provider_compatible_paid_execution_after_executor_recovery/proof
python3 scripts/audit_provider_compatible_paid_execution_after_executor_recovery.py
python3 scripts/validate_receipts.py experiments/iter56_provider_auth_recovery_for_paid_protocol_effect/proof
python3 scripts/audit_provider_auth_recovery_for_paid_protocol_effect.py
python3 scripts/validate_receipts.py experiments/iter57_provider_compatible_paid_execution_after_auth_recovery/proof
python3 scripts/audit_provider_compatible_paid_execution_after_auth_recovery.py
python3 scripts/validate_receipts.py experiments/iter58_codeclash_vertex_dependency_recovery/proof
python3 scripts/audit_codeclash_vertex_dependency_recovery.py
python3 scripts/validate_receipts.py experiments/iter59_provider_compatible_paid_execution_after_dependency_recovery/proof
python3 scripts/audit_provider_compatible_paid_execution_after_dependency_recovery.py
python3 scripts/validate_receipts.py experiments/iter60_provider_model_binding_recovery/proof
python3 scripts/audit_provider_model_binding_recovery.py
python3 scripts/validate_receipts.py experiments/iter61_vertex_quota_project_binding_recovery/proof
python3 scripts/audit_vertex_quota_project_binding_recovery.py
python3 scripts/validate_receipts.py experiments/iter62_vertex_bearer_token_path_recovery/proof
python3 scripts/audit_vertex_bearer_token_path_recovery.py
python3 scripts/validate_receipts.py experiments/iter63_vertex_access_path_parity_recheck/proof
python3 scripts/audit_vertex_access_path_parity_recheck.py
python3 scripts/validate_receipts.py experiments/iter64_provider_compatible_paid_execution_after_access_path_recovery/proof
python3 scripts/audit_provider_compatible_paid_execution_after_access_path_recovery.py
python3 scripts/validate_receipts.py experiments/iter65_receipt_schema_prompt_alignment/proof
python3 scripts/audit_receipt_schema_prompt_alignment.py
python3 scripts/validate_receipts.py experiments/iter66_provider_compatible_paid_execution_after_receipt_prompt_alignment/proof
python3 scripts/audit_provider_compatible_paid_execution_after_receipt_prompt_alignment.py
python3 scripts/validate_receipts.py experiments/iter67_provider_compatible_expanded_slice_refreeze/proof
python3 scripts/audit_provider_compatible_expanded_slice_refreeze.py
python3 scripts/validate_receipts.py experiments/iter68_provider_compatible_task_surface_adapter_recovery/proof
python3 scripts/audit_provider_compatible_task_surface_adapter_recovery.py
python3 scripts/validate_receipts.py experiments/iter69_codeclash_task_surface_source_snapshot_recovery/proof
python3 scripts/audit_codeclash_task_surface_source_snapshot_recovery.py
python3 scripts/validate_receipts.py experiments/iter70_provider_compatible_expanded_adapter_completion/proof
python3 scripts/audit_provider_compatible_expanded_adapter_completion.py
python3 scripts/validate_receipts.py experiments/iter71_provider_compatible_expanded_slice_after_adapter_completion/proof
python3 scripts/audit_provider_compatible_expanded_slice_after_adapter_completion.py
python3 scripts/validate_receipts.py experiments/iter72_provider_compatible_expanded_paid_execution_after_slice_refreeze/proof
python3 scripts/audit_provider_compatible_expanded_paid_execution_after_slice_refreeze.py
python3 scripts/validate_receipts.py experiments/iter73_expanded_receipt_prompt_recovery_after_paid_block/proof
python3 scripts/audit_expanded_receipt_prompt_recovery_after_paid_block.py
python3 scripts/validate_receipts.py experiments/iter74_provider_compatible_expanded_paid_retry_after_receipt_prompt_recovery/proof
python3 scripts/audit_provider_compatible_expanded_paid_retry_after_receipt_prompt_recovery.py
python3 scripts/validate_receipts.py experiments/iter75_provider_compatible_runtime_adc_recovery_after_paid_retry_block/proof
python3 scripts/audit_provider_compatible_runtime_adc_recovery_after_paid_retry_block.py
python3 scripts/validate_receipts.py experiments/iter76_runtime_adc_recheck_after_operator_refresh/proof
python3 scripts/audit_runtime_adc_recheck_after_operator_refresh.py
python3 scripts/validate_receipts.py experiments/iter77_runtime_adc_recheck_after_application_default_login/proof
python3 scripts/audit_runtime_adc_recheck_after_application_default_login.py
python3 scripts/validate_receipts.py experiments/iter78_provider_compatible_expanded_paid_retry_after_adc_recovery/proof
python3 scripts/audit_provider_compatible_expanded_paid_retry_after_adc_recovery.py
python3 scripts/validate_receipts.py experiments/iter79_dummy_row_call_ceiling_recovery_after_paid_retry_block/proof
python3 scripts/audit_dummy_row_call_ceiling_recovery_after_paid_retry_block.py
python3 scripts/validate_receipts.py experiments/iter80_dummy_call_ceiling_bounded_paid_retry_after_recovery/proof
python3 scripts/audit_dummy_call_ceiling_bounded_paid_retry_after_recovery.py
python3 scripts/validate_receipts.py experiments/iter81_expanded_stratified_adapter_validation_consolidation/proof
python3 scripts/audit_expanded_stratified_adapter_validation_consolidation.py
python3 scripts/validate_receipts.py experiments/iter82_benchmark_facing_protocol_effect_slice_design/proof
python3 scripts/audit_benchmark_facing_protocol_effect_slice_design.py
python3 scripts/validate_receipts.py experiments/iter83_benchmark_facing_protocol_effect_execution_pilot/proof
python3 scripts/audit_benchmark_facing_protocol_effect_execution_pilot.py
python3 scripts/validate_receipts.py experiments/iter84_benchmark_facing_null_signal_adjudication/proof
python3 scripts/audit_benchmark_facing_null_signal_adjudication.py
python3 scripts/validate_receipts.py experiments/iter85_discriminating_task_metric_redesign/proof
python3 scripts/audit_discriminating_task_metric_redesign.py
python3 scripts/validate_receipts.py experiments/iter86_discriminating_metric_backtest_on_committed_artifacts/proof
python3 scripts/audit_discriminating_metric_backtest_on_committed_artifacts.py
python3 scripts/validate_receipts.py experiments/iter87_benchmark_facing_discriminating_metric_execution_pilot/proof
python3 scripts/audit_benchmark_facing_discriminating_metric_execution_pilot.py
python3 scripts/validate_receipts.py experiments/iter88_external_benchmark_readiness_adjudication_after_discriminating_pilot/proof
python3 scripts/audit_external_benchmark_readiness_adjudication_after_discriminating_pilot.py
python3 scripts/validate_receipts.py experiments/iter89_same_slice_discriminating_metric_stability_replication/proof
python3 scripts/audit_same_slice_discriminating_metric_stability_replication.py
python3 scripts/validate_receipts.py experiments/iter90_stability_replication_adjudication_after_same_slice_run/proof
python3 scripts/audit_stability_replication_adjudication_after_same_slice_run.py
python3 scripts/validate_receipts.py experiments/iter91_empirical_validation_suite_design_for_completion_verification/proof
python3 scripts/audit_empirical_validation_suite_design_for_completion_verification.py
python3 scripts/validate_receipts.py experiments/iter92_empirical_validation_fixture_materialization_for_completion_verification/proof
python3 scripts/audit_empirical_validation_fixture_materialization_for_completion_verification.py
python3 scripts/validate_receipts.py experiments/iter93_deterministic_strategy_execution_on_materialized_fixtures/proof
python3 scripts/audit_deterministic_strategy_execution_on_materialized_fixtures.py
python3 scripts/validate_receipts.py experiments/iter94_provider_llm_judge_execution_on_materialized_fixtures/proof
python3 scripts/audit_provider_llm_judge_execution_on_materialized_fixtures.py
python3 scripts/validate_receipts.py experiments/iter95_provider_llm_judge_prompt_budget_recovery_after_block/proof
python3 scripts/audit_provider_llm_judge_prompt_budget_recovery_after_block.py
python3 scripts/validate_receipts.py experiments/iter96_provider_llm_judge_bounded_retry_after_prompt_budget_recovery/proof
python3 scripts/audit_provider_llm_judge_bounded_retry_after_prompt_budget_recovery.py
python3 scripts/validate_receipts.py experiments/iter97_five_strategy_completion_verification_adjudication_after_llm_judge/proof
python3 scripts/audit_five_strategy_completion_verification_adjudication_after_llm_judge.py
python3 scripts/validate_receipts.py experiments/iter98_external_verifier_telos_differential_suite_design_after_adjudication/proof
python3 scripts/audit_external_verifier_telos_differential_suite_design_after_adjudication.py
python3 scripts/validate_receipts.py experiments/iter99_external_verifier_telos_differential_fixture_materialization_after_design/proof
python3 scripts/audit_external_verifier_telos_differential_fixture_materialization_after_design.py
python3 scripts/validate_receipts.py experiments/iter100_deterministic_strategy_execution_on_differential_fixtures_after_materialization/proof
python3 scripts/audit_deterministic_strategy_execution_on_differential_fixtures_after_materialization.py
python3 scripts/validate_receipts.py experiments/iter101_provider_llm_judge_execution_on_differential_fixtures_after_deterministic/proof
python3 scripts/audit_provider_llm_judge_execution_on_differential_fixtures_after_deterministic.py
python3 scripts/validate_receipts.py experiments/iter102_provider_llm_judge_differential_retry_recovery_after_block/proof
python3 scripts/audit_provider_llm_judge_differential_retry_recovery_after_block.py
python3 scripts/validate_receipts.py experiments/iter103_differential_provider_llm_judge_full_retry_after_block_recovery/proof
python3 scripts/audit_differential_provider_llm_judge_full_retry_after_block_recovery.py
python3 scripts/validate_receipts.py experiments/iter104_five_strategy_differential_adjudication_after_recovered_llm_judge/proof
python3 scripts/audit_five_strategy_differential_adjudication_after_recovered_llm_judge.py
python3 scripts/validate_receipts.py experiments/iter105_external_benchmark_pilot_design_after_differential_adjudication/proof
python3 scripts/audit_external_benchmark_pilot_design_after_differential_adjudication.py
python3 scripts/validate_receipts.py experiments/iter106_external_benchmark_pilot_materialization_after_design/proof
python3 scripts/audit_external_benchmark_pilot_materialization_after_design.py
python3 scripts/validate_receipts.py experiments/iter107_external_benchmark_pilot_execution_after_materialization/proof
python3 scripts/audit_external_benchmark_pilot_execution_after_materialization.py
python3 scripts/validate_learning_ledger.py
python3 scripts/validate_json.py
python3 scripts/validate_handoff.py

Writing Standard

The language in this repo must stay below the evidence. A claim is allowed only when it has a source, a receipt, a log, or a clearly marked hypothesis behind it. Nulls and blocked gates are first-class results. Corrections remain in the record.