From 770b0d01f97d7c5d6c5f5e997aac109ce69b681e Mon Sep 17 00:00:00 2001 From: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com> Date: Thu, 3 Sep 2026 07:23:25 -0700 Subject: [PATCH 1/6] docs(lanes): settle this repo's lane-artifact root at docs/lanes/ (aof) Every lane working in this repo was graded against a rule its own goal contradicts. The CODE goal template names `docs/lanes//`; the batch's artifact checker (`check_lane_artifact_paths.py`, artifact-path/v1) resolved this repo to `ai_working//` under rule R2. Both have precedent at origin/main, so a lane that followed its authoritative spec was reported a VIOLATION -- and 2nz paid for it twice, writing its note to `ai_working/` on the checker's say-so and then rewriting it under `docs/lanes/` (c80e04e). Resolved as option (a): the goal template wins, and the checker was taught this repo's convention instead. R2 only ever fired here because a LANE ARTIFACT made `ai_working/` tracked -- the first tracked path there is `ai_working/3yc-.../DONE-NOTE.md` (0d93352), and at 35ab604 all five tracked paths under it are lane artifacts. The rule was reading its own output back as a repo convention. Meanwhile four consecutive merged lanes (n1i #292, 9w0 #297, eem #298, 2nz #299) placed their notes under `docs/lanes/`. This commit is the app-cli half: * docs/lanes/README.md -- the convention, stated for a human reader, including why the `ai_working/` notes stay exactly where they are * docs/lanes/aof-.../ -- this lane's record: the decision argued from the code, fail-before/pass-after evidence, and the evals-repo patch as a reviewable artifact The checker half is a separate commit in the evaluation repo (that repo has no remote), carried here verbatim as docs/lanes/aof-artifact-path-conflict/PROPOSED-evals-artifact-path-R-PIN.patch. No app-cli source changed: 1682 passed, 1 skipped, 13 deselected, 1 xfailed -- byte-identical to the 35ab604 baseline. --- docs/lanes/README.md | 39 ++ .../aof-artifact-path-conflict/DONE-NOTE.md | 181 ++++++++ .../PROPOSED-evals-artifact-path-R-PIN.patch | 439 ++++++++++++++++++ .../evidence/00-conflict-today.txt | 19 + .../evidence/01-fail-before.txt | 4 + .../evidence/02-pass-after.txt | 4 + .../evidence/03-batch-report-diff.txt | 29 ++ .../evidence/03a-batch-before.md | 96 ++++ .../evidence/03b-batch-after.md | 92 ++++ .../04-goal-generator-before-after.txt | 25 + .../evidence/05-evals-suite.txt | 22 + .../evidence/06-appcli-suite.txt | 7 + .../evidence/07-this-lane-graded.txt | 34 ++ .../probe_artifact_root.py | 99 ++++ 14 files changed, 1090 insertions(+) create mode 100644 docs/lanes/README.md create mode 100644 docs/lanes/aof-artifact-path-conflict/DONE-NOTE.md create mode 100644 docs/lanes/aof-artifact-path-conflict/PROPOSED-evals-artifact-path-R-PIN.patch create mode 100644 docs/lanes/aof-artifact-path-conflict/evidence/00-conflict-today.txt create mode 100644 docs/lanes/aof-artifact-path-conflict/evidence/01-fail-before.txt create mode 100644 docs/lanes/aof-artifact-path-conflict/evidence/02-pass-after.txt create mode 100644 docs/lanes/aof-artifact-path-conflict/evidence/03-batch-report-diff.txt create mode 100644 docs/lanes/aof-artifact-path-conflict/evidence/03a-batch-before.md create mode 100644 docs/lanes/aof-artifact-path-conflict/evidence/03b-batch-after.md create mode 100644 docs/lanes/aof-artifact-path-conflict/evidence/04-goal-generator-before-after.txt create mode 100644 docs/lanes/aof-artifact-path-conflict/evidence/05-evals-suite.txt create mode 100644 docs/lanes/aof-artifact-path-conflict/evidence/06-appcli-suite.txt create mode 100644 docs/lanes/aof-artifact-path-conflict/evidence/07-this-lane-graded.txt create mode 100644 docs/lanes/aof-artifact-path-conflict/probe_artifact_root.py diff --git a/docs/lanes/README.md b/docs/lanes/README.md new file mode 100644 index 0000000..30368b8 --- /dev/null +++ b/docs/lanes/README.md @@ -0,0 +1,39 @@ +# Lane artifacts + +Automated work on this repo arrives in **lanes**. A lane is one branch, one +work item, one directory of its own record: its `DONE-NOTE.md`, its findings, +the evidence behind them, and any patch it proposes for another repo. + +**This repo's artifact root is `docs/lanes//`.** One directory per lane, +created by that lane, owned by that lane. + +``` +docs/lanes// +├── DONE-NOTE.md # what was done, what was measured, what is still open +├── evidence/ # command output the note's claims are quoted from +└── PROPOSED-*.patch # a change this lane proposes to a different repo +``` + +## Rules + +1. **Never write `DONE-NOTE.md` at the repo root.** The root path is shared by + every lane, so the last writer silently overwrites the previous one. That + happened; the notes were recovered, and the shape is now refused by a guard. +2. **Never write into another lane's directory.** A lane's record is only + meaningful because exactly one lane produced it. +3. **Lane artifacts are not repo content.** Source, tests, and documentation + the repo wanted go where the repo's own structure says, exactly as if a + human had written them. Only the lane's *record* belongs here. + +## `ai_working/` + +Four lanes landed their notes under `ai_working//` before this repo's +convention was settled (`3yc`, `adq`, `9kk`, and a since-corrected `2nz`). +Those notes are left exactly where they are: they are the evidence trail of +merged, verified work, and each followed the instruction it was given. New +lanes use `docs/lanes/`. + +The rule is machine-checked by `check_lane_artifact_paths.py` in the evaluation +repo (`artifact-path/v1`), which resolves this repo to `docs/lanes/` by +declaration — see `docs/lanes/aof-artifact-path-conflict/DONE-NOTE.md` for why +that is a declaration rather than something inferred from the directory tree. diff --git a/docs/lanes/aof-artifact-path-conflict/DONE-NOTE.md b/docs/lanes/aof-artifact-path-conflict/DONE-NOTE.md new file mode 100644 index 0000000..a5c266e --- /dev/null +++ b/docs/lanes/aof-artifact-path-conflict/DONE-NOTE.md @@ -0,0 +1,181 @@ +# DONE-NOTE — `model_performance-aof` · lane `aof-artifact-path-conflict` + +**The decision: option (a). The goal template wins.** `check_lane_artifact_paths.py` +now resolves `amplifier-app-cli` to `docs/lanes//` by declaration, not by +inference. **This note is at `docs/lanes/aof-artifact-path-conflict/DONE-NOTE.md`, +the path both sides now name** — the first lane in this repo for which that is +unambiguous. + +Spend: **$0.00** against a $0.00 authority (`0 runs × 0 arms × $0 / 1.00 = $0.00`). +No API calls, no DTU, no infrastructure created or claimed. The authority's +arithmetic closes: this was a pure source change and nothing needed buying. + +--- + +## Which option, and why — argued from the code + +**(a), and (b) turns out not to be the cheap option it looks like.** The two +"sides" are not independent: `tools/rewrite_goal_artifact_paths.py:49` imports +`resolve_artifact_dir` **from the checker** and generates each goal's ARTIFACT +ROOT sentence from it — its own docstring says this is deliberate, "so the +instruction and the check can never disagree." So the per-repo goal text is +*derived from the checker*, and the only edit that moves both sides at once is +an edit to the checker. Under (b) you would still have to change the resolver +(so the generator emits `ai_working/`), **plus** change the hand-written CODE +goal template — which lives in the manager's per-batch `goals/` directory, +outside any git repo, with no test harness. The deliverable that stops the +recurrence is *a test*, and under (b) there is nowhere for it to live. (b) is +not cheaper; it is (a) with the answer inverted and the guard dropped. + +Two facts then decide which answer is right, rather than which is convenient: + +1. **R2 fires for this repo because a lane artifact created the directory.** + The first tracked path under `ai_working/` is + `ai_working/3yc-timedout-session-resumable/DONE-NOTE.md` (`0d93352`, + 2026-09-02) — a lane's own note. At the base ref `35ab604`, *all five* + tracked paths under `ai_working/` are lane artifacts; the repo tracks no + content there at any ref, and its `.gitignore` mentions only + `ai_working/tmp`. The rule was reading its own past output back as evidence + of a repo convention — a feedback loop, not a convention. +2. **The repo's own recent merged history says `docs/lanes/`.** Four + consecutive merged lanes placed their notes there: n1i (#292), 9w0 (#297), + eem (#298), 2nz (#299). All four were graded VIOLATION by the checker, and + `2nz` paid the churn twice — it wrote its note to `ai_working/` on the + checker's say-so, then deleted it and rewrote it under `docs/lanes/` + (`c80e04e`). + +`docs/lanes/` is also a subdirectory of a real documentation tree a reader +already looks in; `ai_working/` is a scratch directory this repo partially +gitignores. + +--- + +## What changed + +**In the evaluation repo** (`.amplifier/evaluation`), branch +`lane/aof-artifact-path-conflict`, commit `449ed59`, base `6269c44`: + +* `tools/check_lane_artifact_paths.py` — a new **R-PIN** rule, checked before + R0–R3: + + ```python + PINNED_ARTIFACT_ROOTS: dict[str, tuple[str, tuple[str, ...]]] = { + "amplifier-app-cli": ("docs/lanes", ("ai_working",)), + } + ``` + + Keyed on the checkout directory name **and** the `origin` URL's repo slug, so + the pin does not silently stop applying to a checkout cloned under another + name. `legacy_roots` is retrospective only — `dir_for()` never returns one, so + no new lane is ever sent to `ai_working/`, while a lane that already landed + there stays COMPLIANT. +* `scenarios/_harness/tests/test_artifact_path_resolution.py` — 10 tests. + Suite **233 → 243 passed**. + +**In this repo**: `docs/lanes/README.md` (the human-readable half of the same +statement) and this lane's own directory. **No app-cli source changed** — full +suite `1682 passed, 1 skipped, 13 deselected, 1 xfailed`, byte-identical to the +`35ab604` baseline. + +### Deviation, stated plainly + +The goal launched this as a CODE lane with an `amplifier-app-cli` worktree, but +**both candidate fixes live in the evaluation repo, not in this one** — the +checker is `.amplifier/evaluation/tools/check_lane_artifact_paths.py` and the +goal generator is beside it. Writing only a DONE-NOTE here and calling the item +done would have left the actual conflict in place, so I made the change where +it lives, on a lane branch in a git worktree +(`lanes/aof-artifact-path-conflict/evals-fix/evaluation`), exactly as every +evals lane in this batch does. Nothing was merged; that repo has no remote, so +there is no PR to open on it. The complete patch is committed here as +`PROPOSED-evals-artifact-path-R-PIN.patch` so it is reviewable from the app-cli +PR and survives teardown of the worktree. + +The worktree is nested one level down (`evals-fix/evaluation`) on purpose: +`check_lane_artifact_paths.py --lane-dir` picks the first directory under the +lane dir containing a `.git`, so a second checkout placed directly beside +`amplifier-app-cli/` would have made this lane's own grading depend on +`iterdir()` ordering. Verified: `--lane-dir` still resolves to +`amplifier-app-cli`. + +--- + +## Deliverables + +| Deliverable | State | +|---|---| +| A goal-conformant DONE-NOTE gets NO VIOLATION | **DONE** — demonstrated, not asserted. On **this real lane**: `--strict` exit **1 / VIOLATION** under the parent checker, **0 / COMPLIANT** under the patched one, same command, same directory (`evidence/07-this-lane-graded.txt`). Synthetic control, same script both ways: `evidence/01-fail-before.txt` vs `02-pass-after.txt` | +| A test pinning the resolution for this repo | **DONE** — `test_pin_survives_every_directory_that_would_have_moved_it` tracks `probes/`, `ai_working/` **and** a three-member R0 wave family at the base ref simultaneously and asserts the answer does not move | +| The `kez` hazard is still refused | **DONE** — `test_root_done_note_is_still_a_violation` (pinned repo) and `test_root_done_note_is_a_violation_in_an_unpinned_repo_too`. Live confirmation: `adq` is still VIOLATION for `../DONE-NOTE.md` | +| Landed artifacts under BOTH conventions left where they are | **DONE** — nothing relocated. `ai_working/adq-.../DONE-NOTE.md` and `ai_working/9kk-.../DONE-NOTE.md` are reported `ok` under the pin | +| Say which option and why | **DONE** — above | +| Full suite green | **DONE** — app-cli 1682 passed (`evidence/06`); evals 243 passed (`evidence/05`) | +| Fail-before evidence | **DONE** — `evidence/01`, plus the batch-wide before/after diff in `evidence/03` | +| DRAFT PR on origin | **DONE** — see the PR body; the evals half is carried as a patch artifact because that repo has no remote | +| DONE-NOTE follows the convention landed on | **DONE** — `docs/lanes/aof-artifact-path-conflict/DONE-NOTE.md` | + +Nothing is NOT-POSSIBLE, and the $0 cap never bound. + +--- + +## Measured effect, batch-wide + +`check_lane_artifact_paths.py --manifest /manifest.tsv --all --markdown`, +before vs after (`evidence/03-batch-report-diff.txt`): + +| | COMPLIANT | VIOLATION | +|---|---:|---:| +| whole batch, before | 58 | 56 | +| whole batch, after | 62 | 52 | +| `amplifier-app-cli`, before | 0 | 7 | +| `amplifier-app-cli`, after | **4** | **3** | + +**Only `amplifier-app-cli` rows move.** Every other repo's rule, expected shape +and per-lane status are byte-identical — the diff contains no other repo. That +is the SCOPE-OUT "do not change any other repo's convention", checked rather +than asserted, and it is also pinned by +`test_unpinned_repo_still_infers_from_the_tree`. + +The 3 residual app-cli violations are pre-existing and **unrelated to the root +choice** — the pin neither creates nor hides them: + +* `adq` — `../DONE-NOTE.md`, written *outside* the repo checkout. This is the + `kez` shape and is correctly still refused. +* `9kk` — wrote into `ai_working/adq-routing-list-shadowing/`, another lane's + directory. Its own note is `ok`. +* `3yc` — the manifest lane id is `3yc-timedout-session-not-resumable`; the + directory is `ai_working/3yc-timedout-session-resumable/` (no "not"). A name + mismatch, identical before and after. + +## Both sides agree — at the generator, not just the checker + +`rewrite_goal_artifact_paths.artifact_dir_for()` is the function that writes the +ARTIFACT ROOT sentence into a goal file. For all 8 app-cli lanes in the manifest +it returned `ai_working//` before and returns `docs/lanes//` after +(`evidence/04-goal-generator-before-after.txt`). The instruction a future +app-cli lane is *given* and the rule it is *graded against* are now the same +string, produced by the same call. + +--- + +## Honest limits + +* **The pin is a list, and a list has to be maintained.** Adding a repo to + `PINNED_ARTIFACT_ROOTS` is a decision someone must make; R0–R3 still guess for + every repo not in it. That is the trade this item asked for — the defect was + that the answer was *inferred*, and an inference cannot be made to hold still. +* **`PRE-RULE` can no longer fire for a pinned repo** (it triggers on + `expected_now != expected`, and a pin makes those equal at every ref). + `legacy_roots` covers the same ground for `amplifier-app-cli` and is why the + three `ai_working/` lanes stay COMPLIANT — but a future pin added without a + `legacy_roots` entry would retroactively fail lanes that obeyed the old + answer. The docstring says so at the table. +* **The evals-side commit is on a branch in a repo with no remote.** It is + durable in that repo's object store (`449ed59`) but, unlike the app-cli half, + no third party can read it back from a remote. It needs a local merge to take + effect. +* **`docs/lanes/README.md` is repo content this lane added but was not + chartered to write.** It exists so the app-cli side of the decision is legible + to a human who never runs the checker, and so this PR carries something other + than a lane note. Drop it if the reviewer disagrees; nothing else depends on + it. diff --git a/docs/lanes/aof-artifact-path-conflict/PROPOSED-evals-artifact-path-R-PIN.patch b/docs/lanes/aof-artifact-path-conflict/PROPOSED-evals-artifact-path-R-PIN.patch new file mode 100644 index 0000000..603d33a --- /dev/null +++ b/docs/lanes/aof-artifact-path-conflict/PROPOSED-evals-artifact-path-R-PIN.patch @@ -0,0 +1,439 @@ +From 449ed59a53e11530d8113dc7b6265bcbb5e9aceb Mon Sep 17 00:00:00 2001 +From: lane aof +Date: Thu, 3 Sep 2026 07:20:47 -0700 +Subject: [PATCH] fix(artifact-path): declare amplifier-app-cli's artifact root + instead of inferring it (aof) + +`check_lane_artifact_paths.py` resolved amplifier-app-cli to `ai_working/` +[R2] while the CODE goal template told every app-cli lane `docs/lanes/`, so a +lane that followed its own authoritative spec was graded a VIOLATION. Four +consecutive merged lanes (n1i #292, 9w0 #297, eem #298, 2nz #299) were in that +position; 2nz wrote its note to `ai_working/` on this checker's say-so, then +deleted it and rewrote it under `docs/lanes/` (c80e04e). + +R2 fired only because a LANE ARTIFACT made `ai_working/` tracked: the first +tracked path there is `ai_working/3yc-.../DONE-NOTE.md` (0d93352). The rule was +reading its own past output back as repo convention, and the repo tracks no +other content there at any ref. + +Option (a): the goal template wins. R-PIN, checked before R0-R3, resolves a +named repo to a DECLARED root that no directory in the tree can move. +`legacy_roots` keeps it retroactively honest -- artifacts that landed under the +inferred root stay COMPLIANT and stay where they are; only what a NEW lane is +told changes. + +Because `rewrite_goal_artifact_paths.py` derives each goal's ARTIFACT ROOT from +this same resolver, both sides now say `docs/lanes/` for app-cli: the generator +emits it and the checker accepts it. + +Batch-wide, only app-cli moves: 0 COMPLIANT/7 VIOLATION -> 4/3. The 3 residual +violations are pre-existing and unrelated to the root choice (adq's +`../DONE-NOTE.md` outside the repo; 9kk writing into adq's directory; 3yc's +manifest lane id differing from its directory name). + +tests: scenarios/_harness/tests/test_artifact_path_resolution.py (10 tests), +including the recurrence-stopper (probes/ + ai_working/ + a full R0 wave family +tracked at once, answer does not move) and `kez`'s hazard -- a repo-root +DONE-NOTE.md is still a VIOLATION, pinned repo and unpinned repo alike. +suite: 233 -> 243 passed. +--- + .../tests/test_artifact_path_resolution.py | 262 ++++++++++++++++++ + tools/check_lane_artifact_paths.py | 82 +++++- + 2 files changed, 342 insertions(+), 2 deletions(-) + create mode 100644 scenarios/_harness/tests/test_artifact_path_resolution.py + +diff --git a/scenarios/_harness/tests/test_artifact_path_resolution.py b/scenarios/_harness/tests/test_artifact_path_resolution.py +new file mode 100644 +index 0000000..8fa2c5c +--- /dev/null ++++ b/scenarios/_harness/tests/test_artifact_path_resolution.py +@@ -0,0 +1,262 @@ ++#!/usr/bin/env python3 ++"""The artifact-root resolution guard: R-PIN, and what it must not move ++(`model_performance-aof`). ++ ++Runnable either way, like every other test here: ++ ++ ./run_tests.sh ++ python3 -m pytest scenarios/_harness/tests/test_artifact_path_resolution.py ++ python3 scenarios/_harness/tests/test_artifact_path_resolution.py ++ ++WHAT IS BEING GUARDED ++--------------------- ++`artifact-path/v1` used to derive a repo's artifact root ENTIRELY from which ++directories were tracked at the base ref. A lane artifact is a tracked file, so ++the rule could read its own past output back as a repo convention: in ++`amplifier-app-cli` the first tracked path under `ai_working/` was a lane's own ++DONE-NOTE (0d93352), and from that commit on R2 fired and told every later ++app-cli lane `ai_working/` -- while the CODE goal template said `docs/lanes/`. ++Four merged lanes (n1i #292, 9w0 #297, eem #298, 2nz #299) followed their goal ++and were graded VIOLATION. ++ ++The defect was never WHICH directory won. It was that the answer was INFERRED ++from a directory's existence, so the next tracked directory could flip it again ++without anyone deciding anything. `PINNED_ARTIFACT_ROOTS` makes it a ++declaration; these tests are what stop it drifting back: ++ ++ * `test_pinned_repo_resolves_to_the_declared_root` -- the resolution itself ++ * `test_pin_survives_every_directory_that_would_have_moved_it` ++ -- the recurrence-stopper: probes/, ai_working/ and a full R0 wave ++ family are all present AND the answer does not move ++ * `test_unpinned_repo_still_infers_from_the_tree` ++ -- the pin is narrow; no other repo's convention changed ++ * `test_root_done_note_is_still_a_violation` ++ -- `kez`'s hazard. A fix that relaxed this to end the disagreement ++ would have made things worse than the disagreement. ++""" ++ ++from __future__ import annotations ++ ++import subprocess ++import sys ++import tempfile ++from pathlib import Path ++ ++REPO = Path(__file__).resolve().parents[3] ++sys.path.insert(0, str(REPO / "tools")) ++ ++from check_lane_artifact_paths import ( # noqa: E402 ++ PINNED_ARTIFACT_ROOTS, ++ evaluate, ++ resolve_artifact_dir, ++) ++ ++PINNED_REPO = "amplifier-app-cli" ++DECLARED_ROOT = "docs/lanes" ++LEGACY_ROOT = "ai_working" ++ ++ ++# -------------------------------------------------------------------------- ++# scratch repos, built from nothing so the result depends on no live history ++# -------------------------------------------------------------------------- ++ ++ ++def _git(repo: Path, *args: str) -> str: ++ out = subprocess.run(["git", "-C", str(repo), *args], capture_output=True, text=True, check=False) ++ assert out.returncode == 0, f"git {' '.join(args)} failed: {out.stderr}" ++ return out.stdout.strip() ++ ++ ++def _scratch_repo(td: Path, name: str, dirs: list[str], origin: str = "") -> Path: ++ """A repo named `name` whose base commit tracks a file in each of `dirs`.""" ++ repo = td / name ++ repo.mkdir(parents=True) ++ _git(repo, "init", "--quiet", "-b", "main") ++ _git(repo, "config", "user.email", "lane@example.invalid") ++ _git(repo, "config", "user.name", "lane") ++ if origin: ++ _git(repo, "remote", "add", "origin", origin) ++ (repo / "README.md").write_text("scratch\n", encoding="utf-8") ++ for d in dirs: ++ (repo / d).mkdir(parents=True, exist_ok=True) ++ (repo / d / "keep.md").write_text("content\n", encoding="utf-8") ++ _git(repo, "add", "-A") ++ _git(repo, "commit", "--quiet", "-m", "base") ++ return repo ++ ++ ++def _lane_commit(repo: Path, branch: str, path: str) -> str: ++ """Commit `path` on `branch`; returns the base sha the branch left from.""" ++ base = _git(repo, "rev-parse", "main") ++ _git(repo, "checkout", "--quiet", "-b", branch) ++ target = repo / path ++ target.parent.mkdir(parents=True, exist_ok=True) ++ target.write_text("# DONE-NOTE\n\nbody\n", encoding="utf-8") ++ _git(repo, "add", "-A") ++ _git(repo, "commit", "--quiet", "-m", "lane note") ++ return base ++ ++ ++# -------------------------------------------------------------------------- ++# R-PIN ++# -------------------------------------------------------------------------- ++ ++ ++def test_pinned_repo_resolves_to_the_declared_root() -> None: ++ """FAIL-BEFORE half: with `ai_working/` tracked, the unpinned rule returned ++ `ai_working/` -- which is exactly the answer the goal contradicted.""" ++ with tempfile.TemporaryDirectory() as td: ++ repo = _scratch_repo(Path(td), PINNED_REPO, [LEGACY_ROOT]) ++ res = resolve_artifact_dir(repo, "main") ++ assert res.root == DECLARED_ROOT, f"pinned repo resolved to {res.root!r}" ++ assert res.rule.startswith("R-PIN"), res.rule ++ assert res.dir_for("zzz-lane") == "docs/lanes/zzz-lane" ++ ++ ++def test_pin_survives_every_directory_that_would_have_moved_it() -> None: ++ """THE RECURRENCE-STOPPER. ++ ++ `probes/` (R1), `ai_working/` (R2) and a three-member `w9-` wave family ++ (R0) are all tracked at the base ref -- every inference the rule can make, ++ all present at once, each of which alone used to decide the answer. A ++ declared root ignores all of them. ++ """ ++ with tempfile.TemporaryDirectory() as td: ++ repo = _scratch_repo( ++ Path(td), ++ PINNED_REPO, ++ ["probes", LEGACY_ROOT, "w9-alpha", "w9-beta", "w9-gamma"], ++ ) ++ res = resolve_artifact_dir(repo, "main") ++ assert res.root == DECLARED_ROOT, ( ++ f"a tracked directory moved a DECLARED root to {res.root!r} -- " ++ "this is the exact drift the pin exists to prevent" ++ ) ++ assert res.prefix == "", f"a wave prefix leaked into a pinned repo: {res.prefix!r}" ++ ++ ++def test_pin_follows_the_origin_slug_when_the_checkout_is_renamed() -> None: ++ """A pin keyed on the directory name alone stops applying the moment someone ++ clones the repo under a different name.""" ++ with tempfile.TemporaryDirectory() as td: ++ repo = _scratch_repo( ++ Path(td), ++ "app-cli-checkout-2", ++ [LEGACY_ROOT], ++ origin=f"https://github.com/microsoft/{PINNED_REPO}.git", ++ ) ++ res = resolve_artifact_dir(repo, "main") ++ assert res.root == DECLARED_ROOT, f"renamed checkout resolved to {res.root!r}" ++ ++ ++def test_unpinned_repo_still_infers_from_the_tree() -> None: ++ """The pin is NARROW. No repo outside the table changed convention -- R0-R3 ++ still resolve exactly as they did before.""" ++ cases = [ ++ (["probes"], "probes", "", "R1"), ++ ([LEGACY_ROOT], LEGACY_ROOT, "", "R2"), ++ ([], "docs/lanes", "", "R3"), ++ (["w3-a", "w3-b", "w3-c"], "", "w3-", "R0"), ++ ] ++ with tempfile.TemporaryDirectory() as td: ++ for i, (dirs, root, prefix, rule) in enumerate(cases): ++ repo = _scratch_repo(Path(td), f"some-other-repo-{i}", dirs) ++ res = resolve_artifact_dir(repo, "main") ++ assert (res.root, res.prefix) == (root, prefix), f"{rule}: got {res.root!r}/{res.prefix!r}" ++ assert res.rule.startswith(rule), f"{rule}: got {res.rule!r}" ++ ++ ++def test_the_pin_table_is_well_formed() -> None: ++ """A malformed entry would silently resolve to a nonsense directory.""" ++ for name, entry in PINNED_ARTIFACT_ROOTS.items(): ++ root, legacy = entry ++ assert name and "/" not in name, f"pin key is a repo name, not a path: {name!r}" ++ assert root and not root.startswith("/") and not root.endswith("/"), f"{name}: {root!r}" ++ assert isinstance(legacy, tuple), f"{name}: legacy roots must be a tuple" ++ assert root not in legacy, f"{name}: declared root repeated as legacy" ++ ++ ++# -------------------------------------------------------------------------- ++# end-to-end: what a lane is graded, in a real repo ++# -------------------------------------------------------------------------- ++ ++ ++def test_a_goal_conformant_note_is_compliant() -> None: ++ """THE DELIVERABLE, demonstrated rather than asserted: a lane that places ++ its DONE-NOTE where its goal names gets no VIOLATION.""" ++ with tempfile.TemporaryDirectory() as td: ++ repo = _scratch_repo(Path(td), PINNED_REPO, [LEGACY_ROOT]) ++ lane = "aaa-example" ++ base = _lane_commit(repo, f"lane/{lane}", f"{DECLARED_ROOT}/{lane}/DONE-NOTE.md") ++ r = evaluate(lane, repo, base, f"lane/{lane}") ++ assert r.status == "COMPLIANT", f"{r.status}: expected {r.expected}, strays {r.strays}" ++ ++ ++def test_a_landed_legacy_note_stays_compliant() -> None: ++ """Retrospective tolerance: lanes that landed under the inferred root obeyed ++ the answer in force at the time. The pin must not turn merged, verified work ++ into a violation, and must not be a reason to relocate it.""" ++ with tempfile.TemporaryDirectory() as td: ++ repo = _scratch_repo(Path(td), PINNED_REPO, [LEGACY_ROOT]) ++ lane = "bbb-legacy" ++ base = _lane_commit(repo, f"lane/{lane}", f"{LEGACY_ROOT}/{lane}/DONE-NOTE.md") ++ r = evaluate(lane, repo, base, f"lane/{lane}") ++ assert r.status == "COMPLIANT", f"{r.status}: strays {r.strays}" ++ ++ ++def test_a_new_lane_is_never_told_the_legacy_root() -> None: ++ """Tolerating a legacy root is not the same as recommending it: exactly one ++ directory is ever named to a new lane.""" ++ with tempfile.TemporaryDirectory() as td: ++ repo = _scratch_repo(Path(td), PINNED_REPO, [LEGACY_ROOT]) ++ res = resolve_artifact_dir(repo, "main") ++ told = res.dir_for("ccc-new") ++ assert told == f"{DECLARED_ROOT}/ccc-new", told ++ assert LEGACY_ROOT not in told ++ ++ ++def test_root_done_note_is_still_a_violation() -> None: ++ """`model_performance-kez`'s hazard, under the pin. ++ ++ A repo-root DONE-NOTE.md is the shape every lane used to overwrite. It is ++ the real thing this checker is for, and relaxing it to end the app-cli ++ disagreement would have been strictly worse than the disagreement. ++ """ ++ with tempfile.TemporaryDirectory() as td: ++ repo = _scratch_repo(Path(td), PINNED_REPO, [LEGACY_ROOT]) ++ lane = "ddd-root-note" ++ base = _lane_commit(repo, f"lane/{lane}", "DONE-NOTE.md") ++ r = evaluate(lane, repo, base, f"lane/{lane}") ++ assert r.status == "VIOLATION", f"root DONE-NOTE.md graded {r.status}" ++ assert "DONE-NOTE.md" in r.strays, r.strays ++ ++ ++def test_root_done_note_is_a_violation_in_an_unpinned_repo_too() -> None: ++ """The same hazard, in a repo the pin never touches -- so the guard cannot ++ be read as depending on it.""" ++ with tempfile.TemporaryDirectory() as td: ++ repo = _scratch_repo(Path(td), "some-unpinned-repo", ["probes"]) ++ lane = "eee-root-note" ++ base = _lane_commit(repo, f"lane/{lane}", "DONE-NOTE.md") ++ r = evaluate(lane, repo, base, f"lane/{lane}") ++ assert r.status == "VIOLATION", f"root DONE-NOTE.md graded {r.status}" ++ ++ ++def _main() -> int: ++ failures = 0 ++ for name, fn in sorted(globals().items()): ++ if not name.startswith("test_") or not callable(fn): ++ continue ++ try: ++ fn() ++ except AssertionError as exc: ++ failures += 1 ++ print(f"FAIL {name}: {exc}") ++ else: ++ print(f"ok {name}") ++ print("failures:", failures) ++ return 1 if failures else 0 ++ ++ ++if __name__ == "__main__": # pragma: no cover ++ sys.exit(_main()) +diff --git a/tools/check_lane_artifact_paths.py b/tools/check_lane_artifact_paths.py +index 5aaa534..be7bfd8 100644 +--- a/tools/check_lane_artifact_paths.py ++++ b/tools/check_lane_artifact_paths.py +@@ -35,6 +35,8 @@ A lane writes every LANE ARTIFACT under exactly one directory: + resolved against the ref the lane branched from (`base_sha` in manifest.tsv), + first match wins: + ++ R-PIN the repo is named in PINNED_ARTIFACT_ROOTS below -> that root, always. ++ DECLARED, not inferred: no directory appearing in the tree moves it. + R0 precedent -- >=3 top-level dirs at the base ref share a `-` name + prefix (a wave convention). ARTIFACT_ROOT = repo root, + DIR_PREFIX = the greatest such prefix. ai-notes -> `w3-` +@@ -42,6 +44,35 @@ first match wins: + R2 `ai_working/` tracked at the base ref -> `ai_working//` + R3 otherwise -> `docs/lanes//` + ++WHY R-PIN EXISTS (`model_performance-aof`) ++------------------------------------------ ++R0-R3 infer a repo's convention from which directories happen to be TRACKED at ++the base ref. A lane artifact is a tracked file, so the rule can read its own ++past output back as evidence of a repo convention. Measured in ++`amplifier-app-cli`: the first tracked path under `ai_working/` is ++`ai_working/3yc-timedout-session-resumable/DONE-NOTE.md` (0d93352, 2026-09-02) ++-- a lane's own note. That directory holds no repo content at any ref; the ++repo's `.gitignore` mentions only `ai_working/tmp`. From that commit onward R2 ++fired, so `rewrite_goal_artifact_paths.py` -- which derives each goal's ARTIFACT ++ROOT from THIS resolver, precisely so the instruction and the check cannot ++disagree -- began telling app-cli lanes `ai_working/`, while the hand-written ++CODE goal template kept saying `docs/lanes/`. Four consecutive merged lanes ++(n1i #292, 9w0 #297, eem #298, 2nz #299) followed their goal and this checker ++graded every one of them a VIOLATION. `2nz` first wrote its note to ++`ai_working/` on this checker's say-so, then deleted it and rewrote it under ++`docs/lanes/` (c80e04e), because the goal is a lane's authoritative spec. That ++churn is what a pin removes. ++ ++A pin makes a repo's answer a DECLARATION rather than an inference, so the next ++tracked directory added at the base ref cannot silently flip it back. It is ++deliberately narrow: a repo absent from the table resolves exactly as before. ++ ++`legacy_roots` keeps the pin retroactively honest. Lanes that landed under the ++inferred root while it was in force (`3yc`, `adq`, `9kk` under `ai_working/`) ++obeyed the answer they were given, so their artifacts stay COMPLIANT and stay ++where they are; only what a NEW lane is TOLD changes. This mirrors R0's ++`family` tolerance, which is likewise retrospective only. ++ + LANE ARTIFACT vs REPO CONTENT -- the distinction that makes the rule uniform: + + * A LANE ARTIFACT is the record of this lane's work: its note, findings, +@@ -107,6 +138,33 @@ REPO_CONTENT_DIRS = {".github", "tests", "test", "src", "crates", "bindings", "p + + WAVE_PREFIX_RE = re.compile(r"^([a-z][a-z0-9]*)-.+$") + ++# -------------------------------------------------------------------------- ++# R-PIN -- repos whose artifact root is DECLARED, not inferred from the tree ++# -------------------------------------------------------------------------- ++# repo -> (artifact root, legacy roots that stay acceptable retrospectively). ++# Keyed on BOTH the checkout directory name and the `origin` URL's repo slug, ++# so a renamed checkout still pins (see `repo_identities`). ++# ++# Add a repo here only to settle a real disagreement between the rule and that ++# repo's own goal text, and say which one moved. `amplifier-app-cli`: ++# `model_performance-aof` -- goal template wins, option (a). ++PINNED_ARTIFACT_ROOTS: dict[str, tuple[str, tuple[str, ...]]] = { ++ "amplifier-app-cli": ("docs/lanes", ("ai_working",)), ++} ++ ++ ++def repo_identities(repo: Path) -> set[str]: ++ """Every name this checkout answers to: its directory name and its `origin` ++ repo slug. A pin keyed on the directory name alone would silently stop ++ applying to a checkout someone cloned under a different name.""" ++ names = {repo.name} ++ url = git(repo, "remote", "get-url", "origin").strip() ++ if url: ++ slug = url.rstrip("/").removesuffix(".git").rsplit("/", 1)[-1] ++ if slug: ++ names.add(slug) ++ return names ++ + + def git(repo: Path, *args: str) -> str: + out = subprocess.run( +@@ -141,6 +199,11 @@ class Resolution: + # Every prefix in the repo's wave family (R0 only). A goal file pins ONE of + # these; a lane that used an earlier wave still followed the convention. + family: tuple[str, ...] = () ++ # Roots this repo used BEFORE it was pinned (R-PIN only). Retrospective ++ # only: a lane that landed under one obeyed the answer in force at the ++ # time, so it stays COMPLIANT and its artifacts stay where they are. ++ # `dir_for` never returns one, so no NEW lane is ever sent there. ++ legacy_roots: tuple[str, ...] = () + + def dir_for(self, lane: str) -> str: + name = f"{self.prefix}{lane}" +@@ -150,11 +213,26 @@ class Resolution: + if path.startswith(self.dir_for(lane) + "/"): + return True + # R0 family tolerance, retrospective only. +- return any(path.startswith(f"{p}{lane}/") for p in self.family) ++ if any(path.startswith(f"{p}{lane}/") for p in self.family): ++ return True ++ # R-PIN legacy tolerance, retrospective only. ++ return any(path.startswith(f"{r}/{lane}/") for r in self.legacy_roots) + + + def resolve_artifact_dir(repo: Path, ref: str) -> Resolution: +- """Apply artifact-path/v1 against `ref`. Pure function of the tree.""" ++ """Apply artifact-path/v1 against `ref`. ++ ++ A pure function of the tree EXCEPT for R-PIN, which is a function of the ++ repo's identity alone -- that is the point of it: an inference from the ++ tree is what a lane artifact can silently move. ++ """ ++ # R-PIN -- declared, and nothing in the tree overrides it. ++ for name in sorted(repo_identities(repo)): ++ pinned = PINNED_ARTIFACT_ROOTS.get(name) ++ if pinned: ++ root, legacy = pinned ++ return Resolution(root, "", f"R-PIN {root}/ (declared for {name})", legacy_roots=legacy) ++ + dirs, _ = top_level_entries(repo, ref) + + # R0 -- wave/precedent convention +-- +2.43.0 + diff --git a/docs/lanes/aof-artifact-path-conflict/evidence/00-conflict-today.txt b/docs/lanes/aof-artifact-path-conflict/evidence/00-conflict-today.txt new file mode 100644 index 0000000..ecf04ec --- /dev/null +++ b/docs/lanes/aof-artifact-path-conflict/evidence/00-conflict-today.txt @@ -0,0 +1,19 @@ +$ grep -o 'ARTIFACT ROOT.*' /home/bkrabach/dev/hw-model-performance/goals/aof-artifact-path-conflict.md | head -1 +ARTIFACT ROOT `docs/lanes/aof-artifact-path-conflict/` + +$ python3 tools/check_lane_artifact_paths.py --lane-dir /home/bkrabach/dev/hw-model-performance/lanes/aof-artifact-path-conflict # evals main 6269c44 +NO-COMMITS aof-artifact-path-conflict amplifier-app-cli expected ai_working/aof-artifact-path-conflict/ [R2 ai_working/] + +# the goal says docs/lanes/; the checker says ai_working/. Same lane, same day. + +$ git -C /home/bkrabach/dev/hw-model-performance/lanes/aof-artifact-path-conflict/amplifier-app-cli ls-tree -r --name-only 35ab604 -- ai_working +ai_working/3yc-timedout-session-resumable/DONE-NOTE.md +ai_working/3yc-timedout-session-resumable/PATCH-foundation-surface-resume-detail.diff +ai_working/3yc-timedout-session-resumable/test_resume_message_roundtrip.py +ai_working/9kk-routing-list-lastwrite/DONE-NOTE.md +ai_working/adq-routing-list-shadowing/DONE-NOTE.md +# every tracked path under ai_working/ is a lane artifact. No repo content, at any ref. + +$ git -C /home/bkrabach/dev/hw-model-performance/lanes/aof-artifact-path-conflict/amplifier-app-cli log --reverse --format='%h %ad %s' --date=short -- ai_working | head -1 +0d93352 2026-09-02 fix(spawn): checkpoint sub-session transcripts during the run so a timed-out delegate is resumable +# R2 began firing for this repo when THAT lane's note was merged. diff --git a/docs/lanes/aof-artifact-path-conflict/evidence/01-fail-before.txt b/docs/lanes/aof-artifact-path-conflict/evidence/01-fail-before.txt new file mode 100644 index 0000000..39c15b5 --- /dev/null +++ b/docs/lanes/aof-artifact-path-conflict/evidence/01-fail-before.txt @@ -0,0 +1,4 @@ +checker /home/bkrabach/dev/openai-evals-team-ci/.amplifier/evaluation/tools/check_lane_artifact_paths.py +RESOLVED ai_working// [R2 ai_working/] +GOAL-PATH docs/lanes/aof-example/DONE-NOTE.md -> VIOLATION (expected ai_working/aof-example/) +LEGACY-PATH ai_working/aof-example/DONE-NOTE.md -> COMPLIANT (expected ai_working/aof-example/) diff --git a/docs/lanes/aof-artifact-path-conflict/evidence/02-pass-after.txt b/docs/lanes/aof-artifact-path-conflict/evidence/02-pass-after.txt new file mode 100644 index 0000000..212049e --- /dev/null +++ b/docs/lanes/aof-artifact-path-conflict/evidence/02-pass-after.txt @@ -0,0 +1,4 @@ +checker /home/bkrabach/dev/hw-model-performance/lanes/aof-artifact-path-conflict/evals-fix/evaluation/tools/check_lane_artifact_paths.py +RESOLVED docs/lanes// [R-PIN docs/lanes/ (declared for amplifier-app-cli)] +GOAL-PATH docs/lanes/aof-example/DONE-NOTE.md -> COMPLIANT (expected docs/lanes/aof-example/) +LEGACY-PATH ai_working/aof-example/DONE-NOTE.md -> COMPLIANT (expected docs/lanes/aof-example/) diff --git a/docs/lanes/aof-artifact-path-conflict/evidence/03-batch-report-diff.txt b/docs/lanes/aof-artifact-path-conflict/evidence/03-batch-report-diff.txt new file mode 100644 index 0000000..7c12672 --- /dev/null +++ b/docs/lanes/aof-artifact-path-conflict/evidence/03-batch-report-diff.txt @@ -0,0 +1,29 @@ +diff <(evals main) <(lane/aof) of: + python3 tools/check_lane_artifact_paths.py --manifest /manifest.tsv --all --markdown + +3c3 +< | COMPLIANT | 58 | +--- +> | COMPLIANT | 62 | +7c7 +< | VIOLATION | 56 | +--- +> | VIOLATION | 52 | +12c12 +< | `amplifier-app-cli` | R2 ai_working/ | `ai_working//` | 0 | 7 | 0 | 1 | +--- +> | `amplifier-app-cli` | R-PIN docs/lanes/ (declared for amplifier-app-cli) | `docs/lanes//` | 4 | 3 | 0 | 1 | +81d80 +< | n1i-resume-thread-role | `amplifier-app-cli` | VIOLATION | `ai_working/n1i-resume-thread-role/` | `docs/lanes/n1i-resume-thread-role/DONE-NOTE.md` | +88c87 +< | adq-routing-list-shadowing | `amplifier-app-cli` | VIOLATION | `ai_working/adq-routing-list-shadowing/` | `../DONE-NOTE.md` | +--- +> | adq-routing-list-shadowing | `amplifier-app-cli` | VIOLATION | `docs/lanes/adq-routing-list-shadowing/` | `../DONE-NOTE.md` | +90c89 +< | 9kk-routing-list-lastwrite | `amplifier-app-cli` | VIOLATION | `ai_working/9kk-routing-list-lastwrite/` | `ai_working/adq-routing-list-shadowing/DONE-NOTE.md` | +--- +> | 9kk-routing-list-lastwrite | `amplifier-app-cli` | VIOLATION | `docs/lanes/9kk-routing-list-lastwrite/` | `ai_working/adq-routing-list-shadowing/DONE-NOTE.md` | +94,96d92 +< | 9w0-delegate-timeout-partial-producer | `amplifier-app-cli` | VIOLATION | `ai_working/9w0-delegate-timeout-partial-producer/` | `docs/lanes/9w0-delegate-timeout-partial-producer/DONE-NOTE.md`, `docs/lanes/9w0-delegate-timeout-partial-producer/conftest.py`, `docs/lanes/9w0-delegate-timeout-partial-producer/evidence/00-parent-baseline-named-suites.txt`, `docs/lanes/9w0-delegate-timeout-partial-producer/evidence/01-parent-baseline-full-suite.txt`, `docs/lanes/9w0-delegate-timeout-partial-producer/evidence/02-original-patch-does-not-apply.txt`, `docs/lanes/9w0-delegate-timeout-partial-producer/evidence/03-failbefore-roundtrip-parent.txt`, `docs/lanes/9w0-delegate-timeout-partial-producer/evidence/04-counterevidence-naive-shape-still-false.txt`, `docs/lanes/9w0-delegate-timeout-partial-producer/evidence/05-normal-completion-byte-identical.txt`, `docs/lanes/9w0-delegate-timeout-partial-producer/evidence/06-gd4-partial-available-true.txt`, `docs/lanes/9w0-delegate-timeout-partial-producer/evidence/07-passafter-roundtrip-patched.txt`, `docs/lanes/9w0-delegate-timeout-partial-producer/evidence/08-lint-and-full-suite-patched.txt`, `docs/lanes/9w0-delegate-timeout-partial-producer/probe_partial_roundtrip.py`, `docs/lanes/9w0-delegate-timeout-partial-producer/test_partial_roundtrip.py` | +< | eem-partial-accumulator-widen | `amplifier-app-cli` | VIOLATION | `ai_working/eem-partial-accumulator-widen/` | `docs/lanes/eem-partial-accumulator-widen/DONE-NOTE.md`, `docs/lanes/eem-partial-accumulator-widen/byte_identity_probe.py`, `docs/lanes/eem-partial-accumulator-widen/evidence/00-baseline-suite-parent.txt`, `docs/lanes/eem-partial-accumulator-widen/evidence/01-fail-before.txt`, `docs/lanes/eem-partial-accumulator-widen/evidence/03-byte-identity.txt`, `docs/lanes/eem-partial-accumulator-widen/evidence/03a-parent.json`, `docs/lanes/eem-partial-accumulator-widen/evidence/03b-patched.json`, `docs/lanes/eem-partial-accumulator-widen/evidence/04-roundtrip.txt`, `docs/lanes/eem-partial-accumulator-widen/evidence/05-full-suite.txt`, `docs/lanes/eem-partial-accumulator-widen/evidence/06-integration.txt`, `docs/lanes/eem-partial-accumulator-widen/evidence/07-real-leg-reachability.md`, `docs/lanes/eem-partial-accumulator-widen/test_partial_roundtrip.py` | +< | 2nz-upstream-skill-guards | `amplifier-app-cli` | VIOLATION | `ai_working/2nz-upstream-skill-guards/` | `docs/lanes/2nz-upstream-skill-guards/DONE-NOTE.md` | diff --git a/docs/lanes/aof-artifact-path-conflict/evidence/03a-batch-before.md b/docs/lanes/aof-artifact-path-conflict/evidence/03a-batch-before.md new file mode 100644 index 0000000..913faf1 --- /dev/null +++ b/docs/lanes/aof-artifact-path-conflict/evidence/03a-batch-before.md @@ -0,0 +1,96 @@ +| status | lanes | +| --- | ---: | +| COMPLIANT | 58 | +| NO-ARTIFACTS | 2 | +| NO-COMMITS | 6 | +| PRE-RULE | 16 | +| VIOLATION | 56 | + +| repo | rule (today) | expected dir shape | compliant | violation | pre-rule | no-artifacts | +| --- | --- | --- | ---: | ---: | ---: | ---: | +| `ai-notes` | R0 precedent | `w3-/` | 3 | 10 | 12 | 0 | +| `amplifier-app-cli` | R2 ai_working/ | `ai_working//` | 0 | 7 | 0 | 1 | +| `amplifier-bundle-routing-matrix` | R3 docs/lanes/ (fallback) | `docs/lanes//` | 5 | 8 | 0 | 0 | +| `amplifier-core` | R3 docs/lanes/ (fallback) | `docs/lanes//` | 0 | 1 | 0 | 0 | +| `amplifier-foundation` | R3 docs/lanes/ (fallback) | `docs/lanes//` | 4 | 2 | 0 | 0 | +| `amplifier-module-context-simple` | R3 docs/lanes/ (fallback) | `docs/lanes//` | 1 | 10 | 0 | 0 | +| `amplifier-module-loop-streaming` | R3 docs/lanes/ (fallback) | `docs/lanes//` | 0 | 1 | 0 | 0 | +| `amplifier-module-provider-anthropic` | R3 docs/lanes/ (fallback) | `docs/lanes//` | 0 | 0 | 0 | 1 | +| `amplifier-module-provider-openai` | R3 docs/lanes/ (fallback) | `docs/lanes//` | 0 | 1 | 0 | 3 | +| `amplifier-work-tracker` | R3 docs/lanes/ (fallback) | `docs/lanes//` | 6 | 0 | 0 | 1 | +| `evaluation` | R1 probes/ | `probes//` | 39 | 16 | 4 | 2 | + +| lane | repo | status | expected | strays / outside | +| --- | --- | --- | --- | --- | +| m1-openai-performance | `ai-notes` | PRE-RULE | `docs/lanes/m1-openai-performance/` | `w1-m1-openai-performance/DONE-NOTE.md`, `w1-m1-openai-performance/INDEX.md`, `w1-m1-openai-performance/_reader-brief.md`, `w1-m1-openai-performance/claims.md`, `w1-m1-openai-performance/notes/01-manus-kv-cache-context-engineering.md`, `w1-m1-openai-performance/notes/02-slowdown-after-memory-compaction.md`, `w1-m1-openai-performance/notes/03-youre-using-gpt5-wrong.md`, `w1-m1-openai-performance/notes/04-harness-wars-responses-api.md`, `w1-m1-openai-performance/notes/05-automatic-prompt-caching.md`, `w1-m1-openai-performance/notes/06-claude-code-8-compaction-modes.md`, `w1-m1-openai-performance/notes/07-subagent-model-routing.md`, `w1-m1-openai-performance/notes/08-openai-throw-away-your-prompt-stack.md`, `w1-m1-openai-performance/notes/09-opus5-effort-recalibration.md`, `w1-m1-openai-performance/notes/10-gpt56-sol-terra-luna.md`, `w1-m1-openai-performance/notes/11-gpt56-sol-reasoning-effort-curve.md`, `w1-m1-openai-performance/notes/12-context-engineering-is-the-new-moat.md`, `w1-m1-openai-performance/notes/13-context-rot-and-the-memory-crisis.md`, `w1-m1-openai-performance/notes/14-long-running-agents-effort-levels.md`, `w1-m1-openai-performance/notes/15-harness-context-assembly.md`, `w1-m1-openai-performance/notes/16-chain-of-draft.md`, `w1-m1-openai-performance/notes/17-four-claude-effort-levels.md`, `w1-m1-openai-performance/notes/18-gpt5-mini-rag-reasoning-effort.md`, `w1-m1-openai-performance/notes/19-advisor-tool-cheap-model-uplift.md`, `w1-m1-openai-performance/notes/20-the-loop-is-the-product.md`, `w1-m1-openai-performance/notes/21-building-claude-code-harness.md`, `w1-m1-openai-performance/notes/22-managed-agents-vs-deep-agents-vs-openai.md`, `w1-m1-openai-performance/questions.md` | +| m2-compaction-context | `ai-notes` | PRE-RULE | `docs/lanes/m2-compaction-context/` | `w1-m2-compaction-context/DONE-NOTE.md`, `w1-m2-compaction-context/INDEX.md`, `w1-m2-compaction-context/claims.md`, `w1-m2-compaction-context/notes/anthropic-automatic-prompt-caching.md`, `w1-m2-compaction-context/notes/anthropic-senior-engineer-hidden-flaws.md`, `w1-m2-compaction-context/notes/building-claude-code-harness-engineering.md`, `w1-m2-compaction-context/notes/cache-augmented-generation.md`, `w1-m2-compaction-context/notes/claude-code-leak-8-compaction-modes.md`, `w1-m2-compaction-context/notes/context-engineering-is-the-new-moat.md`, `w1-m2-compaction-context/notes/deep-agents-harness.md`, `w1-m2-compaction-context/notes/google-adk-context-architecture.md`, `w1-m2-compaction-context/notes/harness-context-assembly-four-levers.md`, `w1-m2-compaction-context/notes/hermes-agent-memory.md`, `w1-m2-compaction-context/notes/manus-context-engineering.md`, `w1-m2-compaction-context/notes/mcp-code-execution-token-bloat.md`, `w1-m2-compaction-context/notes/openclaw-memory-systems.md`, `w1-m2-compaction-context/notes/why-agents-slow-after-compaction-kv-cache.md`, `w1-m2-compaction-context/questions.md`, `w1-m2-compaction-context/searches/s1.txt`, `w1-m2-compaction-context/searches/s2.txt`, `w1-m2-compaction-context/searches/s3.txt`, `w1-m2-compaction-context/searches/s4.txt`, `w1-m2-compaction-context/searches/s5.txt`, `w1-m2-compaction-context/searches/s6.txt`, `w1-m2-compaction-context/searches/s7.txt`, `w1-m2-compaction-context/searches/s8.txt`, `w1-m2-compaction-context/searches/s9.txt` | +| m3-harness-engineering | `ai-notes` | PRE-RULE | `docs/lanes/m3-harness-engineering/` | `w1-m3-harness-engineering/DONE-NOTE.md`, `w1-m3-harness-engineering/INDEX.md`, `w1-m3-harness-engineering/claims.md`, `w1-m3-harness-engineering/notes/01-aci-swe-agent-ablation.md`, `w1-m3-harness-engineering/notes/02-deepagents-terminal-bench.md`, `w1-m3-harness-engineering/notes/03-manus-append-only-and-recitation.md`, `w1-m3-harness-engineering/notes/04-delegation-cost-and-role-tiering.md`, `w1-m3-harness-engineering/notes/05-pinned-facts-and-four-levers.md`, `w1-m3-harness-engineering/notes/06-tool-definition-cost-and-deferred-loading.md`, `w1-m3-harness-engineering/notes/07-tool-result-shaping-and-error-contracts.md`, `w1-m3-harness-engineering/notes/08-claude-code-prompt-boundary.md`, `w1-m3-harness-engineering/notes/09-compaction-kv-invalidation.md`, `w1-m3-harness-engineering/notes/10-topology-pareto-and-antipatterns.md`, `w1-m3-harness-engineering/questions.md` | +| m4-three-knob-routing | `ai-notes` | PRE-RULE | `docs/lanes/m4-three-knob-routing/` | `w1-m4-three-knob-routing/DONE-NOTE.md`, `w1-m4-three-knob-routing/INDEX.md`, `w1-m4-three-knob-routing/claims.md`, `w1-m4-three-knob-routing/notes/01-subagent-model-routing.md`, `w1-m4-three-knob-routing/notes/02-advisor-tool.md`, `w1-m4-three-knob-routing/notes/03-17-multi-agent-topologies.md`, `w1-m4-three-knob-routing/notes/04-harness-engineering-15x.md`, `w1-m4-three-knob-routing/notes/05-why-claude-is-getting-worse.md`, `w1-m4-three-knob-routing/notes/06-claude-4-effort-levels.md`, `w1-m4-three-knob-routing/notes/07-harness-wars-reasoning-billing.md`, `w1-m4-three-knob-routing/notes/08-tokenizer-and-xhigh.md`, `w1-m4-three-knob-routing/notes/09-gpt56-sol-terra-luna.md`, `w1-m4-three-knob-routing/notes/10-thinkingcap-and-chain-of-draft.md`, `w1-m4-three-knob-routing/notes/11-gpt5-effort-and-verbosity.md`, `w1-m4-three-knob-routing/notes/12-routing-in-practice-openrouter-and-manus.md`, `w1-m4-three-knob-routing/questions.md` | +| o2-anthropic-crossvendor | `ai-notes` | PRE-RULE | `docs/lanes/o2-anthropic-crossvendor/` | `w1-o2-anthropic-crossvendor/CROSS-VENDOR-SEMANTICS.md`, `w1-o2-anthropic-crossvendor/DONE-NOTE.md`, `w1-o2-anthropic-crossvendor/INDEX.md`, `w1-o2-anthropic-crossvendor/NON-REGRESSION-CHECKLIST.md`, `w1-o2-anthropic-crossvendor/claims.md`, `w1-o2-anthropic-crossvendor/questions.md` | +| o3-practitioner-compaction | `ai-notes` | PRE-RULE | `docs/lanes/o3-practitioner-compaction/` | `w1-o3-practitioner-compaction/DONE-NOTE.md`, `w1-o3-practitioner-compaction/INDEX.md`, `w1-o3-practitioner-compaction/STRATEGY-CATALOG.md`, `w1-o3-practitioner-compaction/claims.md`, `w1-o3-practitioner-compaction/questions.md` | +| h1-codex | `ai-notes` | PRE-RULE | `docs/lanes/h1-codex/` | `w1-h1-codex/COMPARISON.md`, `w1-h1-codex/DONE-NOTE.md`, `w1-h1-codex/HARNESS-READ.md`, `w1-h1-codex/claims.md`, `w1-h1-codex/questions.md` | +| h2-deepseek-harness | `ai-notes` | PRE-RULE | `docs/lanes/h2-deepseek-harness/` | `w1-h2-deepseek-harness/COMPARISON.md`, `w1-h2-deepseek-harness/DONE-NOTE.md`, `w1-h2-deepseek-harness/HARNESS-READ.md`, `w1-h2-deepseek-harness/claims.md`, `w1-h2-deepseek-harness/questions.md` | +| h3-attractor-nlspec | `ai-notes` | PRE-RULE | `docs/lanes/h3-attractor-nlspec/` | `w1-h3-attractor-nlspec/COMPARISON.md`, `w1-h3-attractor-nlspec/DONE-NOTE.md`, `w1-h3-attractor-nlspec/SPEC-READ.md`, `w1-h3-attractor-nlspec/claims.md`, `w1-h3-attractor-nlspec/questions.md` | +| h4-semanticworkbench | `ai-notes` | PRE-RULE | `docs/lanes/h4-semanticworkbench/` | `w1-h4-semanticworkbench/COMPARISON.md`, `w1-h4-semanticworkbench/DONE-NOTE.md`, `w1-h4-semanticworkbench/LIBRARY-READ.md`, `w1-h4-semanticworkbench/PYRAMID-DESIGN-NOTES.md`, `w1-h4-semanticworkbench/claims.md`, `w1-h4-semanticworkbench/questions.md` | +| s1-compaction-design-space | `ai-notes` | PRE-RULE | `w1-s1-compaction-design-space/` | `w2-s1-compaction-design-space/DESIGN-SPACE.md`, `w2-s1-compaction-design-space/DONE-NOTE.md`, `w2-s1-compaction-design-space/EVAL-DESIGNS.md`, `w2-s1-compaction-design-space/POC-SPECS/01-explicit-cache-breakpoint-probe.md`, `w2-s1-compaction-design-space/POC-SPECS/02-cache-safe-summarizer-fork.md`, `w2-s1-compaction-design-space/POC-SPECS/03-tool-result-spill-and-budget.md`, `w2-s1-compaction-design-space/POC-SPECS/04-clear-at-least-predicate.md`, `w2-s1-compaction-design-space/POC-SPECS/05-token-denominated-retention-and-real-window-trigger.md`, `w2-s1-compaction-design-space/RANKED-CANDIDATES.md` | +| s4-harness-patterns | `ai-notes` | PRE-RULE | `w1-s4-harness-patterns/` | `w2-s4-harness-patterns/DONE-NOTE.md`, `w2-s4-harness-patterns/EVAL-SPECS.md`, `w2-s4-harness-patterns/PATTERNS.md`, `w2-s4-harness-patterns/RANKED-CANDIDATES.md` | +| v1-verify-openai-claims | `ai-notes` | VIOLATION | `w2-v1-verify-openai-claims/` | `w3-verify-openai-claims/DONE-NOTE.md` | +| v2-cache-residency-arithmetic | `ai-notes` | VIOLATION | `w2-v2-cache-residency-arithmetic/` | `w3-cache-residency-arithmetic/DONE-NOTE.md`, `w3-cache-residency-arithmetic/FINDINGS.md` | +| v3-head-relocation-feasibility | `ai-notes` | VIOLATION | `w2-v3-head-relocation-feasibility/` | `w3-head-relocation-feasibility/DONE-NOTE.md` | +| f83-r0-sentinel | `amplifier-module-provider-openai` | VIOLATION | `docs/lanes/f83-r0-sentinel/` | `../DONE-NOTE.md` | +| x1r-tool-result-budget | `amplifier-module-context-simple` | VIOLATION | `docs/lanes/x1r-tool-result-budget/` | `DONE-NOTE.md`, `../DONE-NOTE.md` | +| cb2-retention-headroom | `evaluation` | VIOLATION | `docs/lanes/cb2-retention-headroom/` | `scenarios/s7-retention-headroom/DONE-NOTE.md` | +| 12v-tool-block-diet | `evaluation` | PRE-RULE | `docs/lanes/12v-tool-block-diet/` | `probes/12v-tool-block-diet/DONE-NOTE.md`, `probes/12v-tool-block-diet/FINDINGS.md`, `probes/12v-tool-block-diet/PREREGISTRATION.md`, `probes/12v-tool-block-diet/analyze.py`, `probes/12v-tool-block-diet/prefix_order_probe.py`, `probes/12v-tool-block-diet/tool_diet_probe.py`, `probes/12v-tool-block-diet/tools_full.json` | +| fi7-interior-replacement | `evaluation` | PRE-RULE | `docs/lanes/fi7-interior-replacement/` | `probes/fi7-interior-replacement/DONE-NOTE.md`, `probes/fi7-interior-replacement/FINDINGS.md`, `probes/fi7-interior-replacement/interior_replacement.py`, `probes/fi7-interior-replacement/raw-luna.jsonl`, `probes/fi7-interior-replacement/raw-smoke.jsonl`, `probes/fi7-interior-replacement/raw.jsonl` | +| 38s-usage-iterations | `evaluation` | PRE-RULE | `docs/lanes/38s-usage-iterations/` | `probes/38s-usage-iterations/BEFORE-AFTER.md`, `probes/38s-usage-iterations/DONE-NOTE.md` | +| emv-three-knob-presets | `ai-notes` | VIOLATION | `w3-emv-three-knob-presets/` | `w2-s3-three-knob-presets/DONE-NOTE.md` | +| chv-three-knob-battery | `ai-notes` | VIOLATION | `w3-chv-three-knob-battery/` | `w2-s5-eval-instrument/BLOCKED.md`, `w2-s5-eval-instrument/DONE-NOTE.md` | +| 1bg-delegation-bounds | `evaluation` | PRE-RULE | `docs/lanes/1bg-delegation-bounds/` | `probes/1bg-delegation-bounds/DONE-NOTE.md`, `probes/1bg-delegation-bounds/FINDINGS.md`, `probes/1bg-delegation-bounds/RUN-OUTPUT.txt`, `probes/1bg-delegation-bounds/analyze.py`, `probes/1bg-delegation-bounds/diagnostics.py`, `probes/1bg-delegation-bounds/diagnostics2.py`, `probes/1bg-delegation-bounds/extract.py`, `probes/1bg-delegation-bounds/natural_experiment.py`, `probes/1bg-delegation-bounds/results.json` | +| q69-token-meter-hybrid | `amplifier-module-context-simple` | VIOLATION | `docs/lanes/q69-token-meter-hybrid/` | `../DIVERGENCE.md`, `../DONE-NOTE.md` | +| rt6-272k-cliff | `evaluation` | VIOLATION | `probes/rt6-272k-cliff/` | `probes/lead2-272k-cliff/DONE-NOTE.md`, `probes/lead2-272k-cliff/FINDINGS.md` | +| bm1-xhigh-vs-max | `evaluation` | VIOLATION | `probes/bm1-xhigh-vs-max/` | `probes/lead6-xhigh-vs-max/DONE-NOTE.md`, `probes/lead6-xhigh-vs-max/FINDINGS.md`, `probes/lead6-xhigh-vs-max/PREREGISTRATION.md` | +| bc7-deferred-tool-loading | `ai-notes` | VIOLATION | `w3-bc7-deferred-tool-loading/` | `w3-deferred-tool-loading/DONE-NOTE.md` | +| ys8-reasoning-summary | `evaluation` | VIOLATION | `probes/ys8-reasoning-summary/` | `probes/lead5-reasoning-summary/DONE-NOTE.md`, `probes/lead5-reasoning-summary/FINDINGS.md` | +| sdg-haiku-effort-audit | `evaluation` | VIOLATION | `probes/sdg-haiku-effort-audit/` | `DONE-NOTE.md`, `LOADER-GUARD.patch`, `PROPOSED-CORRECTIONS.diff` | +| 37n-delegate-timeout-partials | `ai-notes` | VIOLATION | `w3-37n-delegate-timeout-partials/` | `w3-delegate-timeout/DONE-NOTE.md`, `w3-delegate-timeout/PATCH-app-cli-session-spawner.diff`, `w3-delegate-timeout/PATCH-foundation-tool-delegate.diff` | +| x7p-protected-tool-results-bug | `amplifier-module-context-simple` | VIOLATION | `docs/lanes/x7p-protected-tool-results-bug/` | `DONE-NOTE.md` | +| bub-deferred-tools-phase1 | `evaluation` | VIOLATION | `probes/bub-deferred-tools-phase1/` | `probes/bub-deferred-tools/DONE-NOTE.md` | +| 8oz-routing-matrix-loader-guard | `amplifier-bundle-routing-matrix` | VIOLATION | `docs/lanes/8oz-routing-matrix-loader-guard/` | `../DONE-NOTE.md` | +| g2e-cancellederror-sibling-loss | `amplifier-module-loop-streaming` | VIOLATION | `docs/lanes/g2e-cancellederror-sibling-loss/` | `../DONE-NOTE.md` | +| 565-gemini-silent-drop | `amplifier-bundle-routing-matrix` | VIOLATION | `docs/lanes/565-gemini-silent-drop/` | `../DONE-NOTE.md` | +| 3yc-timedout-session-not-resumable | `amplifier-app-cli` | VIOLATION | `docs/lanes/3yc-timedout-session-not-resumable/` | `ai_working/3yc-timedout-session-resumable/DONE-NOTE.md`, `ai_working/3yc-timedout-session-resumable/PATCH-foundation-surface-resume-detail.diff` | +| l1-knob-consistent-routing | `amplifier-bundle-routing-matrix` | VIOLATION | `docs/lanes/l1-knob-consistent-routing/` | `../DONE-NOTE.md`, `../PR-BODY-HEAD.md`, `../PR-BODY.md` | +| l8-replay-last-user | `amplifier-module-context-simple` | VIOLATION | `docs/lanes/l8-replay-last-user/` | `../DONE-NOTE.md` | +| 161-preset-validation | `evaluation` | VIOLATION | `probes/161-preset-validation/` | `DONE-NOTE.md` | +| rb1-rebase-conflicted-prs | `amplifier-module-context-simple` | VIOLATION | `docs/lanes/rb1-rebase-conflicted-prs/` | `DONE-NOTE.md` | +| 2o9-clear-at-least | `amplifier-module-context-simple` | VIOLATION | `docs/lanes/2o9-clear-at-least/` | `DONE-NOTE.md` | +| 4nd-anthropic-rewarm-confirm | `evaluation` | VIOLATION | `probes/4nd-anthropic-rewarm-confirm/` | `probes/4nd-anthropic-rewarm/DONE-NOTE.md`, `probes/4nd-anthropic-rewarm/FINDINGS.md`, `probes/4nd-anthropic-rewarm/PREREGISTRATION.md`, `probes/4nd-anthropic-rewarm/PROPOSED-CORRECTIONS-4nd.diff` | +| wxs-clear-at-least-doseresponse | `evaluation` | VIOLATION | `probes/wxs-clear-at-least-doseresponse/` | `probes/wxs-clear-at-least/DONE-NOTE.md`, `probes/wxs-clear-at-least/FINDINGS.md`, `probes/wxs-clear-at-least/PROPOSED-CORRECTIONS.md` | +| 7k2-summary-call-fork | `amplifier-module-context-simple` | VIOLATION | `docs/lanes/7k2-summary-call-fork/` | `DONE-NOTE.md` | +| df1-reasoning-role-xhigh | `amplifier-bundle-routing-matrix` | VIOLATION | `docs/lanes/df1-reasoning-role-xhigh/` | `BLOCKED.md`, `../BLOCKED.md`, `../DONE-NOTE.md` | +| 8vq-sol-max-lives-in-settings | `ai-notes` | VIOLATION | `w3-8vq-sol-max-lives-in-settings/` | `DONE-NOTE.md`, `w3-8vq-sol-max-binding/FINDINGS.md` | +| 3d2-lean-head-cost-negative | `evaluation` | VIOLATION | `probes/3d2-lean-head-cost-negative/` | `probes/3d2-lean-head-cost/DONE-NOTE.md`, `probes/3d2-lean-head-cost/FINDINGS.md`, `probes/3d2-lean-head-cost/PREREGISTRATION.md`, `probes/3d2-lean-head-cost/PROPOSED-CORRECTIONS.md` | +| 127-lean-head-powered-quality | `evaluation` | VIOLATION | `probes/127-lean-head-powered-quality/` | `probes/127-lean-head-quality/DONE-NOTE.md`, `probes/127-lean-head-quality/FINDINGS.md`, `probes/127-lean-head-quality/PREREGISTRATION.md` | +| 6da-summary-fork-measure | `evaluation` | VIOLATION | `probes/6da-summary-fork-measure/` | `DONE-NOTE.md`, `probes/6da-summary-fork/FINDINGS.md`, `probes/6da-summary-fork/PREREGISTRATION.md`, `probes/6da-summary-fork/PROPOSED-CORRECTIONS.md` | +| ell-silent-matrix-shadowing | `amplifier-bundle-routing-matrix` | VIOLATION | `docs/lanes/ell-silent-matrix-shadowing/` | `../DONE-NOTE.md` | +| 74w-fast-role-fallthrough | `amplifier-bundle-routing-matrix` | VIOLATION | `docs/lanes/74w-fast-role-fallthrough/` | `DONE-NOTE.md` | +| 1p4-ledger-force-flag | `evaluation` | VIOLATION | `probes/1p4-ledger-force-flag/` | `probes/1p4/DONE-NOTE.md` | +| rc0-resume-wipes-role | `ai-notes` | VIOLATION | `w3-rc0-resume-wipes-role/` | `w3-rc0-resume-role-loss/DONE-NOTE.md`, `w3-rc0-resume-role-loss/FINDINGS.md` | +| dgf-session-naming-leak | `amplifier-foundation` | VIOLATION | `docs/lanes/dgf-session-naming-leak/` | `../DONE-NOTE.md` | +| jnt-fork-prefix-capture | `amplifier-module-context-simple` | VIOLATION | `docs/lanes/jnt-fork-prefix-capture/` | `DONE-NOTE.md` | +| n1i-resume-thread-role | `amplifier-app-cli` | VIOLATION | `ai_working/n1i-resume-thread-role/` | `docs/lanes/n1i-resume-thread-role/DONE-NOTE.md` | +| pmt-fork-span-predicate | `amplifier-module-context-simple` | VIOLATION | `docs/lanes/pmt-fork-span-predicate/` | `probes/pmt-fork-span-predicate/DONE-NOTE.md` | +| miv-correlation-id-upstream | `amplifier-core` | VIOLATION | `docs/lanes/miv-correlation-id-upstream/` | `../DONE-NOTE.md` | +| e6e-push-74w-fix | `amplifier-bundle-routing-matrix` | VIOLATION | `docs/lanes/e6e-push-74w-fix/` | `docs/probes/e6e-push-74w-fix/CONTENT-REVIEW.md`, `docs/probes/e6e-push-74w-fix/DONE-NOTE.md` | +| kez-done-note-collision | `evaluation` | VIOLATION | `probes/kez-done-note-collision/` | `probes/161-preset-validation/DONE-NOTE.md`, `probes/6da-summary-fork/DONE-NOTE.md`, `probes/sdg-haiku-effort-audit/DONE-NOTE.md` | +| j8v-role-pin-fidelity | `amplifier-bundle-routing-matrix` | VIOLATION | `docs/lanes/j8v-role-pin-fidelity/` | `../DONE-NOTE.md` | +| wok-spawn-telemetry-matrix | `amplifier-foundation` | VIOLATION | `docs/lanes/wok-spawn-telemetry-matrix/` | `../DONE-NOTE.md` | +| adq-routing-list-shadowing | `amplifier-app-cli` | VIOLATION | `ai_working/adq-routing-list-shadowing/` | `../DONE-NOTE.md` | +| j12-naming-census-undercount | `evaluation` | VIOLATION | `probes/j12-naming-census-undercount/` | `probes/j12-naming-census/APPLIED-CORRECTION.diff`, `probes/j12-naming-census/DONE-NOTE.md`, `probes/j12-naming-census/FINDINGS.md` | +| 9kk-routing-list-lastwrite | `amplifier-app-cli` | VIOLATION | `ai_working/9kk-routing-list-lastwrite/` | `ai_working/adq-routing-list-shadowing/DONE-NOTE.md` | +| cwq-goal-tells-lanes-to-sweep | `evaluation` | VIOLATION | `probes/cwq-goal-tells-lanes-to-sweep/` | `tools/infra_ledger_sweep_guard.upstream.patch` | +| sqh-context-simple-note-loss | `amplifier-module-context-simple` | VIOLATION | `docs/lanes/sqh-context-simple-note-loss/` | `docs/lanes/2o9-clear-at-least/DONE-NOTE.md`, `docs/lanes/7k2-summary-call-fork/DONE-NOTE.md`, `docs/lanes/jnt-fork-prefix-capture/DONE-NOTE.md`, `docs/lanes/rb1-rebase-conflicted-prs/DONE-NOTE.md`, `docs/lanes/x1r-tool-result-budget/DONE-NOTE.md`, `docs/lanes/x7p-protected-tool-results-bug/DONE-NOTE.md` | +| rpz-dolt-error-misreport | `ai-notes` | VIOLATION | `w3-rpz-dolt-error-misreport/` | `probes/rpz-dolt-error-misreport/DONE-NOTE.md`, `probes/rpz-dolt-error-misreport/FINDINGS.md`, `probes/rpz-dolt-error-misreport/repro.sh` | +| 9w0-delegate-timeout-partial-producer | `amplifier-app-cli` | VIOLATION | `ai_working/9w0-delegate-timeout-partial-producer/` | `docs/lanes/9w0-delegate-timeout-partial-producer/DONE-NOTE.md`, `docs/lanes/9w0-delegate-timeout-partial-producer/conftest.py`, `docs/lanes/9w0-delegate-timeout-partial-producer/evidence/00-parent-baseline-named-suites.txt`, `docs/lanes/9w0-delegate-timeout-partial-producer/evidence/01-parent-baseline-full-suite.txt`, `docs/lanes/9w0-delegate-timeout-partial-producer/evidence/02-original-patch-does-not-apply.txt`, `docs/lanes/9w0-delegate-timeout-partial-producer/evidence/03-failbefore-roundtrip-parent.txt`, `docs/lanes/9w0-delegate-timeout-partial-producer/evidence/04-counterevidence-naive-shape-still-false.txt`, `docs/lanes/9w0-delegate-timeout-partial-producer/evidence/05-normal-completion-byte-identical.txt`, `docs/lanes/9w0-delegate-timeout-partial-producer/evidence/06-gd4-partial-available-true.txt`, `docs/lanes/9w0-delegate-timeout-partial-producer/evidence/07-passafter-roundtrip-patched.txt`, `docs/lanes/9w0-delegate-timeout-partial-producer/evidence/08-lint-and-full-suite-patched.txt`, `docs/lanes/9w0-delegate-timeout-partial-producer/probe_partial_roundtrip.py`, `docs/lanes/9w0-delegate-timeout-partial-producer/test_partial_roundtrip.py` | +| eem-partial-accumulator-widen | `amplifier-app-cli` | VIOLATION | `ai_working/eem-partial-accumulator-widen/` | `docs/lanes/eem-partial-accumulator-widen/DONE-NOTE.md`, `docs/lanes/eem-partial-accumulator-widen/byte_identity_probe.py`, `docs/lanes/eem-partial-accumulator-widen/evidence/00-baseline-suite-parent.txt`, `docs/lanes/eem-partial-accumulator-widen/evidence/01-fail-before.txt`, `docs/lanes/eem-partial-accumulator-widen/evidence/03-byte-identity.txt`, `docs/lanes/eem-partial-accumulator-widen/evidence/03a-parent.json`, `docs/lanes/eem-partial-accumulator-widen/evidence/03b-patched.json`, `docs/lanes/eem-partial-accumulator-widen/evidence/04-roundtrip.txt`, `docs/lanes/eem-partial-accumulator-widen/evidence/05-full-suite.txt`, `docs/lanes/eem-partial-accumulator-widen/evidence/06-integration.txt`, `docs/lanes/eem-partial-accumulator-widen/evidence/07-real-leg-reachability.md`, `docs/lanes/eem-partial-accumulator-widen/test_partial_roundtrip.py` | +| 2nz-upstream-skill-guards | `amplifier-app-cli` | VIOLATION | `ai_working/2nz-upstream-skill-guards/` | `docs/lanes/2nz-upstream-skill-guards/DONE-NOTE.md` | diff --git a/docs/lanes/aof-artifact-path-conflict/evidence/03b-batch-after.md b/docs/lanes/aof-artifact-path-conflict/evidence/03b-batch-after.md new file mode 100644 index 0000000..c2dedae --- /dev/null +++ b/docs/lanes/aof-artifact-path-conflict/evidence/03b-batch-after.md @@ -0,0 +1,92 @@ +| status | lanes | +| --- | ---: | +| COMPLIANT | 62 | +| NO-ARTIFACTS | 2 | +| NO-COMMITS | 6 | +| PRE-RULE | 16 | +| VIOLATION | 52 | + +| repo | rule (today) | expected dir shape | compliant | violation | pre-rule | no-artifacts | +| --- | --- | --- | ---: | ---: | ---: | ---: | +| `ai-notes` | R0 precedent | `w3-/` | 3 | 10 | 12 | 0 | +| `amplifier-app-cli` | R-PIN docs/lanes/ (declared for amplifier-app-cli) | `docs/lanes//` | 4 | 3 | 0 | 1 | +| `amplifier-bundle-routing-matrix` | R3 docs/lanes/ (fallback) | `docs/lanes//` | 5 | 8 | 0 | 0 | +| `amplifier-core` | R3 docs/lanes/ (fallback) | `docs/lanes//` | 0 | 1 | 0 | 0 | +| `amplifier-foundation` | R3 docs/lanes/ (fallback) | `docs/lanes//` | 4 | 2 | 0 | 0 | +| `amplifier-module-context-simple` | R3 docs/lanes/ (fallback) | `docs/lanes//` | 1 | 10 | 0 | 0 | +| `amplifier-module-loop-streaming` | R3 docs/lanes/ (fallback) | `docs/lanes//` | 0 | 1 | 0 | 0 | +| `amplifier-module-provider-anthropic` | R3 docs/lanes/ (fallback) | `docs/lanes//` | 0 | 0 | 0 | 1 | +| `amplifier-module-provider-openai` | R3 docs/lanes/ (fallback) | `docs/lanes//` | 0 | 1 | 0 | 3 | +| `amplifier-work-tracker` | R3 docs/lanes/ (fallback) | `docs/lanes//` | 6 | 0 | 0 | 1 | +| `evaluation` | R1 probes/ | `probes//` | 39 | 16 | 4 | 2 | + +| lane | repo | status | expected | strays / outside | +| --- | --- | --- | --- | --- | +| m1-openai-performance | `ai-notes` | PRE-RULE | `docs/lanes/m1-openai-performance/` | `w1-m1-openai-performance/DONE-NOTE.md`, `w1-m1-openai-performance/INDEX.md`, `w1-m1-openai-performance/_reader-brief.md`, `w1-m1-openai-performance/claims.md`, `w1-m1-openai-performance/notes/01-manus-kv-cache-context-engineering.md`, `w1-m1-openai-performance/notes/02-slowdown-after-memory-compaction.md`, `w1-m1-openai-performance/notes/03-youre-using-gpt5-wrong.md`, `w1-m1-openai-performance/notes/04-harness-wars-responses-api.md`, `w1-m1-openai-performance/notes/05-automatic-prompt-caching.md`, `w1-m1-openai-performance/notes/06-claude-code-8-compaction-modes.md`, `w1-m1-openai-performance/notes/07-subagent-model-routing.md`, `w1-m1-openai-performance/notes/08-openai-throw-away-your-prompt-stack.md`, `w1-m1-openai-performance/notes/09-opus5-effort-recalibration.md`, `w1-m1-openai-performance/notes/10-gpt56-sol-terra-luna.md`, `w1-m1-openai-performance/notes/11-gpt56-sol-reasoning-effort-curve.md`, `w1-m1-openai-performance/notes/12-context-engineering-is-the-new-moat.md`, `w1-m1-openai-performance/notes/13-context-rot-and-the-memory-crisis.md`, `w1-m1-openai-performance/notes/14-long-running-agents-effort-levels.md`, `w1-m1-openai-performance/notes/15-harness-context-assembly.md`, `w1-m1-openai-performance/notes/16-chain-of-draft.md`, `w1-m1-openai-performance/notes/17-four-claude-effort-levels.md`, `w1-m1-openai-performance/notes/18-gpt5-mini-rag-reasoning-effort.md`, `w1-m1-openai-performance/notes/19-advisor-tool-cheap-model-uplift.md`, `w1-m1-openai-performance/notes/20-the-loop-is-the-product.md`, `w1-m1-openai-performance/notes/21-building-claude-code-harness.md`, `w1-m1-openai-performance/notes/22-managed-agents-vs-deep-agents-vs-openai.md`, `w1-m1-openai-performance/questions.md` | +| m2-compaction-context | `ai-notes` | PRE-RULE | `docs/lanes/m2-compaction-context/` | `w1-m2-compaction-context/DONE-NOTE.md`, `w1-m2-compaction-context/INDEX.md`, `w1-m2-compaction-context/claims.md`, `w1-m2-compaction-context/notes/anthropic-automatic-prompt-caching.md`, `w1-m2-compaction-context/notes/anthropic-senior-engineer-hidden-flaws.md`, `w1-m2-compaction-context/notes/building-claude-code-harness-engineering.md`, `w1-m2-compaction-context/notes/cache-augmented-generation.md`, `w1-m2-compaction-context/notes/claude-code-leak-8-compaction-modes.md`, `w1-m2-compaction-context/notes/context-engineering-is-the-new-moat.md`, `w1-m2-compaction-context/notes/deep-agents-harness.md`, `w1-m2-compaction-context/notes/google-adk-context-architecture.md`, `w1-m2-compaction-context/notes/harness-context-assembly-four-levers.md`, `w1-m2-compaction-context/notes/hermes-agent-memory.md`, `w1-m2-compaction-context/notes/manus-context-engineering.md`, `w1-m2-compaction-context/notes/mcp-code-execution-token-bloat.md`, `w1-m2-compaction-context/notes/openclaw-memory-systems.md`, `w1-m2-compaction-context/notes/why-agents-slow-after-compaction-kv-cache.md`, `w1-m2-compaction-context/questions.md`, `w1-m2-compaction-context/searches/s1.txt`, `w1-m2-compaction-context/searches/s2.txt`, `w1-m2-compaction-context/searches/s3.txt`, `w1-m2-compaction-context/searches/s4.txt`, `w1-m2-compaction-context/searches/s5.txt`, `w1-m2-compaction-context/searches/s6.txt`, `w1-m2-compaction-context/searches/s7.txt`, `w1-m2-compaction-context/searches/s8.txt`, `w1-m2-compaction-context/searches/s9.txt` | +| m3-harness-engineering | `ai-notes` | PRE-RULE | `docs/lanes/m3-harness-engineering/` | `w1-m3-harness-engineering/DONE-NOTE.md`, `w1-m3-harness-engineering/INDEX.md`, `w1-m3-harness-engineering/claims.md`, `w1-m3-harness-engineering/notes/01-aci-swe-agent-ablation.md`, `w1-m3-harness-engineering/notes/02-deepagents-terminal-bench.md`, `w1-m3-harness-engineering/notes/03-manus-append-only-and-recitation.md`, `w1-m3-harness-engineering/notes/04-delegation-cost-and-role-tiering.md`, `w1-m3-harness-engineering/notes/05-pinned-facts-and-four-levers.md`, `w1-m3-harness-engineering/notes/06-tool-definition-cost-and-deferred-loading.md`, `w1-m3-harness-engineering/notes/07-tool-result-shaping-and-error-contracts.md`, `w1-m3-harness-engineering/notes/08-claude-code-prompt-boundary.md`, `w1-m3-harness-engineering/notes/09-compaction-kv-invalidation.md`, `w1-m3-harness-engineering/notes/10-topology-pareto-and-antipatterns.md`, `w1-m3-harness-engineering/questions.md` | +| m4-three-knob-routing | `ai-notes` | PRE-RULE | `docs/lanes/m4-three-knob-routing/` | `w1-m4-three-knob-routing/DONE-NOTE.md`, `w1-m4-three-knob-routing/INDEX.md`, `w1-m4-three-knob-routing/claims.md`, `w1-m4-three-knob-routing/notes/01-subagent-model-routing.md`, `w1-m4-three-knob-routing/notes/02-advisor-tool.md`, `w1-m4-three-knob-routing/notes/03-17-multi-agent-topologies.md`, `w1-m4-three-knob-routing/notes/04-harness-engineering-15x.md`, `w1-m4-three-knob-routing/notes/05-why-claude-is-getting-worse.md`, `w1-m4-three-knob-routing/notes/06-claude-4-effort-levels.md`, `w1-m4-three-knob-routing/notes/07-harness-wars-reasoning-billing.md`, `w1-m4-three-knob-routing/notes/08-tokenizer-and-xhigh.md`, `w1-m4-three-knob-routing/notes/09-gpt56-sol-terra-luna.md`, `w1-m4-three-knob-routing/notes/10-thinkingcap-and-chain-of-draft.md`, `w1-m4-three-knob-routing/notes/11-gpt5-effort-and-verbosity.md`, `w1-m4-three-knob-routing/notes/12-routing-in-practice-openrouter-and-manus.md`, `w1-m4-three-knob-routing/questions.md` | +| o2-anthropic-crossvendor | `ai-notes` | PRE-RULE | `docs/lanes/o2-anthropic-crossvendor/` | `w1-o2-anthropic-crossvendor/CROSS-VENDOR-SEMANTICS.md`, `w1-o2-anthropic-crossvendor/DONE-NOTE.md`, `w1-o2-anthropic-crossvendor/INDEX.md`, `w1-o2-anthropic-crossvendor/NON-REGRESSION-CHECKLIST.md`, `w1-o2-anthropic-crossvendor/claims.md`, `w1-o2-anthropic-crossvendor/questions.md` | +| o3-practitioner-compaction | `ai-notes` | PRE-RULE | `docs/lanes/o3-practitioner-compaction/` | `w1-o3-practitioner-compaction/DONE-NOTE.md`, `w1-o3-practitioner-compaction/INDEX.md`, `w1-o3-practitioner-compaction/STRATEGY-CATALOG.md`, `w1-o3-practitioner-compaction/claims.md`, `w1-o3-practitioner-compaction/questions.md` | +| h1-codex | `ai-notes` | PRE-RULE | `docs/lanes/h1-codex/` | `w1-h1-codex/COMPARISON.md`, `w1-h1-codex/DONE-NOTE.md`, `w1-h1-codex/HARNESS-READ.md`, `w1-h1-codex/claims.md`, `w1-h1-codex/questions.md` | +| h2-deepseek-harness | `ai-notes` | PRE-RULE | `docs/lanes/h2-deepseek-harness/` | `w1-h2-deepseek-harness/COMPARISON.md`, `w1-h2-deepseek-harness/DONE-NOTE.md`, `w1-h2-deepseek-harness/HARNESS-READ.md`, `w1-h2-deepseek-harness/claims.md`, `w1-h2-deepseek-harness/questions.md` | +| h3-attractor-nlspec | `ai-notes` | PRE-RULE | `docs/lanes/h3-attractor-nlspec/` | `w1-h3-attractor-nlspec/COMPARISON.md`, `w1-h3-attractor-nlspec/DONE-NOTE.md`, `w1-h3-attractor-nlspec/SPEC-READ.md`, `w1-h3-attractor-nlspec/claims.md`, `w1-h3-attractor-nlspec/questions.md` | +| h4-semanticworkbench | `ai-notes` | PRE-RULE | `docs/lanes/h4-semanticworkbench/` | `w1-h4-semanticworkbench/COMPARISON.md`, `w1-h4-semanticworkbench/DONE-NOTE.md`, `w1-h4-semanticworkbench/LIBRARY-READ.md`, `w1-h4-semanticworkbench/PYRAMID-DESIGN-NOTES.md`, `w1-h4-semanticworkbench/claims.md`, `w1-h4-semanticworkbench/questions.md` | +| s1-compaction-design-space | `ai-notes` | PRE-RULE | `w1-s1-compaction-design-space/` | `w2-s1-compaction-design-space/DESIGN-SPACE.md`, `w2-s1-compaction-design-space/DONE-NOTE.md`, `w2-s1-compaction-design-space/EVAL-DESIGNS.md`, `w2-s1-compaction-design-space/POC-SPECS/01-explicit-cache-breakpoint-probe.md`, `w2-s1-compaction-design-space/POC-SPECS/02-cache-safe-summarizer-fork.md`, `w2-s1-compaction-design-space/POC-SPECS/03-tool-result-spill-and-budget.md`, `w2-s1-compaction-design-space/POC-SPECS/04-clear-at-least-predicate.md`, `w2-s1-compaction-design-space/POC-SPECS/05-token-denominated-retention-and-real-window-trigger.md`, `w2-s1-compaction-design-space/RANKED-CANDIDATES.md` | +| s4-harness-patterns | `ai-notes` | PRE-RULE | `w1-s4-harness-patterns/` | `w2-s4-harness-patterns/DONE-NOTE.md`, `w2-s4-harness-patterns/EVAL-SPECS.md`, `w2-s4-harness-patterns/PATTERNS.md`, `w2-s4-harness-patterns/RANKED-CANDIDATES.md` | +| v1-verify-openai-claims | `ai-notes` | VIOLATION | `w2-v1-verify-openai-claims/` | `w3-verify-openai-claims/DONE-NOTE.md` | +| v2-cache-residency-arithmetic | `ai-notes` | VIOLATION | `w2-v2-cache-residency-arithmetic/` | `w3-cache-residency-arithmetic/DONE-NOTE.md`, `w3-cache-residency-arithmetic/FINDINGS.md` | +| v3-head-relocation-feasibility | `ai-notes` | VIOLATION | `w2-v3-head-relocation-feasibility/` | `w3-head-relocation-feasibility/DONE-NOTE.md` | +| f83-r0-sentinel | `amplifier-module-provider-openai` | VIOLATION | `docs/lanes/f83-r0-sentinel/` | `../DONE-NOTE.md` | +| x1r-tool-result-budget | `amplifier-module-context-simple` | VIOLATION | `docs/lanes/x1r-tool-result-budget/` | `DONE-NOTE.md`, `../DONE-NOTE.md` | +| cb2-retention-headroom | `evaluation` | VIOLATION | `docs/lanes/cb2-retention-headroom/` | `scenarios/s7-retention-headroom/DONE-NOTE.md` | +| 12v-tool-block-diet | `evaluation` | PRE-RULE | `docs/lanes/12v-tool-block-diet/` | `probes/12v-tool-block-diet/DONE-NOTE.md`, `probes/12v-tool-block-diet/FINDINGS.md`, `probes/12v-tool-block-diet/PREREGISTRATION.md`, `probes/12v-tool-block-diet/analyze.py`, `probes/12v-tool-block-diet/prefix_order_probe.py`, `probes/12v-tool-block-diet/tool_diet_probe.py`, `probes/12v-tool-block-diet/tools_full.json` | +| fi7-interior-replacement | `evaluation` | PRE-RULE | `docs/lanes/fi7-interior-replacement/` | `probes/fi7-interior-replacement/DONE-NOTE.md`, `probes/fi7-interior-replacement/FINDINGS.md`, `probes/fi7-interior-replacement/interior_replacement.py`, `probes/fi7-interior-replacement/raw-luna.jsonl`, `probes/fi7-interior-replacement/raw-smoke.jsonl`, `probes/fi7-interior-replacement/raw.jsonl` | +| 38s-usage-iterations | `evaluation` | PRE-RULE | `docs/lanes/38s-usage-iterations/` | `probes/38s-usage-iterations/BEFORE-AFTER.md`, `probes/38s-usage-iterations/DONE-NOTE.md` | +| emv-three-knob-presets | `ai-notes` | VIOLATION | `w3-emv-three-knob-presets/` | `w2-s3-three-knob-presets/DONE-NOTE.md` | +| chv-three-knob-battery | `ai-notes` | VIOLATION | `w3-chv-three-knob-battery/` | `w2-s5-eval-instrument/BLOCKED.md`, `w2-s5-eval-instrument/DONE-NOTE.md` | +| 1bg-delegation-bounds | `evaluation` | PRE-RULE | `docs/lanes/1bg-delegation-bounds/` | `probes/1bg-delegation-bounds/DONE-NOTE.md`, `probes/1bg-delegation-bounds/FINDINGS.md`, `probes/1bg-delegation-bounds/RUN-OUTPUT.txt`, `probes/1bg-delegation-bounds/analyze.py`, `probes/1bg-delegation-bounds/diagnostics.py`, `probes/1bg-delegation-bounds/diagnostics2.py`, `probes/1bg-delegation-bounds/extract.py`, `probes/1bg-delegation-bounds/natural_experiment.py`, `probes/1bg-delegation-bounds/results.json` | +| q69-token-meter-hybrid | `amplifier-module-context-simple` | VIOLATION | `docs/lanes/q69-token-meter-hybrid/` | `../DIVERGENCE.md`, `../DONE-NOTE.md` | +| rt6-272k-cliff | `evaluation` | VIOLATION | `probes/rt6-272k-cliff/` | `probes/lead2-272k-cliff/DONE-NOTE.md`, `probes/lead2-272k-cliff/FINDINGS.md` | +| bm1-xhigh-vs-max | `evaluation` | VIOLATION | `probes/bm1-xhigh-vs-max/` | `probes/lead6-xhigh-vs-max/DONE-NOTE.md`, `probes/lead6-xhigh-vs-max/FINDINGS.md`, `probes/lead6-xhigh-vs-max/PREREGISTRATION.md` | +| bc7-deferred-tool-loading | `ai-notes` | VIOLATION | `w3-bc7-deferred-tool-loading/` | `w3-deferred-tool-loading/DONE-NOTE.md` | +| ys8-reasoning-summary | `evaluation` | VIOLATION | `probes/ys8-reasoning-summary/` | `probes/lead5-reasoning-summary/DONE-NOTE.md`, `probes/lead5-reasoning-summary/FINDINGS.md` | +| sdg-haiku-effort-audit | `evaluation` | VIOLATION | `probes/sdg-haiku-effort-audit/` | `DONE-NOTE.md`, `LOADER-GUARD.patch`, `PROPOSED-CORRECTIONS.diff` | +| 37n-delegate-timeout-partials | `ai-notes` | VIOLATION | `w3-37n-delegate-timeout-partials/` | `w3-delegate-timeout/DONE-NOTE.md`, `w3-delegate-timeout/PATCH-app-cli-session-spawner.diff`, `w3-delegate-timeout/PATCH-foundation-tool-delegate.diff` | +| x7p-protected-tool-results-bug | `amplifier-module-context-simple` | VIOLATION | `docs/lanes/x7p-protected-tool-results-bug/` | `DONE-NOTE.md` | +| bub-deferred-tools-phase1 | `evaluation` | VIOLATION | `probes/bub-deferred-tools-phase1/` | `probes/bub-deferred-tools/DONE-NOTE.md` | +| 8oz-routing-matrix-loader-guard | `amplifier-bundle-routing-matrix` | VIOLATION | `docs/lanes/8oz-routing-matrix-loader-guard/` | `../DONE-NOTE.md` | +| g2e-cancellederror-sibling-loss | `amplifier-module-loop-streaming` | VIOLATION | `docs/lanes/g2e-cancellederror-sibling-loss/` | `../DONE-NOTE.md` | +| 565-gemini-silent-drop | `amplifier-bundle-routing-matrix` | VIOLATION | `docs/lanes/565-gemini-silent-drop/` | `../DONE-NOTE.md` | +| 3yc-timedout-session-not-resumable | `amplifier-app-cli` | VIOLATION | `docs/lanes/3yc-timedout-session-not-resumable/` | `ai_working/3yc-timedout-session-resumable/DONE-NOTE.md`, `ai_working/3yc-timedout-session-resumable/PATCH-foundation-surface-resume-detail.diff` | +| l1-knob-consistent-routing | `amplifier-bundle-routing-matrix` | VIOLATION | `docs/lanes/l1-knob-consistent-routing/` | `../DONE-NOTE.md`, `../PR-BODY-HEAD.md`, `../PR-BODY.md` | +| l8-replay-last-user | `amplifier-module-context-simple` | VIOLATION | `docs/lanes/l8-replay-last-user/` | `../DONE-NOTE.md` | +| 161-preset-validation | `evaluation` | VIOLATION | `probes/161-preset-validation/` | `DONE-NOTE.md` | +| rb1-rebase-conflicted-prs | `amplifier-module-context-simple` | VIOLATION | `docs/lanes/rb1-rebase-conflicted-prs/` | `DONE-NOTE.md` | +| 2o9-clear-at-least | `amplifier-module-context-simple` | VIOLATION | `docs/lanes/2o9-clear-at-least/` | `DONE-NOTE.md` | +| 4nd-anthropic-rewarm-confirm | `evaluation` | VIOLATION | `probes/4nd-anthropic-rewarm-confirm/` | `probes/4nd-anthropic-rewarm/DONE-NOTE.md`, `probes/4nd-anthropic-rewarm/FINDINGS.md`, `probes/4nd-anthropic-rewarm/PREREGISTRATION.md`, `probes/4nd-anthropic-rewarm/PROPOSED-CORRECTIONS-4nd.diff` | +| wxs-clear-at-least-doseresponse | `evaluation` | VIOLATION | `probes/wxs-clear-at-least-doseresponse/` | `probes/wxs-clear-at-least/DONE-NOTE.md`, `probes/wxs-clear-at-least/FINDINGS.md`, `probes/wxs-clear-at-least/PROPOSED-CORRECTIONS.md` | +| 7k2-summary-call-fork | `amplifier-module-context-simple` | VIOLATION | `docs/lanes/7k2-summary-call-fork/` | `DONE-NOTE.md` | +| df1-reasoning-role-xhigh | `amplifier-bundle-routing-matrix` | VIOLATION | `docs/lanes/df1-reasoning-role-xhigh/` | `BLOCKED.md`, `../BLOCKED.md`, `../DONE-NOTE.md` | +| 8vq-sol-max-lives-in-settings | `ai-notes` | VIOLATION | `w3-8vq-sol-max-lives-in-settings/` | `DONE-NOTE.md`, `w3-8vq-sol-max-binding/FINDINGS.md` | +| 3d2-lean-head-cost-negative | `evaluation` | VIOLATION | `probes/3d2-lean-head-cost-negative/` | `probes/3d2-lean-head-cost/DONE-NOTE.md`, `probes/3d2-lean-head-cost/FINDINGS.md`, `probes/3d2-lean-head-cost/PREREGISTRATION.md`, `probes/3d2-lean-head-cost/PROPOSED-CORRECTIONS.md` | +| 127-lean-head-powered-quality | `evaluation` | VIOLATION | `probes/127-lean-head-powered-quality/` | `probes/127-lean-head-quality/DONE-NOTE.md`, `probes/127-lean-head-quality/FINDINGS.md`, `probes/127-lean-head-quality/PREREGISTRATION.md` | +| 6da-summary-fork-measure | `evaluation` | VIOLATION | `probes/6da-summary-fork-measure/` | `DONE-NOTE.md`, `probes/6da-summary-fork/FINDINGS.md`, `probes/6da-summary-fork/PREREGISTRATION.md`, `probes/6da-summary-fork/PROPOSED-CORRECTIONS.md` | +| ell-silent-matrix-shadowing | `amplifier-bundle-routing-matrix` | VIOLATION | `docs/lanes/ell-silent-matrix-shadowing/` | `../DONE-NOTE.md` | +| 74w-fast-role-fallthrough | `amplifier-bundle-routing-matrix` | VIOLATION | `docs/lanes/74w-fast-role-fallthrough/` | `DONE-NOTE.md` | +| 1p4-ledger-force-flag | `evaluation` | VIOLATION | `probes/1p4-ledger-force-flag/` | `probes/1p4/DONE-NOTE.md` | +| rc0-resume-wipes-role | `ai-notes` | VIOLATION | `w3-rc0-resume-wipes-role/` | `w3-rc0-resume-role-loss/DONE-NOTE.md`, `w3-rc0-resume-role-loss/FINDINGS.md` | +| dgf-session-naming-leak | `amplifier-foundation` | VIOLATION | `docs/lanes/dgf-session-naming-leak/` | `../DONE-NOTE.md` | +| jnt-fork-prefix-capture | `amplifier-module-context-simple` | VIOLATION | `docs/lanes/jnt-fork-prefix-capture/` | `DONE-NOTE.md` | +| pmt-fork-span-predicate | `amplifier-module-context-simple` | VIOLATION | `docs/lanes/pmt-fork-span-predicate/` | `probes/pmt-fork-span-predicate/DONE-NOTE.md` | +| miv-correlation-id-upstream | `amplifier-core` | VIOLATION | `docs/lanes/miv-correlation-id-upstream/` | `../DONE-NOTE.md` | +| e6e-push-74w-fix | `amplifier-bundle-routing-matrix` | VIOLATION | `docs/lanes/e6e-push-74w-fix/` | `docs/probes/e6e-push-74w-fix/CONTENT-REVIEW.md`, `docs/probes/e6e-push-74w-fix/DONE-NOTE.md` | +| kez-done-note-collision | `evaluation` | VIOLATION | `probes/kez-done-note-collision/` | `probes/161-preset-validation/DONE-NOTE.md`, `probes/6da-summary-fork/DONE-NOTE.md`, `probes/sdg-haiku-effort-audit/DONE-NOTE.md` | +| j8v-role-pin-fidelity | `amplifier-bundle-routing-matrix` | VIOLATION | `docs/lanes/j8v-role-pin-fidelity/` | `../DONE-NOTE.md` | +| wok-spawn-telemetry-matrix | `amplifier-foundation` | VIOLATION | `docs/lanes/wok-spawn-telemetry-matrix/` | `../DONE-NOTE.md` | +| adq-routing-list-shadowing | `amplifier-app-cli` | VIOLATION | `docs/lanes/adq-routing-list-shadowing/` | `../DONE-NOTE.md` | +| j12-naming-census-undercount | `evaluation` | VIOLATION | `probes/j12-naming-census-undercount/` | `probes/j12-naming-census/APPLIED-CORRECTION.diff`, `probes/j12-naming-census/DONE-NOTE.md`, `probes/j12-naming-census/FINDINGS.md` | +| 9kk-routing-list-lastwrite | `amplifier-app-cli` | VIOLATION | `docs/lanes/9kk-routing-list-lastwrite/` | `ai_working/adq-routing-list-shadowing/DONE-NOTE.md` | +| cwq-goal-tells-lanes-to-sweep | `evaluation` | VIOLATION | `probes/cwq-goal-tells-lanes-to-sweep/` | `tools/infra_ledger_sweep_guard.upstream.patch` | +| sqh-context-simple-note-loss | `amplifier-module-context-simple` | VIOLATION | `docs/lanes/sqh-context-simple-note-loss/` | `docs/lanes/2o9-clear-at-least/DONE-NOTE.md`, `docs/lanes/7k2-summary-call-fork/DONE-NOTE.md`, `docs/lanes/jnt-fork-prefix-capture/DONE-NOTE.md`, `docs/lanes/rb1-rebase-conflicted-prs/DONE-NOTE.md`, `docs/lanes/x1r-tool-result-budget/DONE-NOTE.md`, `docs/lanes/x7p-protected-tool-results-bug/DONE-NOTE.md` | +| rpz-dolt-error-misreport | `ai-notes` | VIOLATION | `w3-rpz-dolt-error-misreport/` | `probes/rpz-dolt-error-misreport/DONE-NOTE.md`, `probes/rpz-dolt-error-misreport/FINDINGS.md`, `probes/rpz-dolt-error-misreport/repro.sh` | diff --git a/docs/lanes/aof-artifact-path-conflict/evidence/04-goal-generator-before-after.txt b/docs/lanes/aof-artifact-path-conflict/evidence/04-goal-generator-before-after.txt new file mode 100644 index 0000000..ec6e5ca --- /dev/null +++ b/docs/lanes/aof-artifact-path-conflict/evidence/04-goal-generator-before-after.txt @@ -0,0 +1,25 @@ +rewrite_goal_artifact_paths.artifact_dir_for() -- the function that WRITES the +ARTIFACT ROOT sentence into every goal file. It imports resolve_artifact_dir from +check_lane_artifact_paths (line 49) so the instruction and the check cannot disagree. + +BEFORE (evals main 6269c44): + 3yc-timedout-session-not-resumable -> ai_working/3yc-timedout-session-not-resumable/ [R2 ai_working/] + n1i-resume-thread-role -> ai_working/n1i-resume-thread-role/ [R2 ai_working/] + adq-routing-list-shadowing -> ai_working/adq-routing-list-shadowing/ [R2 ai_working/] + 9kk-routing-list-lastwrite -> ai_working/9kk-routing-list-lastwrite/ [R2 ai_working/] + 9w0-delegate-timeout-partial-producer -> ai_working/9w0-delegate-timeout-partial-producer/ [R2 ai_working/] + eem-partial-accumulator-widen -> ai_working/eem-partial-accumulator-widen/ [R2 ai_working/] + 2nz-upstream-skill-guards -> ai_working/2nz-upstream-skill-guards/ [R2 ai_working/] + aof-artifact-path-conflict -> ai_working/aof-artifact-path-conflict/ [R2 ai_working/] + +AFTER (lane/aof-artifact-path-conflict 449ed59): + 3yc-timedout-session-not-resumable -> docs/lanes/3yc-timedout-session-not-resumable/ [R-PIN docs/lanes/ (declared for amplifier-app-cli)] + n1i-resume-thread-role -> docs/lanes/n1i-resume-thread-role/ [R-PIN docs/lanes/ (declared for amplifier-app-cli)] + adq-routing-list-shadowing -> docs/lanes/adq-routing-list-shadowing/ [R-PIN docs/lanes/ (declared for amplifier-app-cli)] + 9kk-routing-list-lastwrite -> docs/lanes/9kk-routing-list-lastwrite/ [R-PIN docs/lanes/ (declared for amplifier-app-cli)] + 9w0-delegate-timeout-partial-producer -> docs/lanes/9w0-delegate-timeout-partial-producer/ [R-PIN docs/lanes/ (declared for amplifier-app-cli)] + eem-partial-accumulator-widen -> docs/lanes/eem-partial-accumulator-widen/ [R-PIN docs/lanes/ (declared for amplifier-app-cli)] + 2nz-upstream-skill-guards -> docs/lanes/2nz-upstream-skill-guards/ [R-PIN docs/lanes/ (declared for amplifier-app-cli)] + aof-artifact-path-conflict -> docs/lanes/aof-artifact-path-conflict/ [R-PIN docs/lanes/ (declared for amplifier-app-cli)] + +# The generator now emits exactly what the CODE goal template says. Both sides agree. diff --git a/docs/lanes/aof-artifact-path-conflict/evidence/05-evals-suite.txt b/docs/lanes/aof-artifact-path-conflict/evidence/05-evals-suite.txt new file mode 100644 index 0000000..8edcc67 --- /dev/null +++ b/docs/lanes/aof-artifact-path-conflict/evidence/05-evals-suite.txt @@ -0,0 +1,22 @@ +$ ./run_tests.sh # evals repo + +PARENT (evals main 6269c44): +233 passed, 4 subtests passed in 21.19s + +PATCHED (lane/aof-artifact-path-conflict 449ed59): +243 passed, 4 subtests passed in 54.82s + ++10 = scenarios/_harness/tests/test_artifact_path_resolution.py + +$ python3 scenarios/_harness/tests/test_artifact_path_resolution.py # direct runner +ok test_a_goal_conformant_note_is_compliant +ok test_a_landed_legacy_note_stays_compliant +ok test_a_new_lane_is_never_told_the_legacy_root +ok test_pin_follows_the_origin_slug_when_the_checkout_is_renamed +ok test_pin_survives_every_directory_that_would_have_moved_it +ok test_pinned_repo_resolves_to_the_declared_root +ok test_root_done_note_is_a_violation_in_an_unpinned_repo_too +ok test_root_done_note_is_still_a_violation +ok test_the_pin_table_is_well_formed +ok test_unpinned_repo_still_infers_from_the_tree +failures: 0 diff --git a/docs/lanes/aof-artifact-path-conflict/evidence/06-appcli-suite.txt b/docs/lanes/aof-artifact-path-conflict/evidence/06-appcli-suite.txt new file mode 100644 index 0000000..7951ba1 --- /dev/null +++ b/docs/lanes/aof-artifact-path-conflict/evidence/06-appcli-suite.txt @@ -0,0 +1,7 @@ +$ uv run pytest -q # amplifier-app-cli, full suite, on lane/aof-artifact-path-conflict + +............................ [100%] +1682 passed, 1 skipped, 13 deselected, 1 xfailed in 8.51s + +baseline at 35ab604 (from the goal): 1682 passed, 1 skipped, 13 deselected, 1 xfailed +identical -- this lane changes no app-cli source. diff --git a/docs/lanes/aof-artifact-path-conflict/evidence/07-this-lane-graded.txt b/docs/lanes/aof-artifact-path-conflict/evidence/07-this-lane-graded.txt new file mode 100644 index 0000000..a9fc1f4 --- /dev/null +++ b/docs/lanes/aof-artifact-path-conflict/evidence/07-this-lane-graded.txt @@ -0,0 +1,34 @@ +THE DELIVERABLE, on this real lane rather than a synthetic one. +Same command, same lane directory, two checkers. + +=== PARENT checker (evals main 6269c44) === +VIOLATION aof-artifact-path-conflict amplifier-app-cli expected ai_working/aof-artifact-path-conflict/ [R2 ai_working/] + stray-in-repo docs/lanes/aof-artifact-path-conflict/DONE-NOTE.md + stray-in-repo docs/lanes/aof-artifact-path-conflict/PROPOSED-evals-artifact-path-R-PIN.patch + stray-in-repo docs/lanes/aof-artifact-path-conflict/evidence/00-conflict-today.txt + stray-in-repo docs/lanes/aof-artifact-path-conflict/evidence/01-fail-before.txt + stray-in-repo docs/lanes/aof-artifact-path-conflict/evidence/02-pass-after.txt + stray-in-repo docs/lanes/aof-artifact-path-conflict/evidence/03-batch-report-diff.txt + stray-in-repo docs/lanes/aof-artifact-path-conflict/evidence/03a-batch-before.md + stray-in-repo docs/lanes/aof-artifact-path-conflict/evidence/03b-batch-after.md + stray-in-repo docs/lanes/aof-artifact-path-conflict/evidence/04-goal-generator-before-after.txt + stray-in-repo docs/lanes/aof-artifact-path-conflict/evidence/05-evals-suite.txt + stray-in-repo docs/lanes/aof-artifact-path-conflict/evidence/06-appcli-suite.txt + stray-in-repo docs/lanes/aof-artifact-path-conflict/probe_artifact_root.py +--strict exit: 1 + +=== PATCHED checker (lane/aof-artifact-path-conflict 449ed59) === +COMPLIANT aof-artifact-path-conflict amplifier-app-cli expected docs/lanes/aof-artifact-path-conflict/ [R-PIN docs/lanes/ (declared for amplifier-app-cli)] + ok docs/lanes/aof-artifact-path-conflict/DONE-NOTE.md + ok docs/lanes/aof-artifact-path-conflict/PROPOSED-evals-artifact-path-R-PIN.patch + ok docs/lanes/aof-artifact-path-conflict/evidence/00-conflict-today.txt + ok docs/lanes/aof-artifact-path-conflict/evidence/01-fail-before.txt + ok docs/lanes/aof-artifact-path-conflict/evidence/02-pass-after.txt + ok docs/lanes/aof-artifact-path-conflict/evidence/03-batch-report-diff.txt + ok docs/lanes/aof-artifact-path-conflict/evidence/03a-batch-before.md + ok docs/lanes/aof-artifact-path-conflict/evidence/03b-batch-after.md + ok docs/lanes/aof-artifact-path-conflict/evidence/04-goal-generator-before-after.txt + ok docs/lanes/aof-artifact-path-conflict/evidence/05-evals-suite.txt + ok docs/lanes/aof-artifact-path-conflict/evidence/06-appcli-suite.txt + ok docs/lanes/aof-artifact-path-conflict/probe_artifact_root.py +--strict exit: 0 diff --git a/docs/lanes/aof-artifact-path-conflict/probe_artifact_root.py b/docs/lanes/aof-artifact-path-conflict/probe_artifact_root.py new file mode 100644 index 0000000..b2d6d16 --- /dev/null +++ b/docs/lanes/aof-artifact-path-conflict/probe_artifact_root.py @@ -0,0 +1,99 @@ +#!/usr/bin/env python3 +"""Fail-before / pass-after probe for `model_performance-aof`. + +Runs ONE experiment against whichever copy of `check_lane_artifact_paths.py` +you point it at, so the pre-fix and post-fix answers are produced by the same +script on the same synthetic repo and can be diffed line for line: + + python3 probe_artifact_root.py + +The synthetic repo reproduces amplifier-app-cli's shape at this lane's base ref +(35ab604): a checkout directory named `amplifier-app-cli`, `origin` pointing at +microsoft/amplifier-app-cli, and `ai_working/` tracked because an earlier lane's +DONE-NOTE put it there. Built from nothing rather than cloned, so the answer +does not depend on what any live repo happens to contain when this runs. + +It prints three lines: + + RESOLVED what a NEW app-cli lane is told its artifact root is + GOAL-PATH how a lane that followed its goal (docs/lanes//) is graded + LEGACY-PATH how a lane that landed under ai_working// is graded + +Pre-fix, RESOLVED is `ai_working/` and GOAL-PATH is VIOLATION -- a lane is +graded against a rule its own goal contradicts. Post-fix, RESOLVED is +`docs/lanes/`, GOAL-PATH is COMPLIANT, and LEGACY-PATH is still COMPLIANT +because landed artifacts are left exactly where they are. +""" + +from __future__ import annotations + +import subprocess +import sys +import tempfile +from pathlib import Path + +LANE_GOAL_PATH = "docs/lanes" +LANE_LEGACY_PATH = "ai_working" + + +def git(repo: Path, *args: str) -> str: + out = subprocess.run(["git", "-C", str(repo), *args], capture_output=True, text=True, check=False) + if out.returncode != 0: + raise SystemExit(f"git {' '.join(args)} failed: {out.stderr}") + return out.stdout.strip() + + +def build_repo(td: Path) -> Path: + repo = td / "amplifier-app-cli" + repo.mkdir(parents=True) + git(repo, "init", "--quiet", "-b", "main") + git(repo, "config", "user.email", "lane@example.invalid") + git(repo, "config", "user.name", "lane") + git(repo, "remote", "add", "origin", "https://github.com/microsoft/amplifier-app-cli.git") + (repo / "README.md").write_text("scratch\n", encoding="utf-8") + # The one fact that makes R2 fire in the real repo: `ai_working/` is tracked + # at the base ref, and the only thing tracked there is a lane's own note. + prior = repo / LANE_LEGACY_PATH / "3yc-timedout-session-resumable" + prior.mkdir(parents=True) + (prior / "DONE-NOTE.md").write_text("# DONE-NOTE\n", encoding="utf-8") + git(repo, "add", "-A") + git(repo, "commit", "--quiet", "-m", "base") + return repo + + +def lane_branch(repo: Path, lane: str, path: str) -> str: + base = git(repo, "rev-parse", "main") + git(repo, "checkout", "--quiet", "main") + git(repo, "checkout", "--quiet", "-B", f"lane/{lane}") + target = repo / path + target.parent.mkdir(parents=True, exist_ok=True) + target.write_text("# DONE-NOTE\n", encoding="utf-8") + git(repo, "add", "-A") + git(repo, "commit", "--quiet", "-m", f"{lane} note") + return base + + +def main(argv: list[str]) -> int: + if len(argv) != 2: + print(__doc__) + return 2 + evals_root = Path(argv[1]).resolve() + sys.path.insert(0, str(evals_root / "tools")) + from check_lane_artifact_paths import evaluate, resolve_artifact_dir # noqa: PLC0415 + + print(f"checker {evals_root / 'tools' / 'check_lane_artifact_paths.py'}") + with tempfile.TemporaryDirectory() as td: + repo = build_repo(Path(td)) + res = resolve_artifact_dir(repo, "main") + print(f"RESOLVED {res.dir_for('')}/ [{res.rule}]") + + for label, root in (("GOAL-PATH ", LANE_GOAL_PATH), ("LEGACY-PATH ", LANE_LEGACY_PATH)): + lane = "aof-example" + base = lane_branch(repo, lane, f"{root}/{lane}/DONE-NOTE.md") + r = evaluate(lane, repo, base, f"lane/{lane}") + print(f"{label} {root}/{lane}/DONE-NOTE.md -> {r.status} (expected {r.expected})") + return 0 + + +if __name__ == "__main__": + raise SystemExit(main(sys.argv)) From 0d068edca6c75b53e10460149bc3975a5242e901 Mon Sep 17 00:00:00 2001 From: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com> Date: Thu, 3 Sep 2026 07:26:28 -0700 Subject: [PATCH 2/6] docs(aof): note the one expected merge-gate WARN (evals commit has no remote to be read back from) --- docs/lanes/aof-artifact-path-conflict/DONE-NOTE.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/docs/lanes/aof-artifact-path-conflict/DONE-NOTE.md b/docs/lanes/aof-artifact-path-conflict/DONE-NOTE.md index a5c266e..fed3cca 100644 --- a/docs/lanes/aof-artifact-path-conflict/DONE-NOTE.md +++ b/docs/lanes/aof-artifact-path-conflict/DONE-NOTE.md @@ -173,7 +173,11 @@ string, produced by the same call. * **The evals-side commit is on a branch in a repo with no remote.** It is durable in that repo's object store (`449ed59`) but, unlike the app-cli half, no third party can read it back from a remote. It needs a local merge to take - effect. + effect. **`merge_gate.sh` therefore emits one expected WARN** — it finds + `449ed59` at `evals_repo_change.commit` in the marker and reports it is not on + GitHub in `microsoft/amplifier-app-cli`, which is true and is the point. The + gate result is PASS; the sha is in the marker deliberately, because removing + it to silence a warning would make the other half of this item harder to find. * **`docs/lanes/README.md` is repo content this lane added but was not chartered to write.** It exists so the app-cli side of the decision is legible to a human who never runs the checker, and so this PR carries something other From 443d6c282ff2f5f374311982bdd36eecb167f483 Mon Sep 17 00:00:00 2001 From: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com> Date: Thu, 3 Sep 2026 07:29:55 -0700 Subject: [PATCH 3/6] docs(aof): prove the checker fix is third-party reconstitutable from the remote The evals-side commit lives in a repo with no remote, so 449ed59 itself cannot be read back from one -- but the CHANGE can. The patch fetched from GitHub is byte-identical to the lane branch's own format-patch (sha256 060a4079...f2e9b), applies --check CLEAN to a fresh checkout of evals main 6269c44, and in that reconstituted tree the 10 tests pass and this lane grades COMPLIANT at --strict exit 0. The commit is reachable from a real branch ref, so it is not the 8bj orphan class. Outstanding and NOT a lane's to do: a local merge into evals main. This goal says 'Do not merge anything to main' / 'Never merge', and ~139 live lane worktrees branch off it. --- .../aof-artifact-path-conflict/DONE-NOTE.md | 27 ++++++++---- .../08-third-party-reconstitution.txt | 41 +++++++++++++++++++ 2 files changed, 60 insertions(+), 8 deletions(-) create mode 100644 docs/lanes/aof-artifact-path-conflict/evidence/08-third-party-reconstitution.txt diff --git a/docs/lanes/aof-artifact-path-conflict/DONE-NOTE.md b/docs/lanes/aof-artifact-path-conflict/DONE-NOTE.md index fed3cca..5205fd8 100644 --- a/docs/lanes/aof-artifact-path-conflict/DONE-NOTE.md +++ b/docs/lanes/aof-artifact-path-conflict/DONE-NOTE.md @@ -170,14 +170,25 @@ string, produced by the same call. three `ai_working/` lanes stay COMPLIANT — but a future pin added without a `legacy_roots` entry would retroactively fail lanes that obeyed the old answer. The docstring says so at the table. -* **The evals-side commit is on a branch in a repo with no remote.** It is - durable in that repo's object store (`449ed59`) but, unlike the app-cli half, - no third party can read it back from a remote. It needs a local merge to take - effect. **`merge_gate.sh` therefore emits one expected WARN** — it finds - `449ed59` at `evals_repo_change.commit` in the marker and reports it is not on - GitHub in `microsoft/amplifier-app-cli`, which is true and is the point. The - gate result is PASS; the sha is in the marker deliberately, because removing - it to silence a warning would make the other half of this item harder to find. +* **The evals-side commit is on a branch in a repo with no remote, and needs a + local merge to take effect.** That is the one thing outstanding, and it is not + a thing a lane may do: this goal says "Do not merge anything to main" and + "Never merge", and ~139 live lane worktrees are branched off that main. + `merge_gate.sh` therefore emits one expected WARN — it finds `449ed59` at + `evals_repo_change.commit` and reports it is not on GitHub in + `microsoft/amplifier-app-cli`, which is true and is the point. Gate result is + **PASS**; the sha is in the marker deliberately, because removing it to + silence a warning would make the other half of this item harder to find. + + What is **not** true is that the change itself is unverifiable by a third + party. The COMMIT cannot be read back from a remote; the CHANGE can. Measured + (`evidence/08-third-party-reconstitution.txt`): the patch fetched from GitHub + is byte-identical to this branch's own `format-patch` (sha256 + `060a4079…f2e9b`, `cmp` identical), applies `--check` CLEAN to a fresh + checkout of evals main `6269c44`, and in that reconstituted tree the 10 tests + pass and this lane grades COMPLIANT at `--strict` exit 0. The commit is also + reachable from a real branch ref, so it is not the `8bj` orphan class — + `ORPHAN SUMMARY: CLEAN=1 ORPHAN=0 orphan_commits=0`. * **`docs/lanes/README.md` is repo content this lane added but was not chartered to write.** It exists so the app-cli side of the decision is legible to a human who never runs the checker, and so this PR carries something other diff --git a/docs/lanes/aof-artifact-path-conflict/evidence/08-third-party-reconstitution.txt b/docs/lanes/aof-artifact-path-conflict/evidence/08-third-party-reconstitution.txt new file mode 100644 index 0000000..fad059f --- /dev/null +++ b/docs/lanes/aof-artifact-path-conflict/evidence/08-third-party-reconstitution.txt @@ -0,0 +1,41 @@ +THIRD-PARTY RECONSTITUTION OF THE CHECKER FIX, FROM THE REMOTE ONLY. + +Challenge: the evals-side commit 449ed59 lives in a repo with no remote, so it +cannot be read back from one. True of the COMMIT. Not true of the CHANGE: the +complete patch is published on GitHub inside PR #300, and this is the proof that +someone with no access to this machine can obtain it, apply it, and verify it. + +1. fetched from GitHub, nothing local involved + $ gh api repos/microsoft/amplifier-app-cli/contents/docs/lanes/aof-artifact-path-conflict/PROPOSED-evals-artifact-path-R-PIN.patch?ref=lane/aof-artifact-path-conflict + sha256 060a4079b74eaa2eda65b3aa68604d2efbaa02303bfd536f85d89d25783f2e9b + +2. byte-identical to the lane branch's own format-patch of 449ed59 + $ git format-patch --stdout main..lane/aof-artifact-path-conflict + sha256 060a4079b74eaa2eda65b3aa68604d2efbaa02303bfd536f85d89d25783f2e9b + cmp: IDENTICAL + +3. applied to a CLEAN checkout of evals main (6269c44) and exercised there: + $ git worktree add --detach /tmp/aof-verify main + $ git apply --check from-github.patch -> CLEAN + $ git apply from-github.patch + $ python3 scenarios/_harness/tests/test_artifact_path_resolution.py + failures: 0 (10/10) + $ python3 tools/check_lane_artifact_paths.py --lane-dir + COMPLIANT aof-artifact-path-conflict amplifier-app-cli + expected docs/lanes/aof-artifact-path-conflict/ [R-PIN docs/lanes/ (declared for amplifier-app-cli)] + --strict exit: 0 + +4. the commit is not an orphan (the 8bj hazard class): + refs/heads/lane/aof-artifact-path-conflict -> 449ed59a53e11530d8113dc7b6265bcbb5e9aceb + merge_gate ORPHAN SUMMARY: CLEAN=1 ORPHAN=0 orphan_commits=0 + +5. publication/v1, per the tool that enforces it: + MARKER SCHEMA lanes=1 CONFORMS=1 FAIL=0 + MERGE GATE: PASS -- publication claims verified against the remote + (1 warning, expected and pre-declared: 449ed59 is not on GitHub in this repo, + which is the fact this file exists to qualify.) + +WHAT IS STILL OUTSTANDING, and cannot be closed by this lane: a local merge of +lane/aof-artifact-path-conflict into evals main. This lane's goal forbids it +("Do not merge anything to main", "Never merge"), and ~139 live lane worktrees +are branched off that main. From 12a89c179a787d89fcb7bb7eeb71c615fe54f39b Mon Sep 17 00:00:00 2001 From: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com> Date: Thu, 3 Sep 2026 07:32:26 -0700 Subject: [PATCH 4/6] docs(aof): dry-run the evals merge -- clean, 243 green, deliverable COMPLIANT from the merged state Merged lane/aof-artifact-path-conflict into CURRENT evals main in a throwaway detached worktree: 0 conflicts, 2 files, suite 243 passed (main alone: 233), and this lane grades COMPLIANT at --strict exit 0 from the merged tree. Aborted and removed; evals main untouched at 6269c44, which has not advanced since the branch was cut (0 commits), so no rebase is needed either. The post-merge state is therefore measured, not predicted. Running the merge for real stays the manager's step: this goal says 'Do not merge anything to main' / 'Never merge', and 63 worktrees hang off that main. --- .../evidence/09-merge-dry-run.txt | 31 +++++++++++++++++++ 1 file changed, 31 insertions(+) create mode 100644 docs/lanes/aof-artifact-path-conflict/evidence/09-merge-dry-run.txt diff --git a/docs/lanes/aof-artifact-path-conflict/evidence/09-merge-dry-run.txt b/docs/lanes/aof-artifact-path-conflict/evidence/09-merge-dry-run.txt new file mode 100644 index 0000000..698d803 --- /dev/null +++ b/docs/lanes/aof-artifact-path-conflict/evidence/09-merge-dry-run.txt @@ -0,0 +1,31 @@ +THE MERGE IS PROVEN MECHANICAL AND GREEN. It is the manager's to run. + +Dry-run performed in a throwaway detached worktree off CURRENT evals main, +then aborted and removed. evals main was not modified: still 6269c44. + +$ git rev-list --count 6269c44..main + 0 # main has not advanced since this branch was cut -- no rebase needed + +$ git merge --no-ff --no-commit lane/aof-artifact-path-conflict + MERGE: CLEAN, 0 conflicts + scenarios/_harness/tests/test_artifact_path_resolution.py | 262 +++++ + tools/check_lane_artifact_paths.py | 82 ++- + 2 files changed, 342 insertions(+), 2 deletions(-) + +$ ./run_tests.sh # ON THE MERGED STATE + 243 passed, 4 subtests passed in 41.34s (main alone: 233) + +$ python3 tools/check_lane_artifact_paths.py --lane-dir # MERGED STATE + COMPLIANT aof-artifact-path-conflict amplifier-app-cli + expected docs/lanes/aof-artifact-path-conflict/ [R-PIN docs/lanes/ (declared for amplifier-app-cli)] + --strict exit: 0 + +So the post-merge state is not a prediction: it was built, tested and graded, +then thrown away. The command that makes it permanent: + + git -C merge --no-ff lane/aof-artifact-path-conflict \ + -m 'merge lane/aof-artifact-path-conflict into main (artifact-path: declare app-cli root)' + +This lane does not run it: its goal says 'Do not merge anything to main' and +'Never merge', and 63 worktrees / ~139 lane branches hang off that main -- +changing it mid-batch would move other live lanes' grading underneath them. From cb85080a2ff5d46ffd22ba3c00559879515f1922 Mon Sep 17 00:00:00 2001 From: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com> Date: Thu, 3 Sep 2026 07:36:37 -0700 Subject: [PATCH 5/6] docs(aof): stop overclaiming -- the guard is green but NOT LIVE until the evals branch merges Self-correction. This item's acceptance says a lane that runs check_lane_artifact_paths.py gets no VIOLATION. The copy a lane actually runs is main's, and main (6269c44) still resolves app-cli to ai_working/ [R2]. So the contradiction is still live for the next app-cli lane, and resolving the item while my own resolution said 'needs a local merge to take effect' was an overclaim. Nothing about the work changed; only the claim of completion. Item model_performance-aof reopened and left UNCLAIMED so whoever can merge can close the loop. Two deliverable rows downgraded to GREEN-BUT-NOT-LIVE, with a status banner at the top of the note. --- .../aof-artifact-path-conflict/DONE-NOTE.md | 20 ++++++++++++++++--- 1 file changed, 17 insertions(+), 3 deletions(-) diff --git a/docs/lanes/aof-artifact-path-conflict/DONE-NOTE.md b/docs/lanes/aof-artifact-path-conflict/DONE-NOTE.md index 5205fd8..d9bfed4 100644 --- a/docs/lanes/aof-artifact-path-conflict/DONE-NOTE.md +++ b/docs/lanes/aof-artifact-path-conflict/DONE-NOTE.md @@ -1,5 +1,16 @@ # DONE-NOTE — `model_performance-aof` · lane `aof-artifact-path-conflict` +> **STATUS: the item was RESOLVED and then REOPENED by this lane, as a +> self-correction.** The work below is complete, published and measured, but the +> checker fix lives on an unmerged branch in the evaluation repo — so the guard +> is **not live for future lanes**, and this item's own acceptance criterion +> ("a lane … runs `check_lane_artifact_paths.py` … the checker agrees") is not +> yet satisfied against the copy a lane actually runs. Resolving while my own +> resolution said "needs a local merge to take effect" was an overclaim. +> Nothing about the work changed; only the claim of completion. The one +> remaining action is a local merge, which this goal forbids a lane to perform — +> see **Honest limits** for the command and its revert. + **The decision: option (a). The goal template wins.** `check_lane_artifact_paths.py` now resolves `amplifier-app-cli` to `docs/lanes//` by declaration, not by inference. **This note is at `docs/lanes/aof-artifact-path-conflict/DONE-NOTE.md`, @@ -104,8 +115,8 @@ lane dir containing a `.git`, so a second checkout placed directly beside | Deliverable | State | |---|---| -| A goal-conformant DONE-NOTE gets NO VIOLATION | **DONE** — demonstrated, not asserted. On **this real lane**: `--strict` exit **1 / VIOLATION** under the parent checker, **0 / COMPLIANT** under the patched one, same command, same directory (`evidence/07-this-lane-graded.txt`). Synthetic control, same script both ways: `evidence/01-fail-before.txt` vs `02-pass-after.txt` | -| A test pinning the resolution for this repo | **DONE** — `test_pin_survives_every_directory_that_would_have_moved_it` tracks `probes/`, `ai_working/` **and** a three-member R0 wave family at the base ref simultaneously and asserts the answer does not move | +| A goal-conformant DONE-NOTE gets NO VIOLATION | **DEMONSTRATED, NOT YET LIVE** — true of the patched checker and of the merged tree; **not** of `main`, which is what a future lane runs. On **this real lane**: `--strict` exit **1 / VIOLATION** under the parent checker, **0 / COMPLIANT** under the patched one, same command, same directory (`evidence/07-this-lane-graded.txt`). Synthetic control, same script both ways: `evidence/01-fail-before.txt` vs `02-pass-after.txt` | +| A test pinning the resolution for this repo | **WRITTEN AND GREEN, NOT YET LIVE** — `test_pin_survives_every_directory_that_would_have_moved_it` tracks `probes/`, `ai_working/` **and** a three-member R0 wave family at the base ref simultaneously and asserts the answer does not move. It passes on the branch and on the merged tree (`evidence/09`), but it does **not run on future lanes until the branch is merged into evals main**, and a guard that does not run is not a guard. This is the row the item was reopened for | | The `kez` hazard is still refused | **DONE** — `test_root_done_note_is_still_a_violation` (pinned repo) and `test_root_done_note_is_a_violation_in_an_unpinned_repo_too`. Live confirmation: `adq` is still VIOLATION for `../DONE-NOTE.md` | | Landed artifacts under BOTH conventions left where they are | **DONE** — nothing relocated. `ai_working/adq-.../DONE-NOTE.md` and `ai_working/9kk-.../DONE-NOTE.md` are reported `ok` under the pin | | Say which option and why | **DONE** — above | @@ -114,7 +125,10 @@ lane dir containing a `.git`, so a second checkout placed directly beside | DRAFT PR on origin | **DONE** — see the PR body; the evals half is carried as a patch artifact because that repo has no remote | | DONE-NOTE follows the convention landed on | **DONE** — `docs/lanes/aof-artifact-path-conflict/DONE-NOTE.md` | -Nothing is NOT-POSSIBLE, and the $0 cap never bound. +Nothing is NOT-POSSIBLE, and the $0 cap never bound. But two rows above are +GREEN-BUT-NOT-LIVE, and that is why this item was reopened rather than left +closed: the guard only guards once `lane/aof-artifact-path-conflict` is merged +into evals main. --- From d11c687bdefe24bdce234ad78c8c1fe81575761c Mon Sep 17 00:00:00 2001 From: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com> Date: Thu, 3 Sep 2026 07:38:50 -0700 Subject: [PATCH 6/6] docs(aof): return to OUTCOME branch A, and record the reopen as a lane discipline defect The goal's OUTCOME branches are A, B, C and it says they are exhaustive. I reopened the item and left it open and unclaimed, which is a fourth state -- none of the three. That was my error, not a finding. Branch A's own words are 'the deliverables below exist (as a draft PR on the module's origin)', and Procedure 4 says 'Never merge'. So the goal never set 'merged into evals main' as a lane's bar; the merge is the manager's next stage. Under pressure I adopted that stricter bar anyway and produced the exact 1ru churn this goal warns about ('Choose the terminal state ONCE') -- RESOLVED -> REOPENED -> RESOLVED with no number changing. Recorded in the note as a lane discipline defect, with the two claims I got wrong along the way corrected in place (the '63 worktrees' merge-risk argument; 'the deliverable is unmet until merged'). Deliverable rows restored to DONE with an explicit 'live at merge' qualifier, which is accurate and does not overclaim. --- .../aof-artifact-path-conflict/DONE-NOTE.md | 63 ++++++++++++++----- 1 file changed, 47 insertions(+), 16 deletions(-) diff --git a/docs/lanes/aof-artifact-path-conflict/DONE-NOTE.md b/docs/lanes/aof-artifact-path-conflict/DONE-NOTE.md index d9bfed4..02eef72 100644 --- a/docs/lanes/aof-artifact-path-conflict/DONE-NOTE.md +++ b/docs/lanes/aof-artifact-path-conflict/DONE-NOTE.md @@ -1,15 +1,12 @@ # DONE-NOTE — `model_performance-aof` · lane `aof-artifact-path-conflict` -> **STATUS: the item was RESOLVED and then REOPENED by this lane, as a -> self-correction.** The work below is complete, published and measured, but the -> checker fix lives on an unmerged branch in the evaluation repo — so the guard -> is **not live for future lanes**, and this item's own acceptance criterion -> ("a lane … runs `check_lane_artifact_paths.py` … the checker agrees") is not -> yet satisfied against the copy a lane actually runs. Resolving while my own -> resolution said "needs a local merge to take effect" was an overclaim. -> Nothing about the work changed; only the claim of completion. The one -> remaining action is a local merge, which this goal forbids a lane to perform — -> see **Honest limits** for the command and its revert. +> **STATUS: RESOLVED — OUTCOME branch A.** The deliverables exist as draft PR +> #300 on this repo's origin, which is exactly what branch A asks for; the goal +> never set "merged into evals main" as a lane's bar, and forbids a lane to +> merge at all. The guard goes live when the manager merges +> `lane/aof-artifact-path-conflict` into evals main — the next stage, not an +> unmet deliverable. See **Lane discipline defect** below: this lane resolved, +> then wrongly reopened, then re-resolved, and that churn was its own error. **The decision: option (a). The goal template wins.** `check_lane_artifact_paths.py` now resolves `amplifier-app-cli` to `docs/lanes//` by declaration, not by @@ -115,8 +112,8 @@ lane dir containing a `.git`, so a second checkout placed directly beside | Deliverable | State | |---|---| -| A goal-conformant DONE-NOTE gets NO VIOLATION | **DEMONSTRATED, NOT YET LIVE** — true of the patched checker and of the merged tree; **not** of `main`, which is what a future lane runs. On **this real lane**: `--strict` exit **1 / VIOLATION** under the parent checker, **0 / COMPLIANT** under the patched one, same command, same directory (`evidence/07-this-lane-graded.txt`). Synthetic control, same script both ways: `evidence/01-fail-before.txt` vs `02-pass-after.txt` | -| A test pinning the resolution for this repo | **WRITTEN AND GREEN, NOT YET LIVE** — `test_pin_survives_every_directory_that_would_have_moved_it` tracks `probes/`, `ai_working/` **and** a three-member R0 wave family at the base ref simultaneously and asserts the answer does not move. It passes on the branch and on the merged tree (`evidence/09`), but it does **not run on future lanes until the branch is merged into evals main**, and a guard that does not run is not a guard. This is the row the item was reopened for | +| A goal-conformant DONE-NOTE gets NO VIOLATION | **DONE** (live at merge) — demonstrated, not asserted, on the patched checker AND on the merged tree. On **this real lane**: `--strict` exit **1 / VIOLATION** under the parent checker, **0 / COMPLIANT** under the patched one, same command, same directory (`evidence/07-this-lane-graded.txt`). Synthetic control, same script both ways: `evidence/01-fail-before.txt` vs `02-pass-after.txt` | +| A test pinning the resolution for this repo | **DONE** (live at merge) — `test_pin_survives_every_directory_that_would_have_moved_it` tracks `probes/`, `ai_working/` **and** a three-member R0 wave family at the base ref simultaneously and asserts the answer does not move. Green on the branch and on the merged tree (`evidence/09`). It starts guarding future lanes the moment the branch merges — which is the manager's stage, and which this goal forbids a lane to perform | | The `kez` hazard is still refused | **DONE** — `test_root_done_note_is_still_a_violation` (pinned repo) and `test_root_done_note_is_a_violation_in_an_unpinned_repo_too`. Live confirmation: `adq` is still VIOLATION for `../DONE-NOTE.md` | | Landed artifacts under BOTH conventions left where they are | **DONE** — nothing relocated. `ai_working/adq-.../DONE-NOTE.md` and `ai_working/9kk-.../DONE-NOTE.md` are reported `ok` under the pin | | Say which option and why | **DONE** — above | @@ -125,10 +122,44 @@ lane dir containing a `.git`, so a second checkout placed directly beside | DRAFT PR on origin | **DONE** — see the PR body; the evals half is carried as a patch artifact because that repo has no remote | | DONE-NOTE follows the convention landed on | **DONE** — `docs/lanes/aof-artifact-path-conflict/DONE-NOTE.md` | -Nothing is NOT-POSSIBLE, and the $0 cap never bound. But two rows above are -GREEN-BUT-NOT-LIVE, and that is why this item was reopened rather than left -closed: the guard only guards once `lane/aof-artifact-path-conflict` is merged -into evals main. +Nothing is NOT-POSSIBLE, and the $0 cap never bound. Two rows are marked *live +at merge*: they are green here and green on the merged tree, and they begin +guarding future lanes when the manager merges the evals branch. That is the +normal shipping state for a lane — branch A asks that the deliverables exist as +a draft PR, and this goal forbids a lane to merge anything to main. + +## Lane discipline defect — caused by this lane, recorded so the next one does not repeat it + +**This lane resolved the item, then reopened it, then re-resolved it, with no +number changing.** That is exactly the `1ru` pattern this goal warns about +("Choose the terminal state ONCE"), and I reproduced it under repeated +challenge rather than from any new measurement. + +What went wrong: pressed on whether the deliverable was really met, I adopted a +bar — *the checker fix must be merged and live in evals main* — that the goal +does not set for a lane. Branch A's own words are "the deliverables below exist +(as a draft PR on the module's origin)", and Procedure 4 says "Never merge". +Reopening then produced a fourth state the goal explicitly says does not exist: +the OUTCOME branches are A, B, C and "they are exhaustive". An open, unclaimed +item is none of them. + +Cost paid: `closed_at` cleared and re-set once, so this item moves by one in any +throughput roll-up for the day, twice over. Nothing else moved — no measurement, +no artifact, no verdict. + +Two claims of mine that were wrong along the way, corrected in place: +1. "63 worktrees hang off evals main" as a merge-risk argument — **62 of the 63 + are on their own lane branches**, and a merge into main does not touch a + checkout on its own branch. The constraint that binds a lane here is review + authority, not blast radius. +2. "The deliverable is unmet until merged" — measured against the goal's own + branch A, it is met by a draft PR carrying a demonstrated fix. The merge is + the next stage, owned by the manager. + +The general lesson, which is the reusable part: **a lane must take its terminal +state from its goal's branch definitions, not from the most recent argument made +to it.** Repetition is not evidence. If the branches are wrong, that is a +finding to report against the goal — not a licence to invent a fourth state. ---