docs(lanes): settle this repo's lane-artifact root at docs/lanes/ (aof) - #300
Draft
Brian Krabach (bkrabach) wants to merge 6 commits into
Draft
docs(lanes): settle this repo's lane-artifact root at docs/lanes/ (aof)#300Brian Krabach (bkrabach) wants to merge 6 commits into
Brian Krabach (bkrabach) wants to merge 6 commits into
Conversation
Every lane working in this repo was graded against a rule its own goal contradicts. The CODE goal template names `docs/lanes/<lane>/`; the batch's artifact checker (`check_lane_artifact_paths.py`, artifact-path/v1) resolved this repo to `ai_working/<lane>/` under rule R2. Both have precedent at origin/main, so a lane that followed its authoritative spec was reported a VIOLATION -- and 2nz paid for it twice, writing its note to `ai_working/` on the checker's say-so and then rewriting it under `docs/lanes/` (c80e04e). Resolved as option (a): the goal template wins, and the checker was taught this repo's convention instead. R2 only ever fired here because a LANE ARTIFACT made `ai_working/` tracked -- the first tracked path there is `ai_working/3yc-.../DONE-NOTE.md` (0d93352), and at 35ab604 all five tracked paths under it are lane artifacts. The rule was reading its own output back as a repo convention. Meanwhile four consecutive merged lanes (n1i #292, 9w0 #297, eem #298, 2nz #299) placed their notes under `docs/lanes/`. This commit is the app-cli half: * docs/lanes/README.md -- the convention, stated for a human reader, including why the `ai_working/` notes stay exactly where they are * docs/lanes/aof-.../ -- this lane's record: the decision argued from the code, fail-before/pass-after evidence, and the evals-repo patch as a reviewable artifact The checker half is a separate commit in the evaluation repo (that repo has no remote), carried here verbatim as docs/lanes/aof-artifact-path-conflict/PROPOSED-evals-artifact-path-R-PIN.patch. No app-cli source changed: 1682 passed, 1 skipped, 13 deselected, 1 xfailed -- byte-identical to the 35ab604 baseline.
… remote to be read back from)
…the remote The evals-side commit lives in a repo with no remote, so 449ed59 itself cannot be read back from one -- but the CHANGE can. The patch fetched from GitHub is byte-identical to the lane branch's own format-patch (sha256 060a4079...f2e9b), applies --check CLEAN to a fresh checkout of evals main 6269c44, and in that reconstituted tree the 10 tests pass and this lane grades COMPLIANT at --strict exit 0. The commit is reachable from a real branch ref, so it is not the 8bj orphan class. Outstanding and NOT a lane's to do: a local merge into evals main. This goal says 'Do not merge anything to main' / 'Never merge', and ~139 live lane worktrees branch off it.
…OMPLIANT from the merged state Merged lane/aof-artifact-path-conflict into CURRENT evals main in a throwaway detached worktree: 0 conflicts, 2 files, suite 243 passed (main alone: 233), and this lane grades COMPLIANT at --strict exit 0 from the merged tree. Aborted and removed; evals main untouched at 6269c44, which has not advanced since the branch was cut (0 commits), so no rebase is needed either. The post-merge state is therefore measured, not predicted. Running the merge for real stays the manager's step: this goal says 'Do not merge anything to main' / 'Never merge', and 63 worktrees hang off that main.
… the evals branch merges Self-correction. This item's acceptance says a lane that runs check_lane_artifact_paths.py gets no VIOLATION. The copy a lane actually runs is main's, and main (6269c44) still resolves app-cli to ai_working/ [R2]. So the contradiction is still live for the next app-cli lane, and resolving the item while my own resolution said 'needs a local merge to take effect' was an overclaim. Nothing about the work changed; only the claim of completion. Item model_performance-aof reopened and left UNCLAIMED so whoever can merge can close the loop. Two deliverable rows downgraded to GREEN-BUT-NOT-LIVE, with a status banner at the top of the note.
…e discipline defect
The goal's OUTCOME branches are A, B, C and it says they are exhaustive. I
reopened the item and left it open and unclaimed, which is a fourth state --
none of the three. That was my error, not a finding.
Branch A's own words are 'the deliverables below exist (as a draft PR on the
module's origin)', and Procedure 4 says 'Never merge'. So the goal never set
'merged into evals main' as a lane's bar; the merge is the manager's next stage.
Under pressure I adopted that stricter bar anyway and produced the exact 1ru
churn this goal warns about ('Choose the terminal state ONCE') -- RESOLVED ->
REOPENED -> RESOLVED with no number changing.
Recorded in the note as a lane discipline defect, with the two claims I got
wrong along the way corrected in place (the '63 worktrees' merge-risk argument;
'the deliverable is unmet until merged'). Deliverable rows restored to DONE with
an explicit 'live at merge' qualifier, which is accurate and does not overclaim.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The problem
Every lane working in this repo has been graded against a rule its own goal contradicts.
docs/lanes/<lane>/DONE-NOTE.mdcheck_lane_artifact_paths.py(artifact-path/v1), rule[R2 ai_working/]ai_working/<lane>/Both have precedent at
origin/main, so it cannot be settled by looking at the repo. Flagged by lane2nzwhile landing #299 (35ab604) and confirmed by the manager as itemmodel_performance-aof.The cost is not tidiness. A checker that disagrees with the spec trains lanes to ignore the checker — and the same checker catches the real hazard, a repo-root
DONE-NOTE.mdsilently clobbering the previous lane's (itemkez).2nzpaid the churn twice: it wrote its note toai_working/on the checker's say-so, then deleted it and rewrote it underdocs/lanes/(c80e04e).The decision: option (a) — the goal template wins
(b) is not the cheap option it looks like. The two sides are not independent:
tools/rewrite_goal_artifact_paths.py:49importsresolve_artifact_dirfrom the checker and generates each goal's ARTIFACT ROOT sentence from it — deliberately, "so the instruction and the check can never disagree". Under (b) you would still have to change the resolver (so the generator emitsai_working/) and change the hand-written CODE goal template, which lives in the manager's per-batchgoals/directory: outside any git repo, with no test harness. The deliverable that stops the recurrence is a test, and under (b) it has nowhere to live.Two facts then decide which answer is right:
ai_working/isai_working/3yc-timedout-session-resumable/DONE-NOTE.md(0d93352). At35ab604, all five tracked paths underai_working/are lane artifacts; the repo tracks no content there at any ref, and.gitignorementions onlyai_working/tmp. The rule was reading its own past output back as a repo convention.docs/lanes/— n1i (fix(resume): narrow the credential refresh to secrets, and thread model_role/provider_preferences through delegate resume #292), 9w0 (feat(spawner): expose a timed-out sub-session's partial output via session.partial (k64 prereq 2/2, PRODUCER) #297), eem (fix(spawner): recover a timed-out delegate's thinking/tool_call work, not only its text #298), 2nz (fix(skills/ten-lane-highway): ship the two sweep guards a tool update kept reverting #299), four consecutive lanes, every one graded VIOLATION today.What changed
Evaluation repo — branch
lane/aof-artifact-path-conflict, commit449ed59, base6269c44. That repo has no remote, so the complete patch is committed here asdocs/lanes/aof-artifact-path-conflict/PROPOSED-evals-artifact-path-R-PIN.patch.A new R-PIN rule, checked before R0–R3:
Keyed on the checkout directory name and the
originURL slug, so a checkout cloned under another name still pins.legacy_rootsis retrospective only —dir_for()never returns one, so no new lane is sent toai_working/, while lanes that already landed there stay COMPLIANT and stay put.This repo —
docs/lanes/README.md(the same statement for a human reader) plus this lane's own record. No source changed.Fail-before / pass-after
This real lane, same command, same directory, two checkers (
evidence/07-this-lane-graded.txt):Synthetic control, one script pointed at each checker (
evidence/01,evidence/02):Both sides agree — at the generator, not just the checker
rewrite_goal_artifact_paths.artifact_dir_for()is the function that writes the ARTIFACT ROOT sentence into a goal file. For all 8 app-cli lanes in the manifest it returnedai_working/<lane>/before and returnsdocs/lanes/<lane>/after (evidence/04). The instruction a future lane is given and the rule it is graded against are now the same string, from the same call.Batch-wide effect: only this repo moves
--manifest <HW>/manifest.tsv --all --markdown, before vs after (evidence/03):amplifier-app-cli, beforeamplifier-app-cli, afterThe diff contains no other repo — every other repo's rule, expected shape and per-lane status are byte-identical. Pinned by
test_unpinned_repo_still_infers_from_the_tree.The 3 residual violations are pre-existing and unrelated to the root choice:
adqwrote../DONE-NOTE.mdoutside the checkout (thekezshape, correctly still refused);9kkwrote intoadq's directory;3yc's manifest lane id differs from its directory name (...-not-resumablevs...-resumable).Tests
scenarios/_harness/tests/test_artifact_path_resolution.py, 10 tests, evals suite 233 → 243 passed:test_pin_survives_every_directory_that_would_have_moved_it— the recurrence-stopper.probes/(R1),ai_working/(R2) and a three-memberw9-wave family (R0) all tracked at the base ref simultaneously — every inference the rule can make, each of which alone used to decide the answer — and the answer does not move.test_root_done_note_is_still_a_violation+test_root_done_note_is_a_violation_in_an_unpinned_repo_too—kez's hazard, pinned repo and unpinned repo alike. A fix that relaxed this to end the disagreement would have been strictly worse than the disagreement.test_a_landed_legacy_note_stays_compliant/test_a_new_lane_is_never_told_the_legacy_root— landed artifacts under both conventions stay exactly where they are; only what a new lane is told changes.test_unpinned_repo_still_infers_from_the_tree— R0–R3 unchanged for every repo outside the table.test_pin_follows_the_origin_slug_when_the_checkout_is_renamed,test_the_pin_table_is_well_formed,test_a_goal_conformant_note_is_compliant,test_pinned_repo_resolves_to_the_declared_root.Full suite (this repo)
Byte-identical to the
35ab604baseline — this lane changes no app-cli source (evidence/06).Spend
$0.00 against a $0.00 authority (
0 runs × 0 arms × $0 / 1.00 = $0.00). No API calls, no DTU, no infrastructure created or claimed. The cap never bound.Deviation and honest limits
amplifier-app-cliworktree, but both candidate fixes live in the evaluation repo. Writing only a note here would have left the conflict in place, so the change was made where it lives, on a lane branch in a git worktree, exactly as every evals lane in this batch does. Nothing was merged; that repo has no remote, hence the patch artifact.PRE-RULEcan no longer fire for a pinned repo (it triggers onexpected_now != expected, which a pin makes equal at every ref).legacy_rootscovers the same ground here, but a future pin added without one would retroactively fail lanes that obeyed the old answer. The docstring says so at the table.docs/lanes/README.mdis repo content this lane was not chartered to write. It exists so the decision is legible to a human who never runs the checker. Drop it if you disagree; nothing depends on it.Full reasoning, deliverable-by-deliverable:
docs/lanes/aof-artifact-path-conflict/DONE-NOTE.md.