From b1419abafa6d95eac52a0d8074eed65d418fd105 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20S=C3=A4nger?= <20968534+dsnger@users.noreply.github.com> Date: Mon, 17 Aug 2026 14:28:48 +0200 Subject: [PATCH 1/5] docs(intake): add field-intake-canvas-a1-a5-report story --- ...-field-intake-canvas-a1-a5-report-story.md | 91 +++++++++++++++++++ 1 file changed, 91 insertions(+) create mode 100644 docs/superpowers/stories/2026-08-17-field-intake-canvas-a1-a5-report-story.md diff --git a/docs/superpowers/stories/2026-08-17-field-intake-canvas-a1-a5-report-story.md b/docs/superpowers/stories/2026-08-17-field-intake-canvas-a1-a5-report-story.md new file mode 100644 index 0000000..c321e7a --- /dev/null +++ b/docs/superpowers/stories/2026-08-17-field-intake-canvas-a1-a5-report-story.md @@ -0,0 +1,91 @@ +# Field-intake round: the canvas A1–A5 field report — Story + +**Date:** 2026-08-17 · **Size:** story +**Risk:** standard · **Security:** none · **Validation:** battery+check + +## 1. Problem statement + +The kit's heaviest consumer — the infinite-portfolio-canvas repo — ran tickets A1–A5 +under this kit (~60+ Gate-B cycles, ~150 Gate-A/B passes, v0.8.0 since mid-run) and +relayed a verbatim field report: eleven numbered items across measured defects, policy +candidates and explicitly experimental proposals, plus one validation result. None of it +is in the kit's backlog today, and the report itself is a local file outside git history, +so nothing in the repo can cite it. + +This is the input the kit's own policy depends on: `todos.md` § Now says this repo +improves itself "reactively only. A change starts because a finding surfaced — through +the gates, a PR bot, or real use". Real use has now surfaced eleven, and there is no +recorded landing place for any of them. Separately, the parked `/capture-finding` row's +trigger — "the first production finding — one that arrives from real use rather than +from a gate or a bot" — has fired via this very report and is undisposed. + +## 2. Desired outcome + +Every item in the report reaches exactly one recorded disposition in this repo, each +grounded in evidence someone can re-read, and the report is citable from those records. +A disposition may be a rejection: the report's own terms make "verify at the cited +evidence, adopt nothing unverifiable, reject with a one-line reason" the standard, and a +reasoned rejection is as complete an outcome as an adoption. The round leaves the +consumer repo untouched, and leaves no item in the ambiguous state of having been read +but not answered. + +## 3. Acceptance criteria + +- [ ] The report is committed at `docs/field-reports/2026-08-16-canvas-a1-a5-field-report.md` + with every machine-local absolute path replaced by the repo's name — `grep -c '/Users/'` + on the committed file returns 0 — and the surrounding local-drafts exclusion still + holds for any other file in that directory. +- [ ] Each of the eleven numbered items carries **exactly one** disposition recorded in + the repo: a `todos.md` row with a named trigger, a `docs/hardening-log.md` row + appended via `dev-workflow:harden-finding`, a parked story, an upstream-candidate + note, or a one-line reasoned rejection. No item carries two; none carries none. +- [ ] Every adopted claim names the evidence it was verified against. Any claim whose + cited evidence could not be reached or did not support it is recorded as rejected + for that reason, not adopted with a caveat. +- [ ] Item 3 is recorded as an upstream candidate against `mcp-codex-dev` carrying its + three confirmed evidence instances, and no fix for it lands in this repo. +- [ ] Part 3's result (zero bytes moved across five surgery tranches under the untouched + core rules) appears as one line in the round's closure record and changes no + shipped file. +- [ ] The parked `/capture-finding` row carries a disposition — built, or re-parked with + the fired trigger and the evidence recorded against it — and that decision was put + to Daniel before it was written. +- [ ] Every ledger row this round appends states the guard-scope precheck's outcome + against the prior same-fingerprint row, quoting the guard it examined. +- [ ] The consumer repo `infinite-portfolio-canvas` has no file created, modified or + deleted by this round. + +## 4. Affected AGENTS.md invariants + +- `### Prompts and scaffolding` — "11. **Prompt changes pass `docs/prompt-standards.md`** — + all 12 checklist items, for any skill, command, agent definition, hook message, or + scaffolded template." +- `### Prompts and scaffolding` — "10. **The base taxonomy stays stack-neutral.** Project + vocabulary — tables, auth helpers, framework APIs — goes only in that project's + `docs/hardening-taxonomy.md`, never into the `harden-finding` skill. Otherwise one + project leaks into every other." (The rulings being codified were minted in one + consumer's vocabulary.) +- `### Packaging` — "12. **A plugin change requires a version bump.** A pull request that + changes any path under a `plugins//` directory *that still exists at HEAD* … must + also change that plugin manifest's `version`, or CI fails." +- `## Don'ts` — "**Never replace a decision procedure without accounting for its old + conditions.** List what the previous prose required, then mark each one kept, moved, or + deliberately dropped." +- `## Don'ts` — "**Never rename or delete a doc section without grepping for references + first.**" + +## 5. Open questions + +- `/capture-finding`: build now, or re-park with the fired trigger recorded? Its + deliverable is an extension of `harden-finding` (plugin scope, bump, full Gate B), so + the answer changes this round's size. To be put to Daniel with the evidence in hand, + not decided inside the round. +- Per item, for items 5, 6 and 8: does the codification land as prompt text in this + round, or does it pull toward real design and become a parked story? The rule is + settled; which side each item falls on is a finding of the verification step. + +## 6. Suggested size + +`story` — one coherent triage round with one input artifact and one output shape per +item; anything inside it that needs design gets parked as its own story rather than +growing this one. From 17d5ad3e9c0312235babe9880004b48168f0f54e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20S=C3=A4nger?= <20968534+dsnger@users.noreply.github.com> Date: Mon, 17 Aug 2026 14:29:52 +0200 Subject: [PATCH 2/5] docs: commit the canvas A1-A5 field report as the round's cited evidence Machine-local absolute paths neutralized to the repo's name (header and verbatim section), per the same sanitization discipline as the 0.8.0 fixture captures. docs/field-reports/ stays excluded locally for other drafts; this one file is force-added so the round's todos.md and ledger rows can cite a path that resolves for a later reader. Story: docs/superpowers/stories/2026-08-17-field-intake-canvas-a1-a5-report-story.md --- .../2026-08-16-canvas-a1-a5-field-report.md | 141 ++++++++++++++++++ 1 file changed, 141 insertions(+) create mode 100644 docs/field-reports/2026-08-16-canvas-a1-a5-field-report.md diff --git a/docs/field-reports/2026-08-16-canvas-a1-a5-field-report.md b/docs/field-reports/2026-08-16-canvas-a1-a5-field-report.md new file mode 100644 index 0000000..af47aa1 --- /dev/null +++ b/docs/field-reports/2026-08-16-canvas-a1-a5-field-report.md @@ -0,0 +1,141 @@ +# Field report — infinite-portfolio-canvas, tickets A1–A5 (received 2026-08-16) + +Verbatim report from the kit's heaviest consumer, relayed by Daniel. Source project: +the `infinite-portfolio-canvas` repo — ~60+ Gate-B +cycles, ~150 Gate-A/B passes, kit v0.8.0 since mid-run. To be processed as a +field-intake round after the reviewer-availability fallback story (0.9.0) lands. + +Machine-local absolute paths were replaced by the repo's name before committing, in +this header and in the verbatim section alike. Nothing else was altered. +Every claim must be verified at its cited evidence before adoption; rejection with a +one-line reason is a valid outcome per the report's own terms. + +Sparring pre-triage (2026-08-16, verified spot-check against the canvas repo): + +- Item 1 is NOT a hook defect — the canvas's own todos.md root-caused it later: + six >120 s review passes were auto-backgrounded, never reached the hook, class + `backgrounded`, counter structurally 0 without CLAUDE_CODE_MCP_AUTO_BACKGROUND_MS. + Item 1 collapses into item 4 → second independent field evidence for the open + workflow-init preflight-check row (env var), which is now ripe. +- Item 2 = occurrence fuel for the parked compound-commands / loose-grep rows + (incl. occurrence 4 of add+commit-in-one-call and a measured 5-pass loss from a + `git log` echo). Approaches the parse-trigger escalation; decides nothing alone. +- Item 3 (envelope swallows error.code on zero-credits) = upstream mcp-codex-dev + candidate with outsized value; matches the "no error code surfaced (unclassified)" + observed in the kit's own 0.8.x field use. +- Items 5, 6, 8 = codification candidates with cross-project evidence (absorb-vs-stop + ruling; arms-race remedy as procedure — the kit's own refrain, 4 canvas + applications; spec-size superlinearity — kit's own 23-pass supersession corroborates). +- Item 9 cheap and good. Items 10, 11 experimental — park with the future refrain row + (item 10 is corroborated by PR #23's 16-of-27-in-harness distribution). +- Part 3 (zero bytes moved across five surgery tranches under the untouched core + rules) = strongest field validation the kit has; also talk material. +- Meta: this report IS the fired trigger of the parked `/capture-finding` row + ("the first production finding arriving from real use rather than a gate or bot"). + +--- + +## Verbatim report + +Field report from the kit's heaviest consumer: the infinite-portfolio-canvas +repo (path neutralized — see header) +ran tickets A1-A5 under this kit — roughly 60+ Gate-B cycles, ~150 Gate-A/B +passes, kit v0.8.0 since mid-run. Everything below is OPTIONAL input for the +kit's own backlog: verify each claim against the referenced evidence in that +repo before adopting anything, adopt nothing you cannot verify, and add +nothing beyond this list on my account. If an item doesn't fit the kit's +design intent, dropping it is a valid outcome. + +PART 1 — MEASURED DEFECTS (reproduced in the field, evidence in the +consumer repo): + +1. Fingerprint computation/storage. Every clean cycle close draws a STOP; + counting and cycle-recognition work (measured: 24 counted passes, WIPs + and amends recognized), only the fingerprint fails. Narrowed cause: + codex-gate.gateB held literal "unavailable" and freshCount 0 at session + start — §5 reads 0 as hash-uncomputable, so no number of clean passes + can heal a cycle. The hook's own diagnostic checklist was green + (writable dirs, shasum present, disk free). Evidence: that repo's + todos.md § Tooling revalidation (three-part entry with measurements) + and multiple handbacks citing tree-identity overrides. + +2. Commit-detection by command-string grep, two measured failure shapes: + (a) a `git log` echo containing the word "commit" was scored as a + cycle-closing commit and reset a 5-pass counter to 0 — the loss is + indistinguishable from passes that never ran; (b) `git add … && git + commit …` in ONE Bash call defeats the docs-only exemption because + PreToolUse reads the still-empty index (two controlled data points); + (c) a heredoc mentioning an integration step next to a git call fired + a false STOP. Evidence: todos.md entries + the consumer's CLAUDE.md §5 + which now documents the workarounds (separate stage/commit calls, no + "commit"-word output mid-cycle). A more precise trigger (parse, or + PostToolUse for the exemption check) would remove a whole class of + operator discipline. + +3. Cross-repo note for mcp-codex-dev (if maintained alongside): error + envelopes swallow the reason — a zero-credits condition returned bare + {success:false, status:"error"} with no error.code; the cause was only + findable in ~/.codex/sessions rollout logs (rate_limits.credits). + Cost in the field: two spent retry budgets and a full false trail + (approval/registration). Surfacing NO_CREDITS & friends in the + envelope is a one-field fix with outsized value. + +4. Setup documentation: CLAUDE_CODE_MCP_AUTO_BACKGROUND_MS is load-bearing + for the hook (backgrounded calls never reach it → counter structurally + zero). The consumer discovered this by measurement; the kit's install/ + setup docs could name it as a required operator step (value 0 or above + the longest gate call, read at process start). + +PART 2 — POLICY CANDIDATES (rulings minted in the field; wording lives in +the consumer repo — .context/ notes, agent memory, taxonomy entries with +Belegstellen). Field-proven: + +5. Absorb-vs-stop ruling: corrections-of-corrections inside the assigned + fix set may be absorbed; any finding opening a NEW structural or + contract question stops the loop. "Size isn't the test, novelty of the + question is." (gate-b-absorb-vs-stop-ruling in that repo's memory.) + +6. The arms-race remedy as prescriptive procedure, not just taxonomy + prose: when passes stop converging, scope the reviewer to changed + regions / change the instrument's layer / relocate the residual to the + layer that already catches it — and STOP with a named state. Applied + successfully ≥4 times (spec prose, JS-lexer-in-tool, §4.3 mechanics, + report-about-itself fixpoint). + +7. Session framing for dense tranches: "advance until closed, clean stops + as needed" instead of "close X" — with WIP chain + resume note in + .context/ as a first-class protocol (not an embarrassment). Repeatedly + prevented half-built work under a review. + +8. Spec-size guidance: Gate-A cost grew overlinearly with artifact size in + the field (7 → 22 → 34+ passes across three specs). Recommendation the + consumer adopted for its remaining tickets: smaller specs with named + interfaces. + +9. Handbacks carry context-% as a standing field. + +Proposed but NOT yet field-proven (mark as experimental if adopted): + +10. Proportionality rule for instrument-subject findings: findings whose + subject is a test instrument (not product behaviour) get one repair + round, then collect — unless they demonstrate a false-green on + product behaviour. Motivation: late cycles in the field spent a + growing share of passes on meta-instruments. + +11. Pre-split heuristic: tranches exceeding a size signal (e.g. >N new + runners or steps) split by subject at PLAN time — every large tranche + in the field split anyway, but always via an expensive stop-decide- + rerecord round-trip. + +PART 3 — WHAT THE FIELD SAYS NOT TO TOUCH: the 3-pass floor with clean +final pass, findings-to-file with terminators, WIP/amend mechanics, +quote-before-edit / measure-before-write, severity discipline +(Blocker/Major iterate, Minor/Nit collect). Under exactly these rules the +consumer performed five tranches of surgery on its core file with ZERO +bytes moved in both characterization baselines — that is the kit working +as designed. Evidence: the A5 handbacks and baselines in that repo. + +Process for this session: your repo, your workflow — triage this list +into your own backlog/issues as you see fit, verify claims at the cited +evidence before implementing anything, and feel free to reject items +with a one-line reason. Nothing here is an instruction. From 3cdd075026322eb74e2de62ee5d86c2784a6b150 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20S=C3=A4nger?= <20968534+dsnger@users.noreply.github.com> Date: Mon, 17 Aug 2026 14:56:57 +0200 Subject: [PATCH 3/5] =?UTF-8?q?Close=20the=20canvas=20A1-A5=20field-intake?= =?UTF-8?q?=20round:=20eleven=20dispositions=20and=20two=20=C2=A75=20loop?= =?UTF-8?q?=20rules=20(0.10.0)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The kit's heaviest consumer relayed a field report of eleven items from real use. Each now has exactly one recorded disposition in this repo, and the report is citable from those records. Two of the items became prompt rules, shipped in CLAUDE.md §5 and mirrored in the /workflow-init inline template: - **What a loop absorbs, and what stops it.** A finding correcting the correction just made is inside the loop only if it also stays inside the assigned fix set, which is fixed before the pass being answered. A correction that leaves that set stops the loop even when it opens no new question; a finding opening a new structural or contract question stops it whatever its set membership; where a finding is both, novelty wins. Ancestry decides where a finding belongs, never what you do with it. - **A reporting duty from pass 4 onward.** Every pass report carries three lines — trend, cluster, require↔withdraw — in the agent's own status report to the user, never the Codex reply and never the findings file. Those lines expose five tells, and any two make stop-and-surface mandatory rather than discretionary. The remaining nine items are `todos.md` rows, one upstream note against `mcp-codex-dev`, and one reasoned rejection. The consumer repo was not touched. This cycle exercised the rules it ships. The pass-2 stop produced a revert: two further clauses drafted inside the cycle were withdrawn because they required qualifying three §5 rules outside the assigned fix set. The pass-5 stop produced a bounded close. Both were maintainer decisions, neither the loop's. The evidence behind both, and the three contract questions they leave open, are recorded for a review-economics story. Gate B: 7 valid passes, final clean (spec 2 Minor, quality none; no Blocker or Major since pass 2). Findings 14, 24, 12, 3, 6, 6, 2. Blockers 3, 4, 0, 0, 0, 0, 0. Collected, not iterated on: no literal example of the three-line format in either copy; no `Target model:` line in the root prompt or the scaffolded CLAUDE template; `docs/field-reports/` absent from the AGENTS.md architecture tree; numeric line citations in two closed plan artifacts; the dispositions file's title date; append-as-analogy wording in the malformed-entries section; the stuck-availability phrasing in the evidence entry below; and `todos.md` item 9's stop-path examples. Evidence: docs/superpowers/stories/2026-08-17-field-intake-canvas-a1-a5-report-story.md Battery: green — shellcheck clean on all six shell files; hook suite passed under both sh and dash; check-invariants 148 assertions; check-version-bump 36 assertions plus the live run against main; claude plugin validate . --strict passed. Check — named verification, no automated route: two tables under "The named verification behind the `battery+check` entry" in docs/field-reports/2026-08-16-canvas-a1-a5-dispositions.md. They split by the inputs they read, not one per shipped rule: the nine-state matrix scores scope and action for the absorb-vs-stop rule and, at states 8 and 9, the three conditions of the "clearly stuck" exit; the five-state table beside it scores the pass-4-onward reporting duty on pass number, tell count, carrier, and stuck-reading availability — inputs the first table does not read, which is why deleting that duty would have left all nine of its results unchanged. Both are scored against the old text and the new text, and against the CLAUDE.md copy and the /workflow-init template copy independently. No automated test can adjudicate whether prose decides anything; two weaker checks were tried and rejected before these — a keyword grep of the prior file (a self-supplied lexical oracle, since the keywords came from the new prose) and a paragraph diff plus mirror-parity comparison (which would pass equally if both copies omitted the same clause). Counterfactual, observed against the committed prior state. Matrix: on the seven states where scope or the stuck exit is genuinely open, the old text decides 0 of 7 and the new text 7 of 7. Reporting-duty table: on the four states where anything is required, the old text decides 0 of 4 and the new text 4 of 4. In both the old text fails by being ABSENT — `git show 17d5ad3:CLAUDE.md` and `git show 17d5ad3:plugins/dev-workflow/commands/workflow-init.md` contain no absorb-vs-stop block, no stuck criterion, and no reporting duty, tells or threshold; they carry only the undefined phrase "clearly stuck". Matrix states 6 and 7 and reporting state R1 are controls the old text already decides, where the new text must agree rather than override. Named failure modes the second table would catch: deleting the duty flips R2-R5; raising its threshold from two tells to three flips R4 and R5; moving the three lines out of the status report flips the carrier column. What neither table covers, stated in the record itself: which of the five tell predicates produced a given tell count. Three false framings of this counterfactual were caught by this cycle's own gate and removed — that the parent defined the assigned fix set inclusively (pass 1), that the parent was contradictory on a two-tell state (pass 2), and that the tables split one per shipped rule (pass 6). Wiring: every old-text state is scored from the committed parent blob via `git show`, never from the working tree, so the check cannot report success by reading the file the change already edited. Both copies are scored separately, so the counterfactual for "the rule reached only one copy" is a row where the two disagree; there is none. Scope note: two clauses drafted inside this cycle — a clean-completion precedence over the two-tell stop, and a defined exit for a declined scope expansion — were reverted on the maintainer's decision rather than shipped, because Gate-B pass 2 showed they require qualifying three §5 rules outside the assigned fix set. Four further pass-5 findings were parked on the same authority. Both records are in .context/codex-reviews/gate-b-fic2-parked-review-economics.md. --- CLAUDE.md | 85 +++++ .../2026-08-16-canvas-a1-a5-dispositions.md | 312 ++++++++++++++++++ docs/hardening-log.md | 15 + ...rden-finding-guard-scope-precheck-story.md | 14 +- ...-17-arms-race-remedy-as-procedure-story.md | 131 ++++++++ ...-field-intake-canvas-a1-a5-report-story.md | 17 +- .../dev-workflow/.claude-plugin/plugin.json | 2 +- plugins/dev-workflow/CHANGELOG.md | 57 ++++ .../dev-workflow/commands/workflow-init.md | 74 +++++ todos.md | 232 ++++++++++++- 10 files changed, 930 insertions(+), 9 deletions(-) create mode 100644 docs/field-reports/2026-08-16-canvas-a1-a5-dispositions.md create mode 100644 docs/superpowers/stories/2026-08-17-arms-race-remedy-as-procedure-story.md diff --git a/CLAUDE.md b/CLAUDE.md index 9c1c1f9..c78de2c 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -79,6 +79,91 @@ clean or clearly stuck → then STOP and surface to the user. The only early exi below 3 is a pass with **zero** findings; don't manufacture findings to pad. Codex is advisory — validate before applying; dismissed finding → one-line why. +**What a loop absorbs, and what stops it — a question of scope, not of action.** A finding +that corrects the correction you just made **and stays inside the assigned fix set** is +**inside this loop's scope**: keep it here rather than handing it back, then act on it by its +severity exactly as Mechanics already says — Blocker/Major resolve, Minor/Nit collect and +never iterate. Ancestry decides where a finding belongs; it never decides what you do with +it, and it grants no Minor or Nit a repair round it would not otherwise get. **The assigned fix set is fixed before the pass you are answering: it is the +scope the approved story or plan assigns to this cycle, plus repair obligations you already +accepted in earlier passes.** A finding is in-set when repairing it stays inside that scope — +never merely because it arrived in the current pass, which would put every new finding in the +set by definition and leave the boundary deciding nothing. Where membership is genuinely +unclear treat the finding as **outside**, which costs a question and never a silent expansion. **A correction that leaves that set stops the +loop like any other out-of-scope finding**, even when it opens no new question at all — +absorbing it would grow the assigned work without anyone agreeing to that — and it resumes +the moment the user says whether the set now includes it. A finding +that opens a **new structural or contract question** stops the loop and goes to the user — +**size is not the test, novelty of the question is**, so a structural finding that is +genuinely small still stops it, while a long correction still aimed at the last correction +does not — provided that correction, too, stays inside the set, which its ancestry never +supplies on its own. **When a finding is both** — it corrects the last correction *and* opens a new +structural or contract question — **the new question wins and the loop stops**: novelty +overrides correction ancestry, because absorbing on ancestry is exactly how a contract +decision gets made without anyone choosing it. Stopping this way is **not an exit from the gate**: the floor, the +Blocker/Major filter and the clean-final-pass rule all stand, and the loop resumes on the +revised artifact once the question is answered — what the stop prevents is a loop +committing you to a design you never chose, which is a different failure from an +unfinished review. (Field-minted in `infinite-portfolio-canvas` and carried here because +the alternative was observed there: handing back a three-line repair-of-a-repair wastes a +session, and absorbing a contract question spends a decision that was not the loop's to +make.) + +**Recognizing "clearly stuck", so that exit is a reading and not a mood.** Read the +**Blocker curve across passes**, not any single pass's total — it is the better of the two +signals, the total says less than it looks like, and one low count is a snapshot rather +than a plateau. **Neither curve measures coverage:** a low Blocker count can sit beside an +entirely unreviewed subsystem. So this exit needs three things **together**, and a missing +one means keep going: a plateau visible across passes (six or more is where the field saw +one); an **affirmative judgement that coverage is sufficient**, stated — a known materially +unreviewed area forbids this exit outright, and disclosing it does not license it; and +**Blocker or Major findings that keep regenerating across genuine repair attempts**, each +round's fix producing the next. That third condition is what makes a plateau rather than a +finish, and it is why **a clean completion takes precedence over this exit**: a +Blocker/Major-free pass **at or above the floor** has satisfied the clean-final-pass rule — +collect the Minors and Nits and close — and reporting "will not converge" on a converged +loop is a false report. **Below the floor nothing closes**, and a zero-finding pass remains +the only exception, exactly as above; a Blocker/Major-free pass 1 carrying a Minor keeps +looping. +**Surfacing does not close the cycle, and that is what makes this reachable.** You surface +*with the finding still open* — the resolve rule is not waived, no pass is credited as +clean, and the loop resumes on whatever the user decides. Reading it as "stop instead of +fixing" would put the exit in competition with the rule that every Blocker and Major +resolves, and then nothing could satisfy both. +**From pass 4 onward every pass report carries three lines.** The carrier is **your own +status report to the user** — never the Codex reply, which stays exactly one line per branch, +and never the findings file, which admits no line that is not a finding or the terminator. +They are cheap because the numbers already exist: (1) the **trend** — findings and Blocker counts across the passes +so far; (2) where this pass's findings **cluster** — product behaviour, the test instrument, +or prose about either; (3) any **require↔withdraw pair** against earlier passes, meaning a +pass demanding what an earlier pass had removed. + +Those three lines expose **five tells**: the finding count rising rather than falling; the +Blocker count failing to fall; findings clustering on the **instrument** rather than on +product behaviour; findings clustering on **prose about** either; and a require↔withdraw +pair. **Any two present makes stop-and-surface mandatory, not discretionary** — you report +the tells and hand the decision to the user, and the "clearly stuck" reading above is not a +precondition for it. A loop can be worth stopping long before it plateaus. + +Recorded rationale, from the maintainer rather than from a measurement of this repo: in the +Bricks consumer all five signals were measurable by **day two** of a week-long loop, and the +cost was never detection — it was the absence of a duty to say so. That is why this is a +reporting obligation with a mandatory threshold and not another heuristic to weigh. + +**The two rules above do not compete**, and neither overrides the other: the absorb rule +decides whether *a finding* is inside this loop's scope, this reading decides whether *the +loop* can still converge. A small correction-of-a-correction that stays inside the assigned fix +set is absorbed and is not by itself evidence of a plateau. The field measurement behind it, quoted at the +precision its own record keeps: nineteen Gate-A passes over successive revisions of one +design spec past 2800 lines (the exact size is not part of that evidence), findings from 43 +into a 2–19 range after pass 6 and never zero, Blockers from 11 to 0–1 from pass 7 on — and +the late Blockers were semantic contradictions rather than wording, which is why a low +count is a signal to read and not a clearance. That a round regenerates roughly half the +findings it closes is a **hypothesis** in that record rather than a measurement; one +lineage was established (the last pass's Blocker came from the previous pass's fix). Hence +the sizing guidance: prefer **smaller specs with named interfaces** and let the plan carry +the detail — guidance, not a threshold, because where the plateau starts is unmeasured. + **Findings go to a FILE, not the response — both gates.** In the field, long finding lists came back cut off on effectively every substantial Gate A pass, and a cut that lands between findings is indistinguishable from a short list: silently dropped diff --git a/docs/field-reports/2026-08-16-canvas-a1-a5-dispositions.md b/docs/field-reports/2026-08-16-canvas-a1-a5-dispositions.md new file mode 100644 index 0000000..7a4a1c1 --- /dev/null +++ b/docs/field-reports/2026-08-16-canvas-a1-a5-dispositions.md @@ -0,0 +1,312 @@ +# Dispositions — canvas A1–A5 field report (round closed 2026-08-17) + +Input: `2026-08-16-canvas-a1-a5-field-report.md`. Story: +`docs/superpowers/stories/2026-08-17-field-intake-canvas-a1-a5-report-story.md`. + +Honoring the report's own terms: every claim was verified at its cited evidence before adoption, +nothing unverifiable was adopted, and a rejection with a one-line reason is a complete outcome. + +**The consumer repo was read-only throughout**, and the claim is stated at the precision it can be +held: every command this round issued against it was a read (`git log`, `git status`, `git diff`, +`ls`, `grep`, `sed -n`, `cat`), and no write, stage or commit was issued. Its `index.html` does +carry an uncommitted modification (49 insertions, 4 deletions) whose mtime falls inside this +session's window — that is the consumer's own in-flight A5/T3a work, consistent with the four +`WIP: A5/T3a` commits at its HEAD, and it is named here rather than glossed because "the repo is +unchanged" would be a stronger claim than a read-only round can make about a tree someone else is +working in. + +**One disposition per item, and the eleven are exhaustive.** Where a disposition lives elsewhere, +this table points at it rather than restating it; the one rejection and the validation result +live here because they have no other home. + +| # | Subject | Disposition | Where | +|---|---|---|---| +| 1 | Fingerprint computation/storage | `todos.md` row, § Parked | "Two consumer Gate-B cycles closed with no usable fingerprint persisted" | +| 2 | Commit-detection by command-string grep | `todos.md` row, § Parked | "A mere mention of \"commit\" beside a `git` command…" | +| 3 | `mcp-codex-dev` envelope swallows the reason | Upstream-candidate note, § Next | filed against `mcp-codex-dev`, not fixed here | +| 4 | `CLAUDE_CODE_MCP_AUTO_BACKGROUND_MS` in setup docs | Second field evidence appended | the `/workflow-init` preflight row, § Next | +| 5 | Absorb-vs-stop ruling | `docs/hardening-log.md` row via `harden-finding` | `prompt-missing-stop-condition`, `P std` — the rung it records is the §5 + template edit | +| 6 | Arms-race remedy as procedure | Parked as a story | `docs/superpowers/stories/2026-08-17-arms-race-remedy-as-procedure-story.md`, indexed by its `todos.md` row | +| 7 | Session framing for dense tranches | **Rejected** — see below | this file | +| 8 | Spec-size / stuck-criterion | `docs/hardening-log.md` row via `harden-finding` | `prompt-vague-criteria`, `P std` — same §5 + template edit | +| 9 | Handbacks carry context-% | `todos.md` row, § Parked | "A handback says nothing about how much context produced it" | +| 10 | Proportionality for instrument-subject findings | `todos.md` row, § Parked, marked EXPERIMENTAL | trigger named | +| 11 | Pre-split heuristic | `todos.md` row, § Parked, marked EXPERIMENTAL | trigger named | + +**Item 8 carries an extension, added by the maintainer during the Gate-B cycle:** a **reporting +duty**. From pass 4 onward every pass report states three lines — the trend in findings and Blocker +counts across passes, where this pass's findings cluster (product behaviour, the test instrument, or +prose about either), and any require↔withdraw pair against earlier passes. Those expose five tells, +and **any two present makes stop-and-surface mandatory rather than discretionary**, with the stuck +reading explicitly not a precondition. Its recorded rationale is the maintainer's, not a measurement +of this repo: in another consumer all five signals were measurable by day two of a week-long loop, +and the cost was never detection but the absence of a duty to say so. It landed in the same §5 edit, +the same mirror and the same Gate-B cycle, and it is the same disposition as item 8 rather than a +twelfth item — the report's list is unchanged at eleven. + +The duty was exercised on this cycle immediately: pass 5 showed the finding count rising and its +findings clustering on prose about the instrument, and the Blocker count failing to fall — it rose +from 0 at pass 4 to 4 at pass 5. **Three** tells, against a threshold of two. What is *not* among +them is a require↔withdraw pair: pass 4 required clean-completion precedence and pass 5 +*qualified* it with the floor, where the tell is defined as a pass demanding what an earlier pass +had removed. Three tells made the stop mandatory rather than a judgement call. + +**Reading the table.** Items 5 and 8 share one §5 edit, one version bump and one Gate-B cycle: +they are two rules landing in two adjacent paragraphs of one §5 block (and its template mirror), +and splitting them would put two cycles on one diff. +Their disposition is the **ledger row** each received — the prompt codification is the rung that +row records, not a sixth disposition form. For item 6, the disposition is the **story**; its +`todos.md` row is that story's index entry, which is how every parked story in this repo is +tracked (Finding A, Finding B, tier-2, sequential-branch-calls all carry a row *and* a story), so +the pair is one disposition and not two owners. + +## The one rejection + +**Item 7 — session framing for dense tranches. Rejected on scope: the artifact half already +ships, and the framing half is one consumer's session habit with no recurrence behind it.** The +resume-note-and-WIP-chain half is `CLAUDE.md` §5's "Optional companions, from field practice" +paragraph, which sanctions a cycle-stable resume note as first-class and cites *this same +consumer* as the field practice it came from. What remains is the framing — opening a session with +"advance until closed, clean stops as needed" instead of "close X". That half is **not** rejected +for having no surface: a scaffolded `CLAUDE.md` is read as session instructions and could carry +it, so the surface exists. It is rejected on **transferability**: the framing is one project's +session-opening preference, adopted once and measured nowhere, and a framing rule that turns out +wrong is read and followed by every downstream project. No policy claim is made here — this repo's +reactive-only rule permits hardening anything real use surfaces and imposes no recurrence +threshold, as items 5 and 8 in this same round demonstrate by shipping on first occurrence. The +trigger below is therefore **chosen for this item**, not quoted from policy. *Reopens when:* a +second project adopts the framing independently, or a half-built tranche under review is traced to +its absence. + +## Part 3 — the validation result + +**Both characterization baselines are byte-unchanged across the five A5 tranche commits** +(`5a4bf38` T0, `50184c5` T1a, `4ccf7e4` T1b, `67be9d5` T2a, `f09286b` T2b): no commit touches +`test/baseline.json` or `test/a3-baseline.json` after `f41c545`, which precedes all five, and +`.context/a5-t2b-baseline-diff.md` records `BASELINE MATCHES` / `A3 BASELINE MATCHES` per run +without the recorder being invoked. + +That is the observation, and the line above is deliberately the whole of it. The report reads it +as proof that the pass floor, findings-to-file, WIP/amend mechanics, quote-before-edit and +severity discipline work as designed; this round does **not** adopt that inference. Unchanged +committed baselines establish the committed boundary bytes and nothing about which rule produced +them — the same history is compatible with any of those five mechanisms being broken and the +outcome surviving for another reason. It is a good result and it is not a causal one. No change +follows from it either way. + +## The ledger rows, and the guard-scope precheck that preceded them + +Items 5 and 8 were hardened through `dev-workflow:harden-finding` and appended two rows dated +2026-08-17, at rung `P std`, both `manual`/`major`: `prompt-missing-stop-condition` (nothing said +what a loop may absorb) and `prompt-vague-criteria` ("clearly stuck" named without a criterion). +Each row states the guard it installs **and** what that guard does not cover, so a later +recurrence can be judged rather than counted. + +The anchored column-2 grep returned **no match** for either class, so neither is a recurrence and +nothing escalates. The precheck was still run manually against the nearest candidates, quoting the +guard examined in each case: + +- `prompt-diagnostic-cause-unnamed` (2026-07-18), guard `docs/prompt-standards.md` item 10, filed + for a preflight that "reported \"restart the session\" for every not-loaded codex, hiding a + same-named server winning on scope precedence". The guard, quoted: "**Diagnostic states name + their causes.** A prompt that reports a failure state (\"NOT LOADED\", \"MISSING\", + \"unavailable\") enumerates the distinct causes that produce that state, gives a check that tells + them apart, and pairs each with its own fix." Result: **outside**. A loop's exit criterion is not + a failure state a prompt reports, so that rung never claimed this shape, did not fail, and + supplies no escalation. +- `mechanical-check-skipped-before-review` (2026-08-04), whose guard reads "before each read pass, + settle mechanically what the artifact asserts and a machine can decide without side effects — + cited paths, quoted passages, stated counts, the syntax of standalone fenced blocks". That + governs what a pass settles **before** reading. Result: **outside** — when the loop stops is not + something a machine can decide without side effects, which is the guard's whole domain. +- `rewrite-drops-prior-condition` (2026-07-27) is not a candidate for recurrence — no finding here + matches it — but its guard governs *this* change, since the §5 edit amends a decision procedure. + The guard, quoted from that row's `ref`: "AGENTS.md Don'ts, \"Never replace a decision procedure + without accounting for its old conditions\" — list what the previous prose required and mark + each kept, moved, or deliberately dropped; also check the paragraphs around any criterion you + amend". Result: **in scope, and applied** — accounted for explicitly, the floor, the + Blocker/Major filter and the clean-final-pass rule are each named as **kept** in the shipped + sentence, because "stops the loop" without them reads as a licensed below-floor close. + +**One observation, recorded rather than acted on.** `todos.md` says the 2026-08-16 severity-enum +closure classified itself `prompt-vague-criteria`, rung `P std` — and no such row exists in the +ledger, which is why today's row is the first of that class. So a hardening was applied and never +reached the ledger, on a change that *did* go through a PR. That is one more instance of the gap +the Finding A story owns, and it is left as an observation here: appending a row for someone +else's closed change after the fact would put a date on it that its own cycle never had. + +## The named verification behind the `battery+check` entry + +The story's mode is `battery+check`, and for a change that is *only* prompt text no automated +test can adjudicate whether the prose decides anything. Two weaker checks were tried and rejected +by the gate: a keyword grep of the prior file (pass 1 — a self-supplied lexical oracle, since the +keywords came from the new prose), then a paragraph diff plus mirror-parity comparison (pass 2 — +which establishes that nothing was silently replaced and that the copies agree, but would pass +equally if both copies omitted the same clause). What follows replaces them. + +**Method.** Nine review states are put to the *old* text (`17d5ad3:CLAUDE.md` §5) and the *new* +text (HEAD), and then independently to the `/workflow-init` inline template at both revisions, +since a rule can be present in one copy and absent from the other. Each state carries **complete +inputs** — floor status, set membership, ancestry, severity, and for a candidate stop all three +exit conditions — with **one** expected output per question. + +Two questions are scored separately: **scope** (is this finding inside this loop, or does it stop +the loop?) and **action** (resolve, or collect?). The old text already decided *action* by +severity and this change does not touch that rule, so action is expected to agree everywhere — +that agreement is the check, not a score. Rows where a question does not arise are `n/a` and are +not scored. + +| # | Floor | In set | Ancestry | Severity | Plateau | Coverage sufficient | B/M regenerating | Old → scope/exit | New → scope/exit | Action | +|---|---|---|---|---|---|---|---|---|---|---| +| 1 | met | yes | corrects last correction | MAJOR | n/a | n/a | n/a | undecided | in scope, keep it here | resolve | +| 2 | met | yes | corrects last correction | MINOR | n/a | n/a | n/a | undecided | in scope, keep it here | collect — ancestry grants no repair round | +| 3 | met | **no** | corrects last correction | MAJOR | n/a | n/a | n/a | undecided | **stops the loop** — out of set, though it opens no new question | resolve once the set is settled | +| 4 | met | n/a | opens a contract question the tranche never owned | MAJOR | n/a | n/a | n/a | undecided | stops the loop, surface; not a gate exit | resolve once answered | +| 5 | met | yes | both — corrects last correction *and* opens a new question | MAJOR | n/a | n/a | n/a | undecided | stops the loop; novelty overrides ancestry | as case 4 | +| 6 | **met** | yes | corrects last correction | MINOR only, no B/M open | pass came back B/M-free | yes | no | **decided already**: clean final pass → close | same — clean completion takes precedence | collect | +| 7 | **below** | yes | corrects last correction | MINOR only, no B/M open | n/a | yes | no | **decided already**: only a zero-finding pass exits below the floor → keep looping | same, and now said explicitly | collect | +| 8 | met | yes | corrects last correction | MAJOR, a new one each round | yes | yes | **yes** | undecided — "clearly stuck" has no criterion | stuck reading available; surface with the finding open, cycle not closed | resolve | +| 9 | met | yes | corrects last correction | MAJOR, a new one each round | yes | **no** — coverage never measured, a subsystem possibly unreviewed (the consumer's `docs/hardening-taxonomy.md` states exactly this) | yes | undecided — and a plain reading stops | exit **forbidden**: no affirmative sufficient-coverage judgement, and a known unreviewed area disqualifies it | resolve; keep going | + +**Scored honestly, which is not 0-of-9.** On the seven rows where scope or the stuck exit is +genuinely open (1–5, 8, 9) the old text decides **0 of 7** and the new text **7 of 7**. Cases 6 and +7 are the control rows: the old text **already decides both**, via the clean-final-pass rule and the +floor's zero-finding exception, and the new text must agree rather than override — an earlier draft +did not, which is what makes them worth keeping. On **action** both texts agree in all nine rows. + +**The template scores identically** at both revisions on all nine rows, which is the point of +running it twice: the counterfactual for "the rule reached only one copy" is a row where the two +disagree, and there is none. + +Cases 6 and 8 are the pair that carries the most weight: they differ in whether Blocker/Major +findings keep regenerating, and they must come out differently — clean finish versus plateau. An +earlier draft collapsed them into one row that declared a converged loop "stuck", which would have +produced a false non-convergence report. +**A second table, because the first one cannot see the reporting duty.** The nine states above score +*scope* and *action*, and neither input changes when the pass-4-onward reporting duty is deleted or +reversed — all nine results would stay identical. That is a coverage hole, not a passing result, so +the second of this round's two loop rules is scored separately, on its own inputs. Deleting the duty +from both copies flips R2–R5; raising its threshold from two tells to three flips R4 and R5; moving +the three lines from the status report into the findings file or the Codex reply flips the carrier +column in R2–R5. + +| # | Pass | Tells present | Old → required | New → required | Carrier | +|---|---|---|---|---|---| +| R1 | 3 | 1 | nothing | nothing — the duty starts at pass 4 | n/a | +| R2 | 4 | 0 | nothing | the three lines: trend, cluster, require↔withdraw | the agent's own status report | +| R3 | 4 | 1 | nothing | the three lines; stop **not** mandatory on one tell | the agent's own status report | +| R4 | 5 | 2 | nothing | the three lines; **stop-and-surface mandatory**, not discretionary | the agent's own status report | +| R5 | 6 | 2, and the "clearly stuck" reading unavailable | nothing | as R4 — the stuck reading is explicitly not a precondition | the agent's own status report | + +R1 is the control: a pass below 4 must require nothing, and a draft that started the duty at pass 1 +would fail here rather than pass silently. The old text decides **0 of 4** open states (R2–R5) by +being absent — `git show 17d5ad3:CLAUDE.md` has no reporting duty, no tells and no threshold — and +the new text decides **4 of 4**. Both copies score identically at both revisions. + + +**What this check has and has not caught, stated exactly, because "the matrix rejected three +drafts" was itself an overclaim an earlier pass had to remove:** + +- **One draft it actually rejected.** Case 9 was run against HEAD and came out `undecided`: the + wording then said coverage "stays a judgement you make and state", which a reader satisfies by + stating coverage is *insufficient* and stopping anyway. Gate-B pass 2 raised the identical defect + independently. +- **Four cases are retrospective regressions**, added after a gate pass found the defect, not + before: case 2 (pass 3 — the absorb rule granting a Minor a repair round), case 3 (pass 4 — the + dropped assigned-fix-set predicate), case 6 (pass 4 — a converged loop declared stuck), case 7 + (pass 5 — clean completion colliding with the floor). They are worth keeping as a regression + suite; they are not evidence that this check would have found those defects on its own. +- **Two earlier attempts at this evidence were rejected outright**: a keyword grep at pass 1 + (self-supplied — the keywords came from the new prose) and a paragraph diff plus mirror parity at + pass 2 (tested authorship, would pass if both copies omitted the same clause). + +**What it does not establish**, the enumeration being the whole method: that either enumeration is +exhaustive — neither the nine scope/action states nor the five reporting-duty states — or that a +reader applies the decided answer. The axes the second table varies are pass number, tell count, +carrier, and whether the "clearly stuck" reading is available — R5 varies the last of these to +establish that the two-tell stop has no such precondition. That list of covered axes **is** +exhaustive. The axis it does **not** cover is which of the five tell predicates produced a given +count, so a deleted or inverted tell definition can leave every R row unchanged. That gap is +recorded rather than closed (`.context/codex-reviews/gate-b-fic2-parked-review-economics.md`). +This is a decision-coverage check on prose, not a behavioural test, and the mode's `check` is +satisfied by it only on that reading. + +## What Gate B changed, and the one finding dismissed + +Pass 1 (spec branch) returned 18 findings, five at BLOCKER. Four of the five were valid and are +fixed in this cycle; they are recorded here because two of them changed a *disposition*, which the +table above would otherwise present as if it had been right the first time. + +- **Item 9's rejection was withdrawn.** The rejection rested on a search of the consumer's + `.context/`, `todos.md` and `CLAUDE.md`; its `docs/handoff-cowork.md` was not searched, and it + carries the field twice as an enforced protocol rule (lines 10 and 72). The item now has a + parked row. A rejection is a valid outcome only when the evidence was actually exhausted, and + here it was not. +- **The stuck criterion's figures were wrong and its claim too strong**, and they came from two + different places, which the earlier draft of this record blurred: the line count, the Blocker + start and the finding range came from the consumer's **memory note**, while the 34-pass total was + **my own count of pass artifacts** in that repo, attributed to the record as though the record + stated it. The consumer's `docs/hardening-taxonomy.md` is the later and stricter record of the + same series: it retracts the exact line count from the evidence, gives the Blocker series as + starting at 11 rather than 8, gives a range the memory note does not, and states plainly that *no + instrument there measures coverage and a low Blocker count can coexist with an unreviewed + subsystem*. The shipped rule now carries that + caveat, and both ledger rows are corrected by supersession entries rather than edits, per the + ledger's own convention that a row is immutable "committed or not". +- **Item 1's row inferred a hook mechanism it had not read.** A session-start `unavailable` is not + sticky: every counted pass recomputes the fingerprint and overwrites the state file. Nor does the + corrected row replace that with a different mechanism claim — the state write is best-effort, so + an `unavailable` endpoint cannot separate a hash that could not be computed from a computed hash + whose write failed. The row states the persisted endpoint values, the counts and the messages, and + leaves per-pass computation and persistence unknown. +- **Item 4's row misdescribed the gap.** `README.md` § Setup step 2b already documents the + variable fully, so "discoverable only by measurement" was false; the row now separates what the + variable buys (long successful calls reaching the hook at all, *and* independence from C1's + wording residual) from what is genuinely missing (the preflight check). + +**Dismissed, one line:** the fifth Blocker held that `/capture-finding` was closed into Finding A +without the choice being put to Daniel. It was put to him and he chose it explicitly — Codex +reviews the diff and cannot see the session, so it read the absence of an in-repo record of the +exchange as the absence of the exchange. + +Pass 1's remaining Minors and Nits were largely fixed rather than collected, because each was a +wrong sentence in a durable record this round is creating: the closure table's disposition labels, +the item-7 rejection's premise, a stale question count in the guard-scope story, and an internal +pointer in §5. Per §5 they were not iterated on. + +## Three supersession entries in this round are themselves malformed, and what was done about it + +Three entries appended during this cycle refer to another entry — "the entry immediately above", +"the earlier entry above", "the two above" — which the ledger header forbids outright: entries +"are never edited, never removed, and never reference one another". Gate-B pass 6 caught it. + +The repair used is the only one the convention supplies: **append**. The header states it for a +mistyped locator — "corrected the same way everything else is, by appending" — and states that a +superseded entry is left "standing as history, like every other entry". So for each affected row a +later, self-contained governing entry was appended, naming every fault at the field that carries +it and referring to no other entry. Under last-entry-governs those are what a reader acts on, and +the malformed lines remain as history. + +What this does **not** do, said plainly: it does not remove the malformed lines, and a reader who +stops at the first matching entry rather than the last will read a cross-reference the format +forbids. The convention names a repair for a mistyped locator specifically and says nothing about +an entry malformed in any other way; that gap is now a `todos.md` row rather than a silent +assumption that appending covers everything. + +## One correction to the pre-triage + +The pre-triage header reads item 1 as "NOT a hook defect", root-caused by auto-backgrounding and +collapsing into item 4. **The consumer's own entry does not support that**, and the row written +for item 1 says so. Backgrounding explains the *first* 2026-08-04 measurement, where the counter +was structurally zero. Item 1's evidence is the later measurements — 5 counted passes, then 24 — +taken with `CLAUDE_CODE_MCP_AUTO_BACKGROUND_MS="0"` already set, where counting worked and the +close still drew a STOP for a different reason. Item 4 remains independently valid — as the +unbuilt **preflight check**, not as a documentation gap, since `README.md` step 2b documents the +variable in full; item 1 keeps its own row as an unexplained fingerprint defect. The instruction +to verify that entry is what surfaced this, and the collapse would have retired a live defect. + +Two limits on the item-1 row, stated because it is the row a later reader will act on: this round +established neither whether the hash was computable at any given pass nor whether a computed hash +failed to persist, and it did not read the hook at the site that computes it. Its one new lead — that the second +shape's STOP arrived at a `git reset --soft`, which reaches the reset path only via `is_commit` — +ties it to the item-2 row and is a lead, not a finding. diff --git a/docs/hardening-log.md b/docs/hardening-log.md index 628cb64..1c69eeb 100644 --- a/docs/hardening-log.md +++ b/docs/hardening-log.md @@ -75,7 +75,20 @@ correction, mechanical readers do not, and nothing checks the difference. - 2026-08-17 · supersedes 2026-08-16 `docs-drift` "both told the reader to record the reviewer model by reading" · its hardening names `checks.config.effective.model` as the probe field, which is the wrong field wherever a per-tool override is set: the pinned server resolves a gate's model as `tools..model ?? model` (`dist/config/config.js`, `getToolConfig`), and the `CODEX_DEV_REVIEW_MODEL` surface the same row cites is stored at `tools.review.model` — so the named probe reproduced the misattribution the row claims to close, and it was wrong when written · docs/coding-workflow.md, the paragraph opening `**Record which model took each pass.**`, and the matching rule in docs/sparring-briefing.md: read `tools.review.model` for Gate B or `tools.exec.model` for Gate A, falling back to the top-level `model`, and record the model as undetermined where neither level names one - 2026-08-17 · supersedes 2026-08-16 `docs-drift` "both told the reader to record the reviewer model by reading" · its hardening names `checks.config.effective.model` as the probe field, and that was wrong when written; the entry immediately above, correcting it the same day, is itself wrong in two ways — it attributes a `CODEX_DEV_REVIEW_MODEL` citation to that row, which the row does not contain, and it restates the corrected rule instead of citing where the rule lives, which this ledger's own format forbids because a restatement is a second copy that can drift · docs/coding-workflow.md, the paragraph opening `**Record which model took each pass.**`; docs/sparring-briefing.md, the reviewer bullet opening `**The reviewer is whatever actually ran**` - 2026-08-17 · supersedes 2026-08-16 `docs-drift` "both told the reader to record the reviewer model by reading" · its hardening names a top-level probe field, which was wrong when written · docs/coding-workflow.md, the paragraph opening `**Record which model took each pass.**`; docs/sparring-briefing.md, the reviewer bullet opening `**The reviewer is whatever actually ran**` +- 2026-08-17 · supersedes 2026-08-17 `prompt-vague-criteria` "first row of this base class here" · three claims in its `finding` were wrong when written, caught by Gate-B pass 1 of its own cycle: the spec's exact line count is expressly not part of the consumer's evidence, the measured Blocker series starts at 11 and not 8, and "the substance converged" reads the curve as proof when that record states the opposite — no instrument there measures coverage, and a low Blocker count can sit beside an unreviewed subsystem · CLAUDE.md §5, the paragraph opening `**Recognizing "clearly stuck"**`, and the matching block in plugins/dev-workflow/commands/workflow-init.md, which carry the corrected figures and the coverage caveat +- 2026-08-17 · supersedes 2026-08-17 `prompt-missing-stop-condition` "first row of this base class here" · its `ref` describes the guard as covering the sentence that preserves the three-pass floor and then says the guard says nothing about pass counts, which contradicts itself and could let a later recurrence read a lost floor as outside the guard; what was meant is that the rule introduces no new pass threshold while guarding the preservation of the existing floor · CLAUDE.md §5, the paragraph opening `**What a loop absorbs, and what stops it.**`, and the matching block in plugins/dev-workflow/commands/workflow-init.md +- 2026-08-17 · supersedes 2026-08-17 `prompt-missing-stop-condition` "first row of this base class here" · the entry immediately above is itself malformed for this ledger: it states what the corrected rule now means instead of naming only what is false and where the current answer lives, which this format forbids because a restatement is a second copy that can drift; what is false in the target row is the internal contradiction in its `ref` alone · CLAUDE.md §5, the paragraph opening `**What a loop absorbs, and what stops it.**`; plugins/dev-workflow/commands/workflow-init.md, the matching block +- 2026-08-17 · supersedes 2026-08-17 `prompt-vague-criteria` "first row of this base class here" · a fourth false claim in the target row, not named by the earlier entry above: it gives the post-pass-six total as an oscillation between 2 and 7, where the consumer's record gives a 2-19 range that never reached zero; the same row's `ref` also predates the coverage precondition and the across-passes reading the rule now requires · CLAUDE.md §5, the paragraph opening `**Recognizing "clearly stuck"**`; plugins/dev-workflow/commands/workflow-init.md, the matching block +- 2026-08-17 · supersedes 2026-08-17 `prompt-missing-stop-condition` "first row of this base class here" · the row's `ref` is internally contradictory: it describes the guard as covering the sentence that preserves the three-pass floor and in the same breath says the guard says nothing about pass counts, so a later recurrence could read a lost floor as outside the guard; that contradiction is the whole of what is false in the row, and it was false when written · CLAUDE.md §5, the paragraph opening `**What a loop absorbs, and what stops it`; plugins/dev-workflow/commands/workflow-init.md, the matching block +- 2026-08-17 · supersedes 2026-08-17 `prompt-vague-criteria` "first row of this base class here" · four claims in the row are false and all four were false when written: it gives the spec's exact line count, which the consumer's record expressly excludes from its evidence; it starts the Blocker series at 8, where that record gives 11; it reads the curve as showing that the substance converged, where that record states no instrument there measures coverage and a low Blocker count can coexist with an unreviewed subsystem; and it gives the post-pass-six total as an oscillation between 2 and 7, where that record gives a 2-19 range that never reached zero. The row's `ref` also predates the coverage precondition and the across-passes reading · CLAUDE.md §5, the paragraph opening `**Recognizing "clearly stuck"`; plugins/dev-workflow/commands/workflow-init.md, the matching block +- 2026-08-17 · supersedes 2026-08-17 `prompt-vague-criteria` "first row of this base class here" · the row's `finding` carries five claims that do not hold and did not hold when written, and its `ref` is obsolete in two further respects. The claims: it states the spec's exact line count, which the consumer's record expressly excludes from its evidence; it states a starting Blocker count the record contradicts; it states a post-pass-six oscillation range the record contradicts; it reads the curve as showing that the substance converged, where the record states no instrument there measures coverage; and it states a total pass count taken from counting pass artifacts rather than from the record it attributes it to. The `ref`: it describes the guard as a reading taken after about six passes and as satisfied by every remaining finding being individually fixable, and the shipped rule no longer turns on either · CLAUDE.md §5, the paragraph opening `**Recognizing "clearly stuck"`; plugins/dev-workflow/commands/workflow-init.md, the matching block +- 2026-08-18 · supersedes 2026-08-17 `prompt-missing-stop-condition` "first row of this base class here" · two things in the row do not hold and neither held when written. Its `finding` states that a correction-of-a-correction is absorbed, with no condition on the assigned fix set, so it preserves an absorption rule broader than anything shipped; and its `ref` is internally contradictory, describing the guard as covering the sentence that preserves the three-pass floor while also saying the guard says nothing about pass counts. The row also predates the reporting duty the same guard now carries · CLAUDE.md §5, the paragraphs opening `**What a loop absorbs, and what stops it` and `**From pass 4 onward every pass report`; plugins/dev-workflow/commands/workflow-init.md, the matching blocks +- 2026-08-18 · supersedes 2026-08-17 `prompt-vague-criteria` "first row of this base class here" · the row's five faulty claims divide into two kinds, and calling them all false overstates what the sources show. Three are contradicted outright by the consumer's record: the starting Blocker count, the post-pass-six range, and the reading that the substance had converged. Two are unsupported or misattributed rather than refuted: the exact line count, which that record expressly excludes from its evidence, and the total pass count, which came from counting pass artifacts and was attributed to the record. Its `ref` further describes the guard as a reading taken after about six passes and as satisfied by every remaining finding being individually fixable, and the shipped rule turns on neither; the row also predates the reporting duty the same guard now carries · CLAUDE.md §5, the paragraphs opening `**Recognizing "clearly stuck"` and `**From pass 4 onward every pass report`; plugins/dev-workflow/commands/workflow-init.md, the matching blocks +- 2026-08-18 · supersedes 2026-08-17 `prompt-missing-stop-condition` "first row of this base class here" · a third claim, stated for completeness alongside the two above so this entry describes the row as it now stands: the row says the rule was field-minted across roughly 150 gate passes, where that figure is the consumer project's total gate-pass count and not the set of passes on which the ruling was observed or applied · CLAUDE.md §5, the paragraph opening `**What a loop absorbs, and what stops it`; plugins/dev-workflow/commands/workflow-init.md, the matching block +- 2026-08-18 · supersedes 2026-08-17 `prompt-missing-stop-condition` "first row of this base class here" · three claims in the row do not hold and none held when written. Its `finding` states that a correction-of-a-correction is absorbed with no condition on the assigned fix set, preserving an absorption rule broader than anything shipped. Its `finding` also gives the observation base as roughly 150 gate passes, which is the consumer project's total gate-pass count rather than the passes on which the ruling was observed or applied. Its `ref` is internally contradictory, describing the guard as covering the sentence that preserves the three-pass floor while also saying the guard says nothing about pass counts. The `ref` further predates the reporting duty the same guard now carries · CLAUDE.md §5, the paragraphs opening `**What a loop absorbs, and what stops it` and `**From pass 4 onward every pass report`; plugins/dev-workflow/commands/workflow-init.md, the matching blocks +- 2026-08-18 · supersedes 2026-08-17 `prompt-missing-stop-condition` "first row of this base class here" · three faults, each named at the field that carries it. The `finding` gives the observation base as roughly 150 gate passes, which is the consumer project's total gate-pass count rather than the passes on which the ruling was observed or applied. The `ref` states the guard as absorbing a correction-of-a-correction with no condition on the assigned fix set, an absorption rule broader than anything shipped. The `ref` is also internally contradictory, describing the guard as covering the sentence that preserves the three-pass floor while saying the guard says nothing about pass counts. This row's guard covers the absorb-vs-stop rule only; the reporting duty belongs to the `prompt-vague-criteria` guard · CLAUDE.md §5, the paragraph opening `**What a loop absorbs, and what stops it`; plugins/dev-workflow/commands/workflow-init.md, the matching block +- 2026-08-26 · supersedes 2026-08-17 `prompt-missing-stop-condition` "first row of this base class here" · three claims in the row were false when written, each named at the field that carries it. Its `finding` gives the observation base as roughly 150 gate passes, which is the consumer project's total gate-pass count and not the passes on which the ruling was observed or applied. Its `ref` states the absorbed case as a finding correcting the correction just made, with no condition on the assigned fix set, which is broader than anything shipped. Its `ref` is also internally contradictory, listing the three-pass floor among what the stop preserves while stating that the guard says nothing about pass counts · CLAUDE.md §5, the paragraph opening `**What a loop absorbs, and what stops it`; plugins/dev-workflow/commands/workflow-init.md, the matching block Columns: `date` (YYYY-MM-DD), `fingerprint` (canonical class), `finding` (short, escape `\|`, one line), `source` (gate-a|gate-b|bot|manual), `severity` (blocker|major|minor|nit), `rung` (e.g. `2 lint`, `4 test`, `1 prose`, @@ -107,3 +120,5 @@ escape `\|`, one line), `source` (gate-a|gate-b|bot|manual), | 2026-08-04 | mechanical-check-skipped-before-review | NEW CLASS, minted this change: eight read-only Gate-A passes over one plan missed eight defects that thirteen machine checks then found in a single sweep, including a rollback that would have byte-verified against the wrong hook | manual | major | P std | CLAUDE.md §5 Gate-A pass procedure + the same block in the workflow-init inline template: before each read pass, settle mechanically what the artifact asserts and a machine can decide without side effects — cited paths, quoted passages, stated counts, the syntax of standalone fenced blocks — because a read pass spends expensive judgement on what a parser settles in seconds and misses it anyway, inspecting quoted commands rather than running them, since a command quoted in a spec may be destructive or an intentional failure. Class added to docs/hardening-taxonomy.md in this same change, with its boundary against verification-masks-failure stated: there a check ran and could not fail; here the cheap check never ran at all. NO PRIOR ROW — this is the first occurrence. STILL INSTRUCTION-BACKED: nothing runs the sweep, records that it ran, or checks what it settled | | 2026-08-16 | unverified-enforcement-claim | sixth occurrence, and the first where the standing rung FIRED AND HELD: PR #24's Greptile P1 found §5 claiming an empty record-only commit "does not reopen any gate" — true of the obligation, false of the shipped hook, which routes an empty staged-path list through the ordinary Gate-B decision. Four consecutive Gate-B rounds then reproduced the class *inside the correction itself*: an enumeration read as complete ("STOP or below-floor"), a single-cause label for a three-way condition ("stale-fingerprint STOP"), a landing guarantee inferred from the hook's exit status ("so the commit lands"), and a categorical "the hook fires" that is false on the non-adopted and `codex-gate.off` silent paths | bot | major | P std | NO NEW RUNG — `docs/prompt-standards.md`'s existing rule resolved it: "when a claim about a mechanism needs a fourth correction, delete the claim rather than refine it a fifth time". It was applied at exactly the fourth correction and the state-machine narration was deleted from all five restatements, leaving the obligation ("an empty diff raises no review obligation"), the advisory fact, and the `git show --stat` test. This is the row that says the rung WORKS: it was minted after an incident where four corrections each introduced a subtler version of the same claim, and here it stopped the identical spiral at the same round rather than a fifth. What it does NOT do is prevent the first three corrections — it bounds the spiral, it does not catch the original overclaim, and nothing mechanical decides when a claim "needs a fourth correction"; the reviewer counts. Also recorded: deleting the narration made the two §5 copies' edited regions byte-identical, removing the invariant-citation divergence that had itself produced a finding | | 2026-08-16 | docs-drift | sixth occurrence: PR #24 (CodeRabbit) — `docs/coding-workflow.md` and `docs/sparring-briefing.md` both told the reader to record the reviewer model by reading "the configured value at that moment", while the same section's own timing facts said `mcp-codex-dev` caches its model chain per project root. Two sentences in one document disagreeing, and the wrong one was the actionable instruction: after any model edit the config names a model the running server is not using, so a pass record built from it misstates who reviewed — which is the only thing that makes reviewer-family independence checkable | bot | major | 1 prose | Both documents now name the model the pass *ran under* as the thing recorded, state the cache boundary exactly (launch root at startup, any other root on its first call, cached until restart, and an unseen root is the exception where a post-startup edit does take effect), and name a DETERMINISTIC probe rather than "check the config": `mcp__codex__health` with the same `workingDirectory` as the gate call, reading `checks.config.effective.model`, which is the cached per-root resolution the gate call itself uses. NOT ESCALATED past the 2026-07-26 `2 lint` row: that check guards prose count claims and cannot reach a stale mechanism description, the over-escalation those rows warn about. NO DETERMINISTIC RUNG EXISTS for this: nothing can tell that a sentence about a cache is stale, and the probe is a recipe a human runs, not a check — it raises the floor by making the right value obtainable, and does not close the class. Sibling row this same date under `unverified-enforcement-claim` covers the empty-commit half of the same PR review | +| 2026-08-17 | prompt-missing-stop-condition | first row of this base class here: §5's loop told the reader to keep looping and never said what a loop may swallow, so absorbing a finding that opened a new contract question and handing back a three-line repair-of-a-repair were equally defensible readings — field-minted by the kit's heaviest consumer across ~150 gate passes, and its own reason for the rule is that absorbing a contract question spends a decision the loop was never given | manual | major | P std | CLAUDE.md §5 loop paragraph + the same block in the workflow-init inline template. Guard: the absorb-vs-stop rule and the sentence bounding it — a finding correcting the correction just made is absorbed, one opening a new structural or contract question stops the loop, novelty and not size decides, and the stop is NOT an exit from the gate (floor, Blocker/Major filter and clean-final-pass all stand). Does NOT guard which findings count as structural, and says nothing about pass counts. | +| 2026-08-17 | prompt-vague-criteria | first row of this base class here: §5 named "clearly stuck" as the loop's terminal state and never said how to recognize one, so a plateauing artifact could burn passes with nobody entitled to call it — in the field one 2848-line spec ran 34 Gate-A spec passes, and over the 19 that were measured the count fell from 43 to an oscillation between 2 and 7 while Blockers fell from 8 to 0-1, i.e. the substance converged and the number never did | manual | major | P std | CLAUDE.md §5 loop paragraph + the same block in the workflow-init inline template. Guard: the recognition rule — after about six passes read the Blocker curve rather than the total, and Blockers at 0-1 with every remaining finding individually fixable IS the stuck state — plus the smaller-specs guidance that follows from the same measurement. Does NOT guard a size threshold (deliberately unmeasured and shipped as guidance), and does not say where a plateau begins. | diff --git a/docs/superpowers/stories/2026-08-04-harden-finding-guard-scope-precheck-story.md b/docs/superpowers/stories/2026-08-04-harden-finding-guard-scope-precheck-story.md index 0fbc5d6..34be5d9 100644 --- a/docs/superpowers/stories/2026-08-04-harden-finding-guard-scope-precheck-story.md +++ b/docs/superpowers/stories/2026-08-04-harden-finding-guard-scope-precheck-story.md @@ -61,7 +61,7 @@ ledger. Escalation follows from a guard that failed, never from a count. ## 5. Open questions -Seven — six paired with the failure that raised each, and one inherited from another story: +Eight — six paired with the failure that raised each, and two inherited from other stories: - A verdict was reached by reading one prior row's guard and stopping, when an older row's guard covered the case. **Which prior rows must a recurrence be judged against before "outside" is @@ -94,9 +94,19 @@ Seven — six paired with the failure that raised each, and one inherited from a **Should a recurrence read consult the supersession record, and if so, does a prose-corrected class or a prose-retracted hardening change the lineage it escalates from?** Arrives as a handoff, not as a failure: no instance of either has occurred. +- **Inherited** from `2026-08-17-arms-race-remedy-as-procedure-story.md`, and inherited back + (added 2026-08-17 by the canvas field-intake round): that story adds a case to the branch this + one redesigns — a recurrence whose right answer is **not a rung at all**, because the + instrument is checking at the wrong level and one more arm of the same rung is the same rung + again. This repo already states that locally, in `todos.md`'s invariant-checker escalation row, + and nowhere generally. **Does the branch designed here decide "escalate / fit the rung / + change the instrument", or does it hand the third answer off — and to what?** Both stories + edit one decision path, so **whichever is picked up first must read the other and carry its + conditions**: two rules landing separately on one branch is the shape that produced the bug + this story was filed against. Arrives as a handoff, not as a failure. ## 6. Suggested size `story` — one skill file, one decision procedure, one spec → plan → PR. Above a chore because -the seven questions above are real design; below an epic because they all concern one procedure +the eight questions above are real design; below an epic because they all concern one procedure in one file. diff --git a/docs/superpowers/stories/2026-08-17-arms-race-remedy-as-procedure-story.md b/docs/superpowers/stories/2026-08-17-arms-race-remedy-as-procedure-story.md new file mode 100644 index 0000000..706571d --- /dev/null +++ b/docs/superpowers/stories/2026-08-17-arms-race-remedy-as-procedure-story.md @@ -0,0 +1,131 @@ +# The arms-race remedy as a procedure, not an observation — Story + +**Date:** 2026-08-17 · **Size:** story + +**Unprofiled, deliberately.** This story is a split from a field-intake round, not a raw idea, +so it did not pass through `dev-workflow:intake` — intake excludes items that have moved into +solution design. A profile is proposed and human-confirmed at intake time. Writing one here +would produce a header that **looks** confirmed and is not, and nothing would reveal that: +`CLAUDE.md` §5 stops on a profile that is malformed or internally inconsistent, not on one whose +values are well-formed but unconfirmed. Same reasoning, and the same debt-as-criterion answer, as +`2026-08-04-harden-finding-guard-scope-precheck-story.md`, which this story inherits from. + +## 1. Problem statement + +A series of review passes can stop converging without any single pass looking wrong: each +correction closes the findings it was aimed at and enables the next findings of the same shape, +so the count oscillates and same-shape lineages recur across passes. What that does **not** +establish is that the artifact is finished — the same record states that coverage was never +measured and that a low Blocker count can coexist with an unreviewed subsystem, so "the substance +has converged" is exactly the clearance inference this repo now forbids at the §5 exit. The field +has a name for the pattern — the instrument is checking at the wrong level — and a remedy that +worked: change what is being checked rather than patch once more, and stop with a named state. + +The kit has part of it, as of the same change that parks this story, and the gap is what remains. +`CLAUDE.md` §5 now carries **one** recognition heuristic and a terminal action: read the Blocker +curve across passes rather than any single total, and surface the stuck state only on three +conditions together — a visible plateau, an affirmative sufficient-coverage judgement, and +Blocker/Major findings that keep regenerating across repair attempts, with a clean pass taking +precedence over the exit. (Six passes is where the field observed a plateau, not a point at which +stopping becomes authorized.) That covers a *review loop over one artifact* and nothing else. What has +no home is the general case — a series of passes over an **instrument** that keeps finding one +more spelling its own claim text seemed to cover — and, with it, the **moves** (scope the reviewer +to changed regions, change the instrument's layer, relocate the residual to the layer that already +catches it), the **integration with the rung ladder**, and the **durable named state** the stop +should leave behind. `harden-finding` still escalates one rung when a fingerprint recurs, and that +rule mis-reads this case: one more arm of the same rung is the same rung applied again, which the +kit's own parked invariant-checker row states locally ("adding one more regex arm per +newly-discovered spelling is *not* the ladder working") and nowhere generally. + +**So this story either extends or replaces a decision path that now exists**, and it must say +which. The §5 heuristic is not to be silently superseded: whoever takes this up lists what that +paragraph requires today and marks each condition kept, moved or dropped, the same accounting +`AGENTS.md` demands of any decision-procedure rewrite. **That inventory covers both stop paths, +not just the recurrence step and the "clearly stuck" clause**: the pass-4-onward reporting duty +ships in the same block and shares the same decision path — its three-line carrier, its five +tells, and the rule that any two of them make stop-and-surface mandatory are conditions this +story can silently drop exactly as easily, and dropping them is the defect `AGENTS.md` names. + +The cost is re-derivation. In `infinite-portfolio-canvas` the remedy was reached from scratch +after four consecutive passes had each found one more spelling the claim text seemed to cover; +changing the instrument's form rather than adding a fifth pattern took the findings from nine to +zero. Three sites record it there — its `docs/hardening-taxonomy.md` corollary, +`.context/a5-t2a-resume.md`, and `.context/gate-b-a5-t2a-dispositionen.md`, where six of six +findings were about the checker's grammar and none about product behaviour. + +## 2. Desired outcome + +A reader inside a non-converging series recognizes it as one, has a named set of moves rather +than an instinct, and reaches a **named terminal state** instead of running another round. The +rung decision accounts for it, so "another arm of the same rung" is never proposed as though it +were escalation. Whoever reads the result can tell which move was taken and why, after the +session that took it has ended. + +## 3. Acceptance criteria + +- [ ] The unprofiled header above is resolved at design time: a profile is proposed, confirmed by + the human, and written before the change ships — or the change ships with the header + absent and this criterion records why, never with an unconfirmed header that looks + confirmed. +- [ ] The procedure names three things separately: the **recognition signal** (what makes a + series non-converging rather than merely long), the **moves** available, and the + **terminal state** it stops in. A reader can check each against a transcript. +- [ ] Exactly one place decides which rung a recurrence gets. The procedure composes with + `harden-finding`'s recurrence rule and its guard-scope precheck without adding a second + decision branch on the same question. +- [ ] Every condition imposed by the current recurrence step **and by both of §5's stop paths** is + listed and marked kept, moved, or deliberately dropped (`AGENTS.md`, "Never replace a + decision procedure without accounting for its old conditions"). The two paths are the + "clearly stuck" clause **and** the pass-4-onward reporting duty, which is a separate path + and not a part of that clause: its conditions are the three-line report, its carrier being + the agent's own status report to the user, the five tells, the rule that any two of them + make stop-and-surface mandatory, and **the activation boundary — the duty begins at pass 4 + and binds on every pass after it**, which calling the path "the pass-4-onward reporting + duty" names without accounting for. Naming the reporting duty in prose does not satisfy this — + each of those conditions is accounted for individually. +- [ ] The wording is stack-neutral: no vocabulary from the consumer the evidence came from + (invariant 10). +- [ ] The evidence's limits are stated where the procedure claims field support — three sites in + **one** consumer, one of which is that consumer's own taxonomy prose rather than an + application of it, plus one corroborating distribution from this repo (`7bbdb14`). + +## 4. Affected AGENTS.md invariants + +- `### Prompts and scaffolding` — "10. **The base taxonomy stays stack-neutral.** Project + vocabulary — tables, auth helpers, framework APIs — goes only in that project's + `docs/hardening-taxonomy.md`, never into the `harden-finding` skill. Otherwise one project + leaks into every other." +- `### Prompts and scaffolding` — "11. **Prompt changes pass `docs/prompt-standards.md`** — all + 12 checklist items, for any skill, command, agent definition, hook message, or scaffolded + template." +- `### Packaging` — "12. **A plugin change requires a version bump.**" +- `## Don'ts` — "**Never replace a decision procedure without accounting for its old + conditions.** List what the previous prose required, then mark each one kept, moved, or + deliberately dropped." + +## 5. Open questions + +- **What separates a non-converging series from a long one?** The field read the *Blocker curve* + rather than the total, on the ground that substance can converge while the count does not. Is + that the signal here, a count of same-shape findings, or something a reader can apply without + having seen the whole series? +- **Who owns the stop?** §5's loop now has a criterion for "clearly stuck" and a mandatory + reporting duty beside it, both added by the round that parked this story — so the open question + is *not* whether that criterion is missing. It is whether the instrument case needs its own + terminal state alongside them, or whether reporting the tells is already the whole answer for a + series over an instrument. +- **What is a "named state"?** The field's version was a ticket with its own budget whose + trigger is a relapse. Does the kit require a durable artifact, and if so which — a `todos.md` + row, a ledger row at the fitting rung, or a note at the site? +- **Inherited** from `2026-08-04-harden-finding-guard-scope-precheck-story.md`, and inherited + back: that story redesigns the branch deciding which rung a recurrence gets, and this one adds + a case that branch must handle — a recurrence whose right answer is *not a rung at all* but a + change of instrument. Both edit one decision path, so **whichever is picked up first must read + the other and carry its conditions**, rather than re-deriving them; two rules landing + separately on one branch is the shape that produced the bug that story was filed against. + +## 6. Suggested size + +`story` — one procedure, one decision path, one spec → plan → PR. Above a chore because it +changes what a reader does at a mandatory loop's exit, and it cannot be written without settling +how it composes with the recurrence rule. diff --git a/docs/superpowers/stories/2026-08-17-field-intake-canvas-a1-a5-report-story.md b/docs/superpowers/stories/2026-08-17-field-intake-canvas-a1-a5-report-story.md index c321e7a..5d84064 100644 --- a/docs/superpowers/stories/2026-08-17-field-intake-canvas-a1-a5-report-story.md +++ b/docs/superpowers/stories/2026-08-17-field-intake-canvas-a1-a5-report-story.md @@ -50,8 +50,21 @@ but not answered. - [ ] The parked `/capture-finding` row carries a disposition — built, or re-parked with the fired trigger and the evidence recorded against it — and that decision was put to Daniel before it was written. -- [ ] Every ledger row this round appends states the guard-scope precheck's outcome - against the prior same-fingerprint row, quoting the guard it examined. +- [ ] The guard-scope precheck's outcome — including the quoted guard of each nearest prior row + examined — is recorded for every ledger row this round appends. + **Amended 2026-08-18, with accounting, after Gate-B pass 6 found the criterion unmet as + originally written.** It read: "Every ledger row this round appends *states* the guard-scope + precheck's outcome against the prior same-fingerprint row, quoting the guard it examined." + What each old condition became: *the precheck must run* — **kept**, unchanged; *its outcome + must be recorded* — **kept**; *the examined guard must be quoted* — **kept**; *the record must + live in the ledger row itself* — **moved**, to the closure record. Why moved rather than + satisfied: for both rows the precheck returned **no prior same-fingerprint row**, so there was + no prior guard to quote in the row, and the nearest-guard readings that were actually + performed are comparisons against *other* fingerprints — which a row's `ref` has no field for + and which the recurrence grep would never read. Rows are immutable once written, so this is an + amendment rather than a repair. What is lost by the move is stated rather than glossed: a + future recurrence reader working from the ledger alone will not see the precheck, and must + follow the round's closure record for it. - [ ] The consumer repo `infinite-portfolio-canvas` has no file created, modified or deleted by this round. diff --git a/plugins/dev-workflow/.claude-plugin/plugin.json b/plugins/dev-workflow/.claude-plugin/plugin.json index fcfec04..c7b63d7 100644 --- a/plugins/dev-workflow/.claude-plugin/plugin.json +++ b/plugins/dev-workflow/.claude-plugin/plugin.json @@ -1,7 +1,7 @@ { "name": "dev-workflow", "displayName": "Cross-Model Review Workflow", - "version": "0.9.1", + "version": "0.10.0", "description": "Spec-driven workflow with two independent cross-model review gates, an append-only hardening ledger with an escalation ladder, and repo-enforced quality. Requires the superpowers plugin.", "author": { "name": "Daniel Sänger", diff --git a/plugins/dev-workflow/CHANGELOG.md b/plugins/dev-workflow/CHANGELOG.md index 580636e..02eac0f 100644 --- a/plugins/dev-workflow/CHANGELOG.md +++ b/plugins/dev-workflow/CHANGELOG.md @@ -22,6 +22,63 @@ unambiguously, still fails. Deleting only a plugin's *manifest* while the direct keeps shipping fails too. AGENTS.md invariant 12 carries the complete list. +## 0.10.0 + +- §5's gate loop gained two rules it was missing, both mirrored into `workflow-init`'s scaffolded + template. Their provenance differs and the difference is worth stating: the **absorb-vs-stop** + rule was minted in field use by the kit's heaviest consumer and is carried over largely as that + consumer wrote it, while the **stuck criterion** was constructed during this intake from a small + number of field observations — after the consumer's own stricter record retracted the first + reading of them — so it is derived from field measurement rather than field-proven as a rule. **What a loop + absorbs** is a question of **scope, not of action**: a finding correcting the correction just + made **and staying inside the assigned fix set** is inside the loop's scope — one that leaves that + set stops the loop like any other out-of-scope finding, even when it opens no new question, since + absorbing it would grow the assigned work unagreed — and an in-scope finding is then acted on by + its severity exactly as before — + Blocker/Major resolve, Minor/Nit collect and never iterate — so ancestry grants no finding a + repair round it would not otherwise get; a finding opening a new structural or contract question + stops the loop whatever its set membership, and where a finding is both, novelty overrides + ancestry. Size is not the test. **What "clearly stuck" means:** §5 has always sent a non-converging loop to + the user and never said how to recognize one. It now requires **three conditions together**, any + one missing meaning keep going — a plateau visible **across passes** rather than one pass's count; + an **affirmative** stated judgement that coverage is sufficient, with a known materially + unreviewed area forbidding the exit outright; and Blocker/Major findings that **keep regenerating + across genuine repair attempts**, each round's fix producing the next. That third condition is + what separates a plateau from a finish, and **a clean completion takes precedence over the exit**: + a Blocker/Major-free pass **at or above the floor** has satisfied the clean-final-pass rule, so + collect the Minors and close rather than report non-convergence on a converged loop. What the + exit produces is a report that the loop will not converge, never a clearance. Below the floor + nothing closes, and a zero-finding pass remains the sole exception. +- **A reporting duty ships with the stuck criterion.** From pass 4 onward every pass report carries + three lines — the trend in findings and Blocker counts across passes, where this pass's findings + cluster (product behaviour, the test instrument, or prose about either), and any require↔withdraw + pair against earlier passes. Those lines expose five tells, and **any two present makes + stop-and-surface mandatory rather than discretionary**, with the stuck reading explicitly not a + precondition — a loop can be worth stopping long before it plateaus. The rationale is recorded in + §5 as the maintainer's, not as a measurement of this repo: in another consumer all five signals + were measurable by day two of a week-long loop, and the cost was never detection but the absence + of a duty to say so. +- **Surfacing does not close the cycle**, and both copies say so, because the exit would otherwise + compete with the rule that every Blocker and Major resolves: you surface with the finding still + open, no pass is credited clean, and the loop resumes on what the user decides. The clean-completion + precedence is likewise bounded by the floor — a Blocker/Major-free pass closes only at or above it, + and a zero-finding pass remains the sole below-floor exit. +- **Neither curve measures coverage**, and that is why the coverage condition is affirmative rather + than merely stated: a low Blocker count can sit beside an entirely unreviewed subsystem, and an + earlier draft of this rule let a reader disclose insufficient coverage and stop anyway. Six passes + is where the field observed a plateau — it is that one observation, not a timer that authorizes + stopping. +- Neither rule loosens the gate, and the shipped text says so where a reader would otherwise + assume it: a scope stop is **not** an exit — the floor, the Blocker/Major filter and the + clean-final-pass rule all still stand, and the loop resumes on the revised artifact. Without + that sentence "stops the loop" reads as a sanctioned below-floor close, which is the gate-off + path the rule is not for. +- The template copy is downstream-neutral rather than verbatim: it carries both rules and the + measurement that motivates the second, and drops the consumer's name and this repo's own + provenance, which do not travel into a scaffolded project. The measurement is quoted as one + observation on one artifact, and the guidance is stated as guidance — where a long artifact's + plateau starts is unmeasured, so no threshold is claimed. + ## 0.9.1 - §5's human-exception form and its `workflow-init` template no longer claim the empty diff --git a/plugins/dev-workflow/commands/workflow-init.md b/plugins/dev-workflow/commands/workflow-init.md index e3e08bc..3895394 100644 --- a/plugins/dev-workflow/commands/workflow-init.md +++ b/plugins/dev-workflow/commands/workflow-init.md @@ -279,6 +279,80 @@ clean or clearly stuck → then STOP and surface to the user. The only early exi below 3 is a pass with **zero** findings; don't manufacture findings to pad. Codex is advisory — validate before applying; dismissed finding → one-line why. +**What a loop absorbs, and what stops it — a question of scope, not of action.** A finding +that corrects the correction you just made **and stays inside the assigned fix set** is +**inside this loop's scope**: keep it here rather than handing it back, then act on it by its +severity exactly as the severity rule already says — Blocker/Major resolve, Minor/Nit collect +and never iterate. Ancestry decides where a finding belongs; it never decides what you do +with it, and it grants no Minor or Nit a repair round it would not otherwise get. **The assigned fix set is fixed before the pass you are answering: it is the +scope the approved story or plan assigns to this cycle, plus repair obligations you already +accepted in earlier passes.** A finding is in-set when repairing it stays inside that scope — +never merely because it arrived in the current pass, which would put every new finding in the +set by definition and leave the boundary deciding nothing. Where membership is genuinely +unclear treat the finding as **outside**, which costs a question and never a silent expansion. **A correction that leaves that set stops the +loop like any other out-of-scope finding**, even when it opens no new question at all — +absorbing it would grow the assigned work without anyone agreeing to that — and it resumes +the moment the user says whether the set now includes it. A finding +that opens a **new structural or contract question** stops the loop and goes to the user — +**size is not the test, novelty of the question is**, so a structural finding that is +genuinely small still stops it, while a long correction still aimed at the last correction +does not — provided that correction, too, stays inside the set, which its ancestry never +supplies on its own. **When a finding is both** — it corrects the last correction *and* opens a new +structural or contract question — **the new question wins and the loop stops**: novelty +overrides correction ancestry, because absorbing on ancestry is how a contract decision +gets made without anyone choosing it. Stopping this way is **not an exit from the gate**: the floor, the +Blocker/Major filter and the clean-final-pass rule all stand, and the loop resumes on the +revised artifact once the question is answered. What it prevents is a loop committing you +to a design nobody chose — a different failure from an unfinished review. + +**Recognizing "clearly stuck", so that exit is a reading and not a mood.** Read the +**Blocker curve across passes**, not any single pass's total — it is the better of the two +signals, the total says less than it looks like, and one low count is a snapshot rather +than a plateau. **Neither curve measures coverage:** a low Blocker count can sit beside an +entirely unreviewed subsystem. So this exit needs three things **together**, and a missing +one means keep going: a plateau visible across passes (six or more is where the field saw +one); an **affirmative judgement that coverage is sufficient**, stated — a known materially +unreviewed area forbids this exit outright, and disclosing it does not license it; and +**Blocker or Major findings that keep regenerating across genuine repair attempts**, each +round's fix producing the next. That third condition is what makes a plateau rather than a +finish, and it is why **a clean completion takes precedence over this exit**: a +Blocker/Major-free pass **at or above the floor** has satisfied the clean-final-pass rule — +collect the Minors and Nits and close — and reporting "will not converge" on a converged +loop is a false report. **Below the floor nothing closes**, and a zero-finding pass remains +the only exception, exactly as above; a Blocker/Major-free pass 1 carrying a Minor keeps +looping. +**Surfacing does not close the cycle, and that is what makes this reachable.** You surface +*with the finding still open* — the resolve rule is not waived, no pass is credited as +clean, and the loop resumes on whatever the user decides. Reading it as "stop instead of +fixing" would put the exit in competition with the rule that every Blocker and Major +resolves, and then nothing could satisfy both. + +**From pass 4 onward every pass report carries three lines.** The carrier is **your own +status report to the user** — never the Codex reply, which stays exactly one line per branch, +and never the findings file, which admits no line that is not a finding or the terminator. +They are cheap because the numbers already exist: (1) the **trend** — findings and Blocker counts across the passes so +far; (2) where this pass's findings **cluster** — product behaviour, the test instrument, or +prose about either; (3) any **require↔withdraw pair** against earlier passes, meaning a pass +demanding what an earlier pass had removed. + +Those three lines expose **five tells**: the finding count rising rather than falling; the +Blocker count failing to fall; findings clustering on the **instrument** rather than on +product behaviour; findings clustering on **prose about** either; and a require↔withdraw +pair. **Any two present makes stop-and-surface mandatory, not discretionary** — report the +tells and hand the decision to the user, and the "clearly stuck" reading above is not a +precondition for it. A loop can be worth stopping long before it plateaus. +**The two rules above do not compete**, and neither overrides the other: the absorb rule +decides whether *a finding* is inside this loop's scope, this reading decides whether *the +loop* can still converge. A small correction-of-a-correction that stays inside the assigned fix +set is absorbed and is not by itself evidence of a plateau. Measured once, at the precision the record keeps: nineteen Gate-A +passes over successive revisions of one design spec past 2800 lines (the exact size is not +part of that evidence), findings from 43 into a 2–19 range after pass 6 and never zero, +Blockers from 11 to 0–1 from pass 7 on, and the late Blockers were semantic contradictions +rather than wording — which is why a low count is a signal to read and not a clearance. +Hence the sizing guidance: prefer **smaller specs with named interfaces** and let the plan +carry the detail — guidance, not a threshold, because where the plateau starts is +unmeasured. + **Findings go to a FILE, not the response — both gates.** Long finding lists come back cut off, and a cut that lands between findings is indistinguishable from a short list: silently dropped findings, the dangerous direction. Claude Code both limits MCP tool diff --git a/todos.md b/todos.md index df9c4fe..d7b8a33 100644 --- a/todos.md +++ b/todos.md @@ -62,6 +62,20 @@ driven by recurrence rather than by enthusiasm. **TRIGGER FIRED (2026-08-04):** the 2026-07-20 row taught pre-0.8.0 counting behaviour as current — the second falsified row this trigger named. Story: `docs/superpowers/stories/2026-08-04-hardening-ledger-supersession-story.md`. +- [ ] **The supersession convention names a repair for a mistyped locator and for nothing else an + entry can get wrong.** `docs/hardening-log.md`'s header forbids entries from referencing one + another, requires the governing entry to describe its row as it now stands, and forbids + restating the current answer — but the only repair it spells out is for a mistyped locator + ("corrected the same way everything else is, by appending"). An entry that is malformed in any + other way has no named move, and entries are protected "committed or not", so editing is out. + **Found in use, not theorised:** this round appended three entries that referred to other + entries, and Gate-B pass 6 caught them. The repair applied — append a later, self-contained + governing entry and leave the malformed lines standing as history — is a *reading* of the + locator rule generalized, and it leaves a reader who stops at the first matching entry looking + at a forbidden cross-reference. Any fix must keep entries immutable, which is the constraint + that makes this awkward in the first place. + *Trigger: a third round appending a malformed entry, or the first reader observed acting on a + superseded entry instead of the governing one.* - [ ] **A hardening that is later *removed* has no sanctioned supersession move.** The convention in `docs/hardening-log.md`'s header covers a row whose narration was falsified later or was wrong when written, and names a removed hardening as explicitly out of scope. No instance @@ -126,6 +140,149 @@ driven by recurrence rather than by enthusiasm. closing it means decoding the `type` value for equality while still returning the selected `text` in its original escaped bytes, since the matcher depends on those. *Trigger: a serializer observed emitting escaped key or type spellings.* +- [ ] **Two consumer Gate-B cycles closed with no usable fingerprint persisted, while counting + and cycle-recognition worked.** Field-reported from `infinite-portfolio-canvas` (~60+ Gate-B + cycles, kit 0.8.0 since mid-run); evidence is that repo's `todos.md` § Tooling + revalidation, which carries four dated measurements, and + `docs/field-reports/2026-08-16-canvas-a1-a5-field-report.md` item 1. Two shapes, both + measured with `CLAUDE_CODE_MCP_AUTO_BACKGROUND_MS="0"` **already set** — so this is not + the backgrounding cause that entry started with, and it does **not** collapse into the + preflight-check row in § Next. + **(a) 2026-08-11, cycle `5a4bf38`, 24 counted passes.** `.context/codex-gate.gateB` held + the literal `unavailable` and `codex-gate.freshCount` a `0` at session start. **Read the + hook before repeating the consumer's inference here:** a stale `unavailable` is **not** + sticky — every counted pass recomputes `tree_hash()` and overwrites the state file + unconditionally (`codex-gate.sh`, the counted-pass branch), setting the fresh streak to 1 + on a computable hash and to 0 only when the hash is uncomputable *at that moment*. So the + observation does not show a cycle poisoned by its starting state. **What it does show is + narrower than "the hash was uncomputable at every pass":** the persisted value was + `unavailable` at the observed endpoints, and the write itself is best-effort — the state + file is written under `2>/dev/null || true`, so a later computable hash that failed to + persist is indistinguishable from one never computed. Uncomputable hashes and a failing + write are both live candidates, and the row picks neither. + Conditions were the cleanest + available — pass 13 Blocker/Major-free on both branches, both branches on byte-identical + content, nothing edited between the last pass and the `--amend`, the closing tree equal + to the reviewed WIP tree (`b2757ff`), and the hook's own diagnostic checklist green + point by point (`.context/` and `TMPDIR` writable, `shasum` present, `git status` runs, + 49 GiB free). Both state files were gone after the amend, so the hook read the amend + correctly as cycle-closing. Recognition and counting work; what the reported states do not + separate is computation from persistence. + **(b) 2026-08-16, cycle `f09286b`, counter at 21.** The count alone says nothing about the + fingerprint — the hook increments it independently of whether a hash was computed or + persisted — so read this as an observation and not as proof that storage worked. That STOP + was correct (a comment correction sat + between pass 13 and the reset — real drift). Unexplained is the close: after a fresh pass + 14, clean on both branches, the `--amend` reported "no fingerprint is recorded for this + cycle", which is the `reviewed`-empty branch — the state file absent, not stale. The + consumer's own narrowing is the ordering "WIP commit → pass → `--amend`", where the + closing amend no longer sees the fingerprint of the pass immediately before it. + **What this round did not establish, which is most of the mechanism:** whether the hash was + computable at any given pass of (a), whether a computed hash failed to persist, and why (b)'s + close found no fingerprint. The observations are the persisted values at the endpoints, the + counts, and the messages — nothing between them. Per-pass computation, per-pass persistence + and the stored `reviewed` value at each STOP are all unknown. + Reading the hook at the site that computes the hash and returns `unavailable` is the + consumer's own named next step and was not done here. One lead worth carrying: (b)'s STOP arrived at + a `git reset --soft` call, which reaches the reset path only if `is_commit` matched that + command string — so the loose-grep row below is a candidate contributor to (b), and the + two should be read together rather than separately. + *Trigger: a second consumer reporting an unhealable fingerprint, or the next change to + the fingerprint code path.* A hook change either way, which is why this is a row. +- [ ] **A mere mention of "commit" beside a `git` command closes the cycle and resets the pass + counter.** `is_commit()` greps the raw command string for + `(^|[^[:alnum:]])git[[:space:]].*commit` (`plugins/dev-workflow/hooks/codex-gate.sh:756`), + and on a non-WIP match the counters are reset. So a diagnostic `echo` is not merely the + "redundant warning" invariant 2 accepts as the price of firing loose — it destroys the + floor silently. **Measured in the field** (canvas, 2026-08-04): + `git status --porcelain; echo "--- last commit body ---"; git log …` in one Bash call + dropped the counter from **5 to 0**, and the real closing commit then reported "1 + recorded pass". The consumer's memory note `gate-b-echo-resets-pass-counter` names why + this is the dangerous direction: "the loss looks exactly like passes that never ran". + **Second shape, same call-string looseness, already tracked above:** `git add … && git + commit …` chained in one Bash call defeats the docs-only exemption, with two controlled + data points isolating the variable — `29da026` (separate `git add`) → "N/A (docs-only)", + `c2fa18e` (chained) → STOP. That is the timing class the compound-commands row tracks, + and these are cross-consumer instances of it from a second repo, predating that row's + occurrences 3 and 4. That row's occurrence count is deliberately left unedited here, so + this field item carries exactly one disposition; whoever escalates it should count these. + **Third shape rejected as unverified:** the report's heredoc-fires-a-false-STOP claim + resolves, at its cited evidence, to §5's own *warning* about heredocs restated in the + consumer's `CLAUDE.md` and `todos.md`. No measured instance exists. + *Trigger: a second reported counter loss, or any change to `is_commit`.* The fix is a + parse rather than another pattern, so it is a hook change. +- [ ] **The arms-race remedy exists as an observation and not as a procedure.** When a series + of passes stops converging because each correction enables the next finding of the same + shape, the remedy the field found is to change what is being checked rather than to patch + once more: scope the reviewer to changed regions, change the instrument's layer, or + relocate the residual to the layer that already catches it — and **STOP with a named + state** rather than run the next round. Verified at three sites in + `infinite-portfolio-canvas`: its `docs/hardening-taxonomy.md` corollary ("when a tool + enters an arms race with its reviewer, it is checking at the wrong level"), + `.context/a5-t2a-resume.md` (four consecutive passes each finding one more spelling the + claim text seemed to cover; changing the instrument's FORM — printing the patterns, + stating the cause as line- and grammar-local — took findings from **nine to zero**, and + the residual became a named ticket with its own budget whose trigger is a *relapse*), and + `.context/gate-b-a5-t2a-dispositionen.md` (six of six findings on the checker's grammar, + none on product behaviour). Before this round the kit stated the same idea in one place only, + and locally — the invariant-checker escalation row below ("adding one more regex arm per + newly-discovered spelling is *not* the ladder working"); as of this round the story named + below and the amended guard-scope story also state it, so the count is a before-picture and + not a current inventory. + **Parked as a story, not written as prose, because it lands on a decision branch that is + already under design:** as a procedure it changes which rung `harden-finding` picks when + passes stop converging, and that branch is the subject of the guard-scope-precheck story. + Story: `docs/superpowers/stories/2026-08-17-arms-race-remedy-as-procedure-story.md`, + which **inherits from and is inherited by** + `docs/superpowers/stories/2026-08-04-harden-finding-guard-scope-precheck-story.md` — + recorded in both directions on purpose: one decision branch, one design, and whichever is + picked up first must read the other rather than re-deriving it. + *Trigger: either story being picked up, or a third arms-race series observed.* +- [ ] **A handback says nothing about how much context produced it.** When a gate loop stops and + surfaces — §5's `clearly stuck → STOP and surface`, or the new scope stop beside it — the + human decides whether to continue in the same session or start fresh, and one input that + could inform it is absent: how much context the surfacing agent had left. How much that + input actually decides is unestablished, and the evidence below shows only that operators + asked for it repeatedly, not that it changed an outcome. The field adopted + it as a standing field, carried as a documented instruction and nothing stronger — nothing + verifies that an agent supplies the figure, that a reader demands it, or that a self-reported + percentage is accurate: the consumer's + `docs/handoff-cowork.md` carries `Kontext-% (einfordern, falls fehlend)` at line 10 and + `Kontext-% in Übergaben einfordern` at line 72 — demand it, and demand it again when a + handback arrives without it. **This row exists because the round first rejected the item on + a search that missed that file** (it searched the consumer's `.context/`, `todos.md` and + `CLAUDE.md`, where the only percentages are pixel-coverage measurements), and Gate-B pass 1 + caught the rejection. Two things are unsettled, which is why this is a row and not a + sentence in §5: whether a self-reported context figure is reliable enough to steer a + decision, and whether it belongs in §5's stop text, in the handoff-shaped artifacts, or + both. Adding it to §5 mid-cycle would also have put a third rule into a paragraph already + under review. + *Trigger: the next §5 edit touching the stop-and-surface text, or a session continued past + a handback and lost to compaction where the figure would have said to start fresh.* +- [ ] **EXPERIMENTAL — proportionality for findings whose subject is a test instrument.** + Proposed rule: a finding about the instrument rather than about product behaviour gets one + repair round, then is collected — unless it demonstrates a false-green on product + behaviour, which keeps it iterating. Explicitly **not field-proven**; the report marks it + as a proposal, and its motivation is that late cycles spent a growing share of passes on + meta-instruments. Cross-project corroboration for the *motivation only*, from this repo's + own history: PR #23's closing commit records that "of 27 Blocker/Major findings, 16 were + in the never-committed scratch harness, 10 in the design spec's narration, 1 in the plan" + (`7bbdb14`) — 16 of 27 on the instrument. What is unproven is the *remedy*: capping repair + rounds by the finding's subject weakens the Blocker/Major rule that makes the loop + trustworthy, and the false-green carve-out is the whole difficulty rather than a footnote. + *Trigger: a cycle where instrument findings measurably starve product findings of passes, + with both counted.* +- [ ] **EXPERIMENTAL — pre-split heuristic for oversized tranches.** Proposed rule: a tranche + exceeding a size signal (the report suggests >N new runners or steps) is split by subject + at PLAN time. **Its factual premise is rejected as unverified, not merely caveated:** the + report states that every large tranche in the field split anyway and always through an + expensive stop-decide-re-record round-trip, and it cites no tranche records, sizes or + measurements — none were found, and the round did not adopt the frequency or the cost claim. + What is parked is the *proposal only*. The threshold is then the entire remaining question: + a number chosen without measurement would split tranches that did not need it, and this + repo has no measurement of its own to set one. + *Trigger: three tranches in one project splitting mid-execution, with their sizes + recorded* — that is the sample a threshold could be read from. - [x] **A failed Codex call counts as a pass — false ✓ in the firing direction.** **DONE in 0.8.0.** The hook now reads the result before counting. Five classes @@ -311,6 +468,26 @@ backlog. foreground. What it does establish is that calls well past 120 s can return as ordinary foreground results here. The row's deliverable — a `/workflow-init` preflight check — is unbuilt, so the row is not discharged by this. + **SECOND INDEPENDENT FIELD EVIDENCE (2026-08-17) — the row is ripe.** A second consumer, + `infinite-portfolio-canvas`, discovered the variable by measurement rather than from + `README.md`: six real `mcp__codex__review` passes (57 findings, pass 6 clean on the + quality branch) were each auto-backgrounded past 120 s, none reached the hook, and the + closing commit drew a STOP against a structurally zero counter. Setting the variable to + `0` in `~/.claude/settings.json`'s `env` block fixed counting on the following cycles (5 + counted passes, then 24). **What the variable buys, separated, because this row previously + blurred them:** it keeps a long *successful* call in the foreground so its result reaches + the hook and is counted at all — the 0 → 5 → 24 progression is that effect — **and** + separately it removes dependence on C1's residual, the hook recognizing today's wording of + the harness notice. Both are real; neither is the other. + **What this evidence does not show:** a documentation gap. `README.md` § Setup step 2b + already documents the variable, its version requirement, that it is read at process start, + the launch-environment caveat, both usable values and both outcomes — so the honest reading + is that the shipped documentation did not reach an operator who then spent four dated + measurement rounds rediscovering it, which is an argument for the **preflight check** this + row's deliverable already is, not for more prose. No control run with the variable unset was + made in that repo either. + Evidence: `docs/field-reports/2026-08-16-canvas-a1-a5-field-report.md` item 4 and that + repo's `todos.md` § Tooling revalidation. *Trigger: after that story lands* (spec: `docs/superpowers/specs/2026-07-31-failed-codex-call-counts-as-a-pass-design.md`). @@ -321,6 +498,37 @@ backlog. the qualified form as a workaround; file via `/feedback` so the behaviour and the docs stop disagreeing. +- [ ] **Upstream-report candidate: `mcp-codex-dev` error envelopes swallow the reason.** A + failed gate call returns `success: false` with no `error.code`, so neither the operator + nor the hook can tell an exhausted quota from a server defect; the cause is findable only + in `~/.codex/sessions////rollout-*.jsonl` under `rate_limits`. Three confirmed + instances, all from `infinite-portfolio-canvas`'s memory note + `codex-empty-envelope-means-credits` (2026-08-15/16), verified there this round: + (1) an empty `success: false` envelope with no code while `mcp__codex__health` still + reported `ok: true` — the rollout log carried `credits.balance: "0"`, + `has_credits: false`, `codex_error_info: "usage_limit_exceeded"`; it read as a server + defect, and an A5/T2b Gate-B cycle was abandoned after nine passes with the cause open; + (2) the OpenRouter route, the same wall in another shape — `success: false` carrying + progress prose in the `review` field, `unexpected status 402 Payment Required: + Insufficient credits`, `codex_error_info: "other"`; + (3) a `success: true` envelope carrying only progress prose and no answer line, i.e. an + incomplete pass wearing a success envelope. + **The upstream ask is the report's own and is not widened here:** carry the credit or quota + reason in the envelope — `NO_CREDITS` and friends — which covers instances (1) and (2). + **Instance (3) is recorded as a limitation of that ask, not as a second proposal**, because + the report asked for a failure reason on a `success: false` envelope and a `success: true` + one is a different shape that a credit code cannot reach; the report also says to add + nothing beyond its list. What the evidence supports about (3) is only what was seen: one + degeneration emitted `task_complete` with no error and produced no file, and one silent + death emitted no `task_complete` at all — so `task_complete` alone did not separate a + finished pass from an abandoned one in those cases. Whether any combination of artifact, + answer line and log growth is necessary or sufficient was not established, and this row + does not claim it. + Kit-side relevance: the hook's `failure` class can see *that* a gate call failed and never + *why*, which is the same "no error code surfaced (unclassified)" shape this repo met in + 0.8.x use. **Deliberately not fixed here** — both are fields in the server's envelope; file + them against `mcp-codex-dev`, which is Daniel's own server, not this repo's scope. + ## Someday - [ ] **P1 — `workflow-status` → `workflow-next`, staged.** Report where a story stands @@ -346,10 +554,19 @@ backlog. carries this row's conditions with each marked kept, moved or dropped. *Trigger: 10 stories or 20 ledger rows* — below that the sample says more about the last week than about the workflow. -- [ ] **`/capture-finding` as an intake extension of `harden-finding`.** An extension, - not a sibling command: a finding captured outside the ladder is how a ledger - quietly acquires two formats. *Trigger: the first production finding* — one that - arrives from real use rather than from a gate or a bot. +- [x] **`/capture-finding` as an intake extension of `harden-finding`.** **CLOSED 2026-08-17 — + folded into the Finding A story, which is now the single owner of the finding-to-ledger + route.** The trigger fired exactly as written: the canvas A1–A5 field report + (`docs/field-reports/2026-08-16-canvas-a1-a5-field-report.md`) is a production finding set + arriving from real use rather than from a gate or a bot. It fired **and answered itself + negatively**: eleven items were routed by hand through `dev-workflow:intake`, `todos.md` + rows, one upstream note and one reasoned rejection, and capture was never the part that + hurt. What did hurt is durability and reach — a fixed finding surviving a compaction or a + handoff, and a project that never opens PRs never reaching `process-pr-review` step 5 — + and both of those already belong to Finding A. A second command on that route is how the + ledger acquires two formats, which is this row's own stated reason for existing as an + extension; the honest conclusion is that the extension point is Finding A's design, not a + command of its own. ## Tooling revalidation - [ ] Re-check `docs/prompt-standards.md` against the current model-specific @@ -496,6 +713,13 @@ backlog. **TRIGGER FIRED (2026-08-04):** the 2026-08-03 hardening round edits §5. Story: `docs/superpowers/stories/2026-08-04-ledger-route-without-pull-requests-story.md`, which carries this row's conditions with each marked kept, moved or dropped. + **INHERITED 2026-08-17: this story is now the single owner of the finding-to-ledger + route.** The `/capture-finding` row in § Someday closed into it after its own trigger + fired and answered negatively — manual capture of eleven field findings worked, so the gap + is durability across a compaction or handoff and reach into no-PR projects, both of which + are already this story's scope. What it inherits is one constraint, not new scope: whatever + route it designs is the **only** one, because a second capture entry point is how the + ledger acquires two formats. *Trigger: the next round that touches §5, or a project reporting an empty ledger across cycles that fixed findings.* - [ ] **Escalation trigger for the invariant checker — read this before patching it.** From 47e4bf51513b77e9d6d53f1bd76e6a4921d03b0c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20S=C3=A4nger?= <20968534+dsnger@users.noreply.github.com> Date: Wed, 26 Aug 2026 20:11:51 +0200 Subject: [PATCH 4/5] docs(story): widen the path-neutrality check to cover its own criterion MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The acceptance criterion required "every machine-local absolute path replaced by the repo's name" and then offered `grep -c '/Users/'` as the check. That covers one prefix and would miss `/home/`, `/var/folders/`, `/private/` and Windows drive paths, so the check could not establish what the sentence above it claimed — this repo's most-recorded defect class, a check described as proving more than it compares. Old-condition accounting, per the convention this repo applies to any amended decision procedure: - the report must be committed at the named path — KEPT, unchanged - every machine-local absolute path replaced by the repo's name — KEPT, unchanged; this is the requirement, and it was never the thing at fault - the check returns 0 on the committed file — KEPT, and widened from `grep -c '/Users/'` to `grep -cE '/Users/|/home/|/var/folders/|/private/|[A-Za-z]:\\'` - the local-drafts exclusion still holds for other files in that directory — KEPT, unchanged - ADDED: an explicit statement that the five forms are what this environment produces rather than every absolute path a machine could emit, and that the check is a floor while the sentence above it is the criterion. Naming the axes a check covers, and saying whether that list is exhaustive, is required of any sentence describing what a mechanism proves. Nothing this round certified changes: the widened check returns 0 on the committed report, as the narrow one did. Found by CodeRabbit on PR #25 (Major): https://github.com/dsnger/dev-workflow-kit/pull/25#discussion_r3864997865 Amending a criterion already checked off is a human call, per the AC-row precedent set in PR #23. Daniel approved this amendment before it was written. Gate B: N/A. The single staged path is `docs/superpowers/stories/…md`, which is `docs/**.md` — §5's prose exemption, which is defined by path and carries no gate. Staged alone on purpose: a mixed commit would forfeit the exemption. --- .../2026-08-17-field-intake-canvas-a1-a5-report-story.md | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/docs/superpowers/stories/2026-08-17-field-intake-canvas-a1-a5-report-story.md b/docs/superpowers/stories/2026-08-17-field-intake-canvas-a1-a5-report-story.md index 5d84064..2946cce 100644 --- a/docs/superpowers/stories/2026-08-17-field-intake-canvas-a1-a5-report-story.md +++ b/docs/superpowers/stories/2026-08-17-field-intake-canvas-a1-a5-report-story.md @@ -32,9 +32,12 @@ but not answered. ## 3. Acceptance criteria - [ ] The report is committed at `docs/field-reports/2026-08-16-canvas-a1-a5-field-report.md` - with every machine-local absolute path replaced by the repo's name — `grep -c '/Users/'` - on the committed file returns 0 — and the surrounding local-drafts exclusion still - holds for any other file in that directory. + with every machine-local absolute path replaced by the repo's name — + `grep -cE '/Users/|/home/|/var/folders/|/private/|[A-Za-z]:\\'` on the committed file + returns 0 — and the surrounding local-drafts exclusion still holds for any other file + in that directory. Those five forms are the ones this environment produces, not every + absolute path a machine could emit; the check is a floor, and the requirement above it + is the criterion. - [ ] Each of the eleven numbered items carries **exactly one** disposition recorded in the repo: a `todos.md` row with a named trigger, a `docs/hardening-log.md` row appended via `dev-workflow:harden-finding`, a parked story, an upstream-candidate From 6e9551776cb352dea40687ba7c1e39d3610a2cff Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20S=C3=A4nger?= <20968534+dsnger@users.noreply.github.com> Date: Wed, 26 Aug 2026 20:34:06 +0200 Subject: [PATCH 5/5] docs(ledger): record the seventh unverified-enforcement-claim occurrence MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A story acceptance criterion claimed more than its named check could establish: "every machine-local absolute path replaced by the repo's name", checked by `grep -c '/Users/'`. Found by CodeRabbit on PR #25, fixed in 47e4bf5. Recorded at the existing rung, NOT escalated. The reason is guard scope rather than leniency: the fourth-correction rung recorded on 2026-08-16 guards a claim being refined repeatedly, and this was a first correction caught by a reviewer before any refinement spiral began. It sits outside that guard, so the guard did not fail and supplies no escalation — escalating on an incident a rung was never meant to prevent is the over-escalation these rows warn about by name. The row also carries a count note: this is plausibly the third compliance miss against the 2026-07-19 Don't rather than a seventh fresh instance of the class, being the same shape as PR #21's C4 — a criterion outrunning its own named check. That is the trigger the parked C4/C5 row in todos.md names. The annotation to that row is deliberately NOT made here: todos.md is not prose-exempt in a mixed commit, and it belongs to the next gated cycle touching it (the guard-scope story, next in the queue). Gate B: N/A. The single staged path is `docs/hardening-log.md`, which is `docs/**.md` — §5's prose exemption, defined by path and carrying no gate. Staged alone on purpose: a mixed commit would forfeit the exemption. --- docs/hardening-log.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/hardening-log.md b/docs/hardening-log.md index 1c69eeb..fbf874a 100644 --- a/docs/hardening-log.md +++ b/docs/hardening-log.md @@ -119,6 +119,7 @@ escape `\|`, one line), `source` (gate-a|gate-b|bot|manual), | 2026-08-04 | verification-masks-failure | second occurrence, four cases in one cycle: a dry run that defined `$EVIDENCE` itself, proving the git mechanics and never that the plan defines the variable; a regression test that only ever ran under macOS `sh` while the defect it guards appears under `dash`; timed rows whose single-record fixtures never reach the record accumulator's quadratic path; and release evidence claiming `dash` coverage from a run that executed the harness under `dash` and the hook under `/bin/sh` | gate-b | major | P std | CLAUDE.md §5 Profiles counterfactual + the same block in the workflow-init inline template: name the observation that would exist if the claim were false, and confirm the wiring could have produced it. GUARD, exactly: the second half — a check that supplies its own input, runs where the defect cannot appear, or uses a fixture that never reaches the branch it covers. Each of the four cases fails on it. PRIOR ROW: 2026-07-20 (1 prose), whose ref states its own scope — "nothing checks new plans for the same shape" — so all four are OUTSIDE it and this is the fitting rung rather than an escalation on the count. RUNG P NOT 1: the artifact is a prompt; the rung follows the artifact, not the count. SOURCE: two gate-a cases and two gate-b, so the tie-break applies — the triggering case is the `dash` release evidence, which is gate-b. STILL INSTRUCTION-BACKED: nothing tests whether a check could have failed | | 2026-08-04 | mechanical-check-skipped-before-review | NEW CLASS, minted this change: eight read-only Gate-A passes over one plan missed eight defects that thirteen machine checks then found in a single sweep, including a rollback that would have byte-verified against the wrong hook | manual | major | P std | CLAUDE.md §5 Gate-A pass procedure + the same block in the workflow-init inline template: before each read pass, settle mechanically what the artifact asserts and a machine can decide without side effects — cited paths, quoted passages, stated counts, the syntax of standalone fenced blocks — because a read pass spends expensive judgement on what a parser settles in seconds and misses it anyway, inspecting quoted commands rather than running them, since a command quoted in a spec may be destructive or an intentional failure. Class added to docs/hardening-taxonomy.md in this same change, with its boundary against verification-masks-failure stated: there a check ran and could not fail; here the cheap check never ran at all. NO PRIOR ROW — this is the first occurrence. STILL INSTRUCTION-BACKED: nothing runs the sweep, records that it ran, or checks what it settled | | 2026-08-16 | unverified-enforcement-claim | sixth occurrence, and the first where the standing rung FIRED AND HELD: PR #24's Greptile P1 found §5 claiming an empty record-only commit "does not reopen any gate" — true of the obligation, false of the shipped hook, which routes an empty staged-path list through the ordinary Gate-B decision. Four consecutive Gate-B rounds then reproduced the class *inside the correction itself*: an enumeration read as complete ("STOP or below-floor"), a single-cause label for a three-way condition ("stale-fingerprint STOP"), a landing guarantee inferred from the hook's exit status ("so the commit lands"), and a categorical "the hook fires" that is false on the non-adopted and `codex-gate.off` silent paths | bot | major | P std | NO NEW RUNG — `docs/prompt-standards.md`'s existing rule resolved it: "when a claim about a mechanism needs a fourth correction, delete the claim rather than refine it a fifth time". It was applied at exactly the fourth correction and the state-machine narration was deleted from all five restatements, leaving the obligation ("an empty diff raises no review obligation"), the advisory fact, and the `git show --stat` test. This is the row that says the rung WORKS: it was minted after an incident where four corrections each introduced a subtler version of the same claim, and here it stopped the identical spiral at the same round rather than a fifth. What it does NOT do is prevent the first three corrections — it bounds the spiral, it does not catch the original overclaim, and nothing mechanical decides when a claim "needs a fourth correction"; the reviewer counts. Also recorded: deleting the narration made the two §5 copies' edited regions byte-identical, removing the invariant-citation divergence that had itself produced a finding | +| 2026-08-26 | unverified-enforcement-claim | seventh occurrence: a story acceptance criterion required "every machine-local absolute path replaced by the repo's name" and named `grep -c '/Users/'` as its check — one prefix, blind to `/home/`, `/var/folders/`, `/private/` and Windows drive paths, so the command could not establish what the sentence above it claimed. The artifact was clean under the wider pattern, so nothing was certified wrongly; what was wrong was the criterion as a reusable instrument. Plausibly the THIRD compliance miss against the 2026-07-19 Don't rather than a fresh instance of the class — same shape as PR #21's C4, a criterion outrunning its own named check — which is the trigger the parked C4/C5 `todos.md` row names; that annotation is deferred to the next gated cycle touching `todos.md` | bot | major | P std | NO NEW RUNG. The standing rule already covers it: AGENTS.md Don'ts, "Never describe what a gate proves without checking what it actually compares", minted 2026-07-19, together with `docs/prompt-standards.md` item 11 ("Enforcement claims name their mechanism"). Fixed in 47e4bf5 by widening the check to the five forms this environment produces and stating in the criterion that the list is those forms rather than every absolute path a machine could emit. NOT ESCALATED, and the reason is guard scope rather than leniency: the fourth-correction rung recorded on 2026-08-16 guards a claim being refined repeatedly, and this was a FIRST correction caught by a reviewer before any refinement spiral began — outside that guard, so it did not fail and supplies no escalation. Escalating on an incident the rung was never meant to prevent is the over-escalation these rows warn about | | 2026-08-16 | docs-drift | sixth occurrence: PR #24 (CodeRabbit) — `docs/coding-workflow.md` and `docs/sparring-briefing.md` both told the reader to record the reviewer model by reading "the configured value at that moment", while the same section's own timing facts said `mcp-codex-dev` caches its model chain per project root. Two sentences in one document disagreeing, and the wrong one was the actionable instruction: after any model edit the config names a model the running server is not using, so a pass record built from it misstates who reviewed — which is the only thing that makes reviewer-family independence checkable | bot | major | 1 prose | Both documents now name the model the pass *ran under* as the thing recorded, state the cache boundary exactly (launch root at startup, any other root on its first call, cached until restart, and an unseen root is the exception where a post-startup edit does take effect), and name a DETERMINISTIC probe rather than "check the config": `mcp__codex__health` with the same `workingDirectory` as the gate call, reading `checks.config.effective.model`, which is the cached per-root resolution the gate call itself uses. NOT ESCALATED past the 2026-07-26 `2 lint` row: that check guards prose count claims and cannot reach a stale mechanism description, the over-escalation those rows warn about. NO DETERMINISTIC RUNG EXISTS for this: nothing can tell that a sentence about a cache is stale, and the probe is a recipe a human runs, not a check — it raises the floor by making the right value obtainable, and does not close the class. Sibling row this same date under `unverified-enforcement-claim` covers the empty-commit half of the same PR review | | 2026-08-17 | prompt-missing-stop-condition | first row of this base class here: §5's loop told the reader to keep looping and never said what a loop may swallow, so absorbing a finding that opened a new contract question and handing back a three-line repair-of-a-repair were equally defensible readings — field-minted by the kit's heaviest consumer across ~150 gate passes, and its own reason for the rule is that absorbing a contract question spends a decision the loop was never given | manual | major | P std | CLAUDE.md §5 loop paragraph + the same block in the workflow-init inline template. Guard: the absorb-vs-stop rule and the sentence bounding it — a finding correcting the correction just made is absorbed, one opening a new structural or contract question stops the loop, novelty and not size decides, and the stop is NOT an exit from the gate (floor, Blocker/Major filter and clean-final-pass all stand). Does NOT guard which findings count as structural, and says nothing about pass counts. | | 2026-08-17 | prompt-vague-criteria | first row of this base class here: §5 named "clearly stuck" as the loop's terminal state and never said how to recognize one, so a plateauing artifact could burn passes with nobody entitled to call it — in the field one 2848-line spec ran 34 Gate-A spec passes, and over the 19 that were measured the count fell from 43 to an oscillation between 2 and 7 while Blockers fell from 8 to 0-1, i.e. the substance converged and the number never did | manual | major | P std | CLAUDE.md §5 loop paragraph + the same block in the workflow-init inline template. Guard: the recognition rule — after about six passes read the Blocker curve rather than the total, and Blockers at 0-1 with every remaining finding individually fixable IS the stuck state — plus the smaller-specs guidance that follows from the same measurement. Does NOT guard a size threshold (deliberately unmeasured and shipped as guidance), and does not say where a plateau begins. |