Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -215,6 +215,16 @@ reader can judge whether it still holds.
words escapes it entirely. It raises the floor; it does not close the class. The
underlying rule is the check itself: for every sentence about a gate, name the exact
comparison the code performs, and delete any part of the sentence that outruns it.
- **Never replace a decision procedure without accounting for its old conditions.** List
what the previous prose required, then mark each one kept, moved, or deliberately
dropped. A rewrite reliably preserves the condition that motivated it and silently loses
the others: in the profiles cycle this happened ten times across `process-pr-review`,
§5, the spec and a story criterion — once making an eligible profile *sufficient* for a
Gate-B skip, which is the gate-off path that change existed to close. Amending one
criterion also does not touch the prose that motivated it, so check the surrounding
paragraphs of any rule you amend. No tool decides this: a dropped condition looks
exactly like text that was never there.

- **Never rename or delete a doc section without grepping for references first.**
`ci.yml` once pointed at a deleted README section; `MANIFEST.md` listed a `CLAUDE.md`
that did not exist. Docs-drift is this plugin's own taxonomy class and this repo is
Expand Down
9 changes: 9 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -234,6 +234,15 @@ not all of `.context/`, which would strip the committed `codex-gate.on` adoption
`NO FINDINGS` if clean" in `additionalContext`, with the same one-line format.
You filter to Blocker/Major, Codex never does.

**Standing lens, every Gate-B call: "which existing statements does this diff falsify?"**
A change makes sentences wrong in files it never touches. Checks scoped to the edited
paths — a parity diff, a resync, a grep of your own edits — do not look there, because
the file was correct until your change landed elsewhere. This lens is prompt text: it
asks, nothing enforces the ask or validates the answer, and no comprehensive check
covers arbitrary semantic drift. Ask anyway — in one cycle it surfaced a shipped command
that would have let a one-line fix skip Gate B entirely, plus two user-facing docs
teaching a rule the same change had just narrowed.

**What counts as prose (the only Gate-B exemption).** Every staged path is
explanatory documentation — `docs/**.md`, `README.md`, `MANIFEST.md` → N/A.
These describe the product rather than being it, so a wrong sentence costs a
Expand Down
3 changes: 3 additions & 0 deletions docs/hardening-log.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,3 +29,6 @@ escape `\|`, one line), `source` (gate-a|gate-b|bot|manual),
| 2026-07-26 | unverified-enforcement-claim | third occurrence, resolved: a prompt asserting it follows docs/prompt-standards.md while naming no executing model — the PR #12 sparring-briefing said `Target model: any capable chat model` | bot | major | 2 lint | scripts/check-invariants.sh check 4a + fixtures in .test.sh. **Resolves the 2026-07-25 `pending` row** (same fingerprint, appended not edited). GUARDED SPELLING, exactly: in a `*.md` file containing the literal `prompt artifact and follows`, the column-zero `^Target model:` line must appear exactly once, its value must BEGIN with a tested token (`Claude\|Codex\|GPT`, portable ERE boundary — token-anywhere is not enough, since the real defect line contained "Claude" as provenance), and it must name exactly one DISTINCT token. STILL INSTRUCTION-BACKED: a bare `Target model: Claude` naming no execution surface passes; a value whose prose is wrong in a way no token test sees passes; a column-zero declaration inside a fenced block counts (no fence awareness in POSIX grep); an unknown future model fails until the token list is extended. BLIND SPOT, stated because it is structural: this ledger is EXCLUDED from the check it records — a ledger that quotes defects self-rejects the checks that detect them, and scanning it would fail the repo forever on rows like this one. OVER-ESCALATION WARNING: `harden-finding` compares fingerprints only, so a later in-class defect OUTSIDE the guarded spelling will be proposed for a stronger rung than anything justifies; that is a proposal a human validates, and this row names the guard so they can judge it. The skill's scope-blind recurrence rule is parked in todos.md |
| 2026-07-26 | docs-drift | third occurrence, resolved: a prose count contradicting the checklist it counts — PR #12's `docs/prompt-standards.md` said "all ten items" while its own checklist ran 1-12 | bot | minor | 2 lint | scripts/check-invariants.sh check 4b + fixtures in .test.sh. **Resolves the 2026-07-25 `pending` row** (same fingerprint, appended not edited). GUARDED SPELLING, exactly: two-stage claim recognition over `*.md` — stage 1 matches `all <digits\|one..twenty>( checklist)? items` with outer word boundaries, stage 2 requires canonical decimal, so `all 012 items` fires as malformed rather than being invisible; every recognised claim must equal N, where N comes from exactly one `^## Checklist([[:space:]].*)?$` heading whose body holds labels forming the contiguous canonical sequence 1..N, in BOTH docs/prompt-standards.md and the workflow-init template, which must agree. Word forms are in scope BECAUSE the motivating occurrence was one — a digit-only check would have sailed past the very defect this row claims to harden. STILL INSTRUCTION-BACKED: counts above twenty in word form, ordinals, hyphenated compounds (`all twenty-one items`), split-line claims, and any count of something other than this checklist. Same ledger blind spot and over-escalation warning as the row above |
| 2026-07-26 | session-bound-context-not-durable | Gate A/B dispositions and interrupted-cycle state lived only in chat history, so a dismissal's reasoning and a cycle's resume point died with the session; the field had independently invented per-pass companion files (7 dispositions files and a Gate-A resume note in infinite-portfolio-canvas) because the shipped protocol knew only about the findings file | manual | minor | P std | CLAUDE.md §5 "Optional companions, from field practice" + the same block in the workflow-init inline template: `<slot>-dispositions.md` and a cycle-stable resume note (`gate-a-spec-resume.md`, `gate-a-plan-resume.md`, `gate-b-resume.md`). MECHANISM, stated exactly: this is a paragraph of prompt text and **nothing enforces it** — the companions are optional, no step requires one to exist, none participates in pass validation, and a cycle that writes none behaves exactly as before. It recommends durable notes; it does not make context durable. Reading `P std` as more than that would be `unverified-enforcement-claim`, the class resolved at rung 2 in #13. BLIND SPOT inherited from #13: checks 4a/4b do not scan docs/hardening-log.md, so nothing mechanical reads this row either. Gate A rejected `truncated-tool-output-read-as-complete` for this: nothing is truncated, the content was never written |
| 2026-07-27 | unverified-enforcement-claim | FOURTH occurrence: a methodology paragraph describing the gate hook's path matcher was corrected in four consecutive Gate-B rounds — false rationale, then a categorical claim that agent definitions and hook messages always fire Gate B, then a partial-matcher claim ("a bare `agents/` matches nothing", while non-`.md` paths reach the reminder anyway), then an enforcement blur between an advisory hook and the policy — each correction a subtler version of the last, inside a paragraph being rewritten to fix that very class | gate-b | major | P std | docs/prompt-standards.md item 11, new sub-paragraph "Cite the source; do not paraphrase it" + "when a claim about a mechanism needs a fourth correction, delete the claim rather than refine it a fifth time". Sharpens item 11 rather than adding item 13, deliberately: a new numbered item ripples the checklist count through prompt-standards, the workflow-init template and AGENTS.md invariant 11, all pinned by check 4b. COUNT, stated because the escalation depends on it: the prior lineage is 2026-07-18 (first), 2026-07-19 (second), 2026-07-25 (third, `pending`) and 2026-07-26, which RESOLVES that pending row rather than recording a new defect — so this is the fourth occurrence, not the fifth. NOT ESCALATED to a mechanical rung despite the 2026-07-26 `2 lint` row being the latest: that check guards one spelling (`Target model:` lines) and this defect is outside it — exactly the over-escalation the 2026-07-26 row warns about by name. NO CURRENT DETERMINISTIC RUNG covers this paraphrase shape — the one candidate evaluated, a grep for the matcher's terms, fires on the matcher and the policy file themselves; that rules out the naive check, not every conceivable one, and the recurrence count here is a signal a human reads rather than anything that blocks a commit. The fix that finally held was DELETION — the enumeration was replaced by a pointer to the policy file; the same shape closed a second instance in the same cycle, where three successive `git status` recipes were replaced by naming the paths you edited |
| 2026-07-27 | rewrite-drops-prior-condition | NEW CLASS, minted this change: ten instances in one cycle where a multi-condition rule was rewritten, the new condition survived and an old one vanished with the prose it replaced — `process-pr-review` losing the no-story and all-stories branches, then losing fix-triviality so an eligible profile alone authorized a Gate-B skip (a gate-off path invented by the change that exists to close one); a spec surfaces sentence describing the skip as keyed "on the story profile, not on fix size"; story AC 6 dropping triviality in the very edit that added effective-level eligibility; and the §5 authority leaving triviality implicit inside the phrase "triviality skip", which is what every downstream summary copied and then lost | gate-b | major | 1 prose | AGENTS.md Don'ts, "Never replace a decision procedure without accounting for its old conditions" — list what the previous prose required and mark each kept, moved, or deliberately dropped; also check the paragraphs around any criterion you amend, since amending one does not touch the prose that motivated it. Class added to docs/hardening-taxonomy.md in this same change, with its boundary against `docs-drift` stated (drift is two artifacts disagreeing and is fixed by updating the stale one; this is one rewritten rule wrong on its own terms, fixed by restoring the condition). NO DETERMINISTIC RUNG: a dropped condition is textually indistinguishable from text that was never there, so no grep or checker decides it |
| 2026-07-27 | docs-drift | FOURTH occurrence, different mechanism from the first three: a change invalidated sentences in files it never touched — `plugins/dev-workflow/commands/process-pr-review.md` (a SHIPPED command still skipping Gate B on fix size alone, which would have let a one-line fix on a security-relevant story bypass the narrowed rule entirely), `docs/getting-started.md` twice, and `docs/coding-workflow.md`. Every one was correct before the change and false after it | gate-b | major | P std | CLAUDE.md §5 Gate B + the workflow-init inline template: a standing lens on every Gate-B call — "which existing statements does this diff falsify?". A TARGETED ASK found all four; the checks that ran — parity diffs, resyncs, greps of the edited paths — are scoped to files the change touched and do not look where these live, since those files were correct until the change landed elsewhere and nothing in the diff points at them. WHAT THIS RUNG DOES NOT DO, stated because the lens is the whole hardening: it is prompt text, nothing enforces that a reviewer asks it or checks the answer, and it guarantees no discovery. A narrow deterministic check could pin one stale spelling the way check 4b pins one count spelling; what does not exist is a comprehensive check for arbitrary semantic drift, which is why the rung is P and not 2. NOT ESCALATED past the 2026-07-26 `2 lint` row for the same reason as the sibling row above: that check guards prose count claims, and this defect is outside its spelling — the over-escalation the row warns about. Escalation from the 2026-07-18 `P std` row (spec-updated-with-fix) is real but narrow: that rule reaches the spec of the change being made, not arbitrary prose elsewhere in the repo |
12 changes: 12 additions & 0 deletions docs/hardening-taxonomy.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,18 @@ the synonyms a future reader might search for instead.
(Defined here retroactively: the 2026-07-20 ledger row used this class before any
definition existed. Recorded now so the recurrence grep has something to land on.)

- `rewrite-drops-prior-condition` — a rule with several conditions is rewritten, the new
condition survives, and one of the conditions the old prose carried disappears with it. Aliases: lost
precondition, half a rule, the summary that kept the new half, condition dropped in
restatement, "we fixed it and it got weaker".
Comment thread
coderabbitai[bot] marked this conversation as resolved.

**Not `docs-drift`.** Drift is two artifacts disagreeing and is fixed by updating the
stale one; here a single rewritten rule is wrong on its own terms, and the fix is to
restore the condition. Detect it by diffing the old prose against the new for
conditions, not by comparing artifacts. Ten instances in the profiles cycle, one of
which briefly authorized a substantial fix to skip Gate B because its story's profile
was eligible — a gate-off path invented by a rewrite meant to narrow one.

- `session-bound-context-not-durable` — reasoning that has to outlive a session is left in
chat history, so it is gone at the next session, model change or tool switch. Aliases:
lost rationale, why-did-we-dismiss-that, undocumented decision, no resume state,
Expand Down
24 changes: 17 additions & 7 deletions docs/pr-review-bots.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ uses hangs the loop, and treating a channel as context silently drops real findi

| Bot | Enabled | Where findings appear | Notes (plan/tier limits, completion signal, quirks) |
|---|---|---|---|
| CodeRabbit | yes | **inline** | Observed on PR #1 (plan: Pro Plus, profile CHILL): posts real inline review comments on the diff, each with severity and a committable suggestion, plus a walkthrough summary comment. Read the inline comments — the walkthrough is not a findings source. **Its status check can pass while the comment says "Review rate limited" — see the completion-signal note: a green check does not prove the final head was reviewed.** |
| CodeRabbit | yes | **inline** | Observed on PR #1 (plan: Pro Plus, profile CHILL): posts real inline review comments on the diff, each with severity and a committable suggestion, plus a walkthrough summary comment. Read the inline comments — the walkthrough is not a findings source. **Its status check passes while the comment says "Review rate limited" — observed on four PRs (#12, #13, #15, #16), so treat it as this bot's normal behaviour rather than an edge case. A green check does not prove the final head was reviewed; the review count is the arbiter. See the completion-signal note.** |
| Greptile | yes | **summary always; inline usually** | Four PRs observed (#1, #2, #4, #5): a PR-level **summary comment every time**, with findings sometimes only inside it under "Comments Outside Diff". **Inline** comments on #2 (1), #4 (2), #5 (1) but **none on #1** — so inline is usual, not guaranteed. Read both channels; the summary is the one that has never been missing. **Completion signal: none you can block on.** `gh pr checks` displayed a "Greptile Review" entry for #4 and #5, but the check-runs and statuses APIs return no Greptile entry for any of those heads — the two tools disagree, so neither proves it has finished. Posts within ~4–11 min. |
| Cursor Bugbot | no | n/a | Comments only to say it is disabled for this account. Ignore. |

Expand All @@ -29,14 +29,24 @@ uses hangs the loop, and treating a channel as context silently drops real findi
you can block on it. **Two different things, and conflating them merges unreviewed heads.**

- *The check stopped pending* — the blocking signal. Block on this.
- *The final head was reviewed* — a separate verification. Observed on #12 and #13: the
check passed while the issue comment read "Review rate limited", and on #13 the only
CodeRabbit **review record** carried `commit_id` `eed589c` while the merged head was
`92de0d2`. The head that merged was never reviewed, and the green check said nothing
about it.
- *The final head was reviewed* — a separate verification, and the one that decides
whether you may merge. **This is settled behaviour, not a hazard that might occur:
never merge on the check alone — the review count is the arbiter.** Four occurrences,
the last two caught by running the count rather than by luck:
- **#12 and #13** — the check passed while the issue comment read "Review rate
limited"; on #13 the only CodeRabbit **review record** carried `commit_id` `eed589c`
while the merged head was `92de0d2`. The head that merged was never reviewed.
- **#15** — green check, "Review rate limited", and the count returned `0` for the live
head. Merging there would have shipped an unreviewed head; waiting until the count
reached `1` cost about three minutes.
- **#16** — the same, observed while writing this row: green check, "Review rate
limited", count `0`.

The check and the review are independent facts, and the check is the one that lies.
What the green tick establishes is that CodeRabbit's *check* finished — nothing more.

Verify the second before merging — a deterministic boolean, so it can gate rather than be
eyeballed:
eyeballed. Run it on every merge, including the ones where the check looks unambiguous:

```sh
head=$(gh pr view <n> --json headRefOid --jq .headRefOid) # the LIVE head, not local HEAD
Expand Down
18 changes: 18 additions & 0 deletions docs/prompt-standards.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,24 @@ Living references (consult, don't copy — copies go stale):
When the mechanism turns out not to exist, say what actually happens instead —
"this is a rule the agent keeps; nothing counts for it".

**Where the reader can reach the authoritative source, cite it instead of restating
it.** When user-facing text needs a rule that lives somewhere authoritative — a
classifier, a policy section, a config — point at that source. Every restatement is a
copy that can drift and a fresh chance to overclaim. **Exception, by invariant 8:**
text that must be self-contained for its reader — the inline templates
`/workflow-init` scaffolds, which cannot point at a file the downstream project does
not have — restates deliberately and keeps the copies in sync instead. **And when a claim about a mechanism needs a fourth
correction, delete the claim rather than refine it a fifth time.** Both halves come
from one incident: a methodology paragraph describing the gate hook's path matcher was
corrected in four consecutive review rounds — a false rationale, then a categorical
claim, then a partial-matcher claim, then an enforcement blur — each correction a
subtler version of the last, in a paragraph being rewritten to fix that very class. It
closed only when the enumeration was removed and replaced with a pointer to the policy
file. No *current* deterministic rung covers this paraphrase shape — the one candidate
evaluated, a grep for the matcher's terms, fires on the matcher and the policy file
themselves — and the ledger's recurrence count is a signal a human reads, not a check
that blocks anything.

Worth knowing how that count was arrived at: it was first asserted as six without
checking, then "corrected" to five from memory, and only settled at six by reading
the commits. Both wrong numbers were themselves unverified claims — which is the
Expand Down
2 changes: 1 addition & 1 deletion plugins/dev-workflow/.claude-plugin/plugin.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "dev-workflow",
"displayName": "Cross-Model Review Workflow",
"version": "0.7.0",
"version": "0.7.1",
"description": "Spec-driven workflow with two independent cross-model review gates, an append-only hardening ledger with an escalation ladder, and repo-enforced quality. Requires the superpowers plugin.",
"author": {
"name": "Daniel Sänger",
Expand Down
Loading