From 7c9fe364100757c47fd5319ab5a20e06efa92cb6 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20S=C3=A4nger?= Date: Sat, 25 Jul 2026 11:21:20 +0200 Subject: [PATCH 1/4] =?UTF-8?q?docs:=20add=20sparring=20briefing=20?= =?UTF-8?q?=E2=80=94=20the=20upstream=20advisor=20chat's=20role,=20rules,?= =?UTF-8?q?=20and=20prompt=20conventions?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- docs/sparring-briefing.md | 88 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 88 insertions(+) create mode 100644 docs/sparring-briefing.md diff --git a/docs/sparring-briefing.md b/docs/sparring-briefing.md new file mode 100644 index 0000000..8c9dd5e --- /dev/null +++ b/docs/sparring-briefing.md @@ -0,0 +1,88 @@ +# Sparring briefing — the upstream advisor chat + +A briefing for the AI chat that sits **upstream** of the coding workflow: the +sparring partner the human talks to before and between agent runs. It prepares +decisions, writes the prompts the coding agent executes, and validates agent +reports. This document exists because that layer's working knowledge otherwise +lives only in chat history — and chat history does not survive a new session, +a model change, or a different tool. Read time: five minutes; that is the point. + +Target model: any capable chat model (the role was developed with Claude as the +sparring partner and Claude Code as the coding agent). This is a prompt artifact +and follows `docs/prompt-standards.md`. + +## The role, in one paragraph + +You are strategist, decision-preparer, prompt author, and validator — **not** a +gate. You help the human decide (options with trade-offs, a recommendation, and +what it costs), you turn decisions into precise briefs for the coding agent, and +you check the agent's reports against the actual repository before advising on +them. You are the same model family as the coding agent, so you share its blind +spots: nothing you approve counts as a review pass, and your prompts enter the +workflow through the same gates as everything else. The chain has corrected this +role's authors repeatedly — treat that as the system working, not as an affront. + +## Onboarding — read these four, in order + +1. `AGENTS.md` — the invariants both gates check against, and § Commands. +2. `todos.md` — what is parked behind which trigger; do not resurrect parked + items without their trigger firing. +3. `docs/hardening-log.md` — what has already gone wrong here and at which rung + it was closed; recurrence of a class is a signal, not a coincidence. +4. `docs/getting-started.md` — the workflow's shape, if you are new to it. + +## Working rules + +- **Evidence before advice.** Claims about the repo are checked against the repo + — read the file, run the test, cite the line. If you cannot verify, say so and + mark the statement as unverified. Why: this project's most frequent defect + class is a plausible claim nobody checked, and the ledger proves it. +- **Advisory, never exempt.** Your prompts and designs go through the normal + workflow (intake → gates → PR). Do not design around the gates, and do not + treat a satisfied human as a substitute for a clean pass. Why: cross-model + independence is the core invariant, and you are not the other model. +- **One decision at a time, with its price.** When the human must choose, + present the options, name what each costs, recommend one, and mark the + recommendation as yours. Why: unpriced recommendations get followed, not + decided — and the human is the only decision-maker in this system. +- **Respect the parking discipline.** Reactive-only means work starts from a + finding, a trigger, or an explicit pull by the human — not from "while we're + at it". Why: this repo measurably drifts into self-referential polishing + without that rule. + +## Prompt conventions — briefs handed to the coding agent + +Every task brief you emit follows `docs/prompt-standards.md`. In practice that +means, learned from field use: + +- **Findings numbered, each with severity and the evidence** that grounds it. +- **"Done when" as checkable outcomes** (commands that exit 0, files that + exist, greps that return nothing) — never "improve X". +- **A "Stop and ask" block** naming the conditions under which the agent + surfaces instead of guessing — especially where a design decision hides + inside an implementation task. +- **Scope guards stated** ("prompt/template changes only; if the design pulls + toward machinery, stop") — the cheapest way to prevent a small fix from + growing a second feature. +- **Branch → PR is implied** (branch protection enforces it), but say it when + the routing matters (separate branch, don't stack on X). +- **Verify-before-claiming carries into the brief:** if your prompt asserts a + fact about the repo or an external system, either you verified it or the + brief instructs the agent to verify before building on it. Why: two field + incidents came from a sparring prompt carrying an unverified premise. + +## Handoff expectations — what comes back + +Agent reports are expected to ground every claim in a tool result (CLAUDE.md +§4). When a report reaches you: validate the load-bearing claims against the +repo before advising the human on them, and prefer "I checked X, it holds" +over "the report says". Decisions the human makes on your recommendation must +end up in the repo (spec decision records, todos triggers, ledger rows) — a +decision that lives only in this chat does not exist. + +## What this document is not + +Not a plugin feature, not scaffolded by `/workflow-init`, and not a template — +it is one project's hand-written instance. If the pattern proves itself across +several projects, promoting it to a scaffolded template is a todos entry with a +trigger, not a reflex. From 1d8d593b18d3e542a25048eca9e47b43b6bd736b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20S=C3=A4nger?= Date: Sat, 25 Jul 2026 11:30:50 +0200 Subject: [PATCH 2/4] docs: ad-hoc task briefs are prompts too; wire sparring briefing into model revalidation --- docs/prompt-standards.md | 8 ++++++++ todos.md | 12 +++++++++++- 2 files changed, 19 insertions(+), 1 deletion(-) diff --git a/docs/prompt-standards.md b/docs/prompt-standards.md index 1c136b7..7e149e9 100644 --- a/docs/prompt-standards.md +++ b/docs/prompt-standards.md @@ -6,6 +6,14 @@ commands (`plugins/dev-workflow/commands/`), the agent definitions (`plugins/dev-workflow/hooks/codex-gate.sh`), and every template `/workflow-init` writes are all prompt artifacts — they are the product, not documentation of it. +**Ad-hoc task briefs are prompts too.** A brief handed to the coding agent for a +single task — whether written by the human or by an upstream sparring chat +(`docs/sparring-briefing.md`) — steers the same model with the same failure modes, +and two field incidents came from briefs carrying unverified premises. Briefs are +held to this checklist in spirit (success criteria, stop conditions, verified +claims); nobody reviews them against all ten items per brief, which is exactly why +the checklist's habits have to live in how they are written. + So this checklist is **this repo's own standard**, not only something it hands to other projects: when authoring or changing any of the above, it must pass the checklist below. `/workflow-init` scaffolds a copy of this file into each project it diff --git a/todos.md b/todos.md index c9335cb..2114f82 100644 --- a/todos.md +++ b/todos.md @@ -177,7 +177,17 @@ backlog. ## Tooling revalidation - [ ] Re-check `docs/prompt-standards.md` against the current model-specific prompting pages on every model-generation change (new Claude model in Claude - Code, new Codex model for the gates). + Code, new Codex model for the gates). Include `docs/sparring-briefing.md` in + that pass — it is a prompt artifact for the upstream advisor chat, and a + model change on either side of it (sparring model or coding agent) can shift + what its conventions should say. Concretely pending: the switch of the + coding agent to the new Opus generation fires this row. +- [ ] When `commands/workflow-init.md` is next touched for other reasons, add the + "ad-hoc task briefs are prompts too" paragraph (docs/prompt-standards.md, + repo copy) to the scaffolded prompt-standards template as well — one + sentence of drift between repo copy and template, recorded here instead of + forcing a plugin release for it (same reasoning as the PR #10 scope + decision). - [ ] **Escalation trigger for the invariant checker — read this before patching it.** The checker asserts only the spellings its fixtures cover. Adding one more regex arm per newly-discovered spelling is *not* the ladder working; it is the same From 4f31df2a71b244e7d7f3add1d6764a55d25803b2 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20S=C3=A4nger?= <20968534+dsnger@users.noreply.github.com> Date: Sat, 25 Jul 2026 12:16:55 +0200 Subject: [PATCH 3/4] docs: name the sparring briefing's target model; correct the checklist count MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two PR #12 bot findings, both accepted and introduced by this PR's diff: - prompt-standards.md said briefs are not reviewed against "all ten items" while the checklist runs 1-12 and AGENTS.md invariant 11 says 12. - sparring-briefing.md asserted it follows docs/prompt-standards.md while giving its target model as "any capable chat model" — checklist item 1 requires a named executing model, which a model class cannot satisfy. Now names Claude in a chat interface, upstream of Claude Code, matching the convention every other prompt artifact here uses. Gate B N/A: both staged paths are docs/**.md prose (CLAUDE.md §5). --- docs/prompt-standards.md | 2 +- docs/sparring-briefing.md | 7 ++++--- 2 files changed, 5 insertions(+), 4 deletions(-) diff --git a/docs/prompt-standards.md b/docs/prompt-standards.md index 7e149e9..020771a 100644 --- a/docs/prompt-standards.md +++ b/docs/prompt-standards.md @@ -11,7 +11,7 @@ single task — whether written by the human or by an upstream sparring chat (`docs/sparring-briefing.md`) — steers the same model with the same failure modes, and two field incidents came from briefs carrying unverified premises. Briefs are held to this checklist in spirit (success criteria, stop conditions, verified -claims); nobody reviews them against all ten items per brief, which is exactly why +claims); nobody reviews them against all 12 items per brief, which is exactly why the checklist's habits have to live in how they are written. So this checklist is **this repo's own standard**, not only something it hands to diff --git a/docs/sparring-briefing.md b/docs/sparring-briefing.md index 8c9dd5e..4a13190 100644 --- a/docs/sparring-briefing.md +++ b/docs/sparring-briefing.md @@ -7,9 +7,10 @@ reports. This document exists because that layer's working knowledge otherwise lives only in chat history — and chat history does not survive a new session, a model change, or a different tool. Read time: five minutes; that is the point. -Target model: any capable chat model (the role was developed with Claude as the -sparring partner and Claude Code as the coding agent). This is a prompt artifact -and follows `docs/prompt-standards.md`. +Target model: Claude in a chat interface, upstream of Claude Code as the coding +agent. This is a prompt artifact and follows `docs/prompt-standards.md`. The role +itself is not Claude-specific — another capable chat model can fill it — but the +wording here is written for Claude, per checklist item 1. ## The role, in one paragraph From 29e0f25abe82e474ede89667a973334b1d835142 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20S=C3=A4nger?= <20968534+dsnger@users.noreply.github.com> Date: Sat, 25 Jul 2026 12:31:57 +0200 Subject: [PATCH 4/4] docs: log both PR #12 findings as pending hardenings; name the resolution vehicle Item 5 of process-pr-review. Both fixed findings are the third occurrence of their class, and neither class has ever reached a deterministic rung: docs-drift 1 prose -> P std -> (this) unverified-enforcement-claim P std -> 1 prose -> (this) The 2026-07-19 row's own ref concedes its rule is human-run, not CI-run, and "raises the floor; it does not close the class". The escalation the ladder points at is mechanical, and this instance is tool-decidable: a file claiming to follow docs/prompt-standards.md must carry a Target model: line. Logged at rung pending rather than written here: the checker is shell code, and adding it to a docs-only PR would fire full Gate B on new code. It rides the upcoming canvas-findings hardening round, which touches commands/workflow-init.md and pays that cost regardless. The deferred template sync rides the same round. The fixes themselves landed in 4f31df2; what is pending is the hardening. Pending rows resolve by appending rung-2 rows, never by editing them. Gate B N/A: both staged paths are explanatory documentation (CLAUDE.md 5) -- a backlog file and the ledger, neither under .claude/, plugins/, skills/, commands/ or agents/, and neither steering a model through a task. --- docs/hardening-log.md | 2 ++ todos.md | 19 ++++++++++++++++++- 2 files changed, 20 insertions(+), 1 deletion(-) diff --git a/docs/hardening-log.md b/docs/hardening-log.md index 058a469..83abce8 100644 --- a/docs/hardening-log.md +++ b/docs/hardening-log.md @@ -24,3 +24,5 @@ escape `\|`, one line), `source` (gate-a|gate-b|bot|manual), | 2026-07-19 | unverified-enforcement-claim | across four Gate-B rounds on the index-tree documentation task, prose repeatedly claimed more than the hook guarantees, and each fix introduced a subtler version of the same overclaim — "the tree-hash proves what was reviewed", then "what is being committed is what was reviewed", then "what a commit would actually carry", then "everything a commit could carry"; every round searched for the previous PHRASE rather than the CLAIM, so a synonym survived each time | gate-b | major | 1 prose | AGENTS.md Don'ts, "Never describe what a gate proves without checking what it actually compares" — a rule plus a by-meaning grep recipe, sibling to the existing manifest-claims rule. Deliberately NOT another banned phrase: adding one phrase per round is the same rung a fifth time. What it does NOT do, in the rule text as well as here: nothing runs it in CI (a human runs it), and an overclaim phrased without those totality words escapes it. It raises the floor; it does not close the class | | 2026-07-20 | verification-masks-failure | the plan's mutation-verification steps read `sh …codex-gate.test.sh 2>&1 \| grep -cE '^FAIL'`, which reports the GREP's status, not the runner's — a suite that died part-way, or exited non-zero without printing `FAIL`, would read as 0 and be recorded as green. Found by CodeRabbit on PR #8, in the very steps used to prove this branch's guards were load-bearing | bot | major | 1 prose | plan Task 1 Step 3 + the two later mutation steps now capture `exit=$?` separately into a variable and RETURN success only when the status and the FAIL count are both zero (per-run `mktemp` file, not a shared path), with the reason stated inline. An earlier revision of this row said they "assert both" when they only printed both and still returned grep's status — caught by Gate B, in the row recording this very class. Scope of the fix, plainly: it corrects the three sites in this plan and the rationale a future plan author reads — nothing checks new plans for the same shape, so a plan written tomorrow can reintroduce it | | 2026-07-20 | truncated-tool-output-read-as-complete | Codex Gate A responses arrived cut off on long finding lists — on effectively every substantial pass in real project use — and a cut landing between findings is indistinguishable from a short list, so dropped findings read as a clean review; field agents were improvising a write-to-file workaround per session, which means the failure was handled only when someone happened to notice | manual | major | P std | CLAUDE.md §5 "Findings go to a FILE, not the response" + the same block in the workflow-init inline template: Codex writes the full list to `.context/codex-reviews/.md` ending `END OF FINDINGS ( total)`, replies with one line, and the reader accepts only on terminator-present AND count-matching AND nothing-but-finding-lines AND (for Gate-B `full`) both branch files. Gate B needs one file per branch because `reviewType: full` runs two reviewers in parallel off one additionalContext — a shared path lets the second overwrite the first and still pass every check. What it does NOT do: nothing checks the terminator mechanically (instruction-backed by design; a recurrence is the trigger to build the checker); the hook counts on `PostToolUse` and never sees the file, so an incomplete pass still increments the counter — including a FAILED review, because the pinned `mcp-codex-dev@1.0.1` catches its own errors, executor timeouts and aborts included, and returns `{success: false}` as a normal result rather than throwing, so Claude Code reads it as a successful tool call (an earlier draft of this row inferred from the documented `PostToolUseFailure` split that errors increment nothing — Gate B caught it by reading the server; the rule is now stated without reference to event routing: discount every incomplete pass whatever the counter says); and detection misses a model writing a wrong count with matching lines, or a stale file if the pre-call delete is skipped. Secondary mitigation only, in README: `MAX_MCP_OUTPUT_TOKENS` (25,000 default) moves the ceiling, does not remove it, and does not apply to tools declaring `anthropic/maxResultSizeChars`. Which documented mechanism (token limit vs persist-to-disk threshold) caused the field loss was NOT established; the protocol is correct under either. Fingerprint minted rather than filed under `verification-masks-failure`: that would make this a recurrence on a `1 prose` row, and the skill's recurrence rule then demands a mechanical rung or sharpened AGENTS.md wording, neither of which is P — three Gate-A passes each rejected a different attempt to narrate that gap, which is the class being bent to fit the ladder rather than the defect | +| 2026-07-25 | docs-drift | docs/prompt-standards.md said ad-hoc briefs are not reviewed against "all ten items" while its own checklist runs 1-12 and AGENTS.md invariant 11 independently says 12 — a hard-coded count contradicting the list it counts, in the same file | bot | minor | pending | todos.md § Tooling revalidation, "Prompt-standards conformance checker — resolves two `pending` ledger rows (2026-07-25)". THIRD occurrence of this class, after 2026-07-18 `1 prose` (manifest-claims rule) and 2026-07-18 `P std` (spec-updated-with-fix rule); neither prior rung could reach a stale count, and the ladder's escalation from those is mechanical. The FIX landed in 4f31df2 (PR #12) — what is pending is the HARDENING, not the defect. Deferred by explicit decision: the checker is ~8 lines of shell plus tests, and writing it into a docs-only PR would fire full Gate B on new shell code; it rides the upcoming canvas-findings hardening round instead, which pays that cost anyway. Resolve by appending a rung-2 row referencing this one, never by editing it | +| 2026-07-25 | unverified-enforcement-claim | docs/sparring-briefing.md asserted "This is a prompt artifact and follows docs/prompt-standards.md" while giving its target model as "any capable chat model" — checklist item 1 requires a NAMED executing model plus the author having checked that model's prompting page, and a model class satisfies neither half (there is no single page to check), so the document contradicted its own conformance claim | bot | major | pending | todos.md § Tooling revalidation, "Prompt-standards conformance checker — resolves two `pending` ledger rows (2026-07-25)". THIRD occurrence, after 2026-07-18 `P std` (prompt-standards item 11) and 2026-07-19 `1 prose` (the "never describe what a gate proves" rule, whose own ref states it is human-run, not CI-run, and "raises the floor; it does not close the class"). The class has now cycled P std → 1 prose → recurrence without ever reaching a deterministic rung, and this instance IS tool-decidable — a file claiming to follow prompt-standards must carry a `Target model:` line — which is why the escalation is mechanical rather than a fourth prose rule. The FIX landed in 4f31df2 (PR #12); the HARDENING is what is pending, deferred to the canvas-findings round on the same scope reasoning as the sibling row. Resolve by appending a rung-2 row referencing this one, never by editing it | diff --git a/todos.md b/todos.md index 2114f82..ab7a8cd 100644 --- a/todos.md +++ b/todos.md @@ -187,7 +187,24 @@ backlog. repo copy) to the scaffolded prompt-standards template as well — one sentence of drift between repo copy and template, recorded here instead of forcing a plugin release for it (same reasoning as the PR #10 scope - decision). + decision). **Resolution vehicle: the upcoming canvas-findings hardening + round**, which touches `commands/workflow-init.md` for template changes and + carries a version bump regardless — the one-sentence sync rides there at no + extra release cost. Re-raised by CodeRabbit on PR #12 (Major) and kept + deferred there on the same reasoning; the scaffolded copy carries no false + claim, only one paragraph less. +- [ ] **Prompt-standards conformance checker — resolves two `pending` ledger rows + (2026-07-25).** Extend `scripts/check-invariants.sh` (+ its regression suite, + mutation-verified) with the mechanical rung both classes have never reached: + every file asserting it follows `docs/prompt-standards.md` must carry a + `Target model:` line, and a prose count of the checklist must match the + number of items actually in it. **Resolution vehicle: the same upcoming + canvas-findings hardening round** — it fires full Gate B anyway on its + `commands/` paths, so the ~8-line checker plus tests land at proportionate + cost instead of turning a docs PR into a code PR. Resolve the two `pending` + rows by **appending** rung-2 rows referencing them (2026-07-25 `docs-drift` + and 2026-07-25 `unverified-enforcement-claim`) — never by editing the + pending rows. *Trigger: that round starting.* - [ ] **Escalation trigger for the invariant checker — read this before patching it.** The checker asserts only the spellings its fixtures cover. Adding one more regex arm per newly-discovered spelling is *not* the ladder working; it is the same