Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
118 changes: 118 additions & 0 deletions .agents/skills/dual-evaluation/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,118 @@
---
name: dual-evaluation
description: Prepare a contested architectural or sanction/defer decision by commissioning two independent evaluations under deliberately different lenses (and ideally different models), then writing a comparison document that extracts convergence, divergence, and uniquely-caught deliverables for the human decision owner. Use when a decision is hard to reverse or scope-expanding and a single recommendation would under-inform it — e.g. "should we sanction this slice", "evaluate both options independently", "run a dual evaluation", or when Erik asks for independent counsel on a fork in the road.
---

# Dual Evaluation — Independent Counsel for Contested Decisions

> First lived use: the M8 ops-feed-completion sanction (2026-06-10) —
> `docs/research/ops-feed-completion-evaluation.md` (staff architect, Fable 5),
> `…-evaluation-es-specialist.md` (event-sourcing specialist, Opus 4.6),
> `…-evaluation-comparison.md` (comparison + recorded decision). Those three files are
> the reference implementation; this skill generalizes them.

## When to apply — and when not to

Use it when **all three** hold:

1. The decision is genuinely contested or hard to reverse (sanction-vs-defer, a scope
expansion, a structural pivot) — the cost of a wrong call exceeds the ~session the
method costs.
2. A recommendation is wanted, not just options — the output feeds a human decision,
it does not make one.
3. Two *materially different* lenses exist for the question (pragmatic/observable vs
structural/invariant; product vs platform; cost vs correctness).

**Exit ramp:** if during framing the evidence turns out to be one-sided, say so and skip
the ceremony. Precedent: the M8-S6b dispute-control decision was folded into the same
comparison doc as an Addendum *without* an evaluation pass, because the backend was
already fully proven — "no evaluation needed" is a legitimate, recordable outcome.

## The method

### 1. Frame once, neutrally

Write a single decision statement both evaluators receive **identically**: the question,
the 2–3 options, the background facts (lived code state, prior precedent, constraints),
and what kind of answer is wanted (recommendation + reasoning, not a survey). Do not
bake a preferred option into the framing — a tilted prompt produces two echoes, not two
evaluations. Name the decision owner and date.

### 2. Commission two independent evaluations

- **Different lenses, chosen to disagree about what matters.** The lived pair: a
*staff architect* (modular-monolith guardian, conventions, pedagogy, "what does the
operator see?") vs an *event-sourcing specialist* (first principles, topology
invariants, "what guarantee is violated?"). Two flavors of the same reviewer is an
echo chamber, not a second opinion.
- **Different models when available** (lived case: Codex Fable 5 + Codex Opus 4.6) —
cheap extra decorrelation on top of the persona split.
- **True independence:** separate sessions/agents; neither sees the other's output or
knows a second evaluation exists. Independence is the load-bearing property — step 3's
convergence signal is worthless if the chains could have copied each other.
- Each evaluation is **self-contained**: decision framing restated, numbered evaluation
points, an explicit recommendation, references into the lived code. It must stand
alone as counsel even if the other evaluation were lost.

### 3. Write the comparison document

The comparison is its own artifact (`…-evaluation-comparison.md`), authored only after
both evaluations are complete. Its sections, in order:

1. **Header table** — file, lens, model per evaluation; decision owner; date.
2. **Where they agree (strong signal)** — for each agreement, show *both reasoning
chains*. Convergence through different reasoning is the method's whole product;
agreement restated in the same words is suspicious, not reassuring.
3. **Where they diverge (the interesting differences)** — each divergence gets a
**Significance** note: what turns on it, and who should care.
4. **"Anything only one evaluation caught?"** — a first-class question, not a footnote.
These uniquely-caught items are often real deliverables that survive *regardless* of
which overall recommendation wins (lived case: A's topology-test invariant and
`onreconnected` reconciliation; B's precise 10-event gap count — all three entered
the slice prompt).
5. **Summary for the decision** — a question/answer table the owner can act on, ending
with the residual open questions only the human can close.
6. **Decision (owner, date)** — left for the human; recorded *in this file* when made,
naming which pieces of each evaluation are accepted. Later decisions on the same
topic may land as dated Addenda (the M8-S6b Decision-2 precedent).

### 4. Route the outcome

The comparison's Decision section is the citable record: downstream docs (milestone
Document History, slice prompts, STATUS) cite it rather than restating the reasoning.
Uniquely-caught deliverables go into the resulting prompt's scope explicitly — they are
the easiest thing to lose between decision and execution.

## Artifact conventions (CritterBids)

| Artifact | Path |
|---|---|
| Evaluation (first/default lens) | `docs/research/{slug}-evaluation.md` |
| Evaluation (second lens) | `docs/research/{slug}-evaluation-{lens}.md` |
| Comparison + recorded decision | `docs/research/{slug}-evaluation-comparison.md` |

All three are committed; `docs/research/README.md` indexes them.

## Pitfalls

- **Same lens twice** → two echoes; the comparison has nothing to compare.
- **Evaluator 2 sees evaluator 1** (or the framing leaks "the other reviewer said…")
→ independence broken; convergence becomes noise.
- **A tilted framing** ("should we do this obviously-good thing?") → both evaluations
inherit the tilt; write the decision statement as the *undecided* owner would.
- **Treating unanimity as the decision** → the lived case was unanimous on *what* and
split on *when* (separate slice vs folded into S7); the human closed it. Residual
disagreements are the owner's, always.
- **Skipping the comparison** → two stapled documents make the owner do the synthesis;
the comparison *is* the deliverable, the evaluations are its inputs.
- **Running it for everything** → the method costs about a session; the exit ramp
exists so cheap, evidence-one-sided decisions get an Addendum, not a ceremony.

## See also

- `docs/research/ops-feed-completion-evaluation-comparison.md` — the reference
comparison, including a recorded Decision and a no-evaluation-needed Addendum.
- `docs/decisions/README.md` — when the *outcome* warrants an ADR, the comparison doc
is the ADR's evidence trail, not a substitute for it.
- `docs/prompts/README.md` — the resulting slice prompt carries the accepted
deliverables; the prompt cites the comparison's Decision section.
167 changes: 167 additions & 0 deletions .agents/skills/frontend-slice-discipline/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,167 @@
---
name: frontend-slice-discipline
description: >-
CritterBids frontend working discipline, distilled from the M8 retrospectives:
read the lived backend surface before writing client code, render the lived
subset and record gaps as carry-forwards (never invent shapes or silently widen
into backend changes), verify the installed toolchain instead of remembered
APIs, close every slice with a live smoke against the Aspire host (with the
browser-fallback playbook), and the recurring test conventions (injectable
seams, isolated test routers, tests in the production type-check, StrictMode
double-effect handling). Use when starting, implementing, or closing any
frontend slice or feature in client/.
---

# CritterBids Frontend Slice Discipline

> Process rules for working in `client/` — the lessons every M8 slice paid for once so the
> next slice doesn't. The *wire contract* itself lives in
> `docs/skills/wolverine-http-frontend-contract/SKILL.md` (HTTP) and
> `.Codex/skills/signalr/SKILL.md` (push); this skill is about **how to work**, not what the
> bytes look like.

## When to apply

- Starting a new frontend slice or feature in `client/bidder/` or `client/ops/`.
- Mid-slice, when a spec (milestone table, narrative, prompt) names something you can't find
in the backend.
- Closing a slice: deciding what "verified" means before the retro claims it.

## Rule 1 — Read the lived backend before writing client code

Milestone tables, narratives, and prompts describe **intent**; `src/` is **truth**. Before the
first client line, read the actual endpoint signatures, read-model records, and notification
types the slice consumes. Every M8 slice that skipped ahead would have shipped against fiction:

- The milestone named a `ListingDetailView`; the lived endpoint returns `CatalogListingView`
for both list and detail (M8-S2). The client binds to the real shape — no invented schema.
- Narrative 001 names a targeted `Outbid` push; the lived Relay surface fans out `BidPlaced`
only (M8-S3b). "Outbid" became a client-side derivation from view transitions — reading
`Relay/Notifications/` first reshaped the whole design.
- The narrative names `remainingCredit`; the lived `SettlementCompletedNotification` omits it
(M8-S4). The view renders the subset that exists.
- The prompt said `accessTokenFactory` puts the token on the negotiate; the installed client's
source said Bearer header (M8-S5). Package source in `node_modules` counts as lived backend.

## Rule 2 — Render the lived subset; gaps become carry-forwards

When the spec names something the backend doesn't expose, there are exactly two moves, and
inventing a client-side workaround is neither:

1. **Render what exists** and record the gap — in the retro's findings and
"What remains", with the backend change the gap implies (e.g. "display name needs a
`GET /api/participants/{id}` read path"). A recorded carry-forward survives; a silent
omission evaporates.
2. **Escalate a sanctioned backend slice** when the journey cannot ship without it — the
M8-S3a precedent (`POST /api/auctions/bids` had no HTTP surface at all). Its own slice,
its own prompt, never a quiet `.cs` touch inside a frontend slice.

M8's frontend slices shipped with **zero** unsanctioned backend changes because every gap took
move 1 or move 2.

## Rule 3 — Verify the installed toolchain, not remembered APIs

The 2026 ecosystem moved under every slice that assumed muscle memory. Check the **installed**
package (its `node_modules` source or current docs) at scaffold time:

- Tailwind v4: the v3 PostCSS + `@tailwind` directives wiring is gone — v4 is
`@tailwindcss/vite` + one `@import "tailwindcss";` (M8-S1).
- Zod 4 moved string-format validators off `z.string()` (M8-S2).
- TypeScript deprecated `baseUrl` — the `@/*` alias works without it under
`moduleResolution: "bundler"` (M8-S2).
- SignalR 7+ changed the access-token transport (M8-S5) — found by grepping
`node_modules/@microsoft/signalr/dist/esm/`, not by docs.

The pattern: **the library *choice* is usually stable; the *wiring* is what changed.** A
five-minute source check beats a debugging session against a green-looking scaffold.

## Rule 4 — Close every slice with a live smoke

Mocked-fetch tests verify **response handling**; only a live host verifies **request shape**
(body presence, `Content-Type`, header names, key casing) and **cross-client propagation**.
The two canonical failures:

- M8-S2: green build + green tests shipped a session POST that 400'd on its first live
request (no body).
- M8-S3b: a fully green, correctly-built frontend — and the live cross-client loop didn't
close, because two backend bugs sat between it and a working demo. Only the integrated
manual run found them.

A skipped smoke is an **explicitly unchecked acceptance criterion** in the retro (the M8-S4
precedent), never a silent pass.

### The smoke playbook

1. **Host:** `dotnet run --project src/CritterBids.AppHost --launch-profile http`. Dev-only
secrets (the ADR-024 staff token) go in the launching shell's environment —
`$env:OperationsAuth__StaffToken = "..."` — child projects inherit it (no repo change; see
`docs/skills/aspire/SKILL.md`).
2. **Probe the HTTP contract** with direct requests first (PowerShell `Invoke-WebRequest` /
curl): expected status with and without credentials, body casing, `Location` headers.
3. **Drive the real client library from Node** through the Vite dev proxy when a browser is
unavailable — same library, same options as the app, deterministic, and it exercises the
proxy (`ws: true`) specifically. **Credential-transport caveat (M8-S6b Finding 2):** the
signalr client picks its WebSocket implementation by resolvability, not platform — from
inside `client/` it resolves the `ws` package, which CAN set headers, so an
`accessTokenFactory` token goes out as an `Authorization` header (which the
query-string-only StaffToken scheme does not read) instead of `?access_token=`. Node 22's
built-in header-less `WebSocket` behaves like a browser only when `ws` is NOT resolvable.
To reproduce the browser credential transport faithfully from Node, **pin the token in the
URL** (`/hub/operations?access_token=…`) rather than using `accessTokenFactory`.
4. **Real browser pass** when feasible: the Playwright MCP needs Chrome; when it's missing
(admin install), `playwright-core` driving system **Edge** (`channel: "msedge"`,
`headless: true`) works without any browser download (M8-S5).
5. **Tear down** what you started (background hosts, dev servers, temp harnesses) and record
the smoke's findings — including expected console noise — in the retro.

## Test conventions that recurred

- **Injectable seams over module mocks where a boundary exists:** the SignalR Providers take a
`createConnection` prop; fetch wrappers take a `fetchImpl`. Tests drive pushes/responses
through fakes without patching globals. (Module-mock only when asserting on the *production*
factory itself — the M8-S5 `withUrl`-capture test.)
- **Isolated test router, not the real shell.** Render the page under a minimal router with
stub routes (so typed `Link`s resolve) instead of mounting `AppShell` — keeps SignalR and
the session POST out of jsdom (M8-S2).
- **Tests compile in the production type-check.** `tsconfig.json` `include: ["src"]` puts
`*.test.tsx` inside `tsc --noEmit`, which runs in `npm run build` — a type error in a test
breaks the build. Tests must satisfy `strict` + `noUnusedLocals`.
- **Reset browser storage between tests** (`sessionStorage.clear()` in the Vitest setup) —
both apps key auth/session state off it.
- **StrictMode double-effects are a design constraint, not noise.** For a one-shot effect
(the session POST), a `startedRef` guard ensures one real request — and deliberately **no**
cancelled-flag on the result write: under StrictMode the surviving fetch belongs to the
torn-down first effect instance, and a cancelled flag drops the only result on the floor
(the M8 Bug #2 fix-up walkthrough found exactly this hang). For connection effects, the
cleanup `stop()` makes dev log one benign
`Failed to start the HttpConnection before stop() was called` — expected with the cleanup
present, a bug without it.

## Common pitfalls

- **Coding against the milestone/narrative table** without opening the backend types → binds
to views and pushes that don't exist (Rule 1).
- **"Fixing" a backend gap from the frontend** (inventing fields, fake data, client-side
workarounds for missing reads) → Rule 2: render the subset + carry-forward, or escalate a
slice.
- **Scaffolding from training-data muscle memory** → Rule 3; check the installed version.
- **Calling a slice done on green unit tests** → Rule 4; request contracts and cross-client
loops are only verifiable live.
- **Mounting the full app shell in component tests** → jsdom fights SignalR and the session
bootstrap; isolate with a test router.
- **Adding a cancelled-flag to every async effect reflexively** → for one-shot
effects under StrictMode it can discard the only result; reason about which effect instance
owns the in-flight work.

## See also

- `docs/skills/wolverine-http-frontend-contract/SKILL.md` — the HTTP wire contract this
discipline verifies (body rules, `CreationResponse`, ProblemDetails, retry rules).
- `.Codex/skills/signalr/SKILL.md` — the push-surface client conventions (ADR 026 pattern,
hub auth, dedupe).
- `docs/skills/aspire/SKILL.md` — host startup + dev-secret environment inheritance for the
smoke playbook.
- ADR-013 / ADR-025 / ADR-026 in `docs/decisions/` — the stack, layout, and SignalR pattern
decisions slices point at rather than re-deciding.
- M8 retrospectives (`docs/retrospectives/M8-S1…S5-*.md`) — the evidence base; each rule cites
its slice.
Loading
Loading