Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
# ADR: Quality Model — Single-Document Resolution Cascade (Argument > Adoption > KB Default)

## Status

Accepted

## Date

2026-07-11

## Context

- Quality criteria for this repo (and any pair-adopting project) — risk classification, cost class, security relevance, per-tier gate/review requirements — were previously fragmented: implicit in review habits and skill prose rather than written down in one resolvable place.
- Multiple skills need the *same* rules: `classify`, `assess-cost` (#226), `assess-security` (#227), `pair-process-review`, `pair-capability-setup-gates`, and `pair-process-bootstrap` all need to classify risk/cost/tier consistently. Without one source, each would re-derive or duplicate criteria, and drift between skills becomes inevitable and hard to detect.
- Epic #208 (Unified classification & assessments → tags) requires a shared classification model whose tier and tag output every consuming skill and downstream automation (CI gates, `pair-next --filter`) reads identically.
- The KB already has an established convention-over-configuration principle (D21, "Adoption = solo delta"): a new schema should have a complete default in the KB, with adoption files holding only the delta a project actually needs — no new artifact should be mandatory.
- D13 (prior decision, referenced in story #221) rejected splitting this into a separate "quality grid" one-pager alongside the resolution logic — two artifacts that could drift apart.

## Options Considered

### Option 1: Per-skill embedded criteria

- **Description**: Each consuming skill (`classify`, `assess-cost`, `assess-security`, `pair-process-review`, `setup-gates`, `pair-process-bootstrap`) defines its own thresholds and classification rules inline in its own `SKILL.md`.
- **Pros**: No shared document to author or keep in sync; each skill is self-contained.
- **Cons**: Six-plus independent copies of "what counts as high risk" or "what tier requires how many reviewers" — drift across skills is inevitable and undetectable by inspection; no single audit point for "what are pair's quality rules"; violates the KB's own DRY conventions for shared rules.

### Option 2: Dedicated "quality grid" artifact, separate from resolution logic

- **Description**: A standalone visual/tabular "quality grid" document distinct from the prose resolution rules (the shape rejected by prior decision D13).
- **Pros**: Simpler to skim as a grid.
- **Cons**: Explicitly rejected per D13 — creates two artifacts (grid + logic) that can silently drift from each other; the grid becomes stale the moment resolution logic changes without a corresponding grid edit.

### Option 3: Single `quality-model.md` document + `Argument > Adoption > KB default` resolution cascade (chosen)

- **Description**: One document (`packages/knowledge-hub/dataset/.pair/knowledge/guidelines/quality-assurance/quality-model.md`) holds the 3-layer principle, 3 pillars, all classification dimensions, tier resolution (`tier = max(assessed dimensions)`, never lowered once raised — D17), per-tier requirements, and the nested taxonomy. Every consuming skill resolves quality rules through the same three-level cascade: an explicit argument overrides adoption, which overrides the KB default. Tag emission is chromatic-only and gated on a `## Tag Projection` declaration in `tech/risk-matrix.md` — but `risk:*` is the only family the KB names and proposes by default: `classify` asks to activate it the first time it runs and writes the declaration once confirmed. Every other model parameter (`cost:*` included) stays opt-in, exposed only if a project explicitly adds it to the declaration — the KB does not pre-select a second tag family. `tech/risk-matrix.md` is otherwise an *optional* adoption delta (criticality table + threshold overrides); its total absence means KB defaults apply completely and nothing fails (D21).
- **Pros**: Single source of truth — one document every consuming skill points at; near-zero adoption footprint (a project that declines the risk-tag proposal, or hasn't run `classify` yet, writes nothing extra); grep-verifiable, chromatic-only tag scheme instead of ad hoc per-project vocabularies; automation eligibility is a plain adoption-declared filter over existing tags rather than a bespoke tag family, keeping the tag surface small; the propose-then-write pattern means a project gets a sensible tagging default without needing to know `tech/risk-matrix.md` exists.
- **Cons**: Every consuming skill (`assess-cost` #226, `assess-security` #227, `classify` #233, extended review template #228) must independently implement the *same* resolution cascade correctly — the KB is prose, not executable code, so nothing mechanically enforces that all six skills apply the cascade identically; drift is possible if a future skill author misreads the order. Two nested-taxonomy entries (Architecture/modularity → `coupling-balance.md`, Cost signals → the cost-assessment guideline) are forward references to guidelines not yet published (tracked via #263 and #226 respectively).

## Decision

Adopt Option 3. `quality-model.md` is the single KB source of truth for quality classification. Resolution order for every rule in the document is **Argument > Adoption > KB default**: an explicit argument passed to a skill invocation wins; failing that, `tech/risk-matrix.md` (the adoption delta) wins; failing that, the KB default in `quality-model.md` applies. A malformed adoption delta is treated as absent — skills warn and fall back to KB defaults entirely, never partially.

This decision also settles four refinement-session questions, three cited inline in the story only informally (not previously recorded in a durable decision) and one settled during this review cycle:

- **Q1 (document location)**: the model lives under `knowledge/guidelines/quality-assurance/`, alongside the guidelines it governs, not in a separate top-level location.
- **Q2 (tag naming scheme)**: tags are chromatic-only (`risk:green|yellow|red`, `cost:green|yellow|orange|red`) — no semantic tag names beyond color, keeping the scheme small and provider-agnostic.
- **Q2b (no dedicated eligibility tag)**: automation eligibility (e.g., what `pair-next --filter` treats as auto-mergeable) is an adoption-declared *filter* over the classification tags above (e.g., `risk:green`), not its own tag family — one fewer tag surface to maintain.
- **Q2c (default vs. opt-in tag families, settled 2026-07-11)**: not every tag family is equally universal — risk classification is relevant to essentially every change, while cost exposure is situational (real for infra-heavy projects, noise for a static content site). So `risk` is the only family the KB proposes by default; `classify` (#233) is the skill responsible for asking to activate it on its first run and writing the `## Tag Projection` declaration into `tech/risk-matrix.md` once confirmed (the same propose-then-write-if-confirmed idiom `pair-capability-verify-quality` already uses for its first-time Custom Gate Registry setup). `cost` — and any tag family this model grows in the future — stays opt-in, activated only by explicitly adding it to the declaration. The declaration lives in `tech/risk-matrix.md` rather than `way-of-working.md`: it's a classification-behavior toggle, not a process/workflow convention, so it belongs alongside the criticality table and threshold overrides it's the natural sibling of, keeping `way-of-working.md` free of quality-model-specific content as the model grows more adoption surface over time (cost-signal catalog overrides, coupling thresholds, etc.).

The risk tier is `max(assessed dimensions)` across service/domain criticality, change/diff risk, business impact, security relevance, and coupling balance; built twice per story (D17) — once from story context at refinement, once from the diff at review — and the review value is a floor: it may raise the tier but never lower it.

## Consequences

### Benefits

- One document every consuming skill (`classify`, `assess-cost`, `assess-security`, `pair-process-review`, `setup-gates`, `pair-process-bootstrap`) reads from — no re-derivation, no duplicated thresholds.
- Near-zero mandatory adoption footprint: a project that declines the risk-tag proposal, or hasn't run `classify` yet, writes nothing to `tech/risk-matrix.md` at all; even accepting the proposal is a couple of written lines, not a rewrite (D21).
- A sensible default the project doesn't have to discover on its own: `classify` proposes `risk` tagging proactively instead of requiring a team to know `tech/risk-matrix.md`'s Tag Projection schema exists.
- Grep-verifiable, chromatic-only tag scheme instead of bespoke per-project vocabularies.
- Automation eligibility stays a plain tag filter, not a new tag family — smaller surface for `pair-next` and CI gates to reason about.
- Situational tag families (`cost`, and any future ones) stay opt-in — no project is forced to carry classification noise it has no use for.

### Trade-offs and Limitations

- No shared executable code enforces that every consuming skill implements the cascade identically — each of #226/#227/#233/#228 must independently apply `Argument > Adoption > KB default` correctly; a future review of those stories should explicitly check this against `quality-model.md`, not just against each skill's own prose.
- `classify` (#233) now owns an extra piece of first-run behavior (propose → write-if-confirmed → never re-propose) beyond just resolving and applying the model; that story's own review should verify the propose-once idempotency explicitly, not just the tag values.
- Two taxonomy entries are forward references to not-yet-published guidelines (coupling-balance under #263, cost-assessment guideline under #226) — deliberately left as non-hyperlinked code spans (tested for) until those land.
- The coupling dimension depends on `assess-coupling` (#263) and scoped `map-contexts` output; until those exist, coupling is reported "not assessed" and excluded from the tier calculation, never blocking (D21).

## Adoption Impact

- `packages/knowledge-hub/dataset/.pair/knowledge/guidelines/quality-assurance/quality-model.md` (new) — the resolution cascade, classification dimensions, tier/SLA table, tag projection (default/opt-in split + `classify`'s propose-flow), and nested taxonomy.
- `packages/knowledge-hub/dataset/.pair/knowledge/guidelines/quality-assurance/README.md` — indexes the new document.
- `packages/knowledge-hub/dataset/.pair/knowledge/assets/risk-matrix-example.md` (new) — a filled-in `tech/risk-matrix.md` example, including a `## Tag Projection` section, usable as a real adoption starting point.
- No project's `way-of-working.md` requires any change to adopt this decision — the Tag Projection declaration lives in `tech/risk-matrix.md`, not `way-of-working.md`, so this decision adds no new section there. A project's `tech/risk-matrix.md` commonly ends up with at least a `## Tag Projection` section once `classify` has run and its proposal is accepted — that's expected, not a gap.
6 changes: 4 additions & 2 deletions apps/website/content/docs/reference/guidelines-catalog.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ description: Complete catalog of all pair Knowledge Base guidelines (9 categorie

The pair Knowledge Base ships with opinionated guidelines organized into 9 categories and 9 sequential how-to guides covering the full AI-assisted development lifecycle. Guidelines live in `.pair/knowledge/guidelines/`, how-to guides in `.pair/knowledge/how-to/`.

> **Last updated:** 2026-07-07. Source: `packages/knowledge-hub/dataset/.pair/knowledge/`
> **Last updated:** 2026-07-11. Source: `packages/knowledge-hub/dataset/.pair/knowledge/`

## How-To Guides

Expand Down Expand Up @@ -106,6 +106,7 @@ Security, accessibility, and performance standards.

| Topic | What it covers |
| --- | --- |
| **Quality Model** | 3-layer principle, 3 pillars (Cost/Security/Delivery), classification dimensions and tier resolution, per-tier requirements, tag projection, `tech/risk-matrix.md` adoption delta — see the [dedicated page](/docs/reference/quality-model) |
| **Accessibility** | WCAG compliance, screen reader support, keyboard navigation |
| **Performance** | Load times, Core Web Vitals, optimization |
| **Quality Monitoring** | Quality dashboards, trend analysis |
Expand Down Expand Up @@ -155,7 +156,8 @@ Design systems, research, and content strategy.

## Related

- [Skills Catalog](/docs/reference/skills-catalog) — all 32 pair skills with commands and composition
- [Quality Model](/docs/reference/quality-model) — the single source of default quality rules (classification, tiers, tags)
- [Skills Catalog](/docs/reference/skills-catalog) — all 34 pair skills with commands and composition
- [Developer Journey](/docs/developer-journey) — step-by-step walkthrough of the full process
- [KB Structure](/docs/reference/kb-structure) — directory layout and file purposes
- [Customize for Your Team](/docs/customization/team) — override guidelines via adoption files
1 change: 1 addition & 0 deletions apps/website/content/docs/reference/meta.json
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
"specs",
"skills-catalog",
"guidelines-catalog",
"quality-model",
"skill-management",
"kb-structure",
"configuration"
Expand Down
54 changes: 54 additions & 0 deletions apps/website/content/docs/reference/quality-model.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
---
title: Quality Model
description: The single source of default quality rules — 3-layer principle, 3 pillars, classification dimensions, per-tier requirements, tag projection, and the risk-matrix adoption delta.
---

The Quality Model is the single source of default quality rules in the pair Knowledge Base. `classify`, `assess-cost`, `assess-security`, code review, quality gates, and bootstrap all resolve their behavior from this one document — no criteria live anywhere else. Source: [`quality-model.md`](https://github.com/foomakers/pair/blob/main/packages/knowledge-hub/dataset/.pair/knowledge/guidelines/quality-assurance/quality-model.md).

> **Resolution order** for every rule below: **Argument > Adoption > KB default**. A malformed adoption delta is treated as absent — skills warn and fall back to KB defaults.

## Three-Layer Principle

| Layer | Role | Examples |
| --- | --- | --- |
| **Doc** | Rules, written once | this document + pillar guidelines |
| **Skill** | Applies the rules on demand | `classify`, `assess-cost`, `assess-security`, code review |
| **Automation** | Consumes tags deterministically, zero judgment | CI gates, `pair-next --filter` |

Quality is classified in refinement — before code exists — not only at review time (shift-left). The matrix is built twice: once from the story context in refinement, once from the diff in review. Automation never carries its own criteria — it only reads tags.

## Three Pillars

| Pillar | Covers | Tag family (if exposed) |
| --- | --- | --- |
| **Cost** | Financial exposure of building/running the change | `cost:*` — opt-in |
| **Security** | Vulnerabilities, compliance, secure-by-design | feeds `risk:*` + deterministic CI scanning |
| **Delivery** | Correctness, performance, a11y, observability, docs, planning, architecture, release, AI metrics | `risk:*` — KB default |

Every quality theme nests under one of these three pillars, pointing at its existing guideline — no dedicated status page per theme.

## Classification & Tiers

The compiled matrix has one row per dimension: service/domain criticality, change/diff risk, business impact, security relevance, and coupling balance. Each resolves to green/yellow/red; the **risk tier is the max across assessed dimensions** and is never lowered once raised by review. Cost is tracked separately as its own class, always computed and written to the matrix — it's projected as the `cost:green|yellow|orange|red` tag only if a project chooses to expose it (see Tag Projection below).

| Tier | Merge | Reviewers | SLA | Approval |
| --- | --- | --- | --- | --- |
| 🟢 Green | Self-merge at green gate checks | 0 (AI review informational) | — | none |
| 🟡 Yellow | Blocked until reviewed | 1 reviewer | 1 working day | reviewer approval |
| 🔴 Red | Blocked until reviewed and approved | 1 reviewer | 2 working days | explicit approval |

Reviewer counts and SLAs above are KB defaults, not fixed rules — override either per tier in `tech/risk-matrix.md`'s Overrides section (e.g. `tier.red.reviewers: 2`). A PR without classification tags is treated as 🔴 (fail-safe).

## Tag Projection & Eligibility

Tags are a **chromatic projection** of the matrix. `risk:green|yellow|red` is the only tag family the KB names and proposes by default — `classify` asks to activate it the first time it runs and writes the declaration once you confirm, so you get sensible tagging without needing to know the schema exists. Every other parameter the model computes (cost class, security relevance, business impact, coupling balance) is available to expose as its own tag, but the KB doesn't pre-select which, if any — that's entirely your call, added to the declaration when you want it. There is no dedicated "eligibility" tag: automation eligibility is a **filter you declare over classification tags** (e.g. `risk:green`), consumed generically by `pair-next`.

## Project Overrides — `tech/risk-matrix.md`

Optional file with up to three independent sections — none imply the others: **Tag Projection** (which tags are active — typically the first section a project ends up with, since `classify` proposes it), a **criticality table** (service/domain → High/Medium/Low), and **overrides** — threshold tweaks plus optional per-tier reviewer count / SLA overrides (e.g. `tier.red.reviewers: 2`). Fully absent means KB defaults apply completely and no tags are emitted — nothing fails. See the full schema in [`quality-model.md` §5–§6](https://github.com/foomakers/pair/blob/main/packages/knowledge-hub/dataset/.pair/knowledge/guidelines/quality-assurance/quality-model.md#6-techrisk-matrixmd--adoption-delta) and a filled-in example at [`risk-matrix-example.md`](https://github.com/foomakers/pair/blob/main/packages/knowledge-hub/dataset/.pair/knowledge/assets/risk-matrix-example.md).

## Related

- [Guidelines Catalog](/docs/reference/guidelines-catalog) — where the Quality Model sits among the other 9 guideline categories
- [Customize for Your Team](/docs/customization/team) — how to override guidelines via adoption files
- [KB Structure](/docs/reference/kb-structure) — directory layout, `knowledge/` vs `adoption/`
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# `tech/risk-matrix.md` — Example

Illustrative adoption delta for a fictitious e-commerce project. Copy the sections you need — this file is only committed as a reference; a real project's `tech/risk-matrix.md` normally has just a few rows. See the schema and resolution rules in the [quality model](../guidelines/quality-assurance/quality-model.md), §5–§6.

## Tag Projection

Active: risk, cost

`risk` is the only tag family the KB proposes by default — `classify` writes this section itself the first time it runs, once the proposal to activate it is confirmed (§5). This project also added `cost` to the list, since it tracks cloud spend closely.

## Criticality Table

| Service/Domain | Criticality |
| --- | --- |
| payments | High |
| checkout | High |
| catalog | Medium |
| marketing-site | Low |
| internal-admin-tool | Low |

Any service/domain not listed here is treated as unclassified and resolves to High (conservative) for the service-criticality dimension — not to the file-absent Medium default.

## Overrides

Optional threshold tweaks for the change-risk dimension (§3.1 of the quality model). Omit entirely if the KB defaults are fine.

- `change-risk.shared-paths`: `["packages/billing/**", "packages/checkout-core/**"]` — diffs touching these paths are always classified `yellow` or higher for change/diff risk, even if the diff is small.
Loading
Loading