diff --git a/.pair/adoption/tech/adr/2026-07-11-quality-model-resolution-cascade.md b/.pair/adoption/tech/adr/2026-07-11-quality-model-resolution-cascade.md new file mode 100644 index 00000000..0b74360d --- /dev/null +++ b/.pair/adoption/tech/adr/2026-07-11-quality-model-resolution-cascade.md @@ -0,0 +1,75 @@ +# ADR: Quality Model — Single-Document Resolution Cascade (Argument > Adoption > KB Default) + +## Status + +Accepted + +## Date + +2026-07-11 + +## Context + +- Quality criteria for this repo (and any pair-adopting project) — risk classification, cost class, security relevance, per-tier gate/review requirements — were previously fragmented: implicit in review habits and skill prose rather than written down in one resolvable place. +- Multiple skills need the *same* rules: `classify`, `assess-cost` (#226), `assess-security` (#227), `pair-process-review`, `pair-capability-setup-gates`, and `pair-process-bootstrap` all need to classify risk/cost/tier consistently. Without one source, each would re-derive or duplicate criteria, and drift between skills becomes inevitable and hard to detect. +- Epic #208 (Unified classification & assessments → tags) requires a shared classification model whose tier and tag output every consuming skill and downstream automation (CI gates, `pair-next --filter`) reads identically. +- The KB already has an established convention-over-configuration principle (D21, "Adoption = solo delta"): a new schema should have a complete default in the KB, with adoption files holding only the delta a project actually needs — no new artifact should be mandatory. +- D13 (prior decision, referenced in story #221) rejected splitting this into a separate "quality grid" one-pager alongside the resolution logic — two artifacts that could drift apart. + +## Options Considered + +### Option 1: Per-skill embedded criteria + +- **Description**: Each consuming skill (`classify`, `assess-cost`, `assess-security`, `pair-process-review`, `setup-gates`, `pair-process-bootstrap`) defines its own thresholds and classification rules inline in its own `SKILL.md`. +- **Pros**: No shared document to author or keep in sync; each skill is self-contained. +- **Cons**: Six-plus independent copies of "what counts as high risk" or "what tier requires how many reviewers" — drift across skills is inevitable and undetectable by inspection; no single audit point for "what are pair's quality rules"; violates the KB's own DRY conventions for shared rules. + +### Option 2: Dedicated "quality grid" artifact, separate from resolution logic + +- **Description**: A standalone visual/tabular "quality grid" document distinct from the prose resolution rules (the shape rejected by prior decision D13). +- **Pros**: Simpler to skim as a grid. +- **Cons**: Explicitly rejected per D13 — creates two artifacts (grid + logic) that can silently drift from each other; the grid becomes stale the moment resolution logic changes without a corresponding grid edit. + +### Option 3: Single `quality-model.md` document + `Argument > Adoption > KB default` resolution cascade (chosen) + +- **Description**: One document (`packages/knowledge-hub/dataset/.pair/knowledge/guidelines/quality-assurance/quality-model.md`) holds the 3-layer principle, 3 pillars, all classification dimensions, tier resolution (`tier = max(assessed dimensions)`, never lowered once raised — D17), per-tier requirements, and the nested taxonomy. Every consuming skill resolves quality rules through the same three-level cascade: an explicit argument overrides adoption, which overrides the KB default. Tag emission is chromatic-only and gated on a `## Tag Projection` declaration in `tech/risk-matrix.md` — but `risk:*` is the only family the KB names and proposes by default: `classify` asks to activate it the first time it runs and writes the declaration once confirmed. Every other model parameter (`cost:*` included) stays opt-in, exposed only if a project explicitly adds it to the declaration — the KB does not pre-select a second tag family. `tech/risk-matrix.md` is otherwise an *optional* adoption delta (criticality table + threshold overrides); its total absence means KB defaults apply completely and nothing fails (D21). +- **Pros**: Single source of truth — one document every consuming skill points at; near-zero adoption footprint (a project that declines the risk-tag proposal, or hasn't run `classify` yet, writes nothing extra); grep-verifiable, chromatic-only tag scheme instead of ad hoc per-project vocabularies; automation eligibility is a plain adoption-declared filter over existing tags rather than a bespoke tag family, keeping the tag surface small; the propose-then-write pattern means a project gets a sensible tagging default without needing to know `tech/risk-matrix.md` exists. +- **Cons**: Every consuming skill (`assess-cost` #226, `assess-security` #227, `classify` #233, extended review template #228) must independently implement the *same* resolution cascade correctly — the KB is prose, not executable code, so nothing mechanically enforces that all six skills apply the cascade identically; drift is possible if a future skill author misreads the order. Two nested-taxonomy entries (Architecture/modularity → `coupling-balance.md`, Cost signals → the cost-assessment guideline) are forward references to guidelines not yet published (tracked via #263 and #226 respectively). + +## Decision + +Adopt Option 3. `quality-model.md` is the single KB source of truth for quality classification. Resolution order for every rule in the document is **Argument > Adoption > KB default**: an explicit argument passed to a skill invocation wins; failing that, `tech/risk-matrix.md` (the adoption delta) wins; failing that, the KB default in `quality-model.md` applies. A malformed adoption delta is treated as absent — skills warn and fall back to KB defaults entirely, never partially. + +This decision also settles four refinement-session questions, three cited inline in the story only informally (not previously recorded in a durable decision) and one settled during this review cycle: + +- **Q1 (document location)**: the model lives under `knowledge/guidelines/quality-assurance/`, alongside the guidelines it governs, not in a separate top-level location. +- **Q2 (tag naming scheme)**: tags are chromatic-only (`risk:green|yellow|red`, `cost:green|yellow|orange|red`) — no semantic tag names beyond color, keeping the scheme small and provider-agnostic. +- **Q2b (no dedicated eligibility tag)**: automation eligibility (e.g., what `pair-next --filter` treats as auto-mergeable) is an adoption-declared *filter* over the classification tags above (e.g., `risk:green`), not its own tag family — one fewer tag surface to maintain. +- **Q2c (default vs. opt-in tag families, settled 2026-07-11)**: not every tag family is equally universal — risk classification is relevant to essentially every change, while cost exposure is situational (real for infra-heavy projects, noise for a static content site). So `risk` is the only family the KB proposes by default; `classify` (#233) is the skill responsible for asking to activate it on its first run and writing the `## Tag Projection` declaration into `tech/risk-matrix.md` once confirmed (the same propose-then-write-if-confirmed idiom `pair-capability-verify-quality` already uses for its first-time Custom Gate Registry setup). `cost` — and any tag family this model grows in the future — stays opt-in, activated only by explicitly adding it to the declaration. The declaration lives in `tech/risk-matrix.md` rather than `way-of-working.md`: it's a classification-behavior toggle, not a process/workflow convention, so it belongs alongside the criticality table and threshold overrides it's the natural sibling of, keeping `way-of-working.md` free of quality-model-specific content as the model grows more adoption surface over time (cost-signal catalog overrides, coupling thresholds, etc.). + +The risk tier is `max(assessed dimensions)` across service/domain criticality, change/diff risk, business impact, security relevance, and coupling balance; built twice per story (D17) — once from story context at refinement, once from the diff at review — and the review value is a floor: it may raise the tier but never lower it. + +## Consequences + +### Benefits + +- One document every consuming skill (`classify`, `assess-cost`, `assess-security`, `pair-process-review`, `setup-gates`, `pair-process-bootstrap`) reads from — no re-derivation, no duplicated thresholds. +- Near-zero mandatory adoption footprint: a project that declines the risk-tag proposal, or hasn't run `classify` yet, writes nothing to `tech/risk-matrix.md` at all; even accepting the proposal is a couple of written lines, not a rewrite (D21). +- A sensible default the project doesn't have to discover on its own: `classify` proposes `risk` tagging proactively instead of requiring a team to know `tech/risk-matrix.md`'s Tag Projection schema exists. +- Grep-verifiable, chromatic-only tag scheme instead of bespoke per-project vocabularies. +- Automation eligibility stays a plain tag filter, not a new tag family — smaller surface for `pair-next` and CI gates to reason about. +- Situational tag families (`cost`, and any future ones) stay opt-in — no project is forced to carry classification noise it has no use for. + +### Trade-offs and Limitations + +- No shared executable code enforces that every consuming skill implements the cascade identically — each of #226/#227/#233/#228 must independently apply `Argument > Adoption > KB default` correctly; a future review of those stories should explicitly check this against `quality-model.md`, not just against each skill's own prose. +- `classify` (#233) now owns an extra piece of first-run behavior (propose → write-if-confirmed → never re-propose) beyond just resolving and applying the model; that story's own review should verify the propose-once idempotency explicitly, not just the tag values. +- Two taxonomy entries are forward references to not-yet-published guidelines (coupling-balance under #263, cost-assessment guideline under #226) — deliberately left as non-hyperlinked code spans (tested for) until those land. +- The coupling dimension depends on `assess-coupling` (#263) and scoped `map-contexts` output; until those exist, coupling is reported "not assessed" and excluded from the tier calculation, never blocking (D21). + +## Adoption Impact + +- `packages/knowledge-hub/dataset/.pair/knowledge/guidelines/quality-assurance/quality-model.md` (new) — the resolution cascade, classification dimensions, tier/SLA table, tag projection (default/opt-in split + `classify`'s propose-flow), and nested taxonomy. +- `packages/knowledge-hub/dataset/.pair/knowledge/guidelines/quality-assurance/README.md` — indexes the new document. +- `packages/knowledge-hub/dataset/.pair/knowledge/assets/risk-matrix-example.md` (new) — a filled-in `tech/risk-matrix.md` example, including a `## Tag Projection` section, usable as a real adoption starting point. +- No project's `way-of-working.md` requires any change to adopt this decision — the Tag Projection declaration lives in `tech/risk-matrix.md`, not `way-of-working.md`, so this decision adds no new section there. A project's `tech/risk-matrix.md` commonly ends up with at least a `## Tag Projection` section once `classify` has run and its proposal is accepted — that's expected, not a gap. diff --git a/apps/website/content/docs/reference/guidelines-catalog.mdx b/apps/website/content/docs/reference/guidelines-catalog.mdx index 7f233892..871b6175 100644 --- a/apps/website/content/docs/reference/guidelines-catalog.mdx +++ b/apps/website/content/docs/reference/guidelines-catalog.mdx @@ -5,7 +5,7 @@ description: Complete catalog of all pair Knowledge Base guidelines (9 categorie The pair Knowledge Base ships with opinionated guidelines organized into 9 categories and 9 sequential how-to guides covering the full AI-assisted development lifecycle. Guidelines live in `.pair/knowledge/guidelines/`, how-to guides in `.pair/knowledge/how-to/`. -> **Last updated:** 2026-07-07. Source: `packages/knowledge-hub/dataset/.pair/knowledge/` +> **Last updated:** 2026-07-11. Source: `packages/knowledge-hub/dataset/.pair/knowledge/` ## How-To Guides @@ -106,6 +106,7 @@ Security, accessibility, and performance standards. | Topic | What it covers | | --- | --- | +| **Quality Model** | 3-layer principle, 3 pillars (Cost/Security/Delivery), classification dimensions and tier resolution, per-tier requirements, tag projection, `tech/risk-matrix.md` adoption delta — see the [dedicated page](/docs/reference/quality-model) | | **Accessibility** | WCAG compliance, screen reader support, keyboard navigation | | **Performance** | Load times, Core Web Vitals, optimization | | **Quality Monitoring** | Quality dashboards, trend analysis | @@ -155,7 +156,8 @@ Design systems, research, and content strategy. ## Related -- [Skills Catalog](/docs/reference/skills-catalog) — all 32 pair skills with commands and composition +- [Quality Model](/docs/reference/quality-model) — the single source of default quality rules (classification, tiers, tags) +- [Skills Catalog](/docs/reference/skills-catalog) — all 34 pair skills with commands and composition - [Developer Journey](/docs/developer-journey) — step-by-step walkthrough of the full process - [KB Structure](/docs/reference/kb-structure) — directory layout and file purposes - [Customize for Your Team](/docs/customization/team) — override guidelines via adoption files diff --git a/apps/website/content/docs/reference/meta.json b/apps/website/content/docs/reference/meta.json index 26935077..ded1c1f4 100644 --- a/apps/website/content/docs/reference/meta.json +++ b/apps/website/content/docs/reference/meta.json @@ -5,6 +5,7 @@ "specs", "skills-catalog", "guidelines-catalog", + "quality-model", "skill-management", "kb-structure", "configuration" diff --git a/apps/website/content/docs/reference/quality-model.mdx b/apps/website/content/docs/reference/quality-model.mdx new file mode 100644 index 00000000..12d73062 --- /dev/null +++ b/apps/website/content/docs/reference/quality-model.mdx @@ -0,0 +1,54 @@ +--- +title: Quality Model +description: The single source of default quality rules — 3-layer principle, 3 pillars, classification dimensions, per-tier requirements, tag projection, and the risk-matrix adoption delta. +--- + +The Quality Model is the single source of default quality rules in the pair Knowledge Base. `classify`, `assess-cost`, `assess-security`, code review, quality gates, and bootstrap all resolve their behavior from this one document — no criteria live anywhere else. Source: [`quality-model.md`](https://github.com/foomakers/pair/blob/main/packages/knowledge-hub/dataset/.pair/knowledge/guidelines/quality-assurance/quality-model.md). + +> **Resolution order** for every rule below: **Argument > Adoption > KB default**. A malformed adoption delta is treated as absent — skills warn and fall back to KB defaults. + +## Three-Layer Principle + +| Layer | Role | Examples | +| --- | --- | --- | +| **Doc** | Rules, written once | this document + pillar guidelines | +| **Skill** | Applies the rules on demand | `classify`, `assess-cost`, `assess-security`, code review | +| **Automation** | Consumes tags deterministically, zero judgment | CI gates, `pair-next --filter` | + +Quality is classified in refinement — before code exists — not only at review time (shift-left). The matrix is built twice: once from the story context in refinement, once from the diff in review. Automation never carries its own criteria — it only reads tags. + +## Three Pillars + +| Pillar | Covers | Tag family (if exposed) | +| --- | --- | --- | +| **Cost** | Financial exposure of building/running the change | `cost:*` — opt-in | +| **Security** | Vulnerabilities, compliance, secure-by-design | feeds `risk:*` + deterministic CI scanning | +| **Delivery** | Correctness, performance, a11y, observability, docs, planning, architecture, release, AI metrics | `risk:*` — KB default | + +Every quality theme nests under one of these three pillars, pointing at its existing guideline — no dedicated status page per theme. + +## Classification & Tiers + +The compiled matrix has one row per dimension: service/domain criticality, change/diff risk, business impact, security relevance, and coupling balance. Each resolves to green/yellow/red; the **risk tier is the max across assessed dimensions** and is never lowered once raised by review. Cost is tracked separately as its own class, always computed and written to the matrix — it's projected as the `cost:green|yellow|orange|red` tag only if a project chooses to expose it (see Tag Projection below). + +| Tier | Merge | Reviewers | SLA | Approval | +| --- | --- | --- | --- | --- | +| 🟢 Green | Self-merge at green gate checks | 0 (AI review informational) | — | none | +| 🟡 Yellow | Blocked until reviewed | 1 reviewer | 1 working day | reviewer approval | +| 🔴 Red | Blocked until reviewed and approved | 1 reviewer | 2 working days | explicit approval | + +Reviewer counts and SLAs above are KB defaults, not fixed rules — override either per tier in `tech/risk-matrix.md`'s Overrides section (e.g. `tier.red.reviewers: 2`). A PR without classification tags is treated as 🔴 (fail-safe). + +## Tag Projection & Eligibility + +Tags are a **chromatic projection** of the matrix. `risk:green|yellow|red` is the only tag family the KB names and proposes by default — `classify` asks to activate it the first time it runs and writes the declaration once you confirm, so you get sensible tagging without needing to know the schema exists. Every other parameter the model computes (cost class, security relevance, business impact, coupling balance) is available to expose as its own tag, but the KB doesn't pre-select which, if any — that's entirely your call, added to the declaration when you want it. There is no dedicated "eligibility" tag: automation eligibility is a **filter you declare over classification tags** (e.g. `risk:green`), consumed generically by `pair-next`. + +## Project Overrides — `tech/risk-matrix.md` + +Optional file with up to three independent sections — none imply the others: **Tag Projection** (which tags are active — typically the first section a project ends up with, since `classify` proposes it), a **criticality table** (service/domain → High/Medium/Low), and **overrides** — threshold tweaks plus optional per-tier reviewer count / SLA overrides (e.g. `tier.red.reviewers: 2`). Fully absent means KB defaults apply completely and no tags are emitted — nothing fails. See the full schema in [`quality-model.md` §5–§6](https://github.com/foomakers/pair/blob/main/packages/knowledge-hub/dataset/.pair/knowledge/guidelines/quality-assurance/quality-model.md#6-techrisk-matrixmd--adoption-delta) and a filled-in example at [`risk-matrix-example.md`](https://github.com/foomakers/pair/blob/main/packages/knowledge-hub/dataset/.pair/knowledge/assets/risk-matrix-example.md). + +## Related + +- [Guidelines Catalog](/docs/reference/guidelines-catalog) — where the Quality Model sits among the other 9 guideline categories +- [Customize for Your Team](/docs/customization/team) — how to override guidelines via adoption files +- [KB Structure](/docs/reference/kb-structure) — directory layout, `knowledge/` vs `adoption/` diff --git a/packages/knowledge-hub/dataset/.pair/knowledge/assets/risk-matrix-example.md b/packages/knowledge-hub/dataset/.pair/knowledge/assets/risk-matrix-example.md new file mode 100644 index 00000000..24d567d0 --- /dev/null +++ b/packages/knowledge-hub/dataset/.pair/knowledge/assets/risk-matrix-example.md @@ -0,0 +1,27 @@ +# `tech/risk-matrix.md` — Example + +Illustrative adoption delta for a fictitious e-commerce project. Copy the sections you need — this file is only committed as a reference; a real project's `tech/risk-matrix.md` normally has just a few rows. See the schema and resolution rules in the [quality model](../guidelines/quality-assurance/quality-model.md), §5–§6. + +## Tag Projection + +Active: risk, cost + +`risk` is the only tag family the KB proposes by default — `classify` writes this section itself the first time it runs, once the proposal to activate it is confirmed (§5). This project also added `cost` to the list, since it tracks cloud spend closely. + +## Criticality Table + +| Service/Domain | Criticality | +| --- | --- | +| payments | High | +| checkout | High | +| catalog | Medium | +| marketing-site | Low | +| internal-admin-tool | Low | + +Any service/domain not listed here is treated as unclassified and resolves to High (conservative) for the service-criticality dimension — not to the file-absent Medium default. + +## Overrides + +Optional threshold tweaks for the change-risk dimension (§3.1 of the quality model). Omit entirely if the KB defaults are fine. + +- `change-risk.shared-paths`: `["packages/billing/**", "packages/checkout-core/**"]` — diffs touching these paths are always classified `yellow` or higher for change/diff risk, even if the diff is small. diff --git a/packages/knowledge-hub/dataset/.pair/knowledge/guidelines/quality-assurance/README.md b/packages/knowledge-hub/dataset/.pair/knowledge/guidelines/quality-assurance/README.md index 63ea7de3..ea9d5664 100644 --- a/packages/knowledge-hub/dataset/.pair/knowledge/guidelines/quality-assurance/README.md +++ b/packages/knowledge-hub/dataset/.pair/knowledge/guidelines/quality-assurance/README.md @@ -24,6 +24,7 @@ This comprehensive quality assurance framework ensures systematic quality manage ### **Core Quality Framework** +- **quality-model.md** - Single source of default quality rules: 3-layer principle, 3 pillars (Cost/Security/Delivery), classification dimensions and tier resolution, per-tier review/gate requirements, tag projection, `tech/risk-matrix.md` adoption delta schema, nested taxonomy - **automated-verification.md** - Comprehensive automated verification system for tool-based validation and quality gate enforcement - **manual-verification.md** - Manual verification processes and human-driven quality assessment methodologies - **manual-testing.md** - Guidelines for designing, organizing, and executing manual test suites for post-release validation diff --git a/packages/knowledge-hub/dataset/.pair/knowledge/guidelines/quality-assurance/quality-model.md b/packages/knowledge-hub/dataset/.pair/knowledge/guidelines/quality-assurance/quality-model.md new file mode 100644 index 00000000..0506e576 --- /dev/null +++ b/packages/knowledge-hub/dataset/.pair/knowledge/guidelines/quality-assurance/quality-model.md @@ -0,0 +1,167 @@ +# Quality Model + +The single source of default quality rules for this KB. `classify`, `assess-cost`, `assess-security`, `pair-process-review`, `setup-gates`, and `pair-process-bootstrap` all resolve their behavior from this one document — no criteria live anywhere else. Project deviations are a delta in `tech/risk-matrix.md` (§6); absent, KB defaults apply completely. + +**Resolution order** for every rule below: **Argument > Adoption > KB default**. Argument = an explicit override passed to a skill invocation by a human. Adoption = `tech/risk-matrix.md` (§6). KB default = this document. A malformed adoption file is treated as absent: skills warn and fall back to KB defaults. + +## 1. Three-Layer Principle + +| Layer | Role | Examples | +| --- | --- | --- | +| **Doc** | Rules, written once, human-readable | this document + pillar guidelines (§7) | +| **Skill** | Applies the rules on demand, produces artifacts | `classify`, `assess-cost`, `assess-security`, `pair-process-review` | +| **Automation** | Consumes artifacts deterministically, zero judgment | CI gates, `pair-next --filter` | + +**Shift-left** (R1.3): quality is classified in refinement — before code exists — not only at review time (see [Shift-Left Quality](README.md) in the QA framework overview). The matrix is built twice, refinement and review (§3.2); automation never adds its own criteria (D18) — it only reads tags. + +## 2. Three Pillars + +| Pillar | Covers | Tag family (if exposed) | Primary skill | +| --- | --- | --- | --- | +| **Cost** | Financial exposure of building/running the change | `cost:*` — opt-in, §5 | `assess-cost` (cost-signal catalog, forthcoming) | +| **Security** | Vulnerabilities, compliance, secure-by-design | none dedicated — feeds `risk:*` (§3) + deterministic CI scanning | `assess-security`, [security/](security/README.md) | +| **Delivery** | Everything else: correctness, performance, a11y, observability, docs, planning, architecture, release, AI metrics | `risk:*` (correctness/blast-radius facets) — KB default, §5 | `pair-process-review`, `classify` | + +Every theme not directly named here nests under one of these three — see §7. No status pages, no dedicated backlog per theme: a theme gets a card only when there is real work. + +## 3. Classification Model + +### 3.1 Risk dimensions + +The compiled matrix has one row per dimension below. Each row resolves to `green`/`yellow`/`red`. + +| Dimension | Req. | Source (refinement → review) | green | yellow | red | +| --- | --- | --- | --- | --- | --- | +| Service/domain criticality | R5.1 | `tech/risk-matrix.md` criticality table | Low | Medium (default when the file is absent) | High (default for a service/domain **not listed** in an existing table — conservative) | +| Change/diff risk | R5.2 | story scope → diff footprint | isolated, localized change | touches multiple modules or shared code | schema/migration, contract-breaking change, or infra provisioning change | +| Business impact | R4.3 | subdomain classification of what the story/diff touches | `generic` subdomain | `supporting` subdomain | `core` subdomain | +| Security relevance | — | heuristic over touched paths | no security-sensitive surface | security-adjacent (new external dependency, input validation on a non-critical path) | authn/authz, secrets/credentials, cryptography, PII, untrusted-input parsing | +| Coupling balance | — | story context (touched subdomains' volatility + cross-context integrations) → diff (`assess-coupling` verdict) | balanced | unbalanced + stable | unbalanced + volatile | + +Coupling sources absent (no subdomain/bounded-context artifacts, no `assess-coupling` available) ⇒ reported **not assessed**, excluded from the max below, never blocks (D21). See `architecture/design-patterns/coupling-balance.md` (nested taxonomy entry, §7, not yet published) — the single home for the coupling model itself; this document never duplicates that content, only the classification rule above. + +### 3.2 Tier resolution + +**Risk tier = max(assessed dimensions above)**, projected as `risk:green|yellow|red`. + +- Built **twice** per story (D17): in refinement from the **story context** (declared/estimated), in review from the **code/diff** (observed). The review value is a floor: it may raise the tier, **never lower it**. +- A PR with no classification present is treated as `red` (fail-safe). +- Cost (§3.3) is not part of this max — it is its own class, computed independently and carried in the same compiled matrix; it gets its own tag format, `cost:green|yellow|orange|red`, only if a project chooses to expose it as a tag (§5) — it is never a KB default. + +### 3.3 Cost class (R6.2) + +Cost class = **highest detected signal**. The signal catalog (paid-SDK imports, API-key env vars, IaC/provisioning changes, cron/queues, media processing, LLM calls) is maintained in the cost-assessment guideline (forthcoming, `assess-cost`); no signal detected ⇒ `green`. General + provider-specific heuristics: [infrastructure/cloud-providers/cost-optimization.md](../infrastructure/cloud-providers/cost-optimization.md). This value is always computed and written to the story/PR body's matrix (§1); it is projected as the `cost:green|yellow|orange|red` tag only if a project adds `cost` to its Tag Projection declaration (§5) — the KB does not do this by default. + +## 4. Per-Tier Requirements + +| Tier | Merge | Reviewers | SLA | Checklist | Approval | +| --- | --- | --- | --- | --- | --- | +| 🟢 Green | Self-merge once gate checks are green | 0 (AI review informational, ≤4h) | — | standard | none | +| 🟡 Yellow | Blocked until reviewed | 1 reviewer | 1 working day | standard | reviewer approval | +| 🔴 Red | Blocked until reviewed and approved | 1 reviewer | 2 working days | extended | explicit approval required | + +Reviewer counts and SLAs are **KB defaults** (D10), resolved through the same **Argument > Adoption > KB default** cascade as every other rule in this document — not fixed forever. A project may override either per tier in `tech/risk-matrix.md`'s Overrides section (§6), e.g. requiring 2 reviewers at 🔴 Red for a larger team: + +```markdown +## Overrides + +- tier.red.reviewers: 2 +- tier.red.sla_days: 3 +``` + +Review always runs, tests are always green, at every tier (R5.3 + D10) — this part is not overridable, only the reviewer count and SLA are. Gate (mechanical) and review (judgment) are distinct enforcers — gate blocks first, review starts only once gates are green: + +| Tier | Gate checks | +| --- | --- | +| 🟢 | lint + type + build | +| 🟡 | + unit | +| 🔴 | + integration/E2E | + +## 5. Tag Projection + +Chromatic, no semantic tag beyond color. **`risk:green|yellow|red` (§3.2) is the only tag family the KB names and proposes by default.** + +**Tag emission is declared, not implicit.** `classify` only creates tags once a `## Tag Projection` declaration exists in `tech/risk-matrix.md` (§6) — but that declaration is not something a project has to remember to write from scratch: + +- **Only `risk` is a KB default.** Every other parameter this model computes — cost class (§3.3), security relevance, business impact, coupling balance, or any dimension added later — is available to expose as its own tag, but the KB does not pre-select which, if any: that choice belongs entirely to the project. Adding a parameter to the declaration (e.g. `Active: risk, cost`) is what exposes it; nothing beyond `risk` is projected until a project explicitly says so. +- **`classify` proposes only the `risk` default on its own first run**, the same propose-then-write-if-confirmed pattern already used elsewhere in this KB (e.g. `pair-capability-verify-quality`'s first-time Custom Gate Registry setup): + 1. **Check**: does `tech/risk-matrix.md` have a `## Tag Projection` section? + 2. **Skip**: if yes, use it exactly as written — including an explicit opt-out (see below) — and never propose again. + 3. **Act**: if no, ask before creating any tag: + + > No Tag Projection declared yet. Activate `risk:green|yellow|red` on stories and PRs? (recommended — other model parameters can be exposed as tags later, if you decide you want them) + > 1. Yes, activate `risk` (writes the declaration below) + > 2. No, don't tag anything (records the opt-out so this isn't asked again) + + 4. **Verify**: the compiled matrix is written to the story/PR body **regardless of the answer** — §3.2/§3.3's body output never depends on tag projection; only tag *emission* is gated by it. +- Until the proposal is answered, or if it's explicitly declined, the matrix still exists in the story/PR body — it is simply not projected onto tags. + +```markdown +## Tag Projection + +Active: risk +``` + +A project decides which other model parameters, if any, to expose by adding them to the `Active` list — e.g. `Active: risk, cost` if it also wants the cost class (§3.3) projected as a tag; the choice, and the resulting tag's color scheme, follows whichever parameter was added. Write `Active: none` to explicitly opt out of all tag emission (`classify` reads this as a durable "don't ask again," not as "not yet configured"). A project may also rename `risk` itself here (e.g. `risk` → `priority`) — the color values and their meaning stay the same, only the label changes. + +**No dedicated eligibility tag**: automation eligibility is an **adoption-declared filter over classification tags** (e.g. `risk:green`, optionally combined with project tags), not a special tag of its own. `pair-next` consumes it generically, like any other tag filter, re-evaluated on every run (tags can change between runs, e.g. review raising the tier). + +## 6. `tech/risk-matrix.md` — Adoption Delta + +Optional file holding up to three independent sections — a project may have none, one, or all three; the presence of one never implies the others: + +- **`## Tag Projection`** (§5) — which classification tags get emitted. In practice the section most projects end up with first, since `classify` proactively proposes it the first time it runs (§5) — a project doesn't have to know this file exists to get a sensible default. +- **`## Criticality Table`** — per-service/domain criticality overrides (§3.1). +- **`## Overrides`** — threshold overrides for other dimensions, plus optional per-tier reviewer-count/SLA overrides (§4). + +Absent entirely ⇒ KB defaults (§3.1) apply completely to the matrix, and no tags are emitted (§5) — nothing fails (D21). This is the state before `classify` has ever run, or before its Tag Projection proposal has been answered. + +```markdown +## Tag Projection + +Active: risk + +## Criticality Table + +| Service/Domain | Criticality | +| --- | --- | +| payments | High | +| marketing-site | Low | + +## Overrides + +- change-risk.shared-paths: ["packages/billing/**"] +``` + +- **Malformed file** (unparseable table, unknown keys): skills warn and fall back to KB defaults entirely (D21) — including no tag emission, exactly as if the whole file were absent. +- **Unknown service/domain** (queried but not in the criticality table): treated as unclassified ⇒ conservative High for that dimension. +- A filled-in example (also usable as adoption starting point) is at [risk-matrix-example.md](../../assets/risk-matrix-example.md). + +### Resolution-cascade walkthrough + +| Scenario | `tech/risk-matrix.md` | Resolution | +| --- | --- | --- | +| No file, or Tag Projection proposal not yet answered | absent, or missing `## Tag Projection` | Matrix computed and written to the story/PR body per §3.1 defaults; no tags emitted — `classify` proposes the Tag Projection declaration on its next run | +| Tag Projection declared, `risk` active | `## Tag Projection` → `Active: risk` | `risk:*` tag applied to the story/PR alongside the body matrix; `classify` never re-proposes | +| Tag Projection explicitly opted out | `## Tag Projection` → `Active: none` | Matrix written to the body; no tags applied; `classify` never re-proposes | +| File present, service listed | `payments: High` | `payments` resolves to red for that dimension, overriding the Medium default (AC3) | +| File present, service **not** listed | table has other entries only | Conservative High (red) for that dimension, not the absent-file Medium default | +| File present but malformed | unparseable | Warn, fall back to KB defaults as if absent (including no tag emission) | + +## 7. Nested Taxonomy + +Every quality theme not covered by §1–§6 lives under one of the three pillars, pointing at its existing guideline — no new status page, no dedicated backlog structure per theme (D13). + +| Theme | Pillar | Guideline | +| --- | --- | --- | +| Performance | Delivery | [performance/README.md](performance/README.md) | +| Accessibility | Delivery | [accessibility/README.md](accessibility/README.md) | +| Observability | Delivery | [../observability/README.md](../observability/README.md) | +| Documentation | Delivery | [../technical-standards/ai-development/documentation-standards.md](../technical-standards/ai-development/documentation-standards.md) | +| Planning | Delivery | [../collaboration/methodology/README.md](../collaboration/methodology/README.md) | +| Code design / code quality | Delivery | [../code-design/README.md](../code-design/README.md) | +| Architecture / modularity | Delivery | `architecture/design-patterns/coupling-balance.md` (not yet published — single home for the coupling model, see §3.1) | +| Release | Delivery | [../technical-standards/deployment-workflow/release-management.md](../technical-standards/deployment-workflow/release-management.md) | +| AI metrics / retro | Delivery | [../collaboration/project-tracking/README.md](../collaboration/project-tracking/README.md) (reports land in `.pair/working/reports/`, once available) | +| Vulnerabilities / compliance | Security | [security/vulnerability-prevention.md](security/vulnerability-prevention.md), [security/compliance.md](security/compliance.md) | +| Cost signals | Cost | cost-assessment guideline (not yet published, see §3.3) | diff --git a/packages/knowledge-hub/src/quality-model.test.ts b/packages/knowledge-hub/src/quality-model.test.ts new file mode 100644 index 00000000..fa748dc8 --- /dev/null +++ b/packages/knowledge-hub/src/quality-model.test.ts @@ -0,0 +1,171 @@ +import { describe, it, expect } from 'vitest' +import { readFileSync, existsSync } from 'fs' +import { join } from 'path' + +const QUALITY_ASSURANCE_DIR = join( + __dirname, + '../dataset/.pair/knowledge/guidelines/quality-assurance', +) + +const QUALITY_MODEL = readFileSync( + join( + __dirname, + '../dataset/.pair/knowledge/guidelines/quality-assurance/quality-model.md', + ), + 'utf-8', +) +const RISK_MATRIX_EXAMPLE = readFileSync( + join(__dirname, '../dataset/.pair/knowledge/assets/risk-matrix-example.md'), + 'utf-8', +) +const QA_README = readFileSync( + join(__dirname, '../dataset/.pair/knowledge/guidelines/quality-assurance/README.md'), + 'utf-8', +) + +describe('quality-model.md — structure', () => { + it('has the expected title', () => { + expect(QUALITY_MODEL).toMatch(/^# Quality Model/m) + }) + + it('documents the 3-layer principle', () => { + expect(QUALITY_MODEL).toMatch(/Three-Layer Principle/) + expect(QUALITY_MODEL).toMatch(/\*\*Doc\*\*/) + expect(QUALITY_MODEL).toMatch(/\*\*Skill\*\*/) + expect(QUALITY_MODEL).toMatch(/\*\*Automation\*\*/) + }) + + it('documents the 3 pillars', () => { + expect(QUALITY_MODEL).toMatch(/\*\*Cost\*\*/) + expect(QUALITY_MODEL).toMatch(/\*\*Security\*\*/) + expect(QUALITY_MODEL).toMatch(/\*\*Delivery\*\*/) + }) + + it('documents all 5 risk dimensions with tier = max', () => { + for (const dim of [ + 'Service/domain criticality', + 'Change/diff risk', + 'Business impact', + 'Security relevance', + 'Coupling balance', + ]) { + expect(QUALITY_MODEL).toContain(dim) + } + expect(QUALITY_MODEL).toMatch(/max\(assessed dimensions/i) + }) + + it('matches the D10 SLA table exactly — 1 reviewer even at red tier', () => { + expect(QUALITY_MODEL).toMatch(/🟢[^\n]*Self-merge/) + expect(QUALITY_MODEL).toMatch(/🟡[^\n]*\|\s*1 reviewer\s*\|\s*1 working day/) + expect(QUALITY_MODEL).toMatch(/🔴[^\n]*\|\s*1 reviewer\s*\|\s*2 working days/) + }) + + it('states reviewer counts and SLAs are adoption-overridable KB defaults, not fixed', () => { + expect(QUALITY_MODEL).toMatch( + /Reviewer counts and SLAs are \*\*KB defaults\*\*[\s\S]{0,300}Argument > Adoption > KB default/, + ) + expect(QUALITY_MODEL).toContain('tier.red.reviewers: 2') + expect(QUALITY_MODEL).toMatch(/this part is not overridable, only the reviewer count and SLA are/) + }) + + it('defines the chromatic tag projection', () => { + expect(QUALITY_MODEL).toContain('risk:green|yellow|red') + expect(QUALITY_MODEL).toContain('cost:green|yellow|orange|red') + }) + + it('states risk is the only KB-named/proposed tag family and does not special-case cost as a second default', () => { + expect(QUALITY_MODEL).toMatch( + /`risk:green\|yellow\|red` \(§3\.2\) is the only tag family the KB names and proposes by default/, + ) + expect(QUALITY_MODEL).toMatch(/Only `risk` is a KB default/) + expect(QUALITY_MODEL).toMatch( + /the KB does not pre-select which, if any: that choice belongs entirely to the project/, + ) + }) + + it("documents classify's propose-then-write Tag Projection flow", () => { + expect(QUALITY_MODEL).toMatch(/does `tech\/risk-matrix\.md` have a `## Tag Projection` section/) + expect(QUALITY_MODEL).toMatch(/No Tag Projection declared yet\. Activate `risk:green\|yellow\|red`/) + expect(QUALITY_MODEL).toMatch(/records the opt-out so this isn't asked again/) + expect(QUALITY_MODEL).toMatch( + /the compiled matrix is written to the story\/PR body \*\*regardless of the answer\*\*/, + ) + }) + + it('shows the Tag Projection declaration schema (default, multi-family, opt-out)', () => { + expect(QUALITY_MODEL).toMatch(/## Tag Projection\n\nActive: risk\n/) + expect(QUALITY_MODEL).toContain('Active: risk, cost') + expect(QUALITY_MODEL).toContain('Active: none') + }) + + it('states there is no dedicated eligibility tag', () => { + expect(QUALITY_MODEL).toMatch(/No dedicated eligibility tag/) + }) + + it('states the Argument > Adoption > KB default resolution order', () => { + expect(QUALITY_MODEL).toContain('Argument > Adoption > KB default') + }) + + it('states tech/risk-matrix.md holds 3 independent sections that do not imply each other', () => { + expect(QUALITY_MODEL).toMatch(/up to three independent sections/) + expect(QUALITY_MODEL).toMatch(/the presence of one never implies the others/) + }) + + it('walks through the resolution cascade with and without the delta file', () => { + expect(QUALITY_MODEL).toMatch(/No file[^|]*\|\s*absent/) + expect(QUALITY_MODEL).toMatch(/Tag Projection declared, `risk` active/) + expect(QUALITY_MODEL).toMatch(/Tag Projection explicitly opted out/) + expect(QUALITY_MODEL).toMatch(/File present, service listed/) + expect(QUALITY_MODEL).toMatch(/File present, service \*\*not\*\* listed/) + expect(QUALITY_MODEL).toMatch(/File present but malformed/) + }) + + it('does not create dead hyperlinks for guidelines that do not exist yet', () => { + expect(QUALITY_MODEL).not.toMatch(/\]\([^)]*coupling-balance\.md\)/) + expect(QUALITY_MODEL).toContain('`architecture/design-patterns/coupling-balance.md`') + }) + + it('resolves every §7 nested-taxonomy pointer link to a file on disk', () => { + const section = QUALITY_MODEL.split('## 7. Nested Taxonomy')[1] + const links = [...section.matchAll(/\]\(([^)]+)\)/g)].map((m) => m[1]) + expect(links.length).toBeGreaterThanOrEqual(9) + for (const link of links) { + expect(existsSync(join(QUALITY_ASSURANCE_DIR, link))).toBe(true) + } + }) + + it('nests every listed theme under a pillar with a pointer', () => { + for (const theme of [ + 'Performance', + 'Accessibility', + 'Observability', + 'Documentation', + 'Planning', + 'Code design / code quality', + 'Architecture / modularity', + 'Release', + 'AI metrics / retro', + 'Vulnerabilities / compliance', + 'Cost signals', + ]) { + expect(QUALITY_MODEL).toContain(theme) + } + }) +}) + +describe('risk-matrix-example.md', () => { + it('provides a criticality table with at least one High entry', () => { + expect(RISK_MATRIX_EXAMPLE).toMatch(/## Criticality Table/) + expect(RISK_MATRIX_EXAMPLE).toMatch(/\|\s*payments\s*\|\s*High\s*\|/) + }) + + it('documents the unknown-service default separately from the file-absent default', () => { + expect(RISK_MATRIX_EXAMPLE).toMatch(/resolves to High/) + }) +}) + +describe('quality-assurance README — index', () => { + it('lists quality-model.md in the core quality framework section', () => { + expect(QA_README).toMatch(/quality-model\.md/) + }) +})