The check gate has grown slow enough that it shapes how work gets done. bash scripts/check.sh runs four groups — python, frontend, generated, browser — and the two browser suites in that last group are minutes each on their own. The cost is paid on every push by every contributor and on every pull request by CI, and it has reached the point where the honest response to "run the full gate before you push" is a sigh rather than a command.
This issue tracks a program to bring that cost down by removing tests that no longer earn their runtime, and by consolidating variants that differ only in boilerplate. The outcome being measured is wall time. The constraint that outranks it is that the suite must lose no defect-detection power at all: a faster gate that catches less is a worse gate, and there is no amount of time saving that buys a regression reaching main unnoticed.
The two stages, and the gate between them
Stage 1 produces a catalog and changes nothing. Every test in the repository is read against the code it exercises at HEAD — not against what its name or its docstring claims it does — and gets exactly one verdict: keep, keep (protected), remove (redundant), remove (irrelevant), consolidate, demote-to-nightly, or fix-or-remove. The unit of analysis is what defect would have to exist in the product for the test to go red. Two tests are redundant only when that answer is genuinely the same defect set, which is a stricter test than their names sounding alike. No file in the repository is modified during this stage; the deliverable is comments on this issue, one per area.
Each verdict carries its own burden of proof. A remove (redundant) names the surviving test that catches the same defects and says in a sentence why the sets are identical. A remove (irrelevant) names the commit or pull request after which the behavior it guards stopped existing — and if that aging event cannot be found in the history, the verdict reverts to keep. A consolidate shows a table proving every distinguishing input of every absorbed variant survives as a parameter case, because consolidation is allowed to reduce boilerplate and count but never the input space. demote-to-nightly is reserved for tests that are expensive and valuable and guard slow-changing behavior, and since it changes CI policy rather than test content it ships only if the ratification names it explicitly.
The pass also looks for a failure mode that is neither keep nor remove. A vacuous test — one that passes without exercising anything — is broken rather than redundant, and gets fix-or-remove and a separate flag. The shape already seen in this repository is an absence assertion made against a test double built from a remembered signature: the double's field name does not match the real model, the code under test never runs, and nothing happened is satisfied for the wrong reason. Deleting such a test quietly would hide the fact that whatever it claimed to guard may be unguarded.
Stage 2 executes, and only what was ratified. It follows the refactor-protocol skill in full: isolated worktrees, the complete gate per pull request with both browser suites in Chromium, one pull request per catalog area, manual merge after CI is green, and the three-strike rule on repeated CI failure. Removals are followed by a coverage re-measurement against the Stage 1 baseline, and coverage may not decrease because of one; a drop means the catalog was wrong about that test and the test comes back. Consolidations are verified by targeted mutation before any original is deleted — for each absorbed variant, the defect that variant guarded is introduced into the product, the consolidated test is confirmed red, and the mutation is reverted. Only when every absorbed variant's defect turns the replacement red do the originals go.
Nothing crosses from Stage 1 to Stage 2 without a ratification comment on this issue, in the Decision (Armando, <date>): … format. That comment, not the catalog, is the executable list; where it amends the catalog, the amendment wins. An approving reply that does not carry the decision format is not ratification.
What is never a candidate
Some tests are policy enforcement rather than coverage, and they appear in the catalog with verdict keep (protected) and the reason they are protected, regardless of how redundant or expensive they look. That list covers everything under tests/architecture/ together with the import-linter contracts' companion tests, which are the machine enforcement of kernel purity; tests/scripts/annotator_boundary.test.mjs and the three gates it proves fire, which are the same thing for the headless annotator core; the token parity suite and the hardcoded-color gate; the parametrized allowed_actions contract suite, which is the proof that the frontend renders what the wire declares; the annotator stage-and-background contrast assertion in the browser suite; and any test that a skill, DESIGN.md or AGENTS.md names as the enforcement of a rule. The order-dependent kernel test test_configuring_a_connection_reaches_no_model_runtime is also outside this sweep, having been dealt with on its own terms already.
Runnable scripts under examples/ are out of scope entirely. CI exercises them, but they are documentation rather than tests.
Measurement
A baseline is posted on this issue before any analysis begins: full gate wall time with the per-step breakdown, a complete per-test timing list from pytest, per-package vitest timing, per-spec Playwright timing, and per-package coverage saved for the Stage 2 comparison. The same measurement is re-run on main after the last pull request merges, and the final report states tests and wall time before and after per suite, plus every deviation from the ratified list with the reason it was deviated from. A test that turns red during execution is a discovery to be reported, never an obstacle to be argued out of the diff.
The check gate has grown slow enough that it shapes how work gets done.
bash scripts/check.shruns four groups — python, frontend, generated, browser — and the two browser suites in that last group are minutes each on their own. The cost is paid on every push by every contributor and on every pull request by CI, and it has reached the point where the honest response to "run the full gate before you push" is a sigh rather than a command.This issue tracks a program to bring that cost down by removing tests that no longer earn their runtime, and by consolidating variants that differ only in boilerplate. The outcome being measured is wall time. The constraint that outranks it is that the suite must lose no defect-detection power at all: a faster gate that catches less is a worse gate, and there is no amount of time saving that buys a regression reaching
mainunnoticed.The two stages, and the gate between them
Stage 1 produces a catalog and changes nothing. Every test in the repository is read against the code it exercises at
HEAD— not against what its name or its docstring claims it does — and gets exactly one verdict:keep,keep (protected),remove (redundant),remove (irrelevant),consolidate,demote-to-nightly, orfix-or-remove. The unit of analysis is what defect would have to exist in the product for the test to go red. Two tests are redundant only when that answer is genuinely the same defect set, which is a stricter test than their names sounding alike. No file in the repository is modified during this stage; the deliverable is comments on this issue, one per area.Each verdict carries its own burden of proof. A
remove (redundant)names the surviving test that catches the same defects and says in a sentence why the sets are identical. Aremove (irrelevant)names the commit or pull request after which the behavior it guards stopped existing — and if that aging event cannot be found in the history, the verdict reverts tokeep. Aconsolidateshows a table proving every distinguishing input of every absorbed variant survives as a parameter case, because consolidation is allowed to reduce boilerplate and count but never the input space.demote-to-nightlyis reserved for tests that are expensive and valuable and guard slow-changing behavior, and since it changes CI policy rather than test content it ships only if the ratification names it explicitly.The pass also looks for a failure mode that is neither keep nor remove. A vacuous test — one that passes without exercising anything — is broken rather than redundant, and gets
fix-or-removeand a separate flag. The shape already seen in this repository is an absence assertion made against a test double built from a remembered signature: the double's field name does not match the real model, the code under test never runs, and nothing happened is satisfied for the wrong reason. Deleting such a test quietly would hide the fact that whatever it claimed to guard may be unguarded.Stage 2 executes, and only what was ratified. It follows the
refactor-protocolskill in full: isolated worktrees, the complete gate per pull request with both browser suites in Chromium, one pull request per catalog area, manual merge after CI is green, and the three-strike rule on repeated CI failure. Removals are followed by a coverage re-measurement against the Stage 1 baseline, and coverage may not decrease because of one; a drop means the catalog was wrong about that test and the test comes back. Consolidations are verified by targeted mutation before any original is deleted — for each absorbed variant, the defect that variant guarded is introduced into the product, the consolidated test is confirmed red, and the mutation is reverted. Only when every absorbed variant's defect turns the replacement red do the originals go.Nothing crosses from Stage 1 to Stage 2 without a ratification comment on this issue, in the
Decision (Armando, <date>): …format. That comment, not the catalog, is the executable list; where it amends the catalog, the amendment wins. An approving reply that does not carry the decision format is not ratification.What is never a candidate
Some tests are policy enforcement rather than coverage, and they appear in the catalog with verdict
keep (protected)and the reason they are protected, regardless of how redundant or expensive they look. That list covers everything undertests/architecture/together with the import-linter contracts' companion tests, which are the machine enforcement of kernel purity;tests/scripts/annotator_boundary.test.mjsand the three gates it proves fire, which are the same thing for the headless annotator core; the token parity suite and the hardcoded-color gate; the parametrizedallowed_actionscontract suite, which is the proof that the frontend renders what the wire declares; the annotator stage-and-background contrast assertion in the browser suite; and any test that a skill,DESIGN.mdorAGENTS.mdnames as the enforcement of a rule. The order-dependent kernel testtest_configuring_a_connection_reaches_no_model_runtimeis also outside this sweep, having been dealt with on its own terms already.Runnable scripts under
examples/are out of scope entirely. CI exercises them, but they are documentation rather than tests.Measurement
A baseline is posted on this issue before any analysis begins: full gate wall time with the per-step breakdown, a complete per-test timing list from pytest, per-package vitest timing, per-spec Playwright timing, and per-package coverage saved for the Stage 2 comparison. The same measurement is re-run on
mainafter the last pull request merges, and the final report states tests and wall time before and after per suite, plus every deviation from the ratified list with the reason it was deviated from. A test that turns red during execution is a discovery to be reported, never an obstacle to be argued out of the diff.