RUE-506: ADR-0081 — rue test MVP (test declarations, runner, event protocol)

[DorianListens]

Refs RUE-506. Successor to #2239, scoped down per the review consensus there: this is the MVP ADR, and only the MVP ADR. The document stands alone — the process history is two sentences in Status, and reading #2239 is not required.

What the MVP commits to

• Tests are language items: test "name" { ... } blocks (contextual keyword, preview-gated as test_declarations), placement-is-visibility, tests rooted only by test requests. ? in test bodies gets unwrap-and-report semantics with the skipped-destructor consequence accepted explicitly.
• rue test as the driver's first subcommand, with worked CLI examples in the text: NDJSON event stream v1.0, byte-safe bounded capture, asymmetric verbosity, stable IDs, repro argv on every failure, and capability_summary present from v1.0 with an explicit unavailable status.
• Execution as a contract: one test image per target, one process per test, both the loader-visible and test-visible inventories pinned to exact values.
• The structured failure channel (dedicated-pipe recommendation, ABI call under ADR-0055) plus Phase 2.5 structured assertion payloads — agent-first in content, not just transport.
• An explicit compiler-vs-build boundary section: everything in the ADR is compiler behavior; build integration supplies exactly one optional input (--test-candidates for the orphan warning).

The open discovery question

File discovery is a first-class maintainer call, to be taken before Phase 2: import-closure-only with the orphan warning, vs a closure-anchored naming convention that auto-roots conventional test files (a bounded compiler demand over directories the closure occupies — not a directory walk). The import-wiring papercut, its Zig history, and the convention's costs (spec-pinned name, directory contents as request inputs, whole-run blast radius until per-test compile_error lands) are stated in §1 and Open Questions.

What this defers, and where it went

Capability inference, hermetic verdict caching/selection, scheduling/flake policy, and the public provider protocol are deferred to focused follow-up ADRs — deferred, not rejected. §6 records what the MVP does now so each lands additively.

The full design capture stays on #2239 (closed unmerged, five review rounds). Follow-up issues seeded with the extracted sections: RUE-1621 (capability inference), RUE-1622 (verdict caching, blocked by 1621), RUE-1623 (scheduling, partially blocked by 1621), RUE-1624 (provider protocol) — Linear project "rue test follow-ups". MVP phases are RUE-1618/1619/1620 in project "rue test MVP".

Registry and doc-link gates pass (82 records).

🤖 Generated with Claude Code

https://claude.ai/code/session_01NPBpmZU3DpwkumLHNwHsgu

[DorianListens]

Pushed 345daade: fixes from an adversarial review pass (Fable subagent) over the distillation. All nine findings were remnants of the pre-narrowing scope rather than new design content; the grounding spot-checks (E0503/E0505, abort-101, rue-test-runner mechanics, both manifest scripts, the 11 value-taking flags, the 46-helper manifest, ADR-0061/0063 citations) all verified clean against current trunk.

The four that mattered:

1. §6 scheduling bullet miscounted its own deferral list — it named five items ("…and the seedable @random_* maintainer call") while keeping the original's "two of its four items require capability inference," making the dependency unrecoverable. Now four items with the gated two named explicitly (--reruns-for-non-hermetic, hermetic-mismatch reporting) and the seedable-@random_* call attached as its own sentence.
2. §4's shuffle bullet claimed present-tense verified isolation the MVP disclaims — the one un-rewired hermeticity claim left in a Decision section. Now: shuffling keeps order dependence visible today; the impossible-for-hermetic-tests claim arrives with inference.
3. Per-test compile_error verdicts were falsely gated on capability inference by the blanket "Requires capability inference" on the RUE-1622 bullet — §3's mechanism (independent per-test roots, exclusion not stubbing) needs no summaries and no cache. Now scoped: caching/selection items require inference; the compile_error mechanism may land independently.
4. The seedable-@random_* call was deferred to two different places (§4 said scheduling and caching; §6 said RUE-1623). Now consistent: the call is RUE-1623's; its cache-key interaction travels with RUE-1622.

Smaller: the Go/Bazel prior-art bullets no longer assert "Rue verifies/enforces" in present tense; Phase 2's bold note becomes "the minimum usable runner" so "the MVP" consistently means Phases 1–2.5 as the title and Summary use it; ice is reserved alongside compile_error/cached_pass per the document's own unproducible-verdict principle (in the MVP a compiler crash is a whole-run failure on stderr, not a per-test event); the dangling HyRTS citation (it supported the dropped selection-economics argument) is removed; ADR-0038 and ADR-0005 join relates, both load-bearing in the body; undefined --keep-going leaves the synopsis.

Registry and doc-link gates still pass (82 records).

[DorianListens]

Pushed 4426d161: a full editorial pass per maintainer notes, 1,259 → 846 lines. Four changes of substance beyond compression:

1. The document now stands alone. Process history is two sentences in Status; Context is reduced to the facts the design is built on plus one-lesson-each prior art.
2. The UX is shown, not described: worked rue test invocations with illustrative human and NDJSON output open §2, and §1 shows the import-wiring idiom as code with its tradeoff stated plainly.
3. A new top-level boundary section pins compiler-CLI vs build-integration: everything specified is compiler behavior; the build supplies exactly one optional input (--test-candidates).
4. File discovery is now an explicit maintainer call (before Phase 2): import-closure + orphan warning as drafted, vs a closure-anchored naming convention that auto-roots conventional test files — bounded probe over closure directories, not a walk; kills the Zig wiring papercut for sibling tests and makes orphan detection self-contained, at the cost of a spec-pinned name pattern and (until per-test compile_error lands) whole-run blast radius for a broken conventional file. The package-model end state (declared sources as canonical inventory, auto-rooted) is recorded under deferred questions.

Every contract value, reserved schema surface, and adversarial-review fix from 345daade is preserved. Gates pass.

[DorianListens]

Pushed 2e7be7a8: one ruling taken, and Open Questions reorganized by impact.

Ruled: no implicit preview enable. rue test requires --preview test_declarations like every other preview feature. The deciding observation: the gate covers a parser change, so any request whose closure contains test items — executable builds included, which parse test items for the warnings scan — needs the flag to compile at all. Auto-enabling in test mode alone would leave those same files failing ordinary builds while making rue test the first flag to bypass ADR-0005's explicit opt-in, for no net convenience. §1's preview-gate bullet now records the decision and rationale; the open question is removed.

Open Questions now splits into two groups. "Needs a ruling before acceptance": the test declaration surface, the file discovery mechanism, and --filter root-vs-run semantics — the three calls that shape the language surface and discovery machinery. "Lower-impact decisions (decidable within their phase)": failure-channel mechanism, @assert/exit-code stabilization, the 0/1/2/3 contract, the skipped verdict, the scripts/rue test homonym, and naming.

[steveklabnik]

Codex review, requested by Steve.

The overall MVP boundary is much clearer after the editorial pass, and I recommend resolving the three acceptance-level questions as follows: keep the test "name" { ... } surface; use import-closure discovery for the MVP (with the optional declared candidate inventory warning) rather than adding directory contents as compiler inputs; and have --filter narrow the run set, not the analysis root set. The dedicated inherited failure pipe is also the right Phase 2 mechanism.

I found two correctness/sequencing gaps that should be resolved in the ADR:

1. Phase 1 says test-body ? becomes legal and test bodies semantically analyze, while the failure-arm lowering is deferred to Phase 2. Current semantic analysis does not have a separable legality-only step: analyze_try immediately builds the Option/Result match and early-return AIR against the enclosing return type. A ()-typed test body therefore cannot analyze successfully until the test-specific failure arm exists. Move legality and lowering together into Phase 2, or specify a real Phase 1 lowering that makes the claimed emit-level analysis possible.

2. The accepted std.exit(0) blind spot makes a test that exits before its assertions report as a pass. With one process per test and the dedicated parent-child channel already in the MVP, require a terminal completion record that only the dispatcher writes after the test body returns normally. Exit 0/EOF without that record should be a runner failure. This is false-positive test evidence, so it should not be deferred as optional hygiene.

The ADR-registry and documentation-link gates pass at this head.

— Codex

[DorianListens]

Both gaps resolved and all four rulings recorded, as f245ecac1.

Gap 1 — the Phase 1/2 ? split. You're right that there's no separable legality-only step: analyze_try builds the Option/Result match and early return against the enclosing return type as it analyzes, so a ()-typed body can't analyze successfully until the test-specific failure arm exists. Taken as recommended: legality and lowering move together into Phase 2, Phase 1 keeps today's compile error for test-body ?, and §1 states the mechanism reason so the constraint doesn't get re-derived at implementation time.

Gap 2 — the std.exit(0) blind spot. Agreed it's false-positive evidence, not hygiene debt. The dispatcher now writes a terminal completion record on the dedicated channel after the body returns normally, and exit 0 with end-of-stream but no record is a failure with new kind incomplete. The pass definition (§1), the event sketch (§2), the execution contract (§3), and Phase 2's item all carry it. One honest caveat is stated where the mechanism lives: the channel is not a security boundary, so deliberate forgery remains possible — the defect closed is the accidental early exit.

Rulings recorded: test "name" { ... } blocks; import-closure discovery for the MVP with directory contents staying out of compiler inputs (the convention alternative is preserved under Rejected alternatives with the package-model revisit named); --filter narrows the run set, never the root set (§2 now states the verdict-stability rationale inline); dedicated inherited pipe. Open Questions' front group is now the ruling record, and the Status section reflects it.

Registry and doc-link gates pass at this head.