feat(trace): ratchet on scope code no bound test's package can link (#1405) - #1420
Merged
Merged
Conversation
…1405) `req-mutation.sh` mutates every file in a requirement's scope and runs its bound tests under cargo-mutants `--test-package`. A test binary cannot link code outside its own package closure, so a mutant in an unlinkable package is unkillable BY CONSTRUCTION and can only be recorded MISSED. The script's verdict is `killed > 0`, so REQ-FUN-11 passed on the strength of one linkable file while 401 of its 408 mutants could never die — a PASS that was partly about the ownership map. A GRANDFATHERED RATCHET, which is the repo's existing third mode (reachability.sh, NOT-GRANDFATHERED), because the issue's fail-or-warn pair are both wrong here: failing is red-on-arrival (#1074) with three requirements blocked on #1234, and warning repeats DANGLING-CODE, which correctly detects CAP-70's dead path and has never failed a build. NEW unlinkable code fails; the 15 baselined pairs warn; `baseline - current` also FAILS, because #1371 found 74 of 86 orphan-baseline entries stale in one pass under a header that merely asked for pruning. Semantics VERIFIED against cargo-mutants 27.1.0 rather than inferred: `lab.rs` passes only the `--test-package` list (the mutated package is not added) and `cargo.rs` runs `cargo test --package=<p> --no-default-features`, so linkable is P + P's normal/dev/build deps, then normal/build transitively. Deliberately NOT `_workspace_graph()`, which filters dev and optional edges because its question is production reach; a test target does link its own package's dev-deps. Excluding optional edges was right for the wrong reason — the rule is really feature resolution rooted at P under `--no-default-features` — so it is now a CHECKED precondition, not a constant: a graph-self-test probe fails when any internal dep spec activates an optional internal edge. Also: a file that is a `bin` root is linkable only from its own package (a dependent's tests never build a dependency's bin); `unwired` is in scope so REQ-CTL-04's three files do not all fail the day #1234 lands; findings carry best-effort `cargo mutants --list` counts, never a silent 0 (#1279), because the spread is 46x (186 mutants in linksec/async_channel.rs against 4 in modem/envelope_codec.rs). Named UNLINKABLE, not UNREACHABLE: reachability.sh already uses "reachable" for production reach, and a second sense invites exactly the over-read this avoids. Linkability is NECESSARY, NOT SUFFICIENT (#1415) — stated in the finding text, the baseline header and the code. It cannot see a vacuous binding in the right package, nor cfg-gated code inside a linkable file. Sabotage-verified in both directions, each failing its own case, with the unmodified-tree positive control still passing: scope file no bound test's package can link -> UNLINKABLE-FROM-BINDING baseline entry that is no longer unlinkable -> STALE-LINK-BASELINE Verification-objective: a requirement's mutation verdict must not be silently bounded by its own scope; scope code no bound test can link is reported and ratcheted Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0188ATCj6DZ9aRVQ2vSirua6
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1405.
req-mutation.shmutates every file in a requirement's scope and runs its bound tests undercargo-mutants
--test-package. A test binary cannot link code outside its own package closure, soa mutant in an unlinkable package is unkillable by construction and can only ever be recorded MISSED.
The script's verdict is
killed > 0, so REQ-FUN-11 passed on the strength of one linkable filewhile 401 of its 408 mutants could never die — a per-requirement PASS that was partly a statement
about the ownership map rather than about the tests.
Why a ratchet, and not the issue's fail-or-warn
Both options the issue framed are wrong here. Failing is red-on-arrival (#1074): three of the six
affected requirements are blocked on #1234, where nothing depends on
openpulse-keystore, and nobinding placement can fix that. Warning repeats
DANGLING-CODE, which correctly detects CAP-70'sdead path and has never failed a build.
A grandfathered ratchet is this repo's existing third mode (
reachability.sh,NOT-GRANDFATHERED): NEW unlinkable code fails, the 15 baselined pairs warn, andbaseline − currentalso fails — because #1371 found 74 of 86 orphan-baseline entries stale inone pass under a header that merely asked for pruning.
Semantics verified against the tool, not inferred
This was the part most likely to sink the design, and I had guessed it. Fable read cargo-mutants
27.1.0:
lab.rsselectsTestsForMutant::Explicit(packages)— the--test-packagelist only, themutated package is not added — and
cargo.rsissuescargo test --package=<p> --no-default-features. So linkable = P + P's normal/dev/build deps, then normal/build transitively.Deliberately not
_workspace_graph(), which filters dev and optional edges because its questionis production reach. A test target does link its own package's dev-dependencies.
Excluding optional edges turned out to be right for the wrong reason — the real rule is feature
resolution rooted at P under
--no-default-features— so it is now a checked precondition, not aconstant.
Sabotage-verified, every direction
Each fails its own case, and the unmodified-tree positive control still passes — which is what proves
the plants were restored rather than the probes being loud.
Stated blindnesses
Linkability is necessary, not sufficient (#1415), and the name says "link" because
reachability.shalready uses "reachable" for production reach. It cannot see a vacuous binding inthe right package — that is
req-mutation.sh's job — norcfg-gated code inside a linkable file,which compiles out and can also only be MISSED. Said in the finding text, the baseline header and the
code.
The 15 baselined pairs
Nine are blocked on #1234; one (
dict-trainer) is a bin target with no tests, reachable by no bindinganywhere; two (REQ-FUN-05, REQ-FUN-11) are ordinary binding-placement debt and the cheapest to pay
off.
unwiredrequirements are in scope on purpose, so REQ-CTL-04's three files do not all fail onthe day #1234 lands and it flips to
enforced.Test results
scripts/gate.shon the branch tip, clean tree:All 12 steps
ok. held-out (runtime, #1274): notch_rescues_interferer, ota_channel_adaptation — run scripts/slow-tests.shscripts/trace.sh --self-test→ SELF-TEST: PASS, 26okprobes.python3 scripts/lib/trace.py check→TRACE: PASS — 15 grandfathered unlinkable scope pair(s) (0 new, 0 stale).Correction
I told the maintainer #1403 would change these ratios via function-level
code:scoping, and thatthis argued for waiting. #1403 is closed and never proposed that — I asserted it from the issue body
without checking. Dropped from the plan; linkability is a package property regardless.
Verification-objective: a requirement's mutation verdict must not be silently bounded by its own
scope; scope code no bound test can link is reported and ratcheted
Review: docs/dev/reviews/review-1405-link-ratchet.md
🤖 Generated with Claude Code
https://claude.ai/code/session_0188ATCj6DZ9aRVQ2vSirua6