Skip to content

feat(trace): ratchet on scope code no bound test's package can link (#1405) - #1420

Merged
dc0sk merged 1 commit into
mainfrom
fix/1405-link-ratchet
Sep 19, 2026
Merged

dc0sk merged 1 commit into
mainfrom
fix/1405-link-ratchet

Conversation

@dc0sk

@dc0sk dc0sk commented Sep 19, 2026

Copy link
Copy Markdown
Owner

Closes #1405.

req-mutation.sh mutates every file in a requirement's scope and runs its bound tests under
cargo-mutants --test-package. A test binary cannot link code outside its own package closure, so
a mutant in an unlinkable package is unkillable by construction and can only ever be recorded MISSED.
The script's verdict is killed > 0, so REQ-FUN-11 passed on the strength of one linkable file
while 401 of its 408 mutants could never die
— a per-requirement PASS that was partly a statement
about the ownership map rather than about the tests.

Why a ratchet, and not the issue's fail-or-warn

Both options the issue framed are wrong here. Failing is red-on-arrival (#1074): three of the six
affected requirements are blocked on #1234, where nothing depends on openpulse-keystore, and no
binding placement can fix that. Warning repeats DANGLING-CODE, which correctly detects CAP-70's
dead path and has never failed a build.

A grandfathered ratchet is this repo's existing third mode (reachability.sh,
NOT-GRANDFATHERED): NEW unlinkable code fails, the 15 baselined pairs warn, and
baseline − current also fails — because #1371 found 74 of 86 orphan-baseline entries stale in
one pass under a header that merely asked for pruning.

Semantics verified against the tool, not inferred

This was the part most likely to sink the design, and I had guessed it. Fable read cargo-mutants
27.1.0: lab.rs selects TestsForMutant::Explicit(packages) — the --test-package list only, the
mutated package is not added — and cargo.rs issues cargo test --package=<p> --no-default-features. So linkable = P + P's normal/dev/build deps, then normal/build transitively.

Deliberately not _workspace_graph(), which filters dev and optional edges because its question
is production reach. A test target does link its own package's dev-dependencies.

Excluding optional edges turned out to be right for the wrong reason — the real rule is feature
resolution rooted at P under --no-default-features — so it is now a checked precondition, not a
constant.

Sabotage-verified, every direction

scope file no bound test's package can link   -> UNLINKABLE-FROM-BINDING
baseline entry that is no longer unlinkable   -> STALE-LINK-BASELINE
optional-edge precondition (planted features=["gpu"] on openpulse-cli -> bpsk-plugin)
    -> GRAPH-SELF-TEST FAIL: openpulse-cli -> bpsk-plugin activates optional openpulse-gpu

Each fails its own case, and the unmodified-tree positive control still passes — which is what proves
the plants were restored rather than the probes being loud.

Stated blindnesses

Linkability is necessary, not sufficient (#1415), and the name says "link" because
reachability.sh already uses "reachable" for production reach. It cannot see a vacuous binding in
the right package — that is req-mutation.sh's job — nor cfg-gated code inside a linkable file,
which compiles out and can also only be MISSED. Said in the finding text, the baseline header and the
code.

The 15 baselined pairs

Nine are blocked on #1234; one (dict-trainer) is a bin target with no tests, reachable by no binding
anywhere; two (REQ-FUN-05, REQ-FUN-11) are ordinary binding-placement debt and the cheapest to pay
off. unwired requirements are in scope on purpose, so REQ-CTL-04's three files do not all fail on
the day #1234 lands and it flips to enforced.

Test results

scripts/gate.sh on the branch tip, clean tree:

suites=339 tests_passed=2574 tests_failed=0
GATE: PASS c07d92d3021f39b139909790b34d43f5bbbd3335 clean 20260919T195637Z

All 12 steps ok. held-out (runtime, #1274): notch_rescues_interferer, ota_channel_adaptation — run scripts/slow-tests.sh

scripts/trace.sh --self-test → SELF-TEST: PASS, 26 ok probes.
python3 scripts/lib/trace.py check → TRACE: PASS — 15 grandfathered unlinkable scope pair(s) (0 new, 0 stale).

Correction

I told the maintainer #1403 would change these ratios via function-level code: scoping, and that
this argued for waiting. #1403 is closed and never proposed that — I asserted it from the issue body
without checking. Dropped from the plan; linkability is a package property regardless.

Verification-objective: a requirement's mutation verdict must not be silently bounded by its own
scope; scope code no bound test can link is reported and ratcheted

Review: docs/dev/reviews/review-1405-link-ratchet.md

🤖 Generated with Claude Code

https://claude.ai/code/session_0188ATCj6DZ9aRVQ2vSirua6

…1405)

`req-mutation.sh` mutates every file in a requirement's scope and runs its bound
tests under cargo-mutants `--test-package`. A test binary cannot link code outside
its own package closure, so a mutant in an unlinkable package is unkillable BY
CONSTRUCTION and can only be recorded MISSED. The script's verdict is `killed > 0`,
so REQ-FUN-11 passed on the strength of one linkable file while 401 of its 408
mutants could never die — a PASS that was partly about the ownership map.

A GRANDFATHERED RATCHET, which is the repo's existing third mode (reachability.sh,
NOT-GRANDFATHERED), because the issue's fail-or-warn pair are both wrong here:
failing is red-on-arrival (#1074) with three requirements blocked on #1234, and
warning repeats DANGLING-CODE, which correctly detects CAP-70's dead path and has
never failed a build. NEW unlinkable code fails; the 15 baselined pairs warn;
`baseline - current` also FAILS, because #1371 found 74 of 86 orphan-baseline
entries stale in one pass under a header that merely asked for pruning.

Semantics VERIFIED against cargo-mutants 27.1.0 rather than inferred: `lab.rs`
passes only the `--test-package` list (the mutated package is not added) and
`cargo.rs` runs `cargo test --package=<p> --no-default-features`, so linkable is
P + P's normal/dev/build deps, then normal/build transitively. Deliberately NOT
`_workspace_graph()`, which filters dev and optional edges because its question is
production reach; a test target does link its own package's dev-deps.

Excluding optional edges was right for the wrong reason — the rule is really feature
resolution rooted at P under `--no-default-features` — so it is now a CHECKED
precondition, not a constant: a graph-self-test probe fails when any internal dep
spec activates an optional internal edge.

Also: a file that is a `bin` root is linkable only from its own package (a
dependent's tests never build a dependency's bin); `unwired` is in scope so
REQ-CTL-04's three files do not all fail the day #1234 lands; findings carry
best-effort `cargo mutants --list` counts, never a silent 0 (#1279), because the
spread is 46x (186 mutants in linksec/async_channel.rs against 4 in
modem/envelope_codec.rs).

Named UNLINKABLE, not UNREACHABLE: reachability.sh already uses "reachable" for
production reach, and a second sense invites exactly the over-read this avoids.
Linkability is NECESSARY, NOT SUFFICIENT (#1415) — stated in the finding text, the
baseline header and the code. It cannot see a vacuous binding in the right package,
nor cfg-gated code inside a linkable file.

Sabotage-verified in both directions, each failing its own case, with the
unmodified-tree positive control still passing:
  scope file no bound test's package can link -> UNLINKABLE-FROM-BINDING
  baseline entry that is no longer unlinkable -> STALE-LINK-BASELINE

Verification-objective: a requirement's mutation verdict must not be silently
bounded by its own scope; scope code no bound test can link is reported and
ratcheted

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0188ATCj6DZ9aRVQ2vSirua6
@dc0sk
dc0sk merged commit bc070a0 into main Sep 19, 2026
12 checks passed
@dc0sk
dc0sk deleted the fix/1405-link-ratchet branch September 19, 2026 20:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

A requirement's mutation scope can contain mutants its bound test cannot link, so a PASS is bounded by scope rather than by test quality

1 participant