From c07d92d3021f39b139909790b34d43f5bbbd3335 Mon Sep 17 00:00:00 2001 From: Simon Keimer Date: Sat, 19 Sep 2026 21:56:19 +0200 Subject: [PATCH] feat(trace): ratchet on scope code no bound test's package can link (#1405) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `req-mutation.sh` mutates every file in a requirement's scope and runs its bound tests under cargo-mutants `--test-package`. A test binary cannot link code outside its own package closure, so a mutant in an unlinkable package is unkillable BY CONSTRUCTION and can only be recorded MISSED. The script's verdict is `killed > 0`, so REQ-FUN-11 passed on the strength of one linkable file while 401 of its 408 mutants could never die — a PASS that was partly about the ownership map. A GRANDFATHERED RATCHET, which is the repo's existing third mode (reachability.sh, NOT-GRANDFATHERED), because the issue's fail-or-warn pair are both wrong here: failing is red-on-arrival (#1074) with three requirements blocked on #1234, and warning repeats DANGLING-CODE, which correctly detects CAP-70's dead path and has never failed a build. NEW unlinkable code fails; the 15 baselined pairs warn; `baseline - current` also FAILS, because #1371 found 74 of 86 orphan-baseline entries stale in one pass under a header that merely asked for pruning. Semantics VERIFIED against cargo-mutants 27.1.0 rather than inferred: `lab.rs` passes only the `--test-package` list (the mutated package is not added) and `cargo.rs` runs `cargo test --package=

--no-default-features`, so linkable is P + P's normal/dev/build deps, then normal/build transitively. Deliberately NOT `_workspace_graph()`, which filters dev and optional edges because its question is production reach; a test target does link its own package's dev-deps. Excluding optional edges was right for the wrong reason — the rule is really feature resolution rooted at P under `--no-default-features` — so it is now a CHECKED precondition, not a constant: a graph-self-test probe fails when any internal dep spec activates an optional internal edge. Also: a file that is a `bin` root is linkable only from its own package (a dependent's tests never build a dependency's bin); `unwired` is in scope so REQ-CTL-04's three files do not all fail the day #1234 lands; findings carry best-effort `cargo mutants --list` counts, never a silent 0 (#1279), because the spread is 46x (186 mutants in linksec/async_channel.rs against 4 in modem/envelope_codec.rs). Named UNLINKABLE, not UNREACHABLE: reachability.sh already uses "reachable" for production reach, and a second sense invites exactly the over-read this avoids. Linkability is NECESSARY, NOT SUFFICIENT (#1415) — stated in the finding text, the baseline header and the code. It cannot see a vacuous binding in the right package, nor cfg-gated code inside a linkable file. Sabotage-verified in both directions, each failing its own case, with the unmodified-tree positive control still passing: scope file no bound test's package can link -> UNLINKABLE-FROM-BINDING baseline entry that is no longer unlinkable -> STALE-LINK-BASELINE Verification-objective: a requirement's mutation verdict must not be silently bounded by its own scope; scope code no bound test can link is reported and ratcheted Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_0188ATCj6DZ9aRVQ2vSirua6 --- docs/dev/project/trace-link-baseline.txt | 52 ++++ docs/dev/project/traceability.md | 58 ++++ docs/dev/reviews/review-1405-link-ratchet.md | 92 +++++++ scripts/lib/trace.py | 267 +++++++++++++++++++ scripts/trace.sh | 43 +++ 5 files changed, 512 insertions(+) create mode 100644 docs/dev/project/trace-link-baseline.txt create mode 100644 docs/dev/reviews/review-1405-link-ratchet.md diff --git a/docs/dev/project/trace-link-baseline.txt b/docs/dev/project/trace-link-baseline.txt new file mode 100644 index 00000000..5c2b45e8 --- /dev/null +++ b/docs/dev/project/trace-link-baseline.txt @@ -0,0 +1,52 @@ +# Grandfathered UNLINKABLE-FROM-BINDING pairs (#1405). Format: REQ-IDpath +# +# A requirement's bound tests run, under cargo-mutants `--test-package`, in their OWN package only. +# A test binary cannot link code outside that package's closure, so a mutant in a file listed here +# is unkillable BY CONSTRUCTION and can only ever be recorded MISSED. `req-mutation.sh` passes a +# requirement as soon as ONE mutant dies, so without this list a PASS can be a statement about the +# ownership map rather than about the tests. +# +# THIS LIST ONLY SHRINKS. A NEW pair fails the trace check; an entry here that is no longer +# unlinkable ALSO fails (STALE-LINK-BASELINE), because a baseline nobody prunes stops ratcheting — +# #1371 found 74 of 86 orphan-baseline entries stale in one pass. +# +# It is NOT a coverage list. Linkability is necessary, not sufficient (#1415): moving a binding into +# the right package makes mutants linkable, never killable. It is also blind to `cfg`-gated code +# inside a linkable file, which compiles out and can only be MISSED too. +# +# Why each group is here, and what would let it go: +# +# REQ-CTL-01 / -02 / -04 / -05 -> openpulse-keystore, openpulse-linksec +# Blocked on #1234: NOTHING depends on `openpulse-keystore`, so the daemon test bound to +# CTL-01/02 cannot link it, and no binding placement can fix that while the crate has no +# consumer (putting a `// VERIFIES:` in the keystore would trip DORMANT-ENFORCED instead). +# REQ-CTL-04 is `unwired` for exactly this reason and is listed now so that the day #1234 lands +# and it flips to `enforced`, it does not fail on arrival. +# CTL-05 carries a SECOND, separable cause: CAP-68 spans three crates and serves four +# requirements, so each inherits the union. Splitting it along the auth / key-storage seam is +# the CAP-70/JS8 treatment (#1400) and would pay down CTL-01/02/05 at once. +# +# REQ-CMP-01 -> tools/openpulse-dict-trainer/src/main.rs +# A bin target with no tests. No binding anywhere can reach it — a dependent's tests never build +# a dependency's `bin`. Payable only by giving the trainer a test of its own. +# +# REQ-FUN-05 -> crates/openpulse-modem/src/envelope_codec.rs +# REQ-FUN-11 -> crates/openpulse-daemon/src/filexfer.rs +# Ordinary binding-placement debt: a test in the owning package would pay either off. Both are +# single files against otherwise-linkable scopes (FUN-11 is 8/9 since #1406 added its filexfer +# binding), so these are the two cheapest entries on the list. +REQ-CMP-01 tools/openpulse-dict-trainer/src/main.rs +REQ-CTL-01 crates/openpulse-keystore/src/lib.rs +REQ-CTL-01 crates/openpulse-keystore/src/store.rs +REQ-CTL-02 crates/openpulse-keystore/src/lib.rs +REQ-CTL-02 crates/openpulse-keystore/src/store.rs +REQ-CTL-04 crates/openpulse-linksec/src/async_channel.rs +REQ-CTL-04 crates/openpulse-linksec/src/lib.rs +REQ-CTL-04 crates/openpulse-linksec/src/sync_channel.rs +REQ-CTL-05 crates/openpulse-keystore/src/lib.rs +REQ-CTL-05 crates/openpulse-keystore/src/store.rs +REQ-CTL-05 crates/openpulse-linksec/src/async_channel.rs +REQ-CTL-05 crates/openpulse-linksec/src/lib.rs +REQ-CTL-05 crates/openpulse-linksec/src/sync_channel.rs +REQ-FUN-05 crates/openpulse-modem/src/envelope_codec.rs +REQ-FUN-11 crates/openpulse-daemon/src/filexfer.rs diff --git a/docs/dev/project/traceability.md b/docs/dev/project/traceability.md index c3797c1f..13691859 100644 --- a/docs/dev/project/traceability.md +++ b/docs/dev/project/traceability.md @@ -15,6 +15,64 @@ and the actually-observed results per change. --- +## 2026-09-19 — a bound test that cannot LINK the code it claims; #1405 + +**Change.** `req-mutation.sh` mutates every file in a requirement's scope and runs its bound tests +under cargo-mutants `--test-package`. A test binary cannot link code outside its own package closure, +so a mutant in an unlinkable package is unkillable **by construction** and can only be recorded +MISSED. The script's verdict is `killed > 0`, so REQ-FUN-11 passed on the strength of one linkable +file while 401 of its 408 mutants could never die — a per-requirement PASS that was partly a +statement about the ownership map. + +**Design decision (maintainer chose the ratchet; reviewed by Fable, +`docs/dev/reviews/review-1405-link-ratchet.md`).** The issue framed this as fail-or-warn and both are +wrong here: failing is red-on-arrival (#1074) because three of the affected requirements are blocked +on #1234, and warning repeats `DANGLING-CODE`, which correctly detects CAP-70's dead path and has +never failed a build. A **grandfathered ratchet** is the repo's existing third mode +(`reachability.sh`, `NOT-GRANDFATHERED`): baseline today's set, fail only on growth. + +**What the review changed, all implemented.** The linkability semantics were *verified against +cargo-mutants 27.1.0's source* rather than inferred — `lab.rs` passes only the `--test-package` list +and `cargo.rs` runs `cargo test --package=

--no-default-features`, so linkable = P + P's +normal/dev/build deps then normal/build transitively. Excluding optional edges was right for the +wrong reason, and is now a **checked precondition** rather than a constant. A file that is a `bin` +root is linkable only from its own package. `unwired` requirements are in scope so REQ-CTL-04's three +files do not all fail on the day #1234 lands. `baseline − current` FAILS. Findings carry best-effort +mutant counts, because the spread is 46× (186 in `linksec/async_channel.rs` against 4 in +`modem/envelope_codec.rs`). Renamed to **UNLINKABLE**-FROM-BINDING: `reachability.sh` already uses +"reachable" for production reach. + +**A false premise of mine the review caught:** I claimed #1403 would change these ratios through +function-level `code:` scoping and that this argued for waiting. #1403 is closed and never proposed +that; I asserted it from the issue body without checking. + +**Implementation.** `scripts/lib/trace.py` — `_cargo_metadata()` (parsed once), `_link_graph()`, +`_linkable_packages()`, `_mutant_counts()`, the `UNLINKABLE-FROM-BINDING` / `STALE-LINK-BASELINE` +arms in `do_check`, and an optional-edge precondition probe in `graph-self-test`. +`docs/dev/project/trace-link-baseline.txt` — 15 grandfathered pairs across 6 requirements, with a +header saying what would pay each group off. `scripts/trace.sh` — two self-test probes. + +**Tests → results (actually run, at this branch).** + +- `scripts/trace.sh --self-test` → **SELF-TEST: PASS**, 26 `ok` probes, including the two new ones: + `scope file no bound test's package can link -> UNLINKABLE-FROM-BINDING` and `baseline entry that + is no longer unlinkable -> STALE-LINK-BASELINE`. The unmodified-tree positive control still passes, + which is what proves the plants were restored. +- Precondition probe sabotage: planting `features = ["gpu"]` on `openpulse-cli -> bpsk-plugin` gives + `GRAPH-SELF-TEST FAIL: openpulse-cli -> bpsk-plugin activates optional openpulse-gpu`; tree + restored in the same command. +- `python3 scripts/lib/trace.py check` → `TRACE: PASS`, `15 grandfathered unlinkable scope pair(s) + (0 new, 0 stale)`. +- Full gate: see the `GATE:` line on PR #1420. + +**Stated blindnesses.** Linkability is necessary, **not** sufficient (#1415): this cannot see a +vacuous binding in the right package — that is `req-mutation.sh`'s job — nor `cfg`-gated code inside +a linkable file, which compiles out and can also only be MISSED. Two of the 15 entries (REQ-FUN-05, +REQ-FUN-11) are ordinary binding-placement debt and are the cheapest to pay down; nine are blocked on +#1234; one is a bin target with no tests. + +--- + ## 2026-09-19 — the gate's `--all-targets` hid the shipped configuration; #1418 **Change.** `scripts/gate.sh` and `.cargo-husky/hooks/pre-push` each ran exactly one clippy pass, diff --git a/docs/dev/reviews/review-1405-link-ratchet.md b/docs/dev/reviews/review-1405-link-ratchet.md new file mode 100644 index 00000000..76ed18f6 --- /dev/null +++ b/docs/dev/reviews/review-1405-link-ratchet.md @@ -0,0 +1,92 @@ +--- +project: openpulsehf +doc: docs/dev/reviews/review-1405-link-ratchet.md +status: resolved +last_updated: 2026-09-19 +--- + +# Design review — the #1405 binding-linkability ratchet + +## Prompt + +Fable was asked to **falsify** a design before implementation, not confirm it. The design: a new +`trace.py` finding `UNREACHABLE-FROM-BINDING` as a **grandfathered ratchet** (baseline today's +unlinkable set, fail only on growth), after the maintainer chose that over the fail-or-warn pair the +issue framed. Seven numbered attack points were sent with the apparatus (15 measured baseline +candidates, the linkability semantics, the `_workspace_graph` contrast). The hardest, stated as such: +*"are my linkability semantics correct, and do they match cargo-mutants `--test-package`? I inferred +this and have NOT verified it — if cargo-mutants builds something else, the metric measures the wrong +thing."* It was also asked to argue the case for closing #1405 instead. + +## Verdict + +**Build, with seven changes.** All are implemented. + +1. **Semantics confirmed against the tool, not assumed.** Fable read cargo-mutants 27.1.0's source: + `lab.rs` selects `TestsForMutant::Explicit(packages)` — the `--test-package` list only, the mutated + package is *not* added — and `cargo.rs` issues `cargo test --package= + --no-default-features`. So linkable = P + P's normal/dev/build deps, then normal/build + transitively. That is the rule implemented in `_link_graph`. Verified three further ways, + including a `cargo check --tests --message-format=json` artifact set equal to the `cargo tree` set. +2. **"Exclude optional" was right for the wrong reason.** The real rule is feature resolution rooted + at P under `--no-default-features`; exclusion is its *consequence* today, holding only because no + internal dep spec names an optional internal target, plugin defaults are empty, and the daemon's + default-on `gpu` is suppressed by the flag `req-mutation.sh` passes. Now a checked claim: a new + `graph-self-test` probe fails when any internal dep spec activates an optional internal edge. + Sabotage-verified — planting `features = ["gpu"]` on `openpulse-cli -> bpsk-plugin` produces + `GRAPH-SELF-TEST FAIL: openpulse-cli -> bpsk-plugin activates optional openpulse-gpu`. +3. **Bin targets over-approximate.** A dependent's tests never build a dependency's `bin`. A file + that is a bin root is linkable only from its own package. Latent rather than live: it matters the + day CAP-55/CAP-67's `daemon/src/main.rs` goes enforced. +4. **`unwired` belongs in scope.** Excluding it parks REQ-CTL-04's three linksec files outside the + baseline, all of which would fail on the day #1234 lands and it flips to `enforced` — a ratchet + that ambushes whoever fixes the blocker. Baseline is 15 entries, not 12. +5. **`baseline − current` must FAIL, not warn** — the #1371 shape, where 74 of 86 orphan entries were + found stale in one pass because "shrink this over time" was enforced by nobody. Implemented as + `STALE-LINK-BASELINE`, naming all four causes rather than guessing between them. +6. **File-level key, mutant counts in the report.** Linkability is a *package* property, so file is + already finer than the truth. But the disparity is real — measured 186 mutants in + `linksec/async_channel.rs` against 4 in `modem/envelope_codec.rs`, 46× — so NEW findings carry a + best-effort `cargo mutants --list` count, never a silent 0 (#1279). +7. **Naming.** `reachability.sh` already uses "reachable" for *production* reach; a second sense + invites the over-read the design exists to prevent. Renamed to `UNLINKABLE-FROM-BINDING`. + +**A false premise of mine, caught here.** I claimed #1403 would change these ratios via function-level +`code:` scoping and that this argued for waiting. #1403 is **closed** (8b5896fa, "no map change") and +neither its body nor its comments ever proposed function-level scoping — I asserted it from the issue +body without checking. Dropped from the plan; linkability is a package property regardless. + +**On closing instead:** rejected, and the strongest argument is #1415's own history — the person +fixing #1405 created a fresh instance of it (REQ-PTT-04, 82/363 unlinkable) in the same week, caught +only by re-measuring by hand. That is the signature of a defect needing a mechanical check. +`req-mutation.sh` is structurally blind to it: its verdict is `killed > 0`, which is how REQ-FUN-11 +passed at 7/408. + +**Stated blindnesses**, in the finding text, the baseline header and the code: linkability is +necessary but not sufficient (#1415) — it cannot see a *vacuous* binding in the right package, which +is `req-mutation.sh`'s job — and it cannot see `cfg`-gated code inside a linkable file, which compiles +out and can also only be MISSED. + +## Consumer + +`scripts/lib/trace.py` `do_check`, which runs in `scripts/gate.sh:203` and per-PR in +`.github/workflows/traceability.yml:45`. The consumer of the *property* is `scripts/req-mutation.sh`, +whose per-requirement verdict this qualifies. + +## Prior art + +`grep -n "baseline" scripts/lib/trace.py` → the `NEW-ORPHAN` / `STALE-BASELINE` / `DEAD-BASELINE` +trio at lines ~962-985 and `docs/dev/project/{trace-orphan-baseline,reachability-baseline}.txt`; this +check mirrors that shape rather than inventing one. `_workspace_graph()` (trace.py:216) already +resolves the package graph but filters dev and optional edges for *production* reach, so it could not +be reused directly — `_link_graph()` is the test-linkability sibling, and the docstring says why. +Fable independently confirmed no existing mechanism covers this. + +## Twins + +`req-mutation.sh` is the twin consumer and is deliberately **not** changed: its blindness is a +separate defect (`killed > 0` as a verdict) tracked in the issue, and fixing both at once would make +neither attributable. The `reachability.sh` ratchet is the structural twin and was read for its +baseline discipline. Within this check, the two directions are themselves twins and both are +sabotage-verified — NEW and STALE each fail on their own planted defect, with the unmodified-tree +positive control still passing. diff --git a/scripts/lib/trace.py b/scripts/lib/trace.py index 6fb825bb..bad6e987 100644 --- a/scripts/lib/trace.py +++ b/scripts/lib/trace.py @@ -78,6 +78,7 @@ # is the intended end state: a baseline may SHRINK by hand as entries are paid down, and must never # grow back by regeneration. ORPHAN_BASELINE = ROOT / "docs/dev/project/trace-orphan-baseline.txt" +LINK_BASELINE = ROOT / "docs/dev/project/trace-link-baseline.txt" GRANDFATHERED = ROOT / "docs/dev/project/trace-grandfathered-ids.txt" # Production source roots. A file here that no capability claims is an orphan. @@ -215,6 +216,31 @@ class CargoUnavailable(Exception): """ +_CARGO_META = None + + +def _cargo_metadata(): + """`cargo metadata --no-deps`, parsed once per process. Raises CargoUnavailable.""" + global _CARGO_META + if _CARGO_META is not None: + return _CARGO_META + try: + out = subprocess.run( + ["cargo", "metadata", "--no-deps", "--format-version", "1"], + capture_output=True, text=True, cwd=str(ROOT), timeout=300, + ) + except (OSError, subprocess.SubprocessError) as e: + raise CargoUnavailable(f"could not run `cargo metadata`: {e}") + if out.returncode != 0: + raise CargoUnavailable( + f"`cargo metadata` exited {out.returncode}: {out.stderr.strip()[:400]}") + try: + _CARGO_META = json.loads(out.stdout) + except ValueError as e: + raise CargoUnavailable(f"could not parse `cargo metadata` output: {e}") + return _CARGO_META + + def _workspace_graph(): """Return `(pkg_of_dir, prod_rdeps, bin_pkgs)` from `cargo metadata`. @@ -295,6 +321,101 @@ def _optional_only_packages(): if total > 0 and optional_incoming.get(n, 0) == total} +def _link_graph(): + """Return `(normal, dev, build, bin_srcs, pkg_of_dir)` for TEST-LINKABILITY, not production reach. + + Deliberately NOT `_workspace_graph()`, which filters dev and optional edges because its question + is "does production reach this". The question here is the opposite one — "can a test binary in + package P link this file" — and a test target DOES link its own package's dev-dependencies. + Verified against cargo-mutants 27.1.0 rather than assumed: per mutant `lab.rs` selects + `TestsForMutant::Explicit(packages)` (the `--test-package` list only; the mutated package is NOT + added) and `cargo.rs` issues `cargo test --package= --no-default-features`. So the + linkable set is exactly what that command compiles: P, plus P's normal/dev/build dependencies, + then normal/build transitively — a dependency's dev-deps do not link in. + + PRECONDITION on optional edges, checked by `graph-self-test` rather than trusted. Optional + internal edges are excluded, which is correct only while no internal dep spec activates one. + That holds today for three separate reasons — no internal dep spec names an optional internal + target in `features = [...]`, every plugin's `default` feature set is empty, and the one + default-on internal optional (`openpulse-daemon`'s `gpu`) is switched off at the root by the + `--no-default-features` that `req-mutation.sh` passes. Any one of those changing silently makes + this under-report. The rule is really "feature resolution rooted at P under + `--no-default-features`"; exclusion is its consequence today, not the rule itself. + """ + meta = _cargo_metadata() + pkgs = meta.get("packages", []) + if not pkgs: + raise CargoUnavailable("`cargo metadata` reported no packages") + names = {p["name"] for p in pkgs} + pkg_of_dir = {os.path.dirname(p["manifest_path"]): p["name"] for p in pkgs} + normal, dev, build = {}, {}, {} + bin_srcs = set() + for pkg in pkgs: + for t in pkg.get("targets", []): + if "bin" in t.get("kind", []) and t.get("src_path"): + bin_srcs.add(os.path.realpath(t["src_path"])) + for dep in pkg.get("dependencies", []): + if dep["name"] not in names or dep.get("optional"): + continue + kind = dep.get("kind") + tgt = {None: normal, "dev": dev, "build": build}.get(kind) + if tgt is not None: + tgt.setdefault(pkg["name"], set()).add(dep["name"]) + return normal, dev, build, bin_srcs, pkg_of_dir + + +def _mutant_counts(paths): + """`cargo mutants --list` counts per file, best-effort. Never a silent 0 (#1279). + + Only called for NEW findings, so the usual cost is nothing. `--list` returns before any build or + tree copy, and the counts are syn-level UPPER bounds: they include `cfg`-gated mutants that the + gate's `--no-default-features` build compiles out and which therefore could not be killed even + inside a linkable file. + """ + if not paths: + return {} + try: + ver = subprocess.run(["cargo", "mutants", "--version"], + capture_output=True, text=True, cwd=str(ROOT), timeout=60) + if ver.returncode != 0: + return {"__why__": "cargo-mutants not installed"} + except (OSError, subprocess.SubprocessError) as e: + return {"__why__": f"cargo-mutants not runnable: {e}"} + args = ["cargo", "mutants", "--list"] + for f in paths: + args += ["--file", f] + try: + out = subprocess.run(args, capture_output=True, text=True, cwd=str(ROOT), timeout=600) + except (OSError, subprocess.SubprocessError) as e: + return {"__why__": f"`cargo mutants --list` failed: {e}"} + if out.returncode != 0: + return {"__why__": f"`cargo mutants --list` exited {out.returncode}"} + counts = {} + for line in out.stdout.splitlines(): + head = line.split(":", 1)[0].strip() + if head: + counts[head] = counts.get(head, 0) + 1 + # A file we asked about and got nothing for is 0 mutants, which is a real answer, not an absence. + for f in paths: + counts.setdefault(f, 0) + return counts + + +def _linkable_packages(test_pkgs, normal, dev, build): + """Packages a test binary in any of `test_pkgs` can link. See `_link_graph` for the semantics.""" + seed = set(test_pkgs) + for p in test_pkgs: + seed |= normal.get(p, set()) | dev.get(p, set()) | build.get(p, set()) + seen, stack = set(), list(seed) + while stack: + n = stack.pop() + if n in seen: + continue + seen.add(n) + stack.extend((normal.get(n, set()) | build.get(n, set())) - seen) + return seen + + def _package_of(rel, pkg_of_dir): """The package owning source file `rel`, by longest manifest-directory prefix.""" full = os.path.abspath(str(ROOT / rel)) @@ -654,6 +775,13 @@ def do_check(release=False): if ORPHAN_BASELINE.exists(): baseline_orphans = {l.strip() for l in ORPHAN_BASELINE.read_text().splitlines() if l.strip() and not l.startswith("#")} + baseline_links = set() + if LINK_BASELINE.exists(): + for l in LINK_BASELINE.read_text().splitlines(): + l = l.strip() + if l and not l.startswith("#") and "\t" in l: + rid, path = l.split("\t", 1) + baseline_links.add((rid.strip(), path.strip())) binds = _scan_verifies() # The grandfathered set. `traceability: baseline` is legal ONLY for an id in this file, which @@ -952,6 +1080,85 @@ def bad_shape(tok): if rid not in reqs: fails.append(f"binding: DANGLING-BINDING — `// VERIFIES: {rid}` names a requirement not in requirements.yaml") + # ---- UNLINKABLE-FROM-BINDING (#1405): can the bound test BINARY link the code it claims? ---- + # + # `req-mutation.sh` mutates every file in a requirement's scope and runs that requirement's bound + # tests via cargo-mutants `--test-package`. A test binary cannot link code outside its own + # package closure, so a mutant in an unlinkable package is unkillable BY CONSTRUCTION and can + # only be recorded MISSED. The script's verdict is `killed > 0`, so REQ-FUN-11 passed on the + # strength of one reachable file while 401 of its 408 mutants could never die. That makes a + # per-requirement mutation PASS partly a statement about the ownership map. + # + # NOT a coverage metric, and the name says "link" deliberately: `reachability.sh` already uses + # "reachable" for PRODUCTION reach, and #1415 established that linkability is necessary but NOT + # sufficient — a binding placed in the right package makes mutants linkable, never killable. Two + # blindnesses follow and are stated rather than hidden: this cannot see a VACUOUS binding in the + # right package (that is `req-mutation.sh`'s job), and it cannot see `cfg`-gated code inside a + # linkable file, which compiles out and can also only be MISSED. + # + # A RATCHET, not a gate: three of the baselined requirements are blocked on #1234 (nothing + # depends on `openpulse-keystore`), and a check that fails on conditions its owner cannot resolve + # is the red-on-arrival shape (#1074). NEW unlinkable code fails; the baseline warns. + link_new, link_stale, link_note = [], [], None + try: + _normal, _dev, _build, _bin_srcs, _pkgdir = _link_graph() + except CargoUnavailable as e: + link_note = str(e) + else: + current_links = set() + for rid, r in reqs.items(): + # `unwired` is in scope alongside `enforced` on purpose: REQ-CTL-04 is unwired precisely + # BECAUSE its package has no consumer, so its three linksec files are unlinkable today. + # Excluding it would park three entries outside the baseline that all fail on the day + # #1234 lands and it flips to enforced — a ratchet that ambushes the person who fixes + # the blocker. + if (r or {}).get("traceability") not in ("enforced", "unwired"): + continue + files = set() + for cid in (r or {}).get("covered_by", []): + for c in caps.get(cid, {}).get("code", []): + files.update(_matches(c)) + if not files: + continue + test_pkgs = {pkg for b in binds.get(rid, []) + if (pkg := _package_of(b["file"], _pkgdir))} + if not test_pkgs: + continue # a requirement with no binding at all is MISSING-BINDING's business + linkable = _linkable_packages(test_pkgs, _normal, _dev, _build) + for f in sorted(files): + owner = _package_of(f, _pkgdir) + if owner is None: + continue + # A dependent's tests never build a dependency's `bin` target, so a file that IS a + # bin root is linkable only from its OWN package. Without this, `daemon/src/main.rs` + # (claimed by CAP-55 and CAP-67) would read as linkable from any daemon-dependent + # binding the day either capability's requirement goes enforced. + is_bin = os.path.realpath(str(ROOT / f)) in _bin_srcs + ok = owner in linkable and (not is_bin or owner in test_pkgs) + if not ok: + current_links.add((rid, f)) + _counts = _mutant_counts(sorted({f for _r, f in current_links - baseline_links})) + for rid, f in sorted(current_links - baseline_links): + n = _counts.get(f) + howmany = (f" ({n} mutants)" if isinstance(n, int) + else f" (mutant count unavailable: {_counts.get('__why__', 'unknown')})") + fails.append(f"{rid}: UNLINKABLE-FROM-BINDING{howmany} — `{f}` is in this requirement's scope but " + f"no bound test's package can link it, so every mutant in it is unkillable " + f"by construction; move the binding into a package that reaches it, narrow " + f"the capability, or add the pair to {LINK_BASELINE.name} with a reason") + # `baseline - current` FAILS rather than warns: #1371 found 74 of 86 orphan-baseline entries + # stale in one pass because "shrink this over time" was enforced by nobody. Four distinct + # causes collapse into this one condition, so the message must not guess between them. + for rid, f in sorted(baseline_links - current_links): + link_stale.append((rid, f)) + for rid, f in link_stale: + why = ("the requirement is no longer enforced/unwired, the file left its `code:` scope, " + "the file is gone, or a bound test's package now links it") + fails.append(f"{rid}: STALE-LINK-BASELINE — `{f}` is listed in {LINK_BASELINE.name} but is " + f"not unlinkable now ({why}); remove the entry, or the ratchet stops " + f"ratcheting for it") + link_new = sorted(current_links - baseline_links) + # code orphans: production files no capability claims. Baseline grandfathered; NEW ones fail. claimed = _claimed_files(caps) orphans = sorted(f for f in src if f not in claimed) @@ -976,6 +1183,13 @@ def bad_shape(tok): print(f"trace check: {len(reqs)} requirements, {len(caps)} capabilities, " f"{len(binds)} in-code bindings, {len(orphans)} code orphans " f"({len(new_orphans)} new)") + # Print the link ratchet even when it passes: a baseline nobody sees is a baseline nobody prunes. + if link_note: + print(f" NOTE: link ratchet not run — {link_note}") + else: + print(f"trace check: {len(baseline_links)} grandfathered unlinkable scope pair(s) " + f"({len(link_new)} new, {len(link_stale)} stale) — linkability is NECESSARY, not " + f"sufficient (#1415), and is blind to cfg-gated code inside a linkable file") if passed is None: # Say WHICH absence this is. The old text claimed "no gate run found" while several logs sat # in target/, so a reader could take it as "nothing to see" when the real cause was that @@ -1097,6 +1311,59 @@ def bad(msg): ok(f"{len(optional_only)} optional-only package(s) are dormant " f"(e.g. {sorted(optional_only)[0]})") + # #1405 precondition. `_link_graph` EXCLUDES optional internal edges, which is right only while + # nothing activates one. Cargo activates an optional dep when a requested feature names it, and a + # dep spec requests the target's `default` feature set unless it says `default-features = false`. + # So the checkable claim is: no internal dep spec activates an optional INTERNAL dependency. If + # that ever breaks, the link join under-reports — silently, and in the flattering direction. + try: + _meta = _cargo_metadata() + except CargoUnavailable as e: + print(f" GRAPH-SELF-TEST SKIP: {e}") + else: + _pkgs = {p["name"]: p for p in _meta.get("packages", [])} + def _activated(pkgname, feats): + """Optional deps of `pkgname` switched on by feature set `feats`, transitively.""" + pkg = _pkgs.get(pkgname) + if not pkg: + return set() + table, on, seen = pkg.get("features", {}), set(), set() + stack = list(feats) + while stack: + f = stack.pop() + if f in seen: + continue + seen.add(f) + if f.startswith("dep:"): + on.add(f[4:]); continue + if f in table: + stack.extend(table[f]) + else: + on.add(f) # a bare feature name matching an optional dep enables it + return on + offenders = [] + for pkg in _meta.get("packages", []): + for dep in pkg.get("dependencies", []): + if dep["name"] not in _pkgs: + continue + req = list(dep.get("features") or []) + if dep.get("uses_default_features", True): + req.append("default") + for act in _activated(dep["name"], req): + tgt = _pkgs.get(dep["name"], {}) + for d2 in tgt.get("dependencies", []): + if d2.get("optional") and d2["name"] in _pkgs and d2["name"] == act: + offenders.append(f"{pkg['name']} -> {dep['name']} activates optional {act}") + if offenders: + print(" GRAPH-SELF-TEST FAIL: an internal dep spec activates an optional internal edge, " + "so _link_graph's exclusion under-reports:") + for o in sorted(set(offenders)): + print(f" {o}") + rc = 1 + else: + print(" ok: no internal dep spec activates an optional internal edge " + "(_link_graph's exclusion precondition holds)") + print("GRAPH-SELF-TEST: " + ("PASS" if rc == 0 else "FAIL")) return rc diff --git a/scripts/trace.sh b/scripts/trace.sh index 04c02007..bb77a593 100755 --- a/scripts/trace.sh +++ b/scripts/trace.sh @@ -232,6 +232,49 @@ plant_binding_and_expect "unwired binding in a package that reaches a binary" "U # POSITIVE CONTROL. Without this, every assertion above is satisfied by a checker that fails on # everything — including the empty-yaml refusal, which also exits non-zero. +# --- #1405 UNLINKABLE-FROM-BINDING ratchet probes ----------------------------------------------- +# +# Both directions, because a one-sided ratchet is half a ratchet. The NEW arm proves the check can +# fire at all; the STALE arm proves the baseline is pruned, which is the half #1371 found rotted (74 +# of 86 orphan entries stale in one pass, enforced by nobody). +# +# The NEW probe adds an unlinkable file to a requirement whose scope is otherwise fully linkable, so +# a hit is attributable to the plant rather than to pre-existing debt — REQ-FUN-10 is 5/5 since +# #1406 moved its binding into the daemon. `openpulse-keystore` is the plant because nothing depends +# on it (#1234), which is the very condition the check tests for. +LINK_BASELINE_FILE="docs/dev/project/trace-link-baseline.txt" +link_backup="$(mktemp)"; cp "$LINK_BASELINE_FILE" "$link_backup" + +python3 - <<'PYEOF' +import io, re, yaml +p = "docs/dev/project/requirements.yaml" +lines = io.open(p, encoding="utf-8").read().split("\n") +cid = yaml.safe_load("\n".join(lines))["requirements"]["REQ-FUN-10"]["covered_by"][0] +i = next(k for k, l in enumerate(lines) if re.match(rf"^ {re.escape(cid)}:\s*$", l)) +c = next(k for k in range(i, len(lines)) if lines[k].strip() == "code:") +lines.insert(c + 1, " - crates/openpulse-keystore/src/store.rs") +io.open(p, "w", encoding="utf-8").write("\n".join(lines)) +PYEOF +python3 scripts/lib/trace.py check > "$out" 2>&1 +rc=$? +restore +if [ "$rc" -ne 0 ] && grep -q "UNLINKABLE-FROM-BINDING" "$out"; then + echo " ok: scope file no bound test's package can link -> UNLINKABLE-FROM-BINDING" +else + echo " SELF-TEST FAIL: planted unlinkable scope file was NOT caught (exit $rc)"; rc_all=1 +fi + +# STALE: a baseline entry for a pair that IS linkable must fail, or the list silently accumulates. +printf 'REQ-FUN-10\tcrates/openpulse-core/src/handshake.rs\n' >> "$LINK_BASELINE_FILE" +python3 scripts/lib/trace.py check > "$out" 2>&1 +rc=$? +cp "$link_backup" "$LINK_BASELINE_FILE"; rm -f "$link_backup" +if [ "$rc" -ne 0 ] && grep -q "STALE-LINK-BASELINE" "$out"; then + echo " ok: baseline entry that is no longer unlinkable -> STALE-LINK-BASELINE" +else + echo " SELF-TEST FAIL: stale link-baseline entry was NOT caught (exit $rc)"; rc_all=1 +fi + python3 scripts/lib/trace.py check > "$out" 2>&1 rc=$? if [ "$rc" -eq 0 ]; then