Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 52 additions & 0 deletions docs/dev/project/trace-link-baseline.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
# Grandfathered UNLINKABLE-FROM-BINDING pairs (#1405). Format: REQ-ID<TAB>path
#
# A requirement's bound tests run, under cargo-mutants `--test-package`, in their OWN package only.
# A test binary cannot link code outside that package's closure, so a mutant in a file listed here
# is unkillable BY CONSTRUCTION and can only ever be recorded MISSED. `req-mutation.sh` passes a
# requirement as soon as ONE mutant dies, so without this list a PASS can be a statement about the
# ownership map rather than about the tests.
#
# THIS LIST ONLY SHRINKS. A NEW pair fails the trace check; an entry here that is no longer
# unlinkable ALSO fails (STALE-LINK-BASELINE), because a baseline nobody prunes stops ratcheting —
# #1371 found 74 of 86 orphan-baseline entries stale in one pass.
#
# It is NOT a coverage list. Linkability is necessary, not sufficient (#1415): moving a binding into
# the right package makes mutants linkable, never killable. It is also blind to `cfg`-gated code
# inside a linkable file, which compiles out and can only be MISSED too.
#
# Why each group is here, and what would let it go:
#
# REQ-CTL-01 / -02 / -04 / -05 -> openpulse-keystore, openpulse-linksec
# Blocked on #1234: NOTHING depends on `openpulse-keystore`, so the daemon test bound to
# CTL-01/02 cannot link it, and no binding placement can fix that while the crate has no
# consumer (putting a `// VERIFIES:` in the keystore would trip DORMANT-ENFORCED instead).
# REQ-CTL-04 is `unwired` for exactly this reason and is listed now so that the day #1234 lands
# and it flips to `enforced`, it does not fail on arrival.
# CTL-05 carries a SECOND, separable cause: CAP-68 spans three crates and serves four
# requirements, so each inherits the union. Splitting it along the auth / key-storage seam is
# the CAP-70/JS8 treatment (#1400) and would pay down CTL-01/02/05 at once.
#
# REQ-CMP-01 -> tools/openpulse-dict-trainer/src/main.rs
# A bin target with no tests. No binding anywhere can reach it — a dependent's tests never build
# a dependency's `bin`. Payable only by giving the trainer a test of its own.
#
# REQ-FUN-05 -> crates/openpulse-modem/src/envelope_codec.rs
# REQ-FUN-11 -> crates/openpulse-daemon/src/filexfer.rs
# Ordinary binding-placement debt: a test in the owning package would pay either off. Both are
# single files against otherwise-linkable scopes (FUN-11 is 8/9 since #1406 added its filexfer
# binding), so these are the two cheapest entries on the list.
REQ-CMP-01 tools/openpulse-dict-trainer/src/main.rs
REQ-CTL-01 crates/openpulse-keystore/src/lib.rs
REQ-CTL-01 crates/openpulse-keystore/src/store.rs
REQ-CTL-02 crates/openpulse-keystore/src/lib.rs
REQ-CTL-02 crates/openpulse-keystore/src/store.rs
REQ-CTL-04 crates/openpulse-linksec/src/async_channel.rs
REQ-CTL-04 crates/openpulse-linksec/src/lib.rs
REQ-CTL-04 crates/openpulse-linksec/src/sync_channel.rs
REQ-CTL-05 crates/openpulse-keystore/src/lib.rs
REQ-CTL-05 crates/openpulse-keystore/src/store.rs
REQ-CTL-05 crates/openpulse-linksec/src/async_channel.rs
REQ-CTL-05 crates/openpulse-linksec/src/lib.rs
REQ-CTL-05 crates/openpulse-linksec/src/sync_channel.rs
REQ-FUN-05 crates/openpulse-modem/src/envelope_codec.rs
REQ-FUN-11 crates/openpulse-daemon/src/filexfer.rs
58 changes: 58 additions & 0 deletions docs/dev/project/traceability.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,64 @@ and the actually-observed results per change.

---

## 2026-09-19 — a bound test that cannot LINK the code it claims; #1405

**Change.** `req-mutation.sh` mutates every file in a requirement's scope and runs its bound tests
under cargo-mutants `--test-package`. A test binary cannot link code outside its own package closure,
so a mutant in an unlinkable package is unkillable **by construction** and can only be recorded
MISSED. The script's verdict is `killed > 0`, so REQ-FUN-11 passed on the strength of one linkable
file while 401 of its 408 mutants could never die — a per-requirement PASS that was partly a
statement about the ownership map.

**Design decision (maintainer chose the ratchet; reviewed by Fable,
`docs/dev/reviews/review-1405-link-ratchet.md`).** The issue framed this as fail-or-warn and both are
wrong here: failing is red-on-arrival (#1074) because three of the affected requirements are blocked
on #1234, and warning repeats `DANGLING-CODE`, which correctly detects CAP-70's dead path and has
never failed a build. A **grandfathered ratchet** is the repo's existing third mode
(`reachability.sh`, `NOT-GRANDFATHERED`): baseline today's set, fail only on growth.

**What the review changed, all implemented.** The linkability semantics were *verified against
cargo-mutants 27.1.0's source* rather than inferred — `lab.rs` passes only the `--test-package` list
and `cargo.rs` runs `cargo test --package=<p> --no-default-features`, so linkable = P + P's
normal/dev/build deps then normal/build transitively. Excluding optional edges was right for the
wrong reason, and is now a **checked precondition** rather than a constant. A file that is a `bin`
root is linkable only from its own package. `unwired` requirements are in scope so REQ-CTL-04's three
files do not all fail on the day #1234 lands. `baseline − current` FAILS. Findings carry best-effort
mutant counts, because the spread is 46× (186 in `linksec/async_channel.rs` against 4 in
`modem/envelope_codec.rs`). Renamed to **UNLINKABLE**-FROM-BINDING: `reachability.sh` already uses
"reachable" for production reach.

**A false premise of mine the review caught:** I claimed #1403 would change these ratios through
function-level `code:` scoping and that this argued for waiting. #1403 is closed and never proposed
that; I asserted it from the issue body without checking.

**Implementation.** `scripts/lib/trace.py` — `_cargo_metadata()` (parsed once), `_link_graph()`,
`_linkable_packages()`, `_mutant_counts()`, the `UNLINKABLE-FROM-BINDING` / `STALE-LINK-BASELINE`
arms in `do_check`, and an optional-edge precondition probe in `graph-self-test`.
`docs/dev/project/trace-link-baseline.txt` — 15 grandfathered pairs across 6 requirements, with a
header saying what would pay each group off. `scripts/trace.sh` — two self-test probes.

**Tests → results (actually run, at this branch).**

- `scripts/trace.sh --self-test` → **SELF-TEST: PASS**, 26 `ok` probes, including the two new ones:
`scope file no bound test's package can link -> UNLINKABLE-FROM-BINDING` and `baseline entry that
is no longer unlinkable -> STALE-LINK-BASELINE`. The unmodified-tree positive control still passes,
which is what proves the plants were restored.
- Precondition probe sabotage: planting `features = ["gpu"]` on `openpulse-cli -> bpsk-plugin` gives
`GRAPH-SELF-TEST FAIL: openpulse-cli -> bpsk-plugin activates optional openpulse-gpu`; tree
restored in the same command.
- `python3 scripts/lib/trace.py check` → `TRACE: PASS`, `15 grandfathered unlinkable scope pair(s)
(0 new, 0 stale)`.
- Full gate: see the `GATE:` line on PR #1420.

**Stated blindnesses.** Linkability is necessary, **not** sufficient (#1415): this cannot see a
vacuous binding in the right package — that is `req-mutation.sh`'s job — nor `cfg`-gated code inside
a linkable file, which compiles out and can also only be MISSED. Two of the 15 entries (REQ-FUN-05,
REQ-FUN-11) are ordinary binding-placement debt and are the cheapest to pay down; nine are blocked on
#1234; one is a bin target with no tests.

---

## 2026-09-19 — the gate's `--all-targets` hid the shipped configuration; #1418

**Change.** `scripts/gate.sh` and `.cargo-husky/hooks/pre-push` each ran exactly one clippy pass,
Expand Down
92 changes: 92 additions & 0 deletions docs/dev/reviews/review-1405-link-ratchet.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
---
project: openpulsehf
doc: docs/dev/reviews/review-1405-link-ratchet.md
status: resolved
last_updated: 2026-09-19
---

# Design review — the #1405 binding-linkability ratchet

## Prompt

Fable was asked to **falsify** a design before implementation, not confirm it. The design: a new
`trace.py` finding `UNREACHABLE-FROM-BINDING` as a **grandfathered ratchet** (baseline today's
unlinkable set, fail only on growth), after the maintainer chose that over the fail-or-warn pair the
issue framed. Seven numbered attack points were sent with the apparatus (15 measured baseline
candidates, the linkability semantics, the `_workspace_graph` contrast). The hardest, stated as such:
*"are my linkability semantics correct, and do they match cargo-mutants `--test-package`? I inferred
this and have NOT verified it — if cargo-mutants builds something else, the metric measures the wrong
thing."* It was also asked to argue the case for closing #1405 instead.

## Verdict

**Build, with seven changes.** All are implemented.

1. **Semantics confirmed against the tool, not assumed.** Fable read cargo-mutants 27.1.0's source:
`lab.rs` selects `TestsForMutant::Explicit(packages)` — the `--test-package` list only, the mutated
package is *not* added — and `cargo.rs` issues `cargo test --package=<testpkg>
--no-default-features`. So linkable = P + P's normal/dev/build deps, then normal/build
transitively. That is the rule implemented in `_link_graph`. Verified three further ways,
including a `cargo check --tests --message-format=json` artifact set equal to the `cargo tree` set.
2. **"Exclude optional" was right for the wrong reason.** The real rule is feature resolution rooted
at P under `--no-default-features`; exclusion is its *consequence* today, holding only because no
internal dep spec names an optional internal target, plugin defaults are empty, and the daemon's
default-on `gpu` is suppressed by the flag `req-mutation.sh` passes. Now a checked claim: a new
`graph-self-test` probe fails when any internal dep spec activates an optional internal edge.
Sabotage-verified — planting `features = ["gpu"]` on `openpulse-cli -> bpsk-plugin` produces
`GRAPH-SELF-TEST FAIL: openpulse-cli -> bpsk-plugin activates optional openpulse-gpu`.
3. **Bin targets over-approximate.** A dependent's tests never build a dependency's `bin`. A file
that is a bin root is linkable only from its own package. Latent rather than live: it matters the
day CAP-55/CAP-67's `daemon/src/main.rs` goes enforced.
4. **`unwired` belongs in scope.** Excluding it parks REQ-CTL-04's three linksec files outside the
baseline, all of which would fail on the day #1234 lands and it flips to `enforced` — a ratchet
that ambushes whoever fixes the blocker. Baseline is 15 entries, not 12.
5. **`baseline − current` must FAIL, not warn** — the #1371 shape, where 74 of 86 orphan entries were
found stale in one pass because "shrink this over time" was enforced by nobody. Implemented as
`STALE-LINK-BASELINE`, naming all four causes rather than guessing between them.
6. **File-level key, mutant counts in the report.** Linkability is a *package* property, so file is
already finer than the truth. But the disparity is real — measured 186 mutants in
`linksec/async_channel.rs` against 4 in `modem/envelope_codec.rs`, 46× — so NEW findings carry a
best-effort `cargo mutants --list` count, never a silent 0 (#1279).
7. **Naming.** `reachability.sh` already uses "reachable" for *production* reach; a second sense
invites the over-read the design exists to prevent. Renamed to `UNLINKABLE-FROM-BINDING`.

**A false premise of mine, caught here.** I claimed #1403 would change these ratios via function-level
`code:` scoping and that this argued for waiting. #1403 is **closed** (8b5896fa, "no map change") and
neither its body nor its comments ever proposed function-level scoping — I asserted it from the issue
body without checking. Dropped from the plan; linkability is a package property regardless.

**On closing instead:** rejected, and the strongest argument is #1415's own history — the person
fixing #1405 created a fresh instance of it (REQ-PTT-04, 82/363 unlinkable) in the same week, caught
only by re-measuring by hand. That is the signature of a defect needing a mechanical check.
`req-mutation.sh` is structurally blind to it: its verdict is `killed > 0`, which is how REQ-FUN-11
passed at 7/408.

**Stated blindnesses**, in the finding text, the baseline header and the code: linkability is
necessary but not sufficient (#1415) — it cannot see a *vacuous* binding in the right package, which
is `req-mutation.sh`'s job — and it cannot see `cfg`-gated code inside a linkable file, which compiles
out and can also only be MISSED.

## Consumer

`scripts/lib/trace.py` `do_check`, which runs in `scripts/gate.sh:203` and per-PR in
`.github/workflows/traceability.yml:45`. The consumer of the *property* is `scripts/req-mutation.sh`,
whose per-requirement verdict this qualifies.

## Prior art

`grep -n "baseline" scripts/lib/trace.py` → the `NEW-ORPHAN` / `STALE-BASELINE` / `DEAD-BASELINE`
trio at lines ~962-985 and `docs/dev/project/{trace-orphan-baseline,reachability-baseline}.txt`; this
check mirrors that shape rather than inventing one. `_workspace_graph()` (trace.py:216) already
resolves the package graph but filters dev and optional edges for *production* reach, so it could not
be reused directly — `_link_graph()` is the test-linkability sibling, and the docstring says why.
Fable independently confirmed no existing mechanism covers this.

## Twins

`req-mutation.sh` is the twin consumer and is deliberately **not** changed: its blindness is a
separate defect (`killed > 0` as a verdict) tracked in the issue, and fixing both at once would make
neither attributable. The `reachability.sh` ratchet is the structural twin and was read for its
baseline discipline. Within this check, the two directions are themselves twins and both are
sabotage-verified — NEW and STALE each fail on their own planted defect, with the unmodified-tree
positive control still passing.
Loading
Loading