fix(nebula): pin the interpreter fix tier-independently, and stop reporting an exec fault as a config fault

[ZacxDev]

Closes the two 🟡 findings from #1407's round-1 audit.

Finding 1 — the fix shipped without a regression guard, and its only coverage was on the tier that is blind to a revert

MEASURED: revert #1407's run_check to a direct "$CHECK" exec and the dev-host tier still reports 30 passed — the dev host has /usr/bin/env. Only nix build .#checks.x86_64-linux.pytests goes red. So scripts/gate.sh --tier both, the command most sessions actually run, went fully green on the exact defect #1407 existed to remove. An agent "simplifying" run_check back to what #1272 wrote would have seen green and shipped it.

Two source-level guards, which fail wherever the suite runs:

• test_the_verifier_is_never_execed_via_its_own_shebang — pinned as a pair: the helper must exist and no direct call site may reappear. Presence alone passes if someone adds a second, direct site beside the helper; absence alone passes if the helper is deleted outright.
• test_the_verifiers_exit_codes_are_a_closed_set — pins finding 2's load-bearing precondition on both sides, so the verifier growing an exit 3 fails loudly instead of being silently reclassified as an exec fault.

Finding 2 — the misdiagnosing abort was unchanged

check-nebula-relays.sh only ever exits 0, 1 or 2. Every other code fell into the same *) arm as a genuine rc 2:

ABORT: the verifier could not read the current config (rc=126); fix that first

— asserting a fact about $CFG that is false in every one of those cases, and precisely how the /usr/bin/env fault presented, which is why it read as a config problem rather than an exec one.

Split into 2) (a real refusal — message unchanged) and *) (it did not run), via one verifier_answered classifier used at both sites rather than open-coded twice.

🔴 The consequential site is the post-rebuild verify, not the preflight. There a non-verifier rc reaches die after nixos-rebuild test has activated, so the EXIT trap rolls back a change that worked and prints the PERSISTED paragraphs for what was really a signal or exec fault.

Verification

Both tiers, merged tree (branch is off origin/main @ 0477e52a, fast-forward):

tier	result
nix build …#checks…pytests	PASS — collected=21135 passed=21133 failed=0
nix build …#checks…nodetests	PASS — tests=1449 pass=1449 fail=0

collected rose 21127 → 21135, exactly +8 = the test cases added. Nothing silently skipped.

Mutation battery (scripts/tests/mutants-nebula-relay.sh), clean /tmp — pass=26 fail=0, all three controls green, four new mutants each killed by the named test with its own message:

mutant	killed by
M-FH-1 revert $BASH → direct exec	the shebang guard
M-FH-2 verifier grows exit 3	the closed-set test
M-FH-3 widen verifier_answered to accept 126	post-rebuild behavioural
M-FH-4 drop site-B classification	post-rebuild behavioural

Two things the work itself caught, recorded because they generalise

🔴 The repo's battery caught a defect my own ad-hoc sweep scored as PASSING. M-FH-1 died to the right test but not to the right assertion — I had reworded the message after writing the mutant entry. My sweep only checked which test failed; the battery also checks the test's own assertion fired. That is the exact "green for the wrong reason" hole its header documents, hit while writing a guard against a different instance of the same class. The phrase is now asserted present in the test source before being written into the battery.

🔴 The post-rebuild test exists because the sweep found a hole in my tests. Widening verifier_answered was first caught by the structural test alone: every behavioural case aborted at the preflight, so verifier_answered — used only at the post-rebuild site — never executed. A guard that is never reached is not a guard. apply_beside_sequenced_verifier (answer 1, then fail) is the only way to reach that site.

⚠ The stub verifier goes through write_exec, not a hand-written shebang. A #!/usr/bin/env bash stub execs on the dev host and ENOENTs in the sandbox — the test would have fabricated rc 126 from its own stub rather than from the code under test: circular, and green for the wrong reason on one tier.

Also corrected — audit finding 5, docstring only

test_apply_declares_every_tool_it_execs claimed the relationship its name implies while its body inspects one side (a hard-coded list ⊆ declared). MEASURED: head, id and rm are exec'd and undeclared right now, and it is green. ($BASH is a fourth omission but a correct one — the interpreter is already running.)

Body unchanged. Closing it means deriving the exec'd set from the source, which is real work and out of scope here. The docstring now says what the body does rather than what the name suggests, because a guard that reads as coverage while providing none is worse than none — it stops anyone looking. Recorded as open, not fixed.

Not a finding, recorded for the next reader

An earlier full battery run reported CONTROL-KILL as WRONG-KILLER. Not a defect in this branch — it passes in isolation on origin/main and here. A prior run was killed at a 10-minute tool cap before its cleanup trap fired, leaving 29 /tmp/nebula-relay-pre.* files, the residue class the battery's own header documents as making later runs read stale artifacts as live defects. Clearing them made the full run green.

[ZacxDev]

Round 2 complete — and one thing NOT certified

Round 2 confirmed all three round-1 findings genuinely fixed (the auditor re-derived each independently rather than trusting the diff). It returned one SHOULD-FIX and four NITs; all are addressed in 9815a3cc.

The SHOULD-FIX was a docstring my own fix falsified. test_the_verifier_is_never_execed_via_its_own_shebang still claimed "only the sandbox tier goes red" — the behavioural guard I added made the dev-host tier catch that revert too (re-measured: 2 failed, 37 passed). That sentence was the whole justification for keeping the structural half, and it was written in the commit whose subject was "stop a guard's docstring overclaiming."

One NIT was a real payload hole: ${PATCHED:-0} read an inherited environment variable, because the trap flags were declared below the preflight. This script's header documents sudo env "PATH=$PATH" bash …, which preserves the caller's env — so an exported PATCHED=1 made the preflight claim work had been done when none had. Reproduced, fixed by moving the declarations above their first read, regression test watched red on the pre-fix shape.

Two harness defects found and fixed along the way

1. The battery READ a stale bytecode cache. Its header claims PYTHONDONTWRITEBYTECODE=1 "throughout" — that stops it writing one, not reading one an ordinary pytest run left behind. An edit adding assertion messages was invisible for two consecutive runs, both reporting WRONG-KILLER while the phrase was demonstrably in the same command's output run by hand. run_tests now purges __pycache__; verified by deliberately recreating the stale cache.
2. Several assertions passed combined as their own message, so under a mutant the expected phrase appeared nowhere and the battery scored a correct kill as WRONG-KILLER.

🔴 Not certified — read before merging

I have not obtained a single full-green battery run. Three consecutive full runs failed a different pair each time, always exactly two:

run	failed
1	M-FA-3, M-FH-4
2	M-FH-4, M-FH-6
3	M-FI-2, M-FH-1 — reported SURVIVED

Every one of those passes repeatedly in isolation, including M-FH-1, whose mutation I measured by hand as failing two tests — so run 3's SURVIVED was a false SURVIVED, the dangerous direction.

Box load was 64–90 throughout, from other sessions. Conclusion: the per-mutant results are trustworthy; the full-run aggregate on a loaded box is not. Two earlier fail=0 full runs in this PR's history should be read as luck at lower load, not as certification — including one I quoted as evidence.

The gate itself is unaffected: both sandbox tiers were green at 12df41d0 and the changes since are the round-2 fixes above.

Recorded as open, not fixed

• No test reaches the post-switch verify (site 3); apply_beside_sequenced_verifier is two-stage by construction. Round 2 checked the reasoning that site 2 covers the same branch and found it holds — but that is reasoning, not a run.
• test_apply_declares_every_tool_it_execs still cannot catch an undeclared tool; head/id/rm remain exec'd-and-undeclared. Docstring corrected, body unchanged.
• Round-1 NITs untouched: closed-set regex spellings, rc-2 ambiguity, the unquoted counter path, and apply-tailscale.sh solving the same hazard with bash "$CHECK" and no equivalent guard.