Skip to content

Make tests report an absent prerequisite as a skip - #563

Open
Ticed wants to merge 5 commits into
infernode-os:masterfrom
Ticed:test/skip-absent-prerequisites
Open

Make tests report an absent prerequisite as a skip#563
Ticed wants to merge 5 commits into
infernode-os:masterfrom
Ticed:test/skip-absent-prerequisites

Conversation

@Ticed

@Ticed Ticed commented Aug 28, 2026

Copy link
Copy Markdown

What this changes

Four tests reported something other than the truth when a prerequisite was
absent from the environment.

wallet_capability_test printed a failure line and returned without raising,
so the runner recorded it as a pass. wallet_policy_test raised fail:, so
five of its eleven cases failed only because /n/wallet was not mounted.
msg_inject_test and msg_triage_test printed a FAIL line and returned. Each
now raises skip:<reason> naming the driver script that supplies the
prerequisite, which runner.b records as a clean SKIP.

wallet9p_test is rebuilt around a factotum-backed fixture. It starts factotum
only when one is not already running, waits for the mount instead of sleeping a
fixed 1.5s, unmounts what it mounted, and drops the case for the raw signing
oracle that no longer exists. It also gains a _marker export: without one,
joiniface() conflates Wallet9pTest with Wallet9p and the module does not
load. luciuisrv_test.b and task_consistency_test.b already carry the same
workaround.

veltro_cc_alignment_test asserts the planning mandate text the prompt
actually contains.

Tests

wallet_capability_test and wallet_policy_test are in TARG, so these two run
today. Before:

--- PASS: wallet_capability (1.51s)
--- FAIL: wallet_policy (0.01s)
    Account/Setup, Budget/Uint256, Budget/Gas,
    Authorize/ApprovalFlow, Authorize/Denial

After:

--- SKIP: wallet_capability (0.00s)
--- SKIP: wallet_policy (0.00s)

The other four are not in TARG, so they are inert until they are wired. Wiring
them is a separate change that depends on #559 and #560.

@pdfinn pdfinn left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right problem and the right mechanism — runner.b:170 already catches
"fail:skip" or "skip:*" and the comment there cites INFR-312, so this is
using an established convention rather than inventing one. A test that prints
FAIL and returns without raising is recorded as a pass, which is the worst of
the three outcomes.

I measured it rather than reading it. Because the branch is based on
pre-#559/#560 master, I applied the tests/ diff onto current master to
isolate your change from the base:

master            12 passed, 1 failed, 5 skipped
master + this PR  13 passed, 0 failed, 5 skipped

So SystemTxtTodoMandate is genuinely fixed, and nothing regressed.
wallet_policy_test now raises
skip:wallet9p not mounted at /n/wallet (run tests/host/wallet9p...), which
runner.b records as SKIP instead of five false failures. Good.

(First time round I compared the branch against master directly and saw four
extra failures. That was the stale base, not your change — flagging it in case
anyone else measures it the same way and draws the same wrong conclusion.)

Three of the six tests never run

msg_inject_test, msg_triage_test and wallet9p_test are not in
tests/mkfile's TARG, so they are not built and not executed:

msg_inject_test        TARG=0  built=no
msg_triage_test        TARG=0  built=no
wallet9p_test          TARG=0  built=no

The fixes to them are correct but inert. That is the same class #559 dealt with
for 45 modules elsewhere — a source with no target is never compiled — and it
is one line each here. Worth adding in this PR, since otherwise the
wallet9p_test rework in particular (the factotum fixture, waiting on the
mount rather than sleeping 1.5s, the _marker export) is unverifiable.

If any of them is deliberately excluded, a comment in tests/mkfile saying so
would stop the next person restoring it and finding out the hard way.

Rebase needed

This branch predates #560 and still tracks 1011 .dis files. It applies
cleanly to current master, so it is only a rebase, not a rework. Note that
#559 added veltro_cc_alignment_test.dis to TARG — that is why the test runs
at all now, and why its failure was visible for you to fix.

Fix the three missing TARG entries and rebase, and this is good.

@Ticed

Ticed commented Aug 31, 2026

Copy link
Copy Markdown
Author

Rebased onto current master — zero tracked .dis remain — and wired the three
tests you found never run:

tests/mkfile TARG        +msg_inject_test +msg_triage_test +wallet9p_test
tools/dis-manifest.txt   +the three matching lines

verify-dis-build.sh passes at 967 modules. None of the three was deliberately
excluded, so a tests/mkfile comment now records that the two msg tests are
helper-driven rather than leaving the next person to rediscover it.

All three verified through the wired path rather than in isolation: runner.dis
records msg_inject and msg_triage as clean SKIPs and wallet9p as PASS.
wallet9p_test standalone passes 3+1 and exits 0 in 0.28s, which is what proves
the fixture teardown fixed the pass-then-hang — the rework you noted was
unverifiable is now verifiable.

The full-suite before/after you measured, I could not reproduce, and it is
environmental rather than anything in this change. Three causes, all
pre-existing: nondeterministic hangs in modules identical to upstream's
(msg_capability), git_integration needing GitHub, and concurrent runs from
other checkouts on this machine.

The underlying reason is worth your attention separately. git_integration_test.sh
has an environmental skip-guard, but it only fires when the dial fails. A host
firewall that prompts does not fail the connect — it blocks it, so the guard
never runs and the emulator hangs indefinitely holding its ports. That is a real
gap in the suite for any unattended runner, and I am fixing it separately rather
than smuggling it in here.

In place of the full totals: a controlled 85-module before/after, whose only
delta is the expected +2 SKIPs, no regressions.

Fuzz (PR) is red here as it is on every open PR — that is the ClusterFuzzLite
link break from #570, fixed in #578, not anything in this branch.

All three had sources in tests/ but no TARG entry, so they were never
compiled and never executed — the same never-built class the manifest
guard exists for. Add the three targets and the matching
tools/dis-manifest.txt lines so the build must produce them.

msg_inject_test and msg_triage_test raise skip:* without their
tests/inferno drivers; runner.b records that as SKIP. wallet9p_test
starts its own factotum-backed fixture and tears it down, so it runs
and exits inside the runner.
@Ticed
Ticed force-pushed the test/skip-absent-prerequisites branch from 73f7c9d to 482fb78 Compare September 2, 2026 03:15
@Ticed

Ticed commented Sep 2, 2026

Copy link
Copy Markdown
Author

Rebased onto current master (df34b02). Force-pushed 73f7c9d -> 482fb78.

git range-diff reports every commit unchanged — same content, same messages,
only the base moved. The five commits that landed underneath are #583, #582,
#566, #578 and #580; none of them touches this branch's files.

Re-verified on macOS after the rebase.

runner.dis records msg_inject and msg_triage as clean SKIPs and wallet9p
as PASS. wallet9p_test standalone passes and exits, which is what proves the
fixture teardown fixed the pass-then-hang. All three are still wired into
tests/mkfile TARG and tools/dis-manifest.txt after the rebase, and all three
still compile.

#578 fixed the ClusterFuzzLite link break, so Fuzz should now be green here
rather than red for a reason that was never this branch's.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants