spar has been running a self-audit and filing what it finds (pulseengine/spar
#358, #369, #382–#385, #388–#391). The taxonomy is the same one synth kept
hitting in v0.54 — gates that pass without checking — so I tested synth
against each spar finding rather than assuming we differ. Three replicate.
1. spar#388 replicates — cargo test -- <no match> exits 0
Two rivet verification artifacts cite a test-name filter that exists nowhere
under crates/:
| artifact |
cited command |
artifacts/e2e-verification.yaml:363 |
cargo test -p synth-backend -- test_meld_import_table |
artifacts/e2e-verification.yaml:387 |
cargo test -p synth-backend -- test_startup_memory_base |
Proven, not inferred:
$ cargo test -p synth-cli -- test_meld_import_table
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 56 filtered out
→ exit 0
A filter that matches nothing is indistinguishable from a passing test. Both
artifacts therefore claim verification by evidence that never runs — the same
shape as the sret_decide_differential.py gate v0.54 found printing
MISMATCH <-- BUG and exiting 0.
Fix has two parts: correct or remove those two citations, and make the class
unrepresentable — a check that every run: in artifacts/*.yaml naming a test
filter resolves to a real test, and/or --exact/--no-tests=fail where the
runner supports it. Fixing only the two instances repeats the #890 mistake of
fixing output instead of the factory.
Scope measured while checking: 29 cargo test commands are cited across the
rivet artifacts; 13 use a -- name filter; 2 of those 13 resolve to
nothing. All --test <target> citations (the other form) resolve correctly.
2. spar#369 replicates — nothing gates the MSRV
Cargo.toml:33 declares rust-version = "1.88", and CLAUDE.md repeats it as a
project constraint. No workflow job builds at 1.88 — every CI job uses
dtolnay/rust-toolchain@stable. So the real minimum supported version drifts
upward silently on any dependency bump that raises its own MSRV, with no repo
diff and no red build. The declared number is currently an aspiration, not a
tested property.
Cheap fix: one cargo check --workspace --locked job pinned to the declared
rust-version, so the claim and the artifact move together.
3. spar#358 shape — the verification-scope question
spar found its verification gate checking 20 of 161 feature artifacts. synth has
255 rivet artifacts; I have not established how many have their cited
evidence actually executed in CI, and the answer is not obviously good given
finding 1 above. This is the rivet-artifact twin of #890 (which asked the same
question of scripts/repro/*.py and found 70 undeclared). Worth measuring
before claiming the trace is closed.
What I checked and found CLEAN
- spar#383 (format check green on unformatted code —
cargo fmt --all
covering only part of the repo): synth declares 18 workspace members and
has 18 crate directories, none missing. cargo fmt --all --check genuinely
covers everything.
- spar#380 (
strict: true with no merge queue making concurrent CI dead
work): N/A — synth sets strict: false deliberately, recorded when branch
protection was configured.
- The
--test <target> form of artifact citations: 29 checked, 0 broken.
Why port these at all
The v0.54 release fixed nine defects that were in checkers rather than in the
code they check. spar independently found the same class in its own tooling.
That two sibling projects converge on it suggests it is a property of
verification-heavy repos generally: once the artifact is well-tested, the
residual defects migrate into the instruments, where they are invisible because
the instrument's whole job is to report "fine".
Credit: pulseengine/spar#388 (no-match exit 0), #369 (MSRV closure drift),
#358 (verification gate scope).
spar has been running a self-audit and filing what it finds (pulseengine/spar
#358, #369, #382–#385, #388–#391). The taxonomy is the same one synth kept
hitting in v0.54 — gates that pass without checking — so I tested synth
against each spar finding rather than assuming we differ. Three replicate.
1. spar#388 replicates —
cargo test -- <no match>exits 0Two rivet verification artifacts cite a test-name filter that exists nowhere
under
crates/:artifacts/e2e-verification.yaml:363cargo test -p synth-backend -- test_meld_import_tableartifacts/e2e-verification.yaml:387cargo test -p synth-backend -- test_startup_memory_baseProven, not inferred:
A filter that matches nothing is indistinguishable from a passing test. Both
artifacts therefore claim verification by evidence that never runs — the same
shape as the
sret_decide_differential.pygate v0.54 found printingMISMATCH <-- BUGand exiting 0.Fix has two parts: correct or remove those two citations, and make the class
unrepresentable — a check that every
run:inartifacts/*.yamlnaming a testfilter resolves to a real test, and/or
--exact/--no-tests=failwhere therunner supports it. Fixing only the two instances repeats the #890 mistake of
fixing output instead of the factory.
Scope measured while checking: 29
cargo testcommands are cited across therivet artifacts; 13 use a
--name filter; 2 of those 13 resolve tonothing. All
--test <target>citations (the other form) resolve correctly.2. spar#369 replicates — nothing gates the MSRV
Cargo.toml:33declaresrust-version = "1.88", and CLAUDE.md repeats it as aproject constraint. No workflow job builds at 1.88 — every CI job uses
dtolnay/rust-toolchain@stable. So the real minimum supported version driftsupward silently on any dependency bump that raises its own MSRV, with no repo
diff and no red build. The declared number is currently an aspiration, not a
tested property.
Cheap fix: one
cargo check --workspace --lockedjob pinned to the declaredrust-version, so the claim and the artifact move together.3. spar#358 shape — the verification-scope question
spar found its verification gate checking 20 of 161 feature artifacts. synth has
255 rivet artifacts; I have not established how many have their cited
evidence actually executed in CI, and the answer is not obviously good given
finding 1 above. This is the rivet-artifact twin of #890 (which asked the same
question of
scripts/repro/*.pyand found 70 undeclared). Worth measuringbefore claiming the trace is closed.
What I checked and found CLEAN
cargo fmt --allcovering only part of the repo): synth declares 18 workspace members and
has 18 crate directories, none missing.
cargo fmt --all --checkgenuinelycovers everything.
strict: truewith no merge queue making concurrent CI deadwork): N/A — synth sets
strict: falsedeliberately, recorded when branchprotection was configured.
--test <target>form of artifact citations: 29 checked, 0 broken.Why port these at all
The v0.54 release fixed nine defects that were in checkers rather than in the
code they check. spar independently found the same class in its own tooling.
That two sibling projects converge on it suggests it is a property of
verification-heavy repos generally: once the artifact is well-tested, the
residual defects migrate into the instruments, where they are invisible because
the instrument's whole job is to report "fine".
Credit: pulseengine/spar#388 (no-match exit 0), #369 (MSRV closure drift),
#358 (verification gate scope).