test(security): kill 21 of the 22 surviving mutants in content_labels - #78
Merged
Merged
Conversation
First suite in this sweep written against the gate's ACTUAL surviving-mutant
list rather than a reading of the code. The 22 names came from the 12 shard
artifacts of Mutation Testing run 34917215492, so each test targets a mutant
that is known to be alive rather than one I guessed at.
This file is the prompt-injection boundary: it decides which sources are
untrusted, scores content risk, and strips injected instructions before they
reach the model. It had 22 missed / 12 caught.
Added 15 tests (the 4 pre-existing ones are moved verbatim into the same
file, which also brings content_labels.rs from 284 to 247 lines, back under
the 300-line guidance).
What the mutants needed, and why obvious tests miss them:
- as_str on both enums: mutants return "" or "xyzzy". A test asserting one
variant kills only that one, so every variant is asserted by exact literal.
- SourceType::parse: 7 "delete match arm" mutants. A deleted arm falls
through to the _ => File default, so each arm gets its own assertion plus
a round-trip through as_str.
- analyze_content line 148 needs BOTH directions: a High-risk hit must stop
the medium scan (kills <= and ==), and a medium pattern on a trusted
source must still escalate Low -> Medium (kills >).
- sanitize_content line 170 needs an analysis marked Medium over content
that DOES contain a high-risk phrase - only then do < and > diverge.
- line 200's end-offset arithmetic is killed by asserting the EXACT
sanitized string; a contains("[SANITIZED]") check passes for all variants.
- record_content_label is asserted to write a row under the id it returns,
so a constant-returning mutant cannot pass.
Verified by applying 13 representative mutants: 12 KILLED.
The 13th, line 152 "<" -> "<=", SURVIVED - and that is the correct result,
not a gap. The guarded statement is risk_level = RiskLevel::Medium, so when
risk_level is already Medium the mutant assigns Medium over Medium and
nothing is observable. It is an EQUIVALENT mutant. I predicted this before
running it and verified the prediction rather than asserting it; the file
records it so nobody spends time trying to kill it.
cargo test --lib security::content_labels passes (19); clippy --all-targets
--all-features -D warnings and cargo fmt --check both exit 0.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
First suite in this sweep written against the gate's actual surviving-mutant list rather than a reading of the code. The 22 mutant names came from the 12 shard artifacts of Mutation Testing run 34917215492, so every test targets a mutant known to be alive.
src/security/content_labels.rsis the prompt-injection boundary — it decides which sources are untrusted, scores content risk, and strips injected instructions before they reach the model. It stood at 22 missed / 12 caught.15 tests added; the 4 pre-existing ones move verbatim into the same file, which also brings
content_labels.rsfrom 284 to 247 lines, back under the 300-line guidance.What each mutant needed, and why an obvious test misses it
as_str -> ""/"xyzzy"(both enums)parse: 7 × delete match arm_ => File, so each arm needs its own assertion (plus a round-trip throughas_str)<→<=,==<→><→>idx + i + 1contains("[SANITIZED]")passes for all variantsrecord_content_label -> Ok("")Verification — 12 of 13 killed, and the 13th is supposed to survive
SourceType::as_str -> ""parse: delete armgit_logparse: delete armuser_inputRiskLevel::as_str -> ""<→<=<→><→==<→><→<=<→>+ 1→- 1idx +→idx *record_content_label -> Ok("")The survivor is the correct result, not a gap. The guarded statement is
risk_level = RiskLevel::Medium. Whenrisk_levelis alreadyMedium,<=makes the assignment run and assignMediumoverMedium— nothing is observable, by any test. It is an equivalent mutant.I predicted that before running it and then verified the prediction rather than asserting it. The file records the reasoning so nobody burns time trying to kill it, and so the eventual gate number is read correctly: this file's ceiling is 21/22, not 22/22.
cargo test --lib security::content_labelspasses (19);clippy --all-targets --all-features -- -D warningsandcargo fmt --checkboth exit 0.