Skip to content

test(security): kill 21 of the 22 surviving mutants in content_labels - #78

Merged
acamarata merged 1 commit into
mainfrom
test/content-labels-coverage
Sep 15, 2026
Merged

acamarata merged 1 commit into
mainfrom
test/content-labels-coverage

Conversation

@acamarata

Copy link
Copy Markdown
Contributor

First suite in this sweep written against the gate's actual surviving-mutant list rather than a reading of the code. The 22 mutant names came from the 12 shard artifacts of Mutation Testing run 34917215492, so every test targets a mutant known to be alive.

src/security/content_labels.rs is the prompt-injection boundary — it decides which sources are untrusted, scores content risk, and strips injected instructions before they reach the model. It stood at 22 missed / 12 caught.

15 tests added; the 4 pre-existing ones move verbatim into the same file, which also brings content_labels.rs from 284 to 247 lines, back under the 300-line guidance.

What each mutant needed, and why an obvious test misses it

Mutant Why a naive test fails to kill it
as_str -> "" / "xyzzy" (both enums) asserting one variant kills only that one — every variant is asserted by exact literal
parse: 7 × delete match arm a deleted arm falls through to _ => File, so each arm needs its own assertion (plus a round-trip through as_str)
line 148 <<=, == needs a High hit with a medium pattern present, proving the medium scan is skipped
line 148 <> needs the opposite case: a medium pattern on a trusted source must still escalate Low → Medium
line 170 <> needs an analysis marked Medium over content that does contain a high-risk phrase — only then do the branches diverge
line 200 idx + i + 1 killed only by asserting the exact sanitized string; contains("[SANITIZED]") passes for all variants
record_content_label -> Ok("") asserted to write a row under the id it returns, so a constant cannot pass

Verification — 12 of 13 killed, and the 13th is supposed to survive

Mutant Result
SourceType::as_str -> "" KILLED
parse: delete arm git_log KILLED
parse: delete arm user_input KILLED
RiskLevel::as_str -> "" KILLED
L148 <<= KILLED
L148 <> KILLED
L152 <== KILLED
L152 <> KILLED
L152 <<= SURVIVED — equivalent, see below
L170 <> KILLED
L200 + 1- 1 KILLED
L200 idx +idx * KILLED
record_content_label -> Ok("") KILLED

The survivor is the correct result, not a gap. The guarded statement is risk_level = RiskLevel::Medium. When risk_level is already Medium, <= makes the assignment run and assign Medium over Medium — nothing is observable, by any test. It is an equivalent mutant.

I predicted that before running it and then verified the prediction rather than asserting it. The file records the reasoning so nobody burns time trying to kill it, and so the eventual gate number is read correctly: this file's ceiling is 21/22, not 22/22.

cargo test --lib security::content_labels passes (19); clippy --all-targets --all-features -- -D warnings and cargo fmt --check both exit 0.

First suite in this sweep written against the gate's ACTUAL surviving-mutant
list rather than a reading of the code. The 22 names came from the 12 shard
artifacts of Mutation Testing run 34917215492, so each test targets a mutant
that is known to be alive rather than one I guessed at.

This file is the prompt-injection boundary: it decides which sources are
untrusted, scores content risk, and strips injected instructions before they
reach the model. It had 22 missed / 12 caught.

Added 15 tests (the 4 pre-existing ones are moved verbatim into the same
file, which also brings content_labels.rs from 284 to 247 lines, back under
the 300-line guidance).

What the mutants needed, and why obvious tests miss them:

  - as_str on both enums: mutants return "" or "xyzzy". A test asserting one
    variant kills only that one, so every variant is asserted by exact literal.
  - SourceType::parse: 7 "delete match arm" mutants. A deleted arm falls
    through to the _ => File default, so each arm gets its own assertion plus
    a round-trip through as_str.
  - analyze_content line 148 needs BOTH directions: a High-risk hit must stop
    the medium scan (kills <= and ==), and a medium pattern on a trusted
    source must still escalate Low -> Medium (kills >).
  - sanitize_content line 170 needs an analysis marked Medium over content
    that DOES contain a high-risk phrase - only then do < and > diverge.
  - line 200's end-offset arithmetic is killed by asserting the EXACT
    sanitized string; a contains("[SANITIZED]") check passes for all variants.
  - record_content_label is asserted to write a row under the id it returns,
    so a constant-returning mutant cannot pass.

Verified by applying 13 representative mutants: 12 KILLED.

The 13th, line 152 "<" -> "<=", SURVIVED - and that is the correct result,
not a gap. The guarded statement is risk_level = RiskLevel::Medium, so when
risk_level is already Medium the mutant assigns Medium over Medium and
nothing is observable. It is an EQUIVALENT mutant. I predicted this before
running it and verified the prediction rather than asserting it; the file
records it so nobody spends time trying to kill it.

cargo test --lib security::content_labels passes (19); clippy --all-targets
--all-features -D warnings and cargo fmt --check both exit 0.
@acamarata
acamarata merged commit aa4f7ce into main Sep 15, 2026
12 checks passed
@acamarata
acamarata deleted the test/content-labels-coverage branch September 15, 2026 23:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant