Gremlins mutation testing tracker

[cplieger]

Gremlins mutation testing tracker

Auto-updated weekly by cplieger/ci/.github/workflows/weekly-gremlins.yaml.
Last update: 2026-08-10 04:03

This week: 98.2% efficacy (±0.0% across 3 runs), 81.4% mutant coverage. 1 confirmed live mutant.
Trend: ↘ -0.7% from 12-week mean (98.9%).

Rolling 12-week history

Run (UTC)	Mean efficacy	Stddev	Mutant coverage	Live mutants	Δ efficacy
2026-08-10 04:03	98.2%	±0.0%	81.4%	1	+0.0%
2026-07-27 04:17	98.2%	±0.0%	81.4%	1	+0.0%
2026-07-20 04:11	98.2%	±0.0%	81.4%	1	-1.8%
2026-07-13 02:16	100.0%	±0.0%	89.3%	0	+0.0%
2026-07-07 23:28	100.0%	±0.0%	89.5%	0	+0.0%
2026-07-06 01:31	100.0%	±0.0%	89.5%	0	+2.6%
2026-06-29 19:29	97.4%	±0.0%	89.5%	2	+0.0%
2026-06-29 00:22	97.4%	±0.0%	89.5%	2	-2.6%
2026-06-22	100.0%	±0.0%	78.8%	0	+0.0%
2026-06-22	100.0%	±0.0%	78.3%	0	+0.0%
2026-06-18	100.0%	±0.0%	78.3%	0	+0.0%
2026-06-18	100.0%	±0.0%	78.3%	0	+4.5%

Current live mutants (1, this week)

Solid gap — LIVED in all 3 runs (known coverage hole) — 1

internal/sessions/tracker.go

• L180 — CONDITIONALS_NEGATION

How to read

• Mean efficacy: % of runnable mutants killed (or timed-out, treated as caught), averaged across the N runs
• Stddev: variance across runs — high stddev (>3%) signals flaky tests
• Mutant coverage: % of mutants reached by the test suite (test depth)

The "Current live mutants" section is bucketed by how many of the N runs the
mutant LIVED in (N = attempts that week, normally 3):

• Rare flake (1/N): a mutant your tests usually KILL but occasionally let
  through. Most actionable — almost always means a flaky test that's not
  reliable. Open the section to see the file:line; expect to fix the test, not
  the production code.
• Weakly flaky (k/N for 1<k<N): tests catch this mutant inconsistently.
  Less ideal than rare-flake but still worth investigating.
• Solid gap (N/N): every run lets this mutant through. Either there's no
  test exercising the path, or the test asserts the wrong thing. Action: add a
  test or strengthen an existing assertion.

The mutation-regression label is added when this week's mean efficacy drops

5% below the rolling 12-week mean.

Free-form notes

Add anything below — won't be touched by the auto-updater.