Skip to content

Mutation testing baseline fails: INPUT_MODULE_PREFIX_STRIP leaks into module-glob test #369

Description

@leynos

Summary

The scheduled Mutation testing workflow (.github/workflows/mutation-testing-caller.yml, calling mutation-mutmut.yml against shared-actions' own workflow_scripts/) has failed on every recent run within seconds of starting the "Run mutation testing" step. This is estate machinery — the same reusable workflow that every caller repository depends on for mutation testing — so a broken baseline here blocks mutation testing of shared-actions' own scripts on every scheduled trigger.

Occurrences

  • 29097292753 — 2026-07-10
  • 29417371506 — 2026-07-15
  • 29581784412 — 2026-07-17
  • 29687426591 — 2026-07-19

All four fail at the "Run mutation testing" step with the same pytest assertion failure, roughly 25-30 seconds after the job starts, against different commits of main (confirmed different headSha per run), so this is not a one-off flake tied to a single commit.

Root cause

mutation-testing-caller.yml sets module-prefix-strip: "" (line 31) when calling mutation-mutmut.yml for shared-actions itself, because shared-actions' own workflow_scripts/ live at the repository root rather than under a src/ layout. That value flows through as the INPUT_MODULE_PREFIX_STRIP environment variable on the "Run mutation testing" step, which invokes mutmut, which in turn runs the workflow_scripts test suite as its baseline check in the same process environment.

workflow_scripts/tests/test_mutation_run_mutmut.py::TestMainEntry::test_scoped_run_passes_module_globs does not clear or override INPUT_MODULE_PREFIX_STRIP before invoking run_mutmut.app([]), so it inherits the ambient "" value instead of exercising the script's documented default ("src/"). With prefix_strip="", _module_glob_for treats the prefix as falsy and skips stripping (workflow_scripts/mutation_run_mutmut.py:104), so src/mypkg/calc.py translates to src.mypkg.calc.* instead of the expected mypkg.calc.*, and the assertion fails:

AssertionError: changed files should reach mutmut run as module globs
assert 'mutmut run mypkg.calc.*' in 'run --with mutmut==3.6.0 mutmut run src.mypkg.calc.* \nrun --with mutmut==3.6.0 mutmut results --all true \n'

Because this failure occurs during mutmut's baseline test run, mutmut aborts before mutating anything, and the job exits with Process completed with exit code 1 a few seconds later.

Proposed next step

Isolate the test from the calling environment's real INPUT_MODULE_PREFIX_STRIP, either by having test_scoped_run_passes_module_globs (and its siblings) explicitly monkeypatch.delenv("INPUT_MODULE_PREFIX_STRIP", raising=False) before invoking run_mutmut.app([]), or by having the test set INPUT_MODULE_PREFIX_STRIP explicitly to the value it means to exercise, rather than relying on an implicit script default that the real CI environment no longer provides. Whichever fix lands, the baseline should also be run once locally with INPUT_MODULE_PREFIX_STRIP="" set (matching the real caller configuration) to confirm the whole workflow_scripts suite tolerates it, since other tests may share the same blind spot.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions