Skip to content

feat: add --max-memory to bound each scenario's memory use - #649

Open
ykhrustalev wants to merge 1 commit into
sourcefrog:mainfrom
ykhrustalev:ykhrustalev/max-memory
Open

ykhrustalev wants to merge 1 commit into
sourcefrog:mainfrom
ykhrustalev:ykhrustalev/max-memory

Conversation

@ykhrustalev

@ykhrustalev ykhrustalev commented Sep 16, 2026

Copy link
Copy Markdown

Problem
A mutant can turn a bounded loop into an unbounded allocator. A test process growing at hundreds of MB/s exhausts the machine long before a 5×-baseline test timeout fires — on CI the VM is torn down with no log, and the shard's mutants are never recorded.

Solution

  • Adds --max-memory SIZE and the max_memory config key to bound each scenario's cargo process tree.
  • Enforces it with setrlimit(RLIMIT_AS) on the cargo process, inherited by everything it spawns, and logs the mechanism at startup.
  • Fails before testing any mutant when the option is given and cannot be applied, rather than running with no limit. macOS accepts but ignores RLIMIT_AS, so the option warns and is a no-op there.
  • Rejects values under 1M: unlike -t 0, --max-memory=0 would mean "stop everything", not "no limit".

Testing

  • Unit tests for size parsing, CLI-over-config precedence, and mechanism selection.
  • Integration tests: a too-small limit is rejected; an 8G limit doesn't disturb a normal run.

Split out of #647 as requested. RLIMIT_AS limits address space rather than resident memory and has to be set generously; next in the stack: #651 adds a cgroup v2 memory.max backend that limits what actually matters and prefers it where available, and #653 reports its OOM kills.

A mutant can turn a bounded loop into an unbounded allocator, and a test
process growing at hundreds of MB/s exhausts the machine well before a
5x-baseline test timeout arrives. On a CI runner the VM is then torn down
with no log, and the shard's mutants are never recorded.

--max-memory SIZE, and the max_memory config key, put a ceiling on each
scenario's cargo process tree instead. It is enforced with setrlimit
(RLIMIT_AS), which limits address space rather than resident memory and so
has to be set generously; which mechanism is in use is logged at startup.

macOS accepts RLIMIT_AS and ignores it, so there the option warns and does
nothing, as documented. Where it cannot be applied at all, giving the option
is an error raised before any mutant runs, rather than a long run that
silently had no limit. Zero is rejected too: unlike -t 0 it would mean 'stop
everything', not 'no limit'.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant