fix: sweep the child's process group after every phase

NEWS.md

@@ -2,6 +2,10 @@
 
 ## Unreleased
 
+- Fixed: Each cargo invocation now runs as the leader of its own process group, and that group is swept after every phase, not only after a timeout. Processes left running by a scenario's tests are `SIGTERM`ed, given a short grace period, and then `SIGKILL`ed, so they can't keep consuming memory or CPU while later mutants are tested. Unix only.
+
+- Fixed: After a timeout, cargo-mutants waits only a short grace period for the child to exit after `SIGTERM` before sending `SIGKILL`. Previously it waited indefinitely, so a process that ignored `SIGTERM`, or that had been stopped, hung the whole run.
+
 - New: `#[mutants::exclude_re("pattern")]` attribute to exclude specific mutations by regex, without disabling all mutations on the function. The attribute can be placed on functions, `impl` blocks, `trait` blocks, modules, files, and on expressions that can carry an attribute (such as `match`, struct literals, call expressions, method calls, and unary expressions). Multiple patterns can be applied. Also supported within `cfg_attr`. Requires the [mutants](https://crates.io/crates/mutants) crate version `0.0.5` or later.
 
 - Fixed: `#[mutants::skip]` (and `#[cfg_attr(..., mutants::skip)]`) is now honoured when placed on `const` and `static` items, including associated constants in `impl` and `trait` blocks. Previously the attribute was silently ignored on these items and operator mutants inside the initializer expression were still generated ([#508](https://github.com/sourcefrog/cargo-mutants/issues/508)).

book/src/timeouts.md

@@ -33,6 +33,11 @@ You can set an explicit timeouts with the `--timeout` option, also measured in s
 You can also set the test timeout as a multiple of the duration of the baseline test, with the `--timeout-multiplier` option and the `timeout_multiplier` configuration key.
 The multiplier only has an effect if the baseline is not skipped and if `--timeout` is not specified.
 
+When a timeout expires, the process is sent `SIGTERM` so that it can clean up, and then
+`SIGKILL` if it has not exited within a short grace period. A test process that ignores
+`SIGTERM`, or that has been stopped and so never receives it, therefore cannot stall the
+whole run.
+
 ## Build timeouts
 
 `const` expressions may be evaluated at compile time. In the same way that mutations can cause tests to hang, mutations to const code may potentially cause the compiler to enter an infinite loop.
@@ -45,6 +50,23 @@ In this case you can use the `--build-timeout` or `--build-timeout-multiplier` o
 
 You might also choose to skip mutants that can cause long-running const evaluation.
 
+## Leftover processes
+
+A test can leave processes running after it returns: a daemon it started, a helper it
+forgot to wait for, or a test binary that was not reaped. Those processes keep running
+— and keep allocating — while cargo-mutants moves on to the next mutant, so they can
+exhaust the machine's memory in a window where no cargo phase is running at all.
+
+To prevent this, cargo-mutants starts each cargo invocation as the leader of its own
+process group, and sweeps that group after *every* phase, not only after a timeout.
+Once the cargo process itself exits, anything left in the group is sent `SIGTERM`, given
+a short grace period, and then `SIGKILL`ed. What was reaped is recorded in the
+scenario's log, and the pids are shown at `--level=debug`.
+
+This has no effect on how a mutant is classified; it only stops work from one scenario
+leaking into the next. Windows has no process groups, and cargo-mutants does not yet use
+job objects, so this sweep is Unix-only.
+
 ## Exceptions
 
 The multiplier timeout options cannot be used when the baseline is skipped

src/cargo.rs

@@ -55,7 +55,7 @@ pub fn run_cargo(
         debug!(?encoded_rustflags);
         env.push(("CARGO_ENCODED_RUSTFLAGS".to_owned(), encoded_rustflags));
     }
-    let process_status = Process::run(
+    let (process_status, _sweep) = Process::run(
         &argv,
         &env,
         build_dir.path(),

src/process.rs

@@ -15,6 +15,7 @@ use std::time::{Duration, Instant};
 
 use anyhow::Context;
 use camino::Utf8Path;
+use itertools::Itertools;
 use serde::Serialize;
 use tracing::{Level, debug, span, trace};
 
@@ -26,15 +27,57 @@ use crate::output::ScenarioOutput;
 /// How frequently to check if a subprocess finished.
 const WAIT_POLL_INTERVAL: Duration = Duration::from_millis(50);
 
+/// How long to let a process, or a process group, wind up after `SIGTERM` before
+/// sending `SIGKILL`.
+const TERM_GRACE: Duration = Duration::from_millis(500);
+
+/// How often to check whether a signalled process or group has gone away.
+const TERM_POLL_INTERVAL: Duration = Duration::from_millis(20);
+
 #[cfg(windows)]
 mod windows;
 #[cfg(windows)]
-use windows::{configure_command, terminate_child};
+use windows::{configure_command, kill_child, sweep_process_group, terminate_child};
 
 #[cfg(unix)]
 mod unix;
 #[cfg(unix)]
-use unix::{configure_command, terminate_child};
+use unix::{configure_command, kill_child, sweep_process_group, terminate_child};
+
+/// What sweeping a finished child's process group found and did.
+///
+/// A scenario's tests can leave processes running: a test binary that was not waited
+/// for, or anything a test spawned and forgot. Those processes stay in the child's
+/// process group, and would otherwise keep running (and keep allocating) while
+/// cargo-mutants moves on to later scenarios.
+#[derive(Debug, Default, Clone, PartialEq, Eq, Serialize)]
+pub struct Sweep {
+    /// The pids that were in the group, where the platform can enumerate them.
+    pub pids: Option<Vec<i32>>,
+    /// Whether any process was still in the group after the direct child exited.
+    pub strays: bool,
+    /// Whether `SIGTERM` was not enough, and `SIGKILL` had to be sent.
+    pub killed: bool,
+}
+
+impl Sweep {
+    /// Describe what was reaped, or None if nothing was left over.
+    pub fn describe(&self) -> Option<String> {
+        if !self.strays {
+            return None;
+        }
+        let how = if self.killed { "SIGKILLed" } else { "reaped" };
+        Some(match &self.pids {
+            Some(pids) if !pids.is_empty() => format!(
+                "left {n} stray process{es} behind ({how}: {list})",
+                n = pids.len(),
+                es = if pids.len() == 1 { "" } else { "es" },
+                list = pids.iter().join(", ")
+            ),
+            _ => format!("left stray processes behind ({how})"),
+        })
+    }
+}
 
 pub struct Process {
     child: Child,
@@ -45,6 +88,9 @@ pub struct Process {
 impl Process {
     /// Run a subprocess to completion, watching for interrupts, with a timeout, while
     /// ticking the progress bar.
+    ///
+    /// Whatever the outcome, the child's process group is swept before returning, so
+    /// that nothing it left running survives into the next scenario.
     pub fn run(
         argv: &[String],
         env: &[(String, String)],
@@ -53,17 +99,42 @@ impl Process {
         jobserver: Option<&jobserver::Client>,
         scenario_output: &mut ScenarioOutput,
         console: &Console,
-    ) -> Result<Exit> {
+    ) -> Result<(Exit, Sweep)> {
         let mut child = Process::start(argv, env, cwd, timeout, jobserver, scenario_output)?;
-        let process_status = loop {
-            if let Some(exit_status) = child.poll()? {
-                break exit_status;
+        let result = loop {
+            match child.poll() {
+                Ok(Some(exit_status)) => break Ok(exit_status),
+                Ok(None) => {}
+                Err(err) => break Err(err),
             }
             console.tick();
             sleep(WAIT_POLL_INTERVAL);
         };
+        let sweep = child.sweep()?;
+        let process_status = result?;
         scenario_output.message(&format!("result: {process_status:?}"))?;
-        Ok(process_status)
+        if let Some(description) = sweep.describe() {
+            scenario_output.message(&description)?;
+        }
+        Ok((process_status, sweep))
+    }
+
+    /// Kill anything the child left running in its process group.
+    ///
+    /// This runs after every phase, not only after a timeout: a scenario that exited
+    /// cleanly can still have left a test binary or a process spawned by a test behind.
+    fn sweep(&mut self) -> Result<Sweep> {
+        let sweep = sweep_process_group(&self.child)?;
+        if sweep.strays {
+            debug!(
+                pids = ?sweep.pids,
+                killed = sweep.killed,
+                "swept processes left over in the child's process group"
+            );
+        } else {
+            trace!("no processes left in the child's process group");
+        }
+        Ok(sweep)
     }
 
     /// Launch a process, and return an object representing the child.
@@ -124,16 +195,41 @@ impl Process {
     ///
     /// Blocks until the subprocess is terminated and then returns the exit status.
     ///
+    /// `SIGTERM` first, so it gets a chance to clean up, but only for a bounded grace
+    /// period: a process that ignores it, or that is stopped and so never receives it,
+    /// would otherwise hang the run here forever. Whatever else is left in the process
+    /// group is dealt with by the sweep in [`Process::run`].
+    ///
     /// The status might not be `Timeout` if this raced with a normal exit.
     #[mutants::skip] // would leak processes from tests if skipped
     fn terminate(&mut self) -> Result<()> {
         let _span = span!(Level::DEBUG, "terminate_child", pid = self.child.id()).entered();
         debug!("terminating child process");
         terminate_child(&mut self.child)?;
         trace!("wait for child after termination");
+        let deadline = Instant::now() + TERM_GRACE;
+        loop {
+            match self.child.try_wait() {
+                Ok(Some(exit)) => {
+                    debug!("terminated child exit status {exit:?}");
+                    return Ok(());
+                }
+                Ok(None) => {}
+                Err(err) => {
+                    debug!(?err, "Failed to wait for child after termination");
+                    return Ok(());
+                }
+            }
+            if Instant::now() >= deadline {
+                debug!("child did not exit after SIGTERM; killing it");
+                kill_child(&mut self.child)?;
+                break;
+            }
+            sleep(TERM_POLL_INTERVAL);
+        }
         match self.child.wait() {
-            Err(err) => debug!(?err, "Failed to wait for child after termination"),
-            Ok(exit) => debug!("terminated child exit status {exit:?}"),
+            Err(err) => debug!(?err, "Failed to wait for child after kill"),
+            Ok(exit) => debug!("killed child exit status {exit:?}"),
         }
         Ok(())
     }

src/process/unix.rs

@@ -1,5 +1,7 @@
 use std::os::unix::process::{CommandExt, ExitStatusExt};
 use std::process::{Child, Command, ExitStatus};
+use std::thread::sleep;
+use std::time::Instant;
 
 use anyhow::bail;
 use nix::errno::Errno;
@@ -9,29 +11,123 @@ use tracing::warn;
 
 use crate::Result;
 
-use super::Exit;
+use super::{Exit, Sweep, TERM_GRACE, TERM_POLL_INTERVAL};
 
-#[allow(unknown_lints, clippy::needless_pass_by_ref_mut)] // To match Windows
+/// Send a signal to a process group, returning whether the group still had any member.
+///
+/// A signal of `None` sends nothing and just probes for members.
 #[mutants::skip] // hard to exercise the ESRCH edge case
-pub(super) fn terminate_child(child: &mut Child) -> Result<()> {
-    let pid = Pid::from_raw(child.id().try_into().unwrap());
-    match killpg(pid, Signal::SIGTERM) {
-        Ok(()) => Ok(()),
-        Err(Errno::ESRCH) => {
-            Ok(()) // Probably already gone
-        }
+fn signal_group(pgid: Pid, signal: Option<Signal>) -> Result<bool> {
+    match killpg(pgid, signal) {
+        Ok(()) => Ok(true),
+        Err(Errno::ESRCH) => Ok(false), // the group is empty
         Err(Errno::EPERM) if cfg!(target_os = "macos") => {
-            Ok(()) // If the process no longer exists then macos can return EPERM (maybe?)
+            Ok(false) // If the process no longer exists then macos can return EPERM (maybe?)
         }
         Err(errno) => {
             // TODO: Maybe strerror?
-            let message = format!("failed to terminate child: error {errno}");
+            let message = format!("failed to signal process group {pgid}: error {errno}");
             warn!("{}", message);
             bail!(message);
         }
     }
 }
 
+#[allow(unknown_lints, clippy::needless_pass_by_ref_mut)] // To match Windows
+#[mutants::skip] // would leak processes from tests if skipped
+pub(super) fn terminate_child(child: &mut Child) -> Result<()> {
+    signal_group(child_pgid(child), Some(Signal::SIGTERM))?;
+    Ok(())
+}
+
+#[allow(unknown_lints, clippy::needless_pass_by_ref_mut)] // To match Windows
+#[mutants::skip] // would leak processes from tests if skipped
+pub(super) fn kill_child(child: &mut Child) -> Result<()> {
+    signal_group(child_pgid(child), Some(Signal::SIGKILL))?;
+    Ok(())
+}
+
+/// Kill anything left in a child's process group once the child itself has exited.
+///
+/// The child was started as the leader of its own process group, so anything it or its
+/// descendants spawned and left running is still in that group, even after being
+/// reparented. Send `SIGTERM`, give the group a moment to wind up, then `SIGKILL`
+/// whatever is left.
+#[mutants::skip] // would leak processes from tests if skipped
+pub(super) fn sweep_process_group(child: &Child) -> Result<Sweep> {
+    let pgid = child_pgid(child);
+    // Probe before enumerating: almost every phase leaves nothing behind, and listing
+    // the group means reading every /proc/<pid>/stat on the machine.
+    if !signal_group(pgid, None)? {
+        return Ok(Sweep::default());
+    }
+    let pids = group_members(pgid);
+    signal_group(pgid, Some(Signal::SIGTERM))?;
+    let deadline = Instant::now() + TERM_GRACE;
+    loop {
+        if !signal_group(pgid, None)? {
+            return Ok(Sweep {
+                pids,
+                strays: true,
+                killed: false,
+            });
+        } else if Instant::now() >= deadline {
+            break;
+        }
+        sleep(TERM_POLL_INTERVAL);
+    }
+    signal_group(pgid, Some(Signal::SIGKILL))?;
+    Ok(Sweep {
+        pids,
+        strays: true,
+        killed: true,
+    })
+}
+
+/// The process group id of a child, which (because we start it with `process_group(0)`)
+/// is the same as its pid.
+///
+/// Callers signal this after the child has been reaped, by which point the group may be
+/// empty and the kernel free to hand the same number to something else. Hitting that
+/// would take a full wrap of the pid space inside the microseconds between reaping and
+/// signalling, so we live with it.
+fn child_pgid(child: &Child) -> Pid {
+    Pid::from_raw(child.id().try_into().expect("child pid fits in pid_t"))
+}
+
+/// List the pids currently in a process group.
+///
+/// Returns `None` on platforms where we can't enumerate processes: the sweep still
+/// works there, it just can't say what it killed.
+#[cfg(target_os = "linux")]
+#[mutants::skip] // only affects what we can say in the debug log
+fn group_members(pgid: Pid) -> Option<Vec<i32>> {
+    let members = std::fs::read_dir("/proc")
+        .ok()?
+        .flatten()
+        .filter_map(|dir_entry| dir_entry.file_name().to_string_lossy().parse::<i32>().ok())
+        .filter(|pid| pgid_of(*pid) == Some(pgid.as_raw()))
+        .collect();
+    Some(members)
+}
+
+/// Read a process's group id out of `/proc/<pid>/stat`.
+#[cfg(target_os = "linux")]
+#[mutants::skip] // only affects what we can say in the debug log
+fn pgid_of(pid: i32) -> Option<i32> {
+    let stat = std::fs::read_to_string(format!("/proc/{pid}/stat")).ok()?;
+    // The second field is the command name in parentheses, and may itself contain spaces
+    // and parentheses, so only split the fields after its closing paren. Counting from
+    // there, the fields are: state, ppid, pgrp.
+    let (_, after_comm) = stat.rsplit_once(')')?;
+    after_comm.split_whitespace().nth(2)?.parse().ok()
+}
+
+#[cfg(not(target_os = "linux"))]
+fn group_members(_pgid: Pid) -> Option<Vec<i32>> {
+    None
+}
+
 #[mutants::skip]
 pub(super) fn configure_command(command: &mut Command) {
     command.process_group(0);

src/process/windows.rs

@@ -4,13 +4,26 @@ use anyhow::Context;
 
 use crate::Result;
 
-use super::Exit;
+use super::{Exit, Sweep};
 
 #[mutants::skip] // hard to exercise the ESRCH edge case
 pub(super) fn terminate_child(child: &mut Child) -> Result<()> {
     child.kill().context("Kill child")
 }
 
+/// Windows has no `SIGTERM`, so `terminate_child` already killed it outright.
+#[mutants::skip] // would leak processes from tests if skipped
+pub(super) fn kill_child(child: &mut Child) -> Result<()> {
+    child.kill().context("Kill child")
+}
+
+/// Windows has no process groups; the equivalent would be a job object, which we don't
+/// use yet, so there is nothing to sweep.
+#[allow(clippy::unnecessary_wraps)] // To match Unix
+pub(super) fn sweep_process_group(_child: &Child) -> Result<Sweep> {
+    Ok(Sweep::default())
+}
+
 #[mutants::skip]
 pub(super) fn configure_command(_command: &mut Command) {}