Description
When a WorkerPool teardown exceeds its shutdown timeout, the pool abandons the unresponsive worker by design (pool.py logs the abandoned worker uid) — but the abandoned WorkerProcess then holds the parent interpreter's exit hostage. multiprocessing registers an atexit handler that joins every live non-daemon child, and an abandoned worker never exits on its own: its parent-death watchdog only fires when the parent dies, which is precisely what the join prevents. The parent blocks in os.waitpid indefinitely.
Observed twice running the full test suite: tests/integration/test_dispatch_hypothesis timed out under machine load, its pool teardown abandoned a live worker, and pytest — with all tests complete and results buffered — wedged at interpreter exit until the orphan was killed externally. Stack at the wedge: Py_RunMain → _Py_Finalize → atexit_callfuncs → os_waitpid → __wait4, with the leaked worker idle in its event loop.
The trigger test is itself load-sensitive (a heavy Hypothesis exploration that passes in isolation in ~100s), but the wedge amplifies any timed-out or crashed test that strands a worker — a one-test flake becomes a whole-suite hang.
Expected behavior
An abandoned worker must not block the parent's exit. Either spawn WorkerProcess with daemon=True — safe because the parent-death watchdog already guarantees the reverse direction (workers self-terminate when the parent goes away) — or explicitly detach the process when the pool's shutdown-timeout path abandons it. Secondarily, test_dispatch_hypothesis deserves timeout headroom or retry coverage so a slow machine doesn't strand workers in the first place.
Root cause
WorkerProcess extends multiprocessing.Process without daemon=True (process.py), so multiprocessing.util._exit_function joins it at interpreter exit. The pool's shutdown-timeout path (pool.py:667-716) deliberately abandons workers that fail to stop in time but leaves them subject to that join. The abandoned worker keeps serving: its _parent_watchdog fires only on parent death, which the atexit join itself forestalls — a deadlock between the parent's exit and the child's exit condition.
Distinct from #268: that issue covers stranded in-loop ResourcePool._schedule_cleanup tasks from unstopped inline services in worker unit tests — benign, GC-timed teardown warnings with no production change required. This issue is an OS-subprocess leak that hard-wedges interpreter exit, surfaced from the integration suite, and implicates the production spawn/abandonment path.
Description
When a
WorkerPoolteardown exceeds its shutdown timeout, the pool abandons the unresponsive worker by design (pool.pylogs the abandoned worker uid) — but the abandonedWorkerProcessthen holds the parent interpreter's exit hostage.multiprocessingregisters an atexit handler that joins every live non-daemon child, and an abandoned worker never exits on its own: its parent-death watchdog only fires when the parent dies, which is precisely what the join prevents. The parent blocks inos.waitpidindefinitely.Observed twice running the full test suite:
tests/integration/test_dispatch_hypothesistimed out under machine load, its pool teardown abandoned a live worker, and pytest — with all tests complete and results buffered — wedged at interpreter exit until the orphan was killed externally. Stack at the wedge:Py_RunMain → _Py_Finalize → atexit_callfuncs → os_waitpid → __wait4, with the leaked worker idle in its event loop.The trigger test is itself load-sensitive (a heavy Hypothesis exploration that passes in isolation in ~100s), but the wedge amplifies any timed-out or crashed test that strands a worker — a one-test flake becomes a whole-suite hang.
Expected behavior
An abandoned worker must not block the parent's exit. Either spawn
WorkerProcesswithdaemon=True— safe because the parent-death watchdog already guarantees the reverse direction (workers self-terminate when the parent goes away) — or explicitly detach the process when the pool's shutdown-timeout path abandons it. Secondarily,test_dispatch_hypothesisdeserves timeout headroom or retry coverage so a slow machine doesn't strand workers in the first place.Root cause
WorkerProcessextendsmultiprocessing.Processwithoutdaemon=True(process.py), somultiprocessing.util._exit_functionjoins it at interpreter exit. The pool's shutdown-timeout path (pool.py:667-716) deliberately abandons workers that fail to stop in time but leaves them subject to that join. The abandoned worker keeps serving: its_parent_watchdogfires only on parent death, which the atexit join itself forestalls — a deadlock between the parent's exit and the child's exit condition.Distinct from #268: that issue covers stranded in-loop
ResourcePool._schedule_cleanuptasks from unstopped inline services in worker unit tests — benign, GC-timed teardown warnings with no production change required. This issue is an OS-subprocess leak that hard-wedges interpreter exit, surfaced from the integration suite, and implicates the production spawn/abandonment path.