Repository navigation
test(docs): test_docs_snippets never terminates when run as a narrow selection — CPU-bound, silent, and can hang ci.yml's python job #297
Description
tests/test_docs_snippets.py never terminates when it is run as a narrow selection, and completes in seconds when the whole tests/ directory runs. It burns CPU the whole time rather than blocking, so it presents as a hang with no diagnostic output at all.
Reproduction (on main @ 42262c3)
uv run pytest tests/test_docs_snippets.py -q # never returns
uv run pytest "tests/test_docs_snippets.py::test_documented_python_resolves_against_the_wheel" -q # sameSampling the process while it runs:
pid elapsed cpu state
40894 00:45 0:44.62 RN
40894 01:15 1:09.74 RN
CPU time tracks elapsed almost exactly, and the state is R — it is spinning in compute, not deadlocked on a lock or waiting on I/O. Killed manually at 75s and (in an earlier trial) at 100s; it has never been observed to finish in this mode.
Against that:
uv run pytest tests -n 4 -q # 215 passed in ~6-8s, INCLUDING this moduleSo the module is fine when its whole directory is collected, and does not terminate when it is selected on its own. Observed with and without -n, so xdist is not the variable.
Consequence beyond the narrow selection
The combined invocation pytest packages/laterite/tests tests -n 4 — which is what ci.yml's python job runs — can also hang on it. Under --dist load the module's tests are distributed across workers, so whether the prerequisite condition holds appears to depend on what else lands in that worker. When it hangs there it is worse than here, because the run produces no failure and no output: the job simply sits until it is killed.
That is how it was found. Chasing it cost roughly an hour of a session on the assumption that a coverage run was merely slow.
Where to look
Both tests in the module share one module-scoped fixture, audited (tests/test_docs_snippets.py:250), which walks every documented page and execs its snippets via _audit (:208). Deselecting one test therefore does not avoid the fixture — the other still triggers it, which is worth knowing before trying to work around this with --deselect.
The shape of the evidence — fine with siblings collected, non-terminating without them — points at the fixture depending on state some other test in tests/ establishes, with something failing to terminate when that state is absent rather than failing fast. _workdir() (:196) copies a fixture file into a temp cwd, and _audit execs each statement under contextlib.chdir(tmp), so the environment a snippet sees is assembled per-run; a snippet that scans or loops over that directory is one candidate, but this has not been narrowed further.
Not the same bug as #294
#294 is a heap corruption with a mechanism: a use-after-free reached through compat.dataframe_to_AGS4, surfacing as a SIGSEGV or a pyo3 panic depending on what the next native call touches. This one has no mechanism yet, and the two have different shapes — corruption versus an ordering/prerequisite dependency.
They meet only at test_docs_snippets, and that is a shared entry point, not a shared cause: a fixture that execs every documented snippet will keep reaching whatever native paths the docs advertise, so it is unsurprising that it is where both surfaced. Filed separately so this does not get closed when the segfault is fixed.
(The separation argument is another contributor's, working this repo in parallel; it corrected my initial assumption that the two were one story.)
Activity
niko86 commentedon Aug 17, 2026
This does not reproduce on current
main, and an A/B says why: it is #294 afterall, and #301 fixed it.
It no longer reproduces
Both commands from the report, on the shipped wheel:
Twenty consecutive runs of the narrow selection: 20 passed, 0 hung. The
detector was falsified first, because a harness that cannot see a hang reports
the same thing as a fix — a deliberate
while True: passunder the same bound iscaught as a kill (
rc=142), and a normal exit as0.The A/B
mimallocunpinned to v3 inrust-packages/laterite-py/Cargo.toml, rebuilt, andthe positive control run first — without it, a green docs result would only
mean the probe build was not faulty:
test_pyarrow_buffers_survive_a_coresident_native_moduleUnicodeDecodeError: 'utf-8' codec can't decode byte 0xf7 in position 16Stopped at six because 6/6 against 0/20 already settles it. Note it hangs every
time here, rather than the "never observed to finish, sometimes fine" of the
original report — consistent with a fault whose expression depends on allocation
history, which is also why collecting the whole
tests/directory changed theoutcome.
Environment restored afterwards: pin reverted, wheel rebuilt, control green.
Why the filing separated them
The report distinguished this from #294 on the grounds that #294 "has a mechanism
(a use-after-free ... surfacing as a SIGSEGV or a pyo3 panic)" while this one had
none, and that corruption and an ordering dependency are different shapes.
That reasoning was sound given what was known, but the premise did not survive:
#294 turned out not to be a use-after-free. It was two co-resident mimalloc
instances handing out overlapping memory, and a corrupted free-list spins on a
core exactly as described here — CPU time tracking elapsed, state
R, nodiagnostic. The two "different shapes" were one fault seen through two symptoms.
The timeline fits with nothing left over: #297 was filed 2026-08-12 17:47Z; #301
merged 2026-08-13 04:23Z, about eleven hours later. And this module is a
plausible place for it to land — it execs snippets from nine pages that reach
dataframe_to_AGS4/to_pandas/string_dtype, so it imports pyarrow andcalls into the cdylib, which is the whole condition.
Recommendation
Close as fixed by #301 — over to you rather than done here.
No new gate is owed.
test_pyarrow_buffers_survive_a_coresident_native_modulealready covers the cause, runs in a subprocess so it can express the
startup-ordering condition, and is red on a v3 build as shown above. A test
asserting "the docs module terminates" would be a slower, flakier proxy for a
mechanism that is already gated at its source.
Worth keeping from this: the "if it recurs" watch item in #346 is a different
mechanism and is untouched by any of the above.
niko86 commentedon Aug 18, 2026
Does not reproduce on current
main(6b957ad). Recording the evidence rather than acting on it, because there is no failing state here to fix against.Both documented reproductions complete, and the combined CI invocation does too:
Three consecutive narrow runs: 0.43s / 0.42s / 0.46s. The original report describes the narrow selection as never having been observed to finish, so a deterministic hang has become a deterministic pass.
ci.yml'spythonjob has also been green on recent PRs (~6m20–6m40).The harness was not restructured, so that is not the explanation. The shape this issue describes is intact —
_workdir(:197),_audit(:209), the module-scopedauditedfixture (:263). The line numbers moved only becausebefca86("Contain the process-wide state the docs correctly document", #328 via #372) added 59 lines to the module.That commit is the obvious suspect but I do not think it is the cause: it adds a guard that the module does not leak process-wide state, whereas this issue's hypothesis was the fixture depending on state a sibling test establishes. Opposite directions. I could not establish causation cheaply, and would rather say so than assert a tidy story.
More likely something the snippets exec changed underneath — the fixture runs whatever the docs advertise, so a fault in a native path reached that way would present here and vanish when that path was fixed elsewhere.
Suggested disposition: someone who can reproduce it should try current
mainbefore this is worked. If it is gone for them too, close it as fixed-by-unknown rather than leaving a ticket that costs a session to disprove. I have deliberately not added a timeout guard: with no reproducible hang there is nothing to show the guard firing, and a guard never seen red is the non-falsifiable-test smell this repo names.Environment: macOS, this checkout's
uvvenv. Since the original was environment-sensitive ("depends on what else lands in that worker"), one clean machine is evidence and not proof.niko86 commentedon Aug 19, 2026
No longer reproduces, and the reason is that it was never its own bug — it is #294.
The reconstruction
Worktree at the commit this issue names (
42262c3), the module's own narrow selection as the probe, killed at 55s:R, CPU tracking elapsed almost exactly, matching the sampling recorded aboveBoth halves of the fault were present when this was filed and neither is now:
uv.lockat42262c3had pyarrow 25.0.0, inside the 24.0.0–25.0.0 window, and the v2 pin (#301) landed after this issue. Either fix alone clears it.On the separation argument
The "Not the same bug as #294" section reasoned from shape — corruption versus an ordering dependency — and that is what made it look like two stories. The mechanism collapses them: a corrupted free list can spin as readily as it can crash, and which one surfaces depends on allocation order. That is also the whole of the puzzle in this issue's title. Collecting the siblings changes the allocation order, so the module was fine with them and not alone. No prerequisite state, no fixture dependency, nothing in
_workdir()or_audit.The "where to look" note above is therefore a dead end for anyone picking this up — worth knowing before it costs another hour.
What came out of it
Nothing here needs fixing. But the runtime guard for #294 states in its own comment that it can only go red on pyarrow 24.0.0–25.0.0, and the lock has been past that since 25.0.1 — so nothing was watching the pin itself, which is the half we control and the half that protects a user whose pyarrow we do not choose. #447 adds a manifest gate that does, and records the second face on
concepts/arrow-c-ffi-allocator-ownership.md.Suggest closing once #447 lands.
test: gate the mimalloc v2 pin off the manifests, where a fixed pyarr…