test: gate the mimalloc v2 pin off the manifests — and close out #297, which was #294 all along

[niko86]

✅ Based on main — no stacking, safe to merge when CI is green.

Started as "fix #297" and finished somewhere else: the hang no longer reproduces, because it was never its own bug.

What #297 actually was

#297 recorded tests/test_docs_snippets.py as never terminating when selected on its own — CPU-bound, no output, no diagnostic — and argued it was not #294, on the grounds that corruption and a spin are different shapes. Reasonable at the time, and wrong: it is the same co-resident-allocator fault.

Reconstructed against a worktree at the commit the issue names (42262c3), using the module's own selection as the probe. Two variables, four cells:

our pin	pyarrow	result
v3	25.0.0	HUNG — state R, CPU tracking elapsed, killed at 55s
v2	25.0.0	passed
v3	25.0.1	passed
v2	25.0.1	passed — today's environment, and 20/20 consecutive runs

The lock at 42262c3 had pyarrow 25.0.0, and the v2 pin (#301) landed after the issue was filed, so both halves of the fault were present then and neither is now. Either fix alone clears it.

The "narrow selection versus whole directory" behaviour that made it look like an ordering bug falls out of the mechanism: a corrupted free list can spin as readily as it can crash, and which one you get depends on allocation order. Collecting the siblings changes the allocation order.

What this PR adds, and why it is not "nothing to do"

Both halves are fixed, so there is no hang to fix. There is a gap, though, and it is the interesting part.

The runtime guard (test_pyarrow_buffers_survive_a_coresident_native_module) says of itself, in its own comment, that it can only go red on pyarrow 24.0.0–25.0.0. The lock has been past that window since 25.0.1. So today that guard passes no matter what the pin says — the pin being the half we control, and the half that protects a user whose pyarrow we do not choose.

tests/test_allocator_pin_faithful.py reads the pin off the manifests instead:

• discovers the crates rather than listing them — a #[global_allocator] can only be set by a final artifact, so a new binding or binary is covered the day it is added, not the day someone remembers this file;
• asserts features = ["v2"] on each one's mimalloc dep, with the reason in the failure message (omitting the feature is not a neutral default — libmimalloc-sys resolves v3 from 0.1.47 on, so the faulting configuration is three deleted characters away);
• buildless, so it runs in repo-gates in about a tenth of a second rather than behind a release build.

Falsified by dropping the feature from laterite-node: red on that crate alone, green again on restore.

Wiki

concepts/arrow-c-ffi-allocator-ownership.md owns this fault and did not know it had a second face. It does now, with the instruction that matters more than the anecdote: triage this one by its precondition (two co-resident allocators), never by its symptom. A fault you triage by symptom is one you re-file.

Follow-up

#297 can close — recommend the maintainer does so; the evidence is posted there. Nothing in it needs fixing, and its "Not the same bug as #294" section is superseded by the table above.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!