Linux-only debug-profile SIGSEGV in cargo-test from #10619 (shadow-stack TLS hot-cache), mechanism unconfirmed

[proggeramlug]

Summary

PR #10619 (perf/try-entry, "route the shadow-stack TLS through the
hot-cache fast path") caused CI's cargo-test job (Linux, debug profile,
ubuntu-latest) to fail with a SIGSEGV:

error: test failed, to rerun pass `-p perry-runtime --lib`
Caused by:
  process didn't exit successfully: `.../target/debug/build/perry-runtime/.../perry_runtime-...` (signal: 11, SIGSEGV: invalid memory reference)

Run: https://github.com/PerryTS/perry/actions/runs/35374727647/job/105594641738

The job's stdout is block-buffered, so no FAILED line and no panic message
survived; the last visible line was
test async_hooks::test_support::tests::before_after_restore_execution_ids ... ok.

The PR's own change is small: crates/perry-runtime/src/gc/roots/shadow_stack.rs
swaps the SHADOW: UnsafeCell<ShadowStackState> thread-local from a raw
thread_local! to crate::perry_thread_local! (the tls_hot.rs mechanism
that caches a resolved TLS address in a per-thread slot table), for an ~8%
win on try/catch entry (_tlv_get_addr was a real per-entry leaf cost on
Darwin). It was confirmed clean on cargo-test runs for #10644, #10647,
#10650, #10651 against the same base — so this is #10619's own failure, not
pre-existing flake.

What was tried

Reproduction, debug profile, RUST_TEST_THREADS=1 cargo test -p perry-runtime --lib:

• macOS (arm64), Darwin perry_thread_local! path (as shipped): clean.
  4013 passed, 0 failed, 4 ignored, ~167s. The test immediately after the
  CI log's last visible line
  (async_hooks::test_support::tests::native_async_resource_accepts_string_and_symbol_expandos)
  ran fine, back to back with no gap.
• macOS (arm64) with the Darwin pthread-TSD path forced off (all 11
  target_vendor = "apple" cfg predicates in tls_hot.rs flipped to an
  unsatisfiable vendor, so hot() resolves through the generic
  hot_via_tls() that every non-Darwin-aarch64 target already uses): also
  clean. 4010 passed, 0 failed (same 4010/4013 shape as the release
  numbers below, matching the compiled-out debug_assert! count for a debug
  build vs. what's expected — no crash). This weakens the theory that the
  generic (non-Darwin) hot_via_tls() code path is inherently broken for
  SHADOW: on this host it isn't.
• Linux x86_64, real target (qemu-emulated Ubuntu VM, matching
  ubuntu-latest's triple and the CI-pinned nightly-2026-08-20 toolchain),
  same debug command: ran clean through at least 1968 of ~4013 tests in
  alphabetical run order (comfortably past the async_hooks region where CI's
  log cuts off — no crash there or anywhere in between). The run stalled
  (not crashed — process stayed alive, just made no forward progress for
  10+ minutes) at
  gc::tests::runtime_roots::perex_replace::perex_replace_output_is_not_capped_by_the_scratch_limit,
  which is unrelated to shadow-stack/TLS (a regex-replace scratch-buffer
  stress test) and runs in well under a second natively on macOS — almost
  certainly a qemu-emulation-specific slowdown for that one test, not a hang
  tied to this PR. A second run skipping that one test
  (-- --skip perex_replace_output_is_not_capped_by_the_scratch_limit) was
  in progress at the time of writing; this issue will be updated with its
  result.

So: three independent environments (Darwin fast path, Darwin with the
fast path forced off, and Linux/x86_64 on the actual target triple) all ran
clean, and none reproduced the SIGSEGV.

What's still unexplained

The CI SIGSEGV is real (three separate PRs' clean cargo-test runs against
the same base rule out pre-existing flake) but was never pinned to a
specific test, a specific line, or a backtrace. Two hypotheses were floated
during the investigation and neither is confirmed or ruled out:

1. Re-entrancy in tls_hot.rs's fill(). fill() (tls_hot.rs:~296)
   resolves the 17 named HotTls fields and writes temp_roots last,
   with a comment that this ordering stops "a re-entrant call from inside
   one of the providers above" from observing a half-filled cache as ready.
   That ordering prevents using a half-filled cache — it does not prevent
   recursion: a re-entrant caller mid-fill() still sees
   temp_roots.is_null() and calls fill() again. Whether anything on the
   resolution path (HotKey::resolve_and_cache → hot() →
   hot_via_tls()/hot_uncached() → fill()) can actually re-enter itself
   for SHADOW specifically was not confirmed either way.
2. Debug-profile stack depth under CI's default thread, since this repo
   has hit exactly this signature before (a deep recursive-descent test,
   green locally under --release, SIGSEGV in CI's debug cargo-test, same
   missing FAILED line from stdout buffering). Not confirmed here either.

Given the reproduction environments above are the closest available proxies
for CI's actual runner and all came back clean, closing this without a
confirmed mechanism is reasonable if a future maintainer wants to; it's
filed instead so the SIGSEGV doesn't evaporate silently. If it recurs, the
fill() ordering note above is where to look first, and a real
ubuntu-latest GitHub-hosted runner (not qemu-emulated) would be worth
trying before a from-scratch re-investigation.

Resolution shipped alongside this issue

Independent of the crash's cause, tls_hot.rs's own module docs already say
the published-cache shortcut only pays for itself on Darwin aarch64 — on
every other target "resolving a thread-local is already a fixed offset and
the extra cache indirection has no demonstrated benefit." So routing
SHADOW through perry_thread_local! unconditionally was strictly more
work off that platform even before the SIGSEGV: one extra thread-local
resolution (HOT) plus a slot-array indirection, replacing the single
direct TLS access a raw thread_local! already was. The try-entry win was
only ever measured on Darwin.

crates/perry-runtime/src/gc/roots/shadow_stack.rs's SHADOW declaration
is now cfg-split: perry_thread_local! only under
all(target_vendor = "apple", target_arch = "aarch64", target_pointer_width = "64"),
a plain thread_local! everywhere else (which is what shipped before
#10619). This keeps the measured win where it was measured and removes the
pessimization — and, as a side effect, removes CI's only exposure to
whatever caused this SIGSEGV, since the generic hot_via_tls() path no
longer touches SHADOW on Linux at all. That is a mitigation, not a
diagnosis: the mechanism is still not understood, which is the reason this
issue stays open.

[proggeramlug]

Update: causation confirmed via CI, mechanism still open

Local reproduction (documented above) never settled whether #10619 actually
caused the SIGSEGV — macOS came back clean on both the Darwin
perry_thread_local! path and with it forced off, and a qemu-emulated
Linux x86_64 VM ran clean through 1968+ of ~4013 tests before turning out to
be unusable as an instrument for the rest of the suite (one unrelated
regex-replace stress test, perex_replace_output_is_not_capped_by_the_scratch_limit,
is pathologically slow under emulation — fast natively — and an attempted
A/B there produced two SIGKILLs that were briefly misread as a reproduced
crash before being traced to an operator pkill -f pattern self-matching
its own remote shell session, a known pitfall of running multi-line scripts
over ssh/limactl shell, not a real fault).

So the question was settled the direct way instead: the PR's #[cfg]-gated
fix (SHADOW goes through perry_thread_local! only on
all(target_vendor = "apple", target_arch = "aarch64", target_pointer_width = "64"),
a plain thread_local! everywhere else) was pushed to the draft PR, and
CI's own cargo-test job — the actual failing runner — was read directly:

• Before (unconditional perry_thread_local! swap): red, SIGSEGV. Run
  35374727647, job 105594641738.
• After (this issue's fix): green. Run 35433215970, job 105871415920.

Same job, same runner class, same test suite, only the SHADOW declaration
changed between the two. This confirms #10619 did cause the SIGSEGV —
the earlier "three clean local environments" framing in the issue body above
undersold it; it should have been read as "local reproduction is
inconclusive," not "probably not this PR."

What is still not understood: the actual mechanism inside
tls_hot.rs's resolution path that faults on Linux debug builds. The
fill() re-entrancy note from the issue body remains the leading unconfirmed
lead — fill() (tls_hot.rs:~296) writes temp_roots last specifically so
a re-entrant caller can't observe a half-filled cache as ready, but that
ordering does not stop fill() from being called again re-entrantly. This
was never confirmed with a backtrace, because qemu-emulated Linux (the only
Linux target available) was too slow to get one economically, and the fix
landing removes CI's only exposure to whatever it is — so there's no longer
a red build motivating further digging.

Leaving this open as originally intended: the fix mitigates by removing the
change from every platform it had no measured benefit on anyway (independent
of the crash), not by understanding the fault. If the mechanism becomes
relevant again — e.g. some other Linux-only perry_thread_local! conversion
trips the same thing — this is where to start, ideally on real (non-emulated)
x86_64 hardware with RUST_BACKTRACE=1 and a debugger attached, rather than
qemu.