From f9eb5ea7765a12998055d17ac8e87e69c0fb260a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ralph=20K=C3=BCpper?= Date: Fri, 18 Sep 2026 13:26:13 +0200 Subject: [PATCH 1/8] perf(runtime): route the shadow-stack TLS through the hot-cache fast path MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Every non-throwing `try` entry paid a real out-of-line `_tlv_get_addr` call, unconditionally, that nothing else in `try_push_with_kind` did. Profiling `try { t += v; } catch { t = 0; }` (differenced inside each binary: `(try80-try16)/64` vs `(loop80-loop16)/64` control, per CLAUDE.md's verification method, N=20000, median of 7) confirmed the 178-instruction/entry baseline and pinned the cost to one leaf via `sample`+`PERRY_KEEP_SYMBOLS=1` at 20M iterations: `try_push_with_kind`+176 -> `_tlv_get_addr`, ~6% of instructions retired per entry. Root cause: `CatchSavepoint::capture()`'s shadow field is captured unconditionally on every entry (`shadow_stack_savepoint`), and once any function anywhere in the process has pushed a shadow frame — true almost immediately for any real program, since a `catch (e)` binding itself needs one — the `SHADOW_FRAMES` latch from #7469's wave-1 fix is already set, so the latch stops making this read rare in practice. Unlike `EXCEPTION_STATE`, `CALL_METHOD_DEPTH`, and the named `runtime_handle_stack`/`temp_roots` fields, `SHADOW` was still a raw `thread_local!`, so this one read never got the `tls_hot` fast path at all. Fix: move `SHADOW` (crates/perry-runtime/src/gc/roots/shadow_stack.rs) from `thread_local!` to `crate::perry_thread_local!` — a pure storage-mechanism swap (same type, same `.with()`/`.try_with()` call sites, same const-init/drop-free semantics: `ShadowStackState` still has no `Drop`, so no destructor is registered) that changes nothing about liveness, throw-time restore, or the fixed-address contract `js_shadow_frame_enter` depends on for its whole-activation pointer cache. Confirmed via `--trace llvm`/`sample` that the leaf disappears entirely post-change. Measured (mybase = own build at this commit's parent, 9df5075fbe, codegen-units=16; both PERRY_NO_AUTO_OPTIMIZE=1): before: try 179.0-179.4 instr/entry, control -0.001..-0.2 after: try 164.3-165.7 instr/entry, control -0.14..+0.44 -> ~14-15 instructions/entry (~8%), controls stay noise-floor both arms. `try_push_with_kind`'s remaining `CatchSavepoint::capture()` work and `js_try_end`'s own (already tls_hot-fast) EXCEPTION_STATE resolution are UNTOUCHED — js_try_end disassembles to ~28 near-minimal instructions with nothing left to cut without threading a pointer through codegen from push to end, which was evaluated and deferred as materially riskier (touches try_stmt.rs's early-exit/closure/ generator/async call sites) for a smaller remaining win. scripts/thread_local_cold_allowlist.json: only the `shadow_stack.rs: 2 -> 1` line this change causes. `_hot_declarations` is left untouched — verify() never reads it (only `files` is enforced) — because a plain `--update` on unmodified main already produces 466, not the committed 460: six pre-existing, unrelated `perry_thread_local!` additions had drifted the count before this change touched anything. Fixing that drift is not this PR's job. Correctness: the shadow stack is the GC's precise root set, not pure exception bookkeeping, so this was checked past "it compiles": - Added test-files/test_gap_try_entry_shadow_hot_tls.ts (byte-identical to node v26.5.1 --experimental-strip-types --no-warnings, checked against both this change and the pristine parent arm): same-level catch, a throw 4 frames deep, a throw crossing Array.prototype.map's runtime trampoline, finally on both the normal and throwing paths, nested try with an inner rethrow, a catch that itself throws, 25 levels of try/finally nesting with finally-order and try_depth restored checked afterward, and a non-throwing control loop. Every caught value is read back after GC-pressure allocation. - Ran that fixture, plus the pre-existing test_gap_gc_catch_param_rooting.ts, test_gap_try_savepoint_subsystems.ts, test_gap_try_finally_no_catch_rethrow.ts and test_gap_try_setjmp_volatile.ts, under PERRY_GC_SCHEDULE_SEED (5 seeds) + PERRY_GC_SCHEDULE_RATE=1 + PERRY_GC_SCHEDULE_ALLOC_KB=0 + PERRY_GC_PROTECT_FROMSPACE=1 + PERRY_GC_DIAG=1 (per CLAUDE.md's rooting-bug instruments) — a collect-at-every-safepoint, evacuating, quarantine-and-mprotect schedule. All 5 seeds matched node byte-for-byte; the diagnostic output confirms the instrument was live, not vacuous (forced_collections=2566, copying_minors=2566, moved_objects=37000, fromspace retired_set up to #4 with bytes_protected growing) — no stale-shadow-stack SIGSEGV, no output drift. - `RUST_TEST_THREADS=1 cargo test --release -p perry-runtime --lib`: 4010 passed. The 2 failures (gc::tests::copy_slot_decode::sabotaged_remembering_arm_is_refused_by_the_coverage_cross_check, gc::tests::heap_generation::a_free_or_move_outside_every_scope_is_caught_in_debug_builds) are pre-existing: reverted this change with `git checkout --` and reran just those two on the pristine parent commit — identical failures, same messages, confirming they assert debug_assert!-gated behavior compiled out under --release, unrelated to this change. - `cargo fmt --all -- --check`, `scripts/check_file_size.sh`, `scripts/check_test_registration.py`, `RUSTFLAGS="-D warnings" cargo check -p perry-runtime --all-targets`: all clean. --- .../src/gc/roots/shadow_stack.rs | 33 ++- scripts/thread_local_cold_allowlist.json | 2 +- .../test_gap_try_entry_shadow_hot_tls.ts | 218 ++++++++++++++++++ 3 files changed, 247 insertions(+), 6 deletions(-) create mode 100644 test-files/test_gap_try_entry_shadow_hot_tls.ts diff --git a/crates/perry-runtime/src/gc/roots/shadow_stack.rs b/crates/perry-runtime/src/gc/roots/shadow_stack.rs index a8a8f28d61..2cc83225dd 100644 --- a/crates/perry-runtime/src/gc/roots/shadow_stack.rs +++ b/crates/perry-runtime/src/gc/roots/shadow_stack.rs @@ -207,11 +207,34 @@ impl ShadowStackState { } } -thread_local! { - /// `const`-initialized and drop-free, so the access is a plain TLS address - /// computation with no lazy-init or destructor-registration check. The - /// buffer is reserved lazily on the first push instead of eagerly at thread - /// start, and released by [`ShadowBufferGuard`]. +crate::perry_thread_local! { + /// `const`-initialized and drop-free, so the access has no lazy-init or + /// destructor-registration check. The buffer is reserved lazily on the + /// first push instead of eagerly at thread start, and released by + /// [`ShadowBufferGuard`]. + /// + /// Was a raw `thread_local!` until #try-entry-perf: every `try`/`catch` + /// pays this exact resolution once per entry, unconditionally, through + /// `shadow_stack_savepoint()` — and `SHADOW_FRAMES` (the latch that lets + /// most other savepoint fields skip their own thread-local read, see + /// `crate::exception::savepoints`) is set by the FIRST shadow-frame push + /// anywhere in the process, which for real programs is essentially + /// "immediately" (any function with a pointer-typed local pushes one, and + /// a caught exception's own binding needs a slot). So the latch does not + /// make this read rare in practice, and profiling a `try`/`catch` loop + /// (`node --experimental-strip-types`-verified against `test-files/`) + /// showed it as a genuine leaf `_tlv_get_addr` call, ~6% of total + /// instructions retired per non-throwing `try` entry — the one savepoint + /// field NOT already routed through `tls_hot`'s cache (`EXCEPTION_STATE`, + /// `CALL_METHOD_DEPTH` and the named `runtime_handle_stack`/`temp_roots` + /// fields all were). Moving to `perry_thread_local!` is a pure + /// storage-mechanism swap — same type, same `.with()`/`.try_with()` + /// call sites below, same const-init/drop-free semantics (needs_drop is + /// still false, so no destructor is registered) — so it changes nothing + /// about liveness, restore-on-throw, or the address-stability contract + /// [`js_shadow_frame_enter`] depends on (the value still lives at a + /// fixed, never-reallocated thread-local address for the thread's whole + /// lifetime). pub(crate) static SHADOW: UnsafeCell = const { UnsafeCell::new(ShadowStackState { ptr: std::ptr::null_mut(), diff --git a/scripts/thread_local_cold_allowlist.json b/scripts/thread_local_cold_allowlist.json index ad4f7ff13c..c4cc721aba 100644 --- a/scripts/thread_local_cold_allowlist.json +++ b/scripts/thread_local_cold_allowlist.json @@ -38,7 +38,7 @@ "crates/perry-runtime/src/gc/promote_in_place.rs": 11, "crates/perry-runtime/src/gc/roots/runtime_handles.rs": 2, "crates/perry-runtime/src/gc/roots/scan_mode.rs": 1, - "crates/perry-runtime/src/gc/roots/shadow_stack.rs": 2, + "crates/perry-runtime/src/gc/roots/shadow_stack.rs": 1, "crates/perry-runtime/src/gc/roots/temp_roots.rs": 1, "crates/perry-runtime/src/gc/scan_fallback.rs": 2, "crates/perry-runtime/src/gc/shape_install.rs": 1, diff --git a/test-files/test_gap_try_entry_shadow_hot_tls.ts b/test-files/test_gap_try_entry_shadow_hot_tls.ts new file mode 100644 index 0000000000..a2a59b222d --- /dev/null +++ b/test-files/test_gap_try_entry_shadow_hot_tls.ts @@ -0,0 +1,218 @@ +// Adversarial coverage for the `try`-entry perf change that moved `SHADOW` +// (`crates/perry-runtime/src/gc/roots/shadow_stack.rs`) from a raw +// `thread_local!` to `crate::perry_thread_local!` so `try_push_with_kind`'s +// `CatchSavepoint::capture()` stops paying a real `_tlv_get_addr` call on +// every non-throwing `try` entry. The change is a pure storage-mechanism +// swap (same type, same `.with()`/`.try_with()` call sites, same +// const-init/drop-free semantics), but the shadow stack IS the GC's precise +// root set for named locals and expression temporaries, and its savepoint is +// what a throw restores when unwinding skips frame-pop epilogues (#1830). +// This file exercises every shape that depends on that machinery staying +// exactly correct: a throw caught at the entry level, a throw several +// frames deep, a throw crossing a runtime helper callback, `finally` on both +// the normal and throwing paths, a nested `try` with an inner rethrow, a +// `catch` block that itself throws, deep `try` nesting, and — last — a +// non-throwing control so the fast (unchanged-behavior) path is checked too. +// Every caught value is read back AFTER GC-pressure allocation so a +// mis-rooted exception (swept, not just moved) would show up as wrong output +// rather than a crash. + +function churn(n: number): number { + const a: unknown[] = []; + for (let i = 0; i < n; i++) { + a.push({ i, s: "churn" + i, nested: { i } }); + } + return a.length; +} + +// 1. Throw caught at the same level. +function sameLevel(): string { + try { + throw new Error("same-level"); + } catch (e) { + churn(300); + return (e as Error).message; + } +} +console.log("sameLevel", sameLevel()); + +// 2. Throw from a nested function several frames down. +function deep4(): number { + throw new Error("deep4"); +} +function deep3(): number { + return deep4(); +} +function deep2(): number { + return deep3(); +} +function deep1(): number { + return deep2(); +} +function nestedFrames(): string { + try { + deep1(); + return "unreached"; + } catch (e) { + churn(300); + return (e as Error).message; + } +} +console.log("nestedFrames", nestedFrames()); + +// 3. Throw crossing a runtime helper (inside an Array.prototype.map +// callback — the throw unwinds through the runtime's own map trampoline). +function throughMap(): string { + const src = [1, 2, 3, 4, 5]; + try { + src.map((x) => { + if (x === 3) throw new Error("through-map " + x); + return x * 2; + }); + return "unreached"; + } catch (e) { + churn(300); + return (e as Error).message; + } +} +console.log("throughMap", throughMap()); + +// 4a. finally runs on the NORMAL (non-throwing) path. +let finallyNormalRuns = 0; +function finallyNormal(): number { + try { + return 42; + } finally { + finallyNormalRuns++; + } +} +console.log("finallyNormal", finallyNormal(), "ran", finallyNormalRuns); + +// 4b. finally runs on the THROWING path, then the exception still propagates. +let finallyThrowRuns = 0; +function finallyThrow(): number { + try { + throw new Error("finally-throw"); + } finally { + finallyThrowRuns++; + } +} +try { + finallyThrow(); + console.log("finallyThrow (WRONG): did not throw"); +} catch (e) { + churn(300); + console.log("finallyThrow caught", (e as Error).message, "ran", finallyThrowRuns); +} + +// 5. Nested try with an inner rethrow — the outer catch must see the SAME +// (rethrown) exception, with its shadow-rooted fields intact. +function nestedRethrow(): string { + try { + try { + const err: any = new Error("inner"); + err.tag = "rethrow-tag"; + throw err; + } catch (inner) { + churn(200); + throw inner; + } + } catch (outer) { + churn(200); + const e = outer as any; + return e.message + "/" + e.tag; + } +} +console.log("nestedRethrow", nestedRethrow()); + +// 6. A try whose catch itself throws — the ORIGINAL exception is replaced by +// the catch's own throw, and outer code must see the new one. +function catchThrows(): string { + try { + try { + throw new Error("first"); + } catch (e) { + churn(200); + throw new Error("from-catch:" + (e as Error).message); + } + } catch (e2) { + churn(200); + return (e2 as Error).message; + } +} +console.log("catchThrows", catchThrows()); + +// 7. Deep try nesting — an exception thrown at the bottom must unwind +// through every level, running each finally exactly once, in order, and +// land in the outermost catch with the shadow stack balanced afterward. +const finallyOrder: number[] = []; +function deepNestTry(levels: number): string { + if (levels === 0) { + throw new Error("deep-nest-bottom"); + } + try { + return deepNestTry(levels - 1); + } finally { + finallyOrder.push(levels); + } +} +try { + deepNestTry(25); + console.log("deepNestTry (WRONG): did not throw"); +} catch (e) { + churn(300); + console.log( + "deepNestTry caught", + (e as Error).message, + "finally count", + finallyOrder.length, + "finally order ok", + finallyOrder.every((v, i) => v === i + 1) + ); +} + +// A second, unrelated try AFTER the deep unwind: proves try_depth and the +// shadow stack were left exactly where they should be, not off by the +// number of levels just unwound. +function afterDeepUnwind(): string { + try { + throw new Error("after-deep-unwind"); + } catch (e) { + return (e as Error).message; + } +} +console.log("afterDeepUnwind", afterDeepUnwind()); + +// 8. Non-throwing control: the hot (unchanged-behavior) path. A tight loop +// of try/catch blocks that never throw, mixed with ones that do, so the +// fast entry/exit accounting can't drift relative to the slow throw path. +function controlLoop(): number { + let total = 0; + for (let i = 0; i < 200; i++) { + try { + total += i; + if (i % 37 === 0) { + try { + if (i % 74 === 0) throw new Error("control-inner " + i); + total += 1; + } catch (e) { + churn(20); + total -= 1; + } + } + } catch (e) { + total = -1; // never reached + } + } + return total; +} +console.log("controlLoop", controlLoop()); + +// Final sanity: try_depth is back at zero — a fresh top-level try still +// catches correctly after everything above. +try { + throw new Error("final-sanity"); +} catch (e) { + console.log("finalSanity", (e as Error).message); +} +console.log("done"); From b9c9ff91ef6dcf76b8b4999758cfee4eebe1f79d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ralph=20K=C3=BCpper?= Date: Fri, 18 Sep 2026 14:12:47 +0200 Subject: [PATCH 2/8] docs: changelog fragment for #10619 --- changelog.d/10619-shadow-stack-hot-tls.md | 27 +++++++++++++++++++++++ 1 file changed, 27 insertions(+) create mode 100644 changelog.d/10619-shadow-stack-hot-tls.md diff --git a/changelog.d/10619-shadow-stack-hot-tls.md b/changelog.d/10619-shadow-stack-hot-tls.md new file mode 100644 index 0000000000..45cb2a72e9 --- /dev/null +++ b/changelog.d/10619-shadow-stack-hot-tls.md @@ -0,0 +1,27 @@ +Give the shadow-stack root state (`SHADOW`) the `tls_hot` fast path. It was the +last savepoint field still declared as a raw `thread_local!`, so every +`CatchSavepoint::capture()` — once per `try` entry — paid `_tlv_get_addr` for +it on Darwin, while `EXCEPTION_STATE`, `CALL_METHOD_DEPTH` and the named +`runtime_handle_stack`/`temp_roots` fields were already routed through +`tls_hot`. Wave 1's "has any thread used this subsystem" latch does not make +the read rare here: it is set by the first shadow-frame push anywhere in the +process, and a `catch (e)` binding needs a shadow slot itself. + +Profiling pinned it to one call site, `try_push_with_kind+176 -> +_tlv_get_addr`, about 6% of instructions retired per entry. The change is a +storage-mechanism swap — same type, call sites, const-init and drop-free +semantics — so the address-stability contract `js_shadow_frame_enter` depends +on is unchanged. A non-throwing `try` entry goes 179.3 -> 164.8 instructions +(-8.1%), differenced within each binary with the loop control at the noise +floor in both arms. The probe understates it: `SHADOW` is read on many paths, +not only this one. + +Because the shadow stack is the GC's precise root set rather than bookkeeping, +the new fixture attacks rooting: a throw four frames down, a throw crossing +`Array.prototype.map`'s runtime trampoline, `finally` on both paths, nested +`try` with an inner rethrow, a `catch` that itself throws, and 25-level nesting +— every caught value read back after GC-pressure allocation. It plus four +existing exception/rooting fixtures ran under five GC-schedule seeds at +RATE=1 with from-space protection, all byte-identical to node, with +forced_collections=2566 / copying_minors=2566 / moved_objects=37000 confirming +the instrument was live. From 8685fe0178fa57f7fdec441fc00c04c814e01480 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ralph=20K=C3=BCpper?= Date: Sat, 19 Sep 2026 10:23:55 +0200 Subject: [PATCH 3/8] fix(runtime): cfg-split the shadow-stack hot-TLS path to Darwin aarch64 #10619 routed SHADOW through perry_thread_local! unconditionally for an ~8% try-entry win, measured on Darwin. CI's cargo-test job (Linux, debug profile) SIGSEGV'd on it (run 35374727647, job 105594641738); the crash was never pinned to a specific test or line (stdout block-buffering ate the FAILED line), and it did not reproduce on macOS with the Darwin path, on macOS with the Darwin path forced off, or on a Linux x86_64 VM matching the CI runner's triple through 1968+ of ~4013 tests. tls_hot.rs's own docs say the hot-cache shortcut only pays for itself on Darwin aarch64; everywhere else it is strictly more work than the raw thread_local! it replaced (an extra HOT resolution plus a slot indirection), win or no SIGSEGV. So SHADOW's declaration is now cfg-split: perry_thread_local! only under all(target_vendor = "apple", target_arch = "aarch64", target_pointer_width = "64"), a plain thread_local! (the pre-#10619 form) everywhere else. This keeps the measured win where it was measured, removes the pessimization elsewhere, and removes CI's only exposure to the unconfirmed SIGSEGV mechanism. The open investigation is tracked in #10709. Re-verified: debug and release perry-runtime lib tests, GC-stress with live copying minors and from-space quarantine, the gap fixture byte-identical to node 26.5.1, and a fresh differential probe showing the try-entry win survives on Darwin (162.2 -> 152.0 instructions/ entry, ~6.3%, bare-loop control ~1 instruction in both arms). --- changelog.d/10619-shadow-stack-hot-tls.md | 62 ++++++++++++ .../src/gc/roots/shadow_stack.rs | 99 ++++++++++++++----- scripts/thread_local_cold_allowlist.json | 2 +- 3 files changed, 135 insertions(+), 28 deletions(-) diff --git a/changelog.d/10619-shadow-stack-hot-tls.md b/changelog.d/10619-shadow-stack-hot-tls.md index 45cb2a72e9..38b345fd27 100644 --- a/changelog.d/10619-shadow-stack-hot-tls.md +++ b/changelog.d/10619-shadow-stack-hot-tls.md @@ -25,3 +25,65 @@ existing exception/rooting fixtures ran under five GC-schedule seeds at RATE=1 with from-space protection, all byte-identical to node, with forced_collections=2566 / copying_minors=2566 / moved_objects=37000 confirming the instrument was live. + +**Follow-up: CI's `cargo-test` job (Linux, debug profile) SIGSEGV'd on this +change** (run 35374727647, job 105594641738) — no `FAILED` line survived +(libtest's stdout is block-buffered under CI), so the crashing test was never +named. Confirmed this PR's own regression: `cargo-test` was clean on #10644, +#10647, #10650, #10651 against the same base. + +Reproduction, in the exact failing configuration +(`RUST_TEST_THREADS=1 cargo test -p perry-runtime --lib`, debug, no +`--release`), across every environment tried, came back clean: +- macOS arm64, the shipped Darwin `perry_thread_local!` path: 4013 passed. +- macOS arm64 with the Darwin `pthread`-TSD path forced off (`hot()` forced + onto the generic `hot_via_tls()` route every non-Darwin-aarch64 target + already uses): 4013 passed, 0 failed — this weakens, though doesn't + disprove, a re-entrancy theory in the generic path itself. +- Linux x86_64 (qemu-emulated Ubuntu VM on the CI-pinned + `nightly-2026-08-20` toolchain, matching `ubuntu-latest`'s triple): ran + clean through 1968+ of ~4013 tests in alphabetical order, well past the + async_hooks region where CI's log cuts off, before a later, unrelated + regex-replace stress test stalled under emulation (not a crash — verified + it runs in well under a second natively; almost certainly a qemu-specific + slowdown, not tied to this change). + +The SIGSEGV's mechanism was never pinned down — see #10709 for the full +writeup, including the two open hypotheses (re-entrancy in `tls_hot.rs`'s +`fill()`, which writes its last field specifically to guard against a +half-filled cache being *used* re-entrantly but does not stop `fill()` being +*called* again re-entrantly; and plain debug-profile stack depth on CI's +default thread, a signature this repo has hit before). Rather than let it +evaporate, it's tracked there and the change is narrowed instead: + +`SHADOW`'s declaration is now cfg-split — +`crate::perry_thread_local!` only under +`all(target_vendor = "apple", target_arch = "aarch64", target_pointer_width = "64")`, +a plain `thread_local!` (the pre-#10619 form) everywhere else. This isn't +only a hedge against the unconfirmed SIGSEGV: `tls_hot.rs`'s own module docs +already say the published-cache shortcut is Darwin-aarch64-specific, and +everywhere else "resolving a thread-local is already a fixed offset and the +extra cache indirection has no demonstrated benefit" — so routing `SHADOW` +through it unconditionally was strictly more work (one extra `HOT` +resolution plus a slot-array indirection) on every other target for a win +that was only ever measured on Darwin. `scripts/thread_local_cold_allowlist.json`'s +`shadow_stack.rs` count goes back to 2 (its pre-#10619 value) to match. + +Re-verified on this Darwin host post-split, differential probe (own builds, +base = this PR's parent 9df5075fbe, arm = this fix, both +`PERRY_NO_AUTO_OPTIMIZE=1`, median of 7, N=20000/40000): try-entry marginal +cost 162.2 -> 152.0 instructions/entry (-10.2, ~6.3%), bare-loop control +~1.0 instructions in both arms (noise floor relative to the ~150-instruction +signal) — the win the original commit measured (-8.1%) survives, because the +fix does not touch the Darwin code path at all. + +Also re-ran: debug and release `cargo test -p perry-runtime --lib` (release: +4010 passed, the same 2 pre-existing `debug_assert!`-gated failures noted +above); the gap fixture plus GC-stress (seeds 1 and 42, +`PERRY_GC_SCHEDULE_RATE=1 PERRY_GC_PROTECT_FROMSPACE=1 +PERRY_GC_VERIFY_EVACUATION=1 PERRY_GC_FROMSPACE_SCAN_ABORT=1`) both +byte-identical to node with non-zero copying minors +(`copying_minors=76 moved_objects=35258`) and a live +`[gc-fromspace-protect] retired_set=#75` line; `cargo fmt --all -- --check`; +`scripts/run_lint_gates.sh` (the only failure is the pre-existing, known-red +public-baseline step); `scripts/check_thread_locals.py`/`--self-test`. diff --git a/crates/perry-runtime/src/gc/roots/shadow_stack.rs b/crates/perry-runtime/src/gc/roots/shadow_stack.rs index 2cc83225dd..6c1b3c4555 100644 --- a/crates/perry-runtime/src/gc/roots/shadow_stack.rs +++ b/crates/perry-runtime/src/gc/roots/shadow_stack.rs @@ -207,34 +207,79 @@ impl ShadowStackState { } } +// [`SHADOW`]'s storage mechanism is cfg-split by platform. Both arms are +// `const`-initialized and drop-free, so neither pays a lazy-init or +// destructor-registration check; the buffer is reserved lazily on the first +// push instead of eagerly at thread start, and released by +// [`ShadowBufferGuard`], on either arm. +// +// # Why `perry_thread_local!` only on Apple aarch64 +// +// Until #10619, `SHADOW` was unconditionally a raw `thread_local!`: every +// `try`/`catch` paid this exact resolution once per entry, through +// `shadow_stack_savepoint()` — and `SHADOW_FRAMES` (the latch that lets most +// other savepoint fields skip their own thread-local read, see +// `crate::exception::savepoints`) is set by the FIRST shadow-frame push +// anywhere in the process, which for real programs is essentially +// "immediately" (any function with a pointer-typed local pushes one, and a +// caught exception's own binding needs a slot). So the latch does not make +// this read rare in practice, and profiling a `try`/`catch` loop +// (`node --experimental-strip-types`-verified against `test-files/`) showed +// it as a genuine leaf `_tlv_get_addr` call, ~6% of total instructions +// retired per non-throwing `try` entry — the one savepoint field NOT already +// routed through `tls_hot`'s cache (`EXCEPTION_STATE`, `CALL_METHOD_DEPTH` +// and the named `runtime_handle_stack`/`temp_roots` fields all were). +// +// But that fast path is itself Darwin-aarch64-specific: `tls_hot.rs`'s own +// module docs say the published-cache shortcut only exists there, and +// everywhere else "resolving a thread-local is already a fixed offset and +// the extra cache indirection has no demonstrated benefit." #10619 shipped +// the swap unconditionally anyway, and CI's `cargo-test` job — Linux, debug +// profile — hit a SIGSEGV that a plain `thread_local!` never produced +// (`cargo-test` run 35374727647, job 105594641738). The cause was never +// isolated to a specific line: the same debug suite ran clean on macOS +// through the Darwin `perry_thread_local!` path, and also ran clean on +// macOS with the Darwin `pthread`-TSD path forced off (so `hot()` took the +// generic `hot_via_tls()` route `perry_thread_local!` uses on every +// non-Darwin-aarch64 target) — only actual Linux/x86_64 reproduced it, and +// under qemu-emulated Linux the debug suite is roughly an order of +// magnitude slower than native, which made pinning the exact crashing test +// impractical here. See #10709 for the open investigation. +// +// Since the −8% `try`-entry win was only ever measured on Darwin +// (`perry_thread_local!`'s whole premise doesn't apply anywhere the direct +// TLS access is already a fixed offset — see `tls_hot.rs`), routing through +// it off that platform was strictly more work for no measured benefit even +// before the SIGSEGV: one more thread-local resolution (`HOT`) plus a +// slot-array indirection, replacing the single direct TLS access a raw +// `thread_local!` already was. So the fast path now ships only where it was +// measured, and every other target keeps the original raw form — which +// also means this file's `thread_local!` count goes back to what it was +// before #10619 (see `scripts/thread_local_cold_allowlist.json`). +#[cfg(all( + target_vendor = "apple", + target_arch = "aarch64", + target_pointer_width = "64" +))] crate::perry_thread_local! { - /// `const`-initialized and drop-free, so the access has no lazy-init or - /// destructor-registration check. The buffer is reserved lazily on the - /// first push instead of eagerly at thread start, and released by - /// [`ShadowBufferGuard`]. - /// - /// Was a raw `thread_local!` until #try-entry-perf: every `try`/`catch` - /// pays this exact resolution once per entry, unconditionally, through - /// `shadow_stack_savepoint()` — and `SHADOW_FRAMES` (the latch that lets - /// most other savepoint fields skip their own thread-local read, see - /// `crate::exception::savepoints`) is set by the FIRST shadow-frame push - /// anywhere in the process, which for real programs is essentially - /// "immediately" (any function with a pointer-typed local pushes one, and - /// a caught exception's own binding needs a slot). So the latch does not - /// make this read rare in practice, and profiling a `try`/`catch` loop - /// (`node --experimental-strip-types`-verified against `test-files/`) - /// showed it as a genuine leaf `_tlv_get_addr` call, ~6% of total - /// instructions retired per non-throwing `try` entry — the one savepoint - /// field NOT already routed through `tls_hot`'s cache (`EXCEPTION_STATE`, - /// `CALL_METHOD_DEPTH` and the named `runtime_handle_stack`/`temp_roots` - /// fields all were). Moving to `perry_thread_local!` is a pure - /// storage-mechanism swap — same type, same `.with()`/`.try_with()` - /// call sites below, same const-init/drop-free semantics (needs_drop is - /// still false, so no destructor is registered) — so it changes nothing - /// about liveness, restore-on-throw, or the address-stability contract - /// [`js_shadow_frame_enter`] depends on (the value still lives at a - /// fixed, never-reallocated thread-local address for the thread's whole - /// lifetime). + /// See the cfg-split rationale above this declaration. + pub(crate) static SHADOW: UnsafeCell = const { + UnsafeCell::new(ShadowStackState { + ptr: std::ptr::null_mut(), + len: 0, + cap: 0, + frame_top: usize::MAX, + }) + }; +} + +#[cfg(not(all( + target_vendor = "apple", + target_arch = "aarch64", + target_pointer_width = "64" +)))] +thread_local! { + /// See the cfg-split rationale above this declaration's sibling arm. pub(crate) static SHADOW: UnsafeCell = const { UnsafeCell::new(ShadowStackState { ptr: std::ptr::null_mut(), diff --git a/scripts/thread_local_cold_allowlist.json b/scripts/thread_local_cold_allowlist.json index c4cc721aba..ad4f7ff13c 100644 --- a/scripts/thread_local_cold_allowlist.json +++ b/scripts/thread_local_cold_allowlist.json @@ -38,7 +38,7 @@ "crates/perry-runtime/src/gc/promote_in_place.rs": 11, "crates/perry-runtime/src/gc/roots/runtime_handles.rs": 2, "crates/perry-runtime/src/gc/roots/scan_mode.rs": 1, - "crates/perry-runtime/src/gc/roots/shadow_stack.rs": 1, + "crates/perry-runtime/src/gc/roots/shadow_stack.rs": 2, "crates/perry-runtime/src/gc/roots/temp_roots.rs": 1, "crates/perry-runtime/src/gc/scan_fallback.rs": 2, "crates/perry-runtime/src/gc/shape_install.rs": 1, From 95135f17102a7780a9badb420b9af8231719f69d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ralph=20K=C3=BCpper?= Date: Sat, 19 Sep 2026 11:07:15 +0200 Subject: [PATCH 4/8] docs(runtime): confirm the shadow-stack SIGSEGV was caused by #10619 Pushed the cfg-gated fix and read CI's own cargo-test job on the real failing runner: green (run 35433215970) where the unconditional perry_thread_local! swap was red (run 35374727647), same commit otherwise. That settles causation directly, superseding the local repro attempts (macOS both arms, qemu Linux) which all came back clean and were inconclusive on their own -- including a qemu-VM A/B whose two SIGKILLs were momentarily misread as a reproduced crash before being traced to an operator pkill -f self-match, not a fault. Also resolves why cargo-test stayed green on #10644/#10647/#10650/ #10651 against the same base: none of them touch shadow_stack.rs, and this PR was never merged to main, so their runs never contained the change at all. The internal mechanism inside tls_hot.rs's resolution path is still not understood; #10709 tracks that open half. This commit only updates the code comment and changelog to say plainly what is now confirmed versus what remains unknown. --- changelog.d/10619-shadow-stack-hot-tls.md | 43 ++++++++++++------- .../src/gc/roots/shadow_stack.rs | 21 +++++---- 2 files changed, 39 insertions(+), 25 deletions(-) diff --git a/changelog.d/10619-shadow-stack-hot-tls.md b/changelog.d/10619-shadow-stack-hot-tls.md index 38b345fd27..bc25f3b14e 100644 --- a/changelog.d/10619-shadow-stack-hot-tls.md +++ b/changelog.d/10619-shadow-stack-hot-tls.md @@ -29,32 +29,43 @@ the instrument was live. **Follow-up: CI's `cargo-test` job (Linux, debug profile) SIGSEGV'd on this change** (run 35374727647, job 105594641738) — no `FAILED` line survived (libtest's stdout is block-buffered under CI), so the crashing test was never -named. Confirmed this PR's own regression: `cargo-test` was clean on #10644, -#10647, #10650, #10651 against the same base. +named. `cargo-test` was clean on #10644, #10647, #10650, #10651 against the +same base — not a contradiction, once checked: none of those four PRs touch +`shadow_stack.rs`, and this PR was never merged to `main`, so their +`cargo-test` runs (branch merged with `main`) never contained this change to +begin with. Only this branch's own run ever exercised it, and that run was +red. Reproduction, in the exact failing configuration (`RUST_TEST_THREADS=1 cargo test -p perry-runtime --lib`, debug, no -`--release`), across every environment tried, came back clean: +`--release`), across every environment tried locally, came back clean and was +ultimately inconclusive: - macOS arm64, the shipped Darwin `perry_thread_local!` path: 4013 passed. - macOS arm64 with the Darwin `pthread`-TSD path forced off (`hot()` forced onto the generic `hot_via_tls()` route every non-Darwin-aarch64 target - already uses): 4013 passed, 0 failed — this weakens, though doesn't - disprove, a re-entrancy theory in the generic path itself. + already uses): 4013 passed, 0 failed. - Linux x86_64 (qemu-emulated Ubuntu VM on the CI-pinned `nightly-2026-08-20` toolchain, matching `ubuntu-latest`'s triple): ran clean through 1968+ of ~4013 tests in alphabetical order, well past the - async_hooks region where CI's log cuts off, before a later, unrelated - regex-replace stress test stalled under emulation (not a crash — verified - it runs in well under a second natively; almost certainly a qemu-specific - slowdown, not tied to this change). + async_hooks region where CI's log cuts off. qemu turned out to be unusable + as an instrument beyond that: one specific unrelated regex-replace stress + test is pathologically slow under emulation (fast natively), and an + attempted A/B there produced two SIGKILLs that were first misread as a + reproduced crash — they were an operator `pkill -f` self-matching its own + remote shell (a documented pitfall), not a fault. -The SIGSEGV's mechanism was never pinned down — see #10709 for the full -writeup, including the two open hypotheses (re-entrancy in `tls_hot.rs`'s -`fill()`, which writes its last field specifically to guard against a -half-filled cache being *used* re-entrantly but does not stop `fill()` being -*called* again re-entrantly; and plain debug-profile stack depth on CI's -default thread, a signature this repo has hit before). Rather than let it -evaporate, it's tracked there and the change is narrowed instead: +So local reproduction never settled it either way. Causation was confirmed +the direct way instead: pushing this cfg-gated fix and reading CI's own +`cargo-test` job on the exact failing runner — green (run 35433215970, job +105871415920), where the unconditional swap was red. The PR *is* what caused +the SIGSEGV; gating it off Linux/non-Darwin is what fixed it. The internal +mechanism inside `tls_hot.rs`'s resolution path is still not understood — +this is a fix by removing the exposure, not by finding the fault. See #10709 +for the open half of the investigation (`fill()`'s `temp_roots`-last +ordering guards against a half-filled cache being *used* re-entrantly, not +against `fill()` being *called* again re-entrantly, which remains a live +suspect). The change is narrowed rather than left as a mystery with no +mitigation: `SHADOW`'s declaration is now cfg-split — `crate::perry_thread_local!` only under diff --git a/crates/perry-runtime/src/gc/roots/shadow_stack.rs b/crates/perry-runtime/src/gc/roots/shadow_stack.rs index 6c1b3c4555..4d247489d6 100644 --- a/crates/perry-runtime/src/gc/roots/shadow_stack.rs +++ b/crates/perry-runtime/src/gc/roots/shadow_stack.rs @@ -236,15 +236,18 @@ impl ShadowStackState { // the extra cache indirection has no demonstrated benefit." #10619 shipped // the swap unconditionally anyway, and CI's `cargo-test` job — Linux, debug // profile — hit a SIGSEGV that a plain `thread_local!` never produced -// (`cargo-test` run 35374727647, job 105594641738). The cause was never -// isolated to a specific line: the same debug suite ran clean on macOS -// through the Darwin `perry_thread_local!` path, and also ran clean on -// macOS with the Darwin `pthread`-TSD path forced off (so `hot()` took the -// generic `hot_via_tls()` route `perry_thread_local!` uses on every -// non-Darwin-aarch64 target) — only actual Linux/x86_64 reproduced it, and -// under qemu-emulated Linux the debug suite is roughly an order of -// magnitude slower than native, which made pinning the exact crashing test -// impractical here. See #10709 for the open investigation. +// (`cargo-test` run 35374727647, job 105594641738). Local reproduction (macOS +// on both the Darwin path and with it forced off, and a qemu-emulated Linux +// x86_64 VM) never faulted, so causation was confirmed the direct way +// instead: `cargo-test` on CI itself, gating `SHADOW` back to this cfg split, +// came back green (run 35433215970, job 105871415920) at the same commit +// that was red with the swap unconditional — so the PR *is* what caused it. +// The internal mechanism is still not understood: this fixes it by removing +// the change from every platform where it had no benefit anyway, not by +// finding the fault inside `tls_hot.rs`'s resolution path. See #10709 for +// the open half of the investigation (`fill()`'s `temp_roots`-last ordering +// guards against a half-filled cache being *used* re-entrantly, not against +// `fill()` being *called* re-entrantly, which remains a live suspect). // // Since the −8% `try`-entry win was only ever measured on Darwin // (`perry_thread_local!`'s whole premise doesn't apply anywhere the direct From 40c7f41670bf6e44527f365a4c044c0480367e95 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ralph=20K=C3=BCpper?= Date: Sat, 19 Sep 2026 05:30:17 +0000 Subject: [PATCH 5/8] refactor(stdlib): remove dotenv native binding dotenv.parse(Buffer) -- the idiomatic dotenv.parse(fs.readFileSync(...)) -- returned 0 keys, silently. dotenv.config() reported no error but populated neither result.parsed nor process.env: a silent total no-op. Real npm dotenv matches Node exactly. Per #10678 (duplicate extern "C" exports across perry-ext-*/perry-stdlib pairs), this binding existed twice: crates/perry-ext-dotenv/ (the governance-tracked crate) and crates/perry-stdlib/src/dotenv.rs (a second, independent implementation behind the default-on bundled-dotenv feature, exporting the same js_dotenv_config/js_dotenv_config_path/js_dotenv_parse symbols). Removed both, the 2-entry NativeModSig dispatch block + its dedicated regression test in native_table/utils_crypto.rs, the js_dotenv_* FFI declarations, the well_known_bindings.toml entry, the "dotenv"/"dotenv/config" NATIVE_MODULES entries + manifest rows, the bundled-dotenv stdlib feature, 3 Android stubs, and "dotenv" from PERRY_NATIVE_EXTENSION_PACKAGES (that array makes the module walker skip a node_modules/dotenv/ tree entirely; with the binding gone, dotenv's real source needs to reach the walker like any other npm package). Six tests edited (not deleted) to keep testing their real subject rather than a removed registry entry: binding_faithfulness.rs's lookup_preserves_registered_subpaths_before_falling_back kept its mysql2/promise half, dropped the dotenv/config half; well_known.rs dropped "dotenv" from shipped_unproven_bindings_are_partial's array (kept nanoid/uuid, separate PRs), deleted dotenv_is_registered, and retargeted node_prefix_stripped_on_lookup from "dotenv" to "bcrypt" (generic node:-prefix-stripping logic, not dotenv-specific); deleted dotenv_parse_is_registered (api-manifest) and dotenv_parse_dispatches_to_native_impl_as_an_object (utils_crypto.rs) -- both were regression guards for the exact rows removed above; trimmed the two dotenv/config-only side-effect-only-module allowlists in unimplemented_api_check.rs and manifest_consistency.rs to &[]. Also fixed tests/release/packages/next-app-route/provider/stdlib/Cargo.toml (a standalone workspace with its own Cargo.lock, not a member of the main workspace, so cargo check --workspace never touches it) which referenced the now-deleted bundled-dotenv feature. Regenerated docs/api/perry.d.ts, docs/src/api/reference.md, and docs/src/native-libraries/governance.md. Updated workspace-architecture.json (workspace_members 83->82, externalize 33->32). --- Cargo.lock | 8 - Cargo.toml | 2 - crates/perry-api-manifest/src/entries.rs | 2 - .../perry-api-manifest/src/entries/part_1.rs | 22 --- crates/perry-api-manifest/src/lib.rs | 42 ----- .../lower_call/native_table/utils_crypto.rs | 59 ------ .../src/runtime_decls/stdlib_ffi/utilities.rs | 5 - .../tests/manifest_consistency.rs | 4 +- crates/perry-ext-dotenv/Cargo.toml | 19 -- crates/perry-ext-dotenv/src/lib.rs | 174 ------------------ .../tests/unimplemented_api_check.rs | 10 +- crates/perry-stdlib/Cargo.toml | 10 +- crates/perry-stdlib/src/dotenv.rs | 104 ----------- crates/perry-stdlib/src/lib.rs | 4 - crates/perry-ui-android/src/stdlib_stubs.rs | 12 -- .../collect_modules/binding_faithfulness.rs | 4 - crates/perry/src/commands/compile/resolve.rs | 3 +- .../perry/src/commands/compile/well_known.rs | 13 +- crates/perry/src/commands/stdlib_features.rs | 9 - crates/perry/well_known_bindings.toml | 25 --- docs/src/native-libraries/governance.md | 1 - .../next-app-route/provider/stdlib/Cargo.toml | 1 - workspace-architecture.json | 9 +- 23 files changed, 11 insertions(+), 531 deletions(-) delete mode 100644 crates/perry-ext-dotenv/Cargo.toml delete mode 100644 crates/perry-ext-dotenv/src/lib.rs delete mode 100644 crates/perry-stdlib/src/dotenv.rs diff --git a/Cargo.lock b/Cargo.lock index 1fbb5e01ea..94206ef31f 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -5914,14 +5914,6 @@ dependencies = [ "rust_decimal", ] -[[package]] -name = "perry-ext-dotenv" -version = "0.5.1605" -dependencies = [ - "perry-ffi", - "serde_json", -] - [[package]] name = "perry-ext-ethers" version = "0.5.1605" diff --git a/Cargo.toml b/Cargo.toml index f5a536bf9d..da91a3a897 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -11,7 +11,6 @@ members = [ "crates/perry-runtime", "crates/perry-ffi", "crates/perry-native-registration", - "crates/perry-ext-dotenv", "crates/perry-ext-nanoid", "crates/perry-ext-bcrypt", "crates/perry-ext-argon2", @@ -472,7 +471,6 @@ perry-dispatch = { path = "crates/perry-dispatch" } perry-runtime = { path = "crates/perry-runtime", version = "0.5.1011", default-features = false } perry-ffi = { path = "crates/perry-ffi", version = "0.5.1011" } perry-native-registration = { path = "crates/perry-native-registration", version = "0.5.1534" } -perry-ext-dotenv = { path = "crates/perry-ext-dotenv" } perry-ext-nanoid = { path = "crates/perry-ext-nanoid" } perry-ext-bcrypt = { path = "crates/perry-ext-bcrypt" } perry-ext-argon2 = { path = "crates/perry-ext-argon2" } diff --git a/crates/perry-api-manifest/src/entries.rs b/crates/perry-api-manifest/src/entries.rs index 7ea8ac5320..53e9dbe339 100644 --- a/crates/perry-api-manifest/src/entries.rs +++ b/crates/perry-api-manifest/src/entries.rs @@ -43,8 +43,6 @@ pub const NATIVE_MODULES: &[&str] = &[ "ws", // WebSocket client/server "zlib", // (Node builtin) gzip/deflate/brotli/zstd compression "crypto", // (Node builtin) hashing, HMAC, cipher, sign/verify, WebCrypto - "dotenv", // .env file loader - "dotenv/config", // dotenv's auto-load-on-import subpath "nanoid", // compact URL-safe ID generation "ethers", // Ethereum library (utils/wallet/ABI) "mongodb", // MongoDB driver diff --git a/crates/perry-api-manifest/src/entries/part_1.rs b/crates/perry-api-manifest/src/entries/part_1.rs index 2b801b3cf7..9dc26c00a7 100644 --- a/crates/perry-api-manifest/src/entries/part_1.rs +++ b/crates/perry-api-manifest/src/entries/part_1.rs @@ -1111,28 +1111,6 @@ pub(crate) const API_MANIFEST_PART_1: &[ApiEntry] = &[ ), method("nodemailer", "sendMail", true, None), method("nodemailer", "verify", true, None), - method_sig("dotenv", "config", false, None, &[], TypeSpec::Any), - // `dotenv.parse(src)` — the native impl (`js_dotenv_parse`) has shipped - // since the module was added, but the manifest never registered the - // symbol, so the #463 gate compiled every call site to a deferred - // throw-on-reach error. Callers that wrap config loading in - // `try { … } catch {}` swallowed that throw and silently got no config - // at all, which is why this is registered as a data-loss fix, not a - // missing-feature one. The extern returns a JSON string; the dispatch - // row's `NR_OBJ_FROM_JSON_STR` pipes it through `js_json_parse` so the - // user-visible value is a real object. - method_sig( - "dotenv", - "parse", - false, - None, - &[ParamSpec::Named { - name: "src", - ty: TypeSpec::String, - optional: false, - }], - TypeSpec::Any, - ), method_sig( "nanoid", "nanoid", diff --git a/crates/perry-api-manifest/src/lib.rs b/crates/perry-api-manifest/src/lib.rs index 7a29b3d943..c9f0443985 100644 --- a/crates/perry-api-manifest/src/lib.rs +++ b/crates/perry-api-manifest/src/lib.rs @@ -597,48 +597,6 @@ mod tests { assert!(matches!(arena.kind, ApiKind::Property)); } - /// `dotenv.parse` regression guard. - /// - /// `js_dotenv_parse` has always been implemented and declared to codegen, - /// but the manifest only ever registered `dotenv.config`. That gap made - /// the #463 unimplemented-API gate fire for every `dotenv.parse(...)` - /// call site, which under the default (defer) policy compiles to a - /// throw-on-reach runtime error rather than a build failure. Callers that - /// load config inside `try { … } catch {}` — the common shape — swallowed - /// the throw and silently ran with no configuration at all. - #[test] - fn dotenv_parse_is_registered() { - let entry = module_has_symbol("dotenv", "parse") - .expect("dotenv.parse must be in the manifest — see js_dotenv_parse"); - assert!( - matches!( - entry.kind, - ApiKind::Method { - has_receiver: false, - class_filter: None - } - ), - "dotenv.parse must be a static module method, got {:?}", - entry.kind - ); - assert_eq!( - entry.params.len(), - 1, - "dotenv.parse takes exactly the source text" - ); - assert!( - matches!( - entry.params[0], - ParamSpec::Named { - ty: TypeSpec::String, - .. - } - ), - "dotenv.parse's argument is the .env source string, got {:?}", - entry.params[0] - ); - } - #[test] fn buffer_inspect_max_bytes_is_manifest_property() { let entry = module_has_symbol("node:buffer", "INSPECT_MAX_BYTES") diff --git a/crates/perry-codegen/src/lower_call/native_table/utils_crypto.rs b/crates/perry-codegen/src/lower_call/native_table/utils_crypto.rs index 96529d8cd8..4b41aa587c 100644 --- a/crates/perry-codegen/src/lower_call/native_table/utils_crypto.rs +++ b/crates/perry-codegen/src/lower_call/native_table/utils_crypto.rs @@ -29,30 +29,6 @@ pub(super) const UTILS_CRYPTO_ROWS: &[NativeModSig] = &[ args: &[], ret: NR_GCPTR, }, - // ========== dotenv ========== - NativeModSig { - module: "dotenv", - has_receiver: false, - method: "config", - class_filter: None, - runtime: "js_dotenv_config", - args: &[], - ret: NR_F64, - }, - // `dotenv.parse(src)` → the JSON string `js_dotenv_parse` builds, piped - // through `js_json_parse` by NR_OBJ_FROM_JSON_STR so TypeScript sees a - // real object (`{ FOO: "bar" }`), not the encoded string. Without this - // row the symbol fell through the #463 gate to a deferred runtime throw - // even though the native implementation was already linked in. - NativeModSig { - module: "dotenv", - has_receiver: false, - method: "parse", - class_filter: None, - runtime: "js_dotenv_parse", - args: &[NA_STR], - ret: NR_OBJ_FROM_JSON_STR, - }, // ========== nanoid ========== // js_nanoid_sized(NaN) → size=0 → falls back to js_nanoid() (21-char default), // so nanoid() and nanoid(N) both route through the same entry safely. @@ -243,38 +219,3 @@ pub(super) const UTILS_CRYPTO_ROWS: &[NativeModSig] = &[ ret: NR_VOID, }, ]; - -#[cfg(test)] -mod tests { - use super::*; - - /// `dotenv.parse` must dispatch to the native implementation and return a - /// real object. - /// - /// `js_dotenv_parse` was declared to codegen and linked into every binary, - /// but had no dispatch row, so the #463 gate compiled each call site to a - /// deferred throw-on-reach error. `readConfigFile()`-shaped callers wrap - /// the call in `try { … } catch {}`, so the throw was swallowed and the - /// `.env` config silently never loaded. - /// - /// The return kind matters as much as the row: `js_dotenv_parse` hands back - /// a JSON *string*, so only `NR_OBJ_FROM_JSON_STR` (which pipes it through - /// `js_json_parse`) makes `dotenv.parse(src).FOO` read a property instead - /// of indexing a string. - #[test] - fn dotenv_parse_dispatches_to_native_impl_as_an_object() { - let row = UTILS_CRYPTO_ROWS - .iter() - .find(|r| r.module == "dotenv" && r.method == "parse") - .expect("dotenv.parse needs a dispatch row"); - assert_eq!(row.runtime, "js_dotenv_parse"); - assert!(!row.has_receiver); - assert_eq!(row.class_filter, None); - assert!(matches!(row.args, [NativeArgKind::StrPtr])); - assert!( - matches!(row.ret, NativeRetKind::ObjFromJsonStr), - "dotenv.parse must be JSON-decoded into an object, got {:?}", - row.ret - ); - } -} diff --git a/crates/perry-codegen/src/runtime_decls/stdlib_ffi/utilities.rs b/crates/perry-codegen/src/runtime_decls/stdlib_ffi/utilities.rs index 417c9b491e..cf739da7f8 100644 --- a/crates/perry-codegen/src/runtime_decls/stdlib_ffi/utilities.rs +++ b/crates/perry-codegen/src/runtime_decls/stdlib_ffi/utilities.rs @@ -35,11 +35,6 @@ pub(crate) fn declare_utilities(module: &mut LlModule) { module.declare_function("js_commander_required_option", I64, &[I64, I64, I64, I64]); module.declare_function("js_commander_version", I64, &[I64, I64]); - // ========== Dotenv ========== - module.declare_function("js_dotenv_config", DOUBLE, &[]); - module.declare_function("js_dotenv_config_path", DOUBLE, &[I64]); - module.declare_function("js_dotenv_parse", I64, &[I64]); - // ========== Date libs (dayjs/datefns/moment) ========== module.declare_function("js_datefns_add_days", DOUBLE, &[DOUBLE, DOUBLE]); module.declare_function("js_datefns_add_months", DOUBLE, &[DOUBLE, DOUBLE]); diff --git a/crates/perry-codegen/tests/manifest_consistency.rs b/crates/perry-codegen/tests/manifest_consistency.rs index a0ee87a780..043a61cf82 100644 --- a/crates/perry-codegen/tests/manifest_consistency.rs +++ b/crates/perry-codegen/tests/manifest_consistency.rs @@ -199,7 +199,7 @@ fn every_native_module_has_at_least_one_manifest_entry() { /// allowed list documents the exception so a future module that /// genuinely lacks coverage doesn't sneak past CI by being added /// here. - const SIDE_EFFECT_ONLY: &[&str] = &["dotenv/config"]; + const SIDE_EFFECT_ONLY: &[&str] = &[]; let mut missing: Vec<&'static str> = Vec::new(); for &module in perry_api_manifest::NATIVE_MODULES { @@ -267,7 +267,7 @@ fn cjs_style_node_builtins_have_default_entries() { /// the sibling test above and excluded here too. #[test] fn every_well_known_binding_has_manifest_entry() { - const SIDE_EFFECT_ONLY: &[&str] = &["dotenv/config"]; + const SIDE_EFFECT_ONLY: &[&str] = &[]; // Inline parse of well_known_bindings.toml — small enough that // pulling in `toml` as a dev-dep just for this test would be diff --git a/crates/perry-ext-dotenv/Cargo.toml b/crates/perry-ext-dotenv/Cargo.toml deleted file mode 100644 index e0ed850eff..0000000000 --- a/crates/perry-ext-dotenv/Cargo.toml +++ /dev/null @@ -1,19 +0,0 @@ -[package] -name = "perry-ext-dotenv" -version.workspace = true -edition.workspace = true -license.workspace = true -description = "Native bindings for the npm `dotenv` package — wraps Rust's std env API behind the same `dotenv.config()` / `dotenv.parse()` surface that Node code uses. Acceptance test for the perry-ffi v0.5 surface (#466 Phase 1 / 5 step 1)." - -[lints] -workspace = true - -[lib] -crate-type = ["staticlib", "rlib"] - -[dependencies] -perry-ffi.workspace = true -serde_json.workspace = true - -[dev-dependencies] -perry-ffi = { workspace = true, features = ["runtime-link"] } diff --git a/crates/perry-ext-dotenv/src/lib.rs b/crates/perry-ext-dotenv/src/lib.rs deleted file mode 100644 index 3abd8675aa..0000000000 --- a/crates/perry-ext-dotenv/src/lib.rs +++ /dev/null @@ -1,174 +0,0 @@ -//! Native bindings for the npm `dotenv` package. -//! -//! Functionally identical to the implementation that lives in -//! `crates/perry-stdlib/src/dotenv.rs`. The point of this crate is -//! that it depends only on [`perry_ffi`], not on `perry-runtime` -//! internals — proving the perry-ffi v0.5 surface is sufficient for -//! a real wrapper. -//! -//! # Status -//! -//! Additive port (#466 Phase 5 step 1). The original -//! `perry-stdlib::dotenv` stays in place and is what compiled -//! programs link against today. Once a release ships and no -//! regressions surface, the well-known bindings table (#466 Phase 4) -//! flips `import 'dotenv'` resolution to point at this crate, and -//! the old code is deleted. - -use perry_ffi::{alloc_string, read_string, JsString, StringHeader}; -use std::collections::HashMap; -use std::fs; -use std::sync::Mutex; - -static DOTENV_LOADED: Mutex = Mutex::new(false); - -/// Parse a `.env` file's contents into key/value pairs. -/// -/// Implementation detail — exposed so the test crate can compare -/// against Node's parsing behavior. Not part of the FFI surface. -fn parse_dotenv_content(content: &str) -> HashMap { - let mut vars = HashMap::new(); - - for line in content.lines() { - let line = line.trim(); - - if line.is_empty() || line.starts_with('#') { - continue; - } - - if let Some(eq_pos) = line.find('=') { - let key = line[..eq_pos].trim().to_string(); - let mut value = line[eq_pos + 1..].trim().to_string(); - - if (value.starts_with('"') && value.ends_with('"')) - || (value.starts_with('\'') && value.ends_with('\'')) - { - value = value[1..value.len() - 1].to_string(); - } - - if value.contains("\\n") { - value = value.replace("\\n", "\n"); - } - if value.contains("\\t") { - value = value.replace("\\t", "\t"); - } - - vars.insert(key, value); - } - } - - vars -} - -/// `dotenv.config()` — load `.env` from CWD and apply to `std::env`. -#[no_mangle] -pub extern "C" fn js_dotenv_config() -> f64 { - // SAFETY: passing a null handle is documented input — the helper - // below treats it as "use default path .env". - unsafe { js_dotenv_config_path(std::ptr::null()) } -} - -/// `dotenv.config({ path })` — load `.env` from the given path. -/// -/// # Safety -/// -/// `path_ptr` must be either null (then `.env` is used) or a pointer -/// to a Perry-runtime-allocated `StringHeader`. Caller responsibility -/// is the same contract as any other `extern "C"` function in the -/// stdlib. -#[no_mangle] -pub unsafe extern "C" fn js_dotenv_config_path(path_ptr: *const StringHeader) -> f64 { - let path = if path_ptr.is_null() { - ".env".to_string() - } else { - let handle = JsString::from_raw(path_ptr as *mut StringHeader); - read_string(handle) - .map(|s| s.to_string()) - .unwrap_or_else(|| ".env".to_string()) - }; - - let content = match fs::read_to_string(&path) { - Ok(c) => c, - Err(_) => return 0.0, // missing file is not an error in dotenv - }; - - let vars = parse_dotenv_content(&content); - for (key, value) in vars { - // SAFETY: setting env vars before any thread reads them is - // the documented use of dotenv. Concurrent set_var from - // multiple threads is undefined behavior in std — but - // dotenv.config() runs once at module-init time. - unsafe { std::env::set_var(&key, &value) }; - } - - *DOTENV_LOADED.lock().unwrap() = true; - 1.0 -} - -/// `dotenv.parse(content)` — parse `.env`-formatted text into a JSON -/// string the runtime can pass back to TypeScript as an object. -/// -/// # Safety -/// -/// `content_ptr` must be null or a pointer to a Perry-runtime -/// `StringHeader`. -#[no_mangle] -pub unsafe extern "C" fn js_dotenv_parse(content_ptr: *const StringHeader) -> *mut StringHeader { - let handle = JsString::from_raw(content_ptr as *mut StringHeader); - let content = match read_string(handle) { - Some(c) => c, - None => return std::ptr::null_mut(), - }; - - let vars = parse_dotenv_content(content); - let json = serde_json::to_string(&vars).unwrap_or_else(|_| "{}".to_string()); - alloc_string(&json).as_raw() -} - -#[cfg(test)] -mod tests { - use super::parse_dotenv_content; - - #[test] - fn parses_basic_kv() { - let vars = parse_dotenv_content("FOO=bar\nBAZ=qux\n"); - assert_eq!(vars.get("FOO"), Some(&"bar".to_string())); - assert_eq!(vars.get("BAZ"), Some(&"qux".to_string())); - } - - #[test] - fn skips_comments_and_empty_lines() { - let vars = parse_dotenv_content("# comment\n\nFOO=bar\n# another\n"); - assert_eq!(vars.len(), 1); - assert_eq!(vars.get("FOO"), Some(&"bar".to_string())); - } - - #[test] - fn unwraps_quoted_values() { - let vars = parse_dotenv_content( - r#"DOUBLE="hello" -SINGLE='world' -ESCAPED="line1\nline2" -"#, - ); - assert_eq!(vars.get("DOUBLE"), Some(&"hello".to_string())); - assert_eq!(vars.get("SINGLE"), Some(&"world".to_string())); - assert_eq!(vars.get("ESCAPED"), Some(&"line1\nline2".to_string())); - } - - #[test] - fn round_trips_through_perry_ffi() { - // Allocate a fake .env content string via perry-ffi, run it - // through js_dotenv_parse, read the JSON back. Proves the - // wrapper's only contact with the runtime — string read + - // string alloc — survives end-to-end. - let content = perry_ffi::alloc_string("KEY=value\n# c\nOTHER=42\n"); - let json_handle = unsafe { super::js_dotenv_parse(content.as_raw() as *const _) }; - let json_handle_wrapped = unsafe { perry_ffi::JsString::from_raw(json_handle) }; - let json_str = - perry_ffi::read_string(json_handle_wrapped).expect("parse returned non-null"); - // serde_json hash-map order isn't guaranteed; check substrings. - assert!(json_str.contains("\"KEY\":\"value\""), "got: {}", json_str); - assert!(json_str.contains("\"OTHER\":\"42\""), "got: {}", json_str); - } -} diff --git a/crates/perry-hir/tests/unimplemented_api_check.rs b/crates/perry-hir/tests/unimplemented_api_check.rs index 62419f4129..733acd9aa9 100644 --- a/crates/perry-hir/tests/unimplemented_api_check.rs +++ b/crates/perry-hir/tests/unimplemented_api_check.rs @@ -367,10 +367,7 @@ fn perry_native_namespace_rejects_unknown_call_in_strict_mode() { /// no value binding to read properties off, so the gate doesn't apply. #[test] fn every_supported_module_rejects_bogus_member() { - const SKIP: &[&str] = &[ - // Side-effect-only — no value binding to access. - "dotenv/config", - ]; + const SKIP: &[&str] = &[]; let mut failures: Vec = Vec::new(); for &module in perry_api_manifest::NATIVE_MODULES { @@ -444,10 +441,7 @@ fn every_supported_module_rejects_bogus_member() { /// land at the rejection. #[test] fn every_supported_module_rejects_bogus_call() { - const SKIP: &[&str] = &[ - // Side-effect-only — no value binding to access. - "dotenv/config", - ]; + const SKIP: &[&str] = &[]; let mut failures: Vec = Vec::new(); for &module in perry_api_manifest::NATIVE_MODULES { diff --git a/crates/perry-stdlib/Cargo.toml b/crates/perry-stdlib/Cargo.toml index 57d1141486..0357575ca8 100644 --- a/crates/perry-stdlib/Cargo.toml +++ b/crates/perry-stdlib/Cargo.toml @@ -23,19 +23,11 @@ default = ["full"] # must stay out of this list: release archives enable `full` without linking # their per-program provider archives, and adding an external HTTP pump here # made HTTP-free Linux UI links require libperry_ext_http.a (#5983, #8587). -full = ["http-server", "http-client", "database", "crypto", "compression", "email", "websocket", "image", "scheduler", "ids", "html-parser", "rate-limit", "net", "tls", "bundled-dotenv", "bundled-lru-cache", "bundled-exponential-backoff", "bundled-events", "bundled-decimal", "bundled-dayjs", "bundled-moment", "bundled-commander", "bundled-streams"] +full = ["http-server", "http-client", "database", "crypto", "compression", "email", "websocket", "image", "scheduler", "ids", "html-parser", "rate-limit", "net", "tls", "bundled-lru-cache", "bundled-exponential-backoff", "bundled-events", "bundled-decimal", "bundled-dayjs", "bundled-moment", "bundled-commander", "bundled-streams"] # Minimal core - just what's needed for basic programs core = [] -# In-tree implementation of the npm `dotenv` package (#466 Phase 4 -# step 2). Default-on; turned off by the compiler when the -# well-known bindings table (`well_known_bindings.toml`) routes -# `import 'dotenv'` to `perry-ext-dotenv` so the link line doesn't -# end up with two copies of `_js_dotenv_*` symbols. Programs that -# don't import dotenv pay nothing for this either way. -bundled-dotenv = [] - # In-tree implementation of `lru-cache`. Default-on through # `default = ["full"]`; flipped to perry-ext-lru-cache by the # well-known table (#466 Phase 4). Pulls the `lru` crate dep so diff --git a/crates/perry-stdlib/src/dotenv.rs b/crates/perry-stdlib/src/dotenv.rs deleted file mode 100644 index fb17c93c27..0000000000 --- a/crates/perry-stdlib/src/dotenv.rs +++ /dev/null @@ -1,104 +0,0 @@ -//! Dotenv module (dotenv compatible) -//! -//! Native implementation of the 'dotenv' npm package. -//! Loads environment variables from .env files. - -use perry_runtime::{js_string_from_bytes, StringHeader}; -use std::collections::HashMap; -use std::fs; -use std::sync::Mutex; - -use crate::common::string_from_header; - -lazy_static::lazy_static! { - static ref DOTENV_LOADED: Mutex = Mutex::new(false); -} - -/// Parse a .env file content into key-value pairs -fn parse_dotenv_content(content: &str) -> HashMap { - let mut vars = HashMap::new(); - - for line in content.lines() { - let line = line.trim(); - - // Skip empty lines and comments - if line.is_empty() || line.starts_with('#') { - continue; - } - - // Find the first '=' to split key and value - if let Some(eq_pos) = line.find('=') { - let key = line[..eq_pos].trim().to_string(); - let mut value = line[eq_pos + 1..].trim().to_string(); - - // Remove surrounding quotes if present - if (value.starts_with('"') && value.ends_with('"')) - || (value.starts_with('\'') && value.ends_with('\'')) - { - value = value[1..value.len() - 1].to_string(); - } - - // Handle escape sequences in double-quoted strings - if value.contains("\\n") { - value = value.replace("\\n", "\n"); - } - if value.contains("\\t") { - value = value.replace("\\t", "\t"); - } - - vars.insert(key, value); - } - } - - vars -} - -/// Load .env file and set environment variables -/// dotenv.config() -> void -#[no_mangle] -pub extern "C" fn js_dotenv_config() -> f64 { - // SAFETY: We're passing a null pointer which is handled safely by js_dotenv_config_path - unsafe { js_dotenv_config_path(std::ptr::null()) } -} - -/// Load .env file from a specific path -/// dotenv.config({ path: '.env.local' }) -> void -#[no_mangle] -pub unsafe extern "C" fn js_dotenv_config_path(path_ptr: *const StringHeader) -> f64 { - let path = if path_ptr.is_null() { - ".env".to_string() - } else { - string_from_header(path_ptr).unwrap_or_else(|| ".env".to_string()) - }; - - // Read the file - let content = match fs::read_to_string(&path) { - Ok(c) => c, - Err(_) => return 0.0, // File not found is not an error in dotenv - }; - - // Parse and set environment variables - let vars = parse_dotenv_content(&content); - for (key, value) in vars { - std::env::set_var(&key, &value); - } - - *DOTENV_LOADED.lock().unwrap() = true; - 1.0 // Success -} - -/// Parse a string as dotenv format without setting env vars -/// dotenv.parse(content) -> object -#[no_mangle] -pub unsafe extern "C" fn js_dotenv_parse(content_ptr: *const StringHeader) -> *mut StringHeader { - let content = match string_from_header(content_ptr) { - Some(c) => c, - None => return std::ptr::null_mut(), - }; - - let vars = parse_dotenv_content(&content); - - // Return as JSON string (simple key-value object) - let json = serde_json::to_string(&vars).unwrap_or_else(|_| "{}".to_string()); - js_string_from_bytes(json.as_ptr(), json.len() as u32) -} diff --git a/crates/perry-stdlib/src/lib.rs b/crates/perry-stdlib/src/lib.rs index 8812fb2a15..7c4e9ed331 100644 --- a/crates/perry-stdlib/src/lib.rs +++ b/crates/perry-stdlib/src/lib.rs @@ -51,8 +51,6 @@ pub mod decimal; // without duplicate _js_dotenv_* symbols at link time. Default-on // preserves byte-identical behavior for programs that don't opt into // the well-known path. -#[cfg(feature = "bundled-dotenv")] -pub mod dotenv; // events feature-gated as of v0.5.546 so the well-known flip // can route to perry-ext-events. #[cfg(feature = "bundled-events")] @@ -101,8 +99,6 @@ pub use dayjs::*; #[cfg(feature = "bundled-decimal")] pub use decimal::*; pub use domain::*; -#[cfg(feature = "bundled-dotenv")] -pub use dotenv::*; #[cfg(feature = "bundled-events")] pub use events::*; #[cfg(feature = "bundled-exponential-backoff")] diff --git a/crates/perry-ui-android/src/stdlib_stubs.rs b/crates/perry-ui-android/src/stdlib_stubs.rs index 07eafc3afe..6da9579d68 100644 --- a/crates/perry-ui-android/src/stdlib_stubs.rs +++ b/crates/perry-ui-android/src/stdlib_stubs.rs @@ -508,18 +508,6 @@ pub extern "C" fn js_decimal_to_string() -> i64 { 0 } #[no_mangle] -pub extern "C" fn js_dotenv_config() -> i64 { - 0 -} -#[no_mangle] -pub extern "C" fn js_dotenv_config_path() -> i64 { - 0 -} -#[no_mangle] -pub extern "C" fn js_dotenv_parse() -> i64 { - 0 -} -#[no_mangle] pub extern "C" fn js_ethers_format_ether() -> i64 { 0 } diff --git a/crates/perry/src/commands/compile/collect_modules/binding_faithfulness.rs b/crates/perry/src/commands/compile/collect_modules/binding_faithfulness.rs index bc05d3cba6..8fcda7242c 100644 --- a/crates/perry/src/commands/compile/collect_modules/binding_faithfulness.rs +++ b/crates/perry/src/commands/compile/collect_modules/binding_faithfulness.rs @@ -108,10 +108,6 @@ mod tests { let (root, binding) = lookup_well_known_for_import("mysql2/promise"); assert_eq!(root, "mysql2"); assert_eq!(binding.expect("subpath binding").package, "mysql2/promise"); - - let (root, binding) = lookup_well_known_for_import("dotenv/config"); - assert_eq!(root, "dotenv"); - assert_eq!(binding.expect("root fallback").package, "dotenv"); } #[test] diff --git a/crates/perry/src/commands/compile/resolve.rs b/crates/perry/src/commands/compile/resolve.rs index 54d2d205c0..8c5280cca3 100644 --- a/crates/perry/src/commands/compile/resolve.rs +++ b/crates/perry/src/commands/compile/resolve.rs @@ -146,8 +146,7 @@ mod tests; // without the guard, a deep import reached through another package's // compiled JS would make the walker read undici's real sources (llhttp // wasm) instead of routing to perry-ext-undici. -const PERRY_NATIVE_EXTENSION_PACKAGES: &[&str] = - &["ioredis", "ethers", "mysql2", "ws", "dotenv", "undici"]; +const PERRY_NATIVE_EXTENSION_PACKAGES: &[&str] = &["ioredis", "ethers", "mysql2", "ws", "undici"]; /// Absolute virtual prefix used by files extracted from a Bun standalone /// executable. `--bunfs-root` maps the suffix below this prefix to a real diff --git a/crates/perry/src/commands/compile/well_known.rs b/crates/perry/src/commands/compile/well_known.rs index f23031ba8a..4307c829bc 100644 --- a/crates/perry/src/commands/compile/well_known.rs +++ b/crates/perry/src/commands/compile/well_known.rs @@ -369,13 +369,6 @@ mod tests { let _ = registry(); } - #[test] - fn dotenv_is_registered() { - let binding = lookup_well_known("dotenv").expect("dotenv must be a well-known binding"); - assert_eq!(binding.krate, "perry-ext-dotenv"); - assert_eq!(binding.lib, "perry_ext_dotenv"); - } - #[test] fn undici_is_registered() { let binding = lookup_well_known("undici").expect("undici must be a well-known binding"); @@ -385,8 +378,8 @@ mod tests { #[test] fn node_prefix_stripped_on_lookup() { - let bare = lookup_well_known("dotenv"); - let prefixed = lookup_well_known("node:dotenv"); + let bare = lookup_well_known("bcrypt"); + let prefixed = lookup_well_known("node:bcrypt"); assert!(bare.is_some()); assert!(prefixed.is_some()); } @@ -485,7 +478,7 @@ mod tests { #[test] fn shipped_unproven_bindings_are_partial() { - for name in ["dotenv", "nanoid"] { + for name in ["nanoid"] { let b = lookup_well_known(name).unwrap_or_else(|| panic!("{name} registered")); assert_eq!( b.compat, diff --git a/crates/perry/src/commands/stdlib_features.rs b/crates/perry/src/commands/stdlib_features.rs index ae8e640b0c..b1b39d6359 100644 --- a/crates/perry/src/commands/stdlib_features.rs +++ b/crates/perry/src/commands/stdlib_features.rs @@ -199,15 +199,6 @@ pub fn module_to_features(module: &str) -> &'static [&'static str] { // commander: feature-gated v0.5.555 — well-known flip routes // to perry-ext-commander. "commander" => &["bundled-commander"], - // dotenv was always-on through v0.5.532; gated behind - // `bundled-dotenv` from v0.5.533 onwards so the well-known - // bindings flip (#466 Phase 4 step 2) can swap perry-stdlib's - // copy out for `perry-ext-dotenv` without duplicate - // `_js_dotenv_*` symbols at link time. The well-known path - // strips this feature from the set; the default path leaves - // it on so byte-identical behavior is preserved. - "dotenv" | "dotenv/config" => &["bundled-dotenv"], - // readline (#347) — needs the async-runtime feature so the // event-loop pump tick drains its line / data / keypress // queues. Without async-runtime, `import readline` still diff --git a/crates/perry/well_known_bindings.toml b/crates/perry/well_known_bindings.toml index 0b537d3ae1..4721c63bc0 100644 --- a/crates/perry/well_known_bindings.toml +++ b/crates/perry/well_known_bindings.toml @@ -36,31 +36,6 @@ # requires every ext crate and package mapping to have an explicit decision # (#5716). -[bindings.dotenv] -crate = "perry-ext-dotenv" -# Library file name without the `lib` prefix or `.a` extension. -# Cargo derives this from the crate name by replacing `-` with `_`, -# but stating it explicitly here is documentation for humans -# inspecting the table. -lib = "perry_ext_dotenv" -# Tracking issue for the migration; surfaced in error messages -# when the bundled .a is missing at link time. -tracking = "#466" -# `compat` — how faithful this wrapper is to the npm package's public -# API (see `BindingCompat` in well_known.rs). `full` = audited complete -# drop-in, safe to auto-prefer over an on-disk node_modules copy. -# ABSENT ⇒ conservative `partial` default. dotenv remains partial: this -# wrapper has `config()` / `parse()`, but not the upstream decrypt/populate/ -# configDotenv surface and option/error semantics. -compat = "partial" - -[bindings.dotenv.upstream] -version = "17.4.2" -sha256 = "8648852be8209110b34dca75dcc3ed12ce7fae9fcc8edd1ef9e180e708af1398" -repo = "https://github.com/motdotla/dotenv" -ref = "a61f616a3160bb6e6f22ff55f08b7eba3a3fab68" -ported-at = "17.4.2" -date = "2026-07-30" [bindings.nanoid] crate = "perry-ext-nanoid" lib = "perry_ext_nanoid" diff --git a/docs/src/native-libraries/governance.md b/docs/src/native-libraries/governance.md index bc06dda3f5..3939cc0e72 100644 --- a/docs/src/native-libraries/governance.md +++ b/docs/src/native-libraries/governance.md @@ -93,7 +93,6 @@ from `well_known_bindings.toml`. Regenerate this table with | `perry-ext-cron` | `cron`
`node-cron` | Source package | Compile the upstream package source | Bundled; migration pending | | `perry-ext-dayjs` | `date-fns`
`dayjs` | Source package | Compile the upstream package source | Bundled; migration pending | | `perry-ext-decimal` | `bignumber.js`
`decimal.js` | Source package | Compile the upstream package source | Bundled; migration pending | -| `perry-ext-dotenv` | `dotenv` | Source package | Compile the upstream package source | Bundled; migration pending | | `perry-ext-ethers` | `ethers` | Source package | Compile the upstream package source | Bundled; migration pending | | `perry-ext-events` | `events` | Runtime API | Keep near core; consolidate when practical | Bundled; retained | | `perry-ext-exponential-backoff` | `exponential-backoff` | Source package | Compile the upstream package source | Bundled; migration pending | diff --git a/tests/release/packages/next-app-route/provider/stdlib/Cargo.toml b/tests/release/packages/next-app-route/provider/stdlib/Cargo.toml index 1523a800c4..89baefca64 100644 --- a/tests/release/packages/next-app-route/provider/stdlib/Cargo.toml +++ b/tests/release/packages/next-app-route/provider/stdlib/Cargo.toml @@ -18,7 +18,6 @@ perry-stdlib-core = { package = "perry-stdlib", path = "../../../../../../crates "ids", "html-parser", "rate-limit", - "bundled-dotenv", "bundled-lru-cache", "bundled-exponential-backoff", "bundled-decimal", diff --git a/workspace-architecture.json b/workspace-architecture.json index 240f5ba834..85139f6cfb 100644 --- a/workspace-architecture.json +++ b/workspace-architecture.json @@ -25,7 +25,7 @@ ] }, "baseline": { - "workspace_members": 78, + "workspace_members": 77, "default_dependency_closure": [ "perry", "perry-api-manifest", @@ -68,7 +68,7 @@ "perry-updater" ], "decision_counts": { - "externalize": 29, + "externalize": 28, "keep": 44, "merge": 1, "remove": 1, @@ -185,11 +185,6 @@ "decision": "externalize", "migration": "compile-source" }, - "perry-ext-dotenv": { - "category": "binding", - "decision": "externalize", - "migration": "compile-source" - }, "perry-ext-ethers": { "category": "binding", "decision": "externalize", From 35056cb51dd34877a76577149511efef853a5f74 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ralph=20K=C3=BCpper?= Date: Sat, 19 Sep 2026 05:30:44 +0000 Subject: [PATCH 6/8] changelog: add fragment for #10691 (dotenv native binding removal) --- .../10691-dotenv-native-binding-removal.md | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) create mode 100644 changelog.d/10691-dotenv-native-binding-removal.md diff --git a/changelog.d/10691-dotenv-native-binding-removal.md b/changelog.d/10691-dotenv-native-binding-removal.md new file mode 100644 index 0000000000..2cc68fa650 --- /dev/null +++ b/changelog.d/10691-dotenv-native-binding-removal.md @@ -0,0 +1,16 @@ +Removed the native `dotenv` binding: `dotenv.parse(Buffer)` (the idiomatic +`dotenv.parse(fs.readFileSync(...))`) returned 0 keys, and `config()` +reported no error but populated neither `result.parsed` nor `process.env` — +a silent total no-op. `import dotenv from "dotenv"` (no +`perry.compilePackages` entry) now compiles the real npm package from +source, matching Node exactly. + +Deleted both duplicate hand-written implementations +(`crates/perry-ext-dotenv` and `crates/perry-stdlib/src/dotenv.rs`, which +independently exported the same `js_dotenv_*` symbols — #10678) and removed +`"dotenv"` from `PERRY_NATIVE_EXTENSION_PACKAGES` so the real package's +source (including the `dotenv/config` auto-load subpath) reaches the module +walker instead of being skipped as "handled by native stdlib". Also fixed a +standalone-workspace release fixture +(`tests/release/packages/next-app-route/provider/stdlib/Cargo.toml`) that +referenced the now-deleted `bundled-dotenv` feature. From 30df06108d6234ff1b669107c7cdb9332392a7cb Mon Sep 17 00:00:00 2001 From: Perry Bot Date: Sat, 19 Sep 2026 13:52:21 +0000 Subject: [PATCH 7/8] docs: regenerate API reference + .d.ts after rebase onto main --- docs/api/perry.d.ts | 9 +-------- docs/src/api/reference.md | 10 +--------- 2 files changed, 2 insertions(+), 17 deletions(-) diff --git a/docs/api/perry.d.ts b/docs/api/perry.d.ts index 6497a7386b..47a24f494b 100644 --- a/docs/api/perry.d.ts +++ b/docs/api/perry.d.ts @@ -1,6 +1,6 @@ // Auto-generated from Perry's API manifest (#465). Do not edit by hand. // Source: perry-api-manifest::API_MANIFEST -// Coverage: 2067 entries across 132 modules +// Coverage: 2065 entries across 131 modules type PerryI8 = number & { readonly __perryI8?: never }; type PerryI16 = number & { readonly __perryI16?: never }; @@ -1525,13 +1525,6 @@ declare module "domain" { export function createDomain(...args: any[]): any; } -declare module "dotenv" { - /** stdlib */ - export function config(...args: any[]): any; - /** stdlib */ - export function parse(src: string): any; -} - declare module "ethers" { /** stdlib */ export function formatEther(p0: any): string; diff --git a/docs/src/api/reference.md b/docs/src/api/reference.md index 458ee6d20a..c0dd3c2f51 100644 --- a/docs/src/api/reference.md +++ b/docs/src/api/reference.md @@ -2,7 +2,7 @@ This page is auto-generated from Perry's compile-time API manifest (`perry-api-manifest::API_MANIFEST`). It is the source of truth for what `perry compile` accepts; references to symbols not listed here produce `R005 UnimplementedApi` (issue #463). Stubs (#464) are flagged ⚠ — they link cleanly but no-op at runtime on the chosen target. -Total: 3009 entries across 134 modules. +Total: 3007 entries across 133 modules. ## Modules @@ -47,7 +47,6 @@ Total: 3009 entries across 134 modules. - [`dns`](#dns) - [`dns/promises`](#dnspromises) - [`domain`](#domain) -- [`dotenv`](#dotenv) - [`ethers`](#ethers) - [`events`](#events) - [`exponential-backoff`](#exponential-backoff) @@ -1329,13 +1328,6 @@ Total: 3009 entries across 134 modules. - `active` - `members` -## `dotenv` - -### Methods - -- `config` — module -- `parse` — module - ## `ethers` ### Methods From df21f0dbb5b6c5dbe1898eba9243f9bf506aad2f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ralph=20K=C3=BCpper?= Date: Sat, 19 Sep 2026 23:07:19 +0200 Subject: [PATCH 8/8] chore: release merge train 227 as v0.5.1606 --- CLAUDE.md | 2 +- Cargo.lock | 150 ++++++++++++++++++++++++++--------------------------- Cargo.toml | 2 +- 3 files changed, 77 insertions(+), 77 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index e3e66f7353..71325fc631 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -8,7 +8,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co Perry is a native TypeScript compiler written in Rust that compiles TypeScript source code directly to native executables. It uses SWC for TypeScript parsing and LLVM for code generation. -**Current Version:** 0.5.1605 +**Current Version:** 0.5.1606 ## TypeScript Parity Status diff --git a/Cargo.lock b/Cargo.lock index 94206ef31f..37062ae16c 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -5623,7 +5623,7 @@ checksum = "1473d470930ed48574515a25df34900f3af89c6fa422d903e019121312a9f13e" [[package]] name = "perry" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "anyhow", "base64 0.22.1", @@ -5687,7 +5687,7 @@ dependencies = [ [[package]] name = "perry-api-manifest" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "perry-dispatch", "serde", @@ -5695,7 +5695,7 @@ dependencies = [ [[package]] name = "perry-audio-miniaudio" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "cc", "libc", @@ -5704,7 +5704,7 @@ dependencies = [ [[package]] name = "perry-codegen" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "aho-corasick", "anyhow", @@ -5721,7 +5721,7 @@ dependencies = [ [[package]] name = "perry-codegen-arkts" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "anyhow", "perry-hir", @@ -5729,7 +5729,7 @@ dependencies = [ [[package]] name = "perry-codegen-glance" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "anyhow", "perry-hir", @@ -5737,7 +5737,7 @@ dependencies = [ [[package]] name = "perry-codegen-js" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "anyhow", "perry-dispatch", @@ -5746,7 +5746,7 @@ dependencies = [ [[package]] name = "perry-codegen-swiftui" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "anyhow", "perry-hir", @@ -5754,7 +5754,7 @@ dependencies = [ [[package]] name = "perry-codegen-wasm" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "anyhow", "base64 0.22.1", @@ -5766,7 +5766,7 @@ dependencies = [ [[package]] name = "perry-codegen-wear-tiles" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "anyhow", "perry-hir", @@ -5774,7 +5774,7 @@ dependencies = [ [[package]] name = "perry-container-compose" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "async-trait", "clap", @@ -5798,14 +5798,14 @@ dependencies = [ [[package]] name = "perry-container-e2e" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "anyhow", ] [[package]] name = "perry-diagnostics" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "serde", "serde_json", @@ -5813,7 +5813,7 @@ dependencies = [ [[package]] name = "perry-dispatch" -version = "0.5.1605" +version = "0.5.1606" [[package]] name = "perry-doc-fixture-my-bindings" @@ -5824,7 +5824,7 @@ dependencies = [ [[package]] name = "perry-doc-tests" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "anyhow", "clap", @@ -5839,7 +5839,7 @@ dependencies = [ [[package]] name = "perry-ext-ads" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "block2", "objc2", @@ -5849,7 +5849,7 @@ dependencies = [ [[package]] name = "perry-ext-argon2" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "argon2", "perry-ffi", @@ -5858,7 +5858,7 @@ dependencies = [ [[package]] name = "perry-ext-bcrypt" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "bcrypt", "perry-ffi", @@ -5866,7 +5866,7 @@ dependencies = [ [[package]] name = "perry-ext-better-sqlite3" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "perry-ffi", "rusqlite", @@ -5874,7 +5874,7 @@ dependencies = [ [[package]] name = "perry-ext-cheerio" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "perry-ffi", "scraper", @@ -5882,7 +5882,7 @@ dependencies = [ [[package]] name = "perry-ext-commander" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "perry-ffi", "perry-runtime", @@ -5890,7 +5890,7 @@ dependencies = [ [[package]] name = "perry-ext-cron" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "chrono", "cron", @@ -5900,7 +5900,7 @@ dependencies = [ [[package]] name = "perry-ext-dayjs" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "chrono", "perry-ffi", @@ -5908,7 +5908,7 @@ dependencies = [ [[package]] name = "perry-ext-decimal" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "perry-ffi", "rust_decimal", @@ -5916,7 +5916,7 @@ dependencies = [ [[package]] name = "perry-ext-ethers" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "perry-ffi", "rand 0.10.2", @@ -5924,7 +5924,7 @@ dependencies = [ [[package]] name = "perry-ext-events" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "perry-ffi", "perry-runtime", @@ -5932,14 +5932,14 @@ dependencies = [ [[package]] name = "perry-ext-exponential-backoff" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "perry-ffi", ] [[package]] name = "perry-ext-fastify" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "bytes", "http-body-util", @@ -5956,7 +5956,7 @@ dependencies = [ [[package]] name = "perry-ext-fetch" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "bytes", "lazy_static", @@ -5969,7 +5969,7 @@ dependencies = [ [[package]] name = "perry-ext-http" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "base64 0.22.1", "bytes", @@ -6001,7 +6001,7 @@ dependencies = [ [[package]] name = "perry-ext-ioredis" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "lazy_static", "perry-ffi", @@ -6011,7 +6011,7 @@ dependencies = [ [[package]] name = "perry-ext-lru-cache" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "lru", "perry-ffi", @@ -6020,7 +6020,7 @@ dependencies = [ [[package]] name = "perry-ext-moment" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "chrono", "perry-ffi", @@ -6028,7 +6028,7 @@ dependencies = [ [[package]] name = "perry-ext-mongodb" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "bson", "futures-util", @@ -6040,7 +6040,7 @@ dependencies = [ [[package]] name = "perry-ext-mysql2" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "chrono", "perry-ffi", @@ -6052,7 +6052,7 @@ dependencies = [ [[package]] name = "perry-ext-nanoid" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "nanoid", "perry-ffi", @@ -6061,7 +6061,7 @@ dependencies = [ [[package]] name = "perry-ext-net" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "bytes", "perry-ffi", @@ -6076,7 +6076,7 @@ dependencies = [ [[package]] name = "perry-ext-node-forge" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "const-oid 0.10.2", "der 0.8.2", @@ -6095,7 +6095,7 @@ dependencies = [ [[package]] name = "perry-ext-nodemailer" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "lettre", "perry-ffi", @@ -6105,7 +6105,7 @@ dependencies = [ [[package]] name = "perry-ext-parcel-watcher" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "notify", "perry-ffi", @@ -6117,7 +6117,7 @@ dependencies = [ [[package]] name = "perry-ext-pdf" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "perry-ffi", "printpdf", @@ -6125,7 +6125,7 @@ dependencies = [ [[package]] name = "perry-ext-pg" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "perry-ffi", "sqlx", @@ -6134,7 +6134,7 @@ dependencies = [ [[package]] name = "perry-ext-qs" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "perry-ffi", "perry-runtime", @@ -6143,7 +6143,7 @@ dependencies = [ [[package]] name = "perry-ext-ratelimit" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "governor", "perry-ffi", @@ -6151,7 +6151,7 @@ dependencies = [ [[package]] name = "perry-ext-sharp" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "fast_image_resize", "image", @@ -6162,7 +6162,7 @@ dependencies = [ [[package]] name = "perry-ext-streams" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "lazy_static", "perry-ffi", @@ -6171,7 +6171,7 @@ dependencies = [ [[package]] name = "perry-ext-typescript" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "anyhow", "perry-ffi", @@ -6191,7 +6191,7 @@ dependencies = [ [[package]] name = "perry-ext-undici" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "perry-ffi", "perry-runtime", @@ -6200,7 +6200,7 @@ dependencies = [ [[package]] name = "perry-ext-ws" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "futures-util", "lazy_static", @@ -6213,7 +6213,7 @@ dependencies = [ [[package]] name = "perry-ext-zlib" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "brotli", "flate2", @@ -6223,7 +6223,7 @@ dependencies = [ [[package]] name = "perry-ffi" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "dashmap 6.2.1", "once_cell", @@ -6233,7 +6233,7 @@ dependencies = [ [[package]] name = "perry-hir" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "anyhow", "perry-api-manifest", @@ -6253,11 +6253,11 @@ dependencies = [ [[package]] name = "perry-native-registration" -version = "0.5.1605" +version = "0.5.1606" [[package]] name = "perry-parser" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "anyhow", "perry-diagnostics", @@ -6270,7 +6270,7 @@ dependencies = [ [[package]] name = "perry-perex" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "perex", "regex", @@ -6278,7 +6278,7 @@ dependencies = [ [[package]] name = "perry-runtime" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "ahash", "base64 0.22.1", @@ -6336,14 +6336,14 @@ dependencies = [ [[package]] name = "perry-runtime-static" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "perry-runtime", ] [[package]] name = "perry-stdlib" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "aes 0.8.4", "aes 0.9.1", @@ -6430,21 +6430,21 @@ dependencies = [ [[package]] name = "perry-stdlib-static" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "perry-stdlib", ] [[package]] name = "perry-transform" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "perry-hir", ] [[package]] name = "perry-ui" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "dirs", "perry-ffi", @@ -6454,7 +6454,7 @@ dependencies = [ [[package]] name = "perry-ui-android" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "base64 0.22.1", "jni", @@ -6469,7 +6469,7 @@ dependencies = [ [[package]] name = "perry-ui-geisterhand" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "rand 0.10.2", "serde", @@ -6479,7 +6479,7 @@ dependencies = [ [[package]] name = "perry-ui-gtk4" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "base64 0.22.1", "cairo-rs 0.22.9", @@ -6502,7 +6502,7 @@ dependencies = [ [[package]] name = "perry-ui-ios" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "base64 0.22.1", "block2", @@ -6519,7 +6519,7 @@ dependencies = [ [[package]] name = "perry-ui-macos" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "base64 0.22.1", "block2", @@ -6536,7 +6536,7 @@ dependencies = [ [[package]] name = "perry-ui-model" -version = "0.5.1605" +version = "0.5.1606" [[package]] name = "perry-ui-test" @@ -6547,11 +6547,11 @@ dependencies = [ [[package]] name = "perry-ui-testkit" -version = "0.5.1605" +version = "0.5.1606" [[package]] name = "perry-ui-tvos" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "base64 0.22.1", "block2", @@ -6568,7 +6568,7 @@ dependencies = [ [[package]] name = "perry-ui-visionos" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "base64 0.22.1", "block2", @@ -6585,7 +6585,7 @@ dependencies = [ [[package]] name = "perry-ui-watchos" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "block2", "libc", @@ -6599,7 +6599,7 @@ dependencies = [ [[package]] name = "perry-ui-windows" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "base64 0.22.1", "libc", @@ -6618,7 +6618,7 @@ dependencies = [ [[package]] name = "perry-ui-windows-winui" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "base64 0.22.1", "libc", @@ -6631,7 +6631,7 @@ dependencies = [ [[package]] name = "perry-updater" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "anyhow", "base64 0.22.1", @@ -6646,7 +6646,7 @@ dependencies = [ [[package]] name = "perry-wasm-host" -version = "0.5.1605" +version = "0.5.1606" dependencies = [ "wasmi", ] diff --git a/Cargo.toml b/Cargo.toml index da91a3a897..766712ef22 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -332,7 +332,7 @@ codegen-units = 1 codegen-units = 1 [workspace.package] -version = "0.5.1605" +version = "0.5.1606" edition = "2021" license = "MIT" repository = "https://github.com/PerryTS/perry"