Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
140 changes: 140 additions & 0 deletions benchmarks/catch_frames/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,140 @@
# Catch-frame setup

Grouping the per-depth exception state and inlining its savepoint providers
reduces non-throwing setup by 121 instructions per runtime catch in this probe.
The existing C trampoline still owns `setjmp`. Generated user `try`/`catch`
calls `js_eh_try_push` in `perry-codegen/src/stmt/try_stmt.rs` and shares the
same snapshot capture, although its exception transport uses the system unwinder.

## Design and correctness

`exception::TryFrame` stores the jump buffer, handler kind and `CatchSavepoint`
in one fixed, heap-allocated slab. A depth indexes one allocation instead of
separate boxed arrays. The slab never grows, so jump-buffer addresses remain
stable. The 1,024-handler limit and try-depth accounting are unchanged.

`exception/savepoints.rs` declares each snapshot field together with its
capture provider, restore provider and required real-throw witness. The macro
generates storage, capture, ordered restoration and the test. This removes
the independently maintained lists, including the incomplete legacy test
replay. The small capture providers inline so the optimizer can reuse hot
TLS accesses; live subsystem state stays with its current owner. This is an
incremental grouping change, without a journal or additional live TLS mirrors.

Every existing restore remains, in its original order. Async-context cleanup
still runs first, using the unchanged handler depth. No callers lose their
catch frames. A future frame-elision optimization needs separate evidence
that its protected operation cannot throw.

The generated tests establish nonempty state outside a trap, change it again
inside two nested traps, and throw through `js_throw` and the C trampoline.
They require both catches to recover exactly their enclosing state, and also
verify that normal completion does not replay cleanup. The shadow witness
changes both shadow frames and expression temp roots; the dyn-eval witness
changes both interpreter roots and call depth.

`fault_restore.py` temporarily omits one registered restore at a time in the
production path. All 13 omissions fail their corresponding test at the
enclosing-state assertion; results are in `evidence/fault-results.json`.
Each mutant runs in a separate process. The expected assertion aborts across
the C callback boundary, so its exit is SIGABRT. An unrelated crash or an empty
test filter does not count as detection. The script restores and rebuilds the
original source in `finally`, then requires the exception suite to pass.

## Instruction measurements, 2026-09-14

Baseline: main `eb13fa188d` (0.5.1564), which already includes the #10215
large-array corruption fix. Host: perrymaster, AMD Ryzen 7 7700X, Linux
6.17.0-23-generic, perf 6.17.13, Rust nightly 2026-08-20
(`1.100.0-nightly f7d782a3b`), LLVM 22. Node oracle: 26.5.1.

Before changing runtime code, the promise/async workload pushed 100,007 frames.
Its initial instruction profile attributed 96 of 4,221 samples (2.27%) to
capture. A later independent baseline sample attributed 29 of 1,098 (2.64%).
The JSON allocation workload pushed only six frames, making it a useful
negative control. The hoisted regex test pushed 3,500,011 frames. Uprobe counts
were collected separately, with the probe removed before performance runs.

| Workload | Baseline instructions | Candidate instructions | Change |
|---|---:|---:|---:|
| Promise/async loop, 100k | 1,098,667,245 | 1,077,465,852 | -1.93% |
| JSON parse/stringify loop, 100k | 3,182,841,469 | 3,171,165,684 | -0.37% |
| Hoisted regex `test`, 1M | 13,816,754,889 | 13,266,297,479 | -3.98% |
| Regex `exec`, 1M | 31,217,404,185 | 30,183,325,430 | -3.31% |
| Runtime catch microprobe, 1M | 472,151,423 | 351,152,420 | -25.63% |
| Generated-handler setup microprobe, 1M | 366,173,508 | 244,175,238 | -33.32% |
| Plain microprobe control, 1M | 27,068,237 | 27,068,989 | +0.00% |

Subtracting each build's plain loop gives **445.08 → 324.08 instructions per
runtime catch (-27.19%)**, and **339.11 → 217.11 per generated-handler setup
(-35.98%)**. These include frame entry and exit; they do not measure throwing.
The Rust example has its own Cargo link graph, so its absolute per-frame cost
should not be substituted into compiled-program profiles.

All four TypeScript outputs match Node. The regex probes are copied from
`/root/claude-regex/repro/tonly/{hoist,exec1}.ts`. No cc application claim is
made. The small JSON delta cannot be attributed to its six catch frames:
provider inlining can also change other call sites and compiler output.

`evidence/instructions.json` records totals, sampled attribution, frame counts
and archive/compiler hashes. Raw stat outputs and gzip-compressed folded
stacks are alongside it. Symbol attribution is approximate under LTO and
sampling. The verdict uses instruction totals. All retained A/B folded stacks
use `perf script --no-inline` consistently. Raw perf data and build logs are
retained under `/root/js-throw-evidence` on perrymaster.

Integration exposed a Thin LTO identity hazard: separately linked archives
could give `global_this_builtin_noop_thunk` different addresses, breaking
builtin constructor recognition. The final patch gives this address sentinel
one external symbol and a distinct non-inlined body. The unguarded optimized
build fails the EventTarget subclass fixture; changing only its constructor
pointer to the comparison target in GDB makes that fixture pass. The guarded
build passes that fixture and four related constructor fixtures. The final
measurements above and the full parity sweep both use a fresh, frozen build
including this guard. See `evidence/constructor-identity.md` for the witness.

## Reproduction

Use isolated baseline and candidate worktrees, each with its own target.
Keep the package selection and build settings identical on both sides:

```sh
export LLVM_SYS_221_PREFIX=/usr/lib/llvm-22
export CARGO_BUILD_JOBS=4 CARGO_INCREMENTAL=0
export CARGO_PROFILE_RELEASE_CODEGEN_UNITS=16
export CARGO_PROFILE_RELEASE_DEBUG=1 CARGO_PROFILE_RELEASE_STRIP=none
cargo build --release -p perry -p perry-runtime-static -p perry-stdlib-static
cargo build --release -p perry-runtime --example catch_frames
export PERRY_RUNTIME_DIR="$PWD/target/release"
python3 benchmarks/catch_frames/measure.py \
--perry "$PWD/target/release/perry" \
--micro "$PWD/target/release/examples/catch_frames" --output /tmp/catch-measure
python3 benchmarks/catch_frames/count_frames.py \
/tmp/catch-measure/promises /tmp/catch-measure/promises.frames
```

Put pinned Node 26.5.1 on PATH. Do not override the repository's unwind-table
and x86 frame-pointer flags. Both measurements use thin LTO and the same
16-codegen-unit override; these are not claims about a separate dist build.
Copy the benchmark sources and example into the baseline worktree without
the runtime patch. Freeze the three compiler/archive outputs before building
any extra package set for other validation.

The measurement script uses `perf stat -r 3 -e instructions:u` and a separate
`perf record -e instructions:u --call-graph dwarf` run. It retains oracle
outputs and folded stacks. Run fault injection only in a disposable worktree,
with no other builds or source copying in progress:

```sh
python3 benchmarks/catch_frames/fault_restore.py --output /tmp/catch-faults
```

## Validation

The local replay and its baseline comparisons are recorded in
`evidence/validation.md`. GC ratchet measurements use seven repetitions of
each of 14 probes. The candidate passes against a fresh Linux baseline from
the same main commit, with all 126 heap/collector medians identical. This
does not replace the repository's pinned artifact with a new baseline or
claim equivalence to its different-platform capture. GC wall/RSS observations
are retained by the harness but are not performance evidence for this change.
32 changes: 32 additions & 0 deletions benchmarks/catch_frames/count_frames.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
#!/usr/bin/env python3
"""Count all try-frame pushes with a temporary uprobe, outside perf A/B runs."""
import argparse
import json
import os
from pathlib import Path
import subprocess


def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("binary", type=Path)
parser.add_argument("output", type=Path)
args = parser.parse_args()
symbols = subprocess.check_output(["nm", "--defined-only", str(args.binary)], text=True)
push = [line.split()[-1] for line in symbols.splitlines() if "try_push_with_kind" in line]
assert len(push) == 1, push
event = f"perry_catch_{os.getpid()}:push"
# Use the DWARF function name: an LTO-generated `.llvm.<hash>` suffix in
# the linkage name is parsed as probe syntax by perf.
subprocess.run(["perf", "probe", "-x", str(args.binary), "--add", f"{event}=try_push_with_kind"], stdout=subprocess.DEVNULL, check=True)
try:
with args.output.with_suffix(".out").open("w") as stream:
subprocess.run(["perf", "stat", "-x", ";", "-e", event, "-o", str(args.output), "--", str(args.binary)], stdout=stream, check=True)
finally:
subprocess.run(["perf", "probe", "-q", "--del", event], check=True)
count = next(int(line.split(";")[0]) for line in args.output.read_text().splitlines() if event in line)
print(json.dumps({"binary": str(args.binary), "frame_pushes": count}))


if __name__ == "__main__":
main()
20 changes: 20 additions & 0 deletions benchmarks/catch_frames/evidence/additional-baseline-checks.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
[
{
"test": "test_issue_4826_array_headers",
"baseline": "node_fail",
"candidate": "node_fail",
"detail": "Node oracle times out: server fixture never closes"
},
{
"test": "test_issue_4975_http_agent_keep_alive_connection_options",
"baseline": "crash",
"candidate": "crash",
"detail": "Perry prints true and 1250, then exceeds the 10-second shutdown deadline. Existing Linux allowlist entry cites #8841, which was closed when checked on 2026-09-14."
},
{
"test": "test_parity_stream_web",
"baseline": "crash",
"candidate": "crash",
"detail": "Both print the Web Streams constructor error and exceed the 10-second deadline. Existing module-inventory entry cites closed roadmap #793."
}
]
12 changes: 12 additions & 0 deletions benchmarks/catch_frames/evidence/api-docs-comparison.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
[
{
"path": "docs/src/api/reference.md",
"baseline_generated_equals_candidate_generated": true,
"baseline_generated_equals_committed": false
},
{
"path": "docs/api/perry.d.ts",
"baseline_generated_equals_candidate_generated": true,
"baseline_generated_equals_committed": false
}
]
Binary file not shown.
Binary file not shown.
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
public baseline error: public artifact benchmark inputs changed; regenerate it with ./benchmarks/run_public_baseline.sh
Binary file not shown.
Binary file not shown.
3 changes: 3 additions & 0 deletions benchmarks/catch_frames/evidence/baseline/catch.stat
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# started on Mon Sep 14 04:34:13 2026

472151423;;instructions:u;0.00%;45711787;100.00;;
1 change: 1 addition & 0 deletions benchmarks/catch_frames/evidence/baseline/exec1.expected
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
5888890
Binary file not shown.
3 changes: 3 additions & 0 deletions benchmarks/catch_frames/evidence/baseline/exec1.stat
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# started on Mon Sep 14 04:32:12 2026

31217404185;;instructions:u;0.00%;2411220179;100.00;;
1 change: 1 addition & 0 deletions benchmarks/catch_frames/evidence/baseline/hoist.expected
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
500000
Binary file not shown.
3 changes: 3 additions & 0 deletions benchmarks/catch_frames/evidence/baseline/hoist.frames
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# started on Mon Sep 14 04:35:34 2026

3500011;;perry_catch_201827:push;3490348955;100.00;;
3 changes: 3 additions & 0 deletions benchmarks/catch_frames/evidence/baseline/hoist.stat
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# started on Mon Sep 14 04:30:55 2026

13816754889;;instructions:u;0.00%;862134599;100.00;;
1 change: 1 addition & 0 deletions benchmarks/catch_frames/evidence/baseline/json.expected
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
5008938890
Binary file not shown.
3 changes: 3 additions & 0 deletions benchmarks/catch_frames/evidence/baseline/json.frames
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# started on Mon Sep 14 04:32:53 2026

6;;perry_catch_192702:push;228354174;100.00;;
3 changes: 3 additions & 0 deletions benchmarks/catch_frames/evidence/baseline/json.stat
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# started on Mon Sep 14 04:29:53 2026

3182841469;;instructions:u;0.00%;190945873;100.00;;
Binary file not shown.
3 changes: 3 additions & 0 deletions benchmarks/catch_frames/evidence/baseline/plain.stat
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# started on Mon Sep 14 04:34:03 2026

27068237;;instructions:u;0.00%;1910641;100.00;;
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
5000050000
Binary file not shown.
3 changes: 3 additions & 0 deletions benchmarks/catch_frames/evidence/baseline/promises.frames
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# started on Mon Sep 14 04:35:33 2026

100007;;perry_catch_201756:push;150141188;100.00;;
3 changes: 3 additions & 0 deletions benchmarks/catch_frames/evidence/baseline/promises.stat
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# started on Mon Sep 14 04:29:25 2026

1098667245;;instructions:u;0.00%;106550647;100.00;;
Binary file not shown.
3 changes: 3 additions & 0 deletions benchmarks/catch_frames/evidence/baseline/unwind.stat
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# started on Mon Sep 14 04:34:32 2026

366173508;;instructions:u;0.00%;25498367;100.00;;
5 changes: 5 additions & 0 deletions benchmarks/catch_frames/evidence/candidate-artifacts.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{
"perry": "08a1aff537958e855fbdf5b9f25ccd22deb3cc3fc1a97c2f3063ed98dd2cbd9d",
"libperry_runtime.a": "2c0b05fbad66bb569241ccf742831504c9ef8c4819a31845c3d92c70d0efaa72",
"libperry_stdlib.a": "86ffd21fcd187c748661fd85015d6824998a43f70610b98b98cb7e8fd86a0206"
}
Binary file not shown.
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
499999500000
3 changes: 3 additions & 0 deletions benchmarks/catch_frames/evidence/candidate/catch.stat
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# started on Mon Sep 14 05:49:25 2026

351152420;;instructions:u;0.00%;27154989;100.00;;
3 changes: 3 additions & 0 deletions benchmarks/catch_frames/evidence/candidate/catch.stat.out
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
499999500000
499999500000
499999500000
1 change: 1 addition & 0 deletions benchmarks/catch_frames/evidence/candidate/exec1.expected
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
5888890
Binary file not shown.
1 change: 1 addition & 0 deletions benchmarks/catch_frames/evidence/candidate/exec1.out
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
5888890
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
5888890
3 changes: 3 additions & 0 deletions benchmarks/catch_frames/evidence/candidate/exec1.stat
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# started on Mon Sep 14 05:47:42 2026

30183325430;;instructions:u;0.00%;3248216039;100.00;;
3 changes: 3 additions & 0 deletions benchmarks/catch_frames/evidence/candidate/exec1.stat.out
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
5888890
5888890
5888890
1 change: 1 addition & 0 deletions benchmarks/catch_frames/evidence/candidate/hoist.expected
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
500000
Binary file not shown.
1 change: 1 addition & 0 deletions benchmarks/catch_frames/evidence/candidate/hoist.out
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
500000
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
500000
3 changes: 3 additions & 0 deletions benchmarks/catch_frames/evidence/candidate/hoist.stat
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# started on Mon Sep 14 05:46:49 2026

13266297479;;instructions:u;0.00%;830465765;100.00;;
3 changes: 3 additions & 0 deletions benchmarks/catch_frames/evidence/candidate/hoist.stat.out
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
500000
500000
500000
1 change: 1 addition & 0 deletions benchmarks/catch_frames/evidence/candidate/json.expected
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
5008938890
Binary file not shown.
1 change: 1 addition & 0 deletions benchmarks/catch_frames/evidence/candidate/json.out
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
5008938890
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
5008938890
3 changes: 3 additions & 0 deletions benchmarks/catch_frames/evidence/candidate/json.stat
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# started on Mon Sep 14 05:46:19 2026

3171165684;;instructions:u;0.00%;196440960;100.00;;
3 changes: 3 additions & 0 deletions benchmarks/catch_frames/evidence/candidate/json.stat.out
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
5008938890
5008938890
5008938890
Binary file not shown.
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
499999500000
3 changes: 3 additions & 0 deletions benchmarks/catch_frames/evidence/candidate/plain.stat
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# started on Mon Sep 14 05:49:18 2026

27068989;;instructions:u;0.00%;3532580;100.00;;
3 changes: 3 additions & 0 deletions benchmarks/catch_frames/evidence/candidate/plain.stat.out
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
499999500000
499999500000
499999500000
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
5000050000
Binary file not shown.
1 change: 1 addition & 0 deletions benchmarks/catch_frames/evidence/candidate/promises.out
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
5000050000
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
5000050000
3 changes: 3 additions & 0 deletions benchmarks/catch_frames/evidence/candidate/promises.stat
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# started on Mon Sep 14 05:46:01 2026

1077465852;;instructions:u;0.00%;67069381;100.00;;
3 changes: 3 additions & 0 deletions benchmarks/catch_frames/evidence/candidate/promises.stat.out
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
5000050000
5000050000
5000050000
51 changes: 51 additions & 0 deletions benchmarks/catch_frames/evidence/candidate/results.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
{
"promises": {
"instructions": 1077465852.0,
"samples": 1077,
"capture_samples": 4,
"transport_leaf_samples": 0,
"capture_percent": 0.3714020427112349
},
"json": {
"instructions": 3171165684.0,
"samples": 3168,
"capture_samples": 0,
"transport_leaf_samples": 0,
"capture_percent": 0.0
},
"hoist": {
"instructions": 13266297479.0,
"samples": 13263,
"capture_samples": 447,
"transport_leaf_samples": 483,
"capture_percent": 3.3702782175978285
},
"exec1": {
"instructions": 30183325430.0,
"samples": 30182,
"capture_samples": 999,
"transport_leaf_samples": 1080,
"capture_percent": 3.309919819760122
},
"plain": {
"instructions": 27068989.0,
"samples": 26,
"capture_samples": 0,
"transport_leaf_samples": 0,
"capture_percent": 0.0
},
"catch": {
"instructions": 351152420.0,
"samples": 351,
"capture_samples": 6,
"transport_leaf_samples": 43,
"capture_percent": 1.7094017094017093
},
"unwind": {
"instructions": 244175238.0,
"samples": 244,
"capture_samples": 129,
"transport_leaf_samples": 0,
"capture_percent": 52.868852459016395
}
}
Binary file not shown.
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
499999500000
3 changes: 3 additions & 0 deletions benchmarks/catch_frames/evidence/candidate/unwind.stat
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# started on Mon Sep 14 05:49:46 2026

244175238;;instructions:u;0.00%;19181172;100.00;;
Loading
Loading