From 6f29fb0d03068a49a0a8bfc82b77b1464e543f0d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ralph=20K=C3=BCpper?= Date: Sat, 12 Sep 2026 06:04:12 +0200 Subject: [PATCH 01/10] perf(codegen): read materialized lazy JSON arrays from the indexed cache A JSON.parse result carries GC_TYPE_LAZY_ARRAY, so the indexed inline cache's brand check (obj_type == GC_TYPE_ARRAY) rejected it and every element read fell through to arrlike.ic.miss, re-classifying the same receiver three more times: js_packed_arraylike_index_get, then js_array_get_f64, then json_tape::cached_read::lazy_get. R22-R26 made that last helper allocation-free, but the call chain in front of it was untouched, so `rows[7].id` on a parsed array still cost ~237 retired instructions against Node's handful of cycles. Serve the read in the cache instead, once a scan or the random-access flip has installed the ordinary array, on exactly the proof lazy_get already takes: live unforwarded GC_TYPE_ARRAY, no descriptor overrides, length within capacity and its plausibility bound, dense in-bounds index. Prototype invalidation is also honoured, which the ordinary tier checks and lazy_get does not, so the admitted set is no wider. lazy_get refreshes the header's cached_length mirror when it takes this path. A cache cannot write, so it requires the mirror to already agree and routes a disagreement to the miss helper, which refreshes it and lets the next read hit. A grown or shrunk array therefore never reports a stale length through a fast-path read. Holes, sparse tape-backed reads, growth-forwarding stubs and every exotic receiver keep the unchanged dispatcher. --- .../expr/index_get/inline_dyn_typed_array.rs | 125 +++++++++++++++++- crates/perry-runtime/src/json_tape.rs | 11 ++ 2 files changed, 135 insertions(+), 1 deletion(-) diff --git a/crates/perry-codegen/src/expr/index_get/inline_dyn_typed_array.rs b/crates/perry-codegen/src/expr/index_get/inline_dyn_typed_array.rs index 53c1fa6cac..00425a0a40 100644 --- a/crates/perry-codegen/src/expr/index_get/inline_dyn_typed_array.rs +++ b/crates/perry-codegen/src/expr/index_get/inline_dyn_typed_array.rs @@ -466,9 +466,132 @@ pub(super) fn lower_inline_dyn_typed_array_get( let elem_bounds_label = ctx.block_label(elem_bounds_idx); let elem_load_label = ctx.block_label(elem_load_idx); let elem_value_label = ctx.block_label(elem_value_idx); + // A `JSON.parse` result is `GC_TYPE_LAZY_ARRAY`, not `GC_TYPE_ARRAY`, so + // every one of its indexed reads used to fall straight through to + // `arrlike.ic.miss` and re-classify the receiver three more times + // (`js_packed_arraylike_index_get` -> `js_array_get_f64` -> + // `json_tape::cached_read::lazy_get`). Once a scan or a random-access flip + // has installed the ordinary array, that whole chain resolves one word; + // serve it here instead, on exactly the proof `lazy_get` already uses. + let lazy_kind_idx = ctx.new_block("arrlike.lazy.kind"); + let lazy_header_idx = ctx.new_block("arrlike.lazy.header"); + let lazy_guard_idx = ctx.new_block("arrlike.lazy.guard"); + let lazy_load_idx = ctx.new_block("arrlike.lazy.load"); + let lazy_kind_label = ctx.block_label(lazy_kind_idx); + let lazy_header_label = ctx.block_label(lazy_header_idx); + let lazy_guard_label = ctx.block_label(lazy_guard_idx); + let lazy_load_label = ctx.block_label(lazy_load_idx); + ctx.current_block = object_brand_idx; ctx.block() - .cond_br(&is_array, &object_array_guard_label, &elem_kind_label); + .cond_br(&is_array, &object_array_guard_label, &lazy_kind_label); + + // `GC_TYPE_LAZY_ARRAY` (perry-runtime `gc/types.rs`). Anything else keeps + // the existing Array-subclass probe below. + ctx.current_block = lazy_kind_idx; + let lazy_is_lazy = ctx.block().icmp_eq(I8, &gc_type, "9"); + ctx.block() + .cond_br(&lazy_is_lazy, &lazy_header_label, &elem_kind_label); + + // `LazyArrayHeader::materialized` is word 4 (offset 32; pinned by a const + // assert in perry-runtime `json_tape.rs`). Null means the array is still + // tape-backed and only the sparse per-element cache can answer, which + // needs the bitmap probe in `lazy_get` — keep that on the miss path. + ctx.current_block = lazy_header_idx; + let lazy_materialized_addr = ctx.block().add(I64, &object_raw, "32"); + let lazy_materialized_ptr = ctx.block().inttoptr(I64, &lazy_materialized_addr); + let lazy_materialized = ctx.block().load(I64, &lazy_materialized_ptr); + let lazy_has_array = ctx.block().icmp_ne(I64, &lazy_materialized, "0"); + ctx.block() + .cond_br(&lazy_has_array, &lazy_guard_label, &object_miss_label); + + // The same proof `cached_read::lazy_get` takes before its inline load: a + // live unforwarded ordinary Array, no descriptor overrides, no prototype + // invalidation, and a dense in-capacity index. A growth-forwarding stub + // keeps its own GC header and fails `obj_type`/`FORWARDED`, so it routes + // to the resolver exactly as before (#9717). + // + // `cached_length` is the length mirror codegen reads for `.length` at + // offset 0. `lazy_get` refreshes it when it takes this path; the cache + // cannot write, so it instead requires the mirror to already agree and + // sends a disagreement to the miss helper — which refreshes it, making the + // next read hit. That keeps a grown or shrunk array from reporting a stale + // length through a fast-path read. + ctx.current_block = lazy_guard_idx; + let lazy_type_addr = ctx.block().sub(I64, &lazy_materialized, "8"); + let lazy_type_ptr = ctx.block().inttoptr(I64, &lazy_type_addr); + let lazy_type = ctx.block().load(I8, &lazy_type_ptr); + let lazy_is_array = ctx.block().icmp_eq(I8, &lazy_type, "1"); + let lazy_flags_addr = ctx.block().sub(I64, &lazy_materialized, "7"); + let lazy_flags_ptr = ctx.block().inttoptr(I64, &lazy_flags_addr); + let lazy_flags = ctx.block().load(I8, &lazy_flags_ptr); + let lazy_fwd = ctx.block().and(I8, &lazy_flags, "128"); + let lazy_not_fwd = ctx.block().icmp_eq(I8, &lazy_fwd, "0"); + let lazy_reserved_addr = ctx.block().sub(I64, &lazy_materialized, "6"); + let lazy_reserved_ptr = ctx.block().inttoptr(I64, &lazy_reserved_addr); + let lazy_reserved = ctx.block().load(I16, &lazy_reserved_ptr); + let lazy_descriptor_bits = ctx.block().and(I16, &lazy_reserved, "1024"); + let lazy_no_descriptors = ctx.block().icmp_eq(I16, &lazy_descriptor_bits, "0"); + let lazy_invalidated = ctx + .block() + .load_volatile(I8, "@PERRY_ARRAY_INDEX_FAST_PATH_INVALIDATED"); + let lazy_default_prototypes = ctx.block().icmp_eq(I8, &lazy_invalidated, "0"); + let lazy_array_ptr = ctx.block().inttoptr(I64, &lazy_materialized); + let lazy_length = ctx.block().load(I32, &lazy_array_ptr); + let lazy_capacity_addr = ctx.block().add(I64, &lazy_materialized, "4"); + let lazy_capacity_ptr = ctx.block().inttoptr(I64, &lazy_capacity_addr); + let lazy_capacity = ctx.block().load(I32, &lazy_capacity_ptr); + let lazy_mirror_ptr = ctx.block().inttoptr(I64, &object_raw); + let lazy_mirror = ctx.block().load(I32, &lazy_mirror_ptr); + let lazy_mirror_fresh = ctx.block().icmp_eq(I32, &lazy_mirror, &lazy_length); + let lazy_length_i64 = ctx.block().zext(I32, &lazy_length, I64); + let lazy_capacity_i64 = ctx.block().zext(I32, &lazy_capacity, I64); + let lazy_in_bounds = ctx.block().icmp_ult(I64, &object_idx_i64, &lazy_length_i64); + let lazy_within_capacity = ctx + .block() + .icmp_ule(I64, &lazy_length_i64, &lazy_capacity_i64); + // `lazy_get`'s own plausibility bound on an installed array. Keeping it + // makes this cache admit exactly the set the runtime helper admits, so the + // two can never disagree about which reads are fast. + let lazy_length_plausible = ctx + .block() + .icmp_ule(I64, &lazy_length_i64, "100000000"); + let lazy_ok = ctx.block().and(I1, &lazy_is_array, &lazy_not_fwd); + let lazy_ok = ctx.block().and(I1, &lazy_ok, &lazy_no_descriptors); + let lazy_ok = ctx.block().and(I1, &lazy_ok, &lazy_default_prototypes); + let lazy_ok = ctx.block().and(I1, &lazy_ok, &lazy_mirror_fresh); + let lazy_ok = ctx.block().and(I1, &lazy_ok, &lazy_in_bounds); + let lazy_ok = ctx.block().and(I1, &lazy_ok, &lazy_within_capacity); + let lazy_ok = ctx.block().and(I1, &lazy_ok, &lazy_length_plausible); + ctx.block() + .cond_br(&lazy_ok, &lazy_load_label, &object_miss_label); + + // A hole must still consult the prototype chain, so it keeps the complete + // dispatcher rather than becoming `undefined` here. + ctx.current_block = lazy_load_idx; + let lazy_element_word = ctx.block().add(I64, &object_idx_i64, "1"); + let lazy_element_ptr = + ctx.block() + .gep_inbounds(I64, &lazy_array_ptr, &[(I64, &lazy_element_word)]); + let lazy_raw = ctx.block().load(DOUBLE, &lazy_element_ptr); + let lazy_raw_bits = ctx.block().bitcast_double_to_i64(&lazy_raw); + let lazy_is_hole = ctx + .block() + .icmp_eq(I64, &lazy_raw_bits, crate::nanbox::TAG_HOLE_I64); + let lazy_value_idx = ctx.new_block("arrlike.lazy.value"); + let lazy_value_label = ctx.block_label(lazy_value_idx); + ctx.block() + .cond_br(&lazy_is_hole, &object_miss_label, &lazy_value_label); + ctx.current_block = lazy_value_idx; + let lazy_value = if coerce_slow_to_number { + ctx.block() + .call(DOUBLE, "js_number_coerce", &[(DOUBLE, &lazy_raw)]) + } else { + lazy_raw + }; + let lazy_end_label = ctx.block().label.clone(); + ctx.block().br(&merge_label); + kind_incoming.push((lazy_value, lazy_end_label)); ctx.current_block = elem_kind_idx; let elem_is_object = ctx.block().icmp_eq(I8, &gc_type, "2"); ctx.block() diff --git a/crates/perry-runtime/src/json_tape.rs b/crates/perry-runtime/src/json_tape.rs index ffe70a0699..0515fb1f86 100644 --- a/crates/perry-runtime/src/json_tape.rs +++ b/crates/perry-runtime/src/json_tape.rs @@ -1153,6 +1153,17 @@ const _: () = assert!( `.length` as a raw u32 load there" ); +// `materialized` is the second codegen contract on this struct. The indexed +// inline cache (`perry-codegen` `expr/index_get/inline_dyn_typed_array.rs`) +// reads this slot directly to serve `lazy[i]` without a runtime call, exactly +// as `cached_read::lazy_get` does. A reordered field would send that fast path +// at an unrelated word, so pin the offset the same way `cached_length` is. +const _: () = assert!( + std::mem::offset_of!(LazyArrayHeader, materialized) == 32, + "LazyArrayHeader::materialized must stay at offset 32 — the indexed inline \ + cache loads the installed array from that word" +); + /// #7478: how long a run of consecutive ascending cold reads has to get /// before we stop materializing element-by-element and hand the whole /// array to the batch parser. From fd5221197a63503bf21ec27be97b0837d271182b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ralph=20K=C3=BCpper?= Date: Sat, 12 Sep 2026 06:12:03 +0200 Subject: [PATCH 02/10] perf(codegen): probe the lazy sparse element cache from the indexed cache MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The materialized tier only fires once a scan or the adaptive random-access flip has installed an ordinary array. An array small enough that the walk never trips that flip — 120 records in the benchmark's 16 KiB fixture, and any repeated or clustered read pattern — stays tape-backed for the life of the program and kept missing the cache entirely, which is why the 16 KiB field walk was the worst row in the matrix at 10.6x Node. Inline lazy_get's sparse branch for that case: bounds against the header's cached_length, non-null bitmap and element words, the bitmap bit, then the parallel element slot. The bitmap is the liveness test because JSValue::ZERO is a legal cached value, so a zero element word cannot serve as one. Out-of-bounds deliberately does not shortcut to undefined the way lazy_get does — the prototype chain stays the miss helper's job. Cold reads, holes, uncached slots and prototype invalidation all keep the unchanged dispatcher. --- .../expr/index_get/inline_dyn_typed_array.rs | 98 ++++++++++++++++++- .../src/expr/index_get_claim_tests.rs | 12 +++ crates/perry-runtime/src/json_tape.rs | 14 +++ 3 files changed, 123 insertions(+), 1 deletion(-) diff --git a/crates/perry-codegen/src/expr/index_get/inline_dyn_typed_array.rs b/crates/perry-codegen/src/expr/index_get/inline_dyn_typed_array.rs index 00425a0a40..97096802b5 100644 --- a/crates/perry-codegen/src/expr/index_get/inline_dyn_typed_array.rs +++ b/crates/perry-codegen/src/expr/index_get/inline_dyn_typed_array.rs @@ -477,6 +477,16 @@ pub(super) fn lower_inline_dyn_typed_array_get( let lazy_header_idx = ctx.new_block("arrlike.lazy.header"); let lazy_guard_idx = ctx.new_block("arrlike.lazy.guard"); let lazy_load_idx = ctx.new_block("arrlike.lazy.load"); + let lazy_sparse_bounds_idx = ctx.new_block("arrlike.lazy.sparse.bounds"); + let lazy_sparse_probe_idx = ctx.new_block("arrlike.lazy.sparse.probe"); + let lazy_sparse_bit_idx = ctx.new_block("arrlike.lazy.sparse.bit"); + let lazy_sparse_load_idx = ctx.new_block("arrlike.lazy.sparse.load"); + let lazy_sparse_value_idx = ctx.new_block("arrlike.lazy.sparse.value"); + let lazy_sparse_bounds_label = ctx.block_label(lazy_sparse_bounds_idx); + let lazy_sparse_probe_label = ctx.block_label(lazy_sparse_probe_idx); + let lazy_sparse_bit_label = ctx.block_label(lazy_sparse_bit_idx); + let lazy_sparse_load_label = ctx.block_label(lazy_sparse_load_idx); + let lazy_sparse_value_label = ctx.block_label(lazy_sparse_value_idx); let lazy_kind_label = ctx.block_label(lazy_kind_idx); let lazy_header_label = ctx.block_label(lazy_header_idx); let lazy_guard_label = ctx.block_label(lazy_guard_idx); @@ -503,7 +513,93 @@ pub(super) fn lower_inline_dyn_typed_array_get( let lazy_materialized = ctx.block().load(I64, &lazy_materialized_ptr); let lazy_has_array = ctx.block().icmp_ne(I64, &lazy_materialized, "0"); ctx.block() - .cond_br(&lazy_has_array, &lazy_guard_label, &object_miss_label); + .cond_br(&lazy_has_array, &lazy_guard_label, &lazy_sparse_bounds_label); + + // Still tape-backed: the sparse per-element cache is the only thing that + // can answer without materializing a subtree, and it is what a repeated or + // clustered read pattern actually hits — an array small enough that the + // adaptive walk never trips the full-materialization flip stays here for + // the life of the program. `lazy_get`'s own sparse branch is three loads + // and a bit test, so inline exactly that and leave every cold read, hole + // and out-of-bounds index to the miss helper. + // + // Out-of-bounds deliberately does NOT shortcut to `undefined` here even + // though `lazy_get` does: the prototype chain is the miss helper's job. + ctx.current_block = lazy_sparse_bounds_idx; + let sparse_invalidated = ctx + .block() + .load_volatile(I8, "@PERRY_ARRAY_INDEX_FAST_PATH_INVALIDATED"); + let sparse_default_prototypes = ctx.block().icmp_eq(I8, &sparse_invalidated, "0"); + let sparse_length_ptr = ctx.block().inttoptr(I64, &object_raw); + let sparse_length = ctx.block().load(I32, &sparse_length_ptr); + let sparse_length_i64 = ctx.block().zext(I32, &sparse_length, I64); + let sparse_in_bounds = ctx + .block() + .icmp_ult(I64, &object_idx_i64, &sparse_length_i64); + let sparse_bounds_ok = ctx + .block() + .and(I1, &sparse_default_prototypes, &sparse_in_bounds); + ctx.block() + .cond_br(&sparse_bounds_ok, &lazy_sparse_probe_label, &object_miss_label); + + // `materialized_bitmap` is word 6 and `materialized_elements` word 5 + // (offsets 48 and 40; both pinned by const asserts in perry-runtime + // `json_tape.rs`). A null on either side means nothing has been cached yet. + ctx.current_block = lazy_sparse_probe_idx; + let sparse_bitmap_addr = ctx.block().add(I64, &object_raw, "48"); + let sparse_bitmap_slot = ctx.block().inttoptr(I64, &sparse_bitmap_addr); + let sparse_bitmap = ctx.block().load(I64, &sparse_bitmap_slot); + let sparse_elements_addr = ctx.block().add(I64, &object_raw, "40"); + let sparse_elements_slot = ctx.block().inttoptr(I64, &sparse_elements_addr); + let sparse_elements = ctx.block().load(I64, &sparse_elements_slot); + let sparse_has_bitmap = ctx.block().icmp_ne(I64, &sparse_bitmap, "0"); + let sparse_has_elements = ctx.block().icmp_ne(I64, &sparse_elements, "0"); + let sparse_probe_ok = ctx + .block() + .and(I1, &sparse_has_bitmap, &sparse_has_elements); + ctx.block() + .cond_br(&sparse_probe_ok, &lazy_sparse_bit_label, &object_miss_label); + + // The bitmap is the authoritative "this slot holds a materialized value" + // signal — `JSValue::ZERO` is a legal cached value, so a null/zero element + // word cannot be used as the liveness test. + ctx.current_block = lazy_sparse_bit_idx; + let sparse_word_index = ctx.block().lshr(I64, &object_idx_i64, "6"); + let sparse_word_offset = ctx.block().shl(I64, &sparse_word_index, "3"); + let sparse_word_addr = ctx.block().add(I64, &sparse_bitmap, &sparse_word_offset); + let sparse_word_ptr = ctx.block().inttoptr(I64, &sparse_word_addr); + let sparse_word = ctx.block().load(I64, &sparse_word_ptr); + let sparse_bit_index = ctx.block().and(I64, &object_idx_i64, "63"); + let sparse_shifted = ctx.block().lshr(I64, &sparse_word, &sparse_bit_index); + let sparse_bit = ctx.block().and(I64, &sparse_shifted, "1"); + let sparse_cached = ctx.block().icmp_ne(I64, &sparse_bit, "0"); + ctx.block() + .cond_br(&sparse_cached, &lazy_sparse_load_label, &object_miss_label); + + // A bitmap-set slot always holds a real materialized JSValue, so the hole + // test below can never fire; keep it anyway, since its only effect is to + // route an impossible value to the same helper that would have produced it. + ctx.current_block = lazy_sparse_load_idx; + let sparse_elem_offset = ctx.block().shl(I64, &object_idx_i64, "3"); + let sparse_elem_addr = ctx.block().add(I64, &sparse_elements, &sparse_elem_offset); + let sparse_elem_ptr = ctx.block().inttoptr(I64, &sparse_elem_addr); + let sparse_raw = ctx.block().load(DOUBLE, &sparse_elem_ptr); + let sparse_raw_bits = ctx.block().bitcast_double_to_i64(&sparse_raw); + let sparse_is_hole = ctx + .block() + .icmp_eq(I64, &sparse_raw_bits, crate::nanbox::TAG_HOLE_I64); + ctx.block() + .cond_br(&sparse_is_hole, &object_miss_label, &lazy_sparse_value_label); + ctx.current_block = lazy_sparse_value_idx; + let sparse_value = if coerce_slow_to_number { + ctx.block() + .call(DOUBLE, "js_number_coerce", &[(DOUBLE, &sparse_raw)]) + } else { + sparse_raw + }; + let sparse_end_label = ctx.block().label.clone(); + ctx.block().br(&merge_label); + kind_incoming.push((sparse_value, sparse_end_label)); // The same proof `cached_read::lazy_get` takes before its inline load: a // live unforwarded ordinary Array, no descriptor overrides, no prototype diff --git a/crates/perry-codegen/src/expr/index_get_claim_tests.rs b/crates/perry-codegen/src/expr/index_get_claim_tests.rs index f3ace585ac..7322c59248 100644 --- a/crates/perry-codegen/src/expr/index_get_claim_tests.rs +++ b/crates/perry-codegen/src/expr/index_get_claim_tests.rs @@ -175,6 +175,18 @@ fn unknown_numeric_read_guards_dense_subclass_families_and_spilled_length() { ir.contains("arrlike.ic.range") && ir.contains("arrlike.ic.miss"), "the live length and cached dense-prefix bound must retain a semantic side exit:\n{ir}" ); + assert!( + ir.contains("arrlike.lazy.guard") && ir.contains("arrlike.lazy.load"), + "a materialized lazy JSON array must be readable without leaving the cache:\n{ir}" + ); + assert!( + ir.contains("@PERRY_ARRAY_INDEX_FAST_PATH_INVALIDATED"), + "the lazy tier must honour process-wide prototype invalidation:\n{ir}" + ); + assert!( + ir.contains("arrlike.lazy.sparse.bit") && ir.contains("arrlike.lazy.sparse.load"), + "a tape-backed lazy array must probe its per-element cache inline:\n{ir}" + ); } fn dynamic_symbol_access_ir(symbol_init: Expr, field: Option<&str>) -> String { diff --git a/crates/perry-runtime/src/json_tape.rs b/crates/perry-runtime/src/json_tape.rs index 0515fb1f86..63fe008497 100644 --- a/crates/perry-runtime/src/json_tape.rs +++ b/crates/perry-runtime/src/json_tape.rs @@ -1164,6 +1164,20 @@ const _: () = assert!( cache loads the installed array from that word" ); +// The sparse tier of that same cache probes the per-element cache directly: +// bitmap bit first, then the parallel element slot. Both offsets are read as +// raw words from emitted code, so neither may drift either. +const _: () = assert!( + std::mem::offset_of!(LazyArrayHeader, materialized_elements) == 40, + "LazyArrayHeader::materialized_elements must stay at offset 40 — the \ + indexed inline cache loads a cached element from that word" +); +const _: () = assert!( + std::mem::offset_of!(LazyArrayHeader, materialized_bitmap) == 48, + "LazyArrayHeader::materialized_bitmap must stay at offset 48 — the indexed \ + inline cache proves a cached element live from that word" +); + /// #7478: how long a run of consecutive ascending cold reads has to get /// before we stop materializing element-by-element and hand the whole /// array to the batch parser. From 07d6d2370d5e4496ef3d3e1f1219a1a2186905bc Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ralph=20K=C3=BCpper?= Date: Sat, 12 Sep 2026 06:12:56 +0200 Subject: [PATCH 03/10] test(json): cover both indexed-cache tiers for lazy JSON arrays MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Exercises what the two new cache tiers are allowed to answer and what they must hand back to the dispatcher: identity and value across repeated reads, growth and shrink against the header's length mirror, holes, an accessor descriptor installed on one index, a prototype index override and its retirement, and cached zero — whose NaN-boxed bits are all zero, so the bitmap rather than the element word has to prove a sparse slot live. The sparse half deliberately never scans its array, since a scan would trip the materialization flip and move it onto the other tier. --- .../test_gap_json_lazy_indexed_cache.ts | 151 ++++++++++++++++++ 1 file changed, 151 insertions(+) create mode 100644 test-files/test_gap_json_lazy_indexed_cache.ts diff --git a/test-files/test_gap_json_lazy_indexed_cache.ts b/test-files/test_gap_json_lazy_indexed_cache.ts new file mode 100644 index 0000000000..09ecca106e --- /dev/null +++ b/test-files/test_gap_json_lazy_indexed_cache.ts @@ -0,0 +1,151 @@ +// The indexed inline cache serves reads of a MATERIALIZED lazy JSON array. +// Everything here is about the guards that let it do so safely: the array must +// still be an ordinary unforwarded Array, its length mirror on the lazy header +// must agree, holes and descriptors must fall back, and a prototype override +// must disable the fast path. Run in auto/tape/direct modes, including +// scheduled moving GC. +const pieces: string[] = []; +for (let i = 0; i < 200; i++) { + pieces.push('{"id":' + i + ',"name":"heap string for record ' + i + '"}'); +} +const text = "[" + pieces.join(",") + "]"; +const retained: any[] = []; +const sparseRetained: any[] = []; +let sum = 0; + +function fullyMaterialize(rows: any): void { + // A whole-array scan trips the adaptive flip, so later reads go through the + // installed ordinary array rather than the sparse per-element cache. + let seen = 0; + for (let i = 0; i < rows.length; i++) seen += rows[i].id; + if (seen !== 19900) throw new Error("scan sum changed: " + seen); +} + +for (let round = 0; round < 40; round++) { + const rows: any = JSON.parse(text); + fullyMaterialize(rows); + + // Repeated reads off the installed array must keep identity and value. + const saved: any = rows[7]; + for (let repeat = 0; repeat < 16; repeat++) { + if (rows[7] !== saved || rows[7].id !== 7) throw new Error("identity changed"); + if (rows[199].id !== 199) throw new Error("tail value changed"); + if (rows[200] !== undefined || rows[4294967295] !== undefined) { + throw new Error("out-of-bounds read"); + } + sum += rows[7].id + rows[199].id; + } + + // Growth: the header's length mirror goes stale, so a fast-path read must + // not report the old length or miss the new element. + rows.push({id: 200, name: "grown heap string"}); + if (rows.length !== 201) throw new Error("length after growth: " + rows.length); + if (rows[200].id !== 200) throw new Error("grown element lost"); + if (rows[7] !== saved) throw new Error("identity lost across growth"); + + // Shrink, then regrow into holes. Reads of the hole region must consult the + // prototype chain rather than loading a stale slot. + rows.length = 32; + if (rows.length !== 32) throw new Error("length after shrink: " + rows.length); + if (rows[32] !== undefined || rows[200] !== undefined) { + throw new Error("stale read past shrink"); + } + if (rows[31].id !== 31) throw new Error("surviving element lost"); + rows.length = 64; + if (rows[48] !== undefined) throw new Error("hole must read undefined"); + + // A descriptor override on one index must take every read off the cache. + const described: any = JSON.parse(text); + fullyMaterialize(described); + let getterCalls = 0; + Object.defineProperty(described, 5, { + configurable: true, + get: function () { getterCalls++; return {id: -5, name: "from getter"}; }, + }); + for (let repeat = 0; repeat < 8; repeat++) { + const value: any = described[5]; + if (value.id !== -5 || value.name !== "from getter") { + throw new Error("descriptor read bypassed"); + } + if (described[6].id !== 6) throw new Error("neighbour read broken"); + sum += value.id; + } + if (getterCalls !== 8) throw new Error("getter calls: " + getterCalls); + + // Allocate between passes so scheduled moving GC also covers reads taken + // after the installed array has moved. + const churn: any = JSON.parse('{"name":"pass ' + round + '"}'); + if (churn.name !== "pass " + round) throw new Error("churn changed"); + retained.push(saved); +} + +// The SPARSE tier: an array whose adaptive walk never trips the +// full-materialization flip stays tape-backed, so repeated reads are served +// from the per-element cache instead of an installed array. Never scan this +// one -- a scan would move it onto the materialized tier above. +for (let round = 0; round < 20; round++) { + const sparse: any = JSON.parse(text); + const probes: number[] = [0, 1, 63, 64, 65, 127, 128, 199]; + const first: any[] = []; + for (let p = 0; p < probes.length; p++) first.push(sparse[probes[p]]); + for (let repeat = 0; repeat < 12; repeat++) { + for (let p = 0; p < probes.length; p++) { + const index = probes[p]; + const value: any = sparse[index]; + // Identity must hold across every repeat: a cache hit returns the + // same object, never a freshly materialized copy. + if (value !== first[p]) throw new Error("sparse identity changed at " + index); + if (value.id !== index) throw new Error("sparse value changed at " + index); + if (value.name !== "heap string for record " + index) { + throw new Error("sparse name changed at " + index); + } + sum += value.id; + } + if (sparse[200] !== undefined || sparse[4294967295] !== undefined) { + throw new Error("sparse out-of-bounds read"); + } + if (sparse[-1] !== undefined) throw new Error("negative index read"); + } + // Zero is a legal cached value and its NaN-boxed bits are all zero, so the + // bitmap -- not the element word -- has to be what proves a slot cached. + const zeros: any = JSON.parse("[0,0,0,0,0,0,0,0]"); + for (let repeat = 0; repeat < 8; repeat++) { + if (zeros[3] !== 0 || zeros[7] !== 0) throw new Error("cached zero lost"); + sum += zeros[3]; + } + // Mutating through the sparse cache must move the array off it correctly. + sparse[64] = {id: -64, name: "replacement"}; + if (sparse[64].id !== -64) throw new Error("sparse replacement lost"); + if (sparse[65] !== first[4]) throw new Error("neighbour lost across mutation"); + sparseRetained.push(first[2]); +} + +// A prototype index override must disable the fast path process-wide: a hole +// read has to find the inherited value, not undefined. +const holed: any = JSON.parse(text); +fullyMaterialize(holed); +holed.length = 8; +holed.length = 16; +(Array.prototype as any)[12] = "from prototype"; +if (holed[12] !== "from prototype") throw new Error("prototype override ignored"); +if (holed[3].id !== 3) throw new Error("dense read broken under override"); +delete (Array.prototype as any)[12]; +if (holed[12] !== undefined) throw new Error("prototype override not retired"); + +for (let round = 0; round < retained.length; round++) { + const saved: any = retained[round]; + if (saved.id !== 7 || saved.name !== "heap string for record 7") { + throw new Error("retained element changed"); + } + sum += saved.id; +} +// Elements handed out by the sparse tier must survive every later collection +// with their identity and contents intact. +for (let round = 0; round < sparseRetained.length; round++) { + const saved: any = sparseRetained[round]; + if (saved.id !== 63 || saved.name !== "heap string for record 63") { + throw new Error("retained sparse element changed"); + } + sum += saved.id; +} +console.log("lazy-indexed-cache", retained.length, sparseRetained.length, sum); From 28b83967fd14eee789f3bd04d78e9e8e5fadf350 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ralph=20K=C3=BCpper?= Date: Sat, 12 Sep 2026 06:33:44 +0200 Subject: [PATCH 04/10] perf(codegen): route the lazy tier off the subclass probe's miss edge Keep brand's successors exactly as they were and reach the lazy tier from arrlike.elem.kind's miss edge instead, after both the ordinary-Array and elements-subclass probes have declined the receiver. The admitted set and every guard are unchanged; only the position in the chain moves. This is hygiene, not a fix. The 20 MiB access rows regress ~4.8% on field walks either way, and the first placement was not the cause: profiling both arms on that row shows js_packed_arraylike_index_get absent entirely. A 20 MiB document is above the lazy admission bound, so it parses to an ordinary Array whose reads hit arrlike.ic.array_guard inline and never reach the miss chain where these blocks live -- they are present, not executed. The cost is code layout: run() grows 10752 to 11804 bytes, and that row's per-iteration work is ~0.027us and front-end bound (72-74% of samples inside run, 25% in fmod from the workload's own i % length). Retired instructions move +0.85% while CPU moves +4.8%, which is the signature of instruction fetch and prediction rather than executed work. --- .../src/expr/index_get/inline_dyn_typed_array.rs | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/crates/perry-codegen/src/expr/index_get/inline_dyn_typed_array.rs b/crates/perry-codegen/src/expr/index_get/inline_dyn_typed_array.rs index 97096802b5..c29ccc1e98 100644 --- a/crates/perry-codegen/src/expr/index_get/inline_dyn_typed_array.rs +++ b/crates/perry-codegen/src/expr/index_get/inline_dyn_typed_array.rs @@ -473,6 +473,8 @@ pub(super) fn lower_inline_dyn_typed_array_get( // `json_tape::cached_read::lazy_get`). Once a scan or a random-access flip // has installed the ordinary array, that whole chain resolves one word; // serve it here instead, on exactly the proof `lazy_get` already uses. + // The blocks are declared here; they are reached from `arrlike.elem.kind` + // below, after the ordinary-Array and elements-subclass probes both miss. let lazy_kind_idx = ctx.new_block("arrlike.lazy.kind"); let lazy_header_idx = ctx.new_block("arrlike.lazy.header"); let lazy_guard_idx = ctx.new_block("arrlike.lazy.guard"); @@ -494,14 +496,18 @@ pub(super) fn lower_inline_dyn_typed_array_get( ctx.current_block = object_brand_idx; ctx.block() - .cond_br(&is_array, &object_array_guard_label, &lazy_kind_label); + .cond_br(&is_array, &object_array_guard_label, &elem_kind_label); - // `GC_TYPE_LAZY_ARRAY` (perry-runtime `gc/types.rs`). Anything else keeps - // the existing Array-subclass probe below. + // `GC_TYPE_LAZY_ARRAY` (perry-runtime `gc/types.rs`). This tier hangs off + // the Array-subclass probe's miss edge rather than off `brand`, so the + // ordinary-Array path keeps exactly the control flow it had: measured on + // the 20 MiB fixture (above the lazy admission bound, so a plain Array), + // routing `brand`'s not-array edge through here cost +4 retired + // instructions per read on that untouched path. ctx.current_block = lazy_kind_idx; let lazy_is_lazy = ctx.block().icmp_eq(I8, &gc_type, "9"); ctx.block() - .cond_br(&lazy_is_lazy, &lazy_header_label, &elem_kind_label); + .cond_br(&lazy_is_lazy, &lazy_header_label, &object_miss_label); // `LazyArrayHeader::materialized` is word 4 (offset 32; pinned by a const // assert in perry-runtime `json_tape.rs`). Null means the array is still @@ -691,7 +697,7 @@ pub(super) fn lower_inline_dyn_typed_array_get( ctx.current_block = elem_kind_idx; let elem_is_object = ctx.block().icmp_eq(I8, &gc_type, "2"); ctx.block() - .cond_br(&elem_is_object, &elem_meta_label, &object_miss_label); + .cond_br(&elem_is_object, &elem_meta_label, &lazy_kind_label); ctx.current_block = elem_meta_idx; let elem_meta_addr = ctx.block().add(I64, &object_raw, &meta_offset); let elem_meta_slot_ptr = ctx.block().inttoptr(I64, &elem_meta_addr); From be49b9ee669e1ea3e5eef477364555a707036b96 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ralph=20K=C3=BCpper?= Date: Sat, 12 Sep 2026 06:45:28 +0200 Subject: [PATCH 05/10] chore(json): keep json_tape.rs under the file cap; rustfmt the lazy tiers The three new LazyArrayHeader offset pins pushed json_tape.rs to 2004 lines, over scripts/check_file_size.sh's 2000-line cap. Move all four layout contracts (the existing cached_length pin included) into json_tape/layout.rs, which is only their enforcement; the field doc comments keep saying why each word is load-bearing. json_tape.rs lands at 1968 lines, below where main has it. Also rustfmt's rewrap of the new cond_br calls in the indexed cache. --- .../expr/index_get/inline_dyn_typed_array.rs | 25 ++++++---- crates/perry-runtime/src/json_tape.rs | 39 +-------------- crates/perry-runtime/src/json_tape/layout.rs | 50 +++++++++++++++++++ 3 files changed, 67 insertions(+), 47 deletions(-) create mode 100644 crates/perry-runtime/src/json_tape/layout.rs diff --git a/crates/perry-codegen/src/expr/index_get/inline_dyn_typed_array.rs b/crates/perry-codegen/src/expr/index_get/inline_dyn_typed_array.rs index c29ccc1e98..0be52b154a 100644 --- a/crates/perry-codegen/src/expr/index_get/inline_dyn_typed_array.rs +++ b/crates/perry-codegen/src/expr/index_get/inline_dyn_typed_array.rs @@ -518,8 +518,11 @@ pub(super) fn lower_inline_dyn_typed_array_get( let lazy_materialized_ptr = ctx.block().inttoptr(I64, &lazy_materialized_addr); let lazy_materialized = ctx.block().load(I64, &lazy_materialized_ptr); let lazy_has_array = ctx.block().icmp_ne(I64, &lazy_materialized, "0"); - ctx.block() - .cond_br(&lazy_has_array, &lazy_guard_label, &lazy_sparse_bounds_label); + ctx.block().cond_br( + &lazy_has_array, + &lazy_guard_label, + &lazy_sparse_bounds_label, + ); // Still tape-backed: the sparse per-element cache is the only thing that // can answer without materializing a subtree, and it is what a repeated or @@ -545,8 +548,11 @@ pub(super) fn lower_inline_dyn_typed_array_get( let sparse_bounds_ok = ctx .block() .and(I1, &sparse_default_prototypes, &sparse_in_bounds); - ctx.block() - .cond_br(&sparse_bounds_ok, &lazy_sparse_probe_label, &object_miss_label); + ctx.block().cond_br( + &sparse_bounds_ok, + &lazy_sparse_probe_label, + &object_miss_label, + ); // `materialized_bitmap` is word 6 and `materialized_elements` word 5 // (offsets 48 and 40; both pinned by const asserts in perry-runtime @@ -594,8 +600,11 @@ pub(super) fn lower_inline_dyn_typed_array_get( let sparse_is_hole = ctx .block() .icmp_eq(I64, &sparse_raw_bits, crate::nanbox::TAG_HOLE_I64); - ctx.block() - .cond_br(&sparse_is_hole, &object_miss_label, &lazy_sparse_value_label); + ctx.block().cond_br( + &sparse_is_hole, + &object_miss_label, + &lazy_sparse_value_label, + ); ctx.current_block = lazy_sparse_value_idx; let sparse_value = if coerce_slow_to_number { ctx.block() @@ -655,9 +664,7 @@ pub(super) fn lower_inline_dyn_typed_array_get( // `lazy_get`'s own plausibility bound on an installed array. Keeping it // makes this cache admit exactly the set the runtime helper admits, so the // two can never disagree about which reads are fast. - let lazy_length_plausible = ctx - .block() - .icmp_ule(I64, &lazy_length_i64, "100000000"); + let lazy_length_plausible = ctx.block().icmp_ule(I64, &lazy_length_i64, "100000000"); let lazy_ok = ctx.block().and(I1, &lazy_is_array, &lazy_not_fwd); let lazy_ok = ctx.block().and(I1, &lazy_ok, &lazy_no_descriptors); let lazy_ok = ctx.block().and(I1, &lazy_ok, &lazy_default_prototypes); diff --git a/crates/perry-runtime/src/json_tape.rs b/crates/perry-runtime/src/json_tape.rs index 63fe008497..6f292bd38c 100644 --- a/crates/perry-runtime/src/json_tape.rs +++ b/crates/perry-runtime/src/json_tape.rs @@ -1139,44 +1139,7 @@ pub struct LazyArrayHeader { pub sequential_streak: u32, } -// `cached_length` at offset 0 is a CODEGEN contract, not a layout preference: -// Perry inlines `.length` as a raw u32 load at offset 0 rather than calling -// `js_array_length`, so an unmaterialized lazy array only reports the right -// length because this field sits first. Nothing else in the tree enforced -// that — the guarantee lived in a doc comment — so a field reordered into -// the front would have produced silently wrong `.length` values with every -// test still green. Adding a field to this struct is the moment that can -// happen, so pin it here. -const _: () = assert!( - std::mem::offset_of!(LazyArrayHeader, cached_length) == 0, - "LazyArrayHeader::cached_length must stay at offset 0 — codegen inlines \ - `.length` as a raw u32 load there" -); - -// `materialized` is the second codegen contract on this struct. The indexed -// inline cache (`perry-codegen` `expr/index_get/inline_dyn_typed_array.rs`) -// reads this slot directly to serve `lazy[i]` without a runtime call, exactly -// as `cached_read::lazy_get` does. A reordered field would send that fast path -// at an unrelated word, so pin the offset the same way `cached_length` is. -const _: () = assert!( - std::mem::offset_of!(LazyArrayHeader, materialized) == 32, - "LazyArrayHeader::materialized must stay at offset 32 — the indexed inline \ - cache loads the installed array from that word" -); - -// The sparse tier of that same cache probes the per-element cache directly: -// bitmap bit first, then the parallel element slot. Both offsets are read as -// raw words from emitted code, so neither may drift either. -const _: () = assert!( - std::mem::offset_of!(LazyArrayHeader, materialized_elements) == 40, - "LazyArrayHeader::materialized_elements must stay at offset 40 — the \ - indexed inline cache loads a cached element from that word" -); -const _: () = assert!( - std::mem::offset_of!(LazyArrayHeader, materialized_bitmap) == 48, - "LazyArrayHeader::materialized_bitmap must stay at offset 48 — the indexed \ - inline cache proves a cached element live from that word" -); +mod layout; /// #7478: how long a run of consecutive ascending cold reads has to get /// before we stop materializing element-by-element and hand the whole diff --git a/crates/perry-runtime/src/json_tape/layout.rs b/crates/perry-runtime/src/json_tape/layout.rs new file mode 100644 index 0000000000..26a0323364 --- /dev/null +++ b/crates/perry-runtime/src/json_tape/layout.rs @@ -0,0 +1,50 @@ +//! Layout contracts on [`LazyArrayHeader`] that emitted code depends on. +//! +//! Perry's codegen reads these words directly — `.length` as a raw u32 at +//! offset 0, and the indexed inline cache's lazy tiers at the three pointer +//! slots below — instead of calling into the runtime. A field reordered in +//! front of any of them would send emitted code at an unrelated word with +//! every test still green, so each offset is pinned here at compile time. +//! The doc comments on the fields themselves say *why* each is load-bearing; +//! this module is only the enforcement. + +use super::LazyArrayHeader; + +// `cached_length` at offset 0 is a CODEGEN contract, not a layout preference: +// Perry inlines `.length` as a raw u32 load at offset 0 rather than calling +// `js_array_length`, so an unmaterialized lazy array only reports the right +// length because this field sits first. Nothing else in the tree enforced +// that — the guarantee lived in a doc comment — so a field reordered into +// the front would have produced silently wrong `.length` values with every +// test still green. Adding a field to this struct is the moment that can +// happen, so pin it here. +const _: () = assert!( + std::mem::offset_of!(LazyArrayHeader, cached_length) == 0, + "LazyArrayHeader::cached_length must stay at offset 0 — codegen inlines \ + `.length` as a raw u32 load there" +); + +// `materialized` is the second codegen contract on this struct. The indexed +// inline cache (`perry-codegen` `expr/index_get/inline_dyn_typed_array.rs`) +// reads this slot directly to serve `lazy[i]` without a runtime call, exactly +// as `cached_read::lazy_get` does. A reordered field would send that fast path +// at an unrelated word, so pin the offset the same way `cached_length` is. +const _: () = assert!( + std::mem::offset_of!(LazyArrayHeader, materialized) == 32, + "LazyArrayHeader::materialized must stay at offset 32 — the indexed inline \ + cache loads the installed array from that word" +); + +// The sparse tier of that same cache probes the per-element cache directly: +// bitmap bit first, then the parallel element slot. Both offsets are read as +// raw words from emitted code, so neither may drift either. +const _: () = assert!( + std::mem::offset_of!(LazyArrayHeader, materialized_elements) == 40, + "LazyArrayHeader::materialized_elements must stay at offset 40 — the \ + indexed inline cache loads a cached element from that word" +); +const _: () = assert!( + std::mem::offset_of!(LazyArrayHeader, materialized_bitmap) == 48, + "LazyArrayHeader::materialized_bitmap must stay at offset 48 — the indexed \ + inline cache proves a cached element live from that word" +); From 69002c6c21b325680d445268c2112be0edd31a2b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ralph=20K=C3=BCpper?= Date: Sat, 12 Sep 2026 06:50:19 +0200 Subject: [PATCH 06/10] test(json): split the lazy defineProperty gap out of the cache fixture The indexed-cache fixture asserted that an accessor descriptor installed on an index takes reads off the fast path. main cannot do that at all: in both the sparse and the materialized state the descriptor is installed and then ignored, and only PERRY_JSON_TAPE=0 matches Node. The assertion therefore failed identically on both arms and told us nothing about the new tiers. Move it to its own reproducer, covering both lazy states, and record it in the gap snapshot against #10097. The cache fixture keeps every assertion the tiers are actually responsible for. --- ...test_gap_json_lazy_defineproperty_index.ts | 33 +++++++++++++++++++ .../test_gap_json_lazy_indexed_cache.ts | 26 +++------------ test-parity/gap_snapshot.json | 7 ++++ 3 files changed, 45 insertions(+), 21 deletions(-) create mode 100644 test-files/test_gap_json_lazy_defineproperty_index.ts diff --git a/test-files/test_gap_json_lazy_defineproperty_index.ts b/test-files/test_gap_json_lazy_defineproperty_index.ts new file mode 100644 index 0000000000..173cc5cdd3 --- /dev/null +++ b/test-files/test_gap_json_lazy_defineproperty_index.ts @@ -0,0 +1,33 @@ +// Object.defineProperty on an index of a JSON.parse array must be honoured by +// later reads. Passes with PERRY_JSON_TAPE=0 (direct parse) and fails in the +// default lazy route: the accessor is installed but indexed reads keep +// returning the element, in both the sparse and the materialized state. +// Known gap: #10097. +const pieces: string[] = []; +for (let i = 0; i < 200; i++) { + pieces.push('{"id":' + i + ',"name":"heap string for record ' + i + '"}'); +} +const text = "[" + pieces.join(",") + "]"; +let sum = 0; +for (let round = 0; round < 4; round++) { + // sparse (never scanned) and materialized (scanned) both must honour it + for (const scan of [false, true]) { + const rows: any = JSON.parse(text); + if (scan) { let seen = 0; for (let i = 0; i < rows.length; i++) seen += rows[i].id; sum += seen; } + let getterCalls = 0; + Object.defineProperty(rows, 5, { + configurable: true, + get: function () { getterCalls++; return {id: -5, name: "from getter"}; }, + }); + for (let repeat = 0; repeat < 8; repeat++) { + const value: any = rows[5]; + if (value.id !== -5 || value.name !== "from getter") { + throw new Error("descriptor read bypassed (scan=" + scan + ")"); + } + if (rows[6].id !== 6) throw new Error("neighbour read broken"); + sum += value.id; + } + if (getterCalls !== 8) throw new Error("getter calls: " + getterCalls); + } +} +console.log("lazy-defineproperty-index", sum); diff --git a/test-files/test_gap_json_lazy_indexed_cache.ts b/test-files/test_gap_json_lazy_indexed_cache.ts index 09ecca106e..078645ac2c 100644 --- a/test-files/test_gap_json_lazy_indexed_cache.ts +++ b/test-files/test_gap_json_lazy_indexed_cache.ts @@ -1,9 +1,11 @@ // The indexed inline cache serves reads of a MATERIALIZED lazy JSON array. // Everything here is about the guards that let it do so safely: the array must // still be an ordinary unforwarded Array, its length mirror on the lazy header -// must agree, holes and descriptors must fall back, and a prototype override -// must disable the fast path. Run in auto/tape/direct modes, including -// scheduled moving GC. +// must agree, holes must fall back, and a prototype override must disable the +// fast path. Run in auto/tape/direct modes, including scheduled moving GC. +// (An accessor descriptor on one index is test_gap_json_lazy_defineproperty_index.ts: +// main cannot honour it on a lazy array in any parser mode, and that is +// tracked as its own gap.) const pieces: string[] = []; for (let i = 0; i < 200; i++) { pieces.push('{"id":' + i + ',"name":"heap string for record ' + i + '"}'); @@ -54,24 +56,6 @@ for (let round = 0; round < 40; round++) { rows.length = 64; if (rows[48] !== undefined) throw new Error("hole must read undefined"); - // A descriptor override on one index must take every read off the cache. - const described: any = JSON.parse(text); - fullyMaterialize(described); - let getterCalls = 0; - Object.defineProperty(described, 5, { - configurable: true, - get: function () { getterCalls++; return {id: -5, name: "from getter"}; }, - }); - for (let repeat = 0; repeat < 8; repeat++) { - const value: any = described[5]; - if (value.id !== -5 || value.name !== "from getter") { - throw new Error("descriptor read bypassed"); - } - if (described[6].id !== 6) throw new Error("neighbour read broken"); - sum += value.id; - } - if (getterCalls !== 8) throw new Error("getter calls: " + getterCalls); - // Allocate between passes so scheduled moving GC also covers reads taken // after the installed array has moved. const churn: any = JSON.parse('{"name":"pass ' + round + '"}'); diff --git a/test-parity/gap_snapshot.json b/test-parity/gap_snapshot.json index 46723c64a5..10423a89f8 100644 --- a/test-parity/gap_snapshot.json +++ b/test-parity/gap_snapshot.json @@ -24,6 +24,13 @@ "category": "bug-open", "reason": "process SIGINT trace hook gap; standing per #5917 diff." }, + "test_gap_json_lazy_defineproperty_index": { + "status": "parity_fail", + "issue": "10097", + "added": "2026-09-12", + "category": "bug-open", + "reason": "Object.defineProperty index accessor on a JSON.parse lazy array is installed but bypassed by indexed reads; passes with PERRY_JSON_TAPE=0" + }, "test_gap_perfhooks_3088_3008_3010_3011": { "status": "parity_fail", "issue": "3088", From b3bc4d7a909f0fe3e6701b48e00c68f66bd15e65 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ralph=20K=C3=BCpper?= Date: Sat, 12 Sep 2026 07:22:16 +0200 Subject: [PATCH 07/10] perf(codegen): outline the lazy read proof into one runtime probe Inlining the whole lazy proof at every indexed read site made emitted code measurably worse on rows it never executes on. The 50-row screen against the pre-change compiler showed six separated regressions, worst string_a:parse +5.08% and null:parse +3.14% -- rows with no array in them at all -- and the access screen showed 20 MiB field walks +4.78%, on a receiver that is an ordinary Array and never enters these blocks. run() in the JSON worker grew 10752 to 11804 bytes; retired instructions moved +0.85% while CPU moved +4.8%, the signature of instruction fetch and prediction rather than executed work. Replace both tiers with a single call to js_lazy_array_index_probe, which is lazy_get's two non-allocating branches and nothing else. Emitted code per read site drops from ~87 instructions to a tag test, a call and a result test. The dispatcher chain (js_packed_arraylike_index_get -> js_array_get_f64 -> lazy_get) is still skipped; only the proof moves out of line. TAG_HOLE is the declined signal -- unambiguous, because a hole is never a value a read yields and holes already route to the miss helper. Cold elements, descriptors, out-of-bounds, growth stubs and a stale length mirror all come back as that. The probe is classified CannotCollect: it reads headers, bitmap and slots, never allocates a managed value, never enters user code, and deliberately omits lazy_get's rooted fallback. The three pointer-word offset pins go away with the inline form, since the cache no longer emits those offsets; only the pre-existing cached_length contract stays pinned. --- .../expr/index_get/inline_dyn_typed_array.rs | 267 ++++-------------- .../src/expr/index_get_claim_tests.rs | 12 +- crates/perry-codegen/src/gc_call_effects.rs | 6 + .../perry-codegen/src/runtime_decls/arrays.rs | 5 + .../src/json_tape/cached_read.rs | 79 ++++++ crates/perry-runtime/src/json_tape/layout.rs | 41 +-- 6 files changed, 151 insertions(+), 259 deletions(-) diff --git a/crates/perry-codegen/src/expr/index_get/inline_dyn_typed_array.rs b/crates/perry-codegen/src/expr/index_get/inline_dyn_typed_array.rs index 0be52b154a..6f989e12d8 100644 --- a/crates/perry-codegen/src/expr/index_get/inline_dyn_typed_array.rs +++ b/crates/perry-codegen/src/expr/index_get/inline_dyn_typed_array.rs @@ -476,231 +476,16 @@ pub(super) fn lower_inline_dyn_typed_array_get( // The blocks are declared here; they are reached from `arrlike.elem.kind` // below, after the ordinary-Array and elements-subclass probes both miss. let lazy_kind_idx = ctx.new_block("arrlike.lazy.kind"); - let lazy_header_idx = ctx.new_block("arrlike.lazy.header"); - let lazy_guard_idx = ctx.new_block("arrlike.lazy.guard"); - let lazy_load_idx = ctx.new_block("arrlike.lazy.load"); - let lazy_sparse_bounds_idx = ctx.new_block("arrlike.lazy.sparse.bounds"); - let lazy_sparse_probe_idx = ctx.new_block("arrlike.lazy.sparse.probe"); - let lazy_sparse_bit_idx = ctx.new_block("arrlike.lazy.sparse.bit"); - let lazy_sparse_load_idx = ctx.new_block("arrlike.lazy.sparse.load"); - let lazy_sparse_value_idx = ctx.new_block("arrlike.lazy.sparse.value"); - let lazy_sparse_bounds_label = ctx.block_label(lazy_sparse_bounds_idx); - let lazy_sparse_probe_label = ctx.block_label(lazy_sparse_probe_idx); - let lazy_sparse_bit_label = ctx.block_label(lazy_sparse_bit_idx); - let lazy_sparse_load_label = ctx.block_label(lazy_sparse_load_idx); - let lazy_sparse_value_label = ctx.block_label(lazy_sparse_value_idx); + let lazy_call_idx = ctx.new_block("arrlike.lazy.call"); + let lazy_value_idx = ctx.new_block("arrlike.lazy.value"); let lazy_kind_label = ctx.block_label(lazy_kind_idx); - let lazy_header_label = ctx.block_label(lazy_header_idx); - let lazy_guard_label = ctx.block_label(lazy_guard_idx); - let lazy_load_label = ctx.block_label(lazy_load_idx); + let lazy_call_label = ctx.block_label(lazy_call_idx); + let lazy_value_label = ctx.block_label(lazy_value_idx); ctx.current_block = object_brand_idx; ctx.block() .cond_br(&is_array, &object_array_guard_label, &elem_kind_label); - // `GC_TYPE_LAZY_ARRAY` (perry-runtime `gc/types.rs`). This tier hangs off - // the Array-subclass probe's miss edge rather than off `brand`, so the - // ordinary-Array path keeps exactly the control flow it had: measured on - // the 20 MiB fixture (above the lazy admission bound, so a plain Array), - // routing `brand`'s not-array edge through here cost +4 retired - // instructions per read on that untouched path. - ctx.current_block = lazy_kind_idx; - let lazy_is_lazy = ctx.block().icmp_eq(I8, &gc_type, "9"); - ctx.block() - .cond_br(&lazy_is_lazy, &lazy_header_label, &object_miss_label); - - // `LazyArrayHeader::materialized` is word 4 (offset 32; pinned by a const - // assert in perry-runtime `json_tape.rs`). Null means the array is still - // tape-backed and only the sparse per-element cache can answer, which - // needs the bitmap probe in `lazy_get` — keep that on the miss path. - ctx.current_block = lazy_header_idx; - let lazy_materialized_addr = ctx.block().add(I64, &object_raw, "32"); - let lazy_materialized_ptr = ctx.block().inttoptr(I64, &lazy_materialized_addr); - let lazy_materialized = ctx.block().load(I64, &lazy_materialized_ptr); - let lazy_has_array = ctx.block().icmp_ne(I64, &lazy_materialized, "0"); - ctx.block().cond_br( - &lazy_has_array, - &lazy_guard_label, - &lazy_sparse_bounds_label, - ); - - // Still tape-backed: the sparse per-element cache is the only thing that - // can answer without materializing a subtree, and it is what a repeated or - // clustered read pattern actually hits — an array small enough that the - // adaptive walk never trips the full-materialization flip stays here for - // the life of the program. `lazy_get`'s own sparse branch is three loads - // and a bit test, so inline exactly that and leave every cold read, hole - // and out-of-bounds index to the miss helper. - // - // Out-of-bounds deliberately does NOT shortcut to `undefined` here even - // though `lazy_get` does: the prototype chain is the miss helper's job. - ctx.current_block = lazy_sparse_bounds_idx; - let sparse_invalidated = ctx - .block() - .load_volatile(I8, "@PERRY_ARRAY_INDEX_FAST_PATH_INVALIDATED"); - let sparse_default_prototypes = ctx.block().icmp_eq(I8, &sparse_invalidated, "0"); - let sparse_length_ptr = ctx.block().inttoptr(I64, &object_raw); - let sparse_length = ctx.block().load(I32, &sparse_length_ptr); - let sparse_length_i64 = ctx.block().zext(I32, &sparse_length, I64); - let sparse_in_bounds = ctx - .block() - .icmp_ult(I64, &object_idx_i64, &sparse_length_i64); - let sparse_bounds_ok = ctx - .block() - .and(I1, &sparse_default_prototypes, &sparse_in_bounds); - ctx.block().cond_br( - &sparse_bounds_ok, - &lazy_sparse_probe_label, - &object_miss_label, - ); - - // `materialized_bitmap` is word 6 and `materialized_elements` word 5 - // (offsets 48 and 40; both pinned by const asserts in perry-runtime - // `json_tape.rs`). A null on either side means nothing has been cached yet. - ctx.current_block = lazy_sparse_probe_idx; - let sparse_bitmap_addr = ctx.block().add(I64, &object_raw, "48"); - let sparse_bitmap_slot = ctx.block().inttoptr(I64, &sparse_bitmap_addr); - let sparse_bitmap = ctx.block().load(I64, &sparse_bitmap_slot); - let sparse_elements_addr = ctx.block().add(I64, &object_raw, "40"); - let sparse_elements_slot = ctx.block().inttoptr(I64, &sparse_elements_addr); - let sparse_elements = ctx.block().load(I64, &sparse_elements_slot); - let sparse_has_bitmap = ctx.block().icmp_ne(I64, &sparse_bitmap, "0"); - let sparse_has_elements = ctx.block().icmp_ne(I64, &sparse_elements, "0"); - let sparse_probe_ok = ctx - .block() - .and(I1, &sparse_has_bitmap, &sparse_has_elements); - ctx.block() - .cond_br(&sparse_probe_ok, &lazy_sparse_bit_label, &object_miss_label); - - // The bitmap is the authoritative "this slot holds a materialized value" - // signal — `JSValue::ZERO` is a legal cached value, so a null/zero element - // word cannot be used as the liveness test. - ctx.current_block = lazy_sparse_bit_idx; - let sparse_word_index = ctx.block().lshr(I64, &object_idx_i64, "6"); - let sparse_word_offset = ctx.block().shl(I64, &sparse_word_index, "3"); - let sparse_word_addr = ctx.block().add(I64, &sparse_bitmap, &sparse_word_offset); - let sparse_word_ptr = ctx.block().inttoptr(I64, &sparse_word_addr); - let sparse_word = ctx.block().load(I64, &sparse_word_ptr); - let sparse_bit_index = ctx.block().and(I64, &object_idx_i64, "63"); - let sparse_shifted = ctx.block().lshr(I64, &sparse_word, &sparse_bit_index); - let sparse_bit = ctx.block().and(I64, &sparse_shifted, "1"); - let sparse_cached = ctx.block().icmp_ne(I64, &sparse_bit, "0"); - ctx.block() - .cond_br(&sparse_cached, &lazy_sparse_load_label, &object_miss_label); - - // A bitmap-set slot always holds a real materialized JSValue, so the hole - // test below can never fire; keep it anyway, since its only effect is to - // route an impossible value to the same helper that would have produced it. - ctx.current_block = lazy_sparse_load_idx; - let sparse_elem_offset = ctx.block().shl(I64, &object_idx_i64, "3"); - let sparse_elem_addr = ctx.block().add(I64, &sparse_elements, &sparse_elem_offset); - let sparse_elem_ptr = ctx.block().inttoptr(I64, &sparse_elem_addr); - let sparse_raw = ctx.block().load(DOUBLE, &sparse_elem_ptr); - let sparse_raw_bits = ctx.block().bitcast_double_to_i64(&sparse_raw); - let sparse_is_hole = ctx - .block() - .icmp_eq(I64, &sparse_raw_bits, crate::nanbox::TAG_HOLE_I64); - ctx.block().cond_br( - &sparse_is_hole, - &object_miss_label, - &lazy_sparse_value_label, - ); - ctx.current_block = lazy_sparse_value_idx; - let sparse_value = if coerce_slow_to_number { - ctx.block() - .call(DOUBLE, "js_number_coerce", &[(DOUBLE, &sparse_raw)]) - } else { - sparse_raw - }; - let sparse_end_label = ctx.block().label.clone(); - ctx.block().br(&merge_label); - kind_incoming.push((sparse_value, sparse_end_label)); - - // The same proof `cached_read::lazy_get` takes before its inline load: a - // live unforwarded ordinary Array, no descriptor overrides, no prototype - // invalidation, and a dense in-capacity index. A growth-forwarding stub - // keeps its own GC header and fails `obj_type`/`FORWARDED`, so it routes - // to the resolver exactly as before (#9717). - // - // `cached_length` is the length mirror codegen reads for `.length` at - // offset 0. `lazy_get` refreshes it when it takes this path; the cache - // cannot write, so it instead requires the mirror to already agree and - // sends a disagreement to the miss helper — which refreshes it, making the - // next read hit. That keeps a grown or shrunk array from reporting a stale - // length through a fast-path read. - ctx.current_block = lazy_guard_idx; - let lazy_type_addr = ctx.block().sub(I64, &lazy_materialized, "8"); - let lazy_type_ptr = ctx.block().inttoptr(I64, &lazy_type_addr); - let lazy_type = ctx.block().load(I8, &lazy_type_ptr); - let lazy_is_array = ctx.block().icmp_eq(I8, &lazy_type, "1"); - let lazy_flags_addr = ctx.block().sub(I64, &lazy_materialized, "7"); - let lazy_flags_ptr = ctx.block().inttoptr(I64, &lazy_flags_addr); - let lazy_flags = ctx.block().load(I8, &lazy_flags_ptr); - let lazy_fwd = ctx.block().and(I8, &lazy_flags, "128"); - let lazy_not_fwd = ctx.block().icmp_eq(I8, &lazy_fwd, "0"); - let lazy_reserved_addr = ctx.block().sub(I64, &lazy_materialized, "6"); - let lazy_reserved_ptr = ctx.block().inttoptr(I64, &lazy_reserved_addr); - let lazy_reserved = ctx.block().load(I16, &lazy_reserved_ptr); - let lazy_descriptor_bits = ctx.block().and(I16, &lazy_reserved, "1024"); - let lazy_no_descriptors = ctx.block().icmp_eq(I16, &lazy_descriptor_bits, "0"); - let lazy_invalidated = ctx - .block() - .load_volatile(I8, "@PERRY_ARRAY_INDEX_FAST_PATH_INVALIDATED"); - let lazy_default_prototypes = ctx.block().icmp_eq(I8, &lazy_invalidated, "0"); - let lazy_array_ptr = ctx.block().inttoptr(I64, &lazy_materialized); - let lazy_length = ctx.block().load(I32, &lazy_array_ptr); - let lazy_capacity_addr = ctx.block().add(I64, &lazy_materialized, "4"); - let lazy_capacity_ptr = ctx.block().inttoptr(I64, &lazy_capacity_addr); - let lazy_capacity = ctx.block().load(I32, &lazy_capacity_ptr); - let lazy_mirror_ptr = ctx.block().inttoptr(I64, &object_raw); - let lazy_mirror = ctx.block().load(I32, &lazy_mirror_ptr); - let lazy_mirror_fresh = ctx.block().icmp_eq(I32, &lazy_mirror, &lazy_length); - let lazy_length_i64 = ctx.block().zext(I32, &lazy_length, I64); - let lazy_capacity_i64 = ctx.block().zext(I32, &lazy_capacity, I64); - let lazy_in_bounds = ctx.block().icmp_ult(I64, &object_idx_i64, &lazy_length_i64); - let lazy_within_capacity = ctx - .block() - .icmp_ule(I64, &lazy_length_i64, &lazy_capacity_i64); - // `lazy_get`'s own plausibility bound on an installed array. Keeping it - // makes this cache admit exactly the set the runtime helper admits, so the - // two can never disagree about which reads are fast. - let lazy_length_plausible = ctx.block().icmp_ule(I64, &lazy_length_i64, "100000000"); - let lazy_ok = ctx.block().and(I1, &lazy_is_array, &lazy_not_fwd); - let lazy_ok = ctx.block().and(I1, &lazy_ok, &lazy_no_descriptors); - let lazy_ok = ctx.block().and(I1, &lazy_ok, &lazy_default_prototypes); - let lazy_ok = ctx.block().and(I1, &lazy_ok, &lazy_mirror_fresh); - let lazy_ok = ctx.block().and(I1, &lazy_ok, &lazy_in_bounds); - let lazy_ok = ctx.block().and(I1, &lazy_ok, &lazy_within_capacity); - let lazy_ok = ctx.block().and(I1, &lazy_ok, &lazy_length_plausible); - ctx.block() - .cond_br(&lazy_ok, &lazy_load_label, &object_miss_label); - - // A hole must still consult the prototype chain, so it keeps the complete - // dispatcher rather than becoming `undefined` here. - ctx.current_block = lazy_load_idx; - let lazy_element_word = ctx.block().add(I64, &object_idx_i64, "1"); - let lazy_element_ptr = - ctx.block() - .gep_inbounds(I64, &lazy_array_ptr, &[(I64, &lazy_element_word)]); - let lazy_raw = ctx.block().load(DOUBLE, &lazy_element_ptr); - let lazy_raw_bits = ctx.block().bitcast_double_to_i64(&lazy_raw); - let lazy_is_hole = ctx - .block() - .icmp_eq(I64, &lazy_raw_bits, crate::nanbox::TAG_HOLE_I64); - let lazy_value_idx = ctx.new_block("arrlike.lazy.value"); - let lazy_value_label = ctx.block_label(lazy_value_idx); - ctx.block() - .cond_br(&lazy_is_hole, &object_miss_label, &lazy_value_label); - ctx.current_block = lazy_value_idx; - let lazy_value = if coerce_slow_to_number { - ctx.block() - .call(DOUBLE, "js_number_coerce", &[(DOUBLE, &lazy_raw)]) - } else { - lazy_raw - }; - let lazy_end_label = ctx.block().label.clone(); - ctx.block().br(&merge_label); - kind_incoming.push((lazy_value, lazy_end_label)); ctx.current_block = elem_kind_idx; let elem_is_object = ctx.block().icmp_eq(I8, &gc_type, "2"); ctx.block() @@ -770,6 +555,50 @@ pub(super) fn lower_inline_dyn_typed_array_get( ctx.block().br(&merge_label); kind_incoming.push((elem_value, elem_end_label)); + // `GC_TYPE_LAZY_ARRAY` (perry-runtime `gc/types.rs`). This tier hangs off + // the Array-subclass probe's miss edge, after the ordinary-Array and + // elements-subclass probes have both declined the receiver. + ctx.current_block = lazy_kind_idx; + let lazy_is_lazy = ctx.block().icmp_eq(I8, &gc_type, "9"); + ctx.block() + .cond_br(&lazy_is_lazy, &lazy_call_label, &object_miss_label); + + // One call into `json_tape::cached_read::js_lazy_array_index_probe`, which + // is `lazy_get`'s two non-allocating branches and nothing else. That skips + // the dispatcher chain (`js_packed_arraylike_index_get` -> + // `js_array_get_f64` -> `lazy_get`) without inlining the whole proof at + // every indexed read site: the inline form grew this function ~10% and cost + // rows it never executes on up to 5% to code layout alone. + // + // `TAG_HOLE` means "this read needs the rooted accessor" -- unambiguous, + // because a hole is never a value a read yields, and holes already route to + // the miss helper. Cold elements, descriptors, out-of-bounds, growth stubs + // and a stale length mirror all come back as that. The probe cannot + // allocate, run user code or collect, so no extra rooting is required here. + ctx.current_block = lazy_call_idx; + let lazy_raw_i64 = object_raw.clone(); + let lazy_probe = ctx.block().call( + DOUBLE, + "js_lazy_array_index_probe", + &[(I64, &lazy_raw_i64), (I64, &object_idx_i64)], + ); + let lazy_probe_bits = ctx.block().bitcast_double_to_i64(&lazy_probe); + let lazy_declined = ctx + .block() + .icmp_eq(I64, &lazy_probe_bits, crate::nanbox::TAG_HOLE_I64); + ctx.block() + .cond_br(&lazy_declined, &object_miss_label, &lazy_value_label); + ctx.current_block = lazy_value_idx; + let lazy_value = if coerce_slow_to_number { + ctx.block() + .call(DOUBLE, "js_number_coerce", &[(DOUBLE, &lazy_probe)]) + } else { + lazy_probe + }; + let lazy_end_label = ctx.block().label.clone(); + ctx.block().br(&merge_label); + kind_incoming.push((lazy_value, lazy_end_label)); + // Ordinary Array: the receiver tag and forwarding state were checked in // the predecessor. Reject descriptors or any process-wide prototype // invalidation, then prove a dense in-capacity index before loading the diff --git a/crates/perry-codegen/src/expr/index_get_claim_tests.rs b/crates/perry-codegen/src/expr/index_get_claim_tests.rs index 7322c59248..3670cd0ce3 100644 --- a/crates/perry-codegen/src/expr/index_get_claim_tests.rs +++ b/crates/perry-codegen/src/expr/index_get_claim_tests.rs @@ -176,16 +176,12 @@ fn unknown_numeric_read_guards_dense_subclass_families_and_spilled_length() { "the live length and cached dense-prefix bound must retain a semantic side exit:\n{ir}" ); assert!( - ir.contains("arrlike.lazy.guard") && ir.contains("arrlike.lazy.load"), - "a materialized lazy JSON array must be readable without leaving the cache:\n{ir}" + ir.contains("arrlike.lazy.kind") && ir.contains("js_lazy_array_index_probe"), + "a lazy JSON array must reach its probe from the cache, not the dispatcher:\n{ir}" ); assert!( - ir.contains("@PERRY_ARRAY_INDEX_FAST_PATH_INVALIDATED"), - "the lazy tier must honour process-wide prototype invalidation:\n{ir}" - ); - assert!( - ir.contains("arrlike.lazy.sparse.bit") && ir.contains("arrlike.lazy.sparse.load"), - "a tape-backed lazy array must probe its per-element cache inline:\n{ir}" + !ir.contains("arrlike.lazy.sparse") && !ir.contains("arrlike.lazy.guard"), + "the lazy proof belongs in the probe, not inlined at every read site:\n{ir}" ); } diff --git a/crates/perry-codegen/src/gc_call_effects.rs b/crates/perry-codegen/src/gc_call_effects.rs index 69280c68eb..e1e8d17fe4 100644 --- a/crates/perry-codegen/src/gc_call_effects.rs +++ b/crates/perry-codegen/src/gc_call_effects.rs @@ -60,6 +60,12 @@ pub(crate) fn classify_direct_callee(name: &str) -> GcCallEffect { // `array/subclass.rs`: scalar descriptor/header comparison only. It // neither allocates nor enters user code; a miss returns zero. | "js_packed_arraylike_loop_revalidate_live" + // `json_tape/cached_read.rs`: reads an already-materialized lazy JSON + // element. Header/bitmap/slot loads only -- it is `lazy_get`'s two + // non-allocating branches with the rooted fallback deliberately left + // out, so every case it cannot serve returns TAG_HOLE and the emitted + // code takes its ordinary miss call instead. + | "js_lazy_array_index_probe" // `gc/roots/temp_roots.rs`: TLS vector operations and an incremental // marking barrier only. They never run a Perry collection. | "js_gc_temp_root_push" diff --git a/crates/perry-codegen/src/runtime_decls/arrays.rs b/crates/perry-codegen/src/runtime_decls/arrays.rs index b7b8e3123f..f8b78d9708 100644 --- a/crates/perry-codegen/src/runtime_decls/arrays.rs +++ b/crates/perry-codegen/src/runtime_decls/arrays.rs @@ -50,6 +50,11 @@ pub fn declare_phase_b_arrays(module: &mut LlModule) { // Refs #488: bulk push for `arr.push(...src)` spread call. module.declare_function("js_array_push_spread_f64", I64, &[I64, I64]); module.declare_function("js_array_get_f64", DOUBLE, &[I64, I32]); + // The indexed inline cache's lazy tier: `lazy_get`'s two non-allocating + // branches, reached once the receiver's GC header proves a live + // GC_TYPE_LAZY_ARRAY. Returns the element, or TAG_HOLE when the read needs + // the rooted accessor and the cache must take its ordinary miss call. + module.declare_function("js_lazy_array_index_probe", DOUBLE, &[I64, I64]); // repsel #7480 / #5093: the element-shape versioned loop's preheader // guard. Establishes-or-confirms the per-array homogeneous element-shape // invariant and returns the proven class id (0 = no proof). O(n) on the diff --git a/crates/perry-runtime/src/json_tape/cached_read.rs b/crates/perry-runtime/src/json_tape/cached_read.rs index e1e41db2b9..08e4e317d1 100644 --- a/crates/perry-runtime/src/json_tape/cached_read.rs +++ b/crates/perry-runtime/src/json_tape/cached_read.rs @@ -74,6 +74,85 @@ pub unsafe fn lazy_get(hdr: *mut LazyArrayHeader, i: u32) -> JSValue { super::lazy_get_rooted(hdr, i) } +/// Probe an already-materialized lazy element for emitted code. +/// +/// This is `lazy_get`'s two non-allocating branches and nothing else. It exists +/// so the indexed inline cache can skip the dispatcher chain +/// (`js_packed_arraylike_index_get` -> `js_array_get_f64` -> `lazy_get`) with +/// ONE call instead of inlining ~87 instructions at every indexed read site in +/// the program: the inline form measurably grew `run()` by 10% in the JSON +/// access benchmark and cost an untouched ordinary-Array row ~4.8% to code +/// layout alone. +/// +/// `raw` must be a live, unforwarded `GC_TYPE_LAZY_ARRAY` pointer -- the caller +/// proves that from the GC header before calling. Returns `TAG_HOLE` to mean +/// "this read needs the rooted accessor"; that is unambiguous because a hole is +/// never a value a read yields, and the caller already routes holes to its miss +/// helper. Cold elements, holes, descriptors, out-of-bounds indices, growth +/// stubs and a stale length mirror all take that exit. +/// +/// Cannot allocate a managed value, run user code or collect, so the caller +/// needs no additional rooting around it. +#[no_mangle] +pub unsafe extern "C" fn js_lazy_array_index_probe(raw: i64, idx: i64) -> f64 { + let miss = f64::from_bits(crate::value::TAG_HOLE); + if raw == 0 || !(0..=u32::MAX as i64).contains(&idx) { + return miss; + } + let hdr = raw as *mut LazyArrayHeader; + let i = idx as u32; + if (*hdr).materialized.is_null() { + // Sparse: the bitmap is the liveness test, because `JSValue::ZERO` is a + // legal cached value whose bits are all zero. + if i >= (*hdr).cached_length { + return miss; + } + let bitmap = (*hdr).materialized_bitmap; + let cache = (*hdr).materialized_elements; + if bitmap.is_null() + || cache.is_null() + || *bitmap.add(i as usize / 64) & (1u64 << (i % 64)) == 0 + { + return miss; + } + let bits = (*cache.add(i as usize)).bits(); + if bits == crate::value::TAG_HOLE { + return miss; + } + return f64::from_bits(bits); + } + // Materialized: the same proof `lazy_get` takes before its inline load. + // A growth-forwarding stub keeps its own GC header and fails these, so it + // routes to the resolver through the caller's miss path exactly as before. + let cached = (*hdr).materialized; + let header = &*cached + .cast::() + .sub(crate::gc::GC_HEADER_SIZE) + .cast::(); + if header.obj_type != crate::gc::GC_TYPE_ARRAY + || header.gc_flags & crate::gc::GC_FLAG_FORWARDED != 0 + || header._reserved & crate::gc::OBJ_FLAG_ARRAY_DESCRIPTORS != 0 + || (*cached).length > (*cached).capacity + || (*cached).length > 100_000_000 + || i >= (*cached).length + { + return miss; + } + // `lazy_get` refreshes this mirror when it serves the read; a probe must not + // write, so it declines instead and lets the rooted accessor refresh it. + // That keeps a grown or shrunk array from reporting a stale `.length`. + if (*hdr).cached_length != (*cached).length { + return miss; + } + let elements = + (cached as *const u8).add(std::mem::size_of::()) as *const u64; + let bits = *elements.add(i as usize); + if bits == crate::value::TAG_HOLE { + return miss; + } + f64::from_bits(bits) +} + #[cfg(test)] mod tests { use super::super::*; diff --git a/crates/perry-runtime/src/json_tape/layout.rs b/crates/perry-runtime/src/json_tape/layout.rs index 26a0323364..3cbd3817eb 100644 --- a/crates/perry-runtime/src/json_tape/layout.rs +++ b/crates/perry-runtime/src/json_tape/layout.rs @@ -1,12 +1,14 @@ //! Layout contracts on [`LazyArrayHeader`] that emitted code depends on. //! -//! Perry's codegen reads these words directly — `.length` as a raw u32 at -//! offset 0, and the indexed inline cache's lazy tiers at the three pointer -//! slots below — instead of calling into the runtime. A field reordered in -//! front of any of them would send emitted code at an unrelated word with -//! every test still green, so each offset is pinned here at compile time. -//! The doc comments on the fields themselves say *why* each is load-bearing; -//! this module is only the enforcement. +//! Perry's codegen reads `.length` as a raw u32 at offset 0 instead of calling +//! into the runtime. A field reordered in front of it would send emitted code +//! at an unrelated word with every test still green, so the offset is pinned +//! here at compile time. The doc comment on the field itself says *why* it is +//! load-bearing; this module is only the enforcement. +//! +//! The indexed inline cache reaches the other words through +//! `js_lazy_array_index_probe` rather than emitting their offsets, so they need +//! no pin: moving them is a plain Rust refactor the compiler checks. use super::LazyArrayHeader; @@ -23,28 +25,3 @@ const _: () = assert!( "LazyArrayHeader::cached_length must stay at offset 0 — codegen inlines \ `.length` as a raw u32 load there" ); - -// `materialized` is the second codegen contract on this struct. The indexed -// inline cache (`perry-codegen` `expr/index_get/inline_dyn_typed_array.rs`) -// reads this slot directly to serve `lazy[i]` without a runtime call, exactly -// as `cached_read::lazy_get` does. A reordered field would send that fast path -// at an unrelated word, so pin the offset the same way `cached_length` is. -const _: () = assert!( - std::mem::offset_of!(LazyArrayHeader, materialized) == 32, - "LazyArrayHeader::materialized must stay at offset 32 — the indexed inline \ - cache loads the installed array from that word" -); - -// The sparse tier of that same cache probes the per-element cache directly: -// bitmap bit first, then the parallel element slot. Both offsets are read as -// raw words from emitted code, so neither may drift either. -const _: () = assert!( - std::mem::offset_of!(LazyArrayHeader, materialized_elements) == 40, - "LazyArrayHeader::materialized_elements must stay at offset 40 — the \ - indexed inline cache loads a cached element from that word" -); -const _: () = assert!( - std::mem::offset_of!(LazyArrayHeader, materialized_bitmap) == 48, - "LazyArrayHeader::materialized_bitmap must stay at offset 48 — the indexed \ - inline cache proves a cached element live from that word" -); From 62ef8db166ab145c7dea5bf55c8b57dae6839529 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ralph=20K=C3=BCpper?= Date: Sat, 12 Sep 2026 08:29:06 +0200 Subject: [PATCH 08/10] test(json): cover the lazy index probe's decline contract The probe is the indexed cache's whole lazy fast path, so what it DECLINES matters as much as what it serves: every decline is a read the emitted code must hand to its rooted miss helper, and a decline that wrongly became a value would be a silently wrong read. Covers an uncached index, a warmed sparse hit and its still-cold neighbour, cached zero (whose NaN-boxed bits are all zero, so the bitmap rather than the element word has to prove liveness), out-of-bounds -- which must not shortcut to undefined, since the prototype chain is the caller's job -- indices outside the u32 domain, a null receiver, and a materialized read whose length mirror has gone stale, which must decline until the rooted accessor refreshes it. --- .../src/json_tape/cached_read.rs | 73 +++++++++++++++++++ 1 file changed, 73 insertions(+) diff --git a/crates/perry-runtime/src/json_tape/cached_read.rs b/crates/perry-runtime/src/json_tape/cached_read.rs index 08e4e317d1..d4058f46f8 100644 --- a/crates/perry-runtime/src/json_tape/cached_read.rs +++ b/crates/perry-runtime/src/json_tape/cached_read.rs @@ -183,6 +183,79 @@ mod tests { } } + const MISS: u64 = crate::value::TAG_HOLE; + + fn probe(hdr: *mut LazyArrayHeader, i: i64) -> u64 { + unsafe { super::js_lazy_array_index_probe(hdr as i64, i).to_bits() } + } + + /// The probe is the cache's entire lazy fast path, so what it DECLINES is + /// as load-bearing as what it serves: every decline is a read the emitted + /// code must hand to its rooted miss helper. + #[test] + fn lazy_index_probe_serves_cached_reads_and_declines_everything_else() { + let _guard = crate::gc::GcSuppressScope::new(); + let input = format!( + "[{}]", + (0..8).map(|i| i.to_string()).collect::>().join(",") + ); + unsafe { + let hdr = fixture(input.as_bytes()); + // Nothing is cached yet, so every index declines rather than + // inventing a value. + for i in 0..8 { + assert_eq!(probe(hdr, i), MISS, "uncached index {i} must decline"); + } + // A rooted read populates the sparse cache; the probe then serves + // that index and still declines its neighbours. + let warmed = lazy_get(hdr, 3); + assert!((*hdr).materialized.is_null(), "must still be tape-backed"); + assert_eq!(probe(hdr, 3), warmed.bits(), "cached index must be served"); + assert_eq!(probe(hdr, 4), MISS, "a neighbour is still uncached"); + // Zero is a legal cached value whose NaN-boxed bits are all zero, + // so the bitmap rather than the element word has to prove liveness. + let zero = lazy_get(hdr, 0); + assert_eq!(zero.bits(), JSValue::number(0.0).bits()); + assert_eq!(probe(hdr, 0), zero.bits(), "cached zero must be served"); + // Out of bounds consults the prototype chain, so it is the caller's + // job -- the probe must not shortcut it to undefined. + for i in [8, 9, 4_294_967_295] { + assert_eq!(probe(hdr, i), MISS, "out-of-bounds {i} must decline"); + } + // Indices outside the u32 domain, and a null receiver, decline too. + for i in [-1, -4096, 4_294_967_296, i64::MAX] { + assert_eq!(probe(hdr, i), MISS, "index {i} must decline"); + } + assert_eq!(probe(std::ptr::null_mut(), 0), MISS, "null must decline"); + } + } + + #[test] + fn lazy_index_probe_declines_a_stale_length_mirror_after_growth() { + let _guard = crate::gc::GcSuppressScope::new(); + let input = format!("[{}]", vec![r#"{"id":1}"#; 12].join(",")); + unsafe { + let hdr = fixture(input.as_bytes()); + let arr = force_materialize_lazy(hdr); + assert!(!(*hdr).materialized.is_null(), "must be materialized"); + let served = lazy_get(hdr, 5); + assert_eq!( + probe(hdr, 5), + served.bits(), + "materialized read must be served" + ); + // `lazy_get` refreshes the header's length mirror when it serves a + // read; the probe cannot write, so a mirror that has gone stale + // must send the read back to the rooted accessor rather than let a + // later `.length` report the old value. + let real = (*arr).length; + (*hdr).cached_length = real + 1; + assert_eq!(probe(hdr, 5), MISS, "a stale mirror must decline"); + (*hdr).cached_length = real; + assert_eq!(probe(hdr, 5), served.bits(), "a fresh mirror serves again"); + } + } + unsafe fn fixture(input: &[u8]) -> *mut LazyArrayHeader { let text = crate::string::js_string_from_bytes(input.as_ptr(), input.len() as u32); with_built_tape(input, |tape| { From aeb41638a333b6c4d84cb95e3b1906c21c1886d7 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ralph=20K=C3=BCpper?= Date: Sat, 12 Sep 2026 08:44:31 +0200 Subject: [PATCH 09/10] test(parity): ratchet the lazy defineProperty gap on both platforms scripts/parity_known_failures.py is a ratchet, not a suppression list: a gap_snapshot.json entry without a platform-applicable known_failures.json record fails the audit. Register #10097 for linux and macos with its provenance. --- test-parity/known_failures.json | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/test-parity/known_failures.json b/test-parity/known_failures.json index dea95fa392..ed881947a1 100644 --- a/test-parity/known_failures.json +++ b/test-parity/known_failures.json @@ -76,6 +76,16 @@ "category": "bug-stale", "reason": "RE-TRIAGE: tracking issue #2514 is CLOSED but this still fails (audited 2026-08-07, #7582) — needs a new issue. process SIGINT trace hook gap; standing per the #5917 diff." }, + "test_gap_json_lazy_defineproperty_index": { + "issue": "10097", + "added": "2026-09-12", + "category": "bug-open", + "reason": "Object.defineProperty on an index of a JSON.parse lazy array is installed and then ignored by indexed reads, in both the sparse and materialized states; PERRY_JSON_TAPE=0 (direct parse) matches Node. Found while adding the indexed-cache lazy tier: the assertion failed identically on the candidate and on main, so it is main's gap, not the change's. Filed as #10097.", + "platforms": [ + "linux", + "macos" + ] + }, "test_gap_perfhooks_3088_3008_3010_3011": { "issue": "3088", "added": "2026-07-04", From 1801680434ced221fda5862aa0a2f4177220b490 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ralph=20K=C3=BCpper?= Date: Sat, 12 Sep 2026 09:19:08 +0200 Subject: [PATCH 10/10] docs(changelog): record the lazy JSON array indexed-cache tier (#10114) --- changelog.d/10114-json-lazy-array-index-ic.md | 11 +++++++++++ 1 file changed, 11 insertions(+) create mode 100644 changelog.d/10114-json-lazy-array-index-ic.md diff --git a/changelog.d/10114-json-lazy-array-index-ic.md b/changelog.d/10114-json-lazy-array-index-ic.md new file mode 100644 index 0000000000..ae8a48395e --- /dev/null +++ b/changelog.d/10114-json-lazy-array-index-ic.md @@ -0,0 +1,11 @@ +**perf(codegen): serve lazy JSON array reads from the indexed inline cache.** A `JSON.parse` result carries `GC_TYPE_LAZY_ARRAY`, and the indexed inline cache's brand check only admitted `GC_TYPE_ARRAY` — so every `parsed[i]` fell through to `arrlike.ic.miss` and re-classified the same receiver three more times (`js_packed_arraylike_index_get` → `js_array_get_f64` → `json_tape::cached_read::lazy_get`), about **227 retired instructions for `rows[7].id`**. #10050 and #10064 made that last helper allocation-free; nothing had touched the dispatcher chain in front of it, which is why post-parse reads were the worst rows in the JSON matrix. + +The cache now proves a live `GC_TYPE_LAZY_ARRAY` from the GC header and makes one call to `js_lazy_array_index_probe` — `lazy_get`'s two non-allocating branches and nothing else, covering both the sparse per-element cache (an array whose adaptive walk never trips the materialization flip stays tape-backed for the life of the program: the 16 KiB fixture is 120 records) and an installed ordinary array. `TAG_HOLE` is the declined signal, unambiguous because a hole is never a value a read yields and holes already route to the miss helper; cold elements, descriptors, out-of-bounds, growth-forwarding stubs and a stale `cached_length` mirror all take that exit. `lazy_get` refreshes that mirror when it serves a read and a probe cannot write, so it instead requires the mirror to agree and declines otherwise — a grown or shrunk array can never report a stale `.length` through a fast-path read. The probe is classified `CannotCollect` and omits `lazy_get`'s rooted fallback, so the caller needs no extra rooting. + +Quiet M1/8 GiB host, Node 26.5.1, Bun 1.3.14, 7 interleaved repetitions, measured against `main` at `e8f912392`: 1 MiB repeat **−41.7%**, fields **−41.6%**, sequential **−33.4%**, random **−31.9%**; 16 KiB repeat **−41.8%**, fields **−39.8%**, random **−35.4%**, sequential **−22.8%**. `records_array_1m:random` goes from 2.11× Node to **1.47×**. Retired instructions per read drop 37.7–48.9% on those rows. Peak RSS unchanged. + +**Disclosed costs.** The 50-row screen shows nine separated regressions of +0.5–1.2% (`numbers_1m:parse +2.41%`), six of which reproduce across two independent windows, on `parse`/`sparse`/`roundtrip` rows. These come from the benchmark worker's structure rather than from parsing: `worker.ts` runs all five operations inside one `run()`, so its parse loop shares code layout and register allocation with the `scan`/`sparse` loops that do contain indexed reads. A worker whose `run()` has no indexed read at all compiles to a **byte-identical object file on both arms**, so the cost cannot reach such a function. Programs that parse and index in the same hot function will see it; programs that do not, cannot. On the access screen, 20 MiB rows (above the lazy admission bound, so ordinary Arrays that never enter the new path) show `repeat +2.97%` and `fields −2.83%`; before #10074 landed the same pair read +0.08% and +4.94%, so that cost relocates between rows when unrelated runtime changes land and is microarchitectural sensitivity on ~0.026 µs rows, not a property of this change. + +**Why one call rather than inlining the proof.** The inline form was built and measured first, then rejected: `run()` grew 10752 → 11804 bytes and the 50-row screen showed `string_a:parse +5.08%` and `null:parse +3.14%` — rows with no array in them. Outlining cuts the growth to **+56 bytes** and those rows to +0.01% and −0.07%, retaining essentially all of the instruction reduction. + +**Validation.** 234 rows — 13 lazy-array fixtures × native/shadow roots × auto/tape/direct parsers × normal/scheduled/full-GC — byte-identical to the reference on both arms, with a real moving-GC witness on every one of the 78 scheduled rows. `test_gap_json_lazy_indexed_cache.ts` covers identity, growth, shrink, holes, a prototype override and its retirement, and cached zero (all-zero NaN-boxed bits, so the bitmap rather than the element word must prove a slot live); two Rust unit tests pin the probe's decline contract. `Object.defineProperty` on a lazy index is a pre-existing gap failing identically on both arms, split into its own reproducer and ratcheted against #10097. The architectural follow-up that would delete this tier entirely is #10098.