diff --git a/changelog.d/10114-json-lazy-array-index-ic.md b/changelog.d/10114-json-lazy-array-index-ic.md new file mode 100644 index 0000000000..ae8a48395e --- /dev/null +++ b/changelog.d/10114-json-lazy-array-index-ic.md @@ -0,0 +1,11 @@ +**perf(codegen): serve lazy JSON array reads from the indexed inline cache.** A `JSON.parse` result carries `GC_TYPE_LAZY_ARRAY`, and the indexed inline cache's brand check only admitted `GC_TYPE_ARRAY` — so every `parsed[i]` fell through to `arrlike.ic.miss` and re-classified the same receiver three more times (`js_packed_arraylike_index_get` → `js_array_get_f64` → `json_tape::cached_read::lazy_get`), about **227 retired instructions for `rows[7].id`**. #10050 and #10064 made that last helper allocation-free; nothing had touched the dispatcher chain in front of it, which is why post-parse reads were the worst rows in the JSON matrix. + +The cache now proves a live `GC_TYPE_LAZY_ARRAY` from the GC header and makes one call to `js_lazy_array_index_probe` — `lazy_get`'s two non-allocating branches and nothing else, covering both the sparse per-element cache (an array whose adaptive walk never trips the materialization flip stays tape-backed for the life of the program: the 16 KiB fixture is 120 records) and an installed ordinary array. `TAG_HOLE` is the declined signal, unambiguous because a hole is never a value a read yields and holes already route to the miss helper; cold elements, descriptors, out-of-bounds, growth-forwarding stubs and a stale `cached_length` mirror all take that exit. `lazy_get` refreshes that mirror when it serves a read and a probe cannot write, so it instead requires the mirror to agree and declines otherwise — a grown or shrunk array can never report a stale `.length` through a fast-path read. The probe is classified `CannotCollect` and omits `lazy_get`'s rooted fallback, so the caller needs no extra rooting. + +Quiet M1/8 GiB host, Node 26.5.1, Bun 1.3.14, 7 interleaved repetitions, measured against `main` at `e8f912392`: 1 MiB repeat **−41.7%**, fields **−41.6%**, sequential **−33.4%**, random **−31.9%**; 16 KiB repeat **−41.8%**, fields **−39.8%**, random **−35.4%**, sequential **−22.8%**. `records_array_1m:random` goes from 2.11× Node to **1.47×**. Retired instructions per read drop 37.7–48.9% on those rows. Peak RSS unchanged. + +**Disclosed costs.** The 50-row screen shows nine separated regressions of +0.5–1.2% (`numbers_1m:parse +2.41%`), six of which reproduce across two independent windows, on `parse`/`sparse`/`roundtrip` rows. These come from the benchmark worker's structure rather than from parsing: `worker.ts` runs all five operations inside one `run()`, so its parse loop shares code layout and register allocation with the `scan`/`sparse` loops that do contain indexed reads. A worker whose `run()` has no indexed read at all compiles to a **byte-identical object file on both arms**, so the cost cannot reach such a function. Programs that parse and index in the same hot function will see it; programs that do not, cannot. On the access screen, 20 MiB rows (above the lazy admission bound, so ordinary Arrays that never enter the new path) show `repeat +2.97%` and `fields −2.83%`; before #10074 landed the same pair read +0.08% and +4.94%, so that cost relocates between rows when unrelated runtime changes land and is microarchitectural sensitivity on ~0.026 µs rows, not a property of this change. + +**Why one call rather than inlining the proof.** The inline form was built and measured first, then rejected: `run()` grew 10752 → 11804 bytes and the 50-row screen showed `string_a:parse +5.08%` and `null:parse +3.14%` — rows with no array in them. Outlining cuts the growth to **+56 bytes** and those rows to +0.01% and −0.07%, retaining essentially all of the instruction reduction. + +**Validation.** 234 rows — 13 lazy-array fixtures × native/shadow roots × auto/tape/direct parsers × normal/scheduled/full-GC — byte-identical to the reference on both arms, with a real moving-GC witness on every one of the 78 scheduled rows. `test_gap_json_lazy_indexed_cache.ts` covers identity, growth, shrink, holes, a prototype override and its retirement, and cached zero (all-zero NaN-boxed bits, so the bitmap rather than the element word must prove a slot live); two Rust unit tests pin the probe's decline contract. `Object.defineProperty` on a lazy index is a pre-existing gap failing identically on both arms, split into its own reproducer and ratcheted against #10097. The architectural follow-up that would delete this tier entirely is #10098. diff --git a/crates/perry-codegen/src/expr/index_get/inline_dyn_typed_array.rs b/crates/perry-codegen/src/expr/index_get/inline_dyn_typed_array.rs index 53c1fa6cac..6f989e12d8 100644 --- a/crates/perry-codegen/src/expr/index_get/inline_dyn_typed_array.rs +++ b/crates/perry-codegen/src/expr/index_get/inline_dyn_typed_array.rs @@ -466,13 +466,30 @@ pub(super) fn lower_inline_dyn_typed_array_get( let elem_bounds_label = ctx.block_label(elem_bounds_idx); let elem_load_label = ctx.block_label(elem_load_idx); let elem_value_label = ctx.block_label(elem_value_idx); + // A `JSON.parse` result is `GC_TYPE_LAZY_ARRAY`, not `GC_TYPE_ARRAY`, so + // every one of its indexed reads used to fall straight through to + // `arrlike.ic.miss` and re-classify the receiver three more times + // (`js_packed_arraylike_index_get` -> `js_array_get_f64` -> + // `json_tape::cached_read::lazy_get`). Once a scan or a random-access flip + // has installed the ordinary array, that whole chain resolves one word; + // serve it here instead, on exactly the proof `lazy_get` already uses. + // The blocks are declared here; they are reached from `arrlike.elem.kind` + // below, after the ordinary-Array and elements-subclass probes both miss. + let lazy_kind_idx = ctx.new_block("arrlike.lazy.kind"); + let lazy_call_idx = ctx.new_block("arrlike.lazy.call"); + let lazy_value_idx = ctx.new_block("arrlike.lazy.value"); + let lazy_kind_label = ctx.block_label(lazy_kind_idx); + let lazy_call_label = ctx.block_label(lazy_call_idx); + let lazy_value_label = ctx.block_label(lazy_value_idx); + ctx.current_block = object_brand_idx; ctx.block() .cond_br(&is_array, &object_array_guard_label, &elem_kind_label); + ctx.current_block = elem_kind_idx; let elem_is_object = ctx.block().icmp_eq(I8, &gc_type, "2"); ctx.block() - .cond_br(&elem_is_object, &elem_meta_label, &object_miss_label); + .cond_br(&elem_is_object, &elem_meta_label, &lazy_kind_label); ctx.current_block = elem_meta_idx; let elem_meta_addr = ctx.block().add(I64, &object_raw, &meta_offset); let elem_meta_slot_ptr = ctx.block().inttoptr(I64, &elem_meta_addr); @@ -538,6 +555,50 @@ pub(super) fn lower_inline_dyn_typed_array_get( ctx.block().br(&merge_label); kind_incoming.push((elem_value, elem_end_label)); + // `GC_TYPE_LAZY_ARRAY` (perry-runtime `gc/types.rs`). This tier hangs off + // the Array-subclass probe's miss edge, after the ordinary-Array and + // elements-subclass probes have both declined the receiver. + ctx.current_block = lazy_kind_idx; + let lazy_is_lazy = ctx.block().icmp_eq(I8, &gc_type, "9"); + ctx.block() + .cond_br(&lazy_is_lazy, &lazy_call_label, &object_miss_label); + + // One call into `json_tape::cached_read::js_lazy_array_index_probe`, which + // is `lazy_get`'s two non-allocating branches and nothing else. That skips + // the dispatcher chain (`js_packed_arraylike_index_get` -> + // `js_array_get_f64` -> `lazy_get`) without inlining the whole proof at + // every indexed read site: the inline form grew this function ~10% and cost + // rows it never executes on up to 5% to code layout alone. + // + // `TAG_HOLE` means "this read needs the rooted accessor" -- unambiguous, + // because a hole is never a value a read yields, and holes already route to + // the miss helper. Cold elements, descriptors, out-of-bounds, growth stubs + // and a stale length mirror all come back as that. The probe cannot + // allocate, run user code or collect, so no extra rooting is required here. + ctx.current_block = lazy_call_idx; + let lazy_raw_i64 = object_raw.clone(); + let lazy_probe = ctx.block().call( + DOUBLE, + "js_lazy_array_index_probe", + &[(I64, &lazy_raw_i64), (I64, &object_idx_i64)], + ); + let lazy_probe_bits = ctx.block().bitcast_double_to_i64(&lazy_probe); + let lazy_declined = ctx + .block() + .icmp_eq(I64, &lazy_probe_bits, crate::nanbox::TAG_HOLE_I64); + ctx.block() + .cond_br(&lazy_declined, &object_miss_label, &lazy_value_label); + ctx.current_block = lazy_value_idx; + let lazy_value = if coerce_slow_to_number { + ctx.block() + .call(DOUBLE, "js_number_coerce", &[(DOUBLE, &lazy_probe)]) + } else { + lazy_probe + }; + let lazy_end_label = ctx.block().label.clone(); + ctx.block().br(&merge_label); + kind_incoming.push((lazy_value, lazy_end_label)); + // Ordinary Array: the receiver tag and forwarding state were checked in // the predecessor. Reject descriptors or any process-wide prototype // invalidation, then prove a dense in-capacity index before loading the diff --git a/crates/perry-codegen/src/expr/index_get_claim_tests.rs b/crates/perry-codegen/src/expr/index_get_claim_tests.rs index f3ace585ac..3670cd0ce3 100644 --- a/crates/perry-codegen/src/expr/index_get_claim_tests.rs +++ b/crates/perry-codegen/src/expr/index_get_claim_tests.rs @@ -175,6 +175,14 @@ fn unknown_numeric_read_guards_dense_subclass_families_and_spilled_length() { ir.contains("arrlike.ic.range") && ir.contains("arrlike.ic.miss"), "the live length and cached dense-prefix bound must retain a semantic side exit:\n{ir}" ); + assert!( + ir.contains("arrlike.lazy.kind") && ir.contains("js_lazy_array_index_probe"), + "a lazy JSON array must reach its probe from the cache, not the dispatcher:\n{ir}" + ); + assert!( + !ir.contains("arrlike.lazy.sparse") && !ir.contains("arrlike.lazy.guard"), + "the lazy proof belongs in the probe, not inlined at every read site:\n{ir}" + ); } fn dynamic_symbol_access_ir(symbol_init: Expr, field: Option<&str>) -> String { diff --git a/crates/perry-codegen/src/gc_call_effects.rs b/crates/perry-codegen/src/gc_call_effects.rs index 69280c68eb..e1e8d17fe4 100644 --- a/crates/perry-codegen/src/gc_call_effects.rs +++ b/crates/perry-codegen/src/gc_call_effects.rs @@ -60,6 +60,12 @@ pub(crate) fn classify_direct_callee(name: &str) -> GcCallEffect { // `array/subclass.rs`: scalar descriptor/header comparison only. It // neither allocates nor enters user code; a miss returns zero. | "js_packed_arraylike_loop_revalidate_live" + // `json_tape/cached_read.rs`: reads an already-materialized lazy JSON + // element. Header/bitmap/slot loads only -- it is `lazy_get`'s two + // non-allocating branches with the rooted fallback deliberately left + // out, so every case it cannot serve returns TAG_HOLE and the emitted + // code takes its ordinary miss call instead. + | "js_lazy_array_index_probe" // `gc/roots/temp_roots.rs`: TLS vector operations and an incremental // marking barrier only. They never run a Perry collection. | "js_gc_temp_root_push" diff --git a/crates/perry-codegen/src/runtime_decls/arrays.rs b/crates/perry-codegen/src/runtime_decls/arrays.rs index b7b8e3123f..f8b78d9708 100644 --- a/crates/perry-codegen/src/runtime_decls/arrays.rs +++ b/crates/perry-codegen/src/runtime_decls/arrays.rs @@ -50,6 +50,11 @@ pub fn declare_phase_b_arrays(module: &mut LlModule) { // Refs #488: bulk push for `arr.push(...src)` spread call. module.declare_function("js_array_push_spread_f64", I64, &[I64, I64]); module.declare_function("js_array_get_f64", DOUBLE, &[I64, I32]); + // The indexed inline cache's lazy tier: `lazy_get`'s two non-allocating + // branches, reached once the receiver's GC header proves a live + // GC_TYPE_LAZY_ARRAY. Returns the element, or TAG_HOLE when the read needs + // the rooted accessor and the cache must take its ordinary miss call. + module.declare_function("js_lazy_array_index_probe", DOUBLE, &[I64, I64]); // repsel #7480 / #5093: the element-shape versioned loop's preheader // guard. Establishes-or-confirms the per-array homogeneous element-shape // invariant and returns the proven class id (0 = no proof). O(n) on the diff --git a/crates/perry-runtime/src/json_tape.rs b/crates/perry-runtime/src/json_tape.rs index ffe70a0699..6f292bd38c 100644 --- a/crates/perry-runtime/src/json_tape.rs +++ b/crates/perry-runtime/src/json_tape.rs @@ -1139,19 +1139,7 @@ pub struct LazyArrayHeader { pub sequential_streak: u32, } -// `cached_length` at offset 0 is a CODEGEN contract, not a layout preference: -// Perry inlines `.length` as a raw u32 load at offset 0 rather than calling -// `js_array_length`, so an unmaterialized lazy array only reports the right -// length because this field sits first. Nothing else in the tree enforced -// that — the guarantee lived in a doc comment — so a field reordered into -// the front would have produced silently wrong `.length` values with every -// test still green. Adding a field to this struct is the moment that can -// happen, so pin it here. -const _: () = assert!( - std::mem::offset_of!(LazyArrayHeader, cached_length) == 0, - "LazyArrayHeader::cached_length must stay at offset 0 — codegen inlines \ - `.length` as a raw u32 load there" -); +mod layout; /// #7478: how long a run of consecutive ascending cold reads has to get /// before we stop materializing element-by-element and hand the whole diff --git a/crates/perry-runtime/src/json_tape/cached_read.rs b/crates/perry-runtime/src/json_tape/cached_read.rs index e1e41db2b9..d4058f46f8 100644 --- a/crates/perry-runtime/src/json_tape/cached_read.rs +++ b/crates/perry-runtime/src/json_tape/cached_read.rs @@ -74,6 +74,85 @@ pub unsafe fn lazy_get(hdr: *mut LazyArrayHeader, i: u32) -> JSValue { super::lazy_get_rooted(hdr, i) } +/// Probe an already-materialized lazy element for emitted code. +/// +/// This is `lazy_get`'s two non-allocating branches and nothing else. It exists +/// so the indexed inline cache can skip the dispatcher chain +/// (`js_packed_arraylike_index_get` -> `js_array_get_f64` -> `lazy_get`) with +/// ONE call instead of inlining ~87 instructions at every indexed read site in +/// the program: the inline form measurably grew `run()` by 10% in the JSON +/// access benchmark and cost an untouched ordinary-Array row ~4.8% to code +/// layout alone. +/// +/// `raw` must be a live, unforwarded `GC_TYPE_LAZY_ARRAY` pointer -- the caller +/// proves that from the GC header before calling. Returns `TAG_HOLE` to mean +/// "this read needs the rooted accessor"; that is unambiguous because a hole is +/// never a value a read yields, and the caller already routes holes to its miss +/// helper. Cold elements, holes, descriptors, out-of-bounds indices, growth +/// stubs and a stale length mirror all take that exit. +/// +/// Cannot allocate a managed value, run user code or collect, so the caller +/// needs no additional rooting around it. +#[no_mangle] +pub unsafe extern "C" fn js_lazy_array_index_probe(raw: i64, idx: i64) -> f64 { + let miss = f64::from_bits(crate::value::TAG_HOLE); + if raw == 0 || !(0..=u32::MAX as i64).contains(&idx) { + return miss; + } + let hdr = raw as *mut LazyArrayHeader; + let i = idx as u32; + if (*hdr).materialized.is_null() { + // Sparse: the bitmap is the liveness test, because `JSValue::ZERO` is a + // legal cached value whose bits are all zero. + if i >= (*hdr).cached_length { + return miss; + } + let bitmap = (*hdr).materialized_bitmap; + let cache = (*hdr).materialized_elements; + if bitmap.is_null() + || cache.is_null() + || *bitmap.add(i as usize / 64) & (1u64 << (i % 64)) == 0 + { + return miss; + } + let bits = (*cache.add(i as usize)).bits(); + if bits == crate::value::TAG_HOLE { + return miss; + } + return f64::from_bits(bits); + } + // Materialized: the same proof `lazy_get` takes before its inline load. + // A growth-forwarding stub keeps its own GC header and fails these, so it + // routes to the resolver through the caller's miss path exactly as before. + let cached = (*hdr).materialized; + let header = &*cached + .cast::() + .sub(crate::gc::GC_HEADER_SIZE) + .cast::(); + if header.obj_type != crate::gc::GC_TYPE_ARRAY + || header.gc_flags & crate::gc::GC_FLAG_FORWARDED != 0 + || header._reserved & crate::gc::OBJ_FLAG_ARRAY_DESCRIPTORS != 0 + || (*cached).length > (*cached).capacity + || (*cached).length > 100_000_000 + || i >= (*cached).length + { + return miss; + } + // `lazy_get` refreshes this mirror when it serves the read; a probe must not + // write, so it declines instead and lets the rooted accessor refresh it. + // That keeps a grown or shrunk array from reporting a stale `.length`. + if (*hdr).cached_length != (*cached).length { + return miss; + } + let elements = + (cached as *const u8).add(std::mem::size_of::()) as *const u64; + let bits = *elements.add(i as usize); + if bits == crate::value::TAG_HOLE { + return miss; + } + f64::from_bits(bits) +} + #[cfg(test)] mod tests { use super::super::*; @@ -104,6 +183,79 @@ mod tests { } } + const MISS: u64 = crate::value::TAG_HOLE; + + fn probe(hdr: *mut LazyArrayHeader, i: i64) -> u64 { + unsafe { super::js_lazy_array_index_probe(hdr as i64, i).to_bits() } + } + + /// The probe is the cache's entire lazy fast path, so what it DECLINES is + /// as load-bearing as what it serves: every decline is a read the emitted + /// code must hand to its rooted miss helper. + #[test] + fn lazy_index_probe_serves_cached_reads_and_declines_everything_else() { + let _guard = crate::gc::GcSuppressScope::new(); + let input = format!( + "[{}]", + (0..8).map(|i| i.to_string()).collect::>().join(",") + ); + unsafe { + let hdr = fixture(input.as_bytes()); + // Nothing is cached yet, so every index declines rather than + // inventing a value. + for i in 0..8 { + assert_eq!(probe(hdr, i), MISS, "uncached index {i} must decline"); + } + // A rooted read populates the sparse cache; the probe then serves + // that index and still declines its neighbours. + let warmed = lazy_get(hdr, 3); + assert!((*hdr).materialized.is_null(), "must still be tape-backed"); + assert_eq!(probe(hdr, 3), warmed.bits(), "cached index must be served"); + assert_eq!(probe(hdr, 4), MISS, "a neighbour is still uncached"); + // Zero is a legal cached value whose NaN-boxed bits are all zero, + // so the bitmap rather than the element word has to prove liveness. + let zero = lazy_get(hdr, 0); + assert_eq!(zero.bits(), JSValue::number(0.0).bits()); + assert_eq!(probe(hdr, 0), zero.bits(), "cached zero must be served"); + // Out of bounds consults the prototype chain, so it is the caller's + // job -- the probe must not shortcut it to undefined. + for i in [8, 9, 4_294_967_295] { + assert_eq!(probe(hdr, i), MISS, "out-of-bounds {i} must decline"); + } + // Indices outside the u32 domain, and a null receiver, decline too. + for i in [-1, -4096, 4_294_967_296, i64::MAX] { + assert_eq!(probe(hdr, i), MISS, "index {i} must decline"); + } + assert_eq!(probe(std::ptr::null_mut(), 0), MISS, "null must decline"); + } + } + + #[test] + fn lazy_index_probe_declines_a_stale_length_mirror_after_growth() { + let _guard = crate::gc::GcSuppressScope::new(); + let input = format!("[{}]", vec![r#"{"id":1}"#; 12].join(",")); + unsafe { + let hdr = fixture(input.as_bytes()); + let arr = force_materialize_lazy(hdr); + assert!(!(*hdr).materialized.is_null(), "must be materialized"); + let served = lazy_get(hdr, 5); + assert_eq!( + probe(hdr, 5), + served.bits(), + "materialized read must be served" + ); + // `lazy_get` refreshes the header's length mirror when it serves a + // read; the probe cannot write, so a mirror that has gone stale + // must send the read back to the rooted accessor rather than let a + // later `.length` report the old value. + let real = (*arr).length; + (*hdr).cached_length = real + 1; + assert_eq!(probe(hdr, 5), MISS, "a stale mirror must decline"); + (*hdr).cached_length = real; + assert_eq!(probe(hdr, 5), served.bits(), "a fresh mirror serves again"); + } + } + unsafe fn fixture(input: &[u8]) -> *mut LazyArrayHeader { let text = crate::string::js_string_from_bytes(input.as_ptr(), input.len() as u32); with_built_tape(input, |tape| { diff --git a/crates/perry-runtime/src/json_tape/layout.rs b/crates/perry-runtime/src/json_tape/layout.rs new file mode 100644 index 0000000000..3cbd3817eb --- /dev/null +++ b/crates/perry-runtime/src/json_tape/layout.rs @@ -0,0 +1,27 @@ +//! Layout contracts on [`LazyArrayHeader`] that emitted code depends on. +//! +//! Perry's codegen reads `.length` as a raw u32 at offset 0 instead of calling +//! into the runtime. A field reordered in front of it would send emitted code +//! at an unrelated word with every test still green, so the offset is pinned +//! here at compile time. The doc comment on the field itself says *why* it is +//! load-bearing; this module is only the enforcement. +//! +//! The indexed inline cache reaches the other words through +//! `js_lazy_array_index_probe` rather than emitting their offsets, so they need +//! no pin: moving them is a plain Rust refactor the compiler checks. + +use super::LazyArrayHeader; + +// `cached_length` at offset 0 is a CODEGEN contract, not a layout preference: +// Perry inlines `.length` as a raw u32 load at offset 0 rather than calling +// `js_array_length`, so an unmaterialized lazy array only reports the right +// length because this field sits first. Nothing else in the tree enforced +// that — the guarantee lived in a doc comment — so a field reordered into +// the front would have produced silently wrong `.length` values with every +// test still green. Adding a field to this struct is the moment that can +// happen, so pin it here. +const _: () = assert!( + std::mem::offset_of!(LazyArrayHeader, cached_length) == 0, + "LazyArrayHeader::cached_length must stay at offset 0 — codegen inlines \ + `.length` as a raw u32 load there" +); diff --git a/test-files/test_gap_json_lazy_defineproperty_index.ts b/test-files/test_gap_json_lazy_defineproperty_index.ts new file mode 100644 index 0000000000..173cc5cdd3 --- /dev/null +++ b/test-files/test_gap_json_lazy_defineproperty_index.ts @@ -0,0 +1,33 @@ +// Object.defineProperty on an index of a JSON.parse array must be honoured by +// later reads. Passes with PERRY_JSON_TAPE=0 (direct parse) and fails in the +// default lazy route: the accessor is installed but indexed reads keep +// returning the element, in both the sparse and the materialized state. +// Known gap: #10097. +const pieces: string[] = []; +for (let i = 0; i < 200; i++) { + pieces.push('{"id":' + i + ',"name":"heap string for record ' + i + '"}'); +} +const text = "[" + pieces.join(",") + "]"; +let sum = 0; +for (let round = 0; round < 4; round++) { + // sparse (never scanned) and materialized (scanned) both must honour it + for (const scan of [false, true]) { + const rows: any = JSON.parse(text); + if (scan) { let seen = 0; for (let i = 0; i < rows.length; i++) seen += rows[i].id; sum += seen; } + let getterCalls = 0; + Object.defineProperty(rows, 5, { + configurable: true, + get: function () { getterCalls++; return {id: -5, name: "from getter"}; }, + }); + for (let repeat = 0; repeat < 8; repeat++) { + const value: any = rows[5]; + if (value.id !== -5 || value.name !== "from getter") { + throw new Error("descriptor read bypassed (scan=" + scan + ")"); + } + if (rows[6].id !== 6) throw new Error("neighbour read broken"); + sum += value.id; + } + if (getterCalls !== 8) throw new Error("getter calls: " + getterCalls); + } +} +console.log("lazy-defineproperty-index", sum); diff --git a/test-files/test_gap_json_lazy_indexed_cache.ts b/test-files/test_gap_json_lazy_indexed_cache.ts new file mode 100644 index 0000000000..078645ac2c --- /dev/null +++ b/test-files/test_gap_json_lazy_indexed_cache.ts @@ -0,0 +1,135 @@ +// The indexed inline cache serves reads of a MATERIALIZED lazy JSON array. +// Everything here is about the guards that let it do so safely: the array must +// still be an ordinary unforwarded Array, its length mirror on the lazy header +// must agree, holes must fall back, and a prototype override must disable the +// fast path. Run in auto/tape/direct modes, including scheduled moving GC. +// (An accessor descriptor on one index is test_gap_json_lazy_defineproperty_index.ts: +// main cannot honour it on a lazy array in any parser mode, and that is +// tracked as its own gap.) +const pieces: string[] = []; +for (let i = 0; i < 200; i++) { + pieces.push('{"id":' + i + ',"name":"heap string for record ' + i + '"}'); +} +const text = "[" + pieces.join(",") + "]"; +const retained: any[] = []; +const sparseRetained: any[] = []; +let sum = 0; + +function fullyMaterialize(rows: any): void { + // A whole-array scan trips the adaptive flip, so later reads go through the + // installed ordinary array rather than the sparse per-element cache. + let seen = 0; + for (let i = 0; i < rows.length; i++) seen += rows[i].id; + if (seen !== 19900) throw new Error("scan sum changed: " + seen); +} + +for (let round = 0; round < 40; round++) { + const rows: any = JSON.parse(text); + fullyMaterialize(rows); + + // Repeated reads off the installed array must keep identity and value. + const saved: any = rows[7]; + for (let repeat = 0; repeat < 16; repeat++) { + if (rows[7] !== saved || rows[7].id !== 7) throw new Error("identity changed"); + if (rows[199].id !== 199) throw new Error("tail value changed"); + if (rows[200] !== undefined || rows[4294967295] !== undefined) { + throw new Error("out-of-bounds read"); + } + sum += rows[7].id + rows[199].id; + } + + // Growth: the header's length mirror goes stale, so a fast-path read must + // not report the old length or miss the new element. + rows.push({id: 200, name: "grown heap string"}); + if (rows.length !== 201) throw new Error("length after growth: " + rows.length); + if (rows[200].id !== 200) throw new Error("grown element lost"); + if (rows[7] !== saved) throw new Error("identity lost across growth"); + + // Shrink, then regrow into holes. Reads of the hole region must consult the + // prototype chain rather than loading a stale slot. + rows.length = 32; + if (rows.length !== 32) throw new Error("length after shrink: " + rows.length); + if (rows[32] !== undefined || rows[200] !== undefined) { + throw new Error("stale read past shrink"); + } + if (rows[31].id !== 31) throw new Error("surviving element lost"); + rows.length = 64; + if (rows[48] !== undefined) throw new Error("hole must read undefined"); + + // Allocate between passes so scheduled moving GC also covers reads taken + // after the installed array has moved. + const churn: any = JSON.parse('{"name":"pass ' + round + '"}'); + if (churn.name !== "pass " + round) throw new Error("churn changed"); + retained.push(saved); +} + +// The SPARSE tier: an array whose adaptive walk never trips the +// full-materialization flip stays tape-backed, so repeated reads are served +// from the per-element cache instead of an installed array. Never scan this +// one -- a scan would move it onto the materialized tier above. +for (let round = 0; round < 20; round++) { + const sparse: any = JSON.parse(text); + const probes: number[] = [0, 1, 63, 64, 65, 127, 128, 199]; + const first: any[] = []; + for (let p = 0; p < probes.length; p++) first.push(sparse[probes[p]]); + for (let repeat = 0; repeat < 12; repeat++) { + for (let p = 0; p < probes.length; p++) { + const index = probes[p]; + const value: any = sparse[index]; + // Identity must hold across every repeat: a cache hit returns the + // same object, never a freshly materialized copy. + if (value !== first[p]) throw new Error("sparse identity changed at " + index); + if (value.id !== index) throw new Error("sparse value changed at " + index); + if (value.name !== "heap string for record " + index) { + throw new Error("sparse name changed at " + index); + } + sum += value.id; + } + if (sparse[200] !== undefined || sparse[4294967295] !== undefined) { + throw new Error("sparse out-of-bounds read"); + } + if (sparse[-1] !== undefined) throw new Error("negative index read"); + } + // Zero is a legal cached value and its NaN-boxed bits are all zero, so the + // bitmap -- not the element word -- has to be what proves a slot cached. + const zeros: any = JSON.parse("[0,0,0,0,0,0,0,0]"); + for (let repeat = 0; repeat < 8; repeat++) { + if (zeros[3] !== 0 || zeros[7] !== 0) throw new Error("cached zero lost"); + sum += zeros[3]; + } + // Mutating through the sparse cache must move the array off it correctly. + sparse[64] = {id: -64, name: "replacement"}; + if (sparse[64].id !== -64) throw new Error("sparse replacement lost"); + if (sparse[65] !== first[4]) throw new Error("neighbour lost across mutation"); + sparseRetained.push(first[2]); +} + +// A prototype index override must disable the fast path process-wide: a hole +// read has to find the inherited value, not undefined. +const holed: any = JSON.parse(text); +fullyMaterialize(holed); +holed.length = 8; +holed.length = 16; +(Array.prototype as any)[12] = "from prototype"; +if (holed[12] !== "from prototype") throw new Error("prototype override ignored"); +if (holed[3].id !== 3) throw new Error("dense read broken under override"); +delete (Array.prototype as any)[12]; +if (holed[12] !== undefined) throw new Error("prototype override not retired"); + +for (let round = 0; round < retained.length; round++) { + const saved: any = retained[round]; + if (saved.id !== 7 || saved.name !== "heap string for record 7") { + throw new Error("retained element changed"); + } + sum += saved.id; +} +// Elements handed out by the sparse tier must survive every later collection +// with their identity and contents intact. +for (let round = 0; round < sparseRetained.length; round++) { + const saved: any = sparseRetained[round]; + if (saved.id !== 63 || saved.name !== "heap string for record 63") { + throw new Error("retained sparse element changed"); + } + sum += saved.id; +} +console.log("lazy-indexed-cache", retained.length, sparseRetained.length, sum); diff --git a/test-parity/gap_snapshot.json b/test-parity/gap_snapshot.json index 46723c64a5..10423a89f8 100644 --- a/test-parity/gap_snapshot.json +++ b/test-parity/gap_snapshot.json @@ -24,6 +24,13 @@ "category": "bug-open", "reason": "process SIGINT trace hook gap; standing per #5917 diff." }, + "test_gap_json_lazy_defineproperty_index": { + "status": "parity_fail", + "issue": "10097", + "added": "2026-09-12", + "category": "bug-open", + "reason": "Object.defineProperty index accessor on a JSON.parse lazy array is installed but bypassed by indexed reads; passes with PERRY_JSON_TAPE=0" + }, "test_gap_perfhooks_3088_3008_3010_3011": { "status": "parity_fail", "issue": "3088", diff --git a/test-parity/known_failures.json b/test-parity/known_failures.json index dea95fa392..ed881947a1 100644 --- a/test-parity/known_failures.json +++ b/test-parity/known_failures.json @@ -76,6 +76,16 @@ "category": "bug-stale", "reason": "RE-TRIAGE: tracking issue #2514 is CLOSED but this still fails (audited 2026-08-07, #7582) — needs a new issue. process SIGINT trace hook gap; standing per the #5917 diff." }, + "test_gap_json_lazy_defineproperty_index": { + "issue": "10097", + "added": "2026-09-12", + "category": "bug-open", + "reason": "Object.defineProperty on an index of a JSON.parse lazy array is installed and then ignored by indexed reads, in both the sparse and materialized states; PERRY_JSON_TAPE=0 (direct parse) matches Node. Found while adding the indexed-cache lazy tier: the assertion failed identically on the candidate and on main, so it is main's gap, not the change's. Filed as #10097.", + "platforms": [ + "linux", + "macos" + ] + }, "test_gap_perfhooks_3088_3008_3010_3011": { "issue": "3088", "added": "2026-07-04",