perf(codegen,runtime): retire the per-access class-field latch, and gate the compiler's copy of the GC header layout

.github/workflows/test.yml

@@ -586,6 +586,24 @@ jobs:
           python3 scripts/check_node_version_consistency.py --self-test
           python3 scripts/check_node_version_consistency.py
 
+      # perry-codegen does NOT depend on perry-runtime, yet it bakes the
+      # collector's header layout into emitted code: the inline `new` path
+      # stores a packed GcHeader word as a compile-time constant (#8122) and
+      # every class-field / element-shape guard masks that word against a
+      # literal. Both sides carried their own copy of GC_TYPE_OBJECT,
+      # GC_FLAG_FORWARDED, GC_OBJ_TYPED_LAYOUT_INTACT and friends, held together
+      # only by a comment -- the `debug_assert_eq!` that looks like it checks
+      # them compares codegen's constant to a string literal (a tautology) and
+      # is compiled out of release and perry-dev anyway. A renumbering in the
+      # runtime therefore compiled clean, passed every suite, and shipped a
+      # binary whose objects the collector misreads. Build-free, so it belongs
+      # in `lint`, which IS a required context.
+      - name: GC header constant consistency
+        if: ${{ !cancelled() }}
+        run: |
+          python3 scripts/check_gc_header_constants.py --self-test
+          python3 scripts/check_gc_header_constants.py
+
       # #7341 layer 3. A RuntimeHandleScope gives an object liveness; it does
       # nothing for a raw pointer already read out of the slot. Every rooting bug
       # in the quarantine sweep had rooting ALREADY -- what was missing was

changelog.d/10646-retire-class-field-latch.md

@@ -0,0 +1,59 @@
+### Static-key class-field reads drop the per-access latch (−18% on the guard's fast path)
+
+Every static-key class-field read gated its fast path on
+`@PERRY_CLASS_FIELD_INLINE_GUARD_DISABLED`. That is an `external global`, so on
+arm64 reading it cost `adrp` + a GOT `ldr` + a dependent `ldrb` through it + a
+compare — four instructions and two dependent loads, before the guard had
+looked at the receiver at all. It could not be hoisted: the runtime flips it
+mid-execution when a descriptor or accessor lands on a class prototype, so the
+load was `volatile` by necessity.
+
+That authority now rides on a value the guard already had to load. Each class
+gains a `@perry_class_guard_shape_*` expectation, seeded at module init with
+the class ShapeId and registered with the runtime;
+`disable_class_field_inline_guard` poisons every registered slot with
+`u32::MAX`. ShapeIds are allocated from `[0x8000_0000, 0xC000_0000)` and never
+reused, so a poisoned expectation can never match a live object — every guard
+misses and routes to the IC, exactly what the latch bought.
+
+It is deliberately a SEPARATE global from `@perry_class_shape_id_*`: that one is
+stamped into every new instance by `js_object_alloc_class_inline_keys_stamped`,
+so poisoning it would brand live objects with a bogus ShapeId rather than close
+a fast path. Subclass arms and imported-class stubs carry the poisonable
+expectation too, and an imported-stub rewrite that lands after a disable
+re-poisons rather than resurrects.
+
+Measured on arm64 (`-Os` + `llc -O2 -mcpu=apple-m1`): one `o.a` on a typed
+receiver goes from 28 to 23 executed fast-path instructions (−18%) with one
+fewer dependent load; a probe making 16 reads on one receiver goes from 595.2
+to 563.0 executed instructions per call (−5.4%, `/usr/bin/time -l` instructions
+retired, differenced over iteration count). The per-read marginal is −2 rather
+than −5 because LLVM already hoisted the latch's GOT base register across
+accesses within a function. The latch is gone from `$generic`, `$spec_b` and
+the copy the inliner leaves in the caller — the last of which is the code that
+actually executes.
+
+### The compiler's copy of the GC header layout is now gated
+
+`perry-codegen` does not depend on `perry-runtime`, yet it bakes the collector's
+header layout into emitted code: the inline `new` path stores a packed
+`GcHeader` word as a compile-time constant, and every class-field /
+element-shape / method-probe guard masks that word against a literal. Thirty-six
+restatements across ten files, with the agreement held by a code comment — the
+`debug_assert_eq!` that looked like enforcement compared codegen's constant to a
+string literal, a tautology that never referenced the runtime and is compiled
+out of `release` and `perry-dev` anyway. A flag renumbered in the runtime
+compiled clean, passed every suite, and shipped a compiler whose allocator baked
+one bit layout while the collector read another.
+
+`scripts/check_gc_header_constants.py` (in `lint`) re-derives every restatement
+from the runtime constant it quotes, including composites and the fused 32-bit
+masks. A registered constant that stops existing fails, so a fix deletes its own
+entry, and a new header-shaped `const` in a watched file must be registered or
+exempted with a reason. Writing the registry found five restatements a
+module-scope grep misses, because they are declared inside function bodies.
+
+`shape_descriptor_census` gains a matching requirement: the class-field
+precheck must read its expectation VOLATILE from the poisonable global, since a
+lowering that hoisted that load would reopen a fast path the runtime has closed
+and would still satisfy a shape-only assertion.

crates/perry-codegen/src/codegen/mod.rs

@@ -1196,6 +1196,14 @@ pub fn compile_module(hir: &HirModule, opts: CompileOptions) -> Result<Vec<u8>>
             I32,
             "0",
         );
+        // The poisonable twin of the ShapeId global: same value, same linkage,
+        // but only ever COMPARED against — see
+        // `typed_shape::guard_shape_global_name_from_keys_global`.
+        llmod.add_global(
+            &crate::typed_shape::guard_shape_global_name_from_keys_global(&global_name),
+            I32,
+            "0",
+        );
         // #8122: the inline-`new` header image, composed at module init
         // (`string_pool.rs`) for the classes `class_header_images` admits.
         llmod.add_internal_global(
@@ -1389,6 +1397,11 @@ pub fn compile_module(hir: &HirModule, opts: CompileOptions) -> Result<Vec<u8>>
             I32,
             "0",
         );
+        llmod.add_internal_global(
+            &crate::typed_shape::guard_shape_global_name_from_keys_global(&global_name),
+            I32,
+            "0",
+        );
         // #8122: the inline-`new` header image, composed at module init
         // (`string_pool.rs`) for the classes `class_header_images` admits.
         llmod.add_internal_global(

crates/perry-codegen/src/codegen/string_pool.rs

@@ -647,6 +647,18 @@ pub(super) fn emit_string_pool(
         );
         blk.store(I32, &shape_id, &shape_global);
 
+        // Seed the guard expectation with the same ShapeId and hand the
+        // runtime its address, so `disable_class_field_inline_guard` can poison
+        // it. Registration happens AFTER the seed, and the runtime poisons on
+        // the spot if the latch already flipped — so a module initialised late
+        // cannot reopen a fast path the process has closed.
+        let guard_global = format!(
+            "@{}",
+            crate::typed_shape::guard_shape_global_name_from_keys_global(global_name)
+        );
+        blk.store(I32, &shape_id, &guard_global);
+        blk.call_void("js_register_class_guard_shape", &[(PTR, &guard_global)]);
+
         // #8122: compose the class's inline-`new` header image —
         // `[packed GcHeader word | class_id | ShapeId << 32]` — beside the
         // ShapeId it consumes, ONCE. Every inline allocation of this class
@@ -703,6 +715,7 @@ pub(super) fn emit_string_pool(
                     (PTR, &global_ref),
                     (PTR, &shape_global),
                     (PTR, &image_ref),
+                    (PTR, &guard_global),
                 ],
             );
         }

crates/perry-codegen/src/expr/class_field_inline_guard.rs

@@ -155,7 +155,7 @@ pub(crate) fn class_field_subclass_arms(
         seen_ids.push(sub_id);
         arms.push(ClassFieldSubclassArm {
             class_id: sub_id,
-            shape_id_global: crate::typed_shape::shape_id_global_name_from_keys_global(
+            shape_id_global: crate::typed_shape::guard_shape_global_name_from_keys_global(
                 &keys_global,
             ),
         });
@@ -441,12 +441,14 @@ pub(crate) fn emit_class_field_inline_precheck(
     obj_bits: &str,
     obj_handle: &str,
     expected_class_id: &str,
-    expected_shape_id: &str,
     require_raw_f64: bool,
     set_value_bits: Option<&str>,
     fast_label: &str,
     subclass_arms: &[ClassFieldSubclassArm],
+    keys_global_name: &str,
 ) -> String {
+    let guard_shape_global =
+        crate::typed_shape::guard_shape_global_name_from_keys_global(keys_global_name);
     let deref_idx = ctx.new_block("class_field_inline.deref");
     let guardcall_idx = ctx.new_block("class_field_inline.guardcall");
     let deref_label = ctx.block_label(deref_idx);
@@ -467,14 +469,11 @@ pub(crate) fn emit_class_field_inline_precheck(
     // relaxed-atomic read the guard itself performs.
     {
         let blk = ctx.block();
-        let flag = blk.load_volatile(I8, "@PERRY_CLASS_FIELD_INLINE_GUARD_DISABLED");
-        let flag_ok = blk.icmp_eq(I8, &flag, "0");
         let tag = blk.lshr(I64, obj_bits, "48");
         let is_ptr = blk.icmp_eq(I64, &tag, POINTER_TAG_HI16);
         let above_band = blk.icmp_ugt(I64, obj_handle, HANDLE_BAND_TOP);
         let ptr_safe = blk.and(I1, &is_ptr, &above_band);
-        let can_inline = blk.and(I1, &ptr_safe, &flag_ok);
-        blk.cond_br(&can_inline, &deref_label, &guardcall_label);
+        blk.cond_br(&ptr_safe, &deref_label, &guardcall_label);
     }
 
     ctx.current_block = deref_idx;
@@ -522,13 +521,20 @@ pub(crate) fn emit_class_field_inline_precheck(
         // ObjectHeader word 0 is class_id @0 and the authoritative ShapeId @4
         // (#8113): one 64-bit compare against `(shape << 32) | class_id`.
         let identity = blk.load(I64, &obj_ptr);
-        let declared = expected_class_identity(blk, expected_class_id, expected_shape_id);
+        // The displaced latch's authority lives here now: this expectation is
+        // what `disable_class_field_inline_guard` poisons, so the compare the
+        // guard already had to make now also answers "is the inline path still
+        // open?". VOLATILE for exactly the reason the latch load was — the
+        // runtime flips it mid-execution and a cached expectation would take a
+        // fast path the process has closed.
+        let live_shape = blk.load_volatile(I32, &format!("@{guard_shape_global}"));
+        let declared = expected_class_identity(blk, expected_class_id, &live_shape);
         let mut shape_ok = blk.icmp_eq(I64, &identity, &declared);
         // The declared class's own (class id, ShapeId) pair, OR any subclass
         // arm's. Each arm is a full pair — matching a class id without its
         // canonical descriptor would accept a diverged layout.
         for arm in subclass_arms {
-            let arm_shape = blk.load(I32, &format!("@{}", arm.shape_id_global));
+            let arm_shape = blk.load_volatile(I32, &format!("@{}", arm.shape_id_global));
             let arm_expected = expected_class_identity(blk, &arm.class_id.to_string(), &arm_shape);
             let arm_ok = blk.icmp_eq(I64, &identity, &arm_expected);
             shape_ok = blk.or(I1, &shape_ok, &arm_ok);

crates/perry-codegen/src/expr/hit_path_access_tests.rs

@@ -272,6 +272,16 @@ fn point_class() -> Class {
 /// `probe(p: Point) { return p.x }` — the inline class-field guard tests the
 /// GcHeader with one masked 32-bit compare and the class/shape identity with
 /// one 64-bit compare, instead of five separate header loads.
+///
+/// Three loads now, not two: the third is the poisonable
+/// `@perry_class_guard_shape_*` expectation, which carries the authority the
+/// `@PERRY_CLASS_FIELD_INLINE_GUARD_DISABLED` latch used to. That is a
+/// REDUCTION, not an addition — the latch it displaced was an `external
+/// global`, so reading it cost a GOT load plus a dependent `ldrb` through it
+/// plus a compare, in the gate block, on every access. Net per access: one
+/// fewer machine instruction pair and one fewer dependent load. The assertions
+/// below pin both halves, because "three loads" alone would also be satisfied
+/// by a lowering that kept the latch and added the expectation.
 #[test]
 fn class_field_inline_guard_uses_two_fused_loads() {
     let mut m = module(
@@ -290,8 +300,20 @@ fn class_field_inline_guard_uses_two_fused_loads() {
     let loads: Vec<&str> = deref.lines().filter(|l| l.contains(" = load ")).collect();
     assert_eq!(
         loads.len(),
-        2,
-        "the guard must load the header word and the identity word only:\n{deref}"
+        3,
+        "the guard must load the header word, the identity word and the live \
+         expectation, and nothing else:\n{deref}"
+    );
+    assert!(
+        !ir.contains("@PERRY_CLASS_FIELD_INLINE_GUARD_DISABLED"),
+        "the per-access latch must be GONE from the class-field guard — its \
+         authority moved onto the expectation this guard already loads:\n{ir}"
+    );
+    assert!(
+        deref.contains("load volatile i32, ptr @perry_class_guard_shape_"),
+        "the expectation must be read VOLATILE per access: the runtime poisons \
+         it mid-execution and a cached copy would reopen a closed fast \
+         path:\n{deref}"
     );
     assert!(
         loads.iter().any(|l| l.contains("load i32"))

crates/perry-codegen/src/expr/property_get.rs

@@ -1712,11 +1712,11 @@ pub(crate) fn lower(ctx: &mut FnCtx<'_>, expr: &Expr) -> Result<String> {
                                 &obj_bits,
                                 &obj_handle,
                                 &expected_class_id_str,
-                                &expected_shape_id,
                                 requires_raw_f64,
                                 None,
                                 &fast_label,
                                 &subclass_arms,
+                                &keys_global_name,
                             );
                         // ONE EXIT. Everything the pre-check could not prove —
                         // the guard call, the guard-PASS slot load, the nullish
@@ -1757,14 +1757,27 @@ pub(crate) fn lower(ctx: &mut FnCtx<'_>, expr: &Expr) -> Result<String> {
                             let key_bits = blk.bitcast_double_to_i64(&key_box);
                             blk.and(I64, &key_bits, POINTER_MASK_I64)
                         };
+                        // Loaded HERE, not through the function-entry cache
+                        // `load_class_shape_id` keeps: since the inline
+                        // precheck moved to the poisonable
+                        // `@perry_class_guard_shape_*` expectation, the
+                        // truthful ShapeId is a cold-arm-only operand, and an
+                        // entry-block load of it is two instructions the fast
+                        // path pays and never reads.
+                        let ic_shape_id = {
+                            let global = crate::typed_shape::shape_id_global_name_from_keys_global(
+                                &keys_global_name,
+                            );
+                            ctx.block().load(I32, &format!("@{global}"))
+                        };
                         let val_ic = ctx.block().call(
                             DOUBLE,
                             "js_class_field_get_ic",
                             &[
                                 (I64, &site_id),
                                 (DOUBLE, &recv_box),
                                 (I32, &expected_class_id_str),
-                                (I32, &expected_shape_id),
+                                (I32, &ic_shape_id),
                                 (I64, &key_raw),
                                 (I32, &field_idx_str),
                                 (I32, requires_raw_f64_str),

crates/perry-codegen/src/expr/property_get/helpers.rs

@@ -765,11 +765,11 @@ pub(crate) fn lower_raw_f64_class_field_get_for_number_context(
         &obj_bits,
         &obj_handle,
         &expected_class_id_str,
-        &expected_shape_id,
         true,
         None,
         &fast_label,
         &subclass_arms,
+        &keys_global_name,
     );
     let guard_ok = ctx.block().call(
         I32,

crates/perry-codegen/src/expr/property_set.rs

@@ -1261,11 +1261,11 @@ pub(crate) fn lower(ctx: &mut FnCtx<'_>, expr: &Expr, assignment_strict: bool) -
                                 &obj_bits,
                                 &obj_handle,
                                 &expected_class_id_str,
-                                &expected_shape_id,
                                 requires_raw_f64,
                                 Some(&val_bits),
                                 &fast_label,
                                 &subclass_arms,
+                                &keys_global_name,
                             );
                                 let guard_ok = ctx.block().call(
                                     I32,

crates/perry-codegen/src/expr/property_set/sloppy_class_field.rs

@@ -158,8 +158,6 @@ pub(crate) fn try_lower_sloppy_class_field_store(
         let key_handle_global = format!("@{}", ctx.strings.entry(key_idx).handle_global);
         let field_idx_str = field_index.to_string();
         let expected_class_id_str = expected_class_id.to_string();
-        let expected_shape_id =
-            crate::typed_shape::load_class_shape_id(ctx, &class_name, &keys_global_name);
 
         let (obj_bits, obj_handle, key_box, val_bits) = {
             let blk = ctx.block();
@@ -189,11 +187,11 @@ pub(crate) fn try_lower_sloppy_class_field_store(
             &obj_bits,
             &obj_handle,
             &expected_class_id_str,
-            &expected_shape_id,
             true,
             Some(&val_bits),
             &fast_label,
             &subclass_arms,
+            &keys_global_name,
         );
 
         // Miss: the strict-aware runtime with `strict = 0`, so a rejected write
@@ -291,8 +289,6 @@ fn try_lower_sloppy_class_field_boxed_store(
         let key_handle_global = format!("@{}", ctx.strings.entry(key_idx).handle_global);
         let field_idx_str = field_index.to_string();
         let expected_class_id_str = expected_class_id.to_string();
-        let expected_shape_id =
-            crate::typed_shape::load_class_shape_id(ctx, class_name, keys_global_name);
 
         let (obj_bits, obj_handle, key_box, val_bits) = {
             let blk = ctx.block();
@@ -322,11 +318,11 @@ fn try_lower_sloppy_class_field_boxed_store(
             &obj_bits,
             &obj_handle,
             &expected_class_id_str,
-            &expected_shape_id,
             false,
             Some(&val_bits),
             &fast_label,
             &subclass_arms,
+            &keys_global_name,
         );
 
         {

crates/perry-codegen/src/lower_call/method_override.rs

@@ -1203,11 +1203,11 @@ pub(super) fn emit_guarded_direct_method_call(
                         &obj_bits,
                         &obj_handle,
                         &expected_class_id_str,
-                        &expected_shape_id,
                         true,
                         None,
                         &proven_label,
                         &[],
+                        &keys_global_name,
                     );
                 // Created after the precheck's own blocks so the merge (and the
                 // typed/generic branch it feeds) follows the per-field guard