diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 58596a561..27ad61c10 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -338,6 +338,7 @@ jobs: wasm_gc_games_differential_mir \ wasm_gc_handler_carrier \ wasm_gc_map_versions_spec \ + wasm_gc_vector_versions_spec \ wasm_gc_optimize_trunc_sat \ wasm_gc_packed_sequence \ wasm_gc_perslot_int_unboxing_differential \ diff --git a/CHANGELOG.md b/CHANGELOG.md index 9f529eee7..0bd7e4870 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -64,6 +64,7 @@ The generated loop is now written from the program's source alone, and the manif - **Generated Rust moves the rest of a nested record into its update.** `Setting.update(setting, window = Window.update(setting.window, created = Map.set(setting.window.created, k, v)), height = setting.height + 1)` used to clone `setting.window` and `setting.window.created`, so every `Map.set` copied the Map. When nothing reads that part of `setting` again, the Map now moves into `Map.set` and the other fields of `setting.window` move into the new `Window`. - **`check` no longer asks a verify block of a function taking a `Tcp.Socket` or a `Wait.Item`.** Every constructor of either carries a capability resource, so no verify case can write one, but the exemption only looked into the types of the function's own module. It now looks into a capability's own records and sums by the same rule. - **Generated Rust builds when a record is updated after one of its fields was read.** `progress = flight.progress` followed by `Flight.update(flight, progress = f(progress))` inside a pair of functions that tail-call each other generated Rust that moved the field out and then moved the whole record, which rustc rejects (E0382). A loop that read a field in a `let` and later returned the whole record failed the same way. Such a field read now moves only when nothing reads that part of the record again and is copied otherwise, and the field read into `f` moves in both spellings (`f(progress)` and `f(flight.progress)`), so a Map `f` updates is not copied. +- **wasm-gc: `Vector.set` no longer copies the vector.** `set` copied the whole array unless the compiler could prove nothing else held the vector, which it cannot for a vector held in a record field. A record holding a 100,000-cell Vector took 0.21 s for 2000 sets and 1.8 s for 20,000 under `aver run --wasm-gc`. A Vector is now a version over an array, like a Map: `set` writes the cell in place and returns a new version, and the vector it was given stays valid because the old cell is kept with it. The same runs now take 0.04 s. Reading an older version again costs one step for each set made since. `bench/scenarios/vector_ops` runs in 0.37 ms instead of 18 ms on wasmtime and 0.29 ms instead of 13 ms on V8. `==` on two Vectors now compiles on wasm-gc; before, the module failed validation. A program with no Vector value compiles to the same bytes as before. A function that touches a Vector is no longer offered for certification, because the certificate wall models a Vector as a plain array; `tools/certkit/fixtures/cell_at.av` loses its certificate. - **wasm-gc: `Map.set` no longer copies the map, and `Map.remove` no longer changes the map it was given.** `set` copied every bucket unless the compiler could prove the map had no other holder, which it cannot for a map held in a record field such as an answer module's state. A 100 000-entry state map served 2000 `Map.set` requests in 11 s under `aver run --wasm-gc` and 1.9 s on Node 26. `remove` wrote into the map it was given, so a caller that still held that map saw the key gone. Both now write into the map's arrays in place and return a new version. The version they were given stays valid, because a record of what the write replaced is kept with it. The same run now takes 0.4 s under `aver run --wasm-gc` and 0.3 s on Node. Reading an older version again costs one step for each write made since. - **A program with the generated loop can key its own waits by a type of its own.** The loop keys its wait by `Int`, and a hand-written `Wait.poll` keyed by a sum beside it used to be refused by the Rust door ("this program keys one wait set by 'Int' and another by 'Watch'") and to fail wasm-gc validation. In a program that answers a capability of its own, each such wait, in the entry or in a dependency, now goes through helpers generated for its key type that carry it through an `Int`-keyed wait. It answers the same keys in the same order. Its recording holds the `Int`-keyed wait. The wasm-gc wait ABI is unchanged, and `--target wasip2` runs such waits too. A program that answers no capability still keys all its waits one way. - **`check` no longer asks a verify block of a function no verify case can call.** A parameter of a capability resource type (`Tcp.Connection`, `Work.Job`, a job kind's handle), or of a tuple, record or sum of the module that always carries one, has no value a case can write, so such a pure branching helper failed `error[missing-verify]` with no way to satisfy it. It is now exempt, the way effectful functions are. A parameter with an empty value (`List`, `Option`, a sum with a resource-free variant) still needs its verify block. diff --git a/docs/certification.md b/docs/certification.md index 9e1e3aa27..1b2da35ac 100644 --- a/docs/certification.md +++ b/docs/certification.md @@ -105,7 +105,7 @@ L3 is derived by the wall (`GrammarTotal.checkTermGroup`), never read from a man Every certified export reports one class, `source-plan-v1`, with facets the wall derives from the plan: `recursive`, `mutual`, `calls`, `records`, `variants`, `strings`, `floats`. -The plan grammar is the admitted subset of optimized MIR. It covers Int, Bool, Float and String literals; locals and named `let`; calls to other planned functions, including self and mutual recursion and tail calls; Int `+ - *` and the six comparisons; Bool `and`, `or`, `not`, `==` and `!=`; Float comparisons other than `!=`; String `+`, `==`, `!=` and interpolation of String parts; `if`; records with two or more fields (create in declared order, and project); user variants, `Option` and `Result` (construct and match); matches on Int, Bool and String literals and flat tuple destructuring; `Option.withDefault` and `Result.withDefault`; `Option.withDefault(Vector.get(v, i), )`; `Int.div` and `Int.mod` by a nonzero literal, or fused under `Result.withDefault` with an Int default; the empty list and `List.prepend`. +The plan grammar is the admitted subset of optimized MIR. It covers Int, Bool, Float and String literals; locals and named `let`; calls to other planned functions, including self and mutual recursion and tail calls; Int `+ - *` and the six comparisons; Bool `and`, `or`, `not`, `==` and `!=`; Float comparisons other than `!=`; String `+`, `==`, `!=` and interpolation of String parts; `if`; records with two or more fields (create in declared order, and project); user variants, `Option` and `Result` (construct and match); matches on Int, Bool and String literals and flat tuple destructuring; `Option.withDefault` and `Result.withDefault`; `Int.div` and `Int.mod` by a nonzero literal, or fused under `Result.withDefault` with an Int default; the empty list and `List.prepend`. A function that touches a `Vector` is declined: on wasm-gc a `Vector` value is a version struct over a shared array, and the wall models it as the plain array of its elements. A function is declined, with the MIR node or type named in the reason, when it has effects, uses raw i64 slots, negates an Int (the negation helper has no wall template yet), uses an Int literal outside the i64 range, does Float arithmetic, calls through a function value, matches on a list, or uses any other node outside the subset. A function is also declined when the producer's check finds that its plan does not lower to exactly its code entry. diff --git a/docs/wasm-gc-custom-capabilities.md b/docs/wasm-gc-custom-capabilities.md index b2ac327f4..521eda646 100644 --- a/docs/wasm-gc-custom-capabilities.md +++ b/docs/wasm-gc-custom-capabilities.md @@ -66,7 +66,7 @@ embedded runner lifts these values directly into transport-neutral | `Result`, `Option`, tuple, record | typed GC struct reference | | sum type | nominal root reference carrying a typed variant struct | | `List` | nullable typed cons reference | -| `Vector` | typed mutable GC array reference | +| `Vector` | typed GC struct reference to one version over a mutable array; read and fill it through the `Vector` helpers below | | `Map` | Aver's typed deterministic map reference | | proof-packed `List` refinement | typed GC array reference; record factories/projectors bridge its declared carrier | diff --git a/src/codegen/cert/plan_from_mir.rs b/src/codegen/cert/plan_from_mir.rs index 3bc31c4a0..890e16a76 100644 --- a/src/codegen/cert/plan_from_mir.rs +++ b/src/codegen/cert/plan_from_mir.rs @@ -278,16 +278,16 @@ impl TypeTableBuilder { } Ok(PlanTy::Result(Box::new(t), Box::new(e))) } - ("Vector", 1) => { - let t = arg(self, 0)?; - let idx = layout - .vector(&canon) - .ok_or_else(|| format!("type `{canon}` is not registered"))?; - if !self.table.vecs.iter().any(|v| v.0 == t) { - self.table.vecs.push((t.clone(), idx)); - } - Ok(PlanTy::Vec(Box::new(t))) - } + // A `Vector` value is a version struct over its array on + // wasm-gc (`codegen::wasm_gc::vectors`), and reading it may + // reroot the versions sharing that array. The wall models a + // Vector as the plain array of its elements, so a function that + // touches one is declined rather than certified against the + // wrong representation. + ("Vector", 1) => Err(format!( + "type `{canon}`: a Vector is a versioned struct on wasm-gc, and the wall \ + models it as a plain array" + )), ("List", 1) => { let t = arg(self, 0)?; let idx = layout diff --git a/src/codegen/wasm_gc/README.md b/src/codegen/wasm_gc/README.md index 2ff869c0c..2426fd606 100644 --- a/src/codegen/wasm_gc/README.md +++ b/src/codegen/wasm_gc/README.md @@ -81,7 +81,7 @@ Aver is statically typed; the type checker has already proven what each value is | `List` | `(ref null $List_T)` where `$List_T = (struct (field T) (field (ref null $List_T)))` | | `Tuple` | `(ref null $Tuple_T1_T2_…)` | | `Map` | `(ref null $Map_K_V)` — flat hashtable struct | -| `Vector` | `(ref null (array T))` (or `$Vector_T` wrapping array + len) | +| `Vector` | `(ref null $Vector_T)`: a version over `(array (mut T))` (see `vectors.rs`) | | `Record name` | `(ref null $Record_name)` — named struct | | `Constructor name` | `(ref null $Constr_name)` — named struct subtype of the variant root | diff --git a/src/codegen/wasm_gc/body/emit.rs b/src/codegen/wasm_gc/body/emit.rs index db5c23259..c0a0a143c 100644 --- a/src/codegen/wasm_gc/body/emit.rs +++ b/src/codegen/wasm_gc/body/emit.rs @@ -168,13 +168,13 @@ pub(super) fn sum_or_record_eq_fn(ty: &crate::types::Type, ctx: &EmitCtx<'_>) -> .collect(); ctx.fn_map.list_ops.get(&canonical).and_then(|ops| ops.eq) } - crate::types::Type::Vector(_) => { - let canonical: String = ty - .display() + // The vector helpers are registered per `List` pair. + crate::types::Type::Vector(inner) => { + let canonical: String = format!("List<{}>", inner.display()) .chars() .filter(|c| !c.is_whitespace()) .collect(); - ctx.fn_map.vfl_ops.get(&canonical).and_then(|ops| ops.eq) + ctx.fn_map.vfl_ops_lookup(&canonical).and_then(|ops| ops.eq) } // Map structural eq — `__eq_Map` slot lives in // MapHelperRegistry but we mirror the fn idx into diff --git a/src/codegen/wasm_gc/body/from_mir/builtins.rs b/src/codegen/wasm_gc/body/from_mir/builtins.rs index 98bc9402c..ea10fa382 100644 --- a/src/codegen/wasm_gc/body/from_mir/builtins.rs +++ b/src/codegen/wasm_gc/body/from_mir/builtins.rs @@ -575,12 +575,8 @@ pub(crate) fn emit_mir_option_with_default( let index = &inner.args[1]; let vec_aver = aver_type_str_of(vector); let canonical: String = vec_aver.chars().filter(|c| !c.is_whitespace()).collect(); - let vec_idx = - ctx.registry - .vector_type_idx(&canonical) - .ok_or(WasmGcError::Validation(format!( - "Vector.get: vector arg of type `{vec_aver}` is not a registered Vector" - )))?; + let (vslots, vops) = vector_helpers(&canonical, ctx)?; + let vec_idx = vslots.array; let element = TypeRegistry::vector_element_type(&canonical).ok_or(WasmGcError::Validation( format!("Vector.get: cannot parse element type from `{canonical}`"), @@ -609,11 +605,12 @@ pub(crate) fn emit_mir_option_with_default( idx_nonneg!(); idx_i32!(); e!(vector); - func.instruction(&Instruction::ArrayLen); + emit_vector_len(func, vslots); func.instruction(&Instruction::I32LtU); func.instruction(&Instruction::I32And); func.instruction(&Instruction::If(block_ty)); e!(vector); + func.instruction(&Instruction::Call(vops.current)); idx_i32!(); func.instruction(&Instruction::ArrayGet(vec_idx)); func.instruction(&Instruction::Else); @@ -697,16 +694,52 @@ pub(crate) fn emit_mir_option_with_default( Ok(MirBuiltinEmit::Produced(true)) } -/// Mirror of `emit_vector_set_or_default`: the fused -/// `Option.withDefault(Vector.set(v, i, x), v)`. When -/// `mir_arg_uniquely_owned` says `v` is a dead, non-aliased binding the -/// engine array is mutated in place (`array.set` on the original handle, -/// no allocation); otherwise the array is cloned (`array.new_default` + -/// `array.copy`) and the copy mutated. The ownership verdict and the -/// instruction sequence match the HIR oracle byte-for-byte — -/// `mir_arg_uniquely_owned` reads the same `last_use` / `aliased_slots` -/// the oracle's `arg_uniquely_owned` does, on the same `v` occurrence -/// (`Vector.set`'s first arg). +/// The versioned helpers of one `Vector` (`vectors.rs`). +#[derive(Clone, Copy)] +struct VectorHelpers { + current: u32, + set: u32, +} + +/// The version slots and the helpers of `canonical` (`Vector`); see +/// `vectors.rs`. +fn vector_helpers( + canonical: &str, + ctx: &EmitCtx<'_>, +) -> Result<(crate::codegen::wasm_gc::types::VectorSlots, VectorHelpers), WasmGcError> { + let slots = ctx + .registry + .vector_slots(canonical) + .ok_or(WasmGcError::Validation(format!( + "`{canonical}` is not a registered Vector" + )))?; + let element = TypeRegistry::vector_element_type(canonical).ok_or(WasmGcError::Validation( + format!("cannot parse element type from `{canonical}`"), + ))?; + let ops = ctx + .fn_map + .vfl_ops_lookup(&format!("List<{}>", element.trim())) + .copied() + .ok_or(WasmGcError::Validation(format!( + "the helpers of `{canonical}` were not registered" + )))?; + match (ops.current, ops.set) { + (Some(current), Some(set)) => Ok((slots, VectorHelpers { current, set })), + _ => Err(WasmGcError::Validation(format!( + "`{canonical}` has no version helpers" + ))), + } +} + +/// With a vector on the stack, leave its array's length. +fn emit_vector_len(func: &mut Function, slots: crate::codegen::wasm_gc::types::VectorSlots) { + crate::codegen::wasm_gc::vectors::emit_version_len(func, slots); +} + +/// The fused `Option.withDefault(Vector.set(v, i, x), v)`: a new version +/// with the cell written when `i` is in range, `v` itself otherwise. +/// `mir_arg_uniquely_owned` (a dead, non-aliased binding) lets the `set` +/// helper skip recording the old cell when no other version needs it. fn emit_mir_vector_set_or_default( func: &mut Function, vector: &Spanned, @@ -717,12 +750,11 @@ fn emit_mir_vector_set_or_default( ) -> Result { let vec_aver = aver_type_str_of(vector); let canonical: String = vec_aver.chars().filter(|c| !c.is_whitespace()).collect(); - let vec_idx = ctx - .registry - .vector_type_idx(&canonical) - .ok_or(WasmGcError::Validation(format!( - "Vector.set: vector arg of type `{vec_aver}` is not a registered Vector" - )))?; + let (vslots, vops) = vector_helpers(&canonical, ctx)?; + let version_ref = wasm_encoder::ValType::Ref(wasm_encoder::RefType { + nullable: true, + heap_type: wasm_encoder::HeapType::Concrete(vslots.version), + }); macro_rules! e { ($x:expr) => { @@ -747,93 +779,28 @@ fn emit_mir_vector_set_or_default( }; } - // Fast path: dead, non-aliased binding → mutate the engine array in - // place and return the same handle. No scratch, no allocation. - // - // Re-emitting `vector` three times is free HERE and nowhere else: the - // call-site guard above only reaches this emitter when the receiver and + // The call-site guard only reaches this emitter when the receiver and // the `withDefault` default are the same `MirExpr::Local`, so every - // emission is one `local.get` of one cell. (The boxed spelling, - // `emit_mir_vector_set_boxed`, has no such guard — its receiver can be a - // provably-fresh non-local, and re-emitting THAT builds another array.) - if mir_arg_uniquely_owned(vector, ctx) { - idx_nonneg!(); - idx_i32!(); - e!(vector); - func.instruction(&Instruction::ArrayLen); - func.instruction(&Instruction::I32LtU); - func.instruction(&Instruction::I32And); - func.instruction(&Instruction::If(wasm_encoder::BlockType::Empty)); - e!(vector); - idx_i32!(); - e!(value); - func.instruction(&Instruction::ArraySet(vec_idx)); - func.instruction(&Instruction::End); - e!(vector); - return Ok(MirBuiltinEmit::Produced(true)); - } - - // Slow path (clone-on-write): the slot may share its engine array - // with another live binding. Allocate a fresh array, copy every - // cell, mutate the copy. - let scratch = slots - .vector_set_scratch - .get(&canonical) - .copied() - .ok_or_else(|| { - WasmGcError::Validation(format!( - "Vector.set: scratch local for `{canonical}` not reserved \ - (slot-pre-pass missed this site)" - )) - })?; - let vec_ref = wasm_encoder::ValType::Ref(wasm_encoder::RefType { - nullable: true, - heap_type: wasm_encoder::HeapType::Concrete(vec_idx), - }); - - // The bounds test runs BEFORE the copy, and the whole clone lives inside - // the taken branch. Out of bounds this fusion's answer is the default, - // which the call-site guard proved is the receiver itself, so the else - // branch hands back the original instead of an identical copy of it — - // the same value, and no allocation at all on the miss. - // - // Scratch discipline, the same one `emit_mir_vector_set_boxed` states in - // full: there is ONE scratch local per `Vector` per fn and this - // emitter is re-entrant, so every read of `scratch` happens before any - // operand that could contain another `Vector.set` of the same type is - // re-emitted. Both reads — the result and the `array.set` target — are - // therefore pushed before the index is re-emitted and before `value` is. - // The index used to be emitted between the store and the reads, so an - // index like `Vector.len(Option.withDefault(Vector.set(…), …))` handed - // back the array the NESTED set had left in the local. + // emission of `vector` is one `local.get` of one cell. In range, the + // versioned `set` (`vectors.rs`) writes the cell and returns the new + // version; a dead, non-aliased receiver that no other version is + // described against is written with nothing recorded. Out of range the + // answer is the receiver itself. + let owned = mir_arg_uniquely_owned(vector, ctx); idx_nonneg!(); idx_i32!(); e!(vector); - func.instruction(&Instruction::ArrayLen); + emit_vector_len(func, vslots); func.instruction(&Instruction::I32LtU); func.instruction(&Instruction::I32And); - func.instruction(&Instruction::If(wasm_encoder::BlockType::Result(vec_ref))); - - e!(vector); - func.instruction(&Instruction::ArrayLen); - func.instruction(&Instruction::ArrayNewDefault(vec_idx)); - func.instruction(&Instruction::LocalSet(scratch)); - func.instruction(&Instruction::LocalGet(scratch)); - func.instruction(&Instruction::I32Const(0)); - e!(vector); - func.instruction(&Instruction::I32Const(0)); + func.instruction(&Instruction::If(wasm_encoder::BlockType::Result( + version_ref, + ))); e!(vector); - func.instruction(&Instruction::ArrayLen); - func.instruction(&Instruction::ArrayCopy { - array_type_index_dst: vec_idx, - array_type_index_src: vec_idx, - }); - func.instruction(&Instruction::LocalGet(scratch)); - func.instruction(&Instruction::LocalGet(scratch)); idx_i32!(); e!(value); - func.instruction(&Instruction::ArraySet(vec_idx)); - + func.instruction(&Instruction::I32Const(i32::from(owned))); + func.instruction(&Instruction::Call(vops.set)); func.instruction(&Instruction::Else); e!(vector); func.instruction(&Instruction::End); @@ -2392,10 +2359,15 @@ pub(crate) fn emit_mir_vector_builtin( ) -> Result { match dotted { "Vector.len" if args.len() == 1 => { + let canonical: String = aver_type_str_of(&args[0]) + .chars() + .filter(|c| !c.is_whitespace()) + .collect(); + let (vslots, _) = vector_helpers(&canonical, ctx)?; if emit_mir_expr(func, &args[0], slots, ctx)?.is_none() { return Ok(MirBuiltinEmit::Fallback); } - func.instruction(&Instruction::ArrayLen); + emit_vector_len(func, vslots); func.instruction(&Instruction::I64ExtendI32U); // `Vector.len` returns `Int` — lift the i64 length into the // `$AverInt` carrier. @@ -2427,6 +2399,8 @@ pub(crate) fn emit_mir_vector_builtin( return Ok(MirBuiltinEmit::Fallback); } func.instruction(&Instruction::ArrayNew(vec_idx)); + let (vslots, _) = vector_helpers(&canonical, ctx)?; + crate::codegen::wasm_gc::vectors::emit_wrap_array(func, vslots); } "Vector.new" if args.len() == 2 => { let elem_aver = aver_type_str_of(&args[1]); @@ -2490,6 +2464,8 @@ pub(crate) fn emit_mir_vector_builtin( emit_size_i64(func)?; func.instruction(&Instruction::I32WrapI64); func.instruction(&Instruction::ArrayNew(vec_idx)); + let (vslots, _) = vector_helpers(&canonical, ctx)?; + crate::codegen::wasm_gc::vectors::emit_wrap_array(func, vslots); emit_default_value(func, "String", ctx.registry)?; func.instruction(&Instruction::StructNew(res_idx)); func.instruction(&Instruction::Else); @@ -2536,8 +2512,9 @@ pub(crate) fn emit_mir_vector_new_literal( emit_mir_vector_builtin(func, "__vector_new", args, slots, ctx) } -/// Mirror of `emit_vector_get_boxed` (builtins.rs): bounds-checked -/// `Option` — `Some(arr[i])` in range, `None` otherwise. +/// Bounds-checked `Vector.get` as an `Option`: `Some(arr[i])` in range, +/// `None` otherwise. The elements are read from the vector's current array +/// (`vectors.rs`). pub(crate) fn emit_mir_vector_get_boxed( func: &mut Function, vector: &Spanned, @@ -2547,12 +2524,7 @@ pub(crate) fn emit_mir_vector_get_boxed( ) -> Result { let vec_aver = aver_type_str_of(vector); let canonical: String = vec_aver.chars().filter(|c| !c.is_whitespace()).collect(); - let vec_idx = ctx - .registry - .vector_type_idx(&canonical) - .ok_or(WasmGcError::Validation(format!( - "Vector.get: vector arg of type `{vec_aver}` is not a registered Vector" - )))?; + let (vslots, vops) = vector_helpers(&canonical, ctx)?; let element = TypeRegistry::vector_element_type(&canonical).ok_or(WasmGcError::Validation( format!("Vector.get: cannot parse element type from `{canonical}`"), ))?; @@ -2593,14 +2565,15 @@ pub(crate) fn emit_mir_vector_get_boxed( idx_nonneg!(); idx_i32!(); e!(vector); - func.instruction(&Instruction::ArrayLen); + emit_vector_len(func, vslots); func.instruction(&Instruction::I32LtU); func.instruction(&Instruction::I32And); func.instruction(&Instruction::If(block_ty)); func.instruction(&Instruction::I32Const(1)); e!(vector); + func.instruction(&Instruction::Call(vops.current)); idx_i32!(); - func.instruction(&Instruction::ArrayGet(vec_idx)); + func.instruction(&Instruction::ArrayGet(vslots.array)); func.instruction(&Instruction::StructNew(opt_idx)); func.instruction(&Instruction::Else); func.instruction(&Instruction::I32Const(0)); @@ -2610,9 +2583,9 @@ pub(crate) fn emit_mir_vector_get_boxed( Ok(MirBuiltinEmit::Produced(true)) } -/// Mirror of `emit_vector_set_boxed` (builtins.rs): `Option>`. -/// Fast path (uniquely-owned arg) mutates the engine array in place; -/// slow path clone-on-writes through the per-`Vector` scratch local. +/// `Vector.set` as an `Option>`: in range, `Some` of the new +/// version the versioned `set` helper returns (`vectors.rs`); otherwise +/// `None`. pub(crate) fn emit_mir_vector_set_boxed( func: &mut Function, vector: &Spanned, @@ -2623,12 +2596,7 @@ pub(crate) fn emit_mir_vector_set_boxed( ) -> Result { let vec_aver = aver_type_str_of(vector); let canonical: String = vec_aver.chars().filter(|c| !c.is_whitespace()).collect(); - let vec_idx = ctx - .registry - .vector_type_idx(&canonical) - .ok_or(WasmGcError::Validation(format!( - "Vector.set: vector arg of type `{vec_aver}` is not a registered Vector" - )))?; + let (vslots, vops) = vector_helpers(&canonical, ctx)?; let opt_canonical = format!("Option<{canonical}>"); let opt_idx = ctx .registry @@ -2673,100 +2641,39 @@ pub(crate) fn emit_mir_vector_set_boxed( (slot-pre-pass missed this site)" )) })?; - if mir_arg_uniquely_owned(vector, ctx) { - // The receiver is emitted ONCE, into the scratch local, and read back - // from there at all three sites. Re-emitting it is only free for a - // LOCAL; a non-local receiver is owned-eligible when it is a provably - // FRESH collection, and re-emitting a fresh collection BUILDS ANOTHER - // ONE. Three emissions then meant three different arrays: the bounds - // check measured the first, `array.set` wrote the second, and the - // `Some` wrapped the third — so - // `Vector.set(Vector.fromList([x, 9]), 0, x + 5000)` answered `x` - // where every other backend answered `x + 5000` (#953 round 3, probe - // h). Freshness earns the write, it does not make the value free. - // - // ## Scratch discipline: every read of it before any re-emission - // - // There is ONE scratch local per `Vector` per fn, and the emitter - // is re-entrant: an operand re-emitted here can be another - // `Vector.set` of the same `Vector`, which stores into that same - // local. So the whole body obeys one rule — emit everything that can - // re-enter, THEN store, THEN read, and let the tail re-emissions - // clobber a local nothing will read again: - // - // - the index goes first, before the store. It is emitted up to three - // times (non-negative test, bounds test, `array.set` operand), and - // a nested set inside it would otherwise overwrite the receiver - // between the store and the first read; - // - the `Some` is built BEFORE the write, so both remaining reads of - // `scratch` — the payload and the `array.set` target — happen - // before the index is re-emitted and before `value` is. It still - // observes the mutation: the struct holds the array by reference. - idx_nonneg!(); - idx_i32!(); - e!(vector); - func.instruction(&Instruction::LocalSet(scratch)); - func.instruction(&Instruction::LocalGet(scratch)); - func.instruction(&Instruction::ArrayLen); - func.instruction(&Instruction::I32LtU); - func.instruction(&Instruction::I32And); - func.instruction(&Instruction::If(block_ty)); - func.instruction(&Instruction::I32Const(1)); - func.instruction(&Instruction::LocalGet(scratch)); - func.instruction(&Instruction::StructNew(opt_idx)); - func.instruction(&Instruction::LocalGet(scratch)); - idx_i32!(); - e!(value); - func.instruction(&Instruction::ArraySet(vec_idx)); - func.instruction(&Instruction::Else); - func.instruction(&Instruction::I32Const(0)); - func.instruction(&Instruction::RefNull(wasm_encoder::HeapType::Concrete( - vec_idx, - ))); - func.instruction(&Instruction::StructNew(opt_idx)); - func.instruction(&Instruction::End); - return Ok(MirBuiltinEmit::Produced(true)); - } - + let owned = mir_arg_uniquely_owned(vector, ctx); + // The receiver is emitted ONCE, into the scratch local: a non-local + // receiver is owned-eligible when it is a provably FRESH collection, and + // re-emitting a fresh collection builds another one (#953 round 3). + // + // ## Scratch discipline + // + // There is ONE scratch local per `Vector` per fn, and the emitter is + // re-entrant: an operand re-emitted here can be another `Vector.set` of + // the same `Vector`, which stores into that same local. So the index + // goes first, before the store, and the only read of `scratch` after the + // bounds test (the `set` receiver) is pushed before the index is + // re-emitted and before `value` is. idx_nonneg!(); idx_i32!(); e!(vector); - func.instruction(&Instruction::ArrayLen); + func.instruction(&Instruction::LocalSet(scratch)); + func.instruction(&Instruction::LocalGet(scratch)); + emit_vector_len(func, vslots); func.instruction(&Instruction::I32LtU); func.instruction(&Instruction::I32And); func.instruction(&Instruction::If(block_ty)); - e!(vector); - func.instruction(&Instruction::ArrayLen); - func.instruction(&Instruction::ArrayNewDefault(vec_idx)); - func.instruction(&Instruction::LocalSet(scratch)); - func.instruction(&Instruction::LocalGet(scratch)); - func.instruction(&Instruction::I32Const(0)); - e!(vector); - func.instruction(&Instruction::I32Const(0)); - e!(vector); - func.instruction(&Instruction::ArrayLen); - func.instruction(&Instruction::ArrayCopy { - array_type_index_dst: vec_idx, - array_type_index_src: vec_idx, - }); - // Same scratch discipline as the owned path above: the `Some` wraps the - // copy BEFORE the copy is written, so both reads of `scratch` are done - // before the index is re-emitted and before `value` is. Without it a - // nested `Vector.set` of the same `Vector` in either operand — an - // index like `Vector.len(Option.withDefault(Vector.set(…), …))` is enough - // — left the payload pointing at the nested set's array while the write - // landed correctly on this one. func.instruction(&Instruction::I32Const(1)); func.instruction(&Instruction::LocalGet(scratch)); - func.instruction(&Instruction::StructNew(opt_idx)); - func.instruction(&Instruction::LocalGet(scratch)); idx_i32!(); e!(value); - func.instruction(&Instruction::ArraySet(vec_idx)); + func.instruction(&Instruction::I32Const(i32::from(owned))); + func.instruction(&Instruction::Call(vops.set)); + func.instruction(&Instruction::StructNew(opt_idx)); func.instruction(&Instruction::Else); func.instruction(&Instruction::I32Const(0)); func.instruction(&Instruction::RefNull(wasm_encoder::HeapType::Concrete( - vec_idx, + vslots.version, ))); func.instruction(&Instruction::StructNew(opt_idx)); func.instruction(&Instruction::End); diff --git a/src/codegen/wasm_gc/body/slots.rs b/src/codegen/wasm_gc/body/slots.rs index 21eb5c410..fc7b57ccb 100644 --- a/src/codegen/wasm_gc/body/slots.rs +++ b/src/codegen/wasm_gc/body/slots.rs @@ -323,11 +323,9 @@ impl SlotTable { }; // Allocate one scratch local per unique `Vector` instantiation // that appears as the first argument of any `Vector.set` call in - // this fn body. The clone-on-write emit (`emit_vector_set_*`) - // builds the new vector via `array.new_default` + `array.copy` - // and conditionally writes the changed cell on the copy — that - // requires a typed local to hold the copy ref between - // `array.copy` and the subsequent `array.set`. + // this fn body. The boxed `Vector.set` emit evaluates its receiver + // once into it, a version (`vectors.rs`), and reads it for the + // bounds test and the versioned `set`. let mut vector_set_canonicals: HashSet = HashSet::new(); let mut vector_new_canonicals: HashSet = HashSet::new(); collect_vector_scratch_canonicals( @@ -339,10 +337,10 @@ impl SlotTable { let mut sorted: Vec = vector_set_canonicals.into_iter().collect(); sorted.sort(); // deterministic local order for canonical in sorted { - if let Some(vec_idx) = registry.vector_type_idx(&canonical) { + if let Some(vector) = registry.vector_slots(&canonical) { let ty = ValType::Ref(wasm_encoder::RefType { nullable: true, - heap_type: wasm_encoder::HeapType::Concrete(vec_idx), + heap_type: wasm_encoder::HeapType::Concrete(vector.version), }); let local_idx = by_slot.len() as u32; by_slot.push(ty); diff --git a/src/codegen/wasm_gc/capability_abi.rs b/src/codegen/wasm_gc/capability_abi.rs index ee69820c6..729c0aa07 100644 --- a/src/codegen/wasm_gc/capability_abi.rs +++ b/src/codegen/wasm_gc/capability_abi.rs @@ -57,15 +57,19 @@ enum HelperKind { }, ListIsEmpty, VectorNew { - type_idx: u32, + slots: super::types::VectorSlots, + }, + VectorLen { + slots: super::types::VectorSlots, }, - VectorLen, VectorGet { - type_idx: u32, + slots: super::types::VectorSlots, + current: u32, value: Option, }, VectorSet { - type_idx: u32, + slots: super::types::VectorSlots, + current: u32, value: Option, }, SumKind { @@ -105,6 +109,8 @@ pub(super) struct IntAbiHelpers { pub(super) struct CollectionAbiHelpers<'a> { pub(super) maps: &'a dyn Fn(&str) -> Option, + /// The `current` helper of a `Vector` (`vectors.rs`), by canonical. + pub(super) vector_current: &'a dyn Fn(&str) -> Option, pub(super) packed_sequences: &'a dyn Fn(&str) -> Option, } @@ -407,32 +413,36 @@ impl CapabilityAbi { } } Type::Vector(inner) => { - let type_idx = *registry - .vector_types - .get(&canonical.replace(' ', "")) - .ok_or_else(|| { - WasmGcError::Validation(format!("capability ABI lacks `{canonical}` slot")) - })?; + let compact = canonical.replace(' ', ""); + let lacks = + || WasmGcError::Validation(format!("capability ABI lacks `{canonical}` slot")); + let slots = registry + .vector_versions + .get(&compact) + .copied() + .ok_or_else(lacks)?; + let current = (collection_helpers.vector_current)(&compact).ok_or_else(lacks)?; let vector = value(ty)?.expect("Vector has a wasm value"); let inner = value(inner)?; push( format!("{stem}_new"), vec![ValType::I32], vec![vector], - HelperKind::VectorNew { type_idx }, + HelperKind::VectorNew { slots }, ); push( format!("{stem}_len"), vec![vector], vec![ValType::I32], - HelperKind::VectorLen, + HelperKind::VectorLen { slots }, ); push( format!("{stem}_get"), vec![vector, ValType::I32], inner.into_iter().collect(), HelperKind::VectorGet { - type_idx, + slots, + current, value: inner, }, ); @@ -443,7 +453,8 @@ impl CapabilityAbi { set_params, vec![], HelperKind::VectorSet { - type_idx, + slots, + current, value: inner, }, ); @@ -929,31 +940,44 @@ fn emit_helper(function: &mut Function, kind: &HelperKind) { function.instruction(&Instruction::LocalGet(0)); function.instruction(&Instruction::RefIsNull); } - HelperKind::VectorNew { type_idx } => { + HelperKind::VectorNew { slots } => { function.instruction(&Instruction::LocalGet(0)); - function.instruction(&Instruction::ArrayNewDefault(*type_idx)); + function.instruction(&Instruction::ArrayNewDefault(slots.array)); + super::vectors::emit_wrap_array(function, *slots); } - HelperKind::VectorLen => { + HelperKind::VectorLen { slots } => { function.instruction(&Instruction::LocalGet(0)); - function.instruction(&Instruction::ArrayLen); + super::vectors::emit_version_len(function, *slots); } - HelperKind::VectorGet { type_idx, value } => { + HelperKind::VectorGet { + slots, + current, + value, + } => { function.instruction(&Instruction::LocalGet(0)); + function.instruction(&Instruction::Call(*current)); function.instruction(&Instruction::LocalGet(1)); - function.instruction(&Instruction::ArrayGet(*type_idx)); + function.instruction(&Instruction::ArrayGet(slots.array)); if value.is_none() { function.instruction(&Instruction::Drop); } } - HelperKind::VectorSet { type_idx, value } => { + // The host fills a vector it has just made with `_new`: no other + // version exists yet, so the cell is written in place. + HelperKind::VectorSet { + slots, + current, + value, + } => { function.instruction(&Instruction::LocalGet(0)); + function.instruction(&Instruction::Call(*current)); function.instruction(&Instruction::LocalGet(1)); if value.is_some() { function.instruction(&Instruction::LocalGet(2)); } else { function.instruction(&Instruction::I32Const(0)); } - function.instruction(&Instruction::ArraySet(*type_idx)); + function.instruction(&Instruction::ArraySet(slots.array)); } HelperKind::SumKind { variants } => { for (tag, variant) in variants.iter().enumerate() { diff --git a/src/codegen/wasm_gc/lists.rs b/src/codegen/wasm_gc/lists.rs index be60f81ca..b137ba7a4 100644 --- a/src/codegen/wasm_gc/lists.rs +++ b/src/codegen/wasm_gc/lists.rs @@ -76,6 +76,15 @@ pub(super) struct VectorFromListOps { /// reverse. Slotted alongside `from_list` so any pair of /// `(List, Vector)` registers both helpers together. pub(super) to_list: u32, + /// `current : (Vector) -> array`: the version made current, and + /// its array (`vectors.rs`). Every read of the elements goes through it. + /// `None` when the program has no Vector value: the helpers here then + /// work on plain arrays. + pub(super) current: Option, + /// `set : (Vector, i32, T, i32 owned) -> Vector`: a new version + /// with one cell written, index already checked (`vectors.rs`). `None` + /// exactly when `current` is. + pub(super) set: Option, /// `eq : (Vector, Vector) -> i32`. Length-match + per-T /// element eq. None when T isn't `list_eq_kind`-able. pub(super) eq: Option, @@ -83,6 +92,17 @@ pub(super) struct VectorFromListOps { pub(super) hash: Option, } +/// Function type indices of one `(List, Vector)` pair's helpers. +#[derive(Debug, Clone, Copy)] +struct VflTypeIdx { + from_list: u32, + to_list: u32, + /// `current` and `set`, when the vector is versioned. + versions: Option<(u32, u32)>, + eq: Option, + hash: Option, +} + #[derive(Debug, Clone, Copy)] pub(super) struct StringSplitOps { pub(super) split: u32, @@ -104,8 +124,8 @@ pub(super) struct ListHelperRegistry { /// registry). vfl_ops: HashMap, vfl_order: Vec, - /// Per-pair: `(from_list_type_idx, to_list_type_idx)`. - vfl_type_indices: HashMap, Option)>, + /// Per-pair function type indices, in registration order. + vfl_type_indices: HashMap, /// `Tuple` canonical → `List.zip` fn idx. Registered when /// the program has `List`, `List`, and `List>` @@ -266,10 +286,25 @@ impl ListHelperRegistry { *next_type_idx += 1; let to_ty = *next_type_idx; *next_type_idx += 1; + let versioned = registry.vector_slots(&vec_canonical).is_some(); + let version_tys = versioned.then(|| { + let current_ty = *next_type_idx; + *next_type_idx += 1; + let set_ty = *next_type_idx; + *next_type_idx += 1; + (current_ty, set_ty) + }); let from_fn = *next_wasm_fn_idx; *next_wasm_fn_idx += 1; let to_fn = *next_wasm_fn_idx; *next_wasm_fn_idx += 1; + let version_fns = versioned.then(|| { + let current_fn = *next_wasm_fn_idx; + *next_wasm_fn_idx += 1; + let set_fn = *next_wasm_fn_idx; + *next_wasm_fn_idx += 1; + (current_fn, set_fn) + }); // Vector eq + hash slots match the list cap — same // resolvable kinds (primitive / String / nominal record- // sum since 0.16.3). @@ -292,12 +327,22 @@ impl ListHelperRegistry { VectorFromListOps { from_list: from_fn, to_list: to_fn, + current: version_fns.map(|v| v.0), + set: version_fns.map(|v| v.1), eq: vec_eq_fn, hash: vec_hash_fn, }, ); - self.vfl_type_indices - .insert(canonical.clone(), (from_ty, to_ty, vec_eq_ty, vec_hash_ty)); + self.vfl_type_indices.insert( + canonical.clone(), + VflTypeIdx { + from_list: from_ty, + to_list: to_ty, + versions: version_tys, + eq: vec_eq_ty, + hash: vec_hash_ty, + }, + ); self.vfl_order.push(canonical.clone()); } @@ -424,12 +469,14 @@ impl ListHelperRegistry { )))?; let elem = TypeRegistry::list_element_type(canonical).unwrap(); let vec_canonical = format!("Vector<{}>", elem.trim()); - let vec_idx = - registry + let vec_idx = match registry.vector_slots(&vec_canonical) { + Some(slots) => slots.version, + None => registry .vector_type_idx(&vec_canonical) .ok_or(WasmGcError::Validation(format!( "vector `{vec_canonical}` not registered for from_list" - )))?; + )))?, + }; let list_ref = ValType::Ref(RefType { nullable: true, heap_type: HeapType::Concrete(list_idx), @@ -442,6 +489,12 @@ impl ListHelperRegistry { types.ty().function([list_ref], [vec_ref]); // to_list : (Vector) -> List types.ty().function([vec_ref], [list_ref]); + // current, set (`vectors.rs`), for a versioned vector + if registry.vector_slots(&vec_canonical).is_some() { + for (params, results) in super::vectors::helper_types(&vec_canonical, registry)? { + types.ty().function(params, results); + } + } let elem = TypeRegistry::list_element_type(canonical).unwrap(); if list_eq_kind(elem.trim(), registry).is_some() { // eq : (Vector, Vector) -> i32 @@ -534,13 +587,17 @@ impl ListHelperRegistry { } } for canonical in &self.vfl_order { - let (from_t, to_t, eq_t, hash_t) = self.vfl_type_indices[canonical]; - funcs.function(from_t); - funcs.function(to_t); - if let Some(t) = eq_t { + let idx = self.vfl_type_indices[canonical]; + funcs.function(idx.from_list); + funcs.function(idx.to_list); + if let Some((current, set)) = idx.versions { + funcs.function(current); + funcs.function(set); + } + if let Some(t) = idx.eq { funcs.function(t); } - if let Some(t) = hash_t { + if let Some(t) = idx.hash { funcs.function(t); } } @@ -608,9 +665,24 @@ impl ListHelperRegistry { } } for canonical in &self.vfl_order { - codes.function(&emit_vec_from_list(canonical, registry)?); - codes.function(&emit_vec_to_list(canonical, registry)?); let ops = self.vfl_ops[canonical]; + let vec_canonical = format!( + "Vector<{}>", + TypeRegistry::list_element_type(canonical).unwrap().trim() + ); + codes.function(&emit_vec_from_list(canonical, registry)?); + codes.function(&emit_vec_to_list(canonical, registry, ops.current)?); + if let Some(current) = ops.current { + codes.function(&super::vectors::emit_vector_current( + &vec_canonical, + registry, + )?); + codes.function(&super::vectors::emit_vector_set( + &vec_canonical, + registry, + current, + )?); + } if ops.eq.is_some() { let elem = TypeRegistry::list_element_type(canonical).unwrap(); let kind = list_eq_kind(elem.trim(), registry).unwrap(); @@ -621,6 +693,7 @@ impl ListHelperRegistry { string_eq_fn_idx, eq_helper_fn_idx, aint_eq_fn_idx, + ops.current, )?); codes.function(&emit_vec_hash( canonical, @@ -628,6 +701,7 @@ impl ListHelperRegistry { kind, string_eq_fn_idx, hash_helper_fn_idx, + ops.current, )?); } } @@ -1120,6 +1194,16 @@ fn vec_idx_of_pair( Ok((vec_idx, elem_val)) } +/// The version slots of the `Vector` paired with `List`, when the +/// program has Vector values. +fn vector_slots_of_pair( + list_canonical: &str, + registry: &TypeRegistry, +) -> Option { + let elem = TypeRegistry::list_element_type(list_canonical).unwrap(); + registry.vector_slots(&format!("Vector<{}>", elem.trim())) +} + /// `len : (List) -> i64`. fn emit_list_len(canonical: &str, registry: &TypeRegistry) -> Result { let list_idx = list_idx_of(canonical, registry)?; @@ -1286,6 +1370,9 @@ fn emit_vec_from_list(canonical: &str, registry: &TypeRegistry) -> Result Result`, `List`) pair — `T` reads off /// the registered list canonical. -fn emit_vec_to_list(canonical: &str, registry: &TypeRegistry) -> Result { +fn emit_vec_to_list( + canonical: &str, + registry: &TypeRegistry, + current_fn: Option, +) -> Result { let list_idx = list_idx_of(canonical, registry)?; let (vec_idx, _) = vec_idx_of_pair(canonical, registry)?; let list_ref = ValType::Ref(RefType { nullable: true, heap_type: HeapType::Concrete(list_idx), }); - // params: 0=vec. locals: 1=acc, 2=i. - let mut f = Function::new([(1, list_ref), (1, ValType::I32)]); + let array_ref = ValType::Ref(RefType { + nullable: true, + heap_type: HeapType::Concrete(vec_idx), + }); + // params: 0=vector. locals: 1=acc, 2=i, and for a versioned vector + // 3=its current array; a plain array is read straight from 0. + let mut locals = vec![(1, list_ref), (1, ValType::I32)]; + let array = match current_fn { + Some(_) => { + locals.push((1, array_ref)); + 3 + } + None => 0, + }; + let mut f = Function::new(locals); + if let Some(current_fn) = current_fn { + f.instruction(&Instruction::LocalGet(0)); + f.instruction(&Instruction::Call(current_fn)); + f.instruction(&Instruction::LocalSet(array)); + } // acc = null f.instruction(&Instruction::RefNull(HeapType::Concrete(list_idx))); f.instruction(&Instruction::LocalSet(1)); // i = vec.len - 1 - f.instruction(&Instruction::LocalGet(0)); + f.instruction(&Instruction::LocalGet(array)); f.instruction(&Instruction::ArrayLen); f.instruction(&Instruction::I32Const(1)); f.instruction(&Instruction::I32Sub); @@ -2341,7 +2450,7 @@ fn emit_vec_to_list(canonical: &str, registry: &TypeRegistry) -> Result, Vector) -> i32`. Length check + element- /// wise eq via per-T instruction. Same `T must be eq-able` rule as /// list_eq. @@ -2957,20 +3108,37 @@ fn emit_vec_eq( string_eq_fn_idx: Option, eq_helper_fn_idx: &std::collections::HashMap, aint_eq_fn_idx: Option, + current_fn: Option, ) -> Result { let (vec_idx, _) = vec_idx_of_pair(canonical, registry)?; - // params: 0=va, 1=vb. locals: 2=len, 3=i. - let mut f = Function::new([(1, ValType::I32), (1, ValType::I32)]); - f.instruction(&Instruction::LocalGet(0)); + let array_ref = ValType::Ref(RefType { + nullable: true, + heap_type: HeapType::Concrete(vec_idx), + }); + // params: 0=va, 1=vb. locals: 2=len, 3=i, and for versioned vectors + // 4/5=their arrays; plain arrays are read straight from 0 and 1. + let mut locals = vec![(1, ValType::I32), (1, ValType::I32)]; + let (a, b) = match current_fn { + Some(_) => { + locals.extend([(1, array_ref), (1, array_ref)]); + (4, 5) + } + None => (0, 1), + }; + let mut f = Function::new(locals); + if let Some(current_fn) = current_fn { + emit_both_arrays(&mut f, vec_idx, current_fn); + } + f.instruction(&Instruction::LocalGet(a)); f.instruction(&Instruction::ArrayLen); - f.instruction(&Instruction::LocalGet(1)); + f.instruction(&Instruction::LocalGet(b)); f.instruction(&Instruction::ArrayLen); f.instruction(&Instruction::I32Ne); f.instruction(&Instruction::If(BlockType::Empty)); f.instruction(&Instruction::I32Const(0)); f.instruction(&Instruction::Return); f.instruction(&Instruction::End); - f.instruction(&Instruction::LocalGet(0)); + f.instruction(&Instruction::LocalGet(a)); f.instruction(&Instruction::ArrayLen); f.instruction(&Instruction::LocalSet(2)); f.instruction(&Instruction::I32Const(0)); @@ -2981,10 +3149,10 @@ fn emit_vec_eq( f.instruction(&Instruction::LocalGet(2)); f.instruction(&Instruction::I32GeU); f.instruction(&Instruction::BrIf(1)); - f.instruction(&Instruction::LocalGet(0)); + f.instruction(&Instruction::LocalGet(a)); f.instruction(&Instruction::LocalGet(3)); f.instruction(&Instruction::ArrayGet(vec_idx)); - f.instruction(&Instruction::LocalGet(1)); + f.instruction(&Instruction::LocalGet(b)); f.instruction(&Instruction::LocalGet(3)); f.instruction(&Instruction::ArrayGet(vec_idx)); match &kind { @@ -3043,6 +3211,7 @@ fn emit_vec_hash( kind: ListEqKind, _string_eq_fn_idx: Option, hash_helper_fn_idx: &std::collections::HashMap, + current_fn: Option, ) -> Result { let (vec_idx, _) = vec_idx_of_pair(canonical, registry)?; let elem = TypeRegistry::list_element_type(canonical).unwrap(); @@ -3080,10 +3249,31 @@ fn emit_vec_hash( } _ => {} } + // A versioned vector's current array, after every other local; a plain + // array is read straight from 0. + let array_local = match current_fn { + Some(_) => { + let local = 1 + locals.iter().map(|(n, _)| n).sum::(); + locals.push(( + 1, + ValType::Ref(RefType { + nullable: true, + heap_type: HeapType::Concrete(vec_idx), + }), + )); + local + } + None => 0, + }; let mut f = Function::new(locals); + if let Some(current_fn) = current_fn { + f.instruction(&Instruction::LocalGet(0)); + f.instruction(&Instruction::Call(current_fn)); + f.instruction(&Instruction::LocalSet(array_local)); + } f.instruction(&Instruction::I32Const(5381)); f.instruction(&Instruction::LocalSet(1)); - f.instruction(&Instruction::LocalGet(0)); + f.instruction(&Instruction::LocalGet(array_local)); f.instruction(&Instruction::ArrayLen); f.instruction(&Instruction::LocalSet(2)); f.instruction(&Instruction::I32Const(0)); @@ -3100,7 +3290,7 @@ fn emit_vec_hash( f.instruction(&Instruction::I32Shl); f.instruction(&Instruction::LocalGet(1)); f.instruction(&Instruction::I32Add); - f.instruction(&Instruction::LocalGet(0)); + f.instruction(&Instruction::LocalGet(array_local)); f.instruction(&Instruction::LocalGet(3)); f.instruction(&Instruction::ArrayGet(vec_idx)); let _ = elem; diff --git a/src/codegen/wasm_gc/maps.rs b/src/codegen/wasm_gc/maps.rs index f745921b6..fabbd051c 100644 --- a/src/codegen/wasm_gc/maps.rs +++ b/src/codegen/wasm_gc/maps.rs @@ -1363,8 +1363,8 @@ fn key_storage_null_heap(k_aver: &str, registry: &TypeRegistry) -> HeapType { if let Some(l) = registry.list_type_idx(k_aver) { return HeapType::Concrete(l); } - if let Some(v) = registry.vector_type_idx(k_aver) { - return HeapType::Concrete(v); + if let Some(v) = registry.vector_slots(k_aver) { + return HeapType::Concrete(v.version); } if let Some(slots) = registry.map_slots(k_aver) { return HeapType::Concrete(slots.map); diff --git a/src/codegen/wasm_gc/mod.rs b/src/codegen/wasm_gc/mod.rs index 6ced06528..526cb931e 100644 --- a/src/codegen/wasm_gc/mod.rs +++ b/src/codegen/wasm_gc/mod.rs @@ -77,6 +77,7 @@ mod run_fail; mod tests; mod types; mod types_discovery; +mod vectors; mod view; mod wasip2_capability_imports; mod wasip2_disk_bytes; diff --git a/src/codegen/wasm_gc/module.rs b/src/codegen/wasm_gc/module.rs index 1af2b8085..b8eadd50c 100644 --- a/src/codegen/wasm_gc/module.rs +++ b/src/codegen/wasm_gc/module.rs @@ -2349,6 +2349,12 @@ pub(super) fn emit_module_with( capability_int_abi, &super::capability_abi::CollectionAbiHelpers { maps: &|canonical| map_helpers.kv_helpers(canonical), + vector_current: &|canonical| { + let element = super::types::TypeRegistry::vector_element_type(canonical)?; + list_helpers + .vfl_ops_for(&format!("List<{}>", element.trim())) + .and_then(|ops| ops.current) + }, packed_sequences: &|name| packed_sequence_helpers.ops_for(name), }, &mut types, @@ -6611,6 +6617,38 @@ fn emit_user_types( mutable: true, }), )); + // A `Vector` value: a version over that array (`vectors.rs`), + // and what one cell held in an older version. A program with no + // Vector value has neither. + let Some(&slots) = registry.vector_versions.get(canonical) else { + continue; + }; + let nullable = |heap: u32| { + wasm_encoder::StorageType::Val(ValType::Ref(wasm_encoder::RefType { + nullable: true, + heap_type: wasm_encoder::HeapType::Concrete(heap), + })) + }; + let field = |element_type| wasm_encoder::FieldType { + element_type, + mutable: true, + }; + entries.push(( + slots.version, + mk_struct(vec![ + field(nullable(slots.array)), + field(nullable(slots.diff)), + field(wasm_encoder::StorageType::Val(ValType::I32)), + ]), + )); + entries.push(( + slots.diff, + mk_struct(vec![ + field(nullable(slots.version)), + field(wasm_encoder::StorageType::Val(ValType::I32)), + field(wasm_encoder::StorageType::Val(elem_val)), + ]), + )); } // `Result` — `(struct (mut i32 tag) (mut T ok) (mut E err))`. diff --git a/src/codegen/wasm_gc/types.rs b/src/codegen/wasm_gc/types.rs index 4ffdb074d..c9cb7ebfc 100644 --- a/src/codegen/wasm_gc/types.rs +++ b/src/codegen/wasm_gc/types.rs @@ -85,6 +85,10 @@ pub(super) struct TypeRegistry { /// Insertion order for `vector_types` — used by module emit so /// type-section entries land at the indices the registry recorded. pub(super) vector_order: Vec, + /// The version and diff struct of every `Vector` in + /// `vector_order`. A `Vector` value is a version over the array in + /// `vector_types`; see `vectors.rs`. + pub(super) vector_versions: HashMap, /// Per-instantiation `Option` slot. Same monomorphisation /// strategy as `vector_types`. Each `Option` lowers to a /// `(struct (mut i32 tag) (mut T value))` — tag=0 None, tag=1 @@ -357,6 +361,20 @@ pub(super) struct MapSlots { pub(super) diff: u32, } +/// The wasm types of one `Vector` instantiation (see `vectors.rs`). +#[derive(Debug, Clone, Copy)] +pub(super) struct VectorSlots { + /// `(array (mut T))` — the elements. + pub(super) array: u32, + /// `(struct (mut array_ref arr) (mut diff_ref diff) (mut i32 held))` — + /// a `Vector` value. A null `diff` marks the version that owns the + /// array's current contents. + pub(super) version: u32, + /// `(struct (mut version_ref next) (mut i32 idx) (mut T value))` — what + /// one cell held in an older version. + pub(super) diff: u32, +} + impl TypeRegistry { /// Build the registry with a `--handler` shape — pre-register /// HttpRequest/Http.Response refs in case the handler fn is the @@ -1277,6 +1295,34 @@ impl TypeRegistry { next_idx += 1; } + // Every `Vector` is known by now: give each its version and diff + // structs, above its array, when the program has a Vector value at + // all. Every `List` and every string interpolation registers a + // `Vector` array for helpers the program may never call; a + // program with no Vector value keeps them as plain arrays, and its + // module stays what it was before versions. + let mut vector_versions: HashMap = HashMap::new(); + let versioned = program_uses_vector( + resolved_fn_defs, + &record_fields, + &variants, + capability_boundary_types, + ); + for canonical in vector_order.iter().filter(|_| versioned) { + let array = vector_types[canonical]; + let version = next_idx; + let diff = next_idx + 1; + next_idx += 2; + vector_versions.insert( + canonical.clone(), + VectorSlots { + array, + version, + diff, + }, + ); + } + // Discover unique String literals — each gets a passive data // segment idx assigned in encounter order. Walk fn bodies + any // string literals embedded in expressions; canonicalise on @@ -1466,6 +1512,7 @@ impl TypeRegistry { record_fields, vector_types, vector_order, + vector_versions, option_types, option_order, list_types, @@ -1794,6 +1841,21 @@ impl TypeRegistry { } } + /// The array, version and diff types of a registered `Vector`. + pub(super) fn vector_slots(&self, canonical: &str) -> Option { + let normalized = normalize_compound(canonical); + let aliased = apply_type_name_aliases(&normalized, &self.type_name_aliases); + if let Some(slots) = self.vector_versions.get(&aliased).copied() { + return Some(slots); + } + let bare = strip_inner_dotted_prefixes(&aliased); + if bare != aliased { + self.vector_versions.get(&bare).copied() + } else { + None + } + } + /// Element-type Aver string for a registered `Vector`. Used by /// module emit to resolve the wasm storage type of array elements. pub(super) fn vector_element_type(canonical: &str) -> Option<&str> { @@ -2336,6 +2398,44 @@ fn expr_uses_string(expr: &crate::ir::hir::ResolvedExpr) -> bool { /// table. Both `Literal::Str` and the `Literal` parts of an /// `InterpolatedStr` count — each unique byte sequence gets a passive /// data segment. +/// Whether the program has a `Vector` value anywhere: a type that names one +/// (a signature, a binding annotation, a record or variant field, a +/// capability boundary type) or a call that makes or reads one. A Vector +/// value can only come from one of those. When there is none, the +/// `Vector` arrays the registry keeps for `List` helpers and string +/// concatenation stay plain arrays and get no versions (`vectors.rs`). +fn program_uses_vector( + resolved_fn_defs: &[crate::ir::hir::ResolvedFnDef], + record_fields: &HashMap>, + variants: &HashMap>, + capability_boundary_types: &[String], +) -> bool { + use crate::ir::hir::{BuiltinIntrinsic, ResolvedCallee, ResolvedFnBody, ResolvedStmt}; + let names = |ty: &str| ty.contains("Vector<"); + let makes_or_reads = |callee: &ResolvedCallee| match callee { + ResolvedCallee::Builtin(name) => name.starts_with("Vector.") || name == "List.fromVector", + ResolvedCallee::Intrinsic(BuiltinIntrinsic::VectorNew) => true, + _ => false, + }; + resolved_fn_defs.iter().any(|fd| { + let ResolvedFnBody::Block(stmts) = fd.body.as_ref(); + names(&fd.return_type.display()) + || fd.params.iter().any(|(_, ty)| names(&ty.display())) + || stmts.iter().any(|stmt| { + matches!(stmt, ResolvedStmt::Binding { ty_ann: Some(ty), .. } if names(&ty.display())) + }) + || fn_body_reaches(fd, &makes_or_reads) + }) || record_fields + .values() + .flatten() + .any(|(_, ty)| names(ty)) + || variants + .values() + .flatten() + .any(|v| v.fields.iter().any(|ty| names(ty))) + || capability_boundary_types.iter().any(|ty| names(ty)) +} + fn fn_body_calls_builtin(fd: &crate::ir::hir::ResolvedFnDef, dotted: &str) -> bool { use crate::ir::hir::ResolvedCallee; fn_body_reaches( @@ -2877,15 +2977,22 @@ pub(super) fn aver_to_wasm( "String.Index reached wasm-gc without its hidden array slot".into(), )); } - // `Vector` resolves to `(ref null $vector_T)`. The registry's - // `vector_types` map is keyed on whitespace-stripped canonical - // form so `Vector` and `Vector< Int >` collide on the same - // slot. + // `Vector` resolves to `(ref null $vector_version_T)`, a version + // over the `(array (mut T))` (`vectors.rs`). The registry's maps are + // keyed on whitespace-stripped canonical form so `Vector` and + // `Vector< Int >` collide on the same slot. if trimmed.starts_with("Vector<") && trimmed.ends_with('>') && let Some(reg) = registry { let canonical: String = trimmed.chars().filter(|c| !c.is_whitespace()).collect(); + if let Some(slots) = reg.vector_slots(&canonical) { + return Ok(Some(ValType::Ref(RefType { + nullable: true, + heap_type: HeapType::Concrete(slots.version), + }))); + } + // A program with no Vector value: only the helpers' plain arrays. if let Some(idx) = reg.vector_type_idx(&canonical) { return Ok(Some(ValType::Ref(RefType { nullable: true, diff --git a/src/codegen/wasm_gc/vectors.rs b/src/codegen/wasm_gc/vectors.rs new file mode 100644 index 000000000..e743e4775 --- /dev/null +++ b/src/codegen/wasm_gc/vectors.rs @@ -0,0 +1,324 @@ +//! Versions of one Vector that share its array. +//! +//! A `Vector` value is a version struct over an `(array (mut T))`: +//! `arr`, `diff` and `held`. `Vector.set` writes the cell in place and +//! returns a new version over the same array, so a set costs one write and +//! two small structs, not a copy of the array. The vector it was given stays +//! a valid value: before the write, the old cell is recorded in a diff +//! struct hung on the old version, which is then "the new version, except +//! that cell `idx` holds `value`". This is the scheme `maps/versions.rs` +//! uses for a Map's buckets, after Baker ("Shallow binding makes functional +//! arrays fast", 1991). +//! +//! Exactly one version of a lineage owns the array's current contents: the +//! one whose `diff` is null. [`emit_vector_current`] makes a given version +//! that one and returns the array. It follows the `diff` chain to the +//! current version and walks back, swapping each recorded cell into the +//! array and hanging the swapped-out cell on the version it just left, so +//! every version stays readable and the chain now points the other way. +//! Every read of the elements goes through it; `len` reads the array's +//! length, which no version changes. +//! +//! `held` is set on a version once another version's diff points at it, +//! and never cleared. A set whose target is uniquely owned (nothing reads +//! that binding again) and that no other version is described against may +//! write the cell and return the same version, with nothing recorded: +//! nobody can see the old contents any more. That keeps a loop that builds +//! a vector by setting it free of allocation. +//! +//! Only a program with a Vector value gets versions (`types.rs`, +//! `program_uses_vector`). The registry also keeps a `Vector` array for +//! every `List` helper pair and for the string concatenation helper; in a +//! program with no Vector value those stay plain arrays, with no version or +//! diff struct and no `current` / `set` helper, so its module is what it was +//! before versions. +//! +//! Two versions of one lineage cannot both be current, so a helper that +//! reads two vectors at once (`eq`) copies one side when both share an +//! array. A host reads and writes a vector only through the capability ABI +//! helpers, which go through the same current version. + +use wasm_encoder::{BlockType, Function, HeapType, Instruction, RefType, ValType}; + +use super::WasmGcError; +use super::types::{TypeRegistry, VectorSlots}; + +/// Version fields. +pub(super) const ARR_FIELD: u32 = 0; +const DIFF_FIELD: u32 = 1; +const HELD_FIELD: u32 = 2; + +/// Diff fields. +const DIFF_NEXT: u32 = 0; +const DIFF_IDX: u32 = 1; +const DIFF_VALUE: u32 = 2; + +fn nullable(idx: u32) -> ValType { + ValType::Ref(RefType { + nullable: true, + heap_type: HeapType::Concrete(idx), + }) +} + +/// The slots and the wasm element type of `canonical` (`Vector`). +fn slots_and_element( + canonical: &str, + registry: &TypeRegistry, +) -> Result<(VectorSlots, ValType), WasmGcError> { + let slots = registry.vector_slots(canonical).ok_or_else(|| { + WasmGcError::Validation(format!("vector `{canonical}` has no version slots")) + })?; + let element = TypeRegistry::vector_element_type(canonical).ok_or_else(|| { + WasmGcError::Validation(format!("vector `{canonical}` has no parsable element type")) + })?; + let element = super::types::aver_to_wasm(element, Some(registry))?.unwrap_or(ValType::I32); + Ok((slots, element)) +} + +/// With an array on the stack, leave the version that owns it: a fresh +/// vector no other version is described against. +pub(super) fn emit_wrap_array(f: &mut Function, slots: VectorSlots) { + f.instruction(&Instruction::RefNull(HeapType::Concrete(slots.diff))); + f.instruction(&Instruction::I32Const(0)); + f.instruction(&Instruction::StructNew(slots.version)); +} + +/// With a version on the stack, leave its array's length. Every version of +/// a lineage has the same length, so this needs no reroot. +pub(super) fn emit_version_len(f: &mut Function, slots: VectorSlots) { + f.instruction(&Instruction::StructGet { + struct_type_index: slots.version, + field_index: ARR_FIELD, + }); + f.instruction(&Instruction::ArrayLen); +} + +/// `current(vector) -> array`: make `vector` the version that owns its +/// array's contents, and return the array. +pub(super) fn emit_vector_current( + canonical: &str, + registry: &TypeRegistry, +) -> Result { + let (slots, element) = slots_and_element(canonical, registry)?; + let version_ref = nullable(slots.version); + // params: 0=vector. locals: 1=prev, 2=cur, 3=diff, 4=next, 5=current + // version, 6=version being restored, 7=idx, 8=array, 9=cell swapped out. + let mut f = Function::new([ + (1, version_ref), + (1, version_ref), + (1, nullable(slots.diff)), + (1, version_ref), + (1, version_ref), + (1, version_ref), + (1, ValType::I32), + (1, nullable(slots.array)), + (1, element), + ]); + let get_version = |f: &mut Function, field| { + f.instruction(&Instruction::StructGet { + struct_type_index: slots.version, + field_index: field, + }); + }; + let set_version = |f: &mut Function, field| { + f.instruction(&Instruction::StructSet { + struct_type_index: slots.version, + field_index: field, + }); + }; + let get_diff = |f: &mut Function, field| { + f.instruction(&Instruction::StructGet { + struct_type_index: slots.diff, + field_index: field, + }); + }; + let set_diff = |f: &mut Function, field| { + f.instruction(&Instruction::StructSet { + struct_type_index: slots.diff, + field_index: field, + }); + }; + + // Already current: every read of a vector used once. + f.instruction(&Instruction::LocalGet(0)); + get_version(&mut f, DIFF_FIELD); + f.instruction(&Instruction::RefIsNull); + f.instruction(&Instruction::If(BlockType::Empty)); + f.instruction(&Instruction::LocalGet(0)); + get_version(&mut f, ARR_FIELD); + f.instruction(&Instruction::Return); + f.instruction(&Instruction::End); + + // Walk to the current version, turning each diff's `next` round to + // point at the version before it. `prev` ends on the last stale + // version, whose diff is relative to the current one. + f.instruction(&Instruction::RefNull(HeapType::Concrete(slots.version))); + f.instruction(&Instruction::LocalSet(1)); + f.instruction(&Instruction::LocalGet(0)); + f.instruction(&Instruction::LocalSet(2)); + f.instruction(&Instruction::Block(BlockType::Empty)); + f.instruction(&Instruction::Loop(BlockType::Empty)); + f.instruction(&Instruction::LocalGet(2)); + get_version(&mut f, DIFF_FIELD); + f.instruction(&Instruction::LocalTee(3)); + f.instruction(&Instruction::RefIsNull); + f.instruction(&Instruction::BrIf(1)); + f.instruction(&Instruction::LocalGet(3)); + get_diff(&mut f, DIFF_NEXT); + f.instruction(&Instruction::LocalSet(4)); + f.instruction(&Instruction::LocalGet(3)); + f.instruction(&Instruction::LocalGet(1)); + set_diff(&mut f, DIFF_NEXT); + f.instruction(&Instruction::LocalGet(2)); + f.instruction(&Instruction::LocalSet(1)); + f.instruction(&Instruction::LocalGet(4)); + f.instruction(&Instruction::LocalSet(2)); + f.instruction(&Instruction::Br(0)); + f.instruction(&Instruction::End); + f.instruction(&Instruction::End); + + // Every version in the chain shares the current version's array. + f.instruction(&Instruction::LocalGet(2)); + f.instruction(&Instruction::LocalSet(5)); + f.instruction(&Instruction::LocalGet(5)); + get_version(&mut f, ARR_FIELD); + f.instruction(&Instruction::LocalSet(8)); + + // Walk back towards `vector`, one version at a time: put that version's + // cell back into the array, and keep what was there as the diff of the + // version just left, pointing forward to the restored one. + f.instruction(&Instruction::Block(BlockType::Empty)); + f.instruction(&Instruction::Loop(BlockType::Empty)); + f.instruction(&Instruction::LocalGet(1)); + f.instruction(&Instruction::RefIsNull); + f.instruction(&Instruction::BrIf(1)); + f.instruction(&Instruction::LocalGet(1)); + f.instruction(&Instruction::LocalSet(6)); + f.instruction(&Instruction::LocalGet(6)); + get_version(&mut f, DIFF_FIELD); + f.instruction(&Instruction::LocalSet(3)); + // prev = the version before this one (the reversed link). + f.instruction(&Instruction::LocalGet(3)); + get_diff(&mut f, DIFF_NEXT); + f.instruction(&Instruction::LocalSet(1)); + f.instruction(&Instruction::LocalGet(3)); + get_diff(&mut f, DIFF_IDX); + f.instruction(&Instruction::LocalSet(7)); + // Swap the cell out, and the restored version's cell in. + f.instruction(&Instruction::LocalGet(8)); + f.instruction(&Instruction::LocalGet(7)); + f.instruction(&Instruction::ArrayGet(slots.array)); + f.instruction(&Instruction::LocalSet(9)); + f.instruction(&Instruction::LocalGet(8)); + f.instruction(&Instruction::LocalGet(7)); + f.instruction(&Instruction::LocalGet(3)); + get_diff(&mut f, DIFF_VALUE); + f.instruction(&Instruction::ArraySet(slots.array)); + // The diff now describes the version just left, relative to the + // restored one, which is therefore held. + f.instruction(&Instruction::LocalGet(3)); + f.instruction(&Instruction::LocalGet(9)); + set_diff(&mut f, DIFF_VALUE); + f.instruction(&Instruction::LocalGet(3)); + f.instruction(&Instruction::LocalGet(6)); + set_diff(&mut f, DIFF_NEXT); + f.instruction(&Instruction::LocalGet(5)); + f.instruction(&Instruction::LocalGet(3)); + set_version(&mut f, DIFF_FIELD); + f.instruction(&Instruction::LocalGet(6)); + f.instruction(&Instruction::RefNull(HeapType::Concrete(slots.diff))); + set_version(&mut f, DIFF_FIELD); + f.instruction(&Instruction::LocalGet(6)); + f.instruction(&Instruction::I32Const(1)); + set_version(&mut f, HELD_FIELD); + f.instruction(&Instruction::LocalGet(6)); + f.instruction(&Instruction::LocalSet(5)); + f.instruction(&Instruction::Br(0)); + f.instruction(&Instruction::End); + f.instruction(&Instruction::End); + + f.instruction(&Instruction::LocalGet(8)); + f.instruction(&Instruction::End); + Ok(f) +} + +/// `set(vector, idx, value, owned) -> vector`: the vector with cell `idx` +/// holding `value`. The caller has checked that `idx` is in range. `owned` +/// is non-zero when nothing reads `vector` after the set; the cell is then +/// written with nothing recorded when no other version is described +/// against `vector`. +pub(super) fn emit_vector_set( + canonical: &str, + registry: &TypeRegistry, + current_fn: u32, +) -> Result { + let (slots, _) = slots_and_element(canonical, registry)?; + // params: 0=vector, 1=idx, 2=value, 3=owned. locals: 4=array, 5=new version. + let mut f = Function::new([(1, nullable(slots.array)), (1, nullable(slots.version))]); + f.instruction(&Instruction::LocalGet(0)); + f.instruction(&Instruction::Call(current_fn)); + f.instruction(&Instruction::LocalSet(4)); + + f.instruction(&Instruction::LocalGet(3)); + f.instruction(&Instruction::LocalGet(0)); + f.instruction(&Instruction::StructGet { + struct_type_index: slots.version, + field_index: HELD_FIELD, + }); + f.instruction(&Instruction::I32Eqz); + f.instruction(&Instruction::I32And); + f.instruction(&Instruction::If(BlockType::Empty)); + f.instruction(&Instruction::LocalGet(4)); + f.instruction(&Instruction::LocalGet(1)); + f.instruction(&Instruction::LocalGet(2)); + f.instruction(&Instruction::ArraySet(slots.array)); + f.instruction(&Instruction::LocalGet(0)); + f.instruction(&Instruction::Return); + f.instruction(&Instruction::End); + + // The new version, held by the old one's diff. + f.instruction(&Instruction::LocalGet(4)); + f.instruction(&Instruction::RefNull(HeapType::Concrete(slots.diff))); + f.instruction(&Instruction::I32Const(1)); + f.instruction(&Instruction::StructNew(slots.version)); + f.instruction(&Instruction::LocalSet(5)); + // The old version: the new one, except for the old cell. + f.instruction(&Instruction::LocalGet(0)); + f.instruction(&Instruction::LocalGet(5)); + f.instruction(&Instruction::LocalGet(1)); + f.instruction(&Instruction::LocalGet(4)); + f.instruction(&Instruction::LocalGet(1)); + f.instruction(&Instruction::ArrayGet(slots.array)); + f.instruction(&Instruction::StructNew(slots.diff)); + f.instruction(&Instruction::StructSet { + struct_type_index: slots.version, + field_index: DIFF_FIELD, + }); + f.instruction(&Instruction::LocalGet(4)); + f.instruction(&Instruction::LocalGet(1)); + f.instruction(&Instruction::LocalGet(2)); + f.instruction(&Instruction::ArraySet(slots.array)); + f.instruction(&Instruction::LocalGet(5)); + f.instruction(&Instruction::End); + Ok(f) +} + +/// A function type: its params and results. +pub(super) type FuncType = (Vec, Vec); + +/// The function types of the two helpers, in registration order: +/// `current : (vector) -> array`, `set : (vector, i32, T, i32) -> vector`. +pub(super) fn helper_types( + canonical: &str, + registry: &TypeRegistry, +) -> Result<[FuncType; 2], WasmGcError> { + let (slots, element) = slots_and_element(canonical, registry)?; + let version_ref = nullable(slots.version); + Ok([ + (vec![version_ref], vec![nullable(slots.array)]), + ( + vec![version_ref, ValType::I32, element, ValType::I32], + vec![version_ref], + ), + ]) +} diff --git a/tests/cert_verify_spec.rs b/tests/cert_verify_spec.rs index 2ed17ba20..19d6c8e84 100644 --- a/tests/cert_verify_spec.rs +++ b/tests/cert_verify_spec.rs @@ -4935,21 +4935,15 @@ fn cert_verify_declines_tampered_int_dispatch_plan() { ); } -/// End-to-end acceptance and fail-closed tamper coverage for the fused -/// `Option.withDefault(Vector.get(vec, idx), d)` read: a `cellAt`-shaped export -/// reaches CERTIFIED, and each of the three holes an attacker could try to -/// move — the literal default, the declared vector array type, and the -/// to-index/box helper wiring — is pinned, so a consistent rewrite of the -/// attacker-editable package data is DECLINED, never re-credited. +/// A function that reads a Vector is declined, and says why. A `Vector` +/// value is a version struct over its array on wasm-gc, and a read may reroot +/// the versions sharing that array; the wall models a Vector as the plain +/// array of its elements, so `cellAt` (the fused +/// `Option.withDefault(Vector.get(vec, idx), d)` read) must not be offered. #[test] -fn cert_verify_accepts_fused_vector_read_and_declines_three_tampers() { - if !lean_required::lake_available() { - eprintln!("skipping fused vector-read verify test: `lake` not available"); - return; - } - +fn cert_declines_a_function_that_reads_a_vector() { let repo_root = PathBuf::from(env!("CARGO_MANIFEST_DIR")); - let out_dir = temp_dir("cert-fused-vector-read"); + let out_dir = temp_dir("cert-vector-declined"); let compile = aver_command() .current_dir(&repo_root) .arg("compile") @@ -4967,102 +4961,27 @@ fn cert_verify_accepts_fused_vector_read_and_declines_three_tampers() { String::from_utf8_lossy(&compile.stdout), String::from_utf8_lossy(&compile.stderr) ); - let wasm = out_dir.join("cell_at.wasm"); - let cert = out_dir.join("cert"); - - let (ok, report) = aver_verify(&wasm, &cert); - assert!(ok, "fused vector read must verify CERTIFIED:\n{report}"); + let manifest: serde_json::Value = serde_json::from_str( + &std::fs::read_to_string(out_dir.join("cert").join("cert-manifest.json")).unwrap(), + ) + .unwrap(); + let certified = manifest["certified"].as_array().expect("certified list"); assert!( - report.contains("CERTIFIED") && report.contains("cellAt"), - "verdict must credit cellAt:\n{report}" + certified.iter().all(|entry| entry["name"] != "cellAt"), + "cellAt must not be certified: {certified:?}" + ); + let reason = manifest["declaredUncertified"] + .as_array() + .expect("declaredUncertified list") + .iter() + .find(|entry| entry["name"] == "cellAt") + .and_then(|entry| entry["reason"].as_str()) + .expect("cellAt is declared uncertified with a reason") + .to_string(); + assert!( + reason.contains("Vector") && reason.contains("versioned struct"), + "the reason names the Vector representation: {reason}" ); - - let manifest: serde_json::Value = - serde_json::from_str(&std::fs::read_to_string(cert.join("cert-manifest.json")).unwrap()) - .unwrap(); - let to_index_idx = manifest["hostRoleTable"]["toIndex"] - .as_u64() - .expect("cell_at declares the index helper"); - let box_idx = manifest["hostRoleTable"]["box"] - .as_u64() - .expect("cell_at declares the box helper"); - assert_ne!(to_index_idx, box_idx, "helper roles must be distinct"); - let plans = std::fs::read_to_string(cert.join("Plans.lean")).unwrap(); - let vecs_at = plans - .find("vecs := [(.int, ") - .expect("cell_at declares Vector") - + "vecs := [(.int, ".len(); - let arr_ty: u32 = plans[vecs_at..] - .split(')') - .next() - .unwrap() - .parse() - .expect("the Vector array index"); - - // (1) the literal default `0` becomes `1`; (2) the declared Vector - // array type index moves; (3) the index and box helpers swap in the - // subject's role table. - let helper_swap_from = format!("box := some {box_idx}, add := "); - let helper_swap_to = format!("box := some {to_index_idx}, add := "); - let vec_from = format!("vecs := [(.int, {arr_ty})]"); - let vec_to = format!("vecs := [(.int, {})]", arr_ty + 1); - let to_index_from = format!("toIndex := some {to_index_idx}"); - let to_index_to = format!("toIndex := some {box_idx}"); - for (label, file, from, to) in [ - ( - "default literal", - "plan:cellAt", - "(.literal (.int 0))", - "(.literal (.int 1))", - ), - ( - "array type", - "Plans.lean", - vec_from.as_str(), - vec_to.as_str(), - ), - ( - "helper swap", - "Manifest.lean", - helper_swap_from.as_str(), - helper_swap_to.as_str(), - ), - ] { - let dir = temp_dir(&format!( - "cert-fused-vector-read-{}", - label.replace(' ', "-") - )); - copy_dir(&out_dir, &dir); - let tampered = dir.join("cert"); - if let Some(export) = file.strip_prefix("plan:") { - tamper_export_plan(&tampered.join("Plans.lean"), export, from, to); - } else { - replace_once(&tampered.join(file), from, to); - } - if label == "helper swap" { - replace_once( - &tampered.join("Manifest.lean"), - &to_index_from, - &to_index_to, - ); - let mf = tampered.join("cert-manifest.json"); - let mut m: serde_json::Value = - serde_json::from_str(&std::fs::read_to_string(&mf).unwrap()).unwrap(); - m["hostRoleTable"]["box"] = serde_json::json!(to_index_idx); - m["hostRoleTable"]["toIndex"] = serde_json::json!(box_idx); - std::fs::write(&mf, serde_json::to_string_pretty(&m).unwrap()).unwrap(); - } - let (ok, out) = aver_verify(&dir.join("cell_at.wasm"), &tampered); - assert!(!ok, "tamper `{label}` must be DECLINED:\n{out}"); - assert!( - out.contains("DECLINED"), - "tamper `{label}` must report a decline verdict, not an error:\n{out}" - ); - assert!( - !out.contains("CERTIFIED"), - "tamper `{label}` must never re-credit the export:\n{out}" - ); - } } /// The five real source functions the Int value-comparison faces were built diff --git a/tests/wasm_gc_container_read_ownership.rs b/tests/wasm_gc_container_read_ownership.rs index 2e3f59d80..55587ded4 100644 --- a/tests/wasm_gc_container_read_ownership.rs +++ b/tests/wasm_gc_container_read_ownership.rs @@ -24,9 +24,11 @@ //! asserted against the same hand-computed literal — three backends //! agreeing on a wrong value still fails. The two green-by-design cells //! (fresh local, fresh chain) pin that the conservative fix did not -//! swallow the owned fast path, and `fresh_local_set_stays_in_place` -//! additionally pins the *emitted code*: the clone (`array.new_default`) -//! exists only in the container-read variant. +//! swallow the owned fast path. A `Vector.set` no longer clones in either +//! case: it writes the cell and keeps the old one with the vector it was +//! given (`src/codegen/wasm_gc/vectors.rs`), so the map's vector stays what +//! it was without a copy, and `neither_a_fresh_nor_a_container_read_set_copies` pins that +//! neither variant allocates an array for the set. #![cfg(feature = "wasm")] @@ -852,13 +854,10 @@ fn wat_of(source: &str) -> String { /// The two programs are identical except for `held`'s provenance — /// fresh `Vector.fromList` vs a read out of the map — so their type / /// helper sets match and the WAT `array.new_default` count isolates the -/// ownership decision at the `Vector.set` site. The container-read -/// variant must carry exactly one more (its clone-before-mutate); if -/// the fresh variant ever gains one, the fast path was silently -/// pessimized, and if the read variant loses its extra one, the copy -/// guard regressed. +/// `Vector.set` site. Neither variant clones: the set is versioned, and +/// the map's vector keeps its old cell through the version it is. #[test] -fn fresh_local_set_stays_in_place_while_container_read_copies() { +fn neither_a_fresh_nor_a_container_read_set_copies() { let fresh = r#" fn probe(x: Int) -> Int ? "Fresh receiver: the set may mutate in place." @@ -892,11 +891,8 @@ fn main() -> Unit let fresh_clones = wat_of(fresh).matches("array.new_default").count(); let read_clones = wat_of(read).matches("array.new_default").count(); assert_eq!( - read_clones, - fresh_clones + 1, - "expected the container-read variant to carry exactly one more \ - array.new_default (the clone-before-mutate) than the fresh \ - variant ({read_clones} vs {fresh_clones}) — either the fresh \ - fast path was pessimized or the copy guard regressed" + read_clones, fresh_clones, + "expected neither variant to clone for the set ({read_clones} vs \ + {fresh_clones} array.new_default)" ); } diff --git a/tests/wasm_gc_vector_versions_spec.rs b/tests/wasm_gc_vector_versions_spec.rs new file mode 100644 index 000000000..7cb43c9e6 --- /dev/null +++ b/tests/wasm_gc_vector_versions_spec.rs @@ -0,0 +1,411 @@ +//! Versions of one wasm-gc Vector. +//! +//! `Vector.set` writes the cell in place and leaves the vector it was given +//! valid through a diff (see `src/codegen/wasm_gc/vectors.rs`). These tests +//! keep older versions in use after newer ones are made from them: a chain of +//! writes read back in any order, two branches from one base, equality +//! between versions that share an array, a builder loop whose base is kept, +//! and a Vector in a record field that the caller keeps. Each program runs on +//! the VM and on wasm-gc, and both must print the hand-checked answer. +//! +//! Before versions, `Vector.set` copied the whole array whenever it could not +//! prove its receiver unique, which it cannot for a Vector held in a record +//! field. + +#![cfg(feature = "wasm")] + +#[path = "support/aver_cmd.rs"] +mod aver_cmd; + +use aver_cmd::{cleanup, format_output, temp_module}; + +use std::path::PathBuf; +use std::process::Command; + +fn run_cli(prefix: &str, source: &str, extra_args: &[&str]) -> String { + let repo_root = PathBuf::from(env!("CARGO_MANIFEST_DIR")); + let path = temp_module(prefix, source); + let out = Command::new(env!("CARGO_BIN_EXE_aver")) + .current_dir(&repo_root) + .arg("run") + .arg(&path) + .args(extra_args) + .output() + .expect("expected `aver run` to execute"); + cleanup(&path); + assert!( + out.status.success(), + "{prefix} run {extra_args:?} failed:\n{}", + format_output(&out) + ); + String::from_utf8_lossy(&out.stdout).trim().to_string() +} + +fn assert_vm_and_wasm_gc(name: &str, source: &str, expected: &str) { + let vm = run_cli(name, source, &[]); + assert_eq!( + vm, expected, + "{name}: the VM diverged from the checked answer" + ); + let wasm = run_cli(name, source, &["--wasm-gc"]); + assert_eq!( + wasm, expected, + "{name}: wasm-gc diverged from the checked answer — a version of a vector \ + saw a write made to another one" + ); +} + +const AT: &str = r#" +fn at(v: Vector, i: Int) -> Int + ? "The cell, or -1 outside the vector." + Option.withDefault(Vector.get(v, i), 0 - 1) + +fn put(v: Vector, i: Int, x: Int) -> Vector + ? "The vector with one cell written, or the vector itself outside it." + match Vector.set(v, i, x) + Option.Some(w) -> w + Option.None -> v + +fn show(v: Vector) -> String + ? "The cells, comma-separated." + showFrom(List.fromVector(v), "") + +fn showFrom(cells: List, acc: String) -> String + ? "The cells after acc." + match cells + [] -> acc + [c, ..rest] -> showFrom(rest, "{acc}{c},") +"#; + +fn program(body: &str) -> String { + format!( + "module Main\n intent = \"Versions of one vector.\"\n effects [Console.print]\n{AT}\n{body}" + ) +} + +#[test] +fn every_version_of_a_chain_reads_as_itself() { + assert_vm_and_wasm_gc( + "vector-versions-chain", + &program( + r#" +fn main() -> Unit + ! [Console.print] + v0 = Vector.new(4, 0) + v1 = put(v0, 0, 1) + v2 = put(v1, 1, 2) + v3 = put(v2, 0, 3) + v4 = put(v3, 3, 4) + Console.print("{show(v0)} {show(v2)} {show(v4)} {show(v1)} {show(v3)} {show(v0)} {show(v4)}") +"#, + ), + "0,0,0,0, 1,2,0,0, 3,2,0,4, 1,0,0,0, 3,2,0,0, 0,0,0,0, 3,2,0,4,", + ); +} + +#[test] +fn two_branches_from_one_base_stay_apart() { + assert_vm_and_wasm_gc( + "vector-versions-branches", + &program( + r#" +fn main() -> Unit + ! [Console.print] + base = put(put(Vector.new(3, 0), 0, 5), 2, 7) + left = put(base, 1, 1) + right = put(put(base, 1, 2), 0, 9) + leftAgain = put(left, 2, 8) + Console.print("{at(left, 1)} {at(right, 1)} {at(base, 1)} {show(leftAgain)} {show(right)} {show(base)} {show(left)}") +"#, + ), + "1 2 0 5,1,8, 9,2,7, 5,0,7, 5,1,7,", + ); +} + +#[test] +fn equality_between_versions_that_share_an_array() { + assert_vm_and_wasm_gc( + "vector-versions-eq", + &program( + r#" +fn main() -> Unit + ! [Console.print] + base = Vector.new(3, 0) + a = put(base, 1, 4) + b = put(base, 1, 4) + c = put(a, 2, 1) + back = put(c, 2, 0) + Console.print("{a == b} {a == c} {back == a} {base == a} {base == base} {show(a)} {show(c)}") +"#, + ), + "true false true false true 0,4,0, 0,4,1,", + ); +} + +#[test] +fn a_builder_loop_keeps_the_base_it_started_from() { + assert_vm_and_wasm_gc( + "vector-versions-builder", + &program( + r#" +fn fill(v: Vector, i: Int) -> Vector + ? "Writes i * i at every index from i down to 0." + match i < 0 + true -> v + false -> fill(Option.withDefault(Vector.set(v, i, i * i), v), i - 1) + +fn main() -> Unit + ! [Console.print] + base = Vector.new(5, 7) + built = fill(base, 4) + again = fill(built, 1) + Console.print("{show(base)} {show(built)} {show(again)} {show(fill(Vector.new(3, 1), 2))}") +"#, + ), + "7,7,7,7,7, 0,1,4,9,16, 0,1,4,9,16, 0,1,4,", + ); +} + +#[test] +fn a_vector_in_a_record_the_caller_keeps_is_not_changed() { + assert_vm_and_wasm_gc( + "vector-versions-record", + &program( + r#" +record State + cells: Vector + count: Int + +fn step(s: State, i: Int) -> State + ? "Writes i at i; the None arm hands s back whole." + match Vector.set(s.cells, i, i + 10) + Option.Some(updated) -> State.update(s, cells = updated, count = s.count + 1) + Option.None -> s + +fn run(s: State, i: Int, n: Int) -> State + ? "step for i up to n." + match i >= n + true -> s + false -> run(step(s, i), i + 1, n) + +fn main() -> Unit + ! [Console.print] + start = State(cells = Vector.new(4, 0), count = 0) + half = run(start, 0, 2) + done = run(half, 2, 6) + Console.print("{show(start.cells)} {show(half.cells)} {show(done.cells)} {done.count} {half.count}") +"#, + ), + "0,0,0,0, 10,11,0,0, 10,11,12,13, 4 2", + ); +} + +/// The measured fixture: 2000 sets of a 100,000-cell Vector held by a record. +#[test] +fn the_record_field_fixture_answers_as_the_vm_does() { + let source = std::fs::read_to_string(concat!( + env!("CARGO_MANIFEST_DIR"), + "/tests/fixtures/vector_field_set/main.av" + )) + .expect("read the fixture"); + assert_vm_and_wasm_gc("vector-versions-fixture", &source, "total 2000"); +} + +/// Nothing reads `b` after the second set, but `a` is described against +/// `b`, so that set must still record the cell it overwrites. +#[test] +fn a_dead_version_an_older_one_needs_is_not_written_over() { + assert_vm_and_wasm_gc( + "vector-versions-held", + &program( + r#" +fn main() -> Unit + ! [Console.print] + a = Vector.new(3, 0) + c = match Vector.set(a, 0, 1) + Option.Some(b) -> Option.withDefault(Vector.set(b, 1, 2), b) + Option.None -> a + Console.print("{show(a)} {show(c)}") +"#, + ), + "0,0,0, 1,2,0,", + ); +} + +#[test] +fn a_vector_of_strings_and_a_miss_keep_their_versions() { + assert_vm_and_wasm_gc( + "vector-versions-strings", + r#" +module Main + intent = "Versions of a vector of strings, and a set past the end." + effects [Console.print] + +fn word(v: Vector, i: Int) -> String + ? "The cell, or ? outside the vector." + Option.withDefault(Vector.get(v, i), "?") + +fn main() -> Unit + ! [Console.print] + base = Vector.fromList(["a", "b", "c"]) + changed = Option.withDefault(Vector.set(base, 1, "x"), base) + missed = Vector.set(changed, 3, "y") + Console.print("{word(base, 1)} {word(changed, 1)} {word(base, 1)} {Vector.len(changed)} {missed == Option.None} {List.len(List.fromVector(base))}") +"#, + "b x b 3 true 3", + ); +} + +fn wat_of(source: &str) -> String { + let mut items = aver::source::parse_source(source).expect("parse"); + let neutral_policy = aver::ir::NeutralAllocPolicy; + let result = aver::ir::pipeline::run( + &mut items, + aver::ir::PipelineConfig { + typecheck: Some(aver::ir::TypecheckMode::Full { base_dir: None }), + alloc_policy: Some(&neutral_policy), + run_interp_lower: false, + run_buffer_build: false, + run_chars_fusion: false, + run_list_build: false, + ..Default::default() + }, + ); + let tc = result.typecheck.as_ref().expect("typecheck requested"); + assert!(tc.errors.is_empty(), "typecheck failed: {:?}", tc.errors); + let bytes = aver::codegen::wasm_gc::compile_to_wasm_gc(&items, result.analysis.as_ref()) + .expect("wasm-gc compile"); + wasmprinter::print_bytes(&bytes).expect("print wat") +} + +/// The bodies of the functions in `wat` whose signature line contains +/// `signature`. +fn func_bodies<'w>(wat: &'w str, signature: &str) -> Vec<&'w str> { + wat.match_indices("\n (func ") + .map(|(start, _)| { + let body = &wat[start + 1..]; + &body[..body[1..] + .find("\n (func") + .map_or(body.len(), |end| end + 1)] + }) + .filter(|body| { + body.lines() + .next() + .is_some_and(|line| line.contains(signature)) + }) + .collect() +} + +/// A state record reaches `Vector.set` as a field of a record the function +/// was handed, which no ownership fact covers. The versioned `set` helper it +/// calls (the one function taking a version, an index, a `Bool` element and +/// the owned flag) must neither copy nor allocate an array. +#[test] +fn set_on_a_record_field_does_not_copy_the_array() { + let wat = wat_of( + r#" +module Main + intent = "A state record whose vector every step sets." + effects [Console.print] + +record State + cells: Vector + count: Int + +fn step(s: State, i: Int) -> State + ? "Sets cell i." + match Vector.set(s.cells, i, true) + Option.Some(updated) -> State.update(s, cells = updated, count = s.count + 1) + Option.None -> s + +verify step + step(State(cells = Vector.fromList([false]), count = 0), 0).count => 1 + +fn main() -> Unit + ! [Console.print] + Console.print("{step(State(cells = Vector.fromList([false, false]), count = 0), 1).count}") +"#, + ); + let sets = func_bodies(&wat, " i32 i32 i32) (result (ref null "); + assert_eq!(sets.len(), 1, "expected one Vector.set helper in:\n{wat}"); + assert!( + !sets[0].contains("array.copy") && !sets[0].contains("array.new"), + "Vector.set copies or allocates an array:\n{}", + sets[0] + ); + assert!( + sets[0].contains("array.set") && sets[0].contains("struct.new"), + "Vector.set writes the cell and makes a new version:\n{}", + sets[0] + ); +} + +/// A program with lists, string interpolation and `+` but no Vector value. +/// The registry still keeps a `Vector` array for every `List` and the +/// concatenation helper's `Vector`, but they stay plain arrays with +/// no version or diff struct and no `current` / `set` helper, so the module +/// is byte for byte what it was before vectors had versions. The hash is the +/// one the compiler produced before them; a change to codegen that moves it +/// for another reason updates it, one that moves it because of versions is +/// the regression this pins. +#[test] +fn a_program_without_a_vector_value_is_unchanged_by_versions() { + use sha2::{Digest, Sha256}; + let source = r#"module Main + intent = "Lists, string interpolation and concatenation, and no Vector." + effects [Console.print] + +fn total(xs: List, acc: Int) -> Int + ? "The sum of the list." + match xs + [] -> acc + [x, ..rest] -> total(rest, acc + x) + +fn words(xs: List, acc: String) -> String + ? "The words, joined." + match xs + [] -> acc + [w, ..rest] -> words(rest, acc + w + " ") + +fn main() -> Unit + ! [Console.print] + xs = [1, 2, 3] + Console.print("total {total(xs, 0)} of {List.len(xs)}: {words(["a", "b"], "")}") +"#; + let repo_root = PathBuf::from(env!("CARGO_MANIFEST_DIR")); + let path = temp_module("vector-versions-none", source); + let out_dir = path.with_extension("out"); + let out = Command::new(env!("CARGO_BIN_EXE_aver")) + .current_dir(&repo_root) + .arg("compile") + .arg(&path) + .args(["--target", "wasm-gc", "-o"]) + .arg(&out_dir) + .output() + .expect("expected `aver compile` to execute"); + assert!( + out.status.success(), + "compile failed:\n{}", + format_output(&out) + ); + let wasm = std::fs::read_dir(&out_dir) + .expect("output dir") + .filter_map(|entry| entry.ok().map(|e| e.path())) + .find(|p| p.extension().is_some_and(|ext| ext == "wasm")) + .expect("a .wasm in the output"); + let bytes = std::fs::read(&wasm).expect("read the module"); + cleanup(&path); + let _ = std::fs::remove_dir_all(&out_dir); + let digest: String = Sha256::digest(&bytes) + .iter() + .map(|b| format!("{b:02x}")) + .collect(); + assert_eq!( + (bytes.len(), digest.as_str()), + ( + 10243, + "385c20d44393408caeee6f08c21233534de16bd47f3672a22400976c4813b0b4" + ), + "a module with no Vector value changed" + ); +} diff --git a/tools/cert-baseline.json b/tools/cert-baseline.json index 3fcc1b71a..8d80112df 100644 --- a/tools/cert-baseline.json +++ b/tools/cert-baseline.json @@ -253,9 +253,7 @@ }, "tools/certkit/fixtures/cell_at.av": { "bridges": [], - "certified": { - "cellAt": "L1" - }, + "certified": {}, "laws": [] }, "tools/certkit/fixtures/cert_goals.av": {