Grow into memory instead of reserving it, and raise the ceiling

CHANGELOG.md

@@ -8,6 +8,35 @@ The public API is the exports of `swisstable` and the two `.wasm` subpaths.
 The `*WasmExports` interfaces mirror the native sources, are marked
 `@internal`, and change without a major version.
 
+## 1.1.0 - 2026-08-10
+
+### Changed
+
+- The capacity ceiling is 117,440,512 entries for `SwissU32ToU32` and
+  58,720,256 for `SwissU32ToU64`, up from 917,504. The hash is 64 bits now,
+  which gives the probe position enough bits to address the whole slot
+  space. The two modules differ because a u64 entry is wider.
+- Tables grow into their memory instead of reserving it. The banks are no
+  longer static arrays; they sit at computed offsets above the module's
+  static data, and linear memory grows to reach them. An instance starts at
+  1.25 MiB (`SwissU32ToU32`) or 1.75 MiB (`SwissU32ToU64`) rather than
+  21 MiB or 29 MiB, and grows with what it holds.
+- Each module now declares a maximum linear memory of 3.4 GiB
+  (`SwissU32ToU32`) or 2.4 GiB (`SwissU32ToU64`). This is address space, not
+  a reservation, but a memory-constrained or 32-bit host may decline it at
+  instantiation where the previous 21 MiB succeeded.
+- `SWISS_MAX_CAPACITY_LOG2` accepts `[4, 29]`, and each source keeps its own
+  default. Lowering it caps how far a table can grow and how much address
+  space the module declares. It no longer changes what an instance reserves.
+- Memory is never returned to the host. `clear()` and `shrinkToFit()` ready
+  a table for reuse, but an instance holds the high-water mark of every bank
+  it used until `dispose()`.
+
+### Fixed
+
+- A host that refuses to grow linear memory now reports a capacity error and
+  leaves the table unchanged, instead of trapping.
+
 ## 1.0.0 - 2026-08-09
 
 Initial release.

CONTRIBUTING.md

@@ -71,6 +71,8 @@ examples/       runnable examples, start at 01-basic.ts
 test/           bun test suites
 docs/           api.md, design.md, performance.md, and plans/
 dist/wasm/      build output (generated, not committed)
+                also holds two test-only modules, swiss_u32_capped and
+                swiss_u32_starved, which are never embedded or published
 ```
 
 ## The loop
@@ -81,7 +83,7 @@ bun run build      # both steps below
 bun run build:wasm # native/*.c -> dist/wasm/*.wasm + src/generated/*.ts
 bun run build:js   # src/*.ts   -> dist/js/*.js + .d.ts
 
-bun test           # 246 tests across 24 suites
+bun test           # 252 tests across 24 suites
 bun run typecheck  # tsc --noEmit
 bun run check:ubsan  # rebuild with UBSan trapping and exercise the arithmetic
 bun run smoke      # the built package under plain Node
@@ -132,17 +134,22 @@ If you add or rename an export, update the `exports` list in
 The linker does not fail on a missing `--export=` symbol — the error surfaces
 at `load()` as a `TypeError`, so tests are what catch it.
 
-`MAX_CAPACITY` is set by `MAX_CAPACITY_LOG2`, a power-of-two exponent
-defaulting to 20 via `#ifndef` and overridable with the
-`SWISS_MAX_CAPACITY_LOG2` environment variable at build time. It is accepted
-in `[4, 25]`, enforced identically in `native/swiss_core.h` and
-`scripts/build-wasm.ts`: below 4 a bank holds less than one SIMD group, and
-above 25 `h1()` has no bits left to address the extra slots with. Linear memory is
-derived from it in `scripts/build-wasm.ts` (bytes-per-slot times slots, plus
-fixed overhead), so there is no second figure to keep in step — but if you add
-a static array, raise that target's `overheadBytes`. The statics have to fit
-in the linked linear memory, and the link fails with "initial memory too
-small" if they do not, so the arithmetic is checked by every build.
+`MAX_CAPACITY` is set by `MAX_CAPACITY_LOG2`, a power-of-two exponent. Each
+source sets its own via `#ifndef` (27 for `swiss_u32.c`, 26 for
+`swiss_u64.c`), and `SWISS_MAX_CAPACITY_LOG2` overrides both at build time.
+The accepted range is `[4, 29]`, checked in `native/swiss_core.h` and
+`scripts/build-wasm.ts`. Below 4 a bank holds less than one SIMD group;
+above 29 the 32-bit product in `MAX_LIVE()` wraps.
+
+What binds first is narrower and depends on the entry width: three banks
+have to address inside wasm32. A `_Static_assert` in `swiss_core.h` checks
+that against `sizeof(Entry)`, so an exponent a module cannot address fails
+the build. That is why the two modules differ by one.
+
+If you add a static array, raise that target's `staticBytes` in
+`scripts/build-wasm.ts`. Statics have to fit below the initial memory, and
+the link fails with "initial memory too small" if they do not, so every
+build checks it.
 
 ## Changing the benchmarks
 

README.md

@@ -31,21 +31,21 @@ Docs: [API](docs/api.md) · [Design](docs/design.md) ·
 - One byte of metadata per slot, compared sixteen at a time with `wasm_simd128`.
 - 10.3 bytes per entry at full occupancy, 20.6 with the standby bank a rehash
   needs, against a measured 37 for `Map` on V8 and 67 on JavaScriptCore.
-- An instance reserves 21 MiB (u32) or 29 MiB (u64) up front and commits it
-  by page: an empty table costs 1.7 MiB RSS, each further one about 50 KiB.
-  Lower `SWISS_MAX_CAPACITY_LOG2` for many small tables — see
+- Holds up to 117,440,512 entries (u32) or 58,720,256 (u64).
+- An instance starts at 1.25 MiB (u32) or 1.75 MiB (u64) and grows with the
+  table, so a small table costs what a small table costs. See
   [Footprint](docs/design.md#footprint). `dispose()`, or a `using`
   declaration, hands an instance back without waiting for the collector.
 - Bulk `setMany`/`getMany`/`deleteMany` on both tables cross once per batch:
-  a 100,000-entry u32 fill is 6.2–6.7 ns/op against `Map`'s 44–65, and the
-  matching lookup 4.1–4.7 against 10–22.
+  a 100,000-entry u32 fill is 6.1–6.9 ns/op against `Map`'s 48–70, and the
+  matching lookup 3.9–4.9 against 11–24.
 - `getOrInsert` and `increment` do a read-modify-write in one crossing and
-  one probe: counting is 1.25–1.8x faster than `get` plus `set`.
-- No allocator — fixed linear memory, linked `-nostdlib`, never calls
-  `memory.grow`, nothing allocated on a hot path.
+  one probe. Inserting a missing key is 5.0–6.3x faster than `Map`.
+- No allocator. Linked `-nostdlib`, with the banks at computed offsets and
+  linear memory grown to reach them. Nothing allocates on a hot path.
 - Ships compiled ESM with type declarations, and the modules are compiled in:
   no `.wasm` to serve, no loader to write, no install step, no dependencies.
-- Runs in Node, Bun, Deno, bundlers, and browsers — anything with WebAssembly
+- Runs in Node, Bun, Deno, bundlers, and browsers. Anything with WebAssembly
   SIMD (Node 16.9+, Chrome 91+, Firefox 89+, Safari 16.4+), which
   `supportsSimd()` reports for the current runtime.
 
@@ -83,11 +83,11 @@ Four exports:
 | `StringInterner` | `string -> u32` | stable IDs in first-seen order |
 | `InternedSwissMap` | `string -> V` | string keys over a numeric table |
 
-Keys and values are strictly `u32` and anything else throws `RangeError`;
-capacity is fixed at build time (917,504 entries); a stored `0` is always
-distinguishable from an absent key. See [docs/api.md](docs/api.md) for every
-method and thrown error, and [`examples/`](examples/README.md) for five
-runnable programs.
+Keys and values are strictly `u32` and anything else throws `RangeError`.
+Capacity is bounded at 117,440,512 entries (u32) or 58,720,256 (u64). A
+stored `0` is always distinguishable from an absent key. See
+[docs/api.md](docs/api.md) for every method and thrown error, and
+[`examples/`](examples/README.md) for five runnable programs.
 
 Two operational notes. Each table seeds its hash from the runtime's CSPRNG,
 so a colliding key set cannot be computed offline and reused across
@@ -117,18 +117,18 @@ its own, probed in a shuffled order. i9-13900K on x64 Linux.
 
 | Workload | Bun 1.3 | Node 24 | Deno 2.9 | Chrome 151 | Firefox 153 |
 | --- | --- | --- | --- | --- | --- |
-| fill (pre-sized) | 8.3x | 6.4x | 5.6x | 4.2x | 5.2x |
-| lookup hit | 1.60x | 2.8x | 3.2x | 2.6x | 1.79x |
-| lookup miss | 1.38x | 3.3x | 3.4x | 2.6x | 1.73x |
-| `has` | 1.84x | 3.4x | 3.6x | 3.1x | 1.98x |
-| overwrite existing key | 2.3x | 2.7x | 2.9x | 2.7x | 3.2x |
-| delete | 5.6x | 5.5x | 5.4x | 4.3x | 4.3x |
-| churn (delete + reinsert) | 3.2x | 3.6x | 3.5x | 2.9x | 3.2x |
-| count (`increment`) | 1.41x | 1.39x | 1.47x | 1.49x | 0.96x |
-| u32 bulk fill (`setMany`) | 9.8x | 9.6x | 8.0x | 6.6x | 8.6x |
-| u32 bulk lookup (`getMany`) | 2.4x | 5.4x | 4.8x | 5.0x | 3.8x |
-| u64 bulk fill (`setMany`) | 6.0x | 8.0x | 6.9x | 6.2x | 9.1x |
-| u64 bulk lookup (`getMany`) | 1.56x | 3.8x | 3.7x | 3.2x | 2.3x |
+| fill (pre-sized) | 5.2x | 6.2x | 5.0x | 3.6x | 4.7x |
+| lookup hit | 1.50x | 2.7x | 2.8x | 2.3x | 1.60x |
+| lookup miss | 1.29x | 3.2x | 3.4x | 2.5x | 1.70x |
+| `has` | 1.77x | 3.2x | 3.5x | 2.9x | 1.86x |
+| overwrite existing key | 2.2x | 2.5x | 2.5x | 2.2x | 2.8x |
+| delete | 6.1x | 5.5x | 5.8x | 4.1x | 4.7x |
+| churn (delete + reinsert) | 3.3x | 3.6x | 3.5x | 2.6x | 3.2x |
+| count (`increment`) | 1.26x | 1.33x | 1.35x | 1.28x | 0.89x |
+| u32 bulk fill (`setMany`) | 8.4x | 11x | 8.3x | 7.6x | 9.2x |
+| u32 bulk lookup (`getMany`) | 2.7x | 5.6x | 4.9x | 5.4x | 3.6x |
+| u64 bulk fill (`setMany`) | 10x | 9.7x | 8.6x | 6.4x | 8.5x |
+| u64 bulk lookup (`getMany`) | 2.1x | 4.1x | 3.8x | 3.6x | 2.3x |
 
 The table costs about the same on every engine. The columns differ because
 `Map` does. Counting is the one row a browser engine takes: at 1,000 distinct

SECURITY.md

@@ -22,9 +22,10 @@ memory and nothing else, and cannot reach past it — a bug in the C is
 confined to the instance's memory by the WebAssembly sandbox, which is the
 main reason the table is compiled rather than written in JavaScript.
 
-The modules are built `-nostdlib` with no allocator and never call
-`memory.grow`, so there is no allocation path to exhaust and no `malloc` to
-corrupt.
+The modules are built `-nostdlib` with no allocator, so there is no `malloc`
+to corrupt. Linear memory grows only when a rehash reaches for its next
+bank. A host that refuses the growth gets a capacity error, and the table is
+left exactly as it was.
 
 ## In scope
 
@@ -64,14 +65,25 @@ plain memory with no atomics, and sharing an instance across workers
 corrupts it. Share the compiled `WebAssembly.Module` instead and give each
 worker its own instance.
 
-**Capacity is fixed at build time.** Exceeding it throws `RangeError`
-rather than growing. An input that drives a table past 917,504 entries is a
-capacity-planning problem in the calling service.
-
-**Every instance reserves its full linear memory up front** — 21 MiB for
-`SwissU32ToU32`, 29 MiB for `SwissU32ToU64` — committed by page, so an empty
-one costs about 1.7 MiB RSS. Code that creates a table per request and never
-calls `dispose()` will accumulate them until the collector runs.
+**Capacity is bounded** at 117,440,512 entries for `SwissU32ToU32` and
+58,720,256 for `SwissU32ToU64`. Exceeding it throws `RangeError` rather than
+growing. If untrusted input decides how many entries a table holds, that is
+a capacity-planning problem in the calling service. The ceiling is high
+enough that such input reaches host memory first.
+
+**An instance never hands memory back.** It starts at 1.25 MiB
+(`SwissU32ToU32`) or 1.75 MiB (`SwissU32ToU64`) and grows with the table.
+Neither `clear()` nor `shrinkToFit()` returns pages, so an instance holds
+the high-water mark of every bank it used. Code that creates a table per
+request and never calls `dispose()` accumulates them until the collector
+runs.
+
+**Each module declares a large maximum memory**, 3.4 GiB for
+`SwissU32ToU32` and 2.4 GiB for `SwissU32ToU64`. This is address space, not
+a reservation, and no page is committed until a table reaches it. A
+memory-constrained or 32-bit host may still decline it at instantiation,
+which surfaces as a construction failure. Build with a lower
+`SWISS_MAX_CAPACITY_LOG2` for those hosts.
 
 ## Supported versions
 

benches/bench.ts

@@ -1424,11 +1424,10 @@ async function shrinkScenario(peak: number, remaining: number): Promise<void> {
 /**
  * Bytes of linear memory each slot costs.
  *
- * Two banks are reserved so a rehash has somewhere to move entries to, and
- * both are static arrays sized to the compiled ceiling — so a slot costs its
- * per-bank layout twice, whether or not a rehash is in progress. Kept in
- * step with `bankBytesPerSlot` in scripts/build-wasm.ts, which is what the
- * modules are actually linked against.
+ * Each module addresses two banks so a rehash has somewhere to move entries
+ * to, and the address extent a table holds covers both — so a slot costs its
+ * per-bank layout twice. Kept in step with `bankBytesPerSlot` in
+ * scripts/build-wasm.ts, which is the per-bank half of the same number.
  */
 const U32_BYTES_PER_SLOT = 2 * (1 + 8);
 const U64_BYTES_PER_SLOT = 2 * (1 + 12);
@@ -1681,9 +1680,9 @@ async function memoryScenario(keys: Uint32Array): Promise<void> {
       `in the live bank alone`,
   );
   console.log(
-    `  linear memory reserved per instance: ` +
-      `${(reserved.u32 / (1024 * 1024)).toFixed(0)} MiB (u32), ` +
-      `${(reserved.u64 / (1024 * 1024)).toFixed(0)} MiB (u64)`,
+    `  linear memory an instance starts at: ` +
+      `${(reserved.u32 / (1024 * 1024)).toFixed(2)} MiB (u32), ` +
+      `${(reserved.u64 / (1024 * 1024)).toFixed(2)} MiB (u64)`,
   );
 }
 

benches/compare.ts

@@ -544,15 +544,15 @@ function memorySummary(): string | null {
         `The tables are not on the heap and are not measured the same way. A ` +
         `slot costs ${report.slotBytes.u32} B (u32) or ` +
         `${report.slotBytes.u64} B (u64) of linear memory, both banks ` +
-        `counted — at the 7/8 load ceiling that is ` +
+        `counted. At the 7/8 load factor that is ` +
         `${perEntry(report.slotBytes.u32)} B/entry and ` +
         `${perEntry(report.slotBytes.u64)} B/entry, of which ` +
         `${perEntry(report.slotBytes.u32 / 2)} B and ` +
         `${perEntry(report.slotBytes.u64 / 2)} B are the live bank the ` +
-        `entries are actually in. An instance reserves ` +
-        `${(report.reservedBytes.u32 / (1024 * 1024)).toFixed(0)} MiB (u32) or ` +
-        `${(report.reservedBytes.u64 / (1024 * 1024)).toFixed(0)} MiB (u64) of ` +
-        `address space up front, committed page by page as it is touched.`
+        `entries are actually in. An instance starts at ` +
+        `${(report.reservedBytes.u32 / (1024 * 1024)).toFixed(2)} MiB (u32) or ` +
+        `${(report.reservedBytes.u64 / (1024 * 1024)).toFixed(2)} MiB (u64) ` +
+        `and grows linear memory as the table reaches each new bank.`
       );
     }
   }