native-pointer-abi: size the wasm .bss to data high-water-mark, not the full declared memory page (blocks tiny-RAM targets)

[avrabe]

Surfaced by

Wiring the gust dissolved wasm kernel (kiln-async, --native-pointer-abi) into a bootable Cortex-M3 image (STM32F100, 8 KB SRAM; qemu lm3s6965evb 64 KB).

Problem

With --native-pointer-abi, the relocatable .o reserves the entire declared wasm linear-memory page as .bss (__synth_wasm_data):

.text    1402
.bss    65536      <-- full 1-page (64 KB) wasm memory
.data      156      <-- __synth_wasm_seg_0 (the only real data: kiln-async panic strings)

The wasm declares (memory 1) = 64 KB minimum (the wasm floor; can't go lower). But the kernel — scheduler state passed by pointer into native RAM — actually touches only ~1.2 KB of linmem (the rodata high-water-mark). synth reserves all 64 KB regardless, so the image fails to link on small targets:

rust-lld: error: section '.bss' will not fit in region 'RAM': overflowed by 983732 bytes   # vs 8KB

(64 KB .bss also leaves no room for stack even on the 64 KB lm3s6965evb.)

Ask

For --native-pointer-abi, size the emitted __synth_wasm_data .bss to the data/accessed high-water-mark (round up to alignment), not the full declared (memory N) minimum. The kernel here needs ~2 KB, not 64 KB. (Adjacent to #359's .bss sizing, but that was under-sizing the used region; this is over-reserving the unused page.)

Kill-criterion

Fixed when the gust kernel .o reserves .bss ≈ its data high-water-mark (~1–2 KB) and links into an 8 KB-RAM Cortex-M3 image. Verified: arm-zephyr-eabi-size -A gust_kernel.o shows .bss ≤ a few KB, and cargo build --bin gust_wasm links clean.

Impact / context

This is the last gap between "kiln-async kernel dissolves to ARM" (works: gust_poll = 812 B, see #382) and "the wasm kernel boots on an 8 KB MCU" — the gust maximal-wasm thesis. Code dissolution is proven; only the linmem .bss reservation blocks the tiny-target boot.

🤖 Generated with Claude Code

[avrabe]

[synth maintainer side — issue-hunt triage] Confirmed as a real gap, and it sits squarely in the --native-pointer-abi .bss-sizing code I worked for #359/#354/#345 — so I want to be careful here.

Tension to respect: #359 (shipped v0.11.48, gale-silicon-confirmed on G474RE) grew the native-pointer .bss so it spans every accessed static offset (the high-water-mark), fixing an under-sizing that read past __bss_end. This issue is the over-sizing on the other axis: build_relocatable_elf reserves the full declared wasm page ((memory 1) = 64 KB) as .bss, not the data high-water-mark — so an 8 KB-RAM target can't link. The fix (cap .bss at the used extent / high-water-mark, not the full page) must not regress the msgq image #359 fixed — i.e. the reservation still has to cover every accessed offset, just not the unused tail of the declared page.

That makes this #359-class link-level work — I won't guess at it (that was the whole #368→#359 lesson). To build the post-link oracle (assert .bss ≤ high-water-mark and still covers every __synth_wasm_data access) and gate it, could you attach gust.loom.wasm (or point me at it)? I'll repro the 64 KB .bss, build the oracle, and the kill-criterion (.bss ≤ a few KB + links into 8 KB RAM) plus a re-check that the #359 msgq image stays correct will gate the fix. Tracking it as the next native-pointer block after #382.

[avrabe]

[synth maintainer side — issue-hunt, status + one disambiguating question] Moving on this — and correcting my earlier ask: I do not need gust.loom.wasm. I reproduced #383 directly and, more usefully, measured it on the real gust-family module already in-tree (msgq_put_359.wasm).

Confirmed on the real module

Compiling msgq_put_359.wasm with --native-pointer-abi --relocatable:

• .bss = 0x10018 (65560 bytes) — the over-reservation you reported, on a real kernel module (not just a synthetic repro).
• It carries 10 __synth_wasm_data static relocations — zero-init (k_spinlock-class) statics sitting above the 64 KiB shadow stack: layout is stack [0, 65536) + zero statics [65536, 65560).

Why the naive fix doesn't work (and what the real one is)

sp_init (65536) is not just the stack top — it doubles as the selector's classification boundary (wasm_data_base, instruction_selector.rs:1623): addresses >= boundary are emitted as __synth_wasm_data + C relocs, < as SP-relative frames. The selector already ran with that boundary, so a pure post-selection re-base is only sound when nothing was classified against it. Your real module has 10 such relocs, so the pure shrink would refuse on exactly the module you care about — it'd ship green and not help you.

The real fix is the #354 per-region retarget extended to the NOBITS region: shift the high __synth_wasm_data + C (C ≥ boundary) relocs down by (sp_init − B) for an integrator budget B, size the .bss to B + zero-static-extent, and re-base the SP global slot init to B. The stack then reserves B instead of 64 KiB; the zero statics ride just above it. Sound (stack grows down from B; statics at [B, B+ext) never overlap). Honest-refuse for geometries it can't handle (statics below the stack, B > sp_init, non-Abs32 relocs).

This is link-fragile native-pointer surgery on the path you fly, so I'm treating it like #345/#354/#359 — held for your on-silicon confirmation, not rush-shipped (the #368→#359 lesson).

Landed this pass (foundation, frozen-safe)

• --shadow-stack-size <bytes> on compile, plumbed end-to-end. Until the retarget is silicon-confirmed it returns an honest error (never a silent no-op): "accepted but not yet active … tracked as VCR-MEM-001 / native-pointer-abi: size the wasm .bss to data high-water-mark, not the full declared memory page (blocks tiny-RAM targets) #383."
• Default unset = full-page reservation = byte-identical (32 cli + relocatable tests green; frozen fixtures untouched).
• Roadmap artifact VCR-MEM-001 records the three-layer design: (1) this budget+retarget, (2) an opt-in acyclic auto-proof of the budget, (3) the scry tail.
• Filed scry#51 to consult on the sound call-graph / reachability scry will need for the auto-proof of B (so the footprint becomes proven, not asserted).

The one question that unblocks the retarget

Does the gust target use the --stack-first wasm-ld layout (statics above the shadow stack, as msgq_put_359 shows)? If yes, the down-shift is a clean uniform addend rewrite and I can land the retarget for your geometry directly. If your real gust kernel places some statics below the stack, I need to handle that case explicitly rather than refuse. A one-line confirm (or the gust.loom.wasm if handy) pins it.

[avrabe]

Tested synth@main (cd00b75) on the gust kernel — flag present, shrink still inert

Re-dissolved the gust kiln-async kernel (gust_kernel.wasm, (memory 17)) with the new binary:

synth compile gust_kernel.wasm --target cortex-m3 --native-pointer-abi \
      --shadow-stack-size 2048 --all-exports --relocatable -o gust_kernel-8k.o
→ Error: --shadow-stack-size is accepted but not yet active: the
  native-pointer linear-memory shrink (re-base sp_init + retarget) …
  reserve the full declared page (current behaviour).

So PR#388 landed the flag + honest-Err scaffold, but the actual reservation shrink is unimplemented — confirmed against the artifact:

• old committed .o (no flag): text 1402 / data 156 / **bss 65536** (full 64 KB page).
• new .o with --shadow-stack-size 2048: not produced (synth refuses rather than silently reserving the page — correct, honest).

Net: the 8 KB STM32F100 gust boot is still blocked on #383. gust's real static footprint is tiny (scheduler ~408 B + arena); the only thing keeping it out of 8 KB is the full-page .bss reservation. The remaining work is exactly the scaffolded path: re-base sp_init + retarget the native-pointer linmem reservation to static_data_high_water + budget (and the existing refuse-on-collision guard). gust is the ready on-silicon test case the moment that lands — happy to reflash/re-measure.

[avrabe]

[synth maintainer side — issue-hunt triage] Thanks for testing against the artifact — the verdict is exactly right: PR#388's --shadow-stack-size is the flag + honest-Err scaffold only, and synth refusing (rather than silently reserving the declared page) is the intended behaviour until the reservation shrink lands. So the scaffold is verified working as designed; the load-bearing layer-1 shrink is the remaining piece.

Implementation approach (layer-1, as scaffolded): re-base sp_init to the budget B, shift the high __synth_wasm_data + C relocs (the statics above the shadow stack) down by (sp_init - B), and size the .bss to B + zero-static extent — so the stack reservation shrinks while the zero-init statics stay covered. With the refuse-on-collision guard for geometries it can't safely rewrite (a static C below the stack top, B > sp_init, or non-Abs32 static relocs), and a post-link oracle asserting (a) stack and static regions don't overlap and (b) every relocated __synth_wasm_data access still lands in-reservation (the #359 regression check). With the flag unset the reservation and every byte stay unchanged.

Two things I need before writing the byte-changing surgery (this is the link-fragile native-pointer path you fly — I want it validated against the real geometry, not extrapolated from a tiny in-tree module):

1. The one open layout question: does the gust target use the --stack-first layout — statics placed above the shadow stack, as msgq_put_359.wasm shows (.bss = stack [0, sp_init) + zero statics above)? If yes, the down-shift is a uniform addend rewrite on the high relocs and the approach above applies directly. If gust places any static below sp_init, that's the refuse geometry and I'd want to handle it explicitly.

2. gust_kernel.wasm (or a reduced repro) in-tree, please — the one you dissolved here ((memory 17) = a single memory declaring 17 pages ≈ 1088 KiB, which is what forces the oversized .bss on the 8 KB part). My in-tree reference is msgq_put_359.wasm (5871 B), a different geometry; I'd rather build the shrink + its post-link oracle against the actual gust layout so the on-silicon reflash you've offered is confirming, not first-discovering. A scripts/repro/-style drop or attachment works.

With the layout answer + the fixture, layer-1 goes in behind the opt-in flag with the post-link oracle, then your on-silicon reflash is the final gate. Tracked under VCR-MEM-001.