Skip to content

worker-build --emscripten - #1061

Open
guybedford wants to merge 26 commits into
mainfrom
gbedford/worker-build-emscripten
Open

worker-build --emscripten#1061
guybedford wants to merge 26 commits into
mainfrom
gbedford/worker-build-emscripten

Conversation

@guybedford

@guybedford guybedford commented Sep 11, 2026

Copy link
Copy Markdown
Collaborator

This adds worker-build --emscripten, building Workers for wasm32-unknown-emscripten so crates that need a libc, epoll sockets or blocking calls (stock Tokio net, etc.) run unmodified, parking through JSPI.

  • worker-build installs the pinned emsdk release (6.0.9) into its cache directory and applies the patches under worker-build/patches/emscripten/ to the frontend: the marker-based -sWASM_BINDGEN backport (Unify wasm-bindgen output under -sWASM_BINDGEN with marker-based detection emscripten-core/emscripten#27208) and NODERAWSOCKETS DNS (Real DNS resolution for getaddrinfo under -sNODERAWSOCKETS emscripten-core/emscripten#27693). Each is dropped as the pin moves past it. EMSCRIPTEN / EMSDK env vars override for local toolchain work.
  • rustc drives emcc as the linker for a bin target (cdylib cannot link on emscripten); emcc runs wasm-bindgen post-link, and the output is collected into the standard build/index.js + wasm shape, so wrangler config only changes the build command (plus the new_module_registry compat flag for import.meta.url). Codegen and link flags go through RUSTFLAGS, as --panic-unwind does.
  • worker::init becomes a private start function (its public init export collided with Emscripten's own) and no longer registers the abort reinit hook on emscripten, where Emscripten owns instantiation.
  • Rust-side dependency patches (tokio, mio, libc, wasm-streams) stay user-side; the emscripten-tcp example documents the [patch.crates-io] block.

--release builds work with the released wasm-bindgen CLI. Debuginfo builds need a CLI with wasm-bindgen/wasm-bindgen#5328 (DWARF that survives the exnref translation) until 0.2.129; the submodule is updated to include it along with wasm-bindgen/wasm-bindgen#5332.

Tested: the emscripten-tcp example (worker types + tokio::net::TcpStream under JSPI) builds with --dev and --release and serves /?host=example.com under wrangler dev; CI adds an Emscripten example job running both builds and that smoke test. Unit tests cover the patch applier.

guybedford added a commit to guybedford/rust-workers-minecraft that referenced this pull request Sep 16, 2026
@guybedford
Build with the worker crate and worker-build --emscripten --tokio
490f181 
The Worker now uses workers-rs (cloudflare/workers-rs#1061): the `worker`
crate for the fetch handler, Env and Durable Object plumbing, and
`worker-build --emscripten --tokio` to link the bin through emcc, wrap the
exports into the entrypoint and DurableObject-derived class, and emit the
module Wrangler serves. The JS shim, the factory link arguments in
.cargo/config.toml and the wasm-bindgen CLI plumbing in the scripts go away;
build.rs keeps only the application's own settings.

The Durable Object no longer owns a runtime. `connect` is a
`#[wasm_bindgen(tokio)]` export (wasm-bindgen/wasm-bindgen#5334) whose
future runs on the thread's Tokio `LocalEventLoop` (tokio-rs/tokio#8484),
the host event loop driving the scheduler. The server lifetime is spawned
as a task so a panic arrives as a JoinError and fails the object.

Pins move to the event-loop stack: tokio `emscripten-event-loop-host`, mio
`emscripten`, workers-rs `gbedford/worker-build-emscripten` with its
wasm-bindgen submodule supplying every wasm-bindgen crate, and emscripten
`cf-final` with the emsdk release its main is paired with. Binaryen comes
from that emsdk, so the separate Binaryen checkout and build are dropped.
@guybedford
worker-build --emscripten
b7682dc 
Build Workers for wasm32-unknown-emscripten through a worker-build
provisioned toolchain: the pinned emsdk release is installed into the
cache directory and the frontend patched with backports the Rust link
needs (marker-based -sWASM_BINDGEN, NODERAWSOCKETS DNS). rustc drives
emcc as the linker for a bin target, emcc runs wasm-bindgen post-link,
and the output is collected into the standard build/index.js shape.

The worker crate's start function becomes private, since its public
init export collided with Emscripten's own.
@guybedford
Keep DWARF in emscripten builds
01004fd 
The exnref translation failure on debug builds was wasm-bindgen 0.2.128's
--keep-debug output (walrus 0.27.1 .debug_loc), fixed on main by #5328,
not binaryen on the linked module. The CLI floor already covers it.
@guybedford
Skip reinit on emscripten and support minified emcc output
230d6c9 
The worker crate no longer registers the abort reinit hook on
wasm32-unknown-emscripten, so the released wasm-bindgen CLI builds
--emscripten --release; only debuginfo builds still need #5328. Export
and wasm import detection now handle emcc's minified release output.
@guybedford
worker-build: leave every cloudflare: module external when bundling
45600c7 
The runtime serves the whole cloudflare: namespace (cloudflare:node,
cloudflare:test, ...), not only the three that were listed, and a
wasm-bindgen module import of one of them must reach the output as an
import rather than fail to resolve in esbuild.
@guybedford
Reentrant JSPI toolchain
9ce6451 
Backport the JSPI lifecycle hooks, REENTRANT_JSPI fiber stacks and epoll
listener API to the 6.0.9 frontend, and take Binaryen from a release
carrying the jspi-hooks pass (WebAssembly/binaryen#9102). The release
sysroot stamp is dropped after patching so emcc installs the new
headers.
@guybedford
emscripten-tcp: note reentrant activations
c49dff8
@guybedford
Use wasm-bindgen JSPI lifecycle hooks integration
3d9eaec 
wasm-bindgen/wasm-bindgen#5333 instruments jspi exports and suspending
imports with emscripten's fiber hooks, so under -sREENTRANT_JSPI each
fetch activation runs on its own stack. The example takes wasm-bindgen
from the submodule until it is released.
@guybedford
ci: build the emscripten example with the submodule wasm-bindgen
d1b6a43
@guybedford
ci: accept any HTTP/1.x status line in the emscripten smoke test
c9aafb3
@guybedford
Fiber-owned Tokio context under REENTRANT_JSPI
e2d6724 
Pass --cfg tokio_jspi_hooks so Tokio's emscripten port registers the JSPI
lifecycle hooks, and move the example to that Tokio branch. Concurrent
fetch activations each drive their own runtime.
@guybedford
worker-build --emscripten: derive exported classes from DurableObject
1235d78 
The runtime exposes RPC on a Durable Object class only when it derives
from cloudflare:workers' DurableObject; the wasm-bindgen classes are
plain, so splice its prototype in.
@guybedford
JSPI exports for event and Durable Object handlers on emscripten
7c7cd9e 
On wasm32-unknown-emscripten the handler wrappers are #[wasm_bindgen(jspi)]
exports suspending on the handler future, so each activation is its own
fiber. The wrappers are left for rustc to expand: expanding both cfg
variants in the proc macro deduplicated their export descriptors.
@guybedford
worker-build --tokio: event-loop and JSPI Tokio integration
1df4e6c 
`--emscripten --tokio` schedules each handler on a Tokio event-loop
runtime per invocation (tokio-rs/tokio#8479), whose wait is the host
event loop, so plain async handlers use tokio::net and tokio::time with
no stack switching. `--emscripten --tokio=jspi` is the previous
behaviour: JSPI exports on their own fibers blocking on a runtime. The
mode reaches the macros as cfg(worker_tokio = ...).

The epoll listener backport now delivers readiness through a macrotask:
workerd drains microtasks synchronously inside builtin module loads, so
a microtask delivery ran one runtime's drive under another's connect().
@guybedford
Update Cargo.lock for wasm-bindgen-futures tokio feature
d03fde0
@guybedford
Regenerate emscripten patches from the current upstream PR heads
cb5eac6 
reentrant-jspi.patch now carries #27698, #27699 and #27547 at their heads
(listener keepalive holds, teardown wakes, macrotask delivery) and
noderawsockets-dns.patch carries #27693 at its head (node:dns only, no
virtual /etc/hosts), both as exact diffs against the 6.0.9 tree.
@guybedford
Regenerate emscripten patches from the rebased upstream stack
99800f3 
epoll-listeners.patch carries #27547 at 34e2cc057 and #27720 (timeout
keepalive release), noderawsockets-dns.patch carries #27693 as rebased
onto that, and the JSPI hooks / REENTRANT_JSPI backport (#27698, #27699)
moves to a trailing jspi-hooks.patch that only --tokio=jspi depends on.
@guybedford
Track epoll listener PR heads: MINIMAL_RUNTIME guard around maybeExit
0916e34
@guybedford
Record the rebaselined epoll listener PR heads
eef9c63
@guybedford
Track the epoll listener PR heads: native_sigs.py ordering
04d2abd
@guybedford
Drop the JSPI Tokio mode: --tokio is the host-driven event loop
9c694bb 
The event-loop integration never needed stack switching, so --tokio=jspi,
its lifecycle-hooks backport (jspi-hooks.patch), the -sJSPI link, the
Binaryen release carrying the jspi-hooks pass, the tokio_jspi_hooks cfg
and the JSPI export variant in worker-macros go. --tokio is a plain flag
and the emsdk release supplies the whole backend.

The noderawsockets-dns patch stays: without stack switching getaddrinfo
returns EAI_AGAIN for hostnames, a clean io::Error instead of the host's
uncaught "Invalid IP address" from net.connect.
@guybedford
Move --tokio onto tokio#8484 host-driven event loops
fccf113 
tokio-rs/tokio#8484 replaces EventLoopRuntime with EventLoop /
LocalEventLoop; the example takes tokio from its branch and mio from its
emscripten branch (tokio-rs/mio#1969). The wasm-bindgen submodule moves to
the rebuilt emscripten stack: main plus the reinit fix (#5332), legalized
export names and the tokio attribute (#5334) adapted to LocalEventLoop,
without the emscripten JSPI support. worker's schedule_isolated use is
unchanged.
@guybedford
Restore the wasm-bindgen path patches for the submodule stack
de73cc0 
0.8.6 dropped them to release on stable wasm-bindgen; worker's tokio
feature on wasm-bindgen-futures and #[wasm_bindgen(tokio)] come from the
submodule until they ship.
@guybedford
Export the connect handler through async_export_mod
35fde18 
Brings #[event(connect)] (#1041) in line with the other handlers so it
runs on the Tokio event loop under --tokio.
@guybedford
guybedford force-pushed the gbedford/worker-build-emscripten branch from 0873a5d to 35fde18 Compare September 16, 2026 19:40
guybedford added a commit to guybedford/rust-workers-minecraft that referenced this pull request Sep 16, 2026
@guybedford
Build with the worker crate and worker-build --emscripten --tokio
dac5a64 
The Worker now uses workers-rs (cloudflare/workers-rs#1061): the `worker`
crate for the fetch handler, Env and Durable Object plumbing, and
`worker-build --emscripten --tokio` to link the bin through emcc, wrap the
exports into the entrypoint and DurableObject-derived class, and emit the
module Wrangler serves. The JS shim, the factory link arguments in
.cargo/config.toml and the wasm-bindgen CLI plumbing in the scripts go away;
build.rs keeps only the application's own settings.

The Durable Object no longer owns a runtime. `connect` is a
`#[wasm_bindgen(tokio)]` export (wasm-bindgen/wasm-bindgen#5334) whose
future runs on the thread's Tokio `LocalEventLoop` (tokio-rs/tokio#8484),
the host event loop driving the scheduler. The server lifetime is spawned
as a task so a panic arrives as a JoinError and fails the object.

Pins move to the event-loop stack: tokio `emscripten-event-loop-host`, mio
`emscripten`, workers-rs `gbedford/worker-build-emscripten` with its
wasm-bindgen submodule supplying every wasm-bindgen crate, and emscripten
`cf-final` with the emsdk release its main is paired with. Binaryen comes
from that emsdk, so the separate Binaryen checkout and build are dropped.
guybedford added a commit to guybedford/rust-workers-minecraft that referenced this pull request Sep 16, 2026
@guybedford
Build with the worker crate and worker-build --emscripten --tokio
0ba7da1 
The Worker now uses workers-rs (cloudflare/workers-rs#1061): the `worker`
crate for the fetch handler, Env and Durable Object plumbing, and
`worker-build --emscripten --tokio` to link the bin through emcc, wrap the
exports into the entrypoint and DurableObject-derived class, and emit the
module Wrangler serves. The JS shim, the factory link arguments in
.cargo/config.toml and the wasm-bindgen CLI plumbing in the scripts go away;
build.rs keeps only the application's own settings.

The Durable Object no longer owns a runtime. `connect` is a
`#[wasm_bindgen(tokio)]` export (wasm-bindgen/wasm-bindgen#5334) whose
future runs on the thread's Tokio `LocalEventLoop` (tokio-rs/tokio#8484),
the host event loop driving the scheduler. The server lifetime is spawned
as a task so a panic arrives as a JoinError and fails the object.

Pins move to the event-loop stack: tokio `emscripten-event-loop-host`, mio
`emscripten`, workers-rs `gbedford/worker-build-emscripten` with its
wasm-bindgen submodule supplying every wasm-bindgen crate, and emscripten
`cf-final` with the emsdk release its main is paired with. Binaryen comes
from that emsdk, so the separate Binaryen checkout and build are dropped.
@guybedford
Track the simplified epoll listeners and tokio#8484 runtime keepalive
ce55db3 
emscripten#27547 now delivers through unref'd handles with no listener
keepalive of its own, and #27720 / #27693 are regenerated from their
merged main commits. The example's tokio moves to 5ef5ab3d, where the
event loop holds the runtime keepalive itself.
guybedford added a commit to guybedford/rust-workers-minecraft that referenced this pull request Sep 17, 2026
@guybedford
Build with the worker crate and worker-build --emscripten --tokio
1d5812e 
The Worker now uses workers-rs (cloudflare/workers-rs#1061): the `worker`
crate for the fetch handler, Env and Durable Object plumbing, and
`worker-build --emscripten --tokio` to link the bin through emcc, wrap the
exports into the entrypoint and DurableObject-derived class, and emit the
module Wrangler serves. The JS shim, the factory link arguments in
.cargo/config.toml and the wasm-bindgen CLI plumbing in the scripts go away;
build.rs keeps only the application's own settings.

The Durable Object no longer owns a runtime. `connect` is a
`#[wasm_bindgen(tokio)]` export (wasm-bindgen/wasm-bindgen#5334) whose
future runs on the thread's Tokio `LocalEventLoop` (tokio-rs/tokio#8484),
the host event loop driving the scheduler. The server lifetime is spawned
as a task so a panic arrives as a JoinError and fails the object.

Pins move to the event-loop stack: tokio `emscripten-event-loop-host`, mio
`emscripten`, workers-rs `gbedford/worker-build-emscripten` with its
wasm-bindgen submodule supplying every wasm-bindgen crate, and emscripten
`cf-final` with the emsdk release its main is paired with. Binaryen comes
from that emsdk, so the separate Binaryen checkout and build are dropped.
@guybedford
Move onto tokio#8484 hosted event loops
3ffd51b 
The example's tokio moves to 067f92b6, where the emscripten event loop
is built hosted: it schedules its own drives and holds the runtime
keepalive while it has tasks. The wasm-bindgen submodule follows with
the tokio attribute built on it; worker's use of schedule_isolated is
unchanged.
guybedford added a commit to guybedford/rust-workers-minecraft that referenced this pull request Sep 18, 2026
@guybedford
Build with the worker crate and worker-build --emscripten --tokio
ef6fdd3 
The Worker now uses workers-rs (cloudflare/workers-rs#1061): the `worker`
crate for the fetch handler, Env and Durable Object plumbing, and
`worker-build --emscripten --tokio` to link the bin through emcc, wrap the
exports into the entrypoint and DurableObject-derived class, and emit the
module Wrangler serves. The JS shim, the factory link arguments in
.cargo/config.toml and the wasm-bindgen CLI plumbing in the scripts go away;
build.rs keeps only the application's own settings.

The Durable Object no longer owns a runtime. `connect` is a
`#[wasm_bindgen(tokio)]` export (wasm-bindgen/wasm-bindgen#5334) whose
future runs on the thread's Tokio `LocalEventLoop` (tokio-rs/tokio#8484),
the host event loop driving the scheduler. The server lifetime is spawned
as a task so a panic arrives as a JoinError and fails the object.

Pins move to the event-loop stack: tokio `emscripten-event-loop-host`, mio
`emscripten`, workers-rs `gbedford/worker-build-emscripten` with its
wasm-bindgen submodule supplying every wasm-bindgen crate, and emscripten
`cf-final` with the emsdk release its main is paired with. Binaryen comes
from that emsdk, so the separate Binaryen checkout and build are dropped.
@guybedford
Track the emscripten socket stack heads
f4546f5 
Regenerates the 6.0.9 backports from the current pull request heads:
the epoll listeners are unchanged in content, the DNS patch gains
emscripten_dns_lookup_async / emscripten_dns_lookup_result (#27742),
and a new accept-blocking.patch carries the part of #27724 that reaches
the single-threaded Node build, recv surfacing a pending socket error
ahead of EOF.
guybedford added a commit to guybedford/rust-workers-minecraft that referenced this pull request Sep 18, 2026
@guybedford
Build with the worker crate and worker-build --emscripten --tokio
6600809 
The Worker now uses workers-rs (cloudflare/workers-rs#1061): the `worker`
crate for the fetch handler, Env and Durable Object plumbing, and
`worker-build --emscripten --tokio` to link the bin through emcc, wrap the
exports into the entrypoint and DurableObject-derived class, and emit the
module Wrangler serves. The JS shim, the factory link arguments in
.cargo/config.toml and the wasm-bindgen CLI plumbing in the scripts go away;
build.rs keeps only the application's own settings.

The Durable Object no longer owns a runtime. `connect` is a
`#[wasm_bindgen(tokio)]` export (wasm-bindgen/wasm-bindgen#5334) whose
future runs on the thread's Tokio `LocalEventLoop` (tokio-rs/tokio#8484),
the host event loop driving the scheduler. The server lifetime is spawned
as a task so a panic arrives as a JoinError and fails the object.

Pins move to the event-loop stack: tokio `emscripten-event-loop-host`, mio
`emscripten`, workers-rs `gbedford/worker-build-emscripten` with its
wasm-bindgen submodule supplying every wasm-bindgen crate, and emscripten
`cf-final` with the emsdk release its main is paired with. Binaryen comes
from that emsdk, so the separate Binaryen checkout and build are dropped.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@guybedford