Add wasm32-unknown-emscripten target support - #1969
guybedford wants to merge 3 commits into
Conversation
|
Yeah that could work - let me see if I can come up with a way for the
callback mechanism to only be on the Tokio side instead.
…On Sun, Jul 12, 2026 at 14:34 Alice Ryhl ***@***.***> wrote:
***@***.**** commented on this pull request.
------------------------------
In src/poll.rs
<#1969 (comment)>:
> @@ -444,6 +444,40 @@ impl Poll {
}
}
+#[cfg(all(target_os = "emscripten", feature = "os-poll"))]
+impl Poll {
+ /// Create a `Poll` whose epoll set carries a persistent, non-blocking
+ /// readiness callback for the lifetime of the `Poll`. Instead of blocking in
+ /// [`Poll::poll`], the emscripten runtime delivers up to `capacity` ready
+ /// events to `callback` on a fresh host tick whenever the set makes progress
+ /// (needs no JSPI/ASYNCIFY). The callback is unregistered when the `Poll` is
+ /// dropped.
+ ///
+ /// The `callback` may freely re-enter this `Poll`/`Registry` (e.g. register
+ /// or deregister sources): it holds no internal borrow while running.
+ pub fn new_with_callback<F>(capacity: usize, callback: F) -> io::Result<Poll>
The code already copies the events into the Events buffer, so instead of
passing them to a callback, I don't see why you can't just keep them around
in said buffer until the next call to poll and return them to the caller
then. That wouldn't need any user-facing API change.
—
Reply to this email directly, view it on GitHub
<#1969?email_source=notifications&email_token=AAESFSWFKSGVAOT73HVO2FT5EP75RA5CNFSNUABKM5UWIORPF5TWS5BNNB2WEL2QOVWGYUTFOF2WK43UKJSXM2LFO4XTINRYGA4DEMJSGI4KM4TFMFZW63VGMF2XI2DPOKSWK5TFNZ2KYZTPN52GK4S7MNWGSY3L#discussion_r3567166992>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAESFSVF7QLBRUCJ7LS35NL5EP75RAVCNFSNUABEKJSXA33TNF2G64TZHMZDGMJTHA4TQNB3JFZXG5LFHM2DQNRQGAZDGMJUHGQXMAQ>
.
You are receiving this because you authored the thread.Message ID:
***@***.***>
|
|
Why do you want the callback mechanism on the Tokio side rather than just storing them in this buffer until the next time Tokio calls |
e3ecd61 to
4c43fd5
Compare
Darksonn
left a comment
There was a problem hiding this comment.
It looks like you were able to reduce this down to merely adjusting cfgs, which is great, thanks!
Thomasdezeeuw
left a comment
There was a problem hiding this comment.
I'm not sure what to do about the missing public api (UnixStream::pair and UnixDatagram). Removing them, as this API does, is one option, we can also include them, but always return an unsupported error.
What do you think @Darksonn?
|
My suggestion would be to remove the unsupported APIs. IMO it's no different from the fact that |
I've gone ahead and implemented this. |
876f031 to
a62c9e4
Compare
|
To give a progress update here - Emscripten landed support for epoll yesterday in emscripten-core/emscripten#27207 🥳 This PR is now just awaiting two more PRs expected to land this week (libc and Emscripten as per the description checklist), at which point it'll be ready for final review. In the mean time, the code remains feature complete, so reviews continue to be very much welcome. |
Follow-on to tokio-rs#8281 to support real async under Emscripten pthreads and JSPI. For JSPI, a `runtime::jspi` module provides the `__asyncjs__` suspension import timer, the only suspending import, as well as a feature check for JSPI being used. The park leaf suspends the whole Wasm stack on a host timer, so the host loop runs while Tokio waits and resumes it when the timer fires. For pthreads it actually just works naturally, with the test suite change to use `-sPROXY_TO_PTHREAD`. Emscripten runs filesystem operations on the main thread, so `tokio::fs` deadlocks under a runtime that parks it. The scheduler, task system and time driver are the canonical ones, tokio-macros is untouched, and no public API is added. The scheduler's `event_interval` maintenance park arrives as a zero-duration park and resumes through a host immediate, so a greedy self-waking task cannot starve host timers: the immediate it schedules next lands in the following event loop iteration, which begins by running expired timers. `setTimeout(0)` also works but is clamped to a millisecond, and a microtask queue does not work at all, since those drain before the loop advances. Coverage expands from one run of the suite in which a timed wait blocked the thread, plus a single pthread test file, to three lanes: * non-JSPI: 4 passed, 0 failed * JSPI: 763 passed, 0 failed, 10 ignored * pthreads (`-pthread -sPROXY_TO_PTHREAD`): 762 passed, 0 failed, 10 ignored Real-timer tests for Emscripten are consolidated into the single `rt_emscripten_jspi.rs` file. * `spawn_blocking` remains unsupported in non-pthread builds * `net` (epoll, over tokio-rs/mio#1969) remains the follow-up * Without `-sJSPI`, a timed wait now panics rather than blocking the thread through `std::thread::sleep`. This is the timer behaviour change the Wasm docs already anticipate, and freezing the host loop is not a useful way to wait on this target. * A wait with no deadline panics in either mode. There is no reactor here, so host timers are the only mid-park wake source and nothing could ever deliver the wake. * JSPI can support re-entrancy through stack saving and restoring, but that is not implemented here, so this is sound only under non-reentrant JSPI currently. Refs: tokio-rs#8281
Follow-on to tokio-rs#8281 to support real async under Emscripten pthreads and JSPI. For JSPI, a `runtime::jspi` module provides the `__asyncjs__` suspension import timer, the only suspending import, as well as a feature check for JSPI being used. The park leaf suspends the whole Wasm stack on a host timer, so the host loop runs while Tokio waits and resumes it when the timer fires. For pthreads it works naturally, with the test suite change to use `-sPROXY_TO_PTHREAD`. Emscripten runs filesystem operations on the main thread, so `tokio::fs` deadlocks under a runtime that parks it. The scheduler, task system and time driver are the canonical ones, tokio-macros is untouched, and no public API is added. The scheduler's `event_interval` maintenance park arrives as a zero-duration park and resumes through a host immediate, so a busy scheduler still gives the host loop a turn, the role the non-blocking I/O poll plays on native: the immediate it schedules next lands in the following event loop iteration, which begins by running expired timers. `setTimeout(0)` also works but is clamped to a millisecond, and a microtask queue does not work at all, since those drain before the loop advances. Without JSPI the zero-duration park is a no-op, as on native. Coverage expands from one run of the suite in which a timed wait blocked the thread, plus a single pthread test file, to three lanes: * non-JSPI: 4 passed, 0 failed * JSPI: 763 passed, 0 failed, 10 ignored * pthreads (`-pthread -sPROXY_TO_PTHREAD`): 762 passed, 0 failed, 10 ignored Real-timer tests for Emscripten are consolidated into the single `rt_emscripten_jspi.rs` file. * `spawn_blocking` remains unsupported in non-pthread builds * `net` (epoll, over tokio-rs/mio#1969) remains the follow-up * Without `-sJSPI`, a timed wait now panics rather than blocking the thread through `std::thread::sleep`. This is the timer behaviour change the Wasm docs already anticipate, and freezing the host loop is not a useful way to wait on this target. * A wait with no deadline panics in either mode. There is no reactor here, so host timers are the only mid-park wake source and nothing could ever deliver the wake. * JSPI can support re-entrancy through stack saving and restoring, but that is not implemented here, so this is sound only under non-reentrant JSPI currently. Refs: tokio-rs#8281
Follow-on to tokio-rs#8281 to support real async under Emscripten pthreads and JSPI. For JSPI, a `runtime::jspi` module provides the `__asyncjs__` suspension import timer, the only suspending import, as well as a feature check for JSPI being used. The park leaf suspends the whole Wasm stack on a host timer, so the host loop runs while Tokio waits and resumes it when the timer fires. For pthreads it works naturally. Emscripten's filesystem is synchronous, so `tokio::fs` and `io-std` keep using the inline blocking shim there too, and the pthread lane needs no `-sPROXY_TO_PTHREAD`. The scheduler, task system and time driver are the canonical ones, tokio-macros is untouched, and no public API is added. The scheduler's `event_interval` maintenance park arrives as a zero-duration park and resumes through a host immediate, so a busy scheduler still gives the host loop a turn, the role the non-blocking I/O poll plays on native: the immediate it schedules next lands in the following event loop iteration, which begins by running expired timers. `setTimeout(0)` also works but is clamped to a millisecond, and a microtask queue does not work at all, since those drain before the loop advances. Without JSPI the zero-duration park is a no-op, as on native. Coverage expands from one run of the suite in which a timed wait blocked the thread, plus a single pthread test file, to three lanes: * non-JSPI: 4 passed, 0 failed * JSPI: 763 passed, 0 failed, 10 ignored * pthreads (`-pthread`): 762 passed, 0 failed, 10 ignored Real-timer tests for Emscripten are consolidated into the single `rt_emscripten_jspi.rs` file. * `spawn_blocking` remains unsupported in non-pthread builds * `net` (epoll, over tokio-rs/mio#1969) remains the follow-up * Without `-sJSPI`, a timed wait now panics rather than blocking the thread through `std::thread::sleep`. This is the timer behaviour change the Wasm docs already anticipate, and freezing the host loop is not a useful way to wait on this target. * A wait with no deadline panics in either mode. There is no reactor here, so host timers are the only mid-park wake source and nothing could ever deliver the wake. * JSPI can support re-entrancy through stack saving and restoring, but that is not implemented here, so this is sound only under non-reentrant JSPI currently. Refs: tokio-rs#8281
…nd pthreads Follow-on to tokio-rs#8281. That PR enabled the `wasm32-unknown-emscripten` target, but a timed wait on the single-threaded runtime blocked the thread through `std::thread::sleep`, and the pthread lane covered a single test file. For JSPI, a `runtime::jspi` module provides the `__asyncjs__` suspension import timer, the only suspending import, as well as a feature check for JSPI being used. The park leaf suspends the whole Wasm stack on a host timer, so the host loop runs while Tokio waits and resumes it when the timer fires. For pthreads nothing in the runtime changes: the native condvar parker already works over `Atomics.wait`. The pthread CI lane now runs the full suite. Emscripten's filesystem is synchronous, so `tokio::fs` and `io-std` use the inline blocking shim in pthread builds too, which is what lets that lane run without `-sPROXY_TO_PTHREAD`. The scheduler, task system and time driver are the canonical ones, tokio-macros is untouched, and no public API is added. The scheduler's `event_interval` maintenance park arrives as a zero-duration park and resumes through a host immediate, so a busy scheduler still gives the host loop a turn, the role the non-blocking I/O poll plays on native: the immediate it schedules next lands in the following event loop iteration, which begins by running expired timers. `setTimeout(0)` also works but is clamped to a millisecond, and a microtask queue does not work at all, since those drain before the loop advances. Without JSPI the zero-duration park is a no-op, as on native. Coverage expands from one run of the suite in which a timed wait blocked the thread, plus a single pthread test file, to three lanes: * non-JSPI: 4 passed, 0 failed * JSPI: 763 passed, 0 failed, 10 ignored * pthreads (`-pthread`): 762 passed, 0 failed, 10 ignored Real-timer tests for Emscripten are consolidated into the single `rt_emscripten_jspi.rs` file. * `spawn_blocking` remains unsupported in non-pthread builds * `net` (epoll, over tokio-rs/mio#1969) remains the follow-up * Without `-sJSPI`, a timed wait now panics rather than blocking the thread through `std::thread::sleep`. This is the timer behaviour change the Wasm docs already anticipate, and freezing the host loop is not a useful way to wait on this target. * A wait with no deadline panics in either mode. There is no reactor here, so host timers are the only mid-park wake source and nothing could ever deliver the wake. * JSPI can support re-entrancy through stack saving and restoring, but that is not implemented here, so this is sound only under non-reentrant JSPI currently. Refs: tokio-rs#8281
Follow-on to tokio-rs#8281, supporting JSPI as an alternative approach to suspension. A `runtime::jspi` module provides the `__asyncjs__` suspension import timer, the only suspending import, as well as a feature check for JSPI being used. The park leaf suspends the whole Wasm stack on a host timer, so the host loop runs while Tokio waits and resumes it when the timer fires. The scheduler, task system and time driver are the canonical ones, tokio-macros is untouched, and no public API is added. The scheduler's `event_interval` maintenance park arrives as a zero-duration park and resumes through a host immediate, so a busy scheduler still gives the host loop a turn, the role the non-blocking I/O poll plays on native: the immediate it schedules next lands in the following event loop iteration, which begins by running expired timers. Without JSPI the zero-duration park is a no-op, as on native. `setTimeout(0)` also works but is clamped to a millisecond, and a microtask queue does not work at all, since those drain before the loop advances. Coverage expands from one run of the suite in which a timed wait blocked the thread, plus a single pthread test file, to three lanes: * non-JSPI: 4 passed, 0 failed * JSPI: 763 passed, 0 failed, 10 ignored * pthreads (`-pthread`): 762 passed, 0 failed, 10 ignored In the process, the pthreads test suite is ungated as well, and changed to share the blocking shim, since Emscripten has a sync-only FS. Real-timer tests for Emscripten are consolidated into the single `rt_emscripten_jspi.rs` file. * `spawn_blocking` remains unsupported in non-pthread builds * `net` (epoll, over tokio-rs/mio#1969) remains the follow-up * Without `-sJSPI`, a non-zero wait now panics rather than blocking the thread through `std::thread::sleep`. * A wait with no deadline panics in either mode. There is no reactor here, so host timers are the only mid-park wake source and nothing could ever deliver the wake. * JSPI can support re-entrancy through stack saving and restoring, but that is not implemented here, so this is sound only under non-reentrant JSPI currently. Refs: tokio-rs#8281
9b373cb to
7a0702f
Compare
Follow-on to tokio-rs#8285, enabling `tokio::net` on Emscripten over mio's epoll selector and Node's raw sockets (`-sNODERAWSOCKETS`). The I/O driver is the native one. Under JSPI, Emscripten's `epoll_wait` is a blocking wait that suspends on the host event loop, resuming on readiness (or the mio waker pipe) or the deadline, so `park` and `park_timeout` need nothing target-specific. The one addition is for the scheduler's zero-duration maintenance park: a zero-timeout `epoll_wait` is a synchronous probe, and the host loop is the only producer of readiness, so the driver yields one host turn first, as the zero-duration `ParkThread` park already does. Without JSPI a real wait panics rather than spinning, matching the existing park semantics. Under pthreads with `-sPROXY_TO_PTHREAD` the wait blocks on the worker as on native, with no target-specific code at all. `ScheduledIo::set_readiness` moves from `fetch_update` to an explicit `compare_exchange_weak` loop: Rust 1.99 deprecates `fetch_update`, and the MSRV predates `try_update`. TcpStream/TcpListener/UdpSocket, stream `AF_UNIX` sockets, `lookup_host` and `AsyncFd` work as on native. Gated where Node lacks the primitive: datagram `AF_UNIX` (`UnixDatagram`, `UnixSocket::new_datagram`), `socketpair(2)` (`UnixStream::pair`), and `SO_PEERCRED` (`peer_cred` reports unsupported). Name resolution goes through Emscripten's synchronous `getaddrinfo`, which maps hostnames to synthetic addresses, so the `localhost` tests are ignored on this target. CI adds `net` to both emscripten test runs on the released emsdk: JSPI (815 passed, 0 failed, 19 ignored) and pthreads, which regains `-sPROXY_TO_PTHREAD` (818 passed, 0 failed, 19 ignored). The JSPI run uses Rust beta until 1.99 is stable, which is where `OwnedFd::try_clone` (mio's registry handle) gains Emscripten support. TEMPORARY: `mio` comes from tokio-rs/mio#1969 until released.
Follow-on to tokio-rs#8285, enabling `tokio::net` on Emscripten over mio's epoll selector and Node's raw sockets (`-sNODERAWSOCKETS`). The I/O driver is the native one. Under JSPI, Emscripten's `epoll_wait` is a blocking wait that suspends on the host event loop, resuming on readiness (or the mio waker pipe) or the deadline, so `park` and `park_timeout` need nothing target-specific. The one addition is for the scheduler's zero-duration maintenance park: a zero-timeout `epoll_wait` is a synchronous probe, and the host loop is the only producer of readiness, so the driver yields one host turn first, as the zero-duration `ParkThread` park already does. Without JSPI a real wait panics rather than spinning, matching the existing park semantics. Under pthreads with `-sPROXY_TO_PTHREAD` the wait blocks on the worker as on native, with no target-specific code at all. TcpStream/TcpListener/UdpSocket, stream `AF_UNIX` sockets, `lookup_host` and `AsyncFd` work as on native. Gated where Node lacks the primitive: datagram `AF_UNIX` (`UnixDatagram`, `UnixSocket::new_datagram`), `socketpair(2)` (`UnixStream::pair`), and `SO_PEERCRED` (`peer_cred` reports unsupported). Name resolution goes through Emscripten's synchronous `getaddrinfo`, which maps hostnames to synthetic addresses, so the `localhost` tests are ignored on this target. CI adds `net` to both emscripten test runs on the released emsdk: JSPI (815 passed, 0 failed, 19 ignored) and pthreads, which regains `-sPROXY_TO_PTHREAD` (818 passed, 0 failed, 19 ignored). The JSPI run uses Rust beta until 1.99 is stable, which is where `OwnedFd::try_clone` (mio's registry handle) gains Emscripten support. TEMPORARY: `mio` comes from tokio-rs/mio#1969 until released.
One perpetual `#[wasm_bindgen(jspi)]` export builds a current-thread runtime and `block_on`s the whole server lifetime; every park suspends the Wasm stack on `epoll_wait`, so the hosted runtime adapter is gone. Pumpkin binds its stock `TcpListener` on 25565 inside the Durable Object's port table, and the object routes each inbound socket to it with `handleAsNodeConnection`, replacing the injected-stream entry point, wasm-streams, and the workers-rs dependency. `stop` cancels the server and the run promise settling is the checkpoint signal. Toolchain: Rust beta, emscripten main frontend over an emsdk backend, wasm-bindgen 0.2.128 CLI via `-sWASM_BINDGEN`, exnref exception handling throughout, tokio `emscripten-epoll`, mio tokio-rs/mio#1969, libc `libc-0.2`. rustc needs a larger compile-thread stack for pumpkin-data. Requires a workerd with per-Durable-Object port tables and `net.Server` inbound routing (`MINIFLARE_WORKERD_PATH`).
One perpetual `#[wasm_bindgen(jspi)]` export builds a current-thread runtime and `block_on`s the whole server lifetime; every park suspends the Wasm stack on `epoll_wait`, so the hosted runtime adapter is gone. Pumpkin binds its stock `TcpListener` on 25565 inside the Durable Object's port table, and the object routes each inbound socket to it with `handleAsNodeConnection`, replacing the injected-stream entry point, wasm-streams, and the workers-rs dependency. `stop` cancels the server and the run promise settling is the checkpoint signal. Toolchain: Rust beta, emscripten main frontend paired with the matching emscripten-releases backend through emsdk, wasm-bindgen 0.2.128 CLI via `-sWASM_BINDGEN`, exnref exception handling throughout, tokio `emscripten-epoll`, mio tokio-rs/mio#1969, libc `libc-0.2`. Setup provisions all of it under .work/ (or reuses EMSDK/EMSCRIPTEN). rustc needs a larger compile-thread stack for pumpkin-data. The wasm-bindgen and workers-rs patches and the CLI lockfile are gone; the Pumpkin patch drops the injected-stream entry point. Requires a workerd with per-Durable-Object port tables and `net.Server` inbound routing (`MINIFLARE_WORKERD_PATH`). CI moves to Linux.
One perpetual `#[wasm_bindgen(jspi)]` export builds a current-thread runtime and `block_on`s the whole server lifetime; every park suspends the Wasm stack on `epoll_wait`, so the hosted runtime adapter is gone. Pumpkin binds its stock `TcpListener` on 25565 inside the Durable Object's port table, and the object routes each inbound socket to it with `handleAsNodeConnection`, replacing the injected-stream entry point, wasm-streams, and the workers-rs dependency. `stop` cancels the server and the run promise settling is the checkpoint signal. Toolchain: Rust beta, the emscripten 6.0.9 release frontend with a backport of emscripten-core/emscripten#27208 over the Homebrew or emsdk 6.0.9 backend, wasm-bindgen 0.2.128 CLI via `-sWASM_BINDGEN`, exnref exception handling throughout, tokio `emscripten-epoll`, mio tokio-rs/mio#1969, libc `libc-0.2`. Setup provisions the sources, backend and CLI under .work/. rustc needs a larger compile-thread stack for pumpkin-data. The wasm-bindgen and workers-rs patches and the CLI lockfile are gone; the Pumpkin patch drops the injected-stream entry point. Requires a workerd with per-Durable-Object port tables and `net.Server` inbound routing (`MINIFLARE_WORKERD_PATH`). CI moves to Linux.
Follow-on to tokio-rs#8281, supporting JSPI as an alternative approach to suspension. A `context::jspi` module provides the `__asyncjs__` suspension import timer, the only suspending import, as well as a feature check for JSPI being used. The park leaf suspends the whole Wasm stack on a host timer, so the host loop runs while Tokio waits and resumes it when the timer fires. The scheduler, task system and time driver are the canonical ones, tokio-macros is untouched, and no public API is added. The scheduler's `event_interval` maintenance park arrives as a zero-duration park and resumes through a host immediate, so a busy scheduler still gives the host loop a turn, the role the non-blocking I/O poll plays on native: the immediate it schedules next lands in the following event loop iteration, which begins by running expired timers. Without JSPI the zero-duration park is a no-op, as on native. `setTimeout(0)` also works but is clamped to a millisecond, and a microtask queue does not work at all, since those drain before the loop advances. A park leaves the runtime. Under single-threaded JS semantics, parked and "another activation is running" are the same state, so a sibling promising activation may call `block_on` on the same thread while one is suspended. `enter_runtime` records a `Snapshot` of the context it was entered from (what it and `set_scheduler` write: entered flag, rng, current handle and depth, scheduler pointer) and its guard restores the previous one; `jspi::suspended` wraps the suspending call, restoring the entry snapshot around it and its own after, on unwind too: a JS error out of the import (`SuspendError` from a non-promising activation) unwinds the runtime's guards cleanly rather than leaving the thread half-left. The record is part of the snapshot so nesting through the host composes. A suspension Tokio does not issue, such as a suspending import called from task code, keeps the runtime entered, so a sibling `block_on` during it panics as nested, and nested `block_on` from Rust is unchanged. Coverage expands from one run of the suite in which a timed wait blocked the thread, plus a single pthread test file, to three lanes: * non-JSPI: 4 passed, 0 failed * JSPI: 791 passed, 0 failed, 10 ignored * pthreads (`-pthread`): 762 passed, 0 failed, 10 ignored In the process, the pthreads test suite is ungated as well, and changed to share the blocking shim, since Emscripten has a sync-only FS. Real-timer tests for Emscripten are consolidated into the single `rt_emscripten_jspi.rs` file. Its sibling-activation tests are driven from a `--js-library` shim, since rustc internalizes a test binary's `__em_js__` statics. The `interleaved_suspended_runtimes` test is ignored: Emscripten shares one shadow stack between promising activations, so a runtime doing work while a sibling is suspended overwrites the sibling's frames until the toolchain switches stacks. * `spawn_blocking` remains unsupported in non-pthread builds * `net` (epoll, over tokio-rs/mio#1969) remains the follow-up * Without `-sJSPI`, a non-zero wait now panics rather than blocking the thread through `std::thread::sleep`. * A wait with no deadline panics in either mode. There is no reactor here, so host timers are the only mid-park wake source and nothing could ever deliver the wake. * `task::local::CURRENT` is not part of the swapped context; a `LocalSet` entered in a suspended sibling would be visible to a resumed activation once interleaving is possible. Refs: tokio-rs#8281
Follow-on to tokio-rs#8285, enabling `tokio::net` on Emscripten over mio's epoll selector and Node's raw sockets (`-sNODERAWSOCKETS`). The I/O driver is the native one. Under JSPI, Emscripten's `epoll_wait` is a blocking wait that suspends on the host event loop, resuming on readiness (or the mio waker pipe) or the deadline, so `park` and `park_timeout` need nothing target-specific beyond `jspi::io_wait`, which makes the wait a park in the JSPI sense: it leaves the runtime around the `epoll_wait` as the timer park does, so sibling activations may drive their own runtime while it is suspended. It also handles the scheduler's zero-duration maintenance park: a zero-timeout `epoll_wait` is a synchronous probe, and the host loop is the only producer of readiness, so the driver yields one host turn first, as the zero-duration `ParkThread` park already does. Without JSPI a real wait panics rather than spinning, matching the existing park semantics. Under pthreads with `-sPROXY_TO_PTHREAD` the wait blocks on the worker as on native, with no target-specific code at all. A wait with no deadline is a real `epoll_wait` that a socket could wake, so it suspends rather than panicking as the reactor-less park does; the `rt_emscripten_block_on` test of that panic is gated off `net`. TcpStream/TcpListener/UdpSocket, stream `AF_UNIX` sockets, `lookup_host` and `AsyncFd` work as on native. Gated where Node lacks the primitive: datagram `AF_UNIX` (`UnixDatagram`, `UnixSocket::new_datagram`), `socketpair(2)` (`UnixStream::pair`), and `SO_PEERCRED` (`peer_cred` reports unsupported). Name resolution goes through Emscripten's synchronous `getaddrinfo`, which maps hostnames to synthetic addresses, so the `localhost` tests are ignored on this target. CI adds `net` to both emscripten test runs on the released emsdk: JSPI (846 passed, 0 failed, 19 ignored) and pthreads, which regains `-sPROXY_TO_PTHREAD` (818 passed, 0 failed, 19 ignored). The JSPI lane links a `--pre-js` that fails a test binary whose `main` never returns: a park that suspends with no wake source leaves Node's event loop to drain and the process to exit 0, which cargo would otherwise take as success. The JSPI run uses Rust beta until 1.99 is stable, which is where `OwnedFd::try_clone` (mio's registry handle) gains Emscripten support. TEMPORARY: `mio` comes from tokio-rs/mio#1969 until released, which in turn takes `libc` from the `libc-0.2` branch for its emscripten epoll bindings (rust-lang/libc#5427).
Follow-on to tokio-rs#8285, enabling `tokio::net` on Emscripten over mio's epoll selector and Node's raw sockets (`-sNODERAWSOCKETS`). The I/O driver is the native one. Under JSPI, Emscripten's `epoll_wait` is a blocking wait that suspends on the host event loop, resuming on readiness (or the mio waker pipe) or the deadline, so `park` and `park_timeout` need nothing target-specific beyond `jspi::io_wait`, which makes the wait a park in the JSPI sense: it leaves the runtime around the `epoll_wait` as the timer park does, so sibling activations may drive their own runtime while it is suspended. It also handles the scheduler's zero-duration maintenance park: a zero-timeout `epoll_wait` is a synchronous probe, and the host loop is the only producer of readiness, so the driver yields one host turn first, as the zero-duration `ParkThread` park already does. Without JSPI a real wait panics rather than spinning, matching the existing park semantics. Under pthreads with `-sPROXY_TO_PTHREAD` the wait blocks on the worker as on native, with no target-specific code at all. A wait with no deadline is a real `epoll_wait` that a socket could wake, so it suspends rather than panicking as the reactor-less park does; the `rt_emscripten_block_on` test of that panic is gated off `net`. TcpStream/TcpListener/UdpSocket, stream `AF_UNIX` sockets, `lookup_host` and `AsyncFd` work as on native. Gated where Node lacks the primitive: datagram `AF_UNIX` (`UnixDatagram`, `UnixSocket::new_datagram`), `socketpair(2)` (`UnixStream::pair`), and `SO_PEERCRED` (`peer_cred` reports unsupported). Name resolution goes through Emscripten's synchronous `getaddrinfo`, which maps hostnames to synthetic addresses, so the `localhost` tests are ignored on this target. CI adds `net` to both emscripten test runs on the released emsdk: JSPI (846 passed, 0 failed, 19 ignored) and pthreads, which regains `-sPROXY_TO_PTHREAD` (818 passed, 0 failed, 19 ignored). The JSPI lane links a `--pre-js` that fails a test binary whose `main` never returns: a park that suspends with no wake source leaves Node's event loop to drain and the process to exit 0, which cargo would otherwise take as success. The JSPI run uses Rust beta until 1.99 is stable, which is where `OwnedFd::try_clone` (mio's registry handle) gains Emscripten support. TEMPORARY: `mio` comes from tokio-rs/mio#1969 until released, which in turn takes `libc` from the `libc-0.2` branch for its emscripten epoll bindings (rust-lang/libc#5427).
Follow-on to tokio-rs#8281, supporting JSPI as an alternative approach to suspension. A `context::jspi` module provides the `__asyncjs__` suspension import timer, the only suspending import, as well as a feature check for JSPI being used. The park leaf suspends the whole Wasm stack on a host timer, so the host loop runs while Tokio waits and resumes it when the timer fires. The scheduler, task system and time driver are the canonical ones, tokio-macros is untouched, and no public API is added. The scheduler's `event_interval` maintenance park arrives as a zero-duration park and resumes through a host immediate, so a busy scheduler still gives the host loop a turn, the role the non-blocking I/O poll plays on native: the immediate it schedules next lands in the following event loop iteration, which begins by running expired timers. Without JSPI the zero-duration park is a no-op, as on native. `setTimeout(0)` also works but is clamped to a millisecond, and a microtask queue does not work at all, since those drain before the loop advances. A park leaves the runtime. Under single-threaded JS semantics, parked and "another activation is running" are the same state, so a sibling promising activation may call `block_on` on the same thread while one is suspended. `enter_runtime` records a `Snapshot` of the context it was entered from (what it and `set_scheduler` write: entered flag, rng, current handle and depth, scheduler pointer) and its guard restores the previous one; `jspi::suspended` wraps the suspending call, restoring the entry snapshot around it and its own after, on unwind too: a JS error out of the import (`SuspendError` from a non-promising activation) unwinds the runtime's guards cleanly rather than leaving the thread half-left. The record is part of the snapshot so nesting through the host composes. A suspension Tokio does not issue, such as a suspending import called from task code, keeps the runtime entered, so a sibling `block_on` during it panics as nested, and nested `block_on` from Rust is unchanged. Coverage expands from one run of the suite in which a timed wait blocked the thread, plus a single pthread test file, to three lanes: * non-JSPI: 4 passed, 0 failed * JSPI: 791 passed, 0 failed, 10 ignored * pthreads (`-pthread`): 762 passed, 0 failed, 10 ignored In the process, the pthreads test suite is ungated as well, and changed to share the blocking shim, since Emscripten has a sync-only FS. Real-timer tests for Emscripten are consolidated into the single `rt_emscripten_jspi.rs` file. Its sibling-activation tests are driven from a `--js-library` shim, since rustc internalizes a test binary's `__em_js__` statics. The `interleaved_suspended_runtimes` test is ignored: Emscripten shares one shadow stack between promising activations, so a runtime doing work while a sibling is suspended overwrites the sibling's frames until the toolchain switches stacks. * `spawn_blocking` remains unsupported in non-pthread builds * `net` (epoll, over tokio-rs/mio#1969) remains the follow-up * Without `-sJSPI`, a non-zero wait now panics rather than blocking the thread through `std::thread::sleep`. * A wait with no deadline panics in either mode. There is no reactor here, so host timers are the only mid-park wake source and nothing could ever deliver the wake. * `task::local::CURRENT` is not part of the swapped context; a `LocalSet` entered in a suspended sibling would be visible to a resumed activation once interleaving is possible. Refs: tokio-rs#8281
One perpetual `#[wasm_bindgen(jspi)]` export builds a current-thread runtime and `block_on`s the whole server lifetime; every park suspends the Wasm stack on `epoll_wait`, so the hosted runtime adapter is gone. Pumpkin binds its stock `TcpListener` on 25565 inside the Durable Object's port table, and the object routes each inbound socket to it with `handleAsNodeConnection`, replacing the injected-stream entry point, wasm-streams, and the workers-rs dependency. `stop` cancels the server and the run promise settling is the checkpoint signal. `-sREENTRANT_JSPI` gives each activation its own shadow stack, so other entries into the module while the server is suspended cannot clobber its frames. Toolchain: Rust beta; emscripten main plus the JSPI hooks, reentrant JSPI and epoll listener PRs, with the paired emscripten-releases LLVM and the jspi-hooks Binaryen branch built by setup; wasm-bindgen 0.2.128 CLI via `-sWASM_BINDGEN`; exnref exception handling throughout; tokio `emscripten-epoll`, mio tokio-rs/mio#1969, libc `libc-0.2`. rustc needs a larger compile-thread stack for pumpkin-data. The wasm-bindgen and workers-rs patches and the CLI lockfile are gone; the Pumpkin patch drops the injected-stream entry point. Requires a workerd with per-Durable-Object port tables and `net.Server` inbound routing (`MINIFLARE_WORKERD_PATH`). CI moves to Linux.
|
@Darksonn thanks, I've pushed up a change to fix it. |
Follow-on to tokio-rs#8281, supporting JSPI as an alternative approach to suspension. A `context::jspi` module provides the `__asyncjs__` suspension import timer, the only suspending import, as well as a feature check for JSPI being used. The park leaf suspends the whole Wasm stack on a host timer, so the host loop runs while Tokio waits and resumes it when the timer fires. The scheduler, task system and time driver are the canonical ones, tokio-macros is untouched, and no public API is added. The scheduler's `event_interval` maintenance park arrives as a zero-duration park and resumes through a host immediate, so a busy scheduler still gives the host loop a turn, the role the non-blocking I/O poll plays on native: the immediate it schedules next lands in the following event loop iteration, which begins by running expired timers. Without JSPI the zero-duration park is a no-op, as on native. `setTimeout(0)` also works but is clamped to a millisecond, and a microtask queue does not work at all, since those drain before the loop advances. The runtime stays entered while parked, as on native: a `block_on` from another promising activation on the same thread during the park panics as a nested runtime. Driving the runtime from several activations is a follow-on. Coverage expands from one run of the suite in which a timed wait blocked the thread, plus a single pthread test file, to three lanes: * non-JSPI: 4 passed, 0 failed * JSPI: 788 passed, 0 failed, 10 ignored * pthreads (`-pthread`): 762 passed, 0 failed, 10 ignored In the process, the pthreads test suite is ungated as well, and changed to share the blocking shim, since Emscripten has a sync-only FS. Real-timer tests for Emscripten are consolidated into the single `rt_emscripten_jspi.rs` file. * `spawn_blocking` remains unsupported in non-pthread builds * `net` (epoll, over tokio-rs/mio#1969) remains the follow-up * Without `-sJSPI`, a non-zero wait now panics rather than blocking the thread through `std::thread::sleep`. * A wait with no deadline panics in either mode. There is no reactor here, so host timers are the only mid-park wake source and nothing could ever deliver the wake. Refs: tokio-rs#8281
Follow-on to tokio-rs#8281, supporting JSPI as an alternative approach to suspension. A `context::jspi` module provides the `__asyncjs__` suspension import timer, the only suspending import, as well as a feature check for JSPI being used. The park leaf suspends the whole Wasm stack on a host timer, so the host loop runs while Tokio waits and resumes it when the timer fires. The scheduler, task system and time driver are the canonical ones, tokio-macros is untouched, and no public API is added. The scheduler's `event_interval` maintenance park arrives as a zero-duration park and resumes through a host immediate, so a busy scheduler still gives the host loop a turn, the role the non-blocking I/O poll plays on native: the immediate it schedules next lands in the following event loop iteration, which begins by running expired timers. Without JSPI the zero-duration park is a no-op, as on native. `setTimeout(0)` also works but is clamped to a millisecond, and a microtask queue does not work at all, since those drain before the loop advances. The runtime stays entered while parked, as on native: a `block_on` from another promising activation on the same thread during the park panics as a nested runtime. Driving the runtime from several activations is a follow-on. Coverage expands from one run of the suite in which a timed wait blocked the thread, plus a single pthread test file, to three lanes: * non-JSPI: 4 passed, 0 failed * JSPI: 788 passed, 0 failed, 10 ignored * pthreads (`-pthread`): 762 passed, 0 failed, 10 ignored In the process, the pthreads test suite is ungated as well, and changed to share the blocking shim, since Emscripten has a sync-only FS. Real-timer tests for Emscripten are consolidated into the single `rt_emscripten_jspi.rs` file. * `spawn_blocking` remains unsupported in non-pthread builds * `net` (epoll, over tokio-rs/mio#1969) remains the follow-up * Without `-sJSPI`, a non-zero wait now panics rather than blocking the thread through `std::thread::sleep`. * A wait with no deadline panics in either mode. There is no reactor here, so host timers are the only mid-park wake source and nothing could ever deliver the wake. Refs: tokio-rs#8281
Follow-on to tokio-rs#8285, enabling `tokio::net` on Emscripten over mio's epoll selector and Node's raw sockets (`-sNODERAWSOCKETS`). The I/O driver is the native one. Under JSPI, Emscripten's `epoll_wait` is a blocking wait that suspends on the host event loop, resuming on readiness (or the mio waker pipe) or the deadline, so `park` and `park_timeout` need nothing target-specific beyond `jspi::io_wait`, which handles the scheduler's zero-duration maintenance park: a zero-timeout `epoll_wait` is a synchronous probe, and the host loop is the only producer of readiness, so the driver yields one host turn first, as the zero-duration `ParkThread` park already does. Without JSPI a real wait panics rather than spinning, matching the existing park semantics. Under pthreads with `-sPROXY_TO_PTHREAD` the wait blocks on the worker as on native, with no target-specific code at all. A wait with no deadline is a real `epoll_wait` that a socket could wake, so it suspends rather than panicking as the reactor-less park does; the `rt_emscripten_block_on` test of that panic is gated off `net`. TcpStream/TcpListener/UdpSocket, stream `AF_UNIX` sockets, `lookup_host` and `AsyncFd` work as on native. Gated where Node lacks the primitive: datagram `AF_UNIX` (`UnixDatagram`, `UnixSocket::new_datagram`), `socketpair(2)` (`UnixStream::pair`), and `SO_PEERCRED` (`peer_cred` reports unsupported). Name resolution goes through Emscripten's synchronous `getaddrinfo`, which maps hostnames to synthetic addresses, so the `localhost` tests are ignored on this target. CI adds `net` to both emscripten test runs on the released emsdk: JSPI (846 passed, 0 failed, 19 ignored) and pthreads, which regains `-sPROXY_TO_PTHREAD` (818 passed, 0 failed, 19 ignored). The JSPI lane links a `--pre-js` that fails a test binary whose `main` never returns: a park that suspends with no wake source leaves Node's event loop to drain and the process to exit 0, which cargo would otherwise take as success. The JSPI run uses Rust beta until 1.99 is stable, which is where `OwnedFd::try_clone` (mio's registry handle) gains Emscripten support. TEMPORARY: `mio` comes from tokio-rs/mio#1969 until released, which in turn takes `libc` from the `libc-0.2` branch for its emscripten epoll bindings (rust-lang/libc#5427).
fa8787c to
56d1887
Compare
Adds `wasm32-unknown-emscripten` as a target for mio, plus a CI job that runs the suite under Node. Resolves tokio-rs#642. Emscripten exposes a real epoll backed by its runtime event loop, so the existing Linux epoll selector is reused rather than adding a new backend. The wasm `compile_error!` guard is relaxed to let emscripten through, and the `epoll`/`eventfd`/pipe cfg lists gain emscripten. AF_UNIX support is stream-only: emscripten's node-backed sockets have no datagram primitive, so `UnixDatagram` and the `socketpair`-based helpers are not compiled there. Sockets set `O_NONBLOCK` via `fcntl` since emscripten's `socket(2)` silently strips `SOCK_NONBLOCK`/`SOCK_CLOEXEC`. The test peers are std sockets on helper threads, so std is rebuilt with atomics via -Zbuild-std and linked -pthread with -sPROXY_TO_PTHREAD, and NODERAWFS/NODERAWSOCKETS back the filesystem and sockets with node's. Emscripten sockets never block - a call that would block returns EAGAIN - so the peers' blocking `accept`/`read` go through `util::accept`/`util::read`, which on emscripten wait for readiness with `poll(2)` (which does park a pthread) and retry; elsewhere they are the plain std calls. This runs on a released emsdk (6.0.9) with no emscripten patches. Nightly + rust-src are needed for -Zbuild-std; no custom target spec is required since nightly emits the __main_argc_argv entry point (rust-lang/rust#158937). Doctests are skipped on this target: rustdoc does not apply the emcc link args. Temporary, until released: Cargo.toml takes libc from the `libc-0.2` branch for the emscripten epoll bindings (rust-lang/libc#5427). Suite result: 147 passed, 0 failed, 3 ignored under Node. The one emscripten-specific ignore is `tcp_stream::raw_fd` (`getsockname` after a non-blocking connect can transiently report an unbound local address).
56d1887 to
8f57be2
Compare
|
This PR is now ready for review, and no longer draft. All upstream patchsets have now landed. We are just waiting for the libc@0.2.190 release further. In the mean time, reviews are very welcome. //cc @Darksonn |
| // and the caller is expected to wait for readiness itself. `poll(2)` does | ||
| // block on a pthread, which is where these peers run. | ||
| #[cfg(target_os = "emscripten")] | ||
| pub fn accept<L: Accept + AsRawFd>(listener: &L) -> io::Result<L::Conn> { |
There was a problem hiding this comment.
Aren't we using JSPI precisely to make blocking syscalls such as this one possible without special code such as this?
There was a problem hiding this comment.
This Mio implementation doesn't use JSPI, as that requires extra ceremony per the Tokio PR. Instead pthreads uses a JS worker to support blocking calls which gets things to mostly just work, and avoiding the need for the blocking JSPI Emscripten PR.
There was a problem hiding this comment.
Still, if you're using pthreads, why can't we just use a blocking call here instead of looping on a non-blocking call?
There was a problem hiding this comment.
I'm also not a fan of all this special code for testing. I would prefer if we can do without it.
There was a problem hiding this comment.
I had an Emscripten PR to do just that open for two months here - emscripten-core/emscripten#27342.
I just closed it to use this approach instead, because there was resistance to supporting a dual path sync/async function upstream. Accept behaves sync without jspi/pthreads, and async under jspi/pthreads, but Emscripten doesn't have a way to switch on those modes, which is what my PR added. Otherwise you need two variants of every blocking function in JS - a sync version and an async version. All difficult sells for Emscripten.
So my test refactor here was done to ensure this PR can land sooner (as soon as the next libc release).
Ideally we could just not use blocking sockets in the test suite here and everything would work out. I haven't really never needed support for blocking accept yet in test applications.
There was a problem hiding this comment.
I've posted a new attempt towards this in emscripten-core/emscripten#27724.
Depending on progress, we can decide if we want this PR to block on that or not.
I do think ideally we should land this since it only affects the tests, and do the refactoring as a follow-on, now that it has tracking PR again.
There was a problem hiding this comment.
I think if emscripten's goal is to get C programs to work verbatim without changes, you'll need these blocking calls to work. I would be ok with doing it temporarily if we can fix it in future emscripten, but it's Thomas's call.
There was a problem hiding this comment.
Certianly, we can see how progress goes with that - but I would strong suggest that once libc@0.2.190 is released, we shouldn't hold back the patchset on it if it is still pending.
| // and the caller is expected to wait for readiness itself. `poll(2)` does | ||
| // block on a pthread, which is where these peers run. | ||
| #[cfg(target_os = "emscripten")] | ||
| pub fn accept<L: Accept + AsRawFd>(listener: &L) -> io::Result<L::Conn> { |
There was a problem hiding this comment.
I'm also not a fan of all this special code for testing. I would prefer if we can do without it.
|
|
||
| [target.'cfg(any(unix, target_os = "hermit", target_os = "wasi"))'.dependencies] | ||
| libc = "0.2.183" | ||
| libc = { git = "https://github.com/rust-lang/libc", branch = "libc-0.2" } |
There was a problem hiding this comment.
Blocking: this needs a released version (I assume this is just a matter of time).
There was a problem hiding this comment.
Yes, we are just waiting on 0.2.190.
tokio-rs/tokio#8484 replaces EventLoopRuntime with EventLoop / LocalEventLoop: spawn_local queues only, drive runs one batch, and the root's outcome is read from its JoinHandle. The ambient and isolated schedulers now spawn the root plus a sibling task that awaits it and delivers the result, keeping the synchronous first poll (drive outside a runtime context) and panic -> JoinError semantics. The isolated event loop's drop is deferred to a microtask, since a runtime cannot be dropped from inside its own drive. The event loop builders are gated on tokio's net feature, so mio is patched to its emscripten branch (tokio-rs/mio#1969).
tokio-rs/tokio#8484 replaces EventLoopRuntime with EventLoop / LocalEventLoop: spawn_local queues only, drive runs one batch, and the root's outcome is read from its JoinHandle. The ambient and isolated schedulers now spawn the root plus a sibling task that awaits it and delivers the result, keeping the synchronous first poll (drive outside a runtime context) and panic -> JoinError semantics. The isolated event loop's drop is deferred to a microtask, since a runtime cannot be dropped from inside its own drive. The event loop builders are gated on tokio's net feature, so mio is patched to its emscripten branch (tokio-rs/mio#1969).
tokio-rs/tokio#8484 replaces EventLoopRuntime with EventLoop / LocalEventLoop; the example takes tokio from its branch and mio from its emscripten branch (tokio-rs/mio#1969). The wasm-bindgen submodule moves to the rebuilt emscripten stack: main plus the reinit fix (#5332), legalized export names and the tokio attribute (#5334) adapted to LocalEventLoop, without the emscripten JSPI support. worker's schedule_isolated use is unchanged.
tokio-rs/tokio#8484 replaces EventLoopRuntime with EventLoop / LocalEventLoop; the example takes tokio from its branch and mio from its emscripten branch (tokio-rs/mio#1969). The wasm-bindgen submodule moves to the rebuilt emscripten stack: main plus the reinit fix (#5332), legalized export names and the tokio attribute (#5334) adapted to LocalEventLoop, without the emscripten JSPI support. worker's schedule_isolated use is unchanged.
Follow-on to tokio-rs#8281, supporting JSPI as an alternative approach to suspension. A `context::jspi` module provides the `__asyncjs__` suspension import timer, the only suspending import, as well as a feature check for JSPI being used. The park leaf suspends the whole Wasm stack on a host timer, so the host loop runs while Tokio waits and resumes it when the timer fires. The scheduler, task system and time driver are the canonical ones, tokio-macros is untouched, and no public API is added. The scheduler's `event_interval` maintenance park arrives as a zero-duration park and resumes through a host immediate, so a busy scheduler still gives the host loop a turn, the role the non-blocking I/O poll plays on native: the immediate it schedules next lands in the following event loop iteration, which begins by running expired timers. Without JSPI the zero-duration park is a no-op, as on native. `setTimeout(0)` also works but is clamped to a millisecond, and a microtask queue does not work at all, since those drain before the loop advances. The runtime stays entered while parked, as on native: a `block_on` from another promising activation on the same thread during the park panics as a nested runtime. Driving the runtime from several activations is a follow-on. Coverage expands from one run of the suite in which a timed wait blocked the thread, plus a single pthread test file, to three lanes: * non-JSPI: 4 passed, 0 failed * JSPI: 788 passed, 0 failed, 10 ignored * pthreads (`-pthread`): 762 passed, 0 failed, 10 ignored In the process, the pthreads test suite is ungated as well, and changed to share the blocking shim, since Emscripten has a sync-only FS. Real-timer tests for Emscripten are consolidated into the single `rt_emscripten_jspi.rs` file. * `spawn_blocking` remains unsupported in non-pthread builds * `net` (epoll, over tokio-rs/mio#1969) remains the follow-up * Without `-sJSPI`, a non-zero wait now panics rather than blocking the thread through `std::thread::sleep`. * A wait with no deadline panics in either mode. There is no reactor here, so host timers are the only mid-park wake source and nothing could ever deliver the wake. Refs: tokio-rs#8281
Follow-on to tokio-rs#8285, enabling `tokio::net` on Emscripten over mio's epoll selector and Node's raw sockets (`-sNODERAWSOCKETS`). The I/O driver is the native one. Under JSPI, Emscripten's `epoll_wait` is a blocking wait that suspends on the host event loop, resuming on readiness (or the mio waker pipe) or the deadline, so `park` and `park_timeout` need nothing target-specific beyond `jspi::io_wait`, which handles the scheduler's zero-duration maintenance park: a zero-timeout `epoll_wait` is a synchronous probe, and the host loop is the only producer of readiness, so the driver yields one host turn first, as the zero-duration `ParkThread` park already does. Without JSPI a real wait panics rather than spinning, matching the existing park semantics. Under pthreads with `-sPROXY_TO_PTHREAD` the wait blocks on the worker as on native, with no target-specific code at all. A wait with no deadline is a real `epoll_wait` that a socket could wake, so it suspends rather than panicking as the reactor-less park does; the `rt_emscripten_block_on` test of that panic is gated off `net`. TcpStream/TcpListener/UdpSocket, stream `AF_UNIX` sockets, `lookup_host` and `AsyncFd` work as on native. Gated where Node lacks the primitive: datagram `AF_UNIX` (`UnixDatagram`, `UnixSocket::new_datagram`), `socketpair(2)` (`UnixStream::pair`), and `SO_PEERCRED` (`peer_cred` reports unsupported). Name resolution goes through Emscripten's synchronous `getaddrinfo`, which maps hostnames to synthetic addresses, so the `localhost` tests are ignored on this target. CI adds `net` to both emscripten test runs on the released emsdk: JSPI (846 passed, 0 failed, 19 ignored) and pthreads, which regains `-sPROXY_TO_PTHREAD` (818 passed, 0 failed, 19 ignored). The JSPI lane links a `--pre-js` that fails a test binary whose `main` never returns: a park that suspends with no wake source leaves Node's event loop to drain and the process to exit 0, which cargo would otherwise take as success. The JSPI run uses Rust beta until 1.99 is stable, which is where `OwnedFd::try_clone` (mio's registry handle) gains Emscripten support. TEMPORARY: `mio` comes from tokio-rs/mio#1969 until released, which in turn takes `libc` from the `libc-0.2` branch for its emscripten epoll bindings (rust-lang/libc#5427).
tokio-rs/tokio#8484 replaces EventLoopRuntime with EventLoop / LocalEventLoop: spawn_local queues only, drive runs one batch, and the root's outcome is read from its JoinHandle. The ambient and isolated schedulers now spawn the root plus a sibling task that awaits it and delivers the result, keeping the synchronous first poll (drive outside a runtime context) and panic -> JoinError semantics. The isolated event loop's drop is deferred to a microtask, since a runtime cannot be dropped from inside its own drive. The event loop builders are gated on tokio's net feature, so mio is patched to its emscripten branch (tokio-rs/mio#1969).
Resolves #642.
Adds
wasm32-unknown-emscriptenas a target for mio, with a CI job running the suite under Node. Opening as a draft for now since it depends on unreleased patchsets for Emscripten and Rust's libc.This lays the groundwork for Tokio support for Emscripten. Initially I did not plan to PR Mio, but in discussion with @Darksonn and @Noah-Kennedy it was suggested to use Mio if possible, and this has worked out well in my opinion.
With emscripten-core/emscripten#27207 landed, Emscripten now exposes a real epoll, so the existing Linux epoll selector can reused rather than adding a new backend.
Patch Sets
The only remaining dependency needed to land this is libc@0.2.190, which for now is pinned to its 0.2 branch.
This PR required the following upstream patches, now all merged:
Test status
The tests run under
-pthread -sPROXY_TO_PTHREADin Emscripten.In order to support the blocking accept, it was necessary to refactor the test suite to support non-blocking sockets via a
util::accept/util::readabstraction.Tests run directly on the Emscripten Node.js build with
NODERAWSOCKETSandNODERAWFSto provide a transparent runner without further harness configuration being necessary.tcp_stream::raw_fd(Emscriptengetsocknamereports0.0.0.0as the local address until a non-blocking connect completes; port is stable since [NODERAWSOCKETS] Bind-first connect for synchronous getsockname emscripten-core/emscripten#27566, address fix pending).