Skip to content

Conversation

@ktock
Copy link
Contributor

@ktock ktock commented Oct 8, 2025

This PR fixes an issue that the timeout argument for the select syscall wasn't applied to pipes and it returned immedeately instead of waiting for events.

Although the select syscall passes the timeout value to the poll method of the stream implementations, this can't handle multiple fds because a fd can't notify readiness while another is blocking in poll. As a result, using the select syscall with a combination of pipes and other streams (e.g. PTY) can be problematic.

To address this, this PR implements a callback-based event notification. Each stream implementation's poll invocation receives a callback, allowing it asynchronously notify the select syscall when it becomes ready. The select syscall blocks until one of these callbacks is triggered or the timeout expires. This behviour is enabled only when PROXY_TO_PTHREAD is enabled to avoid blocking the main worker.

To maintain compatibility with non-pipefs streams, the select syscall allows stream implementations to ignore the callback and synchronously return the event status instead. In such cases, the select syscall still updates the flags accordingly.

@ktock ktock marked this pull request as draft October 8, 2025 03:53
@ktock ktock force-pushed the fixselect branch 3 times, most recently from 89b85b8 to 05bd5fd Compare October 8, 2025 18:24
@ktock ktock marked this pull request as ready for review October 8, 2025 23:23
@sbc100
Copy link
Collaborator

sbc100 commented Oct 16, 2025

Thanks for the PR @ktock. This looks like a fairly large and complex PR, and I'm out sick right now I'm afraid so I'm not sure when I will have time to go through it.

CC'ing @atrosinenko who wrote the original pipefs back in #4935.

Also @kripken and @tlively who might be interesting the multithreaded event stuff.

@ktock ktock force-pushed the fixselect branch 3 times, most recently from 5fce398 to 3938766 Compare October 24, 2025 02:48
@ktock
Copy link
Contributor Author

ktock commented Oct 24, 2025

Rebased. Could we move this forward?

@kripken
Copy link
Member

kripken commented Oct 30, 2025

I am unfortunately not familiar with this code (or with native pipes either). It looks like the PIPEFS code began in #4378 / #4935 by @cynecx and @atrosinenko - would one of you be interested to review this perhaps?

@ktock ktock force-pushed the fixselect branch 3 times, most recently from d7f75dc to 16b7865 Compare December 11, 2025 02:42
@ktock
Copy link
Contributor Author

ktock commented Dec 11, 2025

@sbc100 @kripken

This looks like a fairly large and complex PR

I've just simplified the patch by removing changes in libpthread.js. Instead, this patch uses emscripten_proxy_sync_with_ctx to synchronize the thread worker and the main worker. This patch allows pipefs and setTimeout to unblock the worker by calling emscripten_proxy_finish when an event occurs. I've added some wrapper functions for the proxying-related APIs so that the JS implementation of newselect can use them.

@ktock ktock force-pushed the fixselect branch 2 times, most recently from 5cef212 to ac6c045 Compare December 12, 2025 08:40
@ktock ktock force-pushed the fixselect branch 5 times, most recently from aaa6575 to 6079e85 Compare December 15, 2025 07:45
@ktock
Copy link
Contributor Author

ktock commented Dec 15, 2025

The failure in ci/circleci: test-mac-arm64 looks like unrelated to this PR:

This job was rejected because the resource class is unavailable

@sbc100
Copy link
Collaborator

sbc100 commented Dec 15, 2025

Circle CI should be fixed in #25955

sbc100 added a commit that referenced this pull request Dec 19, 2025
sbc100 added a commit to sbc100/emscripten that referenced this pull request Dec 19, 2025
After emscripten-core#25523 this function is no longer proxied itself and therefore
doesn't need to live in JS at all.
sbc100 added a commit to sbc100/emscripten that referenced this pull request Dec 19, 2025
After emscripten-core#25523 this function is no longer proxied itself and therefore
doesn't need to live in JS at all.
sbc100 added a commit to sbc100/emscripten that referenced this pull request Dec 19, 2025
After emscripten-core#25523 this function is no longer proxied itself and therefore
doesn't need to live in JS at all.
sbc100 added a commit to sbc100/emscripten that referenced this pull request Dec 19, 2025
After emscripten-core#25523 this function is no longer proxied itself and therefore
doesn't need to live in JS at all.
sbc100 added a commit to sbc100/emscripten that referenced this pull request Dec 19, 2025
After emscripten-core#25523 this function is no longer proxied itself and therefore
doesn't need to live in JS at all.
sbc100 added a commit to sbc100/emscripten that referenced this pull request Dec 19, 2025
After emscripten-core#25523 this function is no longer proxied itself and therefore
doesn't need to live in JS at all.
sbc100 added a commit that referenced this pull request Dec 19, 2025
After #25523 this function is no longer proxied itself and therefore
doesn't need to live in JS at all.
sbc100 added a commit to sbc100/emscripten that referenced this pull request Dec 19, 2025
These were added unnecessarily in emscripten-core#25523.
`_emscripten_proxy_newselect_finish` was added to
`create_pointer_conversion_wrappers` in emscripten.py which makes these
conversions automatic.
sbc100 added a commit that referenced this pull request Dec 19, 2025
These were added unnecessarily in #25523.
`_emscripten_proxy_newselect_finish` was added to
`create_pointer_conversion_wrappers` in emscripten.py which makes these
conversions automatic.
sbc100 added a commit to sbc100/emscripten that referenced this pull request Dec 20, 2025
This change improves the implementation of poll() to support blocking
when called from threads (see emscripten-core#25523) and moves the select-based-on-poll
implemenation from being wasmfs specific to be always being used.
@ktock
Copy link
Contributor Author

ktock commented Dec 22, 2025

There seem to be a couple of issues with the tests here.

Firstly I've seen some new flakes which I guess are timing related?

Secondly they seems to be failing under lsan/asan:https://logs.chromium.org/logs/emscripten-releases/buildbucket/cr-buildbucket/8695064874929935185/+/u/Emscripten_testsuite__ASan_/stdout

emcc: error: -pthread + ALLOW_MEMORY_GROWTH may run non-wasm code slowly, see WebAssembly/design#1271 [-Wpthreads-mem-growth] [-Werror]
[240/1180] None
None
test_select_blocking (test_core.asan.test_select_blocking) ... FAIL

maybe an easy fix?

Sorry for the slow reply, and thank you for the follow-up PRs. I've also seen the ongoing PR #25990 and added comments there as well. Please let me know if there is something I can do for these features.

sbc100 added a commit to sbc100/emscripten that referenced this pull request Dec 22, 2025
This change improves the implementation of poll() to support blocking
when called from threads (see emscripten-core#25523) and moves the select-based-on-poll
implemenation from being wasmfs specific to be always being used.
sbc100 added a commit to sbc100/emscripten that referenced this pull request Dec 22, 2025
This change improves the implementation of poll() to support blocking
when called from threads (see emscripten-core#25523) and moves the select-based-on-poll
implemenation from being wasmfs specific to be always being used.
sbc100 added a commit to sbc100/emscripten that referenced this pull request Dec 22, 2025
This change improves the implementation of poll() to support blocking
when called from threads (see emscripten-core#25523) and moves the select-based-on-poll
implemenation from being wasmfs specific to be always being used.
sbc100 added a commit to sbc100/emscripten that referenced this pull request Dec 22, 2025
This change improves the implementation of poll() to support blocking
when called from threads (see emscripten-core#25523) and moves the select-based-on-poll
implemenation from being wasmfs specific to be always being used.
sbc100 added a commit to sbc100/emscripten that referenced this pull request Dec 22, 2025
This change improves the implementation of poll() to support blocking
when called from threads (see emscripten-core#25523) and moves the select-based-on-poll
implemenation from being wasmfs specific to be always being used.
sbc100 added a commit to sbc100/emscripten that referenced this pull request Dec 22, 2025
This change improves the implementation of poll() to support blocking
when called from threads (see emscripten-core#25523) and moves the select-based-on-poll
implemenation from being wasmfs specific to be always being used.
sbc100 added a commit to sbc100/emscripten that referenced this pull request Dec 22, 2025
This change improves the implementation of poll() to support blocking
when called from threads (see emscripten-core#25523) and moves the select-based-on-poll
implemenation from being wasmfs specific to be always being used.
sbc100 added a commit to sbc100/emscripten that referenced this pull request Dec 22, 2025
This change improves the implementation of poll() to support blocking
when called from threads (see emscripten-core#25523) and moves the select-based-on-poll
implemenation from being wasmfs specific to be always being used.
sbc100 added a commit to sbc100/emscripten that referenced this pull request Dec 22, 2025
This change improves the implementation of poll() to support blocking
when called from threads (see emscripten-core#25523) and moves the select-based-on-poll
implemenation from being wasmfs specific to be always being used.
sbc100 added a commit to sbc100/emscripten that referenced this pull request Dec 22, 2025
This change improves the implementation of poll() to support blocking
when called from threads (see emscripten-core#25523) and moves the select-based-on-poll
implemenation from being wasmfs specific to be always being used.
sbc100 added a commit that referenced this pull request Dec 22, 2025
This change improves the implementation of poll() to support blocking
when called from threads (see #25523) and moves the select-based-on-poll
implemenation from being wasmfs specific to be always being used.
sbc100 added a commit to sbc100/emscripten that referenced this pull request Dec 22, 2025
This change improves the implementation of poll() to support blocking
when called from threads (see emscripten-core#25523) and moves the select-based-on-poll
implemenation from being wasmfs specific to be always being used.
sbc100 added a commit to sbc100/emscripten that referenced this pull request Dec 22, 2025
This change improves the implementation of poll() to support blocking
when called from threads (see emscripten-core#25523) and moves the select-based-on-poll
implemenation from being wasmfs specific to be always being used.
sbc100 added a commit to sbc100/emscripten that referenced this pull request Dec 22, 2025
This change improves the implementation of poll() to support blocking
when called from threads (see emscripten-core#25523) and moves the select-based-on-poll
implemenation from being wasmfs specific to be always being used.
sbc100 added a commit to sbc100/emscripten that referenced this pull request Dec 22, 2025
This change improves the implementation of poll() to support blocking
when called from threads (see emscripten-core#25523) and moves the select-based-on-poll
implemenation from being wasmfs specific to be always being used.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants