`for await (const chunk of process.stdin)` intermittently stalls forever on piped input ≥ ~1 MiB (4 MiB+ hangs ~always)

[proggeramlug]

Summary

for await (const chunk of process.stdin) on a pipe intermittently stops delivering chunks and never resumes. The process sits idle (0% CPU) with input still unread, the writer blocks, and nothing ever completes. It is a race: 1 MiB stalls about half the time, 4 MiB and up essentially always. 64 KiB is fine.

Repro

// rd.ts
import * as process from "node:process";
async function main(): Promise<void> {
  let total = 0, chunks = 0;
  for await (const data of process.stdin) { total += data.length; chunks++; }
  console.log("total", total, "chunks", chunks);
}
main();

# drive.py — writes N bytes to stdin, closes it, waits 12 s
import subprocess
def run(cmd, size):
    p = subprocess.Popen(cmd, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
    try: out, err = p.communicate(bytes([1]) * size, timeout=12); st = 'done'
    except subprocess.TimeoutExpired: p.kill(); out, err = p.communicate(); st = 'HANG'
    return f'{st} {out.decode().strip()}'
for size in (65536, 1 << 20, 3 << 19, 2 << 20, 4 << 20):
    print(size, run(['./rd'], size), '|', run(['./rd'], size))

perry 0.5.1618 (b9ba951ff8, macOS arm64, 16 KiB pipe chunks), two runs per size:

bytes	run 1	run 2
65,536	done, 7 chunks	done, 7 chunks
1,048,576	HANG	done, 170 chunks
1,572,864	done, 239 chunks	HANG
2,097,152	done, 323 chunks	HANG
4,194,304	HANG	HANG
8,388,608	HANG ×3

node 24 on the same driver: done total 8388608 chunks 8261.

When stalled, a sample shows the main thread in pthread_cond_wait and the stdin reader thread in read(). Logging per chunk shows delivery simply stops part-way (e.g. at 890,364 of 2,097,151 bytes), so it looks like a lost wakeup / pause-resume hand-off between the reader thread and the loop rather than EOF handling. Linux not tested yet.

Also seen downstream, not yet isolated (may be separate bugs)

Same stdin path inside the Native Messaging host from https://github.com/guest271314/NativeMessagingHosts (each chunk copied into one large Uint8Array with .set, then echoed with stdout.write):

• 2 MiB message: one run hung, one run finished but emitted a frame whose 4-byte length header was garbage (4022190063) — i.e. corrupted output, not just a stall.
• 64 MiB message: SIGSEGV (rc −11) after ~75 ms on one run, hang on another.

Those need their own reduction once the stall is out of the way; noting them here so they are not lost.

Impact

Any CLI / filter / Native Messaging host that reads more than about a megabyte from a pipe. Browser→host Native Messaging messages go up to 64 MiB.

Related: #10873 and the Uint8Array<ArrayBuffer> indexOf issue (same program). Context: jlucaso1/js-compiled#2

[proggeramlug]

Classification: a regression, not macOS-specific — and root-caused.

build	host	hangs
published 0.5.1520	Linux x86_64	0 / 15 (64 KiB … 64 MiB × 3)
main 841b605c97 (v0.5.1632)	Linux x86_64	1 / 350 at 16 MiB; 0 / 20 at 64 MiB; 0 / 90 at ≤ 4 MiB
main 841b605c97 (v0.5.1632)	macOS arm64	3 / 6 at 1 MiB, 6 / 6 at 4 MiB, 5 / 8 at 8 MiB fed in 256-byte writes

A stalled process has no reader thread at all — sample shows only the main thread, parked in js_wait_for_event. (The "reader thread in read()" in the issue text was a different, healthy process.)

Root cause (crates/perry-runtime/src/os_process_streams.rs, ensure_stdin_reader): the async iterator pauses its source after every delivered chunk and resumes it on the next pull (node_stream/async_iterator.rs, ns_readable_iter_on_data / ns_readable_iterator_next). On process.stdin, pause() latches STDIN_DETACHED — the fd-0 reader exits when it sees the latch at the top of its loop — and resume() clears the latch and respawns the reader unless STDIN_READER_STARTED is still true. The reader's stop decision and its STARTED reset (a drop guard) were separate steps:

reader: sees STDIN_DETACHED == true, decides to exit
main:   resume() → STDIN_DETACHED = false; CAS(STARTED false→true) FAILS — the dying reader has not cleared it yet
reader: clears STARTED, gone

fd 0 then has no reader, while js_readline_has_active still reports a started, un-paused, flowing stdin — so the loop stays alive and idles forever. One roll of that dice per delivered chunk: ~170 chunks/MiB through a 16 KiB macOS pipe fed in 512-byte writes, which is why 1 MiB is a coin flip there and 4 MiB is near-certain; a 64 KiB Linux pipe delivers ~16 chunks/MiB and the window is a handful of instructions, so it needs preemption to land (the one Linux hang was under load average ~30).

Introduced by bb57392 (2026-09-04, "unify process.stdin reader surface"). Before it, a piped stdin was read by perry-stdlib readline's own reader, which never consulted the latch — that is why 0.5.1520 is clean.

Fix + tests coming in a PR. The "garbage frame header" noted at the bottom of this issue is a separate, deterministic bug — process.stdout.write(bytes) UTF-8-decodes binary chunks — now #10903.

[proggeramlug]

Verdict on the two "also seen downstream" items at the bottom of this issue — neither is the stall, neither is GC:

• Garbage frame header (4022190063) — deterministic: process.stdout.write(Uint8Array) UTF-8-decodes the chunk, so F7 FF 0F 00 goes out as EF BF BD EF …. Tracked separately in process.stdout.write(bytes) is not binary-safe: a Buffer/Uint8Array chunk is UTF-8 decoded, every invalid byte is written as EF BF BD #10903.
• 64 MiB SIGSEGV — deterministic: it is Uint8Array<ArrayBuffer>-annotated receiver: indexOf / lastIndexOf silently return -1 (plain Uint8Array, any and Float32Array<ArrayBuffer> are correct) #10894. The Uint8Array<ArrayBuffer> annotation sends message.indexOf(COMMA, from) to the plain-Array js_array_indexOf_jsvalue, which reads the byte buffer as 8-byte slots and runs off the end (backtrace + a 6-line repro that crashes at 2 MiB are on Uint8Array<ArrayBuffer>-annotated receiver: indexOf / lastIndexOf silently return -1 (plain Uint8Array, any and Float32Array<ArrayBuffer> are correct) #10894).

With the stall fixed (#10913) and the receiver annotated plain Uint8Array, the host's read side is clean: 1 MiB … 64 MiB, 0 hangs / 0 crashes in 220 runs on macOS.