[fix] Announce the VM ready only once its QMP monitor answers - #229
Open
NimbleAINinja wants to merge 1 commit into
Open
NimbleAINinja wants to merge 1 commit into
NimbleAINinja wants to merge 1 commit into
Conversation
Launching intermittently failed with "The virtual machine started, but Try Omarchy could not enable safe Mac sleep ... cannot connect to QMP socket: Connection refused", most often on the first launch after installing a build, and the guest console log was empty afterwards. run-qemu-gpu.sh printed "Ready. QMP:" as soon as QEMU had created its socket files. QEMU creates those early in initialisation, but accepts monitor connections only from its main loop, which starts when initialisation finishes, and the socket listens with a backlog of one. The helper connected on Ready and waited two seconds for a greeting. On a slow start -- an 8 GiB guest, VirGL and ANGLE setup, cold caches after an install -- initialisation outlasted that wait. The connection had been queued by the kernel, and a queued connection stays queued when its client gives up, so the backlog was now full and every later connect was refused until the main loop ran. The helper reads that as a broken monitor and sends QEMU SIGTERM, which is why the console log was empty. The mechanism was reproduced against the shipped QEMU with a deliberately slow initialisation: one probe that gave up, four refused retries, then success the instant the main loop came up. Spacing the helper's retries cannot help; they retry into a full queue. Make Ready mean what its consumer assumes. A new sourced library waits until a connection to the QMP socket receives a greeting, retrying a refused connect or a silent monitor every 100 ms. A probe that gives up may itself occupy the backlog, and the main loop drains it as its first act, so the next probe gets through. It fails if QEMU exits or after 60 seconds, so a monitor that never answers still fails startup. The test drives the function against a stand-in that listens immediately but accepts only after a delay, the way QEMU does, and checks that Ready waits for the answer, does not hold up a monitor that answers at once, and reports a dead QEMU promptly; it also checks that the launcher consults the monitor before printing the Ready line. The two launcher contract tests stage the new library into their bundle and their fake QEMU now answers the monitor with a greeting, as QEMU does. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Launching intermittently fails with:
Most often on the first launch after installing a build; the guest console log is empty afterwards; reopening usually works. Seen on an M1 Max / macOS 27.
Cause
run-qemu-gpu.shprints[qemu-gpu] Ready. QMP: …as soon as QEMU has created its socket files. QEMU creates those early inqemu_init, but only accepts monitor connections from its main loop, which starts when init finishes — and the QMP chardev listens with a backlog of one (chardev/char-socket.c,qio_net_listener_open_sync(…, 1, …)).The helper connects on Ready and gives the greeting two seconds. On a slow start (8 GiB guest, VirGL/ANGLE setup, cold caches after an install) init outlasts that. The connection had already been queued by the kernel, and a queued connection stays queued when its client gives up, so the backlog is now full: every later connect returns
ECONNREFUSEDuntil the main loop runs. The helper reads that as a broken monitor and sends QEMUSIGTERM— hence the empty console log.Reproduced deterministically against the shipped QEMU with a deliberately slow init: one probe that gives up, then every connect refused, then success the instant the main loop came up. Pacing the helper's retries (#228) cannot help — they retry into a full queue — which is why that PR is closed in favour of this one.
Fix
Make Ready mean what its consumer assumes. A new sourced library,
macos/qemu-monitor-ready.sh, waits until a connection to the QMP socket receives a greeting, retrying a refused connect or a silent monitor every 100 ms. A probe that gives up may itself occupy the backlog, and the main loop drains it as its first act, so the next probe gets through. It fails if QEMU exits, or after 60 s, so a monitor that never answers still fails startup. The launcher calls it immediately before printing Ready. The helper is unchanged.Validation
make testpasses.macos/Tests/qemu-monitor-ready.test.shdrives the function against a stand-in that listens immediately but only accepts after a delay, the way QEMU does: Ready waits for the answer (≥ 1.4 s on a 1.5 s delay), does not hold up a monitor that answers at once, and reports a dead QEMU promptly. It also asserts the launcher consults the monitor before the Ready line.🤖 Generated with Claude Code