Conversation
Timing each worker in place on a quiet 32 core box showed a query asking for eight hands regularly getting seven. The last job of the batch would start after the other seven had finished the scan between them, and the six before it started in a ramp, each about a wakeup behind the one before. The pool pushed one job at a time, taking the queue lock and waking one worker for each. A wakeup is not addressed to a job, though. A worker that is already awake takes the job the notify was meant for, the notified worker finds the queue empty and parks again, and the job it should have taken sits there until somebody finishes and loops around. For a morsel scan that is the whole query, since a worker does not come back until the morsels run out. The batch now goes under one lock and one broadcast, so every parked worker wakes with jobs already queued and takes one. Spawning moved outside the lock while it was here: a spawn is long enough that holding the queue through it stalls the workers waiting to dequeue, which is the opposite of what spawning is for. The submitting thread also spins on the latch before it parks now. It is not a worker, so the reason workers park at once does not apply to it: it has just finished its own share and the hands it waits on are inside microseconds of finishing theirs, so the core it holds is one nobody wants. Parking there cost more than a tenth of a millisecond against queries that run in about one.
The spin was a Windows-only thing, on the reading that a parked thread wakes in microseconds everywhere else. Timing every worker in place said otherwise. On a quiet 32 core Linux box a batch of eight started in a ramp of about twenty five microseconds a worker, and the last one or two of them regularly did not arrive at all until the others had finished the scan between them, which is a query asking for eight hands and getting six. A tenth of a millisecond is not microseconds against a query that runs in one. With a hundred microseconds of spin every one of the eight is inside the scan and starts within a tenth of a millisecond of the first, and the stragglers are gone. A tenth of what Windows spins, because the wakeups here cost that much less and because a spinning worker is a worker not parked, which matters on a host with fewer cores than the query asked for.
This was referenced Aug 25, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The scale gate has been reading around 4.9x at eight workers on gamingpc against a milestone that wants six, and the usual suspects, fork cost and morsel granularity and the finish, all came back small when I timed them. So I timed the workers themselves instead of the query around them, printing when each one started, when it ended, and what the morsel claim stood at when it arrived.
The last of eight regularly did not arrive until the other seven had finished the scan between them, sometimes with the claim counter already past the last morsel, so it did nothing at all. The six before it started in a ramp about twenty five microseconds apart. A query asking for eight hands was getting six and a bit.
Two causes, both in the pool.
The batch was queued a job at a time, each push taking the lock and waking one worker. A wakeup is not addressed to a job. A worker that is already awake takes the job the notify was meant for, the notified worker finds the queue empty and parks again, and the job it should have taken sits there until somebody finishes and loops around, which for a morsel scan is the end of the query. The batch goes under one lock and one broadcast now, so every parked worker wakes with jobs already queued and takes one. Spawning moved out of the lock while I was in there, since a spawn is long enough that holding the queue through it stalls the workers waiting to dequeue.
The other is that the spin before parking was Windows only, on the reading that a parked thread wakes in microseconds elsewhere. On this box it does not. A hundred microseconds of spin, a tenth of what Windows uses, puts all eight inside the scan and within a tenth of a millisecond of the first. The submitting thread spins on the completion latch too, for a different reason: it is not a worker, it has just finished its own share, and the hands it waits on are inside microseconds of finishing theirs, so the core it holds is one nobody else wants.
Measured on gamingpc under WSL, medians of nine paired alternating runs of two separately built binaries. The expand query went 4.9x to 5.4x at eight workers, 1.12 ms to 1.02, and the branch was faster in all nine pairs. The scan query beside it went 4.2x to 5.3x, 0.80 ms to 0.67, faster in eight of nine with one tie. One worker times moved about 2 percent the wrong way on both, which is a single worker run not going near the pool at all. The group by bench is unmoved either way.
The floor stays at 2.0. It is set by server3's worst run on eight shared vCPUs, not by this box.
There is a test now that every job in a batch gets a worker of its own: each job counts itself in, waits for the rest, and counts itself out, so the high water mark is how many the pool had running at once, and a batch that leaves one job queued while another worker runs two never reaches it.
Part of P2, zu#75.