diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md index d1e8f0905..f35201ad6 100644 --- a/.github/copilot-instructions.md +++ b/.github/copilot-instructions.md @@ -492,8 +492,7 @@ the same time, so both halves of the evidence disappear together. This is not a small correction. Measured on two runs here: a `windows-topology-sys` sweep reported 61 survivors of which **57 were in `#[cfg(feature = "serde")]` code**, and a `windows-file-watcher` sweep reported 247 of which **147 were in `scenario-tool` and -`test-util` modules**. In both cases roughly 60% of the "gaps" were artifacts of the -invocation. So: +`test-util` modules**. In both cases the gated code dominated the survivor list. So: ``` cargo mutants -p --all-features @@ -1182,7 +1181,7 @@ hypothetical: across three consecutive review rounds on PR #42, *five of six fin corrections that had not propagated* rather than original defects. See [DESIGN-NOTES.md](../DESIGN-NOTES.md) -> [Restatement drift](../DESIGN-NOTES.md#restatement-drift). -Three rules follow, ordered by how little each depends on anyone remembering. +Five rules follow, ordered by how little each depends on anyone remembering. ### 1. Prefer a derived fact to a restated one @@ -1271,6 +1270,155 @@ Two corollaries that have each already cost a review round: written while the old reading was current — generators, test doubles, examples — because those encode the reading rather than citing it. +### 4. Prose carries the claim; a number belongs in an artifact + +Measured data pasted into prose becomes a copy somebody must keep true by hand, in every place it +was pasted, forever. Markdown has no include at all; rustdoc has one, and this repository uses it -- +`windows-waitable-queues`' [lib.rs](../crates/windows-waitable-queues/src/lib.rs) splices its whole +README in with `#[doc = include_str!("../README.md")]`. What neither has is a way to pull a *single +measured value* out of a data file and into a sentence: the include is whole-file or nothing, so a +figure quoted mid-paragraph must be typed there. That is why pasting is the path +of least resistance — and it is where this repository's documentation defects overwhelmingly come +from. Measured on one pull request's review history: almost none of its measurement findings were +*wrong measurements*; they were transcriptions that drifted — a table disagreeing with its own copy +one file away, an attribution naming a superseded capture, one horizon left unqualified across seven +sites in three wordings. + +- **Write the claim, not the digits, wherever the digits are not the point.** "Measured faster under + contention, over a spread that overlaps the same-code control at every producer count" carries no + transcribed figure, so it cannot drift *from* the artifact the way a pasted number does. It can + still be made false by a retake -- so **it links the artifact**, and a reader who follows the link + can settle it. Omitting digits removes the transcription failure, not the obligation to cite. +- **When a figure must appear, it has exactly one home.** Prefer a committed capture the prose links + to ([mutation-sweeps/2026-09-02/](../mutation-sweeps/2026-09-02) is this repository's existing + example) over the same figure typed + into two documents. Provenance — host, commit, date — travels with the data rather than in a + hand-maintained table beside it. +- **Never restate a proportion over data you already showed.** A ratio over counts in the same + document is not a finding; it is a hand-computed copy of one, checked by nobody and stale the + moment any input moves. The counts are the finding. This rule was earned: an instructions file in + this repository claimed "in both cases roughly 60%" about two figures given four words earlier, + neither of which rounded to it. +- **The same applies to incidental tallies** — test counts, file counts, line counts. If the number + is not itself the finding, leave it out; "the gate is green" says what "308 lib tests" pretends to. + +**This is the data-side twin of rule 1.** Rule 1 says define a fact once in code and have everything +ask. This says the same of measurements: hold the number once, and have prose point rather than +paraphrase. + +### 5. Present what was observed; never write the conclusion + +Rule 4 governs where a number lives. This governs whether you state what it *means*. Give the +figures and the mechanism. Stop. Do not tell the reader what follows for them. + +**Where it applies is syntactic, so there is nothing to adjudicate.** It governs `.md` files and +rustdoc — `///` and `//!`. It does not govern an ordinary `//` comment. + +That split is about purpose, not about how public the text happens to be. **Rustdoc is not a code +comment; it is the `.md` colocated with the code** — the same published prose, read by the same +people, deciding whether to adopt the thing. A `//` comment exists to carry the *developer's +mind-state* to whoever edits that line next: why this was done this way, what was uncertain, what +was traded. `// effectively unreachable` there is not a lapse to be tolerated — **conveying a +judgment, hedges and all, is that comment's whole job**, and flattening it into a bare figure would +destroy the thing it was written to preserve. + +An earlier draft of this rule got the boundary wrong twice: first exempting "code comments" as a +single category, then withdrawing the exemption on the grounds that rustdoc made the boundary +undecidable. It is decidable, and it is the slash count. + +The failure does not look like an error, which is why it survives review. It reads as helpfulness: + +- A table gives one layout a horizon of years and another a horizon hundreds of times longer at the + same rate; the paragraph above it says the shorter one "reaches the same practical headroom a + 128-bit word gives." Nothing is inconsistent — + the prose has simply decided, on the reader's behalf, that a factor of that size does not matter to + them. +- A table gives a layout a horizon of months; the prose calls it "the first row that is not + reachable." +- **Flipping the verdict is not the fix.** Replacing "not reachable" with "reachable by a busy + long-lived process" is the same move with the opposite conclusion. Delete the conclusion, do not + correct it: *"every row recurs; what changes down the column is how long that takes at a given + rate — the table's own figures say how much."* + +Three words are the usual tell, and each is a conclusion wearing a measurement's clothes: +**practical**, **effectively**, **reachable**. So are "enough", "negligible", "safe to", and any +sentence whose subject is the reader. In `.md` and rustdoc they mark a violation; in a `//` comment +they mark a developer saying what they thought, which is wanted. + +This is **not a new rule** — it is [D-no-client-prescriptions](../crates/windows-platform-probes/DESIGN-NOTES.md#d-no-client-prescriptions) +("state what was observed … stop there"), stated once for the repository rather than once for the +crate that happens to publish measurements. Every instance found so far has been a violation of that +existing decision rather than a gap in it. Apply it while writing: no checker can find these, +because nothing is inconsistent. + +## REVIEW FEEDBACK — answer it where it was raised, not only in the commit + +**A review round is not finished when the code changes. It is finished when the reviewer has +been told what happened.** Fixing the code and pushing is half the transaction; the other half +is a reply on GitHub, and omitting it is the default failure mode because the fix *feels* like +completion. It is not, for three reasons: + +- **A commit is not an answer.** The reviewer sees a new SHA, not your reasoning. Nothing + connects "I changed `format_ratio`" to the finding that asked for it, so the next round + re-raises what was already addressed — which has repeatedly cost rounds on this repository. +- **Some findings are correctly declined, and silence cannot say so.** A declined finding that + is never answered is indistinguishable from one that was missed. Declining is legitimate; + declining *silently* is not. +- **Suppressed comments have no thread at all.** They arrive in the review summary rather than + attached to a line, so there is no place a reply can land by default and no automatic record + that they were read. They are the easiest feedback to drop and the most likely to be re-raised + verbatim in the next round. + +### What to do, by where the feedback lives + +- **Inline review comments (a thread on a line).** Reply *on that thread*, naming what changed + and the commit SHA that changed it. Then resolve the thread — but only if the finding is + genuinely discharged; never resolve to clear the queue. Use the `resolveReviewThread` tool, or + `gh api repos/{owner}/{repo}/pulls/{number}/comments/{comment_id}/replies -f body=...`. +- **Suppressed comments, review-summary findings, and anything pasted to you out of band** — no + thread exists, so post **one new PR comment** covering that round: + `gh pr comment --body-file .scratch/.md`. One comment per round, not one per + finding; a reviewer reads the round as a unit. +- **No PR** (work committed straight to a branch, or feedback on a commit): + `gh api repos/{owner}/{repo}/commits/{sha}/comments -f body=...` against the commit that + carries the response. + +### What the response must contain + +Every finding in the round gets a line, and each line is one of exactly two things: + +1. **Changed** — what was changed and the SHA. Where the fix was a *sweep* rather than a + single-line edit (per CONTRACT INTEGRITY rule 3 above), say so and give the count: "swept + `QueueFull`: 13 files, 4 updated". A reviewer who sees only the cited line fixed has no way + to know the population was covered. +2. **Declined** — the argument for why, in enough detail to be argued back against. "Not + applicable" is not an argument; "this is gated behind `test-util`, so the mutant sits in code + the shipping build never compiles" is. + +Two further rules, each of which has already cost a round here: + +- **Do not claim a fix you have not verified.** The same standard applies as anywhere else in + this file: verify by execution. Where the fix was a test, say what sabotage showed it is + load-bearing — an unverified "added a test" is exactly the cosmetic binding CONTRACT INTEGRITY + rule 1 warns about. +- **Report what the round taught, not just what it touched.** When a review round reveals that + several findings were one underlying error, say that — it is more useful to the reviewer than + five separate acknowledgements, and it is how a recurring defect gets named instead of + repeatedly re-fixed. + +### The PR description drifts too, and nothing greps it + +A PR body is prose that restates measured claims, gate results, and design rationale — so it +rots exactly like the documents CONTRACT INTEGRITY governs, with one difference: **it is not a +file in the tree, so no sweep, grep, or CI check will ever catch it.** When a round corrects a +claim, check whether the PR description states the same claim, and correct it in the same round. + +Keep out of the PR body anything that drifts without carrying information. **Test counts are the +standing example**: "308 lib tests" changes on almost every commit, tells a reader nothing that +"tests pass" does not, and creates a restatement-drift instance out of nothing. State that the +gate is green and which parts of it ran; do not enumerate. The same goes for file counts, line +counts, and any other incidental tally that is not itself the finding. + ## CHECKLIST file hygiene CHECKLIST files are **action-only**: they contain pending, in-progress, and recently diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 117c87126..fb18cf5dc 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -500,6 +500,54 @@ jobs: RUSTDOCFLAGS: "-D rustdoc::broken_intra_doc_links -D rustdoc::private_intra_doc_links" run: cargo doc -p windows-placement-probe --no-deps --no-default-features --locked + waitable-queues-default-features: + name: windows-waitable-queues (default features) + runs-on: windows-latest + # **This job exists because adding `probe-queue-contention` took the default + # configuration away from every other job.** That probe needs `dwcas` and + # `experimental-permit-claim`, and enabling them on a workspace member + # unifies them across the whole workspace -- so the `--workspace` steps that + # deliberately omit `--all-features` stopped being a default-features build + # of THIS crate, and no job was left compiling it without `dwcas`. + # + # That matters because `dwcas` is additive: `Wide` and its `ClaimLayout` impl + # exist only under it, so the configuration that loses them is the one that + # can break unnoticed. The crate's manifest says `dwcas` is "non-default so + # nothing depends on it by accident" and is "the only thing in this crate + # that costs a third-party dependency" -- a claim that is only true while + # something still builds without it. + # + # Named per-crate rather than fixed by making the probe's dependency + # optional: the probe must stay buildable by a plain `cargo build`, and a + # feature that has to be remembered before the probe compiles is a worse + # trade than a job that cannot be forgotten. + env: + RUSTUP_TOOLCHAIN: stable + steps: + - uses: actions/checkout@v7 + - uses: dtolnay/rust-toolchain@stable + with: + components: clippy + - uses: Swatinem/rust-cache@v2 + - name: cargo build + run: cargo build -p windows-waitable-queues --all-targets --locked + - name: cargo clippy + run: cargo clippy -p windows-waitable-queues --all-targets --locked -- -D warnings + - name: cargo test + env: + RUST_BACKTRACE: 1 + RUST_LIB_BACKTRACE: 1 + run: cargo test -p windows-waitable-queues --locked --no-fail-fast + # The `docs` job documents `--all-features` only, so a link into a + # `dwcas`-gated item resolves there and dangles in this configuration. + # `private_intra_doc_links` is denied here for the same reason it is in the + # placement-probe job: locally it is only a warning, so nothing but a deny + # catches a public item documented against a private one. + - name: cargo doc (deny broken intra-doc links) + env: + RUSTDOCFLAGS: "-D rustdoc::broken_intra_doc_links -D rustdoc::private_intra_doc_links" + run: cargo doc -p windows-waitable-queues --no-deps --locked + fmt: name: rustfmt runs-on: windows-latest @@ -571,6 +619,37 @@ jobs: - name: cargo check (MSRV) run: cargo check --workspace --all-targets --all-features --locked + dwcas-gating: + name: probes build where a 128-bit exchange is unavailable + runs-on: windows-latest + # See build-test's comment: forces the floating stable toolchain over the + # repo's rust-toolchain.toml pin. + env: + RUSTUP_TOOLCHAIN: stable + steps: + - uses: actions/checkout@v7 + - uses: dtolnay/rust-toolchain@stable + with: + targets: i686-pc-windows-msvc + - uses: Swatinem/rust-cache@v2 + # `windows-platform-probes` takes the queue crate's `dwcas` feature from + # two `[target.'cfg(...)']` tables, so the manifest predicate and the + # source's `cfg(feature = "dwcas")` gating have to agree. Nothing else in + # CI builds a configuration where they can disagree: every other job is + # x86-64 or aarch64 with `cmpxchg16b` on, where the feature is always + # enabled and the ungated path is never compiled. + # + # These two are the configurations the gating exists for. A 32-bit target + # has no `AtomicU128` at all, and `-C target-feature=-cmpxchg16b` removes + # it from x86-64 -- each was verified by hand when the gating was written, + # which is precisely the kind of check that stops being run. + - name: cargo check (i686, no 128-bit atomic) + run: cargo check -p windows-platform-probes --all-targets --target i686-pc-windows-msvc --locked + - name: cargo check (x86-64 with cmpxchg16b disabled) + env: + RUSTFLAGS: "-C target-feature=-cmpxchg16b" + run: cargo check -p windows-platform-probes --all-targets --target x86_64-pc-windows-msvc --locked + alloc-only: name: wtf-string (alloc-only, no_std) runs-on: windows-latest diff --git a/Cargo.lock b/Cargo.lock index 539b402e0..76ddfb335 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -224,6 +224,7 @@ dependencies = [ "windows-sys", "windows-threadpool-sys", "windows-topology-sys", + "windows-waitable-queues", "wtf-string", ] diff --git a/DESIGN-NOTES.md b/DESIGN-NOTES.md index 2ce928d50..bcb098811 100644 --- a/DESIGN-NOTES.md +++ b/DESIGN-NOTES.md @@ -1859,3 +1859,52 @@ written. The rule is about **discarded failure information**, not about discarde The audit this decision implies is queued as [CHECKLIST.md](CHECKLIST.md) -> `M22.1`; it is not scheduled by this note alone. +## Prose volume is not the error surface; restatement count is + +**Decided: the error surface is proportional to how often a fact is restated, not to how much prose +carries it.** A uniform cut to the prose leaves every restatement in place, just in fewer words. +What is watched is therefore the number of hand-maintained copies of a single fact, not a word or +line count -- and no prose-to-code ratio is quoted here, because this note takes no position on one. + +**Decided: prose carries the claim; an artifact carries the number.** + +- **A claim belongs in prose.** "`reserving_mpsc` measured faster than `slotwise_mpsc` under + contention, over a spread that overlaps the same-code control at every producer count" is a + claim. It transcribes no figure, so it cannot drift from the artifact the way a pasted number + does -- but it is not thereby permanent: a retake can make it false, and a reader cannot tell from + the sentence alone. That is why the claim cites the artifact. Dropping the digits removes the + transcription failure and leaves the citation obligation exactly where it was. +- **A number belongs in an artifact.** A measured cost, a capture's commit, a count of occurrences: + one copy, with its provenance travelling *with* it rather than in a hand-maintained attribution + table beside it. +- **A proportion over data we hold is not a finding, it is a restatement of one.** Computed by hand, + checked by nobody, and stale the moment any input moves. The counts are the finding. A reader who + wants a ratio can take one, against a denominator they chose and at a moment they know. + +[`mutation-sweeps/2026-09-02/`](mutation-sweeps/2026-09-02) and +[`crates/windows-platform-probes/captures/2026-09-16-drained-handshake/`](crates/windows-platform-probes/captures/2026-09-16-drained-handshake/README.md) +are the worked examples: dated, committed capture directories carrying raw runs, the script that +derives the summary, and its output. The seven-run sweep the variance argument rests on still has no +committed capture, so the gap is narrowed rather than closed. + +**Decided: formal specification and restatement control address different classes, and neither +substitutes for the other.** TLA+ and `loom` +([D-31](crates/windows-waitable-queues/DESIGN-NOTES.md#d-31)) target algorithm properties; +restatement control targets documented facts. A formal spec's most useful property here is not proof +-- it is that prose can point at it instead of paraphrasing it, which is +[restatement drift](#restatement-drift)'s first remedy applied one level up. + +Neither of those instruments has been run, and neither is scheduled: `D-31` records the `loom` +verification as planned and several documents name `M31.6` as its owner, but no checklist contains +that item -- `windows-waitable-queues` has an archive, +[COMPLETED-CHECKLIST.md](crates/windows-waitable-queues/COMPLETED-CHECKLIST.md), and no open +checklist at all. + +**Decided: the cut is to restated assertions, never to rationale.** Rationale is what makes a +decision re-checkable years later; cutting it to hit a volume target would remove the only prose +that has never been wrong while leaving the prose that keeps being wrong in proportion. + +**No work is scheduled by this note**, per "design notes are not a work queue". How the measurement +was taken, which findings it is drawn from, what was rejected on the way, and the one cheap remedy +that was costed but not adopted are in +[DESIGN-RATIONALE.md](DESIGN-RATIONALE.md#why-restatement-count-is-what-is-watched). diff --git a/DESIGN-RATIONALE.md b/DESIGN-RATIONALE.md index 4b6d3b18d..cce9fa325 100644 --- a/DESIGN-RATIONALE.md +++ b/DESIGN-RATIONALE.md @@ -205,6 +205,242 @@ following the rule that a binding which cannot be shown to fail is cosmetic. Fiv mutations -- three manifest values, a deleted claim, and a stale version planted in prose -- each produce a distinct, located failure. +## Why a measured figure is asked to have one home + +[DESIGN-NOTES.md](DESIGN-NOTES.md#prose-volume-and-error-surface) records the rule; this is how it +was reached, and what was rejected on the way. + +The evidence was a review history, not an argument. Across the rounds on PR #90, most findings were +not wrong measurements -- they were transcriptions that had drifted from the thing they restated: a +table disagreeing with its own copy, a control quoted for the wrong regime, a wrap horizon stated in +minutes that the crate's own published rate contradicts. The measurements were fine. The +copies were not. + +Two weaker rules were considered and rejected. **"Keep the copies in sync"** is what had already +been happening, and the failure mode is that nothing enforces it; every drifted figure on that +branch was written by someone intending to keep it in sync. **"Never publish a figure"** fails the +other way: a caller choosing a layout needs a number, and hiding it behind a link that may not be +followed trades one failure for another. What survived is narrower -- the *figure* lives in a +committed artifact, and prose carries the *claim* plus a link to it. + +A correction from review is recorded with the rule itself: dropping the digits does **not** make a +claim permanent. A qualitative sentence cannot suffer transcription drift, because it transcribes +nothing, but a retake can still falsify it and a reader cannot see that from the sentence. So the +citation obligation is unchanged by the wording; only the transcription failure is removed. An +earlier draft of the rule said the digit-free form "cannot drift", which overstated it. + +The mechanism -- how a figure gets from an artifact into rendered prose -- is deliberately left +open; markdown has no include, and rustdoc's include is whole-file. That is stated in the decision as an +unsettled trade rather than resolved here, and no work is scheduled against it. + +## Why restatement count is what is watched + +[DESIGN-NOTES.md](DESIGN-NOTES.md#prose-volume-and-error-surface) records the decision. This is how +it was reached, what was rejected on the way, and the remedy that was costed but not adopted. It was +moved here from that file, where it had been written inline: Tier 1 is the current decision, and a +section carrying its own motivating question, census procedure and superseded drafts had made the +decision harder to find inside it. + +[Restatement drift](#restatement-drift) explains the mechanism and gives the remedy. This note +records something that section does not: a measurement of **where** the drift actually lives, taken +after PR #90's eighteenth review round, and what follows from it about formal specification. + +The question that prompted it was whether this repository simply says too much -- whether English, +which must be inexact to serve human readers, is being asked to carry a specification load it cannot +bear, and whether some formal specification plus substantially less prose would shrink the error +surface. + +### The measurement, and why it is not written down here + +Prose volume was looked at first and set aside: whatever the ratio of prose to code is here, it is +not what the findings track. That ratio is deliberately not quoted, because quoting a measurement +this section takes no position on would be an uncited figure inside the argument against uncited +figures. + +The thing to watch is that in `windows-waitable-queues`, a handful of single facts -- `Perpetual`'s +reservation-count ceiling, `Balanced`'s recurrence horizon, `Perpetual`'s position span, `Balanced`'s +field ceiling -- are each restated many times across several files, by hand, with nothing checking +any of them. Which of them has the most copies was counted once, during the review rounds that +produced this section, and has not been counted since; no census is committed, so that ordering is +recorded here as a historical observation rather than a current fact. + +**The exact counts are deliberately not recorded here.** An earlier version of this section carried +them as a table, and the table drifted within days: one row gained an occurrence when a qualifier was +added to a rustdoc elsewhere in this same branch, so the census of restatements became a restatement +that needed maintaining. That is the section's own subject, demonstrated on the section. + +Anyone who wants current numbers can compute them, which is the point of the principle below -- the +counts are a finding, and a finding should be computed rather than quoted: + +```powershell +# Occurrences of a figure across the crate, and how many files carry it. +$files = git ls-files 'crates/windows-waitable-queues/*' | + Where-Object { $_ -match '\.(rs|md|toml)$' } +foreach ($pattern in '\b255\b', '37 seconds', '2\^56', '4,294,967,295', 'about 20 years') { + $hits = 0; $carrying = 0 + foreach ($file in $files) { + $n = ([regex]::Matches([System.IO.File]::ReadAllText($file), $pattern)).Count + if ($n) { $hits += $n; $carrying++ } + } + "{0,-16} {1,3} occurrences across {2} files" -f $pattern, $hits, $carrying +} +``` + +**All of these are restated by hand with nothing checking them.** Three of those facts -- the +ceiling, the span and the field ceiling -- follow from `ClaimLayout`'s associated constants. The +time figures follow from a field width *and* an assumed sustained push rate, so a +constants-versus-table check would validate the constant-derived facts outright and the time figures +only once the +rate is pinned somewhere single. That distinction bounds what the cheapest remedy below can do -- an +earlier version of this paragraph said every one was derivable from the constants, which overstated +it, in a note about overstatement. The error surface is proportional to how often a fact is restated, +not to total prose volume: a uniform cut to the prose leaves every restatement in place, just in +fewer words. + +### Which errors this predicts, and which it does not + +Sorting PR #90's findings across all rounds by class: + +- **Restated derivable facts** -- the `2^31`/`2^30` target-dependent capacity, `MAX_RESERVED` + conflated with capacity, "`Wide` removes it" for a bound that is finite, stale recurrence tables, + a test count that matched no crate, the same horizon left unqualified across seven sites, a + withdrawn magnitude surviving in two public rustdocs. +- **Structural** -- an unmarked supersedence row in a decision index, an orphaned milestone + reference. A linter's job, not a specification's. +- **Evidence overclaiming** -- a noise floor computed from two runs, a refusal-count argument that + did not reproduce in direction or magnitude across three re-measurements. These were the most + valuable findings of the whole PR, and *more* measurement is what fixes them, not less prose. +- **Policy** -- client prescriptions surviving [D-no-client-prescriptions](crates/windows-platform-probes/DESIGN-NOTES.md#d-no-client-prescriptions). + Only a reviewer catches these. +- **Algorithm properties** -- **zero findings, in any round.** + +That last line is the one to be careful with, because it has two readings and only the second is +honest. There are no findings in that class because **there is no instrument for it**, not because +the algorithms are known good. `SH-14.1` is a live, known defect in the claim protocol; it was found +by a person reasoning carefully, and nothing in the toolchain would have caught it. Absence of +findings where nothing looks is not evidence of correctness -- the same error this repository has +corrected in its own measurements more than once. + +### What follows + +Three conclusions, of which the middle one is the one that changes practice. + +**Formal specification and prose reduction address different classes.** TLA+ and `loom` +([D-31](crates/windows-waitable-queues/DESIGN-NOTES.md#d-31)) target algorithm properties. Neither +has been run, and neither is scheduled: `D-31` records the `loom` verification as planned, and +several documents name `M31.6` as its owner, but no checklist contains that item -- +`windows-waitable-queues` has an archive, +[COMPLETED-CHECKLIST.md](crates/windows-waitable-queues/COMPLETED-CHECKLIST.md), and no open +checklist at all. So what can be said about that class is +that it produced no findings in any review round of PR #90 +while carrying one known unfound defect, which is a statement about the reviews rather than a +result from either instrument. Restatement targets documented facts, +which have produced most findings. Both are worth doing; conflating them would aim the expensive +instrument at the cheap problem. + +**The cut must be to restated assertions, not to rationale.** No finding in any round of PR #90 was +against a passage explaining *why* a decision was made. The findings were against duplicated +*assertions* of fact, against overclaims from evidence, and against prescriptions. Rationale is what +makes a decision re-checkable years later and is the reason this file exists at all; cutting it +uniformly to hit a volume target would remove the only prose that has never been wrong, while +leaving the prose that keeps being wrong in proportion. + +**A formal spec's most useful property here is not proof -- it is that prose can point at it instead +of paraphrasing it.** That is [restatement drift](#restatement-drift)'s first remedy applied one +level up: define the protocol once in a form that can be checked, and let every document cite it. +This is the real connection between the two ideas, and it is why they belong in the same +conversation despite fixing different things. + +### Prose carries the claim; an artifact carries the number + +The sharper question, asked after several rounds of the above: **why is measured data living in +prose at all?** + +There is no principled reason. It is an accident of what is easy. Markdown has no include and +rustdoc has no data include, so the only way to put a figure in front of a reader is to paste it -- +and a pasted figure is a copy somebody must keep true by hand, in every place they pasted it, +forever. + +The cost is measurable in this PR's own review history. Almost none of its measurement-related +findings were *wrong measurements*. They were **transcription failures**: the same table in the +README and the crate rustdoc disagreeing because one was retaken; an attribution naming a capture +the figures no longer came from; one recurrence horizon left unqualified across seven sites in three +wordings; a withdrawn magnitude surviving in two public rustdocs. The most instructive was a +proportion that restated two counts **given four words earlier in the same sentence** and got one of +them wrong -- it said "in both cases roughly 60%" where one of the two cases was 57 of 61. The data +was adjacent and the summary of it was false, because prose is not checkable and nobody checks it. + +**This repository already contains the better pattern, and this branch was the first to apply it +in the probe crate.** +[`mutation-sweeps/2026-09-02/`](mutation-sweeps/2026-09-02) is a dated, committed capture directory: data as an artifact, cited +rather than retyped. `windows-platform-probes` produces the most-cited numbers in the workspace and +committed no capture at all when this section was written -- every figure it had published reached +its document by hand. The re-measurement that `M4.3` forced is the first exception: +[`crates/windows-platform-probes/captures/2026-09-16-drained-handshake/`](crates/windows-platform-probes/captures/2026-09-16-drained-handshake/README.md) +commits the raw runs, the script that derives the summary, and its output. The seven-run sweep that +the variance argument rests on still has no committed capture, so the gap this section describes is +narrowed rather than closed. + +So the principle, which holds regardless of which mechanism is eventually chosen: + +- **A claim belongs in prose.** "`reserving_mpsc` measured faster than `slotwise_mpsc` under + contention, over a spread that overlaps the same-code control at every producer count" is a + claim. It transcribes no figure, so it cannot drift from the artifact the way a pasted number + does -- but it is not thereby permanent: a retake can make it false, and a reader cannot tell from + the sentence alone. That is why the claim cites the artifact. Dropping the digits removes the + transcription failure and leaves the citation obligation exactly where it was. +- **A number belongs in an artifact.** A measured cost, a capture's commit, a count of occurrences: + one copy, with its provenance travelling *with* it rather than in a hand-maintained attribution + table beside it. +- **A proportion over data we hold is not a finding, it is a restatement of one.** Computed by hand, + checked by nobody, and stale the moment any input moves. The counts are the finding. A reader who + wants a ratio can take one, against a denominator they chose and at a moment they know. + +If this were adopted, the "which restatements are mechanically checkable" question earlier in this +note **dissolves** rather than being answered: all of them, because none would be restated. + +**The mechanism is undecided and no work is scheduled here.** The reader-experience trade is real -- +a figure in the prose is read by whoever reads the sentence, and a figure behind a link is read by +whoever follows it, which is a different and unmeasured set -- and it has not been settled. +Recorded as a principle so the next person choosing where to paste a number has the argument in front +of them, not as a queued change. Per "design notes are not a work queue", the absence of a checklist +item is deliberate. + +### The cheapest available move, recorded but not scheduled + +[README.md](crates/windows-waitable-queues/README.md) is already a build input for +`windows-waitable-queues` (`#[doc = include_str!]` in +[lib.rs](crates/windows-waitable-queues/src/lib.rs)), so a test can parse the published layout +tables and assert every row against `ClaimLayout`'s constants -- turning the occurrences that sit in +table rows into checked derivations of one definition, with no generator and no new tooling. It +reaches only those; the occurrences in prose are untouched by it. + +**Be precise about what that would and would not catch, because this paragraph has now overstated it +twice.** The layout table's columns are the layout name, the reservation-count field ceiling, the +pushes-to-recurrence count, and a time. A constants check covers the **ceiling and push-count +columns** outright. The time column additionally needs the assumed rate pinned somewhere single. And +the two errors this note originally named -- the `2^31`/`2^30` target-dependent capacity and the +`MAX_RESERVED`-as-capacity conflation -- it would **not** have caught at all: both are prose +assertions in the surrounding text, not cells in any table. + +That bound is the useful part rather than a caveat on it. A constants-versus-table check reaches the +occurrences that sit in table rows and none of the ones in prose, and both populations are +substantial -- which is the shape of the result, and a reason to build the check rather than not to. +The prose occurrences need something that reads assertions rather than rows. A remedy that covers the +tabular ones is worth having; claiming it covers both is how a partial instrument comes to be trusted +as a complete one. + +*(An earlier version of this paragraph put a proportion here. It is gone deliberately: a ratio over +the counts above is a restatement of them, computed by hand and checked by nobody, and it drifts the +moment any file is edited -- which is the defect this whole note is about. The counts are the +finding. Anyone who needs a proportion can take one, against a denominator they chose and at a moment +they know.)* + +**No work is scheduled by this note.** It was written to inform a decision that has not been taken, +and the deliberate absence of a checklist item is per the "design notes are not a work queue" rule +rather than an oversight. If the table-versus-constants test or a +prose-reduction pass is adopted, each needs its own item at that time. + ## References - [`QueueUserWorkItem` and `WT_TRANSFER_IMPERSONATION`](https://learn.microsoft.com/windows/win32/api/threadpoollegacyapiset/nf-threadpoollegacyapiset-queueuserworkitem) diff --git a/crates/windows-platform-probes/CHECKLIST.md b/crates/windows-platform-probes/CHECKLIST.md index 68d9862ea..bca71d524 100644 --- a/crates/windows-platform-probes/CHECKLIST.md +++ b/crates/windows-platform-probes/CHECKLIST.md @@ -67,6 +67,88 @@ correctness in the archive. own arithmetic does NOT belong, and the honest outcome for such a one is a line in the module header saying so by name rather than a silent absence. +- [ ] **M4.4** -- Interleave each candidate with a nearby control instead of measuring the control + four runs away from it, and re-measure everything that changes. + + **Gap:** `measure()` runs, per producer count, `baseline_fetch_add`, `slotwise_mpsc`, + `reserving_mpsc`, `permit_mpsc`, then the three drained shapes, then the layout rows starting with + `reserving(32/32)`. The same-code control is the `reserving_mpsc` row against the + `reserving(32/32)` row -- **four measurements apart**, each five repetitions of 50,000 pushes per + producer. Frequency, thermal and scheduler drift across that interval is folded into the control, + and into every candidate the control is used to judge. At sixteen and thirty-two producers, where + the machine is oversubscribed and the layout differences are smallest, that is exactly where it + matters most. + + This is the first *specific* mechanism proposed for the 7-61% same-configuration spread recorded in + [DESIGN-NOTES.md](DESIGN-NOTES.md#d-variance-is-a-finding); the other candidates there are general. + Reported by review. + + **Target:** measure each candidate adjacent to a control run of the same code, or randomise and + balance the order across repetitions so drift cannot align with position in the sequence. Whichever + is chosen, the control must end up measuring the same interval the candidate did. + + **BLOCKER, same as M4.3:** interleaving changes the measurement, so every figure published in + [DESIGN-NOTES.md](DESIGN-NOTES.md) becomes a measurement of a different procedure. The item is + "change it *and* re-run the sweep *and* rewrite the sections", not a reordering. Doing it + mid-branch would invalidate figures that ten review rounds have been read against. Raised rather + than silently deferred, per the PRIME DIRECTIVE. + + **It is placed ahead of M4.2 deliberately**, since a control that is not paired cannot answer + whether lengthening the run narrows the spread -- that answer would be confounded by the same + drift this item removes. Taking M4.2 first would produce a diagnosis nobody could trust. + +- [ ] **M4.2** -- Give the measurement probes the controls needed to act on a dispersion finding, + so "gather more data along this axis" does not require editing a `const` and rebuilding. + + **Ordered after M4.4, which is why it appears second despite the lower number.** The first + diagnostic step this item unblocks is "lengthen the run and see whether the control narrows", and + that cannot be read while the control is measured four runs away from its candidate -- drift would + confound it either way. + + **This item is deliberately small in software and large in guidance.** The diagnostic method + belongs in [DESIGN-NOTES.md](DESIGN-NOTES.md) -- see + [What to try first, and how to tell when you have reached the + floor](DESIGN-NOTES.md#d-variance-is-a-finding) -- and this item exists only to make that method + executable. The judgement stays with the person; the probe stops being the obstacle. + + **Gap:** [src/queue_contention.rs](src/queue_contention.rs) fixes every sampling parameter as a + compile-time constant -- `PUSHES_PER_PRODUCER` (50,000) and `REPETITIONS` (5) -- and `measure()` + takes no arguments. They were made `pub` and are now printed in the report, so a captured run at + least says what produced it; but reading a constant is not setting one. The first move the design + note prescribes on seeing a wide control is to lengthen the span and raise the repetition count on + the unchanged configuration, which is still a source edit and a rebuild. A control that cannot be + turned is not a control, and the cheapest diagnostic step is the one being blocked. + + **Target:** `measure()` takes a settings value carrying at least the pushes-per-producer count, + the repetition count, and the producer counts to sweep (`PRODUCER_COUNTS` is already public and + is the model for the others). Existing defaults stay exactly as they are, so a default run remains + the run the notes describe and every published figure stays reproducible. The binary exposes the + same knobs so a human or an agent can act without a rebuild. + + Apply the same treatment to the sibling cost probes where the sampling parameters are equally + fixed; the axes we anticipate varying are **duration, repetitions, and concurrency**, so those are + the ones that need to be reachable. Do not add knobs beyond what a stated diagnostic step needs -- + an unused parameter is a configuration surface to maintain and a way for two runs to differ + without anyone noticing. + + **Report what was used.** Whatever settings a run was given must appear in its output beside the + host banner, for the reason + [D-observations-not-verdicts](DESIGN-NOTES.md#d-observations-not-verdicts) already gives: a figure + is only interpretable with its capture parameters, and these are now among them. Making the + sampling adjustable without recording it would turn one reproducibility problem into a worse one. + + **Not in scope:** deciding why the control is wide. That is the judgement this tooling supports, + and per the design note a negative result -- "lengthening and repeating do not narrow it, so the + floor is here" -- is a real answer that gets recorded beside the figures. + + **Also not in scope, because it is done:** emitting the dispersion. See M4.5 below. + +- [x] **M4.5** -- Emit the dispersion, not just the median. -> [completed 2026-09-15 UTC-07:00](COMPLETED-CHECKLIST.md#m45) + +- [x] **M4.3** -- Close the undrained window at the start of the drained regime with a readiness handshake, and re-measure everything that changes. -> [completed 2026-09-16 UTC-04:00](COMPLETED-CHECKLIST.md#m43) + +- [x] **M4.6** -- Make the start gate releasable, so a failed thread spawn cannot deadlock the probe. -> [completed 2026-09-16 UTC-04:00](COMPLETED-CHECKLIST.md#m46) + - [ ] **M2.5** -- Make the banner describe the read the body describes. Gated by M3.1 and M3.3, both landed: establishing that the middle of three discoveries agreed @@ -109,6 +191,8 @@ correctness in the archive. explicitly after being pointed at the question. Nothing in the suite decides it either way, which is itself the argument for making it an invariant rather than a test. +- [x] **M4.7** -- Make the queue-contention report renderer testable, by taking the observation as an argument instead of measuring inside it. -> [completed 2026-09-16 UTC-04:00](COMPLETED-CHECKLIST.md#m47) + - [ ] **M2.15** -- Run the probe suite on a second architecture in CI. **Keeps its conclusion but loses its evidence.** The five failures cited below were all @@ -153,6 +237,27 @@ correctness in the archive. them. Until this lands, a shape that needs two dimensions must be added by hand, which is exactly the imagination-driven process M2.12 exists to replace. +- [ ] **M4.8** -- Have the queue-contention report carry the build identity that produced it, and + have the capture scripts require it to agree. + + **Gap:** a run's report states its `host:`, `profile:` and `sampling:`, and the capture scripts + now refuse a set whose runs disagree on any of those. None of it identifies the *instrument*. Two + runs from different probe commits, on one machine, under one profile, pass that check -- and this + is the capture where that matters most, because `M4.3` changed the drained procedure, so a + pre-handshake and a post-handshake run would have their medians combined as though one procedure + produced both. The instrument commit is recorded in the capture README, which is an assertion by + whoever took the capture rather than something anything verifies. + + **Target:** `windows-placement-probe`'s `build_identity` module is the worked example -- a build + script stamps the commit, the dirty flag and the build source into env vars that the binary reads + at run time, and `BuildIdentity::current()` renders them. `windows-platform-probes` has no build + script today, so this adds one. The report prints the identity beside the existing attribution + lines, `requireOneConfiguration` in both capture scripts includes it, and the sabotage is two runs + of different commits being refused. + + **Blocker recorded when queued:** none. The dependency exists next door and is already proven by + that crate's own tests. + ## M5 -- Carried over from M2: unblocked hygiene @@ -241,21 +346,4 @@ IDs keep their M2 numbers, for the reason given under M4. - [x] **M2.14.2** -- Add to CONTRACT INTEGRITY rule 1 the one thing this branch learned that it does NOT already say. -> [completed 2026-09-13](COMPLETED-CHECKLIST.md#m2142) -- [ ] **M2.16** -- Repair the garbled `Report` doc comment, and drop the two counts that have already - rotted beside it. - - [src/report.rs](src/report.rs) opens its `Report` sink doc with a dangling fragment -- "A [`Report`] - a renderer can `writeln!` into directly." followed by a blank line and then "is arithmetic. Every - renderer writes through ..." -- so a sentence was lost in an edit, and "moves only 18 renderer - signatures." is followed by a bare repeat of the word "signatures." Introduced 2026-09-09 by - `b5594860` and `3827dc32`, both already on main; found while sweeping a count defect on the report - -oracle branch, where the file was out of scope to touch. - - Both surviving numbers in that passage are censuses that have since drifted. It claims **332 - `writeln!` sites**; measured now, 354. [DESIGN-NOTES.md](DESIGN-NOTES.md) restates the same 332, - so the two must be fixed together or they drift apart again. Replace them with the invariant the - passage is actually arguing -- that `String` already implements `fmt::Write`, so every existing - write site stands untouched and only the renderer signatures move -- which is what makes the point - and cannot rot. This is the same defect class as CONTRACT INTEGRITY rule 1 in - [.github/copilot-instructions.md](../../.github/copilot-instructions.md), which M2.14 exists to - make bite. +- [x] **M2.16** -- Repair the garbled `Report` doc comment, and drop the two counts that had rotted beside it. -> [completed 2026-09-16 UTC-04:00](COMPLETED-CHECKLIST.md#m216) diff --git a/crates/windows-platform-probes/COMPLETED-CHECKLIST.md b/crates/windows-platform-probes/COMPLETED-CHECKLIST.md index 93044d433..06a962a08 100644 --- a/crates/windows-platform-probes/COMPLETED-CHECKLIST.md +++ b/crates/windows-platform-probes/COMPLETED-CHECKLIST.md @@ -1114,3 +1114,190 @@ M4 below, six in M5. M2.18 is the exception, dissolved rather than moved. currently has no sentence about claims. > **-> DEPENDS ON M2.14.1:** the pointer has nothing to point at until the manifest exists. + +## Moved 2026-09-15 20:04:09 UTC-07:00 -- M4.5: the probe carries its own dispersion + +### M4.5 -- Emit the dispersion, not just the median. *(completed 2026-09-15 20:04:09 UTC-07:00)* + +Split out of M4.2 when it landed, because M4.2's remaining work -- the sampling controls -- is +independent of it and an unchecked item must not embed a completed deliverable. + +`median_run` took the five timed repetitions, sorted them, kept the middle one and **discarded the +other four**. `Run` then carried a median with no spread, so every figure derived from the probe was +published without dispersion and any range quoted elsewhere had been computed by hand outside the +instrument. + +That was a contract failure rather than a gap in polish. +[D-observations-not-verdicts](DESIGN-NOTES.md#d-observations-not-verdicts) requires every published +figure to carry "the number of runs with their dispersion", and says a ratio quoted without those +"is an anecdote, not data a reader can compare against their own hardware". The probe was the source +of the figures that decision governs and did not satisfy it. Reported by review, and correctly. + +`Run` gained `fastest_nanos_per_op` and `slowest_nanos_per_op`, taken from the ends of the sort that +already existed, plus a `spread()` accessor returning `Option` -- `None` for a shape that did +not run. `render_table` publishes an `ns/op range` column and a `spread` column. + +*(Corrected before merge. This item first shipped `spread()` returning `0.0` for the unmeasurable +case, recorded here as "the same guard `format_ratio` carries for the same reason". A later review +on the same branch found that `0.0` renders as `0.00x`, which reads as perfect stability -- the most +reassuring cell the column can hold, produced by a row that measured nothing. `spread()` now returns +`Option`, and every renderer routes through `Run::is_measured` rather than restating the test. The +original wording is noted rather than quietly replaced because it did not merely go stale: it +presented returning zero as the fix, when returning zero was the defect.)* + +Verified load-bearing by sabotage: taking the fastest from the median index instead of the minimum +fails `median_run_carries_the_fastest_and_slowest_repetitions`. + +The dispersion justified itself on first capture. `slotwise_mpsc` at two producers spans 19.3 to +59.5 ns/op within one configuration on one host, which the median alone had +concealed entirely, in a table that had already been published twice. + +## Moved 2026-09-16 16:46:50 UTC-04:00 -- M4.6 and M4.3: the start gate, and the window it left open + +### M4.6 -- Make the start gate releasable, so a failed thread spawn cannot deadlock the probe. *(completed 2026-09-16 16:46:50 UTC-04:00)* + +Every timer sized a `std::sync::Barrier` for all planned workers plus the coordinator, then spawned +the workers with `Scope::spawn`, which **panics** if the OS cannot create a thread. If that happened +after an earlier worker had already parked, the coordinator never reached its own arrival, the party +count was never met, and `thread::scope` joined a permanently parked worker while unwinding. The +probe hung rather than failed, which is the worse of the two. In the drained timers the consumer was +parked on the same barrier, and `StopOnDrop` could not help, because the scope could not finish +unwinding to drop it. + +A `Barrier`'s party count, once set, must be met, so the fix was a change of primitive rather than a +change of call. `StartGate` keeps the property the measurement depends on -- a complete party +releases every member together, which is what `measured_span` relies on when it argues that each +worker must time itself -- and adds the operation `Barrier` lacks. `arrive_and_wait` now reports +whether the party completed, so a worker freed by a release returns instead of timing an abandoned +run; `ReleaseOnDrop`, held inside each scope's closure, performs the release while that closure +unwinds, which is before the join loop it has to unblock. Poisoning is stepped over rather than +propagated, because panicking out of a `Drop` that is already unwinding would abort the process +instead of unblocking it. + +Applied to all nine timers at once, since they share the defect and this branch had three times +shipped a correct fix applied to a subset of its call sites. + +**Some of the new gate tests were first written to assert on the test thread, and sabotage caught +them**: with `release` neutered they parked the test rather than failing it, which would wedge a +suite that runs its tests as threads in one process. Every gate test now arrives off-thread and +polls, so a regression reddens in five seconds. + +### M4.3 -- Close the undrained window at the start of the drained regime with a readiness handshake, and re-measure everything that changes. *(completed 2026-09-16 16:46:50 UTC-04:00)* + +The gate proves the consumer exists, is scheduled and is past thread start-up; it does not prove the +consumer has reached its first `pop`, and it releases every party together. So a producer could push +into a queue nobody was draining yet -- an undrained opening to a run whose whole subject is that it +is drained. `await_consumer` closes it: the consumer announces that it is draining, and producers +hold until they see that before starting their clocks. Applied to all four drained timers, with +`Acquire`/`Release` per the queue crate's `D-38` standing answer on promoting the load. + +**The blocker recorded when this was queued was real, and it is what made the item large.** The +change moves the drained numbers, so every drained figure already published measured a different +piece of code. Re-measured on the same host, three whole-probe invocations, committed as a capture +at [captures/2026-09-16-drained-handshake/](captures/2026-09-16-drained-handshake/README.md) with the summarising script beside the raw runs so the +derivation can be checked rather than trusted. + +**The figures are amended rather than replaced**, because this is new data and not a correction: the +earlier capture remains what the earlier instrument measured, and both are labelled with the code +that produced them. + +**What the re-measurement establishes was overstated when this entry was written, and the correction +belongs here.** It originally said the finding survived -- every layout median still inside the +same-code control band. That rested on pooling every control observation into one band, and the +pooling produced the answer: the control is not independent of producer count, so a pooled band is +wider than any count's own and containment follows from the method. Compared per count, several +medians fall outside their own count's range; compared per count the other way, three runs give +three control observations, which is not a band. Three runs do not settle the drained comparison in +either direction. The capture now reports per-count figures and emits no verdict, and the +pre-handshake reading rests on the seven-run sweep, which this does not replace. + +**Two further corrections, appended because this archive is append-only.** The +paragraph above names `await_consumer` as what closes the window; that helper is +only the *waiting* half -- a producer spinning until the flag is set. The +ordering the item is actually about lives in `drain_then_announce`, which pops +once and only then publishes readiness, and which was extracted into a single +definition later (see the M4.3 review round) precisely because four timers had +been writing it by hand with nothing able to test it. + +And "closes it" overstates what any flag can do. The window is *narrowed to a +stated guarantee*: no producer begins timing until the consumer has executed its +pop path at least once. Continuous draining is not guaranteed and cannot be -- +the consumer can be descheduled immediately afterwards, as it can at any point +during the run. `await_consumer`'s own doc said so from the start, which is what +makes this entry's wording a restatement that drifted from the thing it restated. + +## Moved 2026-09-16 17:31:31 UTC-04:00 -- M2.16: the census that broke the prose around it + +### M2.16 -- Repair the garbled `Report` doc comment, and drop the two counts that had rotted beside it. *(completed 2026-09-16 17:31:31 UTC-04:00)* + +[src/report.rs](src/report.rs) opened its `Report` sink doc with a dangling fragment -- a title line, a blank line, +then "is arithmetic. Every renderer writes through ..." -- and further down repeated the bare word +"signatures." after the sentence that already ended in it. Both were introduced on 2026-09-09. + +**The history is the point, and it was recovered rather than guessed.** `b5594860` wrote the passage +with a full sentence -- "This is the answer to 'how does a formatted line reach the sink', and the +reason it is a `std::fmt::Write` adapter rather than a method on `Report` is arithmetic" -- and a +count of **504** write sites. `3827dc32`, titled "Update comments in report.rs for better clarity", +revised that count to **332** and, in the same edit, deleted the two lines that carried the sentence. +So the edit that maintained the census is the edit that broke the prose, which is as direct an +argument for CONTRACT INTEGRITY rule 4 as this repository has produced. + +The two homes were fixed together, because fixing one alone would have created a fresh disagreement: + +- **[src/report.rs](src/report.rs)** states the property instead of the count. `String` already implements + `fmt::Write`, so a sink that does too leaves every write site untouched and moves only the renderer + signatures. That is what makes the point, and it cannot rot. The lost sentence is restored, the + duplicated word removed, and the census delegated to the design note by link. +- **[DESIGN-NOTES.md](DESIGN-NOTES.md)** keeps the numbers, because there they *are* the finding: the decision was + taken by counting, and the entry contrasts M1's estimate of "upwards of 160" with what re-measuring + found. They are now pinned as the census *as it stood on 2026-09-09 when the decision was taken* + rather than stated in the present tense as a description of the crate now, and the passage no longer + repeats the figure four times to make its argument. + +**A third home was found by sweeping and deliberately left alone.** `COMPLETED-CHECKLIST.md`'s +archived `M1.1` entry restates 332 and 18. It is pre-existing on main rather than authored by this +branch, it sits in an append-only archive dated by its own heading, and there the count is the whole +of the record -- the entry exists to say that the estimate was wrong and that re-measuring decided +the question. Rewriting it would have been an archive rewrite in service of tidiness. The design +note's new wording agrees with it rather than contradicting it. + +## Moved 2026-09-16 18:40:00 UTC-04:00 -- M4.7: the report renderer, driven by a corpus + +### M4.7 -- Make the queue-contention report renderer testable, by taking the observation as an argument instead of measuring inside it. *(completed 2026-09-16 18:40:00 UTC-04:00)* + +`render` called `measure()` itself, so the only way to exercise it was a ~65-second host-dependent +pass. Everything beyond the library's `render_table` was therefore reached by nothing in the suite: +the three tables' assembly, the derived column widths, the `cfg`-gated `Wide` rows, and the prose +between them. Two defects shipped through that gap on this branch -- two tables hard-coding a column +width for formatters whose output has no fixed maximum, and a drained footer printing a seven-run +result beneath a table produced by one invocation. + +`render` now writes the banner and calls `render_observation(out, &measure())`. The banner stays +outside because it is a fresh topology read rather than a function of the observation, which is what +keeps the rendering half pure and therefore drivable by a fixture. The binary moved to +[src/bin/queue_contention/main.rs](src/bin/queue_contention/main.rs) so it can carry a sibling +[tests.rs](src/bin/queue_contention/tests.rs), following +`windows-placement-probe`'s layout; git recorded it as a rename, so history follows. + +**The cases are data, not code.** [corpus.json](src/bin/queue_contention/corpus.json) holds an +observation and what the rendered report +must be true of, so adding a case needs no Rust. The central check is *derived rather than +restated*: `aligned_tables` asserts every line of a named table is the same length, which is exactly +the property a cell wider than its column breaks. It therefore catches width bugs the corpus never +anticipated, where a golden would only catch what somebody thought to record and would need +regenerating whenever the prose moved. + +Two things the corpus established on first run, both of which a hand-written fixture would have +missed: + +- With an empty observation the `ns/op range` table emits a header and no rows, while the layout and + scaling tables still emit six `--` rows, because those iterate `PRODUCER_COUNTS` and that one + iterates the runs. The first expectation written was wrong about this, not the renderer. +- **A width of 22 was not overrun by an ordinary outlier.** The 300ms-against-4ns repetition that + motivated the original finding renders 21 characters; reaching 23 needs a hundredfold ratio as + well. The argument for deriving the width is that no constant can be *established* as sufficient, + since the cell's width is a function of measured data -- not that 22 was visibly too small. The + corpus case says so in its own `why`, having been corrected once for claiming otherwise. + +Sabotage-verified: replacing the derived widths with the constant they had before fails the corpus +case, naming the table and the overrun line. diff --git a/crates/windows-platform-probes/Cargo.toml b/crates/windows-platform-probes/Cargo.toml index db464bcba..1b3ff7259 100644 --- a/crates/windows-platform-probes/Cargo.toml +++ b/crates/windows-platform-probes/Cargo.toml @@ -82,6 +82,10 @@ path = "src/bin/doorbell_cost.rs" name = "probe-request-cost" path = "src/bin/request_cost.rs" +[[bin]] +name = "probe-queue-contention" +path = "src/bin/queue_contention/main.rs" + # These two are the same code, and that is the measurement: they differ only in # whether `build.rs` embeds the `longPathAware` manifest, which is not a runtime # switch and so cannot be a flag on one binary. @@ -140,6 +144,16 @@ windows-placement-probe = { path = "../windows-placement-probe" } # a queue, not a stand-in, for the same reason the topology probe reads the # shipping parse: a reimplementation would measure the reimplementation. windows-namespace-request-sys = { path = "../windows-namespace-request-sys" } +# The contention probe measures the shipping queue shapes rather than a +# reimplementation, for the same reason: a stand-in would only measure itself, +# and the whole question is what the real tail claim costs. +# +# The experimental permit claim is enabled here because this probe is what +# decides its fate (SH-15.5): it must be measured against the shipping shapes on +# the same host, in the same run, by the same harness. +windows-waitable-queues = { path = "../windows-waitable-queues", features = [ + "experimental-permit-claim", +] } # The long-path probe measures a length against `MAX_PATH`, and `MAX_PATH` counts # UTF-16 code units. `OsStr::len` counts Rust's platform encoding -- WTF-8 here -- # so the two disagree the moment a non-ASCII character appears in `%TEMP%`, which @@ -161,6 +175,43 @@ wtf-string = { path = "../wtf-string" } # still the real parser's and this crate hand-writes no string walking at all. serde = { version = "1.0", optional = true } +# The 128-bit claim layout is measured only where a 128-bit exchange is a native +# instruction, and the feature that supplies it is added per-target rather than +# unconditionally. +# +# This is not a tuning choice, it is the queue crate's `D-18`/`D-37` contract: +# `dwcas` is non-default precisely so `reserving_mpsc`'s narrow word ships on +# every target while the wide one ships only where it is genuinely lock-free. +# `i686-pc-windows-msvc` has no 128-bit atomic at all, and `D-18` records that +# the workspace deliberately keeps that target supported -- so enabling `dwcas` +# from here without a target gate would propagate through feature unification and +# fail the i686 build in the queue crate (`cannot find AtomicU128 in +# portable_atomic`). Measured, not assumed: that is the exact error this gate was +# added to remove. +# +# `target_has_atomic = "128"` would be the WRONG condition -- `D-37` records that +# rustc emits it even with `cmpxchg16b` disabled, and that is measured here: +# `rustc --print cfg --target x86_64-pc-windows-msvc -C target-feature=-cmpxchg16b` +# still prints `target_has_atomic="128"` while dropping +# `target_feature="cmpxchg16b"`. So the feature is the discriminator, and the +# x86-64 branch tests it: naming the architecture alone would enable `dwcas` on a +# build with the instruction switched off, where `AtomicU128` does not exist and +# the queue crate fails to compile. aarch64 needs no such test -- `ldxp`/`stxp` is +# ARMv8-A baseline and requires no target feature. +# +# `src/queue_contention.rs` gates the `Wide` rows on the SAME conditions. The two +# must agree; the source side names this comment so a reader changing one finds +# the other. +[target.'cfg(all(target_arch = "x86_64", target_feature = "cmpxchg16b"))'.dependencies] +windows-waitable-queues = { path = "../windows-waitable-queues", features = [ + "dwcas", +] } + +[target.'cfg(target_arch = "aarch64")'.dependencies] +windows-waitable-queues = { path = "../windows-waitable-queues", features = [ + "dwcas", +] } + [dev-dependencies] # Here because [tests/a_real_report_agrees_with_itself.rs] uses `serde_json` # DIRECTLY, to empty the row's diagnostic lists by parsing and re-rendering diff --git a/crates/windows-platform-probes/DESIGN-NOTES.md b/crates/windows-platform-probes/DESIGN-NOTES.md index 5417ffd1f..70356fb77 100644 --- a/crates/windows-platform-probes/DESIGN-NOTES.md +++ b/crates/windows-platform-probes/DESIGN-NOTES.md @@ -19,6 +19,204 @@ concentrated form: a hand-written second copy of a platform check is not a check of the platform, it is a check of the copy. When the two disagree, nothing detects it. +## A probe reports observations parameterized by their capture; it does not draw the client's conclusion + + + +Every figure this crate publishes is **what one machine did on one day**, and it +is recorded together with the parameters of its capture -- at minimum the probe's +host banner (architecture, logical and core counts, SMT, cache groupings, +efficiency classes, NUMA nodes), the build profile, and the number of runs with +their dispersion. A ratio quoted without those is an anecdote, not data a reader +can compare against their own hardware. + +**Some capture parameters are approximated, and some are simply absent; the +difference is stated rather than elided.** Windows exposes no NUMA *distance* +table -- there is no Win32 equivalent of reading ACPI SLIT, as recorded at the +`proximity` site in +[core_affinity.rs](../windows-placement-probe/src/core_affinity.rs). What it does +expose, and what this crate uses as the best available analog, is the +**assignment of processors to NUMA nodes** -- which is what the banner's +`numa[...]` field carries, as processors-per-node. Device-to-node mapping is +obtainable on the same footing. That analog answers "are these two things in the +same domain", which is the question most placement decisions actually turn on; it +does not answer "how much further is node 2 than node 1", and no amount of +probing on Windows will. Memory configuration and BIOS state are not captured at +all. + +**Reading the analog is part of reading the figure.** A banner of `numa[16]` is +a *single-domain* machine, so measurements taken on it say nothing whatever about +cross-domain behaviour -- not "a little", nothing. A figure is only evidence +about the domain structure its banner records. + +**The conclusions this crate is willing to draw are coarse, mechanically +reasoned, and observation-backed** -- "the buffers should be in the same memory +domain as the executor" is the shape of a claim that earns its place, because it +follows from how the hardware works *and* the measurements agree. **Fine-grained +topological and layout choices are handed to the client, not made for them.** +Which position/reservation apportionment a queue should use is exactly such a +choice: the layout is a type parameter of the shipping queue, the probe measures +every candidate, and the note reports what it saw. It does not name a winner. + +### Do not tell a reader what to do about a measurement + + + +State what was observed, under what parameters, and what the procedure could not +determine. Stop there. Advice built on top of a measurement -- "prefer this +layout", "most callers do not need that", "measure on your own target if +throughput matters", "start with this shape" -- **takes on a responsibility this +crate has no standing to hold**: it has measured one machine, and the reader's +deployment is not that machine. A recommendation converts an observation into an +undertaking, and the undertaking is one nobody here can honour. + +This is not the same as withholding information. Everything needed to decide +still gets published: the figures, their capture parameters, their dispersion, +the hazards, the structural differences between the shapes, and the limits of +what the instrument can separate. What is removed is the imperative mood. + +Two forms are easy to miss because they sound like caution rather than advice. +"Measure on your own hardware before choosing" is still an instruction, and it +implies the reader's not having done so is a mistake we warned them about. +"This is the answer for almost every caller" is a recommendation wearing a +hedge. Both are struck. A statement about what a *layout provides* is a fact and +stays; a statement about what a *reader should pick* is not. + +This is why the apportionment claim in the queue-contention section was +*withdrawn in both directions* rather than reversed. The measurement stopped +supporting "re-apportioning is free", but it equally did not support "it costs +30%" -- one host, seven runs, against a control that wanders. The correct output +of a probe that cannot call something is **the record that it could not**, never +a verdict chosen because a verdict reads better. What a reader does with that is +the reader's decision, and stating it for them would take a responsibility this +crate has no standing to hold: it has measured one machine. + +The failure this prevents is a reader inheriting a number as though it were a +property of the code. It is a property of the code **on that machine**, and the +distinction is the whole value of shipping the probe rather than only its output. + +**A figure here is the output of a procedure, and that is what review checks.** +An observation is not an opinion, a position, or a point of view, so it is not +something to be argued for or against. The reviewable questions are procedural: +was the right instrument used, were the capture parameters recorded, was the +dispersion reported, does the stated conclusion stay inside what the procedure can +support. If the answer to those is yes, the figure stands as an observation -- +including when a reader would have characterised it differently. + +**So do not re-litigate a number or its wording.** A challenge of the form "this +should be called 1.4x rather than 1.1x", or "this ought to be phrased as a bound +rather than a comparison", is answered by pointing at the procedure and the table, +not by renegotiating the prose. The exception is a statement that the procedure +cannot support at all -- an attribution to a cause the measurement does not +isolate, a claim of significance a control contradicts, a figure restated +somewhere it can drift from its source. Those are procedural defects and are +fixed. The distinction is worth holding because the two look alike in a review +comment and only one is worth the time. + +This is a consequence of the decision above rather than a separate rule: once +figures are published as parameterised observations rather than as verdicts, there +is no verdict left to disagree with, and treating each characterisation as a +position to defend re-introduces exactly the authority the decision removes. + +### High variance in our own control is a finding about the instrument, not just a wider yardstick + + + +When the same code measured twice in the same run disagrees by tens of percent, +the first thing that has been measured is **the method**. It is tempting to treat +a wide control as merely a coarser ruler -- to widen the band and carry on +judging ratios against it -- and that is the mistake this decision exists to stop. +A control that wanders is a defect report against the measurement, and it is +logged as one even when the measurement is still used. + +The candidate causes are not distinguishable from the dispersion alone, and all +of them are live here: + +- **The wrong instrument for the variable.** A probe that moves several things at + once cannot attribute a difference to the one under test. +- **A control that is not a paired control.** This probe's same-code control is + the `reserving_mpsc` row against the `reserving(32/32)` row, and `measure()` + runs them **four measurements apart** -- the permit shape and all three drained + shapes fall between them, each five repetitions of 50,000 pushes per producer. + Any frequency, thermal or scheduler drift across that interval lands inside the + control, and by the same token inside every candidate row it is compared + against. This is a specific, mechanical candidate for the spread above rather + than a general worry, and it was proposed by review rather than found here. + Queued as M4.4 in [CHECKLIST.md](CHECKLIST.md), because interleaving changes the + measurement and so obsoletes the published figures. +- **A defect in the probe itself.** This crate has already shipped one -- the + timing window that read the coordinator's clock rather than the producers'. + That defect was invisible in the numbers until it was found by reading, and it + moved high-producer figures by roughly 45%. +- **Insufficient runs or too short a duration** -- straightforward hygiene, and + the cheapest to rule out. +- **A noisy machine.** These are fine-grained measurements taken on a shared, + general-purpose desktop running everything else it normally runs. Scheduling, + frequency scaling, and other tenants all land inside the timed region. + +**How much this matters depends entirely on what the number is for, and that +calibration is recorded rather than assumed.** In benchmarking or marketing +literature it would be disqualifying: those documents exist to support a +comparative claim, and a comparative claim resting on a control this wide is not +supported. Here the purpose is *planning for deployment environments resembling +the measured one* -- and a figure gathered on an ordinary loaded machine is not +obviously the wrong input for planning on ordinary loaded machines. So the +honest treatment is neither to suppress the data nor to promote it: **record it, +record the dispersion beside it, and record that the dispersion is itself +unexplained.** + +### What to try first, and how to tell when you have reached the floor + +**The cheapest move is always to gather more of the same before gathering +anything different.** Lengthen the timed span, raise the repetition count, or +both, on the *unchanged* configuration. This costs only wall time and it +partitions the problem in one step: if the control narrows, the dispersion was +sampling noise and the previous run simply had too few samples to resolve +anything; if it does not, the width is structural and the remaining candidates +are the interesting ones. Do this before pinning threads, before quiescing the +machine, and before suspecting the probe -- each of those changes what is being +measured, and a change made before the cheap check cannot be evaluated. + +**A warmup pass separates transient cost from ongoing noise, and the two are +different things.** This probe already discards one untimed pass -- though not for +the reason an earlier version of this paragraph gave. Every timed repetition +builds and drops its own queue, so the discarded pass cannot fault in any +allocation a timed pass will use; what it warms is process state, the allocator's +size class, the OS page cache, the instruction cache and the branch predictors. +Cold caches, predictors, and CPU +frequency ramp are the same *kind* of cost -- one-time, front-loaded, not a +property of the steady state -- and lengthening the timed span dilutes them +whether or not a warmup removes them. + +It is worth being clear that this does **not** contradict the position that some +noise is inherent to a shared machine. A warmup removes *transients*; contention +with other tenants continues for the whole run and is not removable by any amount +of warming. The two widen dispersion for unrelated reasons, and removing the +transients is what makes the inherent floor *visible* rather than what hides it. +Expect warming and lengthening to shrink the spread to some value and then stop +shrinking it, and treat that plateau as the interesting result. + +**There is always a floor, and recognising it is the skill this decision is +really about.** A measurement cannot resolve a difference smaller than the noise +in the quantity being differenced, and past that point more runs buy nothing -- +continuing to gather them is how a project spends a week proving that two numbers +are the same. The floor is a real, findable property of the setup, not a failure. + +**The floor is not necessarily a percentage of the measured value**, and assuming +it is will mislead you in both directions. It can be set by the sampling regime +instead: the granularity of the clock, how many independent samples the run +actually takes, or how the measured span is constructed. This probe times a whole +pass and divides -- two timestamps per worker per repetition -- so at small +absolute values the resolvable difference is governed by how many independent +passes were taken, not by any fixed fraction of the nanoseconds reported. That is +why "the 1-producer rows are noisy because the numbers are small" is a guess +rather than a diagnosis, and why the first move above is to add samples: it tests +that guess directly. + +What this decision forbids is the quiet version -- reporting a wide control as +though a wide control were normal. It is not normal. It is an open question, and +where it is open, the note says so and the checklist carries the work. + ## Three tiers, because "run all the probes" is not a safe instruction @@ -481,6 +679,502 @@ architecture. It would fail on any host with a longer user name, on either architecture. Recorded here because it is exactly the kind of result this comparison exists to classify correctly: a red build that is **not** a finding. +## The queue-contention probe, and why it must not run in the CI probe job + +`probe-queue-contention` measures two things a design decision is waiting on: whether the bounded +array queue's tail claim contends badly enough to justify the linked and sharded MPSC shapes, and how +[`reserving_mpsc`](../windows-waitable-queues/src/reserving_mpsc.rs) and `slotwise_mpsc` compare end +to end in the regime where `reserving_mpsc`'s extra read of the consumer's position is most expensive. + +**An end-to-end comparison, and deliberately nothing finer.** Two earlier wordings of this sentence +were both wrong: the first said the probe *prices* that read, the second said it *bounds* it from +above. Neither holds. Writing `R` and `S` for the two shapes' total push costs, `R - S` contains the +read plus the differences in claim protocol, slot metadata and retry behaviour, and those terms are +not ordered -- so the difference constrains the read in neither direction. Isolating it would need a +matched control this probe does not have. + +**The checklists carrying those decisions are not in this repository yet** -- they arrive with the +rest of the queue work -- so this note deliberately names the QUESTIONS rather than linking to items +that would dangle. The probe is the instrument; it is useful before the plan that consumes it lands, +and it is landed first precisely so the decision is made against measurement rather than argument. + +**It is deliberately absent from the `platform-probes` CI job, and the reasons are a core count and a +clock rather than a preference.** A contention curve needs more cores than a hosted runner has: the +sixteen- and thirty-two-producer rows on a four-core runner would measure the scheduler and report it as +contention. And the run costs about 65 seconds, against a job whose other probes are seconds apiece. + +**It must be run in release, which is a measurement and not a preference.** In a debug build +`slotwise_mpsc` and `reserving_mpsc` come out at 249.7 and 254.0 ns/push at sixteen producers -- +indistinguishable. In release, on the same machine in the same minute, 193.5 and 52.2. The un-inlined +overhead of a debug build swamps the cache-coherence effects that *are* the finding, so a debug run does +not merely lose precision: it reports the two shapes as equivalent, which is a confident wrong answer of +exactly the kind this crate's `doorbell_cost` notes warn about. + +**Those four figures predate a correction to the timing window and have not been retaken.** The +qualitative finding is unaffected -- a debug build still swamps the effect -- but the numbers themselves +were measured while the probe timed from this thread's clock rather than from the producers' own, which +overstated throughput at high producer counts. Measured on `x86_64 16p/8c` after the correction: +`reserving_mpsc` at sixteen producers moved from 35.0 to a median of 52.3 ns/push across seven runs +(46.3-55.6). The move is larger than that shape's own run-to-run spread on this host, so the direction +is not in doubt; the magnitude is a single host's observation. Any figure in this note taken before the +correction should be read as optimistic until retaken. + +**An earlier version of this paragraph put that run-to-run spread at "2-6%", which seven runs do not +support** -- the same shape and configuration ranges 18% at sixteen producers, and the layout rows below +range considerably wider. The 2-6% figure came from comparing two runs, which cannot measure a spread; it +is corrected here rather than quietly dropped because several conclusions in this note were written +against it, and one of them did not survive the correction (see the layout section below). + +**That is a constraint on HOW it runs, not an argument for keeping it out**, and an earlier draft of this +paragraph confused the two -- it said the CI job "runs `cargo run` without `--release`", which is not true +of the job it describes: `probe-doorbell-cost` and `probe-request-cost` already run there with `--release`, +under a comment establishing exactly the rule this probe would fall under. It also said "unlike every other +probe", and `probe-cancel-io` is likewise absent. Corrected by a review. The release precedent exists; what +keeps this one out is that it costs an order of magnitude more than the two probes that use it, on hardware +that cannot answer the question anyway. + +So this one is run by hand, on a known machine, and its numbers are recorded with the machine attached. + +### Reading it + +Two regimes, and the pair is the point. + +**Isolated** gives producers a capacity large enough that nothing is ever refused and runs no consumer, so +whatever curve appears against N is the producer side alone, with no consumer traffic in it. It is not +the claim alone -- what is timed is each shape's whole push path, tail claim and slot write and +publication and doorbell together, so a difference here is a difference in PUSH COST rather than +evidence about the claim on its own. **Drained** runs a consumer looping on `pop`, +which is the regime in which `reserving_mpsc`'s read of `head` is most expensive -- that +read is +cheap until a consumer is *writing* the line, and measuring it in isolation would report it as free. +It neither isolates that read nor bounds it: the ratio is between two complete push paths whose other +differences are not ordered. + +The drained regime has a **single** consumer, because that is what MPSC means, so at high producer counts +it becomes consumer-bound and a plateau there says nothing about the claim. Each row carries the refusal +count from the queue's own `Observable` counters precisely so that is visible as a fact rather than +mistaken for contention: the sixteen- and thirty-two-producer drained rows show millions of refusals and +should be read as measurements of the consumer. + +## The claim word's width costs 1.1x to 3.8x in isolation, and the drained figure is withdrawn + +Measured by `probe-queue-contention` on one host, `x86_64-pc-windows-msvc`. +Four apportionments of `reserving_mpsc`'s claim word: 32/32, 16/48 and 8/56 over +`AtomicU64`, and 64/64 over `AtomicU128`. The last is measured only where a +128-bit exchange is native: aarch64, and x86-64 **built with `cmpxchg16b`** -- +the gate is the target feature rather than the architecture, because an x86-64 +build with the instruction switched off has no `AtomicU128` either. On a target +without one the report carries the other three and leaves its column empty. + +**These were duplicated scaffolding when the measurement was taken, and they +ship now.** The layouts were built as copies so the shipping crate was not +disturbed while the question was open; the measurement below is what closed it, +and they are now +[`ClaimLayout`](../windows-waitable-queues/src/reserving_mpsc.rs) with +`Balanced`, `Enduring`, `Perpetual` and `Wide` as its implementations -- which is +what this probe imports. Recorded because the original wording still described +the scaffolding, and a reader who went looking for `claim_layout.rs` would not +find it. + +`AtomicU128::is_always_lock_free()` is **true** on this target and +`cfg(target_feature = "cmpxchg16b")` is enabled by default, so the 128-bit +exchange is a compile-time-guaranteed native instruction here and no CPUID +branch was measured as though it were the algorithm. + +| producers | 16/48 vs 32/32 (isolated) | 64/64 vs 32/32 (isolated) | 64/64 vs 32/32 (drained) | +|---|---|---|---| +| 1 | 1.14x | 2.05x | 1.05x | +| 4 | 1.21x | 1.37x | 1.12x | +| 8 | 1.00x | 2.33x | 1.07x | +| 16 | 0.88x | 2.37x | 1.00x | +| 32 | 0.98x | 2.99x | 1.11x | + +**Re-apportioning the bits looked free here, and that reading was withdrawn.** +The reasoning was that both layouts issue the same `lock cmpxchg` on the same +`u64`, so only the shift and mask constants differ, and the table above was read +as confirming it. The table cannot carry that weight: these are single-run +figures, and the same-code control measured later ranges 0.69-1.12x isolated and +0.68-1.27x drained, either of which is +wider than most of the differences being called "noise" -- note that this very +table has 16/48 at 1.14x and 1.21x while the prose beneath it says "within +noise". See +[Re-measured on the shipping type](#d-queue-layout-observations) +below for the seven-run figures and the withdrawal. What the re-apportionment +buys is not in dispute: the recurrence moves from 2^32 to 2^48, from about 37 +seconds of sustained maximum-rate pushing to about 28 days. + +**Widening the word is not free in the isolated regime, and the drained figure +below does not survive the re-measurement.** Isolated, where no consumer touches +the queue, `cmpxchg16b` cost 2-3x on the stand-in and the penalty *grows* with +contention; that is the one conclusion in this section the seven-run +re-measurement strengthened, to 3.45x and 3.81x at sixteen and thirty-two +producers. (These are shares of total push cost, not of the exchange: the +isolated regime times the whole push path, and only the layout differs between +these rows.) The drained figure of 5-12% is **withdrawn** -- not because the +number moved, but because nothing was measuring whether it meant anything. The +re-measured drained 128-bit rows run 2-13%, which resembles the old figure +closely enough to look like confirmation, while every one of them sits inside a +same-code control spanning -32% to +27%. A number that agrees with its +predecessor is not thereby established; that is precisely the trap the control +exists to catch, and this is the case where it catches it. + +### The drained regime is hard to read, and the refusal counts do not settle it + +The two regimes must not be averaged, and the drained one must not be read as +the answer on its own. The mechanism is structural: **a slower producer is less +backpressured**, so it earns fewer refusals, and refusal retries are inside the +timed region -- which means part of what makes a slower shape's per-push number +look close may be that it spent less time being turned away. + +**The refusal counts were offered here as evidence of that, and they do not +support it.** An earlier version of this section reported the 64/64 layout taking +12,149 refusals at eight producers against 32/32's 74,181, and read the +asymmetry as the mechanism showing through. Re-measured three times on the same +host, the counts are neither stable nor consistently ordered: + +| run | 32/32 refusals | 64/64 refusals | +|---|---|---| +| 1 | 925 | 14,461 | +| 2 | 12,613 | 2,152 | +| 3 | 1,814 | 10,337 | + +The ordering reverses between runs and the magnitudes span more than an order of +magnitude either way, so no single run's counts establish anything about which +shape was more backpressured. The original figures were one run, and they are not +reproducible in direction or in size. + +What survives is the confound, not a measurement of it: the drained numbers +contain retry time whose amount is unknown and varies between runs, so a drained +difference cannot be read as a difference in push cost. That is a reason to +distrust the drained ratios, which is weaker than the claim this section +previously made and is what the data supports. + +**The isolated regime removes consumer traffic; it does not isolate the claim.** +Both regimes time the whole push path -- the tail claim, the slot-sequence load, +the item write, the publication store and the doorbell's fence. An earlier +version of this paragraph called the isolated regime "the clean measurement of +the claim itself", said the drained one shows the claim "is not the dominant +cost", and placed a real application between the two. None of the three follows: +the first attributes a whole-path number to one operation, the second rests on +drained figures the paragraph above has just shown to be confounded, and the +third is an interpolation between two regimes that measure different things, +offered about deployments this crate has not seen. + +### What the control caught + +The first run reported 3.7x against the shipping shape and a completely +different scaling curve. The cause was that the duplicate had not padded `head` +and the claim word onto separate cache lines, which `reserving_mpsc` does +deliberately -- every producer reads `head` on every push, so sharing a line +puts the consumer's writes in their path. Aligned, the duplicate tracks the +shipping shape's curve. + +A residual gap remains: the duplicate runs about 1.26x slower than +`reserving_mpsc` at high producer counts. That offset applies equally to all +three layouts, so the ratios above stand, but it means these figures are **not** +absolute numbers for the shipping shape and must not be quoted as such. + +**Comparing a duplicate against the original it stands in for is what made both +of these visible.** A run of three layouts that agreed with each other and +disagreed with reality would have looked entirely healthy. + +### What each apportionment actually buys + +The rollover figures for candidate splits, computed from the rates above. The +rate model reproduces the crate's own published figure -- 32/32 at 116M/s gives +37 seconds, which is what `reserving_mpsc`'s module documentation discloses -- so +these are an extension of that disclosure rather than a competing estimate. + +| split (reserved/position) | reservation field width | @257M/s | @116M/s | @33M/s | +|---|---|---|---|---| +| 32/32 (ships) | 2^32 | 17 s | 37 s | 2.2 min | +| 24/40 | 2^24 | 71 min | 2.6 hr | 9.2 hr | +| 21/43 | 2^21 | 9.5 hr | 21.1 hr | 3.1 days | +| 20/44 | 2^20 | 19.0 hr | 42.1 hr | 6.1 days | +| 16/48 | 2^16 | 12.7 days | 28.1 days | 98 days | +| 12/52 | 2^12 | 202 days | 449 days | 4 yr | +| 8/56 | 2^8 | 9 yr | 20 yr | 69 yr | +| 64/64 (`u128`) | 2^64 | 2,270 yr | 5,039 yr | 17,607 yr | + +**The second column is a field width, not a reachable reservation count**, and +the distinction matters twice. A field of 2^n encodings holds a maximum count of +2^n - 1; and the shipping type caps the count at `u32::MAX` however wide the +field is, because it is handed back to callers as a `u32` -- so the 64/64 row's +2^64 encodings buy no more reservations than 32/32's. Admission is bounded by +capacity as well, which for the shipping 32/32 layout binds first at 2^31 slots. +The time columns are unaffected: they depend on the *position* half, which is +what this table exists to compare. + +Rates: 257M/s is the measured isolated peak at one producer, which has no +consumer and so is not a rate any draining queue can sustain -- it is a +conservative floor on time-to-wrap. 33M/s is the measured drained rate at one +producer. 116M/s is the crate's own disclosed figure and is the honest planning +number. + +**The reservation half is where the bits are being spent, and the trade it makes +is a real one.** The field currently *encodes* up to four billion outstanding +reservations, which is a field ceiling rather than a reachable count: on +`Balanced` the ring's capacity binds first (at most 2^31 slots on a 64-bit +target, 2^30 on a 32-bit one), and a smaller queue binds it sooner still. +Narrowing the field is what buys the position bits: 2^21 reservations leaves +about a +day, 2^12 leaves over a year, and 2^8 leaves twenty years. That last is a plain +`AtomicU64`, so it reaches twenty years without a third-party dependency and +without reopening `D-18`'s i686 question -- against the 128-bit word's 5,039 at +the same rate. Both are finite and both scale with the caller's rate; which of +them is enough is a question about a deployment, not one this table answers. + +**An earlier version of this paragraph called the reservation half "the half +worth least", on the premise that outstanding reservations are bounded by how +many producers are mid-flight -- hundreds, perhaps thousands -- so that narrowing +the field gave up "reservations nobody will allocate". That premise is +withdrawn as false.** `Producer::reserve` takes `&self` and returns an owned +`Reservation`, so one producer can hold as many as the field allows: the bound is +the lesser of the ring capacity and the field, not a producer count. The queue +crate's `one_producer_alone_can_exhaust_the_reservation_field` fills `Perpetual`'s +255 from a single thread and is then refused. The arithmetic above is unaffected, +but what it costs is not free -- a caller holding many simultaneous reservations +is choosing against the narrower layouts. See +[D-41](../windows-waitable-queues/DESIGN-NOTES.md#d-41). + +**This paragraph previously added "at no measured cost", and that clause is +withdrawn** -- it was the same claim the layout section below withdrew, restated +a third time in a section about counter arithmetic rather than about speed. The +arithmetic above is unaffected, because time-to-wrap follows from the field width +and a rate, not from a measurement of either layout; what does not follow is any +statement about what the re-apportionment costs to run. See +[Re-measured on the shipping type](#d-queue-layout-observations). + +So the arithmetic separates the rows this way: every row recurs, and what changes +down the column is how long that takes at a given rate -- 16/48 at 12.7 days +against 12/52 at 202 days, at the conservative floor. Which of +them a caller wants is the caller's question, +and the shipping type takes the layout as a parameter so it stays theirs -- see +[D-no-client-prescriptions](#d-no-client-prescriptions). + +### Re-measured on the shipping type, with the probe's own control to read it against + + + +`CW-1.6` deleted the duplicated protocol in this crate once +`windows-waitable-queues` took the layout as a parameter, so the probe now +instantiates the real type at each layout. The numbers below supersede the ones +above, which were taken from the stand-in. + +**Read every figure here as one host's observation, not as a portable result.** +The capture parameters are the probe's own banner, reproduced in full because a +ratio without them is an anecdote rather than data someone else can use: + +``` +host: x86_64 16p/8c smt+ L2[2,2,2,2,2,2,2,2] ec[0:16] numa[16] +``` + +Seven runs, median of the per-run ratios with the observed range beside it, +release build. **The sampling parameters are capture parameters too**: each run +is a whole probe invocation, within which every configuration is measured five +times and the median reported, each measurement being 50,000 pushes per producer +thread, preceded by one untimed pass. That pass does **not** pre-touch any +allocation a timed pass will use -- every repetition builds and drops its own +queue -- so what it warms is process state: the allocator's size class, the OS +page cache, the instruction cache and the branch predictors. So a figure below +rests on 35 timed passes per +configuration, and "seven runs" alone would not let anyone reproduce it. These +are fixed at +[src/queue_contention.rs](src/queue_contention.rs)`::PUSHES_PER_PRODUCER` and +`REPETITIONS`; M4.2 in [CHECKLIST.md](CHECKLIST.md) makes them adjustable, which +is what the first diagnostic step above needs and cannot currently do. + +**The banner's `numa[16]` is load-bearing here: it means a single +NUMA node holding all sixteen processors**, so every figure below was taken +inside one memory domain and says nothing about cross-domain behaviour. What is +not pinned down at all is memory configuration and BIOS state; NUMA *distances* +are unavailable on Windows by platform limit rather than by omission, and the +processor-to-node assignment in the banner is the analog this crate uses in their +place (see [A probe reports observations parameterized by their +capture](#d-observations-not-verdicts)). + +The layout is a *parameter* of the shipping type, so this note's job is to report +what this machine did and hand the reader the tooling -- the probe -- to measure +the machine they actually care about. It is not to pick a winner on their behalf. + +**The probe emits its own noise control, and it is the only honest yardstick for +these ratios.** The `reserving_mpsc` row and the `reserving(32/32)` row are the +same code at the same layout, measured twice in the same run, so their ratio is +what "no difference" looks like on this host: + +| regime | same-code control (`reserving_mpsc` vs `32/32`) | +|---|---| +| isolated | median 0.94-1.05x, observed 0.69-1.12x | +| drained | median 0.98-1.07x, observed 0.68-1.27x | + +**Amended 2026-09-16: the drained row measures a probe that no longer exists.** +`M4.3` added a readiness handshake, so producers now hold until the consumer +announces that it is draining rather than starting the moment the gate releases. +The row above was taken before that and is kept as what the earlier instrument +measured. A capture taken after it, on the same host, is in +[captures/2026-09-16-drained-handshake/](captures/2026-09-16-drained-handshake/README.md); +its control span is in that capture's `summary.txt` rather than restated here. +The two are not a like-for-like comparison of dispersion -- the row above spans +seven runs and the new capture three -- so the amendment records that the +measurement changed, not that the spread narrowed. The isolated row is +unaffected: those timers have no consumer, and so no handshake. + +So a ratio inside roughly 0.9-1.1x is indistinguishable from zero effect here, +and at sixteen and thirty-two producers the control alone wanders past 1.12x. + +**That control is far too wide, and saying so is part of reporting it.** Two +measurements of *the same code in the same run* should not differ by 27%, and +the same-configuration spread across seven runs reaches 61%. Used above as a +yardstick, this is the honest yardstick available -- but a yardstick this elastic +is first a defect report against the probe, not a fact about the queue. The cause +is not determined: it could be the probe measuring more than the variable under +test, a residual defect like the timing window already found and fixed here, too +few runs or too short a measured span, or simply that these are nanosecond-scale +measurements taken on a shared desktop that is doing other things. The dispersion +alone cannot distinguish them, and this note does not guess. See +[High variance in our own control is a finding about the +instrument](#d-variance-is-a-finding) for what to try first and how to recognise +the floor, and M4.2 in [CHECKLIST.md](CHECKLIST.md) for the probe controls that +make those steps executable without a source edit. + +What follows is therefore reported as *data with a known-unexplained spread*. + +**Isolated regime**, median of the per-run ratios with the observed range beside +it. The drained regime is reported in the paragraph below the table, and mixing +the two is the reading this label exists to prevent: + +| producers | 16/48 vs 32/32 (isolated) | 8/56 vs 32/32 (isolated) | 64/64 vs 32/32 (isolated) | +|---|---|---|---| +| 1 | 1.00x [0.74-1.00] | 1.00x [0.67-1.04] | 1.37x [1.16-1.57] | +| 2 | 0.94x [0.89-1.05] | 0.96x [0.80-0.98] | 1.13x [1.02-1.15] | +| 4 | 0.96x [0.83-1.03] | 1.00x [0.90-1.10] | 1.29x [1.14-1.36] | +| 8 | 1.01x [0.95-1.13] | 0.94x [0.92-1.08] | 1.82x [1.64-2.20] | +| 16 | 1.23x [1.09-1.35] | 1.26x [1.16-1.33] | **3.45x [2.91-4.27]** | +| 32 | 1.30x [1.15-1.41] | 1.28x [1.11-1.42] | **3.81x [2.70-4.31]** | + +In the drained regime nothing separates at all -- every u64 layout *and* the +128-bit word sit inside the control band at every producer count (the widest +median is 1.13x at one producer, against a control that reaches 1.27x). + +**Amended 2026-09-16: the isolated table above has a committed cross-check, and +it agrees.** The seven-run sweep's raw runs were not kept, which is the gap the +drained amendment below is about. The three-run capture taken for `M4.3` records +the isolated regime as well, and +[isolated.js](captures/2026-09-16-drained-handshake/isolated.js) derives it -- +so the isolated figures, unlike the drained ones, can be checked against a +second independent measurement. The two agree on 64/64 at every producer count: + +| producers | seven-run (above) | three-run capture | same-code control, capture | +|---|---|---|---| +| 1 | 1.37x [1.16-1.57] | 1.39x [1.32-1.74] | 0.96x [0.79-1.06] | +| 2 | 1.13x [1.02-1.15] | 1.10x [1.10-1.16] | 0.97x [0.96-1.00] | +| 4 | 1.29x [1.14-1.36] | 1.37x [1.36-1.70] | 1.04x [0.97-1.06] | +| 8 | 1.82x [1.64-2.20] | 1.67x [1.59-2.05] | 1.01x [0.99-1.12] | +| 16 | 3.45x [2.91-4.27] | 4.00x [3.94-4.10] | 1.09x [0.87-1.14] | +| 32 | 3.81x [2.70-4.31] | 4.77x [3.54-4.99] | 1.02x [0.94-1.07] | + +In the capture, all three runs put 64/64 above the control's whole observed +range at **every** producer count; 16/48 and 8/56 each do so at 16 and 32. +Three control observations per count is not a band, so this reports what these +runs did rather than what a fresh run would do. + +**Corrected 2026-09-17: the control column above was inverted when first +published.** `isolated.js` divided the control the other way round -- +`reserving(32/32)` over `reserving_mpsc`, while every layout ratio beside it +divides *by* `reserving(32/32)` -- so a reciprocal was tabulated as though it +were comparable. These ranges are not symmetric about 1.00, so the difference is +real: the reciprocal of `[0.95-1.26]` is `[0.79-1.05]`. It moved a reported +result, not just a column: 8/56 sits above its control at 16 producers as well as +32, where the inverted control had shown 32 alone. `summarise.js` computed its +control in the correct direction throughout, which is why the drained figures are +unaffected. Found by review. + +**An earlier reading of the small-count end said "near parity at one or two", +and that is withdrawn.** It was restated in six places across the queue crate -- +rustdoc, the crate doc, the README twice, `Cargo.toml`, and D-41 -- while the +table directly above it read 1.37x at one producer, and the capture reads 1.39x +against a control of 1.04x. Whatever 1.37x is, it is not parity, and the phrase +asserted an absence of difference that neither measurement shows. The six sites +now say the path was measured as slower at every producer count measured, +smallest at one or two. Found by review of the isolated figures against the +capture. + +**Amended 2026-09-16: re-measured after the `M4.3` handshake, and the +re-measurement does not settle it.** That paragraph was taken before producers +held for the consumer, so it describes a drained regime whose opening was +briefly undrained. Re-measured on the same host without that window, the figures +are in +[captures/2026-09-16-drained-handshake/](captures/2026-09-16-drained-handshake/README.md). + +**What the re-measurement establishes is less than first claimed here, and the +correction is worth stating plainly.** This amendment originally said every +layout median still sat inside the same-code control band. That rested on +pooling every control observation into one band, and the pooling is what +produced the answer: the control is not independent of producer count -- it +spans about 0.82-0.98x at one producer against 0.95-1.23x at thirty-two in that +capture -- so pooling builds a band wider than any count's own, and containment +follows from the method. Compared per count, several medians fall outside their +own count's range. Compared per count the other way, three runs give three +control observations, and the range of three samples is not a band to judge +anything against. + +So the drained comparison is **not established by three runs**, in either +direction. The capture reports the per-count figures and declines a verdict; +the pre-handshake reading above rests on the seven-run sweep, which this does +not replace. Reported by review, after the pooled framing had already been +published here. + +**The same caveat reaches the paragraph above, and saying so is the honest +scope of this correction.** That reading compares a per-count median against a +control quoted as a single pooled figure -- "the widest median is 1.13x at one +producer, against a control that reaches 1.27x" -- which is the same comparison +this amendment has just withdrawn for the three-run capture. Its raw runs are +not committed, so the per-count bands behind it cannot be recomputed here and +the conclusion is neither confirmed nor refuted. What can be said is that it +rests on the same framing, and that settling it needs the sweep re-run with its +data kept. + +**Widening the word is the one effect this probe establishes.** At sixteen and +thirty-two producers the isolated 128-bit rows fall outside the same-code +control band by a wide margin. The `u64` re-apportionments also sit outside it at +those counts, but too close to it to establish an ordering or a cost -- which is +the distinction the next paragraph withdraws the apportionment claim over, and +stating it as "they do not" contradicted both that paragraph and the capture. +That is a real effect on this machine, and its direction is mechanically +unsurprising -- `cmpxchg16b` against `lock cmpxchg`. Whether it reproduces on +another microarchitecture is a question for the probe, not for this note. + +**The apportionment claim is withdrawn, in both directions.** This section +previously said the `u64` re-apportionments "track the default within noise" and +that twenty years of headroom is therefore "free". That was asserted from a +single run against a noise floor quoted as 2-6%, and neither half holds: the +measured control is far wider than 2-6%, and the re-apportionments do not sit +inside it at sixteen and thirty-two producers. But the replacement is *not* the +opposite claim. 1.23-1.30x against a control that itself reaches 1.12x is a +flag, not a finding -- what it records is that this probe, on this host, at seven +runs, could not separate the layouts at high producer counts. Nothing here +establishes an ordering between them, in either direction. This is +[the rule for what this crate concludes](#d-observations-not-verdicts) applied to +the case that earned it. + +**The residual offset is gone, which is the point of the deletion.** The +duplicate ran about 1.26x slower than `reserving_mpsc` at high producer counts, +an error that had to be carried as a caveat on every figure. That the same-code +control now sits on 1.00x is what says the offset is gone -- and building that +control into the probe's output, rather than asserting a noise floor in prose, +is what let every ratio above be read honestly. + +The general lesson is worth keeping even though the duplicate is gone: +**a stand-in is only evidence about the thing it stands in for while something +checks that it still does.** This one was checked, which is how the missing +cache padding was caught; but the checking only ever bounded the error, and the +bound was loose enough to hide a third of the wide word's cost. + +A second lesson the correction above earned: **a ratio means nothing without the +dispersion of the thing it is a ratio of.** Two runs cannot measure a spread, so +quoting one to two decimal places invites exactly the over-reading that produced +the withdrawn claim. Where this note gives a ratio it now gives the range too. + ## The report is buffered, and what that costs @@ -539,18 +1233,28 @@ The choice was between giving `Report` a method taking `fmt::Arguments` (with a `report_line!` macro), implementing `fmt::Write` on a sink so existing `writeln!` calls keep working, and keeping the `String` while flushing it at line boundaries. **It was decided by counting rather than by taste.** Every -renderer already writes through `writeln!(out, ...)` against a `String`'s -`fmt::Write`, at **332 sites** in this crate; only 18 functions take the `&mut -String` those sites write into. A sink method would have been the most explicit -option and would have rewritten all 332; `fmt::Write` moves the 18 and leaves -the 332 untouched, because `String` implements `fmt::Write` too and the call -sites cannot tell the difference. - -Worth recording that M1 estimated "upwards of 160" of those sites. The real -figure is twice that, and it is the whole of the argument -- an option whose -cost is "rewrite every call site" is affordable at 160 and is not at 332. A -plan's estimate is worth re-measuring at the moment it becomes a decision. - +renderer already wrote through `writeln!(out, ...)` against a `String`'s +`fmt::Write`, at far more sites than there were functions taking the `&mut +String` they wrote into. A sink method would have been the most explicit option +and would have rewritten every one of those write sites; `fmt::Write` moves the +signatures and leaves the write sites untouched, because `String` implements +`fmt::Write` too and the call sites cannot tell the difference. + +**The census as it stood when the decision was taken, on 2026-09-09: 332 write +sites against 18 signatures.** It is recorded as what was measured that day, not +as a description of the crate now, and the distinction earns its place here. The +same passage in the sink's own doc comment first carried 504 sites and "about +twenty" functions; the edit that revised those figures is the same edit that +dropped a sentence out of that comment and left it reading "is arithmetic" with +nothing before it. Keeping the census true by hand is what broke the prose +around it. The relation above is what the decision actually turned on, and it +cannot rot. + +Worth recording that M1 estimated "upwards of 160" of those sites, and the +measured figure was roughly twice that. The gap is the whole of the argument -- +an option whose cost is "rewrite every call site" is affordable at 160 and is +not at twice that. A plan's estimate is worth re-measuring at the moment it +becomes a decision. ### What the adapter has to reassemble, and why that is not a detail `fmt::Write` is **line-agnostic**: `write_str` receives whatever slices the diff --git a/crates/windows-platform-probes/DESIGN-RATIONALE.md b/crates/windows-platform-probes/DESIGN-RATIONALE.md index 2ae0566e9..034557fd9 100644 --- a/crates/windows-platform-probes/DESIGN-RATIONALE.md +++ b/crates/windows-platform-probes/DESIGN-RATIONALE.md @@ -528,3 +528,162 @@ about THEM. The instruments remain exactly as good as the hand-sabotage that built them -- which is where several of this branch's defects were found, and where the next one will be. A clean sweep is evidence about the oracle, not about the things measuring it. + +## Why the crate reports observations instead of verdicts + +Recorded for [D-observations-not-verdicts](DESIGN-NOTES.md#d-observations-not-verdicts). + +The rule was earned, not designed. The queue-contention note had carried the +claim that re-apportioning a queue's position/reservation bits was **free** -- +that 16/48 and 8/56 "track the default within noise" -- and therefore that buying +twenty years of counter headroom cost nothing. Two independent defects sat under +that sentence. + +The first was arithmetic-shaped: the ratios in the table directly beneath it were +several times the noise floor the same document quoted. The prose +contradicted its own evidence, in adjacent lines, and survived several review +passes anyway -- because "within noise" reads as a conclusion rather than as a +claim about a measured quantity, so nobody checked it against the number. + +The second was deeper. That floor had itself been obtained by comparing **two +runs**, which cannot measure a spread at all. Seven runs of the same probe put +the same-configuration spread roughly an order of magnitude wider. So the +floor every "within noise" judgement in the section had been made against was +wrong by about that much, and the judgements were not recoverable by +adjusting it. + +**The figures are deliberately not repeated here.** Both tables live in +[DESIGN-NOTES.md](DESIGN-NOTES.md#d-variance-is-a-finding), which is the one home +for them, and the seven-run sweep's own gap -- its raw runs were never committed +-- is recorded there rather than in a second copy. A rationale that restated the +numbers would be a second surface to keep true by hand, which is the defect this +very section is explaining. + +What made the repair possible was already in the probe's output. `reserving_mpsc` +and `reserving(32/32)` are the same code at the same layout, measured twice per +run, so their ratio is an *empirical* answer to "what does no difference look +like here". That is a control the instrument +derives rather than a floor the prose asserts, which is +[D-derived-not-restated](DESIGN-NOTES.md#d-derived-not-restated) applied to a +measurement instead of to a fact. Its measured span is in the control table in +[DESIGN-NOTES.md](DESIGN-NOTES.md#d-variance-is-a-finding), which is where it is +recorded rather than here. + +**The tempting repair was to invert the claim**, since at high producer counts +the seven-run medians put the re-apportionments above the same-code control's +own excursions. That would have been +the same error with the opposite sign: one host, one microarchitecture, a single +NUMA domain, against a control wide enough that the gap is not an ordering. The +claim was +withdrawn in both directions instead, and the section now says which +configuration is worth measuring locally rather than what the answer is. + +This generalises to where the crate draws its line. Coarse claims that follow +from how the hardware works *and* are backed by observation -- "the buffers +should be in the same memory domain as the executor" -- are worth making, and +portable enough to be useful. Fine-grained topological and layout choices are +not: they depend on parameters the capture does not record and the reader's +machine does not share. The shipping queue takes its layout as a type parameter +precisely so that choice belongs to the client; a design note that quietly picks +one on their behalf takes it back. + +**On the capture parameters themselves.** Windows exposes no NUMA distance table, +so "how far apart are these nodes" is unanswerable on this platform. The analog +the crate uses is the processor-to-node assignment carried in the banner's +`numa[...]` field, with device-to-node mapping available on the same footing. It +answers the same-domain question, which is what most placement decisions turn on, +and it is why `numa[16]` on the measurement host is worth stating plainly: a +single domain means the queue figures say nothing about cross-domain behaviour at +all. + +### Why a wide control is logged as a defect rather than absorbed + +Recorded for [D-variance-is-a-finding](DESIGN-NOTES.md#d-variance-is-a-finding). + +The queue-contention repair produced a genuinely useful artifact -- a noise +control the probe derives rather than asserts, built from two rows that are the +same code at the same layout. It immediately did its job, withdrawing a claim +that had survived several reviews. + +It also very nearly produced a second error. Having measured the control's span, +the natural next move is to use it: judge every ratio against that +band, mark what falls outside, and report the result. That is what the first +draft of the section did. But two measurements of identical code in the same run +differing by as much as they did is not a fact about the queue at all -- it is +the instrument +telling you something, and using it as a ruler while declining to ask why it is +elastic is how a methodological problem becomes permanent. The control had been +promoted from *symptom* to *tool* without anyone deciding to do that. + +The causes are not separable from the dispersion itself, which is precisely why +the decision refuses to guess among them. The list is short and every entry is +plausible here: the probe may be moving more than the variable under test; it may +carry a residual defect, as it demonstrably did until the timing window was +corrected; seven runs of a ~65-second probe may simply be too few; or a +nanosecond-scale measurement on a shared desktop running everything else may be +dominated by the machine. + +**The ordering of the diagnosis is the practical content, and it follows from +cost rather than from likelihood.** Lengthening the span and raising the +repetition count is the only step that changes nothing about what is being +measured -- so it is the only step whose result is interpretable before the +others have been tried. Pinning threads, quiescing the machine, or altering the +probe all move the measurement as well as the noise, and a change made ahead of +the cheap check cannot be evaluated against anything. That this also happens to +be the least effortful step is a convenience, not the reason. + +**That ordering has since been revised, and the revision is in +[CHECKLIST.md](CHECKLIST.md): `M4.4` comes before `M4.2`.** The argument above +holds only if the control is comparable to the candidate, and in this probe it is +not: `measure()` runs the control roughly four configurations away from the row +it is a control for, so a lengthened run still carries whatever drifts across +that distance. Lengthening an unpaired control buys a narrower interval around a +quantity that is still confounded by sequence, which is not an interpretable +result -- so interleaving the control with its candidate has to land first. The +paragraph above is kept as the reasoning that was current when the diagnosis was +first written down; where the two disagree, the checklist is the execution order. + +The other half is knowing when to stop. Every setup has a floor, and past it more +runs buy nothing; the failure mode is a week spent establishing that two numbers +are the same. What makes the floor easy to misjudge is the assumption that it +scales with the measured value -- that small numbers are inherently noisy. It can +just as well be set by the sampling regime: clock granularity, how many +independent samples the run takes, how the span is constructed. This probe takes +two timestamps per worker per repetition and divides by the pass, so what limits +resolution at small values is the number of passes, not a fraction of the +nanoseconds printed. The distinction matters because the two readings prescribe +opposite actions -- one says the measurement is hopeless, the other says take +more samples. + +**A warmup pass looks like it contradicts the "some noise is inherent" position, +and the objection is worth answering rather than smoothing over.** If dispersion +were genuinely inherent to a shared machine, warming could not remove it -- so +proposing a warmup appears to concede that it is really an artifact after all. +The resolution is that the two are different quantities that happen to widen the +same spread. A warmup addresses *transients*: page faults on a fresh allocation, +cold caches and predictors, frequency ramp -- all front-loaded, all one-time, none +a property of the steady state. Contention with other tenants is not front-loaded; +it continues for the whole run and no amount of warming touches it. Removing the +transients therefore does not hide the inherent floor, it uncovers it, which is +why the expected signature is a spread that narrows as warming and length +increase and then stops narrowing. The plateau is the inherent part. This probe +already discards one untimed pass -- which warms process and allocator state +rather than the timed allocation, since each repetition builds its own queue -- +so part of this is done and the rest is unmeasured. + +**The calibration is the part worth writing down, because it is not obvious and +it cuts both ways.** A spread like this in a benchmark or a marketing document +would be fatal -- such documents exist to carry a comparative claim, and a +comparative claim resting on a control this wide is simply unsupported. But this +crate's figures exist to support *planning for deployment environments like the +measured one*, and the measured one is an ordinary machine under ordinary load. +Data gathered there is not obviously the wrong input for planning there. So the +answer is neither suppression nor promotion: publish it, publish the dispersion, +and publish the fact that the dispersion is unexplained. A reader can then weigh +it for their own purpose, which is the same principle as +[D-observations-not-verdicts](DESIGN-NOTES.md#d-observations-not-verdicts) applied +to the quality of the measurement instead of to its portability. + +The rule exists to forbid the quiet version: reporting a wide control as though a +wide control were ordinary. It is not ordinary, and the moment it stops being +remarked upon is the moment nobody investigates it. diff --git a/crates/windows-platform-probes/captures/2026-09-16-drained-handshake/README.md b/crates/windows-platform-probes/captures/2026-09-16-drained-handshake/README.md new file mode 100644 index 000000000..d824500af --- /dev/null +++ b/crates/windows-platform-probes/captures/2026-09-16-drained-handshake/README.md @@ -0,0 +1,96 @@ +# Queue contention, drained regime, after the M4.3 readiness handshake + +Three whole-probe invocations taken to answer one question: does closing the +undrained opening at the start of each drained run change what the drained rows +say? + +**This is a second capture, not a replacement for the first.** The figures in +[DESIGN-NOTES.md](../../DESIGN-NOTES.md) that predate `M4.3` measured a probe +whose producers could begin pushing before the consumer had run at all. These +measured a probe where no producer begins timing until the consumer has executed +its pop path at least once. That is the guarantee, stated exactly: continuous +draining is not guaranteed and no flag could express it, since the consumer can +be descheduled afterwards as it can at any point in the run. Both are real +measurements of two different pieces of code, so they are kept side by side and +each is labelled with the instrument that produced it. + +## Attribution + +| | | +|---|---| +| Host | `x86_64 16p/8c smt+ L2[2,2,2,2,2,2,2,2] ec[0:16] numa[16]` | +| Profile | release | +| Sampling | 50,000 pushes per producer, median of 5 repetitions, one untimed warmup pass | +| Runs | 3 whole-probe invocations, in [run1.txt](run1.txt), [run2.txt](run2.txt), [run3.txt](run3.txt) | +| Instrument | `probe-queue-contention`, built from `68198359` (the commit that made the consumer drain once before announcing readiness) | +| Taken | 2026-09-16 18:12 UTC-04:00 | + +The host is the same machine as the capture the crate README carries, so the two +are comparable; nothing here says anything about any other hardware, and the +banner's `numa[16]` is a single node holding all sixteen processors. + +The runs' regime banner reads `a consumer popping continuously`, which is what +`68198359` printed; a later commit changed that label to `a consumer looping on +pop`, because the handshake guarantees the consumer's pop path has run once, not +that it is scheduled without gaps. A later commit also rewrote the paragraph +under the comparison table, which used to read the point estimate as a verdict +and now directs the reader to the interval. + +Both are prose the probe prints around its tables, not measurements, so the runs +below are not retaken for them: nothing about what was measured moved. Expect the +committed runs to differ from a fresh one in wording of this kind, and compare +the figures rather than the surrounding text. + +## Reading it + +[summary.txt](summary.txt) is the output of [summarise.js](summarise.js) over the +three runs, regenerated with: + +``` +node summarise.js run1.txt run2.txt run3.txt +``` + +[isolated.txt](isolated.txt) is the output of [isolated.js](isolated.js) over the +same three runs, regenerated with: + +``` +node isolated.js run1.txt run2.txt run3.txt +``` + +The two cover different regimes and are kept apart for that reason: `summarise.js` +derives the **drained** tables, `isolated.js` the **isolated** ones. + +The isolated figures matter beyond this directory because the queue crate's +documentation makes a claim about them -- that the whole push path was measured +as slower under `Wide` at every producer count. That claim rests on a **separate** +seven-run sweep whose raw runs were never committed, and on the crate's own +attributed table, which is a **third** capture built from `fecd352`. This +directory is neither of those: it is three runs from `68198359`, and what +`isolated.js` provides is an independent cross-check that can actually be run, +against figures that otherwise have none. Where this and the crate's table +disagree, the crate's table is the attributed figure for that crate; this one is +evidence about how far such a figure moves. + +Both are committed so the derivation can be checked rather than taken on trust, and +so nothing downstream has to retype a figure. The scripts derive what the runs do +not state individually: the across-run median per producer count, and the +same-code control -- which is a relation *between* two tables, since +`reserving_mpsc` in the comparison table and `32/32` in the layout table are the +same configuration measured twice in the same run. + +**It reports per producer count and emits no verdict, deliberately.** An earlier +version pooled every control observation into one band and asked whether each +layout median fell inside it. It answered `true`, and the pooling is what +produced that answer: the control is not independent of producer count -- about +0.82-0.98x at one producer against 0.95-1.23x at thirty-two here -- so a pooled +band is wider than any count's own, and containment follows from the method +rather than from the data. Comparing per count does not rescue a verdict either, +because three runs give three control observations per count, and the range of +three samples is not a band to judge anything against. + +So three runs do not settle the drained comparison in either direction. This +capture reports figures; the claim that nothing separates in the drained regime +is the **pre-handshake** reading, resting on the seven-run sweep, which this does +not replace and which measured a probe `M4.3` has since changed. Carrying that +reading forward as a current statement about the drained regime is the thing this +paragraph exists to prevent. diff --git a/crates/windows-platform-probes/captures/2026-09-16-drained-handshake/isolated.js b/crates/windows-platform-probes/captures/2026-09-16-drained-handshake/isolated.js new file mode 100644 index 000000000..1c523a219 --- /dev/null +++ b/crates/windows-platform-probes/captures/2026-09-16-drained-handshake/isolated.js @@ -0,0 +1,276 @@ +// Copyright (c) Mike Grier. + +// Isolated regime: each claim-word layout against the default, read against a +// same-code control. +// +// `summarise.js` beside this derives the DRAINED tables only. The isolated +// figures are cited too -- by the queue crate's docs, which say the whole push +// path was measured as slower under `Wide` at every producer count -- so they +// need a derivation a reader can run, not a number taken on trust. +// +// The control is the point. `reserving_mpsc` and `reserving(32/32)` are the SAME +// CODE under two names, so the gap between them is what "no difference" looks +// like on this host; a layout ratio only means something read against it. +// +// Three runs give three control observations per count. The range of three +// samples is NOT a band -- a fresh draw falls outside the range of three priors +// about half the time by construction -- so the last column reports what these +// runs did, and is not a claim about what the next run would do. +// +// Usage: node isolated.js run1.txt run2.txt run3.txt +"use strict"; + +const fs = require("fs"); +const crypto = require("crypto"); + +// One writer for the generated artifact, per this repository's output rule: no +// formatting site picks a destination, so retargeting the report to a file is a +// change here and nowhere else. `fail` is the diagnostic path and stays separate +// from the artifact, which is why they are two sinks rather than one with a flag. +let sink = (text) => process.stdout.write(text + "\n"); +const out = (text = "") => sink(text); +const fail = (text) => process.stderr.write(text + "\n"); + +const files = process.argv.slice(2); +if (files.length === 0) { + fail("usage: node isolated.js [run.txt ...]"); + process.exit(2); +} +// `files.length` is reported as the number of independent runs, so the same +// path given twice would publish a two-run comparison -- with a zero-width +// control range -- derived from one set of bytes. A typo must not make the +// capture look more reproducible than it is. +{ + // Compared by CONTENT, not by name. A resolved path catches `./run1.txt` and + // a symlink, but not `copy-of-run1.txt`, which has a different real path and + // identical bytes -- and the run count would still present it as a second + // observation, narrowing the reported range without adding data. What is being + // claimed here is independent runs, so identical bytes cannot be two of them. + // + // **This is a heuristic, and its assumption is worth stating.** The report + // carries no per-invocation identifier, so "identical bytes" stands in for + // "same run". Two genuinely independent runs producing identical bytes would + // be refused -- possible in principle, since the figures are rounded, and + // vanishingly unlikely across this many of them. The refusal is loud and + // diagnosable; accepting a duplicated run would silently fabricate agreement, + // which is the worse of the two. `M4.8` replaces the heuristic with a real + // identity once the report carries one. + const seen = new Map(); + for (const file of files) { + const digest = crypto.createHash("sha256").update(fs.readFileSync(file)).digest("hex"); + if (seen.has(digest)) { + fail(`the same capture was given twice: ${file} is byte-identical to ${seen.get(digest)}`); + process.exit(2); + } + seen.set(digest, file); + } +} + + +// The attribution a run prints about itself. A capture is runs of one build on +// one host under one sampling regime, so these must agree across the inputs -- +// mixing a debug run, or another machine's, yields a median and a control span +// that describe no configuration that was ever measured, and nothing else here +// would notice. +function attribution(text, where) { + const line = (prefix) => { + const found = text.split(/\r?\n/).find((l) => l.startsWith(prefix)); + if (found === undefined) { + fail(`${where}: no "${prefix}" line; this does not look like a probe report`); + process.exit(2); + } + return found.trim(); + }; + const profile = line("profile:"); + // Agreement is not enough on its own: three debug reports agree with each + // other, and the probe stamps a debug run "NOT A MEASUREMENT" precisely + // because its figures are not one. A capture built from them would be + // internally consistent and meaningless. + if (profile !== "profile: release") { + fail(`${where}: ${profile} -- only a release run is a measurement`); + process.exit(2); + } + // Included because it can differ while the host banner does not: an affinity + // mask changes how many processors the process may use without changing the + // machine it names, and producer counts are read against that number. + return [line("host:"), profile, line("sampling:"), line("processors available to this process:")].join( + " | ", + ); +} + +function requireOneConfiguration(entries) { + // **What this cannot check.** The report carries no build identity, so two + // runs of DIFFERENT probe commits on one host under one profile agree here. + // That matters most for exactly this capture: `M4.3` changed the drained + // procedure, so a pre-handshake and a post-handshake run would pass and have + // their medians combined as though one procedure produced both. The + // instrument commit is asserted by the capture README, which is a claim by + // the person who took the capture rather than something these scripts verify. + // Closing it needs the probe to stamp its own build identity into the report; + // `M4.8` in CHECKLIST.md owns that. + const first = entries[0]; + for (const entry of entries.slice(1)) { + if (entry.attribution !== first.attribution) { + fail( + `${entry.name} was taken under different conditions from ${first.name}:\n` + + ` ${first.name}: ${first.attribution}\n` + + ` ${entry.name}: ${entry.attribution}`, + ); + process.exit(2); + } + } +} + +const COUNTS = [1, 2, 4, 8, 16, 32]; +const DEFAULT_LAYOUT = "reserving(32/32)"; +// Same code as DEFAULT_LAYOUT, under the shipping type's own name. +const CONTROL_TWIN = "reserving_mpsc"; +const LAYOUTS = ["reserving(16/48)", "reserving(8/56)", "reserving(64/64)"]; + +// The isolated raw table only. The drained table repeats every shape name, so a +// whole-file scan would silently average the two regimes together. +function isolatedRows(file) { + const lines = fs.readFileSync(file, "utf8").split("\n"); + const start = lines.findIndex((l) => l.startsWith("-- isolated:")); + const end = lines.findIndex((l) => l.startsWith("-- drained:")); + if (start < 0 || end < 0 || end <= start) { + throw new Error(`${file}: expected an isolated marker followed by a drained one`); + } + const rows = new Map(); + for (const line of lines.slice(start, end)) { + const m = line.match(/^(\S+)\s+(\d+)\s+([\d.]+)\s/); + if (m) { + const key = `${m[1]}@${m[2]}`; + // A `Map` keeps the last write. A run carrying the whole expected producer + // set plus one duplicated row would pass every completeness check while + // one measurement was silently discarded in favour of another. + if (rows.has(key)) { + throw new Error(`${file}: a second isolated row for ${key}`); + } + rows.set(key, Number(m[3])); + } + } + // Every shape this script reads must appear at exactly the swept producer + // counts, and no *layout* it does not read may appear at all. `ratios()` looks + // up only the counts in `COUNTS` for only the shapes it knows, so both an + // extra count and an unrecognised layout would sit in the map unread while the + // summary reported success over a capture it had only partly used. + // + // Scoped to `reserving(...)` rows deliberately: the isolated table also + // carries `baseline_fetch_add`, `slotwise_mpsc` and `permit_mpsc`, which this + // script does not derive from and which are not anomalies. An earlier version + // rejected every unknown shape and refused the committed capture on its first + // row. + const known = [DEFAULT_LAYOUT, CONTROL_TWIN, ...LAYOUTS]; + for (const key of rows.keys()) { + const shape = key.slice(0, key.lastIndexOf("@")); + if (shape.startsWith("reserving(") && !known.includes(shape)) { + throw new Error(`${file}: unexpected claim-word layout ${shape}`); + } + } + for (const shape of known) { + const seen = [...rows.keys()] + .filter((k) => k.startsWith(`${shape}@`)) + .map((k) => Number(k.slice(shape.length + 1))) + .sort((a, b) => a - b); + if (seen.length !== 0 && seen.join(",") !== COUNTS.join(",")) { + throw new Error( + `${file}: ${shape} covers producers [${seen}], expected [${COUNTS}]`, + ); + } + } + return rows; +} + +requireOneConfiguration( + files.map((f) => ({ name: f, attribution: attribution(fs.readFileSync(f, "utf8"), f) })), +); +const tables = files.map(isolatedRows); + +function ratios(numerator, denominator) { + const out = new Map(); + for (const n of COUNTS) { + out.set( + n, + tables.map((t, i) => { + const a = t.get(`${numerator}@${n}`); + const b = t.get(`${denominator}@${n}`); + // A missing or unusable row is an error, not a skipped count: silently + // dropping one would quietly narrow every range printed below. + // Both operands must be positive, not merely finite and the denominator + // non-zero: a zero or negative numerator divides cleanly and publishes + // an ordinary-looking `0.00x`. The probe's did-not-run sentinel used to + // be `0.0`, so that is the shape a legacy capture actually takes. + if (!Number.isFinite(a) || !Number.isFinite(b) || a <= 0 || b <= 0) { + throw new Error(`${files[i]}: no usable ${numerator}/${denominator} at ${n} producers`); + } + return a / b; + }), + ); + } + return out; +} + +// The middle of an odd set, and the mean of the two middle values otherwise -- +// matching `summarise.js`. The CLI takes any number of runs, and picking the +// upper-middle for an even set would report the slower of two runs as their +// median, which is a different statistic under the same name. +const median = (xs) => { + const sorted = [...xs].sort((a, b) => a - b); + const mid = Math.floor(sorted.length / 2); + return sorted.length % 2 === 1 ? sorted[mid] : (sorted[mid - 1] + sorted[mid]) / 2; +}; +const fmt = (x) => x.toFixed(2); +const cell = (xs) => `${fmt(median(xs))}x [${fmt(Math.min(...xs))}-${fmt(Math.max(...xs))}]`; + +// Same code under two names. The denominator is `DEFAULT_LAYOUT`, exactly as it +// is for every layout ratio above -- a control read against ratios computed the +// other way round is not a control, because these ranges are not symmetric about +// 1.00 and the reciprocal of [0.95-1.26] is [0.79-1.05]. `summarise.js` computes +// its control in this same direction. +const control = ratios(CONTROL_TWIN, DEFAULT_LAYOUT); +const measured = new Map(LAYOUTS.map((l) => [l, ratios(l, DEFAULT_LAYOUT)])); + +// Widths derived from the cells, not fixed. `cell()` renders a median and a +// range of measured ratios, which have no upper bound, so a fixed field shifts +// every column after it the first time a value outgrows it -- the same argument +// `column_width` makes for the Rust report's columns. +const COLUMNS = ["control", ...LAYOUTS.map((l) => l.replace("reserving", ""))]; +const body = COUNTS.map((n) => [cell(control.get(n)), ...LAYOUTS.map((l) => cell(measured.get(l).get(n)))]); +// The widths this table has always used, kept as floors so the committed +// output is unchanged; the derivation only ever widens. +const width = COLUMNS.map((name, column) => + Math.max(18, name.length, ...body.map((row) => row[column].length)), +); +const PRODUCERS_WIDTH = Math.max(9, ...COUNTS.map((n) => String(n).length)); + +out(`isolated regime, ${files.length} run(s): ${files.join(", ")}`); +out(`each layout against ${DEFAULT_LAYOUT}; control is ${CONTROL_TWIN} against ${DEFAULT_LAYOUT}`); +out("median of the per-run ratios, with the observed range beside it\n"); + +out( + ["producers".padEnd(PRODUCERS_WIDTH + 2), ...COLUMNS.map((name, i) => name.padEnd(width[i] + 2))].join(""), +); +COUNTS.forEach((n, row) => { + out( + [ + String(n).padEnd(PRODUCERS_WIDTH + 2), + ...body[row].map((c, i) => c.padEnd(width[i] + 2)), + ].join(""), + ); +}); + +out("\nwhere every run sat above the control's whole observed range:"); +for (const l of LAYOUTS) { + const above = COUNTS.filter((n) => { + const top = Math.max(...control.get(n)); + return measured.get(l).get(n).every((x) => x > top); + }); + out(` ${l.padEnd(18)} ${above.length ? above.join(", ") + " producers" : "no producer count"}`); +} + +out( + `\nThe control's range here is ${files.length} observation(s) per count, which is not\n` + + "a band. This reports what these runs did; it does not establish that a fresh\n" + + "run would land the same way.", +); diff --git a/crates/windows-platform-probes/captures/2026-09-16-drained-handshake/isolated.txt b/crates/windows-platform-probes/captures/2026-09-16-drained-handshake/isolated.txt new file mode 100644 index 000000000..ba9089eed --- /dev/null +++ b/crates/windows-platform-probes/captures/2026-09-16-drained-handshake/isolated.txt @@ -0,0 +1,20 @@ +isolated regime, 3 run(s): run1.txt, run2.txt, run3.txt +each layout against reserving(32/32); control is reserving_mpsc against reserving(32/32) +median of the per-run ratios, with the observed range beside it + +producers control (16/48) (8/56) (64/64) +1 0.96x [0.79-1.06] 0.95x [0.94-1.00] 0.97x [0.95-1.00] 1.39x [1.32-1.74] +2 0.97x [0.96-1.00] 0.96x [0.93-0.97] 0.91x [0.90-0.95] 1.10x [1.10-1.16] +4 1.04x [0.97-1.06] 0.96x [0.86-1.02] 0.96x [0.93-0.99] 1.37x [1.36-1.70] +8 1.01x [0.99-1.12] 0.95x [0.95-1.11] 0.92x [0.88-1.14] 1.67x [1.59-2.05] +16 1.09x [0.87-1.14] 1.25x [1.18-1.29] 1.27x [1.14-1.29] 4.00x [3.94-4.10] +32 1.02x [0.94-1.07] 1.35x [1.34-1.42] 1.43x [1.26-1.45] 4.77x [3.54-4.99] + +where every run sat above the control's whole observed range: + reserving(16/48) 16, 32 producers + reserving(8/56) 16, 32 producers + reserving(64/64) 1, 2, 4, 8, 16, 32 producers + +The control's range here is 3 observation(s) per count, which is not +a band. This reports what these runs did; it does not establish that a fresh +run would land the same way. diff --git a/crates/windows-platform-probes/captures/2026-09-16-drained-handshake/run1.txt b/crates/windows-platform-probes/captures/2026-09-16-drained-handshake/run1.txt new file mode 100644 index 000000000..e1e2f63e4 --- /dev/null +++ b/crates/windows-platform-probes/captures/2026-09-16-drained-handshake/run1.txt @@ -0,0 +1,209 @@ +host: x86_64 16p/8c smt+ L2[2,2,2,2,2,2,2,2] ec[0:16] numa[16] +== how does the array queue's push path scale with producer count? == + +processors available to this process: 16 + +profile: release +sampling: 50000 pushes per producer, median of 5 repetitions, one untimed warmup pass + +-- isolated: producers only, capacity large enough that nothing is refused -- +shape producers ns/op ops/sec refusals ns/op range spread +baseline_fetch_add 1 2.3 432900433 0 2.2-2.3 1.04x +slotwise_mpsc 1 5.9 169204738 0 5.9-6.4 1.08x +reserving_mpsc 1 5.5 182415177 0 5.4-5.9 1.10x +permit_mpsc 1 7.9 127097102 0 7.8-7.9 1.01x +reserving(32/32) 1 5.7 175870559 0 5.4-5.9 1.09x +reserving(16/48) 1 5.4 186289121 0 5.3-5.7 1.06x +reserving(8/56) 1 5.4 185735513 0 5.4-5.8 1.07x +reserving(64/64) 1 7.5 134120172 0 7.4-8.0 1.08x +baseline_fetch_add 2 11.8 84774500 0 10.7-12.8 1.19x +slotwise_mpsc 2 57.5 17395237 0 56.9-63.7 1.12x +reserving_mpsc 2 34.1 29330674 0 32.6-37.9 1.16x +permit_mpsc 2 39.7 25160398 0 17.9-42.5 2.38x +reserving(32/32) 2 35.0 28573878 0 30.2-37.2 1.23x +reserving(16/48) 2 33.7 29638411 0 33.2-36.7 1.11x +reserving(8/56) 2 33.2 30110506 0 32.0-38.0 1.19x +reserving(64/64) 2 40.5 24717601 0 37.6-41.0 1.09x +baseline_fetch_add 4 14.9 67098333 0 14.1-15.9 1.12x +slotwise_mpsc 4 87.3 11450163 0 85.0-89.9 1.06x +reserving_mpsc 4 37.8 26456776 0 34.9-39.2 1.12x +permit_mpsc 4 30.7 32614722 0 30.1-89.4 2.97x +reserving(32/32) 4 36.3 27581261 0 35.4-39.6 1.12x +reserving(16/48) 4 37.1 26989825 0 34.0-40.0 1.18x +reserving(8/56) 4 35.0 28553481 0 30.6-40.9 1.34x +reserving(64/64) 4 61.8 16178743 0 49.2-64.3 1.31x +baseline_fetch_add 8 14.9 66969144 0 14.6-15.3 1.05x +slotwise_mpsc 8 132.4 7550033 0 129.2-147.7 1.14x +reserving_mpsc 8 38.4 26065764 0 36.0-49.9 1.39x +permit_mpsc 8 24.4 40931603 0 22.1-27.1 1.23x +reserving(32/32) 8 38.8 25798296 0 34.6-40.4 1.17x +reserving(16/48) 8 37.0 27019359 0 33.9-45.9 1.35x +reserving(8/56) 8 35.6 28093439 0 34.8-41.2 1.18x +reserving(64/64) 8 61.7 16203319 0 54.1-88.1 1.63x +baseline_fetch_add 16 14.7 67896153 0 14.6-14.8 1.02x +slotwise_mpsc 16 250.7 3988420 0 193.4-270.0 1.40x +reserving_mpsc 16 45.8 21854759 0 44.1-52.6 1.19x +permit_mpsc 16 22.4 44640117 0 21.1-23.8 1.13x +reserving(32/32) 16 52.9 18894528 0 50.7-55.8 1.10x +reserving(16/48) 16 62.2 16083150 0 59.8-69.2 1.16x +reserving(8/56) 16 68.0 14702963 0 66.4-74.4 1.12x +reserving(64/64) 16 216.7 4615734 0 206.7-218.5 1.06x +baseline_fetch_add 32 14.8 67525079 0 14.5-15.3 1.05x +slotwise_mpsc 32 227.3 4400348 0 223.9-268.5 1.20x +reserving_mpsc 32 53.0 18873756 0 51.0-56.1 1.10x +permit_mpsc 32 19.9 50292798 0 19.8-20.0 1.01x +reserving(32/32) 32 49.4 20238050 0 46.9-52.0 1.11x +reserving(16/48) 32 66.0 15141493 0 64.5-75.7 1.17x +reserving(8/56) 32 70.6 14172549 0 62.7-72.4 1.15x +reserving(64/64) 32 246.7 4054253 0 191.4-252.7 1.32x + +-- drained: a consumer popping continuously, capacity 1024 -- +shape producers ns/op ops/sec refusals ns/op range spread +slotwise_mpsc 1 10.5 95274390 30 9.6-11.0 1.15x +reserving_mpsc 1 24.5 40889761 706 23.6-26.6 1.13x +permit_mpsc 1 60.5 16541503 437 59.7-70.6 1.18x +reserving(32/32) 1 25.0 40048058 707 21.3-26.7 1.26x +reserving(16/48) 1 25.3 39522567 1549 25.2-27.4 1.09x +reserving(8/56) 1 25.6 39047247 1231 24.6-27.9 1.13x +reserving(64/64) 1 29.3 34176350 2067 28.8-31.3 1.08x +slotwise_mpsc 2 74.3 13450804 360 68.3-81.1 1.19x +reserving_mpsc 2 66.5 15041892 2633 60.1-70.9 1.18x +permit_mpsc 2 55.6 17980114 10 54.4-56.3 1.03x +reserving(32/32) 2 67.0 14915355 92 65.0-67.4 1.04x +reserving(16/48) 2 66.2 15097302 1183 64.9-71.8 1.11x +reserving(8/56) 2 69.2 14454418 281 64.7-70.0 1.08x +reserving(64/64) 2 71.1 14056394 1050 65.6-73.8 1.13x +slotwise_mpsc 4 108.0 9255746 5979 97.8-112.9 1.16x +reserving_mpsc 4 92.8 10773598 0 91.4-100.6 1.10x +permit_mpsc 4 82.9 12055818 87923 71.3-84.5 1.19x +reserving(32/32) 4 94.5 10584867 808 90.2-100.9 1.12x +reserving(16/48) 4 100.0 9997001 0 98.5-105.6 1.07x +reserving(8/56) 4 94.3 10606141 1151 93.3-97.0 1.04x +reserving(64/64) 4 103.3 9683167 4388 92.7-112.6 1.21x +slotwise_mpsc 8 157.6 6343376 575 153.3-159.2 1.04x +reserving_mpsc 8 164.5 6078270 1755 147.9-172.7 1.17x +permit_mpsc 8 147.7 6772246 608108 136.6-159.7 1.17x +reserving(32/32) 8 165.0 6061488 4788 158.9-173.9 1.09x +reserving(16/48) 8 167.0 5989432 12075 163.2-168.3 1.03x +reserving(8/56) 8 165.9 6026556 6065 160.1-169.3 1.06x +reserving(64/64) 8 176.0 5681859 8970 166.3-185.4 1.11x +slotwise_mpsc 16 248.1 4030921 996020 244.6-333.4 1.36x +reserving_mpsc 16 275.5 3629124 1273380 258.4-346.1 1.34x +permit_mpsc 16 278.2 3594693 3179439 244.9-311.0 1.27x +reserving(32/32) 16 265.9 3760339 1045393 245.3-295.9 1.21x +reserving(16/48) 16 283.7 3524417 956012 257.9-319.5 1.24x +reserving(8/56) 16 267.3 3740662 1219744 249.3-347.0 1.39x +reserving(64/64) 16 255.7 3910307 1510719 250.3-328.6 1.31x +slotwise_mpsc 32 566.2 1766305 16022148 486.2-717.5 1.48x +reserving_mpsc 32 619.1 1615374 23168414 494.6-784.9 1.59x +permit_mpsc 32 483.8 2066835 11753534 416.8-572.5 1.37x +reserving(32/32) 32 654.8 1527084 25476853 466.5-765.1 1.64x +reserving(16/48) 32 806.1 1240549 29246593 574.9-905.9 1.58x +reserving(8/56) 32 586.1 1706172 17994934 497.6-800.6 1.61x +reserving(64/64) 32 445.5 2244654 9291698 429.6-826.5 1.92x + +interpretation: + + 1. push-path scaling with producer count (isolated regime) + + producers slotwise reserving permit atomic floor + 1 1.00x [1.00-1.00] 1.00x [1.00-1.00] 1.00x [1.00-1.00] 1.00x [1.00-1.00] + 2 0.10x [0.09-0.11] 0.16x [0.14-0.18] 0.20x [0.18-0.44] 0.20x [0.17-0.22] + 4 0.07x [0.07-0.07] 0.15x [0.14-0.17] 0.26x [0.09-0.26] 0.15x [0.14-0.16] + 8 0.04x [0.04-0.05] 0.14x [0.11-0.16] 0.32x [0.29-0.36] 0.15x [0.14-0.16] + 16 0.02x [0.02-0.03] 0.12x [0.10-0.13] 0.35x [0.33-0.37] 0.16x [0.15-0.16] + 32 0.03x [0.02-0.03] 0.10x [0.10-0.12] 0.40x [0.39-0.40] 0.16x [0.15-0.16] + + Read as: throughput at N producers divided by throughput at one. + 1.00 means N threads together push no faster than one did. + The atomic floor is the cheapest possible contended operation, + so it says how much of any curve is the queue and how much is + simply what this processor does to a fought-over cache line. + + 2. reserving vs slotwise, drained (where `head` is written) + + The ratio is the WHOLE push path of two different shapes, not the + price of reserving's extra `head` load on its own: they use + different claim protocols, slot metadata and retry behaviour. This + regime is where that load is at its most expensive -- but the + ratio still does not isolate it, or bound it either way. + + producers slotwise reserving reserving/slotwise permit permit/reserving + ns/op ns/op ratio [bound] ns/op ratio [bound] + 1 10.5 24.5 2.33x [2.14-2.77] 60.5 2.47x [2.24-2.99] + 2 74.3 66.5 0.89x [0.74-1.04] 55.6 0.84x [0.77-0.94] + 4 108.0 92.8 0.86x [0.81-1.03] 82.9 0.89x [0.71-0.93] + 8 157.6 164.5 1.04x [0.93-1.13] 147.7 0.90x [0.79-1.08] + 16 248.1 275.5 1.11x [0.77-1.41] 278.2 1.01x [0.71-1.20] + 32 566.2 619.1 1.09x [0.69-1.61] 483.8 0.78x [0.53-1.16] + + `reserving_mpsc` reads the consumer's position on every push and + `slotwise_mpsc` does not. This regime is where that read is at its + most expensive, because a consumer is writing the line being read + -- but the ratio does not decompose. It is an END-TO-END + comparison of two shapes: they also differ in claim protocol, + slot metadata and retry behaviour, and those differences are not + ordered. So this ratio neither isolates the read nor bounds it. + + `permit_mpsc` is experimental and is the candidate replacement + for `reserving_mpsc`: it removes that read entirely, and with it + the stale room decision behind SH-14.1, by making admission a + read-modify-write on a permit count instead. The last column is + the trade -- below 1.00 and the safer claim is also the cheaper + one; above 1.00 and closing the hole costs throughput. + + 3. claim-word layout + + 4 apportionments of reserving_mpsc's claim word, measured on + the shipping type itself rather than on a stand-in. 32/32 is the + default; 16/48 and 8/56 are the same u64 exchange with the bits + apportioned differently; 64/64 is a u128 exchange (cmpxchg16b on + x86-64, ldxp/stxp on aarch64), measured only where that is native. + The three u64 rows issue the same instruction and differ only in + shift and mask constants, so there is no structural reason for one + to be slower -- but these rows time the WHOLE push path, so a + difference between them is not thereby noise. Read it against a + control before calling it either way: the reserving_mpsc row and + the 32/32 row above are the same code, so the gap between them is + what 'no difference' looks like on this host -- read it against + the layout rows before calling any of them apart. + 64/64 vs 32/32 is the double-width layout's effect on the whole + push path -- what moving the recurrence to 2^64 costs, against + 8/56 moving it to 2^56. Both defer the recurrence rather than + removing it. Not the exchange in isolation. + + -- isolated -- + producers 32/32 ns/op 16/48 ns/op 8/56 ns/op 64/64 ns/op 16/48 vs 8/56 vs 64/64 vs + 1 5.7 5.4 5.4 7.5 0.94x [0.91-1.05] 0.95x [0.91-1.07] 1.31x [1.26-1.49] + 2 35.0 33.7 33.2 40.5 0.96x [0.89-1.21] 0.95x [0.86-1.26] 1.16x [1.01-1.36] + 4 36.3 37.1 35.0 61.8 1.02x [0.86-1.13] 0.97x [0.77-1.16] 1.70x [1.24-1.82] + 8 38.8 37.0 35.6 61.7 0.95x [0.84-1.32] 0.92x [0.86-1.19] 1.59x [1.34-2.54] + 16 52.9 62.2 68.0 216.7 1.17x [1.07-1.37] 1.29x [1.19-1.47] 4.09x [3.71-4.31] + 32 49.4 66.0 70.6 246.7 1.34x [1.24-1.61] 1.43x [1.21-1.54] 4.99x [3.68-5.39] + + -- drained -- + producers 32/32 ns/op 16/48 ns/op 8/56 ns/op 64/64 ns/op 16/48 vs 8/56 vs 64/64 vs + 1 25.0 25.3 25.6 29.3 1.01x [0.94-1.29] 1.03x [0.92-1.31] 1.17x [1.08-1.47] + 2 67.0 66.2 69.2 71.1 0.99x [0.96-1.10] 1.03x [0.96-1.08] 1.06x [0.97-1.14] + 4 94.5 100.0 94.3 103.3 1.06x [0.98-1.17] 1.00x [0.93-1.07] 1.09x [0.92-1.25] + 8 165.0 167.0 165.9 176.0 1.01x [0.94-1.06] 1.01x [0.92-1.07] 1.07x [0.96-1.17] + 16 265.9 283.7 267.3 255.7 1.07x [0.87-1.30] 1.01x [0.84-1.41] 0.96x [0.85-1.34] + 32 654.8 806.1 586.1 445.5 1.23x [0.75-1.94] 0.90x [0.65-1.72] 0.68x [0.56-1.77] + + the 32/32 row and the reserving_mpsc row above are the same + configuration run twice, so the gap between them is this host's + same-code control: whatever it shows is dispersion, not a + difference between shapes. Do not read it as noise that can be + discounted -- its width is an open question about this + instrument. They + are no longer a control against a duplicated implementation: the + shipping type takes the layout as a parameter, so there is nothing + left that could drift away from what callers actually run. + + CAUTION: the drained regime has ONE consumer, because that is what + MPSC means. At high producer counts it is expected to become + consumer-bound, and a plateau there says nothing about the claim. + The refusal counts above are what make that visible: a run with + many refusals met a full queue often, so the consumer is one term + in what it measured. That does not rule the tail out -- both can + bind at once, and these counts do not separate them. diff --git a/crates/windows-platform-probes/captures/2026-09-16-drained-handshake/run2.txt b/crates/windows-platform-probes/captures/2026-09-16-drained-handshake/run2.txt new file mode 100644 index 000000000..d9e203762 --- /dev/null +++ b/crates/windows-platform-probes/captures/2026-09-16-drained-handshake/run2.txt @@ -0,0 +1,209 @@ +host: x86_64 16p/8c smt+ L2[2,2,2,2,2,2,2,2] ec[0:16] numa[16] +== how does the array queue's push path scale with producer count? == + +processors available to this process: 16 + +profile: release +sampling: 50000 pushes per producer, median of 5 repetitions, one untimed warmup pass + +-- isolated: producers only, capacity large enough that nothing is refused -- +shape producers ns/op ops/sec refusals ns/op range spread +baseline_fetch_add 1 2.3 432152118 0 2.3-2.6 1.14x +slotwise_mpsc 1 6.0 166722241 0 5.9-6.3 1.07x +reserving_mpsc 1 5.4 184774575 0 5.4-5.9 1.09x +permit_mpsc 1 8.1 123609394 0 8.1-8.7 1.08x +reserving(32/32) 1 6.8 147405660 0 5.4-9.1 1.67x +reserving(16/48) 1 6.4 157331655 0 6.2-6.8 1.11x +reserving(8/56) 1 6.6 152207002 0 6.0-6.8 1.14x +reserving(64/64) 1 11.8 85048478 0 10.1-12.0 1.19x +baseline_fetch_add 2 12.5 80179602 0 11.8-12.8 1.08x +slotwise_mpsc 2 58.8 17014326 0 44.4-61.3 1.38x +reserving_mpsc 2 35.9 27845845 0 29.5-37.8 1.28x +permit_mpsc 2 43.1 23208856 0 40.4-43.9 1.09x +reserving(32/32) 2 36.0 27808676 0 32.6-37.8 1.16x +reserving(16/48) 2 33.6 29717682 0 31.1-35.3 1.13x +reserving(8/56) 2 32.8 30522235 0 24.4-36.5 1.50x +reserving(64/64) 2 39.5 25342119 0 35.8-40.0 1.12x +baseline_fetch_add 4 13.8 72513687 0 13.4-15.8 1.18x +slotwise_mpsc 4 92.3 10837406 0 90.4-93.0 1.03x +reserving_mpsc 4 36.8 27173175 0 35.5-39.2 1.11x +permit_mpsc 4 32.0 31285685 0 31.3-32.5 1.04x +reserving(32/32) 4 38.0 26332420 0 34.6-40.1 1.16x +reserving(16/48) 4 32.7 30609122 0 30.5-35.8 1.17x +reserving(8/56) 4 35.3 28292945 0 30.6-41.1 1.35x +reserving(64/64) 4 52.1 19205654 0 42.5-57.7 1.36x +baseline_fetch_add 8 15.8 63477957 0 14.3-17.2 1.21x +slotwise_mpsc 8 141.9 7044933 0 136.3-148.1 1.09x +reserving_mpsc 8 38.5 25992254 0 35.8-40.3 1.13x +permit_mpsc 8 25.7 38934745 0 25.7-26.9 1.05x +reserving(32/32) 8 38.2 26204077 0 36.2-44.6 1.23x +reserving(16/48) 8 36.4 27492543 0 36.1-38.4 1.06x +reserving(8/56) 8 43.6 22933018 0 34.7-45.3 1.31x +reserving(64/64) 8 78.3 12770006 0 62.6-103.8 1.66x +baseline_fetch_add 16 14.9 66962417 0 14.5-15.1 1.04x +slotwise_mpsc 16 248.2 4029196 0 196.7-268.3 1.36x +reserving_mpsc 16 50.4 19839203 0 50.0-54.6 1.09x +permit_mpsc 16 20.2 49499437 0 19.7-22.9 1.16x +reserving(32/32) 16 46.2 21639635 0 44.7-46.9 1.05x +reserving(16/48) 16 57.9 17263293 0 55.6-60.7 1.09x +reserving(8/56) 16 52.7 18991098 0 49.5-57.7 1.17x +reserving(64/64) 16 184.6 5417144 0 132.3-188.8 1.43x +baseline_fetch_add 32 15.1 66207901 0 14.9-15.4 1.03x +slotwise_mpsc 32 227.5 4395744 0 217.2-229.7 1.06x +reserving_mpsc 32 53.4 18722143 0 48.7-55.5 1.14x +permit_mpsc 32 22.1 45231856 0 21.8-22.5 1.03x +reserving(32/32) 32 52.3 19129466 0 50.7-54.7 1.08x +reserving(16/48) 32 74.5 13425229 0 67.2-76.2 1.13x +reserving(8/56) 32 66.0 15140432 0 64.0-71.6 1.12x +reserving(64/64) 32 185.3 5395613 0 173.8-231.3 1.33x + +-- drained: a consumer popping continuously, capacity 1024 -- +shape producers ns/op ops/sec refusals ns/op range spread +slotwise_mpsc 1 10.2 97675327 37 9.7-11.1 1.14x +reserving_mpsc 1 23.8 42052145 648 22.2-25.2 1.14x +permit_mpsc 1 59.7 16762211 487 57.3-68.7 1.20x +reserving(32/32) 1 29.1 34340659 0 27.7-32.4 1.17x +reserving(16/48) 1 28.5 35075412 450 23.9-30.3 1.27x +reserving(8/56) 1 26.8 37366415 1285 25.2-30.1 1.19x +reserving(64/64) 1 27.4 36456435 397 25.4-36.3 1.43x +slotwise_mpsc 2 72.2 13845621 322 69.4-80.0 1.15x +reserving_mpsc 2 68.4 14619028 7678 67.2-69.1 1.03x +permit_mpsc 2 54.7 18293241 43 53.5-55.0 1.03x +reserving(32/32) 2 64.6 15480595 2799 62.8-69.8 1.11x +reserving(16/48) 2 62.0 16128512 1640 59.7-69.8 1.17x +reserving(8/56) 2 71.3 14028000 206 65.7-74.5 1.13x +reserving(64/64) 2 69.6 14377732 66 62.3-73.9 1.19x +slotwise_mpsc 4 107.0 9344266 1293 97.7-109.3 1.12x +reserving_mpsc 4 97.9 10219097 1407 83.6-100.7 1.21x +permit_mpsc 4 71.7 13950504 61931 66.3-78.1 1.18x +reserving(32/32) 4 96.1 10405936 0 79.8-98.7 1.24x +reserving(16/48) 4 96.5 10363822 263 84.1-100.8 1.20x +reserving(8/56) 4 103.0 9707984 348 88.8-107.4 1.21x +reserving(64/64) 4 106.0 9435164 428 96.6-113.6 1.18x +slotwise_mpsc 8 146.6 6821375 2645 145.8-160.4 1.10x +reserving_mpsc 8 160.4 6236329 4435 154.3-170.8 1.11x +permit_mpsc 8 119.8 8345782 475034 96.8-150.6 1.56x +reserving(32/32) 8 173.6 5758926 8263 170.6-176.3 1.03x +reserving(16/48) 8 169.8 5888189 0 167.0-175.2 1.05x +reserving(8/56) 8 168.6 5930943 7305 161.6-173.8 1.08x +reserving(64/64) 8 179.6 5566937 7573 170.7-188.5 1.10x +slotwise_mpsc 16 329.5 3035177 2119296 274.9-353.4 1.29x +reserving_mpsc 16 233.1 4290545 738295 215.5-254.0 1.18x +permit_mpsc 16 219.5 4556538 2265316 197.9-333.9 1.69x +reserving(32/32) 16 308.8 3238569 3098327 201.4-327.2 1.62x +reserving(16/48) 16 264.9 3775086 1766041 211.4-428.0 2.02x +reserving(8/56) 16 285.8 3498926 2084768 241.0-346.0 1.44x +reserving(64/64) 16 246.4 4058429 1156019 215.3-307.4 1.43x +slotwise_mpsc 32 558.7 1790020 16403774 394.2-768.9 1.95x +reserving_mpsc 32 630.0 1587180 23588446 331.1-711.0 2.15x +permit_mpsc 32 531.5 1881620 12610199 419.9-644.1 1.53x +reserving(32/32) 32 510.4 1959073 16810770 395.0-627.1 1.59x +reserving(16/48) 32 640.9 1560202 21611025 518.3-903.3 1.74x +reserving(8/56) 32 677.9 1475132 22815835 527.2-956.4 1.81x +reserving(64/64) 32 568.1 1760288 15246419 501.2-631.0 1.26x + +interpretation: + + 1. push-path scaling with producer count (isolated regime) + + producers slotwise reserving permit atomic floor + 1 1.00x [1.00-1.00] 1.00x [1.00-1.00] 1.00x [1.00-1.00] 1.00x [1.00-1.00] + 2 0.10x [0.10-0.14] 0.15x [0.14-0.20] 0.19x [0.18-0.22] 0.19x [0.18-0.22] + 4 0.07x [0.06-0.07] 0.15x [0.14-0.17] 0.25x [0.25-0.28] 0.17x [0.15-0.20] + 8 0.04x [0.04-0.05] 0.14x [0.13-0.16] 0.31x [0.30-0.34] 0.15x [0.13-0.18] + 16 0.02x [0.02-0.03] 0.11x [0.10-0.12] 0.40x [0.35-0.44] 0.15x [0.15-0.18] + 32 0.03x [0.03-0.03] 0.10x [0.10-0.12] 0.37x [0.36-0.40] 0.15x [0.15-0.18] + + Read as: throughput at N producers divided by throughput at one. + 1.00 means N threads together push no faster than one did. + The atomic floor is the cheapest possible contended operation, + so it says how much of any curve is the queue and how much is + simply what this processor does to a fought-over cache line. + + 2. reserving vs slotwise, drained (where `head` is written) + + The ratio is the WHOLE push path of two different shapes, not the + price of reserving's extra `head` load on its own: they use + different claim protocols, slot metadata and retry behaviour. This + regime is where that load is at its most expensive -- but the + ratio still does not isolate it, or bound it either way. + + producers slotwise reserving reserving/slotwise permit permit/reserving + ns/op ns/op ratio [bound] ns/op ratio [bound] + 1 10.2 23.8 2.32x [1.99-2.60] 59.7 2.51x [2.27-3.10] + 2 72.2 68.4 0.95x [0.84-1.00] 54.7 0.80x [0.77-0.82] + 4 107.0 97.9 0.91x [0.76-1.03] 71.7 0.73x [0.66-0.93] + 8 146.6 160.4 1.09x [0.96-1.17] 119.8 0.75x [0.57-0.98] + 16 329.5 233.1 0.71x [0.61-0.92] 219.5 0.94x [0.78-1.55] + 32 558.7 630.0 1.13x [0.43-1.80] 531.5 0.84x [0.59-1.95] + + `reserving_mpsc` reads the consumer's position on every push and + `slotwise_mpsc` does not. This regime is where that read is at its + most expensive, because a consumer is writing the line being read + -- but the ratio does not decompose. It is an END-TO-END + comparison of two shapes: they also differ in claim protocol, + slot metadata and retry behaviour, and those differences are not + ordered. So this ratio neither isolates the read nor bounds it. + + `permit_mpsc` is experimental and is the candidate replacement + for `reserving_mpsc`: it removes that read entirely, and with it + the stale room decision behind SH-14.1, by making admission a + read-modify-write on a permit count instead. The last column is + the trade -- below 1.00 and the safer claim is also the cheaper + one; above 1.00 and closing the hole costs throughput. + + 3. claim-word layout + + 4 apportionments of reserving_mpsc's claim word, measured on + the shipping type itself rather than on a stand-in. 32/32 is the + default; 16/48 and 8/56 are the same u64 exchange with the bits + apportioned differently; 64/64 is a u128 exchange (cmpxchg16b on + x86-64, ldxp/stxp on aarch64), measured only where that is native. + The three u64 rows issue the same instruction and differ only in + shift and mask constants, so there is no structural reason for one + to be slower -- but these rows time the WHOLE push path, so a + difference between them is not thereby noise. Read it against a + control before calling it either way: the reserving_mpsc row and + the 32/32 row above are the same code, so the gap between them is + what 'no difference' looks like on this host -- read it against + the layout rows before calling any of them apart. + 64/64 vs 32/32 is the double-width layout's effect on the whole + push path -- what moving the recurrence to 2^64 costs, against + 8/56 moving it to 2^56. Both defer the recurrence rather than + removing it. Not the exchange in isolation. + + -- isolated -- + producers 32/32 ns/op 16/48 ns/op 8/56 ns/op 64/64 ns/op 16/48 vs 8/56 vs 64/64 vs + 1 6.8 6.4 6.6 11.8 0.94x [0.68-1.26] 0.97x [0.66-1.25] 1.73x [1.11-2.21] + 2 36.0 33.6 32.8 39.5 0.94x [0.82-1.08] 0.91x [0.64-1.12] 1.10x [0.95-1.23] + 4 38.0 32.7 35.3 52.1 0.86x [0.76-1.03] 0.93x [0.76-1.19] 1.37x [1.06-1.67] + 8 38.2 36.4 43.6 78.3 0.95x [0.81-1.06] 1.14x [0.78-1.25] 2.05x [1.40-2.87] + 16 46.2 57.9 52.7 184.6 1.25x [1.18-1.36] 1.14x [1.06-1.29] 3.99x [2.82-4.22] + 32 52.3 74.5 66.0 185.3 1.42x [1.23-1.50] 1.26x [1.17-1.41] 3.55x [3.18-4.56] + + -- drained -- + producers 32/32 ns/op 16/48 ns/op 8/56 ns/op 64/64 ns/op 16/48 vs 8/56 vs 64/64 vs + 1 29.1 28.5 26.8 27.4 0.98x [0.74-1.10] 0.92x [0.78-1.09] 0.94x [0.78-1.31] + 2 64.6 62.0 71.3 69.6 0.96x [0.85-1.11] 1.10x [0.94-1.19] 1.08x [0.89-1.18] + 4 96.1 96.5 103.0 106.0 1.00x [0.85-1.26] 1.07x [0.90-1.35] 1.10x [0.98-1.42] + 8 173.6 169.8 168.6 179.6 0.98x [0.95-1.03] 0.97x [0.92-1.02] 1.03x [0.97-1.10] + 16 308.8 264.9 285.8 246.4 0.86x [0.65-2.13] 0.93x [0.74-1.72] 0.80x [0.66-1.53] + 32 510.4 640.9 677.9 568.1 1.26x [0.83-2.29] 1.33x [0.84-2.42] 1.11x [0.80-1.60] + + the 32/32 row and the reserving_mpsc row above are the same + configuration run twice, so the gap between them is this host's + same-code control: whatever it shows is dispersion, not a + difference between shapes. Do not read it as noise that can be + discounted -- its width is an open question about this + instrument. They + are no longer a control against a duplicated implementation: the + shipping type takes the layout as a parameter, so there is nothing + left that could drift away from what callers actually run. + + CAUTION: the drained regime has ONE consumer, because that is what + MPSC means. At high producer counts it is expected to become + consumer-bound, and a plateau there says nothing about the claim. + The refusal counts above are what make that visible: a run with + many refusals met a full queue often, so the consumer is one term + in what it measured. That does not rule the tail out -- both can + bind at once, and these counts do not separate them. diff --git a/crates/windows-platform-probes/captures/2026-09-16-drained-handshake/run3.txt b/crates/windows-platform-probes/captures/2026-09-16-drained-handshake/run3.txt new file mode 100644 index 000000000..4dd771606 --- /dev/null +++ b/crates/windows-platform-probes/captures/2026-09-16-drained-handshake/run3.txt @@ -0,0 +1,209 @@ +host: x86_64 16p/8c smt+ L2[2,2,2,2,2,2,2,2] ec[0:16] numa[16] +== how does the array queue's push path scale with producer count? == + +processors available to this process: 16 + +profile: release +sampling: 50000 pushes per producer, median of 5 repetitions, one untimed warmup pass + +-- isolated: producers only, capacity large enough that nothing is refused -- +shape producers ns/op ops/sec refusals ns/op range spread +baseline_fetch_add 1 2.3 432152118 0 2.3-2.6 1.13x +slotwise_mpsc 1 6.2 161082474 0 5.9-6.4 1.09x +reserving_mpsc 1 5.7 175377061 0 5.4-6.3 1.15x +permit_mpsc 1 8.0 125031258 0 7.9-8.4 1.07x +reserving(32/32) 1 5.4 184774575 0 5.4-5.8 1.08x +reserving(16/48) 1 5.4 186706497 0 5.3-5.7 1.08x +reserving(8/56) 1 5.4 185597624 0 5.4-5.7 1.06x +reserving(64/64) 1 7.5 134120172 0 7.4-7.7 1.03x +baseline_fetch_add 2 12.2 82209799 0 10.8-12.7 1.17x +slotwise_mpsc 2 56.8 17613076 0 54.0-57.6 1.07x +reserving_mpsc 2 35.1 28489217 0 24.2-35.6 1.47x +permit_mpsc 2 42.5 23510051 0 41.0-43.1 1.05x +reserving(32/32) 2 36.5 27398762 0 33.5-37.7 1.13x +reserving(16/48) 2 35.5 28190455 0 33.7-37.1 1.10x +reserving(8/56) 2 33.0 30335204 0 32.0-36.8 1.15x +reserving(64/64) 2 40.2 24857690 0 39.2-41.1 1.05x +baseline_fetch_add 4 14.9 66961296 0 13.7-15.7 1.14x +slotwise_mpsc 4 87.5 11426156 0 84.5-89.6 1.06x +reserving_mpsc 4 39.2 25497520 0 37.0-39.5 1.07x +permit_mpsc 4 30.8 32489157 0 30.3-31.5 1.04x +reserving(32/32) 4 37.1 26948004 0 33.8-37.4 1.11x +reserving(16/48) 4 35.6 28082394 0 35.2-36.1 1.03x +reserving(8/56) 4 36.7 27236453 0 35.0-38.3 1.10x +reserving(64/64) 4 50.6 19743726 0 45.6-56.3 1.23x +baseline_fetch_add 8 15.8 63281126 0 15.2-16.6 1.09x +slotwise_mpsc 8 133.8 7474428 0 125.0-140.7 1.13x +reserving_mpsc 8 46.9 21300729 0 39.4-48.5 1.23x +permit_mpsc 8 24.2 41306099 0 22.1-24.4 1.10x +reserving(32/32) 8 42.0 23781213 0 35.3-43.3 1.23x +reserving(16/48) 8 46.6 21479970 0 37.5-53.9 1.43x +reserving(8/56) 8 37.0 26998022 0 36.4-44.9 1.23x +reserving(64/64) 8 70.1 14259030 0 59.9-71.5 1.19x +baseline_fetch_add 16 15.1 66365808 0 14.9-15.1 1.02x +slotwise_mpsc 16 202.0 4950869 0 171.0-242.4 1.42x +reserving_mpsc 16 52.6 19004768 0 46.4-55.2 1.19x +permit_mpsc 16 22.3 44793084 0 20.0-22.8 1.14x +reserving(32/32) 16 46.3 21597048 0 40.2-47.3 1.18x +reserving(16/48) 16 59.5 16805664 0 55.8-60.6 1.09x +reserving(8/56) 16 59.0 16961658 0 52.9-61.1 1.15x +reserving(64/64) 16 182.6 5476841 0 148.8-228.1 1.53x +baseline_fetch_add 32 14.7 68208753 0 14.6-15.1 1.03x +slotwise_mpsc 32 258.7 3865605 0 211.5-264.6 1.25x +reserving_mpsc 32 47.6 21022619 0 47.1-51.5 1.09x +permit_mpsc 32 22.0 45478060 0 21.2-22.2 1.05x +reserving(32/32) 32 50.6 19763773 0 46.8-53.2 1.14x +reserving(16/48) 32 68.4 14622341 0 67.9-75.9 1.12x +reserving(8/56) 32 73.3 13650386 0 65.8-77.5 1.18x +reserving(64/64) 32 241.4 4141800 0 191.4-245.5 1.28x + +-- drained: a consumer popping continuously, capacity 1024 -- +shape producers ns/op ops/sec refusals ns/op range spread +slotwise_mpsc 1 10.7 93370682 40 8.7-11.1 1.27x +reserving_mpsc 1 23.1 43230157 225 22.3-24.9 1.11x +permit_mpsc 1 62.2 16065806 195 59.9-63.1 1.05x +reserving(32/32) 1 24.0 41673612 628 21.9-25.0 1.15x +reserving(16/48) 1 26.8 37302298 1161 25.9-27.8 1.07x +reserving(8/56) 1 27.0 37100245 1419 25.0-31.5 1.26x +reserving(64/64) 1 29.1 34385531 679 27.6-31.2 1.13x +slotwise_mpsc 2 70.3 14227584 605 67.7-77.1 1.14x +reserving_mpsc 2 63.1 15853891 2858 62.1-66.7 1.07x +permit_mpsc 2 54.2 18465857 52 52.3-54.5 1.04x +reserving(32/32) 2 68.6 14582786 995 67.3-69.1 1.03x +reserving(16/48) 2 67.2 14874091 847 66.1-69.6 1.05x +reserving(8/56) 2 67.4 14839217 169 64.4-72.4 1.13x +reserving(64/64) 2 69.7 14355647 44 69.3-71.2 1.03x +slotwise_mpsc 4 103.8 9637300 1991 96.4-106.2 1.10x +reserving_mpsc 4 91.4 10938765 2017 83.2-97.2 1.17x +permit_mpsc 4 73.6 13587695 78415 61.2-89.6 1.46x +reserving(32/32) 4 96.4 10370916 8168 93.5-101.9 1.09x +reserving(16/48) 4 94.2 10610248 2135 92.6-96.5 1.04x +reserving(8/56) 4 94.4 10593669 917 80.2-101.5 1.27x +reserving(64/64) 4 92.4 10827960 3433 89.0-98.8 1.11x +slotwise_mpsc 8 151.3 6607748 2909 146.2-153.3 1.05x +reserving_mpsc 8 157.2 6360241 1794 153.3-164.1 1.07x +permit_mpsc 8 131.6 7597529 553659 122.3-148.6 1.22x +reserving(32/32) 8 160.2 6241428 4894 156.0-176.2 1.13x +reserving(16/48) 8 157.6 6343387 4510 152.0-174.4 1.15x +reserving(8/56) 8 163.7 6110257 15254 142.7-173.8 1.22x +reserving(64/64) 8 170.3 5871422 0 155.7-180.1 1.16x +slotwise_mpsc 16 295.5 3384563 1724125 198.4-346.7 1.75x +reserving_mpsc 16 268.5 3723944 1563173 193.3-334.3 1.73x +permit_mpsc 16 272.8 3665271 3043100 196.5-310.3 1.58x +reserving(32/32) 16 306.4 3263624 2683848 234.3-359.3 1.53x +reserving(16/48) 16 278.7 3587611 1198515 227.4-304.6 1.34x +reserving(8/56) 16 272.1 3675748 960270 234.4-365.6 1.56x +reserving(64/64) 16 237.5 4210548 31703 225.1-314.8 1.40x +slotwise_mpsc 32 635.7 1573041 19058153 432.0-717.3 1.66x +reserving_mpsc 32 675.4 1480673 25956490 475.2-768.1 1.62x +permit_mpsc 32 431.6 2316787 10629824 329.4-547.5 1.66x +reserving(32/32) 32 691.7 1445741 26425860 492.6-814.3 1.65x +reserving(16/48) 32 633.9 1577466 21810973 619.2-949.4 1.53x +reserving(8/56) 32 829.0 1206298 28202713 570.4-956.6 1.68x +reserving(64/64) 32 647.7 1544019 19247556 531.5-718.7 1.35x + +interpretation: + + 1. push-path scaling with producer count (isolated regime) + + producers slotwise reserving permit atomic floor + 1 1.00x [1.00-1.00] 1.00x [1.00-1.00] 1.00x [1.00-1.00] 1.00x [1.00-1.00] + 2 0.11x [0.10-0.12] 0.16x [0.15-0.26] 0.19x [0.18-0.20] 0.19x [0.18-0.24] + 4 0.07x [0.07-0.08] 0.15x [0.14-0.17] 0.26x [0.25-0.28] 0.15x [0.15-0.19] + 8 0.05x [0.04-0.05] 0.12x [0.11-0.16] 0.33x [0.32-0.38] 0.15x [0.14-0.17] + 16 0.03x [0.02-0.04] 0.11x [0.10-0.14] 0.36x [0.35-0.42] 0.15x [0.15-0.17] + 32 0.02x [0.02-0.03] 0.12x [0.11-0.13] 0.36x [0.35-0.39] 0.16x [0.15-0.18] + + Read as: throughput at N producers divided by throughput at one. + 1.00 means N threads together push no faster than one did. + The atomic floor is the cheapest possible contended operation, + so it says how much of any curve is the queue and how much is + simply what this processor does to a fought-over cache line. + + 2. reserving vs slotwise, drained (where `head` is written) + + The ratio is the WHOLE push path of two different shapes, not the + price of reserving's extra `head` load on its own: they use + different claim protocols, slot metadata and retry behaviour. This + regime is where that load is at its most expensive -- but the + ratio still does not isolate it, or bound it either way. + + producers slotwise reserving reserving/slotwise permit permit/reserving + ns/op ns/op ratio [bound] ns/op ratio [bound] + 1 10.7 23.1 2.16x [2.01-2.85] 62.2 2.69x [2.41-2.83] + 2 70.3 63.1 0.90x [0.81-0.99] 54.2 0.86x [0.78-0.88] + 4 103.8 91.4 0.88x [0.78-1.01] 73.6 0.81x [0.63-1.08] + 8 151.3 157.2 1.04x [1.00-1.12] 131.6 0.84x [0.75-0.97] + 16 295.5 268.5 0.91x [0.56-1.68] 272.8 1.02x [0.59-1.61] + 32 635.7 675.4 1.06x [0.66-1.78] 431.6 0.64x [0.43-1.15] + + `reserving_mpsc` reads the consumer's position on every push and + `slotwise_mpsc` does not. This regime is where that read is at its + most expensive, because a consumer is writing the line being read + -- but the ratio does not decompose. It is an END-TO-END + comparison of two shapes: they also differ in claim protocol, + slot metadata and retry behaviour, and those differences are not + ordered. So this ratio neither isolates the read nor bounds it. + + `permit_mpsc` is experimental and is the candidate replacement + for `reserving_mpsc`: it removes that read entirely, and with it + the stale room decision behind SH-14.1, by making admission a + read-modify-write on a permit count instead. The last column is + the trade -- below 1.00 and the safer claim is also the cheaper + one; above 1.00 and closing the hole costs throughput. + + 3. claim-word layout + + 4 apportionments of reserving_mpsc's claim word, measured on + the shipping type itself rather than on a stand-in. 32/32 is the + default; 16/48 and 8/56 are the same u64 exchange with the bits + apportioned differently; 64/64 is a u128 exchange (cmpxchg16b on + x86-64, ldxp/stxp on aarch64), measured only where that is native. + The three u64 rows issue the same instruction and differ only in + shift and mask constants, so there is no structural reason for one + to be slower -- but these rows time the WHOLE push path, so a + difference between them is not thereby noise. Read it against a + control before calling it either way: the reserving_mpsc row and + the 32/32 row above are the same code, so the gap between them is + what 'no difference' looks like on this host -- read it against + the layout rows before calling any of them apart. + 64/64 vs 32/32 is the double-width layout's effect on the whole + push path -- what moving the recurrence to 2^64 costs, against + 8/56 moving it to 2^56. Both defer the recurrence rather than + removing it. Not the exchange in isolation. + + -- isolated -- + producers 32/32 ns/op 16/48 ns/op 8/56 ns/op 64/64 ns/op 16/48 vs 8/56 vs 64/64 vs + 1 5.4 5.4 5.4 7.5 0.99x [0.92-1.06] 1.00x [0.93-1.06] 1.38x [1.28-1.42] + 2 36.5 35.5 33.0 40.2 0.97x [0.89-1.11] 0.90x [0.85-1.10] 1.10x [1.04-1.23] + 4 37.1 35.6 36.7 50.6 0.96x [0.94-1.07] 0.99x [0.93-1.13] 1.36x [1.22-1.66] + 8 42.0 46.6 37.0 70.1 1.11x [0.87-1.52] 0.88x [0.84-1.27] 1.67x [1.38-2.02] + 16 46.3 59.5 59.0 182.6 1.29x [1.18-1.51] 1.27x [1.12-1.52] 3.94x [3.15-5.67] + 32 50.6 68.4 73.3 241.4 1.35x [1.28-1.62] 1.45x [1.24-1.66] 4.77x [3.60-5.25] + + -- drained -- + producers 32/32 ns/op 16/48 ns/op 8/56 ns/op 64/64 ns/op 16/48 vs 8/56 vs 64/64 vs + 1 24.0 26.8 27.0 29.1 1.12x [1.04-1.27] 1.12x [1.00-1.44] 1.21x [1.10-1.43] + 2 68.6 67.2 67.4 69.7 0.98x [0.96-1.03] 0.98x [0.93-1.08] 1.02x [1.00-1.06] + 4 96.4 94.2 94.4 92.4 0.98x [0.91-1.03] 0.98x [0.79-1.09] 0.96x [0.87-1.06] + 8 160.2 157.6 163.7 170.3 0.98x [0.86-1.12] 1.02x [0.81-1.11] 1.06x [0.88-1.15] + 16 306.4 278.7 272.1 237.5 0.91x [0.63-1.30] 0.89x [0.65-1.56] 0.78x [0.63-1.34] + 32 691.7 633.9 829.0 647.7 0.92x [0.76-1.93] 1.20x [0.70-1.94] 0.94x [0.65-1.46] + + the 32/32 row and the reserving_mpsc row above are the same + configuration run twice, so the gap between them is this host's + same-code control: whatever it shows is dispersion, not a + difference between shapes. Do not read it as noise that can be + discounted -- its width is an open question about this + instrument. They + are no longer a control against a duplicated implementation: the + shipping type takes the layout as a parameter, so there is nothing + left that could drift away from what callers actually run. + + CAUTION: the drained regime has ONE consumer, because that is what + MPSC means. At high producer counts it is expected to become + consumer-bound, and a plateau there says nothing about the claim. + The refusal counts above are what make that visible: a run with + many refusals met a full queue often, so the consumer is one term + in what it measured. That does not rule the tail out -- both can + bind at once, and these counts do not separate them. diff --git a/crates/windows-platform-probes/captures/2026-09-16-drained-handshake/summarise.js b/crates/windows-platform-probes/captures/2026-09-16-drained-handshake/summarise.js new file mode 100644 index 000000000..ddbc3baca --- /dev/null +++ b/crates/windows-platform-probes/captures/2026-09-16-drained-handshake/summarise.js @@ -0,0 +1,436 @@ +// Copyright (c) Mike Grier. +// Summarise the drained tables of a queue-contention capture. +// +// Reads the probe's own report text rather than re-deriving anything: the +// medians and bounds are whatever the instrument printed. What this adds is the +// across-run median per producer count, and the same-code control span, which +// no single run states because it is a relation between two tables. + +const fs = require("fs"); +const crypto = require("crypto"); + +// One writer for the generated artifact, per this repository's output rule: no +// formatting site picks a destination, so retargeting the report to a file is a +// change here and nowhere else. `fail` is the diagnostic path and stays separate +// from the artifact, which is why they are two sinks rather than one with a flag. +let sink = (text) => process.stdout.write(text + "\n"); +const out = (text = "") => sink(text); +const fail = (text) => process.stderr.write(text + "\n"); + +// A ratio cell, with all three numbers required to be plain decimals, and the +// whole cell required to stand alone between whitespace. Unanchored, the pattern +// matches a SUFFIX of malformed text -- `1.2.3x [1.0-2.0]` yields `2.3x +// [1.0-2.0]`, which then passes every downstream check as an ordinary cell. +const RATIO = /(?<=^|\s)(\d+\.\d+)x \[(\d+\.\d+)-(\d+\.\d+)\](?=\s|$)/g; + +function median(values) { + const sorted = [...values].sort((a, b) => a - b); + const middle = Math.floor(sorted.length / 2); + return sorted.length % 2 === 0 + ? (sorted[middle - 1] + sorted[middle]) / 2 + : sorted[middle]; +} + +// Problems are collected rather than thrown, so one malformed capture reports +// everything wrong with it instead of only the first thing. +// +// **Nothing here may quietly degrade to `NaN`.** Every comparison against `NaN` +// is false, so a `NaN` bound makes the containment filter below find nothing +// outside it and print `true` -- certifying a capture it could not read. A +// report renders `--` for a shape that did not run, and `Number("--")` is +// exactly that `NaN`. +const problems = []; + +function finite(text, what) { + const value = Number(text); + if (!Number.isFinite(value)) { + problems.push(`${what}: ${JSON.stringify(text)} is not a number`); + return null; + } + return value; +} + +// Every quantity this script reads is a cost or a ratio of costs, so zero and +// negative are as invalid as NaN -- and more dangerous, because `0.00x` is +// finite and renders as an ordinary-looking measurement. The probe's own +// did-not-run sentinel used to be `0.0`, so this is the shape a legacy or +// malformed capture actually takes. +function positive(text, what) { + const value = finite(text, what); + if (value === null) return null; + if (value <= 0) { + problems.push(`${what}: ${JSON.stringify(text)} is not positive`); + return null; + } + return value; +} + +// A producer count is a positive whole number, and is used as a `Map` key. Left +// to `finite`, a label of `1.0` becomes the number 1 and matches the expected +// count 1, so a malformed capture would be normalised into a well-formed one on +// the way past the completeness check. +function wholeCount(text, what) { + // The check is on the TEXT, not the parsed value: `Number("2.0")` is 2 and + // `Number.isInteger(2)` is true, so a value-level test normalises the very + // label it is meant to reject. A producer count is written as digits. + if (!/^\d+$/.test(text)) { + problems.push(`${what}: ${JSON.stringify(text)} is not a whole number`); + return null; + } + return positive(text, what); +} + +// A ratio triple must be ordered and must contain its own point estimate. +// `positive` accepts each number on its own, so `2.00x [3.00-1.00]` passes +// three separate checks and is still not an interval any instrument produced. +function orderedTriple(point, low, high, where) { + if (point === null || low === null || high === null) return false; + if (low > high) { + problems.push(`${where}: bound ${low} to ${high} is inverted`); + return false; + } + if (point < low || point > high) { + problems.push(`${where}: median ${point} is outside its own bound ${low}-${high}`); + return false; + } + return true; +} + +// A table's rows, from `from` to the blank line that ends it. +// +// A fixed six-row slice discards a seventh row without reporting it, so a +// capture carrying an extra or duplicated producer row past the sixth would +// satisfy `EXPECTED_PRODUCERS` on the truncated map and be certified while its +// data was silently dropped. Reading to the terminator puts every row in front +// of the duplicate and expected-set checks instead. +function tableBody(lines, from) { + const body = []; + for (let i = from; i < lines.length; i += 1) { + if (lines[i].trim() === "") break; + body.push(lines[i]); + } + return body; +} + +// producers -> { narrowNanos, ratios: [16/48, 8/56, 64/64] } +function drainedLayout(lines, path) { + let start = -1; + lines.forEach((line, i) => { + if (line.includes("-- drained --")) start = i; + }); + // The raw table's marker is `-- drained: ... --`, which this does not match; + // the one it finds is the claim-layout interpretation table, which is the one + // carrying ratios. A missing marker must stop the run rather than silently + // slice from line 1, which would summarise whatever happened to be there. + if (start < 0) { + problems.push(`${path}: no drained layout table (expected a "-- drained --" marker)`); + return new Map(); + } + const rows = new Map(); + for (const line of tableBody(lines, start + 2)) { + const fields = line.trim().split(/\s+/); + const producers = wholeCount(fields[0], `${path}: a drained layout producer count`); + if (producers === null) continue; + const where = `${path}, drained layout, ${producers} producers`; + const narrowNanos = positive(fields[1], `${where}: the 32/32 cost`); + // Pass every value through `positive`, like every other captured quantity, + // and check all three numbers of each cell -- the two bounds are not used by + // this script, but a capture carrying an unreadable bound is not a capture + // this script should certify as summarised. + const ratios = [...line.matchAll(RATIO)].flatMap((m, i) => [ + positive(m[1], `${where}: layout ratio ${i + 1}`), + positive(m[2], `${where}: layout ratio ${i + 1} lower bound`), + positive(m[3], `${where}: layout ratio ${i + 1} upper bound`), + ]); + // A row that did not run renders `--`, which the ratio pattern does not + // match, so a short list is the signal that this row cannot be summarised. + if (ratios.length !== 9) { + problems.push( + `${where}: found ${ratios.length / 3} layout ratios, expected 3`, + ); + continue; + } + if (ratios.some((r) => r === null)) continue; + if (narrowNanos === null) continue; + // Each triple must be an interval containing its own median. + const ordered = [0, 3, 6].every((i) => + orderedTriple(ratios[i], ratios[i + 1], ratios[i + 2], `${where}: layout ratio ${i / 3 + 1}`), + ); + if (!ordered) continue; + // A `Map` keeps the last write, so a duplicated producer row would silently + // discard the earlier measurement while the completeness check -- which sees + // only distinct keys -- still reported a whole capture. + if (rows.has(producers)) { + problems.push(`${where}: a second layout row for this producer count`); + continue; + } + rows.set(producers, { narrowNanos, ratios: [ratios[0], ratios[3], ratios[6]] }); + } + return rows; +} + +// producers -> reserving ns/op, from the comparison table +function drainedComparison(lines, path) { + const start = lines.findIndex((line) => line.includes("reserving/slotwise")); + const rows = new Map(); + for (const line of tableBody(lines, start + 2)) { + const fields = line.trim().split(/\s+/); + const producers = wholeCount(fields[0], `${path}: a comparison producer count`); + if (producers === null) continue; + const reserving = positive( + fields[2], + `${path}, drained comparison, ${producers} producers: the reserving_mpsc cost`, + ); + if (reserving === null) continue; + // Same overwrite hazard as the layout table, and it matters more here: this + // value is the same-code control, so silently keeping the last of two + // duplicates would move the published control range. + if (rows.has(producers)) { + problems.push( + `${path}, drained comparison, ${producers} producers: a second row for this producer count`, + ); + continue; + } + rows.set(producers, reserving); + } + return rows; +} + +const paths = process.argv.slice(2); +// An empty argument list summarises nothing: every completeness check below +// iterates the captures, so with none of them there is nothing to complain +// about, `median([])` is NaN, and the script prints an empty summary and exits +// successfully. Rejected here, as `isolated.js` does. +if (paths.length === 0) { + fail("usage: node summarise.js [run.txt ...]"); + process.exit(2); +} +// The run count and the control's `(n)` are both `paths.length`, so the same +// capture given twice would be certified as two independent runs -- repeated +// bytes reported as agreement. +{ + // Compared by CONTENT, not by name. A resolved path catches `./run1.txt` and + // a symlink, but not `copy-of-run1.txt`, which has a different real path and + // identical bytes -- and `paths.length` is both the run count and the control + // band's `(n)`, so a copy would narrow the reported range and inflate the + // sample size without adding an observation. + // + // **This is a heuristic, and its assumption is worth stating.** The report + // carries no per-invocation identifier, so "identical bytes" stands in for + // "same run". Two genuinely independent runs producing identical bytes would + // be refused -- possible in principle, since the figures are rounded, and + // vanishingly unlikely across this many of them. The refusal is loud and + // diagnosable; accepting a duplicated run would silently fabricate agreement, + // which is the worse of the two. `M4.8` replaces the heuristic with a real + // identity once the report carries one. + const seen = new Map(); + for (const path of paths) { + const digest = crypto.createHash("sha256").update(fs.readFileSync(path)).digest("hex"); + if (seen.has(digest)) { + fail(`the same capture was given twice: ${path} is byte-identical to ${seen.get(digest)}`); + process.exit(2); + } + seen.set(digest, path); + } +} + +// The attribution a run prints about itself. A capture is runs of one build on +// one host under one sampling regime, so these must agree across the inputs -- +// mixing a debug run, or another machine's, yields a median and a control span +// that describe no configuration that was ever measured, and nothing else here +// would notice. +function attribution(text, where) { + const line = (prefix) => { + const found = text.split(/\r?\n/).find((l) => l.startsWith(prefix)); + if (found === undefined) { + fail(`${where}: no "${prefix}" line; this does not look like a probe report`); + process.exit(2); + } + return found.trim(); + }; + const profile = line("profile:"); + // Agreement is not enough on its own: three debug reports agree with each + // other, and the probe stamps a debug run "NOT A MEASUREMENT" precisely + // because its figures are not one. A capture built from them would be + // internally consistent and meaningless. + if (profile !== "profile: release") { + fail(`${where}: ${profile} -- only a release run is a measurement`); + process.exit(2); + } + // Included because it can differ while the host banner does not: an affinity + // mask changes how many processors the process may use without changing the + // machine it names, and producer counts are read against that number. + return [line("host:"), profile, line("sampling:"), line("processors available to this process:")].join( + " | ", + ); +} + +function requireOneConfiguration(entries) { + // **What this cannot check.** The report carries no build identity, so two + // runs of DIFFERENT probe commits on one host under one profile agree here. + // That matters most for exactly this capture: `M4.3` changed the drained + // procedure, so a pre-handshake and a post-handshake run would pass and have + // their medians combined as though one procedure produced both. The + // instrument commit is asserted by the capture README, which is a claim by + // the person who took the capture rather than something these scripts verify. + // Closing it needs the probe to stamp its own build identity into the report; + // `M4.8` in CHECKLIST.md owns that. + const first = entries[0]; + for (const entry of entries.slice(1)) { + if (entry.attribution !== first.attribution) { + fail( + `${entry.name} was taken under different conditions from ${first.name}:\n` + + ` ${first.name}: ${first.attribution}\n` + + ` ${entry.name}: ${entry.attribution}`, + ); + process.exit(2); + } + } +} + +const layouts = []; +const comparisons = []; +const controls = []; +requireOneConfiguration( + paths.map((p) => ({ name: p, attribution: attribution(fs.readFileSync(p, "utf8"), p) })), +); +for (const path of paths) { + const lines = fs.readFileSync(path, "utf8").split(/\r?\n/); + const layout = drainedLayout(lines, path); + const comparison = drainedComparison(lines, path); + layouts.push(layout); + comparisons.push(comparison); + // The same code measured twice in one run: `reserving_mpsc` in the comparison + // table against `32/32` in the layout table. + const control = new Map(); + for (const [producers, row] of layout) { + const reserving = comparison.get(producers); + if (reserving === undefined) { + problems.push( + `${path}: ${producers} producers has a layout row but no comparison row`, + ); + continue; + } + if (row.narrowNanos > 0) { + control.set(producers, reserving / row.narrowNanos); + } + } + controls.push(control); +} + +// The sweep the probe runs, stated rather than inferred. Deriving the expected +// set from the first capture makes the completeness check circular: three runs +// all truncated at the same producer count agree with each other, `rows` stays +// non-empty, and the script reports a partial capture as a whole one. +const EXPECTED_PRODUCERS = [1, 2, 4, 8, 16, 32]; + +layouts.forEach((layout, i) => { + const seen = [...layout.keys()].sort((a, b) => a - b); + if (seen.join(",") !== EXPECTED_PRODUCERS.join(",")) { + problems.push( + `${paths[i]}: drained layout covers producers [${seen}], expected [${EXPECTED_PRODUCERS}]`, + ); + } +}); + +// The comparison table needs the same check, and for the same reason: the loop +// that builds the control iterates the LAYOUT's keys, so an extra comparison row +// is accepted into its map and then never looked at. Only the layout side was +// checked, which left half the capture able to carry rows nothing reported. +comparisons.forEach((comparison, i) => { + const seen = [...comparison.keys()].sort((a, b) => a - b); + if (seen.join(",") !== EXPECTED_PRODUCERS.join(",")) { + problems.push( + `${paths[i]}: drained comparison covers producers [${seen}], expected [${EXPECTED_PRODUCERS}]`, + ); + } +}); + +const producers = EXPECTED_PRODUCERS; +out(`runs: ${paths.length}`); + +// **Reported per producer count, and deliberately without a verdict.** +// +// An earlier version pooled every control observation into one band and asked +// whether each layout median fell inside it. That answered `true`, and the +// answer was an artifact of the pooling: the control is not independent of +// producer count -- it spans about 0.82-0.98x at one producer against +// 0.95-1.23x at thirty-two on this capture -- so pooling builds a band wider +// than any count's own and containment follows from the method rather than +// from the data. +// +// Comparing per count instead does not rescue a verdict either. Three runs +// give three control observations per count, and the range of three samples is +// not a band: a fresh independent draw falls outside the range of three priors +// about half the time. So neither comparison is strong enough to say a median +// is inside or outside, and this script says so rather than picking whichever +// framing yields an answer. +const rows = []; +for (const p of producers) { + const present = layouts.filter((layout) => layout.has(p)); + if (present.length !== layouts.length) { + problems.push( + `${p} producers is missing from ${layouts.length - present.length} run(s)`, + ); + continue; + } + const control = controls.map((c) => c.get(p)).filter((v) => Number.isFinite(v)); + if (control.length !== controls.length) { + problems.push(`${p} producers has no control ratio in every run`); + continue; + } + rows.push({ + producers: p, + control, + medians: [0, 1, 2].map((column) => + median(present.map((layout) => layout.get(p).ratios[column])), + ), + }); +} + +// **Refuse to summarise a capture that could not be read.** A report renders +// `--` for a shape that did not run, `Number("--")` is `NaN`, and every +// comparison against `NaN` is false -- so an unreadable cell used to empty the +// "outside the band" list and print `true`. +if (problems.length > 0 || rows.length === 0) { + out(""); + out("CAPTURE INCOMPLETE -- not summarised:"); + if (rows.length === 0) out(" - no complete producer counts were read"); + for (const problem of problems) out(` - ${problem}`); + process.exitCode = 1; + return; +} + +out(""); +out( + "drained, per producer count: the same-code control's observed range, then", +); +out("each layout's median ratio against 32/32, across runs."); +out(""); +// Derived, with the table's original width as a floor: a control band is built +// from measured ratios and has no upper bound, so a fixed field would shift the +// layout columns the first time one outgrew it. +const bands = rows.map((row) => { + const low = Math.min(...row.control); + const high = Math.max(...row.control); + return `${low.toFixed(2)}-${high.toFixed(2)}(${row.control.length})`; +}); +const bandWidth = Math.max(16, "control(n)".length, ...bands.map((b) => b.length)); +out(`producers ${"control(n)".padEnd(bandWidth)} 16/48 8/56 64/64`); +rows.forEach((row, i) => { + out( + `${String(row.producers).padStart(9)} ${bands[i].padEnd(bandWidth)} ` + + row.medians.map((m) => `${m.toFixed(2)}x`).join(" "), + ); +}); + +const everyControl = rows.flatMap((row) => row.control); +out(""); +out( + `control observations: ${everyControl.length} across ${rows.length} producer counts, ` + + `${Math.min(...everyControl).toFixed(2)}x to ${Math.max(...everyControl).toFixed(2)}x pooled`, +); +out( + "Pooled only to show the spread; it is not a band to judge a median against,", +); +out("for the reason recorded in this script beside the table above."); diff --git a/crates/windows-platform-probes/captures/2026-09-16-drained-handshake/summary.txt b/crates/windows-platform-probes/captures/2026-09-16-drained-handshake/summary.txt new file mode 100644 index 000000000..2e16335be --- /dev/null +++ b/crates/windows-platform-probes/captures/2026-09-16-drained-handshake/summary.txt @@ -0,0 +1,16 @@ +runs: 3 + +drained, per producer count: the same-code control's observed range, then +each layout's median ratio against 32/32, across runs. + +producers control(n) 16/48 8/56 64/64 + 1 0.82-0.98(3) 1.01x 1.03x 1.17x + 2 0.92-1.06(3) 0.98x 1.03x 1.06x + 4 0.95-1.02(3) 1.00x 1.00x 1.09x + 8 0.92-1.00(3) 0.98x 1.01x 1.06x + 16 0.75-1.04(3) 0.91x 0.93x 0.80x + 32 0.95-1.23(3) 1.23x 1.20x 0.94x + +control observations: 18 across 6 producer counts, 0.75x to 1.23x pooled +Pooled only to show the spread; it is not a band to judge a median against, +for the reason recorded in this script beside the table above. diff --git a/crates/windows-platform-probes/src/bin/queue_contention/corpus.json b/crates/windows-platform-probes/src/bin/queue_contention/corpus.json new file mode 100644 index 000000000..f3e312f12 --- /dev/null +++ b/crates/windows-platform-probes/src/bin/queue_contention/corpus.json @@ -0,0 +1,894 @@ +{ + "_comment": [ + "Cases that drive the queue-contention report renderer. Each supplies an", + "observation and what the rendered report must be true of. Adding a case is", + "adding data -- no Rust changes -- which is the point: the report's tables", + "were unreachable by the suite until M4.7, and two defects shipped through", + "that gap.", + "", + "`aligned_tables` maps a header substring to how many tables the report must", + "head with it. Every non-blank line of each of those tables, header included,", + "must be the same length. That is exactly the", + "property a cell wider than its column breaks, and it is derived from the", + "output rather than restated as a golden -- so it catches a width bug the", + "corpus never anticipated, which a golden cannot.", + "The count is part of the expectation rather than a minimum: the raw table", + "and the claim-layout table are each rendered twice, once per regime, under", + "identical headers. Checking only that at least one matched would let a", + "regime stop being reported without any assertion failing -- there would", + "simply be one fewer table to check, which is silence rather than failure.", + "A table that is legitimately empty is named in `contains` instead of here:", + "when nothing was measured the raw table is still headed but has no data", + "rows, and a header on its own always aligns with itself, so listing it", + "would buy a vacuous assertion rather than a check." + ], + "cases": [ + { + "name": "ordinary", + "why": "A plausible run. Establishes that the tables render and line up at all, so the stressed cases below are testing something. It also carries measured `reserving(64/64)` rows in both regimes: that layout is cfg-gated behind a native 128-bit exchange, so without them the renderer's 64/64 columns were only ever exercised by the did-not-run path, and a width or alignment fault in a real Wide measurement would have rendered unseen.", + "observation": { + "available_parallelism": 8, + "isolated": [ + [ + "baseline_fetch_add", + 1, + 2.3, + 434782608, + 0, + 2.1, + 2.5 + ], + [ + "baseline_fetch_add", + 2, + 12.1, + 165289256, + 0, + 11, + 13 + ], + [ + "slotwise_mpsc", + 1, + 6.3, + 158730158, + 0, + 6, + 6.8 + ], + [ + "slotwise_mpsc", + 2, + 50.6, + 39525691, + 0, + 19.3, + 59.5 + ], + [ + "reserving_mpsc", + 1, + 5.4, + 185185185, + 0, + 5.2, + 5.9 + ], + [ + "reserving_mpsc", + 2, + 31.9, + 31347962, + 0, + 22.5, + 35.2 + ], + [ + "permit_mpsc", + 1, + 7.9, + 126582278, + 0, + 7.5, + 8.3 + ], + [ + "permit_mpsc", + 2, + 44.2, + 22624434, + 0, + 37.4, + 45.9 + ], + [ + "reserving(32/32)", + 1, + 5.5, + 181818181, + 0, + 5.3, + 6 + ], + [ + "reserving(32/32)", + 2, + 32.4, + 30864197, + 0, + 23, + 35.9 + ], + [ + "reserving(16/48)", + 1, + 5.6, + 178571428, + 0, + 5.4, + 6.1 + ], + [ + "reserving(16/48)", + 2, + 33.1, + 30211480, + 0, + 23.4, + 36.2 + ], + [ + "reserving(8/56)", + 1, + 5.7, + 175438596, + 0, + 5.5, + 6.2 + ], + [ + "reserving(8/56)", + 2, + 33.4, + 29940119, + 0, + 23.8, + 36.5 + ], + [ + "reserving(64/64)", + 1, + 7.5, + 133333333, + 0, + 7.1, + 7.9 + ], + [ + "reserving(64/64)", + 8, + 61.7, + 16207455, + 0, + 54.1, + 88.1 + ] + ], + "drained": [ + [ + "slotwise_mpsc", + 1, + 11.4, + 87719298, + 0, + 10.9, + 12.1 + ], + [ + "slotwise_mpsc", + 2, + 66.8, + 29940119, + 1200, + 60.1, + 70.2 + ], + [ + "reserving_mpsc", + 1, + 26.2, + 38167938, + 0, + 24.9, + 27.8 + ], + [ + "reserving_mpsc", + 2, + 64.2, + 31152647, + 1100, + 58.3, + 68 + ], + [ + "permit_mpsc", + 1, + 64.3, + 15552099, + 0, + 61, + 67.1 + ], + [ + "permit_mpsc", + 2, + 56.2, + 35587188, + 900, + 51.4, + 59.9 + ], + [ + "reserving(32/32)", + 1, + 24.8, + 40322580, + 0, + 23.6, + 26.3 + ], + [ + "reserving(32/32)", + 2, + 65.1, + 30721966, + 1150, + 59, + 69.1 + ], + [ + "reserving(16/48)", + 1, + 26.8, + 37313432, + 0, + 25.4, + 28.2 + ], + [ + "reserving(16/48)", + 2, + 67.9, + 29455081, + 1180, + 61.2, + 71.4 + ], + [ + "reserving(8/56)", + 1, + 27.6, + 36231884, + 0, + 26.1, + 29 + ], + [ + "reserving(8/56)", + 2, + 67, + 29850746, + 1160, + 60.5, + 70.8 + ], + [ + "reserving(64/64)", + 1, + 29.3, + 34129692, + 12, + 27.4, + 31 + ], + [ + "reserving(64/64)", + 8, + 103.3, + 9680542, + 480, + 92.1, + 118.4 + ] + ] + }, + "expect": { + "aligned_tables": { + "16/48 vs": 2, + "reserving/slotwise": 1, + "atomic floor": 1, + "ns/op range": 2 + }, + "contains": [ + "processors available to this process: 8", + "reserving(64/64)" + ], + "absent": [] + } + }, + { + "name": "spans_wide_enough_to_overrun_a_fixed_column", + "why": "A cell's width is a function of measured data: `format_ratio_bounded` sizes its interval from the observed span, so no constant is provably sufficient. These values are chosen to exceed a fixed width rather than to depict a likely run -- and the margin matters, because the width of 22 that two tables carried was NOT overrun by an ordinary outlier. A 300ms repetition against a 4ns one renders 21 characters. Reaching 23 needs a hundredfold ratio as well. The argument for deriving the width is that the bound cannot be established, not that 22 was obviously too small.", + "observation": { + "available_parallelism": 8, + "isolated": [ + [ + "baseline_fetch_add", + 1, + 2.3, + 434782608, + 0, + 2.1, + 2.5 + ], + [ + "baseline_fetch_add", + 2, + 12.1, + 165289256, + 0, + 11, + 13 + ], + [ + "slotwise_mpsc", + 1, + 6, + 166666666, + 0, + 4, + 6 + ], + [ + "slotwise_mpsc", + 2, + 30, + 66666666, + 0, + 4, + 60 + ], + [ + "reserving_mpsc", + 1, + 5, + 200000000, + 0, + 4, + 6 + ], + [ + "reserving_mpsc", + 2, + 3000, + 666666, + 0, + 40, + 90000 + ], + [ + "permit_mpsc", + 1, + 8, + 125000000, + 0, + 4, + 6 + ], + [ + "permit_mpsc", + 2, + 3000, + 666666, + 0, + 40, + 90000 + ], + [ + "reserving(32/32)", + 1, + 5, + 200000000, + 0, + 4, + 6 + ], + [ + "reserving(32/32)", + 2, + 30, + 66666666, + 0, + 4, + 6 + ], + [ + "reserving(16/48)", + 1, + 50, + 20000000, + 0, + 40, + 6000 + ], + [ + "reserving(16/48)", + 2, + 3000, + 666666, + 0, + 40, + 90000 + ], + [ + "reserving(8/56)", + 1, + 60, + 16666666, + 0, + 40, + 7000 + ], + [ + "reserving(8/56)", + 2, + 3200, + 625000, + 0, + 40, + 95000 + ] + ], + "drained": [ + [ + "slotwise_mpsc", + 1, + 11, + 90909090, + 0, + 4, + 6 + ], + [ + "slotwise_mpsc", + 2, + 30, + 66666666, + 1200, + 4, + 60 + ], + [ + "reserving_mpsc", + 1, + 26, + 38461538, + 0, + 4, + 60 + ], + [ + "reserving_mpsc", + 2, + 3000, + 666666, + 1100, + 40, + 90000 + ], + [ + "permit_mpsc", + 1, + 64, + 15625000, + 0, + 4, + 60 + ], + [ + "permit_mpsc", + 2, + 3000, + 666666, + 900, + 40, + 90000 + ], + [ + "reserving(32/32)", + 1, + 25, + 40000000, + 0, + 4, + 6 + ], + [ + "reserving(32/32)", + 2, + 30, + 66666666, + 1150, + 4, + 6 + ], + [ + "reserving(16/48)", + 1, + 250, + 4000000, + 0, + 40, + 9000 + ], + [ + "reserving(16/48)", + 2, + 3000, + 666666, + 1180, + 40, + 90000 + ], + [ + "reserving(8/56)", + 1, + 270, + 3703703, + 0, + 40, + 9000 + ], + [ + "reserving(8/56)", + 2, + 3200, + 625000, + 1160, + 40, + 95000 + ] + ] + }, + "expect": { + "aligned_tables": { + "16/48 vs": 2, + "reserving/slotwise": 1, + "atomic floor": 1, + "ns/op range": 2 + }, + "contains": [ + "100.00x [6.67-22500.00]" + ], + "absent": [] + } + }, + { + "name": "a_shape_that_did_not_run", + "why": "A row present but carrying the did-not-run sentinel. Zero is not an obviously broken value in any of this report's columns, so the rendered cells must be marked rather than published: `0.0` ns/op reads as immeasurably fast and `0.00x` as a ratio of one.", + "observation": { + "available_parallelism": 4, + "isolated": [ + [ + "baseline_fetch_add", + 1, + 2.3, + 434782608, + 0, + 2.1, + 2.5 + ], + [ + "slotwise_mpsc", + 1, + 6.3, + 158730158, + 0, + 6, + 6.8 + ], + [ + "reserving_mpsc", + 1, + 5.4, + 185185185, + 0, + 5.2, + 5.9 + ], + [ + "permit_mpsc", + 1, + 0, + 0, + 0, + 0, + 0 + ], + [ + "reserving(32/32)", + 1, + 5.5, + 181818181, + 0, + 5.3, + 6 + ], + [ + "reserving(16/48)", + 1, + 0, + 0, + 0, + 0, + 0 + ], + [ + "reserving(8/56)", + 1, + 5.7, + 175438596, + 0, + 5.5, + 6.2 + ] + ], + "drained": [ + [ + "slotwise_mpsc", + 1, + 11.4, + 87719298, + 0, + 10.9, + 12.1 + ], + [ + "reserving_mpsc", + 1, + 26.2, + 38167938, + 0, + 24.9, + 27.8 + ], + [ + "permit_mpsc", + 1, + 0, + 0, + 0, + 0, + 0 + ], + [ + "reserving(32/32)", + 1, + 24.8, + 40322580, + 0, + 23.6, + 26.3 + ], + [ + "reserving(16/48)", + 1, + 0, + 0, + 0, + 0, + 0 + ], + [ + "reserving(8/56)", + 1, + 27.6, + 36231884, + 0, + 26.1, + 29 + ] + ] + }, + "expect": { + "aligned_tables": { + "16/48 vs": 2, + "reserving/slotwise": 1, + "atomic floor": 1, + "ns/op range": 2 + }, + "contains": [ + "--" + ], + "absent": [ + "0.00x", + " 0.0 ", + "infx", + "NaN" + ] + } + }, + { + "name": "nothing_measured_at_all", + "why": "Every shape absent, which is what a `cfg`-elided target or a failed run looks like from the renderer's side. It must still produce a report with its headers rather than panicking or printing figures it does not have.", + "observation": { + "available_parallelism": null, + "isolated": [], + "drained": [] + }, + "expect": { + "aligned_tables": { + "16/48 vs": 2, + "reserving/slotwise": 1, + "atomic floor": 1 + }, + "contains": [ + "processors available to this process: unknown (the query failed)", + "ns/op range" + ], + "absent": [ + "0.00x", + "infx", + "NaN" + ] + } + }, + { + "name": "a_row_whose_median_looks_fine_but_whose_endpoints_do_not", + "why": "A median is not the only field a row publishes. `render_table` prints ops/sec and both range endpoints directly once `is_measured` says the row counts, so a plausible median beside a poisoned endpoint reached the report as an ordinary-looking cell. These rows keep an ordinary median and spoil one other field each: a zero slowest endpoint, a negative fastest one, a zero ops/sec. JSON cannot express infinity, so the non-finite half of the guard is not reachable from a corpus case and is covered by the same predicate.", + "observation": { + "available_parallelism": 4, + "isolated": [ + [ + "baseline_fetch_add", + 1, + 2.3, + 0, + 0, + 2.1, + 2.5 + ], + [ + "slotwise_mpsc", + 1, + 6.3, + 158730158, + 0, + 6, + 0 + ], + [ + "reserving_mpsc", + 1, + 5.4, + 185185185, + 0, + -1, + 5.9 + ], + [ + "permit_mpsc", + 1, + 0, + 0, + 0, + 0, + 0 + ], + [ + "reserving(32/32)", + 1, + 5.5, + 181818181, + 0, + 5.3, + 6 + ], + [ + "reserving(16/48)", + 1, + 0, + 0, + 0, + 0, + 0 + ], + [ + "reserving(8/56)", + 1, + 5.7, + 175438596, + 0, + 5.5, + 6.2 + ] + ], + "drained": [ + [ + "slotwise_mpsc", + 1, + 11.4, + 87719298, + 0, + 10.9, + 12.1 + ], + [ + "reserving_mpsc", + 1, + 26.2, + 38167938, + 0, + 24.9, + 27.8 + ], + [ + "permit_mpsc", + 1, + 0, + 0, + 0, + 0, + 0 + ], + [ + "reserving(32/32)", + 1, + 24.8, + 40322580, + 0, + 23.6, + 26.3 + ], + [ + "reserving(16/48)", + 1, + 0, + 0, + 0, + 0, + 0 + ], + [ + "reserving(8/56)", + 1, + 27.6, + 36231884, + 0, + 26.1, + 29 + ] + ] + }, + "expect": { + "aligned_tables": { + "16/48 vs": 2, + "reserving/slotwise": 1, + "atomic floor": 1, + "ns/op range": 2 + }, + "contains": [ + "--" + ], + "absent": [ + "6.0-0.0", + "-1.0-5.9", + "0.00x", + "infx", + "NaN" + ] + } + } + ] +} diff --git a/crates/windows-platform-probes/src/bin/queue_contention/main.rs b/crates/windows-platform-probes/src/bin/queue_contention/main.rs new file mode 100644 index 000000000..a34602898 --- /dev/null +++ b/crates/windows-platform-probes/src/bin/queue_contention/main.rs @@ -0,0 +1,562 @@ +// Copyright (c) Mike Grier. + +//! Prints how the array queue's tail claim behaves as producers are added. +//! +//! **An experiment, not a component.** These probes measure platform behaviour +//! and are not for production use. Do not call them from production code, and +//! do not lift a technique out of here. See this crate's +//! [DESIGN-NOTES.md](../../../DESIGN-NOTES.md). +//! +//! This reports observations that bear on two questions otherwise settled by +//! taste: whether the linked and sharded MPSC shapes are ever needed, and +//! whether `slotwise_mpsc` and `reserving_mpsc` should merge. It does not settle +//! either -- see `queue_contention`'s module docs for what the regimes can and +//! cannot separate. + +use windows_platform_probes::queue_contention::{ + DRAINED_CAPACITY, Observation, PRODUCER_COUNTS, PUSHES_PER_PRODUCER, REPETITIONS, column_width, + format_nanos, format_ratio_bounded, format_scaling_bounded, measure, ratio_column_width, + render_table, shapes, +}; +use windows_platform_probes::report::emit_report; + +fn main() { + // The probe's whole output policy, and it is one line: hand the renderer to + // the sink. Nothing here or below names a stream -- that is chosen once, in + // `report`, so retargeting a probe is not a rewrite. + emit_report(render); +} + +#[cfg(test)] +mod tests; + +/// Measure, then render what was measured. +/// +/// **The two halves are separate so the second one can be tested.** Rendering +/// used to measure inside itself, which made the only way to exercise it a +/// ~65-second host-dependent run -- so the table assembly, the derived column +/// widths, the `cfg`-gated rows and the prose between them were reached by +/// nothing in the suite. Two defects shipped through that gap on this branch: +/// two tables hard-coded a column width for formatters whose output has no +/// fixed maximum, and the drained footer printed a seven-run result beneath a +/// table produced by one invocation. See `M4.7`. +fn render(out: &mut dyn std::fmt::Write) { + // The banner is the one line that is not a function of the observation -- + // it is a fresh topology read -- so it is written here and + // `render_observation` stays a pure function of what was measured. That + // purity is the whole point of the split: it is what lets a fixture drive + // the entire report. + // + // First line of the report, and part of the returned text rather than + // written out here: a captured report must carry the line naming the + // machine that produced it, and the taint marker with it. + let _ = writeln!( + out, + "{}", + windows_placement_probe::fingerprint::banner_line() + ); + render_observation(out, &measure()); +} + +/// Everything the report says about an observation. +fn render_observation(out: &mut dyn std::fmt::Write, observation: &Observation) { + let _ = writeln!( + out, + "== how does the array queue's push path scale with producer count? ==\n" + ); + + // `available_parallelism`, not the host count -- an affinity mask or job + // object narrows it, and saying "host reports" under either would contradict + // the banner three lines up. The host's shape is already there; this is what + // decides whether a producer count oversubscribes THIS run. + let _ = match observation.available_parallelism { + Some(count) => writeln!(out, "processors available to this process: {count}\n"), + None => writeln!( + out, + "processors available to this process: unknown (the query failed)\n" + ), + }; + // The sampling parameters are capture parameters, and a figure is only + // interpretable with them -- see D-observations-not-verdicts. The build + // profile is one of them too: a captured report has to be able to show it + // was produced by a build that can measure, not merely stay silent when it + // was. The dispersion belongs here as well, and now is: each row carries the + // range across its repetitions and the resulting spread. What M4.2 still + // covers is making the sampling parameters settable rather than fixed. + let _ = writeln!( + out, + "profile: {}", + if cfg!(debug_assertions) { + "debug -- NOT A MEASUREMENT, see below" + } else { + "release" + } + ); + let _ = writeln!( + out, + "sampling: {} pushes per producer, median of {} repetitions, one untimed \ + warmup pass\n", + PUSHES_PER_PRODUCER, REPETITIONS + ); + // A debug build does not merely lose precision here: the un-inlined overhead + // swamps the cache-coherence effects that ARE the finding, and the two MPSC + // shapes come out indistinguishable -- a confident wrong answer. The banner + // goes in the report rather than in a refusal to run, because it has to + // travel with a captured report: whoever pastes these numbers somewhere is + // the person who needs to see it, and a binary that refused would tell only + // the person who already had the terminal open. + if cfg!(debug_assertions) { + let _ = writeln!( + out, + "!! DEBUG BUILD -- THESE NUMBERS ARE NOT A MEASUREMENT !!" + ); + let _ = writeln!( + out, + "!! Un-inlined overhead swamps the effect being measured, and the" + ); + let _ = writeln!( + out, + "!! MPSC shapes report as equivalent when they are not. Rebuild" + ); + let _ = writeln!(out, "!! with --release before reading anything below.\n"); + } + + let _ = writeln!( + out, + "-- isolated: producers only, capacity large enough that nothing is refused --" + ); + render_table(out, &observation.isolated); + + let _ = writeln!( + out, + "\n-- drained: a consumer looping on pop, capacity {DRAINED_CAPACITY} --" + ); + render_table(out, &observation.drained); + + let _ = writeln!(out, "\ninterpretation:\n"); + + // Question 1: does the claim collapse as producers are added? + let _ = writeln!( + out, + " 1. push-path scaling with producer count (isolated regime)\n" + ); + // Same two-pass shape as the claim-layout table below, and for the same + // reason: `format_scaling_bounded` renders a point plus a measured interval, + // whose width follows the data and has no fixed maximum. See + // `ratio_column_width`. + let scaling_rows: Vec<(usize, [String; 4])> = PRODUCER_COUNTS + .iter() + .map(|&producers| { + let cell = |shape: &str| { + format_scaling_bounded( + observation.scaling(&observation.isolated, shape, producers), + observation.scaling_bounds(&observation.isolated, shape, producers), + ) + }; + ( + producers, + [ + cell(shapes::SLOTWISE_MPSC), + cell(shapes::RESERVING_MPSC), + cell(shapes::PERMIT_MPSC), + cell(shapes::BASELINE_FETCH_ADD), + ], + ) + }) + .collect(); + let w = ratio_column_width( + scaling_rows + .iter() + .flat_map(|(_, cells)| cells) + .map(String::as_str), + ); + let _ = writeln!( + out, + " {:<12} {:>w$} {:>w$} {:>w$} {:>w$}", + "producers", "slotwise", "reserving", "permit", "atomic floor" + ); + for (producers, cells) in &scaling_rows { + let _ = writeln!( + out, + " {producers:<12} {:>w$} {:>w$} {:>w$} {:>w$}", + cells[0], cells[1], cells[2], cells[3], + ); + } + let _ = writeln!( + out, + "\n Read as: throughput at N producers divided by throughput at one." + ); + let _ = writeln!( + out, + " 1.00 means N threads together push no faster than one did." + ); + let _ = writeln!( + out, + " The atomic floor is the cheapest possible contended operation," + ); + let _ = writeln!( + out, + " so it says how much of any curve is the queue and how much is" + ); + let _ = writeln!( + out, + " simply what this processor does to a fought-over cache line." + ); + + // Question 2: what does reserving_mpsc's read of `head` actually cost? + let _ = writeln!( + out, + "\n 2. reserving vs slotwise, drained (where `head` is written)\n" + ); + let _ = writeln!( + out, + " The ratio is the WHOLE push path of two different shapes, not the" + ); + let _ = writeln!( + out, + " price of reserving's extra `head` load on its own: they use" + ); + let _ = writeln!( + out, + " different claim protocols, slot metadata and retry behaviour. This" + ); + let _ = writeln!( + out, + " regime is where that load is at its most expensive -- but the" + ); + let _ = writeln!( + out, + " ratio still does not isolate it, or bound it either way.\n" + ); + // Two-pass again: both ratio columns hold `format_ratio_bounded` output, + // whose width follows the measured span. The header labels join the + // derivation rather than being assumed to fit, so the column is correct by + // construction instead of by the floor happening to exceed them. + let drained_rows: Vec<(usize, String, String, String, String, String)> = PRODUCER_COUNTS + .iter() + .map(|&producers| { + let plain = observation.find(&observation.drained, shapes::SLOTWISE_MPSC, producers); + let reserving = + observation.find(&observation.drained, shapes::RESERVING_MPSC, producers); + let permit = observation.find(&observation.drained, shapes::PERMIT_MPSC, producers); + ( + producers, + format_nanos(plain), + format_nanos(reserving), + format_ratio_bounded(reserving, plain), + format_nanos(permit), + // The column SH-15.5 exists to fill: the experimental claim + // against the shipping shape it would replace. It is an + // end-to-end ratio of two whole push paths, not a price on the + // room-decision race: the shapes also differ in refusal and + // retry behaviour, and the queue crate's D-35 records that in + // this regime their refusal counts differ by orders of + // magnitude the harness cannot attribute -- which is what + // SH-15.5.1 exists to settle. + format_ratio_bounded(permit, reserving), + ) + }) + .collect(); + let r = ratio_column_width( + drained_rows + .iter() + .flat_map(|(_, _, _, ratio, _, permit_ratio)| [ratio.as_str(), permit_ratio.as_str()]) + .chain(["reserving/slotwise", "permit/reserving", "ratio [bound]"]), + ); + let _ = writeln!( + out, + " {:<10} {:>12} {:>12} {:>r$} {:>12} {:>r$}", + "producers", "slotwise", "reserving", "reserving/slotwise", "permit", "permit/reserving" + ); + let _ = writeln!( + out, + " {:<10} {:>12} {:>12} {:>r$} {:>12} {:>r$}", + "", "ns/op", "ns/op", "ratio [bound]", "ns/op", "ratio [bound]" + ); + for (producers, plain, reserving, ratio, permit, permit_ratio) in &drained_rows { + let _ = writeln!( + out, + " {producers:<10} {:>12} {:>12} {:>r$} {:>12} {:>r$}", + plain, reserving, ratio, permit, permit_ratio, + ); + } + let _ = writeln!( + out, + "\n `reserving_mpsc` reads the consumer's position on every push and" + ); + let _ = writeln!( + out, + " `slotwise_mpsc` does not. This regime is where that read is at its" + ); + let _ = writeln!( + out, + " most expensive, because a consumer is writing the line being read" + ); + let _ = writeln!( + out, + " -- but the ratio does not decompose. It is an END-TO-END" + ); + let _ = writeln!( + out, + " comparison of two shapes: they also differ in claim protocol," + ); + let _ = writeln!( + out, + " slot metadata and retry behaviour, and those differences are not" + ); + let _ = writeln!( + out, + " ordered. So this ratio neither isolates the read nor bounds it." + ); + let _ = writeln!( + out, + "\n `permit_mpsc` is experimental and is the candidate replacement" + ); + let _ = writeln!( + out, + " for `reserving_mpsc`: it removes that read entirely, and with it" + ); + let _ = writeln!( + out, + " the stale room decision behind SH-14.1, by making admission a" + ); + let _ = writeln!( + out, + " read-modify-write on a permit count instead. The last column is" + ); + let _ = writeln!( + out, + " the trade. Read the INTERVAL, not the point: one that crosses" + ); + let _ = writeln!( + out, + " 1.00 orders nothing, however far the point estimate sits from" + ); + let _ = writeln!(out, " it. An interval clear of"); + let _ = writeln!( + out, + " 1.00 orders the two WHOLE PUSH PATHS and not the room-decision" + ); + let _ = writeln!( + out, + " race on its own: the shapes differ in refusal and retry" + ); + let _ = writeln!( + out, + " behaviour too, and here their refusal counts differ by orders" + ); + let _ = writeln!(out, " of magnitude this harness cannot attribute."); + + // Question 3: what does the claim word's apportionment and width cost? + let _ = writeln!(out, "\n 3. claim-word layout\n"); + // Counted from what was actually MEASURED rather than from what is present: + // the 64/64 rows are cfg-elided on a target with no native 128-bit exchange, + // and a hardcoded "four" would be false there -- but a row can also be + // present while carrying the did-not-run sentinel, which `render_table` + // marks `--` and this sentence would otherwise still count. See + // `Observation::count_measured`. + let layouts_measured = observation.count_measured( + &observation.isolated, + &[ + shapes::CLAIM_NARROW, + shapes::CLAIM_DEEP, + shapes::CLAIM_PERPETUAL, + shapes::CLAIM_WIDE, + ], + PRODUCER_COUNTS[0], + ); + let _ = writeln!( + out, + " {layouts_measured} apportionments of reserving_mpsc's claim word, measured on" + ); + let _ = writeln!( + out, + " the shipping type itself rather than on a stand-in. 32/32 is the" + ); + let _ = writeln!( + out, + " default; 16/48 and 8/56 are the same u64 exchange with the bits" + ); + let _ = writeln!( + out, + " apportioned differently; 64/64 is a u128 exchange (cmpxchg16b on" + ); + let _ = writeln!( + out, + " x86-64, ldxp/stxp on aarch64), measured only where that is native." + ); + let _ = writeln!( + out, + " The three u64 rows issue the same instruction and differ only in" + ); + let _ = writeln!( + out, + " shift and mask constants, so there is no structural reason for one" + ); + let _ = writeln!( + out, + " to be slower -- but these rows time the WHOLE push path, so a" + ); + let _ = writeln!( + out, + " difference between them is not thereby noise. Read it against a" + ); + let _ = writeln!( + out, + " control before calling it either way: the reserving_mpsc row and" + ); + let _ = writeln!( + out, + " the 32/32 row above are the same code, so the gap between them is" + ); + let _ = writeln!( + out, + " what 'no difference' looks like on this host -- read it against" + ); + let _ = writeln!( + out, + " the layout rows before calling any of them apart." + ); + let _ = writeln!( + out, + " 64/64 vs 32/32 is the double-width layout's effect on the whole" + ); + let _ = writeln!( + out, + " push path -- what moving the recurrence to 2^64 costs, against" + ); + let _ = writeln!( + out, + " 8/56 moving it to 2^56. Both defer the recurrence rather than" + ); + let _ = writeln!(out, " removing it. Not the exchange in isolation.\n"); + for (label, regime) in [ + ("isolated", &observation.isolated), + ("drained", &observation.drained), + ] { + let _ = writeln!(out, " -- {label} --"); + // Every cell is rendered before the header is emitted, because the ratio + // column's width is derived from the widest value it must hold. A Rust + // width is a minimum, so sizing the header first and discovering a wider + // cell later does not truncate that cell -- it silently pushes the two + // columns after it out of line. See `ratio_column_width`. + let rows: Vec<(usize, [String; 4], [String; 3])> = PRODUCER_COUNTS + .iter() + .map(|&producers| { + let narrow = observation.find(regime, shapes::CLAIM_NARROW, producers); + let deep = observation.find(regime, shapes::CLAIM_DEEP, producers); + let perpetual = observation.find(regime, shapes::CLAIM_PERPETUAL, producers); + let wide = observation.find(regime, shapes::CLAIM_WIDE, producers); + ( + producers, + [ + format_nanos(narrow), + format_nanos(deep), + format_nanos(perpetual), + format_nanos(wide), + ], + [ + format_ratio_bounded(deep, narrow), + format_ratio_bounded(perpetual, narrow), + format_ratio_bounded(wide, narrow), + ], + ) + }) + .collect(); + let w = ratio_column_width( + rows.iter() + .flat_map(|(_, _, ratios)| ratios) + .map(String::as_str) + .chain(["16/48 vs", "8/56 vs", "64/64 vs"]), + ); + // `format_nanos` renders a measurement, so these columns are no more + // bounded than the ratio columns beside them. + let n = column_width( + rows.iter() + .flat_map(|(_, nanos, _)| nanos) + .map(String::as_str), + 11, + ); + let _ = writeln!( + out, + " {:<10} {:>n$} {:>n$} {:>n$} {:>n$} {:>w$} {:>w$} {:>w$}", + "producers", + "32/32 ns/op", + "16/48 ns/op", + "8/56 ns/op", + "64/64 ns/op", + "16/48 vs", + "8/56 vs", + "64/64 vs", + ); + for (producers, nanos, ratios) in &rows { + let _ = writeln!( + out, + " {:<10} {:>n$} {:>n$} {:>n$} {:>n$} {:>w$} {:>w$} {:>w$}", + producers, nanos[0], nanos[1], nanos[2], nanos[3], ratios[0], ratios[1], ratios[2], + ); + } + let _ = writeln!(out); + } + let _ = writeln!( + out, + " the 32/32 row and the reserving_mpsc row above are the same" + ); + let _ = writeln!( + out, + " configuration run twice, so the gap between them is this host's" + ); + let _ = writeln!( + out, + " same-code control: whatever it shows is dispersion, not a" + ); + let _ = writeln!( + out, + " difference between shapes. Do not read it as noise that can be" + ); + let _ = writeln!( + out, + " discounted -- its width is an open question about this" + ); + let _ = writeln!(out, " instrument. They"); + let _ = writeln!( + out, + " are no longer a control against a duplicated implementation: the" + ); + let _ = writeln!( + out, + " shipping type takes the layout as a parameter, so there is nothing" + ); + let _ = writeln!( + out, + " left that could drift away from what callers actually run." + ); + let _ = writeln!( + out, + "\n CAUTION: the drained regime has ONE consumer, because that is what" + ); + let _ = writeln!( + out, + " MPSC means. At high producer counts it is expected to become" + ); + let _ = writeln!( + out, + " consumer-bound, and a plateau there says nothing about the claim." + ); + let _ = writeln!( + out, + " The refusal counts above are what make that visible: a run with" + ); + let _ = writeln!( + out, + " many refusals met a full queue often, so the consumer is one term" + ); + let _ = writeln!( + out, + " in what it measured. That does not rule the tail out -- both can" + ); + let _ = writeln!( + out, + " bind at once, and these counts do not separate them." + ); +} diff --git a/crates/windows-platform-probes/src/bin/queue_contention/tests.rs b/crates/windows-platform-probes/src/bin/queue_contention/tests.rs new file mode 100644 index 000000000..6eb6992d3 --- /dev/null +++ b/crates/windows-platform-probes/src/bin/queue_contention/tests.rs @@ -0,0 +1,297 @@ +// Copyright (c) Mike Grier. + +//! The report renderer, driven by a corpus rather than by hand-written cases. +//! +//! Adding a case is adding data to [corpus.json](corpus.json) -- an observation, and what the +//! rendered report must be true of. Nothing here needs to change, which is the +//! point: the report's tables were reachable by nothing in the suite until +//! `M4.7`, because rendering measured inside itself and the only way to run it +//! was a ~65-second host-dependent pass. +//! +//! **The central check is derived, not restated.** `aligned_tables` asserts that +//! every line of a named table is the same length. That is exactly the property +//! a cell wider than its column breaks, and it holds for inputs the corpus never +//! anticipated -- where a golden would only catch what somebody thought to +//! record, and would have to be regenerated every time the prose moved. + +use super::render_observation; +use serde_json::Value; +use std::panic::AssertUnwindSafe; +use windows_platform_probes::queue_contention::{Observation, Run}; + +/// Asserts every line of a table shares its header's width. +/// +/// **The one statement of the alignment rule.** The corpus check calls it, and +/// so does the anti-vacuity test at the bottom of this file, so that test +/// exercises the assertion the corpus actually relies on. A test that compared +/// the line lengths itself would be a second copy of the rule, and would keep +/// passing after this body was deleted. +#[track_caller] +fn assert_aligned(name: &str, lines: &[&str], why: &str) { + let width = lines[0].len(); + for line in lines { + assert_eq!( + line.len(), + width, + "[{name}] a cell overran its column, so every column after it \ + no longer lines up with its header.\n{why}\n\ + header ({width}): {:?}\n line ({}): {line:?}", + lines[0], + line.len() + ); + } +} + +/// Compiled in, so a missing corpus is a build failure rather than a test that +/// silently runs nothing. Malformed JSON is caught a step later, when the corpus +/// is parsed: that is a test failure, not a build failure, because +/// `include_str!` checks only that the bytes exist. +const CORPUS: &str = include_str!("corpus.json"); + +/// A row is `[shape, producers, nanos_per_op, ops_per_second, refusals, +/// fastest, slowest]`, positionally. +/// +/// The struct literal is exhaustive, so adding a field to [`Run`] stops this +/// file compiling -- which is the reminder to decide what the corpus should say +/// about it, rather than letting a new field go unexercised. +fn run_from(value: &Value) -> Run { + let row = value.as_array().expect("a run is an array"); + let number = |index: usize| -> f64 { + row[index] + .as_f64() + .unwrap_or_else(|| panic!("field {index} of a run is a number")) + }; + // Read as an integer, not through `f64`. `as_f64` rounds past 2^53, so a + // refusal count of 9007199254740993 would silently become ...992; and `as` + // on the way back truncates a fraction and saturates a negative, so `1.5` + // would become producer count 1. A fixture must fail rather than quietly + // exercise a case it does not name. + let whole = |index: usize| -> u64 { + row[index] + .as_u64() + .unwrap_or_else(|| panic!("field {index} of a run is a non-negative whole number")) + }; + Run { + // Leaked so the fixture can hand back the `&'static str` the field + // wants. A test process is the one place that is the cheap answer, and + // the corpus is a fixed compiled-in set, so this cannot grow. + shape: Box::leak( + row[0] + .as_str() + .expect("a shape is a string") + .to_owned() + .into_boxed_str(), + ), + producers: usize::try_from(whole(1)).expect("a producer count fits a usize"), + nanos_per_op: number(2), + ops_per_second: number(3), + refusals: whole(4), + fastest_nanos_per_op: number(5), + slowest_nanos_per_op: number(6), + } +} + +fn observation_from(value: &Value) -> Observation { + let rows = |key: &str| -> Vec { + value[key] + .as_array() + .unwrap_or_else(|| panic!("`{key}` is an array")) + .iter() + .map(run_from) + .collect() + }; + Observation { + isolated: rows("isolated"), + drained: rows("drained"), + // `null` is a real observation -- the processor-count query failed -- so + // it cannot also be what a typo produces. `as_u64().map(...)` would + // return `None` for a missing field, a string, a negative, or a + // fraction, rendering the unknown-parallelism case and passing, while + // the corpus said something else entirely. + // + // Indexing with `[]` yields `Value::Null` for an absent key, which would + // have made an *omitted* field indistinguishable from an explicit one -- + // the same conflation one layer up, and exactly what the sentence above + // claims is impossible. `get` separates them. + available_parallelism: match value.get("available_parallelism").unwrap_or_else(|| { + panic!("a case states `available_parallelism`, using null where the query failed") + }) { + Value::Null => None, + other => { + // The real value comes from `NonZeroUsize`, so zero is not a + // processor count the probe can ever observe. `null` is the only + // representation of a failed query; accepting `0` as well would + // let a malformed corpus publish an impossible figure. + let count = other.as_u64().unwrap_or_else(|| { + panic!( + "`available_parallelism` is a positive whole number or null, not {other}" + ) + }); + assert!( + count > 0, + "`available_parallelism` is null when the query failed, not 0" + ); + Some(usize::try_from(count).expect("a processor count fits a usize")) + } + }, + } +} + +/// Every table whose header contains `header`, each with its header included. +/// +/// A table runs from its header to the first blank line. The drained tables +/// carry a second header row, which is part of the table and has to line up +/// with the rest of it, so it is not skipped. +/// +/// **All occurrences, not the first.** The report emits the claim-word layout +/// table twice under identical headers -- once isolated, once drained -- and +/// `ns/op range` heads both raw tables. Returning only the first meant the +/// corpus checked the isolated table and a width regression in the drained one +/// passed unseen, which is half the report unguarded. +fn tables<'a>(report: &'a str, header: &str) -> Vec> { + let all: Vec<&str> = report.lines().collect(); + let mut found = Vec::new(); + let mut index = 0; + while index < all.len() { + if all[index].contains(header) && is_columnar(all[index]) { + let table: Vec<&str> = all[index..] + .iter() + .take_while(|line| !line.trim().is_empty()) + .copied() + .collect(); + // Step past this table so its own rows cannot match again. + index += table.len().max(1); + found.push(table); + } else { + index += 1; + } + } + found +} + +/// Whether a line is a table header rather than prose that mentions one. +/// +/// Header names are matched as substrings, and the report's prose discusses the +/// columns it prints -- "The atomic floor is the cheapest possible contended +/// operation" contains `atomic floor` and is a sentence. Slicing from there +/// gathers a paragraph and compares the lengths of its lines, which fails for +/// the ordinary reason that prose is ragged. +/// +/// A header is columnar: its fields are separated by gaps of multiple spaces, so +/// it splits into two or more parts. Prose is single-spaced and splits into one. +/// That one test tells them apart without the fixture having to enumerate +/// either. +fn is_columnar(line: &str) -> bool { + line.trim() + .split(" ") + .filter(|part| !part.is_empty()) + .count() + >= 2 +} + +#[test] +fn every_corpus_case_renders_a_report_whose_tables_line_up() { + let corpus: Value = serde_json::from_str(CORPUS).expect("the corpus parses"); + let cases = corpus["cases"].as_array().expect("`cases` is an array"); + assert!( + !cases.is_empty(), + "an empty corpus would pass every assertion below without testing anything" + ); + + for case in cases { + let name = case["name"].as_str().expect("a case is named"); + let why = case["why"].as_str().expect("a case says why it exists"); + let observation = observation_from(&case["observation"]); + + let mut report = String::new(); + render_observation(&mut report, &observation); + + let expect = &case["expect"]; + let aligned = expect["aligned_tables"] + .as_object() + .expect("`aligned_tables` maps a header to how many times it must appear"); + for (header, count) in aligned { + let expected = count + .as_u64() + .expect("an expected occurrence count is a number") + as usize; + let found = tables(&report, header); + assert_eq!( + found.len(), + expected, + "[{name}] the report has {} table(s) headed {header:?}, expected {expected}. \ + A count that has dropped means a table stopped being rendered, which every \ + alignment assertion below would otherwise pass in silence.\n{why}\n{report}", + found.len() + ); + for (occurrence, lines) in found.iter().enumerate() { + assert!( + lines.len() > 1, + "[{name}] the table at {header:?} (occurrence {}) has no rows, \ + so its alignment is not being checked\n{why}", + occurrence + 1 + ); + assert_aligned(name, lines, why); + } + } + + for needle in expect["contains"] + .as_array() + .expect("`contains` is an array") + { + let needle = needle.as_str().expect("a needle is a string"); + assert!( + report.contains(needle), + "[{name}] expected {needle:?} in the report.\n{why}\n{report}" + ); + } + + for needle in expect["absent"].as_array().expect("`absent` is an array") { + let needle = needle.as_str().expect("a needle is a string"); + assert!( + !report.contains(needle), + "[{name}] {needle:?} must not reach the report.\n{why}\n{report}" + ); + } + } +} + +/// The alignment check must be able to fail, or the corpus proves nothing. +/// +/// Every case above passes, which is indistinguishable from a check that cannot +/// fail. This hands [`assert_aligned`] a deliberately overrun table and asserts +/// it panics -- the anti-vacuity half of a sabotage run, made in-suite because +/// the detector is the instrument here. +/// +/// It calls the same function the corpus calls rather than re-deriving the rule +/// from the fixture's line lengths. A test that compared the lengths itself +/// would keep passing after [`assert_aligned`]'s body was deleted, which is +/// exactly the failure it exists to rule out. +/// +/// The caught panic prints its message through the default hook, so a backtrace +/// line appears in this test's output on success. That is left alone: silencing +/// it means installing a process-global no-op panic hook, and this suite runs +/// its tests as threads in one process, so the window would swallow a concurrent +/// test's failure message. +#[test] +fn the_alignment_check_can_tell_a_misaligned_table_from_an_aligned_one() { + let misaligned = tables( + "producers ratio\n1 1.00x [1.00-1.00]\n", + "ratio", + ) + .remove(0); + assert_eq!(misaligned.len(), 2, "the fixture has a header and one row"); + let caught = std::panic::catch_unwind(AssertUnwindSafe(|| { + assert_aligned("fixture", &misaligned, "a deliberately overrun table"); + })); + assert!( + caught.is_err(), + "the detector passed a table whose row is {} wide against a {} header; \ + it cannot report a real overrun either", + misaligned[1].len(), + misaligned[0].len() + ); + + let aligned = tables("producers ratio\n 1 1.00x\n", "ratio").remove(0); + assert_aligned("fixture", &aligned, "an aligned table must not be reported"); +} diff --git a/crates/windows-platform-probes/src/lib.rs b/crates/windows-platform-probes/src/lib.rs index a507d54f7..d73d9fcd7 100644 --- a/crates/windows-platform-probes/src/lib.rs +++ b/crates/windows-platform-probes/src/lib.rs @@ -96,6 +96,11 @@ //! cargo test -p windows-platform-probes -- --include-ignored # both tiers //! cargo test -p windows-platform-probes -- --ignored # ignored tier only //! cargo run -p windows-platform-probes --bin probe-cancel-io # binary only +//! +//! # binary only, and --release is not optional: a debug build reports +//! # slotwise_mpsc and reserving_mpsc as equivalent, which is a confident +//! # wrong answer rather than a merely imprecise one. Takes about a minute. +//! cargo run --release -p windows-platform-probes --bin probe-queue-contention //! ``` //! //! `--include-ignored` is what CI runs, and is almost always what a human @@ -134,6 +139,7 @@ //! | [`doorbell_cost::measure`] | binary only | the absolute cost of `SetEvent`, a set/reset cycle and a satisfied wait against an uncontended atomic, and how much batching drives the doorbell below the push it accompanies | //! | [`doorbell_cost::measure_park_and_wake`] | asserted | that the park-and-wake handshake completes rather than deadlocking, which its first implementation did | //! | [`request_cost::measure`] | binary only | the absolute cost of preparing a path, building an owned `OpenFile`, and duplicating a handle | +//! | [`queue_contention::measure`] | binary only | how the bounded array queue's whole push path scales with producer count, against a contended `fetch_add` floor -- the tail claim is one term in that path, not the whole of it; how `reserving_mpsc` and `slotwise_mpsc` compare end to end in the regime where `reserving_mpsc`'s read of the consumer's position is most expensive (an end-to-end shape comparison -- it neither isolates that read nor bounds it, since the shapes differ in claim protocol and slot metadata too and those differences are not ordered); and each claim-word apportionment available on the target -- the 128-bit `64/64` row is measured only where that exchange is native, so a 32-bit or `cmpxchg16b`-less build reports three -- and its effect on that same whole path -- binary only because it wants more cores than a hosted runner has, takes about a minute, and reports the two shapes as equivalent unless built with `--release` | #![cfg(windows)] #![forbid(unsafe_op_in_unsafe_fn)] #![warn(missing_docs)] @@ -148,6 +154,7 @@ pub mod ioring; pub mod long_path; pub mod long_path_report; pub mod pool_growth; +pub mod queue_contention; pub mod report; /// The report oracle. **Test-support: present only where it is used.** /// diff --git a/crates/windows-platform-probes/src/queue_contention.rs b/crates/windows-platform-probes/src/queue_contention.rs new file mode 100644 index 000000000..9fe291a3c --- /dev/null +++ b/crates/windows-platform-probes/src/queue_contention.rs @@ -0,0 +1,1686 @@ +// Copyright (c) Mike Grier. + +//! How does the array queue's push path scale with producer count? +//! +//! The question behind it is whether the **tail claim** contends badly enough to +//! justify other MPSC shapes -- but what is timed is each shape's whole push +//! path, so the curve is push-path scaling and the claim is one term in it. See +//! the regime notes below before attributing any difference to the claim. +//! +//! **An experiment, not a component.** These probes measure platform behaviour +//! and are not for production use. Do not call them from production code, and +//! do not lift a technique out of here. See this crate's +//! [DESIGN-NOTES.md](../DESIGN-NOTES.md). +//! +//! # The two decisions this exists to force +//! +//! **1. Are the linked and sharded MPSC shapes needed at all?** They are parked +//! in a plan that is not in this repository yet, gated on this measurement +//! rather than on taste. (The named checklist file arrives with the rest of the +//! queue work; naming a path that does not resolve is what this crate's own +//! link rule forbids, and an earlier draft did it here.) +//! If N threads compare-and-swapping one tail does not collapse +//! at the producer counts a real system reaches, the bounded array queue is the +//! only MPSC the queue crate ever needs, and two speculative shapes never get +//! written. +//! +//! **2. Should `slotwise_mpsc` and `reserving_mpsc` merge?** They ship as peers because +//! honouring a reservation costs the producer a read of the consumer's +//! position -- one line every thread touches -- and *how much* that costs was a +//! judgement rather than a measurement. This probe does not turn it into one: +//! the drained rows compare two complete push paths and cannot separate that +//! read from the other differences between the shapes. What they supply is an +//! end-to-end comparison in the regime where the read is most expensive, which +//! is an input to that decision rather than the decision. +//! +//! # Two regimes, because one of them cannot answer the second question +//! +//! Producers are timed twice, and the pair is the point. +//! +//! - **Isolated** -- capacity large enough that nothing is ever refused, and no +//! consumer running. Nothing else touches the queue, so the curve against N +//! is the producer side alone, with no consumer traffic in it. +//! +//! **It is not the compare-and-swap alone, and an earlier draft said it +//! was.** What is timed is each shape's whole push path: the tail claim, but +//! also the slot-sequence load, the item write, the publication store, and +//! the doorbell's fence. `permit_mpsc` takes two shared read-modify-writes +//! where the others take one. So a difference between shapes here is a +//! difference in PUSH COST, and attributing it to the claim alone would be +//! reading more out of the number than is in it. Found by a review. +//! +//! - **Drained** -- a consumer looping on `pop` while the producers push. +//! This is the regime in which `reserving_mpsc`'s read of `head` is at its +//! most expensive, because `head` is only costly to read when a consumer is +//! *writing* it. Measured in isolation that read hits a clean, shared line and +//! looks free -- which would be a confident wrong answer. +//! +//! **It does not isolate that read, and it does not bound it either** -- an +//! earlier correction here claimed a bound, which is no better than the +//! over-claim it replaced. The ratio is between two complete push paths, and +//! `reserving_mpsc` and `slotwise_mpsc` differ in claim protocol, slot metadata +//! and retry behaviour as well as in that one load. Writing `R` and `S` for the +//! two totals, `R - S` is the read plus those other differences, and **those +//! terms are not ordered** -- so the difference constrains the read in neither +//! direction. Read these rows as an end-to-end comparison of two shapes in the +//! regime where the read is most expensive, and nothing finer. Found by a +//! review -- the second one to correct this sentence. +//! +//! # What is deliberately not claimed +//! +//! The drained regime has a **single** consumer, because that is what MPSC +//! means. At high producer counts it is therefore expected to become +//! consumer-bound, and a throughput plateau there says nothing about the tail +//! claim. The probe reports each run's refusal count -- from the queue's own +//! `Observable` counters -- so a backpressure-bound run is visible as a fact +//! rather than mistaken for contention. Read the isolated regime for push-path +//! scaling with producer count, and the drained one for the end-to-end shape +//! comparison taken while `head` is being written. + +use std::fmt; +use std::sync::Arc; +use std::sync::atomic::{AtomicBool, AtomicU64, Ordering}; +use std::sync::{Condvar, Mutex, MutexGuard, PoisonError}; +use std::thread; +use std::time::Instant; + +use windows_waitable_queues::{permit_mpsc, reserving_mpsc, slotwise_mpsc}; + +use windows_waitable_queues::reserving_mpsc::{Balanced, ClaimLayout, Enduring, Perpetual}; + +/// The 128-bit layout exists only where a 128-bit exchange is native. +/// +/// The condition is duplicated in this crate's `Cargo.toml`, which adds the +/// `dwcas` feature under the same `cfg`; see the comment there for why the +/// architectures are named rather than testing `target_has_atomic = "128"`, and +/// why enabling the feature unconditionally breaks the workspace's deliberately +/// supported `i686-pc-windows-msvc` build. Changing one without the other yields +/// either a missing type or an unused feature. +#[cfg(any( + all(target_arch = "x86_64", target_feature = "cmpxchg16b"), + target_arch = "aarch64" +))] +use windows_waitable_queues::reserving_mpsc::Wide; + +#[cfg(test)] +mod tests; + +/// How many pushes each producer thread performs in one timed run. +pub const PUSHES_PER_PRODUCER: usize = 50_000; + +/// How many times each configuration is repeated; the median is reported. +/// +/// Odd, so the median is an observed value rather than an average of two. Five +/// because these probes run on a virtual machine, where a single run can be +/// perturbed by something entirely outside the process. +pub const REPETITIONS: usize = 5; + +/// The producer counts measured, in order. +/// +/// Fixed rather than derived from the host's processor count, so two runs on +/// different machines produce comparable rows. The host's own count is reported +/// alongside, since the interesting region is around and beyond it. +pub const PRODUCER_COUNTS: &[usize] = &[1, 2, 4, 8, 16, 32]; + +/// The names a run is filed under. +/// +/// **Named once because a lookup by string literal is a rename waiting to +/// fail, and this one already did.** The `mpsc` -> `slotwise_mpsc` rename +/// updated the recording side and not the reporting binary, which went on +/// asking for `"mpsc"`; every lookup returned `None` and two entire columns of +/// the report rendered as `--` without anything erroring. A wrong shape name is +/// not a compile error, so the only defence is that both sides read the same +/// definition. +pub mod shapes { + /// The bounded-array MPSC. + pub const SLOTWISE_MPSC: &str = "slotwise_mpsc"; + /// The reservation-based MPSC. + pub const RESERVING_MPSC: &str = "reserving_mpsc"; + /// The experimental permit-claiming MPSC, measured against + /// [`RESERVING_MPSC`] because it is a candidate replacement for it. + pub const PERMIT_MPSC: &str = "permit_mpsc"; + /// The contended-atomic floor the queues are measured against. + /// + /// Contended, not uncontended: every producer thread increments the **same** + /// `AtomicU64`, which is the point -- it is the cheapest possible thing N + /// threads can do to one cache line, so it separates what the queue costs + /// from what this processor does to a fought-over line. + pub const BASELINE_FETCH_ADD: &str = "baseline_fetch_add"; + /// `reserving_mpsc` on its default layout: a `u64` split 32 / 32. + /// + /// The same configuration as [`RESERVING_MPSC`], run again under its own + /// name so the layout comparison reads without a reader having to know + /// which layout the default is. + pub const CLAIM_NARROW: &str = "reserving(32/32)"; + /// `reserving_mpsc` on `Enduring`: a `u64` split 16 / 48. + pub const CLAIM_DEEP: &str = "reserving(16/48)"; + /// `reserving_mpsc` on `Perpetual`: a `u64` split 8 / 56. + pub const CLAIM_PERPETUAL: &str = "reserving(8/56)"; + /// `reserving_mpsc` on `Wide`: a `u128` split 64 / 64. + pub const CLAIM_WIDE: &str = "reserving(64/64)"; +} +/// One configuration's result. +#[derive(Debug, Clone, Copy, PartialEq)] +pub struct Run { + /// Which queue shape, or the baseline. + pub shape: &'static str, + /// How many producer threads pushed concurrently. + pub producers: usize, + /// Median nanoseconds per successful operation, across all producers. + /// + /// An *operation* is one successful push for every queue shape. For + /// [`shapes::BASELINE_FETCH_ADD`] it is one `fetch_add` on a shared + /// `AtomicU64` -- that row is a floor rather than a queue, so it has no + /// pushes to report, and labelling this field per-push would publish it with + /// units it does not have. + pub nanos_per_op: f64, + /// Successful operations per second, summed across producers. See + /// [`Run::nanos_per_op`] for what counts as an operation in each row. + pub ops_per_second: f64, + /// Pushes refused for want of room during the median run. + /// + /// Non-zero means the run was at least partly bounded by the consumer + /// rather than by the claim, which is a fact about the measurement and not + /// about the queue. + pub refusals: u64, + /// Fastest of the [`REPETITIONS`] timed repetitions, in nanoseconds per + /// operation. + /// + /// Carried because [`d-observations-not-verdicts`] obliges every published + /// figure to arrive with its run count *and its dispersion*: a median alone + /// is an anecdote a reader cannot compare against their own hardware. The + /// four repetitions the median discards are the only evidence this probe has + /// about its own stability within a run, and discarding them silently was + /// the crate publishing a figure its own contract forbids. + /// + /// [`d-observations-not-verdicts`]: ../DESIGN-NOTES.md#d-observations-not-verdicts + pub fastest_nanos_per_op: f64, + /// Slowest of the [`REPETITIONS`] timed repetitions, in nanoseconds per + /// operation. See [`Run::fastest_nanos_per_op`]. + pub slowest_nanos_per_op: f64, +} + +impl Run { + /// Whether this row carries a measurement at all. + /// + /// **This is the single definition of the "did not run" sentinel, and every + /// renderer asks it rather than restating the test.** A shape that did not + /// run reports zero, and zero is not an obviously broken value in any of + /// this report's columns: `0.0` ns/op reads as immeasurably fast, `0.00x` + /// as a ratio of one, `0.0-0.0` as perfect stability. Each is the most + /// flattering cell its column can hold, produced by a row that measured + /// nothing. + /// + /// It lives here because the test was previously written inline in the + /// renderers that remembered it and simply absent from those that did not + /// -- which is how [`Run::spread`] came to render `0.00x` for a shape that + /// never ran. Fixing that one accessor left the same hole in four other + /// paths, because the sentinel was a convention rather than a definition. + /// A renderer can now only get this wrong by not asking. + /// + /// **Every field a renderer prints, not just the median.** An earlier + /// version tested `nanos_per_op` alone, so a row with a plausible median and + /// a poisoned `ops_per_second` or range endpoint answered `true` and + /// [`render_table`] then formatted those fields directly -- publishing `NaN` + /// or `inf` in a column of measurements, which is the failure the sentinel + /// exists to prevent. `refusals` is an integer and carries no such value. + #[must_use] + pub fn is_measured(&self) -> bool { + [ + self.nanos_per_op, + self.ops_per_second, + self.fastest_nanos_per_op, + self.slowest_nanos_per_op, + ] + .iter() + .all(|value| value.is_finite() && *value > 0.0) + } + + /// The spread across this configuration's repetitions, as a multiple. + /// + /// `1.00` would mean every repetition took the same time. A wide spread + /// says the figure beside it is one draw from a distribution this host does + /// not hold still, which is the reading the median alone hides. + /// + /// `None` when the fastest repetition took no measurable time, which cannot + /// happen for a real run and marks a shape that did not run at all. + /// + /// **This returns an `Option` rather than a sentinel, and that is the whole + /// point.** An earlier version returned `0.0` for the unmeasurable case, + /// reasoning that zero is not a plausible spread. It renders as `0.00x`, + /// which reads as *perfect stability* -- the most reassuring cell the column + /// can contain, produced by a row that measured nothing. Every neighbouring + /// accessor already returns `Option` for the same situation; this one was + /// the exception, and the exception is what a renderer got wrong. + #[must_use] + pub fn spread(&self) -> Option { + // `> 0.0` alone is not the test: `f64::INFINITY > 0.0` is true, and a + // finite slowest over an infinite fastest is `0.0` -- a spread of zero, + // which is the reassuring end of this column. Both endpoints must be + // real numbers before dividing them. + if self.fastest_nanos_per_op.is_finite() + && self.fastest_nanos_per_op > 0.0 + && self.slowest_nanos_per_op.is_finite() + // Positive too, not merely finite: a zero slowest endpoint divides + // cleanly to a spread of zero -- the most reassuring value this + // column can hold -- and a negative one yields a negative spread, + // which is not a dispersion at all. `is_measured` keeps both out of + // the report, but `spread` is public and answers callers directly. + && self.slowest_nanos_per_op > 0.0 + { + Some(self.slowest_nanos_per_op / self.fastest_nanos_per_op) + } else { + None + } + } +} + +/// Everything one invocation measured. +#[derive(Debug, Clone)] +pub struct Observation { + /// Producers timed with no consumer and no possibility of refusal. + pub isolated: Vec, + /// Producers timed against a consumer looping on `pop`. + /// + /// The handshake in `await_consumer` guarantees that loop has executed at + /// least once before any producer starts timing. It does not guarantee the + /// consumer is never descheduled afterwards, so "looping" describes what the + /// consumer thread runs, not how continuously it is scheduled to run it. + pub drained: Vec, + /// Processors available to **this process**, when it could be determined. + /// + /// This is `available_parallelism`, which is the process-available estimate + /// and not the host's logical-processor count: an affinity mask or a job + /// object narrows it, so under either it is legitimately smaller than the + /// banner's `16p`. It is reported because it is what decides whether a + /// producer count oversubscribes *this run*, which is the question a reader + /// of these rows actually has; the host's own shape is already on the banner. + /// + /// `None` when the query failed. An earlier version mapped failure to `0`, + /// which the report then printed as a zero-processor host -- a value no host + /// has, presented with the same confidence as a measured one. + pub available_parallelism: Option, +} + +impl Observation { + /// Look one run up. + #[must_use] + pub fn find(&self, regime: &[Run], shape: &str, producers: usize) -> Option { + regime + .iter() + .find(|run| run.shape == shape && run.producers == producers) + .copied() + } + + /// How many of `shapes` actually produced a measurement at `producers`. + /// + /// Counted rather than written as a literal because the 64/64 rows are + /// `cfg`-elided on a target with no native 128-bit exchange, so a hardcoded + /// count would be false there. + /// + /// **Presence is not measurement.** A row can be present and still carry the + /// did-not-run sentinel, and the report's prose is where that distinction + /// escapes: `render_table` marks such a row `--` in every measured cell while + /// a sentence above it counts the shape as measured. Asking + /// [`Run::is_measured`] here is what keeps the two halves of the report + /// telling the same story. + #[must_use] + pub fn count_measured(&self, regime: &[Run], shapes: &[&str], producers: usize) -> usize { + shapes + .iter() + .filter(|shape| { + self.find(regime, shape, producers) + .is_some_and(|run| run.is_measured()) + }) + .count() + } + + /// How far throughput scaled from one producer to `producers`. + /// + /// 1.0 means N producers together push no faster than one did, which is + /// what a badly contended claim looks like. Perfect scaling would be N, + /// which no shared-tail queue can reach. + #[must_use] + pub fn scaling(&self, regime: &[Run], shape: &str, producers: usize) -> Option { + let one = self.find(regime, shape, 1)?; + let many = self.find(regime, shape, producers)?; + Some(many.ops_per_second / one.ops_per_second) + } + + /// The interval [`Observation::scaling`] could occupy, given the two rows' + /// observed spans. + /// + /// **This is a bound, not a sampled distribution, and the difference + /// matters.** The probe measures each configuration in its own pass, so the + /// repetitions behind the numerator and the denominator are not paired: + /// there is no set of per-repetition ratios to take a median or a range + /// over. What can be said is that if one producer's cost lies in `[a, b]` + /// and N producers' in `[c, d]`, their ratio cannot fall outside + /// `[a / d, b / c]` -- so this is the widest the scaling could be, which is + /// conservative in the direction that matters. + /// + /// Carried because [`d-observations-not-verdicts`] obliges every published + /// figure to arrive with its dispersion, and a *derived* figure is exactly + /// where a bare point estimate is most likely to be over-read. Reporting + /// the ratio alone, while the rows beneath it carry ranges, would put the + /// least certain number on the page in the most confident dress. + /// + /// Pairing the repetitions would give a real distribution rather than a + /// bound, and that is a change to how the probe measures rather than to how + /// it reports -- see `M4.4`, which asks for candidates to be interleaved + /// with their controls. + /// + /// [`d-observations-not-verdicts`]: ../DESIGN-NOTES.md#d-observations-not-verdicts + #[must_use] + pub fn scaling_bounds( + &self, + regime: &[Run], + shape: &str, + producers: usize, + ) -> Option<(f64, f64)> { + let one = self.find(regime, shape, 1)?; + let many = self.find(regime, shape, producers)?; + // Asked before the identity case below, not after: a row that did not + // run is still a row, so `find` returns it and `producers == 1` would + // otherwise hand back an exact `(1.0, 1.0)` for a shape that measured + // nothing. The point estimate goes `NaN` in that case and the current + // renderer suppresses it, but this is a public method and a caller + // reading the bound on its own would see a certainty that is not there. + if !one.is_measured() || !many.is_measured() { + return None; + } + if producers == 1 { + // **The one-producer row is a row divided by itself.** Its scaling is + // exactly 1.0 by construction, not approximately 1.0 by measurement, + // so there is no interval to report. Handing both spans to + // `ratio_bounds` would treat one measurement as two independent ones + // and manufacture a bound like `[0.84-1.20]` around a quantity that + // cannot be anything but 1 -- uncertainty invented by the arithmetic + // rather than observed, printed in the report's first row. + // + // The bound stays sound either way, since it contains 1.0. It is + // tightness that is at stake, and at the identity it is exact. + return Some((1.0, 1.0)); + } + // Scaling is a RATE ratio -- many over one -- which is the COST ratio + // one over many, so the rows go in that order. + ratio_bounds(one, many) + } +} + +/// The interval the cost ratio `numerator / denominator` could occupy, given +/// each row's observed span. +/// +/// Both arguments are rows whose costs are nanoseconds per operation, and the +/// result is in those same terms: `0.20x` means the numerator cost a fifth of +/// what the denominator cost. A caller wanting a *rate* ratio -- "how much +/// faster" -- passes the two rows the other way round, which is what +/// [`Observation::scaling_bounds`] does. +/// +/// See [`Observation::scaling_bounds`] for why this is a bound rather than a +/// sample. +/// +/// **An earlier version of this function documented a cost ratio and returned +/// the inverse**, leaving both internal callers to compensate by swapping their +/// arguments. That works until somebody calls it directly, which is the defect a +/// review caught: a public contract that is only correct if you read the +/// implementation is not a contract. +/// +/// `None` when either span touches zero, which cannot happen for a real run and +/// is reported rather than divided by. +#[must_use] +pub fn ratio_bounds(numerator: Run, denominator: Run) -> Option<(f64, f64)> { + // **A `<= 0.0` test does not reject `NaN`.** IEEE comparison against NaN is + // false whichever way it is written, so every one of these four disjuncts is + // false for a NaN endpoint and the guard falls through to the division, + // which then renders `[NaN-NaN]`. Testing `is_finite()` first is what + // actually excludes it, and it excludes infinities in the same move. + let endpoints = [ + numerator.fastest_nanos_per_op, + numerator.slowest_nanos_per_op, + denominator.fastest_nanos_per_op, + denominator.slowest_nanos_per_op, + ]; + if endpoints + .iter() + .any(|endpoint| !endpoint.is_finite() || *endpoint <= 0.0) + { + return None; + } + // Widest is the numerator at its worst over the denominator at its best; + // narrowest is the reverse. + let low = numerator.fastest_nanos_per_op / denominator.slowest_nanos_per_op; + let high = numerator.slowest_nanos_per_op / denominator.fastest_nanos_per_op; + Some((low, high)) +} + +/// Renders a ratio together with the interval it could occupy, or `--`. +/// +/// The bound is printed in square brackets to mark it as *not* a sampled range: +/// the row ranges above it are observed spans, this is arithmetic over two of +/// them. +/// +/// **Both rows must have measured something.** A shape that failed to run +/// reports zero nanoseconds, and zero is the sentinel for "no measurement here" +/// on either side of the division -- a zero numerator would render `0.00x`, +/// which is a number a reader takes for a result rather than for the absence of +/// one. An earlier version guarded only the denominator, on the reasoning that +/// division is what breaks; publishing a plausible figure from a row that never +/// ran is the worse failure of the two. +#[must_use] +pub fn format_ratio_bounded(numerator: Option, denominator: Option) -> String { + match (numerator, denominator) { + (Some(numerator), Some(denominator)) + if numerator.is_measured() && denominator.is_measured() => + { + let point = numerator.nanos_per_op / denominator.nanos_per_op; + match ratio_bounds(numerator, denominator) { + Some((low, high)) => format!("{point:.2}x [{low:.2}-{high:.2}]"), + None => format!("{point:.2}x"), + } + } + _ => "--".to_owned(), + } +} + +/// The **minimum** width the report gives a bounded ratio column. +/// +/// Named here, beside the formatter, rather than written as a literal in the +/// report's format string. [`format_ratio_bounded`] emits a point estimate *and* +/// its interval -- `1.00x [1.00-1.00]` is 17 characters, not the 5 a bare +/// `1.00x` would take -- and a Rust width is a minimum rather than a maximum, so +/// a field narrower than the value does not truncate it, it pushes every later +/// column out of line with its header. The report had been allocating 10. +/// +/// **This is a floor, not a bound, because the formatter has no bound.** The +/// interval's endpoints come from measured spans, so a slow outlier -- an +/// ordinary event on a loaded or virtualized host, and the reason `median_run` +/// takes a median at all -- widens the cell without limit: a 300 ms repetition +/// against a 4 ns one renders `10.00x [6.67-1500.00]`, which is 21. Use +/// [`ratio_column_width`] to size the column against the values it must actually +/// hold; this constant only stops a table of narrow values from looking cramped. +pub const RATIO_COLUMN_WIDTH: usize = 20; + +/// The width a ratio column must take to keep its rows aligned with its header. +/// +/// The widest cell the column has to hold, or [`RATIO_COLUMN_WIDTH`] when that +/// is wider. Derived from the rendered cells rather than assumed, because +/// [`format_ratio_bounded`]'s output length is a function of measured data and +/// therefore has no compile-time bound -- see [`RATIO_COLUMN_WIDTH`] for the +/// case that overruns it. A caller must render every cell of the column before +/// emitting the header, which is the only ordering that can get this right. +#[must_use] +pub fn ratio_column_width<'a>(cells: impl IntoIterator) -> usize { + column_width(cells, RATIO_COLUMN_WIDTH) +} + +/// The width a column must take to keep its rows aligned with its header. +/// +/// The widest cell, or `floor` when that is wider. Every column in this report +/// holds rendered measurements, whose length is a function of the data rather +/// than a constant, so a fixed field silently shifts everything to its right the +/// first time a value outgrows it. `floor` only stops a table of narrow values +/// from looking cramped; it is never an upper bound. +#[must_use] +pub fn column_width<'a>(cells: impl IntoIterator, floor: usize) -> usize { + cells + .into_iter() + .map(str::len) + .max() + .map_or(floor, |widest| widest.max(floor)) +} + +/// Renders a scaling factor together with the interval it could occupy. +/// +/// See [`Observation::scaling_bounds`]: the bracketed interval is a bound over +/// two unpaired spans, not a distribution. +#[must_use] +pub fn format_scaling_bounded(point: Option, bounds: Option<(f64, f64)>) -> String { + match (point, bounds) { + // A scaling of zero is the sentinel, not a measurement: it means the + // many-producer row reported no throughput at all. Guarded here as well + // as against non-finite values, because zero is the half that renders + // plausibly -- `0.00x` looks like a contended queue, `infx` does not. + (Some(point), _) if !point.is_finite() || point <= 0.0 => "--".to_owned(), + (Some(point), Some((low, high))) if low.is_finite() && high.is_finite() => { + format!("{point:.2}x [{low:.2}-{high:.2}]") + } + (Some(point), _) => format!("{point:.2}x"), + (None, _) => "--".to_owned(), + } +} + +/// Renders one regime's rows as the report's table body. +/// +/// Here rather than in the binary so it can be tested without running the +/// measurement. It takes a sink rather than writing to stdout for the reason +/// [`crate::report`] records: a helper writing to stdout while its caller +/// composes a string emits its lines first, reordering the report without losing +/// any of it. +pub fn render_table(out: &mut dyn fmt::Write, runs: &[Run]) { + const HEADERS: [&str; 7] = [ + "shape", + "producers", + "ns/op", + "ops/sec", + "refusals", + "ns/op range", + "spread", + ]; + // The widths this table has always used, kept as floors so an ordinary + // report is unchanged. Every column is a measurement rendered at its natural + // width, and `spread` and `ns/op range` are quotients and pairs of measured + // endpoints with no upper bound -- a long pause, the very outlier + // `median_run` exists to tolerate, renders wider than any fixed field and + // pushes every column after it out of line. See `ratio_column_width`, which + // is the same argument for the ratio tables. + const FLOOR: [usize; 7] = [18, 10, 14, 16, 14, 18, 9]; + + let mut rows: Vec<[String; 7]> = Vec::with_capacity(runs.len()); + for run in runs { + // `shape` and `producers` are configuration and always mean something. + // Every other column is a measurement, so a row that did not run has + // nothing to put in any of them -- including `refusals`, whose zero + // would otherwise read as "nothing was refused" rather than "nothing + // was attempted". See `Run::is_measured`. + let (nanos, ops, refusals, range, spread) = if run.is_measured() { + ( + format!("{:.1}", run.nanos_per_op), + format!("{:.0}", run.ops_per_second), + run.refusals.to_string(), + format!( + "{:.1}-{:.1}", + run.fastest_nanos_per_op, run.slowest_nanos_per_op + ), + format_scaling(run.spread()), + ) + } else { + ( + "--".to_owned(), + "--".to_owned(), + "--".to_owned(), + "--".to_owned(), + "--".to_owned(), + ) + }; + rows.push([ + run.shape.to_owned(), + run.producers.to_string(), + nanos, + ops, + refusals, + range, + spread, + ]); + } + + let mut width = FLOOR; + for row in &rows { + for (column, cell) in row.iter().enumerate() { + width[column] = width[column].max(cell.len()); + } + } + + let _ = writeln!( + out, + "{:b$} {:>c$} {:>d$} {:>e$} {:>f$} {:>g$}", + HEADERS[0], + HEADERS[1], + HEADERS[2], + HEADERS[3], + HEADERS[4], + HEADERS[5], + HEADERS[6], + a = width[0], + b = width[1], + c = width[2], + d = width[3], + e = width[4], + f = width[5], + g = width[6], + ); + for row in &rows { + let _ = writeln!( + out, + "{:b$} {:>c$} {:>d$} {:>e$} {:>f$} {:>g$}", + row[0], + row[1], + row[2], + row[3], + row[4], + row[5], + row[6], + a = width[0], + b = width[1], + c = width[2], + d = width[3], + e = width[4], + f = width[5], + g = width[6], + ); + } +} + +/// A scaling factor, or `--` when it is missing or not a number. +/// +/// Guards non-finite values for the same reason [`format_ratio`] guards its +/// denominator, and the guard belongs here rather than in [`Observation::scaling`]: +/// a shape whose one-producer row reports zero makes the quotient infinite, and +/// `infx` in a column of measurements reads as a measurement. `scaling` is +/// deliberately allowed to return the non-finite value -- it is arithmetic, not +/// a renderer -- so the display layer is where it has to be caught. +#[must_use] +pub fn format_scaling(scaling: Option) -> String { + match scaling { + Some(value) if value.is_finite() && value > 0.0 => format!("{value:.2}x"), + _ => "--".to_owned(), + } +} + +/// `numerator / denominator` as a cost ratio, or `--` when either is missing. +/// +/// Guards both rows rather than trusting them: a shape that failed to run +/// reports zero, so a zero denominator would print `inf` or `NaN` and a zero +/// numerator would print `0.00x` -- and of those two the second is the more +/// dangerous, because it looks like a measurement rather than like a failure. +#[must_use] +pub fn format_ratio(numerator: Option, denominator: Option) -> String { + match (numerator, denominator) { + (Some(numerator), Some(denominator)) + if numerator.is_measured() && denominator.is_measured() => + { + format!("{:.2}x", numerator.nanos_per_op / denominator.nanos_per_op) + } + _ => "--".to_owned(), + } +} + +/// One row's nanoseconds per operation, or `--` when the row is missing or did +/// not run. +/// +/// Guards the sentinel for the reason [`Run::is_measured`] records: this column +/// is the report's most-read number, and `0.0` in it reads as a shape too fast +/// to time rather than as one that never ran. +#[must_use] +pub fn format_nanos(run: Option) -> String { + match run { + Some(run) if run.is_measured() => format!("{:.1}", run.nanos_per_op), + _ => "--".to_owned(), + } +} + +/// Time every configuration. +#[must_use] +pub fn measure() -> Observation { + let mut isolated = Vec::new(); + let mut drained = Vec::new(); + + for &producers in PRODUCER_COUNTS { + isolated.push(median_run(shapes::BASELINE_FETCH_ADD, producers, |count| { + time_contended_atomic(count) + })); + isolated.push(median_run(shapes::SLOTWISE_MPSC, producers, |count| { + time_isolated_mpsc(count) + })); + isolated.push(median_run(shapes::RESERVING_MPSC, producers, |count| { + time_isolated_reserving(count) + })); + isolated.push(median_run(shapes::PERMIT_MPSC, producers, |count| { + time_isolated_permit(count) + })); + + drained.push(median_run(shapes::SLOTWISE_MPSC, producers, |count| { + time_drained_mpsc(count) + })); + drained.push(median_run(shapes::RESERVING_MPSC, producers, |count| { + time_drained_reserving(count) + })); + drained.push(median_run(shapes::PERMIT_MPSC, producers, |count| { + time_drained_permit(count) + })); + + isolated.push(median_run(shapes::CLAIM_NARROW, producers, |count| { + time_isolated_layout::(count) + })); + isolated.push(median_run(shapes::CLAIM_DEEP, producers, |count| { + time_isolated_layout::(count) + })); + isolated.push(median_run(shapes::CLAIM_PERPETUAL, producers, |count| { + time_isolated_layout::(count) + })); + // Gated on the architectures where a 128-bit exchange is native; see the + // `Wide` import above. `#[cfg]` governs only the statement that follows + // it, so each of the two pushes carries its own. + #[cfg(any( + all(target_arch = "x86_64", target_feature = "cmpxchg16b"), + target_arch = "aarch64" + ))] + isolated.push(median_run(shapes::CLAIM_WIDE, producers, |count| { + time_isolated_layout::(count) + })); + + drained.push(median_run(shapes::CLAIM_NARROW, producers, |count| { + time_drained_layout::(count) + })); + drained.push(median_run(shapes::CLAIM_DEEP, producers, |count| { + time_drained_layout::(count) + })); + drained.push(median_run(shapes::CLAIM_PERPETUAL, producers, |count| { + time_drained_layout::(count) + })); + #[cfg(any( + all(target_arch = "x86_64", target_feature = "cmpxchg16b"), + target_arch = "aarch64" + ))] + drained.push(median_run(shapes::CLAIM_WIDE, producers, |count| { + time_drained_layout::(count) + })); + } + + Observation { + isolated, + drained, + available_parallelism: thread::available_parallelism() + .ok() + .map(std::num::NonZeroUsize::get), + } +} + +/// Raw result of one timed repetition: elapsed nanoseconds and refusals. +type Repetition = (f64, u64); + +/// Run one configuration [`REPETITIONS`] times and keep the median. +/// +/// The median rather than the mean, because on a virtual machine the failure +/// mode is one run being hugely slower rather than a spread around a centre, +/// and a mean would carry that outlier into the reported number. +fn median_run( + shape: &'static str, + producers: usize, + mut timer: impl FnMut(usize) -> Repetition, +) -> Run { + // One untimed pass first. Be exact about what this does and does not warm: + // for the queue timers, every call to `timer` builds and drops its OWN + // queue, so this does not pre-touch the allocation any timed repetition will + // use. The baseline timer allocates no queue at all -- one `AtomicU64` and a + // barrier -- so for that row there is no allocation to pre-touch either way. + // What the pass warms in both cases is the process: the allocator's size + // class, the OS page cache, the instruction cache, and the branch + // predictors, which is why the first timed repetition is no longer an + // outlier. An earlier comment here claimed it faulted in "the" allocation, + // which is not true of an allocation made fresh each pass, and a later one + // said every timer builds a queue, which is not true of the baseline. + // Both found by review. + let _ = timer(producers); + + let mut results: Vec = (0..REPETITIONS).map(|_| timer(producers)).collect(); + results.sort_by(|left, right| left.0.total_cmp(&right.0)); + let (elapsed_nanos, refusals) = results[REPETITIONS / 2]; + // The sort is ascending by elapsed time, so the extremes are the ends. They + // are carried rather than discarded because a median without its dispersion + // is what `d-observations-not-verdicts` forbids publishing. + let (fastest_nanos, _) = results[0]; + let (slowest_nanos, _) = results[REPETITIONS - 1]; + + let pushes = (producers * PUSHES_PER_PRODUCER) as f64; + Run { + shape, + producers, + nanos_per_op: elapsed_nanos / pushes, + ops_per_second: pushes / (elapsed_nanos / 1e9), + refusals, + fastest_nanos_per_op: fastest_nanos / pushes, + slowest_nanos_per_op: slowest_nanos / pushes, + } +} + +/// The floor: N threads incrementing one shared counter. +/// +/// Not a queue, and not trying to be. It is the cheapest possible operation on +/// a contended line, so it says how much of a queue's scaling curve is the +/// queue and how much is simply what this processor does when N cores fight +/// over one cache line. +fn time_contended_atomic(producers: usize) -> Repetition { + let counter = Arc::new(AtomicU64::new(0)); + // One party per worker plus this thread. Every worker is created, then waits + // here; the clock starts as the barrier releases, so neither thread creation + // nor a solo head start by an early worker is inside the measurement. See + // `start_barrier`'s note for why that matters at these producer counts. + let gate = start_gate(producers); + let spans = thread::scope(|scope| { + let _release = ReleaseOnDrop(Arc::clone(&gate)); + let mut workers = Vec::with_capacity(producers); + for _ in 0..producers { + let counter = Arc::clone(&counter); + let gate = Arc::clone(&gate); + workers.push(scope.spawn(move || { + if !gate.arrive_and_wait() { + // Abandoned before the party completed; see `StartGate`. + return (Instant::now(), Instant::now()); + } + let began = Instant::now(); + for _ in 0..PUSHES_PER_PRODUCER { + counter.fetch_add(1, Ordering::Relaxed); + } + (began, Instant::now()) + })); + } + let _ = gate.arrive_and_wait(); + workers + .into_iter() + .map(|worker| worker.join().expect("a producer must not panic")) + .collect::>() + }); + (measured_span(&spans), 0) +} + +/// Capacity big enough that a whole run fits, so nothing is ever refused. +fn capacity_for(producers: usize) -> usize { + (producers * PUSHES_PER_PRODUCER).next_power_of_two() +} + +/// A gate holding every participant until all of them exist, or until the +/// coordinator gives up on the ones that do not. +/// +/// **Without this the row labelled N producers need not have measured N of +/// them.** Spawning is not instant, and each worker used to start pushing the +/// moment it was created, so at 50,000 pushes an early producer could complete +/// a long uncontended prefix -- or finish entirely -- before the last thread was +/// spawned. The reported interval also began before any worker existed, folding +/// thread-creation cost into a per-push number. The curve against N is the whole +/// output of this probe, and both effects bend it downward exactly where it is +/// steepest. +/// +/// The count includes the coordinating thread, so no worker can start before the +/// last one exists. It does NOT start the clock -- see [`measured_span`] for why +/// that is a separate job. +/// +/// **Why this is not `std::sync::Barrier`.** A `Barrier`'s party count, once +/// set, must be met: there is no way to say "nobody else is coming". The timers +/// size the gate for every planned worker and then spawn them with +/// `Scope::spawn`, which *panics* if the OS refuses a thread. If that happened +/// after an earlier worker had already parked, the coordinator never reached its +/// own arrival, the count was never met, and `thread::scope` joined a +/// permanently parked worker while unwinding -- so the probe **hung rather than +/// failed**, which is the worse of the two. [`StartGate::release`] is the +/// missing operation, and [`ReleaseOnDrop`] performs it on the unwind path. +/// +/// **The property the measurement depends on is preserved.** Every parked party +/// is woken by one `notify_all` and returns as a group, exactly as +/// `Barrier::wait` does -- which is what [`measured_span`] relies on when it +/// argues that this thread cannot time the workers and each must time itself. +struct StartGate { + state: Mutex, + opened: Condvar, +} + +struct GateState { + /// Participants still to arrive. Reaching zero opens the gate. + remaining: usize, + /// Whether waiters may proceed, for either reason. + open: bool, + /// Whether the gate opened because everyone arrived, rather than because + /// the coordinator released it. A participant that reads `false` learns its + /// run was abandoned and should not do the work. + complete: bool, +} + +impl StartGate { + /// `participants` workers plus the coordinating thread. + fn new(participants: usize) -> Arc { + Arc::new(Self { + state: Mutex::new(GateState { + remaining: participants + 1, + open: false, + complete: false, + }), + opened: Condvar::new(), + }) + } + + /// Poisoning is stepped over rather than propagated. + /// + /// Nothing but the gate's own bookkeeping runs under this lock, so a + /// poisoned mutex means some *other* thread panicked while parked here. The + /// whole point of this type is to unblock that situation; panicking on the + /// way -- from inside a `Drop` that is already unwinding -- would abort the + /// process instead. + fn locked(&self) -> MutexGuard<'_, GateState> { + self.state.lock().unwrap_or_else(PoisonError::into_inner) + } + + /// Arrive, then block until every participant has, or until the gate is + /// released. + /// + /// `true` when the gate opened because the party was complete, which is the + /// only case in which a run's timings mean anything. `false` says the + /// coordinator gave up; the caller should return without doing the work. + #[must_use] + fn arrive_and_wait(&self) -> bool { + let mut state = self.locked(); + state.remaining = state.remaining.saturating_sub(1); + if state.remaining == 0 && !state.open { + state.open = true; + state.complete = true; + } + if state.open { + let complete = state.complete; + drop(state); + self.opened.notify_all(); + return complete; + } + while !state.open { + state = self + .opened + .wait(state) + .unwrap_or_else(PoisonError::into_inner); + } + state.complete + } + + /// Open the gate now, however many participants are missing. + /// + /// A no-op once the gate is open, so the guard that calls this on the + /// ordinary path costs nothing. + fn release(&self) { + let mut state = self.locked(); + state.open = true; + drop(state); + self.opened.notify_all(); + } +} + +/// Releases the start gate however the spawning phase ends. +/// +/// The failure this exists for is a `Scope::spawn` panic partway through +/// creating the workers: without it, the parties already parked wait for a count +/// that will never be met, and the join that `thread::scope` performs while +/// unwinding never returns. Held inside the scope's closure, so it drops while +/// that closure unwinds -- before the join loop it needs to unblock. +struct ReleaseOnDrop(Arc); + +impl Drop for ReleaseOnDrop { + fn drop(&mut self) { + self.0.release(); + } +} + +fn start_gate(participants: usize) -> Arc { + StartGate::new(participants) +} + +/// The wall-clock window the producers were actually inside: from the first to +/// begin to the last to finish. +/// +/// **Each worker times itself, because this thread cannot time them.** The +/// obvious arrangement -- release the barrier, call `Instant::now()` here, and +/// read `elapsed()` after the scope ends -- is wrong at both ends, and a review +/// caught it: +/// +/// - `Barrier::wait` releases every party together, and this thread is just +/// another party. A worker can return from `wait` and run an arbitrary prefix +/// of its pushes before this thread is scheduled again to read the clock, so +/// the start could land after work had already happened. That understates the +/// interval, which OVERSTATES throughput. +/// - `thread::scope` joins every worker before it returns, so an `elapsed()` +/// read after it includes thread exit and join. That overstates the interval, +/// which understates throughput. +/// +/// Neither error is bounded by anything this probe controls, and both bite +/// hardest on the fast low-producer rows where a run is only hundreds of +/// microseconds. Taking the earliest start and the latest finish measures the +/// span the producers were contending over and nothing else. +fn measured_span(spans: &[(Instant, Instant)]) -> f64 { + let began = spans + .iter() + .map(|(began, _)| *began) + .min() + .expect("a run has at least one producer"); + let ended = spans + .iter() + .map(|(_, ended)| *ended) + .max() + .expect("a run has at least one producer"); + + ended.duration_since(began).as_nanos() as f64 +} + +fn time_isolated_mpsc(producers: usize) -> Repetition { + let (tx, rx) = + slotwise_mpsc::bounded::(capacity_for(producers)).expect("a valid capacity"); + let gate = start_gate(producers); + let spans = thread::scope(|scope| { + let _release = ReleaseOnDrop(Arc::clone(&gate)); + let mut workers = Vec::with_capacity(producers); + for producer in 0..producers { + let tx = tx.clone(); + let gate = Arc::clone(&gate); + workers.push(scope.spawn(move || { + if !gate.arrive_and_wait() { + // Abandoned before the party completed; see `StartGate`. + return (Instant::now(), Instant::now()); + } + let began = Instant::now(); + for index in 0..PUSHES_PER_PRODUCER { + tx.push((producer * PUSHES_PER_PRODUCER + index) as u64) + .expect("the run fits in the capacity"); + } + (began, Instant::now()) + })); + } + let _ = gate.arrive_and_wait(); + workers + .into_iter() + .map(|worker| worker.join().expect("a producer must not panic")) + .collect::>() + }); + let elapsed = measured_span(&spans); + let refusals = tx.refused(); + // Drain before dropping: teardown would otherwise walk every slot, and that + // is not part of what is being timed. + while rx.pop().is_ok() {} + (elapsed, refusals) +} + +fn time_isolated_reserving(producers: usize) -> Repetition { + let (tx, rx) = + reserving_mpsc::bounded::(capacity_for(producers)).expect("a valid capacity"); + let gate = start_gate(producers); + let spans = thread::scope(|scope| { + let _release = ReleaseOnDrop(Arc::clone(&gate)); + let mut workers = Vec::with_capacity(producers); + for producer in 0..producers { + let tx = tx.clone(); + let gate = Arc::clone(&gate); + workers.push(scope.spawn(move || { + if !gate.arrive_and_wait() { + // Abandoned before the party completed; see `StartGate`. + return (Instant::now(), Instant::now()); + } + let began = Instant::now(); + for index in 0..PUSHES_PER_PRODUCER { + tx.push((producer * PUSHES_PER_PRODUCER + index) as u64) + .expect("the run fits in the capacity"); + } + (began, Instant::now()) + })); + } + let _ = gate.arrive_and_wait(); + workers + .into_iter() + .map(|worker| worker.join().expect("a producer must not panic")) + .collect::>() + }); + let elapsed = measured_span(&spans); + let refusals = tx.refused(); + while rx.pop().is_ok() {} + (elapsed, refusals) +} + +/// The experimental permit claim, with no consumer and no possibility of refusal. +/// +/// Not "the regime that isolates the claim", which an earlier wording said: this +/// times the whole push path, including slot metadata, the item write, +/// publication and the doorbell's fence. See the module header. +/// +/// A line-for-line twin of [`time_isolated_reserving`] with one shape +/// substituted. Deliberately not factored into a generic over the two, which +/// would need a trait both implement and would put a dynamic or monomorphised +/// indirection inside the timed region -- in a measurement whose whole output is +/// a difference of a few nanoseconds per push. +fn time_isolated_permit(producers: usize) -> Repetition { + let (tx, rx) = permit_mpsc::bounded::(capacity_for(producers)).expect("a valid capacity"); + let gate = start_gate(producers); + let spans = thread::scope(|scope| { + let _release = ReleaseOnDrop(Arc::clone(&gate)); + let mut workers = Vec::with_capacity(producers); + for producer in 0..producers { + let tx = tx.clone(); + let gate = Arc::clone(&gate); + workers.push(scope.spawn(move || { + if !gate.arrive_and_wait() { + // Abandoned before the party completed; see `StartGate`. + return (Instant::now(), Instant::now()); + } + let began = Instant::now(); + for index in 0..PUSHES_PER_PRODUCER { + tx.push((producer * PUSHES_PER_PRODUCER + index) as u64) + .expect("the run fits in the capacity"); + } + (began, Instant::now()) + })); + } + let _ = gate.arrive_and_wait(); + workers + .into_iter() + .map(|worker| worker.join().expect("a producer must not panic")) + .collect::>() + }); + let elapsed = measured_span(&spans); + let refusals = tx.refused(); + while rx.pop().is_ok() {} + (elapsed, refusals) +} + +/// A capacity a real system would choose, so the drained regime exercises +/// backpressure the way a real one would. +pub const DRAINED_CAPACITY: usize = 1024; + +/// Stops a drained timer's consumer however the producer phase ends. +/// +/// Each drained timer parks a consumer in `while !done { ... }` and clears the +/// flag once the producer scope returns. On the **failure** path that line is +/// never reached: a producer's assertion unwinds straight past it, so the +/// consumer keeps spinning on a flag nobody will ever set, and its `JoinHandle` +/// is dropped without a join. The probe then leaves a thread burning a core for +/// the life of the process -- in exactly the run someone is trying to read a +/// failure out of. +/// +/// Clearing the flag from `Drop` runs on both paths, so the consumer observes +/// the stop and finishes whether the producers succeeded or panicked. The join +/// is still done explicitly on the success path, where its return value is the +/// refusal count; on the unwind path the thread is detached, but it terminates +/// promptly rather than spinning, which is the part that mattered. +struct StopOnDrop(Arc); + +impl Drop for StopOnDrop { + fn drop(&mut self) { + self.0.store(true, Ordering::Relaxed); + } +} + +/// Holds a producer until the consumer has drained at least once. +/// +/// **The gate is not enough, and the difference is the regime being measured.** +/// Arriving at the gate proves the consumer exists, is scheduled and is past +/// thread start-up; it does not prove the consumer has reached its first `pop`. +/// The gate releases every party together, so a producer could push into a queue +/// nobody was taking from yet -- an undrained opening to a run whose whole point +/// is that it is drained. The window was bounded by a scheduling quantum rather +/// than by thread creation, which is why the gate was still worth having, but it +/// was not zero. +/// +/// **What this guarantees, stated exactly.** No producer begins timing until the +/// consumer has executed its pop path at least once. It does *not* guarantee the +/// consumer is draining continuously from then on -- nothing a flag can express +/// would, since the consumer can be descheduled at any point afterwards, as it +/// can at any point during the run. What it removes is the case where producers +/// push into a queue whose consumer has not yet run at all. +/// +/// `Acquire`/`Release` rather than `Relaxed`, though the flag carries no data: +/// this is the standing "promote the load" answer recorded in the queue crate's +/// [D-38](../../windows-waitable-queues/DESIGN-NOTES.md#d-38) -- an acquire that +/// proves unnecessary costs little, while a relaxed load that turns out to have +/// been load-bearing fails only on hardware nobody here owns. +/// +/// **This changes what the drained rows measure**, which is why it is `M4.3` and +/// why the figures taken before it are kept beside the ones taken after rather +/// than replaced: they are measurements of two different pieces of code, and +/// both are real. +fn await_consumer(ready: &AtomicBool) { + while !ready.load(Ordering::Acquire) { + std::hint::spin_loop(); + } +} + +/// Drains once, then announces -- the producing half of the `M4.3` handshake. +/// +/// **The one statement of the ordering.** Four drained timers need it, and a +/// hand-written `pop` followed by a `store` in each of them is four chances for +/// the two lines to end up the other way round, which no test could see: the +/// timers cannot run without running the whole probe. Defining it here gives the +/// ordering a single home that the `the_handshake_drains_before_it_announces` +/// test can drive with a recording fake. +/// +/// Swapping these two statements reintroduces the undrained opening in its +/// narrower form -- the announcement would mean "this consumer is about to +/// drain", which a descheduling can falsify, rather than "this consumer has +/// executed the pop path", which nothing can. +/// +/// `Release` pairs with the `Acquire` in [`await_consumer`], so a producer that +/// observes the flag has the pop ordered before it. +fn drain_then_announce(pop_once: impl FnOnce(), ready: &AtomicBool) { + pop_once(); + ready.store(true, Ordering::Release); +} + +fn time_drained_mpsc(producers: usize) -> Repetition { + let (tx, rx) = slotwise_mpsc::bounded::(DRAINED_CAPACITY).expect("a valid capacity"); + let done = Arc::new(AtomicBool::new(false)); + let consumer_done = Arc::clone(&done); + // Set on every exit path, not just the one that returns. See StopOnDrop. + let stop = StopOnDrop(done); + // Closes the undrained opening the gate alone leaves. See await_consumer. + let ready = Arc::new(AtomicBool::new(false)); + let consumer_ready = Arc::clone(&ready); + // The consumer is a gate participant, not merely spawned: spawning is not + // readiness, and a consumer still in thread start-up while producers push + // turns the opening of the run into an undrained regime. + // + // The gate alone does not finish the job, which is why `await_consumer` + // exists. Arriving proves the consumer exists, is scheduled and is past + // start-up; it does not prove the consumer has reached its first `pop`, and + // the gate releases every party together. The handshake narrows that + // remainder to a stated guarantee: no producer begins timing until the + // consumer has executed its pop path at least once. Continuous draining is + // not guaranteed and cannot be by a flag. This is the M4.3 change, and it + // MOVED the drained numbers -- the figures taken before it are kept beside + // the ones taken after rather than replaced, because they measure two + // different pieces of code. + let gate = start_gate(producers + 1); + let consumer_gate = Arc::clone(&gate); + + let consumer = thread::spawn(move || { + if !consumer_gate.arrive_and_wait() { + return rx.refused(); + } + // **One drain attempt BEFORE announcing, not merely reaching the loop.** + // Publishing first proves only that the consumer is about to drain: it + // can be descheduled between the store and its first `pop`, which is the + // same undrained opening in a narrower form. Popping first makes the + // announcement mean `this consumer has executed the pop path`, which is a + // fact rather than an intention. The queue is empty here, so it costs one + // failed pop, and it happens before any producer has started timing. + drain_then_announce( + || { + let _ = rx.pop(); + }, + &consumer_ready, + ); + // Spin rather than park: the doorbell's cost is `doorbell_cost`'s + // question, and parking here would measure that instead of the claim. + while !consumer_done.load(Ordering::Relaxed) { + while rx.pop().is_ok() {} + std::hint::spin_loop(); + } + while rx.pop().is_ok() {} + rx.refused() + }); + + let spans = thread::scope(|scope| { + let _release = ReleaseOnDrop(Arc::clone(&gate)); + let mut workers = Vec::with_capacity(producers); + for producer in 0..producers { + let tx = tx.clone(); + let gate = Arc::clone(&gate); + let ready = Arc::clone(&ready); + workers.push(scope.spawn(move || { + if !gate.arrive_and_wait() { + // Abandoned before the party completed; see `StartGate`. + return (Instant::now(), Instant::now()); + } + await_consumer(&ready); + let began = Instant::now(); + for index in 0..PUSHES_PER_PRODUCER { + let mut item = (producer * PUSHES_PER_PRODUCER + index) as u64; + // Retry a FULL queue, which is what a real producer does; + // the refusal count is what makes that visible. Anything + // else is not retryable -- a disconnected queue never + // drains -- and retrying it is an infinite spin that + // presents as a hung probe rather than as the consumer + // failure it actually is. The queue crate says so itself: + // "retrying the first is sensible and retrying the second + // is a spin". + while let Err(error) = tx.push(item) { + assert!( + error.is_retryable(), + "the consumer is gone, so this push can never \ + succeed: {error}" + ); + item = error.into_inner(); + std::hint::spin_loop(); + } + } + (began, Instant::now()) + })); + } + let _ = gate.arrive_and_wait(); + workers + .into_iter() + .map(|worker| worker.join().expect("a producer must not panic")) + .collect::>() + }); + let elapsed = measured_span(&spans); + + drop(stop); + drop(tx); + let refusals = consumer.join().expect("the consumer must not panic"); + (elapsed, refusals) +} + +fn time_drained_reserving(producers: usize) -> Repetition { + // **Defaults on both sides, and that is a correction.** This row previously + // enabled high-water tracking here and nowhere else, to "also price the + // switch M31.4 made opt-in". But the number it feeds is presented as the + // cost of *reservation*, and tracking adds work to this shape's push path + // alone, so the ratio measured reservation plus a handicap with no way for a + // reader to separate them. + // + // **What the handicap actually is, corrected:** an earlier version of this + // comment said tracking adds a load of the consumer's position. It does not. + // `reserving_mpsc::publish` loads `head` **unconditionally** -- the slot + // write needs that acquire edge whether or not anything is measured, as the + // comment at that load says in as many words. What the switch adds is the + // depth arithmetic and the metric update on the far side of a branch that is + // taken either way. Smaller than claimed, and still not part of what this row + // is presented as measuring. + // + // Nothing consumes the high-water figure here either, so the tracking was + // paying a cost to produce a number nobody read. Pricing that switch is a + // worthwhile measurement and needs its own row, with both shapes tracking, + // rather than being folded into this comparison. + let (tx, rx) = reserving_mpsc::bounded::(DRAINED_CAPACITY).expect("a valid capacity"); + let done = Arc::new(AtomicBool::new(false)); + let consumer_done = Arc::clone(&done); + // Set on every exit path, not just the one that returns. See StopOnDrop. + let stop = StopOnDrop(done); + // Closes the undrained opening the gate alone leaves. See await_consumer. + let ready = Arc::new(AtomicBool::new(false)); + let consumer_ready = Arc::clone(&ready); + // The consumer joins the gate here for the reason it does in the slotwise + // twin: a run whose opening is undrained is not the regime being measured. + let gate = start_gate(producers + 1); + let consumer_gate = Arc::clone(&gate); + + let consumer = thread::spawn(move || { + if !consumer_gate.arrive_and_wait() { + return rx.refused(); + } + // **One drain attempt BEFORE announcing, not merely reaching the loop.** + // Publishing first proves only that the consumer is about to drain: it + // can be descheduled between the store and its first `pop`, which is the + // same undrained opening in a narrower form. Popping first makes the + // announcement mean `this consumer has executed the pop path`, which is a + // fact rather than an intention. The queue is empty here, so it costs one + // failed pop, and it happens before any producer has started timing. + drain_then_announce( + || { + let _ = rx.pop(); + }, + &consumer_ready, + ); + while !consumer_done.load(Ordering::Relaxed) { + while rx.pop().is_ok() {} + std::hint::spin_loop(); + } + while rx.pop().is_ok() {} + rx.refused() + }); + + let spans = thread::scope(|scope| { + let _release = ReleaseOnDrop(Arc::clone(&gate)); + let mut workers = Vec::with_capacity(producers); + for producer in 0..producers { + let tx = tx.clone(); + let gate = Arc::clone(&gate); + let ready = Arc::clone(&ready); + workers.push(scope.spawn(move || { + if !gate.arrive_and_wait() { + // Abandoned before the party completed; see `StartGate`. + return (Instant::now(), Instant::now()); + } + await_consumer(&ready); + let began = Instant::now(); + for index in 0..PUSHES_PER_PRODUCER { + let mut item = (producer * PUSHES_PER_PRODUCER + index) as u64; + while let Err(error) = tx.push(item) { + // Only a FULL queue is retryable; see the note on the + // first of these loops. + assert!( + error.is_retryable(), + "the consumer is gone, so this push can never \ + succeed: {error}" + ); + item = error.into_inner(); + std::hint::spin_loop(); + } + } + (began, Instant::now()) + })); + } + let _ = gate.arrive_and_wait(); + workers + .into_iter() + .map(|worker| worker.join().expect("a producer must not panic")) + .collect::>() + }); + let elapsed = measured_span(&spans); + + drop(stop); + drop(tx); + let refusals = consumer.join().expect("the consumer must not panic"); + (elapsed, refusals) +} + +/// The experimental permit claim, against a consumer looping on `pop`. +/// +/// The regime that can price the claim honestly, for the same reason the +/// reserving twin needs it: the shared line a producer touches is only +/// expensive when a consumer is writing it. Measured in isolation, an +/// uncontended line looks free -- which would be a confident wrong answer, and +/// this shape has more riding on that answer than the others, because it trades +/// `reserving_mpsc`'s *load* of the consumer's position for a read-modify-write +/// on a count the consumer also writes. +fn time_drained_permit(producers: usize) -> Repetition { + let (tx, rx) = permit_mpsc::bounded::(DRAINED_CAPACITY).expect("a valid capacity"); + let done = Arc::new(AtomicBool::new(false)); + let consumer_done = Arc::clone(&done); + // Set on every exit path, not just the one that returns. See StopOnDrop. + let stop = StopOnDrop(done); + // Closes the undrained opening the gate alone leaves. See await_consumer. + let ready = Arc::new(AtomicBool::new(false)); + let consumer_ready = Arc::clone(&ready); + let gate = start_gate(producers + 1); + let consumer_gate = Arc::clone(&gate); + + let consumer = thread::spawn(move || { + if !consumer_gate.arrive_and_wait() { + return rx.refused(); + } + // **One drain attempt BEFORE announcing, not merely reaching the loop.** + // Publishing first proves only that the consumer is about to drain: it + // can be descheduled between the store and its first `pop`, which is the + // same undrained opening in a narrower form. Popping first makes the + // announcement mean `this consumer has executed the pop path`, which is a + // fact rather than an intention. The queue is empty here, so it costs one + // failed pop, and it happens before any producer has started timing. + drain_then_announce( + || { + let _ = rx.pop(); + }, + &consumer_ready, + ); + while !consumer_done.load(Ordering::Relaxed) { + while rx.pop().is_ok() {} + std::hint::spin_loop(); + } + while rx.pop().is_ok() {} + rx.refused() + }); + + let spans = thread::scope(|scope| { + let _release = ReleaseOnDrop(Arc::clone(&gate)); + let mut workers = Vec::with_capacity(producers); + for producer in 0..producers { + let tx = tx.clone(); + let gate = Arc::clone(&gate); + let ready = Arc::clone(&ready); + workers.push(scope.spawn(move || { + if !gate.arrive_and_wait() { + // Abandoned before the party completed; see `StartGate`. + return (Instant::now(), Instant::now()); + } + await_consumer(&ready); + let began = Instant::now(); + for index in 0..PUSHES_PER_PRODUCER { + let mut item = (producer * PUSHES_PER_PRODUCER + index) as u64; + while let Err(error) = tx.push(item) { + // Only a FULL queue is retryable; see the note on the + // first of these loops. + assert!( + error.is_retryable(), + "the consumer is gone, so this push can never \ + succeed: {error}" + ); + item = error.into_inner(); + std::hint::spin_loop(); + } + } + (began, Instant::now()) + })); + } + let _ = gate.arrive_and_wait(); + workers + .into_iter() + .map(|worker| worker.join().expect("a producer must not panic")) + .collect::>() + }); + let elapsed = measured_span(&spans); + + drop(stop); + drop(tx); + let refusals = consumer.join().expect("the consumer must not panic"); + (elapsed, refusals) +} + +/// One claim-word layout, with no consumer and no possibility of refusal. +/// +/// As with the other isolated timers, this is the whole push path and not the +/// claim word alone; only the layout differs between these rows, so a difference +/// is still attributable to the layout, but its magnitude is a share of total +/// push cost rather than of the exchange. +/// +/// **Generic over the layout, where [`time_isolated_permit`] is deliberately +/// duplicated, and the difference is the point.** That twin compares two +/// *different types*, which a generic could only unify behind a trait, putting +/// an indirection that might not inline identically inside the timed region. +/// These are the *same type* at different layout parameters, so this +/// monomorphises to exactly the code a hand-written copy would produce -- there +/// is nothing left to dispatch. +/// +/// Measures `reserving_mpsc` itself rather than a stand-in. An earlier form of +/// this probe carried its own duplicated implementation of the claim protocol, +/// built so the layouts could be compared before the shipping crate had them; +/// it drifted from the original twice while doing so. The shipping type takes +/// the layout as a parameter now, so the duplicate is gone. +fn time_isolated_layout(producers: usize) -> Repetition { + let (tx, rx) = + reserving_mpsc::bounded_as::(capacity_for(producers)).expect("a valid capacity"); + let gate = start_gate(producers); + let spans = thread::scope(|scope| { + let _release = ReleaseOnDrop(Arc::clone(&gate)); + let mut workers = Vec::with_capacity(producers); + for producer in 0..producers { + let tx = tx.clone(); + let gate = Arc::clone(&gate); + workers.push(scope.spawn(move || { + if !gate.arrive_and_wait() { + // Abandoned before the party completed; see `StartGate`. + return (Instant::now(), Instant::now()); + } + let began = Instant::now(); + for index in 0..PUSHES_PER_PRODUCER { + tx.push((producer * PUSHES_PER_PRODUCER + index) as u64) + .expect("the run fits in the capacity"); + } + (began, Instant::now()) + })); + } + let _ = gate.arrive_and_wait(); + workers + .into_iter() + .map(|worker| worker.join().expect("a producer must not panic")) + .collect::>() + }); + let elapsed = measured_span(&spans); + let refusals = tx.refused(); + while rx.pop().is_ok() {} + (elapsed, refusals) +} + +/// One claim-word layout, against a consumer looping on `pop`. +/// +/// Generic for [`time_isolated_layout`]'s reason. +fn time_drained_layout(producers: usize) -> Repetition { + let (tx, rx) = + reserving_mpsc::bounded_as::(DRAINED_CAPACITY).expect("a valid capacity"); + let done = Arc::new(AtomicBool::new(false)); + let consumer_done = Arc::clone(&done); + // Set on every exit path, not just the one that returns. See StopOnDrop. + let stop = StopOnDrop(done); + // Closes the undrained opening the gate alone leaves. See await_consumer. + let ready = Arc::new(AtomicBool::new(false)); + let consumer_ready = Arc::clone(&ready); + // The consumer joins the gate for the reason its twins do: a run whose + // opening is undrained is not the regime being measured. + let gate = start_gate(producers + 1); + let consumer_gate = Arc::clone(&gate); + + let consumer = thread::spawn(move || { + if !consumer_gate.arrive_and_wait() { + return rx.refused(); + } + // **One drain attempt BEFORE announcing, not merely reaching the loop.** + // Publishing first proves only that the consumer is about to drain: it + // can be descheduled between the store and its first `pop`, which is the + // same undrained opening in a narrower form. Popping first makes the + // announcement mean `this consumer has executed the pop path`, which is a + // fact rather than an intention. The queue is empty here, so it costs one + // failed pop, and it happens before any producer has started timing. + drain_then_announce( + || { + let _ = rx.pop(); + }, + &consumer_ready, + ); + while !consumer_done.load(Ordering::Relaxed) { + while rx.pop().is_ok() {} + std::hint::spin_loop(); + } + while rx.pop().is_ok() {} + rx.refused() + }); + + let spans = thread::scope(|scope| { + let _release = ReleaseOnDrop(Arc::clone(&gate)); + let mut workers = Vec::with_capacity(producers); + for producer in 0..producers { + let tx = tx.clone(); + let gate = Arc::clone(&gate); + let ready = Arc::clone(&ready); + workers.push(scope.spawn(move || { + if !gate.arrive_and_wait() { + // Abandoned before the party completed; see `StartGate`. + return (Instant::now(), Instant::now()); + } + await_consumer(&ready); + let began = Instant::now(); + for index in 0..PUSHES_PER_PRODUCER { + let mut item = (producer * PUSHES_PER_PRODUCER + index) as u64; + while let Err(error) = tx.push(item) { + // Only a FULL queue is retryable; see the note on the + // first of these loops. + assert!( + error.is_retryable(), + "the consumer is gone, so this push can never \ + succeed: {error}" + ); + item = error.into_inner(); + std::hint::spin_loop(); + } + } + (began, Instant::now()) + })); + } + let _ = gate.arrive_and_wait(); + workers + .into_iter() + .map(|worker| worker.join().expect("a producer must not panic")) + .collect::>() + }); + let elapsed = measured_span(&spans); + drop(stop); + drop(tx); + let refusals = consumer.join().expect("the consumer must not panic"); + (elapsed, refusals) +} diff --git a/crates/windows-platform-probes/src/queue_contention/tests.rs b/crates/windows-platform-probes/src/queue_contention/tests.rs new file mode 100644 index 000000000..76f2b04e1 --- /dev/null +++ b/crates/windows-platform-probes/src/queue_contention/tests.rs @@ -0,0 +1,1606 @@ +// Copyright (c) Mike Grier. + +//! Tests for the pure lookup helpers. +//! +//! These decide whether the report prints a ratio or `--`, so they are worth +//! testing directly; none of them needs the 65-second measurement. + +use super::*; +use std::time::Duration; + +/// A `Run` with everything but the fields under test held constant. +fn run(shape: &'static str, producers: usize, ops_per_second: f64) -> Run { + Run { + shape, + producers, + nanos_per_op: if ops_per_second > 0.0 { + 1_000_000_000.0 / ops_per_second + } else { + 0.0 + }, + ops_per_second, + refusals: 0, + fastest_nanos_per_op: if ops_per_second > 0.0 { + 1_000_000_000.0 / ops_per_second + } else { + 0.0 + }, + slowest_nanos_per_op: if ops_per_second > 0.0 { + 1_000_000_000.0 / ops_per_second + } else { + 0.0 + }, + } +} + +fn observation(isolated: Vec, drained: Vec) -> Observation { + Observation { + isolated, + drained, + available_parallelism: Some(8), + } +} + +fn sample() -> Observation { + observation( + vec![ + run(shapes::BASELINE_FETCH_ADD, 1, 400_000_000.0), + run(shapes::BASELINE_FETCH_ADD, 4, 800_000_000.0), + run(shapes::RESERVING_MPSC, 1, 200_000_000.0), + run(shapes::RESERVING_MPSC, 4, 100_000_000.0), + run(shapes::SLOTWISE_MPSC, 1, 100_000_000.0), + ], + vec![ + run(shapes::RESERVING_MPSC, 1, 40_000_000.0), + run(shapes::RESERVING_MPSC, 4, 10_000_000.0), + ], + ) +} + +#[test] +fn find_returns_the_row_matching_both_shape_and_producer_count() { + let observed = sample(); + let found = observed + .find(&observed.isolated, shapes::RESERVING_MPSC, 4) + .expect("the row is present"); + assert_eq!(found.shape, shapes::RESERVING_MPSC); + assert_eq!(found.producers, 4); +} + +#[test] +fn find_distinguishes_rows_that_share_a_shape() { + let observed = sample(); + let one = observed + .find(&observed.isolated, shapes::RESERVING_MPSC, 1) + .expect("present"); + let four = observed + .find(&observed.isolated, shapes::RESERVING_MPSC, 4) + .expect("present"); + assert_ne!(one.ops_per_second, four.ops_per_second); +} + +#[test] +fn find_distinguishes_rows_that_share_a_producer_count() { + let observed = sample(); + let reserving = observed + .find(&observed.isolated, shapes::RESERVING_MPSC, 1) + .expect("present"); + let slotwise = observed + .find(&observed.isolated, shapes::SLOTWISE_MPSC, 1) + .expect("present"); + assert_ne!(reserving.ops_per_second, slotwise.ops_per_second); +} + +#[test] +fn find_returns_none_for_a_shape_that_was_not_measured() { + let observed = sample(); + assert!( + observed + .find(&observed.isolated, shapes::CLAIM_WIDE, 1) + .is_none() + ); +} + +#[test] +fn find_returns_none_for_a_producer_count_that_was_not_measured() { + let observed = sample(); + assert!( + observed + .find(&observed.isolated, shapes::RESERVING_MPSC, 32) + .is_none() + ); +} + +/// The regime is a parameter, so the same shape and count must not leak across. +#[test] +fn find_reads_only_the_regime_it_is_given() { + let observed = sample(); + let isolated = observed + .find(&observed.isolated, shapes::RESERVING_MPSC, 1) + .expect("present in isolated"); + let drained = observed + .find(&observed.drained, shapes::RESERVING_MPSC, 1) + .expect("present in drained"); + assert_ne!(isolated.ops_per_second, drained.ops_per_second); + assert!( + observed + .find(&observed.drained, shapes::SLOTWISE_MPSC, 1) + .is_none(), + "slotwise was measured only in the isolated regime here" + ); +} + +#[test] +fn find_on_an_empty_regime_is_none_rather_than_a_panic() { + let observed = observation(Vec::new(), Vec::new()); + assert!( + observed + .find(&observed.isolated, shapes::RESERVING_MPSC, 1) + .is_none() + ); +} + +#[test] +fn scaling_divides_the_many_producer_rate_by_the_one_producer_rate() { + let observed = sample(); + // baseline: 800M at four producers against 400M at one. + let scaled = observed + .scaling(&observed.isolated, shapes::BASELINE_FETCH_ADD, 4) + .expect("both rows present"); + assert!( + (scaled - 2.0).abs() < f64::EPSILON, + "expected 2.0, got {scaled}" + ); +} + +/// The direction matters: a contended claim scales *below* one, and reporting +/// the reciprocal would turn the finding upside down. +#[test] +fn scaling_below_one_means_more_producers_pushed_fewer_items() { + let observed = sample(); + let scaled = observed + .scaling(&observed.isolated, shapes::RESERVING_MPSC, 4) + .expect("both rows present"); + assert!( + (scaled - 0.5).abs() < f64::EPSILON, + "expected 0.5, got {scaled}" + ); + assert!(scaled < 1.0, "this is what a contended claim looks like"); +} + +#[test] +fn scaling_at_one_producer_is_one_by_construction() { + let observed = sample(); + let scaled = observed + .scaling(&observed.isolated, shapes::RESERVING_MPSC, 1) + .expect("the one-producer row is present"); + assert!( + (scaled - 1.0).abs() < f64::EPSILON, + "expected 1.0, got {scaled}" + ); +} + +#[test] +fn scaling_is_none_when_the_one_producer_row_is_missing() { + let observed = observation( + vec![run(shapes::RESERVING_MPSC, 4, 100_000_000.0)], + Vec::new(), + ); + assert!( + observed + .scaling(&observed.isolated, shapes::RESERVING_MPSC, 4) + .is_none(), + "without the one-producer row there is nothing to scale against" + ); +} + +#[test] +fn scaling_is_none_when_the_many_producer_row_is_missing() { + let observed = observation( + vec![run(shapes::RESERVING_MPSC, 1, 200_000_000.0)], + Vec::new(), + ); + assert!( + observed + .scaling(&observed.isolated, shapes::RESERVING_MPSC, 32) + .is_none() + ); +} + +#[test] +fn scaling_is_none_for_a_shape_absent_from_the_regime() { + let observed = sample(); + assert!( + observed + .scaling(&observed.drained, shapes::BASELINE_FETCH_ADD, 4) + .is_none() + ); +} + +/// A zero denominator yields a non-finite value rather than a panic. That is +/// deliberate -- `scaling` is arithmetic, not a renderer -- and it is why +/// `format_scaling` in the binary must filter non-finite values before +/// formatting, or a degenerate observation prints `infx` in a column of +/// measurements. This test pins the half of that contract the library owns. +#[test] +fn scaling_against_a_zero_rate_is_non_finite_rather_than_a_panic() { + let observed = observation( + vec![ + run(shapes::RESERVING_MPSC, 1, 0.0), + run(shapes::RESERVING_MPSC, 4, 100_000_000.0), + ], + Vec::new(), + ); + let scaled = observed + .scaling(&observed.isolated, shapes::RESERVING_MPSC, 4) + .expect("both rows are present"); + assert!(!scaled.is_finite(), "expected non-finite, got {scaled}"); +} + +/// Every shape name must be distinct, or `find` would return whichever row +/// happened to come first and two columns would silently show one measurement. +#[test] +fn every_shape_name_is_distinct() { + let names = [ + shapes::BASELINE_FETCH_ADD, + shapes::SLOTWISE_MPSC, + shapes::RESERVING_MPSC, + shapes::PERMIT_MPSC, + shapes::CLAIM_NARROW, + shapes::CLAIM_DEEP, + shapes::CLAIM_PERPETUAL, + shapes::CLAIM_WIDE, + ]; + for (index, name) in names.iter().enumerate() { + assert!( + !names[..index].contains(name), + "{name} appears more than once" + ); + } +} + +/// `capacity_for` must leave room for every push, or the isolated regime would +/// refuse and stop being the regime it claims to be. +/// +/// `>=` rather than `>`: a `bounded(n)` queue accepts exactly `n` items +/// (measured, not assumed), so an exact fit is sufficient. Today the product is +/// never a power of two, so the distinction is unreachable -- but M4.2 makes the +/// push count settable, and a stricter assertion than the contract requires would +/// reject a valid configuration then. +#[test] +fn capacity_for_leaves_room_for_every_push_at_every_producer_count() { + for &producers in PRODUCER_COUNTS { + let capacity = capacity_for(producers); + let pushes = producers * PUSHES_PER_PRODUCER; + assert!( + capacity >= pushes, + "{producers} producers push {pushes} but capacity is {capacity}" + ); + assert!( + capacity.is_power_of_two(), + "{capacity} must be a power of two" + ); + } +} + +/// `measured_span` is the correction that round two of this branch's review +/// produced, and it had no test until round seventeen asked for one. The probe +/// previously timed from the coordinator's clock, which understated elapsed time +/// and overstated throughput by roughly 45% at high producer counts. These pin +/// the shape of the replacement: the span runs from the EARLIEST worker start to +/// the LATEST worker finish, so no worker's time is outside it. +/// +/// `Instant` cannot be constructed from a literal, so each case builds one from +/// a single `now` and offsets it. That keeps the arithmetic exact without making +/// the test depend on how long it takes to run. +#[test] +fn measured_span_runs_from_the_earliest_start_to_the_latest_finish() { + let base = Instant::now(); + let ms = Duration::from_millis(1); + // Three workers, deliberately out of order and overlapping: the earliest + // start belongs to the second, the latest finish to the third. + let spans = vec![ + (base + 10 * ms, base + 40 * ms), + (base + 5 * ms, base + 20 * ms), + (base + 30 * ms, base + 60 * ms), + ]; + let nanos = measured_span(&spans); + // 5ms..60ms + let expected = (55 * ms).as_nanos() as f64; + assert!( + (nanos - expected).abs() < f64::EPSILON, + "expected {expected} ns, got {nanos}" + ); +} + +/// The defect the correction replaced would have measured one worker's slice, or +/// the coordinator's view of it. Any narrower aggregation than min-start to +/// max-end is therefore what this guards against. +#[test] +fn measured_span_is_wider_than_any_single_worker() { + let base = Instant::now(); + let ms = Duration::from_millis(1); + let spans = vec![ + (base + 10 * ms, base + 20 * ms), + (base + 15 * ms, base + 50 * ms), + (base, base + 5 * ms), + ]; + let nanos = measured_span(&spans); + for (began, ended) in &spans { + let worker = ended.duration_since(*began).as_nanos() as f64; + assert!( + nanos >= worker, + "span {nanos} must cover every worker, but one ran {worker}" + ); + } + assert!( + (nanos - (50 * ms).as_nanos() as f64).abs() < f64::EPSILON, + "expected the full 0..50ms window, got {nanos}" + ); +} + +#[test] +fn measured_span_of_one_worker_is_that_worker() { + let base = Instant::now(); + let ms = Duration::from_millis(1); + let nanos = measured_span(&[(base + 3 * ms, base + 11 * ms)]); + assert!( + (nanos - (8 * ms).as_nanos() as f64).abs() < f64::EPSILON, + "expected 8ms, got {nanos}" + ); +} + +/// Workers that never overlap still yield one span covering both, because the +/// question the probe asks is how long the whole configuration took. +#[test] +fn measured_span_covers_disjoint_workers() { + let base = Instant::now(); + let ms = Duration::from_millis(1); + let nanos = measured_span(&[(base, base + ms), (base + 100 * ms, base + 101 * ms)]); + assert!( + (nanos - (101 * ms).as_nanos() as f64).abs() < f64::EPSILON, + "expected 101ms, got {nanos}" + ); +} + +/// The renderer's cells. These were in the binary and therefore untestable until +/// they moved into this module; the non-finite case in particular is documented +/// by `scaling_against_a_zero_rate_is_non_finite_rather_than_a_panic` above and +/// was relying on that documentation rather than on a check. +#[test] +fn format_scaling_renders_a_finite_value_and_marks_everything_else() { + assert_eq!(format_scaling(Some(1.0)), "1.00x"); + assert_eq!(format_scaling(Some(0.5)), "0.50x"); + assert_eq!(format_scaling(Some(12.345)), "12.35x"); + assert_eq!(format_scaling(None), "--"); + assert_eq!(format_scaling(Some(f64::INFINITY)), "--"); + assert_eq!(format_scaling(Some(f64::NEG_INFINITY)), "--"); + assert_eq!(format_scaling(Some(f64::NAN)), "--"); +} + +#[test] +fn format_ratio_divides_and_guards_its_denominator() { + let fast = run(shapes::RESERVING_MPSC, 4, 200_000_000.0); + let slow = run(shapes::SLOTWISE_MPSC, 4, 100_000_000.0); + // slow is 10.0 ns/op, fast is 5.0, so fast/slow is 0.50x. + assert_eq!(format_ratio(Some(fast), Some(slow)), "0.50x"); + assert_eq!(format_ratio(Some(slow), Some(fast)), "2.00x"); + assert_eq!(format_ratio(None, Some(slow)), "--"); + assert_eq!(format_ratio(Some(fast), None), "--"); + assert_eq!(format_ratio(None, None), "--"); +} + +/// A shape that failed to run reports zero nanoseconds, and dividing by it would +/// put `inf` in a column a reader takes for a measurement. +#[test] +fn format_ratio_refuses_a_zero_denominator() { + let measured = run(shapes::RESERVING_MPSC, 4, 100_000_000.0); + let absent = run(shapes::SLOTWISE_MPSC, 4, 0.0); + assert_eq!(absent.nanos_per_op, 0.0, "the fixture must have zero cost"); + assert_eq!(format_ratio(Some(measured), Some(absent)), "--"); +} + +#[test] +fn format_nanos_renders_one_decimal_or_the_marker() { + assert_eq!( + format_nanos(Some(run(shapes::RESERVING_MPSC, 1, 1e9))), + "1.0" + ); + assert_eq!(format_nanos(None), "--"); +} + +#[test] +fn render_table_writes_a_header_and_one_line_per_run() { + let rows = vec![ + run(shapes::BASELINE_FETCH_ADD, 1, 400_000_000.0), + run(shapes::RESERVING_MPSC, 4, 100_000_000.0), + ]; + let mut out = String::new(); + render_table(&mut out, &rows); + let lines: Vec<&str> = out.lines().collect(); + assert_eq!(lines.len(), 3, "a header and two rows, got {out:?}"); + assert!(lines[0].contains("shape") && lines[0].contains("ns/op")); + assert!(lines[1].contains(shapes::BASELINE_FETCH_ADD)); + assert!(lines[2].contains(shapes::RESERVING_MPSC)); + assert!( + lines[2].contains("10.0"), + "100M ops/sec is 10.0 ns/op, got {:?}", + lines[2] + ); +} + +#[test] +fn render_table_of_nothing_still_writes_its_header() { + let mut out = String::new(); + render_table(&mut out, &[]); + assert_eq!(out.lines().count(), 1, "header only, got {out:?}"); +} + +/// A timer that replays a scripted sequence instead of measuring anything, so +/// `median_run`'s selection can be checked exactly. The first value is consumed +/// by the untimed warmup pass. +fn scripted(values: Vec) -> impl FnMut(usize) -> Repetition { + let mut next = 0usize; + move |_producers| { + let value = values[next]; + next += 1; + value + } +} + +/// The scripted repetitions used by the tests below, in the order `median_run` +/// calls for them. Three properties are deliberate and each catches a different +/// regression: +/// +/// - the durations are **not** in ascending order, so failing to sort at all +/// selects 5e6 rather than the median 3e6; +/// - the refusal counts are **not** monotonic in duration, so sorting by the +/// wrong tuple element also selects 5e6; +/// - no two durations are equal, so the median is unambiguous. +/// +/// Sort *direction* is deliberately not covered, because it cannot be: with +/// `REPETITIONS == 5`, `results[REPETITIONS / 2]` is index 2 of five, which is +/// the median whether the sort ascends or descends. A test claiming to pin +/// direction here would pass under both and be theatre. +fn scripted_repetitions() -> Vec { + vec![ + (999e6, 9_999), // warmup, discarded + (4e6, 40), + (1e6, 10), + (5e6, 30), + (2e6, 50), + (3e6, 20), + ] +} + +#[test] +fn median_run_reports_the_median_repetition_rather_than_the_first_or_last() { + let measured = median_run(shapes::RESERVING_MPSC, 1, scripted(scripted_repetitions())); + // 3e6 ns over 1 * 50,000 pushes is 60 ns per op. + assert!( + (measured.nanos_per_op - 60.0).abs() < 1e-9, + "expected the 3e6 ns median, got {} ns/op", + measured.nanos_per_op + ); + assert_eq!(measured.shape, shapes::RESERVING_MPSC); + assert_eq!(measured.producers, 1); +} + +/// The refusal count travels with the repetition whose duration was chosen. It +/// is the probe's only signal that a drained row was limited by the consumer +/// rather than by claim contention, so pairing it with a different repetition +/// would misattribute the cause while leaving the timing plausible. +#[test] +fn median_run_pairs_the_refusal_count_with_the_median_repetition() { + let measured = median_run(shapes::RESERVING_MPSC, 1, scripted(scripted_repetitions())); + assert_eq!( + measured.refusals, 20, + "3e6 ns is the median and its repetition refused 20; got {}", + measured.refusals + ); +} + +/// The warmup pass exists to take the first-call costs out of the sample, so its +/// value must not reach the report. Its scripted duration is the largest in the +/// sequence and its refusal count is unique, so either leaking into the result +/// is visible. +#[test] +fn median_run_discards_the_warmup_pass() { + let measured = median_run(shapes::RESERVING_MPSC, 1, scripted(scripted_repetitions())); + assert_ne!( + measured.refusals, 9_999, + "the warmup's refusals were reported" + ); + assert!( + measured.nanos_per_op < 100.0, + "the warmup's 999e6 ns reached the report as {} ns/op", + measured.nanos_per_op + ); +} + +/// Both published rates come from the same chosen repetition, so they must agree +/// with each other. A regression that derived one from the median and the other +/// from some different element would leave a report whose two columns describe +/// different runs. +#[test] +fn median_run_derives_both_rates_from_the_one_chosen_repetition() { + let measured = median_run(shapes::RESERVING_MPSC, 1, scripted(scripted_repetitions())); + let round_trip = 1_000_000_000.0 / measured.nanos_per_op; + assert!( + (measured.ops_per_second - round_trip).abs() < 1e-3, + "{} ops/sec does not agree with {} ns/op", + measured.ops_per_second, + measured.nanos_per_op + ); +} + +/// `median_run` scales by the producer count, so the same per-repetition +/// durations must report a lower per-push cost when more producers shared them. +#[test] +fn median_run_divides_the_median_by_every_producers_pushes() { + let one = median_run(shapes::RESERVING_MPSC, 1, scripted(scripted_repetitions())); + let four = median_run(shapes::RESERVING_MPSC, 4, scripted(scripted_repetitions())); + assert!( + (one.nanos_per_op / four.nanos_per_op - 4.0).abs() < 1e-9, + "four producers push four times as many items in the same span: {} vs {}", + one.nanos_per_op, + four.nanos_per_op + ); +} + +/// The refusal column is the probe's diagnosis of *why* a drained row is slow -- +/// consumer backpressure rather than claim contention -- so a row that dropped +/// or misformatted it would leave the report looking complete while the central +/// signal was silently absent. Every other renderer test builds rows refusing +/// nothing, which cannot catch that. +#[test] +fn render_table_shows_a_nonzero_refusal_count() { + let mut refused = run(shapes::SLOTWISE_MPSC, 8, 100_000_000.0); + refused.refusals = 123_456; + let mut out = String::new(); + render_table(&mut out, &[refused]); + let row = out.lines().nth(1).expect("one row was rendered"); + assert!( + row.contains("123456"), + "the refusal count is missing from {row:?}" + ); + assert!( + out.lines().next().expect("a header").contains("refusals"), + "the refusal column is unlabelled" + ); +} + +/// The dispersion `d-observations-not-verdicts` obliges the crate to publish. +/// `median_run` sorts ascending, so the extremes are the ends of that sort -- +/// these pin that the reported range is the whole sample rather than, say, the +/// median repeated or two adjacent repetitions. +#[test] +fn median_run_carries_the_fastest_and_slowest_repetitions() { + let measured = median_run(shapes::RESERVING_MPSC, 1, scripted(scripted_repetitions())); + // Scripted timed repetitions are 1e6..5e6 ns over 50,000 pushes: 20..100 ns/op. + assert!( + (measured.fastest_nanos_per_op - 20.0).abs() < 1e-9, + "expected the 1e6 ns repetition as fastest, got {} ns/op", + measured.fastest_nanos_per_op + ); + assert!( + (measured.slowest_nanos_per_op - 100.0).abs() < 1e-9, + "expected the 5e6 ns repetition as slowest, got {} ns/op", + measured.slowest_nanos_per_op + ); +} + +/// The median must lie inside the range, or the two are describing different +/// samples. This is the cheap invariant that catches a range computed from the +/// wrong vector or from an unsorted one. +#[test] +fn median_run_brackets_its_median_with_the_range() { + let measured = median_run(shapes::RESERVING_MPSC, 1, scripted(scripted_repetitions())); + assert!( + measured.fastest_nanos_per_op <= measured.nanos_per_op, + "fastest {} must not exceed the median {}", + measured.fastest_nanos_per_op, + measured.nanos_per_op + ); + assert!( + measured.nanos_per_op <= measured.slowest_nanos_per_op, + "median {} must not exceed the slowest {}", + measured.nanos_per_op, + measured.slowest_nanos_per_op + ); +} + +/// The warmup is excluded from the dispersion as well as from the median. Its +/// scripted 999e6 ns would otherwise dominate the range and make every row look +/// wildly unstable. +#[test] +fn median_run_excludes_the_warmup_from_the_range() { + let measured = median_run(shapes::RESERVING_MPSC, 1, scripted(scripted_repetitions())); + assert!( + measured.slowest_nanos_per_op < 1_000.0, + "the warmup's 999e6 ns reached the range as {} ns/op", + measured.slowest_nanos_per_op + ); +} + +#[test] +fn spread_is_the_slowest_over_the_fastest() { + let mut run = run(shapes::RESERVING_MPSC, 4, 100_000_000.0); + run.fastest_nanos_per_op = 10.0; + run.slowest_nanos_per_op = 13.0; + assert!( + (run.spread().expect("a measured row") - 1.3).abs() < 1e-9, + "got {:?}", + run.spread() + ); +} + +/// A configuration whose repetitions all took the same time has a spread of +/// exactly one, which is what "this host held still" looks like. +#[test] +fn spread_of_an_identical_sample_is_one() { + let mut run = run(shapes::RESERVING_MPSC, 4, 100_000_000.0); + run.fastest_nanos_per_op = 42.0; + run.slowest_nanos_per_op = 42.0; + assert!( + (run.spread().expect("a measured row") - 1.0).abs() < 1e-9, + "got {:?}", + run.spread() + ); +} + +/// A shape that failed to run reports zero, and dividing by it would put `inf` +/// in a column a reader takes for a measurement -- the same guard +/// `format_ratio` carries. +/// A row that never ran has no spread to report. It used to answer `0.0` here, +/// which the report rendered as `0.00x` -- the most reassuring value the column +/// can hold, meaning "perfectly stable", produced by a shape that measured +/// nothing. `None` is the honest answer and the renderer turns it into `--`. +#[test] +fn spread_of_a_zero_sample_is_none_rather_than_a_reassuring_number() { + let mut run = run(shapes::SLOTWISE_MPSC, 4, 0.0); + run.fastest_nanos_per_op = 0.0; + run.slowest_nanos_per_op = 0.0; + assert_eq!(run.spread(), None, "an unmeasured row has no spread"); + assert_eq!( + format_scaling(run.spread()), + "--", + "and it must not render as a number" + ); +} + +#[test] +fn render_table_publishes_the_range_and_the_spread() { + let mut row = run(shapes::RESERVING_MPSC, 8, 100_000_000.0); + row.fastest_nanos_per_op = 9.5; + row.slowest_nanos_per_op = 12.5; + let mut out = String::new(); + render_table(&mut out, &[row]); + let header = out.lines().next().expect("a header"); + let line = out.lines().nth(1).expect("one row"); + assert!( + header.contains("range") && header.contains("spread"), + "the dispersion columns are unlabelled: {header:?}" + ); + assert!( + line.contains("9.5-12.5"), + "the range is missing from {line:?}" + ); + assert!( + line.contains("1.32x"), + "the spread is missing from {line:?}" + ); +} + +/// Helper: a row with an explicit cost span, for the bound arithmetic. +fn run_spanning( + shape: &'static str, + producers: usize, + fastest: f64, + median: f64, + slowest: f64, +) -> Run { + Run { + shape, + producers, + nanos_per_op: median, + ops_per_second: if median > 0.0 { + 1_000_000_000.0 / median + } else { + 0.0 + }, + refusals: 0, + fastest_nanos_per_op: fastest, + slowest_nanos_per_op: slowest, + } +} + +/// The bound pairs each side's extreme against the other's opposite extreme, +/// because that is the widest the ratio could be. Rate ratio inverts cost, so +/// the smallest rate ratio is the numerator at its slowest against the +/// denominator at its fastest. +#[test] +fn ratio_bounds_pairs_opposing_extremes() { + let numerator = run_spanning(shapes::RESERVING_MPSC, 4, 8.0, 10.0, 12.0); + let denominator = run_spanning(shapes::SLOTWISE_MPSC, 4, 40.0, 50.0, 60.0); + let (low, high) = ratio_bounds(numerator, denominator).expect("both spans are positive"); + // cost ratio low = num.fastest / den.slowest = 8 / 60 + // cost ratio high = num.slowest / den.fastest = 12 / 40 + assert!((low - (8.0 / 60.0)).abs() < 1e-9, "low was {low}"); + assert!((high - (12.0 / 40.0)).abs() < 1e-9, "high was {high}"); +} + +/// The invariant that makes the bound meaningful: whatever point estimate the +/// medians produce must lie inside it. A bound that excluded its own point +/// estimate would be arithmetic nobody should trust. +#[test] +fn ratio_bounds_contain_the_point_estimate() { + let numerator = run_spanning(shapes::RESERVING_MPSC, 4, 8.0, 10.0, 12.0); + let denominator = run_spanning(shapes::SLOTWISE_MPSC, 4, 40.0, 50.0, 60.0); + let point = numerator.nanos_per_op / denominator.nanos_per_op; + let (low, high) = ratio_bounds(numerator, denominator).expect("both spans are positive"); + assert!( + low <= point && point <= high, + "the point estimate {point} falls outside its own bound [{low}, {high}]" + ); +} + +/// A row whose span touches zero cannot be divided by, the same case +/// `format_ratio` and `spread` already guard. +#[test] +fn ratio_bounds_of_a_zero_span_is_none() { + let real = run_spanning(shapes::RESERVING_MPSC, 4, 8.0, 10.0, 12.0); + let absent = run_spanning(shapes::SLOTWISE_MPSC, 4, 0.0, 0.0, 0.0); + assert!(ratio_bounds(real, absent).is_none()); + assert!(ratio_bounds(absent, real).is_none()); +} + +/// A configuration whose repetitions all agreed gives a bound of zero width, +/// which is what "this host held still" looks like for a derived figure. +#[test] +fn ratio_bounds_of_two_exact_samples_is_a_point() { + let numerator = run_spanning(shapes::RESERVING_MPSC, 4, 10.0, 10.0, 10.0); + let denominator = run_spanning(shapes::SLOTWISE_MPSC, 4, 50.0, 50.0, 50.0); + let (low, high) = ratio_bounds(numerator, denominator).expect("positive spans"); + // Cost ratio: 10 over 50. + assert!( + (low - 0.2).abs() < 1e-9 && (high - 0.2).abs() < 1e-9, + "[{low},{high}]" + ); +} + +#[test] +fn format_ratio_bounded_renders_the_point_and_its_interval() { + let numerator = run_spanning(shapes::RESERVING_MPSC, 4, 8.0, 10.0, 12.0); + let denominator = run_spanning(shapes::SLOTWISE_MPSC, 4, 40.0, 50.0, 60.0); + let rendered = format_ratio_bounded(Some(numerator), Some(denominator)); + assert!( + rendered.starts_with("0.20x ["), + "expected the point estimate first, got {rendered:?}" + ); + assert!( + rendered.contains('[') && rendered.contains(']'), + "the bound must be bracketed to mark it as not a sampled range: {rendered:?}" + ); +} + +#[test] +fn format_ratio_bounded_marks_a_missing_or_zero_row() { + let real = run_spanning(shapes::RESERVING_MPSC, 4, 8.0, 10.0, 12.0); + let zero = run_spanning(shapes::SLOTWISE_MPSC, 4, 0.0, 0.0, 0.0); + assert_eq!(format_ratio_bounded(None, Some(real)), "--"); + assert_eq!(format_ratio_bounded(Some(real), None), "--"); + assert_eq!(format_ratio_bounded(Some(real), Some(zero)), "--"); +} + +#[test] +fn format_scaling_bounded_renders_point_and_interval_or_the_marker() { + assert_eq!( + format_scaling_bounded(Some(2.0), Some((1.5, 2.5))), + "2.00x [1.50-2.50]" + ); + assert_eq!(format_scaling_bounded(None, Some((1.5, 2.5))), "--"); + assert_eq!( + format_scaling_bounded(Some(f64::NAN), Some((1.0, 2.0))), + "--" + ); + assert_eq!( + format_scaling_bounded(Some(f64::INFINITY), Some((1.0, 2.0))), + "--" + ); + // A point estimate with no computable bound still renders, unbracketed. + assert_eq!(format_scaling_bounded(Some(2.0), None), "2.00x"); +} + +#[test] +fn scaling_bounds_reads_the_one_and_many_producer_rows() { + let observation = Observation { + isolated: vec![ + run_spanning(shapes::RESERVING_MPSC, 1, 4.0, 5.0, 6.0), + run_spanning(shapes::RESERVING_MPSC, 8, 40.0, 50.0, 60.0), + ], + drained: Vec::new(), + available_parallelism: Some(8), + }; + let (low, high) = observation + .scaling_bounds(&observation.isolated, shapes::RESERVING_MPSC, 8) + .expect("both rows present with positive spans"); + let point = observation + .scaling(&observation.isolated, shapes::RESERVING_MPSC, 8) + .expect("both rows present"); + assert!( + low <= point && point <= high, + "scaling {point} outside its bound [{low}, {high}]" + ); +} + +#[test] +fn scaling_bounds_is_none_when_a_row_is_missing() { + let observation = Observation { + isolated: vec![run_spanning(shapes::RESERVING_MPSC, 8, 40.0, 50.0, 60.0)], + drained: Vec::new(), + available_parallelism: Some(8), + }; + assert!( + observation + .scaling_bounds(&observation.isolated, shapes::RESERVING_MPSC, 8) + .is_none(), + "the one-producer row is absent, so no bound exists" + ); +} + +/// Zero is the sentinel for "this shape did not run", and it is as meaningful on +/// the numerator side as on the denominator. A zero denominator would render +/// `inf`; a zero numerator renders `0.00x`, which is worse -- `inf` announces +/// itself as broken, and `0.00x` reads as a shape that was immeasurably fast. +#[test] +fn format_ratio_refuses_a_zero_numerator() { + let measured = run(shapes::RESERVING_MPSC, 4, 100_000_000.0); + let absent = run(shapes::SLOTWISE_MPSC, 4, 0.0); + assert_eq!(absent.nanos_per_op, 0.0, "the fixture must have zero cost"); + assert_eq!( + format_ratio(Some(absent), Some(measured)), + "--", + "a row that never ran must not render as a ratio" + ); +} + +#[test] +fn format_ratio_bounded_refuses_a_zero_numerator() { + let measured = run_spanning(shapes::RESERVING_MPSC, 4, 8.0, 10.0, 12.0); + let absent = run_spanning(shapes::SLOTWISE_MPSC, 4, 0.0, 0.0, 0.0); + assert_eq!( + format_ratio_bounded(Some(absent), Some(measured)), + "--", + "a row that never ran must not render as a ratio" + ); +} + +/// Both formatters agree about what is unmeasurable, in both positions. They are +/// separate functions with separate guards, which is exactly how one of them +/// came to guard only half the cases. +#[test] +fn both_ratio_formatters_reject_the_same_unmeasurable_rows() { + let measured = run_spanning(shapes::RESERVING_MPSC, 4, 8.0, 10.0, 12.0); + let absent = run_spanning(shapes::SLOTWISE_MPSC, 4, 0.0, 0.0, 0.0); + for (numerator, denominator) in [ + (Some(absent), Some(measured)), + (Some(measured), Some(absent)), + (Some(absent), Some(absent)), + (None, Some(measured)), + (Some(measured), None), + ] { + assert_eq!( + format_ratio(numerator, denominator), + "--", + "format_ratio accepted an unmeasurable pair" + ); + assert_eq!( + format_ratio_bounded(numerator, denominator), + "--", + "format_ratio_bounded accepted an unmeasurable pair" + ); + } +} + +/// The renderer is where the sentinel did its damage, so the guard is asserted +/// there and not only on the accessor. A row that measured nothing must show +/// `--` in the spread column rather than a number a reader would take for +/// stability. +#[test] +fn render_table_marks_an_unmeasured_spread_rather_than_printing_zero() { + let mut absent = run(shapes::SLOTWISE_MPSC, 8, 0.0); + absent.fastest_nanos_per_op = 0.0; + absent.slowest_nanos_per_op = 0.0; + let mut out = String::new(); + render_table(&mut out, &[absent]); + let row = out.lines().nth(1).expect("one row was rendered"); + assert!( + !row.contains("0.00x"), + "an unmeasured row rendered a spread that reads as perfect stability: {row:?}" + ); + assert!( + row.contains("--"), + "an unmeasured spread must be marked: {row:?}" + ); +} + +/// The sentinel is one predicate now, so it is pinned directly. +/// +/// [`Run::is_measured`] exists so a renderer cannot forget the test by writing +/// it slightly differently. That only helps if the predicate is itself right, +/// which a test routed through a formatter would not establish. +#[test] +fn is_measured_rejects_every_shape_of_unmeasured_row() { + assert!(run(shapes::RESERVING_MPSC, 1, 1e9).is_measured()); + + let mut row = run(shapes::RESERVING_MPSC, 1, 0.0); + assert!( + !row.is_measured(), + "a zero cost is the did-not-run sentinel" + ); + + row.nanos_per_op = f64::NAN; + assert!(!row.is_measured(), "NaN is not a measurement"); + + row.nanos_per_op = f64::INFINITY; + assert!(!row.is_measured(), "infinity is not a measurement"); + + row.nanos_per_op = -1.0; + assert!(!row.is_measured(), "a negative cost is not a measurement"); +} + +/// A row that did not run must not put a number in *any* measured column. +/// +/// The test above this one checked the spread cell alone, and passed while the +/// very same row published `0.0` ns/op, `0` ops/sec and a `0.0-0.0` range -- +/// three cells that read as a shape too fast to time. Checking one cell of a +/// row is what let the other four drift, so this asserts over the whole row. +#[test] +fn render_table_marks_every_measured_cell_of_a_row_that_did_not_run() { + let mut absent = run(shapes::SLOTWISE_MPSC, 8, 0.0); + absent.fastest_nanos_per_op = 0.0; + absent.slowest_nanos_per_op = 0.0; + let mut out = String::new(); + render_table(&mut out, &[absent]); + let row = out.lines().nth(1).expect("one row was rendered"); + + // Shape and producer count are configuration, not measurement: they are + // known whether or not the row ran, and must survive. + assert!( + row.contains(shapes::SLOTWISE_MPSC), + "the shape is configuration and must still be named: {row:?}" + ); + assert!( + row.contains('8'), + "the producer count is configuration and must survive: {row:?}" + ); + + assert_eq!( + row.matches("--").count(), + 5, + "ns/op, ops/sec, refusals, range and spread must all be marked: {row:?}" + ); + assert!( + !row.contains("0.0"), + "an unmeasured row published a number: {row:?}" + ); +} + +/// A shape that did not run must not publish a cost. +#[test] +fn format_nanos_marks_a_row_that_did_not_run() { + assert_eq!(format_nanos(Some(run(shapes::SLOTWISE_MPSC, 8, 0.0))), "--"); + + let mut broken = run(shapes::SLOTWISE_MPSC, 8, 1e9); + broken.nanos_per_op = f64::NAN; + assert_eq!(format_nanos(Some(broken)), "--"); +} + +/// Zero scaling is the sentinel, and it is the half that renders plausibly. +/// +/// [`Observation::scaling`] divides the many-producer rate by the one-producer +/// rate, so a many-producer row that did not run yields exactly `Some(0.0)`. +/// Rendered, that is `0.00x` in a column where values near `1` are the normal +/// reading -- it looks like a queue that failed to *scale* rather than one that +/// failed to *run*. The other sentinel, `infx`, at least announces itself. +#[test] +fn format_scaling_marks_a_zero_rather_than_publishing_it() { + assert_eq!(format_scaling(Some(0.0)), "--"); + assert_eq!(format_scaling_bounded(Some(0.0), None), "--"); + assert_eq!(format_scaling_bounded(Some(0.0), Some((0.0, 0.0))), "--"); +} + +/// The whole path, so the guard is pinned where a reader would meet it. +/// +/// The formatter tests above supply the sentinel by hand. This one makes the +/// probe's own arithmetic produce it, which is the only way to show the two +/// halves agree about what a did-not-run row looks like. +#[test] +fn a_many_producer_row_that_did_not_run_scales_to_the_marker() { + let observation = Observation { + isolated: vec![ + run(shapes::RESERVING_MPSC, 1, 1e8), + run(shapes::RESERVING_MPSC, 8, 0.0), + ], + drained: Vec::new(), + available_parallelism: Some(8), + }; + let point = observation.scaling(&observation.isolated, shapes::RESERVING_MPSC, 8); + assert_eq!( + point, + Some(0.0), + "the sentinel reaches the renderer as a plain zero" + ); + let bounds = observation.scaling_bounds(&observation.isolated, shapes::RESERVING_MPSC, 8); + assert_eq!( + format_scaling_bounded(point, bounds), + "--", + "a shape that never ran must not publish a scaling factor" + ); +} + +/// The ratio column must fit every cell it is asked to hold. +/// +/// A Rust width is a *minimum*, so a value wider than its field is not +/// truncated -- it pushes every column after it out of alignment with its +/// header, silently. The layout table allocated 10 characters to a formatter +/// whose ordinary output is 17, so the second and third ratio columns had been +/// rendering seven and fourteen characters adrift. +/// +/// **An earlier version of this test asserted `cell.len() <= RATIO_COLUMN_WIDTH`, +/// and that invariant is not available.** The interval's endpoints are measured +/// spans, so the cell's width is a function of data and has no compile-time +/// bound; the test passed only because its fixtures happened to be narrow. The +/// width is now derived from the cells, and what is pinned is that derivation. +#[test] +fn ratio_column_fits_every_cell_it_must_hold() { + let ordinary = format_ratio_bounded( + Some(run_spanning(shapes::CLAIM_WIDE, 32, 40.0, 50.0, 60.0)), + Some(run_spanning(shapes::CLAIM_NARROW, 32, 4.0, 5.0, 6.0)), + ); + // The guard is only meaningful if the formatter really does emit the wide + // point-and-interval form -- otherwise it would pass against a bare `1.00x`. + assert!( + ordinary.contains('['), + "expected a bounded ratio, got {ordinary:?}" + ); + + // A slow outlier -- a 300 ms repetition against a 4 ns one, which is exactly + // what `median_run` takes a median to survive -- overruns the floor. + let outlier = format_ratio_bounded( + Some(run_spanning(shapes::CLAIM_WIDE, 32, 40.0, 50.0, 6000.0)), + Some(run_spanning(shapes::CLAIM_NARROW, 32, 4.0, 5.0, 6.0)), + ); + assert!( + outlier.len() > RATIO_COLUMN_WIDTH, + "this fixture exists to exceed the floor; if it no longer does, the \ + case it guards has stopped being exercised: {outlier:?}" + ); + + let cells = [ordinary.as_str(), outlier.as_str(), "--"]; + let width = ratio_column_width(cells); + for cell in cells { + assert!( + cell.len() <= width, + "{cell:?} is {} characters and would push the next column out of \ + line with its header, which allows {width}", + cell.len() + ); + } +} + +/// The floor applies when every cell is narrower than it. +#[test] +fn ratio_column_never_narrows_below_its_floor() { + assert_eq!(ratio_column_width(["1.00x", "--"]), RATIO_COLUMN_WIDTH); + assert_eq!( + ratio_column_width(std::iter::empty()), + RATIO_COLUMN_WIDTH, + "a regime with no rows still needs a header that lines up" + ); +} + +/// A row that is present but never ran must not be counted as measured. +/// +/// The report says "N apportionments ... measured" above a table in which +/// `render_table` marks every cell of an unmeasured row `--`. Counting presence +/// rather than measurement let those two halves disagree: the sentence claimed +/// four layouts while the table showed one of them as having produced nothing. +#[test] +fn count_measured_counts_rows_that_ran_rather_than_rows_that_exist() { + let layouts = [ + shapes::CLAIM_NARROW, + shapes::CLAIM_DEEP, + shapes::CLAIM_PERPETUAL, + shapes::CLAIM_WIDE, + ]; + let observation = Observation { + isolated: vec![ + run(shapes::CLAIM_NARROW, 1, 1e8), + run(shapes::CLAIM_DEEP, 1, 1e8), + // Present, but carrying the did-not-run sentinel. + run(shapes::CLAIM_PERPETUAL, 1, 0.0), + // CLAIM_WIDE absent entirely, as it is when cfg-elided. + ], + drained: Vec::new(), + available_parallelism: Some(8), + }; + assert_eq!( + observation.count_measured(&observation.isolated, &layouts, 1), + 2, + "a present-but-unmeasured row must not be counted, and an absent one \ + must not be either" + ); +} + +/// A `<= 0.0` test does not reject `NaN`, and this pins that it is rejected. +/// +/// Every comparison against `NaN` is false, so the four-way `<= 0.0` guard this +/// function used to carry fell straight through to the division for a `NaN` +/// endpoint and produced `[NaN-NaN]`. The endpoints are public fields, so this +/// is reachable without going through `median_run`. +#[test] +fn ratio_bounds_rejects_non_finite_endpoints() { + let sound = run_spanning(shapes::RESERVING_MPSC, 8, 4.0, 5.0, 6.0); + assert!( + ratio_bounds(sound, sound).is_some(), + "the fixture must otherwise produce a bound, or this proves nothing" + ); + + // **All four endpoints, each on its own.** An earlier version of this test + // poisoned `numerator.fastest` and `denominator.slowest` only -- the two + // that feed the LOWER bound -- so dropping either of the upper bound's + // endpoints from the guard would have left it green while `ratio_bounds` + // returned a `NaN` high. + /// A named span endpoint, so each can be poisoned independently. + type Endpoint = (&'static str, fn(&mut Run, f64)); + let fields: [Endpoint; 2] = [ + ("fastest", |run, value| run.fastest_nanos_per_op = value), + ("slowest", |run, value| run.slowest_nanos_per_op = value), + ]; + for poison in [f64::NAN, f64::INFINITY, f64::NEG_INFINITY] { + for (field, set) in fields { + let mut numerator = sound; + set(&mut numerator, poison); + assert_eq!( + ratio_bounds(numerator, sound), + None, + "numerator.{field} = {poison} must not reach the division" + ); + + let mut denominator = sound; + set(&mut denominator, poison); + assert_eq!( + ratio_bounds(sound, denominator), + None, + "denominator.{field} = {poison} must not reach the division" + ); + + // The medians are still sound, so the point estimate is publishable + // and the *bound* is not. What must never appear is a bracket built + // from a poisoned endpoint. + for (label, rendered) in [ + ( + "numerator", + format_ratio_bounded(Some(numerator), Some(sound)), + ), + ( + "denominator", + format_ratio_bounded(Some(sound), Some(denominator)), + ), + ] { + assert!( + !rendered.contains('['), + "{label}.{field} = {poison} rendered an interval: {rendered:?}" + ); + } + } + } +} + +/// `> 0.0` admits infinity, and a finite slowest over it is a spread of zero. +/// +/// Zero is the reassuring end of the spread column, so this is the same class +/// of defect as the sentinel that reached the renderer earlier: a row that +/// measured nothing coherent reporting perfect stability. +#[test] +fn spread_rejects_non_finite_span_endpoints() { + let mut row = run(shapes::RESERVING_MPSC, 8, 1e8); + row.fastest_nanos_per_op = 10.0; + row.slowest_nanos_per_op = 13.0; + assert!(row.spread().is_some(), "the fixture must otherwise measure"); + + let mut infinite_fastest = row; + infinite_fastest.fastest_nanos_per_op = f64::INFINITY; + assert_eq!( + infinite_fastest.spread(), + None, + "an infinite fastest divides to a spread of zero, which reads as \ + perfect stability" + ); + + for poison in [f64::NAN, f64::INFINITY] { + let mut broken = row; + broken.slowest_nanos_per_op = poison; + assert_eq!(broken.spread(), None, "a {poison} slowest has no spread"); + + let mut broken = row; + broken.fastest_nanos_per_op = poison; + assert_eq!(broken.spread(), None, "a {poison} fastest has no spread"); + } +} + +/// At one producer the bound is exact, because the row is divided by itself. +/// +/// The report's first row is this case. Passing both spans to `ratio_bounds` +/// treats one measurement as two independent ones and manufactures an interval +/// around a quantity that is 1 by construction -- uncertainty invented by the +/// arithmetic rather than observed, in the most prominent row on the page. +#[test] +fn scaling_bounds_at_one_producer_is_exactly_one() { + // A deliberately wide span: if the identity case were not special-cased, + // this row would publish a correspondingly wide bound. + let observation = Observation { + isolated: vec![run_spanning(shapes::RESERVING_MPSC, 1, 4.0, 5.0, 6.0)], + drained: Vec::new(), + available_parallelism: Some(8), + }; + assert_eq!( + observation.scaling(&observation.isolated, shapes::RESERVING_MPSC, 1), + Some(1.0), + "the point estimate is one by construction" + ); + assert_eq!( + observation.scaling_bounds(&observation.isolated, shapes::RESERVING_MPSC, 1), + Some((1.0, 1.0)), + "and so is the bound; a wider one would be invented, not measured" + ); + + // The general case must keep its real bound, or this special case has + // simply broken the function. + let observation = Observation { + isolated: vec![ + run_spanning(shapes::RESERVING_MPSC, 1, 4.0, 5.0, 6.0), + run_spanning(shapes::RESERVING_MPSC, 8, 40.0, 50.0, 60.0), + ], + drained: Vec::new(), + available_parallelism: Some(8), + }; + let (low, high) = observation + .scaling_bounds(&observation.isolated, shapes::RESERVING_MPSC, 8) + .expect("both rows present"); + assert!( + low < high, + "a genuine comparison of two rows still spans an interval, got \ + [{low}, {high}]" + ); + + // A present-but-unmeasured one-producer row must not get the exact bound. + // `find` returns it, so the identity case would otherwise report perfect + // certainty about a shape that measured nothing. + for absent in [0.0, f64::NAN, f64::INFINITY] { + let mut row = run_spanning(shapes::RESERVING_MPSC, 1, 4.0, 5.0, 6.0); + row.nanos_per_op = absent; + let observation = Observation { + isolated: vec![row], + drained: Vec::new(), + available_parallelism: Some(8), + }; + assert_eq!( + observation.scaling_bounds(&observation.isolated, shapes::RESERVING_MPSC, 1), + None, + "a row reporting {absent} has no scaling to bound" + ); + } +} + +/// Either row being unmeasured is enough; it does not take both. +/// +/// Found by `cargo mutants`: replacing the `||` in that guard with `&&` survived +/// the suite, because every case written supplied the same row twice. A bound +/// over one real row and one that never ran is exactly the case the guard is +/// for, and nothing reached it. +#[test] +fn scaling_bounds_needs_both_rows_measured_not_merely_one() { + let measured = run_spanning(shapes::RESERVING_MPSC, 1, 4.0, 5.0, 6.0); + let mut absent = run_spanning(shapes::RESERVING_MPSC, 8, 40.0, 50.0, 60.0); + absent.nanos_per_op = 0.0; + + let one_ran = Observation { + isolated: vec![measured, absent], + drained: Vec::new(), + available_parallelism: Some(8), + }; + assert_eq!( + one_ran.scaling_bounds(&one_ran.isolated, shapes::RESERVING_MPSC, 8), + None, + "the many-producer row measured nothing, so there is no bound" + ); + + let mut absent_one = measured; + absent_one.nanos_per_op = 0.0; + let other_ran = Observation { + isolated: vec![ + absent_one, + run_spanning(shapes::RESERVING_MPSC, 8, 40.0, 50.0, 60.0), + ], + drained: Vec::new(), + available_parallelism: Some(8), + }; + assert_eq!( + other_ran.scaling_bounds(&other_ran.isolated, shapes::RESERVING_MPSC, 8), + None, + "and the one-producer row measuring nothing is equally disqualifying" + ); +} + +/// A non-finite bound is not printed, even beside a perfectly good point. +/// +/// Found by `cargo mutants`: the `low.is_finite() && high.is_finite()` guard +/// could be replaced with `true`, or its `&&` with `||`, and the suite stayed +/// green -- every case gave the bound a finite pair or no pair at all, so the +/// guard was never asked to reject one. The point estimate is still +/// publishable in that case; only the interval is not. +#[test] +fn format_scaling_bounded_drops_a_non_finite_interval_and_keeps_the_point() { + for (low, high) in [ + (f64::NAN, 2.0), + (1.5, f64::NAN), + (f64::NEG_INFINITY, 2.0), + (1.5, f64::INFINITY), + ] { + let rendered = format_scaling_bounded(Some(2.0), Some((low, high))); + assert_eq!( + rendered, "2.00x", + "[{low}, {high}] is not an interval, so only the point may be \ + published -- got {rendered:?}" + ); + } + + // And a finite pair must still be printed, or the guard has simply been + // turned into "never show an interval". + assert_eq!( + format_scaling_bounded(Some(2.0), Some((1.5, 2.5))), + "2.00x [1.50-2.50]" + ); +} + +/// The stop flag must be set on the path where nobody sets it explicitly. +/// +/// The drained timers cleared the flag on the line after their producer scope, +/// which a producer's assertion unwinds straight past -- leaving the consumer +/// spinning on a flag nobody would ever set, its handle dropped unjoined, and a +/// core burning for the life of the process. The success path was never in +/// doubt; this pins the failure path, which is the one that was broken. +/// +/// Note this asserts the *observable* effect through an `Arc` the guard does not +/// own, rather than reading the guard back: a consumer sees the flag through +/// exactly such a clone. +#[test] +fn stop_on_drop_sets_the_flag_when_the_producer_phase_unwinds() { + let flag = Arc::new(AtomicBool::new(false)); + + // The success path, for contrast. + let observed = Arc::clone(&flag); + { + let _stop = StopOnDrop(Arc::clone(&flag)); + assert!( + !observed.load(Ordering::Relaxed), + "the flag must stay clear while the guard is alive, or a consumer \ + would stop before the producers had finished" + ); + } + assert!( + observed.load(Ordering::Relaxed), + "a normal drop must stop it" + ); + + // The path that was broken. `catch_unwind` prints the panic to stderr; no + // panic hook is installed to silence it, because a hook is process-global + // and this suite runs its tests as threads in one process. + let unwound = Arc::new(AtomicBool::new(false)); + let observed = Arc::clone(&unwound); + let result = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| { + let _stop = StopOnDrop(Arc::clone(&unwound)); + panic!("a producer failed, as one does when the consumer is gone"); + })); + assert!(result.is_err(), "the fixture must actually unwind"); + assert!( + observed.load(Ordering::Relaxed), + "the consumer would spin forever on a flag nobody sets" + ); +} + +/// Arrives at a gate on its own thread and hands back somewhere to read the +/// answer. +/// +/// **Every gate test goes through this rather than calling `arrive_and_wait` +/// directly.** A gate regression parks its caller forever, so an assertion made +/// on the test thread would hang the whole suite -- which runs its tests as +/// threads in one process -- instead of failing it. The spawned thread is +/// deliberately never joined, for the same reason. +/// +/// Two of these tests were first written the direct way, and sabotage caught +/// both: neutering `release` wedged the run rather than reddening it. +fn spawn_arrival(gate: &Arc) -> Arc>> { + let gate = Arc::clone(gate); + let outcome = Arc::new(Mutex::new(None)); + let observed = Arc::clone(&outcome); + thread::spawn(move || { + let complete = gate.arrive_and_wait(); + *observed.lock().expect("not poisoned") = Some(complete); + }); + outcome +} + +/// The answer from [`spawn_arrival`], or `None` if the party never came back. +fn await_arrival(outcome: &Arc>>) -> Option { + let deadline = Instant::now() + Duration::from_secs(5); + loop { + if let Some(complete) = *outcome.lock().expect("not poisoned") { + return Some(complete); + } + if Instant::now() >= deadline { + return None; + } + thread::sleep(Duration::from_millis(10)); + } +} + +/// A complete party opens the gate and every member learns that it was complete. +#[test] +fn a_complete_party_opens_the_gate_for_everyone() { + let gate = StartGate::new(3); + let arrivals: Vec<_> = (0..4).map(|_| spawn_arrival(&gate)).collect(); + for outcome in &arrivals { + assert_eq!( + await_arrival(outcome), + Some(true), + "every member of a complete party must be freed, and told so" + ); + } +} + +/// A released gate frees parties waiting for arrivals that will never come. +/// +/// This is the deadlock `M4.6` was queued for, reproduced without needing the +/// OS to refuse a thread: the gate is sized for a party that never completes, +/// which is what a panicking `Scope::spawn` leaves behind. +#[test] +fn a_released_gate_frees_parties_that_will_never_be_completed() { + let gate = StartGate::new(2); + let parked = spawn_arrival(&gate); + + // Let it reach the gate, then give up on the members that never arrive. + thread::sleep(Duration::from_millis(50)); + gate.release(); + + assert_eq!( + await_arrival(&parked), + Some(false), + "the parked party must be freed and told the party was incomplete, so \ + a worker knows its run was abandoned" + ); +} + +/// Arriving at an already-released gate reports the party incomplete. +/// +/// The ordering matters: a worker spawned before the failure may arrive after +/// the coordinator has given up, and must reach the same conclusion as one that +/// was already parked. +#[test] +fn arriving_after_a_release_still_reports_an_incomplete_party() { + let gate = StartGate::new(4); + gate.release(); + for _ in 0..2 { + assert_eq!( + await_arrival(&spawn_arrival(&gate)), + Some(false), + "a late arrival must not be told the party completed, and the \ + answer must not change on a second look" + ); + } +} + +/// The guard releases the gate while its scope unwinds. +#[test] +fn release_on_drop_frees_the_gate_when_spawning_panics() { + let gate = StartGate::new(8); + let parked = spawn_arrival(&gate); + thread::sleep(Duration::from_millis(50)); + + let result = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| { + let _release = ReleaseOnDrop(Arc::clone(&gate)); + panic!("the OS refused a thread, as `Scope::spawn` does by panicking"); + })); + assert!(result.is_err(), "the fixture must actually unwind"); + + assert_eq!( + await_arrival(&parked), + Some(false), + "the guard did not release the gate as its scope unwound" + ); +} + +/// A producer does not start timing until the consumer says it is draining. +/// +/// This is the window `M4.3` closed: the gate proves the consumer exists and is +/// scheduled, not that it has reached its first `pop`, so a producer released by +/// the gate could push into a queue nobody was draining yet -- an undrained +/// opening to a run whose whole subject is that it is drained. +/// +/// Asserted by holding the flag clear and showing the producer stays put, then +/// setting it and showing the producer moves. A test that only set the flag +/// first would pass against a missing handshake, which is the shape that has +/// slipped through on this branch before. +#[test] +fn a_producer_waits_for_the_consumer_to_announce_that_it_is_draining() { + let ready = Arc::new(AtomicBool::new(false)); + let waited = Arc::new(AtomicBool::new(false)); + + let consumer_ready = Arc::clone(&ready); + let observed = Arc::clone(&waited); + // Detached rather than joined: a regression leaves this parked forever, and + // joining would hang the suite instead of failing it. + thread::spawn(move || { + await_consumer(&consumer_ready); + observed.store(true, Ordering::Release); + }); + + // While the consumer has not announced itself, the producer must not pass. + thread::sleep(Duration::from_millis(100)); + assert!( + !waited.load(Ordering::Acquire), + "a producer started before the consumer was draining" + ); + + ready.store(true, Ordering::Release); + let deadline = Instant::now() + Duration::from_secs(5); + while !waited.load(Ordering::Acquire) { + assert!( + Instant::now() < deadline, + "the producer never observed the consumer's announcement" + ); + thread::sleep(Duration::from_millis(10)); + } +} + +/// The handshake drains before it announces, not after. +/// +/// The test above proves a producer waits for the announcement. It says nothing +/// about what the announcement means, so it would pass just as happily if the +/// consumer announced first and drained second -- and that ordering is the whole +/// of `M4.3`. Announcing first makes the flag mean "about to drain", which a +/// descheduling between the store and the first `pop` falsifies; draining first +/// makes it mean "has executed the pop path", which nothing can. +/// +/// Reachable only because [`drain_then_announce`] states the ordering once. The +/// four drained timers that use it cannot be tested directly -- running one runs +/// the whole probe -- so a hand-written `pop`-then-`store` in each was four +/// copies of a guarantee nothing could check. +/// +/// The fake records what the flag said *at the moment the pop ran*. If the store +/// had already happened, it sees `true`. +#[test] +fn the_handshake_drains_before_it_announces() { + let ready = AtomicBool::new(false); + let already_announced = AtomicBool::new(false); + + drain_then_announce( + || already_announced.store(ready.load(Ordering::Acquire), Ordering::Release), + &ready, + ); + + assert!( + !already_announced.load(Ordering::Acquire), + "readiness was published before the consumer drained, so a producer \ + released by it can push into a queue whose consumer has not run -- the \ + undrained opening M4.3 closed, in its narrower form" + ); + assert!( + ready.load(Ordering::Acquire), + "the handshake drained but never announced, so every producer would spin \ + forever in await_consumer" + ); +} diff --git a/crates/windows-platform-probes/src/report.rs b/crates/windows-platform-probes/src/report.rs index 781e09834..fbbec13b0 100644 --- a/crates/windows-platform-probes/src/report.rs +++ b/crates/windows-platform-probes/src/report.rs @@ -92,13 +92,22 @@ impl Captured { /// A [`Report`] a renderer can `writeln!` into directly. /// -/// is arithmetic. Every renderer writes through `writeln!(out, ...)` against a -/// `String`, at **332 sites** across this crate; a sink method taking +/// This is the answer to "how does a formatted line reach the sink", and the +/// reason it is a [`std::fmt::Write`] adapter rather than a method on [`Report`] +/// is a property rather than a count. Every renderer already wrote through +/// `writeln!(out, ...)` against a `String`; a sink method taking /// `fmt::Arguments` would have been explicit but would have rewritten every one -/// of them, while `String` already implements `fmt::Write`, so a sink that does -/// too lets those sites stand untouched and moves only 18 renderer signatures. -/// signatures. The recorded reasoning is in -/// [DESIGN-NOTES.md](../DESIGN-NOTES.md#d-streaming-report). +/// of those sites, while `String` already implements `fmt::Write` -- so a sink +/// that does too lets every write site stand untouched and moves only the +/// renderer signatures, which now take `&mut dyn fmt::Write` and are handed a +/// [`LineSink`] by `emit_report_to`. +/// +/// The census that decided it is recorded in +/// [DESIGN-NOTES.md](../DESIGN-NOTES.md#d-streaming-report) rather than repeated +/// here, because a count in prose is a copy somebody has to keep true. This +/// passage carried one, and the edit that revised it is the same edit that +/// dropped the sentence above and left this comment reading "is arithmetic" +/// with nothing before it. /// /// # Lines are reassembled here, because `fmt::Write` does not speak in them /// diff --git a/crates/windows-topology-sys/DESIGN-NOTES.md b/crates/windows-topology-sys/DESIGN-NOTES.md index 3138f0229..47e95a31c 100644 --- a/crates/windows-topology-sys/DESIGN-NOTES.md +++ b/crates/windows-topology-sys/DESIGN-NOTES.md @@ -54,7 +54,7 @@ additional CPU and memory cost; do not solve the consumer's architecture for the Three ways to obtain a `MachineMemoryTopology` are supported on purpose, and the crate's own front page advertises the third: "deserialize one from JSON written for a machine you do not have". That is a feature -- it is how a consumer tests against hardware it lacks, and this workspace needs it right now, because -`probe-core-affinity` must exercise NUMA selection logic on hosts that have exactly one NUMA node. +`placement-probe` must exercise NUMA selection logic on hosts that have exactly one NUMA node. The hazard is that **the resulting value looked exactly like a discovered one**. There is a passing test in this crate that parses a *Linux-shaped* description, complete with an ACPI SLIT-style distance diff --git a/crates/windows-waitable-queues/Cargo.toml b/crates/windows-waitable-queues/Cargo.toml index f77ddb42e..35c35fd2c 100644 --- a/crates/windows-waitable-queues/Cargo.toml +++ b/crates/windows-waitable-queues/Cargo.toml @@ -57,10 +57,15 @@ experimental-permit-claim = [] # while still compiling. With them off, `AtomicU128` does not exist on such a # target and the build fails naming it. # -# Most callers should not need this. `Perpetual` reaches roughly twenty years -# before its claim position recurs, on a plain `AtomicU64` at no measured cost, -# whereas the 128-bit exchange measured 2-3x slower on the claim itself. See -# `ClaimLayout` for the comparison. +# This feature adds only the `Wide` layout. The whole push path was measured as +# slower under it at every producer count measured -- smallest at one or two, several times by +# thirty-two, in the isolated regime. The probe times the complete push, so that +# is the layout's effect on that path, not the exchange in isolation. +# +# `Perpetual` needs no feature: it reaches roughly twenty years before its claim +# position recurs on a plain `AtomicU64`, and what THAT costs in throughput is not +# established -- a separate question from the figures above, which are `Wide`'s. +# See `ClaimLayout` for both. dwcas = ["dep:portable-atomic"] [lib] diff --git a/crates/windows-waitable-queues/DESIGN-NOTES.md b/crates/windows-waitable-queues/DESIGN-NOTES.md index b3360cd1e..94d477814 100644 --- a/crates/windows-waitable-queues/DESIGN-NOTES.md +++ b/crates/windows-waitable-queues/DESIGN-NOTES.md @@ -51,7 +51,7 @@ preferred. | D-14 | **`slotwise_mpsc`'s arming asks "would `pop` find something", not "is `len` zero".** The two disagree over a slot a producer has claimed but not published, and only the first answer lets the consumer park on it instead of spinning until that producer is rescheduled. | | D-15 | **`Doorbell::clear` resets the event *before* clearing the flag that mirrors it, and the original order was a lost wakeup.** A producer signalling between the two lines set the flag and issued a real `SetEvent`; the `ResetEvent` that followed erased the signal and left the flag set, wedging the doorbell dark while it claimed to be lit. **Amends [D-9](#d-9)**, whose "there is no third case" holds only for a queue whose emptiness is one position comparison. | | D-16 | **Its cost premise is falsified by [D-26](#d-26); the conclusion stands on capability instead -- see [D-29](#d-29).** Reservation is a capability a shape may lack, so the reserving multi-producer queue ships as a peer of `slotwise_mpsc` rather than replacing it. Honouring a reservation requires counting free slots, which requires the consumer's position -- a single shared line `slotwise_mpsc`'s push deliberately never reads. The original rationale added that this made reserving the *more expensive* shape and that both should ship rather than charge every caller for it; measurement reversed that, and the split is now justified by the capability alone. **Amends [D-6](#d-6)**, which assumed one queue would carry every policy. | -| D-17 | **The reservation count and the claim position live in one word, because a check-and-claim over both must be a single atomic operation.** Two atomics cannot be made correct with any amount of fencing: the pushing producer is load-then-store and the reserving one store-then-load, so the Dekker argument does not apply and both can miss each other. The 32/32 split is forced by the arithmetic, and caps this shape at 2^31 items. | +| D-17 | **Partly superseded by [D-41](#d-41): the 32/32 split is no longer forced, and the capacity ceiling is `Balanced`'s rather than the shape's.** **The reservation count and the claim position live in one word, because a check-and-claim over both must be a single atomic operation.** Two atomics cannot be made correct with any amount of fencing: the pushing producer is load-then-store and the reserving one store-then-load, so the Dekker argument does not apply and both can miss each other. The 32/32 split is forced *given the premise D-41 dropped* -- that the count must be able to reach the whole capacity -- and under that split the shape caps at 2^31 items on a 64-bit target, 2^30 on a 32-bit one. | | D-18 | **Superseded by [D-37](#d-37), which adopts a 128-bit compare-and-swap for a separate wide shape.** Retained because its analysis of the *costs* is still correct and D-37 depends on it; what changed is that those costs are now paid by a **separate shape** rather than imposed on this one. **Originally: a 128-bit compare-and-swap is refused.** **Amended once before being superseded, because three of the four reasons originally given were wrong or incomplete, and the decisive one was missing.** It would *not* lift the cap "and nothing else": a 64-bit position also collapses SH-14.1's ABA recurrence, which was unknown when this was written. It is *not* outside the x86-64 baseline -- `rustc 1.98.0` emits `target_feature="cmpxchg16b"` for `x86_64-pc-windows-msvc`, so there is no floor to raise and no runtime detection to pay. What stands is the dependency (`AtomicU128` is still unstable, rust-lang/rust#99069) and, decisively, that **`i686-pc-windows-msvc` has no 128-bit atomic at all**: adopting this is not "widen the word" but "widen the word *and* drop 32-bit support". Revisit for a tagged pointer, or if 32-bit support is dropped for other reasons -- not before. | | D-19 | **The coalesced loss latch is deliberately not generalised from the file watcher.** Coalescing there is sound because a desync is *idempotent* -- two mean the same as one, and the answer to both is a re-scan. A queue of arbitrary `T` has no such property, so what generalises is a loss *count*, which is M31.4's observability rather than a policy. | | D-20 | **Undrained items are handed to a caller-supplied sink at teardown, and the sink is chosen at construction because `Drop` has nowhere to hand them back to.** Without one they are destroyed on whichever thread released the last handle -- which may be a pool callback that must not block, and closing a handle to a dead network path can block for a long time. The default is unchanged; what changes is that it is now a named choice. | @@ -60,22 +60,22 @@ preferred. | D-23 | **High-water tracking is opt-in at construction; refusals and doorbell rings are always on.** The difference is where each can be paid for: refusals sit on the failure path and rings on a path that already costs a syscall, but a peak has to observe *every* change -- and on `slotwise_mpsc` that means the producer reading the consumer's position, the shared line [D-16](#d-16) built a separate shape to avoid. Untracked reports `None`, not `0`. | | D-24 | **Counting the doorbell's rings turns the skip optimisation into part of the observable contract, and that is the point rather than a side effect.** R9 asks for the count precisely so "disabling the skip must change the number" -- so the sabotage entry for removing the skip changed from a control expecting `survives` to a defect expecting `caught`. An optimisation nobody can measure is an assumption. | | D-25 | **`Observable` deliberately does not restate depth.** [D-2](#d-2)'s sketch listed it, but `Bounded::len` already reports it from positions the queue keeps anyway. Naming it twice would give one number two spellings and two places to drift. What belongs on `Observable` is only what must be *accumulated*. | -| D-26 | **Measured: the tail claim contends badly, and `reserving_mpsc` is up to 4x FASTER than `slotwise_mpsc` under contention -- the opposite of what [D-16](#d-16) assumed.** Aggregate throughput *falls* as producers are added, for both shapes and far more than a bare contended atomic explains. D-16's premise, that reading the consumer's position makes the reserving shape the expensive one, is falsified everywhere except a single producer with a live consumer. | -| D-27 | **The gap is intrinsic to Vyukov's sequence protocol, not a fixable flaw in `slotwise_mpsc`'s retry loop.** Its producer must read a slot's sequence *before* claiming, and that slot marches through memory as the tail advances while other producers write it. Padding slots onto their own cache lines was tested and rejected: it recovers about a fifth at eight producers, for four times the memory, and leaves the shape still 2.8x slower. | +| D-26 | **Measured: the tail claim contends badly, and `reserving_mpsc` is FASTER than `slotwise_mpsc` under contention -- the opposite of what [D-16](#d-16) assumed.** Aggregate throughput *falls* as producers are added, for both shapes and far more than a bare contended atomic explains. D-16's premise, that reading the consumer's position makes the reserving shape the expensive one, is falsified everywhere except a single producer with a live consumer. **The magnitude originally stated here -- "up to 4x" -- predates a correction to the probe's timing window and is withdrawn**; per [D-29](#d-29) no current public figure derives from that capture. The direction survives the correction; the magnitude does not. | +| D-27 | **Mechanism not established; see the section, which is marked historical.** **Originally: the gap is intrinsic to Vyukov's sequence protocol, not a fixable flaw in `slotwise_mpsc`'s retry loop.** Its producer must read a slot's sequence *before* claiming, and that slot marches through memory as the tail advances while other producers write it. Padding slots onto their own cache lines was tested and rejected: it recovers about a fifth at eight producers, for four times the memory, and leaves the shape still 2.8x slower. | | D-28 | **Amended -- the blanket rejection is withdrawn; the verdict depends on thread placement, and the open question is an open question queued outside this crate.** Caching the peer's index was measured, and it engaged as designed. It cost ~1.8x on x64 with the threads across cores, and *won* 17x on ARM64 and 1.8x on x64 SMT siblings. Batch depth decides the sign, and batch depth is set by where the two threads are scheduled -- not by the architecture and not by our code. A prefetch-only "warming" control changed nothing on any host. | -| D-29 | **Both multi-producer shapes ship. The crate publishes what it measured and declines to choose for the caller.** [D-26](#d-26) falsified [D-16](#d-16)'s cost premise, which reopened merge-or-delete; the answer is neither. Vyukov's sequence protocol and the head-based one are independently researched designs, both in production use, and our own workload having settled which *we* want is not evidence about anyone else's. Deleting a shape because no visible consumer wants it is what PLATFORM INTEGRITY forbids. What the crate owes instead is the data and, through `probe-core-affinity`, the means to gather it on the caller's own hardware. | +| D-29 | **Both multi-producer shapes ship. The crate publishes what it measured and declines to choose for the caller.** [D-26](#d-26) falsified [D-16](#d-16)'s cost premise, which reopened merge-or-delete; the answer is neither. Vyukov's sequence protocol and the head-based one are independently researched designs, both in production use, and our own workload having settled which *we* want is not evidence about anyone else's. Deleting a shape because no visible consumer wants it is what PLATFORM INTEGRITY forbids. What the crate owes instead is the data and, through `placement-probe`, the means to gather it on the caller's own hardware. | | D-30 | **Both MPSC shapes are qualified by name; neither is `mpsc`.** A bare `mpsc` beside `reserving_mpsc` makes one canonical by implication, which contradicts this crate's own "no shape is the canonical one" and, after [D-29](#d-29), is simply false. `slotwise_mpsc` names its claim protocol -- it claims slot by slot, with no shared counter -- and avoids the reading `sequence_mpsc` invites, that it alone preserves FIFO order when both shapes do. Renamed before first publish, where it is free. | | D-31 | **0.1.0 ships without machine-checked memory orderings, and says so in its own documentation.** Model-checking gates 1.0, not 0.1.0. It would close the *demonstrated* gap -- a weakened `Acquire` survives the whole suite -- but not the dangerous one: it cannot model `SetEvent`/`ResetEvent`, so it cannot cover the doorbell, and [D-15](#d-15)'s lost wakeup, the only ordering bug this crate has had, was found by sabotage instead. The risk it addresses is mostly regression risk, which is lowest before there are consumers. The disclosure, not the deferral, is the decision. | | D-32 | **`Reserving::Reservation<'a>` gains a bound, before the crate publishes.** The associated type is currently unbounded, so a caller generic over the trait can claim a slot and drop it but never redeem it -- the trait cannot express the operation it exists for. Both implementors already have identical `send` and `is_disconnected` signatures, so the bound is additive; adding it after publication is a breaking change to every implementor. Done as SH-1.5: the [`Claim`](src/traits.rs) trait carries `send` and `is_disconnected`, and both reservation types implement it as forwarders. `Claim` must be in scope to call those methods on a claim whose concrete type the caller has not named, which is why it is re-exported at the crate root. | | D-33 | **`PushError` is `#[non_exhaustive]`, and the one-directional doorbell is disclosed rather than fixed before 0.1.0.** The receive-side errors already carried the attribute and the send side lacked it by omission; adding it after publication is itself breaking, so it is taken now while the crate has no external consumers. Whether a producer can *wait* for room stays open as M32.3 -- it is additive, so it does not gate the release -- but the absence is stated in both the crate docs and the README, because `crossbeam-channel`'s `send` blocks and a reader arriving from it will assume this one does too. | | D-34 | **Every bounded queue surveyed is ABA-safe for one of two reasons, and this crate's `reserving_mpsc` has neither.** Either the claim counter is a whole machine word, so recurrence is unreachable -- crossbeam, concurrent-queue, thingbuf, Vyukov, SCQ's `Head`/`Tail` -- or the authorizing compare-exchange is moved onto the cell, so the decision and the write are validated together (CRQ, SCQ). Ours packs the position into a 32-bit *subfield* and authorizes with an exchange that does not cover the separately-read `head`. Nikolaev (DISC 2019, section 3) states the width assumption the field relies on and states it for **CPU-word** width, which a subfield does not satisfy; DPDK's `rte_ring` is the same protocol as ours and its published justification covers modular arithmetic only. The generalisation -- ours, unstated in any source -- is that **the atomic operation authorizing the write must cover everything the decision depended on.** Survey in [DESIGN-SESSION-2026-09-02](design-sessions/DESIGN-SESSION-2026-09-02-claim-protocol-prior-art.md); the fix is the claim-protocol replacement recorded there. | -| D-35 | **Measured: the permit claim is 2.7x faster than `reserving_mpsc` at 16-32 producers, and 1.45x slower at one.** The safer claim is also the faster one everywhere contention exists, which was not the expected result -- it touches *two* shared lines where the shipping shape touches one plus a read, and [D-26](#d-26) had established that the shared line is what collapses. The mechanism is that both of its operations are unconditional read-modify-writes that never retry, where the shipping shape's compare-exchange retries once per lost race; the retries dominate long before the second line does. It is the only shape measured that gets *faster* per push as producers are added (42.8 ns at two to 19.5 at thirty-two) and the only one that stays within 1.5x of a bare contended `fetch_add`. **This decides the shape of the fix but not the fix**: the drained regime's refusal counts differ by orders of magnitude in a way this harness cannot attribute, which SH-15.5.1 exists to settle before SH-15.6 adopts anything. | +| D-35 | **The magnitudes below predate the timing correction and are retained as the record of what the measurement showed, not as current figures.** **Measured: the permit claim is faster than `reserving_mpsc` at high producer counts and slower at one** -- originally recorded as 2.7x and 1.45x respectively, from a capture taken before the probe's timing window was corrected; per [D-29](#d-29) no current public figure derives from that capture. The direction survives the correction; the magnitudes do not. The safer claim is also the faster one everywhere contention exists, which was not the expected result -- it touches *two* shared lines where the shipping shape touches one plus a read, and [D-26](#d-26) had established that the shared line is what collapses. The mechanism is that both of its operations are unconditional read-modify-writes that never retry, where the shipping shape's compare-exchange retries once per lost race; the retries dominate long before the second line does. It is the only shape measured that gets *faster* per push as producers are added (42.8 ns at two to 19.5 at thirty-two) and the only one that stays within 1.5x of a bare contended `fetch_add`. **This decides the shape of the fix but not the fix**: the drained regime's refusal counts differ by orders of magnitude in a way this harness cannot attribute, which SH-15.5.1 exists to settle before SH-15.6 adopts anything. | | D-36 | **Superseded by [D-41](#d-41): the hazard is now a layout choice, not a defect that must ship.** The reasoning below stands as the record of why it was right to disclose rather than delay while the only known fix was the claim-protocol replacement. **0.1.0 ships SH-14.1 disclosed rather than fixed, and the disclosure is a release blocker.** Following [D-31](#d-31)'s principle -- the disclosure, not the deferral, is the decision -- because the fix is a claim-protocol replacement ([D-35](#d-35)) whose adoption is still gated on an open question, and holding the release for it would trade a *documented* hazard for an undocumented rush. **The two gaps are not equally forgiving and the text says so**: an unverified ordering is a risk of a bug, this is a known one with a computed exposure, and its failure mode is silent -- no error, panic, or counter -- so a caller can neither detect nor mitigate it. That is precisely why it may not ship in silence. Stated in the crate docs, the README, and the shape's own module docs, each leading with **"on every target, not only 32-bit ones"**, because the natural spelling "32-bit position" invites the opposite reading and SH-6.1 already had to be corrected for exactly that. The shape-selection guidance in both documents was also amended: it previously said "start with `reserving_mpsc`" with no caveat, pointing callers at the hazardous shape by default. | -| D-37 | **Partly superseded by [D-41](#d-41): the wide word ships as a *layout* behind the non-default `dwcas` feature, not as a separate `reserving_mpsc_wide` shape, and the gate is the feature rather than the target.** What stands is the reasoning below about `portable-atomic`: `default-features = false` is load-bearing, because with defaults on it silently substitutes a global lock, and D-7's burden of proof is discharged rather than waived. What does not is the shape's name and the premise that the narrow word must keep SH-14.1 -- re-apportioning the narrow word removes the exposure for free, so the wide word is no longer the only way out. **The reserving claim word ships in two widths: the narrow one on every target, the wide one only where a 128-bit exchange is genuinely lock-free.** `reserving_mpsc` keeps its packed 64-bit word, keeps SH-14.1's hazard, keeps [D-36](#d-36)'s warnings, and is **never silently swapped** for the wide shape on targets that could host one -- a contract that changes with the target is what PLATFORM INTEGRITY rule 2 forbids, and a caller who read "2^32" must get 2^32. `reserving_mpsc_wide` is the same protocol with a `u128` word split 64/64: recurrence needs 2^64 pushes, and the capacity ceiling rises to 2^62. **The gate is one line of `Cargo.toml`: `default-features = false`.** Measured, not designed -- with the default feature set `portable-atomic` compiles on i686 and silently substitutes a global lock, but with defaults off `AtomicU128` **does not exist** there (`no AtomicU128 in the root`), nor on x86_64 built without `cmpxchg16b`. It exists exactly where a native lock-free exchange is guaranteed at compile time, so the `use` statement is the gate and it fails loudly. A `cfg(target_has_atomic = "128")` would be the *wrong* gate -- it is emitted even with `cmpxchg16b` disabled -- and a `const` assertion on `is_always_lock_free()`, though genuinely const-evaluable, is redundant where the type exists and unreachable where it does not. That is the standard SH-14.2 already set when it probed i686 to confirm `AtomicU64` was lock-free before widening `slotwise_mpsc`, recording that a hidden mutex "would have made this a bad trade". [D-7](#d-7)'s burden of proof for adding a Cargo feature is **discharged, not waived**: D-7 rejected feature-gating because the only benefit was compile time, and the cost here is a third-party dependency, which dead-code elimination does not remove from `Cargo.lock` or from an auditor's review. | +| D-37 | **Partly superseded by [D-41](#d-41): the wide word ships as a *layout* behind the non-default `dwcas` feature, not as a separate `reserving_mpsc_wide` shape, and the gate is the feature rather than the target.** What stands is the reasoning below about `portable-atomic`: `default-features = false` is load-bearing, because with defaults on it silently substitutes a global lock, and D-7's burden of proof is discharged rather than waived. What does not is the shape's name and the premise that the narrow word must keep SH-14.1 -- re-apportioning the narrow word moves the exposure to a later recurrence (`2^48` under `Enduring`, `2^56` under `Perpetual`) without a third-party dependency -- what it costs in throughput is unestablished, see [D-41](#d-41) -- so the wide word is no longer the only way out. **The reserving claim word ships in two widths: the narrow one on every target, the wide one only where a 128-bit exchange is genuinely lock-free.** `reserving_mpsc` keeps its packed 64-bit word, keeps SH-14.1's hazard, keeps [D-36](#d-36)'s warnings, and is **never silently swapped** for the wide shape on targets that could host one -- a contract that changes with the target is what PLATFORM INTEGRITY rule 2 forbids, and a caller who read "2^32" must get 2^32. `reserving_mpsc_wide` is the same protocol with a `u128` word split 64/64: recurrence needs 2^64 pushes, and the capacity ceiling rises to 2^62. **The gate is one line of `Cargo.toml`: `default-features = false`.** Measured, not designed -- with the default feature set `portable-atomic` compiles on i686 and silently substitutes a global lock, but with defaults off `AtomicU128` **does not exist** there (`no AtomicU128 in the root`), nor on x86_64 built without `cmpxchg16b`. It exists exactly where a native lock-free exchange is guaranteed at compile time, so the `use` statement is the gate and it fails loudly. A `cfg(target_has_atomic = "128")` would be the *wrong* gate -- it is emitted even with `cmpxchg16b` disabled -- and a `const` assertion on `is_always_lock_free()`, though genuinely const-evaluable, is redundant where the type exists and unreachable where it does not. That is the standard SH-14.2 already set when it probed i686 to confirm `AtomicU64` was lock-free before widening `slotwise_mpsc`, recording that a hidden mutex "would have made this a bad trade". [D-7](#d-7)'s burden of proof for adding a Cargo feature is **discharged, not waived**: D-7 rejected feature-gating because the only benefit was compile time, and the cost here is a third-party dependency, which dead-code elimination does not remove from `Cargo.lock` or from an auditor's review. | | D-38 | **One atomic, one discipline: an atomic that carries any acquire/release operation has acquire/release on *every* operation, and a relaxed load is never mixed in.** A relaxed operation is still **atomic** -- indivisible, untorn, and free of data-race UB -- but it is *unordered*: it behaves like a plain load or store with respect to placement, unanchored relative to the ordered operations on the same object and free to be moved by the optimizer or the processor. It is not pinned to its textual site, so reasoning about it in statement order is nonsense. **The two axes are independent, and conflating them is the mistake in both directions:** relaxed does not mean "no guarantees" (see [D-40](#d-40) -- the atomicity is often the whole point), and it does not mean "ordered but weakly". The failure mode is that it usually does what the source appears to say, until a change of code generator or a weaker processor makes it not; on x86-64 TSO a decorative `Acquire` and a `Relaxed` load emit near-identical code, so a test suite on this host cannot see the difference at all -- which is the same blindness [D-31](#d-31) measured. **When the two resolutions differ, promote the load.** The exception is a reference count ([D-39](#d-39)), and it is an exception for a stated reason rather than by convention. | | D-39 | **The reference counts keep a relaxed increment against an `AcqRel` decrement, and that is the one sanctioned departure from [D-38](#d-38).** It is sanctioned because *no dependent memory is read on the strength of the relaxed increment*: a thread incrementing `producers` already holds a handle, so it needs no edge to learn the object exists, and the cache-coherence effect an acquire would buy is not required until the count reaches zero -- at which point the `AcqRel` decrement supplies it. That is the same argument `std::sync::Arc` makes, and it is a property of what the count is used for, not a general licence. A count whose value ever decided whether to *dereference* something would not qualify. | | D-40 | **A relaxed atomic is chosen for its *atomicity*, and dropping to a plain field is never the way to "simplify" one.** [D-38](#d-38) says a relaxed operation is unordered; it is still indivisible, and that is frequently the entire reason the field is an atomic at all. Without it the implementation may synthesise a wide access out of narrower ones and observe a **torn** value, and concurrent access to a non-atomic field is a data race and therefore UB regardless. `reserving_mpsc`'s claim word is the worked example: it packs `reserved` and `position` into one `u64` and is uniformly relaxed ([D-38](#d-38)), yet a torn read would yield a pair that never existed as a state and would break the compare-and-swap protocol outright. On `i686-pc-windows-msvc` -- which [D-18](#d-18) deliberately keeps supported -- that load must be a `cmpxchg8b` or an 8-byte SSE load, *more* expensive than the two `mov`s a plain `u64` would get, and the compiler is obliged to pay it for exactly this reason. **Relaxed is a statement about ordering only; it is never a step toward removing the atomic.** | -| D-41 | **The claim word's apportionment is a caller's choice, and the recurrence behind SH-14.1 is a number the caller sets rather than one this crate imposes.** Supersedes [D-36](#d-36), whose premise was that the only fix was the [D-35](#d-35) claim-protocol replacement, gated on an open question -- so disclosing beat delaying. That premise was false, and measurement is what showed it: the 32/32 split followed from requiring the reservation half to hold the *entire capacity*, because every slot could be reserved at once. Capping outstanding reservations instead leaves the capacity bounded only by the ring, and the position is free to take 48 or 56 bits. `Balanced` (32/32), `Enduring` (16/48) and `Perpetual` (8/56) issue the **same** `lock cmpxchg` on the same `u64`, differing only in shift constants, and measured indistinguishable outside noise -- so the recurrence moves from about 37 seconds to about 20 years for no throughput and no dependency. The reservation half was the wrong half to spend bits on: it held 2^32 where the real bound is however many producers are mid-send. `Wide` (64/64 over a `u128`) is offered behind the non-default `dwcas` feature, because it is the one thing here that costs a third-party crate -- the standard library has no 128-bit atomic -- and it measured 2-3x slower on the claim; it buys a guarantee rather than a lifetime argument. **The default stays `Balanced`** so introducing the choice changed no existing caller's behaviour, and it is documented as *not* the recommended layout: leaving it because it is the status quo would preserve the hazard by inertia. | +| D-41 | **The claim word's apportionment is a caller's choice, and the recurrence behind SH-14.1 is a number the caller sets rather than one this crate imposes.** Supersedes [D-36](#d-36), whose premise was that the only fix was the [D-35](#d-35) claim-protocol replacement, gated on an open question -- so disclosing beat delaying. That premise was false, and measurement is what showed it: the 32/32 split followed from requiring the reservation half to hold the *entire capacity*, because every slot could be reserved at once. Capping outstanding reservations instead leaves the capacity bounded only by the ring, and the position is free to take 48 or 56 bits. `Balanced` (32/32), `Enduring` (16/48) and `Perpetual` (8/56) issue the **same** atomic compare-exchange on the same `u64`, differing only in shift and mask constants -- so the recurrence moves from about 37 seconds to about 20 years without a dependency. **An earlier version of this row added "and measured indistinguishable outside noise", and that clause is withdrawn**: it rested on a single probe run read against a noise floor of 2-6% that seven runs put at 7-61%. Re-measured, the deeper layouts are indistinguishable from `Balanced` at low producer counts, and at high counts sit outside the same-code control but too close to it to establish an ordering or a cost on this host: a flag to measure locally, not a finding. The figures are in the probe's note rather than restated here. The layout is the caller's choice and the throughput question belongs on the caller's hardware; see the queue-contention section of [DESIGN-NOTES.md](../windows-platform-probes/DESIGN-NOTES.md#d-queue-layout-observations). Which half the bits go to is the trade this decision exposes rather than settles: a 32-bit reservation field, whose count the shipping type caps at `u32::MAX` and whose achievable value capacity caps lower still, against a position whose width sets the recurrence horizon. The bound that matters on the reservation side is how many reservations a caller holds at once -- the lesser of the ring capacity and the field, reachable by one producer alone, since `reserve` takes `&self`. Which of the two a deployment needs is the deployment's question. **An earlier version of this row said the real bound was "however many producers are mid-send", and that is withdrawn as false**: one producer fills `Perpetual`'s 255 in a loop and is then refused, which `one_producer_alone_can_exhaust_the_reservation_field` pins. The correction matters because the false premise made the narrower fields look unreachable, which is the argument for spending bits on the position. `Wide` (64/64 over a `u128`) is offered behind the non-default `dwcas` feature, because it is the one thing here that costs a third-party crate -- the standard library has no 128-bit atomic -- and the whole push path was measured as slower under it at every producer count measured in the isolated regime -- smallest at one or two, several times by thirty-two -- while the drained comparison is not settled in either direction, its control having been pooled across producer counts it is not independent of. The probe times the complete push, so that is the layout's effect on that path rather than a measurement of the exchange alone; it buys a longer lifetime argument -- a recurrence at 2^64 rather than at 2^56 -- and not a different kind of argument. Like every horizon in that column it scales with the caller's push rate rather than being absolute. **The default stays `Balanced`** so introducing the choice changed no existing caller's behaviour, and its documentation states the defect outright -- that past 2^32 pushes from two or more producers the queue can silently lose an item -- rather than leaving the status quo to look safe by inertia. It states the defect rather than recommending against the layout, per [D-no-client-prescriptions](../windows-platform-probes/DESIGN-NOTES.md#d-no-client-prescriptions): a caller told what the layout does can decide; a caller told what to prefer has been handed our judgement about their deployment. | ## D-2: capabilities are sliced, not gathered @@ -541,6 +541,14 @@ M31.5 rather than as an intention here. ## D-17: the reservation count and the claim position share one word +**Partly superseded by [D-41](#d-41): the 32/32 split is no longer forced.** The packing argument below +is unchanged and still describes `Balanced`, but its premise -- that the count must be able to reach the +whole capacity -- was the thing D-41 dropped. Capping outstanding reservations instead frees the position +to take 48 or 56 bits, so the split became a caller-selected layout rather than the only division of the +word. Read "forced" below as "forced *given that premise*", and the 2^31 ceiling as `Balanced`'s rather +than the shape's -- and on a 32-bit target as 2^30, where the crate-wide `usize` capacity bound binds +first. + **The obvious implementation is broken, and it is worth writing down why, because the brokenness is not visible from reading either side on its own.** With the count in its own atomic: @@ -571,10 +579,13 @@ Three consequences fall out, and all three are improvements: consumer's position anyway. So `reserving_mpsc`'s `pop` is one store shorter than `slotwise_mpsc`'s: nothing writes a "free again" sequence. -**The 32/32 split is forced, not chosen.** A position of `b` bits keeps a wrapping difference unambiguous -only up to `2^(b-1)`; the count can reach the capacity, so it needs `b` bits too; `b + b = 64` gives -`b = 32`. There is no cleverer division of the word, and the resulting ceiling is 2^31 items -- a ring -this shape allocates in full at construction, so at eight bytes an item it is already 17 GB. +**The 32/32 split is forced by the premise above, which [D-41](#d-41) later dropped.** A position of `b` bits +keeps a wrapping difference unambiguous +only up to `2^(b-1)`; if the count must be able to reach the capacity it needs `b` bits too; `b + b = 64` gives +`b = 32`. Given that requirement there is no cleverer division of the word, and the resulting ceiling is 2^31 items -- a ring +this shape allocates in full at construction, so at eight bytes an item it is already 17 GB. D-41 removed the +requirement rather than the arithmetic: capping outstanding reservations well below capacity lets the position +take 48 or 56 bits, which is what `Enduring` and `Perpetual` do. The 2^31 ceiling is therefore `Balanced`'s. That ceiling is reported through `CapacityError`'s `max_valid`, which [D-12](#d-12) had already made a property of the shape rather than of the crate. D-12 introduced that for the *minimum* and argued the @@ -589,9 +600,12 @@ constraint that binds: the count's half must be wide enough to hold the whole ca ## D-18: a 128-bit compare-and-swap is refused -**Superseded by [D-37](#d-37).** A 128-bit exchange is now adopted, but for a **separate wide -shape** rather than for this one: `reserving_mpsc` keeps its packed 64-bit word on every target, and -`reserving_mpsc_wide` is a peer beside it. Read this decision for the cost analysis, which D-37 +**Superseded by [D-37](#d-37).** A 128-bit exchange is now adopted, though not in the form D-37 first +proposed: it planned a **separate wide shape**, `reserving_mpsc_wide`, as a peer beside this one, and +[D-41](#d-41) replaced that with a `ClaimLayout` inside `reserving_mpsc`. The peer shape was never +built, so the name appears in this file only as history. What survives unchanged is that +`reserving_mpsc` keeps its packed 64-bit word on every target and the wide word is reached by asking +for it. Read this decision for the cost analysis, which D-37 depends on and does not repeat -- and note one correction it needs, below, that D-37's gate is built around. @@ -844,7 +858,26 @@ refused; a consumer wants to know how deep the backlog got and how often it was Measured by `probe-queue-contention` in a **release** build on an AMD EPYC 7763, 8 cores / 16 logical processors, Windows 11 Enterprise 10.0.26200, `x86_64`. Median of five repetitions after a discarded -warm-up; three independent invocations agreed to within noise. **Note the architecture**: every previous +warm-up; three independent invocations agreed to within noise. + +**That "within noise" rests on a floor this workspace has since measured to be far +wider.** The figure was read against a 2-6% run-to-run spread; seven runs of the +same probe later put the same-configuration spread at 7-61% depending on producer +count, and the probe's own same-code control spans 0.69-1.12x isolated and +0.68-1.27x drained. The figures below +are not retracted -- the direction of `D-26` survived a re-measurement on the +shipping type -- but "agreed within noise" is a weaker statement than it reads as, +and any difference here smaller than that control should not be treated as +established. See +[DESIGN-NOTES.md](../windows-platform-probes/DESIGN-NOTES.md#d-variance-is-a-finding). + +**These figures also predate the correction to the probe's timing window and have +not been retaken.** The probe timed from the coordinator's clock rather than from +the producers' own, which overstated throughput by a margin that grew with +producer count. Read the absolute values as optimistic, the high-producer rows +most so. + +**Note the architecture**: every previous measurement in this workspace was taken on the ARM64 development machine, so these numbers fill the x64 gap rather than extending the ARM64 record, and the two are not interchangeable. @@ -880,6 +913,16 @@ single consumer rather than the claim. ## D-27: why, and why it is not a bug to fix +**The causal mechanism below is a hypothesis this workspace has not established, and the figures in it +predate a correction to the probe's timing window.** Read the whole section as the historical argument +that made `D-26`'s result explicable rather than as a measured finding. What survives is the end-to-end +observation -- `reserving_mpsc` measured faster than `slotwise_mpsc` under contention on the hosts tried +-- and the fact that the difference is a property of the two *protocols* rather than of two +implementations of one. What does not survive is the attribution: the probe times the complete push and +so cannot isolate or bound the sequence read, which is the quantity this explanation rests on. The +padding experiment below is a real measurement and still rejects the false-sharing hypothesis; its +numbers, being pre-correction, should be read as optimistic. + The obvious response to D-26 is that `slotwise_mpsc` must have a defect. It does not, and the difference is worth understanding because it is a property of the two *protocols* rather than of two implementations of one. @@ -1025,7 +1068,7 @@ The ARM64 host asked the x64 host to test a specific prediction: **that SMT sibl stay in lockstep, giving shallow batches, and that this was the condition making caching lose.** ARM64 has no SMT and physically cannot express that placement, so only the x64 host could answer it. -`probe-core-affinity`, x64, medians of three runs (all three agreed to within 3%): +`placement-probe`, x64, medians of three runs (all three agreed to within 3%): | placement | base ns/item | cached ns/item | cached batch depth | verdict | |---|---|---|---|---| @@ -1076,7 +1119,7 @@ single machine and none cross-checks another. The per-placement coverage matrix, (`same cache, cross class`) that neither host can express, is kept with the open question in that open question rather than duplicated here. -**A probe defect found while doing this, now fixed.** `probe-core-affinity` printed its placement +**A probe defect found while doing this, now fixed.** `placement-probe` printed its placement table from a hard-coded list of four variants that omitted `SameCoreSiblings`, while the interpretation beneath it iterated over the placements actually measured. On an SMT host the table therefore showed the sibling row as absent while the interpretation quoted a number for it -- the @@ -1130,7 +1173,13 @@ information we have. A `0.x` version number carries the rest, and is meant liter [D-26](#d-26) falsified [D-16](#d-16)'s premise -- reading the consumer's position was supposed to make `reserving_mpsc` the expensive shape, and it is instead the faster one under contention, by up to 4x on -x64 and 6.4x on ARM64. That reopened a question D-16 had treated as settled: if the split does not buy +x64 and 6.4x on ARM64. **Those two figures predate the timing correction described in +[DESIGN-NOTES.md](../windows-platform-probes/DESIGN-NOTES.md#d-queue-layout-observations) and are +retained only as the record of what reopened the question**; the two-host capture they came from has +been withdrawn from the README rather than carried forward, and no current public figure is derived +from them. The direction they established -- that `reserving_mpsc` is not the expensive shape -- +survives the correction; the magnitudes do not. That reopened a question D-16 had treated as settled: +if the split does not buy what it claimed, should the shapes merge, or should one be deleted? **Neither. Both ship, and the crate declines to choose between them on the caller's behalf.** @@ -1147,7 +1196,7 @@ What the crate owes a caller instead is honesty and equipment: - **The measurements, stated plainly**, including the regimes where each wins and the fact that the answer inverted once already when a second architecture was tried. -- **The means to measure their own domain.** `probe-core-affinity` and the placement tool exist so a +- **The means to measure their own domain.** `placement-probe` and the placement tool exist so a caller can settle this on their own hardware and workload rather than inheriting ours. A queue library that publishes one benchmark and calls it a recommendation is asserting a conclusion about machines it has never seen. @@ -1247,13 +1296,33 @@ obviously cheaper. This decision records only the landscape and the criterion. ## D-35: the permit claim measured, and the result that inverts the expectation +**Every magnitude in this section predates the correction to the probe's timing window, and is +retained as the record of what was measured rather than as a current figure.** Per +[D-29](#d-29) no current public figure derives from that capture. The direction the section +establishes -- that the permit claim is faster where contention exists and slower at one producer -- +survives the correction; the multipliers do not. + Run by `probe-queue-contention` on the reference host (x86-64, 16 logical / 8 physical, SMT on), release build, five repetitions per configuration with the median kept. The whole run was repeated three times; the isolated numbers reproduced within noise except one outlier noted below. +**Read "within noise" here against the wider floor measured later**: seven runs +of this probe put the same-configuration spread at 7-61%, and its same-code +control at 0.69-1.12x isolated -- the regime these numbers are in -- against +0.68-1.27x drained, so three agreeing runs establish less than the phrase +suggests. See +[DESIGN-NOTES.md](../windows-platform-probes/DESIGN-NOTES.md#d-variance-is-a-finding). + +**These figures also predate the correction to the probe's timing window and have +not been retaken.** The probe timed from the coordinator's clock rather than from +the producers' own, which overstated throughput by a margin that grew with +producer count. Read the absolute values as optimistic, the high-producer rows +most so. + ### Isolated regime -- producers only, nothing ever refused -The cleanest measurement of the claim, because nothing else touches the queue. Nanoseconds per push: +The cleanest comparison between the shapes, because nothing else touches the queue -- but still a +measurement of each shape's whole push path, not of the claim alone. Nanoseconds per push: | producers | `slotwise_mpsc` | `reserving_mpsc` | `permit_mpsc` | contended `fetch_add` | |---|---|---|---|---| diff --git a/crates/windows-waitable-queues/README.md b/crates/windows-waitable-queues/README.md index 0cfb87090..805c2998f 100644 --- a/crates/windows-waitable-queues/README.md +++ b/crates/windows-waitable-queues/README.md @@ -64,12 +64,12 @@ how many threads push, whether a slot can be claimed before the message exists -- decides its *algorithm*, not merely its configuration, so these are separate shapes rather than one type with switches. A caller names the shape it wants. -| Shape | Producers | What it adds | Choose it when | +| Shape | Producers | What it adds | Applies when | |---|---|---|---| | `spsc` | one | nothing -- no compare-and-swap on either side | exactly one thread pushes | -| `slotwise_mpsc` | many | Vyukov's per-slot sequence protocol, so producers push without a lock | **the default** for many producers | +| `slotwise_mpsc` | many | Vyukov's per-slot sequence protocol, so producers push without a lock | many threads push and a full queue may refuse | | `reserving_mpsc` | many | claiming a slot *before* the message exists | a message must not be lost to a full queue | -| `permit_mpsc` | many | an experimental claim protocol | never in production -- see below | +| `permit_mpsc` | many | an experimental claim protocol | behind `experimental-permit-claim`, outside the semver promise -- see below | Every shape has one consumer. `permit_mpsc` is behind the non-default `experimental-permit-claim` feature and is outside the semver promise; it will @@ -119,16 +119,36 @@ to correct that misreading once. **This is a property of the default layout, not of the shape**, and that is a change: it was previously a defect a caller had to live with. The claim word packs an outstanding-reservation count beside the position, and how its bits are -divided is now a caller's choice. Reservations are bounded by how many producers -are mid-send -- hundreds at most -- so giving up a ceiling nobody reaches buys -positions: +divided is now a caller's choice. A narrower count field buys position bits, and +what it costs is reservations held simultaneously: `Producer::reserve` takes +`&self` and returns an owned `Reservation`, so a single producer can hold as +many as the field allows, and a caller that holds many at once is choosing +against the narrower layouts rather than against a producer count. -| Layout | Outstanding reservations | Pushes to recurrence | At sustained maximum rate | +| Layout | Reservation-count field ceiling | Pushes to recurrence | At the pre-correction planning rate | |---|---|---|---| -| `Balanced` (default) | 2^32 | 2^32 | about 37 seconds | +| `Balanced` (default) | 4,294,967,295 | 2^32 | about 37 seconds | | `Enduring` | 65,535 | 2^48 | about 28 days | | `Perpetual` | 255 | 2^56 | about 20 years | -| `Wide` (needs `dwcas`) | 2^32 | 2^64 | unreachable | +| `Wide` (needs `dwcas`) | 4,294,967,295 | 2^64 | about 5,000 years | + +The last column is arithmetic, not a measurement: pushes-to-recurrence divided by +a sustained rate of about 116 million pushes per second. **That rate predates a +correction to the probe's timing window**, which had overstated throughput -- so +the true sustained rate is lower and these horizons longer. They are kept as a +floor, saying the wrap arrives sooner than it does, which is the conservative +direction for a hazard. The horizon that matters is the one on your hardware at +your rate. + +The middle column is the field's ceiling rather than the count any particular +queue reaches: admission is also bounded by capacity -- `reserve` refuses once +the ring has no room beyond the reservations already outstanding -- so the +achievable count is the lesser of the two. It is reachable where capacity allows: +one producer alone fills `Perpetual`'s 255 in a loop given a ring that large, +which `one_producer_alone_can_exhaust_the_reservation_field` pins. For `Balanced` +the capacity bound binds first, since that +layout accepts at most 2^31 slots on a 64-bit target, and 2^30 on a 32-bit +one. For the others the field is the smaller number only once the queue is at least that large: a `Perpetual` queue of capacity 64 can hold 64 reservations, not 255. The achievable count is always the lesser of the two. ```rust use windows_waitable_queues::reserving_mpsc::{self, Perpetual}; @@ -139,15 +159,21 @@ let (tx, rx) = reserving_mpsc::bounded_as::(64)?; # Ok::<(), windows_waitable_queues::CapacityError>(()) ``` -**A deeper position costs nothing measurable.** `Balanced`, `Enduring`, and -`Perpetual` all issue the same exchange on the same 64-bit word and differ only -in shift and mask constants; a probe comparing them found no difference outside -noise. `Wide` is the exception: it needs a 128-bit exchange, which measured 2-3x -slower on the claim, and it is the only thing in this crate that costs a -third-party dependency. +**A deeper position is the same exchange on the same word.** `Balanced`, +`Enduring`, and `Perpetual` all issue the same exchange on the same 64-bit word +and differ only in shift and mask constants, so there is no structural reason for +one to be slower -- but **what that costs in throughput is not established**: a +probe comparing them found them indistinguishable at low producer counts, and at high counts sat outside the probe's same-code control but too close to it to establish an ordering or a cost on this host. `Wide` is a separate matter: it needs a 128-bit exchange, +and the whole push path was measured as slower under it at every producer count +measured -- smallest at one or two, several times by thirty-two, in the isolated +regime -- and it is the only thing in +this crate that costs a third-party dependency. The default remains `Balanced` so that no existing caller's behaviour changed -when the choice was introduced. It is not the recommended layout. +when the choice was introduced. Under it, a queue driven past 2^32 pushes by two +or more producers can **silently lose an item** -- the defect described above. +`Enduring` and `Perpetual` move that point out by 2^16 and 2^24 respectively, and +`Wide` moves it to 2^64 pushes. **What happens.** A producer checks that there is room, is descheduled, and resumes after other producers have driven the position field through a complete @@ -161,31 +187,39 @@ receives a different item than the one that was sent, and nothing observable says so -- which is why this is documented here rather than left to a caller to discover, and why it cannot be mitigated after the fact. -**The exposure, measured rather than estimated.** Under `Balanced`, 2^32 pushes -is 37 seconds to roughly four minutes of *sustained* pushing at this crate's own -measured rates -- about two minutes at two producers, which is the smallest -count that can trigger it at all. That is sustained throughput, not a total +**The exposure, as arithmetic over a disclosed rate.** Under `Balanced`, 2^32 +pushes is about 37 seconds of *sustained* pushing at the rate the layout table +above discloses. Two producers is the smallest count that can trigger the defect +at all. **That rate predates a correction to +the probe's timing window** and is kept as a floor for the reason the layout +table above gives: the correction lowers the rate and lengthens the horizon, so +these figures name a shorter horizon than the corrected rate gives. That is sustained throughput, not a total accumulated over an uptime. Reaching the wrap is necessary but not sufficient: a producer must also be stalled inside a window a few instructions wide. Rare, but a preemption is enough, and "rare" over billions of pushes is not "never". -The figures in the table above scale that same measurement by the position +The figures in the table above scale that same rate model by the position width, so they are a floor on time rather than a forecast: a queue that must -drain cannot sustain the fastest rate measured, and a slower producer takes +drain cannot sustain the fastest rate shown, and a slower producer takes proportionally longer to reach its wrap. -**What to do about it.** +**What bears on it.** -- **Name a layout.** `Perpetual` puts the recurrence about twenty years out at - no measured cost, which takes it past any real deployment. This is the answer - for almost every caller who is exposed at all. +- **Naming a layout moves it.** `Perpetual` puts the recurrence about twenty + years out. **What it costs in throughput is not established** -- it issues the + same atomic compare-exchange on the same `u64` as the default, and was measured + as indistinguishable from it at low producer counts; at high counts sat outside the probe's same-code control but too close to it to establish an ordering or a cost on this host. - **`slotwise_mpsc` does not have this hazard** under any layout. Its positions - are 64 bits on every target, so the equivalent wrap needs 2^64 claims. Prefer - it unless you need `Reserving`. + are 64 bits on every target, so the equivalent wrap needs 2^64 claims. It does + not offer `Reserving`. - **`spsc` never had it**, having no contended claim to race. -- **The default layout is sound below its wrap.** A queue that will not push 4.3 - billion items in one run, or that is not driven at sustained maximum rate by - two or more producers, is not exposed even on `Balanced`. +- **The default layout's exposure is a count, not a rate.** Two conditions must + both hold: two or more producers (one producer has no race to lose), and 4.3 + billion pushes accumulated over the life of one queue. A lower sustained rate + does not remove the exposure -- the position advances once per push regardless + of how fast they arrive, so a slow queue with two or more producers reaches the + same wrap, just later. An earlier version of this bullet listed a low rate as + its own exemption, which was wrong. This is disclosed on the same principle as the ordering gap below: an adopter gets the information we have rather than an assurance we cannot support. The @@ -202,11 +236,21 @@ Both are off by default, and the default build depends on `windows-sys` alone. `reserving_mpsc`. This is the only thing in the crate that costs a third-party dependency: Rust's standard library has no 128-bit atomic -- `core::sync::atomic` stops at 64 bits -- so the double-width compare-and-swap comes from -`portable-atomic`. Most callers do not need it; `Perpetual` reaches roughly -twenty years before its claim position recurs with no dependency and no measured -cost, while the 128-bit exchange measured 2-4x slower on the claim itself. Take -it when you want the recurrence gone as a guarantee rather than deferred by an -argument about deployment lifetimes. +`portable-atomic`. `Perpetual` reaches roughly +twenty years before its claim position recurs with no dependency, though what +that costs in throughput is not established, while under `Wide` the whole push +path was measured as slower at every producer count measured -- smallest at one +or two, several times by thirty-two, in the isolated regime. That claim rests on +a seven-run sweep whose raw runs were never committed, and on a three-run capture +that was: +[captures/2026-09-16-drained-handshake/](../windows-platform-probes/captures/2026-09-16-drained-handshake/README.md), +whose `isolated.js` derives the per-count layout ratios against a same-code +control. The two agree on the direction; the capture is the part a reader can +run. What `Wide` provides +that the `u64` layouts do not is a 64-bit position: the recurrence moves to +2^64 pushes -- about 5,000 years at the same rate the table above uses, rather +than the twenty `Perpetual` buys. That is a longer horizon, not the absence of +one, and it moves with the caller's rate like every other figure in that column. **`experimental-permit-claim`** adds `permit_mpsc`, a different claim protocol in which the decision and the operation are one atomic rather than two. It is @@ -337,55 +381,126 @@ position, which is the only way a reservation can be answered at all. Both are well-studied designs in production use elsewhere, which is why this crate ships both rather than picking one for you. -**Start here:** +**What distinguishes them:** -- **Pushing more than ~4 billion items in one run, from two or more producers?** - Either use `slotwise_mpsc`, whose positions are 64 bits under every - configuration, or name a deeper layout on `reserving_mpsc` -- `Perpetual` - puts the recurrence about twenty years out at no measured cost. Under its - default layout `reserving_mpsc` can lose an item past that volume; see +- **Pushing more than ~4 billion items through one queue, from two or more producers?** + Under its default layout `reserving_mpsc` can lose an item past that volume. + The count is cumulative over that queue's whole life, not per run: many short + bursts reach the wrap as surely as one long one. + `slotwise_mpsc`'s positions are 64 bits under every configuration, and naming a + deeper layout on `reserving_mpsc` moves the recurrence out -- `Perpetual` to + about twenty years -- though what that costs in throughput is not established. + The mechanism is in [the section on recurrence](#how-long-reserving_mpsc-runs-before-its-claim-position-recurs) - above, which you should read before choosing. -- Need `reserve`? Only `reserving_mpsc` has it, and `slotwise_mpsc` structurally - cannot. That no longer forces a trade against the recurrence: choosing a - layout addresses it, so the capability can settle the choice on its own - merits. -- Otherwise, **start with `reserving_mpsc`.** It was the faster of the two at - every producer count we measured above one. -- Only one producer *and* one consumer? Use `spsc`, which beats both. - -**What we measured**, in ns per push, isolated regime, median of three runs. -Higher producer counts oversubscribe both hosts: - -| producers | `slotwise_mpsc` (x64) | `reserving` (x64) | `slotwise_mpsc` (ARM64) | `reserving` (ARM64) | + above. +- **Of the two MPSC shapes, only `reserving_mpsc` offers `reserve`**; + `slotwise_mpsc` structurally cannot. (`spsc` has it too, and the experimental + `permit_mpsc` exposes its own.) Wanting `reserve` no longer means accepting the + default layout's recurrence, but the trade is not gone -- it changes axis: a + deeper position is paid for with a lower ceiling on outstanding reservations, + 65,535 under `Enduring` and 255 under `Perpetual` against `u32::MAX` under the + default. +- **`spsc` requires exactly one producer and one consumer**, and does less work + than either MPSC shape because of it. + +The measurements below are one host's observation, recorded with the parameters +that produced them. They are not a ranking, and which shape suits a given +deployment is the deployment's question. + +### What was measured + +In ns per operation, isolated regime (producers only, +capacity large enough that nothing is refused). Each cell is the median of three +whole-probe runs, followed by the full range across all fifteen repetitions those +runs contain. An operation is one successful push for the three queue shapes; +for `baseline_fetch_add` it is one `fetch_add`, which is why the column is +labelled per operation rather than per push: + +| producers | `slotwise_mpsc` | `reserving_mpsc` | `permit_mpsc` | `baseline_fetch_add` | |---|---|---|---|---| -| 1 | 9.0 | 8.6 | 6.5 | 6.1 | -| 2 | 49.0 | 28.0 | 29.8 | 9.4 | -| 4 | 84.4 | 33.3 | 60.6 | 12.9 | -| 8 | 140.8 | 38.5 | 167.4 | 29.8 | -| 16 | 193.5 | 52.2 | 194.9 | 30.6 | -| 32 | 239.7 | 56.9 | 195.0 | 30.6 | - -x64 is an AMD EPYC 7763 slice (8 cores, 16 threads); ARM64 is a Snapdragon X2 -Elite (12 cores, no SMT). **Read these as two data points, not as a law.** This -comparison has already inverted once: it was designed on the assumption that -`slotwise_mpsc` would be the cheaper shape, and measurement said otherwise on both -machines. - -**Measure your own workload before treating any of this as settled.** Producer -count, how hard the consumer drains, and where the threads are scheduled all -move the answer -- thread placement alone moved an SPSC handoff by 5.6x on one -of these hosts. The `probe-core-affinity` tool in this repository exists so you -can run that measurement on your hardware instead of inheriting ours. +| 1 | 6.3 (6.3-7.5) | 5.4 (5.4-6.4) | 7.9 (7.9-8.3) | 2.3 (2.3-2.7) | +| 2 | 50.6 (19.3-59.5) | 31.9 (22.5-35.2) | 44.2 (37.4-45.9) | 12.1 (5.8-14.3) | +| 4 | 91.6 (89.7-99.9) | 37.2 (31.6-41.5) | 31.8 (30.4-32.9) | 13.8 (12.6-17.6) | +| 8 | 138.6 (126.9-157.6) | 37.8 (34.3-41.7) | 25.9 (25.0-27.4) | 14.7 (13.9-15.9) | +| 16 | 218.0 (188.9-272.7) | 47.9 (44.9-56.0) | 21.8 (20.9-25.6) | 14.8 (14.4-15.9) | +| 32 | 224.7 (131.4-268.3) | 51.3 (40.7-55.4) | 21.9 (20.7-39.0) | 15.0 (14.7-15.7) | + +**The ranges are the point, not a footnote.** `slotwise_mpsc` at two producers +spans 19.3 to 59.5 within one configuration on one host, +and at thirty-two, 131.4 to 268.3. A median quoted without that is an anecdote, +which is why +[D-observations-not-verdicts](../windows-platform-probes/DESIGN-NOTES.md#d-observations-not-verdicts) +obliges every published figure here to carry its run count *and* its dispersion. +An earlier version of this table published the medians alone and did not meet +that obligation; the probe now carries the range through to the report so it +cannot be omitted again. + +**Attribution, because a figure without it is not reusable data:** + +| | | +|---|---| +| Host | `x86_64 16p/8c smt+ L2[2,2,2,2,2,2,2,2] ec[0:16] numa[16]` | +| Profile | release | +| Sampling | 50,000 pushes per producer, median of 5 repetitions, one untimed warmup pass | +| Runs | 3 whole-probe invocations; cells are the median of the three, ranges span all 15 repetitions | +| Instrument | `probe-queue-contention`, built from `fecd352` (the commit that added the range columns) | +| Taken | 2026-09-15 UTC-07:00 | + +The banner's `numa[16]` is a single NUMA node holding all sixteen processors, so +nothing here says anything about cross-domain behaviour. `permit_mpsc` is behind +`experimental-permit-claim` and is not covered by the semver promise. +`baseline_fetch_add` is N threads incrementing one `AtomicU64` -- the cheapest +thing N threads can do to a contended line, included so the queue figures can be +read against what this processor does to such a line at all. + +**Read these as one machine's numbers.** Producer counts above 8 oversubscribe +this host's 8 physical cores, and the spread is not small at either scale. +*Between* runs: `slotwise_mpsc` at sixteen producers gave whole-run medians of +225.7, 218.0 and 192.9. *Within* a run the probe reports its own per-row spread +-- a fourth, separate invocation of the same build gave that row a median of +226.5 over a 181.5-242.3 range across its five repetitions. +The parenthesised ranges in the table above are the wider quantity: the extremes +over all fifteen repetitions of the three captured runs. The probe's same-code +control for this regime has been measured at 0.69-1.12x over seven runs -- the +drained regime's is wider, at 0.68-1.27x, and does not apply to the isolated +figures above; see +[DESIGN-NOTES.md](../windows-platform-probes/DESIGN-NOTES.md#d-variance-is-a-finding). +That seven-run sweep is a **separate capture** taken to size the noise floor, not +a longer version of this table -- its medians differ from the ones above, which is +the point it was making. Where the two disagree, this table is the attributed +figure for this crate and the sweep is the evidence about how much such a figure +moves. + +**A previous version of this table compared two hosts** -- an AMD EPYC 7763 slice +and a Snapdragon X2 Elite -- and has been removed rather than carried forward. Its +figures predate a correction to the probe's timing window, which timed from the +coordinator's clock rather than the producers' own and overstated throughput by a +margin that grew with producer count; and neither of those machines is available +here to retake them. The ARM64 data point is therefore gone rather than stale, +which is the lesser of the two problems. Restoring one is what M2.15 in the +probe crate's [CHECKLIST.md](../windows-platform-probes/CHECKLIST.md) is for. + +That comparison did carry one finding worth keeping, because it was structural +rather than numeric: it was designed on the assumption that `slotwise_mpsc` would +be the cheaper shape, and measurement said otherwise on both machines. + +**What moves these numbers.** Producer count, how hard the consumer drains, and +where the threads are scheduled all change the answer -- thread placement alone +moved an SPSC handoff by 5.6x on an earlier host this workspace measured. The +`placement-probe` tool in this repository runs that measurement, and +`probe-queue-contention` runs the one above. Two things that look like reasons to choose and are not: -- **Capacity.** On a 64-bit target `slotwise_mpsc` reaches 2^62 slots and - `reserving_mpsc` 2^31. On a 32-bit one the crate-wide ceiling is 2^30 and - **both** shapes land there -- `reserving_mpsc`'s packed 2^31 is clamped down - to it as well -- so the difference disappears and the comparison means - nothing. Either way it counts slots allocated up front, not items ever pushed: - a ring of 2^31 slots is tens of gigabytes before it holds anything useful. +- **Capacity.** `reserving_mpsc` has no single ceiling: it is the layout's, one + bit narrower than that layout's position. On a 64-bit target `Balanced` + reaches 2^31, `Enduring` 2^47, `Perpetual` 2^55, and `Wide` 2^62 -- the last + being the crate-wide ceiling, which is also `slotwise_mpsc`'s, so under `Wide` + the two shapes reach the same number and there is nothing to compare. On a + 32-bit target the crate-wide ceiling is 2^30 and every layout of both shapes + lands there, so the difference disappears again. Either way it counts slots + allocated up front, not items ever pushed: a ring of 2^31 slots is tens of + gigabytes before it holds anything useful. - **`slotwise_mpsc` winning at one producer.** True in one regime, and at one producer you want `spsc` anyway. diff --git a/crates/windows-waitable-queues/src/lib.rs b/crates/windows-waitable-queues/src/lib.rs index 1c8817227..d7ac006c4 100644 --- a/crates/windows-waitable-queues/src/lib.rs +++ b/crates/windows-waitable-queues/src/lib.rs @@ -87,16 +87,39 @@ //! **This is a property of the default layout, not of the shape**, and that is //! a change: it was previously a defect a caller had to live with. The claim //! word packs an outstanding-reservation count beside the position, and how its -//! bits are divided is now a caller's choice. Reservations are bounded by how -//! many producers are mid-send -- hundreds at most -- so giving up a ceiling -//! nobody reaches buys positions: -//! -//! | Layout | Outstanding reservations | Pushes to recurrence | At sustained maximum rate | +//! bits are divided is now a caller's choice. A narrower count field buys +//! position bits, and what it costs is reservations held simultaneously: +//! `Producer::reserve` takes `&self` and returns an owned `Reservation`, so a +//! single producer can hold as many as the field allows, and a caller that +//! holds many at once is choosing against the narrower layouts rather than +//! against a producer count. +//! +//! | Layout | Reservation-count field ceiling | Pushes to recurrence | At the pre-correction planning rate | //! |---|---|---|---| -//! | `Balanced` (default) | 2^32 | 2^32 | about 37 seconds | +//! | `Balanced` (default) | 4,294,967,295 | 2^32 | about 37 seconds | //! | `Enduring` | 65,535 | 2^48 | about 28 days | //! | `Perpetual` | 255 | 2^56 | about 20 years | -//! | `Wide` (needs `dwcas`) | 2^32 | 2^64 | unreachable | +//! | `Wide` (needs `dwcas`) | 4,294,967,295 | 2^64 | about 5,000 years | +//! +//! The last column is arithmetic, not a measurement: pushes-to-recurrence +//! divided by a sustained rate of about 116 million pushes per second. **That +//! rate predates a correction to the probe's timing window**, which had +//! overstated throughput -- so the true sustained rate is lower and these +//! horizons longer. They are kept as a floor, saying the wrap arrives sooner +//! than it does -- a shorter horizon than the corrected rate gives. The horizon +//! that matters is the one on your hardware at your rate. +//! +//! The reservation-count column is the field's ceiling rather than the count any +//! particular queue reaches: admission is also bounded by capacity, so the +//! achievable count is the lesser of the two. It is reachable where capacity +//! allows -- one producer alone fills `Perpetual`'s 255 in a loop given a ring +//! that large, which `one_producer_alone_can_exhaust_the_reservation_field` +//! pins. For `Balanced` the capacity bound binds first -- +//! that layout accepts at most 2^31 slots on a 64-bit target, and 2^30 on a +//! 32-bit one. For the others the field is the smaller number only once +//! the queue is at least that large: a `Perpetual` queue of capacity 64 can +//! hold 64 reservations, not 255. The achievable count is always the lesser +//! of the two. //! //! ``` //! use windows_waitable_queues::reserving_mpsc::{self, Perpetual}; @@ -107,16 +130,39 @@ //! # Ok::<(), windows_waitable_queues::CapacityError>(()) //! ``` //! -//! **A deeper position costs nothing measurable.** `Balanced`, `Enduring`, and -//! `Perpetual` all issue the same exchange on the same 64-bit word and differ -//! only in shift and mask constants; a probe comparing them found no difference -//! outside noise. `Wide` is the exception: it needs a 128-bit exchange, which -//! measured 2-3x slower on the claim, and it is the only thing in this crate -//! that costs a third-party dependency. Prefer `Perpetual` unless you want the -//! guarantee rather than the twenty years. +//! **A deeper position is the same exchange on the same word.** `Balanced`, +//! `Enduring`, and `Perpetual` all issue the same exchange on the same 64-bit +//! word and differ only in shift and mask constants, so there is no structural +//! reason for one to be slower -- but **what that costs in throughput is not +//! established**: a probe comparing them found them indistinguishable at low +//! producer counts, and at high counts sat outside the probe's same-code control but too close to it to establish an ordering or a cost on this host. `Wide` is a separate +//! matter: it needs a 128-bit exchange, and the whole push path was measured as +//! slower under it at every producer count measured -- smallest at one or two, +//! several times by thirty-two, in the isolated regime -- and it is the only +//! thing in +//! this crate +//! that costs a third-party dependency. What it provides that `Perpetual` does +//! not is a 64-bit position: the recurrence moves to 2^64 pushes -- about 5,000 +//! years at the rate the table above uses, against the twenty `Perpetual` buys. +//! A longer horizon rather than the absence of one, and it moves with the +//! caller's rate like every other figure in that column. +//! +//! **Where the layout evidence is.** The `What was measured` table this section +//! links carries the queue *shapes*, not the claim-word layouts, so it does not +//! contain the `Wide` figures. Those come from two other places: a seven-run +//! sweep whose raw runs were never committed, and a three-run capture that was, +//! at +//! [captures/2026-09-16-drained-handshake/](https://github.com/MikeGrier/windows-threadpool-sys/blob/main/crates/windows-platform-probes/captures/2026-09-16-drained-handshake/README.md), +//! whose [isolated.js](https://github.com/MikeGrier/windows-threadpool-sys/blob/main/crates/windows-platform-probes/captures/2026-09-16-drained-handshake/isolated.js) derives the +//! per-count layout ratios against a same-code control. The two agree on the +//! direction. Until the sweep is re-run with its data kept, the capture is the +//! part of this claim a reader can check. //! //! The default remains `Balanced` so that no existing caller's behaviour -//! changed when the choice was introduced. It is not the recommended layout. +//! changed when the choice was introduced. Under it, a queue driven past 2^32 +//! pushes by two or more producers can **silently lose an item** -- the defect +//! described above. `Enduring` and `Perpetual` move that point out, and `Wide` +//! moves it to 2^64 pushes. //! //! **What happens.** A producer checks that there is room, is descheduled, and //! resumes after other producers have driven the position field through a @@ -130,10 +176,15 @@ //! observable says so -- which is why this is documented here rather than left //! to a caller to discover, and why it cannot be mitigated after the fact. //! -//! **The exposure, measured rather than estimated.** Under `Balanced`, 2^32 -//! pushes is 37 seconds to roughly four minutes of *sustained* pushing at this -//! crate's own measured rates -- about two minutes at two producers, which is -//! the smallest count that can trigger it at all. That is sustained throughput, +//! **The exposure, as arithmetic over a disclosed rate.** Under `Balanced`, 2^32 +//! pushes is about 37 seconds of *sustained* pushing at the rate the layout +//! table above discloses. Two producers is the smallest count that can trigger +//! the defect at all. **That rate predates a +//! correction to the probe's timing window** and is kept as a floor for the +//! reason the layout table above gives: the correction lowers the rate and +//! lengthens the horizon, so these figures say the wrap arrives sooner than it +//! does -- a shorter horizon than the corrected rate gives. That is sustained +//! throughput, //! not a total accumulated over an uptime. Reaching the wrap is necessary but //! not sufficient: a producer must also be stalled inside a window a few //! instructions wide. Rare, but a preemption is enough, and "rare" over @@ -144,18 +195,23 @@ //! drain cannot sustain the fastest rate measured, and a slower producer takes //! proportionally longer to reach its wrap. //! -//! **What to do about it.** +//! **What bears on it.** //! -//! - **Name a layout.** `Perpetual` puts the recurrence about twenty years out -//! at no measured cost, which takes it past any real deployment. This is the -//! answer for almost every caller who is exposed at all. +//! - **Naming a layout moves it.** `Perpetual` puts the recurrence about twenty +//! years out. **What it costs in throughput is not established** -- it issues +//! the same atomic compare-exchange on the same `u64` as the default, and was +//! measured as indistinguishable from it at low producer counts; at high counts sat outside the probe's same-code control but too close to it to establish an ordering or a cost on this host. //! - **[`slotwise_mpsc`] does not have this hazard** under any layout. Its //! positions are 64 bits on every target, so the equivalent wrap needs 2^64 -//! claims. Prefer it unless you need [`Reserving`]. +//! claims. It does not offer [`Reserving`]. //! - **[`spsc`] never had it**, having no contended claim to race. -//! - **The default layout is sound below its wrap.** A queue that will not push -//! 4.3 billion items in one run, or that is not driven at sustained maximum -//! rate by two or more producers, is not exposed even on `Balanced`. +//! - **The default layout's exposure is a count, not a rate.** Two conditions +//! must both hold: two or more producers (one producer has no race to lose), +//! and 4.3 billion pushes accumulated over the life of one queue. A lower +//! sustained rate does not remove the exposure -- the position advances once +//! per push regardless of how fast they arrive, so a slow queue with two or +//! more producers reaches the same wrap, just later. An earlier version of +//! this bullet listed a low rate as its own exemption, which was wrong. //! //! This is disclosed on the same principle as the ordering gap below: an //! adopter gets the information we have rather than an assurance we cannot @@ -262,41 +318,78 @@ //! answered at all. Both are well-studied designs in production use elsewhere, //! which is why this crate ships both instead of picking one for you. //! -//! - **Pushing more than ~4 billion items in one run, from two or more -//! producers?** Use [`slotwise_mpsc`]. [`reserving_mpsc`] has a known -//! item-loss defect past that volume, on every target -- see the section -//! above, which you should read before choosing. -//! - Need [`Reserving`]? Only [`reserving_mpsc`] has it; [`slotwise_mpsc`] structurally -//! cannot. Weigh that against the defect above rather than treating the -//! capability as settling the choice. -//! - Otherwise **start with [`reserving_mpsc`]**: it was the faster of the two -//! at every producer count above one that we measured. -//! - One producer *and* one consumer? Use [`spsc`], which beats both. -//! -//! Measured ns per push, isolated regime, median of three. An AMD EPYC 7763 -//! slice (8 cores, 16 threads) and a Snapdragon X2 Elite (12 cores, no SMT): -//! -//! | producers | `slotwise_mpsc` x64 | `reserving` x64 | `slotwise_mpsc` ARM64 | `reserving` ARM64 | -//! |---|---|---|---|---| -//! | 1 | 9.0 | 8.6 | 6.5 | 6.1 | -//! | 2 | 49.0 | 28.0 | 29.8 | 9.4 | -//! | 4 | 84.4 | 33.3 | 60.6 | 12.9 | -//! | 8 | 140.8 | 38.5 | 167.4 | 29.8 | -//! | 16 | 193.5 | 52.2 | 194.9 | 30.6 | -//! | 32 | 239.7 | 56.9 | 195.0 | 30.6 | -//! -//! **Read these as two data points, not as a law**, and measure your own -//! workload before treating them as settled. This comparison has already -//! inverted once: the split was designed on the assumption that `slotwise_mpsc` would be -//! the cheaper shape, and measurement disagreed on both machines. Producer -//! count, how hard the consumer drains, and where the threads are scheduled all -//! move the answer -- placement alone moved an SPSC handoff by 5.6x on one of -//! these hosts. -//! -//! Two things that look like reasons to choose and are not. **Capacity**: on a -//! 64-bit target `slotwise_mpsc` reaches 2^62 slots and `reserving_mpsc` 2^31. +//! - **Pushing more than ~4 billion items through one queue, from two or more +//! producers?** [`reserving_mpsc`] under its default layout has a known +//! item-loss defect past that volume, on every target. The count is cumulative +//! over that queue's whole life, not per run: many short bursts reach the wrap +//! as surely as one long one. [`slotwise_mpsc`]'s +//! positions are 64 bits under every configuration, and naming a deeper layout +//! on [`reserving_mpsc`] moves the recurrence out. The mechanism is in the +//! section above. +//! - **Of the two MPSC shapes, only [`reserving_mpsc`] implements +//! [`Reserving`]**; [`slotwise_mpsc`] structurally cannot. ([`spsc`] +//! implements it too, and the experimental `permit_mpsc` exposes its own +//! `reserve`.) Wanting it no longer means accepting the default layout's +//! recurrence, but the trade is not gone -- it changes axis: a deeper position +//! is paid for with a lower ceiling on outstanding reservations -- a *field* +//! ceiling of 65,535 under +//! `Enduring` and 255 under `Perpetual` against `u32::MAX` under the default, +//! with the count a given queue can actually hold being the lesser of that and +//! its capacity. +//! - **[`spsc`] requires exactly one producer and one consumer**, and does less +//! work than either MPSC shape because of it. +//! +//! **The measurements live in one place, not two.** This crate's +//! [README][readme-measurements] carries the capture: the isolated-regime table, +//! every cell's observed range, and the attribution -- host banner, profile, +//! sampling parameters, run count, the instrument's commit, and when it was +//! taken. That is deliberately not duplicated here, because a figure with two +//! hand-maintained homes is a figure that will disagree with itself the first +//! time one of them is retaken. This branch did exactly that once already. +//! +//! What is worth saying without the digits: +//! +//! - The figures are **one host's observation**, not a ranking, and which shape +//! suits a deployment is the deployment's question. +//! - **The ranges matter more than the medians.** Every cell carries the span +//! its repetitions covered, because +//! [`D-observations-not-verdicts`] obliges a published figure to arrive with +//! its run count *and* its dispersion. At some producer counts that span is +//! wide enough to swallow the difference between shapes. +//! - An *operation* is one successful push for the three queue shapes; for +//! `baseline_fetch_add` it is one `fetch_add` on a shared `AtomicU64`, which is +//! why the column is labelled per operation rather than per push. It is +//! included so the queue figures can be read against what this processor does +//! to a contended line at all. +//! - The host is a single NUMA node holding all its processors, so **nothing +//! there says anything about cross-domain behaviour**, and producer counts +//! above its physical core count oversubscribe it. +//! - `permit_mpsc` is behind `experimental-permit-claim` and is outside the +//! semver promise. +//! +//! An earlier capture compared two machines and was removed rather than carried +//! forward: its figures predate a correction to the probe's timing window, and +//! neither machine is available here to retake them. One finding from it was +//! structural rather than numeric and is worth keeping -- the split was designed +//! on the assumption that `slotwise_mpsc` would be the cheaper shape, and +//! measurement disagreed on both machines. +//! +//! [readme-measurements]: https://github.com/MikeGrier/windows-threadpool-sys/blob/main/crates/windows-waitable-queues/README.md#what-was-measured +//! [`D-observations-not-verdicts`]: https://github.com/MikeGrier/windows-threadpool-sys/blob/main/crates/windows-platform-probes/DESIGN-NOTES.md#d-observations-not-verdicts +//! +//! **What moves these numbers.** Producer count, how hard the consumer drains, +//! and where the threads are scheduled -- placement alone moved an SPSC handoff +//! by several times on an earlier host this workspace measured; the figure is in +//! the README's measurement section rather than repeated here. +//! +//! Two things that look like reasons to choose and are not. **Capacity**: the +//! ceiling is the layout's, not the shape's. On a 64-bit target `slotwise_mpsc` +//! reaches 2^62 slots, and `reserving_mpsc` reaches 2^31 under `Balanced`, 2^47 +//! under `Enduring`, 2^55 under `Perpetual` and 2^62 under `Wide` -- the last +//! being the crate-wide ceiling, so under `Wide` the two shapes reach the same +//! number. //! On a 32-bit one the crate-wide ceiling is 2^30 and **both** shapes land -//! there -- `reserving_mpsc`'s packed 2^31 is clamped down to it too -- so the +//! there -- every `reserving_mpsc` layout is clamped down to it too -- so the //! difference disappears entirely and the comparison means nothing at all. //! Either way it counts slots allocated up front rather than items ever pushed, //! and 2^31 slots is tens of gigabytes before the ring holds anything useful. diff --git a/crates/windows-waitable-queues/src/permit_mpsc.rs b/crates/windows-waitable-queues/src/permit_mpsc.rs index d078ab044..041250ca4 100644 --- a/crates/windows-waitable-queues/src/permit_mpsc.rs +++ b/crates/windows-waitable-queues/src/permit_mpsc.rs @@ -26,10 +26,10 @@ //! //! How wide that field is, and so how many pushes recurrence takes, is a layout //! choice there -- 32 bits under the default and up to 64 under -//! [`reserving_mpsc::Wide`](crate::reserving_mpsc). **That moves the recurrence -//! out of reach without removing the separation that causes it**, which is why -//! this shape remains interesting: it addresses the structure rather than the -//! interval. +//! [`reserving_mpsc::Wide`](crate::reserving_mpsc). **That widens the interval +//! before recurrence without removing the separation that causes it**, which is +//! why this shape remains interesting: it addresses the structure rather than +//! the interval. //! //! Here the decision *is* the operation. A producer takes a permit from a count //! of unspoken-for slots with one atomic, and that single modification both @@ -78,7 +78,9 @@ use crate::metrics::Metrics; /// /// **64 bits on every target, deliberately, rather than `usize`**, for the same /// reason [`slotwise_mpsc`](crate::slotwise_mpsc) made the same choice: a -/// 32-bit counter laps in minutes at this crate's measured rates, and a shape +/// 32-bit counter laps in about thirty-seven seconds at the reference rate +/// [`reserving_mpsc::ClaimLayout`] documents -- an arithmetic input taken from +/// another shape rather than a bound on this one -- and a shape /// whose soundness depends on the target's pointer width is not one this crate /// ships twice over. /// diff --git a/crates/windows-waitable-queues/src/reserving_mpsc.rs b/crates/windows-waitable-queues/src/reserving_mpsc.rs index e1cfa4dfc..b9b1c8cd7 100644 --- a/crates/windows-waitable-queues/src/reserving_mpsc.rs +++ b/crates/windows-waitable-queues/src/reserving_mpsc.rs @@ -20,17 +20,22 @@ //! silent**: the consumer receives a different item than was sent, and no error, //! panic, or counter reports it. //! -//! Under `Balanced`, 2^32 pushes is 37 seconds to about four minutes of -//! *sustained* pushing at this crate's measured rates, roughly two minutes at -//! two producers. The wrap alone is not enough -- a producer must also stall +//! Under `Balanced`, 2^32 pushes is about 37 seconds of +//! *sustained* pushing at the rate [`ClaimLayout`] discloses; two producers is +//! the smallest count that can trigger it. That rate predates a correction to the probe's timing window, +//! so it is a floor rather than a forecast -- the correction lowers the rate +//! and lengthens the horizon, so the figures below name a shorter one than it gives; +//! see [`ClaimLayout`]. The wrap alone is not enough -- a producer must also stall //! inside a window a few instructions wide -- but a preemption suffices. //! //! **[`ClaimLayout`] is how far away that is.** [`Perpetual`] moves it to 2^56 //! pushes, about twenty years at the same rate, for the cost of a reservation -//! ceiling of 255 and nothing measurable besides -- it is the same exchange on -//! the same word, differing only in shift constants. [`Enduring`] sits between -//! them, and the `dwcas` feature adds a 128-bit word that removes the -//! recurrence outright. +//! ceiling of 255 -- it is the same exchange on +//! the same word, differing only in shift and mask constants, though what that costs in +//! throughput is not established (see [`ClaimLayout`]). [`Enduring`] sits between +//! them, and the `dwcas` feature adds a 128-bit word whose 64-bit position moves +//! the recurrence to 2^64 pushes -- about 5,000 years at the rate +//! [`ClaimLayout`] documents, which is a longer horizon rather than no horizon. //! //! ``` //! use windows_waitable_queues::reserving_mpsc::{self, Perpetual}; @@ -41,7 +46,8 @@ //! ``` //! //! The default stays `Balanced` so that introducing the choice changed no -//! existing caller's behaviour; it is not the recommended layout. +//! existing caller's behaviour; under it the queue can silently lose an item past +//! 2^32 pushes from two or more producers. //! [`slotwise_mpsc`](crate::slotwise_mpsc) does not have this hazard under any //! layout, its positions being 64 bits on every target; [`spsc`](crate::spsc) //! never had it. The full statement is in the [crate documentation](crate). @@ -56,11 +62,26 @@ //! on **every** push, including the pushes that never reserve anything -- which //! is what `slotwise_mpsc` avoids and why it cannot offer reservation at all. //! -//! **That cost is not what makes either shape slower.** This one measured -//! *faster* than `slotwise_mpsc` under contention on both architectures tried, by up to -//! 6.4x, because the slot sequence `slotwise_mpsc` reads instead marches through memory -//! while other producers write it. See the crate documentation for the numbers -//! and for how to choose. +//! **What that read costs is not established, and this comparison cannot +//! establish it.** This shape measured *faster* than `slotwise_mpsc` under +//! contention on the hosts tried -- an end-to-end result over the complete push +//! path, in which the consumer-position read is one term among the protocol, +//! metadata and retry costs and is not separated from them. So the honest +//! statement is that the extra read did not stop this shape being the faster of +//! the two here; it is **not** that the read is cheap, nor that it is "not what +//! makes either shape slower", which is a causal negative the measurement cannot +//! support either. The magnitude belongs with the capture that produced it +//! rather than here, so see the crate documentation's attributed table for the +//! figures and the conditions they were taken under. +//! +//! An earlier version of this paragraph gave a figure ("by up to 6.4x") taken +//! from a two-host comparison that has since been withdrawn for predating a +//! correction to the probe's timing window, and attributed the difference to +//! `slotwise_mpsc`'s slot sequence marching through memory while other producers +//! write it. That mechanism is plausible and is **not** established: the probe +//! times the complete push and cannot isolate or bound that read, so the causal +//! claim went further than the measurement supports. The direction survives; the +//! magnitude and the cause do not. //! //! `slotwise_mpsc`'s producer never reads the consumer's position. It asks a different //! question -- "is the slot I am about to claim free?" -- and reads that from @@ -75,8 +96,12 @@ //! line every thread in the system touches. //! //! So the two ship as peers ([D-16](../DESIGN-NOTES.md#d-16)): `slotwise_mpsc` for a -//! caller who wants the cheapest possible push and can treat a refusal as -//! backpressure, this shape for a caller with a message it must not lose. That +//! caller who can treat a refusal as backpressure, this shape for a caller with +//! a message it must not lose. The distinction is what each offers, not which +//! one is quicker -- an earlier version of this sentence said `slotwise_mpsc` +//! was for "a caller who wants the cheapest possible push", which asserts a cost +//! ordering this crate does not establish and which its own end-to-end +//! measurement did not find. That //! is the narrow-trait argument from [D-2](../DESIGN-NOTES.md#d-2) reaching //! its sharpest case -- `slotwise_mpsc` does not implement //! [`Reserving`](crate::Reserving) because it genuinely cannot, not because @@ -113,26 +138,43 @@ //! //! # What the packing costs, and what it does not //! -//! Splitting a 64-bit word 32/32 caps this shape at -//! a maximum of 2^31 items, and that split is forced rather than chosen: +//! Splitting a 64-bit word 32/32 caps `Balanced` at +//! a maximum of 2^31 items on a 64-bit target -- 2^30 on a 32-bit one, where +//! the crate-wide ceiling binds first -- and that split is forced *given one +//! premise*: //! a position of `b` bits keeps a wrapping difference unambiguous only up to -//! `2^(b-1)`, and the count needs `b` bits because it can reach the capacity, so -//! `b + b = 64` gives `b = 32`. There is no cleverer division of the word. +//! `2^(b-1)`, and if the count must be able to reach the capacity it needs `b` +//! bits too, so `b + b = 64` gives `b = 32`. Given that requirement there is no +//! cleverer division of the word. +//! +//! **[`ClaimLayout`] drops the requirement rather than the arithmetic.** Capping +//! outstanding reservations well below capacity -- 65,535 under [`Enduring`], 255 +//! under [`Perpetual`] -- frees the position to take 48 or 56 bits, which is +//! where the deeper layouts come from. So the derivation above describes +//! [`Balanced`] and the capacity-filling case, not every layout this module +//! offers. //! -//! **A 128-bit compare-and-swap is deliberately not used *here*** -//! ([D-37](../DESIGN-NOTES.md#d-37)). It would not remove the cost that -//! matters -- the consumer's position still has to be read -- and 2^31 slots is -//! a ring this shape allocates in full at construction. +//! **The 128-bit word ships as a layout, not as a separate shape** +//! ([D-37](../DESIGN-NOTES.md#d-37), amended by +//! [D-41](../DESIGN-NOTES.md#d-41)). An earlier plan put it in a shape of its own +//! called `reserving_mpsc_wide`; that shape was never built, and this paragraph +//! described it as forthcoming for longer than the plan survived. //! -//! The operative reason is that widening *this* shape's word would change what -//! it offers depending on the target: `i686-pc-windows-msvc` has no lock-free -//! 128-bit exchange, so the same module would be lock-free on one target and -//! silently mutex-backed on another. A wider claim ships instead as its own -//! shape (`reserving_mpsc_wide`, not yet built -- see D-37), to exist only -//! where the exchange is genuinely lock-free. That keeps *this* module's -//! contract the same on every target, which is the property being protected -//! here: a caller who wants 2^62 slots and no wrap hazard will ask for it by -//! name rather than get it by accident of where they compiled. +//! The reason for keeping it out of the default is unchanged: widening this +//! shape's word unconditionally would change what the module offers depending on +//! the target, because `i686-pc-windows-msvc` has no lock-free 128-bit exchange +//! and neither does an x86-64 build without `cmpxchg16b`. On such a target this +//! crate does not silently substitute a lock: the `portable-atomic` dependency +//! is taken with `default-features = false`, which is load-bearing precisely +//! because its defaults *would* supply a global lock and still compile. With +//! them off, `AtomicU128` does not exist there and enabling `dwcas` fails the +//! build naming it. So the hazard an unconditional wide word would carry is a +//! module that is lock-free on one target and mutex-backed on another; the +//! hazard the feature gate actually trades it for is a build that stops. So +//! `Wide` (which exists only under `dwcas`, so this names it without linking) is +//! reached by naming it, and the +//! narrow word's contract is identical on every target -- a caller gets the wide +//! one by asking, never by accident of where they compiled. use core::cell::{Cell, UnsafeCell}; use core::fmt; @@ -163,30 +205,67 @@ use crate::options::Options; /// them is therefore a trade, and this trait is where a caller chooses which /// side to spend them on. /// -/// **The two things being traded are not equally valuable, and the shipping -/// default spends the bits on the less valuable one.** The reservation count -/// bounds how many messages may be held in flight at once -- in practice the -/// number of producers mid-send, so hundreds or thousands. The position decides +/// **What the two halves buy is different in kind, and which matters depends on +/// the deployment.** The reservation count +/// bounds how many messages may be held in flight at once. That bound is the +/// lesser of the ring's capacity and the layout's count field, and it is +/// reachable by a *single* producer: [`Producer::reserve`] takes `&self` and +/// returns an owned [`Reservation`], so one thread can hold as many as the +/// field allows. (An earlier version of this paragraph said the practical bound +/// was "the number of producers mid-send, so hundreds or thousands", and called +/// the reservation half "the less valuable one" on that basis. The bound was +/// wrong, and it mattered -- it made the narrower fields look unreachable, which +/// is what made the trade look one-sided. One +/// producer alone fills `Perpetual`'s 255 and is then refused, which +/// `one_producer_alone_can_exhaust_the_reservation_field` pins.) The position +/// decides /// how many pushes occur before it recurs, and a recurrence is the `SH-14.1` /// hazard: a producer descheduled across a full wrap can claim against a /// numerically identical but generations-later value. /// -/// | Layout | reserved / position | Outstanding reservations | Pushes to recurrence | +/// | Layout | reserved / position | Reservation-count field ceiling | Pushes to recurrence | /// |---|---|---|---| -/// | [`Balanced`] | 32 / 32 | 2^32 | 2^32 | +/// | [`Balanced`] | 32 / 32 | 4,294,967,295 | 2^32 | /// | [`Enduring`] | 16 / 48 | 65,535 | 2^48 | /// | [`Perpetual`] | 8 / 56 | 255 | 2^56 | /// +/// A fourth layout, `Wide` (64 / 64 over a `u128`), exists when the `dwcas` +/// feature is enabled; it is omitted from this table because it does not exist +/// in a default build, and named without a link here for the same reason. +/// +/// **The middle column is the field's ceiling, not a reachable number of +/// reservations.** Admission is also bounded by capacity -- `reserve` refuses +/// once the ring has no room beyond the reservations already outstanding -- so +/// the achievable count is the lesser of the two. For [`Balanced`] the capacity +/// bound is the binding one: this layout accepts at most 2^31 slots on a 64-bit +/// target and 2^30 on a 32-bit one, so no more than that many reservations can +/// be outstanding whatever the field could hold. For +/// [`Enduring`] and [`Perpetual`] the field is the smaller of the two only once +/// the queue is at least that large -- a `Perpetual` queue of capacity 64 admits +/// 64 reservations, not 255. The achievable count is always the lesser of +/// capacity and field ceiling; this column is the field half of that pair. +/// /// At this crate's disclosed sustained rate of about 116 million pushes per /// second, those recurrences are roughly **37 seconds**, **28 days**, and /// **20 years** respectively. The rate is the one `reserving_mpsc`'s own hazard /// note quotes; a queue that must drain cannot sustain the fastest rate /// measured, so treat these as a floor on time rather than a forecast. /// -/// **Choosing a deeper position costs nothing measurable.** All three issue the -/// same `lock cmpxchg` on the same `u64` and differ only in shift and mask -/// constants; a probe comparing them found no difference outside noise. The -/// trade is entirely against the reservation ceiling. +/// **That rate premise predates a correction to the probe's timing window**, +/// which had overstated throughput. The correction therefore moves the true +/// sustained rate *down* and these horizons *up*, so the figures above remain a +/// floor -- they say the wrap arrives sooner than it does, which is the +/// corrected rate gives. They have not been recomputed, because +/// the horizon a caller needs is the one on their own hardware and at their own +/// rate; the arithmetic is field width divided by rate. +/// +/// **Choosing a deeper position is the same instruction on the same word.** All +/// three issue the same atomic compare-exchange on the same `u64` and differ +/// shift and mask constants, so there is no structural reason for one to be +/// slower. **What that costs in throughput is not established**: a probe +/// comparing them found them indistinguishable at low producer counts, and at high counts sat outside the probe's same-code control but too close to it to establish an ordering or a cost on this host. The settled trade is the +/// reservation ceiling; throughput is target-dependent and this crate does not +/// characterise it beyond the one host in the note above. /// /// This trait is sealed: the layouts are a fixed set because each one's /// constants are checked against each other at compile time, and a caller @@ -468,13 +547,29 @@ impl ClaimWord for u128 { /// The shipping division: 32 bits each. /// -/// Holds 2^32 outstanding reservations and recurs after 2^32 pushes -- about -/// **37 seconds** of sustained maximum-rate pushing. This is the default -/// because it is what the shape shipped with, not because it is the best -/// choice: the reservation ceiling it buys is far beyond any real use, and it -/// is paid for with the whole of the `SH-14.1` exposure. Prefer [`Enduring`] or -/// [`Perpetual`] unless you genuinely hold more than 65,535 reservations at -/// once. +/// Its reservation-count field tops out at [`u32::MAX`], though capacity binds +/// first: this layout accepts at most 2^31 slots on a 64-bit target and 2^30 on +/// a 32-bit one, so no instance can hold more than that many outstanding +/// reservations whatever the field could encode. It +/// recurs after 2^32 pushes -- +/// about +/// **37 seconds** at the pre-correction planning rate ([`ClaimLayout`] says why +/// that is a floor). Past that point, with two +/// or more producers, the queue can **silently lose an item**: that is the whole +/// of the `SH-14.1` exposure, and this layout carries it. +/// +/// It is the default because it is what the shape shipped with. The reservation +/// field it buys is far beyond any use this crate has seen -- and beyond what +/// its own capacity permits -- while the position half is what sets the +/// exposure. +/// [`Enduring`] and [`Perpetual`] spend that field the other way -- +/// [`Enduring`] holds up to 65,535 outstanding reservations, [`Perpetual`] up to +/// 255 -- each reachable only when the queue's capacity is at least that +/// large -- +/// and `Wide` moves it to 2^64 pushes -- a longer horizon in years, not the absence of +/// one. (`Wide` exists +/// only under the `dwcas` feature, so this names it without linking: an +/// intra-doc link here would not resolve in a default-feature rustdoc build.) #[derive(Debug, Clone, Copy, PartialEq, Eq)] pub struct Balanced; impl sealed::Sealed for Balanced {} @@ -486,8 +581,10 @@ impl ClaimLayout for Balanced { /// A deeper position: 16 bits of reservations, 48 of position. /// -/// Holds 65,535 outstanding reservations and recurs after 2^48 pushes -- about -/// **28 days** of sustained maximum-rate pushing. +/// Its field holds at most 65,535 outstanding reservations -- reachable only +/// when capacity is at least that large, since the achievable count is the +/// lesser of the two -- and its position recurs after 2^48 pushes, about +/// **28 days** at the pre-correction planning rate; see [`ClaimLayout`]. #[derive(Debug, Clone, Copy, PartialEq, Eq)] pub struct Enduring; impl sealed::Sealed for Enduring {} @@ -499,9 +596,13 @@ impl ClaimLayout for Enduring { /// The deepest position: 8 bits of reservations, 56 of position. /// -/// Holds 255 outstanding reservations and recurs after 2^56 pushes -- about -/// **20 years** of sustained maximum-rate pushing, which puts the recurrence -/// beyond any real deployment rather than merely far away. +/// Its field holds at most 255 outstanding reservations -- reachable only when +/// capacity is at least that large, since the achievable count is the lesser of +/// the two -- and its position recurs after 2^56 pushes, about +/// **20 years** at the pre-correction planning rate ([`ClaimLayout`]). That is a +/// longer horizon, not the absence of one, and like every figure in that column +/// it scales with the caller's rate: a deployment pushing an order of magnitude +/// faster reaches it in about two years. /// /// 255 reservations is the whole of the trade, and it is a real limit rather /// than a nominal one: [`Producer::reserve`] returns `None` once that many are @@ -518,16 +619,31 @@ impl ClaimLayout for Perpetual { /// A 128-bit claim word: 64 bits of position, and the count in the other half. /// /// Requires the `dwcas` feature, which is what brings in the `portable-atomic` -/// dependency this crate otherwise does not have. The position needs 2^64 -/// pushes to recur, which no deployment reaches -- not "not for twenty years", -/// but not at all. +/// dependency this crate otherwise does not have. The position is 64 bits, so it +/// recurs after 2^64 pushes -- about 5,000 years at the rate [`ClaimLayout`] +/// documents, against the twenty [`Perpetual`] buys. Longer, not unbounded. /// -/// **Read the cost before choosing it.** The 128-bit exchange measured 2-3x -/// slower than a `u64` one on the claim itself, and the penalty grows with -/// producer count; against a draining consumer the difference is much smaller. -/// [`Perpetual`] reaches about twenty years on a plain `AtomicU64` at no -/// measured cost, so this is worth taking only when a guarantee is wanted in -/// place of an argument about deployment lifetimes. +/// The whole push path was measured as slower under this layout than under a +/// `u64` one at every producer count measured, and the difference **grows with +/// producer count** -- smallest at one or two, several times by thirty-two, in +/// the isolated regime on one x86-64 host. Against a draining consumer the +/// comparison is not settled in either direction: the control there was pooled +/// across producer counts, and it is not independent of producer count, so +/// containment followed from the pooling rather than from the data. The probe +/// times the +/// complete push, so this is the layout's effect on that path and not a +/// measurement of the 128-bit exchange on its own. The per-count table is in the +/// queue-contention section of +/// [DESIGN-NOTES.md](../../windows-platform-probes/DESIGN-NOTES.md), which is +/// the one place it is recorded. +/// +/// [`Perpetual`] reaches about twenty years on a plain `AtomicU64`, and what +/// that costs in throughput is not established -- see [`ClaimLayout`]. What this +/// layout provides that the others do not is a 64-bit position: the recurrence +/// moves to 2^64 pushes -- about 5,000 years at the rate [`ClaimLayout`] +/// documents, rather than the twenty [`Perpetual`] buys. A longer horizon, not +/// the absence of one, and it scales with the caller's rate like the rest of +/// that column. /// /// The reservation ceiling is [`u32::MAX`] rather than the 64 bits the field /// could hold, because the count is reported to callers as a `u32`. @@ -544,6 +660,24 @@ impl ClaimLayout for Wide { const POSITION_BITS: u32 = 64; } +/// The per-layout capacity ceilings the README publishes, pinned to the source. +/// +/// [The README](../README.md) names 2^31, 2^47, 2^55 and 2^62 for the four +/// layouts on a 64-bit target, and the reader is entitled to rely on them. +/// Nothing in a markdown file can fail a build, so the numbers are asserted +/// against `BOUNDS_MAX` here: widening or narrowing a layout's position moves +/// one of these and stops the build, which is the prompt to go and correct the +/// prose. Held to a 64-bit target because on a 32-bit one the crate-wide +/// ceiling binds first and every layout lands on it instead. +#[cfg(target_pointer_width = "64")] +const _: () = { + assert!(::BOUNDS_MAX == 1usize << 31); + assert!(::BOUNDS_MAX == 1usize << 47); + assert!(::BOUNDS_MAX == 1usize << 55); + #[cfg(feature = "dwcas")] + assert!(::BOUNDS_MAX == 1usize << 62); +}; + /// The position after `position`, wrapping at the width the layout gives it. /// /// **Centralised because the width is no longer the type's.** A position is diff --git a/crates/windows-waitable-queues/src/reserving_mpsc/tests.rs b/crates/windows-waitable-queues/src/reserving_mpsc/tests.rs index 7223392d3..9d2ee5cc5 100644 --- a/crates/windows-waitable-queues/src/reserving_mpsc/tests.rs +++ b/crates/windows-waitable-queues/src/reserving_mpsc/tests.rs @@ -1765,3 +1765,39 @@ fn try_iter_and_drain_are_the_same_iterator_and_need_no_import() { let taken: Vec = rx.drain().collect(); assert_eq!(taken, vec![4, 5]); } + +/// A single producer handle can hold many reservations at once, because +/// `reserve` takes `&self` and returns an owned `Reservation`. The +/// reservation-count field is therefore reachable by ONE producer in a loop, +/// and the bound has nothing to do with how many producers exist. +/// +/// This is asserted because the crate's documentation once claimed the opposite +/// -- that reservations were "bounded by how many producers are mid-send" -- +/// which would have made `Perpetual`'s 255 ceiling unreachable in practice. It +/// is reachable by one thread, and this pins that. +#[test] +fn one_producer_alone_can_exhaust_the_reservation_field() { + // Perpetual is 8/56: the count field holds at most 255. + let (tx, _rx) = bounded_as::(1024).expect("a valid capacity"); + + // Bounded deliberately. Were the guard to regress to never refusing, an + // unbounded `repeat_with` would allocate until the process died -- and this + // suite runs its tests as threads in one process, so that takes every other + // test with it. One attempt past the ceiling is enough: the assertion below + // then reports a count one too high instead of the harness disappearing. + let held: Vec<_> = std::iter::repeat_with(|| tx.reserve()) + .take(Perpetual::MAX_RESERVED as usize + 1) + .take_while(Option::is_some) + .flatten() + .collect(); + + assert_eq!( + held.len(), + Perpetual::MAX_RESERVED as usize, + "one producer filled the field to its ceiling, not to a producer count" + ); + assert!( + tx.reserve().is_none(), + "the field is full, so the next reservation must be refused" + ); +} diff --git a/crates/windows-waitable-queues/src/slotwise_mpsc.rs b/crates/windows-waitable-queues/src/slotwise_mpsc.rs index 89b8f2197..d1efe1b3a 100644 --- a/crates/windows-waitable-queues/src/slotwise_mpsc.rs +++ b/crates/windows-waitable-queues/src/slotwise_mpsc.rs @@ -84,7 +84,12 @@ use core::sync::atomic::{AtomicBool, AtomicU64, AtomicUsize, Ordering}; /// counter cannot lap. /// /// With `usize` it can. On a 32-bit target the counter laps after 2^32 claims, -/// which at this crate's measured rates is a matter of minutes: the stalled +/// which is about thirty-seven seconds at the reference rate +/// [`reserving_mpsc::ClaimLayout`] documents. **That rate is `reserving_mpsc`'s, +/// and is used here only as an arithmetic input rather than as a bound on this +/// shape**: this shape's own measured throughput differs, and at low producer +/// counts exceeds it, so the interval is neither a floor nor a forecast for it. +/// The stalled /// producer then sees the same tail bits, succeeds, and writes a slot that has /// since been refilled from the previous lap of the ring. Every other guard in /// this shape holds -- the position really is claimed by exactly one producer; @@ -192,9 +197,17 @@ pub fn bounded(capacity: usize) -> Result<(Producer, Consumer), Capacit /// branch on a field that is written once at construction. /// /// That avoidance is what distinguishes the two multi-producer shapes, but -/// **it is not what makes either one faster**: measurement found this shape the -/// slower of the two under contention, by up to 6.4x. See the crate -/// documentation for the numbers and for how to choose. +/// **it is not what makes either one faster**: in the **isolated** regime, +/// measurement found this shape the slower of the two under contention on the +/// host the crate's table was taken on. The qualifier is load-bearing -- the +/// crate's table is isolated, and the drained captures on the same host have +/// rows going both ways, so an unqualified reading of this sentence is +/// contradicted by the committed data. See the crate +/// documentation's attributed table for the figures and the conditions they were +/// taken under. (An earlier version of this sentence gave "by up to 6.4x", a +/// figure from a two-host capture withdrawn for predating a correction to the +/// probe's timing window, and pointed at the crate documentation "for how to +/// choose"; which shape suits a deployment is the deployment's question.) /// /// # Errors /// @@ -547,10 +560,17 @@ impl Producer { // asked for the answer. // // Note what this property does *not* buy: measurement found this shape - // slower than `reserving_mpsc` under contention despite it, because the - // slot sequence a producer must read instead marches through memory - // while other producers write it. Staying off the shared line is why - // the two shapes are different, not why either is quick. + // slower than `reserving_mpsc` under contention in the ISOLATED regime + // despite it -- the drained capture on the same host has rows going both + // ways, so the unqualified claim is contradicted by committed data. Why + // even the isolated result is + // so is not established -- the probe times the complete push, so the + // sequence read + // is one term among several and is never isolated. An earlier version of + // this comment attributed it to the slot sequence marching through + // memory while other producers write it; that mechanism is plausible and + // unmeasured. Staying off the shared line is why the two shapes are + // different, not a claim about which is quick. // // Off, the cost is one predictable branch on a field written once at // construction, so the line is shared but read-only -- the cheap kind.