diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md index 7fe6b98c9..6bc6c13fd 100644 --- a/.github/copilot-instructions.md +++ b/.github/copilot-instructions.md @@ -1195,6 +1195,28 @@ When you add such a predicate, **verify the binding by sabotage**: change the de confirm the consumer's *behavior* changes. If only a test's expectation moves, or nothing moves, the binding is cosmetic and the copy is still there. +**Sabotage the CLAIM, not the symptom.** When a change asserts that it *removed* a weakness — a +census eliminated, a check made exhaustive, a duplicate collapsed — the sabotage that matters is +the one that would falsify *that* assertion, which is rarely the same as the one that reproduces +the original symptom. This is the failure mode the rule above does not cover on its own: an author +who believes they derived a fact when they only **moved** the restatement will happily sabotage the +old site, watch it go red, and record the class as closed. Measured on one guard in +`windows-platform-probes`, three times in succession — strings, then a hand-written `ALL`, then +generation — each fix relocating the census somewhere harder to see while its commit message +claimed the class was closed, and each caught by a review rather than by its author. Ask what the +commit message asserts, then break *that*. + +**A sabotage is worth nothing once it is discarded**, so record it where something re-runs it +rather than in a terminal you will close. Put it in the component's `sabotage.json` and sweep it +with [tools/run-sabotage.ps1](../tools/run-sabotage.ps1) — see +[tools/README-sabotage.md](../tools/README-sabotage.md) for the manifest format and for why a +manifest with no `expect: "survives"` control can only tell you the tests are sensitive, never that +they are sensitive to the right things. The harness is the authority on whether a patch site is +unique; do not re-implement that check beside it. This is not a preference: the `sabotage-harness` +CI job exists because eleven review rounds on that harness produced thirteen later defects +*introduced by earlier fixes*, "because every verification was a one-off command that was then +discarded and nothing re-checked an earlier guarantee." + Know the limit — but know that it is narrower than it first appears. **Sequencing rules (ordering, bracket entry states, what may follow what) are not value-level, and are still derivable**: define them once as a shared executable oracle — a state machine over the @@ -1434,13 +1456,44 @@ step breakdowns described during planning. When a group of related items is fully complete: 1. Move the completed group to `COMPLETED-CHECKLIST.md` in the same directory. -2. Prefix the moved block with a heading: `## Moved YYYY-MM-DD — `. +2. Prefix the moved block with a heading: + `## Moved YYYY-MM-DD HH:MM:SS ±hh:mm — `, taken from + `Get-Date -Format "yyyy-MM-dd HH:mm:ss zzz"`. Carry the offset, per "Timestamps carry their + offset" below; existing headings without one are fine and are not worth going back to change. 3. `COMPLETED-CHECKLIST.md` is **append-only**; always add new groups at the bottom. 4. Leave only the remaining pending or in-progress items in the source `CHECKLIST.md`. Named feature files (`CHECKLIST-.md`) should be **deleted entirely** once all items are complete. Move their content to `COMPLETED-CHECKLIST.md` in the same directory before deleting. +### Timestamps carry their offset + +**When you write a date into a repository file, make it UTC or give it its offset.** This governs +`## Moved`, `## Resolved`, and the completed-item stamp. It costs a few characters and it is worth +them. + +**This is a presentation concern, not a correctness one.** A timestamp with an offset is +self-describing: two readers in different zones, or one reader on a machine whose zone is set +wrongly, still agree on the instant it denotes. A bare local date leaves that to be inferred. Nothing +is *wrong* with a bare date — the event happened when it happened — it is just ambiguous in a way +that a few extra characters remove. + +Note what the offset does and does not buy. It does **not** prevent a misconfigured machine from +writing a misleading date; it makes that date convertible afterwards. Worked example from this +repository: a laptop reporting `-04:00` while its owner sat in `-07:00` ran three hours fast, so +timestamps taken late in the evening carried the next day's date. Every affected commit was still +unambiguous, because git stores the instant with its offset — `2026-09-13T01:14:28-04:00` is exactly +`2026-09-12 22:14 -07:00`. A bare `2026-09-13` in a checklist was the only thing that had to be +re-derived from elsewhere. + +- **Do not go back and repair bare dates already written**, and do not rewrite history to relabel an + offset. The instants are correct either way, the ambiguity is small, and rewriting changes every + hash — on the branch that prompted this, two tracked files cited commit hashes, one of them a + split's mandatory `Split from ... at ` provenance line, and both would have dangled. Fix such + a date only when something visible actually depends on it. +- **`Get-Date` reports what the machine believes.** If its offset looks implausible for where the + engineer actually is, say so rather than quietly writing the date down. + ## Design note files Any directory in the repository may have a DESIGN-NOTES.md file. diff --git a/Cargo.lock b/Cargo.lock index a26461a7f..539b402e0 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -216,6 +216,8 @@ dependencies = [ name = "windows-platform-probes" version = "0.0.1" dependencies = [ + "serde", + "serde_json", "windows-namespace-request-sys", "windows-placement-probe", "windows-platform-probes", diff --git a/PLANS.md b/PLANS.md index 8eb8844b4..8441a8849 100644 --- a/PLANS.md +++ b/PLANS.md @@ -20,7 +20,7 @@ plans tracker: [crates/windows-file-enumeration-sys/PLANS.md](crates/windows-fil | [CHECKLIST-thread-ambient.md](CHECKLIST-thread-ambient.md) | in progress | M22-M23: extract the captured-context composite into `windows-thread-ambient-sys`, a standalone platform layer that captures a thread's ambient state and applies it on another thread. M24-M26: `windows-namespace-request-sys`, marshalable Win32 namespace call parameter sets, over a round-one entry list audited from three real consumers (this repository's watcher and enumeration crates, and `MikeGrier/Globazog-rs`) rather than guessed. M27: `windows-platform-probes`, a durable home for the measurements this workspace's designs rest on, under a three-tier scheme (asserted / ignored / binary-only) where every tier is compiled by an ordinary build. Feature-scoped and deleted when complete; it is the whole of the `mikegrier/thread-ambient` branch's work, and is deliberately separate from the deferred namespace-facility items in [CHECKLIST.md](CHECKLIST.md). | [crates/windows-thread-ambient-sys/DESIGN-NOTES.md](crates/windows-thread-ambient-sys/DESIGN-NOTES.md) | | [crates/windows-overlapped-io-sys/CHECKLIST.md](crates/windows-overlapped-io-sys/CHECKLIST.md) | not started | M14: finish the contract audit -- categories 1, 2, 6, 8, 9 were not examined -- and sweep `outstanding()` for the advisory-predicate hazard. | [crates/windows-overlapped-io-sys/DESIGN-NOTES.md](crates/windows-overlapped-io-sys/DESIGN-NOTES.md) | | [crates/windows-ioring-sys/CHECKLIST.md](crates/windows-ioring-sys/CHECKLIST.md) | in progress | Memory-safe Rust over the Windows `IoRing` submission/completion ring, as a new crate. M1-M7 (ring lifecycle through the `ring-copy` topology-aligned sample) are complete and archived. The parked, pinned-thread `M6+` work and the new M10 contract audit remain. | [crates/windows-ioring-sys/DESIGN-NOTES.md](crates/windows-ioring-sys/DESIGN-NOTES.md) | -| [crates/windows-platform-probes/CHECKLIST.md](crates/windows-platform-probes/CHECKLIST.md) | in progress | M1 (streaming reports) is done and archived: every probe now writes into the sink as it measures, through a `fmt::Write` adapter that left all 332 `writeln!` call sites untouched, and the `catch_unwind`/`resume_unwind` pair is gone because there is no longer a buffer to rescue. Measured with a control -- a probe killed 300 ms into a 0.8 s run keeps its banner and heading on six runs of six, where the previous build kept nothing on six of six. M2 remains: check correspondence *between* a report's parts, which is the defect class no per-part instrument in this crate can see. | [crates/windows-platform-probes/DESIGN-NOTES.md](crates/windows-platform-probes/DESIGN-NOTES.md#d-streaming-report) | +| [crates/windows-platform-probes/CHECKLIST.md](crates/windows-platform-probes/CHECKLIST.md) | in progress | M1 (streaming reports) is done and archived: every probe now writes into the sink as it measures, through a `fmt::Write` adapter that left all 332 `writeln!` call sites untouched, and the `catch_unwind`/`resume_unwind` pair is gone because there is no longer a buffer to rescue. Measured with a control -- a probe killed 300 ms into a 0.8 s run keeps its banner and heading on six runs of six, where the previous build kept nothing on six of six. M2 built the report oracle -- one executable definition of the correspondences between a report's prose and NDJSON halves, bound inside the renderers so every test that renders inherits it -- along with a derived fact set and a corpus of report shapes. It is complete; its ten unrelated leftovers -- CI hygiene, a doc repair, probe-prose corrections -- were re-sequenced into M4 (gated on M3) and M5 (gated on nothing). M3 then supersedes its central rule. Re-reading M2's own evidence showed that both defects which motivated the oracle were defects in the ENCODED ROW, not in the relation between two renderings, and that the row published its three diagnostic lists as bare counts -- so a survey reading `"parse_incomplete":1` could not tell a probe self-bug from host flakiness. The row is the machine contract and gets the facts and the invariants; the prose is for a reader and gets review. M3 is complete and archived (ten items): those three fields publish arrays of OBJECTS, each carrying a stable `code` plus the values its variant holds -- `{"code":"partitioning_summary_missing","level":9}` rather than the bare `"partitioning_summary_missing"` of the superseded M3.1 form; the surviving correspondences became invariants over the observation rather than over two renderings, so a rule that reads the diagnostic lists -- which would be a restatement of `verdict()` and blind to a deleted push site -- was rewritten to read the observation; the row is emitted from a typed value through one writer with total escaping, which is the crate's only defence against caller text reaching the mined artifact; the prose oracle and every parser serving it were deleted, and no test extracts structured data from prose anywhere in the crate. Four later items came from reviews and are the more instructive half: three instruments were found asserting less than their names claimed, `BlockingState::ALL` was found to be a census the compiler did not check despite a doc comment claiming it did, and the row's hand-written JSON well-formedness check was measured against a real parser over 1807 generated corruptions -- 159 disagreements, every one a FALSE accept -- and replaced by `serde_json`, after which the remaining hand-written string scanners were deleted too. What remains is M4 (four M2 leftovers M3 gated, now unblocked and re-scoped) and M5 (six ungated hygiene items). | [crates/windows-platform-probes/DESIGN-NOTES.md](crates/windows-platform-probes/DESIGN-NOTES.md#d-streaming-report), [#d-encoded-row-is-the-contract](crates/windows-platform-probes/DESIGN-NOTES.md#d-encoded-row-is-the-contract) | | [CHECKLIST-mutation-survivors.md](CHECKLIST-mutation-survivors.md) | not started | Work queued from the workspace-wide cargo-mutants sweep of 2026-09-02, whose findings are kept in [mutation-sweeps/2026-09-02/](mutation-sweeps/2026-09-02/README.md) rather than re-derived -- the run took roughly fourteen hours. 2,792 caught, 1,112 survived, 198 timed out. **The headline numbers mislead in three ways and the README says how**: a timeout in a blocking-API crate is usually a detection that lost its name rather than a gap (measured: one of `windows-waitable-queues`' 120 timeouts fails four tests in 0.00s when re-injected alone), a low score on an executable probe crate is measuring the wrong thing, and three kinds of survivor -- equivalent mutants, unreachable code, and constants that want a `const` assertion -- are not missing tests at all. M1 covers the shipping crates; M2 holds the two crates that are not libraries and whose scope is an engineer's decision; M3 re-runs and prunes rather than hand-editing the tool's output into a second source of truth. | [mutation-sweeps/2026-09-02/README.md](mutation-sweeps/2026-09-02/README.md) | Add a row here when new work is planned, against [CHECKLIST.md](CHECKLIST.md) or any crate's. diff --git a/crates/windows-ioring-sys/tests/handover.rs b/crates/windows-ioring-sys/tests/handover.rs index 40baf6935..89023636a 100644 --- a/crates/windows-ioring-sys/tests/handover.rs +++ b/crates/windows-ioring-sys/tests/handover.rs @@ -63,6 +63,18 @@ const CHUNKS: usize = 8; const CHUNK_LEN: usize = 512; const WAVES: usize = 3; +/// Why a host cannot run these tests, named once because two sites say it. +/// +/// A `const` rather than the literal at each site because the literal does not +/// FIT at the deeper one: nested in a loop it pushes the line past `max_width`, +/// rustfmt responds by leaving the whole expression alone, and the result is +/// mis-indented code that `cargo fmt --check` reports as clean. Found by a +/// review, which read the misalignment as a formatting failure that would break +/// CI -- it would not, and that is the more interesting half: rustfmt giving up +/// is silent, so the only guard here is the shorter line. +const NEEDS_COMPLETION_EVENT: &str = + "this host must report IORING_FEATURE_SET_COMPLETION_EVENT to run the handover tests"; + /// Generous, because a positive wait must not flake on a loaded machine. /// Every test that pays it in full is one that would otherwise hang. const SIGNAL_TIMEOUT_MS: u32 = 5_000; @@ -82,12 +94,32 @@ type Pending> = HashMap>; const DIRECT_OPS: usize = 8; const DIRECT_LEN: usize = 1024 * 1024; -/// Attempts at catching the unbuffered reads mid-flight. A handful, because -/// each one issues `DIRECT_OPS * DIRECT_LEN` of real device I/O; the test -/// asserts that *at least one* attempt reached the in-flight state rather than -/// requiring every attempt to, so an unlucky one cannot make it flake. +/// Attempts at catching the unbuffered reads mid-flight, at each width in +/// [`DIRECT_WIDTHS`]. The test asserts that *at least one* attempt reached the +/// in-flight state rather than requiring every attempt to, so an unlucky one +/// cannot make it flake. const DIRECT_ATTEMPTS: usize = 4; +/// How many reads to have in flight, escalating until one attempt catches them. +/// +/// **Four attempts at one width was not enough, and the reason is worth stating +/// because it is not the obvious one.** Eight 1 MiB unbuffered reads cannot +/// finish inside the ~6us an attach takes on an idle machine -- measured, with +/// zero of eight landed. What defeats the test is not a fast device but a +/// DESCHEDULED THREAD: if this thread loses its quantum between `submit` and +/// `completion_event`, the reads have milliseconds to finish and the attempt +/// degenerates. A shared CI runner does that occasionally, and four attempts in +/// a row were unlucky once. +/// +/// More attempts at the same width only buys more coin flips against the same +/// coin. Widening the flight buys HEADROOM: at 64 reads the device has eight +/// times the work to get through, so a stall has to be eight times longer to +/// beat it. So escalate rather than merely repeat, and stop at the first width +/// that works -- the common case still costs one attempt at eight. +/// +/// Bounded by the ring, which is created with 64 submission entries. +const DIRECT_WIDTHS: [usize; 4] = [8, 16, 32, 64]; + /// `FILE_FLAG_NO_BUFFERING` requires the buffer address, the file offset, and /// the length to be sector-aligned. 4096 satisfies both 512e and 4Kn devices. const ALIGN: usize = 4096; @@ -320,9 +352,7 @@ fn an_attach_serves_both_the_backlog_and_the_wave_that_follows_it() { // signal can account for it. submit_wave(&mut ring, &file, 0, &mut contract, &mut pending); - let event = ring.completion_event().expect( - "this host must report IORING_FEATURE_SET_COMPLETION_EVENT to run the handover tests", - ); + let event = ring.completion_event().expect(NEEDS_COMPLETION_EVENT); // Wave 1 lands *after* the attach, into a queue wave 0 already made // non-empty -- so it raises no edge of its own and is only ever seen by a @@ -429,60 +459,166 @@ fn attaching_while_unbuffered_reads_are_still_in_flight_strands_nothing() { let handle = file.as_raw_handle(); let mut caught_in_flight = false; - - for attempt in 0..DIRECT_ATTEMPTS { - let mut ring = IoRing::new(64, 64).expect("create ring"); - let mut contract = RingContract::new(); - let mut pending: Pending = Pending::new(); - - { - let mut batch = Batch::new(&mut ring); - for index in 0..DIRECT_OPS { - let buffer = Aligned::new(DIRECT_LEN); - let offset = (index * DIRECT_LEN) as u64; - // SAFETY: `file` outlives every operation queued here -- this - // attempt drains to completion before the next one starts, and - // the handle lives for the whole test. - let token = unsafe { batch.read_raw(handle, buffer, offset, PushOptions::new()) } - .expect("queue unbuffered read"); - contract.observe_push(token.id()); - pending.insert(token.id(), token); + // **What each attempt observed, kept so a failure can say WHY it + // degenerated rather than only THAT it did.** + // + // This guard fires when every read landed before the completion event was + // attached, and the interesting question is then which side moved: did the + // reads get faster, or did the attach get slower? Observed once in CI on a + // shared runner while passing on an idle developer machine, where the answer + // could not be recovered from the failure message at all -- it reported the + // conclusion and none of the evidence, so the only way to investigate was to + // re-run and hope. + // + // `submit_to_attach` is the number that usually settles it. It spans exactly + // the window this test depends on: the reads are in flight for it, so a + // large value means the attach was starved rather than the device being + // quick. + let mut trace: Vec<( + usize, + usize, + usize, + std::time::Duration, + std::time::Duration, + )> = Vec::new(); + + 'widths: for width in DIRECT_WIDTHS { + for attempt in 0..DIRECT_ATTEMPTS { + let mut ring = IoRing::new(64, 64).expect("create ring"); + let mut contract = RingContract::new(); + let mut pending: Pending = Pending::new(); + // Assigned once inside the block below and read after it, so no + // initial value is needed -- and giving it one would be a value + // nothing reads. + let submitted; + + { + let mut batch = Batch::new(&mut ring); + for index in 0..width { + let buffer = Aligned::new(DIRECT_LEN); + // Wraps, so a wider flight re-reads the fixture rather than + // needing a proportionally larger one. This test cares only + // that the reads are real and outstanding, never what they + // return. + let offset = ((index % DIRECT_OPS) * DIRECT_LEN) as u64; + // SAFETY: `file` outlives every operation queued here -- this + // attempt drains to completion before the next one starts, and + // the handle lives for the whole test. + let token = + unsafe { batch.read_raw(handle, buffer, offset, PushOptions::new()) } + .expect("queue unbuffered read"); + contract.observe_push(token.id()); + pending.insert(token.id(), token); + } + // **Started BEFORE the submit, not after it.** The reads begin + // executing inside `submit_and_wait`, so a clock started once it + // returns omits part of the very window the precondition depends + // on -- and a read that finished during the call is invisible to + // it. Measuring from here spans every instant a read could have + // used. + submitted = std::time::Instant::now(); + batch.submit_and_wait(0, 0).expect("submit without waiting"); } - batch.submit_and_wait(0, 0).expect("submit without waiting"); - } - let event = ring.completion_event().expect( - "this host must report IORING_FEATURE_SET_COMPLETION_EVENT to run the handover tests", - ); + let event = ring.completion_event().expect(NEEDS_COMPLETION_EVENT); + let attached = std::time::Instant::now(); + + // Non-blocking, so this measures what the attach actually found rather + // than waiting for a state to develop. + let already_queued = drain_to_empty(&mut ring, &mut contract, &mut pending); + trace.push(( + width, + attempt, + already_queued, + attached.duration_since(submitted), + // The NON-BLOCKING poll only. `wait_and_drain` runs after this + // and is not included, so the label says `polled` rather than + // `drained`: the earlier name claimed the whole drain, and a + // reader chasing a slow one would have been misled by a number + // that never contained it. + attached.elapsed(), + )); + // **Why this proves the precondition, stated because a review read + // it the other way round.** The concern was that a read finishing + // between the attach and this poll makes the test pass while only + // exercising the already-completed case. It cannot, and the + // direction is what settles it. + // + // `drain_to_empty` loops until `try_pop` reports the queue EMPTY -- + // no cap, no early exit -- so `already_queued` is the total observed + // at a moment strictly AFTER the attach. A read that finishes in + // that window is therefore COUNTED, which pushes `already_queued` + // toward `width` and makes this branch LESS likely to be taken. The + // already-completed case it warns about is exactly the case where + // all `width` are drained and the flag is never set. + // + // So the error this can make is a false NEGATIVE, never a false + // positive -- and completion is monotonic, so a read outstanding at + // the (later) poll was outstanding at the (earlier) attach. The + // escalation over widths and attempts exists for the false + // negatives. + // + // The one assumption is that `try_pop` reports emptiness truthfully. + // A ring that claimed empty while holding completions would forge + // this precondition -- but that is a defect in the crate under test, + // and `contract.assert_quiescent()` below is what would catch it. + if already_queued < width { + caught_in_flight = true; + } - // Non-blocking, so this measures what the attach actually found rather - // than waiting for a state to develop. - let already_queued = drain_to_empty(&mut ring, &mut contract, &mut pending); - if already_queued < DIRECT_OPS { - caught_in_flight = true; + wait_and_drain( + &mut ring, + &event, + &mut contract, + &mut pending, + width - already_queued, + &format!( + "width {width}, attempt {attempt}, {already_queued} already queued at attach" + ), + ); + + assert!( + pending.is_empty(), + "width {width}, attempt {attempt}: a token was never claimed" + ); + contract.assert_quiescent(); + + // The state was reached, and every assertion above has now run + // against it. Wider flights would only cost device I/O to + // re-establish what this one already showed. + if caught_in_flight { + break 'widths; + } } - - wait_and_drain( - &mut ring, - &event, - &mut contract, - &mut pending, - DIRECT_OPS - already_queued, - &format!("attempt {attempt}, {already_queued} already queued at attach"), - ); - - assert!( - pending.is_empty(), - "attempt {attempt}: a token was never claimed" - ); - contract.assert_quiescent(); } + let observed = trace + .iter() + .map(|(width, attempt, queued, to_attach, to_polled)| { + format!( + " {width} reads, attempt {attempt}: {queued}/{width} already queued at attach; \ + submit+attach {to_attach:?}, attach->polled {to_polled:?}" + ) + }) + .collect::>() + .join("\n"); + let widest = DIRECT_WIDTHS[DIRECT_WIDTHS.len() - 1]; + assert!( caught_in_flight, - "no attempt caught a read in flight: every unbuffered read had already completed by the \ - time the event was attached, so this test degenerated into the already-queued case and \ - is no longer covering what it claims" + "no attempt caught a read in flight at ANY width: every unbuffered read had already \ + completed by the time the event was attached, so this test degenerated into the \ + already-queued case and is no longer covering what it claims.\n\n\ + Escalated to {widest} reads of {DIRECT_LEN} bytes, {DIRECT_ATTEMPTS} attempt(s) per \ + width:\n{observed}\n\n\ + **Do not simply re-run.** A busy machine is already accounted for -- that is what the \ + escalation is for, since widening the flight multiplies the stall needed to beat it. \ + Reaching the widest row above means the stall outlasted {widest} reads, which a shared \ + runner does not usually manage, or the device now resolves them faster than one \ + `completion_event` call. Read `submit+attach`, which spans from just before the \ + submit to the completed attach: large says the ATTACH was starved and \ + this host is pathologically loaded; small says the DEVICE won, and the fix is a larger \ + `DIRECT_LEN` rather than more attempts." ); drop(file); diff --git a/crates/windows-platform-probes/CHECKLIST.md b/crates/windows-platform-probes/CHECKLIST.md index 830acc3af..68d9862ea 100644 --- a/crates/windows-platform-probes/CHECKLIST.md +++ b/crates/windows-platform-probes/CHECKLIST.md @@ -5,45 +5,76 @@ separately, in the workspace [CHECKLIST-thread-ambient.md](../../CHECKLIST-threa M27; that file is feature-scoped and is deleted when its feature completes, so durable follow-up work for the crate belongs here instead. -## M2 -- Check correspondence between the report's parts, not just each part - -A pull-request review found a state where [src/topology_report.rs](src/topology_report.rs) printed -`BUG IN THIS PROBE ... Nothing below about cache partitioning can be trusted` while `cross_check` had -no branch for that state, so the verdict could print `=> agree` two paragraphs below. Twenty-eight -rounds of per-artifact review and a zero-surviving-mutant `cargo-mutants` result had both passed over -it, because every function involved was correct on its own terms and the defect lived in the relation -between two of them. - -See [DESIGN-NOTES.md](DESIGN-NOTES.md) -> [The defects that survived were correspondence -failures](DESIGN-NOTES.md#d-correspondence-failures) for why each instrument was structurally -incapable of finding it, and for the matrix-as-exploration / oracle-as-durable split this milestone -implements. - -Every correlation the oracle admits is one the report already renders twice, found either by -catching a contradiction or by walking the artifact field by field. It is not a -speculative list to extend by imagination -- one is added when a contradiction is found, and the -authoritative set is the `Correspondence` enum rather than any count written here. (This said "the -three correlations below" while the enum already had four.) - -- [x] **M2.1** -- Add a report oracle to this crate: one shared executable definition of the correlations that must hold between the parts of a rendered report. -> [completed 2026-09-10](COMPLETED-CHECKLIST.md#m21) - -- [x] **M2.2** -- Route every test that renders a report through the oracle, by binding it in the renderer. -> [completed 2026-09-10](COMPLETED-CHECKLIST.md#m22) - -- [x] **M2.3** -- Run `measure()` against the real host, render the report, and apply the oracle. -> [completed 2026-09-10](COMPLETED-CHECKLIST.md#m23) - -- [ ] **M2.4** -- Explore, with the sparse matrix as the instrument, whether the same correspondence - failures exist for `Coherence`, `BracketOutcome` and `Verdict`, and in the sibling probes' renderers. - Expect the matrix to be mostly empty; that is the expected shape and not a sign the exercise failed. - **Record the vacuous results as well as the findings** -- "X and Y were examined and need not - correspond" is what stops the next person re-exploring the same cells, and is the half that normally - evaporates. Promote only what proves meaningful into the oracle from M2.1. +## M4 -- Carried over from M2: the items M3 gated + +These were written under M2 and were blocked on M3, which is **now complete and archived** in +[COMPLETED-CHECKLIST.md](COMPLETED-CHECKLIST.md). They are unblocked, and each was RE-SCOPED rather +than merely delayed -- so each item below carries its own re-scoping note, in the item, where +somebody executing the list will actually meet it. + +**The IDs keep their M2 numbers deliberately.** [COMPLETED-CHECKLIST.md](COMPLETED-CHECKLIST.md) is +append-only and its entries are immutable, and two archived entries already cite M2.4 and M2.14 -- +so renumbering would leave dangling references in a file that may not be edited to repair them. +Stable IDs cost a mismatch between an item number and its milestone; renumbering would cost +correctness in the archive. + +- [ ] **M2.4** -- Explore, with the sparse matrix as the instrument, whether `Coherence`, + `BracketOutcome` and `Verdict` carry invariants the row does not yet publish -- as VALUES on the + observation, not as correspondences between two renderings -- and whether the sibling probes' + renderers have the same gaps. Expect the matrix to be mostly empty; that is the expected shape and + not a sign the exercise failed. **Record the vacuous results as well as the findings** -- "X and Y + were examined and need not be related" is what stops the next person re-exploring the same cells, + and is the half that normally evaporates. Promote only what proves meaningful into the invariant + set from M3.2. + + Re-scoped by M3.2; the note at the top of this milestone gives the reasoning. **The item text + above was rewritten when M3 was archived, to match**: it still asked for "the same correspondence + failures" and for promotion "into the oracle from M2.1", both retired by M3, so a reader working + the list linearly would have been sent after the half that no longer exists. Found by a review -- + and the lesson generalises, since a re-scoping note 25 lines above an item does not reach someone + executing the item. > **-> OPEN QUESTION for the engineer:** M2.4 may show this generalises past this crate, in which case > the oracle belongs somewhere shared and the question becomes a repository-wide convention rather than > a probe-crate one. That is a design decision, not a mechanical follow-on, and is deliberately left > unanswered here. -- [ ] **M2.5** -- Make the banner describe the read the body describes. A probe run performs +- [ ] **M4.1** -- Model the observation-readable `ParseIncomplete` conditions that + `blocking_states` currently omits, so a deleted push site in `cross_check` is caught for all of + them rather than for the subset. + + **Gap:** `blocking_states` justified its absentees as "derived counts whose only source IS the + cross-check's own arithmetic". That is false for most of them -- + `CacheLevelsWithoutPartitions`, `NumaDomainsOnlyInCpuSets`, `CoresOnlyInCpuSets`, + `RelationsWithoutProcessors`, `UnreportedRelations`, `DescribedRelations`, + `OverlappingWalkRelations`, `ProcessorAttributeConflicts`, `NumaDomainsWithConflictingLabels`, + `NumaDomainsUnreported` and `MeasuredButCountsAbsent` all read fields sitting on `Observation` + in plain sight. Deleting one of those push sites lets `verdict()` reach `agree` with + `blocking_states` silent. Reported across three review rounds against two wordings of the claim; + the claims in [src/topology/invariant.rs](src/topology/invariant.rs) and in + `every_numa_counter_branch_...` were narrowed in the same review round that reported this, + to stop overstating the coverage, + which is why this item is the fix rather than the discovery. + + **Target:** each gains a `BlockingState` variant, a `blocking_states` branch, a `codes_for` arm, + a perturbation in the invariant tests, and a corpus shape in + [tests/a_real_report_agrees_with_itself.rs](tests/a_real_report_agrees_with_itself.rs) so + `the_corpus_reaches_every_blocking_state` still holds. **Do not add a variant without its corpus + shape** -- that test is what stops the mapping being written and never exercised, which is the + defect this whole area keeps producing. + + Apply the tautology test to each before adding it: a state whose only source is the cross-check's + own arithmetic does NOT belong, and the honest outcome for such a one is a line in the module + header saying so by name rather than a silent absence. + +- [ ] **M2.5** -- Make the banner describe the read the body describes. + + Gated by M3.1 and M3.3, both landed: establishing that the middle of three discoveries agreed + produces a new FACT, which M3.1 says must reach the row rather than only the banner, and M3.3 + changed how the banner is built. Written before those, it would have been written into machinery + about to move. + + A probe run performs **three** independent `MachineMemoryTopology::discover()` calls: `Fingerprint::discover()` for the banner, `measure()`'s own discovery for the body, and `Fingerprint::discover()` again. `attribution` compares only the two endpoints, so equal endpoints print an unqualified banner without establishing @@ -65,16 +96,72 @@ three correlations below" while the enum already had four.) comparison that is itself new prose able to drift. The endpoint reads still earn their place: they catch structural change across the wider window that the counter bracket cannot see. - **This belongs to M2 rather than beside it:** "the banner describes the measured read" is a - correspondence invariant, so it should be expressed in the M2.1 oracle and checked on every rendered - report, not asserted once in a single test. + **Express it where M3 put the invariants, not in the M2.1 oracle.** "The banner describes the + measured read" is an invariant over the OBSERVATION, so it belongs in the invariant set from M3.2 + and, if the fact reaches the artifact, in the row schema -- checked on every rendered report through + the renderer binding rather than asserted once in a single test. **These two paragraphs were + rewritten when M3 landed**: they asked for the relation to be expressed in the M2.1 prose oracle, + which M3.4 deleted, so an executor would have gone looking for machinery that no longer exists. + Found by a review, and the same defect M2.4 carried. Reviewer disagreement is recorded deliberately, because it is evidence about the instrument rather than noise: across two rounds one reader raised this twice while two others cleared it, one of them explicitly after being pointed at the question. Nothing in the suite decides it either way, which is - itself the argument for the oracle. + itself the argument for making it an invariant rather than a test. + +- [ ] **M2.15** -- Run the probe suite on a second architecture in CI. + + **Keeps its conclusion but loses its evidence.** The five failures cited below were all + `prose: "x86_64"` against `ndjson: "x86"` -- instances of exactly the correspondence M3.4 + retired, so they can no longer occur and a re-run now looks clean. The point stands without them: + CI builds `aarch64` and never tests it, and architecture is the one shape dimension a corpus + cannot vary because it is fixed at compile time. Restate it on that basis when picked up. + + A reviewer asked whether the suite was portable and it was not: three renderer fixtures and the + shape corpus' banner builder each hard-coded `x86_64` while the row they are compared against + publishes `std::env::consts::ARCH`. Measured on `i686-pc-windows-msvc`: five failures, every one + `prose: "x86_64"` against `ndjson: "x86"`. CI BUILDS `aarch64` and never TESTS it, so a + build-and-clippy matrix cannot see this class at all. + + Architecture is the one shape dimension the M2.12 corpus cannot vary, because it is fixed at + compile time rather than chosen per report -- so the corpus that exists precisely to defeat shape + blindness is blind here by construction, and only a second test target can close it. Add one + (`i686-pc-windows-msvc` runs natively on the existing runners; `aarch64` would need its own). -- [x] **M2.6** -- Say what `GetFullPathNameW` does, in the crate that owns it, and whether it stays. -> [completed 2026-09-09](COMPLETED-CHECKLIST.md#m26) +- [ ] **M2.17** -- Cross the corpus dimensions instead of varying one at a time. + + Re-scoped by M3.5: the dimensions worth crossing are the ROW's. Crossing prose shapes that have + since stopped being checked would have aimed at the retiring half. + + [tests/a_real_report_agrees_with_itself.rs](tests/a_real_report_agrees_with_itself.rs)'s `shapes()` + builds each shape by taking `base()` and changing ONE thing. That makes every shape easy to read and + is why the corpus found what it found -- but it means any renderer branch selected by TWO + dimensions at once is unreachable by construction, and the corpus cannot report the gap because it + does not know the branch exists. + + Measured: `CrossCheck` tags a `parse_incomplete` entry `- {caveat}` under `INCOMPLETE` but + `(parse incomplete) {caveat}` under `DISAGREE`. Anomalies appeared only in an agreeing-counter + shape and disagreements only in a zero-anomaly shape, so the second spelling was never rendered, + and the anomaly-count rule was silently unread on every disagreeing report -- while the comment + above it said it was read for every verdict. One hand-written crossed shape closed it, and the + accounting test went red under sabotage only once that shape existed. + + Enumerate the dimensions the renderer actually branches on (verdict, bracket, coherence, the + partitioning arm, presence of each diagnostic list) and generate the cross product, or a pairwise + covering set if the full product is too slow. The corpus already asserts self-consistency and runs + the fact accounting per shape, so nothing new has to be written to check them -- only to produce + them. Until this lands, a shape that needs two dimensions must be added by hand, which is exactly + the imagination-driven process M2.12 exists to replace. + + +## M5 -- Carried over from M2: unblocked hygiene + +**Nothing gates these.** They are grouped last by priority, not by dependency -- none of them touches +the report pipeline, so any of them may be pulled forward ahead of M4 at any time. They were +discovered during M2 and parked there under a heading none of them fit. M3, which gated M4 but never +gated these, is complete and archived. + +IDs keep their M2 numbers, for the reason given under M4. - [ ] **M2.7** -- Decide whether the other nine probe steps in CI should carry `if: '!cancelled()'`, and apply or record the decision. @@ -136,10 +223,6 @@ three correlations below" while the enum already had four.) and cheap. Same defect class as PR #86's subject -- a claim stated more strongly than the evidence supports -- so whichever is chosen, the wording has to end up matching what the numbers can carry. -- [x] **M2.10** -- Derive the oracle's set of checked facts from the renderer instead of extending it by hand. -> [completed 2026-09-10](COMPLETED-CHECKLIST.md#m210) -- [x] **M2.11** -- Compare the `outermost_partitioning_cache` discriminator against the prose conclusion. -> [completed 2026-09-10](COMPLETED-CHECKLIST.md#m211) - -- [x] **M2.12** -- Validate the oracle and its instruments against a corpus of report SHAPES generated from the renderer. -> [completed 2026-09-11](COMPLETED-CHECKLIST.md#m212) - [ ] **M2.13** -- Lint the completed-checklist archive mechanically in CI. Three bookkeeping defects reached review on this branch, and all three are decidable by a script: a @@ -151,32 +234,12 @@ three correlations below" while the enum already had four.) `- [ ]` remains; and the file has ZERO deleted lines against the merge base. The last one is the append-only invariant, and it is the one a human reviewer is least likely to notice. -- [ ] **M2.14** -- Write two authoring rules into the repository instructions, both earned on this - branch. - **State the invariant, not the census.** "14 keys read, 3 unread" added nothing that "every key is - classified" does not, and it was wrong -- written by eyeballing a list rather than counting it, in - the commit documenting a fix for exactly that defect class. Where a number is genuinely load - bearing, it must come from a command run in the same action that writes it. +- [x] **M2.14** -- Make the two authoring rules this branch earned actually bite. -> [completed 2026-09-13](COMPLETED-CHECKLIST.md#m214) - **A new test is not done until it has been observed to fail.** Every vacuous test on this branch was - written green and stayed green until a reviewer thought to break something: a guard that matched a - violation's VARIANT where only its FACT established the point, and a fixture whose `.replace()` of - `[1]` matched nothing because the report rendered `[0]`. Sabotage belongs at authoring time, not at - review time. +- [x] **M2.14.1** -- Give this crate a sabotage manifest, so "observed to fail" is a recorded artifact rather than a habit. -> [completed 2026-09-13](COMPLETED-CHECKLIST.md#m2141) -- [ ] **M2.15** -- Run the probe suite on a second architecture in CI. - - A reviewer asked whether the suite was portable and it was not: three renderer fixtures and the - shape corpus' banner builder each hard-coded `x86_64` while the row they are compared against - publishes `std::env::consts::ARCH`. Measured on `i686-pc-windows-msvc`: five failures, every one - `prose: "x86_64"` against `ndjson: "x86"`. CI BUILDS `aarch64` and never TESTS it, so a - build-and-clippy matrix cannot see this class at all. - - Architecture is the one shape dimension the M2.12 corpus cannot vary, because it is fixed at - compile time rather than chosen per report -- so the corpus that exists precisely to defeat shape - blindness is blind here by construction, and only a second test target can close it. Add one - (`i686-pc-windows-msvc` runs natively on the existing runners; `aarch64` would need its own). +- [x] **M2.14.2** -- Add to CONTRACT INTEGRITY rule 1 the one thing this branch learned that it does NOT already say. -> [completed 2026-09-13](COMPLETED-CHECKLIST.md#m2142) - [ ] **M2.16** -- Repair the garbled `Report` doc comment, and drop the two counts that have already rotted beside it. @@ -193,51 +256,6 @@ three correlations below" while the enum already had four.) so the two must be fixed together or they drift apart again. Replace them with the invariant the passage is actually arguing -- that `String` already implements `fmt::Write`, so every existing write site stands untouched and only the renderer signatures move -- which is what makes the point - and cannot rot. This is the same defect class as M2.14's first authoring rule. - -- [ ] **M2.17** -- Cross the corpus dimensions instead of varying one at a time. - - [tests/a_real_report_agrees_with_itself.rs](tests/a_real_report_agrees_with_itself.rs)'s `shapes()` - builds each shape by taking `base()` and changing ONE thing. That makes every shape easy to read and - is why the corpus found what it found -- but it means any renderer branch selected by TWO - dimensions at once is unreachable by construction, and the corpus cannot report the gap because it - does not know the branch exists. - - Measured: `CrossCheck` tags a `parse_incomplete` entry `- {caveat}` under `INCOMPLETE` but - `(parse incomplete) {caveat}` under `DISAGREE`. Anomalies appeared only in an agreeing-counter - shape and disagreements only in a zero-anomaly shape, so the second spelling was never rendered, - and the anomaly-count rule was silently unread on every disagreeing report -- while the comment - above it said it was read for every verdict. One hand-written crossed shape closed it, and the - accounting test went red under sabotage only once that shape existed. - - Enumerate the dimensions the renderer actually branches on (verdict, bracket, coherence, the - partitioning arm, presence of each diagnostic list) and generate the cross product, or a pairwise - covering set if the full product is too slow. The corpus already asserts self-consistency and runs - the fact accounting per shape, so nothing new has to be written to check them -- only to produce - them. Until this lands, a shape that needs two dimensions must be added by hand, which is exactly - the imagination-driven process M2.12 exists to replace. - -- [ ] **M2.18** -- Decide whether a banner should be a TYPE rather than a `&str`. - - **This is a design decision for the engineer, not a defect to fix in passing.** A review observed - that `is_attribution_shaped` recognises a SHAPE, not a provenance: any two `host:` lines followed - by the exact disclaimer pass through `preamble` verbatim. Since `report` and `report_unmeasured` - both take `&str`, there is a public path where caller text decides a renderer-owned question. - Measured: a hand-built banner of `host: 999p/1c` / `host: 1p/1c` / the disclaimer - renders verbatim, and the oracle's exemption for unestablished attribution then skips the - banner-against-body processor-count check -- so the banner suppressed a correspondence. - - **The honest scope of it.** The suppression is not silent: the report visibly states that its two - readings disagree, which is exactly the condition under which declining to compare counts is - CORRECT. The oracle reads the artifact, and the artifact says so. Every production caller composes - its banner with `attribution()`, so nothing reaches this by accident today. Validating the - per-line shape more strictly does not close it either -- `attribution` legitimately emits - `host: UNKNOWN -- topology discovery failed: {error}` with arbitrary error text, so arbitrary - text can always ride inside a well-formed banner line. - - The fix that would actually close it is a typed banner with a private constructor, so only - `attribution()` can produce one and the renderer's signature carries the guarantee. The cost is - every fixture and corpus shape that builds a banner by hand, plus a test-only escape hatch that - partially reopens the hole for the tests that need odd banners. Worth doing if the renderer's - input contract is meant to be enforced rather than documented; not worth doing if `&str` in, and - containment on the way out, is the intended boundary. Raise before implementing. + and cannot rot. This is the same defect class as CONTRACT INTEGRITY rule 1 in + [.github/copilot-instructions.md](../../.github/copilot-instructions.md), which M2.14 exists to + make bite. diff --git a/crates/windows-platform-probes/COMPLETED-CHECKLIST.md b/crates/windows-platform-probes/COMPLETED-CHECKLIST.md index 820fe6d2f..93044d433 100644 --- a/crates/windows-platform-probes/COMPLETED-CHECKLIST.md +++ b/crates/windows-platform-probes/COMPLETED-CHECKLIST.md @@ -495,4 +495,622 @@ request as it was written, and quotes the module doc as it read before the corre is now `every_fact_is_accounted_for_on_every_representative_shape`. Both said "every" while `shapes()` is a sample, so a green run read as proof of coverage it does not have. The old names are left standing above because this file is history; this line is how a reader following them - finds where they went.)* \ No newline at end of file + finds where they went.)* + +## Moved 2026-09-12 -- M2 completes: the report oracle, its fact set and its shape corpus + +M2's own work is done. Its completed items -- M2.1 (the oracle), M2.2 (the renderer binding), +M2.3 (the real-host test), M2.6 (`GetFullPathNameW`), M2.10 (the derived fact set), M2.11 (the +partitioning discriminator) and M2.12 (the shape corpus) -- were each archived above as they landed, +so their stubs in [CHECKLIST.md](CHECKLIST.md) carried nothing this file does not already hold and +were deleted with the milestone. + +**The ten open items were re-sequenced, not reworked.** They had accumulated under a heading none of +them fit -- a CI `if:` condition and a doc-comment repair are not correspondence work -- and they +split by whether [DESIGN-NOTES.md](DESIGN-NOTES.md) -> +[#d-encoded-row-is-the-contract](DESIGN-NOTES.md#d-encoded-row-is-the-contract) gates them: + +- **M4** (gated on M3): M2.4, M2.5, M2.15, M2.17. +- **M5** (gated on nothing): M2.7, M2.8, M2.9, M2.13, M2.14, M2.16. +- **M2.18 is dissolved** into M3.3 rather than moved. It asked whether a banner should be a type; + M3.3 answers the general form of that question, and answering the banner alone would have typed + one parameter while leaving the shape everywhere else. + +**Their IDs deliberately keep the `M2.` prefix.** This file is append-only and its entries are +immutable, and two entries above already cite M2.4 and M2.14 -- so renumbering would leave dangling +references here that may not be edited to repair them. A stable ID costs a mismatch between an item +number and its milestone heading; renumbering would cost correctness in the archive. + +*(Recorded 2026-09-12 18:45:27 -04:00. This entry closes a milestone rather than completing an item, +so it carries no `###` item heading and nothing links to it by anchor.)* + +## Moved 2026-09-12 -- M3: the encoded row became the contract, and the prose stopped being checked + + +Decided in [DESIGN-NOTES.md](DESIGN-NOTES.md) -> [The encoded row is the contract; the prose is +not](DESIGN-NOTES.md#d-encoded-row-is-the-contract), from the session in +[design-sessions/DESIGN-SESSION-2026-09-12-what-the-oracle-should-read.md](design-sessions/DESIGN-SESSION-2026-09-12-what-the-oracle-should-read.md). + +The row is a machine contract mined across a fleet; the prose is for a reader. They carry different +obligations -- the row must be **correct**, enforced by machine; the prose must be **accurate and +readable**, enforced by review. Nothing is required to hold *between* them. + +Re-checked against the code rather than against M2's account of it: the original defect was fixed, +and what it left behind was larger. The row published `not_compared`, `parse_incomplete` and +`enumeration_anomalies` as **counts**, where the prose printed each entry's text. A survey reading +`"parse_incomplete":1` could not tell *the probe detected a bug in itself* from *a core record +contradicted itself* from *this topology was not measured from a running machine*. **The row was +impoverished relative to the prose** -- the artifact that gets mined carried less than the artifact +that gets read. M3.1 has since closed that particular gap; the rest of the milestone is about which +artifact carries the contract, and stands whole. + +**M2 completed with this decision**, and its ten open items were re-sequenced rather than reworked. +The milestone's own work -- the oracle, the binding, the real-host test, the derived fact set, the +partitioning discriminator and the shape corpus -- is done and archived. The leftovers had +accumulated under a heading none of them fit, and they split by whether M3 gates them: four are in +M4 below, six in M5. M2.18 is the exception, dissolved rather than moved. + +- **M2.18 (typed banner) is dissolved into M3.3**, not carried over. It was the smallest instance + of "should a report be a value a writer renders, or a string the renderer concatenates", and + answering it alone would have typed one parameter while leaving the shape everywhere else. +- **The per-item re-scoping notes for M2.4, M2.5, M2.15 and M2.17 now live in M4's preamble**, not + here. They were written during this milestone but they instruct work that is still open, and a + pending instruction does not belong in an append-only archive nobody may edit to correct it. +- **M2.7, M2.8, M2.9, M2.13, M2.14 and M2.16 are gated by nothing** and are in M5. M2.9 (a + cross-host ratio called "the finding") and M2.14 (making two authoring rules bite) are if anything + reinforced: + under this decision prose accuracy is a review obligation rather than a machine-checked one, which + puts more weight on both. + +- [x] **M3.1** -- Publish each diagnostic as itself, not as a count. + + `not_compared`, `parse_incomplete` and `enumeration_anomalies` reach the row as + `check.parse_incomplete.len()` and its two siblings, so the fact that a mining pass most needs -- + *which* condition occurred -- exists only in prose. Publish the entries, and give each a stable + machine-readable discriminant rather than the human sentence, so a survey can group by condition + without matching on English that is free to be reworded. The sentences stay in the prose, where + rewording them is harmless. + + **The rule this establishes, which is the durable half:** a renderer may not tell a reader + something the row cannot tell a survey. A cardinality is not a statement of the fact. + + **Done.** Three enums in `topology::diagnostic` -- 21 + 6 + 3 variants, one per condition -- each + carrying its data, rendering its sentence through `Display`, and naming itself through `code()`. + `CrossCheck`'s three `Vec` became `Vec`, and the row publishes arrays of codes + where it published `.len()`. The prose is byte-identical: the loops write `{entry}` and `Display` + emits the same sentences. + + **The wire format changed**, deliberately and not additively: `"parse_incomplete":1` is now + `"parse_incomplete":["partitioning_summary_missing"]`. The count is still available as the list's + length, so nothing is lost, and publishing both would be a restatement that can drift. Same shape + as the `efficiency_classes` correction that preceded it. + + Sabotage-verified, each mutation injected on its own line and reverted: renaming + `PartitioningSummaryMissing`'s code reddens only + `the_row_names_the_probes_own_bug_when_it_detects_one`; making the row keep only the first + condition reddens the two list tests, through the bound oracle's count rule; mislabelling + `TrailingBytes` reddens only `an_anomaly_reaches_the_row_as_its_kind`. + + **The substring-to-variant conversion cost two assertions their discrimination, found by review.** + `c.contains("no online processors and processor groups")` became + `matches!(c, MeasuredButCountsAbsent { .. })`, which holds when the entry names only ONE of the + two -- exactly what the test forbids -- and the loop's labels stopped being asserted at all. + Measured: with `absent` truncated to its first entry the whole suite stayed green at 249 passed. + Both now assert the variant's `absent` payload, and both were observed to fail -- the truncation + reddens the both-absent test, and swapping the two names reddens both. The general lesson is that + converting an assertion from a substring to a variant DROPS whatever the substring discriminated + inside the payload; the variant is the weaker claim unless the payload comes with it. + + **Which conditions are listed is deliberately not compared against the prose.** The code and the + sentence come from one variant, so there is no second implementation to disagree through -- the + correspondence holds by construction, which is stronger than a check. What remains checkable, and + is checked, is that both renderings list the same NUMBER. Found while converting the accounting + instrument: a mutation that swapped one code for another went unnoticed on the + `verdict incomplete` shape, because the oracle reads the length. `corruptions` now APPENDS a code + rather than substituting one, so the length always differs. + +- [x] **M3.2** -- Assert the surviving correspondences as invariants on the observation, before + rendering. + + Alarm-against-verdict, diagnostics-against-verdict and counters-against-verdict are the three + oracle rules that survive the decision. They stop being comparisons of two rendered texts and + become predicates over `Observation` and `CrossCheck` -- `SummaryMissing` implies the verdict is not + `agree`, a non-empty `parse_incomplete` implies the verdict is not `agree`, and so on. No parser is + involved, and the check runs whether or not anything was rendered. + + Each one must be sabotage-verified on arrival: delete the invariant, confirm the suite reddens, + restore it. A predicate that cannot fail is the failure mode this crate keeps meeting. + + **Done, and the item's own framing was wrong in a way worth recording.** It named + "diagnostics-against-verdict" and "counters-against-verdict" as rules to move. Two of those read + `CrossCheck`'s lists -- and `verdict` is a pure function of those lists, so such a rule restates + the definition, cannot fail for any input, and CANNOT CATCH A DELETED PUSH SITE: the deletion + empties the list, the rule sees nothing, and the verdict is `agree` legitimately. Written that + way first, with three tests that asserted acceptance under violation-sounding names. + + Every rule now reads the OBSERVATION. `blocking_states` names ten states that forbid an agreeing + verdict, each with a push site in `cross_check` that it does not consult, plus the two counter + rules. `check` takes the verdict rather than deriving it, so a test can supply the answer a + broken `cross_check` would give -- otherwise every branch is reachable only by editing the source + and a green run says nothing. + + Bound at `observe` (every observation MEASURED, rendered or not -- what this item asked for) and + at `report` (every observation RENDERED, which on the test side is most of them, since the suite + builds observations by hand). Not in `cross_check`, which would recurse. + + Sabotage: deleting the `PartitioningSummaryMissing` push reddens four tests, two of them new -- + the invariant's own accounting test, and a render test through `assert_holds` at the renderer + binding. The invariant is not the sole detector for that push site; its value is the nine others, + several of which have no dedicated test. + +- [x] **M3.3** -- Emit the row from a typed value through one writer. + + > **-> PREREQUISITE: M3.4 lands first.** The reason is on M3.4: this item's nested per-entry data + > makes `ndjson_list_len` silently miscount, so the parsers it would break should be gone before + > the row changes shape rather than taught a shape they are about to lose. + + The row is built today by interpolating every value positionally into a `concat!` template. + Two defect classes follow from that construction and both are closed by replacing it, not by + checking it: + + **Injection.** Measured on PR #88: an `io::Error` containing `{` was selected as the report's + machine-readable row, so the oracle checked the caller's text instead of the probe's. Caller text + reaching the mined artifact is contamination of the contract. + + **Field order and labelling.** A field's name and its value are related only by counting + positions, so a reordered argument or a miscounted placeholder yields mislabelled data that + still parses, which nothing downstream can detect. Stated as the coupling rather than as a + count of placeholders: that count was written twice and wrong twice within an hour. + + A typed row struct plus a single writer that escapes strings makes both unrepresentable. Write the + writer here rather than adding a serialization dependency -- this crate has none and the row is + one flat object. + + **Carry each diagnostic's DATA, which M3.1 left behind.** M3.1 publishes a condition's code but + not the values its variant holds -- a survey learns `contradictory_cores` without learning that + three cores contradicted themselves. The variants already carry those values, for `Display`; what + stopped M3.1 publishing them is that the row is still a positional `concat!` template, where a + nested per-entry object has to be hand-assembled. Once the row is typed this is a field like any + other. Not deferred for want of a consumer -- the shape of the row is the blocker, and it is this + item. This subsumes M2.18: the banner becomes a typed field like any other, and the + question of who may construct one is answered by the row's constructor rather than separately. + + **Done.** `crate::row` holds a `Value` and a `Row` whose members are name-and-value pairs, with + one writer that escapes strings. Both defect classes are now unrepresentable rather than + detected: a name and its value move together or not at all, and a `Value::Text` cannot end the + string it is in. + + Each diagnostic publishes its DATA through `published()`, so a survey learns + `{"code":"contradictory_cores","count":3}` rather than the code alone -- what M3.1 had to leave + behind because the row was a positional template. Anomalies carry `source` and `offset` too: + the same kind at the same offset across a fleet is a different finding from the same kind + scattered, and neither is visible from a count. + + `report_unmeasured` goes through the same writer, and that is the shape that most needed it -- + it is the only renderer that interpolates caller text, a failed discovery's `io::Error`. The + error now reaches the row as a `discovery_error` field, so a survey can group failures by cause + instead of parsing the prose sentence. + + The key-set check M3.4 deferred here now exists -- but NOT in the form M3.4 predicted, and the + first attempt at it was vacuous. See the correction recorded under M3.7. + + **Two silent behaviour changes were caught by checking the old code rather than trusting the + rewrite.** `PartitioningCache` has FIVE variants, not the four a rewrite naturally reaches for; + and `SummaryMissing` publishes its level rather than `null` -- which matters precisely because + that arm is the report telling a reader the probe has a bug, and WHICH level went unchecked is + what they need. + + Sabotage-verified: removing the quote escape reddens three row tests, including the + brace-injection one. The clean row is byte-identical to what the template produced, confirmed + against a real `probe-topology` run. + +- [x] **M3.4** -- Retire the prose-against-row correspondences and the parsers that serve only them. + + > **-> DO THIS BEFORE M3.3, and leave both IDs where they are.** M3.3 carries each diagnostic's + > data, which turns the flat code arrays into arrays of OBJECTS -- and `ndjson_list_len` splits on + > `,`, documented as safe for flat code arrays and nothing else. Pointed at + > `[{"code":"contradictory_cores","cores":3}]` it counts members rather than entries and returns 2 + > for one entry. It does not fail; it silently answers wrong, and every prose-comparison rule then + > compares that against the prose. Running M3.3 first therefore means teaching parsers a nested + > shape and deleting them one item later, with a silent-wrong-answer window in between. The IDs + > stay put because renumbering costs more than the mismatch, the same trade as M4/M5. + > + > Intended order for the rest of M3: **M3.2 -> M3.4 -> M3.3 -> M3.5**. + + > **-> CODE REVIEW RESUMES HERE.** Reviews are paused by the engineer's decision of 2026-09-12 + > until this crate no longer depends on prose as the oracle's subject, and this is the item that + > ends that dependence. The reasoning: a large share of PR #88's fifteen fix commits were defects + > in the prose-reading machinery -- the multibyte panic in `processors_in_banner`, `trim_matches` + > collapsing `[[0]]` and `[0]`, `prose_field` selecting the wrong line -- and every one of them is + > code this item deletes. Reviewing it closely is polishing something already scheduled for + > demolition. + > + > Recorded with the honest counterweight, so the decision can be re-judged on evidence rather than + > re-argued: of the six findings across the two reviews run on 2026-09-12, none was a defect in + > the prose oracle. Two were documentation drift, one was a test that had lost its + > discrimination, and the most valuable -- `disagreements` reaching the prose and not the row at + > all -- was about the ROW being incomplete and survives this item untouched. + + Of 38 top-level functions in [src/report_oracle.rs](src/report_oracle.rs), ten are correspondence + rules, four are comparison helpers, and **twenty-three exist only to extract values back out of + rendered text**. With M3.2 and M3.3 landed, that extraction layer has no remaining consumer. + + What stays is a thin check that the row is **well-formed** -- it parses, it carries the expected + key set, and it is the only such line in the report. That is not a correspondence; it is the + writer's own output being checked, and the writer is the one place structure cannot check itself. + + Retire, do not merely stop calling. Dead extraction helpers left in place are a second grammar for + a format that no longer has two readers. + + **Done, together with M3.5, because they cannot be separated.** The fact-accounting instrument is + built entirely on `report_oracle::check` and the `Correspondence` variants, so deleting the + correspondences leaves it measuring nothing and the suite red between the two items. Committed as + one commit citing both IDs, per the checklist rule for coupled items, rather than split into a + commit that does not pass. + + Measured: `report_oracle.rs` 79,394 -> 8,874 bytes, its tests 105,299 -> 5,598, the integration + instrument 71,007 -> 25,689. All eight prose correspondences and all twenty-three extraction + helpers are gone. + + What survives is the row's well-formedness: exactly one machine-readable line, brackets balanced + (string-aware, because a failed discovery's `io::Error` is interpolated into a string value and an + OS message is free to contain a bracket), and no repeated top-level key. That last one is the + malformation that survives a consumer's parse and changes what it reads, since most JSON readers + take the last. + + (The bracket check was weaker than this sentence implies -- it counted depth, so a trailing or + misplaced separator passed. Strengthened in M3.7.) + + **The key-set check is deliberately NOT here.** Asserting it needs a list of expected keys, and a + list written here is a census -- this component re-corrected the same census three times in one + day. M3.3 makes the row a typed value, at which point the key set is derivable from the type + rather than declared beside it. Moved there rather than approximated here. + + (**The second sentence is wrong, and M3.7 corrects it.** A key set is NOT derivable from a typed + row: the type says "a row is a map of names to values", which is satisfied by every key set, + including the one missing a field. The census this note was right to fear is a count; a schema is + not one, and refusing to write it down bought nothing.) + +- [x] **M3.5** -- Re-aim the shape corpus and the fact accounting at the row. + + **The instrument enumerates in one direction only, and the other direction is where M3.1's rule + lives.** `ndjson_keys` reads the ROW's keys and requires each to be classified, so it asks "does + anything read this key?" -- never "does the prose state a fact the row omits?". A fact with no key + is outside the set of things it can have an opinion about. + + Measured, and this is how it was found rather than reasoned: `CrossCheck::disagreements` reached + the prose as a listed entry per disagreement and reached the row as nothing at all. `cross_check` + said `disagree` without saying WHICH counter did, which is the same shape as the defect the + milestone came from. It survived 41 review rounds, a zero-survivor mutation sweep and the fact + accounting, because every one of those instruments starts from what the row publishes. A review + found it by reading the enum and asking who called `code()` -- the answer was nobody. + + So the accounting needs a second enumeration, from the PROSE's facts to the row's keys, or the + rule "a renderer may not tell a reader something the row cannot tell a survey" has no instrument + behind it and holds only as long as someone remembers it. + + **Done, with M3.4, and the second enumeration exists.** The instrument no longer asks "which prose + facts does the oracle read" -- there are none. It asks, for every state `topology::invariant` knows + forbids agreement, whether the row publishes a condition for it; and it holds the row's published + conditions against what the cross-check found, across the corpus. + + (As first written this said the second rule held the row against a count of PROSE lines, which it + did at the time. The follow-up commit that removed the last prose parsing replaced that with the + comparison against the cross-check -- recorded further down this same item, so the item disagreed + with itself. Found by a review.) + + Sabotage-verified against the defect that motivated it: dropping `disagreements` from the row -- + the omission that survived 41 review rounds, a zero-survivor mutation sweep and the old accounting + -- now reddens the rule that holds the row against the cross-check. That rule was named + `the_row_lists_a_condition_for_every_diagnostic_the_prose_lists` when this evidence was recorded + and is `the_row_lists_exactly_the_conditions_the_cross_check_found` now; the sabotage was re-run + against the current name. Recorded evidence that cannot be re-run as written is evidence nobody + will re-run. + + **One asymmetry, found by the instrument rather than reasoned.** Counting all four lists against + prose lines failed: the prose folds every anomaly into ONE + `windows-topology-sys recorded N enumeration anomal...` sentence while the row lists one code per + anomaly, so three anomalies read as two dropped entries. `enumeration_anomalies` counts on its own + axis and is checked against the OBSERVATION -- one published code per anomaly recorded -- which is + the artifact the row owes fidelity to. Checking it against the number inside that sentence would + be the prose-reading this milestone retired. + + Both publication rules carry a corpus guard, because both skip a shape in no blocking state and a + drifted all-healthy corpus would leave them green while checking nothing. + + **The last prose parsing in the matrix is gone.** M3.5 left one site: a rule that filtered + rendered lines by prefix, counted them, and compared that number against the row -- the only place + left where the test matrix obtained structured data by reading sentences. It had a unit-test twin + in `src/tests.rs` that the first sweep missed and a second, wider sweep found. + + Both are replaced by the same claim against `cross_check`: the row's codes must EQUAL the + cross-check's, in order. Strictly stronger -- a count catches only a dropped entry, this catches a + drop, a reorder and a substitution -- and it never reads a sentence. It also covers all three + lists, which the prose count could not: under INCOMPLETE the renderer gives `not_compared` and + `parse_incomplete` the same bare `- ` prefix, so only their total was recoverable from prose. + + **The ordering half was vacuous, and the guard is what found it.** Reversing the row's + `parse_incomplete` order reddened nothing: every corpus shape varied one dimension, so each landed + at most one entry per list, and a one-element list has no order to get wrong. A first version of + the guard summed the three lists and passed while the sabotage still did nothing -- one entry in + each of two lists is two conditions and no order. Corrected to measure the largest SINGLE list, + and a `several conditions at once, in one list` shape added. The reorder now reddens. + + What remains that touches rendered text at all: selecting the row line, and asserting positional + containment -- the banner is the first line, the banner is one line, there is exactly one row. + None reads prose for its content. + + + [tests/a_real_report_agrees_with_itself.rs](tests/a_real_report_agrees_with_itself.rs) enumerates + the facts a report publishes and measures, by mutation, which are read. The instrument is sound and + the target changes: enumerate the row's fields, and require each to be read by an invariant or + explicitly classified as unread. Its corpus of shapes keeps its purpose -- it exists to defeat the + imagination-driven fixture, which the decision does not change. + + The prose half becomes a rendering test: the renderer emits what it is supposed to emit, judged on + its own terms rather than against the row. + +- [x] **M3.6** -- Split [DESIGN-NOTES.md](DESIGN-NOTES.md) into Tier 1 and Tier 2. + + Measured: 88 KiB, which is **XL** on the repository's byte scale, and the default posture at XL is + to split unless the module is indivisible. It is not -- it carries current decisions and a large + volume of how-we-got-here reasoning, which is exactly the Tier 1 / Tier 2 fracture the repository + instructions describe. + + Move the rationale to `DESIGN-RATIONALE.md`, cross-referenced by decision anchor, leaving Tier 1 + stating what was decided and what forced it. The decision added by this milestone is written to be + split that way already, so it is the worked example rather than the hard case. + + **Done, and the result is still XL -- say so rather than imply otherwise.** 25,942 bytes moved; + DESIGN-NOTES.md went 92,467 -> 67,807, which is over the 64 KiB threshold still. The split was + made at the one unambiguous Tier 2 fracture rather than trimmed to hit a number. + + The fracture: the correspondence-oracle investigation. It is Tier 2 on both tests -- a record of + how a decision was reached rather than a statement of one, AND a decision since superseded by + [#d-encoded-row-is-the-contract](DESIGN-NOTES.md#d-encoded-row-is-the-contract). Moving it also + resolved latent drift: it cites `Correspondence`, the fact-accounting instrument and the prose + rules, none of which survived M3.4 and M3.5. As history those sentences are accurate; as Tier 1 + they described deleted code. + + Pure relocation, verified byte-for-byte against the pre-split file (439 lines, identical). The + two moved anchors are kept in Tier 1 beside a pointer, so existing links land somewhere that says + where the content went, and every in-repo reference was repointed at the content. + +> **-> OPEN QUESTION for the engineer:** the remaining bulk of DESIGN-NOTES.md is neither current +> decisions nor rationale -- it is FINDINGS, measurements about Windows that are this crate's actual +> product (the completion-port fork, the thread-agnosticism probe, the x64 comparison, the long-path +> pair, the topology cross-check). They do not belong in a rationale file, and filing them as +> decisions is what keeps Tier 1 XL. Whether they want a tier of their own is a structural choice +> about this component's documentation scheme, so it is raised rather than taken. + +- [x] **M3.7** -- Make four instruments as strong as their names claim. + + A review of the completed M3 found no wrong behaviour and four weak instruments -- tests and + guards whose names assert a property they could not actually fail to satisfy. That is the + recurring defect class of this whole branch, so the four are recorded with what each one was + measured to miss. + + **1. The row's key set was unenforced.** M3.3's note above claimed the check "is derived: + `Row::keys` reads the value, and a test asserts the reader and the writer agree". Both halves + read the same `Row`, so the test says only that the writer is self-consistent. Measured: deleting + `.with("packages", ...)` from the renderer left the ENTIRE suite green -- a field silently + vanishes from every downstream survey and nothing objects. Fixed by declaring + `MEASURED_ROW_KEYS` / `UNMEASURED_ROW_KEYS` as the contract the renderer is held to. This is not + the census M3.4 feared: a count is derivable from the thing it counts, so restating it is drift + waiting to happen; a schema is NOT derivable from the row, which is exactly why writing it down + buys something. + + **2. The well-formedness oracle accepted invalid JSON.** `balanced()` counted bracket depth, so + `{"a":1,}` (trailing separator) and `{"a":1]` (mismatched closer) both passed -- and a consumer + would reject both. Replaced by `malformation()`, a typed delimiter stack that also checks + separator placement. Sabotage-verified: making the writer emit a leading separator produces a + balanced but invalid row, which the old check passed and the new one reddens. + + (**That replacement was itself replaced, by M3.9.** The typed delimiter stack was a second + hand-written opinion about what JSON is, and a generated test found 159 more rows it accepted + and a real parser rejected.) + + **3. A sabotage asserted only that it had sabotaged.** The publication-accounting sabotage stripped + a condition from the report and then asserted the condition was absent -- which is a fact about + the string edit, not about the rule. It would have passed with the rule deleted. The rule is now + `publication_holds(observation, text)`, and the sabotage asserts it REJECTS the stripped report + and ACCEPTS the original. + + **4. A completeness guard compared a table against itself.** `blocking_states` returned strings, + and the guard that checked every blocking state was described derived both sides from that one + table. Introduced a `BlockingState` enum with `ALL`, so the guard holds the table against the + type's variants and a new state that nobody describes fails to build past it. + + (**The last clause was false, and M3.8 corrects it.** `ALL` was a hand-written array; nothing + tied it to the enum.) + +- [x] **M3.8** -- Make `BlockingState::ALL` exhaustive by construction rather than by assertion. + + The same defect as M3.7's fourth finding, one level up, and introduced by the fix for it. The + doc on `ALL` claimed "an exhaustive list the compiler checks: adding a variant without adding it + there fails to build". That is not what the compiler checks. The `match` in `described()` is + exhaustive-checked, which is what made the claim look right -- but it forces a new variant to + acquire an ARM, never an ENTRY in a separate array. + + Measured, not read: a new variant plus the `described()` arm the match demands compiled cleanly + and left all ten invariant tests green, reached by none of them. `ALL` is the list the + completeness guard iterates, so a variant missing from it is a blocking state nothing tests -- + which is the exact failure the guard was added to prevent, reintroduced by the shape of its fix. + + The reverse loop in the guard is not a substitute. It catches a state `blocking_states` produces + and `ALL` omits, but only once some perturbation reaches it -- and a state with no perturbation + entry is precisely what the test exists to catch, so it is circular in the case that matters. + + Fixed by declaring the enum, `ALL` and `described()` from one list through a macro, so a variant + that is not in the list does not exist. The claim is now true rather than deleted. + + Sabotage-verified in both directions: the original sabotage is now inexpressible (there is no + second place to omit the variant from), and its reachable equivalent -- a new state in the list + with no perturbation entry -- reddens `every_blocking_state_has_a_perturbation`, where before + the whole suite stayed green. + + **Three rounds on one guard: strings, then a hand-written `ALL`, then generation.** Each fix + moved the census somewhere harder to see rather than removing it. Worth stating because the + reviewer's finding was not a new defect -- it was the same defect wearing the previous fix. + +- [x] **M3.9** -- Decide the row's well-formedness by a real parse, and delete the hand-written one. + + **The question the oracle asks is "could a consumer read this row", and a consumer uses a JSON + parser.** Anything hand-written here is a second opinion about what JSON is, and a second opinion + is a thing that can disagree -- so `malformation` now calls `serde_json` and the scanner is gone. + + **Measured, and the measurement is why this happened at all.** The hand-written check had already + been through a review, which strengthened it after finding it accepted `{"a":1,}`. A generated + test -- 1807 single-character corruptions of a real row, judged against `serde_json` -- then found + **159 more disagreements, every single one a FALSE ACCEPT**: 129 stray backslashes forming invalid + escapes, 10 missing `:`, 13 `,` where a `:` belonged, 3 the reverse, 3 missing values, 1 string + following a number. The review had found one instance of a class with 160 members. + + Closing the last ~26 required tracking whether an object expects a name or a value next, which is + a JSON parser. So the choice was to write one or to depend on one. + + **The agreement test was deleted in the same commit, deliberately.** With the parse delegated it + would compare `serde_json` against `serde_json` -- green by construction, and exactly the + tautology this milestone keeps deleting. What replaced it asks a question that is still open: not + "is the verdict right" but "does the verdict REACH the caller", which is a property of `check` and + not guaranteed by any parser. It found a real boundary while being written: 8 corruptions destroy + the leading brace, and those are `Missing` rather than `Malformed` -- not a row at all, which for + a survey asking "did this host report a row" is the right answer and a different one. Both + branches are asserted. + + **Three tests stopped asserting the defect's wording.** The message is `serde_json`'s now, so + this crate does not own it; pinning it would let a dependency's patch release redden tests about + unclosed delimiters, a false finding about this crate. They assert rejection and the carried row. + + **A parse does NOT subsume `RepeatedKey`,** which is why that check stays hand-written: + `serde_json` accepts a duplicated key and silently keeps the last, which is precisely the + malformation that survives a consumer's parse and changes what it reads. + + `report_oracle` is now gated `cfg(any(test, feature = "oracle-in-renderer"))` -- every caller + already was -- which is what keeps the parser out of a shipping probe. Verified by inspecting the + binaries, per the precedent in that feature's own comment: the default `probe-topology.exe` + contains no `serde_json`, no oracle panic string, and no parser message; the `--all-features` one + contains all three. + +- [x] **M3.10** -- Delete the last hand-written string scanners in the oracle. + + M3.9 removed one of six; this removes the rest. **The module hand-writes no string walking at + all now** -- every byte-level decision about quotes, escapes and delimiters comes from + `serde_json`. + + **Two of the four were provably unsafe, on an argument enforced by nothing.** `list_codes` found + `"code":"` and took the next `"` as the end, and `list_span_end` counted brackets with no notion + of being inside a string. Both were safe only because every code is a `&'static str` from an enum + and no caller text reaches a diagnostic list -- true, load-bearing, and guarded by no test. + `keys` had already proved the class reachable: it made `assert_corresponds` panic from inside + `report_unmeasured` on a quoted `discovery_error`. Parsing makes the argument unnecessary rather + than merely correct, which is the difference between a property and a hope. + + **`keys` needed a visitor rather than a parsed map, and the reason is a contract.** It must return + the row's names in ORDER and WITH DUPLICATES. `serde_json::Map` sorts, and silently keeps the last + of a repeated key -- which would delete the evidence for `RowDefect::RepeatedKey`, the one + malformation that survives a consumer's parse. A `MapAccess` visitor reads each name as the parser + reads it, so both properties survive while every scanning decision stays `serde_json`'s. That + reasoning is now a sabotage entry rather than a comment: replacing the visitor with the obvious + `Map` one-liner is `caught`. + + **`list_span_end` was deleted, not moved.** Its only caller was a sabotage doing text surgery on a + list. That sabotage now parses, empties the lists and re-renders -- its third implementation, after + one that split on commas (which sliced entries in half once they became objects) and one that used + this helper. A sabotage that hand-parses is a sabotage that can quietly stop sabotaging, and it + leaves the rule it guards unguarded while still passing. + + **The stale sabotage entry is itself the evidence.** After the change the sweep reported + `MANIFEST STALE: pattern found 0 times` for the escape-awareness entry -- the defect it injected + can no longer be expressed, because the code that could hold it is gone. Replaced with the + parsed-map entry above; 7 of 7 behave as declared. + + Default build re-verified by binary inspection: neither `serde` nor `serde_json` appears on a + normal dependency edge, and `probe-topology.exe` contains no parser string. + +## Moved 2026-09-13 22:03:09 -07:00 -- M2.14: making the two authoring rules this branch earned actually bite + +### M2.14 -- Make the two authoring rules this branch earned actually bite. *(completed 2026-09-13 22:03:09 UTC-07:00)* + + Re-planned + 2026-09-12; see the rationale below before implementing either sub-step. Both sub-steps done. + + **As originally written this item said "write two authoring rules into the repository + instructions". Measurement says that would have been worse than useless.** The two rules it + proposed -- *state the invariant, not the census*, and *a new test is not done until it has been + observed to fail* -- ALREADY EXIST, in + [.github/copilot-instructions.md](../../.github/copilot-instructions.md) under CONTRACT INTEGRITY + rule 1 ("Prefer a derived fact to a restated one", and beneath it "verify the binding by + sabotage: change the definition and confirm the consumer's BEHAVIOR changes"). Writing them again + would add a second copy of a rule, which is the exact defect that section forbids and the exact + mechanism -- restatement drift -- it exists to prevent. + + **And statement is demonstrably not the gap.** Both rules were in force on 2026-09-12 and both + were violated: `09da7e9` claims "Sabotage-verified, EACH against the instrument it was meant to + strengthen" and then names two sabotages for four fixes. The one that got none is the completeness + guard, which a review found broken an hour later (M3.8). A rule cited in the commit that breaks it + will not be repaired by a third copy of itself. + + **This repository has already solved this problem once, and not with a rule.** The + [ci.yml](../../.github/workflows/ci.yml) `sabotage-harness` job records that the harness "accumulated + fixes over eleven review rounds and thirteen of the later defects were introduced by earlier fixes, + because every verification was a one-off command that was then discarded and nothing re-checked an + earlier guarantee." That is M3.8's story verbatim. The answer then was a CI ratchet. + + +### M2.14.1 -- Give this crate a sabotage manifest, so "observed to fail" is a recorded artifact rather than a habit. *(completed 2026-09-13 22:03:09 UTC-07:00)* + + **Done.** [sabotage.json](sabotage.json), 7 entries, swept green: six `caught`, one `survives`, + all behaving as declared. Not wired into CI, matching the two sibling manifests and the + `sabotage-harness` job's own note that a sweep "rebuilds a crate per entry and is deliberately an + occasional instrument". + + **The control is the entry that matters most here.** It rewords a prose line to carry the same + fact and must SURVIVE, which turns this component's central decision -- the row is the machine + contract, the prose is for a reader -- from a sentence into a measurement. If it is ever reported + as caught, a test has started reading the prose again and that test is the defect. + + **The harness found a defect in the manifest that the authoring script missed, which is the + lesson.** The entry for the escape-aware key reader anchored on `'\\' => escaped = true,`; the + script checked uniqueness by whole-LINE equality and found one match, while the harness matches by + SUBSTRING and found two -- the same arm appears in `malformation` at a deeper indent, and the + shallower line is a substring of the deeper one. The script's check was a second, weaker + implementation of the harness's rule, which is precisely the defect class this manifest exists to + catch. The anchor was widened to the function signature; the harness remains the only authority on + uniqueness. + + `tools/run-sabotage.ps1` exists, has its own tests, and runs in CI; + [windows-placement-probe](../windows-placement-probe/sabotage.json) (9 entries) and + [windows-waitable-queues](../windows-waitable-queues/sabotage.json) (39 entries) each carry a + `sabotage.json`. **This crate carried none**, so every sabotage run while building M3 was ad-hoc + PowerShell, discarded on the spot -- which is why a `git checkout` destroyed uncommitted work + twice and a `.Replace` pattern silently matched two sites once. The manifest format's `find` must + match EXACTLY ONCE, which is precisely the guard that hand-running lacked. + + Eight commits on this branch recorded their sabotages in the message, so the first pass was + transcription rather than invention: the defect, the file and the test expected to redden were + already written down. + + Include at least one `expect: "survives"` control. A manifest of nothing but `caught` cannot + distinguish a suite that is watching from a suite that fails on any edit. + + +### M2.14.2 -- Add to CONTRACT INTEGRITY rule 1 the one thing this branch learned that it does NOT already say. *(completed 2026-09-13 22:03:09 UTC-07:00)* + + A pointer, not a restatement. + + **Done.** Two paragraphs added to CONTRACT INTEGRITY rule 1 in + [.github/copilot-instructions.md](../../.github/copilot-instructions.md). Neither restates the + existing rule: the first says to sabotage the CLAIM a change makes rather than the symptom it + cites, and the second says a sabotage is worth nothing once discarded and points at the manifest + and the harness. The original M2.14 wording is nowhere in the diff, which was the point. + + The genuinely new fact: **when a fix claims to have removed a weakness, sabotage the claim rather + than the symptom.** Rule 1 tells an author to prefer a derived fact over a restated one; it does + not warn that an author may believe they derived one when they only MOVED the census. That is + exactly what happened three times on a single guard -- strings, then a hand-written `ALL`, then + generation -- each fix relocating the census somewhere harder to see while its commit message + claimed the class was closed. The wording that would have caught it is about the CLAIM, and rule 1 + currently has no sentence about claims. + + > **-> DEPENDS ON M2.14.1:** the pointer has nothing to point at until the manifest exists. diff --git a/crates/windows-platform-probes/Cargo.toml b/crates/windows-platform-probes/Cargo.toml index a8c77b8cd..db464bcba 100644 --- a/crates/windows-platform-probes/Cargo.toml +++ b/crates/windows-platform-probes/Cargo.toml @@ -34,7 +34,9 @@ path = "src/lib.rs" # workspace does use `--all-features`, for cargo-mutants runs. The boundary is # stated rather than claimed away: the DEFAULT build is the one that carries the # print-rather-than-panic guarantee. -oracle-in-renderer = [] +# Pulls in `serde_json` because the oracle's well-formedness check IS a real +# parse. See the `[dependencies]` entry for why that is not hand-written. +oracle-in-renderer = ["dep:serde_json", "dep:serde"] [[bin]] name = "probe-error-mode" @@ -92,6 +94,24 @@ name = "probe-long-path-unaware" path = "src/bin/long_path_unaware.rs" [dependencies] +# **Optional, and reached only through `oracle-in-renderer`.** The report +# oracle's job is to answer "would a consumer be able to parse this row", and +# the honest way to answer that is to PARSE IT with the kind of parser a +# consumer uses. This replaced a hand-written structural check. +# +# That check was not obviously wrong; it had been through a review, which +# strengthened it after finding it accepted `{"a":1,}`. Then a generated test -- +# 1807 single-character corruptions of a real row, judged against this crate -- +# found **159 more disagreements, every one a false accept**: a stray backslash +# forming an invalid escape, a missing `:`, a `,` where a `:` belonged. Closing +# the last of those meant tracking whether an object expects a name or a value, +# which is a JSON parser. So the choice was to write one or to depend on one. +# +# It is NOT in the default build, because the oracle is not: the gate here is +# the same one at the call sites in `topology_report`, and a shipping probe must +# print a self-contradicting report rather than panic on it. The module itself +# carries the gate, which is also the clearest statement of what that module is. +serde_json = { version = "1.0", optional = true } # **Every workspace dependency below is path-only, with no `version`**, because # this crate is never distributed at all -- not to a registry, and not as a # released binary either, unlike `windows-placement-probe` next door. These @@ -133,8 +153,33 @@ windows-namespace-request-sys = { path = "../windows-namespace-request-sys" } # over that is a vocabulary one -- the workspace's own WTF-16 type naming the # quantity rather than an ad-hoc count that happens to agree. wtf-string = { path = "../wtf-string" } +# Beside `serde_json`, and gated with it, for ONE reason: `keys` must report the +# row's top-level names in ORDER and WITH DUPLICATES, and a parsed map can give +# neither -- `serde_json::Map` sorts, and silently keeps the last of a repeated +# key, which is the very defect `RowDefect::RepeatedKey` exists to report. A +# `MapAccess` visitor sees the names as the parser reads them, so the scan is +# still the real parser's and this crate hand-writes no string walking at all. +serde = { version = "1.0", optional = true } [dev-dependencies] +# Here because [tests/a_real_report_agrees_with_itself.rs] uses `serde_json` +# DIRECTLY, to empty the row's diagnostic lists by parsing and re-rendering +# rather than by cutting the text. +# +# **Not to satisfy `report_oracle`'s gate, which an earlier version of this +# comment claimed.** That claim was backwards: it said `cfg(test)` builds compile +# the module without `oracle-in-renderer`. They do not. The self dev-dependency +# below is an edge from this package to itself, so under resolver 2 the feature +# unifies onto the package in ANY build that includes dev-dependencies -- the +# `cfg(test)` lib build included. Measured with a `compile_error!` probe in both +# directions: `cargo check --tests` succeeds under `not(feature = ...)` and fails +# under `feature = ...`, while a plain `--lib` build has the feature off. +# +# So the two arms of `cfg(any(test, feature = "oracle-in-renderer"))` are never +# exercised separately, the optional `serde`/`serde_json` are already active in +# every test build, and a matching `serde` dev-dependency was dead weight -- +# removed after confirming all 213 tests still pass without it. +serde_json = "1.0" # A dependency on ITSELF, which cargo permits for dev-dependencies and which is # the only way to turn a feature on for this package's own integration tests: # a test under `tests/` links the library as a plain dependency, so `cfg(test)` diff --git a/crates/windows-platform-probes/DESIGN-NOTES.md b/crates/windows-platform-probes/DESIGN-NOTES.md index 8020fa745..5417ffd1f 100644 --- a/crates/windows-platform-probes/DESIGN-NOTES.md +++ b/crates/windows-platform-probes/DESIGN-NOTES.md @@ -1025,415 +1025,229 @@ it: it closes over the whole run including both banner reads, where `measure` closes only over the counters. Neither subsumes the other, and no work is scheduled by this decision. -## The defects that survived were correspondence failures, and no instrument here could see them +## The correspondence-oracle investigation, and what it concluded - -This probe was reviewed twenty-eight times before it opened as a pull request, -by two independent readers per round on different models, with `cargo-mutants` -reporting **zero surviving mutants** on both of its modules. A review on the -pull request then found, in code none of that had touched, a state where the -renderer printed + + +**Superseded by [The encoded row is the contract; the prose is not](#d-encoded-row-is-the-contract).** + +**Moved to Tier 2: [DESIGN-RATIONALE.md](DESIGN-RATIONALE.md).** The anchors +above are kept here so every existing link still lands somewhere that says where +the content went. + +What it covers: the two defects a pull-request review found after twenty-eight +rounds of per-artifact review and a zero-surviving-mutant sweep; why each +instrument in use was structurally incapable of seeing them; the sparse-matrix +and oracle split that followed; and the oracle's own design, failure modes and +mutation evidence. + +It is Tier 2 rather than Tier 1 for two reasons. It is a record of how a decision +was reached rather than a statement of one -- and the decision it reached has +since been superseded by +[The encoded row is the contract; the prose is not](#d-encoded-row-is-the-contract). +The code it describes is also gone: the prose-reading oracle, its correspondence +enum and its fact-accounting instrument were retired by M3.4 and M3.5, so several +of its sentences name types that no longer exist. + +What survives into Tier 1 is the conclusion the two defects actually support, +which is the decision below. + +## The encoded row is the contract; the prose is not + + + +A probe is a data pipeline that renders, at its tail, to two artifacts: an NDJSON +row and prose. **They are not peers.** The row is a machine contract -- mined +across a fleet, joined against other runs, and the thing this workspace's designs +end up resting on. The prose is for a reader. + +So they carry different obligations: + +- **The row must be CORRECT**, and that is machine-enforced. Its values, its + invariants and its shape are asserted. +- **The prose must be ACCURATE AND READABLE**, and that is enforced by review. + It is not required to be programmatically comparable against the row, and + nothing here checks that it is. + +This supersedes the rule in +[#d-oracle-refuses-to-know](DESIGN-RATIONALE.md#d-oracle-refuses-to-know), which said the oracle +must read the rendered artifact rather than the state behind it. + +### What forced it: both originating defects were defects in the row + +The reason the earlier rule looked right was a misreading of its own evidence. +Re-checked against the code, for the two defects in +[#d-correspondence-failures](DESIGN-RATIONALE.md#d-correspondence-failures): + +**The alarm beside the agreeing verdict.** `report` emits +`BUG IN THIS PROBE: ...` with a `writeln!` into the prose, and the NDJSON row has +**no key for it** -- while `cross_check` IS a key, and read `agree` on the +defective run. So the row certified a clean agreeing measurement on a host where +the probe had detected its own bug, and said nothing about the bug. A survey +mining that row would have been wrong and had no way to know. The prose alarm was +not the defect; it was the only trace that the row was wrong, which is why a +human found it and no instrument did. + +**That defect is fixed, and what it left behind was the live gap.** Checked +rather than assumed, because the paragraph above describes the code as it was: +`Observation::cross_check` pushes `PartitioningCache::SummaryMissing` onto +`parse_incomplete`, which forces the verdict away from `agree`, so the row could +not certify that run. But the row published `parse_incomplete` as a **count** -- +as it did `not_compared` and `enumeration_anomalies` -- where the prose published +each entry's text. A survey reading `"parse_incomplete":1` could not tell *the +probe detected a bug in itself* from *a core record contradicted itself* from +*this topology was not measured from a running machine*. Those are categorically +different facts, and only the prose distinguished them. + +So the shape of the problem was not that the row is out of step with the prose. +It is that **the row was impoverished relative to the prose** -- the artifact +that gets mined carried less than the artifact that gets read -- which is +backwards given which of the two the designs rest on. + +**M3.1 closed this**, and the past tense above is deliberate: the three fields +publish the conditions themselves, minted by `topology::diagnostic`, so a survey +reads which one fired rather than how many there were. The count remains +available as the list's length. + +M3.3 then gave each entry its DATA, so the published form is an object rather +than a bare code: ``` -BUG IN THIS PROBE: the topology crate named L3 as the outermost -partitioning cache and this survey carries no summary for it. Nothing -below about cache partitioning can be trusted. +"parse_incomplete":[{"code":"partitioning_summary_missing","level":9}] ``` -while `cross_check` had no branch for that state at all, so `verdict()` could -return `Agree` for the same run and print `=> agree` two paragraphs below. A -second finding in the same review had the same shape: one fact rendered twice -in one report -- `efficiency classes: [0]` in prose, `"efficiency_classes":1` -in the NDJSON -- in two shapes a consumer cannot reconcile, where the numeral -happens to read as a plausible class *label*. - -Neither is a bug inside a function. Every function involved was correct on its -own terms, and each had been read repeatedly and found so. The defect lived in -the **relation between two artifacts**, and that is a place none of the -instruments in use could look. - -### Why each instrument was structurally incapable, not merely unlucky - -**Mutation testing cannot find absent code.** `cargo-mutants` perturbs what is -written and asks whether a test notices. A missing branch has no mutants, so -the missing `SummaryMissing` check did not lower the score -- it was invisible -to it. The 180/0 result was true and said nothing about the gap. A perfect -mutation score is compatible with an entirely missing feature, and this -component is the proof. - -The same run also shows the weaker half of what a mutation score means. A test -existed asserting `"efficiency_classes":2`, so every mutant of that line died. -It was pinning the wrong shape faithfully. **Mutation testing measures whether -behavior is pinned by tests; it is silent on whether the pinned behavior is -right.** Both halves were over-read here for many rounds as though they were -evidence of correctness. - -**Exhaustiveness checking protects `match` expressions, not concepts.** -`PartitioningCache` exists precisely to force a decision -- its own doc says a -renderer or serialiser "cannot emit the absent case without having decided -which absent case it is" -- and it worked, in the two consumers that wrote a -`match`. It bought nothing in the two that did not: `domain_counts` reached the -same information through `outermost_partitioning_cache`, a second accessor -returning `Option`, which launders five states into two; and `cross_check` -never asked. A type can only compel a consumer that consults it. - -**Per-artifact review finds per-artifact defects.** Two readers checking each -function against its own documentation will confirm both sides of a -contradiction, because each side is locally true. Worse, the readers were -answering questions posed in a prompt, and across rounds that prompt -accumulated focus areas and "already verified, do not re-litigate" facts. The -shared prompt correlated the readers far more strongly than their differing -models decorrelated them; the instrument was being shaped to agree with its -author. Removing that framing in the final round is what got a reader to trace -`simultaneous_multithreading` out of this crate into `windows-topology-sys` and -check it against the Win32 `LTP_PC_SMT` contract. - -The single sentence that covers all three: **every instrument in use verified -properties of things that exist.** Tests assert existing behavior, mutation -perturbs existing code, reviewers check written claims. A correspondence -failure is a property of a *pair*, and an absent branch is not a thing at all. - -### Integration-level analysis was absent, which is where these live - -At the time of the pull request the crate had one integration test, asserting -that a probe writes something to stdout. Of twenty-five `report()` calls in the -suite, **none rendered from a real host's `measure()`** -- every one used a -synthetic `Observation` built by hand. A hand-built fixture can only contain -states its author already imagined, and each assertion checked one local fact -about it. Nothing anywhere rendered the artifact a consumer actually reads and -asked whether it was self-consistent. - -### What to do instead: a sparse matrix to explore with, an oracle to keep - -The obvious response -- tabulate every state against every consumer and fill -the grid -- is wrong, and was proposed and rejected during this analysis. Such -a table grows combinatorially, most of its cells are meaningless, and a version -of it committed beside the code would be a second copy of the code's structure -that nothing verifies. It would rot exactly as every restatement in this -component rotted, and a stale "all cells covered" table is more dangerous than -no table. - -The division that does work: - -- **The matrix is a transient, exploratory instrument.** Draw it for one type - at one boundary to find out which correlations exist. It is expected to be - **sparse**; most cells are empty and discovering that is cheap. Correlations - cannot be derived -- which is why twenty-eight rounds of reading produced - none -- so populating it is exploration, not specification. -- **An oracle is the durable artifact.** Only cells that turn out to mean - something graduate into it. It stays small because discovery, not - enumeration, fills it. - -`windows-file-watcher`'s `ContractChecker` is this repository's worked example -of the oracle half: a shared executable definition of the rules, owned by the -crate that owns the contract, that the producing crate's own tests and every -consumer's test doubles all bind to. It already existed while this probe was -being written, and was not reached for. - -Every correlation admitted here is one the report already renders twice, with -nothing relating the two -- a property of the artifact rather than of anyone's -intuition about it. The ones this decision was written against are: - -1. an alarm in the report implies the verdict is not `agree`; -2. a fact rendered twice must agree across its renderings; -3. an uncaveated hardware claim implies `!parse_in_doubt`; -4. the banner names the same machine the body describes. - -That list is the seed, not the census: the admission RULE is what governs, and -the authoritative set is the `Correspondence` enum in -[src/report_oracle.rs](src/report_oracle.rs). An earlier version of this -paragraph said "three correlations" and listed the first three while the enum -already had the fourth -- the count was wrong when written and would have -rotted again at the next addition, so it is stated as a rule here instead. - -The replacement rule was then itself overstated, as "known to be real because it -was violated" -- which excludes the correspondences found by the M2.4 matrix, -where no defect had occurred and walking every NDJSON field against the prose is -what showed the fact rendered twice with nothing comparing it. Both routes are -admissible; what is not is inventing a correspondence between things the report -does not actually render twice. Corrected the same day it was written, after a -review noticed it contradicted `check_structured_pairs`' own history. - -What makes an oracle different from more tests is where it is invoked: if every -test renders *through* it, every existing call site inherits the checks and so -does every future one. A test added beside them checks one case; an oracle -checks every case anyone ever writes. (Also stated without a number on purpose --- this said "all twenty-five existing call sites" and there are now 35.) - -**Record the vacuous findings too.** "We examined whether X and Y must -correspond, and they need not" is a result, and it is the half that normally -evaporates -- without it the next person re-explores the same empty cells. - -An oracle is a forcing function for correlations already discovered. It will -not find a new one. The discipline that makes it compound is that each newly -found cross-artifact contradiction adds an invariant to the oracle rather than -a one-off test. - -Whether this generalises to `Coherence`, `BracketOutcome`, `Verdict` and the -sibling probes is **an open question, deliberately not answered here.** The work -this decision implies is queued as M2 in [CHECKLIST.md](CHECKLIST.md); this -section schedules nothing on its own. - -## The oracle exists, and what it deliberately refuses to know - -M2.1 built it: [src/report_oracle.rs](src/report_oracle.rs), admitting only -correlations the report already renders twice. The defect that forced -it is the section above. - -**It reads the rendered artifact, never the state behind it.** Checking state -would miss precisely this defect class -- in the original finding the state was -consistent and the two *renderings* of it were not. - -**It relates two things already visible in the report, and re-derives nothing.** -A second implementation of the rendering rules would be a check of the copy -rather than of the contract, and would drift the moment either moved. So the -alarm rule compares an alarm line against a verdict line, the double-rendering -rule compares prose against NDJSON, and the gating rule compares a claim against -the report's own published evidence of doubt. - -That last one is the interesting boundary. `CrossCheck::parse_in_doubt` is -`!disagreements.is_empty() || !parse_incomplete.is_empty()`, and the NDJSON -publishes `parse_incomplete` as a **count** rather than the predicate -- so the -oracle reads the count and the `disagree` verdict, which are the two visible -shadows of that definition. The coupling is deliberate, and confirming it still -holds is what M2.2's sabotage check is for when the call sites are bound. - -**Half the tests assert acceptance**, following -[../windows-file-watcher/src/contract.rs](../windows-file-watcher/src/contract.rs)'s -`ContractChecker`: an alarm beside a non-agreeing verdict is legal and is what -the fix produced, a caveated claim under doubt is legal and is what the renderer -emits on every heterogeneous host with a short parse, and a prose-only report is -silence rather than violation. Over-constraining is the same defect as -under-specifying and fails in the more expensive direction, because noise trains -a reader to ignore the instrument. - -### The failure mode that would look exactly like success - -An oracle whose prose labels do not match the renderer reads nothing, finds -nothing, and passes everything. So the labels were confirmed against a real -`probe-topology` run, and a test corrupts each double-rendered value in turn and -requires a violation -- if a label ever drifts, that test fails rather than the -oracle going quietly blind. - -**The first attempt at that injection silently did nothing**, and is worth -recording because it nearly produced the opposite conclusion. The anchor used -was `cross-check:`, which does not occur -- the real text is `cross-check -against independently read Win32 counters:` -- so the "defective" report was -identical to the clean one, the oracle correctly reported no violation, and the -reading was almost "the oracle is blind". A sabotage that fails to apply is -indistinguishable from an instrument that fails to fire, unless the injection -asserts it changed something. It now does. - -**The mirror-image hazard: a RESTORE that fails to rebuild.** Sabotage work in -this crate is a loop -- break it, run it, put it back, run it again -- and the -put-it-back step has its own way of lying. On Windows, PowerShell's `Copy-Item` -preserves the source file's `LastWriteTime`, so restoring a file from a backup -taken earlier gives it an mtime OLDER than the artifacts built from the -sabotaged version. Cargo fingerprints by mtime, decides nothing has changed, and -reruns the previous binary. Measured here: a restored, correct oracle reported -the fixed defect as still present, and the reading was almost "the fix does not -work" -- the conclusion was only avoided by printing the intermediate values and -finding that the function returned the right answer while the test insisted it -did not. - -After restoring a file by copy, set its timestamp forward -(`(Get-ChildItem ).LastWriteTime = Get-Date`) or rewrite it through a -read-then-write, which stamps it as a matter of course. The general rule is the -same one as above, pointed the other way: **a green result proves nothing until -you know the code you are testing is the code you just wrote.** - -### The real-host test, and the guard that stops it passing for nothing - -[tests/a_real_report_agrees_with_itself.rs](tests/a_real_report_agrees_with_itself.rs) -composes the report the way `probe-topology` does and applies the oracle -explicitly. - -**Why it has to exist.** The oracle's unit tests pin it against fixtures, and a -fixture is a report somebody wrote down -- so a fixture-bound oracle checks -correspondences over states its author already imagined, and the defect it -exists for was a state nobody had imagined. More narrowly, a fixture cannot -notice the *renderer* drifting away from the prose labels the oracle reads: -both sides would still agree with each other. Only the real artifact disagrees. - -Some unit tests in this crate do call `measure()` and so do read this host. -What none of them does is run the **oracle** over a report rendered from that -reading, which is the gap this test closes. On CI it runs across the hosted -runner fleet, a survey of shapes no fixture anticipates. - -**It asserts nothing about this machine, deliberately.** A test expecting a -processor count, a cache level or a verdict would fail on the next runner shape -rather than on a defect, and would have to be loosened until it asserted -nothing. What it checks is that whatever this host produced, the report's parts -agree with each other -- a property every host must satisfy, including one whose -topology cannot be read at all. - -#### The primary assertion can pass having checked nothing - -On a host whose report the oracle cannot parse, every lookup returns `None`, -every comparison is skipped, and -`a_report_rendered_from_this_host_agrees_with_itself` passes having checked -exactly zero correspondences. That is why the second test corrupts each -double-rendered fact in a report **this host really produced** and requires the -oracle to report a violation **naming that fact** -- and asserts first that the -corruption changed the text at all, for the reason recorded above. -Eight facts rather than one, because corrupting a single field would leave the -others unguarded: the renderer could drift away from the oracle's other prose -labels and the test would still pass on the strength of the one that remained. - -**"Naming that fact" is load-bearing, and took three attempts to get right.** -The guard first required only that the violation list was non-empty. That was -strengthened to require a `ProseAndNdjsonDisagree`, with a comment correctly -observing that corrupting the NDJSON `processors` count also trips the -cross-check counter rule -- and then not acting on the observation, because that -counter rule emits precisely that variant. So the strengthened guard still -passed while the reader it was written to protect was blind. - -Measured, by blinding one `DOUBLE_RENDERED` prose label at a time: under the -variant-only assertion, `processors` and `groups` both stayed **green**, masked -by their counter rules, while `packages` went red because nothing else reads it. -Two of the four facts the guard names as required were unchecked by the guard -whose entire purpose is to establish that they are checked. Matching on the -`fact` string closes it, because no neighbouring rule can supply another rule's -fact name. - -The general lesson, which this crate has now paid for four times: **a guard -against vacuity is itself a claim, and is subject to the same discipline as any -other.** Each of the three earlier versions was written to fix the previous -one's vacuity and introduced the next-narrower version of it. The only thing -that has ever settled the question is sabotaging the mechanism the guard is -supposed to protect and watching the guard go red -- never reading the guard and -judging it sufficient. - -### The fact set is derived, because the gaps were never in the rules - -Six unread double-renderings were found on this branch by six different -reviewers, and none by the oracle's own coverage. Each individual gap was real, -and fixing each one by hand was correct -- but the pattern is the finding, and -the pattern is structural: **a rule is added per fact, so the SET of facts is -what drifts.** Nothing derived that set from -[src/topology_report.rs](src/topology_report.rs), so a field added to the -renderer was unread until a person happened to notice. - -The clearest demonstration is gap 6. It was created BY the hand-written rule -that closed gap 2: M2.11 added a comparison for the partitioning discriminator, -that arm renders a level number as well, and nothing compared the number. Closing -a gap added a fact, and the new fact was unread by exactly the mechanism that -produced the previous five. - -`every_fact_the_renderer_publishes_is_accounted_for` in -[tests/a_real_report_agrees_with_itself.rs](tests/a_real_report_agrees_with_itself.rs) -is the answer, and its shape matters more than its code. Both halves come from -places that cannot fall out of step: - -- **The set of facts is enumerated from the artifact** -- the NDJSON line of a - report this host really rendered. Not a list in the test. A new field appears - in the enumeration by itself, without anyone maintaining anything. -- **Read-or-unread is measured, not declared** -- each value is corrupted in turn - and the oracle is asked. A rule that quietly stops reading a fact is caught - even though no list changed. - -Only the classification of each key is written down, because which of the three -kinds a key belongs to is a judgement: compared, conditional on its prose being -rendered, or having no second rendering at all. **A key in no list fails**, and -that is the whole point -- adding a fact to the renderer forces the judgement to -be made deliberately instead of being discovered by the seventh reviewer. - -This is the same discipline the oracle itself rests on, turned on the oracle: -relate two things that already exist rather than restate one of them. A second -list of "facts the rules look at" would have been another copy to drift, which -is the defect rather than the fix. - -**The limit, stated because a reader will otherwise assume it is closed.** The -enumeration is keyed to the machine-readable line, because that is the side with -an enumerable structure. A fact rendered only in PROSE, with no field beside it, -is invisible here. Prose is not enumerable without parsing English, so that half -remains a thing only a reader notices -- and saying so is better than implying a -coverage that does not exist. -### The instrument is code too, and it is where the defects were - -Eleven review rounds across six models ran over this branch. Classifying every -finding by what would have caught it earlier is more useful than the findings -themselves, because the classes are very unevenly sized. - -**The largest class by far is shape blindness**, and it has one root: every -instrument here was validated against a single artifact, the developer machine's. -This host produces exactly one shape -- measured, `agree`, the `Level` -partitioning arm, non-empty caches, classes `[0]`, zero anomalies, zero -`not_compared`, x86_64 -- and every defect in that class lived in the complement -of it. The architecture uncompared on an unmeasured report, `efficiency_classes` -comparing contents so a scalar and a one-element list were identical, the anomaly -count unread under two of three verdicts, the expected fact name differing on the -`SummaryMissing` arm, an empty container that could not be corrupted: none of -them can occur here, and each was found only because a reviewer imagined a shape -by hand. [CHECKLIST.md](CHECKLIST.md) -> M2.12 is the structural answer, and it -is M2.10's move one level up: derive the set of SHAPES from the renderer's -branches, as M2.10 derives the set of FACTS from the artifact. - -**The second observation is the one worth carrying to other crates.** Most of -these defects were not in the oracle. They were in the guard that checks the -oracle, the table of required facts, the fact name the guard expects, the -declaration of when silence is legitimate. The thing under test came through the -last rounds clean; the things doing the testing did not. - -An instrument feels like it sits outside the system under test, so it escapes the -discipline applied to production code -- and then it fails on a CI runner -reporting a defect in the renderer that is really a defect in the instrument, -which is worse than no check at all because it sends the reader to the wrong -place. Instruments need what production code gets: derivation instead of -restatement, sabotage before they are believed, and a stated boundary. - -**And a fix is new code.** Twice on this branch the fix for one gap created the -next: M2.11's discriminator rule gave the level a second fact name, which broke -the guard that expected one; M2.10's accounting inherited a silence assumption -that only holds under an `agree` verdict. After a fix, re-run the derivation or -the sweep -- not only the test that prompted it. -### Mutation testing is how the instruments got checked, and what it cannot reach - -Every defect on this branch was found by a person reading code -- reviewers, -mostly, and this file records how often they found the same shape. Late on, the -obvious question got asked: is there a mechanical way to ask whether a test -establishes anything, rather than trusting that it does? - -There is, and it was already installed. `cargo-mutants` changes the source and -asks whether anything notices, which is the sabotage loop this crate has been -running by hand all along, done exhaustively. It answers a stronger question than -coverage: not *was this branch executed* but *does anything DETECT a change to -it*. On a branch whose recurring defect is a test that runs code without -establishing anything about it, that difference is the whole point. - -Three sweeps, run through [tools/run-mutants.ps1](../../tools/run-mutants.ps1): - -| file | tested | caught | unviable | survivors | -|---|---|---|---|---| -| `topology_report.rs` | 28 | 28 | 0 | 2, then none | -| `report_oracle.rs` | 143 | 138 | 5 | 6, then none | -| `topology.rs` | 186 | 180 | 6 | none, first run | - -The unviable column is why a caught-count does not equal a tested-count: those -mutants did not compile, so they say nothing either way. An earlier version of -this table gave only the caught figures, and the commit that wrote it summed them -to a total that matched neither -- 357 were tested and 346 caught, and it claimed -352. Stated in full here so the arithmetic is checkable rather than asserted, -which is the same rule this branch keeps having to relearn about numbers. - -**The survivor worth remembering is `assert_corresponds`.** Replacing its body -with `()` survived, because every instrument that would notice goes THROUGH it -- -both renderers are bound to it, the real-host test calls it, the corpus reaches -it by rendering -- so a no-op assertion makes all of them pass together. Nothing -asserted that the assertion asserts. That is a blind spot no amount of adding -tests *through* an instrument can find, and it is exactly what a tool that -attacks the code rather than the tests is for. - -The others were smaller and of one kind: guards that only fire on malformed -input, which every fixture was too well-formed to reach. A `> 0` that stops an -empty value being reported as a value; the `&&` that makes `!!...!!` a shape -requiring both ends; a string branch whose only caller always passes an array. - -**What it cannot reach, and this matters here.** `cargo-mutants` mutates `src/`, -not `tests/`. The accounting table, the shape corpus and the fact-name guard all -live in `tests/`, so the tool validates the code they check and says nothing -about THEM. The instruments remain exactly as good as the hand-sabotage that -built them -- which is where several of this branch's defects were found, and -where the next one will be. A clean sweep is evidence about the oracle, not about -the things measuring it. \ No newline at end of file +Stated here because this is Tier 1 and the wire format is what a reader comes to +it for. The bare-code form this paragraph first showed was M3.1-era and was +superseded three commits later on the same branch -- the drift class this +component keeps meeting, caught by a review. The rest of +this decision is unaffected -- it is about which artifact carries the contract, +not about these three fields. + +**`efficiency classes: [0]` against `"efficiency_classes":1`.** Both halves were +correct derivations of one consistent value -- the prose rendered the set, the row +rendered the cardinality -- so no invariant was violated. Note how it was +repaired: the row now publishes `"efficiency_classes":[...]`, the set. **The fix +was to change what the row publishes.** The prose comparison was how a reviewer +noticed, not the repair. + +Neither defect needed a prose-against-row oracle to fix. Both needed the +structured output to be made right. + +### The rule that falls out, and it is the load-bearing one + +**A renderer may not tell a reader something the row cannot tell a survey.** A +state worth naming to a human is a state worth publishing to a mining pass; if +only the prose can say it, the fact exists solely in the artifact nothing +queries, and the only detector is a person reading. A cardinality is not a +statement of the fact -- `"parse_incomplete":1` names no condition -- so a count +beside a prose list is an instance of this rule being broken, not an exception +to it. + +With that rule in place the surviving correspondences stop being text +comparisons and become **invariants on the observation, checked before +rendering** -- `summary_missing` implies the verdict is not `agree`, and likewise +for the other diagnostics and the counters. No parser is involved. + +**This rule is enforced, and the first thing it would have caught was already +broken when the rule was written.** Every instrument in this crate used to start +from what the row publishes -- the fact accounting enumerated the row's keys, the +mutation sweep perturbed code the row's construction reached -- so all of them +asked "does anything read this key?" and none asked "does the prose state a fact +the row omits?". Measured: `CrossCheck::disagreements` was rendered per-entry in +the prose and published in the row as nothing at all, so a survey could see +`"cross_check":"disagree"` and not which counter disagreed. It survived 41 review +rounds and a zero-survivor mutation sweep. A reviewer found it by reading the +enum and asking who called `code()`. + +The second enumeration -- every state that forbids agreement to a published +condition -- now exists, in +[tests/a_real_report_agrees_with_itself.rs](tests/a_real_report_agrees_with_itself.rs): +`every_state_that_blocks_agreement_reaches_the_row` holds `topology::invariant`'s +blocking states against the row's keys, and `publication_holds` is the shared +predicate the corpus rule and its sabotage both call. Landed as M3.5, archived in +[COMPLETED-CHECKLIST.md](COMPLETED-CHECKLIST.md). + +(Until M3.5 landed this section said "nothing here enforces that rule yet" and +pointed at CHECKLIST.md for the queued item. Both statements outlived the +milestone that made them false -- the reason this Tier 1 file is swept against +the code rather than trusted, and an instance of the restatement drift the +repository instructions describe.) + +### What the text-reading design cost + +Counted in [src/report_oracle.rs](src/report_oracle.rs) **as it stood before this decision**: of 38 +top-level functions, ten were correspondence rules and four were comparison +helpers. **Twenty-three existed only to extract values back out of rendered +text.** None of them survives: [src/report_oracle.rs](src/report_oracle.rs) reads no +rendered PROSE at all now, and hand-writes no string scanning -- the row's +well-formedness is a `serde_json` parse and its keys come from that parser's own +tokens. (This said "reads no rendered text at all", which a review correctly read +as contradicting the module: the prose reader is gone, the ROW parser is not, and +the row is rendered text. What changed is that nothing here infers a value from a +sentence -- the one parse left is of a format with a specification, performed by a +library rather than by this crate.) So the counts above are what the design cost, not what the file holds. +(They are also the only counts kept here, because they describe a file that no +longer exists in that form and so cannot drift; a count of the CURRENT file would +be a census, and is deliberately absent.) + +That is a parser for a format this crate itself writes, and it behaved like one. +A large share of PR #88's review rounds were defects in the READER rather than in +the thing read: a multi-byte panic in `processors_in_banner`, a `p/` substring +matching inside an opaque `io::Error`, `trim_matches` collapsing `[[0]]` and +`[0]`, a prose lookup selecting the wrong line when two began alike. None of +those is a defect in a probe. They are a defect source the design created for +itself. + +### Where structure replaces checking, prefer structure + +Three of the four hazards this component has actually met are made +*unrepresentable* by construction rather than detected after the fact, and that +is the stronger move: + +- **Injection.** Caller text reaching the row is contamination of the mined + artifact. Measured on PR #88: an `io::Error` containing `{` was selected as the + report's machine-readable row. A typed row emitted by one writer cannot have + this. +- **Field order and labelling.** The row was built by interpolating every value + positionally through a `concat!` template, so a field's name and its value were + related only by counting -- and a reordered argument or a miscounted `{}` gave + mislabelled data that still parses. A typed row with one writer cannot have + this either, and that is what M3.3 built: the template is gone, and + [src/row.rs](src/row.rs) is the one writer. + + Stated as the coupling rather than as a count, deliberately, and the reason is + on the record: this said "eighteen values", was corrected to "seventeen" when + a review counted the placeholders, and was falsified again within the hour by + M3.1's follow-up adding `disagreements`. The hazard is that the correspondence + is positional at all; how many positions there are is exactly the sort of + census this component keeps having to re-correct. +- **Value divergence.** Two renderings of one field cannot disagree about its + value when both read the field. + +What structure does NOT cover, and so still needs something reading bytes: **the +writer itself.** Several of PR #88's defects lived there -- a disclaimer matched +as a suffix so it could be welded onto the line above, a flattening that ate the +disclaimer, a containment that produced `host: host: ...`. The residual text +check is therefore small and about well-formedness, not about correspondence. + +### What this does not say + +It does not say the prose does not matter. An overstated finding in prose +propagates into the design notes that cite it, which is a live concern in this +crate rather than a hypothetical -- M2.9 in [CHECKLIST.md](CHECKLIST.md) is an +open item about exactly that. What changes is that prose accuracy is a **review** +obligation, discharged by a person reading the report, rather than a +correspondence a machine asserts. + +It also does not delete the correspondence rules. They relocate onto the +observation, losing the parser in front of them. The containment work in +[src/topology_report.rs](src/topology_report.rs) matters MORE under this +decision, not less, because what it keeps out is now keeping it out of the +contract artifact. + +The work this implies was M3, which is complete and archived in +[COMPLETED-CHECKLIST.md](COMPLETED-CHECKLIST.md). (This said "is queued as M3 in +CHECKLIST.md" until a review pointed out that the canonical design note was +advertising landed work as pending.) The session that produced it is +[design-sessions/DESIGN-SESSION-2026-09-12-what-the-oracle-should-read.md](design-sessions/DESIGN-SESSION-2026-09-12-what-the-oracle-should-read.md). diff --git a/crates/windows-platform-probes/DESIGN-RATIONALE.md b/crates/windows-platform-probes/DESIGN-RATIONALE.md new file mode 100644 index 000000000..2ae0566e9 --- /dev/null +++ b/crates/windows-platform-probes/DESIGN-RATIONALE.md @@ -0,0 +1,530 @@ +# Design rationale: windows-platform-probes + +Tier 2. **How decisions were reached** -- the investigations behind them, the +alternatives weighed, and the reasoning that has since been superseded. The +current decisions are in [DESIGN-NOTES.md](DESIGN-NOTES.md), which is +authoritative; where the two disagree, Tier 1 wins. + +Split from DESIGN-NOTES.md at ba786ce. + +Read this for "why did it end up like this", never for "what is it now". Several +sections below describe code that no longer exists -- the prose-reading oracle +and its instruments were retired by M3.4 and M3.5 -- and they are kept because +the reasoning is what a future reader needs when the same question comes round +again, not because the code is still there. + +--- + +## The defects that survived were correspondence failures, and no instrument here could see them + + + +**The diagnosis here is refined by [The encoded row is the contract; the prose is +not](DESIGN-NOTES.md#d-encoded-row-is-the-contract).** What each instrument could not see is +unchanged and is still the reason this component has an oracle at all. What this +section got wrong is WHERE the two defects lived: both were defects in the +ENCODED ROW, not in the relation between two renderings of a consistent state. + +This probe was reviewed twenty-eight times before it opened as a pull request, +by two independent readers per round on different models, with `cargo-mutants` +reporting **zero surviving mutants** on both of its modules. A review on the +pull request then found, in code none of that had touched, a state where the +renderer printed + +``` +BUG IN THIS PROBE: the topology crate named L3 as the outermost +partitioning cache and this survey carries no summary for it. Nothing +below about cache partitioning can be trusted. +``` + +while `cross_check` had no branch for that state at all, so `verdict()` could +return `Agree` for the same run and print `=> agree` two paragraphs below. A +second finding in the same review had the same shape: one fact rendered twice +in one report -- `efficiency classes: [0]` in prose, `"efficiency_classes":1` +in the NDJSON -- in two shapes a consumer cannot reconcile, where the numeral +happens to read as a plausible class *label*. + +Neither is a bug inside a function. Every function involved was correct on its +own terms, and each had been read repeatedly and found so. The defect lived in +the **relation between two artifacts**, and that is a place none of the +instruments in use could look. + +### Why each instrument was structurally incapable, not merely unlucky + +**Mutation testing cannot find absent code.** `cargo-mutants` perturbs what is +written and asks whether a test notices. A missing branch has no mutants, so +the missing `SummaryMissing` check did not lower the score -- it was invisible +to it. The 180/0 result was true and said nothing about the gap. A perfect +mutation score is compatible with an entirely missing feature, and this +component is the proof. + +The same run also shows the weaker half of what a mutation score means. A test +existed asserting `"efficiency_classes":2`, so every mutant of that line died. +It was pinning the wrong shape faithfully. **Mutation testing measures whether +behavior is pinned by tests; it is silent on whether the pinned behavior is +right.** Both halves were over-read here for many rounds as though they were +evidence of correctness. + +**Exhaustiveness checking protects `match` expressions, not concepts.** +`PartitioningCache` exists precisely to force a decision -- its own doc says a +renderer or serialiser "cannot emit the absent case without having decided +which absent case it is" -- and it worked, in the two consumers that wrote a +`match`. It bought nothing in the two that did not: `domain_counts` reached the +same information through `outermost_partitioning_cache`, a second accessor +returning `Option`, which launders five states into two; and `cross_check` +never asked. A type can only compel a consumer that consults it. + +**Per-artifact review finds per-artifact defects.** Two readers checking each +function against its own documentation will confirm both sides of a +contradiction, because each side is locally true. Worse, the readers were +answering questions posed in a prompt, and across rounds that prompt +accumulated focus areas and "already verified, do not re-litigate" facts. The +shared prompt correlated the readers far more strongly than their differing +models decorrelated them; the instrument was being shaped to agree with its +author. Removing that framing in the final round is what got a reader to trace +`simultaneous_multithreading` out of this crate into `windows-topology-sys` and +check it against the Win32 `LTP_PC_SMT` contract. + +The single sentence that covers all three: **every instrument in use verified +properties of things that exist.** Tests assert existing behavior, mutation +perturbs existing code, reviewers check written claims. A correspondence +failure is a property of a *pair*, and an absent branch is not a thing at all. + +### Integration-level analysis was absent, which is where these live + +At the time of the pull request the crate had one integration test, asserting +that a probe writes something to stdout. Of twenty-five `report()` calls in the +suite, **none rendered from a real host's `measure()`** -- every one used a +synthetic `Observation` built by hand. A hand-built fixture can only contain +states its author already imagined, and each assertion checked one local fact +about it. Nothing anywhere rendered the artifact a consumer actually reads and +asked whether it was self-consistent. + +### What to do instead: a sparse matrix to explore with, an oracle to keep + +The obvious response -- tabulate every state against every consumer and fill +the grid -- is wrong, and was proposed and rejected during this analysis. Such +a table grows combinatorially, most of its cells are meaningless, and a version +of it committed beside the code would be a second copy of the code's structure +that nothing verifies. It would rot exactly as every restatement in this +component rotted, and a stale "all cells covered" table is more dangerous than +no table. + +The division that does work: + +- **The matrix is a transient, exploratory instrument.** Draw it for one type + at one boundary to find out which correlations exist. It is expected to be + **sparse**; most cells are empty and discovering that is cheap. Correlations + cannot be derived -- which is why twenty-eight rounds of reading produced + none -- so populating it is exploration, not specification. +- **An oracle is the durable artifact.** Only cells that turn out to mean + something graduate into it. It stays small because discovery, not + enumeration, fills it. + +`windows-file-watcher`'s `ContractChecker` is this repository's worked example +of the oracle half: a shared executable definition of the rules, owned by the +crate that owns the contract, that the producing crate's own tests and every +consumer's test doubles all bind to. It already existed while this probe was +being written, and was not reached for. + +Every correlation admitted here is one the report already renders twice, with +nothing relating the two -- a property of the artifact rather than of anyone's +intuition about it. The ones this decision was written against are: + +1. an alarm in the report implies the verdict is not `agree`; +2. a fact rendered twice must agree across its renderings; +3. an uncaveated hardware claim implies `!parse_in_doubt`; +4. the banner names the same machine the body describes. + +That list is the seed, not the census: the admission RULE is what governs, and +the authoritative set is the `Correspondence` enum in +[src/report_oracle.rs](src/report_oracle.rs). An earlier version of this +paragraph said "three correlations" and listed the first three while the enum +already had the fourth -- the count was wrong when written and would have +rotted again at the next addition, so it is stated as a rule here instead. + +The replacement rule was then itself overstated, as "known to be real because it +was violated" -- which excludes the correspondences found by the M2.4 matrix, +where no defect had occurred and walking every NDJSON field against the prose is +what showed the fact rendered twice with nothing comparing it. Both routes are +admissible; what is not is inventing a correspondence between things the report +does not actually render twice. Corrected the same day it was written, after a +review noticed it contradicted `check_structured_pairs`' own history. + +What makes an oracle different from more tests is where it is invoked: if every +test renders *through* it, every existing call site inherits the checks and so +does every future one. A test added beside them checks one case; an oracle +checks every case anyone ever writes. (Also stated without a number on purpose +-- this said "all twenty-five existing call sites" and there are now 35.) + +**Record the vacuous findings too.** "We examined whether X and Y must +correspond, and they need not" is a result, and it is the half that normally +evaporates -- without it the next person re-explores the same empty cells. + +An oracle is a forcing function for correlations already discovered. It will +not find a new one. The discipline that makes it compound is that each newly +found cross-artifact contradiction adds an invariant to the oracle rather than +a one-off test. + +Whether this generalises to `Coherence`, `BracketOutcome`, `Verdict` and the +sibling probes is **an open question, deliberately not answered here.** The work +this decision implies is queued as M2 in [CHECKLIST.md](CHECKLIST.md); this +section schedules nothing on its own. + +## The oracle exists, and what it deliberately refuses to know + + + +**Superseded by [The encoded row is the contract; the prose is +not](DESIGN-NOTES.md#d-encoded-row-is-the-contract), and the code it describes no +longer exists.** Everything below is a record of what was built and why, in the +past tense whatever its grammar says: M3.4 deleted the prose correspondences, +the `Correspondence` enum and the twenty-three extraction helpers, and M3.5 +replaced the fact-accounting instrument. `report_oracle` today is a +well-formedness check on the row and nothing more. + +This paragraph read "the rest of this section ... still describes what is in the +tree and still holds", which was true when it was written -- before M3.4, three +commits earlier on the same branch -- and was carried through the Tier 1 / Tier 2 +split unchanged. Found by a review. It is the same drift this component keeps +paying for, and it is worth leaving the correction visible rather than quietly +deleting the sentence. + +M2.1 built it: [src/report_oracle.rs](src/report_oracle.rs), admitting only +correlations the report already renders twice. The defect that forced +it is the section above. + +**It reads the rendered artifact, never the state behind it.** Checking state +would miss precisely this defect class -- in the original finding the state was +consistent and the two *renderings* of it were not. + +That last sentence is the superseded one, and it is wrong about its own +evidence. Re-checked against the code: the alarm has no NDJSON key, and +`cross_check` does -- so the original finding was a run whose ENCODED ROW said +`agree` while the probe had detected its own bug, and published nothing about +that bug. The state was not consistent; the row was wrong. See +[#d-encoded-row-is-the-contract](DESIGN-NOTES.md#d-encoded-row-is-the-contract). + +**It relates two things already visible in the report, and re-derives nothing.** +A second implementation of the rendering rules would be a check of the copy +rather than of the contract, and would drift the moment either moved. So the +alarm rule compares an alarm line against a verdict line, the double-rendering +rule compares prose against NDJSON, and the gating rule compares a claim against +the report's own published evidence of doubt. + +That last one is the interesting boundary. `CrossCheck::parse_in_doubt` is +`!disagreements.is_empty() || !parse_incomplete.is_empty()`, and the NDJSON +publishes `parse_incomplete` as a **list of conditions** rather than the +predicate -- so the oracle reads whether that list is empty, together with the +`disagree` verdict, and those are the two visible shadows of that definition. The +coupling is deliberate, and confirming it still holds is what M2.2's sabotage +check is for when the call sites are bound. + +(This said "as a **count**", which M3.1 made false when the three diagnostic +fields began publishing their conditions. The shape of the argument is +unchanged -- the row still renders a shadow of the predicate rather than the +predicate -- but the shadow is now a list, and an emptiness test rather than a +comparison against `0`.) + +**Half the tests assert acceptance**, following +[../windows-file-watcher/src/contract.rs](../windows-file-watcher/src/contract.rs)'s +`ContractChecker`: an alarm beside a non-agreeing verdict is legal and is what +the fix produced, a caveated claim under doubt is legal and is what the renderer +emits on every heterogeneous host with a short parse, and a prose-only report is +silence rather than violation. Over-constraining is the same defect as +under-specifying and fails in the more expensive direction, because noise trains +a reader to ignore the instrument. + +### The failure mode that would look exactly like success + +An oracle whose prose labels do not match the renderer reads nothing, finds +nothing, and passes everything. So the labels were confirmed against a real +`probe-topology` run, and a test corrupts each double-rendered value in turn and +requires a violation -- if a label ever drifts, that test fails rather than the +oracle going quietly blind. + +**The first attempt at that injection silently did nothing**, and is worth +recording because it nearly produced the opposite conclusion. The anchor used +was `cross-check:`, which does not occur -- the real text is `cross-check +against independently read Win32 counters:` -- so the "defective" report was +identical to the clean one, the oracle correctly reported no violation, and the +reading was almost "the oracle is blind". A sabotage that fails to apply is +indistinguishable from an instrument that fails to fire, unless the injection +asserts it changed something. It now does. + +**The mirror-image hazard: a RESTORE that fails to rebuild.** Sabotage work in +this crate is a loop -- break it, run it, put it back, run it again -- and the +put-it-back step has its own way of lying. On Windows, PowerShell's `Copy-Item` +preserves the source file's `LastWriteTime`, so restoring a file from a backup +taken earlier gives it an mtime OLDER than the artifacts built from the +sabotaged version. Cargo fingerprints by mtime, decides nothing has changed, and +reruns the previous binary. Measured here: a restored, correct oracle reported +the fixed defect as still present, and the reading was almost "the fix does not +work" -- the conclusion was only avoided by printing the intermediate values and +finding that the function returned the right answer while the test insisted it +did not. + +After restoring a file by copy, set its timestamp forward +(`(Get-ChildItem ).LastWriteTime = Get-Date`) or rewrite it through a +read-then-write, which stamps it as a matter of course. The general rule is the +same one as above, pointed the other way: **a green result proves nothing until +you know the code you are testing is the code you just wrote.** + +### The real-host test, and the guard that stops it passing for nothing + +[tests/a_real_report_agrees_with_itself.rs](tests/a_real_report_agrees_with_itself.rs) +composes the report the way `probe-topology` does and applies the oracle +explicitly. + +**Why it has to exist.** The oracle's unit tests pin it against fixtures, and a +fixture is a report somebody wrote down -- so a fixture-bound oracle checks +correspondences over states its author already imagined, and the defect it +exists for was a state nobody had imagined. More narrowly, a fixture cannot +notice the *renderer* drifting away from the prose labels the oracle reads: +both sides would still agree with each other. Only the real artifact disagrees. + +Some unit tests in this crate do call `measure()` and so do read this host. +What none of them does is run the **oracle** over a report rendered from that +reading, which is the gap this test closes. On CI it runs across the hosted +runner fleet, a survey of shapes no fixture anticipates. + +**It asserts nothing about this machine, deliberately.** A test expecting a +processor count, a cache level or a verdict would fail on the next runner shape +rather than on a defect, and would have to be loosened until it asserted +nothing. What it checks is that whatever this host produced, the report's parts +agree with each other -- a property every host must satisfy, including one whose +topology cannot be read at all. + +#### The primary assertion can pass having checked nothing + +On a host whose report the oracle cannot parse, every lookup returns `None`, +every comparison is skipped, and +`a_report_rendered_from_this_host_agrees_with_itself` passes having checked +exactly zero correspondences. That is why the second test corrupts each +double-rendered fact in a report **this host really produced** and requires the +oracle to report a violation **naming that fact** -- and asserts first that the +corruption changed the text at all, for the reason recorded above. +Eight facts rather than one, because corrupting a single field would leave the +others unguarded: the renderer could drift away from the oracle's other prose +labels and the test would still pass on the strength of the one that remained. + +**"Naming that fact" is load-bearing, and took three attempts to get right.** +The guard first required only that the violation list was non-empty. That was +strengthened to require a `ProseAndNdjsonDisagree`, with a comment correctly +observing that corrupting the NDJSON `processors` count also trips the +cross-check counter rule -- and then not acting on the observation, because that +counter rule emits precisely that variant. So the strengthened guard still +passed while the reader it was written to protect was blind. + +Measured, by blinding one `DOUBLE_RENDERED` prose label at a time: under the +variant-only assertion, `processors` and `groups` both stayed **green**, masked +by their counter rules, while `packages` went red because nothing else reads it. +Two of the four facts the guard names as required were unchecked by the guard +whose entire purpose is to establish that they are checked. Matching on the +`fact` string closes it, because no neighbouring rule can supply another rule's +fact name. + +The general lesson, which this crate has now paid for four times: **a guard +against vacuity is itself a claim, and is subject to the same discipline as any +other.** Each of the three earlier versions was written to fix the previous +one's vacuity and introduced the next-narrower version of it. The only thing +that has ever settled the question is sabotaging the mechanism the guard is +supposed to protect and watching the guard go red -- never reading the guard and +judging it sufficient. + +### The fact set is derived, because the gaps were never in the rules + +Six unread double-renderings were found on this branch by six different +reviewers, and none by the oracle's own coverage. Each individual gap was real, +and fixing each one by hand was correct -- but the pattern is the finding, and +the pattern is structural: **a rule is added per fact, so the SET of facts is +what drifts.** Nothing derived that set from +[src/topology_report.rs](src/topology_report.rs), so a field added to the +renderer was unread until a person happened to notice. + +The clearest demonstration is gap 6. It was created BY the hand-written rule +that closed gap 2: M2.11 added a comparison for the partitioning discriminator, +that arm renders a level number as well, and nothing compared the number. Closing +a gap added a fact, and the new fact was unread by exactly the mechanism that +produced the previous five. + +`every_fact_the_renderer_publishes_is_accounted_for` in +[tests/a_real_report_agrees_with_itself.rs](tests/a_real_report_agrees_with_itself.rs) +is the answer, and its shape matters more than its code. Both halves come from +places that cannot fall out of step: + +- **The set of facts is enumerated from the artifact** -- the NDJSON line of a + report this host really rendered. Not a list in the test. A new field appears + in the enumeration by itself, without anyone maintaining anything. +- **Read-or-unread is measured, not declared** -- each value is corrupted in turn + and the oracle is asked. A rule that quietly stops reading a fact is caught + even though no list changed. + +Only the classification of each key is written down, because which of the three +kinds a key belongs to is a judgement: compared, conditional on its prose being +rendered, or having no second rendering at all. **A key in no list fails**, and +that is the whole point -- adding a fact to the renderer forces the judgement to +be made deliberately instead of being discovered by the seventh reviewer. + +This is the same discipline the oracle itself rests on, turned on the oracle: +relate two things that already exist rather than restate one of them. A second +list of "facts the rules look at" would have been another copy to drift, which +is the defect rather than the fix. + +**The limit, stated because a reader will otherwise assume it is closed.** The +enumeration is keyed to the machine-readable line, because that is the side with +an enumerable structure. A fact rendered only in PROSE, with no field beside it, +is invisible here. Prose is not enumerable without parsing English, so that half +remains a thing only a reader notices -- and saying so is better than implying a +coverage that does not exist. + +### The instrument is code too, and it is where the defects were + +Eleven review rounds across six models ran over this branch. Classifying every +finding by what would have caught it earlier is more useful than the findings +themselves, because the classes are very unevenly sized. + +**The largest class by far is shape blindness**, and it has one root: every +instrument here was validated against a single artifact, the developer machine's. +This host produces exactly one shape -- measured, `agree`, the `Level` +partitioning arm, non-empty caches, classes `[0]`, zero anomalies, zero +`not_compared`, x86_64 -- and every defect in that class lived in the complement +of it. The architecture uncompared on an unmeasured report, `efficiency_classes` +comparing contents so a scalar and a one-element list were identical, the anomaly +count unread under two of three verdicts, the expected fact name differing on the +`SummaryMissing` arm, an empty container that could not be corrupted: none of +them can occur here, and each was found only because a reviewer imagined a shape +by hand. [CHECKLIST.md](CHECKLIST.md) -> M2.12 is the structural answer, and it +is M2.10's move one level up: derive the set of SHAPES from the renderer's +branches, as M2.10 derives the set of FACTS from the artifact. + +**The second observation is the one worth carrying to other crates.** Most of +these defects were not in the oracle. They were in the guard that checks the +oracle, the table of required facts, the fact name the guard expects, the +declaration of when silence is legitimate. The thing under test came through the +last rounds clean; the things doing the testing did not. + +An instrument feels like it sits outside the system under test, so it escapes the +discipline applied to production code -- and then it fails on a CI runner +reporting a defect in the renderer that is really a defect in the instrument, +which is worse than no check at all because it sends the reader to the wrong +place. Instruments need what production code gets: derivation instead of +restatement, sabotage before they are believed, and a stated boundary. + +**And a fix is new code.** Twice on this branch the fix for one gap created the +next: M2.11's discriminator rule gave the level a second fact name, which broke +the guard that expected one; M2.10's accounting inherited a silence assumption +that only holds under an `agree` verdict. After a fix, re-run the derivation or +the sweep -- not only the test that prompted it. + +### Mutation testing is how the instruments got checked, and what it cannot reach + +Every defect on this branch was found by a person reading code -- reviewers, +mostly, and this file records how often they found the same shape. Late on, the +obvious question got asked: is there a mechanical way to ask whether a test +establishes anything, rather than trusting that it does? + +There is, and it was already installed. `cargo-mutants` changes the source and +asks whether anything notices, which is the sabotage loop this crate has been +running by hand all along, done exhaustively. It answers a stronger question than +coverage: not *was this branch executed* but *does anything DETECT a change to +it*. On a branch whose recurring defect is a test that runs code without +establishing anything about it, that difference is the whole point. + +Six sweeps, run through [tools/run-mutants.ps1](../../tools/run-mutants.ps1). The +first three predate the M3 rewrite and are kept for the arithmetic note below; +the last three cover the three modules M3 created, none of which any sweep had +reached: + +| file | tested | caught | unviable | survivors | +|---|---|---|---|---| +| `topology_report.rs` | 28 | 28 | 0 | 2, then none | +| `report_oracle.rs` | 143 | 138 | 5 | 6, then none | +| `topology.rs` | 186 | 180 | 6 | none, first run | +| `row.rs` | 25 | 19 | 6 | none, first run | +| `topology/invariant.rs` | 26 | 21 | 2 | 3, then one equivalent | +| `topology/diagnostic.rs` | 28 | 23 | 5 | **12**, then two prose, then none | + +**The three later sweeps are the argument for running them at all**, and each +made a different case. `row.rs` -- the crate's only defence against caller text +reaching the mined artifact -- came back clean on its first run, which no amount +of review could have established. `topology/invariant.rs` gave up a real gap that +five review rounds across four models had not: relaxing `online_processors > 0` +to `>= 0` survived, because the test that NAMES that boundary asserts on +`cross_check` and so covered only one of the two deliberate copies of the +condition. + +**`topology/diagnostic.rs` is the one that mattered.** It survived 12 of 26 -- +46% -- and the survivors were the row's own payloads: `NotCompared::code` could +be replaced wholesale with `""`, every arm of `published_anomaly` deleted, and +two arms of `anomaly_code` deleted so that a real buffer overrun would publish as +`unclassified`, whose documented meaning is the opposite. Measured separately: +rewriting the `count` helper so every published count was wrong left 218 tests +green. + +This is the field-labelling defect [src/row.rs](src/row.rs) exists to make +unrepresentable, reappearing one level down. `row.rs` pairs a name with its value +so position cannot mislabel them; these functions then hand-pair names with +values INSIDE each entry, and nothing was watching. The tests that looked like +they covered it built their expectation from `code()` and compared it against a +row the writer had built from `code()` -- both sides moving together, which is +the tautology class this branch deletes elsewhere. + +Closed with goldens in [src/topology/diagnostic/tests.rs](src/topology/diagnostic/tests.rs), +written as literals on purpose: a code is a SCHEMA, not a predicate, and a schema +is not derivable from the thing that emits it. Completeness is compiler-checked +where the enum belongs to this crate, via an exhaustive `match` in the test. + +**The last two survivors were both `Display` impls, and how they were closed is +the part worth keeping.** Blanking either left the suite green, and a reader +would have got ` - ` with nothing after the dash. Both obvious fixes were +wrong: pinning the sentences would make the prose machine-checked, which +[DESIGN-NOTES.md](DESIGN-NOTES.md) -> `d-encoded-row-is-the-contract` deliberately +does not do, and leaving them ships a blank line. + +The engineer's question -- "a blank line is perhaps wrong, perhaps it should be +called out?" -- named a third answer better than either. **A diagnostic's WORDING +is a review obligation; its PRESENCE is not.** The renderer now routes every +entry through `described`, which substitutes an explicit `BUG IN THIS PROBE` line +for a rendering that is empty or blank, and a test asserts that no diagnostic +renders as that line. That pins THAT each entry describes itself without pinning +WHAT it says. + +Both improvements fall out of the same branch. A reader gets a stated defect +instead of a blank -- which is indistinguishable from a rendering bug, from a +finding with genuinely nothing to say, and from a stray newline -- and the +mutants become catchable, because blanking a `Display` now produces the callout +the test forbids. The sweep went to **zero survivors**, including the two mutants +`described` itself introduced. + +The `assert_holds` survivor is equivalent, for the same reason `assert_corresponds` +survived below, and the argument is now recorded at the function rather than left +for the next sweep to rediscover. + +The unviable column is why a caught-count does not equal a tested-count: those +mutants did not compile, so they say nothing either way. An earlier version of +this table gave only the caught figures, and the commit that wrote it summed them +to a total that matched neither -- 357 were tested and 346 caught, and it claimed +352. Stated in full here so the arithmetic is checkable rather than asserted, +which is the same rule this branch keeps having to relearn about numbers. + +**The survivor worth remembering is `assert_corresponds`.** Replacing its body +with `()` survived, because every instrument that would notice goes THROUGH it -- +both renderers are bound to it, the real-host test calls it, the corpus reaches +it by rendering -- so a no-op assertion makes all of them pass together. Nothing +asserted that the assertion asserts. That is a blind spot no amount of adding +tests *through* an instrument can find, and it is exactly what a tool that +attacks the code rather than the tests is for. + +The others were smaller and of one kind: guards that only fire on malformed +input, which every fixture was too well-formed to reach. A `> 0` that stops an +empty value being reported as a value; the `&&` that makes `!!...!!` a shape +requiring both ends; a string branch whose only caller always passes an array. + +**What it cannot reach, and this matters here.** `cargo-mutants` mutates `src/`, +not `tests/`. The accounting table, the shape corpus and the fact-name guard all +live in `tests/`, so the tool validates the code they check and says nothing +about THEM. The instruments remain exactly as good as the hand-sabotage that +built them -- which is where several of this branch's defects were found, and +where the next one will be. A clean sweep is evidence about the oracle, not about +the things measuring it. diff --git a/crates/windows-platform-probes/PLANS.md b/crates/windows-platform-probes/PLANS.md index 00a4f7252..95765ff82 100644 --- a/crates/windows-platform-probes/PLANS.md +++ b/crates/windows-platform-probes/PLANS.md @@ -4,5 +4,5 @@ Design decisions are in [DESIGN-NOTES.md](DESIGN-NOTES.md). | Path to CHECKLIST.md | Status | Brief description | Design Notes | |---|---|---|---| -| [CHECKLIST.md](CHECKLIST.md) | in progress | M1: stream a probe's report as it is measured. The report sink buffers each report into a `String`, so a termination that does not unwind -- Ctrl-C, or an abort during unwinding -- discards it, where the line-by-line printing it replaced kept it. Costs most on `probe-cancel-io`, which runs about twenty seconds precisely when the wedge it hunts for occurs. M2: check correspondence between a report's parts rather than each part alone, after a pull-request review found the renderer calling a state a bug while the verdict certified the same run as `agree`. | [DESIGN-NOTES.md](DESIGN-NOTES.md#d-buffered-report), [#d-correspondence-failures](DESIGN-NOTES.md#d-correspondence-failures) | +| [CHECKLIST.md](CHECKLIST.md) | in progress | M1: stream a probe's report as it is measured; **done and archived**. The sink USED TO buffer each report into a `String`, so a termination that did not unwind -- Ctrl-C, or an abort during unwinding -- discarded it, where the line-by-line printing it replaced kept it; that cost most on `probe-cancel-io`, which runs about twenty seconds precisely when the wedge it hunts for occurs. Every probe now writes into the sink as it measures, through a `fmt::Write` adapter, and completed lines reach `Stdout` via `LineSink`. M2: check correspondence between a report's parts rather than each part alone, after a pull-request review found the renderer calling a state a bug while the verdict certified the same run as `agree`; complete -- the oracle, the renderer binding, the real-host test, the derived fact set, the partitioning discriminator and the shape corpus all landed, and its ten unrelated leftovers were re-sequenced into M4 and M5. M3: make the encoded row the contract and stop checking the prose against it. Re-reading M2's own evidence showed both defects that motivated the oracle were defects in the ROW -- and that the row published its three diagnostic lists as bare counts, so a survey could not tell a probe self-bug from host flakiness. The row gets the facts and the invariants; the prose gets review. **Complete and archived, ten items.** The three fields publish arrays of OBJECTS, each carrying a stable `code` plus the values its variant holds -- `{"code":"partitioning_summary_missing","level":9}` rather than the bare code of the superseded M3.1 form -- and each key's value SHAPE is now declared beside its name in `MEASURED_ROW_SHAPES`; the surviving correspondences became invariants over the observation rather than over two renderings; the row is emitted from a typed value through one writer with total escaping; the prose oracle and its parsers are gone, and no test extracts structured data from prose anywhere in the crate. The last four items came from reviews: three instruments asserted less than their names claimed, `BlockingState::ALL` was a census the compiler did not check while a doc comment said it did, and the hand-written JSON check was measured against a real parser over 1807 generated corruptions -- 159 disagreements, every one a false ACCEPT -- then replaced by `serde_json`, after which the remaining hand-written string scanners were deleted. M4 holds the four carried-over items M3 gated, now unblocked and re-scoped; M5 the six that nothing gates, which may be pulled forward at any time. | [DESIGN-NOTES.md](DESIGN-NOTES.md#d-buffered-report), [#d-encoded-row-is-the-contract](DESIGN-NOTES.md#d-encoded-row-is-the-contract), [DESIGN-RATIONALE.md](DESIGN-RATIONALE.md#d-correspondence-failures) | | [../../CHECKLIST-thread-ambient.md](../../CHECKLIST-thread-ambient.md) | in progress | M27: create the crate, migrate this session's probes into it under the three-tier scheme, and queue migration of the nine earlier measurements that still live only in git-ignored scratch. | [DESIGN-NOTES.md](DESIGN-NOTES.md) | diff --git a/crates/windows-platform-probes/design-sessions/DESIGN-SESSION-2026-09-12-what-the-oracle-should-read.md b/crates/windows-platform-probes/design-sessions/DESIGN-SESSION-2026-09-12-what-the-oracle-should-read.md new file mode 100644 index 000000000..56676e502 --- /dev/null +++ b/crates/windows-platform-probes/design-sessions/DESIGN-SESSION-2026-09-12-what-the-oracle-should-read.md @@ -0,0 +1,199 @@ +# Design session 2026-09-12: what the oracle should read + +Decisions resulting from this session: + +- [DESIGN-NOTES.md](../DESIGN-NOTES.md) -> [The encoded row is the contract; the prose is + not](../DESIGN-NOTES.md#d-encoded-row-is-the-contract) (new, and supersedes the + artifact-reading rule recorded in [The oracle exists, and what it deliberately refuses to + know](../DESIGN-RATIONALE.md#d-oracle-refuses-to-know)). + +Work queued from it: [CHECKLIST.md](../CHECKLIST.md) milestone M3. + +## The question + +Held immediately after PR #88 merged. The engineer asked, of the oracle that PR had just +built: **why does it have to be based on the formatted string?** + +The recorded answer at that moment was the one in the module doc of +[src/report_oracle.rs](../src/report_oracle.rs) and in Tier 1: it reads the rendered +artifact, never the state behind it, because "in the original finding the state was +consistent and the two renderings of it were not." + +The session set out to test that sentence against its own evidence rather than to defend +it. + +## What the evidence turned out to be + +The oracle's charter names two originating defects, both found by a pull-request review +after twenty-eight rounds of per-artifact review and a zero-surviving-mutant `cargo-mutants` +result had passed over them. They are recorded in [DESIGN-NOTES.md](../DESIGN-NOTES.md) -> +[The defects that survived were correspondence +failures](../DESIGN-RATIONALE.md#d-correspondence-failures). + +### Defect 1 -- the alarm beside the agreeing verdict + +The renderer printed `BUG IN THIS PROBE: the topology crate named L3 as the outermost +partitioning cache and this survey carries no summary for it. Nothing below about cache +partitioning can be trusted.` while `cross_check` had no branch for that state, so +`verdict()` could return `Agree` and print `=> agree` two paragraphs below. + +Checked during the session, in the code as it stands on `main`: + +- The alarm is emitted by a `writeln!` into the prose, in `report`'s + `PartitioningCache::Level` arm of [src/topology_report.rs](../src/topology_report.rs). +- The NDJSON row carries eighteen keys **as of this session** -- it carries nineteen now, + because the work this session set off added `disagreements` (in `77a83fc`, after + [4d7544e](../DESIGN-NOTES.md) recorded this). The count is left at eighteen on purpose: a + review reported it as a stale census, and it is not one. The diagnosis below turns on + what the row held AT THE TIME, so correcting the number to nineteen would make the next + bullet -- "there is no key for the alarm" -- read as an error rather than as the finding. + For the current schema read `MEASURED_ROW_KEYS` and `MEASURED_ROW_SHAPES` in + [src/topology_report.rs](../src/topology_report.rs), which are the one authority; the list + that follows is **the eighteen as they stood at this session** and is not reproduced here as + current. (A review read it as a current-schema list missing `disagreements`, which is a fair + reading of how it was introduced -- hence this sentence.) + + `reason`, `arch`, `processors`, `groups`, + `packages`, `numa_domains`, `numa_domains_without_processors`, `cores`, + `efficiency_classes`, `caches`, `outermost_partitioning_cache_level`, + `outermost_partitioning_cache`, `policies`, `cross_check`, `not_compared`, + `parse_incomplete`, `enumeration_anomalies`, `numa_domains_only_in_cpu_sets`. +- **There is no key for the alarm.** `cross_check` is there, and on the defective run it + read `agree`. + +That changes the diagnosis. The defect was not a disagreement between two renderings of a +consistent state. **The encoded row was wrong**: it certified a run as `agree` on which the +probe had detected its own bug, and it published nothing at all about that bug. A +fleet-survey pass mining the NDJSON would have counted the host as a clean agreeing +measurement and never known otherwise. + +The prose alarm was not the defect. It was the only visible trace that the encoded row was +wrong -- which is exactly why a human reviewer found it and no instrument did. + +So the repair that actually addresses defect 1 is not "compare the prose against the +NDJSON". It is **publish the alarming condition structurally, and assert the invariant on +the data**: a run that detected a missing summary cannot also be `agree`. That is +checkable on the observation, before any rendering, with no parser. + +### Defect 2 -- `efficiency classes: [0]` against `"efficiency_classes":1` + +One fact rendered twice, in two shapes a consumer cannot reconcile: the prose printed the +set of class labels, the NDJSON printed the cardinality, and the numeral `1` reads as a +plausible class *label*. + +Both halves were correct derivations from one consistent value, so no struct-level +predicate was violated. At first reading this looks like the strongest case for an oracle +that reads text -- a contradiction that exists only in the representation. + +But look at how it was actually repaired. The row now emits +`"efficiency_classes":[0]` -- the class LABELS as a list. **The fix was to change what the row publishes.** +The prose comparison was the route by which a reviewer noticed, not the repair. + +### A correction made while writing this up, and it strengthened the case + +The first draft of the Tier 1 decision said the alarm's condition is unpublished and +the verdict can therefore still read `agree`. Checked before committing, and the second +half is **no longer true**: `Observation::cross_check` now pushes +`PartitioningCache::SummaryMissing` onto `parse_incomplete`, which forces the verdict away +from `agree`. That was the original repair, and it worked. + +Writing the superseding decision on a description of the code as it used to be would have +been the exact defect class this branch has spent fifteen review rounds on. What the check +found instead is a gap that is live, and larger: + +The row publishes `not_compared`, `parse_incomplete` and `enumeration_anomalies` as +**counts** -- `check.parse_incomplete.len()` -- where the prose prints each entry's text. A +survey reading `"parse_incomplete":1` cannot tell *the probe detected a bug in itself* from +*a core record contradicted itself* from *cache levels numbered 0* from *this topology was +not measured from a running machine*. Four categorically different facts, one cardinality, +and only the prose separates them. + +So the accurate statement of the problem is not that the row disagrees with the prose. It +is that **the row is impoverished relative to the prose**: the artifact that gets mined +carries strictly less than the artifact that gets read. Which is the engineer's point, +sharpened -- the effort went into comparing the two halves when the encoded half was the +one missing content. + +### Both defects were defects in the encoded data + +That is the session's central finding, and it was not what either the module doc or Tier 1 +said. Defect 1 was a wrong value in the row plus a missing field. Defect 2 was the row +publishing a cardinality where the useful fact was the set. Neither required a +prose-against-NDJSON oracle to *fix*; both required the structured output to be made +right. + +## The cost that was being paid for the other reading + +Counted during the session, in [src/report_oracle.rs](../src/report_oracle.rs): of 38 +top-level functions, ten are correspondence rules and four are comparison helpers -- +and **twenty-three exist purely to extract values back out of rendered text** +(`fingerprint_tokens`, `ndjson_field`, `ndjson_raw_field`, `balanced_end`, `claim_block`, +`prose_field`, `cache_object`, `cache_rows`, `policy_rows`, `leading_digits`, +`has_line_beginning`, `object_keys`, and the rest). + +That ratio is the indictment, and the branch's own review history is the evidence for it. A +large share of PR #88's review rounds were defects in **the reader, not in the thing read**: +a multi-byte panic in `processors_in_banner`, a `p/` substring matching inside an opaque +`io::Error`, `trim_matches` collapsing `[[0]]` and `[0]` to the same value, +`prose_field(" (")` selecting the wrong line when two lines began the same way. + +Those are not defects in the probe. They are defects in a hand-written parser of output +this crate had just finished writing -- a defect source the design invented for itself. + +## The engineer's framing, which the evidence supports + +> There is a data pipeline, and at some point it renders either to a structured format or +> to prose. The prose has to be accurate and readable, but I really do not see value in +> ensuring it is programmatically comparable. The structured format? Yes, absolutely. +> +> Yes, humans read the prose. But we should present prose which you and then optionally I +> will inspect for correspondence, and then once that's done, we'll move along. And there +> may be defects there. But it's the defects in the encoded data that really matter. + +This is the decision recorded in Tier 1. The session's contribution is that it is not +merely a preference about where to spend effort -- the two defects that motivated the +oracle in the first place were both defects in the encoded data, so the evidence that was +taken to argue for reading prose argues for the opposite. + +## Alternatives considered + +**Keep the oracle as it is.** Rejected. It is not wrong, and it does catch real things -- +but it aims the expensive machinery at the artifact with the lower stakes, and pays for it +with twenty-three parser functions that are themselves a defect source. The prose is +reviewable by a human; the row is mined by a machine across a fleet and is what this +workspace's designs rest on. + +**A typed banner (the previous M2.18).** Dissolved into this decision rather than answered +on its own terms. It was the smallest instance of a general question -- should a report be a +value that a writer renders, or a string the renderer concatenates -- and answering it in +isolation would have fixed one parameter while leaving the same shape everywhere else. + +**A structured report checked as a struct, with the oracle moved wholesale onto it.** +Tempting and half right. It closes value-divergence and injection by construction, which is +better than detecting them. But it must not be mistaken for a complete answer: a check on +the struct says nothing about the writer, and the writer is where several of PR #88's +defects lived (`is_attribution_shaped` matching a suffix, the flattening that ate the +disclaimer, the `host: host:` doubling). The conclusion taken was the narrower one -- assert +the invariants on the data, emit the row through one typed writer so injection and +field-order defects are unrepresentable, and keep only a thin check that the row is +well-formed. + +**Rejected framing: "the prose does not matter".** Not what was decided, and worth stating +because it is the easy misreading. The prose must be accurate; an overstated finding in +prose propagates into design notes that cite it, which is a live concern in this crate -- +M2.9 is an open item about exactly that. What was decided is that prose accuracy is a +**human review** obligation rather than a machine-checked correspondence. + +## What survives from PR #88 + +Worth recording, because the decision reads as a larger reversal than it is: + +- The correspondence *rules* survive, relocated. Alarm-against-verdict, + diagnostics-against-verdict and counters-against-verdict all become invariants on the + observation. What they lose is the parser in front of them. +- The containment work in [src/topology_report.rs](../src/topology_report.rs) survives and + matters more under this decision, not less: it stops caller text reaching the row. +- The shape corpus and the fact-accounting instrument survive in shape, re-aimed at the + row's fields rather than at prose labels. +- What retires is the prose-against-NDJSON family and the extraction helpers that serve + only it. diff --git a/crates/windows-platform-probes/sabotage.json b/crates/windows-platform-probes/sabotage.json new file mode 100644 index 000000000..9ef82648b --- /dev/null +++ b/crates/windows-platform-probes/sabotage.json @@ -0,0 +1,117 @@ +{ + "description": "Sabotage manifest for windows-platform-probes. Each entry is a defect that was actually run by hand while building M3; the manifest exists so those one-off verifications stop being discarded. See CHECKLIST.md M2.14.1.", + "package": "windows-platform-probes", + "sabotages": [ + { + "name": "row drops the packages field", + "file": "src/topology_report.rs", + "expect": "caught", + "why": "A field silently vanishes from every mined row. This is the defect that was MEASURED to leave the whole suite green before MEASURED_ROW_KEYS existed, so this entry is the regression test for the key-set contract itself.", + "find": [ + " .with(\"packages\", observation.packages)" + ], + "replace": [] + }, + { + "name": "row writer emits a leading separator", + "file": "src/row.rs", + "expect": "caught", + "why": "Produces a row that is bracket-balanced but not valid JSON, so a consumer's parser rejects it. The oracle's old `balanced` check counted depth and passed exactly this; `malformation` must not.", + "find": [ + " Self::Object(members) => {", + " out.push('{');", + " for (at, (name, value)) in members.iter().enumerate() {", + " if at > 0 {", + " out.push(',');", + " }" + ], + "replace": [ + " Self::Object(members) => {", + " out.push('{');", + " for (_at, (name, value)) in members.iter().enumerate() {", + " out.push(',');" + ] + }, + { + "name": "row writer does not escape quotes", + "file": "src/row.rs", + "expect": "caught", + "why": "A quote in a caller's `io::Error` ends the JSON string early and lets the rest of the message forge keys in the mined row. `write_escaped` is the whole of the injection fix.", + "find": [ + " '\"' => out.push_str(\"\\\\\\\"\")," + ], + "replace": [ + " '\"' => out.push('\"')," + ] + }, + { + "name": "the row's key reader uses a parsed map", + "file": "src/report_oracle.rs", + "expect": "caught", + "why": "The obvious simplification, and it is wrong twice over: a serde_json::Map SORTS its names and silently keeps the LAST of a repeated key -- so this loses the row's key ORDER, which is part of the contract MEASURED_ROW_KEYS states, and it deletes the evidence for RowDefect::RepeatedKey, the one malformation that survives a consumer's parse. This entry exists so that the reason the visitor is not a map cannot be forgotten.", + "find": [ + " let mut reader = serde_json::Deserializer::from_str(row);", + " let Ok(names) = serde::Deserializer::deserialize_map(&mut reader, TopLevelNames) else {", + " return Vec::new();", + " };", + " if reader.end().is_err() {", + " return Vec::new();", + " }", + " names" + ], + "replace": [ + " serde_json::from_str::>(row)", + " .map(|members| members.into_iter().map(|(name, _)| name).collect())", + " .unwrap_or_default()" + ] + }, + { + "name": "cross_check forgets the changed bracket", + "file": "src/topology.rs", + "expect": "caught", + "why": "THE entry this crate's invariant module exists for. Deleting the push empties the list, so `verdict()` -- a pure function of the lists -- legitimately returns `agree` beside an observation whose bracket did not hold. Any rule reading the LISTS is blind to this by construction; only a rule reading the OBSERVATION catches it.", + "find": [ + " check.not_compared.push(NotCompared::MachineChanged);" + ], + "replace": [] + }, + { + "name": "a blocking state with no perturbation", + "file": "src/topology/invariant.rs", + "expect": "caught", + "why": "An untested blocking state. Adds rather than deletes, deliberately: the completeness guard exists to catch an ADDITION, and a hand-written `ALL` was measured to let exactly this through while all ten invariant tests stayed green.", + "find": [ + " BracketNotHeld => \"the bracket did not establish that the machine held still\"," + ], + "replace": [ + " BracketNotHeld => \"the bracket did not establish that the machine held still\",", + " /// A state added by a sabotage, with no perturbation behind it.", + " SabotageSentinel => \"a state no perturbation produces\"," + ] + }, + { + "name": "prose reworded to carry the same fact", + "file": "src/topology_report.rs", + "expect": "survives", + "why": "THE CONTROL, and it encodes this component's central decision as a measurement rather than as prose: the row is the machine contract and the prose is for a reader, so no test may derive a MACHINE-CONTRACT requirement from the wording. Note the narrowness -- tests may and do assert prose (about 35 of them read report text with contains), because the human report is a deliverable and its content is worth covering. What none of them may do is parse the prose, or pin a sentence the row is separately responsible for. So this entry rewords a line no test asserts, and if it is ever reported as caught, the question to ask is WHICH test caught it: one that reads the wording to learn something the row already publishes is the defect, and one that simply covers this sentence means the entry needs a different sentence.", + "find": [ + " let _ = writeln!(out, \"cache boundary divides this machine.\");" + ], + "replace": [ + " let _ = writeln!(out, \"cache boundary splits this machine.\");" + ] + }, + { + "name": "blocking_states relaxes the processor guard", + "file": "src/topology/invariant.rs", + "expect": "caught", + "why": "The guard says a machine with processors and no packages is a finding; relaxing it to >= makes a topology that reported NOTHING a finding too, accusing a synthetic observation of hiding packages it never claimed. This condition exists TWICE on purpose -- cross_check has its own copy, and blocking_states must recompute rather than read cross_check's output, or it restates verdict() and goes blind to a deleted push site. Independence is the design; agreement is the property. A mutation sweep found the property untested: the test that names this boundary asserts on cross_check, so it covered only the other copy, and this mutation survived five review rounds across four models. Kept here so the pairing is re-run rather than re-discovered.", + "find": [ + " if observation.online_processors > 0 && observation.packages == 0 {" + ], + "replace": [ + " if observation.online_processors >= 0 && observation.packages == 0 {" + ] + } + ] +} diff --git a/crates/windows-platform-probes/src/lib.rs b/crates/windows-platform-probes/src/lib.rs index 8b0e7b2f6..a507d54f7 100644 --- a/crates/windows-platform-probes/src/lib.rs +++ b/crates/windows-platform-probes/src/lib.rs @@ -149,8 +149,17 @@ pub mod long_path; pub mod long_path_report; pub mod pool_growth; pub mod report; +/// The report oracle. **Test-support: present only where it is used.** +/// +/// Every caller is already behind this gate -- the renderers' `assert_row_is_well_formed` +/// bindings, the unit tests, and the integration tests, which reach it through the +/// self dev-dependency. Stating that here rather than leaving it implied is what +/// lets the module depend on a real JSON parser without putting one in a shipping +/// probe binary. +#[cfg(any(test, feature = "oracle-in-renderer"))] pub mod report_oracle; pub mod request_cost; +pub mod row; pub mod topology; pub mod topology_report; pub mod worker_context; diff --git a/crates/windows-platform-probes/src/report_oracle.rs b/crates/windows-platform-probes/src/report_oracle.rs index 1b9b3a082..b2590f620 100644 --- a/crates/windows-platform-probes/src/report_oracle.rs +++ b/crates/windows-platform-probes/src/report_oracle.rs @@ -1,1664 +1,453 @@ // Copyright (c) Mike Grier. -//! Correspondences a rendered report must satisfy *between* its parts. +//! Whether a rendered report's machine-readable row is well-formed. //! -//! # Why this exists rather than more per-part tests +//! # What this used to be, and why it is not that any more //! -//! A pull-request review found [`crate::topology_report`] printing -//! `BUG IN THIS PROBE ... Nothing below about cache partitioning can be -//! trusted` while the verdict two paragraphs below read `=> agree`. Twenty-eight -//! rounds of per-artifact review and a zero-surviving-mutant `cargo-mutants` run -//! had both passed over it, because **every function involved was correct on its -//! own terms** and the defect lived in the relation between two of them. +//! This module was an oracle over the report's PROSE: it read the rendered +//! sentences, extracted values back out of them, and compared those against the +//! NDJSON row. It was built after a pull-request review found a report calling a +//! state a bug while the verdict two paragraphs below certified the same run as +//! `agree`. //! -//! That is the shape no per-part instrument can see. A test asserts one -//! function's output; a mutant perturbs one function's behaviour; a reviewer -//! reads one artifact and finds it locally true. A contradiction between two -//! locally-true parts is invisible to all three. +//! [DESIGN-NOTES.md](../DESIGN-NOTES.md#d-encoded-row-is-the-contract) retired +//! that design. The row is a machine contract -- mined across a fleet, and what +//! this workspace's designs rest on -- and the prose is for a reader. They carry +//! different obligations: the row must be CORRECT, machine-enforced; the prose +//! must be ACCURATE AND READABLE, enforced by review. Nothing is required to +//! hold *between* them. //! -//! # It reads the artifact, not the state that produced it +//! The correspondences worth keeping were never about rendering. They related a +//! STATE to the verdict, and they live in [`crate::topology::invariant`] now, as +//! predicates over the observation that run whether or not anything was +//! rendered. Of the thirty-eight functions this module carried, twenty-three +//! existed only to extract values back out of rendered text -- a parser for a +//! format this crate itself writes, and it behaved like one: a multi-byte panic, +//! a substring matching inside an opaque `io::Error`, a `trim_matches` +//! collapsing `[[0]]` and `[0]`. None of those was a defect in a probe. //! -//! Every check here works on the **rendered text** -- the thing a reader and a -//! log-mining pass actually receive. Checking internal state instead would miss -//! precisely the defect class this exists for: the state was consistent in the -//! case above, and the two renderings of it were not. +//! # What is left, and why anything is left at all //! -//! # What it deliberately does not do +//! Structure makes most of the old checks unrepresentable rather than detected, +//! which is the stronger move. What structure cannot check is **the writer** -- +//! whatever turns values into bytes is downstream of every type, and several of +//! this crate's defects lived exactly there. So one check survives: the report +//! carries exactly one machine-readable row, and that row is a well-formed JSON +//! object. **Not flat** -- an earlier version of this sentence said flat, which +//! the row has not been since it began publishing diagnostics: `caches`, +//! `policies` and the three diagnostic lists are nested arrays and objects. A +//! reader who believed it would have taken the nested data for a defect. //! -//! It does not re-derive what the renderer should have printed. A second -//! implementation of the rendering rules would be a check of the copy rather -//! than of the contract, and would drift from the original the moment either -//! moved. Each rule below relates **two things already visible in the report**, -//! so the oracle has no opinion of its own to go stale. +//! That is not a correspondence. It is the writer's own output being read back, +//! which is the one thing no amount of typing upstream can do for itself. //! -//! Over-constraining is the same defect as under-specifying, so a report that -//! omits a fact is not a violation -- absence is checked only where the report -//! itself makes a claim that requires the other part to agree. -//! -//! # Every correlation here is one the REPORT already renders twice -//! -//! That is the admission rule. A correspondence is added when the report states -//! the same fact in both halves and nothing relates the two -- never because -//! someone imagined that two things ought to correspond. The test is a property -//! of the artifact, not of anyone's intuition about it. -//! -//! **Two ways of finding one, and the rule admits both.** Most were caught the -//! hard way, by a rendered report contradicting itself; each variant below names -//! the contradiction that earned it. The structured pairs in -//! `check_structured_pairs` (private, so named rather than linked) were found -//! the systematic way instead, by walking -//! every NDJSON field against the prose and seeing which facts were rendered -//! twice with nothing comparing them -- no defect had occurred, and waiting for -//! one would have been the worse plan. -//! -//! This section said "every correlation here was observed violated", which -//! excluded the second route and so contradicted that function's own history -//! four screens below. Written today, while correcting a different overstatement -//! in the same paragraph; a rule stated more strongly than the code supports is -//! the exact defect this module exists to catch, and it went in as part of the -//! fix for one. Found by a review. -//! -//! **Stated as a rule rather than a count, deliberately.** This paragraph used -//! to read "seeded with three correlations ... a fourth is added when a fourth -//! contradiction is found" -- while the enum directly below it already had four -//! variants. The census was wrong when it was written, survived every review of -//! this branch, and would have gone stale again at the next addition even if it -//! had been right. The rule cannot: it stays true however many there are, and -//! it is the part a reader actually needs, since what matters is that nothing -//! here is speculative rather than how much of it there is. - -/// A correspondence between two parts of a report that did not hold. +//! **The key set is checked, but not here.** This module reads a row it is +//! handed and has no way to know which keys were owed; the contract lives beside +//! the renderer that owes them, as `topology_report::MEASURED_ROW_KEYS`. An +//! earlier version of this paragraph said the check was deferred until the row +//! became a typed value, "at which point the key set is derivable from the type" +//! -- which is false, and was measured to be: a type says "a map of names to +//! values", which every key set satisfies, including the one missing a field. + +/// A way the report's machine-readable row is not well-formed. #[derive(Clone, Debug, PartialEq, Eq)] -pub enum Correspondence { - /// The prose raised an alarm while the verdict said everything matched. - /// - /// The original defect. An alarm is a statement that some part of the - /// report cannot be trusted; a verdict of `agree` is a statement that every - /// check made was made and matched. Both can be locally true and they - /// cannot both describe the same run. - AlarmWithAgreeingVerdict { - /// The alarm line found in the prose. - alarm: String, - /// Where the agreeing verdict was found: `prose` or `ndjson`. - verdict_source: &'static str, - }, - /// One fact rendered twice, with the two renderings disagreeing. - /// - /// A report carries its findings for a human in prose and for a mining pass - /// in NDJSON. A consumer that reconciles the two cannot, and neither - /// rendering is self-evidently the wrong one. - ProseAndNdjsonDisagree { - /// What the fact is called, for the reader of the failure. - fact: &'static str, - /// As the prose rendered it. - prose: String, - /// As the NDJSON rendered it. - ndjson: String, - }, - /// A hardware claim was stated without its caveat while the report's own - /// evidence says the parse was in doubt. +pub enum RowDefect { + /// The report carries no machine-readable row. /// - /// The renderer's rule is that every hardware conclusion is gated on the - /// parse being whole. A claim printed bare, in a report that elsewhere - /// reports doubt, is that rule with an exception -- and a rule with an - /// exception is not a rule. - UncaveatedClaimUnderDoubt { - /// The claim that was stated bare. - claim: &'static str, - /// The report's own visible evidence of doubt. - evidence: String, - }, - /// The banner names one machine and the body describes another. + /// Every report has one, including the unmeasured shape -- that is what lets + /// a fleet survey tell a host where discovery FAILED from a job that never + /// ran the probe. A report without one silently excludes exactly the hosts + /// most worth counting. + Missing, + /// The report carries more than one. /// - /// The banner is the line a reader uses to decide whether two runs are - /// comparable at all, so a banner describing a different machine from the - /// body under it invalidates every comparison drawn from the report -- - /// while each half stays locally correct, which is what let this survive - /// review. - BannerDisagreesWithBody { - /// The processor count the banner named. - banner: String, - /// The processor count the body reported. - body: String, + /// A mining pass reads the first line that looks like a row, so a second one + /// is not extra data -- it is an ambiguity about which line is the contract. + /// Measured before containment existed: an `io::Error` whose text contained + /// `{` was selected as the row, so a reader checked the caller's text + /// instead of the probe's. + Duplicated { + /// How many lines look like a row. + count: usize, }, - /// The prose states a fact the machine-readable line does not carry at all. - /// - /// **Absence where the report has already made the claim.** This module - /// holds that a report omitting a fact is not a violation -- but that rule - /// is about facts the report never mentions. Once the PROSE states one, the - /// report has made a claim that the other rendering is required to agree - /// with, and a missing counterpart is that requirement going unmet rather - /// than the fact being absent. - /// - /// Every comparison here was written as "both sides present, do they - /// match", so a rendering that DROPPED a field read as silence: measured, - /// deleting `"processors"` from the row, deleting the whole `policies` - /// object, and deleting the partitioning discriminator each left a report - /// the oracle accepted, with the prose still making all three claims. A - /// mining pass reading such a row gets no value and no warning. + /// The row is not a syntactically valid JSON object. /// - /// `report_unmeasured` needs no exemption: it renders neither side of any - /// TOPOLOGY fact, so there is no prose claim for a missing counter, cache or - /// policy to leave unanswered. - /// - /// Not "renders neither side" flatly, which is what this said and is false: - /// the short object publishes `arch`, and the banner can name the same - /// architecture, so that one correspondence IS rendered twice on an - /// unmeasured report -- and is checked there, by - /// `an_architecture_contradiction_survives_an_attribution_disclaimer`. The - /// exemption this paragraph explains is about the facts the short object - /// omits, not about the whole shape. - RenderedOnlyInProse { - /// The fact the prose stated. - fact: &'static str, - /// What the prose said, with nothing to compare it against. - prose: String, + /// **Decided by a real parse, not by a check written here.** The question + /// this answers is "could a consumer read this row", and a consumer uses a + /// JSON parser -- so the only answer that cannot drift from the question is + /// one a JSON parser gives. Two hand-written versions preceded this: the + /// first counted bracket depth, which `{"a":1,}` and `{"a":1]` both satisfy; + /// the second matched delimiters by kind and checked separators, and a + /// generated test still found 159 rows it accepted and `serde_json` did not. + Malformed { + /// What is wrong, in the parser's own words. + what: String, + /// The row, as rendered. + row: String, }, - /// The verdict says every check was made, and a check's evidence is absent. - /// - /// **An `agree` verdict is a claim about what the run DID, not only about - /// what matched.** `CrossCheck` reports `agree` to mean every check this - /// probe could make was made and matched -- so a report that agrees while - /// omitting the line a check reads is contradicting its own verdict, even - /// though the two halves it still renders agree perfectly. + /// The row repeats a key. /// - /// Measured: deleting the `GetActiveProcessorCount` line from an otherwise - /// untouched agreeing report left `check` returning nothing at all. The - /// counter comparison simply skipped, because it was written to compare two - /// present values and to say nothing otherwise -- the same shape as the - /// dropped-counterpart class, in the one place where the VERDICT is what the - /// missing side contradicts. - EvidenceMissingWithAgreeingVerdict { - /// The correspondence whose rendering the report did not carry. - fact: &'static str, + /// A repeated key is not a parse error in every JSON reader -- most take the + /// last -- so this is precisely the kind of malformation that survives a + /// consumer's parse and changes what it reads. + RepeatedKey { + /// The key rendered more than once. + key: String, }, } -/// Every correspondence `report` violates, in the order they were checked. -/// -/// An empty result means every correlation this oracle knows about held. It -/// does **not** mean the report is correct: an oracle is a floor, not a -/// specification. -#[must_use] -pub fn check(report: &str) -> Vec { - let mut found = Vec::new(); - let ndjson = ndjson_line(report); - - check_alarm_against_verdict(report, ndjson, &mut found); - check_prose_against_ndjson(report, ndjson, &mut found); - check_claims_against_doubt(report, ndjson, &mut found); - check_structured_pairs(report, ndjson, &mut found); - check_counters_against_verdict(report, ndjson, &mut found); - check_banner_against_body(report, ndjson, &mut found); - check_diagnostics_against_verdict(report, ndjson, &mut found); - check_partitioning_answer(report, ndjson, &mut found); - - found -} - -/// Each arm of the partitioning discriminator, and the prose that announces it. -/// -/// **Mapped from the renderer, arm by arm, rather than guessed.** A rule written -/// against a subset produces false violations on the arms it guessed wrong, and -/// that failure mode is not hypothetical here: this same module once shipped a -/// banner rule asserting a correspondence `attribution()` explicitly declines to -/// claim, and it had to be narrowed after a review reproduced the false positive. -/// -/// The markers are each arm's OPENING sentence, which is the part that cannot be -/// confused with another arm's. Changing any of these strings is a change to the -/// report's contract with its readers, not a rewording. -const PARTITIONING_ARMS: &[(&str, &str)] = &[ - ( - "outermost cache that partitions the processors it covers: ", - "level", - ), - ("no cache level reported more than one domain", "none"), - ("no cache levels were reported at all", "no_levels_reported"), - ( - "at least one cache level reported more than one distinct domain", - "not_unique", - ), - (SUMMARY_MISSING_MARKER, "summary_missing"), -]; - -/// The partitioning answer the prose gives, against the one the NDJSON publishes. -/// -/// Gap 2 of M2.10, found by a review corrupting a real report and watching the -/// oracle accept it. The oracle already compared -/// `outermost_partitioning_cache_level` -- the NUMBER -- so the level was -/// checked. What went unread was the DISCRIMINATOR, the field saying whether a -/// level was selected at all: changing `"level"` to another value left the -/// oracle silent while the prose still read `outermost cache that partitions the -/// processors it covers: L2 (8 domains)`. -/// -/// The two are opposite answers to this probe's central question -- "a level -/// partitions" against "none does" -- and the renderer is explicit about why the -/// field exists: the level alone said `null` for every absent case alike, so a -/// query counting nulls as "machines no cache level partitions" folded in -/// machines where a level DOES partition. This rule is what stops the two -/// renderings of that answer drifting apart. -fn check_partitioning_answer(report: &str, ndjson: Option<&str>, found: &mut Vec) { - let Some(ndjson) = ndjson else { - return; - }; - let announced: Vec<&str> = PARTITIONING_ARMS - .iter() - .filter(|(marker, _)| has_line_beginning(report, marker)) - .map(|(_, arm)| *arm) - .collect(); - - // **Absent WITH a prose arm is a dropped discriminator, not silence.** - // `report_unmeasured` carries neither side and so announces no arm, which is - // why the exemption needed no special case -- but returning early on the - // field alone also excused a MEASURED report that printed an arm and lost - // its discriminator. Measured: deleting the field from a `level` report left - // the prose conclusion standing with nothing to relate it to, and the oracle - // accepted it. - let Some(published) = ndjson_field(ndjson, "outermost_partitioning_cache") else { - // ANY announced arm, not just a single one. Written for the single-arm - // slice first, which excused the worse report of the two: two prose - // answers AND no discriminator. - if !announced.is_empty() { - found.push(Correspondence::RenderedOnlyInProse { - fact: "outermost partitioning answer", - prose: announced.join(" and "), - }); +impl std::fmt::Display for RowDefect { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + match self { + Self::Missing => f.write_str("the report carries no machine-readable row"), + Self::Duplicated { count } => write!( + f, + "the report carries {count} machine-readable rows, so which one \ + is the contract is ambiguous" + ), + Self::Malformed { what, row } => { + write!(f, "the row is not a valid JSON object -- {what}: {row}") + } + Self::RepeatedKey { key } => write!( + f, + "the row renders `{key}` more than once, which most JSON readers \ + resolve silently by taking the last" + ), } - return; - }; - - match announced.as_slice() { - // The prose names no partitioning answer at all. Not a contradiction -- - // the rule fires only where the report makes the claim twice. - [] => {} - [announced] if *announced == published => {} - [announced] => found.push(Correspondence::ProseAndNdjsonDisagree { - fact: "outermost partitioning answer", - prose: (*announced).to_owned(), - ndjson: published.to_owned(), - }), - // Two arms' prose in one report. The arms are exclusive by construction - // -- they are one `match` -- so this is the prose giving two answers to a - // question that has one, whatever the NDJSON says. - several => found.push(Correspondence::ProseAndNdjsonDisagree { - fact: "outermost partitioning answer", - prose: several.join(" and "), - ndjson: published.to_owned(), - }), } - - // **`summary_missing` names a LEVEL in both renderings, and it was the one - // arm whose number nothing compared.** Found by a review, after the rule - // above had closed the discriminator itself -- which is the pattern this - // module keeps repeating: a rule is added per fact, so the fact added - // alongside it goes unread. - // - // The oracle's other level comparison is keyed to the `Level` arm's prose - // label, `outermost cache that partitions the processors it covers: `, so it - // never fires here. And `summary_missing` is the only non-`Level` arm whose - // NDJSON level is a NUMBER rather than `null`, which is exactly what made - // the omission invisible: the three arms beside it have no number to - // disagree about. - // - // It matters most precisely where it was missing. This arm is the state the - // renderer prints as `BUG IN THIS PROBE ... Nothing below about cache - // partitioning can be trusted` -- a report already telling its reader it is - // unreliable, in which the two renderings of WHICH level went unchecked. - if let Some(prose_level) = summary_missing_level(report) { - compare( - found, - "summary-missing outermost level", - prose_level, - ndjson_field(ndjson, "outermost_partitioning_cache_level"), - ); - } -} - -/// The level the `summary_missing` prose names, if the report carries that arm. -/// -/// Keyed to the same opening sentence [`PARTITIONING_ARMS`] uses, so the two -/// cannot drift apart: if that sentence is reworded, both stop matching together -/// rather than one silently continuing to match a report the other no longer -/// recognises. -fn summary_missing_level(report: &str) -> Option<&str> { - leading_digits(after_marker_leading_a_line(report, SUMMARY_MISSING_MARKER)?) } -/// The text following `marker` on a line the RENDERER leads with it. -/// -/// **A marker search over the whole report reads the caller's text as the -/// probe's.** The banner is contained into the first line rather than dropped, -/// so its content survives -- and an unanchored `find` then picks a marker out -/// of it wherever it lands. Measured before this: a banner ending -/// `... named L99 as the outermost` produced a `summary-missing outermost level` -/// of 99 against the row's 1, and one ending -/// `windows-topology-sys recorded 99 enumeration anomalies` produced an anomaly -/// count of 99 against the row's 0. Both are the oracle raising a violation -/// about a report that does not contain the defect -- a false alarm invented out -/// of caller text, which is the failure mode that costs a reader the most. -/// -/// The rest of the module reads by line start, and these two were what was left -/// of the older style. The bullet is part of the renderer's shape, not a -/// concession: `CrossCheck` writes its diagnostic entries as ` - `, -/// so the marker genuinely never begins its line. Accepting the bullet cannot -/// re-open the hole it closes, because a contained banner is one line beginning -/// `host:` and an attribution-shaped one is `host:` lines and a disclaimer -- -/// neither can present a line whose first content is `- `. -fn after_marker_leading_a_line<'a>(report: &'a str, marker: &str) -> Option<&'a str> { - report +/// Every way `report`'s machine-readable row is not well-formed. +#[must_use] +pub fn check(report: &str) -> Vec { + let rows: Vec<&str> = report .lines() - .find_map(|line| strip_entry_tag(line.trim_start()).strip_prefix(marker)) -} - -/// A diagnostic entry's leading tag, removed. -/// -/// **`CrossCheck`'s entries carry a tag whose spelling depends on the verdict, -/// and reading only one of them left the rule unread on the others.** The -/// `INCOMPLETE` arm writes ` - {caveat}`; the `DISAGREE` arm writes -/// ` (parse incomplete) {caveat}` and ` (not compared) {skipped}` so a -/// reader can tell the disagreement from what was merely not established. -/// Anchoring to the bullet alone therefore went blind to the anomaly count on -/// exactly the verdict where a reader most needs it. Measured: the -/// `(parse incomplete) ` rendering returned `None` where the `- ` rendering -/// returned `Some("2")`, so corrupting the count on a disagreeing report raised -/// no violation at all. -/// -/// Matches the SHAPE of a tag rather than restating the renderer's two literal -/// strings, which would be a second copy to drift. Containment makes that safe: -/// a contained banner is one line beginning `host:` and an attribution-shaped -/// one is `host:` lines and a disclaimer, so neither can present a line whose -/// first content is a bullet or a parenthesised tag. -fn strip_entry_tag(line: &str) -> &str { - if let Some(rest) = line.strip_prefix("- ") { - return rest; - } - - line.strip_prefix('(') - .and_then(|rest| rest.split_once(") ")) - .map_or(line, |(_tag, rest)| rest) -} - -/// The run of ASCII digits `text` opens with, if it opens with one. -fn leading_digits(text: &str) -> Option<&str> { - let end = text - .find(|character: char| !character.is_ascii_digit()) - .unwrap_or(text.len()); - - (end > 0).then(|| &text[..end]) -} - -/// The opening sentence of the `summary_missing` arm, up to the level it names. -const SUMMARY_MISSING_MARKER: &str = "BUG IN THIS PROBE: the topology crate named L"; + .filter(|line| line.starts_with('{')) + .collect(); -/// The diagnostic counts, against the verdict drawn beside them and against the -/// prose that lists the same entries. -/// -/// `not_compared`, `parse_incomplete` and `enumeration_anomalies` are published -/// in the NDJSON and their entries are listed in the prose, and until now -/// nothing compared any of the three. `parse_incomplete` was read only as a -/// nonzero predicate for the heterogeneity caveat, which is a different question -/// from whether the two renderings agree. -/// -/// **The verdict rule binds to a SPECIFIED contract, not to current behaviour.** -/// [`crate::topology_report`] states it where the NDJSON is emitted: "anomalies -/// populate `parse_incomplete`, and a non-empty `parse_incomplete` forces the -/// verdict away from `agree`, so `cross_check == "agree"` IMPLIES no record -/// failed to decode", and the crate pins it with -/// `a_dropped_enumeration_record_blocks_agreement_even_when_every_counter_matches` -/// rather than leaving it a promise in a comment. So an `agree` verdict beside a -/// nonzero count of either is the report contradicting its own published rule -- -/// the same shape as the alarm-beside-an-agreeing-verdict defect this module was -/// built for, in the field a mining pass trusts most. -/// -/// The implication is stated ONE WAY and is read that way here: a run whose -/// counter failed to read has a complete parse and still reports `incomplete`, -/// so a nonzero `not_compared` is NOT asserted to force the verdict, and no rule -/// below claims it does. -fn check_diagnostics_against_verdict( - report: &str, - ndjson: Option<&str>, - found: &mut Vec, -) { - let Some(ndjson) = ndjson else { - return; + let [row] = rows.as_slice() else { + return vec![if rows.is_empty() { + RowDefect::Missing + } else { + RowDefect::Duplicated { count: rows.len() } + }]; }; - if ndjson_field(ndjson, "cross_check") == Some("agree") { - // `not_compared` belongs here too, and was missing. `CrossCheck`'s - // verdict makes `agree` imply that nothing was skipped as well as that - // nothing failed to decode, so an agreeing report publishing skipped - // work contradicts its own rule exactly as a nonzero `parse_incomplete` - // does. Measured, before this: `"cross_check":"agree"` beside - // `"not_compared":3` was accepted with no violation. Found by a review. - for key in ["parse_incomplete", "enumeration_anomalies", "not_compared"] { - if let Some(count) = ndjson_field(ndjson, key) - && count != "0" - { - found.push(Correspondence::AlarmWithAgreeingVerdict { - alarm: format!("\"{key}\":{count}"), - verdict_source: "ndjson", - }); - } - } - } + let mut found = Vec::new(); - // **The anomaly COUNT is inside the prose sentence, not inferable from the - // line count.** `CrossCheck` emits one `parse_incomplete` entry reading - // `windows-topology-sys recorded N enumeration anomal...` however many there - // were, so counting lines can never check N -- a report can say it recorded - // 2 while publishing 99 and agree about every total. Found by a review, - // which also caught that the checklist claimed all three diagnostic counts - // were compared "against the prose listings" when this one was only ever - // checked as a nonzero predicate under an `agree` verdict. - // - // Read for EVERY verdict, deliberately. The `agree` rule above is about a - // contradiction with the verdict; this is about the two renderings of one - // number, which must agree whatever the verdict says. - if let Some(prose_anomalies) = anomaly_count_in_prose(report) { - compare( - found, - "enumeration anomaly count", - prose_anomalies, - ndjson_field(ndjson, "enumeration_anomalies"), - ); + if let Some(what) = malformation(row) { + found.push(RowDefect::Malformed { + what, + row: (*row).to_owned(), + }); + // Every check below reads the object's members, which is not a question + // that means anything about text that is not an object. + return found; } - // The prose lists these entries one per line, and how it marks them depends - // on the verdict: the DISAGREE arm labels each kind, so the two are counted - // separately, while the INCOMPLETE arm renders both as a bare `- `, which - // makes only their total recoverable. Counting what the prose can actually - // distinguish, rather than a number it does not render, is the whole habit - // this module is built on. - if has_line_beginning(report, "=> DISAGREE") { - // **Zero lines is not a prose claim.** These compare a COUNT OF LINES - // against a field, and the renderer emits no line when the count is - // zero -- so routing them through `compare`'s absence path reported a - // dropped counterpart for a prose that had said nothing, on a fixture - // that was previously accepted. Absence matters only where the prose - // actually listed entries. - for (label, key, fact) in [ - (" (not compared) ", "not_compared", "not compared count"), - ( - " (parse incomplete) ", - "parse_incomplete", - "parse incomplete count", - ), - ] { - let listed = prose_lines_beginning(report, label); - match ndjson_field(ndjson, key) { - Some(json) => compare_counts(found, fact, listed, json.parse().unwrap_or_default()), - None if listed > 0 => found.push(Correspondence::RenderedOnlyInProse { - fact, - prose: listed.to_string(), - }), - None => {} - } + let mut seen: Vec = Vec::new(); + for key in keys(row) { + if seen.contains(&key) { + found.push(RowDefect::RepeatedKey { key }); + } else { + seen.push(key); } } - if has_line_beginning(report, "=> INCOMPLETE") { - let listed = prose_lines_beginning(report, " - "); - let (Some(skipped), Some(caveats)) = ( - ndjson_count(ndjson, "not_compared"), - ndjson_count(ndjson, "parse_incomplete"), - ) else { - // **The prose has already listed the entries here.** This arm sums - // two fields, so it was written to return unless BOTH are present -- - // and that made a row which dropped either one silent on exactly the - // verdict whose reason those counts carry. Found by the deletion - // sweep, on a corpus shape rather than on this host: the counts are - // zero here, so the entries are absent and there is nothing to drop. - if listed > 0 { - found.push(Correspondence::RenderedOnlyInProse { - fact: "incomplete-verdict listing count", - prose: listed.to_string(), - }); - } - return; - }; - compare_counts( - found, - "incomplete-verdict listing count", - listed, - skipped + caveats, - ); - } + found } -/// The anomaly count named inside the diagnostic sentence, if it is present. -/// -/// Keyed to the sentence `CrossCheck` writes, and reading the number that -/// follows it. The count is rendered INSIDE one entry rather than as one entry -/// per anomaly, so nothing about the number is recoverable from counting lines. -fn anomaly_count_in_prose(report: &str) -> Option<&str> { - const MARKER: &str = "windows-topology-sys recorded "; - leading_digits(after_marker_leading_a_line(report, MARKER)?) -} +/// What is wrong with `row` as a JSON object, if anything. +/// +/// **A real parse, because the question is whether a consumer can parse it.** +/// Anything else here is a second opinion about what JSON is, and a second +/// opinion is a thing that can disagree. Both hand-written predecessors did: +/// the first counted bracket depth and accepted `{"a":1,}`; the second matched +/// delimiters by kind and checked separators, and a test that generated 1807 +/// single-character corruptions of a real row found **159 it accepted and +/// `serde_json` rejected -- every one a false accept.** Closing the last of them +/// required tracking whether an object expects a name or a value next, which is +/// a JSON parser; so this depends on one rather than growing one. +/// +/// **A parse does not subsume [`RowDefect::RepeatedKey`].** `serde_json` accepts +/// a duplicated key and silently keeps the last, which is exactly why that +/// defect is worth a check of its own: it survives the consumer's parse and +/// changes what the consumer reads. The two checks answer different questions +/// and neither replaces the other. +fn malformation(row: &str) -> Option { + // The row is required to be an OBJECT, not merely valid JSON. A bare `[1,2]` + // parses and would satisfy a laxer check, while carrying no keys at all. + match serde_json::from_str::>(row) { + Ok(_) => None, + Err(error) => Some(error.to_string()), + } +} + +/// Every key `row` renders at its top level, in the order it renders them, and +/// INCLUDING repeats. +/// +/// **Read from the parser's own tokens, not by walking the bytes.** Both +/// properties this returns are ones a parsed map destroys: `serde_json::Map` +/// sorts its names, and silently keeps the last of a repeated key -- which is +/// exactly the defect [`RowDefect::RepeatedKey`] reports, so parsing into a map +/// would delete the evidence. A `MapAccess` visitor sees each name as the parser +/// reads it, which keeps both while leaving every escape, quote and delimiter +/// decision to `serde_json`. +/// +/// The hand-written version of this was the last string scanner here, and it had +/// already produced a real defect: it used `find('"')`, which takes `\"` for a +/// terminator, so a `discovery_error` carrying an escaped quote shifted where it +/// thought strings began and text INSIDE the error was emitted as top-level +/// keys. Measured: an `io::Error` of `q":1,"q":1,"q` rendered a row that +/// `JSON.parse` accepts with four keys, and `assert_row_is_well_formed` panicked from +/// inside the renderer. Fixing that added escape-awareness to one of the +/// scanners and left the others to be argued about; this removes the question. +/// +/// Top level only, deliberately: a nested object's members are that object's +/// keys, and repeating one there is a different question from repeating one in +/// the row. The visitor reads nested values as [`serde::de::IgnoredAny`], which +/// consumes them without collecting their names. +#[must_use] +pub fn keys(row: &str) -> Vec { + struct TopLevelNames; -/// How many lines of `report` begin with `prefix`. -fn prose_lines_beginning(report: &str, prefix: &str) -> usize { - report - .lines() - .filter(|line| line.starts_with(prefix)) - .count() -} + impl<'de> serde::de::Visitor<'de> for TopLevelNames { + type Value = Vec; -/// An NDJSON field read as a count, or `None` when it renders no number. -fn ndjson_count(ndjson: &str, key: &str) -> Option { - ndjson_field(ndjson, key)?.parse().ok() -} + fn expecting(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.write_str("the probe's machine-readable row, a JSON object") + } -/// Push a disagreement between two counts of the same thing. -fn compare_counts(found: &mut Vec, fact: &'static str, prose: usize, json: usize) { - if prose != json { - found.push(Correspondence::ProseAndNdjsonDisagree { - fact, - prose: prose.to_string(), - ndjson: json.to_string(), - }); + fn visit_map>( + self, + mut members: A, + ) -> Result { + let mut names = Vec::new(); + while let Some(name) = members.next_key::()? { + names.push(name); + members.next_value::()?; + } + Ok(names) + } } -} -/// The banner names the machine the body describes. -/// -/// A run makes three discoveries of the host -- one before the measurement, -/// `measure`'s own, and one after -- and the banner used to be built from an -/// endpoint, so it could name a different topology from the body beneath it -/// with nothing in the report saying so. -/// -/// **The rule applies only where the report claims the correspondence**, and -/// that qualifier is load-bearing here. `attribution` takes the two readings -/// that bracket the measurement and renders the first as the banner; the body -/// comes from a `measure()` between them. When the two endpoints differ, or -/// either fails, the report says so in as many words -- "which of them names the -/// machine the body below describes was not established" -- and on such a report -/// the first banner line need not describe the body at all. Asserting a -/// contradiction there would be the oracle over-claiming exactly where the -/// renderer went to trouble not to, so it returns instead. -/// -/// **On the origin branch this rule was unconditional, and that was correct -/// there**: `measure_observed` builds the banner from the body's own topology, -/// which makes the mismatch unrepresentable. That construction change is a -/// separate peel, so the ambiguity is live on this branch and the rule has to -/// respect it. When the construction lands, the guard below stops being reached -/// rather than becoming wrong. -/// -/// What the rule still buys where attribution IS determinate: the banner and the -/// body remain two independent *derivations* from one topology -- -/// `Fingerprint::from_topology` and `observe`, each with its own filter for -/// which processors count. Those have already disagreed once, when -/// `from_topology` summed core-domain membership and printed `0p` for a machine -/// about to be measured on four processors. -fn check_banner_against_body(report: &str, ndjson: Option<&str>, found: &mut Vec) { - let Some(ndjson) = ndjson else { - return; - }; - - // The architecture is checked FIRST and outside the attribution exemption, - // for the reason given on the function below. - check_architecture_against_body(report, ndjson, found); - // The report's own statement that it cannot attribute the body to either - // reading. Matched on the rendered disclaimers rather than on the count of - // `host:` lines, because the count is incidental to how `attribution` - // happens to render today and these sentences are the contract. + // **`end` matters, and its absence made two public functions disagree.** + // `serde_json` stops at the end of the first value and does not care what + // follows, so `{"reason":"x"}garbage` yielded `["reason"]` here while + // `check` reported `Malformed { what: "trailing characters ..." }` for the + // same row. A caller reading keys directly was told a malformed artifact + // was readable. Measured, then fixed; reported by a review. // - // It exempts the PROCESSOR COUNT specifically, which is what the disclaimer - // is about: the two readings name different topologies and this run cannot - // say which describes the body. - if has_line_beginning(report, "HOST READINGS DISAGREE:") - || has_line_beginning(report, "HOST NOT ESTABLISHED:") - { - return; - } - - let Some(banner) = report.lines().find(|line| line.starts_with("host:")) else { - return; + // The block below is one sabotage anchor and is deliberately free of + // comments, so replacing the visitor with a parsed map stays a single + // contiguous substitution that still compiles. + let mut reader = serde_json::Deserializer::from_str(row); + let Ok(names) = serde::Deserializer::deserialize_map(&mut reader, TopLevelNames) else { + return Vec::new(); }; - - // Absent on a report whose discovery failed: the banner reads `UNKNOWN` and - // `report_unmeasured` emits no processor count, so there is nothing to - // relate and no violation to claim. - let (Some(banner_count), Some(body_count)) = ( - processors_in_banner(banner), - ndjson_field(ndjson, "processors"), - ) else { - return; - }; - - if banner_count != body_count { - found.push(Correspondence::BannerDisagreesWithBody { - banner: banner_count.to_owned(), - body: body_count.to_owned(), - }); + if reader.end().is_err() { + return Vec::new(); } + names } - -/// The architecture every banner names, against the one the body publishes. -/// -/// The architecture is rendered twice -- in the banner and as the NDJSON's -/// `arch` -- and went unread until a review corrupted one and watched the oracle -/// accept it. Both come from `std::env::consts::ARCH` today, so they cannot -/// currently differ; that is a fact about the renderer rather than a contract, -/// and exactly the kind of coincidence this oracle is built not to lean on. -/// -/// **Outside the attribution exemption, and that is the point of separating it.** -/// The disclaimers say which of the two bracket READINGS describes the body was -/// not established. That is a statement about the machine's topology, not about -/// its instruction set: when every banner the report managed to parse names the -/// same architecture, then whichever reading describes the body, the -/// architecture is that one -- so a body naming a different one is a -/// contradiction the disclaimer does not excuse. Found by a review; before this, -/// two `x86_64` banners under a disclaimer beside an `aarch64` body produced no -/// violation at all. -/// -/// **When the banners disagree with EACH OTHER about the architecture, this -/// returns**, because then the question really is unestablished and asserting -/// anything would be the over-claim the exemption exists to prevent. -/// -/// Checked before the processor-count guard as well, so it still reaches -/// `report_unmeasured`, which emits `arch` but no `processors`. -fn check_architecture_against_body(report: &str, ndjson: &str, found: &mut Vec) { - let announced: Vec<&str> = report - .lines() - .filter(|line| line.starts_with("host:")) - .filter_map(architecture_in_banner) - .collect(); - - let Some(first) = announced.first() else { - return; - }; - if announced.iter().any(|architecture| architecture != first) { - return; - } - - // Past the two returns above, the banner HAS named an architecture and every - // reading agrees on it -- so a row without `arch` is the report stating the - // fact once, not a report that never stated it. - compare( - found, - "architecture", - first, - ndjson_field(ndjson, "arch").map(|body| body.trim_matches('"')), - ); -} -/// The architecture the banner names. -/// -/// The fingerprint renders as ` p/c ...`, optionally behind one or -/// more `!!MARKER!!` prefixes, so the architecture is the first token that is -/// not one of those. -/// -/// **Every marker is skipped, not a named list of them.** The first version of -/// this stripped `!!taint!!` alone and read `!!assumed!!` as an architecture, -/// reporting a contradiction against a perfectly good banner -- caught by an -/// existing acceptance test, which is what those are for. -/// -/// A first attempt at justifying that generality said "the renderers emit at -/// least `!!SYNTHETIC!!`, `!!UNOFFICIAL!!`, `!!RESTORED!!`, `!!assumed!!` and -/// `!!taint!!`", which is an overclaim of the kind this module exists to catch, -/// written while fixing another one. Counted: a `host:` line is rendered by -/// `banner_line`/`banner_line_for` from a `Fingerprint`, which emits -/// `!!{provenance}!!` only when the provenance is not `Measured` -- so the only -/// markers this function can meet today are **`!!SYNTHETIC!!` and -/// `!!RESTORED!!`**. `!!UNOFFICIAL!!` belongs to `BuildIdentity`, which never -/// reaches this line, and `!!assumed!!` and `!!taint!!` are not emitted anywhere -/// -- they exist only as invented fixtures in this module's own tests. -/// -/// The generality is still right, and on a better argument than a miscounted -/// list: the marker is a *shape* the banner reserves for provenance, and this -/// module observes the renderer rather than mirroring it. Matching the shape -/// cannot fall out of step; enumerating today's two spellings would. -/// -/// **Only a FINGERPRINT names an architecture, so the count shape is required -/// here.** A failed read names no architecture, and a caller may pass any string -/// as a banner. Without that guard the first token of such a line is read as an -/// architecture and contradicts the NDJSON every time: measured, `UNKNOWN` -/// against a real `"arch"` produced a false violation, and the crate's own -/// `host: TEST-FIXTURE` fixture produced fourteen more. What the failed-read -/// line actually looks like -- and why the guard has to read by position rather -/// than by substring -- is on `fingerprint_tokens`, which owns that decision. -/// (Named without a link: it is private, and a public doc may not link to it.) -/// -/// This paragraph used to describe that line as "the bare word `UNKNOWN`", which -/// is not what `banner_line_for` writes. The correction was made on -/// `fingerprint_tokens` and not here, so the two docs contradicted each other -/// in the same module -- a fix applied to one statement of a fact while another -/// statement of it survived, which is the drift this crate keeps paying for. -/// -/// Note carefully which side this constrains. Requiring the BANNER to carry -/// `p/c` is what establishes it is a fingerprint; requiring the BODY to -/// carry a processor count is the coupling that wrongly confined this rule to -/// measured reports. The first is the renderer's contract, the second was an -/// accident of where the code sat. -/// **Public because the question has to have ONE answer.** A test helper asked -/// the same thing -- "does this `host:` line name an architecture?" -- with its -/// own cheaper rule, `line.contains("p/")`. That agreed with this module until -/// this module started reading the tokens by position, and then it did not: -/// measured, a failed-discovery banner of -/// `host: UNKNOWN -- topology discovery failed: 16p/foo something opaque` -/// satisfied the helper and not the oracle, so the fact accounting demanded that -/// `arch` be read on a report where the oracle is right to say nothing, and a -/// perfectly valid unmeasured report failed the suite. -/// -/// A second implementation of a predicate is not a check of it; it is a copy -/// that agrees until it does not. Consumers ask here instead. -pub fn architecture_in_banner(banner: &str) -> Option<&str> { - fingerprint_tokens(banner).map(|(architecture, _)| architecture) -} - -/// The `` and `p/c` tokens a banner names, if it names a fingerprint. +/// The `code` of every entry in `row`'s list-valued `key`. /// -/// **Read BY POSITION, because a banner that is not a fingerprint can still -/// contain the substrings one would have.** Both readers used to search the -/// whole line -- the count as "digits before the first `p/`", the architecture -/// as "first token that is not a taint marker and does not contain `p/`" -- and -/// a banner only has to mention `p/` somewhere for that to find a fingerprint -/// in text that is not one. +/// **One definition, because two instruments need it.** The diagnostic lists +/// hold objects -- `{"code":"contradictory_cores","count":3}` -- and both the +/// unit tests and the publication accounting ask this question. A second +/// implementation of it is the kind of copy that agrees until it does not. /// -/// That is not hypothetical, and the doc this replaces had the renderer's own -/// shape wrong: `banner_line_for` does NOT render a failed read as the bare -/// word `UNKNOWN`. It renders -/// `host: UNKNOWN -- topology discovery failed: {error}`, with the `io::Error` -/// verbatim. Measured: an error text of `16p/foo something opaque` made the -/// count reader answer `16`, which satisfied the guard, so the architecture -/// reader then answered `UNKNOWN` and the bound assertion PANICKED on a -/// perfectly valid unmeasured report -- a probe crashing on the host whose -/// discovery failed, which is the host it exists to report. +/// Read from a parse. The previous version searched for `"code":"` and then took +/// the next `"` as the end, which is not escape-aware: a code containing a quote +/// would have truncated. That was argued safe because every code is a +/// `&'static str` from an enum and no caller text reaches a list -- an argument +/// that was true, load-bearing, and enforced by nothing. Parsing makes the +/// argument unnecessary rather than merely correct, which is the difference +/// between a property and a hope. /// -/// The fingerprint renders ` p/c smt` behind any number of -/// `!!taint!!` markers, so the two tokens are taken from their positions and -/// the second is required to have the count SHAPE. `UNKNOWN -- ...` fails that -/// on its second token and is silent, which is the right answer: a failed read -/// names no architecture, so there is nothing to compare. -/// -/// Found together rather than separately because both readers need exactly the -/// same decision -- "is this a fingerprint, and where are its parts" -- and two -/// copies of that decision are what let them disagree about it before. -fn fingerprint_tokens(banner: &str) -> Option<(&str, &str)> { - let mut tokens = banner - .strip_prefix("host:")? - .split_whitespace() - .skip_while(|token| token.starts_with("!!") && token.ends_with("!!")); - - let architecture = tokens.next()?; - let counts = tokens.next()?; - - is_count_shaped(counts).then_some((architecture, counts)) -} - -/// Whether `token` is the `p/c` the fingerprint writes. -fn is_count_shaped(token: &str) -> bool { - let Some((processors, rest)) = token.split_once("p/") else { - return false; +/// Returns empty for a key that is absent or not a list, which is the same +/// answer as an empty list on purpose: a consumer of this is asking "what +/// conditions are published", and "none" is the answer in both cases. +#[must_use] +pub fn list_codes(row: &str, key: &str) -> Vec { + let Ok(parsed) = serde_json::from_str::(row) else { + return Vec::new(); }; - let Some(cores) = rest.strip_suffix('c') else { - return false; + let Some(entries) = parsed.get(key).and_then(serde_json::Value::as_array) else { + return Vec::new(); }; - [processors, cores] + entries .iter() - .all(|part| !part.is_empty() && part.bytes().all(|byte| byte.is_ascii_digit())) -} - -/// The processor count a banner line names, as it was rendered. -/// -/// Returned as text rather than parsed, so a malformed count is reported as the -/// mismatch it is instead of being silently discarded by a failed parse. -fn processors_in_banner(banner: &str) -> Option<&str> { - let (_, counts) = fingerprint_tokens(banner)?; - - counts.split_once("p/").map(|(processors, _)| processors) -} - -/// [`check`], as an assertion, for tests that render a report. -/// -/// # Panics -/// -/// Panics listing every correspondence the report violated. -pub fn assert_corresponds(report: &str) { - let violations = check(report); - assert!( - violations.is_empty(), - "the report's parts contradict each other: {violations:#?}\n\n\ - --- the report ---\n{report}" - ); + .filter_map(|entry| entry.get("code")) + .filter_map(serde_json::Value::as_str) + .map(str::to_owned) + .collect() } - -/// The report's machine-readable line, if it has one. +/// The report's machine-readable row, if it carries exactly one well-formed one. /// -/// A report is prose with at most one NDJSON line in it. `report_unmeasured` -/// emits a much shorter object than `report`, so every field read below is -/// optional by construction. -fn ndjson_line(report: &str) -> Option<&str> { - report.lines().find(|line| line.starts_with('{')) -} - -/// The raw text of one field of a flat JSON object. +/// Public because the instruments in `tests/` read the row to ask what it +/// publishes, and a second implementation of "which line is the row" is the kind +/// of copy that agrees until it does not. /// -/// Hand-written rather than pulled from a JSON crate because this crate has no -/// such dependency and the object is emitted a few lines away in this same -/// crate: a top-level object of machine-generated fields, whose only nesting is -/// the `caches` array of objects and the `policies` object, both of which the -/// balanced scan below handles. An earlier version of this sentence called the -/// object "flat, unnested", which would lead a future change to assume nested -/// values are unsupported when they are read here every run. It returns the value's source -/// text -- quotes stripped for a string, otherwise verbatim -- so a caller -/// compares renderings rather than parsed values, which is the point. -fn ndjson_field<'a>(line: &'a str, key: &str) -> Option<&'a str> { - let needle = format!("\"{key}\":"); - let start = line.find(&needle)? + needle.len(); - let rest = &line[start..]; - - let value = if let Some(stripped) = rest.strip_prefix('"') { - let end = stripped.find('"')?; - &stripped[..end] - } else if rest.starts_with('[') || rest.starts_with('{') { - // Balanced, not first-closer. `caches` is an array OF objects and - // `policies` is an object, so stopping at the first `]` or `}` would - // truncate both -- returning `[{"level":1,"domains":8` for a three-level - // machine, which then compares unequal against anything and reports a - // contradiction that is the reader's own parse. - let end = balanced_end(rest)?; - &rest[1..end] - } else { - let end = rest.find([',', '}']).unwrap_or(rest.len()); - &rest[..end] - }; - - Some(value.trim()) -} - -/// One field of a flat JSON object, with its delimiters left on. +/// **Defined by [`check`], so the accessor and the oracle cannot disagree.** It +/// ran its own subset -- one row, and `malformation` -- and `serde_json` accepts +/// a duplicated key, so a row that `check` reported as +/// [`RowDefect::RepeatedKey`] was handed back here as well-formed. A caller +/// asking "may I read this row" got yes for a row the crate had already judged +/// ambiguous, which is the one malformation that survives a consumer's parse and +/// changes what it reads. Found by a review. /// -/// [`ndjson_field`] strips the quotes from a string and the brackets from an -/// array or object, which is what most callers want -- they are comparing -/// contents. A caller that cares whether the value IS an array needs the -/// delimiter, because a scalar and a one-element list have the same contents. -fn ndjson_raw_field<'a>(line: &'a str, key: &str) -> Option<&'a str> { - let needle = format!("\"{key}\":"); - let start = line.find(&needle)? + needle.len(); - let rest = &line[start..]; - - let end = if rest.starts_with('[') || rest.starts_with('{') { - balanced_end(rest)? + 1 - } else if let Some(after_quote) = rest.strip_prefix('"') { - after_quote.find('"')? + 2 - } else { - rest.find([',', '}']).unwrap_or(rest.len()) - }; - - Some(rest[..end].trim()) -} - -/// The index of the bracket closing the one `text` opens with. -fn balanced_end(text: &str) -> Option { - let mut depth = 0_i32; - for (index, character) in text.char_indices() { - match character { - '[' | '{' => depth += 1, - ']' | '}' => { - depth -= 1; - if depth == 0 { - return Some(index); - } - } - _ => {} - } +/// This is the same defect the module keeps warning about, in the function whose +/// doc comment warns about it: two implementations of one question, agreeing +/// until they did not. +#[must_use] +pub fn row(report: &str) -> Option<&str> { + if !check(report).is_empty() { + return None; } - None + report.lines().find(|line| line.starts_with('{')) } -/// Whether some line of the report BEGINS with this renderer token. -/// -/// **Anchored, because a report carries text the renderer does not own.** -/// `report_unmeasured` embeds the caller's `io::Error`, so an unanchored -/// substring search reads that error as if it were the probe speaking. Measured, -/// before this: an error reading `BUG IN THIS PROBE => agree` made a perfectly -/// valid unmeasured report trip the alarm rule and panic in the renderer's own -/// binding -- the oracle inventing a contradiction out of a message it should -/// have treated as opaque. Found by a review. -/// -/// **The renderer now contains that text, and this anchoring is still what -/// stops it being read.** `renderer_owns_every_line` flattens the error, so it -/// can no longer introduce a LINE -- but its words still sit inside the -/// discovery-failure line, and an unanchored search finds them there just the -/// same. The two fixes answer different halves: containment stops caller text -/// impersonating a line, anchoring stops it being read as one. This paragraph -/// said "prints verbatim", which described the state before containment and -/// made the anchoring look redundant. -/// -/// Leading whitespace is trimmed rather than matched, because the renderer -/// indents some of these lines and not others, and the indentation is -/// presentation rather than contract. What matters is that the token begins a -/// line: every token passed here is one the renderer writes at the start of a -/// line it owns, and the caller's error is embedded mid-line after -/// `MachineMemoryTopology::discover failed: `. -/// -/// **The residue, stated rather than implied, and narrower than it was.** An -/// error containing a newline followed by one of these tokens would once have -/// put that token at the start of its own line, where anchoring cannot help. The -/// renderer closed that: `report_unmeasured` passes the error through -/// `renderer_owns_every_line`, so no error can create a line any more. Measured: -/// an error of `first\nBUG IN THIS PROBE: second` produces no line beginning -/// with the alarm, and the report is accepted. -/// -/// What remains is only for a caller that hands `check` text the renderer never -/// produced -- a test constructing a report by hand, say. Bounding even that -/// needs the renderer to tell the oracle which region is opaque, which is a -/// change to the report format rather than to this reader. -fn has_line_beginning(report: &str, token: &str) -> bool { - report - .lines() - .any(|line| line.trim_start().starts_with(token)) -} +use crate::row::Shape; -/// The claim's line and the indented continuation beneath it. -/// -/// A gated claim and the caveat that excuses it are one rendered block: the -/// renderer writes the claim, then the caveat as an indented continuation, then -/// a blank line. Ending at that blank line is what keeps another block's caveat -/// from answering for this one. +/// Every way `row` departs from `schema`'s value SHAPES, as sentences. /// -/// **This also subsumes the caller-text problem, which is why the wider -/// `renderer_prose` filter it replaced is gone.** The caveat is matched -/// mid-line, so it is the one search here that cannot anchor to a line start -- -/// and that is the SUPPRESSING direction: a caveat found where none was written -/// removes a violation. Measured, before any of this: with the caveat sentence -/// appended to the banner, `UncaveatedClaimUnderDoubt` disappeared from a report -/// that still carried the claim and still said `parse_incomplete=2`. -/// -/// A block cannot start in the banner, because the claim line is found by its -/// own opening text and containment guarantees a caller's banner is one line -/// beginning `host:`. So scoping to the block excludes caller text for a -/// structural reason rather than by listing the places caller text can appear, -/// which is what the previous filter had to do -- and got wrong once, by -/// anchoring to a title that `clean_report()` does not use. -/// `a_caveat_in_the_banner_does_not_excuse_an_uncaveated_claim` still pins it. -fn claim_block<'a>(report: &'a str, claim: &str) -> impl Iterator { - report - .lines() - .skip_while(move |line| !line.trim_start().starts_with(claim)) - .take_while(|line| !line.trim().is_empty()) -} - -/// The text after `label` on the line that begins with it. -fn prose_field<'a>(report: &'a str, label: &str) -> Option<&'a str> { - report - .lines() - .find(|line| line.starts_with(label)) - .map(|line| line[label.len()..].trim()) -} - -/// Alarms the prose can raise. Each is a statement that part of the report is -/// not to be trusted. -const ALARMS: &[&str] = &["BUG IN THIS PROBE"]; - -fn check_alarm_against_verdict( - report: &str, - ndjson: Option<&str>, - found: &mut Vec, -) { - let Some(alarm) = report.lines().find(|line| { - ALARMS - .iter() - .any(|marker| line.trim_start().starts_with(marker)) - }) else { - return; - }; - - if has_line_beginning(report, "=> agree") { - found.push(Correspondence::AlarmWithAgreeingVerdict { - alarm: alarm.trim().to_owned(), - verdict_source: "prose", - }); - } - - if ndjson.and_then(|line| ndjson_field(line, "cross_check")) == Some("agree") { - found.push(Correspondence::AlarmWithAgreeingVerdict { - alarm: alarm.trim().to_owned(), - verdict_source: "ndjson", - }); - } -} - -/// Facts this report renders twice: the prose label, the NDJSON key, and the -/// name to use when they disagree. +/// **The half the key contract was missing.** `MEASURED_ROW_KEYS` pins which +/// names appear and in what order, and says nothing about what they hold -- so +/// a renderer could publish `"processors":"16"` and satisfy the key test, the +/// well-formedness check and every renderer assertion at once. Measured before +/// this existed: rendering that one field through `.to_string()` left all 230 +/// library tests and all 10 real-host integration tests green. Reported by a +/// review. /// -/// Counts only. A prose line reads `processors (online) : 16` and the NDJSON -/// `"processors":16`, so the comparison is of the rendered values with the -/// prose label removed. -const DOUBLE_RENDERED: &[(&str, &str, &str)] = &[ - ("processors (online) : ", "processors", "online processors"), - ("processor groups : ", "groups", "processor groups"), - ("packages : ", "packages", "packages"), - ("physical cores : ", "cores", "physical cores"), -]; +/// Keys are not re-checked here; that is the key test's job, and doing it in +/// both places would make one of them the copy. A key the schema names and the +/// row lacks is reported, because a shape cannot be checked against nothing. +#[must_use] +pub fn shape_violations(row: &str, schema: &[(&str, Shape)]) -> Vec { + use serde_json::Value as Json; -fn check_prose_against_ndjson(report: &str, ndjson: Option<&str>, found: &mut Vec) { - let Some(ndjson) = ndjson else { - return; + let Ok(parsed) = serde_json::from_str::>(row) else { + return vec![format!("the row is not a JSON object: {row}")]; }; - for (label, key, fact) in DOUBLE_RENDERED { - let Some(prose) = prose_field(report, label) else { - continue; - }; - let Some(json) = ndjson_field(ndjson, key) else { - // The prose made the claim; the row is required to carry it. - found.push(Correspondence::RenderedOnlyInProse { - fact, - prose: prose.to_owned(), - }); + let mut found = Vec::new(); + for (name, shape) in schema { + let Some(value) = parsed.get(*name) else { + found.push(format!("`{name}` is missing, so its shape cannot hold")); continue; }; - if prose != json { - found.push(Correspondence::ProseAndNdjsonDisagree { - fact, - prose: prose.to_owned(), - ndjson: json.to_owned(), - }); - } - } - - // The efficiency classes, whose two renderings differ in punctuation and so - // cannot be compared as text. This pair is here because it was wrong: the - // NDJSON once emitted the class COUNT under a plural name, so a - // single-class host printed `"efficiency_classes":1` beside a prose - // `efficiency classes: [0]` -- the same fact, in one report, in two - // renderings a consumer cannot reconcile. - // Gated on the PROSE, so a row that lost the field is a dropped counterpart - // rather than silence -- the same rule `compare` applies, restated here only - // because this pair needs RAW values and so cannot route through it. - let classes_prose = prose_field(report, " efficiency classes: "); - let classes_json = ndjson_raw_field(ndjson, "efficiency_classes"); - - if let Some(prose) = classes_prose - && classes_json.is_none() - { - found.push(Correspondence::RenderedOnlyInProse { - fact: "efficiency classes", - prose: prose.to_owned(), - }); + check_shape(name, *shape, value, &mut found); } - if let (Some(prose), Some(json)) = (classes_prose, classes_json) { - // **The container is part of the fact, and comparing only the contents - // threw it away.** `normalise_list` strips `[` and `]` from both sides, - // so a scalar `1` and a list `[1]` normalise to the same `"1"` -- which - // means the very regression this pair exists for, a class COUNT emitted - // under a plural name, survives undetected on any host whose single - // class is `1`. The old test caught the historical case only because it - // used class `[0]` against a count of `1`, so the VALUES differed; it - // established nothing about the shape. Found by a review. - // - // A host with a single class `1` is a supported shape, not a contrived - // one, so this is a live hole rather than a theoretical one. - // **Both sides, because the first version of this checked one.** It - // required the NDJSON to be a list and said nothing about the prose, so - // the mirror drift -- prose falling to `efficiency classes: 1` while the - // NDJSON still renders `[1]` -- was accepted: `normalise_list` strips - // the brackets from the JSON side and the two compare equal. Measured, - // before this: `check()` returned no violation for exactly that report. - // Found by a review, in the fix for the other direction. - if prose.starts_with('[') != json.starts_with('[') { - found.push(Correspondence::ProseAndNdjsonDisagree { - fact: "efficiency classes", - prose: prose.to_owned(), - ndjson: json.to_owned(), - }); - } else { - let prose_classes = normalise_list(prose); - let json_classes = normalise_list(json); - if prose_classes != json_classes { - found.push(Correspondence::ProseAndNdjsonDisagree { - fact: "efficiency classes", - prose: prose_classes, - ndjson: json_classes, - }); - } - } - } - - // The verdict, which the prose states as a sentence and the NDJSON as a - // token. - let prose_verdict = if has_line_beginning(report, "=> agree") { - Some("agree") - } else if has_line_beginning(report, "=> DISAGREE") { - Some("disagree") - } else if has_line_beginning(report, "=> INCOMPLETE") { - Some("incomplete") - } else { - None - }; - - // The verdict is the report's central claim, so a row that lost it is the - // worst case of the dropped-counterpart class rather than an exception to - // it: the prose still announces an answer and nothing machine-readable - // carries it. - if let Some(prose) = prose_verdict { - compare( - found, - "cross-check verdict", - prose, - ndjson_field(ndjson, "cross_check"), - ); - } + found } -/// The facts whose two renderings differ in shape rather than punctuation. -/// -/// Found by the M2.4 matrix rather than by a defect. The four counts already -/// checked above were the ones a reviewer had happened to look at; walking every -/// NDJSON field against the prose showed these carrying the same fact twice as -/// well, with nothing comparing them. -fn check_structured_pairs(report: &str, ndjson: Option<&str>, found: &mut Vec) { - let Some(ndjson) = ndjson else { - return; - }; - - // `NUMA domains : 1 (0 with no processors)` against two fields. - if let Some(prose) = prose_field(report, "NUMA domains : ") { - let total = prose.split_whitespace().next().unwrap_or_default(); - let without = prose - .split_once('(') - .and_then(|(_, rest)| rest.split_whitespace().next()) - .unwrap_or_default(); - - compare( - found, - "NUMA domains", - total, - ndjson_field(ndjson, "numa_domains"), - ); - compare( - found, - "NUMA domains without processors", - without, - ndjson_field(ndjson, "numa_domains_without_processors"), - ); - } - - // ` (3 reported only by CPU Sets, never by the relationship walk:` against - // the NDJSON's count of the same thing. - // - // **Rendered CONDITIONALLY**, and that is why it needs its own lookup rather - // than joining the pair above: the renderer emits the prose line only when - // the count is above zero, so on most hosts there is no line to find and - // `prose_field` returns `None`. That is silence, not agreement -- the rule - // fires only where the report actually makes the claim twice. - // - // Found by a review, and it is the third double-rendered fact this module - // shipped without reading. The other two were `arch` and the - // `outermost_partitioning_cache` discriminator, now read by - // `check_partitioning_answer` with all five arms mapped. The lesson the - // three share: - // a rule is added per fact, so the set of facts is the thing that drifts, - // and nothing here derives that set from the renderer. - // **Selected by what the line SAYS, not by being the first ` (` line.** - // `prose_field` takes the first line with the label, and ` (` is not a - // label -- it is the opening of any parenthesised continuation. A report - // with heterogeneous efficiency classes writes - // ` (heterogeneous: ...` ABOVE this one, so on that shape the first match - // was the wrong line, the `contains` guard below rejected it, and the - // CPU-Sets count went unread with no sign that it had. - // - // The accounting test reports that as an unread fact rather than hiding it, - // which is how it was found -- but only on a report carrying both, and this - // host renders neither. - if let Some(prose) = report - .lines() - .find(|line| line.starts_with(" (") && line.contains("reported only by CPU Sets")) - .map(|line| line[" (".len()..].trim()) - { - compare( - found, - "NUMA domains reported only by CPU Sets", - prose.split_whitespace().next().unwrap_or_default(), - ndjson_field(ndjson, "numa_domains_only_in_cpu_sets"), - ); - } - - // `outermost cache that partitions the processors it covers: L2 (8 domains)` - // against the level the NDJSON names. This pair is the one the original - // defect lived next to: the prose can name a level the machine-readable - // line does not. - if let Some(prose) = prose_field( - report, - "outermost cache that partitions the processors it covers: ", - ) { - let level = prose - .trim_start_matches('L') - .split_whitespace() - .next() - .unwrap_or_default(); - compare( - found, - "outermost partitioning cache level", - level, - ndjson_field(ndjson, "outermost_partitioning_cache_level"), - ); - } - - // The policy table against the `policies` object. A policy's domain count is - // what the whole report is for, so two renderings of it disagreeing would - // mislead exactly the reader who came for the answer. - // - // **The NAMES are compared before the counts, because the name is a - // double-rendered fact and not merely a lookup aid.** Locating the NDJSON - // entry by the prose name and comparing only the value makes a failed lookup - // silent: `find()` returns `None`, `compare` returns early, and renaming - // `by-core` on one side alone -- or dropping the entry, or adding one the - // prose never mentions -- is accepted. Found by a review. - // **The prose section is what decides whether the pair is required, so the - // CONTAINER's absence is checked the same way a member's is.** Gating the - // whole block on the object existing made "the renderer dropped the entire - // `policies` object" silent while the prose table still stood beside it -- - // the member-level tests covered an entry going missing from the object, - // never the object going missing from the row. Measured: deleting it left a - // report the oracle accepted. - let policies_section = has_line_beginning(report, "domains each policy would produce:"); - let policies = ndjson_field(ndjson, "policies"); - - if policies_section && policies.is_none() { - found.push(Correspondence::RenderedOnlyInProse { - fact: "policy names", - prose: "domains each policy would produce:".to_owned(), - }); - } - - if let Some(policies) = policies { - // Gated on the prose SECTION, not on the rows parsing. The header is - // what establishes that the report renders this set twice; requiring a - // non-empty row list instead would make "the prose table lost all its - // rows" look like silence rather than the disagreement it is. - if policies_section { - compare_membership( - found, - "policy names", - &policy_rows(report) - .into_iter() - .map(|(name, _)| name) - .collect::>(), - &object_keys(policies), - ); - } - - for (name, count) in policy_rows(report) { - let key = format!("\"{name}\":"); - let json = policies - .find(&key) - .map(|at| &policies[at + key.len()..]) - .map(|rest| { - let end = rest.find(',').unwrap_or(rest.len()); - rest[..end].trim() - }); - - // **An entry the object does not carry is a MEMBERSHIP finding, and - // `compare_membership` above has already reported it by name.** - // Letting the per-entry rule report absence as well produced a - // second, weaker violation for the same defect -- one that says a - // count had no counterpart without saying which policy it belonged - // to. Each rule reports its own concern once. - if json.is_some() { - compare(found, "policy domain count", &count, json); +/// Whether `value` has `shape`, appending one sentence per departure. +/// +/// **Recursive, and reporting WHERE rather than only THAT.** A lone +/// "`caches` should be a list of objects" cannot tell a reader which element +/// lost which member -- and the nested contract is the half a review found +/// unchecked, because `[{}]` satisfied "a list of objects" while publishing +/// none of the cache fields a survey mines. +fn check_shape(at: &str, shape: Shape, value: &serde_json::Value, found: &mut Vec) { + use serde_json::Value as Json; + + fn is_number(value: &Json) -> bool { + value.is_u64() || value.is_i64() + } + + let holds = match shape { + Shape::Text => value.is_string(), + Shape::Number => is_number(value), + Shape::NumberOrNull => is_number(value) || value.is_null(), + Shape::ListOfNumbers => value.as_array().is_some_and(|l| l.iter().all(is_number)), + Shape::ObjectOfNumbers => value.as_object().is_some_and(|o| o.values().all(is_number)), + Shape::ListOfObjectsWith(required) => { + let Some(entries) = value.as_array() else { + found.push(format!("`{at}` should be a list but is `{value}`")); + return; + }; + + for (index, entry) in entries.iter().enumerate() { + let Some(members) = entry.as_object() else { + found.push(format!( + "`{at}`[{index}] should be an object but is `{entry}`" + )); + continue; + }; + + for (member, member_shape) in required { + let Some(held) = members.get(*member) else { + found.push(format!("`{at}`[{index}] is missing `{member}`")); + continue; + }; + check_shape( + &format!("{at}[{index}].{member}"), + *member_shape, + held, + found, + ); + } } - } - } - - // The cache table against the `caches` array, level by level -- and the set - // of LEVELS first, for the reason given above. A cache moved from level 3 to - // level 9 in one rendering only is the same silent-lookup defect: the prose - // still reads `L3`, nothing matches it, and the report is accepted. - // The same container rule as `policies`: the prose SECTION is what makes the - // pair required, so losing the whole array is a dropped counterpart and not - // silence. - let caches_section = has_line_beginning(report, "caches:"); - let caches_field = ndjson_field(ndjson, "caches"); - - if caches_section && caches_field.is_none() { - found.push(Correspondence::RenderedOnlyInProse { - fact: "cache levels", - prose: "caches:".to_owned(), - }); - } - - if let Some(caches) = caches_field { - if caches_section { - compare_membership( - found, - "cache levels", - &cache_rows(report) - .into_iter() - .map(|(level, _)| level) - .collect::>(), - &cache_levels(caches), - ); - } - for (level, domains) in cache_rows(report) { - // **The object is located by its `level` member, then read for its - // `domains` member -- two independent steps, so member order and - // spelling are separate questions.** - // - // (History, because the shape of the bug is the reason for the - // shape of the code. This used to match one literal, - // `"level":N,"domains":`, which silently required the two members to - // be adjacent and in that order. Renaming, reordering or dropping - // `domains` alone made the lookup miss, and the prose domain count - // went uncompared while `cache_levels` still found every level and - // reported membership as agreeing. Measured then: both - // `{"level":1,"x-domains":8}` and `{"domains":8,"level":1}` were - // accepted, where `{"level":1,"domains":9}` was caught. The tests - // below now require the reordered object to stay readable and the - // renamed one to be reported.) - // - // A level the array does not carry at all is `compare_membership`'s - // finding above, reported by level number, so it is skipped here -- - // but an object that IS there and cannot answer is a dropped - // counterpart, which is `compare`'s business. - if let Some(object) = cache_object(caches, &level) { - compare( - found, - "cache domain count", - &domains, - ndjson_field(object, "domains"), - ); - } + return; } - } -} - -/// Push a disagreement when two renderings of one SET differ. -/// -/// Sorted before comparing, so a renderer free to emit its entries in a -/// different order from the prose is not accused of disagreeing about which -/// entries exist. Both sides get the same comparator, so the comparison stays -/// consistent whatever that order is. -fn compare_membership( - found: &mut Vec, - fact: &'static str, - prose: &[String], - ndjson: &[String], -) { - let mut prose_sorted = prose.to_vec(); - let mut ndjson_sorted = ndjson.to_vec(); - prose_sorted.sort(); - ndjson_sorted.sort(); - - if prose_sorted != ndjson_sorted { - found.push(Correspondence::ProseAndNdjsonDisagree { - fact, - prose: prose_sorted.join(", "), - ndjson: ndjson_sorted.join(", "), - }); - } -} - -/// The keys of a flat JSON object, in the order it renders them. -/// -/// A key is a quoted string followed immediately by `:`. `policies` is flat -- -/// name to count -- so no nesting has to be tracked here, and a value that -/// happened to be a string could not be mistaken for a key because it is not -/// followed by a colon. -fn object_keys(object: &str) -> Vec { - let mut keys = Vec::new(); - let mut rest = object; + }; - while let Some(open) = rest.find('"') { - let after_open = &rest[open + 1..]; - let Some(close) = after_open.find('"') else { - break; - }; - let (key, tail) = after_open.split_at(close); - let tail = &tail[1..]; - if tail.starts_with(':') { - keys.push(key.to_owned()); - } - rest = tail; + if !holds { + found.push(format!("`{at}` should be {shape:?} but is `{value}`")); } - - keys } -/// The `level` of each entry of the `caches` array, as rendered. -fn cache_levels(caches: &str) -> Vec { - const NEEDLE: &str = "\"level\":"; - let mut levels = Vec::new(); - let mut rest = caches; - - while let Some(at) = rest.find(NEEDLE) { - let after = &rest[at + NEEDLE.len()..]; - let end = after.find([',', '}']).unwrap_or(after.len()); - levels.push(after[..end].trim().to_owned()); - rest = &after[end..]; - } - - levels -} -/// Push a disagreement when both renderings are present and differ. -/// One prose reading against its machine-readable counterpart. +/// [`shape_violations`], as an assertion. /// -/// **Absence is reported HERE, so every rule inherits it.** Each caller reaches -/// this only after finding the prose, so a `None` counterpart is not "the report -/// does not mention this fact" -- it is "the report states this fact once and -/// the row lost it". Returning early on `None` made that silent for every -/// comparison routed through this helper at once. +/// # Panics /// -/// Fixing it at the three reported call sites first, rather than here, is what -/// left the rest: a later review named five more, and a deletion sweep of the -/// real report then found twelve keys whose removal the oracle accepted. The -/// helper is the only place the rule cannot be forgotten for the next fact -/// somebody adds. -fn compare(found: &mut Vec, fact: &'static str, prose: &str, json: Option<&str>) { - let Some(json) = json else { - found.push(Correspondence::RenderedOnlyInProse { - fact, - prose: prose.to_owned(), - }); - return; - }; - if prose != json { - found.push(Correspondence::ProseAndNdjsonDisagree { - fact, - prose: prose.to_owned(), - ndjson: json.to_owned(), - }); - } -} - -/// `(policy name, domain count)` for each row of the policy table. -fn policy_rows(report: &str) -> Vec<(String, String)> { - report - .lines() - .skip_while(|line| !line.starts_with("domains each policy would produce:")) - .skip(1) - .take_while(|line| line.starts_with(" ")) - .filter_map(|line| { - let mut parts = line.split_whitespace(); - Some((parts.next()?.to_owned(), parts.next()?.to_owned())) - }) - .collect() +/// Panics listing every value whose shape the schema forbids. +pub fn assert_row_has_the_schemas_shapes(report: &str, schema: &[(&str, Shape)]) { + let row = row(report).unwrap_or_else(|| panic!("no single well-formed row in:\n{report}")); + let violations = shape_violations(row, schema); + assert!( + violations.is_empty(), + "the row departs from its schema in {} way(s):\n{}\n\n--- the row ---\n{row}", + violations.len(), + violations + .iter() + .map(|what| format!(" - {what}")) + .collect::>() + .join("\n"), + ); } -/// The object in the `caches` array that names `level`, if one does. +/// [`check`], as an assertion, for tests that render a report. /// -/// Compares whole members rather than searching for a prefix, so `"level":1` -/// does not match the object for level 10, and does not care what order the -/// members are written in. -fn cache_object<'a>(caches: &'a str, level: &str) -> Option<&'a str> { - let named = format!("\"level\":{level}"); - - caches.split('{').find_map(|chunk| { - let object = chunk.split('}').next()?; - - object - .split(',') - .any(|member| member.trim() == named) - .then_some(object) - }) -} - -/// `(level, domain count)` for each row of the cache table. -fn cache_rows(report: &str) -> Vec<(String, String)> { - report - .lines() - .skip_while(|line| !line.starts_with("caches:")) - .skip(1) - .take_while(|line| line.starts_with(" ")) - .filter_map(|line| { - let mut parts = line.split_whitespace(); - let level = parts.next()?.strip_prefix('L')?.to_owned(); - Some((level, parts.next()?.to_owned())) - }) - .collect() -} - -/// The independently-read Win32 counters against the verdict drawn from them. +/// **Named `assert_corresponds` until 2026-09-13, and the name outlived what it +/// did.** Before M3 this compared a report's prose against its encoded row; M3 +/// made the row the machine contract and retired that comparison, leaving a +/// function that validates ONE thing -- that the report carries exactly one +/// well-formed JSON row. The old name went on promising a cross-part guarantee +/// to every reader of its four call sites. Renamed after a review read those +/// call sites as still enforcing correspondence, which is exactly the mistake +/// the name invited. /// -/// A different shape from the rules above, and the one closest to what this -/// probe is *for*. The prose prints each counter beside the enumerated value it -/// was read to check; the whole point of the run is that a mismatch is a -/// finding. So a counter that disagrees with the enumeration while the verdict -/// reads `agree` is the original defect in its purest form -- the report -/// showing its own contradicting evidence directly above a verdict denying it. -fn check_counters_against_verdict( - report: &str, - ndjson: Option<&str>, - found: &mut Vec, -) { - let Some(ndjson) = ndjson else { - return; - }; - if ndjson_field(ndjson, "cross_check") != Some("agree") { - return; - } - - // Only the two counters that are a direct count of an enumerated quantity. - // `GetNumaHighestNodeNumber` is deliberately absent: it reports the largest - // node NUMBER, which the report itself says is not a count, so comparing it - // against `numa_domains` would manufacture a disagreement on any machine - // with sparse node numbering. - for (label, key, fact) in [ - ( - " GetActiveProcessorCount : ", - "processors", - "active processor count against the enumeration", - ), - ( - " GetActiveProcessorGroupCount: ", - "groups", - "active group count against the enumeration", - ), - ] { - // **Reached only under an agreeing verdict**, which is what makes the - // absence of either side a contradiction rather than silence: `agree` - // asserts the check was MADE, and the counter line is the evidence it - // was. Skipping quietly accepted a report that claimed a check it did - // not show -- measured, by deleting the `GetActiveProcessorCount` line - // and watching `check` return nothing. - let (Some(counter), Some(enumerated)) = - (prose_field(report, label), ndjson_field(ndjson, key)) - else { - found.push(Correspondence::EvidenceMissingWithAgreeingVerdict { fact }); - continue; - }; - if counter != enumerated { - found.push(Correspondence::ProseAndNdjsonDisagree { - fact, - prose: counter.to_owned(), - ndjson: enumerated.to_owned(), - }); - } - } -} - -/// A list of numbers as a comparable string, whichever way it was punctuated. +/// # Panics /// -/// **Exactly one outer pair of brackets comes off, so nesting survives the -/// normalisation.** `trim_matches` removes every consecutive bracket, which -/// collapsed `[0]` and `[[0]]` to the same `0` -- so a renderer that regressed -/// to a nested array beside one-level prose would have compared equal. The -/// punctuation this is meant to forgive is one side writing `[0, 1]` where the -/// other writes `0,1`; a difference in DEPTH is a real disagreement and must -/// survive to be reported. -fn normalise_list(rendered: &str) -> String { - rendered - .strip_prefix('[') - .and_then(|inner| inner.strip_suffix(']')) - .unwrap_or(rendered) - .split(',') - .map(str::trim) - .filter(|piece| !piece.is_empty()) - .collect::>() - .join(",") -} - -/// Hardware claims the prose can make, each with the caveat that must accompany -/// it when the parse is in doubt. -const GATED_CLAIMS: &[(&str, &str, &str)] = &[( - "(heterogeneous: an I/O thread left unconstrained can land on an", - "This run did not establish that the parse is whole", - "heterogeneity", -)]; - -fn check_claims_against_doubt(report: &str, ndjson: Option<&str>, found: &mut Vec) { - let Some(ndjson) = ndjson else { - return; - }; - - // The report's own visible evidence that its parse was in doubt. - // - // These two are exactly what `CrossCheck::parse_in_doubt` is defined as -- - // a non-empty `parse_incomplete` or a non-empty `disagreements`, the latter - // being what makes the verdict `disagree`. That correspondence is the point - // rather than a coincidence: if the definition changes and this does not, - // the sabotage check in M2.2 is what should notice. - let mut evidence = Vec::new(); - if let Some(count) = ndjson_field(ndjson, "parse_incomplete") - && count != "0" - { - evidence.push(format!("parse_incomplete={count}")); - } - if ndjson_field(ndjson, "cross_check") == Some("disagree") { - evidence.push("cross_check=disagree".to_owned()); - } - - if evidence.is_empty() { - return; - } - - for (claim, caveat, name) in GATED_CLAIMS { - // The CLAIM is anchored and the CAVEAT is not, deliberately. A claim - // found where none was made invents a violation, which is the direction - // that must not be fooled by embedded text; a caveat found where none - // was made only SUPPRESSES one. And the caveat token is written to match - // mid-line on purpose -- the renderer prints it after `(of the levels - // that decoded. ` in one arm and after `(` in another -- so anchoring it - // would stop it matching the lines it exists for. - // **Scoped to the claim's OWN block, not the whole report.** A review - // predicted that another block's caveat could satisfy this one: the - // `Level` cache arm under `parse_in_doubt` writes the same sentence, and - // a report can be heterogeneous and in doubt and take that arm at once. - // - // Measured on exactly that crossed shape: it does NOT mask, because the - // renderer WRAPS the cache arm's sentence -- `... did not establish that - // the parse` ends one line and `is whole ...` begins the next -- so no - // single line carries the token. The finding was wrong about today's - // renderer and right about the code: that protection is an accident of - // where a line happens to break, and reflowing that sentence would - // silently turn the oracle blind to an uncaveated hardware claim. - // - // The block is the claim's line and the indented continuation under it, - // which is what the renderer actually emits and what the caveat belongs - // to. An accident that holds is still an accident. - if has_line_beginning(report, claim) - && !claim_block(report, claim).any(|line| line.contains(caveat)) - { - found.push(Correspondence::UncaveatedClaimUnderDoubt { - claim: name, - evidence: evidence.join(", "), - }); - } - } +/// Panics listing every way the row is malformed. +pub fn assert_row_is_well_formed(report: &str) { + let defects = check(report); + assert!( + defects.is_empty(), + "a rendered report's machine-readable row is malformed in {} way(s):\n{}\n\n\ + --- the report ---\n{report}", + defects.len(), + defects + .iter() + .map(|defect| format!(" - {defect}")) + .collect::>() + .join("\n"), + ); } #[cfg(test)] diff --git a/crates/windows-platform-probes/src/report_oracle/tests.rs b/crates/windows-platform-probes/src/report_oracle/tests.rs index dd8c5b083..487b1d33c 100644 --- a/crates/windows-platform-probes/src/report_oracle/tests.rs +++ b/crates/windows-platform-probes/src/report_oracle/tests.rs @@ -1,2517 +1,499 @@ // Copyright (c) Mike Grier. -//! Tests for the report oracle. -//! -//! Half of these assert the oracle **accepts** something. That is deliberate, -//! and follows `windows-file-watcher`'s `ContractChecker`: an oracle that -//! rejects legal reports is as broken as one that passes illegal ones, and it -//! fails in the more expensive direction, because the noise trains a reader to -//! ignore it. -//! -//! The reports here are hand-written text rather than rendered from an -//! `Observation`. That is the point: the oracle reads artifacts, so it must be -//! testable with artifacts, including ones no renderer would currently produce. - -use super::{Correspondence, check}; - -/// A fingerprint-shaped banner naming the architecture this build actually -/// targets. -/// -/// **Not a literal, because the renderer publishes `std::env::consts::ARCH`.** -/// A banner hard-coding `x86_64` agrees with the body on an x86_64 host and -/// contradicts it everywhere else, so every test that feeds a banner to the -/// RENDERER failed on `i686-pc-windows-msvc` -- five of them, each reporting -/// `prose: "x86_64"` against `ndjson: "x86"`. CI builds `aarch64` but does not -/// run the suite there, so the fleet never saw it either. -/// -/// Fixtures built entirely by hand are unaffected: their banner and their NDJSON -/// both say `x86_64`, so they agree with each other whatever the host is. Only -/// the ones that mix a literal banner with a rendered body were wrong. -fn host_banner(rest: &str) -> String { - format!("host: {} {rest}", std::env::consts::ARCH) -} - -/// A fingerprint this build can always produce, naming the build's architecture. -/// -/// Tests that need a successful bracket reading construct one rather than -/// calling `Fingerprint::discover()`, so they do not depend on the host being -/// able to read its own topology -- which is a failure this crate exists to -/// report, not one a test should be defeated by. -fn built_fingerprint() -> windows_placement_probe::fingerprint::Fingerprint { - windows_placement_probe::fingerprint::Fingerprint { - arch: std::env::consts::ARCH, - processors: 16, - cores: 8, - smt: true, - partitioning_cache_level: Some(2), - cache_domain_sizes: vec![8, 8], - efficiency_classes: vec![(0, 16)], - numa_node_sizes: vec![16], - provenance: windows_topology_sys::Provenance::Measured, - } -} - -/// A report body with the shape the topology probe emits, for a host that is -/// unremarkable and agrees with itself. -fn clean_report() -> String { - [ - "host: x86_64 16p/8c", - "== what does this machine look like? ==", - "", - "processors (online) : 16", - "processor groups : 1", - "packages : 1", - "NUMA domains : 1 (0 with no processors)", - "physical cores : 8", - " cores with SMT : 8", - " efficiency classes: [0]", - "", - "caches:", - " L1 8 domain(s), processors per domain: [2, 2, 2, 2, 2, 2, 2, 2]", - " L3 1 domain(s), processors per domain: [16]", - "", - "outermost cache that partitions the processors it covers: L1 (8 domains)", - "", - "domains each policy would produce:", - " single 1", - " by-core 8", - "", - "cross-check against independently read Win32 counters:", - " GetActiveProcessorCount : 16", - " GetActiveProcessorGroupCount: 1", - " GetNumaHighestNodeNumber : 0", - " => agree. Every check this probe could make was made and matched.", - r#"{"reason":"x-probe-topology","arch":"x86_64","processors":16,"groups":1,"packages":1,"numa_domains":1,"numa_domains_without_processors":0,"cores":8,"efficiency_classes":[0],"caches":[{"level":1,"domains":8},{"level":3,"domains":1}],"outermost_partitioning_cache_level":1,"outermost_partitioning_cache":"level","policies":{"single":1,"by-core":8},"cross_check":"agree","parse_incomplete":0}"#, - ] - .join("\n") -} - -// --- must reject ------------------------------------------------------------ - -#[test] -fn an_alarm_beside_an_agreeing_prose_verdict_is_a_violation() { - // The original defect, reduced: `BUG IN THIS PROBE` and `=> agree` in one - // report. Both statements were locally true and they cannot both describe - // the same run. - let report = clean_report().replace( - "cross-check against independently read Win32 counters:", - "BUG IN THIS PROBE: the topology crate named L3 as the outermost\ncross-check against independently read Win32 counters:", - ); - - let violations = check(&report); - - assert!( - violations.iter().any(|v| matches!( - v, - Correspondence::AlarmWithAgreeingVerdict { - verdict_source: "prose", - .. - } - )), - "an alarm printed beside `=> agree` must be reported, got {violations:#?}" - ); -} - -#[test] -fn an_alarm_beside_an_agreeing_ndjson_verdict_is_a_violation() { - // The same contradiction reaching a mining pass instead of a reader. It is - // reported separately because the two consumers are separate: a fleet - // survey never sees the prose. - let report = clean_report() - .replace( - " => agree. Every check this probe could make was made and matched.", - " => INCOMPLETE. Nothing this probe compared disagreed.", - ) - .replace( - "cross-check against independently read Win32 counters:", - "BUG IN THIS PROBE: something\ncross-check against independently read Win32 counters:", - ); - let violations = check(&report); - - assert!( - violations.iter().any(|v| matches!( - v, - Correspondence::AlarmWithAgreeingVerdict { - verdict_source: "ndjson", - .. - } - )), - "an alarm beside `\"cross_check\":\"agree\"` must be reported, got {violations:#?}" - ); -} - -#[test] -fn a_count_the_prose_and_the_ndjson_disagree_about_is_a_violation() { - let report = clean_report().replace("processors (online) : 16", "processors (online) : 8"); - - let violations = check(&report); - - assert!( - violations.iter().any(|v| matches!( - v, - Correspondence::ProseAndNdjsonDisagree { - fact: "online processors", - .. - } - )), - "two renderings of the processor count must agree, got {violations:#?}" - ); -} - -#[test] -fn a_class_count_rendered_where_the_class_list_belongs_is_a_violation() { - // The historical defect this pair exists for: the NDJSON emitted the class - // COUNT under a plural name, so a single-class host printed - // `"efficiency_classes":1` beside a prose `efficiency classes: [0]`. Same - // fact, same report, and the two readings differ -- one says "one class", - // the other says "class one". - let report = clean_report().replace(r#""efficiency_classes":[0]"#, r#""efficiency_classes":1"#); - - let violations = check(&report); - - assert!( - violations.iter().any(|v| matches!( - v, - Correspondence::ProseAndNdjsonDisagree { - fact: "efficiency classes", - .. - } - )), - "a class count where the class list belongs must be reported, got {violations:#?}" - ); -} - -#[test] -fn a_verdict_the_prose_and_the_ndjson_disagree_about_is_a_violation() { - let report = clean_report().replace(r#""cross_check":"agree""#, r#""cross_check":"disagree""#); - - let violations = check(&report); - - assert!( - violations.iter().any(|v| matches!( - v, - Correspondence::ProseAndNdjsonDisagree { - fact: "cross-check verdict", - .. - } - )), - "the verdict must read the same in both renderings, got {violations:#?}" - ); -} - -#[test] -fn a_bare_hardware_claim_under_an_incomplete_parse_is_a_violation() { - let report = clean_report() - .replace( - " efficiency classes: [0]", - " efficiency classes: [0, 1]\n (heterogeneous: an I/O thread left unconstrained can land on an", - ) - .replace(r#""efficiency_classes":[0]"#, r#""efficiency_classes":[0,1]"#) - .replace(r#""parse_incomplete":0"#, r#""parse_incomplete":2"#); - - let violations = check(&report); - - assert!( - violations.iter().any(|v| matches!( - v, - Correspondence::UncaveatedClaimUnderDoubt { - claim: "heterogeneity", - .. - } - )), - "a hardware claim stated bare under a short parse must be reported, got {violations:#?}" - ); -} - -#[test] -fn a_bare_hardware_claim_under_a_disagreeing_cross_check_is_a_violation() { - // The other half of `parse_in_doubt`. A disagreement is doubt about the - // parse just as much as an incomplete one, and the renderer's gate covers - // both -- so an oracle that only knew about `parse_incomplete` would pass - // exactly half the cases the rule is written for. - let report = clean_report() - .replace( - " efficiency classes: [0]", - " efficiency classes: [0, 1]\n (heterogeneous: an I/O thread left unconstrained can land on an", - ) - .replace(r#""efficiency_classes":[0]"#, r#""efficiency_classes":[0,1]"#) - .replace( - " => agree. Every check this probe could make was made and matched.", - " => DISAGREE. This is a finding, not a nuisance:", - ) - .replace(r#""cross_check":"agree""#, r#""cross_check":"disagree""#); - - let violations = check(&report); - - assert!( - violations.iter().any(|v| matches!( - v, - Correspondence::UncaveatedClaimUnderDoubt { - claim: "heterogeneity", - .. - } - )), - "a hardware claim stated bare under a disagreement must be reported, got {violations:#?}" - ); -} - -#[test] -fn every_double_rendered_fact_is_actually_read() { - // The failure mode that would make this whole oracle worthless, and would - // look exactly like success: a prose label that does not match what the - // renderer emits makes the lookup return `None`, the comparison is skipped, - // and the report passes having been checked for nothing. - // - // So each pair is exercised individually rather than trusted. Corrupting - // one prose value must produce one violation naming that fact; if a label - // ever drifts from the renderer, the corresponding case here stops firing - // and this test fails rather than the oracle going quietly blind. - // - // The labels themselves were confirmed against a real `probe-topology` run, - // which is what makes the fixture above a fixture and not a guess. - for (label, wrong) in [ - ("processors (online) : 16", "processors (online) : 99"), - ("processor groups : 1", "processor groups : 99"), - ("packages : 1", "packages : 99"), - ("physical cores : 8", "physical cores : 99"), - (" efficiency classes: [0]", " efficiency classes: [9]"), - ] { - let report = clean_report().replace(label, wrong); - let violations = check(&report); - - assert!( - violations - .iter() - .any(|v| matches!(v, Correspondence::ProseAndNdjsonDisagree { .. })), - "corrupting `{label}` produced no violation, so the oracle is not \ - reading that line at all -- got {violations:#?}" - ); - } -} - -// --- the cells the M2.4 matrix walk added ------------------------------------ - -#[test] -fn a_counter_that_contradicts_the_enumeration_under_an_agreeing_verdict_is_a_violation() { - // The rule closest to what this probe is *for*, and the original defect in - // its purest form: the report printing its own contradicting evidence - // directly above a verdict denying it. The whole run exists to compare an - // independently read counter against the enumeration, so a mismatch is the - // finding -- and `agree` says there was none. - let report = clean_report().replace( - " GetActiveProcessorCount : 16", - " GetActiveProcessorCount : 8", - ); - - let violations = check(&report); - - assert!( - violations.iter().any(|v| matches!( - v, - Correspondence::ProseAndNdjsonDisagree { - fact: "active processor count against the enumeration", - .. - } - )), - "a counter disagreeing with the enumeration under `agree` must be \ - reported, got {violations:#?}" - ); -} - -#[test] -fn a_contradicting_counter_is_accepted_when_the_verdict_reports_it() { - // The legal shape, and the one the probe exists to produce. Rejecting it - // would fire on every host that actually has the disagreement this probe - // hunts for -- the run most worth reading. - let report = clean_report() - .replace( - " GetActiveProcessorCount : 16", - " GetActiveProcessorCount : 8", - ) - .replace( - " => agree. Every check this probe could make was made and matched.", - " => DISAGREE. This is a finding, not a nuisance:", - ) - .replace(r#""cross_check":"agree""#, r#""cross_check":"disagree""#); - - assert_eq!(check(&report), Vec::new()); -} - -#[test] -fn the_highest_numa_node_number_is_not_compared_against_the_domain_count() { - // Deliberately absent from the counter rule, and pinned so it stays absent. - // `GetNumaHighestNodeNumber` reports the largest node NUMBER, which the - // report itself says is not a count; comparing it against `numa_domains` - // would manufacture a disagreement on any machine with sparse node - // numbering. Over-constraining is the same defect as under-specifying. - let report = clean_report().replace( - " GetNumaHighestNodeNumber : 0", - " GetNumaHighestNodeNumber : 7", - ); - - assert_eq!(check(&report), Vec::new()); -} - -#[test] -fn a_policy_count_the_two_renderings_disagree_about_is_a_violation() { - // The domain count per policy is the answer the whole report exists to - // give, so two renderings of it disagreeing misleads exactly the reader who - // came for it. - let report = clean_report().replace(r#""by-core":8"#, r#""by-core":4"#); - - let violations = check(&report); - - assert!( - violations.iter().any(|v| matches!( - v, - Correspondence::ProseAndNdjsonDisagree { - fact: "policy domain count", - .. - } - )), - "the policy table and the policies object must agree, got {violations:#?}" - ); -} - -#[test] -fn a_cache_domain_count_the_two_renderings_disagree_about_is_a_violation() { - let report = clean_report().replace(r#"{"level":3,"domains":1}"#, r#"{"level":3,"domains":9}"#); - - let violations = check(&report); - - assert!( - violations.iter().any(|v| matches!( - v, - Correspondence::ProseAndNdjsonDisagree { - fact: "cache domain count", - .. - } - )), - "the cache table and the caches array must agree, got {violations:#?}" - ); -} - -#[test] -fn an_outermost_level_the_two_renderings_disagree_about_is_a_violation() { - let report = clean_report().replace( - r#""outermost_partitioning_cache_level":1"#, - r#""outermost_partitioning_cache_level":3"#, - ); - - let violations = check(&report); - - assert!( - violations.iter().any(|v| matches!( - v, - Correspondence::ProseAndNdjsonDisagree { - fact: "outermost partitioning cache level", - .. - } - )), - "the named outermost level must match the machine-readable one, got {violations:#?}" - ); -} - -#[test] -fn a_nested_container_is_read_whole_rather_than_to_its_first_closer() { - // `caches` is an array OF objects, so a reader stopping at the first `}` - // would see only its first entry -- and would then silently skip every - // later level rather than compare it. This corrupts the LAST cache entry, - // which only a balanced read can reach. - let report = clean_report().replace(r#"{"level":3,"domains":1}"#, r#"{"level":3,"domains":5}"#); - - assert!( - !check(&report).is_empty(), - "a disagreement in the last element of a nested container must still be \ - found, or the container is being truncated at its first closer" - ); -} - -// --- must accept ------------------------------------------------------------ - -#[test] -fn a_report_that_agrees_with_itself_is_accepted() { - assert_eq!(check(&clean_report()), Vec::new()); -} - -#[test] -fn an_alarm_with_a_verdict_that_is_not_agree_is_accepted() { - // The legal shape of an alarm, and the one the fix produced. Rejecting it - // would make the oracle unusable on exactly the reports it was written to - // protect. - let report = clean_report() - .replace( - " => agree. Every check this probe could make was made and matched.", - " => INCOMPLETE. Nothing this probe compared disagreed, but this run", - ) - .replace(r#""cross_check":"agree""#, r#""cross_check":"incomplete""#) - .replace( - "cross-check against independently read Win32 counters:", - "BUG IN THIS PROBE: something\ncross-check against independently read Win32 counters:", - ); - assert_eq!(check(&report), Vec::new()); -} - -#[test] -fn a_hardware_claim_with_its_caveat_under_doubt_is_accepted() { - // Doubt plus a claim is legal when the claim is caveated. This is the - // shape the renderer actually produces, so an oracle that rejected it - // would fire on every heterogeneous host with a short parse. - // - // **The verdict moves with the doubt, and an earlier version of this - // fixture forgot that.** It set `"parse_incomplete":2` while leaving the - // verdict at `agree`, which the renderer cannot emit: it publishes the rule - // that a non-empty `parse_incomplete` forces the verdict away from `agree`. - // So the comment above claimed "the shape the renderer actually produces" - // about a shape it cannot produce -- caught when the rule that reads those - // counts was added and rejected this fixture. Both renderings of the verdict - // move together here, because the report renders it twice. - let report = clean_report() - .replace( - " efficiency classes: [0]", - " efficiency classes: [0, 1]\n (heterogeneous: an I/O thread left unconstrained can land on an\n (This run did not establish that the parse is whole, and the classes", - ) - .replace( - " => agree. Every check this probe could make was made and matched.", - " => INCOMPLETE. Nothing this probe compared disagreed, but this run\n\ - \x20 did not establish that the parse is consistent:\n\ - \x20 - a cache record failed to decode\n\ - \x20 - a second cache record failed to decode", - ) - .replace(r#""efficiency_classes":[0]"#, r#""efficiency_classes":[0,1]"#) - .replace(r#""cross_check":"agree""#, r#""cross_check":"incomplete""#) - .replace( - r#""parse_incomplete":0}"#, - r#""parse_incomplete":2,"not_compared":0,"enumeration_anomalies":0}"#, - ); - - assert_eq!(check(&report), Vec::new()); -} - -#[test] -fn a_bare_hardware_claim_with_no_doubt_reported_is_accepted() { - // The common case on a healthy heterogeneous host: the claim is bare - // because there is nothing to caveat. - let report = clean_report() - .replace( - " efficiency classes: [0]", - " efficiency classes: [0, 1]\n (heterogeneous: an I/O thread left unconstrained can land on an", - ) - .replace(r#""efficiency_classes":[0]"#, r#""efficiency_classes":[0,1]"#); - - assert_eq!(check(&report), Vec::new()); -} - -#[test] -fn a_report_with_no_ndjson_line_is_accepted() { - // `report_unmeasured` and any future prose-only report. Every NDJSON read - // is optional by construction, so a missing line is silence rather than a - // violation. - let prose = clean_report() - .lines() - .filter(|line| !line.starts_with('{')) - .collect::>() - .join("\n"); - - assert_eq!(check(&prose), Vec::new()); -} - -#[test] -fn a_shorter_ndjson_object_is_accepted() { - // `report_unmeasured` emits only `reason`, `arch` and `cross_check`. Fields - // the oracle knows about but the report does not carry are absent, not - // wrong. - let report = [ - "host: x86_64 16p/8c", - "the topology could not be read: something went wrong", - r#"{"reason":"x-probe-topology","arch":"x86_64","cross_check":"not_measured"}"#, - ] - .join("\n"); - - assert_eq!(check(&report), Vec::new()); -} - -#[test] -fn the_two_class_list_punctuations_are_read_as_the_same_list() { - // The prose renders `[0, 1]` through `Debug` and the NDJSON `[0,1]` through - // a join. They are the same fact, and an oracle that compared them as text - // would report every multi-class host as a contradiction. - let report = clean_report() - .replace(" efficiency classes: [0]", " efficiency classes: [0, 1]") - .replace( - r#""efficiency_classes":[0]"#, - r#""efficiency_classes":[0,1]"#, - ); - - assert_eq!(check(&report), Vec::new()); -} - -#[test] -fn a_banner_naming_a_different_machine_from_the_body_is_a_violation() { - // The M2.5 defect. Both halves are locally correct -- the banner faithfully - // renders one topology and the body another -- and the report reconciles - // them nowhere, so a reader deciding whether two runs are comparable is - // reading a line about a machine the numbers did not come from. - let report = clean_report().replace("host: x86_64 16p/8c", "host: x86_64 8p/8c"); - assert!( - !report.contains("16p/8c"), - "the sabotage must actually have landed, or this test proves nothing" - ); - - assert_eq!( - check(&report), - vec![Correspondence::BannerDisagreesWithBody { - banner: "8".to_owned(), - body: "16".to_owned(), - }] - ); -} - -#[test] -fn a_banner_for_a_host_that_could_not_be_read_is_accepted() { - // `Fingerprint::discover` failing renders `UNKNOWN` with no count in it. - // There is nothing to relate, and reporting a contradiction would turn a - // gap in the measurement into a claim about the report -- the inversion - // this whole crate is built to avoid. - let report = clean_report().replace( - "host: x86_64 16p/8c", - "host: UNKNOWN -- topology discovery failed: access denied", - ); - - assert_eq!(check(&report), Vec::new()); -} - -#[test] -fn a_tainted_banner_is_still_read_for_its_count() { - // An unmeasured topology renders behind a `!!...!! ` prefix. The taint says - // the numbers are not to be trusted as hardware -- it does not excuse the - // banner from naming the same numbers the body does, and a reader - // reconciling the two is exactly who the marker is for. - let report = clean_report().replace("host: x86_64 16p/8c", "host: !!assumed!! x86_64 8p/8c"); - - assert_eq!( - check(&report), - vec![Correspondence::BannerDisagreesWithBody { - banner: "8".to_owned(), - body: "16".to_owned(), - }] - ); -} - -#[test] -fn the_second_banner_line_of_a_disagreeing_bracket_is_not_compared() { - // When the endpoint readings differ, `attribution` prints the other reading - // too and says plainly that which one names the machine was not - // established. The body deliberately does not describe that second reading, - // so comparing it here would report a contradiction as a defect when it is - // the renderer being honest -- an oracle that over-constrains fails in the - // more expensive direction. - let report = clean_report().replace( - "host: x86_64 16p/8c", - "host: x86_64 16p/8c\nhost: x86_64 8p/8c\nHOST READINGS DISAGREE: the two readings above \ - bracket the measurement\nand differ, so which of them names the machine the body below \ - describes\nwas not established.", - ); - - assert_eq!(check(&report), Vec::new()); -} - -#[test] -fn an_indeterminate_attribution_is_not_a_banner_violation() { - // **The report says the correspondence was not established, so the oracle - // must not assert it.** `attribution` on this branch takes the two bracket - // readings and renders, when they differ: - // - // HOST READINGS DISAGREE: ... which of them names the machine the body - // below describes was not established. - // - // The body comes from a `measure()` between them, so the FIRST banner line - // is one endpoint and need not describe the body. Reporting a contradiction - // there would be the oracle over-claiming exactly as the renderer went to - // trouble not to -- and over-constraining is the failure this file's other - // half exists to catch. - // - // This is the seam the origin branch closed with `measure_observed`, which - // builds the banner from the body's own topology. That construction change - // is not in this peel, so the ambiguity is real here and the oracle has to - // respect it. - let report = clean_report().replace( - "host: x86_64 16p/8c", - "host: x86_64 8p/4c\n\ - host: x86_64 16p/8c\n\ - HOST READINGS DISAGREE: the two readings above bracket the measurement\n\ - and differ, so which of them names the machine the body below describes\n\ - was not established.", - ); - - assert_eq!( - check(&report), - Vec::new(), - "the first banner reading names 8 processors and the body 16, which the \ - report itself declines to call a contradiction" - ); -} - -#[test] -fn an_unestablished_host_is_not_a_banner_violation() { - // The other indeterminate form: at least one bracket reading failed, so - // nothing confirmed the host held still. Same reasoning, different text, - // and it is a separate arm of `attribution`'s match -- checking only the - // disagree case would leave this one asserting a correspondence the report - // does not claim. - // - // **The FAILED reading is the second one, and that ordering is the test.** - // An earlier version put `UNKNOWN` first, which is equally legal -- the - // `_` arm of `attribution` covers (Err, Ok), (Ok, Err) and (Err, Err) -- - // but it made this test vacuous: `processors_in_banner("host: UNKNOWN")` - // finds no count, so the rule returned at the count guard and never - // reached the disclaimer. Measured: deleting the `HOST NOT ESTABLISHED` - // arm of the exemption left that version, and the whole suite, green. - // Putting the successful reading first gives the rule a count to compare, - // so the exemption is the only thing that can suppress the violation. - let report = clean_report().replace( - "host: x86_64 16p/8c", - "host: x86_64 8p/4c\n\ - host: UNKNOWN\n\ - HOST NOT ESTABLISHED: at least one of the two readings that bracket the measurement\n\ - failed, so nothing confirmed the machine held still under it.", - ); - - assert_eq!( - check(&report), - Vec::new(), - "a failed bracket reading is not evidence the banner contradicts the body" - ); -} - -#[test] -fn the_unestablished_host_fixture_would_be_a_violation_without_the_disclaimer() { - // The control that keeps the test above honest. Same report, same 8-versus-16 - // contradiction, with only the disclaimer removed -- and now it MUST be a - // violation. Without this, a future edit that stops the rule reaching the - // comparison at all would leave the acceptance test passing for the wrong - // reason, which is precisely how the previous version went vacuous. - let report = clean_report().replace( - "host: x86_64 16p/8c", - "host: x86_64 8p/4c\n\ - host: UNKNOWN", - ); - - assert_eq!( - check(&report), - vec![Correspondence::BannerDisagreesWithBody { - banner: "8".to_owned(), - body: "16".to_owned(), - }], - "with no disclaimer the banner's 8 processors contradict the body's 16, so \ - the acceptance test above is established by the exemption rather than by \ - the comparison being unreachable" - ); -} -#[test] -fn an_architecture_the_banner_and_the_ndjson_disagree_about_is_a_violation() { - // Found by a review corrupting the NDJSON `arch` and watching the oracle - // accept it. Both renderings come from `std::env::consts::ARCH` today, so - // they cannot currently differ -- which is a fact about the renderer rather - // than a contract, and exactly the kind of coincidence this oracle is built - // not to lean on. - let report = clean_report().replace(r#""arch":"x86_64""#, r#""arch":"aarch64""#); - - assert!( - check(&report).iter().any(|violation| matches!( - violation, - Correspondence::ProseAndNdjsonDisagree { - fact: "architecture", - .. - } - )), - "the banner names x86_64 and the body aarch64: {:#?}", - check(&report) - ); -} - -#[test] -fn a_tainted_banner_is_still_read_for_its_architecture() { - // The taint prefix is a rendering of doubt about the READING, not a - // different machine, so the architecture behind it is still the one the - // banner claims. Skipping it here would quietly drop the correspondence on - // exactly the reports where a reader most wants it checked. - let report = clean_report() - .replace( - "host: x86_64 16p/8c", - "host: !!assumed!! !!taint!! x86_64 16p/8c", - ) - .replace(r#""arch":"x86_64""#, r#""arch":"aarch64""#); - - assert!( - check(&report).iter().any(|violation| matches!( - violation, - Correspondence::ProseAndNdjsonDisagree { - fact: "architecture", - .. - } - )), - "a tainted banner still names an architecture: {:#?}", - check(&report) - ); -} - -#[test] -fn an_architecture_disagreement_is_read_on_an_unmeasured_report_too() { - // The architecture rule was written below the processor-count guard, which - // confined it to MEASURED reports without saying so: `report_unmeasured` - // renders `arch` and no `processors`, so the guard returned first and the - // architecture went uncompared on exactly the reports that carry least - // else. Found by a review; measured before the fix as `check()` returning - // no violation at all for the report below. - // - // This is the shape `report_unmeasured` emits when the bracket reading - // succeeded and the measurement did not, so the banner names a real - // architecture while the body is the short object. - let report = "host: x86_64 16p/8c\n\ - MachineMemoryTopology::discover failed: a simulated failure\n\ - {\"reason\":\"x-probe-topology\",\"arch\":\"aarch64\",\"cross_check\":\"not_measured\"}\n"; - - assert_eq!( - check(report), - vec![Correspondence::ProseAndNdjsonDisagree { - fact: "architecture", - prose: "x86_64".to_owned(), - ndjson: "aarch64".to_owned(), - }], - "an unmeasured report renders the architecture twice like any other, so \ - the absence of a processor count must not suppress the comparison" - ); -} - -#[test] -fn a_cpu_set_only_numa_count_the_two_renderings_disagree_about_is_a_violation() { - // The third double-rendered fact this module shipped without reading, after - // `arch` and the outermost-cache discriminator. Found by a review, not by - // the oracle's own coverage -- which is the argument for M2.10's approach of - // deriving the fact set rather than extending it by hand. - // - // The prose line is rendered ONLY when the count is above zero, so the - // fixture has to add it: `clean_report()` describes a host with none. - let report = clean_report().replace( - "NUMA domains : 1 (0 with no processors)", - "NUMA domains : 1 (0 with no processors)\n (5 reported only by CPU Sets, never by the relationship walk:", - ); - let report = report.replace( - r#""numa_domains_without_processors":0"#, - r#""numa_domains_without_processors":0,"numa_domains_only_in_cpu_sets":9"#, - ); - - assert!( - check(&report).iter().any(|violation| matches!( - violation, - Correspondence::ProseAndNdjsonDisagree { - fact: "NUMA domains reported only by CPU Sets", - .. - } - )), - "the prose says 5 and the body 9: {:#?}", - check(&report) - ); -} - -#[test] -fn a_report_with_no_cpu_set_only_line_is_accepted() { - // The conditional half. On a host where the count is zero the renderer emits - // no such line, and absence is silence rather than a disagreement with the - // NDJSON's `0`. An oracle that read the missing line as a mismatch would - // fire on almost every host -- the over-constraining failure this file's - // acceptance half exists to catch. - let report = clean_report().replace( - r#""numa_domains_without_processors":0"#, - r#""numa_domains_without_processors":0,"numa_domains_only_in_cpu_sets":0"#, - ); - - assert_eq!( - check(&report), - Vec::new(), - "a host with no CPU-Set-only domains renders no line to compare" - ); -} - -#[test] -fn a_policy_the_ndjson_renames_is_a_violation() { - // Gap 4, found by a review: the policy NAME is a double-rendered fact, and - // locating the NDJSON entry by it made a failed lookup silent. Before this - // rule, renaming one side alone left the oracle with nothing to compare and - // the report was accepted. - let report = clean_report().replace(r#""by-core":8"#, r#""by-cores":8"#); - - assert_eq!( - check(&report), - vec![Correspondence::ProseAndNdjsonDisagree { - fact: "policy names", - prose: "by-core, single".to_owned(), - ndjson: "by-cores, single".to_owned(), - }], - "the prose names a policy the NDJSON does not" - ); -} - -#[test] -fn a_policy_missing_from_the_ndjson_is_a_violation() { - // The same silence in its other form. Dropping the entry leaves the prose - // row with nothing to match, which the per-entry comparison cannot report. - let report = clean_report().replace(r#","by-core":8"#, ""); - - assert_eq!( - check(&report), - vec![Correspondence::ProseAndNdjsonDisagree { - fact: "policy names", - prose: "by-core, single".to_owned(), - ndjson: "single".to_owned(), - }], - "a policy the prose reports is absent from the machine-readable line" - ); -} - -#[test] -fn a_policy_only_the_ndjson_reports_is_a_violation() { - // The third form, and the one a per-entry loop over PROSE rows can never - // see: an entry the prose never mentions is not iterated at all. - let report = clean_report().replace(r#""by-core":8}"#, r#""by-core":8,"by-l3":2}"#); - - assert_eq!( - check(&report), - vec![Correspondence::ProseAndNdjsonDisagree { - fact: "policy names", - prose: "by-core, single".to_owned(), - ndjson: "by-core, by-l3, single".to_owned(), - }], - "the NDJSON carries a policy the prose does not report" - ); -} - -#[test] -fn a_cache_the_ndjson_moves_to_another_level_is_a_violation() { - // The cache half of gap 4, and the exact case the review named: the prose - // still reads `L3` while the NDJSON calls it level 9, so the level lookup - // matches nothing and the domain count goes uncompared. - let report = clean_report().replace(r#"{"level":3,"domains":1}"#, r#"{"level":9,"domains":1}"#); - - assert_eq!( - check(&report), - vec![Correspondence::ProseAndNdjsonDisagree { - fact: "cache levels", - prose: "1, 3".to_owned(), - ndjson: "1, 9".to_owned(), - }], - "the two renderings disagree about which cache levels exist" - ); -} - -#[test] -fn collections_rendered_in_different_orders_are_accepted() { - // The acceptance half, and the reason the comparison sorts. Nothing obliges - // the NDJSON to emit its entries in the prose's order, so a rule comparing - // sequences would report a contradiction about ordering that neither - // rendering claims. Both sides get the same comparator, so this stays - // consistent whatever order either chooses. - let report = clean_report() - .replace( - r#""policies":{"single":1,"by-core":8}"#, - r#""policies":{"by-core":8,"single":1}"#, - ) - .replace( - r#""caches":[{"level":1,"domains":8},{"level":3,"domains":1}]"#, - r#""caches":[{"level":3,"domains":1},{"level":1,"domains":8}]"#, - ); - - assert_eq!( - check(&report), - Vec::new(), - "the same entries in a different order are the same entries" - ); -} - -#[test] -fn an_agreeing_verdict_beside_a_nonzero_parse_incomplete_is_a_violation() { - // Gap 5, and the reason it carries a correctness question rather than only - // a completeness one. The renderer publishes the rule where it emits the - // NDJSON: anomalies populate `parse_incomplete`, a non-empty - // `parse_incomplete` forces the verdict away from `agree`, so - // `cross_check == "agree"` implies no record failed to decode. An `agree` - // beside a nonzero count is therefore the report contradicting its own - // published rule, in the field a mining pass trusts before any other. - let report = clean_report().replace(r#""parse_incomplete":0"#, r#""parse_incomplete":2"#); - - assert_eq!( - check(&report), - vec![Correspondence::AlarmWithAgreeingVerdict { - alarm: r#""parse_incomplete":2"#.to_owned(), - verdict_source: "ndjson", - }], - "a parse that did not complete cannot sit beside a verdict saying every \ - check matched" - ); -} - -#[test] -fn an_agreeing_verdict_beside_a_nonzero_anomaly_count_is_a_violation() { - // The same rule reached through the other field it names. Anomalies populate - // `parse_incomplete`, so an `agree` verdict rules both out. - let report = clean_report().replace( - r#""parse_incomplete":0}"#, - r#""parse_incomplete":0,"enumeration_anomalies":1}"#, - ); - - assert_eq!( - check(&report), - vec![Correspondence::AlarmWithAgreeingVerdict { - alarm: r#""enumeration_anomalies":1"#.to_owned(), - verdict_source: "ndjson", - }], - "a dropped enumeration record cannot sit beside an agreeing verdict" - ); -} - -#[test] -fn a_not_compared_count_the_two_renderings_disagree_about_is_a_violation() { - // The completeness half. Under DISAGREE the prose labels each skipped check - // on its own line, so the two renderings of how many there were can be - // compared directly. - // `clean_report()` renders no `not_compared` at all -- it is narrower than - // the real renderer -- so the field has to be added here rather than - // replaced. Written as a replacement of an absent key, this test passed - // while checking nothing, which is how the first draft of it went green. - let report = clean_report() - .replace( - " => agree. Every check this probe could make was made and matched.", - " => DISAGREE. This is a finding, not a nuisance:\n\ - \x20 - the group count disagrees\n\ - \x20 (not compared) GetNumaHighestNodeNumber is not a count", - ) - .replace(r#""cross_check":"agree""#, r#""cross_check":"disagree""#) - .replace( - r#""parse_incomplete":0}"#, - r#""parse_incomplete":0,"not_compared":4,"enumeration_anomalies":0}"#, - ); - - assert_eq!( - check(&report), - vec![Correspondence::ProseAndNdjsonDisagree { - fact: "not compared count", - prose: "1".to_owned(), - ndjson: "4".to_owned(), - }], - "the prose lists one skipped check and the NDJSON claims four" - ); -} - -#[test] -fn an_incomplete_verdict_listing_fewer_entries_than_it_counts_is_a_violation() { - // Under INCOMPLETE both kinds render as a bare `- `, so only their TOTAL is - // recoverable from the prose -- and that total is what this compares. - // Claiming to separate them here would be reading a distinction the prose - // does not draw. - let report = clean_report() - .replace( - " => agree. Every check this probe could make was made and matched.", - " => INCOMPLETE. Nothing this probe compared disagreed, but this run\n\ - \x20 did not establish that the parse is consistent:\n\ - \x20 - a cache record failed to decode", - ) - .replace(r#""cross_check":"agree""#, r#""cross_check":"incomplete""#) - .replace( - r#""parse_incomplete":0}"#, - r#""parse_incomplete":3,"not_compared":0,"enumeration_anomalies":0}"#, - ); - - assert_eq!( - check(&report), - vec![Correspondence::ProseAndNdjsonDisagree { - fact: "incomplete-verdict listing count", - prose: "1".to_owned(), - ndjson: "3".to_owned(), - }], - "one entry is listed where the counts total three" - ); -} - -#[test] -fn a_nonzero_not_compared_beside_an_incomplete_verdict_is_accepted() { - // The acceptance half, and it pins the direction of the rule. The renderer - // states the implication ONE WAY: a run whose counter failed to read has a - // complete parse and still reports `incomplete`. So a nonzero - // `not_compared` is legal here, and a rule asserting it forces the verdict - // would be claiming more than the contract does. - let report = clean_report() - .replace( - " => agree. Every check this probe could make was made and matched.", - " => INCOMPLETE. Nothing this probe compared disagreed, but this run\n\ - \x20 did not establish that the parse is consistent:\n\ - \x20 - GetActiveProcessorCount could not be read", - ) - .replace(r#""cross_check":"agree""#, r#""cross_check":"incomplete""#) - .replace( - r#""parse_incomplete":0}"#, - r#""parse_incomplete":0,"not_compared":1,"enumeration_anomalies":0}"#, - ); - - assert_eq!( - check(&report), - Vec::new(), - "a counter that could not be read is exactly what an incomplete verdict \ - reports, and the parse is untouched by it" - ); -} - -/// The prose each discriminator arm announces itself with, as the renderer -/// writes it. Used to exercise every arm rather than the one this host happens -/// to produce. -fn partitioning_prose(arm: &str) -> &'static str { - match arm { - "level" => "\noutermost cache that partitions the processors it covers: L1 (8 domains)", - "none" => { - "\nno cache level reported more than one domain, so nothing here divides\nthe work by cache." - } - "no_levels_reported" => { - "\nno cache levels were reported at all, so nothing here says whether a\ncache boundary divides this machine." - } - "not_unique" => { - "\nat least one cache level reported more than one distinct domain, but\nno unique outermost one was established: either two partition this\nmachine incomparably, or the candidates were rejected as overlapping\n-- in which case none of them partitions it at all." - } - "summary_missing" => { - "\nBUG IN THIS PROBE: the topology crate named L1 as the outermost\npartitioning cache and this survey carries no summary for it. Nothing\nbelow about cache partitioning can be trusted." - } - other => unreachable!("unmapped arm {other}"), - } -} - -/// `clean_report()` with its partitioning prose and discriminator set -/// independently, so the two can be made to disagree. -fn report_with_partitioning(prose_arm: &str, published: &str) -> String { - clean_report() - .replace( - "\noutermost cache that partitions the processors it covers: L1 (8 domains)", - partitioning_prose(prose_arm), - ) - // **Replaces the discriminator rather than adding one.** This appended a - // second member, which was harmless while `clean_report()` carried no - // discriminator at all -- and stopped being harmless the moment the - // fixture was corrected to publish the field the renderer always emits. - // Measured then: every fixture from this builder carried - // `"outermost_partitioning_cache"` TWICE, so the row was not JSON any - // consumer could parse, and the tests passed only because - // `ndjson_field` happens to read the first of the two. - .replace( - r#""outermost_partitioning_cache":"level""#, - &format!(r#""outermost_partitioning_cache":"{published}""#), - ) -} - -#[test] -fn the_partitioning_fixtures_publish_one_discriminator_each() { - // **The fixture builder must produce a row a consumer could parse.** It - // appended the discriminator rather than replacing it, which was invisible - // while `clean_report()` carried none -- and the moment that fixture was - // corrected to publish what the renderer always emits, every report from - // this builder carried the key TWICE. The suite stayed green because - // `ndjson_field` reads the first of the two, so the tests were right by - // accident about an artifact the crate cannot emit. - for arm in [ - "level", - "none", - "no_levels_reported", - "not_unique", - "summary_missing", - ] { - let text = report_with_partitioning(arm, arm); - let row = text - .lines() - .find(|line| line.starts_with('{')) - .unwrap_or_default(); - - assert_eq!( - row.matches(r#""outermost_partitioning_cache":"#).count(), - 1, - "the {arm} fixture must name the discriminator once:\n{row}" - ); - assert!( - row.contains(&format!(r#""outermost_partitioning_cache":"{arm}""#)), - "and it must be the arm asked for:\n{row}" - ); - } -} - -#[test] -fn every_partitioning_arm_agreeing_with_its_prose_is_accepted() { - // The acceptance half, walked over EVERY arm rather than the one this host - // produces. The checklist item that queued this work said to map every arm - // to its prose before writing the rule, because a rule against a guessed - // subset fires falsely on the arms it guessed wrong -- which this module has - // already done once, in the banner rule. - for arm in [ - "level", - "none", - "no_levels_reported", - "not_unique", - "summary_missing", - ] { - let report = report_with_partitioning(arm, arm); - let violations = check(&report); - - // `summary_missing` opens with `BUG IN THIS PROBE`, which the alarm rule - // reads -- correctly, and beside an agreeing verdict. That is a real - // correspondence about a different fact, so it is expected here rather - // than suppressed. - let partitioning: Vec<_> = violations - .iter() - .filter(|violation| { - matches!( - violation, - Correspondence::ProseAndNdjsonDisagree { - fact: "outermost partitioning answer", - .. - } - ) - }) - .collect(); - - assert!( - partitioning.is_empty(), - "arm {arm} agrees with its own prose and must not be reported: {violations:#?}" - ); - } -} - -#[test] -fn every_partitioning_arm_contradicting_its_prose_is_a_violation() { - // Corrupting each arm in turn, which is the other half of what the item - // asked for. Each arm is paired with a DIFFERENT published value, so no arm - // is left resting on another's coverage. - for (prose_arm, published) in [ - ("level", "none"), - ("none", "level"), - ("no_levels_reported", "not_unique"), - ("not_unique", "no_levels_reported"), - ("summary_missing", "level"), - ] { - let report = report_with_partitioning(prose_arm, published); - - assert!( - check(&report).contains(&Correspondence::ProseAndNdjsonDisagree { - fact: "outermost partitioning answer", - prose: prose_arm.to_owned(), - ndjson: published.to_owned(), - }), - "prose announcing {prose_arm} beside a published {published} is two \ - opposite answers to this probe's central question, and went unread \ - until this rule: {:#?}", - check(&report) - ); - } -} - -#[test] -fn the_defect_the_review_found_is_a_violation() { - // The concrete case reported: a real report whose prose names a partitioning - // level while the discriminator says no level partitions. Before this rule - // the oracle accepted it, because it compared only the LEVEL NUMBER, which - // both renderings still agreed about. - let report = report_with_partitioning("level", "none"); - - assert_eq!( - check(&report), - vec![Correspondence::ProseAndNdjsonDisagree { - fact: "outermost partitioning answer", - prose: "level".to_owned(), - ndjson: "none".to_owned(), - }], - "the prose names L1 as partitioning while the NDJSON says none does" - ); -} - -#[test] -fn a_report_publishing_no_discriminator_is_accepted() { - // **Asserted against the artifact that really carries neither.** This - // asserted on `clean_report()`, whose row had no discriminator when the test - // was written -- and then the fixture was corrected to publish the field the - // renderer always emits, and the test went on passing while testing the - // opposite of its name. It would have stayed green if the missing - // -counterpart rule had regressed. - // - // `report_unmeasured` is the shape where the question actually arises: every - // MEASURED report announces a partitioning arm, because `PartitioningCache` - // has no silent variant, so a measured row that dropped the discriminator is - // a dropped counterpart and not silence -- which is - // `a_dropped_counterpart_is_a_violation_and_not_silence`. Here there is no - // prose claim, so there is nothing to relate. - let unmeasured = crate::topology_report::report_unmeasured( - &host_banner("16p/8c"), - &std::io::Error::other("a simulated failure"), - ); - - assert!( - !unmeasured.contains("outermost_partitioning_cache"), - "this test is vacuous unless the row really lacks the field:\n{unmeasured}" - ); - assert_eq!( - check(&unmeasured), - Vec::new(), - "a report that makes the claim in neither rendering has nothing to relate" - ); -} - -#[test] -fn a_summary_missing_level_the_two_renderings_disagree_about_is_a_violation() { - // Gap 6, and the sixth unread double-rendering found by a sixth reviewer - // rather than by this module's own coverage -- which is the finding M2.10 - // exists for, arriving on schedule while that item sat open. - // - // `summary_missing` is the one non-`Level` arm whose NDJSON level is a - // NUMBER rather than `null`, so it is the only one of the three that can - // disagree with the prose at all. The oracle's other level comparison is - // keyed to the `Level` arm's prose label and never fires here, so before - // this rule the two numbers were rendered side by side and never related. - let report = report_with_partitioning("summary_missing", "summary_missing").replace( - r#""outermost_partitioning_cache_level":1"#, - r#""outermost_partitioning_cache_level":99"#, - ); - - assert!( - check(&report).contains(&Correspondence::ProseAndNdjsonDisagree { - fact: "summary-missing outermost level", - prose: "1".to_owned(), - ndjson: "99".to_owned(), - }), - "the prose names L1 and the NDJSON publishes 99: {:#?}", - check(&report) - ); -} - -#[test] -fn a_summary_missing_level_both_renderings_agree_about_is_accepted() { - // The acceptance half. The arm always reports `BUG IN THIS PROBE`, so this - // report is not silent -- the alarm rule reads it correctly, and that is a - // true correspondence about a different fact. What must NOT appear is a - // disagreement about the level, which both renderings give as 1. - let report = report_with_partitioning("summary_missing", "summary_missing"); - - assert!( - !check(&report).iter().any(|violation| matches!( - violation, - Correspondence::ProseAndNdjsonDisagree { - fact: "summary-missing outermost level", - .. - } - )), - "both renderings name L1, so the level is not a disagreement: {:#?}", - check(&report) - ); -} -#[test] -fn a_report_with_no_summary_missing_arm_reports_no_level_of_it() { - // The rule is keyed to the arm's own sentence, so a report that does not - // carry that arm has nothing to relate. Pinned because a marker matched too - // loosely would fire on every report that happens to mention a level. - assert!( - !check(&clean_report()).iter().any(|violation| matches!( - violation, - Correspondence::ProseAndNdjsonDisagree { - fact: "summary-missing outermost level", - .. - } - )), - "the clean report announces the `level` arm, not `summary_missing`" - ); -} +//! Tests for the row's well-formedness check. +//! +//! Half of these assert ACCEPTANCE. A check that fires on a legal row costs a +//! reader more than one that misses an illegal one, because noise trains them to +//! ignore the instrument -- and the reports this runs against are the ones a +//! fleet survey mines, so a false alarm is a false finding about a host. -#[test] -fn a_class_count_that_matches_the_single_class_is_still_a_violation() { - // The regression this pair exists for, on the host that hides it. The NDJSON - // once emitted the class COUNT under a plural name; the original test caught - // that with prose `[0]` against a count of `1`, where the VALUES differ. On a - // host whose single class is `1`, the count and the list have the same - // contents, and stripping the brackets from both made them identical. - // - // Found by a review. The container is part of the fact. - let report = clean_report() - .replace(" efficiency classes: [0]", " efficiency classes: [1]") - .replace(r#""efficiency_classes":[0]"#, r#""efficiency_classes":1"#); +use super::{RowDefect, check, keys, malformation, row}; - assert_eq!( - check(&report), - vec![Correspondence::ProseAndNdjsonDisagree { - fact: "efficiency classes", - prose: "[1]".to_owned(), - ndjson: "1".to_owned(), - }], - "a scalar where a list belongs is the historical defect, whatever the value" - ); +/// A well-formed row, in the shape the renderer emits. +fn clean_row() -> String { + concat!( + r#"{"reason":"x-probe-topology","arch":"x86_64","processors":16,"#, + r#""efficiency_classes":[0],"caches":[{"level":1,"domains":8}],"#, + r#""policies":{"single":1},"cross_check":"agree","disagreements":[],"#, + r#""parse_incomplete":[]}"# + ) + .to_owned() } -#[test] -fn a_single_class_list_rendered_as_a_list_is_accepted() { - // The acceptance half, so the rule reads the CONTAINER rather than merely - // rejecting anything whose text is short. - // - // **The first draft of this test replaced `[1]` with `[1]`**, which matches - // nothing in a fixture that renders `[0]` -- so it re-checked the clean - // report and established nothing about single-class hosts at all. Written - // while fixing a defect of exactly that shape, which is how persistent it - // is. - let report = clean_report() - .replace(" efficiency classes: [0]", " efficiency classes: [1]") - .replace(r#""efficiency_classes":[0]"#, r#""efficiency_classes":[1]"#); - - assert_eq!( - check(&report), - Vec::new(), - "a single class rendered as a list on both sides agrees, so the rule reads \ - the container and not merely the digit" - ); +/// A report carrying `row` under a line of prose. +fn report_with(row: &str) -> String { + format!("host: x86_64 16p/8c\nsome prose the reader gets\n{row}") } #[test] -fn an_anomaly_count_the_two_renderings_disagree_about_is_a_violation() { - // `CrossCheck` renders the anomaly count INSIDE one diagnostic sentence, - // however many anomalies there were, so no count of prose lines can check - // it. Before this rule the field was read only as a nonzero predicate under - // an `agree` verdict, which left the number itself unrelated for every other - // verdict. Found by a review. - let report = clean_report() - .replace( - " => agree. Every check this probe could make was made and matched.", - " => INCOMPLETE. Nothing this probe compared disagreed, but this run\n\ - \x20 did not establish that the parse is consistent:\n\ - \x20 - windows-topology-sys recorded 2 enumeration anomalies, so what Windows returned", - ) - .replace(r#""cross_check":"agree""#, r#""cross_check":"incomplete""#) - .replace( - r#""parse_incomplete":0}"#, - r#""parse_incomplete":1,"not_compared":0,"enumeration_anomalies":99}"#, - ); - - assert!( - check(&report).contains(&Correspondence::ProseAndNdjsonDisagree { - fact: "enumeration anomaly count", - prose: "2".to_owned(), - ndjson: "99".to_owned(), - }), - "the prose says it recorded 2 and the field publishes 99: {:#?}", - check(&report) - ); +fn a_well_formed_row_is_accepted() { + assert_eq!(check(&report_with(&clean_row())), Vec::new()); } #[test] -fn an_anomaly_count_both_renderings_agree_about_is_accepted() { - // The acceptance half, on the same shape. - let report = clean_report() - .replace( - " => agree. Every check this probe could make was made and matched.", - " => INCOMPLETE. Nothing this probe compared disagreed, but this run\n\ - \x20 did not establish that the parse is consistent:\n\ - \x20 - windows-topology-sys recorded 2 enumeration anomalies, so what Windows returned", - ) - .replace(r#""cross_check":"agree""#, r#""cross_check":"incomplete""#) - .replace( - r#""parse_incomplete":0}"#, - r#""parse_incomplete":1,"not_compared":0,"enumeration_anomalies":2}"#, - ); - +fn a_report_with_no_row_is_a_defect() { + // Every report has one, including the unmeasured shape -- that is what lets + // a survey tell a host where discovery failed from a job that never ran the + // probe. assert_eq!( - check(&report), - Vec::new(), - "two and two agree, and the incomplete listing totals one entry" + check("host: x86_64 16p/8c\nprose only, no row"), + vec![RowDefect::Missing] ); } #[test] -fn an_architecture_contradiction_survives_an_attribution_disclaimer() { - // The disclaimer says which of the two bracket READINGS describes the body - // was not established -- a statement about the machine's topology, not its - // instruction set. When both readings name the same architecture, whichever - // one describes the body, the architecture is that one. So a body naming a - // different one contradicts them both, and the exemption does not cover it. - // - // Found by a review: before this, the disclaimer returned before the - // architecture was ever compared, and this report produced no violation. - let report = clean_report() - .replace( - "host: x86_64 16p/8c", - "host: x86_64 8p/4c\n\ - host: x86_64 16p/8c\n\ - HOST READINGS DISAGREE: the two readings above bracket the measurement\n\ - and differ, so which of them names the machine the body below describes\n\ - was not established.", - ) - .replace(r#""arch":"x86_64""#, r#""arch":"aarch64""#); +fn a_report_with_two_rows_is_a_defect() { + // A mining pass reads the first line that looks like a row, so a second is + // not extra data -- it is an ambiguity about which line is the contract. + let two = format!("{}\n{}", report_with(&clean_row()), clean_row()); - assert_eq!( - check(&report), - vec![Correspondence::ProseAndNdjsonDisagree { - fact: "architecture", - prose: "x86_64".to_owned(), - ndjson: "aarch64".to_owned(), - }], - "both readings say x86_64 and the body says aarch64; the processor-count \ - exemption does not reach this" - ); + assert_eq!(check(&two), vec![RowDefect::Duplicated { count: 2 }]); } -#[test] -fn banners_disagreeing_about_the_architecture_are_not_a_violation() { - // The limit of the rule above, and the reason it is stated as "every banner - // agrees" rather than "the first banner". When the two readings name - // DIFFERENT architectures, which one describes the body really is - // unestablished, and asserting either would be the over-claim the exemption - // exists to prevent. - let report = clean_report().replace( - "host: x86_64 16p/8c", - "host: aarch64 8p/4c\n\ - host: x86_64 16p/8c\n\ - HOST READINGS DISAGREE: the two readings above bracket the measurement\n\ - and differ, so which of them names the machine the body below describes\n\ - was not established.", - ); - - assert_eq!( - check(&report), - Vec::new(), - "the readings disagree with each other, so the body agrees with one of them \ - and this run cannot say which should have described it" - ); +/// What the parser says about `row`, for a test asserting a defect's identity +/// rather than its wording. +/// +/// **The wording is `serde_json`'s, so this crate does not get to assert it.** +/// These three tests used to name the message, which was right while the check +/// was ours -- the message WAS the finding, and a test naming it pinned which +/// branch fired. It is now a dependency's string, and pinning it would assert a +/// thing we neither own nor promise: a wording change in a patch release would +/// redden tests about unclosed delimiters, which is a false finding about this +/// crate. What survives is what these tests are actually for -- that this input +/// is rejected, and that the whole row is carried back for a reader. +fn malformation_of(row: &str) -> String { + serde_json::from_str::>(row) + .expect_err("the fixture is meant to be malformed") + .to_string() } #[test] -fn a_class_list_the_prose_renders_as_a_scalar_is_a_violation() { - // The mirror of `a_class_count_that_matches_the_single_class_is_still_a_violation`. - // That one pinned the NDJSON side; this pins the prose side, because the - // first fix checked only one of them and `normalise_list` strips the - // brackets from whichever side has them. Found by a review of that fix. - let report = clean_report().replace(" efficiency classes: [0]", " efficiency classes: 0"); +fn an_unclosed_delimiter_is_a_defect() { + let truncated = r#"{"reason":"x-probe-topology","caches":[{"level":1}"#; assert_eq!( - check(&report), - vec![Correspondence::ProseAndNdjsonDisagree { - fact: "efficiency classes", - prose: "0".to_owned(), - ndjson: "[0]".to_owned(), - }], - "the container is part of the fact in BOTH renderings, not just the \ - machine-readable one" + check(&report_with(truncated)), + vec![RowDefect::Malformed { + what: malformation_of(truncated), + row: truncated.to_owned() + }] ); } #[test] -fn an_agreeing_verdict_beside_skipped_work_is_a_violation() { - // `CrossCheck`'s verdict makes `agree` imply that nothing was skipped as - // well as that nothing failed to decode, so an agreeing report publishing a - // nonzero `not_compared` contradicts its own published rule -- the same - // shape as a nonzero `parse_incomplete` beside `agree`, which this module - // already read. Found by a review. - let report = clean_report().replace( - r#""parse_incomplete":0}"#, - r#""parse_incomplete":0,"not_compared":3,"enumeration_anomalies":0}"#, - ); +fn a_trailing_separator_is_a_defect() { + // **Balanced but invalid**, which the depth-only check accepted. A writer + // that emitted a separator for a member it then skipped produces exactly + // this, and every bracket still matches. + let trailing = r#"{"reason":"x-probe-topology","arch":"x86_64",}"#; assert_eq!( - check(&report), - vec![Correspondence::AlarmWithAgreeingVerdict { - alarm: r#""not_compared":3"#.to_owned(), - verdict_source: "ndjson", - }], - "work the probe skipped cannot sit beside a verdict saying every check \ - it could make was made" + check(&report_with(trailing)), + vec![RowDefect::Malformed { + what: malformation_of(trailing), + row: trailing.to_owned() + }] ); } #[test] -fn a_caller_error_that_mimics_the_probe_is_not_read_as_the_probe() { - // `report_unmeasured` embeds the caller's `io::Error`, so the report carries - // text the renderer does not own. An unanchored substring search reads that - // text as if the probe had spoken it. - // - // The renderer CONTAINS that text now -- `renderer_owns_every_line` flattens - // it, so it cannot introduce a line -- but its words still sit inside the - // discovery-failure line, which is why this test is about anchoring rather - // than about containment. The two answer different halves. - // - // Measured, before the anchoring: this report tripped the alarm rule and - // panicked inside the renderer's own binding -- the oracle inventing a - // contradiction out of a message it should treat as opaque. Found by a - // review. - let report = crate::topology_report::report_unmeasured( - &host_banner("16p/8c"), - &std::io::Error::other("BUG IN THIS PROBE => agree"), - ); +fn the_key_reader_and_the_oracle_agree_about_trailing_garbage() { + // **Two public readers disagreed about the same row.** `serde_json` stops + // at the end of the first value and does not care what follows, so without + // `Deserializer::end()` this row gave `keys` a clean `["reason"]` while + // `check` reported `Malformed { what: "trailing characters ..." }`. + // Measured before the fix, exactly that pair. A caller reading keys + // directly was told a malformed artifact was readable. Found by a review. + let row = r#"{"reason":"x-probe-topology"}garbage"#; + assert!(malformation(row).is_some(), "the oracle rejects it"); assert_eq!( - check(&report), - Vec::new(), - "the probe reported a failure whose MESSAGE mentions an alarm and a \ - verdict; neither is a line this renderer wrote" + keys(row), + Vec::::new(), + "and the key reader must not read it as though it were whole" ); -} -#[test] -fn a_real_alarm_is_still_read_when_the_renderer_writes_it() { - // The other direction, so the anchoring cannot quietly turn the alarm rule - // off. The renderer writes `BUG IN THIS PROBE` at the start of its own line. - let report = report_with_partitioning("summary_missing", "summary_missing"); - - assert!( - check(&report) - .iter() - .any(|violation| matches!(violation, Correspondence::AlarmWithAgreeingVerdict { .. })), - "an alarm the renderer itself wrote, beside an agreeing verdict, is still \ - a violation: {:#?}", - check(&report) + // The control: the same row WITHOUT the garbage is read normally, so the + // rule is not simply refusing everything. + assert_eq!( + keys(r#"{"reason":"x-probe-topology"}"#), + vec!["reason".to_owned()] ); } #[test] -fn a_caller_error_cannot_introduce_a_line_of_its_own() { - // The stronger form of the same defect, and the one that is worse than a - // false alarm. `report_unmeasured` interpolated the caller's `io::Error` - // VERBATIM when this was written, so an error carrying NEWLINES could put - // lines into the report that this renderer never wrote -- and anchoring to - // line starts does not help when the injected text starts its own line. +fn valid_json_that_is_not_an_object_is_a_malformation() { + // **Pins the `Map` in `malformation`, which the corpus cannot reach.** The + // generated corruptions are one-character mutations of a row, and none can + // turn an object into a valid NON-object -- so a regression from + // `serde_json::Map` to `serde_json::Value` would have left every test green. + // Reported by a review. // - // Stated in the past tense because the renderer has since been fixed: - // `renderer_owns_every_line` flattens the error, so it can no longer create - // a line at all. This test is what keeps that true, so it describes the - // regression it prevents rather than a hazard that is still open. - // - // Two of them, because they fail differently: - // - // `\nBUG IN THIS PROBE\n=> agree` invents an alarm beside a verdict; - // `\n{"cross_check":"agree"}` is selected as the machine-readable row, - // so the oracle checks the CALLER's text - // instead of the probe's. - // - // The renderer now flattens caller text, so neither can create a line. - // Found by a review, which was right that documenting the hole was not the - // same as closing it. - for injection in [ - "x\nBUG IN THIS PROBE\n=> agree", - "x\n{\"reason\":\"x-probe-topology\",\"arch\":\"aarch64\",\"cross_check\":\"agree\"}", - ] { - let report = crate::topology_report::report_unmeasured( - &host_banner("16p/8c"), - &std::io::Error::other(injection), - ); - - assert_eq!( - report.lines().filter(|line| line.starts_with('{')).count(), - 1, - "the report must carry exactly one machine-readable row, and it must \ - be the renderer's:\n{report}" - ); - assert_eq!( - check(&report), - Vec::new(), - "an error message is opaque payload, not the probe speaking:\n{report}" + // Called directly rather than through `check`, because `check` selects rows + // by a leading `{` and these never get that far: through the public path a + // bare list is `RowDefect::Missing`, not a malformed row. That makes the + // requirement defence in depth rather than a reachable case -- said plainly + // here, because the comment beside it reads as though `[1,2]` arrives, and + // the honest claim is that the type is what stops it ever mattering. + for not_an_object in [r#"[1,2]"#, "null", "3", r#""a string""#, "true"] { + assert!( + malformation(not_an_object).is_some(), + "{not_an_object} is valid JSON but carries no keys, so it is not a row" ); } -} - -#[test] -fn a_banner_cannot_introduce_a_line_of_its_own() { - // The banner is caller-supplied too, and reaches every report through - // `preamble` rather than only the unmeasured one. - let report = crate::topology_report::report_unmeasured( - &host_banner("16p/8c\n=> agree\nBUG IN THIS PROBE"), - &std::io::Error::other("a simulated failure"), - ); - assert_eq!( - check(&report), - Vec::new(), - "a banner cannot smuggle in a verdict or an alarm:\n{report}" - ); + // The control: the same call accepts an object, so the assertions above are + // not passing merely because `malformation` rejects everything. + assert_eq!(malformation(&clean_row()), None); } #[test] -fn an_attribution_banner_keeps_its_lines() { - // **The regression this pair exists for.** `attribution` renders two `host:` - // readings and a disclaimer when they differ, so the banner legitimately - // spans several lines. An earlier containment flattened it unconditionally: - // the disclaimer stopped being a line of its own, the oracle's exemption for - // it stopped firing, and the second reading vanished. - // - // Nothing caught it because this host's two readings agree, so every report - // rendered here carries a one-line banner -- the shape blindness this branch - // keeps paying for, this time in the renderer rather than an instrument. - // **Built rather than discovered.** This called `Fingerprint::discover()` and - // panicked if it failed -- on a crate whose whole point is that discovery can - // fail, and whose renderer has a dedicated arm for exactly that. A host that - // could not read its own topology would have failed this test for a reason it - // is not about. Found by a review. - // - // The architecture comes from the build so the banner agrees with the row; - // a literal would contradict it off that architecture, which is the same - // portability defect that cost five failures on `i686-pc-windows-msvc`. - let banner = crate::topology_report::attribution( - &Ok(built_fingerprint()), - &Err(std::io::Error::other("the second reading failed")), - ); - let text = crate::topology_report::report_unmeasured( - &banner, - &std::io::Error::other("a simulated failure"), - ); +fn a_mismatched_closing_delimiter_is_a_defect() { + // Also balanced by depth, also invalid: an object closed by a bracket. + let mismatched = r#"{"reason":"x-probe-topology","arch":"x86_64"]"#; - assert!( - text.lines() - .any(|line| line.starts_with("HOST NOT ESTABLISHED:")), - "the disclaimer must remain a line of its own, or the oracle's exemption \ - for it cannot fire:\n{text}" - ); assert_eq!( - text.lines() - .filter(|line| line.starts_with("host:")) - .count(), - 2, - "both bracket readings must survive as their own lines:\n{text}" - ); -} - -#[test] -fn a_banner_cannot_occupy_a_reserved_line_position() { - // The other half. The banner is a `&str` any caller can supply and it - // occupies the first line, so an arbitrary string could impersonate a line - // the renderer reserves -- which flattening newlines did not stop, because - // the banner IS a line. - // - // Measured before this: a banner of a whole NDJSON object gave the report - // TWO machine-readable rows and the oracle read the caller's rather than the - // renderer's; a banner of `=> agree` was read as a verdict and panicked a - // valid unmeasured report. - for impersonation in [ - r#"{"reason":"x-probe-topology","arch":"aarch64","cross_check":"agree"}"#, - "=> agree. Every check this probe could make was made and matched.", - "BUG IN THIS PROBE: pretending to be an alarm", - ] { - let text = crate::topology_report::report_unmeasured( - impersonation, - &std::io::Error::other("a simulated failure"), - ); - - assert_eq!( - text.lines().filter(|line| line.starts_with('{')).count(), - 1, - "exactly one machine-readable row, and it is the renderer's:\n{text}" - ); - assert_eq!( - check(&text), - Vec::new(), - "a banner names a machine; it cannot be a verdict, an alarm, or a \ - row:\n{text}" - ); - } -} - -#[test] -fn a_contained_banner_still_says_what_it_said() { - // **Containment must not be destruction, and an earlier test could not tell - // the difference.** It asserted the report was SAFE -- one machine-readable - // row, no violations -- which a containment that threw the banner away - // entirely also satisfies. Mutation testing found exactly that: replacing - // `renderer_owns_every_line` with `String::new()` or a constant survived, - // because nothing checked the banner still named the machine. - // - // A banner a reader cannot read is not a fixed banner. The first line has to - // remain the host's, whatever had to be done to make it safe to print. - let text = crate::topology_report::report_unmeasured( - "an-odd-machine\nrunning-something-unusual", - &std::io::Error::other("a simulated failure"), - ); - let first = text.lines().next().unwrap_or_default(); - - assert!( - first.starts_with("host:"), - "the first line is the host banner:\n{text}" - ); - for word in ["an-odd-machine", "running-something-unusual"] { - assert!( - first.contains(word), - "containment flattens the banner; it does not discard it -- {word} is \ - missing from {first:?}:\n{text}" - ); - } -} - -// --- what the mutation sweep found nothing pinned -------------------------- - -#[test] -#[should_panic(expected = "the report's parts contradict each other")] -fn assert_corresponds_panics_on_a_contradicting_report() { - // **The deepest thing nothing checked.** Every other instrument in this - // crate trusts `assert_corresponds`: the renderers are bound to it, the - // real-host test calls it, and the corpus reaches it by rendering. Replacing - // its body with `()` survived the mutation sweep -- because every test that - // would notice goes THROUGH it, so a no-op assertion makes them all pass. - // - // The one direction nothing could establish from the inside. - super::assert_corresponds( - &clean_report().replace("processors (online) : 16", "processors (online) : 8"), + check(&report_with(mismatched)), + vec![RowDefect::Malformed { + what: malformation_of(mismatched), + row: mismatched.to_owned() + }] ); } #[test] -fn assert_corresponds_accepts_a_report_that_agrees_with_itself() { - // The other half, so the fix above cannot be "always panic". - super::assert_corresponds(&clean_report()); -} - -#[test] -fn a_summary_missing_marker_with_no_level_number_names_no_level() { - // `(end > 0)` is what stops a marker with no digits after it reporting an - // EMPTY level as though it were one. Mutating it to `>=` survived, because - // every fixture puts a number there. - let report = report_with_partitioning("summary_missing", "summary_missing") - .replace("named L1 as the outermost", "named Lx as the outermost"); +fn a_nested_list_closed_as_an_object_is_a_defect() { + // The inner case, so the stack is shown to be a stack rather than a pair of + // counters that happen to agree at the end. + let mismatched = r#"{"reason":"x","caches":[{"level":1}}}"#; assert!( - !check(&report).iter().any(|violation| matches!( - violation, - Correspondence::ProseAndNdjsonDisagree { - fact: "summary-missing outermost level", - .. - } - )), - "a marker with no level number names no level, so there is nothing to \ - relate: {:#?}", - check(&report) + matches!( + check(&report_with(mismatched)).as_slice(), + [RowDefect::Malformed { .. }] + ), + "{:?}", + check(&report_with(mismatched)) ); } #[test] -fn an_anomaly_sentence_with_no_count_names_no_count() { - // The same guard in the anomaly reader, and the same reason it survived. - let report = clean_report() - .replace( - " => agree. Every check this probe could make was made and matched.", - " => INCOMPLETE. Nothing this probe compared disagreed, but this run\n\ - \x20 did not establish that the parse is consistent:\n\ - \x20 - windows-topology-sys recorded many enumeration anomalies", - ) - .replace(r#""cross_check":"agree""#, r#""cross_check":"incomplete""#) - .replace( - r#""parse_incomplete":0}"#, - r#""parse_incomplete":1,"not_compared":0,"enumeration_anomalies":3}"#, - ); +fn a_brace_inside_a_string_does_not_confuse_the_delimiter_stack() { + // The acceptance half of the stack: `discovery_error` carries an OS message, + // which may contain any delimiter. Mis-stacking those would report every + // such host as malformed. + let row = r#"{"reason":"x","discovery_error":"failed at {[ and never closed"}"#; - assert!( - !check(&report).iter().any(|violation| matches!( - violation, - Correspondence::ProseAndNdjsonDisagree { - fact: "enumeration anomaly count", - .. - } - )), - "a sentence with no number in it states no count: {:#?}", - check(&report) - ); + assert_eq!(check(&report_with(row)), Vec::new()); } #[test] -fn a_half_marker_is_not_a_marker() { - // A provenance marker is the SHAPE `!!...!!`, and both ends are required. - // Mutating the `&&` to `||` survived, because no fixture carried a token - // with only one end -- so nothing established that a half-marker is read as - // an ordinary token rather than skipped as decoration. - let report = clean_report().replace("host: x86_64 16p/8c", "host: !!SYNTHETIC 16p/8c"); +fn a_bracket_inside_a_string_does_not_unbalance_a_row() { + // **The acceptance half that matters most**, because the renderer does emit + // brackets inside strings: a failed discovery's `io::Error` is interpolated + // into a string value, and an OS message is free to contain one. A balance + // check that counted them would report every such host as malformed. + let with_brackets = r#"{"reason":"x-probe-topology","error":"failed at [0] {oops}"}"#; - assert!( - check(&report).contains(&Correspondence::ProseAndNdjsonDisagree { - fact: "architecture", - prose: "!!SYNTHETIC".to_owned(), - ndjson: "x86_64".to_owned(), - }), - "`!!SYNTHETIC` is not the marker shape, so it is where the architecture \ - should have been: {:#?}", - check(&report) - ); + assert_eq!(check(&report_with(with_brackets)), Vec::new()); } #[test] -fn a_string_field_is_read_with_its_quotes_when_the_container_matters() { - // `ndjson_raw_field` keeps the delimiters so a caller can tell a scalar from - // a one-element list. Its STRING branch had no exercise at all: the only - // caller asks about `efficiency_classes`, which is always an array, so - // mutating the `+ 2` that steps over both quotes survived twice. - let line = r#"{"reason":"x-probe-topology","arch":"x86_64","processors":16}"#; +fn an_escaped_quote_does_not_end_a_string() { + // The other half of the same hazard: a `\"` inside an error message would + // otherwise close the string early and put the rest of the message into + // bracket-counting. + let escaped = r#"{"reason":"x-probe-topology","error":"he said \"[\" and left"}"#; - assert_eq!(super::ndjson_raw_field(line, "arch"), Some("\"x86_64\"")); - assert_eq!(super::ndjson_raw_field(line, "processors"), Some("16")); - assert_eq!(super::ndjson_raw_field(line, "absent"), None); + assert_eq!(check(&report_with(escaped)), Vec::new()); } #[test] -fn a_marker_in_the_banner_is_not_the_probe_speaking() { - // **The banner is caller text, and containment keeps its CONTENT.** It is - // flattened onto one line beginning `host:`, which stops it impersonating a - // verdict, an alarm or a row -- but the words survive, so a reader that - // searches the whole report still finds a marker inside them. +fn an_escaped_quote_inside_a_value_does_not_forge_a_key() { + // **A correct report crashed the probe, and this is the shape that did it.** + // `keys` used `find('"')`, which takes `\"` for a terminator, so an escaped + // quote shifted where it thought strings began and ended and text INSIDE a + // value was emitted as a top-level key. Two equal ones read as a repeated + // key, and `assert_row_is_well_formed` panicked from inside `report_unmeasured`. // - // Two readers did. Measured before the fix: this banner yielded a - // summary-missing level of 99 against the row's 1, and the anomaly banner - // below yielded a count of 99 against the row's 0. Both are the oracle - // raising a violation about a report that does not contain the defect -- - // worse than missing one, because a false alarm sends a reader hunting a - // contradiction the probe never rendered. - let level = report_with_partitioning("level", "level").replace( - "host: x86_64 16p/8c", - "host: x86_64 16p/8c BUG IN THIS PROBE: the topology crate named L99 as the outermost", - ); - - assert_eq!( - check(&level), - [], - "a banner quoting the summary-missing sentence is not a summary-missing arm" + // Reachable, not hypothetical: `discovery_error` carries a failed + // discovery's `io::Error`, whose message is whatever the OS said. + let forged = concat!( + r#"{"reason":"x-probe-topology","arch":"x86_64","#, + r#""discovery_error":"q\":1,\"q\":1,\"q"}"# ); - let anomalies = clean_report() - .replace( - r#""parse_incomplete":0}"#, - r#""parse_incomplete":0,"not_compared":0,"enumeration_anomalies":0}"#, - ) - .replace( - "host: x86_64 16p/8c", - "host: x86_64 16p/8c windows-topology-sys recorded 99 enumeration anomalies", - ); - assert_eq!( - check(&anomalies), - [], - "a banner quoting the anomaly sentence is not a diagnostic entry" + keys(forged), + vec!["reason", "arch", "discovery_error"], + "the error's contents are a VALUE, however many quotes it contains" ); + assert_eq!(check(&report_with(forged)), Vec::new()); } #[test] -fn the_anchored_readers_still_read_the_lines_the_renderer_writes() { - // The acceptance half. Anchoring is only worth having if it still reads the - // real thing, and the two markers sit differently on their lines: the - // summary-missing sentence BEGINS its line, while `CrossCheck` writes its - // diagnostic entries as ` - `, so the anomaly marker never - // does. A reader anchored to line starts alone would have gone blind to the - // second while looking fixed. - let summary = report_with_partitioning("summary_missing", "summary_missing"); - - assert_eq!( - super::summary_missing_level(&summary), - Some("1"), - "the renderer's own summary-missing line is still read" - ); - - let bulleted = " => INCOMPLETE. Nothing this probe compared disagreed:\n\ - \x20 - windows-topology-sys recorded 2 enumeration anomalies, so what Windows returned"; +fn a_backslash_before_the_closing_quote_does_not_swallow_the_rest_of_the_row() { + // The other half: a value ending in an escaped backslash closes normally, + // so the keys after it are still found. Getting this wrong in the other + // direction would silently drop every key that follows. + let row = r#"{"reason":"x","path":"C:\\temp\\","cross_check":"agree"}"#; - assert_eq!( - super::anomaly_count_in_prose(bulleted), - Some("2"), - "the renderer's bulleted diagnostic entry is still read" - ); + assert_eq!(keys(row), vec!["reason", "path", "cross_check"]); + assert_eq!(check(&report_with(row)), Vec::new()); } #[test] -fn a_banner_cannot_state_a_disagreement_and_deny_it() { - // **A shape `attribution` cannot produce must not reach the reader.** - // `attribution` prints ONE `host:` line when the two bracket readings agree - // and TWO with a disclaimer when they do not; the disclaimer is the sentence - // that says which reading describes the body was not established. - // - // The recogniser used to accept any number of `host:` lines with the - // disclaimer optional, so two readings naming DIFFERENT machines passed - // through verbatim with nothing saying they conflicted. Measured then: that - // banner rendered beside an `"arch":"x86_64"` row and `check` returned no - // violations, because the architecture rule's exemption for an unestablished - // host covered a banner that had never claimed to be unestablished. - // Both readings name the BUILD's architecture, not a literal: the row - // publishes `std::env::consts::ARCH`, so a hard-coded one contradicts it off - // x86_64 and this test would fail for a reason it is not about. Measured - // while writing it -- the first draft said `aarch64` and the bound oracle - // fired on the contradiction rather than the shape. - let reading = format!("host: {} 16p/8c", std::env::consts::ARCH); - let two_readings = crate::topology_report::report_unmeasured( - &format!("{reading}\n{reading}"), - &std::io::Error::other("a simulated failure"), - ); +fn a_repeated_key_is_a_defect() { + // Not a parse error in most readers -- they take the last -- so this is + // precisely the malformation that survives a consumer's parse and changes + // what it reads. + let repeated = r#"{"reason":"x-probe-topology","processors":16,"processors":8}"#; - assert!( - two_readings.starts_with(&format!("{reading} {reading}\n")), - "an unattributable banner is contained onto one line, not trusted: {two_readings}" + assert_eq!( + check(&report_with(repeated)), + vec![RowDefect::RepeatedKey { + key: "processors".to_owned() + }] ); - - // The two readings agree here so containment cannot manufacture an - // architecture contradiction, which is what lets this test assert the SHAPE - // on its own. The contradicting case is the one the bound oracle now - // catches. - // - // The acceptance half -- that a banner `attribution` really did write still - // passes through with its lines intact -- is - // `an_attribution_banner_keeps_its_lines`, which renders the two-readings - // -and-a-disclaimer shape and asserts both the disclaimer line and the host - // line count. Tightening the cardinality wrongly would turn that test red, - // so it is not restated here. } #[test] -fn a_banner_no_one_wrote_in_ascii_does_not_panic_the_oracle() { - // **`check` is public and the banner is caller text, so a slice that is - // wrong on a multi-byte char is a panic out of the oracle rather than a - // violation.** Containment neutralises `\n` and `\r` only; every other byte - // reaches the readers. Measured before the fix, on the banner below: - // `start byte index 8 is not a char boundary; it is inside '-'` raised from - // `processors_in_banner`. - // - // A localised `io::Error` interpolated by `banner_line_for` is the - // plausible route on a non-English host, so this is not only a fuzzing - // curiosity. - let report = clean_report().replace("host: x86_64 16p/8c", "host: \u{2013}16p/8c"); - - // The banner's count is now unreadable as a number, which is a - // disagreement with the body and not an error -- what matters here is that - // the oracle ANSWERS instead of unwinding. - let violations = check(&report); - - assert!( - violations - .iter() - .all(|found| !matches!(found, Correspondence::AlarmWithAgreeingVerdict { .. })), - "the odd banner must not be read as an alarm: {violations:?}" - ); +fn a_key_repeated_inside_a_nested_object_is_not_the_rows_key() { + // Top level only: a nested object's members are that object's keys, and + // repeating one there is a different question. `policies` renders arbitrary + // policy names, so a name colliding with a row key is possible. + let nested = r#"{"reason":"x-probe-topology","processors":16,"policies":{"processors":2}}"#; - // The same char in every other position the readers touch. - for banner in [ - "host: \u{2013} 16p/8c", - "host: x86_64 16p/8c \u{2013}", - "\u{2013}", - "host: \u{65e5}16p/8c", - ] { - let report = clean_report().replace("host: x86_64 16p/8c", banner); - let _ = check(&report); - } + assert_eq!(check(&report_with(nested)), Vec::new()); } #[test] -fn the_anomaly_count_is_read_under_a_disagreeing_verdict_too() { - // **The entry's tag depends on the verdict.** `CrossCheck` writes - // `parse_incomplete` entries as `- {caveat}` under `INCOMPLETE`, but under - // `DISAGREE` as `(parse incomplete) {caveat}` so a reader can tell the - // disagreement from what was merely not established. The reader anchored to - // the bullet alone, so on a disagreeing report the anomaly sentence was - // never found and the two renderings of the count went uncompared -- - // directly contradicting the comment above the rule, which says it is read - // for every verdict. - // - // Measured before the fix: the `(parse incomplete) ` rendering returned - // `None` where the `- ` rendering returned `Some("2")`. - let entry = "windows-topology-sys recorded 2 enumeration anomalies, so what Windows returned"; - - for line in [ - format!(" - {entry}"), - format!(" (parse incomplete) {entry}"), - format!(" (not compared) {entry}"), - ] { - assert_eq!( - super::anomaly_count_in_prose(&line), - Some("2"), - "every tag the renderer can write must leave the sentence readable: {line}" - ); - } +fn the_rows_keys_are_read_at_the_top_level_only() { + let row = clean_row(); - // The rejection half stays intact: a tag is renderer-owned because a - // contained banner is one line beginning `host:`, so caller text cannot - // present one. assert_eq!( - super::anomaly_count_in_prose(&format!("host: x86_64 16p/8c {entry}")), - None, - "a banner quoting the sentence is still not a diagnostic entry" + keys(&row), + vec![ + "reason", + "arch", + "processors", + "efficiency_classes", + "caches", + "policies", + "cross_check", + "disagreements", + "parse_incomplete", + ], + "the nested `level`, `domains` and `single` are not the ROW's keys" ); } #[test] -fn a_failed_discovery_banner_is_not_read_as_a_fingerprint() { - // **A banner only has to MENTION `p/` for a search to find a fingerprint in - // text that is not one.** `banner_line_for` renders a failed read as - // `host: UNKNOWN -- topology discovery failed: {error}` with the - // `io::Error` verbatim -- not as the bare word `UNKNOWN`, which is what this - // module's doc claimed until this test was written. +fn the_row_accessor_declines_every_report_the_oracle_faults() { + // **The accessor and the oracle answer one question, so they may not + // disagree.** `row` ran its own subset -- one row, and `malformation` -- and + // `serde_json` accepts a duplicated key, so a report `check` faulted as + // `RepeatedKey` was handed back here as readable. A caller asking "may I + // read this row" got yes for a row already judged ambiguous. // - // So an error text of `16p/foo something opaque` made the count reader - // answer `16`, which satisfied the guard, and the architecture reader then - // answered `UNKNOWN` against a real `arch`. Measured before the fix: the - // bound assertion PANICKED on a valid unmeasured report -- the probe - // crashing on the host whose discovery failed, which is the host it exists - // to report. - let error = std::io::Error::other("16p/foo something opaque"); - let banner = windows_placement_probe::fingerprint::banner_line_for(&Err( - std::io::Error::other("16p/foo something opaque"), - )); - - assert!( - banner.contains("p/"), - "this test is pointless unless the error text reaches the banner: {banner}" - ); - assert_eq!( - super::processors_in_banner(&banner), - None, - "a failed read names no fingerprint, whatever its error text spells" - ); - assert_eq!(super::architecture_in_banner(&banner), None); - - // End to end: rendering asserts on its own output under this build, so a - // false violation here is a panic rather than a return value. - let text = crate::topology_report::report_unmeasured(&banner, &error); - - assert!(text.contains("UNKNOWN"), "{text}"); - - // The acceptance half: a real fingerprint is still read. - let real = host_banner("16p/8c"); - - assert_eq!(super::processors_in_banner(&real), Some("16")); - assert_eq!( - super::architecture_in_banner(&real), - Some(std::env::consts::ARCH) - ); - - // And a taint marker still does not displace the two tokens. - let tainted = format!("host: !!SYNTHETIC!! {} 16p/8c", std::env::consts::ARCH); - - assert_eq!( - super::architecture_in_banner(&tainted), - Some(std::env::consts::ARCH) - ); -} - -#[test] -fn a_nested_list_does_not_compare_equal_to_a_flat_one() { - // `trim_matches` removes EVERY consecutive bracket, so `[0]` and `[[0]]` - // both normalised to `0` and a renderer that regressed to a nested array - // beside one-level prose would have compared equal. The punctuation this is - // meant to forgive is `[0, 1]` against `0,1`; a difference in DEPTH is a - // real disagreement. - assert_eq!( - super::normalise_list("[0, 1]"), - super::normalise_list("0,1") - ); - assert_eq!(super::normalise_list("[0]"), "0"); - assert_ne!( - super::normalise_list("[[0]]"), - super::normalise_list("[0]"), - "a nested list is structurally different and must not normalise away" - ); -} - -#[test] -fn a_caveat_in_the_banner_does_not_excuse_an_uncaveated_claim() { - // **The caveat is matched mid-line, so it is the one search here that - // cannot anchor to a line start -- and it is the dangerous direction.** A - // claim found where none was made invents a violation; a caveat found where - // none was made SUPPRESSES one. Measured before the fix: appending the - // caveat sentence to the banner made `UncaveatedClaimUnderDoubt` vanish from - // a report that still carried the claim and still said `parse_incomplete=2`. - let claimed = clean_report() - .replace(r#""parse_incomplete":0}"#, r#""parse_incomplete":2}"#) - .replace( - " efficiency classes: [0]", - " efficiency classes: [0, 1]\n (heterogeneous: an I/O thread left unconstrained can land on an", - ); - - let uncaveated = |text: &str| { - check(text) - .iter() - .any(|found| matches!(found, Correspondence::UncaveatedClaimUnderDoubt { .. })) - }; - - assert!( - uncaveated(&claimed), - "the claim is bare and the parse is in doubt: {:#?}", - check(&claimed) - ); - - let banner_says_it = claimed.replace( - "host: x86_64 16p/8c", - "host: x86_64 16p/8c This run did not establish that the parse is whole", - ); - - assert!( - uncaveated(&banner_says_it), - "a caveat the RENDERER did not write must not excuse the claim: {:#?}", - check(&banner_says_it) - ); - - // The acceptance half: the renderer's own caveat still excuses it. - let properly_caveated = claimed.replace( - " (heterogeneous: an I/O thread left unconstrained can land on an", - " (heterogeneous: an I/O thread left unconstrained can land on an\n (This run did not establish that the parse is whole, and the classes", - ); - - assert!( - !uncaveated(&properly_caveated), - "the caveat beside the claim is what the rule exists to accept: {:#?}", - check(&properly_caveated) - ); -} + // Stated as the correspondence rather than as the one case, because the + // gap was not in the case anyone wrote down: it was in the SECOND + // implementation existing at all. Any future defect `check` learns is + // covered here without a new test. + let faulted = [ + report_with(r#"{"reason":"x","processors":4,"processors":8}"#), + report_with(r#"{"reason":"x","arch":"x86_64",}"#), + report_with(r#"{"reason":"x","arch":"x86_64"]"#), + report_with(r#"{"unclosed":["#), + "a report with no row at all".to_owned(), + format!( + "{}\n{}", + r#"{"reason":"x","arch":"x86_64"}"#, r#"{"reason":"y","arch":"x86"}"# + ), + ]; -#[test] -fn the_cpu_sets_count_is_selected_by_what_the_line_says_not_by_its_position() { - // **This guards an order dependency, not a defect that was live.** A review - // reported that the heterogeneity line shadows this one, because both open - // ` (` and the reader took the FIRST such line. Checked against the - // renderer: the CPU-Sets line is written in the NUMA block - // (`topology_report.rs:271`) and the heterogeneity line in the - // efficiency-class block (`:295`), so the CPU-Sets line comes first and the - // finding does not reproduce. A fixture with both, in the renderer's order, - // passes under the old reader too -- measured. - // - // The reader is still fixed, because ` (` is not a label. It opens ANY - // parenthesised continuation, so the old code was correct only by the - // accident that nothing else opens one above this. The failure mode that - // accident was hiding is silent: a new continuation added above would make - // the rule read the wrong line, find no CPU-Sets text, and compare nothing - // -- with no error and no sign that a fact had stopped being checked. - // - // So the shape below is one the renderer cannot currently emit. It is the - // only shape that can distinguish selecting by CONTENT from selecting by - // POSITION, which is the property actually being asserted. - let report = clean_report() - .replace( - "packages : 1", - "packages : 1\n (a continuation this renderer does not write today)", - ) - .replace( - "NUMA domains : 1 (0 with no processors)", - "NUMA domains : 1 (0 with no processors)\n (5 reported only by CPU Sets, never by the relationship walk:", - ) - .replace( - r#""numa_domains_without_processors":0"#, - r#""numa_domains_without_processors":0,"numa_domains_only_in_cpu_sets":9"#, + for report in &faulted { + let defects = check(report); + assert!( + !defects.is_empty(), + "the fixture must be faulted for this to mean anything: {report}" ); - - assert!( - check(&report).iter().any(|found| matches!( - found, - Correspondence::ProseAndNdjsonDisagree { - fact: "NUMA domains reported only by CPU Sets", - .. - } - )), - "the prose says 5 and the body 9, below an unrelated continuation: {:#?}", - check(&report) - ); - - // The renderer's real order still reads, so the fix did not trade one - // position dependency for another. - let as_rendered = clean_report() - .replace( - "NUMA domains : 1 (0 with no processors)", - "NUMA domains : 1 (0 with no processors)\n (5 reported only by CPU Sets, never by the relationship walk:", - ) - .replace( - " efficiency classes: [0]", - " efficiency classes: [0, 1]\n (heterogeneous: an I/O thread left unconstrained can land on an", - ) - .replace( - r#""numa_domains_without_processors":0"#, - r#""numa_domains_without_processors":0,"numa_domains_only_in_cpu_sets":5"#, + assert_eq!( + row(report), + None, + "`check` reports {defects:?} and `row` handed the row back anyway" ); + } + // And the other direction, or the rule is satisfied by refusing everything. + let clean = report_with(&clean_row()); + assert!(check(&clean).is_empty()); assert!( - !check(&as_rendered).iter().any(|found| matches!( - found, - Correspondence::ProseAndNdjsonDisagree { - fact: "NUMA domains reported only by CPU Sets", - .. - } - )), - "both say 5, so there is nothing to report: {:#?}", - check(&as_rendered) + row(&clean).is_some(), + "a clean report must still be readable" ); } #[test] -fn a_dropped_counterpart_is_a_violation_and_not_silence() { - // **Every comparison here was "both sides present, do they match", so a - // rendering that DROPPED a field read as silence.** The module's rule that - // an omitted fact is not a violation is about facts the report never - // mentions; once the prose states one, the report has made a claim the - // other rendering is required to answer. - // - // Measured before this: each of the three deletions below left a report the - // oracle accepted, with the prose still making all three claims. A mining - // pass reading such a row gets no value and no warning. - let named = |text: &str, wanted: &str| { - check(text).iter().any(|found| { - matches!(found, Correspondence::RenderedOnlyInProse { fact, .. } if *fact == wanted) - }) - }; - - // A single count, from the four the prose and the row both carry. - let no_processors = clean_report().replace(r#""processors":16,"#, ""); - - assert!( - named(&no_processors, "online processors"), - "the prose still says 16: {:#?}", - check(&no_processors) - ); +fn the_row_accessor_declines_an_ambiguous_or_malformed_report() { + let one = report_with(&clean_row()); + let two = format!("{}\n{}", report_with(&clean_row()), clean_row()); + let malformed = report_with(r#"{"unbalanced":["#); - // A whole CONTAINER, which the member-level tests never covered: they check - // an entry going missing from the object, not the object from the row. - let no_policies = clean_report().replace(r#""policies":{"single":1,"by-core":8},"#, ""); - - assert!( - named(&no_policies, "policy names"), - "the prose table still stands: {:#?}", - check(&no_policies) - ); - - // The discriminator, whose early return excused a measured report that - // printed an arm and lost it. - let no_discriminator = clean_report().replace(r#""outermost_partitioning_cache":"level","#, ""); - - assert!( - named(&no_discriminator, "outermost partitioning answer"), - "the prose still announces the level arm: {:#?}", - check(&no_discriminator) - ); - - // **The acceptance half, and the reason no exemption was needed.** - // `report_unmeasured` renders neither side of the TOPOLOGY facts deleted - // above, so it makes no prose claim for those missing fields to leave - // unanswered -- the shape that would have forced a special case if the rule - // had been keyed to the field instead. - // - // Narrowed deliberately: the short object DOES publish `arch`, and the - // banner can name the same architecture, so that correspondence is rendered - // twice here and is checked. "Renders neither side" would have been a - // tidier sentence and a false one. - let unmeasured = crate::topology_report::report_unmeasured( - &host_banner("16p/8c"), - &std::io::Error::other("a simulated failure"), - ); - - assert_eq!( - check(&unmeasured), - [], - "a report that claims nothing cannot leave a claim unanswered" - ); + assert_eq!(row(&one), Some(clean_row().as_str())); + assert_eq!(row("prose only"), None); + assert_eq!(row(&two), None); + assert_eq!(row(&malformed), None); } #[test] -fn a_multiline_discovery_error_does_not_cost_the_disclaimer_its_own_line() { - // **A banner line has to be a line.** `banner_line_for` interpolates a - // failed read's `io::Error` verbatim and an OS error may contain a newline, - // so a single READING could arrive as two lines. `attribution` then composes - // a banner with more lines than readings, `is_attribution_shaped` stops - // recognising the renderer's own output, and `preamble` contains the whole - // value -- taking the renderer-owned disclaimer down with it. +fn a_defect_is_reported_once_per_extra_rendering_of_a_key() { + // **One entry per EXTRA rendering, so the count reads as how many times the + // row said it again.** Three renderings of `a` give two defects, not one and + // not three. // - // Measured before the fix: a two-line error gave a six-line attribution and - // a report with no `HOST NOT ESTABLISHED:` line at all, so the oracle's - // exemption for an unestablished host stopped firing on a report that had - // legitimately earned it. - let error = || std::io::Error::other("line one\nline two"); - let banner = crate::topology_report::attribution(&Err(error()), &Err(error())); - - assert_eq!( - banner.lines().count(), - 4, - "two readings and a two-line disclaimer, whatever the OS wrote: {banner}" - ); + // This was named `..._once_per_repeated_key_rather_than_per_occurrence` and + // opened by claiming de-duplication the code does not do -- while asserting + // the per-occurrence behaviour its own failure message describes. A reader + // taking the name for the contract got it backwards. The assertion was + // right; the name and the comment were the defect. + let thrice = r#"{"a":1,"a":2,"a":3,"b":1,"b":2}"#; - let text = crate::topology_report::report_unmeasured(&banner, &error()); - - assert!( - text.lines() - .any(|line| line.starts_with("HOST NOT ESTABLISHED:")), - "the disclaimer must survive as a line of its own: {text}" - ); assert_eq!( - text.lines() - .filter(|line| line.starts_with("host:")) - .count(), - 2, - "one line per reading, not one per line of error text: {text}" + check(&report_with(thrice)), + vec![ + RowDefect::RepeatedKey { + key: "a".to_owned() + }, + RowDefect::RepeatedKey { + key: "a".to_owned() + }, + RowDefect::RepeatedKey { + key: "b".to_owned() + }, + ], + "one entry per EXTRA rendering, so the count reads as how many times the \ + row said it again" ); } -#[test] -fn a_cache_object_is_read_however_its_members_are_written() { - // **The lookup matched `"level":N,"domains":` as one literal**, which - // requires the two members to be adjacent and in that order. So renaming or - // moving `domains` made the lookup miss, `compare` was never reached, and - // the prose domain count went unchecked -- while `cache_levels` still found - // every level and reported membership as agreeing, which is what made the - // report look whole. - let with_cache = |object: &str| clean_report().replace(r#"{"level":1,"domains":8}"#, object); - - let names = |text: &str, fact: &str| { - check(text).iter().any(|found| { - matches!(found, Correspondence::RenderedOnlyInProse { fact: named, .. } if *named == fact) - || matches!(found, Correspondence::ProseAndNdjsonDisagree { fact: named, .. } if *named == fact) - }) - }; +/// Every single-character corruption of `row`, as (what was done, the result). +/// +/// Deletion and structural substitution, which between them reach the defects a +/// writer actually produces: a lost delimiter, a doubled separator, a `:` where +/// a `,` belonged, a quote that ends a string early. +fn corruptions(row: &str) -> Vec<(String, String)> { + const STRUCTURAL: [char; 8] = ['{', '}', '[', ']', ',', ':', '"', '\\']; + let mut out = Vec::new(); + + for (at, character) in row.char_indices() { + let after = at + character.len_utf8(); + + let mut deleted = String::with_capacity(row.len()); + deleted.push_str(&row[..at]); + deleted.push_str(&row[after..]); + out.push((format!("deleted {character:?} at {at}"), deleted)); + + for replacement in STRUCTURAL { + if replacement == character { + continue; + } + let mut swapped = String::with_capacity(row.len() + 1); + swapped.push_str(&row[..at]); + swapped.push(replacement); + swapped.push_str(&row[after..]); + out.push(( + format!("replaced {character:?} at {at} with {replacement:?}"), + swapped, + )); + } + } - // The member is gone: the object is present and cannot answer, which is a - // dropped counterpart rather than a membership problem. - assert!( - names( - &with_cache(r#"{"level":1,"x-domains":8}"#), - "cache domain count" - ), - "a renamed member leaves the prose count with nothing to agree with: {:#?}", - check(&with_cache(r#"{"level":1,"x-domains":8}"#)) - ); + out +} + +#[test] +fn every_unparseable_row_reaches_the_caller_as_a_defect() { + // **This is a WIRING test, and saying so matters.** An earlier version of it + // compared `malformation` against `serde_json` and was worth running, + // because `malformation` was then a hand-written check that could disagree + // -- it generated 1807 corruptions and found 159 disagreements, every one a + // false accept, which is why the hand-written version is gone. + // + // With the parse itself delegated, that comparison would be `serde_json` + // against `serde_json`: green by construction, and exactly the kind of + // tautology this crate keeps having to delete. So the question changed. It + // is no longer "is the verdict right" -- nothing here is entitled to an + // opinion on that -- but "does the verdict REACH the caller", which is a + // property of `check` and is not guaranteed by anything upstream. A `check` + // that dropped the result, or looked at the wrong line, would still be + // delegating to a correct parser. + let clean = clean_row(); + assert!( + serde_json::from_str::(&clean).is_ok(), + "the fixture must be valid JSON before corrupting it means anything" + ); + + // A corruption that destroys the LEADING BRACE is a different finding, and + // both halves are asserted rather than one being waved through. `check` + // selects the row by `starts_with('{')`, so such a line is not a malformed + // row -- it is not a row, and the report has none. `Missing` is the right + // answer there and `Malformed` would be the wrong one, because a survey's + // question is "did this host report a row", not "was the text well-formed". + // Measured: exactly the 8 corruptions of position 0, which is what makes the + // two branches worth separating instead of accepting any defect at all. + let mut escaped = Vec::new(); + let cases = corruptions(&clean); + for (what, candidate) in &cases { + if serde_json::from_str::(candidate).is_ok() { + continue; + } + let defects = check(&report_with(candidate)); + let expected = if candidate.starts_with('{') { + defects + .iter() + .any(|defect| matches!(defect, RowDefect::Malformed { .. })) + } else { + defects.contains(&RowDefect::Missing) + }; + if !expected { + escaped.push(format!("{what}: got {defects:?}")); + } + } - // **Reordering alone is NOT a contradiction**, so it cannot distinguish a - // reader that handles it from one that is blind -- both say nothing. The - // case that separates them reorders AND disagrees. - assert_eq!( - check(&with_cache(r#"{"domains":8,"level":1}"#)), - [], - "member order is not a disagreement; the value still agrees" - ); assert!( - names( - &with_cache(r#"{"domains":9,"level":1}"#), - "cache domain count" - ), - "reordered members must still be READ, which only a wrong value can show: {:#?}", - check(&with_cache(r#"{"domains":9,"level":1}"#)) - ); - - // And an object for a level the prose does not list is membership's - // business, not this rule's -- `"level":1` must not match level 10. - assert_eq!( - check(&with_cache( - r#"{"level":1,"domains":8},{"level":10,"domains":4}"# - )) - .iter() - .filter(|found| matches!( - found, - Correspondence::ProseAndNdjsonDisagree { - fact: "cache domain count", - .. - } - )) - .count(), - 0, - "an extra level is a membership finding, reported by level number" + escaped.is_empty(), + "{} of {} corruptions are unparseable and were not reported as the \ + defect they are:\n{}", + escaped.len(), + cases.len(), + escaped.join("\n") ); } #[test] -fn a_disclaimer_welded_to_a_reading_is_not_attribution_shaped() { - // **The recogniser matched the disclaimer as a SUFFIX, which says nothing - // about whether it starts a line.** `attribution` always writes it after a - // newline, but `trim_end_matches('\n')` accepted however many newlines it - // found -- including none -- so a caller could weld the disclaimer onto the - // second reading and have the whole thing passed through verbatim. - // - // Measured before the fix: the report's second line came out as - // `host: 16p/8cHOST READINGS DISAGREE: ...`, which this renderer - // cannot produce. Containment exists to stop caller text occupying a line - // the renderer reserves; trusting an unproducible shape hands it one. - let reading = format!("host: {} 16p/8c", std::env::consts::ARCH); - let welded = format!( - "{reading}\n{reading}HOST READINGS DISAGREE: the two readings above bracket the measurement\n\ - and differ, so which of them names the machine the body below describes\n\ - was not established." - ); - - let text = crate::topology_report::report_unmeasured( - &welded, - &std::io::Error::other("a simulated failure"), - ); +fn the_corruption_generator_reaches_defects_of_every_kind() { + // **A generator that produced nothing, or only legal strings, would leave the + // test above vacuous and green.** A generator cannot report the shape it + // never reaches, so it has to be asked what it reached. + let cases = corruptions(&clean_row()); + assert!(cases.len() > 500, "only {} corruptions", cases.len()); - // **Containment means ONE LINE, and that is what to assert.** A first - // version asserted the welded substring was gone, which it is not and should - // not be: flattening replaces newlines, and the weld had none, so the two - // stay adjacent. What changes is that the whole banner now occupies a single - // line the renderer prefixed, instead of contributing three lines of its own - // with a disclaimer that looks renderer-owned. - let banner_lines = text - .lines() - .take_while(|line| !line.starts_with("== processor topology")) + let unparseable = cases + .iter() + .filter(|(_, candidate)| serde_json::from_str::(candidate).is_err()) .count(); - - assert_eq!( - banner_lines, 1, - "a welded disclaimer must be contained onto one line, not trusted as \ - three:\n{text}" - ); assert!( - !text - .lines() - .any(|line| line.starts_with("HOST READINGS DISAGREE")), - "and it must not be left standing as a renderer-owned disclaimer:\n{text}" - ); - - // The acceptance half: what `attribution` really writes still passes - // through with its lines intact, which is what - // `an_attribution_banner_keeps_its_lines` asserts in full. Checked here too - // because the fix tightened the very predicate that test depends on. - let genuine = crate::topology_report::attribution( - &Ok(built_fingerprint()), - &Err(std::io::Error::other("the second reading failed")), - ); - let rendered = crate::topology_report::report_unmeasured( - &genuine, - &std::io::Error::other("a simulated failure"), + unparseable > 100, + "only {unparseable} of {} corruptions are unparseable, so the wiring \ + test is mostly skipping its own body", + cases.len() ); + let parseable = cases.len() - unparseable; assert!( - rendered.starts_with(&format!("{genuine}\n")), - "a banner attribution really did write must still pass through:\n{rendered}" - ); -} - -#[test] -fn an_agreeing_verdict_must_show_the_counters_it_claims_to_have_checked() { - // **`agree` is a claim about what the run DID, not only about what - // matched.** `CrossCheck` reports it to mean every check this probe could - // make was made and matched -- so a report that agrees while omitting the - // line a check reads contradicts its own verdict, even though the halves it - // still renders agree perfectly. - // - // The rule was written to compare two present values and to say nothing - // otherwise, which is the dropped-counterpart shape in the one place where - // the VERDICT is what the missing side contradicts. Measured before the fix: - // deleting the counter line left `check` returning nothing at all. - for label in [ - " GetActiveProcessorCount : ", - " GetActiveProcessorGroupCount: ", - ] { - let line = clean_report() - .lines() - .find(|line| line.starts_with(label)) - .map(str::to_owned) - .unwrap_or_else(|| { - panic!("the fixture must carry {label:?} for this to mean anything") - }); - let without = clean_report().replace(&format!("{line}\n"), ""); - - assert_ne!(without, clean_report(), "the removal must apply: {label:?}"); - assert!( - without.contains(r#""cross_check":"agree""#), - "the verdict must still claim agreement, or there is nothing to \ - contradict: {label:?}" - ); - assert!( - check(&without).iter().any(|found| matches!( - found, - Correspondence::EvidenceMissingWithAgreeingVerdict { .. } - )), - "a counter the verdict claims to have checked is missing, and the \ - oracle accepted it: {:#?}", - check(&without) - ); - } - - // The acceptance half: the untouched fixture renders both counters and is - // accepted, so the rule fires on absence rather than on everything. - assert_eq!( - check(&clean_report()), - [], - "a report that shows its counters must still be accepted" + parseable > 0, + "every corruption is unparseable, so a `check` that reported `Malformed` \ + unconditionally would satisfy the wiring test" ); } diff --git a/crates/windows-platform-probes/src/row.rs b/crates/windows-platform-probes/src/row.rs new file mode 100644 index 000000000..ea6018d82 --- /dev/null +++ b/crates/windows-platform-probes/src/row.rs @@ -0,0 +1,327 @@ +// Copyright (c) Mike Grier. + +//! The machine-readable row, as a value with one writer. +//! +//! # Why this exists +//! +//! The row was built by interpolating every value positionally into a `concat!` +//! template. Two defect classes follow from that construction, and both are +//! closed by replacing it rather than by checking it -- see +//! [DESIGN-NOTES.md](../DESIGN-NOTES.md#d-encoded-row-is-the-contract). +//! +//! **Injection.** Caller text reaching the mined artifact is contamination of +//! the contract. Measured on PR #88: an `io::Error` whose text contained `{` was +//! selected as the report's machine-readable row, so a reader checked the +//! caller's text instead of the probe's. A [`Value::Text`] is escaped by the +//! writer, so no string it carries can end the string it is in, let alone start +//! a new row. +//! +//! **Field order and labelling.** A field's name and its value were related only +//! by counting positions, so a reordered argument or a miscounted placeholder +//! yielded mislabelled data that still parses and that nothing downstream can +//! detect. Here a name and its value are one pair, moved together or not at all. +//! +//! # What [`Row::keys`] derives, and what it cannot +//! +//! It reads back the names a caller actually supplied, which is what the +//! writer's own tests need: a row renders the members it was given, in order, +//! and that property is derivable rather than restated. +//! +//! **It is NOT the contract, and this section used to say it was.** The claim +//! here was that the well-formedness check "no longer needs a list of expected +//! keys written beside it". That is false, and falsifiably so: `keys` reports +//! what the builder happened to supply, so a row missing a required field is +//! still perfectly self-consistent. Measured -- with `.with("packages", ...)` +//! deleted from the renderer, the whole suite stayed green. +//! +//! The contract is `topology_report::MEASURED_ROW_KEYS` and its `_SHAPES` +//! sibling, owned by the renderer that owes those fields and stated +//! independently of it. A schema is not derivable from the thing it constrains; +//! the anti-census rule is about facts that CAN be derived, and this is not one. +//! Reported by a review, which found this passage still steering a reader back +//! toward the vacuous check. + +use std::fmt::Write as _; + +#[cfg(test)] +mod tests; + +/// The shape a row's value must have, as a schema states it. +/// +/// **The type-level counterpart of [`Value`], and the half the key list was +/// missing.** A schema of names alone pins WHICH fields a row carries and says +/// nothing about what they hold, so a renderer could publish `"processors"` as +/// a string and satisfy every check the crate had. Measured, before this +/// existed: `.with("processors", observation.online_processors.to_string())` +/// left all 230 library tests and all 10 real-host integration tests green. +/// Reported by a review. +/// +/// **Stated independently rather than derived from the renderer**, which is the +/// same reasoning as the diagnostic goldens: a shape read back out of the +/// `Value` the renderer produced would move whenever the renderer moved, and so +/// could never disagree with it. A schema is not derivable from the thing it +/// constrains -- writing it down twice is what makes it a schema rather than a +/// restatement. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum Shape { + /// A JSON string. + Text, + /// A JSON number. + Number, + /// A number, or `null` where the answer is genuinely "there is none". + NumberOrNull, + /// A list whose every element is a number. + ListOfNumbers, + /// A list whose every element is an object carrying at least these members, + /// each with the shape named beside it. + /// + /// **Recursive, because "a list of objects" was not a contract.** That is + /// what this replaced, and `[{}]` satisfied it -- so `caches` could stop + /// publishing `level` and `domains` with the shape oracle green, and a + /// diagnostic entry could lose the `code` a survey groups by. Reported by a + /// review, and measured: an empty cache object produced zero violations. + /// + /// Extra members are allowed. What a schema owes a consumer is that the + /// fields it promises are present and typed; forbidding additions would + /// make every new field a breaking change to the checker rather than to the + /// contract. + ListOfObjectsWith(&'static [(&'static str, Shape)]), + /// An object whose every member is a number. + ObjectOfNumbers, +} + +/// A value the row can carry. +/// +/// Deliberately not every JSON shape: there is no floating point, because every +/// quantity this crate publishes is a count, an identifier or a list of them, +/// and a float in a mined artifact invites a consumer to compare values that +/// were never measured that precisely. +#[derive(Clone, Debug, PartialEq, Eq)] +pub enum Value { + /// A string, escaped on the way out. + Text(String), + /// A count. + Number(usize), + /// The absent case, for a field whose answer is "there is none". + /// + /// A variant rather than an omitted key, because a consumer can tell `null` + /// from a field this probe is too old to publish, and cannot tell an + /// omission from either. + Null, + /// An ordered list. + List(Vec), + /// An ordered set of named members. + Object(Vec<(&'static str, Value)>), +} + +/// Panics if any object anywhere inside `value` repeats a member name. +/// +/// **Every level, because the row's guarantee is about the artifact, not about +/// its first level.** `Row::with` rejects a repeat among the row's own members; +/// this is the same rule applied to what those members contain. Without it a +/// diagnostic entry could render `{"code":"a","code":"b"}` -- measured, exactly +/// that -- which `serde_json` and `JSON.parse` both accept while keeping one +/// value, and which the oracle cannot report either, because `keys` reads +/// top-level names by design. Reported by a review. +/// +/// Uniqueness is PER OBJECT, not across the row: every diagnostic entry carries +/// its own `code`, and a rule that forbade that would reject every real report. +fn assert_unique_names(at: &str, value: &Value) { + match value { + Value::Object(members) => { + let mut seen: Vec<&str> = Vec::new(); + for (name, held) in members { + assert!( + !seen.contains(name), + "the object at `{at}` already carries `{name}`, and a repeated \ + key survives a consumer's parse as whichever value happened \ + to come last" + ); + seen.push(name); + assert_unique_names(name, held); + } + } + Value::List(entries) => { + for entry in entries { + assert_unique_names(at, entry); + } + } + Value::Text(_) | Value::Number(_) | Value::Null => {} + } +} + +impl From<&str> for Value { + fn from(text: &str) -> Self { + Self::Text(text.to_owned()) + } +} + +impl From for Value { + fn from(text: String) -> Self { + Self::Text(text) + } +} + +impl From for Value { + fn from(count: usize) -> Self { + Self::Number(count) + } +} + +impl From for Value { + fn from(count: u8) -> Self { + Self::Number(count as usize) + } +} + +impl> From> for Value { + fn from(value: Option) -> Self { + value.map_or(Self::Null, Into::into) + } +} + +impl> FromIterator for Value { + fn from_iter>(items: I) -> Self { + Self::List(items.into_iter().map(Into::into).collect()) + } +} + +impl Value { + /// Append this value's rendering to `out`. + fn write(&self, out: &mut String) { + match self { + Self::Text(text) => write_escaped(out, text), + Self::Number(count) => { + let _ = write!(out, "{count}"); + } + Self::Null => out.push_str("null"), + Self::List(items) => { + out.push('['); + for (at, item) in items.iter().enumerate() { + if at > 0 { + out.push(','); + } + item.write(out); + } + out.push(']'); + } + Self::Object(members) => { + out.push('{'); + for (at, (name, value)) in members.iter().enumerate() { + if at > 0 { + out.push(','); + } + write_escaped(out, name); + out.push(':'); + value.write(out); + } + out.push('}'); + } + } + } +} + +/// Write `text` as a JSON string, escaped. +/// +/// **This is the whole of the injection fix**, so it is deliberately total: a +/// caller's `io::Error` can contain a quote, a backslash, a newline, or a +/// control character from a localised message, and each of those would otherwise +/// end the string or the line. +fn write_escaped(out: &mut String, text: &str) { + out.push('"'); + for character in text.chars() { + match character { + '"' => out.push_str("\\\""), + '\\' => out.push_str("\\\\"), + '\n' => out.push_str("\\n"), + '\r' => out.push_str("\\r"), + '\t' => out.push_str("\\t"), + // Everything below a space is a control character JSON forbids + // unescaped. `\u` form rather than a name, because the named escapes + // above are the only ones JSON defines. + control if control < ' ' => { + let _ = write!(out, "\\u{:04x}", control as u32); + } + other => out.push(other), + } + } + out.push('"'); +} + +/// The report's machine-readable row. +/// +/// Members are ordered, and the order is the order they were added. That is a +/// property worth keeping even though JSON readers do not care: a human reading +/// accumulated CI output reads them in order, and a stable order makes a diff +/// between two runs legible. +#[derive(Clone, Debug, Default, PartialEq, Eq)] +pub struct Row { + members: Vec<(&'static str, Value)>, +} + +impl Row { + /// A row of the given `reason`, which is how a mining pass selects it. + #[must_use] + pub fn new(reason: &'static str) -> Self { + Self { + members: vec![("reason", Value::Text(reason.to_owned()))], + } + } + + /// Add a member. + /// + /// Takes the name and the value together, which is the point: they cannot be + /// reordered apart, and there is no position to miscount. + /// + /// # Panics + /// + /// If `name` is already present. A repeated top-level key is the one + /// malformation that SURVIVES a consumer's parse -- `serde_json` and + /// `JSON.parse` both accept it and silently keep the last value -- so a row + /// carrying one is not a broken artifact a survey discards but an ambiguous + /// one it mines, which is worse. The crate reports it as + /// `report_oracle::RowDefect::RepeatedKey`; this is the writer being unable + /// to produce it in the first place. + /// + /// (Deliberately not an intra-doc link. `report_oracle` is compiled only + /// under `cfg(any(test, feature = "oracle-in-renderer"))` while this module + /// is always built, so a link here cannot resolve in a default `cargo doc` + /// and emits a broken-intra-doc-link warning. Found by a review, and + /// confirmed by running `cargo doc -p windows-platform-probes --lib`.) + /// + /// **Why a panic and not a `Result`.** Every caller is a renderer in this + /// crate composing a fixed schema, so a repeat is a programming error at the + /// call site, not a condition to handle -- and a fallible builder would put + /// a `?` on nineteen infallible calls to describe a case that must never + /// happen. Reported by a review, which observed that this public writer + /// could emit a row the crate's own oracle faults. + #[must_use] + pub fn with(mut self, name: &'static str, value: impl Into) -> Self { + assert!( + !self.members.iter().any(|(present, _)| *present == name), + "the row already carries `{name}`, and a repeated key survives a \ + consumer's parse as whichever value happened to come last" + ); + let value = value.into(); + assert_unique_names(name, &value); + self.members.push((name, value)); + self + } + + /// Every key this row carries, in order. + /// + /// Derived rather than declared, so a check over the key set cannot drift + /// from what is published. + #[must_use] + pub fn keys(&self) -> Vec<&'static str> { + self.members.iter().map(|(name, _)| *name).collect() + } + + /// The row, rendered as one line of JSON. + #[must_use] + pub fn render(&self) -> String { + let mut out = String::new(); + Value::Object(self.members.clone()).write(&mut out); + out + } +} diff --git a/crates/windows-platform-probes/src/row/tests.rs b/crates/windows-platform-probes/src/row/tests.rs new file mode 100644 index 000000000..a11624850 --- /dev/null +++ b/crates/windows-platform-probes/src/row/tests.rs @@ -0,0 +1,234 @@ +// Copyright (c) Mike Grier. + +//! Tests for the row's value model and its writer. + +use super::{Row, Value}; + +#[test] +#[should_panic(expected = "the row already carries `processors`")] +fn the_writer_refuses_to_render_a_key_twice() { + // **The one malformation that survives a consumer's parse**, so the writer + // is made unable to produce it rather than the oracle being left to notice. + // `serde_json` and `JSON.parse` both accept a repeated key and keep the + // last, which turns a broken row into an AMBIGUOUS one -- mined rather than + // discarded. Reported by a review against this public builder. + let _ = Row::new("x") + .with("processors", 16_usize) + .with("processors", 32_usize); +} + +#[test] +#[should_panic(expected = "already carries `code`")] +fn the_writer_refuses_a_repeated_name_inside_a_nested_object_too() { + // **The other half of the duplicate-key guarantee.** `with` rejected a + // repeat at the TOP level only, so a nested entry could render + // `{"code":"a","code":"b"}` -- measured, exactly that string -- and the + // oracle could not see it either, because `keys` reads top-level names by + // design. A consumer's parser keeps whichever came last. + // + // Reported by a review as the gap left by the top-level check, which is + // what it was: the guarantee was stated for the row and enforced for one + // level of it. + let _ = Row::new("x").with( + "parse_incomplete", + Value::List(vec![Value::Object(vec![ + ("code", Value::Text("a".to_owned())), + ("code", Value::Text("b".to_owned())), + ])]), + ); +} + +#[test] +fn a_repeated_name_in_two_sibling_objects_is_fine() { + // The control. Uniqueness is per object, not across the row -- every + // diagnostic entry carries its own `code`, and a check that forbade that + // would reject every real report. + let row = Row::new("x").with( + "parse_incomplete", + Value::List(vec![ + Value::Object(vec![("code", Value::Text("a".to_owned()))]), + Value::Object(vec![("code", Value::Text("b".to_owned()))]), + ]), + ); + + assert_eq!( + row.render(), + r#"{"reason":"x","parse_incomplete":[{"code":"a"},{"code":"b"}]}"# + ); +} + +#[test] +fn a_row_renders_its_members_in_the_order_they_were_added() { + let row = Row::new("x-probe-topology") + .with("arch", "x86_64") + .with("processors", 16_usize); + + assert_eq!( + row.render(), + r#"{"reason":"x-probe-topology","arch":"x86_64","processors":16}"# + ); +} + +#[test] +fn the_key_set_is_derived_from_the_value() { + // **What `keys` is for, and what it is not.** It reports the names the + // builder was given, in order -- a property of the writer, worth pinning + // because the row's key ORDER is part of the contract. + // + // This said it was "the property that lets the well-formedness check stop + // carrying a census". It is not: a row missing a required field is still + // self-consistent, so a check reading `keys` back would only ever watch the + // writer agree with itself. The contract lives in + // `topology_report::MEASURED_ROW_KEYS` and its `_SHAPES` sibling, checked + // against the renderer that owes those fields. Found by a review. + let row = Row::new("x-probe-topology") + .with("arch", "x86_64") + .with("cores", 8_usize); + + assert_eq!(row.keys(), vec!["reason", "arch", "cores"]); +} + +#[test] +fn a_quote_in_caller_text_cannot_end_the_string_it_is_in() { + // **The injection fix, on the value that carries caller text.** A failed + // discovery interpolates an `io::Error`, and an OS message is free to + // contain a quote. + let row = Row::new("x-probe-topology").with("error", r#"he said "no" and left"#); + + assert_eq!( + row.render(), + r#"{"reason":"x-probe-topology","error":"he said \"no\" and left"}"# + ); +} + +#[test] +fn a_brace_in_caller_text_cannot_start_a_second_row() { + // Measured on PR #88: an `io::Error` containing `{` was selected as the + // report's machine-readable row, so a reader checked the caller's text + // instead of the probe's. A brace inside a string is inert -- this asserts + // the rendering keeps it there. + let row = Row::new("x-probe-topology").with("error", r#"failed at {"reason":"fake"}"#); + let rendered = row.render(); + + assert!( + rendered.lines().count() == 1, + "one line, so there is no second row to select: {rendered}" + ); + assert_eq!( + rendered, + r#"{"reason":"x-probe-topology","error":"failed at {\"reason\":\"fake\"}"}"# + ); +} + +#[test] +fn a_newline_in_caller_text_cannot_end_the_row() { + // The row is one LINE, and a mining pass splits on lines. An unescaped + // newline would put the rest of an error message on a line of its own, + // where it is neither the row nor prose. + let row = Row::new("x-probe-topology").with("error", "first\nsecond\r\nthird"); + let rendered = row.render(); + + assert_eq!(rendered.lines().count(), 1, "{rendered}"); + assert_eq!( + rendered, + r#"{"reason":"x-probe-topology","error":"first\nsecond\r\nthird"}"# + ); +} + +#[test] +fn a_backslash_is_escaped_so_it_cannot_escape_the_quote_after_it() { + // The subtle one: a message ending in a backslash -- a Windows path, say -- + // would otherwise escape the closing quote and swallow the rest of the row. + let row = Row::new("x-probe-topology").with("path", r"C:\temp\"); + + assert_eq!( + row.render(), + r#"{"reason":"x-probe-topology","path":"C:\\temp\\"}"# + ); +} + +#[test] +fn a_control_character_is_escaped_to_its_json_form() { + // A localised OS message can carry one, and JSON forbids them unescaped. + let row = Row::new("x-probe-topology").with("error", "bell\u{7}null\u{0}"); + + assert_eq!( + row.render(), + r#"{"reason":"x-probe-topology","error":"bell\u0007null\u0000"}"# + ); +} + +#[test] +fn a_tab_uses_its_named_escape_rather_than_the_numeric_one() { + let row = Row::new("x").with("t", "a\tb"); + + assert_eq!(row.render(), r#"{"reason":"x","t":"a\tb"}"#); +} + +#[test] +fn the_absent_case_renders_as_null_rather_than_being_omitted() { + // A consumer can tell `null` from a field this probe is too old to publish, + // and cannot tell an omission from either. + let row = Row::new("x").with("highest_numa_node", Option::::None); + + assert_eq!(row.render(), r#"{"reason":"x","highest_numa_node":null}"#); +} + +#[test] +fn a_present_option_renders_as_its_value() { + let row = Row::new("x").with("highest_numa_node", Some(2_usize)); + + assert_eq!(row.render(), r#"{"reason":"x","highest_numa_node":2}"#); +} + +#[test] +fn an_empty_list_renders_as_an_empty_list() { + // Not omitted, for the same reason `null` is not: "no conditions" and "this + // probe does not publish conditions" are different answers. + let row = Row::new("x").with("parse_incomplete", Value::List(Vec::new())); + + assert_eq!(row.render(), r#"{"reason":"x","parse_incomplete":[]}"#); +} + +#[test] +fn a_list_of_objects_renders_each_member_in_order() { + let row = Row::new("x").with( + "caches", + Value::List(vec![ + Value::Object(vec![ + ("level", Value::Number(1)), + ("domains", 8_usize.into()), + ]), + Value::Object(vec![ + ("level", Value::Number(3)), + ("domains", 1_usize.into()), + ]), + ]), + ); + + assert_eq!( + row.render(), + r#"{"reason":"x","caches":[{"level":1,"domains":8},{"level":3,"domains":1}]}"# + ); +} + +#[test] +fn a_list_collects_from_an_iterator_of_anything_a_value_accepts() { + let row = Row::new("x").with( + "efficiency_classes", + [0_u8, 1].into_iter().collect::(), + ); + + assert_eq!(row.render(), r#"{"reason":"x","efficiency_classes":[0,1]}"#); +} + +#[test] +fn a_key_is_escaped_too() { + // Keys are `&'static str` minted in this crate, so none needs escaping + // today -- but the writer has one path for strings, so a key that ever did + // is handled rather than being a hole waiting for the first policy name + // with a quote in it. + let row = Row::new("x").with("odd\"name", 1_usize); + + assert_eq!(row.render(), r#"{"reason":"x","odd\"name":1}"#); +} diff --git a/crates/windows-platform-probes/src/tests.rs b/crates/windows-platform-probes/src/tests.rs index a53d9a232..2ad53f00c 100644 --- a/crates/windows-platform-probes/src/tests.rs +++ b/crates/windows-platform-probes/src/tests.rs @@ -850,7 +850,13 @@ fn a_numa_node_the_topology_crate_never_saw_is_still_reported() { "{check:?}" ); assert_eq!(check.disagreements.len(), 1, "{check:?}"); - assert!(check.disagreements[0].contains("NUMA nodes"), "{check:?}"); + assert!( + matches!( + check.disagreements[0], + crate::topology::Disagreement::HighestNumaNode { .. } + ), + "{check:?}" + ); assert!(check.not_compared.is_empty(), "{check:?}"); } @@ -869,7 +875,13 @@ fn a_topology_reporting_no_numa_node_at_all_disagrees_with_a_raw_one() { "{check:?}" ); assert_eq!(check.disagreements.len(), 1, "{check:?}"); - assert!(check.disagreements[0].contains("none"), "{check:?}"); + assert!( + matches!( + check.disagreements[0], + crate::topology::Disagreement::HighestNumaNode { parsed: None, .. } + ), + "{check:?}" + ); } // A counter that could not be read must never read as agreement. Each of these @@ -895,7 +907,7 @@ fn a_failed_numa_read_is_incomplete_rather_than_agreement() { assert!(check.disagreements.is_empty(), "{check:?}"); assert_eq!(check.not_compared.len(), 1, "{check:?}"); assert!( - check.not_compared[0].contains("GetNumaHighestNodeNumber"), + check.not_compared[0] == crate::topology::NotCompared::HighestNumaNodeFailed, "{check:?}" ); } @@ -917,7 +929,7 @@ fn a_failed_processor_count_is_incomplete_rather_than_a_parse_disagreement() { ); assert!(check.disagreements.is_empty(), "{check:?}"); assert!( - check.not_compared[0].contains("GetActiveProcessorCount"), + check.not_compared[0] == crate::topology::NotCompared::ActiveProcessorCountFailed, "{check:?}" ); } @@ -935,7 +947,7 @@ fn a_failed_group_count_is_incomplete_rather_than_a_parse_disagreement() { ); assert!(check.disagreements.is_empty(), "{check:?}"); assert!( - check.not_compared[0].contains("GetActiveProcessorGroupCount"), + check.not_compared[0] == crate::topology::NotCompared::ActiveProcessorGroupCountFailed, "{check:?}" ); } @@ -1003,7 +1015,10 @@ fn disagreeing_source_enumerations_block_agreement_even_when_every_counter_match assert!(check.not_compared.is_empty(), "{check:?}"); assert_eq!(check.parse_incomplete.len(), 1, "{check:?}"); assert!( - check.parse_incomplete[0].contains("never agreed"), + matches!( + check.parse_incomplete[0], + crate::topology::ParseIncomplete::EnumerationsDisagreed { .. } + ), "{check:?}" ); assert_eq!( @@ -1374,9 +1389,11 @@ fn a_topology_with_no_processors_at_all_is_not_accused_of_hiding_packages() { let check = observation.cross_check(); assert!( - !check.parse_incomplete.iter().any( - |c| c.contains("no packages were reported") || c.contains("no cores were reported") - ), + !check.parse_incomplete.iter().any(|c| matches!( + c, + crate::topology::ParseIncomplete::NoPackages + | crate::topology::ParseIncomplete::NoCores + )), "with no processors reported, absent packages and cores are not a separate finding: \ {check:?}" ); @@ -1466,10 +1483,10 @@ fn observe_reports_a_numa_domain_whose_sources_number_it_differently() { // so the empty-survey entry fires alongside; asserting a total would couple // this test to causes it is not about. assert!( - check - .parse_incomplete - .iter() - .any(|c| c.contains("more than one distinct node number")), + check.parse_incomplete.iter().any(|c| matches!( + c, + crate::topology::ParseIncomplete::NumaDomainsWithConflictingLabels { .. } + )), "{check:?}" ); assert_eq!( @@ -1536,10 +1553,10 @@ fn a_machine_that_changed_still_reports_what_the_parse_itself_lost() { "the timing skew is still recorded: {check:?}" ); assert!( - check - .parse_incomplete - .iter() - .any(|c| c.contains("enumeration anomal")), + check.parse_incomplete.iter().any(|c| matches!( + c, + crate::topology::ParseIncomplete::EnumerationAnomalies { .. } + )), "and the dropped record is NOT suppressed by it: {check:?}" ); assert!( @@ -1556,12 +1573,12 @@ fn a_core_or_attribute_the_sources_disagree_about_blocks_agreement() { // cores, or give the same processor different efficiency classes. for (label, mutate) in [ ( - "core(s) were reported only by CPU Sets", + "cores_only_in_cpu_sets", Box::new(|o: &mut crate::topology::Observation| o.cores_only_in_cpu_sets = 1) as Box, ), ( - "attribute(s) carry more than one distinct value", + "processor_attribute_conflicts", Box::new(|o: &mut crate::topology::Observation| o.processor_attribute_conflicts = 1), ), ] { @@ -1575,7 +1592,7 @@ fn a_core_or_attribute_the_sources_disagree_about_blocks_agreement() { counter: {check:?}" ); assert!( - check.parse_incomplete.iter().any(|c| c.contains(label)), + check.parse_incomplete.iter().any(|c| c.code() == label), "{label}: {check:?}" ); assert_eq!( @@ -1710,7 +1727,10 @@ fn observe_carries_the_crates_attribute_conflicts() { .cross_check() .parse_incomplete .iter() - .any(|c| c.contains("attribute(s) carry more than one distinct value")), + .any(|c| matches!( + c, + crate::topology::ParseIncomplete::ProcessorAttributeConflicts { .. } + )), ); } @@ -1773,7 +1793,7 @@ fn a_topology_nobody_measured_cannot_be_certified_against_this_machine() { .cross_check() .parse_incomplete .iter() - .any(|c| c.contains("not measured from a running machine")), + .any(|c| matches!(c, crate::topology::ParseIncomplete::NotMeasured)), "{:?}", synthetic.cross_check() ); @@ -1793,7 +1813,7 @@ fn a_topology_nobody_measured_cannot_be_certified_against_this_machine() { .cross_check() .parse_incomplete .iter() - .any(|c| c.contains("not measured from a running machine")), + .any(|c| matches!(c, crate::topology::ParseIncomplete::NotMeasured)), "{:?}", measured.cross_check() ); @@ -1856,12 +1876,12 @@ fn a_report_of_no_packages_or_no_cores_is_a_finding_not_a_machine() { // exactly as an empty cache survey is. for (label, mutate) in [ ( - "packages", + "no_packages", Box::new(|o: &mut crate::topology::Observation| o.packages = 0) as Box, ), ( - "cores", + "no_cores", Box::new(|o: &mut crate::topology::Observation| o.cores = Vec::new()), ), ] { @@ -1874,7 +1894,7 @@ fn a_report_of_no_packages_or_no_cores_is_a_finding_not_a_machine() { "{label}: an absent relationship is not the crate contradicting a counter: {check:?}" ); assert!( - check.parse_incomplete.iter().any(|c| c.contains(label)), + check.parse_incomplete.iter().any(|c| c.code() == label), "{label}: {check:?}" ); assert_eq!( @@ -1904,7 +1924,10 @@ fn a_numa_domain_no_source_reported_blocks_agreement_rather_than_accusing_the_pa assert!(check.disagreements.is_empty(), "{check:?}"); assert_eq!(check.parse_incomplete.len(), 1, "{check:?}"); assert!( - !check.parse_incomplete[0].contains("CPU Sets"), + !matches!( + check.parse_incomplete[0], + crate::topology::ParseIncomplete::NumaDomainsOnlyInCpuSets { .. } + ), "nobody reported it, so the message must not name a reporter: {check:?}" ); assert_eq!( @@ -2801,15 +2824,27 @@ fn an_empty_core_or_package_record_blocks_agreement() { // count and the policy derived from it while nothing else notices. The // live-host test already calls an empty core a parse error; without this, // cross_check would still certify one. - for (label, mutate) in [ + // **The expected pair travels with each case, because the variant alone + // cannot tell them apart.** Both arms raise `RelationsWithoutProcessors`, so + // matching the variant would run this loop twice and establish one thing -- + // an empty CORE record and an empty PACKAGE record would be + // interchangeable, and a `cross_check` that counted one for the other would + // pass. The old substrings did not separate them either: `Display` renders + // `{cores} core(s) and {packages} package(s) cover no processors` in one + // sentence, so both `"core(s) and"` and `"package(s) cover no processors"` + // matched whichever field was set. That gap predates the move to variants; + // the payload is what closes it. + for (label, mutate, expected) in [ ( - "core(s) and", + "an empty core record", Box::new(|o: &mut crate::topology::Observation| o.cores_without_processors = 1) as Box, + (1, 0), ), ( - "package(s) cover no processors", + "an empty package record", Box::new(|o: &mut crate::topology::Observation| o.packages_without_processors = 1), + (0, 1), ), ] { let mut observation = agreeing_observation(); @@ -2818,8 +2853,13 @@ fn an_empty_core_or_package_record_blocks_agreement() { let check = observation.cross_check(); assert!(check.disagreements.is_empty(), "{label}: {check:?}"); assert!( - check.parse_incomplete.iter().any(|c| c.contains(label)), - "{label}: {check:?}" + check.parse_incomplete.iter().any(|c| matches!( + c, + crate::topology::ParseIncomplete::RelationsWithoutProcessors { cores, packages } + if (*cores, *packages) == expected + )), + "{label}: the entry must count the record that was empty, and only \ + it -- expected {expected:?}: {check:?}" ); assert_eq!( check.verdict(), @@ -2949,7 +2989,10 @@ fn a_relation_a_caller_described_is_not_a_measurement() { .cross_check() .parse_incomplete .iter() - .any(|c| c.contains("described by a caller")), + .any(|c| matches!( + c, + crate::topology::ParseIncomplete::DescribedRelations { .. } + )), "a measured topology carrying a described relation is not all measured: {:?}", described.cross_check() ); @@ -2968,7 +3011,10 @@ fn a_relation_a_caller_described_is_not_a_measurement() { .cross_check() .parse_incomplete .iter() - .any(|c| c.contains("described by a caller")), + .any(|c| matches!( + c, + crate::topology::ParseIncomplete::DescribedRelations { .. } + )), "{:?}", walked.cross_check() ); @@ -3006,7 +3052,7 @@ fn a_bracket_left_open_is_not_the_same_as_a_machine_that_held_still() { check .not_compared .iter() - .any(|c| c.contains("bracket around the parse was not closed")), + .any(|c| matches!(c, crate::topology::NotCompared::BracketNotEstablished)), "{check:?}" ); assert!( @@ -3076,7 +3122,7 @@ fn observe_will_not_claim_a_bracket_it_was_not_given() { check .not_compared .iter() - .any(|c| c.contains("bracket around the parse was not closed")), + .any(|c| matches!(c, crate::topology::NotCompared::BracketNotEstablished)), "{check:?}" ); assert_ne!( @@ -3156,10 +3202,10 @@ fn a_relation_no_source_reported_is_counted_whatever_its_kind() { let check = observation.cross_check(); assert!(check.disagreements.is_empty(), "{check:?}"); assert!( - check - .parse_incomplete - .iter() - .any(|c| c.contains("carry no observation from any source")), + check.parse_incomplete.iter().any(|c| matches!( + c, + crate::topology::ParseIncomplete::UnreportedRelations { .. } + )), "{check:?}" ); assert_eq!( @@ -3177,16 +3223,20 @@ fn a_measured_topology_reporting_no_processors_or_groups_blocks_agreement() { // machine its own parse says has no processors at all. The guard on the // package and core checks reads `online_processors > 0`, so the impossible // case silently switched those off as well. + // The label is the name the variant must carry in `absent`, not merely a + // caption for the failure message: zeroing one count must name THAT count. + // Matching the variant alone would pass for either, so the loop would run + // twice and establish one thing. for (label, mutate) in [ ( - "no online processors", + "online processors", Box::new(|o: &mut crate::topology::Observation| { o.online_processors = 0; o.raw_active_processors = 0; }) as Box, ), ( - "no processor groups", + "processor groups", Box::new(|o: &mut crate::topology::Observation| { o.groups = 0; o.raw_group_count = 0; @@ -3203,8 +3253,12 @@ fn a_measured_topology_reporting_no_processors_or_groups_blocks_agreement() { {check:?}" ); assert!( - check.parse_incomplete.iter().any(|c| c.contains(label)), - "{label}: {check:?}" + check.parse_incomplete.iter().any(|c| matches!( + c, + crate::topology::ParseIncomplete::MeasuredButCountsAbsent { absent } + if absent.as_slice() == [label] + )), + "{label}: the entry must name the count that was zero, and only it: {check:?}" ); assert_eq!( check.verdict(), @@ -3230,12 +3284,26 @@ fn both_absent_counts_are_named_together_rather_than_one_standing_for_the_pair() observation.raw_group_count = 0; let check = observation.cross_check(); - assert!( - check - .parse_incomplete - .iter() - .any(|c| c.contains("no online processors and processor groups")), - "{check:?}" + let absent = check + .parse_incomplete + .iter() + .find_map(|entry| match entry { + crate::topology::ParseIncomplete::MeasuredButCountsAbsent { absent } => Some(absent), + _ => None, + }) + .unwrap_or_else(|| panic!("no entry for the impossible machine at all: {check:?}")); + + // **The CONTENTS, not merely the variant.** Matching + // `MeasuredButCountsAbsent { .. }` holds when `absent` names one of the two, + // which is exactly what this test exists to forbid -- so it passed while + // establishing nothing beyond what the loop above already establishes. + // Measured: with `absent` truncated to its first entry, the whole suite + // stayed green at 249 passed. Found by a review. + assert_eq!( + absent.as_slice(), + ["online processors", "processor groups"], + "a machine missing both must name both, in the order the report states \ + them: {check:?}" ); } @@ -3254,17 +3322,17 @@ fn a_topology_nobody_measured_is_not_accused_of_describing_no_machine() { let check = observation.cross_check(); assert!( - !check - .parse_incomplete - .iter() - .any(|c| c.contains("cannot have none")), + !check.parse_incomplete.iter().any(|c| matches!( + c, + crate::topology::ParseIncomplete::MeasuredButCountsAbsent { .. } + )), "an unmeasured topology is not held to what a running machine must have: {check:?}" ); assert!( check .parse_incomplete .iter() - .any(|c| c.contains("was not measured from a running machine")), + .any(|c| matches!(c, crate::topology::ParseIncomplete::NotMeasured)), "and the reason it is exempt is itself reported: {check:?}" ); } @@ -3283,10 +3351,10 @@ fn two_walk_records_of_one_kind_claiming_a_processor_block_agreement() { let check = observation.cross_check(); assert!(check.disagreements.is_empty(), "{check:?}"); assert!( - check - .parse_incomplete - .iter() - .any(|c| c.contains("share a processor with another")), + check.parse_incomplete.iter().any(|c| matches!( + c, + crate::topology::ParseIncomplete::OverlappingWalkRelations { .. } + )), "{check:?}" ); assert_eq!( @@ -3637,10 +3705,10 @@ fn a_count_holding_relations_no_platform_reported_cannot_contradict_a_counter() counter: {check:?}" ); assert!( - check - .not_compared - .iter() - .any(|c| c.contains("could not be attributed to the parse")), + check.not_compared.iter().any(|c| matches!( + c, + crate::topology::NotCompared::CountsIncludeUnparsedRelations + )), "{label}: and the reason no comparison was made is reported: {check:?}" ); assert_eq!( @@ -3661,7 +3729,10 @@ fn a_wholly_parsed_topology_is_still_compared_against_its_counters() { assert!(!observation.counts_include_unparsed_relations()); let check = observation.cross_check(); assert!( - check.disagreements.iter().any(|c| c.contains("groups:")), + check + .disagreements + .iter() + .any(|c| matches!(c, crate::topology::Disagreement::ProcessorGroups { .. })), "{check:?}" ); assert_eq!(check.verdict(), crate::topology::Verdict::Disagree); @@ -3707,10 +3778,10 @@ fn a_core_whose_smt_flag_contradicts_its_own_processor_count_blocks_agreement() let check = observation.cross_check(); assert!(check.disagreements.is_empty(), "{label}: {check:?}"); assert!( - check - .parse_incomplete - .iter() - .any(|c| c.contains("disagrees with the number of processors")), + check.parse_incomplete.iter().any(|c| matches!( + c, + crate::topology::ParseIncomplete::ContradictoryCores { .. } + )), "{label}: {check:?}" ); assert_eq!( @@ -3763,10 +3834,10 @@ fn a_cache_level_numbered_zero_blocks_agreement() { let check = observation.cross_check(); assert!(check.disagreements.is_empty(), "{check:?}"); assert!( - check - .parse_incomplete - .iter() - .any(|c| c.contains("numbered 0")), + check.parse_incomplete.iter().any(|c| matches!( + c, + crate::topology::ParseIncomplete::UnnumberedCacheLevels { .. } + )), "{check:?}" ); assert_eq!(check.verdict(), crate::topology::Verdict::Incomplete); @@ -4060,10 +4131,10 @@ fn observe_counts_walk_numa_nodes_that_claim_the_same_processor() { "the counter agreed, so nothing is filed against the parse: {check:?}" ); assert!( - check - .parse_incomplete - .iter() - .any(|c| c.contains("share a processor with another")), + check.parse_incomplete.iter().any(|c| matches!( + c, + crate::topology::ParseIncomplete::OverlappingWalkRelations { .. } + )), "{check:?}" ); assert!(check.parse_in_doubt(), "{check:?}"); @@ -4093,7 +4164,12 @@ fn a_conflict_count_says_what_it_counted_rather_than_which_source_said_it() { mutate(&mut observation); let check = observation.cross_check(); - let entries = check.parse_incomplete.join(" "); + let entries = check + .parse_incomplete + .iter() + .map(ToString::to_string) + .collect::>() + .join(" "); assert!( entries.contains("more than one distinct"), "{label}: states what it counted: {check:?}" @@ -4215,10 +4291,10 @@ fn a_named_level_with_no_summary_blocks_agreement_and_sizes_to_one_domain() { let check = observation.cross_check(); assert!(check.disagreements.is_empty(), "{check:?}"); assert!( - check - .parse_incomplete - .iter() - .any(|c| c.contains("carries no summary for it")), + check.parse_incomplete.iter().any(|c| matches!( + c, + crate::topology::ParseIncomplete::PartitioningSummaryMissing { .. } + )), "{check:?}" ); assert_eq!( @@ -4448,3 +4524,507 @@ fn preparing_a_path_needs_no_volume_behind_its_drive_letter() { `request_cost` depends on for its hard-coded long-path sample" ); } + +// --- M3.1: the row names each condition, rather than counting them ----------- + +/// The codes of a diagnostic list, for comparing the row against what the +/// cross-check found. +/// +/// Generic over the three list types because each has its own `code`, and a +/// helper per list would be three copies of one idea. +fn codes_of(entries: &[T]) -> Vec { + entries + .iter() + .map(|entry| entry.code().to_owned()) + .collect() +} + +/// The three diagnostic vocabularies, so `codes_of` can take any of them. +trait HasCode { + fn code(&self) -> &'static str; +} + +impl HasCode for crate::topology::Disagreement { + fn code(&self) -> &'static str { + Self::code(self) + } +} + +impl HasCode for crate::topology::NotCompared { + fn code(&self) -> &'static str { + Self::code(self) + } +} + +impl HasCode for crate::topology::ParseIncomplete { + fn code(&self) -> &'static str { + Self::code(self) + } +} + +/// The codes the row publishes for `key`, in order. +/// +/// Reads the rendered artifact rather than the `CrossCheck` behind it, because +/// what a survey receives is the point: an assertion against the struct would +/// hold even if the writer published nothing at all. +fn row_codes(text: &str, key: &str) -> Vec { + let row = crate::report_oracle::row(text) + .unwrap_or_else(|| panic!("no single well-formed row in:\n{text}")); + + crate::report_oracle::list_codes(row, key) +} +#[test] +fn the_row_names_the_probes_own_bug_when_it_detects_one() { + // **The defect this whole milestone came from, stated as a test.** The + // renderer prints `BUG IN THIS PROBE ...` for a named partitioning level + // carrying no summary. The verdict has been forced away from `agree` since + // that defect was fixed -- but the row published only + // `"parse_incomplete":1`, so a survey could tell the run was in doubt and + // NOT that the doubt was this probe contradicting itself, which is a + // categorically different fact from a flaky host. + let mut observation = clean_observation(); + observation.partitioning_cache_level = Some(9); + + let text = crate::topology_report::report(BANNER, &observation); + + // **The precondition is read off the OBSERVATION, not off the prose.** This + // asserted `text.contains("BUG IN THIS PROBE")` first, which made a + // row-contract test depend on the wording of a sentence -- the one thing M3 + // says the row exists to stop. A reword of that line would have reddened + // this test, and the crate's `survives` sabotage control did not notice + // because it is anchored on a different prose line. Found by a review. + // + // The typed condition is the honest precondition anyway: it is what the + // renderer itself reads to decide whether to alarm. + let check = observation.cross_check(); + assert!( + check.parse_incomplete.iter().any(|entry| matches!( + entry, + crate::topology::diagnostic::ParseIncomplete::PartitioningSummaryMissing { .. } + )), + "the probe must have detected its own bug for this test to mean anything: {check:?}" + ); + assert!( + row_codes(&text, "parse_incomplete").contains(&"partitioning_summary_missing".to_owned()), + "the row must name the condition, not merely count it: {text}" + ); +} + +#[test] +fn the_row_lists_exactly_the_codes_of_the_conditions_the_check_found() { + // **The row against the vocabulary, on a report carrying several + // conditions at once.** A single-condition fixture cannot show that the + // codes travel in order, or that one is not dropped. + let mut observation = clean_observation(); + observation.caches = Vec::new(); + observation.cores_only_in_cpu_sets = 1; + observation.numa_domains_unreported = 2; + + let check = observation.cross_check(); + let expected: Vec = check + .parse_incomplete + .iter() + .map(|entry| entry.code().to_owned()) + .collect(); + + assert!( + expected.len() >= 3, + "the fixture must carry several conditions or it shows nothing: {check:?}" + ); + + let text = crate::topology_report::report(BANNER, &observation); + + assert_eq!( + row_codes(&text, "parse_incomplete"), + expected, + "every condition the check found reaches the row, in order: {text}" + ); +} + +#[test] +fn the_row_lists_a_condition_for_every_kind_the_check_found() { + // **The rule M3.1 establishes: a renderer may not tell a reader something + // the row cannot tell a survey.** + // + // This counted PROSE LINES and compared that number against the row -- the + // last place in the matrix that obtained structured data by reading + // sentences. Replaced by the same claim against the cross-check, which is + // stronger (it catches a reorder or a substitution, not only a drop) and + // never reads a sentence. + // + // Covers all three lists at once, which the prose count could not: under + // INCOMPLETE the renderer gives `not_compared` and `parse_incomplete` the + // same bare `- ` prefix, so only their total was recoverable from prose. + let mut observation = clean_observation(); + observation.caches = Vec::new(); + observation.numa_domains_unreported = 2; + observation.raw_group_count = 0; + + let text = crate::topology_report::report(BANNER, &observation); + let check = observation.cross_check(); + + for (key, expected) in [ + ("disagreements", codes_of(&check.disagreements)), + ("not_compared", codes_of(&check.not_compared)), + ("parse_incomplete", codes_of(&check.parse_incomplete)), + ] { + assert_eq!( + row_codes(&text, key), + expected, + "{key}: the row must publish what the check found, in order: {text}" + ); + } + + assert!( + !check.not_compared.is_empty() && check.parse_incomplete.len() >= 2, + "the fixture must fill more than one list, and one of them more than \ + once, or neither the coverage nor the ordering is exercised: {check:?}" + ); +} +#[test] +fn an_anomaly_reaches_the_row_as_its_kind() { + // Anomalies are published per-anomaly, so a survey can group by WHAT failed + // to decode. `AnomalyKind` is `#[non_exhaustive]`, so a kind this crate has + // no code for lands in `unclassified` -- visible in the row rather than + // silently mislabelled as a kind it is not. + let mut observation = clean_observation(); + observation.enumeration_anomalies = vec![ + windows_topology_sys::EnumerationAnomaly { + source: windows_topology_sys::Source::RelationshipWalk, + offset: 0, + kind: windows_topology_sys::AnomalyKind::TrailingBytes { remaining: 3 }, + }, + windows_topology_sys::EnumerationAnomaly { + source: windows_topology_sys::Source::CpuSets, + offset: 8, + kind: windows_topology_sys::AnomalyKind::Undersized { + declared: 8, + minimum: 48, + }, + }, + ]; + + let text = crate::topology_report::report(BANNER, &observation); + + assert_eq!( + row_codes(&text, "enumeration_anomalies"), + vec!["trailing_bytes".to_owned(), "undersized".to_owned()], + "each anomaly reaches the row as its own kind: {text}" + ); +} + +#[test] +fn the_row_names_which_counter_disagreed() { + // **The list M3.1 built a vocabulary for and did not wire.** `cross_check` + // said `disagree`, and the prose named the counter and both readings -- but + // the row carried no `disagreements` field at all, so a survey could tell + // that a host's parse was contradicted and not WHAT contradicted it. + // + // Same shape as the defect the milestone came from, and it survived every + // instrument here for a structural reason worth remembering: the fact + // accounting in `tests/a_real_report_agrees_with_itself.rs` enumerates the + // ROW's keys, so a fact the row omits entirely is outside the set of things + // it can ask about. + let mut observation = clean_observation(); + observation.raw_group_count = 2; + + let text = crate::topology_report::report(BANNER, &observation); + + assert!( + text.contains("=> DISAGREE"), + "the fixture must actually disagree or this shows nothing: {text}" + ); + assert_eq!( + row_codes(&text, "disagreements"), + vec!["processor_groups".to_owned()], + "the row must name the counter that disagreed: {text}" + ); +} + +#[test] +fn a_report_with_nothing_to_report_lists_no_disagreements() { + // The acceptance half: an agreeing host publishes the key with an empty + // list rather than omitting it, so a survey can tell "no disagreements" + // from "this probe is too old to say". + let text = crate::topology_report::report(BANNER, &clean_observation()); + + assert!(text.contains(r#""disagreements":[]"#), "{text}"); +} + +#[test] +fn a_discovery_error_full_of_json_cannot_manufacture_a_second_row() { + // **The injection defect, end to end through the renderer.** Measured on + // PR #88: an `io::Error` whose text contained `{` was selected as the + // report's machine-readable row, so a reader checked the caller's text + // instead of the probe's. Two containments now answer it -- the prose + // flattening that stops the text occupying a line, and the row's writer + // that escapes it into a string value. + // + // The error is chosen to be as hostile as an OS message can be: a brace, a + // quote, a backslash and a newline, each of which alone would end something. + let hostile = + "failed at {\"reason\":\"x-probe-topology\",\"cross_check\":\"agree\"}\nand C:\\temp\\"; + let text = crate::topology_report::report_unmeasured(BANNER, &std::io::Error::other(hostile)); + + let rows: Vec<&str> = text.lines().filter(|line| line.starts_with('{')).collect(); + assert_eq!( + rows.len(), + 1, + "the caller's text must not be selectable as a row: {text}" + ); + assert!( + rows[0].contains(r#""cross_check":"not_measured""#), + "and the ONE row is the probe's, not the caller's: {}", + rows[0] + ); + crate::report_oracle::assert_row_is_well_formed(&text); +} + +#[test] +fn a_discovery_error_reaches_the_row_as_a_field() { + // A survey counting failures wants to group them by cause, and the prose + // sentence is not something a mining pass should be parsing. + let text = crate::topology_report::report_unmeasured( + BANNER, + &std::io::Error::other("the device is not ready"), + ); + + assert!( + text.contains(r#""discovery_error":"the device is not ready""#), + "{text}" + ); +} + +#[test] +fn the_rendered_row_carries_exactly_the_keys_the_value_declares() { + // **The READER against the WRITER, and nothing more than that.** What this + // pins is that `report_oracle::keys` reads back exactly the names the value + // declared, in order -- a round-trip through the renderer and the parser. + // + // It is deliberately NOT the contract check, and this comment used to claim + // it was: it said a hand-written key list was a census that `Row::keys` + // derives away. False, and the next test is the correction -- a row missing + // a required field is still self-consistent, so two sides agreeing says + // nothing about WHICH keys the survey is owed. That is + // `MEASURED_ROW_KEYS`, checked below. Found by a review, which read this + // paragraph as still steering future edits back to the vacuous check. + let row = crate::row::Row::new("x-probe-topology") + .with("arch", "x86_64") + .with("cross_check", "agree"); + + assert_eq!( + crate::report_oracle::keys(&row.render()), + row.keys(), + "the writer publishes exactly what the value declares" + ); +} + +#[test] +fn the_measured_row_carries_exactly_the_contracts_keys() { + // **The check that was claimed and was not there.** The previous version + // compared `report_oracle::keys` against `Row::keys` -- the reader against + // the writer -- which says nothing about WHICH keys the contract requires. + // Measured: with `.with("packages", ...)` deleted from the builder, the + // entire suite stayed green. + // + // Compared as an exact SEQUENCE, so a dropped field, an added one and a + // reordered one all fail. Order is part of the contract here because + // accumulated CI output is read by humans as well as machines, and a stable + // order makes a diff between two runs legible. + let text = crate::topology_report::report(BANNER, &clean_observation()); + let row = crate::report_oracle::row(&text).expect("one well-formed row"); + + assert_eq!( + crate::report_oracle::keys(row), + crate::topology_report::MEASURED_ROW_KEYS, + "the measured row must carry exactly the contract's keys: {row}" + ); +} + +#[test] +fn the_unmeasured_row_carries_exactly_its_own_contracts_keys() { + // A host whose discovery FAILED publishes a different shape, and that is + // the point -- a survey must be able to tell it from a measured row that + // happens to be missing fields. So it has its own schema rather than being + // checked as a subset of the one above. + let text = crate::topology_report::report_unmeasured( + BANNER, + &std::io::Error::other("the device is not ready"), + ); + let row = crate::report_oracle::row(&text).expect("one well-formed row"); + + assert_eq!( + crate::report_oracle::keys(row), + crate::topology_report::UNMEASURED_ROW_KEYS, + "the unmeasured row must carry exactly its contract's keys: {row}" + ); +} + +#[test] +fn every_row_value_has_the_shape_its_schema_declares() { + // **The half the key contract could not state.** `MEASURED_ROW_KEYS` pins + // which names appear and in what order and says nothing about what they + // hold, so a renderer could publish `"processors":"16"` and satisfy the key + // test, the well-formedness check and every renderer assertion at once. + // Measured: rendering that one field through `.to_string()` left all 230 + // library tests and all 10 real-host integration tests green. Found by a + // review. + // + // Both shapes, because the unmeasured row is its own schema rather than a + // subset, and a check that only ever saw the measured one would leave the + // failed-discovery artifact -- the one a fleet survey sees most on a broken + // host -- unconstrained. + crate::report_oracle::assert_row_has_the_schemas_shapes( + &crate::topology_report::report(BANNER, &clean_observation()), + crate::topology_report::MEASURED_ROW_SHAPES, + ); + + crate::report_oracle::assert_row_has_the_schemas_shapes( + &crate::topology_report::report_unmeasured( + BANNER, + &std::io::Error::other("the device is not ready"), + ), + crate::topology_report::UNMEASURED_ROW_SHAPES, + ); +} + +#[test] +fn a_diagnostic_entry_without_a_code_is_a_shape_violation() { + // The control for the coded schema, and the reason a list of objects was + // not enough on its own: `code` is the stable discriminant a survey groups + // by, so an entry lacking one is unmineable while still being valid JSON. + let row = crate::row::Row::new("x-probe-topology") + .with( + "parse_incomplete", + crate::row::Value::List(vec![crate::row::Value::Object(vec![( + "level", + crate::row::Value::Number(9), + )])]), + ) + .render(); + + let violations = crate::report_oracle::shape_violations( + &row, + &[( + "parse_incomplete", + crate::row::Shape::ListOfObjectsWith(&[("code", crate::row::Shape::Text)]), + )], + ); + assert_eq!(violations.len(), 1, "{violations:?}"); + assert!(violations[0].contains("parse_incomplete"), "{violations:?}"); + + // And the same list WITH a code is accepted, so the rule is not simply + // rejecting every list of objects. + let coded = crate::row::Row::new("x-probe-topology") + .with( + "parse_incomplete", + crate::row::Value::List(vec![crate::row::Value::Object(vec![( + "code", + crate::row::Value::Text("not_measured".to_owned()), + )])]), + ) + .render(); + assert_eq!( + crate::report_oracle::shape_violations( + &coded, + &[( + "parse_incomplete", + crate::row::Shape::ListOfObjectsWith(&[("code", crate::row::Shape::Text)]) + )] + ), + Vec::::new() + ); +} + +#[test] +fn a_cache_entry_missing_its_numeric_members_is_a_shape_violation() { + // **`[{}]` used to satisfy `caches`.** The shape said "a list of objects" + // and stopped there, so the renderer could drop `level` and `domains`, or + // publish them as strings, with the schema oracle green -- measured, zero + // violations for an empty cache object. Reported by a review. + // + // `[{}]` is also, exactly, the bogus shape a design session claimed the row + // emitted and which was corrected earlier on this branch. It was never a + // real rendering; it was reachable through the checker. + let empty = crate::row::Row::new("x-probe-topology") + .with( + "caches", + crate::row::Value::List(vec![crate::row::Value::Object(Vec::new())]), + ) + .render(); + let violations = crate::report_oracle::shape_violations(&empty, &measured_caches()); + assert_eq!(violations.len(), 2, "{violations:?}"); + assert!( + violations + .iter() + .any(|what| what.contains("is missing `level`")), + "{violations:?}" + ); + + // A wrongly TYPED member, not merely an absent one. + let stringly = crate::row::Row::new("x-probe-topology") + .with( + "caches", + crate::row::Value::List(vec![crate::row::Value::Object(vec![ + ("level", crate::row::Value::Text("L1".to_owned())), + ("domains", crate::row::Value::Number(8)), + ])]), + ) + .render(); + let violations = crate::report_oracle::shape_violations(&stringly, &measured_caches()); + assert_eq!(violations.len(), 1, "{violations:?}"); + assert!(violations[0].contains("caches[0].level"), "{violations:?}"); + + // The control: a real cache entry passes, so the rule is not rejecting + // everything. + let good = crate::row::Row::new("x-probe-topology") + .with( + "caches", + crate::row::Value::List(vec![crate::row::Value::Object(vec![ + ("level", crate::row::Value::Number(1)), + ("domains", crate::row::Value::Number(8)), + ])]), + ) + .render(); + assert_eq!( + crate::report_oracle::shape_violations(&good, &measured_caches()), + Vec::::new() + ); +} + +/// The `caches` entry OF the measured schema, taken from the schema rather than +/// restated beside it. +/// +/// Written out, this test would have pinned a shape of its own and passed while +/// `MEASURED_ROW_SHAPES` declared something else -- the copy checking the copy. +fn measured_caches() -> Vec<(&'static str, crate::row::Shape)> { + let entry: Vec<_> = crate::topology_report::MEASURED_ROW_SHAPES + .iter() + .copied() + .filter(|(key, _)| *key == "caches") + .collect(); + + assert_eq!(entry.len(), 1, "the schema declares `caches` exactly once"); + entry +} + +#[test] +fn the_two_row_shapes_are_distinguishable_by_their_keys() { + // The guard that keeps the two schemas from drifting into each other. If + // the unmeasured shape ever became a prefix of the measured one, a survey + // reading a truncated measured row could not tell it from a failed + // discovery -- which is the distinction the unmeasured row exists to make. + assert_ne!( + crate::topology_report::MEASURED_ROW_KEYS, + crate::topology_report::UNMEASURED_ROW_KEYS + ); + assert!( + crate::topology_report::UNMEASURED_ROW_KEYS.contains(&"discovery_error"), + "the failed-discovery shape is identified by a key the measured one \ + does not have, rather than by absence" + ); + assert!( + !crate::topology_report::MEASURED_ROW_KEYS.contains(&"discovery_error"), + "and the measured shape must not carry it" + ); +} diff --git a/crates/windows-platform-probes/src/topology.rs b/crates/windows-platform-probes/src/topology.rs index 3a9353ca3..6ac61ebd4 100644 --- a/crates/windows-platform-probes/src/topology.rs +++ b/crates/windows-platform-probes/src/topology.rs @@ -43,6 +43,11 @@ use windows_topology_sys::{ Source, }; +pub mod diagnostic; +pub mod invariant; + +pub use diagnostic::{Disagreement, NotCompared, ParseIncomplete}; + /// One cache level, summarised across the machine. #[derive(Debug, Clone, PartialEq, Eq)] pub struct CacheLevel { @@ -536,36 +541,24 @@ impl Observation { let mut check = CrossCheck::default(); if !self.enumeration_anomalies.is_empty() { - check.parse_incomplete.push(format!( - // States the CONDITION, not a direction. "Records were dropped - // and the counts are short" was true of the two anomaly kinds - // that decode to nothing and false of `TruncatedArray`, which - // keeps the record with the entries that fit -- and a cache - // record kept with a partial affinity mask can INFLATE a - // partition count rather than shorten it, so the old message - // pointed a reader the wrong way. `AnomalyKind` is - // `#[non_exhaustive]`, so classifying here would need a - // catch-all arm that a future variant falls into silently; - // saying only what is true of all of them cannot rot that way. - "windows-topology-sys recorded {} enumeration anomal{}, so what Windows returned \ - was not fully decoded and the counts above may be short, or overstated where a \ - record was kept with an incomplete processor set", - self.enumeration_anomalies.len(), - if self.enumeration_anomalies.len() == 1 { - "y" - } else { - "ies" - }, - )); + // The wording's reasoning lives on the variant now, with the rest of + // this vocabulary. `AnomalyKind` is `#[non_exhaustive]`, so this + // entry deliberately says only what is true of every kind; the row + // publishes each anomaly's own code separately, where the catch-all + // is visible as `unclassified` rather than hidden in a sentence. + check + .parse_incomplete + .push(ParseIncomplete::EnumerationAnomalies { + count: self.enumeration_anomalies.len(), + }); } if self.numa_domains_only_in_cpu_sets > 0 { - check.parse_incomplete.push(format!( - "{} NUMA domain(s) were reported only by CPU Sets and never by the relationship \ - walk, so the two sources group nodes differently -- which no counter and no \ - coherence check reaches, since coherence compares processor sets", - self.numa_domains_only_in_cpu_sets, - )); + check + .parse_incomplete + .push(ParseIncomplete::NumaDomainsOnlyInCpuSets { + count: self.numa_domains_only_in_cpu_sets, + }); } // No cache levels at all. Distinct from the per-level case below, and @@ -576,11 +569,7 @@ impl Observation { // conclusion the report draws about cache structure would be drawn from // nothing. if self.caches.is_empty() { - check.parse_incomplete.push( - "no cache levels were reported at all, so what divides this machine by cache \ - was not established in either direction" - .to_string(), - ); + check.parse_incomplete.push(ParseIncomplete::NoCacheLevels); } // A level the survey DOES carry, with no partitions at all. @@ -599,18 +588,27 @@ impl Observation { .map(|c| c.level) .collect(); if !empty_levels.is_empty() { - check.parse_incomplete.push(format!( - "cache level(s) {empty_levels:?} decoded to no partitions at all, so what \ - divides this machine at those levels was not established" - )); + check + .parse_incomplete + .push(ParseIncomplete::CacheLevelsWithoutPartitions { + levels: empty_levels, + }); } // A running machine has processors and groups whatever the enumeration // said, so a MEASURED topology reporting none of either did not describe - // its host. Nothing below reaches this: both raw counters report failure - // as zero, so a zero parse beside a failed read is filed as - // `not_compared` -- leaving `parse_in_doubt` false, and the report free - // to state an impossible machine without a caveat. + // its host. + // + // **The gap this closes, in the past tense it belongs in.** Nothing else + // reaches the case: both raw counters report failure as zero, so a zero + // parse beside a failed read WAS filed as `not_compared` alone -- which + // left `parse_in_doubt` false and the report free to state an impossible + // machine without a caveat. The push below is what changed that, and it + // makes `parse_in_doubt` true for exactly this case. + // + // The paragraph above described the old behaviour in the present tense, + // so it read as though this rule did not exist while sitting directly + // over it. Found by a review. // // Stated over the LIST rather than once per count, so a third such count // joins the array instead of needing its own rule to be remembered. @@ -623,11 +621,9 @@ impl Observation { .map(|(name, _)| name) .collect(); if self.topology_was_measured && !absent.is_empty() { - check.parse_incomplete.push(format!( - "this topology was measured from a running machine, which cannot have none, but \ - it reported no {}", - absent.join(" and "), - )); + check + .parse_incomplete + .push(ParseIncomplete::MeasuredButCountsAbsent { absent }); } // The machine has packages and cores whatever the enumeration said, so @@ -637,14 +633,10 @@ impl Observation { // simply not report the relationship, in which case no anomaly fires // and nothing else here notices. if self.online_processors > 0 && self.packages == 0 { - check - .parse_incomplete - .push("no packages were reported at all, though the machine has one".to_string()); + check.parse_incomplete.push(ParseIncomplete::NoPackages); } if self.online_processors > 0 && self.cores.is_empty() { - check - .parse_incomplete - .push("no cores were reported at all, though the machine has one".to_string()); + check.parse_incomplete.push(ParseIncomplete::NoCores); } // A record that contradicts ITSELF, which no counter reaches: nothing @@ -660,20 +652,22 @@ impl Observation { .filter(|core| core.contradicts_itself()) .count(); if contradictory_cores > 0 { - check.parse_incomplete.push(format!( - "{contradictory_cores} core(s) report an SMT flag that disagrees with the number \ - of processors recorded beside it, so the record contradicts itself" - )); + check + .parse_incomplete + .push(ParseIncomplete::ContradictoryCores { + count: contradictory_cores, + }); } // Windows numbers cache levels from 1, so a level of 0 is a level the // parse did not read rather than one the machine has. let unnumbered_levels = self.caches.iter().filter(|c| c.level == 0).count(); if unnumbered_levels > 0 { - check.parse_incomplete.push(format!( - "{unnumbered_levels} cache level(s) are numbered 0, which is not a level Windows \ - reports, so what they describe was not established" - )); + check + .parse_incomplete + .push(ParseIncomplete::UnnumberedCacheLevels { + count: unnumbered_levels, + }); } // The renderer prints this state as "BUG IN THIS PROBE ... Nothing @@ -687,83 +681,77 @@ impl Observation { // `partitioning_cache` are both public, which is the same reason the // clamps in `domain_counts` exist. if let PartitioningCache::SummaryMissing(level) = self.partitioning_cache() { - check.parse_incomplete.push(format!( - "L{level} was named as the outermost partitioning cache and this survey carries \ - no summary for it, so what it divides was not established" - )); + check + .parse_incomplete + .push(ParseIncomplete::PartitioningSummaryMissing { level }); } if !self.topology_was_measured { - check.parse_incomplete.push( - "this topology was not measured from a running machine, so nothing here \ - describes the host it is reported on" - .to_string(), - ); + check.parse_incomplete.push(ParseIncomplete::NotMeasured); } if self.cores_without_processors > 0 || self.packages_without_processors > 0 { - check.parse_incomplete.push(format!( - "{} core(s) and {} package(s) cover no processors, so they raise those counts \ - and the policies derived from them without describing any part of the machine", - self.cores_without_processors, self.packages_without_processors, - )); + check + .parse_incomplete + .push(ParseIncomplete::RelationsWithoutProcessors { + cores: self.cores_without_processors, + packages: self.packages_without_processors, + }); } if self.unreported_relations > 0 { - check.parse_incomplete.push(format!( - "{} relation(s) carry no observation from any source, so they are counted here \ - without any platform API having described them", - self.unreported_relations, - )); + check + .parse_incomplete + .push(ParseIncomplete::UnreportedRelations { + count: self.unreported_relations, + }); } if self.described_relations > 0 { - check.parse_incomplete.push(format!( - "{} relation(s) were described by a caller rather than reported by any platform \ - API, so the counts above are not all of them measured", - self.described_relations, - )); + check + .parse_incomplete + .push(ParseIncomplete::DescribedRelations { + count: self.described_relations, + }); } if self.cores_only_in_cpu_sets > 0 { - check.parse_incomplete.push(format!( - "{} core(s) were reported only by CPU Sets and never by the relationship walk, so \ - the two group processors into cores differently and the core count above holds \ - both groupings", - self.cores_only_in_cpu_sets, - )); + check + .parse_incomplete + .push(ParseIncomplete::CoresOnlyInCpuSets { + count: self.cores_only_in_cpu_sets, + }); } if self.overlapping_walk_relations > 0 { - check.parse_incomplete.push(format!( - "{} relation(s) reported by the relationship walk share a processor with another \ - of the same kind, so one processor is claimed by two packages, two cores or two \ - NUMA nodes and the counts above hold both", - self.overlapping_walk_relations, - )); + check + .parse_incomplete + .push(ParseIncomplete::OverlappingWalkRelations { + count: self.overlapping_walk_relations, + }); } if self.processor_attribute_conflicts > 0 { - check.parse_incomplete.push(format!( - "{} per-processor attribute(s) carry more than one distinct value, so the \ - efficiency classes above are one claim rather than an agreed one", - self.processor_attribute_conflicts, - )); + check + .parse_incomplete + .push(ParseIncomplete::ProcessorAttributeConflicts { + count: self.processor_attribute_conflicts, + }); } if self.numa_domains_with_conflicting_labels > 0 { - check.parse_incomplete.push(format!( - "{} NUMA domain(s) carry more than one distinct node number, so what node they \ - are was not established and the highest below takes the larger", - self.numa_domains_with_conflicting_labels, - )); + check + .parse_incomplete + .push(ParseIncomplete::NumaDomainsWithConflictingLabels { + count: self.numa_domains_with_conflicting_labels, + }); } if self.numa_domains_unreported > 0 { - check.parse_incomplete.push(format!( - "{} NUMA domain(s) carry no observation from either source, so they raise the \ - domain count while contributing no node number to compare", - self.numa_domains_unreported, - )); + check + .parse_incomplete + .push(ParseIncomplete::NumaDomainsUnreported { + count: self.numa_domains_unreported, + }); } match &self.coherence { @@ -772,22 +760,20 @@ impl Observation { walk_only, cpu_sets_only, attempts, - } => check.parse_incomplete.push(format!( - "windows-topology-sys reports its two enumerations never agreed within {attempts} \ - attempt(s): {} processor(s) seen only by the relationship walk, {} seen only by \ - CPU Sets and so absent from the parsed list entirely", - walk_only.len(), - cpu_sets_only.len(), - )), + } => check + .parse_incomplete + .push(ParseIncomplete::EnumerationsDisagreed { + attempts: *attempts, + walk_only: walk_only.len(), + cpu_sets_only: cpu_sets_only.len(), + }), // Unreachable from `discover`, which returns `Agreed` or // `Disagreed`. Reported rather than ignored because reaching it // would mean the parse came from somewhere that read nothing // twice, and a cross-check cannot certify that either. - Coherence::NotCollected => check.parse_incomplete.push( - "windows-topology-sys reports its coherence was never collected, so nothing \ - established that its two enumerations describe the same machine" - .to_string(), - ), + Coherence::NotCollected => check + .parse_incomplete + .push(ParseIncomplete::CoherenceNotCollected), } // Everything above is about the PARSE and is evaluated unconditionally. @@ -806,19 +792,11 @@ impl Observation { match self.bracket { BracketOutcome::HeldStill => {} BracketOutcome::Changed => { - check.not_compared.push( - "the machine changed while this ran -- the counters moved across the parse, \ - so the two readings describe different instants" - .to_string(), - ); + check.not_compared.push(NotCompared::MachineChanged); return check; } BracketOutcome::NotEstablished => { - check.not_compared.push( - "the bracket around the parse was not closed -- a counter failed one of its \ - two reads, so nothing established that the machine held still" - .to_string(), - ); + check.not_compared.push(NotCompared::BracketNotEstablished); return check; } } @@ -835,11 +813,9 @@ impl Observation { // Filed as `not_compared` rather than skipped silently: this is a // reading that could not be trusted, which is exactly what that list is. if self.counts_include_unparsed_relations() { - check.not_compared.push( - "these counts include relations no platform API reported, so a counter mismatch \ - could not be attributed to the parse" - .to_string(), - ); + check + .not_compared + .push(NotCompared::CountsIncludeUnparsedRelations); return check; } @@ -848,31 +824,29 @@ impl Observation { // count to compare against. Treating it as a count reported a failed // measurement as though the crate's parse were wrong. if self.raw_active_processors == 0 { - check.not_compared.push( - "GetActiveProcessorCount returned 0, which is its failure report".to_string(), - ); + check + .not_compared + .push(NotCompared::ActiveProcessorCountFailed); } else if self.online_processors != self.raw_active_processors as usize { - check.disagreements.push(format!( - "online processors: topology crate says {}, GetActiveProcessorCount says {}", - self.online_processors, self.raw_active_processors - )); + check.disagreements.push(Disagreement::OnlineProcessors { + parsed: self.online_processors, + counter: self.raw_active_processors, + }); } if self.raw_group_count == 0 { - check.not_compared.push( - "GetActiveProcessorGroupCount returned 0, which is its failure report".to_string(), - ); + check + .not_compared + .push(NotCompared::ActiveProcessorGroupCountFailed); } else if self.groups != self.raw_group_count as usize { - check.disagreements.push(format!( - "groups: topology crate says {}, GetActiveProcessorGroupCount says {}", - self.groups, self.raw_group_count - )); + check.disagreements.push(Disagreement::ProcessorGroups { + parsed: self.groups, + counter: self.raw_group_count, + }); } let Some(highest) = self.raw_highest_numa_node else { - check.not_compared.push( - "GetNumaHighestNodeNumber failed, so no NUMA comparison was made".to_string(), - ); + check.not_compared.push(NotCompared::HighestNumaNodeFailed); return check; }; @@ -883,12 +857,10 @@ impl Observation { // nodes 0 and 2 are a valid sparse topology, and the count form // would report a regression on hardware that is reporting itself // correctly. - check.disagreements.push(format!( - "NUMA nodes: topology crate's highest node is {}, GetNumaHighestNodeNumber says {}", - self.highest_numa_node - .map_or_else(|| "none".to_string(), |n| n.to_string()), - highest - )); + check.disagreements.push(Disagreement::HighestNumaNode { + parsed: self.highest_numa_node, + counter: highest, + }); } check } @@ -902,7 +874,7 @@ impl Observation { pub struct CrossCheck { /// Counters that were compared and did not match. Each is a finding about /// the shipping crate's parse. - pub disagreements: Vec, + pub disagreements: Vec, /// Comparisons this run could not make, or could not trust. Each is a gap /// in this measurement, not a finding about the parse. /// @@ -913,7 +885,7 @@ pub struct CrossCheck { /// relations no platform API reported. Both are comparisons that were not /// made; neither is a counter that failed. The causes are enumerated in /// exactly one place, [`Observation::cross_check`]'s body. - pub not_compared: Vec, + pub not_compared: Vec, /// Ways the parse is short, or its claims mutually inconsistent, such that /// agreeing counters cannot certify it. /// @@ -926,7 +898,7 @@ pub struct CrossCheck { /// while every entry happened to be a crate self-assessment. One is not: /// a NUMA domain only CPU Sets described is derived here, from provenance /// the crate carries but draws no conclusion about. - pub parse_incomplete: Vec, + pub parse_incomplete: Vec, } impl CrossCheck { @@ -1429,7 +1401,7 @@ pub fn observe( .outermost_partitioning_cache() .map(|(level, _)| level); - Observation { + let observation = Observation { online_processors, groups, numa_domains, @@ -1456,5 +1428,19 @@ pub fn observe( raw_group_count, raw_highest_numa_node, bracket, - } + }; + + // **Bound here, so the invariants hold for every observation this crate + // MEASURES whether or not one is ever rendered.** That is the difference + // between these and the correspondences they came from: the oracle can only + // speak about a report, so a caller who measures and never renders got + // nothing. `report` binds them too, which is what covers the observations + // the tests build by hand. + // + // Never in `cross_check`: `assert_holds` asks it for the verdict, so the + // assertion would recurse. + #[cfg(any(test, feature = "oracle-in-renderer"))] + invariant::assert_holds(&observation); + + observation } diff --git a/crates/windows-platform-probes/src/topology/diagnostic.rs b/crates/windows-platform-probes/src/topology/diagnostic.rs new file mode 100644 index 000000000..90207ef53 --- /dev/null +++ b/crates/windows-platform-probes/src/topology/diagnostic.rs @@ -0,0 +1,675 @@ +// Copyright (c) Mike Grier. + +//! The vocabulary of what a cross-check found, as values rather than sentences. +//! +//! # Why these are types and not `String`s +//! +//! Each of [`CrossCheck`](super::CrossCheck)'s three lists used to hold the +//! human sentence and nothing else, and the NDJSON row published each list's +//! **length**. So the fact a mining pass most needs -- *which* condition +//! occurred -- existed only in the prose: a survey reading +//! `"parse_incomplete":1` could not tell *the probe detected a bug in itself* +//! from *a core record contradicted itself* from *this topology was not +//! measured from a running machine*. +//! +//! That is the gap recorded in +//! [DESIGN-NOTES.md](../../DESIGN-NOTES.md#d-encoded-row-is-the-contract): the +//! row was impoverished relative to the prose, which is backwards given that the +//! row is the artifact a fleet survey mines and the designs rest on. +//! +//! A variant carries its own data, renders its own sentence through +//! [`fmt::Display`], and names itself through `code`. One definition, so the +//! sentence and the discriminant cannot drift apart -- where a parallel +//! `(code, String)` pair could be updated on one side only. +//! +//! # The code is the contract; the sentence is not +//! +//! `code` is a **stable machine discriminant** and changing one is a breaking +//! change to the row, exactly as renaming a field would be. The `Display` text +//! is free to be reworded at any time: it reaches only the prose, where a reader +//! is the consumer and rewording is harmless. +//! +//! This is what lets the prose stay written for a human. Before, a test or a +//! survey wanting to know which condition fired had to match on English, so +//! improving a sentence risked breaking a consumer -- which is a reason not to +//! improve it. + +use std::fmt; + +use windows_topology_sys::{AnomalyKind, EnumerationAnomaly, Source}; + +use crate::row::Value; + +/// A diagnostic as the row publishes it: its code, and the values it carries. +/// +/// **The data, not only the code.** M3.1 published the code alone, so a survey +/// learned `contradictory_cores` without learning that three cores contradicted +/// themselves. The variants already carry those values, for `Display`; what +/// stopped M3.1 publishing them is that the row was a positional template where +/// a nested object had to be hand-assembled. +/// +/// `code` is always first, so a reader scanning accumulated output sees the +/// identity before the detail. +fn entry(code: &'static str, fields: Vec<(&'static str, Value)>) -> Value { + let mut members = vec![("code", Value::Text(code.to_owned()))]; + members.extend(fields); + Value::Object(members) +} + +/// Defines `code` and `ALL_CODES` for a diagnostic enum from one list. +/// +/// The `match` is exhaustive, so a variant added to the enum does not compile +/// until it has a line here -- and that line reaches `ALL_CODES` without anyone +/// having to remember. That is the whole point. The tests assert that a fixture +/// covers `ALL_CODES`, so a new variant is not merely obliged to HAVE a code, it +/// is obliged to be EXERCISED; a hand-written `ALL_CODES` would have relocated +/// the omission rather than closed it. +/// +/// What this deliberately does not do is state the codes twice. The goldens in +/// the tests remain the independent second statement of the VALUES. This is the +/// single statement of the SET, and the two answer different questions. +macro_rules! diagnostic_codes { + ($enum:ident { $($pattern:pat => $code:literal,)+ }) => { + impl $enum { + /// The stable discriminant a survey groups by. + /// + /// Changing one of these is a breaking change to the NDJSON row. + #[must_use] + pub const fn code(&self) -> &'static str { + match self { + $($pattern => $code,)+ + } + } + + /// Every code this enum can produce, one per variant. + /// + /// Generated beside `code` from the same list, so it cannot omit a + /// variant the enum has. + pub const ALL_CODES: &'static [&'static str] = &[$($code,)+]; + } + }; +} + +/// A counter comparison that was made and did not match. +/// +/// Each is a finding about the shipping crate's parse, and is the only list +/// whose entries produce [`Verdict::Disagree`](super::Verdict::Disagree). +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum Disagreement { + /// The parsed processor count and `GetActiveProcessorCount` differ. + OnlineProcessors { + /// What `windows-topology-sys` parsed. + parsed: usize, + /// What the Win32 counter reported. + counter: u32, + }, + /// The parsed group count and `GetActiveProcessorGroupCount` differ. + ProcessorGroups { + /// What `windows-topology-sys` parsed. + parsed: usize, + /// What the Win32 counter reported. + counter: u16, + }, + /// The parsed highest NUMA node and `GetNumaHighestNodeNumber` differ. + /// + /// Highest against highest, never a count against `highest + 1`: Windows + /// does not promise the largest node *number* equals the node count, and + /// nodes 0 and 2 are a valid sparse topology. + HighestNumaNode { + /// The largest node number the parse carries, if it carries any. + parsed: Option, + /// What the Win32 counter reported. + counter: u32, + }, +} + +diagnostic_codes!(Disagreement { + Self::OnlineProcessors { .. } => "online_processors", + Self::ProcessorGroups { .. } => "processor_groups", + Self::HighestNumaNode { .. } => "highest_numa_node", +}); + +impl Disagreement { + /// This disagreement as the row publishes it. + #[must_use] + pub fn published(&self) -> Value { + let fields = match self { + Self::OnlineProcessors { parsed, counter } => vec![ + ("parsed", Value::Number(*parsed)), + ("counter", Value::Number(*counter as usize)), + ], + Self::ProcessorGroups { parsed, counter } => vec![ + ("parsed", Value::Number(*parsed)), + ("counter", Value::Number(*counter as usize)), + ], + Self::HighestNumaNode { parsed, counter } => vec![ + ( + "parsed", + parsed.map_or(Value::Null, |node| Value::Number(node as usize)), + ), + ("counter", Value::Number(*counter as usize)), + ], + }; + + entry(self.code(), fields) + } +} + +impl fmt::Display for Disagreement { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + match self { + Self::OnlineProcessors { parsed, counter } => write!( + f, + "online processors: topology crate says {parsed}, GetActiveProcessorCount says \ + {counter}" + ), + Self::ProcessorGroups { parsed, counter } => write!( + f, + "groups: topology crate says {parsed}, GetActiveProcessorGroupCount says {counter}" + ), + Self::HighestNumaNode { parsed, counter } => write!( + f, + "NUMA nodes: topology crate's highest node is {}, GetNumaHighestNodeNumber says \ + {counter}", + parsed.map_or_else(|| "none".to_string(), |n| n.to_string()), + ), + } + } +} + +/// A comparison this run could not make, or could not trust. +/// +/// Each is a gap in this measurement, not a finding about the parse -- which is +/// why [`parse_in_doubt`](super::CrossCheck::parse_in_doubt) excludes this list. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum NotCompared { + /// The counters moved across the parse, so the two readings describe + /// different instants. + MachineChanged, + /// A counter failed one of its two reads, so nothing established that the + /// machine held still. + BracketNotEstablished, + /// The counts include relations no platform API reported, so a mismatch + /// could not be attributed to the parse. + CountsIncludeUnparsedRelations, + /// `GetActiveProcessorCount` reported failure. + ActiveProcessorCountFailed, + /// `GetActiveProcessorGroupCount` reported failure. + ActiveProcessorGroupCountFailed, + /// `GetNumaHighestNodeNumber` reported failure. + HighestNumaNodeFailed, +} + +#[cfg(test)] +mod tests; + +/// What a report should print instead of an entry that describes itself as +/// nothing. +/// +/// **The wording of a diagnostic is a review obligation; its PRESENCE is not.** +/// Under the decision that the row is the machine contract, no test pins the +/// sentence a `Display` impl produces -- and a mutation sweep showed the cost of +/// stopping there: blanking `Display for Disagreement` or `Display for +/// NotCompared` left the suite green, and a reader would have got ` - ` with +/// nothing after the dash. +/// +/// A blank is the worst of the available answers. It is indistinguishable from a +/// rendering bug, from a finding with genuinely nothing to say, and from a stray +/// newline, so a reader cannot tell whether the probe found something it failed +/// to describe. Saying so explicitly costs one branch and turns an invisible +/// defect into a visible one. +/// +/// This is the seam that lets presence be machine-checked without wording being +/// checked: a test can assert that no diagnostic renders as this string, which +/// pins THAT each entry describes itself without pinning WHAT it says. +pub const UNDESCRIBED: &str = + "BUG IN THIS PROBE: a finding was recorded with nothing to say about it"; + +/// `entry` as the report should print it, or [`UNDESCRIBED`] if it prints blank. +#[must_use] +pub fn described(entry: &impl fmt::Display) -> String { + let text = entry.to_string(); + if text.trim().is_empty() { + UNDESCRIBED.to_owned() + } else { + text + } +} + +diagnostic_codes!(NotCompared { + Self::MachineChanged => "machine_changed", + Self::BracketNotEstablished => "bracket_not_established", + Self::CountsIncludeUnparsedRelations => "counts_include_unparsed_relations", + Self::ActiveProcessorCountFailed => "active_processor_count_failed", + Self::ActiveProcessorGroupCountFailed => "active_processor_group_count_failed", + Self::HighestNumaNodeFailed => "highest_numa_node_failed", +}); + +impl NotCompared { + /// This entry as the row publishes it. + /// + /// Every variant is a bare condition with no values of its own, so each + /// publishes its code and nothing else -- which is the honest rendering + /// rather than an object padded to look like the others. + #[must_use] + pub fn published(&self) -> Value { + entry(self.code(), Vec::new()) + } +} + +impl fmt::Display for NotCompared { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + let text = match self { + Self::MachineChanged => { + "the machine changed while this ran -- the counters moved across the parse, so \ + the two readings describe different instants" + } + Self::BracketNotEstablished => { + "the bracket around the parse was not closed -- a counter failed one of its two \ + reads, so nothing established that the machine held still" + } + Self::CountsIncludeUnparsedRelations => { + "these counts include relations no platform API reported, so a counter mismatch \ + could not be attributed to the parse" + } + Self::ActiveProcessorCountFailed => { + "GetActiveProcessorCount returned 0, which is its failure report" + } + Self::ActiveProcessorGroupCountFailed => { + "GetActiveProcessorGroupCount returned 0, which is its failure report" + } + Self::HighestNumaNodeFailed => { + "GetNumaHighestNodeNumber failed, so no NUMA comparison was made" + } + }; + + f.write_str(text) + } +} + +/// A way the parse is short, or its claims mutually inconsistent, such that +/// agreeing counters cannot certify it. +/// +/// Established from the PARSE rather than from any counter, which is why no +/// counter agreeing can retire an entry here. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum ParseIncomplete { + /// `windows-topology-sys` recorded records it could not decode. + /// + /// States the CONDITION, not a direction. "Records were dropped and the + /// counts are short" was true of the two anomaly kinds that decode to + /// nothing and false of `TruncatedArray`, which keeps the record with the + /// entries that fit -- and a cache record kept with a partial affinity mask + /// can INFLATE a partition count rather than shorten it, so the old message + /// pointed a reader the wrong way. + EnumerationAnomalies { + /// How many anomalies were recorded. + count: usize, + }, + /// NUMA domains only CPU Sets reported, so the two sources group nodes + /// differently. + NumaDomainsOnlyInCpuSets { + /// How many such domains. + count: usize, + }, + /// The survey reported no cache relationships whatsoever. + NoCacheLevels, + /// Levels the survey carries that decoded to no partitions at all. + CacheLevelsWithoutPartitions { + /// Which levels. + levels: Vec, + }, + /// A measured topology reported none of a count a running machine must have. + MeasuredButCountsAbsent { + /// Which counts were zero, in the order the report names them. + absent: Vec<&'static str>, + }, + /// No packages were reported, though the machine has one. + NoPackages, + /// No cores were reported, though the machine has one. + NoCores, + /// Cores whose SMT flag disagrees with the processors recorded beside it. + ContradictoryCores { + /// How many such cores. + count: usize, + }, + /// Cache levels numbered 0, which is not a level Windows reports. + UnnumberedCacheLevels { + /// How many such levels. + count: usize, + }, + /// A level was named as the outermost partitioning cache with no summary + /// for it. + /// + /// This is the probe detecting a bug in itself, and is the condition the + /// renderer prints as `BUG IN THIS PROBE`. + PartitioningSummaryMissing { + /// The level that was named. + level: u8, + }, + /// The topology was not measured from a running machine. + NotMeasured, + /// Cores and packages that cover no processors. + RelationsWithoutProcessors { + /// How many cores. + cores: usize, + /// How many packages. + packages: usize, + }, + /// Relations carrying no observation from any source. + UnreportedRelations { + /// How many relations. + count: usize, + }, + /// Relations a caller described rather than any platform API reporting them. + DescribedRelations { + /// How many relations. + count: usize, + }, + /// Cores only CPU Sets reported, so the two sources group processors into + /// cores differently. + CoresOnlyInCpuSets { + /// How many such cores. + count: usize, + }, + /// Walk relations sharing a processor with another of the same kind. + OverlappingWalkRelations { + /// How many relations. + count: usize, + }, + /// Per-processor attributes carrying more than one distinct value. + ProcessorAttributeConflicts { + /// How many attributes. + count: usize, + }, + /// NUMA domains carrying more than one distinct node number. + NumaDomainsWithConflictingLabels { + /// How many domains. + count: usize, + }, + /// NUMA domains carrying no observation from either source. + NumaDomainsUnreported { + /// How many domains. + count: usize, + }, + /// The crate's two enumerations never agreed. + EnumerationsDisagreed { + /// How many attempts were made. + /// + /// `u32` because that is what `Coherence::Disagreed` carries; taking the + /// upstream type rather than casting keeps this a copy of the value and + /// not a conversion of it. + attempts: u32, + /// Processors seen only by the relationship walk. + walk_only: usize, + /// Processors seen only by CPU Sets. + cpu_sets_only: usize, + }, + /// The crate reports its coherence was never collected. + CoherenceNotCollected, +} + +diagnostic_codes!(ParseIncomplete { + Self::EnumerationAnomalies { .. } => "enumeration_anomalies", + Self::NumaDomainsOnlyInCpuSets { .. } => "numa_domains_only_in_cpu_sets", + Self::NoCacheLevels => "no_cache_levels", + Self::CacheLevelsWithoutPartitions { .. } => "cache_levels_without_partitions", + Self::MeasuredButCountsAbsent { .. } => "measured_but_counts_absent", + Self::NoPackages => "no_packages", + Self::NoCores => "no_cores", + Self::ContradictoryCores { .. } => "contradictory_cores", + Self::UnnumberedCacheLevels { .. } => "unnumbered_cache_levels", + Self::PartitioningSummaryMissing { .. } => "partitioning_summary_missing", + Self::NotMeasured => "not_measured", + Self::RelationsWithoutProcessors { .. } => "relations_without_processors", + Self::UnreportedRelations { .. } => "unreported_relations", + Self::DescribedRelations { .. } => "described_relations", + Self::CoresOnlyInCpuSets { .. } => "cores_only_in_cpu_sets", + Self::OverlappingWalkRelations { .. } => "overlapping_walk_relations", + Self::ProcessorAttributeConflicts { .. } => "processor_attribute_conflicts", + Self::NumaDomainsWithConflictingLabels { .. } => "numa_domains_with_conflicting_labels", + Self::NumaDomainsUnreported { .. } => "numa_domains_unreported", + Self::EnumerationsDisagreed { .. } => "enumerations_disagreed", + Self::CoherenceNotCollected => "coherence_not_collected", +}); + +impl ParseIncomplete { + /// This entry as the row publishes it, with the values its variant carries. + #[must_use] + pub fn published(&self) -> Value { + let count = |value: &usize| vec![("count", Value::Number(*value))]; + let fields = match self { + Self::EnumerationAnomalies { count: n } + | Self::NumaDomainsOnlyInCpuSets { count: n } + | Self::ContradictoryCores { count: n } + | Self::UnnumberedCacheLevels { count: n } + | Self::UnreportedRelations { count: n } + | Self::DescribedRelations { count: n } + | Self::CoresOnlyInCpuSets { count: n } + | Self::OverlappingWalkRelations { count: n } + | Self::ProcessorAttributeConflicts { count: n } + | Self::NumaDomainsWithConflictingLabels { count: n } + | Self::NumaDomainsUnreported { count: n } => count(n), + Self::NoCacheLevels + | Self::NoPackages + | Self::NoCores + | Self::NotMeasured + | Self::CoherenceNotCollected => Vec::new(), + Self::CacheLevelsWithoutPartitions { levels } => { + vec![("levels", levels.iter().copied().collect())] + } + Self::MeasuredButCountsAbsent { absent } => { + vec![("absent", absent.iter().copied().collect())] + } + Self::PartitioningSummaryMissing { level } => { + vec![("level", Value::Number(*level as usize))] + } + Self::RelationsWithoutProcessors { cores, packages } => vec![ + ("cores", Value::Number(*cores)), + ("packages", Value::Number(*packages)), + ], + Self::EnumerationsDisagreed { + attempts, + walk_only, + cpu_sets_only, + } => vec![ + ("attempts", Value::Number(*attempts as usize)), + ("walk_only", Value::Number(*walk_only)), + ("cpu_sets_only", Value::Number(*cpu_sets_only)), + ], + }; + + entry(self.code(), fields) + } +} + +impl fmt::Display for ParseIncomplete { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + match self { + Self::EnumerationAnomalies { count } => write!( + f, + "windows-topology-sys recorded {count} enumeration anomal{}, so what Windows \ + returned was not fully decoded and the counts above may be short, or overstated \ + where a record was kept with an incomplete processor set", + if *count == 1 { "y" } else { "ies" }, + ), + Self::NumaDomainsOnlyInCpuSets { count } => write!( + f, + "{count} NUMA domain(s) were reported only by CPU Sets and never by the \ + relationship walk, so the two sources group nodes differently -- which no \ + counter and no coherence check reaches, since coherence compares processor sets" + ), + Self::NoCacheLevels => f.write_str( + "no cache levels were reported at all, so what divides this machine by cache was \ + not established in either direction", + ), + Self::CacheLevelsWithoutPartitions { levels } => write!( + f, + "cache level(s) {levels:?} decoded to no partitions at all, so what divides this \ + machine at those levels was not established" + ), + Self::MeasuredButCountsAbsent { absent } => write!( + f, + "this topology was measured from a running machine, which cannot have none, but \ + it reported no {}", + absent.join(" and "), + ), + Self::NoPackages => { + f.write_str("no packages were reported at all, though the machine has one") + } + Self::NoCores => { + f.write_str("no cores were reported at all, though the machine has one") + } + Self::ContradictoryCores { count } => write!( + f, + "{count} core(s) report an SMT flag that disagrees with the number of processors \ + recorded beside it, so the record contradicts itself" + ), + Self::UnnumberedCacheLevels { count } => write!( + f, + "{count} cache level(s) are numbered 0, which is not a level Windows reports, so \ + what they describe was not established" + ), + Self::PartitioningSummaryMissing { level } => write!( + f, + "L{level} was named as the outermost partitioning cache and this survey carries \ + no summary for it, so what it divides was not established" + ), + Self::NotMeasured => f.write_str( + "this topology was not measured from a running machine, so nothing here describes \ + the host it is reported on", + ), + Self::RelationsWithoutProcessors { cores, packages } => write!( + f, + "{cores} core(s) and {packages} package(s) cover no processors, so they raise \ + those counts and the policies derived from them without describing any part of \ + the machine" + ), + Self::UnreportedRelations { count } => write!( + f, + "{count} relation(s) carry no observation from any source, so they are counted \ + here without any platform API having described them" + ), + Self::DescribedRelations { count } => write!( + f, + "{count} relation(s) were described by a caller rather than reported by any \ + platform API, so the counts above are not all of them measured" + ), + Self::CoresOnlyInCpuSets { count } => write!( + f, + "{count} core(s) were reported only by CPU Sets and never by the relationship \ + walk, so the two group processors into cores differently and the core count \ + above holds both groupings" + ), + Self::OverlappingWalkRelations { count } => write!( + f, + "{count} relation(s) reported by the relationship walk share a processor with \ + another of the same kind, so one processor is claimed by two packages, two cores \ + or two NUMA nodes and the counts above hold both" + ), + Self::ProcessorAttributeConflicts { count } => write!( + f, + "{count} per-processor attribute(s) carry more than one distinct value, so the \ + efficiency classes above are one claim rather than an agreed one" + ), + Self::NumaDomainsWithConflictingLabels { count } => write!( + f, + "{count} NUMA domain(s) carry more than one distinct node number, so what node \ + they are was not established and the highest below takes the larger" + ), + Self::NumaDomainsUnreported { count } => write!( + f, + "{count} NUMA domain(s) carry no observation from either source, so they raise \ + the domain count while contributing no node number to compare" + ), + Self::EnumerationsDisagreed { + attempts, + walk_only, + cpu_sets_only, + } => write!( + f, + "windows-topology-sys reports its two enumerations never agreed within \ + {attempts} attempt(s): {walk_only} processor(s) seen only by the relationship \ + walk, {cpu_sets_only} seen only by CPU Sets and so absent from the parsed list \ + entirely" + ), + Self::CoherenceNotCollected => f.write_str( + "windows-topology-sys reports its coherence was never collected, so nothing \ + established that its two enumerations describe the same machine", + ), + } + } +} + +/// The stable discriminant for an anomaly the enumeration recorded. +/// +/// **`AnomalyKind` is `#[non_exhaustive]`, so this match needs a catch-all and +/// a variant added upstream lands in it.** That is stated rather than hidden: +/// `unclassified` is a real answer meaning "this probe's vocabulary is older +/// than the crate's", which is more useful to a survey than a code invented +/// here that pretends to name the new kind. The row publishes it, so a sweep +/// over accumulated output finds the day the vocabularies parted rather than +/// silently mislabelling the anomaly. +/// +/// It is deliberately NOT a compile error. Owning the row's vocabulary means +/// this crate decides when a new upstream kind earns a code, and a build that +/// breaks on a dependency bump would force that decision at the worst moment. +#[must_use] +pub fn anomaly_code(anomaly: &EnumerationAnomaly) -> &'static str { + match anomaly.kind { + AnomalyKind::Undersized { .. } => "undersized", + AnomalyKind::OverrunsBuffer { .. } => "overruns_buffer", + AnomalyKind::TrailingBytes { .. } => "trailing_bytes", + AnomalyKind::TruncatedArray { .. } => "truncated_array", + _ => "unclassified", + } +} + +/// An anomaly as the row publishes it. +/// +/// Carries WHERE as well as what: the enumeration it was reading and the byte +/// offset it stopped at. A survey grouping by kind across a fleet wants both -- +/// the same kind at the same offset on many hosts is a different finding from +/// the same kind scattered, and neither is visible from a count. +/// +/// `AnomalyKind` is `#[non_exhaustive]`, so a kind added upstream lands in +/// `unclassified` and publishes no fields rather than a guess at which ones it +/// has. That is the honest rendering: the code says the vocabulary is older than +/// the crate, and inventing fields for it would say more than is known. +#[must_use] +pub fn published_anomaly(anomaly: &EnumerationAnomaly) -> Value { + let source = match anomaly.source { + Source::RelationshipWalk => "relationship_walk", + Source::CpuSets => "cpu_sets", + _ => "unclassified", + }; + + let mut fields = vec![ + ("source", Value::Text(source.to_owned())), + ("offset", Value::Number(anomaly.offset)), + ]; + + fields.extend(match anomaly.kind { + AnomalyKind::Undersized { declared, minimum } => vec![ + ("declared", Value::Number(declared)), + ("minimum", Value::Number(minimum)), + ], + AnomalyKind::OverrunsBuffer { + declared, + remaining, + } => vec![ + ("declared", Value::Number(declared)), + ("remaining", Value::Number(remaining)), + ], + AnomalyKind::TrailingBytes { remaining } => { + vec![("remaining", Value::Number(remaining))] + } + AnomalyKind::TruncatedArray { declared, decoded } => vec![ + ("declared", Value::Number(declared)), + ("decoded", Value::Number(decoded)), + ], + _ => Vec::new(), + }); + + entry(anomaly_code(anomaly), fields) +} diff --git a/crates/windows-platform-probes/src/topology/diagnostic/tests.rs b/crates/windows-platform-probes/src/topology/diagnostic/tests.rs new file mode 100644 index 000000000..87b289ca0 --- /dev/null +++ b/crates/windows-platform-probes/src/topology/diagnostic/tests.rs @@ -0,0 +1,648 @@ +// Copyright (c) Mike Grier. + +//! Tests for what the diagnostics publish into the row. +//! +//! # Why these are literals +//! +//! Every assertion here writes the expected wire form out by hand. That is +//! deliberate, and it is the opposite of what this crate does elsewhere: a rule +//! that is a PREDICATE over values is defined once and asked, never restated, +//! because a hand-written second copy checks the copy rather than the contract. +//! +//! A code and a field name are not predicates. They are a SCHEMA -- the names a +//! fleet survey groups by, which this module's own docs call "a breaking change +//! to the NDJSON row" -- and a schema is not derivable from the thing that emits +//! it. Writing it down twice is how a golden works: the test disagrees when the +//! writer moves, which is the entire point. +//! +//! The distinction matters because the tests that existed before these did the +//! derivable thing to the non-derivable one. They built their expectation from +//! `code()` and compared it against a row the writer had built from `code()`, so +//! both sides moved together and neither pinned anything. +//! +//! # What that left open, measured +//! +//! A mutation sweep of the parent module returned **12 survivors of 26** -- +//! `NotCompared::code` could be replaced wholesale with `""` or `"xyzzy"`, and +//! every arm of `published_anomaly` and two of `anomaly_code` could be deleted, +//! all with a green suite. Separately, rewriting `published`'s count helper to +//! `*value * 7 + 1` -- every count in every entry wrong -- left 218 tests +//! passing. +//! +//! That is the field-labelling defect `row.rs` exists to make unrepresentable, +//! reappearing one level down: `row.rs` pairs a name with its value so position +//! cannot mislabel them, and then these functions hand-pair names with values +//! inside each entry, where nothing was watching. +//! +//! # Completeness +//! +//! Where the enum belongs to this crate, the expectation is written as an +//! exhaustive `match`, so a variant added without a golden does not compile. +//! `AnomalyKind` and `Source` are `#[non_exhaustive]` upstream and cannot be +//! matched exhaustively; for those the goldens are explicit instances, and each +//! named kind is asserted NOT to return `unclassified`, so an arm deleted from +//! the classifier is caught rather than quietly becoming a fall-through. +//! +//! **What that does not cover, stated because the wording here used to claim it +//! did.** This said the `unclassified` fallback "is asserted directly". It is +//! not, and from this crate it cannot be: `#[non_exhaustive]` is precisely the +//! attribute that stops a downstream crate constructing a variant it does not +//! know, so no unknown kind can be built here to drive that arm. The arm is +//! reachable only from a future upstream release, and what is checked is the +//! half that is checkable -- that nothing this crate DOES name reaches it. +//! Found by a review. + +use super::{ + Disagreement, NotCompared, ParseIncomplete, UNDESCRIBED, anomaly_code, described, + published_anomaly, +}; +use crate::row::Row; +use windows_topology_sys::{AnomalyKind, EnumerationAnomaly, Source}; + +/// The published value, rendered through the row's own writer. +/// +/// Wrapped in a row rather than rendered directly, so the bytes under test are +/// the bytes a survey reads -- escaping, separators and all -- rather than a +/// second rendering written for the test. +fn rendered(value: crate::row::Value) -> String { + let row = Row::new("x-test").with("entry", value).render(); + let opened = row.find("\"entry\":").expect("the entry is present") + "\"entry\":".len(); + row[opened..row.len() - 1].to_owned() +} + +fn anomaly(source: Source, offset: usize, kind: AnomalyKind) -> EnumerationAnomaly { + EnumerationAnomaly { + source, + offset, + kind, + } +} + +#[test] +fn every_not_compared_code_is_the_one_the_row_promises() { + // Exhaustive, so a seventh variant does not compile until it has a code + // here. All six were unpinned: the sweep replaced the whole function with + // `""` and with `"xyzzy"` and nothing noticed. + let golden = |reason: &NotCompared| match reason { + NotCompared::MachineChanged => "machine_changed", + NotCompared::BracketNotEstablished => "bracket_not_established", + NotCompared::CountsIncludeUnparsedRelations => "counts_include_unparsed_relations", + NotCompared::ActiveProcessorCountFailed => "active_processor_count_failed", + NotCompared::ActiveProcessorGroupCountFailed => "active_processor_group_count_failed", + NotCompared::HighestNumaNodeFailed => "highest_numa_node_failed", + }; + + // The exhaustive `golden` above obliges a new variant to have a code; it does + // NOT oblige this array to carry one, so a seventh variant could take a wrong + // code with this `every...` test green. Reported by a review against exactly + // this loop. + let every = [ + NotCompared::MachineChanged, + NotCompared::BracketNotEstablished, + NotCompared::CountsIncludeUnparsedRelations, + NotCompared::ActiveProcessorCountFailed, + NotCompared::ActiveProcessorGroupCountFailed, + NotCompared::HighestNumaNodeFailed, + ]; + covers_every_variant( + "NotCompared", + &every.iter().map(NotCompared::code).collect::>(), + NotCompared::ALL_CODES, + ); + + for reason in every { + assert_eq!(reason.code(), golden(&reason), "{reason:?}"); + assert_eq!( + rendered(reason.published()), + format!("{{\"code\":\"{}\"}}", golden(&reason)), + "{reason:?}: a bare condition publishes its code and nothing else" + ); + } +} + +#[test] +fn every_disagreement_publishes_the_pair_it_carries() { + // `parsed` and `counter` are the two numbers a survey compares, so swapping + // the labels is the mislabelling defect in its purest form: the row still + // parses and says the opposite of the truth. + // + // A cases array rather than free-standing assertions, so the set this test + // exercises can be DERIVED and held against `ALL_CODES`. Written out, the + // name's "every disagreement" rested on nobody adding a fourth variant. + let cases = [ + ( + Disagreement::OnlineProcessors { + parsed: 12, + counter: 16, + }, + r#"{"code":"online_processors","parsed":12,"counter":16}"#, + ), + ( + Disagreement::ProcessorGroups { + parsed: 1, + counter: 2, + }, + r#"{"code":"processor_groups","parsed":1,"counter":2}"#, + ), + ( + Disagreement::HighestNumaNode { + parsed: Some(2), + counter: 3, + }, + r#"{"code":"highest_numa_node","parsed":2,"counter":3}"#, + ), + ]; + covers_every_variant( + "Disagreement", + &cases + .iter() + .map(|(found, _)| found.code()) + .collect::>(), + Disagreement::ALL_CODES, + ); + for (found, golden) in &cases { + assert_eq!(rendered(found.published()), *golden, "{found:?}"); + } + // The absent parse renders as `null`, not as a number and not as an omitted + // field: a survey must be able to tell "the parse saw no NUMA node" from + // "the parse saw node 0". + assert_eq!( + rendered( + Disagreement::HighestNumaNode { + parsed: None, + counter: 3, + } + .published() + ), + r#"{"code":"highest_numa_node","parsed":null,"counter":3}"# + ); +} + +/// Asserts that `fixture` contains an instance of EVERY variant, by the codes it +/// covers rather than by how many entries it has. +/// +/// **A count proves nothing.** This replaced `assert!(every.len() > 15)`, which +/// a fixture of any size passes while omitting a variant -- so a variant added +/// to the enum could go unrendered with the suite green, which is exactly the +/// hole the fixture existed to close. Found by a review. +/// +/// `ALL_CODES` is generated beside `code` from one list, so a new variant +/// reaches this check without anyone remembering to widen it. Compared as a SET +/// because a fixture may legitimately carry two instances of one variant to +/// exercise a payload that differs, as the `Disagreement` one does. +fn covers_every_variant(what: &str, covered: &[&str], all: &[&str]) { + // **Distinctness, checked on `ALL_CODES` rather than on the fixture**, and + // checked here so all three enums get it from one site. Set membership + // alone cannot see a duplicate: if two variants were given the same + // literal, `ALL_CODES` would carry it twice and a single fixture entry + // would satisfy both copies in both directions below. `ParseIncomplete` + // had a separate uniqueness assertion; `Disagreement` and `NotCompared` + // had none, so for those two a shared code was invisible. Found by a + // review. + // + // The harm is the same one the row exists to prevent: two conditions that + // publish one code cannot be told apart by a survey. + let mut seen: Vec<&str> = Vec::new(); + let mut repeated: Vec<&str> = Vec::new(); + for code in all { + if seen.contains(code) { + repeated.push(code); + } else { + seen.push(code); + } + } + assert!( + repeated.is_empty(), + "two {what} variants publish {repeated:?}, so a survey cannot tell those \ + conditions apart" + ); + + let missing: Vec<&str> = all + .iter() + .filter(|code| !covered.contains(*code)) + .copied() + .collect(); + assert!( + missing.is_empty(), + "the {what} fixture omits {missing:?}, so those variants are never rendered here" + ); + + // The other direction, so a code retired from the enum does not linger in a + // fixture that then silently tests nothing. + let stale: Vec<&str> = covered + .iter() + .filter(|code| !all.contains(*code)) + .copied() + .collect(); + assert!( + stale.is_empty(), + "the {what} fixture carries {stale:?}, which no variant produces" + ); +} + +/// Every `ParseIncomplete` variant, one instance each. +/// +/// **Shared, because two tests need the same completeness and a second +/// hand-written list is a second chance to omit a variant.** The presence test +/// carried its own seven-element sample and so exercised `Display` for a third +/// of the enum; blanking any omitted arm would have rendered `UNDESCRIBED` in a +/// real report while that test stayed green. Found by a review. +/// +/// The exhaustive `match` in the code test forces a new variant to acquire a +/// golden; this forces it to be EXERCISED. Neither implies the other, which is +/// why both exist. +fn every_parse_incomplete() -> Vec { + vec![ + ParseIncomplete::EnumerationAnomalies { count: 1 }, + ParseIncomplete::NumaDomainsOnlyInCpuSets { count: 1 }, + ParseIncomplete::NoCacheLevels, + ParseIncomplete::CacheLevelsWithoutPartitions { levels: vec![1] }, + ParseIncomplete::MeasuredButCountsAbsent { absent: vec!["x"] }, + ParseIncomplete::NoPackages, + ParseIncomplete::NoCores, + ParseIncomplete::ContradictoryCores { count: 1 }, + ParseIncomplete::UnnumberedCacheLevels { count: 1 }, + ParseIncomplete::PartitioningSummaryMissing { level: 1 }, + ParseIncomplete::NotMeasured, + ParseIncomplete::RelationsWithoutProcessors { + cores: 1, + packages: 1, + }, + ParseIncomplete::UnreportedRelations { count: 1 }, + ParseIncomplete::DescribedRelations { count: 1 }, + ParseIncomplete::CoresOnlyInCpuSets { count: 1 }, + ParseIncomplete::OverlappingWalkRelations { count: 1 }, + ParseIncomplete::ProcessorAttributeConflicts { count: 1 }, + ParseIncomplete::NumaDomainsWithConflictingLabels { count: 1 }, + ParseIncomplete::NumaDomainsUnreported { count: 1 }, + ParseIncomplete::EnumerationsDisagreed { + attempts: 1, + walk_only: 1, + cpu_sets_only: 1, + }, + ParseIncomplete::CoherenceNotCollected, + ] +} +#[test] +fn every_parse_incomplete_variant_has_the_code_the_row_promises() { + // **Every variant, not every payload SHAPE.** The test below covers shapes, + // on the argument that the counted variants share one helper -- true of the + // PAYLOAD and false of the CODE, which is per-variant. So the counted variants + // could be given a wrong code with nothing to notice: the report corpus + // builds its expectation through `code()` itself, so both sides move + // together. Found by a review of the pull request. + // + // Exhaustive, so a variant added without a code here does not compile. The + // shape arguments are `..` because this pins the discriminant only; the + // payloads are the test below. + let golden = |entry: &ParseIncomplete| match entry { + ParseIncomplete::EnumerationAnomalies { .. } => "enumeration_anomalies", + ParseIncomplete::NumaDomainsOnlyInCpuSets { .. } => "numa_domains_only_in_cpu_sets", + ParseIncomplete::NoCacheLevels => "no_cache_levels", + ParseIncomplete::CacheLevelsWithoutPartitions { .. } => "cache_levels_without_partitions", + ParseIncomplete::MeasuredButCountsAbsent { .. } => "measured_but_counts_absent", + ParseIncomplete::NoPackages => "no_packages", + ParseIncomplete::NoCores => "no_cores", + ParseIncomplete::ContradictoryCores { .. } => "contradictory_cores", + ParseIncomplete::UnnumberedCacheLevels { .. } => "unnumbered_cache_levels", + ParseIncomplete::PartitioningSummaryMissing { .. } => "partitioning_summary_missing", + ParseIncomplete::NotMeasured => "not_measured", + ParseIncomplete::RelationsWithoutProcessors { .. } => "relations_without_processors", + ParseIncomplete::UnreportedRelations { .. } => "unreported_relations", + ParseIncomplete::DescribedRelations { .. } => "described_relations", + ParseIncomplete::CoresOnlyInCpuSets { .. } => "cores_only_in_cpu_sets", + ParseIncomplete::OverlappingWalkRelations { .. } => "overlapping_walk_relations", + ParseIncomplete::ProcessorAttributeConflicts { .. } => "processor_attribute_conflicts", + ParseIncomplete::NumaDomainsWithConflictingLabels { .. } => { + "numa_domains_with_conflicting_labels" + } + ParseIncomplete::NumaDomainsUnreported { .. } => "numa_domains_unreported", + ParseIncomplete::EnumerationsDisagreed { .. } => "enumerations_disagreed", + ParseIncomplete::CoherenceNotCollected => "coherence_not_collected", + }; + + let every = every_parse_incomplete(); + + // This test's NAME claims every variant, and the exhaustive `golden` above + // does not deliver that: a `match` obliges a variant to HAVE an arm, never + // obliges the fixture to reach it, so an omitted variant's code would go + // unchecked here. Stated independently rather than leaned on from the + // presence test, which could be deleted without this one noticing. + covers_every_variant( + "ParseIncomplete", + &every.iter().map(ParseIncomplete::code).collect::>(), + ParseIncomplete::ALL_CODES, + ); + + let mut seen: Vec<&str> = Vec::new(); + for entry in &every { + let code = entry.code(); + assert_eq!(code, golden(entry), "{entry:?}"); + assert!( + !seen.contains(&code), + "{entry:?}: `{code}` is already another variant's code, so a survey \ + cannot tell the two conditions apart" + ); + seen.push(code); + } +} + +#[test] +fn every_parse_incomplete_shape_publishes_the_fields_its_variant_carries() { + // One instance of each PAYLOAD SHAPE rather than of each variant: the + // count-carrying variants all share a single helper, and it was rewriting + // every one of them wrongly that left 218 tests green. + // + // No number here on purpose. Two comments in this file said "twelve counted + // variants" and there are eleven -- a census, wrong, in the tests written to + // stop exactly that. The shape argument does not depend on how many there + // are, so stating it buys nothing and rots. + let cases = [ + ( + ParseIncomplete::ContradictoryCores { count: 3 }, + r#"{"code":"contradictory_cores","count":3}"#, + ), + (ParseIncomplete::NoPackages, r#"{"code":"no_packages"}"#), + ( + ParseIncomplete::CacheLevelsWithoutPartitions { levels: vec![1, 2] }, + r#"{"code":"cache_levels_without_partitions","levels":[1,2]}"#, + ), + ( + ParseIncomplete::MeasuredButCountsAbsent { + absent: vec!["packages"], + }, + r#"{"code":"measured_but_counts_absent","absent":["packages"]}"#, + ), + ( + ParseIncomplete::PartitioningSummaryMissing { level: 3 }, + r#"{"code":"partitioning_summary_missing","level":3}"#, + ), + ( + ParseIncomplete::RelationsWithoutProcessors { + cores: 4, + packages: 7, + }, + r#"{"code":"relations_without_processors","cores":4,"packages":7}"#, + ), + ( + ParseIncomplete::EnumerationsDisagreed { + attempts: 2, + walk_only: 5, + cpu_sets_only: 9, + }, + r#"{"code":"enumerations_disagreed","attempts":2,"walk_only":5,"cpu_sets_only":9}"#, + ), + ]; + + for (entry, golden) in cases { + assert_eq!(rendered(entry.published()), golden, "{entry:?}"); + } +} + +#[test] +fn distinct_values_in_one_entry_are_not_interchangeable() { + // **The labelling check, stated as a property rather than as another + // golden.** The goldens above would still pass if two fields were swapped + // AND both goldens were updated to match -- which is exactly what an author + // mid-refactor does. This asks the narrower question a golden cannot: with + // every value distinct, does each name carry ITS value? + let relations = rendered( + ParseIncomplete::RelationsWithoutProcessors { + cores: 4, + packages: 7, + } + .published(), + ); + assert!( + relations.contains(r#""cores":4"#) && relations.contains(r#""packages":7"#), + "cores and packages must not be interchanged: {relations}" + ); + + let undersized = rendered(published_anomaly(&anomaly( + Source::RelationshipWalk, + 64, + AnomalyKind::Undersized { + declared: 8, + minimum: 48, + }, + ))); + assert!( + undersized.contains("\"declared\":8") && undersized.contains("\"minimum\":48"), + "declared and minimum must not be interchanged: {undersized}" + ); +} + +#[test] +fn every_named_anomaly_kind_has_a_code_of_its_own() { + // A deleted arm here does not fail loudly -- it falls through to + // `unclassified`, whose documented meaning is "this probe's vocabulary is + // older than the crate". A real overrun would then be filed as an unknown + // kind and mis-attributed across a fleet. Two of these arms were deletable + // with a green suite. + let cases = [ + ( + AnomalyKind::Undersized { + declared: 1, + minimum: 2, + }, + "undersized", + ), + ( + AnomalyKind::OverrunsBuffer { + declared: 3, + remaining: 4, + }, + "overruns_buffer", + ), + ( + AnomalyKind::TrailingBytes { remaining: 5 }, + "trailing_bytes", + ), + ( + AnomalyKind::TruncatedArray { + declared: 6, + decoded: 7, + }, + "truncated_array", + ), + ]; + + for (kind, golden) in cases { + let described = format!("{kind:?}"); + let found = anomaly_code(&anomaly(Source::RelationshipWalk, 0, kind)); + assert_eq!(found, golden, "{described}"); + assert_ne!( + found, "unclassified", + "{described}: a kind this crate names must not fall through to the \ + catch-all, which says the vocabulary is older than the crate" + ); + } +} + +#[test] +fn every_anomaly_publishes_where_it_was_found_as_well_as_what() { + // `source` and `offset` are the fields the module's docs give the reason + // for -- the same kind at the same offset across a fleet is a different + // finding from the same kind scattered. Every arm below was deletable. + let cases = [ + ( + anomaly( + Source::RelationshipWalk, + 64, + AnomalyKind::Undersized { + declared: 8, + minimum: 48, + }, + ), + r#"{"code":"undersized","source":"relationship_walk","offset":64,"declared":8,"minimum":48}"#, + ), + ( + anomaly( + Source::CpuSets, + 128, + AnomalyKind::OverrunsBuffer { + declared: 96, + remaining: 32, + }, + ), + r#"{"code":"overruns_buffer","source":"cpu_sets","offset":128,"declared":96,"remaining":32}"#, + ), + ( + anomaly( + Source::RelationshipWalk, + 256, + AnomalyKind::TrailingBytes { remaining: 12 }, + ), + r#"{"code":"trailing_bytes","source":"relationship_walk","offset":256,"remaining":12}"#, + ), + ( + anomaly( + Source::CpuSets, + 512, + AnomalyKind::TruncatedArray { + declared: 10, + decoded: 6, + }, + ), + r#"{"code":"truncated_array","source":"cpu_sets","offset":512,"declared":10,"decoded":6}"#, + ), + ]; + + for (found, golden) in cases { + assert_eq!(rendered(published_anomaly(&found)), golden, "{found:?}"); + } +} + +#[test] +fn every_diagnostic_describes_itself() { + // **Presence is machine-checked here; WORDING is not, and that is the + // whole point of the seam.** This asserts only that each entry renders as + // something a reader can act on -- never what it says -- so the prose stays + // a review obligation while a blank stops being possible to ship. + // + // Two `Display` impls could be blanked with a green suite, and a reader + // would have got ` - ` with nothing after the dash: indistinguishable + // from a rendering bug, from a finding with nothing to say, and from a + // stray newline. This test names that case and nothing else. + let disagreements = [ + Disagreement::OnlineProcessors { + parsed: 12, + counter: 16, + }, + Disagreement::ProcessorGroups { + parsed: 1, + counter: 2, + }, + Disagreement::HighestNumaNode { + parsed: Some(2), + counter: 3, + }, + Disagreement::HighestNumaNode { + parsed: None, + counter: 3, + }, + ]; + covers_every_variant( + "Disagreement", + &disagreements + .iter() + .map(Disagreement::code) + .collect::>(), + Disagreement::ALL_CODES, + ); + for entry in &disagreements { + let text = described(entry); + assert_ne!(text, UNDESCRIBED, "{entry:?} renders blank"); + assert!(!text.trim().is_empty(), "{entry:?} renders blank"); + } + + // The comment here used to read "Exhaustive, so a seventh `NotCompared` must + // describe itself to compile" -- of an ARRAY LITERAL, which forces nothing. + // A seventh variant compiles fine and is simply never rendered. Same defect + // as the one a review reported against the count below, in a comment that + // claimed the guarantee outright; found by sweeping the class rather than + // the reported instance. + let not_compared = [ + NotCompared::MachineChanged, + NotCompared::BracketNotEstablished, + NotCompared::CountsIncludeUnparsedRelations, + NotCompared::ActiveProcessorCountFailed, + NotCompared::ActiveProcessorGroupCountFailed, + NotCompared::HighestNumaNodeFailed, + ]; + covers_every_variant( + "NotCompared", + ¬_compared + .iter() + .map(NotCompared::code) + .collect::>(), + NotCompared::ALL_CODES, + ); + for entry in ¬_compared { + let text = described(entry); + assert_ne!(text, UNDESCRIBED, "{entry:?} renders blank"); + assert!(!text.trim().is_empty(), "{entry:?} renders blank"); + } + + // **Every variant, from the shared fixture.** This carried its own + // seven-element sample, so it exercised `Display` for a third of the enum -- + // `NoCacheLevels`, `EnumerationAnomalies`, `NoCores` and the rest were never + // rendered here, and blanking any of their arms would have put UNDESCRIBED + // in a real report while this test stayed green. Found by a review. + let every = every_parse_incomplete(); + covers_every_variant( + "ParseIncomplete", + &every.iter().map(ParseIncomplete::code).collect::>(), + ParseIncomplete::ALL_CODES, + ); + for entry in &every { + let text = described(entry); + assert_ne!(text, UNDESCRIBED, "{entry:?} renders blank"); + assert!(!text.trim().is_empty(), "{entry:?} renders blank"); + } +} + +#[test] +fn an_entry_that_says_nothing_is_called_out_rather_than_left_blank() { + // The other half, and without it the test above cannot distinguish a + // working `described` from one that returns its input unchanged. + struct Silent; + impl std::fmt::Display for Silent { + fn fmt(&self, _: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + Ok(()) + } + } + + struct Blank; + impl std::fmt::Display for Blank { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + // Whitespace, not emptiness: a reader cannot tell the two apart on + // the page, so neither may the check. + f.write_str(" ") + } + } + + assert_eq!(described(&Silent), UNDESCRIBED); + assert_eq!(described(&Blank), UNDESCRIBED); + assert_eq!(described(&"a real description"), "a real description"); +} diff --git a/crates/windows-platform-probes/src/topology/invariant.rs b/crates/windows-platform-probes/src/topology/invariant.rs new file mode 100644 index 000000000..8eac6d82b --- /dev/null +++ b/crates/windows-platform-probes/src/topology/invariant.rs @@ -0,0 +1,452 @@ +// Copyright (c) Mike Grier. + +//! States an observation can be in that forbid an agreeing verdict. +//! +//! # These are the correspondences that survived, moved off the text +//! +//! The report oracle (`crate::report_oracle`, present only in builds that run +//! it, so deliberately not a link from here) once checked relations between a +//! report's two rendered +//! halves. The rules worth keeping are not really about rendering -- they relate +//! a STATE to the verdict, and the report is only where that relation became +//! visible. They are here instead, as predicates over [`Observation`], per +//! [DESIGN-NOTES.md](../../DESIGN-NOTES.md#d-encoded-row-is-the-contract). +//! +//! Two things change by moving them. They run whether or not anything was +//! rendered, so a caller that measures and never builds a report still gets +//! them; and no parser stands between the rule and the values it reads, which is +//! where a large share of this crate's defects lived. +//! +//! # Why every rule here reads the OBSERVATION, and none reads the lists +//! +//! This was got wrong on the first attempt, and the reason is worth recording +//! because it is not obvious. +//! +//! [`CrossCheck::verdict`] is a pure function of the three lists: a non-empty +//! `disagreements` gives `Disagree`, two empty lists beside it give `Agree`, +//! anything else gives `Incomplete`. So a rule of the form "a non-empty +//! `parse_incomplete` forbids `agree`" is not an invariant at all -- it restates +//! the definition, cannot fail for any input, and three such rules were written +//! here before that was noticed. +//! +//! Worse than useless: such a rule **cannot catch the defect this component +//! exists because of.** That defect was a state -- a named partitioning level +//! with no summary -- that `cross_check` had no branch for. A missing branch +//! means the list stays EMPTY, so a list-reading rule sees nothing to complain +//! about and the verdict it produces is `Agree` legitimately. The evidence that +//! something is wrong survives only in the observation. +//! +//! So each rule below names a state, computes it from the observation, and +//! requires the verdict to have moved off `agree`. A push site deleted from +//! `cross_check` leaves the state visible here and fires the rule; that is the +//! whole design, and it is what the sabotage evidence in +//! [COMPLETED-CHECKLIST.md](../../COMPLETED-CHECKLIST.md) M3.2 demonstrates. It +//! is also the manifest entry `cross_check forgets the changed bracket` in +//! [sabotage.json](../../sabotage.json), which re-runs that evidence rather than +//! leaving it as a claim about a sabotage somebody once performed. +//! +//! # What that means for testing them +//! +//! No input can violate these while `cross_check` is correct, because +//! `cross_check` is what makes them hold. They are reachable by CODE CHANGE, not +//! by data -- so [`check`] takes the verdict alongside the observation, letting +//! a test supply the answer a broken `cross_check` would give, and the sabotage +//! loop confirms that a real deletion reddens a real test. + +use std::fmt; + +use super::{Coherence, CrossCheck, Observation, PartitioningCache, Verdict}; + +#[cfg(test)] +mod tests; + +/// Declares [`BlockingState`]: the variants, [`BlockingState::ALL`] and +/// [`BlockingState::described`] all from ONE list. +/// +/// **This exists so that `ALL` cannot drift from the enum.** Writing the two by +/// hand does not prevent it, and the difference is not cosmetic: `ALL` is what +/// the completeness guard iterates, so a variant missing from it is a blocking +/// state nothing tests. A hand-written `ALL` was measured to allow exactly that +/// -- a new variant, its `described()` arm supplied because the `match` forces +/// one, compiled cleanly and left all ten invariant tests green while being +/// reached by none of them. +/// +/// The `match` in `described()` is genuinely exhaustive-checked, which is what +/// made the hand-written version look safe. It is not enough: it forces a new +/// variant to acquire an ARM, never an ENTRY in a separate array. Generating +/// both from one list is what ties them together, because the enum itself comes +/// from that list -- a variant that is not in it does not exist. +macro_rules! blocking_states { + ($( $(#[$doc:meta])* $variant:ident => $described:literal ),+ $(,)?) => { + /// A state an observation can be in that forbids an agreeing verdict. + /// + /// **A type rather than a `&'static str`, so completeness is + /// checkable.** These were strings, and the test that claimed to check + /// every state had a perturbation derived BOTH of its sets from the + /// perturbation table -- so a new branch in [`blocking_states`] that no + /// mutation reached appeared in neither set and both loops stayed green. + /// The guard could only confirm that existing labels described existing + /// mutations. + /// + /// The type is declared by a macro from a single list, so + /// [`BlockingState::ALL`] cannot omit a variant: the variants and `ALL` + /// are the same list. An earlier version wrote them separately and + /// claimed the compiler checked the correspondence, which it did not -- + /// found by a review, two rounds after the strings. + #[derive(Debug, Clone, Copy, PartialEq, Eq)] + pub enum BlockingState { + $( $(#[$doc])* $variant, )+ + } + + impl BlockingState { + /// Every state, so a test can check the perturbation table covers + /// them all. + /// + /// Generated from the same list as the variants, so it is exhaustive + /// by construction rather than by anyone remembering. + pub const ALL: &'static [Self] = &[ $( Self::$variant, )+ ]; + + /// How the report names this state, for a violation a reader has to + /// act on. + #[must_use] + pub const fn described(self) -> &'static str { + match self { + $( Self::$variant => $described, )+ + } + } + } + }; +} + +blocking_states! { + /// A level was named as the outermost partitioning cache with no summary. + /// + /// The state the renderer prints as `BUG IN THIS PROBE`, and the defect this + /// component exists because of. + PartitioningSummaryMissing => "summary missing for the outermost partitioning cache", + /// The enumeration recorded anomalies. + EnumerationAnomalies => "the enumeration recorded anomalies", + /// The topology was not measured from a running machine. + NotMeasured => "the topology was not measured from a running machine", + /// No cache levels were reported. + NoCacheLevels => "no cache levels were reported", + /// No packages were reported, though the machine has one. + NoPackages => "no packages were reported", + /// No cores were reported, though the machine has one. + NoCores => "no cores were reported", + /// A core record contradicts itself. + ContradictoryCore => "a core record contradicts itself", + /// A cache level is numbered 0, which Windows does not report. + UnnumberedCacheLevel => "a cache level is numbered 0", + /// The crate's two enumerations did not agree. + EnumerationsDisagreed => "the crate's two enumerations did not agree", + /// Coherence between the two enumerations was never established. + /// + /// **Distinct from `EnumerationsDisagreed`, and the separation is load + /// bearing.** One branch covered both, on the reading that anything other + /// than `Agreed` forbids agreement -- true of the VERDICT and false of the + /// ROW, which publishes `coherence_not_collected` here and + /// `enumerations_disagreed` there. A state that names the wrong code makes + /// the per-state publication rule demand something the renderer never emits. + /// Latent until a corpus shape reached it. Found by a review. + CoherenceNotCollected => "coherence between the two enumerations was not collected", + /// The bracket did not establish that the machine held still. + BracketNotHeld => "the bracket did not establish that the machine held still", +} + +/// A state that forbids an agreeing verdict, found beside one. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum Violation { + /// The observation is in a state that must prevent `agree`, and did not. + StateWithAgreeingVerdict { + /// The state. + state: BlockingState, + }, + /// `agree` without the comparison it asserts having been made. + /// + /// `agree` is the claim that every check this probe could make WAS made and + /// matched, so a counter that reported failure cannot sit beside it. + AgreedWithoutComparingCounter { + /// The counter that was not compared. + counter: &'static str, + }, + /// `agree` beside a counter that does not equal what was enumerated. + AgreedDespiteCounterMismatch { + /// The counter. + counter: &'static str, + /// What the parse carried. + parsed: usize, + /// What the counter reported. + read: usize, + }, + /// `agree` beside a NUMA highest-node number that does not match the parse. + /// + /// Separate from [`Violation::AgreedDespiteCounterMismatch`] because the + /// quantity is different in kind: a largest node NUMBER, optional on both + /// sides, rather than a count. Folding it into the count-shaped variant + /// would have meant inventing a `usize` for an absent parse. + AgreedDespiteNumaMismatch { + /// What the parse carried, which may be nothing. + parsed: Option, + /// What `GetNumaHighestNodeNumber` reported. + counter: u32, + }, +} + +impl fmt::Display for Violation { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + match self { + Self::StateWithAgreeingVerdict { state } => write!( + f, + "the observation is in the `{}` state, which forbids an \ + agreeing verdict, but the verdict is `agree` -- so whatever in \ + `cross_check` should have reported this state did not", + state.described(), + ), + Self::AgreedWithoutComparingCounter { counter } => write!( + f, + "the verdict is `agree`, which asserts every check was made, \ + but {counter} reported failure so its comparison was not made" + ), + Self::AgreedDespiteCounterMismatch { + counter, + parsed, + read, + } => write!( + f, + "the verdict is `agree` but the parse carries {parsed} where \ + {counter} read {read}" + ), + Self::AgreedDespiteNumaMismatch { parsed, counter } => match parsed { + Some(parsed) => write!( + f, + "the verdict is `agree` but the parse's highest NUMA node is \ + {parsed} where GetNumaHighestNodeNumber read {counter}" + ), + None => write!( + f, + "the verdict is `agree` but the parse reports no NUMA node at \ + all where GetNumaHighestNodeNumber read {counter}" + ), + }, + } + } +} + +/// Every state `observation` is in that forbids an agreeing verdict. +/// +/// **Computed from the observation, never from the cross-check's lists.** Each +/// of these has a push site in [`Observation::cross_check`], and the point of +/// enumerating them separately is that a deleted push site leaves the state +/// here and the list empty -- so this is what notices, and the list could not. +/// +/// **The converse does NOT hold, and the wording here used to imply it did.** +/// This said the absentees were "derived counts whose only source IS the +/// cross-check's own arithmetic", which would make their exclusion principled. +/// It is false for most of them: `CacheLevelsWithoutPartitions`, +/// `NumaDomainsOnlyInCpuSets`, `CoresOnlyInCpuSets`, `RelationsWithoutProcessors` +/// and the rest read fields that sit on [`Observation`] in plain sight. So +/// deleting one of THOSE push sites is not caught here -- the verdict can reach +/// `agree` with `blocking_states` silent -- and the guarantee above covers the +/// states actually listed below, not every condition `cross_check` can find. +/// Reported by three review rounds against two different variants of the claim. +/// +/// The tautology argument is still the right test for what belongs here; it +/// simply was not what excluded these. Closing the gap is queued as **M4.1** in +/// [CHECKLIST.md](../../CHECKLIST.md) rather than recorded only here, because a +/// decision written in a comment schedules nothing. +#[must_use] +pub fn blocking_states(observation: &Observation) -> Vec { + let mut states = Vec::new(); + + // The defect this component exists because of: the renderer prints this + // state as `BUG IN THIS PROBE ... Nothing below about cache partitioning + // can be trusted`, and `cross_check` had no branch for it. + if matches!( + observation.partitioning_cache(), + PartitioningCache::SummaryMissing(_) + ) { + states.push(BlockingState::PartitioningSummaryMissing); + } + + if !observation.enumeration_anomalies.is_empty() { + states.push(BlockingState::EnumerationAnomalies); + } + + if !observation.topology_was_measured { + states.push(BlockingState::NotMeasured); + } + + if observation.caches.is_empty() { + states.push(BlockingState::NoCacheLevels); + } + + if observation.online_processors > 0 && observation.packages == 0 { + states.push(BlockingState::NoPackages); + } + + if observation.online_processors > 0 && observation.cores.is_empty() { + states.push(BlockingState::NoCores); + } + + if observation + .cores + .iter() + .any(super::CoreShape::contradicts_itself) + { + states.push(BlockingState::ContradictoryCore); + } + + if observation.caches.iter().any(|cache| cache.level == 0) { + states.push(BlockingState::UnnumberedCacheLevel); + } + + // **Matched by variant rather than by `!= Agreed`**, because `cross_check` + // files these under different codes and a state must name the code its own + // condition emits. `Coherence` is not `#[non_exhaustive]`, so this match is + // compiler-exhaustive and a fourth variant cannot be silently folded into + // whichever arm happens to be nearest -- which is what the `!= Agreed` form + // did to `NotCollected`. + // + // **No `&` on the scrutinee, and none is needed.** Three review rounds have + // reported this and the two like it in `diagnostic.rs` as moving a + // non-`Copy` field out of a shared reference. Matching a place expression + // behind a `&` is a move only when a PATTERN BINDING moves a non-`Copy` + // value; every arm below is unit-like or `{ .. }`, so nothing is bound at + // all. `cargo check --all-targets` is clean across the workspace, and has + // been on every commit these lines have existed. + match observation.coherence { + Coherence::Agreed => {} + Coherence::Disagreed { .. } => states.push(BlockingState::EnumerationsDisagreed), + Coherence::NotCollected => states.push(BlockingState::CoherenceNotCollected), + } + + if observation.bracket != super::BracketOutcome::HeldStill { + states.push(BlockingState::BracketNotHeld); + } + + states +} + +/// Every invariant relating `observation` to `verdict` that does not hold. +/// +/// Empty is the answer for every pair this crate can produce, because +/// `cross_check` is what makes these hold. A non-empty result means a push site +/// or a guard in `cross_check` stopped reporting a state the observation still +/// shows. +/// +/// **Takes the verdict rather than deriving it**, so a test can supply the +/// answer a broken `cross_check` would give and see the rule fire. Deriving it +/// would leave every branch reachable only by editing the source, and a green +/// run would carry no information about whether the branch works. +#[must_use] +pub fn check(observation: &Observation, verdict: Verdict) -> Vec { + if verdict != Verdict::Agree { + return Vec::new(); + } + + let mut found: Vec = blocking_states(observation) + .into_iter() + .map(|state| Violation::StateWithAgreeingVerdict { state }) + .collect(); + + // Zero is how both counters report failure, so a zero beside `agree` is the + // verdict claiming a comparison that could not have happened. + // + // **The NUMA counter is held too, one rule further down, and the reason it + // was once absent is worth keeping because it was half right.** It reports + // the largest node NUMBER rather than a count, so there is no enumerated + // quantity to hold it to -- nodes 0 and 2 are a valid sparse topology, and + // comparing it against `numa_domains` would manufacture a violation on + // hardware reporting itself correctly. That argument rules out ONE + // comparison. It does not rule out the comparison `cross_check` actually + // makes, which is highest-against-highest, and excluding NUMA from here on + // the strength of it left the module's own claim -- that a push site deleted + // from `cross_check` fires a rule here -- false for precisely those two push + // sites. Found by a review. + for (counter, parsed, read) in [ + ( + "GetActiveProcessorCount", + observation.online_processors, + observation.raw_active_processors as usize, + ), + ( + "GetActiveProcessorGroupCount", + observation.groups, + observation.raw_group_count as usize, + ), + ] { + if read == 0 { + found.push(Violation::AgreedWithoutComparingCounter { counter }); + } else if parsed != read { + found.push(Violation::AgreedDespiteCounterMismatch { + counter, + parsed, + read, + }); + } + } + + // The NUMA comparison, in the shape `cross_check` makes it: highest node + // number against highest node number, never against a count. `agree` is + // reachable only through both of that function's NUMA branches declining to + // fire, so beside `agree` the counter must have been readable AND equal. + match observation.raw_highest_numa_node { + None => found.push(Violation::AgreedWithoutComparingCounter { + counter: "GetNumaHighestNodeNumber", + }), + Some(counter) if observation.highest_numa_node != Some(counter) => { + found.push(Violation::AgreedDespiteNumaMismatch { + parsed: observation.highest_numa_node, + counter, + }); + } + Some(_) => {} + } + + found +} + +/// [`check`], as an assertion, for the call sites that are bound to it. +/// +/// **Replacing this body with `()` survives a mutation sweep, and no test can +/// change that.** Recorded here rather than left for the next sweep to +/// re-discover, because the argument is short and the alternative is a test +/// manufactured to reach code nothing can reach. +/// +/// It derives the verdict from `observation.cross_check()`, so the pair it +/// checks is always the pair the crate itself produces -- and [`check`]'s rules +/// hold for every such pair by construction, as the module header explains: they +/// are reachable by CODE CHANGE, not by data. That is precisely why [`check`] +/// takes the verdict as a PARAMETER, letting a test supply the answer a broken +/// `cross_check` would give; this function has no such seam, so there is no +/// observation for which it panics and nothing to distinguish it from `()`. +/// +/// The same survivor was recorded for `assert_row_is_well_formed` on PR #88 -- see +/// [DESIGN-RATIONALE.md](../../DESIGN-RATIONALE.md) -- for the same reason: every +/// instrument that would notice goes THROUGH it. A binding that cannot fail on +/// data is checked by the sweep's `caught` results on [`check`] itself, which is +/// where the behaviour lives. +/// +/// # Panics +/// +/// Panics listing every invariant the observation violated. +pub fn assert_holds(observation: &Observation) { + let violations = check(observation, observation.cross_check().verdict()); + assert!( + violations.is_empty(), + "an observation violated {} invariant(s) relating it to its verdict:\n{}", + violations.len(), + violations + .iter() + .map(|violation| format!(" - {violation}")) + .collect::>() + .join("\n"), + ); +} + +/// Named so the module's own import of [`CrossCheck`] is not dead. +/// +/// `assert_holds` reaches the verdict through it, and a reader looking for the +/// relation between the two types should find it stated rather than inferred. +const _: fn(&CrossCheck) -> Verdict = CrossCheck::verdict; diff --git a/crates/windows-platform-probes/src/topology/invariant/tests.rs b/crates/windows-platform-probes/src/topology/invariant/tests.rs new file mode 100644 index 000000000..b5fc36149 --- /dev/null +++ b/crates/windows-platform-probes/src/topology/invariant/tests.rs @@ -0,0 +1,634 @@ +// Copyright (c) Mike Grier. + +//! Tests for the observation-to-verdict invariants. +//! +//! Every violation branch is reachable here because [`super::check`] takes the +//! VERDICT rather than deriving it. That is not a convenience: `cross_check` is +//! what makes these invariants hold, so no observation can violate them while it +//! is correct, and a derived version would leave each branch reachable only by +//! editing the source. Supplying the verdict lets a test hand over the answer a +//! broken `cross_check` would give. +//! +//! Half of these assert ACCEPTANCE. An invariant that fires on a legal pair +//! costs a reader more than one that misses an illegal one, because noise trains +//! them to ignore the instrument -- the same rule the report oracle is built on. + +use super::{BlockingState, Violation, blocking_states, check}; + +/// A named mutation of an observation, for the tests that build a corpus. +type Perturb = Box; +use crate::topology::{ + BracketOutcome, CacheLevel, CoreShape, Observation, PartitioningCache, Verdict, +}; + +/// An observation whose every check passes, to perturb one field at a time. +/// +/// Deliberately a copy of the shape `crate::tests::agreeing_observation` builds +/// rather than a call to it: that helper is private to a sibling test module, +/// and a fixture reaching across module boundaries couples two suites that +/// should be free to change apart. +fn agreeing() -> Observation { + Observation { + online_processors: 4, + groups: 1, + numa_domains: 1, + numa_domains_without_processors: 0, + numa_domains_only_in_cpu_sets: 0, + numa_domains_unreported: 0, + numa_domains_with_conflicting_labels: 0, + topology_was_measured: true, + cores_only_in_cpu_sets: 0, + cores_without_processors: 0, + packages_without_processors: 0, + overlapping_walk_relations: 0, + described_relations: 0, + unreported_relations: 0, + processor_attribute_conflicts: 0, + highest_numa_node: Some(0), + packages: 1, + cores: vec![CoreShape { + simultaneous_multithreading: true, + efficiency_class: 0, + processors: 4, + }], + caches: vec![CacheLevel { + level: 1, + processors_per_domain: vec![4], + }], + partitioning_cache_level: None, + enumeration_anomalies: Vec::new(), + coherence: windows_topology_sys::Coherence::Agreed, + raw_active_processors: 4, + raw_group_count: 1, + raw_highest_numa_node: Some(0), + bracket: BracketOutcome::HeldStill, + } +} + +#[test] +fn the_observation_this_crate_produces_is_in_no_blocking_state() { + // The acceptance half, and the premise every perturbation below rests on: + // if the fixture already blocked agreement, each test would be asserting + // against two states instead of the one it introduced. + assert_eq!(blocking_states(&agreeing()), Vec::::new()); + assert_eq!(check(&agreeing(), Verdict::Agree), Vec::new()); +} + +#[test] +fn the_verdict_the_crate_actually_draws_holds_every_invariant() { + // Against the REAL pair rather than a supplied verdict, so this would catch + // an invariant that is wrong about what `cross_check` does. + let observation = agreeing(); + let verdict = observation.cross_check().verdict(); + + assert_eq!(verdict, Verdict::Agree); + assert_eq!(check(&observation, verdict), Vec::new()); +} + +/// Each blocking state, the field that produces it, and the name it reports. +/// +/// A table rather than a test each, because the property is identical and the +/// interesting part is that NONE of them is missing. That completeness is +/// asserted by `every_blocking_state_has_a_perturbation`, which compares this +/// table against what `blocking_states` can actually produce -- so a state added +/// to the invariant with no entry here fails rather than going quietly +/// unexercised. +/// +/// This doc claimed such a relation before one existed. Found by a review: the +/// table happened to match, which is the condition under which nobody notices. +type Perturbation = (BlockingState, Perturb); + +fn perturbations() -> Vec { + vec![ + ( + BlockingState::PartitioningSummaryMissing, + Box::new(|o: &mut Observation| o.partitioning_cache_level = Some(9)), + ), + ( + BlockingState::EnumerationAnomalies, + Box::new(|o: &mut Observation| { + o.enumeration_anomalies = vec![windows_topology_sys::EnumerationAnomaly { + source: windows_topology_sys::Source::CpuSets, + offset: 0, + kind: windows_topology_sys::AnomalyKind::TrailingBytes { remaining: 3 }, + }]; + }), + ), + ( + BlockingState::NotMeasured, + Box::new(|o: &mut Observation| o.topology_was_measured = false), + ), + ( + BlockingState::NoCacheLevels, + Box::new(|o: &mut Observation| o.caches = Vec::new()), + ), + ( + BlockingState::NoPackages, + Box::new(|o: &mut Observation| o.packages = 0), + ), + ( + BlockingState::NoCores, + Box::new(|o: &mut Observation| o.cores = Vec::new()), + ), + ( + BlockingState::ContradictoryCore, + Box::new(|o: &mut Observation| { + o.cores = vec![CoreShape { + simultaneous_multithreading: false, + efficiency_class: 0, + processors: 4, + }]; + }), + ), + ( + BlockingState::UnnumberedCacheLevel, + Box::new(|o: &mut Observation| { + o.caches = vec![CacheLevel { + level: 0, + processors_per_domain: vec![4], + }]; + }), + ), + ( + BlockingState::EnumerationsDisagreed, + Box::new(|o: &mut Observation| { + o.coherence = windows_topology_sys::Coherence::Disagreed { + attempts: 2, + walk_only: Vec::new(), + cpu_sets_only: Vec::new(), + }; + }), + ), + ( + // Kept distinct from the `Disagreed` row above, because the row + // publishes a different code for each and a shared perturbation + // would leave one of the two codes unexercised -- which is exactly + // how the states came to be conflated. + BlockingState::CoherenceNotCollected, + Box::new(|o: &mut Observation| { + o.coherence = windows_topology_sys::Coherence::NotCollected; + }), + ), + ( + BlockingState::BracketNotHeld, + Box::new(|o: &mut Observation| o.bracket = BracketOutcome::Changed), + ), + ] +} + +#[test] +fn every_blocking_state_has_a_perturbation() { + // **Compared against `BlockingState::ALL`, not against the table itself.** + // The first version of this guard derived BOTH of its sets from + // `perturbations()` -- the reached set by applying them, the labelled set by + // reading them -- so a new branch in `blocking_states` that no mutation + // activated appeared in neither, and both loops stayed green. It could only + // confirm that existing labels described existing mutations, which is not + // what its name claims. Found by a review, one round after the guard was + // added in response to an earlier one. + // + // **`ALL` is exhaustive by construction, not by the `described()` match.** + // This comment used to claim the latter, and it was false: the match forces + // a new variant to acquire an ARM, never an entry in a separate array. + // Measured -- a variant absent from a hand-written `ALL` compiled and left + // all ten tests here green. `BlockingState` is now declared by a macro from + // one list, so the variants and `ALL` are the same list. + // + // The reverse loop below is NOT a substitute for that. It catches a state + // `blocking_states` produces and `ALL` omits, but only once some + // perturbation reaches it -- and a state with no perturbation entry is + // precisely the case this test exists to catch, so relying on it would be + // circular in exactly the case that matters. + let mut reached: Vec = Vec::new(); + for (_, mutate) in perturbations() { + let mut observation = agreeing(); + mutate(&mut observation); + for state in blocking_states(&observation) { + if !reached.contains(&state) { + reached.push(state); + } + } + } + + let labelled: Vec = perturbations() + .into_iter() + .map(|(state, _)| state) + .collect(); + + for state in BlockingState::ALL { + assert!( + labelled.contains(state), + "`{state:?}` is a blocking state with no entry in the perturbation \ + table, so nothing shows that it fires or that `cross_check` \ + already forbids it" + ); + assert!( + reached.contains(state), + "`{state:?}` has a table entry whose mutation does not actually \ + produce it, so the row for it tests nothing" + ); + } + + for state in &reached { + assert!( + BlockingState::ALL.contains(state), + "`{state:?}` is reported by `blocking_states` and missing from \ + `BlockingState::ALL`" + ); + } +} + +#[test] +fn every_blocking_state_forbids_an_agreeing_verdict() { + for (state, mutate) in perturbations() { + let mut observation = agreeing(); + mutate(&mut observation); + + assert!( + blocking_states(&observation).contains(&state), + "{state:?}: the observation is in this state and `blocking_states` \ + did not say so" + ); + assert!( + check(&observation, Verdict::Agree) + .contains(&Violation::StateWithAgreeingVerdict { state }), + "{state:?}: the state is present beside an agreeing verdict and the \ + invariant did not fire" + ); + } +} + +#[test] +fn every_blocking_state_is_one_the_real_cross_check_already_reports() { + // **The invariants must agree with `cross_check`, or they are a second + // opinion rather than a postcondition.** For each state, the verdict the + // crate actually draws must already be something other than `agree` -- so + // the invariant is pinning behaviour that exists rather than demanding + // behaviour that does not. + // + // This is what makes the whole module a postcondition. If one of these + // failed, the right response would be to fix `cross_check`, not to relax + // the invariant. + for (state, mutate) in perturbations() { + let mut observation = agreeing(); + mutate(&mut observation); + + let cross_check = observation.cross_check(); + assert_ne!( + cross_check.verdict(), + Verdict::Agree, + "{state:?}: the invariant forbids `agree` here, so `cross_check` must \ + already forbid it: {cross_check:?}" + ); + assert_eq!( + check(&observation, cross_check.verdict()), + Vec::new(), + "{state:?}: and against the real verdict there is nothing to report" + ); + } +} + +#[test] +fn a_blocking_state_is_silent_when_the_verdict_already_admits_it() { + // The acceptance half of the whole module: these rules constrain the + // AGREEING verdict and nothing else. A report that says it is incomplete is + // free to be in any of these states -- that is what incomplete means. + for verdict in [Verdict::Incomplete, Verdict::Disagree] { + for (state, mutate) in perturbations() { + let mut observation = agreeing(); + mutate(&mut observation); + + assert_eq!( + check(&observation, verdict), + Vec::new(), + "{state:?}: {verdict:?} admits the doubt, so there is nothing to \ + contradict" + ); + } + } +} + +#[test] +fn the_partitioning_state_is_read_from_the_observation_not_the_list() { + // **The distinction the module is built on, asserted rather than described.** + // A rule reading `parse_incomplete` cannot catch a DELETED push site: the + // deletion empties the list, so the rule sees nothing and the verdict is + // `agree` legitimately. Reading the observation is what survives that. + // + // Expressed here as: the state is visible with no reference to the + // cross-check at all. + let mut observation = agreeing(); + observation.partitioning_cache_level = Some(9); + + assert!(matches!( + observation.partitioning_cache(), + PartitioningCache::SummaryMissing(9) + )); + assert!( + blocking_states(&observation).contains(&BlockingState::PartitioningSummaryMissing), + "read from the observation, with the cross-check never consulted" + ); +} + +#[test] +fn an_agreeing_verdict_requires_the_counters_to_have_been_read() { + // Zero is how both counters report failure, so `agree` beside a zero is the + // verdict claiming a comparison that could not have happened. + for (counter, mutate) in [ + ( + "GetActiveProcessorCount", + Box::new(|o: &mut Observation| o.raw_active_processors = 0) + as Box, + ), + ( + "GetActiveProcessorGroupCount", + Box::new(|o: &mut Observation| o.raw_group_count = 0), + ), + ] { + let mut observation = agreeing(); + mutate(&mut observation); + + assert_eq!( + check(&observation, Verdict::Agree), + vec![Violation::AgreedWithoutComparingCounter { counter }], + "{counter}: a failed read cannot sit beside a verdict claiming \ + every check was made" + ); + + let cross_check = observation.cross_check(); + assert_ne!( + cross_check.verdict(), + Verdict::Agree, + "{counter}: and the real cross-check already forbids it: \ + {cross_check:?}" + ); + } +} + +#[test] +fn an_agreeing_verdict_requires_the_counters_to_have_matched() { + for (counter, parsed, read, mutate) in [ + ( + "GetActiveProcessorCount", + 4, + 8, + Box::new(|o: &mut Observation| o.raw_active_processors = 8) + as Box, + ), + ( + "GetActiveProcessorGroupCount", + 1, + 2, + Box::new(|o: &mut Observation| o.raw_group_count = 2), + ), + ] { + let mut observation = agreeing(); + mutate(&mut observation); + + assert_eq!( + check(&observation, Verdict::Agree), + vec![Violation::AgreedDespiteCounterMismatch { + counter, + parsed, + read, + }], + "{counter}: the enumeration and the counter differ, so `agree` is \ + not available" + ); + } +} + +#[test] +fn the_numa_counter_is_deliberately_not_held_to_the_enumeration() { + // `GetNumaHighestNodeNumber` reports the largest node NUMBER, which Windows + // does not promise equals the node count -- nodes 0 and 2 are a valid + // sparse topology. Holding it to `numa_domains` would manufacture a + // violation on hardware reporting itself correctly, which is the same + // over-claim `cross_check` was corrected to stop making. + // + // A sparse topology is the whole point of this case: `numa_domains` is 2 + // while the highest node number is also 2, so the two differ by one and a + // count-shaped rule would fire. The highest-against-highest rule below does + // not, which is what makes them different rules rather than one rule that + // was left out. + let mut observation = agreeing(); + observation.numa_domains = 2; + observation.highest_numa_node = Some(2); + observation.raw_highest_numa_node = Some(2); + + assert_eq!(observation.cross_check().verdict(), Verdict::Agree); + assert_eq!(check(&observation, Verdict::Agree), Vec::new()); +} + +#[test] +fn the_processor_guard_agrees_with_the_one_cross_check_applies() { + // **Two copies of one boundary, deliberately, and nothing held them to each + // other.** `blocking_states` recomputes `online_processors > 0 && packages + // == 0` rather than asking `cross_check`, and it MUST: a rule that reads + // `cross_check`'s output restates `verdict()` and is blind to a deleted push + // site, which is the whole reason this module exists. Independence is the + // design; agreement is the property, and the property was untested. + // + // Found by a mutation sweep, not by review. Relaxing either `>` to `>=` here + // survived five review rounds across four models, because the test that + // names this boundary -- + // `a_topology_with_no_processors_at_all_is_not_accused_of_hiding_packages` + // -- asserts on `cross_check` and so covers only the OTHER copy. + // + // Written as a correspondence over a corpus that spans the boundary rather + // than as a single case, so a future guard whose condition drifts on either + // side is caught wherever it drifts. + let shapes: [(&str, Perturb); 4] = [ + ( + "no processors, and nothing else reported either", + Box::new(|o: &mut Observation| { + o.online_processors = 0; + o.raw_active_processors = 0; + o.packages = 0; + o.cores = Vec::new(); + }), + ), + ( + "no processors, but packages and cores reported", + Box::new(|o: &mut Observation| { + o.online_processors = 0; + o.raw_active_processors = 0; + }), + ), + ( + "processors reported, packages absent", + Box::new(|o: &mut Observation| o.packages = 0), + ), + ( + "processors reported, cores absent", + Box::new(|o: &mut Observation| o.cores = Vec::new()), + ), + ]; + + for (shape, mutate) in shapes { + let mut observation = agreeing(); + mutate(&mut observation); + + let states = blocking_states(&observation); + let parse = observation.cross_check().parse_incomplete; + + assert_eq!( + states.contains(&BlockingState::NoPackages), + parse.contains(&crate::topology::ParseIncomplete::NoPackages), + "{shape}: this module and `cross_check` disagree about whether \ + absent packages are a finding -- states {states:?}, parse {parse:?}" + ); + assert_eq!( + states.contains(&BlockingState::NoCores), + parse.contains(&crate::topology::ParseIncomplete::NoCores), + "{shape}: this module and `cross_check` disagree about whether \ + absent cores are a finding -- states {states:?}, parse {parse:?}" + ); + } +} + +#[test] +fn a_corpus_spanning_the_processor_guard_reaches_both_of_its_answers() { + // The correspondence above compares two computations, so it passes + // vacuously if every shape lands on the same side of the boundary. Assert + // that the corpus reaches BOTH answers, or the test is a comparison of two + // constants. + let mut absent = agreeing(); + absent.packages = 0; + absent.cores = Vec::new(); + assert!( + blocking_states(&absent).contains(&BlockingState::NoPackages), + "a machine with processors and no packages must be in the state" + ); + + let mut unmeasured = agreeing(); + unmeasured.online_processors = 0; + unmeasured.raw_active_processors = 0; + unmeasured.packages = 0; + unmeasured.cores = Vec::new(); + assert!( + !blocking_states(&unmeasured).contains(&BlockingState::NoPackages), + "a topology that reported no processors is not accused of hiding \ + packages -- this is the `> 0` the sweep found unguarded" + ); + assert!( + !blocking_states(&unmeasured).contains(&BlockingState::NoCores), + "nor of hiding cores" + ); +} + +#[test] +fn an_agreeing_verdict_requires_the_numa_counter_to_have_been_read() { + // The branch `cross_check` takes when `GetNumaHighestNodeNumber` fails: it + // files `HighestNumaNodeFailed` and returns, so `agree` is unreachable. + // Delete that push and `agree` becomes reachable beside an unread counter, + // which is the defect this rule exists to name. + let mut observation = agreeing(); + observation.raw_highest_numa_node = None; + + assert_eq!( + check(&observation, Verdict::Agree), + vec![Violation::AgreedWithoutComparingCounter { + counter: "GetNumaHighestNodeNumber" + }] + ); +} + +#[test] +fn an_agreeing_verdict_requires_the_numa_counter_to_have_matched() { + // Both directions of the mismatch, because the parse's side is an `Option` + // and the absent case renders differently -- a rule whose message says + // "carries None" where a reader expected a number is a rule that will be + // misread in the one situation it fires. + let mut mismatched = agreeing(); + mismatched.highest_numa_node = Some(1); + mismatched.raw_highest_numa_node = Some(3); + + assert_eq!( + check(&mismatched, Verdict::Agree), + vec![Violation::AgreedDespiteNumaMismatch { + parsed: Some(1), + counter: 3 + }] + ); + + let mut unparsed = agreeing(); + unparsed.highest_numa_node = None; + unparsed.raw_highest_numa_node = Some(3); + + assert_eq!( + check(&unparsed, Verdict::Agree), + vec![Violation::AgreedDespiteNumaMismatch { + parsed: None, + counter: 3 + }] + ); + assert!( + Violation::AgreedDespiteNumaMismatch { + parsed: None, + counter: 3 + } + .to_string() + .contains("no NUMA node at all"), + "the absent case must not render as a number" + ); +} + +#[test] +fn every_numa_counter_branch_in_the_real_cross_check_is_one_this_module_forbids() { + // **The claim in this module's header, checked rather than asserted**: a + // push site deleted from `cross_check` fires a rule here. For each NUMA + // COUNTER branch, the verdict the real `cross_check` draws must already be + // something other than `agree`, AND this module must forbid `agree` for the + // same observation -- so the rule pins behaviour that exists rather than + // demanding behaviour that does not. + // + // **Named for the counter on purpose.** This was + // `every_numa_branch_...`, which was false: `cross_check` also pushes + // `NumaDomainsOnlyInCpuSets`, `NumaDomainsUnreported` and + // `NumaDomainsWithConflictingLabels`, none of which has a `BlockingState`, + // so deleting one of those push sites is not caught. A review found the + // name claiming the whole family while the body covered the two branches + // that read `raw_highest_numa_node`. The absentees are queued as M4.1; the + // name now says which half is guarded. + // + // This is the pairing the other states get from + // `every_blocking_state_is_one_the_real_cross_check_already_reports`; NUMA + // had neither half until a review found the header's claim was false for + // exactly these two branches. + for (what, mutate) in [ + ( + "counter unreadable", + Box::new(|o: &mut Observation| o.raw_highest_numa_node = None) + as Box, + ), + ( + "counter disagrees with the parse", + Box::new(|o: &mut Observation| { + o.highest_numa_node = Some(1); + o.raw_highest_numa_node = Some(3); + }), + ), + ] { + let mut observation = agreeing(); + mutate(&mut observation); + + let cross_check = observation.cross_check(); + assert_ne!( + cross_check.verdict(), + Verdict::Agree, + "{what}: `cross_check` must already forbid `agree`: {cross_check:?}" + ); + assert_ne!( + check(&observation, Verdict::Agree), + Vec::new(), + "{what}: and this module must forbid it too, or a deleted push site \ + here fires nothing" + ); + assert_eq!( + check(&observation, cross_check.verdict()), + Vec::new(), + "{what}: the verdict the crate actually draws must hold every rule" + ); + } +} diff --git a/crates/windows-platform-probes/src/topology_report.rs b/crates/windows-platform-probes/src/topology_report.rs index 869664bba..cb62f5850 100644 --- a/crates/windows-platform-probes/src/topology_report.rs +++ b/crates/windows-platform-probes/src/topology_report.rs @@ -19,7 +19,109 @@ use std::io; use windows_placement_probe::fingerprint::{Fingerprint, banner_line_for}; -use crate::topology::{Observation, PartitioningCache, Verdict}; +use crate::row::{Row, Shape, Value}; + +/// What every entry in one of the four diagnostic lists must carry. +/// +/// `code` is the stable discriminant a survey groups by, so an entry without one +/// is unmineable even though it is valid JSON. The payload beside it is +/// per-variant and so is not a member of this schema -- it is checked against +/// the typed publisher by +/// `the_row_carries_each_diagnostic_entry_whole_and_not_only_its_code`. +const CODED: &[(&str, Shape)] = &[("code", Shape::Text)]; + +/// Declares a row schema once, as names WITH shapes, and derives the key list. +/// +/// One list, so a key cannot gain a shape without gaining a name or the reverse. +/// The alternative -- a `_KEYS` const beside a `_SHAPES` const -- is two +/// statements of one schema, which is the restatement this crate keeps paying +/// for. +macro_rules! row_schema { + ( + $(#[$keys_doc:meta])* $keys:ident, + $(#[$shapes_doc:meta])* $shapes:ident { $($name:literal => $shape:expr,)+ } + ) => { + $(#[$keys_doc])* + pub const $keys: &[&str] = &[$($name,)+]; + + $(#[$shapes_doc])* + pub const $shapes: &[(&str, Shape)] = &[$(($name, $shape),)+]; + }; +} + +row_schema!( + /// Every key a MEASURED topology row carries, in order. + /// + /// **This is the contract, not a census of the code.** The anti-census rule + /// this crate keeps relearning is about restating facts that can be DERIVED + /// -- a count of placeholders, a tally of variants. A schema is not + /// derivable from anything: it IS the agreement with the survey that reads + /// these rows, so writing it down is what makes it checkable at all. + /// + /// It was missing, and the gap was measured: with `packages` deleted from + /// the builder entirely, the whole suite stayed green. `Row::keys` reports + /// what the builder happened to supply, so a test comparing the two only + /// ever showed the reader and the writer agreeing with each other -- never + /// that a field the survey depends on is still there. Found by a review. + /// + /// Changing this list is a breaking change to the row, and + /// `the_measured_row_carries_exactly_the_contracts_keys` is what makes that + /// visible in a diff rather than in a mining pass six months later. + MEASURED_ROW_KEYS, + /// The same schema with each key's value SHAPE, which is the half the key + /// list alone could not state. + /// + /// Names and order say WHICH fields a row carries; they say nothing about + /// what those fields hold. Measured before this existed: publishing + /// `processors` through `.to_string()` -- a number becoming a string in the + /// mined artifact -- left all 230 library tests and all 10 real-host + /// integration tests green. Found by a review. + /// + /// Changing a shape here is a breaking change to the row exactly as + /// changing a name is. + MEASURED_ROW_SHAPES +{ + "reason" => Shape::Text, + "arch" => Shape::Text, + "processors" => Shape::Number, + "groups" => Shape::Number, + "packages" => Shape::Number, + "numa_domains" => Shape::Number, + "numa_domains_without_processors" => Shape::Number, + "cores" => Shape::Number, + "efficiency_classes" => Shape::ListOfNumbers, + "caches" => Shape::ListOfObjectsWith(&[("level", Shape::Number), ("domains", Shape::Number)]), + "outermost_partitioning_cache_level" => Shape::NumberOrNull, + "outermost_partitioning_cache" => Shape::Text, + "policies" => Shape::ObjectOfNumbers, + "cross_check" => Shape::Text, + "disagreements" => Shape::ListOfObjectsWith(CODED), + "not_compared" => Shape::ListOfObjectsWith(CODED), + "parse_incomplete" => Shape::ListOfObjectsWith(CODED), + "enumeration_anomalies" => Shape::ListOfObjectsWith(CODED), + "numa_domains_only_in_cpu_sets" => Shape::Number, +}); + +row_schema!( + /// Every key an UNMEASURED topology row carries, in order. + /// + /// Deliberately short, and deliberately its own schema rather than a subset + /// of the one above: a row from a host whose discovery FAILED is a + /// different shape, and a survey must be able to tell it from a measured + /// row that happens to be missing fields. + UNMEASURED_ROW_KEYS, + /// The same schema with each key's value shape. + UNMEASURED_ROW_SHAPES +{ + "reason" => Shape::Text, + "arch" => Shape::Text, + "cross_check" => Shape::Text, + "discovery_error" => Shape::Text, +}); +use crate::topology::diagnostic::{described, published_anomaly}; +use crate::topology::{ + Disagreement, NotCompared, Observation, ParseIncomplete, PartitioningCache, Verdict, +}; /// The banner and title both reports open with. /// @@ -251,18 +353,41 @@ pub fn report_unmeasured(banner: &str, error: &io::Error) -> String { out, "subject must say so instead of printing a misleading shape.)" ); + // **Through the same writer, and this is the shape that most needs it.** + // The only caller text in any report reaches THIS renderer -- a failed + // discovery's `io::Error`, whose message is whatever the OS said. Publishing + // it as a `Value::Text` is what makes a brace in that message inert rather + // than the start of a second row, which is the contamination measured on + // PR #88. + // + // The error is published as a field rather than only printed, because a + // survey counting failures wants to group them by cause, and the prose + // sentence above is not something a mining pass should be parsing. let _ = writeln!( out, - r#"{{"reason":"x-probe-topology","arch":"{}","cross_check":"not_measured"}}"#, - std::env::consts::ARCH + "{}", + Row::new("x-probe-topology") + .with("arch", std::env::consts::ARCH) + .with("cross_check", "not_measured") + .with("discovery_error", error.to_string()) + .render() ); - // Bound here too, for the reason given on `report` below. This renderer - // makes fewer claims, so fewer correspondences apply -- but "fewer apply" - // is a conclusion the oracle should reach by looking, not one assumed by - // leaving the call out. + // Bound here too, for the reason given on `report` below: an unmeasured + // report is still a report, and a survey still has to parse its row. + // + // This used to say the renderer "makes fewer claims, so fewer + // correspondences apply". There are no correspondences left to apply -- M3 + // retired the prose/row relation, and what is checked is that the row is + // well formed, which is not a thing a renderer can make less of. + // + // The SHAPES are bound here too, for the reason given on `report` below: + // names alone leave a consumer's field types unconstrained. #[cfg(any(test, feature = "oracle-in-renderer"))] - crate::report_oracle::assert_corresponds(&out); + { + crate::report_oracle::assert_row_is_well_formed(&out); + crate::report_oracle::assert_row_has_the_schemas_shapes(&out, UNMEASURED_ROW_SHAPES); + } out } @@ -279,6 +404,15 @@ pub fn report(banner: &str, observation: &Observation) -> String { // draws. A rule with one exception is not a rule, and the exception was the // conclusion drawn from the one field the two sources are known to // contradict each other about. + // **Bound here as well as in `observe`, and the two cover different + // things.** `observe` covers every observation this crate MEASURES; this + // covers every observation anyone RENDERS, which on the test side is most + // of them -- the suite builds observations by hand rather than measuring a + // machine, so binding only at `observe` would leave the invariants + // exercised on one host shape. + #[cfg(any(test, feature = "oracle-in-renderer"))] + crate::topology::invariant::assert_holds(observation); + let check = observation.cross_check(); let parse_in_doubt = check.parse_in_doubt(); @@ -595,15 +729,15 @@ pub fn report(banner: &str, observation: &Observation) -> String { Verdict::Disagree => { let _ = writeln!(out, " => DISAGREE. This is a finding, not a nuisance:"); for complaint in &check.disagreements { - let _ = writeln!(out, " - {complaint}"); + let _ = writeln!(out, " - {}", described(complaint)); } // Listed even here, so a reader knows the disagreement above is not // the whole picture. for skipped in &check.not_compared { - let _ = writeln!(out, " (not compared) {skipped}"); + let _ = writeln!(out, " (not compared) {}", described(skipped)); } for caveat in &check.parse_incomplete { - let _ = writeln!(out, " (parse incomplete) {caveat}"); + let _ = writeln!(out, " (parse incomplete) {}", described(caveat)); } } Verdict::Incomplete => { @@ -613,10 +747,10 @@ pub fn report(banner: &str, observation: &Observation) -> String { ); let _ = writeln!(out, " did not establish that the parse is consistent:"); for skipped in &check.not_compared { - let _ = writeln!(out, " - {skipped}"); + let _ = writeln!(out, " - {}", described(skipped)); } for caveat in &check.parse_incomplete { - let _ = writeln!(out, " - {caveat}"); + let _ = writeln!(out, " - {}", described(caveat)); } } } @@ -655,128 +789,171 @@ pub fn report(banner: &str, observation: &Observation) -> String { // anything sizing itself by cache boundary, on a row already certified. // Hence the `outermost_partitioning_cache` field beside it, which a // consumer must read rather than inferring from the level being absent. - let cache_json: Vec = observation - .caches - .iter() - .map(|c| format!(r#"{{"level":{},"domains":{}}}"#, c.level, c.domains())) - .collect(); - let policy_json: Vec = observation - .domain_counts() - .into_iter() - .map(|(name, count)| format!(r#""{name}":{count}"#)) - .collect(); - let _ = writeln!( - out, - concat!( - r#"{{"reason":"x-probe-topology","arch":"{}","processors":{},"groups":{},"#, - r#""packages":{},"numa_domains":{},"numa_domains_without_processors":{},"cores":{},"#, - r#""efficiency_classes":[{}],"caches":[{}],"outermost_partitioning_cache_level":{},"#, - r#""outermost_partitioning_cache":"{}","#, - r#""policies":{{{}}},"cross_check":"{}","not_compared":{},"parse_incomplete":{},"#, - r#""enumeration_anomalies":{},"numa_domains_only_in_cpu_sets":{}}}"# - ), - std::env::consts::ARCH, - observation.online_processors, - observation.groups, - observation.packages, - observation.numa_domains, - observation.numa_domains_without_processors, - observation.cores.len(), + // **One writer, one value, no positions.** The row was built by + // interpolating every value positionally into a `concat!` template, where a + // field's name and its value were related only by counting -- so a reordered + // argument or a miscounted placeholder yielded mislabelled data that still + // parses. Here a name and its value are one pair. See `crate::row`. + // + // The diagnostics publish their DATA now, not only their code: a survey + // learns `contradictory_cores` AND that three cores contradicted themselves. + // What stopped M3.1 doing that was this template. + let row = Row::new("x-probe-topology") + .with("arch", std::env::consts::ARCH) + .with("processors", observation.online_processors) + .with("groups", observation.groups) + .with("packages", observation.packages) + .with("numa_domains", observation.numa_domains) + .with( + "numa_domains_without_processors", + observation.numa_domains_without_processors, + ) + .with("cores", observation.cores.len()) // The CLASSES, not how many there are. A plural name over a count is - // ambiguous in the one way that matters here: on a single-class host - // this emitted `"efficiency_classes":1`, which reads exactly like a - // machine whose one class is class *1* -- while the prose two lines - // above printed `efficiency classes: [0]`. Same fact, same report, two - // renderings a consumer cannot reconcile. The list is what the name - // promises, agrees with the prose, and carries strictly more: a fleet - // survey can still get the count from its length, and can now also see - // WHICH classes a host reported. - classes - .iter() - .map(u8::to_string) - .collect::>() - .join(","), - cache_json.join(","), - // The level the prose names, read per variant rather than through - // `outermost_partitioning_cache`, whose `None` covers the - // summary-missing case too. Routing through it emitted - // `"outermost_partitioning_cache_level":null` beside - // `"outermost_partitioning_cache":"summary_missing"` while the prose - // printed the number -- one fact, two renderings, no way to reconcile - // them. `domain_counts` was taken off the same accessor for the same - // reason; this consumer was not swept with it. - match observation.partitioning_cache() { - PartitioningCache::Level(cache) => cache.level.to_string(), - PartitioningCache::SummaryMissing(level) => level.to_string(), - PartitioningCache::NoLevelsReported - | PartitioningCache::NoLevelPartitions - | PartitioningCache::NoUniqueOutermost => "null".to_string(), - }, - // The level alone said `null` for every absent case alike, on a line - // the verdict had already certified as "agree" -- an incomparable + // ambiguous in the one way that matters: on a single-class host this + // emitted `"efficiency_classes":1`, which reads exactly like a machine + // whose one class is class *1* -- while the prose printed + // `efficiency classes: [0]`. Same fact, two renderings a consumer cannot + // reconcile. A survey can still get the count from the length. + .with( + "efficiency_classes", + classes.iter().copied().collect::(), + ) + .with( + "caches", + observation + .caches + .iter() + .map(|cache| { + Value::Object(vec![ + ("level", Value::from(cache.level)), + ("domains", Value::from(cache.domains())), + ]) + }) + .collect::(), + ) + // The level alone said `null` for every absent case alike, on a line the + // verdict had already certified as `agree` -- an incomparable // partitioning touches nothing `cross_check` consults. A query counting // nulls as "machines no cache level partitions" then folded in machines - // where a level DOES partition, which is the opposite conclusion for - // anything sizing itself by cache boundary. Always a string, so a - // consumer filters on `== "none"` rather than on the absence of a - // number. - match observation.partitioning_cache() { - PartitioningCache::Level(_) => "level", - PartitioningCache::NoLevelsReported => "no_levels_reported", - PartitioningCache::NoLevelPartitions => "none", - PartitioningCache::NoUniqueOutermost => "not_unique", - PartitioningCache::SummaryMissing(_) => "summary_missing", - }, - policy_json.join(","), - // A tri-state rather than a boolean, for the reason the prose above - // gives: a log-mining pass over accumulated CI output must be able to - // tell "all three counters agreed" from "two agreed and the third was - // never compared". `cross_check_ok:true` said the same thing for both. - match check.verdict() { - Verdict::Agree => "agree", - Verdict::Disagree => "disagree", - Verdict::Incomplete => "incomplete", - }, - check.not_compared.len(), - // Separate from `not_compared`, because a mining pass that finds - // `"cross_check":"incomplete"` needs to know whether this probe failed - // to read a counter or the parse itself was short or disputed -- the - // first is a gap in the measurement, the second a fact about the - // machine worth going and looking at. The two counts beside it say - // which kind, without a consumer having to know what `cross_check` - // currently pushes for. - check.parse_incomplete.len(), - observation.enumeration_anomalies.len(), - observation.numa_domains_only_in_cpu_sets, - ); + // where a level DOES partition, the opposite conclusion for anything + // sizing itself by cache boundary. Hence the discriminator beside it, + // which a consumer must read rather than inferring from the absence. + // **The level a SummaryMissing arm names is published too**, not + // `null`: the two renderings of WHICH level went unchecked are what a + // reader needs when the report is telling them the probe has a bug. + .with( + "outermost_partitioning_cache_level", + match observation.partitioning_cache() { + PartitioningCache::Level(cache) => Value::from(cache.level), + PartitioningCache::SummaryMissing(level) => Value::from(level), + PartitioningCache::NoLevelsReported + | PartitioningCache::NoLevelPartitions + | PartitioningCache::NoUniqueOutermost => Value::Null, + }, + ) + .with( + "outermost_partitioning_cache", + match observation.partitioning_cache() { + PartitioningCache::Level(_) => "level", + PartitioningCache::NoLevelsReported => "no_levels_reported", + PartitioningCache::NoLevelPartitions => "none", + PartitioningCache::NoUniqueOutermost => "not_unique", + PartitioningCache::SummaryMissing(_) => "summary_missing", + }, + ) + .with( + "policies", + Value::Object( + observation + .domain_counts() + .into_iter() + .map(|(name, count)| (name, Value::from(count))) + .collect(), + ), + ) + // A tri-state rather than a boolean: a mining pass must be able to tell + // "all three counters agreed" from "two agreed and the third was never + // compared". `cross_check_ok:true` said the same thing for both. + .with( + "cross_check", + match check.verdict() { + Verdict::Agree => "agree", + Verdict::Disagree => "disagree", + Verdict::Incomplete => "incomplete", + }, + ) + .with( + "disagreements", + check + .disagreements + .iter() + .map(Disagreement::published) + .collect::(), + ) + .with( + "not_compared", + check + .not_compared + .iter() + .map(NotCompared::published) + .collect::(), + ) + .with( + "parse_incomplete", + check + .parse_incomplete + .iter() + .map(ParseIncomplete::published) + .collect::(), + ) + .with( + "enumeration_anomalies", + observation + .enumeration_anomalies + .iter() + .map(published_anomaly) + .collect::(), + ) + .with( + "numa_domains_only_in_cpu_sets", + observation.numa_domains_only_in_cpu_sets, + ); + + let _ = writeln!(out, "{}", row.render()); // **Bound here rather than called from each test, which is the difference // between an oracle and three more tests.** A test added beside the others // checks one case; binding the renderer checks every case anyone writes // later, including the ones nobody thought to add. // - // **Measured, not assumed.** Re-introducing a cross-part contradiction -- - // the NDJSON processor count one higher than the prose -- turns 13 existing - // tests red through this line, none of which was written about processor - // counts: they are about cache notes, efficiency classes and caveats, and - // they inherit the check purely by rendering a report. With the same - // contradiction in place and this line removed, EVERY LIBRARY TEST PASSES: - // the per-part tests cannot see the defect at all. + // **What it catches: a renderer that emits a malformed row.** Nothing here + // compares the prose against the row. M3 retired that relation -- the row is + // the machine contract and the prose is reviewed, not parsed -- so the only + // claim this line supports is that every report this renderer produces + // carries exactly one well-formed JSON row. // - // That sentence used to say THE WHOLE SUITE passes, which was true when it - // was written and stopped being true in the same commit -- this branch adds - // `tests/a_real_report_agrees_with_itself.rs`, whose tests call the oracle - // explicitly and so go red without the binding. Measured just now: the - // library suite is entirely green under that sabotage while the real-host - // integration tests fail. Named without a count on purpose, because the - // count moved between a reviewer measuring it and this correction being - // written, for exactly the reason the next paragraph gives. + // **Measured, not assumed** (2026-09-13, by sabotaging `Row::render` to drop + // the closing brace): 42 library tests fail with this line, 29 without it. + // The 13 that only this line catches are about cache notes, efficiency + // classes, caveats, NUMA lines and partitioning levels -- not one of them + // mentions the row's syntax. They inherit the check purely by rendering a + // report, which is the whole argument for binding the renderer instead of + // adding a test beside the others. // - // Stated as the invariant rather than as a count, because the count rots. - // This read "all 190 pass" when the suite held 190 tests, and it has grown - // several times since -- so a reviewer had to run the suite three times to - // establish that the sentence was merely stale rather than wrong. The - // number was never the point; that nothing else catches the defect is. + // **The previous version of this paragraph was false, and that is worth + // recording.** It claimed a prose/NDJSON processor-count contradiction turned + // 13 tests red through this line. That was true before M3 and silently + // stopped being true when the oracle stopped reading prose: re-run in full, + // the sabotage it names now leaves all 228 library tests AND all 10 real-host + // integration tests green. A reviewer inferred it from the code; the check + // that settled it was running it. An evidence paragraph nothing executes is + // the same rot as a test nothing runs -- so when this mechanism changes + // again, re-measure rather than re-word. + // + // The counts above are dated for that reason. The invariant is the durable + // half: nothing else in the library catches a malformed row, because no + // per-part test parses one. // // **Why the gate is not `cfg(test)` alone.** It was, and the claim above was // then false for half of what "every test" means: cargo compiles this @@ -811,7 +988,25 @@ pub fn report(banner: &str, observation: &Observation) -> String { // case, because the assertion's message carries the whole report -- what is // lost is the NDJSON row a survey would have mined, which is why the default // build is the one that matters and is the one pinned above. + // + // **The schema's SHAPES are bound here for the same reason.** Well-formed + // says the row parses; the schema says `processors` is a number and each + // diagnostic entry carries a code. Measured: publishing `processors` through + // `.to_string()` left all 230 library tests and all 10 real-host integration + // tests green before this line existed. Found by a review. + // + // **Both inside ONE `cfg` block, which the first attempt got wrong.** A + // `#[cfg]` attribute governs the single statement that follows it, so + // adding a second call beneath the gated one left that call ungated -- and + // `report_oracle` does not exist in a default build. Nothing local caught + // it: this crate's dev-dependency on itself turns `oracle-in-renderer` on + // for every `cargo test` and `cargo check --all-targets`, so the + // feature-off arm is never compiled here. CI's `cargo run --bin` is, and + // that is where it broke. #[cfg(any(test, feature = "oracle-in-renderer"))] - crate::report_oracle::assert_corresponds(&out); + { + crate::report_oracle::assert_row_is_well_formed(&out); + crate::report_oracle::assert_row_has_the_schemas_shapes(&out, MEASURED_ROW_SHAPES); + } out } diff --git a/crates/windows-platform-probes/tests/a_real_report_agrees_with_itself.rs b/crates/windows-platform-probes/tests/a_real_report_agrees_with_itself.rs index e06022e5c..9592ddf7d 100644 --- a/crates/windows-platform-probes/tests/a_real_report_agrees_with_itself.rs +++ b/crates/windows-platform-probes/tests/a_real_report_agrees_with_itself.rs @@ -1,38 +1,48 @@ // Copyright (c) Mike Grier. -//! The report this crate renders from a *real* measurement, checked against the -//! oracle. +//! The report this crate renders from a *real* measurement, checked against +//! the OBSERVATION it was rendered from. //! -//! # Why this is not a unit test +//! # What "agrees with itself" means here, since it changed +//! +//! It used to mean the report's two renderings agreed: prose against encoded +//! row. M3 retired that relation -- the row is the machine contract and the +//! prose is reviewed rather than parsed -- and this file's tests moved with it +//! without its header following. Reported by a review, which read the paragraphs +//! below as still describing what the file does. //! -//! The oracle's own tests pin it against fixtures. A fixture is a report -//! somebody wrote down, so a fixture-bound oracle checks correspondences over -//! states its author already imagined -- and the defect the oracle exists for -//! was a state nobody had imagined: `topology_report` printing `BUG IN THIS -//! PROBE ... Nothing below about cache partitioning can be trusted` while the -//! verdict two paragraphs below printed `=> agree`. +//! It now means the row agrees with the observation that produced it: every +//! condition `cross_check` found reaches the row, every state that blocks +//! agreement is published, one code per anomaly, and the row is well formed. +//! The two halves being compared are a value and its encoding, not two +//! sentences -- and that is still self-agreement, which is why the target keeps +//! its name. //! -//! More narrowly, a fixture cannot notice the **renderer** drifting away from -//! the prose labels the oracle looks for. Both sides would still agree with -//! each other; only the real artifact disagrees. +//! # Why this is not a unit test +//! +//! The other tests pin the renderer against fixtures. A fixture is an +//! observation somebody wrote down, so it exercises states its author already +//! imagined -- and the defect this file exists for was a state nobody had +//! imagined: `topology_report` printing `BUG IN THIS PROBE ... Nothing below +//! about cache partitioning can be trusted` while the verdict two paragraphs +//! below printed `=> agree`. //! //! Some unit tests in this crate do call `measure()` and so do read this host. -//! What none of them does is run the ORACLE over a report rendered from that -//! reading, which is the gap this file closes. On CI it runs across the hosted -//! runner fleet -- a slow survey of shapes no fixture anticipates. +//! What none of them does is check a report rendered from that reading, which +//! is the gap this file closes. On CI it runs across the hosted runner fleet -- +//! a slow survey of shapes no fixture anticipates. //! //! # It asserts nothing about this machine //! //! Deliberately. A test that expected a processor count, a cache level or a //! verdict would fail on the next runner shape rather than on a defect, and //! would have to be loosened until it asserted nothing. What it checks is that -//! whatever this host produced, the report's parts agree **with each other** -- -//! a property every host must satisfy, including one whose topology cannot be -//! read at all. +//! whatever this host produced, the row accounts for it -- a property every +//! host must satisfy, including one whose topology cannot be read at all. use windows_placement_probe::fingerprint::Fingerprint; use windows_platform_probes::report_oracle; -use windows_platform_probes::topology::measure; +use windows_platform_probes::topology::{invariant, measure}; use windows_platform_probes::topology_report::{attribution, report, report_unmeasured}; /// The report exactly as `probe-topology` composes it. @@ -55,944 +65,27 @@ fn real_report() -> (String, bool) { } #[test] -fn a_report_rendered_from_this_host_agrees_with_itself() { - let (text, _) = real_report(); - - // The whole assertion. Not "the report says X" -- "the report does not - // contradict itself", which is checkable without knowing anything about - // the machine. - report_oracle::assert_corresponds(&text); -} - -#[test] -fn the_oracle_is_actually_reading_this_host_s_report() { - // Without this, the test above is worth nothing on a host whose report the - // oracle cannot parse: every lookup returns `None`, every comparison is - // skipped, and it passes having checked exactly zero correspondences. +fn a_report_rendered_from_this_host_is_well_formed() { + // **The real host, which no fixture can stand in for.** A fixture is a + // report somebody wrote down, so it can only exercise shapes its author + // imagined -- and the defect this file was built around was a state nobody + // had. On CI this runs across the hosted runner fleet, a slow survey of + // shapes no fixture anticipates. // - // The unit tests pin the oracle against a fixture, which cannot notice the - // renderer drifting away from it. Only the real artifact can, and only if - // something requires a violation to appear. - // - // So a fact this host really rendered is corrupted, and the oracle must - // report it. `"processors":0` is the corruption because no host has zero - // online processors, so it disagrees with the prose on every machine - // without needing to know what the prose says. + // It asserts nothing about this MACHINE, deliberately: a test expecting a + // processor count or a verdict would fail on the next runner shape rather + // than on a defect. What it checks is that whatever this host produced, the + // row a survey will mine is well-formed -- a property every host satisfies, + // including one whose topology cannot be read at all. let (text, measured) = real_report(); - if !measured { - // `report_unmeasured` carries no counts to corrupt. A host whose - // topology cannot be read is a legitimate outcome -- "cannot measure" - // is a third answer in this crate -- and skipping is honest here in a - // way it would not be for the assertion above, which still ran. - note("topology could not be read on this host; corruption check skipped"); - return; - } - - // Every double-rendered fact, not just one. Corrupting a single field would - // leave the other pairs unguarded: the renderer could drift away from the - // oracle's other prose labels and this would still pass on the strength of - // the one. - // - // **Each corruption is chosen against the value actually rendered, and keys - // this host did not render are skipped.** An earlier version replaced every - // count with `0`, on the reasoning that no host has zero processors, groups, - // packages or cores -- which is true of those four and false of the report - // as a whole, and the difference would have failed on legitimate hosts - // rather than on a defect: - // - // * `outermost_partitioning_cache_level` is rendered `null` for the three - // absent-partitioning variants, and no prose level accompanies it, so - // rewriting it to `0` yields no correspondence to violate. - // * `"domains":` does not appear at all on a host that reports no caches. - // * A successfully measured but incomplete parse can render `0` decoded - // packages or cores, where replacing `0` with `0` changes nothing and - // the sabotage silently fails to apply. - // - // So the replacement is `1` where the report says `0` and `0` otherwise, - // which is guaranteed to differ from whatever this host rendered, and a key - // that is absent or `null` is skipped with a reason rather than asserted - // against. - // - // The skip cannot swallow the whole test, and the guard names WHICH facts it - // requires rather than how many. - // - // **A count was not enough, and the difference is not academic.** An earlier - // version asserted `exercised >= 4` against a running total, which the four - // CONDITIONAL entries can satisfy on their own -- so if the four - // unconditional keys drifted out of the renderer, all four would skip, the - // conditional four would make the total, and this test would pass having - // checked none of the facts its comment claimed it required. Demonstrated by - // a reviewer: forcing the four unconditional lookups to miss produced four - // skip messages and a PASS. The comment claimed coverage while the code - // enforced quantity, which is the defect this whole branch is about, in the - // guard written to prevent it. - let mut exercised: Vec<&str> = Vec::new(); - - // `true` where a measured report must carry the fact, so a skip is a defect - // rather than a legal shape. - // - // **Six are unconditional, not four.** An earlier version marked - // `"numa_domains":` and the `"single":` policy optional on the stated - // grounds that "the policy and NUMA entries depend on what the host has". - // They do not, and the renderer says so plainly: `report()` writes the - // `NUMA domains` prose line and the `"numa_domains"` field with no - // condition around either, and `domain_counts()` begins every policy table - // with `("single", 1)` -- a count clamped to one because there is always at - // least one domain. The claim was about the host; the truth was about the - // renderer, which is the same confusion in miniature that this whole test - // exists to catch. - // - // Marking a mandatory fact optional inverts the guard: measured, renaming - // the NDJSON `"numa_domains"` key, and separately renaming the `single` - // policy, each left this test GREEN -- so it passed precisely when the - // renderer-to-oracle binding for a mandatory fact disappeared, which is the - // one event it is here to detect. - // - // The two that remain optional are genuinely conditional: the outermost - // level is `null` on three partitioning variants, and `"domains":` is - // absent when no caches are reported. - // The fact each key must be reported under. A LIST, not one name, because - // one key can be read by different rules depending on the shape the host - // produced -- see the outermost level below. Every entry is still a specific - // fact, so a neighbouring rule cannot stand in for any of them. - for (key, facts, required) in [ - ("\"processors\":", &["online processors"][..], true), - ("\"groups\":", &["processor groups"][..], true), - ("\"packages\":", &["packages"][..], true), - ("\"cores\":", &["physical cores"][..], true), - ("\"numa_domains\":", &["NUMA domains"][..], true), - ( - // **Two facts, because two arms name a level.** The `Level` arm - // renders `outermost cache that partitions the processors it - // covers: L2`, and `SummaryMissing` renders `BUG IN THIS PROBE: the - // topology crate named L2 as the outermost` -- different sentences, - // read by different rules, reported under different fact names. The - // arms come from one `match` and so are mutually exclusive, which is - // what makes accepting either of them precise rather than loose. - // - // Found by a review. With only the first name here, a host that - // produced the `SummaryMissing` shape failed this guard even though - // the field WAS read -- an instrument reporting a defect in the - // renderer that was really a defect in the instrument. - "\"outermost_partitioning_cache_level\":", - &[ - "outermost partitioning cache level", - "summary-missing outermost level", - ][..], - false, - ), - // Nested, and reached by their inner keys: `policies` is an object and - // `caches` an array of objects, so these corrupt the first entry of - // each rather than the container. - ("\"single\":", &["policy domain count"][..], true), - ("\"domains\":", &["cache domain count"][..], false), - ] { - let Some(rendered) = rendered_value(&text, key) else { - assert!( - !required, - "{key} is rendered by every measured report, and this one does \ - not carry it. Either the renderer dropped the field or the key \ - spelling drifted -- both of which make the correspondence stop \ - being checked.\n\n--- the report ---\n{text}" - ); - note(&format!("{key} is not rendered on this host; skipped")); - continue; - }; - if rendered == "null" { - assert!( - !required, - "{key} is rendered null, which no measured report does for this \ - fact.\n\n--- the report ---\n{text}" - ); - note(&format!( - "{key} is rendered null on this host, so no prose accompanies it; skipped" - )); - continue; - } - - let corrupted = corrupt_count(&text, key, if rendered == "0" { "1" } else { "0" }); - assert_ne!( - corrupted, text, - "corrupting {key} changed nothing, so this proves nothing -- a \ - sabotage that fails to apply is indistinguishable from an \ - instrument that fails to fire.\n\n--- the report ---\n{text}" - ); - - // **A violation naming THIS FACT, not merely one of the right kind.** - // This guard has now been wrong twice in the same direction, each fix - // stopping one step short. It first asserted only that the list was - // non-empty. That was strengthened to require a - // `ProseAndNdjsonDisagree`, with a comment correctly explaining that - // corrupting `processors` ALSO trips the cross-check counter rule -- - // and then not acting on it, because the counter rule emits that very - // variant. So the strengthened form still passed while the prose reader - // was blind. - // - // Measured, one entry blinded at a time by breaking its `DOUBLE_RENDERED` - // prose label: with the variant-only assertion, `processors` and `groups` - // both stayed GREEN (masked by their counter rules at - // `check_counters_against_enumeration`), while `packages` went red - // because nothing else reads it. Two of the four required facts were - // unchecked by the guard whose whole purpose is to prove they are - // checked. - // - // Matching on `fact` is what closes it: no neighbouring rule can supply - // another rule's fact name. - let violations = report_oracle::check(&corrupted); - assert!( - violations.iter().any(|violation| matches!( - violation, - report_oracle::Correspondence::ProseAndNdjsonDisagree { fact: named, .. } - if facts.contains(named) - )), - "the oracle read no prose-against-NDJSON correspondence for {key} \ - (expected one of the facts {facts:?}) in a report this host actually \ - produced, so `a_report_rendered_from_this_host_agrees_with_itself` \ - is passing without checking that fact. The renderer has probably \ - drifted from the prose label the oracle looks for.\n\ngot \ - {violations:#?}\n\n--- the report ---\n{text}" - ); - exercised.push(key); - } - - // Named, not counted. Each unconditional fact must appear in what was - // actually exercised, so no number of conditional entries can stand in for - // one of them. - // - // This list is deliberately a SECOND statement of which facts are - // mandatory, rather than being derived from the `required` column above. - // Deriving it would make deleting a table row silently legal -- the row - // would vanish from both the loop and the requirement in one edit. Two - // independent statements mean a row cannot be dropped without this list - // noticing, which is the same reasoning the oracle itself is built on: - // relate two renderings rather than trusting one. - for required in [ - "\"processors\":", - "\"groups\":", - "\"packages\":", - "\"cores\":", - "\"numa_domains\":", - "\"single\":", - ] { - assert!( - exercised.contains(&required), - "{required} was never exercised on this host, so the correspondence \ - it names went unchecked. Exercised: {exercised:?}\n\n\ - --- the report ---\n{text}" - ); - } -} - -/// The value the NDJSON renders for `key`, or `None` when it renders none. -/// -/// Used to decide whether a fact is present at all before corrupting it, and to -/// choose a replacement that differs from what this host actually rendered. -fn rendered_value(report: &str, key: &str) -> Option { - let line = report.lines().find(|line| line.starts_with('{'))?; - let start = line.find(key)? + key.len(); - let end = line[start..] - .find([',', '}']) - .map_or(line.len(), |offset| start + offset); - - Some(line[start..end].to_owned()) -} - -/// Rewrite one NDJSON count to `value`, which the caller picks to differ from -/// what this host rendered. -fn corrupt_count(report: &str, key: &str, value: &str) -> String { - report - .lines() - .map(|line| { - if line.starts_with('{') { - replace_json_number(line, key, value) - } else { - line.to_owned() - } - }) - .collect::>() - .join("\n") -} - -/// Replace the numeric value following `key` in a flat JSON line. -fn replace_json_number(line: &str, key: &str, value: &str) -> String { - let Some(start) = line.find(key) else { - return line.to_owned(); - }; - let after = start + key.len(); - let end = line[after..] - .find([',', '}']) - .map_or(line.len(), |offset| after + offset); - - format!("{}{key}{value}{}", &line[..start], &line[end..]) -} - -// --- M2.10: the fact set, derived from the artifact rather than restated ------ - -/// Why a fact may legitimately go unread on some host. -#[derive(Clone, Copy)] -enum Silence { - /// Never. The prose always states this fact, so it must always be read. - Never, - /// When the key renders this exact value, the prose makes no matching claim. - AtValue(&'static str), - /// When the key renders this value AND the verdict is not `agree`. - /// - /// **Two different rules read these counts, and only one of them is always - /// available.** At zero the prose sentence carrying the count is absent, so - /// the count comparison cannot fire -- but under an AGREEING verdict a - /// nonzero value is a violation in its own right, so the zero-to-nonzero - /// corruption is still caught. Declaring the key silent at zero outright - /// excused both, which made the assertion vacuous on this host and let the - /// declaration's own prose claim a guarantee it had just given away. - /// Measured: with no excuse at all, exactly one mutation went unnoticed -- - /// the deletion -- and on a DISAGREE shape, exactly one went unnoticed -- - /// the corruption. Neither blanket answer is right; the verdict is what - /// separates them. - AtValueUnlessAgreeing(&'static str), - /// When no `host:` line names an architecture. The banner is the only prose - /// rendering of the architecture, and a failed discovery renders `UNKNOWN`, - /// which names none -- so on a host whose bracket reads both failed there is - /// nothing to relate. Conditioned on the REPORT rather than on the key's own - /// value, which is why this cannot be expressed as `AtValue`. - WhenNoBannerArchitecture, -} - -/// Every fact a report may publish, what this test requires of it, and the -/// correspondences that count as reading it. -/// -/// `reads` is the crux. An earlier version treated a key as read when ANY -/// violation appeared after corrupting it, which is the same defect this branch -/// spent rounds fixing elsewhere: corrupting `parse_incomplete` on a -/// heterogeneous report also trips `UncaveatedClaimUnderDoubt`, so the -/// diagnostics reader could be deleted entirely and this test would stay green. -/// Naming the facts that belong to each key is what makes the measurement mean -/// something. Found by a review. -/// -/// `empty_replacement` is the minimal non-empty value to substitute for an empty -/// container. Both empty shapes still have a second rendering -- the renderer -/// emits `efficiency classes: []` unconditionally, and `caches:` followed by -/// `none reported` -- so treating them as silent would let a no-class or -/// no-cache host pass while the reader was gone. Also found by a review. -/// `absent_is_silent_at` is separate from `silence` because the two questions -/// have different answers for the diagnostic counts. At `0` those keys render NO -/// prose entry, so deleting them leaves no claim for the dropped-counterpart -/// rule to answer and the oracle is right to say nothing -- while CORRUPTING -/// them to nonzero is still caught, by the rule that a non-`agree` count -/// contradicts an agreeing verdict. Declaring them `Silence::AtValue("0")` -/// would have excused both and thrown away coverage that exists, so the -/// narrower statement is the true one. -struct Fact { - key: &'static str, - silence: Silence, - reads: &'static [&'static str], - empty_replacement: Option<&'static str>, - /// The value at which the prose renders nothing, so the key's ABSENCE has no - /// prose claim to contradict. Applies only to the deletion mutation. - absent_is_silent_at: Option<&'static str>, - /// The prose label of an independently-read counter for this key, if one - /// exists. Corrupting THAT line moves a side only the counter rule reads, - /// which is what forces that rule to be exercised separately from the - /// prose/NDJSON rule that shares the key. - counter_prose: Option<&'static str>, - why: &'static str, -} - -const fn fact(key: &'static str, reads: &'static [&'static str]) -> Fact { - Fact { - key, - silence: Silence::Never, - reads, - empty_replacement: None, - absent_is_silent_at: None, - counter_prose: None, - why: "", - } -} - -/// The first object in the rendered `caches` array, if the report has one. -/// -/// Taken from the artifact rather than assumed, so a host with no L1 -- or no -/// caches at all -- is described rather than mismatched. -fn first_cache_object(report: &str) -> Option { - let line = report.lines().find(|line| line.starts_with('{'))?; - let at = line.find(r#""caches":[{"#)? + r#""caches":["#.len(); - let rest = &line[at..]; - let end = rest.find('}')? + 1; - - Some(rest[..end].to_owned()) -} - -/// The report with `label`'s prose value replaced by one that disagrees. -/// -/// Moves only the prose side, so the NDJSON still agrees with its own prose and -/// the only rule that can notice is the one reading this counter. -fn corrupt_counter_prose(report: &str, label: &str) -> Option { - let line = report.lines().find(|line| line.starts_with(label))?; - let rendered = line[label.len()..].trim(); - let flipped = if rendered == "0" { "1" } else { "0" }; - - Some(report.replace(line, &format!("{label}{flipped}"))) -} - -const FACTS: &[Fact] = &[ - Fact { - silence: Silence::WhenNoBannerArchitecture, - why: "a report whose bracket reads both failed renders `UNKNOWN`, which \ - names no architecture, so the body's `arch` has no prose to relate", - ..fact("arch", &["architecture"]) - }, - // **Two rules read these keys, so the mutation set has to separate them.** - // Corrupting the NDJSON value moves the enumerated side, which BOTH the - // prose/NDJSON rule and the cross-check counter rule can see -- so the - // ordinary rule's violation satisfied the accounting and masked whether the - // counter rule was read at all. Measured: deleting both counter rules left - // every test in this file green. - // - // `counter_prose` names the line only the counter rule reads. Corrupting it - // leaves the enumeration agreeing with its own prose, so the only violation - // available names the counter fact -- and if that rule is gone, nothing is - // reported and the accounting fails, which is the point. - Fact { - counter_prose: Some(" GetActiveProcessorCount : "), - ..fact( - "processors", - &[ - "online processors", - "active processor count against the enumeration", - ], - ) - }, - Fact { - counter_prose: Some(" GetActiveProcessorGroupCount: "), - ..fact( - "groups", - &[ - "processor groups", - "active group count against the enumeration", - ], - ) - }, - fact("packages", &["packages"]), - fact("numa_domains", &["NUMA domains"]), - fact( - "numa_domains_without_processors", - &["NUMA domains without processors"], - ), - fact("cores", &["physical cores"]), - Fact { - empty_replacement: Some("[0]"), - ..fact("efficiency_classes", &["efficiency classes"]) - }, - Fact { - empty_replacement: Some(r#"[{"level":9,"domains":9}]"#), - ..fact("caches", &["cache levels", "cache domain count"]) - }, - Fact { - silence: Silence::AtValue("null"), - why: "three of the five partitioning variants publish null, and their \ - prose names no level", - ..fact( - "outermost_partitioning_cache_level", - &[ - "outermost partitioning cache level", - "summary-missing outermost level", - ], - ) - }, - fact( - "outermost_partitioning_cache", - &["outermost partitioning answer"], - ), - fact("policies", &["policy domain count", "policy names"]), - Fact { - silence: Silence::AtValue("\"not_measured\""), - why: "`report_unmeasured` renders no verdict line, so the short object's \ - cross_check has no prose to relate", - ..fact("cross_check", &["cross-check verdict"]) - }, - // **Read at every value, but ABSENT only matters when the prose speaks.** - // Both reach the prose as entries the renderer emits only when the count is - // above zero, so at zero there is no line for a dropped counterpart to - // contradict -- while corrupting either to nonzero is still caught, because - // a non-zero count beside an agreeing verdict is a violation in its own - // right. `Silence::AtValue("0")` would have excused both mutations and - // discarded that second guarantee. - Fact { - absent_is_silent_at: Some("0"), - why: "at zero the renderer emits no `(not compared)` entry, so the \ - absence of the field contradicts nothing", - ..fact( - "not_compared", - &["not compared count", "incomplete-verdict listing count"], - ) - }, - Fact { - absent_is_silent_at: Some("0"), - why: "at zero the renderer emits no `(parse incomplete)` entry, so the \ - absence of the field contradicts nothing", - ..fact( - "parse_incomplete", - &["parse incomplete count", "incomplete-verdict listing count"], - ) - }, - // **Silent when ABSENT at zero, not silent at zero.** This was declared - // `Silence::AtValue("0")`, which excuses every mutation on a host rendering - // no anomalies -- including the zero-to-nonzero corruption that the comment - // below calls load-bearing. Measured: with the excuse removed, exactly ONE - // of the two mutations goes unnoticed, and it is the DELETION. The - // corruption is caught, by the rule that a nonzero count cannot sit beside - // an agreeing verdict -- so blanket silence threw away a guarantee this host - // does provide, and let the comment claim one the classification denied. - // - // The narrower declaration is the same one `not_compared` and - // `parse_incomplete` needed, and this key should have been swept with them. - Fact { - silence: Silence::AtValueUnlessAgreeing("0"), - absent_is_silent_at: Some("0"), - why: "the count reaches the prose only inside the `windows-topology-sys \ - recorded N enumeration anomal...` sentence, which the renderer \ - emits only when there are anomalies. At zero that sentence is \ - absent, so on a DISAGREE or INCOMPLETE host there is no second \ - rendering -- and under `agree` a nonzero value is a violation in \ - its own right, so the fact is never silently wrong. **This \ - declaration was made once, in c75d74e, and then lost when this \ - table was restructured two commits later; nothing caught it \ - because THIS host reports `agree`, where the key is read anyway. \ - Found by a review, twice.**", - ..fact("enumeration_anomalies", &["enumeration anomaly count"]) - }, - Fact { - silence: Silence::AtValue("0"), - why: "the renderer emits the `(N reported only by CPU Sets` line only \ - when the count is above zero, so zero is silence rather than \ - agreement", - ..fact( - "numa_domains_only_in_cpu_sets", - &["NUMA domains reported only by CPU Sets"], - ) - }, -]; - -/// Keys with no second rendering at all, and why. -const NEVER_COMPARED: &[(&str, &str)] = &[( - "reason", - "a routing tag for a mining pass, naming which probe emitted the row. The \ - prose never states it, so there is no correspondence to check", -)]; - -#[test] -fn every_fact_the_renderer_publishes_is_accounted_for() { - // **M2.10.** Six unread double-renderings were found by six reviewers and - // none by this suite, because a rule is added per fact and nothing derived - // the SET of facts from the renderer. - // - // The derivation takes both halves from places that cannot drift: the SET is - // enumerated from the NDJSON line of a report the renderer really produced, - // and read-or-unread is MEASURED by corrupting each value and asking the - // oracle. What is declared is the classification -- which facts belong to a - // key, and when silence is legitimate -- and every part of that declaration - // is itself measured. - account_for_every_fact(&real_report().0, "the measured report"); - account_for_every_fact(&unmeasured_report(), "the unmeasured report"); -} - -/// The short report, rendered through the real path with a real banner. -fn unmeasured_report() -> String { - let before = Fingerprint::discover(); - let after = Fingerprint::discover(); - - report_unmeasured( - &attribution(&before, &after), - &std::io::Error::other("a simulated discovery failure"), - ) -} - -/// Require every fact `text` publishes to be classified, and every claim in that -/// classification to hold. -fn account_for_every_fact(text: &str, shape: &str) { - let keys = ndjson_keys(text); + report_oracle::assert_row_is_well_formed(&text); assert!( - !keys.is_empty(), - "{shape} published no machine-readable facts at all, so the enumeration \ - is broken rather than the renderer.\n\n--- the report ---\n{text}" + report_oracle::row(&text).is_some(), + "every report carries exactly one row, including an unmeasured one -- \ + that is what lets a survey tell a host where discovery FAILED from a \ + job that never ran the probe (measured: {measured})\n\n{text}" ); - - for key in &keys { - let classified = FACTS.iter().find(|entry| entry.key == key); - let never = NEVER_COMPARED.iter().find(|(name, _)| name == key); - - assert!( - classified.is_some() || never.is_some(), - "{shape} publishes `{key}`, and nothing here says whether the oracle \ - reads it. THIS IS THE POINT OF THIS TEST: a fact was added to the \ - report and no rule was added to relate it to its prose. Either add \ - the rule and list `{key}` in FACTS, or say why it has no second \ - rendering in NEVER_COMPARED.\n\n--- the report ---\n{text}" - ); - - let rendered = raw_value(text, key).unwrap_or_default(); - - if let Some(fact) = classified { - // Every mutation site in turn, not just the first. Corrupting only - // the first number inside `caches` changes a LEVEL, so the - // domain-count reader could be deleted and the level-membership rule - // would still fire and hide it. Found by a review. - let mut mutations = corruptions(text, fact.key, fact.empty_replacement); - - // The prose-side mutation, where a second rule reads this key from a - // line of its own. Appended BEFORE the deletion-excuse slice below - // would trim the tail, so it is judged like any other corruption. - // - // **Only where the counter rule can fire.** It reads the counters - // solely under an agreeing verdict -- on a report that already says - // its counters disagree, a counter contradiction is the subject - // rather than a violation. Generating the mutation anyway made the - // corpus' `a counter that disagrees with the enumeration` shape fail - // for the rule's correct behaviour, which is the over-constraint - // this module treats as the same defect as under-specifying. - if let Some(label) = fact.counter_prose - && raw_value(text, "cross_check").as_deref() == Some("\"agree\"") - && let Some(corrupted) = corrupt_counter_prose(text, label) - { - let deletion = mutations.pop(); - mutations.push(corrupted); - mutations.extend(deletion); - } - - // The deletion is the LAST mutation `corruptions` appends, and it is - // judged on its own terms: a key whose value renders no prose leaves - // nothing for a dropped counterpart to contradict, so its absence is - // legitimately silent even though corrupting it is not. - let absence_excused = fact - .absent_is_silent_at - .is_some_and(|value| rendered == value); - let judged = if absence_excused { - &mutations[..mutations.len().saturating_sub(1)] - } else { - &mutations[..] - }; - - let unread: Vec<&String> = judged - .iter() - .filter(|corrupted| !reads_the_fact(corrupted, fact)) - .collect(); - - let excused = match fact.silence { - Silence::Never => false, - Silence::AtValue(value) => rendered == value, - Silence::AtValueUnlessAgreeing(value) => { - rendered == value - && raw_value(text, "cross_check").as_deref() != Some("\"agree\"") - } - Silence::WhenNoBannerArchitecture => banner_names_no_architecture(text), - }; - - // **A fact that cannot be mutated is not a fact that was checked.** - // An earlier version let an empty mutation list stand for success, - // so a classified key whose value has no mutation site -- an array - // of strings, say -- would pass while the oracle had no reader for - // it at all. That defeats the anti-drift guarantee this test is - // for. Found by a review. - assert!( - !mutations.is_empty() || excused, - "`{key}` is classified as read, and nothing here knows how to \ - change the {rendered} it renders, so this host cannot show that \ - anything reads it. Teach `corruptions` this value shape, or \ - declare when the key is silent.\n\n--- the report ---\n{text}" - ); - - assert!( - unread.is_empty() || excused, - "nothing in {shape} reads `{key}` as any of {:?} -- {} of {} \ - mutations of it went unnoticed. Either a rule that reads it has \ - drifted from the renderer, or it never read this key. One of \ - those mutations DELETES the key: if that is the one going \ - unnoticed, the rule compares two present values and treats a \ - dropped counterpart as silence.{}\n\n\ - --- the report ---\n{text}", - fact.reads, - unread.len(), - judged.len(), - if fact.why.is_empty() { - String::new() - } else { - format!(" The declared condition for silence is: {}.", fact.why) - } - ); - } - - if let Some((_, why)) = never { - assert!( - corruptions(text, key, None) - .iter() - .all(|corrupted| report_oracle::check(corrupted).is_empty()), - "`{key}` is listed as having no second rendering -- {why} -- but \ - the oracle now reports something when it is corrupted in {shape}. \ - The exemption is stale; move it to FACTS.\n\n\ - --- the report ---\n{text}" - ); - } - } -} - -/// Whether any violation in `corrupted` is one of `fact`'s own correspondences. -/// -/// Attribution matters: a violation from a neighbouring rule proves nothing -/// about whether THIS key is read. `AlarmWithAgreeingVerdict` is matched by the -/// key appearing in the alarm text, which is how the diagnostics rule names the -/// field it is complaining about. -fn reads_the_fact(corrupted: &str, fact: &Fact) -> bool { - report_oracle::check(corrupted).iter().any(|violation| { - match violation { - report_oracle::Correspondence::ProseAndNdjsonDisagree { fact: named, .. } => { - fact.reads.contains(named) - } - report_oracle::Correspondence::AlarmWithAgreeingVerdict { alarm, .. } => { - alarm.contains(fact.key) - } - // The banner rule reports a processor count without naming a fact. - report_oracle::Correspondence::BannerDisagreesWithBody { .. } => { - fact.key == "processors" - } - report_oracle::Correspondence::UncaveatedClaimUnderDoubt { .. } => false, - // Names its fact the same way `ProseAndNdjsonDisagree` does, and - // for the same reason: it IS that comparison, reported when the - // other side turned out not to be there. - report_oracle::Correspondence::RenderedOnlyInProse { fact: named, .. } => { - fact.reads.contains(named) - } - // Names its correspondence directly, so it attributes the same way. - report_oracle::Correspondence::EvidenceMissingWithAgreeingVerdict { fact: named } => { - fact.reads.contains(named) - } - } - }) -} - -/// Whether no `host:` line in the report names an architecture. -/// -/// **Asks the oracle rather than re-deciding.** This used to test -/// `line.contains("p/")`, which is what the oracle once did too -- and when the -/// oracle moved to reading the fingerprint's tokens by position, this copy -/// stayed behind. Measured: a failed-discovery banner of -/// `host: UNKNOWN -- topology discovery failed: 16p/foo something opaque` -/// satisfied the substring test and not the oracle, so this helper reported that -/// the banner named an architecture, the `arch` silence exemption did not apply, -/// and `account_for_every_fact` failed on a report the oracle had accepted. -/// -/// The instrument that measures whether a fact is read must not hold its own -/// opinion about what the oracle does. Found by a review. -fn banner_names_no_architecture(report: &str) -> bool { - !report - .lines() - .filter(|line| line.starts_with("host:")) - .any(|line| report_oracle::architecture_in_banner(line).is_some()) -} - -/// One corrupted copy of the report per mutation site in this key's value, -/// plus one with the key DELETED. -/// -/// A container has one site per number it renders, so each nested property is -/// exercised on its own rather than standing behind the first. -/// -/// **Deletion is a mutation, and leaving it out hid a whole class.** Every -/// mutation here used to rewrite a VALUE, so every rule was asked only "these -/// two renderings differ" and never "one of them is gone". The oracle's -/// comparisons were written as `if let (Some(prose), Some(json))`, which reads a -/// dropped field as silence -- and this instrument, the thing whose whole -/// purpose is to prove each fact is read, could not see it. Five sites were -/// found by review after three had already been fixed by review, which is the -/// signature of a class being patched instance by instance instead of swept. -/// -/// Adding it here rather than beside each rule is deliberate: a rule added later -/// inherits the question automatically, which is the only version of this that -/// cannot rot. -fn corruptions(report: &str, key: &str, empty_replacement: Option<&str>) -> Vec { - let Some(original) = raw_value(report, key) else { - return Vec::new(); - }; - - // A `null` value has no corruption, but it can still be DELETED -- and the - // absence of the key is a different claim from the presence of `null`. - let deleted = report - .replace(&format!("\"{key}\":{original},"), "") - .replace(&format!(",\"{key}\":{original}"), ""); - - if original == "null" { - return vec![deleted]; - } - - let rewrite = |replacement: &str| { - report.replace( - &format!("\"{key}\":{original}"), - &format!("\"{key}\":{replacement}"), - ) - }; - - if original.starts_with('"') { - return vec![rewrite("\"x-corrupted\""), deleted]; - } - - if original.starts_with('[') || original.starts_with('{') { - let sites: Vec = original - .char_indices() - .filter(|(index, character)| { - character.is_ascii_digit() - && !original[..*index].ends_with(|previous: char| previous.is_ascii_digit()) - }) - .map(|(index, _)| index) - .collect(); - - if sites.is_empty() { - // An empty container still has prose beside it, so substitute the - // smallest value that disagrees with an empty one. - let mut mutations = empty_replacement - .map(|replacement| vec![rewrite(replacement)]) - .unwrap_or_default(); - mutations.push(deleted); - return mutations; - } - - let mut mutations: Vec = sites - .into_iter() - .map(|at| { - let mut copy = original.clone(); - let flipped = if &original[at..=at] == "0" { "1" } else { "0" }; - copy.replace_range(at..=at, flipped); - rewrite(©) - }) - .collect(); - - // **A mutation that changes a NAME, not a number.** Every mutation above - // changes a digit, so a rule that relates the SET of entries -- the - // policy names, the cache levels -- is never exercised on its own: the - // per-entry value comparisons fire for the same mutation and report the - // key as read. Removing `compare_membership` for policy names would - // leave this instrument green while that correspondence was dead. - // - // Renaming the first key is what reaches it. With the membership rule - // present the rename is reported as a disagreement about which entries - // exist; with it removed the prose name matches nothing, the per-entry - // lookup returns `None`, and NOTHING is reported -- which this test then - // fails on, as it should. Found by a review. - // **Every distinct key, not just the first one.** Renaming only the - // first reaches `level` in the cache array and never `domains`, so a - // reader of the inner member could go blind while this stayed green: - // measured, `{"level":1,"x-domains":8}` was accepted by the oracle - // because the domain lookup required the two members to be adjacent and - // in order, and nothing here renamed the second one to find out. - for key in object_key_names(&original) { - let from = format!("\"{key}\":"); - let to = format!("\"x-{key}\":"); - mutations.push(rewrite(&original.replacen(&from, &to, 1))); - } - - mutations.push(deleted); - - return mutations; - } - - vec![rewrite(if original == "0" { "1" } else { "0" }), deleted] -} - -/// Every distinct key name a container value writes, in first-seen order. -/// -/// Keys of nested objects included: they are what the renamed-member mutation -/// needs, and they are exactly the ones the top-level enumeration cannot see. -fn object_key_names(value: &str) -> Vec { - let mut names: Vec = Vec::new(); - - for (at, _) in value.match_indices("\":") { - let Some(opening) = value[..at].rfind('"') else { - continue; - }; - let name = &value[opening + 1..at]; - - if !name.is_empty() && !names.iter().any(|seen| seen == name) { - names.push(name.to_owned()); - } - } - - names -} - -/// Every key the NDJSON line renders, in the order it renders them. -fn ndjson_keys(report: &str) -> Vec { - let Some(line) = report.lines().find(|line| line.starts_with('{')) else { - return Vec::new(); - }; - let mut keys = Vec::new(); - let mut rest = line; - let mut depth = 0_i32; - - while let Some(quote) = rest.find('"') { - for character in rest[..quote].chars() { - match character { - '[' | '{' => depth += 1, - ']' | '}' => depth -= 1, - _ => {} - } - } - let after = &rest[quote + 1..]; - let Some(close) = after.find('"') else { break }; - let (name, tail) = after.split_at(close); - let tail = &tail[1..]; - // Depth 1 only. A nested member is not enumerated as a fact of its own - // -- it is part of its container's value, and so is covered by the - // container's mutations instead. `a_nested_fact_nobody_classified_fails_the_accounting` - // is the proof that this is a division of labour rather than a gap. - if tail.starts_with(':') && depth == 1 { - keys.push(name.to_owned()); - } - rest = tail; - } - - keys -} - -/// The raw value the NDJSON renders for `key`, delimiters included. -fn raw_value(report: &str, key: &str) -> Option { - let line = report.lines().find(|line| line.starts_with('{'))?; - let needle = format!("\"{key}\":"); - let start = line.find(&needle)? + needle.len(); - let rest = &line[start..]; - - let end = if rest.starts_with('[') || rest.starts_with('{') { - let mut depth = 0_i32; - let mut close = None; - for (index, character) in rest.char_indices() { - match character { - '[' | '{' => depth += 1, - ']' | '}' => { - depth -= 1; - if depth == 0 { - close = Some(index + 1); - break; - } - } - _ => {} - } - } - close? - } else if let Some(after) = rest.strip_prefix('"') { - after.find('"')? + 2 - } else { - rest.find([',', '}']).unwrap_or(rest.len()) - }; - - Some(rest[..end].to_owned()) -} - -/// Every diagnostic this file writes goes through here. -/// -/// The repository's output rule: once a second call site appears, where the text -/// goes stops being each call site's business. These are skip notes -- a reader -/// of a CI log needs them to tell "this host's shape meant the check could not -/// run" from "the check ran and found nothing", which are very different -/// readings of the same green result. -fn note(message: &str) { - eprintln!("{message}"); } // --- M2.12: the shape corpus ------------------------------------------------ @@ -1221,6 +314,36 @@ fn shapes() -> Vec { }; push("two sources that described different processors", disagreed); + // **The shape whose absence hid a wrong state-to-code mapping.** The corpus + // carried `Agreed` and `Disagreed` and never `NotCollected`, and + // `blocking_states` lumped the latter two together as + // `EnumerationsDisagreed` -- while `cross_check` publishes + // `coherence_not_collected` for it. The per-state rule therefore demanded a + // code the row does not emit for this observation, and nothing noticed, + // because no shape reached it. Found by a review. + let mut uncollected = base(); + uncollected.coherence = windows_topology_sys::Coherence::NotCollected; + push("coherence that was never collected", uncollected); + + // **The three states the corpus declared and never rendered.** `codes_for` + // named a code for each, and no shape produced any of them, so those arms + // were never held against a real row -- the same gap that let + // `CoherenceNotCollected` carry the wrong code. Found by a review. + let mut no_packages = base(); + no_packages.packages = 0; + push("processors reported, no packages", no_packages); + + let mut no_cores = base(); + no_cores.cores = Vec::new(); + push("processors reported, no cores", no_cores); + + let mut unnumbered = base(); + unnumbered.caches = vec![CacheLevel { + level: 0, + processors_per_domain: vec![4], + }]; + push("a cache level Windows does not number", unnumbered); + let mut anomalies = base(); anomalies.enumeration_anomalies = vec![ windows_topology_sys::EnumerationAnomaly { @@ -1265,6 +388,22 @@ fn shapes() -> Vec { anomalies_while_disagreeing, ); + // **Several conditions in ONE list, which nothing else here reaches.** Every + // other shape varies one dimension, so each lands at most one entry per + // list -- and a one-element list has no order to get wrong. That made the + // ordering half of `the_row_lists_exactly_the_conditions_the_cross_check_found` + // vacuous: reversing the row's `parse_incomplete` reddened nothing at all. + // + // Found by the guard in `the_corpus_reaches_states_that_block_agreement`, + // which is there precisely because a corpus cannot report the shape it does + // not reach. + let mut several = base(); + several.caches = Vec::new(); + several.cores_only_in_cpu_sets = 1; + several.numa_domains_unreported = 2; + several.processor_attribute_conflicts = 1; + push("several conditions at once, in one list", several); + shapes } @@ -1294,224 +433,534 @@ fn every_representative_shape_agrees_with_itself() { } } +// --- M3.5: the accounting, re-aimed from the prose at the row ---------------- + +/// The row's lists that render one prose entry each. +/// +/// `enumeration_anomalies` is deliberately absent: it is per-anomaly detail the +/// prose summarises into ONE entry, so it counts on a different axis and is +/// checked against the observation instead. +const DIAGNOSTIC_LISTS: &[&str] = &["disagreements", "not_compared", "parse_incomplete"]; + +/// Every condition code the row publishes under `keys`. +/// +/// **Which keys is a parameter, because the four lists do not all relate to +/// the prose the same way.** The three DIAGNOSTIC lists render one prose entry +/// each. `enumeration_anomalies` does not: the prose folds every anomaly into a +/// single `windows-topology-sys recorded N enumeration anomal...` sentence, so +/// a host with three anomalies publishes three codes beside one prose line. +/// Counting them together made that shape look like a dropped entry. +/// +/// Reads the RENDERED row rather than the `CrossCheck` behind it, because what a +/// survey receives is the subject: an assertion against the struct would hold +/// even if the writer published nothing at all. +fn published_codes(text: &str, keys: &[&str]) -> Vec { + let Some(row) = report_oracle::row(text) else { + panic!("no single well-formed row in:\n{text}"); + }; + + keys.iter() + .flat_map(|key| report_oracle::list_codes(row, key)) + .collect() +} +/// Every diagnostic entry in `text`'s row, as rendered JSON, for one key. +/// +/// **The whole object, not just its `code`.** `published_codes` above discards +/// every payload field, so the renderer-to-row path stayed green if +/// `topology_report` dropped or reshaped `parsed`, `counter`, `level` or an +/// anomaly's metadata. The per-variant tests cover `published()` in isolation; +/// nothing covered it through the renderer until this. Found by a review. +fn published_entries(text: &str, key: &str) -> Vec { + let Some(row) = report_oracle::row(text) else { + panic!("no single well-formed row in:\n{text}"); + }; + + let parsed: serde_json::Value = serde_json::from_str(row).expect("the row parses"); + parsed[key] + .as_array() + .unwrap_or_else(|| panic!("`{key}` should be a list in:\n{row}")) + .iter() + .map(ToString::to_string) + .collect() +} + +/// One diagnostic value as the row writer renders it. +/// +/// Derived from `published()` rather than written out, so the expectation +/// cannot drift from the publisher it is checking; what this pins is that the +/// RENDERER carries that value through unchanged. +fn as_rendered(value: windows_platform_probes::row::Value) -> String { + let row = windows_platform_probes::row::Row::new("x").with("entry", value); + let text = row.render(); + let parsed: serde_json::Value = serde_json::from_str(&text).expect("the row parses"); + parsed["entry"].to_string() +} + +#[test] +fn the_row_carries_each_diagnostic_entry_whole_and_not_only_its_code() { + use windows_platform_probes::topology::diagnostic::{ + Disagreement, NotCompared, ParseIncomplete, published_anomaly, + }; + + for shape in shapes() { + let text = report(&banner_for(&shape.observation), &shape.observation); + let check = shape.observation.cross_check(); + + for (key, expected) in [ + ( + "disagreements", + check + .disagreements + .iter() + .map(Disagreement::published) + .collect::>(), + ), + ( + "not_compared", + check + .not_compared + .iter() + .map(NotCompared::published) + .collect::>(), + ), + ( + "parse_incomplete", + check + .parse_incomplete + .iter() + .map(ParseIncomplete::published) + .collect::>(), + ), + ( + "enumeration_anomalies", + shape + .observation + .enumeration_anomalies + .iter() + .map(published_anomaly) + .collect::>(), + ), + ] { + assert_eq!( + published_entries(&text, key), + expected.into_iter().map(as_rendered).collect::>(), + "{}: the row's `{key}` must carry each entry whole", + shape.what + ); + } + } +} + +/// Whether `text`'s row publishes a condition for an observation in a blocking +/// state. +/// +/// **Extracted so the sabotage can invoke the rule rather than restate it.** +/// `a_state_the_row_does_not_publish_fails_the_accounting` used to strip the +/// row's conditions and then assert only that the stripping had worked -- so it +/// demonstrated the sabotage, never that the accounting REJECTS it. The +/// accounting could have been deleted and that test would have stayed green. +/// Found by a review. +/// +/// **Per state, not "at least one".** This asked only whether the row published +/// SOME code, which an observation in five blocking states satisfies by +/// publishing one -- so dropping four states' conditions left the rule that +/// names itself `every_state_...` perfectly happy. Measured: truncating the +/// row's `parse_incomplete` to its first entry is caught by three tests that +/// compare the row against `cross_check`, and by this one not at all. That +/// matters because this is the only instrument running the other enumeration -- +/// from `invariant`'s states INTO the row -- so its weakness was invisible to +/// everything else. Found by a review of the pull request. +fn publication_holds(observation: &Observation, text: &str) -> bool { + let published = published_codes(text, DIAGNOSTIC_LISTS); + invariant::blocking_states(observation) + .into_iter() + .all(|state| { + codes_for(state) + .iter() + .any(|code| published.iter().any(|found| found == code)) + }) +} + +/// The row code(s) that answer `state`. +/// +/// **A schema, written down, and exhaustive so it cannot fall behind.** This is +/// the correspondence the milestone exists to enforce -- a state the invariants +/// know about must reach a survey -- and it is not derivable from either side: +/// `blocking_states` computes states from an observation and the renderer emits +/// codes, with nothing in between that already knows the pairing. Writing it +/// here is what makes the check possible; a new state that names no code fails +/// to compile. +/// +/// `BracketNotHeld` answers to either code because `cross_check` files a +/// different one depending on how the bracket failed, and both are honest +/// reports of the same blocking state. +fn codes_for(state: invariant::BlockingState) -> &'static [&'static str] { + use invariant::BlockingState as State; + match state { + State::PartitioningSummaryMissing => &["partitioning_summary_missing"], + // **A CODE inside `parse_incomplete`, not the row key of the same + // name.** `cross_check` pushes `ParseIncomplete::EnumerationAnomalies` + // when the observation records any, and that variant's code is this + // string -- so `publication_holds` finds it while scanning + // `DIAGNOSTIC_LISTS`, which deliberately excludes the + // `enumeration_anomalies` LIST. A review read this as the key and + // concluded the assertion must fail for the `anomalies` shape; it does + // not, and `every_state_that_blocks_agreement_reaches_the_row` covers + // exactly that shape. Noted here because the collision is real even + // though the conclusion was not. + State::EnumerationAnomalies => &["enumeration_anomalies"], + State::NotMeasured => &["not_measured"], + State::NoCacheLevels => &["no_cache_levels"], + State::NoPackages => &["no_packages"], + State::NoCores => &["no_cores"], + State::ContradictoryCore => &["contradictory_cores"], + State::UnnumberedCacheLevel => &["unnumbered_cache_levels"], + State::EnumerationsDisagreed => &["enumerations_disagreed"], + State::CoherenceNotCollected => &["coherence_not_collected"], + State::BracketNotHeld => &["machine_changed", "bracket_not_established"], + } +} + #[test] -fn every_fact_is_accounted_for_on_every_representative_shape() { - // The accounting, over the whole corpus rather than over this host alone. - // Its classification -- which facts belong to a key, and when silence is - // legitimate -- was written against one shape, so this is the first thing - // that checks those declarations against the others. +fn every_state_that_blocks_agreement_reaches_the_row() { + // **This is the rule M3.1 established, given an instrument at last.** A + // renderer may not tell a reader something the row cannot tell a survey -- + // and nothing enforced that, because every instrument in this crate + // enumerated the ROW's keys and so could only ask "does anything read this + // key?", never "does the prose state a fact the row omits?". + // + // Measured, which is how the gap was found rather than reasoned: + // `CrossCheck::disagreements` reached the prose as one line per + // disagreement and reached the row as nothing at all, so a survey saw + // `"cross_check":"disagree"` without learning WHICH counter disagreed. It + // survived 41 review rounds and a zero-survivor mutation sweep because all + // of them start from what the row publishes. + // + // The enumeration here runs the other way: for every state + // `topology::invariant` knows forbids an agreeing verdict, render a report + // in that state and require the row to carry a code for it. for shape in shapes() { let text = windows_platform_probes::topology_report::report( &banner_for(&shape.observation), &shape.observation, ); - account_for_every_fact(&text, shape.what); + let blocking = invariant::blocking_states(&shape.observation); + if blocking.is_empty() { + continue; + } + + // **Names the state whose code is missing, and what was published + // instead.** The message used to say "publishes no condition at all", + // which was wrong in the case that actually fired: the row DID publish a + // condition, just not the one the failing state maps to. A reader + // debugging a wrong mapping was told the opposite of the symptom. + let published = published_codes(&text, DIAGNOSTIC_LISTS); + let unpublished: Vec = blocking + .iter() + .filter(|state| { + !codes_for(**state) + .iter() + .any(|code| published.iter().any(|found| found == code)) + }) + .map(|state| format!("{state:?} (wants one of {:?})", codes_for(*state))) + .collect(); + + assert!( + publication_holds(&shape.observation, &text), + "{}: the observation is in {} state(s) that forbid agreement -- \ + {blocking:?} -- and the row publishes no code for {}. Published: \ + {published:?}. A survey reading it would see a verdict it cannot \ + account for.\n\n--- the report ---\n{text}", + shape.what, + blocking.len(), + unpublished.join(", "), + ); } } #[test] -fn a_failed_discovery_whose_error_mentions_a_count_is_still_accounted_for() { - // **The instrument must not hold its own opinion about what the oracle - // does.** `banner_names_no_architecture` decides whether the `arch` silence - // exemption applies, and it used to answer with `line.contains("p/")` -- - // which agreed with the oracle until the oracle began reading the - // fingerprint's tokens by position. - // - // `banner_line_for` interpolates a failed read's `io::Error` verbatim, so an - // error mentioning a processor count puts `p/` in the banner of a report - // that names no architecture at all. Measured before the fix: the helper - // said the banner named one, the exemption did not apply, and the accounting - // demanded that `arch` be read on a report the oracle had accepted -- a - // valid unmeasured report failing the suite for a reason that was not about - // the report. +fn the_corpus_reaches_every_blocking_state() { + // **The per-state rule proves a mapping was WRITTEN; this proves it was + // EXERCISED.** `codes_for` is exhaustive, so every state names a code -- but + // an arm whose state no shape produces is never compared against a rendered + // row, and a wrong code there sits undetected. That is not hypothetical: it + // is exactly how `CoherenceNotCollected` came to be mapped to + // `enumerations_disagreed`, caught only when a shape finally reached it. // - // No corpus shape produces this banner and neither does this host, so - // nothing else here would notice the two definitions drifting apart again. - let error = || std::io::Error::other("16p/foo something opaque"); - let banner = windows_placement_probe::fingerprint::banner_line_for(&Err(error())); + // Found by a review, which observed that this corpus omitted several + // declared states outright. + let mut unreached: Vec = Vec::new(); + for state in invariant::BlockingState::ALL { + let reached = shapes() + .iter() + .any(|shape| invariant::blocking_states(&shape.observation).contains(state)); + if !reached { + unreached.push(format!("{state:?}")); + } + } assert!( - banner.contains("p/"), - "this test is pointless unless the error text reaches the banner: {banner}" + unreached.is_empty(), + "no corpus shape reaches {}, so `codes_for`'s arm(s) for them are never \ + held against a rendered row", + unreached.join(", ") ); +} - let text = windows_platform_probes::topology_report::report_unmeasured(&banner, &error()); +#[test] +fn the_corpus_reaches_an_observation_in_several_blocking_states_at_once() { + // **The rule above is per-state, and a corpus of single-state shapes cannot + // tell that apart from "at least one".** Its previous form was satisfied by + // any one published code, and no shape with two states would have exposed + // that -- which is why the reachability is asserted rather than assumed: a + // corpus cannot report the shape it never reaches. + let deepest = shapes() + .into_iter() + .map(|shape| { + ( + shape.what, + invariant::blocking_states(&shape.observation).len(), + ) + }) + .max_by_key(|(_, states)| *states) + .expect("the corpus is not empty"); assert!( - banner_names_no_architecture(&text), - "a failed read names no architecture, whatever its error text spells:\n{text}" - ); - - account_for_every_fact( - &text, - "a failed discovery whose error text contains a count", + deepest.1 >= 2, + "the deepest shape in the corpus is `{}` with {} blocking state(s), so \ + the per-state rule is never asked to distinguish one state from \ + several", + deepest.0, + deepest.1, ); } #[test] -fn a_nested_fact_nobody_classified_fails_the_accounting() { - // **The enumeration is shallow; the GUARANTEE is not.** `ndjson_keys` records - // only depth-1 keys, so nothing here lists `policies`' entries or a cache - // object's `level` and `domains` -- which reads like a hole in a test called - // `every_fact_the_renderer_publishes_is_accounted_for`, and a review read it - // that way. - // - // It is not one, because read-or-unread is MEASURED rather than enumerated. - // A nested field is part of its container's value, so `corruptions` mutates - // it -- by digit, and now by renaming every distinct key -- and those - // mutations must be noticed by a rule that names one of the container's - // facts. A member nobody wrote a rule for is a mutation nobody notices. +fn the_row_lists_exactly_the_conditions_the_cross_check_found() { + // **The rule that replaced a prose count, and the last prose parsing in the + // matrix went with it.** This compared the row's condition count against a + // count of prose lines -- which meant filtering rendered text by line prefix + // and turning it into a number, the one remaining place the test matrix + // obtained structured data by reading prose. // - // This test is that argument, executed. Without it the property holds by - // reasoning about two functions that do not mention each other, which is - // exactly the kind of claim this branch keeps finding to be false. - // **Injection sites come from what this host actually rendered.** The first - // version assumed a measured report with a level-1 cache and hard-coded - // `"level":1,"domains":` -- the same host dependency a review had just - // removed from another test in this file. A host whose discovery fails - // renders neither container, and a measured one need not have an L1; there - // the injection would silently not apply and the assertion below would fire - // about a report that is perfectly legitimate. - let (text, measured) = real_report(); - - if !measured { - // `report_unmeasured` publishes no container at all, so there is no - // nested fact to leave unclassified and nothing for this test to say. - return; - } - - // **The injected member must be one NO rule reads, or this proves nothing.** - // An earlier version also added `"by-latency":3` to the `policies` object -- - // but policy entries are NAMED, and `compare_membership` reads that name set, - // so the extra entry is a fact the oracle covers rather than an unclassified - // one. Measured: the injected report already carried a `policy names` - // violation before any mutation, and the accounting then panicked on the - // `reason` exemption ("the oracle now reports something when it is - // corrupted") rather than on an unread fact. The assertion held and the - // reason was wrong, which is the failure mode this whole file exists to - // catch. Found by a review. + // What replaces it is strictly stronger and never reads a sentence: the + // row's codes must EQUAL the cross-check's codes, in order. A count could + // only catch a dropped entry; this catches a dropped one, a reordered one, + // and a substituted one. // - // A cache object's members are the case that works: the array is the only - // container of objects whose members are read positionally rather than by - // name, so a member nobody named is genuinely unclassified. Measured on the - // same run: baseline `[]`, and the panic is - // `nothing ... reads 'caches' ... 2 of 11 mutations went unnoticed`. - let Some(cache) = first_cache_object(&text) else { - // A measured report need not carry caches -- `no_levels_reported` is a - // legitimate arm -- and then there is no object to nest a fact in. - return; - }; - - let unclassified = [ - ( - "a number in a cache object", - cache.replace('{', r#"{"latency":7,"#), - ), - ( - "a string in a cache object", - cache.replace('{', r#"{"note":"x","#), - ), - ]; - - for (what, to) in unclassified { - let injected = text.replace(&cache, &to); - - assert_ne!( - injected, text, - "{what}: the injection did not apply, so this proves nothing" + // Be clear about what it is: the row is BUILT from these lists, so this is + // the writer being checked against its input, not an independent reading. + // That is exactly the check worth having here -- the writer is the one thing + // no amount of typing upstream can check for itself -- but it is narrower + // than "the report is correct" and should not be read as that. + for shape in shapes() { + let text = windows_platform_probes::topology_report::report( + &banner_for(&shape.observation), + &shape.observation, ); - // **And the report must still agree with itself before the mutation.** - // If the injection itself creates a violation, every later assertion - // sees it and this test can pass for a reason unrelated to the fact - // being unread -- which is exactly how the `policies` case fooled it. + let check = shape.observation.cross_check(); + let expected: Vec = check + .disagreements + .iter() + .map(|entry| entry.code().to_owned()) + .chain( + check + .not_compared + .iter() + .map(|entry| entry.code().to_owned()), + ) + .chain( + check + .parse_incomplete + .iter() + .map(|entry| entry.code().to_owned()), + ) + .collect(); + assert_eq!( - report_oracle::check(&injected), - [], - "{what}: injecting an unread fact must not itself be a violation, or \ - the accounting's panic below proves nothing about coverage" + published_codes(&text, DIAGNOSTIC_LISTS), + expected, + "{}: the row must publish every condition the cross-check found, in \ + order\n\n--- the report ---\n{text}", + shape.what, ); + } +} +#[test] +fn the_corpus_reaches_states_that_block_agreement() { + // **The guard that stops the two tests above passing for nothing.** Both + // skip or trivially satisfy a shape in no blocking state, so a corpus that + // had drifted to all-healthy shapes would leave them green while checking + // nothing -- the failure mode this crate keeps meeting. + // + // Stated as a relation rather than a count: at least one shape blocks, and + // at least one does not, so both sides of every rule above are exercised. + let blocking = shapes() + .iter() + .filter(|shape| !invariant::blocking_states(&shape.observation).is_empty()) + .count(); - let accounted = std::panic::catch_unwind(|| { - account_for_every_fact(&injected, "a report carrying an unclassified nested fact") - }); + assert!( + blocking > 0, + "no shape in the corpus is in a blocking state, so the publication \ + rules are vacuous" + ); + assert!( + blocking < shapes().len(), + "every shape blocks, so the acceptance half of the publication rules \ + is never exercised" + ); - assert!( - accounted.is_err(), - "{what} was published and no rule reads it, and the accounting accepted \ - the report anyway -- the shallow enumeration has become a real hole" - ); - } + // **ORDER is only a claim where there is more than one entry to order.** + // `the_row_lists_exactly_the_conditions_the_cross_check_found` compares the + // row's codes against the cross-check's as a SEQUENCE, which is what makes + // it stronger than the prose count it replaced -- but a corpus whose shapes + // each carry at most one condition can never tell a sequence from a set. + // + // Measured, and this is why the guard exists: reversing the row's + // `parse_incomplete` order reddened nothing until a multi-condition shape + // was in the corpus. + // Within ONE list, not summed across the three. Summing was the first + // version of this guard and it passed while the sabotage still reddened + // nothing: a shape carrying one `not_compared` and one `parse_incomplete` + // has two conditions and no order to get wrong, because reversing a + // one-element list is the identity. + let most = shapes() + .iter() + .map(|shape| { + let check = shape.observation.cross_check(); + check + .disagreements + .len() + .max(check.not_compared.len()) + .max(check.parse_incomplete.len()) + }) + .max() + .unwrap_or_default(); + + assert!( + most >= 2, + "no shape carries two conditions in ONE list, so the ordering half of \ + the publication rule is vacuous -- it cannot tell a sequence from a set" + ); } #[test] -fn one_blocks_caveat_does_not_excuse_another_blocks_claim() { - // **A crossed shape: heterogeneous AND in doubt AND the `Level` cache arm.** - // That arm writes the same "did not establish that the parse is whole" - // sentence the heterogeneity caveat does, so a report-global search for the - // caveat could be satisfied by the WRONG block and accept an uncaveated - // hardware claim. A review predicted exactly that. +fn a_state_the_row_does_not_publish_fails_the_accounting() { + // **The accounting's own sabotage, so it cannot go quietly blind.** A rule + // that enumerates states and finds them all published is indistinguishable + // from one that enumerates nothing -- unless something shows it failing. // - // Measured on this shape before the fix: it did NOT mask -- but only because - // the renderer wraps the cache arm's sentence, so no single line carries the - // whole token. The protection was an accident of where a line breaks, and - // reflowing that sentence would have turned the oracle blind with nothing to - // notice. The caveat search is now scoped to the claim's own block; this test - // is what keeps that true. - let mut crossed = base(); - crossed.numa_domains_only_in_cpu_sets = 2; - crossed.cores = vec![ - CoreShape { - simultaneous_multithreading: true, - efficiency_class: 0, - processors: 8, - }, - CoreShape { - simultaneous_multithreading: false, - efficiency_class: 1, - processors: 8, - }, - ]; + // Reproduces the `disagreements` defect exactly: a report in a blocking + // state whose row carries no condition for it. + let mut observation = base(); + observation.partitioning_cache_level = Some(9); - let text = windows_platform_probes::topology_report::report(&banner_for(&crossed), &crossed); + let text = + windows_platform_probes::topology_report::report(&banner_for(&observation), &observation); - // The shape really is the crossed one, or this proves nothing. assert!( - text.contains("(heterogeneous:"), - "the shape must make the gated hardware claim:\n{text}" + !invariant::blocking_states(&observation).is_empty(), + "the fixture must be in a blocking state or this shows nothing" ); assert!( - text.contains("of the levels that decoded"), - "and must carry the cache arm's own caveat, which is the masking \ - candidate:\n{text}" - ); - assert_eq!( - report_oracle::check(&text), - [], - "the crossed shape is legitimate and must be accepted as rendered" + !published_codes(&text, DIAGNOSTIC_LISTS).is_empty(), + "the row publishes it today, which is what the rule requires" ); - // Remove ONLY the heterogeneity caveat. The cache arm's caveat stays. - let caveat = text + // Now empty every diagnostic list in the row, which is what a renderer that + // forgot to publish one would produce. + // + // **Done by parsing, emptying and re-rendering, not by cutting the text.** + // Two earlier versions cut it: the first split on commas, which sliced the + // entries in half once they became objects; the second used a span helper in + // the oracle that was not string-aware. Both are the same mistake -- a + // sabotage that hand-parses is a sabotage that can stop sabotaging while + // still passing, and it leaves the rule it guards unguarded. Rebuilding from + // a parse also produces a row that is genuinely valid, so what this feeds the + // accounting is a report a renderer could really have emitted. + let stripped = text .lines() - .find(|line| line.trim_start().starts_with("(This run did not establish")) - .map(str::to_owned) - .expect("the heterogeneity caveat must be present to be removed"); - let uncaveated = text.replace(&format!("{caveat}\n"), ""); + .map(|line| { + if !line.starts_with('{') { + return line.to_owned(); + } + let Ok(mut parsed) = serde_json::from_str::(line) else { + return line.to_owned(); + }; + for key in DIAGNOSTIC_LISTS { + if let Some(list) = parsed.get_mut(*key) + && list.is_array() + { + *list = serde_json::Value::Array(Vec::new()); + } + } + parsed.to_string() + }) + .collect::>() + .join("\n"); - assert_ne!(uncaveated, text, "the caveat removal must apply"); assert!( - report_oracle::check(&uncaveated) + published_codes(&stripped, DIAGNOSTIC_LISTS).is_empty(), + "the sabotage must actually remove the conditions: {stripped}" + ); + + // **And the accounting must REJECT it.** Asserting only that the stripping + // worked demonstrated the sabotage and nothing else -- the rule could have + // been deleted and this stayed green, which is the shape of vacuity this + // whole suite exists to avoid. Calling the same predicate the corpus rule + // calls is what makes this a test of the rule. + assert!( + !publication_holds(&observation, &stripped), + "a report in a blocking state whose row publishes nothing must fail the \ + accounting: {stripped}" + ); + assert!( + publication_holds(&observation, &text), + "and the unsabotaged report must pass it, or the rule rejects \ + everything: {text}" + ); +} + +#[test] +fn the_row_publishes_one_code_per_anomaly_the_observation_carries() { + // **The axis `DIAGNOSTIC_LISTS` deliberately leaves out.** The prose folds + // every anomaly into one sentence, so the prose cannot say how many there + // were beyond the number inside that sentence -- and checking a number + // inside a sentence is the prose-reading this milestone retired. + // + // Checked against the OBSERVATION instead, which is the artifact the row is + // supposed to be faithful to. That is the relation worth having: a survey + // grouping anomalies by kind is reading this list, and it must have one + // entry per anomaly the enumeration actually recorded. + for shape in shapes() { + let text = windows_platform_probes::topology_report::report( + &banner_for(&shape.observation), + &shape.observation, + ); + + assert_eq!( + published_codes(&text, &["enumeration_anomalies"]).len(), + shape.observation.enumeration_anomalies.len(), + "{}: the observation carries {} anomal(ies) and the row publishes \ + {:?}\n\n--- the report ---\n{text}", + shape.what, + shape.observation.enumeration_anomalies.len(), + published_codes(&text, &["enumeration_anomalies"]), + ); + } +} + +#[test] +fn the_corpus_reaches_a_shape_that_records_anomalies() { + // The guard for the rule above: on an all-clean corpus it compares zero + // against zero on every shape and establishes nothing. + assert!( + shapes() .iter() - .any(|found| matches!( - found, - report_oracle::Correspondence::UncaveatedClaimUnderDoubt { - claim: "heterogeneity", - .. - } - )), - "another block's caveat must not answer for this claim: {:#?}", - report_oracle::check(&uncaveated) + .any(|shape| !shape.observation.enumeration_anomalies.is_empty()), + "no shape records an anomaly, so the per-anomaly rule is vacuous" ); }