A floor per cell: shapes may improve, and may not quietly get worse (#198, step 3) - #406
Merged
Merged
Conversation
…198, step 3) The committed report makes every changed answer a reviewed diff. That catches a regression only if somebody reads the diff and knows which direction is bad — and a diff shows `header` becoming `rejected` in exactly the same shade as the reverse. The two questions are different: *"did anything move?"* and *"did anything move **down**?"*, and only the second can be enforced without a reviewer. So each cell that produces something now carries a floor — the level it has been seen to reach — in a committed `GUARANTEES.md`. 130 of them: 45 at `header`, 75 at `compiles`, 10 at `generates`. Rising above a floor is free and silent. Falling below it fails a test naming the cell, the floor and the level it now reaches. **Raising is automatic, lowering is a hand edit.** `--update-guarantees` raises floors and never lowers one. Giving up on a shape that used to work should cost a line in a diff a reviewer can argue with, not a silently regenerated artifact — which is exactly what would happen if the file were rewritten on every run. The ladder is per target, because C runs one stage further than JNI: `header` is C's ceiling, `compiles` is JNI's until the Kotlin compiler runs. A shared ceiling would let a C cell lose its header without falling below any floor. Four tests, three of which exist because a gate that has never been observed failing is a gate nobody can trust: * the ratchet reports exactly the cell that fell, and ignores the one that rose; * `--update-guarantees` leaves a floor standing when the run does worse; * every floor names a cell the matrix still enumerates, so a renamed shape cannot leave an entry behind being satisfied by nothing; * and the gate itself, against the floors this repository commits. Verified end to end as well as in unit form: claiming `header` for a JNI cell fails with array_scalar__field__jni: was guaranteed to reach `header`, now reaches only `compiles` `report::survey()` is memoized, so the report and the ratchet share one pass over both generators rather than doubling the suite's cost to compute the same answer. **Exit: answer-preserving.** `REPORT.md` is byte-identical; this stage adds a second gate over the same measurements and moves none of them. Part of #198, tracked by #399.
The ratchet's third level waits on 2c: there is no runtime state for a floor to stand on yet.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Step 3 of #198, on
shape-coverage(#402).Why the committed report is not enough
REPORT.mdmakes every changed answer a reviewed diff. But it shows a cell goingheader→rejectedin exactly the same shade as one going the other way, so it catches a regression only if a reviewer reads the diff and knows which direction is which.Two different questions:
REPORT.mdGUARANTEES.mdThe ratchet
Each cell that produces something carries a floor — the level it has been seen to reach. 130 floors: 45 at
header, 75 atcompiles, 10 atgenerates(50 C cells, 80 JNI).Rising above a floor is free and silent. Falling below it fails a test that names the cell, the floor, and where it is now:
Raising is automatic, lowering is a hand edit.
cargo run -p shape-matrix -- --update-guaranteesraises floors and never lowers one. That asymmetry is the whole mechanism: if the file were rewritten on every run, a floor would follow its cell downwards and the ratchet would hold nothing. Giving up on a shape that used to work should cost a line in a diff a reviewer can argue with.The ladder is per target, because C runs one stage further:
headeris C's ceiling,compilesis JNI's until the Kotlin compiler runs. A shared ceiling would let a C cell lose its header without falling below any floor.Tests
Four, and three of them exist because a gate nobody has seen fail is a gate nobody can trust:
--update-guaranteesleaves a floor standing when the run does worse — the ratchet's integrity, tested directly;The first three run on synthetic floors rather than on the repository's, deliberately: a rule whose behaviour can only be observed by breaking the repo is a rule nobody checks. I also verified it end-to-end by hand — claiming
headerfor a JNI cell produces the failure above — and reverted.Exit: answer-preserving
REPORT.mdis byte-identical. This stage adds a second gate over the same measurements and moves none of them.One performance note:
report::survey()is now memoized, so the report and the ratchet share one pass over both generators plus onecargo check, rather than doubling the suite's cost to compute the same answer twice. The suite runs in ~4s.Not in this step
The issue also asks for a
must executelevel. There is no runtime state for a floor to stand on until step 2c, so the ladder stops atheader.Checks
cargo test -p shape-matrix(14 tests), clippy-D warningsover all targets and features,cargo fmt --checkwith CI's config,RUSTDOCFLAGS=-D warnings cargo doc, andREPORT.mdunchanged.