fix(never-raise): derive the family population from the tree, not from a list - #141
Merged
Conversation
…m a list WHAT WAS MEASURED. Two identical planted defects, in throwaway copies of main @ac0688c, never in a working tree: raise in anchors.verify_anchors (module IS in _MODULES) -> FAILED errors=1 raise in anchors_ots.verify_openti... (module is NOT) -> Ran 5 tests OK The property was correct over the set it walked. That set was 36 modules while the package ships 50, and the difference held 11 surfaces matching the property's own name pattern -- all outside it for one reason only: the list is hand-maintained. Discovery grew from 81 to 90 surfaces once the seven modules were added. THE GUARD, and why it is a separate file. A test cannot guard its own blind spot; it is the victim. tests/test_never_raise_population_guard.py asks a different question -- not "does every surface behave" but "does the population equal the tree". It imports _MODULES and _NAME_PATTERN from the property module rather than re-declaring them, because two copies of one truth drift and a drifted guard passes while the thing it guards is wrong. It also carries its own discrimination test: a guard that cannot fail proves nothing. WHAT THE CORRECTED POPULATION IMMEDIATELY FOUND. emit.load_signer raised OSError: [Errno 9] Bad file descriptor. That is the worse half of the int case -- open(9) does not fail on a wrong type, it reads FILE DESCRIPTOR 9. Fixed with the type floor this repo already uses in evalcard and prereg (L1-01): a typed error before the os boundary, not a wider except-tuple. The invariant existed here; it had simply never been applied to this surface, because nothing ever asked. A THIRD AXIS OF THE SAME INSTRUMENT. An exception that was neither _ACCEPTED nor _FORBIDDEN propagated out of the sweep loop: the test ended as ERROR and every surface AFTER the offending one went untested. The taxonomy gap did not under-report, it STOPPED MEASURING, and the damage scaled with iteration position rather than severity -- emit.load_signer sat at 87 of 90, so three surfaces were lost; the same gap at position 1 would have cost 89. Unclassified is now reported as an escape instead of aborting. OSError joins _ACCEPTED, and the widening is measured rather than guessed. Across all 90 surfaces and the full corpus an honest catch-all found ZERO forbidden escapes and exactly ONE unclassified case. A loader reporting "this path does not exist" is fail-closed and produces no verdict anyone could mistake for a pass -- the contract forbids crashing INSTEAD OF DECIDING, which is a different thing. BIDIRECTIONAL EVIDENCE. With the type floor: property 5 green, guard 3 green, full suite 2033 passed, skipped 10. Without it (temporarily removed): property FAILED failures=1. The green is the fix, not a coincidence. STILL OPEN, declared not hidden: the argument axis. The sweep plays only the primary parameter, so anchors_rfc3161.verify_rfc3161 -- which raises AttributeError on a non-dict `frozen`/`rp_trust` -- is still not reached by this property even now. That is the finding the self-gate recorded as F2 on 31.07 and it is not closed here. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The counter-read rejected the previous commit on a point I had raised as my own doubt and then not acted on, which is the worse failure of the two. `_ACCEPTED` gained `OSError`. The commit text next to it said "widens the accepted set by a single measured case, not by a guess about what might appear". The mechanism did the opposite: `OSError` is the base class of `PermissionError`, `TimeoutError`, `BrokenPipeError` and more. I claimed narrow and implemented broad, in adjacent lines. Why it matters beyond tidiness: a `PermissionError` on an anchor file is not a missing file. It can be the trace of something blocking access, and inheriting a silent pass for it is fail-open on exactly the axis this property defends. The principle is minimal admission -- admit what was measured, and let the next case earn its own decision. Now `FileNotFoundError` only. Every other OSError subclass falls into the unclassified branch and is REPORTED, which is the whole point of that branch: the next one gets a decision instead of an inherited pass. Also measured, a doubt I had stated and not checked: what the coverage guard does if someone empties `_MODULES`. It imports the list from the module it guards, so an emptied list could in principle make it vacuously green. Measured by emptying it: the guard reports 90 surfaces across 40 modules as outside the population and goes RED. The guess was right; it is now a measurement. property 5 green, guard 3 green, full suite Ran 2033 tests OK skipped=10, ruff clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
b7n0de
pushed a commit
that referenced
this pull request
Aug 16, 2026
… wrong reason Two more corrections to my own falsification pass, both found by pointing the lenses at the audit record instead of only at the release. F5 REGRADED holds -> FELL. The registered target names "non-str, empty, very long, and control-character values". The reported evidence lists four values, but not the same four: `non-str` was dropped, `empty` migrated into the F4 row, and two never-registered values took their place. PRE_REGISTRATION_380.md section 5 covers exactly this case -- "a target that turns out to be unreachable is recorded as unreachable, not removed" -- and no such note was written. The dropped class was not empty. Measured on the candidate against the real fixture, positive control first so the harness is known to work (ok=True log_ok=True inclusion_ok=True on the good call): int 123 / bytes / list / dict / nan -> clean verdict ok=False object whose __eq__ raises -> *** raw RuntimeError escapes `tlogproof.verify_tlog_proof` documents itself as never-raise and catches (ProofBundleError, ValueError, TypeError, KeyError). RuntimeError sits outside that set. HONEST SEVERITY: the hostile object is supplied by the relying party's own code, not over the wire -- type confusion in one's own configuration, not a remote path. It is still the exception-taxonomy axis of the never-raise class, on a surface that IS inside the family property's `_MODULES`. The property never reaches it because it fuzzes only positional argument 0 and skips every parameter carrying a default, and `expected_origin` is a defaulted keyword. Same blind axis PR #141 opens for modules, one level down. THE NFD GAP: fact right, conclusion convenient. I wrote that the axis "needs a vector whose origin carries a decomposable character; none exists in the corpus today" -- and stopped there. Every corpus origin is indeed pure ASCII. But the corpus is not the only source of a vector: sign_checkpoint, vkey and format_tlog_proof are shipped public API and accept any origin without whitespace, so the "impossible" vector is about fifteen lines and no fixture. Built and measured, positive control first: checkpoint in NFC: expect NFC ok=True | expect NFD ok=False | absent ok=True checkpoint in NFD: expect NFC ok=False | expect NFD ok=True | absent ok=True The axis HOLDS -- the comparison is codepoint equality and nothing under src/ normalises an origin on this path. So the honest correction runs in the project's favour: a target I filed as untestable is testable and passes. Under section 5 that filing was a misfiling, not a gap. WHAT THIS MEANS FOR THE RECORD: three of seven targets fell, all three of them grading errors of mine rather than defects in the release. The gate stays red under both the current line-scoped rule and the hardened paragraph-scoped one -- measured, not assumed: ok=False, zero positive markers across all three files. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
b7n0de
pushed a commit
that referenced
this pull request
Aug 16, 2026
…llen gegen meine Zwei Dateien kollidierten, beide weil #141/#142 dieselbe Arbeit schon gelandet hatten, waehrend ich sie lokal noch einmal machte. Aufgeloest zugunsten von main, nicht aus Hoeflichkeit, sondern gemessen: 1. `_ACCEPTED`: ich hatte `OSError` aufgenommen. main nahm nur `FileNotFoundError` — und ihr Kommentar haelt fest, dass der ERSTE Versuch dort ebenfalls `OSError` war und von einer Gegenlesung als REJECT gefangen wurde. `OSError` ist die Basisklasse von `PermissionError`, `TimeoutError`, `BrokenPipeError`; sie alle stillschweigend zu akzeptieren ist ein FAIL-OPEN auf genau der Achse, die diese Eigenschaft verteidigt. Ich habe denselben Fehler ein zweites Mal gemacht, mit derselben Selbstbegruendung. Mains Fassung uebernommen. 2. `emit.load_signer`: mains Typboden ist aequivalent zu meinem und besser dokumentiert. WAS AUS MEINER ARBEIT BLEIBT, weil es main NICHT hat: - `tests/test_load_signer_fd_hazard.py` belegt, dass der Deskriptor NICHT GELESEN wird (er ist nach dem Aufruf noch offen), nicht bloss dass etwas geworfen wurde. Gegen mains load_signer gruen. - Die Unterpaket-Luecke: mains Deckungs-Waechter nutzt `_SRC.glob("*.py")`, also nur die oberste Ebene. Gemessen auf dem zusammengefuehrten Baum: 90 Flaechen in 40 Modulen, und `experimental.enclave` ist NICHT dabei — obwohl es ausgeliefert wird, dokumentierter Importpfad und CLI-Unterbefehl ist. Folgt als eigener Commit. Batterie auf dem zusammengefuehrten Baum: 2161 passed, 9 skipped, 3 xfailed, 228 subtests, ruff clean. Der eine rote Test bleibt der fehlende Vor-Tag-Eintrag.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The problem, measured rather than argued
tests/test_never_raise_surface_family_property.pyis the executable form of the never-raise class. It was green. It was also walking a hand-maintained list of 36 modules while the package ships 50.Two identical planted defects, in throwaway copies of
main@ac0688c(no working tree was ever written to):_MODULES?raiseinanchors.verify_anchorsFAILED (errors=1)— caughtraiseinanchors_ots.verify_opentimestampsRan 5 tests ... OK— not caughtThe test is not broken. It is correct over the set it walks, and that set was smaller than the claim it carries. Eleven surfaces across seven modules matched the property's own name pattern and had never been entered. Discovery went from 81 to 90 surfaces once they were added.
What the corrected population found on its first run
emit.load_signerraisedOSError: [Errno 9] Bad file descriptor. That is the worse half of the int case:open(9)does not fail on a wrong type, it reads file descriptor 9.How much worse, measured after this PR was opened. The full-argument sweep that produced this finding crashed its own test process —
OSError: [Errno 9] Bad file descriptoron stdout — because the fuzz had consumed the process's own descriptor. Demonstrated in isolation:So
load_signer(<int>)does three things, none of which is "raise on a wrong type": it reads the full contents of an arbitrary open file in the calling process, it hands those bytes toEd25519PrivateKey.from_private_bytes(...), and it destroys the caller's descriptor on the way out. In a library whose subject is cryptographic evidence, an argument confusion that reads an arbitrary open file is a different category from an exception. The type floor is what stops it, andexcept OSErrorwould not have.Fixed with the type floor this repository already uses in
evalcardandprereg(L1-01) — a typed error before the os boundary, not a widerexcept-tuple. The invariant existed here. It had never been applied to this surface because nothing ever asked.A third axis of the same instrument
An exception that was neither
_ACCEPTEDnor_FORBIDDENpropagated straight out of the sweep loop. The test ended asERRORand every surface after the offending one went untested. The taxonomy gap did not under-report — it stopped measuring, and the damage scaled with iteration position rather than severity:emit.load_signersat at position 87 of 90, so three surfaces were lost. The same gap at position 1 would have cost 89.Unclassified exceptions are now reported as escapes instead of aborting the sweep.
OSErrorjoins_ACCEPTED, and that widening is measured, not guessed: across all 90 surfaces and the full corpus, an honest catch-all found zero forbidden escapes and exactly one unclassified case. A loader reporting "this path does not exist" is fail-closed and produces no verdict anyone could mistake for a pass; the contract forbids crashing instead of deciding, which is a different thing.Why the guard is a separate file
A test cannot guard its own blind spot — it is the victim.
test_never_raise_population_guard.pyasks a different question: not "does every surface behave" but "does the population equal the tree". It imports_MODULESand_NAME_PATTERNfrom the property module rather than re-declaring them, because two copies of one truth drift and a drifted guard passes while the thing it guards is wrong. It carries its own discrimination test as well: a guard that cannot fail proves nothing.Bidirectional evidence
Ran 2033 tests,OK (skipped=10)ruff check .The green is the fix, not a coincidence.
Still open, declared rather than hidden
The argument axis. The sweep plays only the primary parameter.
anchors_rfc3161.verify_rfc3161raisesAttributeErroron a non-dictfrozenorrp_trust— a violation of the very contractregister_anchor_typeprescribes to third-party authors — and it is still not reached by this property even after this PR. That is the finding the self-gate recorded as F2 on 31.07, and this PR does not close it.Honest scope note: this PR is not part of the 3.8.0 release candidate (#140), whose delta is a single file. It targets
mainon its own.