From 171fc64f1cce1c7881b1c933ae366ff85a565457 Mon Sep 17 00:00:00 2001 From: kraxo Date: Sun, 16 Aug 2026 13:34:52 +0200 Subject: [PATCH 1/2] fix(never-raise): derive the family population from the tree, not from a list WHAT WAS MEASURED. Two identical planted defects, in throwaway copies of main @ac0688c, never in a working tree: raise in anchors.verify_anchors (module IS in _MODULES) -> FAILED errors=1 raise in anchors_ots.verify_openti... (module is NOT) -> Ran 5 tests OK The property was correct over the set it walked. That set was 36 modules while the package ships 50, and the difference held 11 surfaces matching the property's own name pattern -- all outside it for one reason only: the list is hand-maintained. Discovery grew from 81 to 90 surfaces once the seven modules were added. THE GUARD, and why it is a separate file. A test cannot guard its own blind spot; it is the victim. tests/test_never_raise_population_guard.py asks a different question -- not "does every surface behave" but "does the population equal the tree". It imports _MODULES and _NAME_PATTERN from the property module rather than re-declaring them, because two copies of one truth drift and a drifted guard passes while the thing it guards is wrong. It also carries its own discrimination test: a guard that cannot fail proves nothing. WHAT THE CORRECTED POPULATION IMMEDIATELY FOUND. emit.load_signer raised OSError: [Errno 9] Bad file descriptor. That is the worse half of the int case -- open(9) does not fail on a wrong type, it reads FILE DESCRIPTOR 9. Fixed with the type floor this repo already uses in evalcard and prereg (L1-01): a typed error before the os boundary, not a wider except-tuple. The invariant existed here; it had simply never been applied to this surface, because nothing ever asked. A THIRD AXIS OF THE SAME INSTRUMENT. An exception that was neither _ACCEPTED nor _FORBIDDEN propagated out of the sweep loop: the test ended as ERROR and every surface AFTER the offending one went untested. The taxonomy gap did not under-report, it STOPPED MEASURING, and the damage scaled with iteration position rather than severity -- emit.load_signer sat at 87 of 90, so three surfaces were lost; the same gap at position 1 would have cost 89. Unclassified is now reported as an escape instead of aborting. OSError joins _ACCEPTED, and the widening is measured rather than guessed. Across all 90 surfaces and the full corpus an honest catch-all found ZERO forbidden escapes and exactly ONE unclassified case. A loader reporting "this path does not exist" is fail-closed and produces no verdict anyone could mistake for a pass -- the contract forbids crashing INSTEAD OF DECIDING, which is a different thing. BIDIRECTIONAL EVIDENCE. With the type floor: property 5 green, guard 3 green, full suite 2033 passed, skipped 10. Without it (temporarily removed): property FAILED failures=1. The green is the fix, not a coincidence. STILL OPEN, declared not hidden: the argument axis. The sweep plays only the primary parameter, so anchors_rfc3161.verify_rfc3161 -- which raises AttributeError on a non-dict `frozen`/`rp_trust` -- is still not reached by this property even now. That is the finding the self-gate recorded as F2 on 31.07 and it is not closed here. Co-Authored-By: Claude Opus 5 (1M context) --- src/proofbundle/emit.py | 12 ++ tests/test_never_raise_population_guard.py | 120 ++++++++++++++++++ ...est_never_raise_surface_family_property.py | 29 ++++- 3 files changed, 160 insertions(+), 1 deletion(-) create mode 100644 tests/test_never_raise_population_guard.py diff --git a/src/proofbundle/emit.py b/src/proofbundle/emit.py index a2d7d490..28352b88 100644 --- a/src/proofbundle/emit.py +++ b/src/proofbundle/emit.py @@ -64,6 +64,18 @@ def save_signer(key: Ed25519PrivateKey, path: str) -> None: def load_signer(path: str) -> Ed25519PrivateKey: """Load an Ed25519 signing key from a 32 byte raw seed file.""" + # TYPE FLOOR, same invariant as evalcard/prereg (L1-01) — applied here only on 2026-08-16, because + # until then this surface sat OUTSIDE the never-raise family property: `emit` was not in `_MODULES`, + # so nothing ever asked the question. The moment the population was derived from the tree instead of + # a hand-maintained list, the property caught this on its first run. + # + # Measured before the fix: `load_signer(9)` raised `OSError: [Errno 9] Bad file descriptor`. That is + # the worse half of the int case — `open(9)` does not fail on a wrong type, it reads FILE + # DESCRIPTOR 9. A wider except-tuple would hide the escape while leaving the fd read in place; the + # floor is the fix, and it belongs before the os boundary, not after it. + if not isinstance(path, (str, bytes, os.PathLike)): + from .errors import BundleFormatError as _BFE # noqa: PLC0415 + raise _BFE(f"signer key path must be a path string, got {type(path).__name__} (fail-closed)") with open(path, "rb") as handle: return Ed25519PrivateKey.from_private_bytes(handle.read()) diff --git a/tests/test_never_raise_population_guard.py b/tests/test_never_raise_population_guard.py new file mode 100644 index 00000000..8d9e6eee --- /dev/null +++ b/tests/test_never_raise_population_guard.py @@ -0,0 +1,120 @@ +"""The never-raise family property must walk the tree, not a maintained list. + +WHY THIS EXISTS. `tests/test_never_raise_surface_family_property.py` is the executable form of the +never-raise class: no public surface may terminate with a raw exception on hostile input. It carries +its own regression floor on the DENOMINATOR (`test_discovery_finds_the_expected_surface_family`), and +that floor is real — but it guards against the denominator *collapsing*, not against it being +*incomplete by construction*. The module list `_MODULES` is hand-maintained, so a module added to the +package is silently outside the property until somebody remembers to add it. + +MEASURED, 2026-08-16, on `main` @ ac0688c. Two identical planted defects in throwaway copies: + + raise in anchors.verify_anchors (module IS in _MODULES) -> FAILED (errors=1), caught + raise in anchors_ots.verify_openti... (module is NOT) -> Ran 5 tests, OK, NOT caught + +The property is correct over the set it walks. That set was 36 modules while the package shipped 50, +and the difference held 11 surfaces matching the property's own name pattern — one of them a live +contract violation (`anchors_rfc3161.verify_rfc3161` raises `AttributeError` on a non-dict `frozen` +or `rp_trust`, which is exactly what `register_anchor_type` forbids third-party authors from doing). + +WHY A SEPARATE FILE AND NOT AN ASSERTION IN THE PROPERTY ITSELF. A test cannot be the guard of its +own blind spot — it is the victim. This guard asks a different question than the property does: not +"does every surface behave" but "does the property's population equal the tree's". Keeping them apart +means a future edit that shrinks `_MODULES` fails HERE, loudly, instead of silently making the +property cheaper to pass. + +This guard deliberately derives BOTH sides from the same source the property uses, so it cannot drift +from it: the module list and the name pattern are imported from the property module rather than +re-declared here. Re-declaring them would create a second copy of the same truth, which is the class +of defect this file exists to prevent. +""" +from __future__ import annotations + +import importlib +import inspect +import pathlib +import unittest + +_SRC = pathlib.Path(__file__).resolve().parents[1] / "src" / "proofbundle" + +# Import the property module's OWN definitions. Never re-declare them: two copies of one truth drift, +# and a drifted guard would pass while the thing it guards is wrong. +_prop = importlib.import_module("tests.test_never_raise_surface_family_property") +_MODULES = set(_prop._MODULES) +_NAME_PATTERN = _prop._NAME_PATTERN + + +def _package_modules() -> set[str]: + """Every shipped module name, from the tree — the ground truth the population is measured against.""" + return {p.stem for p in _SRC.glob("*.py") if not p.stem.startswith("_")} + + +def _never_raise_surfaces_in(mod_name: str) -> list[str]: + """Functions DEFINED in this module whose name matches the property's own family pattern. + + `__module__` is checked so a name merely imported into the module does not count as one of its + surfaces — otherwise a re-export would inflate every module's apparent population. + """ + try: + mod = importlib.import_module(f"proofbundle.{mod_name}") + except Exception: # noqa: BLE001 — an optional-extra module that will not import is out of scope + return [] + return sorted( + n for n, f in vars(mod).items() + if not n.startswith("_") + and _NAME_PATTERN.match(n) + and inspect.isfunction(f) + and getattr(f, "__module__", "") == f"proofbundle.{mod_name}" + ) + + +class TestNeverRaisePopulationIsDerivedFromTheTree(unittest.TestCase): + + def test_no_shipped_surface_sits_outside_the_property(self): + """The invariant: every never-raise-shaped surface in the package is inside the population. + + A failure here does NOT mean the surface misbehaves. It means nobody has ever asked whether it + does — which is the more expensive state, because a green property is read as covering it. + """ + aussen: dict[str, list[str]] = {} + for mod_name in sorted(_package_modules() - _MODULES): + treffer = _never_raise_surfaces_in(mod_name) + if treffer: + aussen[mod_name] = treffer + gesamt = sum(len(v) for v in aussen.values()) + self.assertEqual( + aussen, {}, + f"{gesamt} never-raise surface(s) in {len(aussen)} module(s) are outside the property's " + f"population, so the property has never entered them: {aussen}. " + "Add the module to _MODULES in tests/test_never_raise_surface_family_property.py — and " + "expect the property to go red if the surface actually violates the contract, which is " + "the point.") + + def test_the_population_names_only_modules_that_exist(self): + """The other direction: a module listed but no longer shipped makes the population lie the + other way — it inflates the apparent denominator with names that walk nothing.""" + verwaist = sorted(_MODULES - _package_modules()) + self.assertEqual(verwaist, [], + f"_MODULES names {len(verwaist)} module(s) that the package does not ship: " + f"{verwaist}. The denominator counts them and nothing walks them.") + + def test_this_guard_actually_discriminates(self): + """Bidirectional validation: the guard must be able to FAIL, or its green means nothing. + + A guard whose predicate can only ever pass is decoration. This exercises the comparison with a + deliberately wrong population and requires it to detect the difference. + """ + paket = _package_modules() + self.assertTrue(paket, "no shipped modules discovered — the ground truth itself is broken") + # A population missing a module that HAS surfaces must be detectable. + kandidaten = [m for m in sorted(paket) if _never_raise_surfaces_in(m)] + self.assertTrue(kandidaten, "no module with never-raise surfaces found — the pattern matched nothing") + kuenstlich = set(kandidaten[1:]) # drop one module that provably has surfaces + fehlend = {m: _never_raise_surfaces_in(m) for m in (paket - kuenstlich) if _never_raise_surfaces_in(m)} + self.assertNotEqual(fehlend, {}, + "the comparison did not notice a module removed from the population — " + "this guard cannot fail and therefore proves nothing") + + +if __name__ == "__main__": # pragma: no cover + unittest.main() diff --git a/tests/test_never_raise_surface_family_property.py b/tests/test_never_raise_surface_family_property.py index b544c570..49544da9 100644 --- a/tests/test_never_raise_surface_family_property.py +++ b/tests/test_never_raise_surface_family_property.py @@ -30,6 +30,12 @@ # the sweep, hiding the decision/outcome/subject_binding RecursionError class): "subject_binding", "relation", "assurance", "automation_verdict", "beacon", "public_transparency", "signature", "policy_profiles", "canonical", + # 2026-08-16: the population was hand-maintained and had drifted 14 modules behind the package. + # A coverage guard (tests/test_never_raise_population_guard.py) now derives the expected set from + # the tree, so a module added to the package can no longer sit outside this property unnoticed. + # These seven carried 11 matching surfaces the property had never entered. + "anchors_chia", "anchors_markovian", "anchors_ots", "anchors_rfc3161", "anchors_rootcommit", + "emit", "pqsig", ] # Broadened name family (round 8): the predicate-validation surfaces a relying party actually calls # (validate_*/require_valid_*/require_derived_*/classify_*/derive_*) were entirely outside the old pattern. @@ -43,7 +49,14 @@ # ACCEPTED terminations: a returned value, or a TYPED fail-closed error. ProofBundleError covers # BundleFormatError / BudgetExceeded / PQUnavailable / UnsupportedError / CanonicalizerUnavailable / PolicyError # / SdjwtVcError / EvalClaimError-as-PBError; ValueError covers EvalClaimError + the rfc8785 domain family. -_ACCEPTED = (ProofBundleError, ValueError) +# `OSError` added 2026-08-16, and the reason belongs here rather than in a commit nobody re-reads. +# The never-raise contract forbids a surface CRASHING INSTEAD OF DECIDING — the type-confusion +# signatures in `_FORBIDDEN`. A loader that reports "this path does not exist" is the opposite of that: +# it is fail-closed, it is informative, and it produces no verdict a relying party could mistake for a +# pass. Measured before adding it, across all 90 discovered surfaces and the full corpus: ZERO +# forbidden escapes and exactly ONE unclassified case (`emit.load_signer` on `b"bytes-not-str"`). +# So this widens the accepted set by a single measured case, not by a guess about what might appear. +_ACCEPTED = (ProofBundleError, ValueError, OSError) # FORBIDDEN raw terminations = the type-confusion crash signatures a public verify surface must never emit. _FORBIDDEN = (AttributeError, TypeError, RecursionError, KeyError, IndexError, UnicodeDecodeError, MemoryError) @@ -158,6 +171,20 @@ def test_no_public_surface_raises_raw_on_hostile_primary(self): except _FORBIDDEN as exc: escapes.append(f"{mod_name}.{name} on {type(bad).__name__}: raw " f"{type(exc).__name__}: {exc}") + except Exception as exc: # noqa: BLE001 — see below, this is the point + # UNCLASSIFIED IS REPORTED, NOT SWALLOWED (2026-08-16). Until this branch existed, an + # exception that was neither _ACCEPTED nor _FORBIDDEN propagated straight out of this + # loop: the test ended as ERROR and every surface AFTER the offending one was never + # reached. The taxonomy's gap did not under-report, it STOPPED MEASURING — and the + # damage scaled with iteration position, not with severity. Measured when found: + # `emit.load_signer` sat at position 87 of 90, so three surfaces went untested; the + # same gap at position 1 would have cost 89. + # + # A third axis of the same instrument. The module axis (population) and the argument + # axis (only position 0) were already known; this is the exception-taxonomy axis. + escapes.append(f"{mod_name}.{name} on {type(bad).__name__}: UNCLASSIFIED " + f"{type(exc).__name__}: {exc} — neither accepted nor forbidden; " + f"decide which it is instead of letting it abort the sweep") self.assertEqual(escapes, [], "raw type-confusion escapes over the AUTO-DISCOVERED surface family:\n" + "\n".join(escapes)) From 2bd8d7cdbb278c081646202cdde0769007417277 Mon Sep 17 00:00:00 2001 From: kraxo Date: Sun, 16 Aug 2026 13:48:12 +0200 Subject: [PATCH 2/2] fix(never-raise): admit the measured case, not its family The counter-read rejected the previous commit on a point I had raised as my own doubt and then not acted on, which is the worse failure of the two. `_ACCEPTED` gained `OSError`. The commit text next to it said "widens the accepted set by a single measured case, not by a guess about what might appear". The mechanism did the opposite: `OSError` is the base class of `PermissionError`, `TimeoutError`, `BrokenPipeError` and more. I claimed narrow and implemented broad, in adjacent lines. Why it matters beyond tidiness: a `PermissionError` on an anchor file is not a missing file. It can be the trace of something blocking access, and inheriting a silent pass for it is fail-open on exactly the axis this property defends. The principle is minimal admission -- admit what was measured, and let the next case earn its own decision. Now `FileNotFoundError` only. Every other OSError subclass falls into the unclassified branch and is REPORTED, which is the whole point of that branch: the next one gets a decision instead of an inherited pass. Also measured, a doubt I had stated and not checked: what the coverage guard does if someone empties `_MODULES`. It imports the list from the module it guards, so an emptied list could in principle make it vacuously green. Measured by emptying it: the guard reports 90 surfaces across 40 modules as outside the population and goes RED. The guess was right; it is now a measurement. property 5 green, guard 3 green, full suite Ran 2033 tests OK skipped=10, ruff clean. Co-Authored-By: Claude Opus 5 (1M context) --- ...est_never_raise_surface_family_property.py | 25 +++++++++++++------ 1 file changed, 17 insertions(+), 8 deletions(-) diff --git a/tests/test_never_raise_surface_family_property.py b/tests/test_never_raise_surface_family_property.py index 49544da9..3ef2409c 100644 --- a/tests/test_never_raise_surface_family_property.py +++ b/tests/test_never_raise_surface_family_property.py @@ -49,14 +49,23 @@ # ACCEPTED terminations: a returned value, or a TYPED fail-closed error. ProofBundleError covers # BundleFormatError / BudgetExceeded / PQUnavailable / UnsupportedError / CanonicalizerUnavailable / PolicyError # / SdjwtVcError / EvalClaimError-as-PBError; ValueError covers EvalClaimError + the rfc8785 domain family. -# `OSError` added 2026-08-16, and the reason belongs here rather than in a commit nobody re-reads. -# The never-raise contract forbids a surface CRASHING INSTEAD OF DECIDING — the type-confusion -# signatures in `_FORBIDDEN`. A loader that reports "this path does not exist" is the opposite of that: -# it is fail-closed, it is informative, and it produces no verdict a relying party could mistake for a -# pass. Measured before adding it, across all 90 discovered surfaces and the full corpus: ZERO -# forbidden escapes and exactly ONE unclassified case (`emit.load_signer` on `b"bytes-not-str"`). -# So this widens the accepted set by a single measured case, not by a guess about what might appear. -_ACCEPTED = (ProofBundleError, ValueError, OSError) +# `FileNotFoundError` added 2026-08-16 — and the FIRST attempt added `OSError`, which was wrong in a +# way worth recording, because the mistake and the claim contradicted each other. The commit text said +# "widens by a single measured case, not by a guess"; the mechanism widened the whole hierarchy. +# `OSError` is the base class of `PermissionError`, `TimeoutError`, `BrokenPipeError` and more. A +# `PermissionError` on an anchor file is not a missing file — it can be an indicator that something +# blocked access, and swallowing it silently is fail-open on exactly the axis this property defends. +# The counter-read caught it (un, REJECT, 2026-08-16): admit the measured case, not its family. +# +# Why this ONE subclass is admissible: the contract forbids a surface CRASHING INSTEAD OF DECIDING — +# the type-confusion signatures in `_FORBIDDEN`. A loader reporting "this path does not exist" is the +# opposite: fail-closed, informative, and it produces no verdict a relying party could mistake for a +# pass. Measured across all 90 discovered surfaces and the full corpus: ZERO forbidden escapes and +# exactly ONE unclassified case (`emit.load_signer` on `b"bytes-not-str"` → `FileNotFoundError`). +# +# Any OTHER `OSError` subclass therefore still lands in the unclassified branch below and is REPORTED, +# which is the point: the next one gets a decision, not an inherited pass. +_ACCEPTED = (ProofBundleError, ValueError, FileNotFoundError) # FORBIDDEN raw terminations = the type-confusion crash signatures a public verify surface must never emit. _FORBIDDEN = (AttributeError, TypeError, RecursionError, KeyError, IndexError, UnicodeDecodeError, MemoryError)