3.8.0 scope: the deep-gate fixes, made reviewable

.github/workflows/release.yml

@@ -63,10 +63,49 @@ jobs:
           sha256sum dist/* | tee dist/SHA256SUMS
 
       - name: Generate SLSA build provenance for the artifacts
+        id: provenance
         uses: actions/attest-build-provenance@4d101475d8b20a2381f78447822ac1eab6504dd8  # v4.2.2
         with:
           subject-path: "dist/*.whl,dist/*.tar.gz"
 
+      - name: Also place the provenance next to the release assets
+        run: |
+          # WHY: OpenSSF Scorecard's Signed-Releases check reads the RELEASE ASSETS and looks for
+          # *.minisig / *.asc / *.sig / *.sign / *.sigstore / *.sigstore.json / *.intoto.jsonl
+          # (ossf/scorecard docs/checks.md, read 2026-08-07). Measured the same day: this repo scores
+          # 0/10 there while the assets of v3.7.0 are exactly the wheel, the sdist and SHA256SUMS.
+          # The provenance is NOT missing — it lives in GitHub's attestation store and on PyPI, i.e.
+          # somewhere the check never looks.
+          #
+          # NOT INTO dist/ — and that is the whole point of this directory existing. The first version
+          # of this step (d3401a7) copied the bundle into dist/, which is ALSO what feeds the PyPI
+          # upload via packages-dir. A deep gate on that very commit measured the consequence with
+          # twine 7.0.0: InvalidDistribution, before any network I/O — the GitHub Release would have
+          # been published and the PyPI leg would have failed, splitting the release in half.
+          # A directory that serves two consumers with different admissible contents is the coupling;
+          # a second directory removes it, whereas extending the removal list downstream would only
+          # have patched this one file.
+          #
+          # NOTHING IS RE-SIGNED HERE. The bundle produced by the step above is copied unchanged. If
+          # it is ever absent, this fails loudly rather than shipping a release whose provenance copy
+          # silently went missing.
+          #
+          # ON THE NAME: `.intoto.jsonl` describes the content — in-toto attestations, one JSON per
+          # line. It is not chosen because that pattern scores 10 while `.sigstore.json` scores 8;
+          # the check does not verify signatures at all, so a name could buy points on its own, and
+          # picking one for that reason is exactly the defect this project exists to make visible.
+          set -euo pipefail
+          src="${{ steps.provenance.outputs.bundle-path }}"
+          if [ ! -s "$src" ]; then
+            echo "provenance bundle missing or empty at '$src' — refusing to publish a release"
+            echo "whose provenance copy would be silently absent."
+            exit 1
+          fi
+          mkdir -p release-assets
+          cp "$src" "release-assets/proofbundle-${GITHUB_REF_NAME#v}-provenance.intoto.jsonl"
+          cp dist/SHA256SUMS release-assets/SHA256SUMS
+          echo "release-only assets:"; ls -l release-assets/
+
       - name: Upload the attested dist for the publish job (exact bytes, no rebuild)
         uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a  # v7.0.1
         with:
@@ -93,7 +132,8 @@ jobs:
           files: |
             dist/*.whl
             dist/*.tar.gz
-            dist/SHA256SUMS
+            release-assets/SHA256SUMS
+            release-assets/*-provenance.intoto.jsonl
           generate_release_notes: true
           prerelease: ${{ steps.relmeta.outputs.prerelease }}
           make_latest: ${{ steps.relmeta.outputs.make_latest }}
@@ -121,6 +161,31 @@ jobs:
         run: |
           set -euo pipefail
           rm -f dist/SHA256SUMS   # not a distributable; keep only the wheel + sdist for upload
+
+          # ALLOWLIST, NOT A REMOVAL LIST — and this is a class fix, not a tidy-up.
+          #
+          # The line above is a hand-maintained blocklist of "things that are not distributables".
+          # A deep gate measured what that costs (finding L6-01 on d3401a7): a new file was added to
+          # dist/, the removal list did not know about it, and twine 7.0.0 rejects the upload with
+          # InvalidDistribution — after the GitHub Release has already been published. The list did
+          # not fail; it simply did not mention the new file, and silence read as approval.
+          #
+          # A blocklist can only ever name what someone already thought of. This asserts the property
+          # instead: after the removals, dist/ contains EXACTLY the wheel and the sdist. Anything
+          # else — a new artefact, a stray log, a future non-distributable — stops the publish here,
+          # loudly, on the leg where it is still cheap.
+          unerwartet=$(find dist -maxdepth 1 -type f ! -name '*.whl' ! -name '*.tar.gz' -printf '%f\n')
+          if [ -n "$unerwartet" ]; then
+            echo "dist/ carries files that are not distributables:"
+            echo "$unerwartet" | sed 's/^/  - /'
+            echo "Publishing would hand these to twine via packages-dir and fail AFTER the GitHub"
+            echo "Release is out. Put release-only assets in release-assets/, or extend this gate"
+            echo "deliberately if they really belong in the upload."
+            exit 1
+          fi
+          test "$(find dist -maxdepth 1 -name '*.whl' | wc -l)" = 1
+          test "$(find dist -maxdepth 1 -name '*.tar.gz' | wc -l)" = 1
+
           got_wheel=$(sha256sum dist/*.whl | cut -d' ' -f1)
           got_sdist=$(sha256sum dist/*.tar.gz | cut -d' ' -f1)
           echo "attested wheel:   ${{ needs.build-and-attest.outputs.digest-wheel }}"

.github/workflows/scorecard.yml

@@ -2,8 +2,12 @@ name: Scorecard
 
 # OpenSSF Scorecard — supply-chain posture (pinned deps, token permissions, branch protection,
 # dangerous workflows, SAST, fuzzing, signed releases). Publishes results so the badge works.
-# The high-leverage checks this repo already maxes: Pinned-Dependencies (all actions SHA-pinned),
-# Token-Permissions (read-all top-level, per-job escalation), SAST (CodeQL), Fuzzing (Hypothesis).
+# Measured 2026-08-07 (v5.5.0, overall 6.5/10), not assumed: Token-Permissions, SAST and Fuzzing do
+# score 10/10. Pinned-Dependencies does NOT — it scores 3/10, while an earlier version of this
+# comment claimed the check was maxed because all actions are SHA-pinned. Actions are only one input;
+# the check also weighs pip/Dockerfile pinning. A comment that states a score nobody re-measured is
+# the same defect class this project reports about version numbers, so it now carries its date.
+# Current state per check: docs/openssf_best_practices_self_assessment.md.
 
 on:
   branch_protection_rule:

CHANGELOG.md

@@ -6,6 +6,96 @@ to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
 _Editorial 2026-07-20: internal gate codename replaced by its external name throughout; content unchanged._
 
+## [Unreleased]
+
+**Semantics: unchanged. Resource ceilings: one deliberate tightening, disclosed below.**
+
+This banner said "Nothing under `src/` changes" until 2026-08-08. That was **false** by then. Measured
+at `bc3ae70` with `git diff --numstat origin/main HEAD -- src/proofbundle/`: **8 files, 196
+insertions, 3 deletions**. The sentence was written when it was true and was not pulled when the tree
+moved past it — a statement nobody re-measured.
+
+The first correction of this banner then repeated the fault it describes. It claimed "204 lines", a
+figure already 9 off when it was written and 17 further off after the revert below landed. A count
+against a moving branch is only true at a named ref, so this one names its ref and its command.
+Found by the mandatory review lane, both times, and not by a check — `scripts/check_version_and_changelog.py`
+reads only headings, and the release-scope checkbox in [RELEASE.md](RELEASE.md) is read by a human,
+not by a gate.
+
+What actually changed, and why each is patch-safe:
+
+* **No public interface gains or loses a field**, and no verdict flips from fail to pass. Every change
+  below is fail-**closed**: input that was accepted and is over a generous ceiling is now refused
+  before the work it would cost.
+* **Structural budget on the direct-dict path.** Six public surfaces that accept an already-parsed
+  structure now apply the same `VerificationBudget` ceilings the string/file path has always applied
+  (`string_len` 1 000 000, `json_nodes` 200 000). On that path the `input_bytes` cap is inert — there
+  are no bytes to measure — so those surfaces were unbounded. This is the same deliberate exception
+  the project shipped in 3.2.3 (Finding 15b) and is disclosed here for the same reason:
+  [COMPATIBILITY.md](COMPATIBILITY.md) requires that a tightening of a previously accepted input say
+  so explicitly. Each surface reports it in **its own** documented failure form — a result dict where
+  the surface returns dicts, `BundleFormatError` where it raises — so no new exception type appears
+  anywhere.
+* **Withdrawn before release: moving the `merkle_path` cap earlier.** An earlier commit in this
+  cycle moved the `merkle_path` (256) check in `verify_sample_opening` ahead of the base64 decode.
+  It was reverted, and this entry records why rather than dropping it silently. The claim it
+  originally carried here — that the outcome is unchanged for every input — was **measured false**.
+  Method, since the repo asks every number to name its object and its source: two worktrees at the
+  commit and its parent, the same `verify-opening` invocation against each, exit codes compared per
+  input class. The CLI exit code moved from 2 to 1 whenever any proof element, or `root_b64`, would
+  have been rejected by `b64decode(validate=True)`. Two independent partitions were counted — one
+  gave at least 12 diverging classes, an independent re-count gave 22; "input class" is not a defined
+  unit here, so the lower bound is what the claim rests on. The verdict itself never flipped (`ok`
+  stays `False`), but [COMPATIBILITY.md](COMPATIBILITY.md) lists the meaning of exit codes as a
+  public surface, and this project already kept `stash@{0}` out of 3.7.1 for the same reason. The
+  change also did not achieve what it was for: an `Omega(n)` structural budget walk runs one line
+  above the cap, so the cap cannot precede the work it bounds. Peak memory at n=190000 was 11867 KiB
+  against 2.2 KiB at n=257 — a linear path, not a flat one. Wall-clock figures for the same runs are
+  deliberately not quoted: they were host-dependent and differed by 28% between two measurements of
+  the same code. The underlying finding stays open for a minor release.
+* **Typed errors on two path arguments.** `evaluation_card_hash` and `prereg_hash` raise
+  `BundleFormatError` on a non-path argument instead of leaking `OverflowError` / `TypeError` /
+  `FileNotFoundError`. The CLI always passes a `str`, no test or doc pinned the old types, and the
+  surrounding failure form in both functions was already `BundleFormatError`.
+
+The planned scope for the next patch is written down in
+[docs/release_scope/3.7.1.md](docs/release_scope/3.7.1.md).
+
+### Fixed
+
+- The post-tag drift check anchored on `git describe --tags`, which returns *whatever was tagged
+  last*. Measured on 2026-08-07 it returned a corpus review tag; `_semver_tuple` reads that as
+  `(0, 0, 0)`, so any real version compares as "bumped past it" and the check stopped applying. It
+  did not fail — it went silent, and silence looked like agreement. Under that blind spot one
+  non-trivial commit sat undelivered since `v3.7.0` with no `## [Unreleased]` section (this one).
+  The check now anchors on the last **release** tag and distinguishes three states: a release tag,
+  no tags at all, or tags that exist but none of them is a release.
+- `pyproject.toml` pins the ruff **rule set**, not just its version, and raises the cap to `<0.17`
+  (#134). Measured on the identical tree: ruff 0.15.x applies 59 default rules and exits 0 over all
+  258 tracked `.py` files, ruff 0.16.x applies 413 and reports 1168 findings. The cap alone would
+  have silently stopped checking the 18 rules 0.16 removed. `mypy` is bounded at `<3` for the same
+  reason, deliberately and without a measured failure.
+
+### Added
+
+- `scripts/check_version_and_changelog.py` also compares the two prose places that state the current
+  version (`RELEASE.md`, `docs/readiness_pack/PROGRESS.md`), and optionally PyPI and the project page
+  (`--external`). External surfaces have three states: agreement, disagreement, and NICHT MESSBAR —
+  unreachable never counts as green, and `--require-external` turns it into a failure for the release
+  checklist. Historical statements (`since vX`, `as of vX`, old changelog headings) are deliberately
+  out of scope: bumping them would turn a fact into a false claim.
+- A release gate in `RELEASE.md`: a checkable list a release answers *before* the Owner-GO is asked
+  for. No date, no cadence — what is slowed down is vagueness, not speed.
+
+### Changed
+
+- `docs/IN_TOTO_PROFILE.md` and `docs/upstream/eval-result.md` now say what was actually submitted as
+  in-toto/attestation#575. Both still listed `anchors[]` as a predicate field, which that PR never
+  had, and neither carried the absence rule, the optional harness `DigestSet`, the non-claim on
+  harness/grader fitness, `passed` as a **signed threshold verdict**, or `assuranceLevel` as
+  **issuer-declared**. The upstream copy now states that the PR is the source of truth when the two
+  differ.
+
 ## [3.7.0] - 2026-07-23 (adapter sample-count provenance, BETA, relation EXPERIMENTAL)
 
 Status boundary (No-Overclaim): 3.7.0 remains audit-candidate BETA, relation/v0.1 EXPERIMENTAL. This is a

COMPATIBILITY.md

@@ -0,0 +1,89 @@
+# Compatibility and deprecation
+
+What SemVer means here concretely, what counts as a breaking change, how long a deprecated thing
+stays, and which parts are exempt because they are labelled EXPERIMENTAL.
+
+**Nothing on this page is a new promise.** README already states the project is SemVer-committed;
+this file writes down what that sentence already implies, so that a reader does not have to infer it
+and a maintainer cannot quietly narrow it.
+
+## What counts as the public interface
+
+Compatibility statements are worthless without naming the surface they cover. For proofbundle it is:
+
+1. **The Python API** — names importable from `proofbundle` and its documented submodules, their
+   parameters, and the **shape of what they return**.
+2. **The CLI** — subcommands, flags, and the meaning of exit codes.
+3. **The emitted bytes** — the receipt/bundle formats, the in-toto Statement and the DSSE envelope.
+   A signed structure is an interface even when no function signature changes.
+4. **The verification verdict** — whether a given input verifies. A change that makes previously
+   valid input invalid is breaking even if every signature stayed the same.
+
+Point 3 is the one that is easy to miss, and it is the reason `stash@{0}` did not ride along in
+3.7.1: adding one key to a per-edge result entry changes a structure that ends up signed. Measured,
+recorded in [`docs/release_scope/3.7.1.md`](docs/release_scope/3.7.1.md), and kept out of a PATCH.
+
+## What is a breaking change
+
+- Removing or renaming anything in the four surfaces above.
+- Changing the **type** or **meaning** of a field, including a field that is only ever read.
+- **Adding** a field to an emitted, signed structure — the bytes change even though nothing was
+  taken away.
+- Making a previously accepted input fail verification.
+- Turning an optional obligation into a required one.
+
+**Not breaking:** new optional CLI flags, new functions, additional keys in a structure that is
+neither signed nor part of a documented return shape, documentation, tests, build tooling, and
+performance. Tightening a check that only ever accepted input the spec already called invalid is a
+fix, not a break — and the CHANGELOG entry has to say so explicitly, because from the outside a
+stricter check and a break look identical.
+
+## What each version step allows
+
+| Step | Allowed |
+|---|---|
+| PATCH (`x.y.Z`) | fixes only. **No semantic change, no new obligation, no changed behaviour at a public interface.** |
+| MINOR (`x.Y.0`) | additive changes, new optional fields, new commands, deprecation *announcements* |
+| MAJOR (`X.0.0`) | removals, renames, meaning changes — that is, everything above |
+
+That PATCH row is not decoration: it is the question the release gate in [RELEASE.md](RELEASE.md)
+makes someone answer per line of the scope list, in writing, before a release is asked for.
+
+## How long a deprecated thing stays
+
+**A deprecated stable element is removed no earlier than the next MAJOR.** That is not an extra
+guarantee — it is what SemVer already means, written down so nobody has to derive it. In practice:
+
+1. The deprecation is announced in the CHANGELOG of the MINOR that announces it, and the element
+   keeps working unchanged.
+2. It stays through every following MINOR and PATCH of that major line.
+3. It may be removed in the next MAJOR, and that removal is named in the CHANGELOG.
+
+**No calendar.** No "six months", no "two releases" — a period nobody schedules is a promise that
+breaks itself. The bound is the next MAJOR, and MAJORs happen when they happen.
+
+**Honest limit:** as of this writing no stable element has gone through the full cycle, so this
+describes a rule, not a track record. It is written down precisely because a rule that only exists
+in someone's head is not one.
+
+## What is EXPERIMENTAL, and what that costs you
+
+EXPERIMENTAL parts are **excluded from all of the above**. They may change or disappear in any
+release, including a PATCH, without a deprecation period. That is the whole point of the label: it
+buys the freedom to get a design wrong in public.
+
+Currently labelled EXPERIMENTAL (see CHANGELOG and README for the authoritative statement per
+release):
+
+- **`relation/v0.1`** — the relation/lineage surface
+- **the `[experimental]` extra** — the TEE-attestation bridge, see
+  [docs/EXPERIMENTAL_ENCLAVE.md](docs/EXPERIMENTAL_ENCLAVE.md)
+
+The `eval-result` predicate is a further case and is labelled separately: its `predicateType` sits
+in a **vendor namespace** until in-toto registers the type, and the migration path (registered URI
+plus alias) is written down in [docs/IN_TOTO_PROFILE.md](docs/IN_TOTO_PROFILE.md). Consumers match
+on the subject digest, so that rename does not affect binding.
+
+The current release line as a whole carries a status boundary of its own (audit-candidate **BETA**),
+stated per release in the CHANGELOG. A BETA line still follows the table above; the label says how
+much external assurance exists, not how freely the interface may move.