Skip to content

Verify evidence digests against committed bytes, not the working tree - #1

Merged
0j0bit merged 1 commit into
mainfrom
fix/digest-gate-committed-bytes
Sep 13, 2026
Merged

0j0bit merged 1 commit into
mainfrom
fix/digest-gate-committed-bytes

Conversation

@mcl-release-governor

Copy link
Copy Markdown

Verify evidence digests against committed bytes, not the working tree

This gate reported PASSED on Windows while two artifacts under mcl-ap would
have failed for anyone else. The first release-gates run on a Linux runner,
minutes after the repositories became public, caught both immediately.

The gate was not wrong about the bytes it saw. It was looking at the wrong
bytes. .gitattributes says * text=auto eol=lf, and text=auto makes Git
compare NORMALISED content, so a file left over from before that rule can sit
in a Windows working tree with CRLF, hash to the CRLF value, and still be
reported by git status as perfectly clean. A digest recorded from that tree
then agrees with itself forever on that one machine.

That is the whole failure mode this gate exists to prevent, reappearing one
level up: nothing was checking that the bytes being checked were the published
ones. An external reviewer clones fresh and gets the committed bytes on every
platform, and that is the only thing a published digest can mean.

So each covered file's working-tree bytes are now compared against its
committed blob before any digest is verified, and a divergence is its own
distinct failure with its own instruction -- refresh the checkout, and do NOT
regenerate the digest from those bytes, because doing so is how a wrong digest
becomes permanent.

Three things this cost, all worth recording:

  • Paths must be resolved before Git is asked about them. A digest file here
    references ../../exp002_probe.wav, and git rev-parse HEAD: does not
    normalise .. the way ls-files does. Comparing the two answers without
    resolving first reported a divergence that did not exist.

  • The command substitutions need || true. Under set -e a failing
    substitution inside an assignment terminates the gate silently, which it
    did on the first run -- printing its header and nothing else.

  • Verified in both directions, not just observed to pass: a clean tree exits
    0 with no failures, a file given CRLF in the working tree only is caught by
    name and exits 1, and restoring it returns the gate to green.

Co-Authored-By: Claude Opus 5 noreply@anthropic.com

This gate reported PASSED on Windows while two artifacts under mcl-ap would
have failed for anyone else. The first release-gates run on a Linux runner,
minutes after the repositories became public, caught both immediately.

The gate was not wrong about the bytes it saw. It was looking at the wrong
bytes. `.gitattributes` says `* text=auto eol=lf`, and `text=auto` makes Git
compare NORMALISED content, so a file left over from before that rule can sit
in a Windows working tree with CRLF, hash to the CRLF value, and still be
reported by `git status` as perfectly clean. A digest recorded from that tree
then agrees with itself forever on that one machine.

That is the whole failure mode this gate exists to prevent, reappearing one
level up: nothing was checking that the bytes being checked were the published
ones. An external reviewer clones fresh and gets the committed bytes on every
platform, and that is the only thing a published digest can mean.

So each covered file's working-tree bytes are now compared against its
committed blob before any digest is verified, and a divergence is its own
distinct failure with its own instruction -- refresh the checkout, and do NOT
regenerate the digest from those bytes, because doing so is how a wrong digest
becomes permanent.

Three things this cost, all worth recording:

  - Paths must be resolved before Git is asked about them. A digest file here
    references ../../exp002_probe.wav, and `git rev-parse HEAD:` does not
    normalise `..` the way `ls-files` does. Comparing the two answers without
    resolving first reported a divergence that did not exist.

  - The command substitutions need `|| true`. Under `set -e` a failing
    substitution inside an assignment terminates the gate silently, which it
    did on the first run -- printing its header and nothing else.

  - Verified in both directions, not just observed to pass: a clean tree exits
    0 with no failures, a file given CRLF in the working tree only is caught by
    name and exits 1, and restoring it returns the gate to green.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

@0j0bit 0j0bit left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed as 0j0bit: only tools/check-evidence-digests.sh changes; it now requires working-tree bytes to equal the committed blob before verifying a digest, resolves .. paths, and guards substitutions under set -e; verified in both directions; build (gcc) and build (clang) green.

@0j0bit
0j0bit merged commit 66c838b into main Sep 13, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant