Skip to content

Run the redraw gate in CI, and take the chart profile off host.json - #79

Merged
tamnd merged 3 commits into
mainfrom
chart-redraw-gate
Sep 9, 2026
Merged

tamnd merged 3 commits into
mainfrom
chart-redraw-gate

Conversation

@tamnd

@tamnd tamnd commented Sep 9, 2026

Copy link
Copy Markdown
Owner

Closes #75, though not the way that issue describes it. The first half of it is already done and has been for a while: chart --manifest writes graphs.sha256, and both committed manifests verify. What was missing is the half the issue itself says matters, which is that nothing runs the check.

CI now redraws every chart in every published results directory from the committed output.json and compares SHA-256 against the committed manifest. That is M8's exit gate as a job rather than as a sentence, and it is the only check here that can see a change to the chart code quietly moving a pixel. It goes in the generated job, beside docs --check, because it is the same kind of claim: the file in the tree is the file the generator would write.

The issue worried that byte for byte reproducibility across platforms might not be free. It is, so far. Both directories were drawn on macOS and both verify on this machine, and the CI runner is Linux, so the first run of this job is also the first cross platform check. If it fails, that is worth knowing now rather than on the day somebody disputes a bar.

Getting there found a footgun. chart --dir X --check X/graphs.sha256 with no --profile draws 146 unstamped charts and reports that all 146 of them do not match, because the stamp along the bottom is part of the picture. The output says the charts are not reproducible when what happened is that an argument was left off, which is the worst possible thing for a determinism check to say. A results directory names its own profile in host.json, so that is where it comes from now when --profile is absent, and the CI job needs no per directory argument. --profile still wins where it is given, and the golden series, which has no directory, still draws unstamped.

A test asserts that every published directory is named for the profile its host.json records. Those two disagreeing would put one name on the directory and another on the charts inside it, and the manifest would stop verifying for anybody who passed the name they could see.

@tamnd tamnd added chart The chart engine and the 154 charts ci Workflows, toolchain and release determinism Byte identical output, golden data, hash manifests labels Sep 9, 2026
@tamnd
tamnd merged commit 850f6d6 into main Sep 9, 2026
14 checks passed
@tamnd
tamnd deleted the chart-redraw-gate branch September 9, 2026 22:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

chart The chart engine and the 154 charts ci Workflows, toolchain and release determinism Byte identical output, golden data, hash manifests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

graphs.sha256 is hand made, checked by nothing, and rots when charts are redrawn

1 participant