test: fail when docs/ serves a stale notebook - #58
Merged
Conversation
Notebooks are `resources:` in _quarto.yml, not `render:` targets, so Quarto copies them into docs/ verbatim. That put them in the blind spot between both existing gates: the regenerate step compares the tracked notebooks against the normalizer and never looks in docs/, and compare_render.py walks *.html only. A notebook committed without a re-render therefore left Pages serving the old copy with every check green. It happened twice, once to nine of the twelve notebooks at a stroke. Check 2 compares each committed notebook against the copy docs/ serves, byte for byte — exact is the right comparison here precisely because Quarto does not transform these files, unlike the HTML gate, which cannot be exact. It also catches a rename that leaves an orphan behind in docs/notebooks/. CI already runs the checker against docs/ as committed before re-rendering, which is the copy Pages is serving, so the gate lands there for free. --notebooks-only still runs the two notebook checks alone: this one reads docs/, so it belongs with the site checks. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0125vs3ARnz2pLp8D5vQ6iKJ
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #57, which is where this came up:
docs/notebooks/had gone stale againstnotebooks/again, and nothing in CI said so.The gap
notebooks/*.ipynbareresources:in_quarto.yml, notrender:targets, so Quarto copies them intodocs/verbatim. That puts them in the blind spot between both existing gates:gen_notebooks.pyand fails only if the trackednotebooks/drift from the normalizer's output — it never looks indocs/;compare_render.pywalks*.htmland skips everything else.So a notebook committed without a re-render leaves Pages serving the old copy with every check green. It has now happened twice, once to nine of the twelve notebooks at a stroke.
The check
New check 2,
docs/notebooks matches notebooks/: each committed notebook compared byte-for-byte against the copydocs/serves, plus orphans indocs/notebooks/that a rename left behind.Byte-exact is the right comparison here for the reason the HTML gate cannot be: Quarto copies these files rather than transforming them, so there is no SCSS-style platform difference to normalize away.
The workflow already runs
check_links.pyagainstdocs/as committed, before it re-renders — the copy Pages is serving — so the gate lands in the right place for free.--notebooks-onlyis unchanged and still numbers its two checks 1 and 2; this one readsdocs/, so it belongs with the site checks. Kahoot's TODO-only check moves from 9 to 10;CLAUDE.mdis updated for both that and the count.Verified
main:14 notebooks served from docs/ are byte-identical to the committed notebooks/.notebooks/02-thinking-in-n-dimensions.ipynbwithout re-rendering:check_links.pyandcheck_links.py --notebooks-onlypass, with the numbering right in each mode.🤖 Generated with Claude Code
https://claude.ai/code/session_0125vs3ARnz2pLp8D5vQ6iKJ