Skip to content

test: fail when docs/ serves a stale notebook - #58

Merged
project-delphi merged 1 commit into
mainfrom
fix/docs-notebooks-staleness-check
Sep 3, 2026
Merged

test: fail when docs/ serves a stale notebook#58
project-delphi merged 1 commit into
mainfrom
fix/docs-notebooks-staleness-check

Conversation

@project-delphi

Copy link
Copy Markdown
Owner

Follow-up to #57, which is where this came up: docs/notebooks/ had gone stale against notebooks/ again, and nothing in CI said so.

The gap

notebooks/*.ipynb are resources: in _quarto.yml, not render: targets, so Quarto copies them into docs/ verbatim. That puts them in the blind spot between both existing gates:

  • the regenerate step reruns gen_notebooks.py and fails only if the tracked notebooks/ drift from the normalizer's output — it never looks in docs/;
  • compare_render.py walks *.html and skips everything else.

So a notebook committed without a re-render leaves Pages serving the old copy with every check green. It has now happened twice, once to nine of the twelve notebooks at a stroke.

The check

New check 2, docs/notebooks matches notebooks/: each committed notebook compared byte-for-byte against the copy docs/ serves, plus orphans in docs/notebooks/ that a rename left behind.

Byte-exact is the right comparison here for the reason the HTML gate cannot be: Quarto copies these files rather than transforming them, so there is no SCSS-style platform difference to normalize away.

The workflow already runs check_links.py against docs/ as committed, before it re-renders — the copy Pages is serving — so the gate lands in the right place for free.

--notebooks-only is unchanged and still numbers its two checks 1 and 2; this one reads docs/, so it belongs with the site checks. Kahoot's TODO-only check moves from 9 to 10; CLAUDE.md is updated for both that and the count.

Verified

  • Passes on current main: 14 notebooks served from docs/ are byte-identical to the committed notebooks/.
  • Fails as intended when they diverge — edited one markdown cell in notebooks/02-thinking-in-n-dimensions.ipynb without re-rendering:
    [2] docs/notebooks matches notebooks/
      FAIL  docs/ serves an old copy of 1 notebook(s): 02-thinking-in-n-dimensions.ipynb — run `quarto render` and commit docs/
    
    Exit code 1. The edit was reverted; it is not in this branch.
  • Both check_links.py and check_links.py --notebooks-only pass, with the numbering right in each mode.

🤖 Generated with Claude Code

https://claude.ai/code/session_0125vs3ARnz2pLp8D5vQ6iKJ

Notebooks are `resources:` in _quarto.yml, not `render:` targets, so Quarto
copies them into docs/ verbatim. That put them in the blind spot between both
existing gates: the regenerate step compares the tracked notebooks against the
normalizer and never looks in docs/, and compare_render.py walks *.html only.
A notebook committed without a re-render therefore left Pages serving the old
copy with every check green. It happened twice, once to nine of the twelve
notebooks at a stroke.

Check 2 compares each committed notebook against the copy docs/ serves, byte
for byte — exact is the right comparison here precisely because Quarto does
not transform these files, unlike the HTML gate, which cannot be exact. It
also catches a rename that leaves an orphan behind in docs/notebooks/.

CI already runs the checker against docs/ as committed before re-rendering,
which is the copy Pages is serving, so the gate lands there for free.

--notebooks-only still runs the two notebook checks alone: this one reads
docs/, so it belongs with the site checks.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0125vs3ARnz2pLp8D5vQ6iKJ
@project-delphi
project-delphi merged commit c1a9b91 into main Sep 3, 2026
1 check passed
@project-delphi
project-delphi deleted the fix/docs-notebooks-staleness-check branch September 3, 2026 17:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant