Publish reuses a stale Sphinx environment: two lectures publish as chapter 57, and mccall_risk is missing from nearly every sidebar

[mmcky]

Summary

publish.yml unpacks the cache.yml artifact into _build whole — including _build/.doctrees, the Sphinx environment from the run that produced the artifact — and never clears it. That environment holds cross-document state: the resolved toctree and the section numbering derived from it. Reusing it makes every page render its navigation from the table of contents as it stood when the artifact was built, however current the sources are.

publish-2026aug24 (37d8e85, run 32688898836) shipped that. The artifact in use was from 2026-08-17, because the three cache runs since had failed, and mccall_risk entered _toc.yml on 2026-08-21 with #264 — so the restored environment had never seen the new lecture.

Reader-visible symptoms, verified live 2026-08-24

Two lectures publish under the same chapter number, because the numbering came from a toctree that does not contain mccall_risk:

Page	<title> as published	Expected from _toc.yml
mccall_risk.html	57. 工作搜寻 V：风险敏感型偏好	57
mccall_persist_trans.html	57. 工作搜寻 V：持续性与暂时性工资冲击	58
career.html	58. 工作搜寻 VI：职业选择建模	59

The offset continues for the rest of the corpus (jv 59, odu 60).

The new lecture is missing from other pages' navigation. In the release asset — the build's own output, so this is not a deploy or CDN artefact — the Search section of career.html's navigation reads mccall_model, mccall_model_with_separation, mccall_model_with_sep_markov, mccall_fitted_vfi, mccall_persist_trans, jv, odu. mccall_risk is absent, though _toc.yml at the published commit lists it exactly once between mccall_fitted_vfi and mccall_persist_trans. Same omission on mccall_persist_trans.html. A reader can reach the new lecture only from the landing page.

This is separate from #266 (the undelivered upstream renumberings), though the two compound on the same two pages: #266 gives both lectures the numeral V, this issue gives both the number 57. Fixing either leaves the other.

What is not wrong

The build did not skip pages, and the deploy is faithful. The run log's HTML phase shows writing output for 145 docnames, including career, mccall_persist_trans and mccall_risk, and the release tarball matches the live site byte-for-byte on the pages checked. _toc.yml at 37d8e85 is correct and there were no toctree warnings. The pages are fresh; only the global state they were rendered against is stale.

Fix

Clear the restored Sphinx environment after the download, keeping _build/.jupyter_cache — the expensive notebook execution and the only part the artifact exists to supply. ci.yml already does this in a step named "Clear stale Sphinx environment"; publish.yml had it only as a commented-out line beside the HTML build.

PR: #272.

Why nothing caught it

Every signal was green: the workflow succeeded, all 145 pages return HTTP 200, bin/check-tofu --site reports no missing glyphs, and no link 404s. The defect lives only in cross-page consistency, which nothing currently checks — a whole-site invariant (every page's own section number agrees with the number the navigation gives it; every page in the ToC appears in every page's navigation) would have caught it before the tag went out.

Remediation

A cache run was dispatched on main 2026-08-24 (32710038106) and gates a re-cut. Sequence: green cache artifact → merge #272 → re-tag → confirm career.html lists mccall_risk and no two lectures share a chapter number.

Refs: #271 (the cache.yml gate defects that let the artifact go stale), #266 (the numeral collision this compounds with), QuantEcon/project-translation#48.

[mmcky]

Correction to the mechanism — the body above has been rewritten, and the fix is up as #272.

As originally filed this issue said the publish build rewrote only three pages and deployed stale HTML for the rest. That was wrong, and I want the error on the record rather than silently edited away.

Checking the build log properly shows the HTML phase emitting writing output for 145 docnames, career and mccall_persist_trans among them, and the release tarball matches the live site on every page I compared. Every page was rebuilt from the tagged sources.

What is actually stale is the Sphinx environment, not the HTML. publish.yml restores _build/.doctrees along with the rest of the artifact, and that environment carries the resolved toctree and the section numbering derived from it. So the pages were regenerated — against a table of contents from 2026-08-17, three days before mccall_risk was added to _toc.yml in #264. Hence a fresh page that omits the new lecture from its navigation, and numbering that is one behind from mccall_persist_trans onward.

Every reader-visible symptom originally reported is real and reproduces; only the explanation changed. The fix is the same one either way — clear the restored environment — so #272 stands as written.

Worth keeping in mind for the delivery-integrity work: the first theory was consistent with all the page-level evidence and still wrong. Distinguishing "stale page" from "fresh page, stale global state" needed the build log and the release asset, not the live site.

[mmcky]

Re-cut and verified — publish-2026aug24b is clean on everything this issue covers.

Tag cut on 6a4b427 (both #272 and #273 merged), build run 32721582162, green in 8m13s — down from 68 minutes, because the cache artifact from the 2026-08-24 run was current so nothing re-executed.

Validated against the release tarball (the build's own output, so free of deploy and CDN doubt) and then confirmed live:

Check	Before	After
Lectures sharing a chapter number	mccall_risk and mccall_persist_trans both 57	none
Lectures sharing a Job Search numeral	both V	none — I…X each used once
Pages linking mccall_risk.html	1 of 158 (intro only)	147 of 158
Pages whose own number disagrees with the navigation	86	0
rel=canonical / og:url on the dead domain	296	0 — all on quantecon.github.io

Live spot-check confirms the series reads mccall_risk 57 · V, mccall_persist_trans 58 · VI, career 59 · VII, jv 60 · VIII, odu 61 · IX, mccall_q 64 · X, and career.html now carries a mccall_risk.html link with a self-referential canonical.

The last row is the one worth keeping: "every page's own section number agrees with the number the navigation gives it" is the whole-site invariant that would have caught this release before it shipped, and no existing check computes it. It is proposed for QuantEcon/project-translation#41.

One new defect surfaced by the re-cut, unrelated to this issue's mechanism: check-tofu --site now reports util_rand_resp losing 19 CJK glyphs. The lecture has no i18n font registration and its figures carry Chinese labels; the previous release rendered them from a 2026-08-17 execution that happened to resolve a CJK face anyway. Re-executing exposed it. Fix in #274. That also means this morning's clean check-tofu was luck rather than correctness.