Read life-expectancy and usa-gini from data-lectures - #61
Merged
Conversation
Wave A4 of the datasets migration: the last two Track A CSVs moved to QuantEcon/data-lectures in QuantEcon/data-lectures#74, which is on `main` and serving both files. Three URLs. All three land on `raw.githubusercontent.com` — one spelling for reads and downloads alike, matching the decision taken during the high_dim_data fold. It drops a redirect hop and avoids the two-hosts-in-one-file split that the "harmonise these forms" class of fix has broken before. Note this differs from the older `github.com/.../raw/` form used by earlier repoint sets in this repo; harmonising those is a separate sweep, not this PR. Left alone on purpose: simple_linear_regression.md `cols = [... 'Life expectancy at birth (historical)' ...]`. The migrated bytes are byte-identical to intro's, so the old column label is still correct. Upstream OWID has since renamed four of eight columns and halved the row count; that delta is recorded in the new manifest and registered on QuantEcon/data-lectures#39. Adopting it is a content change, not a repoint. inequality.md the `[This notebook]` link to intro's `_static/.../data.ipynb`. It points at a notebook, not a dataset, and that notebook stays in lecture-python-intro. Verified rather than assumed. Line numbers re-derived immediately before editing. Both targets serve 200 with `access-control-allow-origin: *` and sha256 matching the manifests (ec5d3235… / bed9074a…). Old and new URLs produce IDENTICAL frames under the lecture's own code: both read_csv calls in simple_linear_regression (10x8 and 62156x4), `.dropna()` at 12,445 rows — which is what the prose asserts — the `Year == 2018` filter at 166 rows that drives the fitted scatter, and inequality's 20x3 with an identical index. So this cannot change a figure. Also deletes this repo's two committed copies. They are mirror-orphans: nothing here or anywhere else reads them — verified before deleting — and both are byte-identical to what data-lectures now serves, so the bytes are not lost. lecture-wasm deploys on push to main rather than on a publish tag, so it needs no two-phase split. CI here cannot validate any of this: ci.yml runs `myst build --html` with no execution, so a dead URL is invisible to it. The equivalence above was measured directly instead.
❌ Deploy Preview for beautiful-dodol-cb9543 failed.
|
There was a problem hiding this comment.
Pull request overview
Updates this repo’s lecture notebooks to read the migrated life-expectancy and usa-gini datasets from QuantEcon/data-lectures (via raw.githubusercontent.com), and removes an unused in-repo CSV mirror now served upstream.
Changes:
- Repoint
simple_linear_regressionto fetchlife-expectancy-vs-gdp-per-capita.csvfromQuantEcon/data-lectures(including the{download}link). - Repoint
inequalityto fetchusa-gini-nwealth-tincome-lincome.csvfromQuantEcon/data-lectures. - Delete the now-redundant committed copy of
usa-gini-nwealth-tincome-lincome.csvfrom this repo.
Reviewed changes
Copilot reviewed 3 out of 4 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| lectures/simple_linear_regression.md | Updates both the download link and data_url to the data-lectures raw URL for the life expectancy vs GDP dataset. |
| lectures/inequality.md | Updates data_url to the data-lectures raw URL for the USA gini/wealth/income dataset. |
| lectures/_static/lecture_specific/inequality/usa-gini-nwealth-tincome-lincome.csv | Removes an unused local mirror of a dataset now served from data-lectures. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
|
📖 Netlify Preview Ready! Preview URL: https://pr-61--beautiful-dodol-cb9543.netlify.app (c0520d0) ✨ Browse the preview at the URL above. |
mmcky
added a commit
to QuantEcon/data-lectures
that referenced
this pull request
Aug 12, 2026
All four consumers now read
raw.githubusercontent.com/QuantEcon/data-lectures/main/lectures/, so the two A4
records move landed -> repointed and both manifests gain their consumers.
THIS PR IS THE WAVE'S ACCEPTANCE TEST, not bookkeeping after it. Run locally in
both directions against the four now-merged consumer repos, before pushing:
status: landed -> exit 1, 2 warnings ("marked landed but some consumer
already reads data-lectures")
status: repointed -> exit 0, 0 warnings
So the red window was real, opened when the first consuming PR merged, and this
closes it. A non-zero strict audit also skips the Pages deploy, and Pages serves
lectures/ as well as the dashboard, so the served data tree would have gone
stale alongside it.
Repoints recorded: QuantEcon/lecture-python-intro#834, QuantEcon/lecture-wasm#61,
QuantEcon/lecture-intro.zh-cn#294, QuantEcon/test-actions-lecture-intro#56.
All four consumers are recorded per manifest, not the two the scan can see —
the convention set by the P3 fold. lecture-intro.zh-cn and
test-actions-lecture-intro are outside SCAN_REPOS, so migration.yml and the
manifests are the only place they exist. Safe because build_audit derives its
own consumer list from the scan and never asserts on the manifest's; the
audit's own count for these two files stays 2.
The generated lecture-python-intro.notebooks mirror is a FIFTH reference-holder
and is deliberately not listed: nobody repoints it, and it self-heals after
intro's next publish tag. Recorded as a comment in both manifests so the
omission reads as a decision rather than an oversight.
Post-flip figures: migrated 24 -> 26, committed_files 43 -> 41 (lecture-wasm
deleted its two mirror-orphans in its repoint PR), orphans unchanged at 26,
CATALOG.md "26 read by lectures today" with no awaiting-repoint remainder.
consumed-file-check: 26 manifests, 27 files hash-checked, 0 errors. Catalog
freshness gate green.
WHAT THIS DOES NOT DO: nothing is deleted. lecture-python-intro still holds the
only live copy of both files, and deleting them is gated on FOUR publishes —
intro, lecture-intro.zh-cn and the canary are all tag-gated, and merging a
repoint changes main, not a published site. lecture-wasm self-heals on merge.
Track A's third item, graph.txt, is untouched and is not a data-lectures
migration at all.
This was referenced Aug 12, 2026
mmcky
added a commit
to QuantEcon/data-lectures
that referenced
this pull request
Aug 12, 2026
Two record defects found by the independent validation, neither caught by the
strict audit — it reconciles manifests, migration.yml and the scan, and cannot
see a dead annotation key or a wrong number in prose.
**Two dead orphan keys.** `scripts/audit_annotations.yml` still carried
`lecture-wasm:lectures/_static/lecture_specific/{inequality/usa-gini…,
simple_linear_regression/life-expectancy…}` as `mirror-orphan`, with notes
saying "wasm's <lecture> reads intro's copy by URL". Both files were deleted in
QuantEcon/lecture-wasm#61 and wasm now reads data-lectures, so the keys name
nothing and the notes describe a state that ended. Same class as the
`lecture-wasm:lectures/graph.txt` key removed in #77 — these two were missed
there because that PR was scoped to graph.txt.
**The "8 repos" figure was wrong and no counting rule produces it.** PLAN.md's
graph.txt deletion caveat said the blob sits in 8 repos. Re-derived by the
validation: **7** hold it byte-identically (4,691 B, sha256 046759fb…), and two
more — QuantEcon.jl and QuantEcon.lectures.code — hold a 4,692-byte variant
differing by a single trailing space, giving 9 under a looser rule. Corrected to
7 with the variant named, so the next person sweeping does not have to
rediscover why the count will not reconcile.
The figure mattered because that sentence is the caveat gating a deletion.
Strict audit exit 0 with zero warnings; catalog gate green.
Both defects, and the reasoning behind the numbers, are recorded on
QuantEcon/workspace-lectures#37.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Wave A4 of the datasets migration. The last two Track A CSVs moved to
QuantEcon/data-lecturesin QuantEcon/data-lectures#74, which is merged and serving both files. Three URLs.All three land on
raw.githubusercontent.com— one spelling for reads and{download}roles alike, matching the decision taken during thehigh_dim_datafold. It drops a redirect hop and avoids the two-hosts-in-one-file split that the "harmonise these forms" class of fix has broken before. This does differ from the oldergithub.com/…/raw/form used by earlier repoint sets; harmonising those is a separate sweep, deliberately not this PR.Left alone on purpose
simple_linear_regression.md—cols = ['Code', 'Year', 'Life expectancy at birth (historical)', 'GDP per capita']. The migrated bytes are byte-identical to intro's, so the old column label is still the correct one. Upstream OWID has since renamed four of eight columns, moved the annotations column, and halved the row count; that delta is recorded in the new manifest and registered on QuantEcon/data-lectures#39. Adopting it would change every life-expectancy value while leaving the 2018 scatter looking identical — a content change with re-reviewed figures, not a repoint.inequality.md— the[This notebook]link to intro's_static/lecture_specific/inequality/data.ipynb. It points at a notebook, not a dataset, and that notebook stays inlecture-python-intro.Verification
Line numbers re-derived immediately before editing, not carried from a table. Both targets serve
HTTP 200withaccess-control-allow-origin: *, and their sha256 match the manifests exactly (ec5d3235…,bed9074a…).Old and new URLs produce identical frames under the lecture's own code — stronger than checking the URL returns 200:
read_csv(data_url, nrows=10)equals: Trueread_csv(data_url, usecols=cols)equals: True.dropna()Year == 2018(the fitted scatter)equals: Trueread_csv(data_url, index_col='year')So this provably cannot change a figure.
Also checked: the commit carries both the added and removed URL lines in each file (a scripted repoint has silently shipped deletions-without-URL-change before), and no reference to intro's copies of either file remains.
Also deletes this repo's two committed copies
They are mirror-orphans — nothing here or anywhere else reads them, verified before deleting, because this repo's own lectures fetch intro's copies by URL. Both are byte-identical to what data-lectures now serves, so no bytes are lost.
scripts/audit_annotations.ymlin data-lectures already classifies both asmirror-orphan.No two-phase split is needed here: this repo deploys via
ci.ymlon push tomain, not on apublish*tag, so it self-heals on merge.Rule 5, and why green CI here means very little
Both new reads use
raw.githubusercontent.com, which is the only form this repo's runtime can fetch — thegithub.com/…/raw/form is a 302 whose response carries an emptyaccess-control-allow-origin, so the browser rejects it before following the redirect. The remaininggithub.com/…data-lectures URLs in this repo are prose links and{download}roles in other lectures, which are plain navigations and CORS-exempt; the audit's rule-5 assertion scans code cells only.ci.ymlrunsmyst build --htmlwith no execution, so a dead data URL is invisible to it — green here is not evidence the repoint works. That is why the equivalence above was measured directly. Worth knowing too:simple_linear_regressionis in this repo's TOC and executes in-browser, whileinequalityis commented out atmyst.yml:44, so only one of these two lectures is currently reader-facing here.Merge together with
QuantEcon/lecture-python-intro#834 and QuantEcon/test-actions-lecture-intro, same branch name. Repoint rule 2 — merging one half while the other sits open partially repoints the dataset and opens the drift window; that was observed on 2026-08-06 when a wasm PR merged ahead of its intro counterpart.
Tracking: QuantEcon/workspace-lectures#23