Notebook 12: matrix factorizations, and what each one costs

[project-delphi]

Closes #54.

Fills in the teaching body of notebooks/12-matrix-factorizations.ipynb — the stub from #53 — and adds the handbook appendix that mirrors it.

The notebook

44 cells: four exercises with folded solutions, five widgets, and a bilingual body throughout. Six factorizations framed as constrained optimizations (§12.1), a cost table derived from Trefethen & Bau's flop counts (§12.2), and a second cost table the reader measures for themselves (§12.3).

Every number quoted in the prose was executed and checked against a full nbconvert --execute run, which completes with zero errors and zero warnings:

Where	What it shows
Ex. 1	Degree-10 polynomial on the real airline series. κ(X) = 2.16e7, κ(XᵀX) = 4.65e14, coefficient errors 1.95e-03 (normal equations) against 2.04e-14 (QR) — while the residuals agree to six decimals, so the check most people run sees nothing wrong
Ex. 2	The 1797×1797 RBF kernel system of the digits, 200 right-hand sides. Factoring once is ~350× a per-column refactor and ~4× an explicit inverse, which is also less accurate. Held-out accuracy 98.3%, so it is a real model
Ex. 3	Size sweep with fitted log-log slopes near 2.2–2.5, not 3 — with the honest reason, and the prediction that does survive a real machine: the SVD/Cholesky ratio at ~32× against the table's 39×
§12.4	Eckart–Young checked to 1e-13 on the astronaut image; rank explorer separating numbers stored from bytes stored (int16 vs float32)
Ex. 4	The smallest rank crossing a dB threshold, read off the singular values without rebuilding anything
§12.5	Digits truncated to rank k feeding a least-squares classifier: best held-out accuracy at k = 41, above full rank. Truncation as regularization, not just compression
§12.6	NMF against the SVD at rank 16: worse by four points of error, with over 80% of its component entries at zero against the SVD's 21%
§12.7	Flop budget calibrated on this machine, labelled as an order-of-magnitude sizing tool rather than a benchmark

No new datasets. The images, digits and airline series are all already in the workshop; the airline CSV is the one notebook 08 fetches.

Handbook

Adds Appendix E — Matrix Factorizations: Which One, and What It Costs: framing paragraph, the cost table, four # TODO blocks mirroring the notebook's exercises, and a Colab link. Facilitator Notes moves to Appendix G, leaving F for the tensor-factorizations deep dive (#55). Cross-links added from §1.4's factorization map and the wrap-up's "Where to go next".

slides/README.md pointed at "Appendix D's facilitator guidance" — already the wrong letter — now corrected to G.

One deviation from the issue, flagged

The issue asked to shrink Appendix D to a pointer, on the grounds that it is "a one-exercise version of material this appendix covers properly." That is true of D's framing but not of its content: D is about Cholesky as a sampler (correlated Monte Carlo, portfolio tails, with quantified verified results), and Appendix E covers Cholesky as a solver. Nothing in E replaces it.

So D keeps its exercises and solution, and loses only the framing paragraph that E now duplicates — replaced by a cross-link naming the split explicitly. Happy to cut it entirely if the redundancy was the point rather than the reasoning; say so and I will.

Checks

All ten checks in check_links.py pass, including the byte-identical docs/notebooks gate from #58 and the internal-fragment check that validates the new appendix anchor. gen_tables.py and gen_notebooks.py are idempotent on the result. Rendered with Quarto 1.6.40 and docs/ is committed.

🤖 Generated with Claude Code

https://claude.ai/code/session_01DP1TzSePjs3YLNLKJF2Zrw

[project-delphi]

Automated review — /code-review (medium effort)

Reviewed all three commits at the time of review — 4cfa900, f654c7a, 781408a — against main.

The reviewer re-executed the numeric claims independently rather than trusting the PR body. Every quoted figure reproduced: κ(X)=2.16e7 / κ(XᵀX)=4.65e14, coefficient errors 1.95e-03 vs 2.04e-14 with residuals identical to six decimals; degree-12 is indeed where the residual itself degrades (549.76 vs 503.69); Eckart–Young equality to ≤5e-14 at k=5…100; rank 16 → 21.09 dB at 12.5% int16 bytes, rank 50 → 27.35 dB at 39.1%, rank 128 → 100.1%; needed_rank → [13, 35, 72, 124] with the k=35 verification at 25.13 dB against 24.96 one rank lower; digits best held-out accuracy 0.950 at k=41, above full-rank 0.937; NMF 0.2567 vs SVD 0.2180 error with 83.2% vs 21.0% near-zero entries. Cell 10 behaves as described (Cholesky raises, NMF raises, eigh silently returns the mirrored matrix's spectrum). check_links.py passes all ten checks, and docs/notebooks/12-…ipynb is byte-identical to notebooks/ at every commit.

Critical Issues

None.

Important Issues

tensors_workshop_plan_with_quizzes.md — the appendix lettering ships with a hole at F. The handbook runs A, B, C, D, E, G. Appendix F is reserved for the tensor-factorizations deep dive in #55, which is not merged, so what Pages serves today has a visibly skipped letter, and slides/README.md:34 plus the facilitator note at line 8 both point readers at "Appendix G" past a gap. Flagged rather than blocking: the deviation is intentional per the PR body, but the PR body only defends keeping Appendix D, not the gap itself.

Suggestions

1. budget() applies square-only formulas to non-square input with no caveat. FLOPS["Cholesky"], ["LU"] and ["Eigendecomposition"] are n³/3, 2n³/3, 9n³, but budget() evaluates all six over whatever the sliders hold. At the default m = 10,000, n = 500 it prints a flop count, factor size and compression ratio for a Cholesky, an LU and an eigendecomposition of a 10,000×500 matrix — none of which exist. §12.7 is explicitly framed as "the matrix you will bring tomorrow", so this is the one cell where a learner is invited to trust output on unfamiliar shapes.
2. Appendix E's TODO 4 snippet has an unguarded divide-by-zero. db = 10 * np.log10(1.0 / (tail / A.size)) divides by tail[-1], exactly 0.0 by construction. The notebook wraps the identical line in np.errstate(divide="ignore"); the handbook copy does not, so pasting it emits a RuntimeWarning and raises under np.seterr(divide="raise").
3. np.linalg.eig(block)[0].real is labelled "(correct)". Real for this fixed block, so the output is right — but .real on a general non-symmetric matrix silently discards imaginary parts, which is precisely the failure mode the cell exists to warn about. Also eig returns eigenvalues unordered while eigh returns them ascending, so the two printed rows do not correspond element-wise even though the layout invites comparison.
4. Number drift across the three places the same measurement is quoted (handbook "~31x", PR body "~32x", prose "2.2 to 2.9" vs "near 2.4"). These are machine properties, and the facilitator note already declares Appendix E's timings non-reproducible.
5. Shared rng is consumed in stream order by cells 19, 21, 23 and 41, so re-running any one shifts the matrices every later cell receives — the §12.3 sweep and §12.7 calibration are not reproducible across a non-linear execution order. The notebook already works around this twice with private generators.

Strengths

• The #appendix-e-… anchors resolve correctly under Quarto's id generation (em dash collapsed, not doubled), verified against the committed HTML, and check 3 covers them going forward.
• Cells 16 and 21 rebind X/y and sq/D2/G from the folded solutions with an explicit comment, which is what keeps check 9 green across 162 visible cells.
• needed_rank computes the whole PSNR-vs-k curve from one SVD via a reversed cumulative sum instead of rebuilding 512 truncations — the exercise's stated point, correctly implemented, including the errstate guard on the terminal zero.
• The §12.6 sparsity headline survives the obvious objection: re-measured with a per-component threshold instead of the global one gives 79.4% vs 19.9%, so the 83%/21% claim is not an artifact of NMF's scale indeterminacy.
• docs/notebooks/ stayed in sync through all three commits, including two mid-review pushes — the gate added in #58 is doing its job.

---

What I did about it

Zero Critical Issues, so this is mergeable. I fixed the Important issue and all five Suggestions rather than deferring them — see the follow-up commit:

• Appendix F is now a short reserved stub pointing at the notebook, so the sequence reads as deliberate instead of broken.
• budget() prints n/a — needs a square matrix for Cholesky, LU and the eigendecomposition when m != n, with a line saying why.
• Appendix E's snippet gained the errstate guard the notebook already had.
• The eig/eigh cell now prints the largest imaginary part (so dropping it is justified rather than assumed, with a note that a rotation matrix genuinely has complex eigenvalues) and sorts both rows so they line up.
• The sweep and calibration cells take private generators, matching the pattern the notebook already used for the two splits.
• The handbook's "~31x" is now "well under the predicted 39x" — the ratio is a machine property and two runs here gave 31.3x and 31.8x.