Improve all workshop notebooks for clear real-data learning (#44)

CLAUDE.md

@@ -19,19 +19,21 @@ counted — and both get it from `scripts/timeline.py` rather than walking it
 twice.
 
 **Never hand-edit generated scaffolding.** For notebooks, that rule applies
-only to the centrally owned header, Setup preamble, Setup code and footer.
-Teaching body cells are deliberately edited directly in the `.ipynb` file,
-including in Colab with Gemini, and the notebook normalizer preserves them.
+only to the centrally owned header (cell 0) and footer (final cell). Every
+cell between them -- including the entire Setup section (its heading, its
+prose and its code) -- is a teaching body cell, edited directly in the
+`.ipynb` file, including in Colab with Gemini, and the notebook normalizer
+preserves them.
 
-Change `_variables.yml` for shared facts/objectives or `scripts/content.py`
-for Setup code, then run the appropriate generator:
+Change `_variables.yml` for shared facts, objectives and the bilingual header
+text, then run the appropriate generator:
 
 | Generated | Owned by |
 |---|---|
 | `_includes/*.md` (every section table, and the agenda both decks show) | `scripts/gen_tables.py` |
 | The marker-delimited table regions inside `README.md` and `notebooks/README.md` — the rest of both files is hand-maintained | `scripts/gen_tables.py` |
-| `notebooks/*.ipynb` — header/objectives/Colab badge, Setup preamble, Setup code and footer | `scripts/gen_notebooks.py` using `_variables.yml` + `scripts/content.py` |
-| `notebooks/*.ipynb` — teaching body cells | the notebook itself; editable directly in Colab/Gemini |
+| `notebooks/*.ipynb` — header (cell 0) and footer (final cell) only | `scripts/gen_notebooks.py` using `_variables.yml` |
+| `notebooks/*.ipynb` — every cell between the header and footer, including the Setup section | the notebook itself; editable directly in Colab/Gemini |
 | `images/ds-*` (dataset cards) | `scripts/gen_thumbnails.py` |
 | `images/hero-band.png`, `images/fig-*` (the handbook's figures) | `scripts/gen_figures.py` |
 | `docs/` | `quarto render` |
@@ -151,9 +153,9 @@ between macOS and ubuntu-latest at the same version).
 `notebooks/*.ipynb` are `resources:` in `_quarto.yml`, not `render:` targets —
 Quarto copies them into `docs/notebooks/` verbatim. So a notebook change that
 is committed without a re-render leaves `docs/notebooks/` serving the old copy,
-and nothing fails: the regenerate gate compares `notebooks/` against
-`content.py` and never looks in `docs/`, while `compare_render.py` skips
-non-HTML entirely. This has already happened once, to nine of the twelve
+and nothing fails: the regenerate gate reruns `scripts/gen_notebooks.py` and
+only fails if the tracked `notebooks/` drift from the normalizer's output,
+never looking in `docs/`, while `compare_render.py` skips non-HTML entirely. This has already happened once, to nine of the twelve
 notebooks at once. Re-render after *any* notebook change, not only after a
 prose or `_variables.yml` change.
 

_includes/agenda-en.md

@@ -4,7 +4,7 @@
 |---|---|---|---|
 | 00:00 | 5 | — | Setup and welcome |
 | 00:05 | 20 | I | What a tensor is |
-| 00:25 | 20 | II | Thinking in N dimensions *(group)* |
+| 00:25 | 20 | II | Thinking in N dimensions |
 | 00:45 | 30 | III | Indexing & broadcasting · Reshape & transpose |
 | 01:15 | 10 | 🎯 | **Kahoot 1** + break |
 | 01:25 | 15 | III | Video pipeline design *(group)* |

_includes/agenda-es.md

@@ -4,7 +4,7 @@
 |---|---|---|---|
 | 00:00 | 5 | — | Preparación y bienvenida |
 | 00:05 | 20 | I | Qué es un tensor |
-| 00:25 | 20 | II | Pensar en N dimensiones *(grupo)* |
+| 00:25 | 20 | II | Pensar en N dimensiones |
 | 00:45 | 30 | III | Indexación y broadcasting · Reshape y transposición |
 | 01:15 | 10 | 🎯 | **Kahoot 1** + pausa |
 | 01:25 | 15 | III | Diseño de un pipeline de vídeo *(grupo)* |

_includes/notebooks-en.md

@@ -2,14 +2,14 @@
 | # | Notebook | Covers | Colab |
 |---|---|---|---|
 | 00 | [`00-setup-and-data.ipynb`](https://github.com/project-delphi/tensors-workshop/blob/main/notebooks/00-setup-and-data.ipynb) | Setup and welcome — Load every dataset and confirm your runtime works before anything else. | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/project-delphi/tensors-workshop/blob/main/notebooks/00-setup-and-data.ipynb) |
-| 01 | [`01-what-a-tensor-is.ipynb`](https://github.com/project-delphi/tensors-workshop/blob/main/notebooks/01-what-a-tensor-is.ipynb) | What a tensor is — The vocabulary, shape in NumPy, and the three operations that matter. | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/project-delphi/tensors-workshop/blob/main/notebooks/01-what-a-tensor-is.ipynb) |
-| 02 | [`02-thinking-in-n-dimensions.ipynb`](https://github.com/project-delphi/tensors-workshop/blob/main/notebooks/02-thinking-in-n-dimensions.ipynb) | Thinking in N dimensions — Argue about what each axis means, and why a batch axis differs from a time axis. | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/project-delphi/tensors-workshop/blob/main/notebooks/02-thinking-in-n-dimensions.ipynb) |
-| 03 | [`03-indexing-and-broadcasting.ipynb`](https://github.com/project-delphi/tensors-workshop/blob/main/notebooks/03-indexing-and-broadcasting.ipynb) | Indexing and broadcasting real data — Select the right column of real tumour data, then meet zero-variance pixels. | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/project-delphi/tensors-workshop/blob/main/notebooks/03-indexing-and-broadcasting.ipynb) |
-| 04 | [`04-reshape-and-transpose.ipynb`](https://github.com/project-delphi/tensors-workshop/blob/main/notebooks/04-reshape-and-transpose.ipynb) | Reshape and transpose real images — HWC to CHW, NHWC to NCHW, and why reshape silently destroys an image. | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/project-delphi/tensors-workshop/blob/main/notebooks/04-reshape-and-transpose.ipynb) |
-| 05 | [`05-video-pipeline-design.ipynb`](https://github.com/project-delphi/tensors-workshop/blob/main/notebooks/05-video-pipeline-design.ipynb) | Video pipeline design — Design the tensor shape at every stage of two real video systems. | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/project-delphi/tensors-workshop/blob/main/notebooks/05-video-pipeline-design.ipynb) |
-| 06 | [`06-contraction-with-einsum.ipynb`](https://github.com/project-delphi/tensors-workshop/blob/main/notebooks/06-contraction-with-einsum.ipynb) | Contraction with einsum — One notation for the dot product, the matrix product, and a batch of images. | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/project-delphi/tensors-workshop/blob/main/notebooks/06-contraction-with-einsum.ipynb) |
-| 07 | [`07-inverses-and-pseudoinverse.ipynb`](https://github.com/project-delphi/tensors-workshop/blob/main/notebooks/07-inverses-and-pseudoinverse.ipynb) | Inverses and the pseudoinverse — Solve a 20,433-equation system that has no exact solution. | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/project-delphi/tensors-workshop/blob/main/notebooks/07-inverses-and-pseudoinverse.ipynb) |
-| 08 | [`08-recursion-with-matrices.ipynb`](https://github.com/project-delphi/tensors-workshop/blob/main/notebooks/08-recursion-with-matrices.ipynb) | Recursion with matrices and vectors — Apply one matrix again and again: Fibonacci, eigenvectors, and a real forecast. | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/project-delphi/tensors-workshop/blob/main/notebooks/08-recursion-with-matrices.ipynb) |
-| 09 | [`09-convolution-and-deconvolution.ipynb`](https://github.com/project-delphi/tensors-workshop/blob/main/notebooks/09-convolution-and-deconvolution.ipynb) | Convolution and deconvolution — Convolution is a structured matrix product, and blur can be partly undone. | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/project-delphi/tensors-workshop/blob/main/notebooks/09-convolution-and-deconvolution.ipynb) |
-| 10 | [`10-tucker-decomposition.ipynb`](https://github.com/project-delphi/tensors-workshop/blob/main/notebooks/10-tucker-decomposition.ipynb) | Tucker decomposition on real data — PCA generalized to every axis, on a real tensor of New York taxi trips. | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/project-delphi/tensors-workshop/blob/main/notebooks/10-tucker-decomposition.ipynb) |
-| 11 | [`11-wrap-up-and-take-homes.ipynb`](https://github.com/project-delphi/tensors-workshop/blob/main/notebooks/11-wrap-up-and-take-homes.ipynb) | Wrap-up and take-homes — What connects Blocks 4, 5 and 6, plus five take-home exercises. | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/project-delphi/tensors-workshop/blob/main/notebooks/11-wrap-up-and-take-homes.ipynb) |
+| 01 | [`01-what-a-tensor-is.ipynb`](https://github.com/project-delphi/tensors-workshop/blob/main/notebooks/01-what-a-tensor-is.ipynb) | What a tensor is — Learn to read a tensor's structure and track what its axes mean as you fix, rearrange, or contract them. | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/project-delphi/tensors-workshop/blob/main/notebooks/01-what-a-tensor-is.ipynb) |
+| 02 | [`02-thinking-in-n-dimensions.ipynb`](https://github.com/project-delphi/tensors-workshop/blob/main/notebooks/02-thinking-in-n-dimensions.ipynb) | Thinking in N dimensions — Learn to read real tensors by asking what every axis counts and why a batch axis is not the same thing as a time axis. | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/project-delphi/tensors-workshop/blob/main/notebooks/02-thinking-in-n-dimensions.ipynb) |
+| 03 | [`03-indexing-and-broadcasting.ipynb`](https://github.com/project-delphi/tensors-workshop/blob/main/notebooks/03-indexing-and-broadcasting.ipynb) | Indexing and broadcasting real data — Select named measurements from real tumour-sample data, compare meaningful subsets, then standardize real image data safely. | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/project-delphi/tensors-workshop/blob/main/notebooks/03-indexing-and-broadcasting.ipynb) |
+| 04 | [`04-reshape-and-transpose.ipynb`](https://github.com/project-delphi/tensors-workshop/blob/main/notebooks/04-reshape-and-transpose.ipynb) | Reshape and transpose real images — Use real images to move between HWC↔CHW and NHWC↔NCHW, then show why matching shapes do not guarantee matching axis semantics. | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/project-delphi/tensors-workshop/blob/main/notebooks/04-reshape-and-transpose.ipynb) |
+| 05 | [`05-video-pipeline-design.ipynb`](https://github.com/project-delphi/tensors-workshop/blob/main/notebooks/05-video-pipeline-design.ipynb) | Video pipeline design — Process one pinned real video end to end, then use what its axes actually mean to design two downstream video pipelines. | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/project-delphi/tensors-workshop/blob/main/notebooks/05-video-pipeline-design.ipynb) |
+| 06 | [`06-contraction-with-einsum.ipynb`](https://github.com/project-delphi/tensors-workshop/blob/main/notebooks/06-contraction-with-einsum.ipynb) | Contraction with einsum — Use one index rule on real data, then change the inputs interactively to test which index disappears. | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/project-delphi/tensors-workshop/blob/main/notebooks/06-contraction-with-einsum.ipynb) |
+| 07 | [`07-inverses-and-pseudoinverse.ipynb`](https://github.com/project-delphi/tensors-workshop/blob/main/notebooks/07-inverses-and-pseudoinverse.ipynb) | Inverses and the pseudoinverse — Use the pseudoinverse on real singular, tall, and wide systems, then inspect the geometry interactively. | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/project-delphi/tensors-workshop/blob/main/notebooks/07-inverses-and-pseudoinverse.ipynb) |
+| 08 | [`08-recursion-with-matrices.ipynb`](https://github.com/project-delphi/tensors-workshop/blob/main/notebooks/08-recursion-with-matrices.ipynb) | Recursion with matrices and vectors — Treat recursion as repeated state updates, connect repeated multiplication with dominant eigen-directions, and test recursive forecasting on real airline-passenger data. | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/project-delphi/tensors-workshop/blob/main/notebooks/08-recursion-with-matrices.ipynb) |
+| 09 | [`09-convolution-and-deconvolution.ipynb`](https://github.com/project-delphi/tensors-workshop/blob/main/notebooks/09-convolution-and-deconvolution.ipynb) | Convolution and deconvolution — Treat convolution as a structured linear operator, separate it from correlation, understand transposed convolution, and partially recover a real blurred image. | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/project-delphi/tensors-workshop/blob/main/notebooks/09-convolution-and-deconvolution.ipynb) |
+| 10 | [`10-tucker-decomposition.ipynb`](https://github.com/project-delphi/tensors-workshop/blob/main/notebooks/10-tucker-decomposition.ipynb) | Tucker decomposition on real data — Build a real tensor from New York taxi trips, compress each mode with HOSVD, and explore the trade-off between size, reconstruction error, and interpretable structure. | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/project-delphi/tensors-workshop/blob/main/notebooks/10-tucker-decomposition.ipynb) |
+| 11 | [`11-wrap-up-and-take-homes.ipynb`](https://github.com/project-delphi/tensors-workshop/blob/main/notebooks/11-wrap-up-and-take-homes.ipynb) | Wrap-up and take-homes — Wrap up the workshop around one connecting idea, then choose among five extensions: PCA, attention, CP, Cholesky, and audio. | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/project-delphi/tensors-workshop/blob/main/notebooks/11-wrap-up-and-take-homes.ipynb) |