Skip to content

Refactor notebooks for direct Colab editing #47

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
project-delphi merged 1 commit into from
Aug 27, 2026
Merged

Refactor notebooks for direct Colab editing #47

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 9 additions & 4 deletions CLAUDE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,15 +18,20 @@ what time each section starts, once the quizzes and breaks between them are
counted — and both get it from `scripts/timeline.py` rather than walking it
twice.

**Never hand-edit generated output.** Change `_variables.yml` (or
`scripts/content.py` for notebook teaching content), then regenerate:
**Never hand-edit generated scaffolding.** For notebooks, that rule applies
only to the centrally owned header, Setup preamble, Setup code and footer.
Teaching body cells are deliberately edited directly in the `.ipynb` file,
including in Colab with Gemini, and the notebook normalizer preserves them.

Change `_variables.yml` for shared facts/objectives or `scripts/content.py`
for Setup code, then run the appropriate generator:

| Generated | Owned by |
|---|---|
| `_includes/*.md` (every section table, and the agenda both decks show) | `scripts/gen_tables.py` |
| The marker-delimited table regions inside `README.md` and `notebooks/README.md` — the rest of both files is hand-maintained | `scripts/gen_tables.py` |
| `notebooks/*.ipynb` — scaffolding: header, objectives, Colab badge, Kahoot footer | `scripts/gen_notebooks.py` |
| `notebooks/*.ipynb` — body cells | `scripts/content.py` |
| `notebooks/*.ipynb` — header/objectives/Colab badge, Setup preamble, Setup code and footer | `scripts/gen_notebooks.py` using `_variables.yml` + `scripts/content.py` |
| `notebooks/*.ipynb` — teaching body cells | the notebook itself; editable directly in Colab/Gemini |
| `images/ds-*` (dataset cards) | `scripts/gen_thumbnails.py` |
| `images/hero-band.png`, `images/fig-*` (the handbook's figures) | `scripts/gen_figures.py` |
| `docs/` | `quarto render` |
Expand Down
130 changes: 130 additions & 0 deletions _variables.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -160,6 +160,14 @@ sections:
title_es: "Preparación y bienvenida"
summary_en: "Load every dataset and confirm your runtime works before anything else."
summary_es: "Carga todos los conjuntos de datos y confirma que tu entorno funciona antes de empezar."
objectives_en:
- "Confirm your Colab runtime can reach every dataset the workshop uses."
- "Know which data ships inside the libraries and which is downloaded."
- "Recognise the shapes you will be working with all day."
objectives_es:
- "Confirmar que tu entorno de Colab puede acceder a todos los conjuntos de datos que usa el taller."
- "Saber qué datos vienen incluidos en las librerías y cuáles se descargan."
- "Reconocer las formas de los tensores con las que trabajarás durante todo el taller."
s01:
n: "01"
slug: "what-a-tensor-is"
Expand All @@ -173,6 +181,18 @@ sections:
title_es: "Qué es un tensor"
summary_en: "The vocabulary, shape in NumPy, and the three operations that matter."
summary_es: "El vocabulario, la forma en NumPy y las tres operaciones que importan."
objectives_en:
- "Use the vocabulary: order, axis, mode, shape, slice, fiber, unfolding, contraction, decomposition."
- "Read `.shape`, `.ndim` and `.size` off any array and say what each axis means."
- "Take slices and fibers, and unfold a tensor into a matrix without losing anything."
- "Write a dot product and a matrix product as `einsum` contractions."
- "Place LU, QR, eigendecomposition, SVD, the pseudoinverse, Cholesky and Tucker in one map."
objectives_es:
- "Usar el vocabulario: orden, eje, modo, forma, slice, fibra, unfolding, contracción y descomposición."
- "Leer `.shape`, `.ndim` y `.size` de cualquier arreglo y explicar qué significa cada eje."
- "Tomar slices y fibras, y desplegar un tensor en una matriz sin perder información."
- "Escribir un producto punto y un producto matricial como contracciones con `einsum`."
- "Ubicar LU, QR, descomposición en valores propios, SVD, pseudoinversa, Cholesky y Tucker en un mismo mapa."
s02:
n: "02"
slug: "thinking-in-n-dimensions"
Expand All @@ -186,6 +206,16 @@ sections:
title_es: "Pensar en N dimensiones"
summary_en: "Argue about what each axis means, and why a batch axis differs from a time axis."
summary_es: "Discutir qué significa cada eje y por qué un eje de lote difiere de un eje temporal."
objectives_en:
- "Say what a new axis *counts*, rather than saying \"we add a dimension\"."
- "Explain why a batch axis and a time axis behave differently despite identical shapes."
- "Propose two ways to batch videos of different lengths, and say what each loses or invents."
- "Map experimental choices onto axes of a real microscopy tensor."
objectives_es:
- "Decir qué *cuenta* un nuevo eje, en lugar de decir simplemente \"agregamos una dimensión\"."
- "Explicar por qué un eje de batch y un eje temporal se comportan de manera diferente aunque tengan formas idénticas."
- "Proponer dos formas de agrupar vídeos de diferentes longitudes y explicar qué pierde o inventa cada una."
- "Representar decisiones experimentales mediante los ejes de un tensor real de microscopía."
s03:
n: "03"
slug: "indexing-and-broadcasting"
Expand All @@ -199,6 +229,16 @@ sections:
title_es: "Indexación y broadcasting con datos reales"
summary_en: "Select the right column of real tumour data, then meet zero-variance pixels."
summary_es: "Seleccionar la columna correcta de datos reales de tumores y encontrar píxeles de varianza cero."
objectives_en:
- "Select a named column of real data by name, never by a hard-coded number."
- "Combine fancy indexing and boolean indexing to pull out sub-tables in one operation."
- "Standardize a data matrix with broadcasting."
- "Recognise a zero-variance column, and know why real images contain them."
objectives_es:
- "Seleccionar una columna de datos reales por su nombre, nunca mediante un número escrito manualmente."
- "Combinar fancy indexing e indexación booleana para extraer subtablas en una sola operación."
- "Estandarizar una matriz de datos mediante broadcasting."
- "Reconocer una columna de varianza cero y comprender por qué las imágenes reales pueden contenerlas."
s04:
n: "04"
slug: "reshape-and-transpose"
Expand All @@ -212,6 +252,14 @@ sections:
title_es: "Reshape y transposición de imágenes reales"
summary_en: "HWC to CHW, NHWC to NCHW, and why reshape silently destroys an image."
summary_es: "De HWC a CHW, de NHWC a NCHW, y por qué reshape destruye una imagen en silencio."
objectives_en:
- "Convert an image between `(H, W, C)` and `(C, H, W)` with `np.transpose`."
- "Convert a batch between NHWC and NCHW, and know which axis is which when two share a size."
- "Explain why `reshape` runs without error and still destroys the image."
objectives_es:
- "Convertir una imagen entre `(H, W, C)` y `(C, H, W)` con `np.transpose`."
- "Convertir un batch entre NHWC y NCHW y distinguir correctamente los ejes cuando dos tienen el mismo tamaño."
- "Explicar por qué `reshape` puede ejecutarse sin error y aun así destruir la estructura de la imagen."
s05:
n: "05"
slug: "video-pipeline-design"
Expand All @@ -225,6 +273,16 @@ sections:
title_es: "Diseño de un pipeline de vídeo"
summary_en: "Design the tensor shape at every stage of two real video systems."
summary_es: "Diseñar la forma del tensor en cada etapa de dos sistemas de vídeo reales."
objectives_en:
- "Read the shape of a real decoded video and say what each of its four axes counts."
- "Design the tensor shape at each of five pipeline stages, for two different systems."
- "Apply one ragged-length strategy from section 02 and give the exact batched shape."
- "Decide where a new axis goes, and say how that choice affects the rest of the pipeline."
objectives_es:
- "Leer la forma de un vídeo real decodificado y explicar qué cuenta cada uno de sus cuatro ejes."
- "Diseñar la forma del tensor en cada una de cinco etapas del pipeline para dos sistemas diferentes."
- "Aplicar una estrategia para longitudes variables de la sección 02 y dar la forma exacta del batch."
- "Decidir dónde se ubica un nuevo eje y explicar cómo esa decisión afecta el resto del pipeline."
s06:
n: "06"
slug: "contraction-with-einsum"
Expand All @@ -238,6 +296,16 @@ sections:
title_es: "Contracción con einsum"
summary_en: "One notation for the dot product, the matrix product, and a batch of images."
summary_es: "Una sola notación para el producto punto, el producto matricial y un lote de imágenes."
objectives_en:
- "State the einsum rule: an index missing after the arrow is summed over."
- "Contract the colour axis of one image, and of a whole batch, with one call each."
- "Write trace, transpose and the matrix product as `einsum` and check them against NumPy."
- "Build a full similarity matrix between 1797 images with a single contraction."
objectives_es:
- "Enunciar la regla de einsum: un índice que no aparece después de la flecha se suma."
- "Contraer el eje de color de una imagen y de un batch completo con una sola llamada en cada caso."
- "Escribir la traza, la transposición y el producto matricial con `einsum` y verificarlos con NumPy."
- "Construir una matriz completa de similitud entre 1797 imágenes mediante una sola contracción."
s07:
n: "07"
slug: "inverses-and-pseudoinverse"
Expand All @@ -251,6 +319,18 @@ sections:
title_es: "Inversas y la pseudoinversa"
summary_en: "Solve a 20,433-equation system that has no exact solution."
summary_es: "Resolver un sistema de 20.433 ecuaciones que no tiene solución exacta."
objectives_en:
- "Say when a square matrix has no inverse, and predict the error before you see it."
- "Compute the Moore-Penrose pseudoinverse and verify its four defining conditions."
- "Say what `x = A⁺b` gives you for a tall matrix and for a wide one."
- "Solve a real 20,433-equation system that has no exact solution."
- "Apply the pseudoinverse to a tensor by unfolding, solving, and folding back."
objectives_es:
- "Identificar cuándo una matriz cuadrada no tiene inversa y anticipar el error antes de observarlo."
- "Calcular la pseudoinversa de Moore-Penrose y verificar sus cuatro condiciones definitorias."
- "Explicar qué produce `x = A⁺b` para una matriz alta y para una matriz ancha."
- "Resolver un sistema real de 20.433 ecuaciones que no tiene solución exacta."
- "Aplicar la pseudoinversa a un tensor mediante unfolding, solución y reconstrucción."
s08:
n: "08"
slug: "recursion-with-matrices"
Expand All @@ -264,6 +344,16 @@ sections:
title_es: "Recursión con matrices y vectores"
summary_en: "Apply one matrix again and again: Fibonacci, eigenvectors, and a real forecast."
summary_es: "Aplicar una misma matriz una y otra vez: Fibonacci, autovectores y un pronóstico real."
objectives_en:
- "Write a recurrence as repeated multiplication by one matrix."
- "Find the dominant eigenvector by power iteration, and check it against `np.linalg.eig`."
- "Fit an autoregressive model with the pseudoinverse and feed its own output back in."
- "Recognise that structure as the skeleton of a recurrent neural network."
objectives_es:
- "Escribir una recurrencia como multiplicaciones repetidas por una misma matriz."
- "Encontrar el vector propio dominante mediante power iteration y comprobarlo con `np.linalg.eig`."
- "Ajustar un modelo autorregresivo con la pseudoinversa y reutilizar su propia salida como entrada."
- "Reconocer esa estructura como el esqueleto de una red neuronal recurrente."
s09:
n: "09"
slug: "convolution-and-deconvolution"
Expand All @@ -277,6 +367,18 @@ sections:
title_es: "Convolución y deconvolución"
summary_en: "Convolution is a structured matrix product, and blur can be partly undone."
summary_es: "La convolución es un producto matricial estructurado, y el desenfoque se puede deshacer en parte."
objectives_en:
- "Predict the output size of a convolution in `full`, `valid` and `same` mode."
- "Say why what deep learning calls convolution is really correlation."
- "Write a convolution as multiplication by a Toeplitz matrix."
- "Distinguish transposed convolution from true deconvolution."
- "Recover a blurred image with Richardson-Lucy, and measure it honestly."
objectives_es:
- "Predecir el tamaño de salida de una convolución en los modos `full`, `valid` y `same`."
- "Explicar por qué lo que deep learning llama convolución es realmente correlación."
- "Escribir una convolución como multiplicación por una matriz de Toeplitz."
- "Distinguir la convolución transpuesta de una deconvolución verdadera."
- "Recuperar una imagen desenfocada con Richardson-Lucy y medir el resultado de manera rigurosa."
s10:
n: "10"
slug: "tucker-decomposition"
Expand All @@ -290,6 +392,18 @@ sections:
title_es: "Descomposición de Tucker con datos reales"
summary_en: "PCA generalized to every axis, on a real tensor of New York taxi trips."
summary_es: "PCA generalizado a todos los ejes, sobre un tensor real de viajes en taxi de Nueva York."
objectives_en:
- "Build a genuine order-3 tensor out of a flat table of real trips."
- "Compute a Tucker decomposition by HOSVD, using only unfolding, SVD and einsum."
- "Contract three axes at once with a single `einsum` string."
- "Measure reconstruction error against compression ratio."
- "Read a factor matrix and recognise a real pattern the decomposition found by itself."
objectives_es:
- "Construir un tensor real de orden 3 a partir de una tabla plana de viajes reales."
- "Calcular una descomposición de Tucker mediante HOSVD usando únicamente unfolding, SVD y einsum."
- "Contraer tres ejes simultáneamente con una sola expresión `einsum`."
- "Comparar el error de reconstrucción con la razón de compresión."
- "Interpretar una matriz de factores y reconocer un patrón real encontrado automáticamente por la descomposición."
s11:
n: "11"
slug: "wrap-up-and-take-homes"
Expand All @@ -303,3 +417,19 @@ sections:
title_es: "Cierre y ejercicios para casa"
summary_en: "What connects Blocks 4, 5 and 6, plus five take-home exercises."
summary_es: "Qué conecta los bloques 4, 5 y 6, más cinco ejercicios para casa."
objectives_en:
- "State the one idea that connects the pseudoinverse, deconvolution and Tucker."
- "Find the scaling trap in PCA on real, unstandardized data (take-home A)."
- "Build attention out of two contractions, and mask padded positions (take-home B)."
- "Run a real CP decomposition and read its components as trip types nobody labelled (take-home C)."
- "Build correlated data from independent noise with Cholesky, and see why ignoring covariance understates portfolio risk (take-home D)."
- "Denoise a real voice recording by truncating the SVD of its STFT, and measure the result in SNR rather than by ear (take-home E)."
- "Trade parameter count against reconstruction error with a rank slider, on a real dense tensor (optional appendix)."
objectives_es:
- "Enunciar la idea común que conecta la pseudoinversa, la deconvolución y Tucker."
- "Identificar la trampa de escala de PCA sobre datos reales sin estandarizar (take-home A)."
- "Construir atención mediante dos contracciones y enmascarar las posiciones de padding (take-home B)."
- "Ejecutar una descomposición CP real e interpretar sus componentes como tipos de viajes que nadie etiquetó previamente (take-home C)."
- "Construir datos correlacionados a partir de ruido independiente con Cholesky y observar por qué ignorar la covarianza subestima el riesgo de portafolio (take-home D)."
- "Eliminar ruido de una grabación de voz real truncando la SVD de su STFT y medir el resultado con SNR en lugar de evaluarlo solo de oído (take-home E)."
- "Comparar número de parámetros y error de reconstrucción mediante un slider de rango sobre un tensor denso real (apéndice opcional)."
35 changes: 25 additions & 10 deletions notebooks/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,19 +54,34 @@ dismissing**.
Every code cell has been executed against the real datasets. The handbook's
verified numbers all reproduce.

## Do not edit these files directly
## Editing notebooks with Colab and Gemini

They are generated:
The teaching body of each notebook is editable directly. Edits made in Colab,
including edits made with Gemini, no longer need to be retyped into
`scripts/content.py`.

```bash
uv run --with pyyaml,nbformat python scripts/gen_notebooks.py
```
Ownership is split deliberately:

- Generated: header, objectives, Colab badge, Setup preamble, Setup code, footer.
- Notebook-owned: teaching cells between Setup and the footer.

Shared objectives and workshop facts live in `_variables.yml`.
Central Setup code lives in `scripts/content.py`.

### Colab to GitHub workflow

1. Open the notebook from its Open in Colab badge.
2. Edit the teaching body with Gemini or by hand.
3. Use File > Save a copy in GitHub and save to your feature branch.
4. Run `uv run --with pyyaml,nbformat python scripts/gen_notebooks.py`.
5. Run `uv run --with pyyaml,nbformat python scripts/check_links.py`.
6. Render with the repository-pinned Quarto version.
7. Review the diff and open a pull request.

The normalizer removes outputs, execution counts and transient Colab per-cell
metadata while preserving teaching cells and folded-solution metadata.

Teaching content lives in [`scripts/content.py`](../scripts/content.py);
the header, Colab badge and Kahoot footer are built by
[`scripts/gen_notebooks.py`](../scripts/gen_notebooks.py) from
[`_variables.yml`](../_variables.yml). Editing a notebook by hand means your
change disappears the next time anyone regenerates.
Running `gen_notebooks.py` twice must produce no additional changes.

## Running them somewhere other than Colab

Expand Down
Loading
Loading