Variational AutoEncoders

A two-part teaching deck on autoencoders and variational autoencoders — from the encoder/decoder bottleneck and its compression analogy, through the ELBO and the reparameterization trick, to worked examples in Keras, PyTorch and JAX.

▶ View the presentation

Built with Quarto and reveal.js. Fifty-one slides; press ? in the deck for keyboard shortcuts, o for the slide overview, b for the chalkboard.

Contents

Part 1 — Generative vs. discriminative models · compression and decompression · the autoencoder · PCA and the linear autoencoder · Jensen's inequality, variational Bayes and the VAE derivation · pros, cons and extensions (β-VAE, VQ-VAE, conditional VAE, InfoMax VAE).

Part 2 — GenAI and autoencoder recap · deep learning frameworks and accelerated hardware · the three Keras APIs (Sequential, Model subclassing, Functional) · the ELBO in detail · generating variants on MNIST and human_nontata_promoters · PyTorch.

Notebooks and references

• Autoencoders notebook (Colab)
• Keras 3.0 VAE (Colab)
• PyTorch-VAE repository
• Kingma & Welling (2019), An Introduction to Variational Autoencoders — arXiv:1906.02691

Building locally

Requires Quarto 1.4 or newer. No Python or R environment is needed — the deck has no executable cells.

quarto preview     # live-reloading preview in the browser
quarto render      # writes the static deck to _site/

Repository layout

index.qmd                       the deck
theme/simple-dark.scss          the "Simple Dark" theme, ported from the original
images/                         all 15 figures extracted from the source .pptx
                                (including the six now rendered as tables or
                                code, kept as provenance for the transcription)
variational_autoencoders.pptx   the original PowerPoint deck

theme/simple-dark.scss reproduces the palette of the original Google Slides theme — #212121 ground, white text, and the teal/cyan/orange accents — read out of the .pptx theme part rather than eyeballed.

Notes on the conversion

The Quarto deck follows the PowerPoint one slide for slide, including the blank slide and the repeated ?s and KAHOOT.IT slides. Three things are rendered rather than reproduced as screenshots:

• The Pros & Cons, Types and Why Frameworks tables were images in the original; they are now real tables, so they stay sharp and searchable.
• Two defects in those screenshots were repaired: the Types table showed a broken LaTeX escape in place of P(y | x), and the Why Frameworks table was cropped mid-sentence at the right edge ("Standard, less efficient without p…" → "…parallelism").
• The Model Class API code and the two ASCII architecture diagrams were images; they are now a syntax-highlighted code block and preformatted text.