Deep Learning Foundations

Streamlit Blog for Digestible Deep Learning Concepts

This section of the blog covers the core foundations of Deep Learning, from the earliest statistical methods to the Transformer architecture.

Topics Covered

• Ordinary Least Squares (OLS)

  • Least squares regression and model fitting

• Linear Regression with Gradient Descent

  • Full Batch Gradient Descent optimization

• Logistic Regression

  • Binary classification with sigmoid activation

• Regularization

  • Ridge (L2)
  • LASSO (L1)
  • Elastic Net

• Perceptron

  • Single-layer perceptron
  • Limitations and early AI winter

• Feed Forward Networks (MLPs)

  • Backpropagation
  • Activation functions (Sigmoid, Tanh, ReLU, etc.)
  • Optimizers (Momentum, AdaGrad, RMSProp, Adam)
  • Regularization methods (Dropout, BatchNorm, LayerNorm)

• Convolutional Neural Networks (CNNs)

  • LeNet, AlexNet, VGG, ResNet

• Recurrent Neural Networks (RNNs)

  • Simple RNNs, LSTMs, GRUs
  • Seq2Seq models

• Transformers

  • Tokenization
  • Embedding
  • Positional Encoding
  • Attention Mechanism
  • Transformer architecture overview

---

📌 Each topic will include:

• Mathematical Intuition (derivations where helpful)
• Interactive Visualizations (Streamlit demos)
• Historical Context (who, when, why it mattered)
• Modern Relevance (how it connects to today’s models)