This project demonstrates the implementation of a simple linear regression model built entirely from scratch, utilizing the core concepts of a lightweight autodifferentiation engine inspired by micrograd.
The goal is to train a model to uncover a hidden linear relationship within noisy, synthetically generated data.
| File Name | Purpose | Description |
|---|---|---|
Parabolic_micrograd.py |
Core Autodiff Engine (Primitives) | Contains the foundational components for building the neural network structure, including the custom Value class. Defines basic mathematical operations (addition, multiplication, power, etc.) and their corresponding gradient calculation logic for the backward pass. |
Regression_model_micrograd.py |
Application and Training Loop | Implements the complete end-to-end regression pipeline, including: 1️ Data Generation – creates parabolic data with noise. 2️ LinearModel Class – defines the regression model. 3️ Loss Function – Mean Squared Error (MSE). 4️ Training Loop – performs gradient descent for parameter optimization. |
🚀 How It Works
- Generate synthetic data with an underlying parabolic relationship and added random noise.
- Define a linear model with parameters (
w,b) represented as customValueobjects. - Compute the loss using Mean Squared Error (MSE).
- Backpropagate gradients through the computation graph using the custom autodiff engine.
- Update parameters via manual gradient descent steps.
- Building a minimal autodifferentiation system from scratch
- Implementing a computation graph and backward propagation manually
- Training a model using gradient-based optimization
- Python 3.8+
- Makes use of libraries like numpy, pandas, matplotlib and graphviz
After training, the model learns to approximate the hidden relationship between x and y, reducing the mean squared error significantly over time.
This project is released under the MIT License.
Author: Shreya
Inspired by: Andrej Karpathy’s micrograd