Comparing Different Residual Block Types for Image Classification

This project investigates how different residual block architectures impact the learning behavior and performance of Residual Networks (ResNet). Specifically, it implements a "Mini-ResNet" and evaluates four distinct modular variants of residual blocks on a 25-class subset of the Tiny-ImageNet dataset.

Project Overview

The objective is to perform an ablation study on residual block designs, comparing "Post-Activation" vs. "Pre-Activation" strategies and "Basic" vs. "Bottleneck" configurations.

Residual Block Variants

1. Basic Post-Activation: Standard ResNet block with ReLU after the addition.
2. Bottleneck Post-Activation: Uses 1x1 convolutions to reduce and then restore dimensions (bottleneck) with post-activation.
3. Basic Pre-Activation: Standard block with Batch Normalization and ReLU before the convolutions.
4. Bottleneck Pre-Activation: Bottleneck configuration with pre-activation.

Architecture: Mini-ResNet

The custom Mini-ResNet is designed for 64x64 input images and consists of:

• Stem: 3x3 Conv (32 filters, stride 1) → BatchNorm → ReLU.
• Stage 1: 2 Residual Blocks (32 channels).
• Stage 2: 2 Residual Blocks (64 channels, stride 2 downsampling in the first block).
• Stage 3: 2 Residual Blocks (128 channels, stride 2 downsampling in the first block).
• Head: Global Average Pooling (GAP) → Fully Connected Layer (25 classes).

Dataset: Tiny-ImageNet-25

• Source: A subset of the Tiny-ImageNet dataset.
• Classes: 25 randomly selected classes (seeded using the last 3 digits of a matriculation number).
• Preprocessing:
  • Images normalized using ImageNet statistics: mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225].
  • Data loaders handle class isolation and label remapping.

Implementation Details

• Framework: PyTorch
• Optimization: SGD with momentum (0.9), Weight Decay (5e-4), and Cross-Entropy Loss.
• Metrics Tracked:
  • Training/Validation Loss and Accuracy.
  • Model Size (Parameters).
  • Computational Complexity (MACs).
  • Inference Latency (averaged over 100 iterations with 10 warm-up runs).

Files in this Repository

• Project3_Results.ipynb: The primary notebook containing the ablation study results, comparative analysis, and visualizations.
• models.py: Implementation of the MiniResNet and the four modular block types.
• data.py: Data loading and preprocessing scripts for Tiny-ImageNet-25.
• train_all.py: Script to train all four variants sequentially and save results.
• utils.py: Utility functions for metrics calculation and visualization.

Results Summary

Detailed empirical results, including loss/accuracy curves and performance tables, are available in the Project3_Results.ipynb notebook.

---

This project was completed as part of the CG3201 Coursework.