Audio Pattern Recognition — ESC-50

Environmental Sound Classification on the ESC-50 benchmark. Academic project for Audio Pattern Recognition (Università degli Studi di Milano, 2026).

Final grade: 10/10

A complete machine-learning pipeline that goes from raw waveform inspection to handcrafted-feature classification, model explainability and a deep-learning extension based on log-Mel spectrograms.

---

1. Project overview

The goal is to automatically classify 5-second environmental audio recordings into 50 sound categories using the ESC-50 dataset (2,000 balanced clips). The notebook covers:

1. Dataset inspection & audio exploration — metadata validation, waveform and spectrogram visualization across multiple classes.
2. Feature extraction — MFCCs with first and second temporal derivatives, log-Mel spectrograms, spectral shape descriptors (centroid, bandwidth, roll-off, contrast) and energy descriptors (RMS, zero-crossing rate).
3. Unsupervised analysis — k-means clustering and t-SNE projection of the feature space.
4. Supervised classification — SVM (RBF) with grid-search tuning, Random Forest, k-NN baseline.
5. Explainability — feature-importance analysis on the Random Forest model.
6. Deep learning extension — Convolutional Neural Network on log-Mel spectrograms with SpecAugment-style data augmentation.

---

2. Results

Model	Notes
SVM (RBF)	Best overall accuracy on handcrafted features
Random Forest	Comparable accuracy, intrinsic feature importance
k-NN	Distance-based baseline, lowest accuracy
CNN (log-Mel)	Does not surpass classical models on this dataset size

The most discriminative descriptors turned out to be the temporal derivatives of MFCCs (Δ, ΔΔ) and spectral contrast — capturing how the timbre evolves over time, which is essential for distinguishing impulsive, harmonic and continuous sound sources.

See report/ for the full technical report and figures/ for the headline plots.

---

3. Repository structure

.
├── APR_RINCON_POLANCO/
│   ├── code_apr_rincon_polanco.ipynb   # Main notebook (entry point)
│   ├── esc50.csv                       # ESC-50 metadata
│   ├── features_mfcc.npy               # Pre-computed MFCC features
│   ├── X_extended.npy                  # Extended feature matrix (MFCC + Δ + spectral + energy)
│   └── labels.npy / y_labels.npy       # Target arrays
├── report/
│   └── final_report.pdf                # Full technical report
├── figures/
│   ├── rf_top10_importances.png        # Top-10 Random Forest feature importances
│   └── tsne_true_vs_kmeans_k50.png     # t-SNE projection: ground truth vs k-means (k=50)
├── requirements.txt
└── README.md

The audio/ directory containing the 2,000 .wav files is not included (≈ 2 GB). Download it from the ESC-50 repository and place it inside APR_RINCON_POLANCO/audio/.

---

4. Reproducing the experiments

# 1. Clone
git clone https://github.com/qiuskye/audio-pattern-recognition-esc50.git
cd audio-pattern-recognition-esc50

# 2. Get the audio data (≈ 600 MB compressed)
curl -L https://github.com/karolpiczak/ESC-50/archive/master.zip -o esc50.zip
unzip esc50.zip "ESC-50-master/audio/*" -d /tmp/
mv /tmp/ESC-50-master/audio APR_RINCON_POLANCO/audio

# 3. Install dependencies
pip install -r requirements.txt

# 4. Run the notebook
jupyter notebook APR_RINCON_POLANCO/code_apr_rincon_polanco.ipynb

The pre-computed feature arrays (.npy) are included so the supervised-learning sections can be re-run without re-extracting features from raw audio.

---

5. Dataset

ESC-50: Dataset for Environmental Sound Classification — Karol J. Piczak, ACM Multimedia 2015. github.com/karolpiczak/ESC-50

---

6. Author

Ana Yang Rincón Polanco — Erasmus exchange student at UNIMI, BSc Data Science & AI @ Universidad Politécnica de Madrid.

GitHub @qiuskye