Multi-Modal Classification Project

📊 Overview

This repository contains the evaluation results of a machine learning classification project that compares three different modeling approaches across 7 distinct classes. The objective of this project was to determine the most effective data modality for predicting the target classes by evaluating text data, numerical/categorical data, and a fusion of both.

🗄️ Dataset

The models were evaluated on a test dataset consisting of 20,107 samples distributed across 7 target classes (labeled 0 through 6).

• Class Imbalance: The dataset exhibits class imbalance, with Classes 0 and 1 making up the majority of the samples (approx. 68%), while Class 4 is the minority class.

🧠 Models Evaluated

1. Multi-modal Fusion Model: Combines text, numerical, and categorical features.
2. Text-only Model: Relies exclusively on textual features.
3. Numerical + Categorical Model: Utilizes only the structured data features.

---

📈 Training History & Visualizations

Based on the provided training logs, the models exhibited the following learning behaviors:

• Multi-modal Fusion Model: Trained over 10 epochs. The training accuracy steadily climbed from ~0.80 to ~0.88, while the validation accuracy stabilized around 0.86, indicating a well-fitted model without severe overfitting.
• Text-only Model: Evaluated over 5 epochs. The validation accuracy hovered between 0.855 and 0.860, closely tracking the training accuracy, resulting in strong final performance.
• Numerical + Categorical Model: Evaluated over 5 epochs. The model struggled significantly, with validation accuracy stagnating around 0.404 - 0.407, demonstrating an inability to capture the underlying patterns in the structured data alone.

---

🏆 Performance Evaluation

Below are the detailed classification reports for each model evaluated on the test set.

1. Multi-modal Fusion Model

Overall Accuracy: 86%

Class	Precision	Recall	F1-Score	Support
0	0.85	0.89	0.87	6905
1	0.94	0.92	0.93	6768
2	0.88	0.91	0.89	2236
3	0.70	0.64	0.67	2165
4	0.81	0.78	0.80	225
5	0.76	0.71	0.74	1461
6	0.73	0.76	0.75	347
Macro Avg	0.81	0.80	0.81	20107

2. Text-only Model

Overall Accuracy: 86%

Class	Precision	Recall	F1-Score	Support
0	0.88	0.85	0.86	6905
1	0.92	0.94	0.93	6768
2	0.89	0.90	0.90	2236
3	0.64	0.72	0.68	2165
4	0.79	0.83	0.81	225
5	0.79	0.69	0.74	1461
6	0.75	0.73	0.74	347
Macro Avg	0.81	0.81	0.81	20107

3. Numerical + Categorical Model

Overall Accuracy: 41%

Class	Precision	Recall	F1-Score	Support
0	0.39	0.56	0.46	6905
1	0.43	0.64	0.51	6768
2	0.00	0.00	0.00	2236
3	0.03	0.00	0.00	2165
4	0.00	0.00	0.00	225
5	0.00	0.00	0.00	1461
6	0.00	0.00	0.00	347
Macro Avg	0.12	0.17	0.14	20107

---

🔍 Key Insights & Conclusion

1. Textual Data is the Primary Driver: The Text-only Model achieves an impressive 86% accuracy, performing nearly identically to the Multi-modal Fusion Model. This indicates that the core predictive signal for these classes lies within the unstructured text data.
2. Structured Data is Insufficient on its Own: The Numerical + Categorical Model performs exceptionally poorly (41% accuracy). It completely fails to predict classes 2, 4, 5, and 6 (scoring 0.00 across Precision, Recall, and F1). It acts merely as a weak classifier guessing the majority classes.
3. Fusion Model Stability: While the Multi-modal Fusion Model does not significantly outperform the Text-only model in overall accuracy (both sit at 86%), it shows slight variations in class-level metrics. For instance, the Fusion model has slightly better Precision for Class 1 (0.94 vs 0.92) but slightly lower Recall for Class 3 (0.64 vs 0.72).

Recommendation: Given the comparable performance between the Fusion Model and the Text-only Model, deploying the Text-only Model may be the most efficient path forward if computational cost or pipeline simplicity is a priority, as the structured data provides negligible lift. However, if robustness is required, the Fusion model remains the strongest comprehensive approach.