error_analysis_summary.md

Error Analysis Report

1. Executive Summary

• Test samples analyzed: 8,976
• Samples with at least one label error: 4,491 (50.03%)
• High-confidence wrong predictions (confidence > 0.90): 930
• Exact-match accuracy from prediction artifact: 0.4997

2. Overall Metrics

Metric	Value
Micro F1	0.5861
Macro F1	0.5208
Exact-match Accuracy	0.4997
Hamming Loss	0.0405

3. Best Performing Emotions

Emotion	Support	Precision	Recall	F1	Average Precision
gratitude	507	0.8486	0.929	0.887	0.9521
love	355	0.7139	0.8507	0.7763	0.831
amusement	421	0.6817	0.9002	0.7758	0.7741
admiration	670	0.6285	0.7776	0.6951	0.7513
remorse	105	0.71	0.6762	0.6927	0.6745

4. Worst Performing Emotions by F1

Emotion	Support	Precision	Recall	F1	Average Precision
realization	152	0.21	0.2763	0.2386	0.233
disappointment	185	0.2554	0.3189	0.2837	0.2636
annoyance	357	0.2938	0.3193	0.306	0.2676
excitement	110	0.2688	0.4545	0.3378	0.3452
approval	464	0.2797	0.5129	0.362	0.3762

5. Emotions with Lowest Recall

Emotion	Support	Recall	F1
realization	152	0.2763	0.2386
disappointment	185	0.3189	0.2837
annoyance	357	0.3193	0.306
excitement	110	0.4545	0.3378
confusion	230	0.4652	0.4534

6. Most Frequent Confusion Pairs

True emotion	Predicted emotion	Count
neutral	approval	321
neutral	disapproval	223
neutral	curiosity	221
approval	neutral	192
disapproval	neutral	168
annoyance	neutral	134
neutral	annoyance	131
curiosity	neutral	88
neutral	realization	87
neutral	admiration	82

7. Interpretation and Recommended Follow-up

• Low F1 indicates that the emotion is difficult to separate from semantically similar classes or is affected by class imbalance.
• Low recall indicates that the model frequently misses examples of that emotion; inspect false negatives and consider additional examples or threshold analysis.
• High-frequency confusion pairs should be reviewed together because they identify systematic semantic overlap between emotion classes.
• High-confidence wrong predictions should be manually annotated using manual_error_annotation.csv to distinguish semantic overlap, sarcasm, implicit emotion, ambiguity, rare-class effects, and annotation errors.

8. Generated Artifacts

• error_analysis_predictions.csv — sample-level probabilities and correctness
• worst_labels.csv — per-emotion performance ordered by F1
• confident_wrong_predictions.csv — confident errors for manual inspection
• confusion_pairs.csv — ranked true-to-predicted emotion pairs
• manual_error_annotation.csv — annotation template for human audit