A Transformer-based scam message detection system built with BERT and HuggingFace Transformers.
The model classifies SMS messages as Scam or Legitimate, provides confidence scores, and visualizes attention weights to improve interpretability.
Scam and phishing messages are increasingly common across SMS, email, and social media platforms.
These messages often use urgent or manipulative language to trick users into:
- clicking malicious links
- revealing sensitive information
- transferring money
This project builds an AI-powered Scam Detection Assistant using a pretrained BERT (Bidirectional Encoder Representations from Transformers) model fine-tuned for SMS scam classification.
The system:
- Detects scam messages
- Returns prediction confidence
- Visualizes attention weights
- Demonstrates practical Transformer-based NLP classification
- ✅ Scam vs Legitimate message classification
- ✅ Transformer-based NLP pipeline
- ✅ Fine-tuned BERT model
- ✅ Confidence score output
- ✅ Attention visualization for explainability
- ✅ Real-world SMS spam dataset
- ✅ High classification accuracy (99.28%)
This project uses:
- Model:
bert-base-uncased - Framework: HuggingFace Transformers
- Backend: PyTorch
- Architecture Type: Encoder-only Transformer
- WordPiece Tokenization
- Bidirectional Self-Attention
- CLS Token Classification
- Softmax Output Layer
Unlike decoder-only models such as GPT, BERT reads the full sentence bidirectionally, making it highly effective for text classification tasks.
Input Message
↓
WordPiece Tokenization
↓
BERT Encoder (12 layers)
↓
[CLS] Token Representation
↓
Linear Classification Head
↓
Softmax Probabilities
↓
Scam / Legitimate Prediction
- Source: UCI ML Repository / Kaggle
- Total Messages: 5,574
- Legitimate: 4,827
- Scam: 747
Dataset Link: https://www.kaggle.com/datasets/uciml/sms-spam-collection-dataset
| Parameter | Value |
|---|---|
| Epochs | 3 |
| Batch Size | 16 |
| Learning Rate | 2e-5 |
| Optimizer | AdamW |
| Max Sequence Length | 128 |
| Loss Function | CrossEntropyLoss |
| Metric | Score |
|---|---|
| Accuracy | 99.28% |
| Precision | 96.49% |
| Recall | 98.21% |
| F1 Score | 97.35% |
| Predicted Legit | Predicted Scam | |
|---|---|---|
| Actual Legit | 721 | 4 |
| Actual Scam | 2 | 110 |
The model successfully detected:
- 110 out of 112 scam messages
- while minimizing false positives
| Message | Prediction | Confidence |
|---|---|---|
| "Congratulations! You've won a $1000 Walmart gift card!" | SCAM | 100.0% |
| "Are you free for lunch tomorrow?" | LEGITIMATE | 98.9% |
| "Verify your account now!" | SCAM | 99.9% |
The project includes attention visualization to improve explainability.
The model focuses strongly on scam-related tokens such as:
- "FREE"
- "WINNER"
- "prize"
- "900"
This helps users understand why the model classified a message as suspicious.
- Python
- PyTorch
- HuggingFace Transformers
- Scikit-learn
- Pandas
- NumPy
- Matplotlib
- Seaborn
- Jupyter Notebook
Through this project, I learned:
- how Transformer architectures work
- the difference between encoder-only and decoder-only models
- fine-tuning pretrained LLMs
- attention mechanisms in NLP
- model evaluation using F1-score and confusion matrices
- explainable AI techniques using attention visualization
Current limitations:
- English-only support
- SMS-focused dataset
- No sender metadata analysis
- No multilingual scam detection
Future improvements:
- Multilingual scam detection
- Real-time API deployment
- Web application interface
- Hybrid detection using metadata + NLP
-
BERT Paper: Devlin et al. (2019) — BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
-
Attention Is All You Need: Vaswani et al. (2017)
-
HuggingFace Transformers: https://huggingface.co/transformers