Banking Intent Classification with LoRA Fine-Tuned FLAN-T5

Parameter-efficient fine-tuning (LoRA/PEFT) of Google FLAN-T5-base on the Banking77 dataset. Classifies 77 customer service intents, trains 1.77M of 250M parameters (~0.71%) in ~2hrs on free Colab GPU. Deployed as a FastAPI REST API with batch prediction, Docker containerisation, and full evaluation metrics.

---

Table of Contents

• Project Overview
• Architecture
• LoRA Configuration
• Training Details
• Model Performance
• Project Structure
• Quick Start
• API Reference
• Docker Deployment
• Evaluation
• Technologies

---

Project Overview

This project demonstrates Parameter-Efficient Fine-Tuning (PEFT) using LoRA on a sequence-to-sequence model for multi-class intent classification. Instead of updating all 250 M parameters, LoRA injects small trainable rank-decomposition matrices into the attention layers — training only ~1.77 M parameters (0.71%) while retaining full expressiveness.

The system handles real-world banking queries such as:

Query	Predicted Intent
"My card was charged twice"	transaction_charged_twice
"I want to activate my card"	activate_my_card
"How do I transfer money abroad?"	transfer_into_account
"I can't remember my PIN"	passcode_forgotten
"Why is my balance not showing the deposit?"	balance_not_updated_after_bank_transfer

---

Architecture

Input Query
    │
    ▼
Prompt Template: "Classify this banking query: {text}\n\nCategory:"
    │
    ▼
FLAN-T5-base (250M params, frozen)
+ LoRA Adapters (1.77M trainable params on q,v attention layers)
    │
    ▼
Beam Search (4 beams, greedy decode)
    │
    ▼
Label Matching (exact → substring → token-overlap)
    │
    ▼
Predicted Intent (one of 77 Banking77 categories)

---

LoRA Configuration

Parameter	Value
Rank (r)	16
Alpha (lora_alpha)	32
Dropout	0.1
Target modules	q, v (query & value attention)
Task type	SEQ_2_SEQ_LM
Trainable parameters	1,769,472
Total parameters	249,867,264
% trainable	0.71%

---

Training Details

Setting	Value
Base model	google/flan-t5-base
Dataset	PolyAI/banking77
Train samples	10,003
Test samples	3,080
Epochs	10
Batch size	2 (effective 16 via gradient accumulation ×8)
Learning rate	3e-4
Optimizer	AdamW (weight decay 0.01)
LR scheduler	Linear with 100 warmup steps
Mixed precision	FP16
Logging steps	50
Output dir	./lora_adapter_improved

Training environment: Google Colab (T4 GPU, 15 GB VRAM)

---

Model Performance

Evaluated on the Banking77 test split (3,080 samples) using evaluate.py:

Metric	Score
Overall Accuracy	~82–85%
Macro F1	~0.82
Weighted F1	~0.83
Macro Precision	~0.84
Macro Recall	~0.82
Inference speed (CPU)	~150–200 ms / query
Inference speed (GPU)	~15–25 ms / query

Note: Exact numbers depend on the training run. Run python evaluate.py to reproduce metrics on your adapter weights. The training loss showed convergence, with occasional spikes due to FP16 on CPU — these stabilise on GPU.

Why Only ~0.71% of Parameters are Trained

LoRA decomposes weight updates into two small matrices A ∈ R^{r×d} and B ∈ R^{d×r}, where r=16. The effective delta ΔW = BA is added to the frozen weight at inference. This gives:

• 4× lower GPU memory vs full fine-tuning
• ~3× faster training
• Adapter file size < 10 MB (vs 1 GB for the full model)

---

Project Structure

bankbot-lora-flant5-fastapi/
├── app/
│   ├── __init__.py
│   └── main.py                    # FastAPI application
├── banking-classifier-api/        # Standalone API copy (for cloud deploy)
├── models/
│   └── lora_adapter_improved/     # Trained LoRA adapter weights
│       ├── adapter_config.json
│       ├── adapter_model.safetensors
│       ├── tokenizer.json
│       ├── tokenizer_config.json
│       ├── special_tokens_map.json
│       └── spiece.model
├── FineTuning_LoRA.ipynb          # Training notebook (run on Colab)
├── evaluate.py                    # Offline evaluation script
├── test_api.py                    # API integration tests
├── requirements.txt               # Python dependencies
├── Dockerfile                     # Container config
├── .dockerignore
├── index.html                     # Simple web UI
└── README.md

---

Quick Start

Prerequisites

• Python 3.9+
• CUDA-capable GPU (optional, but recommended for training/fast inference)

Installation

git clone https://github.com/aneessaheba/peft-banking-classifier.git
cd peft-banking-classifier
pip install -r requirements.txt

Run the API

uvicorn app.main:app --host 0.0.0.0 --port 8000 --reload

Open http://localhost:8000/docs for the interactive Swagger UI.

Quick Prediction

import requests
response = requests.post(
    "http://localhost:8000/predict",
    json={"text": "My card was charged twice"}
)
print(response.json())
# {'text': 'My card was charged twice', 'predicted_category': 'transaction_charged_twice', 'confidence': 'high'}

Training from Scratch

Open FineTuning_LoRA.ipynb in Google Colab (GPU runtime recommended) and run all cells sequentially:

1. Install dependencies
2. Check GPU availability
3. Load Banking77 dataset
4. Load FLAN-T5-base
5. Apply LoRA configuration
6. Preprocess & tokenize data
7. Train (10 epochs, ~30–40 min on T4)
8. Evaluate & save adapter

---

API Reference

GET /

Returns API metadata and available endpoints.

GET /health

{"status": "healthy", "model_loaded": true, "device": "cuda", "categories_count": 77}

GET /categories

Returns the full list of 77 Banking77 intent labels.

POST /predict

Classify a single query.

Request:

{"text": "I want to activate my card"}

Response:

{
  "text": "I want to activate my card",
  "predicted_category": "activate_my_card",
  "confidence": "high"
}

POST /predict/batch

Classify up to 100 queries in one call.

Request:

["My card was stolen", "How do I top up?", "Transfer money abroad"]

Response:

{
  "predictions": [
    {"text": "My card was stolen", "predicted_category": "lost_or_stolen_card"},
    {"text": "How do I top up?", "predicted_category": "topping_up_by_card"},
    {"text": "Transfer money abroad", "predicted_category": "transfer_into_account"}
  ],
  "count": 3,
  "success": true
}

---

Docker Deployment

# Build
docker build -t bankbot-classifier .

# Run
docker run -p 8000:8000 bankbot-classifier

API available at http://localhost:8000

---

Evaluation

Run the offline evaluation script against the Banking77 test set:

# Evaluate on 500 test samples (fast)
python evaluate.py --max-samples 500

# Full test set evaluation (3,080 samples)
python evaluate.py --max-samples 0

# Custom model path
python evaluate.py --model-path ./models/lora_adapter_improved --split test

Outputs:

• Console: accuracy, F1, precision, recall, per-class breakdown
• evaluation_results.json — full metrics as JSON
• confusion_matrix.png — 77×77 confusion matrix heatmap

Run API Integration Tests

# Start server first, then:
python test_api.py --base-url http://localhost:8000

---

Technologies

Component	Library / Tool
Base model	google/flan-t5-base (HuggingFace Transformers)
Fine-tuning	PEFT (LoRA)
Training	PyTorch + HuggingFace Trainer
Dataset	PolyAI/banking77 (HuggingFace Datasets)
API framework	FastAPI + Uvicorn
Validation	Pydantic v2
Containerisation	Docker
Evaluation	scikit-learn
Training environment	Google Colab (T4 GPU)

---

Author

Anees Saheba — aneessaheba.guddi@sjsu.edu San José State University