DisputeShield: Multimodal Chargeback & Payment Fraud Risk Scorer

A multimodal financial chargeback scoring and dispute triage engine that combines classical NLP text representations with tabular transaction velocity features, cost-sensitive gradient boosting, and isotonic probability calibration to minimize payment dispute losses.

---

1. System Architecture Diagram

       =======================================================================
       |                     DISPUTESHIELD ARCHITECTURE                      |
       =======================================================================

  +-------------------------------------+   +-------------------------------+
  | TABULAR PAYMENT & DISPUTE METADATA  |   | UNSTRUCTURED CLAIM TEXT / MEMO|
  | • Amount, IP Geodistance, Velocity  |   | • Customer Dispute Narration  |
  | • Card-Present Flag, User History   |   | • Merchant Response Descriptor|
  +-------------------------------------+   +-------------------------------+
                     │                                     │
                     ▼                                     ▼
  +-------------------------------------+   +-------------------------------+
  | TABULAR FEATURE PIPELINE            |   | CLASSICAL NLP PIPELINE        |
  | • Out-of-Fold Target Encoding (MCC) |   | • Regex & Token Normalization |
  | • Velocity Z-Scores & SQL Windows   |   | • Sublinear TF-IDF (1-2 Ngrams|
  | • Amount-to-History Ratios          |   | • Domain Fraud Lexicon Density|
  +-------------------------------------+   +-------------------------------+
                     │                                     │
                     └──────────────────┬──────────────────┘
                                        ▼
  +-------------------------------------------------------------------------+
  | HORIZONTAL SPARSE-DENSE MATRIX FUSION (scipy.sparse.hstack)             |
  | Combined Feature Vector: Scaled Numerical + Encoded Categorical + TFIDF |
  +-------------------------------------------------------------------------+
                                      │
                                      ▼
  +-------------------------------------------------------------------------+
  | COST-SENSITIVE GRADIENT BOOSTING & ISOTONIC PROBABILITY CALIBRATION     |
  | • LightGBM Classifier with Asymmetric Class Weights (scale_pos_weight)  |
  | • Isotonic Regression Calibration Layer (Minimizing Brier Loss Score)   |
  +-------------------------------------------------------------------------+
                                      │
                                      ▼
  +-------------------------------------------------------------------------+
  | EXPECTED VALUE / BAYES COST OPTIMIZATION MATRIX                         |
  | • False Negative (Chargeback Penalty + Loss) = $150                     |
  | • False Positive (Manual Investigator Review) = $15                     |
  | • Dynamic Cutoff Threshold Selection (Minimizing Total Dollar Loss)     |
  +-------------------------------------------------------------------------+
                                      │
                                      ▼
  +-------------------------------------------------------------------------+
  | FRAUD TRIAGE ROUTING & INTERPRETABILITY ENGINE                          |
  | • Automated Auto-Defend / Accept / Manual Review Routing               |
  | • Feature Attribution & Human-Readable Decision Codes                   |
  +-------------------------------------------------------------------------+

---

2. Directory Structure

project_2_disputeshield/
├── README.md               # Project documentation, architecture, results
├── config.py               # Hyperparameters and cost matrix parameters
├── run_pipeline.py         # Main execution pipeline entrypoint
├── data/
│   ├── __init__.py
│   └── generator.py        # Multimodal payment dispute dataset generator
├── features/
│   ├── __init__.py
│   ├── nlp.py              # Text cleaning & sublinear TF-IDF extraction
│   └── tabular.py          # OOF target encoding & sparse-dense fusion
├── models/
│   ├── __init__.py
│   └── classifier.py       # Calibrated LightGBM classifier
└── evaluation/
    ├── __init__.py
    ├── metrics.py          # ROC-AUC, PR-AUC, and Brier score evaluation
    └── cost_optimizer.py   # Bayes expected cost optimization matrix

---

Visual Diagnostics & Cost Optimization

Figure 1: Expected Value Cost Curve ($150 FN vs $15 FP) showing 54.1% Net Financial Loss Reduction

Figure 2: Probability Calibration Reliability (Left) and Multimodal ROC / Precision-Recall Curves (Right)

---

3. How to Run

Execute the end-to-end pipeline from the repository root:

python3 run_pipeline.py

---

4. Key Performance Metrics

Metric	Measured Value	Technical & Financial Meaning
ROC-AUC Score	0.860	Strong discrimination separating fraudulent claims from legitimate friendly disputes.
PR-AUC (Avg. Precision)	0.731	High precision maintained across the recall spectrum under severe class imbalance.
Brier Score Loss	0.131	High probability calibration accuracy after 3-Fold Isotonic Regression.
Optimal Bayes Cutoff	0.080 – 0.110	Shifted operating threshold reflecting 10:1 business cost asymmetry ($150 FN vs $15 FP).
Net Operational Savings	53.6%	Direct reduction in operational dollar loss relative to standard naive 0.50 cutoff.

---