🇮🇳 Aadhaar Data Analysis

Self-Normalized Anomaly Detection & Geographic Inequality

---

1. Problem & Approach

The Challenge: Detecting anomalies in the world's largest biometric database.

Global thresholds (e.g., "more than 1000 enrolments") fail because they cannot handle the massive scale differences between Indian states. A "normal" day in Uttar Pradesh looks like a massive spike in Goa.

💡 The Solution: Self-Normalized Z-Scores

We calculate a dynamic baseline ($\mu, \sigma$) for each state to identify "statistically surprising" events ($Z > 3$) equitably across villages and metros.

---

2. 🔍 Key Findings

📉 Insight 1: Access & Service Reach

"Store-and-Forward" Risks: We detected significant periods of inactivity followed by massive data spikes. This operational latency creates a risk surface for irregular bulk enrolments entering the system during "dark" periods.

⚠️ Insight 2: Service Load & Stress

Operational Stress: High enrolment volumes often correlate with a high Update Burden (ratio of updates to new enrolments). A high ratio indicates poor initial data quality, forcing residents to return for corrections.

🗺️ Insight 3: Geographic Inequality

The Silent Districts: We identified districts consistently in the bottom 20th percentile of activity. These areas often represent sustained Denial of Service rather than simple low demand.

🚨 Insight 4: Critical Anomalies

73 Critical Events: The model successfully flagged specific dates with Z-Scores $> 3$. The most extreme event had a deviation of 8.57, pinpointing massive irregularities.

---

3. 🧪 Methodology

Our fairness model relies on a simple yet powerful transformation:

$$ Z_{s,d} = \frac{X_{s,d} - \mu_s}{\sigma_s} $$

Where:

• $X_{s,d}$ = Enrolments in State $s$ on Day $d$
• $\mu_s$ = Historical Mean for State $s$
• $\sigma_s$ = Historical Standard Deviation for State $s$

---

4. 📂 Codebase

The analysis is powered by this repository:

File	Description
scripts/run_anomaly.py	🐍 Core Engine: Implements the Self-Normalized Z-Score logic.
notebooks/analysis_access_reach.ipynb	📓 Access Analysis: Quantifies delayed access and burstiness.
notebooks/analysis_geographic_inequality.ipynb	📓 Inequality Analysis: Identifies underserved districts.
notebooks/analysis_service_load.ipynb	📓 Stress Analysis: Operational stress and update burden calculations.

---