fix(kyc-fraud): Replace continent-heuristic geo anomaly detector with Haversine speed check + multi-hop sequence analysis - #210
Merged
Conversation
11 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #198
Summary
Rewrites
checkGeoAnomaly()insrc/services/kycFraud.service.tsto eliminate four compounding bugs that caused frequent false positives against legitimate humanitarian aid beneficiaries — particularly refugees and displaced persons whose country-of-submission changes regularly and rapidly. A Kenyan family that fled to Uganda and submits a KYC from a Ugandan refugee camp would previously be flagged withseverity: 'high'(cross-continent check) and, combined with any other signal, pushed above thehighRiskThreshold = 50, triggering aFRAUD_DETECTIONjob and potentially causing auto-rejection.The fix replaces the continent-membership heuristic with a physics-based impossible-travel detector: compute the great-circle distance between country centroids using the Haversine formula, derive implied speed, and compare against a configurable threshold (
config.kycFraud.geoMaxPlausibleSpeedKmh, default 900 km/h). The continent map is retained as a low-precision fallback for country codes that are absent from the centroid dataset, but its severity output is capped at'medium'to reflect reduced confidence.Problem
Bug 1 — Continent membership instead of physical distance
The detector compared continent strings rather than physical distance. Two countries on the same continent can be 8,000 km apart (Portugal and Russia are both
EUin the hand-rolled map), while two countries on different continents can be 100 km apart (Morocco/Spain, Panama/Colombia).config.kycFraud.geoMaxPlausibleSpeedKmh = 900was defined in config and parsed but never referenced anywhere in the code — it was clearly intended for a Haversine-based calculation that was never implemented.Bug 2 — Incomplete and incorrect continent map
buildContinentMap()had multiple errors:ASandEUarrays; last-write won (EU), silently.EUonly, despite spanning two continents.priorContinent = undefined, which short-circuited the cross-continent check entirely, leaving the detector blind for those submissions.Bug 3 — Only the most-recent prior submission was checked
Given a submission sequence A (Kenya, day 1) → B (UK, day 2) → C (Kenya, day 2, 2h after B), comparing only
B → Cshows a same-continent short-hop with no signal. But theB → Cleg alone (London to Nairobi in 2 hours, ~3,400 km/h) is physically impossible and should fire. The old code happened to miss it because both countries were "different continents" but the 2-hour rule was the only gate — and the logic fired on the wrong pair. More generally, multi-hop impossible-travel patterns (A → B → A round trips, document re-use across geographies) were invisible to the single-pair check.Bug 4 — Time delta used
Date.now()instead of submission timestampsThis measured time between the prior submission and when the assessment job ran, not between the prior submission and the current submission. If the
FRAUD_DETECTIONBullMQ worker was down for 6 hours and processed a backlog of submissions on restart, a genuinely suspicious 30-minute hop (prior at T, current at T+30min) would be evaluated as a ~6-hour gap and produce no signal.Solution
1. Bundled country centroid dataset (
data/country-centroids.json)A static JSON file mapping 250 ISO 3166-1 alpha-2 country codes to
{ lat: number, lng: number }centroids is loaded once at module initialisation:No external API call is made at assessment time. The full centroid lookup is an in-process
O(1)object property access. The file is 11 KB — well within the 500 KB repository budget constraint. Kosovo (XK), Taiwan (TW), and Palestine (PS) are all included.2. Haversine great-circle distance (no npm dependency)
9 lines of trigonometry, zero dependencies. Implements the standard Haversine formula with Earth radius 6,371 km.
3. Physics-based
checkCountryPair()helperA new pure function
checkCountryPair(fromCountry, toCountry, hoursDiff)encapsulates the detection logic for a single (from, to) hop:config.kycFraud.geoMaxPlausibleSpeedKmh. Returnsseverity: 'high'if speed exceeds threshold.severityat'medium'to reflect reduced precision.0.001h(~3.6 seconds) to avoidInfinityspeeds.hoursDiff < 0(out-of-order timestamps) are skipped silently.4. Multi-hop sequence analysis
The N most-recent prior submissions are fetched, reversed into chronological order, and appended with the current submission to form a full sequence. Every consecutive pair
(sequence[i], sequence[i+1])is checked. The loop returns on the first impossible-travel signal found (most-recent pair first), so the hottest anomaly is always surfaced.5. Correct time delta via
submittedAtThe
FraudInputinterface gains an optionalsubmittedAt?: Datefield. When supplied by the caller (recommended), the time delta is computed entirely from stored timestamps and is immune to job-queue delays. When omitted,new Date()is used as a safe default (backward-compatible). Callers should always supplysubmittedAtfrom the KYC submission row'screatedAt.6.
geoAnomalyLookbackconfig + env varThe number of prior submissions to examine is now configurable, defaulting to 5. Operators in high-velocity environments can lower this to reduce DB read load; operators investigating serial fraud can raise it.
7.
getModelVersions()refactor (shadow scoring housekeeping)As a related fix surfaced during the work,
getActiveModelVersion()was refactored intogetModelVersions(), which fetches both the active and shadow-candidate model versions in a singlefindManycall instead of two separatefindFirstcalls. The publicgetActiveModelVersion()wrapper is preserved for backward compatibility. A newapplyPlattScaling()helper extracts the Platt sigmoid into a reusable, testable function.Files Changed
data/country-centroids.jsonsrc/services/kycFraud.service.tscheckGeoAnomaly(), addshaversineKm(),checkCountryPair(),getModelVersions(),applyPlattScaling()src/config/index.tsgeoAnomalyLookbackconfig key.env.exampleGEO_ANOMALY_LOOKBACK=5tests/unit/kycFraud.service.test.tsfindManyAPITest Coverage
All 45 tests pass. The new
checkGeoAnomalytest suite is split into threedescribeblocks:Core behaviour (updated existing tests)
nullwhenclaimedCountryis not providednullwhen no prior submissions existnullwhen prior country matches current country (any time gap)highfor intercontinental impossible speed (US→AU in 30 min, ~28,600 km/h)nullfor plausible slow travel (US→CA in 5 hours, ~300 km/h)Acceptance criteria (new tests)
severity: 'high'severity: 'high''high'ZZ: no centroid, no continent → gracefulnullXK: centroid present → Haversine fires correctly (XK→GB in 30 min →'high')XX→XX): same-country short-circuit →nullsubmittedAt: prior fixed at 2020-01-01T12:00Z,submittedAtat +30min → fires'high'; withoutsubmittedAtthe 4-yearDate.now()delta would produce no signalsubmittedAtomitted: falls back toDate.now(), still correct when prior is genuinely 30 min agoPerformance
checkGeoAnomaly()calls with varied country pairs complete in < 50ms (centroid lookup is pure in-processO(1))Before / After: KE→UG Scenario
KE=AF,UG=AF→ same continent)severity: 'high'(speed > 900 km/h threshold)geoMaxPlausibleSpeedKmhdefined but never usedconfig.kycFraud.geoMaxPlausibleSpeedKmhseverity: 'high'Configuration Reference
KYC_FRAUD_GEO_MAX_SPEED_KMHconfig.kycFraud.geoMaxPlausibleSpeedKmh900GEO_ANOMALY_LOOKBACKconfig.kycFraud.geoAnomalyLookback5Out of Scope
claimedCountryfrom the beneficiary profile is used, not IP address)config.kycFraud.weights.geoAnomaly(still 20)checkDocumentReuse,checkVelocity, orcheckDeviceFingerprintChecklist
data/country-centroids.jsonadded (250 codes, 11 KB, < 500 KB limit)config.kycFraud.geoMaxPlausibleSpeedKmhis the only threshold — no hardcoded900inkycFraud.service.tsGEO_ANOMALY_LOOKBACKenv var documented in.env.exampleFraudSignalreturn type unchanged,'geoAnomaly'signal name unchanged