Skip to content

Repository files navigation

Taintrace — sanctions screening over a Bitcoin contamination graph

A platform that screens Bitcoin transactions against international sanctions lists.

Status: phases 1 to 4 delivered. The lists are ingested and refreshed daily; the Bitcoin feed is consumed live and screened in real time; taint propagates through the graph over n hops; and a web interface makes the whole thing usable — http://localhost:8090. This document describes only what works today.


1. The problem, without jargon

When a country sanctions a person or a company, it bars them from the financial system: no more bank account, no more transfers. For this, banks have official lists of names, which they check before every operation.

Cryptocurrencies complicate this exercise. A bitcoin payment does not go from one name to another, but from one address to another — a string of characters such as 12QtD5BFwRsdNsAZY76UVE1xyCGNTojH9h that nobody signs. So the authorities also publish lists of addresses: the US Treasury currently lists 522 for Bitcoin alone. Checking that a payment is not going to one of them is simple.

The real problem lies elsewhere. The money does not stop at the first address: it is passed on to a second one, which passes it on to a third. After three hops, a perfectly honest company can receive funds that came from a sanctioned address without any way of knowing it. This is called taint: finding, step by step, the indirect beneficiaries. It is the business of companies such as Chainalysis — except that here, all the data is public and free.

This project builds that chain, in stages:

Phase What it brings Mental picture
1 Know reliably, and up to date, which addresses are sanctioned have the list
2 Watch transactions go by and alert on those that touch the list guard the door
3 Walk back up the chain of transfers to find the indirect beneficiaries follow the money

All three are delivered. Here, in one picture, is what phase 3 produces — a real chain, measured on 30/07/2026:

1Kuf2Rd8mDyAViwBozGTNYnvWL8uYFrkVo   [Xiaobing YAN, SDNTK program]      100.00 %
        │  transaction 740d93d8…
        ▼
1CHc8vfaE27QdVB57kniJh4oU4erieVrK8                                     100.00 %
        │  transaction 5a78ae09…
        ▼
1AeFq5RbXiY1vsRqZjcF7fVodCxXwmDcMX                                      11.31 %
        │  transaction 6bafcb3f…
        ▼
1M4wAEiCSDuQLAyqYSn3JwGev41gN7Q4Et            512.25 BTC received       11.07 %

The last address is not on any sanctions list. Yet it received 512 bitcoins, 11% of which trace back to an entity frozen by the US Treasury, three transactions away. Without a graph and a propagation rule, it is invisible.


2. Getting started

What you need

Docker Desktop, about 5 GB of free disk space, and an Internet connection (the refresh downloads 380 MB per day).

Disk footprint measured after a full run:

Airflow image (the official one + 3 dependencies) 2.17 GB
MongoDB image 1.19 GB
Kafka image 0.63 GB
PostgreSQL image 0.42 GB
Neo4j image 0.60 GB
Streaming services image 0.24 GB
MongoDB data (bronze + silver + alerts + graph) 318 MB
Airflow metadata 68 MB
Total ≈ 5.6 GB

Memory actually used by the six containers, in steady state:

Service Memory Limit set
airflow-webserver 838 MB 2 GB
airflow-scheduler 581 MB 3 GB
kafka 446 MB 1 GB
mongo 96 MB 1 GB
postgres 53 MB 512 MB
criblage (screening) 21 MB 512 MB
producteur (producer) 16 MB 512 MB

The command

docker compose up -d

That is all. No configuration, no file to copy, no task to enable by hand.

Durations measured on 30/07/2026, starting from scratch (docker compose down -v then deleting the image):

Step Duration
docker compose up -d returns 62 s
The 4 services become healthy ~3 min
The DAG triggers on its own and finishes ~2 more min

On the very first launch on a new machine, add the download time of the official Airflow image (1.5 GB).

Service Address Credentials
The interface http://localhost:8090 none
Airflow UI http://localhost:8081 admin / admin
Neo4j graph http://localhost:7474 neo4j / sanctions-radar-local
MongoDB localhost:27018 no authentication
Kafka internal to the Docker network not exposed

Nine services start: mongo, postgres, kafka, neo4j, airflow-scheduler, airflow-webserver, producteur, criblage and interface. The producer connects to the Bitcoin feed as soon as it starts — no action is required.

Open http://localhost:8090. That is how the project is shown. The rest — Airflow, Neo4j — is for operations, not for the demo.

Ports deliberately shifted. 27017 and 8080 are often already taken (native MongoDB, Tomcat, Oracle listener). To change them, copy .env.example to .env.

Security. The two services listen only on 127.0.0.1: nothing is reachable from the network. That is what makes it possible to keep no secrets in the repository — see DECISIONS.md § D11.

Replaying the demonstrations

Phase 1 — the lists, and the absence of duplicates:

docker compose exec airflow-scheduler python /opt/airflow/scripts/demo_phase1.py --idempotence

The script runs the full pipeline, shows what actually lands in the database, then replays the ingestion a second time to prove that no duplicate is created.

Phase 2 — watch an alert fire:

docker compose exec producteur python /app/scripts/demo_phase2.py --reinitialiser --rejeu

Why --reinitialiser. The replay always republishes the same historical transactions, and alert writing is idempotent: on the second pass, nothing is recreated and the screen does not move. --reinitialiser (reset) clears the demo alerts and cases so you start from zero. The screening list, the graph and the metrics are not touched. See DECISIONS.md § D41.

The script publishes to Kafka real transactions from addresses actually sanctioned by OFAC, retrieved from the blockchain, and shows the alerts that the screening produces. See section 6 for what this demonstration proves, and what it does not.

To observe the state without injecting anything: add --etat.

Phase 3 — follow the money over n hops:

docker compose exec criblage python /app/scripts/demo_phase3.py --sauts 3

The script builds the graph from the sanctioned addresses, propagates the taint, and shows the indirect beneficiaries with the full chain of transactions linking them to the sanctioned entity. Allow ~135 seconds on the first pass, a few seconds afterwards (the cache).

The depth is free: --sauts 4, --sauts 5. So are the cut-off threshold and the number of addresses followed per hop (--seuil, --plafond).

Running the tests

docker compose exec airflow-scheduler python -m pytest /opt/airflow/tests -q

Or on the host, without Docker: python -m pytest tests -q.


3. The flow, end to end

   US Treasury                             OpenSanctions
   sdn.xml — 28.8 MB                       entities.ftm.json — 350 MB
          │                                        │
          │  download with resume on disconnect (HTTP Range header)
          │  nothing is written to disk: read, filter, forget
          ▼                                        ▼
   ┌──────────────────────────────────────────────────────────┐
   │  BRONZE — the data as the source published it             │
   │  bronze_ofac  ·  bronze_opensanctions   (7 days kept)     │
   └──────────────────────────────────────────────────────────┘
          │
          │  address format recognition + checksum
          │  deduplication on the key (chain, address)
          ▼
   ┌──────────────────────────────────────────────────────────┐
   │  SILVER — the screening list                              │
   │  silver_sanctioned_wallets                                │
   │  one row per address, whatever the sources                │
   └──────────────────────────────────────────────────────────┘
          │
          ▼
   in-memory Python set → constant-time membership test
          │
          │  All driven by Airflow, every day at 03:00 UTC.
          │
══════════╪══════════════════ PHASE 2 ═══════════════════════════════════
          │
   blockchain.info (WebSocket)          mempool.space (history replay)
   unconfirmed transactions             already confirmed transactions
   2 to 3.6 per second                  from real OFAC addresses
          │                                        │
          │  automatic reconnection                │
          │  (doubling wait: 2s, 4s, 8s…)          │
          ▼                                        ▼
   ┌──────────────────────────────────────────────────────────┐
   │  KAFKA — topic `tx.raw`, 1 partition, 6 h retention       │
   │  envelope: { recu_a, origine, statut_confirmation, tx }   │
   └──────────────────────────────────────────────────────────┘
          │
          │  the producer writes even if the consumer is dead
          │  the consumer resumes at its offset: nothing is lost
          ▼
   ┌──────────────────────────────────────────────────────────┐
   │  SCREENING — `address in set` in constant time            │
   │  list reloaded every 300 s (Airflow updates it)           │
   └──────────────────────────────────────────────────────────┘
          │
          ├──────────────► alertes        (one row per address hit)
          ├──────────────► tx_retenues    (raw material for phase 3)
          └──────────────► metriques_flux (throughput, latencies, every 30 s)

══════════════════════════ PHASE 3 ═══════════════════════════════════════

   sanctioned addresses (the seeds)
          │
          │  exploration guided by tainted value,
          │  bounded per hop, cached in MongoDB
          ▼
   blockstream.info → btcscan.org → mempool.space → blockchain.info
          │                          (automatic failover)
          ▼
   ┌──────────────────────────────────────────────────────────┐
   │  BIPARTITE GRAPH                                         │
   │  (:Adresse)-[:ALIMENTE]->(:Transaction)-[:CREDITE]->(:Adresse)
   └──────────────────────────────────────────────────────────┘
          │
          │  "haircut" propagation: share = Σ(vᵢ×pᵢ) / Σvᵢ
          │  cut-off threshold, maximum over paths
          ▼
   ┌──────────────────────────────────────────────────────────┐
   │  Neo4j: proof paths + visualisation :7474                 │
   │  MongoDB `contaminations`: the ranking of beneficiaries   │
   └──────────────────────────────────────────────────────────┘

   Driven by Airflow, every Monday at 04:00 UTC.

Why two layers

Bronze keeps what the source published, dated by day. If we discover tomorrow that the parser misses an address format, we fix the parser and rebuild the list from bronze — without re-downloading 380 MB, and without having lost what the source said on that day. Sanctions lists change; history cannot be recovered.

Silver is the usable list: one row per (chain, address) pair, no matter how many sources cite it.


4. What was measured

All values in this section were recorded on 30/07/2026 on the target machine (Windows, 23 GB of memory, Docker Desktop). None is estimated. To reproduce them: python scripts/mesure_sources.py.

The two sources

OFAC SDN OpenSanctions sanctions
Format XML JSON, one entity per line
Volume transferred 28.77 MB 350.37 MB
Read and processing time 4.2 s 58.3 s
Entities scanned 19,175 291,419
Of which carry addresses 91 2,706
Wallets retained 957 1,577
Rejected (bad checksum or unreadable format) 2 139

The resulting screening list

Total wallets 1,577
Of which Bitcoin 568
Cited by both sources 957
Contributed by OpenSanctions only 620
Contributed by OFAC only 0
Accepted without being verifiable 13
Source / format inconsistencies flagged 1

"Why two sources, when OFAC adds none?"

The question is fair: OpenSanctions contains 100% of the addresses published by the US Treasury. The 620 additional addresses come from elsewhere — 614 from the Israeli Ministry of Defense (il_mod_crypto), 6 from the French Treasury (fr_tresor_gels_avoir).

We still keep reading OFAC directly, for two reasons. Freshness: OpenSanctions harvests its sources periodically, the Treasury publishes live. Outage detection: if the OpenSanctions harvester breaks, we would lose 957 addresses without knowing it. Reading OFAC separately gives a number to compare against — and a sanctions screening system should not depend on a single aggregator for the very thing that justifies its existence.

The network is not reliable, and that is measured

Observation Value
Observed throughput to OpenSanctions 5 to 11 MB/s
Disconnection on a 350 MB download 1 time out of 4 attempts
Position of the disconnection 252.7 MB
Resume via Range header accepted by the server yes (206 Partial Content)
Duration with resume 752 s instead of a failure

It is this measurement, and not a precaution on principle, that justifies the resume mechanism described in DECISIONS.md § D5.

No duplicates, verified

Second run of the pipeline on the same day, immediately after the first:

OFAC          957 portefeuilles -> {'matched': 957, 'upserted': 0, 'modified': 957}
OpenSanctions 1577 portefeuilles -> {'matched': 1577, 'upserted': 0, 'modified': 1577}

avant : 1577 portefeuilles
apres : 1577 portefeuilles
-> AUCUN DOUBLON

upserted: 0 on the second pass: no row was created twice. Reproduce it with the --idempotence option of the demo script.

The screening itself

The 568 Bitcoin addresses fit in a Python set of a few tens of kilobytes. Measured: 1,000,000 membership tests in 0.056 s, i.e. 17.9 million per second. No database is queried per transaction — this is what will make screening the real-time feed possible without heavy infrastructure.

Duration of a full run

Step Duration
OFAC ingestion 4.2 s
OpenSanctions ingestion 58.3 s
Full pipeline, two passes (--idempotence demo) 316.7 s

The DAG, actually executed

These five tasks are not a screenshot of intent: this is the state of a run on 30/07/2026, triggered by the scheduler.

task_id                state    start_date                  end_date
preparer_base          success  2026-07-30T09:19:08+00:00   2026-07-30T09:19:11+00:00
ingerer_opensanctions  success  2026-07-30T09:19:13+00:00   2026-07-30T09:20:11+00:00
ingerer_ofac           success  2026-07-30T09:52:21+00:00   2026-07-30T09:52:50+00:00
purger_bronze          success  2026-07-30T09:52:51+00:00   2026-07-30T09:52:53+00:00
consigner              success  2026-07-30T09:52:55+00:00   2026-07-30T09:52:57+00:00

And the report that the consigner task left in MongoDB:

{ "total_wallets": 1577,
  "by_chain": { "BTC": 568, "TRX": 491, "USDT": 342, "ETH": 109, "LTC": 17,
                "DOGE": 10, "XMR": 9, "BCH": 7, "DASH": 5, "ZEC": 5, "…": 1 },
  "multi_source": 957, "with_disagreement": 1, "unvalidated": 13 }

This is the document to consult when wondering why the list has not changed: it contains, for each source, the bytes read, the rejects, the network disconnections and the duration.


5. Real time, measured

What the Bitcoin feed actually delivers

Recorded over 838 transactions from the blockchain.info feed, on 30/07/2026.

Measure Value
Throughput 2.0 to 3.6 transactions/s
Average message size 1,654 bytes
Largest size observed 169,804 bytes
Median gap between two transactions 511 ms
95th percentile gap 1,029 ms
Input addresses decoded 371 / 371 — 100%
Output addresses decoded 374 / 729 — 51.3%

The 48.7% of outputs without an address are not a defect. Checked on 256 cases: they are all OP_RETURN outputs, carrying zero satoshi. They carry data and cannot receive any funds by construction. Coverage of the outputs that carry value is therefore complete.

Performance of the full chain

Latency measured between the moment the producer receives the message and the moment the screening returns its verdict — two timestamps taken on the same machine.

Measure Value
Sustained throughput in steady state 3.4 to 4.6 tx/s
Throughput while catching up 19 to 54 tx/s
Median latency 57 ms
p95 latency (last 60 seconds) 73 to 75 ms
Unreadable messages 0

Two p95 values are published, and it is not an affectation. After an incident, the messages caught up show a latency of 80 to 120 s — that is accurate, they really did wait. But those values stay in the 5,000-sample window for sixteen minutes. latence_p95_ms shows that an incident happened; latence_p95_recente_ms shows the current state. Confusing the two would make the system look slow when it processes in 57 ms.

Test no. 1 — complete network cut on the producer

The container is disconnected from the Docker network for 75 seconds.

13:22:05  flux interrompu (Connection to remote host was lost) — reconnexion dans 2 s  [n°1]
13:22:07  flux interrompu (Temporary failure in name resolution) — reconnexion dans 4 s [n°2]
13:22:11  flux interrompu (Temporary failure in name resolution) — reconnexion dans 8 s [n°3]
13:22:19  flux interrompu (Temporary failure in name resolution) — reconnexion dans 16 s [n°4]
          ── réseau rétabli ──
13:22:36  connecte a wss://ws.blockchain.info/inv, abonnement aux transactions non confirmees
13:22:42  678 transactions publiees | 2.43 tx/s | 4 reconnexion(s)
13:23:43  967 transactions publiees | 2.84 tx/s | 4 reconnexion(s)

Reconnected 13 seconds after the network came back. The process never died: restart: unless-stopped did not have to step in; the code held.

Test no. 2 — consumer outage, producer kept running

The criblage service is stopped for 90 seconds while the producer keeps writing to Kafka.

GROUP     TOPIC    PARTITION  CURRENT-OFFSET  LOG-END-OFFSET  LAG
criblage  tx.raw   0          7407            7407            0

On restart, the consumer processes 574 transactions at 19 tx/s — six times the feed's throughput — then falls back to 3.3 tx/s once caught up. LAG 0: no transaction lost.

It is this test, and nothing else, that justifies Kafka in this project. With an in-memory queue, those 90 seconds of transactions would have vanished.

Alerts: facts, and cases

The defect that was fixed. First version: one alert row per address hit. Measurement: 49 alerts for 16 transactions, one transaction alone generating 57. An analyst who sees 57 rows for a single transfer stops looking at the tool.

The two-level model:

Unit Key Answers
Alert a transaction txid what happened?
Case an entity holder's name what should I work on?

Verified on the same dataset: 49 → 16 alerts → 1 case.

gravite    entite            alertes   valeur exposee              statut
--------------------------------------------------------------------------
critique   Xiaobing YAN           16   287.64968477 BTC            nouvelle

The four severity levels:

Level Condition Expected response
CRITIQUE an address from the official list is involved immediate block, legal obligation
HAUTE exposed value ≥ 1 BTC handled within the day
MOYENNE exposed value ≥ 0.01 BTC handled within the week
BASSE below that batch review

CRITIQUE (critical) deliberately ignores the amount: 600 satoshis to an address published by an authority trigger a legal obligation. It is also what separates the 8 official addresses from the 26 inferred through co-spending — same risk, but only the former trigger CRITIQUE.

Notification. Before, an alert was a log line: if nobody looks, nobody knows. Now every case above the threshold goes out on two channels:

{"horodatage": "2026-07-31T01:48:41Z", "dossier": "entite:xiaobing-yan",
 "gravite": "critique", "entite": "Xiaobing YAN", "alertes": 16,
 "valeur_exposee_btc": 287.64968477, "statut": "nouvelle", "adresses": [...]}
Channel State Why
dossiers.jsonl file active depends on no service, survives restarts, any tool can follow a file
HTTP hook disabled by default provided without being imposed — no external service is required

To view it:

docker compose exec criblage tail -5 /var/alertes/dossiers.jsonl

An alert, actually fired

24 transactions replayed → 49 alerts in 3.0 seconds. Excerpt:

[6] sortie de fonds sanctionnes
    adresse    : 1EpMiZkQVekM5ij12nMiEwttFPcDK9XhX6  (emetteur)
    detenteur  : Xiaobing YAN
    montant    : 128 000 000 sat (1.28000000 BTC)
    transaction: c326d6af290eb42c6e690d6d57b6c60322ae6bd3910e4d4ebd48f336e485d880
    origine    : rejeu   statut : confirmee
    latence de detection : 107 ms

This transaction c326d6af… touches three sanctioned addresses at once — 1.28 + 0.83 + 0.45 BTC. It is a consolidation in which the entity swept several of its wallets. It was not fabricated: it is in the blockchain, and the screening found it on its own.

Test no. 3 — the history source goes down, not simulated

It happened during the final verification, which makes it the best possible proof. mempool.space stopped responding — timeout at 30 s — while the Bitcoin feed was running normally (5.33 tx/s, no reconnection) and blockchain.info was answering in 1 s.

historique indisponible via mempool.space pour 12QtD5BF… (handshake operation timed out)
rejeu de 12QtD5BF… (Xiaobing YAN) : 47 transactions disponibles via blockchain.info

49 alerts — exactly the same result as with the primary source. The replay tries two sources and gives up after three consecutive failures with a message naming the cause, instead of going through 72 addresses at a 45 s timeout each (54 minutes of silent blocking in the first version).

What the demonstration proves, and what it does not

It proves that the chain works end to end. It does not claim that these transactions just happened.

Sanctioned addresses are frozen, seized or abandoned — which is exactly the effect a sanction seeks. Out of the ~400,000 daily Bitcoin transactions, the probability that one of the 568 monitored addresses appears during a thirty-minute defense is close to zero. A platform you cannot see alert proves nothing.

So we replay real confirmed transactions, retrieved via mempool.space. Every alert coming from the replay carries origine: "rejeu" in the database: it cannot be confused with a live detection. The script's counter explicitly distinguishes the two.


6. Taint — how the money is followed

The problem in one picture

A sanctioned address sends 1 bitcoin. The beneficiary mixes it with 9 perfectly clean bitcoins, then passes the whole lot on. Did the third party receive sanctioned money? Yes — but one tenth.

The whole difficulty lies in that number. Three doctrines compete:

Rule Principle Result here Verdict
Poison one dirty input ⇒ all outputs dirty 100% manufactures dirty value
Haircut each output inherits the dirty proportion 10% chosen
FIFO first in, first out (Clayton's case) 0% or 100% requires an ordering the blockchain does not provide

Haircut is chosen for a verifiable, tested reason: it is the only rule that conserves dirty value. One dirty bitcoin goes in, one dirty bitcoin comes out. Poison would manufacture ten.

part_sortante  =  ( Σ valeur_entrée × part_entrée )  /  ( Σ valeur_entrée )

(outgoing share = Σ(input value × input share) / Σ(input value))

All outputs inherit the same share, whatever their size: this is the direct consequence of the bipartite model — the pot is shared, nobody can say which input funded which output.

Why a bipartite graph

The temptation is to link sender to beneficiary: (Alice) --2 BTC--> (Bob). That is wrong. A Bitcoin transaction takes n inputs and produces m outputs, and nothing in the protocol says which funded which.

Alice   1.0 BTC ─┐                 ┌─> Bob     1.4 BTC
                 ├─> [ tx abc ] ───┤
Carol   1.0 BTC ─┘                 └─> Diane   0.6 BTC

So the transaction is kept as a node in its own right: (:Adresse)-[:ALIMENTE]->(:Transaction)-[:CREDITE]->(:Adresse).

Two choices to own

The maximum over paths, not the average. An address reached by two paths, one at 40% and the other at 3%, is retained at 40%. An analyst wants the strongest link, not an average that would dilute it.

A cut-off threshold, 0.1% by default. Without it, taint spreads to the entire blockchain with infinitesimal shares. An address at 0.0001% is not a lead, it is noise — and following it costs hours of network calls.

The cost, measured — and the workaround

Measurement of 30/07/2026 Value
Branching factor (beneficiaries per address) 9.5
Sustainable API throughput (latency, not quota) 0.26 call/s
Blind exploration at 3 hops ~2,888 addresses — 3.1 hours

A breadth-first exploration is therefore out of reach for a demonstration. And pointless: most of those 2,888 addresses receive only a few thousand tainted satoshis.

The workaround: explore in the order an investigator works. At each hop, the taint is propagated over what is already known, then only the addresses carrying the most tainted value are followed. This is not an optimisation bolted on afterwards; it is the business question itself. And what is dropped at the cap is always what carries the least dirty money.

Measured result

Build of 30/07/2026 at 22:10 UTC, triggered by Airflow alone — with no intervention. Production parameters: 8 starting addresses, 3 hops, a cap of 60 addresses followed per hop, threshold 0.1%.

Total duration 115 s
Network calls 9
Cache reads 107
Transactions collected 861
Addresses in the graph 14,100
Edges 21,634
Tainted addresses 1,391

Nine network calls for 861 transactions: that is the cache at work. Without it, the same build would cost ~6 minutes of network time on every run.

Distribution by distance from a sanctioned address:

Distance Addresses
0 hops — sanctioned (start) 8
1 hop 50
2 hops 289
3 hops 1,044

The top five indirect beneficiaries, by tainted value received:

# Address Share Tainted value Hop
1 1KGf2X44fTz3oXhEn3C9e8YYNsPQQoMsfW 100.00% 913.43 BTC 3
2 1M4wAEiCSDuQLAyqYSn3JwGev41gN7Q4Et 11.07% 512.25 BTC 3
3 1La5BmmoKNntspBodPf6SP8Hjam4vdWaTD 35.13% 361.17 BTC 1
4 16P21ZWgj6ijqviPeHHnLW5vhe8q77yfwG 100.00% 300.00 BTC 1
5 1MJuULqkfMotsyB3xhgHJGUe6NYcAXmMKQ 35.13% 277.53 BTC 2

The first row deserves a pause: 100% taint at three hops. It means those 913 bitcoins went through three transactions without ever being mixed with clean funds. This is not progressive dilution, it is a direct transfer chain — exactly the pattern an analyst is looking for.

The traversal is bounded, not exhaustive, and the program says so: 1,056 addresses remain in the unexplored frontier. Claiming exhaustiveness would be false — one more hop would reveal more.

Following the money is not enough — co-spending

The propagation described above follows the money. It therefore misses the addresses that a sanctioned entity owns but to which it never sent anything directly.

The principle. To spend a bitcoin, you must sign with the private key that controls it. If a transaction consumes funds from two addresses, both keys were in the same wallet. This is not a statistical correlation, it is a cryptographic constraint.

What it corrected. Measurement of 31/07/2026: 26 addresses co-signed with an OFAC address, including four that the system classified at 0% taint.

Address Before After Product verdict
1NNX32J8prJSai3PHV3aiWs4mW9GPfNqcF 0.00% 100% NOTHING FOUND → ALERT
1QJpBDssSJcgyLUwbeiwidSb8wqA3Xa4cW 0.00% 100% NOTHING FOUND → ALERT
13rdxcGzgvBjZKTP2j1maALK6nkyK22Dk3 0.00% 100% NOTHING FOUND → ALERT
1EMgX4EjinCJAJvRFBSUFtixrtaCN8fZFN 0.00% 100% NOTHING FOUND → ALERT

All of them belong to Xiaobing YAN. They were false negatives — the worst possible defect for a compliance tool, and it was the system itself that revealed them.

The co-spending trap: CoinJoin

There is one case in which this heuristic gives a wrong answer. In a CoinJoin, strangers deliberately pool their funds to muddy the trail — each signs their own input; the keys are not in the same wallet.

Applying clustering to a CoinJoin would amount to declaring that dozens of strangers form a single entity, and tainting innocent people.

They are detected by their signature — at least 3 inputs and at least 3 outputs of exactly the same amount, which never happens by chance — and excluded. 8 CoinJoins excluded in the latest build.

The threshold is 3 equal outputs, not 2: a payment and its change can land on the same amount. Better to wrongly exclude an ordinary transaction than to wrongly merge strangers.

What the system distinguishes — and never confuses

Count Nature
Sanctionnee 8 published by an authority — a fact
MemeEntite 26 inferred through co-spending — an inference
FortementContaminee 896 propagation result
Contaminee 487 propagation result

The risk level of the 26 is the same as that of the 8. The claim is not. The system must never make OFAC say what it did not write.

The exact role of Neo4j

Neo4j does not compute the taint: it is a value-weighted aggregation, which is awkward to write in Cypher and hard to test. It remains a pure Python function, covered by 28 tests.

Neo4j brings three things no other component provides:

Contribution Why it matters
Variable-length path queries -[*1..6]-> produces the proof: where the money went, not just a percentage
Graph persistence exploration costs minutes, replay is instant
Visualisation at http://localhost:7474 showing a chain is better than describing it

7. The interface

http://localhost:8090 — four screens, nothing to install.

Screen For whom What it shows
Verification the merchant paste an address, get a verdict and the path of the money
Cases the compliance analyst the work queue, alerts arriving live
Graph everyone the propagation, as a radial layout or in 3D
Metrics the jury throughput and latency, measured continuously

What it weighs

Python dependencies added 0 — http.server from the standard library
npm packages 0 — no build step
Additional Docker image 0 MB — reuses the streaming image
First load 49.6 KB (HTML + CSS + JS)
Three.js, loaded on click on "3D Volume" 675.4 KB
Outgoing network requests 0, enforced by a Content-Security-Policy header

The fonts are the system ones — Segoe UI and Consolas on Windows. None is downloaded: the site works offline, which matters on the day of a defense in a room with unreliable Wi-Fi.

The demo walkthrough — 10 minutes

① The problem (1 min) — Verification screen, click on "a sanctioned address".

Verdict REFUSAL in 15 ms. "This address is on the US Treasury list. That is the easy case: any tool detects it."

② The real problem (2 min) — click on "913 BTC tainted at 100%".

Verdict ALERT, 100%, 3 transfers, answered in 11 ms. The full chain is displayed: Xiaobing YAN [OFAC LIST] → 3 transactions → the address checked.

"This address is on no list. No bank would refuse it. And yet 913 bitcoins it holds come from a frozen entity, in three transfers and with no mixing at all."

Click a blockstream link: the proof is public and verifiable.

③ The false negative corrected (1 min) — click on "revealed by co-spending".

"This address was classified as clean by my first version. It co-signed a transaction with a sanctioned address — so the same person holds both keys. My system revealed its own flaw."

④ Real time (2 min) — Cases tab.

16 alerts grouped into 1 case, critical severity, 287.65 BTC exposed. "One alert per transaction, not per address: otherwise a single transfer generated 57." Change a case's status: the lifecycle is tracked.

Leave it running: alerts appear without reloading.

⑤ The scale (2 min) — Graph tab.

Radial layout: one ring per hop, the 200 most tainted addresses out of 1,404. Hover over a point. Then switch to 3D Volume: depth carries the distance to a sanctioned address.

⑥ Proof that it runs (2 min) — Metrics tab.

Throughput, median and p95 latency, curves over 6 hours. "Everything comes from the database. No value is hard-coded."

API routes

Route Returns
GET /api/etat banner counters
GET /api/verifier?adresse=… the verdict — directe=0 limits to levels 1 and 2
GET /api/chaine?adresse=… the proof path from Neo4j
GET /api/dossiers the work queue
GET /api/alertes?limite=…&depuis=… recent alerts
GET /api/metriques 6-hour series
GET /api/graphe?limite=… nodes and edges, capped at 200
GET /api/dossier/statut?dossier=…&statut=… moves a case forward
GET /api/flux event stream — alerts pushed live

All input is validated at the boundary: length, characters, and checksum. A mistyped address is rejected before it triggers any outgoing network call.


8. The trap we found — and that is worth the detour

The file everyone uses for the US list is sdn.csv. It has no column for crypto addresses: they are concatenated into a free-text field called Remarks. That field is cut off at 1,000 characters.

Consequence, measured on the real file:

sdn.csv sdn.xml
"Digital Currency Address" mentions 463 963
Usable wallets 445 957
Of which Bitcoin 242 522
Addresses cut in the middle 11 0

The truncated entities are exactly the ones that matter, those that hold dozens of addresses: HYDRA MARKET, BLENDER.IO, GARANTEX, CHATEX, SUEX. And the cut does not lose one address per entity — it loses everything after the thousandth character.

How we noticed. The code verifies the checksum of each address before accepting it. Bitcoin addresses published by the US Treasury were failing that check. Rather than silently discarding them, we looked for the reason: they were truncated. So the project reads the XML.


9. Where to look when it breaks

Symptom Where to look Usual cause
docker compose up fails on a port docker compose ps, Get-NetTCPConnection -LocalPort 27018,8081 A local service already holds the port. Copy .env.example to .env and change MONGO_PORT / AIRFLOW_PORT.
The Airflow UI does not respond docker compose logs airflow-webserver Still starting: allow 60 s after up.
The DAG does not appear docker compose exec airflow-scheduler airflow dags list-import-errors Syntax error in the DAG, or a PYTHONPATH that does not see /opt/airflow/src.
An Airflow task keeps failing Airflow UI → task → Logs Source unreachable. The message says how many bytes had been read before giving up.
The list does not update ingestion_runs collection in MongoDB Every run leaves its report there: bytes read, rejects, disconnections.
No transactions arrive docker compose logs producteur Look for flux interrompu: the producer reconnects on its own, with a doubling wait. If the message repeats endlessly, it is the Internet connection.
No alert fires docker compose logs criblage Normal behaviour live: sanctioned addresses barely move any more (see § 5). Use demo_phase2.py to trigger one.
The screening does not see the list docker compose logs criblage | grep "liste surveillee" The Airflow DAG has not run yet. The consumer reloads by itself every 300 s.
The consumer falls behind LAG command below Normal lag after a restart: it catches up at ~19 tx/s. If it does not shrink, check criblage's memory.
producteur or criblage is unhealthy docker compose logs <service> The loop has not advanced for 120 s. This is a real hang, not a dead process — Docker would already have restarted that.
The disk fills up docker system df Purge bronze (BRONZE_RETENTION_DAYS), Kafka retention (KAFKA_RETENTION_HOURS), or the build cache (docker builder prune).

Looking at the data by hand

docker compose exec mongo mongosh sanctions_radar --eval "db.silver_sanctioned_wallets.countDocuments({chain:'BTC'})"
docker compose exec mongo mongosh sanctions_radar --eval "db.alertes.find().sort({detectee_a:-1}).limit(3).pretty()"
docker compose exec mongo mongosh sanctions_radar --eval "db.metriques_flux.find().sort({mesuree_a:-1}).limit(1).pretty()"

Checking the Kafka consumer lag

docker compose exec kafka /opt/kafka/bin/kafka-consumer-groups.sh --bootstrap-server localhost:9092 --group criblage --describe

LAG 0 means the screening is up to date. A LAG that grows without ever coming down is the only real alarm signal of the real-time chain.

Stopping everything, erasing everything

docker compose down            # stops, keeps the data
docker compose down -v         # stops and deletes the data

10. Repository layout

docker-compose.yml           the 8 services
docker/airflow/              Airflow image (the official one + 4 dependencies)
docker/streaming/            streaming services image — 240 MB, plain Python
airflow/dags/
  refresh_sanctions.py       the daily lists DAG
  construire_graphe.py       the weekly graph DAG

src/sanctions_radar/
  chains.py                  address recognition — pure functions, tested
  model.py                   the vocabulary shared by both sources
  http_stream.py             HTTP reading: resume on disconnect, TLS trust
  store.py                   MongoDB: bronze, silver, log
  pipeline.py                the sequence, shared by the DAG and the demo
  sources/ofac.py            US Treasury XML parser
  sources/opensanctions.py   FollowTheMoney feed parser

  streaming/                 ── phase 2 ──
    screening.py             THE screening — pure functions, tested
    bitcoin_ws.py            WebSocket feed, reconnection, idle timeout
    producer.py              WebSocket → Kafka, timestamped envelope
    consumer.py              Kafka → screening → alerts, manual offsets
    alerts.py                alerts, cases, lifecycle
    gravite.py               the 4 severity levels — pure, tested
    notifications.py         on-disk queue + optional HTTP hook
    metrics.py               throughput and latency percentiles
    replay.py                real history replay (mempool.space)
    esplora.py               format translation — pure, testable anywhere
    historique.py            four history sources, fail fast
    heartbeat.py             health: is the loop really advancing?

  graphe/                    ── phase 3 ──
    modele.py                bipartite Address/Transaction graph — pure
    taint.py                 THE "haircut" propagation — pure, 28 tests
    cluster.py               co-spending + CoinJoin guard — pure, 19 tests
    explorateur.py           bounded traversal guided by tainted value
    neo4j_store.py           loading and path queries
    verification.py          THE product: a verdict on an address
    pipeline.py              the sequence, shared by the DAG and the demo

  web/                       ── phase 4 ──
    routes.py                the 8 routes — pure functions, tested without a server
    serveur.py               HTTP glue, standard library, zero dependencies

web/                         the interface — no npm, no build
  index.html · app.css · app.js · graphe3d.js
  vendor/three.module.min.js loaded on click on "3D Volume"

tests/                       152 tests, on actually sanctioned data
  test_chains.py    (40)     recognition and checksums
  test_ofac.py      (16)     XML parser, including the CSV truncation
  test_opensanctions.py (14) FollowTheMoney parser
  test_http_stream.py (12)   simulated network cuts
  test_screening.py (25)     the screening, including its constant-time cost
  test_metrics.py   (13)     percentiles, windows, non-regressions
  test_replay.py    (11)     Esplora translation and full chain
  test_heartbeat.py (11)     detection of a stuck loop
  test_historique.py (10)    source failover and fail fast
  test_taint.py     (28)     propagation, including value conservation
  test_explorateur.py (14)   the traversal follows the money, not the alphabet
  test_cluster.py   (19)     co-spending, transitivity, CoinJoin exclusion
  test_alertes.py   (22)     grouping, severity, notifications
  test_api.py       (30)     boundary validation, limits, routes

scripts/
  demo_phase1.py             the lists and the absence of duplicates
  demo_phase2.py             real time and alerts
  demo_phase3.py             the graph and the taint chain
  mesure_sources.py          the figures in section 4
DECISIONS.md                 the trade-offs, with what was ruled out

11. What real time cannot do on its own

A structural limit, stated right away so that phase 3 does not head into a wall.

The unconfirmed transaction feed only gives what goes by while we are listening. It brings no history. A graph built from a few hours of listening is a cloud of unrelated transactions: there is nothing to propagate. Phase 2 shows this concretely — direct screening works perfectly and finds nothing, which is the expected result.

The workaround, verified: the full history of an address is public and free.

GET https://mempool.space/api/address/{adresse}/txs   →  200, 0,63 s, 47 transactions
GET https://blockchain.info/rawaddr/{adresse}         →  200, 0,51 s

This is already what the phase 2 replay uses. Phase 3 will use it to seed the graph with the real history of the sanctioned addresses, then propagate the taint over it. The live feed keeps its full role: it feeds throughput, metrics and recovery from disconnections — and it will end up touching the graph, because by then the tainted set will no longer number 568 addresses but several thousand.


12. Next steps

Phase Content Status
1 Docker foundation, list ingestion, daily refresh delivered
2 Real-time Bitcoin feed, Kafka, direct screening, alerts delivered
3 Neo4j graph seeded from history, weighted taint over n hops delivered
4 Web interface: verification, cases, graph, metrics delivered

Definition of Done status

Criterion Status
docker compose up -d brings everything up, with no manual step Done — verified from a cold start
The Bitcoin feed runs continuously and survives a network cut Done — test no. 1, § 5
A sanctioned transaction triggers a visible alert Done — 49 alerts in 3.0 s, including with the primary source down
Airflow refreshes the lists every day, without duplicates Done — upserted: 0
Metrics are measured: throughput, p95 latency, alerts Done — § 5
Taint walks up a chain over n hops, configurable Done — § 6 — 458 addresses, 3 hops, chain displayed
README.md and DECISIONS.md written and up to date Done — 30 trade-offs
A stranger can replay the demonstration Done — three scripts, § 2

What remains open, said plainly

Limit Why it exists
The traversal is bounded, not exhaustive 376 addresses dropped at the cap, 285 in the frontier. One more hop would reveal more — this is a cost choice, not an oversight.
Taint has no entity attribution We know an address is 11% tainted, not who owns it. Address clustering is a different discipline.
The 0.1% threshold is a choice, not a truth No authority publishes a regulatory threshold. It is configurable and its value is shown in every run.
Mempool alerts are not reconfirmed An unconfirmed transaction may never be mined. The status is carried; reconfirmation remains to be done.

About

Criblage temps réel des transactions crypto contre les listes de sanctions internationales, avec propagation de la contamination dans le graphe des transactions

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages