Replicate: the mutual-legibility matrix (CPU-only, ~20 min, no GPU)

[fathomlab]

Replicate the mutual-legibility matrix - no GPU, ~20 minutes, laptop CPU.

We published the first map of which AI models can decode each other's concept representations with zero labels (FINDING). Three models form a mutually legible clique across model families; one (Qwen2.5-1.5B) is an island - unreadable from every direction.

The concept extractions are committed to the repo, so no model downloads and no GPU are needed.

git clone https://github.com/fathom-lab/styxx
cd styxx/papers/disjoint-worlds
pip install numpy scipy
python run_b37.py

What to reproduce (deterministic, seed 343):

quantity	committed value
llama_3b -> gemma_2b discovery	0.5918
llama_3b -> qwen_1p5b discovery	0.0536
clique pairs	all discover 0.59-0.83
qwen (island)	<= 0.17 from every direction

The load-bearing claim is the topology (clique + island), not any single decimal.

Breaking it is worth more than confirming it - see REPLICATIONS.md. Post your numbers here either way and you get a named credit line.

Good first issue because: one command, no hardware, and the answer is checkable against a committed receipt.

[fathomlab]

Update 2026-08-06 - the replication surface grew, and the cheapest check is now four seconds.

When this issue was posted the target was the b37 matrix. The arc has since closed at nine sealed acts, and several are far cheaper to check. Ranked by how fast you can falsify us:

check	runtime	what it claims
run_b45.py	~4 seconds	clique concept-frames co-align at 0.848 median vs a 0.0566 random-null p95; the island sits below the clique in 5/5 seeds. No fitting, no Hungarian - pure principal angles.
run_b41.py	minutes	correcting the island's top-20 frame directions takes discovery 0.0612 to 0.9745, while a matched random frame does 0.0
run_b46.py	~1 hour	the cliff: flat, then near-vertical, knee at t=0.8
run_b37.py	~20 min	the original matrix, clique/island topology
run_b42.py	~9 hours	the dose curve, k*=2

All CPU-only from the committed .npz banks. No GPU, no downloads.

If you have four seconds, run b45 - it is the single easiest claim here to break.

Two honest notes for replicators:

1. A claim in REPLICATE_legibility.md was overstated and has been corrected: b44's "every donor x seed above the 0.30 floor" holds only at k=20; the k=2 core transfers partially (medians 0.3214 / 0.3036). Anyone running against the old text would have hit the discrepancy themselves.
2. run_b48.py returns INVALID__null_leaks. That is correct behaviour, not a broken script - we mis-specified a gate and the machinery refused the run. It ships as-is because the refusal is the point.

Still good first issue. A replication that breaks something earns more credit than one that confirms it.