feat(eval): confusion clusters, and the corpus reaches 53 chunks

[mrsibe]

Stacked on #204 (fix/eval-review-followups). Retarget to main after the chain merges.

What does this PR do?

The first two confusion clusters of corpus v2 — deliberately confusable material, not "near-duplicate documents that lower Recall@5". The goal is a benchmark that can tell retrieval strategies apart at all, and lowering Recall is a consequence of that rather than the objective.

The clusters

1. Gateway API v1 / v2 / v3 + a migration guide. Four documents sharing almost all their vocabulary and differing in every number:

	v1	v2	v3
Default timeout	30000 ms	60000 ms	45000 ms
Recommended retries	2	5	3
Backoff base	500 ms	2000 ms	1000 ms
Rate limit	600 rpm	3000 rpm	1200 rpm
Context window	8192 tokens	32768 tokens	65536 tokens

All four discuss timeout, retry, backoff, rate limit, context window in that order. The migration guide restates every number in comparison tables, so it contains v1's timeout and v2's and v3's in a single block — the hardest negative in the set.

2. Monitoring protocols. Four documents that are confusable with each other and with the river-monitoring.md / lake-monitoring.md already in the corpus:

	reservoir	coastal	estuary	groundwater
Sampling interval	fortnightly	hourly	daily	monthly
Replicates	4	3	5	2
Sensor depth	5 m below surface	1 m below surface	2 m below surface	15 m below water table

Each states the network's extreme value ("the least frequent cadence in the network", "the highest of any programme"), so several passages read as plausible and only one is right.

Corpus: 19 → 53 chunks, 13 → 21 documents, 44 → 63 questions.

The tool that made it possible

npm run eval:blocks <file.md> prints the real document_blocks.order values through the same loader and block builder the harness uses. Ground truth block is a pipeline ordinal, not a line number a human counted; authoring a corpus by guessing it is how a dataset quietly drifts. eval/README.md now says so and points at the command.

What it measured

                 before    after
chunks           19        53
Recall@5         0.9211    0.8774
nDCG@10          0.8476    0.7804
MAP@10           0.7965    0.7280

type	n	Recall@5	nDCG@10
cross-lingual	9	0.6667	0.4173
multi-hop	5	0.7000	0.7218
semantic	21	0.9048	0.8542
exact	8	1.0000	0.8663
hard-negative	7	1.0000	0.8758
zh	3	1.0000	1.0000

hard-negative has perfect Recall@5 and a much worse nDCG — the signature the material was built for: the correct passage is in the top 5, and a confusable sibling keeps outranking it. That is a ranking problem the strategy comparison can now see, and it is invisible on a corpus where the right answer is always at rank 1.

multi-hop fell to 0.7000: the comparison questions need four locations each and do not get all four.

A result worth pausing on

The sweep (all 63 questions) now shows hybrid ahead of dense on the deciding metric:

	Recall@5	nDCG@10
dense, candidateK=20	0.8774	0.7804
hybrid, candidateK=20	0.8962	0.8098
hybrid, candidateK=5	0.9104	0.8101

But on the validation split the same comparison is a dead heat (both 0.8684), so the held-out rule still declines to adopt and test reports dense only.

That discrepancy is the honest state of things, and it is a property of the split, not of hybrid: 24 validation questions is not enough to see a gain the full set shows. The conclusion is to grow the validation split, not to go back to selecting on everything — which is exactly what produced the earlier, over-confident "hybrid clears the rule".

Unanswerable is still 0/10

meanCandidatesRetrieved sits at the candidateK cap and meanContextPassages at contextK for every unanswerable question, so abstention remains unmeasurable here. Two of the ten are near-miss questions — "How much GPU memory does Gateway API v2 require?" and "How many litres per second does the reservoir release downstream?" — about subjects the corpus discusses at length and never answers. Those are the realistic hallucination shape; the FIFA questions were only ever a sanity check.

Scope, honestly

53 chunks, not 150–300. Two clusters is not the target either. candidateK ∈ {5,10,20,40} has more room than it did at 19 chunks, but candidateK ≥ 10 still behaves identically, so the axis is only partly unlocked and threshold remains unmeasurable.

The remaining clusters are the same exercise and the same verification loop. This PR stops where the verified work stops.

Testing

• npm test — 504 pass
• npm run eval twice — byte-identical baseline (63 questions, 53 chunks)
• npm run eval:retrieval, eval:threshold, eval:sweep — all regenerated
• eval/splits.json extended: 24 validation / 39 test, stratified by type

Related

Part of #192. Child 2, stage 3 (confusion clusters).