feat(retrieval): fusion candidate layer, BM25/hybrid strategies, and the retrieval experiment

[mrsibe]

Stacked on #169 (feat/96-fts-search): hybrid retrieval needs the FTS index from #96. Retarget to main after that merges.

What does this PR do?

Implements the candidate/fusion layer and the BM25 / hybrid strategies, runs the retrieval experiment #77 asked for, and fixes the bug that made BM25 silently useless.

Related issue

Fixes #77
Related to #154, #96, #78

The result

Every strategy runs the real harness over the same corpus and 30 questions, chunking held fixed at the frozen baseline (baseline-v1.5.json):

Strategy	Recall@1	Recall@5	MRR	nDCG@10	Evidence P@5	p95
dense (vector)	0.8333	1.0000	0.9278	0.9437	0.2133	18.35 ms
sparse (BM25)	0.7333	0.8667	0.8056	0.8184	0.1867	2.73 ms
hybrid (RRF)	0.8667	1.0000	0.9444	0.9561	0.2133	28.52 ms

Hybrid is better on Recall@1, MRR and nDCG@10 — but the frozen rule requires Recall@5 to improve, and dense is already saturated at 1.0000, so no strategy can satisfy it here. Dense stays the default, and the result is recorded as inconclusive rather than adopted or rejected on a metric that cannot move. The report says this plainly.

The bug it exposed

buildFtsMatchQuery ANDed the terms. That is the right default for a lookup box but wrong for a question: a natural-language question's terms virtually never all appear in one chunk, so sparse scored 0.0000 on every metric and hybrid silently degenerated to dense — a "hybrid" that was dense with extra latency. Terms are now ORed; BM25 still ranks a chunk matching more terms higher.

What changed

• candidates.ts: rrfFuse over chunkId + rank. Scores are not comparable across channels (cosine vs BM25), which is exactly why only the ranks are used.
• HybridRetriever serves dense / sparse / hybrid from one path; DenseRetriever gained candidateHits() so the dense channel is not duplicated for fusion.
• RetrievalRequest.strategy, SearchOptions.strategy, --eval-retrieval=, scripts/eval-retrieval.mjs, npm run eval:retrieval.
• docs/eval/retrieval-v1.5.{md,json} are generated and added to .prettierignore.

Not evaluated

Reranking. The issue lists "hybrid + reranker", but a cross-encoder model is not available offline and inventing its numbers would defeat the harness. It stays open until a model can be pinned the way the embedding model is. Stated in the report.

How was this tested?

• npm run typecheck — passes.
• npm test — 398 pass (new RRF tests: agreement wins, single-channel hits still rank, empty channels, rank-not-score).
• npm run check:design — no violations.
• npm run build — passes.
• npm run eval — byte-identical to baseline-v1.5.json (dense is still the default, so the frozen numbers do not move).
• electron . --smoke-test — PASS, 27 checks, with HybridRetriever now the app's retriever.

Checklist

• I have reviewed my own changes.
• npm run typecheck passes.
• npm run build passes.
• I have tested the affected user workflow.
• I have not included unrelated changes.
• I have updated documentation when necessary.

Desktop / build changes

• Not applicable