bench: OpenKnowledge embeddings provider head-to-head (LodeDB vs OpenAI default)

[Davidobot]

Summary

• new benchmarks/openknowledge_embeddings/: what does an OpenKnowledge (inkeep/open-knowledge) user trade by pointing search.semantic.baseUrl at lodedb serve instead of the default OpenAI text-embedding-3-small?
• quality axis: their own semantic-eval.ts harness run unmodified via its env knobs; LodeDB minilm (and bge) pass their pre-registered FR2 gate (held-out MRR 0.753 vs 0.797, recall@5 1.000 vs 0.923, n=13; bge passes but scores below minilm because its query prefix cannot be expressed over the OpenAI wire, which is why minilm is the documented recommendation)
• performance axis on Modal (cpu=8, 16 GB): kubernetes/website content/en pinned at 71d23f81 (2,441 docs / 4,485 chunks), chunked and batched with faithful ports of their chunker (UTF-16 parity-tested against their own test cases) and embedder semantics (96/96k sequential batches, their retry/timeout policy, pooled connections, same client both sides)
• results: OpenAI bulk-indexes 1.4-2.9x faster (median 1.8x); LodeDB query p50 4.5 ms vs 173.3 ms (~38x lower) with max 7.7 ms vs 2,804.8 ms, at $0 and zero content egress vs $0.116 and 159.5 MB per full index; bulk throughput varied ~2x across Modal CPU hosts so medians and ranges are published with raw metrics-only JSONs for all five runs
• benchmarks/README.md index row

Testing

• .venv/bin/python -m pytest benchmarks/openknowledge_embeddings/ -q: 11 passed (chunker UTF-16 parity vs their chunking.test.ts cases, batcher, surrogate sanitization); bench tests are folder-local like the other benchmark suites, exact invocation documented in the README
• ruff check clean
• results reproduced across five Modal runs (one both-provider 300-doc, one both-provider full-corpus, three lodedb-only full-corpus repeats), all committed under results/