feat(rag): EmbeddingGemma-300M via ONNX + hybrid retrieval, doc-relevance gate, reranker

[sagar-develop]

Summary

Replaces the USE-Lite (100-dim TFLite) document embedder with EmbeddingGemma-300M on ONNX Runtime — telemetry-free, no MediaPipe/Play Services — and rebuilds document retrieval as a hybrid pipeline with a set of retrieval-quality fixes verified on-device.

Embedder upgrade

• EmbeddingGemma-300M via ONNX Runtime (OnnxEmbeddingEngine), with an optional ms-marco-MiniLM-L6 cross-encoder reranker (OnnxReranker).
• Device-tiered: USE-Lite (<6 GB) · Gemma@256 (6–8 GB) · Gemma@256 + reranker (8–10 GB) · Gemma@512 + reranker (≥10 GB), driven by EmbedderRecommendation and surfaced as a "Recommended" badge. Models download on-device through the catalogue; nothing is bundled.
• Pure-Kotlin tokenizers (GemmaBpeTokenizer, BertWordPieceTokenizer) reading the HF tokenizer.json, validated against the reference transformers tokenizer — onnxruntime-extensions has no GemmaTokenizer.
• Matryoshka truncation (128/256/512) with per-dim ObjectBox HNSW entities (GemmaChunk128/256/512) + dim routing in the repository.
• Task-aware QUERY/DOCUMENT embedding prompts.

Hybrid retrieval

Dense vector search + BM25 lexical scoring fused via Reciprocal Rank Fusion, with a per-document cap, wider candidate pools, and a larger grounding budget.

Retrieval-quality fixes (the war story)

A real personal-finance project held three policies: a car policy (TATA AIG, ₹8,504), a life policy (Future Generali, ₹41,799), and a health policy (ICICI Lombard).

1. Wrong-document grounding — "car insurance premium" answered ₹41,799 from the life policy because BM25 lexical overlap let the life PDF out-score the car PDF. ₹8,504 is the correct car premium. Fixed by a document-level dominance gate that keeps grounding on the source that genuinely dominates the candidate set.
2. Title-match override — "who is the insurer of my car policy" grounded on the health policy (whose formal "…insurer" phrasing out-scored the car doc). A distinctive query term naming a doc by title now grounds on that doc → answer went ICICI Lombard → TATA AIG.
3. Truncated grounded answers — grounded replies collapsed to 1–2 tokens after a few turns because the stateful LiteRT-LM KV cache accumulated each turn's grounding block. Fixed by a per-grounded-turn session reset (reopenSessionAndAwait) that re-prefills only bounded visible history (MAX_PREFILL_TURNS=16).
4. Self-healing migration — document-level re-index into the active embedder's index on next open; no re-import/OCR.
5. Reranker recall win — enabling the ungated cross-encoder reranker on the 8 GB tier recovered a health-insurer chunk the first-stage fusion ranked too low.

Testing

• 11/11 DefaultDocumentRetrieverTest retriever unit tests pass.
• :sample-app:assembleRelease (R8 + signed) builds green.
• Verified on-device (Realme CPH2723): all four grounding scenarios above behave correctly post-fix.

Known residual

The life policy's sum assured sits inside a garbled extracted table (PDF text-extraction artifact), so it can be missed at the first-stage recall step. This is a PDF-extraction / first-stage recall limitation, not a ranking bug — documented for follow-up.

🤖 Generated with Claude Code