Add measured RAG evaluation and request tracing

[oosuhada]

Summary

• add a committed 120-query portfolio retrieval benchmark with lexical vs entity-aware hybrid comparison
• add request-stage tracing and retrieval telemetry without raw prompt/question logging
• add architecture-specific threat model and CI regression thresholds

Measured evidence

• Recall@5: 0.25 -> 0.53
• MRR@10: 0.2528 -> 0.5348
• nDCG@10: 0.2566 -> 0.5424
• no-evidence recall: 1.0 for both modes

Verification

• unit 13/13, routing 4/4, RAG integration 5/5
• strict 120-query benchmark passed
• Next.js production build passed
• browser E2E passed
• providerless AI eval: FAQ 5/5, failure cases 13/13