Skip to content

Prototype hybrid sparse + dense retrieval #15

Description

@AshwinUgale

Why

The README roadmap flags hybrid sparse + dense retrieval "if any single library's recall plateaus." A contributor could add a sparse (BM25) signal alongside the current dense vector search and measure the delta with the eval harness. Meaty, self-contained research task with a clear success metric (recall on the corpus).

What to do

  • Add an optional sparse retrieval path, fuse scores with the dense hits (e.g. Reciprocal Rank Fusion), gate it behind a setting, and report before/after eval numbers in the PR.

Acceptance criteria

  • Eval harness shows the accuracy/recall delta; feature is opt-in and documented.

Pointers

  • sidecar/src/docchat_sidecar/tools.py (SearchDocsTool); evals/ for measurement.

Larger scope - comment before starting so we can align on the approach.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requesthelp wantedExtra attention is neededresearchExperimental / measured research work

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions