Skip to content

docs(examples): add hybrid BM25 + dense retrieval RAG notebook - #7499

Open
kShaman771 wants to merge 1 commit into
chroma-core:mainfrom
kShaman771:docs/hybrid-retrieval-rag-example
Open

docs(examples): add hybrid BM25 + dense retrieval RAG notebook#7499
kShaman771 wants to merge 1 commit into
chroma-core:mainfrom
kShaman771:docs/hybrid-retrieval-rag-example

Conversation

@kShaman771

Copy link
Copy Markdown

Description of changes

Adds a self-contained Jupyter notebook demonstrating hybrid retrieval for RAG using ChromaDB.

The notebook combines BM25 sparse retrieval with ChromaDB dense semantic retrieval and uses Reciprocal Rank Fusion (RRF) to merge both ranked result sets.

  • Improvements & Bug fixes

    • No bug fixes or changes to existing Chroma functionality.
    • No existing APIs or behavior are modified.
  • New functionality

    • Adds examples/advanced/hybrid_retrieval_rag.ipynb.
    • Demonstrates BM25 sparse retrieval using rank_bm25.
    • Demonstrates dense semantic retrieval using ChromaDB with sentence-transformer embeddings.
    • Implements Reciprocal Rank Fusion (RRF) to combine sparse and dense ranked results.
    • Includes side-by-side retrieval examples to illustrate the behavior of sparse, dense, and hybrid retrieval.
    • Runs locally without requiring external API keys.

Test plan

The notebook has been run and verified end-to-end.

The example verifies the complete workflow, including:

  • Creating and indexing the example document corpus.
  • BM25 sparse retrieval.
  • ChromaDB dense semantic retrieval.
  • Reciprocal Rank Fusion of the sparse and dense ranked result sets.
  • Running example queries and comparing sparse, dense, and hybrid retrieval results.

The change is isolated to a new example notebook and does not modify the Chroma library, existing tests, or production code.

  • Tests pass locally with pytest for python, yarn test for js, cargo test for rust

The repository-wide test suites above are not applicable to this documentation/example-only addition and have therefore not been marked as completed.

Migration Plan

No migrations or forwards/backwards compatibility changes are required.

This PR only adds a new standalone notebook under examples/advanced/ and does not modify existing APIs, schemas, persisted data, configuration, or runtime behavior.

Observability plan

No additional instrumentation or monitoring is required.

This change adds an educational example notebook only and does not introduce or modify production runtime behavior, services, telemetry, or operational infrastructure.

Documentation Changes

This PR adds a self-contained Jupyter notebook at:

examples/advanced/hybrid_retrieval_rag.ipynb

The notebook serves as an advanced usage example demonstrating how ChromaDB dense retrieval can be combined with BM25 sparse retrieval using Reciprocal Rank Fusion (RRF).

The notebook includes the required setup, implementation, explanatory context, and example retrieval comparisons so that the hybrid retrieval workflow can be understood and run independently.

No existing user-facing APIs are changed, so no updates to the existing API documentation are required.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@github-actions

Copy link
Copy Markdown

Reviewer Checklist

Please leverage this checklist to ensure your code review is thorough before approving

Testing, Bugs, Errors, Logs, Documentation

  • Can you think of any use case in which the code does not behave as intended? Have they been tested?
  • Can you think of any inputs or external events that could break the code? Is user input validated and safe? Have they been tested?
  • If appropriate, are there adequate property based tests?
  • If appropriate, are there adequate unit tests?
  • Should any logging, debugging, tracing information be added or removed?
  • Are error messages user-friendly?
  • Have all documentation changes needed been made?
  • Have all non-obvious changes been commented?

System Compatibility

  • Are there any potential impacts on other parts of the system or backward compatibility?
  • Does this change intersect with any items on our roadmap, and if so, is there a plan for fitting them together?

Quality

  • Is this code of a unexpectedly high quality (Readability, Modularity, Intuitiveness)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant