Skip to content

Add OpenAI API embedder behind optional extra - #13

Merged
AshwinUgale merged 1 commit into
AshwinUgale:mainfrom
Dawn-OuYang:add-api-embedder-backend
Aug 26, 2026
Merged

Add OpenAI API embedder behind optional extra#13
AshwinUgale merged 1 commit into
AshwinUgale:mainfrom
Dawn-OuYang:add-api-embedder-backend

Conversation

@Dawn-OuYang

Copy link
Copy Markdown
Contributor

Title:
Add OpenAI API embedder behind optional extra

Body:
Closes #10

This adds the first hosted API embedder backend for Retrieval Lab. The new
api:<model> selector currently targets OpenAI embedding models and keeps the default
install keyless by importing the OpenAI SDK only when the API embedder is selected.

Changes:

  • Add OpenAIEmbedder in retrieval_lab.embedding.api.
  • Add the optional [api-embed] extra with the OpenAI SDK dependency.
  • Wire --embed-models api:<model> and geometry --embed-model api:<model> through the CLI.
  • Reuse the existing EmbeddingCache content-hash cache for hosted embeddings.
  • Add mocked/keyless tests for the API embedder and missing-extra CLI error.
  • Document installation, OPENAI_API_KEY, and provider availability/rate-limit caveats.
  • Update CHANGELOG.md.

Duplicate check:

  • Checked open PRs mentioning #10; only an unrelated Dependabot PR is open.
  • Checked open PRs for api-embed, OpenAIEmbedder, and hosted embedding; no duplicate PR found.

Validation:

  • python -m pytest -q
  • ruff check src tests
  • git diff --check
  • python -m py_compile src/retrieval_lab/embedding/api.py src/retrieval_lab/embedding/__init__.py src/retrieval_lab/cli.py tests/test_embedding.py tests/test_cli.py

Note:

  • ruff format --check src tests was not used as a gate here because the latest local Ruff
    wants to reformat many pre-existing files outside this patch. This PR leaves unrelated
    formatting untouched.

Signed-off-by: Dawn-OuYang <Dawn-OuYang@users.noreply.github.com>
@AshwinUgale

Copy link
Copy Markdown
Owner

Thanks for this, @Dawn-OuYang — really clean work.

Lazy-importing the OpenAI SDK keeps the default install and tests keyless, reusing the existing content-hash cache gives hosted embeddings free dedup across a sweep, and the injected-fake-client tests cover the adapter without a real key. Docs and CHANGELOG updates appreciated too.

Merging now — thanks again! 🚀

@AshwinUgale
AshwinUgale merged commit 6bc5155 into AshwinUgale:main Aug 26, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add an API embedder backend (OpenAI / Cohere) behind an extra

2 participants