Skip to content

providers: add a gemini backend #61

Description

@jothimani-rajendran

context_report.run.runner.PROVIDERS is currently ("anthropic", "claude-cli", "openai-compatible") — the Anthropic API with a key, the local Claude Code CLI with its login,
and anything speaking the OpenAI-compatible shape. A manifest naming any other provider is
rejected with "no backend for judge provider".

Gemini is worth adding: Google's own API, not its OpenAI-compatible shim. The native shape
differs enough that the compatible path loses information, and a judge whose usage accounting is
approximate undermines the cost.* rows.

What to do: add the backend beside the existing ones. The contract is small — the Asker
protocol in src/context_report/efficacy/backends.py is essentially ask(prompt) -> str, with
optional last_usage and last_model. openai_backend.py and cli_backend.py are the two
worked examples.

The thing to get right: a report says how it was obtained and cannot overstate it. Whatever
you add must carry its provenance through to the statement, so a reader can see which backend
produced a row. A backend that makes its rows indistinguishable from another's defeats the point
of the format.

Testing: the existing tests inject a fake asker so no network call is ever made in CI. Follow
that pattern. Do not add a test that needs a live key — it will fail for every contributor who
does not have one, which is most of them.

Sized to an afternoon if you have credentials to try it against; the plumbing is small and the
care is in the provenance. Files: src/context_report/run/runner.py, judge.py,
efficacy/backends.py, and their tests.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions