context_report.run.runner.PROVIDERS is currently ("anthropic", "claude-cli", "openai-compatible"). A manifest naming any other provider is rejected with "no backend for judge
provider".
Bedrock is worth adding: it is how most enterprise users reach Claude, and those are exactly
the users who need a report format they can run inside their own boundary. Today they cannot use
the efficacy engine at all without routing through a provider their procurement said no to.
What to do: add the backend beside the existing ones. The contract is small — the Asker
protocol in src/context_report/efficacy/backends.py is essentially ask(prompt) -> str, with
optional last_usage and last_model. openai_backend.py and cli_backend.py are the two
worked examples.
The thing to get right: a report says how it was obtained and cannot overstate it. Bedrock
model identifiers are not the same strings as the Anthropic API's, and a statement that records
one while having run the other is a provenance bug, not a cosmetic one. Carry the real model
identity through to the statement.
Testing: the existing tests inject a fake asker so no network call is ever made in CI. Follow
that pattern; do not add a test that needs live AWS credentials.
Sized to an afternoon if you have an account to try it against. Files:
src/context_report/run/runner.py, judge.py, efficacy/backends.py, and their tests.
context_report.run.runner.PROVIDERSis currently("anthropic", "claude-cli", "openai-compatible"). A manifest naming any other provider is rejected with "no backend for judgeprovider".
Bedrock is worth adding: it is how most enterprise users reach Claude, and those are exactly
the users who need a report format they can run inside their own boundary. Today they cannot use
the efficacy engine at all without routing through a provider their procurement said no to.
What to do: add the backend beside the existing ones. The contract is small — the
Askerprotocol in
src/context_report/efficacy/backends.pyis essentiallyask(prompt) -> str, withoptional
last_usageandlast_model.openai_backend.pyandcli_backend.pyare the twoworked examples.
The thing to get right: a report says how it was obtained and cannot overstate it. Bedrock
model identifiers are not the same strings as the Anthropic API's, and a statement that records
one while having run the other is a provenance bug, not a cosmetic one. Carry the real model
identity through to the statement.
Testing: the existing tests inject a fake asker so no network call is ever made in CI. Follow
that pattern; do not add a test that needs live AWS credentials.
Sized to an afternoon if you have an account to try it against. Files:
src/context_report/run/runner.py,judge.py,efficacy/backends.py, and their tests.