feat(providers): OpenRouter intelligence — real context, balance, measured cost (v0.21.261) - #63
Merged
Merged
Conversation
…sured cost (v0.21.261)
Slice 4, the last of the providers overhaul. Fixes the bug that started the
track and adds the account detail the product could previously only learn by
leaving the app.
**The `~` alias rename.** OpenRouter moved its `*-latest` family to a tilde
prefix: the catalog lists `~anthropic/claude-sonnet-latest` and the plain form
is gone entirely. Stored ids still *run* — OpenRouter resolves them — but missed
our catalog, so capabilities fell through to DEFAULT_CAPABILITIES' 8192 for a 1M
model. A believable number, wrong by two orders of magnitude, surfacing much
later as premature compaction and "spotty memory".
`resolve_catalog_id` follows the prefix at lookup time (verified live: 8192 →
1,000,000). Resolution is lookup-only — the request still goes out as the caller
asked — so it never changes which model runs, only which capabilities we
believe. The warning fires once per id, and `describe_route` consults the
resolver so a working alias no longer reads as `known: false`.
The *stored* half is `openrouter_migration.py`, and it is **opt-in with the list
shown**: these are the user's own model choices, so the scan reports what it
would change and nothing moves until they confirm. It walks agent profiles, the
global model settings (incl. config-kind ROLE_MEMBERS) and memory-stage models,
warming the catalog first. A replacement is proposed only when **verified present
in the catalog** — never one broken id rewritten into another — and
`catalog_available: false` means "couldn't check", never "all clear".
**Account strip.** `/key` + `/credits` behind a 60s cache, cleared on unlink so a
forgotten key can't keep reporting a balance. Both best-effort: `/credits` is
documented management-key-only but answers ordinary keys today, so its absence
downgrades detail, not availability. `limit: null` is preserved as null — that's
"no cap", and coercing it to 0 would render an empty gauge reading as "nothing
left". Live: $6.30 of $35.00, $0.00 this month.
**Per-endpoint truth + measured cost.** `/models/{author}/{slug}/endpoints`
answers "why is my context 200k and not 1M?" (the catalog reports the best
endpoint; a turn may route to a smaller one). `/generation?id=` reports what was
really billed, including cache reads/writes and long-context tiers an estimate
can't see — 404 is a normal answer there, since the record is written
asynchronously, so callers keep the estimate. Catalog fetch now prefers
`/models/user`, which respects the account's own provider/privacy filtering.
Also fixes a test double: a bare MagicMock auto-creates `resolve_catalog_id`, so
`test_route_reports_an_unlisted_model_as_unknown` started reporting every unknown
model as known. Caught by `task test:sterile` running the full module.
Deferred to backlog: automatic ledger reconciliation against `/generation` (a
background job, not an inline per-turn call — that would add a round trip to
every turn).
1217 API tests (17 new) OK normally and under `task test:sterile`; 437 client
tests (4 new); pyright 0; tsc clean; docs gates green (161 routes in parity).
Assisted-by: Opus 5
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Slice 4 of 4 — the last of the providers overhaul (#59 catalog, #60 surface, #61 OAuth). Fixes the bug that started the track.
The
~alias renameOpenRouter moved its
*-latestfamily to a tilde prefix: the catalog lists~anthropic/claude-sonnet-latestand the plain form is gone entirely. Stored ids still run — OpenRouter resolves them server-side — but missed our catalog, so capabilities fell through toDEFAULT_CAPABILITIES' 8192 for a 1M model. A believable number, wrong by two orders of magnitude, surfacing much later as premature compaction and "spotty memory".Verified live:
resolve_catalog_idfollows the prefix at lookup time only — the request still goes out as the caller asked, so this never changes which model runs, only which capabilities we believe. The warning fires once per id, anddescribe_routeconsults the resolver so a working alias no longer reads asknown: falsein the Supply Line.Repairing stored ids — opt-in, with the list shown
openrouter_migration.pywalks agent profiles, the global model settings (incl. config-kindROLE_MEMBERS) and memory-stage models, warming the catalog first. Nothing moves until the user confirms — these are their own model choices, and silently rewriting them is exactly the "don't change the user's defaults" failure.Two guarantees:
catalog_available: falsemeans "couldn't check", never "all clear". The scan reports it rather than implying an all-clear it can't back.Account strip
/key+/creditsbehind a 60s cache, cleared on unlink so a forgotten key can't keep reporting a balance. Both best-effort:/creditsis documented management-key-only but answers ordinary keys today, so its absence downgrades detail, not availability.limit: nullis preserved as null — that's "no cap", and coercing it to 0 would render an empty gauge that reads as "nothing left" and sends the user hunting for a billing problem.Live on the real account: $6.30 of $35.00, $0.00 this month.
Per-endpoint truth + measured cost
GET /models/{author}/{slug}/endpoints— per-providercontext_length,max_completion_tokens, quantization and price. The answer to "why is my context 200k and not 1M?": the catalog reports the best endpoint, while a turn may route to a smaller one.GET /generation/{id}— what was really billed, including cache reads/writes and long-context rate tiers an estimate can't see. 404 is a normal answer (the record is written asynchronously), so callers keep the estimate and may retry./models/user, which respects the account's own provider/privacy/guardrail filtering, falling back to/models.A test double that had gone wrong
A bare
MagicMockauto-createsresolve_catalog_id— callable, returning a truthy Mock — so once the registry started consulting it,test_route_reports_an_unlisted_model_as_unknownreported every unknown model as known. Fixed by deleting the attribute so the double matches reality (Anthropic has no resolver). Caught bytask test:sterilefrom #62 running the full module.Verification
Driven against the real service and the real UI:
config.jsonbyte-identical to the pre-test backup.1217API tests (17 new) OK normally and undertask test:sterile;437client tests (4 new);pyright0;tscclean;task docs:checkgreen — 161 routes in parity.Deferred
Automatic ledger reconciliation against
/generation— that belongs in a background job, not an inline per-turn call, which would add a round trip to every turn. Noted for the backlog; the primitive and endpoint are here.Docs
Development-Notes section,
OpenApi.yaml+endpoints.md, version + release notes.Assisted-by: Opus 5