Skip to content

feat(providers): OpenRouter intelligence — real context, balance, measured cost (v0.21.261) - #63

Merged
QR-Madness merged 1 commit into
masterfrom
feat/openrouter-intelligence
Aug 1, 2026
Merged

feat(providers): OpenRouter intelligence — real context, balance, measured cost (v0.21.261)#63
QR-Madness merged 1 commit into
masterfrom
feat/openrouter-intelligence

Conversation

@QR-Madness

Copy link
Copy Markdown
Owner

Slice 4 of 4 — the last of the providers overhaul (#59 catalog, #60 surface, #61 OAuth). Fixes the bug that started the track.

The ~ alias rename

OpenRouter moved its *-latest family to a tilde prefix: the catalog lists ~anthropic/claude-sonnet-latest and the plain form is gone entirely. Stored ids still run — OpenRouter resolves them server-side — but missed our catalog, so capabilities fell through to DEFAULT_CAPABILITIES' 8192 for a 1M model. A believable number, wrong by two orders of magnitude, surfacing much later as premature compaction and "spotty memory".

Verified live:

anthropic/claude-sonnet-latest   →  resolved ~anthropic/claude-sonnet-latest   ctx 1,000,000   (was 8192)
google/gemini-flash-latest       →  resolved ~google/gemini-flash-latest       ctx 1,048,576

resolve_catalog_id follows the prefix at lookup time only — the request still goes out as the caller asked, so this never changes which model runs, only which capabilities we believe. The warning fires once per id, and describe_route consults the resolver so a working alias no longer reads as known: false in the Supply Line.

Repairing stored ids — opt-in, with the list shown

openrouter_migration.py walks agent profiles, the global model settings (incl. config-kind ROLE_MEMBERS) and memory-stage models, warming the catalog first. Nothing moves until the user confirms — these are their own model choices, and silently rewriting them is exactly the "don't change the user's defaults" failure.

Two guarantees:

  • A replacement is proposed only when verified present in the catalog — never one broken id rewritten into another.
  • catalog_available: false means "couldn't check", never "all clear". The scan reports it rather than implying an all-clear it can't back.

Account strip

/key + /credits behind a 60s cache, cleared on unlink so a forgotten key can't keep reporting a balance. Both best-effort: /credits is documented management-key-only but answers ordinary keys today, so its absence downgrades detail, not availability.

limit: null is preserved as null — that's "no cap", and coercing it to 0 would render an empty gauge that reads as "nothing left" and sends the user hunting for a billing problem.

Live on the real account: $6.30 of $35.00, $0.00 this month.

Per-endpoint truth + measured cost

  • GET /models/{author}/{slug}/endpoints — per-provider context_length, max_completion_tokens, quantization and price. The answer to "why is my context 200k and not 1M?": the catalog reports the best endpoint, while a turn may route to a smaller one.
  • GET /generation/{id} — what was really billed, including cache reads/writes and long-context rate tiers an estimate can't see. 404 is a normal answer (the record is written asynchronously), so callers keep the estimate and may retry.
  • Catalog fetch now prefers /models/user, which respects the account's own provider/privacy/guardrail filtering, falling back to /models.

A test double that had gone wrong

A bare MagicMock auto-creates resolve_catalog_id — callable, returning a truthy Mock — so once the registry started consulting it, test_route_reports_an_unlisted_model_as_unknown reported every unknown model as known. Fixed by deleting the attribute so the double matches reality (Anthropic has no resolver). Caught by task test:sterile from #62 running the full module.

Verification

Driven against the real service and the real UI:

  • Alias fix live: 8192 → 1,000,000 on a stored id.
  • Account strip live: $6.30 of $35.00, uncapped key rendered without a gauge.
  • Migration scan against the live catalog: 0 stale refs on this install (correct), and 1 detected when fed a stale id — so both the negative and positive paths are proven.
  • Repair affordance and its confirmation list rendered and cancelled; config.json byte-identical to the pre-test backup.
  • 1217 API tests (17 new) OK normally and under task test:sterile; 437 client tests (4 new); pyright 0; tsc clean; task docs:check green — 161 routes in parity.

Deferred

Automatic ledger reconciliation against /generation — that belongs in a background job, not an inline per-turn call, which would add a round trip to every turn. Noted for the backlog; the primitive and endpoint are here.

Docs

Development-Notes section, OpenApi.yaml + endpoints.md, version + release notes.

Assisted-by: Opus 5

…sured cost (v0.21.261)

Slice 4, the last of the providers overhaul. Fixes the bug that started the
track and adds the account detail the product could previously only learn by
leaving the app.

**The `~` alias rename.** OpenRouter moved its `*-latest` family to a tilde
prefix: the catalog lists `~anthropic/claude-sonnet-latest` and the plain form
is gone entirely. Stored ids still *run* — OpenRouter resolves them — but missed
our catalog, so capabilities fell through to DEFAULT_CAPABILITIES' 8192 for a 1M
model. A believable number, wrong by two orders of magnitude, surfacing much
later as premature compaction and "spotty memory".

`resolve_catalog_id` follows the prefix at lookup time (verified live: 8192 →
1,000,000). Resolution is lookup-only — the request still goes out as the caller
asked — so it never changes which model runs, only which capabilities we
believe. The warning fires once per id, and `describe_route` consults the
resolver so a working alias no longer reads as `known: false`.

The *stored* half is `openrouter_migration.py`, and it is **opt-in with the list
shown**: these are the user's own model choices, so the scan reports what it
would change and nothing moves until they confirm. It walks agent profiles, the
global model settings (incl. config-kind ROLE_MEMBERS) and memory-stage models,
warming the catalog first. A replacement is proposed only when **verified present
in the catalog** — never one broken id rewritten into another — and
`catalog_available: false` means "couldn't check", never "all clear".

**Account strip.** `/key` + `/credits` behind a 60s cache, cleared on unlink so a
forgotten key can't keep reporting a balance. Both best-effort: `/credits` is
documented management-key-only but answers ordinary keys today, so its absence
downgrades detail, not availability. `limit: null` is preserved as null — that's
"no cap", and coercing it to 0 would render an empty gauge reading as "nothing
left". Live: $6.30 of $35.00, $0.00 this month.

**Per-endpoint truth + measured cost.** `/models/{author}/{slug}/endpoints`
answers "why is my context 200k and not 1M?" (the catalog reports the best
endpoint; a turn may route to a smaller one). `/generation?id=` reports what was
really billed, including cache reads/writes and long-context tiers an estimate
can't see — 404 is a normal answer there, since the record is written
asynchronously, so callers keep the estimate. Catalog fetch now prefers
`/models/user`, which respects the account's own provider/privacy filtering.

Also fixes a test double: a bare MagicMock auto-creates `resolve_catalog_id`, so
`test_route_reports_an_unlisted_model_as_unknown` started reporting every unknown
model as known. Caught by `task test:sterile` running the full module.

Deferred to backlog: automatic ledger reconciliation against `/generation` (a
background job, not an inline per-turn call — that would add a round trip to
every turn).

1217 API tests (17 new) OK normally and under `task test:sterile`; 437 client
tests (4 new); pyright 0; tsc clean; docs gates green (161 routes in parity).

Assisted-by: Opus 5
@vercel

vercel Bot commented Aug 1, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
agentx-docs-site Ready Ready Preview Aug 1, 2026 11:35pm

@QR-Madness
QR-Madness merged commit 9f04acc into master Aug 1, 2026
4 checks passed
@QR-Madness
QR-Madness deleted the feat/openrouter-intelligence branch August 1, 2026 23:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant