Skip to content

fix(models): let you raise the context window on a model whose provider publishes none - #259

Open
bman654 wants to merge 1 commit into
mainfrom
feat/user-context-override
Open

bman654 wants to merge 1 commit into
mainfrom
feat/user-context-override

Conversation

@bman654

@bman654 bman654 commented Sep 19, 2026

Copy link
Copy Markdown
Owner

Why

A model whose provider publishes no context window was given clodex's invented 200,000 and
persisted as if the server had said so. The user could not raise it:

const ceiling = positiveInteger(limits.maxContextWindow) ?? standardRaw;
const raw     = Math.min(requested, ceiling);

With no maxContextWindow, the ceiling collapsed onto the guess, so
clodex models --context <model>=1m --save saved an inert preference and silently resolved back to
200,000. Measured on a real provider: a model stored at 200,000 against a real 1,048,576 — 848,576
tokens the user could not reach.

What changed

Three tiers sit behind a window: the curated models.dev cache, the id heuristics, and a flat
200,000 when nothing matches. Only the third is invention. A new lookupKnownContextWindow reports
it as absent rather than inventing a number, and the write sites stop persisting it.
resolveContextWindow and lookupContextWindow keep their exact signatures and answers, so no read
site changed.

An absent window is no longer treated as a ceiling. Every real one still clamps.

o1, o3 and o1-mini declared no window and matched no heuristic rule, so they were persisting
at the invented default; they now carry their real windows (200,000 / 200,000 / 128,000) and keep a
correct ceiling from data rather than from invention.

Clamps preserved — measured, not argued

Base vs head through the real buildDesiredPatchConfig, same fixtures both sides:

fixture base raw head raw
declared 272k + max 872k, stop 5m 872,000 same
heuristic-only model, stop 1,048,576 262,144 same
declared 272k, stop max 272,000 same
declared 1,048,576, stop 2m 1,048,576 same
no window + max 872k, stop 900k 872,000 same
legacy stored 200,000, stop 1m 200,000 same (still clamped)
no window, stop 1m 200,000 1,000,000 ← the fix
no window + 50% share, stop 900k 200,000 raw / 100,000 900,000 / 450,000 ← the fix

The only ceiling removed is the flat default standing in for a model nothing declares.

Upgrade safety

The patch config hash does not move for any install that has not already saved a stop above the
default on such a model — verified by computing it at base and head over nine fixtures, with
PATCH_TRANSFORMS_VERSION pinned so only the model config could move the digest. Seven are
byte-identical; the two that move are exactly the population this fix exists for.

That result rests on buildPatchModelConfig treating an absent window and an explicit 200,000
identically — both omit context. A comment now records that, because a future change
distinguishing them would mark every patched install stale.

Existing installs keep their stored guess until clodex providers refresh-models <provider>, which
replaces the cache wholesale; an already-saved stop then takes effect with no further action. The
README says so.

Tests

19 mutants, all killed in full-file runs; feature deletion reds 18 tests, and the ones that stay
green are exactly the under-scope guards pinning preserved behaviour. Suite run in normal and
shuffled order (files and tests, --no-file-parallelism, private Vite cache): 423/423 both, so no
order dependence despite the module-scope memoization in context-window.ts and context-modes.ts.

pnpm typecheck clean · 2943 passed, 20 skipped (2963) · pnpm build success.

Known, and deliberate

  • A heuristic window stays a hard ceiling. It is inference rather than the provider's word, but
    unlike the flat default it is a claim about that model, and it stops a user baking a window the
    server will reject. Revisit only with an endpoint proven to accept more.
  • clodex patch still emits no warning for a model that is in the catalog with no published
    window; unknownWindows only fires for a favorite whose model or provider has left the registry.
    Making it fire needs provenance carried through PatchModelMeta — filed separately rather than
    widened here.

…er publishes none

A model whose provider publishes no context window was given clodex's invented 200,000 and
persisted as if the server had said so. `clodex models --context <model>=1m --save` then could not
raise it: the stop was clamped back to the guess, because an absent ceiling fell back to the
stored window. Saving a stop now works for these models, and `clodex patch` bakes it in.

Three tiers sit behind a window: the curated models.dev cache, the id heuristics, and a flat
200,000 default when nothing matches. Only the third is invention, so only it stops being
persisted; `lookupKnownContextWindow` reports it as absent instead. An absent window is no longer
treated as a ceiling, while every real one still clamps — a provider-declared window, the account
ceiling, the curated tier, the heuristic tier, and a 200,000 an older clodex already stored.

Reachable through `providers add` for any OpenAI-compatible server whose model list omits the
field, a custom Anthropic endpoint, or an OpenCode Go endpoint added under a non-canonical provider
id. The retained `opencode-go` provider is not affected: its template is allowlist-only and every
shipped entry declares a window.

o1, o3 and o1-mini declared no window and matched no rule, so they were being persisted at the
invented default. They now carry their real windows and keep a correct ceiling.

Existing installs keep the stored guess until `clodex providers refresh-models <provider>`, which
replaces the cache wholesale and lets an already-saved stop take effect. The patch config hash is
unchanged for every install except those that have saved a stop above the default on such a
model — `buildPatchModelConfig` treats an absent window and an explicit 200,000 identically, and a
comment there records that this equality is what keeps upgrades from marking installs stale.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant