Conversation
0x0079
force-pushed
the
claude/opencode-model-endpoint-format
branch
from
September 8, 2026 01:57
85c124b to
4f7b94b
Compare
0x0079
changed the base branch from
claude/opencode-go-issues-vt45zi
to
main
September 8, 2026 01:57
0x0079
force-pushed
the
claude/opencode-model-endpoint-format
branch
from
September 8, 2026 02:18
c9ab28e to
51b53f2
Compare
OpenCode Zen decides the wire format per model, not per provider: 32 of its 35 models answer only on /chat/completions, while gpt-5.6-luna, grok-4.5 and grok-4.6 answer only on /responses. The provider-level OpenAIEndpointMode cannot say that, so a Codex client asking for luna was downgraded to Chat and got a bare 500, and declaring the provider "both" would have been worse — Anthropic-shaped requests count as Responses incoming, so Claude Code on kimi/glm would have been routed to /responses and rejected. Add a fourth mode, per_model, and let it learn. It guesses Chat (the safe default and the 32/35 case), and on a rejection that looks like a format mismatch retries the same service once on the other endpoint, remembering the answer for 8h in memory. Every other mode is untouched and still cannot spend an extra round-trip. The declaration is what bounds this. The old AdaptiveProbe (PR #976) failed because it probed every provider at cold start, burned tokens, blocked 10s and cached failures; here nothing happens until a provider is known to be per-model, the retry rides a real request, only verified successes are stored, and nothing is persisted. The mode is derived rather than declared, because the declaration cannot reach these providers: ProviderTemplate.OpenAIEndpointMode is only copied onto a Provider by the OAuth path, so a provider created through the normal API keeps the zero value — which is why openai-com's "both" has never taken effect either. Deriving it from the API base makes existing, new and hand-added providers work without a migration or a restart. Fixing the template plumbing is the deeper fix, but it would change openai-com's behavior and does not belong here. Tolerating a bare 5xx as a mismatch signal is load-bearing and only safe under that gate: Zen reports one condition three ways, and luna's rejection carries no marker at all. A 4xx still needs an explicit marker, and neither a gateway-side setup failure nor an endpoint the rule pinned is retried. The retry runs before the failure is charged, so it costs neither a failover tier nor a breaker strike, and it reuses the existing firstChunkGate rather than nesting a second buffer — which would break the first-chunk signal. A failed retry never masks a retryable original, so a sibling service still gets its turn. Verified against the live upstream (endpoint_learning_e2e_test.go, skipped without OPENCODE_API_KEY): luna is 500 on Chat and 200 on Responses, kimi-k3 the mirror image, and both rejections are recognized by the matcher. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LS5eJDtN9bfnVXPde2CNwk
0x0079
force-pushed
the
claude/opencode-model-endpoint-format
branch
from
September 8, 2026 06:52
51b53f2 to
745c27e
Compare
Review pass over the per_model feature. No behavior change beyond the last two points. One decision object instead of four context keys. The loop↔attempt contract (resolved endpoint, forced endpoint, pinned, setup-failed) is one value with one lifetime, so it travels as one struct with one reset, and the three type-asserting accessors go away. It also memoizes EffectiveEndpointMode, which derives from URL parsing and was recomputed four or five times per request for every provider, not just OpenCode's. One map in the store instead of two. "A success ends the cooldown" was an invariant between two containers kept in step by hand; as one entry holding both the learned endpoint and the last failed retry, it is structural. The key is now loadbalance.FormatServiceID, the same service key the breaker and usage tracking use, and time comes from internal/clock so learned entries expire with breaker and affinity state under the simulator instead of on the wall clock. Writes sweep dead entries past a threshold: model names come from the request, so the map was unbounded. One predicate, DispatchMayRetry, instead of the same condition spelled two ways in five places — the dispatch loop deciding whether to buffer, and four handlers deciding whether to snapshot the request. They must agree: if they drift, a retry re-transforms a request the first attempt already mutated. The retry returns the status to judge rather than a bool the caller reconciles through a second function, and the empty-service-list case goes back through the fast path instead of widening the loop bound for all traffic. Two behavior changes fall out of that predicate. Buffering and snapshotting now stop once the endpoint is learned — for a Claude-Code-sized request body the snapshot is a full JSON round-trip, and after the answer is known there is no second attempt to serve. So that a mapping which later goes stale cannot pin a model to a dead endpoint for the rest of the TTL, forgetStaleEndpoint drops it on a 5xx and the next request re-learns. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LS5eJDtN9bfnVXPde2CNwk
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
OpenCode Zen decides the wire format per model: 32 of its 35 models answer only on
/chat/completions, whilegpt-5.6-luna,grok-4.5andgrok-4.6answer only on/responses.OpenAIEndpointModeis a per-provider fact and cannot say that, so a Codex client asking for luna was downgraded to Chat and got a bare 500.Key Changes
per_model: declares "both endpoints exist, but each model picks one" — notboth, where mirroring the client's protocol is always safe.per_modelcannot spend an extra round-trip, reach the mismatch matcher, or write to the memory — the containment the oldAdaptiveProbelacked.firstChunkGate(nesting a second buffer would break the first-chunk signal).normalizeOpenCodeEndpointModesets the mode on every Zen provider, matching on API base so hand-added and old-snapshot providers are covered too.Notes
endpoint_learning_e2e_test.go, skipped withoutOPENCODE_API_KEY): luna 500/200, kimi-k3 200/401, both rejections recognized by the matcher. Rationale and the Adaptive comparison are in.design/openai-endpoint-routing.md§8.🤖 Generated with Claude Code
https://claude.ai/code/session_01LS5eJDtN9bfnVXPde2CNwk