Skip to content

modelrunner provider - #7

Open
hakankaan wants to merge 1 commit into
flexigpt:mainfrom
hakankaan:modelrunner-provider
Open

hakankaan wants to merge 1 commit into
flexigpt:mainfrom
hakankaan:modelrunner-provider

Conversation

@hakankaan

Copy link
Copy Markdown

Upfront disclosure: I work on ModelRunner, so this PR adds my own company to the catalog. Written with Claude Code; I've read and run everything in it. Close it without ceremony if catalog entries aren't something you take from outside.

What this adds

A modelrunner provider preset. It's an OpenAI chat-completions gateway — one key in front of models from several labs — so it needs no new SDK type and no code outside the catalog:

SDKType ProviderSDKTypeOpenAIChatCompletions
Origin https://queue.modelrunner.run
ChatCompletionPathPrefix spec.DefaultOpenAIChatCompletionsPrefix
APIKeyHeaderKey spec.DefaultAuthorizationHeaderKey
Model names the platform endpoint id, owner/alias — the only value the chat route accepts

Six presets, reusing the existing Preset* ids the way provider_openrouter.go does: gemini35Flash, gemini37Flash, gemini35FlashLite, deepseekv4pro, zaiglm52, qwen38Max. Context and output lengths are the platform's published per-model values, not estimates.

Four files: the new provider, the catalogProviders registration, and the two manifests that are meant to be updated when a provider is added (catalog_model_membership_test.go, catalog_test.go).

Capability declarations, and why several are false

The provider-level override is the intersection that holds for every listed model; the one image-capable entry overrides its own modalities rather than widening the provider.

  • SupportsReasoningConfig: false. Several of these models think, and some expose an effort control upstream — but the gateway documents no reasoning parameter of its own and the models behind it don't agree on a spelling. Offering the control would work for part of the list and fail for the rest, so it's declared unsupported. This is the declaration I'd most expect you to push back on; happy to revisit it if you'd rather the catalog be optimistic here.
  • Tools: function with Auto / None, SupportsParallelToolCalls: false, MaxForcedTools: 0. Function tool calling is what the gateway documents and what its own conformance run covers. Forced and parallel calls are pass-through in the wire format but unverified per model, so I left them off rather than inferring capability from the request schema.
  • SupportsAutomaticCaching: true, no cache-control. Caching happens upstream and is billed at its own rate on every model here; there's no cache_control parameter to set, so the TopLevel block is all false.
  • Output: Text + JSONSchema. response_format is honoured where the model supports it.

Testing

  • go build ./... — clean.
  • go test ./...all packages pass, matching the main baseline I took before starting.
  • gofmt -l . — clean.
  • golangci-lint run ./...0 issues (2.13.2; your CI pins v2.13.0).

Two things your own validation caught that I'd otherwise have shipped wrong, which is a nice advert for those manifests:

  • validateModel requires either Reasoning or Temperature to be set. With reasoning config declared unsupported, five of six presets failed until each got an explicit Temperature — now new(1.0), matching the OpenRouter gateway presets.
  • TestCatalogContainsAllRegisteredProviders and TestCatalogModelMembershipIsExhaustive both failed until updated, exactly as their comments promise.

The flexigpt-app side

flexigpt-app reads its built-in providers entirely from this catalog (populateDataFromInferenceCatalog), so nothing there can be wired until this is tagged. Once it is, the app-side follow-up is one small commit — the name constant, BuiltInProviderNames, BuiltInProviderTimestamps, a default preset id, and the go.mod bump. Say the word and I'll open it.


Happy to revise anything here — the preset selection, the capability declarations, or the naming. Leave a comment and I'll push a change.

🤖 Generated with Claude Code

An OpenAI chat-completions gateway: one key in front of models from several
labs, so the provider-level capabilities are the intersection that holds for
every listed model, with the one image-capable entry overriding its own
modalities.

Reasoning config, forced and parallel tool calls are declared unsupported
rather than assumed from the wire format — the gateway documents no reasoning
parameter and the models behind it do not agree on a spelling, so offering
the control would only work for part of the list. Prompt caching is automatic
and billed at its own rate on every model, with no cache-control parameter to
set.

Context and output lengths are the platform's published values per model.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant