feat(provider): OpenAI-compatible provider - #553
Open
espetro wants to merge 11 commits into
Open
Conversation
…ial source Extend ProviderId with .anthropic and CredentialSource with .anthropic_api_key, mirroring the openai/openai_api_key pair from the OpenAI-compatible transport. Credentials resolve from the ANTHROPIC_API_KEY environment variable or a profile-owned anthropic_api_key setting, never from repository config. Settings store gains the anthropic_model preference and config_runtime validates it alongside the existing per-provider model fields.
Implement the Anthropic Messages wire as a stream_provider.Provider:
POST {base}/v1/messages with x-api-key and anthropic-version headers,
top-level system hoisting, tool_use/tool_result blocks, and SSE
parsing for message_start, content_block_start/delta/stop (text,
thinking, and input_json deltas), message_delta, message_stop, and
error events. Stop reasons map to the shared failure taxonomy
(end_turn to stop, max_tokens to length, refusal to content_filter).
Assistant messages carrying both text and tool calls serialize the
tool_use blocks inside the still-open content array; closing it
before the blocks produced invalid JSON on every agentic replay.
Base URL defaults to https://api.anthropic.com and is overridable
via FX_ANTHROPIC_BASE_URL, with FX_E2E_ANTHROPIC_URL reserved for
loopback e2e fixtures. Boundaries match the existing transports
(1MB SSE line, 64MB aggregate, 128 tool calls, 4MB tool arguments).
anthropic_models.zig fetches /v1/models with the same caps as the
grok catalog and exports the standard model_catalog_provider pair.
Both files keep credential acceptance keyed by name so they compile
independently of the enum-case commit.
…faces Register the anthropic bundle in builtins/providers.zig and thread it through every surface the openai bundle reaches: the auth provider picker (choiceAt now yields the fifth provider entry), acp server and subagent prompt routing, cli_ask and cli_surface config structs, doctor diagnostics, output contracts, credential resolution call sites (profile anthropic key propagated alongside the openai key), and main.zig dependency injection for both the entry config and the ask/acp runner paths. Anthropic carries no permission reviewer yet, matching the optional reviewer slots; openai_compatible.zig rejects the anthropic_api_key source symmetrically. The model menu lists the anthropic credential label. Agent streams flow through the same ProviderRoutes dispatch as the other wire protocols.
… unset configuredProviderSelection returned ModelNotSelected whenever the saved settings lacked a model for the active provider, even when FX_MODEL carried a process override, making 'FX_PROVIDER=openai FX_MODEL=...' fail before the stream provider was ever consulted. Fall back to the trimmed FX_MODEL value before erroring for the non-gateway providers (anthropic included) and scope each TestEnv install in the override test to its own block so the FX_PROVIDER assertions no longer inherit a stale environment. Verified live against an OpenAI-compatible gateway on both the openai and anthropic wires, including a tool-calling round trip.
Gateway already resolves a compiled default model, so consulting FX_MODEL before the default changed the reported configured_model and broke the startup env-override test.
Requests on the anthropic wire returned ProviderAdmissionMissing at the gateway_step boundary because the codec never called admission.admit() after serialization, unlike the other transports.
Registers the openai-compatible provider id (aliases: openai, openai-compatible), its auth catalog entry, and the openai_model settings field mirroring the other provider-scoped model settings. Exhaustive switches across auth, runtime, and cli surfaces gain minimal openai_compatible arms so the provider id compiles everywhere; transport wiring lands in the following commits.
…ider Resolves the openai-compatible provider from OPENAI_API_KEY or LITELLM_API_KEY in the environment first, then falls back to the stored API key, tagging resolved credentials as .stored_key so no new CredentialSource variant is needed. Adds provider-specific missing credential guidance mirroring the anthropic arm.
…t and model catalog Ports pr-168's chat-completions codec to the current provider ABI: request bodies build via openai_json/openai_tools, the SSE stream reduces through an EventSink in openai_client, and openai_compatible implements stream_provider with admission.admit() before HTTP open and delivery.markPossiblySent() before send. Base URL resolves from FX_OPENAI_BASE_URL (default https://api.openai.com/v1) with a loopback- restricted e2e override. The /models catalog provider reuses the same bounded HTTP helpers. Responses-API remnants from pr-168 are cut.
… cli surfaces Adds the .openai_compatible arm to the provider_set native Set with agent_stream and model catalog providers wired from the ported codec. Live smoke tested via Bifrost openai wire (text + tool loop) using OPENAI_API_KEY and FX_OPENAI_BASE_URL env vars.
|
Security review: https://reviewer.vercel-security.com/vercel-labs/fx/pull/553 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds OpenAI-compatible (Chat Completions) as a first-class inference provider on top of the current provider_set architecture (stacked on #536).
What
ProviderId.openai_compatibleenum case + parse aliases, provider catalog entry,openai_modelsettings field, stored-key credential resolution fromOPENAI_API_KEY/LITELLM_API_KEYenvsrc/gateway/openai_compatible.zigChat Completions transport (POST /chat/completions, SSE streaming, Bearer auth, admission + delivery signaling, tool_use/tool_calls fragments reducer) andopenai_compatible_models.zigmodel catalog (GET /v1/models)provider_set.SetBundle, cli surfaces, ACP, subagent, doctor, auth runtime, footer model menuOpenAiModelNotSelectederrorVerification
zig build test: only the 6 pre-existing main failures (tool_runtime, command_runner, workspace_files) — verified in pristine worktreeOPENAI_API_KEYandFX_OPENAI_BASE_URL=http://localhost:8317/openai: single-turn text and a tool-calling round trip (shell exec) both workNotes for reviewers
FX_MODELenv for the model id when switching providers via settings, oropenai_modelkeyopenaioropenai-compatible(both parsed viamodel_provider.parse)