Skip to content

refactor: consolidate OpenAI-compatible provider chat into Nous.Provider macro#65

Merged
nyo16 merged 1 commit into
masterfrom
refactor/provider-chat-consolidation
Jun 27, 2026
Merged

refactor: consolidate OpenAI-compatible provider chat into Nous.Provider macro#65
nyo16 merged 1 commit into
masterfrom
refactor/provider-chat-consolidation

Conversation

@nyo16

@nyo16 nyo16 commented Jun 27, 2026

Copy link
Copy Markdown
Owner

Summary

Six providers — vLLM, SGLang, LM Studio, Mistral, OpenAICompatible, Custom
carried near-identical chat/2 + chat_stream/2 implementations. The local trio
(vLLM/SGLang/LM Studio) were byte-for-byte structurally identical except for an
id, base URL, env prefix, and display name (a clear three-strikes duplication).

This extends use Nous.Provider with an opt-in chat: config that injects the
shared chat/2, chat_stream/2, base-URL resolver, and header helpers. The
duplication reduces to three variation axes:

Option Values Used by
base_url: :plain · :local · :required hosted · local servers · user-supplied URL
headers: :bearer · :bearer_org with/without openai-organization
timeout: / stream_timeout: ints replaces duplicated @default_timeout/@streaming_timeout

Each provider collapses to a moduledoc + a single use block. Adding a new
OpenAI-compatible provider is now just the chat: option.

Changes

  • Nous.Provider macro: new chat_ast/1 injects overridable chat/2,
    chat_stream/2, chat_resolve_base_url/1, and chat_build_headers/2.
  • vLLM / SGLang / LM Studio / Mistral / OpenAICompatible / Custom: removed
    duplicated bodies, replaced with chat: config.

7 files changed, 233 insertions(+), 329 deletions(-) — ~250 lines of
copy-pasted logic replaced by ~190 lines of reusable macro.

Behavior preservation

This is a behavior-preserving refactor:

  • Error message strings reproduced exactly (generated from id/env where
    they were provider-specific — e.g. Custom's SSRF and "requires a base_url"
    messages).
  • One deliberate consistency fix: the local trio now forwards finch_name in
    chat_stream, matching the convention already used by the 7 other providers.
    Previously they silently dropped it — they were the outliers.

Verification

  • ✅ Provider tests: 197 passed
  • ✅ Full suite: 1896 passed, 102 excluded
  • mix credo --strict: 0 issues
  • mix compile --warnings-as-errors: clean
  • mix format --check-formatted: clean

…der macro

Six providers (vLLM, SGLang, LM Studio, Mistral, OpenAICompatible, Custom)
carried near-identical chat/2 + chat_stream/2 implementations — the local
trio (vLLM/SGLang/LM Studio) were byte-for-byte structurally identical
except for an id, base URL, env prefix, and display name.

Extend `use Nous.Provider` with an opt-in `chat:` config that injects the
shared chat/2, chat_stream/2, base-URL resolver, and header helpers. The
duplication reduced to three variation axes:

  * base_url: :plain | :local | :required
  * headers:  :bearer | :bearer_org
  * timeout: / stream_timeout:  (replaces duplicated module-attr constants)

Each provider collapses to a moduledoc + a single `use` block.

Behavior-preserving:
  * Error message strings reproduced exactly (generated from id/env where
    they were provider-specific).
  * Local trio now forwards finch_name in chat_stream, matching the
    convention already used by the other 7 providers (they were the
    outliers that dropped it).

Verified: provider tests 197 passed, full suite 1896 passed,
credo --strict 0 issues, compile --warnings-as-errors clean, format clean.
@nyo16 nyo16 merged commit b7d93d5 into master Jun 27, 2026
6 checks passed
@nyo16 nyo16 deleted the refactor/provider-chat-consolidation branch June 27, 2026 12:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant