Skip to content

feat(litellm): add OrcaRouter as a named cloud provider route - #2934

Open
XiaoHuo888-hue wants to merge 1 commit into
Osmantic:mainfrom
XiaoHuo888-hue:add-orcarouter-provider
Open

feat(litellm): add OrcaRouter as a named cloud provider route#2934
XiaoHuo888-hue wants to merge 1 commit into
Osmantic:mainfrom
XiaoHuo888-hue:add-orcarouter-provider

Conversation

@XiaoHuo888-hue

Copy link
Copy Markdown

Summary

Adds a named orcarouter model route to the LiteLLM gateway so ODS users can route cloud traffic through OrcaRouter in cloud and hybrid modes — mirroring the existing MiniMax integration.

OrcaRouter is an OpenAI-compatible model gateway (https://api.orcarouter.ai/v1). It also runs gateway-level, zero-trust security for AI agents on the same endpoint — screening every prompt/response and governing every tool call on a default-deny basis, with no application code changes.

Changes

  • Runtime config renderer (ods/scripts/render-runtime-configs.py): add an orcarouter route (model: openai/orcarouter/auto, api_base: https://api.orcarouter.ai/v1, api_key: os.environ/ORCAROUTER_API_KEY) to the litellm-cloud, litellm-hybrid, and switchboard hybrid route lists.
  • Checked-in LiteLLM configs (ods/config/litellm/{cloud,hybrid,switchboard}.yaml): add the matching orcarouter route so renderer contract test_checked_in_mode_configs_match_renderer stays byte-identical.
  • Env wiring: new ORCAROUTER_API_KEY in .env.schema.json (secret: true), .env.example, Linux/macOS/Windows installer env generators, phase 06 key preservation, phase 09 offline sanitization, and LiteLLM compose env passthrough.
  • Docs: LiteLLM README (provider list, cloud example, env table, architecture diagram), docs/MODE-SWITCH.md (cloud provider table + env table).
  • Tests: new renderer tests asserting the route is present in cloud/hybrid/switchboard, and ORCAROUTER_API_KEY added to the dashboard-api settings-env secret-key regression test.
  • Changelog: entry under [Unreleased].

AI Assistance

This PR was drafted with AI assistance (change planning, test authoring, and PR description). All changes were reviewed by the human author.

Release Lane

  • Stable hotfix targeting release/2.6.x
  • Mainline change targeting main
  • Next-minor work targeting the next feature/minor release
  • Not sure; reviewer should help classify

Stable hotfix reason:

N/A

Changed Surface

  • Docs only
  • Tests only
  • Dashboard UI
  • Dashboard API / host agent
  • Installer / bootstrap / lifecycle
  • Docker Compose / service manifests
  • Model routing / Hermes / capabilities
  • Network exposure / auth / proxy
  • Dependencies / runtime wiring

Risk And Validation

  • Risk level: Low
  • Validation run:
    • git diff --check
    • Markdown/link sanity for docs
    • Focused tests listed below
    • Dashboard lint/test/build
    • Extension audit / compose validation
    • Release-grade fleet or scoped hardware validation
    • Stable-lane patch validation, if targeting release/2.6.x
    • Not required because:

Commands/results:

$ python3 tests/test-render-runtime-configs.py        # 25 PASS (incl. 2 new OrcaRouter tests)
$ python3 tests/test-runtime-config-wiring.py         # 7 PASS
$ python3 -m pytest extensions/services/dashboard-api/tests/   # 2118 passed, 1 skipped
$ bash tests/test-install-docs.sh; bash tests/test-tier-map.sh; bash tests/test-external-services.sh   # PASS
$ make lint (bash -n all .sh + py_compile)            # PASS
$ git diff --check                                    # clean

L3 live test: POST https://api.orcarouter.ai/v1/chat/completions with
model orcarouter/auto and a real ORCAROUTER_API_KEY → HTTP 200, valid completion.

Note: tests/contracts/test-installer-contracts.sh reports a single FAIL on a
Podman/systemd lifecycle case ("forced session restart selected the stale system
unit") — this reproduces identically on unmodified main and is an environment
limitation (no systemd in this container), unrelated to this change.

Operational Change Check

  • This is not an operational change.
  • This is an operational change and validation is recorded above.
  • This is an operational change and validation is intentionally deferred for:

The new route is additive and only activates when the user sets
ORCAROUTER_API_KEY and selects the orcarouter model name.

Notes For Reviewers

This mirrors the existing MiniMax provider route exactly. The route is inert
until the user supplies ORCAROUTER_API_KEY in .env.

Disclosure: I'm an engineer on the OrcaRouter team.

Add an 'orcarouter' model route to the LiteLLM gateway so ODS users can
route cloud traffic through OrcaRouter (https://www.orcarouter.ai) in
cloud and hybrid modes, mirroring the existing MiniMax integration.

The route targets https://api.orcarouter.ai/v1 with model orcarouter/auto
and is backed by a new ORCAROUTER_API_KEY env var. Wire the key through
the runtime config renderer, checked-in LiteLLM configs, compose env
passthrough, env schema/example, Linux/macOS/Windows installers, and
offline sanitization. Add renderer + settings-env tests.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: XiaoHuo888-hue <jinhao.song@myflashcloud.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant