OrcaRouter as an optional provider for understory's generic OpenAI-compatible model config
This is a proposal issue, not a code change yet.
Why I'm opening this
understory solves the problem I've hit in agent projects: an agent's memory only stays useful if it is maintained, not just stored. Your design rule — "conformance is enforced in code, not prompts" — is what makes that trustworthy enough to let an LLM rewrite a knowledge base at all: the deterministic bundle layer validates frontmatter, regenerates index.md, appends log.md, and sandboxes every path, so the model decides what changes while code guarantees a conformant OKF bundle. Auto-discovering the loaded model from OpenAI-compatible /v1/models endpoints (so a llama.cpp or llama-swap endpoint works with no hardcoded model id) is a thoughtful fit for an agent-memory tool. I think your users would value more choices about which model does that maintenance work.
The concrete ask
I'm an engineer on the OrcaRouter team. I'd like to propose adding OrcaRouter as an optional provider — not replacing anything. understory already configures any OpenAI-compatible endpoint through LLM_API_BASE_URL + LLM_API_KEY + LLM_API_FORMAT + LLM_MODEL (the Vercel AI SDK createOpenAICompatible path in packages/core/src/providers/index.ts), and OrcaRouter exposes an OpenAI-compatible API with standard API-key authentication. The integration would likely be config-only: LLM_API_BASE_URL=https://api.orcarouter.ai/v1 with an OrcaRouter API key and any OrcaRouter model id, with the existing LLM_FALLBACK_* variables still available for failover. Nothing here is implemented or tested yet — this is a request for your feedback first.
What OrcaRouter would give understory's users
Two capabilities fit understory's workflow directly:
- One endpoint, many models. A single OpenAI-compatible base URL covers 200+ models (Claude, GPT, Gemini, DeepSeek, Qwen, and open-weights models). For users who already run local models, OrcaRouter can be the cloud complement in the fallback slot — keep
LLM_API_BASE_URL pointing at llama.cpp and put OrcaRouter in LLM_FALLBACK_API_BASE_URL, so a local outage or 5xx triggers a hosted model instead of a failed memory_maintain.
- Prompt caching and per-key usage tracking. Repeated
memory_query calls already hit understory's own query cache; gateway-level caching complements that, and per-request observability makes it easier to see what a maintenance run actually costs before setting budgets.
Full transparency on the partner angle: OrcaRouter has an optional open-source partner program where approved OSS projects receive a 5% revenue share from OrcaRouter usage attributed to their integration. Taking part is not a prerequisite for the integration, and I'm happy to follow any disclosure or governance requirements understory has — maintainers should feel free to decline that side entirely.
Context and next step
OrcaRouter is already in the open-source ecosystem — RAGFlow, Dify, NocoBase, DB-GPT, and models.dev / OpenCode are among the projects in its orbit — so this is a well-worn path. More context: https://www.orcarouter.ai/built-with.
If this sounds reasonable, I'd welcome your thoughts — and with your go-ahead I'd be glad to submit an implementation PR (docs + env example updates, plus a short OrcaRouter entry in the README provider list) for review. Thanks for reading.
OrcaRouter as an optional provider for understory's generic OpenAI-compatible model config
This is a proposal issue, not a code change yet.
Why I'm opening this
understory solves the problem I've hit in agent projects: an agent's memory only stays useful if it is maintained, not just stored. Your design rule — "conformance is enforced in code, not prompts" — is what makes that trustworthy enough to let an LLM rewrite a knowledge base at all: the deterministic bundle layer validates frontmatter, regenerates
index.md, appendslog.md, and sandboxes every path, so the model decides what changes while code guarantees a conformant OKF bundle. Auto-discovering the loaded model from OpenAI-compatible/v1/modelsendpoints (so a llama.cpp or llama-swap endpoint works with no hardcoded model id) is a thoughtful fit for an agent-memory tool. I think your users would value more choices about which model does that maintenance work.The concrete ask
I'm an engineer on the OrcaRouter team. I'd like to propose adding OrcaRouter as an optional provider — not replacing anything. understory already configures any OpenAI-compatible endpoint through
LLM_API_BASE_URL+LLM_API_KEY+LLM_API_FORMAT+LLM_MODEL(the Vercel AI SDKcreateOpenAICompatiblepath inpackages/core/src/providers/index.ts), and OrcaRouter exposes an OpenAI-compatible API with standard API-key authentication. The integration would likely be config-only:LLM_API_BASE_URL=https://api.orcarouter.ai/v1with an OrcaRouter API key and any OrcaRouter model id, with the existingLLM_FALLBACK_*variables still available for failover. Nothing here is implemented or tested yet — this is a request for your feedback first.What OrcaRouter would give understory's users
Two capabilities fit understory's workflow directly:
LLM_API_BASE_URLpointing at llama.cpp and put OrcaRouter inLLM_FALLBACK_API_BASE_URL, so a local outage or 5xx triggers a hosted model instead of a failedmemory_maintain.memory_querycalls already hit understory's own query cache; gateway-level caching complements that, and per-request observability makes it easier to see what a maintenance run actually costs before setting budgets.Full transparency on the partner angle: OrcaRouter has an optional open-source partner program where approved OSS projects receive a 5% revenue share from OrcaRouter usage attributed to their integration. Taking part is not a prerequisite for the integration, and I'm happy to follow any disclosure or governance requirements understory has — maintainers should feel free to decline that side entirely.
Context and next step
OrcaRouter is already in the open-source ecosystem — RAGFlow, Dify, NocoBase, DB-GPT, and models.dev / OpenCode are among the projects in its orbit — so this is a well-worn path. More context: https://www.orcarouter.ai/built-with.
If this sounds reasonable, I'd welcome your thoughts — and with your go-ahead I'd be glad to submit an implementation PR (docs + env example updates, plus a short OrcaRouter entry in the README provider list) for review. Thanks for reading.