feat(onboard): add custom OpenAI-compatible provider option#624
feat(onboard): add custom OpenAI-compatible provider option#624andy-ratsirarson wants to merge 1 commit intoNVIDIA:mainfrom
Conversation
📝 WalkthroughWalkthroughAdds support for user-provided OpenAI-compatible inference endpoints across onboarding (interactive and non-interactive), CLI provider selection, runtime provider creation, docs, tests, and blueprint updates; Changes
Sequence DiagramsequenceDiagram
actor User
participant Onboard as "bin/lib/onboard.js"
participant Cred as "Credential Store"
participant Config as "bin/lib/inference-config.js"
participant Shell as "OpenShell CLI/API"
participant Blueprint as "Blueprint YAML"
User->>Onboard: select "Custom OpenAI-compatible"
activate Onboard
alt Interactive
Onboard->>User: prompt base URL
User-->>Onboard: base URL
Onboard->>Cred: save base URL
Onboard->>Cred: lookup API key for base URL
alt no saved key
Onboard->>User: prompt API key
User-->>Onboard: API key
Onboard->>Cred: save API key
end
Onboard->>User: prompt model
User-->>Onboard: model
else Non-interactive
Onboard->>Onboard: read NEMOCLAW_CUSTOM_* env vars and validate
Onboard->>Cred: save base URL and API key
end
Onboard->>Config: build provider selection config (provider: custom, model)
Config-->>Onboard: config
Onboard->>Shell: provider create/update (type: openai, base URL, API key)
Shell-->>Onboard: provider created/updated
Onboard->>Shell: inference set --provider custom-provider --model <model> --no-verify
Shell-->>Onboard: inference routing set
Onboard->>Blueprint: update blueprint/profile with custom provider info
Blueprint-->>Onboard: updated
deactivate Onboard
Onboard-->>User: custom provider ready
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
16cf3d4 to
4bc9621
Compare
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (2)
nemoclaw-blueprint/blueprint.yaml (1)
57-63: Exposecustomin the blueprint's declared profile list.This adds
components.inference.profiles.custom, but the top-levelprofiles:array still omitscustom. Anything that enumerates declared profiles will miss the new provider.YAML snippet
profiles: - default - ncp - nim-local - vllm - custom🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@nemoclaw-blueprint/blueprint.yaml` around lines 57 - 63, Add the new custom provider to the blueprint's declared profiles by adding "custom" to the top-level profiles array so it matches components.inference.profiles.custom; update the profiles list (the YAML scalar sequence under the key profiles) to include the entry custom alongside default, ncp, nim-local, and vllm.test/inference-config.test.js (1)
59-76: Please cover the onboarding branch, not just the config mapper.These cases only pin
getProviderSelectionConfig(). The new env-var validation and credential persistence forcustomlive inbin/lib/onboard.js, and that path is still listed as pending in the PR objectives. A non-interactivesetupNim()test with and withoutNEMOCLAW_CUSTOM_*would catch regressions there.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@test/inference-config.test.js` around lines 59 - 76, Add tests that exercise the onboarding flow in bin/lib/onboard.js by invoking setupNim() non-interactively to cover the onboarding branch (not just getProviderSelectionConfig). Specifically, add two tests: one with NEMOCLAW_CUSTOM_* env vars set and one without; for each, call setupNim() (or the exported function that runs the onboarding path) and assert that env-var validation behaves as expected and credentials are persisted/cleared appropriately. Use the same test file pattern (test/inference-config.test.js) and stub/mock any interactive prompts, file/system I/O, and process.env mutations so the tests remain deterministic; reference setupNim, getProviderSelectionConfig, and the NEMOCLAW_CUSTOM_* variables when locating the code to exercise. Ensure tests assert both validation errors when env vars are missing and successful credential persistence when present.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@bin/lib/onboard.js`:
- Around line 714-732: The onboarding flow currently reuses an existing
CUSTOM_PROVIDER_API_KEY even when the entered baseUrl (from prompt stored via
saveCredential("CUSTOM_PROVIDER_BASE_URL")) changes, causing a saved key to be
paired with the wrong endpoint; update the logic in this block to compare the
stored CUSTOM_PROVIDER_BASE_URL (via getCredential("CUSTOM_PROVIDER_BASE_URL"))
against the newly entered baseUrl and if they differ, treat the API key as
stale: prompt for a new API key (using prompt), overwrite
CUSTOM_PROVIDER_API_KEY with saveCredential("CUSTOM_PROVIDER_API_KEY", apiKey),
and then save the new CUSTOM_PROVIDER_BASE_URL; otherwise preserve the existing
behavior of using the saved key.
In `@README.md`:
- Around line 187-191: The README currently says "Get an API key from
build.nvidia.com" in the paragraph that also documents custom OpenAI-compatible
providers; update the text so that the NVIDIA key instruction is scoped to the
NVIDIA Endpoint option only. Locate the paragraph referencing "Get an API key
from build.nvidia.com" near the "Custom OpenAI-compatible" table and change it
to a qualified sentence such as "If using the NVIDIA endpoint, get an API key
from build.nvidia.com" or move that instruction under the NVIDIA Endpoint
subsection referenced by the "nemoclaw onboard" flow and the environment
variables (NEMOCLAW_PROVIDER, NEMOCLAW_CUSTOM_BASE_URL, NEMOCLAW_CUSTOM_API_KEY,
NEMOCLAW_MODEL) so users selecting a custom provider won’t be misled.
---
Nitpick comments:
In `@nemoclaw-blueprint/blueprint.yaml`:
- Around line 57-63: Add the new custom provider to the blueprint's declared
profiles by adding "custom" to the top-level profiles array so it matches
components.inference.profiles.custom; update the profiles list (the YAML scalar
sequence under the key profiles) to include the entry custom alongside default,
ncp, nim-local, and vllm.
In `@test/inference-config.test.js`:
- Around line 59-76: Add tests that exercise the onboarding flow in
bin/lib/onboard.js by invoking setupNim() non-interactively to cover the
onboarding branch (not just getProviderSelectionConfig). Specifically, add two
tests: one with NEMOCLAW_CUSTOM_* env vars set and one without; for each, call
setupNim() (or the exported function that runs the onboarding path) and assert
that env-var validation behaves as expected and credentials are
persisted/cleared appropriately. Use the same test file pattern
(test/inference-config.test.js) and stub/mock any interactive prompts,
file/system I/O, and process.env mutations so the tests remain deterministic;
reference setupNim, getProviderSelectionConfig, and the NEMOCLAW_CUSTOM_*
variables when locating the code to exercise. Ensure tests assert both
validation errors when env vars are missing and successful credential
persistence when present.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 1ed47649-c662-49b1-a651-cc98c4cd1610
📒 Files selected for processing (6)
DockerfileREADME.mdbin/lib/inference-config.jsbin/lib/onboard.jsnemoclaw-blueprint/blueprint.yamltest/inference-config.test.js
There was a problem hiding this comment.
Actionable comments posted: 2
♻️ Duplicate comments (1)
bin/lib/onboard.js (1)
714-732:⚠️ Potential issue | 🟠 MajorRefresh the saved API key when the base URL changes.
This still reuses
CUSTOM_PROVIDER_API_KEYwhenever one exists. Switching the custom endpoint can therefore pair the new base URL with the old key and leave the provider misconfigured. Also, because the new base URL is saved first, aborting the replacement-key prompt can make that stale key look reusable on the next rerun.Suggested fix
- const baseUrl = await prompt(" Base URL: "); + const baseUrl = (await prompt(" Base URL: ")).trim(); if (!baseUrl) { console.error(" Base URL is required."); process.exit(1); } - saveCredential("CUSTOM_PROVIDER_BASE_URL", baseUrl); - let apiKey = getCredential("CUSTOM_PROVIDER_API_KEY"); + const previousBaseUrl = getCredential("CUSTOM_PROVIDER_BASE_URL"); + let apiKey = + previousBaseUrl === baseUrl ? getCredential("CUSTOM_PROVIDER_API_KEY") : null; if (!apiKey) { - apiKey = await prompt(" API Key: "); + apiKey = (await prompt(" API Key: ")).trim(); if (!apiKey) { console.error(" API key is required."); process.exit(1); } saveCredential("CUSTOM_PROVIDER_API_KEY", apiKey); console.log(" Key saved to ~/.nemoclaw/credentials.json"); } else { console.log(" Using saved API key from credentials."); } + saveCredential("CUSTOM_PROVIDER_BASE_URL", baseUrl);🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@bin/lib/onboard.js` around lines 714 - 732, The code saves CUSTOM_PROVIDER_BASE_URL before handling the API key and always reuses CUSTOM_PROVIDER_API_KEY if present, which can mispair a new base URL with an old key; modify the flow in the onboarding logic (the prompt/saveCredential/getCredential sequence) so you first read the existing base URL via getCredential("CUSTOM_PROVIDER_BASE_URL") and if it differs from the newly entered baseUrl, clear or remove the stored API key (e.g. call saveCredential("CUSTOM_PROVIDER_API_KEY", null/empty or a delete function) or force re-prompt) before attempting to reuse/getCredential("CUSTOM_PROVIDER_API_KEY"); alternatively prompt the user to confirm reuse of the saved key when base URLs differ, and only save the new base URL via saveCredential("CUSTOM_PROVIDER_BASE_URL") after handling the API key update.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@bin/lib/onboard.js`:
- Around line 696-697: The current non-interactive flow persists CI secrets by
calling saveCredential("CUSTOM_PROVIDER_BASE_URL", baseUrl) and
saveCredential("CUSTOM_PROVIDER_API_KEY", apiKey); instead, modify setupNim() to
return the discovered custom { baseUrl, apiKey } when run non-interactively and
update setupInference() to accept an optional { baseUrl, apiKey } parameter so
the non-interactive caller passes the creds directly; keep saveCredential calls
only in the interactive branch (and remove or guard the saveCredential calls
currently invoked at the locations referenced by setupNim()/setupInference() and
the lines corresponding to 830-831) so pipeline secrets are never written to
disk.
- Around line 689-697: Validate and normalize the custom base URL before
persisting: replace the current direct checks around baseUrl/apiKey/model
(variables baseUrl, apiKey, model) and the calls to
saveCredential("CUSTOM_PROVIDER_BASE_URL", ...) with logic that constructs a URL
object (new URL(baseUrl)), canonicalizes it (e.g., remove trailing slash), and
enforces only secure endpoints — require protocol === "https:" or allow
loopback/localhost addresses explicitly (or require an explicit opt-in env var
to accept insecure http), otherwise console.error and process.exit(1); apply the
same validation/normalization to the other branch that saves
OPENAI_BASE_URL/OPENAI_API_KEY (the 714-719 area) so both paths use identical
URL parsing and rejection rules before calling saveCredential.
---
Duplicate comments:
In `@bin/lib/onboard.js`:
- Around line 714-732: The code saves CUSTOM_PROVIDER_BASE_URL before handling
the API key and always reuses CUSTOM_PROVIDER_API_KEY if present, which can
mispair a new base URL with an old key; modify the flow in the onboarding logic
(the prompt/saveCredential/getCredential sequence) so you first read the
existing base URL via getCredential("CUSTOM_PROVIDER_BASE_URL") and if it
differs from the newly entered baseUrl, clear or remove the stored API key (e.g.
call saveCredential("CUSTOM_PROVIDER_API_KEY", null/empty or a delete function)
or force re-prompt) before attempting to
reuse/getCredential("CUSTOM_PROVIDER_API_KEY"); alternatively prompt the user to
confirm reuse of the saved key when base URLs differ, and only save the new base
URL via saveCredential("CUSTOM_PROVIDER_BASE_URL") after handling the API key
update.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 9a98e147-f0d0-4e19-aea6-a7cede9598c0
📒 Files selected for processing (7)
DockerfileREADME.mdbin/lib/inference-config.jsbin/lib/onboard.jsdocs/inference/switch-inference-providers.mdnemoclaw-blueprint/blueprint.yamltest/inference-config.test.js
✅ Files skipped from review due to trivial changes (1)
- docs/inference/switch-inference-providers.md
🚧 Files skipped from review as they are similar to previous changes (4)
- bin/lib/inference-config.js
- nemoclaw-blueprint/blueprint.yaml
- test/inference-config.test.js
- README.md
4bc9621 to
30723a3
Compare
There was a problem hiding this comment.
🧹 Nitpick comments (1)
bin/lib/onboard.js (1)
742-746: Trim the model input for consistency.
customBaseUrlandcustomApiKeyare trimmed, butmodelis not. Accidental leading/trailing whitespace could cause model routing failures.Proposed fix
- model = await prompt(" Model name (e.g. gemini-2.5-flash): "); + model = (await prompt(" Model name (e.g. gemini-2.5-flash): ")).trim(); if (!model) {🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@bin/lib/onboard.js` around lines 742 - 746, Trim the user-entered model string the same way as customBaseUrl/customApiKey: after reading model from prompt (the variable model in the onboarding flow), call .trim() and then validate (e.g., model = model && model.trim(); if (!model) { console.error(" Model name is required."); process.exit(1); }) so whitespace-only input is rejected and stored model has no leading/trailing spaces.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Nitpick comments:
In `@bin/lib/onboard.js`:
- Around line 742-746: Trim the user-entered model string the same way as
customBaseUrl/customApiKey: after reading model from prompt (the variable model
in the onboarding flow), call .trim() and then validate (e.g., model = model &&
model.trim(); if (!model) { console.error(" Model name is required.");
process.exit(1); }) so whitespace-only input is rejected and stored model has no
leading/trailing spaces.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 1223ab49-9265-4598-a1e8-d3c91db64f21
📒 Files selected for processing (7)
DockerfileREADME.mdbin/lib/inference-config.jsbin/lib/onboard.jsdocs/inference/switch-inference-providers.mdnemoclaw-blueprint/blueprint.yamltest/inference-config.test.js
✅ Files skipped from review due to trivial changes (1)
- docs/inference/switch-inference-providers.md
🚧 Files skipped from review as they are similar to previous changes (5)
- Dockerfile
- test/inference-config.test.js
- bin/lib/inference-config.js
- nemoclaw-blueprint/blueprint.yaml
- README.md
Add a "Custom OpenAI-compatible endpoint" option to the onboarding wizard, allowing users to bring any provider that exposes an OpenAI-compatible /v1/chat/completions endpoint (e.g. Google Gemini via AI Studio, OpenRouter, Together AI, LiteLLM). The custom provider follows the same gateway-routed architecture as existing providers: the sandbox talks to inference.local, and the OpenShell gateway proxies to the user's endpoint with credential injection and model rewriting. Non-NVIDIA endpoints may reject OpenAI-specific parameters like "store". Set supportsStore: false in the default openclaw.json model compat to prevent 400 rejections from strict endpoints. This is safe for all providers — NVIDIA and Ollama ignore the flag. Interactive mode prompts for base URL, API key, and model name. Non-interactive mode reads NEMOCLAW_CUSTOM_BASE_URL, NEMOCLAW_CUSTOM_API_KEY, and NEMOCLAW_MODEL. Tested with Google Gemini (gemini-2.5-flash) and local Ollama (llama3.2) to verify backward compatibility.
30723a3 to
637d376
Compare
There was a problem hiding this comment.
♻️ Duplicate comments (1)
bin/lib/onboard.js (1)
715-759:⚠️ Potential issue | 🟠 MajorValidate the custom base URL before writing it to the credential store.
Line 722 persists
CUSTOM_PROVIDER_BASE_URLbefore Lines 749-759 validate it and before a replacement key is definitely captured. If the user changes endpoints and then aborts at Line 731, the next run sees the new URL and silently reuses the oldCUSTOM_PROVIDER_API_KEYfor the wrong provider. This block also only rejects remotehttp:URLs, so unsupported schemes still pass, andhttp://[::1]:...needs explicit loopback handling.🛠️ Proposed fix
const previousBaseUrl = getCredential("CUSTOM_PROVIDER_BASE_URL"); - saveCredential("CUSTOM_PROVIDER_BASE_URL", customBaseUrl); customApiKey = previousBaseUrl === customBaseUrl ? getCredential("CUSTOM_PROVIDER_API_KEY") : null; @@ // Validate base URL try { const parsed = new URL(customBaseUrl); - if (parsed.protocol === "http:" && !["localhost", "127.0.0.1", "::1"].includes(parsed.hostname)) { + const isLoopbackHost = ["localhost", "127.0.0.1", "::1", "[::1]"].includes(parsed.hostname); + if (!["http:", "https:"].includes(parsed.protocol)) { + console.error(" Base URL must use https://, or http:// only for localhost."); + process.exit(1); + } + if (parsed.protocol === "http:" && !isLoopbackHost) { console.error(" Insecure http:// URLs are only allowed for localhost. Use https:// for remote endpoints."); process.exit(1); } } catch { console.error(` Invalid URL: ${customBaseUrl}`); process.exit(1); } + + if (!isNonInteractive()) { + saveCredential("CUSTOM_PROVIDER_BASE_URL", customBaseUrl); + }Expected output: a bracketed IPv6 hostname on line 1 and
ftp:on line 2.#!/bin/bash # Verify IPv6 hostname serialization and that non-HTTP schemes parse successfully. node - <<'NODE' console.log(new URL("http://[::1]:4000/v1").hostname); console.log(new URL("ftp://example.com/v1").protocol); NODE🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@bin/lib/onboard.js` around lines 715 - 759, Currently CUSTOM_PROVIDER_BASE_URL is saved via saveCredential("CUSTOM_PROVIDER_BASE_URL", customBaseUrl) before the URL is validated and before deciding whether to keep or prompt for the API key; move the URL validation (the new URL(...) try/catch and checks) to immediately after reading customBaseUrl and before calling saveCredential or deriving customApiKey; reject non-http/https schemes explicitly (e.g., disallow ftp:, file:, etc.), normalize/handle IPv6 loopback by recognizing both "::1" and bracketed "[::1]" as localhost, and only permit http for localhost (127.0.0.1 and ::1) while requiring https for remote hosts; ensure you only call saveCredential for CUSTOM_PROVIDER_BASE_URL and CUSTOM_PROVIDER_API_KEY after validation and after the user has provided/confirmed the API key so an aborted flow does not persist invalid or mismatched credentials.
🧹 Nitpick comments (1)
test/onboard-selection.test.js (1)
95-138: These cases never hit the onboarding validator.They only assert
new URL()plus a duplicatedisLocalhostpredicate, so the checks inbin/lib/onboard.jscan drift without this suite failing. Please drivesetupNim()like the first test in this file, or extract the base-URL validation into a shared helper and test that directly. That would also let this file cover unsupported schemes and the IPv6 loopback form.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Duplicate comments:
In `@bin/lib/onboard.js`:
- Around line 715-759: Currently CUSTOM_PROVIDER_BASE_URL is saved via
saveCredential("CUSTOM_PROVIDER_BASE_URL", customBaseUrl) before the URL is
validated and before deciding whether to keep or prompt for the API key; move
the URL validation (the new URL(...) try/catch and checks) to immediately after
reading customBaseUrl and before calling saveCredential or deriving
customApiKey; reject non-http/https schemes explicitly (e.g., disallow ftp:,
file:, etc.), normalize/handle IPv6 loopback by recognizing both "::1" and
bracketed "[::1]" as localhost, and only permit http for localhost (127.0.0.1
and ::1) while requiring https for remote hosts; ensure you only call
saveCredential for CUSTOM_PROVIDER_BASE_URL and CUSTOM_PROVIDER_API_KEY after
validation and after the user has provided/confirmed the API key so an aborted
flow does not persist invalid or mismatched credentials.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 33cbec8c-51ea-45ea-bf83-2f430ed0cb4b
📒 Files selected for processing (8)
DockerfileREADME.mdbin/lib/inference-config.jsbin/lib/onboard.jsdocs/inference/switch-inference-providers.mdnemoclaw-blueprint/blueprint.yamltest/inference-config.test.jstest/onboard-selection.test.js
✅ Files skipped from review due to trivial changes (3)
- docs/inference/switch-inference-providers.md
- test/inference-config.test.js
- README.md
🚧 Files skipped from review as they are similar to previous changes (3)
- Dockerfile
- nemoclaw-blueprint/blueprint.yaml
- bin/lib/inference-config.js
Summary
Add a "Custom OpenAI-compatible endpoint" option to the onboarding wizard, enabling users to bring any inference provider that exposes an OpenAI-compatible
/v1/chat/completionsendpoint.Problem
NemoClaw currently supports NVIDIA Endpoint, Ollama, and vLLM as inference providers. Users who want to use other providers (Google Gemini, OpenRouter, Together AI, LiteLLM) have no onboarding path.
Additionally, non-NVIDIA endpoints may reject OpenAI-specific request parameters like
store, resulting in400 status code (no body)errors from the sandbox agent.Solution
case "custom"togetProviderSelectionConfig()following the existing switch-case pattern for provider registrationNEMOCLAW_CUSTOM_BASE_URL,NEMOCLAW_CUSTOM_API_KEY, andNEMOCLAW_MODELcompat: { supportsStore: false }on the default inference model entry inopenclaw.jsonto prevent strict endpoints from rejecting thestoreparameter. This is safe for all providers — NVIDIA and Ollama ignore the flagcustomprofile toblueprint.yamlThe custom provider follows the same gateway-routed architecture as existing providers: the sandbox talks to
inference.local, and the OpenShell gateway proxies to the user endpoint with credential injection and model rewriting.Files changed
DockerfilesupportsStore: falsecompat flag to inference model entrybin/lib/inference-config.jscustomprovider casebin/lib/onboard.jsnemoclaw-blueprint/blueprint.yamlcustominference profiletest/inference-config.test.jsdocs/inference/switch-inference-providers.mdREADME.mdTest plan
Summary by CodeRabbit
New Features
Documentation