Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/features/nexus-model-routing.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ Nexus defaults to **Standard** mode. Users see one Nexus experience instead of a
2. Apply the existing K-12 input guardrail before classifier traffic without rewriting allowed request text.
3. Apply deterministic capability rules for image generation, PSD-data, web search/current information, and common instructional requests.
4. Send ambiguous requests to Amazon Nova Micro on Bedrock for a provider-neutral `intent`, `tier`, `confidence`, and reason codes.
5. Resolve an accessible, active Nexus model from the configured ordered candidates. If none is configured, use `providerMetadata.nexusRouterTier` or model-name conventions. If no tier match is available, use the closest tier in the requested family. Auto may finally use the existing client model as a safe fallback; an explicit Advanced family never silently crosses into another family.
5. Resolve an accessible, active Nexus model from the configured ordered candidates. Models whose provider credentials are not configured (checked through the settings manager with environment fallback; Bedrock always counts as configured because it can use the ambient AWS credential chain) are excluded from routed selection, so routing never picks a model whose provider creation would fail at stream time. If none is configured, use `providerMetadata.nexusRouterTier` or model-name conventions. If no tier match is available, use the closest tier in the requested family. Auto may finally use the existing client model as a safe fallback — an explicitly selected client model keeps its honest provider-configuration error rather than being silently rerouted; an explicit Advanced family never silently crosses into another family.
6. Automatically select an image-capable model for image intent, attach the existing database-backed PSD-data MCP server for PSD-data intent, or select a web-search-capable Gemini model and enable its native Google Search tool for web-search intent.
7. Persist the route decision on assistant-message metadata and expose it in `X-Nexus-Routing` for evaluation.

Expand Down
62 changes: 62 additions & 0 deletions lib/ai/__tests__/provider-credentials.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
/** @jest-environment node */

const mockGetOpenAI = jest.fn()
const mockGetGoogleAI = jest.fn()
const mockGetAzureOpenAI = jest.fn()
const mockGetLatimer = jest.fn()

jest.mock("@/lib/settings-manager", () => ({
Settings: {
getOpenAI: () => mockGetOpenAI(),
getGoogleAI: () => mockGetGoogleAI(),
getAzureOpenAI: () => mockGetAzureOpenAI(),
getLatimer: () => mockGetLatimer(),
},
}))
jest.mock("@/lib/logger", () => ({
createLogger: () => ({ info: jest.fn(), warn: jest.fn(), error: jest.fn(), debug: jest.fn() }),
}))

import { getConfiguredChatProviders } from "../provider-credentials"

describe("getConfiguredChatProviders", () => {
beforeEach(() => {
jest.clearAllMocks()
mockGetOpenAI.mockResolvedValue(null)
mockGetGoogleAI.mockResolvedValue(null)
mockGetAzureOpenAI.mockResolvedValue({ key: null, endpoint: null, resourceName: null })
mockGetLatimer.mockResolvedValue(null)
})

it("always includes amazon-bedrock even with no keys configured", async () => {
const configured = await getConfiguredChatProviders()
expect(configured).toEqual(new Set(["amazon-bedrock"]))
})

it("includes every provider whose credential is present", async () => {
mockGetOpenAI.mockResolvedValue("sk-test")
mockGetGoogleAI.mockResolvedValue("google-key")
mockGetAzureOpenAI.mockResolvedValue({ key: "azure-key", endpoint: null, resourceName: "resource" })
mockGetLatimer.mockResolvedValue("latimer-key")
const configured = await getConfiguredChatProviders()
expect(configured).toEqual(new Set(["openai", "google", "amazon-bedrock", "azure", "latimer"]))
})

it("treats whitespace-only keys as not configured", async () => {
mockGetOpenAI.mockResolvedValue(" ")
const configured = await getConfiguredChatProviders()
expect(configured.has("openai")).toBe(false)
})

it("requires both the Azure key and resource name", async () => {
mockGetAzureOpenAI.mockResolvedValue({ key: "azure-key", endpoint: null, resourceName: null })
const configured = await getConfiguredChatProviders()
expect(configured.has("azure")).toBe(false)
})

it("fails open when the credential probe throws", async () => {
mockGetOpenAI.mockRejectedValue(new Error("settings unavailable"))
const configured = await getConfiguredChatProviders()
expect(configured).toEqual(new Set(["openai", "google", "amazon-bedrock", "azure", "latimer"]))
})
})
39 changes: 39 additions & 0 deletions lib/ai/provider-credentials.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
import { Settings } from "@/lib/settings-manager"
import { createLogger } from "@/lib/logger"

const log = createLogger({ module: "provider-credentials" })

const ALL_CHAT_PROVIDERS = ["openai", "google", "amazon-bedrock", "azure", "latimer"] as const

/**
* Returns the lowercase provider identifiers whose credentials are configured
* (database settings with environment fallback, cached by the settings
* manager), so routing can skip models that would fail provider creation.
*
* amazon-bedrock is always included: it can authenticate through the default
* AWS credential chain (ECS/Lambda IAM roles), which cannot be probed here.
*
* Fails open — a probe failure must not take chat down, so every provider is
* treated as configured and a missing key surfaces at stream time as before.
*/
export async function getConfiguredChatProviders(): Promise<Set<string>> {
try {
const [openAiKey, googleKey, azure, latimerKey] = await Promise.all([
Settings.getOpenAI(),
Settings.getGoogleAI(),
Settings.getAzureOpenAI(),
Settings.getLatimer(),
])
const configured = new Set<string>(["amazon-bedrock"])
if (openAiKey?.trim()) configured.add("openai")
if (googleKey?.trim()) configured.add("google")
if (azure.key?.trim() && azure.resourceName?.trim()) configured.add("azure")
if (latimerKey?.trim()) configured.add("latimer")
return configured
} catch (error) {
log.warn("Provider credential probe failed; treating every provider as configured", {
error: error instanceof Error ? error.message : String(error),
})
return new Set(ALL_CHAT_PROVIDERS)
}
}
96 changes: 96 additions & 0 deletions lib/nexus/model-router/__tests__/router.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,12 @@ const mockFilterAccessibleResourceIds = jest.fn()
const mockGetConfig = jest.fn()
const mockClassify = jest.fn()
const mockExecuteQuery = jest.fn()
const mockGetConfiguredChatProviders = jest.fn()

jest.mock("@/lib/db/drizzle", () => ({ getNexusEnabledModels: () => mockGetNexusEnabledModels() }))
jest.mock("@/lib/ai/provider-credentials", () => ({
getConfiguredChatProviders: () => mockGetConfiguredChatProviders(),
}))
jest.mock("@/lib/db/drizzle/resource-access", () => ({
filterAccessibleResourceIds: (...args: unknown[]) => mockFilterAccessibleResourceIds(...args),
}))
Expand Down Expand Up @@ -51,6 +55,9 @@ function defineNexusModelRouterSuite1Part1() {
mockGetNexusEnabledModels.mockResolvedValue(models)
mockFilterAccessibleResourceIds.mockResolvedValue(models.map(model => String(model.id)))
mockGetConfig.mockResolvedValue({ config, mode: "active" })
mockGetConfiguredChatProviders.mockResolvedValue(
new Set(["openai", "google", "amazon-bedrock", "azure", "latimer"])
)
mockClassify.mockResolvedValue({
intent: "general", tier: "medium", confidence: 0.9,
reasonCodes: ["normal_request"], source: "classifier",
Expand Down Expand Up @@ -343,3 +350,92 @@ const defineNexusModelRouterSuite1 = () => {
};

describe("Nexus model router", defineNexusModelRouterSuite1)

describe("Nexus model router credential filtering", () => {
beforeEach(() => {
jest.clearAllMocks()
mockGetNexusEnabledModels.mockResolvedValue(models)
mockFilterAccessibleResourceIds.mockResolvedValue(models.map(model => String(model.id)))
mockGetConfig.mockResolvedValue({ config, mode: "active" })
mockClassify.mockResolvedValue({
intent: "general", tier: "medium", confidence: 0.9,
reasonCodes: ["normal_request"], source: "classifier",
})
})

it("routes AUTO past a candidate whose provider credential is not configured", async () => {
const autoConfig = nexusRouterConfigSchema.parse({
auto: { light: [], medium: ["gpt-terra", "us.anthropic.claude-sonnet"], high: [] },
})
mockGetConfig.mockResolvedValue({ config: autoConfig, mode: "active" })
mockGetConfiguredChatProviders.mockResolvedValue(new Set(["amazon-bedrock", "google"]))

const result = await routeNexusRequest({
text: "Help", fallbackModelId: "gpt-terra", experienceMode: "standard",
requestedFamily: "auto", enabledConnectorIds: [], userId: 7,
})

expect(result.modelId).toBe("us.anthropic.claude-sonnet")
expect(result.metadata.selectedFamily).toBe("anthropic")
})

it("keeps the explicitly selected model when routing is off, even without its provider key", async () => {
mockGetConfig.mockResolvedValue({ config, mode: "off" })
mockGetConfiguredChatProviders.mockResolvedValue(new Set(["amazon-bedrock", "google"]))

const result = await routeNexusRequest({
text: "Help", fallbackModelId: "gpt-terra", experienceMode: "standard",
requestedFamily: "auto", enabledConnectorIds: [], userId: 7,
})

expect(result.modelId).toBe("gpt-terra")
})

it("fails clearly when the requested Advanced family's provider is not configured", async () => {
mockGetConfiguredChatProviders.mockResolvedValue(new Set(["amazon-bedrock", "google"]))

await expect(routeNexusRequest({
text: "Help", fallbackModelId: "us.anthropic.claude-sonnet", experienceMode: "advanced",
requestedFamily: "openai", enabledConnectorIds: [], userId: 7,
})).rejects.toThrow("openai family")
})

it("re-routes required tools to a configured provider when routing is off and the explicit model's provider is not", async () => {
mockGetConfig.mockResolvedValue({ config, mode: "off" })
mockGetConfiguredChatProviders.mockResolvedValue(new Set(["amazon-bedrock", "google"]))

const result = await routeNexusRequest({
text: "Summarize my attachment", fallbackModelId: "gpt-terra", experienceMode: "standard",
requestedFamily: "auto", enabledConnectorIds: [],
enabledToolNames: ["searchNexusAttachments"], userId: 7,
})

expect(result.modelId).toBe("us.anthropic.claude-sonnet")
expect(result.metadata.reasonCodes).toContain("required_tools_enforced")
})

it("fails fast before provider creation when no configured provider has an accessible model", async () => {
// The explicitly selected model's provider being unconfigured must NOT be
// executed as a last resort here — that is the stream-time missing-key 500
// this filter exists to prevent. Exhaustion fails fast with a clear error.
mockGetConfiguredChatProviders.mockResolvedValue(new Set(["azure"]))

await expect(routeNexusRequest({
text: "Help", fallbackModelId: "gpt-terra", experienceMode: "standard",
requestedFamily: "auto", enabledConnectorIds: [], userId: 7,
})).rejects.toThrow("No accessible Nexus model is available")
})

it("does not offer image generation through an unconfigured provider", async () => {
mockClassify.mockResolvedValue({
intent: "image", tier: "medium", confidence: 0.99,
reasonCodes: ["explicit_image_request"], source: "deterministic",
})
mockGetConfiguredChatProviders.mockResolvedValue(new Set(["amazon-bedrock", "openai"]))

await expect(routeNexusRequest({
text: "Create an image", fallbackModelId: "gpt-terra", experienceMode: "standard",
requestedFamily: "auto", enabledConnectorIds: [], userId: 7,
})).rejects.toThrow("Image generation is not available")
})
})
25 changes: 23 additions & 2 deletions lib/nexus/model-router/router.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import { nexusMcpServers } from "@/lib/db/schema"
import { filterAccessibleResourceIds } from "@/lib/db/drizzle/resource-access"
import { createLogger } from "@/lib/logger"
import { hasCapability } from "@/lib/ai/capability-utils"
import { getConfiguredChatProviders } from "@/lib/ai/provider-credentials"
import {
inferModelFamily,
inferModelTier,
Expand Down Expand Up @@ -422,17 +423,37 @@ export async function routeNexusRequest(
"model",
models.map(model => model.id)
))
// The fallback (the model the client explicitly selected) is looked up in
// the unfiltered list on purpose: an explicit selection keeps its honest
// provider-configuration error instead of being silently rerouted.
const fallback = models.find(model =>
model.modelId === args.fallbackModelId
|| String(model.id) === args.fallbackModelId
)
if (!fallback) throw new Error("The fallback Nexus model is unavailable")
// Routed selection must never pick a model whose provider credential is not
// configured — provider creation would throw at stream time and fail the
// whole request, which the router's fallback cannot catch.
const configuredProviders = await getConfiguredChatProviders()
const routableModels = models.filter(model =>
configuredProviders.has(model.provider.toLowerCase())
)
if (routableModels.length < models.length) {
const excludedProviders = [...new Set(
models
.filter(model => !configuredProviders.has(model.provider.toLowerCase()))
.map(model => model.provider)
)]
log.info("Excluding models from routing; provider credentials not configured", {
excludedProviders,
})
}
const requiredTools = [...new Set(args.enabledToolNames ?? [])]
if (mode === "off") {
return buildRouterOffResult({
args,
config,
models,
models: routableModels,
fallback,
accessibleIds,
requiredTools,
Expand All @@ -442,7 +463,7 @@ export async function routeNexusRequest(
args,
config,
mode,
models,
models: routableModels,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve the explicit fallback after routed candidates are exhausted

When active Standard/Auto routing has no accessible model from a configured provider, passing only routableModels means selectRoutedTextModel cannot find the explicitly selected fallback because it searches for that fallback inside its models argument. The request therefore throws No accessible Nexus model is available before provider creation instead of retaining the selected model and surfacing its provider-configuration error as the new comment and routing documentation specify. Keep the unconfigured fallback out of normal candidate selection, but allow it as the final non-routed fallback when no configured alternative exists.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Behavior locked in with tests in 60c8140, but keeping the fail-fast rather than executing the unconfigured explicit fallback: both outcomes are errors, and running a model whose provider creation must throw reproduces the exact stream-time missing-key 500 this PR removes — a misleading mid-stream crash instead of an honest routing error. The docs carve-out applies to the paths that execute the explicit model directly (router-off without required tools, shadow retention), which are unchanged.

fallback,
accessibleIds,
requiredTools,
Expand Down