Skip to content

Refactor Model Selection Logic #425

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
google-labs-jules wants to merge 4 commits into
base: main
Choose a base branch
from
Open

Refactor Model Selection Logic #425

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
b72c0f6
feat: refactor model selection logic for clarity and correctness
google-labs-jules[bot] Jan 13, 2026
5c4f50b
feat: refactor model selection logic for clarity and correctness
google-labs-jules[bot] Jan 13, 2026
5bd124a
feat: refactor model selection logic for clarity and correctness
google-labs-jules[bot] Jan 13, 2026
573bede
fix: correct Gemini model name in selection logic
google-labs-jules[bot] Jan 13, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .env.local.example
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,12 @@ NEXT_PUBLIC_COMPOSIO_USER_ID=user@example.com
# Gemini 3 Pro (Google Generative AI)
GEMINI_3_PRO_API_KEY=your_gemini_3_pro_api_key_here

# AWS Bedrock Credentials
AWS_ACCESS_KEY_ID=YOUR_AWS_ACCESS_KEY_ID_HERE
AWS_SECRET_ACCESS_KEY=YOUR_AWS_SECRET_ACCESS_KEY_HERE
AWS_REGION=YOUR_AWS_REGION_HERE
BEDROCK_MODEL_ID=YOUR_BEDROCK_MODEL_ID_HERE

# Supabase Credentials
NEXT_PUBLIC_SUPABASE_URL=YOUR_SUPABASE_URL_HERE
NEXT_PUBLIC_SUPABASE_ANON_KEY=YOUR_SUPABASE_ANON_KEY_HERE
Expand Down
136 changes: 54 additions & 82 deletions lib/utils/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,103 +19,75 @@ export function generateUUID(): string {

export async function getModel(requireVision: boolean = false) {
const selectedModel = await getSelectedModel();
Comment on lines 20 to 21
Copy link

@charliecreates charliecreates bot Jan 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

requireVision is still accepted by getModel() but is never used in the new selection logic. That’s a behavior smell: callers may set requireVision=true expecting vision-capable models, but the function now ignores it entirely and may return a non-vision model.

If requireVision is not supported anymore, remove it (and update call sites). If it is supported, the model map should encode capabilities and the selection/fallback chain should filter accordingly.

Suggestion

Model the capability in the initializer config and filter by it, e.g.

const modelConfigs = {
  'QCX-Terra': { vision: false, init: () => /*...*/ },
  'GPT-5.1': { vision: true, init: () => /*...*/ },
  // ...
} as const;

const isEligible = (k: keyof typeof modelConfigs) =>
  !requireVision || modelConfigs[k].vision;

Then apply isEligible(...) both for the selected model and the fallback chain.

Reply with "@CharlieHelps yes please" if you'd like me to add a commit with this suggestion.

console.log(`[getModel] User-selected model: ${selectedModel}`);

const xaiApiKey = process.env.XAI_API_KEY;
const gemini3ProApiKey = process.env.GEMINI_3_PRO_API_KEY;
const awsAccessKeyId = process.env.AWS_ACCESS_KEY_ID;
const awsSecretAccessKey = process.env.AWS_SECRET_ACCESS_KEY;
const awsRegion = process.env.AWS_REGION;
const bedrockModelId = process.env.BEDROCK_MODEL_ID || 'anthropic.claude-3-5-sonnet-20241022-v2:0';
const bedrockModelId = process.env.BEDROCK_MODEL_ID || 'anthropic.claude-3-5-sonnet-20240620-v1:0';
const openaiApiKey = process.env.OPENAI_API_KEY;

if (selectedModel) {
switch (selectedModel) {
case 'Grok 4.2':
if (xaiApiKey) {
const xai = createXai({
apiKey: xaiApiKey,
baseURL: 'https://api.x.ai/v1',
});
try {
return xai('grok-4-fast-non-reasoning');
} catch (error) {
console.error('Selected model "Grok 4.2" is configured but failed to initialize.', error);
throw new Error('Failed to initialize selected model.');
}
} else {
console.error('User selected "Grok 4.2" but XAI_API_KEY is not set.');
throw new Error('Selected model is not configured.');
}
case 'Gemini 3':
if (gemini3ProApiKey) {
const google = createGoogleGenerativeAI({
apiKey: gemini3ProApiKey,
});
try {
return google('gemini-3-pro-preview');
} catch (error) {
console.error('Selected model "Gemini 3" is configured but failed to initialize.', error);
throw new Error('Failed to initialize selected model.');
}
} else {
console.error('User selected "Gemini 3" but GEMINI_3_PRO_API_KEY is not set.');
throw new Error('Selected model is not configured.');
}
case 'GPT-5.1':
if (openaiApiKey) {
const openai = createOpenAI({
apiKey: openaiApiKey,
});
return openai('gpt-4o');
} else {
console.error('User selected "GPT-5.1" but OPENAI_API_KEY is not set.');
throw new Error('Selected model is not configured.');
}
}
}

// Default behavior: Grok -> Gemini -> Bedrock -> OpenAI
if (xaiApiKey) {
const xai = createXai({
apiKey: xaiApiKey,
baseURL: 'https://api.x.ai/v1',
});
try {
const modelInitializers = {
'QCX-Terra': () => {
if (!awsAccessKeyId || !awsSecretAccessKey) {
throw new Error('AWS credentials for QCX-Terra are not configured.');
}
const bedrock = createAmazonBedrock({
bedrockOptions: {
region: awsRegion,
credentials: { accessKeyId: awsAccessKeyId, secretAccessKey: awsSecretAccessKey },
},
});
return bedrock(bedrockModelId, { additionalModelRequestFields: { top_k: 250 } });
},
Comment on lines +33 to +44
Copy link

@charliecreates charliecreates bot Jan 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AWS_REGION is read but not validated before being passed into createAmazonBedrock({ bedrockOptions: { region: awsRegion, ... }}). If AWS_REGION is missing, this may fail in a less obvious way or defer failure to later runtime.

Also, QCX-Terra checks only access key + secret; it should include region (and potentially BEDROCK_MODEL_ID if you want to require explicit selection).

Suggestion

Harden the Bedrock initializer to validate the full required configuration:

if (!awsAccessKeyId || !awsSecretAccessKey || !awsRegion) {
  throw new Error('AWS credentials/region for QCX-Terra are not configured.');
}

Optionally validate bedrockModelId if you intend it to be user-configured rather than defaulted.

Reply with "@CharlieHelps yes please" if you'd like me to add a commit with this suggestion.

Comment on lines +29 to +44
Copy link

@charliecreates charliecreates bot Jan 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Bedrock defaults/parameters changed (BEDROCK_MODEL_ID default and top_k: 250). That’s a behavior change that could materially affect output quality/cost/latency. The code doesn’t explain why these specific values are chosen, and it’s not obvious they’re safe defaults.

Suggestion

At minimum, document these defaults with a brief rationale and consider moving them to config/constants so they’re easier to tune and test. If top_k is model-specific, gate it by model ID.

const DEFAULT_BEDROCK_MODEL_ID = process.env.BEDROCK_MODEL_ID ?? '...';
const BEDROCK_TOP_K = Number(process.env.BEDROCK_TOP_K ?? 250);

Reply with "@CharlieHelps yes please" if you want me to add a commit that centralizes these defaults and documents them.

'Grok 4.2': () => {
if (!xaiApiKey) throw new Error('XAI_API_KEY for Grok 4.2 is not set.');
const xai = createXai({ apiKey: xaiApiKey, baseURL: 'https://api.x.ai/v1' });
return xai('grok-4-fast-non-reasoning');
} catch (error) {
console.warn('xAI API unavailable, falling back to next provider:');
}
}
},
'Gemini 3': () => {
if (!gemini3ProApiKey) throw new Error('GEMINI_3_PRO_API_KEY for Gemini 3 is not set.');
const google = createGoogleGenerativeAI({ apiKey: gemini3ProApiKey });
return google('gemini-3-pro-preview');
},
'GPT-5.1': () => {
if (!openaiApiKey) throw new Error('OPENAI_API_KEY for GPT-5.1 is not set.');
const openai = createOpenAI({ apiKey: openaiApiKey });
return openai('gpt-4o');
},
};

if (gemini3ProApiKey) {
const google = createGoogleGenerativeAI({
apiKey: gemini3ProApiKey,
});
if (selectedModel && selectedModel in modelInitializers) {
try {
return google('gemini-3-pro-preview');
console.log(`[getModel] Initializing user-selected model: ${selectedModel}`);
return modelInitializers[selectedModel as keyof typeof modelInitializers]();
} catch (error) {
console.warn('Gemini 3 Pro API unavailable, falling back to next provider:', error);
console.error(`[getModel] Failed to initialize selected model "${selectedModel}":`, error);
// Fallback to default if the selected model fails for any reason.
}
}

if (awsAccessKeyId && awsSecretAccessKey) {
const bedrock = createAmazonBedrock({
bedrockOptions: {
region: awsRegion,
credentials: {
accessKeyId: awsAccessKeyId,
secretAccessKey: awsSecretAccessKey,
},
},
});
const model = bedrock(bedrockModelId, {
additionalModelRequestFields: { top_k: 350 },
});
return model;
// Default fallback logic if no model is selected or initialization fails
console.log('[getModel] No valid model selected, proceeding with default fallback chain.');
const fallbackProviders = [
{ name: 'Grok', key: xaiApiKey, init: modelInitializers['Grok 4.2'] },
{ name: 'Gemini', key: gemini3ProApiKey, init: modelInitializers['Gemini 3'] },
{ name: 'Bedrock', key: awsAccessKeyId && awsSecretAccessKey, init: modelInitializers['QCX-Terra'] },
{ name: 'OpenAI', key: openaiApiKey, init: modelInitializers['GPT-5.1'] },
];

for (const provider of fallbackProviders) {
if (provider.key) {
try {
console.log(`[getModel] Attempting to use default provider: ${provider.name}`);
return provider.init();
} catch (error) {
console.warn(`[getModel] ${provider.name} API unavailable, falling back to next provider:`, error);
}
}
}

const openai = createOpenAI({
apiKey: openaiApiKey,
});
return openai('gpt-4o');
throw new Error('No valid AI providers are configured. Please check your environment variables.');
}