Skip to content

Added static model support for the .forge.toml#3086

Draft
joshuas99 wants to merge 4 commits intotailcallhq:mainfrom
joshuas99:main
Draft

Added static model support for the .forge.toml#3086
joshuas99 wants to merge 4 commits intotailcallhq:mainfrom
joshuas99:main

Conversation

@joshuas99
Copy link
Copy Markdown

@joshuas99 joshuas99 commented Apr 20, 2026

Overview

See my original issue and the initial feature request that I noted was related.

The key issue I noted was the documentation showed it should be possible to add a static model, but the capability was not yet there.

I needed static model support for myself as I was testing the new Qwen3.6 model. I went ahead and created a plan to add the support while still retaining the initial capability for the static models URL that was a part of the [[providers]] definition in the schema.

Project Structure Summary

This is taken directly from the implemented plan.

The implementation spans three key areas:

  1. Schema Layer (forge.schema.json): JSON Schema definition for configuration validation
  2. Config Layer (crates/forge_config/src/config.rs): Rust struct definitions with Serde deserialization
  3. Repository Layer (crates/forge_repo/src/provider/provider_repo.rs): Internal model handling with Models enum

Relevant Files

File Purpose Current State
forge.schema.json:613-689 ProviderEntry schema models accepts only string | null
crates/forge_config/src/config.rs:69-99 ProviderEntry struct models: Option<String>
crates/forge_repo/src/provider/provider_repo.rs:13-21 Internal Models enum Already supports Url and Hardcoded
crates/forge_domain/src/model.rs:23-38 Domain Model struct Defines model fields
crates/forge_repo/src/provider/provider.json Embedded provider configs Uses hardcoded model arrays

Target State

  1. Support Custom Provider Configurations: So model parameters can be overridden
  2. Maintain existing URL support: for discovery endpoints
  3. Make models endpoint graceful: to eliminate failures if there is no such endpoint on a custom provider

Outcome

This fix worked for me to locally support model configuration parameters including crucially modifying the context windows when the values were not presented by oMLX.

@CLAassistant
Copy link
Copy Markdown

CLAassistant commented Apr 20, 2026

CLA assistant check
All committers have signed the CLA.

@joshuas99
Copy link
Copy Markdown
Author

I'm going to update this PR with an additional enhancement. I realized in further testing that the models endpoint is still being required even with custom providers. This causes custom providers to fail unless they provide an expected models API endpoint.

I'm making the models endpoint check optional for the custom provider so it doesn't fail in the case there is no such endpoint.

@amitksingh1490
Copy link
Copy Markdown
Contributor

I'm going to update this PR with an additional enhancement. I realized in further testing that the models endpoint is still being required even with custom providers. This causes custom providers to fail unless they provide an expected models API endpoint.

I'm making the models endpoint check optional for the custom provider so it doesn't fail in the case there is no such endpoint.

sure moving it to draft. Mark it ready for review once done

@amitksingh1490 amitksingh1490 marked this pull request as draft April 21, 2026 06:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants