feat: add model fallback support for resilient AI in DDIL environments#594
Draft
feat: add model fallback support for resilient AI in DDIL environments#594
Conversation
…egration Add service-level model fallback so AI automatically switches between providers when the primary fails (network error, rate limit, model unavailable). Each service can optionally specify a FallbackServiceID pointing to another service, forming a chain (A→B→C) with cycle detection. Backend changes: - Add FallbackServiceID field to ServiceConfig - Add ResolveFallbackChain() for walking the fallback chain with cycle detection - Create multiProviderAccount to register multiple providers with Bifrost - Add FallbackEntry type and wire fallbacks through Config/LLM/New() - Update convertToBifrostRequest and convertToBifrostResponsesRequest to attach fallbacks to every outgoing request - Update NewFromServiceConfig to accept and map fallback services - Update bots.go getLLM() and resolveServiceCfgs() to resolve and pass fallback chains, including change detection for fallback services Co-authored-by: Christopher Speller <crspeller@users.noreply.github.com>
Add a 'Fallback Service' dropdown to the service configuration panel in the system console. When a service is unavailable, requests automatically fall back to the selected service. Chains are supported (A→B→C). - Add fallbackServiceID field to LLMService TypeScript type - Add SelectionItem dropdown showing all other services as fallback options - Self-selection is excluded from the dropdown to prevent trivial cycles - Pass services list from parent Services component to each Service - Include fallbackServiceID in default new service template Co-authored-by: Christopher Speller <crspeller@users.noreply.github.com>
Add tests for: - ResolveFallbackChain: no fallback, simple chain, multi-hop chain, cycle detection, self-cycle, missing/invalid services - FallbackServiceID JSON serialization (round-trip, omitempty) - multiProviderAccount: single/multiple providers, unknown provider, duplicate provider handling, Azure/Bedrock key configs - Fallback request building: both Chat and Responses API with/without fallbacks - NewFromServiceConfig: no fallbacks, single fallback, multiple fallbacks, bot model override isolation, serviceConfigToFallbackEntry mapping Co-authored-by: Christopher Speller <crspeller@users.noreply.github.com>
Co-authored-by: Christopher Speller <crspeller@users.noreply.github.com>
🤖 LLM Evaluation ResultsOpenAI✅ Overall: 19/19 tests passed (100.0%)
Anthropic
❌ Failed EvaluationsShow 2 failuresANTHROPIC1. TestReactEval/[anthropic]_react_cat_message
2. TestThreadsOpenQuestionsFromExportedData/[anthropic]_open_questions_from_eval_timed_dnd.json
This comment was automatically generated by the eval CI pipeline. |
|
Creating a Plugin SpinWick test server |
|
Plugin Spinwick PR #594 🎉 Test server created! Access here: https://agents-pr-594-wuee6.test.mattermost.cloud
Installation ID: Credentials: Posted securely in this Mattermost channel - Look for PR #594 |
Merge origin/master into feature branch. Resolved conflicts: - bifrost/bifrost.go: convertToBifrostResponsesRequest now returns error; kept fallback attachment and adapted to new (req, error) return signature - bifrost/bifrost_test.go: kept both fallback tests and new master tests (structured output enum, multi-type array, proxy routing); updated fallback test calls to match new (req, error) return signature Co-authored-by: Christopher Speller <crspeller@users.noreply.github.com>
|
New commit detected. SpinWick will update the plugin if a new artifact is available. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add a service-level model fallback mechanism so AI automatically switches between LLM providers when the primary is unavailable — using cloud LLMs when connected and falling back to on-prem/local models in DDIL (Denied, Disrupted, Intermittent, Limited) environments.
How it works:
Each AI service can now optionally specify a Fallback Service — another configured service to try when the primary fails (network error, rate limit, model unavailable). Fallback chains are supported (e.g., OpenAI → Anthropic → Local Ollama), with automatic cycle detection.
This leverages Bifrost's built-in fallback mechanism, which handles sequential failover automatically on every request. When the primary provider recovers, it's automatically used again — no manual intervention needed.
Key behavior:
Backend changes:
FallbackServiceIDfield toServiceConfigfor chaining servicesResolveFallbackChain()with cycle detection (visited-set algorithm)multiProviderAccountto register multiple providers with BifrostFrontend changes:
Tests added (28 new test cases):
ResolveFallbackChain: no fallback, simple chain, multi-hop chain, cycle detection, self-cycle, missing/invalid servicesmultiProviderAccount: single/multiple providers, unknown provider, duplicate handling, Azure/Bedrock configsNewFromServiceConfig: no fallbacks, single/multiple fallbacks, bot model override isolationserviceConfigToFallbackEntry: provider mapping, URL normalizationTicket Link
N/A — Feature request for resilient AI in DDIL environments
Release Note