Skip to content

Automatically discover and configure Bifrost-backed LLM providers#569

Draft
crspeller wants to merge 5 commits intomasterfrom
cursor/llm-backend-auto-discovery-97b8
Draft

Automatically discover and configure Bifrost-backed LLM providers#569
crspeller wants to merge 5 commits intomasterfrom
cursor/llm-backend-auto-discovery-97b8

Conversation

@crspeller
Copy link
Copy Markdown
Member

@crspeller crspeller commented Mar 25, 2026

Summary

This PR updates the plugin to derive supported LLM backends from Bifrost instead of a plugin-maintained allowlist.

Key changes:

  • replaced hardcoded service-type mapping and validation with dynamic support based on Bifrost schemas.StandardProviders
  • kept a simple openaicompatible convenience option while adding advanced Bifrost passthrough fields for provider-specific setup:
    • bifrostKeyJSON
    • bifrostProviderConfigJSON
  • refactored runtime setup, model fetching, and transcription setup to use a shared service-to-Bifrost resolver
  • changed model fetching to use the full service config instead of only serviceType, apiKey, apiURL, and orgID
  • normalized fetched model IDs using Bifrost parsing semantics instead of naive slash stripping
  • made the system console provider dropdown dynamic and added advanced Bifrost JSON inputs
  • removed stale provider plumbing and fully removed asage references
  • updated docs and test coverage, including UI coverage for dynamic providers and advanced config persistence
  • fixed a follow-up CI lint regression in the system console service editor

Validation performed:

  • cd /workspace/webapp && npm run check-types
  • docker run --rm -v /workspace:/workspace -w /workspace golang:tip go test ./llm/... ./bifrost/...
  • docker run --rm -v /workspace:/workspace -w /workspace golang:tip go test ./api/... ./server/...
  • cd /workspace/e2e && npx playwright test tests/system-console/service-management-ui.spec.ts --reporter=list

Ticket Link

NONE

Screenshots

NONE

Release Note

Updated Mattermost Agents to discover supported Bifrost-backed LLM providers dynamically, added advanced Bifrost provider configuration fields in the system console, removed asksage references, and updated model fetching to use full service configuration with Bifrost-aware model normalization.
Open in Web Open in Cursor 

cursoragent and others added 4 commits March 23, 2026 13:42
Co-authored-by: Christopher Speller <crspeller@users.noreply.github.com>
Co-authored-by: Christopher Speller <crspeller@users.noreply.github.com>
Co-authored-by: Christopher Speller <crspeller@users.noreply.github.com>
Co-authored-by: Christopher Speller <crspeller@users.noreply.github.com>
@cursor
Copy link
Copy Markdown
Contributor

cursor bot commented Mar 25, 2026

Cursor Agent can help with this pull request. Just @cursor in comments and I'll start working on changes in this branch.
Learn more about Cursor Agents

@github-actions
Copy link
Copy Markdown

github-actions bot commented Mar 25, 2026

🤖 LLM Evaluation Results

OpenAI

⚠️ Overall: 18/19 tests passed (94.7%)

Provider Total Passed Failed Pass Rate
⚠️ OPENAI 19 18 1 94.7%

❌ Failed Evaluations

Show 1 failures

OPENAI

1. TestReactEval/[openai]_react_cat_message

  • Score: 0.00
  • Rubric: The word/emoji is a cat emoji or a heart/love emoji
  • Reason: The output is the literal text "heart_eyes_cat", not an actual cat emoji (e.g., 😺/🐱) or a heart/love emoji (e.g., ❤️/😍).

Anthropic

Overall: 19/19 tests passed (100.0%)

Provider Total Passed Failed Pass Rate
✅ ANTHROPIC 19 19 0 100.0%

This comment was automatically generated by the eval CI pipeline.

Co-authored-by: Christopher Speller <crspeller@users.noreply.github.com>
@cursor cursor bot changed the title Llm backend auto-discovery Automatically discover and configure Bifrost-backed LLM providers Mar 25, 2026
@mm-cloud-bot
Copy link
Copy Markdown

Creating a Plugin SpinWick test server

@mm-cloud-bot
Copy link
Copy Markdown

Plugin Spinwick PR #569 🎉

Test server created!

Access here: https://agents-pr-569-auqk4.test.mattermost.cloud

Plugin Version Artifact
agents c17dafb Download

Installation ID: ko6kci4apfgaxnaj8a7f4c1gxe
Logs: Click here

Credentials: Posted securely in this Mattermost channel - Look for PR #569

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants