v2.11.0 — Routing por escenario + router mode multi-modelo - #11
Merged
Conversation
… llama-server
Dominio:
- models/provider.py: RoutingRule (pattern substring case-insensitive, min_tokens para longContext, provider_id, model) + routing_enabled/routing_rules en ProviderRegistry.
- providers_service: resolve_route (primer match gana; provider desconocido se salta; model vacío cae al active_model del destino) y set_routing.
- llamacpp_service: router_mode en build_cmdline (--models-dir sin --model — llama-server sirve todos los GGUF con load/unload dinámico) y start sin exigir model_path en ese modo.
Aplicación:
- api/providers.py: GET/PUT /providers/routing (declarados antes de /{provider_id} para no colisionar), validación de providers desconocidos.
- api/messages.py: resolución de ruta por request con prompt tokens; provider anthropic ruteado va DIRECTO a api.anthropic.com (native path) porque litellm corre con el config del provider activo; fallback de key nativa a settings cuando el .env no está en os.environ.
- api/openai_compat.py: routing aplicado solo a destinos OpenAI-compat.
Infraestructura:
- Frontend: RoutingPanel (toggle + editor de reglas con select de provider), toggle router_mode en LlamaCppPanel, routingApi + tipos RoutingRule/RoutingConfig.
- CLAUDE.md: documentación de routing y router mode.
Configuración:
- Bump version a 2.11.0.
Pruebas:
- test_routing.py: 11 tests (resolución, gates, endpoints).
- test_messages_native.py: routing override end-to-end (URL y model del provider ruteado).
- test_llamacpp_service.py: cmdline router mode.
Cobertura global del proyecto: suite backend 114 passed; tsc frontend sin errores.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Resumen
haiku(background de Claude Code) → modelo chico local,opus→ Anthropic real (directo a api.anthropic.com), umbralmin_tokenspara longContext. UI en Providers.--models-dir); combinado con routing = un solo server, modelo grande para código + chico para background..envdel config dir no está enos.environ.Test plan
tsc --noEmitlimpio🤖 Generated with Claude Code