v2.10.0 — Proveedor LLM local multi-GPU (llama.cpp) + descarga de modelos HF

.github/copilot-instructions.md

@@ -0,0 +1,3 @@
+# GitHub Copilot — instrucciones
+
+Las instrucciones de proyecto para asistentes AI están en [`CLAUDE.md`](../CLAUDE.md) en la raíz del repo. Contiene arquitectura, layout, comandos de build/test y convenciones del proyecto.

AGENTS.md

@@ -0,0 +1,3 @@
+# AGENTS.md
+
+Las instrucciones de proyecto para asistentes AI están en [`CLAUDE.md`](./CLAUDE.md) en la raíz del repo. Léelo antes de trabajar: contexto de arquitectura, layout, comandos y convenciones.

CLAUDE.md

@@ -53,12 +53,13 @@ pyinstaller bipolar-code.spec   # run from repo root
 | Config | `core/config.py` | Reads `.env` from config dir via pydantic-settings; config dir is `C:\litellm` (Win) / `~/.litellm` (other) |
 | Services | `services/providers_service.py` | Provider CRUD, litellm config YAML generation, kill/start litellm subprocess |
 | Services | `services/proxy_service.py` | Proxy health checks; Claude Code routing (writes `ANTHROPIC_BASE_URL`/`ANTHROPIC_API_KEY` to `~/.claude/settings.json` and Windows registry) |
+| Services | `services/llamacpp_service.py` | llama-server local gestionado: detección de GPUs, tensor-split auto por VRAM libre, start/stop con PID file |
 | API | `api/` | Thin FastAPI routers — each delegates to the matching `*_service.py` |
 | Models | `models/` | Pydantic schemas (`schemas.py`) and provider entity (`provider.py`) |
 | Startup | `main.py` | Mounts `frontend/dist/` as SPA fallback; spawns Copilot token auto-refresh background loop |
 | Logging | `core/logging.py` | structlog setup; use `get_logger(__name__)` throughout |
 
-**Provider registry** persists in `{config_dir}/providers.json`. Three built-in providers: `copilot`, `anthropic`, `lmstudio`. litellm always exposes the aliases `claude-sonnet-4-6`, `claude-opus-4-6`, `gpt-4o` regardless of the active backend.
+**Provider registry** persists in `{config_dir}/providers.json`. Built-in providers include `copilot`, `anthropic`, `lmstudio`, `nvidia_nim`, `openrouter`, `deepseek`, `ollama` and `llamacpp`. litellm always exposes the aliases `claude-sonnet-4-6`, `claude-opus-4-6`, `gpt-4o` regardless of the active backend. Providers with `anthropic_native: true` (llama-server, LM Studio ≥0.4.1, Ollama 2026+) receive `/v1/messages` verbatim — no litellm, no OAI translation. The `llamacpp` provider spawns a managed local `llama-server` (Vulkan multi-GPU, port 4002) via `/api/llamacpp/*`.
 
 **Platform guards**: `providers_service._start_litellm` uses PowerShell on Windows and `subprocess.Popen(start_new_session=True)` on Linux/macOS. `proxy_service._set_user_env` writes the Windows registry only on `sys.platform == "win32"`; it always writes `~/.claude/settings.json`.
 

backend/app/api/llamacpp.py

@@ -0,0 +1,118 @@
+"""
+Router del servidor llama.cpp local y descarga de modelos GGUF desde HF.
+Thin — delega en llamacpp_service y hf_models_service.
+"""
+from fastapi import APIRouter, HTTPException
+from pydantic import BaseModel
+
+from app.core.logging import get_logger
+from app.core.utils import sanitize_error
+from app.services import hf_models_service, llamacpp_service, providers_service
+
+log = get_logger(__name__)
+router = APIRouter(prefix="/llamacpp", tags=["llamacpp"])
+
+
+class DownloadRequest(BaseModel):
+    repo_id: str
+    filename: str
+
+
+class UseModelRequest(BaseModel):
+    path: str
+
+
+def _llamacpp_provider():
+    provider = providers_service.get_provider("llamacpp")
+    if not provider:
+        raise HTTPException(status_code=404, detail="Provider 'llamacpp' no registrado")
+    return provider
+
+
+@router.get("/devices")
+async def devices():
+    provider = _llamacpp_provider()
+    exe = llamacpp_service.resolve_exe(provider)
+    return {
+        "exe_found": bool(exe),
+        "exe_path": exe,
+        "devices": llamacpp_service.list_devices(exe),
+    }
+
+
+@router.get("/status")
+async def status():
+    return await llamacpp_service.get_status(_llamacpp_provider())
+
+
+@router.post("/start")
+async def start():
+    try:
+        return await llamacpp_service.start_llamacpp(_llamacpp_provider())
+    except ValueError as e:
+        raise HTTPException(status_code=400, detail=sanitize_error(str(e)))
+    except Exception as e:
+        log.error("llamacpp_start_failed", error=str(e))
+        raise HTTPException(status_code=500, detail=sanitize_error(str(e)))
+
+
+@router.post("/stop")
+async def stop(force: bool = False):
+    try:
+        return await llamacpp_service.stop_llamacpp(_llamacpp_provider(), force=force)
+    except RuntimeError as e:
+        raise HTTPException(status_code=409, detail=sanitize_error(str(e)))
+
+
+@router.get("/logs")
+async def logs(lines: int = 80):
+    return {"logs": llamacpp_service.tail_logs(lines)}
+
+
+# ── Modelos GGUF (Hugging Face) ──────────────────────────────────────────────
+
+@router.get("/hf/search")
+async def hf_search(q: str):
+    try:
+        return {"results": await hf_models_service.search_gguf(q)}
+    except Exception as e:
+        raise HTTPException(status_code=502, detail=sanitize_error(str(e)))
+
+
+@router.get("/hf/files")
+async def hf_files(repo_id: str):
+    try:
+        return {"files": await hf_models_service.list_repo_gguf_files(repo_id)}
+    except Exception as e:
+        raise HTTPException(status_code=502, detail=sanitize_error(str(e)))
+
+
+@router.post("/hf/download")
+async def hf_download(body: DownloadRequest):
+    try:
+        return await hf_models_service.start_download(body.repo_id, body.filename)
+    except Exception as e:
+        raise HTTPException(status_code=502, detail=sanitize_error(str(e)))
+
+
+@router.get("/hf/downloads")
+async def hf_downloads():
+    return {"downloads": hf_models_service.get_downloads()}
+
+
+@router.delete("/hf/download/{download_id:path}")
+async def hf_cancel(download_id: str):
+    return {"cancelled": hf_models_service.cancel_download(download_id)}
+
+
+@router.get("/models")
+async def local_models():
+    return {"models": hf_models_service.list_local_models()}
+
+
+@router.post("/use-model")
+async def use_model(body: UseModelRequest):
+    provider = _llamacpp_provider()
+    launch = {**provider.local_launch, "model_path": body.path}
+    updated = providers_service.update_provider(provider.id, {"local_launch": launch})
+    return {"model_path": updated.local_launch.get("model_path", "")}

backend/app/api/messages.py

@@ -326,6 +326,7 @@ async def messages_passthrough(request: Request):
     active = providers_service.get_active_provider()
     active_provider_id = active.id if active else "unknown"
     is_anthropic = active and active.litellm_prefix == "anthropic"
+    is_native = bool(active and active.anthropic_native)
 
     ctx_window = token_service.get_context_window(model)
     used = token_service.count_tokens(messages)
@@ -351,18 +352,30 @@ async def generate():
             timeout = httpx.Timeout(connect=10.0, read=120.0, write=10.0, pool=10.0)
             async with httpx.AsyncClient(timeout=timeout) as client:
 
-                if is_anthropic:
-                    # Anthropic provider: passthrough to litellm /v1/messages
-                    forward_headers = {
-                        "Authorization": f"Bearer {settings.proxy_api_key}",
-                        "Content-Type": "application/json",
-                    }
+                if is_anthropic or is_native:
+                    if is_native:
+                        # Provider con /v1/messages nativo (llama-server, LM Studio >=0.4.1,
+                        # Ollama 2026+): reenvío verbatim, solo se reescribe el model
+                        body["model"] = active.active_model or model
+                        native_base = active.api_base.rstrip("/").removesuffix("/v1")
+                        target_url = f"{native_base}/v1/messages"
+                        forward_headers = {"Content-Type": "application/json"}
+                        native_key = os.environ.get(active.auth_env_var, "") if active.auth_env_var else ""
+                        if native_key:
+                            forward_headers["x-api-key"] = native_key
+                    else:
+                        # Anthropic provider: passthrough to litellm /v1/messages
+                        target_url = f"{settings.proxy_url}/v1/messages"
+                        forward_headers = {
+                            "Authorization": f"Bearer {settings.proxy_api_key}",
+                            "Content-Type": "application/json",
+                        }
                     for h in ("anthropic-version", "anthropic-beta"):
                         if h in request.headers:
                             forward_headers[h] = request.headers[h]
 
                     async with client.stream(
-                        "POST", f"{settings.proxy_url}/v1/messages",
+                        "POST", target_url,
                         json=body, headers=forward_headers,
                     ) as resp:
                         if resp.status_code >= 400:
@@ -505,6 +518,16 @@ async def generate():
 
                     _record_usage(active_provider_id, model, usage_buf, truncated)
 
+        except httpx.ConnectError as e:
+            if is_native and active:
+                log.error("native_provider_unreachable", api_base=active.api_base)
+                yield _sse_error(
+                    f"Servidor local no responde en {active.api_base}. "
+                    "Inícialo desde Providers → llama.cpp (Start)."
+                )
+            else:
+                log.error("messages_passthrough_error", error=_sanitize_error(str(e)))
+                yield _sse_error(_sanitize_error(str(e)))
         except Exception as e:
             log.error("messages_passthrough_error", error=_sanitize_error(str(e)))
             yield _sse_error(_sanitize_error(str(e)))

backend/app/api/settings.py

@@ -46,3 +46,22 @@ async def get_full_api_key():
     """Devuelve el API key completo para configurar Claude Code en PCs remotos."""
     s = _get_settings()
     return {"api_key": s.ui_api_key}
+
+
+@router.get("/connection-info")
+async def get_connection_info():
+    """IP LAN + puerto para configurar Claude Code desde otros PCs."""
+    import socket
+    lan_ip = ""
+    try:
+        with socket.socket(socket.AF_INET, socket.SOCK_DGRAM) as sock:
+            # No envía tráfico: connect en UDP solo resuelve la interfaz de salida
+            sock.connect(("8.8.8.8", 80))
+            lan_ip = sock.getsockname()[0]
+    except OSError as e:
+        log.warning("lan_ip_detection_failed", error=str(e))
+    return {
+        "lan_ip": lan_ip,
+        "port": 8000,
+        "anthropic_base_url": f"http://{lan_ip}:8000" if lan_ip else "",
+    }

backend/app/main.py

@@ -11,6 +11,7 @@
 from app.api import providers as providers_router
 from app.api import messages as messages_router
 from app.api import pricing as pricing_router
+from app.api import llamacpp as llamacpp_router
 from app.core.logging import setup_logging, get_logger
 from app.core.config import get_settings
 
@@ -62,7 +63,9 @@ async def _copilot_token_refresh_loop():
 @asynccontextmanager
 async def lifespan(app: FastAPI):
     from app.services import usage_tracker
+    from app.services.llamacpp_service import autostart_if_configured
     await usage_tracker.init_db()
+    asyncio.create_task(autostart_if_configured())
     task = asyncio.create_task(_copilot_token_refresh_loop())
     try:
         yield
@@ -120,6 +123,7 @@ def create_app() -> FastAPI:
     app.include_router(chat_router.router, prefix="/api")
     app.include_router(messages_router.router)
     app.include_router(pricing_router.router, prefix="/api")
+    app.include_router(llamacpp_router.router, prefix="/api")
 
     @app.get("/api/health")
     async def health():

backend/app/models/provider.py

@@ -33,6 +33,14 @@ class Provider(BaseModel):
     max_tools: int = 0               # 0 = sin límite; >0 trunca el array de tools al enviarlo
     rate_limit_rpm: int = 0          # límite externo del proveedor en req/min (0 = desconocido)
 
+    # Providers con Anthropic Messages API nativa (/v1/messages): llama-server,
+    # LM Studio >=0.4.1, Ollama 2026+. El body se reenvía verbatim sin traducción.
+    anthropic_native: bool = False
+
+    # Config de lanzamiento para providers locales gestionados (llama.cpp):
+    # {exe_path, model_path, ctx_size, split_mode, tensor_split, ngl, extra_args}
+    local_launch: dict = {}
+
 
 class ProviderRegistry(BaseModel):
     active_provider_id: str = "copilot"