022: registry-driven LLM serving + operator model selection (offline slice, T461–T488)

README.md

@@ -18,6 +18,9 @@ a heavier reference platform, sized to a laptop.
 > **020 stack-remediation** landed on top: the archived-upstream object store exited to **Garage**
 > (migrated + decommissioned), the serving framework left the children ("Bento-ectomy", byte-identical
 > golden gates), and the agent's HTTP runtime was measured on hardware (verdict: keep-stdlib).
+> **021** added the loop-native operator console; **022** makes the LLM **registry-driven** like the
+> other engines (promote → serve, base + LoRA-adapter resolution, honest served identity — offline
+> slice; the on-hardware switch drills are the [HW] tail).
 > Constitution `v1.5.1`. Reference stack on MLflow `3.14.0`.
 >
 > Per-increment detail lives under [`specs/`](specs/) (each has `spec.md` / `plan.md` / `tasks.md`);
@@ -118,6 +121,26 @@ with the same serving tags **plus lineage** (`base_model`, parent run) so adapte
 Promotion to `@serving` is **operator-driven** (alias-based) and, since **011**, runs through an
 **evaluation gate** (below).
 
+**The LLM is registry-driven too (022).** Pre-022 the `llama.cpp` engine was the sole outlier — it
+loaded a fixed GGUF from the `MODEL` env and ignored the registry, so promoting an LLM did nothing.
+Now **promote = go live**: promoting a text-generation version from the console moves its
+`@serving` alias, records it as the **active serving LLM** (a one-row platform pointer in the
+relational store — it spans model *names*, base vs fine-tune), and triggers an **immediate
+controlled reload** on the agent (evict → load under admission; displacing a resident serving
+model is operator-confirmed; a running job is never preempted — the switch defers instead). At
+each cold load the adapter resolves the active model's `@serving` version from the registry:
+a `kind=full-model` version serves `-m <gguf>`; a `kind=lora-adapter` fine-tune serves
+`-m <base> --lora <adapter>`, with the **base resolved automatically from `base_model` lineage**
+to a registered base GGUF (`scripts/register_base_gguf.py` registers the local zoo bases;
+an unresolvable/absent base **refuses the promotion** and leaves the served LLM unchanged —
+nothing is auto-downloaded). Served identity is **honest end-to-end**: the agent reports the
+actually-loaded model + registry version in its health, and `/serving/state`, the `/infer`
+response, and every logged prediction consume that agent-reported identity (monitoring windows key
+on the model that really produced each output). The `MODEL`/`LORA`/`MODEL_ALIAS` env knobs remain
+only as the **fallback default** for when no serving LLM has been selected. Legacy pre-022 LLM
+versions are recognized by artifact kind and can be batch-tagged with
+`scripts/backfill_llm_task_tags.py` (idempotent, never clobbers an existing tag).
+
 ## Run it
 
 See [quickstart](specs/001-mlops-platform/quickstart.md).

docker-compose.yml

@@ -129,6 +129,10 @@ services:
       # since host.docker.internal can't cross WSL distros.
       AGENT_URL: ${AGENT_URL:-http://host.docker.internal:8100}
       SERVING_MODEL: ${SERVING_MODEL:-qwen2.5-7b-instruct-q4_k_m}
+      # 022 (FR-255): the gated promote requests the agent's /control/reload so a promoted LLM
+      # goes live at once. Empty when the agent runs open (the default); must MATCH the agent's
+      # AGENT_CONTROL_SECRET when one is set (sent as X-Agent-Control).
+      AGENT_CONTROL_SECRET: ${AGENT_CONTROL_SECRET:-}
       # Garage/S3 for the dataset registry (US3) — the store-minted pair (T406 cutover).
       MLFLOW_S3_ENDPOINT_URL: http://garage:3900
       AWS_ACCESS_KEY_ID: "${GARAGE_ACCESS_KEY_ID:?set GARAGE_ACCESS_KEY_ID in .env (scripts/gen_secrets --record-garage)}"

gateway/app/registry.py

@@ -8,14 +8,21 @@
 The MLflow server version is pinned in infra/mlflow (3.14.0, 007 US1); this client matches it.
 """
 import os
+import time
 from typing import Optional
 
 from mlflow.exceptions import MlflowException
 from mlflow.tracking import MlflowClient
 
+from platformlib import llmresolve
+
 MLFLOW_TRACKING_URI = os.getenv("MLFLOW_TRACKING_URI", "http://mlflow:5000")
 SERVING_ALIAS = "serving"
 
+# 022: the configured default base — what serves when no serving-LLM was ever selected (the
+# ActiveServingLLM pointer's documented unset behavior, data-model.md).
+DEFAULT_LLM = os.getenv("SERVING_MODEL", "qwen2.5-7b-instruct-q4_k_m")
+
 # 009 US1 — registry routing metadata (FR-074). Every version SHOULD carry these version tags; the
 # gateway resolves "the serving model+engine for task X" from them rather than a hard-coded
 # endpoint→model map (FR-075). Versions registered before 009 lack them and are tolerated (task=None).
@@ -128,6 +135,91 @@ def promote(name: str, version: str, override: bool = False) -> dict:
             "promoted": True, "verdict": verdict}
 
 
+# --- 022 US1/US4: the active serving-LLM pointer + LLM target resolution ---------------------------
+#
+# The pointer (data-model.md §ActiveServingLLM) lives in the relational store (T461), NOT in an
+# MLflow alias — it spans model NAMES (base vs fine-tune), which a per-model alias cannot express
+# (research R1). Reads are best-effort (unset/unreachable ⇒ the configured default base — the
+# documented unset behavior); the WRITE fails loud (RegistryError) so a promote whose go-live
+# pointer didn't stick is surfaced, never silent.
+
+
+def _store():
+    from platformlib import store
+    return store
+
+
+def get_serving_llm() -> Optional[dict]:
+    """The active serving-LLM selection {model_name, selected_at, selected_by}, or None when unset
+    (or the store is unreachable — both read as 'the configured default base serves')."""
+    try:
+        s = _store()
+        conn = s.connect()
+        try:
+            return s.get_serving_llm(conn)
+        finally:
+            conn.close()
+    except Exception:  # noqa: BLE001 — best-effort read; unset ⇒ default base
+        return None
+
+
+def set_serving_llm(model_name: str, actor: str = "operator") -> dict:
+    """Record `model_name` as the active serving LLM (022 US1 — the gated promote IS the go-live
+    action; this is its persistence half, the agent reload is the other). Fail-loud."""
+    try:
+        s = _store()
+        conn = s.connect()
+        try:
+            s.set_serving_llm(conn, model_name, time.time(), actor)
+        finally:
+            conn.close()
+    except Exception as e:  # noqa: BLE001 — normalize driver/connection errors
+        raise RegistryError(f"active serving-LLM pointer write failed: {e}") from e
+    return {"model_name": model_name, "selected_by": actor}
+
+
+def active_serving_llm_name() -> str:
+    """The model name that SHOULD be serving text-generation: the pointer, else the configured
+    default base (FR-276 — the disambiguator when several LLM models hold a promoted alias)."""
+    rec = get_serving_llm()
+    return rec["model_name"] if rec else DEFAULT_LLM
+
+
+def llm_target_info(name: str, version: str) -> Optional[dict]:
+    """Pre-promotion resolution check for a text-generation version (022 T467/T474, FR-265).
+
+    Returns None when (name, version) is not a text-generation target (task tag or inferred kind
+    — T478), else {task, kind, base, error}: `error` is set when the target cannot resolve at the
+    REGISTRY level (adapter with no/unresolvable base — the artifact-presence half is the agent's
+    probe). The promote route refuses on `error` BEFORE the gate runs, so the alias — and the
+    currently-served LLM — stay unchanged."""
+    c = _client()
+    try:
+        mv = c.get_model_version(name, str(version))
+    except MlflowException as e:
+        raise RegistryError(str(e)) from e
+    tags = dict(mv.tags or {})
+    task = tags.get(TASK_TAG) or llmresolve.task_from_kind(tags)
+    if task != llmresolve.TEXT_GENERATION:
+        return None
+    kind = llmresolve.effective_kind(tags)
+    info = {"task": task, "kind": kind, "base": None, "error": None}
+    if kind == llmresolve.KIND_ADAPTER:
+        base_ref = tags.get(llmresolve.BASE_MODEL_TAG)
+        if not base_ref:
+            info["error"] = (f"{name} v{version} is a LoRA adapter with no base_model lineage — "
+                             f"its base cannot be resolved (FR-264/265)")
+            return info
+        try:
+            bmv = llmresolve.resolve_base_version(c, base_ref)
+            info["base"] = {"name": bmv.name, "version": str(bmv.version), "source": bmv.source}
+        except llmresolve.LLMResolutionError as e:
+            info["error"] = str(e)
+        except MlflowException as e:
+            raise RegistryError(str(e)) from e
+    return info
+
+
 # --- 009 US1: task/serving-engine routing metadata ------------------------------------------------
 
 def set_version_tags(name: str, version: str, tags: dict) -> dict:
@@ -218,15 +310,50 @@ def resolve_serving_target(task: str, prefer_name: Optional[str] = None) -> Opti
             return entry  # the configured backend model wins over any other task-mate
         if first is None:
             first = entry
+    if first is None and task == llmresolve.TEXT_GENERATION:
+        # 022 T478 (FR-267) defense-in-depth: a legacy promoted LLM version with NO task tag
+        # (kind/format only, pre-backfill) is invisible to the tag search above — infer its task
+        # from the artifact kind so no valid LLM version is stranded as unroutable.
+        try:
+            models = c.search_registered_models()
+        except MlflowException as e:
+            raise RegistryError(str(e)) from e
+        for m in models:
+            sv = (dict(m.aliases or {})).get(SERVING_ALIAS)
+            if not sv:
+                continue
+            try:
+                mv = c.get_model_version(m.name, sv)
+            except MlflowException:
+                continue
+            tags = dict(mv.tags or {})
+            if tags.get(TASK_TAG) or llmresolve.task_from_kind(tags) != task:
+                continue  # tagged versions were already considered by the search above
+            entry = {"name": m.name, "version": str(sv), "task": task,
+                     "serving_engine": _resolve_engine(c, mv), "source": mv.source}
+            if prefer_name is not None and m.name == prefer_name:
+                return entry
+            if first is None:
+                first = entry
     return first
 
 
+#: The lineage tags surfaced per LLM serving target (FR-268: base / parent / dataset provenance).
+_LINEAGE_KEYS = ("base_model", "dataset_name", "dataset_version", "parent_version", "lineage")
+
+
 def list_tasks() -> list:
     """Distinct serving tasks discovered from the registry — one entry per registered model's
     `@serving` version, reporting its `task`/`serving_engine` (FR-077). The Infer tab renders one
     panel per task; a serving version with no `task` tag (legacy, pre-009) is reported as task=None
     so the UI degrades it to a read-only "no renderer" placeholder rather than breaking the tab.
-    """
+
+    022 (T478/T479): a text-generation version's task is INFERRED from its artifact kind when the
+    tag is missing (legacy fine-tunes render a real LLM panel, not "no renderer" — FR-267), each
+    LLM entry carries its base-vs-adapter `kind` + lineage (FR-268), and the text-generation rows
+    are filtered to the ACTIVE serving-LLM pointer (FR-276): a promote moves only its own model's
+    `@serving` alias, so several LLM models can each retain one — but only the active-pointer
+    model is the live target; a previously-active LLM must not linger as a stale duplicate."""
     c = _client()
     try:
         models = c.search_registered_models()
@@ -242,10 +369,21 @@ def list_tasks() -> list:
         except MlflowException:
             continue
         tags = dict(mv.tags or {})
-        out.append({
+        task = tags.get(TASK_TAG) or llmresolve.task_from_kind(tags)
+        entry = {
             "model": m.name,
             "version": str(sv),
-            "task": tags.get(TASK_TAG),
+            "task": task,
             "serving_engine": _resolve_engine(c, mv),
-        })
+        }
+        if task == llmresolve.TEXT_GENERATION:
+            entry["kind"] = llmresolve.effective_kind(tags)
+            lineage = {k: tags[k] for k in _LINEAGE_KEYS if tags.get(k)}
+            entry["lineage"] = lineage or None
+        out.append(entry)
+    text_gen = [e for e in out if e["task"] == llmresolve.TEXT_GENERATION]
+    if len(text_gen) > 1:
+        active = active_serving_llm_name()
+        keep = next((e for e in text_gen if e["model"] == active), text_gen[0])
+        out = [e for e in out if e["task"] != llmresolve.TEXT_GENERATION or e is keep]
     return out

gateway/app/routers/infer.py

@@ -12,12 +12,12 @@
 
 from .. import quality, registry, tracing
 from ..serving import (
-    SERVING_MODEL,
     ModelTooLargeError,
     ServingBusyError,
     ServingError,
     gpu_state,
     health,
+    llm_identity,
     run_inference,
 )
 
@@ -35,29 +35,24 @@ class InferRequest(BaseModel):
     preempt: bool = False  # 017: opt-in swap — evict a resident *serving* model first (default 008 refuse)
 
 
-async def _resolve_serving_version() -> str | None:
-    """Registry version currently serving the LLM (FR-006/FR-075).
-
-    /infer always proxies to the single llama supervisor configured by SERVING_MODEL, so SERVING_MODEL's
-    `@serving` alias is authoritative for the reported version — resolve it FIRST. This preserves pre-009
-    /infer behavior exactly (T156): the promoted version is reported even when it predates 009's `task`
-    tags, and a *different* promoted text-generation model can never be mis-reported as the served one.
-    (The earlier task-first resolve mis-fired when SERVING_MODEL's `@serving` version was untagged but a
-    text-generation tag existed elsewhere — `resolve_serving_target` then returned that other model's
-    version, since a candidate must be BOTH `task`-tagged AND its model's `@serving` version.)
-
-    The task-based resolve remains only as a fallback for the edge where SERVING_MODEL has nothing
-    promoted to `@serving`. Best-effort — any failure yields None (the response just omits the version).
-    """
-    try:
-        served = await run_in_threadpool(registry.get_serving, SERVING_MODEL)
-        if served:
-            return served["version"]
-        target = await run_in_threadpool(
-            registry.resolve_serving_target, "text-generation", SERVING_MODEL)
-        return target["version"] if target else None
-    except Exception:
-        return None
+async def _served_identity() -> dict:
+    """The identity every surface attributes this inference to (022 US2, FR-260/261/262): the
+    AGENT-REPORTED {serving_model, serving_version} — the agent is the only component that knows
+    what is actually resident, so the response's registry_model/registry_version, the logged
+    prediction, and `/serving/state` all name the same model+version (the pre-022 divergence —
+    `model: ops-bot-v2` logged as `registry_model: qwen…` — cannot recur). When the agent doesn't
+    report a registry version (env-default binding on a legacy agent), the version falls back to
+    the reported MODEL's own `@serving` alias — still keyed to the served name, never to the
+    fixed SERVING_MODEL config."""
+    ident = await llm_identity()
+    if ident["serving_model"] != "unknown" and ident["serving_version"] is None:
+        try:
+            served = await run_in_threadpool(registry.get_serving, ident["serving_model"])
+            if served:
+                ident["serving_version"] = served["version"]
+        except Exception:
+            pass  # best-effort — the response just omits the version
+    return ident
 
 
 @router.post("/infer")
@@ -69,7 +64,7 @@ async def infer(req: InferRequest):
     """
     start_ns = time.time_ns()
     result = None
-    registry_version = None
+    served_model, registry_version = "unknown", None
     # Pessimistic default: any UNEXPECTED failure (e.g. a malformed supervisor response) keeps the trace
     # errored — only the known-good path flips it to OK just before returning (006 Codex review).
     trace_status = "ERROR"
@@ -106,20 +101,23 @@ async def infer(req: InferRequest):
             LOAD_LATENCY.observe(result["load_ms"] / 1000.0)
         INFER_LATENCY.observe(result.get("infer_ms", 0) / 1000.0)
         INFER_REQUESTS.labels(status="ok").inc()
-        registry_version = await _resolve_serving_version()
+        # 022 US2 (FR-260/261/262): attribute everything — response identity, prediction log,
+        # trace — to the AGENT-REPORTED served identity, not the fixed SERVING_MODEL config.
+        ident = await _served_identity()
+        served_model, registry_version = ident["serving_model"], ident["serving_version"]
         trace_status, outcome = "OK", "completed"  # success is known — flip the pessimistic default
         # 013/FR-119: log the served prediction off the request path (fire-and-forget, fail-open) so it
         # can be scored later against a delayed label. Returns a synchronous id regardless of store state.
         prediction_id = quality.log_prediction(
-            SERVING_MODEL, registry_version, "text-generation", req.prompt, result.get("text"))
+            served_model, registry_version, "text-generation", req.prompt, result.get("text"))
         # 016 (FR-146): also route the prompt to the bounded recoverable-input capture (uniform replay
         # corpus across modalities), so it can be shadow-replayed. The served decoding settings ride along
         # so a replay reproduces them, not the scorer defaults. Fire-and-forget + fail-open.
         quality.capture_input(prediction_id, "text-generation", req.prompt,
                               options={"max_tokens": req.max_tokens, "temperature": req.temperature})
         return {
             "status": "completed",
-            "registry_model": SERVING_MODEL,
+            "registry_model": served_model,
             "registry_version": registry_version,
             "prediction_id": prediction_id,
             **result,
@@ -128,7 +126,7 @@ async def infer(req: InferRequest):
         attrs = {
             "max_tokens": req.max_tokens,
             "temperature": req.temperature,
-            "model": SERVING_MODEL,
+            "model": served_model,
             "status": outcome,
         }
         if result:
@@ -156,10 +154,22 @@ async def serving_health():
 @router.get("/serving/state")
 async def serving_state():
     """GPU/lease state for the Infer status line (008 US3, FR-068): {holder, resident,
-    serving_model, serving_version}. holder ∈ {llm, vision, training, null}; the version is the
-    registry @serving pointer. Read-only; the API key never reaches the browser (BFF contract)."""
+    serving_model, serving_version, base, adapter}. holder ∈ {llm, vision, training, null}.
+
+    022 (FR-260, T470): the model+version are the AGENT-REPORTED served identity (what is actually
+    resident, or what the next load would serve when cold) — `unknown` when the agent is
+    unreachable, never a stale config guess. `base`/`adapter` expose a served fine-tune's resolved
+    provenance (FR-274). Read-only; the API key never reaches the browser (BFF contract)."""
     state = await gpu_state()
-    state["serving_version"] = await _resolve_serving_version()
+    if state["serving_model"] != "unknown" and state["serving_version"] is None:
+        # Legacy-agent tolerance: no registry_version reported → the served NAME's own @serving
+        # alias (still keyed to the reported name — the same rule as _served_identity).
+        try:
+            served = await run_in_threadpool(registry.get_serving, state["serving_model"])
+            if served:
+                state["serving_version"] = served["version"]
+        except Exception:
+            pass
     return state