GenAI Application: one versioned manifest composing an LLM route, RAG, prompt and guardrail policy

[MSKazemi]

Problem

The platform already has four independently useful GenAI building blocks: an LLM gateway (routing,
virtual keys, budgets), a RAG pipeline (retrieval over a versioned knowledge base), a prompt registry
(immutable, labelled prompt versions), and a guardrail layer (PII/injection/safety scanning in
off/monitor/enforce mode). Building one grounded, guarded application on top of them — say, a docs
Q&A assistant — means hand-wiring all four every time: pick a route, remember a knowledge-base name,
remember a prompt@label, remember which guardrail mode applies, and call each piece in the right
order. Nothing versions the combination, nothing gates a change to it before it reaches production
traffic, and nothing stops a caller from silently skipping a step (the guardrail step, in particular, has
already gone missing once in this codebase because nothing forced every caller to include it).

Proposal: GenAIApplication — one versioned manifest

Introduce a typed, versioned, content-addressed manifest that composes these four references into one
deployable, promotable object:

schema_version: 1
name: hpc-docs-assistant
route:
  model: skipper-default        # a gateway logical model / route
  key_ref: gateway/hpc-docs-assistant
rag:                             # optional — omit for a route+guardrail-only app
  kb: hpc-docs
  retrieval: hybrid
  top_k: 5
  encoder: token-hash
prompt:
  name: hpc-docs-system
  label: prod                    # or an immutable `version:` pin
guardrail:
  mode: enforce                  # off | monitor | enforce
  policy: default
eval:                            # required for promotion to Production
  suites: [hpc-docs-groundedness@1, safety@5]

Each field is a reference into an existing subsystem's own registry — this object never
re-implements routing, retrieval, prompt rendering or guardrail scanning; it only composes them in a
fixed, versioned order: guardrail(input) → optional RAG retrieve → gateway route + generate →
guardrail(output).

Versions are content-addressed (version_id = "gaa-" + sha256(canonical manifest)), stored insert-only,
and promoted through movable aliases (Staging / Canary / Production) — the same shape the
platform's agent-version registry already uses for its own composite artifact. A promotion to
Production is gated: the declared evaluation suites must have results for that exact version, every
judge used must be calibrated, the application may not be promoted with guardrails turned off, and
every referenced component (route, knowledge base, prompt, guardrail policy) must currently resolve.

CLI surface

exa genai-app register <manifest.yaml>     # admin — validate, hash, store
exa genai-app show <name>[@alias]          # read
exa genai-app list [--name <name>]         # read
exa genai-app promote <name> <alias> <ref> # admin, gated to Production
exa genai-app invoke <name>[@alias] --message "…"   # admin — makes a real, billed call

Phased plan and acceptance criteria

Phase 1 — manifest registry. Manifest validation + hashing, storage (three additive tables:
versions, aliases, alias history), register/show/list.
Acceptance: registering identical content twice returns the same version id; an unknown field or a
missing required component is rejected with a named reason; list shows resolved aliases per version.

Phase 2 — promotion gate. Reuses the platform's existing evaluation-gate and judge-calibration
machinery (an uncalibrated judge blocks promotion, named); adds the guardrail-not-off-at-Production rule
and the component-resolves check.
Acceptance: a candidate with guardrail.mode: off cannot reach Production even with a passing
evaluation gate; a dangling knowledge-base reference is refused at promotion time, not on first use.

Phase 3 — invoke. Wires the manifest to the real gateway, retrieval and guardrail calls end to
end, with a typed, fail-closed error for every resolution failure (never a silent skip of a declared
component).
Acceptance: an enforce-mode application whose guardrail dependency is unreachable refuses the call
rather than answering unguarded.

Phase 4 — dashboard surface. List/show/register/promote in the operator UI.

Dependencies

This depends on, and must not duplicate, the platform's existing:

• LLM gateway — routing, retries, circuit breaking, credential handling, egress/locality control,
  cost accounting, and its typed error contract. The application manifest only ever names a gateway
  route by reference; it holds no provider credential and implements no routing logic of its own.
• RAG pipeline — retrieval, reranking, knowledge-base versioning and per-tenant isolation.
• Prompt registry — immutable versions, labelled resolution, eval-gated label promotion.
• Guardrail layer — PII/injection/safety scanning in off/monitor/enforce mode.

Likely touched modules

• platform/cli/src/examlops/genai_apps/ (new: manifest.py, service.py)
• platform/cli/src/examlops/data/genai_apps.py (new)
• platform/cli/src/examlops/cli/commands/genai_app_cmd.py (new)
• platform/cli/src/examlops/platform_db.py (additive schema: genai_applications,
  genai_app_aliases, genai_app_alias_history)
• Existing, unmodified, called by reference only: platform/cli/src/examlops/gateway/,
  platform/cli/src/examlops/rag/, platform/cli/src/examlops/prompts/,
  platform/cli/src/examlops/guardrails/