Hermes Model Hub

One dependable OpenAI-compatible endpoint for multiple explicit model pools, with a protected operator console.

Model Hub sits between Hermes Agent and model servers you already run. Humans manage one small catalog; agents use standard model names through one URL. Replica health, qualification, failover, atomic updates, rollback, and the browser control plane stay behind that boundary.

It is designed for Nous Research Hermes Agent and works with other clients that use the OpenAI Chat Completions API. The console adds local, bounded, content-free operator activity and metrics; it does not export telemetry to an external service.

Five-minute first run

1. Install

For a local checkout:

uv tool install .

Or install the public v0.2.0 release tag after publication:

uv tool install git+https://github.com/r0b0tlab/hermes-model-hub.git@v0.2.0

For development:

uv sync --extra dev

2. Launch the console

The primary first-run journey is the browser console:

hermes-pool console

The launcher runs the same FastAPI/uvicorn application used by serve, waits for the application and listener to finish startup, then opens /console/ in the default browser. Use --no-open for SSH or headless sessions; --host, --port, and --log-level override the active listener. Ports must be between 1 and 65535.

With no active catalog, the console is available on loopback and guides setup without first creating current.yaml. The default URL is:

http://127.0.0.1:8080/console/

The state home is ~/.hermes-model-hub by default and can be changed with HERMES_MODEL_HUB_HOME. Use a disposable state home when testing or demonstrating the console.

3. Create a catalog in the browser

In Set up / Edit, add a public alias and one or more OpenAI-compatible /v1 endpoint URLs. You may specify an upstream model ID, an upstream credential environment-variable name, a description, and whether the alias requires tool calling. The server remains the source of truth for validation and normalization; the browser does not implement a second YAML schema.

A catalog can also be written or bootstrapped with the CLI:

hermes-pool init models.yaml
# or
hermes-pool setup --path models.yaml

The normal configuration is intentionally small:

version: 1
default: agent
models:
  agent:
    description: Reliable tool-capable agent model
    require_tools: true
    endpoints:
      - http://qwen-1:8000/v1
      - http://qwen-2:8000/v1
  fast:
    endpoints:
      - http://small-model:8000/v1
health:
  interval_seconds: 60
  failure_threshold: 2
  recovery_successes: 1
listen:
  host: 127.0.0.1
  port: 8080
console:
  enabled: true

If an endpoint advertises several upstream model IDs, specify the right one:

- url: http://server:8000/v1
  model: Qwen3.6-35B-A3B

If an upstream requires authentication, reference an environment variable; never put the key in YAML:

- url: https://models.internal/v1
  api_key_env: INTERNAL_MODEL_KEY

The Hub credential and every upstream credential must use different environment-variable names and different secret values. Candidates that overlap are rejected before activation, and the runtime rechecks this boundary before upstream requests.

4. Qualify, review, and activate

From the console, select Qualify to run real model discovery and chat canaries. Models marked require_tools: true must also produce a real function call. The console shows per-endpoint results, normalized candidate/current/activation views, and the operation lifecycle.

Activation is explicit and server-authoritative:

• strict mode rejects the candidate if any configured endpoint fails;
• Allow degraded can be selected only when every public alias still has a qualified endpoint, and failed replicas are excluded from active traffic;
• the candidate hash and current-configuration hash are checked again before the atomic activation point;
• a failed or stale activation leaves the previously active configuration unchanged.

The equivalent CLI boundary is:

hermes-pool apply models.yaml
# only when every alias still has a qualified endpoint:
hermes-pool apply models.yaml --allow-degraded

5. Connect Hermes or operate the hub

The console can preview and confirmation-gate a Hermes connection using the same connector logic as the CLI. It writes a private timestamped backup before a change and never changes the active Hermes profile during first-run setup.

For the CLI workflow:

hermes-pool connect hermes
# automation in a controlled environment:
hermes-pool connect hermes --yes

Start a new Hermes session after connecting. Use Hermes /model to select another published alias.

Browser console and protected admin workflows

The public shell at /console/ is a package-owned, versioned static bundle. It is intentionally identical whether private state is empty or configured and contains no catalog, endpoint, credential, or event data. The shell calls the protected /admin/v1 control API only after a browser session is established.

The browser workflow covers:

• first-run bootstrap and guided or advanced-YAML draft editing;
• current/empty private draft creation, normalized validation errors, bounded draft persistence, and optimistic hash conflicts;
• real qualification with per-endpoint results;
• candidate-versus-current review, strict/degraded activation policy, and serialized apply operations;
• live overview, model/replica health, bounded request metrics, operation history, and redacted activity events;
• SSE reconnect/resume using event IDs without treating stale browser state as authoritative;
• confirmation-gated rollback preview and activation;
• Hermes provider preview and confirmation-gated connection with a private backup.

The control-plane routes are outside /v1 and use stable {ok, data, error, request_id} envelopes. The main routes are:

• POST/DELETE /admin/v1/session — exchange a Hub bearer for or revoke a browser session;
• GET /admin/v1/snapshot and GET /admin/v1/events — protected state and redacted SSE activity;
• POST/GET/PUT/DELETE /admin/v1/drafts and /admin/v1/drafts/{id} — private candidate drafts;
• POST /admin/v1/drafts/{id}/qualify and /admin/v1/drafts/{id}/apply — qualification and activation jobs;
• GET/DELETE /admin/v1/operations/{id} — inspect or cancel an operation before its irreversible cutoff;
• POST /admin/v1/rollback/preview and /admin/v1/rollback — review and commit a rollback;
• POST /admin/v1/hermes/preview and /admin/v1/hermes/connect — review and commit the Hermes connection.

Browser security mode

• Loopback is the safe default. A non-loopback enabled console is refused unless api_key_env names a non-empty Hub bearer environment variable; wildcard binds remain the listener bind, while the displayed client URL uses a loopback address.
• Browser login accepts the Hub bearer only for the session exchange. The server returns an HttpOnly, SameSite=Strict, path-scoped, process-local signed session cookie; the bearer is not printed, stored in browser storage, or retained as session state. Restarting the process invalidates old sessions.
• Mutating admin requests require both a canonical same-origin Origin and a session-bound CSRF token. Admin and console responses include strict CSP, X-Content-Type-Options: nosniff, Referrer-Policy: no-referrer, and frame denial.
• With an active configuration setting console.enabled: false, the console and admin namespace are unavailable. The launcher refuses before starting that disabled console.

API and compatibility boundary

The public data-plane API remains intentionally small and backward-compatible:

• GET /healthz
• GET /v1/models
• GET /v1/model-capabilities
• POST /v1/chat/completions (streaming and non-streaming)

This four-route list describes the inference/data-plane contract, not the complete Model Hub product. The protected console/admin routes above provide operator setup, qualification, health/activity views, operations, rollback, and Hermes connection workflows. The public API still deliberately does not proxy embeddings, images, audio, or the Responses API.

Hermes and other OpenAI-compatible clients use:

http://127.0.0.1:8080/v1

Discover models:

curl http://127.0.0.1:8080/v1/models

Select one explicitly:

curl http://127.0.0.1:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"fast","messages":[{"role":"user","content":"Hello"}]}'

Omitting model uses the configured default. The Hub never silently changes one public model alias into another.

To protect the Hub data plane itself, configure a Hub bearer environment variable:

api_key_env: HERMES_MODEL_HUB_API_KEY

Clients then send:

Authorization: Bearer ***

The service fails closed if the configured Hub key environment variable is absent. The key is separate from every upstream key and is never forwarded upstream.

Daily operation

hermes-pool status
hermes-pool check
hermes-pool add http://vision:8000/v1 --name vision
hermes-pool apply models.yaml
hermes-pool rollback

add edits the human-owned catalog but does not change live traffic. apply is the explicit qualification and activation boundary.

Reliability and local observability

• Functional qualification before activation—not HTTP 200 alone.
• Replica routing only inside one public model alias.
• Bounded retry on connection failures, timeouts, and HTTP 5xx before response bytes are emitted.
• Circuit breaker quarantines repeatedly failing replicas and functional canaries restore them after recovery.
• Tool-required replicas must pass both chat and tool-call canaries before recovery.
• Streaming requests are never replayed after any response bytes may have reached the caller.
• Atomic current.yaml replacement and private rollback revisions under the state home.
• A failed candidate apply cannot overwrite the last-known-good configuration.
• Hub credentials are validated at the Hub and never forwarded upstream.
• Local activity events, rotating JSONL history, operation summaries, and aggregate request/replica metrics are bounded and content-free. They retain control-plane metadata only; they do not capture prompts, responses, authorization headers, or raw exception text, and they are not exported to a telemetry service.

Development and verification

uv sync --extra dev
uv run pytest
uv run python -m scripts.e2e
uv run ruff check src tests scripts
uv run pyright src tests scripts
uv build
uv run twine check dist/*

The tests and CI definitions cover the protected admin boundary, session/CSRF/origin policy, draft safety, qualification/apply/rollback/connect operations, event and metrics bounds, redaction, packaged assets, CLI console startup, and browser/clean-wheel acceptance. A particular local or hosted release result is recorded in VERDICT.md; the v0.2.0 candidate-preparation record intentionally does not claim those gates until they are run for the exact final candidate.

Scope and explicit non-goals

Model Hub does not launch, stop, schedule, or allocate model servers or GPUs and does not require Kubernetes. It is a reliability and operator-control layer for explicit model pools, not a serving framework or hosted multi-user control plane.

It does not provide Kubernetes/CRDs, GitOps, semantic or cost routing, automatic cross-alias fallback, prompt/response inspection, token traces, external telemetry export, or a Hermes Desktop plugin. serve runs in the foreground; process supervision remains the responsibility of the host's service manager or container runtime. The public inference boundary remains Chat Completions only: no Responses, embeddings, image, or audio proxying.