Part of #9.
Problem
A multi-node distributed model can remain healthy while the single client-facing LLooM leader gateway is unavailable. Without a standby inference path, the gateway remains a single point of failure even after member control-plane degradation is handled correctly.
Naive active/active gateways are unsafe because independent runtime managers could issue conflicting admission, start, stop, or eviction decisions.
Proposed direction
- Support an inference-only standby gateway that can proxy already-serving model endpoints.
- Keep lifecycle ownership explicitly single-leader until LLooM has a durable distributed lease/lock design.
- Expose failover readiness and current lifecycle authority in status output.
- Define a client discovery or virtual endpoint contract that does not expose raw backend ports.
- Ensure the standby cannot mutate runtime state without an explicit authority transition.
Acceptance criteria
- Loss of the leader gateway does not interrupt requests to an already-serving distributed runtime when the standby is healthy.
- The standby rejects lifecycle and configuration writes while it lacks authority.
- No dueling admission, eviction, start, or stop operation is possible.
- Recovery of the leader reconciles cleanly without duplicate work.
- Failure-injection tests cover leader loss during idle and active inference.
Non-goals
- General distributed consensus in the first implementation.
- Public exposure of backend model servers.
Part of #9.
Problem
A multi-node distributed model can remain healthy while the single client-facing LLooM leader gateway is unavailable. Without a standby inference path, the gateway remains a single point of failure even after member control-plane degradation is handled correctly.
Naive active/active gateways are unsafe because independent runtime managers could issue conflicting admission, start, stop, or eviction decisions.
Proposed direction
Acceptance criteria
Non-goals