What
Backend priority groups that cascade failover through ordered tiers. Backends at the same priority level are load-balanced; failover cascades to the next priority group when all backends in a group are unhealthy or exhausted.
Why
Multi-provider AI deployments need deterministic failover ordering. Envoy AI Gateway's routing strategy: Primary (Priority 1, AWS Bedrock, weight 100) -> Secondary (Priority 2, GCP Vertex AI us-east5 weight 80, us-central1 weight 60). Without priority groups, all backends are treated equally regardless of cost, latency, or capacity tier.
Goals
- Per-backend priority level in cluster configuration
- Load balancing within a priority group (existing strategies: round-robin, P2C, etc.)
- Cascade to next group when current group has no healthy backends
- Integration with health checks and circuit breakers for group-level health
What
Backend priority groups that cascade failover through ordered tiers. Backends at the same priority level are load-balanced; failover cascades to the next priority group when all backends in a group are unhealthy or exhausted.
Why
Multi-provider AI deployments need deterministic failover ordering. Envoy AI Gateway's routing strategy: Primary (Priority 1, AWS Bedrock, weight 100) -> Secondary (Priority 2, GCP Vertex AI us-east5 weight 80, us-central1 weight 60). Without priority groups, all backends are treated equally regardless of cost, latency, or capacity tier.
Goals