When intelligent_route is configured with two static candidates for the same model — one fresh: true (inference cluster) and one fresh: false (echo fallback) — the filter consistently selects the stale echo candidate instead of the fresh inference candidate.
Config
- filter: intelligent_route
local_site: site-a
model_header: X-Model
candidates:
- kind: inference_model
name: test-model
site: site-a
cluster: inference
fresh: true
- kind: inference_model
name: test-model
site: site-a
cluster: echo
fresh: falseObserved
Access logs show cluster="echo" upstream="echo-backend" for every POST /v1/chat/completions request with model: test-model. The model_to_header filter runs before and promotes the model name to X-Model.
Expected
The fresh inference candidate should score higher and be selected.
Context
Found while building the OTel benchmark demo (praxis-proxy/experimental#13). The workaround is using router instead. The bug may be a config schema mismatch with the current filter version or a scoring logic issue with static (non-overlay) candidates.
When
intelligent_routeis configured with two static candidates for the same model — onefresh: true(inference cluster) and onefresh: false(echo fallback) — the filter consistently selects the stale echo candidate instead of the fresh inference candidate.Config
Observed
Access logs show
cluster="echo" upstream="echo-backend"for everyPOST /v1/chat/completionsrequest withmodel: test-model. Themodel_to_headerfilter runs before and promotes the model name toX-Model.Expected
The fresh inference candidate should score higher and be selected.
Context
Found while building the OTel benchmark demo (praxis-proxy/experimental#13). The workaround is using
routerinstead. The bug may be a config schema mismatch with the current filter version or a scoring logic issue with static (non-overlay) candidates.