fix(router): session affinity runs in every shipped routing mode - #24
Conversation
sticky_session_id / session_id was dead in every configuration anyone runs
(todos 7dd67da6-9ede-483a-ac3f-acd21cecd7f5): 'cheapest', 'fallback' and
'explicit' returned from sortCandidates before the sticky component was ever
computed, and the scored modes consulted it only after EXACT float score
equality, which no realistic candidate pair produces — minmax normalization
maps any two distinct metric values to 0 and 1, so even near-identical
candidates score far apart. The feature was findable by reading the code and
unreachable at runtime.
The fix is a stateless affinity pass that runs in EVERY mode after the mode's
own ordering: groups of candidates the mode does not distinguish — equal
configured price in cheapest, candidates a provider_order hint leaves unranked
(shared Infinity rank) in fallback/explicit, weighted scores within
SESSION_AFFINITY_SCORE_EPSILON (0.01 on the 0..1 scale) in scored modes — are
ordered by the existing deterministic per-session hash. A candidate the mode
ranks strictly better is never overridden; without a session id the order is
returned untouched, byte-identical to previous behavior. Every decision now
discloses the path under decision.session_affinity
({ session_id_present, applied }), which is what makes the code path
observable and testable per mode. Affinity across requests holds by
construction (same session, same hash, same candidate set) without a
session->provider state store; a persisted store remains out of scope and is
not claimed.
Regression tests written first and red on the pre-fix tree: 17/17 fail at
21fc5db ("0 pass / 17 fail"), 17/17 pass post-fix, covering all seven shipped
modes (explicit, fallback, cheapest, lowest-latency, highest-throughput,
balanced, smart), same-session stability, session-dependence of tie winners,
the pooled-identical-model shape with a worse candidate excluded, clear
winners never sacrificed, ranked provider_order never shuffled, and the
no-session control. Full suite: 259 pass / 0 fail; typecheck and build clean.
Agent: agent-chief-planning
|
[REVIEW] NO_GO — #24 @ d015cac — lens: correctness+security+gates, reviewer unresolved-account001 (1 of 1) Reviewed the full diff against fetched base Commands and gates:
Blocking findings:
Required focused remedy: preserve original order for every candidate whose provider appears in Security findings: none. The session id changes ordering only inside the already policy-filtered eligible set; no new credential, authorization, tenant, or data-exposure path was found. Non-blocking follow-ups: none. |
Keep models within a ranked provider in their configured order while session affinity continues to order only providers the request leaves unranked. Agent: unresolved-account001
|
[REVIEW] GO — #24 @ 87eb350 — lens: correctness+security+gates, reviewer unresolved-account001 (1 of 1) Focused re-review of the one named P1 from the prior Fix verified:
Blocking P0/P1 findings: none. The prior P1 is fixed and its affected lanes pass. Non-blocking follow-ups: none. |
Release @hasna/gateway 0.1.7 after the session-affinity correction landed on main in #24. Validation: - bun run typecheck: exit 0 - bun run test: exit 0, 259 pass, 1 skip, 0 fail Agent: unresolved-account001
Defect
Session affinity (
sticky_session_id/session_id) never ran in any shipped routing mode — todos7dd67da6-9ede-483a-ac3f-acd21cecd7f5, measured at main21fc5dbduring PLA23-00044. Mechanism, confirmed by drivingresolveRoutedirectly in all seven modes with a session id varied across a two-candidate config:cheapest,fallback,explicit:sortCandidatesreturned beforestickyTieBreakerwas ever computed — the code path was literally unreachable.lowest-latency,highest-throughput,balanced,smart): sticky was consulted only after EXACT float score equality. Minmax normalization maps any two distinct metric values to 0 and 1, so even near-identical candidates (500ms vs 505ms latency) score far apart — the tie-break fired only for byte-identical candidate stats, never in a realistic config.Probe evidence (pre-fix): with any differing stats, 16 distinct session ids produced one identical winner in all seven modes; the session id was routing-inert.
Fix
A stateless affinity pass (
applySessionAffinity) runs in EVERY mode after the mode's own ordering. Groups of candidates the mode does not distinguish — equal configured price incheapest, candidates aprovider_orderhint leaves unranked (sharedInfinityrank) infallback/explicit, weighted scores withinSESSION_AFFINITY_SCORE_EPSILON(0.01 on the 0..1 scale) in scored modes — are ordered by the existing deterministic per-session FNV-1a hash.Boundaries, stated so nobody over-reads it:
provider_ordercandidates and configured fallback chains are never shuffled.Every decision now discloses the path:
decision.session_affinity = { session_id_present, applied }— additive optional field, which is what makes the path observable and red/green testable per mode. When affinity ordered a group,decision.reasongains"; session affinity ordered materially-equal candidates".Tests (written first, red on pre-fix tree)
tests/router-affinity.test.ts, 17 tests: at21fc5db+ test file only —0 pass / 17 fail. Post-fix —17 pass / 0 fail / 186 expect() calls. Coverage: all seven shipped modes (affinity path executes AND the session id decides a tie, deterministically, with a probe-derived disagreeing session pair), same-session stability across repeats, the no-session control (unchanged selection, disclosedapplied: false), pooled-identical-model shape with a clearly worse third candidate never joining the tie group, clear winners never sacrificed, rankedprovider_ordernever shuffled.Full suite
bun test:259 pass / 0 fail.bun run typecheckandbun run buildclean.Closes todos row
7dd67da6-9ede-483a-ac3f-acd21cecd7f5. Part of platform wave-2 packagecapacity-cli-and-affinity.Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is disabled.