What
Follow-up from review on #861 (cache-through credential caching, apis::token_cache::TokenCache).
TokenCache::get_or_refresh holds the cache's exclusive lock for the full duration of the fetch call when the cache is stale. Under a genuine network partition to the IdP/metadata server (connection hangs rather than fails fast), every concurrent caller queued behind that lock pays the full fetch timeout serially: request 1 blocks on the write lock, attempts fetch, times out at ~30s; request 2 (queued behind it) then gets the lock, attempts its own fetch, times out at ~30s; and so on. A failed fetch isn't cached, so there's no server-side backoff to break the cycle — each queued request just repeats the same failing attempt.
Under sustained partition, this produces serialized request-latency pile-up proportional to the fetch timeout times the number of concurrent callers, rather than all callers failing fast together.
Why it wasn't addressed in #861
This was a deliberate scope decision — the team chose the simpler cache-through pattern specifically to move faster than the fuller refresh-ahead design (see praxis#555), and this failure mode is a reasonable thing to defer rather than design around up front.
Possible directions (not decided)
- A single shared "in-flight fetch" future that all queued callers await together, rather than each queued caller repeating the fetch attempt
- A shorter fetch timeout specifically for the write-lock-held path, with a distinct (longer) timeout for whichever caller "owns" the retry
- Fail fast for queued callers once one attempt has failed within a short window, rather than each retrying independently
Related
What
Follow-up from review on #861 (cache-through credential caching,
apis::token_cache::TokenCache).TokenCache::get_or_refreshholds the cache's exclusive lock for the full duration of thefetchcall when the cache is stale. Under a genuine network partition to the IdP/metadata server (connection hangs rather than fails fast), every concurrent caller queued behind that lock pays the full fetch timeout serially: request 1 blocks on the write lock, attempts fetch, times out at ~30s; request 2 (queued behind it) then gets the lock, attempts its own fetch, times out at ~30s; and so on. A failed fetch isn't cached, so there's no server-side backoff to break the cycle — each queued request just repeats the same failing attempt.Under sustained partition, this produces serialized request-latency pile-up proportional to the fetch timeout times the number of concurrent callers, rather than all callers failing fast together.
Why it wasn't addressed in #861
This was a deliberate scope decision — the team chose the simpler cache-through pattern specifically to move faster than the fuller refresh-ahead design (see praxis#555), and this failure mode is a reasonable thing to defer rather than design around up front.
Possible directions (not decided)
Related