Repository navigation
fix(limit-count): refresh the delayed-sync quota after the local delta expired - #14015
Merged
Merged
Conversation
…a expired The periodic sync only flushed a key while its local delta was present in the shared dict. That entry is created with a TTL of two windows and is never renewed, so once it expired, a node that kept rejecting never flushed or refreshed its cached quota again. With a sliding window it kept judging on a stale weight of the previous window until the window ended, rejecting requests that the current Redis state would admit. Flush a missing delta as 0, which refreshes the cached quota.
…lta to sync A missing local delta is now flushed as 0 to refresh the cached quota. When Redis is unreachable, that flush must not fall back to charging the local limiter its minimum of 1, or a node that only rejects would add one to its fallback count on every sync.
There was a problem hiding this comment.
🟡 Changes recommended
A successful zero-delta refresh can recreate an empty entry that causes incorrect fallback accounting during a later Redis outage.
1 open finding
What changed in this PR
This PR refreshes cached limit-count quotas after delayed-sync delta entries expire, preventing stale rejections.
Changes:
- Flushes missing deltas as zero to refresh quotas.
- Skips fallback accounting when the delta is absent.
- Adds regression tests for quota refresh and fallback accounting.
| File | Description |
|---|---|
t/plugin/limit-count-redis-delayed-sync3.t |
Tests expired-delta refresh and Redis failure handling. |
apisix/plugins/limit-count/delayed-syncer.lua |
Refreshes missing deltas and guards fallback accounting. |
🧠 Review effort: Balanced
Give feedback about Copilot approvals in this survey to enter a drawing for a $150 gift card.
…flushed A zero-delta quota refresh recreated the expired local delta entry as 0 through incr(-0). If Redis then failed, the sync took that entry for a pending delta and charged the fallback limiter its minimum of 1.
shreemaan-abhishek
approved these changes
Oct 8, 2026
AlinsRan
approved these changes
Oct 8, 2026
membphis
approved these changes
Oct 8, 2026
4 of 5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

With delayed sync, a node that only rejects can stop refreshing its cached quota, and then it keeps rejecting on stale numbers until the quota itself expires.
The periodic sync flushes a key only while its local delta is present in the shared dict (
sync_key,if delta then). That entry is created byincr(..., 0, 2 * window), andinit_ttlonly applies on creation, so the entry expires two windows after it was first written, whatever happens in between. Admitted requests recreate it; rejected ones don't. So once it expires on a node that is rejecting everything, the timer still runs for the key but skips the flush, and the cached quota is never refreshed again. The next refresh only happens when the cached reset runs out.With a sliding window this matters a lot. The cached quota carries the previous window's weight as of the last sync, so the node keeps applying a weight of ~0.9 for the rest of the window instead of letting it decay. In a two-node test at 2× the limit, one 10s window admitted 9 requests where an exact sliding window would have admitted 100, because both nodes stopped syncing about 1s into the window. A fixed window isn't affected in practice: within a window the remote count only grows, so a stale cached quota can't wrongly reject, and the window end coincides with the cached reset, which forces a refresh.
The fix flushes a missing delta as 0 instead of skipping it. That refreshes the cached quota exactly like the existing flush of a zero delta, which already happens whenever the entry is present but empty. A failed shared dict read still skips the flush as before. When there was no local delta and Redis is unreachable, the fallback limiter is skipped: there is nothing to account for locally, and charging it the existing minimum of 1 would let a node that only rejects inflate its fallback count on every sync. The path where the entry exists, including a value of 0, is unchanged. For the same reason,
sync_to_shmno longer creates an empty entry when nothing was flushed, so a zero-delta refresh doesn't later look like a pending delta.t/plugin/limit-count-redis-delayed-sync3.treproduces it without waiting two windows. It exhausts the quota, drops the local delta entry as its expiry would, frees the quota in Redis, and checks that a rejected request schedules a sync that lets the next request through. It returns200 200 503 503 503without the fix and200 200 503 503 200with it. A second case, a fixed window against an unreachable Redis, drops the entry and checks that the failed sync leaves the fallback counter alone. It goes from 1 to 2 without the fallback guard and stays at 1 with it. A third case covers a zero-delta refresh followed by a Redis error (a non-integer counter). Without the change insync_to_shm, the fallback count goes from 0 to 1.No documentation change: the documented delayed sync behavior doesn't change.