Current Status
BFF currently uses InMemoryRateLimiter in bff/src/rate-limit.ts.
There are no obvious issues during single-machine development, but after deploying two or more BFF instances, each instance calculates its own quota. For example, if the limit is 120 requests per minute, two machines might actually allow 240 requests.
docker-compose.yml already has Redis, which can be used as a shared counting base.
Desired Functionality
Replace the current single-machine rate limiting with a Redis-backed rate limiter, supporting the following dimensions:
userIdtenantId- IP
agentIdtoolNamemodelName
Each dimension should be independently configurable:
- window
- request limit
- key prefix
- Enable/Disable
When the limit is exceeded, BFF should return:
HTTP/1.1 429 Too Many Requests
Retry-After: 18
Frontend should display an understandable wait time, not just a generic error.
Example Scenario
Settings:
user limit: 120 requests/minute
model limit: 30 requests/minute
User A has reached their model quota, but user B is using a different model.
Current Status
BFF currently uses
InMemoryRateLimiterinbff/src/rate-limit.ts.There are no obvious issues during single-machine development, but after deploying two or more BFF instances, each instance calculates its own quota. For example, if the limit is 120 requests per minute, two machines might actually allow 240 requests.
docker-compose.ymlalready has Redis, which can be used as a shared counting base.Desired Functionality
Replace the current single-machine rate limiting with a Redis-backed rate limiter, supporting the following dimensions:
userIdtenantIdagentIdtoolNamemodelNameEach dimension should be independently configurable:
When the limit is exceeded, BFF should return:
Example Scenario
Settings:
user limit: 120 requests/minute
model limit: 30 requests/minute
User A has reached their model quota, but user B is using a different model.