Skip to content

[Fix] Services stall during transient Redis outages - #1351

Draft
roomote-roomote[bot] wants to merge 3 commits into
developfrom
fix/redis-connectivity-0h01n7mfsqjou
Draft

[Fix] Services stall during transient Redis outages#1351
roomote-roomote[bot] wants to merge 3 commits into
developfrom
fix/redis-connectivity-0h01n7mfsqjou

Conversation

@roomote-roomote

@roomote-roomote roomote-roomote Bot commented Aug 15, 2026

Copy link
Copy Markdown
Contributor

​Opened on behalf of @mrubens. Follow up by mentioning @roomote-roomote, in the web UI, or in Slack.

What changed

  • Bound ordinary Redis commands to three retries while allowing clients to continue reconnecting with a capped delay.
  • Added a separately observed BullMQ-compatible client for blocking QueueEvents connections that require unbounded request retries.
  • Handle connection errors explicitly, rate-limit repeated outage diagnostics to one report every 30 seconds, and emit a recovery summary.
  • Cover retry configuration, BullMQ client isolation, reconnect bounds, diagnostic coalescing, and QueueEvents wiring in focused tests.

Why this change was made

Transient Redis DNS failures previously produced bursts of unhandled ioredis error events while commands could remain pending indefinitely. The initial retry bound also affected BullMQ blocking connections, which require maxRetriesPerRequest: null to initialize.

Impact

Ordinary Redis-dependent operations now fail within a bounded retry budget instead of hanging indefinitely, while BullMQ queue listeners retain their required blocking semantics. Both client types continue attempting recovery and emit actionable, credential-safe degradation and recovery summaries without one log line per reconnect attempt.

@roomote-community

roomote-community Bot commented Aug 15, 2026

Copy link
Copy Markdown
Contributor

No code issues found. See task

  • packages/redis/src/index.ts:125 Shared Redis retry settings make BullMQ QueueEvents fail to initialize.

Reviewed 1e97c82

Comment thread packages/redis/src/index.ts Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant