feat: add dead letter queue for failed webhook deliveries

[Toyosi5566]

Overview

This PR adds a dead letter queue (DLQ) for webhook deliveries that exhaust their retry attempts. Instead of losing the event payload after a delivery is permanently failed, the full delivery context — including payload, error history, and attempt details — is persisted in a Redis sorted set. New DLQ API endpoints allow operators to list stuck deliveries and retry them, while a TTL automatically expires old entries.

Related Issue

Closes #

Changes

🗂️ DLQ Storage & Lifecycle

• [ADD] src/services/webhookDlq.js

  • Stores permanently failed deliveries in a Redis sorted set with timestamp scores.
  • Persists full payload, error history, and attempt details in each DLQ entry.
  • Applies configurable TTL for automatic cleanup.

• [MODIFY] src/config.js

  • Adds DLQ Redis key prefix, TTL, and retry-related configuration.

• [MODIFY] src/services/cache.js

  • Adds Redis sorted set helpers (zadd, zrange, zrem, expire) used by the DLQ.

🔌 Webhook DLQ API

• [MODIFY] src/routes/webhooks.js

  • Adds GET /webhooks/dlq to list DLQ entries ordered by when they failed.
  • Adds POST /webhooks/dlq/:id/retry to re-dispatch an existing DLQ entry.

• [MODIFY] src/services/webhook.js

  • Wires DLQ listing and retry operations into the webhook service layer.

🔁 Delivery & Retry Integration

• [MODIFY] src/services/webhookDispatcher.js

  • Moves deliveries to the DLQ after the final retry attempt is exhausted.
  • Includes failure timestamps and complete error history in the DLQ entry.

• [MODIFY] src/repositories/deliveryRepository.js

  • Preserves payload/error data when a delivery is marked failed so it can be stored in the DLQ.

• [MODIFY] src/jobs/webhookRetryWorker.js

  • Skips deliveries already moved to the DLQ and triggers DLQ insertion when retries are exhausted.

Verification Results

npm test -- src/services/__tests__/webhookDlq.test.js
✅ 9/9 passed

Live acceptance check:
✅ Failed delivery stored in DLQ after max attempts
✅ Full payload, error history, and attempt details preserved
✅ TTL cleanup verified
✅ GET /webhooks/dlq returns DLQ entries
✅ POST /webhooks/dlq/:id/retry re-dispatches a delivery

Acceptance Criteria	Status
Permanently failed deliveries are stored in a DLQ	✅ Redis sorted set with full context
DLQ entries include payload, error history, and attempts	✅ Entry stores all retry metadata
API endpoint lists DLQ entries	✅ GET /webhooks/dlq returns entries
API endpoint retries a DLQ entry	✅ POST /webhooks/dlq/:id/retry re-dispatches
DLQ entries have TTL for automatic cleanup	✅ Configurable TTL applied on insert

Closes #257

[drips-wave]

@Toyosi5566 Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits