Skip to content

feat: distributed multi-node webhook delivery engine with HMAC signing and DLQ recovery - #450

Merged
jotel-dev merged 2 commits into
Nullifier-Systems:mainfrom
wheval:feat/webhook-delivery-dlq-445
Aug 30, 2026
Merged

feat: distributed multi-node webhook delivery engine with HMAC signing and DLQ recovery#450
jotel-dev merged 2 commits into
Nullifier-Systems:mainfrom
wheval:feat/webhook-delivery-dlq-445

Conversation

@wheval

@wheval wheval commented Aug 30, 2026

Copy link
Copy Markdown
Contributor

Summary

Closes #445. Adds a developer-facing signed webhook delivery system, distinct from the existing single-destination ops alert in lib/webhook.ts (Slack/Discord).

  • Schema (migration 030_add_distributed_webhook_pipeline.sql): webhook_endpoints (one row per developer-registered target URL + HMAC secret) and webhook_delivery_logs (one row per delivery attempt, with status/attempt tracking).
  • Store (webhookDeliveryStore.ts): Pool-or-in-memory, mirroring SwapDisputeStore's shape so tests run with no database. DLQ replay uses SELECT ... FOR UPDATE so a delivery is only ever re-queued once even if an operator and a retry sweep race.
  • Async delivery: every event is enqueued onto velo:webhook-delivery-queue (Redis Stream) rather than sent inline — a slow or dead client endpoint can never block an API response. webhookDeliveryWorker.ts drains it, signs the exact payload bytes with HMAC-SHA256 (x-velo-signature), and retries with exponential backoff + jitter up to 5 attempts before dead-lettering (mirrors sessionRotationWorker's retry shape).
  • Routes (webhooks.ts): POST /webhooks/endpoints, GET /webhooks/endpoints, GET /webhooks/endpoints/:id/deliveries, POST /webhooks/dlq/replay. HTTPS is enforced for target_url when NODE_ENV=production.
  • Wiring: cash.ts's refund flow now also calls notifyDeveloperWebhooks for both trade participants (buyer + seller), alongside the existing sendRefundAlert ops notification — same trigger points, no behavior change to the ops alert itself.
  • Frontend: WebhookSettings.tsx — register an endpoint, view its secret once, monitor recent deliveries, and manually replay anything dead-lettered.

Test plan

  • npx tsc -b / -p apps/api/tsconfig.json --noEmit clean (repo-wide pre-existing errors in batch-auction-engine.test.ts / prekey-vault.ts are unrelated and untouched by this PR)
  • New unit tests: HMAC signature correctness (webhookDeliveryStore.test.ts), store behavior including DLQ replay's single-claim guarantee
  • New worker tests: first-attempt delivery, 5-attempt retry-then-dead-letter, signature header sent correctly
  • New route tests: endpoint registration, URL validation, per-user listing, DLQ replay (including that a second replay of the same delivery is refused, not re-enqueued)
  • Full existing apps/api suite: 562 passed, 0 regressions (had to update cash.test.ts's webhook.js mock to include the new notifyDeveloperWebhooks export)
  • Full existing mobile/frontend suite: 77 passed, 0 regressions

…g and DLQ recovery

Adds a developer-facing webhook system distinct from the existing
single-destination ops alert: developers register a target URL and
receive signed trade-status events instead of nothing, or worse, an
unsigned callback a third party could spoof.

- webhook_endpoints / webhook_delivery_logs (migration 030), with a
  Pool-or-in-memory store mirroring SwapDisputeStore, including
  SELECT ... FOR UPDATE DLQ replay so a delivery is only ever
  re-enqueued once.
- Every delivery is enqueued onto a Redis Stream
  (velo:webhook-delivery-queue) rather than sent inline, so a slow or
  dead client endpoint can never block an API response.
- webhookDeliveryWorker drains the stream, signs with HMAC-SHA256
  (x-velo-signature), and retries with exponential backoff up to 5
  attempts before dead-lettering.
- POST /webhooks/endpoints, GET /webhooks/endpoints,
  GET /webhooks/endpoints/:id/deliveries, POST /webhooks/dlq/replay.
- cash.ts's refund flow now also notifies developer-registered
  webhooks for both trade participants, alongside the existing ops
  alert.
- WebhookSettings.tsx: register an endpoint, view its secret, monitor
  recent deliveries, and manually replay anything dead-lettered.

Closes Nullifier-Systems#445
@vercel

vercel Bot commented Aug 30, 2026

Copy link
Copy Markdown

@wheval is attempting to deploy a commit to the jotelfootball-tech's projects Team on Vercel.

A member of the Team first needs to authorize it.

@vercel

vercel Bot commented Aug 30, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
velo Ready Ready Preview Aug 30, 2026 8:21pm
velo-frontend Ready Ready Preview Aug 30, 2026 8:21pm

@jotel-dev

Copy link
Copy Markdown
Contributor

@wheval The node CI is failing kindly fix that

The repo's localization:check CI gate flags any raw user-facing JSX
text/attribute as an error. Adds a webhookSettings namespace to both
locale catalogs and switches the page to useTranslation()/t().
@jotel-dev

Copy link
Copy Markdown
Contributor

All checks pass, no conflicts. Reviewed and looks good — merging now. Thanks for the contribution!
@wheval

@jotel-dev
jotel-dev merged commit 4837603 into Nullifier-Systems:main Aug 30, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEAT] Distributed Multi-Node Webhook Event Delivery Engine & DLQ Recovery

2 participants