Skip to content

feat(event-process): integrate IdempotencyService dedup into handle() (EVO-1211)#42

Open
nickoliveira23 wants to merge 1 commit into
developfrom
feat/EVO-1211
Open

feat(event-process): integrate IdempotencyService dedup into handle() (EVO-1211)#42
nickoliveira23 wants to merge 1 commit into
developfrom
feat/EVO-1211

Conversation

@nickoliveira23

Copy link
Copy Markdown

Summary

Implements story 3.5 (Webhook Event Pipeline): integrates the shared IdempotencyService (2.4) into EventProcessService.handle() to drop duplicate provider webhooks.

  • After signature validation, computes sha256(rawPayload) via IdempotencyService.computeHash and calls checkAndMark(hash): false (already seen within the 1h TTL) drops silently (ack, no throw) and increments the metric; true (first seen) proceeds.
  • Runs after signature validation, so forged payloads never reach Redis.
  • The hash covers only rawPayload (per the card) — folding in headers/metadata would mask legitimate duplicates that arrive with a different header.
  • A shared rawPayloadString() helper is reused by the signature path.

Security

  • Internal broker consumer (no auth surface). Duplicate detection is atomic across replicas (Redis Lua SET-NX from story 2.4). Duplicate drop is ack (never a throw), so a rejected payload is not redelivered forever.

Notes

  • checkAndMark-only is at-most-once: it marks the hash before downstream processing exists. There is nothing to protect yet (enrichment 3.6 / persist 3.7 are not wired into handle()). When story 3.7 adds the ClickHouse persist, the safety lock (acquireLock/releaseLock, already in IdempotencyService) should bridge the mark→persist window so a persist failure can retry instead of losing the event. Out of scope here per the 3.5 card.
  • The idempotency key is sha256(rawPayload) without platform (per the card's "no headers/metadata"); cross-platform byte-identical bodies are effectively impossible (each provider body carries unique ids).
  • No event-process.module.ts change: IdempotencyModule is @Global, so IdempotencyService injects directly.

Test plan

  • evo-flow: npm run typecheck — clean
  • evo-flow: npx jest src/runners/event-process/services/event-process.service.spec.ts — 9 examples, 0 failures
  • evo-flow: npx eslint <changed .ts> — clean

Covers ACs: second identical message dropped + event_duplicates_dropped increments (AC1); a bit-different payload yields a different hash and is not deduped (AC2); plus idempotency-runs-only-after-signature ordering.

Changed Files

  • src/runners/event-process/services/event-process.service.ts
  • src/runners/event-process/metrics/event-process-metrics.ts
  • src/runners/event-process/services/event-process.service.spec.ts

Linked Issue

  • EVO-1211

… (EVO-1211)

Implements story 3.5: after signature validation, the pipeline computes a
sha256 of rawPayload and calls IdempotencyService.checkAndMark — a duplicate
within the TTL is dropped (ack, no throw) and counted, while a first-seen
event proceeds. Runs after signature validation so forged payloads never
reach Redis; the hash covers only rawPayload so a different header can't mask
a legitimate duplicate.

- EventProcessService: inject IdempotencyService (@global), isDuplicate() gate,
  shared rawPayloadString() helper reused by the signature path
- EventProcessMetrics: evo_webhook_event_duplicates_dropped_total{platform}
- specs: AC1 (second identical dropped + metric), AC2 (bit-different payload
  is not a duplicate), and idempotency-runs-after-signature ordering

The safety lock (acquireLock/releaseLock) is wired by 3.7 (EVO-1213) when
persist exists; checkAndMark-only is the at-most-once gate for 3.5.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry @nickoliveira23, you have reached your weekly rate limit of 500000 diff characters.

Please try again later or upgrade to continue using Sourcery

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant