feat: Implement Circuit Breaker pattern for external services - #222
Open
Opulencechuks wants to merge 1 commit into
Open
feat: Implement Circuit Breaker pattern for external services#222Opulencechuks wants to merge 1 commit into
Opulencechuks wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #211
Description
This PR introduces a production-grade Circuit Breaker pattern to protect the application from cascading failures, resource exhaustion, and degraded performance when interacting with external service dependencies.
The Circuit Breaker evaluates request success rates and latencies over a sliding window (circular buffer) and manages state transitions between
CLOSED,OPEN, andHALF_OPEN.Key Changes
src/utils/circuitBreaker.tsusing zero external dependencies. Supports configurable failure rate thresholds, p95 latency thresholds, and configurable delays. Memory bounds are strictly managed via a circular buffer.FallbackStrategy. IncludedFailFastFallbackfor services that should trigger retries or dead-letter queueing upstream, andDefaultValueFallbackfor soft-degrading services.horizonClient.ts): Wraps network calls so the Soroban indexer doesn't exhaust connections when Horizon is down.stripeProvider.ts,stellarProvider.ts): Ensures timeouts fail fast to gracefully defer to thepledge.worker.tsidempotency flow without hanging requests.kycFraud.service.ts): Uses aDefaultValueFallbackreturningnull. This allows the assessment flow to gracefully omit the 3rd-party score and use local signals instead of blocking.notification.service.ts): Wraps the SMTP NodeMailer client. When open, the fallback throws natively, properly activating the existing internal retry loops and BullMQ's queue mechanics.circuitBreaker.test.tsthoroughly tests edge cases and timeout transitions.Testing
CircularBuffer.