Skip to content

feat(queue): add swap job processor with idempotent AMM execution - #22

Merged
ameeribro4-sudo merged 3 commits into
OpenPeerX:mainfrom
P3az3:fix/issue-15-swap-job-processor
Aug 20, 2026
Merged

feat(queue): add swap job processor with idempotent AMM execution#22
ameeribro4-sudo merged 3 commits into
OpenPeerX:mainfrom
P3az3:fix/issue-15-swap-job-processor

Conversation

@P3az3

@P3az3 P3az3 commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

Summary

Closes #15

Adds SwapJobProcessor, the missing consumer for the registered-but-unprocessed swaps Bull queue. It executes single, multi_leg, and batch swap jobs against the existing AMM swap path (LiquidityPoolService.swap) — the only real swap-execution service in the codebase (StellarService has no swap method), satisfying the issue's requirement to reuse an existing service rather than build a new engine. The single most important design decision is the durable Redis idempotency guard (swap:executed:{swapId}, SET NX EX via the existing RedisPoolService): a Bull retry of an executed swap short-circuits, so a funds-moving job is never executed twice.

Why

QueueModule registers the SWAPS queue and QueueService exposes addSingleSwapJob / addMultiLegSwapJob / addBatchSwapJob, but no processor ever consumed the queue — jobs landed in waiting and wedged silently, and with removeOnComplete: false / removeOnFail: false they accumulated forever. A caller got a jobId and believed a swap was in flight when nothing would ever execute it. Because a swap moves funds and Bull retries with backoff, the naive fix (call poolService.swap(...) in a @Process handler) would double-execute on retry, so an idempotency key was the correct primitive: claimed atomically before execution, released on failure (so a genuinely failed swap is re-attempted), and retained on success (so a retry or duplicate enqueue of the same swapId is a no-op).

What was built

src/queue/processors/swap.processor.ts (new; tests in src/queue/processors/swap.processor.spec.ts):

File What it contains
swap.processor.ts @Processor(QueueName.SWAPS) with @Process('single'), @Process('multi_leg'), @Process('batch'). Executes via LiquidityPoolService.swap(poolId, { userId, tokenIn, amountIn, minAmountOut }). single: guard key swap:executed:{swapId}. multi_leg: per-leg keys ({swapId}:leg:{index}) so a retry resumes at the first unexecuted leg and never re-executes completed legs. batch: best-effort — every sub-swap is attempted independently with its own per-swapId guard; failures are collected and the job fails after all are attempted, so a retry only re-attempts the failed sub-swaps. Missing poolId throws NonRetryableSwapError (no point retrying an unresolvable job). @OnQueueFailed moves permanently-failed jobs to the DLQ with MAX_RETRIES_EXCEEDED or NON_RETRYABLE_ERROR.
swap.processor.spec.ts (new) 11 tests against a faithful in-memory Redis fake (honoring NX/EX/TTL) with mocked AMM + DLQ: successful single swap with correct SwapDto; retry does not execute twice; failure releases the guard and re-executes; missing poolId is non-retryable; DLQ routing for both reasons; multi-leg in-order execution; multi-leg resumes at the failed leg without re-executing completed legs; batch executes all sub-swaps; batch fails the job when any sub-swap fails but still attempts all; batch retry skips already-executed sub-swaps.

Integration changes outside src/queue/processors/

  • src/queue/queue.service.tsSingleSwapJobData gains optional poolId?: number and minAmountOut?: number; MultiLegSwapJobData.legs entries gain the same. Optional fields are backward-compatible (no existing caller constructs these payloads — verified: the only references to the enqueue methods are inside queue.service.ts itself), and the enqueue methods are unchanged.
  • src/queue/queue.module.ts — registers SwapJobProcessor in providers and imports ExchangeModule (exports LiquidityPoolService). No circular dependency: ExchangeModule imports only TypeOrmModule; madge confirms the same 3 pre-existing cycles, none involving queue or exchange.
  • src/queue/dead-letter-queue.service.ts — injects the SWAPS queue and adds it to getQueueByName, so a DLQ'd swap job can be recovered and retried (the issue explicitly allows DLQ wiring where needed to make swap failures recoverable). Also typed the getQueueByName switch predicate as QueueName, fixing the file's pre-existing no-unsafe-enum-comparison lint findings on that switch.
  • README.md — documents the swap execution target (AMM), the idempotency guard, multi-leg resume semantics, and batch best-effort semantics under the Redis setup section.

Acceptance criteria coverage

  • A single swap job enqueued via addSingleSwapJob is picked up by a processor and executed against the chosen swap service. (swap.processor.spec.ts — "executes a successful single swap against the AMM service"; LiquidityPoolService.swap is the chosen service, the only real swap executor in the codebase)
  • multi_leg and batch jobs are processed with defined partial-failure behavior. (swap.processor.spec.ts — multi-leg "resumes at the failed leg on retry without re-executing completed legs" and batch "attempts all swaps and fails the job when any sub-swap fails" + "skips already-executed sub-swaps on retry"; semantics documented in the processor docstring and README: multi-leg resumes at the first unexecuted leg with no rollback of completed legs, batch is best-effort per sub-swap)
  • A retried swap job does not execute the swap twice. (swap.processor.spec.ts — "does not execute the same swap twice on retry (idempotency guard)"; durable Redis SET NX EX guard, reused RedisPoolService)
  • A processor spec covers: successful single swap, idempotent retry, and failure that surfaces as a failed job. (swap.processor.spec.ts — all three, plus the failure-DLQ routing tests)
  • The enqueue methods still pass their existing behavior in queue.service usage. (queue.service.ts — enqueue methods byte-for-byte unchanged; job-data additions are optional fields)
  • The swap execution target and batch semantics are documented in the queue README/Swagger. (README — AMM execution target, swap:executed:{swapId} guard, multi-leg resume, batch best-effort; processor docstring)

Test plan

  • npm run build — succeeds
  • npx jest src/queue/processors/swap.processor.spec.ts — 11/11 passing (11 new tests)
  • npm run test — 461/513 passing; failing suites identical to base branch (pre-existing ts-jest/uuid resolution failures; zero new failures)
  • npx eslint — 0 issues on the two new files; on the three edited source files, no new issues versus base (the one new enum-comparison finding my added case QueueName.SWAPS would have introduced was eliminated by typing the switch predicate, which also fixed the pre-existing findings on that switch)
  • npx madge --circular --extensions ts src/ — same 3 pre-existing cycles, none involving queue or exchange
  • Prettier — new files formatted; the README section added is prettier-clean (file carries 120 pre-existing deviations, untouched)

Env vars / Notes

No new env vars. The idempotency guard rides the existing Redis connection (REDIS_HOST/REDIS_PORT) already used by Bull and the cache layer; Redis must be reachable for the guard to work (the queue already requires Redis). Guard TTL is 24h — shorter than the 7-day removeOnFail retention, so markers age out rather than accumulate. Delivery semantics per job type are documented in the processor docstring and README: single is exactly-once under retry, multi-leg resumes at the first unexecuted leg (completed legs are not rolled back — no compensation mechanism exists in the AMM path), batch is best-effort per sub-swap. Swap jobs now need poolId (and optionally minAmountOut) in their payload; jobs without poolId fail fast as non-retryable and land in the DLQ with a clear reason.

Consume the registered-but-unprocessed swaps Bull queue with a
SwapJobProcessor that executes single, multi_leg, and batch jobs
against the AMM swap path. A durable Redis idempotency guard
(swap:executed:{swapId}, SET NX EX) prevents Bull retries from
executing a completed swap twice; multi-leg jobs resume at the first
unexecuted leg and batch jobs are best-effort per sub-swap. Permanent
failures land in the dead-letter queue, which now tracks the swaps
queue for recovery.
@P3az3
P3az3 force-pushed the fix/issue-15-swap-job-processor branch from f07af0e to e9f751f Compare August 20, 2026 15:52

@ameeribro4-sudo ameeribro4-sudo left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@P3az3 resolve conflicts please

@P3az3

P3az3 commented Aug 20, 2026

Copy link
Copy Markdown
Contributor Author

@ameeribro4-sudo resolved.

@ameeribro4-sudo ameeribro4-sudo left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ameeribro4-sudo
ameeribro4-sudo merged commit b0338ba into OpenPeerX:main Aug 20, 2026
4 checks passed
DeRossa1 added a commit to DeRossa1/Peerx-Backend that referenced this pull request Aug 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Swaps queue has no processor: enqueued swap jobs wedge silently and never execute

2 participants