Rehmy implementation gf/4 - #460
Merged
Wilfred007 merged 3 commits intoAug 16, 2026
Merged
Conversation
added 3 commits
August 16, 2026 07:50
Adds locked_by and locked_at columns to the schedules table to support claim-based concurrency control. Includes an index for efficient lookup of claimable rows (active + due + unlocked).
Replace the plain SELECT in processDueSchedules() with an atomic UPDATE ... FROM (SELECT ... FOR UPDATE SKIP LOCKED) pattern that marks each claimed row with the pod's hostname+pid. This prevents two pods from executing the same schedule simultaneously. Also adds releaseClaim() to clear locks after execution and releaseStaleClaims() to reclaim rows from crashed pods (5min timeout).
Update existing processDueSchedules tests to match the new claim query (BEGIN/UPDATE FOR UPDATE SKIP LOCKED/COMMIT pattern) and add lock-clear assertions to recordExecution tests.
Closed
3 tasks
Contributor
|
Thank you @rhemy-arc for your contributions |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds row-level locking (FOR UPDATE SKIP LOCKED) to the payroll schedule executor to prevent concurrent pods from executing the same schedule simultaneously. Without this fix, every backend pod running the '* * * * *' cron job independently queries and processes the same due schedules, causing duplicate Stellar payments to employees.
Changes
Testing
Tradeoffs
Architecture
Pod A (cron tick) Pod B (cron tick)
│ │
├─ releaseStaleClaims() ├─ releaseStaleClaims()
├─ BEGIN ├─ BEGIN
├─ UPDATE ... FOR UPDATE SKIP LOCKED ├─ UPDATE ... FOR UPDATE SKIP LOCKED
│ → claims schedule 42 │ → skips schedule 42 (locked)
├─ COMMIT ├─ COMMIT
├─ executeSchedule(42) ├─ (no schedules claimed)
├─ recordExecution(42) │
│ → clears locked_by │
└─ releaseClaim(42) └─ done
Out of scope
Closes #448