fix(ci): add a scheduled sweep so merge-queue PRs cannot strand - #151
Merged
Merged
Conversation
The event-driven path alone is not sufficient on a merge-queue repo. When a check_suite completes GitHub has often not recomputed mergeStateStatus, so the PR still reads BLOCKED; the workflow logged 'waiting for a later check_suite event' and exited. When that was the last such event nothing ever retried and the PR sat open while reading CLEAN. Pin PR #148 stranded for four days, which silently froze the deploy chain: build-and-deploy only classifies mode=webhook for a pin commit, so the redeploy never ran and the host kept the old image. Adds two defences: retry in-run while BLOCKED with checks still PENDING, and a 15-minute scheduled sweep that enqueues any open mergeable unqueued PR.
Rebases muninn's automerge.yml onto the homelab-stacks canonical version and
adds the sweep on top.
Two pre-existing standard violations are fixed by the rebase:
- github-actions-monorepo-standard.md requires automerge.yml to mint the
deploy-bot App token and pass it to the step calling enqueuePullRequest.
Muninn ran the mutation under the default GITHUB_TOKEN.
- The UNSTABLE self-check handling from homelab-stacks #406 was missing, so
a fast-CI PR could strand on this job's own in-progress automerge check.
The new part is the scheduled sweep. The event path alone cannot guarantee
convergence on a merge-queue repo: when a check_suite completes GitHub has
often not recomputed mergeStateStatus, the PR reads BLOCKED, and if that was
the last event nothing retried. Pin PR #148 stranded four days that way and
silently froze the deploy chain.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
automerge.ymlfires oncheck_suite: completed, readsmergeStateStatus, and only enqueues when the PR is already mergeable. At that moment GitHub has usually not recomputedmergeStateStatus, so the PR readsBLOCKEDand the job logswaiting for a later check_suite eventand exits.If that was the last
check_suiteevent, nothing ever retries. The PR then goesCLEANand sits open indefinitely.Pin PR #148 stranded exactly this way for four days. Because
build-and-deploy.ymlonly classifiesmode=webhookfor achore: pin muninn ->head commit, the Portainer redeploy never fired and the host kept serving the previous image, with no alert anywhere. Reproduced live on #150: checks wentCLEAN, no further automerge run fired, queue stayed empty until it was enqueued by hand.Muninn is the only Emkraan repo with a merge-queue ruleset, so it is the only repo where this explicit enqueue call is load-bearing.
Fix
BLOCKEDand its check rollup is stillPENDING. Terminal states (failing checks, conflicts) break out immediately rather than burning twelve attempts.Sweep runs a single pass per PR (the next sweep is 15 minutes away); event runs wait, since they are racing GitHub's own recomputation.
Concurrency group now falls back to
run_idso sweeps do not collapse into one group.