Skip to content

fix(ci): stop amd64 builds clobbering the arm64 publish cache - #838

Open
manavgup wants to merge 1 commit into
mainfrom
fix/publish-arm64-cache
Open

fix(ci): stop amd64 builds clobbering the arm64 publish cache#838
manavgup wants to merge 1 commit into
mainfrom
fix/publish-arm64-cache

Conversation

@manavgup

Copy link
Copy Markdown
Owner

Problem

The multi-arch GHCR publish job timed out at exactly 30 minutes on two of the last three main builds (#835's and #831's merge builds), cancelling the Docker run and silently skipping the deploy both times. Each needed a manual re-run to ship.

Root cause: the amd64-only build job (every PR, every push) writes to the same weekly GHA cache scope the publish job uses, evicting the arm64 layers publish caches there. Result: every publish rebuilds arm64 from scratch under QEMU — ~30 minutes, sitting exactly on the timeout.

Solution

  • Dedicated prod-multiarch-<week> cache scope for the publish job. It still reads the shared amd64 scope (layer reuse where arches overlap), but arm64 layers now persist between publishes — subsequent publishes in a week drop from ~30 min to a few minutes.
  • Timeout 30 → 45 min so the one genuinely cold build per week clears the line instead of dying at 99%.

Verification

  • YAML validated; the change only touches the publish job (main-only), so PR checks exercise build+scan as usual.
  • Real proof is the next two merges to main: the first publish will be cold (~30 min, now within the 45-min budget); the second should be dramatically faster off the preserved arm64 cache.

Notes

  • Alternative considered: dropping arm64 from the publish matrix. Rejected — Apple Silicon machines pull these images locally.

🤖 Generated with Claude Code

The multi-arch GHCR publish timed out at 30 minutes on two of the last
three main builds, cancelling the run and skipping the deploy each
time. Root cause: the amd64-only build job writes to the same weekly
GHA cache scope on every PR and push, evicting the arm64 layers the
publish job caches — so every publish rebuilt arm64 from scratch under
QEMU (~30 min, exactly at the limit).

Give the publish job its own prod-multiarch-<week> cache scope: it
still reads the shared amd64 scope for layer reuse, but its arm64
layers now survive between publishes. Bump the timeout to 45 min so
the one cold build per week (or busy day) clears the limit instead of
dying at 99%.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant