feat(process-storms): fail-fast AORC pre-flight before the scan#15
Merged
HenryGeorgist merged 1 commit intoJul 15, 2026
Merged
Conversation
process-storms could run 10+ minutes (or hours) before dying with a cryptic zarr KeyError: '.zmetadata' when a required AORC year was never mirrored (the 2026-05-27 2025.zarr incident). HEAD each year's .zmetadata up front and raise in seconds with a copy-pasteable './run.py mirror' command instead. Ported from the fork's plugin/actions/aorc_preflight.py (stdlib + boto3). No-op when AORC_S3_BASE_URL is unset (local dev); only runs on the scan path, not on a catalog reload.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
process-stormscould run 10+ minutes (or hours) before dying with a crypticzarr KeyError: '.zmetadata'when a required AORC year was never mirrored (the2026-05-27
2025.zarrincident). This HEADs each required year's.zmetadataup front and raises in seconds with a copy-pasteable
./run.py mirrorcommand.plugin/actions/aorc_preflight.py(stdlib + boto3).AORC_S3_BASE_URLis unset (local dev); warns but doesn't raisewhen creds are absent (can't probe a private bucket).
required_yearstacks on the next year when a late-year storm window crossesthe boundary.
Tests:
test/test_aorc_preflight.py(required-years incl. boundary crossing;skip-when-no-base-url; raise-on-missing-year). 5 pass.