Consolidate and fix bugs in GitLab "download from Azure" scripts - #9103
Open
andrewlock wants to merge 6 commits into
Open
Consolidate and fix bugs in GitLab "download from Azure" scripts#9103andrewlock wants to merge 6 commits into
andrewlock wants to merge 6 commits into
Conversation
… scripts download-serverless-artifacts.sh, download-single-step-artifacts.sh and download-nuget-packages-to-sign.sh each duplicated the same ~90 lines to resolve an Azure DevOps build for the current commit and poll for a named artifact. Every fix to this logic so far (#8782, #7870) had to be applied to multiple copies, and the copies had already drifted cosmetically. Move the shared logic into a new sourced helper. The three scripts keep only what's specific to them: their own tag/release fallback paths and post-download layout.
… sleep Fixes wwo bugs in the Azure artifact poll loop, present since it was first written: 1. `(( STARTED < ARTIFACT_TIMEOUT ))` ran as a bare statement under `set -e`. When the loop genuinely timed out, this evaluated false and aborted the script right there, so the "ERROR: No downloadUrl found" block below it - added in #7870 specifically to diagnose timeouts - could never run. Removed; the existing `if [ -z "$downloadUrl" ]` check already covers the timeout case. 2. `sleep "$ARTIFACT_POLL_INTERVAL"` ran unconditionally at the end of the loop body, after the artifact had already been found, adding a pointless 100s wait to every successful run (300s for the three sequential artifacts in download-nuget-packages-to-sign.sh). Moved the sleep and counter increment inside the "not found yet" branch.
…fact download_azure_artifact polled the artifact endpoint for up to 40 minutes without ever checking whether the selected Azure DevOps build was still capable of producing the artifact, so a failed/skipped publishing stage (or a canceled build) meant waiting out the full timeout for nothing - delaying sign-nuget-packages, package-oci and aws-lambda-layer. Build `result` alone is not a safe signal (failed builds routinely still publish every artifact), so this checks the specific pipeline stage that publishes each artifact (via the build timeline) and falls back to the whole-build status only when that stage can't be resolved, so an unmapped artifact or a future pipeline rename degrades gracefully instead of breaking releases. Before failing, it also looks for a different build of the exact same commit that can still supply the artifact, to recover from a newer build of the same commit failing while an earlier one succeeded.
…iven retry Replace the previous commit's stage-timeline doom detection and silent build-switching with a simpler rule: poll until the build finishes; if the artifact still isn't there, that build is done for. download_azure_artifact now returns a documented AZDO_ARTIFACT_UNAVAILABLE code instead of exiting, and the new download_azure_artifacts_from_one_build wrapper owns recovery. This ensures jobs waiting for multiple artifacts can discard all artifacts downloaded so far and retry against a different build so artifacts are never mixed across builds, and fails fast when resolve_azure_build_id has no better alternative left to offer.
andrewlock
added
the
area:builds
BenchmarksBenchmark execution time: 2026-08-24 11:34:42 Comparing candidate commit 8603680 in PR branch Found 0 performance improvements and 1 performance regressions! Performance is the same for 71 metrics, 0 unstable metrics, 66 known flaky benchmarks, 60 flaky benchmarks without significant changes.
|
Execution-Time Benchmarks Report ⏱️Execution-time results for samples comparing This PR (9103) and master. ✅ No regressions detected |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary of changes
Reason for change
As part of #9083, @NachoEchevarria flagged some potential improvements to the scripts, for example a cancelled build today would cause flakiness for the download scripts. We now also have 3 copies of essentially the same code, and AI spotted several other bugs, all fixed here
Implementation details
Test coverage
This is the test
Other details
Note that the
.gitlab/download-serverless-artifacts.shand.gitlab/download-single-step-artifacts.shhave different paths they take when running against tags (during release)..gitlab/download-nuget-packages-to-sign.shdoes not have this path, as it does not need to run on tags (there may be an edge case there where we are running - need to check)