Skip to content

Commit c1d204f

Browse files
EtienneLescotEtienneLescot
committed
fix(build): fall back to a mirror when ffmpeg.org refuses the runner
The macOS x64 leg of v1.8.0-rc.6 died on the ffmpeg tarball fetch, twice, twenty minutes apart: curl: (35) Recv failure: Connection reset by peer This is not congestion and not a bad pin. In both runs the arm64 leg on `macos-latest` fetched the same tarball from the same host with the same cold cache and succeeded, while the x64 leg on `macos-15-intel` was reset about a second after starting. An immediate reset that reproduces on one runner pool and never on the other is an egress-level block, so no amount of retrying clears it — a first attempt at this shipped only `--retry-all-errors` and would not have fixed the build. So the source list grows a second entry. Debian's `.orig.tar.xz` is the upstream tarball unmodified — verified byte-identical to the pinned sha256 — and deb.debian.org is CDN-backed. Each source is tried in turn and must both download and match the checksum; the checksum is what makes a second origin safe to trust, and it gates every source equally. Nothing is downgraded: ffmpeg.org stays first, and a mirror that lacks a future version is skipped rather than fatal. Three flags carry their own reasons: - `--retry-all-errors`, because curl only auto-retries what it classes as transient (timeouts, 429, 5xx) — not a reset, not a handshake failure, which are exactly the errors seen here. Verified against a refused connection: `--retry 2 --retry-delay 1` gives up after 0s, adding `--retry-all-errors` spends 2s retrying. - `--connect-timeout 20`, because a throttled origin hangs rather than refuses. Measured: after a few rapid fetches ffmpeg.org left a connect sitting for 75s before failing. Times four attempts, that is five minutes burned before the second source is even tried. - `-f`, so an HTTP error page is not written to the tarball and resurfaced as a checksum mismatch, which reads like a moved pin rather than a bad response. Exercised by extracting the function and running it against real endpoints: canonical alone succeeds; a dead first source falls through to Debian; a source serving the wrong bytes (ffmpeg 8.1.1) is rejected on checksum and the next one is used; and with every source broken it throws listing each attempt with its reason — curl status or the actual hash. The sibling `scripts/fetch-ffmpeg.mjs` has the same single-source shape at line 359 but is left alone: it is the Windows/Linux path, pulls from GitHub releases rather than ffmpeg.org, and uses node's fetch rather than curl.
1 parent 9bb014d commit c1d204f

1 file changed

Lines changed: 84 additions & 10 deletions

File tree

‎scripts/fetch-ffmpeg-macos.mjs‎

Lines changed: 84 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,89 @@ function run(cmd, args, opts = {}) {
4040
}
4141
}
4242

43+
/**
44+
* Where the pinned tarball can be fetched, in order of preference.
45+
*
46+
* ffmpeg.org is canonical and stays first, but it cannot be the only one:
47+
* GitHub's `macos-15-intel` runner pool cannot reach it. On the v1.8.0-rc.6
48+
* build the x64 leg died twice, twenty minutes apart, on
49+
* `curl: (35) Recv failure: Connection reset by peer` about a second after
50+
* starting — while the arm64 leg on `macos-latest` fetched the same tarball
51+
* from the same host in the same runs and succeeded both times. An immediate
52+
* reset that reproduces on one runner pool and never on the other is an
53+
* egress-level block, not congestion, so retrying alone does not clear it.
54+
*
55+
* Debian's `.orig.tar.xz` is the upstream tarball unmodified — verified
56+
* byte-identical to TARBALL_SHA256 below — and deb.debian.org is CDN-backed.
57+
* It is a fallback, not a replacement: the checksum is what makes trusting a
58+
* second origin safe, and it gates every source equally.
59+
*
60+
* When the pin moves, a mirror may not carry the new version yet. That is not
61+
* a failure mode to design around — the list is tried in order and a source
62+
* that 404s is simply skipped, with every attempt reported if none works.
63+
*/
64+
const TARBALL_URLS = [
65+
`https://ffmpeg.org/releases/ffmpeg-${VERSION}.tar.xz`,
66+
`https://deb.debian.org/debian/pool/main/f/ffmpeg/ffmpeg_${VERSION}.orig.tar.xz`,
67+
];
68+
69+
/**
70+
* Fetches the pinned tarball to `dest`, trying each source until one both
71+
* downloads and matches the checksum.
72+
*
73+
* `--retry-all-errors` rather than a plain `--retry`: curl only auto-retries
74+
* what it classes as transient (timeouts, 429, 5xx), which does not include a
75+
* connection reset or a TLS handshake failure — precisely the errors seen here.
76+
* `-f` keeps an HTTP error page from being written to the tarball and
77+
* resurfacing as a checksum mismatch, which reads like a moved pin.
78+
*/
79+
function downloadTarball(dest) {
80+
const failures = [];
81+
for (const url of TARBALL_URLS) {
82+
console.log(`Downloading ffmpeg ${VERSION} from ${new URL(url).host}…`);
83+
// --connect-timeout bounds the fallback, and is not decoration: a throttled
84+
// origin does not refuse, it hangs. Measured against ffmpeg.org after a few
85+
// rapid fetches, a single connect sat for 75s before failing — times four
86+
// attempts, that is five minutes of a build spent before the second source
87+
// is even tried. 20s is far above any healthy handshake.
88+
const r = spawnSync(
89+
"curl",
90+
[
91+
"-fsSL",
92+
"--connect-timeout",
93+
"20",
94+
"--retry",
95+
"3",
96+
"--retry-delay",
97+
"2",
98+
"--retry-all-errors",
99+
"-o",
100+
dest,
101+
url,
102+
],
103+
{ stdio: "inherit" },
104+
);
105+
if (r.status !== 0) {
106+
failures.push(` ${url}\n curl exited with ${r.status}`);
107+
continue;
108+
}
109+
const actual = crypto.createHash("sha256").update(fs.readFileSync(dest)).digest("hex");
110+
if (actual !== TARBALL_SHA256) {
111+
// Not fatal on its own — a mirror may carry a repacked tarball. It is
112+
// reported in full, so a genuinely moved pin is still legible as
113+
// "every source disagreed the same way" rather than "network down".
114+
failures.push(` ${url}\n checksum ${actual}`);
115+
continue;
116+
}
117+
console.log("Checksum OK.");
118+
return;
119+
}
120+
throw new Error(
121+
`Could not obtain ffmpeg ${VERSION} from any source.\n` +
122+
`Expected sha256 ${TARBALL_SHA256}\n${failures.join("\n")}`,
123+
);
124+
}
125+
43126
/** The binary's own licence banner — the only claim worth trusting. */
44127
function isLgpl(dir) {
45128
const bin = path.join(dir, "bin", "ffmpeg");
@@ -68,16 +151,7 @@ if (fs.existsSync(path.join(DEST, "include"))) {
68151
const work = fs.mkdtempSync(path.join(os.tmpdir(), "openscreen-ffmpeg-"));
69152
const tarball = path.join(work, `ffmpeg-${VERSION}.tar.xz`);
70153

71-
console.log(`Downloading ffmpeg ${VERSION}…`);
72-
run("curl", ["-sSL", "-o", tarball, `https://ffmpeg.org/releases/ffmpeg-${VERSION}.tar.xz`]);
73-
74-
const actual = crypto.createHash("sha256").update(fs.readFileSync(tarball)).digest("hex");
75-
if (actual !== TARBALL_SHA256) {
76-
throw new Error(
77-
`Checksum mismatch for the ffmpeg tarball.\n expected ${TARBALL_SHA256}\n got ${actual}`,
78-
);
79-
}
80-
console.log("Checksum OK.");
154+
downloadTarball(tarball);
81155

82156
run("tar", ["-xJf", tarball, "-C", work]);
83157
const src = path.join(work, `ffmpeg-${VERSION}`);

0 commit comments

Comments
 (0)