You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 68a3123
Browse filesBrowse the repository at this point in the historyBrowse files
fix(ci): accumulate shard-timings runs until every package has three executions
The refresh loop broke at the first accumulation that covered the
workspace, and coverage counts a carried weight as covered, so every
refresh since the hourly run went live was one run deep. It now stops
only when the dataset's provisional list is empty: every package
executed by at least MINIMUM_EXECUTED_RUNS (three) of the runs fed.
A refresh whose retained runs cannot reach that is kept and named
provisional rather than refused, and the PR body carries each
package's run count and min/median/max.
Claude-Session: https://claude.ai/code/session_01VF48aw8RPG6wzDnMgp6rtw
Co-authored-by: Claude <noreply@anthropic.com>
# HIT witnesses that its inputs are unchanged; anything else is left
446
480
# out for the coverage check below to name. The workflow still writes
447
481
# no byte the generator did not emit — the merge happens inside it.
482
+
# `--spread-out`: every package's executed-run count and
483
+
# min/median/max, for the PR body. Written beside the dataset,
484
+
# never into it — nothing machine-reads a spread.
448
485
if ! node scripts/measure-test-shard-timings.mjs "${ARGS[@]}" \
449
486
--merge-into scripts/test-shard-timings.json \
450
-
--out "$WORK/refreshed.json"; then
487
+
--out "$WORK/refreshed.json" \
488
+
--spread-out "$WORK/spread.md"; then
451
489
echo "::warning::The generator refused the set including run $RUN_ID; dropping that run and continuing."
452
490
unset 'ACCEPTED[-1]'
453
491
echo "::endgroup::"; continue
@@ -459,17 +497,41 @@ jobs:
459
497
--refreshed "$WORK/refreshed.json" \
460
498
--workspace "$RUNNER_TEMP/turbo-ls.json" \
461
499
--exclude @objectstack/dogfood; then
462
-
CHOSEN="$RUN_ID"
500
+
COVERED=1
501
+
else
502
+
COVERED=''
503
+
fi
504
+
# How many weights still rest on fewer than the minimum executed
505
+
# runs. Read from the generator's own `provisional` list rather than
506
+
# recounted here, and an absent list is an error, not a zero: a
507
+
# zero read from nothing would end the loop at one run again.
508
+
SHORT=$(node -e '
509
+
const d = JSON.parse(require("fs").readFileSync(process.argv[1], "utf8"));
510
+
if (!Array.isArray(d.provisional)) throw new Error(process.argv[1] + " carries no `provisional` list");
511
+
console.log(d.provisional.length);
512
+
' "$WORK/refreshed.json")
513
+
if [ -n "$COVERED" ] && [ "$SHORT" -eq 0 ]; then
514
+
echo "Every package is covered and executed by at least the minimum number of these runs; stopping."
463
515
echo "::endgroup::"
464
516
break
465
517
fi
518
+
if [ -n "$COVERED" ]; then
519
+
echo "Covered, but $SHORT package weight(s) rest on fewer than the minimum executed runs; adding the next older run."
520
+
fi
466
521
echo "::endgroup::"
467
522
done
468
523
469
-
if [ -z "$CHOSEN" ]; then
524
+
if [ -z "$COVERED" ]; then
470
525
echo "::error::The ${#ACCEPTED[@]} eligible run(s) on main, accumulated and merged with the committed dataset, still leave a package that HAD a measured weight with neither a fresh measurement nor a turbo cache HIT to witness that it is unchanged — the shortfall above names them. Each would drop to the test-file-count ESTIMATE, which is the silent degradation this lane exists to prevent. NOTHING was regenerated and no PR was opened: this is a refusal, not a quiet success. A package that merely went unmeasured is NOT this error — that case is carried on its cache-hit witness — so a shortfall here means a suite failed, a package was renamed or removed, or its slices could not be assembled in any run."
471
526
exit 1
472
527
fi
528
+
# PROVISIONAL, not refused (#22014): the candidates ran out before
529
+
# every package reached the minimum. The measurement is kept, the
530
+
# short packages are named in the dataset and the PR body, and this
531
+
# annotation makes the reading visible on a green job.
532
+
if [ "$SHORT" -gt 0 ]; then
533
+
echo "::warning::Shard timings PROVISIONAL: $SHORT package weight(s) rest on fewer than the minimum executed runs across all ${#ACCEPTED[@]} retained run(s). They are named in the dataset's \`provisional\` list; read them as balancing inputs, not baselines."
534
+
fi
473
535
# Candidates arrive newest-first, so ACCEPTED[0] is the most recent run
474
536
# in the set and its commit is the one the refresh is dated from. The
475
537
# full list travels alongside it: every run in it contributed
@@ -553,6 +615,25 @@ jobs:
553
615
}
554
616
')
555
617
618
+
# Sample depth (#22014), read out of the dataset like the carry line
619
+
# above. An absent list is an error rather than "none provisional".
620
+
DEPTH_LINE=$(node -e '
621
+
const d = JSON.parse(require("fs").readFileSync(process.env.RUNNER_TEMP + "/refresh/refreshed.json", "utf8"));
622
+
if (!Array.isArray(d.provisional)) throw new Error("the refreshed dataset carries no `provisional` list");
623
+
const min = d.provenance.minimumRuns;
624
+
const total = Object.keys(d.packages).length;
625
+
if (d.provisional.length === 0) {
626
+
console.log(`Every one of the ${total} weights rests on at least ${min} executed runs; none is provisional.`);
627
+
} else {
628
+
console.log(
629
+
`⚠️ PROVISIONAL: ${d.provisional.length} of the ${total} weights rest on fewer than ${min} executed ` +
630
+
`runs, because the retained runs replayed them more often than they executed them. They are ` +
631
+
`named in the dataset \`provisional\` list and are fit to balance on, NOT to set a per-package ` +
0 commit comments