Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 19 additions & 3 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,23 @@ concurrency:
env:
# Number of parallel bats shards per OS. The matrix and the shard helper must
# stay in lockstep; the stable summary job below verifies that they do.
SHARD_TOTAL: 4
#
# 4 -> 5 (#1107 stopgap). The suite grew ~2.85x (788 -> 2246 @tests) since the
# partition was tuned, and shard-tests.sh seeds each pinned file at one shard's
# AVERAGE share (total_tests/total = 561 now), which over-reserves the two pin
# shards and pushes the whole real load onto the non-pin ones: measured 4-shard
# @test weights were 281/302/832/831, and macOS 4/4 (831) reached the 30-min
# cap while still passing tests. More shards dilutes "the remaining work over
# the non-pin shards" (worst-case per non-pin shard = total*(N+2)/N^2: 842 at
# N=4, 629 at N=5). N=5 (not 6) because the account runs at most FIVE macOS
# jobs at once (see the concurrency header): a 6th macOS shard would queue
# behind the other five of its own run, inflating the wall clock past what a
# lighter shard buys. Five fits the slots exactly, no self-queue). This does NOT fix the scaling property -- pin_seed still
# grows with the suite and the count weight still under-values wait-dominant
# files (test_remote 7.5m / test_remote_sync 5.0m / test_watch 3.0m on macOS,
# none of them the currently pinned pair) -- tracked in #847 for a real fix
# once #1159's per-file timings land.
SHARD_TOTAL: 5
# The Windows legs' sqlite3, pinned three ways (#824).
#
# This comment used to end "one trip to chocolatey, which is the only time
Expand Down Expand Up @@ -249,7 +265,7 @@ jobs:
# the required contexts are `bats` (the aggregate) and the Windows install
# leg, and neither is renamed, so branch protection sees exactly what it
# saw. `needs` is one of the contexts a job name may read.
name: bats (${{ matrix.os }} ${{ matrix.shard }}/4)${{ needs.changes.outputs.docs_only == 'true' && ' — docs-only, suite skipped' || '' }}
name: bats (${{ matrix.os }} ${{ matrix.shard }}/5)${{ needs.changes.outputs.docs_only == 'true' && ' — docs-only, suite skipped' || '' }}
needs: changes
# Run even if `changes` somehow failed/was skipped — fail open to the full
# suite rather than leaving this REQUIRED check unreported (which would
Expand Down Expand Up @@ -283,7 +299,7 @@ jobs:
fail-fast: false
matrix:
os: [ubuntu-latest, macos-latest]
shard: [1, 2, 3, 4]
shard: [1, 2, 3, 4, 5]
steps:
- uses: actions/checkout@v4

Expand Down
4 changes: 2 additions & 2 deletions tests/test_ci_workflow.bats
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@
local workflow="$BATS_TEST_DIRNAME/../.github/workflows/tests.yml"
# 1. The shard job's name carries the marker, keyed on the docs_only output.
# Shard checks are not required contexts, so the name may vary.
grep -Fq "name: bats (\${{ matrix.os }} \${{ matrix.shard }}/4)\${{ needs.changes.outputs.docs_only == 'true' && ' — docs-only, suite skipped' || '' }}" "$workflow"
grep -Fq "name: bats (\${{ matrix.os }} \${{ matrix.shard }}/5)\${{ needs.changes.outputs.docs_only == 'true' && ' — docs-only, suite skipped' || '' }}" "$workflow"
# 2. The aggregate -- the required context -- keeps its exact name, once,
# unconditionally.
[ "$(grep -c '^ name: bats$' "$workflow")" -eq 1 ]
Expand All @@ -72,7 +72,7 @@
# The mutation took: the marker is gone from the copy.
if grep -Fq "docs-only, suite skipped' || ''" "$mutant"; then false; fi
# ...and the name pin no longer matches it.
if grep -Fq "name: bats (\${{ matrix.os }} \${{ matrix.shard }}/4)\${{ needs.changes.outputs.docs_only == 'true' && ' — docs-only, suite skipped' || '' }}" "$mutant"; then false; fi
if grep -Fq "name: bats (\${{ matrix.os }} \${{ matrix.shard }}/5)\${{ needs.changes.outputs.docs_only == 'true' && ' — docs-only, suite skipped' || '' }}" "$mutant"; then false; fi
}

# The suite has to run on the shape that actually gets dogfooded. A PR is only
Expand Down
Loading