Skip to content

Canonical CI: grouped-tests.yml + root test/test_groups.toml#227

Merged
ChrisRackauckas merged 4 commits into
SciML:masterfrom
ChrisRackauckas-Claude:grouped-tests-ci
Jun 15, 2026
Merged

Canonical CI: grouped-tests.yml + root test/test_groups.toml#227
ChrisRackauckas merged 4 commits into
SciML:masterfrom
ChrisRackauckas-Claude:grouped-tests-ci

Conversation

@ChrisRackauckas-Claude

Copy link
Copy Markdown
Contributor

Summary

Converts the root Tests.yml matrix test job into a thin caller of the canonical SciML/.github/.github/workflows/grouped-tests.yml@v1 reusable workflow, with the group × version (× os) matrix now declared once in test/test_groups.toml.

  • Tests.yml: the old version {1, lts, pre} × os {ubuntu, macos, windows} matrix job is replaced by the thin grouped-tests.yml@v1 caller. on: and concurrency: are preserved verbatim; filename and name: unchanged. No with: overrides needed — the repo reads GROUP (default), uses default check-bounds, and default coverage dirs (src,ext).
  • test/test_groups.toml (new):
    • [Core] on ["lts", "1", "pre"] × ["ubuntu-latest", "macos-latest", "windows-latest"] — reproduces the old functional matrix exactly.
    • [QA] on ["lts", "1"] (ubuntu).

Category B refactor (inline Aqua → isolated QA group)

Aqua had been running inline under GROUP=="All"/"Core" (via test/qa.jl). It is now moved into an isolated QA group gated on GROUP=="QA":

  • test/qa/Project.toml (new): Aqua + JET + Test, package via [sources] path = "../..", [compat] julia = "1.10".
  • test/qa/qa.jl (new): the Aqua checks plus the previously-unwired JET report_call checks (consolidated from test/jet/jet.jl).
  • runtests.jl: basictests.jl stays under GROUP Core/All; a new GROUP=="QA" branch activates test/qa, develops the package, instantiates, and includes qa/qa.jl. (GPU branch unchanged.)
  • Removed the old test/qa.jl and test/jet/ (JET now lives in the QA env).

GPU.yml (self-hosted), Downstream.yml, Downgrade.yml, Documentation.yml, and the other workflows are left untouched.

Matrix match (static verification)

compute_affected_sublibraries.jl . --root-matrix (run against SciML/.github@v1) emits:

  • Core: 9 cells = {lts, 1, pre} × {ubuntu-latest, macos-latest, windows-latest} — a 1:1 reproduction of the old Tests.yml functional matrix.
  • QA: 2 cells = {lts, 1} on ubuntu-latest.

All touched TOML and YAML files parse cleanly. Tests/Aqua/JET were not run locally (structural conversion only).

QA group newly wired; Aqua/JET run in CI — any failures will be triaged in a follow-up.

Project.toml metadata: [compat] julia is already "1.10" (LTS floor) and every [extras] dependency already has a [compat] entry, so no metadata fixes were required.


Ignore until reviewed by @ChrisRackauckas.

🤖 Generated with Claude Code

ChrisRackauckas and others added 3 commits June 9, 2026 17:57
Convert the root Tests.yml matrix test job into a thin caller of the
canonical SciML/.github grouped-tests.yml@v1 reusable workflow, with the
group x version (x os) matrix declared once in test/test_groups.toml.

- Tests.yml: replace the 3-version x 3-OS matrix job with the thin
  grouped-tests.yml@v1 caller (on: + concurrency: preserved verbatim).
- test/test_groups.toml: [Core] on [lts, 1, pre] x [ubuntu, macos,
  windows] (reproduces the old functional matrix); [QA] on [lts, 1].
- Category B refactor: Aqua had been running inline under GROUP
  Core/All. Move Aqua (and the previously-unwired JET checks) into an
  isolated QA group gated on GROUP=="QA", with test/qa/Project.toml
  (Aqua + JET + Test, package via [sources] path="../..", julia 1.10)
  and test/qa/qa.jl. runtests.jl now activates test/qa for GROUP=="QA".
- Functional basictests.jl stays under GROUP Core/All.
- Remove the old test/qa.jl (Aqua) and test/jet/ (JET now in QA env).

GPU.yml, Downstream.yml, Downgrade.yml, Documentation.yml and the other
workflows are left untouched.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The grouped-tests test_groups.toml only listed Core and QA, but
test/runtests.jl dispatches on GROUP == "GPU" and the pre-conversion
CI ran a separate GPU.yml job (GROUP: "GPU") on a self-hosted GPU
runner. As a result the GPU functional group never ran under the new
grouped-tests CI. Re-add it as its own group matching the old GPU.yml:
self-hosted Linux/X64/gpu runner, 60-minute timeout, Julia "1".

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The GPU group in test/test_groups.toml supersedes the standalone
GPU.yml workflow; keeping both ran the GPU tests twice on the same
self-hosted runner. Also add macOS-15-intel to the Core os list,
which the conversion's os array had dropped from the old matrix.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@ChrisRackauckas-Claude

Copy link
Copy Markdown
Contributor Author

Pushed 93c032e with two fixes to the conversion:

  1. Deleted .github/workflows/GPU.yml. The [GPU] group in test/test_groups.toml already runs the CUDA tests via the grouped-tests workflow on the same [self-hosted, Linux, X64, gpu] runner, so keeping the standalone workflow ran the GPU tests twice per CI event. Per the GPU-as-group convention, the group supersedes the workflow.

  2. Added macOS-15-intel to Core's os array. The old functional matrix ran Julia 1 on macOS-15-intel (x64 mac), but the conversion's [ubuntu-latest, macos-latest, windows-latest] list dropped that cell. Note this is a slight expansion: the group matrix runs all of lts/1/pre on the Intel mac rather than just 1 — acceptable, and it gives x64-mac coverage across all tested Julia versions.

Static verification (compute_affected_sublibraries.jl --root-matrix from SciML/.github@v1): Core = 12 cells (lts/1/pre x ubuntu-latest/macos-latest/windows-latest/macOS-15-intel), GPU = exactly 1 cell on ["self-hosted","Linux","X64","gpu"] with timeout 60, QA = 2 cells (lts/1 on ubuntu-latest).

…ts conversion)

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@ChrisRackauckas-Claude

Copy link
Copy Markdown
Contributor Author

Added the SciMLTesting v1.2 folder-based run_tests harness on top of this grouped-tests conversion; merges as one PR (grouped-tests + SciMLTesting v1.2).

@ChrisRackauckas ChrisRackauckas marked this pull request as ready for review June 15, 2026 04:57
@ChrisRackauckas ChrisRackauckas merged commit 1db7df4 into SciML:master Jun 15, 2026
24 of 36 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants