Skip to content

Canonical CI: grouped-tests.yml + root test/test_groups.toml#144

Merged
ChrisRackauckas merged 3 commits into
SciML:masterfrom
ChrisRackauckas-Claude:grouped-tests-ci
Jun 10, 2026
Merged

Canonical CI: grouped-tests.yml + root test/test_groups.toml#144
ChrisRackauckas merged 3 commits into
SciML:masterfrom
ChrisRackauckas-Claude:grouped-tests-ci

Conversation

@ChrisRackauckas-Claude

Copy link
Copy Markdown
Contributor

Converts the root test workflow to the canonical SciML thin caller using SciML/.github grouped-tests.yml@v1, with the test matrix declared once in test/test_groups.toml.

Changes

  • .github/workflows/Tests.yml — now a thin caller of grouped-tests.yml@v1 (secrets: inherit, no with: needed: defaults match — GROUP env name, check-bounds: yes, coverage on, src,ext dirs). name:, on:, and concurrency: preserved verbatim. No other workflows touched.
  • test/test_groups.toml (new) — root matrix. The functional suite is split into two groups solely to reproduce the old non-uniform matrix exactly:
    • [Core] version "1" on ubuntu-latest, windows-latest, macOS-15-intel, macOS-latest
    • [CoreLTSPre] versions lts, pre on ubuntu-latest
    • [QA] versions lts, 1
  • test/runtests.jl — adds GROUP dispatch. Every non-QA group runs the full existing functional suite; QA runs the QA file.
  • test/qa/ (new) — isolated QA environment (Aqua, JET, Test, Evolutionary via [sources] path) and qa.jl running Aqua.test_all(Evolutionary) + JET.test_package(Evolutionary; target_defined_modules=true).

Matrix match (verified statically via compute_affected_sublibraries.jl --root-matrix)

Old Tests.yml matrix:

  • 1 × {ubuntu-latest, windows-latest, macOS-15-intel, macOS-latest}
  • lts × {ubuntu-latest}
  • pre × {ubuntu-latest}

Emitted functional cells are byte-for-byte the same 6-cell set. arch is auto-detected from each runner (aarch64 on macOS-latest, x64 elsewhere), matching the old include arch axis. The QA group (QA/lts, QA/1 on ubuntu-latest) is newly added.

Notes

  • QA group newly wired; Aqua/JET run in CI — any failures will be triaged in follow-up.
  • Project metadata: [compat] julia already "1.10" (LTS floor); all [extras] deps already have [compat] entries — no changes needed.
  • Structural conversion only; tests/Aqua/JET were not run locally. TOML + YAML parse and the matrix computation were verified statically.

Ignore until reviewed by @ChrisRackauckas.

ChrisRackauckas and others added 3 commits June 9, 2026 18:11
Convert the root test workflow (Tests.yml) to the canonical SciML thin
caller using SciML/.github grouped-tests.yml@v1, with the version/OS
matrix declared once in test/test_groups.toml.

- Tests.yml becomes a thin caller (on:/concurrency:/name: preserved).
- test/test_groups.toml reproduces the old matrix exactly: Core/"1" on
  ubuntu/windows/macOS-15-intel/macOS-latest, plus lts+pre on
  ubuntu-latest. arch is auto-detected from each runner (aarch64 on
  macOS-latest, x64 elsewhere), matching the old include arch axis.
- runtests.jl gains GROUP dispatch; the existing suite runs under any
  non-QA group.
- New QA group (test/qa/) wires Aqua.test_all + JET.test_package in an
  isolated environment.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The grouped-tests QA group activates test/qa as its own project, so the QA sub-env must carry Pkg (used by the activation/instantiate path).

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…roken

The grouped-tests QA group could not load under the reusable workflow because
test/qa/qa.jl does `using Pkg` to activate test/qa/Project.toml, but the test
target only listed Test + StableRNGs. Add Pkg to [extras]/[targets].test (with
a Pkg = "1" compat entry so Aqua deps_compat stays green), matching the
FastBroadcast.jl QA pattern.

The QA group then surfaced genuine Aqua and JET findings. Keep all passing
Aqua sub-checks running and mark only the failing checks broken:

- Aqua ambiguities: 18 (optimize overloads, src/api/optimize.jl)
- Aqua unbound_args: 3 methods (src/api/expressions.jl)
- Aqua piracies: 7 methods (Expr ops, replace, value)
- JET report_package: errors (+(::Nothing,::Int), kwcall on deprecated
  crossover/mutation aliases in src/deprecated.jl)

Tracked in SciML#145

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@ChrisRackauckas ChrisRackauckas marked this pull request as ready for review June 10, 2026 09:46
@ChrisRackauckas ChrisRackauckas merged commit 2a620b2 into SciML:master Jun 10, 2026
10 of 14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants