Canonical CI: grouped-tests.yml + root test/test_groups.toml#1484
Open
ChrisRackauckas-Claude wants to merge 2 commits into
Open
Canonical CI: grouped-tests.yml + root test/test_groups.toml#1484ChrisRackauckas-Claude wants to merge 2 commits into
ChrisRackauckas-Claude wants to merge 2 commits into
Conversation
Convert the root test workflow (Test.yml) to the canonical grouped-tests.yml@v1 thin caller, declaring the group x version matrix once in test/test_groups.toml instead of a hand-maintained YAML matrix. - Test.yml: replace the version x group matrix job (tests.yml@v1) with the grouped-tests.yml@v1 thin caller. Preserve filename, name: "Tests", on: triggers, and concurrency: verbatim. Keep coverage: false (the old job disabled coverage); all other inputs are defaults. - test/test_groups.toml: [Core] on [lts, 1, pre]; [QA] on [lts, 1]. Linux-only (no os field -> ubuntu-latest default). - Refactor to canonical Core/QA. The functional suite (formerly the Modeling/Simulation/Hybrid/Misc/Spatial groups) now runs under GROUP "Core" (and the local-default "All"); the fine-grained names still work for local runs. The ExplicitImports import-hygiene check moves out of the inline Misc group into an isolated QA group (test/qa/Project.toml with [sources] path to the repo root + test/qa/qa.jl) gated on GROUP == "QA". - Project.toml: drop ExplicitImports from [extras]/[targets].test and its [compat] entry (now lives only in the QA env). julia compat floor is already "1.10" (LTS). Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Member
|
Why is this eliminating the current group structure? That was intentional as otherwise CI times become gigantic. |
Member
|
Nothing is eliminated. |
Member
|
The matrix is exactly the same. |
Member
|
The PR says it explicitly now runs all tests in one group except for QA though, and most of the prior groups are not present in the group CI file. |
Member
|
Okay that's an error. They aren't supposed to be in the CI file, but they aren't reflected in the test_groups. I'll get that fixed. |
The grouped-tests-ci conversion collapsed Catalyst's six parallel functional test groups into a single [Core] entry and dropped the Extensions group entirely. runtests.jl dispatches on Modeling, Simulation, Hybrid, Misc, Spatial, and Extensions (the last gated on `GROUP == "All" || GROUP == "Extensions"`, which RUN_ALL/Core never covers), so with only [Core]+[QA] the Extensions suite never ran and the other five no longer ran as separate CI jobs. List each named functional group as its own entry, matching the old Test.yml group matrix (Modeling/Simulation/Hybrid/Misc/Spatial on lts/1/pre) plus Extensions (lts/1/pre, from TestExtensions.yml), and keep [QA] on lts/1. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Comment on lines
+1
to
+17
| [Modeling] | ||
| versions = ["lts", "1", "pre"] | ||
|
|
||
| [Simulation] | ||
| versions = ["lts", "1", "pre"] | ||
|
|
||
| [Hybrid] | ||
| versions = ["lts", "1", "pre"] | ||
|
|
||
| [Misc] | ||
| versions = ["lts", "1", "pre"] | ||
|
|
||
| [Spatial] | ||
| versions = ["lts", "1", "pre"] | ||
|
|
||
| [Extensions] | ||
| versions = ["lts", "1", "pre"] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Converts the root test workflow (
.github/workflows/Test.yml,name: "Tests") from a hand-maintainedversion × groupmatrix into the canonicalgrouped-tests.yml@v1thin caller, with the matrix declared once in a roottest/test_groups.toml.What changed
.github/workflows/Test.yml— the matrix test job (formerlytests.yml@v1with a 3-version × 5-group matrix) becomes the thin caller:name: "Tests",on:triggers, andconcurrency:are preserved verbatim.coverage: falseis kept (the old job disabled coverage).group-env-nameis omitted becauseruntests.jlreads the canonicalGROUPvar;check-bounds,coverage-directories,apt-packagesare all left at their defaults (the old job used the defaults). Linux-only: noosfield (defaultubuntu-latest).test/test_groups.toml(new) —[Core]on["lts","1","pre"],[QA]on["lts","1"].ExplicitImportsimport-hygiene test running inline in theMiscgroup. That check is moved into an isolated QA group:test/qa/Project.toml—[deps]=Catalyst(via[sources]path = "../..") +ExplicitImports+Test;[compat]julia = "1.10",ExplicitImports = "1.15".test/qa/qa.jl— the moved ExplicitImports checks.runtests.jl— whenGROUP == "QA", activatetest/qa,Pkg.developthe repo root, instantiate, andinclude("qa/qa.jl"). The functional suite (formerly Modeling/Simulation/Hybrid/Misc/Spatial) now runs underCore(and the local defaultAll); the fine-grained group names still work for local runs.Project.toml—ExplicitImportsdropped from[extras],[targets].test, and[compat](it now lives only in the QA env).juliacompat floor is already"1.10"(LTS), unchanged.Extension tests remain owned by the separate
TestExtensions.ymlworkflow (untouched), as before; the root matrix does not run theExtensionsgroup. All other workflow files are untouched.Matrix match
compute_affected_sublibraries.jl . --root-matrix(fromSciML/.github@v1) emits exactly the intended canonical matrix:= 5 cells, all
ubuntu-latest. This is a deliberate consolidation of the old 15-cell matrix (5 functional groups × 3 versions): the 5 functional groups collapse into a singleCoregroup (running the union of all functional tests) on the same 3 versions, and the inline ExplicitImports check becomes a dedicatedQAgroup on[lts, 1]. Same test coverage, canonical Core/QA shape. Emitted matrix verified to match the authoredtest_groups.tomlexactly (5/5).QA validation
Catalyst is a heavy package (ModelingToolkitBase / Symbolics / SymbolicUtils precompile, > ~20 min cold), so the QA group was not run locally — CI validates it. The QA structure was statically verified:
test_groups.toml,test/qa/Project.toml, andProject.tomlall parse as TOML;Test.ymlparses as YAML;runtests.jlandqa/qa.jlparse as Julia; the[sources]path../..resolves to the CatalystProject.toml. No tests were skipped, broken, or silenced.Ignore until reviewed by @ChrisRackauckas.
🤖 Generated with Claude Code