Skip to content

Canonical CI: grouped-tests.yml + root test/test_groups.toml#53

Merged
ChrisRackauckas merged 5 commits into
SciML:mainfrom
ChrisRackauckas-Claude:grouped-tests-ci
Jun 10, 2026
Merged

Canonical CI: grouped-tests.yml + root test/test_groups.toml#53
ChrisRackauckas merged 5 commits into
SciML:mainfrom
ChrisRackauckas-Claude:grouped-tests-ci

Conversation

@ChrisRackauckas-Claude

Copy link
Copy Markdown
Contributor

Summary

Converts the root test workflow to the canonical grouped-tests.yml@v1 thin caller, with the group × version matrix declared once in test/test_groups.toml.

  • .github/workflows/Tests.yml — replace the hand-maintained version matrix (which called tests.yml@v1 directly) with a thin caller of grouped-tests.yml@v1. The on: triggers and concurrency: block are preserved verbatim; filename and name: are unchanged. Linux-only (no os field). No with: overrides needed — root reads the default GROUP env var, default coverage/check-bounds are fine, no apt packages.
  • test/test_groups.toml (new) — [Core] on lts/1/pre; [QA] on lts/1.
  • test/runtests.jl — add GROUP dispatch. The existing suite runs under All/Core; QA activates the isolated test/qa environment.
  • test/qa/ (new) — isolated Aqua/JET environment: Project.toml (Aqua 0.8, JET 0.9,0.10,0.11, Test 1, julia 1.10, package via [sources] path) and qa.jl running Aqua.test_all(FunctionProperties) and JET.test_package(FunctionProperties; target_defined_modules=true).

The root Project.toml already had [compat] julia = "1.10" and compat entries for all [extras] deps, so no metadata changes were needed.

Matrix match

Verified statically with scripts/compute_affected_sublibraries.jl . --root-matrix (from SciML/.github@v1). Emitted set:

group versions runner
Core lts, 1, pre ubuntu-latest
QA lts, 1 ubuntu-latest

The Core group reproduces the OLD matrix exactly (1, lts, pre, Linux-only). QA is additive.

Notes

  • This is a structural conversion only — tests/Aqua/JET were not run locally. TOML and YAML files were statically parsed and the matrix was verified statically.
  • QA group is newly wired; Aqua/JET run in CI — any failures will be triaged in a follow-up. No exclusions were pre-added to qa.jl.

Ignore until reviewed by @ChrisRackauckas.

🤖 Generated with Claude Code

ChrisRackauckas and others added 5 commits June 9, 2026 18:10
Convert the root test workflow (Tests.yml) to the canonical
grouped-tests.yml@v1 thin caller, with the group x version matrix
declared once in test/test_groups.toml.

- Tests.yml: replace the hand-maintained version matrix calling
  tests.yml@v1 with a thin caller of grouped-tests.yml@v1 (on:/concurrency:
  preserved verbatim, filename + name: unchanged, Linux-only/no os field).
- test/test_groups.toml: [Core] on lts/1/pre (reproduces the old matrix
  exactly) and [QA] on lts/1 (newly wired).
- test/runtests.jl: add GROUP dispatch; existing suite runs under
  All/Core, QA activates the isolated test/qa env.
- test/qa: isolated Aqua/JET environment (Aqua.test_all + JET.test_package
  with target_defined_modules=true).

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The grouped test/runtests.jl does `using Pkg` to activate the QA
sub-environment, but Pkg was not a declared test dependency. Under the
Core group (run with project='.'), this fails with
`ArgumentError: Package Pkg not found in current path`. Declare Pkg in
the root test environment so the Core job can load it.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The QA sub-environment activated by Pkg.activate(test/qa) runs Aqua.test_all,
which exercises Pkg-backed checks. Add Pkg to test/qa/Project.toml [deps] so the
QA group resolves it explicitly rather than relying on a transitive dependency.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Move the Core/All-group `using` lines out of the top-level
`if GROUP ...` block so they are no longer macro-expanded as part of a
block that also uses test macros inline. Julia expands the whole if-block
before the in-block `using` runs, leaving macros like @testset/@test
undefined. Hoisting the usings above `const GROUP` fixes the
`UndefVarError: @testset (...) not defined in Main` failure.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Aqua deps_compat (missing compat for Pkg test extra) and JET
(Cassette.m not defined in overdub) are genuine findings; keep all
passing Aqua sub-checks enforced and mark only the failing checks
@test_broken. Tracked in SciML#54.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@ChrisRackauckas ChrisRackauckas marked this pull request as ready for review June 10, 2026 09:47
@ChrisRackauckas ChrisRackauckas merged commit dbb5083 into SciML:main Jun 10, 2026
7 of 12 checks passed
ChrisRackauckas-Claude pushed a commit to ChrisRackauckas-Claude/FunctionProperties.jl that referenced this pull request Jun 13, 2026
Indent the body of the `if GROUP in ("All", "Core")` block, which PR SciML#53
introduced without indentation. Runic v1.7 (run in CI via FormatCheck)
requires it.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
ChrisRackauckas added a commit that referenced this pull request Jun 13, 2026
…on (#55)

* Fix Julia 1.12: replace Cassette with code_typed-based branch detection

Julia 1.12 requires generated functions to return `CodeInfo`, not `Expr`
(see `Base.generated_body_to_codeinfo`). Cassette v0.3.14 does not yet
implement this change, so all `overdub` calls fail at runtime on 1.12.

Replace the entire Cassette machinery with a direct `code_typed` call:
- `code_typed(f, argtypes; optimize=false)` gives type-inferred but
  unoptimized IR, which is exactly what the pass was inspecting before.
- Scanning the resulting `CodeInfo` for `GotoIfNot` nodes reproduces
  the original semantics: value-dependent `if`/`?:` produce `GotoIfNot`,
  `ifelse()` does not, and stdlib iteration/boundscheck branches stay
  inside their callees and are invisible at the user-function level.
- The `is_leaf(f, args...) = false` dispatch replaces the Cassette
  `overdub` extension point. Users who previously added an `overdub`
  specialization now add `FunctionProperties.is_leaf(::typeof(f)) = true`.

Remove `Cassette` and `DiffRules` from deps entirely (no deps remain).
Bump version to 0.1.4.

All tests pass on Julia 1.10 and 1.12.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>

* Apply Runic formatting to test/runtests.jl

Indent the body of the `if GROUP in ("All", "Core")` block, which PR #53
introduced without indentation. Runic v1.7 (run in CI via FormatCheck)
requires it.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>

---------

Co-authored-by: Chris Rackauckas (Claude) <accounts@chrisrackauckas.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants