Skip to content

Fix QA test group: pass qa body via dedicated run_tests kwarg#622

Merged
ChrisRackauckas merged 2 commits into
SciML:masterfrom
ChrisRackauckas-Claude:fix-qa-runtests-dedicated-kwarg
Jun 17, 2026
Merged

Fix QA test group: pass qa body via dedicated run_tests kwarg#622
ChrisRackauckas merged 2 commits into
SciML:masterfrom
ChrisRackauckas-Claude:fix-qa-runtests-dedicated-kwarg

Conversation

@ChrisRackauckas-Claude

Copy link
Copy Markdown
Contributor

Problem

Both tests / QA jobs (julia 1 and julia lts) fail on master at test startup with:

ERROR: LoadError: ArgumentError: run_tests: GROUP="QA" was requested but no `qa` body was provided

Root cause

SciMLTesting 1.2's run_tests routes the reserved "QA" group through a dedicated qa= keyword argument, not through the groups Dict. In run_tests, the elseif group == "QA" branch is evaluated before groups is consulted, and it throws when qa === nothing:

elseif group == "QA"
    qa === nothing &&
        throw(ArgumentError("run_tests: GROUP=\"QA\" was requested but no `qa` body was provided"))
    _run_group_spec(_group_spec(qa), parent; label = "QA")

test/runtests.jl declared the QA spec inside groups = Dict("QA" => ...) and left qa unset, so the dispatch errored before it ever reached the groups table. This was introduced in the v1.2 folder-model conversion (#619 normalized the group into canonical QA, but the wiring was still via groups).

The reusable CI workflow (grouped-tests.yml@v1) reads test/test_groups.toml, which correctly declares the QA group on lts/1, and dispatches GROUP=QA to Pkg.test. The workflow and test_groups.toml are correct; only runtests.jl was miswired.

Fix

Move the QA spec from groups = Dict("QA" => (; env, body)) to the dedicated qa = (; env, body) keyword, matching the SciMLTesting v1.2 explicit-args API. No other change.

Verification (local)

Against SciMLTesting 1.2.0 on Julia 1.10:

  • Old shape reproduces the CI error: passing QA in groups with GROUP=QA throws the exact ArgumentError: GROUP="QA" was requested but no qa body was provided.
  • New shape works: with qa = (; ...), GROUP=QA reaches the QA body (new QA body reached), and GROUP=Core still reaches the core body — no error.
  • Runic.format_string confirms test/runtests.jl is already correctly formatted (FormatCheck clean).

Please ignore until reviewed by @ChrisRackauckas.

ChrisRackauckas and others added 2 commits June 15, 2026 10:15
SciMLTesting 1.2's `run_tests` dispatches the reserved `"QA"` group through a
dedicated `qa=` keyword, not via the `groups` Dict. The `GROUP == "QA"` branch
fires before the `groups` table is consulted and errors with
`ArgumentError: GROUP="QA" was requested but no qa body was provided` whenever
`qa === nothing`. test/runtests.jl declared QA inside `groups`, so both QA CI
jobs (julia 1, julia lts) failed at test startup.

Move the QA spec from `groups = Dict("QA" => ...)` to the `qa = (; env, body)`
keyword, matching the SciMLTesting v1.2 explicit-args API.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The autoregulation `@example` block in
docs/src/libs/datadrivensparse/example_04.md failed the docs build with
`UndefVarError: @mtkmodel not defined`. ModelingToolkit v10/v11 removed the
`@mtkmodel` macro entirely (it is no longer defined or exported; the
example had already been partially ported, using the v10 `@mtkcompile`).

Replace the `@mtkmodel`/`@mtkcompile` model definition with the MTK v11
direct `System` construction (`@parameters`/`@variables`/`System(eqs, t)` +
`mtkcompile`), which yields the same `sys.x` array variables the rest of
the example consumes. Also drop the now-unused `NoSpecialize` import and
the `ODEProblem{true, NoSpecialize}` type parameters (plain
`ODEProblem(sys, [], tspan)` is the v11 form).

Verified locally on Julia 1.12 (ModelingToolkit v11.26.8): a focused
Documenter build of the example_04 page runs every `@example` block with
no `:example_block` errors (makedocs completes with VERIFY_MAKEDOCS_OK).

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@ChrisRackauckas-Claude

Copy link
Copy Markdown
Contributor Author

Additional fix on this branch: docs build (@mtkmodel removed in MTK v11)

Besides the QA-group wiring, the Documentation job was red on a separate genuine bug. The @example block in docs/src/libs/datadrivensparse/example_04.md (autoregulation) failed with:

UndefVarError: `@mtkmodel` not defined in `Main.__atexample__named__example_04`

Root cause

ModelingToolkit v10/v11 removed the @mtkmodel macro entirely — it is no longer defined or exported (isdefined(ModelingToolkit, Symbol("@mtkmodel")) == false on MTK v11.26.8; only @mtkbuild/@mtkcompile/@mtkcomplete remain). The example had been partially ported (it already used @mtkcompile) but still defined the model with @mtkmodel, so the docs build broke. (The triage hint that this was a "missing using ModelingToolkit" / same-block macro issue was incorrect — using ModelingToolkit was already present; the macro simply does not exist anymore.)

Fix

Replace the @mtkmodel/@mtkcompile model with the MTK v11 direct System construction:

@parameters α = 1.0 β = 1.3 γ = 2.0 δ = 0.5
@variables (x(t))[1:2] = [20.0, 12.0]
eqs = [
    D(x[1]) ~ α / (1 + x[2]) - β * x[1],
    D(x[2]) ~ γ / (1 + x[1]) - δ * x[2],
]
@named autoregulation = System(eqs, t)
sys = mtkcompile(autoregulation)
de_problem = ODEProblem(sys, [], tspan)

This yields the same sys.x array variables the rest of the example consumes. Also dropped the unused NoSpecialize import and the ODEProblem{true, NoSpecialize} type params (plain ODEProblem(sys, [], tspan) is the v11 form).

Verification (local, Julia 1.12 / ModelingToolkit v11.26.8)

A focused Documenter build of the example_04 page (with :example_block NOT in warnonly, so any block failure errors the build) ran every @example block with zero failed to run warnings and makedocs completed successfully (VERIFY_MAKEDOCS_OK, exit 0).

Please ignore until reviewed by @ChrisRackauckas.

@ChrisRackauckas ChrisRackauckas marked this pull request as ready for review June 17, 2026 10:44
@ChrisRackauckas ChrisRackauckas merged commit 821472d into SciML:master Jun 17, 2026
17 of 20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants