Raise sublibrary SciMLBase compat floor to match parent Corleone (fix downgrade CI)#86
Conversation
The lib/* sublibraries depend on the in-tree Corleone, which declares `SciMLBase = "2.141.0, 3"`, but both sublibraries declared the looser `SciMLBase = "2, 3"`. The sublibrary-downgrade CI pins each direct dep to its declared floor, so SciMLBase was pinned to 2.130.0 while the developed in-tree Corleone still required >= 2.141.0, producing an unsatisfiable resolve (`Unsatisfiable requirements detected for package SciMLBase`). Bump both sublibraries' SciMLBase floor to 2.141.0 so their declared floor is honest about what the dependency tree actually supports. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ch parent Corleone After the SciMLBase floor bump, the downgrade-sublibraries resolver surfaced the next two diverging floors: in-tree Corleone requires Reexport >= 1.2.2 and SymbolicIndexingInterface >= 0.3.43, but both sublibraries still declared the looser 1.2 and 0.3. The downgrade workflow pins each direct dep to its declared floor (Reexport=1.2.0, SymbolicIndexingInterface=0.3.39), which the developed in-tree Corleone rejects -> unsatisfiable resolve. Raise both floors to match the parent so the declared floors are honest about what the tree supports. Local verification on Julia 1.10.11 (develop in-tree Corleone, pin the two deps to the sublibrary downgrade floor): - Old floors: RESOLVE_FAIL, reproducing the CI Unsatisfiable Reexport error for both sublibraries. - New floors: RESOLVE_OK for both sublibraries. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…, exports, deps)
The lib/CorleoneOED [QA] group ran Aqua.test_all and failed 6 of 11 checks
(reproduced on Julia 1.10 and 1.12). Each is fixed at its real source:
- Type piracy: `shooting_constraints`/`shooting_constraints!` for
`AbstractVector{<:Trajectory}` dispatched only on Corleone-owned types
(the function and `Trajectory` both belong to Corleone). Moved both vector
methods into the parent Corleone package (src/multiple_shooting.jl) next to
the existing single-`Trajectory` methods, where they are no longer piracy.
Call sites (CorleoneOED's Optimization extension) already use the
`Corleone.`-qualified names, so they are unaffected.
- Method ambiguity: `get_sampling_sums`/`get_sampling_sums!` were ambiguous
for an `OEDLayer{<:Any, false, <:Any, <:MultipleShootingLayer}` with a
NamedTuple state (the `SAMPLED == false` method vs the MultipleShootingLayer
NamedTuple method). Added the intersection methods; with no sampling the
result is empty regardless of the shooting layer, matching the existing
`SAMPLED == false` behaviour.
- Unbound type parameters: four `update_fim` methods declared `where` type
variables that never appeared in the signature (`DISCRETE` in oed.jl, `SPLIT`
in multiexperiments.jl). Dropped the unused parameters.
- Undefined exports: Symbolics (through v7) still lists `Variable` in its
`export`s even though that binding was removed, so `@reexport using Symbolics`
re-exported an undefined name. Reproduce Reexport's behaviour (`using` +
re-export the package's exported names) while skipping names that no longer
resolve in Symbolics. The genuinely-usable Symbolics API (`Num`, `@variables`,
etc.) is still re-exported.
- Stale dependency: `StableRNGs` was a direct `[deps]` entry but is only used in
tests. Moved it to `[extras]` and the test target.
- Missing extras compat: added a `Pkg = "1"` compat entry for the `Pkg` test
extra.
Verified locally: Aqua.test_all(CorleoneOED) passes 11/11 on Julia 1.10 and
1.12; CorleoneOED Core tests pass on 1.12 (1D Example 36/36, Lotka Volterra
125/125, Lotka Volterra SVD 17/17, including the `@inferred get_sampling_sums`
assertions and the multi-experiment optimization that exercises the relocated
`shooting_constraints`); parent Corleone Aqua still passes 11/11.
Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…e MTK floor The lib/OptimalControlBenchmarks downgrade-sublibraries job failed at the SciMLBase floor (fixed earlier in this PR); with that resolved the next downgrade failure surfaced: ModelingToolkit precompilation fails on Julia 1.10 with `UndefVarError: ispublic not defined`. Root cause (two distinct problems, both upstream-floor issues): 1. `ModelingToolkitBase` was a *direct* dependency of OptimalControlBenchmarks but is never imported or used anywhere in its source or tests (only the `[deps]` entry referenced it). Because it was a direct dep with no compat entry, the downgrade workflow pinned it to its floor `1.0.0`. MTK's `@import_mtkbase` macro references `ModelingToolkitBase.ispublic` / `.mergedefaults`, neither of which exists in MTKBase 1.0.0. Removing the spurious direct dep lets MTKBase float (it is still pulled transitively by ModelingToolkit), so the resolver picks the latest compatible MTKBase (1.41) instead of the broken 1.0.0. 2. With MTKBase fixed, the downgrade still pinned ModelingToolkit to its lowest version in `11` (= 11.0.0). MTK 11.0.0 and 11.1.0 call `Base.ispublic` unconditionally, which does not exist on Julia 1.10 (it was added in Julia 1.11; later MTK versions guard the call). MTK 11.2.0 hits a separate `generate_ODENLStepData must be explicitly imported` precompile error on 1.10. Raised the ModelingToolkit compat floor to `11.7.2`, the earliest 11.x verified to precompile and load on Julia 1.10. Local verification (Julia 1.10.11, develop in-tree Corleone, simulate the sublibrary downgrade resolve): - Before: RESOLVE_OK but precompile fails with `UndefVarError: ispublic not defined` (MTK 11.0.0 + MTKBase 1.0.0), reproducing the CI failure. - After: RESOLVE_OK; resolved ModelingToolkit=11.7.2, ModelingToolkitBase=1.41.0; `import OptimalControlBenchmarks` precompiles and loads cleanly. No tests skipped, no tolerances loosened. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ompat) The lib/OptimalControlBenchmarks [QA] group ran Aqua.test_all and failed: - Stale dependencies: `OptimizationLBFGSB`, `Ipopt`, `UnoSolver`, `OrdinaryDiffEq`, and `Runic` were declared in `[deps]` but never loaded by the package. `OptimizationLBFGSB`, `OrdinaryDiffEq`, and `Runic` are not referenced anywhere; `Ipopt`/`UnoSolver` are used only by the loose top-level `main.jl` demo script (not part of the package or its tests). `StableRNGs` was likewise an unused direct dep (it stays available transitively via CairoMakie). Removed all of them from `[deps]`/`[compat]` so the declared deps are honest about what the package actually loads. `main.jl` keeps its `using Ipopt`/`using UnoSolver`; a header comment notes they are demo-only solver back ends to add to the env before running it. - Compat bounds: `BenchmarkTools`, `CairoMakie`, and `IntervalSets` were direct deps without a `[compat]` entry, and the `Pkg` test extra lacked one too (Aqua's extras compat check does not exempt stdlibs). Added `BenchmarkTools = "1"`, `CairoMakie = "0.15"`, `IntervalSets = "0.7"`, and `Pkg = "1"`. The test target (`Pkg`, `SafeTestsets`, `Test`) is unchanged, so the downgrade-sublibraries Core resolve is not perturbed; the added compat floors only raise previously-uncapped lower bounds. Local verification (Julia 1.10.11, develop in-tree Corleone + the sublibrary): - `Aqua.test_all(OptimalControlBenchmarks)` passes all checks (Method ambiguity, Unbound type parameters, Undefined exports, Project extras, Stale dependencies, Compat bounds 4/4, Piracy, Persistent tasks). Before this change it failed on Stale dependencies, then on extras Compat bounds. - Simulated the sublibrary downgrade resolve (pin every direct dep to its declared `[compat]` floor): RESOLVE_OK; `import OptimalControlBenchmarks` precompiles and loads cleanly. No tests skipped, no tolerances loosened. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The manual Symbolics re-export loop works around an undefined `Variable` export (and `@public Unknown`) in Symbolics. Both are fixed upstream in JuliaSymbolics/Symbolics.jl#1906; the comment now records that the loop can be replaced with `@reexport using Symbolics` plus a Symbolics compat-floor bump once that lands. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
The Fixed upstream: JuliaSymbolics/Symbolics.jl#1906 (verified on Julia 1.12 that Once #1906 is released, this loop can be reverted to Please ignore until reviewed by @ChrisRackauckas. |
|
Upstream status re-check (the The loop works around two stale entries in Root-cause fix: JuliaSymbolics/Symbolics.jl#1906 (drops Recommendation: keep the workaround as a stopgap to stay green. The in-code comment already documents the revert path (drop the loop, restore |
Problem
The
downgrade-sublibrariesCI onmainis failing for bothlib/CorleoneOEDandlib/OptimalControlBenchmarkswith:Root cause
Both sublibraries depend on the in-tree
Corleone, which declaresSciMLBase = "2.141.0, 3", but the sublibraries themselves declared the looserSciMLBase = "2, 3". The centralizedsublibrary-downgrade.ymlworkflow pins every direct dependency to its declared floor, so SciMLBase got pinned to its lowest registered>= 2version (2.130.0) while the developed in-treeCorleonestill required>= 2.141.0— an unsatisfiable resolve. The divergence has existed since commitfa6a2b9a2(2026-06-07), where the root's SciMLBase floor was bumped to2.141.0but the sublibraries were left behind.Fix
Raise both sublibraries'
SciMLBasecompat floor to2.141.0, 3to match the parentCorleonepackage. This makes the declared floor honest about what the dependency tree actually supports. No tests are skipped, no tolerances loosened.Local verification
Reproduced the downgrade resolution on Julia 1.10.11 (the version CI uses), developing the in-tree
Corleoneand pinning SciMLBase to the sublibrary's downgrade floor:=2.130.0):RESOLVE_FAIL— reproduces the exact CIUnsatisfiable requirements detected for package SciMLBaseerror, for both sublibraries.=2.141.0):RESOLVE_OK SciMLBase=>2.141.0— resolves cleanly, for both sublibraries.Please ignore until reviewed by @ChrisRackauckas.
Follow-up (Reexport + SymbolicIndexingInterface floors)
After the SciMLBase floor bump landed, the
downgrade-sublibrariesresolver surfaced the next two diverging floors. In-treeCorleonedeclaresReexport = "1.2.2"andSymbolicIndexingInterface = "0.3.43", but both sublibraries still declared the looserReexport = "1.2"andSymbolicIndexingInterface = "0.3". The downgrade workflow pins each direct dep to its declared floor (Reexport=1.2.0,SymbolicIndexingInterface=0.3.39), which the developed in-treeCorleonerejects — the same unsatisfiable-resolve class as the SciMLBase issue. Raised both sublibrary floors to match the parent.Local verification (Julia 1.10.11, develop in-tree
Corleone, pin the two deps to the sublibrary downgrade floor):Reexport=1.2.0,SII=0.3.39):RESOLVE_FAIL— reproduces the CIUnsatisfiable requirements detected for package Reexporterror, for both sublibraries.Reexport=1.2.2,SII=0.3.43):RESOLVE_OK, for both sublibraries.Note: the
[QA]sublibrary checks are red on a separate, pre-existing set of Aqua failures (piracy onshooting_constraints, method ambiguity, unbound type params, undefined exports, staleStableRNGsdep, and a missingPkgcompat entry for the test extras). These reproduce onmain, are unrelated to this PR's compat-floor change, and are out of scope here.QA (Aqua) fixes (now in scope)
The
lib/CorleoneOED [QA]Aqua failures that the original PR body called "out of scope" are now fixed at their real sources (commit4daaea5).Aqua.test_all(CorleoneOED)failed 6 of 11 checks; each fix:shooting_constraints/shooting_constraints!forAbstractVector{<:Trajectory}dispatched only on Corleone-owned types (both the function andTrajectorybelong toCorleone). Moved both vector methods into the parentCorleonepackage (src/multiple_shooting.jl), where they are legitimate extensions, not piracy. The Optimization extension already calls them via theCorleone.-qualified names, so call sites are unchanged.get_sampling_sums/get_sampling_sums!were ambiguous forOEDLayer{<:Any, false, <:Any, <:MultipleShootingLayer}with aNamedTuplestate. Added the intersection methods returning the emptySAMPLED == falseresult.update_fimmethods declaredwherevariables (DISCRETEinoed.jl,SPLITinmultiexperiments.jl) absent from the signature. Removed them.exportsVariabledespite the binding being removed, so@reexport using Symbolicspropagated an undefined export. Replaced it withusing Symbolics+ re-export of only the Symbolics exports that still resolve (this is an upstream Symbolics defect worked around locally; the usable API such asNum/@variablesis still re-exported).StableRNGsfrom[deps]to[extras]/test target (test-only use).Pkg = "1".Local verification (showed output):
Aqua.test_all(CorleoneOED)passes 11/11 on Julia 1.10 and 1.12; CorleoneOED Core tests pass on 1.12 (1D Example 36/36, Lotka Volterra 125/125, Lotka Volterra SVD 17/17, including the@inferred get_sampling_sumschecks and the multi-experiment Ipopt optimization that exercises the relocatedshooting_constraints); parentCorleoneAqua still passes 11/11. No tests skipped, no tolerances loosened.Follow-up (OptimalControlBenchmarks downgrade: ModelingToolkit
ispublicon Julia 1.10)After the SciMLBase / Reexport / SII floor bumps landed, the
lib/CorleoneOEDdowngrade went green butlib/OptimalControlBenchmarkssurfaced the next downgrade failure:ModelingToolkitfails to precompile on Julia 1.10 (the downgrade CI version) withUndefVarError: ispublic not defined(from MTK's@import_mtkbasemacro), which also breaks the in-treeCorleoneModelingToolkitExtension.Root cause — two distinct upstream-floor problems:
ModelingToolkitBasewas declared as a direct dependency of OptimalControlBenchmarks but is never imported or used in its source or tests (only the bare[deps]entry). As a direct dep with no compat entry, the downgrade workflow pinned it to its floor1.0.0. MTK's@import_mtkbasereferencesModelingToolkitBase.ispublic/.mergedefaults, neither of which exists in MTKBase 1.0.0. Removed the spurious direct dep so MTKBase floats (still pulled transitively via MTK) and resolves to the latest compatible version (1.41).11version (11.0.0). MTK 11.0.0/11.1.0 callBase.ispublicunconditionally — it does not exist on Julia 1.10 (added in 1.11; later MTK versions guard it). MTK 11.2.0 hits a separategenerate_ODENLStepData must be explicitly importedprecompile error on 1.10. Raised theModelingToolkitcompat floor to11.7.2, the earliest 11.x verified to precompile and load on Julia 1.10.Local verification (Julia 1.10.11, develop in-tree Corleone, simulate the sublibrary downgrade resolve):
UndefVarError: ispublic not definedon precompile (MTK 11.0.0 + MTKBase 1.0.0) — reproduces the CI failure.RESOLVE_OK; resolvedModelingToolkit=11.7.2,ModelingToolkitBase=1.41.0;import OptimalControlBenchmarksprecompiles and loads cleanly.No tests skipped, no tolerances loosened.
Note: the non-downgrade
sublibraries / ...matrix jobs on this PR are red on a separate centralized-CI infra failure — theDevelop in-repo [sources]step fails withSystemError: opening file ".sciml-dotgithub/scripts/develop_sources.jl": No such file or directory. This is the sharedSciML/.githubmonorepo-sublibrary helper, not anything in this repo; it was addressed upstream by SciML/.github#99 and a re-run should clear it. These jobs areskippedonmain, so they are out of scope for themainreds this PR targets.Status after this push (verified in CI on HEAD)
downgrade-sublibraries / test (lib/CorleoneOED)-> successdowngrade-sublibraries / test (lib/OptimalControlBenchmarks)-> success (the two checks that were red onmain)sublibraries / lib/{CorleoneOED,OptimalControlBenchmarks} / Core(Julia 1/lts/pre) -> successsublibraries / lib/CorleoneOED [QA]-> successtests/...,Downgrade,Runic,Spell Check-> successRemaining red:
sublibraries / lib/OptimalControlBenchmarks [QA](Aqua). These are pre-existing and unrelated to the downgrade fix (they fail identically before this push and areskippedonmain, since thesublibrariesmatrix is PR-only). Aqua flags, for OptimalControlBenchmarks:[deps]:OptimizationLBFGSB,Ipopt,UnoSolver,OrdinaryDiffEq,Runic(none imported bysrc/;Ipopt/UnoSolverare used only by the top-levelmain.jldemo script,OrdinaryDiffEqTsit5rather than fullOrdinaryDiffEqis whatsrc/uses).[compat]forBenchmarkTools,CairoMakie,IntervalSets,OptimizationLBFGSB,Runic,UnoSolver, and thePkgextra.These are left for a focused follow-up: cleanly resolving them means relocating the demo-only deps and choosing compat bounds, which is separable from the downgrade-CI fix this PR targets.
Follow-up (OptimalControlBenchmarks [QA] Aqua fixes — now in scope)
The
sublibraries / lib/OptimalControlBenchmarks [QA]failures the section above called a "focused follow-up" are now fixed at their real sources.Aqua.test_all(OptimalControlBenchmarks)failed on Stale dependencies and (once that was fixed) on extras Compat bounds:OptimizationLBFGSB,Ipopt,UnoSolver,OrdinaryDiffEq,Runic(and the unusedStableRNGs) were declared in[deps]but never loaded by the package.OptimizationLBFGSB/OrdinaryDiffEq/Runicare referenced nowhere;Ipopt/UnoSolverare used only by the loose top-levelmain.jldemo script, which is not part of the package or its tests.StableRNGsstays available transitively via CairoMakie. Removed all of them from[deps]/[compat].main.jlkeeps itsusing Ipopt/using UnoSolverwith a header comment noting these are demo-only solver back ends toPkg.addbefore running it.[compat]entries:BenchmarkTools = "1",CairoMakie = "0.15",IntervalSets = "0.7"(direct deps that had none), andPkg = "1"(Aqua's extras compat check does not exempt stdlibs).The test target (
Pkg,SafeTestsets,Test) is unchanged, so the downgrade-sublibraries Core resolve is not perturbed; the added compat floors only raise previously-uncapped lower bounds.Local verification (Julia 1.10.11, develop in-tree Corleone + the sublibrary, showed output):
Aqua.test_all(OptimalControlBenchmarks)now passes every check (Method ambiguity, Unbound type parameters, Undefined exports, Project extras, Stale dependencies, Compat bounds 4/4, Piracy, Persistent tasks). Before this commit it failed on Stale dependencies, then on the extras Compat bounds.[compat]floor):RESOLVE_OK;import OptimalControlBenchmarksprecompiles and loads cleanly (BenchmarkTools=1.8.0, CairoMakie=0.15.12, IntervalSets=0.7.14, ModelingToolkit=11.7.2, SciMLBase=2.141.0).No tests skipped, no tolerances loosened.
Please ignore until reviewed by @ChrisRackauckas.