Spun out of #2 as the tracked work item.
Problem. baseline_run runs julia --project=<pkg> test/runtests.jl, so test-only dependencies declared in test/Project.toml (or legacy [extras]/[targets]) are missing. On any package with test-specific deps the baseline fails with "test suite failed" even though Pkg.test() passes.
Fix. Resolve the test environment the way Pkg does (the package plus the test deps) once, into the shadow; then run baseline and each mutant against that env. Keep the fast per-mutant --project path — just point it at the resolved test environment.
Acceptance. A package with a non-trivial test/Project.toml (e.g. DataSplits.jl from #2) runs a green baseline and a real campaign. Add a fixture package with a test-only dep to the suite.
Priority: this blocks real-world use; it is the next thing to land.
Spun out of #2 as the tracked work item.
Problem.
baseline_runrunsjulia --project=<pkg> test/runtests.jl, so test-only dependencies declared intest/Project.toml(or legacy[extras]/[targets]) are missing. On any package with test-specific deps the baseline fails with "test suite failed" even thoughPkg.test()passes.Fix. Resolve the test environment the way Pkg does (the package plus the test deps) once, into the shadow; then run baseline and each mutant against that env. Keep the fast per-mutant
--projectpath — just point it at the resolved test environment.Acceptance. A package with a non-trivial
test/Project.toml(e.g. DataSplits.jl from #2) runs a green baseline and a real campaign. Add a fixture package with a test-only dep to the suite.Priority: this blocks real-world use; it is the next thing to land.