Runner refactoring 6/6: Add multi-seed benchmarking support

[eantonini]

This PR adds support for multi-seed solver runs.

• Each solver configuration currently hardcodes a single fixed seed, chosen only for reproducibility. Solvers have documented "performance variability," where the same problem can solve in noticeably different times depending only on the seed. A benchmark run under one arbitrary seed can therefore look faster or slower than typical purely by chance.
• runner.benchmark's new -n/--num-seeds N flag (internally orchestrator.run_benchmark's num_seeds, replacing the never-CLI-exposed iterations) runs each (problem, solver configuration) pair N times, each under a different seed. Seeds are generated as 1, 2, 3, ... (not 0-based) — see below.
• New seed_options map in runner/config/solvers.yaml records which options key holds each solver package's seed; solver.get_solver()/execution.run_solver() gained an optional seed override, threaded through as a new --seed N flag on python -m runner.utils.solver (solving runs as a subprocess, so this crosses a process boundary, not just a function argument).
• The seed used is now recorded in a new Seed column in results/benchmark_results.csv and the mean/stddev summary CSV.
• Harmonized every solver's default seed to 0, except CBC. Verified each solver's own option documentation: 0 is an ordinary, deterministic value for GLPK, SCIP, HiGHS, Gurobi, CPLEX, Knitro, Xpress, and Mosek. CBC is the sole confirmed exception: its randomCbcSeed option treats 0 as "use the time of day," which would make that run silently non-deterministic. CBC stays at 1. The multi-seed loop generated seeds start at 1 for every solver.
• Backward compatible: every new parameter is optional and defaults to today's exact behavior (num_seeds=1 never overrides the seed). Since results/benchmark_results.csv is a real, growing file that gets appended to across runs, adding the Seed column required a general schema-migration safety net (results.ensure_csv_schema): appending to a CSV written by older code now widens it in place to include new columns (old rows get a blank cell), rather than producing a ragged file pd.read_csv can't parse.
• Docs updated: runner/README.md (the -n/--num-seeds flag, the --seed flag on runner.utils.solver, usage examples) and runner/SOLVERS.md ("Adding a New Solver" now covers the seed_options entry too).

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
solver-benchmark	Ready	Preview	Sep 1, 2026 2:04pm UTC