Context
steerbench renders a report card for one vector at a time. When comparing several vectors (across models or concepts, as in the README's cross-model sweep) there is no single view that ranks them side by side. A leaderboard that consumes several already-computed cards into one comparison table makes the "which vector steers best without breaking things" question answerable at a glance.
Proposal
Add a function that takes N sets of the existing sweep CSVs (each already analysed via analyze_dose / analyze_layers) and emits one ranked comparison table: per vector, the sweet-spot effect shift (mean +/- std), the coherence-cliff onset, the best coherent layer depth-fraction, and the worst side-effect delta. Reuse the existing per-seed aggregation; do not re-run any model.
Acceptance criteria
- Add a typed input describing one entry (label plus its three CSV paths) and a
build_leaderboard(entries) -> ... that returns a typed, ordered comparison result. A pydantic model is appropriate here since this is a genuinely new config/result surface; keep numpy out of the public type.
- Ranking key documented (e.g. by sweet-spot standardized effect, ties broken by smaller side-effect degradation).
- Stochastic metric rule: each cell derived from >= 3 seeds, reported as mean +/- std, with seeds/hardware/wall-clock carried in each entry's provenance.
- Render the leaderboard as a markdown + HTML table; a
--leaderboard path in the CLI is a nice-to-have, not required for this issue.
- CPU-only unit test building two tiny CSV triples and asserting the ranked order and the reported cells. No model download.
ruff check ., ruff format --check ., mypy src, pytest -q all pass.
Context
steerbench renders a report card for one vector at a time. When comparing several vectors (across models or concepts, as in the README's cross-model sweep) there is no single view that ranks them side by side. A leaderboard that consumes several already-computed cards into one comparison table makes the "which vector steers best without breaking things" question answerable at a glance.
Proposal
Add a function that takes N sets of the existing sweep CSVs (each already analysed via
analyze_dose/analyze_layers) and emits one ranked comparison table: per vector, the sweet-spot effect shift (mean +/- std), the coherence-cliff onset, the best coherent layer depth-fraction, and the worst side-effect delta. Reuse the existing per-seed aggregation; do not re-run any model.Acceptance criteria
build_leaderboard(entries) -> ...that returns a typed, ordered comparison result. A pydantic model is appropriate here since this is a genuinely new config/result surface; keep numpy out of the public type.--leaderboardpath in the CLI is a nice-to-have, not required for this issue.ruff check .,ruff format --check .,mypy src,pytest -qall pass.