Skip to content

Hide submissions with any failed secret run#492

Merged
msaroufim merged 1 commit into
mainfrom
fix-secret-fail-leaderboard-visibility
Jun 17, 2026
Merged

Hide submissions with any failed secret run#492
msaroufim merged 1 commit into
mainfrom
fix-secret-fail-leaderboard-visibility

Conversation

@msaroufim

Copy link
Copy Markdown
Member

Summary

  • hide public leaderboard scores when the same submission has any failed secret run for that runner
  • add a regression for the QR failure mode: public leaderboard passes, secret benchmark fails

Why

PR #490 only filtered failed secret runs with the same mode as the public scored run. QR submissions can fail secret benchmark before producing a secret leaderboard run, while still having a passing public leaderboard score. Those should not appear on rankings.

Validation

  • uv run --extra dev pytest -q tests/test_leaderboard_db.py -k 'failed_secret or leaderboard_submission_ranked or leaderboard_submission_count'
  • uv run --extra dev python -m py_compile src/libkernelbot/leaderboard_db.py
  • git diff --check
  • verified the updated SQL excludes prod submissions 800895, 800582, and 800601

@github-actions

Copy link
Copy Markdown

Coverage report

Click to see where and how coverage changed

FileStatementsMissingCoverageCoverage
(new stmts)
Lines missing
  src/libkernelbot
  utils.py
Project Total  

This report was generated by python-coverage-comment-action

@msaroufim msaroufim merged commit 972d019 into main Jun 17, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant