Skip to content

Add public benchmark entry point and reusable evaluation contracts - #3

Merged
stevezkw1998 merged 7 commits into
mainfrom
codex/multilingual-readme
Aug 21, 2026
Merged

Add public benchmark entry point and reusable evaluation contracts#3
stevezkw1998 merged 7 commits into
mainfrom
codex/multilingual-readme

Conversation

@stevezkw1998

@stevezkw1998 stevezkw1998 commented Aug 20, 2026

Copy link
Copy Markdown
Owner

Summary

  • add a portable neorepro benchmark predictions.csv one-file entry point
  • report leakage-filtered common support, pooled metrics, tie-aware patient Top-K metrics, patient bootstrap intervals, paired differences, random ranking and available strata
  • add machine-readable JSON and Markdown outputs, a two-predictor synthetic example and public contract documentation
  • retain the standalone frozen-benchmark audit, reusable dataset/predictor contracts, RCC extension and multilingual README synchronization
  • merge current v0.1.1 author and Zenodo DOI metadata from main

Validation

  • ruff check src tests
  • pytest -q — 54 passed
  • clean Python 3.11 wheel install and CLI smoke test
  • all seven localized READMEs synchronized

The PR remains a draft pending final CI confirmation.

@stevezkw1998 stevezkw1998 changed the title Add multilingual README and invite external review Add public benchmark entry point and reusable evaluation contracts Aug 21, 2026
@stevezkw1998
stevezkw1998 marked this pull request as ready for review August 21, 2026 08:47
@stevezkw1998
stevezkw1998 merged commit 94a815c into main Aug 21, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant