Skip to content

Use dataset revision without review metadata - #3

Merged
ma7dev merged 2 commits into
mainfrom
codex/metadata-free-release
Sep 27, 2026
Merged

ma7dev merged 2 commits into
mainfrom
codex/metadata-free-release

Conversation

@ma7dev

@ma7dev ma7dev commented Sep 27, 2026

Copy link
Copy Markdown
Member

Summary

  • Pin the current public metadata-free Najd Benchmark revision and certified case-file hash.
  • Document that review annotations are absent and are not required for scoring.
  • Test eligibility without a review-status field; retain the exact 104 ArabicMMLU case IDs and scoring rules.

Validation

  • 16 tests pass.
  • CLI accepts the new case-file hash and reports the new revision.
  • Identical frozen responses scored against old and new case files produce identical results (104/104 correct in the answer-key fixture; this is a software check, not model performance).

Stacked on benchmark #2. Related datasets change migrates the three public Hugging Face datasets. No semantic-review claim or change to structural quarantine eligibility.

@ma7dev
ma7dev changed the base branch from codex/evaluation-contract to main September 27, 2026 12:56
@ma7dev
ma7dev merged commit ec62d97 into main Sep 27, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant