v0.19: TrainedVerifier (RouterArena 0.7358 snapshot)#61
Merged
Conversation
added 2 commits
May 28, 2026 21:51
- TrainedVerifier loads NadirRouter/cascade-verifier-v1 from
HuggingFace (or a local HF cache). Same interface shape as
HeuristicVerifier — score(prompt, cheap_answer, expect_json=...)
returns a TrainedScore with .score / .accepted / .threshold /
.reasons / .to_dict().
- New n2_trained profile uses the trained verifier; n2_default
stays on the heuristic so users who do not want the transformer
stack pay nothing for it.
- CascadeConfig schema: new `verifier` and `verifier_model` fields.
Validated against {"heuristic", "trained"} so typos fail fast.
Defaults preserve v0.18 behaviour.
- NTierCascade auto-instantiates TrainedVerifier when the loaded
profile specifies verifier: trained. Lazy import keeps the
heuristic-only path free of transformers/torch.
- Optional install: pip install nadirclaw[trained] pulls
transformers>=4.40 and torch>=2.0.
- README: new "Trained verifier" section explains install,
activation (NADIRCLAW_TIERS_PROFILE=n2_trained), and what is and
is not released (frozen weights MIT; training pipeline and
adaptive retraining remain Pro-only).
- 9 new tests; full suite 773/773 passing.
This is the frozen snapshot used in RouterArena PR #112
(arena_F 0.7358). Closes the 14-day reproducibility commitment in
RouteWorks/RouterArena#112
Training pipeline and adaptive retraining loop remain proprietary
to Nadir Pro; only the frozen weights are released.
Repo: https://github.com/NadirRouter/NadirClaw
Service: https://getnadir.com
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Ships the trained DeBERTa-v3-small cross-encoder verifier as an MIT-licensed snapshot, fulfilling the public reproducibility commitment made in RouterArena PR #112:
(Numbered v0.19 because NadirClaw shipped 0.14 through 0.18 in the interim; same content, same intent.)
What lands
Weights release
The HuggingFace upload to `NadirRouter/cascade-verifier-v1` is pending — the local environment running this PR did not have HuggingFace auth, so I am landing the code path now and will push the weights manually right after. The local weights are at `getnadir.dev/verifier/weights/best/` (DeBERTa-v3-small, 6 layers, INT8 quantised `verifier_int8.pt` plus full FP32 `model.safetensors` and tokenizer).
Once the HF repo is live the `TrainedVerifier` constructor reaches it transparently — no code change needed.
Backward compatibility
What's NOT released
Per the RouterArena commitment we are releasing the frozen snapshot, not the pipeline:
Test plan
Repo: https://github.com/NadirRouter/NadirClaw
Service: https://getnadir.com