feat(routing): model-tier routing, fleet parallelism, context handoff (RFC 007 A + RFC 012 A) - #36
Merged
Merged
Conversation
… (RFC 007 A, RFC 012 A) Lands RFC 007 Phase A and a new RFC 012 Phase A. Together they answer three questions for every dispatch: which model, how hard to think, and when to hand off. Model tiers (RFC 007). Three tiers named after task complexity — lite / coding / reasoning — bound to concrete model ids per host in `.nightly/config.yml`. Resolution order is plan frontmatter (`model_tier:`) → specialist role default → per-host binding, with a host that has no binding falling through to its CLI's default model rather than hard-failing. The role defaults amend the RFC's original table. It paired role with seniority; the axis that matters is how expensive it is to be wrong and not notice: - reviewer moves coding → reasoning. Review IS result validation, and it is the one role nothing downstream re-checks — lint, types, and tests have all already passed by the time it runs. - researcher moves reasoning → lite. Despite the name it does file search and summarization over code already on disk: high-volume reading, low-stakes synthesis. - implementer and tester stay at coding; `nightly verify` catches what a cheap model gets wrong. Reasoning effort is a new per-tier dial (lite/coding = low, reasoning = xhigh). Model choice alone under-delivers on cost: a fast model at high effort spends the savings on preamble and exploratory tool calls. Effort is injected as prompt text, not a CLI flag, so it works on every host and no-ops where unsupported. Parallelism + context handoff (RFC 012). A global specialist cap, a worktree cap, and per-tier caps (lite 8 / coding 6 / reasoning 2) — the gradient is the cost control that lets the cheap tiers run wide without the bill scaling with fan-out. Context thresholds are ratios of the *dispatched model's* window, not absolutes, so one setting governs a mixed fleet: at 0.25/0.50 a 1M-context model hands off at 250K and hard-stops at 500K while a 200K model does the same at 50K and 100K. Soft = finish the task, summarize, relaunch fresh with the same goals. Hard = stop now; past halfway, one more step risks truncating a write, which loses work rather than wasting tokens. Also fixes a latent bug found in passing: cli.py and doctor.py each held a private copy of the config template and they had drifted — doctor's had lost the `vault:` and `worktree:` blocks entirely, so a repo repaired by `doctor --fix` got a materially different config than a freshly initialized one. Both now import one `DEFAULT_CONFIG_YML`, with a parity test pinning it. `nightly doctor` gains an advisory check for hosts with no tier binding (pulled forward from Phase C), since inert routing matters more now that the reviewer tier changed. 1162 tests pass (54 new); ruff, pyrefly, and `nightly verify` clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Contributor
📝 WalkthroughWalkthroughThe change adds model-tier routing, per-tier reasoning effort and parallelism configuration, model-aware context handoff thresholds, shared configuration templates, doctor diagnostics, public exports, RFC updates, and comprehensive tests. ChangesRouting and context-handoff foundation
Estimated code review effort: 4 (Complex) | ~60 minutes Sequence Diagram(s)sequenceDiagram
participant PlanRecord
participant Routing
participant Config
participant Context
PlanRecord->>Routing: provide optional model_tier
Config->>Routing: provide role defaults and host bindings
Routing->>Routing: resolve tier, model, and reasoning effort
Routing->>Context: pass resolved model
Context->>Context: calculate soft and hard token thresholds
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.planning/rfcs/007-model-tier-routing.md:
- Around line 43-54: Update decision `#3`’s model_tiers YAML sample in
.planning/rfcs/007-model-tier-routing.md to avoid presenting codex, gemini,
antigravity, and cursor ids as shipped defaults, either by removing those
bindings or clearly marking them operator-supplied/illustrative. Also update the
Context bullets in .planning/rfcs/007-model-tier-routing.md lines 235-269 so
reviewer maps to the reasoning tier and researcher maps to the lite tier,
matching decision `#4`.
In `@packages/nightly-core/src/nightly_core/doctor.py`:
- Around line 184-226: Update _check_model_tiers to treat a host as unbound when
its configured tier map is missing any entry from MODEL_TIERS, not only when the
map is empty; report such hosts in the warning path. Expand the successful
detail to include bindings for every configured host rather than sampling
sorted(hosts)[:1], and add a matching test covering a host bound only for
coding.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: fc120c46-b5ba-4ebb-9d5e-e4fdd0e18a87
📒 Files selected for processing (12)
.planning/rfcs/007-model-tier-routing.md.planning/rfcs/012-fleet-parallelism-and-context-handoff.mdpackages/nightly-core/src/nightly_core/__init__.pypackages/nightly-core/src/nightly_core/cli.pypackages/nightly-core/src/nightly_core/config.pypackages/nightly-core/src/nightly_core/contract.pypackages/nightly-core/src/nightly_core/doctor.pypackages/nightly-core/src/nightly_core/plans.pypackages/nightly-core/src/nightly_core/routing.pypackages/nightly-core/src/nightly_core/specialists.pypackages/nightly-core/tests/test_config_template.pypackages/nightly-core/tests/test_routing.py
Comment on lines
+43
to
+54
| | Tier | Claude | Notes | | ||
| |------|--------|-------| | ||
| | Lite | `claude-haiku-4-5` (200K ctx) | file search, summarization, docs, narrative | | ||
| | Coding | `claude-sonnet-5` (1M ctx) | implementation + test authoring | | ||
| | Reasoning | `claude-opus-5` (1M ctx) | orchestration, result validation, merge adjudication | | ||
|
|
||
| *(Amended 2026-07-28. The original table listed Opus 4.7 / Sonnet 4.6 / | ||
| Haiku 4.5 alongside speculative OpenAI and Google rows. Only the Claude | ||
| column ships as a default: those are ids Nightly can state | ||
| authoritatively. Other vendors' ids are operator-supplied — the schema | ||
| accepts any string, and an unbound host falls through to its CLI's own | ||
| default model rather than hard-failing on a guessed id.)* |
Contributor
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
RFC 007's 2026-07-28 amendment didn't propagate to two other sections of the same doc.
The amendment updated the tier-mapping table's Claude column and decision #4's role defaults, but left two other sections describing the pre-amendment state, now self-contradictory within the same document:
.planning/rfcs/007-model-tier-routing.md#L43-L54: the amendment note states only the Claude column ships as a default and other vendor ids are operator-supplied, but decision#3's samplemodel_tiers:yaml (further down, unchanged) still shows concrete default-looking bindings forcodex/gemini/antigravity/cursor— update that sample (or annotate it as illustrative/operator-supplied) to match..planning/rfcs/007-model-tier-routing.md#L235-L269: decision#4swaps reviewer→reasoningand researcher→lite, but the unchanged Context section still says reviewer routes to "coding tier" and researcher to "reasoning tier" — the exact pre-amendment mapping decision#4says was wrong. Update those bullets to match the new defaults.
📍 Affects 1 file
.planning/rfcs/007-model-tier-routing.md#L43-L54(this comment).planning/rfcs/007-model-tier-routing.md#L235-L269
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.planning/rfcs/007-model-tier-routing.md around lines 43 - 54, Update
decision `#3`’s model_tiers YAML sample in
.planning/rfcs/007-model-tier-routing.md to avoid presenting codex, gemini,
antigravity, and cursor ids as shipped defaults, either by removing those
bindings or clearly marking them operator-supplied/illustrative. Also update the
Context bullets in .planning/rfcs/007-model-tier-routing.md lines 235-269 so
reviewer maps to the reasoning tier and researcher maps to the lite tier,
matching decision `#4`.
Comment on lines
+184
to
+226
| def _check_model_tiers(root: Path) -> DoctorCheck: | ||
| """Warn when the installed hosts have no `model_tiers:` binding — RFC 007. | ||
|
|
||
| Advisory, never repaired. A config predating RFC 007 still works: every | ||
| dispatch falls through to the host CLI's own default model, exactly as | ||
| before. But it also means tier routing is silently inert, which is | ||
| worth saying out loud rather than letting the operator assume their | ||
| `lite` researcher is actually running on a lite model. | ||
| """ | ||
| from nightly_core.config import load_model_tier_config # noqa: PLC0415 | ||
|
|
||
| cfg = load_model_tier_config(root) | ||
| if not cfg.enabled: | ||
| return DoctorCheck( | ||
| name="model_tiers", | ||
| description="model-tier routing", | ||
| status="skipped", | ||
| detail="disabled via model_tiers.enabled: false", | ||
| ) | ||
|
|
||
| hosts = _configured_hosts(root) | ||
| unbound = sorted(h for h in hosts if not cfg.models.get(h)) | ||
| if not unbound: | ||
| bound = ", ".join( | ||
| f"{tier}={cfg.binding(host, tier).model}" | ||
| for host in sorted(hosts)[:1] | ||
| for tier in MODEL_TIERS | ||
| ) | ||
| return DoctorCheck( | ||
| name="model_tiers", | ||
| description="model-tier routing", | ||
| status="ok", | ||
| detail=bound, | ||
| ) | ||
| return DoctorCheck( | ||
| name="model_tiers", | ||
| description="model-tier routing", | ||
| status="warning", | ||
| detail=( | ||
| f"no tier→model binding for: {', '.join(unbound)} " | ||
| "(dispatches use the host CLI's default model)" | ||
| ), | ||
| ) |
Contributor
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Partial per-host tier bindings are reported as "ok."
unbound = sorted(h for h in hosts if not cfg.models.get(h)) only catches a host with zero tier entries. A host with a non-empty but incomplete map (e.g. an operator declares only coding: for gemini) has a truthy dict and is folded into the "ok" branch, even though its lite/reasoning dispatches still silently fall through to the CLI default — the exact silent-inert scenario this check exists to surface per its own docstring.
Separately, the "ok" detail string only samples sorted(hosts)[:1], so with multiple configured hosts only the alphabetically-first one's bindings are shown.
🐛 Proposed fix for the partial-binding gap
hosts = _configured_hosts(root)
- unbound = sorted(h for h in hosts if not cfg.models.get(h))
+ unbound = sorted(
+ h for h in hosts if any(cfg.binding(h, tier).model is None for tier in MODEL_TIERS)
+ )Worth a matching test alongside the existing _check_model_tiers cases (e.g. gemini bound only for coding).
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| def _check_model_tiers(root: Path) -> DoctorCheck: | |
| """Warn when the installed hosts have no `model_tiers:` binding — RFC 007. | |
| Advisory, never repaired. A config predating RFC 007 still works: every | |
| dispatch falls through to the host CLI's own default model, exactly as | |
| before. But it also means tier routing is silently inert, which is | |
| worth saying out loud rather than letting the operator assume their | |
| `lite` researcher is actually running on a lite model. | |
| """ | |
| from nightly_core.config import load_model_tier_config # noqa: PLC0415 | |
| cfg = load_model_tier_config(root) | |
| if not cfg.enabled: | |
| return DoctorCheck( | |
| name="model_tiers", | |
| description="model-tier routing", | |
| status="skipped", | |
| detail="disabled via model_tiers.enabled: false", | |
| ) | |
| hosts = _configured_hosts(root) | |
| unbound = sorted(h for h in hosts if not cfg.models.get(h)) | |
| if not unbound: | |
| bound = ", ".join( | |
| f"{tier}={cfg.binding(host, tier).model}" | |
| for host in sorted(hosts)[:1] | |
| for tier in MODEL_TIERS | |
| ) | |
| return DoctorCheck( | |
| name="model_tiers", | |
| description="model-tier routing", | |
| status="ok", | |
| detail=bound, | |
| ) | |
| return DoctorCheck( | |
| name="model_tiers", | |
| description="model-tier routing", | |
| status="warning", | |
| detail=( | |
| f"no tier→model binding for: {', '.join(unbound)} " | |
| "(dispatches use the host CLI's default model)" | |
| ), | |
| ) | |
| hosts = _configured_hosts(root) | |
| unbound = sorted( | |
| h for h in hosts if any(cfg.binding(h, tier).model is None for tier in MODEL_TIERS) | |
| ) | |
| if not unbound: | |
| bound = ", ".join( | |
| f"{tier}={cfg.binding(host, tier).model}" | |
| for host in sorted(hosts)[:1] | |
| for tier in MODEL_TIERS | |
| ) | |
| return DoctorCheck( | |
| name="model_tiers", | |
| description="model-tier routing", | |
| status="ok", | |
| detail=bound, | |
| ) | |
| return DoctorCheck( | |
| name="model_tiers", | |
| description="model-tier routing", | |
| status="warning", | |
| detail=( | |
| f"no tier→model binding for: {', '.join(unbound)} " | |
| "(dispatches use the host CLI's default model)" | |
| ), | |
| ) |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@packages/nightly-core/src/nightly_core/doctor.py` around lines 184 - 226,
Update _check_model_tiers to treat a host as unbound when its configured tier
map is missing any entry from MODEL_TIERS, not only when the map is empty;
report such hosts in the warning path. Expand the successful detail to include
bindings for every configured host rather than sampling sorted(hosts)[:1], and
add a matching test covering a host bound only for coding.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Lands RFC 007 Phase A and a new RFC 012 Phase A. Together they answer three questions for every dispatch: which model, how hard to think, and when to hand off.
Model tiers (RFC 007)
Three tiers named after task complexity —
lite/coding/reasoning— bound to concrete model ids per host in.nightly/config.yml. Resolution order is plan frontmatter (model_tier:) → specialist role default → per-host binding. A host with no binding falls through to its CLI's default model rather than hard-failing on a guessed id.The role defaults amend the RFC's original table
The original paired role with seniority. The axis that actually matters is how expensive it is to be wrong and not notice:
implementernightly verifytesterreviewerresearcherReview is the last judgment call before a diff becomes a PR: lint, types, and tests have all already passed by the time it runs, which is exactly why review exists. A lite-tier mis-approval is caught by nothing. The original RFC flagged this hazard in its Risks section and then mitigated it with
coding— half a step.Reasoning effort is a new per-tier dial
lite/coding→low,reasoning→xhigh. Model choice alone under-delivers on cost: a fast model run at high effort spends its savings on preamble and exploratory tool calls — the exact behavior the fast tiers exist to avoid. Effort is injected as prompt text rather than a vendor CLI flag, so it works on every host and no-ops where unsupported.Parallelism + context handoff (RFC 012)
Global specialist cap, worktree cap, and per-tier caps (
lite 8 / coding 6 / reasoning 2). The gradient is the cost control: it lets the cheap tiers run wide without the bill scaling with fan-out, since reasoning work largely serializes anyway.Context thresholds are ratios of the dispatched model's window, not absolutes, so one setting governs a mixed fleet:
Soft = finish the task you're on, write a handoff summary, relaunch a fresh agent with the same goals. Hard = stop now, mid-task. Past halfway, "one more step" risks truncating a write, which loses work rather than merely wasting tokens.
Drive-by bug fix
cli.pyanddoctor.pyeach held a private copy of the config template, and they had drifted: doctor's had lost thevault:andworktree:blocks entirely, so a repo repaired bynightly doctor --fixgot a materially different config than a freshly initialized one. Both now import oneDEFAULT_CONFIG_YML, with a parity test pinning it.nightly doctoralso gains an advisory check for configured hosts with no tier binding — pulled forward from Phase C, since inert routing matters more now that the reviewer tier changed.Verification
test_routing.pyandtest_config_template.py)ruff check/ruff format --check/pyreflycleannightly verify: all 5 checks passNot in this PR
RFC 007 Phases B/C (threading the model id into dispatch argv, briefing tier breakdown,
--tierflag, host skill text) and RFC 012 Phases B/C (admission enforcement, keepalive handoff injection). This PR is schema + resolvers + tests; nothing yet changes what argv a dispatch actually gets.🤖 Generated with Claude Code
Summary by CodeRabbit
New Features
nightly doctorfor missing model-tier bindings.Improvements