Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
9417925
feat(model-probe): discover model controls from the harness at init time
ulmentflam Jul 28, 2026
df185df
docs(rfc-007): reconcile Phase B checklist; record discovery as Resol…
ulmentflam Jul 28, 2026
2aec15f
feat(dispatch): enforce parallelism caps at admission (RFC 012 B1/B2/B5)
ulmentflam Jul 28, 2026
1490b04
feat(worktree): honor parallelism.max_worktrees (RFC 012 B3)
ulmentflam Jul 28, 2026
d09c4b3
docs(rules): fleet doctrine as rule 12 (RFC 007 C1/B2, RFC 012 B4/C4)
ulmentflam Jul 28, 2026
b6669a7
docs(readme): cost-aware dispatch, fleet caps, context handoff (RFC 0…
ulmentflam Jul 28, 2026
4b6606f
feat(briefing): dispatches-by-tier panel (RFC 007 B3) — completes RFC…
ulmentflam Jul 28, 2026
b7ec2c6
feat(keepalive): inject handoff instructions at the context threshold…
ulmentflam Jul 28, 2026
cf64eca
feat(digest,briefing): surface pending handoffs (RFC 012 C2/C3) — com…
ulmentflam Jul 28, 2026
c9e7347
fix(cascade): skip RFC items already done on an unmerged local branch
ulmentflam Jul 28, 2026
7ac7d2e
feat(doctor): warn when Nightly work cannot leave the machine
ulmentflam Jul 28, 2026
8b37780
fix(status): drop the doubled `enabled=`; surface tiers, fleet caps, …
ulmentflam Jul 28, 2026
3e9765c
fix(dispatch): derive host-support errors instead of restating them
ulmentflam Jul 28, 2026
6960dad
feat(verify): configurable per-check timeout, actionable timeout message
ulmentflam Jul 28, 2026
dac293e
feat(doctor): warn when a tier is bound to a different band's model
ulmentflam Jul 28, 2026
2aa3b48
feat(doctor): report config blocks an existing config.yml never learned
ulmentflam Jul 28, 2026
4aa1166
refactor(cli): type the dispatch row printer; show each dispatch's tier
ulmentflam Jul 28, 2026
45222ac
feat(rules): pre-flight verification doctrine (RFC 008 A); reconcile …
ulmentflam Jul 28, 2026
6e91197
feat(briefing): auto-ticked RFC items panel (RFC 008 Phase B)
ulmentflam Jul 28, 2026
f17f76e
test(doctor): pin that every check helper is actually wired
ulmentflam Jul 28, 2026
48af086
test: exercise every Literal member through its dispatching function
ulmentflam Jul 28, 2026
84a95ad
test(conclude_skill): pin the companion skills' human-only doctrine
ulmentflam Jul 28, 2026
ed04000
fix(doctor,rfc-007): partial tier bindings; stale ids in the RFC sample
ulmentflam Jul 28, 2026
efe1ba5
test(plans): cover append_pr_feedback, the pr_rescue write path
ulmentflam Jul 28, 2026
c0813e1
test(headless): cover default_subprocess_runner (76% → 100%)
ulmentflam Jul 28, 2026
48daf7d
test(ci_watch): cover the classifier that decides red vs green (86% →…
ulmentflam Jul 28, 2026
ba8686a
test(triage): cover the open-PR issue-reference guard (82% → 97%)
ulmentflam Jul 28, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
143 changes: 107 additions & 36 deletions .planning/rfcs/007-model-tier-routing.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ author: nightly-seed
source: interactive_seed
estimated_effort: ~7h across 3 phases
phase_a: implemented
status_note: all three phases implemented 2026-07-28
---

# RFC 007 — Model-tier routing for cost-aware specialist dispatch
Expand Down Expand Up @@ -64,9 +65,10 @@ task shape to choose a tier:

- `nightly specialist implementer` — coding tier
- `nightly specialist tester` — coding tier
- `nightly specialist reviewer` — coding tier (could be lite for
trivial diffs; conservative default is coding)
- `nightly specialist researcher` — reasoning tier
- `nightly specialist reviewer` — reasoning tier (review *is* result
validation; nothing downstream re-checks it — see Resolved #4)
- `nightly specialist researcher` — lite tier (file search and
summarization over code already on disk — see Resolved #4)
- Plan body says "audit-only" / "doc-only" / "briefing-only" →
lite tier
- Plan frontmatter declares `model_tier: reasoning` → override
Expand Down Expand Up @@ -202,31 +204,29 @@ Sonnet 4.7 doesn't break the config; only the model id under
pointing at concrete model ids:

```yaml
# Shipped defaults — the only ids Nightly states authoritatively.
model_tiers:
claude:
lite: claude-haiku-4-5
coding: claude-sonnet-4-6
reasoning: claude-opus-4-7
codex:
lite: gpt-5-mini
coding: gpt-5
reasoning: gpt-5-reasoning
cursor:
coding: claude-sonnet-5
reasoning: claude-opus-5
cursor: # Anthropic-backed, same ids
lite: claude-haiku-4-5
coding: claude-sonnet-4-6
reasoning: claude-opus-4-7
gemini:
lite: gemini-2.5-flash
coding: gemini-2.5-pro
reasoning: gemini-3.5-pro
antigravity:
lite: gemini-2.5-flash
coding: gemini-2.5-pro
reasoning: gemini-3.5-pro
opencode:
coding: claude-sonnet-5
reasoning: claude-opus-5
opencode: # Anthropic-backed, same ids
lite: claude-haiku-4-5
coding: claude-sonnet-4-6
reasoning: claude-opus-4-7
coding: claude-sonnet-5
reasoning: claude-opus-5

# ILLUSTRATIVE ONLY — not shipped, and the ids below are placeholders.
# `codex`, `gemini`, and `antigravity` resolve to no binding by default;
# their dispatches fall through to the host CLI's own model. Wire real
# ids yourself, or let `nightly init` discover them (Resolved #12).
# codex:
# lite: <vendor model id>
# coding: <vendor model id>
# reasoning: <vendor model id>
```

`nightly init` and `nightly doctor` write this default block.
Expand Down Expand Up @@ -341,6 +341,35 @@ everywhere and degrades to a no-op on a model that ignores it. A
future phase can upgrade specific hosts to a native flag without
changing the config schema.

**12. Model controls are discovered, not declared (ADDED 2026-07-28).**
`nightly init` probes each installed host CLI's `--help` for its
model-selection flag and its advertised model vocabulary, then ranks that
vocabulary into the three tiers and writes the result to
`model_tiers.<host>.flag` and `model_tiers.<host>.<tier>`.

The original plan had Nightly carry a per-host flag table. That table is
wrong the day a vendor renames a flag, and a wrong flag is a hard spawn
failure in the middle of an unattended run — whereas the right one is
readable from the CLI in milliseconds. Discovery immediately found
`--model` on opencode and gemini, neither of which had been verified by
hand when the defaults were written.

Two rules keep discovery safe:

- **It can only add certainty.** Probe results merge over the seeded
defaults; any probe failure degrades to the seeded template, so `init`
can never fail because a host CLI misbehaved.
- **Pinned beats floating.** A discovered *pinned* id (`claude-opus-5`)
overrides a seeded default, but a bare alias (`opus`) does not. Aliases
resolve to whatever shipped most recently — convenient interactively,
wrong for an overnight run whose model should still be identifiable in
the morning.

Host coverage spans all seven major harnesses (Claude, Codex, Cursor,
Gemini, OpenCode, Pi, Hermes) plus Antigravity. `pi` and `hermes` are
recognized at the routing layer only — they ship no integration package,
so skill install and keep-alive hooks are unavailable for them.

## Risks

- **Tier mis-pick at the borderline.** Agent judgment will
Expand All @@ -353,10 +382,25 @@ changing the config schema.

- **Stale config after a model deprecation.** If Anthropic
deprecates Haiku 4.5 in favor of Haiku 5.0, configs still
pointing at the old id will fail at dispatch time. Mitigation:
`nightly doctor` gains a future check that pings each
configured model id; the immediate failure mode is "dispatch
raises" which surfaces in the briefing.
pointing at the old id will fail at dispatch time.

*2026-07-28: the proposed mitigation — a doctor check validating
each configured id — turns out to be unbuildable from local
signal.* The vocabulary a host CLI advertises in `--help` is a
sample, not an enumeration: `claude --help` names four tokens
(`fable`, `opus`, `sonnet`, `claude-fable-5`) and **none** of the
three production ids Nightly ships as defaults appear in it. A
membership test would flag correct configuration as broken, which
is worse than no check at all. Validating for real needs a network
call to the vendor's models endpoint — out of scope for `doctor`,
which must work offline.

What shipped instead is `_check_tier_sanity`: a *family*
consistency check that catches the misconfiguration local signal
can actually see — a tier bound to a model from a different band
(`lite: claude-opus-5`). Unrecognized families are skipped rather
than guessed at. The deprecation case remains covered only by
"dispatch raises", which surfaces in the briefing.

- **Host-side rate limits / billing caps.** Switching to lite tier
for the bulk of doc work could trip the host's rate limit if
Expand Down Expand Up @@ -481,14 +525,41 @@ missing config; README updated.
reviewer-tier change makes a silently-inert routing config more
consequential than it was when C2 was scheduled.

**Phase B — Dispatch integration**
- [ ] B1. `nightly dispatch start` reads resolved model id
- [ ] B2. Task-tool fallback documented across six host skill.md
- [ ] B3. Briefing tier-breakdown line
- [ ] B4. `nightly specialist --tier <tier>` flag
- [ ] B5. End-to-end dispatch tests across tiers + fallback
**Phase B — Dispatch integration** — *core landed 2026-07-28; B2/B3 open*
- [x] B1. `nightly dispatch start` reads resolved model id and passes it
with the **discovered** model flag (see B6)
- [x] B2. Task-tool fallback documented — delivered as rule 12 of the
shared rules block alongside C1, not six skill files. *(Ticked
2026-07-28 by RFC 008's own doctrine: implemented in `173a7e8`,
checklist never reconciled.)*
- [x] B3. Briefing tier-breakdown line — a `dispatches by tier` panel
rendered from each task's `dispatch.json`. Reads the run's task
dirs directly rather than via `list_dispatches`, which resolves the
*current* run; the briefing renders arbitrary (including concluded)
runs. Untiered pre-RFC-007 records are surfaced as `unrouted`
rather than dropped.
- [x] B4. `nightly specialist --tier <tier>` flag
- [x] B5. Dispatch resolution tests across tiers + host-miss fallback
(`tests/test_routing.py`); end-to-end argv assertions still open
- [x] B6. *(unplanned, supersedes part of B1)* `nightly init` **discovers**
each host's model-selection flag and model vocabulary from the host
CLI's own `--help`, rather than Nightly carrying a vendor table.
See `nightly_core.model_probe` and RFC 007 Resolved #12.

**Phase C — Auto-tag heuristic + doctor + docs**
- [ ] C1. Auto-tag scoping paragraph on six host skills
- [ ] C2. Doctor flags missing `model_tiers` block
- [ ] C3. README "Cost-aware dispatch" section
- [x] C1. Tier-scoping guidance — delivered as **rule 12 of the shared
rules block** (`nightly_core.rules`) rather than six near-duplicate
skill files. `seed_rules` propagates one marker-delimited block to
every host's AGENTS.md / CLAUDE.md, so one edit reaches all seven
harnesses and cannot drift between them. B2 is satisfied by the
same change.
- [x] C2. Doctor flags missing `model_tiers` block — shipped early as
`_check_model_tiers` in `9b23ef1`. *(Ticked 2026-07-28; recorded
there as "A8, pulled forward from C2" but the C2 box was left
unchecked — exactly the drift RFC 008 exists to catch.)*
- [x] C3. README "Cost-aware dispatch" section — plus a "Context
handoff" section, the `parallelism:` enforcement note, and the
previously-undocumented `nightly dispatch` command family in the
CLI reference. Backed by `tests/test_readme_claims.py`, which pins
the tier/effort/ratio tables against the code so prose and
behavior cannot drift apart.
34 changes: 26 additions & 8 deletions .planning/rfcs/008-pre-rfc-completion-check.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
---
status: accepted
phase_a: implemented (A3 deferred — see note)
phase_b: implemented
sized: true
title: Pre-RFC completion check — verify deliverable doesn't already exist before dispatching
created: 2026-06-04
Expand Down Expand Up @@ -382,13 +384,29 @@ both branches.
## Sized checklist

**Phase A — Verifier paragraph + auto-tick documentation**
- [ ] A1. `RFC_008_VERIFIER_PARAGRAPH` constant in `nightly_core.specialists` (or `skill_blocks` if extracted)
- [ ] A2. Verifier paragraph added to all six host `skill.md` files in the SCOPE step
- [ ] A3. `_REQUIRED_SKILL_TOKENS` extended with `pre-flight verification` per host
- [ ] A4. `_is_item_in_flight` docstring cross-references this RFC's verifier
- [ ] A5. Tests covering presence + doctor drift detection
- [x] A1. `RFC_008_VERIFIER_PARAGRAPH` constant in `nightly_core.specialists`
- [x] A2. Verifier doctrine reaches every host — as **rule 13 of the
shared rules block**, not six duplicated skill files. This RFC
itself calls the per-host duplication "a doctor-monitored drift
surface" (Resolved #9), which concedes the duplication is a
liability; `seed_rules` propagates one marker-delimited copy to
every host's AGENTS.md / CLAUDE.md and cannot drift between them.
Same delivery as RFC 007 C1/B2.
- [ ] A3. `_REQUIRED_SKILL_TOKENS` per-host token — **not applicable as
written**, given A2's delivery: there is no per-host copy to drift.
The equivalent guard is doctor's existing `_check_rules` plus a test
asserting the doctrine is inside the seeded markers (A5). Left open
rather than silently reinterpreted — a human should confirm the
substitution before it is ticked.
- [x] A4. `_is_item_in_flight` docstring cross-references the verifier
and the RFC 012 local-branch guard as the three composing layers
- [x] A5. Tests covering presence, marker placement, and content

**Phase B — Briefing surface + auto-tick commit format**
- [ ] B1. Briefing template gains "Auto-ticked RFC items" optional section
- [ ] B2. Commit message format documented in the verifier paragraph
- [ ] B3. Tests covering rendered briefing with and without auto-ticks
- [x] B1. Briefing gains a "ticked as already-done (verify these)"
panel, populated by scanning `main..HEAD` for rule 13's commit
format. Omitted when empty.
- [x] B2. Commit message format documented in the verifier paragraph —
shipped with A1/A2 in `15fb1bc`. *(Ticked by rule 13's own process:
verified present before implementing, not re-implemented.)*
- [x] B3. Tests covering parse, render-with, and render-without
50 changes: 37 additions & 13 deletions .planning/rfcs/012-fleet-parallelism-and-context-handoff.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,13 +9,15 @@ author: operator
source: interactive_seed
estimated_effort: ~5h across 3 phases
phase_a: implemented
phase_b: implemented
phase_c: implemented
---

# RFC 012 — Fleet parallelism and context-handoff protocol

## Status

`accepted` — operator seed in the 2026-07-28 interactive session,
`implemented` — all three phases landed 2026-07-28. Operator seed in the 2026-07-28 interactive session,
alongside the RFC 007 amendment. Two knobs that only make sense
together: how *wide* the fleet runs, and what an individual agent does
when its context fills up. Phase A lands the config schema and the
Expand Down Expand Up @@ -256,15 +258,37 @@ produces a handoff summary naming its unfinished goals.
- [x] A6. Config template blocks
- [x] A7. Unit tests (in `tests/test_routing.py`)

**Phase B — Admission control**
- [ ] B1. `dispatch start` admission check
- [ ] B2. `dispatch status` shows cap utilization
- [ ] B3. Worktree creation honors `max_worktrees`
- [ ] B4. Six host skills: fan-out-to-cap guidance
- [ ] B5. Admission tests

**Phase C — Handoff protocol**
- [ ] C1. Keepalive threshold comparison + prompt injection
- [ ] C2. `handoff:` section in `digest.md`
- [ ] C3. Briefing handoff counts
- [ ] C4. Six host skills: two-threshold protocol text
**Phase B — Admission control** — *B1/B2/B5 landed 2026-07-28*
- [x] B1. `dispatch start` admission check — refuses with exit code 3 and
names the cap that blocked it; `--force` overrides. Dispatch state
now persists its tier so counting doesn't re-resolve the plan.
- [x] B2. `dispatch status` shows a `capacity:` line (live/cap per tier
plus the global total)
- [x] B3. Worktree creation honors `max_worktrees` — raises the typed
`WorktreeCapReached` (not a bare RuntimeError, so "at capacity" is
distinguishable from "git failed"), checked before the branch is
cut so a refused request leaves nothing behind. Both callers
(`nightly worktree`, the headless driver) pass the configured cap;
CLI exits 3, matching `dispatch start`.
- [x] B4. Fan-out-to-cap guidance — same delivery as RFC 007 C1: rule 12
of the shared rules block, not six skill files.
- [x] B5. Admission tests (`tests/test_admission.py`, 14 cases) —
including the liveness rule: a dispatch whose PID is gone must not
occupy a slot, or an unpolled crash wedges the fleet

**Phase C — Handoff protocol** — *C1 landed 2026-07-28*
- [x] C1. Keepalive threshold comparison + prompt injection — a three-rung
ladder (hard handoff > soft handoff > the v0.0.12 diet nudge). The
handoff block *replaces* the diet block rather than stacking; two
competing directives in one prompt is how an agent follows neither.
Session thresholds resolve against the reasoning tier's model, per
Resolved #8 and rule 12.
- [x] C2. `Pending handoffs` section in `digest.md` — the digest is what
the `SessionStart(compact)` hook re-injects, so a handoff written
just before a compaction survives the very event it exists for.
- [x] C3. Briefing handoff panel — names the task and its outstanding
work rather than a bare count; "which work was left" is the
operator's actual first question.
- [x] C4. Two-threshold protocol text — in rule 12. (C1–C3, the
keepalive enforcement, remain open: the doctrine is documented but
the hook does not yet compare against the thresholds.)
Loading