feat: Antigravity provider + configurable routing policy (difficulty bands, decision agent, fallback chains) - #2
Open
nuchareviews-beep wants to merge 10 commits into
Conversation
router.ts's isRoutableProvider() and benchmarks.ts's rankAutoModelOptions() each hardcoded their own separate provider allowlist. Adding "antigravity" to router.ts's copy made it eligible for thread creation, but rankAutoModelOptions kept using its own stale copy internally, so Antigravity candidates were silently excluded from every ranked (benchmarked) selection and could only ever be chosen via fallbackSelection -- which only runs when Codex, Claude Code, and Cursor are ALL simultaneously unavailable or quota-exhausted. Verified live against a running bb instance before this fix: two `bb autorouter route` calls (difficulty 2 and 92), plus one with an explicit "route this to agy" instruction and one naming a real agy model id directly, all still picked Codex every time. That's the gap this fix closes. Both files now read one shared ROUTABLE_PROVIDER_IDS from benchmarks.ts, so this class of two-lists-drift-apart bug can't recur. No fabricated benchmark score is introduced for Antigravity/ Gemini models -- rankAutoModelOptions correctly still returns no ranked option for them (no CursorBench entry exists), matching this repo's existing "does not invent scores for unmeasured models" policy. Added two tests: one confirming that policy still holds for Antigravity specifically, one confirming fallbackSelection actually returns a route on Antigravity when it's the only eligible candidate -- the real, narrow condition under which it gets picked in practice. All 30 tests pass, typecheck and build are clean.
Below a difficulty threshold (25/100), skip CursorBench-driven ranking and use a fixed priority instead: Antigravity (local agy, no per-token billing) first, then Codex, then Claude Code. Cursor keeps its normal benchmark-ranked path at every difficulty -- it's not part of this priority list. This is a real preference, not a fallback-of-last-resort: previously Antigravity only ever won when Codex, Claude Code, and Cursor were ALL simultaneously unavailable (see the prior commit). Simple tasks don't need a capability-matched model chosen from a benchmark curve built for harder work; they need the cheapest thing that can do them. An explicit model override (user-requested or from custom instructions) still takes priority over this -- it only applies to the automatic difficulty-based path. Verified live against a running bb instance: `bb autorouter route` at difficulty 0 and 3 now correctly picks Antigravity and produces a real response; difficulty 92 is unaffected and still picks Codex, so this doesn't regress normal-difficulty routing. 6 new tests (36 total, all passing) cover the priority order, quota-exhaustion fallthrough at each tier, the threshold boundary, and deferring to benchmark ranking when none of the three are eligible.
Replace the free-text "provider/model" input with a searchable provider/model list, following the same pattern as bb-plugin-prompt-enhancer's ModelSettingsSection: a listModels RPC backed by a KV-cached, stale-while-revalidate provider/model catalog, rendered as a cmdk-based Command picker. Settings storage is unchanged (decisionAgent stays a "provider/model" string, still readable via `bb autorouter config --decision-agent`); this only changes how it's set from the settings UI. Vendors components/ui/command.tsx adapted from prompt-enhancer's version, swapped to autorouter's own HugeIcons-based <Icon> instead of adding lucide-react as a second icon library.
The classifier's "automatic" mode used to hardcode a fixed provider priority (Cursor gpt-5.6-sol-medium -> Codex gpt-5.6-luna) directly in router.ts. Move it to a new automaticFallbackChain setting (ordered provider/model list, editable via the settings-page picker or `bb autorouter config --automatic-fallback`), so it's a preference users can reorder, extend, or clear rather than an opinion baked into the code. Default value reproduces the previous hardcoded order, so existing "automatic" behavior is unchanged unless a user edits it — verified via the existing router.test.ts suite (all 36 tests still pass unmodified, since they exercise resolveRoute through defaultAutorouterSettings). parseStoredSettings now merges stored settings with defaults before giving up, instead of discarding the whole object on schema mismatch — otherwise settings saved before this field existed (including this session's own decisionAgent override) would have silently reset to every default on first load after the update.
1. Per-difficulty model selection (settings.difficultyBands): replaces the
hardcoded "simple task" shortcut (SIMPLE_TASK_DIFFICULTY_MAX = 25,
SIMPLE_TASK_PROVIDER_PRIORITY = [antigravity, codex, claude-code]) with
a plain, editable list of {maxDifficulty, fallbackChain} bands, checked
low-to-high. Chain entries may be a bare provider (any/default model)
or an exact provider/model pin. Default settings reproduce the old
hardcoded band as ordinary data, not logic baked into router.ts. New
difficultyBandSelection() replaces simpleTaskSelection(); router.test.ts
updated to match (39 tests, was 36) plus new coverage for band ordering,
exact-model pins in a chain, and "no band covers this difficulty".
2. Decision section: merged the "Decision agent" picker and "Automatic
fallback chain" picker into one Command control per the same UX
pattern used for difficulty bands. In Automatic mode the single list
is the fallback-order editor (checking a model toggles its membership
in automaticFallbackChain); switching to a specific model pins the
classifier to it directly, and the fallback-order editor hides since
it's not in play. Same picker component now reused three ways
(Decision agent, and once per difficulty band) via a shared
DifficultyBandsSection component.
Verified: tsc clean, 39/39 tests pass, `bb plugin build` produces real
dist/ output.
Bands used to always mean "difficulty <= maxDifficulty" -- the comparator was implicit and hardcoded in difficultyBandSelection. Add a comparator field (one of <=, <, >=, >, ==) to each band, defaulting to "<=" so existing bands (including the shipped default) keep their exact prior meaning unless explicitly changed. This makes ">= 80" or "== 50" style bands expressible, not just "at or under N". - settings.ts: BAND_COMPARATORS, BandComparator, DEFAULT_BAND_COMPARATOR; difficultyBandSchema requires comparator (strict schema). - router.ts: new bandMatchesDifficulty() switches on the comparator; difficultyBandSelection() uses it instead of a hardcoded `<=`. - app.tsx: band editor gets a comparator <select> next to the threshold input; "Add band" seeds new bands with the default comparator. - server.ts: status output and --difficulty-bands JSON docs mention the field. Migration: parseStoredSettings now backfills a missing `comparator` on each stored band (defaulting to "<=") before re-validating, instead of letting one old-shape band fail validation and silently reset the whole settings object to every default -- the same class of bug fixed for top-level fields last commit, now handled one level deeper. Added settings.test.ts to cover this directly (it wasn't covered anywhere before), including the exact "field added after settings were stored" scenario that broke live earlier this session. Tests: 45 -> 50 (6 new bandMatchesDifficulty/comparator-aware-selection cases in router.test.ts, 5 new migration cases in settings.test.ts). tsc clean, bb plugin build produces real dist/ output.
Docs previously described a fixed, hardcoded "Antigravity priority for simple tasks" rule. Rewrite to document the actual current mechanism: settings.difficultyBands (comparator-aware, user-editable) and the merged Decision agent picker/fallback-order editor, with the old hardcoded behavior noted as just the shipped default value. Also adds a real screenshot of the rendered settings page (Decision agent picker + fallback order editor) for the PR description.
settings.difficultyBands now stores { minDifficulty, maxDifficulty,
fallbackChain } directly instead of { maxDifficulty, comparator,
fallbackChain }. Both bounds are inclusive and independently nullable
("unbounded" on that side), so a band expresses a genuine two-sided range
in one native shape -- e.g. 0-25 for one model, 26-75 for another, 76-100
for a third -- rather than five separate single-sided comparators that
had to be paired up by hand to express a range.
- settings.ts: difficultyBandSchema is now { minDifficulty, maxDifficulty
(both nullable ints, 0-100), fallbackChain }, with a refine() rejecting
min > max. migrateBand() converts both prior shapes -- the original
pre-comparator { maxDifficulty } (implicit "<=") and last commit's
{ maxDifficulty, comparator } -- into the equivalent range, so existing
stored settings (including this session's own live band) keep their
exact prior meaning. BAND_COMPARATORS/BandComparator/
DEFAULT_BAND_COMPARATOR are gone; nothing else referenced them outside
router.ts/app.tsx, both updated in this commit.
- router.ts: bandMatchesDifficulty() checks difficulty against
[minDifficulty, maxDifficulty] directly; band evaluation order sorts by
effective lower bound (null treated as -1) instead of maxDifficulty.
- app.tsx: the band editor's comparator <select> + single number input
become two number inputs ("min" / "max"), each blank = unbounded on
that side.
- server.ts: status output and --difficulty-bands JSON docs/example
updated to the range shape.
- README.md: rewrites the per-difficulty section to document ranges
instead of the comparator model it described in the last commit.
Tests: 53/53 (was 50) -- router.test.ts's comparator suite replaced with
range-equivalent coverage (unbounded-below, unbounded-above, two-sided,
exact-value, fully-unbounded, sort-by-effective-lower-bound, and an
explicit 0-25/26-75/76-100 three-band routing case), settings.test.ts
gets a migration case per prior shape variant. tsc clean, bb plugin build
produces real dist/ output.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Supersedes #1. Includes everything from that PR (the
antigravityprovider becoming a real, working routable provider, plus the two-lists-drift-apart benchmark bug fix it caught and fixed) and a full configurable routing-policy system built on top of it — every routing decision that PR deliberately left as a fixed rule (to keep that PR focused) is now a plain, editable setting instead of logic baked intorouter.ts.What's in this PR
1. Antigravity as a real routable provider (from #1, unchanged)
antigravityprovider (bridges to the localagyCLI viabb-plugin-antigravity-acp) to Autorouter's routable-provider set.router.tsandbenchmarks.tseach hardcoded their own separate provider allowlist, so Antigravity was eligible for candidate discovery but silently excluded fromrankAutoModelOptions's ranking regardless. Both files now read one sharedROUTABLE_PROVIDER_IDS.fallbackSelectionwhen Codex/Claude Code/Cursor are all unavailable — unless a difficulty band below routes to it directly.2. Per-difficulty model selection: native two-sided ranges (new)
settings.difficultyBands: an ordered list of{ minDifficulty, maxDifficulty, fallbackChain }bands, checked low-to-high by effective lower bound. Both bounds are inclusive and independently nullable —minDifficulty: nullmeans unbounded below (matches down to 0),maxDifficulty: nullmeans unbounded above (matches up to 100), andminDifficulty === maxDifficultymatches an exact score. The first band whose range covers a task's difficulty score skips benchmark ranking entirely and routes straight through that band's own fallback chain.This is a genuine two-sided range, not a series of independent one-sided thresholds paired up by hand:
0–25for one model,26–75for another,76–100for a third, with no gaps or overlaps to manage. Chain entries can be a bare provider (any/default model) or an exactprovider/modelpin.This directly replaces what would otherwise have been a hardcoded rule (
SIMPLE_TASK_DIFFICULTY_MAX = 25/SIMPLE_TASK_PROVIDER_PRIORITY = [antigravity, codex, claude-code]) — that logic doesn't exist inrouter.tsat all; it's only the default value of this setting. Add, remove, reorder, or clear bands freely from the settings page or:bb autorouter config --difficulty-bands '[{"minDifficulty":null,"maxDifficulty":25,"fallbackChain":["antigravity","codex","claude-code"]},{"minDifficulty":26,"maxDifficulty":75,"fallbackChain":["codex/gpt-5.6-terra"]}]'(This went through one design iteration during review: the first version of this PR used a single threshold + comparator (
<=,<,>=,>,==) per band, which could express a one-sided cutoff but needed two bands hand-paired to express a real range. Replaced with native two-sided bounds, which subsumes every comparator case directly —<= Nis{null, N},>= Nis{N, null},== Nis{N, N}— while making an actual range a single band instead of an implicit pairing. Existing stored settings (including comparator-shaped bands saved during that iteration, and the original pre-comparator shape before that) migrate automatically — see Verification.)3. Decision agent: one merged control (new)
The model that rates each task's difficulty before routing is itself configurable, via a single searchable provider/model picker instead of a free-text box:
automaticFallbackChain, same shape and same picker pattern as the difficulty bands above. In this mode the picker doubles as the fallback-order editor — checking a model toggles its membership in the chain, shown right below as a reorderable list.If nothing in the fallback order is available, the classifier falls back to any launchable model, then whatever's first — a last-resort safety net, not a preference, so it isn't user-facing.
Screenshots
Decision agent picker + fallback-order editor, rendered live:
Per-difficulty model selection, rendered live — note this screenshot is from the single-comparator iteration (
<=dropdown + one threshold) described above, captured before the native-range rewrite in this update. The range UI replaces that dropdown with two bounded number inputs ("min" / "max", either blank for unbounded) in the same card layout; a refreshed screenshot will follow once available, but the underlying behavior shown here — ordered fallback chain, reorder/remove, add-band, add-model-to-chain — is otherwise unchanged by the rewrite:Verification
npx tsc --noEmit— cleannpx vitest run— 53/53 passing (28 original + 25 new across this PR: routable-provider regression coverage, band-range/exact-model-pin/two-sided-range cases, and settings-migration coverage for every prior stored shape)bb plugin build— produces realdist/outputbb autorouter routecall lands on the configured decision-agent model by tracing the hidden classifier thread's actualproviderId, not just trusting the setting saved.0–25/26–75/76+) correctly routes a difficulty-55 task to the middle band's pinned model (benchmarkScore: null, bypassing ranking) — not just the edges.settings.test.tsfor every shape.Scope
README.md,app.tsx,benchmarks.ts,router.ts,router.test.ts,server.ts,settings.ts,settings.test.ts,components/ui/command.tsx(vendored/adapted frombb-plugin-prompt-enhancer's picker component),docs/screenshots/.