Skip to content

feat: Antigravity provider + configurable routing policy (difficulty bands, decision agent, fallback chains) - #2

Open
nuchareviews-beep wants to merge 10 commits into
jjcm:mainfrom
nuchareviews-beep:local-opinionated
Open

feat: Antigravity provider + configurable routing policy (difficulty bands, decision agent, fallback chains)#2
nuchareviews-beep wants to merge 10 commits into
jjcm:mainfrom
nuchareviews-beep:local-opinionated

Conversation

@nuchareviews-beep

@nuchareviews-beep nuchareviews-beep commented Aug 24, 2026

Copy link
Copy Markdown

Summary

Supersedes #1. Includes everything from that PR (the antigravity provider becoming a real, working routable provider, plus the two-lists-drift-apart benchmark bug fix it caught and fixed) and a full configurable routing-policy system built on top of it — every routing decision that PR deliberately left as a fixed rule (to keep that PR focused) is now a plain, editable setting instead of logic baked into router.ts.

What's in this PR

1. Antigravity as a real routable provider (from #1, unchanged)

  • Adds the locally-registered antigravity provider (bridges to the local agy CLI via bb-plugin-antigravity-acp) to Autorouter's routable-provider set.
  • Fixes a real bug caught by live-testing, not just the unit suite: router.ts and benchmarks.ts each hardcoded their own separate provider allowlist, so Antigravity was eligible for candidate discovery but silently excluded from rankAutoModelOptions's ranking regardless. Both files now read one shared ROUTABLE_PROVIDER_IDS.
  • OmniRoute stays deliberately excluded — reserved for delegated subagent work, not interactive Autorouter threads.
  • Antigravity has no CursorBench entry (no fabricated score), so under plain benchmark ranking it's still only chosen via fallbackSelection when Codex/Claude Code/Cursor are all unavailable — unless a difficulty band below routes to it directly.

2. Per-difficulty model selection: native two-sided ranges (new)

settings.difficultyBands: an ordered list of { minDifficulty, maxDifficulty, fallbackChain } bands, checked low-to-high by effective lower bound. Both bounds are inclusive and independently nullable — minDifficulty: null means unbounded below (matches down to 0), maxDifficulty: null means unbounded above (matches up to 100), and minDifficulty === maxDifficulty matches an exact score. The first band whose range covers a task's difficulty score skips benchmark ranking entirely and routes straight through that band's own fallback chain.

This is a genuine two-sided range, not a series of independent one-sided thresholds paired up by hand: 0–25 for one model, 26–75 for another, 76–100 for a third, with no gaps or overlaps to manage. Chain entries can be a bare provider (any/default model) or an exact provider/model pin.

This directly replaces what would otherwise have been a hardcoded rule (SIMPLE_TASK_DIFFICULTY_MAX = 25 / SIMPLE_TASK_PROVIDER_PRIORITY = [antigravity, codex, claude-code]) — that logic doesn't exist in router.ts at all; it's only the default value of this setting. Add, remove, reorder, or clear bands freely from the settings page or:

bb autorouter config --difficulty-bands '[{"minDifficulty":null,"maxDifficulty":25,"fallbackChain":["antigravity","codex","claude-code"]},{"minDifficulty":26,"maxDifficulty":75,"fallbackChain":["codex/gpt-5.6-terra"]}]'

(This went through one design iteration during review: the first version of this PR used a single threshold + comparator (<=, <, >=, >, ==) per band, which could express a one-sided cutoff but needed two bands hand-paired to express a real range. Replaced with native two-sided bounds, which subsumes every comparator case directly — <= N is {null, N}, >= N is {N, null}, == N is {N, N} — while making an actual range a single band instead of an implicit pairing. Existing stored settings (including comparator-shaped bands saved during that iteration, and the original pre-comparator shape before that) migrate automatically — see Verification.)

3. Decision agent: one merged control (new)

The model that rates each task's difficulty before routing is itself configurable, via a single searchable provider/model picker instead of a free-text box:

  • Automatic (default): tries an ordered automaticFallbackChain, same shape and same picker pattern as the difficulty bands above. In this mode the picker doubles as the fallback-order editor — checking a model toggles its membership in the chain, shown right below as a reorderable list.
  • Picking a specific model instead pins the classifier to it directly, skipping the fallback order entirely.

If nothing in the fallback order is available, the classifier falls back to any launchable model, then whatever's first — a last-resort safety net, not a preference, so it isn't user-facing.

Screenshots

Decision agent picker + fallback-order editor, rendered live:

Decision agent settings

Per-difficulty model selection, rendered live — note this screenshot is from the single-comparator iteration (<= dropdown + one threshold) described above, captured before the native-range rewrite in this update. The range UI replaces that dropdown with two bounded number inputs ("min" / "max", either blank for unbounded) in the same card layout; a refreshed screenshot will follow once available, but the underlying behavior shown here — ordered fallback chain, reorder/remove, add-band, add-model-to-chain — is otherwise unchanged by the rewrite:

Per-difficulty model selection

Verification

  • npx tsc --noEmit — clean
  • npx vitest run53/53 passing (28 original + 25 new across this PR: routable-provider regression coverage, band-range/exact-model-pin/two-sided-range cases, and settings-migration coverage for every prior stored shape)
  • bb plugin build — produces real dist/ output
  • Live-tested against a running bb instance at every step, not just the unit suite:
    • Confirmed a real bb autorouter route call lands on the configured decision-agent model by tracing the hidden classifier thread's actual providerId, not just trusting the setting saved.
    • Confirmed reordering the fallback chain changes which model the classifier actually runs on.
    • Confirmed a genuine three-band range configuration (0–25 / 26–75 / 76+) correctly routes a difficulty-55 task to the middle band's pinned model (benchmarkScore: null, bypassing ranking) — not just the edges.
    • Confirmed the settings-migration path against real stored data at each shape transition: the original pre-comparator shape, the intermediate single-comparator shape, and now the native-range shape all load correctly from what was actually persisted in a live KV store, without wholesale-resetting the whole settings object to defaults — a real regression caught live during development (fields added after a setting was first saved used to silently discard the whole object) and now covered by settings.test.ts for every shape.

Scope

README.md, app.tsx, benchmarks.ts, router.ts, router.test.ts, server.ts, settings.ts, settings.test.ts, components/ui/command.tsx (vendored/adapted from bb-plugin-prompt-enhancer's picker component), docs/screenshots/.

router.ts's isRoutableProvider() and benchmarks.ts's
rankAutoModelOptions() each hardcoded their own separate provider
allowlist. Adding "antigravity" to router.ts's copy made it eligible
for thread creation, but rankAutoModelOptions kept using its own
stale copy internally, so Antigravity candidates were silently
excluded from every ranked (benchmarked) selection and could only
ever be chosen via fallbackSelection -- which only runs when Codex,
Claude Code, and Cursor are ALL simultaneously unavailable or
quota-exhausted.

Verified live against a running bb instance before this fix: two
`bb autorouter route` calls (difficulty 2 and 92), plus one with an
explicit "route this to agy" instruction and one naming a real agy
model id directly, all still picked Codex every time. That's the
gap this fix closes.

Both files now read one shared ROUTABLE_PROVIDER_IDS from
benchmarks.ts, so this class of two-lists-drift-apart bug can't
recur. No fabricated benchmark score is introduced for Antigravity/
Gemini models -- rankAutoModelOptions correctly still returns no
ranked option for them (no CursorBench entry exists), matching this
repo's existing "does not invent scores for unmeasured models"
policy. Added two tests: one confirming that policy still holds
for Antigravity specifically, one confirming fallbackSelection
actually returns a route on Antigravity when it's the only eligible
candidate -- the real, narrow condition under which it gets picked
in practice. All 30 tests pass, typecheck and build are clean.
Below a difficulty threshold (25/100), skip CursorBench-driven
ranking and use a fixed priority instead: Antigravity (local agy, no
per-token billing) first, then Codex, then Claude Code. Cursor keeps
its normal benchmark-ranked path at every difficulty -- it's not part
of this priority list.

This is a real preference, not a fallback-of-last-resort: previously
Antigravity only ever won when Codex, Claude Code, and Cursor were
ALL simultaneously unavailable (see the prior commit). Simple tasks
don't need a capability-matched model chosen from a benchmark curve
built for harder work; they need the cheapest thing that can do them.

An explicit model override (user-requested or from custom
instructions) still takes priority over this -- it only applies to
the automatic difficulty-based path.

Verified live against a running bb instance: `bb autorouter route`
at difficulty 0 and 3 now correctly picks Antigravity and produces a
real response; difficulty 92 is unaffected and still picks Codex, so
this doesn't regress normal-difficulty routing. 6 new tests (36
total, all passing) cover the priority order, quota-exhaustion
fallthrough at each tier, the threshold boundary, and deferring to
benchmark ranking when none of the three are eligible.
Replace the free-text "provider/model" input with a searchable
provider/model list, following the same pattern as
bb-plugin-prompt-enhancer's ModelSettingsSection: a listModels RPC backed
by a KV-cached, stale-while-revalidate provider/model catalog, rendered
as a cmdk-based Command picker. Settings storage is unchanged
(decisionAgent stays a "provider/model" string, still readable via
`bb autorouter config --decision-agent`); this only changes how it's set
from the settings UI.

Vendors components/ui/command.tsx adapted from prompt-enhancer's version,
swapped to autorouter's own HugeIcons-based <Icon> instead of adding
lucide-react as a second icon library.
The classifier's "automatic" mode used to hardcode a fixed provider
priority (Cursor gpt-5.6-sol-medium -> Codex gpt-5.6-luna) directly in
router.ts. Move it to a new automaticFallbackChain setting (ordered
provider/model list, editable via the settings-page picker or
`bb autorouter config --automatic-fallback`), so it's a preference users
can reorder, extend, or clear rather than an opinion baked into the code.

Default value reproduces the previous hardcoded order, so existing
"automatic" behavior is unchanged unless a user edits it — verified via
the existing router.test.ts suite (all 36 tests still pass unmodified,
since they exercise resolveRoute through defaultAutorouterSettings).

parseStoredSettings now merges stored settings with defaults before
giving up, instead of discarding the whole object on schema mismatch —
otherwise settings saved before this field existed (including this
session's own decisionAgent override) would have silently reset to
every default on first load after the update.
1. Per-difficulty model selection (settings.difficultyBands): replaces the
   hardcoded "simple task" shortcut (SIMPLE_TASK_DIFFICULTY_MAX = 25,
   SIMPLE_TASK_PROVIDER_PRIORITY = [antigravity, codex, claude-code]) with
   a plain, editable list of {maxDifficulty, fallbackChain} bands, checked
   low-to-high. Chain entries may be a bare provider (any/default model)
   or an exact provider/model pin. Default settings reproduce the old
   hardcoded band as ordinary data, not logic baked into router.ts. New
   difficultyBandSelection() replaces simpleTaskSelection(); router.test.ts
   updated to match (39 tests, was 36) plus new coverage for band ordering,
   exact-model pins in a chain, and "no band covers this difficulty".

2. Decision section: merged the "Decision agent" picker and "Automatic
   fallback chain" picker into one Command control per the same UX
   pattern used for difficulty bands. In Automatic mode the single list
   is the fallback-order editor (checking a model toggles its membership
   in automaticFallbackChain); switching to a specific model pins the
   classifier to it directly, and the fallback-order editor hides since
   it's not in play. Same picker component now reused three ways
   (Decision agent, and once per difficulty band) via a shared
   DifficultyBandsSection component.

Verified: tsc clean, 39/39 tests pass, `bb plugin build` produces real
dist/ output.
Bands used to always mean "difficulty <= maxDifficulty" -- the comparator
was implicit and hardcoded in difficultyBandSelection. Add a comparator
field (one of <=, <, >=, >, ==) to each band, defaulting to "<=" so
existing bands (including the shipped default) keep their exact prior
meaning unless explicitly changed. This makes ">= 80" or "== 50" style
bands expressible, not just "at or under N".

- settings.ts: BAND_COMPARATORS, BandComparator, DEFAULT_BAND_COMPARATOR;
  difficultyBandSchema requires comparator (strict schema).
- router.ts: new bandMatchesDifficulty() switches on the comparator;
  difficultyBandSelection() uses it instead of a hardcoded `<=`.
- app.tsx: band editor gets a comparator <select> next to the threshold
  input; "Add band" seeds new bands with the default comparator.
- server.ts: status output and --difficulty-bands JSON docs mention the
  field.

Migration: parseStoredSettings now backfills a missing `comparator` on
each stored band (defaulting to "<=") before re-validating, instead of
letting one old-shape band fail validation and silently reset the whole
settings object to every default -- the same class of bug fixed for
top-level fields last commit, now handled one level deeper. Added
settings.test.ts to cover this directly (it wasn't covered anywhere
before), including the exact "field added after settings were stored"
scenario that broke live earlier this session.

Tests: 45 -> 50 (6 new bandMatchesDifficulty/comparator-aware-selection
cases in router.test.ts, 5 new migration cases in settings.test.ts).
tsc clean, bb plugin build produces real dist/ output.
Docs previously described a fixed, hardcoded "Antigravity priority for
simple tasks" rule. Rewrite to document the actual current mechanism:
settings.difficultyBands (comparator-aware, user-editable) and the merged
Decision agent picker/fallback-order editor, with the old hardcoded
behavior noted as just the shipped default value.

Also adds a real screenshot of the rendered settings page (Decision
agent picker + fallback order editor) for the PR description.
settings.difficultyBands now stores { minDifficulty, maxDifficulty,
fallbackChain } directly instead of { maxDifficulty, comparator,
fallbackChain }. Both bounds are inclusive and independently nullable
("unbounded" on that side), so a band expresses a genuine two-sided range
in one native shape -- e.g. 0-25 for one model, 26-75 for another, 76-100
for a third -- rather than five separate single-sided comparators that
had to be paired up by hand to express a range.

- settings.ts: difficultyBandSchema is now { minDifficulty, maxDifficulty
  (both nullable ints, 0-100), fallbackChain }, with a refine() rejecting
  min > max. migrateBand() converts both prior shapes -- the original
  pre-comparator { maxDifficulty } (implicit "<=") and last commit's
  { maxDifficulty, comparator } -- into the equivalent range, so existing
  stored settings (including this session's own live band) keep their
  exact prior meaning. BAND_COMPARATORS/BandComparator/
  DEFAULT_BAND_COMPARATOR are gone; nothing else referenced them outside
  router.ts/app.tsx, both updated in this commit.
- router.ts: bandMatchesDifficulty() checks difficulty against
  [minDifficulty, maxDifficulty] directly; band evaluation order sorts by
  effective lower bound (null treated as -1) instead of maxDifficulty.
- app.tsx: the band editor's comparator <select> + single number input
  become two number inputs ("min" / "max"), each blank = unbounded on
  that side.
- server.ts: status output and --difficulty-bands JSON docs/example
  updated to the range shape.
- README.md: rewrites the per-difficulty section to document ranges
  instead of the comparator model it described in the last commit.

Tests: 53/53 (was 50) -- router.test.ts's comparator suite replaced with
range-equivalent coverage (unbounded-below, unbounded-above, two-sided,
exact-value, fully-unbounded, sort-by-effective-lower-bound, and an
explicit 0-25/26-75/76-100 three-band routing case), settings.test.ts
gets a migration case per prior shape variant. tsc clean, bb plugin build
produces real dist/ output.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant