chore(config): route agents across runtimes and models

[waynesun09]

Depends on fullsend-ai/agents#1211 (pr-review dispatches sub-agents by persona when the runtime registers them). Without it the subagents block below is inert on pi: the current skill tells the orchestrator to pass the frontmatter model and not set subagent_type, so every child runs anonymously on opus/sonnet (measured, see the first review thread). Order: #1211 merges on maintainer approval → next fullsend release (so release-installed repos pick up the fixed skill through the agents tag; repos on main get it at once) → this PR, right after fullsend-ai/agents#1209 (no waiting period).

What

Persist per-agent runtime and model selection in .fullsend/config.yaml (fullsend v0.42.0 agents: entries, written with fullsend agent set), instead of the fleet-wide opus on Claude Code that every harness carries today. Config in the repo is visible and reviewable; repository variables (<ROLE>_FULLSEND_MODEL) are not, so none are used and rollback is also a PR.

agent	runtime	model	children
code	pi	xai/grok-4.6	—
fix	claude (repo default)	sonnet	—
prioritize	pi	google-vertex/gemini-3.8-flash	—
review	pi	sonnet (orchestrator)	correctness, security, challenger → xai/grok-4.6; docs-currency, style-conventions → google-vertex/gemini-3.8-flash; risk-assessment, intent-coherence, cross-repo-contracts keep their frontmatter sonnet; security-triage keeps haiku
retro	claude	sonnet	its children name no persona and pass no model, so on Claude they inherit sonnet from the parent
triage	claude	sonnet	—

Every line was produced by fullsend agent set … --fullsend-dir .fullsend and validated with fullsend agent list; the only hand edit is reverting the CLI's cosmetic re-serialisation (header comment, status_notifications indentation) so the diff is the agents: block alone.

Why

opus everywhere is the cost driver. code is the from-scratch implementer whose commits are kept, so it gets the more capable model; fix fires after every review round and is the run most often superseded by a human takeover, so it gets sonnet. The review keeps no opus persona at all.

Model alias

models:
    aliases:
        sonnet: claude-sonnet-5

fullsend's pinned default for sonnet is one generation older (claude-sonnet-4-6), so every sonnet above would silently run that without this block. The alias applies to the parent run of every agent on both runtimes and to pi sub-agents. It does not reach Claude Code sub-agents (code, fix, retro children on Claude), which resolve sonnet through the agents harness env pin ANTHROPIC_DEFAULT_SONNET_MODEL — bumping that is a fullsend-ai/agents change, tracked separately. This repo's config is meant to be the worked example other repos copy, which is why the alias is explicit rather than left to the default table.

Merge gates (do not merge before all are ticked)

• The repository's Vertex project serves claude-sonnet-5, xai/grok-4.6 and gemini-3.8-flash — confirmed 2026-09-08 (all three enabled there; probed with a one-token request) (a pinned alias or persona model that cannot be served has no fallback: the review fails at Bootstrap after the sandbox is created, code and the Claude agents fail at the first model call). Or point XAI_VERTEX_PROJECT_ID at a project that does.
• chore(#1193): repin harness images to fullsend 0.42.0 digests agents#1194 merged (a35983272: sandbox images repinned to the 0.42.0 digests — pi 0.85.0, pi-xai-vertex 0.2.1 with the Grok keepalive fix).
• The repin is live on the fleet without any tag move: the dispatch workflows build fullsend from main (a source build carries no release tag), and such a build fetches the built-in agents from fullsend-ai/agents at heads/main, not at v0. Verified on a fleet review run on 2026-09-08 19:29 UTC: fullsend version 41e7a66, agents fetched at a35983272, sandbox image fullsend-code@sha256:623fc745… (the 0.42.0 image). v0 only matters to a release-tagged binary, which the fleet runs just between a fullsend release and the next merge to main.
• fix(pr-review): dispatch sub-agents by persona when the runtime registers them agents#1211 merged 2026-09-09 (4749344e9; pr-review skill dispatches by persona when the runtime lists them). Found by the local review run on 2026-09-08: Bootstrap resolved all nine personas as configured, but the current skill tells the orchestrator to pass model from the frontmatter and not set subagent_type, so every child ran as an anonymous sub-agent on opus/sonnet and the subagents block here was inert. Without feat(e2e): add TOTP support for 2FA-enabled GitHub accounts #1211 this PR changes the review's orchestrator model only.
• Local fullsend run review and fullsend run code with this config on 2026-09-08: Bootstrap resolves all nine personas as configured; with the agents#1211 skill all eight review dispatches go out by persona on the configured models (Grok ×3, Gemini ×2, sonnet-5 ×3), exit 0, $5.34 — at parity with the fleet's own Claude/opus reviews of the same PR ($3.81–$5.91 across six runs); code on Grok passed validation twice ($0.72–$1.00, effort: high accepted, gopls plugin skipped as expected). Full evidence on agents#1209's first review thread.
• chore(config): route agents across runtimes and models agents#1209 merged first (agents is the pilot); this PR follows the same day.

What this PR's own checks do and do not prove

The dispatch workflows check out .fullsend from the base SHA, so the review that runs on this PR uses the current config, not this one. Green here proves the file parses and nothing else; the gates above are the real test.

Rollback

A PR: git revert of this commit, or one fullsend agent set <agent> --fullsend-dir .fullsend --runtime claude --model opus line for a single role.

[qodo-code-review]

PR Summary by Qodo

Route agents across runtimes and models

⚙️ Configuration changes 🕐 10-20 Minutes

AI Description

• Persists reviewable per-agent runtime and model routing in repository configuration.
• Routes agents across Pi, Claude, Grok, Gemini, Sonnet, and Haiku.
• Leaves triage and the repository-wide Claude default unchanged.

Diagram

graph TD
  CONFIG["Repo Config"] --> PROFILES["Agent Profiles"]
  PROFILES -->|code, review, retro| PI["Pi Runtime"] -->|code and reviewers| GROK["xai/grok-4.6"]
  PROFILES -->|fix and prioritize| CLAUDE["Claude Runtime"] -->|primary model| SONNET["Sonnet"]
  PI -->|review personas| GEMINI["Gemini Flash"]
  PI -->|orchestrators| SONNET
  PROFILES -->|retro children| HAIKU["Haiku"]

Loading

High-Level Assessment

The following are alternative approaches to this PR:

1. Repository variables

• ➕ Allows routing changes without modifying tracked files.
• ➕ Can support environment-specific overrides.
• ➖ Runtime and model choices are not visible during code review.
• ➖ Rollback and configuration history are less explicit.
• ➖ Requires maintaining role-specific variable naming conventions.

2. Fleet-wide model defaults

• ➕ Minimizes repository configuration.
• ➕ Keeps all agents on a uniform runtime and model policy.
• ➖ Cannot optimize cost and capability by agent responsibility.
• ➖ Continues using expensive models for frequently superseded runs.
• ➖ Prevents specialized reviewer model assignments.

3. Pinned Sonnet alias

• ➕ Centralizes the concrete Sonnet model version.
• ➕ Simplifies future per-agent version upgrades.
• ➖ An unavailable pinned model fails immediately without fallback.
• ➖ Requires confirmed Vertex project support before adoption.

Recommendation: Use the checked-in per-agent configuration because it makes routing reviewable, versioned, and independently tunable while preserving existing defaults for unspecified agents. Defer the pinned Sonnet alias until model availability is confirmed, and treat repository variables as an operational override rather than the source of truth.

Files changed (1) +21 / -0

Other (1) +21 / -0

config.yamlDefine per-agent runtime and model routing +21/-0

Define per-agent runtime and model routing

• Adds explicit profiles for code, fix, prioritize, review, and retro. Code, review, and retro move to Pi where specified, while Grok, Gemini Flash, Sonnet, and Haiku are assigned according to agent and subagent responsibilities; triage remains unchanged.

.fullsend/config.yaml

[fullsend-ai-review]

🤖 Review · ⚠️ Cancelled · Started 7:09 PM UTC · Ended 7:16 PM UTC

Commit: 1b6ff05 · View workflow run →

[qodo-code-review]

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📎 Requirement gaps (0)

Great, no issues found!

Qodo reviewed your code and found no material issues that require review

Tip of the day

💡 Did you know, you can tweak Display preferences with a live preview to see your comment before it ships

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[fullsend-ai-review]

🤖 Review · ⚠️ Cancelled · Ended 7:18 PM UTC

Commit: 4e549b5 · View workflow run →

[fullsend-ai-review]

🤖 Review · ⚠️ Cancelled · Started 7:20 PM UTC · Ended 7:22 PM UTC

Commit: 030eb13 · View workflow run →

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 7:24 PM UTC · Completed 7:44 PM UTC

Commit: 80dcbef · View workflow run →

Runtime: claude · Model: opus → claude-opus-4-6 · Effort: high · Cost: $4.88

[fullsend-ai-review]

Risk Assessment: low (1/5)

Details

Config-only routing change to a single 28-line file by a known author; no protected paths, security-sensitive files, CI workflows, or dependency changes touched.

[fullsend-ai-review]

Review

Findings

Low

• [missing-authorization] .fullsend/config.yaml — No linked issue authorizes this change. The PR introduces per-agent runtime and model routing, a model alias, and subagent overrides. While traceable authorization is desirable for infrastructure decisions, this is a configuration-only change to a non-protected path with clear rationale in the PR body.

• [scope-coherence] .fullsend/config.yaml:27 — The agents: entry - name: fix (with model: sonnet) is not represented in the roles: list (which includes coder but not fix). Cross-dimension verification confirms this is not a functional issue — the dispatch system maps both code and fix to the coder role via roleAliasHints in config.go and reusable-dispatch.yml, and AgentSettingsFor() resolves agent entries by name independently of the roles list. The asymmetry is cosmetic but may confuse future maintainers.

[rh-hemartin]

LGTM

[fullsend-ai-retro]

🤖 Finished Retro · ✅ Success · Started 2:33 PM UTC · Completed 2:44 PM UTC

Commit: 80dcbef · View workflow run →

Runtime: claude · Model: opus → claude-opus-4-6 · Effort: high · Cost: $3.43

[fullsend-ai-retro]

Retro: PR #7116 — route agents across runtimes and models

Workflow went well. This was a manually-authored, 28-line config-only change to .fullsend/config.yaml that routes agents to different runtimes and models for cost optimization (moving away from fleet-wide opus). The review agent correctly approved with 2 low-severity, non-actionable findings. Both human reviewers also approved. No rework was needed.

What worked

• Cross-dimension reconciliation added value. The correctness sub-agent investigated roleAliasHints and found that coder maps to both code and fix, directly contradicting the scope-coherence finding that the fix entry "may be silently ignored." The orchestrator correctly downgraded this from medium to low. The challenger removed 2 false positives (pre-existing indentation, merge preconditions). This is the review pipeline working as designed.
• Concurrency groups handled force-pushes correctly. Four force-pushes between 19:07–19:20 UTC caused 4 cancelled review runs (~15 min wasted compute), but the concurrency mechanism (cancel-in-progress: true) worked as intended — only the final commit was reviewed.
• Agent and human reviews aligned. The agent approved; both humans approved. No review quality gap.

Observations

• Review cost was disproportionate to scope. The review used opus at high effort with 7 sub-agents (6 dimensions + challenger) for a 28-line config file, costing $4.88. The orchestrator alone consumed ~919K cached tokens across 48 turns. Ironically, this PR is about reducing agent costs — once merged, future reviews use sonnet on the pi runtime. This is corroborating evidence for #6891 (pre-flight scope heuristic to adjust review effort for trivially-scoped PRs) and #5777 (review model routing using diff complexity).
• Model availability fallback. The review log showed "Opus 5 not available — using Opus 4.6." Since this PR pins specific models (including non-Anthropic ones like xai/grok-4.6 and google-vertex/gemini-3.8-flash), model unavailability could impact future runs. This is tracked by #7154 (fallback when configured model is unavailable), #7026 (pi fallback on aliased model unavailability), and #6883 (pre-sandbox availability preflight).
• Force-push debouncing is tracked by #4960 and #7107. The ~15 min of wasted compute here is a minor cost but adds up across many PRs.

No new proposals

All candidate improvement areas are well-covered by existing open issues. No novel proposals are warranted from this retro.