feat(agent): gemma4:31b clears all six, and is the slowest model that does - #628
Merged
Conversation
… does Twenty-seven supported models becomes twenty-eight: 840 runs, 840 passes. Investigate 26s · Optimize 378s · Assess 214s · Operate 93s · Analyze 132s · Plan 3s, a 107-second median and a 398-second slowest run — every figure the largest on this roster. Two settings, and they were earned separately. Its plan cell read 0/5 at the defaults, losing to the clock rather than to the plan: the model spends the turn thinking and the deliverable never arrives. Quiet, it answers in three seconds — the fastest plan turn in this family — and the cell locked on the second attempt. Optimize is the other, and it is this model's wall as well as its clock: a passing run there takes six minutes against a 26-second investigation, and at the compiled 90-second per-turn ceiling the cell would not close at all. Earning them separately is what made a second sweep necessary. `suppressPlanReasoning` reaches plan turns only — `investigation.ts` reads it where the mode is not `agent` — so the five agent cells that locked "with" it were unaffected by it, and all five locked at the 90-second ceiling while the profile that ships carries 150. Raising a per-call ceiling is not obviously free: a turn cut at 90 is re-asked, while a turn allowed 150 spends that from the run's own deadline instead. That trade is a measurement rather than an argument, so the five were read again under the pair that ships — twenty-five runs, every surface 5/5, and the figures above are from those runs. It does not carry `suppressAgentReasoning`, which its 12b sibling does. That switch was measured for an illness this size does not have, and a setting a model did not earn is a guess. `docs/llms/README.md` carried two claims this model falsifies and one it inherited. The roster count, the run total and the 150-second enumeration all move. The speed ratio moves with them — 6s to 107s is an eighteenth, not the twelfth it read. And the "fastest local model" row still named `granite4.1:8b` at 11s while `qwen2.5:7b` has sat at the top of the same table at 6s since it landed; the row is now the model the table already says is fastest. `gemma4:31b` also goes into the sweep list in `scripts/agent-model-e2e.sh`, which drives every supported model through the rail a person actually clicks.
yusuf-gundogdu
force-pushed
the
feat/gemma4-31b
branch
from
September 7, 2026 13:47
c2aeb8d to
6ef95b1
Compare
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.



Twenty-seven supported models becomes twenty-eight: 840 runs, 840 passes.
107-second median, 398-second slowest run, 19 GB. Every one of those is the largest on the
roster, and the family page says so plainly: three more gigabytes than the 26b cost it three times
the median and six minutes on one cell, and every verdict it reaches the 12b reaches too, in a
tenth of the time.
Two settings, earned separately
suppressPlanReasoning— its plan cell read 0/5 at the defaults, losing to the clock ratherthan to the plan: the model spends the turn thinking and the deliverable never arrives. Quiet, it
answers in three seconds, the fastest plan turn in this family, and the cell locked on the second
attempt.
turnTimeoutMs: 150000— optimize is this model's wall and its clock. A passing run theretakes six minutes against a 26-second investigation, and at the compiled 90-second per-turn ceiling
the cell would not close.
It does not carry
suppressAgentReasoning, which its 12b sibling does. That switch wasmeasured for an illness this size does not have, and a setting a model did not earn is a guess.
Why a second sweep, and what it measured
Earning the two separately is exactly what made the first result unshippable.
suppressPlanReasoningreaches plan turns only —investigation.ts:2331reads it where the modeis not
agent— so the five agent cells that locked "with" it were unaffected by it, and allfive locked at the 90-second ceiling while the profile that ships carries 150.
Raising a per-call ceiling is not obviously free. A turn cut at 90 is re-asked; a turn allowed 150
runs on and spends that from the run's own deadline instead. Which trade is better is a
measurement, not an argument, so the five surfaces were read again under the pair that ships:
Twenty-five runs, every surface 5/5. The figures in the table above are from those runs. Optimize
was not re-read — it already locked at exactly this set, since the switch it lacked cannot reach it.
Three stale lines in
docs/llms/README.mdTwo this model falsifies (the roster count and the 150-second enumeration, which is now seven
models) and one it inherited:
eighteenth.
granite4.1:8bat 11s, whileqwen2.5:7bhas sat at thetop of the same table at 6s since it landed. The row now names the model the table already says
is fastest.
Verification
Every assertion added here was mutated to confirm it bites:
turnTimeoutMs: 150_000→90_000in the pinned rowsuppressesPlanReasoning: trueremoved from the pinned rowmeasuredstring (378s→379s)planStatementRetries: 0is measured rather than inherited: all five plan runs were driven throughthe operator document that states it, and the ledger shows none of them consumed the plan ask.
format·lint·typecheck·knip·test(38 groups, 0 fail) ·build·build:lib·attw·chart:check·channels:showcase:check·distribution:matrix --check— all green.Coverage: 45991/45991 lines, 100.00%.
No file here is touched by any open PR (#625, #626, #627, #571).