Skip to content

feat(agent): gemma4:31b clears all six, and is the slowest model that does - #628

Merged
cevheri merged 1 commit into
mainfrom
feat/gemma4-31b
Sep 7, 2026
Merged

feat(agent): gemma4:31b clears all six, and is the slowest model that does#628
cevheri merged 1 commit into
mainfrom
feat/gemma4-31b

Conversation

@yusuf-gundogdu

Copy link
Copy Markdown
Member

Twenty-seven supported models becomes twenty-eight: 840 runs, 840 passes.

Surface Investigate Optimize Assess Operate Analyze Plan
Median of the passing runs 26s 378s 214s 93s 132s 3s

107-second median, 398-second slowest run, 19 GB. Every one of those is the largest on the
roster, and the family page says so plainly: three more gigabytes than the 26b cost it three times
the median and six minutes on one cell, and every verdict it reaches the 12b reaches too, in a
tenth of the time.

Two settings, earned separately

suppressPlanReasoning — its plan cell read 0/5 at the defaults, losing to the clock rather
than to the plan: the model spends the turn thinking and the deliverable never arrives. Quiet, it
answers in three seconds, the fastest plan turn in this family, and the cell locked on the second
attempt.

turnTimeoutMs: 150000 — optimize is this model's wall and its clock. A passing run there
takes six minutes against a 26-second investigation, and at the compiled 90-second per-turn ceiling
the cell would not close.

It does not carry suppressAgentReasoning, which its 12b sibling does. That switch was
measured for an illness this size does not have, and a setting a model did not earn is a guess.

Why a second sweep, and what it measured

Earning the two separately is exactly what made the first result unshippable.
suppressPlanReasoning reaches plan turns only — investigation.ts:2331 reads it where the mode
is not agent — so the five agent cells that locked "with" it were unaffected by it, and all
five locked at the 90-second ceiling while the profile that ships carries 150
.

Raising a per-call ceiling is not obviously free. A turn cut at 90 is re-asked; a turn allowed 150
runs on and spends that from the run's own deadline instead. Which trade is better is a
measurement, not an argument, so the five surfaces were read again under the pair that ships:

plan        5/5
investigate 5/5
operate     5/5
analyze     5/5
assess      5/5

Twenty-five runs, every surface 5/5. The figures in the table above are from those runs. Optimize
was not re-read — it already locked at exactly this set, since the switch it lacked cannot reach it.

Three stale lines in docs/llms/README.md

Two this model falsifies (the roster count and the 150-second enumeration, which is now seven
models) and one it inherited:

  • The speed ratio read "a twelfth of the time". Fastest median 6s against slowest 107s is an
    eighteenth
    .
  • The "fastest local model" row named granite4.1:8b at 11s, while qwen2.5:7b has sat at the
    top of the same table at 6s since it landed. The row now names the model the table already says
    is fastest.

Verification

Every assertion added here was mutated to confirm it bites:

Mutation Result
turnTimeoutMs: 150_00090_000 in the pinned row 1 fail
suppressesPlanReasoning: true removed from the pinned row 1 fail
One character changed in the measured string (378s379s) 1 fail

planStatementRetries: 0 is measured rather than inherited: all five plan runs were driven through
the operator document that states it, and the ledger shows none of them consumed the plan ask.

format · lint · typecheck · knip · test (38 groups, 0 fail) · build · build:lib ·
attw · chart:check · channels:showcase:check · distribution:matrix --check — all green.
Coverage: 45991/45991 lines, 100.00%.

No file here is touched by any open PR (#625, #626, #627, #571).

… does

Twenty-seven supported models becomes twenty-eight: 840 runs, 840 passes. Investigate 26s ·
Optimize 378s · Assess 214s · Operate 93s · Analyze 132s · Plan 3s, a 107-second median and a
398-second slowest run — every figure the largest on this roster.

Two settings, and they were earned separately. Its plan cell read 0/5 at the defaults, losing to
the clock rather than to the plan: the model spends the turn thinking and the deliverable never
arrives. Quiet, it answers in three seconds — the fastest plan turn in this family — and the cell
locked on the second attempt. Optimize is the other, and it is this model's wall as well as its
clock: a passing run there takes six minutes against a 26-second investigation, and at the
compiled 90-second per-turn ceiling the cell would not close at all.

Earning them separately is what made a second sweep necessary. `suppressPlanReasoning` reaches
plan turns only — `investigation.ts` reads it where the mode is not `agent` — so the five agent
cells that locked "with" it were unaffected by it, and all five locked at the 90-second ceiling
while the profile that ships carries 150. Raising a per-call ceiling is not obviously free: a turn
cut at 90 is re-asked, while a turn allowed 150 spends that from the run's own deadline instead.
That trade is a measurement rather than an argument, so the five were read again under the pair
that ships — twenty-five runs, every surface 5/5, and the figures above are from those runs.

It does not carry `suppressAgentReasoning`, which its 12b sibling does. That switch was measured
for an illness this size does not have, and a setting a model did not earn is a guess.

`docs/llms/README.md` carried two claims this model falsifies and one it inherited. The roster
count, the run total and the 150-second enumeration all move. The speed ratio moves with them —
6s to 107s is an eighteenth, not the twelfth it read. And the "fastest local model" row still
named `granite4.1:8b` at 11s while `qwen2.5:7b` has sat at the top of the same table at 6s since
it landed; the row is now the model the table already says is fastest.

`gemma4:31b` also goes into the sweep list in `scripts/agent-model-e2e.sh`, which drives every
supported model through the rail a person actually clicks.
@sonarqubecloud

sonarqubecloud Bot commented Sep 7, 2026

Copy link
Copy Markdown

@cevheri
cevheri merged commit d200d7e into main Sep 7, 2026
27 checks passed
@cevheri
cevheri deleted the feat/gemma4-31b branch September 7, 2026 14:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants