Skip to content

fix(agent): a stop that read nothing is answered for whoever stopped - #635

Merged
cevheri merged 1 commit into
mainfrom
fix/unread-stop-is-the-servers-own
Sep 7, 2026
Merged

fix(agent): a stop that read nothing is answered for whoever stopped#635
cevheri merged 1 commit into
mainfrom
fix/unread-stop-is-the-servers-own

Conversation

@yusuf-gundogdu

Copy link
Copy Markdown
Member

The drive holds a sentence written for exactly one ending:

Read it yourself. Call inspect_schema for the tables and their columns, and inspect_plan for
how a statement will run, then call compose_report with what you found.

It was sent only to a model whose profile asked for it. One profile of twenty-eight does. A
model nobody has measured has no profile at all, so the model most in need of the sentence was the
one guaranteed not to receive it.

What the ledgers say

Measured across the sweep behind 0.14.1:

runs
ended model-stopped with no-report 300
of those, having called no tool at all 150
of those, ended with the drive saying nothing 94

By model: mistral-small3.1:24b 51 · granite3.3:8b 41 · mistral:7b 40 · gpt-oss:20b 18.
None of the four is on the shipped roster — they are exactly the unmeasured models the sentence
could not reach. A hundred and ten of the hundred and fifty were on investigation, the first
surface a model meets.

Why the bound was protecting nothing

The gate's own comment had already argued the cost away. It fires only where anyToolCalled is
false, and compose_report is one of the tools that counts, so a run reaching it composed no report
and has already earned no-report — in the comment's words, "the turn cannot cost a pass."

A bound that cannot protect a passing run is not protecting anything. What this one withheld was a
sentence on a run already lost.

The change

investigation.ts now reads answersUnreadStop rather than retriesUnreadStop. The two differ on
one thing:

profile says before after
retryUnreadStop: true sentence sent sentence sent
retryUnreadStop: false not sent not sent — a measurement, obeyed
no profile at all not sent sent — an absence is not a measurement

This is the same split planStatementAsksFor made beside planStatementRetriesFor, and the fourth
setting found in this shape. All twenty-eight shipped entries state the field, so no shipped
model's turn count moves — verified against measured-profiles.json rather than assumed.

Nineteen red tests, which is the point

Nineteen existing tests turned red, and every one of them is a run that stops having called
nothing — the change reaching exactly what it was aimed at rather than spilling past it. Each gets
the extra scripted turn the drive now spends. tests/evals/real-model.ts holds the shared warm-up
that accounted for several of them at once.

The pair that pins the new behaviour lives beside the tests for the gate it belongs to, replacing
the one that asserted the old contract. Both halves were confirmed to bite:

Mutation Result
an absent profile read as false again 1 fail
a stated false ignored 1 fail

Docs

docs/llms/model-tuning.md gains the section this makes necessary: the last column of the settings
table is the compiled default, and two settings answer differently for a model with no entry at
all. docs/AGENT.md, docs/BACKLOG.md B65 and the notes on profile.ts and models/index.ts are
corrected where they said the absent case resolves to the defaults.

Gates

Every step of the Lint, Typecheck and Build job, in order: chart:check (strict) ·
channels:showcase:check · readme:check · security:check · format · lint · typecheck ·
knip · build · build:lib · attw — all green. test:coverage and plain test both exit 0.
Coverage: 45995/45995 lines, 100.00%.

Rebased onto main after #633 merged; no file here is touched by any other open PR.

The drive holds a sentence written for exactly one ending — "Read it yourself. Call inspect_schema
for the tables and their columns, and inspect_plan for how a statement will run" — and sent it only
to a model whose profile asked for it. One profile of twenty-eight does. A model nobody has
measured has no profile at all, so the model most in need of the sentence was the one guaranteed
not to receive it.

Measured across the sweep behind 0.14.1: three hundred runs ended `model-stopped` with `no-report`,
and half of them — a hundred and fifty — had called no tool at all. Ninety-four of those hundred
and fifty were ended without the drive saying anything: `mistral-small3.1:24b` 51,
`granite3.3:8b` 41, `mistral:7b` 40, `gpt-oss:20b` 18, none of them on the shipped roster. A hundred
and ten of the hundred and fifty were on `investigation`, the first surface a model meets.

The gate's own comment had already argued the cost away. It fires only where `anyToolCalled` is
false, and `compose_report` is one of the tools that counts, so a run reaching it composed no report
and has already earned `no-report` — "the turn cannot cost a pass". A bound that cannot protect a
passing run is not protecting anything, and what this one withheld was a sentence on a run already
lost.

So the gate now reads `answersUnreadStop` rather than `retriesUnreadStop`. The two differ on one
thing: a profile stating `false` is a measurement and is obeyed, while no profile at all is the
absence of a measurement and is answered with the turn. This is the same split
`planStatementAsksFor` made beside `planStatementRetriesFor`, and the fourth setting found in this
shape. All twenty-eight shipped entries state the field, so no shipped model's turn count moves.

Nineteen existing tests turned red on it, every one of them a run that stops having called nothing —
which is the change reaching exactly what it was aimed at. Each gets the extra scripted turn the
drive now spends; `tests/evals/real-model.ts` holds the shared warm-up that accounted for several.
The pair that pins the behaviour lives beside the tests for the gate it belongs to, and both halves
were confirmed to bite by mutation: reading an absent profile as `false` again fails one, ignoring a
stated `false` fails the other.

`docs/llms/model-tuning.md` gains the section this makes necessary — the last column of the settings
table is the compiled default, and two settings answer differently for a model with no entry — and
`docs/AGENT.md`, `docs/BACKLOG.md` B65 and the notes on `profile.ts` and `models/index.ts` are
corrected where they said the absent case resolves to the defaults.
@codecov

codecov Bot commented Sep 7, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@sonarqubecloud

sonarqubecloud Bot commented Sep 7, 2026

Copy link
Copy Markdown

@cevheri
cevheri merged commit f96d63b into main Sep 7, 2026
27 checks passed
@cevheri
cevheri deleted the fix/unread-stop-is-the-servers-own branch September 7, 2026 18:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants