offload: summarize's kept-verbatim trim can force a re-summary, and that cache-write is not counted by expand_prefix_flips

[amiddavid]

The gap

expand_prefix_flips (#208) counts one event: a replay declined because the agent had expanded that content (components/offload.reapplyFrozen, its only increment site). #208 also added a second event of the same class, and it is uncounted.

trimSpanForKeptVerbatim lowers summarize's end so the span never contains expanded content. A checkpoint whose boundary reached past the new end then fails tryReuse's boundary > end guard, so summarize re-summarizes a shorter span from scratch — different summary text at a fixed prefix position, which is a suffix cache-write at ~11.5× a read.

That is the correct trade: losing the agent's expanded content is worse than one cache-write, and re-compacting it would be the bounce loop cg:keep: exists to prevent. But it means a deployment can pay expansion-induced prefix churn while expand_prefix_flips reads zero.

Why it was not counted in #208

A second increment site changes what the number means. Today it answers "how often did a replay decline because of an expand", which is per turn per message and already needs a caveat documented in three places. Adding "and also, how often did a checkpoint get invalidated by the trim" makes it a mixed total that no single reading covers — the ambiguous-counter shape #200 states as a rule.

So #208 narrowed the wording instead: metrics.Snapshot, docs/reference/routes.md and the counter's own doc all now say it counts one event and that a zero does not mean expansion cost nothing. This issue is the decision that wording defers.

The options

1. A second counter. expand_resummaries or similar, incremented where the trim shortens a span that had a checkpoint reaching past the new end. Keeps each number readable; costs a second series for a rarer event.
2. One counter with a label. cg_expand_prefix_flips_total{cause="replay_declined|checkpoint_invalidated"}. One series, decomposable — but the /stats JSON field would have to become an object or gain a sibling, and that is a contract change.
3. Leave it uncounted and documented, the current state. Defensible while the event is believed rare — but nobody has measured how rare, and the trim is new.

What would settle it

Whether the trim actually invalidates checkpoints in practice, which needs resummarize_tokens > 0 (checkpointing on) plus an agent that expands content inside the summarizable middle. kept_verbatim_after_expand on summarize's gate row is the leading indicator: non-zero there with resummarize_tokens set is the population at risk.

How it was found

Review of #208 (round 2), by the reviewer tracing what lowering end does to tryReuse's boundary > end guard. Related: #201 (the flip), #200 (the counter-splitting rule), #204.

[amiddavid]

The indicator this issue names has a tighter meaning than it did before #208

Recording it here while the reason is fresh, because from this issue's side it looks like an unrelated ordering change.

kept_verbatim_after_expand on summarize's gate row is raised only below the trigger gate as of #208's round-2 commit (18db066). So on summarize specifically it now counts turns where summarize would have acted and the trim stopped it, rather than every turn carrying expanded content in the span — including the many where the trigger declined and summarize was never going to summarize anything.

That is a tighter denominator, and a better one for this issue: the population at risk of a forced re-summary is exactly the turns where summarize was going to act, since a declined turn invalidates no checkpoint. Reading the pre-18db066 figure as the at-risk population would over-count it.

Two caveats on using it:

• It is a per-component gate. The same label on the other ten offloaders' rows means the ordinary skip (skipReduce declining to re-compact expanded content) and is unrelated to checkpoint invalidation — read summarize's row, not the total.
• It is still an upper bound, not the event. A trim only forces a re-summary when a checkpoint's boundary reached past the new end; with resummarize_tokens: 0 (checkpointing off, which is the default in several presets) there is no checkpoint to invalidate and the answer is zero however high the gate reads.

So the measurement that would settle option 3 is: summarize's kept_verbatim_after_expand non-zero and resummarize_tokens > 0 and frozen_decisions{outcome="hit"} on summarize not tracking it — the last being the tell that checkpoints stopped being reused.

Credit for spotting that this issue's indicator moved: the #208 reviewer.

[amiddavid]

The third clause of the settling measurement cannot be taken

Checking the addition above, as asked. The tighter-denominator point is right and the resummarize_tokens: 0 caveat is right. The third clause is not: frozen_decisions{outcome="hit"} is structurally blind to summarize, so "summarize's frozen-decision hits not tracking it" will read the same whether checkpoints are being reused or not.

Two independent reasons:

1. The counter never sees summarize. cg_frozen_decisions_total is sourced from offload.FrozenStats(), i.e. frozenHits/frozenMisses. Those have exactly four increment sites — state.go:65, :70, :73 (getResult, extract_llm's replay lookup) and state.go:237, :240 (reapplyFrozen). Summarize uses neither path: its checkpoint read is loadCheckpoint at state.go:467, a bare c.Store.Get(sumKey(c.Session)) with no counter on either outcome. A reused checkpoint and a refused one are equally invisible.

2. It could not be read per component even if it did. promexport.go:487-491 emits cg_frozen_decisions_total through promHeaderProc with an outcome label and nothing else — it is process-wide, like the hit/miss pair it comes from. There is no summarize row to read.

I checked for any other shipped signal for checkpoint reuse and there is none: rep.Irreversible on a non-full reuse (summarize.go:196) is a report flag, not a series, and rep.Event("reserve_exhausted_replayed_checkpoint") (:369) is a different condition.

What that does to the options

Option 3 — leave it uncounted and documented — cannot be settled with shipped signals, because the tell it rests on does not exist. That is not an argument against option 3 as a resting state; it is an argument that choosing it means choosing not to find out.

It also makes option 1 stronger than the issue frames it. "Costs a second series for a rarer event" undersells it: the second counter is the instrument this issue says it needs, so it resolves the question and the measurement in one move.

If a counter is added, "a checkpoint was not reused" is the better subject

Narrower than it sounds, and it matters. The same cache-write — summarize re-summarizing and emitting different summary text at a fixed prefix position — has at least two other causes, both already documented at store.go:428-431, in the joint-budget saturation note:

• cg:sum: evicted → "summarize can never checkpoint, so it re-pays its model call every turn and never reuses" — a re-summary every turn, not once.
• cg:keep: evicted → "content the agent just expanded is re-compacted, which is the expand loop the flag exists to stop" — which also silently disables trimSpanForKeptVerbatim, since the trim's predicate is isKeptVerbatim. So under that pressure the protection this issue is about stops working, and a trim-specific counter would read zero at exactly the moment the problem is worst.

A counter on "a checkpoint existed and was not reused, with the reason" distinguishes trim-invalidated from evicted from stale by construction, and keeps expand_prefix_flips meaning the one thing it means today. A trim-specific counter answers only the case that is easiest to reason about and stays silent on the two that arrive under load.

Neither eviction case is new or introduced by #208 — both are documented already. The point is only that they share the event this issue proposes to count, so the counter's subject should be the event and not the cause.

Found while checking the comment above at the author's request; the ordering-change credit there is theirs, not mine.

[amiddavid]

Correction: the third clause of my previous comment names a measurement that does not exist

I wrote that the settling measurement was summarize's kept_verbatim_after_expand non-zero and resummarize_tokens > 0 and frozen_decisions{outcome="hit"} on summarize not tracking it. The first two hold. The third is not measurable, and I checked rather than took the report:

• frozenHits/frozenMisses have four increment sites — state.go:65, :70, :73 in getResult, and :237, :240 in reapplyFrozen. summarize reaches neither path (grep -c 'getResult\|reapplyFrozen' components/offload/summarize.go → 0). Its checkpoint read is loadCheckpoint (state.go:467), a bare Get(sumKey(...)) with no counter on hit or miss.
• cg_frozen_decisions_total is emitted process-wide with only an outcome label (promexport.go:487-491). There is no per-component dimension, so there would be no summarize row to read even if the counter moved.

So there is no shipped signal for checkpoint reuse. rep.Irreversible on a non-full reuse is a report flag rather than a series, and reserve_exhausted_replayed_checkpoint is a different condition.

That changes the ranking, not just the line

Option 3 (leave it uncounted and documented) cannot be settled with shipped signals, because the tell I proposed does not exist. That is not an argument against option 3 as a resting state — it is a statement that choosing it means choosing not to find out.

And it makes option 1 stronger than this issue framed it. The second counter is not just the fix; it is the instrument the issue says it needs. It settles the question and the measurement in one change, which no other option does.

If option 1 is taken, the counter's subject should be the EVENT, not the cause

Name it "a checkpoint was not reused", not "the trim invalidated one". The same cache-write has two other causes, both already documented in the joint-budget note at store.go:428-431, and neither is pinned (DefaultPinPrefixes at store.go:178 covers cg:frz:, cg:res:, cg:len:, cg:xres:, cg:ttl:, cg:seen: — not these two):

• cg:sum: evicted → summarize can never checkpoint, so it re-summarizes every turn. Strictly worse than the trim's occasional invalidation.
• cg:keep: evicted → expanded content is re-compacted, which is the bounce loop cg:keep: exists to prevent — and it silently disables the trim, because the trim's predicate is isKeptVerbatim.

That second one is the reason to name the event: under cg:keep: pressure a trim-specific counter reads zero exactly when the problem is worst. A counter whose value goes to zero as the harm peaks is worse than no counter, and it is the same ambiguous-signal failure #200 states as a rule, reached from the direction of a false negative rather than a false positive.

Neither of those two causes is new and neither comes from #208 — they just share the event, which is precisely why the counter should be keyed on it.

Credit for the correction, and for the event-vs-cause framing: the #208 reviewer.