Skip to content

fix(offload): let a replayed summary/off decision declare its loss, or the pipeline reverts it - #194

Merged
amiddavid merged 1 commit into
mainfrom
fix/summary-mode-replay-revert
Sep 3, 2026
Merged

fix(offload): let a replayed summary/off decision declare its loss, or the pipeline reverts it#194
amiddavid merged 1 commit into
mainfrom
fix/summary-mode-replay-revert

Conversation

@amiddavid

Copy link
Copy Markdown
Collaborator

Closes #193.

The defect

Under marker_mode: summary or off, an offloader takes a deliberate lossy drop: nothing is stashed and no <<cg:HASH>> is written, so it returns no cache keys. components/pipeline.go:135 treats that combination — the request shrank, no cache keys, not Skipped — as a contract violation and reverts the component, unless rep.Irreversible says the loss was chosen.

commitMark's non-full branch sets that flag, so the turn that takes the decision is fine. Every later turn replays it through reapplyFrozen, which never did. Measured on a two-turn mask fixture at marker_mode: summary, same content both turns:

turn 1 (fresh, via commitMark):  keys=[] Irreversible=true   -> kept
turn 2 (replay, reapplyFrozen):  keys=[] Irreversible=false  -> REVERTED

So from turn 2 onward, for the rest of the session, the component is discarded and the transcript is forwarded verbatim.

Why it is worse than a lost saving

Earlier turns sent the reduced bytes. Sending the original back re-writes the provider's whole cached suffix at ~11.5x the read price — every turn, for every message the component had reduced. That is the cache-destructive direction reapplyFrozen exists to avoid, reached through reapplyFrozen.

It is also invisible from the component's side: it acts, it computes a replacement, and the pipeline discards the work afterwards. Nothing counts reverts per component, so a summary-mode deployment reads as one that is simply saving less than hoped.

Scope

Affects mask, collapse, failed_run, readlifecycle, skeleton, cmdfilter and agentdiet whenever configured with marker_mode: summary or off. The default (full) is unaffected — a full-mode frozen replacement always carries a marker, so keys is non-empty and the precondition never holds.

The fix

reapplyFrozen takes rep and sets rep.Irreversible when the replayed replacement carries no markers. Threading rep through the seven call sites is mechanical.

len(keys) == 0 is a sound test for "degraded mode" rather than a proxy for it: every freeze() site is downstream of a tryMark/commitMark pair, so a full-mode frozen replacement always carries a marker — which means the blanket flag cannot mask a full-mode bug. That was the property worth checking before choosing a blanket flag over threading the mode through the store.

Alternatives considered

  • Widen pipeline.go's exemption to "no keys, but the replacement is a pure function of the original". Rejected: the pipeline cannot know that, and it weakens the guard that catches a component which genuinely forgot to stash.
  • Record the marker mode on the frozen entry and consult it on replay. More explicit, but it changes the store's value format for every frozen key and buys nothing over the marker test.

Why this is off main and not part of #188

Found while auditing replay paths for the #188 review, which flagged the same omission in two new replay branches introduced there. This instance is independent and predates that work — at the merge base (51fcd91) reapplyFrozen takes no rep parameter at all.

#188 still carries the open reserve-lifetime question in #190, so folding this in would mean a bug that is live on main today ships only when that lands. #188 contains the same three lines and will be rebased onto this once it merges; the reviewer on #188 independently reached the same conclusion about separating it.

Verification

gofmt -l . clean · CGO_ENABLED=1 go vet ./... clean · CGO_ENABLED=1 go test ./... all packages pass (Go 1.26.4, eval box).

TestASummaryModeReplayIsNotRevertedFromTurnTwoOnward asserts turn 2's exact conjunction. Turn 1 is asserted first, so a fixture that stopped reaching summary mode fails there rather than passing vacuously below — and the test also requires that turn 2 actually rewrote the message and did not report Skipped, since either would make the revert precondition unreachable.

Verified by reverting its subject:

a replayed summary-mode decision rewrote the message, returned NO cache key and did not
set rep.Irreversible. That is exactly what components/pipeline.go reverts as "offload
dropped content without stashing a cache_key" ...

…r the pipeline reverts it

Closes #193.

Under marker_mode summary/off an offloader takes a deliberate LOSSY drop: nothing
is stashed and no <<cg:HASH>> is written, so it returns no cache keys.
components/pipeline.go:135 treats that combination — the request shrank, no cache
keys, not Skipped — as a contract violation and REVERTS the component, unless
rep.Irreversible says the loss was chosen.

commitMark's non-full branch sets that flag, so the turn that TAKES the decision
is fine. Every later turn replays it through reapplyFrozen, which never did.
Measured on a two-turn mask fixture at marker_mode: summary, same content both
turns:

	turn 1 (fresh, via commitMark):  keys=[] Irreversible=true   -> kept
	turn 2 (replay, reapplyFrozen):  keys=[] Irreversible=false  -> REVERTED

So from turn 2 onward, for the rest of the session, the component is discarded
and the transcript is forwarded verbatim.

That is worse than a lost saving. Earlier turns sent the REDUCED bytes, so
sending the original re-writes the provider's whole cached suffix at ~11.5x the
read price — every turn, for every message the component had reduced. It is the
cache-destructive direction reapplyFrozen exists to avoid, reached through
reapplyFrozen. And it is invisible from the component's side: it acts, computes a
replacement, and the pipeline throws the work away afterwards.

Affects mask, collapse, failed_run, readlifecycle, skeleton, cmdfilter and
agentdiet whenever configured with marker_mode summary or off. The default (full)
is unaffected: a full-mode frozen replacement always carries a marker, so keys is
non-empty and the precondition never holds.

len(keys) == 0 is a sound test for "degraded mode" rather than a proxy: every
freeze() site is downstream of a tryMark/commitMark pair, so a full-mode frozen
replacement always carries a marker — which means the blanket flag cannot mask a
full-mode bug. Threading rep through the seven call sites is mechanical.

Found while auditing replay paths for the #188 review, which flagged the same
omission in two NEW replay branches introduced there. This instance predates that
work — at the merge base reapplyFrozen takes no rep parameter at all — so it is
fixed here, off main, rather than shipping only when #188 does.

Verification: gofmt -l . clean, go vet ./... clean, go test ./... all packages
pass (Go 1.26.4, eval box).

TestASummaryModeReplayIsNotRevertedFromTurnTwoOnward asserts turn 2's exact
conjunction, with turn 1 asserted first so a fixture that stopped reaching
summary mode fails loudly instead of passing vacuously. Verified by reverting the
subject: the test fails with

	a replayed summary-mode decision rewrote the message, returned NO cache key
	and did not set rep.Irreversible

Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com>
Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@amiddavid

Copy link
Copy Markdown
Collaborator Author

Context for reviewers: this is split out of #188 on that PR's review recommendation. #188 currently contains the same three lines and will be rebased onto main once this lands, at which point its state.go diff reduces to a single call-site change. The defect itself is independent of #188 and predates it — verified at the merge base 51fcd91, where reapplyFrozen takes no rep parameter at all. Filed as #193.

@amiddavid amiddavid left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed and verified independently, on the box (Go 1.26.4, CGO_ENABLED=1). This is the right shape for the split, and I found nothing to fix. (Recording as a comment rather than an approval only because GitHub will not let this account approve its own PR.)

The split is clean. Based on current origin/main, and the diff is exactly what a main-defect fix should be: 22 lines in state.go of which 3 are logic, 7 mechanical call-site updates threading rep, one existing test adjusted for the signature, and one new test. No reserve work rode along.

Verified:

  • gofmt -l . clean, go vet ./... clean, go test ./... clean — no failures anywhere, dash included.
  • TestASummaryModeReplayIsNotRevertedFromTurnTwoOnward passes, and revert-verified: reverting the fix to _ = rep compiles clean (go vet silent) and the named test fails. So the mutation is a real revert of the bug, not a build break — and go test -run resolved the name (=== RUN emitted), so it is also not the "no tests to run exits 0" trap.
  • The blanket rep.Irreversible cannot mask a full-mode bug: every freeze() site is downstream of a tryMark/commitMark pair, so a full-mode frozen replacement always carries a marker and len(keys) == 0 genuinely implies a degraded mode. The comment states this; I checked all 7 sites.
  • All 7 reapplyFrozen callers updated.
  • Confirmed the defect predates #188: at the merge base reapplyFrozen took no rep parameter at all and did a bare c.Store.Put(k, ...), so summary-mode replacements have been reverted from turn 2 onward independently of any reserve work.

On sequencing: keeping the same lines in #188 until this lands, then rebasing, is the right call — stripping them pre-emptively would make #188 red for reasons unrelated to its own review. The state.go conflict between the two branches is expected; noting it here so it is on the record rather than looking like drift later.

The comment carries the measured evidence (turn 1 Irreversible=true, turn 2 false with the message rewritten and no keys) and explains why the pipeline reverts that combination, which is what a reader needs to not reintroduce it.

Generated with Claude Code

@amiddavid

Copy link
Copy Markdown
Collaborator Author

Thank you for the independent verification, and particularly for revert-verifying it yourself rather than taking my word for it — including checking that the mutation compiles and that go test -run actually resolved the name. Those are the two traps that produced false evidence three times over on #188, so having them checked from the other side is worth more than the fix.

Nothing to change here. Recording what happens next so the sequencing is on the record:

One point of order on your last paragraph. You describe the comment as carrying "the measured evidence (turn 1 Irreversible=true, turn 2 false…)" — worth being precise that the numbers in the commit message and PR body are from my two-turn probe on a mask fixture, and yours independently reproduced the same shape. Two runs agreeing is stronger than one, and it should read that way rather than as a single measurement quoted twice.

@amiddavid

Copy link
Copy Markdown
Collaborator Author

Taking the correction — and it strengthens the record rather than weakening it, so worth stating precisely.

You're right that the numbers quoted in the state.go comment are from your probe, and my write-up above should have said so. What I actually did was write my own two-turn markerSummary fixture against extract_llm during the round-2 review, before this split existed, and run it on a67c597; it moved from turn 2 irreversible=false to turn 2 irreversible=true. Then for this PR I revert-verified your test independently — mutating the fix to _ = rep, confirming go vet stayed silent, and confirming the named test resolved and failed.

So the evidence for this defect is two independently written probes, on different fixtures, agreeing on the shape — yours on mask via reapplyFrozen, mine on extract_llm's replay branch — plus a mutation showing the fix is what makes the difference. That is a materially stronger claim than one measurement quoted twice, and it is the reason I'd treat this defect as settled rather than merely plausible.

Recording it that way here so the PR carries the accurate provenance.

@amiddavid
amiddavid merged commit 764c1ed into main Sep 3, 2026
5 checks passed
@github-project-automation github-project-automation Bot moved this from New/ToDo to Done in Rossoctl Issue Prioritization Sep 3, 2026
amiddavid added a commit that referenced this pull request Sep 3, 2026
…ndant

Comment only; no behaviour and no test outcome changes.

main's TestASummaryModeReplayIsNotRevertedFromTurnTwoOnward (from #194) and this
branch's `mask via reapplyFrozen` subtest assert the same thing, and the obvious
reading is that one is redundant. They are not: they have different SUBJECTS that
happen to share an assertion.

  - the standalone test asserts a property of reapplyFrozen.
  - the subtest asserts that all four replay branches are held to the SAME
    assertion through the same helper — uniformity, not the property.

They fail differently, which is the test of whether both are needed. Special-case
the helper so it suits extract_llm and extract_sweep_drop but not reapplyFrozen
and the table catches it while the standalone test stays green; break
reapplyFrozen itself and both fire. Dropping the subtest loses the first case
entirely, and that is the case that matters for a helper three other branches
depend on.

The real risk is drift in one specific direction — someone sees two tests for the
same thing and deletes the table entry as redundant, which is reasoning that looks
correct and is not. Each now says it is not a substitute for the other, and why.

Also records at the table that it has NO completeness check, unlike
TestNoStateIsRecordedBeforeTheCommitGate's gateExempt. There are five
commitRefresh call sites and three subtests. The coverage is complete today —
only extract_llm and extract_sweep_drop pass a computed eff and so can reach the
eff != markerFull branch, while state.go and summarize.go's two sites pass
markerFull and handle the degraded case themselves — but establishing that took
reading five call sites, and nothing fails if a sixth is added. #198 carries the
mechanism; two instances of that exact defect were found during this review, one
introduced and one pre-existing (#193), which is why it is filed rather than left
as a convention.

Both points are the reviewer's, including the framing that the two tests have
different subjects rather than being duplication-with-a-reason, which is what I
had called it.

Verification: gofmt -l . clean, go vet ./... clean, go test ./... all packages
pass (Go 1.26.4, eval box).

Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com>
Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

offload: a summary/off-mode offloader is reverted on every turn from the second onward, re-writing the provider's cached suffix each time

2 participants