fix(offload): let a replayed summary/off decision declare its loss, or the pipeline reverts it - #194
Conversation
…r the pipeline reverts it Closes #193. Under marker_mode summary/off an offloader takes a deliberate LOSSY drop: nothing is stashed and no <<cg:HASH>> is written, so it returns no cache keys. components/pipeline.go:135 treats that combination — the request shrank, no cache keys, not Skipped — as a contract violation and REVERTS the component, unless rep.Irreversible says the loss was chosen. commitMark's non-full branch sets that flag, so the turn that TAKES the decision is fine. Every later turn replays it through reapplyFrozen, which never did. Measured on a two-turn mask fixture at marker_mode: summary, same content both turns: turn 1 (fresh, via commitMark): keys=[] Irreversible=true -> kept turn 2 (replay, reapplyFrozen): keys=[] Irreversible=false -> REVERTED So from turn 2 onward, for the rest of the session, the component is discarded and the transcript is forwarded verbatim. That is worse than a lost saving. Earlier turns sent the REDUCED bytes, so sending the original re-writes the provider's whole cached suffix at ~11.5x the read price — every turn, for every message the component had reduced. It is the cache-destructive direction reapplyFrozen exists to avoid, reached through reapplyFrozen. And it is invisible from the component's side: it acts, computes a replacement, and the pipeline throws the work away afterwards. Affects mask, collapse, failed_run, readlifecycle, skeleton, cmdfilter and agentdiet whenever configured with marker_mode summary or off. The default (full) is unaffected: a full-mode frozen replacement always carries a marker, so keys is non-empty and the precondition never holds. len(keys) == 0 is a sound test for "degraded mode" rather than a proxy: every freeze() site is downstream of a tryMark/commitMark pair, so a full-mode frozen replacement always carries a marker — which means the blanket flag cannot mask a full-mode bug. Threading rep through the seven call sites is mechanical. Found while auditing replay paths for the #188 review, which flagged the same omission in two NEW replay branches introduced there. This instance predates that work — at the merge base reapplyFrozen takes no rep parameter at all — so it is fixed here, off main, rather than shipping only when #188 does. Verification: gofmt -l . clean, go vet ./... clean, go test ./... all packages pass (Go 1.26.4, eval box). TestASummaryModeReplayIsNotRevertedFromTurnTwoOnward asserts turn 2's exact conjunction, with turn 1 asserted first so a fixture that stopped reaching summary mode fails loudly instead of passing vacuously. Verified by reverting the subject: the test fails with a replayed summary-mode decision rewrote the message, returned NO cache key and did not set rep.Irreversible Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com> Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Context for reviewers: this is split out of #188 on that PR's review recommendation. #188 currently contains the same three lines and will be rebased onto |
amiddavid
left a comment
There was a problem hiding this comment.
Reviewed and verified independently, on the box (Go 1.26.4, CGO_ENABLED=1). This is the right shape for the split, and I found nothing to fix. (Recording as a comment rather than an approval only because GitHub will not let this account approve its own PR.)
The split is clean. Based on current origin/main, and the diff is exactly what a main-defect fix should be: 22 lines in state.go of which 3 are logic, 7 mechanical call-site updates threading rep, one existing test adjusted for the signature, and one new test. No reserve work rode along.
Verified:
gofmt -l .clean,go vet ./...clean,go test ./...clean — no failures anywhere,dashincluded.TestASummaryModeReplayIsNotRevertedFromTurnTwoOnwardpasses, and revert-verified: reverting the fix to_ = repcompiles clean (go vetsilent) and the named test fails. So the mutation is a real revert of the bug, not a build break — andgo test -runresolved the name (=== RUNemitted), so it is also not the "no tests to run exits 0" trap.- The blanket
rep.Irreversiblecannot mask a full-mode bug: everyfreeze()site is downstream of atryMark/commitMarkpair, so a full-mode frozen replacement always carries a marker andlen(keys) == 0genuinely implies a degraded mode. The comment states this; I checked all 7 sites. - All 7
reapplyFrozencallers updated. - Confirmed the defect predates #188: at the merge base
reapplyFrozentook norepparameter at all and did a barec.Store.Put(k, ...), so summary-mode replacements have been reverted from turn 2 onward independently of any reserve work.
On sequencing: keeping the same lines in #188 until this lands, then rebasing, is the right call — stripping them pre-emptively would make #188 red for reasons unrelated to its own review. The state.go conflict between the two branches is expected; noting it here so it is on the record rather than looking like drift later.
The comment carries the measured evidence (turn 1 Irreversible=true, turn 2 false with the message rewritten and no keys) and explains why the pipeline reverts that combination, which is what a reader needs to not reintroduce it.
Generated with Claude Code
|
Thank you for the independent verification, and particularly for revert-verifying it yourself rather than taking my word for it — including checking that the mutation compiles and that Nothing to change here. Recording what happens next so the sequencing is on the record:
One point of order on your last paragraph. You describe the comment as carrying "the measured evidence (turn 1 |
|
Taking the correction — and it strengthens the record rather than weakening it, so worth stating precisely. You're right that the numbers quoted in the So the evidence for this defect is two independently written probes, on different fixtures, agreeing on the shape — yours on Recording it that way here so the PR carries the accurate provenance. |
…ndant Comment only; no behaviour and no test outcome changes. main's TestASummaryModeReplayIsNotRevertedFromTurnTwoOnward (from #194) and this branch's `mask via reapplyFrozen` subtest assert the same thing, and the obvious reading is that one is redundant. They are not: they have different SUBJECTS that happen to share an assertion. - the standalone test asserts a property of reapplyFrozen. - the subtest asserts that all four replay branches are held to the SAME assertion through the same helper — uniformity, not the property. They fail differently, which is the test of whether both are needed. Special-case the helper so it suits extract_llm and extract_sweep_drop but not reapplyFrozen and the table catches it while the standalone test stays green; break reapplyFrozen itself and both fire. Dropping the subtest loses the first case entirely, and that is the case that matters for a helper three other branches depend on. The real risk is drift in one specific direction — someone sees two tests for the same thing and deletes the table entry as redundant, which is reasoning that looks correct and is not. Each now says it is not a substitute for the other, and why. Also records at the table that it has NO completeness check, unlike TestNoStateIsRecordedBeforeTheCommitGate's gateExempt. There are five commitRefresh call sites and three subtests. The coverage is complete today — only extract_llm and extract_sweep_drop pass a computed eff and so can reach the eff != markerFull branch, while state.go and summarize.go's two sites pass markerFull and handle the degraded case themselves — but establishing that took reading five call sites, and nothing fails if a sixth is added. #198 carries the mechanism; two instances of that exact defect were found during this review, one introduced and one pre-existing (#193), which is why it is filed rather than left as a convention. Both points are the reviewer's, including the framing that the two tests have different subjects rather than being duplication-with-a-reason, which is what I had called it. Verification: gofmt -l . clean, go vet ./... clean, go test ./... all packages pass (Go 1.26.4, eval box). Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com> Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Closes #193.
The defect
Under
marker_mode: summaryoroff, an offloader takes a deliberate lossy drop: nothing is stashed and no<<cg:HASH>>is written, so it returns no cache keys.components/pipeline.go:135treats that combination — the request shrank, no cache keys, notSkipped— as a contract violation and reverts the component, unlessrep.Irreversiblesays the loss was chosen.commitMark's non-fullbranch sets that flag, so the turn that takes the decision is fine. Every later turn replays it throughreapplyFrozen, which never did. Measured on a two-turnmaskfixture atmarker_mode: summary, same content both turns:So from turn 2 onward, for the rest of the session, the component is discarded and the transcript is forwarded verbatim.
Why it is worse than a lost saving
Earlier turns sent the reduced bytes. Sending the original back re-writes the provider's whole cached suffix at ~11.5x the read price — every turn, for every message the component had reduced. That is the cache-destructive direction
reapplyFrozenexists to avoid, reached throughreapplyFrozen.It is also invisible from the component's side: it acts, it computes a replacement, and the pipeline discards the work afterwards. Nothing counts reverts per component, so a summary-mode deployment reads as one that is simply saving less than hoped.
Scope
Affects
mask,collapse,failed_run,readlifecycle,skeleton,cmdfilterandagentdietwhenever configured withmarker_mode: summaryoroff. The default (full) is unaffected — a full-mode frozen replacement always carries a marker, sokeysis non-empty and the precondition never holds.The fix
reapplyFrozentakesrepand setsrep.Irreversiblewhen the replayed replacement carries no markers. Threadingrepthrough the seven call sites is mechanical.len(keys) == 0is a sound test for "degraded mode" rather than a proxy for it: everyfreeze()site is downstream of atryMark/commitMarkpair, so a full-mode frozen replacement always carries a marker — which means the blanket flag cannot mask a full-mode bug. That was the property worth checking before choosing a blanket flag over threading the mode through the store.Alternatives considered
pipeline.go's exemption to "no keys, but the replacement is a pure function of the original". Rejected: the pipeline cannot know that, and it weakens the guard that catches a component which genuinely forgot to stash.Why this is off
mainand not part of #188Found while auditing replay paths for the #188 review, which flagged the same omission in two new replay branches introduced there. This instance is independent and predates that work — at the merge base (
51fcd91)reapplyFrozentakes norepparameter at all.#188 still carries the open reserve-lifetime question in #190, so folding this in would mean a bug that is live on
maintoday ships only when that lands. #188 contains the same three lines and will be rebased onto this once it merges; the reviewer on #188 independently reached the same conclusion about separating it.Verification
gofmt -l .clean ·CGO_ENABLED=1 go vet ./...clean ·CGO_ENABLED=1 go test ./...all packages pass (Go 1.26.4, eval box).TestASummaryModeReplayIsNotRevertedFromTurnTwoOnwardasserts turn 2's exact conjunction. Turn 1 is asserted first, so a fixture that stopped reaching summary mode fails there rather than passing vacuously below — and the test also requires that turn 2 actually rewrote the message and did not reportSkipped, since either would make the revert precondition unreachable.Verified by reverting its subject: