Skip to content

offload: a summary/off-mode offloader is reverted on every turn from the second onward, re-writing the provider's cached suffix each time #193

Description

@amiddavid

The defect

Under marker_mode: summary or off, an offloader takes a deliberate lossy drop: nothing is stashed and no <<cg:HASH>> is written, so the component returns no cache keys. components/pipeline.go:135 treats that combination — the request shrank, no cache keys, not Skipped — as a contract violation and reverts the component, unless rep.Irreversible marks the loss as chosen.

commitMark's non-full branch sets that flag, so the turn that takes the decision is fine. Every later turn replays the frozen decision through reapplyFrozen (components/offload/state.go), which never sets it. Measured on a two-turn mask fixture at marker_mode: summary, same content both turns:

turn 1 (fresh, via commitMark):  keys=[] Irreversible=true   -> kept
turn 2 (replay, reapplyFrozen):  keys=[] Irreversible=false  -> REVERTED

So from turn 2 onward, for the rest of the session, the component is discarded and the transcript is forwarded verbatim.

Why it is worse than a lost saving

Earlier turns sent the reduced bytes. Sending the original back re-writes the provider's whole cached suffix at ~11.5x the read price — and it repeats every turn, for every message the component had reduced. The direction is the cache-destructive one this repo goes to some length to avoid elsewhere (reapplyFrozen exists because flipping an already-cached message is expensive).

It is also invisible from the component's side: it acts, it computes a replacement, and the pipeline throws the work away afterwards. Nothing counts a revert per component, so a summary-mode deployment looks like one that is simply saving less than expected.

Affected components

Every offloader that freezes and replays — mask, collapse, failed_run, readlifecycle, skeleton, cmdfilter, agentdiet — whenever configured with marker_mode: summary or off. The default is full, which is unaffected: a full-mode frozen replacement always carries a marker, so keys is non-empty and the revert precondition never holds.

How it was found

Auditing replay paths while addressing a review on #188, which flagged the same omission in two new replay branches introduced there. This instance is independent of that work and predates it — at the merge base (51fcd91) reapplyFrozen takes no rep parameter at all — which is why it is filed and fixed separately rather than folded into #188.

The fix

reapplyFrozen takes rep and sets rep.Irreversible when the replayed replacement carries no markers. len(keys) == 0 is a sound test for "degraded mode": every freeze() site is downstream of a tryMark/commitMark pair, so a full-mode frozen replacement always carries a marker — meaning the flag cannot mask a full-mode bug.

Threading rep through the seven call sites is mechanical. PR to follow, off main.

Alternatives considered

  • Have pipeline.go treat "no keys and the replacement is a pure function of the original" as exempt. Rejected: the pipeline cannot know that, and widening its exemption weakens the guard that catches a component which genuinely forgot to stash.
  • Record the marker mode on the frozen entry and consult it on replay. More explicit, but it changes the store's value format for every frozen key and buys nothing over the marker test above.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions