offload: extract_llm and extract_llm_sweep share the cg:res: namespace with no component tag, so a depth-gate bypass can rest on another component's decision

[amiddavid]

The problem

extract_llm and extract_llm_sweep share the cg:res: result namespace, and the key carries no component tag:

• resultKey(session, id) = cg:res:<session>:<id> — session and content hash only (components/offload/state.go:46).
• Both components derive id from extract.ContentKey(content) (extract_llm.go, extract_sweep.go).
• The shipped housellm preset runs extract_llm immediately before extract_llm_sweep in one pipeline (config/config.go:436).

So a cg:res: record written by extract_llm for content M is indistinguishable, to the sweep, from one the sweep wrote itself — and vice versa.

Why that matters

Both components use a hit on that key to bypass the cache-tail depth gate. The reasoning is sound only for a record the component itself wrote: "this session already sent these bytes, so replaying them is byte-identical at any depth". On another component's record the premise does not hold — the sweep would splice its own shape descriptor at arbitrary depth on the strength of bytes it never sent, and the stored value is a different projection entirely.

Reachability: not established, in either direction

Worth being explicit, because the obvious mechanism turns out to be wrong. The suggested path was extract_llm's replay apply() failing for want of a reserve slot, leaving M verbatim for the sweep to find — but tryMark/markToken never consult the store, and the replay path's commitRefresh never refuses, so apply declines only on the marker-inclusive never-worse check.

Reaching the state needs M present verbatim, at cached depth, with a live cg:res: from extract_llm and no marker on it. That has not been constructed. It may be unreachable today.

What is certain is that the invariant the code relies on is not expressible by the key, so it holds by accident of the current pipeline shapes rather than by construction — and the next pipeline or ordering change re-opens the question without anything failing.

Mitigating factor

The replacement each component writes is a pure function of (content, config), so a replay emits deterministic bytes whichever component decided. The harm is therefore about which projection gets spliced and at what depth, not about non-determinism.

Options

1. Namespace the record per component — cg:res:<component>:<session>:<id>. Makes the claim true by construction. Costs cross-component reuse of session-scoped results, which nothing currently relies on (the cross-session reuse is a separate namespace, cg:xres:, which is already keyed with the model and config).
2. Store a component tag in cachedResult and check it on read. One field, backward-tolerant (a missing tag reads as empty), and preserves the option of deliberate sharing later.
3. Establish unreachability and document it. Cheapest now, and the reviewer's argument against it is the strong one: it has to be re-established after every pipeline change, and nothing fails when it stops being true.

Option 2 looks best: one field, no behaviour change for records written after it, and the invariant becomes checkable.

How it was found

Reviewing #188, where a comment on the sweep's replay path asserted the stronger claim ("the hit says this session already sent these bytes"). The comment has been corrected to state what the hit actually proves; this issue is the underlying design point, which predates #188 — both components have always shared the namespace.

[amiddavid]

Correction: this is not a latent hazard, it is a MEASURED and load-bearing mechanism — and my recommendation above would delete it

The issue as filed treats cross-component reuse of cg:res: as an accident to be closed, and recommends option 2 (a component tag, checked on read). That recommendation is wrong, or at least far more consequential than the issue says. Iteration 024 measured this sharing as a deliberate, documented mechanism and quantified it.

From docs/experiments/loca/iter024/results.md:

Two components share one result cache, and it recovers 57% of the spend. MECHANISM NOW ESTABLISHED.
extract_llm books $11.58 at $0.00, from 0 fresh calls and 364 replays, in arm B only — 4,749 calls avoided at a 99.2% cache hit rate.

The mechanism is precisely the one this issue describes: extract_llm_sweep's drop path calls the shared putResult, and extract_llm's replay path later finds that record by content id and splices it. That analysis also closed two competing explanations (expand-restore, and the tail boundary moving), and established that the reuse is content-addressed and cross-turn rather than positional — which is why the "extract_llm runs first, so the sweep cannot feed it" objection does not apply.

So adding a component tag would remove 364 replays and $11.58 of measured recovery from arm B, changing that arm's economics by more than the change itself is about.

What the issue should therefore ask

Not "how do we close the leak" but: is cross-component reuse of a session-scoped decision intended? Two defensible answers, and the choice needs to be made rather than inherited:

It is intended. Then cg:res: is a shared session decision cache, not "extract_llm's" as store.ResultPrefix's comment claims, and the fix is to say so in the comment, in docs/design.md, and to keep the reuse. The depth-gate justification then needs restating on a basis that does not depend on which component wrote the record — the honest one is that a replacement is a pure function of (content, config), so replaying is byte-stable whoever decided.

It is not intended. Then the tag closes it, iteration 024's recovery channel disappears, and that must be stated as an expected result of the change rather than discovered as a regression in the next run.

The semantic question underneath, which is the real one

The two components' decisions do not mean the same thing:

• extract_llm stores a compaction — "these bytes reduce to this projection", a property of the content alone.
• the sweep stores a drop — "this output is spent for this transcript's obligations", a judgement about conversational state, which is why the sweep deliberately keeps it session-scoped and never publishes it to cg:xres:.

When extract_llm replays a sweep record it applies an obligation judgement as if it were a content projection, at a fresh position, possibly many turns later. That may be fine — same session, same content — or it may be the sweep's judgement being reused after the obligation state that justified it has changed. Nothing currently distinguishes those, and that is a stronger argument for a deliberate decision than the depth-gate hazard the issue was filed about.

Revised recommendation

1. Decide the semantic question first, with the iteration-024 numbers in hand — the recovery is measured, so the cost of closing it is known rather than speculative.
2. If sharing stays, correct store.ResultPrefix's comment and the depth-gate rationale so neither claims what the key cannot express. That was the narrow fix already applied in fix(store): give rewind payloads their own reserve, and refuse a removal whose original cannot be stored #188 to the sweep's replay comment.
3. Only then consider a tag — and if it goes in, land it in a change whose subject is the savings figure, alongside metrics: extract_llm and extract_llm_sweep measure saved tokens against different baselines, so their per-arm savings are not comparable #195, not folded into unrelated work. Both change measured savings, and the iteration-024 re-run cannot absorb two simultaneous accounting changes.

Credit and provenance

The measurement and the mechanism analysis are the coref/iteration-024 work, not mine; I found them while checking which of #188's follow-ups block that validation. Worth reading docs/experiments/loca/iter024/results.md section 2 before acting on this issue — it also notes that #188's refusals suppress the recovery under pressure while the larger entry cap admits more of it, so "the −$7.95 should not be expected to reproduce" independently of anything here.

[amiddavid]

Second correction: the depth bypass is real, but cross-component sharing adds no byte difference

Two facts established, one by reading the control flow and one by execution. Together they narrow this issue considerably — the byte-flip hazard I filed it about is not there.

1. The bypass is real. In extract_llm.go the same-session replay is at :890 and the cache-tail gate at :932, so a cg:res: hit does reach apply without passing the depth gate. The block says so deliberately: "which is why it is safe at any depth, cached prefix included."

2. But a sweep-written record replayed by extract_llm produces byte-identical text to the sweep's own splice. Verified by running both paths over the same content and comparing:

[context-guru removed a spent tool output — tool output, 401 lines, 8400 tokens]
<<cg:9305fd7383898778>> [full output: call context_guru_expand]

Identical by construction, and each part for its own reason:

• the replacement is cached.Projected, which is the sweep's descriptor;
• the sweep stores an empty summary (putResult(c, id, sweepDescriptor(content), "")), so extract_llm's assemble takes its no-summary branch and neither adds a [summary] segment;
• both build the token through tryMark, whose key is hashKey(content) — same content, same marker;
• the recovery hints are the same literal in both files (" [full output: call " + expand.ToolName + "]").

So the cross-component aspect cannot flip bytes. If extract_llm replays a sweep decision, it writes exactly what the sweep would have written.

What that leaves

The residual concern is not about sharing. It is that content-addressed replay bypasses the depth gate on a justification stated in terms of this message — "this session already sent these compacted bytes" — while the lookup is keyed on content, so the same content at a fresh position takes the bypass too. At such a position the provider may hold the original rather than the compacted form, and splicing there is the flip the gate exists to prevent.

That property belongs to extract_llm's own replay just as much as to a cross-component one, and it predates any sharing. Whether it is reachable in practice depends on whether a recurring content id can first appear at a depth position without having been compacted on arrival — a new message normally arrives in the tail and is decided there, so the window is narrow (a moved boundary after an agent compaction, or a turn where the component was throttled or declined). I have not built that case and am not claiming it is reachable.

Revised scope for this issue

1. The semantic question stands and is the substantive one — extract_llm applying the sweep's obligation judgement as a content projection, possibly many turns later. Unchanged by the above.
2. The byte-flip framing should be dropped. A component tag would not prevent any byte difference, because there is none to prevent — which removes the main argument for adding one, and leaves only the semantic argument, against a measured 57% recovery.
3. The content-addressed-replay-versus-depth-gate question is separate and larger, applies to extract_llm alone, and would be worth its own issue if anyone can construct the window. The comment at extract_llm.go:890 justifies the bypass per-message while the key is per-content, and that mismatch is the honest statement of it.

Credit: the measured cross-component replay volume (364 acts, 0 fresh, 99.2% hit rate, arm B only) is the coref/iteration-024 work, which also correctly declined to assert the depth-gate half. This comment is that half, checked.

[amiddavid]

The fix shape: tag the KIND of judgement, not the component

From the coref/iteration-024 session's ruling, and it is a better answer than either option I listed. Both of my options tagged which component wrote the record, which is not the distinction that carries meaning:

• extract_llm stores a compaction — "these bytes reduce to this projection". A property of the content, so it is legitimately reusable wherever that content appears.
• extract_llm_sweep stores a drop — "this output is spent for this transcript's obligations". A judgement about conversational state, which is why the sweep never publishes to cg:xres:.

So the record should say which kind it is, and a compaction should stay reusable at a fresh position while a drop should not be — or should require re-adjudication. That keeps whatever the channel is legitimately buying and stops an obligation judgement being applied as a content property. A component tag would also block legitimate compaction reuse, and would permit drop reuse between two sweep instances; the kind tag gets both right.

Why the timing is structurally bad, not occasionally bad

The argument that changes my view on whether the reuse is "probably fine":

The marker skip at extract_llm.go:848 precedes the lookup at :868, so every replay lands on unmarked content — the same content reappearing at a fresh position. Content reappears because the agent re-ran the command or re-read the file. An agent re-fetches something because it needs it now.

So the replay applies "was spent when the sweep looked" at precisely the moment the agent has demonstrated renewed need. That is the worst available moment for that judgement to be stale, and it is structural rather than intermittent — the replay condition and the renewed-need signal are the same event.

The sweep's own contract makes it concrete: it defines spent as needed for none of (a) the current step, (b) an incomplete instruction, (c) a stated-but-untaken step. A re-fetch is fairly direct evidence that (a) has changed.

A hypothesis that would make this a correctness issue, and the test that settles it

Offered as a hypothesis, not a finding. If the above reading holds, #197 and #187 are the same issue:

the replay drops content the agent just fetched → the agent asks for it back → the payload has been evicted → expand_unresolved_missing.

Iteration 024 measured 209 unresolved expands in arm B and 0 in arm A — and arm A is the arm where no sweep records exist to replay. Consistent with the coupling, and confounded: arm A also stashes nothing, so the comparison cannot separate the two explanations.

The discriminating test, using #177's per-call record: check whether the content ids behind the failed expands are ids extract_llm replayed. If they are, the two issues are one, and the byte-identity result above stops being reassuring — the harm would not be a flipped byte but a removal the agent had already signalled it needed.

That evidence lives in the iteration-024 run artifacts, which belong to the coref validation work; I have not gone into them.

Consequence for this issue's status

• On bytes, closed: verified identical, no flip (comment above).
• On semantics, open and sharper than filed — with a concrete fix shape (kind tag, not component tag).
• On correctness, conditional: if the discriminating test links the replays to the failed expands, this is not a design question at all.

Credit: the compaction-versus-drop distinction, the recurrence-as-evidence-against-spentness argument and the coupling hypothesis are all the coref/iteration-024 session's.