feat(extract_llm)!: extract_llm_sweep — adjudicate spent tool outputs over the model's cached transcript

[amiddavid]

Adds extract_llm_sweep: a component that asks the request's own model, over the transcript
that model already holds in its prompt cache, which tool outputs are spent — and removes those,
leaving a shape descriptor plus a recoverable marker. It never rewrites anything and it never copies
output content into a prompt.

It also splits the cold-sweep surface out of extract_llm, ports the prefix-ask machinery to
main, and fixes three defects in shared machinery found along the way (#119, #121, #111).

Draft until live verification is re-run — see Verification below.

Why it is a separate component, not a mode

extract_llm does one thing in two situations that want different operations. On a warm turn it
works the uncached tail: the output is recent, the agent may still want most of it, and a smaller
version beats none of it. Deep history is the opposite — it is either still load-bearing, in which
case rewriting corrupts content the model has already reasoned about, or it is spent, in which
case the answer is to remove it, not to produce a smaller version nobody will read.

That is not a threshold difference. strategy, rewrite, aggressiveness and max_chars are
meaningless for an adjudicator — it selects no compaction strategy and produces no rewritten
text — so writing one is now a config error naming its replacement rather than a silently ignored
key.

Where this started, and the three corrections

Recorded because each correction came from reading measurements already in this repo more carefully,
not from new data — and a reviewer is likely to want the reasoning as much as the diff.

It began as one call per output. The proposal cited 4ca1f13 as evidence that per-output
adjudication "is a shape that has already run". That inverted what the commit says: it diagnoses
per-output as "the per-output design refuted at 6% live-kept, not the bulk shape that measured
58%", filed as a bug. cc1aa9f adds the direction of the failure — "small batches do not make
it wrong, they make it UNWILLING TO ACT" — which a sweep specifically cannot tolerate.

Then it was batched at 12, which was better but still wrong. Batching exists to amortise copying
content into the prompt, and this design does not copy content at all. a9d666f had already
established the affordable shape: ask the request's model over its cached transcript, shipping an
inventory (label, size, tool-call id, first line) rather than the outputs, because "paying fresh
to send truncated copies of content the model is reading from cache would defeat the mechanism".
That makes it one call per turn rather than one per output, so batch assembly and per-batch
concurrency all went. The ask still carries a ceiling — maxAskItems = 12 — but for a different
reason than batching had: the reply holds a verbatim quote per verdict against a 16,000-token budget,
measured live at ~600 tokens per verdict, and truncation is all-or-nothing since the array never
closes and nothing parses. An uncapped ask on a 50-candidate transcript would sweep nothing while
paying for the call. What the cap leaves unasked is counted (sweep_over_ask_cap); whether a second
ask should cover the remainder is #132.

Then the trigger change silently broke the candidate set. Moving from cold-turn to a pre-expiry
window made c.ColdCache false by construction — and the depth permission was keyed on it, so
candidates collapsed to the uncached tail while the ask reads everything up to the cached boundary.
Two disjoint sets: the model was asked to judge outputs the transcript it reads does not contain.
Found by live verification, not review (#122).

How it works

• Trigger: the window just before the prompt cache expires. Both halves are cheap there — the ask
  still reads a live cache, and the prefix it invalidates has little life left. The TTL is read
  from the request (apply.cacheTTL: 5 min for a bare ephemeral, 1 h explicit), surfaced as
  Ctx.CacheTTLMs rather than re-derived; an unknown TTL does not fire.
• Candidates: the entire transcript, above a per-output floor, capped at 12 per ask
  (largest first, remainder counted). The sweep accepts invalidating the prefix — that is what the
  window buys, and the cost is bounded by the window's width rather than by the TTL.
• The ask: one prefix ask to the request's model. tool_choice: none (free — not part of the
  cache key), tools left intact (they are part of the key), prefix = the previous turn's sent
  body, so the newest tool output is invisible to it — acceptable, since tail content has had no
  turns in which to be superseded.
• The verdict: needed_by (a/b/c/none) with the obligation quoted verbatim, then keep or
  drop. drop requires needed_by: none; unsure defaults to keep.
• The removal: replaced by a descriptor generated by our code from the content's shape, plus
  the <<cg:HASH>> marker, so expand recovers the original. The decision is frozen and replayed
  byte-identically on later turns — without which the output returns verbatim next turn and the
  cached prefix stops being stable.
• The floor: it declines entirely below min_inventory (default 10) rather than asking,
  because yield is a function of how many candidates the model compares: 1 output → 6% live-kept
  on haiku and 14% on sonnet, both inside the drop-everything null model's error bar; ~15 → 58% at
  the lowest cost per output; batch 3–6 dropped a genuinely-spent output 2 times in 4, batch 10 did
  4 in 4. Below the floor a removal is a guess, and the errors are asymmetric — a wrong keep costs
  one turn's tokens, a wrong drop costs content the agent still needs.
• The fallback: when the cache read does not happen, it asks again with a bounded sample of each
  output. On by default (a session's first turn has no prefix), with block_fallback: true to
  decline instead. The miss is counted either way.

Breaking changes

• extract_llm loses per_output and the cold_cache block, along with the "leaves the component
  with nothing to do" error that was the seam between the two behaviours. Removed keys produce a
  config error naming their replacement. The housellm preset is migrated.
• cg_component_gate_declines_total no longer carries successes; they move to
  cg_component_events_total{component,event} (metrics: cg_component_gate_declines_total carries events that are not declines #121). Both ends are in this repo — the exporter and
  the Grafana dashboards — so there is no external scraper to strand.

Safety invariants

Each is a test, and each was verified to fail when its subject was reverted:

1. a drop naming an outstanding obligation is refused, not performed — the one verification
   pointing the dangerous way;
2. unsure defaults to keep (malformed, truncated, missing or unusable verdicts);
3. a fabricated obligation quote is counted — the only remaining signal that the model is
   inventing, since nothing else it returns is content;
4. an unanswered criterion field is tolerated and counted;
5. a dropped output stays recoverable — marker written, stash present, expand resolves it;
6. the descriptor transports nothing — generated by our code from the content's shape. A test
   caught 29 content leaks across 4 fixtures when the merged design's 96-char head peek was
   restored.

Also revert-verified: the depth permission, the inventory floor, the anchor's provenance, the
rep.Gate race, the gates/events split, the reply parser, and the ask cap.

Accounting

Savings come only from real dropped-candidate sizes, through the same per-turn, non-projected pricing
convention every other component uses. Cost is priced from the request's model — what this
component actually calls — using the same rates the request-level figure comes from, so the two
agree. That last part was a defect until d162e10: CostUSD was never set at all, so the per-call
ledger read $0.00 on every firing while the request-level rollup had the real $0.0940 and
$0.1652. A component whose whole justification is cost looked free.

Verification

Unit: 27 packages, 0 failures, gofmt clean, sweep tests also green under -race. Every test
above was neutralised, run, and the actual failure quoted in its commit.

Live, re-run against the real gateway with a verbatim wire recorder. All seven end-to-end
behaviours hold on the current mechanism: cache_creation = 0 on both prefix asks against
cache_read of 41,274 and 41,140; the window firing at 250s with warm negative controls at 0s and
15s in the same session; six drop-carrying turns all HTTP 200 with no internal role="tool" and no
mid-conversation role="system"; model-driven expand recovering 104 lines byte-for-byte; 36
byte-identical replays; both fallback forks with their separate counters; and expand_unresolved_*
at zero across 32 drops.

The strongest result is a discrimination test: 12 outputs, 8 the user had finished with and 4
carrying an explicit outstanding obligation. 8/8 correct drops, 4/4 correct keeps, zero errors,
with keeps citing a verbatim in-transcript quote — and the agent then answered correctly from the
retained rows. sweep_candidate_at_depth = 31 and cached_prefix never fired, which is #122
confirmed live.

Two live findings are not resolved, and both are in the PR rather than hidden:

• The shipped default declined on a real session. 16 turns, 36,306 tokens of tool output: the
  per-output floor rejected 11 of 15 candidates and min_inventory refused the remaining 4. The
  floors compound, and a real agent turn is mostly small greps. Floor tuning is deferred to
  benchmark arms by decision; sweep_offered and sweep_inventory_below_min will show how often it
  bites.
• An independent review run saw 3 firings produce no usable verdict. Two were the parser defect
  fixed in d162e10 (first-[-to-last-] spanning the model's reasoning) and should now parse. The
  third — ask failure, then a fallback hitting its 90s deadline — is latency rather than parsing, and
  one observation does not distinguish a transient failure from a fallback that is simply too slow
  against that gateway. It fails safe. A wider live sample is wanted, and with the parser fixed the
  remaining failure rate is the number that means something.

Five invariants can only ever be unit-verified — they need the model to misbehave on demand — and
should stay that way: the obligation refusal, a fabricated quote, an unoffered/duplicate/missing
label, unsure-defaults-to-keep on malformed replies, and block_fallback. Note that this includes
both alertable counters, sweep_drop_refused_obligation and sweep_quote_fabricated, which
stayed at zero across every live run — so neither has any live exercise, only revert-verified unit
coverage.

Scope limit

This component's cost rationale is specific to Anthropic prompt-caching traffic. PrefixAsk is
nil by construction for any other provider, and the sweep handles nil explicitly — it declines, or
falls back to the content-carrying path. So it never crashes or misbehaves off that path; it simply
forfeits the cache read that is its entire economic justification, and the fallback pays fresh for
content the cached path reads for about a tenth of the price. Worth knowing before enabling it on a
non-caching backend: block_fallback: true turns that into a clean decline rather than an expensive
one.

Known unmeasured, and deliberately left so

• The pre-expiry window's width. One minute, taken from apply.coldMargin, the only figure here
  with a stated purpose for clock uncertainty around cache expiry. Wider fires more often and
  invalidates more remaining TTL; narrower fires rarely. Nothing measures either side. First thing
  to measure.
• Whether the obligation quote earns its tokens per ask. The evidence halving false drops was
  measured at batch size, where one reply covered many outputs.
• The 58% figure was measured with the coref index supplying evidence. Here the model gets no
  reference-tracking hints and reasons from the transcript alone — plausibly better, since it is not
  limited to exact matches and the index's documented blind spot was transformed reuse, but untested.
  This is what changes when feat(coref): co-reference-aware compaction #80 rebases on top.

Prepared for #80's rebase

AdjudicationItem.Evidence renders when populated and is omitted when not, so index signals slot
into the prompt contract without reshaping it. And sweep_inventory_thinned sits at the
candidate-gathering site: 4ca1f13's real defect was a per-candidate pre-filter that removed 149,681
candidates and left about one per request, turning a bulk arm into the refuted shape while it
reported itself as bulk throughout. It cannot fire on main — that is the point. A filter inserted
ahead of the append trips it on the first request.

[amiddavid]

Correction to the proposal in this PR: its justification for per-output adjudication is inverted, and the mistake is mine.

The doc says 4ca1f13 established that "the merged mode was never bulk — it adjudicated one output per call, so per-output adjudication is a shape that has already run". That commit says the opposite:

That is the per-output design refuted at 6% live-kept, not the bulk shape that measured 58%, so iteration 014 measured something other than what it claimed.

It ran accidentally, scored inside the drop-everything null model's error bar, and the commit treats it as a bug — fixed by bypassing the prefix_still_referenced pre-filter so batches actually carry multiple items.

cc1aa9f is more direct:

at batch 3-6 the model dropped a genuinely-spent output only 2 times in 4, at batch 10 it dropped it 4 in 4 and cleared 100% of genuinely-spent candidates. Small batches do not make it wrong, they make it unwilling to act, which is what a 94.6% keep rate looks like from inside.

Batch size is load-bearing for willingness to act, and per-output is batch-of-one — the extreme of unwilling. So the component built here is the shape the measurements refute.

My secondary argument for per-output does not survive either. I claimed it avoids truncated shared replies and quote-fidelity decay; both were already addressed on feat/coref-compaction — 659e7a6 raised CompletePrefixed to 16000 output tokens, and cc1aa9f capped the batch at 12 after measuring 4 of 37 quotes non-verbatim at batch 16 against 0 of 16 at batch 10. And post-trim batching does not transport content: the reply is verdicts and quotes, never output text. So the transport argument, which is the whole basis of this design, does not distinguish per-output from batched at all.

Do not merge on the premise as written. The likely correct shape for the sweep is batched adjudication at ~10–12.

What is unaffected, and worth keeping whichever shape wins: the contract port, the refusal of a drop that names an obligation, unsure-defaults-to-keep, fabricated-quote counting, a descriptor that transports nothing (a test caught 29 content leaks across 4 fixtures when the merged design's 96-char head peek was restored), reversibility, and the drop-replay-on-warm-turns decision that was missing from the spec. Those are orthogonal to batch size; what changes is the call site and a batch assembler.

Also filed from this work: #119 — rep.Gate called from extract_llm's per-candidate goroutines writes an unlocked map, which is a runtime fatal error rather than a recoverable panic, so it kills the proxy process rather than reverting the component.

[amiddavid]

All three findings from live verification are fixed in de6f3fa. #122 and #123 closed.

#122	candidates are the entire transcript — the depth permission is re-keyed off ColdCache, which the pre-expiry window makes false by construction
—	inventory floor, default 10 (min_inventory), declines below it rather than asking; decline counted
#123	the anchor ships the wire's tool_use_id, not our content key

Three tests, each revert-verified. The depth one is worth quoting because it shows the defect and both fixes interacting in a single line:

sweep_offered = 4, want 12 ... gates=map[cached_prefix:8 sweep_inventory_below_min:4 sweep_offered:4]

Tail-only starves the inventory to 4, and the floor then declines it — which is precisely why a floor without #122 would have made the component inert rather than safe.

Suite: 27 packages, 0 failures, gofmt clean, all five changed files sha256-verified local↔remote before committing.

This PR still should not be marked ready

The live verification has to be re-run. Everything the last run observed was measured against tail-only candidates — the cache read, the drop end to end, the inventory sizes, the replay. The mechanism it exercised is not the mechanism now on the branch, so that evidence no longer describes this code. In particular the two things most worth knowing are now different questions: whether inventories in real traffic clear the floor of 10 (they could not before, since the tail of an ordinary turn is one output), and whether the model's judgement changes when the outputs it is judging are actually inside the transcript it reads — which on the old shape they never were.

Two of the three fixes were mine to cause: I asked for the pre-expiry trigger without checking what else keyed off ColdCache, and I asked for the id anchor without checking what filled the field. Both were verifiable in the tree at the time, and both were caught by live traffic rather than by review.

Also worth carrying into the re-run

The last run's report flagged that extract_sweep_test.go:104 asserted coverage in a comment that did not exist, and that all 20 sweep tests disabled the tail gate. The first is fixed; the second is now deliberate and documented — the legacy tests opt into a small inventory via newSweepSmall because they assert per-candidate counts, and the default floor is exercised separately. That is a coverage shape worth a reviewer's eye rather than something to take on trust.

[OsherElhadad]

Reviewed against current head, with live verification through a local proxy build (real Claude Code sessions, real gateway/model) and a re-analysis of the repo's own committed measurements. make build/go vet/gofmt clean; go test ./... (no -race): 27 packages, 0 failures, matching the PR's own claim.

Blocking / correctness

1. CI is red, and the fix isn't fully mechanical. mkdocs build --strict fails:

WARNING -  The following pages exist in the docs directory, but are not included in the "nav" configuration:
  - components/extract_llm_sweep.md
  - proposals/sweep-adjudicator.md
Aborted with 1 warnings in strict mode!

components/extract_llm_sweep.md is mechanical — add - extract_llm_sweep: components/extract_llm_sweep.md to mkdocs.yml's Offload (reversible) list, right after extract_llm. proposals/sweep-adjudicator.md isn't: docs/proposals/ has no existing nav section anywhere in mkdocs.yml, so there's no pattern to copy. Worth asking directly whether an internal design-rationale doc (the per-output-vs-batched argument with itself) belongs in public nav at all, versus living outside docs/ — this is a decision for the author, not something we picked for them.

2. Live-verified: real adjudication calls failed to produce a usable verdict, 3 for 3, against the real gateway/model. This is the re-run the PR's own second comment asked for, and the result is worth attention independent of everything else here. A session with 12 live-eligible file-read candidates correctly cleared min_inventory and fired three separate times at real idle times inside the [240,300)s pre-expiry window (289.5s / 261.1s / 269.8s, against a 300s TTL). All three failed safe — nothing was ever dropped — but for two different real reasons:

• Fire 1: sweep_unparseable — 74 completion tokens, no parseable verdict.
• Fire 2: sweep_unparseable again — 7,191 completion tokens (71.2s latency) this time, still unparseable.
• Fire 3: the prefix-ask itself failed (sweep_ask_failed), fell back to a second completion call, which also failed: context deadline exceeded at the 90s fallback timeout (sweep_fallback_failed).

Fail-open held throughout — no data loss, no leaked content, no crash. But on this sample, the sweep produced zero effective compactions against real traffic on aws/claude-sonnet-5 via the IBM gateway. The reversibility/expand round-trip and the model's-judgment-with-in-transcript-outputs question the PR explicitly wanted re-verified could not be exercised at all, because nothing was ever dropped. Before shipping, this needs either a wider live sample to see if 3/3 was bad luck, or a look at whether the prefix-ask prompt/response-size interaction (7,191 tokens for a reply that still didn't parse) is a repeatable failure mode against this gateway.

3. Real bug: the sweep's own cost ledger is structurally $0.00, always. extraction_calls.cost_usd was 0.0 on all three real firings above, despite real cache reads (449,304 / 449,376 / 0 tokens) and real completion tokens. components/offload/extract_sweep.go never sets CostUSD on its ModelCall record (grepped the whole file — no reference at all), unlike extract_llm.go:1233's CostUSD: pricing.Cost(inTok, outTok, cw, cr). The true cost is captured, but only via a separate, request-level path (proxy/dashcapture.go:386, cg_llm_cost_usd: $0.0940 / $0.1652 for fires 1/2) — so the per-call ledger the dashboard actually shows for this component will report $0.00 forever, while a different rollup elsewhere has the real number. Given the ask was specifically for accurate, non-amortized cost/savings reporting for this component: this is the concrete defect to fix, and it's a one-line addition mirroring extract_llm.go's existing pattern.

4. The author's own retracted premise is still open in the shipped code — this is the single most important design question left unanswered. The author's first comment concluded per-output adjudication is "the shape the measurements refute" and "the likely correct shape is batched adjudication at ~10-12" (quote fidelity: 4/37 non-verbatim at batch 16 vs 0/16 at batch 10). Current code (extract_sweep.go:419-447) makes exactly one uncapped prefix-ask call carrying every eligible candidate — no batching, no ceiling. min_inventory (default 10) is a floor only; there is no cap. Commit bc016af argues in a comment that the old fidelity measurement doesn't transfer (nothing is copied into the prompt now, only inventory lines) — but that's an argument, not a new measurement, and the author's second comment itself says live verification has to be re-run for exactly this reason. A transcript with 50+ eligible candidates gets one uncapped ask, which also risks sweep_reply_truncated (the whole reply discarded) against PrefixAskMaxTokens=16000. This should either get a real batch-ceiling measurement before merge, or an explicit note that it's deferred with a tracking issue.

Floor tuning (re-analysis of existing evidence — no live-DB access was available this session; see note below)

Cold sweep floor (cold_cache.min_tokens) — recommend keeping 1000, not lowering. The existing head-to-head data already settles this at the two floors that were actually compared: two real accounts at 1000 net +$1.20 (+$2.21 saved − $1.01 spent, 63% acceptance); a 28-request replay recovers 53,071 tokens at 1000 vs 45,458 at 3000, comparable cost. There's no data point below 1000 anywhere in the repo — going lower would be extrapolation, not measurement.

Warm/tail floor (top-level min_tokens) — the evidence points the opposite direction from the ask: raise it, don't lower it. The housellm preset comment's own "8,000 is the derived floor" argument uses cost/accepted-call = $0.0193 and a claimed break-even of "4,060 saved tokens" — that's $0.0193/4,060 = $4.75/MTok, which is exactly the rate extract_econ.go's own later, dated comment explicitly repudiates in favor of a measured $2.50/MTok cache-write rate for aws/claude-sonnet-5 on this deployment. Redoing the same arithmetic with the corrected rate: break-even = $0.0193/$0.0000025 = 7,720 tokens; required candidate size ≈ 7,720/0.65 ≈ 11,877, not 6,250→8,000. The "5 warm calls / net −$0.036" figure shows the identical stale-rate fingerprint (implied $4.71/MTok); recomputed at $2.50/MTok it's actually a bigger loss (−$0.055), reinforcing the direction rather than reversing it. Shipped min_tokens: 3000 is below even the flawed 8,000 estimate. Recommend ~12,000.

Both numbers rest entirely on cost-per-call and reduction-ratio figures already in the repo, not fresh measurement — flagging that plainly rather than presenting false precision. Separately: neither extraction_calls.candidate_tokens (censored to candidates that already cleared a floor) nor request_content.before_tokens (censored to content some other component already chose to mutate — apply/apply.go:702 skips the record entirely when nothing changed) gives a clean, uncensored tool-output-size distribution. Getting one would need new instrumentation (e.g. a temporary always-log-size mode) — a suggestion for a follow-up, not something this review could build. A direct attempt to query the live production DB for this analysis (both a direct read-only query and a copy-based approach) was correctly blocked at the tool-permission layer as insufficiently authorized for production tenant data, twice, on two independent review agents — that block was respected rather than routed around, and the numbers above are re-analysis of already-published repo evidence only.

Confirmed solid

• Reversibility. applySweepDrop always stashes and marks; expand.Resolve restores byte-for-byte. TestDroppedOutputStaysRecoverable and the no-persisting-store degraded case both pass.
• The extract_llm: rep.Gate from per-candidate goroutines can crash the proxy (concurrent map writes) #119 race is resolved by construction, and confirmed clean under -race. adjudicate() is a single serial call now — no go func/sync.WaitGroup anywhere in the sweep code — so the unlocked-map-write-from-goroutines shape extract_llm: rep.Gate from per-candidate goroutines can crash the proxy (concurrent map writes) #119 describes cannot recur. go test -race ./components/... (includes components/offload, where the sweep lives): clean, 0 failures. Separately, go test -race ./... (the whole repo) hit one failure in dash, but it's a Go test-binary 10-minute timeout on TestOverviewStaysFastOnALargeWindow (a perf test moving a 10,000-row aggregation window, slow under -race's 5-10x instrumentation overhead) — confirmed via the full log (panic: test timed out after 10m0s, no WARNING: DATA RACE anywhere), and nothing in the stack touches sweep code. Pre-existing, unrelated to this PR, worth its own ticket against dash's test suite.
• Content-leak test. TestSweepDescriptorTransportsNothing checks 4 fixtures word-by-word and passes; the descriptor is built purely from computed shape stats, never a substring of content. Minor coverage gap: no binary/base64 or markdown/code fixture.
• Savings honesty. SavedTokens is set only from real dropped-candidate sizes and flows through the same per-turn, non-projected dash/event.go pricing convention every other component uses. (The cost-side bug in item 3 above is separate from this — savings accounting is fine, cost accounting for this specific component is broken.)
• Config surface. sweepBannedKeys rejects strategy/rewrite/aggressiveness/max_chars/model/context/context_messages/max_calls/economic_gate with a named reason each; tested.
• Generalization. Safe on non-Anthropic providers/backends — PrefixAsk is nil-by-construction for any non-Anthropic provider, and the sweep handles nil explicitly (decline or fall back to the expensive, provider-agnostic path). It never crashes or misbehaves, it just silently forfeits its entire cost rationale outside Anthropic prompt-caching traffic — worth stating as an explicit scope limit in the PR description if it isn't already.

Minor

• sweep_fallback_kept_everything (the fallback-path half of the keep-all split) has no dedicated test — only the prefix-ask-path equivalent is asserted.
• Test coverage is deliberately concentrated with the tail-gate disabled (preExpiryCtx, MaxCachedIdx: -1) in 21 of 23 sweep tests; the 3 depth tests cover the real boundary. Reasonable given the author's own note, flagging only for visibility.

[amiddavid]

Thank you — this is the review that found why the mechanism produced nothing on your traffic, which no amount of unit testing here was going to surface. All four blocking items addressed in d162e10; three were defects.

1. CI green — mkdocs build --strict now has zero warnings

components/extract_llm_sweep.md added to the Offload nav. For proposals/ I did not add a nav section: exclude_docs already carries superpowers/ as "working plans/specs for the skill that authored them, not published site content", and a proposal is the same category — sweep-adjudicator.md argues against an earlier version of its own design and records which measurements refuted it, which is useful in review and actively confusing on a page someone reaches looking for how to configure something. Excluding also means the next proposal needs no nav edit, and it covers the coref proposal #80 will bring.

You were right that this was the author's call rather than a mechanical fix. Following the existing precedent seemed better than inventing a second mechanism. I also chased the remaining INFO line you'd have seen next: the component doc linked to the now-excluded proposal, which would have been a published 404 — it is a repository path now, not a site link.

2. The 3-for-3 failure has a cause, and it is a defect here

ParseVerdicts took the first [ to the last ] and unmarshalled the span. Any bracket anywhere in the model's reasoning made that span unparseable — and a model asked to justify twelve verdicts writes reasoning. That is exactly your 7,191-token non-truncated reply: prose around and between the JSON. Reported as sweep_unparseable, which reads as "the prompt is wrong" when the prompt was fine.

It now tries each [ with a streaming decoder and takes the first span that decodes. Nine reply shapes are pinned, including the three that fail on the old parser — Per criterion [a] …, Candidates: [0, 1] …, and a trailing see criterion [b] above. So your fires 1 and 2 should now parse.

Fire 3 I have not fixed, and I want to be straight about that: sweep_ask_failed → fallback → context deadline exceeded at 90s is a latency/robustness matter, not a parse bug, and one observation does not tell me whether the ask failed transiently or the fallback is simply too slow against that gateway. It fails safe. Worth a wider live sample as you suggested — with the parser fixed, a re-run should also show a much lower unparseable rate, which changes what the remaining failures mean.

3. Cost ledger — fixed, one line as you said

CostUSD was never set, so the per-call figure was structurally $0.00 while the request-level rollup had the real $0.0940/$0.1652. Priced from the request's model, which is what this component calls by construction, using the same rates the request-level figure uses — so the two agree rather than being two independent guesses.

4. The uncapped ask — capped, with the coverage question tracked

Your arithmetic is the argument that settles it, and it needs no new measurement: ~600 tokens per verdict against a 16,000 budget puts the ceiling near 26, and truncation is all-or-nothing — the array never closes, nothing parses, every verdict is discarded. A 50-candidate transcript would have swept nothing while paying for the call.

Capped at 12, largest first, remainder counted as sweep_over_ask_cap. Twelve because two independent arguments agree: the budget arithmetic, and cc1aa9f's fidelity measurement (4/37 non-verbatim at 16 vs 0/16 at 10) — which I cite as corroboration rather than proof, since it was taken when content was copied into the prompt and no longer is. The cap deliberately does not trip sweep_inventory_thinned: that alarm is for a pre-filter starving the comparison, and a ceiling firing it would make it noise on exactly the transcripts where it should be loudest.

Whether the sweep should make a second ask to cover the remainder is #132. One wrinkle recorded there that is not obvious: the prefix an ask reads is the previous turn's sent body, so it does not update between asks within a turn — two asks would judge the same transcript and could contradict each other on a shared candidate.

Minor, also done

sweep_fallback_kept_everything now has its own test, both arms differing only in whether the asker reports a cache read.

A boundary my first parser fix got wrong, caught by your suite not mine

Requiring the first element to carry a verdict field rejected [{"i":1,"needed_by":"none","quote":""}] and broke TestUnsureDefaultsToKeep. That reply must parse so the caller can count it as an unusable verdict and default to keep — a model that answered badly and a reply that could not be read are different failures with different remedies. The guard now accepts any populated verdict field, still rejecting [{}] and [{"note":"x"}].

Not acted on, deliberately

The floor recommendations. The corrected-rate arithmetic ($2.50/MTok rather than $4.75) is careful work and may well be right, and I note it points up — ~12,000 against the shipped 3,000 — which is the opposite of lowering. But it is re-analysis of published figures, not new measurement, and it contradicts the preset's own committed derivation. Changing the live deployment's floor on that basis belongs on #120 with the author. Your point that neither extraction_calls.candidate_tokens nor request_content.before_tokens gives an uncensored size distribution is the part that makes this properly blocked on instrumentation, and it is recorded there.

The dash -race timeout — TestOverviewStaysFastOnALargeWindow, no DATA RACE in the log, nothing touching sweep code. Pre-existing; agreed it wants its own ticket rather than a change here.

The scope limit you asked to be explicit is now worth stating plainly: outside Anthropic prompt-caching traffic PrefixAsk is nil by construction, so this component declines or falls back — it never misbehaves, but it forfeits its entire cost rationale. Going into the PR description.

Suite: 27 packages, 0 failures, gofmt clean, mkdocs --strict clean, all seven changed files hash-verified against the build host before committing.