docs(analysis): evaluate a Claude Code plugin as a fourth transport

[amiddavid]

What this is

A design evaluation, docs only — no code, no behavior change. It asks whether the components core can ship as a Claude Code plugin: a fourth transport beside proxy/, the AuthBridge plugin and adapters/bifrost, reusing components via configuration rather than reimplementing them.

Framed deliberately as a deployment/transport question. Nothing here proposes replacing the proxy.

The one constraint

No plugin surface can read or write the outbound request. Across all ~34 hook events there is no field that rewrites messages[], system[], tools[] or cache_control, and plugin settings.json accepts only agent and subagentStatusLine — so a plugin cannot even point a session at the proxy.

What is interceptable is one tool result at the moment it is produced: PostToolUse → hookSpecificOutput.updatedToolOutput.

Why this applies to offloaders but not to cache management

This is the part worth reviewing, and it is a structural argument rather than a list of blockers. The plugin sits upstream of Claude Code's session transcript; the proxy sits downstream of it. The full request body exists only on the arrow between them.

Offloaders are content-scoped. cmdfilter filtering terraform plan noise, collapse windowing a 2,000-line log, skeleton reducing a source read — each is a pure function from one blob of text to a shorter blob of text. No neighbouring message, no system array, no provider response. PostToolUse hands over exactly that blob and accepts a replacement, so the unit of work and the unit of interception are the same size. Two components get better than they are in the proxy: the hook supplies tool_input.command and tool_input.file_path, so cmdfilter and skeleton stop inferring both from transcript text.

Cache management is envelope-scoped. Not one message — the shape of the entire prefix:

• cachesplit restructures the top-level system array, which components never see; only apply does, from the raw body.
• cacheinject reasons over messages[] positionally (divergence point, turn-stable anchors, the 4-slot budget counted across system + tools + messages together) and its output is a cache_control key — request metadata, not content.
• the keepalive on feat/keepalive-strategies must originate a request that byte-exactly reproduces a prefix, and price the decision from Observation.CachedTokens, which only the provider's response carries.

None of that is a tool output. It is the envelope — assembled after the last hook runs and never written to disk. Verified rather than assumed: a session transcript carries messages and toolUseResult and no system prompt or tool schemas (scripts/inspect_transcript.py). Since tools+system is the front of the cumulative hash and, per prefixsplit.go, ~7,017 tokens of it, the missing piece is not a skippable tail — it is what you would have to reproduce first. (That token figure, like the component percentages below, is this repo's own recorded measurement quoted from a code comment — accurate, but frozen and not re-verified against current traffic here. The argument turns on its order of magnitude, not the digits.)

So the rule is not lossy-vs-lossless or cheap-vs-expensive:

Content-scoped work distributes to the harness. Envelope-scoped work requires being on the wire.

There is a second asymmetry in how the families fail, and it is the deeper reason. A wrong offload wastes one expand round-trip — bounded, and reversibility is type-enforced. Wrong cache work inverts: a mistimed keepalive creates an entry at 1.25× instead of refreshing at 0.1×, a breakpoint over budget is a hard 400, a representation flip inside a cached prefix re-writes the suffix at 11.5×. Envelope work has no fail direction that is merely "no saving", which is exactly why it wants the host that sees the whole request and the provider's answer to it.

The compensating result

The defensive half of the KV-cache layer stops being necessary rather than being ported. state.go names its own premise: an offloader must re-emit identical bytes "otherwise the agent (which re-sends the ORIGINAL each turn)" flips the representation. A hook rewrites the output before it enters the transcript, so the agent never holds the original. Freeze/replay, MaxCachedIdx, Tracker, frozen_flips and sticky ids have nothing left to defend, and extract_llm's sampling nondeterminism stops disqualifying it from repair.

PreCompact/SessionEnd also replace proxy/agentcompaction.go's string match against Claude Code 2.1.215 internals — which has a documented reachable false positive (our own docs page quoting the phrase).

What is given up

• cachesplit — −34.1% cost, 0% → 96.7% hit rate, and it is in every preset. The best-evidenced component in the repo.
• mask (27.5% Terminal-Bench, 12.5% SWE-bench) and failed_run — both rewrite earlier messages, which a hook firing once at birth cannot do.
• /stats cost tiers — they come from the provider's response, so plugin mode cannot be benchmarked the way the proxy is, and deploy/harbor/*.py has nothing to parse.

On implicit prefix-cache backends (vLLM/llm-d) the cache loss is zero, because prefixsplit is already a no-op there. On Anthropic it is real.

Gate 0 is closed — persistence confirmed

The proposal's one blocking unknown was whether updatedToolOutput persists into the transcript or applies to a single request. Review ran it: a PostToolUse hook replacing a Bash output with a sentinel, captured through a raw-logging reverse proxy so the evidence is the literal outbound body rather than the transcript file. Turn 2, messages[3]:

{"tool_use_id": "toolu_bdrk_01SC6Vg1gZAtY1WEVe7xsbsR", "type": "tool_result",
 "content": "SENTINEL-OBJ-999-REPLACED", "is_error": false}

The real content appears in no resent tool_result, and a later turn asked for the exact output answered with the sentinel at input_tokens: 69 — from resent history, not regeneration. A working collapse plugin then measured −6,285 tokens on a real session, showing as the same reduction on turn 1's cache-write and turn 2's cache-read: a permanent reduction of resent context, not a one-turn display trick.

So the defensive KV-cache half really is free, and the recommendation is no longer conditional.

The predicted failure mode arrived on the first attempt. updatedToolOutput must be the object tool_response shape — {stdout, stderr, interrupted, isImage, noOutputExpected} for Bash — and a bare string is silently ignored. That's now a hard requirement on adapters/cchook: emit the object shape, count your own rejections.

And a number in this PR was wrong. I cited a "~10,000-character cap" on updatedToolOutput; that figure governs additionalContext/systemMessage/plain stdout, not this field. Measured: verbatim and uncapped to ~30,000 chars, real threshold in (30,000, 40,000] (likely 32,768), and above it neither truncation nor rejection — the CLI's ordinary large-output handler emits a ~2,260-char <persisted-output> wrapper with a 2 KB preview and a disk pointer, local record intact. For an oversized hook emission that's a token-cost improvement. "Cap" was the wrong frame.

On DAM

Land the proxy in the gateway. DAM is harness-plural (Claude Code, Pi, Bob, Codex, plus any ACP runtime) and its egress already matches our gateway credential model, so a Claude-Code-only plugin covers one harness of four and none of the bring-your-own case. Ship the plugin as the Claude-Code-session layer on top of the gateway, never as the DAM integration.

Status after review

CI is green: the doc moved to docs/superpowers/specs/, which exclude_docs already covers and whose stated purpose — "working plans/specs … not published site content" — is what this is. Verified with a real mkdocs build --strict (1.45s, clean) plus a negative control confirming the same file under docs/analysis/ still aborts strict mode. The alternative offered was a nav entry under Results:; exclusion is the better fit, since publishing a page evaluating a plugin we haven't built would read as a product that exists.

Also sharpened per review: the envelope claim now records the stronger verification method (11 hook events exercised live, plus the installed CLI's own hookSpecificOutput validation schema — 33 events, 22 with output fields, none touching the envelope), and carries the nuance that promptCacheTtl / CLAUDE_CODE_PROMPT_CACHE_TTL is a reachable cache-TTL lever via settings or env — not via any hook or manifest field, and session-wide rather than per-breakpoint — so "zero cache control" would overstate the gap. Component figures quoted throughout are now labelled as frozen historical measurements.

The scope claim, stated the way the review put it better than I had: a plugin can replicate the offloader half of this repo — persistently and measurably — and categorically cannot replicate the cache-management half. Not "a plugin can do what the proxy does."

Related: #130 (local distribution — how an evaluator installs any of this).

Assisted-By: Claude Opus 5 (1M context) noreply@anthropic.com

[OsherElhadad]

Reviewed by fact-checking every structural claim against current code/docs, and by settling the doc's own stated "Gate 0" empirically — building a real probe plugin and a real Claude Code session, checked against the literal outbound wire bytes rather than assumption.

CI is red

mkdocs build --strict fails: docs/analysis/claude-code-plugin.md isn't wired into mkdocs.yml's nav, and — checked directly — there's no existing "docs/analysis" nav convention to copy. The closest precedent, exclude_docs: superpowers/ (used by draft/working docs), fits a doc that's exploratory; this one reads as a finished evaluation with a recommendation, closer to docs/results/improvement-plan.md, which is navigated. Two honest options, not a single obvious fix: (a) extend exclude_docs to cover analysis/, or (b) add a nav entry under Results:. Author's call, not something to pick on their behalf.

Gate 0 — the doc's own central open question — is now empirically confirmed, in the doc's favor

The doc names this exactly right as the gate everything else depends on, and flags two competing hypotheses: persistence (the doc's hoped-for case) vs. single-request-only (the "counter-hypothesis" that "collapses the whole proposition"). We ran the actual experiment: a PostToolUse hook replacing a tool's output with a sentinel, checked against a raw-log reverse proxy capturing the literal outbound wire body (not just the transcript file) across two turns of a real session.

Result: an object-shaped updatedToolOutput (matching the tool_response schema — {stdout, stderr, interrupted, isImage, noOutputExpected}) does persist into the transcript's stored API message and is resent verbatim on later turns. Wire capture, turn 2, messages[3]:

{"tool_use_id": "toolu_bdrk_01SC6Vg1gZAtY1WEVe7xsbsR", "type": "tool_result", "content": "SENTINEL-OBJ-999-REPLACED", "is_error": false}

— the real content never appears in any resent tool_result, and a follow-up turn asking "what was the exact tool output" answered with the sentinel, at input_tokens: 69 (answered from resent/cached history, not fresh generation). One operational gotcha worth calling out because the doc itself pre-flagged exactly this risk under "Check 1" ("shape validation fails silently... we will ship a silently inert plugin again"): a bare string updatedToolOutput is silently ignored — it must be the object shape above. A naive first implementation attempt hit this and looked like a negative result before the shape was corrected.

To be precise about what this does and doesn't establish: it confirms the doc's own preferred hypothesis, it doesn't refute a claim the doc made — the doc never asserted persistence, it flagged non-persistence as the risk to rule out. This is real, first-party evidence — stronger than what the doc itself cites for this point (docs silence) — and it upgrades the central recommendation from a reasoned bet to an empirically-grounded one.

We also went further and shipped a real, working PostToolUse-based collapse plugin (matcher Bash, collapses large stdout to a head/tail with an omitted-count marker) and A/B-measured it on a real session: −6,285 tokens, showing up as an identical reduction on both the cache-write side of turn 1 and the cache-read side of turn 2 — i.e. a genuine, permanent reduction of resent context, not a one-turn display trick, confirmed by two independent measurements on the same session.

The cache-envelope claim is correct, confirmed at the binary level and empirically, with one precision nuance

Exhaustively checked: no hook input or output, across all 11 tested events (including three the doc didn't test — PostToolBatch, PreModelSwitch/PostModelSwitch, InstructionsLoaded), exposes system, tools, or cache_control. This matches an independent binary-level check (inspecting the installed CLI's own hookSpecificOutput validation schema directly, not docs): 33 hook events, only 22 carry any output fields at all, and the exhaustive validation union confirms none of them touch the envelope. cache_control is present in the real captured wire body — the CLI places it on system blocks and the trailing message itself — but it's the CLI's own placement, never visible to or touchable by any hook. Plugin settings.json is confirmed to accept only agent/subagentStatusLine — a plugin genuinely cannot even point its own session at a different endpoint via its manifest.

One nuance worth a one-line sharpening in the doc: CLAUDE_CODE_PROMPT_CACHE_TTL/promptCacheTtl is a real, reachable cache-TTL lever (5m default on an API key, 1h on subscription) — via settings.json/env, not via any hook or plugin field, and it's session-wide/static rather than per-breakpoint. "No plugin surface can set any cache_control" is true; it could be read as implying zero TTL control of any kind exists, which slightly overstates the gap. Doesn't change the conclusion — cachesplit/cacheinject's per-breakpoint reasoning is still unreachable from plugin-land — just worth precision.

Check 1's own cap figure is off by 3-4x, and mischaracterized

The doc's "~10,000-char cap" doesn't match what actually happens. Measured directly (bisected with distinct-letter-per-5000-char-segment payloads): up to ~30,000 chars, updatedToolOutput.stdout goes out on the wire verbatim, uncapped. The real threshold sits in (30,000, 40,000] chars (most likely 32,768). Above it, there's no truncation and no rejection — the CLI's existing large-tool-output handler (the same one that applies to any oversized tool result, hook-produced or not) kicks in: the wire tool_result becomes a fixed ~2,260-char <persisted-output> wrapper with a 2KB preview and a pointer to the full content saved on disk, while the local toolUseResult record still holds everything, untruncated (confirmed at 100,000 chars, fully intact). So: real number ~3-4x higher than cited, and "cap" is the wrong frame — it's graceful degradation to the same preview+pointer pattern Claude Code already uses everywhere else, which if anything is a token-cost improvement for an oversized hook emission, not a failure mode to design around.

Structural claims — all four checked directly against code/data, all hold up

• "cachesplit restructures the top-level system array, which components never see; only apply does" — confirmed: apply/prefixsplit.go operates on raw wire bytes entirely outside the Component/Reformat/Offload interfaces; independently corroborated by Ctx.ExistingBreakpoints's own doc comment in components/component.go.
• "A session transcript carries messages/toolUseResult and no system prompt or tool schemas" — confirmed empirically, not just by reading the script: ran scripts/inspect_transcript.py against a real 16MB transcript file on this box. No system field, no tools array anywhere.
• "tools+system is ~7,017 tokens of the prefix, per prefixsplit.go" — accurately quoted (the doc's own wording is more careful than a paraphrase suggests: ~7,017 tokens describes part of the block, not the whole tools+system figure), but it's a frozen historical measurement from a code comment, not something a currently-runnable test recomputes — worth a one-line "measured [date], not re-verified on current traffic" caveat for maximal honesty, not a misrepresentation.
• "mask/failed_run rewrite earlier messages, which a hook firing once at birth cannot do" — confirmed by direct code read: both iterate over and re-derive their rewrite against every earlier tool output/run on every request, specifically because the agent re-sends the original each turn — structurally something a single-shot PostToolUse hook can't replicate.

Bottom line

Partially the doc's own framing needs sharpening, not partially wrong: the doc is right that no plugin surface touches the cache envelope (now confirmed at the binary level and empirically, today), it correctly anticipated its own biggest operational risk (the silent-shape-mismatch failure mode, hit exactly as predicted), and its central recommendation — gate the build on Gate 0, then build — is now backed by a real positive result rather than an open question. The net technical implication worth stating plainly for whoever reads this next: a Claude Code plugin can do real, persistent, content-scoped context compaction (demonstrated and measured above) but categorically cannot do envelope-scoped cache management — so the strongest version of this proposal is "plugins can replicate the offloader half of this repo," not "plugins can replicate what the proxy does."

[amiddavid]

Thanks — that review did more than check the doc; the Gate 0 experiment and the cap measurement are both things I flagged and couldn't settle. All four points addressed in 9678582 (branch rebased onto current main, so it's a force-push).

CI — fixed, and verified rather than assumed

Took option (a), exclusion, but by moving the file rather than widening exclude_docs: it now lives at docs/superpowers/specs/2026-08-30-claude-code-plugin-transport-design.md. That directory's stated purpose — "working plans/specs … not published site content" — is exactly what this doc is, it matches the dated-spec naming of the keepalive design doc and the #130 proposal, and it needs no config change at all.

Against your read that it's closer to results/improvement-plan.md: the deciding factor for me is that publishing a page which evaluates a plugin we have not built would read on the docs site as a product that exists. improvement-plan.md describes the shipped thing's roadmap; this describes a hypothetical deployment.

Verified with a real mkdocs build --strict — 1.45s, clean — plus a negative control: the same file placed back under docs/analysis/ still aborts strict mode. So the move is the fix, not something incidental passing for another reason.

Gate 0 — folded in, and it changes the shape of the doc

The wire capture is stronger evidence than anything the doc had, so the "three things to verify" section is now a resolved-gate section plus the risks that actually remain, and the recommendation stops being conditional. Recorded the input_tokens: 69 detail too, since "answered from resent history, not regeneration" is the part that makes it airtight.

The object-vs-string gotcha is now a hard requirement on adapters/cchook rather than a general warning — emit the tool_response object shape, count your own rejections. Worth noting the doc predicted this exact failure mode and it still bit on the first attempt; that's a good argument for the counter being non-optional.

You're right that this confirms rather than refutes — the doc flagged non-persistence as the risk to rule out. Kept that framing.

The cap figure — my error, corrected

"~10,000 chars" came from the additionalContext/systemMessage/plain-stdout cap, which doesn't govern this field, and I carried it across without checking. Now: verbatim and uncapped to ~30,000, threshold in (30,000, 40,000], and above it graceful degradation to the <persisted-output> preview+pointer rather than truncation or rejection. Also took your framing point — "cap" was wrong, and for an oversized emission this is a token-cost improvement, not a hazard to design around.

Both precision notes taken

promptCacheTtl / CLAUDE_CODE_PROMPT_CACHE_TTL is now called out as a genuinely reachable TTL lever — via settings or env, not any hook or manifest field, session-wide rather than per-breakpoint — with the explicit note that a flat "zero cache control" reading overstates the gap. It's also directly relevant to #130, where the install is settings-based: worth considering whether the installer should set it. Filed that thought there rather than expanding scope here.

The frozen-measurement caveat now covers the class, not just the 7,017 figure — cachesplit's −34.1%, mask's 27.5%/12.5% and the system-block size are all labelled as this repo's recorded results, quoted accurately but not re-verified against current traffic, with a note that the argument turns on order of magnitude rather than digits.

Scope claim

Adopted your wording as the lead of the recommendation, because it's more precise than mine was: a plugin can replicate the offloader half of this repo, persistently and measurably, and categorically cannot replicate the cache-management half — not "a plugin can do what the proxy does."

Assisted-By: Claude Opus 5 (1M context) noreply@anthropic.com

[OsherElhadad]

Reviewed against main e88f5d8. CI green, mkdocs build --strict clean (2.91 s), and
docs/superpowers/specs/ is correctly excluded from the published site
(mkdocs.yml:81-82 exclude_docs: superpowers/, confirmed absent from the built output) — so the
"fix CI" claim landed, via a file move rather than a config change. That was a better option than
either suggested in review, and the stated reason ("publishing a page that evaluates a plugin we
have not built would read as a product that exists") is sound.

This is the best-reasoned document in the current batch. It identifies the one constraint that
decides everything, and — this is the part worth preserving — it verified that constraint against
the CLI binary rather than the published documentation. I re-derived it independently, and the
docs are wrong: asked whether PostToolUse can replace tool output, a docs-backed source says the
field does not exist. It does:

$ grep -ao '.\{0,60\}updatedToolOutput.\{0,120\}' "$CLI"        # 2.1.252, ELF, not stripped
updatedToolOutput:...describe("Replaces the tool output before it is sent to the model")
updatedMCPToolOutput:...describe("...Prefer updatedToolOutput, which works for all tools")
"PostToolUse hook returned updatedToolOutput that does not match \<tool\>'s output shape; using original output."
$ grep -aoc updatedToolOutput "$CLI"   # 38

The CLI's own error string independently corroborates the object-shape gotcha at :381-388 — it
literally logs "using original output". The :20-26 claim to have used a stronger method than the
docs is justified, and is the single best decision in the PR.

I also confirmed the capability boundary the doc rests on: hooks see tool name/input/output,
prompts and lifecycle; no hook sees or writes messages[], system, tools or cache_control,
and a hook cannot declare a model-callable tool — MCP is the only path. Since plugins bundle MCP
servers, invariant 3 is still satisfiable exactly as :279 proposes, and the doc does not overclaim.
One correction in the doc's favour: invariant 5's frozen replay is not impossible under a plugin,
it is unnecessary — the replacement enters the transcript and is resent verbatim, so there is no
turn-N+1 re-derivation. :174-198 argues this and it is sound.

Three things to fix before this functions as a decision record.

1. Blocking — #141 did not build this transport, and nothing in either PR says so

This document analyses and recommends a PostToolUse → updatedToolOutput offload hook plus an MCP
expand server (:330, :487). #141's entire hook manifest is:

{ "hooks": { "SessionStart": [ { "hooks": [ {
      "type": "command",
      "command": "\"${CLAUDE_PLUGIN_ROOT}\"/scripts/start-proxy.sh",
      "timeout": 60 } ] } ] } }

No PostToolUse. No updatedToolOutput. No MCP server. It starts the HTTP proxy in the
background and writes env.ANTHROPIC_BASE_URL into the user's real settings.json via
scripts/settings.py. Its default preset is cache — i.e. cachesplit, the one component this
document declares "No — categorically" unreachable from a plugin (:236).

And the headline blocker at :16-18 — "Plugin settings.json accepts only agent and
subagentStatusLine, so a plugin cannot even set ANTHROPIC_BASE_URL" — is true only of a plugin
manifest's settings.json. #141 routes around it in the obvious way. The doc is aware the option
exists (:35, "a settings-based install — the shape proposed for local distribution") but mentions it
only as a promptCacheTtl aside and never evaluates it as a transport candidate. It is the
option that won.

Three consequences: (a) the "fourth transport" framing is wrong for what shipped — #141 is transport
#1 in new packaging, and README's three-transport Integrate table is correct and should not
change; (b) every "unreachable from a plugin" verdict in the component tables is irrelevant to
#141, which runs the full proxy, so a reader using this doc to understand the shipped plugin's
capabilities will be wrong about most of the component set; (c) the doc's actual recommendation is
still unbuilt, which is invisible from either PR.

Fix: add a section distinguishing (A) plugin-as-interceptor (PostToolUse, analysed here,
unbuilt) from (B) plugin-as-installer-for-the-proxy (SessionStart + settings edit, what #130/#141
ship, losing none of the component set). Rewrite :16-18 to scope the claim to the manifest. Fix
the transport count — and note :1 currently says "third transport" while :5, the PR title and
both commit subjects say fourth.

2. Blocking — a warm-regime figure is quoted as the expectation for a definitionally cold-regime user

The −34.1% figure is not wrong, and I want to be precise about that: benchmark harnesses run
tasks back-to-back inside the 5-minute TTL, which is the only regime where a volatile-tail split can
pay. $0.0298 across 1,127 sessions / 11,361 requests (docs/dashboard.md:204) is correct for cold
interactive traffic. Both stand; they differ by three orders of magnitude and do not contradict.

The defect is the transfer. Provenance:

• docs/components/cacheinject.md:203 — mean of one Terminal-Bench task × 3 trials; :209 says
  "Treat 34.1% as one task measured three times, not a fleet average", with a second task within
  noise and all figures at Sonnet 5 rates
• docs/dashboard.md:219 — the A/B "ran tasks back-to-back inside the TTL", so arm 2 inherits arm
  1's warm cache. Not a paired one-timeline measurement. :598 names what it measured:
  "cross-session reuse, which this per-request condition cannot see"

Three further facts bound the cold case further: the env snapshot is captured once per session
(a 9-turn session that created and committed four files produced one volatile-tail hash, and 1,105
of 1,127 first requests read zero from cache); cachesplit does nothing for an agent outside
a git repo (three accounts at 125/1,035/1,972 requests mutated on zero — no volatile tail, either
under the 1,024-token minSplitTokens floor or missing all four markers); and it is a no-op on
implicit prefix-cache backends (config/config.go:379-380). docs/results/terminal-bench-comparison.md:155
records cachesplit acted=0 on Terminal-Bench for a structural reason.

So :219's "Real loss" row, :257's "the best-evidenced component we have" (which
cacheinject.md:209 contradicts directly), and the :426 DAM bullet all overprice losing
cachesplit by orders of magnitude. Corrected, the Anthropic row reads close to the vLLM row — the
plugin gives up cents and gains the free defensive half. The decision this misprices is :491
"land the proxy in the gateway, ship the plugin only on top." That call may still be right on
spendgate/tenancy grounds; it is currently justified on a figure worth $0.03 in the regime a
plugin user is actually in.

:44-50's caveat makes it worse rather than better: "not re-verified against current traffic" is
false (it was, to ~$0.03), and asking the reader to trust order of magnitude is asking them to trust
the one thing that is regime-dependent. Name the regime.

3. Blocking — Gate 0 is closed in two places and still open in two others

349:## Gate 0: persistence — RESOLVED, in this proposal's favour
459:   architecture. **It rests entirely on the unverified persistence check (§0).**
476:   Decide it with the §0 experiment before writing adapter code.
493:- **Gate 0 is now closed** (persistence confirmed on the wire ...)

The review commit rewrote §0 and the Recommendation but missed "What the plugin is actually for"
(:445-477) — the ranked go/no-go section, written to be the part a reader skims to. It still calls
the case conditional and unverified, and instructs the reader to run an experiment that already ran.
So the document's own summary gives the pre-review answer to the decision the document exists to
make. Two edits: drop "and it is conditional" / "rests entirely on the unverified persistence
check (§0)" at :454-459, and replace :476 with "§0 settled item 1; decide item 2 on its own
merits."

Worth fixing because it strengthens the recommendation

• rtk is never mentioned (grep -in 'rtk\|token killer' → no match), and rtk is the
  architecture this document recommends — a hook that rewrites tool output before it enters context —
  already benchmarked in this repo as a full fourth arm at −9.0% billed cost, reward-neutral
  (docs/RESULTS.md:14, docs/results/rtk.md:11). Meanwhile the only figure for this proposal is
  :373's "−6,285 tokens" on one session: no denominator, no percentage, no acted, no paired arm,
  so by this repo's own conventions it is not a savings measurement. And the doc misses where its
  design beats rtk: rtk is a shell hook, so Claude Code's built-in Read/Grep/Glob bypass it
  entirely — that is rtk's ceiling. updatedToolOutput has no such ceiling, and :330 already
  proposes matcher ".*". State expected value as "≥ −9.0%, because matcher .* reaches the
  built-in file tools rtk cannot". That is a real edge, currently unclaimed.
• mask's "top measured lever — 27.5%/12.5%" (:272) is quoted accurately from
  docs/components.md:439, but docs/results/terminal-bench-comparison.md:86 says "~29.5% … that
  figure is a single-task replay", from an arm where mask was not enabled. Two numbers for one
  lever, and a replay figure presented as enforced — the same class as item 2. Quote it as
  "~27.5–29.5%, single-task replay, never enforced in a benchmark arm". (The 27.5/29.5 split is
  pre-existing repo drift, not this PR's fault.)

Smaller items

• :14 says "across all ~34 hook events" while :23 says "33 events". Verified: 33 present in
  2.1.252. This matters because the argument is exhaustive — a fuzzy count invites "which one did
  you miss?". One character.
• :398-399 — "threshold in (30,000, 40,000] — most likely 32,768". The bisection is real and the
  reframe at :403 ("'Cap' was the wrong frame") is right, but the bracket is 10,000 wide and no
  governing constant appears near updatedToolOutput in the binary. In a doc this careful, a guessed
  round number reads as measured. Bisect the last decade or drop the guess.
• :270 — extract_llm "~8× underwater" is stale; terminal-bench-comparison.md now says 82×
  ("the plan's earlier 8× priced those tokens as fresh; they sit in the cached prefix").
• scripts/inspect_transcript.py — the code is good. It runs on a real 44 MB live transcript,
  is read-only, prints key names and counts but never values (so it cannot surface a credential), and
  survives a truncated final JSONL line (try/json.loads/except: continue), which is the common
  case for a live-appended transcript and the one thing that would make it broken by default. The
  objection is placement only: deploy/harbor/*.py is this repo's convention for analysis Python
  (~15 files) while scripts/ holds two shell wrappers; there is no test, no Python linter
  anywhere (.pre-commit-config.yaml and ci.yaml have none), and its only reference in the repo
  is one line of an unpublished doc. Move it to deploy/harbor/, or inline it as a fenced block in
  the spec it supports.
• :138 cites the script for "no system prompt or tool schemas". Running it prints
  system-ish keys: NONE but also record types: {... 'system': 67 ...} — it inspects key names,
  not record types, so a reader who checks finds a system record type and has to work out it is a
  transcript event. The conclusion holds (the wire capture establishes it); one clause closes the gap.
• Drift: 3 commits behind main, mechanically irrelevant (docs-only, zero file overlap), but :132
  reasons about the keepalive on feat/keepalive-strategies as unmerged — feat: bulk-create keep-alive strategies from KV-cache suggestions (strategy campaigns) #126 landed it — and
  feat(extract_llm)!: extract_llm_sweep — adjudicate spent tool outputs over the model's cached transcript #118's extract_llm_sweep has no row in the component verdict tables (:231-287). A rebase would
  want one row added and one branch reference dropped.

Nothing here re-chases refuted work: :255 correctly prices cacheinject's placement at zero.

Recommendation: rewrite, do not close as superseded

"Superseded" would be factually wrong. #141 built a different thing, so the transport this document
recommends is still unbuilt and still the live open question — closing would silently retire an
open recommendation and leave the impression that plugin-as-interceptor was tried and rejected. It is
also the repo's only correct record of the hook surface, on a point where the published documentation
is wrong; delete it and that institutional knowledge reverts to a source that cost someone a wire
capture and a bisection to disprove. :289-322 (offload at birth is irrevocable; an expand becomes a
permanent cost; ${CLAUDE_PLUGIN_DATA} realizes more cross-session reuse than the in-memory store)
and :174-198 appear nowhere else, and both are correct.

Carry a header note so the doc cannot be read as the justification for #141 — e.g. "Evaluates
plugin-as-interceptor (unbuilt). #130/#141 ship plugin-as-installer-for-the-proxy, a different
design — see §Two candidates."

Verdict

Approve with changes. Must-fix: items 1, 2 and 3. The two "strengthens the recommendation" items
are worth doing because they cost the author nothing and the rest are cheap. The review round
genuinely improved this document.

[amiddavid]

All three blockers addressed in cdab623, plus both "strengthens it" items and the smaller ones. Branch rebased onto 3ebc65d, so it's a force-push.

1. Two candidates — the section you asked for

You were right that this was the real defect: the document analysed one use of the plugin surface and never weighed the other, and the one it skipped is the one that won. There's now a §Two candidates table separating (A) plugin-as-interceptor from (B) plugin-as-installer-for-the-proxy, an explicit statement that every "unreachable from a plugin" verdict in the component tables describes (A) and is irrelevant to (B), and a header note so the page can't be read as the justification for #130.

The headline blocker is rescoped rather than softened: a plugin manifest's settings.json accepts only agent/subagentStatusLine, which is not the same claim as "a plugin cannot set ANTHROPIC_BASE_URL". An install skill merging that key into the user's own settings file is legitimate, not a loophole — and it pays none of the costs this document enumerates. Transport count is now "fourth" throughout, and the doc states that (B) adds no transport, so README's three-transport Integrate table is correct and shouldn't change.

One deliberate deviation from your text. You wrote the section against #141's hook manifest as it stood at review time; that PR is actively being worked on and its contents have already moved since, so pinning the doc to a snapshot of it would go stale immediately. The doc describes (B) from #130's spec and says the authority on what (B) ships is #130 and its implementing PR, never this page. The design distinction — which is what you were actually after — is unchanged.

2. Regime, named

Checked your provenance and it holds all the way down, so this is corrected rather than caveated. cacheinject.md:203 is one Terminal-Bench task × 3 trials at Sonnet 5 rates and :209 says in terms "treat 34.1% as one task measured three times, not a fleet average"; dashboard.md:219 says the A/B "ran tasks back-to-back inside the TTL"; dashboard.md:204 gives $0.0298 / 1,127 sessions / 11,361 requests cold, with the once-per-session snapshot, the 1,105-of-1,127 zero-cache-read starts and the 9 warm starts all recorded there.

So the Anthropic row now reads "worth −34.1% warm, $0.0298 cold" and lands close to the vLLM row; the "best-evidenced component we have" line is gone, since cacheinject.md:209 contradicts it directly, and terminal-bench-comparison.md:155's structural cachesplit acted=0 is cited alongside. You were also right that the old caveat made it worse — "not re-verified against current traffic" was simply false, and "trust the order of magnitude" was asking the reader to trust the regime-dependent part. Both replaced with name the regime.

The DAM recommendation stands, but on the grounds you identified as the load-bearing ones — spendgate/tenancy/limits/promexport and harness-plurality — not on the figure. The doc now also concedes the one place the warm number is the more relevant one: long-lived platform agents are closer to warm than a human is.

3. Gate 0

Fixed in the ranked go/no-go section, which was the part still giving the pre-review answer: "and it is conditional" and "rests entirely on the unverified persistence check" are gone, and :476 now reads "§0 settled item 1; decide item 2 on its own merits."

rtk — the best catch of the round

You're right that omitting it was the significant gap, and worse than you put it: deploy/harbor/claude_code_rtk_agent.py means we didn't just know about rtk, we ran it as a benchmark arm. There's now a section quoting results/rtk.md:11 — −9.0% billed cost, reward-neutral, zero request-path latency — as the floor for expected value, and conceding that the "−6,285 tokens" figure has no denominator, no acted and no paired arm, so by this repo's conventions it demonstrates the mechanism rather than measuring a saving.

And the edge you pointed at is now claimed: rtk is a shell hook, so Read/Grep/Glob bypass it entirely, while matcher ".*" reaches them — ≥ −9.0% for that reason. With the honest counterweight that rtk gets its −9.0% with no store, no MCP server and no expand path, which is the argument that a first useful version of (A) is small.

Smaller items

• mask → ~27.5–29.5%, single-task replay, never enforced in a benchmark arm, both places, with both sources cited and flagged as an unenforced upper bound.
• extract_llm 8× → 82×, with the reason the old figure was wrong (priced as fresh, sits in the cached prefix).
• Hook events 33, not ~34 — agreed that a fuzzy count invites "which one did you miss?" of an exhaustive argument.
• Dropped the guessed 32,768 and kept the measured (30,000, 40,000], saying explicitly that it's a 10,000-wide bracket left unnarrowed because no governing constant sits near the field in the binary.
• inspect_transcript.py → deploy/harbor/, and the doc now notes it reports key names, so the system entry in its record-type tally is a transcript event rather than a request system array — with §0's wire capture credited as what actually establishes the point.
• Keepalive branch reference → feat: bulk-create keep-alive strategies from KV-cache suggestions (strategy campaigns) #126; extract_llm_sweep (feat(extract_llm)!: extract_llm_sweep — adjudicate spent tool outputs over the model's cached transcript #118) added to the offload table as No, retroactive by construction over the model's already-cached transcript, same wall as mask.

CI should stay green: docs-only apart from the file move, docs/superpowers/ is excluded at mkdocs.yml:88-90, and the three in-page anchors were validated against the file's own heading slugs.

Assisted-By: Claude Opus 5 (1M context) noreply@anthropic.com