docs(spec): propose local distribution for a one-command Claude Code trial - #130
Open
amiddavid wants to merge 5 commits into
Open
docs(spec): propose local distribution for a one-command Claude Code trial#130amiddavid wants to merge 5 commits into
amiddavid wants to merge 5 commits into
Conversation
…trial Someone who finds the repo today has to install Go 1.26 and a C toolchain, build with the right tags, run a binary, work out ANTHROPIC_BASE_URL, and trust that routing their coding agent through an unknown local proxy will not break it. Most evaluators stop at step one. The value we want them to feel first is KV-cache, which requires being on the wire, so this proposes making the PROXY trivial to install rather than avoiding it. Five pieces, ordered by what they unblock. Pure-Go release binaries. The quickstart tells everyone to set CGO_ENABLED=1, and the dependency tree suggests that is unnecessary for the default build: the tiktoken tokenizer is pure Go and says so in internal/tokens, modernc.org/sqlite is pure Go, and go-tree-sitter is the only cgo dep and is already gated behind cg_skeleton with a stub for the negative case. If CGO_ENABLED=0 builds, we get a static binary, a plain GOOS/GOARCH matrix in one CI job and ~20 lines of GoReleaser; if not, four platforms need per-platform runners. That check is Gate A and everything else in the piece scales off it. Homebrew tap as the primary path because it avoids the macOS Gatekeeper dialog that a curl download triggers. An --idle-exit watchdog so nothing outlives use, reusing the graceful-shutdown path already in main.go. Off by default, because a gateway deployment must never self-terminate. Two constraints are load-bearing: the threshold has a floor at store.ttl_seconds, since exit wipes frozen decisions and a freeze dying mid-session is the 11.5x cache-write bug FrozenLost exists to catch; and the keepalive inverts what idle means, because pinging is what it does while there is no client traffic, so a naive watchdog kills the feature it ships. A `cache` preset — cachesplit alone. Fully lossless: no content dropped, no markers, no expand tool, no LLM calls, so the loudest objection to a context proxy does not apply and that is verifiable from one config line. It is also the best-evidenced component we have (-34.1%, 0% -> 96.7% hit). A plugin that installs it. A plugin cannot set ANTHROPIC_BASE_URL — plugin settings.json accepts only agent and subagentStatusLine — so it is the installer and operator surface, not the transport, and it closes the loop by having Claude perform the settings edit. scripts/ does the deterministic steps; the skill does the part a script does badly, which is merging one key into a settings.json that already holds the user's theme, model, permission rules and possibly their own base URL. Lifecycle via a SessionStart hook rather than launchd: nothing left on the machine, no privileged install. Documents the four details that make that hook correct — never async, idempotent because the event also fires on clear/compact/resume/fork, fixed port because the env block and the hook must agree, and the clone-time trust prompt. Gateway conformance, five items, because this funnel puts us on the wire and a broken trial is a lost adopter. Two can make the demo read as negative: we buffer some SSE responses to look for an expand call, and Claude Code aborts a stream silent for 300s; and a rejected cache_control makes Claude Code disable prompt caching for the rest of the conversation, which is a silent switch-off of the thing we are selling — the strongest argument for shipping `cache` rather than a placement preset. Also records the adoption fact missing from our docs: setting ANTHROPIC_BASE_URL without a credential variable keeps the claude.ai subscription active, so an evaluator can trial context-guru with no API key at all. Non-goals are explicit: not a proxy replacement, no offloaders in the funnel default, no measurement-only mode (rejected — it diagnoses a problem it cannot fix without pings), and not the DAM integration. Five open questions for reviewers, and staging where stage 1 ships alone and carries most of the adoption win. Docs only — no code, no behavior change. Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com> Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The proposal left two questions open that could each have reshaped it. Both are now
measured, with the scripts committed so a reviewer can re-run them.
Gate A — can we ship a static binary with no C toolchain? Yes.
scripts/gate-a-purego.sh runs in a golang:1.26 container, so it needs no local Go:
default tags, CGO_ENABLED=0 PASS
-tags cg_skeleton, CGO_ENABLED=0 fails, as intended
GOOS/GOARCH x {linux,darwin} x {amd64,arm64} PASS, all four
artifact 30.5 MB stripped, static
The cg_skeleton failure is "build constraints exclude all Go files in
.../go-sitter-forest/typescript" — the cgo-disabled signature — which confirms
tree-sitter is the only C dependency and that the gating build tag is doing its job.
ldd reports "not a dynamic executable", so there is no libc coupling either. The
first run of this check reported a failure for the wrong reason (proxy.golang.org
timeouts, not cgo), so the script now fetches the grammar modules first and says so:
a network error must not be mistakable for the error we are testing for.
The practical consequence is bigger than the release tooling.
docs/get-started/quickstart-proxy.md tells every evaluator to set CGO_ENABLED=1 and
install a C toolchain, and that is only true for a cg_skeleton build. Fixing that
paragraph removes the largest onboarding gate in the project and depends on nothing
else in this proposal.
Gate B — does an `env` block merge per key across settings files, or does the
highest-precedence file replace the whole object? It merges.
scripts/gate-b-envmerge.sh relocates the config dir with CLAUDE_CONFIG_DIR so it
never touches a real ~/.claude, and reads the result from a SessionStart hook that
dumps the environment it was handed. With CG_USER+CG_BOTH in user scope and
CG_BOTH+CG_PROJ in project scope, the hook saw CG_BOTH=project, CG_PROJ=project and
CG_USER=user — so a higher-precedence file wins only the keys it actually sets, and a
user-scope install survives a repo shipping its own env block.
That removes the failure mode the gate existed to catch: a user-scope install
silently dying in the most-configured repos. It does not change the default-scope
recommendation, because per-key merge was only one argument for project-local and the
blast-radius argument stands on its own. It does mean --global needs no per-repo
caveat, and that init has one conflict to reason about rather than two: an
ANTHROPIC_BASE_URL the user already set themselves.
Open questions are down from five to five, but they are now all judgment calls
(default scope, idle-exit floor, tap ownership, artifact size, whether to ship
skeleton in v1) rather than unknowns that could invalidate the plan.
Docs and test scripts only — no code, no behavior change.
Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com>
Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Collaborator
Author
Both gates run — neither blocks, and Gate A is better news than expectedPushed Gate A — static binary, no C toolchain: yes
So: one CI job, plain One process note: the first run of check 2 "failed" for the wrong reason — The most valuable thing here is not the release tooling. Gate B —
|
The gate results were appended near the end, so a reviewer met the hedged version of the argument first and the evidence several screens later. Three changes, no new claims. A "Findings up front" table now sits directly after the Problem, giving both answers, what each rules out, and the script that reproduces it. The "What we ship" table gains a "Gated on" column so it is visible at a glance that pieces 1 and 4 are unblocked rather than speculative. The trailing "Resolved before review" section is removed as a duplicate of the new one, and Open Questions now opens by saying the remaining five items are judgment calls rather than unknowns. Piece 1 and the Problem statement also stop hedging. "Reading the dependency tree, that looks unnecessary" and "there should be no cgo dependency at all" were written before the check ran; both now state the measured result and the dependency table names go-tree-sitter as the only cgo path rather than implying there might be others. The Problem section says plainly that step one of the current onboarding is avoidable and that our own quickstart is wrong about it, because that is the finding most likely to be acted on independently of the rest of the proposal. Docs only — no code, no behavior change. Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com> Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The previous commit put both checks at the top of the spec, which was the wrong call. They were de-risking, not findings: each came back the way the proposal already assumed, so neither changes a decision. Opening with them made a reviewer read what we had been worried about before reading what is being proposed. The checks now live in an "Appendix: Verification" at the end, with the reproduce commands and the honest note about the cg_skeleton check first failing for the wrong reason. The body keeps only what is material where it is material: piece 1 states that CGO_ENABLED=0 yields a static 30.5 MB binary for all four targets and that the quickstart's C-toolchain instruction is wrong, and piece 4 states in one sentence that env blocks merge per key so a user-scope install is not clobbered. Both point at the appendix rather than carrying the evidence inline. Also drops the "Gated on" column from the ship table and the surviving "Gate A/Gate B" references in prose, which only made sense while the gates were the frame. Net 10 lines shorter and the proposal reads as a proposal. Docs only — no code, no behavior change. Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com> Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…-gating Two gaps, one of them a correctness bug in the design. The proposal described what we build but never what the evaluator actually does, which is the part a reviewer will judge. A "What the user does" section now answers the question it kept raising: nothing is copied into their project and the skill is not something they place. /plugin install writes enabledPlugins to ~/.claude/settings.json, so the plugin and its skills are installed once per machine and reachable from any repo; only the routing decision is per-repo, deliberately, because that is the one with blast radius. It lists the three commands, what the install skill does at each step, the realistic count of approval prompts (three to four, of which two happen once ever), and a table of what is global versus per-project. The correctness bug: the spec described the SessionStart hook as unconditional, but the plugin is installed at user scope, so its hooks run in EVERY project — including ones the user never routed, where starting a proxy is pure waste. The hook now self-gates on ANTHROPIC_BASE_URL. Settings `env` values are written into the process environment and hook processes inherit it, so the hook exits immediately unless the variable names our port. That needs no extra configuration, keeps no second copy of the port to drift, and degrades correctly: remove the env key by hand and the hook stops firing on its own. Also moves the new section above "What we ship" so the ### pieces stay nested under it rather than under the journey, and removes a duplicated paragraph in piece 1 that stated the quickstart problem twice. Docs only — no code, no behavior change. Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com> Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Collaborator
|
Reviewed by re-running both committed gate scripts independently (not trusting the issue comment's reported results) and fact-checking the two claims the doc itself calls "most consequential" against the current, live docs rather than memory. CI is green, no action needed
Gate A (pure-Go static binary) — re-run independently, all four checks confirmedDocker was available but couldn't pull
Gate B (settings.json per-key env merge) — re-run, confirmed exactlyRan The two "most consequential" claims — checked against current, live docs, not training-data memoryBoth confirmed verbatim against
AssessmentEvery claim checked out — both gates, both "most consequential" facts, plus the incidental corroborations. No red flags. The doc's own framing of the quickstart's |
amiddavid
added a commit
that referenced
this pull request
Aug 30, 2026
…p figure Review found one broken thing, settled the doc's central open question, and caught a wrong number. All four points addressed. CI was red: mkdocs build --strict rejected docs/analysis/claude-code-plugin.md because it was in no nav entry, and there is no docs/analysis convention to copy. Moved it to docs/superpowers/specs/2026-08-30-claude-code-plugin-transport-design.md, which exclude_docs already covers and whose stated purpose — "working plans/specs ... not published site content" — is what this doc is. That also matches the dated-spec naming of the keepalive design doc and of the local-distribution proposal in #130. The reviewer offered the alternative of a nav entry under Results:; exclusion is the better fit, because publishing a page that evaluates a plugin we have not built would read on the docs site as a product that exists. Verified with a real strict build (1.45s, no warnings) and a negative control: the same file under docs/analysis/ still aborts strict mode, so the move is the fix rather than something incidental. Gate 0 is closed, in the doc's favour. Review ran the experiment the doc asked for — a PostToolUse hook replacing a Bash output with a sentinel, captured through a raw-logging reverse proxy so the evidence is the literal outbound body rather than the transcript file — and the replacement persists and is resent verbatim on later turns. A working collapse plugin then measured -6,285 tokens on a real session, appearing as the same reduction on turn 1's cache-write and turn 2's cache-read, which is what separates a permanent reduction of resent context from a one-turn display trick. The "three things to verify" section becomes a resolved-gate section plus the risks that actually remain, and the recommendation stops being conditional. The predicted failure mode arrived on the first attempt, which is worth recording rather than smoothing over: updatedToolOutput must be the object tool_response shape ({stdout, stderr, interrupted, isImage, noOutputExpected} for Bash), and a bare string is silently ignored. That is now a hard requirement on adapters/cchook — emit the object shape, count your own rejections — instead of a general warning. The 10,000-character cap figure was wrong, and "cap" was the wrong frame. That number came from the additionalContext / systemMessage / plain-stdout cap, which does not govern this field. Measured: verbatim and uncapped to ~30,000 chars, real threshold in (30,000, 40,000] and most likely 32,768, and above it neither truncation nor rejection — the CLI's ordinary large-output handler produces a ~2,260-char <persisted-output> wrapper with a 2KB preview and a disk pointer while the local record stays intact. For an oversized hook emission that is a token-cost improvement, not a hazard. Two precision fixes. The envelope claim now records that it was verified by a stronger method than the docs (11 events exercised live, plus the installed CLI's own hookSpecificOutput validation schema: 33 events, 22 with output fields, none touching the envelope) and carries the nuance that promptCacheTtl / CLAUDE_CODE_PROMPT_CACHE_TTL is a reachable cache-TTL lever via settings or env — not via any hook or manifest field, and session-wide rather than per-breakpoint — so a flat "zero cache control" reading would overstate the gap. And the component figures quoted throughout (cachesplit's -34.1%, mask's 27.5%/12.5%, the ~7,017-token system block) are now labelled as this repo's frozen historical measurements, quoted accurately but not re-verified against current traffic by this evaluation. Finally, the recommendation leads with the scope the reviewer articulated better than the doc did: a plugin can replicate the offloader half of this repo, persistently and measurably, and categorically cannot replicate the cache-management half — not "a plugin can do what the proxy does." Docs only — no code, no behavior change. Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com> Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This was referenced Aug 30, 2026
Collaborator
Author
|
Implemented in #141 (all three stages, based on Decisions taken on the open questions, all reversible in review: routing scope project-local by default; Two notes on the spec itself:
Also split out of the implementation: #142 (the preset table was stale in every row) and #143 (the same drift in prose, unfixed). |
Collaborator
|
Reviewed against main The spec is well written and the review round improved it (the self-gating The findings are about the gates, which are the executable part of this PR and the whole point of 1. Blocking —
|
Collaborator
|
Three additional findings after running the second gate and checking CI. 8. Blocking —
|
amiddavid
added a commit
that referenced
this pull request
Sep 1, 2026
…way conformance Implements the local-distribution proposal (#130) minus the Claude Code plugin, which ships as its own PR because all six blocking findings from the review of #141 live in it. Nothing here is held behind that. ## 1. The toolchain gate was our own documentation `docs/get-started/quickstart-proxy.md`, `docs/setup.md` and `docs/hosted.md` all told evaluators to install a C toolchain and set `CGO_ENABLED=1`. That is true only for a `cg_skeleton` build. setup.md went further and named bifrost's tokenizer as a cgo dependency, which it is not — o200k_base is embedded (`internal/tokens/tokens.go`). Verified directly on go 1.26.4 rather than taken from the proposal: `CGO_ENABLED=0` with default tags builds all four release targets (linux/darwin x amd64/arm64, 27.1-33.5 MB stripped), `file` reports "statically linked", `ldd` reports "not a dynamic executable", the binary starts and answers /healthz, and `-tags cg_skeleton` fails under CGO_ENABLED=0 with the build-constraints signature — confirming tree-sitter is the only C dependency. - `.goreleaser.yaml`: a plain GOOS/GOARCH matrix, no cross-toolchains, no `brews:` block (the tap repo and release signing are an unowned question, and nothing may depend on a repo that does not exist). - `.github/workflows/release.yaml`: a tag publishes, `workflow_dispatch` builds the same matrix as a snapshot. It asserts the pure-Go claim with `CC=/nonexistent-c-compiler`. - `make build-static`. The Makefile keeps `CGO_ENABLED=1` because `go test -race` needs it, and the comment now says that is a test-time requirement — reading it as a shipping requirement is how the wrong claim reached the docs. ## 2. A `cache` preset: cachesplit alone The funnel's default, chosen so a stranger can verify the claim by reading one line rather than trusting four components. Not `safe`, whose extra components are lossless in meaning but still rewrite the JSON. ## 3. `--idle-exit`: the proxy cleans itself up Off by default; a gateway or eval-containers deployment must never self-terminate. A signal and the watchdog converge on the SAME teardown, so the self-killing path cannot drift from the one known to work. Two properties are load-bearing: - **The keep-alive inverts "idle."** Pinging is what the proxy does precisely while no client traffic arrives — the quiet gap after `end_turn`, where 83.7% of the recoverable dollars sit. A pending ping both vetoes the exit and RESETS the clock, so retiring the last ping buys a full fresh threshold rather than exiting moments later. - **Exit wipes the in-memory store.** `store.ValidateIdleExit` refuses anything below `max(2 x store.ttl_seconds, 1h)` at startup — ~5h34m at the default. 2x because the TTL is a sliding window. `NewMemory` now calls the same `Options.EffectiveTTL` the floor is computed from, so the two cannot drift. ## 4. Gateway conformance All five items from the proposal, under the `cache` preset. Four were already correct and are now pinned by tests; `POST /anthropic/v1/messages/count_tokens` was missing entirely — without it a client counts context by issuing INFERENCE requests, billed calls added by a proxy sold on removing them. ## 5. Review of #141: the `cache` preset advertised a tool whose every call must fail Five places promised it did not: `config/config.go`, `docs/reference/presets.md`, `docs/how-to/choose-a-preset.md`, the plugin doc, and the install skill. Verified before fixing — `[Read Bash]` in, `[Read Bash context_guru_expand]` out on the real gateway route. Root cause was a code-vs-comment contradiction. `Options.InjectExpand` documented the gate as requiring "an expandable marker"; `expand/inject.go` says "No marker condition, deliberately" and the real conditions were mode, store-persists and has-tools. Nothing asked whether the pipeline could produce a marker at all. `components.Pipeline.HasOffload()` answers that by type assertion rather than a name list (a name list is a second copy of "which components are lossy" and drifts the moment somebody adds one). Under `auto`, injection now requires it. `always` still injects unconditionally — an operator who asks for it by name gets it. This also fixes `off`, the A/B control arm, which was carrying an extra tool declaration. Marker independence is preserved, which is the invariant that matters for cache stability: a pipeline does not change turn to turn, so the tools array stays byte-stable across a session. **Ten existing expand tests changed fixture, and that is worth reading.** They hand-seed the Store to simulate an offload, but built their handler with `pipeline: []` — a pipeline that cannot offload anything. That was harmless only while injection ignored the pipeline. They now use `offloadCapablePipeline` (`[linecap]`, which does not act on their short bodies), so each fixture matches its own premise. No assertion was weakened. ## 6. The rest of the review - **`--idle-exit` was defeated by any health probe** (finding 7). `/healthz` and `/metrics` no longer count as activity: a probe on a schedule shorter than the threshold meant the exit NEVER fired and logged nothing to say so — measured, a 1h-threshold proxy reporting "idle for 1h3m0s" after 2h03m. A dashboard poll still counts, deliberately: a probe is not a viewer, and exiting under somebody who is watching is the worse failure. - **A gateway may no longer self-terminate.** `--idle-exit` with `--upstreams` is refused at startup. That safety was previously accidental — it held only because hosted deployments run a liveness probe, which the change above stops counting. - **The floor's refusal was logged after "listening"**, so a rejected configuration read as a crash. Both refusals moved earlier and into one testable `checkIdleExit`. - **`--listen` and `--version` flags** (findings 2 and 12, which are the plugin's, but the flags are the core's). The address reached the process only through the environment, so no supervisor or `ps` could tell which port an instance held; and `buildinfo.Version` was reachable only via `/stats` on a running proxy, so an installer asking `--help` recorded "Usage of context-guru-proxy:" as the installed version. - **Nothing tested the shipped configuration** (finding 9). A tag published without running any tests, and CI runs the suite only with `CGO_ENABLED=1` — so `TestEveryPresetBuilds`, which guards exactly the CGO-free artifact, was never executed in that configuration. The release workflow now runs a CGO-off suite over the packages whose behaviour depends on which components are compiled in, plus the full suite, before publishing. It also asserts `--version` answers. - **`scripts/gate-a-purego.sh` was cited as proof in four places and is not in this PR** (finding 8). Those now cite the release workflow's own assert step, which exists here and fails the release if a cgo dependency escapes the `cg_skeleton` tag. - **The savings claim was measured in the wrong regime.** −34.1% / 96.7% comes from a harness running tasks back-to-back inside the provider's 5-minute TTL, and is one task measured three times; this project's own interactive figure is $0.0298 across 1,127 sessions, with 1,105 of 1,127 session starts reading zero from cache. Both are now stated, with the zero cases (outside a git repo, under the 1,024-token floor, non-Anthropic backend). The old citation pointed at `docs/results/context-guru.md`, which contains neither number. - **`count_tokens` behaviour kept, consequence documented** (finding 10). It answers about the ORIGINAL body — over-reporting is recoverable, under-reporting costs a failed turn when a fail-open component reverts. What was undocumented is the cost: the client self-compacts earlier than needed (115,933 reported vs 32,802 forwarded). Now in `docs/reference/routes.md`, where the route was absent entirely. Its **hosted branch was wholly untested** — that branch is all that stands between the multi-tenant service and an unmetered open forwarder — so it now has one. - `--idle-exit` and `--version` added to both flag tables; `bytes.NewReader` in counttokens. ## Verification Six mutations, each proven to have landed in the source before its result was allowed to count: expand injection ungated (the defect) -> TestCachePresetAdvertisesNoExtraTool FAIL cache: sent [Read Bash], forwarded [Read Bash context_guru_expand] off: sent [Read Bash], forwarded [Read Bash context_guru_expand] HasOffload always true -> same test FAIL, same two subcases HasOffload always false -> FAIL on the offloader subcase: "mints markers but no longer advertises the expand tool, so a model cannot recover what it offloaded" probes count as activity again -> TestProbesDoNotDeferIdleExit FAIL ("two hours of nothing but liveness probes: idle past the threshold, but watchIdle never exited") gateway guard disabled -> TestCheckIdleExitRefusesAGatewaySelfTerminating FAIL count_tokens hosted auth removed -> TestCountTokensHostedRequiresAuth FAIL (502, want 401) The third mutation is the one worth noting: it proves the fix did not trade a silent defect for its mirror image, an offloader whose output nothing can expand. `go build ./...`, `go vet ./...`, `gofmt -l` and the full `go test ./...` are clean. Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com>
amiddavid
added a commit
that referenced
this pull request
Sep 1, 2026
…way conformance Implements the local-distribution proposal (#130) minus the Claude Code plugin, which ships as its own PR because all six blocking findings from the review of #141 live in it. Nothing here is held behind that. ## 1. The toolchain gate was our own documentation `docs/get-started/quickstart-proxy.md`, `docs/setup.md` and `docs/hosted.md` all told evaluators to install a C toolchain and set `CGO_ENABLED=1`. That is true only for a `cg_skeleton` build. setup.md went further and named bifrost's tokenizer as a cgo dependency, which it is not — o200k_base is embedded (`internal/tokens/tokens.go`). Verified directly on go 1.26.4 rather than taken from the proposal: `CGO_ENABLED=0` with default tags builds all four release targets (linux/darwin x amd64/arm64, 27.1-33.5 MB stripped), `file` reports "statically linked", `ldd` reports "not a dynamic executable", the binary starts and answers /healthz, and `-tags cg_skeleton` fails under CGO_ENABLED=0 with the build-constraints signature — confirming tree-sitter is the only C dependency. - `.goreleaser.yaml`: a plain GOOS/GOARCH matrix, no cross-toolchains, no `brews:` block (the tap repo and release signing are an unowned question, and nothing may depend on a repo that does not exist). - `.github/workflows/release.yaml`: a tag publishes, `workflow_dispatch` builds the same matrix as a snapshot. It asserts the pure-Go claim with `CC=/nonexistent-c-compiler`. - `make build-static`. The Makefile keeps `CGO_ENABLED=1` because `go test -race` needs it, and the comment now says that is a test-time requirement — reading it as a shipping requirement is how the wrong claim reached the docs. ## 2. A `cache` preset: cachesplit alone The funnel's default, chosen so a stranger can verify the claim by reading one line rather than trusting four components. Not `safe`, whose extra components are lossless in meaning but still rewrite the JSON. ## 3. `--idle-exit`: the proxy cleans itself up Off by default; a gateway or eval-containers deployment must never self-terminate. A signal and the watchdog converge on the SAME teardown, so the self-killing path cannot drift from the one known to work. Two properties are load-bearing: - **The keep-alive inverts "idle."** Pinging is what the proxy does precisely while no client traffic arrives — the quiet gap after `end_turn`, where 83.7% of the recoverable dollars sit. A pending ping both vetoes the exit and RESETS the clock, so retiring the last ping buys a full fresh threshold rather than exiting moments later. - **Exit wipes the in-memory store.** `store.ValidateIdleExit` refuses anything below `max(2 x store.ttl_seconds, 1h)` at startup — ~5h34m at the default. 2x because the TTL is a sliding window. `NewMemory` now calls the same `Options.EffectiveTTL` the floor is computed from, so the two cannot drift. ## 4. Gateway conformance All five items from the proposal, under the `cache` preset. Four were already correct and are now pinned by tests; `POST /anthropic/v1/messages/count_tokens` was missing entirely — without it a client counts context by issuing INFERENCE requests, billed calls added by a proxy sold on removing them. ## 5. Review of #141: the `cache` preset advertised a tool whose every call must fail Five places promised it did not: `config/config.go`, `docs/reference/presets.md`, `docs/how-to/choose-a-preset.md`, the plugin doc, and the install skill. Verified before fixing — `[Read Bash]` in, `[Read Bash context_guru_expand]` out on the real gateway route. Root cause was a code-vs-comment contradiction. `Options.InjectExpand` documented the gate as requiring "an expandable marker"; `expand/inject.go` says "No marker condition, deliberately" and the real conditions were mode, store-persists and has-tools. Nothing asked whether the pipeline could produce a marker at all. `components.Pipeline.HasOffload()` answers that by type assertion rather than a name list (a name list is a second copy of "which components are lossy" and drifts the moment somebody adds one). Under `auto`, injection now requires it. `always` still injects unconditionally — an operator who asks for it by name gets it. This also fixes `off`, the A/B control arm, which was carrying an extra tool declaration. Marker independence is preserved, which is the invariant that matters for cache stability: a pipeline does not change turn to turn, so the tools array stays byte-stable across a session. **Ten existing expand tests changed fixture, and that is worth reading.** They hand-seed the Store to simulate an offload, but built their handler with `pipeline: []` — a pipeline that cannot offload anything. That was harmless only while injection ignored the pipeline. They now use `offloadCapablePipeline` (`[linecap]`, which does not act on their short bodies), so each fixture matches its own premise. No assertion was weakened. ## 6. The rest of the review - **`--idle-exit` was defeated by any health probe** (finding 7). `/healthz` and `/metrics` no longer count as activity: a probe on a schedule shorter than the threshold meant the exit NEVER fired and logged nothing to say so — measured, a 1h-threshold proxy reporting "idle for 1h3m0s" after 2h03m. A dashboard poll still counts, deliberately: a probe is not a viewer, and exiting under somebody who is watching is the worse failure. - **A gateway may no longer self-terminate.** `--idle-exit` with `--upstreams` is refused at startup. That safety was previously accidental — it held only because hosted deployments run a liveness probe, which the change above stops counting. - **The floor's refusal was logged after "listening"**, so a rejected configuration read as a crash. Both refusals moved earlier and into one testable `checkIdleExit`. - **`--listen` and `--version` flags** (findings 2 and 12, which are the plugin's, but the flags are the core's). The address reached the process only through the environment, so no supervisor or `ps` could tell which port an instance held; and `buildinfo.Version` was reachable only via `/stats` on a running proxy, so an installer asking `--help` recorded "Usage of context-guru-proxy:" as the installed version. - **Nothing tested the shipped configuration** (finding 9). A tag published without running any tests, and CI runs the suite only with `CGO_ENABLED=1` — so `TestEveryPresetBuilds`, which guards exactly the CGO-free artifact, was never executed in that configuration. The release workflow now runs a CGO-off suite over the packages whose behaviour depends on which components are compiled in, plus the full suite, before publishing. It also asserts `--version` answers. - **`scripts/gate-a-purego.sh` was cited as proof in four places and is not in this PR** (finding 8). Those now cite the release workflow's own assert step, which exists here and fails the release if a cgo dependency escapes the `cg_skeleton` tag. - **The savings claim was measured in the wrong regime.** −34.1% / 96.7% comes from a harness running tasks back-to-back inside the provider's 5-minute TTL, and is one task measured three times; this project's own interactive figure is $0.0298 across 1,127 sessions, with 1,105 of 1,127 session starts reading zero from cache. Both are now stated, with the zero cases (outside a git repo, under the 1,024-token floor, non-Anthropic backend). The old citation pointed at `docs/results/context-guru.md`, which contains neither number. - **`count_tokens` behaviour kept, consequence documented** (finding 10). It answers about the ORIGINAL body — over-reporting is recoverable, under-reporting costs a failed turn when a fail-open component reverts. What was undocumented is the cost: the client self-compacts earlier than needed (115,933 reported vs 32,802 forwarded). Now in `docs/reference/routes.md`, where the route was absent entirely. Its **hosted branch was wholly untested** — that branch is all that stands between the multi-tenant service and an unmetered open forwarder — so it now has one. - `--idle-exit` and `--version` added to both flag tables; `bytes.NewReader` in counttokens. ## Verification Six mutations, each proven to have landed in the source before its result was allowed to count: expand injection ungated (the defect) -> TestCachePresetAdvertisesNoExtraTool FAIL cache: sent [Read Bash], forwarded [Read Bash context_guru_expand] off: sent [Read Bash], forwarded [Read Bash context_guru_expand] HasOffload always true -> same test FAIL, same two subcases HasOffload always false -> FAIL on the offloader subcase: "mints markers but no longer advertises the expand tool, so a model cannot recover what it offloaded" probes count as activity again -> TestProbesDoNotDeferIdleExit FAIL ("two hours of nothing but liveness probes: idle past the threshold, but watchIdle never exited") gateway guard disabled -> TestCheckIdleExitRefusesAGatewaySelfTerminating FAIL count_tokens hosted auth removed -> TestCountTokensHostedRequiresAuth FAIL (502, want 401) The third mutation is the one worth noting: it proves the fix did not trade a silent defect for its mirror image, an offloader whose output nothing can expand. `go build ./...`, `go vet ./...`, `gofmt -l` and the full `go test ./...` are clean. Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com>
amiddavid
added a commit
that referenced
this pull request
Sep 1, 2026
…p figure Review found one broken thing, settled the doc's central open question, and caught a wrong number. All four points addressed. CI was red: mkdocs build --strict rejected docs/analysis/claude-code-plugin.md because it was in no nav entry, and there is no docs/analysis convention to copy. Moved it to docs/superpowers/specs/2026-08-30-claude-code-plugin-transport-design.md, which exclude_docs already covers and whose stated purpose — "working plans/specs ... not published site content" — is what this doc is. That also matches the dated-spec naming of the keepalive design doc and of the local-distribution proposal in #130. The reviewer offered the alternative of a nav entry under Results:; exclusion is the better fit, because publishing a page that evaluates a plugin we have not built would read on the docs site as a product that exists. Verified with a real strict build (1.45s, no warnings) and a negative control: the same file under docs/analysis/ still aborts strict mode, so the move is the fix rather than something incidental. Gate 0 is closed, in the doc's favour. Review ran the experiment the doc asked for — a PostToolUse hook replacing a Bash output with a sentinel, captured through a raw-logging reverse proxy so the evidence is the literal outbound body rather than the transcript file — and the replacement persists and is resent verbatim on later turns. A working collapse plugin then measured -6,285 tokens on a real session, appearing as the same reduction on turn 1's cache-write and turn 2's cache-read, which is what separates a permanent reduction of resent context from a one-turn display trick. The "three things to verify" section becomes a resolved-gate section plus the risks that actually remain, and the recommendation stops being conditional. The predicted failure mode arrived on the first attempt, which is worth recording rather than smoothing over: updatedToolOutput must be the object tool_response shape ({stdout, stderr, interrupted, isImage, noOutputExpected} for Bash), and a bare string is silently ignored. That is now a hard requirement on adapters/cchook — emit the object shape, count your own rejections — instead of a general warning. The 10,000-character cap figure was wrong, and "cap" was the wrong frame. That number came from the additionalContext / systemMessage / plain-stdout cap, which does not govern this field. Measured: verbatim and uncapped to ~30,000 chars, real threshold in (30,000, 40,000] and most likely 32,768, and above it neither truncation nor rejection — the CLI's ordinary large-output handler produces a ~2,260-char <persisted-output> wrapper with a 2KB preview and a disk pointer while the local record stays intact. For an oversized hook emission that is a token-cost improvement, not a hazard. Two precision fixes. The envelope claim now records that it was verified by a stronger method than the docs (11 events exercised live, plus the installed CLI's own hookSpecificOutput validation schema: 33 events, 22 with output fields, none touching the envelope) and carries the nuance that promptCacheTtl / CLAUDE_CODE_PROMPT_CACHE_TTL is a reachable cache-TTL lever via settings or env — not via any hook or manifest field, and session-wide rather than per-breakpoint — so a flat "zero cache control" reading would overstate the gap. And the component figures quoted throughout (cachesplit's -34.1%, mask's 27.5%/12.5%, the ~7,017-token system block) are now labelled as this repo's frozen historical measurements, quoted accurately but not re-verified against current traffic by this evaluation. Finally, the recommendation leads with the scope the reviewer articulated better than the doc did: a plugin can replicate the offloader half of this repo, persistently and measurably, and categorically cannot replicate the cache-management half — not "a plugin can do what the proxy does." Docs only — no code, no behavior change. Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com> Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
amiddavid
added a commit
that referenced
this pull request
Sep 1, 2026
…by regime, and add rtk Addresses the three blocking items from review, plus the cheap ones. 1. Two candidates, and only one is a new transport. The document analysed plugin-as-interceptor (PostToolUse -> updatedToolOutput) without ever weighing plugin-as-installer-for-the-proxy (a skill writing env.ANTHROPIC_BASE_URL into the user's own settings.json), which is what #130 proposes and which gives up none of the component set. New section states the distinction, the component tables are scoped explicitly to (A), and a header note stops this page being read as the justification for #130. The headline blocker is rescoped: a plugin *manifest's* settings.json accepts only agent/subagentStatusLine -- that is not the same as "a plugin cannot set ANTHROPIC_BASE_URL", and the difference is the option that won. Title said "third transport" against the body's "fourth"; now fourth throughout. 2. cachesplit is priced by regime. -34.1%/0%->96.7% is a warm-regime figure -- one Terminal-Bench task x 3 trials (cacheinject.md:203, and :209 says treat it as one task, not a fleet average) with the A/B run back-to-back inside the TTL (dashboard.md:219). Cold interactive traffic is $0.0298 across 1,127 sessions / 11,361 requests (dashboard.md:204). Both are right; the document transferred the warm number onto a definitionally cold-regime user, called cachesplit "the best-evidenced component we have" where cacheinject.md:209 says otherwise, and claimed the figures were "not re-verified against current traffic" -- which for cachesplit is false. Corrected, the Anthropic row reads close to the vLLM row. The DAM recommendation stands but is now justified on spendgate/tenancy/limits and harness-plurality rather than on that figure. 3. Gate 0 is closed consistently. The ranked go/no-go section still called the case conditional and told the reader to run the experiment that already ran. Also: rtk is this architecture and we benchmarked it as a full arm -- -9.0% billed cost, reward-neutral, zero request-path latency (results/rtk.md:11) -- so expected value has a floor instead of resting on one session's -6,285 tokens, and the doc now claims the edge it was missing: rtk is a shell hook, so Read/Grep/Glob bypass it, while matcher ".*" does not. mask quoted as ~27.5-29.5%, single-task replay, never enforced in a benchmark arm. extract_llm 8x -> 82x underwater. Hook events ~34 -> 33. Dropped the guessed 32,768 threshold, keeping the measured (30,000, 40,000] bracket. Noted that inspect_transcript.py reports key names, not record types, so its `system` record type is a transcript event and not a request system array. Keepalive branch reference replaced with #126; extract_llm_sweep (#118) added to the offload table. inspect_transcript.py moved to deploy/harbor/, this repo's convention for analysis Python. Rebased onto main (3ebc65d). Docs-only; no code and no behaviour change. mkdocs is unaffected because docs/superpowers/ is excluded from the site build (mkdocs.yml:88-90); the three in-page anchors were validated against the file's own heading slugs and all other links are external. Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com>
amiddavid
added a commit
that referenced
this pull request
Sep 1, 2026
…way conformance Implements the local-distribution proposal (#130) minus the Claude Code plugin, which ships as its own PR because all six blocking findings from the review of #141 live in it. Nothing here is held behind that. `docs/get-started/quickstart-proxy.md`, `docs/setup.md` and `docs/hosted.md` all told evaluators to install a C toolchain and set `CGO_ENABLED=1`. That is true only for a `cg_skeleton` build. setup.md went further and named bifrost's tokenizer as a cgo dependency, which it is not — o200k_base is embedded (`internal/tokens/tokens.go`). Verified directly on go 1.26.4 rather than taken from the proposal: `CGO_ENABLED=0` with default tags builds all four release targets (linux/darwin x amd64/arm64, 27.1-33.5 MB stripped), `file` reports "statically linked", `ldd` reports "not a dynamic executable", the binary starts and answers /healthz, and `-tags cg_skeleton` fails under CGO_ENABLED=0 with the build-constraints signature — confirming tree-sitter is the only C dependency. - `.goreleaser.yaml`: a plain GOOS/GOARCH matrix, no cross-toolchains, no `brews:` block (the tap repo and release signing are an unowned question, and nothing may depend on a repo that does not exist). - `.github/workflows/release.yaml`: a tag publishes, `workflow_dispatch` builds the same matrix as a snapshot. It asserts the pure-Go claim with `CC=/nonexistent-c-compiler`. - `make build-static`. The Makefile keeps `CGO_ENABLED=1` because `go test -race` needs it, and the comment now says that is a test-time requirement — reading it as a shipping requirement is how the wrong claim reached the docs. The funnel's default, chosen so a stranger can verify the claim by reading one line rather than trusting four components. Not `safe`, whose extra components are lossless in meaning but still rewrite the JSON. Off by default; a gateway or eval-containers deployment must never self-terminate. A signal and the watchdog converge on the SAME teardown, so the self-killing path cannot drift from the one known to work. Two properties are load-bearing: - **The keep-alive inverts "idle."** Pinging is what the proxy does precisely while no client traffic arrives — the quiet gap after `end_turn`, where 83.7% of the recoverable dollars sit. A pending ping both vetoes the exit and RESETS the clock, so retiring the last ping buys a full fresh threshold rather than exiting moments later. - **Exit wipes the in-memory store.** `store.ValidateIdleExit` refuses anything below `max(2 x store.ttl_seconds, 1h)` at startup — ~5h34m at the default. 2x because the TTL is a sliding window. `NewMemory` now calls the same `Options.EffectiveTTL` the floor is computed from, so the two cannot drift. All five items from the proposal, under the `cache` preset. Four were already correct and are now pinned by tests; `POST /anthropic/v1/messages/count_tokens` was missing entirely — without it a client counts context by issuing INFERENCE requests, billed calls added by a proxy sold on removing them. Five places promised it did not: `config/config.go`, `docs/reference/presets.md`, `docs/how-to/choose-a-preset.md`, the plugin doc, and the install skill. Verified before fixing — `[Read Bash]` in, `[Read Bash context_guru_expand]` out on the real gateway route. Root cause was a code-vs-comment contradiction. `Options.InjectExpand` documented the gate as requiring "an expandable marker"; `expand/inject.go` says "No marker condition, deliberately" and the real conditions were mode, store-persists and has-tools. Nothing asked whether the pipeline could produce a marker at all. `components.Pipeline.HasOffload()` answers that by type assertion rather than a name list (a name list is a second copy of "which components are lossy" and drifts the moment somebody adds one). Under `auto`, injection now requires it. `always` still injects unconditionally — an operator who asks for it by name gets it. This also fixes `off`, the A/B control arm, which was carrying an extra tool declaration. Marker independence is preserved, which is the invariant that matters for cache stability: a pipeline does not change turn to turn, so the tools array stays byte-stable across a session. **Ten existing expand tests changed fixture, and that is worth reading.** They hand-seed the Store to simulate an offload, but built their handler with `pipeline: []` — a pipeline that cannot offload anything. That was harmless only while injection ignored the pipeline. They now use `offloadCapablePipeline` (`[linecap]`, which does not act on their short bodies), so each fixture matches its own premise. No assertion was weakened. - **`--idle-exit` was defeated by any health probe** (finding 7). `/healthz` and `/metrics` no longer count as activity: a probe on a schedule shorter than the threshold meant the exit NEVER fired and logged nothing to say so — measured, a 1h-threshold proxy reporting "idle for 1h3m0s" after 2h03m. A dashboard poll still counts, deliberately: a probe is not a viewer, and exiting under somebody who is watching is the worse failure. - **A gateway may no longer self-terminate.** `--idle-exit` with `--upstreams` is refused at startup. That safety was previously accidental — it held only because hosted deployments run a liveness probe, which the change above stops counting. - **The floor's refusal was logged after "listening"**, so a rejected configuration read as a crash. Both refusals moved earlier and into one testable `checkIdleExit`. - **`--listen` and `--version` flags** (findings 2 and 12, which are the plugin's, but the flags are the core's). The address reached the process only through the environment, so no supervisor or `ps` could tell which port an instance held; and `buildinfo.Version` was reachable only via `/stats` on a running proxy, so an installer asking `--help` recorded "Usage of context-guru-proxy:" as the installed version. - **Nothing tested the shipped configuration** (finding 9). A tag published without running any tests, and CI runs the suite only with `CGO_ENABLED=1` — so `TestEveryPresetBuilds`, which guards exactly the CGO-free artifact, was never executed in that configuration. The release workflow now runs a CGO-off suite over the packages whose behaviour depends on which components are compiled in, plus the full suite, before publishing. It also asserts `--version` answers. - **`scripts/gate-a-purego.sh` was cited as proof in four places and is not in this PR** (finding 8). Those now cite the release workflow's own assert step, which exists here and fails the release if a cgo dependency escapes the `cg_skeleton` tag. - **The savings claim was measured in the wrong regime.** −34.1% / 96.7% comes from a harness running tasks back-to-back inside the provider's 5-minute TTL, and is one task measured three times; this project's own interactive figure is $0.0298 across 1,127 sessions, with 1,105 of 1,127 session starts reading zero from cache. Both are now stated, with the zero cases (outside a git repo, under the 1,024-token floor, non-Anthropic backend). The old citation pointed at `docs/results/context-guru.md`, which contains neither number. - **`count_tokens` behaviour kept, consequence documented** (finding 10). It answers about the ORIGINAL body — over-reporting is recoverable, under-reporting costs a failed turn when a fail-open component reverts. What was undocumented is the cost: the client self-compacts earlier than needed (115,933 reported vs 32,802 forwarded). Now in `docs/reference/routes.md`, where the route was absent entirely. Its **hosted branch was wholly untested** — that branch is all that stands between the multi-tenant service and an unmetered open forwarder — so it now has one. - `--idle-exit` and `--version` added to both flag tables; `bytes.NewReader` in counttokens. Six mutations, each proven to have landed in the source before its result was allowed to count: expand injection ungated (the defect) -> TestCachePresetAdvertisesNoExtraTool FAIL cache: sent [Read Bash], forwarded [Read Bash context_guru_expand] off: sent [Read Bash], forwarded [Read Bash context_guru_expand] HasOffload always true -> same test FAIL, same two subcases HasOffload always false -> FAIL on the offloader subcase: "mints markers but no longer advertises the expand tool, so a model cannot recover what it offloaded" probes count as activity again -> TestProbesDoNotDeferIdleExit FAIL ("two hours of nothing but liveness probes: idle past the threshold, but watchIdle never exited") gateway guard disabled -> TestCheckIdleExitRefusesAGatewaySelfTerminating FAIL count_tokens hosted auth removed -> TestCountTokensHostedRequiresAuth FAIL (502, want 401) The third mutation is the one worth noting: it proves the fix did not trade a silent defect for its mirror image, an offloader whose output nothing can expand. `go build ./...`, `go vet ./...`, `gofmt -l` and the full `go test ./...` are clean. Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com>
OsherElhadad
pushed a commit
that referenced
this pull request
Sep 1, 2026
…129) * docs(analysis): evaluate a Claude Code plugin as a fourth transport Asks whether the `components` core can ship as a Claude Code plugin — a thin transport beside `proxy/`, the AuthBridge plugin and `adapters/bifrost`, reusing components via configuration. Not a replacement for the proxy. One fact decides the whole evaluation. No plugin surface can read or write the outbound request: across all ~34 hook events there is no field that rewrites `messages[]`, `system[]`, `tools[]` or `cache_control`, and plugin `settings.json` accepts only `agent` and `subagentStatusLine`, so a plugin cannot even point a session at the proxy. What IS interceptable is one tool result at the moment it is produced, via `PostToolUse` → `updatedToolOutput`. That draws the line between content-scoped and envelope-scoped work, and it is the line between the two component families. Offloaders distribute. `cmdfilter`, `format`, `toon`, `extract`, `collapse`, `smartcrush`, `skeleton` and `dedup` are pure functions from one tool output to a shorter one, which is exactly what the hook hands over and accepts back. Two get better than they are in the proxy: the hook supplies `tool_input.command` and `tool_input.file_path` instead of making `cmdfilter` and `skeleton` infer both from transcript text. Cache management does not distribute, and this is the finding worth arguing with. `cachesplit` restructures the top-level `system` array; `cacheinject` reasons over `messages[]` positionally and emits request metadata; the keepalive on `feat/keepalive-strategies` must originate a request that byte-exactly reproduces a prefix and price it from `CachedTokens`, which only the provider's response carries. None of that is a tool output — it is the envelope, assembled after the last hook runs and never persisted. Verified rather than assumed: a session transcript carries `messages` and `toolUseResult` and no system prompt or tool schemas (`scripts/inspect_transcript.py`). The failure modes are asymmetric too, which is the deeper reason. A wrong offload wastes one expand round-trip, bounded and type-enforced reversible. Wrong cache work inverts: a mistimed keepalive creates at 1.25x instead of refreshing at 0.1x, a breakpoint over budget is a 400, a representation flip inside a cached prefix re-writes the suffix at 11.5x. Envelope work has no merely-no-saving fail direction. The compensating result is that the defensive half of our KV-cache layer stops being necessary rather than being ported. `state.go` names its own premise — an offloader must re-emit identical bytes "otherwise the agent (which re-sends the ORIGINAL each turn)" flips the representation. A hook rewrites the output before it enters the transcript, so the agent never holds the original. Freeze/replay, `MaxCachedIdx`, `Tracker`, `frozen_flips` and sticky ids have nothing left to defend, and `extract_llm`'s sampling nondeterminism stops disqualifying it from repair. Also `PreCompact`/`SessionEnd` replace `proxy/agentcompaction.go`'s string match against Claude Code 2.1.215 internals, which has a documented reachable false positive. What is given up: `cachesplit` (-34.1% cost, 0% -> 96.7% hit, and in every preset), plus `mask` (27.5% Terminal-Bench, 12.5% SWE-bench) and `failed_run`, both of which rewrite EARLIER messages and so cannot work at a hook that fires once at birth. On implicit prefix-cache backends (vLLM/llm-d) the cache loss is zero, because `prefixsplit` is already a no-op there. Also documented: permanence cuts both ways (an expand's restored original joins the transcript for good, so the plugin wants a more conservative pipeline than the proxy), `/stats` cost tiers are unobtainable so plugin mode cannot be benchmarked the way the proxy is, and the whole thing is gated on one unverified fact — whether `updatedToolOutput` persists into the transcript. If it does not, the proposition collapses to an expand-only MCP server. That experiment is named as gate 0 and should run before any adapter code. On DAM: land the proxy in the gateway. DAM is harness-plural and its egress already matches our gateway credential model, so a Claude-Code-only plugin covers one harness of four and none of the bring-your-own-ACP case. Ship the plugin as the Claude-Code-session layer on top, never as the DAM integration. Docs only — no code, no behavior change. Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com> Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs(analysis): address review — fix CI, close Gate 0, correct the cap figure Review found one broken thing, settled the doc's central open question, and caught a wrong number. All four points addressed. CI was red: mkdocs build --strict rejected docs/analysis/claude-code-plugin.md because it was in no nav entry, and there is no docs/analysis convention to copy. Moved it to docs/superpowers/specs/2026-08-30-claude-code-plugin-transport-design.md, which exclude_docs already covers and whose stated purpose — "working plans/specs ... not published site content" — is what this doc is. That also matches the dated-spec naming of the keepalive design doc and of the local-distribution proposal in #130. The reviewer offered the alternative of a nav entry under Results:; exclusion is the better fit, because publishing a page that evaluates a plugin we have not built would read on the docs site as a product that exists. Verified with a real strict build (1.45s, no warnings) and a negative control: the same file under docs/analysis/ still aborts strict mode, so the move is the fix rather than something incidental. Gate 0 is closed, in the doc's favour. Review ran the experiment the doc asked for — a PostToolUse hook replacing a Bash output with a sentinel, captured through a raw-logging reverse proxy so the evidence is the literal outbound body rather than the transcript file — and the replacement persists and is resent verbatim on later turns. A working collapse plugin then measured -6,285 tokens on a real session, appearing as the same reduction on turn 1's cache-write and turn 2's cache-read, which is what separates a permanent reduction of resent context from a one-turn display trick. The "three things to verify" section becomes a resolved-gate section plus the risks that actually remain, and the recommendation stops being conditional. The predicted failure mode arrived on the first attempt, which is worth recording rather than smoothing over: updatedToolOutput must be the object tool_response shape ({stdout, stderr, interrupted, isImage, noOutputExpected} for Bash), and a bare string is silently ignored. That is now a hard requirement on adapters/cchook — emit the object shape, count your own rejections — instead of a general warning. The 10,000-character cap figure was wrong, and "cap" was the wrong frame. That number came from the additionalContext / systemMessage / plain-stdout cap, which does not govern this field. Measured: verbatim and uncapped to ~30,000 chars, real threshold in (30,000, 40,000] and most likely 32,768, and above it neither truncation nor rejection — the CLI's ordinary large-output handler produces a ~2,260-char <persisted-output> wrapper with a 2KB preview and a disk pointer while the local record stays intact. For an oversized hook emission that is a token-cost improvement, not a hazard. Two precision fixes. The envelope claim now records that it was verified by a stronger method than the docs (11 events exercised live, plus the installed CLI's own hookSpecificOutput validation schema: 33 events, 22 with output fields, none touching the envelope) and carries the nuance that promptCacheTtl / CLAUDE_CODE_PROMPT_CACHE_TTL is a reachable cache-TTL lever via settings or env — not via any hook or manifest field, and session-wide rather than per-breakpoint — so a flat "zero cache control" reading would overstate the gap. And the component figures quoted throughout (cachesplit's -34.1%, mask's 27.5%/12.5%, the ~7,017-token system block) are now labelled as this repo's frozen historical measurements, quoted accurately but not re-verified against current traffic by this evaluation. Finally, the recommendation leads with the scope the reviewer articulated better than the doc did: a plugin can replicate the offloader half of this repo, persistently and measurably, and categorically cannot replicate the cache-management half — not "a plugin can do what the proxy does." Docs only — no code, no behavior change. Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com> Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs(analysis): separate the two plugin candidates, price cachesplit by regime, and add rtk Addresses the three blocking items from review, plus the cheap ones. 1. Two candidates, and only one is a new transport. The document analysed plugin-as-interceptor (PostToolUse -> updatedToolOutput) without ever weighing plugin-as-installer-for-the-proxy (a skill writing env.ANTHROPIC_BASE_URL into the user's own settings.json), which is what #130 proposes and which gives up none of the component set. New section states the distinction, the component tables are scoped explicitly to (A), and a header note stops this page being read as the justification for #130. The headline blocker is rescoped: a plugin *manifest's* settings.json accepts only agent/subagentStatusLine -- that is not the same as "a plugin cannot set ANTHROPIC_BASE_URL", and the difference is the option that won. Title said "third transport" against the body's "fourth"; now fourth throughout. 2. cachesplit is priced by regime. -34.1%/0%->96.7% is a warm-regime figure -- one Terminal-Bench task x 3 trials (cacheinject.md:203, and :209 says treat it as one task, not a fleet average) with the A/B run back-to-back inside the TTL (dashboard.md:219). Cold interactive traffic is $0.0298 across 1,127 sessions / 11,361 requests (dashboard.md:204). Both are right; the document transferred the warm number onto a definitionally cold-regime user, called cachesplit "the best-evidenced component we have" where cacheinject.md:209 says otherwise, and claimed the figures were "not re-verified against current traffic" -- which for cachesplit is false. Corrected, the Anthropic row reads close to the vLLM row. The DAM recommendation stands but is now justified on spendgate/tenancy/limits and harness-plurality rather than on that figure. 3. Gate 0 is closed consistently. The ranked go/no-go section still called the case conditional and told the reader to run the experiment that already ran. Also: rtk is this architecture and we benchmarked it as a full arm -- -9.0% billed cost, reward-neutral, zero request-path latency (results/rtk.md:11) -- so expected value has a floor instead of resting on one session's -6,285 tokens, and the doc now claims the edge it was missing: rtk is a shell hook, so Read/Grep/Glob bypass it, while matcher ".*" does not. mask quoted as ~27.5-29.5%, single-task replay, never enforced in a benchmark arm. extract_llm 8x -> 82x underwater. Hook events ~34 -> 33. Dropped the guessed 32,768 threshold, keeping the measured (30,000, 40,000] bracket. Noted that inspect_transcript.py reports key names, not record types, so its `system` record type is a transcript event and not a request system array. Keepalive branch reference replaced with #126; extract_llm_sweep (#118) added to the offload table. inspect_transcript.py moved to deploy/harbor/, this repo's convention for analysis Python. Rebased onto main (3ebc65d). Docs-only; no code and no behaviour change. mkdocs is unaffected because docs/superpowers/ is excluded from the site build (mkdocs.yml:88-90); the three in-page anchors were validated against the file's own heading slugs and all other links are external. Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com> --------- Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com>
amiddavid
added a commit
that referenced
this pull request
Sep 1, 2026
…way conformance Implements the local-distribution proposal (#130) minus the Claude Code plugin, which ships as its own PR because all six blocking findings from the review of #141 live in it. Nothing here is held behind that. `docs/get-started/quickstart-proxy.md`, `docs/setup.md` and `docs/hosted.md` all told evaluators to install a C toolchain and set `CGO_ENABLED=1`. That is true only for a `cg_skeleton` build. setup.md went further and named bifrost's tokenizer as a cgo dependency, which it is not — o200k_base is embedded (`internal/tokens/tokens.go`). Verified directly on go 1.26.4 rather than taken from the proposal: `CGO_ENABLED=0` with default tags builds all four release targets (linux/darwin x amd64/arm64, 27.1-33.5 MB stripped), `file` reports "statically linked", `ldd` reports "not a dynamic executable", the binary starts and answers /healthz, and `-tags cg_skeleton` fails under CGO_ENABLED=0 with the build-constraints signature — confirming tree-sitter is the only C dependency. - `.goreleaser.yaml`: a plain GOOS/GOARCH matrix, no cross-toolchains, no `brews:` block (the tap repo and release signing are an unowned question, and nothing may depend on a repo that does not exist). - `.github/workflows/release.yaml`: a tag publishes, `workflow_dispatch` builds the same matrix as a snapshot. It asserts the pure-Go claim with `CC=/nonexistent-c-compiler`. - `make build-static`. The Makefile keeps `CGO_ENABLED=1` because `go test -race` needs it, and the comment now says that is a test-time requirement — reading it as a shipping requirement is how the wrong claim reached the docs. The funnel's default, chosen so a stranger can verify the claim by reading one line rather than trusting four components. Not `safe`, whose extra components are lossless in meaning but still rewrite the JSON. Off by default; a gateway or eval-containers deployment must never self-terminate. A signal and the watchdog converge on the SAME teardown, so the self-killing path cannot drift from the one known to work. Two properties are load-bearing: - **The keep-alive inverts "idle."** Pinging is what the proxy does precisely while no client traffic arrives — the quiet gap after `end_turn`, where 83.7% of the recoverable dollars sit. A pending ping both vetoes the exit and RESETS the clock, so retiring the last ping buys a full fresh threshold rather than exiting moments later. - **Exit wipes the in-memory store.** `store.ValidateIdleExit` refuses anything below `max(2 x store.ttl_seconds, 1h)` at startup — ~5h34m at the default. 2x because the TTL is a sliding window. `NewMemory` now calls the same `Options.EffectiveTTL` the floor is computed from, so the two cannot drift. All five items from the proposal, under the `cache` preset. Four were already correct and are now pinned by tests; `POST /anthropic/v1/messages/count_tokens` was missing entirely — without it a client counts context by issuing INFERENCE requests, billed calls added by a proxy sold on removing them. Five places promised it did not: `config/config.go`, `docs/reference/presets.md`, `docs/how-to/choose-a-preset.md`, the plugin doc, and the install skill. Verified before fixing — `[Read Bash]` in, `[Read Bash context_guru_expand]` out on the real gateway route. Root cause was a code-vs-comment contradiction. `Options.InjectExpand` documented the gate as requiring "an expandable marker"; `expand/inject.go` says "No marker condition, deliberately" and the real conditions were mode, store-persists and has-tools. Nothing asked whether the pipeline could produce a marker at all. `components.Pipeline.HasOffload()` answers that by type assertion rather than a name list (a name list is a second copy of "which components are lossy" and drifts the moment somebody adds one). Under `auto`, injection now requires it. `always` still injects unconditionally — an operator who asks for it by name gets it. This also fixes `off`, the A/B control arm, which was carrying an extra tool declaration. Marker independence is preserved, which is the invariant that matters for cache stability: a pipeline does not change turn to turn, so the tools array stays byte-stable across a session. **Ten existing expand tests changed fixture, and that is worth reading.** They hand-seed the Store to simulate an offload, but built their handler with `pipeline: []` — a pipeline that cannot offload anything. That was harmless only while injection ignored the pipeline. They now use `offloadCapablePipeline` (`[linecap]`, which does not act on their short bodies), so each fixture matches its own premise. No assertion was weakened. - **`--idle-exit` was defeated by any health probe** (finding 7). `/healthz` and `/metrics` no longer count as activity: a probe on a schedule shorter than the threshold meant the exit NEVER fired and logged nothing to say so — measured, a 1h-threshold proxy reporting "idle for 1h3m0s" after 2h03m. A dashboard poll still counts, deliberately: a probe is not a viewer, and exiting under somebody who is watching is the worse failure. - **A gateway may no longer self-terminate.** `--idle-exit` with `--upstreams` is refused at startup. That safety was previously accidental — it held only because hosted deployments run a liveness probe, which the change above stops counting. - **The floor's refusal was logged after "listening"**, so a rejected configuration read as a crash. Both refusals moved earlier and into one testable `checkIdleExit`. - **`--listen` and `--version` flags** (findings 2 and 12, which are the plugin's, but the flags are the core's). The address reached the process only through the environment, so no supervisor or `ps` could tell which port an instance held; and `buildinfo.Version` was reachable only via `/stats` on a running proxy, so an installer asking `--help` recorded "Usage of context-guru-proxy:" as the installed version. - **Nothing tested the shipped configuration** (finding 9). A tag published without running any tests, and CI runs the suite only with `CGO_ENABLED=1` — so `TestEveryPresetBuilds`, which guards exactly the CGO-free artifact, was never executed in that configuration. The release workflow now runs a CGO-off suite over the packages whose behaviour depends on which components are compiled in, plus the full suite, before publishing. It also asserts `--version` answers. - **`scripts/gate-a-purego.sh` was cited as proof in four places and is not in this PR** (finding 8). Those now cite the release workflow's own assert step, which exists here and fails the release if a cgo dependency escapes the `cg_skeleton` tag. - **The savings claim was measured in the wrong regime.** −34.1% / 96.7% comes from a harness running tasks back-to-back inside the provider's 5-minute TTL, and is one task measured three times; this project's own interactive figure is $0.0298 across 1,127 sessions, with 1,105 of 1,127 session starts reading zero from cache. Both are now stated, with the zero cases (outside a git repo, under the 1,024-token floor, non-Anthropic backend). The old citation pointed at `docs/results/context-guru.md`, which contains neither number. - **`count_tokens` behaviour kept, consequence documented** (finding 10). It answers about the ORIGINAL body — over-reporting is recoverable, under-reporting costs a failed turn when a fail-open component reverts. What was undocumented is the cost: the client self-compacts earlier than needed (115,933 reported vs 32,802 forwarded). Now in `docs/reference/routes.md`, where the route was absent entirely. Its **hosted branch was wholly untested** — that branch is all that stands between the multi-tenant service and an unmetered open forwarder — so it now has one. - `--idle-exit` and `--version` added to both flag tables; `bytes.NewReader` in counttokens. Six mutations, each proven to have landed in the source before its result was allowed to count: expand injection ungated (the defect) -> TestCachePresetAdvertisesNoExtraTool FAIL cache: sent [Read Bash], forwarded [Read Bash context_guru_expand] off: sent [Read Bash], forwarded [Read Bash context_guru_expand] HasOffload always true -> same test FAIL, same two subcases HasOffload always false -> FAIL on the offloader subcase: "mints markers but no longer advertises the expand tool, so a model cannot recover what it offloaded" probes count as activity again -> TestProbesDoNotDeferIdleExit FAIL ("two hours of nothing but liveness probes: idle past the threshold, but watchIdle never exited") gateway guard disabled -> TestCheckIdleExitRefusesAGatewaySelfTerminating FAIL count_tokens hosted auth removed -> TestCountTokensHostedRequiresAuth FAIL (502, want 401) The third mutation is the one worth noting: it proves the fix did not trade a silent defect for its mirror image, an offloader whose output nothing can expand. `go build ./...`, `go vet ./...`, `gofmt -l` and the full `go test ./...` are clean. Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com>
amiddavid
added a commit
that referenced
this pull request
Sep 1, 2026
…way conformance Implements the local-distribution proposal (#130) minus the Claude Code plugin, which ships as its own PR because all six blocking findings from the review of #141 live in it. Nothing here is held behind that. `docs/get-started/quickstart-proxy.md`, `docs/setup.md` and `docs/hosted.md` all told evaluators to install a C toolchain and set `CGO_ENABLED=1`. That is true only for a `cg_skeleton` build. setup.md went further and named bifrost's tokenizer as a cgo dependency, which it is not — o200k_base is embedded (`internal/tokens/tokens.go`). Verified directly on go 1.26.4 rather than taken from the proposal: `CGO_ENABLED=0` with default tags builds all four release targets (linux/darwin x amd64/arm64, 27.1-33.5 MB stripped), `file` reports "statically linked", `ldd` reports "not a dynamic executable", the binary starts and answers /healthz, and `-tags cg_skeleton` fails under CGO_ENABLED=0 with the build-constraints signature — confirming tree-sitter is the only C dependency. - `.goreleaser.yaml`: a plain GOOS/GOARCH matrix, no cross-toolchains, no `brews:` block (the tap repo and release signing are an unowned question, and nothing may depend on a repo that does not exist). - `.github/workflows/release.yaml`: a tag publishes, `workflow_dispatch` builds the same matrix as a snapshot. It asserts the pure-Go claim with `CC=/nonexistent-c-compiler`. - `make build-static`. The Makefile keeps `CGO_ENABLED=1` because `go test -race` needs it, and the comment now says that is a test-time requirement — reading it as a shipping requirement is how the wrong claim reached the docs. The funnel's default, chosen so a stranger can verify the claim by reading one line rather than trusting four components. Not `safe`, whose extra components are lossless in meaning but still rewrite the JSON. Off by default; a gateway or eval-containers deployment must never self-terminate. A signal and the watchdog converge on the SAME teardown, so the self-killing path cannot drift from the one known to work. Two properties are load-bearing: - **The keep-alive inverts "idle."** Pinging is what the proxy does precisely while no client traffic arrives — the quiet gap after `end_turn`, where 83.7% of the recoverable dollars sit. A pending ping both vetoes the exit and RESETS the clock, so retiring the last ping buys a full fresh threshold rather than exiting moments later. - **Exit wipes the in-memory store.** `store.ValidateIdleExit` refuses anything below `max(2 x store.ttl_seconds, 1h)` at startup — ~5h34m at the default. 2x because the TTL is a sliding window. `NewMemory` now calls the same `Options.EffectiveTTL` the floor is computed from, so the two cannot drift. All five items from the proposal, under the `cache` preset. Four were already correct and are now pinned by tests; `POST /anthropic/v1/messages/count_tokens` was missing entirely — without it a client counts context by issuing INFERENCE requests, billed calls added by a proxy sold on removing them. Five places promised it did not: `config/config.go`, `docs/reference/presets.md`, `docs/how-to/choose-a-preset.md`, the plugin doc, and the install skill. Verified before fixing — `[Read Bash]` in, `[Read Bash context_guru_expand]` out on the real gateway route. Root cause was a code-vs-comment contradiction. `Options.InjectExpand` documented the gate as requiring "an expandable marker"; `expand/inject.go` says "No marker condition, deliberately" and the real conditions were mode, store-persists and has-tools. Nothing asked whether the pipeline could produce a marker at all. `components.Pipeline.HasOffload()` answers that by type assertion rather than a name list (a name list is a second copy of "which components are lossy" and drifts the moment somebody adds one). Under `auto`, injection now requires it. `always` still injects unconditionally — an operator who asks for it by name gets it. This also fixes `off`, the A/B control arm, which was carrying an extra tool declaration. Marker independence is preserved, which is the invariant that matters for cache stability: a pipeline does not change turn to turn, so the tools array stays byte-stable across a session. **Ten existing expand tests changed fixture, and that is worth reading.** They hand-seed the Store to simulate an offload, but built their handler with `pipeline: []` — a pipeline that cannot offload anything. That was harmless only while injection ignored the pipeline. They now use `offloadCapablePipeline` (`[linecap]`, which does not act on their short bodies), so each fixture matches its own premise. No assertion was weakened. - **`--idle-exit` was defeated by any health probe** (finding 7). `/healthz` and `/metrics` no longer count as activity: a probe on a schedule shorter than the threshold meant the exit NEVER fired and logged nothing to say so — measured, a 1h-threshold proxy reporting "idle for 1h3m0s" after 2h03m. A dashboard poll still counts, deliberately: a probe is not a viewer, and exiting under somebody who is watching is the worse failure. - **A gateway may no longer self-terminate.** `--idle-exit` with `--upstreams` is refused at startup. That safety was previously accidental — it held only because hosted deployments run a liveness probe, which the change above stops counting. - **The floor's refusal was logged after "listening"**, so a rejected configuration read as a crash. Both refusals moved earlier and into one testable `checkIdleExit`. - **`--listen` and `--version` flags** (findings 2 and 12, which are the plugin's, but the flags are the core's). The address reached the process only through the environment, so no supervisor or `ps` could tell which port an instance held; and `buildinfo.Version` was reachable only via `/stats` on a running proxy, so an installer asking `--help` recorded "Usage of context-guru-proxy:" as the installed version. - **Nothing tested the shipped configuration** (finding 9). A tag published without running any tests, and CI runs the suite only with `CGO_ENABLED=1` — so `TestEveryPresetBuilds`, which guards exactly the CGO-free artifact, was never executed in that configuration. The release workflow now runs a CGO-off suite over the packages whose behaviour depends on which components are compiled in, plus the full suite, before publishing. It also asserts `--version` answers. - **`scripts/gate-a-purego.sh` was cited as proof in four places and is not in this PR** (finding 8). Those now cite the release workflow's own assert step, which exists here and fails the release if a cgo dependency escapes the `cg_skeleton` tag. - **The savings claim was measured in the wrong regime.** −34.1% / 96.7% comes from a harness running tasks back-to-back inside the provider's 5-minute TTL, and is one task measured three times; this project's own interactive figure is $0.0298 across 1,127 sessions, with 1,105 of 1,127 session starts reading zero from cache. Both are now stated, with the zero cases (outside a git repo, under the 1,024-token floor, non-Anthropic backend). The old citation pointed at `docs/results/context-guru.md`, which contains neither number. - **`count_tokens` behaviour kept, consequence documented** (finding 10). It answers about the ORIGINAL body — over-reporting is recoverable, under-reporting costs a failed turn when a fail-open component reverts. What was undocumented is the cost: the client self-compacts earlier than needed (115,933 reported vs 32,802 forwarded). Now in `docs/reference/routes.md`, where the route was absent entirely. Its **hosted branch was wholly untested** — that branch is all that stands between the multi-tenant service and an unmetered open forwarder — so it now has one. - `--idle-exit` and `--version` added to both flag tables; `bytes.NewReader` in counttokens. Six mutations, each proven to have landed in the source before its result was allowed to count: expand injection ungated (the defect) -> TestCachePresetAdvertisesNoExtraTool FAIL cache: sent [Read Bash], forwarded [Read Bash context_guru_expand] off: sent [Read Bash], forwarded [Read Bash context_guru_expand] HasOffload always true -> same test FAIL, same two subcases HasOffload always false -> FAIL on the offloader subcase: "mints markers but no longer advertises the expand tool, so a model cannot recover what it offloaded" probes count as activity again -> TestProbesDoNotDeferIdleExit FAIL ("two hours of nothing but liveness probes: idle past the threshold, but watchIdle never exited") gateway guard disabled -> TestCheckIdleExitRefusesAGatewaySelfTerminating FAIL count_tokens hosted auth removed -> TestCountTokensHostedRequiresAuth FAIL (502, want 401) The third mutation is the one worth noting: it proves the fix did not trade a silent defect for its mirror image, an offloader whose output nothing can expand. `go build ./...`, `go vet ./...`, `gofmt -l` and the full `go test ./...` are clean. Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com>
amiddavid
added a commit
that referenced
this pull request
Sep 1, 2026
…way conformance Implements the local-distribution proposal (#130) minus the Claude Code plugin, which ships as its own PR because all six blocking findings from the review of #141 live in it. Nothing here is held behind that. `docs/get-started/quickstart-proxy.md`, `docs/setup.md` and `docs/hosted.md` all told evaluators to install a C toolchain and set `CGO_ENABLED=1`. That is true only for a `cg_skeleton` build. setup.md went further and named bifrost's tokenizer as a cgo dependency, which it is not — o200k_base is embedded (`internal/tokens/tokens.go`). Verified directly on go 1.26.4 rather than taken from the proposal: `CGO_ENABLED=0` with default tags builds all four release targets (linux/darwin x amd64/arm64, 27.1-33.5 MB stripped), `file` reports "statically linked", `ldd` reports "not a dynamic executable", the binary starts and answers /healthz, and `-tags cg_skeleton` fails under CGO_ENABLED=0 with the build-constraints signature — confirming tree-sitter is the only C dependency. - `.goreleaser.yaml`: a plain GOOS/GOARCH matrix, no cross-toolchains, no `brews:` block (the tap repo and release signing are an unowned question, and nothing may depend on a repo that does not exist). - `.github/workflows/release.yaml`: a tag publishes, `workflow_dispatch` builds the same matrix as a snapshot. It asserts the pure-Go claim with `CC=/nonexistent-c-compiler`. - `make build-static`. The Makefile keeps `CGO_ENABLED=1` because `go test -race` needs it, and the comment now says that is a test-time requirement — reading it as a shipping requirement is how the wrong claim reached the docs. The funnel's default, chosen so a stranger can verify the claim by reading one line rather than trusting four components. Not `safe`, whose extra components are lossless in meaning but still rewrite the JSON. Off by default; a gateway or eval-containers deployment must never self-terminate. A signal and the watchdog converge on the SAME teardown, so the self-killing path cannot drift from the one known to work. Two properties are load-bearing: - **The keep-alive inverts "idle."** Pinging is what the proxy does precisely while no client traffic arrives — the quiet gap after `end_turn`, where 83.7% of the recoverable dollars sit. A pending ping both vetoes the exit and RESETS the clock, so retiring the last ping buys a full fresh threshold rather than exiting moments later. - **Exit wipes the in-memory store.** `store.ValidateIdleExit` refuses anything below `max(2 x store.ttl_seconds, 1h)` at startup — ~5h34m at the default. 2x because the TTL is a sliding window. `NewMemory` now calls the same `Options.EffectiveTTL` the floor is computed from, so the two cannot drift. All five items from the proposal, under the `cache` preset. Four were already correct and are now pinned by tests; `POST /anthropic/v1/messages/count_tokens` was missing entirely — without it a client counts context by issuing INFERENCE requests, billed calls added by a proxy sold on removing them. Five places promised it did not: `config/config.go`, `docs/reference/presets.md`, `docs/how-to/choose-a-preset.md`, the plugin doc, and the install skill. Verified before fixing — `[Read Bash]` in, `[Read Bash context_guru_expand]` out on the real gateway route. Root cause was a code-vs-comment contradiction. `Options.InjectExpand` documented the gate as requiring "an expandable marker"; `expand/inject.go` says "No marker condition, deliberately" and the real conditions were mode, store-persists and has-tools. Nothing asked whether the pipeline could produce a marker at all. `components.Pipeline.HasOffload()` answers that by type assertion rather than a name list (a name list is a second copy of "which components are lossy" and drifts the moment somebody adds one). Under `auto`, injection now requires it. `always` still injects unconditionally — an operator who asks for it by name gets it. This also fixes `off`, the A/B control arm, which was carrying an extra tool declaration. Marker independence is preserved, which is the invariant that matters for cache stability: a pipeline does not change turn to turn, so the tools array stays byte-stable across a session. **Ten existing expand tests changed fixture, and that is worth reading.** They hand-seed the Store to simulate an offload, but built their handler with `pipeline: []` — a pipeline that cannot offload anything. That was harmless only while injection ignored the pipeline. They now use `offloadCapablePipeline` (`[linecap]`, which does not act on their short bodies), so each fixture matches its own premise. No assertion was weakened. - **`--idle-exit` was defeated by any health probe** (finding 7). `/healthz` and `/metrics` no longer count as activity: a probe on a schedule shorter than the threshold meant the exit NEVER fired and logged nothing to say so — measured, a 1h-threshold proxy reporting "idle for 1h3m0s" after 2h03m. A dashboard poll still counts, deliberately: a probe is not a viewer, and exiting under somebody who is watching is the worse failure. - **A gateway may no longer self-terminate.** `--idle-exit` with `--upstreams` is refused at startup. That safety was previously accidental — it held only because hosted deployments run a liveness probe, which the change above stops counting. - **The floor's refusal was logged after "listening"**, so a rejected configuration read as a crash. Both refusals moved earlier and into one testable `checkIdleExit`. - **`--listen` and `--version` flags** (findings 2 and 12, which are the plugin's, but the flags are the core's). The address reached the process only through the environment, so no supervisor or `ps` could tell which port an instance held; and `buildinfo.Version` was reachable only via `/stats` on a running proxy, so an installer asking `--help` recorded "Usage of context-guru-proxy:" as the installed version. - **Nothing tested the shipped configuration** (finding 9). A tag published without running any tests, and CI runs the suite only with `CGO_ENABLED=1` — so `TestEveryPresetBuilds`, which guards exactly the CGO-free artifact, was never executed in that configuration. The release workflow now runs a CGO-off suite over the packages whose behaviour depends on which components are compiled in, plus the full suite, before publishing. It also asserts `--version` answers. - **`scripts/gate-a-purego.sh` was cited as proof in four places and is not in this PR** (finding 8). Those now cite the release workflow's own assert step, which exists here and fails the release if a cgo dependency escapes the `cg_skeleton` tag. - **The savings claim was measured in the wrong regime.** −34.1% / 96.7% comes from a harness running tasks back-to-back inside the provider's 5-minute TTL, and is one task measured three times; this project's own interactive figure is $0.0298 across 1,127 sessions, with 1,105 of 1,127 session starts reading zero from cache. Both are now stated, with the zero cases (outside a git repo, under the 1,024-token floor, non-Anthropic backend). The old citation pointed at `docs/results/context-guru.md`, which contains neither number. - **`count_tokens` behaviour kept, consequence documented** (finding 10). It answers about the ORIGINAL body — over-reporting is recoverable, under-reporting costs a failed turn when a fail-open component reverts. What was undocumented is the cost: the client self-compacts earlier than needed (115,933 reported vs 32,802 forwarded). Now in `docs/reference/routes.md`, where the route was absent entirely. Its **hosted branch was wholly untested** — that branch is all that stands between the multi-tenant service and an unmetered open forwarder — so it now has one. - `--idle-exit` and `--version` added to both flag tables; `bytes.NewReader` in counttokens. Six mutations, each proven to have landed in the source before its result was allowed to count: expand injection ungated (the defect) -> TestCachePresetAdvertisesNoExtraTool FAIL cache: sent [Read Bash], forwarded [Read Bash context_guru_expand] off: sent [Read Bash], forwarded [Read Bash context_guru_expand] HasOffload always true -> same test FAIL, same two subcases HasOffload always false -> FAIL on the offloader subcase: "mints markers but no longer advertises the expand tool, so a model cannot recover what it offloaded" probes count as activity again -> TestProbesDoNotDeferIdleExit FAIL ("two hours of nothing but liveness probes: idle past the threshold, but watchIdle never exited") gateway guard disabled -> TestCheckIdleExitRefusesAGatewaySelfTerminating FAIL count_tokens hosted auth removed -> TestCountTokensHostedRequiresAuth FAIL (502, want 401) The third mutation is the one worth noting: it proves the fix did not trade a silent defect for its mirror image, an offloader whose output nothing can expand. `go build ./...`, `go vet ./...`, `gofmt -l` and the full `go test ./...` are clean. Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com>
amiddavid
added a commit
that referenced
this pull request
Sep 1, 2026
…way conformance Implements the local-distribution proposal (#130) minus the Claude Code plugin, which ships as its own PR because all six blocking findings from the review of #141 live in it. Nothing here is held behind that. `docs/get-started/quickstart-proxy.md`, `docs/setup.md` and `docs/hosted.md` all told evaluators to install a C toolchain and set `CGO_ENABLED=1`. That is true only for a `cg_skeleton` build. setup.md went further and named bifrost's tokenizer as a cgo dependency, which it is not — o200k_base is embedded (`internal/tokens/tokens.go`). Verified directly on go 1.26.4 rather than taken from the proposal: `CGO_ENABLED=0` with default tags builds all four release targets (linux/darwin x amd64/arm64, 27.1-33.5 MB stripped), `file` reports "statically linked", `ldd` reports "not a dynamic executable", the binary starts and answers /healthz, and `-tags cg_skeleton` fails under CGO_ENABLED=0 with the build-constraints signature — confirming tree-sitter is the only C dependency. - `.goreleaser.yaml`: a plain GOOS/GOARCH matrix, no cross-toolchains, no `brews:` block (the tap repo and release signing are an unowned question, and nothing may depend on a repo that does not exist). - `.github/workflows/release.yaml`: a tag publishes, `workflow_dispatch` builds the same matrix as a snapshot. It asserts the pure-Go claim with `CC=/nonexistent-c-compiler`. - `make build-static`. The Makefile keeps `CGO_ENABLED=1` because `go test -race` needs it, and the comment now says that is a test-time requirement — reading it as a shipping requirement is how the wrong claim reached the docs. The funnel's default, chosen so a stranger can verify the claim by reading one line rather than trusting four components. Not `safe`, whose extra components are lossless in meaning but still rewrite the JSON. Off by default; a gateway or eval-containers deployment must never self-terminate. A signal and the watchdog converge on the SAME teardown, so the self-killing path cannot drift from the one known to work. Two properties are load-bearing: - **The keep-alive inverts "idle."** Pinging is what the proxy does precisely while no client traffic arrives — the quiet gap after `end_turn`, where 83.7% of the recoverable dollars sit. A pending ping both vetoes the exit and RESETS the clock, so retiring the last ping buys a full fresh threshold rather than exiting moments later. - **Exit wipes the in-memory store.** `store.ValidateIdleExit` refuses anything below `max(2 x store.ttl_seconds, 1h)` at startup — ~5h34m at the default. 2x because the TTL is a sliding window. `NewMemory` now calls the same `Options.EffectiveTTL` the floor is computed from, so the two cannot drift. All five items from the proposal, under the `cache` preset. Four were already correct and are now pinned by tests; `POST /anthropic/v1/messages/count_tokens` was missing entirely — without it a client counts context by issuing INFERENCE requests, billed calls added by a proxy sold on removing them. Five places promised it did not: `config/config.go`, `docs/reference/presets.md`, `docs/how-to/choose-a-preset.md`, the plugin doc, and the install skill. Verified before fixing — `[Read Bash]` in, `[Read Bash context_guru_expand]` out on the real gateway route. Root cause was a code-vs-comment contradiction. `Options.InjectExpand` documented the gate as requiring "an expandable marker"; `expand/inject.go` says "No marker condition, deliberately" and the real conditions were mode, store-persists and has-tools. Nothing asked whether the pipeline could produce a marker at all. `components.Pipeline.HasOffload()` answers that by type assertion rather than a name list (a name list is a second copy of "which components are lossy" and drifts the moment somebody adds one). Under `auto`, injection now requires it. `always` still injects unconditionally — an operator who asks for it by name gets it. This also fixes `off`, the A/B control arm, which was carrying an extra tool declaration. Marker independence is preserved, which is the invariant that matters for cache stability: a pipeline does not change turn to turn, so the tools array stays byte-stable across a session. **Ten existing expand tests changed fixture, and that is worth reading.** They hand-seed the Store to simulate an offload, but built their handler with `pipeline: []` — a pipeline that cannot offload anything. That was harmless only while injection ignored the pipeline. They now use `offloadCapablePipeline` (`[linecap]`, which does not act on their short bodies), so each fixture matches its own premise. No assertion was weakened. - **`--idle-exit` was defeated by any health probe** (finding 7). `/healthz` and `/metrics` no longer count as activity: a probe on a schedule shorter than the threshold meant the exit NEVER fired and logged nothing to say so — measured, a 1h-threshold proxy reporting "idle for 1h3m0s" after 2h03m. A dashboard poll still counts, deliberately: a probe is not a viewer, and exiting under somebody who is watching is the worse failure. - **A gateway may no longer self-terminate.** `--idle-exit` with `--upstreams` is refused at startup. That safety was previously accidental — it held only because hosted deployments run a liveness probe, which the change above stops counting. - **The floor's refusal was logged after "listening"**, so a rejected configuration read as a crash. Both refusals moved earlier and into one testable `checkIdleExit`. - **`--listen` and `--version` flags** (findings 2 and 12, which are the plugin's, but the flags are the core's). The address reached the process only through the environment, so no supervisor or `ps` could tell which port an instance held; and `buildinfo.Version` was reachable only via `/stats` on a running proxy, so an installer asking `--help` recorded "Usage of context-guru-proxy:" as the installed version. - **Nothing tested the shipped configuration** (finding 9). A tag published without running any tests, and CI runs the suite only with `CGO_ENABLED=1` — so `TestEveryPresetBuilds`, which guards exactly the CGO-free artifact, was never executed in that configuration. The release workflow now runs a CGO-off suite over the packages whose behaviour depends on which components are compiled in, plus the full suite, before publishing. It also asserts `--version` answers. - **`scripts/gate-a-purego.sh` was cited as proof in four places and is not in this PR** (finding 8). Those now cite the release workflow's own assert step, which exists here and fails the release if a cgo dependency escapes the `cg_skeleton` tag. - **The savings claim was measured in the wrong regime.** −34.1% / 96.7% comes from a harness running tasks back-to-back inside the provider's 5-minute TTL, and is one task measured three times; this project's own interactive figure is $0.0298 across 1,127 sessions, with 1,105 of 1,127 session starts reading zero from cache. Both are now stated, with the zero cases (outside a git repo, under the 1,024-token floor, non-Anthropic backend). The old citation pointed at `docs/results/context-guru.md`, which contains neither number. - **`count_tokens` behaviour kept, consequence documented** (finding 10). It answers about the ORIGINAL body — over-reporting is recoverable, under-reporting costs a failed turn when a fail-open component reverts. What was undocumented is the cost: the client self-compacts earlier than needed (115,933 reported vs 32,802 forwarded). Now in `docs/reference/routes.md`, where the route was absent entirely. Its **hosted branch was wholly untested** — that branch is all that stands between the multi-tenant service and an unmetered open forwarder — so it now has one. - `--idle-exit` and `--version` added to both flag tables; `bytes.NewReader` in counttokens. Six mutations, each proven to have landed in the source before its result was allowed to count: expand injection ungated (the defect) -> TestCachePresetAdvertisesNoExtraTool FAIL cache: sent [Read Bash], forwarded [Read Bash context_guru_expand] off: sent [Read Bash], forwarded [Read Bash context_guru_expand] HasOffload always true -> same test FAIL, same two subcases HasOffload always false -> FAIL on the offloader subcase: "mints markers but no longer advertises the expand tool, so a model cannot recover what it offloaded" probes count as activity again -> TestProbesDoNotDeferIdleExit FAIL ("two hours of nothing but liveness probes: idle past the threshold, but watchIdle never exited") gateway guard disabled -> TestCheckIdleExitRefusesAGatewaySelfTerminating FAIL count_tokens hosted auth removed -> TestCountTokensHostedRequiresAuth FAIL (502, want 401) The third mutation is the one worth noting: it proves the fix did not trade a silent defect for its mirror image, an offloader whose output nothing can expand. `go build ./...`, `go vet ./...`, `gofmt -l` and the full `go test ./...` are clean. Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this is
A proposal for review, docs only — no code, no behavior change. It asks for approval on how we distribute context-guru to someone who finds the repo and wants to try it on their own Claude Code sessions.
Spec:
docs/superpowers/specs/2026-08-30-local-distribution-design.mdThe problem
An evaluator today has to install Go 1.26, install a C toolchain, build with the right tags, run a binary, work out
ANTHROPIC_BASE_URL, and trust that routing their coding agent through an unknown local proxy won't break it. Most stop at step one.Step one turns out to be avoidable: the C toolchain isn't required.
CGO_ENABLED=0produces a fully static binary for all four release targets —go-tree-sitteris the only cgo dependency and it's already behind//go:build cg_skeleton. Our quickstart tells every evaluator otherwise.The value we want them to feel first is KV-cache, not the offloaders. Cache work requires being on the wire, so this is about making the proxy trivial to install — not about avoiding it.
Target:
What the user does
Nothing is copied into their project, and the skill isn't something they have to place.
/plugin installwrites toenabledPluginsin~/.claude/settings.json(user scope is the default = "available across all projects"), so the plugin and its skills are installed once per machine and reachable from any repo.Step 3 is the whole reason a skill exists rather than a shell script. It detects the platform and installs the binary (one Bash approval), asks which scope to route, reads the target settings file and backs it up before adding one key —
env.ANTHROPIC_BASE_URL(one file-edit approval), notices if the user already has a base URL set and asks rather than overwriting, then starts the proxy, verifies/healthzand prints the dashboard URL.Three to four approval prompts, of which steps 1–2 happen once, ever. Afterwards
/context-guru:statusreads/statsand/context-guru:uninstallremoves the key and stops the proxy.brew/~/.local/binenvblock)--globalopt-inOnly the routing decision is per-repo, and deliberately so — it's the one with blast radius.
What's proposed
--idle-exitself-terminating proxycachepreset (cachesplitalone)scripts/+SessionStarthookThree things worth your attention
Our quickstart is wrong, and fixing it is the cheapest win here.
docs/get-started/quickstart-proxy.mdinstructs every evaluator to setCGO_ENABLED=1and install a C toolchain. That's only true for acg_skeletonbuild. One-line change, removes the largest onboarding gate in the project, and depends on nothing else in this proposal — worth landing even if the rest is rejected.A rejected
cache_controlsilently switches off prompt caching. Per the gateway protocol reference, when the upstream rejects acache_controlmarker Claude Code retries and disables the capability for the rest of the conversation. So a breakpoint-budget mistake isn't an error the user sees — it turns off the thing we're selling, and the demo reads as negative. This is the strongest argument for shippingcache(cachesplitonly, nocacheinject) as the funnel default.A subscription needs no API key. Setting
ANTHROPIC_BASE_URLwithout a credential variable keeps the user's claude.ai login active — their Pro/Max limits and billing continue to apply. An evaluator can trial context-guru with no API key at all. That's the single most important adoption fact about this project and it appears nowhere in our docs.Design decisions I'd like challenged
Default scope is project-local, not global. Precedence is managed →
--settings→.claude/settings.local.json→.claude/settings.json→~/.claude/settings.json. A global base URL pointing atlocalhostmeans a dead proxy breaks Claude Code everywhere, including repos the evaluator never meant to experiment in. That blast radius is the biggest risk in this proposal — bigger than the build. Local is safer, global is the better demo; I picked safer and am happy to be overruled.SessionStarthook rather than launchd/systemd. Session-scoped lifetime leaves nothing on the machine and needs no privileged install. The spec documents the five details that make it correct — neverasync: true(reintroduces the race with the first request), idempotent because the event also fires onclear/compact/resume/fork, fixed port because the env block and the hook must agree, the clone-time trust prompt, and:The hook self-gates on
ANTHROPIC_BASE_URL. Because the plugin installs at user scope, its hooks run in every project — including ones the user never routed, where starting a proxy is pure waste. Settingsenvvalues are written into the process environment and hooks inherit it, so the hook exits immediately unless the variable names our port. No extra configuration, no second copy of the port to drift, and it degrades correctly: remove the env key by hand and the hook stops firing on its own.--idle-exithas a floor atstore.ttl_seconds. Exit wipes the in-memory Store, and a frozen decision dying mid-session is the 11.5× cache-write bugFrozenLostexists to catch. 24h is safely past the 10000s default; a 30-minute threshold would be actively destructive. Proposed as config validation, not a doc comment. Also: the keepalive inverts what "idle" means — pinging is what it does while there's no client traffic — so the watchdog must know about pending ping schedules or it kills the feature it ships.Skill +
scripts/, not a script alone. A skill doesn't reduce permission prompts, it concentrates them. The real reason for it is the part a script does badly: merging one key into a~/.claude/settings.jsonthat already holds the user's theme, model, permission rules and possibly their own base URL. That's judgment work — read, back up, add one key, detect a conflicting base URL, verify, report. Our own settings files are the proof.Non-goals (stated explicitly in the spec)
Open questions
All judgment calls — nothing here is an unknown that could invalidate the plan.
max(2 × store.ttl_seconds, 1h)?rossoctl/homebrew-taprepo — who owns release signing?modernc.org/sqlite. A-slimbuild without the dashboard is possible, but it costs the demo its best surface.skeletonin v1? It's the only thing needing cgo, so a-skeletonvariant means per-platform CI for one component that isn't incodesmartand isn't in the cache story. Proposal: omit from the first release, document the source build.Staging
cachepreset + fix the CGO claim in the quickstart. Shippable alone, and most of the adoption win.--idle-exit+ the plugin.Appendix: what was verified
Two assumptions the plan rests on were checked rather than trusted. Both came back the way the proposal already assumed, so neither changes a decision — they're recorded so you don't have to take the claims on faith, and the scripts are committed so you can re-run them. Full detail in the spec's Verification appendix.
The binary needs no C toolchain —
scripts/gate-a-purego.sh, in agolang:1.26container so it needs no local Go:CGO_ENABLED=0-tags cg_skeleton,CGO_ENABLED=0build constraints exclude all Go files in …/go-sitter-forest/typescript, the cgo-disabled signatureGOOS/GOARCH× {linux,darwin} × {amd64,arm64}ldd→ "not a dynamic executable"The first run of the
cg_skeletoncheck failed onproxy.golang.orgtimeouts rather than on cgo, which would have been recorded as a confirmation it wasn't. The script now fetches the grammar modules first and reportsINCONCLUSIVEif it can't.envblocks merge per key —scripts/gate-b-envmerge.sh. Relocates the config dir viaCLAUDE_CONFIG_DIRso it never touches a real~/.claude, and reads the result from aSessionStarthook rather than a model self-report. WithCG_USER+CG_BOTHin user scope andCG_BOTH+CG_PROJin project scope, the hook sawCG_BOTH=project,CG_PROJ=project,CG_USER=user— the higher-precedence file won only the key it sets, and the user-scope object survived.Still unverified: the five gateway conformance items in piece 5.
Related: #129 (Claude Code plugin as a transport for the offloaders — different question, same plugin surface).
Assisted-By: Claude Opus 5 (1M context) noreply@anthropic.com