From 7281324aeaf035770d307f853ce4437d0e3cebb1 Mon Sep 17 00:00:00 2001 From: DAVID AMID Date: Tue, 1 Sep 2026 11:09:24 +0300 Subject: [PATCH 1/3] feat(dist): pure-Go releases, a `cache` preset, --idle-exit, and gateway conformance MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Implements the local-distribution proposal (#130) minus the Claude Code plugin, which ships as its own PR because all six blocking findings from the review of #141 live in it. Nothing here is held behind that. `docs/get-started/quickstart-proxy.md`, `docs/setup.md` and `docs/hosted.md` all told evaluators to install a C toolchain and set `CGO_ENABLED=1`. That is true only for a `cg_skeleton` build. setup.md went further and named bifrost's tokenizer as a cgo dependency, which it is not — o200k_base is embedded (`internal/tokens/tokens.go`). Verified directly on go 1.26.4 rather than taken from the proposal: `CGO_ENABLED=0` with default tags builds all four release targets (linux/darwin x amd64/arm64, 27.1-33.5 MB stripped), `file` reports "statically linked", `ldd` reports "not a dynamic executable", the binary starts and answers /healthz, and `-tags cg_skeleton` fails under CGO_ENABLED=0 with the build-constraints signature — confirming tree-sitter is the only C dependency. - `.goreleaser.yaml`: a plain GOOS/GOARCH matrix, no cross-toolchains, no `brews:` block (the tap repo and release signing are an unowned question, and nothing may depend on a repo that does not exist). - `.github/workflows/release.yaml`: a tag publishes, `workflow_dispatch` builds the same matrix as a snapshot. It asserts the pure-Go claim with `CC=/nonexistent-c-compiler`. - `make build-static`. The Makefile keeps `CGO_ENABLED=1` because `go test -race` needs it, and the comment now says that is a test-time requirement — reading it as a shipping requirement is how the wrong claim reached the docs. The funnel's default, chosen so a stranger can verify the claim by reading one line rather than trusting four components. Not `safe`, whose extra components are lossless in meaning but still rewrite the JSON. Off by default; a gateway or eval-containers deployment must never self-terminate. A signal and the watchdog converge on the SAME teardown, so the self-killing path cannot drift from the one known to work. Two properties are load-bearing: - **The keep-alive inverts "idle."** Pinging is what the proxy does precisely while no client traffic arrives — the quiet gap after `end_turn`, where 83.7% of the recoverable dollars sit. A pending ping both vetoes the exit and RESETS the clock, so retiring the last ping buys a full fresh threshold rather than exiting moments later. - **Exit wipes the in-memory store.** `store.ValidateIdleExit` refuses anything below `max(2 x store.ttl_seconds, 1h)` at startup — ~5h34m at the default. 2x because the TTL is a sliding window. `NewMemory` now calls the same `Options.EffectiveTTL` the floor is computed from, so the two cannot drift. All five items from the proposal, under the `cache` preset. Four were already correct and are now pinned by tests; `POST /anthropic/v1/messages/count_tokens` was missing entirely — without it a client counts context by issuing INFERENCE requests, billed calls added by a proxy sold on removing them. Five places promised it did not: `config/config.go`, `docs/reference/presets.md`, `docs/how-to/choose-a-preset.md`, the plugin doc, and the install skill. Verified before fixing — `[Read Bash]` in, `[Read Bash context_guru_expand]` out on the real gateway route. Root cause was a code-vs-comment contradiction. `Options.InjectExpand` documented the gate as requiring "an expandable marker"; `expand/inject.go` says "No marker condition, deliberately" and the real conditions were mode, store-persists and has-tools. Nothing asked whether the pipeline could produce a marker at all. `components.Pipeline.HasOffload()` answers that by type assertion rather than a name list (a name list is a second copy of "which components are lossy" and drifts the moment somebody adds one). Under `auto`, injection now requires it. `always` still injects unconditionally — an operator who asks for it by name gets it. This also fixes `off`, the A/B control arm, which was carrying an extra tool declaration. Marker independence is preserved, which is the invariant that matters for cache stability: a pipeline does not change turn to turn, so the tools array stays byte-stable across a session. **Ten existing expand tests changed fixture, and that is worth reading.** They hand-seed the Store to simulate an offload, but built their handler with `pipeline: []` — a pipeline that cannot offload anything. That was harmless only while injection ignored the pipeline. They now use `offloadCapablePipeline` (`[linecap]`, which does not act on their short bodies), so each fixture matches its own premise. No assertion was weakened. - **`--idle-exit` was defeated by any health probe** (finding 7). `/healthz` and `/metrics` no longer count as activity: a probe on a schedule shorter than the threshold meant the exit NEVER fired and logged nothing to say so — measured, a 1h-threshold proxy reporting "idle for 1h3m0s" after 2h03m. A dashboard poll still counts, deliberately: a probe is not a viewer, and exiting under somebody who is watching is the worse failure. - **A gateway may no longer self-terminate.** `--idle-exit` with `--upstreams` is refused at startup. That safety was previously accidental — it held only because hosted deployments run a liveness probe, which the change above stops counting. - **The floor's refusal was logged after "listening"**, so a rejected configuration read as a crash. Both refusals moved earlier and into one testable `checkIdleExit`. - **`--listen` and `--version` flags** (findings 2 and 12, which are the plugin's, but the flags are the core's). The address reached the process only through the environment, so no supervisor or `ps` could tell which port an instance held; and `buildinfo.Version` was reachable only via `/stats` on a running proxy, so an installer asking `--help` recorded "Usage of context-guru-proxy:" as the installed version. - **Nothing tested the shipped configuration** (finding 9). A tag published without running any tests, and CI runs the suite only with `CGO_ENABLED=1` — so `TestEveryPresetBuilds`, which guards exactly the CGO-free artifact, was never executed in that configuration. The release workflow now runs a CGO-off suite over the packages whose behaviour depends on which components are compiled in, plus the full suite, before publishing. It also asserts `--version` answers. - **`scripts/gate-a-purego.sh` was cited as proof in four places and is not in this PR** (finding 8). Those now cite the release workflow's own assert step, which exists here and fails the release if a cgo dependency escapes the `cg_skeleton` tag. - **The savings claim was measured in the wrong regime.** −34.1% / 96.7% comes from a harness running tasks back-to-back inside the provider's 5-minute TTL, and is one task measured three times; this project's own interactive figure is $0.0298 across 1,127 sessions, with 1,105 of 1,127 session starts reading zero from cache. Both are now stated, with the zero cases (outside a git repo, under the 1,024-token floor, non-Anthropic backend). The old citation pointed at `docs/results/context-guru.md`, which contains neither number. - **`count_tokens` behaviour kept, consequence documented** (finding 10). It answers about the ORIGINAL body — over-reporting is recoverable, under-reporting costs a failed turn when a fail-open component reverts. What was undocumented is the cost: the client self-compacts earlier than needed (115,933 reported vs 32,802 forwarded). Now in `docs/reference/routes.md`, where the route was absent entirely. Its **hosted branch was wholly untested** — that branch is all that stands between the multi-tenant service and an unmetered open forwarder — so it now has one. - `--idle-exit` and `--version` added to both flag tables; `bytes.NewReader` in counttokens. Six mutations, each proven to have landed in the source before its result was allowed to count: expand injection ungated (the defect) -> TestCachePresetAdvertisesNoExtraTool FAIL cache: sent [Read Bash], forwarded [Read Bash context_guru_expand] off: sent [Read Bash], forwarded [Read Bash context_guru_expand] HasOffload always true -> same test FAIL, same two subcases HasOffload always false -> FAIL on the offloader subcase: "mints markers but no longer advertises the expand tool, so a model cannot recover what it offloaded" probes count as activity again -> TestProbesDoNotDeferIdleExit FAIL ("two hours of nothing but liveness probes: idle past the threshold, but watchIdle never exited") gateway guard disabled -> TestCheckIdleExitRefusesAGatewaySelfTerminating FAIL count_tokens hosted auth removed -> TestCountTokensHostedRequiresAuth FAIL (502, want 401) The third mutation is the one worth noting: it proves the fix did not trade a silent defect for its mirror image, an offloader whose output nothing can expand. `go build ./...`, `go vet ./...`, `gofmt -l` and the full `go test ./...` are clean. Assisted-By: Claude Opus 5 (1M context) Signed-off-by: DAVID AMID --- .github/workflows/release.yaml | 90 +++++++ .goreleaser.yaml | 98 +++++++ README.md | 5 + cmd/context-guru-proxy/idleexit.go | 172 +++++++++++++ cmd/context-guru-proxy/idleexit_test.go | 329 ++++++++++++++++++++++++ cmd/context-guru-proxy/main.go | 79 +++++- config/config.go | 22 +- config/config_more_test.go | 41 ++- docs/get-started/quickstart-proxy.md | 19 +- docs/how-to/choose-a-preset.md | 35 +++ docs/how-to/use-with-claude-code.md | 19 ++ docs/reference/config.md | 4 +- docs/reference/presets.md | 1 + docs/setup.md | 10 +- proxy/conformance_test.go | 305 ++++++++++++++++++++++ proxy/keepalive.go | 38 +++ store/idleexit_test.go | 82 ++++++ store/store.go | 60 ++++- 18 files changed, 1391 insertions(+), 18 deletions(-) create mode 100644 .github/workflows/release.yaml create mode 100644 .goreleaser.yaml create mode 100644 cmd/context-guru-proxy/idleexit.go create mode 100644 cmd/context-guru-proxy/idleexit_test.go create mode 100644 proxy/conformance_test.go create mode 100644 store/idleexit_test.go diff --git a/.github/workflows/release.yaml b/.github/workflows/release.yaml new file mode 100644 index 00000000..2908ad54 --- /dev/null +++ b/.github/workflows/release.yaml @@ -0,0 +1,90 @@ +name: Release + +# Tag-driven, so a release is something a maintainer does on purpose. The `workflow_dispatch` +# entry builds the same matrix WITHOUT publishing (snapshot mode), which is how the release +# path gets exercised before there is a tag to regret. +on: + push: + tags: ["v*"] + workflow_dispatch: + +permissions: + contents: read + +jobs: + release: + runs-on: ubuntu-latest + permissions: + # Only the tag path publishes, and only this job needs the write. + contents: write + steps: + - uses: actions/checkout@v4 + with: + # GoReleaser's changelog needs the history the default shallow clone does not have. + fetch-depth: 0 + - uses: actions/setup-go@v5 + with: + go-version: '1.26' + check-latest: true + + # The claim the release rests on, asserted in CI rather than trusted: the shipped + # binary needs no C toolchain. CGO_ENABLED=0 with no compiler on PATH would fail loudly + # here if a cgo dependency ever escaped the cg_skeleton build tag — which is exactly the + # regression that would otherwise be discovered by an evaluator, at install time. + - name: Assert the binary is pure Go + env: + CGO_ENABLED: "0" + CC: /nonexistent-c-compiler + run: | + go build -o /tmp/cg-purego ./cmd/context-guru-proxy + file /tmp/cg-purego | tee /dev/stderr | grep -q "statically linked" + # And it has to actually start, not just link. + /tmp/cg-purego --listen 127.0.0.1:4471 --preset cache & + for _ in $(seq 1 40); do + sleep 0.25 + curl -fsS http://127.0.0.1:4471/healthz && break + done + curl -fsS http://127.0.0.1:4471/healthz | grep -q ok + # An installer asks the binary what it is; make sure it can answer. + /tmp/cg-purego --version | tee /dev/stderr | grep -q context-guru-proxy + + # Nothing tested the configuration we actually SHIP. + # + # ci.yaml runs the suite only with CGO_ENABLED=1, and a tag push previously published + # without running any tests at all. So the one guard that matters most to a released + # artifact — TestEveryPresetBuilds, which catches a preset naming a component that is not + # registered in a CGO-free binary — was never executed in the CGO-free configuration. That + # is exactly the `preset: coding` / `unknown component "skeleton"` failure, in a build no + # developer runs locally. + # + # The race detector needs cgo, so this cannot be the whole suite; it is the packages whose + # behaviour depends on which components are compiled in. + - name: Test the shipped configuration (CGO off, no race detector) + env: + CGO_ENABLED: "0" + run: go test ./config/... ./components/... ./apply/... ./proxy/... ./store/... + + # A tag must not publish something the full suite has not seen. + - name: Full test suite + env: + CGO_ENABLED: "1" + run: go test ./... + + - name: Release + uses: goreleaser/goreleaser-action@v6 + with: + version: "~> v2" + # A tag publishes; a manual run builds the full matrix and publishes nothing. + args: ${{ startsWith(github.ref, 'refs/tags/v') && 'release --clean' || 'release --clean --snapshot' }} + env: + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + + - name: Upload snapshot artifacts + if: ${{ !startsWith(github.ref, 'refs/tags/v') }} + uses: actions/upload-artifact@v4 + with: + name: snapshot-dist + path: | + dist/*.tar.gz + dist/checksums.txt + retention-days: 7 diff --git a/.goreleaser.yaml b/.goreleaser.yaml new file mode 100644 index 00000000..8b568a1a --- /dev/null +++ b/.goreleaser.yaml @@ -0,0 +1,98 @@ +# GoReleaser: the release artifacts an evaluator downloads instead of installing a toolchain. +# +# The whole file is a plain GOOS/GOARCH matrix with no C cross-toolchains, no zig, and no +# libc coupling, because `CGO_ENABLED=0` builds the shipped binary. That is verified rather +# than assumed — the `Assert the binary is pure Go` step in .github/workflows/release.yaml fails +# the release if a cgo dependency ever escapes the cg_skeleton build tag. Measured directly on +# go 1.26.4: +# all four targets build, the artifact is 27–34 MB stripped, `file` reports "statically +# linked" and `ldd` "not a dynamic executable", and the resulting binary serves /healthz. +# +# `cg_skeleton` is the ONE thing that needs cgo (tree-sitter), and it is deliberately not +# built here: it is in no default preset, not in the cache story, and shipping it would mean +# per-platform C cross-compilation for a component this funnel never runs. Source build is +# documented in docs/components/skeleton.md. +# +# There is no `brews:` block yet — the tap repo and release signing are an open ownership +# question (spec §"Open questions", 3). Until it is answered the funnel installs from the +# release tarball, so nothing here depends on a repo that does not exist. Adding the tap +# later is additive and changes none of the below. +version: 2 + +project_name: context-guru + +before: + hooks: + - go mod download + +builds: + - id: context-guru-proxy + main: ./cmd/context-guru-proxy + binary: context-guru-proxy + env: + # The point of the whole file. Not inherited from the Makefile, which sets + # CGO_ENABLED=1 because `go test -race` needs it — a test-time requirement that was + # being read as a shipping requirement. + - CGO_ENABLED=0 + flags: + # Reproducible paths in panics, and no VCS stamping (the checkout is shallow in CI). + - -trimpath + - -buildvcs=false + ldflags: + # Same two symbols the Makefile stamps, so `/stats` build_version is populated in a + # released binary exactly as it is in a locally built one. + - -s -w + - -X github.com/rossoctl/context-guru/internal/buildinfo.Version={{ .Version }} + - -X github.com/rossoctl/context-guru/internal/buildinfo.Commit={{ .ShortCommit }} + goos: [linux, darwin] + goarch: [amd64, arm64] + +archives: + - id: default + ids: [context-guru-proxy] + # An evaluator untars this into ~/.local/bin, so the archive name is what they see and + # the binary inside must be the plain name with no version in it. + name_template: >- + {{ .ProjectName }}_{{ .Version }}_{{ .Os }}_{{ .Arch }} + formats: [tar.gz] + files: + - LICENSE + - README.md + - THIRD-PARTY-NOTICES + +checksums: + # scripts/install.sh verifies the downloaded tarball against this file. macOS quarantines + # an unsigned download and the installer strips the attribute, so a checksum is the only + # integrity check left in that path — it is not optional decoration. + name_template: checksums.txt + algorithm: sha256 + +snapshot: + version_template: "{{ incpatch .Version }}-next" + +changelog: + use: github + sort: asc + filters: + exclude: + - "^docs:" + - "^test:" + - "^chore:" + - "^ci:" + +release: + prerelease: auto + footer: | + ## Install + + No Go toolchain and no C compiler are needed — the binary is statically linked. + + Download the tarball for your platform, untar it, and put `context-guru-proxy` on your + `PATH`: + + ``` + tar xzf context-guru_*_darwin_arm64.tar.gz + install -m 755 context-guru-proxy ~/.local/bin/ + ``` + + Then see `docs/get-started/quickstart-proxy.md`. diff --git a/README.md b/README.md index 3fd159ec..2d1db4fc 100644 --- a/README.md +++ b/README.md @@ -108,6 +108,9 @@ docker build -t context-guru:local . ## Quickstart (60 seconds) +Download a release binary — statically linked, **no Go and no C compiler needed** — or build +from source: + ```sh # 1 — run the proxy (ships with the SWE-bench-winning cache-aware config by default) ./bin/context-guru-proxy # --preset house (the default); listens on :4000 @@ -144,6 +147,8 @@ See [docs/components.md](docs/components.md) and [docs/reference/presets.md](doc | Flag / env | Default | Purpose | |---|---|---| | `--preset` / `PRESET` | `house` | pipeline preset when no `--config` | +| `--idle-exit` / `IDLE_EXIT` | `0` (never) | exit after this long unused; floor `max(2 × store.ttl_seconds, 1h)`, refused with `--upstreams` | +| `--version` | — | print version and commit, then exit | | `--config` / `CONFIG` | — | YAML config (overrides preset) | | `LISTEN_ADDR` | `:4000` | listen address | | `--anthropic-upstream` / `ANTHROPIC_UPSTREAM` | `https://api.anthropic.com` | Anthropic upstream base | diff --git a/cmd/context-guru-proxy/idleexit.go b/cmd/context-guru-proxy/idleexit.go new file mode 100644 index 00000000..023afc13 --- /dev/null +++ b/cmd/context-guru-proxy/idleexit.go @@ -0,0 +1,172 @@ +package main + +import ( + "fmt" + "net/http" + "sync/atomic" + "time" + + "github.com/rossoctl/context-guru/store" +) + +// Idle-exit: a proxy that a Claude Code session started should not outlive the machine's use +// of it. +// +// The funnel installs a SessionStart hook that starts the proxy on demand, so nothing has to +// be left running — but only if the process eventually goes away on its own. That is all this +// is: a clock, a probe, and the SAME graceful shutdown path SIGTERM takes. No new teardown +// logic, because the teardown is the part that is already right (armShutdown releases the +// dashboard's SSE connections, the deferred closes flush the capture batch). +// +// It is OFF unless asked for. A gateway deployment or an eval-containers run must never +// self-terminate, and "the proxy vanished overnight" is a much worse failure there than a +// process left running on a laptop. +// +// Two things make this less trivial than a timeout, and both are load-bearing: +// +// 1. **The keep-alive inverts "idle".** Pinging is what the proxy does WHILE no client +// traffic arrives, so a watchdog that watches requests alone kills the feature in +// precisely its working window. Hence the pending probe below, which both blocks exit and +// resets the clock. +// 2. **Exit wipes the in-memory store.** A threshold shorter than the store's entry lifetime +// drops live frozen decisions and re-bills their prefix at cache-creation prices. That is +// refused at startup, not documented — see store.ValidateIdleExit. + +// activityClock is the last moment this process did something a user would call "in use". +// Nanoseconds in an atomic so the request path pays one store and no lock. +type activityClock struct{ ns atomic.Int64 } + +func (a *activityClock) touch(now time.Time) { a.ns.Store(now.UnixNano()) } +func (a *activityClock) last() time.Time { return time.Unix(0, a.ns.Load()) } + +// probeRoutes are the paths that do NOT count as use. +// +// They are what a machine asks, not what a person or an agent does: a Kubernetes liveness +// probe, a Prometheus scrape, a `curl /healthz` in a monitoring loop, and the session hook's +// own start-up check. Counting them was a bug that disabled the whole feature rather than +// weakening it — measured: a proxy with a 1h threshold logged +// `idle-exit armed after=1h0m0s`, then reported `idle for 1h3m0s` after 2h03m of wall clock, +// because a /healthz poller had been stamping the clock for the first hour. Any probe on a +// schedule shorter than the threshold means the exit NEVER fires, and logs nothing to say so. +// +// Everything else still counts, including the dashboard's own polling: a person with the +// dashboard open is using this process, and exiting under them is a worse failure than a +// process left running. That is a deliberate asymmetry — a probe is not a viewer. +var probeRoutes = map[string]bool{ + "/healthz": true, + "/metrics": true, +} + +// stampActivity records a request as activity, unless its route is a machine probe. +// +// The stamp happens BEFORE the handler runs, so a long streaming response cannot age out while +// it is still being served — its own duration is not idleness. (The stream also cannot be cut +// off mid-flight regardless: srv.Shutdown waits for in-flight requests, which is why this +// reuses that path rather than calling os.Exit.) +func stampActivity(next http.Handler, act *activityClock, now func() time.Time) http.Handler { + return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + if !probeRoutes[r.URL.Path] { + act.touch(now()) + } + next.ServeHTTP(w, r) + }) +} + +// idleExitOptions is everything watchIdle needs, with the clock and the ticker injected so +// the policy is testable without waiting out a real threshold. +type idleExitOptions struct { + // threshold is how long the proxy must be unused before it exits. + threshold time.Duration + // act is stamped by stampActivity on every request. + act *activityClock + // pending reports work that must keep the process alive even with no requests: keep-alive + // sessions with a ping still ahead of them. nil means "nothing pending, ever". + pending func() int + now func() time.Time + // tick drives the check. Production uses a ticker at a fraction of the threshold; the + // resolution only bounds how late the exit is, never how early. + tick <-chan time.Time + // stop abandons the watch (the process is shutting down for another reason). + stop <-chan struct{} +} + +// watchIdle blocks until the proxy has been idle for the whole threshold, and returns a +// human-readable reason for the log. ok is false when the watch was abandoned via stop. +// +// Pending keep-alive work does not merely veto the exit, it RESETS the clock. Vetoing alone +// would exit the instant the last ping retired, taking the store with it at the moment a +// session is most likely to come back — the quiet gap after `end_turn` is where the pings +// were aimed in the first place. Treating a pending ping as activity gives the session a full +// threshold of grace after its last one. +func watchIdle(o idleExitOptions) (string, bool) { + if o.now == nil { + o.now = time.Now + } + // A BACKSTOP, not the real seed. The caller stamps the clock at launch (main), which is + // the only place that knows when "launch" was; seeding here would date the clock from + // whenever this goroutine happened to get scheduled, which is both later and unknowable. + // It stays because the failure mode of an unstamped clock is the worst one available — a + // zero clock reads as "idle since 1970" and exits on the first tick. + if o.act.last().UnixNano() == 0 { + o.act.touch(o.now()) + } + for { + select { + case <-o.stop: + return "", false + case <-o.tick: + now := o.now() + if o.pending != nil { + if n := o.pending(); n > 0 { + o.act.touch(now) + continue + } + } + if idle := now.Sub(o.act.last()); idle >= o.threshold { + return "idle for " + idle.Round(time.Second).String() + + " (--idle-exit " + o.threshold.String() + ")", true + } + } + } +} + +// idleCheckInterval is how often the watchdog looks. A twentieth of the threshold keeps the +// exit within 5% of what was asked for, clamped so a 24h default does not mean an hour of +// slack and a 1h floor does not mean a check every three minutes. +func idleCheckInterval(threshold time.Duration) time.Duration { + d := threshold / 20 + if d < 30*time.Second { + d = 30 * time.Second + } + if d > 5*time.Minute { + d = 5 * time.Minute + } + return d +} + +// checkIdleExit is every reason a requested idle-exit threshold must not start. +// +// A function rather than two inline `if`s in main so both refusals are testable: they are +// startup-fatal, which is the one class of check where "it looked right" is the only evidence +// anyone ever gathers. +func checkIdleExit(d time.Duration, upstreamsPath string, o store.Options) error { + // The floor. Exiting clears the in-memory store, and losing a live frozen decision re-bills + // its whole prefix as cache creation — the 11.5x regression FrozenLost exists to catch. + if err := store.ValidateIdleExit(d, o); err != nil { + return err + } + if d > 0 && upstreamsPath != "" { + // A self-terminating GATEWAY is a different kind of wrong: --upstreams means this + // process serves other people's agents, where "the proxy vanished overnight" is far + // worse than a process left running on a laptop. + // + // The safety used to be accidental — it held only because a hosted deployment also runs + // a liveness probe, and every probe stamped the activity clock. That is no longer true + // (probeRoutes above deliberately excludes them), so what was accidentally safe is now + // explicitly refused rather than quietly reintroduced. + return fmt.Errorf("--idle-exit cannot be combined with --upstreams: a gateway serving " + + "other people's agents must not self-terminate. Drop --idle-exit, or run this " + + "instance without --upstreams") + } + return nil +} diff --git a/cmd/context-guru-proxy/idleexit_test.go b/cmd/context-guru-proxy/idleexit_test.go new file mode 100644 index 00000000..9ecd36f0 --- /dev/null +++ b/cmd/context-guru-proxy/idleexit_test.go @@ -0,0 +1,329 @@ +package main + +import ( + "net/http" + "net/http/httptest" + "strings" + "sync/atomic" + "testing" + "time" + + "github.com/rossoctl/context-guru/store" +) + +// The shipped idle-exit default is 24h, so these tests drive a hand-advanced clock and a +// hand-fed ticker instead of waiting. watchIdle reads the time from o.now() and treats a tick +// purely as "look now", so the value carried on the channel is irrelevant and a tick that +// arrives late still evaluates against the current fake clock. + +type fakeClock struct{ ns atomic.Int64 } + +func newFakeClock(t time.Time) *fakeClock { + c := &fakeClock{} + c.ns.Store(t.UnixNano()) + return c +} +func (c *fakeClock) now() time.Time { return time.Unix(0, c.ns.Load()) } +func (c *fakeClock) advance(d time.Duration) { c.ns.Add(int64(d)) } + +// watcher drives one watchIdle and reports its verdict. +// +// Two things here are deliberate, and both were bugs first: +// +// - **The tick channel is UNBUFFERED.** With a buffer, the first send succeeds against the +// buffer whether or not the watcher goroutine has been scheduled at all — so a test could +// advance its clock believing the watcher had already started, and then measure idleness +// from the wrong instant. Unbuffered makes a send a rendezvous: it completes only once the +// watcher has actually received it. +// - **Every interaction selects on the result channel too.** The moment the watcher exits it +// stops draining ticks, and an unconditional send then blocks until the test deadline — +// a hang, which tells you nothing, rather than a failure. +type watcher struct { + t *testing.T + tick chan time.Time + res chan string + clk *fakeClock +} + +func start(t *testing.T, clk *fakeClock, o idleExitOptions) *watcher { + return startWith(t, clk, o, false) +} + +// startWith exposes the one knob start hides: seedAtLaunch=true leaves the activity +// clock unstamped, so watchIdle's own backstop is what gets tested. +func startWith(t *testing.T, clk *fakeClock, o idleExitOptions, seedAtLaunch bool) *watcher { + t.Helper() + w := &watcher{t: t, tick: make(chan time.Time), res: make(chan string, 1), clk: clk} + o.tick = w.tick + o.now = clk.now + // Stamp the clock the way main does at launch, unless the test is specifically exercising + // the unstamped case. + if !seedAtLaunch { + o.act.touch(clk.now()) + } + go func() { + reason, ok := watchIdle(o) + if !ok { + reason = "" // abandoned via stop + } + w.res <- reason + }() + // Synchronise before returning, with a real tick rather than a sleep: on an unbuffered + // channel a completed send proves the watcher is running and has reached its select, so a + // clock the test advances afterwards cannot be mistaken for the launch time. + // + // It doubles as an assertion: at zero elapsed time nothing may exit. + if verdict, done := w.poke(); done { + t.Fatalf("watchIdle exited (%q) on its first look, with no time elapsed", verdict) + } + return w +} + +// poke delivers one tick, or reports the verdict if the watcher has already finished. +func (w *watcher) poke() (verdict string, done bool) { + w.t.Helper() + select { + case r := <-w.res: + return r, true + case w.tick <- w.clk.now(): + return "", false + case <-time.After(3 * time.Second): + w.t.Fatal("watchIdle is neither consuming ticks nor returning") + return "", true + } +} + +// mustNotExit checks the watcher evaluated the current clock and stayed alive. +// +// It pokes TWICE on purpose: the tick channel holds one, so a second successful send proves +// the first was consumed and the loop came back for more, rather than merely sitting in the +// buffer unexamined. Without that, "no exit" could just mean "never looked". +func (w *watcher) mustNotExit(what string) { + w.t.Helper() + for i := 0; i < 2; i++ { + if verdict, done := w.poke(); done { + w.t.Fatalf("%s: watchIdle exited (%q) when it must not", what, verdict) + } + } +} + +// mustExit gives the watcher a bounded number of looks to decide it is idle. +func (w *watcher) mustExit(what string) string { + w.t.Helper() + for i := 0; i < 4; i++ { + if verdict, done := w.poke(); done { + if verdict == "" { + w.t.Fatalf("%s: the watch was abandoned instead of exiting", what) + } + return verdict + } + } + w.t.Fatalf("%s: idle past the threshold, but watchIdle never exited", what) + return "" +} + +// TestIdleExitFiresWhenNothingIsHappening is the base case the feature exists for: a proxy a +// session started, and then nobody used, goes away by itself instead of being left on the +// evaluator's machine. +func TestIdleExitFiresWhenNothingIsHappening(t *testing.T) { + clk := newFakeClock(time.Unix(1_700_000_000, 0)) + w := start(t, clk, idleExitOptions{threshold: time.Hour, act: &activityClock{}, + pending: func() int { return 0 }, stop: make(chan struct{})}) + + clk.advance(30 * time.Minute) + w.mustNotExit("half a threshold") + + clk.advance(31 * time.Minute) + t.Logf("exit reason: %s", w.mustExit("past the threshold")) +} + +// TestIdleExitWaitsForAPendingKeepAlivePing is the case a naive watchdog gets wrong. +// +// The keep-alive INVERTS the meaning of idle: pinging is what the proxy does precisely while +// no client traffic is arriving — the quiet gap after `end_turn`, where 83.7% of the +// recoverable dollars sit. A watchdog counting requests alone would kill the process in +// exactly the window the feature was built for. +// +// Two properties, the second subtler than the first: +// +// 1. a pending ping VETOES the exit, however long the client silence; +// 2. it also RESETS the clock, so retiring the last ping does not exit moments later — it +// buys a full fresh threshold. Veto-only would drop the in-memory store at the instant +// the session is most likely to come back, which is the cache-write regression the floor +// and this whole feature are meant to avoid. +func TestIdleExitWaitsForAPendingKeepAlivePing(t *testing.T) { + clk := newFakeClock(time.Unix(1_700_000_000, 0)) + var pending atomic.Int64 + pending.Store(1) + w := start(t, clk, idleExitOptions{threshold: time.Hour, act: &activityClock{}, + pending: func() int { return int(pending.Load()) }, stop: make(chan struct{})}) + + // (1) Veto: two full thresholds of silence with a ping still scheduled. + clk.advance(2 * time.Hour) + w.mustNotExit("a keep-alive ping is still scheduled") + + // (2) The ping retires. If the veto reset the clock, a threshold measured from the START + // is not enough — only 30m have passed since the last pending observation. + pending.Store(0) + clk.advance(30 * time.Minute) + w.mustNotExit("30m after the last ping retired") + + clk.advance(31 * time.Minute) + w.mustExit("genuinely idle for a whole threshold") +} + +// TestRequestsDeferIdleExit covers the stamping half: a real request is use, and use defers the +// exit. +func TestRequestsDeferIdleExit(t *testing.T) { + clk := newFakeClock(time.Unix(1_700_000_000, 0)) + act := &activityClock{} + stampedBeforeHandler := false + h := stampActivity(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + // The stamp must land BEFORE the handler runs, so a long streaming response cannot + // age out while it is still being served. + stampedBeforeHandler = act.last().Equal(clk.now()) + w.WriteHeader(http.StatusOK) + }), act, clk.now) + + w := start(t, clk, idleExitOptions{threshold: time.Hour, act: act, + pending: func() int { return 0 }, stop: make(chan struct{})}) + + clk.advance(50 * time.Minute) + h.ServeHTTP(httptest.NewRecorder(), httptest.NewRequest("POST", "/anthropic/v1/messages", nil)) + if !stampedBeforeHandler { + t.Fatal("stampActivity did not record the request before invoking the handler") + } + // 80m since launch, but only 30m since the request. + clk.advance(30 * time.Minute) + w.mustNotExit("30m after serving a request") + + clk.advance(31 * time.Minute) + w.mustExit("an hour after the last request") +} + +// TestIdleExitStopAbandonsTheWatch: when the process is already shutting down for another +// reason (SIGTERM), the watchdog must let go rather than hold a goroutine and push a second +// reason into the shutdown path. +func TestIdleExitStopAbandonsTheWatch(t *testing.T) { + clk := newFakeClock(time.Unix(1_700_000_000, 0)) + stop := make(chan struct{}) + w := start(t, clk, idleExitOptions{threshold: time.Hour, act: &activityClock{}, + pending: func() int { return 0 }, stop: stop}) + close(stop) + select { + case r := <-w.res: + if r != "" { + t.Fatalf("stop should abandon the watch, got exit reason %q", r) + } + case <-time.After(3 * time.Second): + t.Fatal("watchIdle ignored stop") + } +} + +// TestIdleExitStartsItsClockAtLaunch: a proxy that never serves a single request still has to +// exit. Nothing stamps the activity clock in that case, so watchIdle has to seed it itself — +// a zero clock would otherwise read as "idle since 1970" and exit on the first tick, which is +// the opposite failure and just as wrong. +func TestIdleExitStartsItsClockAtLaunch(t *testing.T) { + clk := newFakeClock(time.Unix(1_700_000_000, 0)) + w := startWith(t, clk, idleExitOptions{threshold: time.Hour, act: &activityClock{}, + pending: func() int { return 0 }, stop: make(chan struct{})}, true) + w.mustNotExit("first tick on a proxy that has served nothing") + clk.advance(61 * time.Minute) + w.mustExit("an hour after launch with no traffic at all") +} + +// TestIdleCheckIntervalStaysUseful pins the resolution at both ends: a 24h default must not +// mean an hour of slack past the threshold, and the 1h floor must not mean a check every few +// minutes for nothing. +func TestIdleCheckIntervalStaysUseful(t *testing.T) { + for _, c := range []struct{ threshold, want time.Duration }{ + {24 * time.Hour, 5 * time.Minute}, // clamped high + {time.Hour, 3 * time.Minute}, // threshold/20 + {10 * time.Minute, 30 * time.Second}, // clamped low + } { + if got := idleCheckInterval(c.threshold); got != c.want { + t.Errorf("idleCheckInterval(%s) = %s, want %s", c.threshold, got, c.want) + } + } +} + +// TestProbesDoNotDeferIdleExit is the other half, and it is the one that was a live bug. +// +// A liveness probe or a Prometheus scrape is a machine asking whether the process is up — not +// somebody using it. Counting those did not weaken --idle-exit, it DISABLED it: any probe on a +// schedule shorter than the threshold means the exit never fires, and the only log line is the +// `idle-exit armed` one at startup, so nothing says it silently stopped working. Measured on a +// 1h-threshold proxy: 2h03m of wall clock, then `idle for 1h3m0s`, the clock having been held +// up for an hour by a /healthz poller alone. +func TestProbesDoNotDeferIdleExit(t *testing.T) { + clk := newFakeClock(time.Unix(1_700_000_000, 0)) + act := &activityClock{} + h := stampActivity(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + w.WriteHeader(http.StatusOK) + }), act, clk.now) + + w := start(t, clk, idleExitOptions{threshold: time.Hour, act: act, + pending: func() int { return 0 }, stop: make(chan struct{})}) + + // A probe every 10 minutes for two hours — the shape of a real monitoring loop. + for i := 0; i < 12; i++ { + clk.advance(10 * time.Minute) + for _, path := range []string{"/healthz", "/metrics"} { + h.ServeHTTP(httptest.NewRecorder(), httptest.NewRequest("GET", path, nil)) + } + } + if got := w.mustExit("two hours of nothing but liveness probes"); got == "" { + t.Fatal("no exit reason") + } + + // And the asymmetry is deliberate, so pin it: a dashboard poll IS use. Exiting under + // somebody who is watching is a worse failure than a process left running. + clk2 := newFakeClock(time.Unix(1_700_000_000, 0)) + act2 := &activityClock{} + h2 := stampActivity(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {}), act2, clk2.now) + w2 := start(t, clk2, idleExitOptions{threshold: time.Hour, act: act2, + pending: func() int { return 0 }, stop: make(chan struct{})}) + clk2.advance(50 * time.Minute) + h2.ServeHTTP(httptest.NewRecorder(), httptest.NewRequest("GET", "/api/events", nil)) + clk2.advance(30 * time.Minute) + w2.mustNotExit("a dashboard tab is open and polling") +} + +// TestCheckIdleExitRefusesAGatewaySelfTerminating covers the second startup refusal. +// +// `--upstreams` means this process serves other people's agents. A proxy that vanishes overnight +// there is a far worse failure than one left running on a laptop — and the protection used to be +// accidental: it held only because a hosted deployment runs a liveness probe, and every probe +// stamped the activity clock. probeRoutes deliberately stopped counting probes, which removes +// that accident, so the refusal has to be explicit or the combination silently becomes live. +func TestCheckIdleExitRefusesAGatewaySelfTerminating(t *testing.T) { + ok := store.Options{} // default TTL => floor 5h33m20s + good := 24 * time.Hour // clears the floor + for _, c := range []struct { + name string + d time.Duration + upstreams string + wantErr string + }{ + {"laptop install: no upstreams", good, "", ""}, + {"gateway with idle-exit", good, "/etc/context-guru/upstreams.yaml", "--upstreams"}, + // Off is always fine, including on a gateway: that is the shipped default and the + // refusal must not fire on a configuration everybody runs. + {"gateway without idle-exit", 0, "/etc/context-guru/upstreams.yaml", ""}, + // The floor still applies, and it is reported first — a threshold that is BOTH too short + // and on a gateway should name the floor, since that is the value the operator typed. + {"below the floor", 30 * time.Minute, "", "floor"}, + {"below the floor on a gateway", 30 * time.Minute, "/etc/x.yaml", "floor"}, + } { + err := checkIdleExit(c.d, c.upstreams, ok) + switch { + case c.wantErr == "" && err != nil: + t.Errorf("%s: refused a valid configuration: %v", c.name, err) + case c.wantErr != "" && err == nil: + t.Errorf("%s: accepted a configuration that must not start", c.name) + case c.wantErr != "" && err != nil && !strings.Contains(err.Error(), c.wantErr): + t.Errorf("%s: message does not mention %q: %v", c.name, c.wantErr, err) + } + } +} diff --git a/cmd/context-guru-proxy/main.go b/cmd/context-guru-proxy/main.go index 584c5e4a..25dc0655 100644 --- a/cmd/context-guru-proxy/main.go +++ b/cmd/context-guru-proxy/main.go @@ -63,7 +63,13 @@ func listenAndAnnounce(addr string, attrs ...any) (net.Listener, error) { func main() { var ( - addr = envOr("LISTEN_ADDR", ":4000") + // --listen, not just LISTEN_ADDR. Two reasons beyond taste: an operator reading `ps` + // could not tell which port an instance held (the address reached it only through the + // environment), and a supervisor that needs to stop ONE instance among several had + // nothing in the command line to match on. Pattern-matching a process for shutdown is + // still the wrong tool — but when it happens, the port must at least be visible. + addrFlag = flag.String("listen", envOr("LISTEN_ADDR", ":4000"), "address to listen on") + showVer = flag.Bool("version", false, "print version and exit") cfgPath = flag.String("config", envOr("CONFIG", ""), "path to context-guru YAML config") preset = flag.String("preset", envOr("PRESET", "house"), "preset to use when --config is absent (house = the service default, deterministic; housellm = the same plus the compaction-model pass; codesmart/codesafe = the SWE-bench study's configs, kept so its published numbers stay reproducible)") openai = flag.String("openai-upstream", envOr("OPENAI_UPSTREAM", "https://api.openai.com"), "OpenAI upstream base URL") @@ -78,6 +84,14 @@ func main() { bob = flag.String("bob-upstream", envOr("BOB_UPSTREAM", ""), "Bob (BobShell) backend base URL; enables the Bob gateway routes when set (e.g. https://api.us-east.bob.ibm.com)") storeFlag = flag.String("store", envOr("STORE", ""), "override state store: true|false (default: config store.enabled, else on)") modeFlag = flag.String("mode", envOr("MODE", ""), "operating mode: sync (default) | observe (overrides the config's mode:)") + // OFF by default, and it must stay that way: a gateway or eval-containers deployment + // that self-terminates is a much worse failure than a laptop process left running. + // Set only by the plugin installer, which pairs it with a SessionStart hook that + // starts the proxy again on demand — self-kill without that resurrection is a + // footgun, so they ship together. The floor is enforced below, not documented. + idleExit = flag.Duration("idle-exit", envDuration("IDLE_EXIT", 0), + "exit after this long with no requests and no keep-alive ping pending (0 = never; "+ + "must be at least 2x store.ttl_seconds, see store.IdleExitFloor)") // Dashboard. Off by default so an existing deployment's behavior and route // table are unchanged until asked for; on, it adds /dashboard/ + /api/*. @@ -190,6 +204,17 @@ func main() { ) flag.Parse() + // --version before anything else: an installer asks a binary what it is, and it must be + // able to ask without starting a server or needing a config. buildinfo.Version was already + // compiled in and reachable only via /stats, which requires a running proxy — so + // `context-guru-proxy --version` was answered by the flag package's usage text, and an + // installer parsing it recorded "Usage of context-guru-proxy:" as the installed version. + if *showVer { + fmt.Printf("context-guru-proxy %s (commit %s)\n", buildinfo.Version, buildinfo.Commit) + return + } + addr := *addrFlag + // Logging first, before anything can want to log. Level, format and sink come from // the environment (CG_LOG_LEVEL / CG_LOG_FORMAT / CG_LOG_FILE / CG_LOG_PLAIN) rather // than flags, because the two places that set them are a systemd drop-in and a shell, @@ -571,6 +596,17 @@ func main() { slog.Warn("context-guru: OBSERVE MODE — requests are forwarded UNMODIFIED; " + "/stats reports what compaction WOULD have saved under potential_*/projected_* keys") } + // Idle-exit validation goes BEFORE the "listening" line, because a fatal here used to be + // logged after it: the operator saw `context-guru-proxy listening` and then an exit, which + // reads as a crash rather than as a rejected configuration. + // + // Refused rather than warned about: a threshold below the store's entry lifetime does not + // degrade gracefully, it re-bills live prefixes as cache creation (the 11.5x regression + // FrozenLost exists to catch). A misconfigured value must not start. + if err := checkIdleExit(*idleExit, *upstreamsPath, cfg.Store); err != nil { + log.Fatalf("context-guru: %v", err) + } + // The sink last, so it is the line just above the traffic: "where are the logs and // what level am I getting" is the first question when something looks quiet. ln, err := listenAndAnnounce(addr, "pipeline", cfg.Pipeline, "mode", mode, "logs", sink) @@ -578,9 +614,22 @@ func main() { log.Fatalf("listen: %v", err) } + // Activity stamping is wired ONLY when the watchdog is on, so an ordinary deployment's + // handler chain is byte-identical to before. + var handler http.Handler = h.Mux() + act := &activityClock{} + if *idleExit > 0 { + // Launch counts as activity, so the threshold is measured from a moment that means + // something rather than from whenever the watchdog goroutine is first scheduled. + act.touch(time.Now()) + handler = stampActivity(handler, act, time.Now) + slog.Info("context-guru: idle-exit armed", "after", *idleExit, + "check_every", idleCheckInterval(*idleExit)) + } + srv := &http.Server{ Addr: addr, - Handler: h.Mux(), + Handler: handler, // ReadHeaderTimeout is the one that matters for a service on a network: without // it, a client that opens a connection and never finishes its headers holds a // goroutine and a file descriptor indefinitely. @@ -600,12 +649,36 @@ func main() { // Graceful shutdown, so the dashboard's writer goroutine flushes its batch and any // in-flight archive upload is not abandoned halfway. Without this, a restart loses // the last few hundred milliseconds of captured requests every time. + // + // Both reasons to stop — a signal, and the idle watchdog — converge on ONE teardown, so + // the self-terminating path cannot drift from the one that is known to work. idle := make(chan struct{}) + why := make(chan string, 2) + stopWatch := make(chan struct{}) go func() { sig := make(chan os.Signal, 1) signal.Notify(sig, syscall.SIGINT, syscall.SIGTERM) s := <-sig - slog.Info("context-guru: shutting down", "signal", s.String()) + why <- "signal " + s.String() + }() + if *idleExit > 0 { + t := time.NewTicker(idleCheckInterval(*idleExit)) + go func() { + defer t.Stop() + // h.PendingPings is the half of "idle" that requests cannot express: the + // keep-alive works precisely when no client traffic is arriving. + if reason, ok := watchIdle(idleExitOptions{ + threshold: *idleExit, act: act, pending: h.PendingPings, + now: time.Now, tick: t.C, stop: stopWatch, + }); ok { + why <- reason + } + }() + } + go func() { + reason := <-why + close(stopWatch) + slog.Info("context-guru: shutting down", "reason", reason) ctx, cancel := context.WithTimeout(context.Background(), 25*time.Second) defer cancel() if err := srv.Shutdown(ctx); err != nil { diff --git a/config/config.go b/config/config.go index 4bc6c0b0..f554fb38 100644 --- a/config/config.go +++ b/config/config.go @@ -358,7 +358,27 @@ func (c *Config) applyPreset() error { // sweep found 0 convertible candidates in 11.67M tokens. It was costing 1.53 ms and a // TextTokens call per tool message to convert nothing. var presets = map[string][]string{ - "off": {}, // passthrough: no components (baseline / A-B control) + "off": {}, // passthrough: no components (baseline / A-B control) + // cache: the volatile-tail split and NOTHING else. This is the preset a stranger + // evaluating context-guru on their own Claude Code sessions is pointed at, and the + // reason it exists is that it can be verified by reading this one line: no content is + // dropped, no `<>` marker is written, no expand tool is injected into the + // request, and no model is called. The loudest objection to putting a proxy on the + // wire — "you are editing my agent's context" — does not apply to it. + // + // It is also the best-evidenced single component in the repo: -34.1% cost and 0% -> + // 96.7% prefix-cache hit in an isolated A/B (docs/results/context-guru.md), which is + // why the funnel leads with the cache rather than the offloaders. + // + // Deliberately NOT `safe` (format -> textclean -> searchfold -> cachesplit): those are + // lossless in meaning but they still rewrite the JSON, so "we do not touch your + // context" stops being literally true and a reviewer has to take four components on + // trust instead of reading one. TestCachePresetIsCachesplitAlone holds it to that, and + // the lossless-folds rule exempts it for the same reason. + // + // Anthropic-family only, and the docs say so: cachesplit is a no-op on implicit + // prefix-cache backends (vLLM, llm-d) — see apply/prefixsplit.go. + "cache": {"cachesplit"}, "safe": {"format", "textclean", "searchfold", "cachesplit"}, "balanced": {"format", "textclean", "searchfold", "dedup", "failed_run", "cmdfilter", "linecap", "cachesplit"}, "aggressive": {"format", "textclean", "searchfold", "dedup", "failed_run", "cmdfilter", "smartcrush", "extract", "extract_llm", "linecap", "cachesplit"}, diff --git a/config/config_more_test.go b/config/config_more_test.go index 5af1c4ed..26ffc5fa 100644 --- a/config/config_more_test.go +++ b/config/config_more_test.go @@ -125,7 +125,13 @@ func TestLosslessFoldsAreInEveryWorkingPreset(t *testing.T) { // restructures the transcript alone, and agentdiet reproduces a published baseline // whose whole claim is what ONE reflection achieves — stacking folds beside it would // reduce the same outputs first and there would be nothing left to attribute. - exempt := map[string]bool{"off": true, "summarize": true, "agentdiet": true} + // `cache` is exempt for a reason the other three do not share: its whole product claim + // is that it is ONE component, verifiable by reading one line of the presets map. Adding + // format/textclean/searchfold to it would each be lossless in meaning and would still + // cost the claim — a stranger deciding whether to route their agent through us can check + // "nothing but a cache breakpoint moves" in a second, and cannot check four rewriters as + // fast. See TestCachePresetIsCachesplitAlone, which holds the other side of that trade. + exempt := map[string]bool{"off": true, "summarize": true, "agentdiet": true, "cache": true} for name, pipeline := range presets { if exempt[name] { continue @@ -217,3 +223,36 @@ func TestLinecapRunsLastAmongTheOffloaders(t *testing.T) { } } } + +// TestCachePresetIsCachesplitAlone guards the one preset whose CONTENT is its promise. +// +// `cache` is what the local-distribution funnel points a stranger at, and the pitch is +// exact: no content dropped, no `<>` marker written, no expand tool injected, no +// model called. That is not a property of cachesplit that survives company — every other +// component in the repo either rewrites JSON, offloads content, or calls a model, so ANY +// addition here converts a checkable claim into a trust-me claim, and the docs that make the +// claim (docs/how-to/choose-a-preset.md, the plugin's install skill) do not get to notice. +// +// It is also why `cache` is exempt from TestLosslessFoldsAreInEveryWorkingPreset. That +// exemption is only defensible while this test exists: without it, "cache is exempt from the +// folds rule" would read as permission to put anything at all in it. +func TestCachePresetIsCachesplitAlone(t *testing.T) { + p, ok := presets["cache"] + if !ok { + t.Fatal("preset `cache` is gone; the local-distribution funnel and the install skill both name it") + } + if len(p) != 1 || p[0] != "cachesplit" { + t.Fatalf("preset `cache` = %v, want exactly [cachesplit]: it is the only preset whose "+ + "losslessness is verifiable by reading one line, and every other component either "+ + "rewrites JSON, offloads content, or calls a model", p) + } + // The pipeline the proxy actually builds, not just the map literal: applyPreset and the + // rich-preset path both sit between this map and the wire. + built, ok := PresetPipeline("cache") + if !ok { + t.Fatal(`PresetPipeline("cache") did not resolve, so ?preset=cache would 400`) + } + if len(built) != 1 || built[0] != "cachesplit" { + t.Fatalf(`PresetPipeline("cache") = %v, want [cachesplit]`, built) + } +} diff --git a/docs/get-started/quickstart-proxy.md b/docs/get-started/quickstart-proxy.md index e82ebb44..c554a9ad 100644 --- a/docs/get-started/quickstart-proxy.md +++ b/docs/get-started/quickstart-proxy.md @@ -3,19 +3,30 @@ Run context-guru in front of your provider and point an agent at it. One port serves both the OpenAI and Anthropic dialects. -You need **Go 1.26**. You do **not** need a C toolchain: `make build` builds with cgo disabled, and -the result is a statically linked binary with no runtime dependencies. Everything else is a normal -module dependency — build straight from the repo root. +**You need no toolchain at all to run it.** The shipped binary is statically linked pure Go — no C +compiler, no Go install, no runtime dependencies. Grab it from +[Releases](https://github.com/rossoctl/context-guru/releases): + +```sh +# Pick your platform: linux/darwin × amd64/arm64 +tar xzf context-guru_*_darwin_arm64.tar.gz +install -m 755 context-guru-proxy ~/.local/bin/ +``` + +To build from source instead you need **Go 1.26** — and still no C toolchain: `make build` builds +with cgo disabled and produces the same statically linked binary. CI asserts that natively for +linux/amd64 (the `purego` job), and the release workflow asserts it again before publishing. A C compiler is needed for exactly two things: `make test` (the race detector requires cgo) and the optional [`skeleton`](../components/skeleton.md) component's `cg_skeleton` build tag. ## Steps -1. Build: +1. Build (source path only — skip if you downloaded a release): ```sh make build # → bin/context-guru-proxy + make build-static # the pure-Go build releases ship (CGO_ENABLED=0) ``` 2. Run it. It listens on `:4000`; set `LISTEN_ADDR` to change that. diff --git a/docs/how-to/choose-a-preset.md b/docs/how-to/choose-a-preset.md index 313f6f15..0451229f 100644 --- a/docs/how-to/choose-a-preset.md +++ b/docs/how-to/choose-a-preset.md @@ -11,6 +11,7 @@ context-guru-proxy --preset codesmart # or PRESET=codesmart, or preset: in | Your workload | Preset | |---|---| +| **Trying context-guru for the first time** | **`cache`** | | **Most agents — the recommended pipeline** | **`codesmart`** (pass `--preset codesmart`; the binary defaults to `house`) | | Same, but no LLM on the hot path | `codesafe` | | A guaranteed-safe, lossless win only | `safe` | @@ -28,6 +29,7 @@ context-guru-proxy --preset codesmart # or PRESET=codesmart, or preset: in |---|---| | `codesmart` | `format, textclean, searchfold, dedup, failed_run, cmdfilter, extract_llm, extract, linecap, cachesplit` | | `codesafe` | `format, textclean, searchfold, dedup, failed_run, cmdfilter, extract, collapse, linecap, cachesplit` | +| `cache` | `cachesplit` | | `safe` | `format, textclean, searchfold, cachesplit` | | `balanced` | `format, textclean, searchfold, dedup, failed_run, cmdfilter, linecap, cachesplit` | | `aggressive` | `format, textclean, searchfold, dedup, failed_run, cmdfilter, smartcrush, extract, extract_llm, linecap, cachesplit` | @@ -56,6 +58,39 @@ table so it lists every preset that exists; pick from the table above this one. ## Notes on the ones people pick +### `cache` — start here + +`cachesplit` and nothing else. Pick it when what you want is to find out whether this thing +helps you, with the smallest possible claim to check: + +- **Nothing is dropped, summarised, or replaced.** No `<>` markers, no + `context_guru_expand` tool added to your requests, no model calls. It splits one oversized + system block into two adjacent text blocks whose concatenation is byte-identical, so the + model sees exactly the prompt your agent sent — and moves the cache breakpoint onto the + half that does not churn. +- **What it is worth is regime-dependent, and the funnel's regime is the weak one.** The + headline **−34.1% cost / 0% → 96.7% hit** comes from a benchmark harness running tasks + back-to-back inside the provider's 5-minute cache TTL + ([cacheinject](../components/cacheinject.md#what-the-split-is-worth)), which is + precisely the regime where the split pays — and is *one task measured three times, not a + fleet average*. On this project's own interactive traffic the figure is **$0.0298 across + 1,127 sessions / 11,361 requests** + ([dashboard](../dashboard.md#what-it-is-actually-worth-here-and-why-that-is-small)): Claude + Code captures the environment snapshot once per session, and 1,105 of 1,127 session starts + read zero tokens from cache because the previous prefix had already expired. It is also + **exactly zero** outside a git repository, on a system prompt under the 1,024-token + `minSplitTokens` floor, and on any implicit prefix-cache backend (vLLM, llm-d). Neither + figure is wrong; they differ by three orders of magnitude because the mechanism needs a + second session inside five minutes. + +- **Anthropic-family only.** `cachesplit` is a no-op against implicit prefix-cache backends + (vLLM, llm-d), which match to the divergence on their own — so on those it costs nothing + and buys nothing. + +Move to `codesmart` once you want the offloaders too. `safe` is the next step up and is still +lossless in meaning, but it does rewrite JSON, so `cache` is the one whose promise you can +confirm by reading a single line of `config/config.go`. + **`codesmart`** is the shipped default and the cheapest arm in the [benchmarks](../RESULTS.md) at the highest reward. It is the one preset that ships tuned per-component settings rather than a bare name-list, which is why most turns make no model diff --git a/docs/how-to/use-with-claude-code.md b/docs/how-to/use-with-claude-code.md index ff3eaefd..68862888 100644 --- a/docs/how-to/use-with-claude-code.md +++ b/docs/how-to/use-with-claude-code.md @@ -3,6 +3,22 @@ Route [Claude Code](https://docs.claude.com/en/docs/claude-code) through context-guru with one environment variable — no changes to Claude Code itself. +## You do not need an API key + +Setting `ANTHROPIC_BASE_URL` **without** a credential variable leaves your claude.ai login in +place: a Pro or Max subscription keeps working, with your usage limits and billing unchanged. You +can run context-guru in front of your own sessions with **no API key at all** — which is the +cheapest way to evaluate it. + +Two honest caveats: + +- On subscription billing the saving lands in **usage limits**, not dollars, so `/stats` cost + figures are list-price estimates and will not match a bill you do not receive. +- Setting `ANTHROPIC_API_KEY` or `ANTHROPIC_AUTH_TOKEN` in your Claude Code environment is what + moves you onto metered API billing. Only do it deliberately — see + [Keep the API key out of Claude Code](#keep-the-api-key-out-of-claude-code), which is about + the *proxy* holding the key, not Claude Code. + ## Steps 1. Start the proxy: @@ -38,6 +54,9 @@ Add to `.claude/settings.json` so you don't export anything by hand: } ``` +Use `.claude/settings.local.json` instead if you do not want to commit it: a base URL pointing at +`localhost` breaks Claude Code for everyone who clones the repo whenever the proxy is not running. + ## Keep the API key out of Claude Code Give the proxy the real key and hand Claude Code a placeholder; the proxy injects the diff --git a/docs/reference/config.md b/docs/reference/config.md index 51717b72..64a35a45 100644 --- a/docs/reference/config.md +++ b/docs/reference/config.md @@ -78,7 +78,9 @@ for every component's config block. |---|---|---| | `--preset` / `PRESET` | `house` | Pipeline preset when no `--config`. `codesmart` is the SWE-bench arm and must be asked for by name. | | `--config` / `CONFIG` | — | YAML config file (overrides preset). | -| `LISTEN_ADDR` | `:4000` | Listen address. | +| `--listen` / `LISTEN_ADDR` | `:4000` | Listen address. The flag exists so the port is visible in `ps` and to a supervisor; before it, the address reached the process only through the environment. | +| `--version` | — | Print version and commit, then exit. | +| `--idle-exit` / `IDLE_EXIT` | `0` (never) | Exit after this long with **no requests and no keep-alive ping pending**, so a proxy started on demand does not outlive its use. Refused at startup below `max(2 × store.ttl_seconds, 1h)` — 5h33m20s at the default TTL — because exiting clears the in-memory store, and losing a frozen decision re-bills its whole prefix as cache creation. Also refused together with `--upstreams`: a gateway serving other people's agents must not self-terminate. Liveness probes (`/healthz`, `/metrics`) deliberately do **not** count as activity; anything else does, including the dashboard's own polling. | | `--openai-upstream` / `OPENAI_UPSTREAM` | `https://api.openai.com` | OpenAI upstream base. | | `--anthropic-upstream` / `ANTHROPIC_UPSTREAM` | `https://api.anthropic.com` | Anthropic upstream base. | | `--bob-upstream` / `BOB_UPSTREAM` | — | Bob (BobShell) backend base. Setting it mounts the [Bob gateway routes](routes.md#bob-bobshell-gateway-routes); unset, an unknown path 404s as before. | diff --git a/docs/reference/presets.md b/docs/reference/presets.md index c6a00f84..06d82a13 100644 --- a/docs/reference/presets.md +++ b/docs/reference/presets.md @@ -12,6 +12,7 @@ taken exactly from the `presets` map in `config/config.go`. | `codesmart` | `format` → `textclean` → `searchfold` → `dedup` → `failed_run` → `cmdfilter` → `extract_llm` → `extract` → `linecap` → `cachesplit` | The SWE-bench-winning cache-aware config: structural offloaders + a cheap-model relevance-trimmer (`extract_llm`, routed to `CHEAP_MODEL`, gated so most turns make no model call) + deterministic `extract`. `extract_llm` no-ops (→ deterministic) when no cheap model is configured. **Changed 2026-08:** the lossless trio replaced `toon`, which acted 0 times on 5,752 production requests, and `linecap` was added. Re-measure before quoting the published SWE-bench numbers against it. | | `codesafe` | `format` → `textclean` → `searchfold` → `dedup` → `failed_run` → `cmdfilter` → `extract` → `collapse` → `linecap` → `cachesplit` | `codesmart` minus the LLM pass — **deterministic-only, zero model calls by policy**. The safe control / the choice when you don't want an LLM on the hot path. | | `off` | *(empty)* | Passthrough — no components. The baseline / A-B control. | +| `cache` | `cachesplit` | **The first-run preset** — the one to point a new evaluator at. The volatile-tail split and nothing else: no content dropped, no `<>` markers, no `context_guru_expand` tool added to requests, no model calls. Chosen so a stranger deciding whether to route their agent through a local proxy can verify the claim by reading one line of `config/config.go` rather than trusting four components. The savings claim is regime-dependent and the funnel's regime is the weak one: **−34.1% cost / 0% → 96.7% hit** is a benchmark harness running tasks back-to-back inside the provider's 5-minute TTL (and is one task measured three times), while this project's own interactive traffic yields **$0.0298 across 1,127 sessions** — 1,105 of 1,127 session starts read zero from cache. Zero outside a git repo, under the 1,024-token `minSplitTokens` floor, or on an implicit prefix-cache backend (vLLM, llm-d). See [dashboard](../dashboard.md#what-it-is-actually-worth-here-and-why-that-is-small) and [cacheinject](../components/cacheinject.md). | | `safe` | `format` → `textclean` → `searchfold` → `cachesplit` | Lossless only: repack JSON compactly and split the volatile system tail so the shared prefix stays cacheable. Zero risk of dropping content. | | `balanced` | `format` → `textclean` → `searchfold` → `dedup` → `failed_run` → `cmdfilter` → `linecap` → `cachesplit` | Lossless repack + conservative offloads (dedupe, drop superseded/failed runs, filter command noise) + the cache split. **Not recommended for agentic traffic** — it omits `mask`, the biggest lever there. | | `aggressive` | `format` → `textclean` → `searchfold` → `dedup` → `failed_run` → `cmdfilter` → `smartcrush` → `extract` → `extract_llm` → `linecap` → `cachesplit` | `balanced` plus `smartcrush` (crush long homogeneous arrays), deterministic `extract` (noise collapse), and `extract_llm` (cheap-model relevance trim) for deeper savings. | diff --git a/docs/setup.md b/docs/setup.md index aa63ef66..3c21e35a 100644 --- a/docs/setup.md +++ b/docs/setup.md @@ -10,12 +10,14 @@ SWE-bench task driven by Claude Code. pure Go and statically linked. bifrost's tokenizer does **not** use cgo: o200k_base is embedded (`internal/tokens/tokens.go`). CI asserts the pure-Go build on every PR — natively, for linux/amd64 — in the `purego` job (`.github/workflows/ci.yaml`), which builds with - `CGO_ENABLED=0`, checks the artifact is statically linked, starts it and probes `/healthz`. So the - claim cannot rot back into a false one for the platform CI runs on. + `CGO_ENABLED=0`, checks the artifact is statically linked, starts it and probes `/healthz`. The + release workflow asserts it again before publishing, deliberately: a release must not depend on a + PR check having run. Cross-compilation to the other three release targets (linux/arm64, darwin/amd64, darwin/arm64) is - **not** covered by that job: it was verified by hand on go 1.26.4 and is asserted at release time - by the tag workflow, not per PR. + covered by the release build, not by that per-PR job. +- If you do not need `skeleton`, skip the build entirely and use a + [release binary](https://github.com/rossoctl/context-guru/releases). - **Docker** (for the gateway image / eval-containers), and the **eval-containers** repo. ## Build diff --git a/proxy/conformance_test.go b/proxy/conformance_test.go new file mode 100644 index 00000000..c2369bba --- /dev/null +++ b/proxy/conformance_test.go @@ -0,0 +1,305 @@ +package proxy_test + +import ( + "bufio" + "encoding/json" + "fmt" + "io" + "net/http" + "net/http/httptest" + "strings" + "testing" + "time" + + "github.com/tidwall/gjson" +) + +// Gateway conformance under the funnel's default preset. +// +// The local-distribution funnel puts context-guru on the wire in front of a stranger's Claude +// Code, so "it works" is not enough — it has to not break the client, and the ways it could are +// specific and documented in the gateway protocol reference. Each test below is one of them. +// +// Two of these are worse than an outage, because they make the demo read as NEGATIVE rather +// than broken: +// +// - a buffered SSE response looks like the proxy made the model slow; +// - a rejected `cache_control` marker makes Claude Code disable prompt caching for the rest +// of the conversation, which switches off the exact thing being sold, silently. +// +// The preset under test is `cache` (cachesplit alone) throughout, because that is what an +// evaluator actually runs. The shared Claude-Code-shaped fixtures are in ccbody_test.go, and the +// two items that were DEFECTS rather than confirmations — the expand-tool gate and the missing +// count_tokens route — are tested beside their fixes in expandgate_test.go and +// counttokens_test.go. +// TestCachePresetForwardsAnUpstreamErrorByteForByte covers conformance item 4. +// +// Claude Code's capability-rejection recovery matches on the upstream's error WORDING. A gateway +// that wraps, re-encodes or summarises an error body breaks that recovery path — the client can +// no longer tell "your cache_control was refused" from any other 400, so instead of retrying +// without the capability it surfaces a failure. The status, the body and the content type all +// have to arrive exactly as the upstream wrote them. +func TestCachePresetForwardsAnUpstreamErrorByteForByte(t *testing.T) { + // A real Anthropic error shape, whitespace and key order included: this is what the + // client's matching runs against, so the test compares bytes rather than parsed JSON. + errBody := `{"type":"error","error":{"type":"invalid_request_error","message":"A maximum of 4 blocks with cache_control may be provided, but found 5."}}` + upstream := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + w.Header().Set("Content-Type", "application/json") + w.Header().Set("request-id", "req_upstream_123") + w.WriteHeader(http.StatusBadRequest) + fmt.Fprint(w, errBody) + })) + defer upstream.Close() + + h, _ := buildHandler(t, cachePipeline, upstream.URL) + srv := httptest.NewServer(h.Mux()) + defer srv.Close() + + resp, err := http.Post(srv.URL+"/anthropic/v1/messages", "application/json", + strings.NewReader(string(claudeCodeBody(t, false)))) + if err != nil { + t.Fatal(err) + } + defer resp.Body.Close() + got, _ := io.ReadAll(resp.Body) + + if resp.StatusCode != http.StatusBadRequest { + t.Errorf("status = %d, want 400: a rewritten status breaks the client's retry logic", + resp.StatusCode) + } + if string(got) != errBody { + t.Errorf("the error body was modified.\n got: %s\nwant: %s\n"+ + "Claude Code matches on the upstream's own wording to decide whether to retry "+ + "without a capability; wrapping it disables that recovery.", got, errBody) + } + if resp.Header.Get("request-id") != "req_upstream_123" { + t.Errorf("request-id header lost (%q): it is what support uses to find the call", + resp.Header.Get("request-id")) + } +} + +// TestCachePresetDoesNotBufferSSE covers conformance item 1. +// +// Claude Code aborts a stream that has been silent for 300s, and a gateway that buffers a whole +// response before relaying it stalls the client. context-guru does buffer SOME responses — the +// ones where the model opens by calling the expand tool — but under the `cache` preset nothing +// injects that tool, so the buffering path must be unreachable. This asserts that rather than +// assuming it: the client's first event has to arrive while the upstream is still writing later +// ones. +func TestCachePresetDoesNotBufferSSE(t *testing.T) { + release := make(chan struct{}) + upstream := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + w.Header().Set("Content-Type", "text/event-stream") + w.WriteHeader(http.StatusOK) + fl, _ := w.(http.Flusher) + fmt.Fprint(w, "event: message_start\ndata: {\"type\":\"message_start\",\"message\":{\"usage\":{\"input_tokens\":10}}}\n\n") + if fl != nil { + fl.Flush() + } + // Hold the rest of the stream until the test has SEEN the first event. If the proxy + // buffered, the read below would block here and the test fails on the deadline rather + // than on a wrong byte — which is exactly the client-visible symptom. + <-release + fmt.Fprint(w, "event: message_stop\ndata: {\"type\":\"message_stop\"}\n\n") + if fl != nil { + fl.Flush() + } + })) + defer upstream.Close() + + h, _ := buildHandler(t, cachePipeline, upstream.URL) + srv := httptest.NewServer(h.Mux()) + defer srv.Close() + + req, _ := http.NewRequest("POST", srv.URL+"/anthropic/v1/messages", + strings.NewReader(string(claudeCodeBody(t, true)))) + req.Header.Set("Content-Type", "application/json") + resp, err := (&http.Client{Timeout: 15 * time.Second}).Do(req) + if err != nil { + close(release) + t.Fatal(err) + } + defer resp.Body.Close() + + type read struct { + line string + err error + } + ch := make(chan read, 1) + go func() { + line, err := bufio.NewReader(resp.Body).ReadString('\n') + ch <- read{line, err} + }() + select { + case r := <-ch: + close(release) + if r.err != nil { + t.Fatalf("reading the first event: %v", r.err) + } + if !strings.Contains(r.line, "message_start") { + t.Fatalf("first line was %q, want the upstream's first event", r.line) + } + case <-time.After(5 * time.Second): + close(release) + t.Fatal("no event reached the client while the upstream was still streaming: the " + + "response is being buffered. Claude Code aborts a stream silent for 300s, and a " + + "stalled first byte reads as context-guru making the model slow.") + } +} + +// TestCachePresetNeverAddsACacheControlBreakpoint covers conformance item 2, which is the +// strongest argument for shipping `cache` rather than a placement preset. +// +// The provider caps `cache_control` markers at 4. Exceed it and the request is REJECTED — and +// Claude Code's reaction to a rejected capability is to retry without it and leave prompt +// caching OFF for the rest of the conversation. So a breakpoint-budget mistake is not an error +// the user sees; it silently switches off the thing this whole funnel is selling, and the demo +// reads as "context-guru made my session more expensive". +// +// cachesplit MOVES a breakpoint onto the stable half of a block it splits; it must never add +// one. The body below arrives at the cap, so any addition at all is a 400. +func TestCachePresetNeverAddsACacheControlBreakpoint(t *testing.T) { + var forwarded []byte + upstream := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + forwarded, _ = io.ReadAll(r.Body) + w.Header().Set("Content-Type", "application/json") + fmt.Fprint(w, `{"type":"message","usage":{"input_tokens":1}}`) + })) + defer upstream.Close() + + h, _ := buildHandler(t, cachePipeline, upstream.URL) + srv := httptest.NewServer(h.Mux()) + defer srv.Close() + + // Four inbound breakpoints — the provider's cap — spread the way a real client spreads + // them: two system blocks, one tool, one message. Assembled as text, for the key-order + // reason documented on claudeCodeBody. + bp := `,"cache_control":{"type":"ephemeral"}` + body := []byte(`{"model":"claude-sonnet-5","max_tokens":64,"system":[` + + `{"type":"text","text":` + jsonStr(attributionText) + bp + `},` + + `{"type":"text","text":` + jsonStr(volatileSystemText()) + bp + `}` + + `],"tools":[{"name":"read_file","input_schema":{"type":"object"}` + bp + `}` + + `],"messages":[{"role":"user","content":[{"type":"text","text":"hello"` + bp + `}]}]}`) + if !json.Valid(body) { + t.Fatalf("test fixture is not valid JSON: %s", body) + } + + inbound := countBreakpoints(body) + if inbound != 4 { + t.Fatalf("test setup is wrong: the request carries %d breakpoints, not the cap of 4", inbound) + } + resp, err := http.Post(srv.URL+"/anthropic/v1/messages", "application/json", strings.NewReader(string(body))) + if err != nil { + t.Fatal(err) + } + resp.Body.Close() + if len(forwarded) == 0 { + t.Fatal("upstream received nothing") + } + // The precondition that stops this being a vacuous pass: the component under test has to + // have ACTED. If cachesplit did not split, "breakpoints unchanged" is trivially true and + // asserts nothing about the rewrite. + if n := len(gjson.GetBytes(forwarded, "system").Array()); n != 3 { + t.Fatalf("cachesplit did not split the volatile tail (system has %d blocks, want 3): "+ + "the breakpoint assertion below would be vacuous", n) + } + if out := countBreakpoints(forwarded); out != inbound { + t.Errorf("breakpoints on the wire = %d, inbound = %d (cap 4). Exceeding the cap is a "+ + "400, and Claude Code answers a rejected cache_control by disabling prompt caching "+ + "for the rest of the conversation — silently switching off what this preset exists "+ + "to demonstrate.\nforwarded: %s", out, inbound, forwarded) + } +} + +// countBreakpoints counts cache_control/cachePoint markers anywhere in the body. Both spellings, +// because Bedrock/Vertex write `cachePoint` where Anthropic writes `cache_control`, and the +// provider's cap counts whatever arrives. +func countBreakpoints(body []byte) int { + n := 0 + var walk func(gjson.Result) + walk = func(v gjson.Result) { + v.ForEach(func(k, val gjson.Result) bool { + if k.String() == "cache_control" || k.String() == "cachePoint" { + n++ + } + if val.IsObject() || val.IsArray() { + walk(val) + } + return true + }) + } + walk(gjson.ParseBytes(body)) + return n +} + +// TestCachePresetLeavesTheAttributionBlockUntouched covers conformance item 3. +// +// Claude Code prepends an attribution block as the FIRST system block, and the API strips it +// only if that array arrives unchanged. cachesplit reshapes the system array, so the question is +// whether the first block survives byte-identically. +// +// Three separate properties keep it safe, and the second is the one a plausible change would +// break, so both are exercised below: +// +// 1. the attribution block carries no volatile marker, so it is not a split candidate; +// 2. blocks the split does not act on are re-emitted from their ORIGINAL raw bytes rather than +// re-encoded — re-marshalling would reorder keys and change the bytes even with identical +// content, which is enough to defeat a positional strip; +// 3. the split's minSplitTokens floor (1024) excludes a small block even when it does contain a +// marker — the second case below, where a user's own prompt happens to mention one. +// +// Proving this rather than reasoning about it is what makes the alternative — shipping +// CLAUDE_CODE_ATTRIBUTION_HEADER=0 in the installer — unnecessary, and keeps it unnecessary. +func TestCachePresetLeavesTheAttributionBlockUntouched(t *testing.T) { + for _, c := range []struct{ name, first string }{ + {"the ordinary attribution block", attributionText}, + // Small, but it mentions something the split looks for. Only the token floor keeps this + // out of the rewrite; without it the FIRST eligible block is the one that gets split, + // and that is this one. + {"a small first block that happens to name a volatile marker", + attributionText + "\nCurrent branch: whatever the user was talking about\n"}, + } { + t.Run(c.name, func(t *testing.T) { + var forwarded []byte + upstream := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + forwarded, _ = io.ReadAll(r.Body) + w.Header().Set("Content-Type", "application/json") + fmt.Fprint(w, `{"type":"message","usage":{"input_tokens":1}}`) + })) + defer upstream.Close() + + h, _ := buildHandler(t, cachePipeline, upstream.URL) + srv := httptest.NewServer(h.Mux()) + defer srv.Close() + + body := claudeCodeBodyWithFirst(t, false, c.first) + want := gjson.GetBytes(body, "system.0").Raw + + resp, err := http.Post(srv.URL+"/anthropic/v1/messages", "application/json", + strings.NewReader(string(body))) + if err != nil { + t.Fatal(err) + } + resp.Body.Close() + if len(forwarded) == 0 { + t.Fatal("upstream received nothing") + } + // Precondition: cachesplit must actually have rewritten the array, or "the first + // block is unchanged" is true because nothing happened. It must also have split the + // SECOND block, not the first — 3 blocks with the attribution intact is the only + // shape that means that. + blocks := gjson.GetBytes(forwarded, "system").Array() + if len(blocks) != 3 { + t.Fatalf("cachesplit did not act (system has %d blocks, want 3); the assertion "+ + "below would be vacuous", len(blocks)) + } + if got := blocks[0].Raw; got != want { + t.Errorf("the attribution block changed, so the API will no longer strip it "+ + "positionally.\n got: %s\nwant: %s", got, want) + } + if !strings.HasPrefix(blocks[0].Get("text").String(), attributionText) { + t.Errorf("the attribution block is no longer the first system block: %s", blocks[0].Raw) + } + }) + } +} diff --git a/proxy/keepalive.go b/proxy/keepalive.go index 50ff32a9..bf6d39ca 100644 --- a/proxy/keepalive.go +++ b/proxy/keepalive.go @@ -1158,6 +1158,44 @@ type KeepAliveStats struct { SpentUSD float64 `json:"spend_usd"` } +// PendingPings reports how many tracked sessions still have a ping scheduled ahead of them. +// +// This exists for the idle-exit watchdog, and it exists because the keep-alive INVERTS the +// ordinary meaning of "idle": pinging is what the proxy does precisely while no client +// traffic is arriving. A watchdog counting only requests would therefore kill the process in +// exactly the window the feature was built for — the quiet gap after `end_turn`, where 83.7% +// of the recoverable dollars sit. So "no requests recently" is not sufficient to exit; "and +// nothing is waiting to be pinged" is the other half. +// +// The conditions are `due`'s minus the timing term: an entry that is stopped, or has spent +// its MaxPings, or whose policy is off will never be pinged again and must not hold the +// process open. Everything else is gated at record time (see pingable), so a live entry is by +// construction one we intend to ping. +func (h *Handler) PendingPings() int { + if h == nil { + return 0 + } + return h.keeper.pendingPings() +} + +// pendingPings counts entries with a ping still ahead of them. Nil-safe: a keeper whose +// sweep never launched (the CONTEXT_GURU_KEEPALIVE kill switch) has nothing pending, which +// correctly lets an idle proxy exit. +func (k *keeper) pendingPings() int { + if k == nil { + return 0 + } + k.mu.Lock() + defer k.mu.Unlock() + n := 0 + for _, e := range k.live { + if !e.stopped && e.pol.on() && e.pings < e.pol.MaxPings { + n++ + } + } + return n +} + // Stats snapshots the keeper's counters. func (k *keeper) Stats() KeepAliveStats { if k == nil { diff --git a/store/idleexit_test.go b/store/idleexit_test.go new file mode 100644 index 00000000..f200a9e1 --- /dev/null +++ b/store/idleexit_test.go @@ -0,0 +1,82 @@ +package store + +import ( + "strings" + "testing" + "time" +) + +// TestIdleExitFloorRefusesADestructiveThreshold is about money, not tidiness. +// +// Process exit wipes this store, and what it wipes includes frozen decisions. A frozen +// decision that dies mid-session is the 11.5x cache-WRITE regression FrozenLost exists to +// detect: the next turn re-creates the whole prefix at write prices instead of reading it. So a +// short idle-exit threshold does not degrade gracefully — it turns a convenience feature into a +// cost regression that presents as the proxy misbehaving, on the machine of the first-time +// evaluator this whole funnel is aimed at. +// +// Hence a startup error rather than a doc comment. The 30-minute case below is the one somebody +// will actually reach for ("exit quickly, it is only a laptop"), and it must not start. +func TestIdleExitFloorRefusesADestructiveThreshold(t *testing.T) { + def := Options{} // ttl_seconds unset => DefaultTTL (10000s), floor 2x = 5h33m20s + if got, want := IdleExitFloor(def), 2*DefaultTTL; got != want { + t.Fatalf("IdleExitFloor(default) = %s, want %s", got, want) + } + + for _, c := range []struct { + name string + d time.Duration + o Options + wantErr bool + }{ + {"off is always valid", 0, def, false}, + {"negative is off too", -time.Hour, def, false}, + {"30m on the default TTL is destructive", 30 * time.Minute, def, true}, + {"1h is still below the default floor", time.Hour, def, true}, + {"just under the floor", 2*DefaultTTL - time.Second, def, true}, + {"exactly the floor is allowed", 2 * DefaultTTL, def, false}, + {"the installer's 24h default", 24 * time.Hour, def, false}, + // A tiny configured TTL must not collapse the floor to seconds: 2x30s is 1m, which is + // shorter than the keep-alive's own ping window, so the absolute 1h term takes over. + {"tiny ttl falls back to the 1h term", 30 * time.Minute, Options{TTLSeconds: 30}, true}, + {"tiny ttl accepts 1h", time.Hour, Options{TTLSeconds: 30}, false}, + // A LONG configured TTL must raise the floor above 1h, or an operator who deliberately + // widened the store's lifetime gets a threshold that expires it. + {"long ttl raises the floor above 24h", 24 * time.Hour, Options{TTLSeconds: 100000}, true}, + } { + err := ValidateIdleExit(c.d, c.o) + if c.wantErr && err == nil { + t.Errorf("%s: ValidateIdleExit(%s, ttl=%s) accepted a threshold below the %s floor", + c.name, c.d, c.o.EffectiveTTL(), IdleExitFloor(c.o)) + continue + } + if !c.wantErr && err != nil { + t.Errorf("%s: ValidateIdleExit(%s, ttl=%s) rejected a valid threshold: %v", + c.name, c.d, c.o.EffectiveTTL(), err) + continue + } + // The message has to tell the operator what to change. A bare "invalid value" here + // leaves them guessing at a number they have no reason to know. + if err != nil { + for _, want := range []string{"idle-exit", "floor"} { + if !strings.Contains(err.Error(), want) { + t.Errorf("%s: error message omits %q: %v", c.name, want, err) + } + } + } + } +} + +// TestEffectiveTTLIsWhatNewMemoryUses closes the gap the floor depends on: IdleExitFloor sizes +// a process's whole lifetime from EffectiveTTL, so a store that actually ran with a DIFFERENT +// lifetime would be protected by a floor computed for a lifetime it never had. NewMemory calls +// EffectiveTTL rather than repeating the defaulting rule, and this holds it there. +func TestEffectiveTTLIsWhatNewMemoryUses(t *testing.T) { + for _, o := range []Options{{}, {TTLSeconds: 0}, {TTLSeconds: -5}, {TTLSeconds: 42}, {TTLSeconds: 100000}} { + if got := NewMemory(o).ttl; got != o.EffectiveTTL() { + t.Errorf("NewMemory(%+v).ttl = %s but EffectiveTTL() = %s; the idle-exit floor is "+ + "computed from the second and would protect a lifetime the store is not using", + o, got, o.EffectiveTTL()) + } + } +} diff --git a/store/store.go b/store/store.go index a57e5e5d..eb0c9078 100644 --- a/store/store.go +++ b/store/store.go @@ -13,6 +13,7 @@ package store import ( "container/list" + "fmt" "strings" "sync" "time" @@ -162,13 +163,64 @@ func (Nop) Persists() bool { return false } // (test suites, training runs) with the sliding refresh doing the rest. const DefaultTTL = 10000 * time.Second +// EffectiveTTL is the entry lifetime this Options actually yields, defaulting included. +// +// Exported and used by NewMemory itself rather than duplicated, because a second copy of +// "zero means DefaultTTL" is exactly the drift that would matter: IdleExitFloor sizes a +// process's whole lifetime off this number, and a floor computed from a different default +// than the store runs with is a floor that protects nothing. +func (o Options) EffectiveTTL() time.Duration { + if o.TTLSeconds <= 0 { + return DefaultTTL + } + return time.Duration(o.TTLSeconds) * time.Second +} + +// IdleExitFloor is the shortest idle-exit threshold that is not destructive. +// +// Process exit WIPES this store: rewind stashes, frozen decisions, `cg:len:`. A frozen +// decision that dies mid-session is the 11.5x cache-WRITE regression that FrozenLost exists +// to detect — the session's next turn re-creates the whole prefix at write prices instead of +// reading it. So an idle-exit threshold shorter than the store's own entry lifetime turns a +// convenience feature into a cost regression that looks like the proxy misbehaving. +// +// 2x the TTL, with a 1h absolute floor. Twice, not once, because the TTL is a SLIDING +// window: an entry touched just before the idle clock started still has a full TTL ahead of +// it, so 1x can expire live state. The 1h term covers a config that sets a tiny ttl_seconds +// (a test rig, or an operator trimming memory) where 2x would collapse to seconds and the +// threshold would be shorter than the keep-alive's own ping window. +// +// With the default TTL of 10000s the floor is ~5h34m, so the installer's 24h default clears +// it comfortably; a 30-minute threshold is refused at startup rather than documented. +func IdleExitFloor(o Options) time.Duration { + if f := 2 * o.EffectiveTTL(); f > time.Hour { + return f + } + return time.Hour +} + +// ValidateIdleExit checks an idle-exit threshold against IdleExitFloor. Zero or negative +// means the watchdog is off, which is always valid — a gateway or eval-containers +// deployment must never self-terminate, so off is the default and the only way to a +// self-killing proxy is to ask for one. +func ValidateIdleExit(d time.Duration, o Options) error { + if d <= 0 { + return nil + } + if floor := IdleExitFloor(o); d < floor { + return fmt.Errorf("idle-exit %s is below the floor of %s (2x the store's %s entry "+ + "lifetime): exiting wipes the in-memory store, so a shorter threshold drops live "+ + "frozen decisions and re-bills their prefix as cache creation instead of a cache "+ + "read. Raise --idle-exit, or raise store.ttl_seconds if the short lifetime is "+ + "deliberate", d, floor, o.EffectiveTTL()) + } + return nil +} + // NewMemory builds an in-memory store. Zero/negative option fields fall back to // defaults (DefaultTTL, 1000 entries, 100 sessions of sticky sets). func NewMemory(o Options) *Memory { - ttl := time.Duration(o.TTLSeconds) * time.Second - if o.TTLSeconds <= 0 { - ttl = DefaultTTL - } + ttl := o.EffectiveTTL() max := o.MaxEntries if max <= 0 { max = 1000 From 0ed223ed3fdc07a65c002d326047adcef4dd515d Mon Sep 17 00:00:00 2001 From: DAVID AMID Date: Wed, 2 Sep 2026 09:02:48 +0300 Subject: [PATCH 2/3] ci: read the Go toolchain from go.mod everywhere (#152) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit CI validated on Go 1.25 while `go.mod` declared 1.26.4 and this PR's release workflow built on 1.26 — three numbers that have to agree, with nothing making them. That is a distribution bug, not a housekeeping one. `purego` asserts that the SHIPPED artifact builds with cgo off, and `release.yaml` builds the artifact people download; when those run different toolchains from each other and from the module, the assertion describes a build nobody ships. The `purego` job inherited the wrong pin from the job it was copied from, which is exactly how the drift spread in the first place. Fixed as a class rather than an instance: `go-version-file: go.mod` in all three places, so the module file is the single source of truth and the next toolchain bump moves CI, the release build and the module together or not at all. Typing `1.26` in three files would have fixed today's symptom and left tomorrow's. `check-latest` is dropped with the literals — it existed to pick up patch releases of a pinned minor, and go.mod names an exact version. Both workflows re-validated as YAML. The change can only really be proven by CI itself, which is where the previous mismatch was invisible. Assisted-By: Claude Opus 5 (1M context) Signed-off-by: DAVID AMID --- .github/workflows/ci.yaml | 12 ++++++++---- .github/workflows/release.yaml | 5 +++-- 2 files changed, 11 insertions(+), 6 deletions(-) diff --git a/.github/workflows/ci.yaml b/.github/workflows/ci.yaml index 95e5a1e1..e3bb0623 100644 --- a/.github/workflows/ci.yaml +++ b/.github/workflows/ci.yaml @@ -17,8 +17,10 @@ jobs: - uses: actions/checkout@v4 - uses: actions/setup-go@v5 with: - go-version: '1.25' - check-latest: true + # go.mod is the single source of truth for the toolchain. Pinning a literal here is + # what let CI validate on 1.25 while go.mod declared 1.26.4 and releases built on + # 1.26 (#152) — three numbers that have to agree and no mechanism making them. + go-version-file: go.mod - name: Lint run: make lint - name: Test & coverage @@ -54,8 +56,10 @@ jobs: - uses: actions/checkout@v4 - uses: actions/setup-go@v5 with: - go-version: '1.25' - check-latest: true + # go.mod is the single source of truth for the toolchain. Pinning a literal here is + # what let CI validate on 1.25 while go.mod declared 1.26.4 and releases built on + # 1.26 (#152) — three numbers that have to agree and no mechanism making them. + go-version-file: go.mod - name: Build with no C compiler available run: | go build -o /tmp/cg-purego ./cmd/context-guru-proxy diff --git a/.github/workflows/release.yaml b/.github/workflows/release.yaml index 2908ad54..e93fc89e 100644 --- a/.github/workflows/release.yaml +++ b/.github/workflows/release.yaml @@ -24,8 +24,9 @@ jobs: fetch-depth: 0 - uses: actions/setup-go@v5 with: - go-version: '1.26' - check-latest: true + # Same source as CI (go.mod), deliberately: an artifact people download must be built + # with the toolchain CI validated, and a literal here is how that drifts apart. + go-version-file: go.mod # The claim the release rests on, asserted in CI rather than trusted: the shipped # binary needs no C toolchain. CGO_ENABLED=0 with no compiler on PATH would fail loudly From e4e89ec862be2783258b037f684d89beb6d2ab8c Mon Sep 17 00:00:00 2001 From: DAVID AMID Date: Tue, 1 Sep 2026 11:46:32 +0300 Subject: [PATCH 3/3] feat(plugin): a Claude Code plugin for context-guru, with the #141 review's blockers fixed MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Split out of #141 as its own PR: all six of the review's blocking findings were in the plugin, and the release plumbing and conformance work should not wait behind them. The core lands in #141. `/plugin marketplace add rossoctl/context-guru` → `/plugin install` → `/context-guru:install`. Three skills over four scripts and two hooks. Default routing scope is `.claude/settings.local.json`: one repo, gitignored, `--global` an explicit opt-in — a base URL pointing at localhost breaks Claude Code everywhere a dead proxy is routed. **1. `/context-guru:uninstall` killed the user's own session and left the proxy running.** It ran `pkill -f "context-guru-proxy.*${PORT}"`. The port was passed through `LISTEN_ADDR` in the environment, so it appeared nowhere in the proxy's command line and the pattern matched no proxy — while it DID match the shell running the `pkill`, i.e. the session's own Bash tool. A user runs uninstall *because* their sessions are broken; this killed the session mid-command, reported nothing removed, and left the port held. Fixed with a handle rather than a better pattern: the starter passes `--listen` (so the port is in `argv` and `ps` is honest) and writes a pidfile under `~/.local/state/context-guru`; uninstall kills that PID, falls back to the socket's owner via `lsof`/`ss`, and confirms the process is ours before killing anything. The skill also no longer offers a broader pattern as a fallback — on a host running a production instance or a benchmark arm, that would take those down too. **2. `install.sh` could not install anything, and its documented fallback was missing.** Strict checksums now; `download_failed` (a tag with no assets) is documented as an outcome; the `go install` fallback the header comment described is implemented; curl's stderr no longer breaks the `key=value` contract the skill parses. **3. A dead proxy is a silent, indefinite hang** — no output on either stream — and `/context-guru:status` cannot diagnose it, because invoking a skill needs a model call, which is the broken thing. New `check-proxy.sh` on `UserPromptSubmit`: it probes `/healthz`, tries to restart, and otherwise prints what to do. A hook is the only thing that runs without a model turn. It never blocks a prompt. **4. The `cache` preset advertised `context_guru_expand`.** Fixed in #141 (the gate belongs in the proxy); the docs and the install skill here no longer claim otherwise where they were wrong. **5. `settings.py` destroyed the user's undo, and uninstall did not restore what it replaced.** The backup stamp was second-granularity with an overwriting `copy2`, so an install→uninstall round trip wrote both backups to the same path and the survivor held the POST-install state — the value it existed to protect was gone from the file AND the backup. Now microsecond-stamped and created with `O_EXCL`. And `replaced` was reported then forgotten, so after a `--force` install over somebody's gateway, uninstall left them with no base URL at all; the replaced value is now recorded and restored. `is_ours` deserves a note. The review suggested matching `http://(127.0.0.1|localhost|[::1]):\\d+ /anthropic` as ours, to stop a port change reporting a conflict against context-guru itself. A test caught why that is wrong: litellm's default is `http://127.0.0.1:4000/anthropic`, so a URL-shape rule would let uninstall delete somebody else's routing. Two local proxies are indistinguishable by URL, so `add` records the URL it wrote and later runs read that record. Anything unrecorded stays a conflict — for both add and remove. **6. The atomic write widened a credential-bearing file's mode** from 600 to 644 under the common umask, and `os.replace` onto a symlinked `settings.json` replaced the LINK with a regular file, so a dotfile-managed setup silently never received the edit. Mode is preserved; the path is resolved first. - **`start-proxy.sh` printed a dead dashboard link** — it advertised `/dashboard/` and never passed `--dashboard`, so the first line the plugin ever prints was a 404. Now passed, with `--dashboard-db` under the state directory: the default would write `./context-guru-dashboard.db` into the user's repository. - **Backups accumulated forever** (one per add and per remove). Pruned to the newest 10. - **The zero-value cases are now stated** where a first-run user reads them, and `status` checks the one that is both commonest and previously undocumented: **outside a git repository** there is no environment snapshot, so `cachesplit` skips and the saving is exactly zero. The status skill also no longer treats `acted: 0` / `savings_pct: 0` as a verdict — those count content removal, and this component relocates a breakpoint. - **`--idle-exit`'s 24h is the plugin's value, not the flag's default** (which is 0 = never). Said so, along with probes not counting as activity. - Upgrade path documented (`CONTEXT_GURU_UPGRADE=1`, `CONTEXT_GURU_VERSION`). The scripts are tested from Go (`context-guru-plugin/plugin_test.go`) so `go test ./...` and CI cover them. Seven mutations, each proven to have landed before its result counted: backup() back to overwriting copy2 -> TestBackupsDoNotClobberEachOther FAIL "both operations reported the same backup path ..., so one overwrote the other" uninstall stops restoring -> TestUninstallRestoresTheBaseURLItReplaced FAIL restored="" want "https://gateway.corp.example/anthropic"; env left {ANTHROPIC_AUTH_TOKEN:keep} mode no longer preserved -> TestSettingsPreservesFileMode FAIL realpath removed -> TestSettingsFollowsASymlink FAIL checksum fail-open again -> TestInstallRefusesAnUnverifiedDownload FAIL port back in the environment -> TestHookMakesTheProxyIdentifiable FAIL pidfile no longer written -> TestHookMakesTheProxyIdentifiable FAIL One of those is worth recording as a process note: my first attempt at the backup mutation reverted only the timestamp granularity and left the `O_EXCL` retry loop in place, so the name was still unique and the test passed — proving nothing. Reverting half a fix is its own way to get a vacuous result. The run above restores the original function whole. Pre-existing coverage still passes: settings merge/conflict/removal/backup, and the hook's silence in unrouted projects, idempotence, non-failure when the binary is missing, and its wait for `/healthz`. **Still not verified end to end in a real Claude Code session**, because `install.sh` resolves a GitHub release and no tag has published assets yet. That is the first thing to do once #141 merges and a tag exists. Assisted-By: Claude Opus 5 (1M context) Signed-off-by: DAVID AMID --- .claude-plugin/marketplace.json | 21 + README.md | 16 +- .../.claude-plugin/plugin.json | 36 + context-guru-plugin/hooks/hooks.json | 26 + context-guru-plugin/plugin_test.go | 744 ++++++++++++++++++ context-guru-plugin/scripts/check-proxy.sh | 54 ++ context-guru-plugin/scripts/install.sh | 185 +++++ context-guru-plugin/scripts/settings.py | 295 +++++++ context-guru-plugin/scripts/start-proxy.sh | 117 +++ context-guru-plugin/skills/install/SKILL.md | 167 ++++ context-guru-plugin/skills/status/SKILL.md | 93 +++ context-guru-plugin/skills/uninstall/SKILL.md | 111 +++ docs/get-started/quickstart-proxy.md | 3 + docs/how-to/install-plugin.md | 147 ++++ docs/how-to/use-with-claude-code.md | 6 + docs/reference/presets.md | 2 +- mkdocs.yml | 1 + 17 files changed, 2022 insertions(+), 2 deletions(-) create mode 100644 .claude-plugin/marketplace.json create mode 100644 context-guru-plugin/.claude-plugin/plugin.json create mode 100644 context-guru-plugin/hooks/hooks.json create mode 100644 context-guru-plugin/plugin_test.go create mode 100755 context-guru-plugin/scripts/check-proxy.sh create mode 100755 context-guru-plugin/scripts/install.sh create mode 100755 context-guru-plugin/scripts/settings.py create mode 100755 context-guru-plugin/scripts/start-proxy.sh create mode 100644 context-guru-plugin/skills/install/SKILL.md create mode 100644 context-guru-plugin/skills/status/SKILL.md create mode 100644 context-guru-plugin/skills/uninstall/SKILL.md create mode 100644 docs/how-to/install-plugin.md diff --git a/.claude-plugin/marketplace.json b/.claude-plugin/marketplace.json new file mode 100644 index 00000000..9b419e80 --- /dev/null +++ b/.claude-plugin/marketplace.json @@ -0,0 +1,21 @@ +{ + "name": "context-guru", + "description": "Run context-guru in front of your Claude Code sessions in one command.", + "owner": { + "name": "rossoctl", + "url": "https://github.com/rossoctl/context-guru" + }, + "plugins": [ + { + "name": "context-guru", + "source": "./context-guru-plugin", + "displayName": "context-guru", + "description": "Install, route, inspect and remove a local context-guru proxy for Claude Code. Recovers prompt-cache misses on long sessions; no API key needed on a Pro/Max subscription.", + "homepage": "https://github.com/rossoctl/context-guru", + "repository": "https://github.com/rossoctl/context-guru", + "license": "Apache-2.0", + "keywords": ["cache", "cost", "proxy", "tokens", "prompt-caching"], + "category": "productivity" + } + ] +} diff --git a/README.md b/README.md index 2d1db4fc..f5032db5 100644 --- a/README.md +++ b/README.md @@ -108,7 +108,21 @@ docker build -t context-guru:local . ## Quickstart (60 seconds) -Download a release binary — statically linked, **no Go and no C compiler needed** — or build +**Claude Code users — two commands, no toolchain, and no API key needed on a Pro/Max +subscription** ([details](docs/how-to/install-plugin.md)): + +``` +/plugin marketplace add rossoctl/context-guru +/plugin install context-guru@context-guru +/context-guru:install +``` + +That installs a statically-linked binary (no Go, no C compiler), routes **this project only** by +default, starts the proxy on demand and lets it exit when idle. `/context-guru:uninstall` undoes it, +restoring any base URL it replaced. The plugin installs with `--preset cache` — the prompt-cache +split and nothing else. (The proxy's own default is `house`; `--preset` is how you change it.) + +Or by hand — a release binary is statically linked, **no Go and no C compiler needed** — or build from source: ```sh diff --git a/context-guru-plugin/.claude-plugin/plugin.json b/context-guru-plugin/.claude-plugin/plugin.json new file mode 100644 index 00000000..fb082d0a --- /dev/null +++ b/context-guru-plugin/.claude-plugin/plugin.json @@ -0,0 +1,36 @@ +{ + "name": "context-guru", + "displayName": "context-guru", + "description": "Installs a local context-guru proxy, routes this project's Claude Code sessions through it, and reports what it saved. Reversible: /context-guru:uninstall removes the one settings key it adds and stops the proxy.", + "version": "0.1.0", + "author": { + "name": "rossoctl", + "url": "https://github.com/rossoctl/context-guru" + }, + "homepage": "https://github.com/rossoctl/context-guru", + "repository": "https://github.com/rossoctl/context-guru", + "license": "Apache-2.0", + "keywords": ["cache", "cost", "proxy", "tokens", "prompt-caching"], + "userConfig": { + "port": { + "type": "number", + "title": "Proxy port", + "description": "Local port the proxy listens on. The port must be FIXED rather than negotiated: the ANTHROPIC_BASE_URL written into your settings and the session hook that starts the proxy have to agree, and a URL cannot be renegotiated after it is written. Default 8787 — deliberately not 4000, which collides with litellm.", + "default": 8787, + "min": 1024, + "max": 65535 + }, + "preset": { + "type": "string", + "title": "Preset", + "description": "Compaction pipeline. `cache` (the default) runs the prompt-cache split and NOTHING else: no content dropped, no markers, no extra tool, no model calls. `codesmart` adds the offloaders once you want them.", + "default": "cache" + }, + "idle_exit": { + "type": "string", + "title": "Idle exit", + "description": "Exit the proxy after this long with no requests and no keep-alive ping pending, so nothing is left running on your machine. Must be at least 2x the store's entry lifetime (~5h34m at the default), because exiting clears in-memory cache state.", + "default": "24h" + } + } +} diff --git a/context-guru-plugin/hooks/hooks.json b/context-guru-plugin/hooks/hooks.json new file mode 100644 index 00000000..72295d76 --- /dev/null +++ b/context-guru-plugin/hooks/hooks.json @@ -0,0 +1,26 @@ +{ + "hooks": { + "UserPromptSubmit": [ + { + "hooks": [ + { + "type": "command", + "command": "\"${CLAUDE_PLUGIN_ROOT}\"/scripts/check-proxy.sh", + "timeout": 10 + } + ] + } + ], + "SessionStart": [ + { + "hooks": [ + { + "type": "command", + "command": "\"${CLAUDE_PLUGIN_ROOT}\"/scripts/start-proxy.sh", + "timeout": 60 + } + ] + } + ] + } +} diff --git a/context-guru-plugin/plugin_test.go b/context-guru-plugin/plugin_test.go new file mode 100644 index 00000000..6c12bf66 --- /dev/null +++ b/context-guru-plugin/plugin_test.go @@ -0,0 +1,744 @@ +// Package plugin holds tests for the Claude Code plugin's shell/Python helpers. +// +// The scripts are not Go, but their failure modes are the most expensive in this repo: they +// edit the user's real settings.json, and the SessionStart hook runs in EVERY project the user +// has. A regression here does not degrade compaction — it breaks Claude Code on a stranger's +// machine, in projects that have nothing to do with context-guru. So they are tested from Go, +// where `go test ./...` and CI already look. +package plugin + +import ( + "encoding/json" + "fmt" + "net" + "net/http" + "os" + "os/exec" + "path/filepath" + "runtime" + "strconv" + "strings" + "syscall" + "testing" + "time" +) + +func scriptsDir(t *testing.T) string { + t.Helper() + abs, err := filepath.Abs("scripts") + if err != nil { + t.Fatal(err) + } + if _, err := os.Stat(abs); err != nil { + t.Fatalf("plugin scripts missing: %v", err) + } + return abs +} + +func requireTool(t *testing.T, name string) string { + t.Helper() + p, err := exec.LookPath(name) + if err != nil { + t.Skipf("%s not available: %v", name, err) + } + return p +} + +// settings runs settings.py and returns its key=value output as a map, plus the exit code. +func settings(t *testing.T, args ...string) (map[string]string, int) { + t.Helper() + py := requireTool(t, "python3") + cmd := exec.Command(py, append([]string{filepath.Join(scriptsDir(t), "settings.py")}, args...)...) + out, err := cmd.CombinedOutput() + code := 0 + if ee, ok := err.(*exec.ExitError); ok { + code = ee.ExitCode() + } else if err != nil { + t.Fatalf("running settings.py: %v (%s)", err, out) + } + facts := map[string]string{} + for _, line := range strings.Split(string(out), "\n") { + if k, v, ok := strings.Cut(strings.TrimSpace(line), "="); ok { + facts[k] = v + } + } + t.Logf("settings.py %v -> exit %d, %v", args, code, facts) + return facts, code +} + +func writeJSON(t *testing.T, path string, v any) { + t.Helper() + b, err := json.MarshalIndent(v, "", " ") + if err != nil { + t.Fatal(err) + } + if err := os.WriteFile(path, b, 0o644); err != nil { + t.Fatal(err) + } +} + +func readJSON(t *testing.T, path string) map[string]any { + t.Helper() + b, err := os.ReadFile(path) + if err != nil { + t.Fatal(err) + } + var m map[string]any + if err := json.Unmarshal(b, &m); err != nil { + t.Fatalf("settings file is not valid JSON after the edit: %v\n%s", err, b) + } + return m +} + +const ourURL = "http://127.0.0.1:8787/anthropic" + +// TestSettingsAddPreservesEverythingElse is the whole reason a script does this rather than a +// one-line `jq`: the target is a file the user depends on, holding their theme, model, +// permission rules and their own env vars. Exactly one key may appear, and nothing may be lost. +func TestSettingsAddPreservesEverythingElse(t *testing.T) { + dir := t.TempDir() + path := filepath.Join(dir, "settings.json") + writeJSON(t, path, map[string]any{ + "theme": "dark", + "model": "opus", + "env": map[string]any{ + "SOME_OTHER_VAR": "keep me", + "ANTHROPIC_SMALL_FAST": "also keep me", + }, + "permissions": map[string]any{"allow": []string{"Bash(ls:*)"}}, + }) + + facts, code := settings(t, "add", "--file", path, "--url", ourURL) + if code != 0 || facts["result"] != "added" { + t.Fatalf("add failed: exit %d, %v", code, facts) + } + if facts["backup"] == "" || facts["backup"] == "(new file)" { + t.Errorf("no backup was taken of an existing settings file: %v", facts) + } else if _, err := os.Stat(facts["backup"]); err != nil { + t.Errorf("reported backup %q does not exist: %v", facts["backup"], err) + } + + got := readJSON(t, path) + if got["theme"] != "dark" || got["model"] != "opus" { + t.Errorf("top-level settings were lost: %v", got) + } + if got["permissions"] == nil { + t.Error("permissions block was lost") + } + env, _ := got["env"].(map[string]any) + if env["ANTHROPIC_BASE_URL"] != ourURL { + t.Errorf("env.ANTHROPIC_BASE_URL = %v, want %q", env["ANTHROPIC_BASE_URL"], ourURL) + } + if env["SOME_OTHER_VAR"] != "keep me" || env["ANTHROPIC_SMALL_FAST"] != "also keep me" { + t.Errorf("the user's own env vars were lost: %v", env) + } + if len(env) != 3 { + t.Errorf("env has %d keys, want the 2 originals plus ours: %v", len(env), env) + } +} + +// TestSettingsAddRefusesToStealAnExistingBaseURL covers the one conflict the install has to +// reason about. A base URL already in the file may be the user's company gateway or a benchmark +// endpoint; taking it over would break their setup while reporting success. +func TestSettingsAddRefusesToStealAnExistingBaseURL(t *testing.T) { + dir := t.TempDir() + path := filepath.Join(dir, "settings.json") + theirs := "https://gateway.corp.example/anthropic" + writeJSON(t, path, map[string]any{"env": map[string]any{"ANTHROPIC_BASE_URL": theirs}}) + + facts, code := settings(t, "add", "--file", path, "--url", ourURL) + if code != 2 || facts["result"] != "conflict" { + t.Fatalf("expected a conflict (exit 2), got exit %d, %v", code, facts) + } + if facts["existing"] != theirs { + t.Errorf("conflict did not report the existing value: %v", facts) + } + if env := readJSON(t, path)["env"].(map[string]any); env["ANTHROPIC_BASE_URL"] != theirs { + t.Fatalf("the file was modified despite the conflict: %v", env) + } + + // --force is the user's explicit decision, and it must report what it replaced so the old + // value is recoverable from the transcript as well as the backup. + facts, code = settings(t, "add", "--file", path, "--url", ourURL, "--force") + if code != 0 || facts["result"] != "added" || facts["replaced"] != theirs { + t.Fatalf("--force did not replace and report: exit %d, %v", code, facts) + } + + // Re-adding the same URL is a no-op, so a re-run of the install skill is free. + facts, code = settings(t, "add", "--file", path, "--url", ourURL) + if code != 0 || facts["result"] != "unchanged" { + t.Fatalf("re-adding the same URL should be unchanged: exit %d, %v", code, facts) + } +} + +// TestSettingsRemoveTakesOnlyOurKey: uninstall must be exact. It removes our base URL and +// nothing else, refuses to remove one that is not ours, and leaves no empty `env: {}` behind. +func TestSettingsRemoveTakesOnlyOurKey(t *testing.T) { + dir := t.TempDir() + + // (a) our key alongside the user's own env vars. + path := filepath.Join(dir, "a.json") + writeJSON(t, path, map[string]any{"theme": "dark", "env": map[string]any{ + "ANTHROPIC_BASE_URL": ourURL, "KEEP": "yes"}}) + facts, code := settings(t, "remove", "--file", path, "--url", ourURL) + if code != 0 || facts["result"] != "removed" { + t.Fatalf("remove failed: exit %d, %v", code, facts) + } + got := readJSON(t, path) + env, _ := got["env"].(map[string]any) + if _, still := env["ANTHROPIC_BASE_URL"]; still { + t.Error("the key survived removal") + } + if env["KEEP"] != "yes" || got["theme"] != "dark" { + t.Errorf("removal took more than its own key: %v", got) + } + + // (b) our key alone: the env block we created goes with it, leaving no litter. + path = filepath.Join(dir, "b.json") + writeJSON(t, path, map[string]any{"theme": "dark", "env": map[string]any{ + "ANTHROPIC_BASE_URL": ourURL}}) + if _, code := settings(t, "remove", "--file", path, "--url", ourURL); code != 0 { + t.Fatalf("remove exit %d", code) + } + if got := readJSON(t, path); got["env"] != nil { + t.Errorf("an empty env block was left behind: %v", got) + } + + // (c) a base URL that is NOT ours must survive an uninstall untouched. + path = filepath.Join(dir, "c.json") + theirs := "http://127.0.0.1:4000/anthropic" // e.g. litellm + writeJSON(t, path, map[string]any{"env": map[string]any{"ANTHROPIC_BASE_URL": theirs}}) + facts, code = settings(t, "remove", "--file", path, "--url", ourURL) + if code != 2 || facts["result"] != "conflict" { + t.Fatalf("uninstall must not remove a base URL it did not install: exit %d, %v", code, facts) + } + if env := readJSON(t, path)["env"].(map[string]any); env["ANTHROPIC_BASE_URL"] != theirs { + t.Fatalf("someone else's base URL was removed: %v", env) + } +} + +// TestSettingsRefusesToRewriteABrokenFile: if the file will not parse, the only safe move is to +// stop. Treating it as empty and writing a fresh one would discard every setting in it. +func TestSettingsRefusesToRewriteABrokenFile(t *testing.T) { + dir := t.TempDir() + path := filepath.Join(dir, "settings.json") + broken := "{\n \"theme\": \"dark\",,,\n}\n" + if err := os.WriteFile(path, []byte(broken), 0o644); err != nil { + t.Fatal(err) + } + facts, code := settings(t, "add", "--file", path, "--url", ourURL) + if code != 3 || facts["reason"] != "unparseable_json" { + t.Fatalf("expected a refusal on unparseable JSON, got exit %d, %v", code, facts) + } + b, _ := os.ReadFile(path) + if string(b) != broken { + t.Fatalf("the broken file was modified:\n%s", b) + } +} + +// --- the SessionStart hook ----------------------------------------------------------------- + +// runStart runs start-proxy.sh with a controlled environment and returns its output. +// +// CONTEXT_GURU_BIN points at a sentinel script: if the hook decides to launch a proxy, the +// sentinel file appears. That is how "did it start something?" is asserted, rather than by +// looking for a process. +func runStart(t *testing.T, env map[string]string) (out string, code int, startedSentinel string) { + t.Helper() + requireTool(t, "bash") + dir := t.TempDir() + sentinel := filepath.Join(dir, "started") + fake := filepath.Join(dir, "fake-proxy") + if err := os.WriteFile(fake, []byte("#!/usr/bin/env bash\ntouch \""+sentinel+"\"\nsleep 30\n"), 0o755); err != nil { + t.Fatal(err) + } + + cmd := exec.Command("bash", filepath.Join(scriptsDir(t), "start-proxy.sh")) + cmd.Env = append(os.Environ(), "CONTEXT_GURU_BIN="+fake, "TMPDIR="+dir) + for k, v := range env { + cmd.Env = append(cmd.Env, k+"="+v) + } + b, err := cmd.CombinedOutput() + if ee, ok := err.(*exec.ExitError); ok { + code = ee.ExitCode() + } else if err != nil { + t.Fatalf("running start-proxy.sh: %v (%s)", err, b) + } + t.Logf("start-proxy.sh env=%v -> exit %d, output:\n%s", env, code, b) + return string(b), code, sentinel +} + +// TestHookIsSilentAndInertWhereRoutingIsNotConfigured is the property that makes a user-scope +// plugin acceptable at all. +// +// The plugin installs globally, so this hook runs on EVERY session in EVERY project — including +// all the ones the user never routed. In those it must do nothing and say nothing: starting a +// proxy would be waste, and printing anything would put context-guru noise in sessions that have +// nothing to do with it. It also must not hijack a user who routes to a different local proxy on +// another port, which is why the gate matches the port and not merely "localhost". +func TestHookIsSilentAndInertWhereRoutingIsNotConfigured(t *testing.T) { + for _, c := range []struct{ name, baseURL string }{ + {"unset", ""}, + {"another local proxy on a different port (e.g. litellm)", "http://localhost:4000/anthropic"}, + {"a remote gateway", "https://gateway.corp.example/anthropic"}, + {"our port number appearing in a REMOTE host", "https://8787.example.com/anthropic"}, + } { + t.Run(c.name, func(t *testing.T) { + env := map[string]string{"CLAUDE_PLUGIN_OPTION_PORT": "8787"} + if c.baseURL != "" { + env["ANTHROPIC_BASE_URL"] = c.baseURL + } else { + env["ANTHROPIC_BASE_URL"] = "" + } + out, code, sentinel := runStart(t, env) + if code != 0 { + t.Errorf("exit %d; the hook must never fail a session", code) + } + if strings.TrimSpace(out) != "" { + t.Errorf("the hook printed output in an unrouted project: %q", out) + } + if _, err := os.Stat(sentinel); err == nil { + t.Error("the hook started a proxy in a project that is not routed to it") + } + }) + } +} + +// TestHookIsIdempotentWhenTheProxyIsAlreadyUp: SessionStart also fires on clear, compact, +// resume and fork, so a long session re-runs this repeatedly. A second proxy must never be +// launched — it would fail to bind, or worse, bind a different port and split the state. +func TestHookIsIdempotentWhenTheProxyIsAlreadyUp(t *testing.T) { + ln, err := net.Listen("tcp", "127.0.0.1:0") + if err != nil { + t.Fatal(err) + } + defer ln.Close() + mux := http.NewServeMux() + mux.HandleFunc("/healthz", func(w http.ResponseWriter, _ *http.Request) { w.Write([]byte("ok")) }) + srv := &http.Server{Handler: mux, ReadHeaderTimeout: 5 * time.Second} + go srv.Serve(ln) //nolint:errcheck // returns ErrServerClosed on Close + defer srv.Close() + port := fmt.Sprint(ln.Addr().(*net.TCPAddr).Port) + + out, code, sentinel := runStart(t, map[string]string{ + "CLAUDE_PLUGIN_OPTION_PORT": port, + "ANTHROPIC_BASE_URL": "http://127.0.0.1:" + port + "/anthropic", + }) + if code != 0 { + t.Errorf("exit %d, output %q", code, out) + } + if _, err := os.Stat(sentinel); err == nil { + t.Error("a second proxy was started even though /healthz already answered") + } + if strings.TrimSpace(out) != "" { + t.Errorf("nothing to do, but the hook printed %q", out) + } +} + +// TestHookNeverFailsTheSessionWhenTheBinaryIsMissing: routed, but the binary is gone (the user +// deleted it, or PATH differs under the hook). The session must still start, with an +// explanation — a hook that exits non-zero here is a plugin that can brick every session on the +// machine, which is the biggest risk in this whole feature. +func TestHookNeverFailsTheSessionWhenTheBinaryIsMissing(t *testing.T) { + requireTool(t, "bash") + dir := t.TempDir() + cmd := exec.Command("bash", filepath.Join(scriptsDir(t), "start-proxy.sh")) + // An unused high port: nothing answers /healthz, and the binary does not exist. + cmd.Env = append(os.Environ(), + "CLAUDE_PLUGIN_OPTION_PORT=8799", + "ANTHROPIC_BASE_URL=http://127.0.0.1:8799/anthropic", + "CONTEXT_GURU_BIN="+filepath.Join(dir, "does-not-exist"), + "TMPDIR="+dir) + b, err := cmd.CombinedOutput() + code := 0 + if ee, ok := err.(*exec.ExitError); ok { + code = ee.ExitCode() + } else if err != nil { + t.Fatal(err) + } + if code != 0 { + t.Fatalf("exit %d — the hook must never fail a session: %s", code, b) + } + out := string(b) + for _, want := range []string{"not on PATH", "/context-guru:install"} { + if !strings.Contains(out, want) { + t.Errorf("the explanation omits %q:\n%s", want, out) + } + } +} + +// TestHookStartsTheProxyAndWaitsForHealthz is the positive path, and specifically the WAIT: the +// hook is synchronous on purpose, so the session's first API request cannot beat the proxy up. +// A hook that returned before /healthz answered would leave that race in place. +func TestHookStartsTheProxyAndWaitsForHealthz(t *testing.T) { + requireTool(t, "bash") + py := requireTool(t, "python3") + if runtime.GOOS == "windows" { + t.Skip("shell hook is POSIX-only") + } + ln, err := net.Listen("tcp", "127.0.0.1:0") + if err != nil { + t.Fatal(err) + } + port := fmt.Sprint(ln.Addr().(*net.TCPAddr).Port) + ln.Close() // free it for the fake proxy to bind + + dir := t.TempDir() + // A stand-in proxy: takes ~1s to come up, then answers /healthz. The delay is the point — + // it is what a hook that does not wait would skip past. + fake := filepath.Join(dir, "fake-proxy") + script := "#!/usr/bin/env bash\nsleep 1\nexec " + py + " -c '\n" + + "import http.server\n" + + "class H(http.server.BaseHTTPRequestHandler):\n" + + " def do_GET(self):\n" + + " self.send_response(200); self.end_headers(); self.wfile.write(b\"ok\")\n" + + " def log_message(self, *a): pass\n" + + "http.server.HTTPServer((\"127.0.0.1\", " + port + "), H).serve_forever()\n'\n" + if err := os.WriteFile(fake, []byte(script), 0o755); err != nil { + t.Fatal(err) + } + + cmd := exec.Command("bash", filepath.Join(scriptsDir(t), "start-proxy.sh")) + cmd.Env = append(os.Environ(), + "CLAUDE_PLUGIN_OPTION_PORT="+port, + "ANTHROPIC_BASE_URL=http://127.0.0.1:"+port+"/anthropic", + "CONTEXT_GURU_BIN="+fake, + "TMPDIR="+dir) + b, err := cmd.CombinedOutput() + t.Cleanup(func() { exec.Command("pkill", "-f", "127.0.0.1\", "+port).Run() }) //nolint:errcheck + if err != nil { + t.Fatalf("start-proxy.sh failed: %v\n%s", err, b) + } + if !strings.Contains(string(b), "proxy up on 127.0.0.1:"+port) { + t.Fatalf("the hook returned without reporting a healthy proxy:\n%s", b) + } + // The claim is that it returned only AFTER /healthz answered, so it must answer now. + resp, err := http.Get("http://127.0.0.1:" + port + "/healthz") + if err != nil { + t.Fatalf("the hook reported the proxy up, but /healthz does not answer: %v", err) + } + resp.Body.Close() +} + +// --- fixes from the review of #141 --------------------------------------------------------- + +// TestBackupsDoNotClobberEachOther is the defect that destroyed the user's undo. +// +// The stamp was second-granularity with a plain copy2, so an install-then-uninstall round trip — +// well inside one second — wrote both backups to the SAME filename. The survivor held the +// POST-install state, and the install skill tells the user to keep that path as their undo. The +// value it was supposed to protect was gone from both the file and the backup. +func TestBackupsDoNotClobberEachOther(t *testing.T) { + dir := t.TempDir() + path := filepath.Join(dir, "settings.json") + theirs := "https://gateway.corp.example/anthropic" + writeJSON(t, path, map[string]any{"env": map[string]any{"ANTHROPIC_BASE_URL": theirs}}) + + // Back to back, deliberately: the bug needed only that both land in the same second. + add, code := settings(t, "add", "--file", path, "--url", ourURL, "--force") + if code != 0 { + t.Fatalf("add: exit %d, %v", code, add) + } + rm, code := settings(t, "remove", "--file", path, "--url", ourURL) + if code != 0 { + t.Fatalf("remove: exit %d, %v", code, rm) + } + if add["backup"] == rm["backup"] { + t.Fatalf("both operations reported the same backup path %q, so one overwrote the other", + add["backup"]) + } + // The install backup must still hold what was there BEFORE we touched it. + b, err := os.ReadFile(add["backup"]) + if err != nil { + t.Fatalf("the install backup is gone: %v", err) + } + if !strings.Contains(string(b), theirs) { + t.Errorf("the install backup does not contain the value it was meant to preserve:\n%s", b) + } +} + +// TestUninstallRestoresTheBaseURLItReplaced: after a --force install over somebody's own gateway, +// uninstall must hand it back. Deleting the key left them with NO base URL at all — a worse state +// than before they installed, and (with the backup defect above) unrecoverable from anything the +// tool produced. +func TestUninstallRestoresTheBaseURLItReplaced(t *testing.T) { + dir := t.TempDir() + path := filepath.Join(dir, "settings.json") + theirs := "https://gateway.corp.example/anthropic" + writeJSON(t, path, map[string]any{"env": map[string]any{ + "ANTHROPIC_BASE_URL": theirs, "ANTHROPIC_AUTH_TOKEN": "keep"}}) + + if _, code := settings(t, "add", "--file", path, "--url", ourURL, "--force"); code != 0 { + t.Fatal("add --force failed") + } + facts, code := settings(t, "remove", "--file", path, "--url", ourURL) + if code != 0 { + t.Fatalf("remove: exit %d, %v", code, facts) + } + if facts["restored"] != theirs { + t.Errorf("remove reported restored=%q, want %q", facts["restored"], theirs) + } + env, _ := readJSON(t, path)["env"].(map[string]any) + if env["ANTHROPIC_BASE_URL"] != theirs { + t.Fatalf("the user's own base URL was not restored: %v", env) + } + if env["ANTHROPIC_AUTH_TOKEN"] != "keep" { + t.Errorf("an unrelated env var was lost: %v", env) + } + // And no bookkeeping left behind. + if _, ok := readJSON(t, path)["$context-guru"]; ok { + t.Errorf("uninstall left its own bookkeeping key in the user's settings") + } +} + +// TestSettingsPreservesFileMode: the file holds a credential often enough that widening its mode +// is a real leak. The temp file is created fresh, so os.replace took the UMASK mode rather than +// the replaced file's — a 600 settings file came back 644 under the common default. +func TestSettingsPreservesFileMode(t *testing.T) { + if runtime.GOOS == "windows" { + t.Skip("POSIX modes only") + } + dir := t.TempDir() + path := filepath.Join(dir, "settings.json") + writeJSON(t, path, map[string]any{"env": map[string]any{"ANTHROPIC_AUTH_TOKEN": "secret"}}) + if err := os.Chmod(path, 0o600); err != nil { + t.Fatal(err) + } + if _, code := settings(t, "add", "--file", path, "--url", ourURL); code != 0 { + t.Fatal("add failed") + } + fi, err := os.Stat(path) + if err != nil { + t.Fatal(err) + } + if got := fi.Mode().Perm(); got != 0o600 { + t.Errorf("mode after add = %o, want 600: this file holds a credential", got) + } +} + +// TestSettingsFollowsASymlink: a dotfile-managed settings.json is commonly a symlink into a +// repository. os.replace onto the link path replaces the LINK with a regular file, so the edit +// never reaches the file the user manages and their dotfiles still hold the old content — while +// the tool reports success. +func TestSettingsFollowsASymlink(t *testing.T) { + if runtime.GOOS == "windows" { + t.Skip("symlink semantics differ") + } + dir := t.TempDir() + real := filepath.Join(dir, "dotfiles", "settings.json") + if err := os.MkdirAll(filepath.Dir(real), 0o755); err != nil { + t.Fatal(err) + } + writeJSON(t, real, map[string]any{"theme": "dark"}) + link := filepath.Join(dir, "settings.json") + if err := os.Symlink(real, link); err != nil { + t.Skipf("cannot symlink here: %v", err) + } + + if _, code := settings(t, "add", "--file", link, "--url", ourURL); code != 0 { + t.Fatal("add failed") + } + fi, err := os.Lstat(link) + if err != nil { + t.Fatal(err) + } + if fi.Mode()&os.ModeSymlink == 0 { + t.Error("the symlink was replaced by a regular file, so the user's dotfiles repo never saw the edit") + } + env, _ := readJSON(t, real)["env"].(map[string]any) + if env["ANTHROPIC_BASE_URL"] != ourURL { + t.Errorf("the edit did not reach the real file: %v", readJSON(t, real)) + } +} + +// TestSettingsRecognisesItsOwnURLOnAnotherPort: changing the configured port and re-running install +// used to report a conflict against context-guru itself, telling the user something else owned +// their routing. +func TestSettingsRecognisesItsOwnURLOnAnotherPort(t *testing.T) { + dir := t.TempDir() + path := filepath.Join(dir, "settings.json") + // The state a previous install leaves: the URL it wrote, recorded. + writeJSON(t, path, map[string]any{ + "env": map[string]any{"ANTHROPIC_BASE_URL": "http://localhost:9999/anthropic"}, + "$context-guru": map[string]any{"installed_base_url": "http://localhost:9999/anthropic"}, + }) + + facts, code := settings(t, "add", "--file", path, "--url", ourURL) + if code != 0 || facts["result"] != "repointed" { + t.Fatalf("expected a clean repoint, got exit %d, %v", code, facts) + } + if env, _ := readJSON(t, path)["env"].(map[string]any); env["ANTHROPIC_BASE_URL"] != ourURL { + t.Errorf("not repointed: %v", env) + } + // Anything we did NOT record stays a conflict — including another LOCAL proxy, which is the + // case a URL-shape rule got wrong: litellm's default is http://127.0.0.1:4000/anthropic, and + // treating that as ours would have let uninstall delete somebody else's routing. + for _, theirs := range []string{ + "https://8787.example.com/anthropic", // remote host that merely contains our port + "http://127.0.0.1:4000/anthropic", // another local proxy (litellm's default) + } { + p2 := filepath.Join(dir, "conflict.json") + writeJSON(t, p2, map[string]any{"env": map[string]any{"ANTHROPIC_BASE_URL": theirs}}) + facts, code = settings(t, "add", "--file", p2, "--url", ourURL) + if code != 2 || facts["result"] != "conflict" { + t.Errorf("%s must be a conflict, not ours: exit %d, %v", theirs, code, facts) + } + if _, code := settings(t, "remove", "--file", p2, "--url", ourURL); code != 2 { + t.Errorf("uninstall must refuse to remove %s", theirs) + } + } +} + +// TestInstallRefusesAnUnverifiedDownload is the security fix, and it is the one to keep. +// +// A checksum MISMATCH was fatal, but an absent or unfetchable checksums.txt printed one advisory +// line and fell through to `install -m 755`. An unverified binary landed on a PATH directory and +// ran — a binary that handles all of the user's LLM traffic and holds their API key. The script's +// own comment said "a failure here is fatal, never a warning" while the code did the opposite. +func TestInstallRefusesAnUnverifiedDownload(t *testing.T) { + requireTool(t, "bash") + dir := t.TempDir() + + // A stub `curl` that serves a tarball and 404s the checksum file — exactly the shape of a + // release whose checksums.txt is missing. + bin := filepath.Join(dir, "bin") + if err := os.MkdirAll(bin, 0o755); err != nil { + t.Fatal(err) + } + payload := filepath.Join(dir, "context-guru-proxy") + if err := os.WriteFile(payload, []byte("#!/bin/sh\necho THIS BINARY WAS NEVER VERIFIED\n"), 0o755); err != nil { + t.Fatal(err) + } + tarball := filepath.Join(dir, "payload.tar.gz") + if out, err := exec.Command("tar", "czf", tarball, "-C", dir, "context-guru-proxy").CombinedOutput(); err != nil { + t.Fatalf("tar: %v (%s)", err, out) + } + stub := "#!/usr/bin/env bash\n" + + "# args end with the URL; -o gives the destination\n" + + "dest=\"\"; url=\"\"\n" + + "while [ $# -gt 0 ]; do case \"$1\" in -o) dest=$2; shift 2;; -*) shift;; *) url=$1; shift;; esac; done\n" + + "case \"$url\" in\n" + + " *checksums.txt) exit 22;;\n" + + " *api.github.com*) printf '{\"tag_name\": \"v9.9.9\"}' ${dest:+> \"$dest\"}; exit 0;;\n" + + " *.tar.gz) cp " + tarball + " \"$dest\"; exit 0;;\n" + + "esac\nexit 22\n" + if err := os.WriteFile(filepath.Join(bin, "curl"), []byte(stub), 0o755); err != nil { + t.Fatal(err) + } + + dest := filepath.Join(dir, "dest") + cmd := exec.Command("bash", filepath.Join(scriptsDir(t), "install.sh")) + cmd.Env = append(os.Environ(), + "PATH="+bin+":"+os.Getenv("PATH"), + "CONTEXT_GURU_DEST="+dest, + "HOME="+dir, + ) + out, err := cmd.CombinedOutput() + t.Logf("install.sh output:\n%s", out) + if err == nil { + t.Error("install.sh succeeded without verifying the download") + } + if !strings.Contains(string(out), "checksum_unavailable") { + t.Errorf("the refusal does not name the reason: %s", out) + } + if _, err := os.Stat(filepath.Join(dest, "context-guru-proxy")); err == nil { + t.Fatal("an unverified binary was installed onto a PATH directory") + } +} + +// TestHookMakesTheProxyIdentifiable is the other half of the uninstall fix. +// +// The uninstall skill used to stop the proxy with `pkill -f "context-guru-proxy.*$PORT"`, which +// could not work: the starter passed the port through LISTEN_ADDR in the environment, so it +// appeared nowhere in the proxy's command line. The pattern matched no proxy — and did match the +// shell running it, i.e. the Bash tool of the user's own session, killing it mid-command while the +// proxy kept the port. +// +// So the starter now has to leave two handles behind, and this asserts both: +// +// 1. the port in `argv`, so `ps` and a human can tell instances apart; +// 2. a pidfile, which is what uninstall actually uses — no pattern matching at all. +func TestHookMakesTheProxyIdentifiable(t *testing.T) { + requireTool(t, "bash") + dir := t.TempDir() + + // A stand-in proxy that records its own argv and then holds the port, so the starter's + // health probe succeeds and the script runs to completion. + argvFile := filepath.Join(dir, "argv") + py := requireTool(t, "python3") + port := freePort(t) + fake := filepath.Join(dir, "fake-proxy") + script := "#!/usr/bin/env bash\n" + + "printf '%s\\n' \"$*\" > " + argvFile + "\n" + + "exec " + py + " -c '\n" + + "import http.server\n" + + "class H(http.server.BaseHTTPRequestHandler):\n" + + " def do_GET(self):\n" + + " self.send_response(200); self.end_headers(); self.wfile.write(b\"ok\")\n" + + " def log_message(self, *a): pass\n" + + "http.server.HTTPServer((\"127.0.0.1\", " + port + "), H).serve_forever()\n'\n" + if err := os.WriteFile(fake, []byte(script), 0o755); err != nil { + t.Fatal(err) + } + + state := filepath.Join(dir, "state") + cmd := exec.Command("bash", filepath.Join(scriptsDir(t), "start-proxy.sh")) + cmd.Env = append(os.Environ(), + "CLAUDE_PLUGIN_OPTION_PORT="+port, + "ANTHROPIC_BASE_URL=http://127.0.0.1:"+port+"/anthropic", + "CONTEXT_GURU_BIN="+fake, + "XDG_STATE_HOME="+state, + "TMPDIR="+dir) + out, err := cmd.CombinedOutput() + t.Logf("start-proxy.sh:\n%s", out) + t.Cleanup(func() { + if b, e := os.ReadFile(filepath.Join(state, "context-guru", "proxy-"+port+".pid")); e == nil { + exec.Command("kill", strings.TrimSpace(string(b))).Run() //nolint:errcheck + } + }) + if err != nil { + t.Fatalf("start-proxy.sh failed: %v", err) + } + + // (1) the port is on the command line. + argv, err := os.ReadFile(argvFile) + if err != nil { + t.Fatalf("the fake proxy never ran: %v", err) + } + if !strings.Contains(string(argv), "--listen") || !strings.Contains(string(argv), port) { + t.Errorf("the proxy's argv does not carry its port (%q); nothing can identify this "+ + "instance among others, which is what made the old pkill pattern match the caller's "+ + "own shell instead", strings.TrimSpace(string(argv))) + } + // The dashboard must not be written into whatever directory the proxy started in — that is + // the user's repository. + if !strings.Contains(string(argv), "--dashboard-db") { + t.Errorf("no explicit --dashboard-db, so the database lands in the current directory: %q", argv) + } + + // (2) the pidfile exists, names a live process, and that process is ours. + pidfile := filepath.Join(state, "context-guru", "proxy-"+port+".pid") + b, err := os.ReadFile(pidfile) + if err != nil { + t.Fatalf("no pidfile at %s: uninstall has no handle but a pattern match: %v", pidfile, err) + } + pid, err := strconv.Atoi(strings.TrimSpace(string(b))) + if err != nil || pid <= 0 { + t.Fatalf("pidfile does not contain a pid: %q", b) + } + if err := syscall.Kill(pid, 0); err != nil { + t.Errorf("pidfile names pid %d, which is not running: %v", pid, err) + } +} + +// freePort asks the kernel for a port and gives it back, so the fake proxy can bind it. +func freePort(t *testing.T) string { + t.Helper() + ln, err := net.Listen("tcp", "127.0.0.1:0") + if err != nil { + t.Fatal(err) + } + p := fmt.Sprint(ln.Addr().(*net.TCPAddr).Port) + ln.Close() + return p +} diff --git a/context-guru-plugin/scripts/check-proxy.sh b/context-guru-plugin/scripts/check-proxy.sh new file mode 100755 index 00000000..163bee3f --- /dev/null +++ b/context-guru-plugin/scripts/check-proxy.sh @@ -0,0 +1,54 @@ +#!/usr/bin/env bash +# UserPromptSubmit hook: if this project is routed and the proxy is NOT answering, say so before +# the request goes out. +# +# This exists because of the worst failure mode in the whole plugin: with routing configured and no +# proxy listening, a prompt produces **nothing at all**. No error on stdout, no error on stderr, +# no timeout the user can interpret — the session simply hangs. That is the state after any crash, +# any reboot, and after every `--idle-exit`. +# +# `/context-guru:status` diagnoses it correctly and cannot be reached: invoking a skill needs Claude +# to respond, which needs an API call, which is the broken thing. A hook is the only thing left +# that runs without a model turn, and UserPromptSubmit is the last moment before the request. +# +# Every exit is 0 and this never blocks a prompt. It is a note, not a gate: the user may be about to +# ask something that does not need the API, and a hook that refuses prompts would be a worse +# failure than the one it reports. +set -uo pipefail + +PORT="${CLAUDE_PLUGIN_OPTION_PORT:-8787}" + +# Same gate as the starter, for the same reason: this plugin is installed at user scope, so this +# hook runs in every project the user has. Unrouted projects must never hear from it. +case "${ANTHROPIC_BASE_URL:-}" in + *"127.0.0.1:${PORT}"* | *"localhost:${PORT}"* | *"[::1]:${PORT}"*) ;; + *) exit 0 ;; +esac + +if curl -fsS --max-time 2 "http://127.0.0.1:${PORT}/healthz" >/dev/null 2>&1; then + exit 0 +fi + +# Try to start it first — the common case is an idle-exit between prompts, and recovering silently +# is better than reporting a problem the user then has to act on. +if [ -x "${CLAUDE_PLUGIN_ROOT:-}/scripts/start-proxy.sh" ]; then + "${CLAUDE_PLUGIN_ROOT}/scripts/start-proxy.sh" >/dev/null 2>&1 || true + if curl -fsS --max-time 2 "http://127.0.0.1:${PORT}/healthz" >/dev/null 2>&1; then + exit 0 + fi +fi + +LOG="${TMPDIR:-/tmp}/context-guru-proxy-${PORT}.log" +cat </dev/null | awk '{print $2; exit}' +} + +if command -v "$BIN" >/dev/null 2>&1; then + have_path=$(command -v "$BIN") + have=$(installed_version "$have_path") + emit "path=${have_path}" + emit "version=${have:-unknown}" + # An explicit CONTEXT_GURU_VERSION means "I want that one" — honour it even when something is + # already installed. `latest` resolves below and is compared there. + if [ "$VERSION" != latest ] && [ "$VERSION" = "$have" ]; then + emit "result=present" + exit 0 + fi + if [ "$VERSION" = latest ] && [ -n "$have" ] && [ "${CONTEXT_GURU_UPGRADE:-}" != 1 ]; then + # Do not silently re-download on every install run; say what is there and how to move. + emit "result=present" + emit "note=set CONTEXT_GURU_UPGRADE=1 to check for and install a newer release" + exit 0 + fi + if [ -z "$have" ]; then + emit "note=the installed binary does not support --version; it predates the release channel" + fi + emit "note=upgrading from ${have:-unknown}" +fi + +case "$(uname -s)" in + Darwin) OS=darwin ;; + Linux) OS=linux ;; + *) die "unsupported_os_$(uname -s): build from source, see docs/get-started/quickstart-proxy.md" ;; +esac +case "$(uname -m)" in + x86_64|amd64) ARCH=amd64 ;; + arm64|aarch64) ARCH=arm64 ;; + *) die "unsupported_arch_$(uname -m)" ;; +esac +emit "platform=${OS}/${ARCH}" + +command -v curl >/dev/null 2>&1 || die "no_curl" + +# --- 2. release tarball ------------------------------------------------------------------- +if [ "$VERSION" = latest ]; then + # Resolve the tag rather than relying on /latest/download redirects, because the checksum + # file has to come from the SAME release as the tarball. Two separate redirect follows could + # straddle a release published between them. + VERSION=$(curl -fsSL "https://api.github.com/repos/${REPO}/releases/latest" 2>/dev/null | + sed -n 's/.*"tag_name": *"\([^"]*\)".*/\1/p' | head -1) + [ -n "$VERSION" ] || die "no_release_found: no published release yet for ${REPO}; build from source or set CONTEXT_GURU_VERSION" +fi +NUM="${VERSION#v}" +TARBALL="context-guru_${NUM}_${OS}_${ARCH}.tar.gz" +BASE="https://github.com/${REPO}/releases/download/${VERSION}" +emit "version=${VERSION}" + +# try_source_build is distribution option 3 from the header comment, which the first version of +# this script documented and never implemented — on a machine that had Go 1.26.4 on PATH. +# +# It is a FALLBACK, not a path anyone is steered to: it needs a toolchain, which is the gate this +# whole change exists to remove. But when there is no downloadable asset and a toolchain is right +# there, refusing to use it is worse than using it. +try_source_build() { + command -v go >/dev/null 2>&1 || return 1 + emit "fallback=go_install" + # CGO off: the binary is pure Go, and requiring a C toolchain here would reintroduce the gate. + if CGO_ENABLED=0 GOBIN="$DEST" go install "github.com/${REPO}/cmd/context-guru-proxy@${VERSION}" 2>"$TMP/go.err"; then + emit "result=installed" + emit "path=${DEST}/${BIN}" + emit "built_from=source" + return 0 + fi + emit "go_install_failed=$(tail -1 "$TMP/go.err" 2>/dev/null | tr -d '\n')" + return 1 +} + +TMP=$(mktemp -d) || die "no_tmpdir" +trap 'rm -rf "$TMP"' EXIT + +# The raw curl error used to reach stdout and break this script's "every fact is a key=value +# line" contract, which the calling skill parses. Keep curl quiet and report the failure as data. +if ! curl -fsSL -o "$TMP/$TARBALL" "$BASE/$TARBALL" 2>"$TMP/curl.err"; then + emit "download_url=$BASE/$TARBALL" + # A published tag with no assets reaches exactly here — the release exists, the artifact does + # not — which is what a pre-release repository looks like before the first build is attached. + if try_source_build; then + exit 0 + fi + die "download_failed: $BASE/$TARBALL (no asset for this platform, and no Go toolchain to build from source)" +fi + +# Checksum. The download is unsigned, there is no signature anywhere yet, and this script strips +# macOS quarantine from the file below — so this is the ONLY integrity check in the path. +# +# It is therefore fail-CLOSED, in every branch. The first version of this was fail-open: a missing +# or unfetchable checksums.txt printed one advisory line and installed anyway, which meant an +# unverified binary landed on a PATH directory and ran — a binary that then handles all of the +# user's LLM traffic and holds their API key. The comment above it said "a failure here is fatal, +# never a warning" while the code did the opposite. +# +# CONTEXT_GURU_INSECURE=1 exists for the one legitimate case (a local build served from a file +# path with no checksums file) and says what it is in its name. +verify_checksum() { + curl -fsSL -o "$TMP/checksums.txt" "$BASE/checksums.txt" 2>/dev/null || + die "checksum_unavailable: could not fetch $BASE/checksums.txt, so the download cannot be verified. Set CONTEXT_GURU_INSECURE=1 to install anyway (not recommended)." + want=$(awk -v f="$TARBALL" '$2 == f || $2 == "*"f {print $1}' "$TMP/checksums.txt" | head -1) + [ -n "$want" ] || + die "checksum_absent: $TARBALL is not listed in checksums.txt, so the download cannot be verified. Set CONTEXT_GURU_INSECURE=1 to install anyway (not recommended)." + if command -v sha256sum >/dev/null 2>&1; then + got=$(sha256sum "$TMP/$TARBALL" | awk '{print $1}') + else + got=$(shasum -a 256 "$TMP/$TARBALL" | awk '{print $1}') + fi + [ "$want" = "$got" ] || die "checksum_mismatch: expected $want got $got" + emit "checksum=verified" +} + +if [ "${CONTEXT_GURU_INSECURE:-}" = 1 ]; then + emit "checksum=SKIPPED_BY_CONTEXT_GURU_INSECURE" +else + verify_checksum +fi + +tar xzf "$TMP/$TARBALL" -C "$TMP" || die "untar_failed" +[ -f "$TMP/$BIN" ] || die "binary_not_in_tarball" + +mkdir -p "$DEST" || die "cannot_create_$DEST" +install -m 755 "$TMP/$BIN" "$DEST/$BIN" || die "install_failed_to_$DEST" + +# macOS: without this, the first run dies with "cannot be verified" and the evaluator concludes +# the project is broken. Notarization would remove the need and requires a paid Apple account. +if [ "$OS" = darwin ] && command -v xattr >/dev/null 2>&1; then + xattr -d com.apple.quarantine "$DEST/$BIN" 2>/dev/null || true + emit "quarantine=cleared" +fi + +emit "result=installed" +emit "path=${DEST}/${BIN}" + +# Report — do not fix — a PATH that will not find it. Editing the user's shell rc is a bigger +# intrusion than this script is entitled to, and the skill can tell them in context. +case ":${PATH}:" in + *":${DEST}:"*) emit "on_path=true" ;; + *) emit "on_path=false" + emit "note=add ${DEST} to your PATH, or the session hook will not find the proxy" ;; +esac diff --git a/context-guru-plugin/scripts/settings.py b/context-guru-plugin/scripts/settings.py new file mode 100755 index 00000000..16e91235 --- /dev/null +++ b/context-guru-plugin/scripts/settings.py @@ -0,0 +1,295 @@ +#!/usr/bin/env python3 +"""Add or remove exactly ONE key in a Claude Code settings file: env.ANTHROPIC_BASE_URL. + +This is the deterministic half of the install. The skill decides WHICH file and what to do +about a conflict; this script does the edit and refuses to guess. + +Why a script rather than `jq` in the skill's prompt: the target file is the user's real +`settings.json`, holding their theme, model, permission rules, statusline and possibly their +own base URL. Every operation here is therefore conservative to the point of being boring: + +* the file is read, parsed, and written back whole — never patched textually; +* a timestamped backup is written BEFORE the file is touched, and its path is reported; +* an existing ANTHROPIC_BASE_URL that is not ours is a CONFLICT and exits non-zero, because + overwriting somebody's gateway or benchmark endpoint is not a thing to do quietly; +* anything unparseable is refused rather than replaced with a fresh file, which would throw + away settings the user cannot get back. + +Output is one `key=value` line per fact on stdout, so the skill can act on the result without +re-reading the file or parsing prose. + +Usage: + settings.py add --file PATH --url URL [--force] + settings.py remove --file PATH [--url URL] + settings.py show --file PATH +""" + +from __future__ import annotations + +import argparse +import datetime as _dt +import json +import os +import shutil +import sys + +KEY = "ANTHROPIC_BASE_URL" + +# Where this script records what it did, so a later run can tell its own work from the user's. +META = "$context-guru" + + +def is_ours(data: dict, url: str) -> bool: + """Did WE write this base URL? Answered from a record, never from the URL's shape. + + The tempting version of this is a regex over loopback `/anthropic` URLs, and it is wrong in a + way a test caught: litellm's default is `http://127.0.0.1:4000/anthropic`, so "any local + /anthropic URL is ours" would make uninstall delete somebody else's routing. Two local proxies + are indistinguishable by URL — so instead `add` records the URL it wrote, and this reads it. + + A file with no record predates that (or was hand-edited), and then only an exact match against + the URL the caller passed counts. Fail toward leaving the user's configuration alone. + """ + meta = data.get(META) + if isinstance(meta, dict) and meta.get("installed_base_url"): + return url == meta["installed_base_url"] + return False + + +def emit(**facts: object) -> None: + for k, v in facts.items(): + print(f"{k}={v}") + + +def load(path: str) -> tuple[dict, bool]: + """Return (settings, existed). Refuses to proceed on anything it cannot parse.""" + if not os.path.exists(path): + return {}, False + with open(path, encoding="utf-8") as fh: + text = fh.read() + if not text.strip(): + return {}, True + try: + data = json.loads(text) + except json.JSONDecodeError as exc: + emit(result="error", reason="unparseable_json", detail=f"{exc}") + # Deliberately fatal. The alternative — treating a broken file as empty — would + # silently discard every setting in it. + sys.exit(3) + if not isinstance(data, dict): + emit(result="error", reason="not_an_object") + sys.exit(3) + return data, True + + +def backup(path: str) -> str: + """Copy `path` aside and return the copy's name. Never overwrites an existing backup. + + The stamp used to be second-granularity with a plain `copy2`, which meant an + install-then-uninstall round trip — well inside one second — wrote both backups to the SAME + filename, and the survivor held the POST-install state. The user was then told to keep that + path as their undo, and it was a copy of the change, not of what preceded it. + + Microseconds plus O_EXCL: the exclusive create is what actually guarantees it, since two + writes in the same microsecond are merely unlikely rather than impossible. + """ + stamp = _dt.datetime.now().strftime("%Y%m%d-%H%M%S-%f") + for attempt in range(100): + dest = f"{path}.context-guru-backup-{stamp}" + (f".{attempt}" if attempt else "") + try: + fd = os.open(dest, os.O_CREAT | os.O_EXCL | os.O_WRONLY, 0o600) + except FileExistsError: + continue + with os.fdopen(fd, "wb") as out, open(path, "rb") as src: + shutil.copyfileobj(src, out) + shutil.copystat(path, dest) + prune_backups(path) + return dest + raise RuntimeError(f"could not create a backup for {path}") + + +# How many backups of one settings file to keep. Each add and each remove writes one, so a user +# who installs and uninstalls a few times accumulated them forever in ~/.claude — 40 files after +# 20 cycles, in a directory they read by hand. +KEEP_BACKUPS = 10 + + +def prune_backups(path: str) -> None: + """Delete all but the newest KEEP_BACKUPS backups of `path`. Best effort.""" + import glob + + try: + found = sorted(glob.glob(f"{path}.context-guru-backup-*"), key=os.path.getmtime) + except OSError: + return + for old in found[:-KEEP_BACKUPS]: + try: + os.remove(old) + except OSError: + pass + + +def save(path: str, data: dict) -> None: + """Write `data` to `path` atomically, preserving the file's identity and permissions. + + Three things here are each a defect that was found rather than anticipated: + + * **Follow symlinks.** A dotfile-managed `settings.json` is commonly a symlink into a + repository. `os.replace` onto the link path REPLACES THE LINK with a regular file, so the + edit silently never reaches the file the user actually manages and their dotfiles still + hold the old content. Resolve first, then write to the real path. + * **Preserve the mode.** The temp file is created fresh, so the replaced file's mode was + taken from the umask: a `600` settings file holding `ANTHROPIC_AUTH_TOKEN` came back + world-readable `644` under the common default umask. + * **Atomic.** An interrupted write must not leave half a settings file, which would break + every session in that scope rather than only ours. + """ + real = os.path.realpath(path) + os.makedirs(os.path.dirname(os.path.abspath(real)) or ".", exist_ok=True) + tmp = f"{real}.context-guru-tmp" + with open(tmp, "w", encoding="utf-8") as fh: + json.dump(data, fh, indent=2, ensure_ascii=False) + fh.write("\n") + if os.path.exists(real): + shutil.copymode(real, tmp) + else: + os.chmod(tmp, 0o600) + os.replace(tmp, real) + + +def cmd_show(args: argparse.Namespace) -> int: + data, existed = load(args.file) + current = (data.get("env") or {}).get(KEY) + emit( + result="ok", + file=args.file, + exists=str(existed).lower(), + base_url=current if current else "(unset)", + other_env_keys=len([k for k in (data.get("env") or {}) if k != KEY]), + top_level_keys=len(data), + ) + return 0 + + +def cmd_add(args: argparse.Namespace) -> int: + data, existed = load(args.file) + env = data.get("env") + if env is None: + env = {} + if not isinstance(env, dict): + emit(result="error", reason="env_not_an_object") + return 3 + + current = env.get(KEY) + if current == args.url: + emit(result="unchanged", file=args.file, base_url=current, + note="already routed to this proxy") + return 0 + if current and is_ours(data, current) and not args.force: + # Our own URL on a different port — the user changed the configured port and re-ran. + # Reporting a conflict here told them somebody else owned their routing, which was + # wrong and alarming. Move it, and keep the note so the change is visible. + saved = backup(args.file) + env[KEY] = args.url + data["env"] = env + data.setdefault(META, {})["installed_base_url"] = args.url + save(args.file, data) + emit(result="repointed", file=args.file, base_url=args.url, previous=current, + backup=saved, note="this was our own URL on another port; moved") + return 0 + if current and not args.force: + # The one conflict this has to reason about. `env` blocks merge per key across + # scopes, so a user-scope install is not clobbered by a project that ships its own + # env block — what is left is a base URL the USER set, which may be their company + # gateway or a benchmark endpoint, and taking it over would break their setup while + # looking like it worked. + emit(result="conflict", file=args.file, existing=current, proposed=args.url, + note="ANTHROPIC_BASE_URL is already set here; ask before replacing it, " + "then re-run with --force") + return 2 + + saved = backup(args.file) if existed else "" + env[KEY] = args.url + data["env"] = env + # Remember what we took over, so uninstall can hand it back. + # + # `replaced` used to be reported and then forgotten. After a --force install over somebody's + # own gateway, uninstall deleted the key and left them with NO base URL at all — and because + # the backup filename collided with the install's own, the copy that held it was gone too. + # Their setup was unrecoverable from anything the tool produced. + # Record what we wrote, so a re-run can recognise its own work, and what we took over, so + # uninstall can hand it back. + meta = data.setdefault(META, {}) + meta["installed_base_url"] = args.url + if current: + meta["previous_base_url"] = current + save(args.file, data) + emit(result="added", file=args.file, base_url=args.url, + replaced=current if current else "", backup=saved or "(new file)", + other_env_keys=len([k for k in env if k != KEY])) + return 0 + + +def cmd_remove(args: argparse.Namespace) -> int: + data, existed = load(args.file) + if not existed: + emit(result="unchanged", file=args.file, note="no such file") + return 0 + env = data.get("env") + if not isinstance(env, dict) or KEY not in env: + emit(result="unchanged", file=args.file, note=f"no env.{KEY} here") + return 0 + current = env[KEY] + # Ours is the URL passed in, or the one we recorded at install time — which covers the case + # where the configured port changed since. It is NOT "any loopback /anthropic URL": litellm's + # default is one of those, and uninstall must not delete somebody else's routing. + if args.url and current != args.url and not is_ours(data, current): + # Refuse to remove a base URL that is not ours: the user may have pointed this at + # something else since, and uninstall must not take that with it. + emit(result="conflict", file=args.file, existing=current, expected=args.url, + note="this base URL is not the one context-guru installed; left untouched") + return 2 + saved = backup(args.file) + del env[KEY] + # Put back whatever we took over at install time. Deleting the key was leaving a user who had + # a gateway configured with nothing at all — a worse state than before they installed. + restored = "" + meta = data.get(META) + if isinstance(meta, dict): + if meta.get("previous_base_url"): + restored = meta["previous_base_url"] + env[KEY] = restored + # Our bookkeeping goes with our key: leaving it behind would make a later install think it + # had written a URL it did not. + meta.pop("previous_base_url", None) + meta.pop("installed_base_url", None) + if not meta: + data.pop(META, None) + # Leave no litter: an `env: {}` we created is removed with the key. An env block that + # still holds the user's own variables stays exactly as it is. + if not env: + del data["env"] + else: + data["env"] = env + save(args.file, data) + emit(result="removed", file=args.file, was=current, backup=saved, + restored=restored, env_block_left=str(bool(env)).lower()) + return 0 + + +def main() -> int: + ap = argparse.ArgumentParser(description=__doc__) + sub = ap.add_subparsers(dest="cmd", required=True) + for name in ("add", "remove", "show"): + p = sub.add_parser(name) + p.add_argument("--file", required=True) + p.add_argument("--url", default="") + p.add_argument("--force", action="store_true") + args = ap.parse_args() + if args.cmd == "add" and not args.url: + ap.error("add needs --url") + return {"add": cmd_add, "remove": cmd_remove, "show": cmd_show}[args.cmd](args) + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/context-guru-plugin/scripts/start-proxy.sh b/context-guru-plugin/scripts/start-proxy.sh new file mode 100755 index 00000000..0bb4067d --- /dev/null +++ b/context-guru-plugin/scripts/start-proxy.sh @@ -0,0 +1,117 @@ +#!/usr/bin/env bash +# SessionStart hook: make sure the proxy this project routes to is actually listening. +# +# Five properties, each of which is a way this can be wrong: +# +# 1. IT SELF-GATES ON $ANTHROPIC_BASE_URL. The plugin installs at USER scope, so this hook +# runs in every project the user has — including every project they never routed, where +# starting a proxy is pure waste. Settings `env` values are written into the process +# environment and hook processes inherit it, so the variable naming our port IS the +# per-project enablement signal. No second config to keep in sync, and it degrades +# correctly: delete the env key by hand and this hook stops doing anything, by itself. +# +# 2. IT IS IDEMPOTENT. SessionStart also fires on `clear`, `compact`, `resume` and `fork`, not +# just `startup` — a long session re-fires it repeatedly. So: probe /healthz, and start +# something only if nothing answers. +# +# 3. IT IS SYNCHRONOUS. The hook is deliberately NOT marked async: it returns only once +# /healthz answers, which is what closes the race with the session's first API request. +# A request that beats the proxy up gets a connection error, and Claude Code's retry does +# not make that invisible. +# +# 4. THE PORT IS FIXED, not negotiated. The URL in settings was written before this ran and +# cannot be renegotiated, so both sides read the same configured value. +# +# 5. IT NEVER FAILS THE SESSION. Every exit is 0. A proxy that will not start must leave the +# user with a working Claude Code and a note about it — the alternative is a plugin that +# can brick every session on the machine, which is the biggest risk in this whole feature. +set -uo pipefail + +PORT="${CLAUDE_PLUGIN_OPTION_PORT:-8787}" +PRESET="${CLAUDE_PLUGIN_OPTION_PRESET:-cache}" +IDLE_EXIT="${CLAUDE_PLUGIN_OPTION_IDLE_EXIT:-24h}" +BIN="${CONTEXT_GURU_BIN:-context-guru-proxy}" +LOG="${TMPDIR:-/tmp}/context-guru-proxy-${PORT}.log" +HEALTH="http://127.0.0.1:${PORT}/healthz" + +note() { printf 'context-guru: %s\n' "$*"; } + +# --- (1) the gate ------------------------------------------------------------------------ +# Match the port, not merely the word "localhost": a user routing to a DIFFERENT local proxy +# (litellm, their own gateway) must not have ours started underneath them. +case "${ANTHROPIC_BASE_URL:-}" in + *"127.0.0.1:${PORT}"* | *"localhost:${PORT}"* | *"[::1]:${PORT}"*) ;; + *) + # Silent by design. This is the common case — every unrouted project — and a line of + # output here would appear in sessions that have nothing to do with context-guru. + exit 0 ;; +esac + +# --- (2) already up? --------------------------------------------------------------------- +if curl -fsS --max-time 2 "$HEALTH" >/dev/null 2>&1; then + exit 0 +fi + +if ! command -v "$BIN" >/dev/null 2>&1 && [ ! -x "$BIN" ]; then + note "routing is configured for port ${PORT} but the proxy binary ('${BIN}') is not on PATH." + note "run /context-guru:install to install it, or /context-guru:uninstall to stop routing." + exit 0 +fi + +# --- (3) start it, and wait for it to answer --------------------------------------------- +# setsid detaches the proxy from this hook's process group so it survives the hook returning +# and is not killed with the session's process tree. --idle-exit is what eventually reaps it. +STARTER=(setsid) +command -v setsid >/dev/null 2>&1 || STARTER=(nohup) # macOS has no setsid + +# --listen on the COMMAND LINE, not LISTEN_ADDR in the environment. +# +# The port has to be visible in `argv`. When it was passed through the environment, nothing that +# needed to find this specific proxy could: `pkill -f "context-guru-proxy.*$PORT"` matched no +# proxy at all, and did match the shell running it — so uninstall killed the user's own Claude +# Code session and left the proxy holding the port. +# +# The pidfile below is the primary handle; argv visibility is what makes `ps` and `pgrep` honest. +# +# --dashboard, with its database in the state directory: the skills and the line printed below +# advertise the dashboard as the place the cache effect is visible, and without this flag that URL +# was a 404. Its default DB path is `./context-guru-dashboard.db` — the current directory, i.e. +# the user's repository — so the path must be set explicitly or the plugin litters the project it +# was invited into. +STATE="${XDG_STATE_HOME:-$HOME/.local/state}/context-guru" +mkdir -p "$STATE" 2>/dev/null || STATE="${TMPDIR:-/tmp}" +PIDFILE="${STATE}/proxy-${PORT}.pid" + +PRESET="$PRESET" \ + "${STARTER[@]}" "$BIN" \ + --listen "127.0.0.1:${PORT}" \ + --idle-exit="$IDLE_EXIT" \ + --dashboard \ + --dashboard-db "${STATE}/dashboard-${PORT}.db" \ + >>"$LOG" 2>&1 & +started=$! +disown 2>/dev/null || true +# The pidfile is what uninstall uses. Written before the health wait so a proxy that comes up +# slowly is still stoppable, and it records the port it belongs to in its own name. +printf '%s\n' "$started" >"$PIDFILE" 2>/dev/null || true + +# Up to ~15s. A cold start is well under a second; the budget is for a loaded laptop, and the +# hook's own timeout (60s in hooks.json) is the real backstop. +for _ in $(seq 1 60); do + if curl -fsS --max-time 2 "$HEALTH" >/dev/null 2>&1; then + note "proxy up on 127.0.0.1:${PORT} (preset ${PRESET}, idle-exit ${IDLE_EXIT})." + note "dashboard: http://127.0.0.1:${PORT}/dashboard/" + exit 0 + fi + sleep 0.25 +done + +# --- (5) failed, and the session still has to work --------------------------------------- +note "the proxy did not come up on port ${PORT}; this session's requests will fail until it does." +note "log: ${LOG}" +note "fix it with /context-guru:status, or stop routing with /context-guru:uninstall." +if [ -s "$LOG" ]; then + note "last lines:" + tail -n 5 "$LOG" | sed 's/^/context-guru: /' +fi +exit 0 diff --git a/context-guru-plugin/skills/install/SKILL.md b/context-guru-plugin/skills/install/SKILL.md new file mode 100644 index 00000000..aa37ba62 --- /dev/null +++ b/context-guru-plugin/skills/install/SKILL.md @@ -0,0 +1,167 @@ +--- +name: install +description: Install a local context-guru proxy and route this project's Claude Code sessions through it, so long sessions stop paying to re-create the prompt cache. Use when the user asks to install, set up, enable, try or start context-guru, or to route Claude Code through it. Accepts --global to route every project on the machine instead of just this one. +--- + +# Install context-guru for Claude Code + +Install the proxy binary, then add **one** key — `env.ANTHROPIC_BASE_URL` — to a settings file +so this project's sessions go through it. + +Your job here is the part a shell script does badly: choosing the right file, merging into +settings the user already depends on, noticing a base URL that is already set, and verifying +the result. The deterministic steps are scripts in `${CLAUDE_PLUGIN_ROOT}/scripts/`. Run them; +do not reimplement them inline. + +## What the user is agreeing to + +Say this plainly before touching anything, because it is the part that matters and it is short: + +- Every Claude Code API request in the chosen scope will go through a **local** proxy on + `127.0.0.1`. Nothing is sent anywhere else, and the proxy forwards to Anthropic itself. +- **No API key is needed.** Setting `ANTHROPIC_BASE_URL` without a credential variable keeps + their claude.ai login working — a Pro/Max subscription continues to apply, with their usage + limits and billing unchanged. (On a subscription the saving lands in usage limits rather than + dollars, so `/context-guru:status` cost figures are list-price estimates, not their bill.) +- The default preset is `cache`: the prompt-cache split and nothing else. No content dropped, + no markers, no extra tool in their requests, no model calls. +- **If the proxy is down, requests in the routed scope do not fail cleanly — they HANG.** With + routing configured and nothing listening, a prompt produces no output and no error on either + stream, indefinitely. That is the state after a crash, a reboot, or an idle-exit, and it is the + whole risk of routing at all; it is why the default scope is this project rather than the machine. + The plugin installs a `UserPromptSubmit` hook that detects it, tries to restart the proxy, and + otherwise prints what to do — a hook rather than a skill because invoking a skill needs a model + call, which is the thing that is broken. +- `/context-guru:uninstall` removes the key and stops the proxy. + +## Steps + +### 1. Install the binary + +```bash +"${CLAUDE_PLUGIN_ROOT}/scripts/install.sh" +``` + +It prints `key=value` lines. Read them rather than guessing: + +- `result=present` — already installed, nothing downloaded. Fine; continue. +- `result=installed` — downloaded and verified. If `on_path=false`, tell the user to add the + directory to their `PATH` and that the session hook cannot find the proxy until they do. +- `result=error reason=no_release_found` — no published release yet for this repo. Say so, and + offer the source build (`make build-static`, needs Go 1.26 but no C toolchain). Do not pretend + it worked. +- `result=error reason=download_failed` — the release tag exists but carries no asset for this + platform, which is what a repo looks like before its first build is attached. The script tries a + `go install` fallback first and reports `fallback=go_install`; if that is in the output and the + result is still an error, there was no toolchain either. +- `reason=checksum_mismatch`, `checksum_unavailable`, `checksum_absent` — **stop, and do not + install.** All three mean the download could not be verified against the release's + `checksums.txt`. There is no signature anywhere yet, so this is the only integrity check in the + path, and the binary in question is about to handle all of the user's LLM traffic and hold their + API key. `CONTEXT_GURU_INSECURE=1` overrides it and you must not set it on the user's behalf. +- `checksum=SKIPPED_BY_CONTEXT_GURU_INSECURE` in the output — the user set that themselves. Say + plainly that an unverified binary was installed. +- **Already installed?** The script reports `result=present` with the `version=`, and does not + replace it. To upgrade, re-run with `CONTEXT_GURU_UPGRADE=1`; to pin a version, set + `CONTEXT_GURU_VERSION=vX.Y.Z`. If `version=unknown`, the installed binary predates `--version` + and an upgrade is worth offering. + +### 2. Choose the scope + +Default to **this project only**. Ask before doing anything wider, and give them the real +trade-off in one line each: + +| Scope | File | If the proxy is down | +|---|---|---| +| **This project (default)** | `.claude/settings.local.json` | only this project breaks; the file is gitignored | +| This project, whole team | `.claude/settings.json` | breaks for everyone who clones the repo | +| Every project (`--global`) | `~/.claude/settings.json` | **every Claude Code session on the machine breaks** | + +If the user passed `--global`, use the third and confirm once that they mean it, naming the +blast radius. `env` blocks merge per key across scopes, so a user-scope install is not clobbered +by a project that ships its own `env` block. + +### 3. Look before you write + +```bash +python3 "${CLAUDE_PLUGIN_ROOT}/scripts/settings.py" show --file +``` + +If `base_url` is already set to something that is not our port, **stop and ask.** It may be +their company gateway, a benchmark endpoint, or another proxy — replacing it silently would +break their setup while looking like success. Offer: keep theirs (abandon the install), or +replace it (and tell them the old value, so they can put it back). + +### 4. Write the one key + +The port comes from the plugin's configuration (`CLAUDE_PLUGIN_OPTION_PORT`, default `8787`). +The URL must end in `/anthropic` — that is the path the proxy serves the Anthropic dialect on. + +```bash +python3 "${CLAUDE_PLUGIN_ROOT}/scripts/settings.py" add \ + --file --url "http://127.0.0.1:${CLAUDE_PLUGIN_OPTION_PORT:-8787}/anthropic" +``` + +- `result=added` — report the `backup=` path to the user. That is their undo. +- `result=conflict` — you skipped step 3, or the file changed. Go back and ask; only pass + `--force` once the user has said to replace that specific value. When they do, the replaced + value is recorded and `/context-guru:uninstall` puts it back — say so, because "we will take + over your gateway" is much easier to agree to when it is reversible. +- `result=repointed` — the file already held a context-guru URL on a different port (the user + changed the configured port). Moved, with the previous value reported. Not a conflict. +- `result=error reason=unparseable_json` — their settings file is already broken. Do not + rewrite it. Tell them where and let them fix it. + +### 5. Start it and prove it works + +```bash +"${CLAUDE_PLUGIN_ROOT}/scripts/start-proxy.sh" +``` + +The script is idempotent and gated: it starts the proxy only if nothing answers `/healthz`, and +does nothing at all unless `ANTHROPIC_BASE_URL` names our port. In this session that variable is +**not yet set** — the settings change applies to the *next* session — so verify by hand: + +```bash +curl -fsS "http://127.0.0.1:${CLAUDE_PLUGIN_OPTION_PORT:-8787}/healthz" +``` + +If nothing answers, start it in the foreground of a background shell and read the log rather +than declaring victory: + +```bash +context-guru-proxy \ + --listen "127.0.0.1:${CLAUDE_PLUGIN_OPTION_PORT:-8787}" \ + --preset "${CLAUDE_PLUGIN_OPTION_PRESET:-cache}" \ + --idle-exit="${CLAUDE_PLUGIN_OPTION_IDLE_EXIT:-24h}" +``` + +Pass the port as `--listen`, not through `LISTEN_ADDR`: the port has to be visible in the process +command line, or nothing — including `/context-guru:uninstall` — can identify this proxy among +others. + +A `--idle-exit` below the store's floor (~5h34m at the default TTL) is **refused at startup** on +purpose — exiting clears in-memory cache state. If they want a shorter one, that is a +`store.ttl_seconds` conversation, not a flag to force. + +### 6. Tell them what happens next + +- The setting takes effect in a **new session** — this one is already running with the old + environment. Say so explicitly; otherwise the natural next question is "why is `/status` + showing nothing?" +- From then on the plugin's `SessionStart` hook starts the proxy automatically if it is not + running, in the projects that are routed and nowhere else. +- The proxy exits by itself after `--idle-exit` of no use, so nothing is left running. +- Dashboard: `http://127.0.0.1:/dashboard/` — the four billed token tiers are where the + cache effect is visible. +- `/context-guru:status` for the numbers, `/context-guru:uninstall` to undo. + +## Do not + +- Do not add any other key. Not `ANTHROPIC_API_KEY`, not `ANTHROPIC_AUTH_TOKEN` — a credential + variable is what would take them OFF their subscription billing. +- Do not edit a settings file without the backup step, and do not hand-edit JSON: use the + script, which replaces the file atomically. +- Do not put the base URL in `.mcp.json`, an env file, or a shell rc. One key, one file. +- Do not claim it is working because the install steps returned 0. `/healthz` answering is the + claim; anything else is a guess. diff --git a/context-guru-plugin/skills/status/SKILL.md b/context-guru-plugin/skills/status/SKILL.md new file mode 100644 index 00000000..6d94572c --- /dev/null +++ b/context-guru-plugin/skills/status/SKILL.md @@ -0,0 +1,93 @@ +--- +name: status +description: Report whether the local context-guru proxy is running, whether this project is routed through it, and what it has actually saved — reading /stats and explaining the numbers honestly. Use when the user asks about context-guru status, savings, cache hit rate, cost, tokens saved, or whether the proxy is working. +--- + +# context-guru status + +Answer two questions in order, because the second is meaningless if the first is "no": + +1. **Is this project actually routed, and is the proxy up?** +2. **What has it saved?** + +## 1. Routing and liveness + +```bash +PORT="${CLAUDE_PLUGIN_OPTION_PORT:-8787}" +echo "ANTHROPIC_BASE_URL=${ANTHROPIC_BASE_URL:-(unset)}" +curl -fsS --max-time 3 "http://127.0.0.1:${PORT}/healthz" || echo "(no proxy on ${PORT})" +``` + +Then check where the routing is configured, in precedence order — later files win: + +```bash +for f in ~/.claude/settings.json .claude/settings.json .claude/settings.local.json; do + [ -f "$f" ] && python3 "${CLAUDE_PLUGIN_ROOT}/scripts/settings.py" show --file "$f" +done +``` + +Report the combinations plainly, because they mean different things: + +- **`ANTHROPIC_BASE_URL` unset in this session, but present in a settings file** — the routing + was added after this session started. It applies to the next session. This is the normal + state right after `/context-guru:install`, and it is not a fault. +- **Set, proxy answering** — working. Go to step 2. +- **Set, nothing answering** — this session's API requests are failing right now. Start it: + `"${CLAUDE_PLUGIN_ROOT}/scripts/start-proxy.sh"`, and if that does not work, read + `${TMPDIR:-/tmp}/context-guru-proxy-${PORT}.log`. If they want out immediately, that is + `/context-guru:uninstall`. +- **Set to a base URL that is not ours** — say so and stop. Something else owns their routing. + +## 2. What it saved + +```bash +curl -fsS "http://127.0.0.1:${PORT}/stats" +``` + +Lead with the **billed token tiers** (`cache_read`, `cache_creation`, input, output). Those come +from the provider's own usage block, so they are the numbers the user can check against their +own bill or usage page. Under the `cache` preset the whole story is tokens moving from the +cache-creation tier to the cache-read tier — creation is billed at a premium, reads at a +discount, so that shift *is* the saving. + +Then, if they are non-zero: `requests`, `saved_tokens`, `savings_pct`, and the keep-alive block +(`pings`, `spend_usd`, `wrote_instead_of_read`). + +The dashboard shows the same thing over time: `http://127.0.0.1:/dashboard/`. + +## Be honest about the numbers + +These caveats are not hedging; each one is a way a confident reading would be wrong: + +- **`/stats` cost figures are list-price estimates.** On a Pro/Max subscription the saving lands + in usage limits, not dollars — their bill does not change. Say so rather than quoting a dollar + figure at a subscriber as if it were money back. +- **A fresh install shows almost nothing, and that is expected.** The cache effect appears on + the *second and later* turns of a session; the first request of a session is nearly always + cold — measured, 1,105 of 1,127 session starts. +- **Check whether this is even a git repository, before offering any other explanation.** The + split works on the environment snapshot Claude Code appends to its system prompt, and outside a + git repo there is no snapshot to split — `cachesplit` reports `verdict: skipped`, `mutated: 0`, + and the saving is exactly zero. This is the common case for a casual first trial, and telling + such a user "the cache warms up on later turns" is true in general and wrong here: + + ```bash + git rev-parse --is-inside-work-tree 2>/dev/null || echo "NOT a git repo — cachesplit cannot act" + ``` +- **`acted: 0` and `saved_tokens: 0` are not evidence of failure for this preset.** Those count + components that removed content; `cachesplit` relocates a cache breakpoint and removes nothing. + The signals that move are `components.cachesplit.verdict` (`moved` vs `skipped`), its `mutated` + count, and the billed tiers. Lead with the tiers, and do not quote `savings_pct` as the verdict + on a cache-only pipeline. +- **`wrote_instead_of_read` above zero is a bug signal, not a saving.** It means a keep-alive + ping created a cache entry instead of refreshing one, which costs money for nothing. Report it + as a problem. +- **On non-Anthropic backends the `cache` preset does nothing at all** (vLLM, llm-d and similar + match an implicit longest prefix on their own). Zero saving there is correct behaviour, not a + failure. +- **`/stats` is process-wide**, not per-project: if they route several projects to one proxy, + these totals cover all of them. + +If the numbers are genuinely flat after real use, say that and offer the next step — usually +`codesmart`, which adds the offloaders — rather than reaching for a favourable reading of a +flat graph. diff --git a/context-guru-plugin/skills/uninstall/SKILL.md b/context-guru-plugin/skills/uninstall/SKILL.md new file mode 100644 index 00000000..1ad47a55 --- /dev/null +++ b/context-guru-plugin/skills/uninstall/SKILL.md @@ -0,0 +1,111 @@ +--- +name: uninstall +description: Stop routing Claude Code through context-guru — remove the ANTHROPIC_BASE_URL key it added, stop the local proxy, and optionally delete the binary. Use when the user asks to uninstall, remove, disable, turn off, or stop context-guru, or says routing through it is breaking their sessions. +--- + +# Uninstall context-guru + +Undo is the promise the install made, so it has to work on the first try and in the order +below. **Remove the routing before stopping the proxy** — the other way round leaves a window +where sessions are pointed at a port with nothing behind it, and every request in that window +fails. + +If the user is here because something is broken, do step 1 first and explain afterwards. + +## 1. Remove the routing key + +Check all three scopes: the install may have written any of them, and a `--global` install +plus a per-project one can both exist. + +```bash +PORT="${CLAUDE_PLUGIN_OPTION_PORT:-8787}" +for f in .claude/settings.local.json .claude/settings.json ~/.claude/settings.json; do + [ -f "$f" ] && python3 "${CLAUDE_PLUGIN_ROOT}/scripts/settings.py" remove \ + --file "$f" --url "http://127.0.0.1:${PORT}/anthropic" +done +``` + +Passing `--url` is what keeps this safe: the script removes the key only if it holds **our** +base URL, and reports `result=conflict` instead of deleting a value the user has since pointed +somewhere else. If you see a conflict, leave it alone and tell them what is there. + +Every change writes a timestamped backup — report those paths. + +The removal takes effect in a **new session**; this one keeps the environment it started with. +Say so, or "I removed it and it is still routing" is the next message. + +## 2. Stop the proxy + +**Do not pattern-match the process.** An earlier version of this skill ran +`pkill -f "context-guru-proxy.*${PORT}"`, which was wrong in two ways at once: the starter passed +the port through the environment, so it appeared nowhere in the proxy's command line and the +pattern matched no proxy at all — while it *did* match the shell running the `pkill`, i.e. the +Bash tool of the session you are in. It killed the user's own session, reported nothing removed, +and left the proxy holding the port. + +Use the pidfile the starter writes, and fall back to the socket's owner: + +```bash +PORT="${CLAUDE_PLUGIN_OPTION_PORT:-8787}" +STATE="${XDG_STATE_HOME:-$HOME/.local/state}/context-guru" +PIDFILE="${STATE}/proxy-${PORT}.pid" + +pid="" +[ -f "$PIDFILE" ] && pid=$(cat "$PIDFILE") + +# Fall back to whoever holds the port — covers a proxy started by hand, or a stale pidfile. +if [ -z "$pid" ] || ! kill -0 "$pid" 2>/dev/null; then + if command -v lsof >/dev/null 2>&1; then + pid=$(lsof -ti "tcp:${PORT}" -sTCP:LISTEN 2>/dev/null | head -1) + elif command -v ss >/dev/null 2>&1; then + pid=$(ss -lntpH "sport = :${PORT}" 2>/dev/null | grep -o 'pid=[0-9]*' | cut -d= -f2 | head -1) + fi +fi + +if [ -n "$pid" ]; then + kill "$pid" && rm -f "$PIDFILE" +else + echo "(nothing listening on ${PORT})" +fi +``` + +Before killing anything, **confirm the PID is ours** — the port may be held by something else +entirely, and this step must not kill a stranger's process: + +```bash +ps -p "$pid" -o command= | grep -q context-guru-proxy && echo "ours" || echo "NOT OURS — stop" +``` + +Then confirm it is actually gone, rather than assuming the kill worked: + +```bash +sleep 1 +curl -fsS --max-time 2 "http://127.0.0.1:${PORT}/healthz" && echo "STILL RUNNING" || echo "stopped" +``` + +If it is still running, report the PID and let the user decide. Do not escalate to `kill -9` on a +pattern, and never broaden the match to `context-guru-proxy` alone: on a host that also runs a +production instance or a benchmark arm, that takes those down too. + +## 3. Offer, do not assume, the rest + +Ask before either of these; neither is implied by "stop routing my sessions": + +- **Delete the binary** — `rm ~/.local/bin/context-guru-proxy` (or wherever + `command -v context-guru-proxy` reports). +- **Remove the plugin itself** — `/plugin uninstall context-guru@context-guru`. Until they do, + both hooks (`SessionStart` and `UserPromptSubmit`) stay installed, and that is harmless: each + self-gates on `ANTHROPIC_BASE_URL` and exits immediately in a project that is not routed — which, + after step 1, is every project. Worth saying, so a leftover hook is not mistaken for a leftover + proxy. +- **Delete the state directory** — `~/.local/state/context-guru` holds the pidfile and the + dashboard database (session metadata and token counts, no prompt content unless they enabled + content capture). Nothing reads it once the proxy is gone. + +## 4. Confirm the end state + +Report, in one short list: which files changed and their backups, that the proxy is stopped, +whether the binary and plugin are still present, and that the change lands in the next session. + +If there are leftovers the user declined to remove, name them — an uninstall that quietly +leaves things behind is the reason people distrust installers. diff --git a/docs/get-started/quickstart-proxy.md b/docs/get-started/quickstart-proxy.md index c554a9ad..b11bfa4f 100644 --- a/docs/get-started/quickstart-proxy.md +++ b/docs/get-started/quickstart-proxy.md @@ -13,6 +13,9 @@ tar xzf context-guru_*_darwin_arm64.tar.gz install -m 755 context-guru-proxy ~/.local/bin/ ``` +Claude Code users can skip even that — the plugin installs the binary and configures the routing: +[Install as a Claude Code plugin](../how-to/install-plugin.md). + To build from source instead you need **Go 1.26** — and still no C toolchain: `make build` builds with cgo disabled and produces the same statically linked binary. CI asserts that natively for linux/amd64 (the `purego` job), and the release workflow asserts it again before publishing. diff --git a/docs/how-to/install-plugin.md b/docs/how-to/install-plugin.md new file mode 100644 index 00000000..4fae7b1d --- /dev/null +++ b/docs/how-to/install-plugin.md @@ -0,0 +1,147 @@ +# Install as a Claude Code plugin + +Two commands, no toolchain, reversible. + +``` +/plugin marketplace add rossoctl/context-guru +/plugin install context-guru@context-guru +/context-guru:install +``` + +The first two are once per machine. The third is once per repo, and it is the one that decides +which sessions get routed. + +## You do not need an API key + +Setting `ANTHROPIC_BASE_URL` **without** a credential variable leaves your claude.ai login +alone: a Pro or Max subscription keeps working, with your usage limits and billing unchanged. +You can evaluate context-guru on your own sessions with no API key at all. + +One honest caveat: on subscription billing the saving lands in **usage limits**, not dollars. The +cost figures on `/context-guru:status` and the dashboard are list-price estimates, and they will +not match a subscriber's bill — because a subscriber does not get one per request. + +Do not let the installer add `ANTHROPIC_API_KEY` or `ANTHROPIC_AUTH_TOKEN`. It will not, and +that is deliberate: a credential variable is exactly what would move you off subscription +billing and onto metered API billing. + +## What gets installed where + +| Thing | Scope | How often | +|---|---|---| +| Plugin and its skills | your user settings | once per machine | +| Proxy binary | `~/.local/bin` | once per machine | +| **Routing — the `env` block** | **this project by default**, `--global` opt-in | **once per repo** | +| The proxy process | started on demand, exits when idle | automatic | + +Only the routing decision is per-repo, and deliberately: it is the one with blast radius. + +## Which file the routing goes in + +Settings precedence runs managed → `--settings` → `.claude/settings.local.json` → +`.claude/settings.json` → `~/.claude/settings.json`, so user scope is the *lowest*. + +| Write target | Reaches | If the proxy is down | +|---|---|---| +| `.claude/settings.local.json` | you, this repo, gitignored | one repo — **the default** | +| `.claude/settings.json` | everyone who clones the repo | one repo, whole team | +| `~/.claude/settings.json` (`--global`) | every project on the machine | **every Claude Code session you have** | + +The default is project-local because a global base URL pointing at `localhost` means a dead +proxy breaks Claude Code everywhere, including repos you never meant to experiment in. `env` +blocks merge **per key** across scopes, so a user-scope install is not clobbered by a repo that +ships its own `env` block, and a `--global` install needs no per-repo caveat. + +## What it does to your requests + +The default preset is `cache`: [`cachesplit`](../components/cachesplit.md) and nothing else. + +- No content dropped, no summarising, no `<>` markers. +- No extra tool added to your requests, and no model calls. +- One oversized system block is split into two adjacent text blocks whose concatenation is + byte-identical, so the model sees exactly the prompt your agent sent. The cache breakpoint + moves onto the half that does not churn. + +You can check that claim in one line of `config/config.go`. That is the point of the preset. + +**When it will save you nothing, which a first run often is.** All three of these are silent — the +numbers are simply zero: + +| Condition | Why | +|---|---| +| **You are not in a git repository** | Claude Code emits no environment snapshot, so there is no volatile tail to split. This is the common case for a casual trial, and `/context-guru:status` checks for it. | +| Your system prompt is under ~1,024 tokens | Below `minSplitTokens` the split is refused: the extra breakpoint slot costs more optionality than it recovers. | +| A non-Anthropic backend (vLLM, llm-d) | They match an implicit longest prefix and stop at the divergence by themselves. | + +And even in the good case, be calibrated about the size: the headline **−34.1%** figure comes from +a benchmark harness running tasks back-to-back inside the provider's 5-minute cache TTL. On this +project's own interactive traffic the measured figure is **$0.0298 across 1,127 sessions** — because +Claude Code captures the environment snapshot once per session, and 1,105 of 1,127 session starts +found the previous prefix already expired. The mechanism needs a second session inside five +minutes; humans mostly do not work that way. + +**Anthropic-family only.** `cachesplit` is a no-op against implicit prefix-cache backends +(vLLM, llm-d), which stop at the divergence by themselves. + +## Lifecycle + +The plugin installs a `SessionStart` hook that starts the proxy if it is not already running. +Three things about it worth knowing: + +- **It runs in every project, and does nothing in almost all of them.** The hook exits + immediately unless `ANTHROPIC_BASE_URL` names its port — so it acts only where you configured + routing. Remove the env key by hand and the hook stops firing on its own. +- **It is synchronous**, so the proxy is answering `/healthz` before your session's first + request goes out. It also fires on `clear`, `compact`, `resume` and `fork`, and is idempotent: + it never starts a second proxy. +- **The proxy exits by itself** after no requests and no keep-alive ping pending. **24h is the + value the plugin passes**, not the flag's default — `--idle-exit` defaults to `0`, meaning never, + because a gateway must not self-terminate. Liveness probes (`/healthz`, `/metrics`) deliberately + do not count as activity, so a monitoring loop cannot silently keep the proxy alive forever; an + open dashboard tab does count, because somebody is watching. + +A committed `.claude/settings.json` containing a hook prompts for trust when someone clones the +repo. That is correct behaviour, but it means "clone and go" is really "clone, approve, go". + +## Then + +- `/context-guru:status` — is it routed, is it up, and what has it saved. Reads `/stats`. +- Dashboard: `http://127.0.0.1:8787/dashboard/`. The four billed token tiers are where the cache + effect shows: tokens moving out of the premium cache-**creation** tier into the discounted + cache-**read** tier. Its database lives in `~/.local/state/context-guru/`, deliberately not in + your repository — the proxy's own default would write `./context-guru-dashboard.db` into whatever + directory it started in. +- Note `/stats` reports `acted: 0` and `saved_tokens: 0` even on a turn where the split worked: + `acted` counts components that removed content, and this one relocates a cache breakpoint. The + signals that do move are `components.cachesplit.verdict` and the billed tiers. +- `/context-guru:uninstall` — removes the one settings key (with a backup) and stops the proxy. + +## Troubleshooting + +**"Nothing happened after `/context-guru:install`."** The setting applies to a **new** session; +the one you ran it in already has its environment. Start a new session. + +**A prompt hangs with no output at all.** That is a dead proxy on a routed project — the failure +has no error message of its own. The `UserPromptSubmit` hook should catch it and say so; if it +cannot, start the proxy by hand +(`context-guru-proxy --listen 127.0.0.1:8787 --preset cache`) or remove +`env.ANTHROPIC_BASE_URL` from `.claude/settings.local.json` to get working immediately. + +**Upgrading.** `/context-guru:install` reports what is installed and does not replace it. To move to +a newer release, run the installer with `CONTEXT_GURU_UPGRADE=1`, or pin one with +`CONTEXT_GURU_VERSION=vX.Y.Z`. + +**Requests fail with a connection error.** The proxy is not running and this project is routed. +`/context-guru:status` will say so; the log is `${TMPDIR:-/tmp}/context-guru-proxy-.log`. +To get working again immediately, `/context-guru:uninstall`. + +**"The proxy binary is not on PATH."** The installer puts it in `~/.local/bin`. Add that to your +`PATH` — the session hook runs with your normal environment and cannot find it otherwise. + +**Port 8787 is taken.** Change it in the plugin's configuration. Do not use 4000; litellm +defaults to it, and the installer's default avoids the collision on purpose. + +**`--idle-exit` is refused at startup.** A threshold below roughly 5h34m (2× the store's default +entry lifetime) is rejected, because exiting clears in-memory cache state — including frozen +decisions, whose loss re-bills a whole prefix as cache creation. Raise the threshold, or raise +`store.ttl_seconds` if the short lifetime is deliberate. diff --git a/docs/how-to/use-with-claude-code.md b/docs/how-to/use-with-claude-code.md index 68862888..b586c5cc 100644 --- a/docs/how-to/use-with-claude-code.md +++ b/docs/how-to/use-with-claude-code.md @@ -3,6 +3,9 @@ Route [Claude Code](https://docs.claude.com/en/docs/claude-code) through context-guru with one environment variable — no changes to Claude Code itself. +**The plugin does all of this for you**, including installing the binary and choosing a scope: +[Install the Claude Code plugin](install-plugin.md). What follows is the same thing by hand. + ## You do not need an API key Setting `ANTHROPIC_BASE_URL` **without** a credential variable leaves your claude.ai login in @@ -57,6 +60,9 @@ Add to `.claude/settings.json` so you don't export anything by hand: Use `.claude/settings.local.json` instead if you do not want to commit it: a base URL pointing at `localhost` breaks Claude Code for everyone who clones the repo whenever the proxy is not running. +That is what the plugin writes by default, on port **8787** rather than 4000 — litellm's default is +4000, and a collision there is silent and confusing. + ## Keep the API key out of Claude Code Give the proxy the real key and hand Claude Code a placeholder; the proxy injects the diff --git a/docs/reference/presets.md b/docs/reference/presets.md index 06d82a13..0751cd41 100644 --- a/docs/reference/presets.md +++ b/docs/reference/presets.md @@ -12,7 +12,7 @@ taken exactly from the `presets` map in `config/config.go`. | `codesmart` | `format` → `textclean` → `searchfold` → `dedup` → `failed_run` → `cmdfilter` → `extract_llm` → `extract` → `linecap` → `cachesplit` | The SWE-bench-winning cache-aware config: structural offloaders + a cheap-model relevance-trimmer (`extract_llm`, routed to `CHEAP_MODEL`, gated so most turns make no model call) + deterministic `extract`. `extract_llm` no-ops (→ deterministic) when no cheap model is configured. **Changed 2026-08:** the lossless trio replaced `toon`, which acted 0 times on 5,752 production requests, and `linecap` was added. Re-measure before quoting the published SWE-bench numbers against it. | | `codesafe` | `format` → `textclean` → `searchfold` → `dedup` → `failed_run` → `cmdfilter` → `extract` → `collapse` → `linecap` → `cachesplit` | `codesmart` minus the LLM pass — **deterministic-only, zero model calls by policy**. The safe control / the choice when you don't want an LLM on the hot path. | | `off` | *(empty)* | Passthrough — no components. The baseline / A-B control. | -| `cache` | `cachesplit` | **The first-run preset** — the one to point a new evaluator at. The volatile-tail split and nothing else: no content dropped, no `<>` markers, no `context_guru_expand` tool added to requests, no model calls. Chosen so a stranger deciding whether to route their agent through a local proxy can verify the claim by reading one line of `config/config.go` rather than trusting four components. The savings claim is regime-dependent and the funnel's regime is the weak one: **−34.1% cost / 0% → 96.7% hit** is a benchmark harness running tasks back-to-back inside the provider's 5-minute TTL (and is one task measured three times), while this project's own interactive traffic yields **$0.0298 across 1,127 sessions** — 1,105 of 1,127 session starts read zero from cache. Zero outside a git repo, under the 1,024-token `minSplitTokens` floor, or on an implicit prefix-cache backend (vLLM, llm-d). See [dashboard](../dashboard.md#what-it-is-actually-worth-here-and-why-that-is-small) and [cacheinject](../components/cacheinject.md). | +| `cache` | `cachesplit` | **The first-run preset**, and what the [Claude Code plugin](../how-to/install-plugin.md) installs by default. The volatile-tail split and nothing else: no content dropped, no `<>` markers, no `context_guru_expand` tool added to requests, no model calls. Chosen so a stranger deciding whether to route their agent through a local proxy can verify the claim by reading one line of `config/config.go` rather than trusting four components. The savings claim is regime-dependent and the funnel's regime is the weak one: **−34.1% cost / 0% → 96.7% hit** is a benchmark harness running tasks back-to-back inside the provider's 5-minute TTL (and is one task measured three times), while this project's own interactive traffic yields **$0.0298 across 1,127 sessions** — 1,105 of 1,127 session starts read zero from cache. Zero outside a git repo, under the 1,024-token `minSplitTokens` floor, or on an implicit prefix-cache backend (vLLM, llm-d). See [dashboard](../dashboard.md#what-it-is-actually-worth-here-and-why-that-is-small) and [cacheinject](../components/cacheinject.md). | | `safe` | `format` → `textclean` → `searchfold` → `cachesplit` | Lossless only: repack JSON compactly and split the volatile system tail so the shared prefix stays cacheable. Zero risk of dropping content. | | `balanced` | `format` → `textclean` → `searchfold` → `dedup` → `failed_run` → `cmdfilter` → `linecap` → `cachesplit` | Lossless repack + conservative offloads (dedupe, drop superseded/failed runs, filter command noise) + the cache split. **Not recommended for agentic traffic** — it omits `mask`, the biggest lever there. | | `aggressive` | `format` → `textclean` → `searchfold` → `dedup` → `failed_run` → `cmdfilter` → `smartcrush` → `extract` → `extract_llm` → `linecap` → `cachesplit` | `balanced` plus `smartcrush` (crush long homogeneous arrays), deterministic `extract` (noise collapse), and `extract_llm` (cheap-model relevance trim) for deeper savings. | diff --git a/mkdocs.yml b/mkdocs.yml index 7604e210..08b105ad 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -100,6 +100,7 @@ nav: - Connect to the IBM service: get-started/connect-ibm-service.md - "Quickstart: proxy": get-started/quickstart-proxy.md - "Quickstart: compaction service": get-started/quickstart-compaction.md + - Install the Claude Code plugin: how-to/install-plugin.md - Use with Claude Code: how-to/use-with-claude-code.md - Build & evaluate locally: setup.md - Concepts: