From e7108380254211b5c902d2ce7ba413ff3cb139c1 Mon Sep 17 00:00:00 2001 From: Harshaneel Gokhale Date: Tue, 4 Aug 2026 11:56:03 -0700 Subject: [PATCH 01/18] docs: Spec the proxy-only image variant Design for a `:proxy` tag containing the translating proxy and poppler but no inference stack, for users who already run llama.cpp or a shared internal model server. The Go binary already accepts any OpenAI-compatible upstream, so the work is packaging plus the upstream credential the proxy currently cannot send. Also anchors the specs/ gitignore rule to the repo root. It had no leading slash, so it matched at any depth and would have silently swallowed this file. Co-Authored-By: Claude Opus 5 (1M context) --- .gitignore | 2 +- .../2026-08-04-proxy-only-image-design.md | 177 ++++++++++++++++++ 2 files changed, 178 insertions(+), 1 deletion(-) create mode 100644 docs/superpowers/specs/2026-08-04-proxy-only-image-design.md diff --git a/.gitignore b/.gitignore index adb7fe6..0b70a76 100644 --- a/.gitignore +++ b/.gitignore @@ -1,3 +1,3 @@ /localaik .cache/ -specs/ +/specs/ diff --git a/docs/superpowers/specs/2026-08-04-proxy-only-image-design.md b/docs/superpowers/specs/2026-08-04-proxy-only-image-design.md new file mode 100644 index 0000000..6a00dc6 --- /dev/null +++ b/docs/superpowers/specs/2026-08-04-proxy-only-image-design.md @@ -0,0 +1,177 @@ +# Proxy-only image (`:proxy`) + +**Date:** 2026-08-04 +**Status:** Approved, not yet implemented +**Scope:** One new published image variant plus the configuration needed to use it. + +## Problem + +localaik ships one kind of image: llama.cpp, a Gemma model, and the translating +proxy, welded together. The smallest published tag is 3.16 GB compressed. Almost +all of that is model data: the weights are 2.49 GB and the vision projector +another 0.85 GB, against ~130 MB for llama.cpp and ~8 MB for the proxy itself. + +Some users already run an inference server. It may be llama.cpp on their laptop, +or a shared internal deployment. Those users want only the translation layer: +something that accepts Gemini, OpenAI and Anthropic shaped requests and forwards +them to a server they already operate. Today they must pull 3.16 GB and run a +second model they will never call. + +The Go binary already supports this. `cmd/localaik/main.go` takes `--upstream` +for any OpenAI-compatible base URL and assumes nothing about llama.cpp or +locality. What is missing is a container image that omits the inference stack, +and the configuration to authenticate against a remote upstream. + +## Non-goals + +- Authenticating localaik's own callers. It remains a test double that accepts + and ignores client credentials. +- Any model download. That is the separate `:no-model` variant, specified later. +- Any change to `gemma3-4b`, `gemma3-12b` or `latest`. Those keep working + byte-identically. + +## Design + +### The image + +`Dockerfile` gains a third stage. Stage order matters: the llama.cpp stage stays +last so that a bare `docker build .` and the existing `make docker-build` keep +producing the full image. + +``` +FROM golang:1.25-alpine AS proxy-builder # exists, unchanged +FROM alpine AS proxy # new +FROM ghcr.io/ggml-org/llama.cpp@sha256:... # exists, stays last +``` + +The `proxy` stage installs `poppler-utils` and `ca-certificates`, copies the +binary, and runs it under `tini`. No entrypoint script: with configuration read +from the environment there is nothing for a shell to decide. + +`poppler-utils` is required, not optional. `main.go` constructs +`pdf.NewExecRenderer("pdftoppm")`, and without it every PDF request fails at +render time. Dropping it would save roughly 50 MB and silently remove a +documented feature, which is a bad trade against an image that is already about +50x smaller than today's. + +Expected size: 50-90 MB. To be measured during implementation, not asserted here. + +### Configuration + +`main.go` grows environment fallbacks for its two flags, following the pattern +`PORT` already sets for `--port`. + +| Variable | Flag | Default | Purpose | +| --- | --- | --- | --- | +| `LK_UPSTREAM` | `--upstream` | `http://127.0.0.1:8080/v1` | Base URL of the model server | +| `LK_UPSTREAM_AUTH_HEADER` | none | unset | Credential sent to upstream only | +| `PORT` | `--port` | `8090` | Listen port, unchanged | + +Precedence is flag over environment over default, so the full image's entrypoint +keeps working unchanged: it passes `--upstream` explicitly. + +`LK_UPSTREAM_AUTH_HEADER` holds a complete header line, for example +`Authorization: Bearer abc123`, rather than a bare token. This covers `Bearer`, +llama.cpp's `--api-key`, and any custom scheme without the proxy needing to know +which is in use. + +### Upstream authentication + +The proxy currently sends no credential upstream, deliberately. Three separate +places enforce that: + +- `cloneHeaders` strips `Authorization`, `X-Api-Key` and `X-Goog-Api-Key` from + passthrough requests. +- The Gemini and Anthropic handlers build fresh requests carrying only + `Content-Type` and `Accept`. +- `fetchUpstreamJSON` forwards no headers at all, and documents why. + +That is correct when upstream is `127.0.0.1:8080` inside the same container. +Against a remote server that requires a key, every request would 401. + +The credential is therefore injected in the HTTP client's transport rather than +at each call site. All upstream traffic already flows through `s.client.Do`, so a +`RoundTripper` wrapper applies the header to every request and cannot be +forgotten when a sixth upstream path is added later. + +Two properties must hold simultaneously, and both are tested: + +1. Credentials the caller sent are still stripped and never reach upstream. +2. The proxy's own credential is added to every upstream request. + +### Health and readiness + +`handleHealth` already probes upstream on every call and returns 503 when it is +unreachable, so it works unchanged against a remote server. `HEALTHCHECK +--start-period` drops from 60s to 5s in the `proxy` stage, since no model loads. + +### Publishing + +`release.yml` gains a matrix entry carrying a build target. Existing entries +default to the full image. Tags follow the current scheme minus `latest`: +`proxy` and `vX.Y.Z-proxy`. + +`:proxy` must not become `latest`. Anyone pulling `latest` today gets a +self-contained container, and silently turning that into one requiring an +external server would break them. + +## Security + +`:proxy` has a materially different risk profile from every existing tag, and +the README must say so. + +In the baked images llama.cpp binds `127.0.0.1` inside the container. The only +reachable service is the proxy, and behind it a disposable local model. In +`:proxy` the container becomes a network hop into infrastructure the operator +cares about. Because localaik accepts and ignores client credentials by design, +anyone who can reach port 8090 can drive the upstream server unauthenticated. + +The mitigation is documentation, not code: bind to localhost, and do not publish +the port on a shared network. Adding client authentication is explicitly +rejected, because it would invite treating a test double as production +infrastructure. + +`LK_UPSTREAM_AUTH_HEADER` is a secret in an environment variable. That is +acceptable here: the container legitimately needs it, it travels as a request +header rather than a process argument, and it is never echoed to logs or +forwarded to callers. Implementation must not log its value, and must not enable +shell tracing anywhere it is in scope. + +## Testing + +Existing tests need no changes. They already stub upstream through an +`http.RoundTripper`, which is exactly the seam this feature uses. + +New unit tests: + +- Flag beats environment beats default, for both `--upstream` and `--port`. +- `LK_UPSTREAM_AUTH_HEADER` reaches all four upstream paths: chat completions, + `/tokenize`, models list, and the Anthropic messages route. +- Client `Authorization`, `X-Api-Key` and `X-Goog-Api-Key` are still stripped + when the proxy's own credential is configured, verified together in one test so + the two behaviours cannot silently merge. +- No credential is sent when `LK_UPSTREAM_AUTH_HEADER` is unset. + +Image test, behind the existing `docker_integration` tag: build `--target +proxy`, run it against a stub OpenAI-compatible server, and confirm a Gemini, an +OpenAI and an Anthropic request each round-trip. This is fast, since no model is +involved. + +## Verification before merge + +Two claims in this spec are estimates and must be measured: + +1. Final image size. Recorded in the README once known. +2. That `pdftoppm` from alpine's `poppler-utils` behaves the same as the + Debian-based full image for the PDF-to-PNG path. The existing PDF tests + should be run inside the built `:proxy` image, not only on the host. + +## Follow-up + +`:no-model`, a variant keeping llama.cpp but fetching the model from +`LK_MODEL_URL` at startup, is a separate change. Investigation during this design +established that the pinned llama.cpp build already provides `--model-url`, +`--mmproj-url`, `--hf-token`, `LLAMA_CACHE` and `--offline`, so that work is +mostly packaging and documentation rather than download logic. It also needs a +startup-order change so the proxy answers `/health` during a long download, which +`:proxy` does not require. From da4b1dc654f3b8d1dd11a584390330a0f50e9fc8 Mon Sep 17 00:00:00 2001 From: Harshaneel Gokhale Date: Tue, 4 Aug 2026 12:03:08 -0700 Subject: [PATCH 02/18] docs: Add implementation plan for the proxy-only image Eight tasks, TDD throughout. Alpine base, poppler-utils, tini and busybox wget were verified against a real build: pdftoppm 25.12.0 is present and the base plus packages measures 35 MB. Co-Authored-By: Claude Opus 5 (1M context) --- .../plans/2026-08-04-proxy-only-image.md | 1037 +++++++++++++++++ 1 file changed, 1037 insertions(+) create mode 100644 docs/superpowers/plans/2026-08-04-proxy-only-image.md diff --git a/docs/superpowers/plans/2026-08-04-proxy-only-image.md b/docs/superpowers/plans/2026-08-04-proxy-only-image.md new file mode 100644 index 0000000..5b0a772 --- /dev/null +++ b/docs/superpowers/plans/2026-08-04-proxy-only-image.md @@ -0,0 +1,1037 @@ +# Proxy-only image (`:proxy`) Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Publish a `:proxy` image containing only the translating proxy and `pdftoppm`, for users who already run an OpenAI-compatible model server. + +**Architecture:** Three changes, each independently testable. The Go binary learns two environment fallbacks and gains an optional upstream credential injected at the HTTP transport layer, so no upstream call site changes. The `Dockerfile` gains a third build stage. CI publishes it as a new matrix entry. + +**Tech Stack:** Go 1.25, standard library only. Docker multi-stage build, alpine base. GitHub Actions with `docker/build-push-action@v6`. + +## Global Constraints + +- Go version: `1.25` (from `go.mod`). Standard library only; add no dependencies. +- Run `gofmt -w` on every Go file touched. `make lint` runs `gofmt -l` and `go vet ./...` and must stay clean. +- No em-dashes in any file, including comments, docs and commit messages. +- Comments: default to zero. One line only when the WHY is non-obvious. Never restate the code. Never explain rejected alternatives or review feedback. +- The llama.cpp stage must remain the LAST stage in `Dockerfile`, so `docker build .` and `make docker-build` keep producing the full image. +- `:proxy` must never be tagged `latest`. +- `LK_UPSTREAM_AUTH_HEADER` is a secret. Never log its value. Never add `set -x` anywhere it is in scope. +- Existing behaviour must not change: client credentials (`Authorization`, `X-Api-Key`, `X-Goog-Api-Key`) are still stripped and never forwarded upstream. + +## File Structure + +| File | Responsibility | +| --- | --- | +| `internal/server/upstreamauth.go` (create) | The `RoundTripper` that adds the proxy's own credential to upstream requests. Isolated so the security-sensitive logic is one small readable unit. | +| `internal/server/upstreamauth_test.go` (create) | Tests for that transport in isolation. | +| `internal/server/server.go` (modify) | `Config` gains `UpstreamAuthHeader`; `New` wraps the client transport when it is set. | +| `internal/server/auth_integration_test.go` (create) | Proves the header reaches all four upstream paths while client credentials are still stripped. | +| `cmd/localaik/main.go` (modify) | Environment fallbacks for `--upstream`, plus reading `LK_UPSTREAM_AUTH_HEADER`. | +| `cmd/localaik/main_test.go` (create) | Flag over environment over default precedence. | +| `Dockerfile` (modify) | New `proxy` stage inserted before the llama.cpp stage. | +| `.github/workflows/release.yml` (modify) | Matrix entry with a build target. | +| `Makefile` (modify) | `docker-build-proxy` target for local verification. | +| `integration/proxy_image_test.go` (create) | Behind `docker_integration`: builds and exercises the image. | +| `README.md` (modify) | Tag table row, configuration, and the security warning. | + +--- + +### Task 1: Upstream auth transport + +Adds the credential-injecting `RoundTripper` and wires it into `server.New`. Nothing consumes it yet. + +**Files:** +- Create: `internal/server/upstreamauth.go` +- Create: `internal/server/upstreamauth_test.go` +- Modify: `internal/server/server.go:18-22` (Config), `internal/server/server.go:46-49` (client setup) + +**Interfaces:** +- Consumes: nothing. +- Produces: + - `server.Config.UpstreamAuthHeader string` (new field, optional) + - `func newUpstreamAuthTransport(base http.RoundTripper, header string) http.RoundTripper` + - Header format is a full header line, `"Name: value"`, split on the first colon. + +- [ ] **Step 1: Write the failing test** + +Create `internal/server/upstreamauth_test.go`: + +```go +package server + +import ( + "net/http" + "net/http/httptest" + "testing" +) + +type capturingTransport struct { + seen http.Header +} + +func (c *capturingTransport) RoundTrip(req *http.Request) (*http.Response, error) { + c.seen = req.Header.Clone() + recorder := httptest.NewRecorder() + recorder.WriteHeader(http.StatusOK) + return recorder.Result(), nil +} + +func TestUpstreamAuthTransportAddsHeader(t *testing.T) { + capture := &capturingTransport{} + transport := newUpstreamAuthTransport(capture, "Authorization: Bearer secret") + + req := httptest.NewRequest(http.MethodGet, "http://upstream.test/v1/models", nil) + if _, err := transport.RoundTrip(req); err != nil { + t.Fatalf("RoundTrip returned error: %v", err) + } + + if got := capture.seen.Get("Authorization"); got != "Bearer secret" { + t.Fatalf("Authorization = %q, want %q", got, "Bearer secret") + } +} + +func TestUpstreamAuthTransportTrimsWhitespace(t *testing.T) { + capture := &capturingTransport{} + transport := newUpstreamAuthTransport(capture, " X-Api-Key : abc123 ") + + req := httptest.NewRequest(http.MethodGet, "http://upstream.test/v1/models", nil) + if _, err := transport.RoundTrip(req); err != nil { + t.Fatalf("RoundTrip returned error: %v", err) + } + + if got := capture.seen.Get("X-Api-Key"); got != "abc123" { + t.Fatalf("X-Api-Key = %q, want %q", got, "abc123") + } +} + +func TestUpstreamAuthTransportIgnoresMalformedHeader(t *testing.T) { + for _, header := range []string{"", " ", "NoColonHere", ": novalue", "Name:"} { + capture := &capturingTransport{} + transport := newUpstreamAuthTransport(capture, header) + + req := httptest.NewRequest(http.MethodGet, "http://upstream.test/v1/models", nil) + if _, err := transport.RoundTrip(req); err != nil { + t.Fatalf("RoundTrip returned error for %q: %v", header, err) + } + if len(capture.seen) != 0 { + t.Fatalf("header %q produced %v, want none", header, capture.seen) + } + } +} + +func TestUpstreamAuthTransportDoesNotMutateCallerRequest(t *testing.T) { + capture := &capturingTransport{} + transport := newUpstreamAuthTransport(capture, "Authorization: Bearer secret") + + req := httptest.NewRequest(http.MethodGet, "http://upstream.test/v1/models", nil) + if _, err := transport.RoundTrip(req); err != nil { + t.Fatalf("RoundTrip returned error: %v", err) + } + + if req.Header.Get("Authorization") != "" { + t.Fatal("RoundTrip mutated the caller's request") + } +} +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `go test ./internal/server/ -run UpstreamAuthTransport -v` +Expected: FAIL, `undefined: newUpstreamAuthTransport` + +- [ ] **Step 3: Write minimal implementation** + +Create `internal/server/upstreamauth.go`: + +```go +package server + +import ( + "net/http" + "strings" +) + +type upstreamAuthTransport struct { + base http.RoundTripper + name string + value string +} + +// newUpstreamAuthTransport returns base unchanged when header is not a usable +// "Name: value" line, so a misconfigured value cannot silently drop requests. +func newUpstreamAuthTransport(base http.RoundTripper, header string) http.RoundTripper { + if base == nil { + base = http.DefaultTransport + } + + name, value, found := strings.Cut(header, ":") + name = strings.TrimSpace(name) + value = strings.TrimSpace(value) + if !found || name == "" || value == "" { + return base + } + + return &upstreamAuthTransport{base: base, name: name, value: value} +} + +func (t *upstreamAuthTransport) RoundTrip(req *http.Request) (*http.Response, error) { + clone := req.Clone(req.Context()) + clone.Header.Set(t.name, t.value) + return t.base.RoundTrip(clone) +} +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `go test ./internal/server/ -run UpstreamAuthTransport -v` +Expected: PASS, 4 tests + +- [ ] **Step 5: Wire it into Config** + +In `internal/server/server.go`, add the field to `Config`: + +```go +type Config struct { + UpstreamBaseURL string + UpstreamAuthHeader string + HTTPClient *http.Client + PDFRenderer pdf.Renderer +} +``` + +Then in `New`, replace the existing client setup block: + +```go + client := cfg.HTTPClient + if client == nil { + client = &http.Client{} + } +``` + +with: + +```go + client := cfg.HTTPClient + if client == nil { + client = &http.Client{} + } + if cfg.UpstreamAuthHeader != "" { + clone := *client + clone.Transport = newUpstreamAuthTransport(clone.Transport, cfg.UpstreamAuthHeader) + client = &clone + } +``` + +- [ ] **Step 6: Run the full server suite** + +Run: `go test ./internal/server/ -count=1` +Expected: PASS. Copying the client rather than mutating it means every existing test that passes its own `HTTPClient` is unaffected. + +- [ ] **Step 7: Format, lint, commit** + +```bash +gofmt -w internal/server/upstreamauth.go internal/server/upstreamauth_test.go internal/server/server.go +make lint +git add internal/server/upstreamauth.go internal/server/upstreamauth_test.go internal/server/server.go +git commit -m "feat: Add optional upstream auth header to the proxy + +Injected at the transport layer so every upstream request carries it without +each call site opting in." +``` + +--- + +### Task 2: Prove the header reaches every upstream path + +Task 1 tested the transport alone. This proves the wiring covers all four upstream endpoints and that client credentials are still stripped. + +**Files:** +- Create: `internal/server/auth_integration_test.go` + +**Interfaces:** +- Consumes: `server.Config.UpstreamAuthHeader` from Task 1; `roundTripHandler` from `internal/server/server_test.go:147`; `newTestServer` from `internal/server/meta_test.go:17`. +- Produces: nothing. + +- [ ] **Step 1: Write the failing test** + +Create `internal/server/auth_integration_test.go`: + +```go +package server + +import ( + "bytes" + "context" + "net/http" + "net/http/httptest" + "testing" + + "github.com/harshaneel/localaik/internal/pdf" + openaip "github.com/harshaneel/localaik/internal/protocol/openai" +) + +// Every upstream route must carry the proxy's credential and none of the +// caller's. +func TestUpstreamAuthHeaderReachesEveryUpstreamPath(t *testing.T) { + cases := []struct { + name string + method string + path string + body string + }{ + {"openai_chat", http.MethodPost, "/v1/chat/completions", `{"model":"m","messages":[]}`}, + {"openai_models", http.MethodGet, "/v1/models", ""}, + {"gemini_generate", http.MethodPost, "/v1beta/models/m:generateContent", `{"contents":[{"role":"user","parts":[{"text":"hi"}]}]}`}, + {"gemini_count_tokens", http.MethodPost, "/v1beta/models/m:countTokens", `{"contents":[{"parts":[{"text":"hi"}]}]}`}, + {"anthropic_messages", http.MethodPost, "/v1/messages", `{"max_tokens":8,"messages":[{"role":"user","content":"hi"}]}`}, + {"anthropic_count_tokens", http.MethodPost, "/v1/messages/count_tokens", `{"messages":[{"role":"user","content":"hi"}]}`}, + } + + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + var called bool + var seenAuth, seenClientAuth, seenAPIKey, seenGoogKey string + + upstream := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + called = true + seenAuth = r.Header.Get("X-Proxy-Token") + seenClientAuth = r.Header.Get("Authorization") + seenAPIKey = r.Header.Get("X-Api-Key") + seenGoogKey = r.Header.Get("X-Goog-Api-Key") + + switch r.URL.Path { + case "/tokenize": + writeJSON(w, http.StatusOK, map[string]any{"tokens": []int{1, 2}}) + case "/v1/models": + writeJSON(w, http.StatusOK, openaip.ModelList{Object: "list", Data: []openaip.Model{{ID: "m"}}}) + default: + writeJSON(w, http.StatusOK, openaip.ChatCompletionResponse{ + Choices: []openaip.Choice{{Message: openaip.Message{Content: "ok"}, FinishReason: "stop"}}, + }) + } + }) + + srv, err := New(Config{ + UpstreamBaseURL: "http://upstream.test/v1", + UpstreamAuthHeader: "X-Proxy-Token: upstream-secret", + HTTPClient: &http.Client{Transport: roundTripHandler{handler: upstream}}, + PDFRenderer: pdf.RendererFunc(func(context.Context, []byte) ([][]byte, error) { return nil, nil }), + }) + if err != nil { + t.Fatalf("New returned error: %v", err) + } + + var reader *bytes.Buffer + if tc.body != "" { + reader = bytes.NewBufferString(tc.body) + } else { + reader = bytes.NewBuffer(nil) + } + req := httptest.NewRequest(tc.method, tc.path, reader) + req.Header.Set("Authorization", "Bearer client-secret") + req.Header.Set("X-Api-Key", "client-anthropic-key") + req.Header.Set("X-Goog-Api-Key", "client-google-key") + + rec := httptest.NewRecorder() + srv.ServeHTTP(rec, req) + + if rec.Code != http.StatusOK { + t.Fatalf("status = %d, want 200; body=%s", rec.Code, rec.Body.String()) + } + if !called { + t.Fatal("upstream was never called, so the header check proves nothing") + } + if seenAuth != "upstream-secret" { + t.Fatalf("X-Proxy-Token = %q, want the proxy credential", seenAuth) + } + if seenClientAuth != "" || seenAPIKey != "" || seenGoogKey != "" { + t.Fatalf("client credentials leaked upstream: auth=%q apikey=%q googkey=%q", seenClientAuth, seenAPIKey, seenGoogKey) + } + }) + } +} + +func TestNoUpstreamAuthHeaderWhenUnset(t *testing.T) { + var seen http.Header + + upstream := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + seen = r.Header.Clone() + writeJSON(w, http.StatusOK, openaip.ChatCompletionResponse{ + Choices: []openaip.Choice{{Message: openaip.Message{Content: "ok"}, FinishReason: "stop"}}, + }) + }) + + srv := newTestServer(t, upstream) + + req := httptest.NewRequest(http.MethodPost, "/v1/messages", bytes.NewBufferString(`{"max_tokens":8,"messages":[{"role":"user","content":"hi"}]}`)) + rec := httptest.NewRecorder() + srv.ServeHTTP(rec, req) + + if rec.Code != http.StatusOK { + t.Fatalf("status = %d, want 200; body=%s", rec.Code, rec.Body.String()) + } + if got := seen.Get("Authorization"); got != "" { + t.Fatalf("Authorization = %q, want none when no credential is configured", got) + } +} + +// The Gemini streaming route builds its own request; confirm the credential is +// present there too. +func TestUpstreamAuthHeaderOnStreamingRoute(t *testing.T) { + var seen string + + upstream := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + seen = r.Header.Get("X-Proxy-Token") + w.Header().Set("Content-Type", "text/event-stream") + w.WriteHeader(http.StatusOK) + _, _ = w.Write([]byte("data: {\"choices\":[{\"delta\":{\"content\":\"hi\"},\"finish_reason\":\"stop\"}]}\n\ndata: [DONE]\n\n")) + }) + + srv, err := New(Config{ + UpstreamBaseURL: "http://upstream.test/v1", + UpstreamAuthHeader: "X-Proxy-Token: upstream-secret", + HTTPClient: &http.Client{Transport: roundTripHandler{handler: upstream}}, + PDFRenderer: pdf.RendererFunc(func(context.Context, []byte) ([][]byte, error) { return nil, nil }), + }) + if err != nil { + t.Fatalf("New returned error: %v", err) + } + + body := `{"contents":[{"role":"user","parts":[{"text":"hi"}]}]}` + req := httptest.NewRequest(http.MethodPost, "/v1beta/models/m:streamGenerateContent", bytes.NewBufferString(body)) + rec := httptest.NewRecorder() + srv.ServeHTTP(rec, req) + + if seen != "upstream-secret" { + t.Fatalf("X-Proxy-Token = %q on the streaming route, want the proxy credential", seen) + } +} +``` + +- [ ] **Step 2: Run the tests** + +Run: `go test ./internal/server/ -run 'UpstreamAuthHeader|NoUpstreamAuthHeader' -v` +Expected: PASS. Task 1 already made this work; these tests exist to lock the behaviour against future call sites. + +If any subtest fails with the credential missing, the cause is an upstream request that bypasses `s.client`. Find it and route it through `s.client` rather than weakening the test. + +- [ ] **Step 3: Format, lint, commit** + +```bash +gofmt -w internal/server/auth_integration_test.go +make lint +go test ./internal/server/ -count=1 +git add internal/server/auth_integration_test.go +git commit -m "test: Cover upstream auth on every upstream route + +Locks both halves at once: the proxy credential is added, the caller's is not +forwarded." +``` + +--- + +### Task 3: Environment fallbacks in main.go + +**Files:** +- Modify: `cmd/localaik/main.go:14-32` +- Create: `cmd/localaik/main_test.go` + +**Interfaces:** +- Consumes: `server.Config.UpstreamAuthHeader` from Task 1. +- Produces: + - `func resolveFlagDefault(envName, fallback string) string` + - Environment names: `LK_UPSTREAM`, `LK_UPSTREAM_AUTH_HEADER`, existing `PORT`. + +- [ ] **Step 1: Write the failing test** + +Create `cmd/localaik/main_test.go`: + +```go +package main + +import "testing" + +func TestResolveFlagDefaultPrefersEnv(t *testing.T) { + t.Setenv("LK_TEST_VALUE", "from-env") + + if got := resolveFlagDefault("LK_TEST_VALUE", "fallback"); got != "from-env" { + t.Fatalf("resolveFlagDefault = %q, want from-env", got) + } +} + +func TestResolveFlagDefaultFallsBack(t *testing.T) { + t.Setenv("LK_TEST_VALUE", "") + + if got := resolveFlagDefault("LK_TEST_VALUE", "fallback"); got != "fallback" { + t.Fatalf("resolveFlagDefault = %q, want fallback", got) + } +} + +func TestResolveFlagDefaultUnsetFallsBack(t *testing.T) { + if got := resolveFlagDefault("LK_DEFINITELY_UNSET_VALUE", "fallback"); got != "fallback" { + t.Fatalf("resolveFlagDefault = %q, want fallback", got) + } +} +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `go test ./cmd/localaik/ -v` +Expected: FAIL, `undefined: resolveFlagDefault` + +- [ ] **Step 3: Write minimal implementation** + +Replace the body of `main.go` up to and including the `server.New` call: + +```go +func resolveFlagDefault(envName, fallback string) string { + if value := os.Getenv(envName); value != "" { + return value + } + return fallback +} + +func main() { + port := flag.String("port", resolveFlagDefault("PORT", "8090"), "port to listen on") + upstream := flag.String("upstream", resolveFlagDefault("LK_UPSTREAM", "http://127.0.0.1:8080/v1"), "upstream OpenAI-compatible base URL") + flag.Parse() + + handler, err := server.New(server.Config{ + UpstreamBaseURL: *upstream, + UpstreamAuthHeader: os.Getenv("LK_UPSTREAM_AUTH_HEADER"), + HTTPClient: &http.Client{}, + PDFRenderer: pdf.NewExecRenderer("pdftoppm"), + }) + if err != nil { + log.Fatalf("localaik: %v", err) + } +``` + +Leave the rest of `main` unchanged. Delete the old `defaultPort` block that this replaces. + +- [ ] **Step 4: Run test to verify it passes** + +Run: `go test ./cmd/localaik/ -v` +Expected: PASS, 3 tests + +- [ ] **Step 5: Verify flag still beats environment** + +Run: + +```bash +LK_UPSTREAM=http://from-env:9999/v1 go run ./cmd/localaik --upstream http://from-flag:1111/v1 --port 18099 & +sleep 2 +curl -s -o /dev/null -w '%{http_code}\n' http://127.0.0.1:18099/health +kill %1 +``` + +Expected: `503`. A 503 proves it started and tried to reach an upstream. Then confirm the flag won by checking the process was pointed at `from-flag`: rerun without the flag and confirm it still starts. There is no endpoint that echoes the upstream, so this step verifies startup only; precedence itself is covered by the unit tests. + +- [ ] **Step 6: Format, lint, commit** + +```bash +gofmt -w cmd/localaik/main.go cmd/localaik/main_test.go +make lint +go test ./cmd/... ./internal/... -count=1 +git add cmd/localaik/main.go cmd/localaik/main_test.go +git commit -m "feat: Read upstream and auth header from the environment + +Follows the pattern PORT already set, so the container needs no shell wrapper." +``` + +--- + +### Task 4: The `proxy` Dockerfile stage + +**Files:** +- Modify: `Dockerfile` (insert a stage between line 5 and line 9) +- Modify: `Makefile:20-27` area (add a target) + +**Interfaces:** +- Consumes: the `proxy-builder` stage that already exists at `Dockerfile:1`. +- Produces: build target named `proxy`; image entrypoint runs `localaik` under `tini`. + +- [ ] **Step 1: Add the stage** + +In `Dockerfile`, immediately after the `proxy-builder` stage (after line 5) and before the `FROM ghcr.io/ggml-org/llama.cpp@sha256:...` line, insert: + +```dockerfile +FROM alpine:3 AS proxy +RUN apk add --no-cache ca-certificates poppler-utils tini +COPY --from=proxy-builder /out/localaik /usr/local/bin/localaik +ENV PORT=8090 +HEALTHCHECK --interval=5s --timeout=3s --start-period=5s \ + CMD wget -q -O - "http://127.0.0.1:${PORT:-8090}/health" >/dev/null 2>&1 || exit 1 +EXPOSE 8090 +ENTRYPOINT ["tini", "--", "localaik"] +``` + +Two notes, both verified against a real build of this stage: + +`wget` rather than `curl`, because alpine's busybox already provides `wget` and this avoids installing curl solely for the healthcheck. The llama.cpp stage keeps using `curl`, which it already installs. + +`alpine:3` is a moving tag. The repo pins the llama.cpp base by digest, so pinning this one by digest is more consistent. Resolve it during implementation with `docker inspect alpine:3 --format '{{index .RepoDigests 0}}'` and use that. A moving tag is acceptable if you prefer, since nothing here depends on a specific alpine version. + +Confirmed present in `alpine:3` at time of writing: `pdftoppm` 25.12.0 from `poppler-utils`, `tini`, and busybox `wget`. Base plus these three packages measures 35 MB before the binary is copied in. + +- [ ] **Step 2: Confirm the llama.cpp stage is still last** + +Run: `grep -n '^FROM' Dockerfile` +Expected: three lines, with `ghcr.io/ggml-org/llama.cpp` on the last one. + +- [ ] **Step 3: Confirm the default build is unchanged** + +Run: `docker build -t localaik:default-check . && docker image inspect localaik:default-check --format '{{.Size}}'` +Expected: a size over 3000000000, proving the default target is still the full image. + +- [ ] **Step 4: Build the proxy image and record its size** + +Run: + +```bash +docker build --target proxy -t localaik:proxy-check . +docker image inspect localaik:proxy-check --format '{{.Size}}' | awk '{printf "%.0f MB\n", $1/1000000}' +``` + +Expected: roughly 43 MB. The base plus packages measures 35 MB and the static binary adds about 8 MB. Anything above 60 MB means something unintended got copied in. Write the measured number down; Task 7 puts it in the README. + +- [ ] **Step 5: Verify pdftoppm is present and executable** + +Run: `docker run --rm --entrypoint pdftoppm localaik:proxy-check -v` +Expected: `pdftoppm version 25.12.0` or later, printed to stderr. `pdftoppm -v` exits non-zero on some builds while still printing the version, so treat printed output as success. + +- [ ] **Step 6: Verify the binary starts and reports not-ready** + +Run: + +```bash +docker run -d --name proxy-smoke -p 18098:8090 localaik:proxy-check +sleep 3 +curl -s -o /dev/null -w '%{http_code}\n' http://127.0.0.1:18098/health +docker rm -f proxy-smoke +``` + +Expected: `503`. There is no upstream, so 503 is correct and proves the server is listening. + +- [ ] **Step 7: Add the Makefile target** + +In `Makefile`, add `docker-build-proxy` to the `.PHONY` list on line 12, add a help line in the `help` target after the `docker-build` line: + +``` + 'make docker-build-proxy Build the proxy-only image' \ +``` + +and add the target after `docker-build`: + +```makefile +docker-build-proxy: + @docker build --target proxy -t "$(IMAGE)-proxy" . +``` + +- [ ] **Step 8: Verify the target works** + +Run: `make docker-build-proxy && docker images --format '{{.Repository}}:{{.Tag}}' | grep proxy` +Expected: the tagged image is listed. + +- [ ] **Step 9: Commit** + +```bash +git add Dockerfile Makefile +git commit -m "feat: Add a proxy-only Dockerfile stage + +alpine plus poppler-utils and the binary, no inference stack. The llama.cpp +stage stays last so the default build is unchanged." +``` + +--- + +### Task 5: Image integration test + +**Files:** +- Create: `integration/proxy_image_test.go` + +**Interfaces:** +- Consumes: the `proxy` build target from Task 4. +- Produces: nothing. + +- [ ] **Step 1: Write the test** + +Create `integration/proxy_image_test.go`: + +```go +//go:build docker_integration + +package integration + +import ( + "encoding/json" + "fmt" + "net" + "net/http" + "net/http/httptest" + "os/exec" + "strings" + "testing" + "time" +) + +// Exercises the built proxy image against a stub upstream, proving all three +// protocol surfaces round-trip without an inference stack in the container. +func TestProxyImageRoundTripsAllProtocols(t *testing.T) { + image := "localaik:proxy-integration" + + build := exec.Command("docker", "build", "--target", "proxy", "-t", image, "..") + if out, err := build.CombinedOutput(); err != nil { + t.Fatalf("docker build failed: %v\n%s", err, out) + } + + var seenAuth string + stub := httptest.NewUnstartedServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + seenAuth = r.Header.Get("X-Proxy-Token") + w.Header().Set("Content-Type", "application/json") + if strings.HasSuffix(r.URL.Path, "/tokenize") { + _, _ = w.Write([]byte(`{"tokens":[1,2,3]}`)) + return + } + _, _ = w.Write([]byte(`{"choices":[{"index":0,"message":{"role":"assistant","content":"stubbed"},"finish_reason":"stop"}]}`)) + })) + + listener, err := net.Listen("tcp", "0.0.0.0:0") + if err != nil { + t.Fatalf("listen: %v", err) + } + stub.Listener = listener + stub.Start() + defer stub.Close() + + stubPort := listener.Addr().(*net.TCPAddr).Port + upstream := fmt.Sprintf("http://host.docker.internal:%d/v1", stubPort) + + run := exec.Command("docker", "run", "-d", "--name", "proxy-integration", + "--add-host", "host.docker.internal:host-gateway", + "-p", "18097:8090", + "-e", "LK_UPSTREAM="+upstream, + "-e", "LK_UPSTREAM_AUTH_HEADER=X-Proxy-Token: integration-secret", + image) + if out, err := run.CombinedOutput(); err != nil { + t.Fatalf("docker run failed: %v\n%s", err, out) + } + defer exec.Command("docker", "rm", "-f", "proxy-integration").Run() + + waitForHealth(t, "http://127.0.0.1:18097/health") + + cases := []struct { + name string + path string + body string + }{ + {"openai", "/v1/chat/completions", `{"model":"m","messages":[{"role":"user","content":"hi"}]}`}, + {"gemini", "/v1beta/models/m:generateContent", `{"contents":[{"role":"user","parts":[{"text":"hi"}]}]}`}, + {"anthropic", "/v1/messages", `{"max_tokens":16,"messages":[{"role":"user","content":"hi"}]}`}, + } + + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + resp, err := http.Post("http://127.0.0.1:18097"+tc.path, "application/json", strings.NewReader(tc.body)) + if err != nil { + t.Fatalf("post: %v", err) + } + defer resp.Body.Close() + + if resp.StatusCode != http.StatusOK { + t.Fatalf("status = %d, want 200", resp.StatusCode) + } + var decoded map[string]any + if err := json.NewDecoder(resp.Body).Decode(&decoded); err != nil { + t.Fatalf("decode: %v", err) + } + if len(decoded) == 0 { + t.Fatal("empty response body") + } + }) + } + + if seenAuth != "integration-secret" { + t.Fatalf("upstream saw X-Proxy-Token = %q, want integration-secret", seenAuth) + } +} + +func waitForHealth(t *testing.T, url string) { + t.Helper() + for i := 0; i < 60; i++ { + resp, err := http.Get(url) + if err == nil { + resp.Body.Close() + if resp.StatusCode == http.StatusOK { + return + } + } + time.Sleep(time.Second) + } + t.Fatalf("never became healthy: %s", url) +} +``` + +- [ ] **Step 2: Run it** + +Run: `go test -count=1 -tags=docker_integration ./integration -run ProxyImage -v` +Expected: PASS, 3 subtests. Requires a running Docker daemon. + +If `host.docker.internal` does not resolve on Linux, the `--add-host ...:host-gateway` flag is what makes it work; confirm the Docker version supports it with `docker --version` (needs 20.10 or newer). + +- [ ] **Step 3: Confirm it does not run in the default suite** + +Run: `go test ./integration/ -count=1` +Expected: PASS without building any image, since the file is behind the `docker_integration` tag. + +- [ ] **Step 4: Commit** + +```bash +gofmt -w integration/proxy_image_test.go +make lint +git add integration/proxy_image_test.go +git commit -m "test: Exercise the built proxy image on all three protocols" +``` + +--- + +### Task 6: Publish the tag + +**Files:** +- Modify: `.github/workflows/release.yml:52-66` (matrix), `:96-107` (build step) + +**Interfaces:** +- Consumes: the `proxy` build target from Task 4. +- Produces: DockerHub tags `proxy` and `-proxy`. + +- [ ] **Step 1: Add a target to the existing matrix entries** + +In the `docker` job's matrix, add `target: ""` to both existing entries, so all entries carry the same keys: + +```yaml + - variant: gemma3-4b + latest: true + target: "" + model_url: https://huggingface.co/lmstudio-community/gemma-3-4b-it-GGUF/resolve/c536c4707e747055eecad7da65d46b6fb0ebaa79/gemma-3-4b-it-Q4_K_M.gguf +``` + +Do the same for `gemma3-12b`. Leave every existing URL and checksum untouched. + +- [ ] **Step 2: Add the proxy entry** + +Append to the matrix, after the `gemma3-12b` entry: + +```yaml + - variant: proxy + latest: false + target: proxy + model_url: "" + model_sha256: "" + mmproj_url: "" + mmproj_sha256: "" +``` + +The empty model values are required because every matrix entry must define the same keys for the build-args block to render. + +- [ ] **Step 3: Pass the target to the build** + +In the `Build and push image` step, add a `target` key above `platforms`: + +```yaml + context: . + file: ./Dockerfile + target: ${{ matrix.target }} + platforms: linux/amd64,linux/arm64 +``` + +An empty `target` means the default final stage, which is the llama.cpp one. That preserves the existing images exactly. + +- [ ] **Step 4: Verify the workflow parses** + +Run: `python3 -c "import sys,yaml;yaml.safe_load(open('.github/workflows/release.yml'))" 2>/dev/null || docker run --rm -v "$PWD:/w" -w /w mikefarah/yq:4 '.jobs.docker.strategy.matrix.include | length' .github/workflows/release.yml` +Expected: `3`, or no output and exit 0 from the Python check. If neither tool is available, run `gh workflow view ci-release` after pushing and confirm it is not reporting a parse error. + +- [ ] **Step 5: Confirm latest is not applied to proxy** + +Run: `grep -A3 'variant: proxy' .github/workflows/release.yml | grep latest` +Expected: `latest: false` + +- [ ] **Step 6: Commit** + +```bash +git add .github/workflows/release.yml +git commit -m "ci: Publish the proxy image variant + +Adds a build target to the matrix. Empty target keeps the existing entries on +the default final stage." +``` + +--- + +### Task 7: Document it + +**Files:** +- Modify: `README.md` at the Docker tags table (line 129 area), and a new configuration section after it + +**Interfaces:** +- Consumes: the measured image size from Task 4 Step 4. +- Produces: nothing. + +- [ ] **Step 1: Add the tag table row** + +In the `## Docker tags` table, add after the `gemma3-12b` row. Replace `` with the number from Task 4 Step 4: + +``` +| `proxy` | none (you supply) | ~ MB | +``` + +Then extend the sentence below the table: + +``` +Version-pinned tags follow the pattern `v0.1.1-gemma3-4b`, `v0.1.1-gemma3-12b`, +`v0.1.1-proxy`. The `proxy` tag is never published as `latest`. +``` + +- [ ] **Step 2: Add the usage section** + +Insert a new section immediately before `## Implemented routes`: + +```markdown +## Bring your own model server (`:proxy`) + +If you already run llama.cpp, vLLM, or anything else that speaks the OpenAI +chat-completions API, the `proxy` tag gives you the translation layer alone. It +contains no model and no inference engine. + +```bash +docker run -d -p 8090:8090 \ + -e LK_UPSTREAM=http://llama.internal:8080/v1 \ + gokhalh/localaik:proxy +``` + +| Env var | Default | Description | +| --- | --- | --- | +| `LK_UPSTREAM` | `http://127.0.0.1:8080/v1` | Base URL of your model server | +| `LK_UPSTREAM_AUTH_HEADER` | unset | A full header line sent to your server, for example `Authorization: Bearer abc123` | +| `PORT` | `8090` | Port localaik listens on | + +`LK_UPSTREAM_AUTH_HEADER` is sent only to your upstream. Credentials that +clients send to localaik are still discarded and never forwarded. + +`/health` returns 503 until your upstream answers, so existing healthchecks and +CI wait loops work unchanged. + +### Security + +`:proxy` has a different risk profile from the model-bundled tags. Those keep +llama.cpp bound to localhost inside the container, so the only thing reachable +is a disposable local model. `:proxy` forwards into infrastructure you care +about, and localaik does not authenticate its callers by design. + +**Anyone who can reach port 8090 can use your model server without +credentials.** Bind to localhost and do not publish the port on a shared +network. localaik is a testing tool, not a gateway. +``` + +- [ ] **Step 3: Update the tested-SDKs and limitations text if it claims self-containment** + +Run: `grep -n 'one container\|self-contained\|no internet\|No API key' README.md` + +For each hit, confirm the claim is still true or scope it to the model-bundled tags. The `## Motivation` paragraph says "a single Docker container that speaks all three protocols backed by a local model", which remains accurate for the default tags; add ", or the `proxy` tag if you already run your own model server." to the end of that sentence. + +- [ ] **Step 4: Check the rendered result** + +Run: `grep -n '^## ' README.md` +Expected: the new `## Bring your own model server (:proxy)` heading appears between `## Tuning` and `## Implemented routes`. + +- [ ] **Step 5: Commit** + +```bash +git add README.md +git commit -m "docs: Document the proxy tag and its security profile" +``` + +--- + +### Task 8: Full verification and review + +**Files:** none modified. + +- [ ] **Step 1: Run everything** + +```bash +make lint +go test -count=1 ./cmd/... ./internal/... ./integration/ +go test -count=1 -tags=docker_integration ./integration -run ProxyImage +``` + +Expected: all pass. + +- [ ] **Step 2: Confirm the existing images are untouched** + +```bash +git diff main --stat -- Dockerfile +docker build -t localaik:regression-check . +docker run -d --name regression-check -p 18096:8090 localaik:regression-check +sleep 90 +curl -s http://127.0.0.1:18096/health +docker rm -f regression-check +``` + +Expected: `{"status":"ok"}`. The only `Dockerfile` change should be the inserted stage. + +- [ ] **Step 3: Confirm no secret is logged** + +```bash +docker run --rm -e LK_UPSTREAM_AUTH_HEADER="Authorization: Bearer super-secret" \ + -e LK_UPSTREAM=http://127.0.0.1:9/v1 localaik:proxy-check 2>&1 | head -20 | grep -c super-secret +``` + +Expected: `0`. The container will fail to reach its upstream, which is fine; the check is that the credential never appears in output. + +- [ ] **Step 4: Run the three required reviews** + +Per the repo's PR workflow, before opening the PR: + +1. The `everything-claude-code:code-reviewer` agent on the pending diff. +2. The `superpowers:requesting-code-review` skill against `main..HEAD`. +3. The `codex:review` command, falling back to `codex:codex-rescue` with a saved diff. + +Fix anything actionable and re-review. Do not open the PR while findings are outstanding. + +- [ ] **Step 5: Open the PR** + +Use the `newpr` skill to generate the description. + +--- + +## Self-Review + +**Spec coverage:** + +| Spec requirement | Task | +| --- | --- | +| Third Dockerfile stage, alpine plus poppler-utils | 4 | +| llama.cpp stage stays last | 4 (Step 2), 8 (Step 2) | +| poppler required, not optional | 4 (Step 5) | +| `LK_UPSTREAM` env fallback | 3 | +| `LK_UPSTREAM_AUTH_HEADER`, full header line | 1, 3 | +| Flag over env over default | 3 | +| Credential injected in the transport, one place | 1 | +| Client credentials still stripped | 2 | +| Both properties tested together | 2 (Step 1) | +| `/health` unchanged against remote upstream | 4 (Step 6), 5 | +| `HEALTHCHECK` start-period reduced to 5s | 4 (Step 1) | +| Matrix entry, `proxy` and `vX.Y.Z-proxy` | 6 | +| Never `latest` | 6 (Steps 2, 5) | +| README security warning | 7 (Step 2) | +| Measure final image size | 4 (Step 4), 7 (Step 1) | +| Verify alpine pdftoppm matches | 4 (Step 5), 5 | +| Never log the credential | 8 (Step 3) | +| No model download | not implemented, correctly out of scope | +| No client authentication | not implemented, correctly out of scope | + +**Placeholder scan:** `` in Task 7 Step 1 is an intentional handoff from Task 4 Step 4, which produces the number. No other placeholders. + +**Type consistency:** `newUpstreamAuthTransport(base http.RoundTripper, header string) http.RoundTripper` is defined in Task 1 Step 3 and referenced in Task 1 Steps 1 and 5 only. `resolveFlagDefault(envName, fallback string) string` is defined in Task 3 Step 3 and used in the same step. `Config.UpstreamAuthHeader` is added in Task 1 Step 5 and consumed in Tasks 2 and 3. `roundTripHandler` and `newTestServer` are pre-existing and referenced with their file locations. Consistent. From 8bc36e1490b47bcbcb4a456393470f2e5f1b09c7 Mon Sep 17 00:00:00 2001 From: Harshaneel Gokhale Date: Tue, 4 Aug 2026 12:06:45 -0700 Subject: [PATCH 03/18] feat: Add optional upstream auth header to the proxy Injected at the transport layer so every upstream request carries it without each call site opting in. --- internal/server/server.go | 12 +++-- internal/server/upstreamauth.go | 35 +++++++++++++ internal/server/upstreamauth_test.go | 75 ++++++++++++++++++++++++++++ 3 files changed, 119 insertions(+), 3 deletions(-) create mode 100644 internal/server/upstreamauth.go create mode 100644 internal/server/upstreamauth_test.go diff --git a/internal/server/server.go b/internal/server/server.go index 3254b6b..514290a 100644 --- a/internal/server/server.go +++ b/internal/server/server.go @@ -16,9 +16,10 @@ import ( ) type Config struct { - UpstreamBaseURL string - HTTPClient *http.Client - PDFRenderer pdf.Renderer + UpstreamBaseURL string + UpstreamAuthHeader string + HTTPClient *http.Client + PDFRenderer pdf.Renderer } type Server struct { @@ -48,6 +49,11 @@ func New(cfg Config) (*Server, error) { if client == nil { client = &http.Client{} } + if cfg.UpstreamAuthHeader != "" { + clone := *client + clone.Transport = newUpstreamAuthTransport(clone.Transport, cfg.UpstreamAuthHeader) + client = &clone + } renderer := cfg.PDFRenderer if renderer == nil { diff --git a/internal/server/upstreamauth.go b/internal/server/upstreamauth.go new file mode 100644 index 0000000..9dd2da3 --- /dev/null +++ b/internal/server/upstreamauth.go @@ -0,0 +1,35 @@ +package server + +import ( + "net/http" + "strings" +) + +type upstreamAuthTransport struct { + base http.RoundTripper + name string + value string +} + +// newUpstreamAuthTransport returns base unchanged when header is not a usable +// "Name: value" line, so a misconfigured value cannot silently drop requests. +func newUpstreamAuthTransport(base http.RoundTripper, header string) http.RoundTripper { + if base == nil { + base = http.DefaultTransport + } + + name, value, found := strings.Cut(header, ":") + name = strings.TrimSpace(name) + value = strings.TrimSpace(value) + if !found || name == "" || value == "" { + return base + } + + return &upstreamAuthTransport{base: base, name: name, value: value} +} + +func (t *upstreamAuthTransport) RoundTrip(req *http.Request) (*http.Response, error) { + clone := req.Clone(req.Context()) + clone.Header.Set(t.name, t.value) + return t.base.RoundTrip(clone) +} diff --git a/internal/server/upstreamauth_test.go b/internal/server/upstreamauth_test.go new file mode 100644 index 0000000..174ce57 --- /dev/null +++ b/internal/server/upstreamauth_test.go @@ -0,0 +1,75 @@ +package server + +import ( + "net/http" + "net/http/httptest" + "testing" +) + +type capturingTransport struct { + seen http.Header +} + +func (c *capturingTransport) RoundTrip(req *http.Request) (*http.Response, error) { + c.seen = req.Header.Clone() + recorder := httptest.NewRecorder() + recorder.WriteHeader(http.StatusOK) + return recorder.Result(), nil +} + +func TestUpstreamAuthTransportAddsHeader(t *testing.T) { + capture := &capturingTransport{} + transport := newUpstreamAuthTransport(capture, "Authorization: Bearer secret") + + req := httptest.NewRequest(http.MethodGet, "http://upstream.test/v1/models", nil) + if _, err := transport.RoundTrip(req); err != nil { + t.Fatalf("RoundTrip returned error: %v", err) + } + + if got := capture.seen.Get("Authorization"); got != "Bearer secret" { + t.Fatalf("Authorization = %q, want %q", got, "Bearer secret") + } +} + +func TestUpstreamAuthTransportTrimsWhitespace(t *testing.T) { + capture := &capturingTransport{} + transport := newUpstreamAuthTransport(capture, " X-Api-Key : abc123 ") + + req := httptest.NewRequest(http.MethodGet, "http://upstream.test/v1/models", nil) + if _, err := transport.RoundTrip(req); err != nil { + t.Fatalf("RoundTrip returned error: %v", err) + } + + if got := capture.seen.Get("X-Api-Key"); got != "abc123" { + t.Fatalf("X-Api-Key = %q, want %q", got, "abc123") + } +} + +func TestUpstreamAuthTransportIgnoresMalformedHeader(t *testing.T) { + for _, header := range []string{"", " ", "NoColonHere", ": novalue", "Name:"} { + capture := &capturingTransport{} + transport := newUpstreamAuthTransport(capture, header) + + req := httptest.NewRequest(http.MethodGet, "http://upstream.test/v1/models", nil) + if _, err := transport.RoundTrip(req); err != nil { + t.Fatalf("RoundTrip returned error for %q: %v", header, err) + } + if len(capture.seen) != 0 { + t.Fatalf("header %q produced %v, want none", header, capture.seen) + } + } +} + +func TestUpstreamAuthTransportDoesNotMutateCallerRequest(t *testing.T) { + capture := &capturingTransport{} + transport := newUpstreamAuthTransport(capture, "Authorization: Bearer secret") + + req := httptest.NewRequest(http.MethodGet, "http://upstream.test/v1/models", nil) + if _, err := transport.RoundTrip(req); err != nil { + t.Fatalf("RoundTrip returned error: %v", err) + } + + if req.Header.Get("Authorization") != "" { + t.Fatal("RoundTrip mutated the caller's request") + } +} From e6d33e43bd8ff26a8446df189c9a127cd8448fbd Mon Sep 17 00:00:00 2001 From: Harshaneel Gokhale Date: Tue, 4 Aug 2026 12:11:45 -0700 Subject: [PATCH 04/18] test: Cover upstream auth on every upstream route Locks both halves at once: the proxy credential is added, the caller's is not forwarded. --- internal/server/auth_integration_test.go | 168 +++++++++++++++++++++++ 1 file changed, 168 insertions(+) create mode 100644 internal/server/auth_integration_test.go diff --git a/internal/server/auth_integration_test.go b/internal/server/auth_integration_test.go new file mode 100644 index 0000000..87bdd47 --- /dev/null +++ b/internal/server/auth_integration_test.go @@ -0,0 +1,168 @@ +package server + +import ( + "bytes" + "context" + "net/http" + "net/http/httptest" + "testing" + + "github.com/harshaneel/localaik/internal/pdf" + openaip "github.com/harshaneel/localaik/internal/protocol/openai" +) + +// Every upstream route must carry the proxy's credential and none of the +// caller's. +func TestUpstreamAuthHeaderReachesEveryUpstreamPath(t *testing.T) { + cases := []struct { + name string + method string + path string + body string + }{ + {"openai_chat", http.MethodPost, "/v1/chat/completions", `{"model":"m","messages":[]}`}, + {"openai_models", http.MethodGet, "/v1/models", ""}, + {"gemini_generate", http.MethodPost, "/v1beta/models/m:generateContent", `{"contents":[{"role":"user","parts":[{"text":"hi"}]}]}`}, + {"gemini_count_tokens", http.MethodPost, "/v1beta/models/m:countTokens", `{"contents":[{"parts":[{"text":"hi"}]}]}`}, + {"anthropic_messages", http.MethodPost, "/v1/messages", `{"max_tokens":8,"messages":[{"role":"user","content":"hi"}]}`}, + {"anthropic_count_tokens", http.MethodPost, "/v1/messages/count_tokens", `{"messages":[{"role":"user","content":"hi"}]}`}, + } + + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + var called bool + var seenAuth, seenClientAuth, seenAPIKey, seenGoogKey string + + upstream := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + called = true + seenAuth = r.Header.Get("X-Proxy-Token") + seenClientAuth = r.Header.Get("Authorization") + seenAPIKey = r.Header.Get("X-Api-Key") + seenGoogKey = r.Header.Get("X-Goog-Api-Key") + + switch r.URL.Path { + case "/tokenize": + writeJSON(w, http.StatusOK, map[string]any{"tokens": []int{1, 2}}) + case "/v1/models": + writeJSON(w, http.StatusOK, openaip.ModelList{Object: "list", Data: []openaip.Model{{ID: "m"}}}) + default: + writeJSON(w, http.StatusOK, openaip.ChatCompletionResponse{ + Choices: []openaip.Choice{{Message: openaip.Message{Content: "ok"}, FinishReason: "stop"}}, + }) + } + }) + + srv, err := New(Config{ + UpstreamBaseURL: "http://upstream.test/v1", + UpstreamAuthHeader: "X-Proxy-Token: upstream-secret", + HTTPClient: &http.Client{Transport: roundTripHandler{handler: upstream}}, + PDFRenderer: pdf.RendererFunc(func(context.Context, []byte) ([][]byte, error) { return nil, nil }), + }) + if err != nil { + t.Fatalf("New returned error: %v", err) + } + + var reader *bytes.Buffer + if tc.body != "" { + reader = bytes.NewBufferString(tc.body) + } else { + reader = bytes.NewBuffer(nil) + } + req := httptest.NewRequest(tc.method, tc.path, reader) + req.Header.Set("Authorization", "Bearer client-secret") + req.Header.Set("X-Api-Key", "client-anthropic-key") + req.Header.Set("X-Goog-Api-Key", "client-google-key") + + rec := httptest.NewRecorder() + srv.ServeHTTP(rec, req) + + if rec.Code != http.StatusOK { + t.Fatalf("status = %d, want 200; body=%s", rec.Code, rec.Body.String()) + } + if !called { + t.Fatal("upstream was never called, so the header check proves nothing") + } + if seenAuth != "upstream-secret" { + t.Fatalf("X-Proxy-Token = %q, want the proxy credential", seenAuth) + } + if seenClientAuth != "" || seenAPIKey != "" || seenGoogKey != "" { + t.Fatalf("client credentials leaked upstream: auth=%q apikey=%q googkey=%q", seenClientAuth, seenAPIKey, seenGoogKey) + } + }) + } +} + +func TestNoUpstreamAuthHeaderWhenUnset(t *testing.T) { + var seen http.Header + + upstream := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + seen = r.Header.Clone() + writeJSON(w, http.StatusOK, openaip.ChatCompletionResponse{ + Choices: []openaip.Choice{{Message: openaip.Message{Content: "ok"}, FinishReason: "stop"}}, + }) + }) + + srv := newTestServer(t, upstream) + + req := httptest.NewRequest(http.MethodPost, "/v1/messages", bytes.NewBufferString(`{"max_tokens":8,"messages":[{"role":"user","content":"hi"}]}`)) + rec := httptest.NewRecorder() + srv.ServeHTTP(rec, req) + + if rec.Code != http.StatusOK { + t.Fatalf("status = %d, want 200; body=%s", rec.Code, rec.Body.String()) + } + if got := seen.Get("Authorization"); got != "" { + t.Fatalf("Authorization = %q, want none when no credential is configured", got) + } +} + +// The Gemini streaming route builds its own request; confirm the credential is +// present there too. +func TestUpstreamAuthHeaderOnStreamingRoute(t *testing.T) { + var seen string + + upstream := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + seen = r.Header.Get("X-Proxy-Token") + w.Header().Set("Content-Type", "text/event-stream") + w.WriteHeader(http.StatusOK) + _, _ = w.Write([]byte("data: {\"choices\":[{\"delta\":{\"content\":\"hi\"},\"finish_reason\":\"stop\"}]}\n\ndata: [DONE]\n\n")) + }) + + srv, err := New(Config{ + UpstreamBaseURL: "http://upstream.test/v1", + UpstreamAuthHeader: "X-Proxy-Token: upstream-secret", + HTTPClient: &http.Client{Transport: roundTripHandler{handler: upstream}}, + PDFRenderer: pdf.RendererFunc(func(context.Context, []byte) ([][]byte, error) { return nil, nil }), + }) + if err != nil { + t.Fatalf("New returned error: %v", err) + } + + body := `{"contents":[{"role":"user","parts":[{"text":"hi"}]}]}` + req := httptest.NewRequest(http.MethodPost, "/v1beta/models/m:streamGenerateContent", bytes.NewBufferString(body)) + rec := httptest.NewRecorder() + srv.ServeHTTP(rec, req) + + if seen != "upstream-secret" { + t.Fatalf("X-Proxy-Token = %q on the streaming route, want the proxy credential", seen) + } +} + +func TestNewDoesNotMutateCallerClient(t *testing.T) { + sentinelTransport := &http.Transport{DisableCompression: true} + clientToPass := &http.Client{Transport: sentinelTransport} + + _, err := New(Config{ + UpstreamBaseURL: "http://upstream.test/v1", + UpstreamAuthHeader: "X-Proxy-Token: upstream-secret", + HTTPClient: clientToPass, + PDFRenderer: pdf.RendererFunc(func(context.Context, []byte) ([][]byte, error) { return nil, nil }), + }) + if err != nil { + t.Fatalf("New returned error: %v", err) + } + + if clientToPass.Transport != sentinelTransport { + t.Fatal("New mutated the caller's http.Client; it should have made a copy") + } +} From 7e4bcb88516c62e5acb74654813453681cbe90ad Mon Sep 17 00:00:00 2001 From: Harshaneel Gokhale Date: Tue, 4 Aug 2026 12:19:11 -0700 Subject: [PATCH 05/18] test: Add coverage for all upstream-calling routes Adds subtests for /health, /v1/completions, /v1/models/{id}, /v1beta/models, and /v1beta/models/{id} routes. Fixes streaming test to assert status code and response body content. --- internal/server/auth_integration_test.go | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/internal/server/auth_integration_test.go b/internal/server/auth_integration_test.go index 87bdd47..4216fdb 100644 --- a/internal/server/auth_integration_test.go +++ b/internal/server/auth_integration_test.go @@ -22,10 +22,15 @@ func TestUpstreamAuthHeaderReachesEveryUpstreamPath(t *testing.T) { }{ {"openai_chat", http.MethodPost, "/v1/chat/completions", `{"model":"m","messages":[]}`}, {"openai_models", http.MethodGet, "/v1/models", ""}, + {"openai_completions", http.MethodPost, "/v1/completions", `{"prompt":"hello"}`}, + {"openai_model_get", http.MethodGet, "/v1/models/gpt-4", ""}, {"gemini_generate", http.MethodPost, "/v1beta/models/m:generateContent", `{"contents":[{"role":"user","parts":[{"text":"hi"}]}]}`}, {"gemini_count_tokens", http.MethodPost, "/v1beta/models/m:countTokens", `{"contents":[{"parts":[{"text":"hi"}]}]}`}, + {"gemini_models_list", http.MethodGet, "/v1beta/models", ""}, + {"gemini_model_get", http.MethodGet, "/v1beta/models/gemini-2.5-pro", ""}, {"anthropic_messages", http.MethodPost, "/v1/messages", `{"max_tokens":8,"messages":[{"role":"user","content":"hi"}]}`}, {"anthropic_count_tokens", http.MethodPost, "/v1/messages/count_tokens", `{"messages":[{"role":"user","content":"hi"}]}`}, + {"health", http.MethodGet, "/health", ""}, } for _, tc := range cases { @@ -45,6 +50,10 @@ func TestUpstreamAuthHeaderReachesEveryUpstreamPath(t *testing.T) { writeJSON(w, http.StatusOK, map[string]any{"tokens": []int{1, 2}}) case "/v1/models": writeJSON(w, http.StatusOK, openaip.ModelList{Object: "list", Data: []openaip.Model{{ID: "m"}}}) + case "/v1/models/gpt-4": + writeJSON(w, http.StatusOK, openaip.Model{ID: "gpt-4"}) + case "/health": + w.WriteHeader(http.StatusOK) default: writeJSON(w, http.StatusOK, openaip.ChatCompletionResponse{ Choices: []openaip.Choice{{Message: openaip.Message{Content: "ok"}, FinishReason: "stop"}}, @@ -143,9 +152,15 @@ func TestUpstreamAuthHeaderOnStreamingRoute(t *testing.T) { rec := httptest.NewRecorder() srv.ServeHTTP(rec, req) + if rec.Code != http.StatusOK { + t.Fatalf("status = %d, want 200; body=%s", rec.Code, rec.Body.String()) + } if seen != "upstream-secret" { t.Fatalf("X-Proxy-Token = %q on the streaming route, want the proxy credential", seen) } + if !bytes.Contains(rec.Body.Bytes(), []byte("data:")) { + t.Fatalf("response body missing data: frame; got %s", rec.Body.String()) + } } func TestNewDoesNotMutateCallerClient(t *testing.T) { From 1a9885c3e792b7023a31969a523be10c50f9f733 Mon Sep 17 00:00:00 2001 From: Harshaneel Gokhale Date: Tue, 4 Aug 2026 12:27:54 -0700 Subject: [PATCH 06/18] feat: Read upstream and auth header from the environment Follows the pattern PORT already set, so the container needs no shell wrapper. Co-Authored-By: Claude Opus 5 (1M context) --- cmd/localaik/main.go | 40 +++++++++++++++++++------ cmd/localaik/main_test.go | 61 +++++++++++++++++++++++++++++++++++++++ 2 files changed, 92 insertions(+), 9 deletions(-) create mode 100644 cmd/localaik/main_test.go diff --git a/cmd/localaik/main.go b/cmd/localaik/main.go index ccfe049..c561885 100644 --- a/cmd/localaik/main.go +++ b/cmd/localaik/main.go @@ -5,26 +5,48 @@ import ( "log" "net/http" "os" + "strings" "time" "github.com/harshaneel/localaik/internal/pdf" "github.com/harshaneel/localaik/internal/server" ) -func main() { - defaultPort := os.Getenv("PORT") - if defaultPort == "" { - defaultPort = "8090" +func resolveFlagDefault(envName, fallback string) string { + if value := os.Getenv(envName); value != "" { + return value } + return fallback +} - port := flag.String("port", defaultPort, "port to listen on") - upstream := flag.String("upstream", "http://127.0.0.1:8080/v1", "upstream OpenAI-compatible base URL") +func isValidAuthHeader(header string) bool { + if header == "" { + return false + } + idx := strings.Index(header, ":") + if idx == -1 { + return false + } + name := strings.TrimSpace(header[:idx]) + value := strings.TrimSpace(header[idx+1:]) + return name != "" && value != "" +} + +func main() { + port := flag.String("port", resolveFlagDefault("PORT", "8090"), "port to listen on") + upstream := flag.String("upstream", resolveFlagDefault("LK_UPSTREAM", "http://127.0.0.1:8080/v1"), "upstream OpenAI-compatible base URL") flag.Parse() + authHeader := os.Getenv("LK_UPSTREAM_AUTH_HEADER") + if authHeader != "" && !isValidAuthHeader(authHeader) { + log.Printf("localaik: LK_UPSTREAM_AUTH_HEADER is set but is not a \"Name: value\" header line; no credential will be sent upstream") + } + handler, err := server.New(server.Config{ - UpstreamBaseURL: *upstream, - HTTPClient: &http.Client{}, - PDFRenderer: pdf.NewExecRenderer("pdftoppm"), + UpstreamBaseURL: *upstream, + UpstreamAuthHeader: authHeader, + HTTPClient: &http.Client{}, + PDFRenderer: pdf.NewExecRenderer("pdftoppm"), }) if err != nil { log.Fatalf("localaik: %v", err) diff --git a/cmd/localaik/main_test.go b/cmd/localaik/main_test.go new file mode 100644 index 0000000..63f2f60 --- /dev/null +++ b/cmd/localaik/main_test.go @@ -0,0 +1,61 @@ +package main + +import "testing" + +func TestResolveFlagDefaultPrefersEnv(t *testing.T) { + t.Setenv("LK_TEST_VALUE", "from-env") + + if got := resolveFlagDefault("LK_TEST_VALUE", "fallback"); got != "from-env" { + t.Fatalf("resolveFlagDefault = %q, want from-env", got) + } +} + +func TestResolveFlagDefaultFallsBack(t *testing.T) { + t.Setenv("LK_TEST_VALUE", "") + + if got := resolveFlagDefault("LK_TEST_VALUE", "fallback"); got != "fallback" { + t.Fatalf("resolveFlagDefault = %q, want fallback", got) + } +} + +func TestResolveFlagDefaultUnsetFallsBack(t *testing.T) { + if got := resolveFlagDefault("LK_DEFINITELY_UNSET_VALUE", "fallback"); got != "fallback" { + t.Fatalf("resolveFlagDefault = %q, want fallback", got) + } +} + +func TestIsValidAuthHeaderUnset(t *testing.T) { + if got := isValidAuthHeader(""); got != false { + t.Fatalf("isValidAuthHeader(\"\") = %v, want false", got) + } +} + +func TestIsValidAuthHeaderValid(t *testing.T) { + if got := isValidAuthHeader("Authorization: Bearer token123"); got != true { + t.Fatalf("isValidAuthHeader(\"Authorization: Bearer token123\") = %v, want true", got) + } +} + +func TestIsValidAuthHeaderNoColon(t *testing.T) { + if got := isValidAuthHeader("InvalidHeader NoColon"); got != false { + t.Fatalf("isValidAuthHeader(\"InvalidHeader NoColon\") = %v, want false", got) + } +} + +func TestIsValidAuthHeaderEmptyName(t *testing.T) { + if got := isValidAuthHeader(": value"); got != false { + t.Fatalf("isValidAuthHeader(\": value\") = %v, want false", got) + } +} + +func TestIsValidAuthHeaderEmptyValue(t *testing.T) { + if got := isValidAuthHeader("Name:"); got != false { + t.Fatalf("isValidAuthHeader(\"Name:\") = %v, want false", got) + } +} + +func TestIsValidAuthHeaderWhitespaceOnly(t *testing.T) { + if got := isValidAuthHeader(" : "); got != false { + t.Fatalf("isValidAuthHeader(\" : \") = %v, want false", got) + } +} From 6257188882ffb19ec444a85abd6c42b1d72c161a Mon Sep 17 00:00:00 2001 From: Harshaneel Gokhale Date: Tue, 4 Aug 2026 12:38:52 -0700 Subject: [PATCH 07/18] feat: Add a proxy-only Dockerfile stage alpine plus poppler-utils and the binary, no inference stack. The llama.cpp stage stays last so the default build is unchanged. --- Dockerfile | 9 +++++++++ Makefile | 6 +++++- 2 files changed, 14 insertions(+), 1 deletion(-) diff --git a/Dockerfile b/Dockerfile index 49446fb..7484a3d 100644 --- a/Dockerfile +++ b/Dockerfile @@ -4,6 +4,15 @@ WORKDIR /app COPY . . RUN CGO_ENABLED=0 GOOS=$TARGETOS GOARCH=$TARGETARCH go build -ldflags="-s -w" -o /out/localaik ./cmd/localaik +FROM alpine:3@sha256:28bd5fe8b56d1bd048e5babf5b10710ebe0bae67db86916198a6eec434943f8b AS proxy +RUN apk add --no-cache ca-certificates poppler-utils tini +COPY --from=proxy-builder /out/localaik /usr/local/bin/localaik +ENV PORT=8090 +HEALTHCHECK --interval=5s --timeout=3s --start-period=5s \ + CMD wget -q -O - "http://127.0.0.1:${PORT:-8090}/health" >/dev/null 2>&1 || exit 1 +EXPOSE 8090 +ENTRYPOINT ["tini", "--", "localaik"] + # Upstream does not ship semver for the server image; pin by digest for reproducible multi-arch builds. # Logical tag at pin time: server (includes llama-server --mmproj for Gemma 3 vision). Bump digest to upgrade. FROM ghcr.io/ggml-org/llama.cpp@sha256:80910e898e5d9a6b46ca9d1b4674d3e15faf6d32b9692eb6011ccd34b2cb8a06 diff --git a/Makefile b/Makefile index 56d02dd..c601405 100644 --- a/Makefile +++ b/Makefile @@ -10,7 +10,7 @@ GOFILES := $(shell find cmd internal integration -name '*.go' -type f | sort) export GOCACHE -.PHONY: help fmt fmt-check lint test-unit test-integration test build docker-build docker-up docker-down +.PHONY: help fmt fmt-check lint test-unit test-integration test build docker-build docker-build-proxy docker-up docker-down help: @printf '%s\n' \ @@ -22,6 +22,7 @@ help: 'make test Run lint, unit tests, and integration tests' \ 'make build Build the localaik binary' \ 'make docker-build Build the Docker image' \ + 'make docker-build-proxy Build the proxy-only image' \ 'make docker-up Start the Docker image on PORT' \ 'make docker-down Stop and remove the Docker container' @@ -48,6 +49,9 @@ build: docker-build: @docker build -t "$(IMAGE)" . +docker-build-proxy: + @docker build --target proxy -t "$(IMAGE)-proxy" . + docker-up: @if [[ "$(BUILD_IMAGE)" == "1" ]]; then $(MAKE) docker-build IMAGE="$(IMAGE)"; fi @docker rm -f "$(CONTAINER_NAME)" >/dev/null 2>&1 || true From fe1c0ff4b9b4a21da9594c3ffeb797c76cc3df6d Mon Sep 17 00:00:00 2001 From: Harshaneel Gokhale Date: Tue, 4 Aug 2026 13:01:31 -0700 Subject: [PATCH 08/18] test: Exercise the built proxy image on all three protocols Builds the proxy Dockerfile target, runs it against a stub upstream reached via host.docker.internal, and drives OpenAI, Gemini, and Anthropic-shaped requests through it end to end. Also fixes a leaked listener fd, a stale-container name collision, and an unsynchronized cross-goroutine read found during review of the brief's literal code. --- integration/proxy_image_test.go | 125 ++++++++++++++++++++++++++++++++ 1 file changed, 125 insertions(+) create mode 100644 integration/proxy_image_test.go diff --git a/integration/proxy_image_test.go b/integration/proxy_image_test.go new file mode 100644 index 0000000..7f2bf99 --- /dev/null +++ b/integration/proxy_image_test.go @@ -0,0 +1,125 @@ +//go:build docker_integration + +package integration + +import ( + "encoding/json" + "fmt" + "net" + "net/http" + "net/http/httptest" + "os/exec" + "strings" + "sync" + "testing" + "time" +) + +// Exercises the built proxy image against a stub upstream, proving all three +// protocol surfaces round-trip without an inference stack in the container. +func TestProxyImageRoundTripsAllProtocols(t *testing.T) { + image := "localaik:proxy-integration" + + build := exec.Command("docker", "build", "--target", "proxy", "-t", image, "..") + if out, err := build.CombinedOutput(); err != nil { + t.Fatalf("docker build failed: %v\n%s", err, out) + } + + var authMu sync.Mutex + var seenAuth string + stub := httptest.NewUnstartedServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + authMu.Lock() + seenAuth = r.Header.Get("X-Proxy-Token") + authMu.Unlock() + w.Header().Set("Content-Type", "application/json") + if strings.HasSuffix(r.URL.Path, "/tokenize") { + _, _ = w.Write([]byte(`{"tokens":[1,2,3]}`)) + return + } + _, _ = w.Write([]byte(`{"choices":[{"index":0,"message":{"role":"assistant","content":"stubbed"},"finish_reason":"stop"}]}`)) + })) + + // NewUnstartedServer already bound a loopback-only listener; close it and + // swap in one bound to 0.0.0.0 so the container can reach it. + stub.Listener.Close() + listener, err := net.Listen("tcp", "0.0.0.0:0") + if err != nil { + t.Fatalf("listen: %v", err) + } + stub.Listener = listener + stub.Start() + defer stub.Close() + + stubPort := listener.Addr().(*net.TCPAddr).Port + upstream := fmt.Sprintf("http://host.docker.internal:%d/v1", stubPort) + + // Best-effort: clear any container left behind by a prior run that + // crashed before its own cleanup ran, so the name doesn't collide. + _ = exec.Command("docker", "rm", "-f", "proxy-integration").Run() + + run := exec.Command("docker", "run", "-d", "--name", "proxy-integration", + "--add-host", "host.docker.internal:host-gateway", + "-p", "18097:8090", + "-e", "LK_UPSTREAM="+upstream, + "-e", "LK_UPSTREAM_AUTH_HEADER=X-Proxy-Token: integration-secret", + image) + if out, err := run.CombinedOutput(); err != nil { + t.Fatalf("docker run failed: %v\n%s", err, out) + } + defer exec.Command("docker", "rm", "-f", "proxy-integration").Run() + + waitForHealth(t, "http://127.0.0.1:18097/health") + + cases := []struct { + name string + path string + body string + }{ + {"openai", "/v1/chat/completions", `{"model":"m","messages":[{"role":"user","content":"hi"}]}`}, + {"gemini", "/v1beta/models/m:generateContent", `{"contents":[{"role":"user","parts":[{"text":"hi"}]}]}`}, + {"anthropic", "/v1/messages", `{"max_tokens":16,"messages":[{"role":"user","content":"hi"}]}`}, + } + + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + resp, err := http.Post("http://127.0.0.1:18097"+tc.path, "application/json", strings.NewReader(tc.body)) + if err != nil { + t.Fatalf("post: %v", err) + } + defer resp.Body.Close() + + if resp.StatusCode != http.StatusOK { + t.Fatalf("status = %d, want 200", resp.StatusCode) + } + var decoded map[string]any + if err := json.NewDecoder(resp.Body).Decode(&decoded); err != nil { + t.Fatalf("decode: %v", err) + } + if len(decoded) == 0 { + t.Fatal("empty response body") + } + }) + } + + authMu.Lock() + got := seenAuth + authMu.Unlock() + if got != "integration-secret" { + t.Fatalf("upstream saw X-Proxy-Token = %q, want integration-secret", got) + } +} + +func waitForHealth(t *testing.T, url string) { + t.Helper() + for i := 0; i < 60; i++ { + resp, err := http.Get(url) + if err == nil { + resp.Body.Close() + if resp.StatusCode == http.StatusOK { + return + } + } + time.Sleep(time.Second) + } + t.Fatalf("never became healthy: %s", url) +} From 2adc7675c59ca111dd9e5bbf5471b36ee8fdce99 Mon Sep 17 00:00:00 2001 From: Harshaneel Gokhale Date: Tue, 4 Aug 2026 13:10:36 -0700 Subject: [PATCH 09/18] test: Assert per-protocol response shape in proxy image test The stub returns one OpenAI-shaped payload for every route, and the per-subtest assertion only checked for a non-empty JSON object. That would pass even if routing sent a Gemini or Anthropic request to the OpenAI passthrough handler. Assert the expected top-level key per protocol (choices/candidates/content) and the assistant role on the Anthropic response, so a misrouted handler fails the test. --- integration/proxy_image_test.go | 32 ++++++++++++++++++++++++-------- 1 file changed, 24 insertions(+), 8 deletions(-) diff --git a/integration/proxy_image_test.go b/integration/proxy_image_test.go index 7f2bf99..0b1dc2e 100644 --- a/integration/proxy_image_test.go +++ b/integration/proxy_image_test.go @@ -71,13 +71,14 @@ func TestProxyImageRoundTripsAllProtocols(t *testing.T) { waitForHealth(t, "http://127.0.0.1:18097/health") cases := []struct { - name string - path string - body string + name string + path string + body string + expectedKey string }{ - {"openai", "/v1/chat/completions", `{"model":"m","messages":[{"role":"user","content":"hi"}]}`}, - {"gemini", "/v1beta/models/m:generateContent", `{"contents":[{"role":"user","parts":[{"text":"hi"}]}]}`}, - {"anthropic", "/v1/messages", `{"max_tokens":16,"messages":[{"role":"user","content":"hi"}]}`}, + {"openai", "/v1/chat/completions", `{"model":"m","messages":[{"role":"user","content":"hi"}]}`, "choices"}, + {"gemini", "/v1beta/models/m:generateContent", `{"contents":[{"role":"user","parts":[{"text":"hi"}]}]}`, "candidates"}, + {"anthropic", "/v1/messages", `{"max_tokens":16,"messages":[{"role":"user","content":"hi"}]}`, "content"}, } for _, tc := range cases { @@ -95,8 +96,15 @@ func TestProxyImageRoundTripsAllProtocols(t *testing.T) { if err := json.NewDecoder(resp.Body).Decode(&decoded); err != nil { t.Fatalf("decode: %v", err) } - if len(decoded) == 0 { - t.Fatal("empty response body") + // Each protocol reshapes the stub's fixed OpenAI-shaped payload + // differently; the expected key catches a misrouted handler. + if _, ok := decoded[tc.expectedKey]; !ok { + t.Fatalf("response missing %q key, got keys %v", tc.expectedKey, mapKeys(decoded)) + } + if tc.name == "anthropic" { + if role, _ := decoded["role"].(string); role != "assistant" { + t.Fatalf("role = %q, want assistant", role) + } } }) } @@ -109,6 +117,14 @@ func TestProxyImageRoundTripsAllProtocols(t *testing.T) { } } +func mapKeys(m map[string]any) []string { + keys := make([]string, 0, len(m)) + for k := range m { + keys = append(keys, k) + } + return keys +} + func waitForHealth(t *testing.T, url string) { t.Helper() for i := 0; i < 60; i++ { From 6ed2c2ab127b5e9f12848fd285853b9ce31d32fc Mon Sep 17 00:00:00 2001 From: Harshaneel Gokhale Date: Tue, 4 Aug 2026 13:41:12 -0700 Subject: [PATCH 10/18] test: Compress the expected-key comment to one line --- integration/proxy_image_test.go | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/integration/proxy_image_test.go b/integration/proxy_image_test.go index 0b1dc2e..f968c6b 100644 --- a/integration/proxy_image_test.go +++ b/integration/proxy_image_test.go @@ -96,8 +96,7 @@ func TestProxyImageRoundTripsAllProtocols(t *testing.T) { if err := json.NewDecoder(resp.Body).Decode(&decoded); err != nil { t.Fatalf("decode: %v", err) } - // Each protocol reshapes the stub's fixed OpenAI-shaped payload - // differently; the expected key catches a misrouted handler. + // The expected key catches a response shaped by the wrong handler. if _, ok := decoded[tc.expectedKey]; !ok { t.Fatalf("response missing %q key, got keys %v", tc.expectedKey, mapKeys(decoded)) } From 1db880dbe7fab8c642f341a30f9b111d10a9dfd1 Mon Sep 17 00:00:00 2001 From: Harshaneel Gokhale Date: Tue, 4 Aug 2026 13:44:09 -0700 Subject: [PATCH 11/18] ci: Publish the proxy image variant Adds a build target to the matrix. Empty target keeps the existing entries on the default final stage. --- .github/workflows/release.yml | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index a1ce7d7..4053a72 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -53,16 +53,25 @@ jobs: include: - variant: gemma3-4b latest: true + target: "" model_url: https://huggingface.co/lmstudio-community/gemma-3-4b-it-GGUF/resolve/c536c4707e747055eecad7da65d46b6fb0ebaa79/gemma-3-4b-it-Q4_K_M.gguf model_sha256: be49949e48422e4547b00af14179a193d3777eea7fbbd7d6e1b0861304628a01 mmproj_url: https://huggingface.co/lmstudio-community/gemma-3-4b-it-GGUF/resolve/d400f8ba80bfa661d94a756ea3b663db8b00da85/mmproj-model-f16.gguf mmproj_sha256: 8c0fb064b019a6972856aaae2c7e4792858af3ca4561be2dbf649123ba6c40cb - variant: gemma3-12b latest: false + target: "" model_url: https://huggingface.co/lmstudio-community/gemma-3-12b-it-GGUF/resolve/ed6e7d1e8c65a0181cd45e3a194722d1c651f06e/gemma-3-12b-it-Q4_K_M.gguf model_sha256: 9610e3e07375303f6cd89086b496bcc1ab581177f52042eff536475a29283ba2 mmproj_url: https://huggingface.co/lmstudio-community/gemma-3-12b-it-GGUF/resolve/785c151b0dabf00cfaff1421239fafdbb8e5995d/mmproj-model-f16.gguf mmproj_sha256: 30c02d056410848227001830866e0a269fcc28aaf8ca971bded494003de9f5a5 + - variant: proxy + latest: false + target: proxy + model_url: "" + model_sha256: "" + mmproj_url: "" + mmproj_sha256: "" permissions: contents: read steps: @@ -96,6 +105,7 @@ jobs: with: context: . file: ./Dockerfile + target: ${{ matrix.target }} platforms: linux/amd64,linux/arm64 push: true tags: ${{ steps.meta.outputs.tags }} From 930892d5d8f052356b5b5863910be1c47fa4cbdf Mon Sep 17 00:00:00 2001 From: Harshaneel Gokhale Date: Tue, 4 Aug 2026 13:51:35 -0700 Subject: [PATCH 12/18] docs: Document the proxy tag and its security profile --- README.md | 41 +++++++++++++++++++++++++++++++++++++++-- 1 file changed, 39 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 5db182c..5786434 100644 --- a/README.md +++ b/README.md @@ -11,7 +11,7 @@ A local compatibility server for the Gemini, OpenAI, and Anthropic APIs. Run one ## Motivation -Testing code that calls Gemini, OpenAI, or Anthropic is painful: real API calls are slow, cost money, and need network access. localaik gives you a single Docker container that speaks all three protocols backed by a local model — no API key, no internet, deterministic enough for CI. +Testing code that calls Gemini, OpenAI, or Anthropic is painful: real API calls are slow, cost money, and need network access. localaik gives you a single Docker container that speaks all three protocols backed by a local model, or the `proxy` tag if you already run your own model server. No API key, no internet, deterministic enough for CI. ## Architecture @@ -133,9 +133,11 @@ client := anthropic.NewClient( | --------------------- | ------------------ | ---------- | | `latest`, `gemma3-4b` | Gemma 3 4B Q4_K_M | ~3 GB | | `gemma3-12b` | Gemma 3 12B Q4_K_M | ~7 GB | +| `proxy` | none (you supply) | ~41 MB | -Version-pinned tags follow the pattern `v0.1.1-gemma3-4b`, `v0.1.1-gemma3-12b`. +Version-pinned tags follow the pattern `v0.1.1-gemma3-4b`, `v0.1.1-gemma3-12b`, +`v0.1.1-proxy`. The `proxy` tag is never published as `latest`. ## Tuning (v0.1.3 onwards) @@ -182,6 +184,41 @@ services: | `LK_MLOCK` | 0 (off) | Lock model in RAM (`1` to enable) | +## Bring your own model server (`:proxy`) + +If you already run llama.cpp, vLLM, or anything else that speaks the OpenAI +chat-completions API, the `proxy` tag gives you the translation layer alone. It +contains no model and no inference engine. + +```bash +docker run -d -p 8090:8090 \ + -e LK_UPSTREAM=http://llama.internal:8080/v1 \ + gokhalh/localaik:proxy +``` + +| Env var | Default | Description | +| --- | --- | --- | +| `LK_UPSTREAM` | `http://127.0.0.1:8080/v1` | Base URL of your model server | +| `LK_UPSTREAM_AUTH_HEADER` | unset | A full header line sent to your server, for example `Authorization: Bearer abc123` | +| `PORT` | `8090` | Port localaik listens on | + +`LK_UPSTREAM_AUTH_HEADER` is sent only to your upstream. Credentials that +clients send to localaik are still discarded and never forwarded. + +`/health` returns 503 until your upstream answers, so existing healthchecks and +CI wait loops work unchanged. + +### Security + +`:proxy` has a different risk profile from the model-bundled tags. Those keep +llama.cpp bound to localhost inside the container, so the only thing reachable +is a disposable local model. `:proxy` forwards into infrastructure you care +about, and localaik does not authenticate its callers by design. + +**Anyone who can reach port 8090 can use your model server without +credentials.** Bind to localhost and do not publish the port on a shared +network. localaik is a testing tool, not a gateway. + ## Implemented routes From c126b9be01f4bf03d2748d7092f26894221cca14 Mon Sep 17 00:00:00 2001 From: Harshaneel Gokhale Date: Tue, 4 Aug 2026 14:00:27 -0700 Subject: [PATCH 13/18] docs: Scope proxy-inapplicable claims in Motivation and Limitations --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 5786434..089397a 100644 --- a/README.md +++ b/README.md @@ -11,7 +11,7 @@ A local compatibility server for the Gemini, OpenAI, and Anthropic APIs. Run one ## Motivation -Testing code that calls Gemini, OpenAI, or Anthropic is painful: real API calls are slow, cost money, and need network access. localaik gives you a single Docker container that speaks all three protocols backed by a local model, or the `proxy` tag if you already run your own model server. No API key, no internet, deterministic enough for CI. +Testing code that calls Gemini, OpenAI, or Anthropic is painful: real API calls are slow, cost money, and need network access. localaik gives you a single Docker container that speaks all three protocols backed by a local model, or the `proxy` tag if you already run your own model server. The model-bundled tags need no API key and no internet, and are deterministic enough for CI. ## Architecture @@ -399,7 +399,7 @@ docker build \ ## Limitations - Intended for tests and development, not production -- Image size is dominated by model weights +- Image size is dominated by model weights (not applicable to the `proxy` tag, which ships none) - Cold starts can take tens of seconds while the model loads - PDF rendering adds latency per page From d28e31d2c95fed7d59a722014a8f19bc4ec8ad09 Mon Sep 17 00:00:00 2001 From: Harshaneel Gokhale Date: Tue, 4 Aug 2026 15:25:24 -0700 Subject: [PATCH 14/18] fix: Withhold the upstream credential from redirect targets The transport set LK_UPSTREAM_AUTH_HEADER on every request it saw. An http.Client re-enters its RoundTripper for each redirect hop, and the stdlib strips Authorization only on the request it built itself, so setting the header per-request re-added it after that strip. A 302 from the upstream handed the credential to whatever host the Location named. The transport now pins on the configured upstream hostname, and the credentialed client no longer follows redirects at all, since a followed redirect would also send prompt content to the target. Redirect handling is left at the stdlib default when no credential is configured, so the model-bundled images are unaffected. Also extracts one header predicate, now rejecting names and values that net/http would reject at the wire on every request. Co-Authored-By: Claude Opus 5 (1M context) --- go.mod | 2 +- internal/server/server.go | 7 +- internal/server/upstreamauth.go | 42 +++++- internal/server/upstreamauth_redirect_test.go | 139 ++++++++++++++++++ internal/server/upstreamauth_test.go | 56 ++++++- 5 files changed, 234 insertions(+), 12 deletions(-) create mode 100644 internal/server/upstreamauth_redirect_test.go diff --git a/go.mod b/go.mod index 6545104..f8009e4 100644 --- a/go.mod +++ b/go.mod @@ -5,6 +5,7 @@ go 1.25 require ( github.com/anthropics/anthropic-sdk-go v1.61.0 github.com/openai/openai-go/v3 v3.36.0 + golang.org/x/net v0.41.0 google.golang.org/genai v1.57.0 ) @@ -29,7 +30,6 @@ require ( go.opencensus.io v0.24.0 // indirect go.yaml.in/yaml/v4 v4.0.0-rc.2 // indirect golang.org/x/crypto v0.40.0 // indirect - golang.org/x/net v0.41.0 // indirect golang.org/x/sync v0.16.0 // indirect golang.org/x/sys v0.35.0 // indirect golang.org/x/text v0.27.0 // indirect diff --git a/internal/server/server.go b/internal/server/server.go index 514290a..1aca6c1 100644 --- a/internal/server/server.go +++ b/internal/server/server.go @@ -51,7 +51,12 @@ func New(cfg Config) (*Server, error) { } if cfg.UpstreamAuthHeader != "" { clone := *client - clone.Transport = newUpstreamAuthTransport(clone.Transport, cfg.UpstreamAuthHeader) + clone.Transport = newUpstreamAuthTransport(clone.Transport, cfg.UpstreamAuthHeader, parsed.Hostname()) + // Scoped here so the bundled-model images keep the stdlib default; a + // followed redirect would send prompt content to the target. + clone.CheckRedirect = func(*http.Request, []*http.Request) error { + return http.ErrUseLastResponse + } client = &clone } diff --git a/internal/server/upstreamauth.go b/internal/server/upstreamauth.go index 9dd2da3..a036db2 100644 --- a/internal/server/upstreamauth.go +++ b/internal/server/upstreamauth.go @@ -3,32 +3,62 @@ package server import ( "net/http" "strings" + + "golang.org/x/net/http/httpguts" ) type upstreamAuthTransport struct { base http.RoundTripper + host string name string value string } +// ValidUpstreamAuthHeader reports whether header is a line the transport will +// actually send. Startup warnings must use this, not a second predicate. +func ValidUpstreamAuthHeader(header string) bool { + _, _, ok := parseUpstreamAuthHeader(header) + return ok +} + +// Only the first colon separates the pair, so values may contain further ones. +func parseUpstreamAuthHeader(header string) (string, string, bool) { + name, value, found := strings.Cut(header, ":") + name = strings.TrimSpace(name) + value = strings.TrimSpace(value) + if !found || name == "" || value == "" { + return "", "", false + } + // net/http rejects these at the wire on every request, with an error that + // never mentions the env var that caused it. + if !httpguts.ValidHeaderFieldName(name) || !httpguts.ValidHeaderFieldValue(value) { + return "", "", false + } + return name, value, true +} + // newUpstreamAuthTransport returns base unchanged when header is not a usable // "Name: value" line, so a misconfigured value cannot silently drop requests. -func newUpstreamAuthTransport(base http.RoundTripper, header string) http.RoundTripper { +func newUpstreamAuthTransport(base http.RoundTripper, header, host string) http.RoundTripper { if base == nil { base = http.DefaultTransport } - name, value, found := strings.Cut(header, ":") - name = strings.TrimSpace(name) - value = strings.TrimSpace(value) - if !found || name == "" || value == "" { + name, value, ok := parseUpstreamAuthHeader(header) + if !ok { return base } - return &upstreamAuthTransport{base: base, name: name, value: value} + return &upstreamAuthTransport{base: base, host: host, name: name, value: value} } func (t *upstreamAuthTransport) RoundTrip(req *http.Request) (*http.Response, error) { + // A 3xx re-enters here with the target's URL, so setting the header for + // every host would hand the credential to whatever the redirect names. + if !strings.EqualFold(req.URL.Hostname(), t.host) { + return t.base.RoundTrip(req) + } + clone := req.Clone(req.Context()) clone.Header.Set(t.name, t.value) return t.base.RoundTrip(clone) diff --git a/internal/server/upstreamauth_redirect_test.go b/internal/server/upstreamauth_redirect_test.go new file mode 100644 index 0000000..71d3639 --- /dev/null +++ b/internal/server/upstreamauth_redirect_test.go @@ -0,0 +1,139 @@ +package server + +import ( + "bytes" + "context" + "net/http" + "net/http/httptest" + "strings" + "sync" + "testing" + + "github.com/harshaneel/localaik/internal/pdf" +) + +// redirectingTransport answers upstream.test with a 302 to location and every +// other host with a marker body, recording the headers each host received. +type redirectingTransport struct { + location string + + mu sync.Mutex + seen map[string]http.Header +} + +func (r *redirectingTransport) RoundTrip(req *http.Request) (*http.Response, error) { + r.mu.Lock() + if r.seen == nil { + r.seen = make(map[string]http.Header) + } + r.seen[req.URL.Hostname()] = req.Header.Clone() + r.mu.Unlock() + + recorder := httptest.NewRecorder() + if req.URL.Hostname() == "upstream.test" { + recorder.Header().Set("Location", r.location) + recorder.WriteHeader(http.StatusFound) + return recorder.Result(), nil + } + + recorder.Header().Set("Content-Type", "application/json") + recorder.WriteHeader(http.StatusOK) + _, _ = recorder.WriteString(`{"leaked":true}`) + return recorder.Result(), nil +} + +func (r *redirectingTransport) headers(host string) (http.Header, bool) { + r.mu.Lock() + defer r.mu.Unlock() + header, ok := r.seen[host] + return header, ok +} + +// A 3xx re-enters the transport with the target's URL, so a transport that sets +// the credential unconditionally hands it to whatever host the redirect names. +func TestUpstreamAuthTransportWithholdsCredentialFromOtherHosts(t *testing.T) { + for _, name := range []string{"Authorization", "X-Proxy-Token"} { + t.Run(name, func(t *testing.T) { + base := &redirectingTransport{location: "http://redirect.test/v1/models"} + client := &http.Client{ + Transport: newUpstreamAuthTransport(base, name+": upstream-secret", "upstream.test"), + } + + resp, err := client.Get("http://upstream.test/v1/models") + if err != nil { + t.Fatalf("Get returned error: %v", err) + } + defer resp.Body.Close() + + configured, ok := base.headers("upstream.test") + if !ok { + t.Fatal("configured upstream was never called") + } + if got := configured.Get(name); got != "upstream-secret" { + t.Fatalf("configured upstream saw %s = %q, want the credential", name, got) + } + + target, ok := base.headers("redirect.test") + if !ok { + t.Fatal("redirect target was never reached, so this test proves nothing") + } + if got := target.Get(name); got != "" { + t.Fatalf("redirect target received the credential: %s = %q", name, got) + } + }) + } +} + +func TestCredentialedClientDoesNotFollowUpstreamRedirects(t *testing.T) { + base := &redirectingTransport{location: "http://redirect.test/v1/chat/completions"} + + srv, err := New(Config{ + UpstreamBaseURL: "http://upstream.test/v1", + UpstreamAuthHeader: "X-Proxy-Token: upstream-secret", + HTTPClient: &http.Client{Transport: base}, + PDFRenderer: pdf.RendererFunc(func(context.Context, []byte) ([][]byte, error) { return nil, nil }), + }) + if err != nil { + t.Fatalf("New returned error: %v", err) + } + + req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", bytes.NewBufferString(`{"model":"m","messages":[]}`)) + rec := httptest.NewRecorder() + srv.ServeHTTP(rec, req) + + if rec.Code != http.StatusFound { + t.Fatalf("status = %d, want the 302 passed through to the caller", rec.Code) + } + if strings.Contains(rec.Body.String(), "leaked") { + t.Fatalf("caller received the redirect target's body: %s", rec.Body.String()) + } + if _, ok := base.headers("redirect.test"); ok { + t.Fatal("credentialed client followed the redirect") + } +} + +// The model-bundled images configure no credential and must keep the stdlib's +// redirect handling. +func TestClientWithoutCredentialStillFollowsRedirects(t *testing.T) { + base := &redirectingTransport{location: "http://redirect.test/v1/chat/completions"} + + srv, err := New(Config{ + UpstreamBaseURL: "http://upstream.test/v1", + HTTPClient: &http.Client{Transport: base}, + PDFRenderer: pdf.RendererFunc(func(context.Context, []byte) ([][]byte, error) { return nil, nil }), + }) + if err != nil { + t.Fatalf("New returned error: %v", err) + } + + req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", bytes.NewBufferString(`{"model":"m","messages":[]}`)) + rec := httptest.NewRecorder() + srv.ServeHTTP(rec, req) + + if rec.Code != http.StatusOK { + t.Fatalf("status = %d, want 200 after following the redirect", rec.Code) + } + if !strings.Contains(rec.Body.String(), "leaked") { + t.Fatalf("redirect was not followed; body = %s", rec.Body.String()) + } +} diff --git a/internal/server/upstreamauth_test.go b/internal/server/upstreamauth_test.go index 174ce57..b94a008 100644 --- a/internal/server/upstreamauth_test.go +++ b/internal/server/upstreamauth_test.go @@ -17,9 +17,57 @@ func (c *capturingTransport) RoundTrip(req *http.Request) (*http.Response, error return recorder.Result(), nil } +// The single definition of a usable credential line. A capturing transport +// bypasses net/http's wire-level field checks, so these run at the predicate. +func TestValidUpstreamAuthHeader(t *testing.T) { + cases := []struct { + name string + header string + want bool + }{ + {"typical", "Authorization: Bearer token123", true}, + {"value keeps later colons", "Authorization: Bearer a:b", true}, + {"untrimmed", " X-Api-Key : abc123 ", true}, + {"empty", "", false}, + {"whitespace only", " : ", false}, + {"no colon", "InvalidHeader NoColon", false}, + {"no name", ": value", false}, + {"no value", "Name:", false}, + {"space in name", "Bad Name: secret", false}, + {"newline in value", "Authorization: Bearer a\nX-Evil: b", false}, + {"carriage return in value", "Authorization: Bearer a\rb", false}, + {"null in value", "Authorization: Bearer a\x00b", false}, + } + + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + if got := ValidUpstreamAuthHeader(tc.header); got != tc.want { + t.Fatalf("ValidUpstreamAuthHeader(%q) = %v, want %v", tc.header, got, tc.want) + } + }) + } +} + +// A header the predicate rejects must never reach the wire, where net/http +// would fail every request with an error that does not name the env var. +func TestRejectedHeaderIsNeverSentUpstream(t *testing.T) { + for _, header := range []string{"Bad Name: secret", "Authorization: Bearer a\nX-Evil: b"} { + capture := &capturingTransport{} + transport := newUpstreamAuthTransport(capture, header, "upstream.test") + + req := httptest.NewRequest(http.MethodGet, "http://upstream.test/v1/models", nil) + if _, err := transport.RoundTrip(req); err != nil { + t.Fatalf("RoundTrip returned error for %q: %v", header, err) + } + if len(capture.seen) != 0 { + t.Fatalf("header %q produced %v, want none", header, capture.seen) + } + } +} + func TestUpstreamAuthTransportAddsHeader(t *testing.T) { capture := &capturingTransport{} - transport := newUpstreamAuthTransport(capture, "Authorization: Bearer secret") + transport := newUpstreamAuthTransport(capture, "Authorization: Bearer secret", "upstream.test") req := httptest.NewRequest(http.MethodGet, "http://upstream.test/v1/models", nil) if _, err := transport.RoundTrip(req); err != nil { @@ -33,7 +81,7 @@ func TestUpstreamAuthTransportAddsHeader(t *testing.T) { func TestUpstreamAuthTransportTrimsWhitespace(t *testing.T) { capture := &capturingTransport{} - transport := newUpstreamAuthTransport(capture, " X-Api-Key : abc123 ") + transport := newUpstreamAuthTransport(capture, " X-Api-Key : abc123 ", "upstream.test") req := httptest.NewRequest(http.MethodGet, "http://upstream.test/v1/models", nil) if _, err := transport.RoundTrip(req); err != nil { @@ -48,7 +96,7 @@ func TestUpstreamAuthTransportTrimsWhitespace(t *testing.T) { func TestUpstreamAuthTransportIgnoresMalformedHeader(t *testing.T) { for _, header := range []string{"", " ", "NoColonHere", ": novalue", "Name:"} { capture := &capturingTransport{} - transport := newUpstreamAuthTransport(capture, header) + transport := newUpstreamAuthTransport(capture, header, "upstream.test") req := httptest.NewRequest(http.MethodGet, "http://upstream.test/v1/models", nil) if _, err := transport.RoundTrip(req); err != nil { @@ -62,7 +110,7 @@ func TestUpstreamAuthTransportIgnoresMalformedHeader(t *testing.T) { func TestUpstreamAuthTransportDoesNotMutateCallerRequest(t *testing.T) { capture := &capturingTransport{} - transport := newUpstreamAuthTransport(capture, "Authorization: Bearer secret") + transport := newUpstreamAuthTransport(capture, "Authorization: Bearer secret", "upstream.test") req := httptest.NewRequest(http.MethodGet, "http://upstream.test/v1/models", nil) if _, err := transport.RoundTrip(req); err != nil { From 7f8f0afb846fbae90ffc0ec81711fd8664744413 Mon Sep 17 00:00:00 2001 From: Harshaneel Gokhale Date: Tue, 4 Aug 2026 15:25:34 -0700 Subject: [PATCH 15/18] refactor: Call the shared upstream auth header predicate from main main.go had its own copy of the validation rules, so the startup warning could drift from what the transport actually does and claim no credential would be sent while one was. Co-Authored-By: Claude Opus 5 (1M context) --- cmd/localaik/main.go | 18 ++-------------- cmd/localaik/main_test.go | 45 +++++++++++---------------------------- 2 files changed, 14 insertions(+), 49 deletions(-) diff --git a/cmd/localaik/main.go b/cmd/localaik/main.go index c561885..48fccbd 100644 --- a/cmd/localaik/main.go +++ b/cmd/localaik/main.go @@ -5,7 +5,6 @@ import ( "log" "net/http" "os" - "strings" "time" "github.com/harshaneel/localaik/internal/pdf" @@ -19,27 +18,14 @@ func resolveFlagDefault(envName, fallback string) string { return fallback } -func isValidAuthHeader(header string) bool { - if header == "" { - return false - } - idx := strings.Index(header, ":") - if idx == -1 { - return false - } - name := strings.TrimSpace(header[:idx]) - value := strings.TrimSpace(header[idx+1:]) - return name != "" && value != "" -} - func main() { port := flag.String("port", resolveFlagDefault("PORT", "8090"), "port to listen on") upstream := flag.String("upstream", resolveFlagDefault("LK_UPSTREAM", "http://127.0.0.1:8080/v1"), "upstream OpenAI-compatible base URL") flag.Parse() authHeader := os.Getenv("LK_UPSTREAM_AUTH_HEADER") - if authHeader != "" && !isValidAuthHeader(authHeader) { - log.Printf("localaik: LK_UPSTREAM_AUTH_HEADER is set but is not a \"Name: value\" header line; no credential will be sent upstream") + if authHeader != "" && !server.ValidUpstreamAuthHeader(authHeader) { + log.Printf("localaik: LK_UPSTREAM_AUTH_HEADER is set but is not a valid \"Name: value\" header line; no credential will be sent upstream") } handler, err := server.New(server.Config{ diff --git a/cmd/localaik/main_test.go b/cmd/localaik/main_test.go index 63f2f60..6e37a1f 100644 --- a/cmd/localaik/main_test.go +++ b/cmd/localaik/main_test.go @@ -1,6 +1,10 @@ package main -import "testing" +import ( + "testing" + + "github.com/harshaneel/localaik/internal/server" +) func TestResolveFlagDefaultPrefersEnv(t *testing.T) { t.Setenv("LK_TEST_VALUE", "from-env") @@ -24,38 +28,13 @@ func TestResolveFlagDefaultUnsetFallsBack(t *testing.T) { } } -func TestIsValidAuthHeaderUnset(t *testing.T) { - if got := isValidAuthHeader(""); got != false { - t.Fatalf("isValidAuthHeader(\"\") = %v, want false", got) - } -} - -func TestIsValidAuthHeaderValid(t *testing.T) { - if got := isValidAuthHeader("Authorization: Bearer token123"); got != true { - t.Fatalf("isValidAuthHeader(\"Authorization: Bearer token123\") = %v, want true", got) +// The startup warning must be driven by the same predicate the transport uses; +// server.ValidUpstreamAuthHeader owns the table of cases. +func TestStartupWarningUsesTheServerPredicate(t *testing.T) { + if server.ValidUpstreamAuthHeader("Authorization: Bearer token123") != true { + t.Fatal("a valid header line was rejected") } -} - -func TestIsValidAuthHeaderNoColon(t *testing.T) { - if got := isValidAuthHeader("InvalidHeader NoColon"); got != false { - t.Fatalf("isValidAuthHeader(\"InvalidHeader NoColon\") = %v, want false", got) - } -} - -func TestIsValidAuthHeaderEmptyName(t *testing.T) { - if got := isValidAuthHeader(": value"); got != false { - t.Fatalf("isValidAuthHeader(\": value\") = %v, want false", got) - } -} - -func TestIsValidAuthHeaderEmptyValue(t *testing.T) { - if got := isValidAuthHeader("Name:"); got != false { - t.Fatalf("isValidAuthHeader(\"Name:\") = %v, want false", got) - } -} - -func TestIsValidAuthHeaderWhitespaceOnly(t *testing.T) { - if got := isValidAuthHeader(" : "); got != false { - t.Fatalf("isValidAuthHeader(\" : \") = %v, want false", got) + if server.ValidUpstreamAuthHeader("InvalidHeader NoColon") != false { + t.Fatal("a header line with no colon was accepted") } } From 96a098ab5845062a0ab8f847105a776bdb7fbef5 Mon Sep 17 00:00:00 2001 From: Harshaneel Gokhale Date: Tue, 4 Aug 2026 15:25:34 -0700 Subject: [PATCH 16/18] test: Cover PDF rendering in the proxy image, and fix doc findings The spec required the PDF path be exercised inside the proxy image rather than only against the Debian-based one, since alpine's poppler is a different build. The README quickstart also published port 8090 on every interface, which is what its own security warning tells operators not to do. Co-Authored-By: Claude Opus 5 (1M context) --- Makefile | 2 +- README.md | 9 +++- integration/proxy_image_test.go | 79 +++++++++++++++++++++++++++++++++ 3 files changed, 87 insertions(+), 3 deletions(-) diff --git a/Makefile b/Makefile index c601405..2d08e9b 100644 --- a/Makefile +++ b/Makefile @@ -22,7 +22,7 @@ help: 'make test Run lint, unit tests, and integration tests' \ 'make build Build the localaik binary' \ 'make docker-build Build the Docker image' \ - 'make docker-build-proxy Build the proxy-only image' \ + 'make docker-build-proxy Build the proxy-only image' \ 'make docker-up Start the Docker image on PORT' \ 'make docker-down Stop and remove the Docker container' diff --git a/README.md b/README.md index 089397a..23a45d0 100644 --- a/README.md +++ b/README.md @@ -191,7 +191,7 @@ chat-completions API, the `proxy` tag gives you the translation layer alone. It contains no model and no inference engine. ```bash -docker run -d -p 8090:8090 \ +docker run -d -p 127.0.0.1:8090:8090 \ -e LK_UPSTREAM=http://llama.internal:8080/v1 \ gokhalh/localaik:proxy ``` @@ -203,7 +203,9 @@ docker run -d -p 8090:8090 \ | `PORT` | `8090` | Port localaik listens on | `LK_UPSTREAM_AUTH_HEADER` is sent only to your upstream. Credentials that -clients send to localaik are still discarded and never forwarded. +clients send to localaik are still discarded and never forwarded. It is attached +only to requests whose host matches `LK_UPSTREAM`, and while it is set a +redirect from your upstream is returned to the caller rather than followed. `/health` returns 503 until your upstream answers, so existing healthchecks and CI wait loops work unchanged. @@ -394,6 +396,9 @@ docker build \ --build-arg MMPROJ_URL=... \ --build-arg MMPROJ_SHA256=... \ -t gokhalh/localaik:custom . + +# Proxy only, no model or inference engine (make docker-build-proxy) +docker build --target proxy -t gokhalh/localaik:proxy . ``` ## Limitations diff --git a/integration/proxy_image_test.go b/integration/proxy_image_test.go index f968c6b..d8e8e3a 100644 --- a/integration/proxy_image_test.go +++ b/integration/proxy_image_test.go @@ -3,8 +3,12 @@ package integration import ( + "bytes" + "encoding/base64" "encoding/json" "fmt" + "image/png" + "io" "net" "net/http" "net/http/httptest" @@ -27,9 +31,13 @@ func TestProxyImageRoundTripsAllProtocols(t *testing.T) { var authMu sync.Mutex var seenAuth string + var lastChatBody []byte stub := httptest.NewUnstartedServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { authMu.Lock() seenAuth = r.Header.Get("X-Proxy-Token") + if strings.HasSuffix(r.URL.Path, "/chat/completions") { + lastChatBody, _ = io.ReadAll(r.Body) + } authMu.Unlock() w.Header().Set("Content-Type", "application/json") if strings.HasSuffix(r.URL.Path, "/tokenize") { @@ -108,6 +116,69 @@ func TestProxyImageRoundTripsAllProtocols(t *testing.T) { }) } + // Alpine's poppler-utils is a different build from the full image's Debian + // one, so the PDF-to-PNG path has to be proven inside this container. + t.Run("pdf_to_png", func(t *testing.T) { + payload, err := json.Marshal(map[string]any{ + "contents": []any{map[string]any{ + "role": "user", + "parts": []any{ + map[string]any{"text": "Read this document."}, + map[string]any{"inlineData": map[string]string{ + "mimeType": "application/pdf", + "data": base64.StdEncoding.EncodeToString(buildSimplePDF([]string{ + "NAME: ALICE", + "CITY: BOSTON", + })), + }}, + }, + }}, + }) + if err != nil { + t.Fatalf("marshal request: %v", err) + } + + resp, err := http.Post("http://127.0.0.1:18097/v1beta/models/m:generateContent", "application/json", bytes.NewReader(payload)) + if err != nil { + t.Fatalf("post: %v", err) + } + defer resp.Body.Close() + + if resp.StatusCode != http.StatusOK { + body, _ := io.ReadAll(resp.Body) + t.Fatalf("status = %d, want 200; body=%s", resp.StatusCode, body) + } + + authMu.Lock() + body := string(lastChatBody) + authMu.Unlock() + + if strings.Contains(body, "application/pdf") { + t.Fatal("upstream received the raw PDF instead of rendered pages") + } + + const prefix = "data:image/png;base64," + start := strings.Index(body, prefix) + if start == -1 { + t.Fatalf("upstream received no rendered PNG page; body=%s", truncateForLog(body)) + } + encoded := body[start+len(prefix):] + if end := strings.IndexByte(encoded, '"'); end != -1 { + encoded = encoded[:end] + } + page, err := base64.StdEncoding.DecodeString(encoded) + if err != nil { + t.Fatalf("rendered page was not valid base64: %v", err) + } + config, err := png.DecodeConfig(bytes.NewReader(page)) + if err != nil { + t.Fatalf("rendered page was not a valid PNG: %v", err) + } + if config.Width == 0 || config.Height == 0 { + t.Fatalf("rendered page is %dx%d", config.Width, config.Height) + } + }) + authMu.Lock() got := seenAuth authMu.Unlock() @@ -116,6 +187,14 @@ func TestProxyImageRoundTripsAllProtocols(t *testing.T) { } } +func truncateForLog(body string) string { + const limit = 300 + if len(body) <= limit { + return body + } + return body[:limit] + "...(truncated)" +} + func mapKeys(m map[string]any) []string { keys := make([]string, 0, len(m)) for k := range m { From 54cd22e0c44b422e1475fe46009dedd66762ae3c Mon Sep 17 00:00:00 2001 From: Harshaneel Gokhale Date: Tue, 4 Aug 2026 15:34:37 -0700 Subject: [PATCH 17/18] docs: Correct the proxy build tag note and record the pin's limit The make target tags $(IMAGE)-proxy, not gokhalh/localaik:proxy, so the annotation implied a tag the command does not produce. The CheckRedirect comment now names the invariant a reviewer found: the transport pins hostname only, so relaxing redirect handling would reopen cross-port and scheme-downgrade leakage. Co-Authored-By: Claude Opus 5 (1M context) --- README.md | 2 +- internal/server/server.go | 5 +++-- 2 files changed, 4 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 23a45d0..cf55cc8 100644 --- a/README.md +++ b/README.md @@ -397,7 +397,7 @@ docker build \ --build-arg MMPROJ_SHA256=... \ -t gokhalh/localaik:custom . -# Proxy only, no model or inference engine (make docker-build-proxy) +# Proxy only, no model or inference engine docker build --target proxy -t gokhalh/localaik:proxy . ``` diff --git a/internal/server/server.go b/internal/server/server.go index 1aca6c1..47fd268 100644 --- a/internal/server/server.go +++ b/internal/server/server.go @@ -52,8 +52,9 @@ func New(cfg Config) (*Server, error) { if cfg.UpstreamAuthHeader != "" { clone := *client clone.Transport = newUpstreamAuthTransport(clone.Transport, cfg.UpstreamAuthHeader, parsed.Hostname()) - // Scoped here so the bundled-model images keep the stdlib default; a - // followed redirect would send prompt content to the target. + // Scoped here so the bundled-model images keep the stdlib default. The + // transport pins hostname only, so relaxing this reopens port and + // scheme leaks. clone.CheckRedirect = func(*http.Request, []*http.Request) error { return http.ErrUseLastResponse } From db2361142d8a78d3a5e2cd3c1a18c84ce8082fc3 Mon Sep 17 00:00:00 2001 From: Harshaneel Gokhale Date: Tue, 4 Aug 2026 17:41:04 -0700 Subject: [PATCH 18/18] chore: Keep docs out of the repo Design docs and plans are local working notes, the same treatment /specs/ already gets. Untracks the two that landed, which stay on disk. Co-Authored-By: Claude Opus 5 (1M context) --- .gitignore | 1 + .../plans/2026-08-04-proxy-only-image.md | 1037 ----------------- .../2026-08-04-proxy-only-image-design.md | 177 --- 3 files changed, 1 insertion(+), 1214 deletions(-) delete mode 100644 docs/superpowers/plans/2026-08-04-proxy-only-image.md delete mode 100644 docs/superpowers/specs/2026-08-04-proxy-only-image-design.md diff --git a/.gitignore b/.gitignore index 0b70a76..8259863 100644 --- a/.gitignore +++ b/.gitignore @@ -1,3 +1,4 @@ /localaik .cache/ /specs/ +/docs/ diff --git a/docs/superpowers/plans/2026-08-04-proxy-only-image.md b/docs/superpowers/plans/2026-08-04-proxy-only-image.md deleted file mode 100644 index 5b0a772..0000000 --- a/docs/superpowers/plans/2026-08-04-proxy-only-image.md +++ /dev/null @@ -1,1037 +0,0 @@ -# Proxy-only image (`:proxy`) Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** Publish a `:proxy` image containing only the translating proxy and `pdftoppm`, for users who already run an OpenAI-compatible model server. - -**Architecture:** Three changes, each independently testable. The Go binary learns two environment fallbacks and gains an optional upstream credential injected at the HTTP transport layer, so no upstream call site changes. The `Dockerfile` gains a third build stage. CI publishes it as a new matrix entry. - -**Tech Stack:** Go 1.25, standard library only. Docker multi-stage build, alpine base. GitHub Actions with `docker/build-push-action@v6`. - -## Global Constraints - -- Go version: `1.25` (from `go.mod`). Standard library only; add no dependencies. -- Run `gofmt -w` on every Go file touched. `make lint` runs `gofmt -l` and `go vet ./...` and must stay clean. -- No em-dashes in any file, including comments, docs and commit messages. -- Comments: default to zero. One line only when the WHY is non-obvious. Never restate the code. Never explain rejected alternatives or review feedback. -- The llama.cpp stage must remain the LAST stage in `Dockerfile`, so `docker build .` and `make docker-build` keep producing the full image. -- `:proxy` must never be tagged `latest`. -- `LK_UPSTREAM_AUTH_HEADER` is a secret. Never log its value. Never add `set -x` anywhere it is in scope. -- Existing behaviour must not change: client credentials (`Authorization`, `X-Api-Key`, `X-Goog-Api-Key`) are still stripped and never forwarded upstream. - -## File Structure - -| File | Responsibility | -| --- | --- | -| `internal/server/upstreamauth.go` (create) | The `RoundTripper` that adds the proxy's own credential to upstream requests. Isolated so the security-sensitive logic is one small readable unit. | -| `internal/server/upstreamauth_test.go` (create) | Tests for that transport in isolation. | -| `internal/server/server.go` (modify) | `Config` gains `UpstreamAuthHeader`; `New` wraps the client transport when it is set. | -| `internal/server/auth_integration_test.go` (create) | Proves the header reaches all four upstream paths while client credentials are still stripped. | -| `cmd/localaik/main.go` (modify) | Environment fallbacks for `--upstream`, plus reading `LK_UPSTREAM_AUTH_HEADER`. | -| `cmd/localaik/main_test.go` (create) | Flag over environment over default precedence. | -| `Dockerfile` (modify) | New `proxy` stage inserted before the llama.cpp stage. | -| `.github/workflows/release.yml` (modify) | Matrix entry with a build target. | -| `Makefile` (modify) | `docker-build-proxy` target for local verification. | -| `integration/proxy_image_test.go` (create) | Behind `docker_integration`: builds and exercises the image. | -| `README.md` (modify) | Tag table row, configuration, and the security warning. | - ---- - -### Task 1: Upstream auth transport - -Adds the credential-injecting `RoundTripper` and wires it into `server.New`. Nothing consumes it yet. - -**Files:** -- Create: `internal/server/upstreamauth.go` -- Create: `internal/server/upstreamauth_test.go` -- Modify: `internal/server/server.go:18-22` (Config), `internal/server/server.go:46-49` (client setup) - -**Interfaces:** -- Consumes: nothing. -- Produces: - - `server.Config.UpstreamAuthHeader string` (new field, optional) - - `func newUpstreamAuthTransport(base http.RoundTripper, header string) http.RoundTripper` - - Header format is a full header line, `"Name: value"`, split on the first colon. - -- [ ] **Step 1: Write the failing test** - -Create `internal/server/upstreamauth_test.go`: - -```go -package server - -import ( - "net/http" - "net/http/httptest" - "testing" -) - -type capturingTransport struct { - seen http.Header -} - -func (c *capturingTransport) RoundTrip(req *http.Request) (*http.Response, error) { - c.seen = req.Header.Clone() - recorder := httptest.NewRecorder() - recorder.WriteHeader(http.StatusOK) - return recorder.Result(), nil -} - -func TestUpstreamAuthTransportAddsHeader(t *testing.T) { - capture := &capturingTransport{} - transport := newUpstreamAuthTransport(capture, "Authorization: Bearer secret") - - req := httptest.NewRequest(http.MethodGet, "http://upstream.test/v1/models", nil) - if _, err := transport.RoundTrip(req); err != nil { - t.Fatalf("RoundTrip returned error: %v", err) - } - - if got := capture.seen.Get("Authorization"); got != "Bearer secret" { - t.Fatalf("Authorization = %q, want %q", got, "Bearer secret") - } -} - -func TestUpstreamAuthTransportTrimsWhitespace(t *testing.T) { - capture := &capturingTransport{} - transport := newUpstreamAuthTransport(capture, " X-Api-Key : abc123 ") - - req := httptest.NewRequest(http.MethodGet, "http://upstream.test/v1/models", nil) - if _, err := transport.RoundTrip(req); err != nil { - t.Fatalf("RoundTrip returned error: %v", err) - } - - if got := capture.seen.Get("X-Api-Key"); got != "abc123" { - t.Fatalf("X-Api-Key = %q, want %q", got, "abc123") - } -} - -func TestUpstreamAuthTransportIgnoresMalformedHeader(t *testing.T) { - for _, header := range []string{"", " ", "NoColonHere", ": novalue", "Name:"} { - capture := &capturingTransport{} - transport := newUpstreamAuthTransport(capture, header) - - req := httptest.NewRequest(http.MethodGet, "http://upstream.test/v1/models", nil) - if _, err := transport.RoundTrip(req); err != nil { - t.Fatalf("RoundTrip returned error for %q: %v", header, err) - } - if len(capture.seen) != 0 { - t.Fatalf("header %q produced %v, want none", header, capture.seen) - } - } -} - -func TestUpstreamAuthTransportDoesNotMutateCallerRequest(t *testing.T) { - capture := &capturingTransport{} - transport := newUpstreamAuthTransport(capture, "Authorization: Bearer secret") - - req := httptest.NewRequest(http.MethodGet, "http://upstream.test/v1/models", nil) - if _, err := transport.RoundTrip(req); err != nil { - t.Fatalf("RoundTrip returned error: %v", err) - } - - if req.Header.Get("Authorization") != "" { - t.Fatal("RoundTrip mutated the caller's request") - } -} -``` - -- [ ] **Step 2: Run test to verify it fails** - -Run: `go test ./internal/server/ -run UpstreamAuthTransport -v` -Expected: FAIL, `undefined: newUpstreamAuthTransport` - -- [ ] **Step 3: Write minimal implementation** - -Create `internal/server/upstreamauth.go`: - -```go -package server - -import ( - "net/http" - "strings" -) - -type upstreamAuthTransport struct { - base http.RoundTripper - name string - value string -} - -// newUpstreamAuthTransport returns base unchanged when header is not a usable -// "Name: value" line, so a misconfigured value cannot silently drop requests. -func newUpstreamAuthTransport(base http.RoundTripper, header string) http.RoundTripper { - if base == nil { - base = http.DefaultTransport - } - - name, value, found := strings.Cut(header, ":") - name = strings.TrimSpace(name) - value = strings.TrimSpace(value) - if !found || name == "" || value == "" { - return base - } - - return &upstreamAuthTransport{base: base, name: name, value: value} -} - -func (t *upstreamAuthTransport) RoundTrip(req *http.Request) (*http.Response, error) { - clone := req.Clone(req.Context()) - clone.Header.Set(t.name, t.value) - return t.base.RoundTrip(clone) -} -``` - -- [ ] **Step 4: Run test to verify it passes** - -Run: `go test ./internal/server/ -run UpstreamAuthTransport -v` -Expected: PASS, 4 tests - -- [ ] **Step 5: Wire it into Config** - -In `internal/server/server.go`, add the field to `Config`: - -```go -type Config struct { - UpstreamBaseURL string - UpstreamAuthHeader string - HTTPClient *http.Client - PDFRenderer pdf.Renderer -} -``` - -Then in `New`, replace the existing client setup block: - -```go - client := cfg.HTTPClient - if client == nil { - client = &http.Client{} - } -``` - -with: - -```go - client := cfg.HTTPClient - if client == nil { - client = &http.Client{} - } - if cfg.UpstreamAuthHeader != "" { - clone := *client - clone.Transport = newUpstreamAuthTransport(clone.Transport, cfg.UpstreamAuthHeader) - client = &clone - } -``` - -- [ ] **Step 6: Run the full server suite** - -Run: `go test ./internal/server/ -count=1` -Expected: PASS. Copying the client rather than mutating it means every existing test that passes its own `HTTPClient` is unaffected. - -- [ ] **Step 7: Format, lint, commit** - -```bash -gofmt -w internal/server/upstreamauth.go internal/server/upstreamauth_test.go internal/server/server.go -make lint -git add internal/server/upstreamauth.go internal/server/upstreamauth_test.go internal/server/server.go -git commit -m "feat: Add optional upstream auth header to the proxy - -Injected at the transport layer so every upstream request carries it without -each call site opting in." -``` - ---- - -### Task 2: Prove the header reaches every upstream path - -Task 1 tested the transport alone. This proves the wiring covers all four upstream endpoints and that client credentials are still stripped. - -**Files:** -- Create: `internal/server/auth_integration_test.go` - -**Interfaces:** -- Consumes: `server.Config.UpstreamAuthHeader` from Task 1; `roundTripHandler` from `internal/server/server_test.go:147`; `newTestServer` from `internal/server/meta_test.go:17`. -- Produces: nothing. - -- [ ] **Step 1: Write the failing test** - -Create `internal/server/auth_integration_test.go`: - -```go -package server - -import ( - "bytes" - "context" - "net/http" - "net/http/httptest" - "testing" - - "github.com/harshaneel/localaik/internal/pdf" - openaip "github.com/harshaneel/localaik/internal/protocol/openai" -) - -// Every upstream route must carry the proxy's credential and none of the -// caller's. -func TestUpstreamAuthHeaderReachesEveryUpstreamPath(t *testing.T) { - cases := []struct { - name string - method string - path string - body string - }{ - {"openai_chat", http.MethodPost, "/v1/chat/completions", `{"model":"m","messages":[]}`}, - {"openai_models", http.MethodGet, "/v1/models", ""}, - {"gemini_generate", http.MethodPost, "/v1beta/models/m:generateContent", `{"contents":[{"role":"user","parts":[{"text":"hi"}]}]}`}, - {"gemini_count_tokens", http.MethodPost, "/v1beta/models/m:countTokens", `{"contents":[{"parts":[{"text":"hi"}]}]}`}, - {"anthropic_messages", http.MethodPost, "/v1/messages", `{"max_tokens":8,"messages":[{"role":"user","content":"hi"}]}`}, - {"anthropic_count_tokens", http.MethodPost, "/v1/messages/count_tokens", `{"messages":[{"role":"user","content":"hi"}]}`}, - } - - for _, tc := range cases { - t.Run(tc.name, func(t *testing.T) { - var called bool - var seenAuth, seenClientAuth, seenAPIKey, seenGoogKey string - - upstream := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - called = true - seenAuth = r.Header.Get("X-Proxy-Token") - seenClientAuth = r.Header.Get("Authorization") - seenAPIKey = r.Header.Get("X-Api-Key") - seenGoogKey = r.Header.Get("X-Goog-Api-Key") - - switch r.URL.Path { - case "/tokenize": - writeJSON(w, http.StatusOK, map[string]any{"tokens": []int{1, 2}}) - case "/v1/models": - writeJSON(w, http.StatusOK, openaip.ModelList{Object: "list", Data: []openaip.Model{{ID: "m"}}}) - default: - writeJSON(w, http.StatusOK, openaip.ChatCompletionResponse{ - Choices: []openaip.Choice{{Message: openaip.Message{Content: "ok"}, FinishReason: "stop"}}, - }) - } - }) - - srv, err := New(Config{ - UpstreamBaseURL: "http://upstream.test/v1", - UpstreamAuthHeader: "X-Proxy-Token: upstream-secret", - HTTPClient: &http.Client{Transport: roundTripHandler{handler: upstream}}, - PDFRenderer: pdf.RendererFunc(func(context.Context, []byte) ([][]byte, error) { return nil, nil }), - }) - if err != nil { - t.Fatalf("New returned error: %v", err) - } - - var reader *bytes.Buffer - if tc.body != "" { - reader = bytes.NewBufferString(tc.body) - } else { - reader = bytes.NewBuffer(nil) - } - req := httptest.NewRequest(tc.method, tc.path, reader) - req.Header.Set("Authorization", "Bearer client-secret") - req.Header.Set("X-Api-Key", "client-anthropic-key") - req.Header.Set("X-Goog-Api-Key", "client-google-key") - - rec := httptest.NewRecorder() - srv.ServeHTTP(rec, req) - - if rec.Code != http.StatusOK { - t.Fatalf("status = %d, want 200; body=%s", rec.Code, rec.Body.String()) - } - if !called { - t.Fatal("upstream was never called, so the header check proves nothing") - } - if seenAuth != "upstream-secret" { - t.Fatalf("X-Proxy-Token = %q, want the proxy credential", seenAuth) - } - if seenClientAuth != "" || seenAPIKey != "" || seenGoogKey != "" { - t.Fatalf("client credentials leaked upstream: auth=%q apikey=%q googkey=%q", seenClientAuth, seenAPIKey, seenGoogKey) - } - }) - } -} - -func TestNoUpstreamAuthHeaderWhenUnset(t *testing.T) { - var seen http.Header - - upstream := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - seen = r.Header.Clone() - writeJSON(w, http.StatusOK, openaip.ChatCompletionResponse{ - Choices: []openaip.Choice{{Message: openaip.Message{Content: "ok"}, FinishReason: "stop"}}, - }) - }) - - srv := newTestServer(t, upstream) - - req := httptest.NewRequest(http.MethodPost, "/v1/messages", bytes.NewBufferString(`{"max_tokens":8,"messages":[{"role":"user","content":"hi"}]}`)) - rec := httptest.NewRecorder() - srv.ServeHTTP(rec, req) - - if rec.Code != http.StatusOK { - t.Fatalf("status = %d, want 200; body=%s", rec.Code, rec.Body.String()) - } - if got := seen.Get("Authorization"); got != "" { - t.Fatalf("Authorization = %q, want none when no credential is configured", got) - } -} - -// The Gemini streaming route builds its own request; confirm the credential is -// present there too. -func TestUpstreamAuthHeaderOnStreamingRoute(t *testing.T) { - var seen string - - upstream := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - seen = r.Header.Get("X-Proxy-Token") - w.Header().Set("Content-Type", "text/event-stream") - w.WriteHeader(http.StatusOK) - _, _ = w.Write([]byte("data: {\"choices\":[{\"delta\":{\"content\":\"hi\"},\"finish_reason\":\"stop\"}]}\n\ndata: [DONE]\n\n")) - }) - - srv, err := New(Config{ - UpstreamBaseURL: "http://upstream.test/v1", - UpstreamAuthHeader: "X-Proxy-Token: upstream-secret", - HTTPClient: &http.Client{Transport: roundTripHandler{handler: upstream}}, - PDFRenderer: pdf.RendererFunc(func(context.Context, []byte) ([][]byte, error) { return nil, nil }), - }) - if err != nil { - t.Fatalf("New returned error: %v", err) - } - - body := `{"contents":[{"role":"user","parts":[{"text":"hi"}]}]}` - req := httptest.NewRequest(http.MethodPost, "/v1beta/models/m:streamGenerateContent", bytes.NewBufferString(body)) - rec := httptest.NewRecorder() - srv.ServeHTTP(rec, req) - - if seen != "upstream-secret" { - t.Fatalf("X-Proxy-Token = %q on the streaming route, want the proxy credential", seen) - } -} -``` - -- [ ] **Step 2: Run the tests** - -Run: `go test ./internal/server/ -run 'UpstreamAuthHeader|NoUpstreamAuthHeader' -v` -Expected: PASS. Task 1 already made this work; these tests exist to lock the behaviour against future call sites. - -If any subtest fails with the credential missing, the cause is an upstream request that bypasses `s.client`. Find it and route it through `s.client` rather than weakening the test. - -- [ ] **Step 3: Format, lint, commit** - -```bash -gofmt -w internal/server/auth_integration_test.go -make lint -go test ./internal/server/ -count=1 -git add internal/server/auth_integration_test.go -git commit -m "test: Cover upstream auth on every upstream route - -Locks both halves at once: the proxy credential is added, the caller's is not -forwarded." -``` - ---- - -### Task 3: Environment fallbacks in main.go - -**Files:** -- Modify: `cmd/localaik/main.go:14-32` -- Create: `cmd/localaik/main_test.go` - -**Interfaces:** -- Consumes: `server.Config.UpstreamAuthHeader` from Task 1. -- Produces: - - `func resolveFlagDefault(envName, fallback string) string` - - Environment names: `LK_UPSTREAM`, `LK_UPSTREAM_AUTH_HEADER`, existing `PORT`. - -- [ ] **Step 1: Write the failing test** - -Create `cmd/localaik/main_test.go`: - -```go -package main - -import "testing" - -func TestResolveFlagDefaultPrefersEnv(t *testing.T) { - t.Setenv("LK_TEST_VALUE", "from-env") - - if got := resolveFlagDefault("LK_TEST_VALUE", "fallback"); got != "from-env" { - t.Fatalf("resolveFlagDefault = %q, want from-env", got) - } -} - -func TestResolveFlagDefaultFallsBack(t *testing.T) { - t.Setenv("LK_TEST_VALUE", "") - - if got := resolveFlagDefault("LK_TEST_VALUE", "fallback"); got != "fallback" { - t.Fatalf("resolveFlagDefault = %q, want fallback", got) - } -} - -func TestResolveFlagDefaultUnsetFallsBack(t *testing.T) { - if got := resolveFlagDefault("LK_DEFINITELY_UNSET_VALUE", "fallback"); got != "fallback" { - t.Fatalf("resolveFlagDefault = %q, want fallback", got) - } -} -``` - -- [ ] **Step 2: Run test to verify it fails** - -Run: `go test ./cmd/localaik/ -v` -Expected: FAIL, `undefined: resolveFlagDefault` - -- [ ] **Step 3: Write minimal implementation** - -Replace the body of `main.go` up to and including the `server.New` call: - -```go -func resolveFlagDefault(envName, fallback string) string { - if value := os.Getenv(envName); value != "" { - return value - } - return fallback -} - -func main() { - port := flag.String("port", resolveFlagDefault("PORT", "8090"), "port to listen on") - upstream := flag.String("upstream", resolveFlagDefault("LK_UPSTREAM", "http://127.0.0.1:8080/v1"), "upstream OpenAI-compatible base URL") - flag.Parse() - - handler, err := server.New(server.Config{ - UpstreamBaseURL: *upstream, - UpstreamAuthHeader: os.Getenv("LK_UPSTREAM_AUTH_HEADER"), - HTTPClient: &http.Client{}, - PDFRenderer: pdf.NewExecRenderer("pdftoppm"), - }) - if err != nil { - log.Fatalf("localaik: %v", err) - } -``` - -Leave the rest of `main` unchanged. Delete the old `defaultPort` block that this replaces. - -- [ ] **Step 4: Run test to verify it passes** - -Run: `go test ./cmd/localaik/ -v` -Expected: PASS, 3 tests - -- [ ] **Step 5: Verify flag still beats environment** - -Run: - -```bash -LK_UPSTREAM=http://from-env:9999/v1 go run ./cmd/localaik --upstream http://from-flag:1111/v1 --port 18099 & -sleep 2 -curl -s -o /dev/null -w '%{http_code}\n' http://127.0.0.1:18099/health -kill %1 -``` - -Expected: `503`. A 503 proves it started and tried to reach an upstream. Then confirm the flag won by checking the process was pointed at `from-flag`: rerun without the flag and confirm it still starts. There is no endpoint that echoes the upstream, so this step verifies startup only; precedence itself is covered by the unit tests. - -- [ ] **Step 6: Format, lint, commit** - -```bash -gofmt -w cmd/localaik/main.go cmd/localaik/main_test.go -make lint -go test ./cmd/... ./internal/... -count=1 -git add cmd/localaik/main.go cmd/localaik/main_test.go -git commit -m "feat: Read upstream and auth header from the environment - -Follows the pattern PORT already set, so the container needs no shell wrapper." -``` - ---- - -### Task 4: The `proxy` Dockerfile stage - -**Files:** -- Modify: `Dockerfile` (insert a stage between line 5 and line 9) -- Modify: `Makefile:20-27` area (add a target) - -**Interfaces:** -- Consumes: the `proxy-builder` stage that already exists at `Dockerfile:1`. -- Produces: build target named `proxy`; image entrypoint runs `localaik` under `tini`. - -- [ ] **Step 1: Add the stage** - -In `Dockerfile`, immediately after the `proxy-builder` stage (after line 5) and before the `FROM ghcr.io/ggml-org/llama.cpp@sha256:...` line, insert: - -```dockerfile -FROM alpine:3 AS proxy -RUN apk add --no-cache ca-certificates poppler-utils tini -COPY --from=proxy-builder /out/localaik /usr/local/bin/localaik -ENV PORT=8090 -HEALTHCHECK --interval=5s --timeout=3s --start-period=5s \ - CMD wget -q -O - "http://127.0.0.1:${PORT:-8090}/health" >/dev/null 2>&1 || exit 1 -EXPOSE 8090 -ENTRYPOINT ["tini", "--", "localaik"] -``` - -Two notes, both verified against a real build of this stage: - -`wget` rather than `curl`, because alpine's busybox already provides `wget` and this avoids installing curl solely for the healthcheck. The llama.cpp stage keeps using `curl`, which it already installs. - -`alpine:3` is a moving tag. The repo pins the llama.cpp base by digest, so pinning this one by digest is more consistent. Resolve it during implementation with `docker inspect alpine:3 --format '{{index .RepoDigests 0}}'` and use that. A moving tag is acceptable if you prefer, since nothing here depends on a specific alpine version. - -Confirmed present in `alpine:3` at time of writing: `pdftoppm` 25.12.0 from `poppler-utils`, `tini`, and busybox `wget`. Base plus these three packages measures 35 MB before the binary is copied in. - -- [ ] **Step 2: Confirm the llama.cpp stage is still last** - -Run: `grep -n '^FROM' Dockerfile` -Expected: three lines, with `ghcr.io/ggml-org/llama.cpp` on the last one. - -- [ ] **Step 3: Confirm the default build is unchanged** - -Run: `docker build -t localaik:default-check . && docker image inspect localaik:default-check --format '{{.Size}}'` -Expected: a size over 3000000000, proving the default target is still the full image. - -- [ ] **Step 4: Build the proxy image and record its size** - -Run: - -```bash -docker build --target proxy -t localaik:proxy-check . -docker image inspect localaik:proxy-check --format '{{.Size}}' | awk '{printf "%.0f MB\n", $1/1000000}' -``` - -Expected: roughly 43 MB. The base plus packages measures 35 MB and the static binary adds about 8 MB. Anything above 60 MB means something unintended got copied in. Write the measured number down; Task 7 puts it in the README. - -- [ ] **Step 5: Verify pdftoppm is present and executable** - -Run: `docker run --rm --entrypoint pdftoppm localaik:proxy-check -v` -Expected: `pdftoppm version 25.12.0` or later, printed to stderr. `pdftoppm -v` exits non-zero on some builds while still printing the version, so treat printed output as success. - -- [ ] **Step 6: Verify the binary starts and reports not-ready** - -Run: - -```bash -docker run -d --name proxy-smoke -p 18098:8090 localaik:proxy-check -sleep 3 -curl -s -o /dev/null -w '%{http_code}\n' http://127.0.0.1:18098/health -docker rm -f proxy-smoke -``` - -Expected: `503`. There is no upstream, so 503 is correct and proves the server is listening. - -- [ ] **Step 7: Add the Makefile target** - -In `Makefile`, add `docker-build-proxy` to the `.PHONY` list on line 12, add a help line in the `help` target after the `docker-build` line: - -``` - 'make docker-build-proxy Build the proxy-only image' \ -``` - -and add the target after `docker-build`: - -```makefile -docker-build-proxy: - @docker build --target proxy -t "$(IMAGE)-proxy" . -``` - -- [ ] **Step 8: Verify the target works** - -Run: `make docker-build-proxy && docker images --format '{{.Repository}}:{{.Tag}}' | grep proxy` -Expected: the tagged image is listed. - -- [ ] **Step 9: Commit** - -```bash -git add Dockerfile Makefile -git commit -m "feat: Add a proxy-only Dockerfile stage - -alpine plus poppler-utils and the binary, no inference stack. The llama.cpp -stage stays last so the default build is unchanged." -``` - ---- - -### Task 5: Image integration test - -**Files:** -- Create: `integration/proxy_image_test.go` - -**Interfaces:** -- Consumes: the `proxy` build target from Task 4. -- Produces: nothing. - -- [ ] **Step 1: Write the test** - -Create `integration/proxy_image_test.go`: - -```go -//go:build docker_integration - -package integration - -import ( - "encoding/json" - "fmt" - "net" - "net/http" - "net/http/httptest" - "os/exec" - "strings" - "testing" - "time" -) - -// Exercises the built proxy image against a stub upstream, proving all three -// protocol surfaces round-trip without an inference stack in the container. -func TestProxyImageRoundTripsAllProtocols(t *testing.T) { - image := "localaik:proxy-integration" - - build := exec.Command("docker", "build", "--target", "proxy", "-t", image, "..") - if out, err := build.CombinedOutput(); err != nil { - t.Fatalf("docker build failed: %v\n%s", err, out) - } - - var seenAuth string - stub := httptest.NewUnstartedServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - seenAuth = r.Header.Get("X-Proxy-Token") - w.Header().Set("Content-Type", "application/json") - if strings.HasSuffix(r.URL.Path, "/tokenize") { - _, _ = w.Write([]byte(`{"tokens":[1,2,3]}`)) - return - } - _, _ = w.Write([]byte(`{"choices":[{"index":0,"message":{"role":"assistant","content":"stubbed"},"finish_reason":"stop"}]}`)) - })) - - listener, err := net.Listen("tcp", "0.0.0.0:0") - if err != nil { - t.Fatalf("listen: %v", err) - } - stub.Listener = listener - stub.Start() - defer stub.Close() - - stubPort := listener.Addr().(*net.TCPAddr).Port - upstream := fmt.Sprintf("http://host.docker.internal:%d/v1", stubPort) - - run := exec.Command("docker", "run", "-d", "--name", "proxy-integration", - "--add-host", "host.docker.internal:host-gateway", - "-p", "18097:8090", - "-e", "LK_UPSTREAM="+upstream, - "-e", "LK_UPSTREAM_AUTH_HEADER=X-Proxy-Token: integration-secret", - image) - if out, err := run.CombinedOutput(); err != nil { - t.Fatalf("docker run failed: %v\n%s", err, out) - } - defer exec.Command("docker", "rm", "-f", "proxy-integration").Run() - - waitForHealth(t, "http://127.0.0.1:18097/health") - - cases := []struct { - name string - path string - body string - }{ - {"openai", "/v1/chat/completions", `{"model":"m","messages":[{"role":"user","content":"hi"}]}`}, - {"gemini", "/v1beta/models/m:generateContent", `{"contents":[{"role":"user","parts":[{"text":"hi"}]}]}`}, - {"anthropic", "/v1/messages", `{"max_tokens":16,"messages":[{"role":"user","content":"hi"}]}`}, - } - - for _, tc := range cases { - t.Run(tc.name, func(t *testing.T) { - resp, err := http.Post("http://127.0.0.1:18097"+tc.path, "application/json", strings.NewReader(tc.body)) - if err != nil { - t.Fatalf("post: %v", err) - } - defer resp.Body.Close() - - if resp.StatusCode != http.StatusOK { - t.Fatalf("status = %d, want 200", resp.StatusCode) - } - var decoded map[string]any - if err := json.NewDecoder(resp.Body).Decode(&decoded); err != nil { - t.Fatalf("decode: %v", err) - } - if len(decoded) == 0 { - t.Fatal("empty response body") - } - }) - } - - if seenAuth != "integration-secret" { - t.Fatalf("upstream saw X-Proxy-Token = %q, want integration-secret", seenAuth) - } -} - -func waitForHealth(t *testing.T, url string) { - t.Helper() - for i := 0; i < 60; i++ { - resp, err := http.Get(url) - if err == nil { - resp.Body.Close() - if resp.StatusCode == http.StatusOK { - return - } - } - time.Sleep(time.Second) - } - t.Fatalf("never became healthy: %s", url) -} -``` - -- [ ] **Step 2: Run it** - -Run: `go test -count=1 -tags=docker_integration ./integration -run ProxyImage -v` -Expected: PASS, 3 subtests. Requires a running Docker daemon. - -If `host.docker.internal` does not resolve on Linux, the `--add-host ...:host-gateway` flag is what makes it work; confirm the Docker version supports it with `docker --version` (needs 20.10 or newer). - -- [ ] **Step 3: Confirm it does not run in the default suite** - -Run: `go test ./integration/ -count=1` -Expected: PASS without building any image, since the file is behind the `docker_integration` tag. - -- [ ] **Step 4: Commit** - -```bash -gofmt -w integration/proxy_image_test.go -make lint -git add integration/proxy_image_test.go -git commit -m "test: Exercise the built proxy image on all three protocols" -``` - ---- - -### Task 6: Publish the tag - -**Files:** -- Modify: `.github/workflows/release.yml:52-66` (matrix), `:96-107` (build step) - -**Interfaces:** -- Consumes: the `proxy` build target from Task 4. -- Produces: DockerHub tags `proxy` and `-proxy`. - -- [ ] **Step 1: Add a target to the existing matrix entries** - -In the `docker` job's matrix, add `target: ""` to both existing entries, so all entries carry the same keys: - -```yaml - - variant: gemma3-4b - latest: true - target: "" - model_url: https://huggingface.co/lmstudio-community/gemma-3-4b-it-GGUF/resolve/c536c4707e747055eecad7da65d46b6fb0ebaa79/gemma-3-4b-it-Q4_K_M.gguf -``` - -Do the same for `gemma3-12b`. Leave every existing URL and checksum untouched. - -- [ ] **Step 2: Add the proxy entry** - -Append to the matrix, after the `gemma3-12b` entry: - -```yaml - - variant: proxy - latest: false - target: proxy - model_url: "" - model_sha256: "" - mmproj_url: "" - mmproj_sha256: "" -``` - -The empty model values are required because every matrix entry must define the same keys for the build-args block to render. - -- [ ] **Step 3: Pass the target to the build** - -In the `Build and push image` step, add a `target` key above `platforms`: - -```yaml - context: . - file: ./Dockerfile - target: ${{ matrix.target }} - platforms: linux/amd64,linux/arm64 -``` - -An empty `target` means the default final stage, which is the llama.cpp one. That preserves the existing images exactly. - -- [ ] **Step 4: Verify the workflow parses** - -Run: `python3 -c "import sys,yaml;yaml.safe_load(open('.github/workflows/release.yml'))" 2>/dev/null || docker run --rm -v "$PWD:/w" -w /w mikefarah/yq:4 '.jobs.docker.strategy.matrix.include | length' .github/workflows/release.yml` -Expected: `3`, or no output and exit 0 from the Python check. If neither tool is available, run `gh workflow view ci-release` after pushing and confirm it is not reporting a parse error. - -- [ ] **Step 5: Confirm latest is not applied to proxy** - -Run: `grep -A3 'variant: proxy' .github/workflows/release.yml | grep latest` -Expected: `latest: false` - -- [ ] **Step 6: Commit** - -```bash -git add .github/workflows/release.yml -git commit -m "ci: Publish the proxy image variant - -Adds a build target to the matrix. Empty target keeps the existing entries on -the default final stage." -``` - ---- - -### Task 7: Document it - -**Files:** -- Modify: `README.md` at the Docker tags table (line 129 area), and a new configuration section after it - -**Interfaces:** -- Consumes: the measured image size from Task 4 Step 4. -- Produces: nothing. - -- [ ] **Step 1: Add the tag table row** - -In the `## Docker tags` table, add after the `gemma3-12b` row. Replace `` with the number from Task 4 Step 4: - -``` -| `proxy` | none (you supply) | ~ MB | -``` - -Then extend the sentence below the table: - -``` -Version-pinned tags follow the pattern `v0.1.1-gemma3-4b`, `v0.1.1-gemma3-12b`, -`v0.1.1-proxy`. The `proxy` tag is never published as `latest`. -``` - -- [ ] **Step 2: Add the usage section** - -Insert a new section immediately before `## Implemented routes`: - -```markdown -## Bring your own model server (`:proxy`) - -If you already run llama.cpp, vLLM, or anything else that speaks the OpenAI -chat-completions API, the `proxy` tag gives you the translation layer alone. It -contains no model and no inference engine. - -```bash -docker run -d -p 8090:8090 \ - -e LK_UPSTREAM=http://llama.internal:8080/v1 \ - gokhalh/localaik:proxy -``` - -| Env var | Default | Description | -| --- | --- | --- | -| `LK_UPSTREAM` | `http://127.0.0.1:8080/v1` | Base URL of your model server | -| `LK_UPSTREAM_AUTH_HEADER` | unset | A full header line sent to your server, for example `Authorization: Bearer abc123` | -| `PORT` | `8090` | Port localaik listens on | - -`LK_UPSTREAM_AUTH_HEADER` is sent only to your upstream. Credentials that -clients send to localaik are still discarded and never forwarded. - -`/health` returns 503 until your upstream answers, so existing healthchecks and -CI wait loops work unchanged. - -### Security - -`:proxy` has a different risk profile from the model-bundled tags. Those keep -llama.cpp bound to localhost inside the container, so the only thing reachable -is a disposable local model. `:proxy` forwards into infrastructure you care -about, and localaik does not authenticate its callers by design. - -**Anyone who can reach port 8090 can use your model server without -credentials.** Bind to localhost and do not publish the port on a shared -network. localaik is a testing tool, not a gateway. -``` - -- [ ] **Step 3: Update the tested-SDKs and limitations text if it claims self-containment** - -Run: `grep -n 'one container\|self-contained\|no internet\|No API key' README.md` - -For each hit, confirm the claim is still true or scope it to the model-bundled tags. The `## Motivation` paragraph says "a single Docker container that speaks all three protocols backed by a local model", which remains accurate for the default tags; add ", or the `proxy` tag if you already run your own model server." to the end of that sentence. - -- [ ] **Step 4: Check the rendered result** - -Run: `grep -n '^## ' README.md` -Expected: the new `## Bring your own model server (:proxy)` heading appears between `## Tuning` and `## Implemented routes`. - -- [ ] **Step 5: Commit** - -```bash -git add README.md -git commit -m "docs: Document the proxy tag and its security profile" -``` - ---- - -### Task 8: Full verification and review - -**Files:** none modified. - -- [ ] **Step 1: Run everything** - -```bash -make lint -go test -count=1 ./cmd/... ./internal/... ./integration/ -go test -count=1 -tags=docker_integration ./integration -run ProxyImage -``` - -Expected: all pass. - -- [ ] **Step 2: Confirm the existing images are untouched** - -```bash -git diff main --stat -- Dockerfile -docker build -t localaik:regression-check . -docker run -d --name regression-check -p 18096:8090 localaik:regression-check -sleep 90 -curl -s http://127.0.0.1:18096/health -docker rm -f regression-check -``` - -Expected: `{"status":"ok"}`. The only `Dockerfile` change should be the inserted stage. - -- [ ] **Step 3: Confirm no secret is logged** - -```bash -docker run --rm -e LK_UPSTREAM_AUTH_HEADER="Authorization: Bearer super-secret" \ - -e LK_UPSTREAM=http://127.0.0.1:9/v1 localaik:proxy-check 2>&1 | head -20 | grep -c super-secret -``` - -Expected: `0`. The container will fail to reach its upstream, which is fine; the check is that the credential never appears in output. - -- [ ] **Step 4: Run the three required reviews** - -Per the repo's PR workflow, before opening the PR: - -1. The `everything-claude-code:code-reviewer` agent on the pending diff. -2. The `superpowers:requesting-code-review` skill against `main..HEAD`. -3. The `codex:review` command, falling back to `codex:codex-rescue` with a saved diff. - -Fix anything actionable and re-review. Do not open the PR while findings are outstanding. - -- [ ] **Step 5: Open the PR** - -Use the `newpr` skill to generate the description. - ---- - -## Self-Review - -**Spec coverage:** - -| Spec requirement | Task | -| --- | --- | -| Third Dockerfile stage, alpine plus poppler-utils | 4 | -| llama.cpp stage stays last | 4 (Step 2), 8 (Step 2) | -| poppler required, not optional | 4 (Step 5) | -| `LK_UPSTREAM` env fallback | 3 | -| `LK_UPSTREAM_AUTH_HEADER`, full header line | 1, 3 | -| Flag over env over default | 3 | -| Credential injected in the transport, one place | 1 | -| Client credentials still stripped | 2 | -| Both properties tested together | 2 (Step 1) | -| `/health` unchanged against remote upstream | 4 (Step 6), 5 | -| `HEALTHCHECK` start-period reduced to 5s | 4 (Step 1) | -| Matrix entry, `proxy` and `vX.Y.Z-proxy` | 6 | -| Never `latest` | 6 (Steps 2, 5) | -| README security warning | 7 (Step 2) | -| Measure final image size | 4 (Step 4), 7 (Step 1) | -| Verify alpine pdftoppm matches | 4 (Step 5), 5 | -| Never log the credential | 8 (Step 3) | -| No model download | not implemented, correctly out of scope | -| No client authentication | not implemented, correctly out of scope | - -**Placeholder scan:** `` in Task 7 Step 1 is an intentional handoff from Task 4 Step 4, which produces the number. No other placeholders. - -**Type consistency:** `newUpstreamAuthTransport(base http.RoundTripper, header string) http.RoundTripper` is defined in Task 1 Step 3 and referenced in Task 1 Steps 1 and 5 only. `resolveFlagDefault(envName, fallback string) string` is defined in Task 3 Step 3 and used in the same step. `Config.UpstreamAuthHeader` is added in Task 1 Step 5 and consumed in Tasks 2 and 3. `roundTripHandler` and `newTestServer` are pre-existing and referenced with their file locations. Consistent. diff --git a/docs/superpowers/specs/2026-08-04-proxy-only-image-design.md b/docs/superpowers/specs/2026-08-04-proxy-only-image-design.md deleted file mode 100644 index 6a00dc6..0000000 --- a/docs/superpowers/specs/2026-08-04-proxy-only-image-design.md +++ /dev/null @@ -1,177 +0,0 @@ -# Proxy-only image (`:proxy`) - -**Date:** 2026-08-04 -**Status:** Approved, not yet implemented -**Scope:** One new published image variant plus the configuration needed to use it. - -## Problem - -localaik ships one kind of image: llama.cpp, a Gemma model, and the translating -proxy, welded together. The smallest published tag is 3.16 GB compressed. Almost -all of that is model data: the weights are 2.49 GB and the vision projector -another 0.85 GB, against ~130 MB for llama.cpp and ~8 MB for the proxy itself. - -Some users already run an inference server. It may be llama.cpp on their laptop, -or a shared internal deployment. Those users want only the translation layer: -something that accepts Gemini, OpenAI and Anthropic shaped requests and forwards -them to a server they already operate. Today they must pull 3.16 GB and run a -second model they will never call. - -The Go binary already supports this. `cmd/localaik/main.go` takes `--upstream` -for any OpenAI-compatible base URL and assumes nothing about llama.cpp or -locality. What is missing is a container image that omits the inference stack, -and the configuration to authenticate against a remote upstream. - -## Non-goals - -- Authenticating localaik's own callers. It remains a test double that accepts - and ignores client credentials. -- Any model download. That is the separate `:no-model` variant, specified later. -- Any change to `gemma3-4b`, `gemma3-12b` or `latest`. Those keep working - byte-identically. - -## Design - -### The image - -`Dockerfile` gains a third stage. Stage order matters: the llama.cpp stage stays -last so that a bare `docker build .` and the existing `make docker-build` keep -producing the full image. - -``` -FROM golang:1.25-alpine AS proxy-builder # exists, unchanged -FROM alpine AS proxy # new -FROM ghcr.io/ggml-org/llama.cpp@sha256:... # exists, stays last -``` - -The `proxy` stage installs `poppler-utils` and `ca-certificates`, copies the -binary, and runs it under `tini`. No entrypoint script: with configuration read -from the environment there is nothing for a shell to decide. - -`poppler-utils` is required, not optional. `main.go` constructs -`pdf.NewExecRenderer("pdftoppm")`, and without it every PDF request fails at -render time. Dropping it would save roughly 50 MB and silently remove a -documented feature, which is a bad trade against an image that is already about -50x smaller than today's. - -Expected size: 50-90 MB. To be measured during implementation, not asserted here. - -### Configuration - -`main.go` grows environment fallbacks for its two flags, following the pattern -`PORT` already sets for `--port`. - -| Variable | Flag | Default | Purpose | -| --- | --- | --- | --- | -| `LK_UPSTREAM` | `--upstream` | `http://127.0.0.1:8080/v1` | Base URL of the model server | -| `LK_UPSTREAM_AUTH_HEADER` | none | unset | Credential sent to upstream only | -| `PORT` | `--port` | `8090` | Listen port, unchanged | - -Precedence is flag over environment over default, so the full image's entrypoint -keeps working unchanged: it passes `--upstream` explicitly. - -`LK_UPSTREAM_AUTH_HEADER` holds a complete header line, for example -`Authorization: Bearer abc123`, rather than a bare token. This covers `Bearer`, -llama.cpp's `--api-key`, and any custom scheme without the proxy needing to know -which is in use. - -### Upstream authentication - -The proxy currently sends no credential upstream, deliberately. Three separate -places enforce that: - -- `cloneHeaders` strips `Authorization`, `X-Api-Key` and `X-Goog-Api-Key` from - passthrough requests. -- The Gemini and Anthropic handlers build fresh requests carrying only - `Content-Type` and `Accept`. -- `fetchUpstreamJSON` forwards no headers at all, and documents why. - -That is correct when upstream is `127.0.0.1:8080` inside the same container. -Against a remote server that requires a key, every request would 401. - -The credential is therefore injected in the HTTP client's transport rather than -at each call site. All upstream traffic already flows through `s.client.Do`, so a -`RoundTripper` wrapper applies the header to every request and cannot be -forgotten when a sixth upstream path is added later. - -Two properties must hold simultaneously, and both are tested: - -1. Credentials the caller sent are still stripped and never reach upstream. -2. The proxy's own credential is added to every upstream request. - -### Health and readiness - -`handleHealth` already probes upstream on every call and returns 503 when it is -unreachable, so it works unchanged against a remote server. `HEALTHCHECK ---start-period` drops from 60s to 5s in the `proxy` stage, since no model loads. - -### Publishing - -`release.yml` gains a matrix entry carrying a build target. Existing entries -default to the full image. Tags follow the current scheme minus `latest`: -`proxy` and `vX.Y.Z-proxy`. - -`:proxy` must not become `latest`. Anyone pulling `latest` today gets a -self-contained container, and silently turning that into one requiring an -external server would break them. - -## Security - -`:proxy` has a materially different risk profile from every existing tag, and -the README must say so. - -In the baked images llama.cpp binds `127.0.0.1` inside the container. The only -reachable service is the proxy, and behind it a disposable local model. In -`:proxy` the container becomes a network hop into infrastructure the operator -cares about. Because localaik accepts and ignores client credentials by design, -anyone who can reach port 8090 can drive the upstream server unauthenticated. - -The mitigation is documentation, not code: bind to localhost, and do not publish -the port on a shared network. Adding client authentication is explicitly -rejected, because it would invite treating a test double as production -infrastructure. - -`LK_UPSTREAM_AUTH_HEADER` is a secret in an environment variable. That is -acceptable here: the container legitimately needs it, it travels as a request -header rather than a process argument, and it is never echoed to logs or -forwarded to callers. Implementation must not log its value, and must not enable -shell tracing anywhere it is in scope. - -## Testing - -Existing tests need no changes. They already stub upstream through an -`http.RoundTripper`, which is exactly the seam this feature uses. - -New unit tests: - -- Flag beats environment beats default, for both `--upstream` and `--port`. -- `LK_UPSTREAM_AUTH_HEADER` reaches all four upstream paths: chat completions, - `/tokenize`, models list, and the Anthropic messages route. -- Client `Authorization`, `X-Api-Key` and `X-Goog-Api-Key` are still stripped - when the proxy's own credential is configured, verified together in one test so - the two behaviours cannot silently merge. -- No credential is sent when `LK_UPSTREAM_AUTH_HEADER` is unset. - -Image test, behind the existing `docker_integration` tag: build `--target -proxy`, run it against a stub OpenAI-compatible server, and confirm a Gemini, an -OpenAI and an Anthropic request each round-trip. This is fast, since no model is -involved. - -## Verification before merge - -Two claims in this spec are estimates and must be measured: - -1. Final image size. Recorded in the README once known. -2. That `pdftoppm` from alpine's `poppler-utils` behaves the same as the - Debian-based full image for the PDF-to-PNG path. The existing PDF tests - should be run inside the built `:proxy` image, not only on the host. - -## Follow-up - -`:no-model`, a variant keeping llama.cpp but fetching the model from -`LK_MODEL_URL` at startup, is a separate change. Investigation during this design -established that the pinned llama.cpp build already provides `--model-url`, -`--mmproj-url`, `--hf-token`, `LLAMA_CACHE` and `--offline`, so that work is -mostly packaging and documentation rather than download logic. It also needs a -startup-order change so the proxy answers `/health` during a long download, which -`:proxy` does not require.