feat(cli): report the client environment on API requests - #101
Conversation
2cc6cac to
3531cf7
Compare
|
Per Wesley's suggestion, b9e5e41 swaps the hand-rolled detection matrix for github.com/vercel/detect-agent v1.2.0 (their Go implementation, zero transitive deps). clientenv.go went 271 -> 118 lines and its tests 407 -> 104 (now testing only our wrapper: header format, CI, sanitization, agent-beats-CI precedence). Unchanged: the X-Client-Env header format, CI detection, interactive fallback, and the sanitize-to-agent:unknown rule; detection still cannot affect behavior (any Detect error falls through). Three small semantic deltas from adopting upstream as-is: Replit now matches on REPL_ID alone (we previously gated it on no-TTY), the AI_AGENT github-copilot-cli normalization is dropped (raw sanitized value passes through), and Kiro's TTY gate uses upstream's stdout check instead of our stdin isatty. All three seem fine to inherit rather than fork over. |
|
One follow-up in 061b076 + 693791f: the header now preserves the AI_AGENT @Version suffix (the documented detect-agent convention is name@version, e.g. devin@1; our charset was stripping @ and merging version digits into the name), and agent ids are normalized to lowercase-hyphen (agent:claude-code, agent:gemini-cli). Rationale for the normalization: upstream's name strings mix underscores and hyphens with no stability guarantee, and these values become long-lived analytics dimensions on builds, so a one-line separator normalization keeps our recorded history stable across upstream renames and matches the hyphenated ids the ecosystem registries use. Still zero mapping tables to maintain. |
693791f to
f1a2ea0
Compare
Adds an X-Client-Env header alongside the existing source and version headers, reporting whether the CLI was invoked by a named agent, by CI, or interactively. Deploy attribution currently cannot distinguish an agent-driven deploy from a human one. The agent matrix mirrors the vercel/detect-agent registry (agents.json schema version 1) verbatim, including its identifiers and its array order, so our attribution is comparable with anyone reading the same list. Two narrowing divergences: replit is gated on the absence of a TTY because REPL_ID is set for humans in the Replit editor too, applying the registry's own stated reasoning for kiro; and AI_AGENT is evaluated first so an operator wrapping a known agent can name their own harness. Detection reads the environment only and never influences behaviour, output defaults or confirmation prompts. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Replaces the hand-rolled env-var matrix with the maintained github.com/vercel/detect-agent module (v1.2.0, zero deps) per review. Header format, CI detection, and sanitization semantics are unchanged; tests pruned to the wrapper's own behavior. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The detect-agent AI_AGENT convention is name@version (devin@1); the header charset previously stripped @ and merged version digits into the name. The backend attribution sanitizer already preserves @. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
detect-agent's registry names mix underscores and hyphens (claude_code, cursor-cli) with no stability contract, and these values become analytics dimensions. Normalizing separators keeps recorded history stable across upstream churn and matches the hyphenated ids the agent registries converge on (claude-code, gemini-cli). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ErrAgentNotFound stays the silent no-agent path; any other Detect error is surfaced as an slog warning per review instead of being swallowed, and still falls through so detection cannot affect behavior. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
84582d4 to
012fdf1
Compare
Stack position: 2 of 2. Stacked on #102 — this diff is against it, not
main.Problem
Every request already carries
X-Source: cliandX-CLI-Version, but nothing says what invoked the CLI. An agent-driven deploy and an engineer at a terminal are indistinguishable, so deploy-side agent metrics can only split on the calling principal: currently around 32% of deploying principals are automation against around 70% of build volume, and that gap is one undifferentiated blob of service accounts belonging to CI, to coding agents, and to internal tooling.Change
internal/clientenvclassifies the invoking environment into one of three shapes, emitted asX-Client-Env:agent:<name>for a detected agentcifor a CI system (CI, plus 13 vendor variables)interactiveotherwiseWired onto the JSON API client, the multipart
RunApppath and both device-auth endpoints. Computed once per process.The agent matrix mirrors vercel/detect-agent
agents.json(schema version 1): 20 agents, its identifiers verbatim (including its mixedclaude_codeandcursor-clistyles) and its array order, since order decides which of two simultaneous markers wins.TestAgentDetectorsMatchRegistryOrderpins both, so a drifting sync fails instead of silently reclassifying traffic. The registry promotesAI_AGENTas the cross-vendor self-declaration variable, which we honour.Two deliberate narrowings, both flagged in the code:
replitis gated on the absence of a TTY.REPL_IDis set for every process on Replit including a human in the editor, so bareREPL_IDevidences the host, not an agent. This is the registry's own stated reasoning forkiro("set by both the IDE terminal and the CLI agent, so gate onno_ttyto avoid misdetecting a human at the integrated terminal") applied to the variable with the same problem. Following the registry literally here would have put humans in the agent bucket, which is the exact overcount this header exists to avoid.AI_AGENTis evaluated first, not as a fallback, so an operator wrapping a known agent can name their own harness and have that win.Detection never influences behaviour: no output default, no prompt suppression, nothing branches on it. That is why the non-TTY deploy guard from #87 keys off stdin rather than agent detection.
Detecttakes its environment, filesystem and TTY probes as parameters, so the matrix is tested without touching the real environment.AI_AGENTvalues are caller-supplied, so they are lowercased, restricted to[a-z0-9._-]and capped at 64 characters. A value that sanitizes to nothing becomesagent:unknownrather than degrading tointeractive, keeping a declared-but-unparseable agent in the agent bucket.Not in scope
The header is only stored once dashboard-backend #4132 lands; until then it is accepted and discarded.
X-prefixes are discouraged for new headers by RFC 6648, but this joins an existingX-Source/X-CLI-Version/X-Client-OSfamily and consistency won.Test
go test ./internal/clientenv/... ./internal/api/... ./internal/auth/...green: 58 detection cases (one per registry variable, both TTY branches for the two host-gated agents, the three ordering rules, theAI_AGENTsanitizer and the CI matrix), plus header-presence tests on the API client and both OAuth endpoints.gofmtclean.golangci-lint run ./internal/clientenv/...reports 0 issues; on the other touched packages the single finding (nilerrininternal/auth/docker.go) is pre-existing, verified by re-running against a stash. Repo-widemake lintandgo build ./...fail on vendoredcortex/server/node_modulesAWS CDK templates, unrelated and also failing onmain.🤖 Generated with Claude Code