Skip to content

feat(cli): report the client environment on API requests - #101

Merged
kylegani merged 6 commits into
mainfrom
feat/cli-client-env-header
Aug 24, 2026
Merged

feat(cli): report the client environment on API requests#101
kylegani merged 6 commits into
mainfrom
feat/cli-client-env-header

Conversation

@kylegani

@kylegani kylegani commented Aug 23, 2026

Copy link
Copy Markdown
Contributor

Stack position: 2 of 2. Stacked on #102 — this diff is against it, not main.

Problem

Every request already carries X-Source: cli and X-CLI-Version, but nothing says what invoked the CLI. An agent-driven deploy and an engineer at a terminal are indistinguishable, so deploy-side agent metrics can only split on the calling principal: currently around 32% of deploying principals are automation against around 70% of build volume, and that gap is one undifferentiated blob of service accounts belonging to CI, to coding agents, and to internal tooling.

Change

internal/clientenv classifies the invoking environment into one of three shapes, emitted as X-Client-Env:

  • agent:<name> for a detected agent
  • ci for a CI system (CI, plus 13 vendor variables)
  • interactive otherwise

Wired onto the JSON API client, the multipart RunApp path and both device-auth endpoints. Computed once per process.

The agent matrix mirrors vercel/detect-agent agents.json (schema version 1): 20 agents, its identifiers verbatim (including its mixed claude_code and cursor-cli styles) and its array order, since order decides which of two simultaneous markers wins. TestAgentDetectorsMatchRegistryOrder pins both, so a drifting sync fails instead of silently reclassifying traffic. The registry promotes AI_AGENT as the cross-vendor self-declaration variable, which we honour.

Two deliberate narrowings, both flagged in the code:

  • replit is gated on the absence of a TTY. REPL_ID is set for every process on Replit including a human in the editor, so bare REPL_ID evidences the host, not an agent. This is the registry's own stated reasoning for kiro ("set by both the IDE terminal and the CLI agent, so gate on no_tty to avoid misdetecting a human at the integrated terminal") applied to the variable with the same problem. Following the registry literally here would have put humans in the agent bucket, which is the exact overcount this header exists to avoid.
  • AI_AGENT is evaluated first, not as a fallback, so an operator wrapping a known agent can name their own harness and have that win.

Detection never influences behaviour: no output default, no prompt suppression, nothing branches on it. That is why the non-TTY deploy guard from #87 keys off stdin rather than agent detection. Detect takes its environment, filesystem and TTY probes as parameters, so the matrix is tested without touching the real environment.

AI_AGENT values are caller-supplied, so they are lowercased, restricted to [a-z0-9._-] and capped at 64 characters. A value that sanitizes to nothing becomes agent:unknown rather than degrading to interactive, keeping a declared-but-unparseable agent in the agent bucket.

Not in scope

The header is only stored once dashboard-backend #4132 lands; until then it is accepted and discarded. X- prefixes are discouraged for new headers by RFC 6648, but this joins an existing X-Source / X-CLI-Version / X-Client-OS family and consistency won.

Test

go test ./internal/clientenv/... ./internal/api/... ./internal/auth/... green: 58 detection cases (one per registry variable, both TTY branches for the two host-gated agents, the three ordering rules, the AI_AGENT sanitizer and the CI matrix), plus header-presence tests on the API client and both OAuth endpoints.

gofmt clean. golangci-lint run ./internal/clientenv/... reports 0 issues; on the other touched packages the single finding (nilerr in internal/auth/docker.go) is pre-existing, verified by re-running against a stash. Repo-wide make lint and go build ./... fail on vendored cortex/server/node_modules AWS CDK templates, unrelated and also failing on main.

🤖 Generated with Claude Code

@kylegani
kylegani force-pushed the feat/cli-client-env-header branch from 2cc6cac to 3531cf7 Compare August 24, 2026 08:46
@kylegani
kylegani changed the base branch from main to feat/cli-do-not-track August 24, 2026 08:47
@kylegani

Copy link
Copy Markdown
Contributor Author

Per Wesley's suggestion, b9e5e41 swaps the hand-rolled detection matrix for github.com/vercel/detect-agent v1.2.0 (their Go implementation, zero transitive deps). clientenv.go went 271 -> 118 lines and its tests 407 -> 104 (now testing only our wrapper: header format, CI, sanitization, agent-beats-CI precedence).

Unchanged: the X-Client-Env header format, CI detection, interactive fallback, and the sanitize-to-agent:unknown rule; detection still cannot affect behavior (any Detect error falls through).

Three small semantic deltas from adopting upstream as-is: Replit now matches on REPL_ID alone (we previously gated it on no-TTY), the AI_AGENT github-copilot-cli normalization is dropped (raw sanitized value passes through), and Kiro's TTY gate uses upstream's stdout check instead of our stdin isatty. All three seem fine to inherit rather than fork over.

@kylegani

Copy link
Copy Markdown
Contributor Author

One follow-up in 061b076 + 693791f: the header now preserves the AI_AGENT @Version suffix (the documented detect-agent convention is name@version, e.g. devin@1; our charset was stripping @ and merging version digits into the name), and agent ids are normalized to lowercase-hyphen (agent:claude-code, agent:gemini-cli). Rationale for the normalization: upstream's name strings mix underscores and hyphens with no stability guarantee, and these values become long-lived analytics dimensions on builds, so a one-line separator normalization keeps our recorded history stable across upstream renames and matches the hyphenated ids the ecosystem registries use. Still zero mapping tables to maintain.

@kylegani
kylegani force-pushed the feat/cli-client-env-header branch from 693791f to f1a2ea0 Compare August 24, 2026 11:28
@kylegani
kylegani requested a review from wesrobin August 24, 2026 11:51
Comment thread internal/clientenv/clientenv.go Outdated

@wesrobin wesrobin left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Base automatically changed from feat/cli-do-not-track to main August 24, 2026 13:11
kylegani and others added 6 commits August 24, 2026 15:11
Adds an X-Client-Env header alongside the existing source and version
headers, reporting whether the CLI was invoked by a named agent, by CI,
or interactively. Deploy attribution currently cannot distinguish an
agent-driven deploy from a human one.

The agent matrix mirrors the vercel/detect-agent registry (agents.json
schema version 1) verbatim, including its identifiers and its array
order, so our attribution is comparable with anyone reading the same
list. Two narrowing divergences: replit is gated on the absence of a TTY
because REPL_ID is set for humans in the Replit editor too, applying the
registry's own stated reasoning for kiro; and AI_AGENT is evaluated first
so an operator wrapping a known agent can name their own harness.

Detection reads the environment only and never influences behaviour,
output defaults or confirmation prompts.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Replaces the hand-rolled env-var matrix with the maintained
github.com/vercel/detect-agent module (v1.2.0, zero deps) per review.
Header format, CI detection, and sanitization semantics are unchanged;
tests pruned to the wrapper's own behavior.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The detect-agent AI_AGENT convention is name@version (devin@1); the
header charset previously stripped @ and merged version digits into the
name. The backend attribution sanitizer already preserves @.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
detect-agent's registry names mix underscores and hyphens (claude_code,
cursor-cli) with no stability contract, and these values become
analytics dimensions. Normalizing separators keeps recorded history
stable across upstream churn and matches the hyphenated ids the agent
registries converge on (claude-code, gemini-cli).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ErrAgentNotFound stays the silent no-agent path; any other Detect
error is surfaced as an slog warning per review instead of being
swallowed, and still falls through so detection cannot affect
behavior.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@kylegani
kylegani force-pushed the feat/cli-client-env-header branch from 84582d4 to 012fdf1 Compare August 24, 2026 13:11
@kylegani
kylegani merged commit f825f72 into main Aug 24, 2026
4 checks passed
@kylegani
kylegani deleted the feat/cli-client-env-header branch August 24, 2026 13:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants