feat(logs): track normalized API client identity without leaking raw User-Agent

[LIghtJUNction]

Upstream source

• QuantumNous/new-api PR #7488: feat(logs): 显示请求客户端并支持筛选 QuantumNous/new-api#7488
• Upstream head: e289454a2310d20ffc48c4f262b9aba02a0c031f
• Created 2026-09-20 17:34 UTC; currently open and not merged.

The upstream PR adds request-client identification from the original User-Agent, persists a snapshot into consume/error/task logs, adds client_family filtering/statistics, and exposes client details in the usage-log UI. It changes 32 files (+877/-39), including relay/plugin/WebSocket request capture, log schema/querying, async-task snapshots and frontend filters.

Current LMM gap

Checked against current main after 5431b981fac24dd7aa56dc19c1ee76a764dfe578.

• apps/api-go/model/log.go has no normalized client identity fields; GetAllLogs(...) / user-log/stat queries have no client_family filter.
• Repository search finds no ClientIdentity implementation or equivalent request-scoped normalized API-client snapshot.
• Existing user_agent handling is for login/session/audit metadata, not per-API-request usage attribution.
• LMM already has richer usage-log UX, source/attribution analytics and custom relay/task paths, so mechanically applying the upstream 32-file patch would risk coupling unrelated product analytics and privacy behavior.

This is useful for LMM because it can distinguish Codex / Claude Code / Pi / OpenCode / SDK / browser-style callers in request diagnostics and help validate real-client onboarding, but it should remain observability only: UA is spoofable and must never become an auth/trust/routing signal.

Proposed LMM adaptation

Implement this as a small, LMM-native observability slice after upstream #7488 finishes review:

1. Capture the original inbound User-Agent before any relay/header rewriting on HTTP relay and Responses WebSocket paths.
2. Normalize only known explicit prefixes to a stable {family, variant, version, confidence} structure. Unknown callers remain unknown; do not infer identity from model names or arbitrary substrings.
3. Persist normalized identity with consume/error logs and async-task terminal logs so task settlement keeps the submission-time client identity.
4. Add server-side client_family filtering before pagination/counting and use the same condition for usage statistics.
5. Add a compact usage-log filter/detail UI using LMM's existing components and seven-language i18n; do not copy upstream branding/gradients/product-specific visual treatment.
6. Treat raw UA as potentially sensitive/untrusted metadata. Prefer not to persist the full raw string by default; if retained for diagnostics, sanitize control characters, bound length, keep it out of normal user-facing exports/source-attribution analytics, and document retention/visibility explicitly.
7. Do not let client identity affect pricing, groups, model price locks, /fast, OAuth policy, routing weights, trust level, rate limits or administrator-assistant authorization.

Acceptance criteria

• Codex CLI/desktop, Claude Code, Pi, OpenCode and common OpenAI SDK test UAs map deterministically; malformed/control-character input cannot create a false known identity.
• Unknown UA remains unknown and cannot be promoted by sanitization side effects.
• HTTP relay and Responses WebSocket capture the pre-rewrite identity; two sequential WS turns cannot leak identity state between requests.
• Async tasks retain the identity from submission through terminal logging without changing billing/refund/CAS behavior.
• client_family filtering is applied before count/pagination and produces consistent admin/user statistics while preserving existing user-scope isolation.
• Historical rows without identity continue to load and appear as unrecorded/unknown; migration is idempotent on SQLite/MySQL/PostgreSQL/ClickHouse paths used by LMM.
• Raw UA, if stored at all, is bounded/sanitized and does not enter public/source-attribution exports unintentionally.
• Regression tests cover known/unknown/malformed UA, old rows, filters, pagination/stat consistency, task settlement, and WebSocket request isolation.

Why issue instead of immediate backport

Upstream #7488 is still open with automated review in progress and introduces a database/logging/privacy surface across 32 files. The capability is applicable, but LMM should adopt the normalized observability contract rather than copy the full patch before upstream review settles or accidentally mix UA data into LMM's existing attribution/custom business logic.

[LIghtJUNction]

Upstream #7488 received a new follow-up commit after this issue was opened: QuantumNous/new-api@360d634 (fix(logs): recognize Codex TUI user agents).

The only behavior change is a new explicit prefix mapping:

• codex-tui/ → family codex, variant tui, display name Codex TUI
• regression sample: codex-tui/0.155.1 (Mac OS 26.5.2; arm64) Apple_Terminal/470.2 (codex-tui; 0.155.1) → codex:tui

Please include this when implementing #425. Keep it as a distinct tui variant rather than folding it into generic cli, because upstream now treats those prefixes separately. This does not change the earlier observability-only/privacy constraints: the identity remains untrusted metadata and must not affect pricing, groups, /fast, routing, trust, OAuth, rate limits, or admin-assistant authorization.

Suggested extra regression case: verify codex-tui/<version> maps deterministically while arbitrary UAs merely containing the substring codex-tui do not become a known client unless they match the explicit prefix rule.

[LIghtJUNction]

Upstream #7488 has two new commits since the last tracked head 360d634f:

• QuantumNous/new-api@ebf0e8f — feat(logs): generalize client recognition and synchronize portraits
• QuantumNous/new-api@425926f — fix(i18n): use consistent client variant terminology

The second commit is only the Chinese Variant translation (形态 → 变体) and does not need a separate LMM migration. The first commit materially changes the recognizer and should update #425's implementation contract.

Useful upstream changes to carry into an LMM-native implementation:

1. Use one registry for recognition and filter-family options so the two cannot drift.
2. Distinguish application, sdk, and transport identity kinds. Application identity should outrank an SDK/runtime token appearing later in the same UA.
3. Correct Go-http-client/*: it is generic Go transport, not proof that the caller is NewAPI. Historical newapi rows may stay queryable as legacy data, but new generic Go requests must not be labelled NewAPI/LMM.
4. Add exact/anchored handling for useful new real clients such as Claude diagnostic UA, MiMo Code, WorkBuddy/CodeBuddy, Tender, Hertz, AsyncOpenAI/Python and common Python transports. Keep explicit matching bounded rather than substring-based.
5. Preserve the upstream hardening idea: sanitize control/format-invalid characters, cap UA length, and refuse to promote sanitized/truncated/malformed input into a recognized identity. Also avoid matching product names that only occur inside UA comments.
6. A generic leading Product/Version may be shown as an unverified display hint, but it must remain family=unknown / untrusted. This refines the original feat(logs): track normalized API client identity without leaking raw User-Agent #425 wording: unknown products may be presented, but they must never become a known family merely because their syntax looks valid.

Do not port the upstream portrait/easter-egg assets (changzheng, greyfield, taffyOfficial) or other product-specific visual treatment as part of this issue. LMM should keep its existing usage-log design and only add the compact diagnostics/filter surface it needs.

The original security/product boundary remains unchanged: client identity is observability metadata only and must not influence pricing, groups, model price locks, /fast, OAuth policy, routing weights, trust level, rate limits, billing decisions, or administrator-assistant authorization.

Suggested additional regressions:

• Go-http-client/* => transport/Go, never NewAPI/LMM.
• a host application UA containing an OpenAI SDK/runtime token still resolves to the application; an unfamiliar leading product remains unknown with confidence=unverified.
• control chars, Unicode format chars, malformed parentheses, overlong/truncated UAs, and product names only inside comments cannot produce a recognized family.
• recognized filter choices are generated from the same registry and contain no duplicate families.

Current api.lmm.best main still has no ClientIdentity implementation or equivalent PR, so #425 remains the single tracking item. Upstream #7488 is still open; its only current review thread was the terminology nit above and is resolved. I would still avoid copying the full upstream patch because it mixes DB/logging/UI/privacy changes and portrait assets; implement the normalized observability slice described here when #425 is taken.