Skip to content

Add bounded CLI and Hub usage telemetry for 0.8.1 - #188

Merged
theDakshJaitly merged 3 commits into
codex/0.8.1from
codex/0.8.1-telemetry
Sep 9, 2026
Merged

Add bounded CLI and Hub usage telemetry for 0.8.1#188
theDakshJaitly merged 3 commits into
codex/0.8.1from
codex/0.8.1-telemetry

Conversation

@theDakshJaitly

@theDakshJaitly theDakshJaitly commented Sep 9, 2026

Copy link
Copy Markdown
Collaborator

What

Replace attempt-only CLI telemetry with namespaced starts/completions and explicit Hub page, action and job outcomes. CLI and Hub share the approved random installation UUID. Optional context includes the existing random scaffold UUID and a fixed list of configured AI tools; strict catalogs exclude content, paths, queries, project names/remotes, member/artifact IDs and contact details.

Project context is read once per ordinary CLI invocation or Hub lifecycle, without creating identity. Setup/init reread at completion for newly saved configuration. Graph maintenance/status honors its exact local/parent root selection, and Hub capture binds its exact project root. Shared scaffolds are a team-use estimate, and configured tools describe setup rather than the actual invoking agent.

Delivery uses a bounded local outbox and cancellable HTTP/DNS without the PostHog SDK. Opt-out survives concurrent preference writes, and inspection remains read-only. mex feedback now opens the Hub's existing form; the Home card uses “Help shape MEX.”

Why

Measure feature adoption, outcomes and returning installations without turning agent command volume or idle Hub polling into engagement. Keep usage analytics separate from voluntary contact collection, and bound the effect of offline telemetry on command latency.

This draft targets codex/0.8.1, based on graph-performance merge d64f171. It does not target or merge main.

Type of change

  • Bug fix
  • New feature
  • Refactor
  • Docs
  • CI/Tooling

How to test

  • Follow-up workspace typecheck and full build pass; root declaration bytes remain identical. Initial revision 9cbfab8 packed-install smoke covered fresh setup, the production Hub, official skills and temporary real Git repositories; package layout/dependencies are unchanged by this follow-up.
  • Initial telemetry revision 9cbfab8: full root regression 222 files, 3,500 passed, one skipped; all CI passed, including Node22/24, macOS/Windows and the pinned release-performance gate.
  • Project-context follow-up: 353 focused/regression tests across 15 files, plus workspace typecheck/build, pass. New reader/schema/capture tests are also included in macOS/Windows CI. The real DNS cancellation test waits for an observed loopback query before timing, preserving the 25 ms production grace and 150 ms assertion. All required CI checks pass on a37277b (run 34378404972): Node 22/24, Windows/macOS portability and packed installs, Hub browser, and the release-performance gate.
  • Project-context CLI benchmark: 20 samples after five warmups in each of 16 groups, comparing the preserved 9cbfab8 build with the final metadata build. Metadata lookup p95 is 0.210–0.659 ms; capture-call p95 is at most 2.466 ms, and the longest observed flush is 29.130 ms. Exact existing UUID/tools, private-field rejection, canonical bytes, outputs/exit codes, queued outcomes and socket cleanup all pass. Whole-process differences were noisy (successful healthy command: paired median +7.895 ms, p95 +86.162 ms; disabled failure control p95 +89.533 ms), so these results do not establish zero overhead or a speedup.
  • Initial 9cbfab8 frontend verification: 434 Hub web tests and two Playwright checks passed. No frontend changes in the follow-up. Expanded standalone benchmark harness: 7 tests pass.
  • Initial telemetry built-CLI benchmark (9cbfab8, before project context): 16 baseline/candidate × success/failure × transport groups, with five warmups and 20 samples each, plus pristine initialization and capture/flush probes. Healthy loopback delivery adds 3–8 ms paired median; hanging delivery adds 23–27 ms. The old hanging-success path was approximately 645 ms slower. Processes exit naturally with expected output/status and no retained sockets.
  • New telemetry persistence/transport coverage is included in macOS and Windows CI. Platform CI remains a release gate.

See the validation record, initial measurements, and project-context measurements. Loopback timings exclude internet/TLS latency and are local characterization, not portable performance budgets. Delivery remains best effort; short-only usage may defer queued events.

Checklist

  • Relevant local tests and all required CI checks pass on a37277b
  • No breaking changes: root public declarations are byte-identical to the base
  • Follow-up tested with isolated project fixtures; initial revision also passed real Git and packed-install checks

Code-graph changes

No graph implementation, schema, identity or protocol changes. The real checkout's graph database is unchanged.

  • This PR targets main
  • A linked issue agrees on the bounded extractor/resolver scope
  • The change follows the frozen LanguageExtractor or FrameworkResolver interface
  • A focused fixture and assertions for the expected node/edge shape are included
  • Any new grammar WASM, extension mapping, extractor, or resolver is registered
  • No graph identity, reconciliation, schema, or drift-semantics changes are included, or a core / discuss-first issue is linked above

@theDakshJaitly
theDakshJaitly marked this pull request as ready for review September 9, 2026 17:00
Copilot AI lite review requested due to automatic review settings September 9, 2026 17:00

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@theDakshJaitly
theDakshJaitly merged commit 0ad565d into codex/0.8.1 Sep 9, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants