Define low-volume telemetry event schemas - #287
Open
gmemuriuki wants to merge 57 commits into
Open
gmemuriuki wants to merge 57 commits into
gmemuriuki wants to merge 57 commits into
Conversation
Co-authored-by: Codex <codex@openai.com>
Make room for new telemetry pieces without changing behavior.
Use UUIDs so malformed identifiers are rejected when data is read.
Validate stored dates and versions and keep timestamps to whole seconds.
Add shared row metadata and reject schema drift in version 1.
Classify recognized but invalid rows separately from malformed input.
Give each scoped telemetry identifier a fixed prefix and canonical 128-bit hexadecimal representation. Parse the same representation strictly and use it for JSON serialization. Keep identifier domains separate at the type level, and check the published examples so code and the telemetry contract cannot drift.
Separate identifier windows from domain-scoped dimensions so callers cannot exchange them or mix identifiers from different domains. Length-prefix each input field and pin the HMAC domain constants in the recording contract. This keeps component boundaries unambiguous and makes accidental identifier rotation visible in tests.
Create each 256-bit identity key from the operating system random source and return failures to the caller. Keep generation independently testable so the complete buffer and error path are covered without relying on probabilistic assertions.
Keep identity material in strict, versioned TOML before adding file persistence. Share lowercase hexadecimal encoding with scoped identifiers so keys and identifiers follow the same rules. Reject unknown fields, future versions, malformed keys, and noncanonical anchor dates. Pin storage-limit JSON field order in its contract test.
Generate fresh state from one UTC day while leaving the return-cohort anchor absent until a session is observed. Use a deterministic constructor to pin the exact initial TOML in tests. Keep identity resets monotonic with the latest-opened-day high-water mark and document that rule consistently across the RFD.
Represent cohort days as a bounded D0-D30 value and advance the private return-cohort anchor as sessions are observed. Persist state before recording D0 so partial failures keep identifiers stable and undercount returns.
Roll identifier windows over on the first observation at day 30 or later, while keeping the identity key stable. Select identifier and return-cohort anchors before mutating private state, so storage can persist one coherent session transition.
Rotate the identity key, advance the window anchor, and clear the return cohort as one state replacement. Generate the new key before mutation so failures leave the previous state intact. Document reset's clock-clamping rules and future state obligations.
Make room for focused state submodules while preserving the existing implementation unchanged. The telemetry parent continues to load state through the same private module boundary.
Keep the serialized private-state model in the module root and move identifier reset, window, and cohort transitions into lifecycle.rs. Leave each responsibility's tests beside its implementation and record the boundary in the module structure guide.
Prepare the schema for event-family modules without changing its behavior. Adjust the contract fixture path for the file's new directory depth.
Use closed enums for hook agents, platforms, and session starts. Pin serialized names to the version-one contract and reject unknown values.
Define the agent, platform, and start values recorded for a session. Read session-start rows through the versioned classifier and test them against the JSONL contract. Document that platform values describe the binary's compilation target rather than its physical host.
Keep the seven-agent configuration list separate from the smaller hook agent list. Convert every project agent explicitly so future agent support requires a deliberate telemetry update. Pin each version-one serialized name, including the opencode spelling.
Add the version-one row for one entry in the daily agent configuration snapshot. Keep fixed fields inside the constructor and test every value without wiring storage or production recording yet.
Recognize version-one agent configuration rows through the shared envelope classifier. Keep future versions separate from invalid rows and pin serialization to the JSONL contract example.
Link hook agents to the broader supported-agent vocabulary so rows that refer to the same agent cannot drift apart. Group per-agent configuration values at construction sites, making the configured flag explicit and preparing for the daily batch builder.
Add the fixed triggers, outcomes, and unnamed-package reasons used by resolution summaries. Keep reason counters private, increment them through checked operations, and align dropped resolution batches with storage-limit reporting.
Build resolution summary rows from their reason counters so the stored unnamed-package total cannot drift from its breakdown. Apply the same validation when reading JSON, and reject overflow or missing fields before invalid data reaches analysis.
Teach the shared JSONL reader to recognize version one resolution summaries and serialize them through TelemetryRow. Keep future versions distinct from invalid supported rows, including summaries with unknown fields or inconsistent package counts.
Add the closed version one package ecosystem and extension match values before introducing the package resolution row. Pin their JSON names and reject unknown values so later schema changes remain explicit.
A session start's day is derived from its completion timestamp. Reject stored rows where those values disagree instead of admitting inconsistent data to typed readers. Keep file membership checks with the archive reader, which has the daily file context needed for that separate invariant.
Telemetry may name a package only when its resolved coordinate is safe to record. The schema now rejects unsafe names and versions. Document the fixed version 1 grammar and keep package-specific code in its own module.
Package subjects now take their ecosystem, resolved name, and exact version directly from a validated coordinate. Keep framing inside the identity module so producers cannot silently omit or reorder fields. The same writer also defines counted sequences for future structured dimensions.
Add the versioned row for eligible public packages, using validated coordinates and domain-specific package subjects. Pin the serialized contract example and reject future versions, missing fields, and unknown fields before classifier wiring is added.
Route version one package resolution records through the shared schema classifier and serializer. Keep future versions distinguishable from invalid known rows, and verify invalid public coordinates cannot enter supported output.
Give structured identity values one canonical way to encode variant labels and nested sequences. This keeps safe resolution paths aligned with the frozen telemetry contract.
Add the versioned plugin and skill vocabulary used by telemetry. Validate public names against the contract and preserve their spelling. Share name checks and contract-name tests across schema modules.
Use one macro for the traits shared by validated public names. Keep each validator and error type explicit. Validation still runs when names are read from telemetry JSON.
Package resolution rows now derive package_subject from the validated coordinate and private identity state instead of accepting an unrelated identifier. Separate identifier-window and return-cohort scopes make the anchor choice part of each identity domain. Seal the domain registry and pin its anchor mapping in the recorded-data contract.
Represent public plugins and skills as validated type, source, and name coordinates. Reject incomplete, expanded, or invalid wire forms before they can enter telemetry rows.
Add strict recursive nodes for public extension resolution evidence. Reuse validated coordinates and test the published JSON as the contract.
Wrap extension resolution evidence in a validated path type before it is used by an event row. Reject empty paths and enforce the contract's depth, leaf-count, and encoded-size limits during construction and JSON deserialization. Document the exact boundaries and cover recursive all/any paths.
Encode public targets and their validated resolution paths through the shared identity framing rules. Reuse package coordinate encoding so a package node cannot drift from package subjects. Pin every path variant and opaque reason to the wire contract, including an independently checked HMAC vector. Share the fixed identity state fixture between subject tests.
Derive the extension subject from the row's validated target and path. This prevents callers from pairing recorded evidence with an identifier derived from different inputs.
Classify version-one extension resolution rows and preserve their contract shape when reading and writing JSONL. Exercise malformed targets, fields, and paths, including a nested path that round-trips through the complete classifier.
Build each agent configuration subject from the row's agent and active identifier window. Callers can no longer pair a configuration observation with a subject derived for another agent. Pin the wire label and HMAC result so serialized names and derived identities cannot drift independently.
Represent raw vendor session identifiers as a non-serializable input and encode session dimensions from the agent and vendor value in contract order. Use the production retention dimension in state tests so both identity domains are pinned by independently verified vectors.
Keep each completed session transition as a single-use value, then validate both selected anchors before exposing its identity scopes. This prevents stale lifecycle data from being combined with newer state. Record that the persistence layer must provide a successful-write token before binding, so the eventual persist-before-derive order can be enforced by the API.
Keep the agent and its optional scoped session identifier together so callers cannot associate an identifier with a different agent. Agents without vendor session identifiers remain explicitly unidentified without exposing raw vendor values.
Capture the completion timestamp when observing a session and carry it through the bound state transition. Session rows now derive their day, cohort position, and scoped identifiers from that single observation. This removes the post-persistence mismatch failure that could leave a new return cohort without its D0 row.
Add the built-in and public plugin command vocabulary, including a normalized remove operation for cargo agents use --remove. Tie built-in telemetry to the parsed CLI with an exhaustive mapping, and generate shared public-name validation errors from one macro.
Exercise a session completed at the final second of a UTC day. Keep the row date, completion timestamp, and stored return-cohort anchor aligned so a boundary regression cannot silently lose the D0 record.
Use each validated command coordinate as the complete input to its stable telemetry identifier. Tests pin built-in and plugin field order, an independently checked digest, and separation between coordinates.
Add a window-only observation that advances identifier state without touching return cohorts. Bind timestamps, days, and identity scopes to the selected window before rows can consume them. Session observations now reuse the same transition, keeping window selection and stale-state checks in one place.
Define typed built-in and public plugin command coordinates, validate the versioned row on read, and derive command subjects from the coordinate. Build each row from a bound recording observation so its timestamp, day, and identity window cannot disagree. Pin CLI mappings and published examples with tests.
Build daily agent configuration rows from the recording context that selected their identity window. Session-start operations can reuse the same context for their configuration snapshot. Keep raw session identity construction private and exercise identity derivation through complete rows in the tests.
Build resolution and configuration rows from one recording context so their day and scoped identifiers use the selected identity window. Keep raw identity scopes private, share the test recording fixture, and route command rows through versioned classification.
Session start, resolution summary, and command rows repeated the same raw schema and field-by-field conversion. Move that mechanical work into a private macro while keeping each validation function visible. Keep each row kind in one generated constant and apply field attributes to both serialized and raw forms. This preserves strict version, field, and kind checks without allowing the forms to drift.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Defines the version 1 schemas for low-volume telemetry rows:
session_startandagent_configurationresolution_summary,package_resolution, andextension_resolutioncommandrowsPublic coordinates are validated before they can enter a row. Scoped identifiers are derived from those coordinates and the correct identity anchor. Recording contexts bind timestamps, cohort state, and identifier windows so callers cannot combine unrelated values.
Deserialization rejects unknown fields, unsupported versions, incorrect row kinds, invalid resolution paths, and inconsistent cross-field values. The published JSONL examples are used as test fixtures.
This PR does not enable telemetry or connect these rows to production callers. It is stacked on
telemetry/recording-foundation#286.Depends on #286. Until that PR lands, review only the low-volume changes in this branch comparison (gmemuriuki/symposium@telemetry/recording-foundation...telemetry/low-volume-event-schemas).
Disclosure questions
AI disclosure.
Questions for reviewers.