docs(adr): propose ADR 0071 harness snapshot for agent traces - #5505
Closed
HofniGartner wants to merge 1 commit into
Closed
docs(adr): propose ADR 0071 harness snapshot for agent traces#5505HofniGartner wants to merge 1 commit into
HofniGartner wants to merge 1 commit into
Conversation
Record Level-1 harness-snapshot.json and root-span forge pointers as the run-time join contract; implementation follows in a separate issue. Signed-off-by: Hofni Gartner <hgartner@redhat.com> Co-authored-by: Cursor <cursoragent@cursor.com>
|
Thank you for your interest in contributing to fullsend, @HofniGartner. This project uses a vouch system for first-time contributors. Before submitting a pull request, you need to be vouched by a maintainer. To get vouched:
See CONTRIBUTING.md for details. |
E2E tests did not runE2E tests run automatically for org/repo members and collaborators on pull requests. For other contributors, a maintainer must add the See E2E testing guide for details. |
PR Summary by QodoPropose ADR 0071: harness snapshot + forge pointer join contract for traces
AI Description
Diagram
High-Level Assessment
Files changed (2)
|
Code Review by Qodo
1. ADR 0071 missing ## Options
|
![qodo-code-review[bot]](https://avatars.githubusercontent.com/in/484649?s=60&v=4){kind=small}
Comment on lines
+16
to
+19
| - **`harness-snapshot.json`** — proposed in | ||
| [ADR 0071](../../ADRs/0071-forge-pointer-correlation-for-agent-traces.md) | ||
| (run-start harness fingerprint and forge/CI pointers). Implementation follows | ||
| ADR acceptance; until then only `run-telemetry.jsonl` is guaranteed on disk. |
There was a problem hiding this comment.
2. Missing > planned: callout 📜 Skill insight ≡ Correctness
The new text describes a not-yet-implemented feature (harness-snapshot.json) without the required > **Planned:** callout format and issue link. This can mislead readers about current functionality and removes the required tracking reference.
Agent Prompt
## Issue description
A planned feature is documented without the required `> **Planned:**` blockquote callout and without linking to the relevant tracking issue.
## Issue Context
The guide states `harness-snapshot.json` is only proposed / not yet guaranteed, which is planned functionality and must be formatted accordingly.
## Fix Focus Areas
- docs/guides/infrastructure/distributed-tracing.md[16-20]
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
Comment on lines
+1
to
+3
| --- | ||
| title: "71. Harness snapshot and forge pointer correlation for agent traces" | ||
| status: Proposed |
There was a problem hiding this comment.
3. Adr 0071 uses proposed 📘 Rule violation ⚙ Maintainability
ADR 0071 declares status: Proposed in its frontmatter and repeats “Proposed” in the body, but the repository’s ADR lifecycle vocabulary only allows Accepted, Deprecated, or Superseded. This makes the ADR nonconforming with documented conventions and likely to fail CI/lint validation tied to ADR status correctness.
Agent Prompt
## Issue description
ADR 0071 uses a `status` value (`Proposed`) in both frontmatter and the body `## Status` section, but the repository’s ADR policy restricts valid statuses to `Accepted`, `Deprecated`, or `Superseded` and indicates CI linting validates ADR status/frontmatter correctness.
## Issue Context
Allowed ADR `status` values are `Accepted`, `Deprecated`, or `Superseded` only.
- If this ADR is intended to be merged as a finalized decision record, update ADR 0071 to use an allowed status value (typically `Accepted`) and align the body `## Status` section accordingly.
- If the intent is to support “Proposed” ADRs in this repository, then ADR 0001 and the ADR lint configuration would need to be updated to include/accept `Proposed` (those config changes are not shown here).
- If the decision is not yet accepted and the process is to only store accepted decisions as ADRs, consider relocating this document to a non-ADR area (e.g., plans/spikes) until it is accepted.
## Fix Focus Areas
- docs/ADRs/0071-forge-pointer-correlation-for-agent-traces.md[1-20]
- docs/ADRs/0001-use-adrs-for-decision-making.md[42-63]
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
Comment on lines
+40
to
+78
| ## Decision | ||
|
|
||
| ### 1. Harness snapshot artifact (Level 1) | ||
|
|
||
| Every `fullsend run` writes **`harness-snapshot.json`** into the run output | ||
| directory (next to `run-telemetry.jsonl`) at run start, after the root span | ||
| exists. | ||
|
|
||
| Contents are pointers and a config fingerprint only (no diffs, prompts, or | ||
| skill bodies), including at least: | ||
|
|
||
| - Harness identity: agent, role, slug, model, skills, harness path | ||
| - `harness_content_sha` (content hash of the resolved harness file) | ||
| - Forge/CI pointers when known: `forge_platform`, `repository_url`, | ||
| `ref_revision`, `ref_name`, `change_id`, `pipeline_run_id`, | ||
| `pipeline_run_url` | ||
| - Trace join: `trace_id`, `traceparent` | ||
|
|
||
| Forge/CI fields are filled by fullsend when writing the snapshot (`FULLSEND_*` | ||
| overrides first, then standard CI env). Downstream consumers **read the JSON | ||
| file** (and/or later stores that ingest it), not ambient CI env. | ||
|
|
||
| `OTEL_SDK_DISABLED=true` suppresses span export (`run-telemetry.jsonl` / OTLP) | ||
| but **does not** suppress `harness-snapshot.json` — it is the run-start config | ||
| contract, not span export. | ||
|
|
||
| ### 2. Root span attributes (Level 1 + Level 2) | ||
|
|
||
| The same join keys are set on the root `run` span (`vcs.*`, `cicd.*`, | ||
| harness content SHA, forge platform) so OTLP backends receive them without | ||
| parsing the JSON. Local JSON remains the forensic / handoff contract. | ||
|
|
||
| ### 3. What we do not do | ||
|
|
||
| - Do **not** treat eval scripts’ CI env exports as the harness-snapshot path. | ||
| - Do **not** embed PR diffs or prompt content in the snapshot. | ||
| - Do **not** redefine cross-project report field ownership in this ADR; link | ||
| the shared join contract from Related. | ||
|
|
There was a problem hiding this comment.
4. Adr 0071 has multiple decisions 📜 Skill insight ⚙ Maintainability
ADR 0071 makes multiple distinct decisions (snapshot artifact, root span attributes, and explicit non-decisions) instead of recording exactly one decision. This reduces traceability and makes it harder to supersede or refine individual decisions cleanly.
Agent Prompt
## Issue description
The ADR contains multiple separate decisions under `## Decision` (numbered subsections) rather than a single decision.
## Issue Context
Compliance requires each ADR to record exactly one decision; separate decisions should be split into separate ADRs or narrowed so the ADR has one core decision.
## Fix Focus Areas
- docs/ADRs/0071-forge-pointer-correlation-for-agent-traces.md[40-78]
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
Comment on lines
+79
to
+87
| ## Consequences | ||
|
|
||
| **Easier:** One decided artifact per run; operators and eval loggers can join | ||
| runs to forge/CI and to `run-telemetry.jsonl` via `trace_id`. | ||
|
|
||
| **Harder:** Dispatched child runs must inherit forge context via env so the | ||
| child snapshot is complete; GitLab/Bitbucket coverage depends on CI vars or | ||
| `FULLSEND_*` overrides. Implementation is a follow-up issue after this ADR is | ||
| accepted. |
There was a problem hiding this comment.
5. Adr 0071 consequences not bullets 📜 Skill insight ⚙ Maintainability
The Consequences section is not expressed as 3–5 one-sentence bullet points as required. This makes consequences harder to scan and breaks the standardized ADR format.
Agent Prompt
## Issue description
`## Consequences` is currently prose blocks and does not provide 3–5 one-sentence bullet points.
## Issue Context
Compliance requires consequences to be a short bulleted list to preserve scannability and consistent ADR structure.
## Fix Focus Areas
- docs/ADRs/0071-forge-pointer-correlation-for-agent-traces.md[79-87]
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
Comment on lines
+21
to
+42
| ## Context | ||
|
|
||
| [ADR 0050](0050-distributed-tracing-instrumentation.md) defines how fullsend | ||
| generates traces (OTel Go SDK; Level 1 = `run-telemetry.jsonl`; Level 2 = | ||
| OTLP). [Operational observability](../problems/operational-observability.md) | ||
| requires forge **pointers** (repo + SHA, change id, pipeline run) rather than | ||
| duplicating diffs, and a stable answer to: “what harness/config produced this | ||
| run?” | ||
|
|
||
| After the OTel SDK migration, `run-summary.json` was removed; the sole Level 1 | ||
| telemetry file is `run-telemetry.jsonl`. Eval wrappers that `export` ambient CI | ||
| variables are the wrong layer: harness identity is a **run-time** concern of | ||
| `fullsend run`, and downstream tools must not scrape CI env as the source of | ||
| truth. | ||
|
|
||
| Cross-project join field semantics (Provenance / MLflow tag names) live in the | ||
| shared join contract maintained with the certification report schema; this ADR | ||
| decides only what **fullsend** writes. | ||
|
|
||
| ## Decision | ||
|
|
||
| ### 1. Harness snapshot artifact (Level 1) |
There was a problem hiding this comment.
6. Adr 0071 missing ## options 📜 Skill insight ⚙ Maintainability
ADR 0071 does not include the ## Options section from the ADR template. This deviates from the required ADR structure and omits the standardized place to capture alternatives.
Agent Prompt
## Issue description
The ADR omits the `## Options` section required by the ADR template structure.
## Issue Context
The project ADR template includes an `## Options` section (optional only when genuinely unnecessary). This ADR discusses at least one alternative approach (e.g., relying on eval scripts/CI env), so the template section should be present to document options.
## Fix Focus Areas
- docs/ADRs/0071-forge-pointer-correlation-for-agent-traces.md[21-42]
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
Comment on lines
+44
to
+60
| Every `fullsend run` writes **`harness-snapshot.json`** into the run output | ||
| directory (next to `run-telemetry.jsonl`) at run start, after the root span | ||
| exists. | ||
|
|
||
| Contents are pointers and a config fingerprint only (no diffs, prompts, or | ||
| skill bodies), including at least: | ||
|
|
||
| - Harness identity: agent, role, slug, model, skills, harness path | ||
| - `harness_content_sha` (content hash of the resolved harness file) | ||
| - Forge/CI pointers when known: `forge_platform`, `repository_url`, | ||
| `ref_revision`, `ref_name`, `change_id`, `pipeline_run_id`, | ||
| `pipeline_run_url` | ||
| - Trace join: `trace_id`, `traceparent` | ||
|
|
||
| Forge/CI fields are filled by fullsend when writing the snapshot (`FULLSEND_*` | ||
| overrides first, then standard CI env). Downstream consumers **read the JSON | ||
| file** (and/or later stores that ingest it), not ambient CI env. |
There was a problem hiding this comment.
7. harness-snapshot contract inlined 📜 Skill insight ⚙ Maintainability
ADR 0071 defines a field-level contract for harness-snapshot.json inline (required keys and semantics) instead of linking to a versioned normative spec. This risks unversioned drift and makes it harder for downstream consumers to implement against a stable artifact definition.
Agent Prompt
## Issue description
The ADR includes an inline field-level contract for `harness-snapshot.json` (required keys and semantics).
## Issue Context
Compliance requires detailed contracts (field-level/byte-level artifacts, schemas, canonical snapshots) to live in `docs/normative/<topic>/v<major>/...` and be linked from the ADR.
## Fix Focus Areas
- docs/ADRs/0071-forge-pointer-correlation-for-agent-traces.md[44-70]
- docs/normative/** (add versioned contract artifacts)
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
Comment on lines
+10
to
+20
| Every `fullsend run` produces Level 1 artifacts in the run output directory with | ||
| no configuration required: | ||
|
|
||
| - **`run-telemetry.jsonl`** — OTLP JSON spans covering the run lifecycle | ||
| (sandbox creation, agent iterations, validation) with timestamps, durations, | ||
| trace IDs, and token/cost attributes. | ||
|
|
||
| This file is written on every run unless `OTEL_SDK_DISABLED=true`, which | ||
| suppresses all telemetry output including the local file. It contains | ||
| metadata only — no prompts, completions, or source code content. | ||
| - **`harness-snapshot.json`** — proposed in | ||
| [ADR 0071](../../ADRs/0071-forge-pointer-correlation-for-agent-traces.md) | ||
| (run-start harness fingerprint and forge/CI pointers). Implementation follows | ||
| ADR acceptance; until then only `run-telemetry.jsonl` is guaranteed on disk. | ||
|
|
There was a problem hiding this comment.
8. Ambiguous level-1 wording 🐞 Bug ⚙ Maintainability
The distributed tracing guide says every run produces “Level 1 artifacts” while also noting harness-snapshot.json is only proposed and not yet guaranteed, which can momentarily suggest it exists today. Tightening the opening sentence to clearly separate current behavior from the proposed artifact would reduce reader confusion.
Agent Prompt
## Issue description
The guide introduces Level 1 as producing “artifacts” (plural) and then lists `harness-snapshot.json` while also stating it is only proposed and not yet guaranteed; this is slightly ambiguous.
## Issue Context
Make the first sentence explicitly describe the current guaranteed artifact (`run-telemetry.jsonl`) and then separately describe the proposed additional artifact from ADR 0071.
## Fix Focus Areas
- docs/guides/infrastructure/distributed-tracing.md[8-23]
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
This was referenced Jul 23, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Propose ADR 0071: Level-1
harness-snapshot.jsonplus root-span forge pointers as the run-time join contract. Docs only — implementation is a separate issue/PR per CONTRIBUTING.Related Issue
Implementation checklist: #5449 (after this ADR is accepted)
Changes
harness-snapshot.jsonat run start; mirror join keys on root span; consumers read the artifact, not eval-script CI env exports)Testing
mainat merge (/renumber-adrif needed)Checklist
Made with Cursor