docs(tracing): split tracing docs by audience

[rh-hemartin]

Summary

• Split the monolithic distributed-tracing.md into three audience-targeted docs: dev internals (tracing.md), user how-to (how-to-emit-traces.md), and infrastructure reference (distributed-tracing.md)
• Add VitePress sidebar entries for both new pages
• Update GHA workflow sections to reflect OTEL variable forwarding from fix(#5886): forward missing OTEL variables in managed workflows #5887

Closes #5659

🤖 Generated with Claude Code

[qodo-code-review]

PR Summary by Qodo

Docs: split tracing documentation by audience and update VitePress nav

📝 Documentation ⚙️ Configuration changes 🕐 20-40 Minutes

AI Description

• Split tracing docs into contributor, user, and reference guides for clearer navigation.
• Refresh tracing reference with updated OTEL env vars, span/attribute tables, and operational
 notes.
• Add VitePress sidebar links for the new tracing internals and how-to pages.

Diagram

graph TD
  VP["VitePress config"] --> SB["Sidebar nav"] --> HOWTO["How-to: emit traces"] --> REF["Tracing reference"]
  SB --> DEV["Tracing internals"] --> ADR["ADR 0050"]
  DEV -. "cross-links" .-> REF

Loading

High-Level Assessment

The following are alternative approaches to this PR:

1. Keep one page with audience-specific sections + TOC anchors

• ➕ Single canonical URL (fewer links to manage)
• ➕ Less duplicated context between pages
• ➖ Harder for users to find the right section quickly
• ➖ Long page grows over time and becomes harder to maintain

2. Use VitePress front-matter + tags to generate audience views

• ➕ One source of truth with multiple navigational entry points
• ➕ Scales well as more tracing subtopics are added
• ➖ More site/config complexity
• ➖ Requires consistent metadata discipline across docs

3. Auto-generate the reference tables from code/constants

• ➕ Reduces drift between implementation and documentation
• ➕ Makes attribute/reference changes safer over time
• ➖ Non-trivial tooling investment
• ➖ Harder to keep narrative/operational guidance readable

Recommendation: The PR’s approach (three audience-targeted docs + explicit cross-links + sidebar entries) is the best tradeoff for readability and discoverability with minimal tooling overhead. The main alternative worth considering long-term is auto-generating reference tables, but that’s likely overkill for this change.

Files changed (4) +551 / -235

Documentation (3) +549 / -235

tracing.mdAdd contributor-focused tracing internals guide +237/-0

Add contributor-focused tracing internals guide

• Introduces a new developer-facing document explaining tracer provider setup, span lifecycle in run.go, W3C TRACEPARENT propagation, and exporter behavior. Includes guidance for adding attributes and for unit/local testing, with links to the user how-to, reference page, and ADR 0050.

docs/guides/dev/tracing.md

distributed-tracing.mdRefactor distributed tracing doc into a reference page +168/-235

Refactor distributed tracing doc into a reference page

• Reframes the page as a structured reference: telemetry levels, environment variable tables, span hierarchy/kinds, attribute tables, output format, correlation guidance, and operational behavior. Updates GitHub Actions workflow guidance to reflect forwarding of additional OTEL variables/secrets (e.g., base headers, SDK disable switch, CA bundle path) and adds cross-links to the new how-to and dev guides.

docs/guides/infrastructure/distributed-tracing.md

how-to-emit-traces.mdAdd end-user how-to for enabling OTLP trace export +144/-0

Add end-user how-to for enabling OTLP trace export

• Adds a step-by-step setup guide for configuring OTEL endpoint variables and auth header secrets at repo or org scope using the GitHub UI or gh CLI. Documents how to disable export vs disable all telemetry via OTEL_SDK_DISABLED, and links to the tracing reference and internals guide.

docs/guides/user/how-to-emit-traces.md

Other (1) +2 / -0

config.tsExpose new tracing docs in the VitePress sidebar +2/-0

Expose new tracing docs in the VitePress sidebar

• Adds sidebar entries for the new user guide (How To Emit Traces) and dev guide (Tracing Internals) so the split documentation is discoverable from the docs site navigation.

website/.vitepress/config.ts

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 9:12 AM UTC · Completed 9:29 AM UTC
Commit: 126322c · View workflow run →

[github-actions]

Site preview

Preview: https://ab59f466-site.fullsend-ai.workers.dev

Commit: 5fc1772653dbf6c511223c0536519bbba9fca907

[qodo-code-review]

Code Review by Qodo

🐞 Bugs (1) 📘 Rule violations (0) 📜 Skill insights (2)

1. tracing.md missing prerequisites section ✓ Resolved 📜 Skill insight ✧ Quality

Description

docs/guides/dev/tracing.md introduces procedural steps but does not include a clearly labeled
prerequisites section before them. Readers may start procedures without required setup context.

Code

docs/guides/dev/tracing.md[R160-163]

+## How to add a new span attribute
+
+1. Add the `attribute.String` / `attribute.Int` / `attribute.Float64` call
+   in `run.go` at the appropriate point. Use start attributes for values

Relevance

●●● Strong

Team has accepted adding explicit prerequisites sections before procedural steps in guides.

PR-#2663
PR-#2277

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

PR Compliance ID 1062078 requires a prerequisites section before procedural steps in guides. In
docs/guides/dev/tracing.md, the procedural section begins with numbered steps under `## How to add
a new span attribute` without any preceding prerequisites section.

docs/guides/dev/tracing.md[160-166]
Skill: writing-user-docs

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The guide contains procedures (numbered steps) but lacks a clearly labeled prerequisites section before those procedures.

## Issue Context
Compliance requires prerequisites to appear before step 1 of any procedure in a documentation guide.

## Fix Focus Areas
- docs/guides/dev/tracing.md[149-170]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

2. docs/guides/README.md missing new guides ✓ Resolved 📜 Skill insight ⚙ Maintainability

Description

This PR adds new guide pages under docs/guides/ but does not update docs/guides/README.md to
index them. This makes the new guides harder to discover and violates the guide index requirement.

Code

docs/guides/user/how-to-emit-traces.md[R1-4]

+# How To Emit Traces
+
+## Overview
+

Relevance

●●● Strong

Updating docs/guides/README.md index when adding a guide was previously accepted.

PR-#5778

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

PR Compliance ID 1062084 requires docs/guides/README.md to be updated when adding a new guide
under docs/guides/. The PR adds docs/guides/user/how-to-emit-traces.md and
docs/guides/dev/tracing.md, but the README index does not list them.

docs/guides/user/how-to-emit-traces.md[1-12]
docs/guides/dev/tracing.md[1-13]
docs/guides/README.md[21-55]
Skill: writing-user-docs

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
New guide files were added under `docs/guides/`, but `docs/guides/README.md` was not updated to include entries for them.

## Issue Context
The guides README is the index used to find guides by audience; it must be updated whenever new guides are added.

## Fix Focus Areas
- docs/guides/README.md[21-55]
- docs/guides/user/how-to-emit-traces.md[1-20]
- docs/guides/dev/tracing.md[1-20]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

3. Missing OTEL env forwarding ✗ Dismissed 🐞 Bug ≡ Correctness

Description

distributed-tracing.md now claims managed workflows forward OTEL_EXPORTER_OTLP_HEADERS,
OTEL_SDK_DISABLED, and OTEL_EXPORTER_OTLP_CERTIFICATE, but the managed/reusable workflows and
scaffold templates shown in-repo do not pass these into agent runs. Operators who set these
variables/secrets as instructed will see no effect (e.g., cannot disable telemetry via
OTEL_SDK_DISABLED in managed runs).

Code

docs/guides/infrastructure/distributed-tracing.md[R222-224]

+| `OTEL_EXPORTER_OTLP_HEADERS` | Secret | Base headers (fallback when signal-specific headers are not set) |
+| `OTEL_SDK_DISABLED` | Variable | Disable all telemetry output |
+| `OTEL_EXPORTER_OTLP_CERTIFICATE` | Variable | PEM bundle path for private CAs |

Relevance

●●● Strong

They’ve previously accepted fixing docs vs managed-workflow OTEL passthrough mismatches; likely
update docs or workflows.

PR-#3903

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

The reference doc claims these env vars/secrets are forwarded by managed workflows, but the reusable
workflow step that actually runs the agent only sets endpoint vars,
OTEL_EXPORTER_OTLP_TRACES_HEADERS, and OTEL_RESOURCE_ATTRIBUTES, and the scaffold templates only
forward OTEL_EXPORTER_OTLP_TRACES_HEADERS as a secret.

docs/guides/infrastructure/distributed-tracing.md[200-230]
.github/workflows/reusable-dispatch.yml[614-623]
.github/workflows/reusable-triage.yml[175-187]
internal/scaffold/fullsend-repo/.github/workflows/triage.yml[28-42]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The tracing reference claims the managed workflows forward `OTEL_EXPORTER_OTLP_HEADERS`, `OTEL_SDK_DISABLED`, and `OTEL_EXPORTER_OTLP_CERTIFICATE`, but current managed workflows/scaffolds don’t pass these into the `fullsend run` environment. This makes the documentation incorrect and prevents documented controls (notably `OTEL_SDK_DISABLED`) from working in managed runs.

## Issue Context
- Docs explicitly state these variables are forwarded.
- The reusable workflows only export endpoint + `OTEL_EXPORTER_OTLP_TRACES_HEADERS` + `OTEL_RESOURCE_ATTRIBUTES`.

## Fix Focus Areas
Choose one:
- Update managed workflows + scaffolds to actually forward the documented vars/secrets, OR
- Update the docs to remove/qualify the claim and reflect the variables that are truly forwarded today.

Fix locations:
- docs/guides/infrastructure/distributed-tracing.md[200-245]
- .github/workflows/reusable-dispatch.yml[614-623]
- .github/workflows/reusable-triage.yml[175-187]
- internal/scaffold/fullsend-repo/.github/workflows/triage.yml[28-42]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

4. Disable export misses endpoint 🐞 Bug ≡ Correctness

Description

how-to-emit-traces.md instructs disabling trace export by deleting only
OTEL_EXPORTER_OTLP_ENDPOINT, but the exporter is enabled when either OTEL_EXPORTER_OTLP_ENDPOINT
or OTEL_EXPORTER_OTLP_TRACES_ENDPOINT is set. Users who configured the signal-specific endpoint
will keep exporting traces after following these steps.

Code

docs/guides/user/how-to-emit-traces.md[R118-120]

+gh variable delete OTEL_EXPORTER_OTLP_ENDPOINT --repo <owner/repo>
+gh secret delete OTEL_EXPORTER_OTLP_TRACES_HEADERS --repo <owner/repo>
+```

Relevance

●●● Strong

Doc correctness fix; aligns with prior accepted tracing-doc accuracy/endpoint configuration
improvements.

PR-#3903

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

The how-to doc’s disable commands only delete the base endpoint variable, while the implementation
enables OTLP export when *either* the base endpoint or the traces-specific endpoint is set.

docs/guides/user/how-to-emit-traces.md[111-120]
internal/telemetry/telemetry.go[106-109]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The “Disable trace export” instructions only remove `OTEL_EXPORTER_OTLP_ENDPOINT`, but the code enables OTLP export when either `OTEL_EXPORTER_OTLP_ENDPOINT` or `OTEL_EXPORTER_OTLP_TRACES_ENDPOINT` is present. This leaves a documented path where users think export is disabled but it continues.

## Issue Context
`internal/telemetry/telemetry.go` checks both env vars before setting up the exporter.

## Fix Focus Areas
- docs/guides/user/how-to-emit-traces.md[111-120]
- internal/telemetry/telemetry.go[106-109]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

5. Planned feature lacks issue link 📜 Skill insight ≡ Correctness

Description

The updated tracing reference documents a planned feature without using the required `>
**Planned:**` callout format and without an issue link. This can mislead readers into thinking the
feature exists or lacks tracking.

Code

docs/guides/infrastructure/distributed-tracing.md[R13-15]

+| 1 | `run-telemetry.jsonl` file in the run output directory | None |
+| 2 | OTLP/HTTP export to a remote backend (metadata only) | `OTEL_EXPORTER_OTLP_*ENDPOINT` |
+| 3 | Content capture (prompts, completions, tool I/O) in spans | `OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=true` *(planned, not yet implemented)* |

Relevance

● Weak

Closest precedent rejected requiring a Planned callout with tracking issue link for tracing planned
content.

PR-#3903

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

PR Compliance ID 1062082 requires planned functionality to be documented using a > **Planned:**
callout and to include an issue link. The updated table and section label content capture as planned
but provide no required callout or issue reference.

docs/guides/infrastructure/distributed-tracing.md[11-15]
Skill: writing-user-docs

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The doc mentions a not-yet-implemented feature (`Content capture`) but does not use the required `> **Planned:**` blockquote format and does not include a link to the tracking issue.

## Issue Context
Compliance requires planned features to be clearly marked and linked to an issue so readers can verify status and progress.

## Fix Focus Areas
- docs/guides/infrastructure/distributed-tracing.md[11-18]
- docs/guides/infrastructure/distributed-tracing.md[63-72]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

6. Guide under docs/guides/dev/ 📜 Skill insight ⌂ Architecture

Description

docs/guides/dev/tracing.md is a documentation guide but it is not placed under
docs/guides/admin/ or docs/guides/user/ as required. This breaks the required audience-based
guide directory structure.

Code

docs/guides/dev/tracing.md[R1-6]

+# Tracing Internals
+
+Fullsend's distributed tracing system records structured telemetry for every
+agent run. This document explains how the tracing implementation works and
+how to extend it. It is aimed at contributors modifying the telemetry
+package or the span instrumentation in the run command.

Relevance

● Weak

Prior reviews rejecting moves out of docs/guides/dev; repo keeps dev guides despite taxonomy rule.

PR-#4901
PR-#5502
PR-#5454

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

PR Compliance ID 1062077 requires guides under docs/guides/ to be placed in either admin/ or
user/. The PR adds a new guide under docs/guides/dev/, violating the directory requirement.

docs/guides/dev/tracing.md[1-6]
Skill: writing-user-docs

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
A new guide was added at `docs/guides/dev/tracing.md`, but the compliance rule requires that guides under `docs/guides/` live in either `docs/guides/admin/` or `docs/guides/user/`.

## Issue Context
This PR is reorganizing tracing documentation by audience, but the checklist requires a strict directory convention for guides.

## Fix Focus Areas
- docs/guides/dev/tracing.md[1-20]
- website/.vitepress/config.ts[278-282]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

Context used

✅ Compliance rules (platform): 54 rules

✅ Skills: writing-user-docs, writing-adrs

_{To customize comments, go to the Qodo configuration screen, or learn more in the docs.}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[fullsend-ai-review]

Review

Findings

Low

• [internal-consistency] docs/guides/README.md — The link text "Tracing reference" (Infrastructure section) uses sentence case while the target document's H1 is "Tracing Reference" (title case).
  Remediation: Change to [Tracing Reference](infrastructure/distributed-tracing.md).

• [internal-consistency] docs/guides/README.md — The link text "Tracing internals" (Development section) uses sentence case while the target document's H1 is "Tracing Internals" (title case).
  Remediation: Change to [Tracing Internals](dev/tracing.md).

• [internal-consistency] docs/guides/infrastructure/distributed-tracing.md — The introductory paragraph uses "Tracing Development Guide" as link text but the target document's H1 is "Tracing Internals".
  Remediation: Change [Tracing Development Guide](../dev/tracing.md) to [Tracing Internals](../dev/tracing.md).

• [internal-consistency] docs/guides/infrastructure/distributed-tracing.md — The "See also" section uses "Tracing Development Guide" as link text but the target document's H1 is "Tracing Internals".
  Remediation: Change [Tracing Development Guide](../dev/tracing.md) to [Tracing Internals](../dev/tracing.md).

• [internal-consistency] docs/guides/user/how-to-emit-traces.md — The "See also" section uses "Tracing Development Guide" as link text but the target document's H1 is "Tracing Internals".
  Remediation: Change [Tracing Development Guide](../dev/tracing.md) to [Tracing Internals](../dev/tracing.md).

• [internal-consistency] docs/guides/user/tracing-with-mlflow.md — The "See also" link text uses "Tracing reference" (lowercase "r") while the target document's H1 is "Tracing Reference" (title case).
  Remediation: Change to [Tracing Reference](../infrastructure/distributed-tracing.md).

• [internal-consistency] docs/guides/infrastructure/distributed-tracing.md — The "See also" section links the user and dev guides but omits ADR 0050. The dev guide links ADR 0050 directly; adding it here improves discoverability per issue docs(telemetry): reorganize into user guide, reference, and dev guide #5659.
  Remediation: Add - [ADR 0050](../../ADRs/0050-distributed-tracing-instrumentation.md): design decision to the See also section.

• [internal-consistency] docs/guides/user/how-to-emit-traces.md — The "Before you begin" prerequisites use https://mlflow.example.com:4318/v1/traces as the example URL, mixing an MLflow-specific hostname with the generic OTLP port in a backend-agnostic guide.
  Remediation: Use a backend-neutral hostname like https://tracing.example.com:4318/v1/traces.

• [technical-accuracy] docs/guides/user/tracing-with-mlflow.md — The justification for preferring OTEL_EXPORTER_OTLP_TRACES_ENDPOINT states the SDK would "append /v1/traces to the base URL," implying a problem. Since MLflow's path IS {server}/v1/traces, the base endpoint with SDK appending produces the correct URL. Both approaches work; the reasoning is slightly misleading.
  Remediation: Reframe as a best-practice recommendation for explicitness rather than a necessity.

Previous run

Review

Findings

Low

• [internal-consistency] docs/guides/user/tracing-with-mlflow.md — The "See also" link text uses "Tracing reference" (lowercase "r") while the target document's H1 is "Tracing Reference" (title case). The other See also sections in the new tracing docs consistently use title case matching the target H1.
  Remediation: Change "Tracing reference" to "Tracing Reference" to match the target document's H1.

• [incomplete-authorization] docs/guides/infrastructure/distributed-tracing.md — Issue docs(telemetry): reorganize into user guide, reference, and dev guide #5659 notes that "ADR 0050 provides architectural rationale but isn't linked from either guide." The dev guide (tracing.md) now links ADR 0050, but the reference doc does not.
  Remediation: Add an ADR 0050 link to the "See also" section of distributed-tracing.md, e.g.: - [ADR 0050](../../ADRs/0050-distributed-tracing-instrumentation.md): design decision

Previous run (2)

Review

Findings

Low

• [internal-consistency] docs/guides/README.md:29 — The README link text uses sentence case "Tracing reference" while the document's H1 and the VitePress sidebar both use title case "Tracing Reference". The link text in README.md should match the document title.
  Remediation: Change the README link text from "Tracing reference" to "Tracing Reference" to match the H1 and sidebar.

Previous run (3)

Looks good to me

Previous run (4)

Review

Findings

Medium

• [stale-reference] website/.vitepress/config.ts:261 — The VitePress sidebar entry reads "Distributed Tracing" but the document title was changed to "Tracing Reference". The sidebar text and page heading will disagree, confusing users navigating the site.
  Remediation: Update to { text: "Tracing Reference", link: "/guides/infrastructure/distributed-tracing" }

• [stale-reference] docs/guides/README.md:31 — The link text "Distributed tracing" and description "Configuring OpenTelemetry instrumentation and OTLP backends" no longer match the document, whose title is now "Tracing Reference" and whose content was restructured from a configuration guide to a reference document covering telemetry levels, environment variables, span hierarchy, and attributes.
  Remediation: Update to - [Tracing reference](infrastructure/distributed-tracing.md) — Telemetry levels, environment variables, span hierarchy, and attributes

Low

• [see-also-pattern] docs/guides/dev/tracing.md:233 — Uses "## Related resources" for the closing cross-reference section; the codebase convention is "## See also" (found in 8 existing guides, zero instances of "Related resources"). Notably, the other new file in this PR (how-to-emit-traces.md) correctly uses "## See also".
  Remediation: Change to "## See also"

• [documentation-omission] docs/guides/dev/tracing.md:103 — The Root span section lists start and end attributes but omits fullsend.security_trace_id, fullsend.prescript.skipped, and fullsend.prescript.skip_reason, which the reference doc's attribute table lists on the run span.
  Remediation: Add the missing attributes or add a cross-reference to the reference doc's comprehensive attribute table.

• [content-loss] docs/guides/infrastructure/distributed-tracing.md — The old document contained MLflow-specific examples (experiment routing, Basic auth encoding, cost column caveats), "Organizing traces for an org" guidance, and a production deployment checklist. These were removed. The new user guide partially covers MLflow (experiment ID routing in step 5) but loses the Basic auth encoding example and cost column caveat.

Previous run (5)

Review

Findings

Medium

• [stale-reference] website/.vitepress/config.ts — The VitePress sidebar entry still reads "Distributed Tracing" but the document title was changed to "Tracing Reference". The sidebar uses hardcoded text, so the label and page heading will disagree.
  Remediation: Update to { text: "Tracing Reference", link: "/guides/infrastructure/distributed-tracing" }

• [stale-reference] docs/guides/README.md:29 — The link text references "Distributed tracing" but the document title is now "Tracing Reference".
  Remediation: Update line 29 to match the new title.

• [incomplete-implementation] docs/guides/README.md — ADR-0023 specifies docs/guides/README.md as the enforcement point for guide indexing. Two new guides (dev/tracing.md, user/how-to-emit-traces.md) were added with sidebar entries but without README entries.
  Remediation: Add entries for both new guides to docs/guides/README.md.

Low

• [missing-authorization] — Documentation reorganization (786 lines, 4 files) with no linked issue. Consider creating a tracking issue for reviewability.

• [content-loss] docs/guides/infrastructure/distributed-tracing.md — The old document contained MLflow-specific examples (experiment routing, Basic auth encoding, cost column caveats), "Organizing traces for an org" guidance, and a production deployment checklist. These sections were removed. If intentionally simplified for the audience split, this is fine — verify no content was accidentally dropped.

• [see-also-pattern] docs/guides/dev/tracing.md:233 — Uses "## Related resources" for the closing cross-reference section; the rest of the repo uses "## See also".
  Remediation: Rename to "## See also".

• [documentation-omission] docs/guides/dev/tracing.md:103 — The Root span section lists start and end attributes but omits fullsend.security_trace_id, which the reference doc's attribute table lists on the run span.

• [naming-convention] docs/guides/user/how-to-emit-traces.md:1 — Title "How To Emit Traces" uses a "How To" prefix not seen in other user guides (e.g., "Running Agents Locally", "Jira Integration").
  Remediation: Consider "Emitting Traces" or "Trace Export" to match established naming.

---

Labels: PR exclusively modifies documentation files and doc-site config

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 9:31 AM UTC · Completed 9:47 AM UTC
Commit: 6801388 · View workflow run →

[waynesun09]

Review-only sweep: 3 additional findings not covered by existing review comments (2 other candidate findings — stale README index entry and stale VitePress sidebar label — were already raised in an existing review and are skipped here as duplicates).

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 12:31 PM UTC · Completed 12:46 PM UTC
Commit: 37b37e0 · View workflow run →

[waynesun09]

Review-only sweep: 3 additional findings not covered by existing review comments.

[fullsend-ai-review]

🤖 Review · ⚠️ Cancelled · Started 6:11 AM UTC · Ended 6:12 AM UTC
Commit: 1102c21 · View workflow run →

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 6:13 AM UTC · Completed 6:26 AM UTC
Commit: 51b6c7b · View workflow run →

[waynesun09]

Review-only sweep: 4 additional findings not covered by existing review comments (1 candidate finding — unverified otel-gui backend at docs/guides/dev/tracing.md:268 — was already raised in an existing review thread and is skipped here as a duplicate).

[fullsend-ai-review]

🤖 Review · ❌ Terminated · Started 1:50 PM UTC · Ended 2:04 PM UTC
Commit: 0af7020 · View workflow run →

[waynesun09]

Approving — the merge conflict with #5887 (private-CA checkout-path requirement) is resolved, mergeable now shows clean. No blocking issues. One non-blocking MEDIUM fast-follow: a new table duplicates content already in the numbered list right above it (same 3 env vars, two formats back to back) — worth consolidating but not gating. Remaining threads are pre-existing LOW naming/style nits. CI green.

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 6:45 AM UTC · Completed 7:02 AM UTC
Commit: 5fc1772 · View workflow run →

[fullsend-ai-review]

[low] internal-consistency

The link text "Tracing reference" (Infrastructure section) uses sentence case while the target document's H1 is "Tracing Reference" (title case).

Suggested fix: Change to Tracing Reference.

[fullsend-ai-review]

[low] internal-consistency

The link text "Tracing internals" (Development section) uses sentence case while the target document's H1 is "Tracing Internals" (title case).

Suggested fix: Change to Tracing Internals.

[fullsend-ai-review]

[low] internal-consistency

The introductory paragraph uses "Tracing Development Guide" as link text but the target document's H1 is "Tracing Internals".

Suggested fix: Change Tracing Development Guide to Tracing Internals.

[fullsend-ai-review]

[low] internal-consistency

The "Before you begin" prerequisites use https://mlflow.example.com:4318/v1/traces as the example URL, mixing an MLflow-specific hostname with the generic OTLP port in a backend-agnostic guide.

Suggested fix: Use a backend-neutral hostname like https://tracing.example.com:4318/v1/traces.

[fullsend-ai-retro]

🤖 Finished Retro · ✅ Success · Started 7:05 AM UTC · Completed 7:19 AM UTC
Commit: 5fc1772 · View workflow run →

[fullsend-ai-retro]

Retro: PR #5888 — docs(tracing): split tracing docs by audience

Timeline: Human-authored PR by rh-hemartin (opened Aug 4, merged Aug 6) that split the monolithic distributed-tracing.md into three audience-targeted docs: dev internals, user how-to, and infrastructure reference. The review agent ran 7 times (6 succeeded, 1 cancelled) and issued 4 APPROVED reviews. Human reviewer waynesun09 conducted 4 thorough review sweeps, catching 3 HIGH-severity and 8 MEDIUM-severity issues before approving on Aug 5.

Review quality delta: The review agent found ~12 findings — all LOW or MEDIUM cosmetic consistency issues (casing, naming patterns, link text). The human reviewer found 11 issues at substantially higher severity: false factual claims (artifact upload claim not backed by any workflow YAML), a policy reversal (ADR link deliberately removed in commit f058f7d then re-added), a merge conflict with concurrent PR #5887, and 5 specific content omissions. The agent and human had complementary strengths — the agent excelled at internal consistency, while the human excelled at factual verification, policy awareness, and content completeness.

Most notably, the review agent actively recommended adding the ADR 0050 link that the human identified as a HIGH-severity policy violation. The agent's consistency optimization created the very problem the human caught.

Existing issues with new supporting evidence:

• Review agent should fact-check technical claims in documentation PRs #2199 (fact-check technical claims in docs PRs): The false claim that run-telemetry.jsonl is uploaded as a GHA artifact, caught only by the human reviewer, is a clear instance of this gap.
• Review docs-currency sub-agent should cross-validate documentation examples against actual source files #5297 (cross-validate doc examples against source files): The unverified otel-gui backend reference is another instance.
• Review agent should deduplicate findings across re-review iterations on the same PR #2959 (deduplicate findings across re-review iterations): The agent repeated LOW-severity consistency findings across multiple review runs on this PR.

Autonomy readiness: The review agent's first approval came at 12:46 on Aug 4 — more than 26 hours before the human approved at 15:18 on Aug 5. If the agent had autonomous merge authority for this PR, it would have merged with 3 HIGH-severity issues unresolved (false factual claims, policy reversal, merge conflict). Documentation restructuring PRs remain a category where human review is essential.

Workflow volume: 34 source-repo workflow runs were triggered (22 from pull_request_review events, 12 from pushes). Dispatch-side debouncing effectively reduced these to 7 actual review runs — a 78% reduction. Source-side trigger volume is already tracked across multiple open issues (#4960, #1014, #4069).

Proposals filed

• Review agent should check git history before recommending re-addition of deliberately removed content (in fullsend-ai/agents)
• Review agent should itemize and verify content redistribution in documentation restructuring PRs (in fullsend-ai/agents)