docs: add Bring Your Own Agent guide and agent CLI reference

[ggallen]

Summary

• Adds a comprehensive guide (docs/guides/user/bring-your-own-agent.md) covering how to build custom agents and configure existing ones — harness files, agent definitions, skills, scripts, base composition, registration, and migration from the deprecated customized/ directory overlay.
• Adds CLI reference for fullsend agent subcommands (docs/cli/agent.md): add, list, update, remove, and migrate-customizations. Promotes fullsend agent from "Additional commands" to "Command groups" in the CLI overview and adds it to the VitePress sidebar.
• Uses "configuring" (not "customizing") for existing agent changes, aligned with the default-vs-custom reference doc terminology per Ralph's feedback.
• Updates stale cross-references in docs/agents/README.md, docs/glossary.md, docs/agents/topics/default-vs-custom.md, and docs/guides/infrastructure/standalone-mint.md to point to the new BYOA guide instead of the deprecated building-custom-agents.md.
• Adds a supersession annotation to ADR 0065 (Accepted) noting the building-custom-agents guide has been superseded by bring-your-own-agent.md — minor cross-reference addition only, no decision changes.

Test plan

• Verify all internal anchor links resolve correctly (e.g. #configuring-existing-agents, #field-merge-rules-for-base-and-forge)
• Verify cross-doc links to customizing-with-skills.md, customizing-with-agents-md.md, and standalone-mint.md resolve
• Verify /cli/agent renders correctly and appears in the CLI sidebar
• Confirm terminology aligns with Default, derived, and custom agents

🤖 Generated with Claude Code

[qodo-code-review]

PR Summary by Qodo

Docs: add Bring Your Own Agent (BYOA) guide

📝 Documentation 🕐 20-40 Minutes

AI Description

• Add a BYOA guide explaining harnesses, agent definitions, skills, scripts, and sandbox security.
• Document base composition + merge rules for configuring built-in agents safely.
• Describe agent registration and migration from deprecated customized/ overlay to config-driven
 agents.

Diagram

graph TD
U["User / Repo author"] --> H["Harness YAML"] --> R["fullsend run"] --> S["Sandbox"]
H --> A["Agent definition (MD)"]
H --> P["Policy YAML"]
H --> X["Skills + pre/post scripts"]
C[".fullsend/config.yaml (agents list)"] --> R

Loading

High-Level Assessment

The following are alternative approaches to this PR:

1. Split into tutorial + reference pages

• ➕ Easier navigation and future maintenance (smaller, focused pages).
• ➕ Reference sections (merge rules, field listing) can be shared by other docs.
• ➖ More up-front restructuring and internal linking work.
• ➖ Readers lose the single end-to-end narrative in one page.

2. Extend existing customizing/configuring docs instead of a new guide

• ➕ Avoids duplicating concepts already covered elsewhere (skills, AGENTS.md).
• ➕ Keeps the docs tree flatter with fewer entry points.
• ➖ Existing pages may become overly long and harder to scan.
• ➖ Harder to present a coherent start-to-finish workflow (harness → config → CI).

3. Generate harness field reference from schema/source-of-truth

• ➕ Reduces risk of docs drifting from actual behavior over time.
• ➕ Can auto-include allowed values/defaults as they evolve.
• ➖ Requires tooling/integration work not justified for a docs-only PR.
• ➖ Generated output can be less approachable than curated explanations.

Recommendation: Keep the single BYOA guide as introduced (best for onboarding and end-to-end clarity), but strongly consider a follow-up that extracts the harness field reference + merge rules into a dedicated reference page (or generated page) once the structure stabilizes. That reduces long-term drift risk while preserving this guide’s narrative flow.

Files changed (1) +491 / -0

Documentation (1) +491 / -0

bring-your-own-agent.mdAdd end-to-end BYOA guide for building/configuring agents +491/-0

Add end-to-end BYOA guide for building/configuring agents

• Introduces a comprehensive guide covering harness vs. agent definition responsibilities, sandbox security model, and a minimum viable agent layout. Documents harness fields (including 'base' composition and merge rules), registration via '.fullsend/config.yaml' and CLI commands, plus migration guidance from the deprecated 'customized/' overlay and standalone mint identity considerations.

docs/guides/user/bring-your-own-agent.md

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 4:53 PM UTC · Completed 5:08 PM UTC
Commit: d4ceaa5 · View workflow run →

[github-actions]

Site preview

Preview: https://c6bf5395-site.fullsend-ai.workers.dev

Commit: 162cd3ad92a6023b4ee85cb606b7bb0d0c090572

[qodo-code-review]

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📜 Skill insights (3)

Context used

✅ Compliance rules (platform): 54 rules

✅ Skills: writing-user-docs, writing-adrs

1. No Prerequisites section 📜 Skill insight ✧ Quality

Description

The new guide includes procedural steps but does not provide a clearly labeled prerequisites section
before those steps. This makes it easy for readers to start commands/config changes without required
setup, violating the guide structure requirement.

Code

docs/guides/user/bring-your-own-agent.md[R459-472]

+## Advanced: custom identity
+
+By default, agents authenticate using shared fullsend GitHub Apps via the `slug` field. If you need your own GitHub App — for custom permissions, compliance, or branding — you can run a **standalone mint**. Follow the [Standalone mint guide](../infrastructure/standalone-mint.md) to set one up.
+
+Once your standalone mint is running, configure your agent to use it:
+
+1. **Reference your role in the harness:**
+   ```yaml
+   role: my-role
+   slug: my-org-my-role
+   ```
+
+2. **Set `FULLSEND_MINT_URL`** in your repo to point to your standalone mint.
+

Relevance

⭐⭐⭐ High

Prerequisites sections are enforced for guides with procedures; reviewers asked/accepted adding them
(#2277, fullsend-ai/fullsend#2663).

PR-#2277
PR-#2663

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

The checklist requires a prerequisites section in documentation guides before procedural steps. The
guide has numbered steps for configuring a standalone mint but does not include any Prerequisites
section earlier in the document.

docs/guides/user/bring-your-own-agent.md[1-31]
docs/guides/user/bring-your-own-agent.md[459-472]
Skill: writing-user-docs

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`docs/guides/user/bring-your-own-agent.md` contains procedures (numbered steps and command sequences) but lacks a clearly labeled `## Prerequisites` section before any procedure begins.

## Issue Context
Compliance requires every guide under `docs/guides/` to include a prerequisites section that states what the reader needs before step 1 of any procedure.

## Fix Focus Areas
- docs/guides/user/bring-your-own-agent.md[1-31]
- docs/guides/user/bring-your-own-agent.md[459-472]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

2. Guide index missing entry ✓ Resolved 📜 Skill insight ⚙ Maintainability

Description

A new guide was added under docs/guides/, but docs/guides/README.md was not updated to include
it. This makes the guide undiscoverable from the guides index, violating the index update
requirement.

Code

docs/guides/user/bring-your-own-agent.md[R1-5]

+# Bring Your Own Agent
+
+Add a custom agent to fullsend — or change the configuration of an existing one — from harness file to CI.
+
+This guide uses the [fullsend-ai/agents](https://github.com/fullsend-ai/agents) triage agent as a running example.

Relevance

⭐⭐⭐ High

New guide additions historically update docs/guides/README.md index (e.g., guide PRs fullsend-ai/fullsend#665, fullsend-ai/fullsend#1179,
#1190).

PR-#665
PR-#1179
PR-#1190

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

The compliance rule requires docs/guides/README.md to be updated when adding a new guide. The new
guide exists, but the current index list of user guides does not include Bring Your Own Agent.

docs/guides/user/bring-your-own-agent.md[1-5]
docs/guides/README.md[31-41]
Skill: writing-user-docs

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
A new guide file was added under `docs/guides/user/`, but `docs/guides/README.md` does not include a corresponding entry under `## User guides`.

## Issue Context
Compliance requires updating the guides index whenever a new guide is added.

## Fix Focus Areas
- docs/guides/README.md[31-42]
- docs/guides/user/bring-your-own-agent.md[1-5]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

3. Unnumbered procedure blocks 📜 Skill insight ✧ Quality

Description

Several procedural instructions are presented as prose labels followed by command blocks instead of
numbered steps. This violates the requirement that procedures use numbered steps rather than
narrative/prose formatting.

Code

docs/guides/user/bring-your-own-agent.md[R334-337]

+Register it:
+```bash
+fullsend agent add harness/code.yaml --name code --fullsend-dir .fullsend
+```

Relevance

⭐⭐⭐ High

Numbered-step procedures are repeatedly requested and accepted in docs reviews (e.g., fullsend-ai/fullsend#2663, fullsend-ai/fullsend#2277).

PR-#2663
PR-#2277

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

The checklist requires procedures to be expressed as numbered steps. The guide includes imperative
procedural actions like registration and migration commands without using an ordered list format.

docs/guides/user/bring-your-own-agent.md[308-337]
docs/guides/user/bring-your-own-agent.md[439-447]
Skill: writing-user-docs

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Procedural guidance is presented without ordered (numbered) steps (e.g., `Register it:` / `Preview what would change:` followed by a command block).

## Issue Context
Compliance requires procedural content in documentation guides to use numbered steps rather than prose paragraphs.

## Fix Focus Areas
- docs/guides/user/bring-your-own-agent.md[308-339]
- docs/guides/user/bring-your-own-agent.md[439-447]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

View more (1)

4. Architecture explained inline ✓ Resolved 📜 Skill insight ⚙ Maintainability

Description

The guide restates architectural concepts (agent runtime flow, sandboxing, network policy behavior)
inline without linking to architectural references. This violates the requirement to link to
ADRs/specs/docs/architecture.md for architecture context rather than restating it in guides.

Code

docs/guides/user/bring-your-own-agent.md[R7-29]

+## How agents work
+
+A fullsend agent has two parts:
+
+1. **Harness file** (YAML) — _how_ the agent runs: sandbox image, policy, scripts, skills, credentials, timeouts.
+2. **Agent definition** (Markdown) — _what_ the agent does: prompt, tools, model, skills.
+
+The harness is the entry point. `fullsend run triage` reads the harness, provisions a sandbox, and launches the agent inside it.
+
+```
+fullsend run triage
+        │
+        ▼
+┌── harness/triage.yaml ─────────┐
+│  agent: agents/triage.md        │  ◄── prompt & tools
+│  policy: policies/triage.yaml   │  ◄── sandbox rules
+│  skills: [issue-labels]         │  ◄── domain knowledge
+│  pre_script: scripts/pre-...    │  ◄── fetch data (before sandbox)
+│  post_script: scripts/post-...  │  ◄── act on output (after sandbox)
+└─────────────────────────────────┘
+```
+
+**Security model:** agents run inside a sandboxed environment. Network policies control which APIs the agent can reach, and `binaries` restrictions limit which programs can access each endpoint. Pre-scripts run on the trusted runner _before_ the sandbox starts; post-scripts run _after_ it exits.

Relevance

⭐⭐ Medium

Team enforces adding architecture links sometimes (#770), but no clear precedent forbidding inline
architecture restatements in guides.

PR-#770

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

The compliance rule requires guides to link to architectural references instead of restating
architecture inline. The How agents work section and the Security model paragraph describe
architecture and security behavior without linking to docs/architecture.md/ADRs for that context.

docs/guides/user/bring-your-own-agent.md[7-29]
Skill: writing-user-docs

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The guide includes architectural explanations inline (how harnesses launch sandboxes, security model/network policy behavior) instead of linking out to architectural reference docs.

## Issue Context
Guides should avoid duplicating architecture narratives; they should link to ADRs, normative specs, or `docs/architecture.md`.

## Fix Focus Areas
- docs/guides/user/bring-your-own-agent.md[7-29]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

5. Host_files merge rule wrong ✓ Resolved 🐞 Bug ≡ Correctness

Description

The guide states host_files are “Concatenated” under base/forge merge rules, but the
implementation deduplicates by dest and the child entry overrides the base entry on destination
collision. This mismatch can lead users to misunderstand which host_files entry will win when they
intentionally override an existing mount (e.g., to change a credential path), causing surprising
runtime behavior.

Code

docs/guides/user/bring-your-own-agent.md[R226-231]

+| Field type | Behavior |
+|-----------|----------|
+| Scalars (`model`, `pre_script`, etc.) | Child wins |
+| `skills`, `plugins`, `host_files` | Concatenated |
+| `env`, `runner_env` | Merged; child keys win |
+| `validation_loop`, `security` | Child replaces entirely |

Relevance

⭐⭐⭐ High

Reviewers often require docs match implementation behavior (e.g., runtime/docs correctness fixes
accepted in fullsend-ai/fullsend#2584).

PR-#2584

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

The guide explicitly documents host_files as concatenated, but the harness composition code merges
host_files with destination-based conflict resolution where child entries override base entries.
This is a behavioral mismatch between docs and implementation.

docs/guides/user/bring-your-own-agent.md[224-233]
internal/harness/compose.go[1130-1152]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
The documentation claims `host_files` are concatenated during `base`/`forge` composition, but the code actually merges `host_files` with **child-overrides-base when `dest` matches**. This should be documented so users know they *can* override a base `host_files` entry (and that doing so replaces the base entry).

### Issue Context
Current guide table entry:
- `skills`, `plugins`, `host_files` | Concatenated

Actual behavior:
- Base and child lists are combined, but if any base and child entries share the same `dest`, the child entry replaces the base one.

### Fix Focus Areas
- docs/guides/user/bring-your-own-agent.md[224-233]

### Suggested change
Update the merge-rules row to something like:
- `host_files` | Concatenated; **dedup by `dest` (child wins)**

Optionally add a 1–2 sentence note/example showing how to override a base `host_files` mount by reusing the same `dest`.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

6. Mixed admin/user audience 📜 Skill insight ⌂ Architecture

Description

Although placed under docs/guides/user/, the guide includes administrator/operator-focused content
(per-org configuration and standalone mint identity management). This likely mixes audiences within
a single guide, contrary to the single-audience requirement.

Code

docs/guides/user/bring-your-own-agent.md[R407-425]

+### Per-org config
+
+```yaml
+version: "1"
+dispatch:
+  platform: github
+defaults:
+  roles: [triage, coder, review, my-agent]
+agents:
+  - https://raw.githubusercontent.com/fullsend-ai/agents/<sha>/harness/triage.yaml#sha256=abc...
+  - name: my-cool-agent
+    source: harness/my-cool-agent.yaml
+allowed_remote_resources:
+  - https://raw.githubusercontent.com/fullsend-ai/fullsend/
+  - https://raw.githubusercontent.com/fullsend-ai/agents/
+repos:
+  my-repo:
+    enabled: true
+```

Relevance

⭐ Low

Guide audience/directory placement enforcement has been rejected before (relocation request rejected
in fullsend-ai/fullsend#4901).

PR-#4901

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

The compliance rule requires guides to target a single audience and live in the corresponding
directory. The document includes per-org configuration and standalone mint identity setup content
that is typically admin/operator-oriented, while the file is under user/.

docs/guides/user/bring-your-own-agent.md[407-425]
docs/guides/user/bring-your-own-agent.md[459-473]
Skill: writing-user-docs

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The guide appears to target both repository developers and platform/admin operators in one document, while being located under `docs/guides/user/`.

## Issue Context
Compliance requires each guide to target a single audience (administrators or developers) and be placed accordingly.

## Fix Focus Areas
- docs/guides/user/bring-your-own-agent.md[407-425]
- docs/guides/user/bring-your-own-agent.md[459-473]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

7. slug undefined on first use ✓ Resolved 📜 Skill insight ✧ Quality

Description

The term slug is used in examples before it is defined or linked to the glossary on first
occurrence. This violates the requirement to define jargon on first use via glossary link or inline
definition.

Code

docs/guides/user/bring-your-own-agent.md[R41-48]

+**`harness/my-agent.yaml`:**
+```yaml
+agent: agents/my-agent.md
+image: ghcr.io/fullsend-ai/fullsend-sandbox:latest
+policy: https://raw.githubusercontent.com/fullsend-ai/agents/<sha>/policies/triage.yaml#sha256=abc...
+role: my-agent
+slug: fullsend-ai-coder
+timeout_minutes: 15

Relevance

⭐ Low

Similar “define term on first use” doc feedback was rejected (e.g., define env.runner first use in
#2756).

PR-#2756

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

The rule requires defining jargon on first use. The guide first introduces slug in the minimal
harness example, but only later explains it in the harness field reference section.

docs/guides/user/bring-your-own-agent.md[41-49]
docs/guides/user/bring-your-own-agent.md[140-142]
Skill: writing-user-docs

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`slug` appears in the first harness example without an inline definition or glossary link at first occurrence.

## Issue Context
Compliance requires jargon to be defined on first use via a link to `docs/glossary.md` or an inline definition.

## Fix Focus Areas
- docs/guides/user/bring-your-own-agent.md[41-49]
- docs/guides/user/bring-your-own-agent.md[140-143]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[fullsend-ai-review]

Review

Findings

Low

• [technical-accuracy] docs/cli/agent.md:8 — The introduction states "Authentication is via gh CLI or GH_TOKEN environment variable." The actual resolveToken() function also checks GITHUB_TOKEN as a fallback. This phrasing matches the pre-existing convention in docs/cli/README.md, so it is consistent within the documentation set.

Previous run

Review

Findings

Low

• [technical-accuracy] docs/cli/agent.md:8 — The introduction states "Authentication is via gh CLI or GH_TOKEN environment variable." The actual resolveToken() function also checks GITHUB_TOKEN as a fallback. This phrasing matches the pre-existing convention in docs/cli/README.md, so it is consistent within the documentation set.

• [edge-case] docs/guides/user/bring-your-own-agent.md:427 — The harness field reference documents the role field as "lowercase letter first, then a-z, 0-9, _, -" but omits that the validation also rejects double hyphens (--). A user following the documented character set could create a role like my--agent and hit an unexpected validation error.
  Remediation: Change the role comment to: # Role name (lowercase letter first, then a-z, 0-9, _, -; no double hyphens)

• [missing-authorization] — Non-trivial documentation PR (700+ lines across 12 files) with no linked issue. The PR body mentions "Ralph's feedback" suggesting verbal authorization, but no issue or discussion is linked for traceability.

• [missing-cross-reference] docs/guides/user/customizing-with-skills.md:167 — The See also section lacks a reference to the new Bring Your Own Agent guide. The BYOA guide covers agent registration and base composition closely related to skill customization.
  Remediation: Add - [Bring Your Own Agent](bring-your-own-agent.md) to the See also section.

• [missing-cross-reference] docs/guides/user/customizing-agents.md:498 — The See Also section lacks a reference to the new BYOA guide. The BYOA guide's introduction links to customizing-agents.md, but the reverse link is missing.
  Remediation: Add - [Bring Your Own Agent](bring-your-own-agent.md) to the See Also section.

• [missing-cross-reference] docs/ADRs/0058-agent-registration.md:85 — The References section lacks a cross-reference to the BYOA guide, which is the primary user-facing documentation for the agent registration feature this ADR describes.
  Remediation: Add - [Bring Your Own Agent](../guides/user/bring-your-own-agent.md) to the References section.

Previous run (2)

Review

Findings

Medium

• [stale-reference] website/.vitepress/config.ts:229 — The VitePress sidebar retains the old "Building custom agents from scratch" entry without any deprecation indicator, while the PR adds a new "Bring Your Own Agent" entry nearby. By contrast, docs/guides/README.md annotates the old entry with _(deprecated — see Bring Your Own Agent)_. This inconsistency means the docs site sidebar presents both entries as equally current, with no signal that one supersedes the other.
  Remediation: Either add a "(deprecated)" suffix to the sidebar text for the old entry (e.g., text: "Building custom agents (deprecated)"), or remove the old entry from the sidebar entirely (the page still exists for existing links, just not prominently listed).

Low

• [stale-link-text] docs/guides/infrastructure/private-repositories.md:197 — Link text "Configuring agents" does not match the actual document title "Customizing Agents". The PR updates other instances of this link text to "Customizing agents" to match the actual document title, but this reference in private-repositories.md was not updated.
  Remediation: Change the link text from [Configuring agents](../user/customizing-agents.md) to [Customizing agents](../user/customizing-agents.md).

• [technical-accuracy] docs/cli/agent.md:8 — The introduction states "Authentication is via gh CLI or GH_TOKEN environment variable." The actual resolveToken() function also checks GITHUB_TOKEN. However, this phrasing matches the pre-existing convention used in docs/cli/README.md, so it is consistent within the documentation set.

Previous run (3)

Review

Findings

Low

• [missing-authorization] docs/guides/user/bring-your-own-agent.md — This non-trivial documentation change (549-line new guide + 126-line CLI reference) has no linked issue. The PR body mentions "per Ralph's feedback," indicating authorized work, but the authorization chain is not formally tracked.

Previous run (4)

Review

Findings

Medium

• [missing-reference] docs/agents/README.md:30 — The "Custom Agents" section in the agents README links to ADR 0058 for technical details but does not reference the new "Bring Your Own Agent" guide. Users reading the agents reference would not discover the comprehensive workflow guide added in this PR.
  Remediation: Add a reference to the Bring Your Own Agent guide in the Custom Agents section, e.g., "For a complete guide to building and registering custom agents, see Bring Your Own Agent."

Low

• [missing-authorization] docs/guides/user/bring-your-own-agent.md — This non-trivial documentation change (549-line new guide) has no linked issue. The PR body mentions "per Ralph's feedback," indicating authorized work, but the authorization chain is not formally tracked.

• [scope-creep] docs/guides/user/bring-your-own-agent.md — The PR bundles a terminology shift from "customizing" to "configuring" alongside the new guide. The PR body explains this aligns with default-vs-custom.md terminology per reviewer feedback, and the actual ripple is minimal (link text and one-liner description changes in 4 existing files).

• [link-reference-style] docs/guides/user/bring-your-own-agent.md:84 — References building-custom-agents.md#network-access-via-inline-policies-alternative, which this PR marks as deprecated. New content should avoid depending on deprecated content as the primary reference for a feature. See also: [architectural-drift] finding at this location.
  Remediation: Move the inline policies content into this guide, or reframe the link to make clear the deprecated guide is referenced only for this specific legacy section.

• [architectural-drift] docs/guides/user/bring-your-own-agent.md:84 — Directs users to inline network_policies as the fallback for custom endpoints. While factually correct (ADR 0065 allows inline policies), the framing could better reflect ADR 0065's positioning of providers as the recommended approach. See also: [link-reference-style] finding at this location.
  Remediation: Reframe as "advanced users can fall back to inline network_policies as a workaround, but providers are the recommended architecture (see ADR 0065)."

• [stale-reference] docs/ADRs/0065-provider-backed-policy-composition.md:161 — References "building-custom-agents guide," which is being superseded by "Bring Your Own Agent" in this PR. Note: ADR 0065 has Accepted status, so only minor annotations (cross-references, short notes) are permitted per AGENTS.md ADR immutability rules.
  Remediation: Add a brief annotation noting the guide has been superseded by bring-your-own-agent.md.

Previous run (5)

Review

Findings

Medium

• [documentation-accuracy] docs/guides/user/bring-your-own-agent.md:60 — The providers/vertex-ai.yaml example uses list-of-objects syntax for credentials (- env: ANTHROPIC_VERTEX_PROJECT_ID), but the ProviderDef struct in internal/harness/harness.go defines Credentials as map[string]string. The actual scaffold provider file (internal/scaffold/fullsend-repo/providers/vertex-ai.yaml) uses map syntax (_NOOP_VERTEX_AI: ""). A user following this example would get a YAML unmarshal error at runtime because a YAML sequence cannot be decoded into a Go map[string]string.
  Remediation: Change the credentials block to use map syntax matching the real provider definition format, or add a note that users should copy the file from the scaffold rather than using this example literally.

---

Labels: PR adds and modifies user-facing documentation guides and CLI reference

Previous run (6)

Review

Findings

Low

• [documentation-accuracy] docs/guides/user/bring-your-own-agent.md:148 — The harness field reference comment for role says # Role name (a-z, 0-9, _, -) which lists the allowed character set but omits the constraint that the role must start with a lowercase letter. The actual validation regex is ^[a-z][a-z0-9_-]*$, which rejects roles starting with a digit, underscore, or dash.
  Remediation: Change the comment to # Role name (lowercase letter first, then a-z, 0-9, _, -).

• [link-text-accuracy] docs/cli/agent.md:126 — The See also link uses [Configuring with skills] but the target file is customizing-with-skills.md with title Customizing Agents with Skills. Cross-reference link text should match the referenced document's name.
  Remediation: Change [Configuring with skills] to [Customizing with skills].

• [stale cross-reference] docs/agents/topics/default-vs-custom.md:97 — References building-custom-agents.md in the See also section. This PR deprecates that guide in favor of bring-your-own-agent.md. The link is functional (the deprecated guide has a deprecation banner), but updating the reference would direct readers to the current guide.
  Remediation: Update the reference to point to bring-your-own-agent.md.

• [stale cross-reference] docs/glossary.md:48 — The Custom Agent definition references building-custom-agents.md, which this PR deprecates in favor of bring-your-own-agent.md.
  Remediation: Update the reference to bring-your-own-agent.md.

• [stale cross-reference] docs/guides/infrastructure/standalone-mint.md:356 — References building-custom-agents.md in the See also section, which this PR deprecates in favor of bring-your-own-agent.md.
  Remediation: Update the reference to bring-your-own-agent.md.

Previous run (7)

Review

Findings

Low

• [scope-alignment] docs/guides/user/bring-your-own-agent.md:5 — The new BYOA guide references customizing-agents.md for "harness YAML structure and layered resolution," but both guides now cover harness field reference, base composition, and agent configuration. The overlap is partially mitigated by customizing-agents.md's existing deprecation banner directing readers to config-driven registration, but the relationship between the two guides is not explicitly clarified for readers.
  Remediation: Consider adding a note clarifying which guide is authoritative for which topic in a follow-up.

• [internal-consistency] docs/guides/user/bring-your-own-agent.md:185 — The feedback_mode values documented as "stderr" (default), "stdout", or "exit_code" do not match the feedback_mode: append value used in ADR 0024 and the Go test suite (yaml_semantics_test.go:180). The FeedbackMode field is an unvalidated string in the Go struct (internal/harness/harness.go:210), so any value is accepted at parse time. This inconsistency pre-exists in customizing-agents.md and is propagated here.
  Remediation: Verify which values the runtime consumer supports and align all documentation together.

• [section-naming-consistency] docs/guides/user/bring-your-own-agent.md:522 — The closing section uses ## References instead of ## See also, which is the established pattern across CLI docs and user guides (6 occurrences). The new docs/cli/agent.md added in this same PR correctly uses ## See also.
  Remediation: Change ## References to ## See also to match the established pattern.

Previous run (8)

Review

Findings

Medium

• [scope-alignment] docs/guides/user/bring-your-own-agent.md — The new guide overlaps with the existing customizing-agents.md in harness field reference coverage. building-custom-agents.md is explicitly deprecated per ADR-0064 and redirects to config-driven registration, making BYOA its natural successor — that relationship is clear. However, the relationship between BYOA and customizing-agents.md is not clarified: readers may be unsure which guide to consult for which task.
  Remediation: Add a brief note at the top of the BYOA guide or in the References section explaining how it relates to customizing-agents.md — e.g., "This guide covers the end-to-end workflow for building and registering agents. For details on harness YAML structure and layered resolution, see Customizing Agents."

Low

• [internal-consistency] docs/guides/user/bring-your-own-agent.md:189 — The feedback_mode values documented as "stderr" (default), "stdout", or "exit_code" do not match the feedback_mode: append value used in ADR 0024 and the Go test suite (yaml_semantics_test.go:180). The FeedbackMode field is an unvalidated string. This inconsistency pre-exists in customizing-agents.md:69 and is propagated here.
  Remediation: Verify which values the runtime consumer supports and align all documentation together.

• [internal-consistency] docs/guides/user/bring-your-own-agent.md:168 — The field merge rules table correctly describes host_files as "Concatenated; child overrides by dest" but the table row presentation alongside skills and plugins (simple concatenation) may understate the semantic difference.

• [terminology-consistency] — The guide uses "configuring" (aligned with default-vs-custom.md terminology) while existing guides use "customizing" (customizing-agents.md, customizing-with-skills.md). This is an intentional evolution but creates a terminology gradient across the documentation.

• [stale-cross-reference] docs/guides/README.md:41 — The guides README lists "Building custom agents from scratch" adjacent to the new "Bring Your Own Agent" entry without indicating that the former is deprecated per ADR-0064. The file itself has a deprecation header, but readers scanning the README won't see it.
  Remediation: Annotate the building-custom-agents.md entry in the README with a deprecation note, e.g., "(deprecated — see Bring Your Own Agent)".

Previous run (9)

Review

Findings

Medium

• [API contract violation] docs/guides/user/bring-your-own-agent.md:404 — The per-repo config example includes my-agent in the roles list (roles: [triage, coder, review, my-agent]), but PerRepoConfig.Validate() rejects any role not in the fixed ValidRoles() set (fullsend, triage, coder, review, fix, retro, prioritize, e2e). Same issue in the per-org config example at line 420. A user following these examples would get a validation error.
  Remediation: Remove my-agent from the roles lists in both config examples. Custom agents registered via agents: entries do not need to appear in the roles list.

• [API contract violation] docs/guides/user/bring-your-own-agent.md:418 — The per-org config example uses platform: github in the dispatch: block, but OrgConfig.Validate() requires platform: github-actions. The value github would fail validation.
  Remediation: Change platform: github to platform: github-actions.

• [API contract violation] docs/guides/user/bring-your-own-agent.md:359 — The "add org-specific environment variables" example references harness/coder.yaml in the base URL, but no file named coder.yaml exists in the upstream scaffold — the actual file is code.yaml (internal/scaffold/fullsend-repo/harness/code.yaml). A user following this example would get a 404 when resolving the base URL.
  Remediation: Change harness/coder.yaml to harness/code.yaml in the base URL.

• [missing-doc-navigation] website/.vitepress/config.ts:231 — The new user guide is not listed in the VitePress sidebar configuration. The User Guides sidebar section uses manual entries (not getMarkdownFiles() auto-discovery), so the guide won't appear in the website navigation without a manual entry.
  Remediation: Add { text: "Bring Your Own Agent", link: "/guides/user/bring-your-own-agent" } to the User Guides sidebar items in config.ts.

Low

• [internal-consistency] docs/guides/user/bring-your-own-agent.md:235 — The field merge rules table groups host_files with skills and plugins under "Concatenated", but host_files has distinct dedup-by-Dest semantics: when a child and base both have a host_files entry with the same dest path, the child's entry replaces the base's (per mergeHostFiles() in compose.go). The current wording could mislead users into thinking duplicate dest entries would both appear.
  Remediation: Add a separate row for host_files with "Concatenated; child overrides by dest" or add a footnote explaining the dedup behavior.

• [internal-consistency] docs/guides/user/bring-your-own-agent.md:176 — The feedback_mode documented values ("stderr", "stdout", "exit_code") are consistent with the existing customizing-agents.md guide, but ADR 0024 and tests use feedback_mode: append. The FeedbackMode field is an unvalidated string, so the documented values may not match runtime behavior. This inconsistency predates this PR.
  Remediation: Verify which values the runtime consumer supports and align all documentation (this guide and customizing-agents.md) together.

• [unverifiable-version-claim] docs/guides/user/bring-your-own-agent.md:441 — Version numbers v0.27.0 and v0.30.0+ don't appear anywhere else in the documentation and no matching git tags exist. The existing building-custom-agents.md guide references the same migration command without version numbers.
  Remediation: Remove specific version numbers and use temporal markers instead, or verify these are published versions.

[fullsend-ai-review]

🤖 Review · ⚠️ Cancelled · Started 5:10 PM UTC · Ended 5:17 PM UTC
Commit: cc7a526 · View workflow run →

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 5:18 PM UTC · Completed 5:32 PM UTC
Commit: a97a404 · View workflow run →

Superseded by updated review

[waynesun09]

Review Squad Report — #5255

Agents dispatched: 4 (Claude coder, Claude researcher, Grok, Codex)
Models used: Claude, Grok, Codex
Total findings: 15 (after dedup + verification + UX walkthrough)
False positives removed: 3 (incl. sidebar entry — it IS in the PR)

Methodology

Code review findings from 4 parallel agents, verified against the codebase. UX findings from actually porting the gh-classify agent using this guide — discovered gaps firsthand.

Not in this diff but should be in this PR

• docs/guides/user/building-custom-agents.md — deprecation banner says "register in config.yaml" but doesn't link to this new guide. Add: > For the current guide, see [Bring Your Own Agent](bring-your-own-agent.md).

---

Assisted-by: Claude (review), Grok (review)

[waynesun09]

Review Squad Report — CLI Reference (docs/cli/agent.md)

Agents dispatched: 4 (Claude coder, Claude researcher, Grok, Codex)
Models used: Claude, Grok, Codex
Total findings: 4 MEDIUM, 8 LOW (after dedup + verification)
False positives removed: 0

Positive notes

All 5 subcommands, flags, argument counts, and behavior claims verified against Go source (internal/cli/agent.go, internal/cli/migrate.go). No undocumented subcommands. Override classification table matches code exactly. Security properties (pinning, integrity hashes, path traversal validation) are sound.

LOW findings (not posted inline)

• agent list strips #sha256=... from displayed SOURCE — not documented (user may be confused when comparing to config.yaml)
• agent add validates local path existence — not documented as a constraint
• README description truncates migrate-customizations to just migrate — user might try fullsend agent migrate
• (required) annotation in flag descriptions inconsistent with other CLI docs
• ### Behavior heading pattern unique to this doc — other CLI docs use inline prose
• agent list output format not shown with example
• GitHub blob URL → raw URL transformation not detailed
• integrity hash vs SHA-256 terminology inconsistency (line 21 vs 37)

---

Assisted-by: Claude (review), Grok (review), Codex (review)

[fullsend-ai-review]

🤖 Review · ⚠️ Cancelled · Started 7:19 PM UTC · Ended 7:29 PM UTC
Commit: cc7a526 · View workflow run →

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 7:30 PM UTC · Completed 7:42 PM UTC
Commit: e4fbf6d · View workflow run →

[fullsend-ai-review]

🤖 Finished Review · ❌ Failure · Started 2:13 PM UTC · Completed 2:29 PM UTC
Commit: 69c5d53 · View workflow run →

[fullsend-ai-review]

🤖 Review · ⚠️ Cancelled · Started 2:34 PM UTC · Ended 2:43 PM UTC
Commit: 3d48dce · View workflow run →

[waynesun09]

I followed this guide literally end-to-end (created the exact files "Minimum viable agent" shows, ran the exact command "Testing locally" shows, in a scratch environment) rather than just reading it. A reader following it top-to-bottom will not succeed — three compounding blockers, each confirmed by actual reproduction. Inline comments mark where each one lives.

There's also a fourth, more significant finding that isn't part of this diff but is directly related: it turns out fullsend-ai/agents (the repo running-agents-locally.md now points --fullsend-dir at, per the just-merged #5287) has no profiles/ directory either — meaning the same "unsupported provider type" failure described in the third inline comment below will also hit anyone following the default agents locally with real GCP credentials, once they get past pre-script. I'll open a separate issue for that since it's not this PR's fault, but flagging here since it's the same root cause this PR's example surfaces.

Otherwise this is a strong, well-structured guide — the harness field reference, merge-rules table, and cross-reference updates all check out against the actual Go source.

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 2:43 PM UTC · Completed 2:57 PM UTC
Commit: bce12c3 · View workflow run →

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 3:04 PM UTC · Completed 3:20 PM UTC
Commit: 162cd3a · View workflow run →

[waynesun09]

Re-verified all 3 findings end-to-end after the update, not just re-reading the diff.

• Directory naming: .fullsend/ now used consistently in the "Minimum viable agent" tree and every --fullsend-dir example — no more mismatch.
• fullsend init: removed, correctly replaced with fullsend github setup.
• Missing profiles/: added profiles/fullsend-vertex-ai.yaml with real content — diffed it against the actual scaffold file, byte-identical except the cosmetic leading --- marker.

Rebuilt the exact updated "Minimum viable agent" file tree in a scratch dir and ran the exact "Testing locally" command literally. It now proceeds through harness load → openshell/gateway checks → provider creation succeeds (✓ Provider ready: vertex-ai) → sandbox bootstrap. That's the full chain that was broken before.

Approving.

[fullsend-ai-retro]

🤖 Finished Retro · ✅ Success · Started 3:51 PM UTC · Completed 4:01 PM UTC
Commit: 162cd3a · View workflow run →

[fullsend-ai-retro]

PR #5255 was a substantial docs-only PR (15 files, +719/-15 lines) adding a Bring Your Own Agent guide and CLI reference. It went through 13 review agent runs over ~22 hours, consuming approximately 187 minutes of agent compute. The review agent consistently produced low/medium severity findings (config validation mismatches, link consistency, terminology issues) but never surfaced a HIGH finding. Meanwhile, human reviewer waynesun09 found 16 HIGH and 27 MEDIUM findings using a multi-agent 'review squad' methodology plus hands-on reproduction, including security-relevant issues (production slug reuse in examples), non-existent CLI commands, and missing prerequisite files. The most critical blocking issues were only discoverable by literally following the guide's steps. The review agent did catch some valid issues the human missed (config validation regex details, platform field values), demonstrating complementary value.

Three proposals target the most impactful gaps:

1. Add source-code cross-validation for documentation examples in the docs-currency sub-agent
2. Elevate budget priority for docs-focused sub-agents on docs-only PRs
3. Calibrate documentation finding severity to user-facing impact

Related existing issues: #1480 (shallow docs feedback), #5157 (scale detail to complexity), #1900 (fast path for trivial docs changes).

Proposals filed

• Review docs-currency sub-agent should cross-validate documentation examples against actual source files (in fullsend-ai/fullsend)
• Review orchestrator should promote docs sub-agent budget priority for docs-only PRs (in fullsend-ai/fullsend)
• Review agent severity guidance should escalate documentation findings that block users or cause security confusion (in fullsend-ai/fullsend)