From 162cd3ad92a6023b4ee85cb606b7bb0d0c090572 Mon Sep 17 00:00:00 2001 From: Greg Allen Date: Sat, 18 Jul 2026 11:03:33 -0400 Subject: [PATCH] docs: add Bring Your Own Agent guide and agent CLI reference Signed-off-by: Claude Signed-off-by: Greg Allen --- docs/ADRs/0058-agent-registration.md | 1 + ...0065-provider-backed-policy-composition.md | 3 +- docs/agents/README.md | 3 +- docs/agents/topics/default-vs-custom.md | 4 +- docs/cli/README.md | 2 +- docs/cli/agent.md | 126 ++++ docs/glossary.md | 2 +- docs/guides/README.md | 3 +- .../infrastructure/private-repositories.md | 2 +- docs/guides/infrastructure/standalone-mint.md | 2 +- docs/guides/user/bring-your-own-agent.md | 569 ++++++++++++++++++ docs/guides/user/building-custom-agents.md | 11 +- docs/guides/user/customizing-agents.md | 1 + docs/guides/user/customizing-with-skills.md | 1 + website/.vitepress/config.ts | 4 +- 15 files changed, 719 insertions(+), 15 deletions(-) create mode 100644 docs/cli/agent.md create mode 100644 docs/guides/user/bring-your-own-agent.md diff --git a/docs/ADRs/0058-agent-registration.md b/docs/ADRs/0058-agent-registration.md index aef2d469c4..5e31274db8 100644 --- a/docs/ADRs/0058-agent-registration.md +++ b/docs/ADRs/0058-agent-registration.md @@ -88,4 +88,5 @@ phasing, schema details, CLI behavior, and migration mechanics. - [ADR 0038](0038-universal-harness-access.md) -- URL-based resource references and integrity hashes - [ADR 0045](0045-forge-portable-harness-schema.md) -- harness composition via `base:` URLs - [ADR 0057](0057-repos-management.md) -- repos management for per-repo installations +- [Bring Your Own Agent](../guides/user/bring-your-own-agent.md) -- user-facing guide for agent registration - [Implementation plan](../plans/agent-registration.md) diff --git a/docs/ADRs/0065-provider-backed-policy-composition.md b/docs/ADRs/0065-provider-backed-policy-composition.md index d84205bd8a..f3052b17ae 100644 --- a/docs/ADRs/0065-provider-backed-policy-composition.md +++ b/docs/ADRs/0065-provider-backed-policy-composition.md @@ -159,7 +159,8 @@ automatically by `fullsend run`. policies continue to work, but the scaffold and guides recommend providers as the preferred approach. - The building-custom-agents guide is updated to show providers as - the recommended way to grant network access. + the recommended way to grant network access. (Note: this guide has + been superseded by [Bring Your Own Agent](../guides/user/bring-your-own-agent.md).) - Requires OpenShell >= v0.0.37 and the `providers_v2_enabled` gateway setting (set automatically). - GitHub access is split into read-write (`fullsend-github`) and diff --git a/docs/agents/README.md b/docs/agents/README.md index e7bd086be0..fda992a849 100644 --- a/docs/agents/README.md +++ b/docs/agents/README.md @@ -34,4 +34,5 @@ your org-level or per-repo `config.yaml`. Each entry is either a local path (relative to the fullsend directory) or a pinned HTTPS URL with an integrity hash. Config-registered agents override scaffold defaults when names collide (case-insensitive). See [ADR 0058](../ADRs/0058-agent-registration.md) for -details. +details and [Bring Your Own Agent](../guides/user/bring-your-own-agent.md) +for the complete guide to building and registering custom agents. diff --git a/docs/agents/topics/default-vs-custom.md b/docs/agents/topics/default-vs-custom.md index ff99099a08..82c799bf7c 100644 --- a/docs/agents/topics/default-vs-custom.md +++ b/docs/agents/topics/default-vs-custom.md @@ -94,7 +94,7 @@ for a stated purpose, that override does not make the agent derived. — project-wide instructions for all agents - [Customizing with skills](../../guides/user/customizing-with-skills.md) — extending or replacing built-in skills -- [Building custom agents](../../guides/user/building-custom-agents.md) — - creating a new agent from scratch +- [Bring Your Own Agent](../../guides/user/bring-your-own-agent.md) — + building custom agents and configuring existing ones - [ADR 0045](../../ADRs/0045-forge-portable-harness-schema.md) — `base` composition and harness inheritance diff --git a/docs/cli/README.md b/docs/cli/README.md index f392112428..4e93465c63 100644 --- a/docs/cli/README.md +++ b/docs/cli/README.md @@ -14,6 +14,7 @@ Download the latest binary from [GitHub Releases](https://github.com/fullsend-ai | Command group | Description | |--------------|-------------| +| [`fullsend agent`](agent.md) | Manage agent registrations — add, list, update, remove, and migrate-customizations | | [`fullsend github`](github.md) | Configure GitHub orgs and repos — setup, enrollment, day-2 operations | | [`fullsend inference`](inference.md) | Manage GCP Workload Identity Federation for Agent Platform access | | [`fullsend mint`](mint.md) | Deploy and manage the OIDC token mint service | @@ -24,7 +25,6 @@ Download the latest binary from [GitHub Releases](https://github.com/fullsend-ai | Command | Description | |---------|-------------| | `fullsend run` | Execute an agent locally in a sandbox. See [running agents locally](../guides/user/running-agents-locally.md). | -| `fullsend agent` | Manage agent registrations in config. Subcommands: `add`, `list`, `update`, `remove` (CRUD) and `migrate-customizations` (one-time migration of `customized/` overrides to config-driven agents per ADR-0064) | | `fullsend lock [agent-name]` | Pin remote dependencies to `lock.yaml` | | `fullsend scan` | Run security scanners on agent input/output | diff --git a/docs/cli/agent.md b/docs/cli/agent.md new file mode 100644 index 0000000000..7e051dbda7 --- /dev/null +++ b/docs/cli/agent.md @@ -0,0 +1,126 @@ +--- +sidebar_label: fullsend agent +--- + +# fullsend agent + +Manage agent registrations in fullsend config. Add, list, update, and remove agents, or migrate legacy `customized/` overrides to config-driven agents. + +`agent add`, `agent update`, and `agent migrate-customizations` (non-dry-run) fetch remote content and resolve GitHub URLs. Authentication is via `gh` CLI or `GH_TOKEN` environment variable. + +## Commands + +| Command | Description | +|---------|-------------| +| `fullsend agent add ` | Register an agent in config | +| `fullsend agent list` | List registered agents | +| `fullsend agent update [sha]` | Update a URL agent to a new commit SHA | +| `fullsend agent remove ` | Remove an agent from config | +| `fullsend agent migrate-customizations` | Migrate `customized/` overrides to config-driven agents | + +## `agent add` + +Register an agent in config by URL or local path. URL sources are automatically pinned to a specific commit SHA and annotated with a `#sha256=...` integrity hash. The URL prefix is added to `allowed_remote_resources` if not already present. + +```bash +fullsend agent add https://github.com/my-org/agents/blob/main/harness/lint.yaml --fullsend-dir .fullsend +fullsend agent add harness/custom-review.yaml --name my-review --fullsend-dir .fullsend +``` + +### Flags + +| Flag | Default | Description | +|------|---------|-------------| +| `--fullsend-dir` | | Base directory containing the `.fullsend` layout (required) | +| `--name` | derived from filename | Explicit agent name | + +GitHub blob URLs are resolved to pinned `raw.githubusercontent.com` URLs. Non-GitHub URLs must already contain a commit SHA in the path. Local paths must be relative, must not contain path traversal (`..`), and the file must exist. If an agent with the same name already exists, the command fails. + +## `agent list` + +List all agents registered in config, showing each agent's name and source. + +```bash +fullsend agent list --fullsend-dir .fullsend +``` + +### Flags + +| Flag | Default | Description | +|------|---------|-------------| +| `--fullsend-dir` | | Base directory containing the `.fullsend` layout (required) | + +Read-only. Displays a table with `NAME` and `SOURCE` columns. For URL agents, the `#sha256=...` integrity hash suffix is stripped from the displayed source for readability. + +Example output: +``` +NAME SOURCE +triage https://raw.githubusercontent.com/fullsend-ai/agents/abc123/harness/triage.yaml +my-lint harness/my-lint.yaml +``` + +## `agent update` + +Update a URL-based agent to a new commit SHA and recompute the `#sha256=...` integrity hash. If no SHA is provided, the default branch HEAD is resolved automatically. + +```bash +fullsend agent update triage --fullsend-dir .fullsend +fullsend agent update triage a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4e5f6a1b2 --fullsend-dir .fullsend +``` + +### Flags + +| Flag | Default | Description | +|------|---------|-------------| +| `--fullsend-dir` | | Base directory containing the `.fullsend` layout (required) | + +Only URL agents can be updated — local path agents have nothing to pin. Non-GitHub URL agents require an explicit SHA argument. The integrity hash is recomputed by fetching the content at the new SHA. + +## `agent remove` + +Remove an agent from config. If the removed agent was the last one using a given `allowed_remote_resources` prefix, that prefix is also cleaned up. + +```bash +fullsend agent remove triage --fullsend-dir .fullsend +``` + +### Flags + +| Flag | Default | Description | +|------|---------|-------------| +| `--fullsend-dir` | | Base directory containing the `.fullsend` layout (required) | + +## `agent migrate-customizations` + +Scan the `customized/` directory and migrate each override to a config-driven agent. Changes are committed to a branch and delivered via pull request. Use `--dry-run` to preview changes without creating a PR. + +```bash +fullsend agent migrate-customizations --fullsend-dir .fullsend --dry-run +fullsend agent migrate-customizations --fullsend-dir .fullsend --repo owner/repo +``` + +### Flags + +| Flag | Default | Description | +|------|---------|-------------| +| `--fullsend-dir` | | Base directory containing the `.fullsend` layout (required) | +| `--repo` | | Target repository (`owner/repo`) for the migration PR (required unless `--dry-run`) | +| `--dry-run` | `false` | Show what would change without creating a PR | + +### Override classification + +The tool classifies each override and takes the appropriate action: + +| Override type | Detection | Action | +|---------------|-----------|--------| +| Dead | Agent already registered in config | Delete `customized/` files | +| Custom | Not in upstream scaffold | Move files to regular directories, register local path in config | +| Modified | Standard scaffold agent, not yet in config | Generate a `base:` composition harness with the minimal diff, register in config | + +For modified agents, the migration produces a thin `base:` harness containing only the fields that differ from upstream. + +## See also + +- [Bring Your Own Agent](../guides/user/bring-your-own-agent.md) — building custom agents and configuring existing ones +- [Default, derived, and custom agents](../agents/topics/default-vs-custom.md) — terminology and classification +- [Customizing with skills](../guides/user/customizing-with-skills.md) — extending agents with skills diff --git a/docs/glossary.md b/docs/glossary.md index 1c8130a701..e21164a25f 100644 --- a/docs/glossary.md +++ b/docs/glossary.md @@ -45,7 +45,7 @@ See [Default, derived, and custom agents](agents/topics/default-vs-custom.md). ### Custom Agent An agent whose `base` chain does not trace back to a default agent harness in `fullsend-ai/fullsend`, or that has no `base` at all. A custom agent is built from scratch, even if it happens to resemble a default agent. Contrast with [derived agent](#derived-agent), which starts from a default. -See [Default, derived, and custom agents](agents/topics/default-vs-custom.md) and [Building custom agents](guides/user/building-custom-agents.md). +See [Default, derived, and custom agents](agents/topics/default-vs-custom.md) and [Bring Your Own Agent](guides/user/bring-your-own-agent.md). ## D diff --git a/docs/guides/README.md b/docs/guides/README.md index a65cfc5092..18ec9a8961 100644 --- a/docs/guides/README.md +++ b/docs/guides/README.md @@ -37,7 +37,8 @@ Guides for developers working in repositories where fullsend is active. - [Customizing agents](user/customizing-agents.md) — Harness configurations and layered content resolution for your org and repos - [Customizing with AGENTS.md](user/customizing-with-agents-md.md) — Guide agents using your repo's AGENTS.md file - [Customizing with skills](user/customizing-with-skills.md) — Extend or replace built-in agent skills with custom skill documents -- [Building custom agents from scratch](user/building-custom-agents.md) — Create a new agent from scratch on a per-repo fullsend installation +- [Bring Your Own Agent](user/bring-your-own-agent.md) — Add a custom agent or configure an existing one, from harness file to CI +- [Building custom agents from scratch](user/building-custom-agents.md) — _(deprecated — see [Bring Your Own Agent](user/bring-your-own-agent.md))_ - [Default, derived, and custom agents](../agents/topics/default-vs-custom.md) — When configuration crosses into derived or custom agent territory ## Development diff --git a/docs/guides/infrastructure/private-repositories.md b/docs/guides/infrastructure/private-repositories.md index 3f9c4b69f7..043ce30dfc 100644 --- a/docs/guides/infrastructure/private-repositories.md +++ b/docs/guides/infrastructure/private-repositories.md @@ -194,6 +194,6 @@ Not all private repos are equal. A repo containing open-source code that happens ## See also - [Getting Started](../getting-started/) — Initial fullsend setup -- [Configuring agents](../user/customizing-agents.md) — Harness configuration and layered overrides +- [Customizing agents](../user/customizing-agents.md) — Harness configuration and layered overrides - [Security threat model](../../problems/security-threat-model.md) — Threat priority and defense considerations - [#1189](https://github.com/fullsend-ai/fullsend/issues/1189) — Retro agent private content leak risk diff --git a/docs/guides/infrastructure/standalone-mint.md b/docs/guides/infrastructure/standalone-mint.md index f5574fbd46..2c40599860 100644 --- a/docs/guides/infrastructure/standalone-mint.md +++ b/docs/guides/infrastructure/standalone-mint.md @@ -353,5 +353,5 @@ gh api -X POST /orgs/myorg/actions/variables \ ## See also - [Mint service administration](mint-administration.md) — Managing the hosted GCP mint -- [Building custom agents](../user/building-custom-agents.md) — Creating new agents for your repos +- [Bring Your Own Agent](../user/bring-your-own-agent.md) — Building custom agents and configuring existing ones - [Getting Started](../getting-started/) — End-user setup diff --git a/docs/guides/user/bring-your-own-agent.md b/docs/guides/user/bring-your-own-agent.md new file mode 100644 index 0000000000..1690947539 --- /dev/null +++ b/docs/guides/user/bring-your-own-agent.md @@ -0,0 +1,569 @@ +# Bring Your Own Agent + +Add a custom agent to fullsend — or change the configuration of an existing one — from harness file to CI. + +This guide covers the end-to-end workflow for building and registering agents. For details on harness YAML structure and layered resolution, see [Customizing agents](customizing-agents.md). + +This guide uses the [fullsend-ai/agents](https://github.com/fullsend-ai/agents) triage agent as a running example. + +## How agents work + +A fullsend agent has two parts: + +1. **Harness file** (YAML) — _how_ the agent runs: sandbox image, policy, scripts, skills, credentials, timeouts. +2. **Agent definition** (Markdown) — _what_ the agent does: prompt, tools, model, skills. + +The harness is the entry point. `fullsend run triage` reads the harness, provisions a sandbox, and launches the agent inside it. + +``` +fullsend run triage + │ + ▼ +┌── harness/triage.yaml ─────────┐ +│ agent: agents/triage.md │ ◄── prompt & tools +│ policy: policies/base.yaml │ ◄── sandbox rules +│ skills: [issue-labels] │ ◄── domain knowledge +│ pre_script: scripts/pre-... │ ◄── fetch data (before sandbox) +│ post_script: scripts/post-... │ ◄── act on output (after sandbox) +└─────────────────────────────────┘ +``` + +**Security model:** agents run inside a sandboxed environment. The sandbox policy enforces filesystem access, landlock, and process identity. Network access is typically managed via **provider profiles** (YAML files in a `providers/` directory) referenced by name in the harness `providers:` list — the scaffold's shared `policies/base.yaml` contains no network rules, since built-in agents use providers ([ADR 0065](../../ADRs/0065-provider-backed-policy-composition.md)). Custom agents can also use inline `network_policies` in a per-agent policy file if providers don't cover their needs. Pre-scripts run on the trusted runner _before_ the sandbox starts; post-scripts run _after_ it exits. + +## Minimum viable agent + +You need a harness, an agent definition, and supporting scaffold files. If your repo was set up with `fullsend github setup`, the `.fullsend/` directory already contains `policies/`, `providers/`, and `profiles/` from the scaffold — you only need to add `harness/my-agent.yaml` and `agents/my-agent.md`. For a standalone agent repo, copy the scaffold files or create the full layout: + +``` +.fullsend/ +├── harness/my-agent.yaml # Execution config (you create) +├── agents/my-agent.md # Agent prompt (you create) +├── providers/vertex-ai.yaml # Provider definition (from scaffold) +├── profiles/fullsend-vertex-ai.yaml # Profile definition (from scaffold) +└── policies/base.yaml # Sandbox policy (from scaffold) +``` + +**`harness/my-agent.yaml`:** +```yaml +agent: agents/my-agent.md +image: ghcr.io/fullsend-ai/fullsend-sandbox:latest # Pin to a digest before CI use +policy: policies/base.yaml +providers: + - vertex-ai +role: my-agent +slug: my-org-my-agent # GitHub App identity; convention: - (see Advanced: custom identity) +timeout_minutes: 15 +``` + +**`providers/vertex-ai.yaml`** — provider definition (declares a provider by name and type): +```yaml +name: vertex-ai +type: fullsend-vertex-ai +credentials: + _NOOP_VERTEX_AI: "" +``` + +**`profiles/fullsend-vertex-ai.yaml`** — profile definition (tells OpenShell what endpoints the `fullsend-vertex-ai` type grants access to). Copy this from the scaffold or [fullsend-ai/agents](https://github.com/fullsend-ai/agents): +```yaml +id: fullsend-vertex-ai +display_name: Fullsend Vertex AI +description: Anthropic API and Google Cloud APIs for inference +category: inference +endpoints: + - host: api.anthropic.com + port: 443 + protocol: rest + access: read-write + enforcement: enforce + - host: "*.googleapis.com" + port: 443 + protocol: rest + access: read-write + enforcement: enforce +binaries: + - "**/claude" + - "**/node" +``` + +> **Prerequisite (CI only):** for agents running in GitHub Actions, your org or repo must be provisioned for GCP Workload Identity Federation — run [`fullsend inference provision`](../../cli/inference.md) first. The provider profile above controls network access only; real credentials are delivered via `host_files` (see [real-world example](#real-world-example-the-triage-agent)). + +**`agents/my-agent.md`:** +````markdown +--- +name: my-agent +description: One-line description of what this agent does. +tools: Bash(gh,jq) +model: opus +--- + +You are my-agent. Your job is to [task description]. + +## Steps +1. Fetch input from environment variables +2. Analyze and process +3. Write JSON result to `$FULLSEND_OUTPUT_DIR/agent-result.json` + +Do NOT push code, create issues, or modify anything directly. +Your only output is the JSON result file. +```` + +Network access (which APIs the agent can reach) is controlled by provider profiles or inline `network_policies`. The six built-in profiles (`vertex-ai`, `github`, `github-ro`, `github-artifacts`, `gitleaks`, `package-registries`) use framework-known `type` values (e.g. `fullsend-vertex-ai`, `fullsend-github`). To define a fully custom provider type, reference a remote provider definition together with a matching `openshell.profiles` entry (see [Remote Providers and Profiles](customizing-agents.md#remote-providers-and-profiles) and [ADR 0070](../../ADRs/0070-portable-provider-profile-resolution.md)). For endpoints not covered by providers, inline `network_policies` in the policy YAML also work. Providers are the pattern used by fullsend's built-in agents ([ADR 0065](../../ADRs/0065-provider-backed-policy-composition.md)), but custom agents can use whichever approach fits. + +**Next step:** to get your agent running immediately, skip ahead to [Testing locally](#testing-locally) → [Registering your agent](#registering-your-agent). The sections below provide deeper reference material. + +## Real-world example: the triage agent + +The [fullsend-ai/agents](https://github.com/fullsend-ai/agents) triage agent is a full production agent. The harness below is adapted from the current [`harness/triage.yaml`](https://github.com/fullsend-ai/agents/blob/main/harness/triage.yaml) (field order adjusted for readability): + +```yaml +agent: agents/triage.md +doc: docs/triage.md +model: opus +image: ghcr.io/fullsend-ai/fullsend-sandbox:latest +policy: policies/triage.yaml + +role: triage +slug: fullsend-ai-triage + +host_files: + - src: common/env/gcp-vertex.env + dest: /sandbox/workspace/.env.d/gcp-vertex.env + expand: true + - src: ${GOOGLE_APPLICATION_CREDENTIALS} + dest: /tmp/.gcp-credentials.json + - src: ${GCP_OIDC_TOKEN_FILE} + dest: /sandbox/workspace/.gcp-oidc-token + optional: true + - src: env/triage.env + dest: /sandbox/workspace/.env.d/triage.env + expand: true + +skills: + - skills/issue-labels + +pre_script: scripts/pre-triage.sh +post_script: scripts/post-triage.sh + +validation_loop: + script: scripts/validate-output-schema.sh + schema: schemas/triage-result.schema.json + max_iterations: 2 + +timeout_minutes: 10 + +forge: + github: + pre_script: scripts/pre-triage.sh + post_script: scripts/post-triage.sh + env: + runner: + GITHUB_ISSUE_URL: ${GITHUB_ISSUE_URL} + GH_TOKEN: ${GH_TOKEN} + sandbox: + GITHUB_ISSUE_URL: "${GITHUB_ISSUE_URL}" + GH_TOKEN: "${GH_TOKEN}" +``` + +Key patterns to note: + +- **`policy: policies/triage.yaml`** is a per-agent policy that includes filesystem, landlock, process, and network rules (via inline `network_policies`). This agent predates the provider-based pattern — new agents can use `providers:` instead (see [Minimum viable agent](#minimum-viable-agent)). +- **`host_files`** copy credentials from the trusted runner into the sandbox. `expand: true` resolves `${VAR}` references before copying. +- **`validation_loop.schema`** references the JSON schema file directly — the validation script checks agent output against it. +- **`forge.github`** scopes scripts and env vars to GitHub. When running on GitLab, a `forge.gitlab` block would take effect instead. +- **`common/env/gcp-vertex.env`** is referenced by relative path because both files live in the same repo. If your agent lives in a different repo, reference it by URL (see [Remote references](#referencing-resources-local-vs-remote)) or copy it locally. + +## Harness field reference + +```yaml +# ── Required ────────────────────────────────────────────────── +agent: agents/my-agent.md # Path to agent definition +role: my-agent # Role name (lowercase letter first, then a-z, 0-9, _, -; no double hyphens) + +# ── Identity & metadata ────────────────────────────────────── +slug: my-org-my-role # GitHub App identity (convention: -) +description: One-line summary # Human-readable description +doc: docs/agents/my-agent.md # Source-repo-only; not resolved at runtime +trigger: "event.type == 'issue'" # Optional CEL expression over normevent (ADR 0061) + +# ── Composition ─────────────────────────────────────────────── +base: harness/common-base.yaml # Inherit from another harness (local or URL) + +# ── Sandbox ─────────────────────────────────────────────────── +image: ghcr.io/fullsend-ai/fullsend-sandbox:latest +policy: policies/base.yaml # Sandbox policy (filesystem, landlock, process) +model: opus # LLM model override +readonly_repo: false # Mount repo as read-only in sandbox +providers: # Network access via provider profiles (ADR 0065) + - vertex-ai # References providers/vertex-ai.yaml + - github # References providers/github.yaml + +# ── Skills & plugins ────────────────────────────────────────── +skills: + - skills/my-skill # Local path or URL with #sha256=... +plugins: + - plugins/gopls-lsp +openshell: # OpenShell sandbox profiles + profiles: + - https://example.com/profile.yaml#sha256=abc... + +# ── Scripts (local paths only) ──────────────────────────────── +pre_script: scripts/pre-my-agent.sh +post_script: scripts/post-my-agent.sh +agent_input: inputs/my-input.md # File passed as initial input to the agent + +# ── Validation ──────────────────────────────────────────────── +validation_loop: + script: scripts/validate-output-schema.sh + max_iterations: 2 + feedback_mode: stderr # How validation feedback reaches the agent + +# ── Host files ──────────────────────────────────────────────── +host_files: + - src: env/my-agent.env # Runner path (supports ${VAR}) + dest: /sandbox/workspace/.env.d/my-agent.env + expand: true # Resolve ${VAR} in contents + - src: ${SOME_CREDENTIAL} + dest: /tmp/.cred.json + optional: true # Skip if missing + +# ── Environment ─────────────────────────────────────────────── +env: + runner: # Available to pre/post scripts + MY_VAR: "${MY_VAR}" + sandbox: # Available inside sandbox + MY_SETTING: "value" +runner_env: # Legacy (same as env.runner) + MY_VAR: "${MY_VAR}" + +# ── Timeouts ────────────────────────────────────────────────── +timeout_minutes: 20 +sandbox_timeout_seconds: 300 # 30-600 + +# ── Remote resources ────────────────────────────────────────── +allowed_remote_resources: + - https://github.com/my-org/agent-library/ +allow_runtime_fetch: true +max_runtime_fetches: 10 + +# ── API servers ─────────────────────────────────────────────── +api_servers: # Host-side REST proxies exposed to sandbox + - name: my-api + script: scripts/api-server.sh # Local script that runs the server + port: 8080 # Port the sandbox connects to + env: # Env vars for the server process + API_KEY: "${API_KEY}" + +# ── Forge-specific overrides ────────────────────────────────── +forge: + github: + pre_script: scripts/pre-gh.sh + post_script: scripts/post-gh.sh + skills: [skills/github-specific] # Concatenated with top-level + env: + runner: + GH_TOKEN: "${GH_TOKEN}" + gitlab: + pre_script: scripts/pre-gl.sh + +# ── Security ────────────────────────────────────────────────── +security: + fail_mode: closed # "closed" (default) or "open" +``` + +### Field merge rules (for `base` and `forge`) + +| Field type | Behavior | +|-----------|----------| +| Scalars (`model`, `pre_script`, `image`, etc.) | Child wins if non-empty | +| `skills`, `plugins`, `providers`, `api_servers`, `openshell.profiles` | Concatenated (base + child) | +| `host_files` | Concatenated; child overrides by `dest` | +| `env`, `runner_env` | Merged; child keys win | +| `validation_loop`, `security` | Child replaces entirely | +| `allowed_remote_resources`, `allow_runtime_fetch`, `max_runtime_fetches` | NOT inherited (child must declare its own) | + +### Referencing resources: local vs. remote + +**Local paths** resolve relative to the harness file's base directory: +```yaml +agent: agents/triage.md # → {base}/agents/triage.md +``` + +**Remote URLs** require a `#sha256=...` integrity hash: +```yaml +agent: https://raw.githubusercontent.com/org/repo//agents/lint.md#sha256=abc... +``` + +**Scripts are local-only** — `pre_script`, `post_script`, and `validation_loop.script` must be local paths (they run on the trusted runner). Exception: scripts declared in a `base` harness fetched via URL are allowed. + +## Agent definitions + +The agent definition is Markdown with YAML frontmatter: + +| Field | Purpose | +|-------|---------| +| `name` | Must match the filename (sans `.md`) | +| `description` | One-line summary | +| `tools` | Allowed Bash commands (e.g., `Bash(gh,jq)`) | +| `model` | LLM model | +| `skills` | Skill names to mount | +| `disallowedTools` | Forbidden Bash patterns | + +**Design principles:** +- Agent writes a JSON result file; scripts do all mutations. +- Be specific — define scoring dimensions, thresholds, output schemas. +- Include decision points (branch on confidence, clarity scores, etc.). + +## Skills + +A skill is a directory with a `SKILL.md` file that teaches the agent domain knowledge: + +``` +skills/issue-labels/ + SKILL.md # Required: frontmatter + instructions + scripts/ # Optional: helper scripts + references/ # Optional: reference data +``` + +Reference in the agent frontmatter by name (`skills: [issue-labels]`) and in the harness by path (`skills: [skills/issue-labels]`). Skills can also be URLs with integrity hashes. + +## Scripts + +Pre and post scripts run on the trusted runner outside the sandbox. + +- **Pre-scripts** prepare the environment — fetch data, reset state, write files for `host_files` to copy in. +- **Post-scripts** act on agent output — apply labels, post comments, create PRs. + +**Security:** treat agent output as untrusted input. Validate JSON structure, validate field values against allowlists, quote all variables, and limit string lengths. + +## Harness composition with `base` + +Inherit from an existing harness and override only what differs: + +```yaml +base: https://raw.githubusercontent.com/fullsend-ai/agents//harness/triage.yaml#sha256=abc... + +model: sonnet +slug: my-org-triage +skills: + - skills/my-enhancement +timeout_minutes: 15 +``` + +Base chains support up to 5 levels (`MaxBaseDepth` in `internal/harness/compose.go`). Circular references are detected and rejected. Resolution order: base chain → child overrides → forge selection. See [field merge rules](#field-merge-rules-for-base-and-forge) for how each field type combines. + +> **Note:** `allowed_remote_resources`, `allow_runtime_fetch`, and `max_runtime_fetches` are NOT inherited from base harnesses — the child must declare its own. This prevents a base harness from injecting arbitrary URL prefixes or enabling runtime fetching in the child. + +## Configuring existing agents + +You don't need to build from scratch to change how a built-in agent behaves. Use `base` to inherit the built-in harness and override just the fields you want — then register your configured version so it takes precedence. + +### Example: add a skill to the code agent + +Create a thin harness that inherits from the upstream code agent and adds your skill: + +**`harness/code.yaml`:** +```yaml +base: https://raw.githubusercontent.com/fullsend-ai/fullsend//internal/scaffold/fullsend-repo/harness/code.yaml#sha256=abc... + +skills: + - skills/my-custom-linting # Concatenated with base skills + +timeout_minutes: 45 # Override timeout (scalar → child wins) +``` + +**`skills/my-custom-linting/SKILL.md`:** +```markdown +--- +name: my-custom-linting +description: Org-specific linting rules and conventions. +--- + +# My Custom Linting + +[Your skill content...] +``` + +Test it locally first (see [Testing locally](#testing-locally) for all flags): +```bash +fullsend run code --fullsend-dir .fullsend --target-repo ./my-repo --env-file .env.local +``` + +Then register it: +```bash +fullsend agent add harness/code.yaml --name code --fullsend-dir .fullsend +``` + +Because config-registered agents take precedence over built-in agents on name collision, your `code` agent replaces the default — with all of the base agent's scripts, policies, host_files, and plugins still inherited. + +### Example: swap the model for review + +```yaml +base: https://raw.githubusercontent.com/fullsend-ai/fullsend//internal/scaffold/fullsend-repo/harness/review.yaml#sha256=abc... + +model: sonnet +``` + +### Example: add org-specific environment variables + +```yaml +base: https://raw.githubusercontent.com/fullsend-ai/fullsend//internal/scaffold/fullsend-repo/harness/code.yaml#sha256=abc... + +env: + runner: + JIRA_TOKEN: "${JIRA_TOKEN}" # Merged with base env; child keys win + sandbox: + JIRA_PROJECT: "MYPROJ" +``` + +### What you can configure + +Any harness field can be overridden. The [field merge rules](#field-merge-rules-for-base-and-forge) determine how your overrides combine with the base: + +- **Change model, timeout, image, scripts** — scalars replace the base value. +- **Add skills, plugins, or host_files** — your entries are concatenated with the base's. +- **Add or override env vars** — maps are merged; your keys win on collision. +- **Replace validation or security config** — child replaces the entire block. + +## Testing locally + +Before registering, verify your agent works locally. Most agents need additional flags for credentials and target repo — see [Running agents locally](running-agents-locally.md) for the full list: + +```bash +fullsend run my-agent \ + --fullsend-dir .fullsend \ + --target-repo ./my-repo \ + --env-file .env.local +``` + +The `--env-file` supplies variables your harness references (e.g. `GH_TOKEN`, `ANTHROPIC_VERTEX_PROJECT_ID`). See [Running agents locally](running-agents-locally.md) for prerequisites (GCP credentials, sandbox image) and troubleshooting. + +## Registering your agent + +Register agents in `config.yaml` so fullsend discovers them. Both per-repo (`.fullsend/config.yaml`) and per-org configs support the `agents:` list. + +Authentication for CLI commands uses the `gh` CLI or `GH_TOKEN` environment variable. For URL agents, the CLI resolves GitHub blob URLs to `raw.githubusercontent.com` URLs automatically. + +The examples above show customizing built-in agents via `base`. If you've built an entirely new agent from scratch, register it the same way — just point to a local harness instead of a URL. + +### CLI + +```bash +# Add (auto-pins URL with SHA256): +fullsend agent add \ + https://github.com/fullsend-ai/agents/blob/main/harness/triage.yaml \ + --fullsend-dir .fullsend + +# Add local: +fullsend agent add harness/my-agent.yaml --name my-agent --fullsend-dir .fullsend + +# List / update / remove: +fullsend agent list --fullsend-dir .fullsend +fullsend agent update triage --fullsend-dir .fullsend +fullsend agent remove triage --fullsend-dir .fullsend +``` + +### Per-repo config (`.fullsend/config.yaml`) + +```yaml +version: "1" +roles: [triage, coder, review] +agents: + - https://raw.githubusercontent.com/fullsend-ai/agents//harness/triage.yaml#sha256=abc... + - name: my-cool-agent + source: harness/my-cool-agent.yaml +allowed_remote_resources: + - https://raw.githubusercontent.com/fullsend-ai/fullsend/ + - https://raw.githubusercontent.com/fullsend-ai/agents/ +``` + +### Per-org config + +```yaml +version: "1" +dispatch: + platform: github-actions +defaults: + roles: [triage, coder, review] +agents: + - https://raw.githubusercontent.com/fullsend-ai/agents//harness/triage.yaml#sha256=abc... + - name: my-cool-agent + source: harness/my-cool-agent.yaml +allowed_remote_resources: + - https://raw.githubusercontent.com/fullsend-ai/fullsend/ + - https://raw.githubusercontent.com/fullsend-ai/agents/ +repos: + my-repo: + enabled: true +``` + +**Notes:** +- `roles` controls which built-in agent roles are enabled. Valid values: `fullsend`, `triage`, `coder`, `review`, `fix`, `retro`, `prioritize`, `e2e`. Custom agents registered via `agents:` do not need to appear in this list. +- URL entries are automatically pinned with `#sha256=...` by `fullsend agent add`. +- URLs must be covered by `allowed_remote_resources` in the same config. +- On name collision, config-registered agents take precedence over built-in agents. +- Per-repo config is read from the **base branch**, not from PR branches. + +## Migrating from `customized/` + +The `customized/` directory overlay ([ADR 0035](../../ADRs/0035-layered-content-resolution.md)) is deprecated in favor of the `base:` composition and config-driven registration described in this guide ([ADR 0064](../../ADRs/0064-deprecate-customized-directory-overlay.md)). + +If you have existing files in `customized/`, the `fullsend agent migrate-customizations` command automates the conversion to config-driven agents. + +Preview what would change: +```bash +fullsend agent migrate-customizations --fullsend-dir .fullsend --dry-run +``` + +Run the migration (creates a PR with the changes): +```bash +fullsend agent migrate-customizations --fullsend-dir .fullsend --repo owner/repo +``` + +The tool classifies each override and takes the appropriate action: + +| Override type | Detection | Action | +|---------------|-----------|--------| +| Dead | Agent already registered in config | Delete `customized/` files | +| Custom | Not in upstream scaffold | Move files to regular directories, register local path in config | +| Modified | Standard scaffold agent, not yet in config | Generate a `base:` composition harness with the minimal diff, register in config | + +For modified agents, the migration produces exactly the kind of thin `base:` harness shown in [Configuring existing agents](#configuring-existing-agents) — only the fields that differ from upstream are included. + +## Advanced: custom identity + +By default, agents authenticate using shared fullsend GitHub Apps via the `slug` field. If you need your own GitHub App — for custom permissions, compliance, or branding — you can run a **standalone mint**. Follow the [Standalone mint guide](../infrastructure/standalone-mint.md) to set one up. + +Once your standalone mint is running, configure your agent to use it: + +1. **Reference your role in the harness:** + ```yaml + role: my-role + slug: my-org-my-role + ``` + +2. **Set `FULLSEND_MINT_URL`** in your repo to point to your standalone mint. + +When configured with `FALLBACK_MINT_URL`, the standalone mint serves custom roles locally while proxying unhandled roles to the hosted mint (see [Standalone mint — Fallback proxy behavior](../infrastructure/standalone-mint.md#fallback-proxy-behavior)). + +## Troubleshooting + +| Symptom | Fix | +|---------|-----| +| Agent crashes at 0s | Sandbox can't reach Vertex AI — verify that `providers/vertex-ai.yaml` is listed in your harness `providers:` and that `ANTHROPIC_VERTEX_PROJECT_ID`/`CLOUD_ML_REGION` are set (in your `--env-file` for local runs, or in the workflow `env` block for CI) | +| "role field is required" | Add `role:` to harness | +| Agent can't find input files | Pre-script output paths must match `host_files` entries | +| Provider blocks requests | Check that the required provider profile is listed in `providers:` and exists in the `providers/` directory | +| Schema validation fails | Compare the sandbox output (`$FULLSEND_OUTPUT_DIR/.json`) against the schema referenced in `validation_loop` / `FULLSEND_OUTPUT_SCHEMA`; re-run with `--keep-sandbox` to inspect | +| Agent not found | Verify registration: `fullsend agent list` | +| `allowed_remote_resources` error | URL agents require a matching prefix in `allowed_remote_resources` — `fullsend agent add` sets this automatically | +| `fullsend run` fails locally | Missing GCP credentials or sandbox image — see [Running agents locally](running-agents-locally.md) | +| Integrity hash mismatch | Remote content changed — run `fullsend agent update ` to re-pin | + +## See also + +- [fullsend-ai/agents](https://github.com/fullsend-ai/agents) — reference implementation used throughout this guide +- [Customizing Agents with Skills](customizing-with-skills.md) — creating and managing skills +- [Customizing Agents with AGENTS.md](customizing-with-agents-md.md) — repo-level instructions for all agents +- [Customizing agents](customizing-agents.md) — harness configurations and layered content resolution +- [Default, derived, and custom agents](../../agents/topics/default-vs-custom.md) — when configuration crosses into custom agent territory +- [Standalone mint](../infrastructure/standalone-mint.md) — custom agent roles and identity diff --git a/docs/guides/user/building-custom-agents.md b/docs/guides/user/building-custom-agents.md index a34ac31f0d..7411dafb85 100644 --- a/docs/guides/user/building-custom-agents.md +++ b/docs/guides/user/building-custom-agents.md @@ -15,7 +15,8 @@ harness and override only what differs — see [Default, derived, and custom agents](../../agents/topics/default-vs-custom.md) for the distinction and when each approach makes sense. -For configuring existing agents (overriding harnesses, skills, or policies), see [Configuring agents](customizing-agents.md). +For the config-driven approach to building or configuring agents, see +[Bring Your Own Agent](bring-your-own-agent.md). For configuring existing agents (overriding harnesses, skills, or policies), see [Customizing agents](customizing-agents.md). ## Prerequisites @@ -35,7 +36,7 @@ A custom agent is composed of six parts: skills/ # Knowledge documents mounted into the sandbox ``` -At build time, the workflow layers these customized files on top of the upstream fullsend defaults. Your files override the defaults — anything you don't customize uses the standard fullsend configuration. See [Configuring agents — Layered Configuration Resolution](customizing-agents.md#layered-configuration-resolution) for details on how the layering works. +At build time, the workflow layers these customized files on top of the upstream fullsend defaults. Your files override the defaults — anything you don't customize uses the standard fullsend configuration. See [Customizing agents — Layered Configuration Resolution](customizing-agents.md#layered-configuration-resolution) for details on how the layering works. The key security invariant: agents run inside an untrusted [sandbox](../../glossary.md#sandbox) with no credentials. Pre-scripts fetch data *before* the sandbox starts; post-scripts act on agent output *after* the sandbox exits. Agents never have direct write access to external systems. See the [security threat model](../../problems/security-threat-model.md) for the full trust model. @@ -183,7 +184,7 @@ timeout_minutes: 20 # max_runtime_fetches: 10 ``` -See [Configuring agents — Harness YAML Structure](customizing-agents.md#harness-yaml-structure) for the full field reference (including optional `security`, `providers`, `plugins`, and runtime fetch blocks). +See [Customizing agents — Harness YAML Structure](customizing-agents.md#harness-yaml-structure) for the full field reference (including optional `security`, `providers`, `plugins`, and runtime fetch blocks). The key pattern to understand is how data flows into the sandbox through `host_files`: @@ -375,7 +376,7 @@ The post-script runs on the trusted runner with full credentials, but reads outp ## Step 6: Create skills (optional) -[Skills](../../glossary.md#skill) are Markdown documents mounted into the sandbox that provide domain knowledge the agent can reference. See [Configuring agents — Adding a Custom Skill](customizing-agents.md#adding-a-custom-skill) for how to create one. +[Skills](../../glossary.md#skill) are Markdown documents mounted into the sandbox that provide domain knowledge the agent can reference. See [Customizing agents — Adding a Custom Skill](customizing-agents.md#adding-a-custom-skill) for how to create one. Place your skill at `.fullsend/customized/skills/my-skill/SKILL.md`, then reference it in both the agent frontmatter (`skills: [my-skill]`) and the harness (`skills: [customized/skills/my-skill]`). @@ -626,7 +627,7 @@ When creating a new agent, you need these files: ## Reference -- [Configuring agents](customizing-agents.md) — override existing agent harnesses, skills, and policies +- [Customizing agents](customizing-agents.md) — override existing agent harnesses, skills, and policies - [Bugfix workflow](bugfix-workflow.md) — how the built-in agents work together end to end - [Getting Started](../getting-started/README.md) — prerequisite: admin setup guide - [Architecture overview](../../architecture.md) — component vocabulary and execution stack diff --git a/docs/guides/user/customizing-agents.md b/docs/guides/user/customizing-agents.md index 03b11d7e1e..7ee1d934a0 100644 --- a/docs/guides/user/customizing-agents.md +++ b/docs/guides/user/customizing-agents.md @@ -497,6 +497,7 @@ my-repo/ ## See Also +- [Bring Your Own Agent](bring-your-own-agent.md) - Building and registering custom agents from scratch - [Default, derived, and custom agents](../../agents/topics/default-vs-custom.md) - When does configuration cross into derived or custom agent territory? - [Getting Started](../getting-started/) - Initial setup - [Bugfix Workflow](bugfix-workflow.md) - How agents work together diff --git a/docs/guides/user/customizing-with-skills.md b/docs/guides/user/customizing-with-skills.md index 30dc45ecde..1fc755e6e3 100644 --- a/docs/guides/user/customizing-with-skills.md +++ b/docs/guides/user/customizing-with-skills.md @@ -166,5 +166,6 @@ apply to all agents and human contributors alike. ## See also +- [Bring Your Own Agent](bring-your-own-agent.md) — building and registering custom agents - [Default, derived, and custom agents](../../agents/topics/default-vs-custom.md) — adding skills keeps you in "configured default agent" territory diff --git a/website/.vitepress/config.ts b/website/.vitepress/config.ts index f4e556f3c3..8cc2c75fcb 100644 --- a/website/.vitepress/config.ts +++ b/website/.vitepress/config.ts @@ -182,6 +182,7 @@ export default defineConfig({ text: "CLI Reference", items: [ { text: "Overview", link: "/cli/" }, + { text: "fullsend agent", link: "/cli/agent" }, { text: "fullsend github", link: "/cli/github" }, { text: "fullsend inference", link: "/cli/inference" }, { text: "fullsend mint", link: "/cli/mint" }, @@ -220,12 +221,13 @@ export default defineConfig({ collapsed: true, link: "/guides/", items: [ + { text: "Bring Your Own Agent", link: "/guides/user/bring-your-own-agent" }, { text: "Bugfix Workflow", link: "/guides/user/bugfix-workflow" }, { text: "Customizing Agents", link: "/guides/user/customizing-agents" }, { text: "Customizing with AGENTS.md", link: "/guides/user/customizing-with-agents-md" }, { text: "Customizing with Skills", link: "/guides/user/customizing-with-skills" }, { - text: "Building custom agents from scratch", + text: "Building custom agents from scratch (deprecated)", link: "/guides/user/building-custom-agents", }, { text: "Running Agents Locally", link: "/guides/user/running-agents-locally" },