Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion .skillsaw.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -335,7 +335,15 @@ exclude:
# - "**/_template/**"

# Additional markdown files to run content rules on (glob format)
content-paths: []
#
# examples/ holds complete reference agents generated by `fullsend agent new`
# (see examples/link-check/). They are not fleet agents and are not registered
# in config.yaml, so skillsaw does not discover them on its own — but they are
# meant to be copied, so an unfilled placeholder or a stale TODO in one is
# exactly as harmful as it would be in a real agent definition.
content-paths:
- "examples/**/agents/*.md"


# Not enforced here: CI (lint.yml) and `make lint` both pass an explicit
# strict flag that overrides this config value, so this setting only
Expand Down
1 change: 1 addition & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,7 @@ script-test:
$(call run-timed,bash scripts/post-retro-test.sh)
$(call run-timed,bash scripts/pre-scribe-test.sh)
$(call run-timed,bash scripts/post-scribe-test.sh)
$(call run-timed,bash scripts/example-link-check-test.sh)
$(call run-timed,bash scripts/validate-output-schema-test.sh)
$(call run-timed,bash scripts/validate-code-output-test.sh)
$(call run-timed,bash scripts/gitlint-forbidden-type-scope-test.sh)
Expand Down
6 changes: 5 additions & 1 deletion docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,4 +19,8 @@ Reference documentation for the default agents shipped by fullsend.
| [Custom network policy](network-policy.md) | Configure sandbox network access for hosts beyond the default allowlist |

For configuration, customization, and building your own agents, see the
[fullsend docs](https://fullsend.sh/docs).
[fullsend docs](https://fullsend.sh/docs). To build one, run
[`fullsend agent new`](https://github.com/fullsend-ai/fullsend/blob/main/docs/cli/agent.md#agent-new)
and then follow the [`authoring-custom-agents`](../skills/authoring-custom-agents/SKILL.md)
skill to complete the generated prompt; [`examples/link-check/`](../examples/link-check/)
is a finished one.
79 changes: 79 additions & 0 deletions examples/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
# Examples

Each example here was generated by
[`fullsend agent new`](https://github.com/fullsend-ai/fullsend/blob/main/docs/cli/agent.md#agent-new),
which writes every file except the prompt. What was then added by hand is the
body of `agents/<name>.md` — the instructions the agent follows — and the
choice of trigger and scope.

These are not the agents in this repository's `harness/` directory. They are
not registered in `config.yaml`, nothing dispatches them, and no repository
consumes them by URL. They are here so a working custom agent is checked by
this repository's lint and test gates instead of only described in
documentation.

| Example | Role | Trigger | What it does |
|---------|------|---------|--------------|
| [`link-check/`](link-check/) | `review` | pull request opened, updated, or marked ready — not from a fork | Reports Markdown links **added by the pull request** that do not resolve |

To run one locally, register it first — the examples are deliberately not in
this repository's `config.yaml`, so `fullsend run` cannot find them until you
add one to a copy:

```bash
cp -r examples/link-check /tmp/demo-fullsend
printf 'version: "1"\nroles: [review]\n' > /tmp/demo-fullsend/config.yaml
fullsend agent add harness/link-check.yaml --name link-check \
--fullsend-dir /tmp/demo-fullsend
```

Without that step the run fails with `resolving agent "link-check": no config
and agents-repo fallback unavailable`.

## Using one

Generate your own rather than copying this directory. The command records the
exact container image the agents in this repository currently run on, and
writes the network-access files the role you pick needs:

```bash
fullsend agent new my-agent --fullsend-dir .fullsend \
--role review --description "What my agent decides" --validation-loop
```

`--validation-loop` re-runs the agent if its JSON output does not match the
schema. It is off by default in the command, but every agent in this
repository's `harness/` directory that has a schema uses one, so the example
turns it on to match. It needs `python3` and the `jsonschema` package on the
machine running the agent; the generated `preflight_check` checks for them
before starting the agent rather than after it has finished.

Then read the example alongside the
[`authoring-custom-agents`](../skills/authoring-custom-agents/SKILL.md) skill
while you fill in `agents/my-agent.md`.

## What each example must satisfy

- **No placeholders.** Two gates, because neither covers the other.
`skillsaw --strict` runs over `examples/**/agents/*.md` (see `content-paths`
in `.skillsaw.yaml`) and catches a stale `TODO`; it does **not** recognise
the `<!-- FILL IN: ... -->` marker the generator emits, so
`scripts/example-link-check-test.sh` greps for that one and fails
`make test` if any survives.
- **A tested post-script.** The script that turns the agent's output into a
comment is fed whatever the model produced, so it is tested as carefully as
the ones in `scripts/` — see `scripts/example-link-check-test.sh`, wired into
`make test`.
- **It loads.** `fullsend lock <name> --fullsend-dir examples/<name> --offline`
must pass. It resolves the harness by path, so it works against the example
as it sits here, unregistered.
- **The shared assets are what the generator produces.** `policies/`,
`providers/`, `profiles/` and `scripts/validate-output-schema.sh` are
byte-identical to `fullsend agent new` output, and are not hand-edited here:
they are vendored copies of files that live in fullsend, so a change to one
belongs there, not in this copy. Two of them currently differ from what
fullsend's own scaffold ships, which is tracked as
[fullsend-ai/fullsend#6981](https://github.com/fullsend-ai/fullsend/issues/6981)
and [#7014](https://github.com/fullsend-ai/fullsend/issues/7014).
- **The prompt is written by hand.** `agents/<name>.md` is the one file the
generator leaves for you, and completing it is the point of the example.
132 changes: 132 additions & 0 deletions examples/link-check/agents/link-check.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,132 @@
---
name: link-check
description: Check that links in changed documentation resolve
tools: Bash(gh,jq), Read, Glob
model: opus
---

You are the link-check agent. You decide whether the Markdown links a pull
request **adds or changes** point at something that exists, and you report the
ones that do not.

Scope is deliberately narrow: only links on lines the pull request adds. A
pre-existing broken link elsewhere in a touched file is not this agent's
finding — reporting it would blame the author for something they did not
write, which is the fastest way to get an agent's comments ignored.

## Inputs

- `ISSUE_URL` — the HTML URL of the pull request this run was dispatched for.
- `FULLSEND_FORGE` — always `github` for this agent.
- The target repository is checked out at the sandbox working directory. It is
a **shallow checkout of the default branch, not the pull request's head** —
so a file the pull request adds is not on disk, and a file it deletes still
is. Never infer that a path exists because the pull request adds it.

## Steps

1. Read the pull request's changed files. Use the REST API rather than `git`:
the repository is checked out shallow and not at the pull request's head,
so there is no history to diff against locally.

```bash
# ISSUE_URL looks like https://github.com/OWNER/REPO/pull/NUMBER
read -r OWNER REPO NUMBER < <(sed -E 's#^https://github[.]com/([^/]+)/([^/]+)/(pull|issues)/([0-9]+)$#\1 \2 \4#' <<<"$ISSUE_URL")
gh api --paginate "repos/${OWNER}/${REPO}/pulls/${NUMBER}/files" \
--jq '.[] | select(.status != "removed")
| select(.filename | endswith(".md"))
| select(.patch != null)
| {filename, patch}'
```

Interpolate those three values yourself, as above. Do not write
`{owner}`/`{repo}` literally: those are `gh`'s own placeholders for the
*current checkout's* remote, there is no `{number}` placeholder at all,
and a literal `{number}` is sent through unsubstituted and returns 404.

`select(.status != "removed")` drops files the pull request deletes.
`select(.patch != null)` drops files GitHub returned without a diff — a
pure rename, or one it considered too large. If any `.md` file was dropped
for that reason, or the response reached the endpoint's 3,000-file cap,
say so and use `status: "error"`: reporting `ok` would claim links were
checked when they were not.

If the command fails, write a result with `status: "error"`, a `summary`
naming the command that failed, and stop.

2. If no `.md` files changed, write `status: "ok"` with the summary
`No documentation changes` and stop.

3. Each `patch` is a unified diff. Walk it and keep only the **added** lines —
those beginning with a single `+`. Track
the line number in the file at head: each hunk header `@@ -a,b +c,d @@`
restarts the counter at `c`, an added line advances it by one, and a
context line advances it by one. A REST `patch` starts at its first `@@`,
so there are no file headers to skip.

4. From those added lines, extract every Markdown link target: the target in
`[text](target)` and in `[ref]: target` definitions. Classify each:

- **Relative path** (`../guides/x.md`, `./y.md#anchor`) — resolve it against
the directory of the file that contains it. Strip any `#anchor` and any
`?query` suffix, and percent-decode the result (`My%20Guide.md` is
`My Guide.md`), then check whether that path exists in the checkout.
- **Root-relative path** (`/docs/x.md`) — resolve against the repository
root and check the same way.
- **Absolute URL** (any scheme, including `https`, `http` and `mailto`) —
skip it. The sandbox has no general egress, so a network check would be
flaky rather than wrong.
- **Anchor-only** (`#section`) — skip it.

Before classifying, normalise the target: unwrap a `<...>` destination, and
drop an optional title following the destination (`[t](x.md "Title")` has
the target `x.md`, not `x.md "Title"`). Both are valid CommonMark and both
otherwise yield a target that can never exist on disk.

Skip any candidate inside a backtick code span or a fenced code block — a
documentation change that shows Markdown syntax is not adding a link. The
patch alone cannot tell you the fence state, so read the file at head when
a candidate looks like it may be inside one.
- A `[ref]: target` definition that nothing references — skip it. An unused
definition renders nothing, so it cannot be broken for a reader.

5. A link is broken when its resolved path does not exist. Decide that from
two sources, in this order: if the path is one of the files this pull
request adds or renames — you have that list from step 1 — it will exist
once merged, so treat it as resolving even though it is absent from the
checkout. Otherwise check the checkout on disk. Do not assume a path
exists merely because it appears in the diff as a link target.
Report it as `<file>:<line> -> <target>`, using the line number at head
from step 3.

6. Decide:
- No added links, or none broken: `status: "ok"`.
- One or more broken added links: `status: "findings"`.
- A step could not be completed at all: `status: "error"`.

## Output contract

Write exactly one JSON object to `$FULLSEND_OUTPUT_DIR/agent-result.json`:

```json
{
"status": "findings",
"summary": "2 broken links added in docs/",
"comment": "### Broken links\n\n- `docs/a.md:14` -> `../missing.md`\n"
}
```

- `status` — one of `ok`, `findings`, `error`.
- `summary` — one line, at most 200 characters. Used as the comment heading.
- `comment` — Markdown body posted on the pull request, at most 16384
characters. List one broken link per bullet as `` `<file>:<line>` -> `<target>` ``.
When `status` is `ok` the post-script posts nothing, but `comment` is still
required — a single line such as `All added documentation links resolve.` is
fine.

Do not push commits, open issues, apply labels, edit files, or call any
mutating API. The post-script performs every side effect; your only output is
this file.

Before you finish, run `fullsend-check-output "$FULLSEND_OUTPUT_DIR/agent-result.json"`
to catch schema violations while you can still fix them.
57 changes: 57 additions & 0 deletions examples/link-check/harness/link-check.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
---
# Generated by `fullsend agent new link-check`. Edit freely — nothing
# regenerates this file.
#
# `image:` is the container image this agent runs inside, recorded as
# an exact digest so every run uses the same one. Pass --image to
# choose a different one.
#
# This agent handles GitHub only. To add GitLab or Jira, see
# docs/guides/user/bring-your-own-agent.md.
agent: agents/link-check.md
description: Check that links in changed documentation resolve
role: review
slug: fullsend-ai-link-check
image: ghcr.io/fullsend-ai/fullsend-code@sha256:ea2a31f38ee80e2a9a898a898a289fe432aa882fa5a4046c3236ab8e2627d7e7
policy: policies/base.yaml
providers:
- providers/vertex-ai.yaml
- providers/github-ro.yaml
openshell:
profiles:
- profiles/fullsend-vertex-ai.yaml
- profiles/fullsend-github-ro.yaml
host_files:
- src: ${GOOGLE_APPLICATION_CREDENTIALS}
dest: /tmp/.gcp-credentials.json
- src: ${GCP_OIDC_TOKEN_FILE}
dest: /sandbox/workspace/.gcp-oidc-token
optional: true
model: opus
effort: high
post_script: scripts/post-link-check.sh
Comment thread
qodo-code-review[bot] marked this conversation as resolved.
validation_loop:
script: scripts/validate-output-schema.sh
schema: schemas/link-check-result.schema.json
max_iterations: 2
preflight_check: python3 -c "import jsonschema"
env:
runner:
FULLSEND_FORGE: github
GH_TOKEN: ${GH_TOKEN}
ISSUE_URL: ${GITHUB_ISSUE_URL}
sandbox:
ANTHROPIC_VERTEX_PROJECT_ID: ${ANTHROPIC_VERTEX_PROJECT_ID}
CLAUDE_CODE_USE_VERTEX: "1"
CLOUD_ML_REGION: ${CLOUD_ML_REGION}
FULLSEND_FORGE: github
GH_TOKEN: ${GH_TOKEN}
GOOGLE_APPLICATION_CREDENTIALS: /tmp/.gcp-credentials.json
ISSUE_URL: ${GITHUB_ISSUE_URL}
timeout_minutes: 15
trigger: |
event.entity.kind == "change_proposal"
&& (event.transition.kind == "opened"
|| event.transition.kind == "synchronized"
|| event.transition.kind == "marked_ready")
&& !event.state.change_proposal.is_fork
22 changes: 22 additions & 0 deletions examples/link-check/policies/base.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
---
version: 1

# Base sandbox policy shared by all agents.
#
# Defines non-composable sandbox restrictions: filesystem access,
# landlock, and process identity. Network access is provided
# entirely by provider profiles via provider-backed policy
# composition (ADR 0065).
#
# curl is deliberately excluded from all profile binary allowlists
# to prevent raw HTTP access with the injected GH_TOKEN.

filesystem_policy:
include_workdir: true
read_only: [/usr, /lib, /proc, /dev/urandom, /app, /etc, /var/log]
read_write: [/sandbox, /tmp, /dev/null]
landlock:
compatibility: best_effort
process:
run_as_user: sandbox
run_as_group: sandbox
19 changes: 19 additions & 0 deletions examples/link-check/profiles/fullsend-github-ro.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
---
id: fullsend-github-ro
display_name: Fullsend GitHub (read-only)
description: Read-only GitHub API access for fullsend agents
category: source_control
endpoints:
- host: api.github.com
port: 443
protocol: rest
access: read-only
enforcement: enforce
- host: github.com
port: 443
protocol: rest
access: read-only
enforcement: enforce
binaries:
- "**/gh"
- "**/node"
25 changes: 25 additions & 0 deletions examples/link-check/profiles/fullsend-vertex-ai.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
---
id: fullsend-vertex-ai
display_name: Fullsend Vertex AI
description: Anthropic API and Google Cloud APIs for inference
category: inference
endpoints:
- host: api.anthropic.com
port: 443
protocol: rest
access: read-write
enforcement: enforce
# See fullsend-openai.yaml: model request bodies that mention the
# placeholder prefix would otherwise be reset (NVIDIA/OpenShell#2904).
allow_uninspected_credentials: true
- host: "*.googleapis.com"
port: 443
protocol: rest
access: read-write
enforcement: enforce
allow_uninspected_credentials: true
Comment thread
qodo-code-review[bot] marked this conversation as resolved.
binaries:
- "**/claude"
- "**/claude.exe"
- "**/node"
- "**/pi"
6 changes: 6 additions & 0 deletions examples/link-check/providers/github-ro.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
name: github-ro
type: fullsend-github-ro
# Workaround: https://github.com/NVIDIA/OpenShell/issues/1978
credentials:
_NOOP_GITHUB_RO: ""
6 changes: 6 additions & 0 deletions examples/link-check/providers/vertex-ai.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
name: vertex-ai
type: fullsend-vertex-ai
# Workaround: https://github.com/NVIDIA/OpenShell/issues/1978
credentials:
_NOOP_VERTEX_AI: ""
Loading
Loading