-
Notifications
You must be signed in to change notification settings - Fork 15
feat(#6966): add a completed reference agent and an authoring skill #1167
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
waynesun09
wants to merge
9
commits into
main
Choose a base branch
from
agent-6966-agent-templates
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
9 commits
Select commit
Hold shift + click to select a range
98f68fd
feat(#6966): add a completed reference agent and an authoring skill
waynesun09 f894eaf
fix(#6966): scope the example to added links and keep truncation in t…
waynesun09 202d462
docs(#6966): plain-language README for examples/, mention the authori…
waynesun09 91341d6
docs(#6966): finish removing project jargon from the example docs
waynesun09 59dd305
fix(#6966): regenerate the example and test the real run-dir contract
waynesun09 ce2a2af
fix(#6966): the example could not run its own first two steps
waynesun09 897d3af
fix(#6966): correct three claims the example made about itself
waynesun09 ca26a6c
fix(#6966): regenerate the example on the fixed Vertex profile
waynesun09 8f9272f
fix(#6966): the Inputs section described a checkout that does not exist
waynesun09 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,79 @@ | ||
| # Examples | ||
|
|
||
| Each example here was generated by | ||
| [`fullsend agent new`](https://github.com/fullsend-ai/fullsend/blob/main/docs/cli/agent.md#agent-new), | ||
| which writes every file except the prompt. What was then added by hand is the | ||
| body of `agents/<name>.md` — the instructions the agent follows — and the | ||
| choice of trigger and scope. | ||
|
|
||
| These are not the agents in this repository's `harness/` directory. They are | ||
| not registered in `config.yaml`, nothing dispatches them, and no repository | ||
| consumes them by URL. They are here so a working custom agent is checked by | ||
| this repository's lint and test gates instead of only described in | ||
| documentation. | ||
|
|
||
| | Example | Role | Trigger | What it does | | ||
| |---------|------|---------|--------------| | ||
| | [`link-check/`](link-check/) | `review` | pull request opened, updated, or marked ready — not from a fork | Reports Markdown links **added by the pull request** that do not resolve | | ||
|
|
||
| To run one locally, register it first — the examples are deliberately not in | ||
| this repository's `config.yaml`, so `fullsend run` cannot find them until you | ||
| add one to a copy: | ||
|
|
||
| ```bash | ||
| cp -r examples/link-check /tmp/demo-fullsend | ||
| printf 'version: "1"\nroles: [review]\n' > /tmp/demo-fullsend/config.yaml | ||
| fullsend agent add harness/link-check.yaml --name link-check \ | ||
| --fullsend-dir /tmp/demo-fullsend | ||
| ``` | ||
|
|
||
| Without that step the run fails with `resolving agent "link-check": no config | ||
| and agents-repo fallback unavailable`. | ||
|
|
||
| ## Using one | ||
|
|
||
| Generate your own rather than copying this directory. The command records the | ||
| exact container image the agents in this repository currently run on, and | ||
| writes the network-access files the role you pick needs: | ||
|
|
||
| ```bash | ||
| fullsend agent new my-agent --fullsend-dir .fullsend \ | ||
| --role review --description "What my agent decides" --validation-loop | ||
| ``` | ||
|
|
||
| `--validation-loop` re-runs the agent if its JSON output does not match the | ||
| schema. It is off by default in the command, but every agent in this | ||
| repository's `harness/` directory that has a schema uses one, so the example | ||
| turns it on to match. It needs `python3` and the `jsonschema` package on the | ||
| machine running the agent; the generated `preflight_check` checks for them | ||
| before starting the agent rather than after it has finished. | ||
|
|
||
| Then read the example alongside the | ||
| [`authoring-custom-agents`](../skills/authoring-custom-agents/SKILL.md) skill | ||
| while you fill in `agents/my-agent.md`. | ||
|
|
||
| ## What each example must satisfy | ||
|
|
||
| - **No placeholders.** Two gates, because neither covers the other. | ||
| `skillsaw --strict` runs over `examples/**/agents/*.md` (see `content-paths` | ||
| in `.skillsaw.yaml`) and catches a stale `TODO`; it does **not** recognise | ||
| the `<!-- FILL IN: ... -->` marker the generator emits, so | ||
| `scripts/example-link-check-test.sh` greps for that one and fails | ||
| `make test` if any survives. | ||
| - **A tested post-script.** The script that turns the agent's output into a | ||
| comment is fed whatever the model produced, so it is tested as carefully as | ||
| the ones in `scripts/` — see `scripts/example-link-check-test.sh`, wired into | ||
| `make test`. | ||
| - **It loads.** `fullsend lock <name> --fullsend-dir examples/<name> --offline` | ||
| must pass. It resolves the harness by path, so it works against the example | ||
| as it sits here, unregistered. | ||
| - **The shared assets are what the generator produces.** `policies/`, | ||
| `providers/`, `profiles/` and `scripts/validate-output-schema.sh` are | ||
| byte-identical to `fullsend agent new` output, and are not hand-edited here: | ||
| they are vendored copies of files that live in fullsend, so a change to one | ||
| belongs there, not in this copy. Two of them currently differ from what | ||
| fullsend's own scaffold ships, which is tracked as | ||
| [fullsend-ai/fullsend#6981](https://github.com/fullsend-ai/fullsend/issues/6981) | ||
| and [#7014](https://github.com/fullsend-ai/fullsend/issues/7014). | ||
| - **The prompt is written by hand.** `agents/<name>.md` is the one file the | ||
| generator leaves for you, and completing it is the point of the example. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,132 @@ | ||
| --- | ||
| name: link-check | ||
| description: Check that links in changed documentation resolve | ||
| tools: Bash(gh,jq), Read, Glob | ||
| model: opus | ||
| --- | ||
|
|
||
| You are the link-check agent. You decide whether the Markdown links a pull | ||
| request **adds or changes** point at something that exists, and you report the | ||
| ones that do not. | ||
|
|
||
| Scope is deliberately narrow: only links on lines the pull request adds. A | ||
| pre-existing broken link elsewhere in a touched file is not this agent's | ||
| finding — reporting it would blame the author for something they did not | ||
| write, which is the fastest way to get an agent's comments ignored. | ||
|
|
||
| ## Inputs | ||
|
|
||
| - `ISSUE_URL` — the HTML URL of the pull request this run was dispatched for. | ||
| - `FULLSEND_FORGE` — always `github` for this agent. | ||
| - The target repository is checked out at the sandbox working directory. It is | ||
| a **shallow checkout of the default branch, not the pull request's head** — | ||
| so a file the pull request adds is not on disk, and a file it deletes still | ||
| is. Never infer that a path exists because the pull request adds it. | ||
|
|
||
| ## Steps | ||
|
|
||
| 1. Read the pull request's changed files. Use the REST API rather than `git`: | ||
| the repository is checked out shallow and not at the pull request's head, | ||
| so there is no history to diff against locally. | ||
|
|
||
| ```bash | ||
| # ISSUE_URL looks like https://github.com/OWNER/REPO/pull/NUMBER | ||
| read -r OWNER REPO NUMBER < <(sed -E 's#^https://github[.]com/([^/]+)/([^/]+)/(pull|issues)/([0-9]+)$#\1 \2 \4#' <<<"$ISSUE_URL") | ||
| gh api --paginate "repos/${OWNER}/${REPO}/pulls/${NUMBER}/files" \ | ||
| --jq '.[] | select(.status != "removed") | ||
| | select(.filename | endswith(".md")) | ||
| | select(.patch != null) | ||
| | {filename, patch}' | ||
| ``` | ||
|
|
||
| Interpolate those three values yourself, as above. Do not write | ||
| `{owner}`/`{repo}` literally: those are `gh`'s own placeholders for the | ||
| *current checkout's* remote, there is no `{number}` placeholder at all, | ||
| and a literal `{number}` is sent through unsubstituted and returns 404. | ||
|
|
||
| `select(.status != "removed")` drops files the pull request deletes. | ||
| `select(.patch != null)` drops files GitHub returned without a diff — a | ||
| pure rename, or one it considered too large. If any `.md` file was dropped | ||
| for that reason, or the response reached the endpoint's 3,000-file cap, | ||
| say so and use `status: "error"`: reporting `ok` would claim links were | ||
| checked when they were not. | ||
|
|
||
| If the command fails, write a result with `status: "error"`, a `summary` | ||
| naming the command that failed, and stop. | ||
|
|
||
| 2. If no `.md` files changed, write `status: "ok"` with the summary | ||
| `No documentation changes` and stop. | ||
|
|
||
| 3. Each `patch` is a unified diff. Walk it and keep only the **added** lines — | ||
| those beginning with a single `+`. Track | ||
| the line number in the file at head: each hunk header `@@ -a,b +c,d @@` | ||
| restarts the counter at `c`, an added line advances it by one, and a | ||
| context line advances it by one. A REST `patch` starts at its first `@@`, | ||
| so there are no file headers to skip. | ||
|
|
||
| 4. From those added lines, extract every Markdown link target: the target in | ||
| `[text](target)` and in `[ref]: target` definitions. Classify each: | ||
|
|
||
| - **Relative path** (`../guides/x.md`, `./y.md#anchor`) — resolve it against | ||
| the directory of the file that contains it. Strip any `#anchor` and any | ||
| `?query` suffix, and percent-decode the result (`My%20Guide.md` is | ||
| `My Guide.md`), then check whether that path exists in the checkout. | ||
| - **Root-relative path** (`/docs/x.md`) — resolve against the repository | ||
| root and check the same way. | ||
| - **Absolute URL** (any scheme, including `https`, `http` and `mailto`) — | ||
| skip it. The sandbox has no general egress, so a network check would be | ||
| flaky rather than wrong. | ||
| - **Anchor-only** (`#section`) — skip it. | ||
|
|
||
| Before classifying, normalise the target: unwrap a `<...>` destination, and | ||
| drop an optional title following the destination (`[t](x.md "Title")` has | ||
| the target `x.md`, not `x.md "Title"`). Both are valid CommonMark and both | ||
| otherwise yield a target that can never exist on disk. | ||
|
|
||
| Skip any candidate inside a backtick code span or a fenced code block — a | ||
| documentation change that shows Markdown syntax is not adding a link. The | ||
| patch alone cannot tell you the fence state, so read the file at head when | ||
| a candidate looks like it may be inside one. | ||
| - A `[ref]: target` definition that nothing references — skip it. An unused | ||
| definition renders nothing, so it cannot be broken for a reader. | ||
|
|
||
| 5. A link is broken when its resolved path does not exist. Decide that from | ||
| two sources, in this order: if the path is one of the files this pull | ||
| request adds or renames — you have that list from step 1 — it will exist | ||
| once merged, so treat it as resolving even though it is absent from the | ||
| checkout. Otherwise check the checkout on disk. Do not assume a path | ||
| exists merely because it appears in the diff as a link target. | ||
| Report it as `<file>:<line> -> <target>`, using the line number at head | ||
| from step 3. | ||
|
|
||
| 6. Decide: | ||
| - No added links, or none broken: `status: "ok"`. | ||
| - One or more broken added links: `status: "findings"`. | ||
| - A step could not be completed at all: `status: "error"`. | ||
|
|
||
| ## Output contract | ||
|
|
||
| Write exactly one JSON object to `$FULLSEND_OUTPUT_DIR/agent-result.json`: | ||
|
|
||
| ```json | ||
| { | ||
| "status": "findings", | ||
| "summary": "2 broken links added in docs/", | ||
| "comment": "### Broken links\n\n- `docs/a.md:14` -> `../missing.md`\n" | ||
| } | ||
| ``` | ||
|
|
||
| - `status` — one of `ok`, `findings`, `error`. | ||
| - `summary` — one line, at most 200 characters. Used as the comment heading. | ||
| - `comment` — Markdown body posted on the pull request, at most 16384 | ||
| characters. List one broken link per bullet as `` `<file>:<line>` -> `<target>` ``. | ||
| When `status` is `ok` the post-script posts nothing, but `comment` is still | ||
| required — a single line such as `All added documentation links resolve.` is | ||
| fine. | ||
|
|
||
| Do not push commits, open issues, apply labels, edit files, or call any | ||
| mutating API. The post-script performs every side effect; your only output is | ||
| this file. | ||
|
|
||
| Before you finish, run `fullsend-check-output "$FULLSEND_OUTPUT_DIR/agent-result.json"` | ||
| to catch schema violations while you can still fix them. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,57 @@ | ||
| --- | ||
| # Generated by `fullsend agent new link-check`. Edit freely — nothing | ||
| # regenerates this file. | ||
| # | ||
| # `image:` is the container image this agent runs inside, recorded as | ||
| # an exact digest so every run uses the same one. Pass --image to | ||
| # choose a different one. | ||
| # | ||
| # This agent handles GitHub only. To add GitLab or Jira, see | ||
| # docs/guides/user/bring-your-own-agent.md. | ||
| agent: agents/link-check.md | ||
| description: Check that links in changed documentation resolve | ||
| role: review | ||
| slug: fullsend-ai-link-check | ||
| image: ghcr.io/fullsend-ai/fullsend-code@sha256:ea2a31f38ee80e2a9a898a898a289fe432aa882fa5a4046c3236ab8e2627d7e7 | ||
| policy: policies/base.yaml | ||
| providers: | ||
| - providers/vertex-ai.yaml | ||
| - providers/github-ro.yaml | ||
| openshell: | ||
| profiles: | ||
| - profiles/fullsend-vertex-ai.yaml | ||
| - profiles/fullsend-github-ro.yaml | ||
| host_files: | ||
| - src: ${GOOGLE_APPLICATION_CREDENTIALS} | ||
| dest: /tmp/.gcp-credentials.json | ||
| - src: ${GCP_OIDC_TOKEN_FILE} | ||
| dest: /sandbox/workspace/.gcp-oidc-token | ||
| optional: true | ||
| model: opus | ||
| effort: high | ||
| post_script: scripts/post-link-check.sh | ||
| validation_loop: | ||
| script: scripts/validate-output-schema.sh | ||
| schema: schemas/link-check-result.schema.json | ||
| max_iterations: 2 | ||
| preflight_check: python3 -c "import jsonschema" | ||
| env: | ||
| runner: | ||
| FULLSEND_FORGE: github | ||
| GH_TOKEN: ${GH_TOKEN} | ||
| ISSUE_URL: ${GITHUB_ISSUE_URL} | ||
| sandbox: | ||
| ANTHROPIC_VERTEX_PROJECT_ID: ${ANTHROPIC_VERTEX_PROJECT_ID} | ||
| CLAUDE_CODE_USE_VERTEX: "1" | ||
| CLOUD_ML_REGION: ${CLOUD_ML_REGION} | ||
| FULLSEND_FORGE: github | ||
| GH_TOKEN: ${GH_TOKEN} | ||
| GOOGLE_APPLICATION_CREDENTIALS: /tmp/.gcp-credentials.json | ||
| ISSUE_URL: ${GITHUB_ISSUE_URL} | ||
| timeout_minutes: 15 | ||
| trigger: | | ||
| event.entity.kind == "change_proposal" | ||
| && (event.transition.kind == "opened" | ||
| || event.transition.kind == "synchronized" | ||
| || event.transition.kind == "marked_ready") | ||
| && !event.state.change_proposal.is_fork | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,22 @@ | ||
| --- | ||
| version: 1 | ||
|
|
||
| # Base sandbox policy shared by all agents. | ||
| # | ||
| # Defines non-composable sandbox restrictions: filesystem access, | ||
| # landlock, and process identity. Network access is provided | ||
| # entirely by provider profiles via provider-backed policy | ||
| # composition (ADR 0065). | ||
| # | ||
| # curl is deliberately excluded from all profile binary allowlists | ||
| # to prevent raw HTTP access with the injected GH_TOKEN. | ||
|
|
||
| filesystem_policy: | ||
| include_workdir: true | ||
| read_only: [/usr, /lib, /proc, /dev/urandom, /app, /etc, /var/log] | ||
| read_write: [/sandbox, /tmp, /dev/null] | ||
| landlock: | ||
| compatibility: best_effort | ||
| process: | ||
| run_as_user: sandbox | ||
| run_as_group: sandbox |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| --- | ||
| id: fullsend-github-ro | ||
| display_name: Fullsend GitHub (read-only) | ||
| description: Read-only GitHub API access for fullsend agents | ||
| category: source_control | ||
| endpoints: | ||
| - host: api.github.com | ||
| port: 443 | ||
| protocol: rest | ||
| access: read-only | ||
| enforcement: enforce | ||
| - host: github.com | ||
| port: 443 | ||
| protocol: rest | ||
| access: read-only | ||
| enforcement: enforce | ||
| binaries: | ||
| - "**/gh" | ||
| - "**/node" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,25 @@ | ||
| --- | ||
| id: fullsend-vertex-ai | ||
| display_name: Fullsend Vertex AI | ||
| description: Anthropic API and Google Cloud APIs for inference | ||
| category: inference | ||
| endpoints: | ||
| - host: api.anthropic.com | ||
| port: 443 | ||
| protocol: rest | ||
| access: read-write | ||
| enforcement: enforce | ||
| # See fullsend-openai.yaml: model request bodies that mention the | ||
| # placeholder prefix would otherwise be reset (NVIDIA/OpenShell#2904). | ||
| allow_uninspected_credentials: true | ||
| - host: "*.googleapis.com" | ||
| port: 443 | ||
| protocol: rest | ||
| access: read-write | ||
| enforcement: enforce | ||
| allow_uninspected_credentials: true | ||
|
qodo-code-review[bot] marked this conversation as resolved.
|
||
| binaries: | ||
| - "**/claude" | ||
| - "**/claude.exe" | ||
| - "**/node" | ||
| - "**/pi" | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| --- | ||
| name: github-ro | ||
| type: fullsend-github-ro | ||
| # Workaround: https://github.com/NVIDIA/OpenShell/issues/1978 | ||
| credentials: | ||
| _NOOP_GITHUB_RO: "" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| --- | ||
| name: vertex-ai | ||
| type: fullsend-vertex-ai | ||
| # Workaround: https://github.com/NVIDIA/OpenShell/issues/1978 | ||
| credentials: | ||
| _NOOP_VERTEX_AI: "" |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.