feat(#6966): add a completed reference agent and an authoring skill - #1167
Open
waynesun09 wants to merge 2 commits into
Open
feat(#6966): add a completed reference agent and an authoring skill#1167waynesun09 wants to merge 2 commits into
waynesun09 wants to merge 2 commits into
Conversation
`fullsend agent new` (fullsend-ai/fullsend#6966) generates a valid agent skeleton; these are the two things it hands off to. examples/link-check/ is a complete agent generated by that command and then finished — the tree the generator produces, with the prompt filled in. It lives under examples/ rather than in the fleet directories: those are consumed by URL via `agent add`, and anything placed there reads as a first-class agent. It is deliberately not registered in config.yaml. Being an example is not a reason for it to be unchecked, since it exists to be copied: - skillsaw now runs content rules over examples/**/agents/*.md. It did not before — verified by planting a FILL IN marker and a TODO, which passed `make lint --strict` unnoticed; with the content-paths entry the same plant fails the build. - The post-script gets a test wired into `make test`. It consumes untrusted model output, so it is held to the same standard as a fleet one: the 16 cases cover the field allowlist, non-string coercion, length caps, and ISSUE_URL validation including a foreign host and a path-traversal attempt. skills/authoring-custom-agents/ is the procedure for completing a generated skeleton: replace every marker, keep the output contract, the schema and the post-script naming the same fields, keep tools: matching the body, and leave all mutation to the post-script. It is named apart from the existing agent-scaffolding skill, which is a diagnostic lens for evaluating agent infrastructure rather than an authoring procedure. Assisted-by: Claude Signed-off-by: Wayne Sun <gsun@redhat.com>
PR Summary by QodoAdd reference link-check agent and custom-agent authoring skill
AI Description
Diagram
High-Level Assessment
Files changed (14)
|
|
🤖 Review · ❌ Terminated · Started 4:42 PM UTC · Ended 5:24 PM UTC Commit: |
Code Review by Qodo
1.
|
![qodo-code-review[bot]](https://avatars.githubusercontent.com/in/484649?s=60&v=4){kind=small}
| @@ -0,0 +1,166 @@ | |||
| #!/usr/bin/env bash | |||
There was a problem hiding this comment.
2. Protected paths require human approval 📜 Skill insight § Compliance
This PR modifies protected scripts/ and skills/ paths, so it must receive human governance review even though the linked issue and PR description explain the changes. Automated approval is prohibited for these paths.
Member
Author
There was a problem hiding this comment.
Acknowledged — flagging for human governance review, nothing changed in response. scripts/example-link-check-test.sh is a new test file wired into make test, and skills/authoring-custom-agents/SKILL.md is a new skill; both are additive and neither modifies an existing protected file.
|
🤖 Finished Review · ❌ Failure (validation failed after 2 iteration(s)) · Started 4:42 PM UTC · Completed 5:24 PM UTC Commit: Runtime: claude · Model: opus → claude-opus-4-6 · Effort: high |
…he cap Three fixes to the reference agent, from review. The prompt checked every link in each changed file, so a pre-existing broken link in a touched file surfaced as a finding against whoever touched it. It now extracts links only from lines the pull request adds (`git diff -U0 --diff-filter=AMR base...HEAD -- '*.md'`), and says why: blaming an author for a link they did not write is the fastest way to get an agent's comments ignored. `--diff-filter=AMR` also drops files the pull request deletes, which previously survived the filter and then failed to open at head. `tools:` gains git to match the body. The post-script cut the comment at the 16384-character cap and then appended the truncation marker, overshooting the limit the result schema declares. It now reserves the marker's length: a 17,000-character comment prints at exactly 16384. The fix is in the shared generator template, so it reaches every future generated agent, and the shell test now measures the emitted body rather than only looking for the marker. The example is regenerated with --validation-loop. The CLI default is opt-in, but every schema-backed fleet harness here uses one and the runner has python3 with jsonschema, so the example matches fleet convention — and it exercises the generated preflight_check, which reports a missing dependency before sandbox creation instead of after a full inference run. Assisted-by: Claude Signed-off-by: Wayne Sun <gsun@redhat.com>
Member
Author
|
/fs-review |
|
🤖 Review · Commit: |
|
🤖 Finished Review · ❌ Failure (validation failed after 2 iteration(s)) · Started 5:32 PM UTC · Completed 6:14 PM UTC Commit: Runtime: claude · Model: opus → claude-opus-4-6 · Effort: high · Cost: $7.35 |
rh-hemartin
reviewed
Sep 4, 2026
Comment on lines
+3
to
+5
| Complete reference agents, generated by | ||
| [`fullsend agent new`](https://github.com/fullsend-ai/fullsend/blob/main/docs/cli/agent.md#agent-new) | ||
| and then finished. |
Member
There was a problem hiding this comment.
What does "and then finished" means?
rh-hemartin
reviewed
Sep 4, 2026
| [`fullsend agent new`](https://github.com/fullsend-ai/fullsend/blob/main/docs/cli/agent.md#agent-new) | ||
| and then finished. | ||
|
|
||
| These are **not** fleet agents. They are not registered in this repository's |
rh-hemartin
reviewed
Sep 4, 2026
Comment on lines
+7
to
+11
| These are **not** fleet agents. They are not registered in this repository's | ||
| `config.yaml`, nothing dispatches them, and they are not consumed by URL. They | ||
| exist to show what a completed custom agent looks like, so the shape stays | ||
| honest against this repository's lint and test gates rather than drifting in | ||
| documentation. |
Member
There was a problem hiding this comment.
Isn't this obvious because this is an examples directory?
Member
|
At least publicise the skill in our docs, briefly mention it, we don't need a 200 words paragraph to say "we have a skill to do this we are about to do in this document, use it". |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The two things
fullsend agent newhands off to:examples/link-check/— a complete agent generated by that command and then finished. Areview-role agent that reports Markdown links in changed docs which do not resolve.skills/authoring-custom-agents/— the procedure for completing a generated skeleton.Why
examples/and not the fleet directoriesharness/,agents/,schemas/andscripts/are consumed by URL viaagent add, so anything placed there reads as a first-class agent. Per the #631 discussion, custom and community agents are not mixed with tier-one agents. The example lives in its own subtree and is not registered inconfig.yaml— nothing dispatches it.Being an example is not a reason to be unchecked
It exists to be copied, so an unfilled placeholder or an unsafe post-script in it is exactly as harmful as in a real agent.
skillsaw did not lint
examples/at all. I verified this rather than assuming: planting a<!-- FILL IN -->marker and aTODO:into the example's agent definition and runningmake lintgaveWarnings: 0 / ✓ All checks passed. Addingexamples/**/agents/*.mdtocontent-pathsfixes it — the identical plant now givesWarnings: 1andmake lintexits 1 under--strict.The post-script is tested.
scripts/example-link-check-test.sh, wired intomake test, covers 16 cases against untrusted model output:{"summary": {"nested": true}})statusenum, required fields, the 200-char summary capISSUE_URLvalidation: a foreign host (https://evil.example.com/...) and a path-traversal attempt (.../pull/99/../../x) are both refusedAll run with
POST_LINK_CHECK_DRY_RUN=1, so nothing is ever posted.How to test
To confirm the example is a loadable agent, from a fullsend checkout:
Validated commands
fullsend agent new link-check --role review --on pr-opened --slug fullsend-ai-link-check --no-registerbash scripts/example-link-check-test.shmake lintmake lintwith a plantedFILL IN+TODOWarnings: 1, exit 1 — proves the new scope worksmake check-bundle.src.shbundle)make testfullsend lock link-check --fullsend-dir examples/link-check --offlineDocumented but not executed: a real dispatch of the example agent. That needs GCP credentials, a sandbox image and a live pull request, and the post-script's non-dry-run path comments on a real work item.
Pre-existing
make testflake, not from this changescripts/harness-jira-test.shreports 2–4 failures per run, non-deterministically —yq | grep -qreturns 141 underset -o pipefail. Three consecutive runs of the unchanged test on this branch gave 3, then 4, then 2 failures. The values it reports as missing are present inharness/triage.yaml:This branch touches neither
harness/nor that test.Notes
fullsend agent new— generate a complete custom agent from minimal parameters fullsend#6966. Independent of feat(#6966): addfullsend agent newto generate a complete custom agent fullsend#6972 — neither blocks the other, and the CLI does not fetch anything from here.--strictturns into a failure.authoring-custom-agentsrather thanagent-scaffolding: that name is taken, and the existing skill is a diagnostic lens for evaluating agent infrastructure, not an authoring procedure. The new skill cross-references it rather than overlapping.