Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
0937beb
feat(#6966): add role table, trigger presets and spec parsing for age…
waynesun09 Sep 3, 2026
6148b32
feat(#6966): render a complete agent tree from validated options
waynesun09 Sep 3, 2026
7a9f031
feat(#6966): validate a generated agent tree before it is registered
waynesun09 Sep 3, 2026
f8e6c99
feat(#6966): add `fullsend agent new`
waynesun09 Sep 3, 2026
8ede72d
docs(#6966): document `agent new` and fix the slash-command trigger
waynesun09 Sep 3, 2026
55260f4
fix(#6966): derive the slug when generating inside a git worktree
waynesun09 Sep 3, 2026
9706bb6
fix(#6966): close the write-path and validation gaps from review
waynesun09 Sep 3, 2026
c041654
test(#6966): cover the generator from its own packages
waynesun09 Sep 3, 2026
3dafa13
docs(#6966): record the decision in architecture.md and tidy the ADR
waynesun09 Sep 3, 2026
d64d6d8
docs(#6966): order the BYO prerequisites ahead of step 0
waynesun09 Sep 3, 2026
68ecfc7
fix(#6966): keep a truncated comment inside the declared cap
waynesun09 Sep 3, 2026
70458b9
test(#6966): trim golden trees to two and drop duplicated CLI tests
waynesun09 Sep 4, 2026
c3e2c30
docs(#6966): say what things are instead of naming them
waynesun09 Sep 4, 2026
0bbdf3d
fix(#6966): the generated post-script never found its own input
waynesun09 Sep 4, 2026
2216d54
fix(#6966): generated schema forbids a multi-line summary
waynesun09 Sep 4, 2026
366e64a
docs(#6966): show a real local run of a generated agent
waynesun09 Sep 4, 2026
c73568c
docs(#6966): add the claude-runtime run beside the dummy one
waynesun09 Sep 4, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions docs/ADRs/0058-agent-registration.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,3 +94,11 @@ in the sections below.
- [ADR 0045](0045-forge-portable-harness-schema.md) -- harness composition via `base:` URLs
- [ADR 0057](0057-repos-management.md) -- repos management for per-repo installations
- [Bring Your Own Agent](../guides/user/bring-your-own-agent.md) -- user-facing guide for agent registration

## Notes

- `fullsend agent new` (added for
[#6966](https://github.com/fullsend-ai/fullsend/issues/6966)) generates a
Comment thread
qodo-code-review[bot] marked this conversation as resolved.
complete custom agent and then registers it through the `agent add` path
described here. It adds a generator in front of this decision; the
registration model itself is unchanged.
127 changes: 127 additions & 0 deletions docs/ADRs/0102-generate-custom-agents-from-the-cli.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,127 @@
---
title: "102. Generate custom agents from the CLI"
status: Accepted
Comment thread
qodo-code-review[bot] marked this conversation as resolved.
relates_to:
- agent-architecture
topics:
- cli
- harness
- onboarding
---

# 102. Generate custom agents from the CLI

Date: 2026-09-03

## Status

Accepted

## Context

Building a custom agent from scratch is the most common piece of negative user
feedback on fullsend. The
[Bring Your Own Agent guide](../guides/user/bring-your-own-agent.md) asks the
author to hand-write a harness with roughly thirty fields, an agent definition
whose `tools:` must stay consistent with its body, a result schema, a
post-script that safely turns model output into a forge mutation, a CEL
`trigger`, and a `policies/base.yaml` that a per-repo install does not vendor.

Most of those mistakes are invisible until the first dispatch after merge
(#6830). Three of them are invisible even then:

- A harness with no `trigger:` registers, validates, and appears in
`fullsend agent list`, and is then skipped by dispatch with a bare `continue`
and no annotation — while resolve and load failures both emit `::error::`.
None of the seven fleet harnesses has a `trigger:` to copy from, because the
fleet is dispatched by stage workflows instead.
- A `role:` the hosted mint does not serve is only regex-checked locally and
surfaces as an opaque `403` at run time (#6563).
- A `policy:` or provider path naming a file that is not there fails at run
time, or worse degrades to a warning and a sandbox that cannot reach Vertex
(#6834).

[#4839](https://github.com/fullsend-ai/fullsend/issues/4839) records a July
decision to prefer guides over a CLI for this. That decision was taken in Slack
and was never written up as an ADR, so this is a new decision rather than a
superseding one. Two things have changed since: user feedback that the guides
are not enough, and the precedent of
[GitHub Agentic Workflows](https://github.com/github/gh-aw), which solved the
same onboarding problem with `gh aw new` — a generator that writes a minimal
valid source, leaving the user to edit prose rather than plumbing.

## Options

**Improve the guides further.** Cheapest, and the status quo. But the failure
modes above are not comprehension failures — a reader who understands the guide
perfectly still cannot tell that an omitted `trigger:` means the agent will
never run, because nothing reports it.

**Ship the stub templates from `fullsend-ai/agents` and have the CLI fetch and
pin them like `agent add` does.** Keeps template changes on the agents repo's
release cadence. Rejected for the first cut: it makes the generator's happy
path depend on network and `allowed_remote_resources` state, which reintroduces
the "fails only at first dispatch" class this change exists to remove, and it
creates a merge-order dependency between the two repositories. Revisitable
later as an opt-in `--template-ref`.

**Generate from templates embedded in the CLI.** Chosen.

## Decision

Add `fullsend agent new <name>`, which writes a complete, valid, registered
agent from a minimum of parameters and validates the result before returning.

Four properties make it worth having rather than being a scaffolding
convenience:

1. **A trigger is mandatory.** The command refuses to write a trigger-less
harness, and `--on` presets emit expressions taken verbatim from the CEL
reference — a test asserts the generator and the documentation stay the same
text. The `command:` and `pr-opened` presets both refuse events from forks,
which matters because the default trigger is attached to every generated
agent, including agents with role `coder`.
2. **The role table is closed and checked.** `--role` accepts the five fleet
roles the hosted mint serves. It is hardcoded rather than derived from
`mintcore.BuiltInRoles()`, because derivation would re-admit `scribe` —
which `config.ValidRoles()` deliberately excludes as a mint-only dogfood
role — and would fail open for any future canonical role with no provider
pairing. A unit test asserts each row still matches
`mintcore.RolePermissionsFor`.
3. **The generator writes what a per-repo install does not vendor.**
`policies/base.yaml`, and the providers and profiles the chosen role needs,
are written when absent and never overwritten. Providers are referenced by
path rather than bare name, because the embedded provider fallback fills in
only the OpenAI provider.
4. **The result is validated in process.** Everything is rendered into a
scratch directory and loaded through the same loader dispatch uses, so a
harness that would fail validation never leaves a partially written
`.fullsend` behind.

Templates are embedded with `go:embed`, matching how the repository already
ships scaffold content.

## Consequences

- Creating a working custom agent becomes one command, and the three
silently-fatal mistakes above become generation-time errors with actionable
text. The Bring Your Own Agent guide keeps its hand-written path as the
explanation of what was generated.
- Harness shape is now encoded in a second place. Mitigated but not
eliminated: the harness is built as a `harness.Harness` value and marshalled
rather than formatted as text, so the generator cannot emit a field the
validator does not know about; golden tests pin the generated bytes; and the
role table has a drift test against the mint. A new *required* harness field
would still need a matching change here, and the golden tests are what would
catch it.
- `ValidateRunnerEnvWith` is not run at generation time — it requires every
`${VAR}` in the harness `env` blocks to be set in the calling process, which
is true in CI and false on a developer's machine — so an unset variable
still surfaces at `fullsend run`.
- The default sandbox image digests are compiled in, so they are repinned by
hand on the same cadence as the fleet repin PRs. A golden test makes a repin
visible in review.
- `fullsend lock` and `fullsend run` are unchanged and do not share the
generator's check helper: they interleave minting, runner-env validation and
`${VAR}` expansion between the same steps in different orders, so sharing one
helper would change their behaviour.
1 change: 1 addition & 0 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -396,6 +396,7 @@ the inheritance model: fullsend defaults, then repo baseline (`config.base.yaml`
- Runtime resolution: `fullsend run <name>` resolves agents in two tiers: (1) config entries from `OrgConfig.Agents` (highest priority), (2) runtime fallback to the `fullsend-ai/agents` repository for known first-party agents not in config. The agents-repo fallback is a transitional mechanism for the agent extraction; it will be removed once all users have migrated to config-driven registration (ADR 0058 Phase 5).
- Config lookup: config entries are looked up directly via `findConfigAgentEntry`; the agents-repo fallback operates independently when the agent is not found in config. Builds on [ADR 0045](ADRs/0045-forge-portable-harness-schema.md) harness identity model.
- CLI management: `fullsend agent add|list|set|update|remove` manages config entries and auto-pins URLs to a commit SHA with an integrity hash.
- Agent generation: `fullsend agent new <name>` writes a complete custom agent — harness, agent definition, result schema, post-script, and the policy, providers and profiles a per-repo install does not vendor — validates it with the loader dispatch uses, and registers it through the `agent add` path above. A `trigger:` is mandatory, because a trigger-less harness registers and validates and is then silently never dispatched; `--role` is a closed table of the roles the hosted mint serves, so an unservable role fails locally rather than as a `403` at first dispatch ([ADR 0102](ADRs/0102-generate-custom-agents-from-the-cli.md)).

**Open questions:**

Expand Down
Loading
Loading