From 7c63154d553eca37169f70ebdc198ab8892a98ce Mon Sep 17 00:00:00 2001 From: Wayne Sun Date: Sun, 23 Aug 2026 15:35:04 -0400 Subject: [PATCH 1/9] =?UTF-8?q?docs(#6539):=20one=20local-run=20flow=20?= =?UTF-8?q?=E2=80=94=20pi=20is=20a=20--runtime=20flag=20on=20the=20standar?= =?UTF-8?q?d=20examples?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The guide taught pi as a 350-line parallel walkthrough (hand-rolled pi-hello harness, own config/profile/policy) written before the runtime became a per-run choice (#6526) and before the fleet repo carried the pi plumbing (agents#965/#975). Now there is one flow: the standard fleet examples, plus a "Choosing the runtime" subsection with --runtime pi on the same commands, the override precedence table (both runtimes), the verbatim output of a verified run (fullsend main @ 89bf3b8 against agents main @ 299f264 on macOS and Fedora — harness-default values print bare, overrides carry their source), Gemini-on-Vertex selection by model name, and a compact pi-notes list (build the CLI from main until a release ships pi; image 0.37+ and the exit-127 preflight symptom; review/retro single-context; FULLSEND_PI_PROVIDER and the bash allowlist; fail-closed hooks; --debug='*' and pi-debug.log). The pi platform notes fold into the existing Platform notes as one bullet per OS, and docs/runtimes.md points at the new anchor. Closes #6539 Assisted-by: Claude (code) Signed-off-by: Wayne Sun --- docs/guides/user/running-agents-locally.md | 432 ++++----------------- 1 file changed, 72 insertions(+), 360 deletions(-) diff --git a/docs/guides/user/running-agents-locally.md b/docs/guides/user/running-agents-locally.md index a611c0aad8..fc2397481f 100644 --- a/docs/guides/user/running-agents-locally.md +++ b/docs/guides/user/running-agents-locally.md @@ -202,6 +202,76 @@ fullsend run code \ --env-file fullsend-code.env ``` +### Choosing the runtime + +Every example above runs on **Claude Code, the stable default**. The **pi** +runtime ([pi](https://github.com/earendil-works/pi)) is an experimental, +opt-in alternative — same commands, one extra flag: + +```bash +fullsend run triage \ + --fullsend-dir /tmp/fullsend-agents/ \ + --target-repo /tmp/target-repo/ \ + --env-file fullsend-gcp.env \ + --env-file fullsend-triage.env \ + --runtime pi +``` + +The plan block confirms the choice, and overridden values carry their +source (harness defaults print bare): + +``` + Model: opus + Effort: high + Runtime: pi (from --runtime flag) +... +runtime: selected "pi" from --runtime flag +... +→ Agent: claude-opus-4-6 (v0.84.2) +→ Result: stop + ✓ Agent exited with code 0 (131.9s) +``` + +`metrics.json` records the same (`runtime`, `requested_runtime`, +`runtime_source`, `requested_model`, `override_source`). + +**Per-run overrides** work on both runtimes — precedence is +**flag > environment > config/harness > default**: + +| Override | Flag | Environment | +|----------|------|-------------| +| Runtime (`claude`, `pi`) | `--runtime` | `FULLSEND_RUNTIME` | +| Model (alias, id, or `provider/id` on pi) | `--model` | `FULLSEND_MODEL` (`FULLSEND_PI_MODEL` is a pi-only alias) | +| Effort (`low`…`max`) | `--effort` | `FULLSEND_EFFORT` | +| Fallback chain (Claude Code only) | — | `FULLSEND_FALLBACK_MODELS=a,b` | + +On pi, the model name is also the provider choice — the same Vertex +credentials cover Gemini too: + +```bash +fullsend run triage ... --runtime pi --model google-vertex/gemini-2.5-flash +``` + +**pi-specific notes** (see [Agent runtimes](../../runtimes.md) for the full +security matrix and known constraints): + +- **Build the CLI from `main`** — no released fullsend version includes the + pi runtime yet (`make go-build`). +- The sandbox image must carry pi (`ghcr.io/fullsend-ai/fullsend-sandbox` + 0.37+). A stale image fails preflight with + `pi preflight: pi --version exited 127` — fix with + `podman pull ghcr.io/fullsend-ai/fullsend-sandbox:latest`. +- `review` and `retro` run to schema-valid results but in a **single + context** — pi has no sub-agent tool, so the parallel reviewer roster is + not exercised. +- `FULLSEND_PI_PROVIDER` sets the provider prefix for bare model ids + (default `anthropic-vertex`); `FULLSEND_PI_BASH_ALLOWLIST=enforce` makes + the Bash first-token allowlist block instead of warn. +- Security hooks are fail-closed: a missing or modified hook adapter stops + the run with exit 97 by design; repo-owned `.pi/` content is never loaded. +- Debugging: `--debug='*'` (the `=` is required); sandbox-side failures land + in `pi-debug.log` inside the run directory, not the runner's output. + ### Remote resource flags When your harness references URL-based skills with transitive dependencies, @@ -272,362 +342,6 @@ For GitLab repositories, use `--forge gitlab` instead of `--mint-url`. The agent Status comment behavior is configured via `status_notifications` in `config.yaml`. See [Status Notifications](customizing-agents.md#status-notifications). -## Run a minimal agent on the pi runtime - -The pi runtime (`runtime: pi`) lets you run agents using -[pi](https://github.com/earendil-works/pi) instead of Claude Code inside the -sandbox. You can run a minimal pi agent locally without cloning the -`fullsend-ai/agents` fleet repo — `fullsend run --fullsend-dir` resolves a -config-registered agent to a local harness directory. - -> For background on the pi runtime, its security posture, and known -> constraints, see [Agent runtimes — Pi-specific known -> constraints](../../runtimes/pi.md). - -### Prerequisites (pi-specific) - -In addition to the general [prerequisites](#prerequisites) above, you need: - -| Requirement | Details | -|-------------|---------| -| Sandbox image with pi | `ghcr.io/fullsend-ai/fullsend-sandbox:latest` (must include `PI_VERSION`). Pull the latest to avoid stale cached images — see [Troubleshooting](#troubleshooting-pi-runtime) | -| GCP credentials | A service account key or `gcloud` ADC (`application_default_credentials.json`). The existing [GCP credentials](#get-google-cloud-platform-credentials) section applies — the pi Vertex provider reads the same variables | - -### Directory layout - -A working `--fullsend-dir` for the pi runtime needs the harness, a -`config.yaml` that both registers the agent and selects the runtime, and -supporting files for sandbox credentials and network policy. A bare -`config.yaml` + `harness/` is **not sufficient** — the agent starts but -fails when the sandbox has no credentials or Vertex egress: - -``` -pi-hello/ -├── config.yaml # registers the agent and selects runtime: pi -├── harness/ -│ └── pi-smoke.yaml # agent harness: image, model, host_files, policy -├── agents/ -│ └── pi-smoke.md # agent definition (frontmatter + task prompt) -├── policies/ -│ └── base.yaml # OpenShell sandbox policy (Vertex egress) -├── profiles/ -│ └── fullsend-vertex-ai.yaml # OpenShell egress allowlist for Vertex -├── providers/ -│ └── vertex-ai.yaml # OpenShell inference provider -└── env/ - └── gcp-vertex.env # sandbox-side GCP env vars (expand: true) -``` - -You can copy `policies/`, `profiles/`, `providers/` and `env/` from a -`fullsend-ai/agents` clone, or write them yourself — all four are short, -and their contents are given below so this example stays fleet-free. - -#### `config.yaml` - -The config must both register the harness **and** set `defaults.runtime: pi`. -The two are checked in different places and fail differently: - -- **No `agents:` entry** — placing `harness/pi-smoke.yaml` on disk is not - enough. `resolveAgentSource` looks the agent up in the config and, finding - nothing, fails with `resolving agent "pi-smoke": no config and agents-repo - fallback unavailable`. -- **No `defaults.runtime: pi`** — the run *succeeds* and silently uses the - default `claude` runtime (`backendFromConfigFile` → `ResolveFromConfig` - for this org-style config; a per-repo config resolves through - `ResolveForAgent`, which also honours `runtime:` on the agent's `agents:` entry). - The give-away is the `runtime: selected "claude"` line; pi is never - started. - -```yaml -version: "1" -agents: - - source: harness/pi-smoke.yaml -defaults: - runtime: pi -``` - -#### `harness/pi-smoke.yaml` - -The harness must include `host_files` to deliver GCP credentials into the -sandbox, and reference OpenShell profiles/providers for Vertex egress. -The `--env-file` flag sets the **runner** environment only — sandbox -environment comes from the harness via `env.sandbox` and `host_files` -([ADR 0055](../../ADRs/0055-unified-env-var-delivery.md)): - -```yaml -agent: agents/pi-smoke.md -policy: policies/base.yaml -openshell: - profiles: - - profiles/fullsend-vertex-ai.yaml -providers: - - providers/vertex-ai.yaml - -role: triage -slug: fullsend-ai-pi-smoke -model: haiku -image: ghcr.io/fullsend-ai/fullsend-sandbox:latest - -host_files: - - src: env/gcp-vertex.env - dest: /sandbox/workspace/.env.d/gcp-vertex.env - expand: true - - src: ${GOOGLE_APPLICATION_CREDENTIALS} - dest: /tmp/.gcp-credentials.json -``` - -#### `agents/pi-smoke.md` - -A minimal agent definition with a deterministic task: - -```markdown ---- -name: pi-smoke -description: Minimal smoke-test agent for the pi runtime. -tools: Bash(ls), Write -model: haiku ---- - -You are a smoke-test agent. Do exactly this, then stop: run `ls .` with the -bash tool, then use the write tool to create -`/sandbox/workspace/output/agent-result.json` containing exactly: - -{"action": "sufficient", "reasoning": "Smoke run: pi executed a tool call and wrote this file.", "comment": "pi runtime smoke test - no action needed."} - -Do not read or modify anything else. -``` - -The payload matches the `triage` result schema (`action`, `reasoning`, -`comment`) because the harness declares `role: triage`. This example passes -`--no-post-script`, so nothing validates it — but writing a valid result -keeps the example composable if you drop that flag or reuse the harness for -a real agent. - -#### `env/gcp-vertex.env` - -Sandbox-side GCP environment — these variables reach pi inside the -sandbox: - -```bash -export ANTHROPIC_VERTEX_PROJECT_ID={project-id} -export GOOGLE_CLOUD_PROJECT={project-id} -export CLOUD_ML_REGION=global -export GOOGLE_APPLICATION_CREDENTIALS=/tmp/.gcp-credentials.json -``` - -Two details this file depends on: - -- **`export` is required.** The sandbox sources `.env.d/*.env` with plain `.` - and no `set -a` (`internal/cli/run.go`), so a bare `KEY=value` becomes a - shell variable that pi — a child process — never sees. The symptom is the - Vertex extension disabling itself, or a credentials error, with the file - plainly present in the sandbox. -- **The filename must end in `.env`** — the sourcing loop globs `*.env`, so - `gcp-vertex.conf` would be copied and silently ignored. -- `GOOGLE_APPLICATION_CREDENTIALS` here is the **sandbox** path, matching the - `host_files` `dest` above — not the path on your machine. The `host_files` - entry uses `${GOOGLE_APPLICATION_CREDENTIALS}` from your *runner* shell to - find the key locally. - -#### `policies/base.yaml` - -The sandbox policy. Note the `read_only`/`read_write` prefixes — anything the -agent must read has to sit under one of them, which is why the pi Vertex -extension lives under `/usr/local/share` and not `/opt` (fullsend#6504): - -```yaml ---- -version: 1 -filesystem_policy: - include_workdir: true - read_only: [/usr, /lib, /proc, /dev/urandom, /app, /etc, /var/log] - read_write: [/sandbox, /tmp, /dev/null] -landlock: - compatibility: best_effort -process: - run_as_user: sandbox - run_as_group: sandbox -``` - -#### `profiles/fullsend-vertex-ai.yaml` - -The egress allowlist. Without it the sandbox blocks the inference call and pi -reports a model-not-found error rather than a network error. `**/node` is the -entry that matters for pi; `**/claude` serves the Claude Code runtime: - -```yaml ---- -id: fullsend-vertex-ai -display_name: Fullsend Vertex AI -description: Google Cloud APIs for Vertex AI inference -category: inference -endpoints: - - host: "*.googleapis.com" - port: 443 - protocol: rest - access: read-write - enforcement: enforce -binaries: - - "**/claude" - - "**/node" -``` - -#### `providers/vertex-ai.yaml` - -Binds that profile to the sandbox as an OpenShell provider: - -```yaml ---- -name: vertex-ai -type: fullsend-vertex-ai -credentials: - _NOOP_VERTEX_AI: "" -``` - -### Running the agent - -```bash -fullsend run pi-smoke \ - --fullsend-dir ./pi-hello \ - --target-repo /tmp/target-repo \ - --env-file fullsend-gcp.env \ - --no-post-script \ - --output-dir /tmp/fullsend-out -``` - -On a successful run, you see output like: - -``` -runtime: selected "pi" from ./pi-hello/config.yaml -→ Agent: claude-haiku-4-5 (v0.84.2) -→ Result: stop - Turns: 2 - Tokens: in=5169 out=372 reasoning=140 cache_create=0 cache_read=0 - ✓ Agent exited with code 0 (5.5s) -``` - -The `runtime: selected "pi"` line confirms the pi backend was used. - -Add `--keep-sandbox` when a run fails and you want to inspect the sandbox -afterwards — but delete it when you are done (`openshell sandbox delete -`), since kept sandboxes are not cleaned up for you. - -### Run artifacts - -After a successful run, the output directory contains: - -``` -/tmp/fullsend-out// -├── logs/ -│ ├── openshell-sandbox.log # OCSF events (network, policy decisions) -│ └── openshell-gateway.log -├── iteration-1/ -│ ├── output/ -│ │ └── agent-result.json # whatever the agent wrote to output/ -│ ├── output.jsonl # the agent's raw event stream -│ └── transcripts/ -│ └── -_.jsonl # pi session transcript -├── metrics.json # includes "runtime": "pi" -├── run-telemetry.jsonl -└── security/ # findings.jsonl appears only when a - # security hook actually reports something -``` - -Everything lives under a per-run directory named after the sandbox -(`fs--`), so `--output-dir` accumulates one subdirectory per run -rather than being overwritten. A clean run leaves `security/` empty — that is -the expected result, not a missing artifact. - -Key artifacts to verify: - -- **`metrics.json`** — check `"runtime": "pi"` to confirm the pi backend - was used -- **Session transcript** — the `.jsonl` file under `transcripts/` contains - pi's session events; look for `toolCall` / `toolResult` entries -- **`pi-debug.log`** — appears when `--debug='*'` is passed (note the `=` - syntax — see [Troubleshooting](#troubleshooting-pi-runtime)) - -Use the `analyze-transcript` skill to inspect the session: - -```bash -python3 skills/analyze-transcript/analyze-transcript.py summary \ - /tmp/fullsend-out//iteration-1/transcripts/.jsonl -``` - -``` -Agent: pi-smoke -Model: claude-haiku-4-5 -Messages: 7 (4 user, 3 assistant) -Tokens: 5485 in / 531 out / 0 cache-read / 0 cache-create - -Tool calls: - bash 2 - write 1 - -Stop reasons: toolUse=2, stop=1 -``` - -`tools` and `conversation` are the other two subcommands worth knowing — -`tools` for a per-call table, `conversation` for the readable flow. - -### Pi runtime knobs - -| Variable | Description | -|----------|-------------| -| `FULLSEND_MODEL` (or `fullsend run --model`) | Override the model for the run on any runtime; `FULLSEND_PI_MODEL` is kept as a pi-only alias | -| `FULLSEND_RUNTIME` (or `--runtime`) | Override the runtime selected by `config.yaml` | -| `FULLSEND_EFFORT` (or `--effort`) | Override the harness effort level | -| `FULLSEND_PI_PROVIDER` | Override the inference provider (runner env) | -| `FULLSEND_PI_BASH_ALLOWLIST` | Set to `enforce` to make the Bash first-token allowlist block instead of warn | - -### Security hooks - -Security hooks are enabled by default on pi. The run refuses to start -(exit 97) without the hook adapter — this is intentional (fail-closed). -The runner checks the adapter's SHA-256 before sourcing the agent-writable -`.env`, so a tampered adapter is rejected. - -A planted `.pi/extensions/evil.js` in the target repo is **not** loaded -when `--no-approve` is set (the default in fullsend runs). Pi's -`defaultProjectTrust: never` setting in the sandbox config prevents -repo-owned extensions, skills, and settings from loading. - -### Troubleshooting pi runtime - -**`pi preflight: pi --version exited 127: sh: 1: pi: not found`** -- The sandbox image is stale and predates the pi layers. Pull the latest: - ```bash - podman pull ghcr.io/fullsend-ai/fullsend-sandbox:latest - ``` - -**`[pi-anthropic-vertex] disabled: set GOOGLE_CLOUD_PROJECT ...`** -- The harness is missing `host_files` and/or the OpenShell egress profile. - Sandbox environment comes from the harness, not from `--env-file`. See - the [harness layout](#directory-layout) above. - -**`--debug "..."` fails with `accepts 1 arg(s), received 2`** -- `--debug` is an optional-value flag. Use `--debug='*'` (with `=`), not - `--debug "*"`. - -**Agent fails silently — check `pi-debug.log`** -- When a custom harness is missing `host_files` or the OpenShell profile, - the failure appears in `pi-debug.log` inside the run directory, not in - the runner's terminal output. - -### Platform notes (pi) - -**Linux (Fedora, rootless Podman):** verified end-to-end. The general -[Linux platform notes](#linux) apply. - -**macOS (Apple Silicon):** sandbox creation, the pi bootstrap and preflight, -loading the Vertex extension, and model-id translation are verified on -`darwin/arm64` (macOS 26.5.2, podman machine, `openshell` from Homebrew). -The general [macOS platform notes](#macos) apply. In particular: -- Use `/private/tmp/...` for bind mounts (not `/tmp/...`) -- If the sandbox image architecture differs from the host, set - `FULLSEND_SANDBOX_ARCH` and provide a Linux binary with - `--fullsend-binary` - ## Run from a container Instead of downloading the fullsend binary and installing its host-side @@ -702,16 +416,13 @@ approach. ## Platform notes -> Running the **pi runtime**? Its platform-specific notes live with the rest -> of the pi walkthrough: [Platform notes -> (pi)](#platform-notes-pi). - ### macOS - **Podman machine**: ensure the Podman machine is running (`podman machine start`) before invoking fullsend. The CLI does not start it automatically. - **Podman host-gateway**: if sandbox creation fails with `unable to replace "host-gateway"`, set `host_containers_internal_ip = "192.168.127.254"` under `[containers]` in `~/.config/containers/containers.conf` and restart the Podman machine. - **Architecture mismatch**: if your sandbox image uses a different CPU architecture than the host (e.g. amd64 image on an arm64 Mac via QEMU emulation), set `FULLSEND_SANDBOX_ARCH=amd64` so the CLI downloads the correct binary. This is not needed in the typical setup where the Podman VM matches the host arch. - **Container image**: `--network=host` shares the Podman VM's network namespace, not the Mac's, so a gateway configured at `127.0.0.1` is unreachable from inside the container. Fullsend detects this automatically and redirects the containerized CLI to whichever of `host.containers.internal` (Podman) or `host.docker.internal` (Docker) is actually reachable (fullsend-ai/fullsend#5261) — no manual steps needed. This depends on one of those names resolving inside the container; if neither does, see the **Podman host-gateway** note above. To override the detection yourself, set `OPENSHELL_GATEWAY_ENDPOINT` (e.g. `https://host.containers.internal:17670`) before running the container — an explicit value here is never overwritten. Always use `https://`: check `openshell gateway list`'s `AUTH` column, and if it says `mtls`, OpenShell will present your client certificate to whatever host this points at, so only point it at a gateway you trust. +- **pi runtime**: verified end-to-end on Apple Silicon (podman machine, Homebrew `openshell`); the notes above apply unchanged — use `/private/tmp/...` paths, and `FULLSEND_SANDBOX_ARCH` only if the image arch differs from the host. - **Container image mounts**: bind-mounting `/tmp/...` paths fails with `statfs: no such file or directory` on macOS — Podman Desktop's VM shares `/Users`, `/private`, and `/var/folders` via virtiofs, but not the literal `/tmp` path, and Podman does not resolve the `/tmp` → `/private/tmp` symlink before mounting. Use `/private/tmp/...` (and `$(pwd -P)` instead of `$PWD`). The [container example](#run-from-a-container) above already accounts for this. ### Linux @@ -722,6 +433,7 @@ to the server (gateway). It is likely that you need to bind the gateway to `0.0. `OPENSHELL_BIND_ADDRESS` on `$HOME/.config/openshell/gateway.env` and restart the `openshell-gateway` service. - **SELinux**: on Fedora/RHEL, bind-mounted volumes may need the `:z` suffix for standalone `podman run`. OpenShell handles this automatically. +- **pi runtime**: verified end-to-end on Fedora with rootless Podman; the notes above apply unchanged. ## Troubleshooting From 194fd6c8146912a0fdd55ae89aafc014f20e79f5 Mon Sep 17 00:00:00 2001 From: Wayne Sun Date: Sun, 23 Aug 2026 15:41:58 -0400 Subject: [PATCH 2/9] =?UTF-8?q?docs(#6539):=20review=20round=201=20?= =?UTF-8?q?=E2=80=94=20fix=20the=20runtimes.md=20callout,=20restore=20relo?= =?UTF-8?q?cated=20troubleshooting?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The runtimes.md callout no longer promises a fleet-free walkthrough; the image requirement is stated as 'bakes PI_VERSION' instead of an unsourced 0.37+ tag; the operational knowledge from the deleted section (sandbox env via harness host_files not --env-file, export in .env.d, the disabled-provider symptom, the silent claude fallback and where to see it) lives in a pi subheading under Troubleshooting; the container path carries the same build-from-main caveat; the Gemini example is a full command; old anchors get HTML aliases; run-dir tree notes pi-debug.log; guides index no longer claims released binaries cover pi. Assisted-by: Claude (fix), Grok (review) Signed-off-by: Wayne Sun --- docs/guides/README.md | 2 +- docs/guides/user/running-agents-locally.md | 54 ++++++++++++++++++---- 2 files changed, 47 insertions(+), 9 deletions(-) diff --git a/docs/guides/README.md b/docs/guides/README.md index 1f11633dbf..9430c75353 100644 --- a/docs/guides/README.md +++ b/docs/guides/README.md @@ -34,7 +34,7 @@ Advanced guides for platform operators who deploy and manage the GCP-side infras Guides for developers working in repositories where fullsend is active. - [Bugfix workflow](user/bugfix-workflow.md) — End-to-end guide to how fullsend handles a bug report from issue to merge -- [Running agents locally](user/running-agents-locally.md) — Run fullsend agents on your machine using released binaries (macOS + Linux) +- [Running agents locally](user/running-agents-locally.md) — Run fullsend agents on your machine (macOS + Linux; released binaries for Claude Code, a `main` build for the experimental pi runtime) - [Configuring agent behavior](user/customizing-agents.md) — Harness configurations and `base:` composition for your org and repos - [Configuring with AGENTS.md](user/customizing-with-agents-md.md) — Guide agents using your repo's AGENTS.md file - [Configuring with skills](user/customizing-with-skills.md) — Extend built-in agent skills; [authoring augmentations](user/customizing-with-skills.md#authoring-skills-that-augment-defaults) diff --git a/docs/guides/user/running-agents-locally.md b/docs/guides/user/running-agents-locally.md index fc2397481f..32ef690c23 100644 --- a/docs/guides/user/running-agents-locally.md +++ b/docs/guides/user/running-agents-locally.md @@ -204,6 +204,8 @@ fullsend run code \ ### Choosing the runtime + + Every example above runs on **Claude Code, the stable default**. The **pi** runtime ([pi](https://github.com/earendil-works/pi)) is an experimental, opt-in alternative — same commands, one extra flag: @@ -249,16 +251,29 @@ On pi, the model name is also the provider choice — the same Vertex credentials cover Gemini too: ```bash -fullsend run triage ... --runtime pi --model google-vertex/gemini-2.5-flash +fullsend run triage \ + --fullsend-dir /tmp/fullsend-agents/ \ + --target-repo /tmp/target-repo/ \ + --env-file fullsend-gcp.env \ + --env-file fullsend-triage.env \ + --runtime pi \ + --model google-vertex/gemini-2.5-flash ``` -**pi-specific notes** (see [Agent runtimes](../../runtimes.md) for the full -security matrix and known constraints): - -- **Build the CLI from `main`** — no released fullsend version includes the - pi runtime yet (`make go-build`). -- The sandbox image must carry pi (`ghcr.io/fullsend-ai/fullsend-sandbox` - 0.37+). A stale image fails preflight with +**pi-specific notes** (see [Pi-specific known +constraints](../../runtimes.md#pi-specific-known-constraints-6464) for the +full security matrix; per-run overrides are also covered in +[Choose an agent runtime](../getting-started/choosing-a-runtime.md)): + +- **Build the CLI from `main`** — no released fullsend version includes + the pi runtime yet: clone fullsend and `make go-build` (or + `go run ./cmd/fullsend run …`) instead of the + [release download](#download-the-fullsend-cli) above. The + [container image](#run-from-a-container) tracks releases too, so it has + the same limitation until a release ships pi. +- The sandbox image must include pi (it bakes `PI_VERSION`; any + `ghcr.io/fullsend-ai/fullsend-sandbox` built from `main` after + 2026-08-23 does). A stale image fails preflight with `pi preflight: pi --version exited 127` — fix with `podman pull ghcr.io/fullsend-ai/fullsend-sandbox:latest`. - `review` and `retro` run to schema-valid results but in a **single @@ -460,6 +475,26 @@ to the server (gateway). It is likely that you need to bind the gateway to `0.0. **`unable to replace "host-gateway"` on macOS** - Set `host_containers_internal_ip = "192.168.127.254"` under `[containers]` in `~/.config/containers/containers.conf` and restart the Podman machine +### pi runtime + +- **`pi preflight: pi --version exited 127`** — the sandbox image predates + pi. `podman pull ghcr.io/fullsend-ai/fullsend-sandbox:latest`. +- **`[pi-anthropic-vertex] disabled: set GOOGLE_CLOUD_PROJECT …`** — the + sandbox environment comes from the harness (`host_files`, + `env.sandbox`), not from `--env-file`, which only reaches the runner + process (ADR 0055). Files sourced from `.env.d/` need `export` on each + line. The fleet harnesses already wire this; a custom harness must too. +- **Run used Claude instead of pi** — the runtime falls back to `claude` + when neither the config's `runtime:` nor `--runtime`/`FULLSEND_RUNTIME` + selects pi; the plan block's `Runtime:` line and stderr's + `runtime: selected …` show which one ran and why. +- **`--debug "…"` fails with `accepts 1 arg(s)`** — `--debug` takes an + optional value: write `--debug='*'` (with `=`). +- **Agent fails with nothing in the terminal** — sandbox-side pi failures + land in `pi-debug.log` inside the run directory, next to the + transcripts; kept sandboxes must be removed manually + (`openshell sandbox delete `). + ## Debugging network policies locally When customizing network policies, running agents locally lets you inspect @@ -469,6 +504,9 @@ output to iterate on network policy allowlists. ### Run directory structure +(On pi runs the directory additionally contains `pi-debug.log` — pi's +stderr — next to the transcripts.) + Every `fullsend run` creates a run directory. By default this is under `/tmp/fullsend/`; override it with `--output-dir`: From dab994491b7119f0995eb09b0aea25709b5dd815 Mon Sep 17 00:00:00 2001 From: Wayne Sun Date: Sun, 23 Aug 2026 15:43:32 -0400 Subject: [PATCH 3/9] docs(#6539): reference the sandbox image as v0.37.0+ Assisted-by: Claude (fix) Signed-off-by: Wayne Sun --- docs/guides/user/running-agents-locally.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/guides/user/running-agents-locally.md b/docs/guides/user/running-agents-locally.md index 32ef690c23..cd350e134f 100644 --- a/docs/guides/user/running-agents-locally.md +++ b/docs/guides/user/running-agents-locally.md @@ -271,9 +271,9 @@ full security matrix; per-run overrides are also covered in [release download](#download-the-fullsend-cli) above. The [container image](#run-from-a-container) tracks releases too, so it has the same limitation until a release ships pi. -- The sandbox image must include pi (it bakes `PI_VERSION`; any - `ghcr.io/fullsend-ai/fullsend-sandbox` built from `main` after - 2026-08-23 does). A stale image fails preflight with +- The sandbox image must include pi: + `ghcr.io/fullsend-ai/fullsend-sandbox` **v0.37.0+** (the image bakes + `PI_VERSION`). A stale image fails preflight with `pi preflight: pi --version exited 127` — fix with `podman pull ghcr.io/fullsend-ai/fullsend-sandbox:latest`. - `review` and `retro` run to schema-valid results but in a **single From 277f7b75ce7cde04bea4a52b24fd2b4304999d45 Mon Sep 17 00:00:00 2001 From: Wayne Sun Date: Sun, 23 Aug 2026 16:01:28 -0400 Subject: [PATCH 4/9] docs(#6539): number the runtime-selection steps per the review bot Assisted-by: Claude (fix) Signed-off-by: Wayne Sun --- docs/guides/user/running-agents-locally.md | 56 ++++++++++++---------- 1 file changed, 30 insertions(+), 26 deletions(-) diff --git a/docs/guides/user/running-agents-locally.md b/docs/guides/user/running-agents-locally.md index cd350e134f..2284d14fd5 100644 --- a/docs/guides/user/running-agents-locally.md +++ b/docs/guides/user/running-agents-locally.md @@ -208,34 +208,39 @@ fullsend run code \ Every example above runs on **Claude Code, the stable default**. The **pi** runtime ([pi](https://github.com/earendil-works/pi)) is an experimental, -opt-in alternative — same commands, one extra flag: +opt-in alternative. To run any of the examples on pi: -```bash -fullsend run triage \ - --fullsend-dir /tmp/fullsend-agents/ \ - --target-repo /tmp/target-repo/ \ - --env-file fullsend-gcp.env \ - --env-file fullsend-triage.env \ - --runtime pi -``` +1. Add `--runtime pi` to the same command: + + ```bash + fullsend run triage \ + --fullsend-dir /tmp/fullsend-agents/ \ + --target-repo /tmp/target-repo/ \ + --env-file fullsend-gcp.env \ + --env-file fullsend-triage.env \ + --runtime pi + ``` -The plan block confirms the choice, and overridden values carry their -source (harness defaults print bare): +2. Confirm the selection in the plan block — overridden values carry their + source, harness defaults print bare: -``` - Model: opus - Effort: high - Runtime: pi (from --runtime flag) -... -runtime: selected "pi" from --runtime flag -... -→ Agent: claude-opus-4-6 (v0.84.2) -→ Result: stop - ✓ Agent exited with code 0 (131.9s) -``` + ``` + Model: opus + Effort: high + Runtime: pi (from --runtime flag) + ... + runtime: selected "pi" from --runtime flag + ... + → Agent: claude-opus-4-6 (v0.84.2) + → Result: stop + ✓ Agent exited with code 0 (131.9s) + ``` + + `metrics.json` records the same (`runtime`, `requested_runtime`, + `runtime_source`, `requested_model`, `override_source`). -`metrics.json` records the same (`runtime`, `requested_runtime`, -`runtime_source`, `requested_model`, `override_source`). +3. Optionally override the model or effort for the run (see the table + below) — on pi, the model name is also the provider choice. **Per-run overrides** work on both runtimes — precedence is **flag > environment > config/harness > default**: @@ -247,8 +252,7 @@ runtime: selected "pi" from --runtime flag | Effort (`low`…`max`) | `--effort` | `FULLSEND_EFFORT` | | Fallback chain (Claude Code only) | — | `FULLSEND_FALLBACK_MODELS=a,b` | -On pi, the model name is also the provider choice — the same Vertex -credentials cover Gemini too: +The same Vertex credentials cover Gemini too: ```bash fullsend run triage \ From 9c5c79470b44b43d2988431ee713f666b9026304 Mon Sep 17 00:00:00 2001 From: Wayne Sun Date: Tue, 25 Aug 2026 13:22:13 -0400 Subject: [PATCH 5/9] =?UTF-8?q?docs(#6539):=20rebase=20onto=20#6583=20?= =?UTF-8?q?=E2=80=94=20per-agent=20overrides,=20v0.37.0=20ships=20pi,=20re?= =?UTF-8?q?-point=20the=20Pi=20page?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The Pi runtime page (#6574) replaced the runtimes.md callout this PR rewrote, so its two links now target #choosing-the-runtime. The override table gains the per-agent agents: entry column and the precedence from #6583, with a pointer to the durable per-agent section. v0.37.0 is the first release that carries the pi runtime, so the build-from-main bullet and the guides-index caveat are gone. Assisted-by: Claude Signed-off-by: Wayne Sun --- docs/guides/README.md | 2 +- docs/guides/user/running-agents-locally.md | 81 +++++++++++++++++----- docs/runtimes/pi.md | 8 +-- 3 files changed, 68 insertions(+), 23 deletions(-) diff --git a/docs/guides/README.md b/docs/guides/README.md index 9430c75353..cd7cd383d1 100644 --- a/docs/guides/README.md +++ b/docs/guides/README.md @@ -34,7 +34,7 @@ Advanced guides for platform operators who deploy and manage the GCP-side infras Guides for developers working in repositories where fullsend is active. - [Bugfix workflow](user/bugfix-workflow.md) — End-to-end guide to how fullsend handles a bug report from issue to merge -- [Running agents locally](user/running-agents-locally.md) — Run fullsend agents on your machine (macOS + Linux; released binaries for Claude Code, a `main` build for the experimental pi runtime) +- [Running agents locally](user/running-agents-locally.md) — Run fullsend agents on your machine using released binaries (macOS + Linux; Claude Code by default, `--runtime pi` for the experimental pi runtime) - [Configuring agent behavior](user/customizing-agents.md) — Harness configurations and `base:` composition for your org and repos - [Configuring with AGENTS.md](user/customizing-with-agents-md.md) — Guide agents using your repo's AGENTS.md file - [Configuring with skills](user/customizing-with-skills.md) — Extend built-in agent skills; [authoring augmentations](user/customizing-with-skills.md#authoring-skills-that-augment-defaults) diff --git a/docs/guides/user/running-agents-locally.md b/docs/guides/user/running-agents-locally.md index 2284d14fd5..26c2eb2e2e 100644 --- a/docs/guides/user/running-agents-locally.md +++ b/docs/guides/user/running-agents-locally.md @@ -243,14 +243,59 @@ opt-in alternative. To run any of the examples on pi: below) — on pi, the model name is also the provider choice. **Per-run overrides** work on both runtimes — precedence is -**flag > environment > config/harness > default**: +**flag > environment > the agent's `agents:` entry > repo-wide +config/harness > default**: -| Override | Flag | Environment | -|----------|------|-------------| -| Runtime (`claude`, `pi`) | `--runtime` | `FULLSEND_RUNTIME` | -| Model (alias, id, or `provider/id` on pi) | `--model` | `FULLSEND_MODEL` (`FULLSEND_PI_MODEL` is a pi-only alias) | -| Effort (`low`…`max`) | `--effort` | `FULLSEND_EFFORT` | -| Fallback chain (Claude Code only) | — | `FULLSEND_FALLBACK_MODELS=a,b` | +| Override | Flag | Environment | Per-agent config | +|----------|------|-------------|------------------| +| Runtime (`claude`, `pi`) | `--runtime` | `FULLSEND_RUNTIME` | `runtime:` on the agent's `agents:` entry | +| Model (alias, id, or `provider/id` on pi) | `--model` | `FULLSEND_MODEL` (`FULLSEND_PI_MODEL` is a pi-only alias) | `model:` on the agent's `agents:` entry | +| Effort (`low`…`max`) | `--effort` | `FULLSEND_EFFORT` | `effort:` on the agent's `agents:` entry | +| Fallback chain (Claude Code only) | — | `FULLSEND_FALLBACK_MODELS=a,b` | — | + +**Make it stick per agent.** Flags are for trying things out. To keep a +choice for one agent, put it on that agent's `agents:` entry in the +`config.yaml` the run reads — here the fleet clone, in CI the repo's +`.fullsend/config.yaml` — where it is reviewed with the repo and applies +to every run: + +1. Set it. The entry is validated before anything is written: + + ```bash + fullsend agent set code --fullsend-dir /tmp/fullsend-agents/ \ + --runtime claude --model sonnet + ``` + + Or edit the file by hand. A built-in agent needs only its name; a + custom agent carries the settings on its `source:` entry: + + ```yaml + runtime: pi # repo default for agents that set none + agents: + - name: code + runtime: claude + model: sonnet + ``` + +2. Check it — `fullsend agent list --fullsend-dir /tmp/fullsend-agents/` + shows the settings next to each agent, e.g. + `code harness/code.yaml [runtime=claude model=sonnet]`. + +3. Run as before. The plan block names the entry as the source, and the + per-run flags above still win when you pass them: + + ``` + Runtime: claude (from /tmp/fullsend-agents/config.yaml agents.code) + Model: sonnet (from /tmp/fullsend-agents/config.yaml agents.code) + ``` + +Typos are caught early: `agent set` refuses an invalid value, and +`fullsend run` exits 1 before starting a sandbox (`invalid effort +"turbo": must be one of low, medium, high, xhigh, max`; `"coder" is not +one (did you mean "code"?)`). Names are the agent names you pass to +`fullsend run `, matched case-insensitively. Precedence and how +entries layer over `config.base.yaml`: [Per-agent runtime, model and +effort](../../runtimes.md#per-agent-runtime-model-and-effort-in-configyaml). The same Vertex credentials cover Gemini too: @@ -264,17 +309,17 @@ fullsend run triage \ --model google-vertex/gemini-2.5-flash ``` -**pi-specific notes** (see [Pi-specific known -constraints](../../runtimes.md#pi-specific-known-constraints-6464) for the -full security matrix; per-run overrides are also covered in -[Choose an agent runtime](../getting-started/choosing-a-runtime.md)): - -- **Build the CLI from `main`** — no released fullsend version includes - the pi runtime yet: clone fullsend and `make go-build` (or - `go run ./cmd/fullsend run …`) instead of the - [release download](#download-the-fullsend-cli) above. The - [container image](#run-from-a-container) tracks releases too, so it has - the same limitation until a release ships pi. +**pi-specific notes** (see the [Pi runtime page](../../runtimes/pi.md) +for models, behaviour differences and troubleshooting, the [security +feature matrix](../../contributing/runtime-implementation.md#security-feature-matrix) +for what each hook covers on pi, and [Choose an agent +runtime](../getting-started/choosing-a-runtime.md) for the same overrides +in the setup flow): + +- **fullsend v0.37.0+** is the first release that carries the pi runtime, + so the [release download](#download-the-fullsend-cli) and the + [container image](#run-from-a-container) above both work as-is; an + older binary has no pi runtime to select. - The sandbox image must include pi: `ghcr.io/fullsend-ai/fullsend-sandbox` **v0.37.0+** (the image bakes `PI_VERSION`). A stale image fails preflight with diff --git a/docs/runtimes/pi.md b/docs/runtimes/pi.md index d5c7059e72..3d87d1a6ae 100644 --- a/docs/runtimes/pi.md +++ b/docs/runtimes/pi.md @@ -70,9 +70,9 @@ endpoints answer `FAILED_PRECONDITION` — so region variables are deliberately | Extra knobs | `FULLSEND_PI_PROVIDER` (prefix for bare ids), `FULLSEND_PI_BASH_ALLOWLIST=enforce` | | Not supported | Sub-agents, fallback chains, `plugins:`, Bedrock/Azure providers | -**Running it locally?** See [Run a minimal agent on the pi -runtime](../guides/user/running-agents-locally.md#run-a-minimal-agent-on-the-pi-runtime) — no fleet repo -required. +**Running it locally?** The local-run guide's standard fleet examples take `--runtime pi` — see +[Choosing the runtime](../guides/user/running-agents-locally.md#choosing-the-runtime) for the command, +the per-run and per-agent overrides and the pi-specific notes. ## Behaviour differences worth knowing @@ -117,5 +117,5 @@ provider and model per message. ## See also - [Agent runtimes](../runtimes.md) — choosing and selecting a runtime -- [Running agents locally](../guides/user/running-agents-locally.md#run-a-minimal-agent-on-the-pi-runtime) — a local pi run, no fleet repo required +- [Running agents locally — Choosing the runtime](../guides/user/running-agents-locally.md#choosing-the-runtime) — `--runtime pi` on the standard local-run examples - [pi runtime internals](../contributing/runtime-implementation.md#pi-runtime-internals-6464) — verification provenance and what to re-check on a version bump From d3bcc361ddff9a4a5d6fa97d6a2f829fb1173f27 Mon Sep 17 00:00:00 2001 From: Wayne Sun Date: Tue, 25 Aug 2026 13:46:13 -0400 Subject: [PATCH 6/9] =?UTF-8?q?docs(#6539):=20keep=20the=20local=20guide?= =?UTF-8?q?=20runtime-agnostic=20=E2=80=94=20pi=20details=20move=20to=20th?= =?UTF-8?q?e=20Pi=20page=20(review)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Per rh-hemartin's review: the local-run guide stays basic. "Choosing the runtime" is now one command plus links; the verified pi output, the local requirements, platform notes and pi troubleshooting move to a new "Running it locally" section on docs/runtimes/pi.md (whose links now point at itself, not back at the guide), and the agent set → list → plan-block flow grows the existing CLI line in the per-agent section of docs/runtimes.md with generic paths. The guide gains a See also. Assisted-by: Claude Signed-off-by: Wayne Sun --- docs/guides/README.md | 2 +- docs/guides/user/running-agents-locally.md | 161 ++------------------- docs/runtimes.md | 13 +- docs/runtimes/pi.md | 80 +++++++++- 4 files changed, 105 insertions(+), 151 deletions(-) diff --git a/docs/guides/README.md b/docs/guides/README.md index cd7cd383d1..1f11633dbf 100644 --- a/docs/guides/README.md +++ b/docs/guides/README.md @@ -34,7 +34,7 @@ Advanced guides for platform operators who deploy and manage the GCP-side infras Guides for developers working in repositories where fullsend is active. - [Bugfix workflow](user/bugfix-workflow.md) — End-to-end guide to how fullsend handles a bug report from issue to merge -- [Running agents locally](user/running-agents-locally.md) — Run fullsend agents on your machine using released binaries (macOS + Linux; Claude Code by default, `--runtime pi` for the experimental pi runtime) +- [Running agents locally](user/running-agents-locally.md) — Run fullsend agents on your machine using released binaries (macOS + Linux) - [Configuring agent behavior](user/customizing-agents.md) — Harness configurations and `base:` composition for your org and repos - [Configuring with AGENTS.md](user/customizing-with-agents-md.md) — Guide agents using your repo's AGENTS.md file - [Configuring with skills](user/customizing-with-skills.md) — Extend built-in agent skills; [authoring augmentations](user/customizing-with-skills.md#authoring-skills-that-augment-defaults) diff --git a/docs/guides/user/running-agents-locally.md b/docs/guides/user/running-agents-locally.md index 26c2eb2e2e..14c6abb383 100644 --- a/docs/guides/user/running-agents-locally.md +++ b/docs/guides/user/running-agents-locally.md @@ -206,98 +206,9 @@ fullsend run code \ -Every example above runs on **Claude Code, the stable default**. The **pi** -runtime ([pi](https://github.com/earendil-works/pi)) is an experimental, -opt-in alternative. To run any of the examples on pi: - -1. Add `--runtime pi` to the same command: - - ```bash - fullsend run triage \ - --fullsend-dir /tmp/fullsend-agents/ \ - --target-repo /tmp/target-repo/ \ - --env-file fullsend-gcp.env \ - --env-file fullsend-triage.env \ - --runtime pi - ``` - -2. Confirm the selection in the plan block — overridden values carry their - source, harness defaults print bare: - - ``` - Model: opus - Effort: high - Runtime: pi (from --runtime flag) - ... - runtime: selected "pi" from --runtime flag - ... - → Agent: claude-opus-4-6 (v0.84.2) - → Result: stop - ✓ Agent exited with code 0 (131.9s) - ``` - - `metrics.json` records the same (`runtime`, `requested_runtime`, - `runtime_source`, `requested_model`, `override_source`). - -3. Optionally override the model or effort for the run (see the table - below) — on pi, the model name is also the provider choice. - -**Per-run overrides** work on both runtimes — precedence is -**flag > environment > the agent's `agents:` entry > repo-wide -config/harness > default**: - -| Override | Flag | Environment | Per-agent config | -|----------|------|-------------|------------------| -| Runtime (`claude`, `pi`) | `--runtime` | `FULLSEND_RUNTIME` | `runtime:` on the agent's `agents:` entry | -| Model (alias, id, or `provider/id` on pi) | `--model` | `FULLSEND_MODEL` (`FULLSEND_PI_MODEL` is a pi-only alias) | `model:` on the agent's `agents:` entry | -| Effort (`low`…`max`) | `--effort` | `FULLSEND_EFFORT` | `effort:` on the agent's `agents:` entry | -| Fallback chain (Claude Code only) | — | `FULLSEND_FALLBACK_MODELS=a,b` | — | - -**Make it stick per agent.** Flags are for trying things out. To keep a -choice for one agent, put it on that agent's `agents:` entry in the -`config.yaml` the run reads — here the fleet clone, in CI the repo's -`.fullsend/config.yaml` — where it is reviewed with the repo and applies -to every run: - -1. Set it. The entry is validated before anything is written: - - ```bash - fullsend agent set code --fullsend-dir /tmp/fullsend-agents/ \ - --runtime claude --model sonnet - ``` - - Or edit the file by hand. A built-in agent needs only its name; a - custom agent carries the settings on its `source:` entry: - - ```yaml - runtime: pi # repo default for agents that set none - agents: - - name: code - runtime: claude - model: sonnet - ``` - -2. Check it — `fullsend agent list --fullsend-dir /tmp/fullsend-agents/` - shows the settings next to each agent, e.g. - `code harness/code.yaml [runtime=claude model=sonnet]`. - -3. Run as before. The plan block names the entry as the source, and the - per-run flags above still win when you pass them: - - ``` - Runtime: claude (from /tmp/fullsend-agents/config.yaml agents.code) - Model: sonnet (from /tmp/fullsend-agents/config.yaml agents.code) - ``` - -Typos are caught early: `agent set` refuses an invalid value, and -`fullsend run` exits 1 before starting a sandbox (`invalid effort -"turbo": must be one of low, medium, high, xhigh, max`; `"coder" is not -one (did you mean "code"?)`). Names are the agent names you pass to -`fullsend run `, matched case-insensitively. Precedence and how -entries layer over `config.base.yaml`: [Per-agent runtime, model and -effort](../../runtimes.md#per-agent-runtime-model-and-effort-in-configyaml). - -The same Vertex credentials cover Gemini too: +Every example above runs on **Claude Code**, the default runtime. Fullsend +also has an opt-in **pi** runtime, and any example on this page runs on it +by adding one flag to the same command: ```bash fullsend run triage \ @@ -305,36 +216,14 @@ fullsend run triage \ --target-repo /tmp/target-repo/ \ --env-file fullsend-gcp.env \ --env-file fullsend-triage.env \ - --runtime pi \ - --model google-vertex/gemini-2.5-flash + --runtime pi ``` -**pi-specific notes** (see the [Pi runtime page](../../runtimes/pi.md) -for models, behaviour differences and troubleshooting, the [security -feature matrix](../../contributing/runtime-implementation.md#security-feature-matrix) -for what each hook covers on pi, and [Choose an agent -runtime](../getting-started/choosing-a-runtime.md) for the same overrides -in the setup flow): - -- **fullsend v0.37.0+** is the first release that carries the pi runtime, - so the [release download](#download-the-fullsend-cli) and the - [container image](#run-from-a-container) above both work as-is; an - older binary has no pi runtime to select. -- The sandbox image must include pi: - `ghcr.io/fullsend-ai/fullsend-sandbox` **v0.37.0+** (the image bakes - `PI_VERSION`). A stale image fails preflight with - `pi preflight: pi --version exited 127` — fix with - `podman pull ghcr.io/fullsend-ai/fullsend-sandbox:latest`. -- `review` and `retro` run to schema-valid results but in a **single - context** — pi has no sub-agent tool, so the parallel reviewer roster is - not exercised. -- `FULLSEND_PI_PROVIDER` sets the provider prefix for bare model ids - (default `anthropic-vertex`); `FULLSEND_PI_BASH_ALLOWLIST=enforce` makes - the Bash first-token allowlist block instead of warn. -- Security hooks are fail-closed: a missing or modified hook adapter stops - the run with exit 97 by design; repo-owned `.pi/` content is never loaded. -- Debugging: `--debug='*'` (the `=` is required); sandbox-side failures land - in `pi-debug.log` inside the run directory, not the runner's output. +Everything else about runtimes lives in one place: [Agent +runtimes](../../runtimes.md) for selecting and overriding the runtime, +model and effort — per run, or per agent in `config.yaml` — and +[Pi › Running it locally](../../runtimes/pi.md#running-it-locally) for +what a local pi run needs, its models and its troubleshooting. ### Remote resource flags @@ -486,7 +375,6 @@ approach. - **Podman host-gateway**: if sandbox creation fails with `unable to replace "host-gateway"`, set `host_containers_internal_ip = "192.168.127.254"` under `[containers]` in `~/.config/containers/containers.conf` and restart the Podman machine. - **Architecture mismatch**: if your sandbox image uses a different CPU architecture than the host (e.g. amd64 image on an arm64 Mac via QEMU emulation), set `FULLSEND_SANDBOX_ARCH=amd64` so the CLI downloads the correct binary. This is not needed in the typical setup where the Podman VM matches the host arch. - **Container image**: `--network=host` shares the Podman VM's network namespace, not the Mac's, so a gateway configured at `127.0.0.1` is unreachable from inside the container. Fullsend detects this automatically and redirects the containerized CLI to whichever of `host.containers.internal` (Podman) or `host.docker.internal` (Docker) is actually reachable (fullsend-ai/fullsend#5261) — no manual steps needed. This depends on one of those names resolving inside the container; if neither does, see the **Podman host-gateway** note above. To override the detection yourself, set `OPENSHELL_GATEWAY_ENDPOINT` (e.g. `https://host.containers.internal:17670`) before running the container — an explicit value here is never overwritten. Always use `https://`: check `openshell gateway list`'s `AUTH` column, and if it says `mtls`, OpenShell will present your client certificate to whatever host this points at, so only point it at a gateway you trust. -- **pi runtime**: verified end-to-end on Apple Silicon (podman machine, Homebrew `openshell`); the notes above apply unchanged — use `/private/tmp/...` paths, and `FULLSEND_SANDBOX_ARCH` only if the image arch differs from the host. - **Container image mounts**: bind-mounting `/tmp/...` paths fails with `statfs: no such file or directory` on macOS — Podman Desktop's VM shares `/Users`, `/private`, and `/var/folders` via virtiofs, but not the literal `/tmp` path, and Podman does not resolve the `/tmp` → `/private/tmp` symlink before mounting. Use `/private/tmp/...` (and `$(pwd -P)` instead of `$PWD`). The [container example](#run-from-a-container) above already accounts for this. ### Linux @@ -497,7 +385,6 @@ to the server (gateway). It is likely that you need to bind the gateway to `0.0. `OPENSHELL_BIND_ADDRESS` on `$HOME/.config/openshell/gateway.env` and restart the `openshell-gateway` service. - **SELinux**: on Fedora/RHEL, bind-mounted volumes may need the `:z` suffix for standalone `podman run`. OpenShell handles this automatically. -- **pi runtime**: verified end-to-end on Fedora with rootless Podman; the notes above apply unchanged. ## Troubleshooting @@ -524,26 +411,6 @@ to the server (gateway). It is likely that you need to bind the gateway to `0.0. **`unable to replace "host-gateway"` on macOS** - Set `host_containers_internal_ip = "192.168.127.254"` under `[containers]` in `~/.config/containers/containers.conf` and restart the Podman machine -### pi runtime - -- **`pi preflight: pi --version exited 127`** — the sandbox image predates - pi. `podman pull ghcr.io/fullsend-ai/fullsend-sandbox:latest`. -- **`[pi-anthropic-vertex] disabled: set GOOGLE_CLOUD_PROJECT …`** — the - sandbox environment comes from the harness (`host_files`, - `env.sandbox`), not from `--env-file`, which only reaches the runner - process (ADR 0055). Files sourced from `.env.d/` need `export` on each - line. The fleet harnesses already wire this; a custom harness must too. -- **Run used Claude instead of pi** — the runtime falls back to `claude` - when neither the config's `runtime:` nor `--runtime`/`FULLSEND_RUNTIME` - selects pi; the plan block's `Runtime:` line and stderr's - `runtime: selected …` show which one ran and why. -- **`--debug "…"` fails with `accepts 1 arg(s)`** — `--debug` takes an - optional value: write `--debug='*'` (with `=`). -- **Agent fails with nothing in the terminal** — sandbox-side pi failures - land in `pi-debug.log` inside the run directory, next to the - transcripts; kept sandboxes must be removed manually - (`openshell sandbox delete `). - ## Debugging network policies locally When customizing network policies, running agents locally lets you inspect @@ -553,9 +420,6 @@ output to iterate on network policy allowlists. ### Run directory structure -(On pi runs the directory additionally contains `pi-debug.log` — pi's -stderr — next to the transcripts.) - Every `fullsend run` creates a run directory. By default this is under `/tmp/fullsend/`; override it with `--output-dir`: @@ -689,3 +553,10 @@ curl -sf https://api.example.com/healthz diff <(grep DENIED run-1/logs/openshell-sandbox.log) \ <(grep DENIED run-2/logs/openshell-sandbox.log) ``` + +## See also + +- [Agent runtimes](../../runtimes.md) — choosing a runtime and overriding runtime, model and effort per run or per agent +- [Pi › Running it locally](../../runtimes/pi.md#running-it-locally) — what a local pi run needs, its models and troubleshooting +- [fullsend run](../../cli/run.md) — the full flag reference +- [Configuring agent behavior](customizing-agents.md) — harness configuration and `base:` composition diff --git a/docs/runtimes.md b/docs/runtimes.md index 6496103730..a85294b20a 100644 --- a/docs/runtimes.md +++ b/docs/runtimes.md @@ -111,7 +111,18 @@ agents: model: haiku ``` -Or from the CLI: `fullsend agent set code --runtime claude --model sonnet --effort high`. +Or from the CLI, which validates the entry before writing it: + +1. `fullsend agent set code --fullsend-dir .fullsend --runtime claude --model sonnet --effort high` +2. `fullsend agent list --fullsend-dir .fullsend` shows the settings next to each agent — + `code (built-in) [runtime=claude model=sonnet effort=high]`, or the `source:` path for a custom + agent. +3. The next `fullsend run code` names the entry as the source: `Runtime: claude (from + agents.code)` — and a `--runtime`/`--model` flag on that run still wins. + +An invalid value is refused before the write — `invalid effort "turbo": must be one of low, medium, +high, xhigh, max` — and the same check runs on every `fullsend run`, so a hand-edited entry fails the +run before a sandbox starts rather than being skipped. A `source:` entry needs no `name:` — the agent's name is derived from the source file (`harness/lint.yaml` → `lint`, ADR 0058), and that is the name the settings, `fullsend run lint` diff --git a/docs/runtimes/pi.md b/docs/runtimes/pi.md index 3d87d1a6ae..28854f39b7 100644 --- a/docs/runtimes/pi.md +++ b/docs/runtimes/pi.md @@ -70,9 +70,65 @@ endpoints answer `FAILED_PRECONDITION` — so region variables are deliberately | Extra knobs | `FULLSEND_PI_PROVIDER` (prefix for bare ids), `FULLSEND_PI_BASH_ALLOWLIST=enforce` | | Not supported | Sub-agents, fallback chains, `plugins:`, Bedrock/Azure providers | -**Running it locally?** The local-run guide's standard fleet examples take `--runtime pi` — see -[Choosing the runtime](../guides/user/running-agents-locally.md#choosing-the-runtime) for the command, -the per-run and per-agent overrides and the pi-specific notes. +## Running it locally + +Complete [Running agents locally](../guides/user/running-agents-locally.md) first — the CLI, +OpenShell, credentials and the fleet clone are the same. Every example there runs on pi by adding +`--runtime pi` to the same command: + +```bash +fullsend run triage \ + --fullsend-dir /tmp/fullsend-agents/ \ + --target-repo /tmp/target-repo/ \ + --env-file fullsend-gcp.env \ + --env-file fullsend-triage.env \ + --runtime pi +``` + +The plan block confirms the selection — overridden values carry their source, harness defaults +print bare — and `metrics.json` records the same (`runtime`, `runtime_source`, `requested_model`, +`override_source`): + +``` + Model: opus + Effort: high + Runtime: pi (from --runtime flag) +... +runtime: selected "pi" from --runtime flag +... +→ Agent: claude-opus-4-6 (v0.84.2) +→ Result: stop + ✓ Agent exited with code 0 (131.9s) +``` + +Pick a model the same way — on pi the model name is also the provider choice, and the same Vertex +credentials cover Gemini: + +```bash +fullsend run triage ... --runtime pi --model google-vertex/gemini-3.7-flash +``` + +To keep an agent on pi (or off it) without passing flags every time, set `runtime:`/`model:` on +its `agents:` entry in `config.yaml` — see [per-agent settings](../runtimes.md#per-agent-runtime-model-and-effort-in-configyaml). + +What a local pi run needs, beyond the guide: + +- **fullsend v0.37.0+** — the first release that carries the pi runtime; the release download + and the container image both work as-is. +- **A sandbox image that includes pi** — `ghcr.io/fullsend-ai/fullsend-sandbox` v0.37.0+ (the image + bakes `PI_VERSION`). A stale image fails preflight with `pi preflight: pi --version exited 127`; + `podman pull ghcr.io/fullsend-ai/fullsend-sandbox:latest` fixes it. +- **Platforms** — verified end to end on macOS Apple Silicon (podman machine, Homebrew `openshell`) + and Fedora with rootless Podman; the guide's platform notes apply unchanged. +- **`review` and `retro`** complete with schema-valid results but in a single context — pi has no + sub-agent tool, so the parallel reviewer roster is not exercised (see [Not yet exercised](#not-yet-exercised)). +- **Knobs** — `FULLSEND_PI_PROVIDER` sets the provider for bare model ids (default + `anthropic-vertex`); `FULLSEND_PI_BASH_ALLOWLIST=enforce` makes the Bash first-token allowlist + block instead of warn. +- **Security hooks are fail-closed** — a missing or modified hook adapter stops the run with exit + 97 by design; repo-owned `.pi/` content is never loaded. +- **Debugging** — `--debug='*'` (the `=` is required); sandbox-side failures land in `pi-debug.log` + inside the run directory, next to the transcripts, not in the runner's output. ## Behaviour differences worth knowing @@ -109,6 +165,22 @@ table above, not a shared one. **403 `PERMISSION_DENIED` on a Vertex call.** The credentials work but the model is not enabled in that project's Model Garden, or the provider resolved a different project than you expect. +**`[pi-anthropic-vertex] disabled: set GOOGLE_CLOUD_PROJECT ...`.** The sandbox environment comes +from the harness (`host_files`, `env.sandbox`), not from `--env-file`, which only reaches the runner +process (ADR 0055). Files sourced from `.env.d/` need `export` on each line. The fleet harnesses +already wire this; a custom harness must too. + +**The run used Claude instead of pi.** The runtime falls back to `claude` when neither the config's +`runtime:` (repo-wide or on the agent's `agents:` entry) nor `--runtime`/`FULLSEND_RUNTIME` selects +pi; the plan block's `Runtime:` line and stderr's `runtime: selected ...` show which one ran and why. + +**`--debug "..."` fails with `accepts 1 arg(s)`.** `--debug` takes an optional value: write +`--debug='*'` (with `=`). + +**The agent fails with nothing in the terminal.** Sandbox-side pi failures land in `pi-debug.log` +inside the run directory, next to the transcripts; kept sandboxes must be removed manually +(`openshell sandbox delete `). + **The model says it is a different model than you selected.** Do not trust the reply — a model asked about itself will often repeat whatever the conversation history said. `metrics.json` records the model that actually served the run, and the session JSONL under `transcripts/` records the @@ -117,5 +189,5 @@ provider and model per message. ## See also - [Agent runtimes](../runtimes.md) — choosing and selecting a runtime -- [Running agents locally — Choosing the runtime](../guides/user/running-agents-locally.md#choosing-the-runtime) — `--runtime pi` on the standard local-run examples +- [Running agents locally](../guides/user/running-agents-locally.md) — the local-run flow that [Running it locally](#running-it-locally) builds on - [pi runtime internals](../contributing/runtime-implementation.md#pi-runtime-internals-6464) — verification provenance and what to re-check on a version bump From 642b458298eea7c10f1566c47fba87befe099046 Mon Sep 17 00:00:00 2001 From: Wayne Sun Date: Tue, 25 Aug 2026 13:49:14 -0400 Subject: [PATCH 7/9] docs(#6539): reconcile the review/retro-on-pi wording on the Pi page Assisted-by: Claude Signed-off-by: Wayne Sun --- docs/runtimes/pi.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/docs/runtimes/pi.md b/docs/runtimes/pi.md index 28854f39b7..c39eb6cab2 100644 --- a/docs/runtimes/pi.md +++ b/docs/runtimes/pi.md @@ -148,9 +148,10 @@ What a local pi run needs, beyond the guide: ## Not yet exercised `runtime: pi` is selectable and has been run end to end, but no **fleet lifecycle** run on Vertex is -recorded yet. Pilot on a disposable org with `triage`/`prioritize` before `code`/`fix`. `review` and -`retro` are unsupported — they need sub-agents, and would run in a single context without per-persona -models. `extension_error` events are not mapped. +recorded yet. Pilot on a disposable repo with `triage`/`prioritize` before `code`/`fix`. `review` and +`retro` run to schema-valid results, but in a **single context**: pi has no sub-agent tool, so the +parallel persona roster and its per-persona models are never exercised — treat them as unsupported +for that purpose. `extension_error` events are not mapped. ## Troubleshooting From 3d6d93815f4be5227c92a6c5fe6dc0a8e36d9e24 Mon Sep 17 00:00:00 2001 From: Wayne Sun Date: Tue, 25 Aug 2026 13:55:25 -0400 Subject: [PATCH 8/9] docs(#6539): keep the code span on one line so VitePress does not escape it Assisted-by: Claude Signed-off-by: Wayne Sun --- docs/runtimes.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/docs/runtimes.md b/docs/runtimes.md index a85294b20a..cc0935fe28 100644 --- a/docs/runtimes.md +++ b/docs/runtimes.md @@ -117,8 +117,9 @@ Or from the CLI, which validates the entry before writing it: 2. `fullsend agent list --fullsend-dir .fullsend` shows the settings next to each agent — `code (built-in) [runtime=claude model=sonnet effort=high]`, or the `source:` path for a custom agent. -3. The next `fullsend run code` names the entry as the source: `Runtime: claude (from - agents.code)` — and a `--runtime`/`--model` flag on that run still wins. +3. The next `fullsend run code` names the entry as the source — + `Runtime: claude (from agents.code)` — and a `--runtime`/`--model` flag on that + run still wins. An invalid value is refused before the write — `invalid effort "turbo": must be one of low, medium, high, xhigh, max` — and the same check runs on every `fullsend run`, so a hand-edited entry fails the From cb50b8f04398981569e4d8d7650f34a5cb8a2bb5 Mon Sep 17 00:00:00 2001 From: Wayne Sun Date: Tue, 25 Aug 2026 14:00:43 -0400 Subject: [PATCH 9/9] docs(#6539): drop the dot from the per-agent heading so its anchor resolves on the site VitePress slugs "config.yaml" as "config-yaml" while GitHub drops the dot, so every link to #per-agent-runtime-model-and-effort-in-configyaml landed at the top of the Runtimes page on the published site. The heading is now "Per-agent runtime, model and effort" (identical slug on both), with the config.yaml mention moved into the first sentence; the four links follow. Assisted-by: Claude Signed-off-by: Wayne Sun --- docs/cli/agent.md | 2 +- docs/cli/run.md | 2 +- docs/guides/getting-started/choosing-a-runtime.md | 2 +- docs/runtimes.md | 2 +- docs/runtimes/pi.md | 2 +- 5 files changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/cli/agent.md b/docs/cli/agent.md index 04b106013c..972f29f9e1 100644 --- a/docs/cli/agent.md +++ b/docs/cli/agent.md @@ -99,7 +99,7 @@ fullsend agent set triage --fullsend-dir .fullsend --model xai-vertex/xai/grok-4 | `--model` | Model for this agent — an alias, a model id, or `provider/id` on pi | | `--effort` | Effort level for this agent (`low`, `medium`, `high`, `xhigh`, `max`) | -See [Runtimes — per-agent settings](../runtimes.md#per-agent-runtime-model-and-effort-in-configyaml) for precedence. +See [Runtimes — per-agent settings](../runtimes.md#per-agent-runtime-model-and-effort) for precedence. ## `agent remove` diff --git a/docs/cli/run.md b/docs/cli/run.md index aacb999841..7390b3ebd4 100644 --- a/docs/cli/run.md +++ b/docs/cli/run.md @@ -48,7 +48,7 @@ The **Runtime** line shows which runtime was selected and the config source it w ## Runtime selection -The runtime for a run is resolved once, in this order: `--runtime` flag, `FULLSEND_RUNTIME`, `runtime:` on the agent's `agents:` entry in `config.yaml` / `.fullsend/config.yaml`, the repo-wide `runtime:` there, then the built-in `claude`. The same order applies to the model (`--model`, `FULLSEND_MODEL`, `model:` on the agent's `agents:` entry, harness `model:`, agent frontmatter; `FULLSEND_PI_MODEL` is a lower-precedence alias on pi) and to effort (`--effort`, `FULLSEND_EFFORT`, `effort:` on the agent's `agents:` entry, harness `effort:`). `` is the name given to `fullsend run` (`triage`, `code`, …); see [Runtimes — per-agent settings](../runtimes.md#per-agent-runtime-model-and-effort-in-configyaml). `FULLSEND_FALLBACK_MODELS=a,b` becomes Claude Code's `--fallback-model`; pi ignores it with a warning. +The runtime for a run is resolved once, in this order: `--runtime` flag, `FULLSEND_RUNTIME`, `runtime:` on the agent's `agents:` entry in `config.yaml` / `.fullsend/config.yaml`, the repo-wide `runtime:` there, then the built-in `claude`. The same order applies to the model (`--model`, `FULLSEND_MODEL`, `model:` on the agent's `agents:` entry, harness `model:`, agent frontmatter; `FULLSEND_PI_MODEL` is a lower-precedence alias on pi) and to effort (`--effort`, `FULLSEND_EFFORT`, `effort:` on the agent's `agents:` entry, harness `effort:`). `` is the name given to `fullsend run` (`triage`, `code`, …); see [Runtimes — per-agent settings](../runtimes.md#per-agent-runtime-model-and-effort). `FULLSEND_FALLBACK_MODELS=a,b` becomes Claude Code's `--fallback-model`; pi ignores it with a warning. The plan block prints `Runtime: (from )` and, when an override applied, `Model: (from )`; stderr carries `runtime: selected "" from ` (and `model: requested "" from `) for scripts. A value from the config file is labelled with the file path, suffixed ` agents.` when the agent's entry decided. An invalid override (unknown runtime, unknown effort level, an `agents:` entry that names no agent) fails before the sandbox is created. diff --git a/docs/guides/getting-started/choosing-a-runtime.md b/docs/guides/getting-started/choosing-a-runtime.md index dc614efb3c..d4a3f446f7 100644 --- a/docs/guides/getting-started/choosing-a-runtime.md +++ b/docs/guides/getting-started/choosing-a-runtime.md @@ -20,7 +20,7 @@ Fullsend supports multiple agent runtimes. A runtime is the program that runs in ## When and how the runtime is selected 1. **Next step — Configuring GitHub.** `fullsend github setup ` asks which runtime to use when run from a terminal; press Enter to keep `claude`. Passing `--runtime` skips the prompt. The setup PR it opens records the choice in `.fullsend/config.yaml` and describes how to change it. Nothing runs on this page — continue with [Configuring GitHub](configuring-github.md). -2. **Later — changing it.** Edit `runtime:` in the repo's `.fullsend/config.yaml` (the setup PR shows the key), or re-run `fullsend github setup --runtime `. To put one agent on a different runtime or model than the rest — say `code` on Claude Code while `triage` runs Grok on pi — set `runtime:` on that agent's `agents:` entry in the same file (`fullsend agent set code --runtime claude`); see [Runtimes — per-agent settings](../../runtimes.md#per-agent-runtime-model-and-effort-in-configyaml). Fleets managed through `repos.yaml` set `defaults.runtime` (or a per-entry `runtime`) — `fullsend repos set-default defaults.runtime pi` — and run `fullsend repos install`; see [fullsend repos](../../cli/repos.md). +2. **Later — changing it.** Edit `runtime:` in the repo's `.fullsend/config.yaml` (the setup PR shows the key), or re-run `fullsend github setup --runtime `. To put one agent on a different runtime or model than the rest — say `code` on Claude Code while `triage` runs Grok on pi — set `runtime:` on that agent's `agents:` entry in the same file (`fullsend agent set code --runtime claude`); see [Runtimes — per-agent settings](../../runtimes.md#per-agent-runtime-model-and-effort). Fleets managed through `repos.yaml` set `defaults.runtime` (or a per-entry `runtime`) — `fullsend repos set-default defaults.runtime pi` — and run `fullsend repos install`; see [fullsend repos](../../cli/repos.md). 3. **Per run — trying without changing the repo.** `fullsend run --runtime pi --model google-vertex/gemini-2.5-flash`, or the `FULLSEND_RUNTIME` / `FULLSEND_MODEL` / `FULLSEND_EFFORT` environment variables (flag beats environment beats the agent's `agents:` entry beats repo-wide config). In CI the same names work as repository variables. Reference: [fullsend run](../../cli/run.md) and [Runtimes — selecting and overriding](../../runtimes.md#selecting-a-runtime-and-model). ## Where to see what ran diff --git a/docs/runtimes.md b/docs/runtimes.md index cc0935fe28..473beb5d25 100644 --- a/docs/runtimes.md +++ b/docs/runtimes.md @@ -91,7 +91,7 @@ In CI these are repository variables of the same name, plain or role-prefixed the agent's `agents:` entry in `.fullsend/config.yaml` instead. Harness `env.runner` does **not** reach the `fullsend` process. -### Per-agent runtime, model and effort in config.yaml +### Per-agent runtime, model and effort The `agents:` list is the per-agent place in `config.yaml`: an entry names an agent and can set its `runtime`, `model` and `effort`. A built-in agent (`triage`, `code`, `review`, `fix`, `retro`, diff --git a/docs/runtimes/pi.md b/docs/runtimes/pi.md index c39eb6cab2..7843d84a3b 100644 --- a/docs/runtimes/pi.md +++ b/docs/runtimes/pi.md @@ -109,7 +109,7 @@ fullsend run triage ... --runtime pi --model google-vertex/gemini-3.7-flash ``` To keep an agent on pi (or off it) without passing flags every time, set `runtime:`/`model:` on -its `agents:` entry in `config.yaml` — see [per-agent settings](../runtimes.md#per-agent-runtime-model-and-effort-in-configyaml). +its `agents:` entry in `config.yaml` — see [per-agent settings](../runtimes.md#per-agent-runtime-model-and-effort). What a local pi run needs, beyond the guide: