docs(custom): add PATH modification example

[rh-hemartin]

• Add a section documenting how to modify the PATH when you add new tools.

Closes #2883

[qodo-code-review]

PR Summary by Qodo

Document how to modify PATH for custom tools via host_files

📝 Documentation 🕐 Less than 5 minutes

AI Description

• Add a walkthrough for injecting new executables and PATH changes into the sandbox.
• Show using host_files to stage a script and an .env.d file.
• Clarify why env.sandbox cannot be used to modify PATH.

Diagram

graph TD
  A["Harness config: host_files"] --> B["Stage tool: /tmp/bin/my-tool.sh"] --> D["Agent execution"] --> F["PATH includes /tmp/bin"]
  A --> C["Stage env: .env.d/path-modification.env"] --> E["Workspace .env sources .env.d/*"] --> D

Loading

High-Level Assessment

The chosen approach (documenting host_files + .env.d sourcing) is the most practical and least surprising for users because it matches the existing sandbox initialization behavior and avoids relying on restricted environment variables like PATH in env.sandbox.

Files changed (1) +35 / -0

Documentation (1) +35 / -0

customizing-agents.mdAdd example for modifying PATH via host_files and .env.d +35/-0

Add example for modifying PATH via host_files and .env.d

• Introduces a new “Modifying the PATH” section showing how to copy an executable into the sandbox and prepend its directory to PATH using an '.env.d' file. Adds an explanatory note that 'env.sandbox' cannot be used for PATH changes due to variable restrictions.

docs/guides/user/customizing-agents.md

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 6:46 AM UTC · Completed 6:57 AM UTC
Commit: 82f560a · View workflow run →

[rh-hemartin]

cc @ralphbean as stated in the docs env.sandbox is preventing me from modifying the PATH (reservedSandboxKeys) but I can modify them with the environment files. I think I can make the agent malfunction using env files instead of env.sandbox and I think we need to let the users modify the PATH, so maybe there is a followup here to allow PATH to be set on env.sandbox.

[github-actions]

Site preview

Preview: https://7cf3db48-site.fullsend-ai.workers.dev

Commit: 82f560ac9a3fed11418375dd88487e3d4374015d

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[qodo-code-review]

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📜 Skill insights (0)

Context used

✅ Compliance rules (platform): 54 rules

✅ Skills: writing-user-docs, writing-adrs

1. Upload to /tmp/bin fails ✓ Resolved 🐞 Bug ≡ Correctness

Description

The new example copies scripts/my-tool.sh to /tmp/bin/my-tool.sh, but the runner only
pre-creates /sandbox/workspace/bin (not /tmp/bin) before copying host_files, so the upload can
fail and the recipe won’t work. The runner already prepends /sandbox/workspace/bin to PATH, so
using that location avoids both the directory-creation and PATH-modification pitfalls.

Code

docs/guides/user/customizing-agents.md[R306-327]

+```yaml
+host_files:
+  - src: scripts/my-tool.sh
+    dest: /tmp/bin/my-tool.sh
+  - src: env/path-modification.env
+    dest: /sandbox/workspace/.env.d/path-modification.env
+    expand: false  # expand works outside the runner, so we set it to false
+```
+
+The `scripts/my-tool.sh` file:
+
+```bash
+#!/bin/bash
+
+echo "Hello from $0"
+```
+
+And the `env/path-modification.env`:
+
+```bash
+PATH=/tmp/bin:$PATH
+```

Relevance

⭐⭐⭐ High

Team accepts doc fixes to match actual runner/bootstrap behavior; emphasis on runnable examples (PR
#1087, #372).

PR-#1087
PR-#372

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

The runner only creates /sandbox/workspace/bin and /sandbox/workspace/.env.d during bootstrap,
and UploadFile does not create parent directories. Additionally, PATH is already initialized with
/sandbox/workspace/bin first, making /tmp/bin unnecessary for the “new executable” case.

internal/cli/run.go[1198-1205]
internal/cli/run.go[1421-1427]
internal/sandbox/sandbox.go[454-506]
docs/guides/user/customizing-agents.md[301-334]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The doc example copies an executable to `/tmp/bin`, but the runner does not create `/tmp/bin` before `host_files` upload, so the copy can fail and the example is non-functional.

## Issue Context
The runner creates `/sandbox/workspace/bin` and also prepends it to `PATH` during bootstrap.

## Fix Focus Areas
- docs/guides/user/customizing-agents.md[301-334]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

2. PATH procedure lacks numbered steps ✓ Resolved 📜 Skill insight ✧ Quality

Description

The new ### Modifying the PATH section describes a procedure using prose and code blocks, but it
does not present the procedure as a numbered (ordered) list of steps. This makes the procedure
harder to follow and violates the guide requirement to express procedures as numbered steps.

Code

docs/guides/user/customizing-agents.md[R303-334]

+To modify the agent's `PATH` so it gains access to new executables,
+use the harness' `host_files` key:
+
+```yaml
+host_files:
+  - src: scripts/my-tool.sh
+    dest: /tmp/bin/my-tool.sh
+  - src: env/path-modification.env
+    dest: /sandbox/workspace/.env.d/path-modification.env
+    expand: false  # expand works outside the runner, so we set it to false
+```
+
+The `scripts/my-tool.sh` file:
+
+```bash
+#!/bin/bash
+
+echo "Hello from $0"
+```
+
+And the `env/path-modification.env`:
+
+```bash
+PATH=/tmp/bin:$PATH
+```
+
+The agent execution sources the special file  `/sandbox/workspace/.env` which
+sources all the files under `/sandbox/workspace/.env.d/`, including your
+`path-modification.env` that includes `/tmp/bin/` into the `PATH`.
+
+**Note**: the harness' `env.sandbox` does not work to modify the `PATH` as it
+rejects numerous special variables to avoid the malfunction of the agent.

Relevance

⭐⭐⭐ High

Same doc guide previously updated to numbered procedure steps; suggestion accepted in PR #2663.

PR-#2663

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

PR Compliance ID 1062079 requires procedures in documentation guides to use numbered steps rather
than narrative prose. In docs/guides/user/customizing-agents.md, the added `### Modifying the
PATH` procedure is presented as paragraphs with embedded code blocks, with no ordered list steps.

docs/guides/user/customizing-agents.md[303-334]
Skill: writing-user-docs

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The `### Modifying the PATH` section is procedural but written as prose paragraphs instead of a numbered step list.

## Issue Context
This file is a documentation guide under `docs/guides/`, and procedural content in guides must be written as numbered (ordered) steps.

## Fix Focus Areas
- docs/guides/user/customizing-agents.md[303-334]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

3. Overbroad env.d sourcing claim ✓ Resolved 🐞 Bug ⚙ Maintainability

Description

The docs say /sandbox/workspace/.env “sources all the files under /sandbox/workspace/.env.d/”,
but bootstrap only sources files matching *.env. Users who follow the prose and use a different
extension will be silently ignored.

Code

docs/guides/user/customizing-agents.md[R329-331]

+The agent execution sources the special file  `/sandbox/workspace/.env` which
+sources all the files under `/sandbox/workspace/.env.d/`, including your
+`path-modification.env` that includes `/tmp/bin/` into the `PATH`.

Relevance

⭐⭐⭐ High

Bootstrap sources only .env.d/*.env; team often corrects docs to match code paths (PR #2582).

PR-#2582

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

The bootstrap shell loop sources only .env.d/*.env, not arbitrary files in .env.d/, so the
documentation should reflect the filename requirement.

internal/cli/run.go[1470-1472]
docs/guides/user/customizing-agents.md[329-331]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The documentation implies all files in `.env.d/` are sourced, but the implementation only sources `*.env` files.

## Issue Context
Bootstrap uses a glob over `.env.d/*.env`.

## Fix Focus Areas
- docs/guides/user/customizing-agents.md[329-331]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

[fullsend-ai-review]

Review

Verdict: Approve

Docs-only PR adding a well-structured section on custom executables and PATH modification to docs/guides/user/customizing-agents.md. All four technical claims were verified against the codebase:

• /sandbox/workspace/bin is on the sandbox PATH (internal/cli/run.go:1634)
• .env.d/*.env files are sourced at startup (internal/cli/run.go:1683)
• env.sandbox reserved-key handling correctly skips PATH with a warning (internal/cli/run.go:1577-1609)
• host_files works with just src/dest — expand defaults to false (internal/harness/harness.go:36-41)

The prior review's four low-severity findings (technical-accuracy on "rejects" wording, unnecessary expand: false, indentation inconsistency, extra blank line) are all addressed in this revision.

No findings across correctness, security, intent/coherence, style, or docs-currency dimensions.

Previous run

Review

Findings

Low

• [technical-accuracy] docs/guides/user/customizing-agents.md:352 — The note states env.sandbox "cannot modify PATH — the harness rejects special variables to protect sandbox operation." The actual behavior (internal/cli/run.go) is that reserved keys like PATH are silently skipped with a stderr warning (WARNING: env.sandbox key %q is reserved for runner infrastructure; skipping), not rejected in a way that halts processing or produces an error. Saying "rejects" somewhat overstates the behavior, though the end result (the key has no effect) is correctly conveyed.
  Remediation: Revise to: "env.sandbox cannot override PATH — it is a reserved infrastructure variable and any attempt to set it is silently skipped (with a stderr warning). Use a .env.d file as shown above instead."

• [technical-accuracy] docs/guides/user/customizing-agents.md:345 — expand: false is described as "required here" but false is the zero value and default for the Expand field (internal/harness/harness.go, HostFile struct with omitempty). Omitting expand entirely produces identical behavior. Saying it is "required" may confuse users into thinking the default is true.
  Remediation: Clarify that expand must not be set to true for this use case, or note that false is the default and the explicit setting is shown for clarity.

• [indentation] docs/guides/user/customizing-agents.md:347 — Inconsistent indentation in list item continuation text. The third line of the expand: false explanation paragraph has 2 spaces of indentation while the first two lines have 3 spaces, which may affect Markdown rendering under some parsers.
  Remediation: Change from 2 spaces to 3 spaces to match the surrounding continuation lines.

• [blank-lines] docs/guides/user/customizing-agents.md:323 — Extra blank line before h4 heading. The existing convention in this file uses one blank line before h4 headings, but there are two blank lines before #### Advanced: modifying the PATH for external toolchains.
  Remediation: Remove one of the two blank lines.

Previous run

Review

Findings

Low

• [technical-accuracy] docs/guides/user/customizing-agents.md:352 — The note states env.sandbox "cannot modify PATH — the harness rejects special variables to protect sandbox operation." The actual behavior (internal/cli/run.go) is that reserved keys like PATH are silently skipped with a stderr warning (WARNING: env.sandbox key %q is reserved for runner infrastructure; skipping), not rejected in a way that halts processing or produces an error. Saying "rejects" somewhat overstates the behavior, though the end result (the key has no effect) is correctly conveyed.
  Remediation: Revise to: "env.sandbox cannot override PATH — it is a reserved infrastructure variable and any attempt to set it is silently skipped (with a stderr warning). Use a .env.d file as shown above instead."

• [technical-accuracy] docs/guides/user/customizing-agents.md:345 — expand: false is described as "required here" but false is the zero value and default for the Expand field (internal/harness/harness.go, HostFile struct with omitempty). Omitting expand entirely produces identical behavior. Saying it is "required" may confuse users into thinking the default is true.
  Remediation: Clarify that expand must not be set to true for this use case, or note that false is the default and the explicit setting is shown for clarity.

• [indentation] docs/guides/user/customizing-agents.md:347 — Inconsistent indentation in list item continuation text. The third line of the expand: false explanation paragraph has 2 spaces of indentation while the first two lines have 3 spaces, which may affect Markdown rendering under some parsers.
  Remediation: Change from 2 spaces to 3 spaces to match the surrounding continuation lines.

• [blank-lines] docs/guides/user/customizing-agents.md:323 — Extra blank line before h4 heading. The existing convention in this file uses one blank line before h4 headings, but there are two blank lines before #### Advanced: modifying the PATH for external toolchains.
  Remediation: Remove one of the two blank lines.

Previous run (2)

Review

Findings

Medium

• [technical-accuracy] docs/guides/user/customizing-agents.md:333 — The note states the harness' env.sandbox "rejects numerous special variables to avoid the malfunction of the agent." Per internal/cli/run.go:1395-1397, reserved keys (including PATH) are skipped with a stderr warning (WARNING: env.sandbox key %q is reserved for runner infrastructure; skipping). The word "rejects" is reasonable shorthand, but "numerous special variables" is vague — the actual behavior is a specific reserved-keys list, and the skip emits a visible warning.
  Remediation: Revise to: "Note: the harness' env.sandbox cannot modify the PATH because PATH is a reserved infrastructure variable. Attempts to set PATH via env.sandbox are skipped with a stderr warning."

---

Labels: Docs-only PR documenting harness host_files and env.sandbox behavior for PATH customization.

[waynesun09]

Review squad pass (Claude + Claude + Gemini, 3 agents). All findings verified against internal/cli/run.go on main. 3 MEDIUM findings not already covered by qodo's existing comments on this PR — mostly about the doc teaching a more complex PATH-modification pattern than the built-in /sandbox/workspace/bin mechanism, plus one inaccurate explanation of expand: false.

[fullsend-ai-review]

🤖 Review · ⚠️ Cancelled · Started 2:33 PM UTC · Ended 2:37 PM UTC
Commit: bd28f41 · View workflow run →

[fullsend-ai-review]

🤖 Review · ⚠️ Cancelled · Started 2:38 PM UTC · Ended 2:40 PM UTC
Commit: bd28f41 · View workflow run →

[fullsend-ai-review]

🤖 Review · ⚠️ Cancelled · Started 2:41 PM UTC · Ended 2:45 PM UTC
Commit: bd28f41 · View workflow run →

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 2:45 PM UTC · Completed 2:58 PM UTC
Commit: 99d7fc9 · View workflow run →

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 7:22 AM UTC · Completed 7:28 AM UTC
Commit: 1808158 · View workflow run →

[fullsend-ai-retro]

🤖 Finished Retro · ✅ Success · Started 8:04 AM UTC · Completed 8:13 AM UTC
Commit: 1808158 · View workflow run →

[fullsend-ai-retro]

PR #2882 was a docs-only change adding a section on modifying the PATH for custom executables in the sandbox. The review agent initially approved on July 2 with zero findings, but missed three significant issues later caught by Qodo (automated, July 2) and human reviewer waynesun09 (July 13, 11 days later): (1) the docs taught a complex two-file .env.d workaround when the simpler built-in /sandbox/workspace/bin mechanism already existed, (2) the docs referenced /tmp/bin which the runner doesn't pre-create, and (3) the expand: false explanation was misleading. Notably, the review agent verified that /sandbox/workspace/bin is on PATH in run.go:1634 but failed to connect this fact to the docs teaching an unnecessarily complex alternative. After the human review, the author force-pushed 4 times in 12 minutes causing 3 cancelled review runs. The PR was eventually restructured to lead with the simpler /sandbox/workspace/bin pattern. Three proposals filed — one evidence issue for the existing approach-proportionality gap (#3639), and two new proposals for the agents repo.

Proposals filed

• Correctness sub-agent should verify that file paths in documentation examples are reachable in the target runtime (in fullsend-ai/agents)
• Correctness sub-agent should flag docs that present default config values as required settings (in fullsend-ai/agents)

Evidence notes (not filed as issues)

• Evidence for Review agent intent-coherence sub-agent should evaluate approach proportionality #3639: review agent approved docs teaching complex workaround over simpler built-in mechanism on PR docs(custom): add PATH modification example #2882 (fullsend-ai/fullsend): On PR #2882, a docs-only change adding a 'Modifying the PATH' section to docs/guides/user/customizing-agents.md, the review agent ran on July 2 (run 28570996116) and approved with zero findings. The initial docs version taught users a two-file .env.d sourcing pattern (copy a script to /tmp/bin, then create an .env.d env file to prepend that directory to PATH) to make a custom executable available in the sandbox.