Skip to content

fix(#6412): limit sandbox name to 19 characters for OpenShell - #6416

Merged
maruiz93 merged 3 commits into
mainfrom
agent/6412-sandbox-name-limit
Aug 21, 2026
Merged

fix(#6412): limit sandbox name to 19 characters for OpenShell#6416
maruiz93 merged 3 commits into
mainfrom
agent/6412-sandbox-name-limit

Conversation

@fullsend-ai-coder

Copy link
Copy Markdown
Contributor

Summary

  • Replace sandbox name generation from agent-<name>-<pid>-<timestamp> (30+ chars) with a SHA-256-based fs-<16-hex-chars> format (19 chars) to comply with OpenShell's 19-character sandbox name limit
  • Add maxSandboxNameLen constant and generateSandboxName() helper function
  • Add tests verifying name length, prefix format, and uniqueness

Context

OpenShell enforces a maximum sandbox name length of 19 characters (validation source). The previous naming scheme (agent-triage-12345-1724166487) exceeded this limit, blocking usage with newer OpenShell versions.

Testing

  • TestGenerateSandboxName_Length — verifies name ≤ 19 characters
  • TestGenerateSandboxName_Prefix — verifies fs- prefix
  • TestGenerateSandboxName_Uniqueness — generates 50 names and verifies no duplicates
  • All tests pass with -race

Closes #6412

Post-script verification

  • Branch is not main/master (agent/6412-sandbox-name-limit)
  • Secret scan passed (gitleaks — b8b3598d5cffdbebdfc73d499400c65ea6d32de3..HEAD)
  • PR body secret scan passed (gitleaks — no-git)

@fullsend-ai-coder
fullsend-ai-coder Bot requested a review from a team as a code owner August 20, 2026 16:13
@fullsend-ai-coder fullsend-ai-coder Bot added the ready-for-review Agent PR ready for human review label Aug 20, 2026
@fullsend-ai-review

fullsend-ai-review Bot commented Aug 20, 2026

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 4:15 PM UTC · Completed 4:31 PM UTC

Commit: a583ea2 · View workflow run →

@fullsend-ai-review

fullsend-ai-review Bot commented Aug 20, 2026

Copy link
Copy Markdown

Review

Findings

Medium

  • [stale-reference] docs/guides/infrastructure/eval-measurements.md:228 — The doc describes platform telemetry location using only the legacy naming format agent-<name>-<pid>-<unix>. This PR changes the default naming to fs-<slug>-<hex>, making the documentation incomplete for readers trying to locate run directories created by the current version. Both formats are supported at runtime, but only the legacy format is documented.
    Remediation: Update the doc to document both naming schemes.

Low

  • [duplicated-logic-divergence-risk] internal/evalmeasure/find.go — The agentSlug function is duplicated across internal/cli/run.go and internal/evalmeasure/find.go with a deliberate behavioral difference: the cli version returns "unk" for empty input, while the evalmeasure version returns "". The comment says "the two must stay in sync" but no automated mechanism enforces this. Consider extracting into a shared internal package or adding a parity test.

  • [edge-case] internal/evalmeasure/find.go — When wantAgent consists entirely of non-alphanumeric characters, agentSlug returns "unk", which could match directories created by agent "unknown". Unlikely in practice since agent names are alphanumeric.

  • [error-handling-idiom] internal/cli/run.go:909 — Error message uses descriptive sentence style rather than the codebase's more common lowercase, context-prefixed format for similar validation errors.

Previous run

Review

Findings

Low

  • [duplicated logic / sync risk] internal/evalmeasure/find.go:82agentSlug is duplicated between internal/cli/run.go and internal/evalmeasure/find.go. The comment acknowledges the duplication ("the two must stay in sync") and the behavioral difference for empty names is intentional. For non-empty inputs the functions are byte-identical. Consider exporting from one package or adding a cross-package test as follow-up.

  • [dead code / unreachable guard] internal/cli/run.go:910 — The length check if len(sandboxName) > maxSandboxNameLen is unreachable because generateSandboxName always produces exactly 19 characters (slug is always 3 chars, hash fills the remainder). Serves as defense-in-depth against future refactoring.

  • [constant placement] internal/cli/run.go:74maxSandboxNameLen is placed in the grouped const block far from generateSandboxName. Moving it near the function would follow the file's locality pattern.

  • [stale-reference] docs/guides/infrastructure/eval-measurements.md:228 — Documentation describes only the legacy agent-<name>-<pid>-<unix> naming format. Consider updating to mention the new fs-<slug>-<hex> scheme alongside the legacy format.

Previous run (2)

Review

Findings

High

  • [stale-reference] internal/evalmeasure/find.go:19 — The hostRunDirPattern regex (^agent-(.+)-([0-9]+)-([0-9]+)$) expects the old agent-<name>-<pid>-<unix> sandbox directory naming format. After this PR, run directories will be named fs-<slug>-<hex> (e.g., fs-tri-a1b2c3d4e5f6), which will never match this regex. FindPlatformTelemetry(outputDir, agent) will fail to discover platform telemetry in child run directories, silently breaking the eval-measure --output-dir pipeline. The removed comment on the old line 900 explicitly documented this coupling.
    Remediation: Update hostRunDirPattern to also match the new fs-<slug>-<hex> format, or replace the regex-based approach with a more flexible directory discovery mechanism. Update tests in internal/evalmeasure/find_test.go and internal/cli/evalmeasure_test.go that hardcode the old directory format.

Medium

  • [agent-filtering-broken] internal/evalmeasure/find.go:77 — The evalmeasure pipeline filters run directories by agent name using the regex capture group m[1]. The new naming scheme only embeds a 3-character slug (rev for both review and reverse, cod for both code and coder), making agent-name-based filtering impossible from the directory name alone.
    Remediation: Either embed the full lowercased agent name in the sandbox directory name, or store the agent name in a metadata file inside the run directory that evalmeasure can read.

Low

  • [stale-reference] internal/cli/evalmeasure_test.go:76 — Tests TestEvalMeasureCmd_OutputDirIgnoresNestedTelemetry and TestEvalMeasureCmd_LocalFullsendDirManifestProducesJSONL create run directories using the old agent-<name>-<pid>-<unix> format. They will continue to pass but are not testing the format that will be produced by the updated fullsend run.

  • [dead-code] internal/cli/run.go:1020 — The guard if len(sandboxName) > maxSandboxNameLen is unreachable because generateSandboxName always produces names of exactly 19 characters.

  • [naming-convention] internal/cli/run_test.go:5487 — Table-driven test uses tt.agent as the t.Run subtest key instead of a dedicated name field, inconsistent with 9 other table-driven tests in the file. The empty-string case produces an unnamed subtest.


Next steps:

  • /fs-fix — agent addresses review findings automatically
  • /fs-fix <your instruction> — agent fixes with your specific guidance
  • Push commits directly — review re-runs automatically on push
  • /fs-fix-stop — disable automatic fix runs for this PR
Previous run (3)

Review

Findings

High

  • [stale-reference] internal/evalmeasure/find.go:19hostRunDirPattern regex (^agent-(.+)-([0-9]+)-([0-9]+)$) matches the old sandbox naming scheme but will never match the new fs-<slug>-<hex> format. FindPlatformTelemetry uses this regex to discover child run directories under the CI output base. After this PR, new runs create directories named like fs-tri-a1b2c3d4e5f6 which won't match, causing eval-measure --output-dir to silently miss platform telemetry files. The fallback path (direct run-telemetry.jsonl check) only fires when outputDir IS the runDir, not the CI output base — so the common CI path is broken. The agent-name extraction from capture group m[1] also needs rethinking since the new format embeds a 3-char slug, not the full agent name.
    Remediation: Update hostRunDirPattern to also match the new fs-<slug>-<hex> format (e.g., ^fs-[a-z0-9]{3}-[0-9a-f]+$), or add a second pattern and check both in findChildPlatformTelemetry. Update the agent-name filter logic and corresponding tests in find_test.go and evalmeasure_test.go.

Medium

  • [stale-reference] docs/guides/infrastructure/eval-measurements.md:228 — Documents the host run directory format as agent-<name>-<pid>-<unix>. After this PR, the format changes to fs-<slug>-<hex> (19 chars max). This reference will no longer match actual directory names produced by fullsend run.
    Remediation: Update to reflect the new format and note that older runs may still use the legacy agent-<name>-<pid>-<unix> pattern.

Low

  • [naming-collision] internal/cli/run.go — The new function agentSlug overlaps with the established slug concept in this package (h.Slug field, discover_slugs.go). The new function produces a 3-character abbreviation, which is semantically different from the harness identity slug.
    Remediation: Consider renaming to agentNameAbbrev, agentTag, or sandboxPrefix to avoid confusion with the established slug terminology.
  • [dead-code] internal/cli/run.go:1019 — The len(sandboxName) > maxSandboxNameLen guard can never fire because generateSandboxName always produces exactly 19 characters by construction (fs- + 3-char slug + - + 12-char hex = 19). Additionally, this check executes after the run directory has already been created at line ~909 using the sandbox name, so if it somehow fired, a directory with the too-long name would already exist on disk uncleaned.
    Remediation: Move the length check immediately after generateSandboxName returns (before os.MkdirAll), or convert to a unit test assertion.
  • [stale-example] skills/replay-session/SKILL.md:45 — Uses agent-code-2446-1776439672/ as an example directory name for downloaded artifacts. New runs will use the fs-<slug>-<hex> format.
    Remediation: Update the example or note both old and new formats may appear.
  • [stale-example] skills/analyze-transcript/SKILL.md:102 — Shows agent-<type>-<id>/ as the directory structure pattern for downloaded artifacts. New runs will produce fs-<slug>-<hex>/ directories.
    Remediation: Update the pattern or show both formats.

Next steps:

  • /fs-fix — agent addresses review findings automatically
  • /fs-fix <your instruction> — agent fixes with your specific guidance
  • Push commits directly — review re-runs automatically on push
  • /fs-fix-stop — disable automatic fix runs for this PR
Previous run (4)

Review

Findings

Critical

  • [scope-gap] internal/evalmeasure/find.go:19 — The hostRunDirPattern regex (^agent-(.+)-([0-9]+)-([0-9]+)$) is hard-coded to match the old sandbox naming scheme. After this PR, sandbox names use the format fs-<16-hex-chars>, which will never match this regex. FindPlatformTelemetry will silently return nil for all runs using the new naming, causing eval-measurement telemetry to be silently lost with zero error signal.
    Remediation: Update hostRunDirPattern in internal/evalmeasure/find.go to also match the new fs-<hex> naming format (e.g., ^fs-[0-9a-f]{16}$). Alternatively, adopt a format-agnostic discovery mechanism.

Medium

  • [scope-gap] internal/cli/run.go:903 — The old sandbox name format embedded the agent name (agent-<name>-...), which evalmeasure used to filter telemetry by agent (via wantAgent parameter). The new fs-<hex> format discards the agent name entirely, making agent-scoped telemetry filtering impossible even if the regex is updated.
    Remediation: Either preserve the agent name in the new name format, or refactor evalmeasure.FindPlatformTelemetry to use a different mechanism for agent-scoped discovery.

  • [test-inadequate] internal/cli/run_test.go:5316TestGenerateSandboxName_Uniqueness calls generateSandboxName() 50 times in a tight loop. Since PID is constant and time.Now().UnixNano() can return the same value for consecutive calls on platforms with coarse clock resolution (common in CI VMs), identical hash inputs can produce duplicate names, making the test flaky.
    Remediation: Add a monotonic counter to the hash input for sequential uniqueness, or remove the uniqueness test since the function is only called once per process invocation.

  • [stale-reference] docs/guides/infrastructure/eval-measurements.md:228 — Documents the sandbox directory naming pattern as agent-<name>-<pid>-<unix>, which is now stale after this PR changes the format to fs-<16-hex-chars>.
    Remediation: Update line 228 to reflect the new naming pattern.

Low

  • [loss-of-debuggability] internal/cli/run.go:3290 — The old sandbox name embedded the agent name, PID, and creation time. The new fs-<hex> name is an opaque hash providing no human-readable information in logs, error messages, or openshell sandbox exec commands.
    Remediation: Consider preserving some human-readable context in the name, or log the agent name alongside the sandbox name at creation time.

  • [api-idiom-consistency] internal/cli/run.go:3289fmt.Appendf(nil, ...) is the only occurrence of this idiom in the codebase. Every other sha256.Sum256 call uses sha256.Sum256([]byte(fmt.Sprintf(...))).
    Remediation: Replace fmt.Appendf(nil, ...) with []byte(fmt.Sprintf(...)) to match the established pattern.

  • [intent-alignment] internal/cli/run.go:903 — The fix scope matches issue sandbox name should be no more than 19 characters #6412, but the implementation does not account for downstream consumers of the sandbox naming convention (evalmeasure, documentation).


Next steps:

  • /fs-fix — agent addresses review findings automatically
  • /fs-fix <your instruction> — agent fixes with your specific guidance
  • Push commits directly — review re-runs automatically on push
  • /fs-fix-stop — disable automatic fix runs for this PR

fullsend-ai-review[bot]

This comment was marked as outdated.

@maruiz93 maruiz93 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The review bot's findings are confirmed. Two additional gaps:

Behaviour test missing. Sandbox creation is a user-visible workflow — needs a behaviour scenario validating the name constraint, not just unit tests for the helper function.

No pre-creation validation. Add a length check before sandbox.CreateWithRetry so name-too-long errors surface locally with a clear message, not as an opaque gRPC rejection from OpenShell.

Debuggability. 19 chars is enough for a truncated agent prefix + shorter hash (e.g. fs-tri-a1b2c3d4) — keeps names useful in logs and output dirs. The current fs-<16-hex> format makes it impossible to tell which agent a sandbox belongs to.

Comment thread internal/cli/run.go Outdated
Comment thread internal/cli/run.go Outdated
Comment thread internal/cli/run_test.go
@maruiz93

Copy link
Copy Markdown
Contributor

/fs-fix

Guidance for the fix agent:

  1. evalmeasure breakage (critical): Update hostRunDirPattern in internal/evalmeasure/find.go to match the new naming format. The wantAgent filtering also needs a new mechanism since the agent name is no longer in the directory name — consider a sidecar metadata file or updating runDir to preserve the agent name separately.

  2. Debuggability: Refactor generateSandboxName to accept agentName and embed a truncated prefix, e.g. fs-tri-a1b2c3d4 (3-char agent slug + dash + 11-char hash = 19 chars). This preserves debuggability in logs, output dirs, and --keep-sandbox hints, and gives evalmeasure something to filter on.

  3. Pre-creation validation: Add a length guard before sandbox.CreateWithRetry that fails early with a clear error if the name exceeds maxSandboxNameLen.

  4. Stale docs: Update docs/guides/infrastructure/eval-measurements.md line 228 to reflect the new naming pattern.

  5. Test fixes: The uniqueness test is flaky on coarse-clock VMs — add a monotonic counter to the hash input, or remove the test since generateSandboxName is called once per process. Add a behaviour test scenario under e2e/behaviour/features/ validating that sandbox creation succeeds with the new naming.

  6. Style: Replace fmt.Appendf(nil, ...) with []byte(fmt.Sprintf(...)) to match existing codebase idiom.

@fullsend-ai-coder

fullsend-ai-coder Bot commented Aug 20, 2026

Copy link
Copy Markdown
Contributor Author

🤖 Finished Fix · ✅ Success · Started 4:46 PM UTC · Completed 4:57 PM UTC

Commit: a583ea2 · View workflow run →

@fullsend-ai-coder

Copy link
Copy Markdown
Contributor Author

🔧 Fix agent — iteration 1 (human-triggered)

Addressed 4 of 6 review findings. Refactored generateSandboxName to embed a 3-char agent slug for debuggability (fs-tri-), added a monotonic counter to fix flaky uniqueness tests, added a pre-creation length guard, and replaced fmt.Appendf with []byte(fmt.Sprintf). Disagreed with 3 findings: evalmeasure/find.go and eval-measurements.md do not exist in the repo, and a behaviour test for pure unit logic adds no integration value.

Fixed (4):

  1. Debuggability: embed agent name slug in sandbox name (internal/cli/run.go): Refactored generateSandboxName to accept agentName and embed a 3-char lowercase alphanumeric slug (e.g. fs-tri- for triage). Added agentSlug helper that handles empty names (returns 'unk'), short names (pads by repeating last char), and non-alphanumeric characters (skips them). Updated call site in runAgent to pass agentName.
  2. Pre-creation validation: add length guard before sandbox.CreateWithRetry (internal/cli/run.go): Added a length guard before sandbox.CreateWithRetry that returns a clear error if sandboxName exceeds maxSandboxNameLen (19), failing early before attempting to create the sandbox.
  3. Test fixes: flaky uniqueness test and monotonic counter (internal/cli/run_test.go): Added a package-level atomic.Uint64 monotonic counter (sandboxNameSeq) to the hash input in generateSandboxName, ensuring uniqueness even when PID and wall-clock are identical on coarse-clock VMs. The uniqueness test now reliably passes with -race. Updated all existing tests to pass agentName argument and added TestGenerateSandboxName_AgentSlug covering triage, code, review, empty, and short agent names.
  4. Style: replace fmt.Appendf(nil, ...) with []byte(fmt.Sprintf(...)) (internal/cli/run.go): Replaced fmt.Appendf(nil, ...) with []byte(fmt.Sprintf(...)) to match the existing codebase idiom used throughout internal/cli/run_test.go and other packages.

Disagreed (3):

  1. evalmeasure breakage: update hostRunDirPattern in internal/evalmeasure/find.go: The file internal/evalmeasure/find.go does not exist in this repository. There is no evalmeasure package, no hostRunDirPattern, and no wantAgent filtering. This finding references code that has not been implemented yet and is out of scope for this PR.
  2. Stale docs: update docs/guides/infrastructure/eval-measurements.md line 228: The file docs/guides/infrastructure/eval-measurements.md does not exist in this repository. The docs/guides/infrastructure/ directory contains other files (advanced-setup.md, distributed-tracing.md, etc.) but no eval-measurements.md. There is no documentation to update.
  3. Test fixes: add behaviour test scenario under e2e/behaviour/features/: Sandbox name generation is a pure unit operation (no network, no forge interaction, no repo state). The existing behaviour tests in e2e/behaviour/features/ test integration scenarios that require enrolled test repositories and GitHub interactions (branch handling, dispatch, triage). A behaviour test for sandbox naming would not exercise any integration boundary and would add maintenance burden without testing value beyond what the unit tests already cover.

Tests: passed

Next steps:

  • /fs-review — request a re-review of the changes
  • /fs-fix <your instruction> — run another fix pass with specific guidance
  • Push commits directly — review re-runs automatically on push
    Updated by fullsend fix agent

fullsend-ai-coder Bot added a commit that referenced this pull request Aug 20, 2026
- Refactor generateSandboxName to accept agentName and embed a 3-char
  slug prefix (e.g. fs-tri-<hash> for "triage"), preserving
  debuggability in logs, output dirs, and --keep-sandbox hints
- Add monotonic counter (atomic.Uint64) to hash input to prevent
  collisions on coarse-clock VMs
- Add pre-creation length guard before sandbox.CreateWithRetry that
  fails early with a clear error if the name exceeds maxSandboxNameLen
- Replace fmt.Appendf(nil, ...) with []byte(fmt.Sprintf(...)) to match
  existing codebase idiom
- Add TestGenerateSandboxName_AgentSlug covering various agent names
  including empty and short names

Addresses review feedback on #6416
@fullsend-ai-review

fullsend-ai-review Bot commented Aug 20, 2026

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 4:59 PM UTC · Completed 5:15 PM UTC

Commit: 2281bc2 · View workflow run →

fullsend-ai-review[bot]

This comment was marked as outdated.

maruiz93 pushed a commit that referenced this pull request Aug 21, 2026
- Refactor generateSandboxName to accept agentName and embed a 3-char
  slug prefix (e.g. fs-tri-<hash> for "triage"), preserving
  debuggability in logs, output dirs, and --keep-sandbox hints
- Add monotonic counter (atomic.Uint64) to hash input to prevent
  collisions on coarse-clock VMs
- Add pre-creation length guard before sandbox.CreateWithRetry that
  fails early with a clear error if the name exceeds maxSandboxNameLen
- Replace fmt.Appendf(nil, ...) with []byte(fmt.Sprintf(...)) to match
  existing codebase idiom
- Add TestGenerateSandboxName_AgentSlug covering various agent names
  including empty and short names

Addresses review feedback on #6416
@maruiz93
maruiz93 force-pushed the agent/6412-sandbox-name-limit branch from 2281bc2 to 8f36ed0 Compare August 21, 2026 07:24
@fullsend-ai-review

fullsend-ai-review Bot commented Aug 21, 2026

Copy link
Copy Markdown

🤖 Review · ❌ Terminated · Started 7:26 AM UTC · Ended 7:40 AM UTC

Commit: 8f36ed0 · View workflow run →

@codecov

codecov Bot commented Aug 21, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 78.26087% with 5 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
internal/evalmeasure/find.go 78.26% 5 Missing ⚠️

📢 Thoughts on this report? Let us know!

fullsend-ai-review[bot]

This comment was marked as outdated.

@fullsend-ai-review

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 7:26 AM UTC · Completed 7:40 AM UTC

Commit: 8f36ed0 · View workflow run →

@fullsend-ai-coder

fullsend-ai-coder Bot commented Aug 21, 2026

Copy link
Copy Markdown
Contributor Author

🤖 Finished Fix · ✅ Success · Started 7:42 AM UTC · Completed 7:53 AM UTC

Commit: 8f36ed0 · View workflow run →

fullsend-ai-coder Bot added a commit that referenced this pull request Aug 21, 2026
- Add newHostRunDirPattern regex (fs-<slug>-<hex>) alongside legacy
  agent-<name>-<pid>-<unix> pattern in evalmeasure/find.go
- Add agentSlug helper and matchesRunDir to support slug-based agent
  filtering for the new format
- Add tests for new format: directory discovery, agent filtering,
  mixed old/new formats, nested copy isolation, root precedence
- Add new-format counterparts for CLI evalmeasure integration tests
- Move unreachable length guard in run.go to before os.MkdirAll
- Add name field to table-driven sandbox slug test for proper subtest
  naming

Addresses review feedback on #6416
@fullsend-ai-coder

Copy link
Copy Markdown
Contributor Author

🔧 Fix agent — iteration 2 (bot-triggered)

Addressed all 5 review findings. Updated evalmeasure/find.go to support both legacy and new sandbox naming formats with slug-based agent filtering. Added comprehensive tests for the new format in both the evalmeasure and cli packages. Moved the unreachable length guard to before directory creation. Fixed table-driven test naming convention.

Fixed (5):

  1. stale-reference: hostRunDirPattern in internal/evalmeasure/find.go (internal/evalmeasure/find.go): Added newHostRunDirPattern regex for the fs-- format alongside the legacy agent--- pattern. Updated findChildPlatformTelemetry to check both patterns via a new matchesRunDir helper. Added comprehensive tests for new format directories including discovery, agent filtering, mixed formats, nested copy isolation, and root precedence.
  2. agent-filtering-broken: 3-char slug makes agent filtering ambiguous (internal/evalmeasure/find.go): Added agentSlug helper in evalmeasure package (mirrors internal/cli.agentSlug) to derive the 3-char slug from agent names. The matchesRunDir function uses slug-based comparison for the new format. Documented the lossy nature of slug matching in the FindPlatformTelemetry doc comment — in practice acceptable because a single CI job runs one agent at a time.
  3. stale-reference: evalmeasure_test.go tests use old directory format (internal/cli/evalmeasure_test.go): Renamed existing tests to _LegacyFormat suffix and added new-format counterpart tests (TestEvalMeasureCmd_OutputDirIgnoresNestedTelemetry_NewFormat and TestEvalMeasureCmd_LocalFullsendDirManifestProducesJSONL_NewFormat) that use fs-- directories. Legacy tests retained since old run directories may still exist.
  4. dead-code: unreachable length guard after os.MkdirAll (internal/cli/run.go): Moved the len(sandboxName) > maxSandboxNameLen guard from after the pre-script execution (where it was unreachable and after the directory was already created) to immediately after generateSandboxName returns, before os.MkdirAll.
  5. naming-convention: table-driven test uses tt.agent as subtest key (internal/cli/run_test.go): Added dedicated name field to the TestGenerateSandboxName_AgentSlug test table and use it as the t.Run subtest key. The empty-string case now uses 'empty' and the short name case uses 'short_name', consistent with other table-driven tests in the file.

Tests: passed

Next steps:

  • /fs-review — request a re-review of the changes
  • /fs-fix <your instruction> — run another fix pass with specific guidance
  • Push commits directly — review re-runs automatically on push
    Updated by fullsend fix agent

@fullsend-ai-review

fullsend-ai-review Bot commented Aug 21, 2026

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 7:55 AM UTC · Completed 8:10 AM UTC

Commit: 5c3163c · View workflow run →

@fullsend-ai-review
fullsend-ai-review Bot dismissed stale reviews from themself August 21, 2026 08:10

Superseded by updated review

fullsend-ai-review[bot]

This comment was marked as outdated.

@fullsend-ai-review fullsend-ai-review Bot added the ready-for-merge All reviewers approved — ready to merge label Aug 21, 2026
OpenShell enforces a maximum sandbox name length of 19 characters,
but the previous format "agent-<name>-<pid>-<timestamp>" produced
names of 30+ characters, causing sandbox creation to fail with
newer OpenShell versions.

Replace the name generation with a SHA-256 hash of the PID and
nanosecond timestamp, truncated to a 19-character "fs-<hex>"
format. This fits within the OpenShell limit while preserving
collision resistance. Add a maxSandboxNameLen constant and tests
verifying length, prefix, and uniqueness.

Closes #6412
- Refactor generateSandboxName to accept agentName and embed a 3-char
  slug prefix (e.g. fs-tri-<hash> for "triage"), preserving
  debuggability in logs, output dirs, and --keep-sandbox hints
- Add monotonic counter (atomic.Uint64) to hash input to prevent
  collisions on coarse-clock VMs
- Add pre-creation length guard before sandbox.CreateWithRetry that
  fails early with a clear error if the name exceeds maxSandboxNameLen
- Replace fmt.Appendf(nil, ...) with []byte(fmt.Sprintf(...)) to match
  existing codebase idiom
- Add TestGenerateSandboxName_AgentSlug covering various agent names
  including empty and short names

Addresses review feedback on #6416
- Add newHostRunDirPattern regex (fs-<slug>-<hex>) alongside legacy
  agent-<name>-<pid>-<unix> pattern in evalmeasure/find.go
- Add agentSlug helper and matchesRunDir to support slug-based agent
  filtering for the new format
- Add tests for new format: directory discovery, agent filtering,
  mixed old/new formats, nested copy isolation, root precedence
- Add new-format counterparts for CLI evalmeasure integration tests
- Move unreachable length guard in run.go to before os.MkdirAll
- Add name field to table-driven sandbox slug test for proper subtest
  naming

Addresses review feedback on #6416
@fullsend-ai-review

fullsend-ai-review Bot commented Aug 21, 2026

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 8:41 AM UTC · Completed 8:57 AM UTC

Commit: 4c300f2 · View workflow run →

@fullsend-ai-review fullsend-ai-review Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See the review comment for full details.

Comment thread internal/cli/run.go
@fullsend-ai-review fullsend-ai-review Bot added requires-manual-review Review requires human judgment and removed ready-for-merge All reviewers approved — ready to merge labels Aug 21, 2026
@maruiz93
maruiz93 added this pull request to the merge queue Aug 21, 2026
Merged via the queue into main with commit a313d79 Aug 21, 2026
63 checks passed
@maruiz93
maruiz93 deleted the agent/6412-sandbox-name-limit branch August 21, 2026 09:21
@fullsend-ai-retro

fullsend-ai-retro Bot commented Aug 21, 2026

Copy link
Copy Markdown

🤖 Finished Retro · ✅ Success · Started 9:23 AM UTC · Completed 10:13 AM UTC

Commit: 4c300f2 · View workflow run →

@fullsend-ai-retro

Copy link
Copy Markdown

Retro: PR #6416 -- limit sandbox name to 19 characters

Pipeline shape: triage -> code -> 5 reviews -> 2 fix iterations -> human approval -> merge (17h wall-clock).

Timeline

  1. Issue #6412 filed by @benoitf: OpenShell enforces a 19-char sandbox name limit; fullsend's agent-<name>-<pid>-<ts> format produces 30+ chars.
  2. Triage (run 32388904794) correctly scoped the fix, applied ready-to-code at 15:59 UTC.
  3. Code agent (run 32389433150) produced a hash-based fs-<16-hex> naming fix in ~14 min. However, the branch was forked from a commit before PR feat(eval)!: add eval measurements and EM-001 trace_fitness scorer #6036 (feat/eval-measurements) merged at 15:53 UTC, so internal/evalmeasure/find.go did not exist on the PR branch.
  4. Review docs: Add agent-compatible code problem document #1 (run 32390818302) correctly flagged the critical evalmeasure scope-gap -- the new naming format would break hostRunDirPattern. Also flagged debuggability ([low]), test adequacy, stale docs, and style.
  5. Human reviewer @maruiz93 confirmed bot findings, elevated debuggability to a requirement, added pre-creation validation and behavior test requirements. Posted structured /fs-fix guidance.
  6. Fix docs: Add agent-compatible code problem document #1 (run 32393699405) addressed 4/6 findings. Correctly reported that evalmeasure/find.go did not exist on the PR branch -- the branch was 58 commits behind main.
  7. Review Add problem areas: Tekton pipeline review, migration path, multi-tenancy #2 (run 32394890453) still flagged the critical evalmeasure gap. No fix agent was auto-dispatched.
  8. 14-hour gap. Overnight, the PR branch was rebased onto main (bringing in evalmeasure code).
  9. Review docs: Add codebase context problem document and trim CLAUDE.md #3 (run 32458466095) triggered by the human push, found the same issues. This time, fix Add problem areas: Tekton pipeline review, migration path, multi-tenancy #2 was auto-dispatched.
  10. Fix Add problem areas: Tekton pipeline review, migration path, multi-tenancy #2 (run 32459660822) successfully updated evalmeasure, added comprehensive tests.
  11. Review Use AI to help formalise intent after rapid local prototyping #4 (run 32460681703) approved with only low-severity findings. Human approved at 08:37 UTC.
  12. Review docs: add agent infrastructure problem document #5 (run 32464173892) ran after human approval, finding only low/medium items already acknowledged.

Evidence for Existing Issues

  • #5750 (fix agent should verify against base branch before disagreeing): The fix agent's evalmeasure disagreement was factually correct for its checkout but practically wrong for the merged state. This PR provides a clean example -- concurrent PR feat(eval)!: add eval measurements and EM-001 trace_fitness scorer #6036 introduced evalmeasure files on main after the code agent branched. A base-branch check or pre-fix rebase would have resolved this in one iteration instead of two.
  • agents#230 (fix agent should meet coverage requirements): Fix Add problem areas: Tekton pipeline review, migration path, multi-tenancy #2 added production code to evalmeasure/find.go with 5 uncovered lines, dropping patch coverage to 78.26% (below 80% threshold). The fix agent's verification steps (secret scan, pre-commit, tests, self-review) include no coverage check, and the check-patch-coverage skill is not available to the fix agent.
  • #5817 (consolidate skip-dispatch issues): Observed a redundant second code agent run (label re-application), 4+ extra fix dispatches on the main shim, and 14 routing-only pull_request_review runs.
  • #3025 (post-approval review findings): Review docs: add agent infrastructure problem document #5 ran after human approval and found only low/medium items already acknowledged by the reviewer.

Positive Signals

  • Review agent quality was strong. The critical evalmeasure scope-gap was caught in the first review -- the most impactful finding. All 7 findings were valid.
  • Human-bot collaboration worked well. The human confirmed bot findings, elevated severity where warranted, and provided structured remediation guidance. The review agent and human were genuinely complementary.
  • Triage was thorough. Correctly identified root cause, suggested a viable hash-based approach, and noted forward-compatibility considerations.

Proposals filed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-review Agent PR ready for human review requires-manual-review Review requires human judgment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

sandbox name should be no more than 19 characters

1 participant