test(#5715): commit agent fixtures when registering local custom harness

[fullsend-ai-coder]

Summary

givenCustomHarness and givenDisabledCustomHarness in dispatch.go commit the harness YAML and register it in config.yaml, but never create the agent/policy MD files referenced by the harness's agent: field. After scaffold agent files were removed (#5552 / #5588), clean pool repos fail at harness validation with stat .../.fullsend/agents/triage.md: no such file or directory.

Related Issue

Fixes #5715

Changes

• Add commitLocalHarnessResources helper in dispatch.go that parses harness YAML for relative agent: and policy: paths and commits minimal stub files under .fullsend/ on the config repo
• Call the helper from both givenCustomHarness and givenDisabledCustomHarness after committing the harness YAML
• Mirrors the existing commitRelativeResources pattern in url_dispatch.go (used by URL-sourced harnesses) but writes to the config repo with the .fullsend/ prefix
• Reuses the existing minimalAgentContent constant for agent stubs

Testing

• go test ./pkg/behaviourtest/steps/ passes (all existing + new tests)
• go vet ./pkg/behaviourtest/steps/ passes
• Tests added for givenCustomHarness with agent, agent+policy, absolute/URL paths (skipped), disabled harness, and commitLocalHarnessResources directly (agent, URL skip, absolute skip, no-agent-field, invalid YAML)

Checklist

• PR title follows Conventional Commits (correct type, ! for breaking changes)
• No secrets or sensitive data in the diff

Note: pre-commit could not run in sandbox (network restriction on git fetch). The post-script runs it authoritatively.

---

Closes #5715

Post-script verification

• Branch is not main/master (agent/5715-custom-harness-agent-fixture)
• Secret scan passed (gitleaks — a6da8a87d71713bdf4e517cc2c65dae1997972c5..HEAD)
• PR body secret scan passed (gitleaks — no-git)
• Pre-commit hooks passed (authoritative run on runner)
• Tests ran inside sandbox

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 2:08 PM UTC · Completed 2:20 PM UTC
Commit: c196a69 · View workflow run →

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[fullsend-ai-review]

Review

Findings

Low

• [edge-case] pkg/behaviourtest/steps/dispatch.go:163 — The filter for relative paths only checks for https:// URLs and / absolute paths, but does not filter http:// URLs. An agent: http://example.com/agent.md value would be treated as a relative path and committed under .fullsend/http:/example.com/agent.md. This is inherited from the existing commitRelativeResources pattern in url_dispatch.go and is not a regression introduced by this PR.

---

Labels: PR modifies behaviour test harness registration logic in pkg/behaviourtest/steps/

[ifireball]

/fs-fix rebase onto main to pick up the podman 4.x pin from #5738 (ci(#5733)), so merge-queue behaviour tests stop failing on sandbox/podman API errors.

[ifireball]

/fs-fix rebase onto main

[fullsend-ai-coder]

🤖 Finished Fix · ❌ Failure · Started 4:40 AM UTC · Completed 4:45 AM UTC
Commit: 5802cfd · View workflow run →

[fullsend-ai-coder]

⚠️ Post-fix script failed — Structured output processing failed (exit code 1)

The fix agent completed, but the post-fix script failed before finishing.

Workflow run: https://github.com/fullsend-ai/.fullsend/actions/runs/30514420193

Details:
process-fix-result.py failed with exit code 1 (bad input) for PR #5719 in fullsend-ai/fullsend
Please check the workflow logs for full details and retry with /fs-fix if appropriate.

[fullsend-ai-retro]

🤖 Finished Retro · ✅ Success · Started 7:10 AM UTC · Completed 7:23 AM UTC
Commit: bccce6c · View workflow run →

[fullsend-ai-retro]

Retro: PR #5719 — test(#5715): commit agent fixtures when registering local custom harness

Workflow overview

This was a well-executed end-to-end agent workflow. Issue #5715 (custom harness registration not creating referenced agent fixture files) went from creation to code agent PR in 28 minutes, with a clean first-try implementation that required zero rework from review findings.

Timeline

Time (UTC)	Event
Jul 29 13:38	Issue #5715 created by ifireball
13:39–13:43	Triage run #1 — found draft PR #5714, proposed it as a solution
13:48	Human re-triggered triage with instructions to disregard draft PR
13:49–13:56	Triage run #2 — proper root-cause analysis, applied ready-to-code
13:56–14:06	Code agent — created PR #5719 (187 additions, 2 files)
14:06–14:20	Review agent — approved with one low-severity finding
14:18	Human (ifireball) approved
19:50	Fix run #1 — failed at podman install step (5.x installed, 4.x required)
Jul 30 04:38	Fix run #2 — agent succeeded (rebase + push), but post-fix script failed on schema path resolution
07:07	PR merged by ifireball

What went well

• Code agent first-try quality: The code agent produced a clean, well-tested fix that mirrored the existing commitRelativeResources pattern from url_dispatch.go. 8 new test functions, 100% coverage on changed lines per Codecov.
• Review accuracy: The review agent found a valid edge case (http:// URLs not filtered alongside https://) and correctly classified it as inherited from the existing pattern, not a regression. Final verdict aligned with the human reviewer.
• Review challenger value: The challenger sub-agent caught a high-severity false positive ("incomplete implementation") that the intent sub-agent produced because it received a condensed diff with placeholder comments. Without the challenger, this false positive could have blocked approval.

Evidence for existing open issues

• Review orchestrator should not abbreviate source file content with placeholder strings in sub-agent context packages agents#462 (review orchestrator placeholder abbreviation): This workflow produced a concrete example — the intent sub-agent flagged a fully-implemented function as "incomplete" because the orchestrator replaced the function body with placeholder comments before passing to the sub-agent. The challenger caught it, but this wasted tokens ($3.28 total review cost, 30 turns, 81 tool calls) and would have been a blocking false positive without the adversarial check.
• Post-fix failure comment should distinguish pre-push from post-push failures agents#534 (post-fix failure comment clarity): Fix run 30514420193 successfully rebased and force-pushed the branch, but the PR received a "Fix agent run failed" comment that didn't distinguish the post-script failure from actual agent failure. The human had to independently verify the rebase landed.
• Enable post-script-only retry when agent succeeds but post-script fails #2138 (post-script-only retry): The agent work completed successfully but the overall run was marked as failure because post-fix.sh couldn't find fix-result.schema.json at the expected relative path in the content-addressed cache. No retry was attempted despite the agent succeeding.
• Triage agent: check related issues for active PRs before labeling ready-to-code #2000 (triage + draft PRs): The first triage run found draft PR test(#5707): fix missing harness agent fixtures and timeout diagnostics #5714 and concluded the issue was already being addressed, requiring the human to re-trigger with explicit instructions to ignore it. This added ~13 minutes and a manual intervention.

Already-resolved issues encountered

• Post-fix schema path bug (post-fix: read FULLSEND_OUTPUT_SCHEMA from env instead of relative path agents#526, runner: inject FULLSEND_OUTPUT_SCHEMA into post-script environment #5722): Both closed July 29, but the fix had not propagated to the agents commit (8b9e25e) used by the July 30 fix run. Normal deployment lag.
• Podman version pinning (Pin podman version in action.yml to prevent runner image upgrade breakage #5733, PR ci(#5733): pin podman to 4.x to avoid crun incompatibility #5738): The first fix run failed because podman 5.8.4 was installed when 4.x was required. PR ci(#5733): pin podman to 4.x to avoid crun incompatibility #5738 was merged July 29 at 19:44 UTC but the fix run at 19:50 UTC still hit the issue (likely the runner image hadn't been rebuilt yet).

Proposals

No new proposals — all identified improvement areas are already covered by existing open issues. The workflow performed well overall, with the primary friction coming from infrastructure issues (podman version, schema path) that have already been addressed.