Skip to content

docs(#6064): document fstest-* account permission scope - #6069

Merged
rh-hemartin merged 1 commit into
mainfrom
agent/6064-doc-test-actor-permissions
Aug 26, 2026
Merged

docs(#6064): document fstest-* account permission scope#6069
rh-hemartin merged 1 commit into
mainfrom
agent/6064-doc-test-actor-permissions

Conversation

@fullsend-ai-coder

Copy link
Copy Markdown
Contributor

Summary

Add a "Test actor account permission scope" subsection to docs/guides/dev/behaviour-testing.md documenting:

  • fstest-write, fstest-triage, and fstest-outsider are not members of the fullsend-ai org
  • All three have only read permission on fullsend-ai/fullsend and fullsend-ai/agents
  • Write capability is scoped to disposable pool-org test-repo-NN infrastructure only
  • Classic PATs cannot write to production repos even if exfiltrated
  • Re-verification guidance with example gh api commands and a "last verified" date

This eliminates the need for reviewers of test-actor-related PRs (like #6028) to independently re-derive blast-radius assessments via live API queries.

Testing

  • Verified markdown link integrity with lychee (0 errors)
  • Documentation-only change — no production code or test code modified

Closes #6064

Post-script verification

  • Branch is not main/master (agent/6064-doc-test-actor-permissions)
  • Secret scan passed (gitleaks — 483ad9bd31407310940d21003a6cfe1b6b52b5c8..HEAD)
  • PR body secret scan passed (gitleaks — no-git)

Add a "Test actor account permission scope" subsection to the
behaviour testing guide documenting the blast-radius containment
of the three test actor accounts (fstest-write, fstest-triage,
fstest-outsider). This covers org membership status, production
repo permissions, write access scoping, and re-verification
guidance so reviewers of PRs involving test actor PATs do not
need to re-derive permissions via live API queries.

Note: pre-commit could not run (sandbox network restriction).
The post-script runs authoritative pre-commit on the runner.

Closes #6064

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@fullsend-ai-coder
fullsend-ai-coder Bot requested a review from a team as a code owner August 10, 2026 20:02
@fullsend-ai-coder fullsend-ai-coder Bot added the ready-for-review Triggers review agent dispatch label Aug 10, 2026
@fullsend-ai-review

fullsend-ai-review Bot commented Aug 10, 2026

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 8:03 PM UTC · Completed 8:17 PM UTC

Commit: aac990c · View workflow run →

@github-actions

Copy link
Copy Markdown

Site preview

Preview: https://eaa9c537-site.fullsend-ai.workers.dev

Commit: aac990c957878a93a66283144316376509816204

@codecov

codecov Bot commented Aug 10, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@fullsend-ai-review

Copy link
Copy Markdown

Review

Findings

Low

  • [internal-consistency] docs/guides/dev/behaviour-testing.md:249 — The table row "Write access" shows "Pool-org test-repo-NN repos only" for both fstest-write and fstest-triage. The env var comment at line 232 describes TEST_ACTOR_TRIAGE_PAT as a "triage-level human-like actor PAT". If the label reflects the account's GitHub permission level (triage, not write), then the table is misleading by grouping both accounts under the same "Write access" row value. However, "triage-level" may refer to the role the account plays in test scenarios rather than its literal GitHub permission level. The discrepancy is ambiguous without checking actual GitHub permissions — consider clarifying in the table or a footnote whether fstest-triage has GitHub write or triage permission on pool repos.

  • [verification-completeness] docs/guides/dev/behaviour-testing.md:261 — The verification commands only check org membership and repo permission on fullsend-ai/fullsend. They do not verify: (1) permission on fullsend-ai/agents, which the table also claims is read-only, or (2) pool-org repo permissions, which is where the blast-radius containment claim about write access being "scoped exclusively to disposable test-repo-NN infrastructure" would actually be validated. The commands verify the negative ("cannot access production repos") but not the positive ("write access is limited to pool repos only").


Labels: Documentation-only PR adding a subsection to the behaviour testing guide.

Comment thread docs/guides/dev/behaviour-testing.md
Comment thread docs/guides/dev/behaviour-testing.md
@fullsend-ai-review fullsend-ai-review Bot added ready-for-merge All reviewers approved — ready to merge component/docs User-facing documentation labels Aug 10, 2026
@rh-hemartin
rh-hemartin enabled auto-merge August 26, 2026 08:40
@rh-hemartin
rh-hemartin added this pull request to the merge queue Aug 26, 2026
Merged via the queue into main with commit cde0062 Aug 26, 2026
39 checks passed
@rh-hemartin
rh-hemartin deleted the agent/6064-doc-test-actor-permissions branch August 26, 2026 08:48
@fullsend-ai-retro

fullsend-ai-retro Bot commented Aug 26, 2026

Copy link
Copy Markdown

🤖 Finished Retro · ✅ Success · Started 8:49 AM UTC · Completed 9:04 AM UTC

Commit: aac990c · View workflow run →

Runtime: claude · Model: opus → claude-opus-4-6 · Effort: high · Cost: $4.45

@fullsend-ai-retro

Copy link
Copy Markdown

Retro: PR #6069 — docs(#6064): document fstest- account permission scope*

This was a well-functioning end-to-end agent workflow. The retro agent filed issue #6064 from a PR #6028 retrospective. Triage, code, and review agents completed the full pipeline — issue to reviewed PR — in ~26 minutes with zero rework iterations. The code agent produced a clean, single-commit docs change (31 lines added to docs/guides/dev/behaviour-testing.md). The review agent found two valid low-severity findings and appropriately approved.

Review quality: The review agent's two findings were substantive — an internal consistency ambiguity between the new permission table and existing env-var comments regarding fstest-triage permissions, and incomplete verification commands that only cover fullsend-ai/fullsend despite the documentation claiming read-only access on fullsend-ai/agents too. Both were valid, rated low, and the APPROVE verdict was appropriate.

Merge latency: The automated pipeline completed 2026-08-10; human approval and merge occurred 2026-08-26 — a 16-day gap. This is evidence supporting existing open issues #966 (surface agent-approved PRs awaiting human review) and #1974 (priority-aware escalation for stale agent PRs).

Low findings unaddressed: The two review findings were never addressed before merge. This corroborates #5001 (review agent should suggest /fs-fix for auto-fixable low findings) — both findings were auto-fixable (add verification commands for agents repo, clarify a table footnote) and could have been resolved by a fix agent if prompted.

Autonomy signal: The review agent approved; the human approved 16 days later with no additional comments or change requests. This is consistent with #5289 tracking autonomy readiness for docs-only PRs. The review agent's judgment aligned fully with the human reviewer's.

No novel improvement proposals warranted — all identified opportunities are already covered by existing open issues.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

component/docs User-facing documentation ready-for-merge All reviewers approved — ready to merge ready-for-review Triggers review agent dispatch

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Document fstest-* test actor account permission scope for blast-radius assessment

1 participant