docs(problems): sharpen gh-aw comparison in platform-nativeness

[waynesun09]

Summary

Sharpens docs/problems/platform-nativeness.md's fullsend-vs-gh-aw comparison with a few points that came out of a closer look at GITHUB_TOKEN behavior, verified against primary sources.

Related Issue

None — follow-up refinement to the existing platform-nativeness analysis, no tracked issue.

Changes

• Per-role GitHub App creation: adds the GITHUB_TOKEN workflow-retriggering behavior (GitHub's own docs) as a sharper, more concrete justification for per-role Apps than least-privilege alone — PRs opened via the ambient token land in an approval-required state, and issues/labels opened via it don't retrigger the next workflow at all. Notes this is unverified for gh-aw's own safe-outputs write jobs.
• New "Run-trigger authorization (the 'pwn request' gap)" section: documents that GitHub Actions has no built-in approval gate for issue_comment/issues.opened triggers, citing GitHub Security Lab's original 2021 writeup (the primary source for the term, not a vendor blog) and fullsend's own ADR 0054 mitigation.
• New "Cost/token observability" section: notes fullsend already covers this via OpenTelemetry to any OTLP backend; frames gh-aw's gh aw logs/audit as an inline-UX nicety, not a capability gap.
• Multi-agent coordination: clarifies that orchestration primitives (dispatch-workflow/call-workflow) are the same event-driven pattern fullsend already uses, and are orthogonal to the per-role App question; adds the same-repo-only nuance for dispatch-workflow specifically.
• Forge lock-in: adds that multi-forge portability isn't the only (or biggest) reason GitHub-only is limiting — the harder problem is operating safely against wide-open public upstream repos vs. internal/enterprise repos with natural contributor segregation.

Testing

• lychee --offline --include-fragments — 0 errors, new anchor links resolve
• ./hack/lint-docs-links — OK
• make lint (docs-only change; not run)

Checklist

• PR title follows Conventional Commits
• Commit is signed off (DCO)
• I wrote this contribution myself and can explain all changes in it

[qodo-code-review]

PR Summary by Qodo

Docs: sharpen GitHub Actions vs gh-aw comparison in platform nativeness

📝 Documentation 🕐 10-20 Minutes

AI Description

• Document GITHUB_TOKEN event suppression and why per-role bot identities matter for unattended
 pipelines.
• Add “pwn request” run-trigger authorization gap and cite Fullsend’s ADR 0054 mitigation.
• Clarify cost observability, orchestration primitives, and GitHub-only constraints for public
 repos.

Diagram

graph TD
  A["platform-nativeness.md"] --> B["GITHUB_TOKEN behavior"] --> C["Per-role Apps needed"]
  A --> D["Run-trigger gap"] --> E["ADR 0054 gate"]
  A --> F["Cost observability"] --> G["OTel vs CLI"]
  A --> H["Orchestration notes"] --> I["Same-repo nuance"]
  A --> J["Forge lock-in"] --> K["Public repo trust"]

Loading

High-Level Assessment

The PR’s approach—tightening the comparison using primary-source citations and concrete platform behaviors—is the right strategy for this document. Alternatives like reorganizing into separate docs or adding a comparison table are optional formatting choices and don’t materially improve the technical argument being refined here.

Files changed (1) +21 / -1

Documentation (1) +21 / -1

platform-nativeness.mdAdd concrete GitHub Actions behavior-based refinements to gh-aw comparison +21/-1

Add concrete GitHub Actions behavior-based refinements to gh-aw comparison

• Adds a cited explanation of 'GITHUB_TOKEN'-authored event suppression/approval behavior as a concrete reason per-role bot identities are required for unattended, event-driven pipelines. Introduces a new section on ungated run triggers for social-interaction events (the “pwn request” class) and references Fullsend’s ADR 0054 authorization check. Adds a cost/token observability comparison, clarifies orchestration primitives vs identity concerns, and refines the GitHub-only limitation discussion to emphasize public-repo trust/abuse surfaces.

docs/problems/platform-nativeness.md

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 7:04 PM UTC · Completed 7:23 PM UTC
Commit: 2d71d18 · View workflow run →

[github-actions]

Site preview

Preview: https://f25d2790-site.fullsend-ai.workers.dev

Commit: 76b5d0ccd4de795b83627e39c35df87cdffe0b46

[qodo-code-review]

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📜 Skill insights (0)

Context used

✅ Compliance rules (platform): 54 rules

✅ Skills: writing-user-docs, writing-adrs

1. Org-specific ADRs in problem doc ✗ Dismissed 📘 Rule violation § Compliance

Description

docs/problems/platform-nativeness.md includes organization-specific identifiers and internal
decision references (e.g., Fullsend and specific ADR links) in a core problem doc instead of
placing those details under docs/problems/applied/<org-name>/. This reduces
portability/reusability of the core problem document and violates the generic-vs-applied doc
separation requirement.

Code

docs/problems/platform-nativeness.md[R92-98]

+### Run-trigger authorization (the "pwn request" gap)
+
+GitHub Actions has no built-in approval gate for `issue_comment` or `issues.opened` triggers — unlike `pull_request` from public forks, which can require a maintainer's approval before running. Any user who can comment on an issue or open one fires the workflow immediately, full stop. [GitHub Security Lab named this class of gap "pwn request"](https://securitylab.github.com/resources/github-actions-preventing-pwn-requests/) in 2021; their original writeup focuses on `pull_request_target` code execution, but the same absence of a platform-level trigger gate applies to any social-interaction event. For a workflow that runs paid LLM inference, this means external cost exposure and an abuse surface with no rate limit — an attacker doesn't need a security bug, just a repo they can comment on.
+
+Fullsend closes this itself: every dispatch path — slash commands and automatic event triggers alike — calls a shared `has_write_permission` check against the collaborator-permission API before dispatching an agent ([ADR 0054](../ADRs/0054-require-authorization-on-all-agent-dispatch-paths.md)). This was not the default behavior; it had to be built and applied consistently after discovering some dispatch paths were gated and others weren't.
+
+It is not stated in gh-aw's public documentation whether it applies an equivalent check on *who* may trigger a workflow run, as opposed to controlling what a triggered agent's output is allowed to do via safe-outputs and integrity filtering. Until verified, treat gh-aw workflows triggered by `issue_comment`/`issues.opened` as inheriting the same ungated-trigger behavior as any other GitHub Actions workflow.

Relevance

⭐⭐⭐ High

Prior accepted reviews moved org-specific details out of core problem docs into applied/generic
wording.

PR-#3064
PR-#12

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

PR Compliance ID 1062038 requires that organization-specific examples/workflows/constraints be
placed under docs/problems/applied/<org-name>/, not in core problem docs. The added text
explicitly references internal Fullsend mechanisms and internal ADRs (e.g., has_write_permission
behavior and ADR 0054, plus ADR 0002/ADR 0016) directly in the core document.

Rule 1062038: Separate generic and organization-specific problem documents
docs/problems/platform-nativeness.md[92-98]
docs/problems/platform-nativeness.md[120-122]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Core problem documents under `docs/problems/` must remain generic and must not include organization-specific identifiers (e.g., internal system/project names and internal ADR references). Newly added content in `docs/problems/platform-nativeness.md` embeds org-specific Fullsend behavior and internal ADR links inline.

## Issue Context
This PR adds new sections that reference `Fullsend` implementation details and internal ADRs (e.g., `ADR 0054`, `ADR 0002`, `ADR 0016`) inside a core problem doc.

## Fix Focus Areas
- docs/problems/platform-nativeness.md[92-98]
- docs/problems/platform-nativeness.md[120-122]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

_{To customize comments, go to the Qodo configuration screen, or learn more in the docs.}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[fullsend-ai-review]

Review

Findings

Medium

• [API behavior claim accuracy] docs/problems/platform-nativeness.md:77 — The PR claims gh-aw's supply chain protection uses request_review as the default policy ("the default request_review policy still creates the PR but attaches a blocking review requiring human approval before merge"). Searches of gh-aw's documentation found no request_review policy documented for protected-files — blocked is described as the default (most restrictive), with allowed and fallback-to-issue as alternatives. The same claim appears at line 112 (injection defense section). If request_review is not a valid gh-aw policy, both occurrences should revert to the base version's characterization (blocked/allowed/fallback-to-issue).

Low

• [factual inaccuracy] docs/problems/platform-nativeness.md:169 — States "v0.83.x as of July 2026" for gh-aw's version, but the latest release appears to be v0.82.13 (released July 18, 2026). Should read "v0.82.x."

• [sentence-complexity] docs/problems/platform-nativeness.md:36 — The sentence starting "GitHub's own [GITHUB_TOKEN documentation]..." runs ~95 words with multiple nested clauses covering the general rule, the pull_request carve-out, other activity types, and issues/labels. Consider splitting into 2–3 sentences.

• [sentence-complexity] docs/problems/platform-nativeness.md:38 — The sentence starting "That is fatal for a pipeline..." runs ~70 words with em-dashes and nested clauses. Consider splitting after "next stage runs unattended at all."

• [stale-reference-gh-aw-status] docs/landscape.md:292 — Describes gh-aw as "Early development; may change significantly," now inconsistent with this PR's update to "public preview." Consider updating alongside this PR.

• [stale-reference-supply-chain-policy] docs/landscape.md:300 — Lists supply chain policies as blocked/allowed/fallback-to-issue. Whether this needs updating depends on resolving the request_review policy question above — if request_review is not a valid policy, this text may already be more accurate.

Previous run

Review

Findings

Low

• [API behavior claim accuracy] docs/problems/platform-nativeness.md:92 — The section heading borrows the term “pwn request” from GitHub Security Lab’s 2021 writeup, but the original term refers specifically to privilege escalation via pull_request_target code execution, not to absent authorization gates on who can trigger a workflow run. The document acknowledges the original scope (“their original writeup focuses on pull_request_target code execution”) but extends the term to cover a different concern (cost/abuse exposure from ungated issue_comment/issues.opened triggers). The practical analysis is sound; the attribution stretches the cited term beyond its original meaning.

• [sentence-complexity] docs/problems/platform-nativeness.md:120 — The sentence starting with “Note that dispatch-workflow itself is same-repo only by design...” runs 78 words with nested parenthetical clarifications. While the document’s analytical style tolerates long sentences, this one is at the upper edge of readability. Consider breaking into 2–3 shorter sentences.

• [missing-authorization] docs/problems/platform-nativeness.md — Non-trivial documentation change (21 added lines across 5 distinct analysis points) with no linked issue. Problem documents in docs/problems/ are living analytical documents, so this is a process observation rather than a blocking concern.

• [stale-technical-explanation] docs/ADRs/0033-per-repo-installation-mode.md:67 — ADR 0033’s simplified statement about GitHub event suppression covers pushes only, while this PR’s more detailed analysis covers all event types. The ADR’s statement is accurate for its narrow scope, and ADR immutability constraints apply, so this is informational.

---

Labels: PR modifies a problem analysis document under docs/problems/

[fullsend-ai-review]

🤖 Review · ⚠️ Cancelled · Started 7:25 PM UTC · Ended 7:44 PM UTC
Commit: 5152a7c · View workflow run →

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 7:46 PM UTC · Completed 8:28 PM UTC
Commit: ef078bb · View workflow run →

[fullsend-ai-review]

See the review comment for full details.

[fullsend-ai-review]

🤖 Review · ⚠️ Cancelled · Started 7:16 PM UTC · Ended 7:44 PM UTC
Commit: 91b7980 · View workflow run →

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 7:46 PM UTC · Completed 8:28 PM UTC
Commit: 76b5d0c · View workflow run →

[fullsend-ai-retro]

🤖 Finished Retro · ✅ Success · Started 7:53 PM UTC · Completed 8:08 PM UTC
Commit: 76b5d0c · View workflow run →

[fullsend-ai-retro]

Retro: PR #5646 — docs(problems): sharpen gh-aw comparison in platform-nativeness

Timeline

1. Jul 27 19:03 — PR opened with initial commit 5152a7c. Docs-only change (2 files, +36/−14) sharpening the fullsend vs. gh-aw comparison in docs/problems/platform-nativeness.md.
2. Jul 27 19:04–19:23 — First review agent run completes. Posts findings: [medium] questions whether request_review is gh-aw's default policy; [low] claims latest version is v0.82.13; [low] three sentence-complexity findings; [low] stale refs in docs/landscape.md; [low] "pwn request" attribution. Issues an APPROVED verdict despite the medium finding.
3. Jul 27 19:24 — Second review run cancelled (superseded by new commit ef078bb).
4. Jul 27 19:44–20:28 — Third review run succeeds on ef078bb. Applies requires-manual-review label.
5. Jul 27 20:10 — Human reviewer cgwalters posts the most impactful comment: explicit workflow_dispatch avoids the GITHUB_TOKEN retriggering problem, referencing PR docs: Add ADR 074 with a single-auth #5649. This drives a significant rewrite.
6. Jul 27 20:22, Jul 28 06:35 — Two human approvals (ralphbean, rh-hemartin).
7. Jul 28 19:15 — Author pushes 91b7980 addressing both human and agent findings. Fourth review run cancelled (superseded by final commit).
8. Jul 28 19:44 — Final commit 76b5d0c (sentence split). Fifth review run starts.
9. Jul 28 19:50 — PR merged while fifth review run is still in progress.

Review quality assessment

Agent review: low substantive value, net-negative signal on factual claims.

Finding	Severity	Correct?	Outcome
request_review not the default	medium	Wrong	Author verified it IS the default
Latest version v0.82.13	low	Wrong	Actual was v0.83.4 (published same day)
Sentence complexity (×3)	low	Correct	Fixed
Stale refs in landscape.md	low	Correct	Fixed
"pwn request" attribution	low	Partially	Rephrased

The agent's only medium-severity finding was factually incorrect. Both factual errors involved claims about external state the agent could not verify. The correct findings were all stylistic (sentence length) or mechanical (stale cross-references) — valuable but low-impact.

Human review: high substantive value. cgwalters' single comment identified a fundamental weakness in the document's core technical argument and drove a major rewrite. The agent never attempted to evaluate whether the document's claims about GITHUB_TOKEN retriggering were overstated or whether well-known workarounds existed.

Autonomy signal: This PR provides counter-evidence for agents#427 ("review agent competently handles docs-only PRs"). The agent approved with wrong medium findings, missed the most impactful issue, and the human review was essential.

Token cost

5 review dispatches for a 2-file docs change: 2 completed, 2 cancelled mid-run, 1 still running at merge. The final run was entirely wasted since the PR merged 6 minutes after the last push.

Evidence for existing issues (no new proposals needed)

The problems observed in this PR are well-covered by existing open issues:

• fullsend#2174 (hedge on platform knowledge) and fullsend#1928 (cap severity for unverifiable assumptions) — both directly applicable. The agent's medium finding asserted with confidence that an external API default might differ from what the PR claimed. It was wrong. Implementing either issue would have prevented this false positive.
• agents#420 (verify assertions before posting) — the agent asserted specific version numbers and API defaults without verification capability.
• fullsend#1319 (align approval with finding severity) — the agent approved while simultaneously posting a medium finding.
• agents#373 (skip irrelevant dimensions for docs-only PRs) — would reduce token cost.
• fullsend#1014 (debounce review dispatch) — would have eliminated 2 cancelled runs.
• fullsend#2388 (cancel review when PR merges) — would have stopped the final wasted run.
• agents#265 (cross-reference facts to detect misleading narratives) — the agent never evaluated whether the document's core argument was sound or fairly represented the alternative.

[fullsend-ai-review]

Review skipped — this PR is already merged.

The /fs-review command only reviews open pull requests.

_{Posted by fullsend post-review check}