From 116665a57c5d5a1148088d96d9e8b2743cbb6a10 Mon Sep 17 00:00:00 2001 From: Weekendsuperhero <4048475+WeekendSuperhero@users.noreply.github.com> Date: Mon, 13 Jul 2026 23:30:07 -0700 Subject: [PATCH 1/4] feat: reusable label-gated PR review loop (review / security / tests) Three Jules agents per round, each with a distinct lens and a strict JSON contract; designed around two constraints: no wasted agent turns and no PR noise. Loop mechanics: - The trigger label is a button: removed on pickup (re-arms), jules:reviewing while running, outcome labels per agent (review:approved|needs-changes, security:clear|flagged, tests:sufficient|needs-work) plus jules:approved / jules:error rollups. Label taxonomy bootstraps idempotently. - Same-SHA presses no-op unless the force label is used; the caller holds a per-PR job-level concurrency group with an exact-match label guard so our own label writes can never cancel an in-flight round. - New commits (synchronize) strip stale outcome labels and cancel the round. - Delta rounds: each agent's sticky comment carries a hidden marker (reviewed SHA, round, base64 open-findings state); the next round makes agents verify each previous finding (resolved/unresolved) and review only the diff since the last reviewed SHA. - Noise ceiling: exactly one sticky comment per agent, PATCHed in place. Resolved findings collapse into
. - Verdict -> labels is DERIVED from open findings (critical/high = fail, medium = warn); the model's own verdict can only make it stricter, so an agent cannot green-light its own open findings. - Human PR discussion (minus our stickies) is included in the prompt so agents see pushback and decisions. Context gathering reuses the pr-description include/exclude + per-file collapse strategy; per-agent guidelines files from the calling repo are appended to the prompts (e.g. RULES.md for review, CREDENTIALS.md for security). --- .github/workflows/reusable-pr-review.yml | 750 +++++++++++++++++++++++ 1 file changed, 750 insertions(+) create mode 100644 .github/workflows/reusable-pr-review.yml diff --git a/.github/workflows/reusable-pr-review.yml b/.github/workflows/reusable-pr-review.yml new file mode 100644 index 0000000..4d37348 --- /dev/null +++ b/.github/workflows/reusable-pr-review.yml @@ -0,0 +1,750 @@ +# Reusable workflow: label-gated PR review loop with three Jules agents +# (code review · security · test coverage). +# +# Usage in a consuming repo: +# +# ```yaml +# name: PR Review (Jules) +# on: +# pull_request: +# types: [labeled, synchronize] +# +# jobs: +# review: +# # Exact-match guard: outcome/state labels we add ourselves (jules:reviewing, +# # review:approved, …) must NOT re-enter the concurrency group below. +# if: >- +# github.event.action == 'synchronize' +# || github.event.label.name == 'jules:review' +# || github.event.label.name == 'jules:force' +# # Job-level (not workflow-level) so skipped runs never claim the group. +# # A new `jules:review` press or a push cancels a stale in-flight round. +# concurrency: +# group: pr-review-${{ github.event.pull_request.number }} +# cancel-in-progress: true +# permissions: +# contents: read +# issues: write +# pull-requests: write +# uses: //.github/workflows/reusable-pr-review.yml@main +# with: +# runs-on: warp-ubuntu-latest-x64-4x +# review-guidelines: "RULES.md" +# security-guidelines: "CREDENTIALS.md,acp-fs/SANDBOXING.md" +# secrets: inherit +# ``` +# +# The loop: +# 1. Adding the `jules:review` label is the button. The workflow removes it +# immediately (re-arming it) and runs one round: all agents in parallel, +# one Jules session each. +# 2. Each agent maintains exactly ONE sticky comment on the PR (edited in +# place, never appended), with a hidden marker recording the reviewed SHA, +# round number, and the open findings as machine-readable state. +# 3. Outcome labels carry the verdicts (`review:approved|needs-changes`, +# `security:clear|flagged`, `tests:sufficient|needs-work`, plus +# `jules:approved` when everything is positive). +# 4. New commits (`synchronize`) strip the now-stale outcome labels and +# cancel any in-flight round. Re-adding `jules:review` runs a DELTA +# round: agents re-verify each previous finding (resolved/unresolved) +# and review only the changes since the last reviewed SHA — no wasted +# turns re-litigating settled findings. +# 5. Same-SHA presses are no-ops unless the `jules:force` label is used. +# +# Verdict → labels is derived deterministically from the OPEN findings +# (critical/high ⇒ fail, medium ⇒ warn, else the model's own verdict), so a +# model that under-reports severity can't green-light its own findings. + +name: Reusable PR Review + +on: + workflow_call: + inputs: + runs-on: + description: "Runner label for all jobs (e.g. warp-ubuntu-latest-x64-4x)" + type: string + required: false + default: "ubuntu-latest" + agents: + description: "Comma-separated agents to run: any of review,security,tests" + type: string + required: false + default: "review,security,tests" + trigger-label: + description: "Label that starts a review round (removed on pickup so it re-arms)" + type: string + required: false + default: "jules:review" + force-label: + description: "Label that starts a round even when the head SHA was already reviewed" + type: string + required: false + default: "jules:force" + review-guidelines: + description: "Comma-separated repo paths appended to the code-review agent's prompt (e.g. RULES.md)" + type: string + required: false + default: "" + security-guidelines: + description: "Comma-separated repo paths appended to the security agent's prompt" + type: string + required: false + default: "" + tests-guidelines: + description: "Comma-separated repo paths appended to the test-coverage agent's prompt" + type: string + required: false + default: "" + diff-file-patterns: + description: "Space-separated glob patterns for files to include in the diff" + type: string + required: false + default: "'*.rs' '*.ts' '*.tsx' '*.toml' '*.sql' '*.yml' '*.sh' '*.css' '*.go' '*.swift'" + diff-exclude-patterns: + description: "Space-separated patterns to exclude from the diff" + type: string + required: false + default: ":!pnpm-lock.yaml :!Cargo.lock ':!**/bindings/*.ts'" + max-commit-messages: + description: "Maximum number of commit messages to include" + type: number + required: false + default: 30 + max-findings: + description: "Cap on NEW findings per agent per round" + type: number + required: false + default: 12 + max-wait-seconds: + description: "Maximum seconds to wait for each Jules session" + type: number + required: false + default: 600 + secrets: + JULES_API_KEY: + required: true + JULES_PR_CLIENT_ID: + required: false + JULES_PR_PRIVATE_KEY: + required: false + +jobs: + # ── New commits invalidate the last round: strip stale outcome labels. ── + # (The caller's concurrency group has already cancelled any in-flight + # round for this PR.) Sticky comments stay — they self-update next round. + cleanup: + name: Clear stale review labels + if: github.event.action == 'synchronize' + runs-on: ${{ inputs.runs-on }} + permissions: + issues: write + pull-requests: write + steps: + - name: Remove outcome labels + env: + GH_TOKEN: ${{ github.token }} + PR: ${{ github.event.pull_request.number }} + run: | + for label in "jules:reviewing" "jules:approved" "jules:error" \ + "review:approved" "review:needs-changes" \ + "security:clear" "security:flagged" \ + "tests:sufficient" "tests:needs-work"; do + gh pr edit "$PR" --repo "$GITHUB_REPOSITORY" --remove-label "$label" 2>/dev/null || true + done + echo "Stale review labels cleared for PR #$PR (new commits)" + + # ── Gate + shared context. Runs once per round; agents fan out from it. ── + prepare: + name: Prepare review round + if: >- + github.event.action == 'labeled' + && (github.event.label.name == inputs.trigger-label + || github.event.label.name == inputs.force-label) + runs-on: ${{ inputs.runs-on }} + permissions: + contents: read + issues: write + pull-requests: write + outputs: + skip: ${{ steps.state.outputs.skip }} + agents: ${{ steps.state.outputs.agents }} + round: ${{ steps.state.outputs.round }} + steps: + - name: Validate GitHub App secret configuration + id: app-auth-config + shell: bash + env: + JULES_APP_ID: ${{ secrets.JULES_PR_CLIENT_ID }} + JULES_APP_KEY: ${{ secrets.JULES_PR_PRIVATE_KEY }} + run: | + set -euo pipefail + if { [ -n "${JULES_APP_ID}" ] && [ -z "${JULES_APP_KEY}" ]; } || { [ -z "${JULES_APP_ID}" ] && [ -n "${JULES_APP_KEY}" ]; }; then + echo "::error::Set both JULES_PR_CLIENT_ID and JULES_PR_PRIVATE_KEY, or neither." + exit 1 + fi + ENABLED=false + if [ -n "${JULES_APP_ID}" ] && [ -n "${JULES_APP_KEY}" ]; then + ENABLED=true + fi + echo "enabled=${ENABLED}" >> "$GITHUB_OUTPUT" + + - name: Generate GitHub App token + if: steps.app-auth-config.outputs.enabled == 'true' + id: app-token + uses: actions/create-github-app-token@v3 + with: + client-id: ${{ secrets.JULES_PR_CLIENT_ID }} + private-key: ${{ secrets.JULES_PR_PRIVATE_KEY }} + owner: ${{ github.repository_owner }} + + # Remove the pressed label FIRST so the button re-arms even when this + # round later skips or fails, then make sure the whole taxonomy exists + # (idempotent — `--force` updates color/description in place). + - name: Re-arm trigger and bootstrap labels + env: + GH_TOKEN: ${{ steps.app-token.outputs.token || github.token }} + PR: ${{ github.event.pull_request.number }} + PRESSED: ${{ github.event.label.name }} + run: | + gh pr edit "$PR" --repo "$GITHUB_REPOSITORY" --remove-label "$PRESSED" 2>/dev/null || true + + create() { gh label create "$1" --repo "$GITHUB_REPOSITORY" --color "$2" --description "$3" --force >/dev/null 2>&1 || true; } + create "jules:review" "ededed" "Request a Jules review round (re-add to re-run)" + create "jules:force" "ededed" "Force a Jules round even if this SHA was already reviewed" + create "jules:reviewing" "fbca04" "Jules review round in progress" + create "jules:approved" "0e8a16" "All Jules agents positive" + create "jules:error" "b60205" "A Jules agent failed — see workflow run" + create "review:approved" "0e8a16" "Jules code review: no blocking findings" + create "review:needs-changes" "d93f0b" "Jules code review: blocking findings open" + create "security:clear" "0e8a16" "Jules security review: no blocking findings" + create "security:flagged" "b60205" "Jules security review: blocking findings open" + create "tests:sufficient" "0e8a16" "Jules test review: coverage adequate for this diff" + create "tests:needs-work" "d93f0b" "Jules test review: missing tests for this diff" + echo "Trigger re-armed; label taxonomy ensured" + + - name: Checkout PR head + uses: actions/checkout@v6 + with: + ref: ${{ github.event.pull_request.head.sha }} + token: ${{ steps.app-token.outputs.token || github.token }} + fetch-depth: 0 + + # Each agent's sticky comment carries `` + # plus a base64 state block of its open findings. Read them to decide + # same-SHA skip and to seed the delta round. + - name: Read previous round state + id: state + env: + GH_TOKEN: ${{ steps.app-token.outputs.token || github.token }} + PR: ${{ github.event.pull_request.number }} + HEAD_SHA: ${{ github.event.pull_request.head.sha }} + AGENTS: ${{ inputs.agents }} + FORCED: ${{ github.event.label.name == inputs.force-label }} + run: | + set -euo pipefail + mkdir -p context + + # Validate + normalize the agent list. + AGENTS_JSON=$(printf '%s' "$AGENTS" | jq -R 'split(",") | map(gsub("^\\s+|\\s+$"; "")) | map(select(length > 0))') + for a in $(printf '%s' "$AGENTS_JSON" | jq -r '.[]'); do + case "$a" in review|security|tests) ;; *) + echo "::error::Unknown agent '$a' (allowed: review, security, tests)"; exit 1 ;; + esac + done + echo "agents=$(printf '%s' "$AGENTS_JSON" | jq -c .)" >> "$GITHUB_OUTPUT" + + gh api "repos/$GITHUB_REPOSITORY/issues/$PR/comments" --paginate --jq '.[]' | jq -s '.' > /tmp/all_comments.json + + ALL_CURRENT=true + MAX_ROUND=0 + LAST_SHA="" + for a in $(printf '%s' "$AGENTS_JSON" | jq -r '.[]'); do + BODY=$(jq -r --arg m "" + echo "" + } > /tmp/body.md + + # One sticky comment per agent: PATCH in place if it exists. + CID=$(jq -r --arg m " — safe now, nothing is in production yet. - config/sources/repo.labels.json: canonical versioned label set (control + per-agent outcome labels). - scripts/sync-repo-labels.sh: idempotent sync of the manifest to given repos or every non-archived repo in an org (gh label create --force). GitHub's org 'default labels for new repos' has no API — mirror the manifest there once by hand, or re-run the script / rely on the workflow's self-bootstrap. --- .github/workflows/reusable-pr-review.yml | 84 ++++++++++++------------ config/sources/repo.labels.json | 16 +++++ scripts/sync-repo-labels.sh | 42 ++++++++++++ 3 files changed, 100 insertions(+), 42 deletions(-) create mode 100644 config/sources/repo.labels.json create mode 100755 scripts/sync-repo-labels.sh diff --git a/.github/workflows/reusable-pr-review.yml b/.github/workflows/reusable-pr-review.yml index 4d37348..a7d1bf7 100644 --- a/.github/workflows/reusable-pr-review.yml +++ b/.github/workflows/reusable-pr-review.yml @@ -1,24 +1,24 @@ -# Reusable workflow: label-gated PR review loop with three Jules agents +# Reusable workflow: label-gated PR review loop with three Muse review agents (Jules-powered) # (code review · security · test coverage). # # Usage in a consuming repo: # # ```yaml -# name: PR Review (Jules) +# name: PR Review (Muse) # on: # pull_request: # types: [labeled, synchronize] # # jobs: # review: -# # Exact-match guard: outcome/state labels we add ourselves (jules:reviewing, +# # Exact-match guard: outcome/state labels we add ourselves (muse:reviewing, # # review:approved, …) must NOT re-enter the concurrency group below. # if: >- # github.event.action == 'synchronize' -# || github.event.label.name == 'jules:review' -# || github.event.label.name == 'jules:force' +# || github.event.label.name == 'muse:review' +# || github.event.label.name == 'muse:force' # # Job-level (not workflow-level) so skipped runs never claim the group. -# # A new `jules:review` press or a push cancels a stale in-flight round. +# # A new `muse:review` press or a push cancels a stale in-flight round. # concurrency: # group: pr-review-${{ github.event.pull_request.number }} # cancel-in-progress: true @@ -35,7 +35,7 @@ # ``` # # The loop: -# 1. Adding the `jules:review` label is the button. The workflow removes it +# 1. Adding the `muse:review` label is the button. The workflow removes it # immediately (re-arming it) and runs one round: all agents in parallel, # one Jules session each. # 2. Each agent maintains exactly ONE sticky comment on the PR (edited in @@ -43,13 +43,13 @@ # round number, and the open findings as machine-readable state. # 3. Outcome labels carry the verdicts (`review:approved|needs-changes`, # `security:clear|flagged`, `tests:sufficient|needs-work`, plus -# `jules:approved` when everything is positive). +# `muse:approved` when everything is positive). # 4. New commits (`synchronize`) strip the now-stale outcome labels and -# cancel any in-flight round. Re-adding `jules:review` runs a DELTA +# cancel any in-flight round. Re-adding `muse:review` runs a DELTA # round: agents re-verify each previous finding (resolved/unresolved) # and review only the changes since the last reviewed SHA — no wasted # turns re-litigating settled findings. -# 5. Same-SHA presses are no-ops unless the `jules:force` label is used. +# 5. Same-SHA presses are no-ops unless the `muse:force` label is used. # # Verdict → labels is derived deterministically from the OPEN findings # (critical/high ⇒ fail, medium ⇒ warn, else the model's own verdict), so a @@ -74,12 +74,12 @@ on: description: "Label that starts a review round (removed on pickup so it re-arms)" type: string required: false - default: "jules:review" + default: "muse:review" force-label: description: "Label that starts a round even when the head SHA was already reviewed" type: string required: false - default: "jules:force" + default: "muse:force" review-guidelines: description: "Comma-separated repo paths appended to the code-review agent's prompt (e.g. RULES.md)" type: string @@ -145,7 +145,7 @@ jobs: GH_TOKEN: ${{ github.token }} PR: ${{ github.event.pull_request.number }} run: | - for label in "jules:reviewing" "jules:approved" "jules:error" \ + for label in "muse:reviewing" "muse:approved" "muse:error" \ "review:approved" "review:needs-changes" \ "security:clear" "security:flagged" \ "tests:sufficient" "tests:needs-work"; do @@ -209,17 +209,17 @@ jobs: gh pr edit "$PR" --repo "$GITHUB_REPOSITORY" --remove-label "$PRESSED" 2>/dev/null || true create() { gh label create "$1" --repo "$GITHUB_REPOSITORY" --color "$2" --description "$3" --force >/dev/null 2>&1 || true; } - create "jules:review" "ededed" "Request a Jules review round (re-add to re-run)" - create "jules:force" "ededed" "Force a Jules round even if this SHA was already reviewed" - create "jules:reviewing" "fbca04" "Jules review round in progress" - create "jules:approved" "0e8a16" "All Jules agents positive" - create "jules:error" "b60205" "A Jules agent failed — see workflow run" - create "review:approved" "0e8a16" "Jules code review: no blocking findings" - create "review:needs-changes" "d93f0b" "Jules code review: blocking findings open" - create "security:clear" "0e8a16" "Jules security review: no blocking findings" - create "security:flagged" "b60205" "Jules security review: blocking findings open" - create "tests:sufficient" "0e8a16" "Jules test review: coverage adequate for this diff" - create "tests:needs-work" "d93f0b" "Jules test review: missing tests for this diff" + create "muse:review" "ededed" "Request a Muse review round (re-add to re-run)" + create "muse:force" "ededed" "Force a Muse round even if this SHA was already reviewed" + create "muse:reviewing" "fbca04" "Muse review round in progress" + create "muse:approved" "0e8a16" "All Muse review agents positive" + create "muse:error" "b60205" "A Muse review agent failed — see workflow run" + create "review:approved" "0e8a16" "Muse code review: no blocking findings" + create "review:needs-changes" "d93f0b" "Muse code review: blocking findings open" + create "security:clear" "0e8a16" "Muse security review: no blocking findings" + create "security:flagged" "b60205" "Muse security review: blocking findings open" + create "tests:sufficient" "0e8a16" "Muse test review: coverage adequate for this diff" + create "tests:needs-work" "d93f0b" "Muse test review: missing tests for this diff" echo "Trigger re-armed; label taxonomy ensured" - name: Checkout PR head @@ -229,7 +229,7 @@ jobs: token: ${{ steps.app-token.outputs.token || github.token }} fetch-depth: 0 - # Each agent's sticky comment carries `` + # Each agent's sticky comment carries `` # plus a base64 state block of its open findings. Read them to decide # same-SHA skip and to seed the delta round. - name: Read previous round state @@ -259,10 +259,10 @@ jobs: MAX_ROUND=0 LAST_SHA="" for a in $(printf '%s' "$AGENTS_JSON" | jq -r '.[]'); do - BODY=$(jq -r --arg m "" - echo "" + echo "" + echo "" } > /tmp/body.md # One sticky comment per agent: PATCH in place if it exists. - CID=$(jq -r --arg m "` marker records what + was written. If a human edits a verb (e.g. downgrades `Closes` → + `Part of`), the next run preserves it. A no-op body diff skips the write. +4. Optional `LINEAR_API_KEY` secret enriches lines with issue titles and + prunes definitive not-founds — cosmetic, non-fatal, skipped when absent. + +Inputs: `linear-enabled` (default true), `linear-id-pattern`, plus the diff +shaping knobs (`diff-file-patterns`, `diff-exclude-patterns`, +`max-commit-messages`, `commit-format`, `custom-prompt`, `base-branch`). + +--- + +## Changelog (`reusable-changelog.yml`) + +```yaml +name: Update Changelog +on: + workflow_dispatch: + +jobs: + changelog: + permissions: + contents: write + uses: weekendsuperhero-io/platform-tools/.github/workflows/reusable-changelog.yml@main + with: + changelog-path: CHANGELOG.md + internal-changelog-path: CHANGELOG_INTERNAL.md # omit for single-file mode + runs-on: warp-ubuntu-latest-x64-4x + pr-branch-prefix: "chore/changelog-" + secrets: inherit +``` + +Finds the commits since the last changelog-touching commit and opens a PR +with new entries inserted under `## [Unreleased]`. + +- **Single-file mode** (default, `internal-changelog-path` unset): one + markdown blob of Keep-a-Changelog sections — the original behavior. +- **Split mode**: one Jules call classifies every commit as **user-facing** + (anything a user can perceive; ships verbatim as release notes / updater + notes / TestFlight text) vs **internal** (refactors, CI, deps, plumbing, + tests, docs), and both files are patched in one PR. User-facing bullets ban + ticket IDs, crate names, and commit-speak; internal bullets keep full + technical detail. Dual-face changes may appear in both with different + wording; when unsure, entries go internal — the user changelog stays clean. +- Rotation of `[Unreleased]` on release is the **consumer's** job (e.g. the + agent repo's `scripts/rotate-changelog.sh`, called for both files so + version histories stay parallel). + +--- + +## PR review loop (`reusable-pr-review.yml`) + +```yaml +name: PR Review (Muse) +on: + pull_request: + types: [labeled, synchronize] + +jobs: + review: + # Exact-match guard — NOT startsWith('muse:'): the loop writes + # muse:reviewing / outcome labels itself, and those events must not + # re-enter the concurrency group or they'd cancel the running round. + if: >- + github.event.action == 'synchronize' + || github.event.label.name == 'muse:review' + || github.event.label.name == 'muse:force' + # Job-level (not workflow-level) so guard-skipped runs never claim the + # group; a new press or a push cancels a stale in-flight round. + concurrency: + group: pr-review-${{ github.event.pull_request.number }} + cancel-in-progress: true + permissions: + contents: read + issues: write + pull-requests: write + uses: weekendsuperhero-io/platform-tools/.github/workflows/reusable-pr-review.yml@main + with: + runs-on: warp-ubuntu-latest-x64-4x + review-guidelines: "RULES.md" + security-guidelines: "CREDENTIALS.md,acp-fs/SANDBOXING.md" + tests-guidelines: "ARCHITECTURE.md" + secrets: inherit +``` + +Both comments in that snippet are **load-bearing** — copy them with the code. + +### The loop + +1. **`muse:review` is a button.** Adding it starts a round; the workflow + removes it immediately (re-arming it) and adds `muse:reviewing`. +2. **One round = one context gather + three parallel Jules sessions** — + code review, security, test coverage — each with its own lens prompt and + the caller's guidelines files appended. +3. **Each agent owns exactly one sticky comment**, PATCHed in place (max + three review comments per PR, ever). Findings are severity-sorted; the + resolved ones collapse into `
`. +4. **Verdicts land as labels** (see taxonomy below) plus `muse:approved` + (everything positive) or `muse:error` (an agent failed). +5. **New commits** (`synchronize`) strip the stale outcome labels and cancel + an in-flight round. Re-pressing `muse:review` runs a **delta round**. +6. **Same-SHA presses are no-ops** — `muse:force` overrides. + +### Delta rounds (why turns aren't wasted) + +Each sticky comment carries hidden markers: + +``` + + +``` + +The next round hands the agent its previous open findings plus the diff since +that SHA, with instructions to verify each finding (`resolved`/`unresolved` +with evidence) and review only what changed. Carry-forward is computed by the +workflow, not the model: unresolved findings persist automatically, resolved +ones drop, new ones append. + +### Verdict → labels is deterministic + +Labels derive from the **open findings**: any critical/high ⇒ fail, any +medium ⇒ warn, else pass. The model's own verdict can only make the result +stricter — an agent cannot green-light findings it left open. + +### The lenses + +| Agent | Prefix | Hunts for | Guidelines input | +| --- | --- | --- | --- | +| Code review | `REV` | logic errors, error-handling gaps, races, API misuse, breaking contracts, missed reuse | `review-guidelines` | +| Security | `SEC` | injection, secrets in code/logs, trust-boundary validation, sandbox/entitlement changes, SSRF, crypto misuse, risky deps | `security-guidelines` | +| Test coverage | `TST` | diff behaviors mapped to tests; names the concrete missing test cases (bug fix without regression test = high) | `tests-guidelines` | + +Guidelines inputs are comma-separated repo paths whose contents are appended +to the prompt (capped per file). + +### Label taxonomy + +| Label | Meaning | +| --- | --- | +| `muse:review` | the button — request a round (self-removes on pickup) | +| `muse:force` | re-review an already-reviewed SHA | +| `muse:reviewing` | round in progress | +| `muse:approved` / `muse:error` | rollups: all agents positive / an agent failed | +| `review:approved` / `review:needs-changes` | code-review outcome | +| `security:clear` / `security:flagged` | security outcome | +| `tests:sufficient` / `tests:needs-work` | test-coverage outcome | + +The taxonomy self-bootstraps in any repo on first press. The canonical set is +versioned in [`config/sources/repo.labels.json`](../config/sources/repo.labels.json); +[`scripts/sync-repo-labels.sh`](../scripts/sync-repo-labels.sh) syncs it to +given repos or a whole org (`--org `). GitHub's org-level "default labels +for new repositories" has no API — mirror the manifest there by hand once if +wanted (Settings → Repository defaults → Labels). + +### Failure modes + +- An agent job fails (Jules timeout, unparseable output) → `muse:error`, the + other agents' results still land, `muse:reviewing` is cleared. Re-press to + retry. +- Pressing before the reusable workflow exists on `@main` → workflow-not-found + startup failure; merge platform-tools first. +- Human discussion on the PR (minus the sticky comments) is included in the + agents' context, so pushback and decisions are visible to the next round. + +### Inputs + +| Input | Default | Purpose | +| --- | --- | --- | +| `runs-on` | `ubuntu-latest` | runner for all jobs | +| `agents` | `review,security,tests` | subset to run | +| `trigger-label` / `force-label` | `muse:review` / `muse:force` | the buttons | +| `review-guidelines` / `security-guidelines` / `tests-guidelines` | `""` | comma-separated repo paths per lens | +| `max-findings` | `12` | cap on new findings per agent per round | +| `max-wait-seconds` | `600` | per-Jules-session timeout | +| `diff-file-patterns` / `diff-exclude-patterns` / `max-commit-messages` | — | context shaping, same semantics as the other workflows | + +Secrets: `JULES_API_KEY` (required), `JULES_PR_CLIENT_ID` + +`JULES_PR_PRIVATE_KEY` (optional app identity for label re-arm/bootstrap). +Comment and label writes intentionally use `github.token`, which never +triggers other workflows. From 646f4904a562a35f40b5d5d97fcbe3538b617733 Mon Sep 17 00:00:00 2001 From: Weekendsuperhero <4048475+WeekendSuperhero@users.noreply.github.com> Date: Tue, 14 Jul 2026 12:40:40 -0700 Subject: [PATCH 4/4] refactor: move review prompts out of YAML into markdown templates MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The three lens prompts and the shared JSON contract were heredocs inside reusable-pr-review.yml. They now live as plain markdown in .github/actions/muse-prompts/prompts/ (contract.md, review.md, security.md, tests.md) with {{AGENT}}/{{PREFIX}}/{{ROUND}}/{{MAX_FINDINGS}} tokens substituted at runtime — editable and reviewable as prose, not YAML string art. Delivery mechanism: reusable workflows never check out their own repo, so files next to the YAML don't exist on the runner. The tiny muse-prompts composite action solves that — referencing a remote action downloads the platform-tools tarball, and github.action_path exposes the templates directory to the calling job. --- .github/actions/muse-prompts/action.yml | 29 ++++++ .../actions/muse-prompts/prompts/contract.md | 32 ++++++ .../actions/muse-prompts/prompts/review.md | 8 ++ .../actions/muse-prompts/prompts/security.md | 10 ++ .github/actions/muse-prompts/prompts/tests.md | 11 +++ .github/workflows/reusable-pr-review.yml | 98 ++++--------------- docs/muse-workflows.md | 12 +++ 7 files changed, 123 insertions(+), 77 deletions(-) create mode 100644 .github/actions/muse-prompts/action.yml create mode 100644 .github/actions/muse-prompts/prompts/contract.md create mode 100644 .github/actions/muse-prompts/prompts/review.md create mode 100644 .github/actions/muse-prompts/prompts/security.md create mode 100644 .github/actions/muse-prompts/prompts/tests.md diff --git a/.github/actions/muse-prompts/action.yml b/.github/actions/muse-prompts/action.yml new file mode 100644 index 0000000..1ff4a4f --- /dev/null +++ b/.github/actions/muse-prompts/action.yml @@ -0,0 +1,29 @@ +# Ships the Muse review agents' prompt templates to the runner. +# +# The templates are the markdown files in ./prompts — contract.md (the shared +# JSON output contract) plus one lens file per agent (review.md, security.md, +# tests.md). Edit those files to tune the reviewers; {{AGENT}}, {{PREFIX}}, +# {{ROUND}}, and {{MAX_FINDINGS}} are substituted at runtime by the workflow. +# +# Why an action at all: reusable workflows never check out their own repo, so +# files next to the YAML don't exist on the runner. Referencing this composite +# action remotely is what downloads the platform-tools tarball; action_path +# then points at this directory. + +name: "Muse review prompts" +description: "Exposes the Muse review prompt templates directory to the calling job" + +outputs: + dir: + description: "Absolute path to the prompt templates directory" + value: ${{ steps.locate.outputs.dir }} + +runs: + using: composite + steps: + - name: Locate prompt templates + id: locate + shell: bash + env: + ACTION_PATH: ${{ github.action_path }} + run: echo "dir=${ACTION_PATH}/prompts" >> "$GITHUB_OUTPUT" diff --git a/.github/actions/muse-prompts/prompts/contract.md b/.github/actions/muse-prompts/prompts/contract.md new file mode 100644 index 0000000..295a83a --- /dev/null +++ b/.github/actions/muse-prompts/prompts/contract.md @@ -0,0 +1,32 @@ +You are the "{{AGENT}}" reviewer in an automated three-agent pull +request review. Review ONLY this pull request's changes (diff plus +its immediate context). Cite file paths and line numbers as they +appear in the diff. This is round {{ROUND}}. + +Output a single raw JSON object — no markdown fences, no prose: +{ + "verdict": "pass" | "warn" | "fail", + "summary": "2-4 plain-language sentences", + "findings": [ + {"id": "{{PREFIX}}-{{ROUND}}-1", "severity": "critical|high|medium|low|nit", + "file": "path/to/file", "line": 123, "title": "short title", + "detail": "why this matters", "suggestion": "concrete fix"} + ], + "resolved": [ + {"id": "", "status": "resolved" | "unresolved", + "note": "one line of evidence"} + ] +} + +Rules: +- "findings" holds NEW issues only, at most {{MAX_FINDINGS}}, ordered by + severity, ids numbered {{PREFIX}}-{{ROUND}}-1, {{PREFIX}}-{{ROUND}}-2, … + Merge duplicates. No praise, no restating the diff, no formatting + nits (formatters handle style). An empty array is a fine answer. +- If PREVIOUS FINDINGS are provided below: verify EVERY one against + the current code and list each in "resolved" with status + resolved/unresolved plus evidence. Do NOT repeat unresolved ones in + "findings" — they are carried forward automatically. Focus new + review effort on the CHANGES SINCE LAST REVIEW section. +- "verdict": "fail" if any critical/high finding stands (new or + unresolved), "warn" if only medium/low, "pass" when clean or nits. diff --git a/.github/actions/muse-prompts/prompts/review.md b/.github/actions/muse-prompts/prompts/review.md new file mode 100644 index 0000000..3e9d894 --- /dev/null +++ b/.github/actions/muse-prompts/prompts/review.md @@ -0,0 +1,8 @@ +Your lens: GENERAL CODE REVIEW. +Hunt for: logic errors and unhandled edge cases; error-handling gaps +(swallowed errors, unwraps/panics on fallible paths); concurrency and +race hazards; API misuse; breaking changes to public contracts; +duplicated code that should reuse an existing helper; violations of +the project guidelines included below. +Ignore: formatting/style, naming taste, and hypothetical refactors +with no concrete defect. diff --git a/.github/actions/muse-prompts/prompts/security.md b/.github/actions/muse-prompts/prompts/security.md new file mode 100644 index 0000000..24dd47b --- /dev/null +++ b/.github/actions/muse-prompts/prompts/security.md @@ -0,0 +1,10 @@ +Your lens: SECURITY. +Hunt for: injection (command, SQL, path traversal); secrets or tokens +hardcoded, logged, or written to disk; unsafe deserialization of +untrusted input; missing validation at trust boundaries (IPC, +network, filesystem, subprocess arguments); sandbox / entitlement / +permission expansions; SSRF or unvalidated URLs; crypto misuse; +dependency changes that expand the attack surface; PII or sensitive +data exposed in logs and error messages. +Severity discipline: reachable-by-untrusted-input = critical/high; +defense-in-depth hardening = medium/low. diff --git a/.github/actions/muse-prompts/prompts/tests.md b/.github/actions/muse-prompts/prompts/tests.md new file mode 100644 index 0000000..9eb564f --- /dev/null +++ b/.github/actions/muse-prompts/prompts/tests.md @@ -0,0 +1,11 @@ +Your lens: TEST COVERAGE (of this diff, not the whole repository). +Method: (1) enumerate the behavior changes in this diff — new +features, changed logic, bug fixes; (2) map each to test changes in +the diff or clearly-referenced existing tests; (3) report each gap +as a finding naming the exact behavior plus a CONCRETE test to add: +suggested test name, scenario, expected result. +Severity: bug fix without a regression test = high; new core +behavior untested = high/medium; error paths, migrations, and +boundary conditions untested = medium; nice-to-have cases = low. +Non-behavioral changes (docs, CI, pure refactors with existing +coverage) need no tests — say so and pass. diff --git a/.github/workflows/reusable-pr-review.yml b/.github/workflows/reusable-pr-review.yml index ec3080f..1ce612a 100644 --- a/.github/workflows/reusable-pr-review.yml +++ b/.github/workflows/reusable-pr-review.yml @@ -375,11 +375,21 @@ jobs: name: pr-review-context path: context/ + # The prompts are markdown templates shipped by the muse-prompts + # composite action (referencing a remote action is what downloads the + # platform-tools tarball — reusable workflows never check out their own + # repo). Edit .github/actions/muse-prompts/prompts/*.md to tune the + # reviewers. + - name: Fetch prompt templates + id: prompts + uses: weekendsuperhero-io/platform-tools/.github/actions/muse-prompts@main + - name: Build prompt env: AGENT: ${{ matrix.agent }} MAX_FINDINGS: ${{ inputs.max-findings }} PR_TITLE: ${{ github.event.pull_request.title }} + PROMPTS_DIR: ${{ steps.prompts.outputs.dir }} run: | set -euo pipefail ROUND=$(jq -r '.round' context/meta.json) @@ -388,90 +398,24 @@ jobs: PREV_COUNT=$(jq 'length' "context/prev_${AGENT}.json") case "$AGENT" in - review) - PREFIX="REV" - cat > /tmp/lens.txt << 'LENS_END' - Your lens: GENERAL CODE REVIEW. - Hunt for: logic errors and unhandled edge cases; error-handling gaps - (swallowed errors, unwraps/panics on fallible paths); concurrency and - race hazards; API misuse; breaking changes to public contracts; - duplicated code that should reuse an existing helper; violations of - the project guidelines included below. - Ignore: formatting/style, naming taste, and hypothetical refactors - with no concrete defect. - LENS_END - ;; - security) - PREFIX="SEC" - cat > /tmp/lens.txt << 'LENS_END' - Your lens: SECURITY. - Hunt for: injection (command, SQL, path traversal); secrets or tokens - hardcoded, logged, or written to disk; unsafe deserialization of - untrusted input; missing validation at trust boundaries (IPC, - network, filesystem, subprocess arguments); sandbox / entitlement / - permission expansions; SSRF or unvalidated URLs; crypto misuse; - dependency changes that expand the attack surface; PII or sensitive - data exposed in logs and error messages. - Severity discipline: reachable-by-untrusted-input = critical/high; - defense-in-depth hardening = medium/low. - LENS_END - ;; - tests) - PREFIX="TST" - cat > /tmp/lens.txt << 'LENS_END' - Your lens: TEST COVERAGE (of this diff, not the whole repository). - Method: (1) enumerate the behavior changes in this diff — new - features, changed logic, bug fixes; (2) map each to test changes in - the diff or clearly-referenced existing tests; (3) report each gap - as a finding naming the exact behavior plus a CONCRETE test to add: - suggested test name, scenario, expected result. - Severity: bug fix without a regression test = high; new core - behavior untested = high/medium; error paths, migrations, and - boundary conditions untested = medium; nice-to-have cases = low. - Non-behavioral changes (docs, CI, pure refactors with existing - coverage) need no tests — say so and pass. - LENS_END - ;; + review) PREFIX="REV" ;; + security) PREFIX="SEC" ;; + tests) PREFIX="TST" ;; esac - cat > /tmp/prompt.txt << CONTRACT_END - You are the "${AGENT}" reviewer in an automated three-agent pull - request review. Review ONLY this pull request's changes (diff plus - its immediate context). Cite file paths and line numbers as they - appear in the diff. This is round ${ROUND}. - - Output a single raw JSON object — no markdown fences, no prose: - { - "verdict": "pass" | "warn" | "fail", - "summary": "2-4 plain-language sentences", - "findings": [ - {"id": "${PREFIX}-${ROUND}-1", "severity": "critical|high|medium|low|nit", - "file": "path/to/file", "line": 123, "title": "short title", - "detail": "why this matters", "suggestion": "concrete fix"} - ], - "resolved": [ - {"id": "", "status": "resolved" | "unresolved", - "note": "one line of evidence"} - ] + # Substitute {{TOKENS}} in a template (all values are alphanumeric). + render() { + sed -e "s/{{AGENT}}/${AGENT}/g" \ + -e "s/{{PREFIX}}/${PREFIX}/g" \ + -e "s/{{ROUND}}/${ROUND}/g" \ + -e "s/{{MAX_FINDINGS}}/${MAX_FINDINGS}/g" "$1" } - Rules: - - "findings" holds NEW issues only, at most ${MAX_FINDINGS}, ordered by - severity, ids numbered ${PREFIX}-${ROUND}-1, ${PREFIX}-${ROUND}-2, … - Merge duplicates. No praise, no restating the diff, no formatting - nits (formatters handle style). An empty array is a fine answer. - - If PREVIOUS FINDINGS are provided below: verify EVERY one against - the current code and list each in "resolved" with status - resolved/unresolved plus evidence. Do NOT repeat unresolved ones in - "findings" — they are carried forward automatically. Focus new - review effort on the CHANGES SINCE LAST REVIEW section. - - "verdict": "fail" if any critical/high finding stands (new or - unresolved), "warn" if only medium/low, "pass" when clean or nits. - CONTRACT_END + render "${PROMPTS_DIR}/contract.md" > /tmp/prompt.txt { echo "" - cat /tmp/lens.txt + render "${PROMPTS_DIR}/${AGENT}.md" echo "" echo "=== PR TITLE ===" echo "$PR_TITLE" diff --git a/docs/muse-workflows.md b/docs/muse-workflows.md index b7ed759..9e65602 100644 --- a/docs/muse-workflows.md +++ b/docs/muse-workflows.md @@ -197,6 +197,18 @@ stricter — an agent cannot green-light findings it left open. Guidelines inputs are comma-separated repo paths whose contents are appended to the prompt (capped per file). +The prompt texts themselves are markdown templates in +[`.github/actions/muse-prompts/prompts/`](../.github/actions/muse-prompts/prompts/) — +[`contract.md`](../.github/actions/muse-prompts/prompts/contract.md) (the +shared JSON output contract) plus one lens file per agent +([`review.md`](../.github/actions/muse-prompts/prompts/review.md), +[`security.md`](../.github/actions/muse-prompts/prompts/security.md), +[`tests.md`](../.github/actions/muse-prompts/prompts/tests.md)). Edit those +files to tune the reviewers; `{{AGENT}}`, `{{PREFIX}}`, `{{ROUND}}`, and +`{{MAX_FINDINGS}}` are substituted at runtime. They ship to the runner via the +tiny `muse-prompts` composite action (reusable workflows never check out their +own repo; referencing a remote action is what materializes the files). + ### Label taxonomy | Label | Meaning |