Wave Sprint AI — PR Quality Auditor

A GitHub App that automatically pre-audits pull requests opened during Drips Waves so maintainers can review and pay bounty hunters quickly.

When a PR is opened (or updated) during a Wave, the app:

1. Checks out the PR branch into an isolated workspace.
2. Runs AST linting over the changed files (a curated ESLint rule set).
3. Measures cyclomatic complexity of every changed function via a real AST parse.
4. Executes the repository's unit test suite.
5. Compares the diff's size/complexity against the issue's assigned point level (Trivial, Medium, High).
6. Posts a preliminary audit scorecard as a PR comment and creates/updates a Check Run with a verdict for the maintainer.

The scorecard is a heuristic signal to speed up human review — it is not a replacement for it.

---

How it fits the Drips Wave flow

• Bounty issues are labelled (or described) with a point level.
• Hunters submit PRs that reference the issue (e.g. Closes #12).
• The app finds the linked issue, resolves its point level, audits the diff against that level, and posts the scorecard.
• Maintainers get a fast pass/fail signal before the Wave ends, so contributors get paid sooner.

---

Features

• Automatic audit on pull_request events: opened, synchronize, reopened, ready_for_review (drafts are skipped).
• Re-run on demand with a /reaudit comment on the PR.
• Point level detection from issue labels or body keywords (trivial, small, 1 pt, medium, high, 5 pts, numeric point values, …), with a configurable default fallback.
• AST linting — ESLint flat-config with a curated rule set (style + correctness + a complexity warning), run only on the changed files.
• Cyclomatic complexity analysis — per-function complexity for functions touched by the diff, surfaced as a hotspot table.
• Test execution — runs the repo's test script (npm/yarn/pnpm/bun detected from lockfiles), parses common runner output (Jest, Vitest, Mocha, node:test, AVA) and reports pass/fail counts.
• Diff-vs-point-level check — files changed, lines added/removed and worst changed-function complexity are compared against limits per point level.
• Single, self-updating scorecard comment (keyed by an HTML marker) — no spam on every push.
• Check Run with success / neutral / failure conclusion.
• Safety guard — audits are skipped (with a note) when a diff exceeds MAX_TOTAL_CHANGED_LINES.

---

Architecture

GitHub ──webhook──▶ Express (src/index.js)
                      │  @octokit/webhooks middleware
                      ├─▶ pull_request / issue_comment handlers (src/webhooks/)
                      │     └─▶ runAuditFlow (create check run → audit → scorecard)
                      └─▶ src/audit/
                            ├─ index.js        pipeline orchestration
                            ├─ workspace.js    clone/checkout PR ref (or local dir)
                            ├─ diff.js         numstat + hunk line-range parsing
                            ├─ complexity.js   AST cyclomatic complexity (espree)
                            ├─ lint.js         ESLint programmatic API
                            ├─ tests.js        test runner + output parsing
                            ├─ points.js       point-level detection + verdict logic
                            └─ scorecard.js    markdown renderer

• No database. State lives in the PR thread and check runs.
• No frontend. The interface is the GitHub comment + check run.
• No smart contract (per specification — everything runs off-chain as a GitHub App).

Technology choices

Concern	Choice	Why
HTTP server	Express.js	required by the specification
Webhook handling	@octokit/webhooks	HMAC verification, event routing, node middleware
GitHub API	octokit + @octokit/auth-app	app JWT + short-lived installation tokens
AST parsing	espree (ESTree)	the parser ESLint uses; full AST walk for complexity
Linting	eslint programmatic API	runs against a curated flat config, independent of the PR repo's config
Tests (auditor's own)	vitest	fast, ESM-native, zero-config
Runtime	Node.js ≥ 20	modern, ESM, stable fetch

---

Project structure

wave-sprint-auditor/
├── src/
│   ├── index.js            # Express app + webhook wiring
│   ├── server.js           # entrypoint (loads config, listens)
│   ├── config.js           # env → frozen config (validated)
│   ├── logger.js
│   ├── github/
│   │   ├── client.js       # app octokit + installation-token factory
│   │   └── issues.js       # linked-issue resolution + point-level detection
│   ├── webhooks/
│   │   ├── pullRequest.js  # PR event handler
│   │   ├── comment.js      # /reaudit command handler
│   │   ├── audit-flow.js   # shared audit flow
│   │   └── publish.js      # check runs + scorecard comment (upsert)
│   └── audit/              # see architecture diagram
├── scripts/
│   └── audit-local.mjs     # run the audit on a local repo (no GitHub needed)
├── test/
│   ├── fixtures/           # real mini-repos used by integration tests
│   └── *.test.js           # unit + integration tests
├── Dockerfile
├── docker-compose.yml
└── .env.example

---

Installation & configuration

1. Create the GitHub App

1. Go to Settings → Developer settings → GitHub Apps → New GitHub App.
2. Set a name (e.g. wave-sprint-ai-auditor).
3. Webhook URL: https://your-host.example.com/api/github/webhooks.
4. Webhook secret: a long random string (this becomes WEBHOOK_SECRET).
5. Permissions (Repository):
   • Checks: Read & write
   • Issues: Read & write
   • Pull requests: Read & write
6. Events to subscribe:
   • Pull requests
   • Issue comment
   • Ping
7. Download the generated private key (PEM) and note the App ID.

2. Configure the app

cp .env.example .env

Fill in at minimum:

Variable	Description
GITHUB_APP_ID	the app ID from step 7
GITHUB_APP_PRIVATE_KEY_PATH	path to the PEM file (or inline GITHUB_APP_PRIVATE_KEY)
WEBHOOK_SECRET	the webhook secret you set when creating the app

Optional tuning (see .env.example):

• DEFAULT_POINT_LEVEL — trivial | medium | high (default medium).
• MAX_TOTAL_CHANGED_LINES — safety guard (default 1500).
• INSTALL_TIMEOUT_MS, TEST_TIMEOUT_MS, LINT_TIMEOUT_MS.
• LOG_LEVEL — debug | info | warn | error.
• GITHUB_BASE_URL — set for GitHub Enterprise Server.

3. Install the app on the repositories you want to audit

Install the app on the org/repo that hosts the Wave's bounty repos. Point-level detection works best when the linked issue carries a label like point: medium or a body containing points: 5.

---

Local development

npm install
npm run dev          # starts the server with --watch

Test the audit pipeline without GitHub

The app ships with a local CLI that audits a folder directly (no clone, no credentials):

node scripts/audit-local.mjs --repo ../some-repo --points medium
# exit code 0 → PASS, 1 → FAIL/NEEDS_REVIEW (useful for CI gates)

Or try it on the bundled fixtures:

node scripts/audit-local.mjs --repo test/fixtures/sample-repo --points medium   # PASS
node scripts/audit-local.mjs --repo test/fixtures/sample-repo-issues --points trivial   # FAIL

Simulate a webhook locally

Sign a payload and POST it to the running server (the signature uses the webhook secret):

node -e '
const crypto = require("crypto");
const body = JSON.stringify(require("./test/webhook.fixture.json"));
const sig = "sha256=" + crypto.createHmac("sha256", process.env.WEBHOOK_SECRET).update(body).digest("hex");
fetch("http://localhost:3000/api/github/webhooks", {
  method: "POST",
  headers: { "content-type": "application/json", "x-github-event": "ping",
             "x-github-delivery": "dev", "x-hub-signature-256": sig },
  body,
}).then((r) => console.log(r.status));
'

---

Testing

npm test            # vitest run
npm run lint        # eslint on src/scripts/test

The suite covers:

• Unit: complexity counting, diff parsing, point-level detection, verdict/score logic, scorecard rendering, test-output parsing, webhook signature verification.
• Integration: the full audit pipeline runs against real fixture repos — a clean repo (PASS) and a messy repo with lint errors, a failing test, and an over-complex function (FAIL).
• Webhook/E2E: signed-payload routing, draft skipping, /reaudit command, failure reporting through check runs — all with an injected fake GitHub client, so no network access is needed.

---

Deployment

Docker

# place your PEM at ./github-app.private-key.pem, fill .env, then:
docker compose up -d --build

Exposes port 3000; /health is available for liveness checks.

Kubernetes / bare metal

• Run node src/server.js behind TLS (e.g. nginx/caddy) since GitHub webhooks require HTTPS.
• The only external dependencies are git and npm (or the package manager used by the audited repos) in the runtime image.

---

Point level reference

Level	Files	Added lines	Removed lines	Max function complexity
Trivial	≤ 2	≤ 60	≤ 40	≤ 5
Medium	≤ 5	≤ 200	≤ 120	≤ 10
High	≤ 10	≤ 500	≤ 300	≤ 16

These live in src/audit/points.js (POINT_THRESHOLDS) and are applied against the changed lines/functions only.

How the verdict is computed

• Lint errors → fail; warnings → minor penalty.
• Failing tests → fail; missing test script → informational (partial credit).
• Complexity over the level's limits → fail when a changed function exceeds the per-function ceiling, otherwise a warning.
• PASS when there are no blockers and the score is ≥ 85, FAIL when any blocker exists (lint errors, failing tests, over-complex changed functions), otherwise NEEDS_REVIEW.

---

Security notes

• Webhook requests are HMAC-verified (sha256).
• The app authenticates per-installation with short-lived installation tokens (never the app's private key).
• Git cloning uses a GIT_ASKPASS helper so the token never appears in process arguments.
• Audited code runs in an isolated temp workspace that is deleted afterwards (WORKSPACE_DIR).
• Private key is loaded from the environment or a file mount — never committed.
• A diff-size guard prevents resource exhaustion from huge PRs.
• The auditor uses its own ESLint config instead of the PR repo's, so a repo's config can't crash the run or pull in arbitrary plugins.

---

Known limits

• Linting currently targets JavaScript/JSX (.js, .mjs, .cjs, .jsx). TypeScript files are skipped for lint/complexity (listed as non-parsed in the scorecard). Extension is straightforward via @typescript-eslint.
• Test detection relies on the repo's test script and common runner output formats; exotic runners may fall back to the exit-code signal.
• Point levels are mapped from labels/bodies; if the convention differs in your Wave, adjust detectPointLevel in src/audit/points.js.