diff --git a/.agent/skills/onboarding/SKILL.md b/.agent/skills/onboarding/SKILL.md new file mode 100644 index 000000000..5b1e71f2f --- /dev/null +++ b/.agent/skills/onboarding/SKILL.md @@ -0,0 +1,311 @@ +--- +name: onboarding +description: + Deploy your own Open-Inspect instance. Use when the user wants to set up, deploy, or onboard to + Open-Inspect. Guides through repository setup, credential collection, Terraform deployment, and + verification with user handoffs. +user-invocable: true +allowed-tools: Bash, Read, Write, Edit, Glob, Grep, AskUserQuestion, TodoWrite +--- + +# Open-Inspect Deployment Guide + +You are guiding the user through deploying their own instance of Open-Inspect. This is a multi-phase +process requiring user interaction for credential collection and external service configuration. + +## Before Starting + +Use TodoWrite to create a checklist tracking these phases: + +1. Initial setup questions +2. Repository setup +3. Credential collection (Cloudflare, Vercel, Modal, Anthropic) +4. GitHub App creation +5. Slack App creation (if enabled) +6. Security secrets generation +7. Terraform configuration +8. Terraform deployment (two phases) +9. Post-deployment Slack setup (if enabled) +10. Post-deployment GitHub Bot setup (if enabled) +11. Web app deployment +12. Verification +13. CI/CD setup (optional) + +## Phase 1: Initial Questions + +First, generate a random suffix suggestion for the user: + +```bash +echo "Suggested deployment name: $(openssl rand -hex 3)" +``` + +Use AskUserQuestion to gather: + +1. **Directory location** - Where to create the project (default: current directory or + ~/workplace/open-inspect-{suffix}) +2. **GitHub account** - Which account/org hosts the private repo +3. **Deployment name** - A globally unique identifier for URLs (e.g., their GitHub username, company + name, or the random suffix generated above). Explain this creates URLs like + `open-inspect-{deployment_name}.vercel.app` and must be unique across all Vercel users. +4. **Slack integration** - Yes or No +5. **GitHub bot integration** - Yes or No (automated PR reviews and comment-triggered actions) +6. **Prerequisites confirmation** - Confirm they have accounts on Cloudflare, Vercel, Modal, + Anthropic + +## Phase 2: Repository Setup + +Execute these commands (substitute values from Phase 1): + +```bash +mkdir -p {directory_path} +gh repo create {github_account}/open-inspect-{name} --private --description "Open-Inspect deployment" +cd {directory_path} +git clone git@github.com:ColeMurray/open-inspect.git . +git remote rename origin upstream +git remote add origin git@github.com:{github_account}/open-inspect-{name}.git +git push -u origin main +npm install +npm run build -w @open-inspect/shared +``` + +## Phase 3: Credential Collection + +Hand off to user for each service. Use AskUserQuestion to collect credentials. + +### Cloudflare + +Tell the user: + +- **Account ID**: Found in dashboard URL or account overview +- **Workers Subdomain**: Workers & Pages → Overview, **bottom-right** panel shows + `*.YOUR-SUBDOMAIN.workers.dev` +- **API Token**: Create at https://dash.cloudflare.com/profile/api-tokens with template "Edit + Cloudflare Workers" + permissions for Workers KV Storage (Edit), Workers R2 Storage (Edit), D1 + (Edit) + +### R2 Bucket + +Check wrangler login status, then create bucket: + +```bash +wrangler whoami +wrangler r2 bucket create open-inspect-{name}-tf-state +``` + +Tell user to create R2 API Token at R2 → Overview → Manage R2 API Tokens with "Object Read & Write" +permission. + +### Vercel + +- **API Token**: https://vercel.com/account/tokens +- **Team/Account ID**: Settings → "Your ID" (even personal accounts have one, usually starts with + `team_`) + +### Modal + +- **Token ID and Secret**: https://modal.com/settings or `modal token new` +- **Workspace name**: Visible in Modal dashboard URL + +Then set the token: + +```bash +modal token set --token-id {token_id} --token-secret {token_secret} +modal profile current +``` + +### Anthropic + +- **API Key**: https://console.anthropic.com (starts with `sk-ant-`) + +## Phase 4: GitHub App Setup + +Guide user through creating a GitHub App (handles both OAuth and repo access): + +1. Go to https://github.com/settings/apps → "New GitHub App" +2. **Name**: `Open-Inspect-{YourName}` (globally unique) +3. **Homepage URL**: `https://open-inspect-{deployment_name}.vercel.app` +4. **Webhook**: Uncheck "Active" +5. **Callback URL** (under "Identifying and authorizing users"): + `https://open-inspect-{deployment_name}.vercel.app/api/auth/callback/github` + - **CRITICAL**: Must match deployed Vercel URL exactly! +6. **Repository permissions**: Contents (Read & Write), Issues (Read & Write), Pull requests (Read & + Write), Metadata (Read-only) +7. Create app, note **App ID** +8. Generate **Client Secret**, note **Client ID** and **Client Secret** +9. Generate **Private Key** (downloads .pem file) +10. Install app on account, note **Installation ID** from URL + +After receiving the .pem path, convert to PKCS#8: + +```bash +openssl pkcs8 -topk8 -inform PEM -outform PEM -nocrypt -in {pem_path} -out /tmp/github-app-key-pkcs8.pem +cat /tmp/github-app-key-pkcs8.pem +``` + +## Phase 5: Slack App Setup (If Enabled) + +Guide user: + +1. https://api.slack.com/apps → "Create New App" → "From scratch" +2. OAuth & Permissions → Add scopes: `app_mentions:read`, `chat:write`, `channels:history`, + `channels:read`, `groups:history`, `groups:read`, `im:history`, `im:read`, `reactions:write` +3. Install to Workspace, note **Bot Token** (`xoxb-...`) +4. Basic Information → note **Signing Secret** +5. **App Home and Event Subscriptions configured AFTER deployment** (worker must be running for URL + verification) + +## Phase 6: Generate Security Secrets + +```bash +echo "token_encryption_key: $(openssl rand -base64 32)" +echo "repo_secrets_encryption_key: $(openssl rand -base64 32)" +echo "internal_callback_secret: $(openssl rand -base64 32)" +echo "nextauth_secret: $(openssl rand -base64 32)" +echo "modal_api_secret: $(openssl rand -hex 32)" +echo "github_webhook_secret: $(openssl rand -hex 32)" # Only if GitHub bot enabled +``` + +## Phase 7: Terraform Configuration + +Create `terraform/environments/production/backend.tfvars`: + +```hcl +access_key = "{r2_access_key}" +secret_key = "{r2_secret_key}" +bucket = "open-inspect-{name}-tf-state" +endpoints = { + s3 = "https://{cloudflare_account_id}.r2.cloudflarestorage.com" +} +``` + +Create `terraform/environments/production/terraform.tfvars` with all collected values. Set: + +```hcl +enable_durable_object_bindings = false +enable_service_bindings = false +``` + +If GitHub bot is enabled, also set: + +```hcl +enable_github_bot = true +github_webhook_secret = "{generated_value}" +github_bot_username = "{app-slug}[bot]" +``` + +## Phase 8: Terraform Deployment (Two-Phase) + +**Important**: Build the workers before running Terraform (Terraform references the built bundles): + +```bash +npm run build -w @open-inspect/control-plane -w @open-inspect/slack-bot -w @open-inspect/github-bot +``` + +**Phase 1** (bindings disabled): + +```bash +cd terraform/environments/production +terraform init -backend-config=backend.tfvars +terraform apply +``` + +**Phase 2** (after Phase 1 succeeds): Update tfvars to set both bindings to `true`, then: + +```bash +terraform apply +``` + +## Phase 9: Complete Slack Setup (If Enabled) + +After Terraform deployment, guide user: + +### Enable App Home + +1. App Home → Show Tabs → Enable **"Home Tab"** +2. Save Changes + +The App Home provides a settings interface where users can configure their preferred Claude model. + +### Configure Event Subscriptions + +1. Event Subscriptions → Enable → Request URL: + `https://open-inspect-slack-bot-{deployment_name}.{subdomain}.workers.dev/events` +2. Wait for "Verified" checkmark +3. Subscribe to bot events: `app_home_opened`, `app_mention`, `message.im` + +### Configure Interactivity + +4. Interactivity → Enable → Request URL: + `https://open-inspect-slack-bot-{deployment_name}.{subdomain}.workers.dev/interactions` + +### Invite Bot to Channels + +5. Invite bot to channels: `/invite @BotName` + +## Phase 10: Complete GitHub Bot Setup (If Enabled) + +After Terraform deployment, guide user: + +### Configure Webhook on GitHub App + +1. Go to GitHub App settings → your app +2. Under **Webhook**: check **"Active"** +3. **Webhook URL**: + `https://open-inspect-github-bot-{deployment_name}.{subdomain}.workers.dev/webhooks/github` +4. **Webhook secret**: Enter the `github_webhook_secret` value +5. Under **Subscribe to events**, check: **Pull requests**, **Issue comments**, **Pull request + review comments** +6. Save changes + +### Find Bot Username + +The bot username is the App's slug with `[bot]` appended. E.g., if the app is `My-Inspect-App`, the +bot username is `my-inspect-app[bot]`. Confirm this matches `github_bot_username` in +terraform.tfvars. + +### Usage + +- **Code Review**: Assign the bot as a PR reviewer +- **Comment Actions**: @mention the bot in a PR comment with instructions + +## Phase 11: Web App Deployment + +```bash +npx vercel link --project open-inspect-{deployment_name} +npx vercel --prod +``` + +## Phase 12: Verification + +```bash +curl https://open-inspect-control-plane-{deployment_name}.{subdomain}.workers.dev/health +curl https://{workspace}--open-inspect-api-health.modal.run +curl -I https://open-inspect-{deployment_name}.vercel.app +``` + +Present deployment summary table. Instruct user to test: visit web app, sign in with GitHub, create +session, send prompt. + +## Phase 13: CI/CD Setup (Optional) + +Ask if user wants GitHub Actions CI/CD. If yes, use `gh secret set` for all required secrets. + +## Error Handling + +- **"redirect_uri is not associated"**: Callback URL mismatch - update GitHub App settings +- **Durable Object errors**: Must follow two-phase deployment +- **Slack bot not responding**: Check Event Subscriptions URL verified, bot invited to channel, + reinstall if scopes changed +- **GitHub bot not responding**: Check webhook URL, secret, `enable_github_bot = true`, and + `github_bot_username` matches the App's bot login +- **Vercel build fails**: Terraform configures the monorepo build commands automatically +- **"no such file or directory" for dist/index.js**: Build workers before Terraform: + `npm run build -w @open-inspect/control-plane -w @open-inspect/slack-bot -w @open-inspect/github-bot` +- **Worker deployment fails**: Build shared package first: `npm run build -w @open-inspect/shared` + +## Important Notes + +- Track all collected credentials securely throughout the process +- Never log sensitive values +- The callback URL MUST match the actual deployed Vercel URL +- Two-phase Terraform deployment is required due to Cloudflare Durable Object constraints