Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
311 changes: 311 additions & 0 deletions .agent/skills/onboarding/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,311 @@
---
name: onboarding
description:
Deploy your own Open-Inspect instance. Use when the user wants to set up, deploy, or onboard to
Open-Inspect. Guides through repository setup, credential collection, Terraform deployment, and
verification with user handoffs.
user-invocable: true
allowed-tools: Bash, Read, Write, Edit, Glob, Grep, AskUserQuestion, TodoWrite
---

# Open-Inspect Deployment Guide

You are guiding the user through deploying their own instance of Open-Inspect. This is a multi-phase
process requiring user interaction for credential collection and external service configuration.

## Before Starting

Use TodoWrite to create a checklist tracking these phases:

1. Initial setup questions
2. Repository setup
3. Credential collection (Cloudflare, Vercel, Modal, Anthropic)
4. GitHub App creation
5. Slack App creation (if enabled)
6. Security secrets generation
7. Terraform configuration
8. Terraform deployment (two phases)
9. Post-deployment Slack setup (if enabled)
10. Post-deployment GitHub Bot setup (if enabled)
11. Web app deployment
12. Verification
13. CI/CD setup (optional)

## Phase 1: Initial Questions

First, generate a random suffix suggestion for the user:

```bash
echo "Suggested deployment name: $(openssl rand -hex 3)"
```

Use AskUserQuestion to gather:

1. **Directory location** - Where to create the project (default: current directory or
~/workplace/open-inspect-{suffix})
2. **GitHub account** - Which account/org hosts the private repo
3. **Deployment name** - A globally unique identifier for URLs (e.g., their GitHub username, company
name, or the random suffix generated above). Explain this creates URLs like
`open-inspect-{deployment_name}.vercel.app` and must be unique across all Vercel users.
4. **Slack integration** - Yes or No
5. **GitHub bot integration** - Yes or No (automated PR reviews and comment-triggered actions)
6. **Prerequisites confirmation** - Confirm they have accounts on Cloudflare, Vercel, Modal,
Anthropic

## Phase 2: Repository Setup

Execute these commands (substitute values from Phase 1):

```bash
mkdir -p {directory_path}
gh repo create {github_account}/open-inspect-{name} --private --description "Open-Inspect deployment"
cd {directory_path}
git clone git@github.com:ColeMurray/open-inspect.git .
git remote rename origin upstream
git remote add origin git@github.com:{github_account}/open-inspect-{name}.git
git push -u origin main
npm install
npm run build -w @open-inspect/shared
```

## Phase 3: Credential Collection

Hand off to user for each service. Use AskUserQuestion to collect credentials.

### Cloudflare

Tell the user:

- **Account ID**: Found in dashboard URL or account overview
- **Workers Subdomain**: Workers & Pages → Overview, **bottom-right** panel shows
`*.YOUR-SUBDOMAIN.workers.dev`
- **API Token**: Create at https://dash.cloudflare.com/profile/api-tokens with template "Edit
Cloudflare Workers" + permissions for Workers KV Storage (Edit), Workers R2 Storage (Edit), D1
(Edit)

### R2 Bucket

Check wrangler login status, then create bucket:

```bash
wrangler whoami
wrangler r2 bucket create open-inspect-{name}-tf-state
```

Tell user to create R2 API Token at R2 → Overview → Manage R2 API Tokens with "Object Read & Write"
permission.

### Vercel

- **API Token**: https://vercel.com/account/tokens
- **Team/Account ID**: Settings → "Your ID" (even personal accounts have one, usually starts with
`team_`)

### Modal

- **Token ID and Secret**: https://modal.com/settings or `modal token new`
- **Workspace name**: Visible in Modal dashboard URL

Then set the token:

```bash
modal token set --token-id {token_id} --token-secret {token_secret}
modal profile current
```

### Anthropic

- **API Key**: https://console.anthropic.com (starts with `sk-ant-`)

## Phase 4: GitHub App Setup

Guide user through creating a GitHub App (handles both OAuth and repo access):

1. Go to https://github.com/settings/apps → "New GitHub App"
2. **Name**: `Open-Inspect-{YourName}` (globally unique)
3. **Homepage URL**: `https://open-inspect-{deployment_name}.vercel.app`
4. **Webhook**: Uncheck "Active"
5. **Callback URL** (under "Identifying and authorizing users"):
`https://open-inspect-{deployment_name}.vercel.app/api/auth/callback/github`
- **CRITICAL**: Must match deployed Vercel URL exactly!
6. **Repository permissions**: Contents (Read & Write), Issues (Read & Write), Pull requests (Read &
Write), Metadata (Read-only)
7. Create app, note **App ID**
8. Generate **Client Secret**, note **Client ID** and **Client Secret**
9. Generate **Private Key** (downloads .pem file)
10. Install app on account, note **Installation ID** from URL

After receiving the .pem path, convert to PKCS#8:

```bash
openssl pkcs8 -topk8 -inform PEM -outform PEM -nocrypt -in {pem_path} -out /tmp/github-app-key-pkcs8.pem
cat /tmp/github-app-key-pkcs8.pem
```

## Phase 5: Slack App Setup (If Enabled)

Guide user:

1. https://api.slack.com/apps → "Create New App" → "From scratch"
2. OAuth & Permissions → Add scopes: `app_mentions:read`, `chat:write`, `channels:history`,
`channels:read`, `groups:history`, `groups:read`, `im:history`, `im:read`, `reactions:write`
3. Install to Workspace, note **Bot Token** (`xoxb-...`)
4. Basic Information → note **Signing Secret**
5. **App Home and Event Subscriptions configured AFTER deployment** (worker must be running for URL
verification)

## Phase 6: Generate Security Secrets

```bash
echo "token_encryption_key: $(openssl rand -base64 32)"
echo "repo_secrets_encryption_key: $(openssl rand -base64 32)"
echo "internal_callback_secret: $(openssl rand -base64 32)"
echo "nextauth_secret: $(openssl rand -base64 32)"
echo "modal_api_secret: $(openssl rand -hex 32)"
echo "github_webhook_secret: $(openssl rand -hex 32)" # Only if GitHub bot enabled
```

## Phase 7: Terraform Configuration

Create `terraform/environments/production/backend.tfvars`:

```hcl
access_key = "{r2_access_key}"
secret_key = "{r2_secret_key}"
bucket = "open-inspect-{name}-tf-state"
endpoints = {
s3 = "https://{cloudflare_account_id}.r2.cloudflarestorage.com"
}
```

Create `terraform/environments/production/terraform.tfvars` with all collected values. Set:

```hcl
enable_durable_object_bindings = false
enable_service_bindings = false
```

If GitHub bot is enabled, also set:

```hcl
enable_github_bot = true
github_webhook_secret = "{generated_value}"
github_bot_username = "{app-slug}[bot]"
```
Comment on lines +169 to +194
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Add web_platform configuration to Terraform variables.

The guide is missing the web_platform configuration in the terraform.tfvars setup. This variable determines where the web app is deployed.

Based on learnings, set web_platform = "cloudflare" in Terraform variables to deploy the web app to Cloudflare Workers via OpenNext; when using Cloudflare, Vercel credentials are not required.

📝 Add web_platform configuration

Add this to the terraform.tfvars configuration example in Phase 7:

 ```hcl
 enable_durable_object_bindings = false
 enable_service_bindings        = false
+web_platform                   = "cloudflare"

</details>

<!-- suggestion_start -->

<details>
<summary>📝 Committable suggestion</summary>

> ‼️ **IMPORTANT**
> Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

```suggestion
Create `terraform/environments/production/terraform.tfvars` with all collected values. Set:

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In @.agent/skills/onboarding/SKILL.md around lines 169 - 194, Update the
terraform variables example in the Phase 7
`terraform/environments/production/terraform.tfvars` snippet to include the
web_platform variable by adding web_platform = "cloudflare"; reference the
existing variables enable_durable_object_bindings and enable_service_bindings in
that same block so the web app is deployed to Cloudflare Workers via OpenNext
and document that Vercel credentials are not required when web_platform is set
to "cloudflare".


## Phase 8: Terraform Deployment (Two-Phase)

**Important**: Build the workers before running Terraform (Terraform references the built bundles):

```bash
npm run build -w @open-inspect/control-plane -w @open-inspect/slack-bot -w @open-inspect/github-bot
```

**Phase 1** (bindings disabled):

```bash
cd terraform/environments/production
terraform init -backend-config=backend.tfvars
terraform apply
```

**Phase 2** (after Phase 1 succeeds): Update tfvars to set both bindings to `true`, then:

```bash
terraform apply
```

## Phase 9: Complete Slack Setup (If Enabled)

After Terraform deployment, guide user:

### Enable App Home

1. App Home → Show Tabs → Enable **"Home Tab"**
2. Save Changes

The App Home provides a settings interface where users can configure their preferred Claude model.

### Configure Event Subscriptions

1. Event Subscriptions → Enable → Request URL:
`https://open-inspect-slack-bot-{deployment_name}.{subdomain}.workers.dev/events`
2. Wait for "Verified" checkmark
3. Subscribe to bot events: `app_home_opened`, `app_mention`, `message.im`

### Configure Interactivity

4. Interactivity → Enable → Request URL:
`https://open-inspect-slack-bot-{deployment_name}.{subdomain}.workers.dev/interactions`

### Invite Bot to Channels

5. Invite bot to channels: `/invite @BotName`

## Phase 10: Complete GitHub Bot Setup (If Enabled)

After Terraform deployment, guide user:

### Configure Webhook on GitHub App

1. Go to GitHub App settings → your app
2. Under **Webhook**: check **"Active"**
3. **Webhook URL**:
`https://open-inspect-github-bot-{deployment_name}.{subdomain}.workers.dev/webhooks/github`
4. **Webhook secret**: Enter the `github_webhook_secret` value
5. Under **Subscribe to events**, check: **Pull requests**, **Issue comments**, **Pull request
review comments**
6. Save changes

### Find Bot Username

The bot username is the App's slug with `[bot]` appended. E.g., if the app is `My-Inspect-App`, the
bot username is `my-inspect-app[bot]`. Confirm this matches `github_bot_username` in
terraform.tfvars.

### Usage

- **Code Review**: Assign the bot as a PR reviewer
- **Comment Actions**: @mention the bot in a PR comment with instructions

## Phase 11: Web App Deployment

```bash
npx vercel link --project open-inspect-{deployment_name}
npx vercel --prod
```
Comment on lines +271 to +276
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

Configure NEXT_PUBLIC_WS_URL environment variable before web app deployment.

The web app deployment is missing a critical environment variable setup. The NEXT_PUBLIC_WS_URL must be configured to point to the control plane worker URL deployed in Phase 8.

Based on learnings, NEXT_PUBLIC_WS_URL must be available at build time.

🔧 Add NEXT_PUBLIC_WS_URL configuration

Add environment variable setup before the Vercel deployment commands:

 ## Phase 11: Web App Deployment
 
+Set the WebSocket URL environment variable (pointing to your control plane worker):
+
+```bash
+npx vercel env add NEXT_PUBLIC_WS_URL production
+# When prompted, enter: https://open-inspect-control-plane-{deployment_name}.{subdomain}.workers.dev
+```
+
+Then deploy:
+
 ```bash
 npx vercel link --project open-inspect-{deployment_name}
 npx vercel --prod

</details>

<details>
<summary>🤖 Prompt for AI Agents</summary>

Verify each finding against the current code and only fix it if needed.

In @.agent/skills/onboarding/SKILL.md around lines 271 - 276, Phase 11 missing
build-time environment configuration: before running the existing deployment
commands (npx vercel link --project open-inspect-{deployment_name} and npx
vercel --prod) add the NEXT_PUBLIC_WS_URL environment variable so it is
available at build time; run the Vercel env add flow (e.g., npx vercel env add
NEXT_PUBLIC_WS_URL production) and set its value to the control plane worker URL
(https://open-inspect-control-plane-{deployment_name}.{subdomain}.workers.dev),
then proceed with the link and prod deploy commands to ensure NEXT_PUBLIC_WS_URL
is injected during the build.


</details>

<!-- fingerprinting:phantom:triton:puma:e390b80b-10c1-4c36-ba04-da681f1b62bf -->

<!-- This is an auto-generated comment by CodeRabbit -->


## Phase 12: Verification

```bash
curl https://open-inspect-control-plane-{deployment_name}.{subdomain}.workers.dev/health
curl https://{workspace}--open-inspect-api-health.modal.run
curl -I https://open-inspect-{deployment_name}.vercel.app
```

Present deployment summary table. Instruct user to test: visit web app, sign in with GitHub, create
session, send prompt.

## Phase 13: CI/CD Setup (Optional)

Ask if user wants GitHub Actions CI/CD. If yes, use `gh secret set` for all required secrets.

## Error Handling

- **"redirect_uri is not associated"**: Callback URL mismatch - update GitHub App settings
- **Durable Object errors**: Must follow two-phase deployment
- **Slack bot not responding**: Check Event Subscriptions URL verified, bot invited to channel,
reinstall if scopes changed
- **GitHub bot not responding**: Check webhook URL, secret, `enable_github_bot = true`, and
`github_bot_username` matches the App's bot login
- **Vercel build fails**: Terraform configures the monorepo build commands automatically
- **"no such file or directory" for dist/index.js**: Build workers before Terraform:
`npm run build -w @open-inspect/control-plane -w @open-inspect/slack-bot -w @open-inspect/github-bot`
- **Worker deployment fails**: Build shared package first: `npm run build -w @open-inspect/shared`

## Important Notes

- Track all collected credentials securely throughout the process
- Never log sensitive values
- The callback URL MUST match the actual deployed Vercel URL
- Two-phase Terraform deployment is required due to Cloudflare Durable Object constraints