feat: add initial-setup skill for first-time deployment

.claude/skills/initial-setup/SKILL.md

@@ -0,0 +1,282 @@
+---
+name: initial-setup
+description: First-time deployment wizard for databricks-claw — sets up Slack app, UC connection, secrets, and deploys
+user-invocable: true
+allowed-tools: Bash, Read, Edit, Write, Glob, Grep
+---
+
+# Initial Setup Wizard for databricks-claw
+
+Guide the user through a complete first-time deployment of their personal Claw instance.
+Work through each phase IN ORDER. After each phase, confirm everything passed before moving on.
+If something is missing or broken, STOP and help the user fix it before proceeding.
+
+CRITICAL: NEVER ask the user to paste or type a token, secret, or credential into this conversation.
+Always use `databricks secrets` commands which accept input interactively (never echoed).
+
+---
+
+## Phase 1 — Prerequisites Check
+
+Verify all required tools are installed and authenticated.
+
+1. Check `databricks` CLI is installed:
+   ```
+   databricks --version
+   ```
+2. Check the CLI is authenticated and can reach the workspace:
+   ```
+   databricks auth describe
+   ```
+   Confirm the output shows a valid host and authentication method. Note the workspace host URL — you will need it later.
+3. Check `uv` is installed:
+   ```
+   uv --version
+   ```
+4. Verify the user is in the repo root by checking for `databricks.yml`:
+   ```
+   test -f databricks.yml && echo "OK: in repo root" || echo "ERROR: databricks.yml not found"
+   ```
+
+If any check fails, tell the user what to install/configure and STOP.
+
+---
+
+## Phase 2 — Databricks App Configuration (app.yaml)
+
+### 2a. Check if app.yaml exists
+
+If `app.yaml` does NOT exist, copy from the template:
+```
+cp app.example.yml app.yaml
+```
+
+### 2b. Walk through required env vars
+
+Read `app.yaml` and guide the user through each field. The required env vars are:
+
+| Env Var | Source | Description |
+|---|---|---|
+| `DATABRICKS_USER_PAT` | `valueFrom: user-pat` (secret) | User's Personal Access Token — injected from the `user-pat` secret resource in `resources/app.yml` |
+| `ANTHROPIC_AUTH_TOKEN` | `valueFrom: user-pat` (same secret) | Passed to Claude Agent SDK — uses the same PAT |
+| `ANTHROPIC_BASE_URL` | value | AI Gateway URL. Format: `https://<workspace-id>.ai-gateway.cloud.databricks.com/anthropic` |
+| `ANTHROPIC_MODEL` | value | AI Gateway serving endpoint name (default: `databricks-claude-sonnet-4-6`) |
+| `ANTHROPIC_CUSTOM_HEADERS` | value | Must be `x-databricks-use-coding-agent-mode: true` |
+| `CLAUDE_CODE_DISABLE_EXPERIMENTAL_BETAS` | value | Must be `1` |
+| `SLACK_UC_CONNECTION` | value | UC connection name for Slack Web API (e.g. `slack`) |
+| `CLAW_MEMORY_VOLUME_PATH` | value | UC Volume path for memory. Format: `/Volumes/<catalog>/default/claw_memory` |
+
+Optional env vars (can be added later):
+| Env Var | Description |
+|---|---|
+| `UC_MCP_CONNECTIONS` | Comma-separated UC connection names for external MCP servers |
+| `UC_MANAGED_MCPS` | Comma-separated managed MCP specs (`genie:<id>`, `vector-search:<fqn>`, `sql:<warehouse_id>`, `uc-functions:<fqn>`) |
+| `UC_CUSTOM_MCPS` | Comma-separated `name=url` pairs for custom HTTP MCP servers |
+| `CLAW_NAME` | Bot display name (default: `claw`) |
+| `POLL_INTERVAL_SECONDS` | Polling interval in seconds (default: `5`) |
+| `ACTIVE_THREAD_WINDOW_DAYS` | Days before threads are auto-evicted (default: `7`) |
+| `MAX_ACTIVE_THREADS` | Max concurrent active threads (default: `20`) |
+| `CLAW_REPLY_HEADER` | Whether to prefix replies with bot name (default: `true`) |
+| `CLAW_CLOSE_COMMANDS` | Comma-separated close commands (default: `\close,close,close thread,...`) |
+| `CLAW_TRIGGER_REACTION` | Emoji reaction name to trigger Claw (default: `onit`) |
+| `CLAW_REACTION_CHANNELS` | Comma-separated channel IDs where reaction triggers are monitored |
+
+For each required field:
+- If it has a placeholder value (like `<workspace-id>`), ask the user for the real value and update `app.yaml`.
+- If it references a secret (`valueFrom`), do NOT ask for the secret value — that is handled in Phase 4.
+- **Validate `ANTHROPIC_BASE_URL` workspace ID matches the authenticated workspace.** Extract the workspace ID from the URL (the numeric prefix in `https://<workspace-id>.ai-gateway.cloud.databricks.com/anthropic`) and compare it to the actual workspace ID by running:
+  ```bash
+  WORKSPACE_HOST=$(databricks auth describe 2>&1 | awk '/^Host:/{print $2; exit}' | sed 's:/*$::')
+  TOKEN=$(databricks auth token --host "$WORKSPACE_HOST" 2>/dev/null | python3 -c "import sys,json; print(json.load(sys.stdin)['access_token'])")
+  ACTUAL_WS_ID=$(curl -sI "${WORKSPACE_HOST}/api/2.0/clusters/spark-versions" -H "Authorization: Bearer ${TOKEN}" | grep -i '^x-databricks-org-id:' | awk '{print $2}' | tr -d '\r')
+  URL_WS_ID=$(grep 'ai-gateway.cloud.databricks.com' app.yaml | sed 's|.*https://||' | sed 's|\..*||')
+  echo "Workspace ID from auth:  ${ACTUAL_WS_ID}"
+  echo "Workspace ID from URL:   ${URL_WS_ID}"
+  if [ "$ACTUAL_WS_ID" = "$URL_WS_ID" ]; then echo "✓ MATCH"; else echo "✗ MISMATCH — update ANTHROPIC_BASE_URL in app.yaml"; fi
+  ```
+  If they don't match, update `app.yaml` with the correct workspace ID and STOP until fixed.
+- For `CLAW_MEMORY_VOLUME_PATH`, confirm the catalog name matches `var.catalog_name` in `databricks.yml`.
+
+### 2c. Check databricks.yml variables
+
+Read `databricks.yml` and confirm the variable defaults make sense for the user's workspace:
+- `ai_gateway_endpoint` — must match the `ANTHROPIC_MODEL` value in `app.yaml`
+- `catalog_name` — must match the catalog in `CLAW_MEMORY_VOLUME_PATH`
+- `slack_uc_connection` — must match `SLACK_UC_CONNECTION` in `app.yaml`
+- `secret_scope` — the Databricks secret scope name (default: `claw`)
+
+Ask the user to confirm or update these.
+
+---
+
+## Phase 3 — Slack UC Connection Verification
+
+The person running this setup may not have admin access to the Slack app itself.
+Instead of verifying Slack scopes manually, we validate that the UC connection is configured and can authenticate with Slack.
+
+### 3a. Verify the Slack UC Connection exists
+
+Run:
+```
+databricks connections get <SLACK_UC_CONNECTION_NAME>
+```
+
+Where `<SLACK_UC_CONNECTION_NAME>` is the value from `app.yaml` (e.g. `slack`).
+
+If the connection does not exist, guide the user to create it in the Databricks UI:
+1. Go to Catalog > External Connections > Create Connection
+2. Connection type: HTTP
+3. Host: `https://slack.com`
+4. Auth type: Bearer Token
+5. Token: their Slack Bot User OAuth Token (from the Slack app settings)
+
+### 3b. Test the Slack UC Connection
+
+Test the connection by calling `auth.test` through the Databricks external-function endpoint.
+The Slack client in this codebase uses `POST /api/2.0/external-function` with the connection name.
+
+Run this curl command (replacing placeholders with values from `databricks auth describe` and `app.yaml`):
+
+```bash
+WORKSPACE_HOST=$(databricks auth describe 2>&1 | awk '/^Host:/{print $2; exit}' | sed 's:/*$::')
+SLACK_CONN=$(grep -A1 'SLACK_UC_CONNECTION' app.yaml | grep 'value:' | awk '{print $2}' | tr -d '"')
+TOKEN=$(databricks auth token --host "$WORKSPACE_HOST" 2>/dev/null | python3 -c "import sys,json; print(json.load(sys.stdin)['access_token'])")
+
+curl -s -X POST "${WORKSPACE_HOST}/api/2.0/external-function" \
+  -H "Authorization: Bearer ${TOKEN}" \
+  -H "Content-Type: application/json" \
+  -d "{\"connection_name\": \"${SLACK_CONN}\", \"method\": \"POST\", \"path\": \"/auth.test\", \"json\": \"{}\"}" | python3 -m json.tool
+```
+
+The response should contain `"ok": true` and show the bot's `user_id`, `team`, and `bot_id`.
+
+If it fails:
+- Connection not found → go back to 3b
+- `invalid_auth` → the Slack bot token in the UC connection is wrong or expired
+- Network error → check the connection host is `https://slack.com`
+
+STOP and confirm the test passes before proceeding.
+
+---
+
+## Phase 4 — Secrets Verification
+
+The app needs these secrets. The secret scope is defined in `databricks.yml` variable `secret_scope` (default: `claw`).
+
+| Secret Key | Purpose |
+|---|---|
+| `user-pat` | Databricks Personal Access Token — used for AI Gateway auth and UC connection proxy calls |
+
+### 4a. Ensure the secret scope exists
+
+```
+databricks secrets list-scopes | grep <scope>
+```
+
+If the scope does not exist:
+```
+databricks secrets create-scope <scope>
+```
+
+### 4b. Check each secret exists
+
+For each secret key, check if it exists (this returns metadata only, NOT the value):
+```
+databricks secrets list-secrets <scope>
+```
+
+### 4c. Set missing secrets
+
+For any missing secret, instruct the user to run:
+```
+databricks secrets put-secret <scope> <key>
+```
+
+This opens an interactive editor — the value is never echoed or logged. The user pastes the value there.
+
+For the `user-pat` secret specifically: this is a Databricks Personal Access Token. The user can create one at:
+`https://<workspace-host>/settings/user/developer/access-tokens`
+
+CRITICAL: NEVER ask the user to paste or type their token into this conversation.
+Always use `databricks secrets put-secret` which handles input securely.
+
+---
+
+## Phase 5 — Deploy
+
+### 5a. Full deploy
+
+Run the end-to-end deployment (install deps, validate bundle, deploy, run migrations, start app):
+
+```
+make full-deploy
+```
+
+If it fails, read the error output and help the user fix the issue. Common problems:
+- Validation failure → misconfigured variable or missing resource in `databricks.yml`
+- Deploy failure → permissions or resource conflicts
+- Migration failure → Lakebase not yet provisioned (retry after a minute)
+- Start failure → app configuration issue in `app.yaml`
+
+### 5b. Verify deployment
+
+Check that the app is running:
+```
+databricks apps get claw-app
+```
+
+The status should show the app is ACTIVE or RUNNING.
+
+Then tail the logs briefly to confirm startup:
+```
+databricks apps get-logs claw-app --limit 50
+```
+
+Look for:
+- `SlackPoller started` — confirms the poller is running
+- `Resolved Slack user_id=...` — confirms Slack auth worked
+- No stack traces or startup errors
+
+If there are errors, read them and help the user fix the issue before proceeding.
+
+---
+
+## Phase 6 — Smoke Test
+
+### 6a. Identify the DM channel
+
+From the deployment logs, the bot should have resolved its own user_id and opened a self-DM channel.
+Tell the user:
+
+1. Open Slack
+2. Go to your DM with yourself (search for your own name)
+3. Send a message: `@claw hello, are you there?`
+4. Wait 10-15 seconds (the poller runs every `POLL_INTERVAL_SECONDS`)
+5. Claw should reply in a thread
+
+### 6b. Confirm success
+
+Ask the user to confirm they received a reply. If they did — setup is complete!
+
+If not, check:
+- App logs for errors: `databricks apps get-logs claw-app --limit 100`
+- That the Slack bot is in the workspace
+- That the `SLACK_UC_CONNECTION` points to the right Slack bot token
+- That the `ANTHROPIC_BASE_URL` and `ANTHROPIC_MODEL` are correct for their AI Gateway
+
+---
+
+## Completion
+
+When all phases pass, summarize:
+- Workspace host
+- App name: `claw-app`
+- Slack UC connection name
+- Memory volume path
+- AI Gateway endpoint
+
+Remind the user:
+- To update code and redeploy: `make full-deploy`
+- To customize personality: edit `claw/CLAUDE.md`
+- To add MCP tools: use the `/add-external-mcp`, `/add-managed-mcp`, or `/add-custom-mcp` skills
+- To add skills: create `.claude/skills/<name>/SKILL.md`

Makefile

@@ -1,4 +1,4 @@
-.PHONY: help install test lint requirements migrate deploy-dev deploy-prod deploy-destroy clean
+.PHONY: help install test lint requirements migrate full-deploy deploy-destroy clean
 
 help: ## Show this help
 	@grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | awk 'BEGIN {FS = ":.*?## "}; {printf "  %-15s %s\n", $$1, $$2}'
@@ -39,12 +39,8 @@ run: ## Run the app locally (requires .env)
 
 # ── Deploy ────────────────────────────────────────────────────────────────────
 
-deploy-dev: ## Full deploy to dev (build + validate + deploy + migrate + start)
-	bash scripts/deploy.sh full --target dev
-
-deploy-prod: ## Full deploy to prod (requires confirmation)
-	@read -p "Deploy to PROD? [y/N] " confirm && [ "$$confirm" = "y" ] || exit 1
-	bash scripts/deploy.sh full --target prod
+full-deploy: ## Full deploy (build + validate + deploy + migrate + start)
+	bash scripts/deploy.sh full
 
 deploy-destroy: ## Tear down the Databricks App
 	databricks bundle destroy