diff --git a/README.md b/README.md index f60fefa..6d03f5d 100644 --- a/README.md +++ b/README.md @@ -76,10 +76,10 @@ The pattern is always the same: vague prompt → Claude guesses → wrong output ## Quick Start -### Option A: Claude Code CLI (fastest) +### Option A: npx (fastest — no install) ```bash -claude mcp add preflight -- npx tsx /path/to/preflight/src/index.ts +claude mcp add preflight -- npx -y preflight-dev-serve ``` With environment variables: @@ -87,7 +87,7 @@ With environment variables: ```bash claude mcp add preflight \ -e CLAUDE_PROJECT_DIR=/path/to/your/project \ - -- npx tsx /path/to/preflight/src/index.ts + -- npx -y preflight-dev-serve ``` ### Option B: Clone & configure manually @@ -119,9 +119,11 @@ Restart Claude Code. The tools activate automatically. ```bash npm install -g preflight-dev -claude mcp add preflight -- preflight-dev +claude mcp add preflight -- preflight-dev-serve ``` +> **Note:** `preflight-dev` runs the interactive setup wizard. `preflight-dev-serve` starts the MCP server — that's what you want in your Claude Code config. + --- ## How It Works @@ -335,6 +337,85 @@ After onboarding, you get: --- +## Usage Examples + +Real prompts you'd type in Claude Code once preflight is installed. The tools fire automatically via `preflight_check`, or you can call them directly. + +### Catch a vague prompt before it wastes tokens + +``` +> fix the auth bug + +preflight_check intercepts → clarify_intent fires: + "I found 3 auth-related recent changes: + 1. JWT refresh logic in src/auth/jwt.ts (modified 2h ago) + 2. OAuth callback route in src/routes/oauth.ts (3 open TODOs) + 3. Session middleware in src/middleware/auth.ts (failing test) + Which one? Or describe the symptoms." +``` + +### Scope a multi-step task before starting + +``` +> refactor the database layer to use Drizzle instead of Prisma + +preflight_check triages as MULTI-STEP → scope_work + sequence_tasks fire: + Phase 1: Install drizzle-orm, create schema (src/db/schema.ts) + Phase 2: Migrate queries in src/services/ (12 files) + Phase 3: Update tests (8 files reference prisma mocks) + Phase 4: Remove prisma dependencies, delete prisma/ dir + Estimated: 4 checkpoints, ~45 min +``` + +### Search your session history + +``` +> use search_history to find how I set up the rate limiter + +Searches LanceDB across your indexed sessions: + Found 3 relevant events (similarity > 0.82): + - Session abc123 (Jan 15): Added express-rate-limit to src/middleware/ + - Session def456 (Jan 16): Configured per-route limits in src/routes/api.ts + - Session ghi789 (Feb 2): Fixed rate limit bypass via X-Forwarded-For +``` + +### Grade your prompting habits + +``` +> use prompt_score on "update the thing" + + Grade: D + - Specificity: 1/5 — no file paths, no identifiers + - Scope: 1/5 — "the thing" is completely ambiguous + - Actionability: 2/5 — "update" is vague (refactor? fix? add feature?) + - Done-condition: 0/5 — no way to know when it's done + Suggestion: "Update the price formatter in src/utils/currency.ts + to handle JPY (zero-decimal currency)" +``` + +### Check session health mid-work + +``` +> use check_session_health + + ⚠️ 47 uncommitted files (last commit: 38 min ago) + ⚠️ 23 turns since last checkpoint + 💡 Consider running `checkpoint` before context compaction hits +``` + +### Get a cost estimate + +``` +> use estimate_cost + + This session: ~$2.40 (148K tokens) + Waste from corrections: ~$0.85 (3 wrong→fix cycles) + Preflight savings: ~$1.20 (4 vague prompts caught early) + Net: saving roughly 35% vs unchecked prompting +``` + +--- + ## The 12-Category Scorecard `generate_scorecard` evaluates your prompt discipline across 12 categories. Each one measures something specific about how you interact with Claude Code: @@ -500,6 +581,8 @@ Manual contract definitions that supplement auto-extraction: Environment variables are **fallbacks** — `.preflight/` config takes precedence when present. +> 💡 **Ready-to-use examples:** Copy [`examples/.preflight/`](examples/.preflight/) into your project root for a working starter config with detailed comments. + --- ## Embedding Providers @@ -562,6 +645,95 @@ flowchart TB --- +## Troubleshooting + +### "Cannot find module 'vectordb'" or LanceDB import errors + +LanceDB uses native binaries. If you see module resolution errors: + +```bash +# Clean install with native deps rebuilt +rm -rf node_modules package-lock.json +npm install + +# If still failing, check your Node version (20+ required) +node --version +``` + +On Apple Silicon Macs, make sure you're running a native arm64 Node — not Rosetta. Check with `node -e "console.log(process.arch)"` (should print `arm64`). + +### First run is slow (~90MB model download) + +The local embedding provider ([Xenova/all-MiniLM-L6-v2](https://huggingface.co/Xenova/all-MiniLM-L6-v2)) downloads a ~90MB model on first use. This is a one-time cost — subsequent runs use the cached model. If the download hangs behind a corporate proxy, switch to OpenAI embeddings: + +```bash +export OPENAI_API_KEY=sk-... +export EMBEDDING_PROVIDER=openai +``` + +### "OpenAI API key required for openai embedding provider" + +You set `EMBEDDING_PROVIDER=openai` (or `embeddings.provider: openai` in `.preflight/config.yml`) but didn't provide a key. Either: + +- Set `OPENAI_API_KEY` in your environment, or +- Switch back to local: `export EMBEDDING_PROVIDER=local` + +### Tools not showing up in Claude Code + +1. Make sure the MCP server is registered. Run `claude mcp list` — you should see `preflight`. +2. If missing, re-add it: + ```bash + claude mcp add preflight -- npx tsx /path/to/preflight/src/index.ts + ``` +3. Restart Claude Code after adding. + +### `CLAUDE_PROJECT_DIR` not set + +Some tools (onboarding, session search, contracts) need to know your project root. If they return empty results: + +```bash +claude mcp add preflight \ + -e CLAUDE_PROJECT_DIR=/path/to/your/project \ + -- npx tsx /path/to/preflight/src/index.ts +``` + +Or set it globally: `export CLAUDE_PROJECT_DIR=/path/to/your/project` + +### `.preflight/config.yml` parse errors + +If you see `warning - failed to parse .preflight/config.yml`, your YAML is malformed. Common issues: + +- Tabs instead of spaces (YAML requires spaces) +- Missing quotes around values with special characters +- Incorrect indentation under `related_projects` + +Validate with: `npx yaml-lint .preflight/config.yml` or paste into [yamllint.com](https://www.yamllint.com/). + +### No session data found during onboarding + +`onboard_project` looks for JSONL files in `~/.claude/projects//`. If nothing is found: + +- Make sure you've actually used Claude Code on the project (at least one session) +- Check that `CLAUDE_PROJECT_DIR` matches the exact path Claude Code was opened in +- The path encoding is URL-style — `/Users/jack/my-app` becomes `%2FUsers%2Fjack%2Fmy-app` + +### Ollama embeddings connection refused + +If using Ollama as your embedding provider and getting connection errors: + +```bash +# Make sure Ollama is running +ollama serve + +# Pull the embedding model +ollama pull all-minilm + +# Verify it works +curl http://localhost:11434/api/embed -d '{"model":"all-minilm","input":"test"}' +``` + +--- + ## Contributing This project is young and there's plenty to do. Check the [issues](https://github.com/TerminalGravity/preflight/issues) — several are tagged `good first issue`. diff --git a/bin/serve.js b/bin/serve.js new file mode 100755 index 0000000..279bfe7 --- /dev/null +++ b/bin/serve.js @@ -0,0 +1,10 @@ +#!/usr/bin/env node +// Starts the preflight MCP server (as opposed to the init wizard). +// Used by .mcp.json configs generated by `preflight-dev init`. +import { fileURLToPath } from 'url'; +import { dirname, join } from 'path'; + +const __filename = fileURLToPath(import.meta.url); +const __dirname = dirname(__filename); + +await import(join(__dirname, '../dist/index.js')); diff --git a/examples/.preflight/config.yml b/examples/.preflight/config.yml new file mode 100644 index 0000000..f59170f --- /dev/null +++ b/examples/.preflight/config.yml @@ -0,0 +1,35 @@ +# .preflight/config.yml — Drop this in your project root +# +# This is an example config for a typical Next.js + microservices setup. +# Every field is optional — preflight works with sensible defaults out of the box. +# Commit this to your repo so the whole team gets the same preflight behavior. + +# Profile controls how much detail preflight returns. +# "minimal" — only flags ambiguous+ prompts, skips clarification detail +# "standard" — balanced (default) +# "full" — maximum detail on every non-trivial prompt +profile: standard + +# Related projects for cross-service awareness. +# Preflight will search these for shared types, routes, and contracts +# so it can warn you when a change might break a consumer. +related_projects: + - path: /Users/you/code/auth-service + alias: auth + - path: /Users/you/code/billing-api + alias: billing + - path: /Users/you/code/shared-types + alias: types + +# Behavioral thresholds — tune these to your workflow +thresholds: + session_stale_minutes: 30 # Warn if no activity for this long + max_tool_calls_before_checkpoint: 100 # Suggest a checkpoint after N tool calls + correction_pattern_threshold: 3 # Min corrections before flagging a pattern + +# Embedding provider for semantic search over session history. +# "local" uses Xenova transformers (no API key needed, runs on CPU). +# "openai" uses text-embedding-3-small (faster, needs OPENAI_API_KEY). +embeddings: + provider: local + # openai_api_key: sk-... # Uncomment if using openai provider diff --git a/examples/.preflight/contracts/api.yml b/examples/.preflight/contracts/api.yml new file mode 100644 index 0000000..512543f --- /dev/null +++ b/examples/.preflight/contracts/api.yml @@ -0,0 +1,58 @@ +# .preflight/contracts/api.yml — Manual contract definitions +# +# Define shared types and interfaces that preflight should know about. +# These supplement auto-extracted contracts from your codebase. +# Manual definitions win on name conflicts with auto-extracted ones. +# +# Why manual contracts? +# - Document cross-service interfaces that live in docs, not code +# - Define contracts for external APIs your services consume +# - Pin down types that are implicit (e.g., event payloads) + +- name: User + kind: interface + description: Core user model shared across all services + fields: + - name: id + type: string + required: true + - name: email + type: string + required: true + - name: tier + type: "'free' | 'pro' | 'enterprise'" + required: true + - name: createdAt + type: Date + required: true + +- name: AuthToken + kind: interface + description: JWT payload structure from auth-service + fields: + - name: userId + type: string + required: true + - name: permissions + type: string[] + required: true + - name: expiresAt + type: number + required: true + +- name: WebhookPayload + kind: interface + description: Standard webhook envelope for inter-service events + fields: + - name: event + type: string + required: true + - name: timestamp + type: string + required: true + - name: data + type: Record + required: true + - name: source + type: string + required: true diff --git a/examples/.preflight/triage.yml b/examples/.preflight/triage.yml new file mode 100644 index 0000000..b3d394e --- /dev/null +++ b/examples/.preflight/triage.yml @@ -0,0 +1,45 @@ +# .preflight/triage.yml — Controls how preflight classifies your prompts +# +# The triage engine routes prompts into categories: +# TRIVIAL → pass through (commit, format, lint) +# CLEAR → well-specified, no intervention needed +# AMBIGUOUS → needs clarification before proceeding +# MULTI-STEP → complex task, preflight suggests a plan +# CROSS-SERVICE → touches multiple projects, pulls in contracts +# +# Customize the keywords below to match your domain. + +rules: + # Prompts containing these words are always flagged as AMBIGUOUS. + # Add domain-specific terms that tend to produce vague prompts. + always_check: + - rewards + - permissions + - migration + - schema + - pricing # example: your billing domain + - onboarding # example: multi-step user flows + + # Prompts containing these words skip checks entirely (TRIVIAL). + # These are safe, mechanical tasks that don't need guardrails. + skip: + - commit + - format + - lint + - prettier + - "git push" + + # Prompts containing these words trigger CROSS-SERVICE classification. + # Preflight will search related_projects for relevant types and routes. + cross_service_keywords: + - auth + - notification + - event + - webhook + - billing # matches the related_project alias + +# How aggressively to classify prompts. +# "relaxed" — more prompts pass as clear (experienced users) +# "standard" — balanced (default) +# "strict" — more prompts flagged as ambiguous (new teams, complex codebases) +strictness: standard diff --git a/examples/README.md b/examples/README.md new file mode 100644 index 0000000..778f15d --- /dev/null +++ b/examples/README.md @@ -0,0 +1,35 @@ +# Examples + +## `.preflight/` Config Directory + +The `.preflight/` directory contains example configuration files you can copy into your project root: + +``` +.preflight/ +├── config.yml # Main config — profile, related projects, thresholds +├── triage.yml # Triage rules — keywords, strictness +└── contracts/ + └── api.yml # Manual contract definitions for cross-service types +``` + +### Quick setup + +```bash +# From your project root: +cp -r /path/to/preflight/examples/.preflight .preflight + +# Edit paths in config.yml to match your setup: +$EDITOR .preflight/config.yml +``` + +Then commit `.preflight/` to your repo — your whole team gets the same preflight behavior. + +### What each file does + +| File | Purpose | Required? | +|------|---------|-----------| +| `config.yml` | Profile, related projects, thresholds, embedding config | No — sensible defaults | +| `triage.yml` | Keyword rules for prompt classification | No — sensible defaults | +| `contracts/*.yml` | Manual type/interface definitions for cross-service awareness | No — auto-extraction works without it | + +All files are optional. Preflight works out of the box with zero config — these files let you tune it to your codebase. diff --git a/package.json b/package.json index 9cdabf2..628fc13 100644 --- a/package.json +++ b/package.json @@ -5,7 +5,8 @@ "type": "module", "main": "dist/index.js", "bin": { - "preflight-dev": "./bin/cli.js" + "preflight-dev": "./bin/cli.js", + "preflight-dev-serve": "./bin/serve.js" }, "engines": { "node": ">=20" diff --git a/src/cli/init.ts b/src/cli/init.ts index 996906d..dfaaa25 100644 --- a/src/cli/init.ts +++ b/src/cli/init.ts @@ -104,18 +104,11 @@ async function main(): Promise { } } + // Use npx to run the MCP server via the dedicated serve binary. + // "preflight-dev" runs the init wizard; "preflight-dev-serve" starts the server. config.mcpServers["preflight"] = { command: "npx", - args: ["-y", "preflight-dev@latest"], - env, - }; - - // For the actual server entry point, we need to point to index.ts via tsx - // But npx will resolve the bin entry which is the init script - // So use a different approach: command runs the server - config.mcpServers["preflight"] = { - command: "npx", - args: ["-y", "tsx", "node_modules/preflight/src/index.ts"], + args: ["-y", "preflight-dev-serve"], env, };