From 44fb936680a092aa5801a33de543df109235e61b Mon Sep 17 00:00:00 2001 From: Jack Felke Date: Thu, 5 Mar 2026 05:49:58 -0700 Subject: [PATCH 1/4] docs: add TROUBLESHOOTING.md for common setup and runtime issues Covers LanceDB native binary failures, CLAUDE_PROJECT_DIR config, vector search not returning results, MCP handshake debugging, and performance tips. Links from README nav bar. --- README.md | 2 +- TROUBLESHOOTING.md | 131 +++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 132 insertions(+), 1 deletion(-) create mode 100644 TROUBLESHOOTING.md diff --git a/README.md b/README.md index f60fefa..c1a8584 100644 --- a/README.md +++ b/README.md @@ -12,7 +12,7 @@ A 24-tool MCP server for Claude Code that catches ambiguous instructions before [![npm](https://img.shields.io/npm/v/preflight-dev)](https://www.npmjs.com/package/preflight-dev) [![Node 18+](https://img.shields.io/badge/node-18%2B-brightgreen?logo=node.js&logoColor=white)](https://nodejs.org/) -[Quick Start](#quick-start) · [How It Works](#how-it-works) · [Tool Reference](#tool-reference) · [Configuration](#configuration) · [Scoring](#the-12-category-scorecard) +[Quick Start](#quick-start) · [How It Works](#how-it-works) · [Tool Reference](#tool-reference) · [Configuration](#configuration) · [Scoring](#the-12-category-scorecard) · [Troubleshooting](TROUBLESHOOTING.md) diff --git a/TROUBLESHOOTING.md b/TROUBLESHOOTING.md new file mode 100644 index 0000000..8e8ba66 --- /dev/null +++ b/TROUBLESHOOTING.md @@ -0,0 +1,131 @@ +# Troubleshooting + +Common issues and fixes for preflight. + +--- + +## Installation + +### `npm install` fails with LanceDB native binary errors + +LanceDB uses native binaries. If you see errors like `prebuild-install WARN` or `node-gyp` failures: + +``` +npm ERR! @lancedb/lancedb@0.26.2: The platform "linux" is incompatible +``` + +**Fix:** Make sure you're on a supported platform (macOS arm64/x64, Linux x64, Windows x64) and Node >= 20: + +```bash +node -v # must be >= 20 +npm cache clean --force +rm -rf node_modules package-lock.json +npm install +``` + +If you're on an unsupported platform (e.g., Linux arm64), LanceDB won't work. The timeline/vector search tools will be unavailable, but the core preflight tools still function. + +### `npx tsx` not found + +```bash +npm install -g tsx +# or use npx (comes with npm 7+): +npx tsx src/index.ts +``` + +--- + +## Configuration + +### Tools load but `CLAUDE_PROJECT_DIR` warnings appear + +The timeline and contract search tools need `CLAUDE_PROJECT_DIR` to know which project to index. Without it, those tools will error on use. + +**Fix — Claude Code CLI:** +```bash +claude mcp add preflight \ + -e CLAUDE_PROJECT_DIR=/absolute/path/to/your/project \ + -- npx tsx /path/to/preflight/src/index.ts +``` + +**Fix — `.mcp.json`:** +```json +{ + "mcpServers": { + "preflight": { + "command": "npx", + "args": ["tsx", "/path/to/preflight/src/index.ts"], + "env": { + "CLAUDE_PROJECT_DIR": "/absolute/path/to/your/project" + } + } + } +} +``` + +Use absolute paths — relative paths resolve from the MCP server's cwd, which may not be your project. + +### `.preflight/` config not being picked up + +Preflight looks for `.preflight/config.yml` in `CLAUDE_PROJECT_DIR`. If it's not found, defaults are used silently. + +**Check:** +1. File is named exactly `config.yml` (not `config.yaml`) +2. It's inside `.preflight/` at your project root +3. `CLAUDE_PROJECT_DIR` points to the right directory + +--- + +## Runtime + +### "Server started" but no tools appear in Claude Code + +The MCP handshake might be failing silently. + +**Debug steps:** +1. Test the server standalone: `npx tsx src/index.ts` — should print `preflight: server started` +2. Check Claude Code's MCP logs: `claude mcp list` to see registered servers +3. Remove and re-add: `claude mcp remove preflight && claude mcp add preflight -- npx tsx /path/to/preflight/src/index.ts` + +### Vector search returns no results + +Timeline search uses LanceDB to index your Claude Code session history (JSONL files). + +**Common causes:** +- No session history exists yet — use Claude Code for a few sessions first +- `CLAUDE_PROJECT_DIR` not set (search doesn't know where to look) +- Session files are in a non-standard location + +**Where Claude Code stores sessions:** `~/.claude/projects/` with JSONL files per session. + +### `preflight_check` returns generic advice + +The triage system works best with specific prompts. If you're testing with something like "do stuff", the response will be generic by design — that's it telling you the prompt is too vague. + +Try a real prompt: `"add rate limiting to the /api/users endpoint"` — you'll see it route through scope analysis, contract search, and produce actionable guidance. + +--- + +## Performance + +### Slow startup (>5 seconds) + +LanceDB initialization can be slow on first run as it builds the vector index. + +**Fix:** Subsequent runs are faster. If consistently slow, check that `node_modules/@lancedb` isn't corrupted: +```bash +rm -rf node_modules/@lancedb +npm install +``` + +### High memory usage + +Each indexed project maintains an in-memory vector index. If you're indexing many large projects, memory can grow. + +**Fix:** Only set `CLAUDE_PROJECT_DIR` to the project you're actively working on. + +--- + +## Still stuck? + +Open an issue: https://github.com/TerminalGravity/preflight/issues From 02dd6463f874a35036ec518de63aa302d7962728 Mon Sep 17 00:00:00 2001 From: Jack Felke Date: Thu, 5 Mar 2026 09:19:40 -0700 Subject: [PATCH 2/4] add .preflight/ example configs with annotated YAML and setup guide - examples/.preflight/config.yml: profile, related projects, thresholds, embeddings - examples/.preflight/triage.yml: strictness, always_check/skip/cross-service keywords - examples/.preflight/README.md: setup instructions and env var fallback reference - README.md: link to examples from Configuration Reference section --- README.md | 6 +++++ examples/.preflight/README.md | 43 +++++++++++++++++++++++++++++ examples/.preflight/config.yml | 41 ++++++++++++++++++++++++++++ examples/.preflight/triage.yml | 49 ++++++++++++++++++++++++++++++++++ 4 files changed, 139 insertions(+) create mode 100644 examples/.preflight/README.md create mode 100644 examples/.preflight/config.yml create mode 100644 examples/.preflight/triage.yml diff --git a/README.md b/README.md index c1a8584..fde571a 100644 --- a/README.md +++ b/README.md @@ -406,6 +406,12 @@ This prevents the common failure mode: changing a shared type in one service and ## Configuration Reference +> **Quick start:** Copy the example config into your project: +> ```bash +> cp -r examples/.preflight /path/to/your/project/ +> ``` +> See [`examples/.preflight/`](examples/.preflight/) for fully annotated config files you can customize. + ### `.preflight/config.yml` Drop this in your project root. Every field is optional — defaults are sensible. diff --git a/examples/.preflight/README.md b/examples/.preflight/README.md new file mode 100644 index 0000000..8a1ce51 --- /dev/null +++ b/examples/.preflight/README.md @@ -0,0 +1,43 @@ +# `.preflight/` Configuration + +This directory customizes how preflight behaves in your project. + +## Setup + +Copy this entire directory to your project root: + +```bash +cp -r examples/.preflight /path/to/your/project/ +``` + +Make sure `CLAUDE_PROJECT_DIR` points to your project (so preflight knows where to look). + +## Files + +| File | Purpose | +|------|---------| +| `config.yml` | Profile, related projects, thresholds, embedding provider | +| `triage.yml` | Triage strictness, always-check/skip keywords, cross-service triggers | + +Both files are optional. Omit either one (or any field within) and defaults apply. + +## Team Sharing + +Commit `.preflight/` to your repo so the whole team gets the same triage rules. This is especially useful for: + +- Enforcing checks on risky areas (`always_check: [migration, billing]`) +- Skipping noisy checks on safe commands (`skip: [commit, lint]`) +- Cross-service awareness when your project depends on sibling repos + +## Environment Variable Fallback + +If `.preflight/` doesn't exist, preflight falls back to environment variables: + +| Env Var | Maps to | +|---------|---------| +| `PROMPT_DISCIPLINE_PROFILE` | `profile` | +| `PREFLIGHT_RELATED` | `related_projects` (comma-separated paths) | +| `EMBEDDING_PROVIDER` | `embeddings.provider` | +| `OPENAI_API_KEY` | `embeddings.openai_api_key` | + +When `.preflight/` exists, env vars are ignored (config files take precedence). diff --git a/examples/.preflight/config.yml b/examples/.preflight/config.yml new file mode 100644 index 0000000..7d5b351 --- /dev/null +++ b/examples/.preflight/config.yml @@ -0,0 +1,41 @@ +# .preflight/config.yml +# ────────────────────────────────────────────────────────────────────── +# Drop this directory into your project root. Preflight detects it +# automatically when CLAUDE_PROJECT_DIR points to your project. +# +# All fields are optional — anything you omit uses the defaults shown. +# ────────────────────────────────────────────────────────────────────── + +# Profile controls how many tools activate per check. +# minimal — triage + clarify only (fastest, lowest token cost) +# standard — adds contract search, git state, workspace priorities +# full — everything including timeline vector search & trend reports +profile: standard + +# Related projects for cross-service contract awareness. +# Preflight scans these for exported types, routes, and schemas so it +# can warn you when a change in one project affects another. +related_projects: + - path: ../backend-api + alias: api + - path: ../shared-types + alias: types + +# Tuning knobs — adjust these based on your workflow. +thresholds: + # Minutes before a session is considered stale (affects context freshness) + session_stale_minutes: 30 + + # Tool calls before preflight suggests a checkpoint/commit + max_tool_calls_before_checkpoint: 100 + + # How many keyword matches from correction history trigger a warning + # Lower = more sensitive (catches more repeats, but noisier) + correction_pattern_threshold: 3 + +# Embedding provider for timeline vector search. +# local — built-in TF-IDF (no API key needed, works offline) +# openai — OpenAI text-embedding-3-small (better quality, needs key) +embeddings: + provider: local + # openai_api_key: sk-... # or set OPENAI_API_KEY env var diff --git a/examples/.preflight/triage.yml b/examples/.preflight/triage.yml new file mode 100644 index 0000000..d64712f --- /dev/null +++ b/examples/.preflight/triage.yml @@ -0,0 +1,49 @@ +# .preflight/triage.yml +# ────────────────────────────────────────────────────────────────────── +# Customize the triage decision tree — which prompts get checked, +# which pass through, and how strict the classifier is. +# ────────────────────────────────────────────────────────────────────── + +# Strictness controls the triage sensitivity. +# relaxed — only flags very short/vague prompts +# standard — balanced (recommended for most teams) +# strict — flags anything without explicit file references +strictness: standard + +rules: + # Prompts containing these keywords ALWAYS trigger a full check, + # even if they'd otherwise be classified as clear/trivial. + # Great for high-risk areas of your codebase. + always_check: + - migration + - schema + - permissions + - billing + - deploy + # Add your own: + # - payments + # - auth + # - delete + + # Prompts matching these pass through with zero overhead. + # Use for commands you run constantly and never need checked. + skip: + - commit + - format + - lint + - "git status" + - "run tests" + + # Keywords that trigger cross-service contract scanning. + # When a prompt mentions these, preflight also checks related_projects + # for type/route/schema contracts that might be affected. + cross_service_keywords: + - auth + - notification + - event + - webhook + - api + # Add your own: + # - queue + # - cache + # - pubsub From 814b24749d816ea1deab6f575f8e71d85bec0bbc Mon Sep 17 00:00:00 2001 From: Jack Felke Date: Thu, 5 Mar 2026 10:22:45 -0700 Subject: [PATCH 3/4] docs: add concrete usage examples for key tools in README --- README.md | 144 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 144 insertions(+) diff --git a/README.md b/README.md index fde571a..c3e0272 100644 --- a/README.md +++ b/README.md @@ -335,6 +335,150 @@ After onboarding, you get: --- +## Usage Examples + +Real input → output examples showing what the tools actually produce. + +### `preflight_check` — Catching a vague prompt + +**Input:** +``` +preflight_check({ prompt: "fix the auth bug" }) +``` + +**Output:** +```markdown +# 🛫 Preflight Check +_2025-03-05T10:15 | Triage: **ambiguous** (confidence: 0.85)_ +_Reasons: vague verb without file target; short prompt_ + +## Clarification + +### Git State +Branch: `feat/auth-refactor` | Dirty files: 3 + +Recent commits: + a1b2c3d fix: JWT refresh token expiry check + d4e5f6g feat: add OAuth2 callback handler + h7i8j9k fix: session cookie SameSite attribute + +### ⚠️ Clarification Needed +- Contains vague pronouns — clarify what 'it' refers to +- Vague verb without specific file targets +- Very short prompt — likely missing context +``` + +The tool identified 3 recent auth-related commits and asked you to specify which bug — saving a wrong-direction cycle. + +### `preflight_check` — Multi-step task + +**Input:** +``` +preflight_check({ + prompt: "add rate limiting to /api/users, then update the OpenAPI spec, then add integration tests" +}) +``` + +**Output:** +```markdown +# 🛫 Preflight Check +_Triage: **multi-step** (confidence: 0.92)_ + +## Scope: MEDIUM + +## Sequence +### Execution Plan +1. Add rate limiting to /api/users — Risk: 🟡 MEDIUM +2. Update the OpenAPI spec — Risk: 🟢 LOW +3. Add integration tests — Risk: 🟢 LOW + +### Checkpoints +- [ ] Verify after each step before proceeding +- [ ] Run tests between steps that touch different layers +- [ ] Commit after each successful step +``` + +### `prompt_score` — Grading a prompt + +**Input:** +``` +prompt_score({ prompt: "refactor the database layer to use connection pooling in src/db/pool.ts, max 10 connections, with health checks every 30s" }) +``` + +**Output:** +``` +Grade: A (92/100) + Specificity: A — names exact file and parameters + Scope: A — single focused change + Actionability: A — clear implementation details + Done-condition: B+ — could add "verify with load test" +``` + +### `estimate_cost` — Understanding token waste + +**Input:** +``` +estimate_cost({ session_path: "~/.claude/projects/.../session.jsonl" }) +``` + +**Output (abbreviated):** +``` +Session Cost Estimate +───────────────────── +Total tokens: 45,200 +Estimated cost: $0.68 +Corrections: 3 (waste: ~8,400 tokens / $0.13) +Preflight saves: ~$0.25 if checks had caught the vague prompts + +Waste breakdown: + "fix it" → wrong file edited → corrected 2,800 tokens + "update the tests" → wrong test suite 3,100 tokens + "do the same for the other one" 2,500 tokens +``` + +### `search_history` — Finding past decisions + +**Input:** +``` +search_history({ query: "why did we switch from Redis to Postgres for sessions?" }) +``` + +**Output:** +``` +Found 3 relevant events across 2 sessions: + +1. [2025-02-14 session-abc] "switching session store from Redis to Postgres + because we're already paying for Supabase and don't want another + managed service just for sessions" + +2. [2025-02-14 session-abc] Commit: "feat: migrate session store to Postgres + with pgcrypto for token generation" + +3. [2025-02-15 session-def] "confirmed Postgres sessions working in prod, + p95 latency 12ms vs 8ms with Redis — acceptable tradeoff" +``` + +### `verify_completion` — Pre-merge sanity check + +**Input:** +``` +verify_completion({ task: "add rate limiting middleware" }) +``` + +**Output:** +``` +## Verification Results + +✅ TypeScript: No type errors +✅ Tests: 47 passed, 0 failed (2 new) +✅ Build: Clean build in 3.2s +⚠️ Lint: 1 warning — unused import in src/middleware/rate-limit.ts:3 + +Recommendation: Fix the lint warning, then clear to merge. +``` + +--- + ## The 12-Category Scorecard `generate_scorecard` evaluates your prompt discipline across 12 categories. Each one measures something specific about how you interact with Claude Code: From 03682d071229a48573c8f5740d317c8e3f34051b Mon Sep 17 00:00:00 2001 From: Jack Felke Date: Fri, 6 Mar 2026 06:49:22 -0700 Subject: [PATCH 4/4] fix: CLI entry point now starts MCP server by default, init is a subcommand MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Previously, `preflight-dev` (the bin entry) ran the init wizard, making `claude mcp add preflight -- preflight-dev` broken — it would launch the interactive setup instead of the MCP server. Now: - `preflight-dev` → starts MCP server (correct for claude mcp add) - `preflight-dev init` → runs interactive setup wizard - `preflight-dev help` → shows usage info Also fixes init generating a broken .mcp.json (was overwriting the config with a wrong path to node_modules/preflight/src/index.ts). --- README.md | 6 ++++++ bin/cli.js | 35 +++++++++++++++++++++++++++++++---- src/cli/init.ts | 9 --------- 3 files changed, 37 insertions(+), 13 deletions(-) diff --git a/README.md b/README.md index c3e0272..c3be29d 100644 --- a/README.md +++ b/README.md @@ -122,6 +122,12 @@ npm install -g preflight-dev claude mcp add preflight -- preflight-dev ``` +To run the interactive setup wizard instead (creates `.mcp.json` and `.preflight/` config): + +```bash +preflight-dev init +``` + --- ## How It Works diff --git a/bin/cli.js b/bin/cli.js index 69b21ad..60573a8 100755 --- a/bin/cli.js +++ b/bin/cli.js @@ -1,11 +1,38 @@ #!/usr/bin/env node -// This is a shim that loads the compiled TypeScript CLI +// Preflight CLI entry point +// - `preflight-dev` (no args) → starts the MCP server +// - `preflight-dev init` → runs interactive setup wizard import { fileURLToPath } from 'url'; import { dirname, join } from 'path'; const __filename = fileURLToPath(import.meta.url); const __dirname = dirname(__filename); -// Load the compiled CLI -const cliPath = join(__dirname, '../dist/cli/init.js'); -await import(cliPath); \ No newline at end of file +const command = process.argv[2]; + +if (command === 'init') { + const initPath = join(__dirname, '../dist/cli/init.js'); + await import(initPath); +} else if (command === 'help' || command === '--help' || command === '-h') { + console.log(` +✈️ preflight-dev — MCP server for Claude Code prompt discipline + +Usage: + preflight-dev Start the MCP server (default) + preflight-dev init Interactive setup wizard — creates .mcp.json and .preflight/ + preflight-dev help Show this help message + +Environment variables: + CLAUDE_PROJECT_DIR Project root for timeline/contract tools (required for full profile) + PROMPT_DISCIPLINE_PROFILE Tool profile: minimal | standard | full (default: standard) + EMBEDDING_PROVIDER Embedding backend: local | openai (default: local) + OPENAI_API_KEY Required if EMBEDDING_PROVIDER=openai + +Quick start: + claude mcp add preflight -- npx -y preflight-dev@latest +`); +} else { + // Default: start MCP server + const serverPath = join(__dirname, '../dist/index.js'); + await import(serverPath); +} diff --git a/src/cli/init.ts b/src/cli/init.ts index 996906d..2d2cff5 100644 --- a/src/cli/init.ts +++ b/src/cli/init.ts @@ -110,15 +110,6 @@ async function main(): Promise { env, }; - // For the actual server entry point, we need to point to index.ts via tsx - // But npx will resolve the bin entry which is the init script - // So use a different approach: command runs the server - config.mcpServers["preflight"] = { - command: "npx", - args: ["-y", "tsx", "node_modules/preflight/src/index.ts"], - env, - }; - await writeFile(mcpPath, JSON.stringify(config, null, 2) + "\n"); console.log(`\n✅ preflight added! (profile: ${profile})`);