Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
180 changes: 176 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,18 +76,18 @@ The pattern is always the same: vague prompt → Claude guesses → wrong output

## Quick Start

### Option A: Claude Code CLI (fastest)
### Option A: npx (fastest — no install)

```bash
claude mcp add preflight -- npx tsx /path/to/preflight/src/index.ts
claude mcp add preflight -- npx -y preflight-dev-serve
```

With environment variables:

```bash
claude mcp add preflight \
-e CLAUDE_PROJECT_DIR=/path/to/your/project \
-- npx tsx /path/to/preflight/src/index.ts
-- npx -y preflight-dev-serve
```

### Option B: Clone & configure manually
Expand Down Expand Up @@ -119,9 +119,11 @@ Restart Claude Code. The tools activate automatically.

```bash
npm install -g preflight-dev
claude mcp add preflight -- preflight-dev
claude mcp add preflight -- preflight-dev-serve
```

> **Note:** `preflight-dev` runs the interactive setup wizard. `preflight-dev-serve` starts the MCP server — that's what you want in your Claude Code config.

---

## How It Works
Expand Down Expand Up @@ -335,6 +337,85 @@ After onboarding, you get:

---

## Usage Examples

Real prompts you'd type in Claude Code once preflight is installed. The tools fire automatically via `preflight_check`, or you can call them directly.

### Catch a vague prompt before it wastes tokens

```
> fix the auth bug

preflight_check intercepts → clarify_intent fires:
"I found 3 auth-related recent changes:
1. JWT refresh logic in src/auth/jwt.ts (modified 2h ago)
2. OAuth callback route in src/routes/oauth.ts (3 open TODOs)
3. Session middleware in src/middleware/auth.ts (failing test)
Which one? Or describe the symptoms."
```

### Scope a multi-step task before starting

```
> refactor the database layer to use Drizzle instead of Prisma

preflight_check triages as MULTI-STEP → scope_work + sequence_tasks fire:
Phase 1: Install drizzle-orm, create schema (src/db/schema.ts)
Phase 2: Migrate queries in src/services/ (12 files)
Phase 3: Update tests (8 files reference prisma mocks)
Phase 4: Remove prisma dependencies, delete prisma/ dir
Estimated: 4 checkpoints, ~45 min
```

### Search your session history

```
> use search_history to find how I set up the rate limiter

Searches LanceDB across your indexed sessions:
Found 3 relevant events (similarity > 0.82):
- Session abc123 (Jan 15): Added express-rate-limit to src/middleware/
- Session def456 (Jan 16): Configured per-route limits in src/routes/api.ts
- Session ghi789 (Feb 2): Fixed rate limit bypass via X-Forwarded-For
```

### Grade your prompting habits

```
> use prompt_score on "update the thing"

Grade: D
- Specificity: 1/5 — no file paths, no identifiers
- Scope: 1/5 — "the thing" is completely ambiguous
- Actionability: 2/5 — "update" is vague (refactor? fix? add feature?)
- Done-condition: 0/5 — no way to know when it's done
Suggestion: "Update the price formatter in src/utils/currency.ts
to handle JPY (zero-decimal currency)"
```

### Check session health mid-work

```
> use check_session_health

⚠️ 47 uncommitted files (last commit: 38 min ago)
⚠️ 23 turns since last checkpoint
💡 Consider running `checkpoint` before context compaction hits
```

### Get a cost estimate

```
> use estimate_cost

This session: ~$2.40 (148K tokens)
Waste from corrections: ~$0.85 (3 wrong→fix cycles)
Preflight savings: ~$1.20 (4 vague prompts caught early)
Net: saving roughly 35% vs unchecked prompting
```

---

## The 12-Category Scorecard

`generate_scorecard` evaluates your prompt discipline across 12 categories. Each one measures something specific about how you interact with Claude Code:
Expand Down Expand Up @@ -500,6 +581,8 @@ Manual contract definitions that supplement auto-extraction:

Environment variables are **fallbacks** — `.preflight/` config takes precedence when present.

> 💡 **Ready-to-use examples:** Copy [`examples/.preflight/`](examples/.preflight/) into your project root for a working starter config with detailed comments.

---

## Embedding Providers
Expand Down Expand Up @@ -562,6 +645,95 @@ flowchart TB

---

## Troubleshooting

### "Cannot find module 'vectordb'" or LanceDB import errors

LanceDB uses native binaries. If you see module resolution errors:

```bash
# Clean install with native deps rebuilt
rm -rf node_modules package-lock.json
npm install

# If still failing, check your Node version (20+ required)
node --version
```

On Apple Silicon Macs, make sure you're running a native arm64 Node — not Rosetta. Check with `node -e "console.log(process.arch)"` (should print `arm64`).

### First run is slow (~90MB model download)

The local embedding provider ([Xenova/all-MiniLM-L6-v2](https://huggingface.co/Xenova/all-MiniLM-L6-v2)) downloads a ~90MB model on first use. This is a one-time cost — subsequent runs use the cached model. If the download hangs behind a corporate proxy, switch to OpenAI embeddings:

```bash
export OPENAI_API_KEY=sk-...
export EMBEDDING_PROVIDER=openai
```

### "OpenAI API key required for openai embedding provider"

You set `EMBEDDING_PROVIDER=openai` (or `embeddings.provider: openai` in `.preflight/config.yml`) but didn't provide a key. Either:

- Set `OPENAI_API_KEY` in your environment, or
- Switch back to local: `export EMBEDDING_PROVIDER=local`

### Tools not showing up in Claude Code

1. Make sure the MCP server is registered. Run `claude mcp list` — you should see `preflight`.
2. If missing, re-add it:
```bash
claude mcp add preflight -- npx tsx /path/to/preflight/src/index.ts
```
3. Restart Claude Code after adding.

### `CLAUDE_PROJECT_DIR` not set

Some tools (onboarding, session search, contracts) need to know your project root. If they return empty results:

```bash
claude mcp add preflight \
-e CLAUDE_PROJECT_DIR=/path/to/your/project \
-- npx tsx /path/to/preflight/src/index.ts
```

Or set it globally: `export CLAUDE_PROJECT_DIR=/path/to/your/project`

### `.preflight/config.yml` parse errors

If you see `warning - failed to parse .preflight/config.yml`, your YAML is malformed. Common issues:

- Tabs instead of spaces (YAML requires spaces)
- Missing quotes around values with special characters
- Incorrect indentation under `related_projects`

Validate with: `npx yaml-lint .preflight/config.yml` or paste into [yamllint.com](https://www.yamllint.com/).

### No session data found during onboarding

`onboard_project` looks for JSONL files in `~/.claude/projects/<encoded-path>/`. If nothing is found:

- Make sure you've actually used Claude Code on the project (at least one session)
- Check that `CLAUDE_PROJECT_DIR` matches the exact path Claude Code was opened in
- The path encoding is URL-style — `/Users/jack/my-app` becomes `%2FUsers%2Fjack%2Fmy-app`

### Ollama embeddings connection refused

If using Ollama as your embedding provider and getting connection errors:

```bash
# Make sure Ollama is running
ollama serve

# Pull the embedding model
ollama pull all-minilm

# Verify it works
curl http://localhost:11434/api/embed -d '{"model":"all-minilm","input":"test"}'
```

---

## Contributing

This project is young and there's plenty to do. Check the [issues](https://github.com/TerminalGravity/preflight/issues) — several are tagged `good first issue`.
Expand Down
10 changes: 10 additions & 0 deletions bin/serve.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
#!/usr/bin/env node
// Starts the preflight MCP server (as opposed to the init wizard).
// Used by .mcp.json configs generated by `preflight-dev init`.
import { fileURLToPath } from 'url';
import { dirname, join } from 'path';

const __filename = fileURLToPath(import.meta.url);
const __dirname = dirname(__filename);

await import(join(__dirname, '../dist/index.js'));
35 changes: 35 additions & 0 deletions examples/.preflight/config.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# .preflight/config.yml — Drop this in your project root
#
# This is an example config for a typical Next.js + microservices setup.
# Every field is optional — preflight works with sensible defaults out of the box.
# Commit this to your repo so the whole team gets the same preflight behavior.

# Profile controls how much detail preflight returns.
# "minimal" — only flags ambiguous+ prompts, skips clarification detail
# "standard" — balanced (default)
# "full" — maximum detail on every non-trivial prompt
profile: standard

# Related projects for cross-service awareness.
# Preflight will search these for shared types, routes, and contracts
# so it can warn you when a change might break a consumer.
related_projects:
- path: /Users/you/code/auth-service
alias: auth
- path: /Users/you/code/billing-api
alias: billing
- path: /Users/you/code/shared-types
alias: types

# Behavioral thresholds — tune these to your workflow
thresholds:
session_stale_minutes: 30 # Warn if no activity for this long
max_tool_calls_before_checkpoint: 100 # Suggest a checkpoint after N tool calls
correction_pattern_threshold: 3 # Min corrections before flagging a pattern

# Embedding provider for semantic search over session history.
# "local" uses Xenova transformers (no API key needed, runs on CPU).
# "openai" uses text-embedding-3-small (faster, needs OPENAI_API_KEY).
embeddings:
provider: local
# openai_api_key: sk-... # Uncomment if using openai provider
58 changes: 58 additions & 0 deletions examples/.preflight/contracts/api.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
# .preflight/contracts/api.yml — Manual contract definitions
#
# Define shared types and interfaces that preflight should know about.
# These supplement auto-extracted contracts from your codebase.
# Manual definitions win on name conflicts with auto-extracted ones.
#
# Why manual contracts?
# - Document cross-service interfaces that live in docs, not code
# - Define contracts for external APIs your services consume
# - Pin down types that are implicit (e.g., event payloads)

- name: User
kind: interface
description: Core user model shared across all services
fields:
- name: id
type: string
required: true
- name: email
type: string
required: true
- name: tier
type: "'free' | 'pro' | 'enterprise'"
required: true
- name: createdAt
type: Date
required: true

- name: AuthToken
kind: interface
description: JWT payload structure from auth-service
fields:
- name: userId
type: string
required: true
- name: permissions
type: string[]
required: true
- name: expiresAt
type: number
required: true

- name: WebhookPayload
kind: interface
description: Standard webhook envelope for inter-service events
fields:
- name: event
type: string
required: true
- name: timestamp
type: string
required: true
- name: data
type: Record<string, unknown>
required: true
- name: source
type: string
required: true
45 changes: 45 additions & 0 deletions examples/.preflight/triage.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
# .preflight/triage.yml — Controls how preflight classifies your prompts
#
# The triage engine routes prompts into categories:
# TRIVIAL → pass through (commit, format, lint)
# CLEAR → well-specified, no intervention needed
# AMBIGUOUS → needs clarification before proceeding
# MULTI-STEP → complex task, preflight suggests a plan
# CROSS-SERVICE → touches multiple projects, pulls in contracts
#
# Customize the keywords below to match your domain.

rules:
# Prompts containing these words are always flagged as AMBIGUOUS.
# Add domain-specific terms that tend to produce vague prompts.
always_check:
- rewards
- permissions
- migration
- schema
- pricing # example: your billing domain
- onboarding # example: multi-step user flows

# Prompts containing these words skip checks entirely (TRIVIAL).
# These are safe, mechanical tasks that don't need guardrails.
skip:
- commit
- format
- lint
- prettier
- "git push"

# Prompts containing these words trigger CROSS-SERVICE classification.
# Preflight will search related_projects for relevant types and routes.
cross_service_keywords:
- auth
- notification
- event
- webhook
- billing # matches the related_project alias

# How aggressively to classify prompts.
# "relaxed" — more prompts pass as clear (experienced users)
# "standard" — balanced (default)
# "strict" — more prompts flagged as ambiguous (new teams, complex codebases)
strictness: standard
Loading
Loading