From 21337e2bfcd1e050102181b5c05c7539fe8746d1 Mon Sep 17 00:00:00 2001 From: Jack Felke Date: Fri, 13 Mar 2026 08:03:06 -0700 Subject: [PATCH 1/2] docs: add workflow examples showing preflight in real scenarios MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Added examples/WORKFLOWS.md with 6 concrete before/after workflows: - Vague bug fix → clarified scope - Multi-step refactor → structured plan - Repeated mistake → pattern warning - Session drift → context recovery - Quick commands → zero overhead - New codebase onboarding Linked from main README nav and examples README. --- README.md | 2 +- examples/README.md | 6 ++ examples/WORKFLOWS.md | 142 ++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 149 insertions(+), 1 deletion(-) create mode 100644 examples/WORKFLOWS.md diff --git a/README.md b/README.md index 6d03f5d..a1a4310 100644 --- a/README.md +++ b/README.md @@ -12,7 +12,7 @@ A 24-tool MCP server for Claude Code that catches ambiguous instructions before [![npm](https://img.shields.io/npm/v/preflight-dev)](https://www.npmjs.com/package/preflight-dev) [![Node 18+](https://img.shields.io/badge/node-18%2B-brightgreen?logo=node.js&logoColor=white)](https://nodejs.org/) -[Quick Start](#quick-start) · [How It Works](#how-it-works) · [Tool Reference](#tool-reference) · [Configuration](#configuration) · [Scoring](#the-12-category-scorecard) +[Quick Start](#quick-start) · [How It Works](#how-it-works) · [Workflow Examples](examples/WORKFLOWS.md) · [Tool Reference](#tool-reference) · [Configuration](#configuration) · [Scoring](#the-12-category-scorecard) diff --git a/examples/README.md b/examples/README.md index 778f15d..c861466 100644 --- a/examples/README.md +++ b/examples/README.md @@ -1,5 +1,11 @@ # Examples +## [Workflow Examples](WORKFLOWS.md) + +Concrete before/after examples showing preflight in action — vague bug fixes, multi-step refactors, pattern warnings, session recovery, and more. Start here if you want to see what preflight actually does. + +--- + ## `.preflight/` Config Directory The `.preflight/` directory contains example configuration files you can copy into your project root: diff --git a/examples/WORKFLOWS.md b/examples/WORKFLOWS.md new file mode 100644 index 0000000..96bab0c --- /dev/null +++ b/examples/WORKFLOWS.md @@ -0,0 +1,142 @@ +# Workflow Examples + +Real-world examples of preflight catching problems before they cost you tokens. + +--- + +## 1. Vague Bug Fix → Clarified Scope + +**Without preflight:** +``` +You: fix the login bug +Claude: *edits src/auth/login.ts, breaks OAuth flow* +You: no, the other one — the session timeout +Claude: *undoes changes, starts over* +→ 3 rounds, ~15,000 tokens wasted +``` + +**With preflight:** +``` +You: fix the login bug + +preflight_check fires → classifies as AMBIGUOUS +→ "I found 4 login-related issues in recent history: + 1. Session timeout not refreshing (src/auth/session.ts:42) + 2. OAuth callback returning 404 (src/auth/oauth.ts:118) + 3. Password reset email not sending (src/auth/reset.ts:67) + 4. CSRF token mismatch on mobile (src/middleware/csrf.ts:23) + Which one? Or describe the symptoms." + +You: the session timeout one +→ Scoped fix, ~4,000 tokens, done right +``` + +--- + +## 2. Multi-Step Refactor → Structured Plan + +**Without preflight:** +``` +You: refactor the API to use the new auth +Claude: *starts changing files, misses 3 consumers, breaks billing integration* +``` + +**With preflight:** +``` +You: refactor the API to use the new auth + +preflight_check fires → classifies as MULTI-STEP + CROSS-SERVICE +→ Pulls contracts from auth-service and billing-api +→ "This touches 3 services. Here's a suggested plan: + 1. Update shared AuthToken type in shared-types/ + 2. Migrate auth-service endpoints (3 routes) + 3. Update billing-api consumer (imports AuthToken) + 4. Update main API auth middleware + 5. Run cross-service type check + Want me to proceed step by step, or adjust the plan?" +``` + +--- + +## 3. Repeated Mistake → Pattern Warning + +After you've corrected Claude a few times about the same thing: + +``` +You: update the user schema + +preflight_check fires → correction pattern detected +→ "⚠️ Heads up — in past sessions you've corrected me 3 times + about the user schema: + - Don't modify the `tier` field without updating billing-api + - Always run `prisma migrate dev` after schema changes + - The `createdAt` default is set at the DB level, not in Prisma + Proceeding with these constraints." +``` + +--- + +## 4. Session Drift → Context Recovery + +You've been working for 2 hours and the context window is getting full: + +``` +You: where was I on the notification system? + +search_sessions fires → searches LanceDB vector index +→ "Found 3 relevant sessions from this week: + - Tuesday 2pm: Designed webhook payload schema (WebhookPayload type) + - Tuesday 4pm: Implemented notification dispatch in auth-service + - Wednesday 10am: Started consumer in billing-api, stopped at error handling + Last commit: 'feat: add notification dispatch endpoint' + You were about to add retry logic to the billing consumer." +``` + +--- + +## 5. Quick Commands → Zero Overhead + +Preflight doesn't slow you down on simple tasks: + +``` +You: commit this +→ preflight_check classifies as TRIVIAL → passes through instantly + +You: format src/ +→ TRIVIAL → passes through + +You: run the tests +→ TRIVIAL → passes through +``` + +The triage engine only intervenes when it can save you tokens. + +--- + +## 6. Onboarding a New Codebase + +First time running preflight on a project: + +``` +You: onboard this project + +onboard_project fires → +→ Scans project structure, package.json, tsconfig +→ Indexes session history into LanceDB +→ Extracts contracts from source files +→ "Onboarding complete: + - 847 sessions indexed (last 6 months) + - 23 contracts extracted (12 types, 8 interfaces, 3 routes) + - 4 related projects detected from imports + - Suggested .preflight/config.yml written (review it)" +``` + +--- + +## Tips + +- **Start with `npx`** — don't clone unless you want to contribute +- **Let triage do its thing** — you don't need to call specific tools manually +- **Commit `.preflight/`** — your team gets consistent behavior +- **Check your scorecard weekly** — `generate_scorecard` shows where you're improving +- **Use `search_sessions` when you lose context** — it's faster than scrolling From 1f644ca2ae177ac8f78bf9834faaa15997e10942 Mon Sep 17 00:00:00 2001 From: Jack Felke Date: Fri, 13 Mar 2026 08:16:15 -0700 Subject: [PATCH 2/2] feat: add .preflight/ example config with annotated YAML files MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds examples/.preflight/ directory with: - config.yml: profile, related projects, thresholds, embeddings - triage.yml: strictness, skip/always-check/cross-service keywords - README.md: quick setup recipes for minimal, strict, and monorepo configs The README already references this directory — now the files actually exist. --- examples/.preflight/README.md | 51 +++++++++++++++++++++++++++ examples/.preflight/config.yml | 57 +++++++++++++++++------------- examples/.preflight/triage.yml | 63 +++++++++++++++++++--------------- 3 files changed, 120 insertions(+), 51 deletions(-) create mode 100644 examples/.preflight/README.md diff --git a/examples/.preflight/README.md b/examples/.preflight/README.md new file mode 100644 index 0000000..5b8495c --- /dev/null +++ b/examples/.preflight/README.md @@ -0,0 +1,51 @@ +# `.preflight/` Configuration Example + +Copy this directory into your project root to customize preflight behavior. + +```bash +cp -r examples/.preflight /path/to/your/project/ +``` + +## Files + +| File | Purpose | +|------|---------| +| `config.yml` | Profile, related projects, thresholds, embedding settings | +| `triage.yml` | Triage strictness, skip/always-check/cross-service keyword lists | + +## Quick Setup + +**Minimal (just want it quieter):** +```yaml +# config.yml +profile: minimal +``` + +**Strict (payments/healthcare — force clarity on everything):** +```yaml +# config.yml +profile: full + +# triage.yml +strictness: strict +rules: + always_check: [payments, patient, hipaa, pii, migration, deploy] +``` + +**Monorepo with related services:** +```yaml +# config.yml +related_projects: + - path: ../api-gateway + alias: gateway + - path: ../shared-types + alias: types + - path: ../worker-service + alias: worker +``` + +## Notes + +- When `.preflight/` exists, environment variables (`PROMPT_DISCIPLINE_PROFILE`, etc.) are ignored +- All fields are optional — omitted values use sensible defaults +- Commit this directory to share settings across your team diff --git a/examples/.preflight/config.yml b/examples/.preflight/config.yml index f59170f..3ceaf2f 100644 --- a/examples/.preflight/config.yml +++ b/examples/.preflight/config.yml @@ -1,35 +1,44 @@ -# .preflight/config.yml — Drop this in your project root -# -# This is an example config for a typical Next.js + microservices setup. -# Every field is optional — preflight works with sensible defaults out of the box. -# Commit this to your repo so the whole team gets the same preflight behavior. +# ============================================================================= +# .preflight/config.yml — Team-shareable preflight configuration +# ============================================================================= +# Drop this in your project root as .preflight/config.yml +# When present, env vars (PROMPT_DISCIPLINE_PROFILE, etc.) are ignored. +# All fields are optional — defaults are shown below. +# ============================================================================= -# Profile controls how much detail preflight returns. -# "minimal" — only flags ambiguous+ prompts, skips clarification detail -# "standard" — balanced (default) -# "full" — maximum detail on every non-trivial prompt +# Profile controls how aggressively preflight checks your prompts. +# minimal — only catch the most ambiguous prompts, low friction +# standard — balanced (default) — catches vague prompts + cross-service issues +# full — maximum checking, scorecards, cost estimates on every prompt profile: standard -# Related projects for cross-service awareness. -# Preflight will search these for shared types, routes, and contracts -# so it can warn you when a change might break a consumer. +# Related projects for cross-service contract awareness. +# Preflight extracts types, interfaces, routes, and schemas from these +# so it can warn you when a change in your project affects another. related_projects: - - path: /Users/you/code/auth-service + - path: ../auth-service alias: auth - - path: /Users/you/code/billing-api + - path: ../billing-api alias: billing - - path: /Users/you/code/shared-types - alias: types + # - path: ../shared-types + # alias: types -# Behavioral thresholds — tune these to your workflow +# Tuning knobs for preflight behavior. thresholds: - session_stale_minutes: 30 # Warn if no activity for this long - max_tool_calls_before_checkpoint: 100 # Suggest a checkpoint after N tool calls - correction_pattern_threshold: 3 # Min corrections before flagging a pattern + # Minutes before a session is considered "stale" and context recovery kicks in. + session_stale_minutes: 30 -# Embedding provider for semantic search over session history. -# "local" uses Xenova transformers (no API key needed, runs on CPU). -# "openai" uses text-embedding-3-small (faster, needs OPENAI_API_KEY). + # After this many tool calls without a checkpoint, preflight nudges you + # to pause and verify you're still on track. + max_tool_calls_before_checkpoint: 100 + + # How many keyword matches from past corrections before preflight warns you. + # Lower = more warnings. Set to 1 if you keep making the same mistakes. + correction_pattern_threshold: 3 + +# Embedding configuration for session search (search_sessions tool). embeddings: + # "local" — uses built-in embeddings, no API key needed (default) + # "openai" — uses OpenAI text-embedding-3-small for higher quality search provider: local - # openai_api_key: sk-... # Uncomment if using openai provider + # openai_api_key: sk-... # Or set OPENAI_API_KEY env var before switching to openai diff --git a/examples/.preflight/triage.yml b/examples/.preflight/triage.yml index b3d394e..7f1abaf 100644 --- a/examples/.preflight/triage.yml +++ b/examples/.preflight/triage.yml @@ -1,45 +1,54 @@ -# .preflight/triage.yml — Controls how preflight classifies your prompts -# -# The triage engine routes prompts into categories: -# TRIVIAL → pass through (commit, format, lint) -# CLEAR → well-specified, no intervention needed -# AMBIGUOUS → needs clarification before proceeding -# MULTI-STEP → complex task, preflight suggests a plan -# CROSS-SERVICE → touches multiple projects, pulls in contracts -# -# Customize the keywords below to match your domain. +# ============================================================================= +# .preflight/triage.yml — Customize how prompts get classified +# ============================================================================= +# Controls which prompts trigger checks and which pass through. +# Tune this per-project: a payments service needs stricter rules than a blog. +# ============================================================================= + +# Strictness level for the triage engine. +# relaxed — only flag very short/vague prompts, trust the developer more +# standard — balanced heuristics (default) +# strict — flag anything without explicit file refs or clear scope +strictness: standard rules: - # Prompts containing these words are always flagged as AMBIGUOUS. - # Add domain-specific terms that tend to produce vague prompts. + # Keywords that ALWAYS trigger ambiguity checks, even if the prompt looks clear. + # Use this for dangerous areas of your codebase where you want forced clarity. always_check: - rewards - permissions - migration - schema - - pricing # example: your billing domain - - onboarding # example: multi-step user flows + - billing + - delete + # Add your project's sensitive areas: + # - payments + # - user-data + # - deploy - # Prompts containing these words skip checks entirely (TRIVIAL). - # These are safe, mechanical tasks that don't need guardrails. + # Keywords that skip all checks — these are quick, safe commands. + # Preflight passes them through with zero overhead. skip: - commit - format - lint - - prettier - - "git push" + - status + - log + - help + # Add your own safe commands: + # - test + # - build - # Prompts containing these words trigger CROSS-SERVICE classification. - # Preflight will search related_projects for relevant types and routes. + # Keywords that trigger cross-service contract search. + # When detected, preflight pulls types/interfaces from related_projects + # to show you what might break across service boundaries. cross_service_keywords: - auth - notification - event - webhook - - billing # matches the related_project alias - -# How aggressively to classify prompts. -# "relaxed" — more prompts pass as clear (experienced users) -# "standard" — balanced (default) -# "strict" — more prompts flagged as ambiguous (new teams, complex codebases) -strictness: standard + - shared + # Add your inter-service touchpoints: + # - queue + # - pubsub + # - grpc