From b3e519680177172936909585f6d16fcc13e5330a Mon Sep 17 00:00:00 2001 From: Jack Felke Date: Mon, 16 Mar 2026 08:45:19 -0700 Subject: [PATCH 1/3] docs: add usage walkthrough with realistic tool output examples Shows 4 concrete scenarios: vague prompt clarification, multi-step scoping, correction pattern matching, and cross-service awareness. Each example includes the prompt, triage classification, and the actual output users can expect. --- README.md | 123 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 123 insertions(+) diff --git a/README.md b/README.md index 6d03f5d..7e8b3f5 100644 --- a/README.md +++ b/README.md @@ -74,6 +74,129 @@ The pattern is always the same: vague prompt → Claude guesses → wrong output --- +## Usage Walkthrough + +Here's what preflight actually looks like in practice — real tool calls with realistic output. + +### 1. Vague prompt → automatic clarification + +You type into Claude Code: + +``` +> update the dashboard +``` + +Claude calls `preflight_check` with `{ "prompt": "update the dashboard" }`. Triage classifies this as **ambiguous** (< 50 chars, no file refs). You get back: + +``` +⚠️ AMBIGUOUS — needs clarification before proceeding + +📂 Git State: + Branch: feat/analytics + 3 uncommitted files: src/components/Dashboard.tsx, src/hooks/useDashboard.ts, src/api/metrics.ts + +🔍 Ambiguity Signals: + - "update" is vague — add data? change layout? fix a bug? restyle? + - "dashboard" matches 4 files across the project + - No specific file path or line number referenced + +💡 Try something like: + - "add the new revenue chart to src/components/Dashboard.tsx using the metrics API from src/api/metrics.ts" + - "fix the loading spinner that won't dismiss in the analytics dashboard" + - "update dashboard layout to 3-column grid per the Figma mock" +``` + +### 2. Multi-step task → scoped execution plan + +``` +> refactor the payments module to support Stripe and add webhook handling +``` + +Triage: **multi-step** (multiple verbs + cross-concern). `scope_work` and `sequence_tasks` fire automatically: + +``` +🔶 MULTI-STEP — breaking this down before starting + +📋 Execution Plan: + Step 1: Extract payment interface (src/payments/provider.ts) + → Define PaymentProvider interface with charge(), refund(), webhook() + → Risk: LOW — new file, no existing consumers yet + + Step 2: Implement Stripe adapter (src/payments/stripe.ts) + → Implement PaymentProvider using stripe SDK + → Dependencies: stripe npm package + → Risk: MEDIUM — needs API keys, error handling for network failures + + Step 3: Add webhook endpoint (src/api/webhooks/stripe.ts) + → POST /api/webhooks/stripe with signature verification + → Risk: HIGH — security-sensitive, needs crypto verification + → ⚠️ Checkpoint recommended before this step + + Step 4: Update consumers (src/checkout/*, src/subscriptions/*) + → Swap direct calls to new PaymentProvider interface + → 6 files affected + +🧪 Suggested checkpoints: after Step 1, after Step 3 +⏱️ Estimated tokens: ~45,000 across all steps +``` + +### 3. Correction pattern matching + +You previously logged a correction with `log_correction`: + +``` +> log_correction: "When I said 'update types', I meant only the shared types in packages/shared, not the local type files in each service" +``` + +Next time you say: + +``` +> update the types +``` + +Preflight catches it: + +``` +⚠️ PATTERN MATCH — you've corrected this before + +🔄 Previous correction (2 days ago): + "When I said 'update types', I meant only the shared types in packages/shared, + not the local type files in each service" + +Did you mean shared types in packages/shared? If so, try: + "update shared types in packages/shared/types.ts" +``` + +### 4. Cross-service awareness + +``` +> add a loyalty points field to the user profile +``` + +Triage: **cross-service** (detected `user` + field change pattern, config has related projects). Contracts are searched: + +``` +🔗 CROSS-SERVICE — this change affects multiple projects + +📂 Current project: web-app + - src/types/user.ts → UserProfile interface (line 12) + - prisma/schema.prisma → User model (line 34) + +🔗 Related projects: + - mobile-app: src/api/types.ts → UserProfile (mirrors web-app, line 8) + - rewards-service: src/models/user.ts → UserRecord (includes loyalty_points already?) + - analytics-pipeline: src/schemas/user-events.avro → UserEvent schema + +⚠️ Changing UserProfile requires updates in 3 services. + Suggested order: + 1. prisma/schema.prisma (source of truth) + migrate + 2. web-app src/types/user.ts + 3. mobile-app src/api/types.ts (notify mobile team) + 4. analytics-pipeline schema (may need backfill) +``` + +--- + ## Quick Start ### Option A: npx (fastest — no install) From c9707b940beec124a27c60a568b6350629df168e Mon Sep 17 00:00:00 2001 From: Jack Felke Date: Mon, 16 Mar 2026 08:56:35 -0700 Subject: [PATCH 2/3] examples: add .preflight/ starter config files MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The README and examples/README.md referenced examples/.preflight/ but the actual config files didn't exist. Added: - config.yml — profile, related projects, thresholds, embeddings - triage.yml — keyword rules and strictness settings - contracts/api.yml — manual contract definition examples All files are heavily commented so users can copy them into their project root and customize without needing to reference the docs. --- examples/.preflight/config.yml | 63 ++++++++++++++++----------- examples/.preflight/contracts/api.yml | 59 ++++++++++--------------- examples/.preflight/triage.yml | 44 +++++++++---------- 3 files changed, 81 insertions(+), 85 deletions(-) diff --git a/examples/.preflight/config.yml b/examples/.preflight/config.yml index f59170f..297d8d7 100644 --- a/examples/.preflight/config.yml +++ b/examples/.preflight/config.yml @@ -1,35 +1,46 @@ -# .preflight/config.yml — Drop this in your project root -# -# This is an example config for a typical Next.js + microservices setup. -# Every field is optional — preflight works with sensible defaults out of the box. -# Commit this to your repo so the whole team gets the same preflight behavior. - -# Profile controls how much detail preflight returns. -# "minimal" — only flags ambiguous+ prompts, skips clarification detail +# .preflight/config.yml — Drop this in your project root. +# Every field is optional. Preflight works out of the box with sensible defaults. +# Commit this to your repo so your whole team gets the same behavior. + +# Profile controls how much detail preflight surfaces. +# "minimal" — only flag ambiguous+, skip clarification detail # "standard" — balanced (default) # "full" — maximum detail on every non-trivial prompt profile: standard -# Related projects for cross-service awareness. -# Preflight will search these for shared types, routes, and contracts -# so it can warn you when a change might break a consumer. +# Related projects for cross-service contract awareness. +# Preflight scans these for shared types, interfaces, and API routes +# so it can warn when a change in one project might break another. related_projects: - - path: /Users/you/code/auth-service - alias: auth - - path: /Users/you/code/billing-api - alias: billing - - path: /Users/you/code/shared-types - alias: types - -# Behavioral thresholds — tune these to your workflow + # Example: a backend API that your frontend talks to + # - path: /Users/you/projects/my-api + # alias: api + + # Example: a shared types package + # - path: /Users/you/projects/shared-types + # alias: shared + +# Behavioral thresholds — tune these to your workflow. thresholds: - session_stale_minutes: 30 # Warn if no activity for this long - max_tool_calls_before_checkpoint: 100 # Suggest a checkpoint after N tool calls - correction_pattern_threshold: 3 # Min corrections before flagging a pattern + # Warn if no session activity for this many minutes + session_stale_minutes: 30 -# Embedding provider for semantic search over session history. -# "local" uses Xenova transformers (no API key needed, runs on CPU). -# "openai" uses text-embedding-3-small (faster, needs OPENAI_API_KEY). + # Suggest a checkpoint after this many tool calls in one session + max_tool_calls_before_checkpoint: 100 + + # Minimum correction occurrences before flagging as a pattern + # (e.g., if you keep correcting "use the auth module" 3+ times, + # preflight learns to warn you proactively) + correction_pattern_threshold: 3 + +# Embedding configuration for vector search (timeline tools). +# Local embeddings work offline with no API key — they just need +# a one-time ~90MB model download on first use. embeddings: + # "local" — Xenova/all-MiniLM-L6-v2, runs on-device, no API key needed + # "openai" — OpenAI text-embedding-3-small, needs OPENAI_API_KEY + # "ollama" — Local Ollama server, needs `ollama serve` running provider: local - # openai_api_key: sk-... # Uncomment if using openai provider + + # Only needed if provider is "openai": + # openai_api_key: sk-... diff --git a/examples/.preflight/contracts/api.yml b/examples/.preflight/contracts/api.yml index 512543f..7a2ebd3 100644 --- a/examples/.preflight/contracts/api.yml +++ b/examples/.preflight/contracts/api.yml @@ -1,17 +1,15 @@ -# .preflight/contracts/api.yml — Manual contract definitions +# .preflight/contracts/api.yml — Manual contract definitions. # -# Define shared types and interfaces that preflight should know about. -# These supplement auto-extracted contracts from your codebase. -# Manual definitions win on name conflicts with auto-extracted ones. +# These supplement auto-extraction. Use them when: +# - A related service isn't in TypeScript (preflight can't auto-extract) +# - You want to pin a specific contract shape for stability +# - You're documenting an external API your project depends on # -# Why manual contracts? -# - Document cross-service interfaces that live in docs, not code -# - Define contracts for external APIs your services consume -# - Pin down types that are implicit (e.g., event payloads) +# Preflight uses these to warn when a prompt might affect shared types. - name: User kind: interface - description: Core user model shared across all services + description: Core user object shared between frontend and API fields: - name: id type: string @@ -19,40 +17,29 @@ - name: email type: string required: true - - name: tier - type: "'free' | 'pro' | 'enterprise'" + - name: role + type: "'admin' | 'member' | 'viewer'" required: true - name: createdAt type: Date required: true -- name: AuthToken +- name: ApiResponse kind: interface - description: JWT payload structure from auth-service + description: Standard API response envelope fields: - - name: userId - type: string - required: true - - name: permissions - type: string[] - required: true - - name: expiresAt - type: number - required: true - -- name: WebhookPayload - kind: interface - description: Standard webhook envelope for inter-service events - fields: - - name: event - type: string - required: true - - name: timestamp - type: string + - name: success + type: boolean required: true - name: data - type: Record - required: true - - name: source + type: T + required: false + - name: error type: string - required: true + required: false + +# Add your own shared types below: +# - name: OrderStatus +# kind: enum +# description: Order lifecycle states +# values: [pending, confirmed, shipped, delivered, cancelled] diff --git a/examples/.preflight/triage.yml b/examples/.preflight/triage.yml index b3d394e..4aa02d1 100644 --- a/examples/.preflight/triage.yml +++ b/examples/.preflight/triage.yml @@ -1,45 +1,43 @@ -# .preflight/triage.yml — Controls how preflight classifies your prompts -# -# The triage engine routes prompts into categories: -# TRIVIAL → pass through (commit, format, lint) -# CLEAR → well-specified, no intervention needed -# AMBIGUOUS → needs clarification before proceeding -# MULTI-STEP → complex task, preflight suggests a plan -# CROSS-SERVICE → touches multiple projects, pulls in contracts -# -# Customize the keywords below to match your domain. +# .preflight/triage.yml — Controls how preflight classifies your prompts. +# Customize these rules to match your project's domain vocabulary. rules: - # Prompts containing these words are always flagged as AMBIGUOUS. - # Add domain-specific terms that tend to produce vague prompts. + # Prompts containing these keywords are always flagged as AMBIGUOUS or higher. + # Add domain-specific terms that tend to produce vague requests. always_check: - rewards - permissions - migration - schema - - pricing # example: your billing domain - - onboarding # example: multi-step user flows + # Add your own: + # - billing + # - deployment + # - auth - # Prompts containing these words skip checks entirely (TRIVIAL). - # These are safe, mechanical tasks that don't need guardrails. + # Prompts containing these keywords pass through as TRIVIAL. + # These are routine commands where clarification adds no value. skip: - commit - format - lint - - prettier - - "git push" + # Add your own: + # - typecheck + # - "git status" - # Prompts containing these words trigger CROSS-SERVICE classification. - # Preflight will search related_projects for relevant types and routes. + # Prompts containing these keywords trigger CROSS-SERVICE checks. + # These scan related_projects (from config.yml) for shared contracts. cross_service_keywords: - auth - notification - event - webhook - - billing # matches the related_project alias + # Add your own: + # - graphql + # - grpc + # - queue # How aggressively to classify prompts. -# "relaxed" — more prompts pass as clear (experienced users) +# "relaxed" — more prompts pass as clear (less interruption) # "standard" — balanced (default) -# "strict" — more prompts flagged as ambiguous (new teams, complex codebases) +# "strict" — more prompts flagged as ambiguous (maximum savings) strictness: standard From ae3b5dbdbccf786134f949de79d4c5a650544b19 Mon Sep 17 00:00:00 2001 From: Jack Felke Date: Mon, 16 Mar 2026 09:00:36 -0700 Subject: [PATCH 3/3] fix: escape single quotes in timeline-db WHERE filters MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit buildWhereFilter was interpolating user-supplied values (project, branch, etc.) directly into WHERE clauses without escaping single quotes. This would break queries when paths or branch names contain apostrophes. searchExact already escaped quotes — this brings buildWhereFilter in line. Also adds 9 unit tests for lib/files.ts (readIfExists + findWorkspaceDocs). --- src/lib/timeline-db.ts | 15 ++++-- tests/lib/files.test.ts | 100 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 110 insertions(+), 5 deletions(-) create mode 100644 tests/lib/files.test.ts diff --git a/src/lib/timeline-db.ts b/src/lib/timeline-db.ts index 49b4f78..568a202 100644 --- a/src/lib/timeline-db.ts +++ b/src/lib/timeline-db.ts @@ -289,13 +289,18 @@ export async function insertEvents(events: TimelineEvent[], projectDir?: string) } } +/** Escape a string value for use in a LanceDB SQL-like WHERE clause. */ +function escapeValue(val: string): string { + return val.replace(/'/g, "''"); +} + function buildWhereFilter(opts: SearchOptions): string | undefined { const clauses: string[] = []; - if (opts.project) clauses.push(`project = '${opts.project}'`); - if (opts.branch) clauses.push(`branch = '${opts.branch}'`); - if (opts.type) clauses.push(`type = '${opts.type}'`); - if (opts.since) clauses.push(`timestamp >= '${opts.since}'`); - if (opts.until) clauses.push(`timestamp <= '${opts.until}'`); + if (opts.project) clauses.push(`project = '${escapeValue(opts.project)}'`); + if (opts.branch) clauses.push(`branch = '${escapeValue(opts.branch)}'`); + if (opts.type) clauses.push(`type = '${escapeValue(opts.type)}'`); + if (opts.since) clauses.push(`timestamp >= '${escapeValue(opts.since)}'`); + if (opts.until) clauses.push(`timestamp <= '${escapeValue(opts.until)}'`); return clauses.length > 0 ? clauses.join(" AND ") : undefined; } diff --git a/tests/lib/files.test.ts b/tests/lib/files.test.ts new file mode 100644 index 0000000..ec8a6ed --- /dev/null +++ b/tests/lib/files.test.ts @@ -0,0 +1,100 @@ +import { describe, it, expect, beforeEach, afterEach, vi } from "vitest"; +import { mkdtempSync, writeFileSync, mkdirSync, rmSync } from "fs"; +import { join } from "path"; +import { tmpdir } from "os"; + +let readIfExists: typeof import("../../src/lib/files.js").readIfExists; +let findWorkspaceDocs: typeof import("../../src/lib/files.js").findWorkspaceDocs; + +let tempDir: string; + +beforeEach(async () => { + tempDir = mkdtempSync(join(tmpdir(), "preflight-files-test-")); + process.env.CLAUDE_PROJECT_DIR = tempDir; + // Clear module cache so PROJECT_DIR re-evaluates + vi.resetModules(); + const mod = await import("../../src/lib/files.js"); + readIfExists = mod.readIfExists; + findWorkspaceDocs = mod.findWorkspaceDocs; +}); + +afterEach(() => { + rmSync(tempDir, { recursive: true, force: true }); + delete process.env.CLAUDE_PROJECT_DIR; +}); + +describe("readIfExists", () => { + it("returns null for missing files", () => { + expect(readIfExists("nonexistent.md")).toBeNull(); + }); + + it("reads existing text file", () => { + writeFileSync(join(tempDir, "hello.txt"), "line1\nline2\nline3"); + const result = readIfExists("hello.txt"); + expect(result).toBe("line1\nline2\nline3"); + }); + + it("truncates to maxLines", () => { + const lines = Array.from({ length: 100 }, (_, i) => `line ${i}`); + writeFileSync(join(tempDir, "long.txt"), lines.join("\n")); + const result = readIfExists("long.txt", 5); + expect(result!.split("\n")).toHaveLength(5); + expect(result).toBe("line 0\nline 1\nline 2\nline 3\nline 4"); + }); + + it("returns null for binary files", () => { + const buf = Buffer.alloc(100); + buf[10] = 0; // null byte + writeFileSync(join(tempDir, "binary.dat"), buf); + expect(readIfExists("binary.dat")).toBeNull(); + }); +}); + +describe("findWorkspaceDocs", () => { + it("returns empty when .claude dir missing", () => { + expect(findWorkspaceDocs()).toEqual({}); + }); + + it("finds markdown files in .claude/", () => { + const claudeDir = join(tempDir, ".claude"); + mkdirSync(claudeDir, { recursive: true }); + writeFileSync(join(claudeDir, "rules.md"), "# Rules\nBe good"); + writeFileSync(join(claudeDir, "notes.md"), "# Notes\nStuff"); + writeFileSync(join(claudeDir, "ignore.txt"), "not markdown"); + + const docs = findWorkspaceDocs(); + expect(Object.keys(docs)).toEqual(["notes.md", "rules.md"]); + expect(docs["rules.md"].content).toContain("# Rules"); + }); + + it("scans nested directories", () => { + const subDir = join(tempDir, ".claude", "sub"); + mkdirSync(subDir, { recursive: true }); + writeFileSync(join(subDir, "deep.md"), "# Deep"); + + const docs = findWorkspaceDocs(); + expect(docs["sub/deep.md"]).toBeDefined(); + expect(docs["sub/deep.md"].content).toContain("# Deep"); + }); + + it("skips node_modules and preflight-state", () => { + const nmDir = join(tempDir, ".claude", "node_modules"); + const psDir = join(tempDir, ".claude", "preflight-state"); + mkdirSync(nmDir, { recursive: true }); + mkdirSync(psDir, { recursive: true }); + writeFileSync(join(nmDir, "pkg.md"), "skip"); + writeFileSync(join(psDir, "state.md"), "skip"); + + expect(findWorkspaceDocs()).toEqual({}); + }); + + it("metadataOnly skips content", () => { + const claudeDir = join(tempDir, ".claude"); + mkdirSync(claudeDir, { recursive: true }); + writeFileSync(join(claudeDir, "doc.md"), "# Content here"); + + const docs = findWorkspaceDocs({ metadataOnly: true }); + expect(docs["doc.md"].content).toBe(""); + expect(docs["doc.md"].size).toBeGreaterThan(0); + }); +});