Local prompt-repair layer for AI coding agents
Repair messy coding requests into clean, safe prompts — 100% local, no SaaS, no data upload.
Quick Start • Eval Center • Features • Architecture • Contributing
PromptFix is a local, open-source prompt-repair layer for AI coding agents. You write the way you actually think — "kral login token refresh bozuldu başka yeri bozma" — and PromptFix turns it into a clear, safe, scoped prompt your coding agent can act on.
It runs entirely on your machine: no SaaS backend, no accounts, your prompts and API keys never leave your computer. CLI + Chrome extension + a built-in evaluation suite that proves the quality.
Select text anywhere → Right-click or press a hotkey → Get an optimized prompt.
Input:
kral login token refresh bozuldu başka yeri bozma
Output:
Investigate and fix the login token refresh issue with minimal, targeted
changes. Inspect the existing auth/session/token refresh flow first, avoid
unrelated refactors or config/secrets changes, run relevant tests if
available, and summarize the root cause, fix, and verification steps.
| Feature | Description | |
|---|---|---|
| 🔄 | One-Click Rewrite | Right-click menu + global hotkeys (Windows) |
| ⌨️ | Keyboard Shortcuts | Browser-level shortcuts (Ctrl+Shift+1–5) for every mode |
| 🧠 | Intent Detection | Auto-detects bugfix/feature/performance/review in Turkish & English |
| 🛡️ | Output Guard | Validates output, retries on failure, deterministic fallback |
| 📐 | Structured JSON Guard | Automatically extracts optimized prompt from JSON responses |
| Quality Score | Heuristic 0-100 score with breakdown: specificity, actionability, safety, and more | |
| 🔀 | Before/After Diff | See exactly what changed between your original and the optimized prompt |
| 💬 | Threaded Chat | Discord-like chat with streaming, snippets, slash commands |
| 🧪 | Evaluation Center | Built-in benchmark suite: 40 tests, rule-based + LLM judge |
| ⚡ | Sub-Second Speed | Groq for speed, Ollama for privacy, OpenAI-compatible for flexibility |
| 🔁 | Multi-Provider Fallback | Automatically retries with other configured providers if the primary fails |
| 🚦 | Rate Limiting | Per-IP request limits on all endpoints (60 req/min for optimize & chat) |
pip install -e .Option A: .env file (easiest)
cp .env.example .env
# Edit .env and add your keyOption B: Environment variable
setx GROQ_API_KEY "your_groq_api_key"Then open a new terminal.
Option C: Config file
# ~/.promptfix/config.yaml
providers:
groq:
api_key: "your_key_here" # Direct key (not recommended for shared systems)promptfix initpromptfix service- Open Chrome →
chrome://extensions - Enable Developer mode
- Click Load unpacked
- Select the
extension/folder
Browser — context menu: Select text → Right-click → PromptFix → choose a mode:
| Menu item | Mode | Best for |
|---|---|---|
| Optimize Coding Prompt | short |
General code tasks |
| Fast Rewrite | fast |
Quick one-liner fixes |
| Agent Prompt | agent |
Multi-step agent tasks |
| Explain Mode | explain |
Understanding / root-cause analysis |
| Raw Rewrite | raw |
Unstructured free-form rewrite |
Browser — popup: Click the extension icon to see:
- Quick Optimize — type a rough prompt directly in the popup and click Optimize Prompt to get an optimised version without selecting text on the page. The result is shown inline and auto-copied to your clipboard.
- Service status (Connected / Offline) with provider, model, and uptime
- Recent history — each item shows:
- Truncated input and output
- Score badge (colour-coded: 🟢 85+, 🟡 60–84, 🔴 <60) — only shown when score data is available
- Copy button — copies the optimised prompt to the clipboard; shows "Copied!" feedback for 1.5 s; falls back gracefully if Clipboard API is unavailable
- Default mode selector — saved to sync storage
Diff toggle: When a history item has diff data available, a "Diff" button appears next to the copy button. Clicking it expands an inline panel showing the before/after diff with colour coding:
- Green lines (
+) — additions in the optimised prompt - Red lines (
-) — removals from the original prompt - Grey lines — context and diff headers
The panel scrolls if the diff is long. Click "Hide" to collapse it.
Local result cache: Every optimize request sent through the extension (context menu or popup) is cached locally in chrome.storage.local:
- Stores the last 5 results only
- Entries expire after 24 hours automatically
- Cache persists if the service goes offline — popup can still show recent history with diff
- Never stored: API key, service token, provider credentials, or config data
- Cache is read-only from the popup; only the background service worker writes to it
Offline state: When the service is not running the popup shows a red "Service offline — run: promptfix service" banner. Recent locally-cached results are still shown.
Any app (hotkeys): Select text → press Ctrl+Alt+S (requires promptfix tray)
PromptFix ships with a built-in evaluation framework inspired by enterprise prompt testing tools.
# Rule-based scoring (fast, free)
promptfix eval
# With LLM judge (more accurate, uses tokens)
promptfix eval --judge
# Generate HTML report
promptfix eval --report eval-report.html
# CI mode (fails if score < 75)
promptfix eval --ci --threshold 80
# JSON output
promptfix eval --format json- 40 test cases (20 Turkish + 20 English)
- Bugfix, Feature, Performance, Agent, Review intents
- Constraint adherence: "başka yeri bozma" → must include "minimal/targeted"
- Output cleanliness: No markdown fences, no "Here is" prefixes
- Intent alignment: Correct task type & domain detection
┌─────────────────────────┬────────┬─────────┬────────┬──────────┐
│ Test │ Score │ Mode │ Status │ Duration │
├─────────────────────────┼────────┼─────────┼────────┼──────────┤
│ Auth bugfix (TR) │ 94/100│ short │ ✅ PASS│ 450ms │
│ API 500 error (TR) │ 88/100│ short │ ✅ PASS│ 380ms │
│ New API endpoint (EN) │ 92/100│ agent │ ✅ PASS│ 520ms │
│ Dashboard render (TR) │ 85/100│ explain │ ✅ PASS│ 610ms │
└─────────────────────────┴────────┴─────────┴────────┴──────────┘
Total: 40/40 passed | Avg: 89/100 | Provider: groq
Create evals/my-suite.yaml:
tests:
- name: "My custom test"
input: "fix the login bug minimal changes"
mode: short
asserts:
- type: contains
value: ["minimal", "auth"]
- type: not_contains
value: ["refactor"]
- type: intent_match
task_type: bugfix
domain: authRun it:
promptfix eval --suite evals/my-suite.yaml| Mode | Hotkey | Description |
|---|---|---|
| fast | — | Ultra-compact, max 60 words |
| short | Ctrl+Alt+S | Compact paragraph, max 120 words (default) |
| agent | Ctrl+Alt+P | Structured: Task / Context / Instructions / Constraints / Validation / Deliverables |
| raw | Ctrl+Alt+R | Plain rewrite, no structure |
| explain | — | Include root-cause reasoning, max 300 words |
See exactly what changed between your original prompt and the optimized version.
# Show colored diff in the terminal
promptfix once "kral login token refresh bozuldu başka yeri bozma" --diffSample output:
--- original
+++ optimized
-kral login token refresh bozuldu başka yeri bozma
+Investigate and fix the login token refresh issue with minimal, targeted changes.
+Inspect the existing auth/session/token refresh flow first, avoid unrelated refactors,
+run relevant tests if available, and summarize the root cause, fix, and verification steps.
Via the API:
POST /optimize
{ "text": "fix login bug", "mode": "short", "include_diff": true }Response includes a diff object:
{
"optimized": "Investigate and fix the login...",
"diff": {
"unified": "--- original\n+++ optimized\n...",
"unchanged": false
}
}Every rewrite is automatically scored on 5 dimensions (0–20 each, total 0–100):
| Dimension | What it checks |
|---|---|
| specificity | Concrete action verbs vs. vague filler |
| conciseness | Word count in ideal range for the mode |
| actionability | Deliverable / verification signals present |
| safety | No conversational openers or broadening words |
| intent_alignment | Output reflects detected task type and domain |
# Show score breakdown in terminal
promptfix once "login token bozuldu başka yeri bozma" --scoreSample output:
Quality Score Breakdown
┌─────────────────┬───────┬─────┐
│ Dimension │ Score │ Max │
├─────────────────┼───────┼─────┤
│ specificity │ 16 │ 20 │
│ conciseness │ 20 │ 20 │
│ actionability │ 15 │ 20 │
│ safety │ 20 │ 20 │
│ intent_alignment│ 16 │ 20 │
│ TOTAL │ 87 │ 100 │
└─────────────────┴───────┴─────┘
Grade: A
The API always returns score_breakdown in the /optimize response:
{
"optimized": "...",
"quality_score": 87,
"score_breakdown": {
"total": 87,
"grade": "A",
"breakdown": { "specificity": 16, "conciseness": 20, ... },
"suggestions": []
}
}Use the agent-safety-checklist preset to generate prompts that enforce safe coding-agent behavior:
- Read relevant files before writing code
- Produce a short plan before implementing
- Change only the minimum files necessary
- Never touch secrets, API keys, or production config
- Flag database migrations and breaking changes explicitly
- Run existing tests after the change
- Report changed files, test outcome, and risks
# Via CLI preset
promptfix preset use agent-safety-checklist "deploy the new payment service"
# In chat
/preset agent-safety-checklist
# With text flag
promptfix preset use agent-safety-checklist --text "login token refresh bozuldu başka yeri bozma"This is the safest mode for giving tasks to AI coding agents in critical codebases.
┌─────────────────────┐ ┌─────────────────────┐ ┌──────────────┐
│ Browser Extension │────▶│ Local Service │────▶│ Groq / LLM │
│ (right-click menu) │◀────│ 127.0.0.1:52849 │◀────│ Provider │
└─────────────────────┘ └─────────────────────┘ └──────────────┘
▲
┌─────────┴─────────┐
│ Global Hotkeys │
│ (Windows tray) │
└───────────────────┘
One shared PromptFix Core — extension, hotkeys, CLI, and evaluation all use the same rewrite pipeline.
# Interactive first-time setup (choose provider, enter API key, test connection)
promptfix init
# One-shot optimize
promptfix once "login token bozuldu başka yeri bozma" --mode short
# One-shot with diff view
promptfix once "login token bozuldu başka yeri bozma" --diff
# One-shot with quality score breakdown
promptfix once "login token bozuldu başka yeri bozma" --score
# Combine both
promptfix once "login token bozuldu başka yeri bozma" --diff --score
# Start local service
promptfix service
# Start tray with global hotkeys
promptfix tray
# Interactive chat
promptfix chat
promptfix chat --mode agent
# Evaluation suite
promptfix eval
promptfix eval --judge --report report.html
# Provider management
promptfix provider list
promptfix provider use groq
promptfix provider doctor groq
# Debug tools
promptfix debug-intent "login token refresh bozuldu"
promptfix debug-rewrite "login token refresh bozuldu" --mode agent
# Reload config without restarting
curl -X POST http://127.0.0.1:52849/config/reload- Slash commands:
/mode,/clear,/history,/threads,/new,/load,/delete,/snippet,/help - Snippets: Save reusable prompt fragments, expand with
:snippet_name: - Streaming: Real-time token-by-token responses
- Thread persistence: Auto-saved JSON threads in
~/.promptfix/threads/
- API keys stored only in environment variables or
~/.promptfix/config.yaml(never sent to the browser) - Local service binds to 127.0.0.1 only — not reachable from outside your machine
- Browser extension never sees API keys
- CORS restricted to
chrome-extension://andlocalhostorigins — unknown web origins are blocked - Note: The extension targets Chrome/Chromium. Firefox support is planned.
- Input length limited to 32 000 characters per request — oversized payloads are rejected with HTTP 413
- All endpoints protected by the optional service token (
service.tokenin config) — including/history,/threads,/chat, and/suggestions - Thread IDs validated as UUID v4 on every endpoint — path-traversal attempts are rejected with HTTP 400
- Optional service token for extra authentication (
service.tokenin config) - Live config reload —
POST /config/reloadre-reads~/.promptfix/config.yamland resets the provider without restarting the service - Hotkeys are Windows-only — importing
hotkeys.pyon Linux/macOS no longer crashes;promptfix trayexits gracefully with an informative message - Rate limiting — all endpoints have per-IP rate limits (default 300 req/min globally;
/optimize,/chat,/chat/streamcapped at 60 req/min;/historyat 30 req/min;/config/reloadat 10 req/min) - Multi-provider fallback — if the primary LLM provider fails, PromptFix automatically tries all other configured providers before raising an error
- No SaaS backend, no user accounts, no database
PromptFix is designed to be local-first. Binding to 127.0.0.1 (localhost) means:
- The service is not reachable from other devices on your network.
- Your prompts, history, and API keys never leave your machine.
- If you need to change the bind address (e.g. for a VM), run
promptfix service --host 0.0.0.0— but be aware this exposes the service to your local network.
- Never commit your
.envfile orconfig.yamlto version control. - The browser extension never sees your API key; it only talks to the local service.
- If you set a
service.token, the extension stores it in Chrome's local (not sync) storage.
# Option A: .env file
cp .env.example .env
# Edit .env and add: GROQ_API_KEY=gsk_...
# Option B: Environment variable (Windows CMD)
setx GROQ_API_KEY "gsk_your_key"
# Then open a NEW terminal window.- Make sure the service is started:
promptfix service - Check your firewall is not blocking
127.0.0.1:52849. - Verify the extension's Service URL in Options matches
http://127.0.0.1:52849.
- The extension needs activeTab permission. Reload the extension from
chrome://extensions. - Some pages (e.g.
chrome://, PDF viewers) block content scripts — this is expected.
ollama pull qwen2.5:7b
promptfix provider use ollama
promptfix serviceNo API key needed for Ollama. If Ollama is not running, PromptFix will fall back to other configured providers.
promptfix trayis Windows-only. On Linux/macOS, use the browser extension instead.- Install tray extras:
pip install promptfix[tray] - Run as Administrator if hotkeys still don't register.
Without an API key, the eval suite runs in stub mode (deterministic fallback). This still tests rule-based scoring but does not exercise the LLM. To run full evals, set a GROQ_API_KEY or use Ollama.
We welcome contributions! See CONTRIBUTING.md for guidelines.
Quick start for developers:
pip install -e ".[dev]"
pytest -v
promptfix eval --ciMIT License — see LICENSE for details.
Built with ❤️ for the open-source AI community.
If PromptFix helps you write better prompts, consider giving it a ⭐ on GitHub!