config-tester

A terminal UI (TUI) for testing Claude Code configurations (rules, skills, hooks, and settings).

Overview

config-tester discovers test cases under ~/.claude/tests/, runs each prompt through the claude CLI, and uses a second claude call as a judge to decide whether the response meets your written criteria. Results are shown in a Bubble Tea TUI where you confirm or override the judge's verdict. When your vote agrees with the judge, the result is saved to history. The tool is macOS-friendly and includes optional iTerm2 integrations (tab titles, clickable links, desktop notifications).

Requirements

• Go 1.25 or newer (to build).
• The claude CLI on your PATH. Both test execution and judging run claude -p "<prompt>".
• A test directory at ~/.claude/tests/ (see Test directory format below).

Installation and build

Build the binary directly:

go build ./cmd/config-tester

Or use the build script, which formats the code, runs the unit tests, and writes the binary to bin/config-tester:

./build.sh

Usage

Run the binary. It takes no command-line flags. It reads tests from ~/.claude/tests/ and writes history to ~/.claude/tests/history/.

./bin/config-tester

If no tests are found, it prints a message and exits.

Keybindings

Navigation
  up / k          Move up in the tree
  down / j        Move down in the tree
  Enter / Space   Expand or collapse a folder

Running tests
  r               Run all prompts in the selected folder
  x               Run the single selected prompt

Voting (after a test runs)
  p               Vote pass (confirms the judge verdict)
  f               Vote fail (overrides the judge verdict)
  s               Skip without voting

General
  ?               Toggle the help view
  q / Ctrl+C      Quit (cancels any running tests)

Test directory format

Tests live under ~/.claude/tests/, grouped by type. The valid types are rules, skills, hooks, and settings. Each test is a directory containing a green/ folder (prompts that should trigger the configuration), a red/ folder (prompts that should not trigger it), and a criteria.md file describing what counts as a pass.

~/.claude/tests/
└── rules/
    └── no-emojis/
        ├── green/
        │   ├── 01.txt
        │   └── 02.txt
        ├── red/
        │   └── 01.txt
        └── criteria.md

Prompt files must end in .txt. Each file's full contents are sent as one prompt.

How a test run works

1. Discovery scans ~/.claude/tests/ and loads every test (prompts plus criteria).
2. You select a prompt or folder and press x or r.
3. The runner executes each prompt with claude -p (5 minute timeout per call).
4. The judge sends the original prompt, the response, and your criteria back to claude and parses a JSON verdict of the form {"pass": bool, "confidence": 0.0-1.0, "reasoning": "..."}.
5. You vote. If your vote matches the judge, the result is saved under ~/.claude/tests/history/pass/ or ~/.claude/tests/history/fail/ as a timestamped JSON file. If you disagree, the result is not saved.

Development

Run all tests:

go test ./...

Run with verbose output:

go test -v ./...

The e2e/ tests drive a real iTerm2 window with AppleScript and are macOS-only. The unit tests under internal/ and pkg/ do not require iTerm2.

For a deeper walkthrough of the packages, data flow, and models, see ARCHITECTURE.md.