Verity

Verity is an MCP server for QA automation. It connects your AI client directly to your test suite — probe a URL for testable UI modules, generate compilable test files, execute the suite, diagnose failures, and receive a prioritized improvement plan. All through natural language.

---

Use Cases

• Test generation from a live page: Point Verity at a URL. It opens headless Chromium, maps the DOM into typed modules (forms, navs, dialogs, CTAs), intercepts live API calls, and generates a runnable test file with pre-filled selectors and assertions — not a stub.
• Failure diagnosis: After a failing run, ask for failure details. Verity returns the error message, every Playwright API call extracted from the trace, and direct paths to the screenshot, video, and trace file.
• Self-improving suite health: After enough runs, ask for an optimization plan. Verity classifies each test as broken, flaky, slow-regressing, or stable — and surfaces coverage gaps from previously analyzed URLs.
• Domain-aware test generation: Add a qa-knowledge.md with your business rules and historical bugs. Generated tests embed that context directly as comments, escaping generic stub output.

---

Installation

npm install -g qa-mcp-verity

Or run without a global install:

npx qa-mcp-verity

---

Client Setup

Two environment variables are required for every client:

Variable	Description
TEST_RUNNER	Test framework to use. Default: playwright
QA_PROJECT_ROOT	Absolute path to the project under test

Cursor

Add to ~/.cursor/mcp.json (global) or .cursor/mcp.json (per-project):

{
  "mcpServers": {
    "verity": {
      "command": "npx",
      "args": ["qa-mcp-verity"],
      "env": {
        "TEST_RUNNER": "playwright",
        "QA_PROJECT_ROOT": "/absolute/path/to/your-project"
      }
    }
  }
}

Open Cursor Settings → MCP and confirm Verity appears as connected.

Claude Desktop

Edit ~/Library/Application Support/Claude/claude_desktop_config.json (macOS) or %APPDATA%\Claude\claude_desktop_config.json (Windows):

{
  "mcpServers": {
    "verity": {
      "command": "npx",
      "args": ["qa-mcp-verity"],
      "env": {
        "TEST_RUNNER": "playwright",
        "QA_PROJECT_ROOT": "/absolute/path/to/your-project"
      }
    }
  }
}

Restart Claude Desktop after saving.

Claude Code

claude mcp add verity npx qa-mcp-verity \
  -e TEST_RUNNER=playwright \
  -e QA_PROJECT_ROOT=/absolute/path/to/your-project

Verify the server is registered:

claude mcp list

Gemini CLI

Edit ~/.gemini/settings.json:

{
  "mcpServers": {
    "verity": {
      "command": "npx",
      "args": ["qa-mcp-verity"],
      "env": {
        "TEST_RUNNER": "playwright",
        "QA_PROJECT_ROOT": "/absolute/path/to/your-project"
      }
    }
  }
}

Codex CLI

Edit ~/.codex/config.toml:

[mcp_servers.verity]
command = "npx"
args    = ["verity"]

[mcp_servers.verity.env]
TEST_RUNNER     = "playwright"
QA_PROJECT_ROOT = "/absolute/path/to/your-project"

---

Supported Runners

TEST_RUNNER value	Framework	Generated test format
playwright / playwright-ts	@playwright/test	.spec.ts
pytest / pytest-playwright	pytest-playwright	.py
jest	Jest	.test.ts
cypress	Cypress	.cy.ts
go / go-test	go test	_test.go

---

Tools

Discovery

Tool	Description
get_runner_info	Returns the active runner and all available options. Call this first in every session.
list_tests	Native test collection from the active runner.

Execution

Tool	Description
run_tests	Run the full suite. Accepts filter, browser (chromium/firefox/webkit), and headed.
run_failed	Re-run only the tests that failed in the last run.
codegen	Open an interactive Playwright recording session.

Reporting

Tool	Description
get_test_report	Cached summary from the last run (total, passed, failed, skipped, duration).
get_failure_details	Full failure packet: error, trace steps, screenshot/video/trace paths. Optionally filter by test_id.
get_test_history	Per-run pass rates over the last N runs, oldest-first for trend analysis.
generate_html_report	Self-contained HTML report with screenshots inlined. No external dependencies.

Generation

Tool	Description
analyze_url	Probe a URL with headless Chromium. Returns typed modules with selectors and candidate test cases, plus live API calls and layout overflow warnings.
generate_test	Generate a runnable test file from a description. Pass a module from analyze_url to get pre-filled selectors instead of a stub.
auto_generate_tests	analyze_url + generate_test in one call — one test per detected module.

Optimization

Tool	Description
get_optimization_plan	Three-layer analysis: broken/flaky/slow tests, MCP usage patterns, AI test adoption rate. Written to optimization-plan.md in the project root.

Domain Knowledge

Tool	Description
get_qa_context	Read qa-knowledge.md — business rules that produce domain-aware tests. Accepts an optional section name.
init_qa_knowledge	Scaffold a starter qa-knowledge.md in the project root.

Resources

URI	Description
verity://report/html	Latest test report as HTML
verity://report/json	Latest report.json
verity://report/optimization	Latest optimization-plan.md

---

Configuration

Variable	Default	Description
TEST_RUNNER	playwright	Active test framework
QA_PROJECT_ROOT	./tests_project	Root directory for all file I/O
QA_TIMEOUT_SECONDS	600	Subprocess timeout in seconds
QA_KNOWLEDGE_FILE	<root>/qa-knowledge.md	Override path for the domain knowledge file

---

Domain Knowledge

Generated tests use surface behavior by default — they fill forms, click buttons, and check for visible elements. To make them reflect your business rules:

# In your AI client:
init_qa_knowledge

Edit the scaffolded qa-knowledge.md to add business rules, historical bug regression points, exact UI error strings, and user journeys. On the next generate_test call, that context is embedded directly into the generated file.

See docs/qa-knowledge.example.md for a worked example across e-commerce, SaaS, content, and financial domains.

---

Development

npm install
npm run build      # compile src/ → dist/
npm run typecheck  # type-check without emitting
npm run dev        # watch mode

See docs/framework.md for the architecture reference: data flows, internal report format, security model, and how to add a new runner.

---

License

MIT