llm-redteam

Adversarial red-teaming harness that tests any OpenAI-compatible LLM endpoint for prompt injection, jailbreaks, and indirect injection via tool outputs.

Install

pip install -r requirements.txt

Requires Python 3.11+.

Usage

# run against a local Ollama instance
python run.py --base-url http://localhost:11434/v1 --model llama3

# run against Anthropic (via OpenAI-compat layer)
python run.py --base-url https://api.anthropic.com/v1 --api-key $KEY --model claude-sonnet-4-20250514

# only run jailbreak category, with HTML report
python run.py --categories jailbreak --html

# route through llm-firewall and compare bypass rates
python run.py --firewall --firewall-url http://localhost:8080

# use secondary LLM judge for ambiguous cases
python run.py --judge --judge-model gpt-4o

# also works as a module
python -m llm_redteam --base-url http://localhost:11434/v1 --model llama3

Architecture

                         +------------------+
                         |    run.py /      |
                         |  __main__.py     |
                         +--------+---------+
                                  |
                         +--------v---------+
                         |   config.py      |  .env + argparse
                         +--------+---------+
                                  |
                         +--------v---------+
                         |   runner.py      |  orchestration loop
                         +--+-----+-----+--+
                            |     |     |
              +-------------+     |     +-------------+
              |                   |                   |
     +--------v-----+   +--------v-----+   +---------v----+
     |  corpus.py   |   |  client.py   |   | scoring.py   |
     |  (JSONL)     |   |  (httpx)     |   | (heuristics) |
     +--------------+   +------+-------+   +------+-------+
                               |                  |
                        +------v-------+   +------v-------+
                        | LLM endpoint |   |  judge.py    |
                        | (any OpenAI- |   |  (optional)  |
                        |  compatible) |   +--------------+
                        +--------------+
                               |
                    +----------v-----------+
                    |  reporting/          |
                    |  - json_report.py    |
                    |  - html_report.py    |
                    |  - audit_log.py      |
                    +-----------+----------+
                                |
                       +--------v--------+
                       |   outputs/      |
                       +-----------------+

Corpus structure

Attack prompts live in corpus/ as JSONL files, one per category:

• direct_injection.jsonl — override system prompt, leak instructions, force output
• jailbreak.jsonl — DAN, roleplay, encoding tricks, persona hijacking
• indirect_injection.jsonl — payloads embedded in simulated tool/RAG responses

Each line:

{"id": "di-001", "category": "direct_injection", "prompt": "...", "tags": ["system_override"]}

For indirect injection, a context field carries the simulated tool output:

{"id": "ii-001", "category": "indirect_injection", "prompt": "Summarize this.", "context": "...[payload]...", "tags": ["tool_output"]}

llm-firewall integration

Pass --firewall to route all requests through a local llm-firewall instance. The harness sends requests to the firewall proxy instead of directly to the model, then compares bypass rates with vs without the firewall.

python run.py --firewall --firewall-url http://localhost:8080

The summary output will show both direct bypass rate and firewall bypass rate.

CI

GitHub Actions runs the full test suite against the built-in mock server on every push to master. No external API keys required — the mock server simulates both vulnerable and defended responses deterministically.

Known limitations

• Scoring heuristics are regex-based and will miss nuanced partial compliance
• The --judge flag requires a live LLM endpoint (adds cost and latency)
• Mock server responses are simplistic — real models exhibit more varied behavior
• No streaming support yet (responses are collected in full before scoring)
• Windows-only tested; Linux/macOS should work but is not CI-validated