sol-audit-pipeline

A local, fully offline 3-stage Solidity audit pipeline:

Stage 1 (static) → Stage 2 (12-lens LLM adversarial review) → Stage 2.5 (gate/judging pass) → Stage 3 (Foundry invariant scaffolds) → markdown report.

No subscription, no cloud API calls required — everything runs against a local Ollama model. The 12 specialist "attacker lens" prompts, the senior-auditor mindset (SOP), the 4-gate judging logic, and the report format are adapted from pashov/skills (MIT licensed), originally built for Claude Code's parallel Agent-tool orchestration. This reimplements the same prompts/logic as a plain Python pipeline against a local model instead.

Why this exists

The original skill needs Claude Code's Agent tool (parallel background sub-agents) and, in practice, a Claude subscription or API billing. This version swaps that orchestration for a ThreadPoolExecutor + Ollama HTTP calls, so it runs entirely on your own hardware for free — at the cost of speed and, likely, finding quality versus a frontier hosted model.

Requirements

• Python 3.10+
• Slither (pip install slither-analyzer)
• Foundry (for compiling target projects and running generated invariant tests)
• Ollama, running locally, with a coder-capable model pulled

Installing Ollama

curl -fsSL https://ollama.com/install.sh | sh

(or sudo snap install ollama if you prefer snap — either works; the install script tends to be more up to date)

This installs Ollama as a systemd service, listening on http://localhost:11434.

Setup

python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt

Edit config.yaml to match your hardware — key settings:

• ollama.model — which local model to use
• ollama.parallelism — set to 1 on CPU-only machines (no benefit to concurrency without real parallel compute; risks OOM). Raise it if you have a GPU with spare VRAM.
• ollama.num_ctx — context window sent to Ollama; lower it on constrained RAM, raise it if your model/hardware supports more (larger codebases need more context).

Usage

python src/pipeline.py /path/to/solidity/project

The target project must already compile (forge build / hardhat compile) — Slither needs a successful build to analyze it.

Flags:

• --skip-lenses — run Stage 1 only (fast, free, deterministic). Useful for a quick sanity pass or CI.
• --save-json — dump raw stage1/stage2/gates output to output/debug_raw.json for debugging or manual review of everything the lenses said before gating.

Output: a markdown report at output/<project-name>-audit-report-<timestamp>.md.

Architecture

Stage	What it does	Cost
1 — stage1_static.py	Runs Slither, normalizes findings, produces flagged (contract, function) targets	Free, deterministic, fast
2 — stage2_lenses.py	Runs 12 specialist adversarial prompts (lenses/*.md) against the full in-scope source via local LLM	Slow on CPU (~minutes per lens per file)
2.5 — gates.py	Groups Stage 2 output by group_key, runs one LLM "judge" call per group applying the 4-gate logic from references/judging.md	One extra LLM call per distinct claim group
3 — invariant scaffolds (planned)	Generates Foundry invariant test stubs for surviving high-confidence findings	Free, deterministic template fill
— report.py	Renders everything into the final markdown report	Free, deterministic

Honest limitations

• Local 7B-class models are meaningfully weaker than hosted frontier models at this kind of adversarial code reasoning. Expect more format drift, missed subtleties, and both false positives and false negatives compared to running the original skill in Claude Code with a frontier model. Treat every output — CONFIRMED or LEAD — as a lead for manual verification, not a final verdict.
• CPU-only inference is slow. A single small contract can take 45-75+ minutes for a full 12-lens + judging pass. This is not designed for rapid iteration; it's designed for an unattended/overnight pass before you sit down to manually verify hits.
• Cross-contract echo promotion (from judging.md's lead-promotion rules) is not yet implemented — only same-group_key multi-agent convergence is.
• Every lens currently sees the entire in-scope source bundle, not just Stage 1's flagged targets — this is intentional for lenses needing whole-system context (invariants, cross-function flows), but it means Stage 1's "gate_stage2_on_stage1_findings" config option isn't wired up yet to actually narrow scope.
• This tool does not replace competitive audit contest submissions requiring novel human insight, nor bug bounty submissions requiring a working PoC — it's a triage/leverage tool to point you at the right places faster.

License / attribution

Lens prompts, SOP, judging gates, and report format are adapted from pashov/skills (MIT License). Pipeline/orchestration code in this repo is original.