claude-batch-pipeline

Async batch processing for Claude via the Message Batches API. 50% cheaper per token than synchronous calls, processes up to 100,000 requests per batch with 24h SLA.

Use when you have a large pile of prompts (classification, summarization, extraction, embedding-style scoring) that doesn't need real-time response.

What you get

• Submitter: chunks an input file into batches and submits them
• Watcher: polls batch status until done, fans out parallel batch polls
• Collector: streams batch results, writes them to JSONL with original prompt IDs preserved
• Cost reporter: tracks tokens spent and dollars saved vs sync mode
• Resume: state file lets you restart safely after crash or kill
• Retry: per-prompt retry for the small fraction that errors inside a batch

Total: ~200 LOC Python, no LangChain, no framework.

Quick start

git clone https://github.com/arpit2005/claude-batch-pipeline.git
cd claude-batch-pipeline

pip install -r requirements.txt

cp .env.example .env
# Set ANTHROPIC_API_KEY

# Submit a batch (sample: 100 prompts in 30 seconds)
python submit.py sample_prompts.jsonl --model claude-sonnet-4-20250514

# Output: state file with batch_id, polling starts automatically
# When done (typically ~5min for small batches, up to 24h for large):
cat results.jsonl

Input format

JSONL — one prompt per line:

{"custom_id": "doc_001", "prompt": "Summarize: ..."}
{"custom_id": "doc_002", "prompt": "Extract dates from: ..."}

Each prompt becomes a batch request with the original custom_id preserved so you can join results back to your data.

Output format

JSONL — one result per line, same custom_id as input:

{"custom_id": "doc_001", "result": "...", "usage": {"input_tokens": 250, "output_tokens": 120}, "model": "claude-sonnet-4-20250514"}
{"custom_id": "doc_002", "result": "...", "usage": {...}}
{"custom_id": "doc_003", "error": "max_tokens exceeded", "retry_after": "manual"}

Errored requests are kept in the output so you can decide whether to retry.

Cost benchmark

Sample run: 1,000 prompts, ~500 input / ~200 output tokens each.

Mode	Time	Total cost (Sonnet 4)	Cost per 1K prompts
Synchronous (Messages API)	~25 min (parallelized)	$4.50	$4.50
Batch	~12 min (this run)	$2.25	$2.25

50% savings at this scale. Scales linearly — at 100K prompts you save ~$225 per run.

When NOT to use batch

• Real-time UX (chatbot, IDE assistant) — sync only
• < 50 prompts at a time — overhead isn't worth it
• Need streaming output — batches return final responses only

Files

File	LOC	Purpose
submit.py	~80	Read JSONL, build batch payload, submit to Anthropic, write state file
poll.py	~70	Watch batch status, fetch results when done, write JSONL output
cost.py	~30	Read state file, compute USD spent and saved-vs-sync
sample_prompts.jsonl	—	5 sample prompts to test against

License

MIT.