BadgerForge

Our team's agent for the MLM26 Efficient Coder Challenge. Scored on Terminal-Bench 2.1 with open-weight models served by BadgerBrain.

See plan.md for what we're building next and CONTRIBUTING.md for how we work.

Setup

You need Docker running, the campus VPN (GlobalProtect), and a BadgerBrain key. Ask Chris (endemann@wisc.edu) for the key if you don't have one — it takes a day or two and is tied to your NetID.

# 1. Python env
uv venv --python 3.12 && source .venv/bin/activate
uv pip install -e ".[dev]"

# 2. Terminal-Bench 2.1 tasks
git clone https://github.com/harbor-framework/terminal-bench-2-1.git ~/Documents/terminal-bench-2-1

# 3. Check the gateway is reachable (needs VPN; 401 is the correct answer)
curl -s -o /dev/null -w "%{http_code}\n" https://llm-gw01.doit.wisc.edu/v1/models

Your key

The key never goes in a file. Save the 1Password share link Chris sends into your vault, then point .env.op at it:

LLM_API_KEY=op://Employee/<your item>/credential

op vault list and op item list show the names. .env.op in the repo is a template with a placeholder, so copy it and edit your copy — .env.op.local is gitignored:

cp .env.op .env.op.local   # then put your item name in it

The scripts load it with scripts/env.sh, which asks the 1Password CLI for the key and keeps it in that shell's environment only. On Windows the 1Password app won't authorize op.exe from inside WSL, so runs start from scripts\run_full_windows.ps1, which reads the key on the Windows side and passes it in; see docs/setup-windows.md.

./scripts/run_task.sh regex-log

Gotchas we already hit:

• A plain .env file overrides what env.sh loads (the loader uses override=True), so don't create one.
• harbor's own process grows with every finished trial — past 11 GB by task 56 of 89 — and gets OOM-killed on a 16 GB machine unless WSL has swap (docs/setup-windows.md). harbor jobs resume continues; a trial directory without a result.json is rerun, a finished one is kept unless you delete it.
• First request after an idle period waits ~90s for GPU cold start.
• LLM_MAX_TOKENS is 16384 since 2026-10-09; below ~8192 the reasoning model returns empty responses and times out. An .env.op.local copied before then still says 8192: change it.
• Long runs need the machine awake and the VPN up: prefix with caffeinate -i. A dropped connection used to kill the whole task; the client now retries, but a sleeping laptop still stalls everything.
• Apple Silicon can't run the qemu tasks (x86 emulation lacks a syscall), and build-cython-ext has a broken reference solution upstream.

Layout

See CONTRIBUTING.md. Short version: decision logic in agent/, prompts as Markdown in agent/prompts/, benchmark scripts in scripts/.

The agent in agent/ started as the challenge starter (qualiaMachine/MLM26_EfficientCoder), restructured to the layout above.