feat(bench): multi-turn agent harness — read-only tools, loop, runner wiring (#102/#103)

[Connorrmcd6]

Summary

Phases 1–3 of the v2 multi-turn ("agentic") track — the offline foundation that lets a model choose to read a doc's hidden dependency instead of being forced to trust the doc (the single-shot cascade's structural limit). This is the groundwork for measuring verification suppression: does a confident stale doc stop an agent doing the file-read it otherwise would?

No API calls, no spend — everything here runs on a MockToolModel. The first step that touches real keys/money is the provider adapters (phase 4, Connorrmcd6/surface-bench#16), deliberately left out so this can be reviewed/merged as a self-contained unit.

What changed

• tools_runtime.py — a read-only tool surface (read_file, grep, list_dir, final_answer), path-scoped to a per-trial sandbox with escape rejection. dispatch() turns bad calls into recoverable error strings. Provider-neutral TOOL_SPECS for the upcoming adapter translation. Tracks accessed paths + a touched_hidden() helper — the basis of the verification-rate metric. scenario_sandbox() yields a fresh copy of code/ that includes the hidden dependency the prompt withholds.
• agent.py — run_agent() drives a ToolModel over the tools with a documented neutral message history. Terminates on final_answer, a text-only turn, or a forced answer-now nudge at max_turns. Returns a Trajectory (final text, turns, stop_reason, tool calls, accessed paths, per-turn tokens). The final answer still carries VERDICT:/FILE: blocks, so grading is byte-identical to single-shot and the two modes stay comparable.
• models.py — neutral Step/ToolCall types and a ToolModel protocol (step(...)), kept separate from the single-shot Model so existing adapters need no change. Step.provider_msg lets each future adapter echo history losslessly. MockToolModel for offline runs.
• run.py — --mode {single,multi} (+ config mode, default single so v1 is untouched) and --max-turns. Multi mode runs each trial through a fresh sandbox + run_agent, adds per-row fields (turns, stop_reason, tool_calls, verified_hidden, per_turn_tokens), tags every row with mode, and records mode/max_turns in run.json. A guard exits early if multi mode is asked of a model without step().
• config.toml — mode + max_turns keys.
• tests — 20 offline tests: the tool surface, the loop (incl. an end-to-end run against the real cascade-quota-batcher sandbox), and in-process runner integration proving multi writes the agent fields and single omits them.

Why not "Closes"

Part of #102 (agentic loop harness) and #103 (sandboxed execution + per-turn instrumentation). It doesn't fully close them: a real model can't run multi mode until step() lands on the provider adapters (#107), and the verification/compounding metrics + reporting are Connorrmcd6/surface-bench#13. Kept scoped so the offline core merges before any spend.

Verification

• uv run pytest (in bench/) — 20 passed. bench/ is Python and lives outside the cargo workspace, so this is the authoritative gate for the change.
• cargo fmt --all --check — clean (no Rust touched).
• Manual smokes: --mode single is an unchanged regression; --mode multi produces rows with all agent fields correctly populated (e.g. stop_reason=final_answer, verified_hidden plumbed end-to-end) and run.json records mode/max_turns.

🤖 Generated with Claude Code

[Connorrmcd6]

Added: Anthropic step() adapter (phase 4, Anthropic only).

This extends the PR beyond the offline foundation: AnthropicModel is now a ToolModel, so a real Claude model can drive the loop. The translation between the neutral loop format and the Anthropic wire format is in pure, unit-tested converters (_anthropic_tools / _anthropic_messages / _step_from_anthropic) — no network in the test suite (4 new offline tests, 24 total green). OpenAI + Gemini adapters remain separate (#107).

Live smoke (haiku · multi · cascade-quota-batcher-code · C0/C1/C2 · N=1) confirms the real round-trip and already previews the headline:

Cond	verified_hidden	turns	result
C0 (no doc)	✅ read window.py	4	correct
C1 (stale doc)	❌	1	misled
C2 (fresh doc)	❌	2	(n=1 noise)

i.e. with no doc the agent goes and checks the hidden dependency; with a confident stale doc it answers immediately and is misled. The real signal awaits N≥10 + the oracle tripwires (#105/#117).