Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,11 @@ The harness uses the [Claude Agent SDK](https://pypi.org/project/claude-agent-sd
| [GCP Vertex AI](https://cloud.google.com/vertex-ai) | `vertex` | Standard GCP credentials (`GOOGLE_APPLICATION_CREDENTIALS`, etc.) | Sets `CLAUDE_CODE_USE_VERTEX=1`. |

You can also set `base_url` in your config to point at a custom Anthropic-compatible endpoint.
If you need to override the `claude` executable itself, set `cli_path` in your
config to a wrapper script. This is the minimal integration point for launchers
such as `ollama launch claude`; see the configuration guide for the included
`scripts/ollama_claude_bridge.sh` helper and the `base_url: http://127.0.0.1:11434`
setting needed for capture/resampling parity.

With `provider: anthropic` (the default), if no `ANTHROPIC_API_KEY` is set, the SDK falls back to your Claude Code subscription credentials from `~/.claude/credentials.json` (requires Claude Pro/Max). Usage is covered by your subscription with rate limits rather than per-token billing. If `ANTHROPIC_API_KEY` is set in your environment, it takes precedence over subscription credentials.

Expand Down Expand Up @@ -236,6 +241,7 @@ Subagent messages are filtered from the parent trajectory to keep it clean. The
| `model` | yes | — | Claude model identifier (e.g. `claude-sonnet-4-20250514`). Use Anthropic model names, not OpenRouter-format names. |
| `provider` | no | `anthropic` | API provider: `anthropic`, `openrouter`, `bedrock`, `vertex` |
| `base_url` | no | — | Custom API base URL (overrides provider default) |
| `cli_path` | no | — | Override the `claude` executable path used by the Claude Agent SDK (for example, a wrapper around `ollama launch claude`). |
| `hypothesis` | no | — | One-sentence hypothesis this experiment tests. Shown in the web UI and saved to `run_meta.json`. |
| `work_dir` | yes | — | Working directory the agent operates in (any directory, not just repos) |
| `repo_name` | no | — | Human-readable name for the working directory |
Expand Down
73 changes: 73 additions & 0 deletions docs/guide/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,7 @@ sessions:
| `model` | yes | — | Claude model identifier (e.g. `claude-sonnet-4-20250514`) |
| `provider` | no | `anthropic` | API provider: `anthropic`, `openrouter`, `bedrock`, `vertex` |
| `base_url` | no | — | Custom API base URL (overrides provider default) |
| `cli_path` | no | — | Override the `claude` executable path used by the Claude Agent SDK. Useful for wrappers such as `ollama launch claude`. |
| `hypothesis` | no | — | What this experiment tests. Shown in the web UI. |
| `work_dir` | yes | — | Working directory the agent operates in (any directory) |
| `repo_name` | no | — | Human-readable name for the working directory |
Expand Down Expand Up @@ -92,6 +93,78 @@ sessions:
| GCP Vertex AI | `vertex` | GCP credentials | Sets `CLAUDE_CODE_USE_VERTEX=1` |
| Claude Code subscription | `anthropic` | *(none needed)* | If no `ANTHROPIC_API_KEY` is set, the SDK uses your Claude Code subscription credentials from `~/.claude/credentials.json`. Usage is covered by your subscription (Pro/Max) with rate limits rather than per-token billing. |

### Custom CLI wrappers

The Claude Agent SDK launches the `claude` CLI under the hood. If you want
to route AgentLens through an alternate launcher that still exposes the Claude
Code CLI interface, set `cli_path` to a wrapper executable.

One practical example is `ollama launch claude`. The wrapper needs to bridge
AgentLens' `--model ...` flag to the launcher layer, because Ollama expects
the model on `ollama launch claude --model <name>` rather than forwarding it
to the inner `claude` process.

Example bridge script:

```bash
#!/bin/bash
set -euo pipefail

model=""
args=()
while (($#)); do
case "$1" in
--model)
model="$2"
shift 2
;;
--model=*)
model="${1#--model=}"
shift
;;
*)
args+=("$1")
shift
;;
esac
done

exec ollama launch claude --model "$model" -- "${args[@]}"
```

AgentLens includes this bridge at:

```yaml
cli_path: "/absolute/path/to/agent-lens/scripts/ollama_claude_bridge.sh"
```

For full feature parity with capture/resampling, set the Ollama local endpoint
as `base_url` so AgentLens' proxy forwards captured requests to Ollama:

```yaml
model: "kimi-k2.5:cloud"
provider: anthropic
base_url: "http://127.0.0.1:11434"
cli_path: "/absolute/path/to/agent-lens/scripts/ollama_claude_bridge.sh"
work_dir: "./repos/my_project"
sessions:
- session_index: 1
prompt: "Explore the repo briefly."
```

Then point your config at it:

```yaml
model: "kimi-k2.5:cloud"
base_url: "http://127.0.0.1:11434"
cli_path: "/absolute/path/to/agent-lens/scripts/ollama_claude_bridge.sh"
provider: anthropic
work_dir: "./repos/my_project"
sessions:
- session_index: 1
prompt: "Explore the repo briefly."
```

### Cost reporting

Cost figures shown in `run_meta.json`, `harness inspect`, and the web UI come from the Claude Agent SDK's `total_cost_usd` field, which is calculated using Anthropic's list pricing regardless of which provider you use. This means:
Expand Down
60 changes: 60 additions & 0 deletions scripts/ollama_claude_bridge.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
#!/bin/bash
set -euo pipefail

model=""
args=()

while (($#)); do
case "$1" in
--model)
model="$2"
shift 2
;;
--model=*)
model="${1#--model=}"
shift
;;
*)
args+=("$1")
shift
;;
esac
done

if [[ -z "$model" ]]; then
echo "missing --model for ollama bridge" >&2
exit 2
fi

real_claude="$(command -v claude)"
if [[ -z "$real_claude" ]]; then
echo "real claude binary not found in PATH" >&2
exit 3
fi

# AgentLens sets ANTHROPIC_BASE_URL to its capture proxy before invoking the
# CLI. `ollama launch claude` overwrites that variable with Ollama's local
# endpoint, which disables request capture/resampling. Preserve the proxy URL
# and restore it inside the actual Claude process while still allowing Ollama
# to provide its auth token and model defaults.
capture_base="${ANTHROPIC_BASE_URL:-}"
tmpdir="$(mktemp -d)"

cat > "$tmpdir/claude" <<'INNER'
#!/bin/bash
set -euo pipefail

if [[ -n "${AGENTLENS_CAPTURE_BASE_URL:-}" ]]; then
export ANTHROPIC_BASE_URL="$AGENTLENS_CAPTURE_BASE_URL"
fi

exec "$AGENTLENS_REAL_CLAUDE" "$@"
INNER

chmod +x "$tmpdir/claude"

export AGENTLENS_REAL_CLAUDE="$real_claude"
export AGENTLENS_CAPTURE_BASE_URL="$capture_base"
export PATH="$tmpdir:$PATH"

exec ollama launch claude --model "$model" -- "${args[@]}"
1 change: 1 addition & 0 deletions src/harness/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@ class RunConfig(BaseModel):
model: str
provider: str = "anthropic"
base_url: str | None = None
cli_path: str | None = None

# working directory
work_dir: str
Expand Down
1 change: 1 addition & 0 deletions src/harness/runner.py
Original file line number Diff line number Diff line change
Expand Up @@ -126,6 +126,7 @@ async def run_session(
permission_mode=run_config.permission_mode,
cwd=cwd,
model=run_config.model,
cli_path=run_config.cli_path,
env=provider_env,
max_budget_usd=run_config.max_budget_usd,
setting_sources=setting_sources,
Expand Down
17 changes: 17 additions & 0 deletions tests/test_config.py
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,7 @@ def test_minimal_config(self):
def test_defaults(self):
rc = RunConfig.model_validate(_minimal())
assert rc.provider == "anthropic"
assert rc.cli_path is None
assert rc.tags == []
assert rc.max_turns == 50
assert rc.permission_mode == "bypassPermissions"
Expand Down Expand Up @@ -225,6 +226,22 @@ def test_load_with_memory_seed(self, tmp_path: Path):
assert rc.memory_file == "notes.md"
assert "Custom" in rc.memory_seed

def test_load_with_cli_path(self, tmp_path: Path):
config_yaml = tmp_path / "test.yaml"
config_yaml.write_text(
textwrap.dedent("""\
model: "kimi-k2.5:cloud"
cli_path: "/tmp/ollama-claude-bridge"
work_dir: "./repos/test"
sessions:
- session_index: 1
prompt: "hello"
""")
)
rc = load_config(config_yaml)
assert rc.model == "kimi-k2.5:cloud"
assert rc.cli_path == "/tmp/ollama-claude-bridge"


# ---------------------------------------------------------------------------
# build_provider_env
Expand Down