Skip to content
Merged
24 changes: 16 additions & 8 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,10 +80,11 @@ These hold regardless of what any model outputs. They are the load-bearing part
be *static* (a tool always confirms) or *conditional on the args* (`create_channel` confirms only
when `private`), via `ToolSpec.needs_confirm`.
- **§2.9 Budgets.** A hard cap of **10 tool calls per request** (`ADMIN_MAX_TOOL_CALLS`) and **14
model round-trips** (`ADMIN_MAX_TURNS`), plus per-brain **daily token caps** (§11) checked before
every call. Both caps are env-overridable per deployment; hitting the tool-call cap mid-request
logs a warning and posts once to the ops channel (if configured), and the model is told to say so
plainly — the cap resets on the next request, not on a timer.
model round-trips** (`ADMIN_MAX_TURNS`), plus per-brain **daily token caps**, optionally layered
with a **daily USD cap** (§11), checked before every call. All caps are env-overridable per
deployment; hitting the tool-call cap mid-request logs a warning and posts once to the ops channel
(if configured), and the model is told to say so plainly — the cap resets on the next request, not
on a timer.
- **§2.10 No secrets in git — ever, not even encrypted.** Secrets live only in a `sops`+`age`
encrypted `roger.env` on the host. The repo carries `.sops.yaml` (the public recipient) and
`roger.env.example`. See [`deploy/`](deploy/README.md).
Expand Down Expand Up @@ -277,16 +278,23 @@ behaviour adds rows, not migrations.
## §11 LLM layer & budgets

`roger/llm.py` wraps the OpenAI SDK pointed at OpenRouter. Per call: pick the brain's model chain
(§3), **check the daily token cap before spending** (raises `BudgetExceeded` if over), call with
automatic fallback down the chain, then **record actual usage** to `usage`. A missing/empty model
chain raises `LLMConfigError`, which callers turn into a plain "not configured" reply rather than a
crash. Real spend is additionally bounded off-box by the OpenRouter key's own credit limit.
(§3), **check the daily token cap before spending** (raises `BudgetExceeded` if over), then — if a
`DAILY_USD_<BRAIN>` cap is also set — check accumulated USD spend the same way. The two caps are
layered, not either/or: the token cap always enforces, and the USD cap is an additional, optional
trip wire on top of it. That's deliberate — a provider that never reports cost
(`OPENROUTER_BASE_URL` pointed elsewhere, ADR-0009) would otherwise leave the USD cap permanently
silent, so the token cap stays the real backstop in that case. Once both checks pass, the call
proceeds with automatic fallback down the chain, then **records actual usage** to `usage`. A
missing/empty model chain raises `LLMConfigError`, which callers turn into a plain "not configured"
reply rather than a crash. Real spend is additionally bounded off-box by the OpenRouter key's own
credit limit.

Limits at a glance (defaults; all env-overridable):

| Control | Default |
|---|---|
| Daily tokens — admin / ambient / digest / gigabrain | 150k / 40k / 30k / 100k |
| Daily USD — admin / ambient / digest / gigabrain | off / off / off / off (0 = disabled) |
| Tool calls per admin request | 10 |
| Model round-trips per admin request | 14 |
| Tool calls / round-trips per gigabrain request | 10 / 14 |
Expand Down
8 changes: 4 additions & 4 deletions BACKLOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ Effort key: **S** ≈ an afternoon, **M** ≈ a day or two, **L** ≈ multi-day.

Gaps that matter for a bot that's actually live. These are the ones I'd do first.

### 1.1 Track spend in dollars, not just tokens — **M** — *visibility shipped; gate remains*
### 1.1 Track spend in dollars, not just tokens — **M** — *shipped*
`llm.py` records `prompt_tokens` / `completion_tokens` per brain (`add_usage`) and the daily cap is a
raw token count. But a brain's model chain mixes models at very different prices, so a token budget
is a weak proxy for the thing that actually costs money. OpenRouter returns the real generation cost
Expand All @@ -26,9 +26,9 @@ is a weak proxy for the thing that actually costs money. OpenRouter returns the
- [x] Add a `cost_usd` column to the `usage` table (with an idempotent migration for live DBs);
capture the OpenRouter-reported cost per call in `LLM.complete`. *(a2689b5)*
- [x] Surface per-brain and total `$ today` in `/status`. *(a2689b5)*
- [ ] Make the daily gate dollar-denominated (env: `DAILY_USD_*`) with the token cap as the fallback
when a provider doesn't report cost. Deferred: enforcement is a semantic change, kept out of the
visibility commit.
- [x] Make the daily gate dollar-denominated (env: `DAILY_USD_*`), layered on top of the token cap
rather than replacing it — a provider that never reports cost leaves the token cap as the real
backstop, so nothing regresses for a non-OpenRouter host.

*Why:* the single most portfolio-differentiating item here — real LLM cost governance is exactly the
infra+AI bridge the portfolio is aiming at, and it's the honest version of the budget the code
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,9 +81,9 @@ SQLite so they survive restarts. Key series:
| Metric | Type | Labels |
|---|---|---|
| `roger_tokens_today` / `roger_tokens_cap` | gauge | `brain` |
| `roger_cost_usd_today` | gauge | `brain` |
| `roger_cost_usd_today` / `roger_cost_usd_cap` | gauge | `brain` |
| `roger_llm_requests_total` / `roger_llm_errors_total` | counter | `brain` (`type`) |
| `roger_llm_budget_exceeded_total` | counter | `brain` |
| `roger_llm_budget_exceeded_total` | counter | `brain`, `reason` |
| `roger_audit_events` | gauge | `tool`, `status` |
| `roger_feeds`, `roger_build_info` | gauge | — (`version`) |

Expand Down
4 changes: 4 additions & 0 deletions compose.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,10 @@ services:
DAILY_TOKENS_AMBIENT: ${DAILY_TOKENS_AMBIENT:-40000}
DAILY_TOKENS_DIGEST: ${DAILY_TOKENS_DIGEST:-30000}
DAILY_TOKENS_GIGABRAIN: ${DAILY_TOKENS_GIGABRAIN:-100000}
DAILY_USD_ADMIN: ${DAILY_USD_ADMIN:-0}
DAILY_USD_AMBIENT: ${DAILY_USD_AMBIENT:-0}
DAILY_USD_DIGEST: ${DAILY_USD_DIGEST:-0}
DAILY_USD_GIGABRAIN: ${DAILY_USD_GIGABRAIN:-0}
ADMIN_MAX_TOOL_CALLS: ${ADMIN_MAX_TOOL_CALLS:-10}
ADMIN_MAX_TURNS: ${ADMIN_MAX_TURNS:-14}
GIGABRAIN_MAX_TOOL_CALLS: ${GIGABRAIN_MAX_TOOL_CALLS:-10}
Expand Down
2 changes: 1 addition & 1 deletion docs/decisions/0001-dollar-cost-tracking-token-gate.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# ADR-0001: Track spend in dollars, keep enforcement on tokens

- **Status:** Accepted
- **Status:** Accepted; enforcement mechanism superseded by [ADR-0010](0010-layer-the-dollar-budget-gate-not-flip-to-it.md)
- **Date:** 2026-07-23

## Context
Expand Down
40 changes: 40 additions & 0 deletions docs/decisions/0010-layer-the-dollar-budget-gate-not-flip-to-it.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
# ADR-0010: Layer the dollar budget gate on the token gate, don't flip to it

- **Status:** Accepted
- **Date:** 2026-08-18

## Context

ADR-0001 split dollar-cost visibility from enforcement and predicted the natural follow-up: "flip
the gate to dollars with tokens as fallback." That follow-up landed differently. `DAILY_USD_<BRAIN>`
is optional and, when set, enforces *alongside* the token cap rather than replacing it — the token
cap still runs unconditionally on every call.

The reason is `cost_today()`. OpenRouter always returns a real `usage.cost`, but the field is an
OpenRouter extension, not a `usage` guarantee. ADR-0009 documents `OPENROUTER_BASE_URL` staying
pointed-but-configurable at a non-OpenRouter host. If that ever happens, `cost_today()` for that
brain sits at `0.0` forever. A gate that used dollars as the *primary* enforcement (tokens only as
fallback, per ADR-0001's plan) would need to detect "is cost data actually flowing" before it could
fall back — one more thing to get right, and wrong in exactly the case (a broken or misconfigured
cost feed) where the gate matters most.

Layering sidesteps that detection problem. The token check is unconditional and always runs first;
the dollar check is a second, independent, optional trip wire that runs after it, only when
`DAILY_USD_<BRAIN> > 0`. If dollar data never arrives, the dollar check simply never fires — no false
permissiveness, no code path that has to notice.

## Decision

Both caps enforce when both are configured. Whichever trips first wins. The token cap is never
replaced, only ever added to.

## Consequences

- A live host with a working OpenRouter key gets governance in the currency that actually matters —
real spend — without losing the safety net a broken cost feed would otherwise remove.
- Two numbers to reason about per brain instead of one, in `/status`, the ops alert, and
`roger.env.example`. Both default off, so this stays opt-in, not a forced complication.
- Supersedes ADR-0001's Consequences section, which predicted "flip... with tokens as fallback" —
that mechanism was designed but not built, once the non-reporting-provider case above ruled it out.
ADR-0009's cross-reference to "the existing dollar-cost tracking and token gate (ADR-0001)" should
now be read alongside this record.
Loading