From a91c912050618d7a86097a02b09d151af66c51a7 Mon Sep 17 00:00:00 2001 From: Aleksander Slominski Date: Wed, 2 Sep 2026 14:13:52 -0400 Subject: [PATCH] Docs: Clarify litellm-budget-track pipeline placement (inbound vs outbound) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The doc described litellm-budget-track as an inbound-pipeline plugin and its example config placed it under pipeline.inbound. The plugin is actually direction-agnostic (it implements the generic OnRequest/OnResponse/OnResponseFrame hooks and declares no fixed direction), and the correct pipeline depends on topology: inbound when AuthBridge fronts the LLM endpoint as a reverse proxy, outbound when hosting the agent via 'rossoctl authbridge exec', whose forward proxy runs the outbound pipeline on the agent's egress. In an 'authbridge exec' setup nothing reaches the inbound pipeline, so a plugin left under inbound: records $0 — a common 'it ran but tracked nothing' trap. Add a 'Pipeline placement' subsection at the config example and soften the two 'inbound pipeline' assertions in the title and Use Case to be direction-agnostic. Assisted-By: Claude (Anthropic AI) Signed-off-by: Aleksander Slominski --- authbridge/docs/litellm-budgettrack-plugin.md | 16 +++++++++++++--- 1 file changed, 13 insertions(+), 3 deletions(-) diff --git a/authbridge/docs/litellm-budgettrack-plugin.md b/authbridge/docs/litellm-budgettrack-plugin.md index 09785083..c027b6a3 100644 --- a/authbridge/docs/litellm-budgettrack-plugin.md +++ b/authbridge/docs/litellm-budgettrack-plugin.md @@ -1,6 +1,6 @@ # litellm-budget-track Plugin -Design document for the `litellm-budget-track` AuthBridge inbound pipeline plugin. +Design document for the `litellm-budget-track` AuthBridge pipeline plugin. **Issue:** https://github.com/rossoctl/rossoctl/issues/2177 @@ -16,8 +16,9 @@ with HTTP 429 when the configured daily budget is exceeded. When running AI agents through Cortex (local budget proxy), each agent needs a spending cap. LiteLLM returns the cost of each completion in the `x-litellm-response-cost` response header. This plugin reads that header in the -AuthBridge inbound pipeline, tracks cumulative daily spend in a JSON ledger file, -and blocks further requests once the budget is exhausted. +AuthBridge pipeline that carries the LLM traffic (see [Pipeline placement](#pipeline-placement)), +tracks cumulative daily spend in a JSON ledger file, and blocks further requests +once the budget is exhausted. ## Architecture @@ -91,6 +92,15 @@ pipeline: max_budget: 5.00 ``` +### Pipeline placement + +The plugin is direction-agnostic — place it in whichever pipeline carries the LLM +traffic: **`inbound`** when fronting the LLM endpoint (a reverse proxy the LLM requests +arrive at, as shown above), **`outbound`** when hosting the agent via +`rossoctl authbridge exec -- `, whose forward proxy runs the outbound pipeline on +the agent's own egress to LiteLLM. In an `authbridge exec` setup nothing reaches the +inbound pipeline, so a plugin left under `inbound:` there records `$0` — use `outbound:`. + | Field | Type | Required | Description | |-------|------|----------|-------------| | `spend_file` | string | yes | Path to the JSON ledger file (created if missing) |