fix(http): prompt_tokens double-counted cache reads on the OpenAI wire

[Salil Das (sadlilas)]

Summary

extract_usage() computed prompt_tokens as inputTokens + cacheReadTokens + cacheWriteTokens, on the premise that the three are disjoint buckets. They are not — inputTokens already contains the cache reads, so the cached portion was counted twice.

On a cache-heavy turn (the normal case in an agent loop) that roughly doubles the reported prompt. The value flows to the terminal SSE chunk's usage block, so any OpenAI-compatible client doing its own cost arithmetic from prompt_tokens sees close to twice the real prompt.

Why the old premise was wrong

amplifier-core's docs/contracts/PROVIDER_CONTRACT.md specifies the llm:response usage payload:

Key	Description
input_tokens	Total input tokens — gross total (fresh + cache_read combined)
cache_read_tokens	Tokens served from prompt cache
cache_write_tokens	Tokens written to prompt cache (billed on top of gross)

Providers normalize to that shape:

• amplifier-module-provider-anthropic adds cache_read_input_tokens into input_tokens before emitting.
• amplifier-module-provider-openai subtracts only cache_write out of the vendor total, deliberately leaving cache reads inside.

So cacheReadTokens on the wire is a reported subset of inputTokens, surfaced for visibility — not an addend. cacheWriteTokens is the one bucket billed on top, so it is added.

The ecosystem's other consumer of this same event already encodes exactly this. From amplifier-module-hooks-streaming-ui:

All providers report input_tokens as the gross total — cache_read is already counted inside input_tokens, so adding it again would double-count. cache_write_tokens is the exception: cache creation cost is billed on top of gross and is NOT included in input_tokens.

return input_tokens + cache_create

The change

- prompt_total = new_input + cache_read + cache_write
+ # cache_read is deliberately absent: it is already inside gross_input.
+ prompt_total = gross_input + cache_write

Also renames the local new_input to gross_input so the name stops asserting the thing that was wrong, rewrites the docstring's three-bucket explanation, and corrects the matching comment on the accumulator in routes/chat_completions.py.

What is deliberately unchanged

• cached_tokens stays as cacheReadTokens. It is a subset of prompt_tokens under both the old and the new formula — but only the new one gives it a correct whole to be a subset of.
• cost_usd is unaffected. It is summed from the per-call cost the provider computed from the raw buckets, and was already right. Note that this is precisely why the bug was invisible: cost_usd and prompt_tokens in the same usage block did not reconcile, and only the token figure was wrong.

Verification

ruff check src/          All checks passed
ruff format --check      74 files already formatted
pyright src/             0 errors, 0 warnings

No test covered the arithmetic — the POC translator has no unit tests for extract_usage, which is how this survived. Worth a follow-up; not added here to keep the fix reviewable as a one-line correction.

🤖 Generated with Amplifier