Skip to content

Fresh-eyes pass: mid-2026 pricing, specific model matching, codex token fix - #3

Open
fielding wants to merge 3 commits into
mainfrom
fable/fresh-eyes
Open

Fresh-eyes pass: mid-2026 pricing, specific model matching, codex token fix#3
fielding wants to merge 3 commits into
mainfrom
fable/fresh-eyes

Conversation

@fielding

@fielding fielding commented Jul 2, 2026

Copy link
Copy Markdown
Owner

Assessment

Fresh-eyes review of thinktax as of July 2026 (last commit June 7). State of the project:

  • Health: builds clean (tsup), 37 tests passing, CLI runs end-to-end against real local data. The only hiccup was a corrupted node_modules (npm ci fixed it; nothing in-repo).
  • Parsers: the Claude Code collector handles the current session-log format correctly (message.usage with input_tokens / cache_creation_input_tokens / etc.). The Codex collector parses current token_count events but mis-modeled their semantics (fixed here).
  • Pricing data: stale (dated 2026-02-05). Every model that dominates real usage today was missing — claude-opus-4-8 (61k log entries locally), claude-fable-5 (13.8k), claude-sonnet-4-6, claude-opus-4-7, gpt-5.3-codex (14k), gpt-5.4. Two silent failure modes followed: opus-4-8 fuzzy-matched the legacy claude-opus-4 entry at $15/$75 (3x overestimate), and fable-5 matched nothing and was excluded from totals entirely.

Changes

  1. Pricing table refresh — adds Claude Fable 5 / Mythos 5 ($10/$50), Opus 4.8 / 4.7 ($5/$25), Sonnet 5 / 4.6 ($3/$15), and OpenAI gpt-5.5 ($5/$30), gpt-5.4 ($2.50/$15), gpt-5.3-codex ($1.75/$14). Claude prices confirmed against current Anthropic docs; OpenAI against the official pricing page.
  2. Most-specific fuzzy matchingfindPricing now prefers the longest matching entry instead of first-in-table-order, so prefix families (claude-opus-4 vs claude-opus-4-8) can't mis-price each other. Regression tests pin every model seen in real local logs to its bundled rates.
  3. Codex cached/reasoning token fix — Codex payloads satisfy total_tokens = input_tokens + output_tokens; cached_input_tokens and reasoning_output_tokens are subsets. The collector was billing cached input twice (full rate + cache-read rate) and adding reasoning on top of output. With ~90% of codex input cached, estimates ran 5–7x high.

Verification

  • npm test: 53 passing (was 37), including new codex-semantics tests built from a real session payload and a bundled-table regression suite.
  • Ran refresh + status against real local data (~103k events): all current models resolve to their own entries; codex all-time estimate dropped from $2.1k-ish to ~$600 (gpt-5.3-codex $1442 → $197), consistent with the cached-input correction.

Note: this branch is independent of #1 (billing-honesty work) and shouldn't conflict with it beyond trivial proximity.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Xan5eRiAXvcnAVvXHejdTz

fielding and others added 3 commits July 2, 2026 11:39
The table was last updated 2026-02-05 and predates every model that now
dominates real usage. Adds anthropic claude-fable-5 / claude-mythos-5
($10/$50), claude-opus-4-8 / claude-opus-4-7 ($5/$25),
claude-sonnet-5 / claude-sonnet-4-6 ($3/$15), and openai gpt-5.5
($5/$30), gpt-5.4 ($2.50/$15), gpt-5.3-codex ($1.75/$14), with cache
rates at the standard 1.25x write / 0.1x read multipliers.

Before this, claude-opus-4-8 usage fuzzy-matched the legacy
claude-opus-4 entry and was estimated at 3x the real price, while
claude-fable-5 matched nothing and was silently excluded from totals.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Xan5eRiAXvcnAVvXHejdTz
Fuzzy matching took the first table entry that was a substring of the
model id, so matching depended on table order: any new model sharing a
prefix with an older family (claude-opus-4-8 vs claude-opus-4) could
resolve to the wrong rates. Pick the longest matching entry instead,
and add a regression suite that pins the models seen in real mid-2026
Claude Code and Codex CLI logs to their correct bundled rates.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Xan5eRiAXvcnAVvXHejdTz
Codex token_count payloads satisfy total_tokens = input_tokens +
output_tokens: cached_input_tokens is a subset of input_tokens and
reasoning_output_tokens a subset of output_tokens. The collector was
billing cached tokens twice (full input rate plus cache-read rate) and
adding reasoning on top of output. With ~90% of real codex input served
from cache, estimates ran ~5-7x high (gpt-5.3-codex all-time dropped
from $1442 to $197 on this machine's data).

Split cached out of input so it bills at the cache-read rate, keep
output as reported, and cover the semantics with unit tests built from
a real session log payload.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Xan5eRiAXvcnAVvXHejdTz
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant