From 5fedff4f849cae4d5aa2bed671bf91fcb50bea21 Mon Sep 17 00:00:00 2001 From: NessZerra Date: Sun, 12 Jul 2026 06:04:33 +0700 Subject: [PATCH 1/5] Document token-cost freshness design Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- .../2026-07-12-token-cost-freshness-design.md | 77 +++++++++++++++++++ 1 file changed, 77 insertions(+) create mode 100644 docs/superpowers/specs/2026-07-12-token-cost-freshness-design.md diff --git a/docs/superpowers/specs/2026-07-12-token-cost-freshness-design.md b/docs/superpowers/specs/2026-07-12-token-cost-freshness-design.md new file mode 100644 index 0000000000..0f5887429e --- /dev/null +++ b/docs/superpowers/specs/2026-07-12-token-cost-freshness-design.md @@ -0,0 +1,77 @@ +# Token-Cost Freshness Design + +## Goal + +Port the relevant CodexBar v0.42.0 behavior into the active Rust/Tauri +architecture: + +- token-cost age is independent from provider quota freshness; +- fast cost-fetch failures can retry on the next scheduled pass; +- unknown-model pricing refreshes are bounded and coalesced. + +The change remains one focused PR. It does not add a second scheduler, +dependencies, or filesystem paths. + +## Architecture + +The existing Tauri local-usage cache in `commands/chart.rs` remains the +token-cost refresh boundary. Its cached summary gains an explicit token-cost +timestamp sourced from the local scan result. Consumers use this timestamp for +token-cost age; `AppState.provider_cache_updated_at` continues to describe +provider quota data only. + +The shared Rust pricing boundary in `rust/src/core/cost_pricing.rs` owns +unknown-model refresh coordination. A cache-path keyed in-flight guard +coalesces concurrent refreshes. A bounded refresh-attempt timestamp prevents +repeated misses from creating a loop. A miss may trigger at most one refresh +and one retry before the existing fallback behavior is returned. + +All existing cache and log paths remain unchanged. Refresh work uses the +existing execution paths and does not become part of provider quota refresh. + +## Data flow + +1. A local token-cost request checks the token-cost timestamp and its existing + local usage TTL. +2. If stale, the existing refresh worker performs the scan and stores the + scan timestamp with the summary. +3. On a non-timeout fetch failure, the token-cost suppression timestamp is + cleared so a later scheduled/manual pass can retry promptly. Timeout + failures retain suppression until the normal TTL expires. +4. When pricing lookup cannot find a model, one request starts a refresh for + the relevant pricing cache path. Concurrent requests await that same + refresh. After the refresh, the original lookup retries once; if it still + misses, the existing fallback is used without recursion. + +## Error handling + +Refresh failures are surfaced through the existing cost-fetch error/fallback +behavior. They do not silently manufacture fresh timestamps or recursively +retry. In-flight state is cleared on both success and failure, and bounded +attempt state is updated only when a refresh is actually attempted. + +## Tests + +Tests are written before implementation and cover: + +- token-cost timestamp and quota freshness remain independent; +- fast cost-fetch failures clear suppression while timeouts retain it; +- concurrent unknown-model misses result in one refresh; +- an unknown model does not loop after the bounded refresh and retry; +- refresh failure releases coalescing state for a later bounded attempt. + +Validation includes both Cargo format checks, shared and Tauri Rust tests, +shared and Tauri Clippy with `-D warnings`, frontend checks when touched, and +`git diff --check`. + +## Upstream mapping + +This ports behavior, not Swift structure: + +- upstream `CostUsageFetcher` token snapshots preserve the underlying scan + timestamp rather than treating every cache read as fresh; +- upstream `UsageStore` clears token-fetch TTL suppression for fast failures + but keeps it for timed-out scans; +- upstream `ModelsDevPricing` coordinates refreshes by cache path, limits + repeated attempts, and retries an unknown-model lookup once. + From 83ae1143d5fdfd5f0ce161e7db899eb10af890ec Mon Sep 17 00:00:00 2001 From: NessZerra Date: Sun, 12 Jul 2026 06:07:16 +0700 Subject: [PATCH 2/5] Plan token-cost freshness implementation Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- .../plans/2026-07-12-token-cost-freshness.md | 276 ++++++++++++++++++ 1 file changed, 276 insertions(+) create mode 100644 docs/superpowers/plans/2026-07-12-token-cost-freshness.md diff --git a/docs/superpowers/plans/2026-07-12-token-cost-freshness.md b/docs/superpowers/plans/2026-07-12-token-cost-freshness.md new file mode 100644 index 0000000000..e4615fe6fb --- /dev/null +++ b/docs/superpowers/plans/2026-07-12-token-cost-freshness.md @@ -0,0 +1,276 @@ +# Token-Cost Freshness Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Port upstream v0.42.0 token-cost freshness, fast retry, and bounded unknown-model pricing refresh behavior into the existing Rust/Tauri cost pipeline. + +**Architecture:** Keep local token scanning and its in-memory cache in `commands/chart.rs`; add a scan timestamp and a narrow failure-retry policy without coupling it to `AppState.provider_cache_updated_at`. Add the models.dev catalog, disk cache, dynamic pricing lookup, and cache-path keyed refresh coordinator under the shared Rust `core` pricing boundary. The Tauri chart refresh performs one scan, refreshes only when that scan reports unknown models, and rescans once when pricing becomes available. + +**Tech Stack:** Rust 2024, Tokio already present in both crates, existing `reqwest` async client, `serde`/`serde_json`, Tauri commands, existing Vitest frontend tests. + +## Global Constraints + +- Port behavior, not Swift structure, from upstream steipete/CodexBar v0.41.0-v0.42.0. +- Token-cost age is independent from provider quota freshness. +- Fast cost-fetch failures clear token-cost suppression; timed-out scans retain the normal TTL. +- Unknown-model refreshes are bounded, coalesced by cache path, and retry the lookup at most once. +- Do not add a second scheduler or dependencies. +- Preserve all existing filesystem paths. +- Keep provider-specific pricing and refresh logic inside the shared cost/pricing boundary. +- Do not log raw tokens, cookies, API keys, or model-pricing response contents. +- Use TDD: write each focused failing test before its implementation. + +--- + +### Task 1: Track scan age and cost-fetch retry policy + +**Files:** +- Modify: `rust/src/cost_scanner.rs:23-48` to expose unavailable model IDs. +- Modify: `apps/desktop-tauri/src-tauri/src/commands/chart.rs:19-285` to carry scan age, separate cache age from quota age, and classify failures. +- Test: `apps/desktop-tauri/src-tauri/src/commands/chart.rs` in a new `#[cfg(test)]` module. + +**Interfaces:** +- `CostSummary::unknown_models: HashSet`. +- `ProviderLocalUsageSummary::token_cost_updated_at_ms: i64`, serialized as `tokenCostUpdatedAtMs`. +- `fn token_cost_cache_is_fresh(loaded_at: Option, now: Instant, ttl: Duration) -> bool`. +- `fn cost_fetch_failure_allows_early_retry(failure: CostFetchFailure) -> bool`, false only for `TimedOut`. + +- [ ] **Step 1: Write the failing tests** + +```rust +#[test] +fn token_cost_age_does_not_use_provider_quota_age() { + let token_loaded = Instant::now() - Duration::from_secs(31); + let provider_updated = Instant::now(); + assert!(!token_cost_cache_is_fresh( + Some(token_loaded), Instant::now(), Duration::from_secs(30) + )); + assert!(is_provider_cache_fresh(Some(provider_updated), Duration::from_secs(30))); +} + +#[test] +fn fast_cost_failures_allow_the_next_pass_to_retry() { + assert!(cost_fetch_failure_allows_early_retry(CostFetchFailure::Failed)); + assert!(!cost_fetch_failure_allows_early_retry(CostFetchFailure::TimedOut)); +} +``` + +Also assert that serialized local usage contains `tokenCostUpdatedAtMs`, and +update existing Rust summary fixtures with the timestamp. + +- [ ] **Step 2: Run focused tests and verify failure** + +Run `cargo test --manifest-path apps/desktop-tauri/src-tauri/Cargo.toml token_cost_age_does_not_use_provider_quota_age fast_cost_failures_allow_the_next_pass_to_retry`. +Expected: the new field and functions do not compile yet. + +- [ ] **Step 3: Implement the minimal cache/retry change** + +Set the token timestamp from local scan completion, never from +`AppState.provider_cache_updated_at`. Keep the local cache `loaded_at` as a +separate `Instant`. Record unknown model IDs while preserving current fallback +costs. Clear only token-cost suppression for fast failures; retain it for +timeouts. + +- [ ] **Step 4: Run the focused tests and verify success** + +Run the command from Step 2. Expected: cache-age, retry-policy, and +serialization tests pass. + +- [ ] **Step 5: Commit** + +```powershell +git add rust/src/cost_scanner.rs apps/desktop-tauri/src-tauri/src/commands/chart.rs apps/desktop-tauri/src-tauri/src/commands/tests.rs +git commit -m "Separate token-cost age from quota freshness`n`nCo-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>" +``` + +### Task 2: Add models.dev pricing cache and coalesced refresh + +**Files:** +- Create: `rust/src/core/models_dev_pricing.rs` for catalog decoding, cache artifacts, dynamic pricing, and refresh coordination. +- Modify: `rust/src/core/mod.rs:3-33` to register and re-export the pipeline. +- Modify: `rust/src/core/cost_pricing.rs:585-812` to consult refreshed rates after static lookup. +- Test: `rust/src/core/models_dev_pricing.rs` for parser, cache, and coordinator behavior. + +**Interfaces:** +- `pub async fn refresh_unknown_models_if_needed(provider_id: &str, model_ids: &HashSet) -> bool`. +- `pub fn lookup(provider_id: &str, model_id: &str) -> Option`. +- A cache-path keyed coordinator with one in-flight operation and a 15-minute attempt gate. + +- [ ] **Step 1: Write failing coalescing and bounded-attempt tests** + +```rust +#[tokio::test] +async fn concurrent_refreshes_for_one_cache_path_share_one_operation() { + let coordinator = ModelsDevRefreshCoordinator::default(); + let calls = Arc::new(AtomicUsize::new(0)); + let first_calls = Arc::clone(&calls); + let first = coordinator.refresh(path.clone(), now, async move { + first_calls.fetch_add(1, Ordering::SeqCst); + tokio::time::sleep(Duration::from_millis(10)).await; + true + }); + let second = coordinator.refresh(path, now, async { + panic!("the second caller must await the first operation"); + }); + assert!(tokio::join!(first, second).0); + assert_eq!(calls.load(Ordering::SeqCst), 1); +} + +#[tokio::test] +async fn failed_refresh_is_not_retried_within_the_attempt_window() { + let coordinator = ModelsDevRefreshCoordinator::default(); + assert!(!coordinator.refresh(path.clone(), now, async { false }).await); + assert!(!coordinator.refresh(path, now + Duration::from_secs(60), async { + panic!("the 15-minute bound must suppress this attempt"); + }).await); +} +``` + +Add JSON tests for both the top-level provider map and the `providers` envelope, +including conversion from USD per million tokens to USD per token. + +- [ ] **Step 2: Run focused tests and verify failure** + +Run `cargo test --manifest-path rust/Cargo.toml models_dev_pricing`. +Expected: coordinator, catalog, and dynamic lookup types are missing. + +- [ ] **Step 3: Implement the cache and coordinator** + +Decode both catalog envelopes; store a versioned artifact under the existing +per-user cache root without moving any existing path; use a 24-hour cache TTL; +require priceable OpenAI and Anthropic entries; merge still-priceable entries +from the previous catalog; and use the existing async `reqwest::Client` with a +20-second timeout. Key in-flight and last-attempt state by standardized cache +path. Return false on transport, status, decode, or implausible-catalog errors +without logging response contents. + +- [ ] **Step 4: Implement dynamic lookup** + +Add `DynamicModelPricing` with input, output, cache-read, cache-write, and +optional over-200K rates. Have static pricing win first, then consult the +refreshed provider/model catalog, then preserve the existing fallback. + +- [ ] **Step 5: Run focused tests and commit** + +Run `cargo test --manifest-path rust/Cargo.toml models_dev_pricing` and +`cargo test --manifest-path rust/Cargo.toml test_unknown_model_falls_back_to_sonnet`. +Then commit: + +```powershell +git add rust/src/core/mod.rs rust/src/core/models_dev_pricing.rs rust/src/core/cost_pricing.rs +git commit -m "Coalesce unknown-model pricing refreshes`n`nCo-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>" +``` + +### Task 3: Wire one bounded unknown-pricing retry into Tauri + +**Files:** +- Modify: `apps/desktop-tauri/src-tauri/src/commands/chart.rs:70-285` for one scan, refresh, and rescan. +- Modify: `rust/src/cost_scanner.rs` and `rust/src/codex_costs.rs` to record unknown IDs. +- Modify: `apps/desktop-tauri/src-tauri/src/commands/tests.rs` and `powertoys.rs` for the timestamp field. +- Modify: `apps/desktop-tauri/src/types/bridge.ts` and affected frontend fixtures. +- Test: `chart.rs` for one refresh/rescan and no recursion. + +**Interfaces:** +- The local scan result carries the summary and `HashSet` unknown IDs. +- The pricing pipeline is called once; a successful refresh permits one rescan + with retry disabled, and an unsuccessful refresh returns the first result. +- Token-cost timestamp remains independent from provider snapshot `updated_at`. + +- [ ] **Step 1: Write failing retry-bound tests** + +Use a test-only callback seam: + +```rust +#[test] +fn unknown_models_trigger_one_refresh_and_one_rescan() { + let mut refresh_calls = 0; + let mut scan_calls = 0; + let outcome = retry_unknown_pricing_once( + vec!["claude-future-1".to_string()], + || { refresh_calls += 1; true }, + || { scan_calls += 1; vec![] }, + ); + assert_eq!(refresh_calls, 1); + assert_eq!(scan_calls, 1); + assert!(outcome.unknown_models.is_empty()); +} + +#[test] +fn unavailable_pricing_does_not_loop() { + let outcome = retry_unknown_pricing_once( + vec!["claude-future-1".to_string()], + || false, + || panic!("no second scan after an unsuccessful refresh"), + ); + assert_eq!(outcome.unknown_models, ["claude-future-1"]); +} +``` + +- [ ] **Step 2: Run focused tests and verify failure** + +Run `cargo test --manifest-path apps/desktop-tauri/src-tauri/Cargo.toml unknown_models_trigger_one_refresh_and_one_rescan unavailable_pricing_does_not_loop`. +Expected: the retry seam is not present. + +- [ ] **Step 3: Wire the existing chart refresh** + +After the first 30-day/today scan, collect unknown IDs and call the shared +async pricing pipeline through the existing Tauri runtime. If pricing becomes +available, rerun that scan exactly once with retry disabled. Otherwise retain +the first result and fallback costs. Keep `is_refreshing` and the existing +provider scheduler unchanged. Clear only token-cost suppression on fast worker +failure; retain it on timeout; do not let a late cancelled worker overwrite a +newer cache entry. + +- [ ] **Step 4: Update fixtures and run focused checks** + +Run: + +```powershell +cargo test --manifest-path apps/desktop-tauri/src-tauri/Cargo.toml unknown_models_trigger_one_refresh_and_one_rescan unavailable_pricing_does_not_loop +cd apps/desktop-tauri +npm test -- --run src/types/bridge.test.ts src/components/MenuCard.test.tsx src/floatbar/FloatBar.test.tsx +``` + +- [ ] **Step 5: Commit** + +```powershell +git add rust/src/cost_scanner.rs rust/src/codex_costs.rs apps/desktop-tauri/src-tauri/src/commands/chart.rs apps/desktop-tauri/src-tauri/src/commands/tests.rs apps/desktop-tauri/src-tauri/src/powertoys.rs apps/desktop-tauri/src/types/bridge.ts +git commit -m "Retry unknown token pricing once`n`nCo-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>" +``` + +### Task 4: Full validation, reviews, and focused PR + +- [ ] **Step 1: Format and inspect** + +Run `cargo fmt --all -- --check` and `git diff --check`. + +- [ ] **Step 2: Validate shared Rust** + +Run `cargo test --manifest-path rust/Cargo.toml` and +`cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings`. + +- [ ] **Step 3: Validate Tauri Rust** + +Run `cargo test --manifest-path apps/desktop-tauri/src-tauri/Cargo.toml` and +`cargo clippy --manifest-path apps/desktop-tauri/src-tauri/Cargo.toml --all-targets -- -D warnings`. + +- [ ] **Step 4: Validate frontend** + +From `apps/desktop-tauri`, run `npm test -- --run`, `npm run lint`, and +`npm run build` because the bridge type changes. + +- [ ] **Step 5: Run correctness and thermo-nuclear maintainability reviews** + +Review the complete diff against `main` for quota/token age separation, +fast-versus-timeout retry behavior, one cache-path in-flight refresh, one +unknown-model rescan, no recursive loop, preserved paths, provider boundaries, +duplicate schedulers, oversized abstractions, duplicated normalization, broad +error swallowing, and unnecessary public APIs. Fix only findings caused by the +PR and rerun affected tests. + +- [ ] **Step 6: Push and open, without merging** + +Run `git diff main...HEAD --check`, verify the worktree and final diff, then +push `finesssee-port-cost-freshness` and open one PR into `main` documenting +the upstream v0.41.0-v0.42.0 mapping and the exact checks above. + From 510768a8b16dc618a21fcce8477d7266fb68a0a5 Mon Sep 17 00:00:00 2001 From: NessZerra Date: Sun, 12 Jul 2026 06:57:00 +0700 Subject: [PATCH 3/5] Separate token-cost age from quota freshness Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- .../src-tauri/src/commands/chart.rs | 110 +++++++++++++++++- apps/desktop-tauri/src-tauri/src/powertoys.rs | 1 + .../src/components/MenuCard.test.tsx | 1 + .../src/floatbar/FloatBar.test.tsx | 1 + .../src/surfaces/TrayPanel.test.tsx | 1 + apps/desktop-tauri/src/types/bridge.ts | 1 + rust/src/codex_costs.rs | 27 +++++ rust/src/cost_scanner.rs | 21 ++++ 8 files changed, 160 insertions(+), 3 deletions(-) diff --git a/apps/desktop-tauri/src-tauri/src/commands/chart.rs b/apps/desktop-tauri/src-tauri/src/commands/chart.rs index c3b7f254a1..a2262d7e6b 100644 --- a/apps/desktop-tauri/src-tauri/src/commands/chart.rs +++ b/apps/desktop-tauri/src-tauri/src/commands/chart.rs @@ -14,7 +14,7 @@ use std::sync::{ Arc, Mutex, OnceLock, atomic::{AtomicBool, Ordering}, }; -use std::time::{Duration, Instant}; +use std::time::{Duration, Instant, SystemTime, UNIX_EPOCH}; const LOCAL_USAGE_TTL: Duration = Duration::from_secs(30); @@ -53,6 +53,7 @@ pub struct ProviderLocalUsageSummary { pub latest_tokens: Option, pub top_model: Option, pub estimate_note: String, + pub token_cost_updated_at_ms: i64, } /// Full chart data bundle for one provider. @@ -87,10 +88,12 @@ pub async fn get_provider_chart_data( pub async fn get_provider_local_usage_summary( provider_id: String, ) -> Option { + let failure_provider_id = provider_id.clone(); tauri::async_runtime::spawn_blocking(move || load_provider_local_usage_summary(&provider_id)) .await .unwrap_or_else(|err| { tracing::warn!("Provider local usage worker failed: {}", err); + record_local_usage_fetch_failure(&failure_provider_id, CostFetchFailure::Failed); None }) } @@ -187,6 +190,7 @@ fn load_local_usage_summary( latest_tokens: non_zero_u64(latest_tokens), top_model: top_model(&thirty_day), estimate_note: localized_estimate_note(provider_id, lang), + token_cost_updated_at_ms: current_unix_ms(), }) } @@ -228,6 +232,7 @@ pub(crate) async fn refresh_provider_local_usage_cache(provider_ids: Vec return; } + let failure_provider_ids = provider_ids.clone(); if let Err(err) = tauri::async_runtime::spawn_blocking(move || { for provider_id in provider_ids { let summary = load_local_usage_summary(&provider_id, None); @@ -237,6 +242,9 @@ pub(crate) async fn refresh_provider_local_usage_cache(provider_ids: Vec .await { tracing::warn!("Provider local usage refresh worker failed: {err}"); + for provider_id in failure_provider_ids { + record_local_usage_fetch_failure(&provider_id, CostFetchFailure::Failed); + } } } @@ -255,7 +263,7 @@ fn load_local_usage_summary_cached( let cache = local_usage_cache(); if let Ok(guard) = cache.lock() && let Some(entry) = guard.get(provider_id) - && entry.loaded_at.elapsed() <= LOCAL_USAGE_TTL + && token_cost_cache_is_fresh(Some(entry.loaded_at), Instant::now(), LOCAL_USAGE_TTL) { return entry.summary.clone(); } @@ -285,6 +293,52 @@ fn store_local_usage_summary(provider_id: &str, summary: Option, + now: Instant, + ttl: Duration, +) -> bool { + loaded_at + .and_then(|loaded| now.checked_duration_since(loaded)) + .map(|age| age <= ttl) + .unwrap_or(false) +} + +pub(crate) fn cost_fetch_failure_allows_early_retry(failure: CostFetchFailure) -> bool { + !matches!(failure, CostFetchFailure::TimedOut) +} + +fn current_unix_ms() -> i64 { + SystemTime::now() + .duration_since(UNIX_EPOCH) + .map(|duration| duration.as_millis().min(i64::MAX as u128) as i64) + .unwrap_or(0) +} + fn localized_estimate_note(provider_id: &str, lang: codexbar::settings::Language) -> String { match provider_id { "claude" => locale::get_text(lang, LocaleKey::PanelEstimatedFromLocalLogsClaude), @@ -392,8 +446,58 @@ fn load_openai_dashboard_chart_data( #[cfg(test)] mod tests { - use super::localized_estimate_note; + use super::{ + CostFetchFailure, ProviderLocalUsageSummary, cost_fetch_failure_allows_early_retry, + localized_estimate_note, token_cost_cache_is_fresh, + }; + use crate::commands::is_provider_cache_fresh; use codexbar::settings::Language; + use std::time::{Duration, Instant}; + + #[test] + fn token_cost_age_does_not_use_provider_quota_age() { + let now = Instant::now(); + let token_loaded = now - Duration::from_secs(31); + let provider_updated = now; + assert!(!token_cost_cache_is_fresh( + Some(token_loaded), + now, + Duration::from_secs(30) + )); + assert!(is_provider_cache_fresh( + Some(provider_updated), + Duration::from_secs(30) + )); + } + + #[test] + fn fast_cost_failures_allow_the_next_pass_to_retry() { + assert!(cost_fetch_failure_allows_early_retry( + CostFetchFailure::Failed + )); + assert!(!cost_fetch_failure_allows_early_retry( + CostFetchFailure::TimedOut + )); + } + + #[test] + fn local_usage_summary_serializes_token_cost_timestamp() { + let summary = ProviderLocalUsageSummary { + today_cost: Some(1.0), + thirty_day_cost: Some(2.0), + thirty_day_tokens: Some(300), + latest_tokens: Some(40), + top_model: Some("gpt-5".to_string()), + estimate_note: "estimated".to_string(), + token_cost_updated_at_ms: 1234, + }; + + let json = serde_json::to_value(summary).expect("serialize summary"); + assert_eq!( + json.get("tokenCostUpdatedAtMs").and_then(|v| v.as_i64()), + Some(1234) + ); + } #[test] fn japanese_estimate_note_is_localized() { diff --git a/apps/desktop-tauri/src-tauri/src/powertoys.rs b/apps/desktop-tauri/src-tauri/src/powertoys.rs index f6993099a4..ae1c5aedde 100644 --- a/apps/desktop-tauri/src-tauri/src/powertoys.rs +++ b/apps/desktop-tauri/src-tauri/src/powertoys.rs @@ -221,6 +221,7 @@ mod tests { latest_tokens: Some(1_200), top_model: Some("gpt-5".to_string()), estimate_note: "cached".to_string(), + token_cost_updated_at_ms: 1234, }), ); diff --git a/apps/desktop-tauri/src/components/MenuCard.test.tsx b/apps/desktop-tauri/src/components/MenuCard.test.tsx index 502a13666d..c4b5e7ee48 100644 --- a/apps/desktop-tauri/src/components/MenuCard.test.tsx +++ b/apps/desktop-tauri/src/components/MenuCard.test.tsx @@ -124,6 +124,7 @@ describe("MenuCard", () => { latestTokens: null, topModel: "glim-4.6", estimateNote: "Estimated from local logs", + tokenCostUpdatedAtMs: 1234, }, }); eventMocks.listen.mockResolvedValue(() => {}); diff --git a/apps/desktop-tauri/src/floatbar/FloatBar.test.tsx b/apps/desktop-tauri/src/floatbar/FloatBar.test.tsx index cfed699510..5b402de8c4 100644 --- a/apps/desktop-tauri/src/floatbar/FloatBar.test.tsx +++ b/apps/desktop-tauri/src/floatbar/FloatBar.test.tsx @@ -207,6 +207,7 @@ describe("FloatBar", () => { latestTokens: 200, topModel: "gpt-5", estimateNote: "Estimated from local logs", + tokenCostUpdatedAtMs: 1234, }); renderFloatBar(bootstrap()); diff --git a/apps/desktop-tauri/src/surfaces/TrayPanel.test.tsx b/apps/desktop-tauri/src/surfaces/TrayPanel.test.tsx index 1d8663efa1..c6e1538016 100644 --- a/apps/desktop-tauri/src/surfaces/TrayPanel.test.tsx +++ b/apps/desktop-tauri/src/surfaces/TrayPanel.test.tsx @@ -343,6 +343,7 @@ describe("TrayPanel provider grid", () => { latestTokens: 1200, topModel: "gpt-5.5", estimateNote: "Estimated from local logs", + tokenCostUpdatedAtMs: 1234, }, }); diff --git a/apps/desktop-tauri/src/types/bridge.ts b/apps/desktop-tauri/src/types/bridge.ts index 2179f5de49..7fc63db442 100644 --- a/apps/desktop-tauri/src/types/bridge.ts +++ b/apps/desktop-tauri/src/types/bridge.ts @@ -442,6 +442,7 @@ export interface ProviderLocalUsageSummary { latestTokens: number | null; topModel: string | null; estimateNote: string; + tokenCostUpdatedAtMs: number; } export interface ProviderChartData { diff --git a/rust/src/codex_costs.rs b/rust/src/codex_costs.rs index e36f50251d..0888165c91 100644 --- a/rust/src/codex_costs.rs +++ b/rust/src/codex_costs.rs @@ -101,7 +101,14 @@ fn add_codex_tokens_to_summary( return None; } + let uses_fallback_pricing = + CostUsagePricing::codex_cost_usd(model, tokens.input, tokens.cached, tokens.output) + .is_none(); let cost = codex_cost_usd(model, tokens.input, tokens.cached, tokens.output); + if uses_fallback_pricing { + summary.unknown_models.insert(model.to_string()); + } + summary.input_tokens += tokens.input; summary.cached_tokens += tokens.cached; summary.output_tokens += tokens.output; @@ -239,6 +246,26 @@ mod tests { assert!((cost - 2.0).abs() < f64::EPSILON); } + #[test] + fn records_unknown_codex_model_while_using_fallback_cost() { + let target = NaiveDate::from_ymd_opt(2026, 5, 31).unwrap(); + let range = CostUsageDayRange::new(target, target); + let records = vec![CodexUsageRecord { + day_key: "2026-05-31".to_string(), + model: "gpt-mystery".to_string(), + input: 1_000_000, + cached: 0, + output: 1_000_000, + }]; + let mut summary = CostSummary::default(); + + let (cost, has_tokens) = add_codex_records_to_summary(&mut summary, &records, &range); + + assert!(has_tokens); + assert!(cost > 0.0); + assert!(summary.unknown_models.contains("gpt-mystery")); + } + #[test] fn test_codex_speed_bucket() { assert_eq!(codex_speed_bucket("gpt-5.5-fast"), "fast"); diff --git a/rust/src/cost_scanner.rs b/rust/src/cost_scanner.rs index 5e2d0b1217..7e55a0e356 100755 --- a/rust/src/cost_scanner.rs +++ b/rust/src/cost_scanner.rs @@ -41,6 +41,8 @@ pub struct CostSummary { pub by_speed: HashMap, /// Codex token split by speed/tier when local logs expose it. pub by_speed_tokens: HashMap, + /// Model IDs that were priced with fallback rates because no canonical rate is available. + pub unknown_models: HashSet, /// Period start date pub period_start: Option, /// Period end date @@ -524,6 +526,10 @@ fn should_count_claude_record( } fn add_claude_record_to_summary(summary: &mut CostSummary, record: &ClaudeUsageRecord) { + if CostUsagePricing::claude_cost_usd(&record.model, 0, 0, 0, 0).is_none() { + summary.unknown_models.insert(record.model.clone()); + } + summary.input_tokens += record.input; summary.output_tokens += record.output; summary.cached_tokens += record.cache_create + record.cache_read; @@ -656,6 +662,21 @@ mod tests { assert!((cost - 1.80).abs() < 0.001); } + #[test] + fn records_unknown_claude_model_while_using_fallback_cost() { + let event: ClaudeEvent = serde_json::from_str( + r#"{"type":"assistant","timestamp":"2026-01-15T10:00:00Z","requestId":"req_unknown","message":{"id":"msg_unknown","model":"claude-retired-unknown","usage":{"input_tokens":100000,"output_tokens":100000}}}"#, + ) + .unwrap(); + let record = claude_usage_record_from_event(&event).expect("usage record"); + let mut summary = CostSummary::default(); + + add_claude_record_to_summary(&mut summary, &record); + + assert!(summary.total_cost_usd > 0.0); + assert!(summary.unknown_models.contains("claude-retired-unknown")); + } + #[test] fn test_claude_fable_5_pricing() { let cost = ClaudePricing::cost_usd_with_cache_ttl("claude-fable-5", 100, 10, 0, 20, 5); From 8fce49ad2d6692a3198256d5a52015809451655b Mon Sep 17 00:00:00 2001 From: NessZerra Date: Sun, 12 Jul 2026 08:20:51 +0700 Subject: [PATCH 4/5] Refresh unknown model pricing Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- .../src-tauri/src/commands/chart.rs | 87 +- rust/src/core/cost_pricing.rs | 227 +++-- rust/src/core/mod.rs | 2 + rust/src/core/models_dev_pricing.rs | 817 ++++++++++++++++++ 4 files changed, 1050 insertions(+), 83 deletions(-) create mode 100644 rust/src/core/models_dev_pricing.rs diff --git a/apps/desktop-tauri/src-tauri/src/commands/chart.rs b/apps/desktop-tauri/src-tauri/src/commands/chart.rs index a2262d7e6b..f3ff4d2188 100644 --- a/apps/desktop-tauri/src-tauri/src/commands/chart.rs +++ b/apps/desktop-tauri/src-tauri/src/commands/chart.rs @@ -9,7 +9,7 @@ use codexbar::core::OpenAIDashboardCacheStore; use codexbar::cost_scanner::{CostScanner, CostSummary, get_daily_cost_history}; use codexbar::locale::{self, LocaleKey}; use serde::{Deserialize, Serialize}; -use std::collections::HashMap; +use std::collections::{HashMap, HashSet}; use std::sync::{ Arc, Mutex, OnceLock, atomic::{AtomicBool, Ordering}, @@ -168,30 +168,47 @@ fn load_local_usage_summary( provider_id: &str, cancel: Option<&AtomicBool>, ) -> Option { - let thirty_day = scan_local_cost(provider_id, 30, cancel)?; + load_local_usage_summary_with_unknown_models(provider_id, cancel).0 +} + +fn load_local_usage_summary_with_unknown_models( + provider_id: &str, + cancel: Option<&AtomicBool>, +) -> (Option, HashSet) { + let Some(thirty_day) = scan_local_cost(provider_id, 30, cancel) else { + return (None, HashSet::new()); + }; if cancel.is_some_and(|flag| flag.load(Ordering::Relaxed)) { - return None; + return (None, HashSet::new()); } let today = scan_local_cost(provider_id, 1, cancel).unwrap_or_default(); + let unknown_models = thirty_day + .unknown_models + .union(&today.unknown_models) + .cloned() + .collect(); let thirty_day_tokens = total_tokens(&thirty_day); let latest_tokens = total_tokens(&today); let has_usage = thirty_day.sessions_count > 0 || thirty_day.total_cost_usd > 0.0 || thirty_day_tokens > 0; if !has_usage { - return None; + return (None, unknown_models); } let lang = locale::current_language(); - Some(ProviderLocalUsageSummary { - today_cost: non_zero_f64(today.total_cost_usd), - thirty_day_cost: non_zero_f64(thirty_day.total_cost_usd), - thirty_day_tokens: non_zero_u64(thirty_day_tokens), - latest_tokens: non_zero_u64(latest_tokens), - top_model: top_model(&thirty_day), - estimate_note: localized_estimate_note(provider_id, lang), - token_cost_updated_at_ms: current_unix_ms(), - }) + ( + Some(ProviderLocalUsageSummary { + today_cost: non_zero_f64(today.total_cost_usd), + thirty_day_cost: non_zero_f64(thirty_day.total_cost_usd), + thirty_day_tokens: non_zero_u64(thirty_day_tokens), + latest_tokens: non_zero_u64(latest_tokens), + top_model: top_model(&thirty_day), + estimate_note: localized_estimate_note(provider_id, lang), + token_cost_updated_at_ms: current_unix_ms(), + }), + unknown_models, + ) } pub(crate) fn load_provider_local_usage_summary( @@ -233,18 +250,46 @@ pub(crate) async fn refresh_provider_local_usage_cache(provider_ids: Vec } let failure_provider_ids = provider_ids.clone(); - if let Err(err) = tauri::async_runtime::spawn_blocking(move || { - for provider_id in provider_ids { - let summary = load_local_usage_summary(&provider_id, None); - store_local_usage_summary(&provider_id, summary); - } + let scans = match tauri::async_runtime::spawn_blocking(move || { + provider_ids + .into_iter() + .map(|provider_id| { + let (summary, unknown_models) = + load_local_usage_summary_with_unknown_models(&provider_id, None); + (provider_id, summary, unknown_models) + }) + .collect::>() }) .await { - tracing::warn!("Provider local usage refresh worker failed: {err}"); - for provider_id in failure_provider_ids { - record_local_usage_fetch_failure(&provider_id, CostFetchFailure::Failed); + Ok(scans) => scans, + Err(err) => { + tracing::warn!("Provider local usage refresh worker failed: {err}"); + for provider_id in failure_provider_ids { + record_local_usage_fetch_failure(&provider_id, CostFetchFailure::Failed); + } + return; + } + }; + + for (provider_id, mut summary, unknown_models) in scans { + let pricing_provider = match provider_id.as_str() { + "codex" => Some("openai"), + "claude" => Some("anthropic"), + _ => None, + }; + if let Some(pricing_provider) = pricing_provider + && codexbar::core::refresh_unknown_models_if_needed(pricing_provider, &unknown_models) + .await + { + let rescan_provider = provider_id.clone(); + summary = tauri::async_runtime::spawn_blocking(move || { + load_local_usage_summary(&rescan_provider, None) + }) + .await + .unwrap_or(summary); } + store_local_usage_summary(&provider_id, summary); } } diff --git a/rust/src/core/cost_pricing.rs b/rust/src/core/cost_pricing.rs index 39e7eb29c3..b03174b8b5 100755 --- a/rust/src/core/cost_pricing.rs +++ b/rust/src/core/cost_pricing.rs @@ -5,6 +5,7 @@ #![allow(dead_code)] +use super::models_dev_pricing; use std::collections::HashMap; use std::sync::LazyLock; @@ -582,6 +583,21 @@ static CLAUDE_PRICING: LazyLock> = LazyLock m }); +fn codex_cost_from_rates( + input_tokens: u64, + cached_input_tokens: u64, + output_tokens: u64, + input_rate: f64, + cache_read_rate: f64, + output_rate: f64, +) -> f64 { + let cached = cached_input_tokens.min(input_tokens); + let non_cached = input_tokens.saturating_sub(cached); + (non_cached as f64) * input_rate + + (cached as f64) * cache_read_rate + + (output_tokens as f64) * output_rate +} + /// Cost usage pricing utilities pub struct CostUsagePricing; @@ -669,38 +685,72 @@ impl CostUsagePricing { output_tokens: u64, ) -> Option { let key = Self::normalize_codex_model(model); - let pricing = CODEX_PRICING.get(key.as_str())?; - let (input_rate, cache_read_rate, output_rate) = - if input_tokens > CODEX_LONG_CONTEXT_THRESHOLD { - if let Some(long_context) = pricing.long_context { - ( - long_context.input_cost_per_token, - long_context.cache_read_input_cost_per_token, - long_context.output_cost_per_token, - ) + if let Some(pricing) = CODEX_PRICING.get(key.as_str()) { + let (input_rate, cache_read_rate, output_rate) = + if input_tokens > CODEX_LONG_CONTEXT_THRESHOLD { + if let Some(long_context) = pricing.long_context { + ( + long_context.input_cost_per_token, + long_context.cache_read_input_cost_per_token, + long_context.output_cost_per_token, + ) + } else { + ( + pricing.input_cost_per_token, + pricing.cache_read_input_cost_per_token, + pricing.output_cost_per_token, + ) + } } else { ( pricing.input_cost_per_token, pricing.cache_read_input_cost_per_token, pricing.output_cost_per_token, ) - } - } else { - ( - pricing.input_cost_per_token, - pricing.cache_read_input_cost_per_token, - pricing.output_cost_per_token, - ) - }; - - let cached = cached_input_tokens.min(input_tokens); - let non_cached = input_tokens.saturating_sub(cached); - - let cost = (non_cached as f64) * input_rate - + (cached as f64) * cache_read_rate - + (output_tokens as f64) * output_rate; + }; + return Some(codex_cost_from_rates( + input_tokens, + cached_input_tokens, + output_tokens, + input_rate, + cache_read_rate, + output_rate, + )); + } - Some(cost) + let pricing = models_dev_pricing::lookup("openai", model)?; + let use_tier = pricing + .threshold_tokens + .is_some_and(|threshold| input_tokens > threshold); + Some(codex_cost_from_rates( + input_tokens, + cached_input_tokens, + output_tokens, + if use_tier { + pricing + .input_cost_per_token_above_threshold + .unwrap_or(pricing.input_cost_per_token) + } else { + pricing.input_cost_per_token + }, + if use_tier { + pricing + .cache_read_input_cost_per_token_above_threshold + .or(pricing.cache_read_input_cost_per_token) + .unwrap_or(pricing.input_cost_per_token) + } else { + pricing + .cache_read_input_cost_per_token + .unwrap_or(pricing.input_cost_per_token) + }, + if use_tier { + pricing + .output_cost_per_token_above_threshold + .unwrap_or(pricing.output_cost_per_token) + } else { + pricing.output_cost_per_token + }, + )) } /// Calculate cost for Claude usage in USD @@ -712,44 +762,96 @@ impl CostUsagePricing { output_tokens: i32, ) -> Option { let key = Self::normalize_claude_model(model); - let pricing = CLAUDE_PRICING.get(key.as_str())?; - - /// Calculate tiered cost - fn tiered(tokens: i32, base: f64, above: Option, threshold: Option) -> f64 { - let tokens = tokens.max(0); - match (threshold, above) { - (Some(thresh), Some(above_rate)) => { - let below = tokens.min(thresh); - let over = (tokens - thresh).max(0); - (below as f64) * base + (over as f64) * above_rate + if let Some(pricing) = CLAUDE_PRICING.get(key.as_str()) { + /// Calculate tiered cost + fn tiered(tokens: i32, base: f64, above: Option, threshold: Option) -> f64 { + let tokens = tokens.max(0); + match (threshold, above) { + (Some(thresh), Some(above_rate)) => { + let below = tokens.min(thresh); + let over = (tokens - thresh).max(0); + (below as f64) * base + (over as f64) * above_rate + } + _ => (tokens as f64) * base, } - _ => (tokens as f64) * base, } - } - let cost = tiered( - input_tokens, - pricing.input_cost_per_token, - pricing.input_cost_per_token_above_threshold, - pricing.threshold_tokens, - ) + tiered( - cache_read_input_tokens, - pricing.cache_read_input_cost_per_token, - pricing.cache_read_input_cost_per_token_above_threshold, - pricing.threshold_tokens, - ) + tiered( - cache_creation_input_tokens, - pricing.cache_creation_input_cost_per_token, - pricing.cache_creation_input_cost_per_token_above_threshold, - pricing.threshold_tokens, - ) + tiered( - output_tokens, - pricing.output_cost_per_token, - pricing.output_cost_per_token_above_threshold, - pricing.threshold_tokens, - ); + let cost = tiered( + input_tokens, + pricing.input_cost_per_token, + pricing.input_cost_per_token_above_threshold, + pricing.threshold_tokens, + ) + tiered( + cache_read_input_tokens, + pricing.cache_read_input_cost_per_token, + pricing.cache_read_input_cost_per_token_above_threshold, + pricing.threshold_tokens, + ) + tiered( + cache_creation_input_tokens, + pricing.cache_creation_input_cost_per_token, + pricing.cache_creation_input_cost_per_token_above_threshold, + pricing.threshold_tokens, + ) + tiered( + output_tokens, + pricing.output_cost_per_token, + pricing.output_cost_per_token_above_threshold, + pricing.threshold_tokens, + ); - Some(cost) + return Some(cost); + } + + let pricing = models_dev_pricing::lookup("anthropic", model)?; + let input_tokens = input_tokens.max(0); + let cache_read_input_tokens = cache_read_input_tokens.max(0); + let cache_creation_input_tokens = cache_creation_input_tokens.max(0); + let output_tokens = output_tokens.max(0); + let use_tier = pricing.threshold_tokens.is_some_and(|threshold| { + (input_tokens as u64) + + (cache_read_input_tokens as u64) + + (cache_creation_input_tokens as u64) + > threshold + }); + let input_rate = if use_tier { + pricing + .input_cost_per_token_above_threshold + .unwrap_or(pricing.input_cost_per_token) + } else { + pricing.input_cost_per_token + }; + let cache_read_rate = if use_tier { + pricing + .cache_read_input_cost_per_token_above_threshold + .or(pricing.cache_read_input_cost_per_token) + .unwrap_or(input_rate) + } else { + pricing + .cache_read_input_cost_per_token + .unwrap_or(input_rate) + }; + let cache_write_rate = if use_tier { + pricing + .cache_write_input_cost_per_token_above_threshold + .or(pricing.cache_write_input_cost_per_token) + .unwrap_or(input_rate) + } else { + pricing + .cache_write_input_cost_per_token + .unwrap_or(input_rate) + }; + let output_rate = if use_tier { + pricing + .output_cost_per_token_above_threshold + .unwrap_or(pricing.output_cost_per_token) + } else { + pricing.output_cost_per_token + }; + Some( + (input_tokens as f64) * input_rate + + (cache_read_input_tokens as f64) * cache_read_rate + + (cache_creation_input_tokens as f64) * cache_write_rate + + (output_tokens as f64) * output_rate, + ) } /// Base per-token input rate for a Claude model. Exposed for callers that @@ -757,9 +859,10 @@ impl CostUsagePricing { /// scanner's one-hour cache-write premium, billed at 2x the input rate. pub fn claude_input_cost_per_token(model: &str) -> Option { let key = Self::normalize_claude_model(model); - CLAUDE_PRICING - .get(key.as_str()) - .map(|p| p.input_cost_per_token) + if let Some(pricing) = CLAUDE_PRICING.get(key.as_str()) { + return Some(pricing.input_cost_per_token); + } + models_dev_pricing::lookup("anthropic", model).map(|p| p.input_cost_per_token) } /// Format model name for display (e.g., "claude-3.5-sonnet" → "Sonnet 3.5") diff --git a/rust/src/core/mod.rs b/rust/src/core/mod.rs index 41546a5473..d440e93717 100755 --- a/rust/src/core/mod.rs +++ b/rust/src/core/mod.rs @@ -5,6 +5,7 @@ mod credential_migration; mod credentials; mod http; mod jsonl_scanner; +mod models_dev_pricing; mod openai_dashboard; mod provider; mod provider_factory; @@ -21,6 +22,7 @@ pub use credential_migration::*; pub use credentials::*; pub use http::*; pub use jsonl_scanner::*; +pub use models_dev_pricing::*; pub use openai_dashboard::*; pub use provider::*; pub use provider_factory::instantiate as instantiate_provider; diff --git a/rust/src/core/models_dev_pricing.rs b/rust/src/core/models_dev_pricing.rs new file mode 100644 index 0000000000..dc1be873b2 --- /dev/null +++ b/rust/src/core/models_dev_pricing.rs @@ -0,0 +1,817 @@ +#[cfg(test)] +mod tests { + use super::{ + ModelsDevCache, ModelsDevCacheArtifact, ModelsDevCatalog, ModelsDevRefreshCoordinator, + }; + use std::path::PathBuf; + use std::sync::Arc; + use std::sync::atomic::{AtomicUsize, Ordering}; + use std::time::{Duration, UNIX_EPOCH}; + + #[test] + fn decodes_top_level_provider_map_and_converts_million_token_rates() { + let catalog = ModelsDevCatalog::decode( + r#"{ + "openai": { + "id": "openai", + "models": { + "openai/gpt-fresh": { + "id": "openai/gpt-fresh", + "cost": { + "input": 2.5, + "output": 10, + "cache_read": 0.25, + "cache_write": 3.75, + "context_over_200k": { + "input": 5, + "output": 15, + "cache_read": 0.5, + "cache_write": 7.5 + } + } + } + } + }, + "anthropic": { + "models": { + "claude-fresh": { + "id": "claude-fresh", + "cost": { "input": 3, "output": 15 } + } + } + } + }"#, + ) + .expect("top-level catalog"); + + let pricing = catalog.lookup("openai", "gpt-fresh").expect("pricing"); + assert_eq!(pricing.input_cost_per_token, 2.5e-6); + assert_eq!(pricing.output_cost_per_token, 10e-6); + assert_eq!(pricing.cache_read_input_cost_per_token, Some(0.25e-6)); + assert_eq!(pricing.cache_write_input_cost_per_token, Some(3.75e-6)); + assert_eq!(pricing.threshold_tokens, Some(200_000)); + assert_eq!(pricing.input_cost_per_token_above_threshold, Some(5e-6)); + } + + #[test] + fn decodes_providers_envelope() { + let catalog = ModelsDevCatalog::decode( + r#"{ + "providers": { + "anthropic": { + "id": "anthropic", + "models": { + "claude-fresh": { + "id": "claude-fresh", + "cost": { "input": 3, "output": 15 } + } + } + }, + "openai": { + "models": { + "gpt-fresh": { + "id": "gpt-fresh", + "cost": { "input": 2.5, "output": 10 } + } + } + } + } + }"#, + ) + .expect("enveloped catalog"); + + assert_eq!( + catalog + .lookup("anthropic", "claude-fresh") + .expect("pricing") + .output_cost_per_token, + 15e-6 + ); + } + + #[test] + fn cache_artifact_is_versioned_and_expires_after_one_day() { + let catalog = ModelsDevCatalog::decode( + r#"{ + "openai": { + "models": { + "gpt-fresh": { + "id": "gpt-fresh", + "cost": { "input": 2.5, "output": 10 } + } + } + }, + "anthropic": { + "models": { + "claude-fresh": { + "id": "claude-fresh", + "cost": { "input": 3, "output": 15 } + } + } + } + }"#, + ) + .expect("catalog"); + let fetched_at = UNIX_EPOCH + Duration::from_secs(1_000_000); + let artifact = ModelsDevCacheArtifact::new(catalog, fetched_at); + + assert_eq!(artifact.version, ModelsDevCache::ARTIFACT_VERSION); + assert!(!artifact.is_stale(fetched_at + Duration::from_secs(86_400))); + assert!(artifact.is_stale(fetched_at + Duration::from_secs(86_401))); + assert_eq!( + ModelsDevCache::cache_path(Some(PathBuf::from("cache-root").as_path())), + PathBuf::from("cache-root") + .join("model-pricing") + .join("models-dev-v1.json") + ); + } + + #[tokio::test] + async fn concurrent_refreshes_for_one_cache_path_share_one_operation() { + let coordinator = ModelsDevRefreshCoordinator::default(); + let calls = Arc::new(AtomicUsize::new(0)); + let first_calls = Arc::clone(&calls); + let path = PathBuf::from("pricing.json"); + let now = UNIX_EPOCH + Duration::from_secs(1_000_000); + + let first = coordinator.refresh(path.clone(), now, async move { + first_calls.fetch_add(1, Ordering::SeqCst); + tokio::time::sleep(Duration::from_millis(10)).await; + true + }); + let second = coordinator.refresh(path, now, async { + panic!("the second caller must await the first operation"); + }); + + assert!(tokio::join!(first, second).0); + assert_eq!(calls.load(Ordering::SeqCst), 1); + } + + #[tokio::test] + async fn failed_refresh_is_not_retried_within_the_attempt_window() { + let coordinator = ModelsDevRefreshCoordinator::default(); + let path = PathBuf::from("pricing.json"); + let now = UNIX_EPOCH + Duration::from_secs(1_000_000); + + assert!( + !coordinator + .refresh(path.clone(), now, async { false }) + .await + ); + assert!( + !coordinator + .refresh(path, now + Duration::from_secs(60), async { + panic!("the 15-minute bound must suppress this attempt"); + }) + .await + ); + } + + #[test] + fn cache_path_uses_the_existing_per_user_cache_root() { + let cache_root = ModelsDevCache::default_cache_root().expect("per-user cache root"); + assert_eq!( + ModelsDevCache::cache_path(None), + cache_root.join("model-pricing").join("models-dev-v1.json") + ); + } +} + +use serde::{Deserialize, Deserializer, Serialize}; +use std::collections::{HashMap, HashSet}; +use std::fs; +use std::future::Future; +use std::path::{Path, PathBuf}; +use std::sync::{Arc, LazyLock, Mutex}; +use std::time::{Duration, SystemTime, UNIX_EPOCH}; +use tokio::sync::{Mutex as AsyncMutex, watch}; + +const MODELS_DEV_URL: &str = "https://models.dev/api.json"; +const CACHE_TTL: Duration = Duration::from_secs(24 * 60 * 60); +const REFRESH_ATTEMPT_WINDOW: Duration = Duration::from_secs(15 * 60); + +/// Per-token pricing decoded from the models.dev catalog. +#[derive(Debug, Clone, Copy, PartialEq)] +pub struct DynamicModelPricing { + pub input_cost_per_token: f64, + pub output_cost_per_token: f64, + pub cache_read_input_cost_per_token: Option, + pub cache_write_input_cost_per_token: Option, + pub threshold_tokens: Option, + pub input_cost_per_token_above_threshold: Option, + pub output_cost_per_token_above_threshold: Option, + pub cache_read_input_cost_per_token_above_threshold: Option, + pub cache_write_input_cost_per_token_above_threshold: Option, +} + +#[derive(Debug, Clone, Serialize)] +struct ModelsDevCatalog { + providers: HashMap, +} + +#[derive(Debug, Deserialize)] +#[serde(untagged)] +enum ModelsDevCatalogWire { + Envelope { + providers: HashMap, + }, + ProviderMap(HashMap), +} + +impl<'de> Deserialize<'de> for ModelsDevCatalog { + fn deserialize(deserializer: D) -> Result + where + D: Deserializer<'de>, + { + let providers = match ModelsDevCatalogWire::deserialize(deserializer)? { + ModelsDevCatalogWire::Envelope { providers } + | ModelsDevCatalogWire::ProviderMap(providers) => providers, + }; + Ok(Self { + providers: providers + .into_iter() + .map(|(key, provider)| { + ( + normalize_provider_id(provider.id.as_deref().unwrap_or(&key)), + provider, + ) + }) + .collect(), + }) + } +} + +impl ModelsDevCatalog { + #[cfg(test)] + fn decode(json: &str) -> Option { + serde_json::from_str(json).ok() + } + + fn lookup(&self, provider_id: &str, model_id: &str) -> Option { + let provider = self.providers.get(&normalize_provider_id(provider_id))?; + let candidates = model_id_candidates(model_id); + for candidate in &candidates { + if let Some(model) = provider.models.get(candidate) + && let Some(pricing) = DynamicModelPricing::from_model(model) + { + return Some(pricing); + } + } + provider.models.values().find_map(|model| { + let model_candidates = model_id_candidates(&model.id); + candidates + .iter() + .any(|candidate| model_candidates.contains(candidate)) + .then(|| DynamicModelPricing::from_model(model)) + .flatten() + }) + } + + fn is_plausible_refresh(&self) -> bool { + ["openai", "anthropic"].into_iter().all(|provider_id| { + self.providers + .get(provider_id) + .is_some_and(|provider| provider.models.values().any(ModelsDevModel::is_priceable)) + }) + } + + fn merge_priceable_entries_from(&mut self, cached: &Self) { + for (provider_id, cached_provider) in &cached.providers { + let provider = self + .providers + .entry(provider_id.clone()) + .or_insert_with(|| cached_provider.clone()); + let present_ids: HashSet = provider + .models + .values() + .filter(|model| model.is_priceable()) + .map(|model| stable_model_identity(&model.id)) + .collect(); + for (model_key, cached_model) in &cached_provider.models { + if !cached_model.is_priceable() + || present_ids.contains(&stable_model_identity(&cached_model.id)) + { + continue; + } + let mut fallback_key = model_key.clone(); + if provider.models.contains_key(&fallback_key) { + fallback_key = format!( + "codexbar-fallback:{model_key}:{}", + normalize_model_id(&cached_model.id) + ); + } + provider.models.insert(fallback_key, cached_model.clone()); + } + } + } +} + +#[derive(Debug, Clone, Serialize, Deserialize)] +struct ModelsDevProvider { + id: Option, + #[serde(default)] + models: HashMap, +} + +#[derive(Debug, Clone, Serialize, Deserialize)] +struct ModelsDevModel { + id: String, + cost: Option, +} + +impl ModelsDevModel { + fn is_priceable(&self) -> bool { + self.cost.as_ref().is_some_and(|cost| { + cost.input.is_some_and(|rate| valid_number(&rate)) + && cost.output.is_some_and(|rate| valid_number(&rate)) + }) + } +} + +#[derive(Debug, Clone, Serialize, Deserialize)] +struct ModelsDevCost { + input: Option, + output: Option, + #[serde(rename = "cache_read")] + cache_read: Option, + #[serde(rename = "cache_write")] + cache_write: Option, + #[serde(rename = "context_over_200k")] + context_over_200k: Option, +} + +#[derive(Debug, Clone, Serialize, Deserialize)] +struct ModelsDevContextCost { + input: Option, + output: Option, + #[serde(rename = "cache_read")] + cache_read: Option, + #[serde(rename = "cache_write")] + cache_write: Option, +} + +impl DynamicModelPricing { + fn from_model(model: &ModelsDevModel) -> Option { + let cost = model.cost.as_ref()?; + let input = cost.input.filter(valid_number)?; + let output = cost.output.filter(valid_number)?; + let above = cost.context_over_200k.as_ref(); + Some(Self { + input_cost_per_token: per_token(input), + output_cost_per_token: per_token(output), + cache_read_input_cost_per_token: cost.cache_read.filter(valid_number).map(per_token), + cache_write_input_cost_per_token: cost.cache_write.filter(valid_number).map(per_token), + threshold_tokens: above.is_some().then_some(200_000), + input_cost_per_token_above_threshold: above + .and_then(|cost| cost.input) + .filter(valid_number) + .map(per_token), + output_cost_per_token_above_threshold: above + .and_then(|cost| cost.output) + .filter(valid_number) + .map(per_token), + cache_read_input_cost_per_token_above_threshold: above + .and_then(|cost| cost.cache_read) + .filter(valid_number) + .map(per_token), + cache_write_input_cost_per_token_above_threshold: above + .and_then(|cost| cost.cache_write) + .filter(valid_number) + .map(per_token), + }) + } +} + +fn valid_number(rate: &f64) -> bool { + rate.is_finite() && *rate >= 0.0 +} + +fn per_token(rate: f64) -> f64 { + rate / 1_000_000.0 +} + +fn normalize_provider_id(provider_id: &str) -> String { + provider_id.trim().to_ascii_lowercase() +} + +fn normalize_model_id(model_id: &str) -> String { + model_id.trim().to_string() +} + +fn stable_model_identity(model_id: &str) -> String { + let model_id = normalize_model_id(model_id); + if let Some((base, suffix)) = model_id.split_once('@') { + if suffix == "default" { + return base.to_string(); + } + if suffix.len() == 8 && suffix.bytes().all(|byte| byte.is_ascii_digit()) { + return format!("{base}-{suffix}"); + } + } + model_id +} + +fn model_id_candidates(model_id: &str) -> Vec { + let mut candidates = Vec::new(); + append_model_candidate(&mut candidates, model_id.to_string()); + let mut index = 0; + while index < candidates.len() { + let candidate = candidates[index].clone(); + if let Some(rest) = candidate.strip_prefix("openai/") { + append_model_candidate(&mut candidates, rest.to_string()); + } + if let Some(rest) = candidate.strip_prefix("anthropic.") { + append_model_candidate(&mut candidates, rest.to_string()); + } + if candidate.contains("claude-") + && let Some((_, tail)) = candidate.rsplit_once('.') + && tail.starts_with("claude-") + { + append_model_candidate(&mut candidates, tail.to_string()); + } + if let Some((base, suffix)) = candidate.split_once('@') { + if suffix.len() == 8 && suffix.bytes().all(|byte| byte.is_ascii_digit()) { + append_model_candidate(&mut candidates, format!("{base}-{suffix}")); + } + append_model_candidate(&mut candidates, base.to_string()); + } else if candidate.starts_with("claude-") { + append_model_candidate(&mut candidates, format!("{candidate}@default")); + } + if let Some(base) = candidate.strip_suffix("-v1:0") { + append_model_candidate(&mut candidates, base.to_string()); + } + if let Some(base) = strip_date_suffix(&candidate) { + append_model_candidate(&mut candidates, base.to_string()); + } + index += 1; + } + candidates +} + +fn append_model_candidate(candidates: &mut Vec, candidate: String) { + let candidate = normalize_model_id(&candidate); + if !candidate.is_empty() && !candidates.contains(&candidate) { + candidates.push(candidate); + } +} + +fn strip_date_suffix(model_id: &str) -> Option<&str> { + let suffix = model_id.rsplit_once('-')?.1; + if suffix.len() == 8 && suffix.bytes().all(|byte| byte.is_ascii_digit()) { + return Some(&model_id[..model_id.len() - suffix.len() - 1]); + } + if suffix.len() != 2 || !suffix.bytes().all(|byte| byte.is_ascii_digit()) { + return None; + } + let without_day = &model_id[..model_id.len() - 3]; + let month = without_day.rsplit_once('-')?.1; + if month.len() != 2 || !month.bytes().all(|byte| byte.is_ascii_digit()) { + return None; + } + let without_month = &without_day[..without_day.len() - 3]; + let year = without_month.rsplit_once('-')?.1; + if year.len() == 4 && year.bytes().all(|byte| byte.is_ascii_digit()) { + Some(&without_month[..without_month.len() - 5]) + } else { + None + } +} + +#[derive(Debug, Clone, Serialize, Deserialize)] +struct ModelsDevCacheArtifact { + version: u32, + fetched_at_unix_ms: u64, + catalog: ModelsDevCatalog, +} + +impl ModelsDevCacheArtifact { + fn new(catalog: ModelsDevCatalog, fetched_at: SystemTime) -> Self { + Self { + version: ModelsDevCache::ARTIFACT_VERSION, + fetched_at_unix_ms: unix_ms(fetched_at), + catalog, + } + } + + fn fetched_at(&self) -> SystemTime { + UNIX_EPOCH + Duration::from_millis(self.fetched_at_unix_ms) + } + + fn is_stale(&self, now: SystemTime) -> bool { + now.duration_since(self.fetched_at()).unwrap_or_default() > CACHE_TTL + } +} + +struct ModelsDevCacheLoad { + artifact: Option>, + is_stale: bool, +} + +struct ModelsDevCacheMemoEntry { + modified_at: Option, + size: Option, + artifact: Option>, +} + +static CACHE_MEMO: LazyLock>> = + LazyLock::new(|| Mutex::new(HashMap::new())); + +struct ModelsDevCache; + +impl ModelsDevCache { + const ARTIFACT_VERSION: u32 = 1; + + fn default_cache_root() -> Option { + dirs::cache_dir().map(|path| path.join("CodexBar")) + } + + fn cache_path(cache_root: Option<&Path>) -> PathBuf { + cache_root + .map(Path::to_path_buf) + .or_else(Self::default_cache_root) + .map(|root| { + root.join("model-pricing") + .join(format!("models-dev-v{}.json", Self::ARTIFACT_VERSION)) + }) + .unwrap_or_default() + } +} + +fn unix_ms(time: SystemTime) -> u64 { + time.duration_since(UNIX_EPOCH) + .unwrap_or_default() + .as_millis() + .try_into() + .unwrap_or(u64::MAX) +} + +impl ModelsDevCache { + fn load(now: SystemTime, cache_root: Option<&Path>) -> ModelsDevCacheLoad { + let cache_path = Self::cache_path(cache_root); + if cache_path.as_os_str().is_empty() { + return ModelsDevCacheLoad { + artifact: None, + is_stale: true, + }; + } + let cache_path = standardized_cache_path(&cache_path); + let (modified_at, size) = file_identity(&cache_path); + let memoized = { + let memo = CACHE_MEMO.lock().expect("models.dev cache memo lock"); + memo.get(&cache_path) + .filter(|entry| entry.modified_at == modified_at && entry.size == size) + .map(|entry| entry.artifact.clone()) + }; + let artifact = memoized.unwrap_or_else(|| { + let artifact = fs::read(&cache_path) + .ok() + .and_then(|contents| { + serde_json::from_slice::(&contents).ok() + }) + .filter(|artifact| artifact.version == Self::ARTIFACT_VERSION) + .map(Arc::new); + CACHE_MEMO + .lock() + .expect("models.dev cache memo lock") + .insert( + cache_path, + ModelsDevCacheMemoEntry { + modified_at, + size, + artifact: artifact.clone(), + }, + ); + artifact + }); + let is_stale = artifact + .as_ref() + .is_none_or(|artifact| artifact.is_stale(now)); + ModelsDevCacheLoad { artifact, is_stale } + } +} + +fn file_identity(path: &Path) -> (Option, Option) { + let Ok(metadata) = fs::metadata(path) else { + return (None, None); + }; + (metadata.modified().ok(), Some(metadata.len())) +} + +fn standardized_cache_path(path: &Path) -> PathBuf { + fs::canonicalize(path).unwrap_or_else(|_| { + if path.is_absolute() { + path.to_path_buf() + } else { + std::env::current_dir() + .map(|current_dir| current_dir.join(path)) + .unwrap_or_else(|_| path.to_path_buf()) + } + }) +} + +impl ModelsDevCache { + fn save(catalog: ModelsDevCatalog, fetched_at: SystemTime, cache_root: Option<&Path>) -> bool { + let cache_path = Self::cache_path(cache_root); + if cache_path.as_os_str().is_empty() { + return false; + } + let cache_path = standardized_cache_path(&cache_path); + let Some(parent) = cache_path.parent() else { + return false; + }; + if fs::create_dir_all(parent).is_err() { + return false; + } + let artifact = Arc::new(ModelsDevCacheArtifact::new(catalog, fetched_at)); + let Ok(contents) = serde_json::to_vec(&*artifact) else { + return false; + }; + let Ok(mut file) = std::fs::File::create(&cache_path) else { + return false; + }; + if std::io::Write::write_all(&mut file, &contents).is_err() { + return false; + } + let (modified_at, size) = file_identity(&cache_path); + CACHE_MEMO + .lock() + .expect("models.dev cache memo lock") + .insert( + cache_path, + ModelsDevCacheMemoEntry { + modified_at, + size, + artifact: Some(artifact), + }, + ); + true + } +} + +#[derive(Default)] +struct ModelsDevRefreshCoordinator { + state: Arc>, +} + +#[derive(Default)] +struct ModelsDevRefreshState { + in_flight: HashMap>>, + last_attempt: HashMap, +} + +impl ModelsDevRefreshCoordinator { + async fn refresh(&self, cache_path: PathBuf, now: SystemTime, operation: F) -> bool + where + F: Future + Send + 'static, + { + let cache_path = standardized_cache_path(&cache_path); + let mut state = self.state.lock().await; + if let Some(in_flight) = state.in_flight.get(&cache_path) { + let receiver = in_flight.clone(); + drop(state); + return wait_for_refresh(receiver).await; + } + if state + .last_attempt + .get(&cache_path) + .is_some_and(|last_attempt| { + now.duration_since(*last_attempt).unwrap_or_default() < REFRESH_ATTEMPT_WINDOW + }) + { + return false; + } + + state.last_attempt.insert(cache_path.clone(), now); + let (sender, receiver) = watch::channel(None); + state.in_flight.insert(cache_path.clone(), receiver.clone()); + drop(state); + + let state = Arc::clone(&self.state); + tokio::spawn(async move { + let result = operation.await; + let _ = sender.send(Some(result)); + state + .lock() + .await + .in_flight + .retain(|path, _| path != &cache_path); + }); + wait_for_refresh(receiver).await + } +} + +async fn wait_for_refresh(mut receiver: watch::Receiver>) -> bool { + loop { + if let Some(result) = *receiver.borrow() { + return result; + } + if receiver.changed().await.is_err() { + return false; + } + } +} + +static REFRESH_COORDINATOR: LazyLock = + LazyLock::new(ModelsDevRefreshCoordinator::default); + +/// Looks up a cached models.dev price for a provider/model pair. +pub fn lookup(provider_id: &str, model_id: &str) -> Option { + let load = ModelsDevCache::load(SystemTime::now(), None); + (!load.is_stale) + .then_some(load.artifact) + .flatten() + .and_then(|artifact| artifact.catalog.lookup(provider_id, model_id)) +} + +/// Refreshes the models.dev cache once when supplied models lack cached pricing. +/// +/// Returns true only if at least one supplied model has pricing after the coordinated refresh. +pub async fn refresh_unknown_models_if_needed( + provider_id: &str, + model_ids: &HashSet, +) -> bool { + if model_ids.is_empty() { + return false; + } + refresh_unknown_models_at(provider_id, model_ids, SystemTime::now(), None).await +} + +async fn refresh_unknown_models_at( + provider_id: &str, + model_ids: &HashSet, + now: SystemTime, + cache_root: Option<&Path>, +) -> bool { + let load = ModelsDevCache::load(now, cache_root); + let unknown_models: Vec = if load.is_stale { + model_ids.iter().cloned().collect() + } else { + model_ids + .iter() + .filter(|model_id| { + load.artifact + .as_ref() + .and_then(|artifact| artifact.catalog.lookup(provider_id, model_id)) + .is_none() + }) + .cloned() + .collect() + }; + if unknown_models.is_empty() { + return true; + } + if load.artifact.as_ref().is_some_and(|artifact| { + now.duration_since(artifact.fetched_at()) + .unwrap_or_default() + < REFRESH_ATTEMPT_WINDOW + }) { + return false; + } + + let cache_path = ModelsDevCache::cache_path(cache_root); + if cache_path.as_os_str().is_empty() { + return false; + } + let cache_root = cache_root.map(Path::to_path_buf); + let refresh_cache_root = cache_root.clone(); + let _ = REFRESH_COORDINATOR + .refresh(cache_path, now, async move { + refresh_catalog(now, refresh_cache_root.as_deref()).await + }) + .await; + + let refreshed = ModelsDevCache::load(now, cache_root.as_deref()); + !refreshed.is_stale + && unknown_models.iter().any(|model_id| { + refreshed + .artifact + .as_ref() + .and_then(|artifact| artifact.catalog.lookup(provider_id, model_id)) + .is_some() + }) +} + +async fn refresh_catalog(now: SystemTime, cache_root: Option<&Path>) -> bool { + let Ok(client) = reqwest::Client::builder() + .timeout(Duration::from_secs(20)) + .build() + else { + return false; + }; + let Ok(response) = client.get(MODELS_DEV_URL).send().await else { + return false; + }; + if !response.status().is_success() { + return false; + } + let Ok(mut catalog) = response.json::().await else { + return false; + }; + if !catalog.is_plausible_refresh() { + return false; + } + if let Some(cached) = ModelsDevCache::load(now, cache_root).artifact { + catalog.merge_priceable_entries_from(&cached.catalog); + } + ModelsDevCache::save(catalog, now, cache_root) +} From 4fc95861909e2a27f3ec61c1aca941f0d98db72e Mon Sep 17 00:00:00 2001 From: NessZerra Date: Sun, 12 Jul 2026 08:26:09 +0700 Subject: [PATCH 5/5] Fix cost freshness documentation whitespace Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- docs/superpowers/plans/2026-07-12-token-cost-freshness.md | 1 - docs/superpowers/specs/2026-07-12-token-cost-freshness-design.md | 1 - 2 files changed, 2 deletions(-) diff --git a/docs/superpowers/plans/2026-07-12-token-cost-freshness.md b/docs/superpowers/plans/2026-07-12-token-cost-freshness.md index e4615fe6fb..6f2b5aead2 100644 --- a/docs/superpowers/plans/2026-07-12-token-cost-freshness.md +++ b/docs/superpowers/plans/2026-07-12-token-cost-freshness.md @@ -273,4 +273,3 @@ PR and rerun affected tests. Run `git diff main...HEAD --check`, verify the worktree and final diff, then push `finesssee-port-cost-freshness` and open one PR into `main` documenting the upstream v0.41.0-v0.42.0 mapping and the exact checks above. - diff --git a/docs/superpowers/specs/2026-07-12-token-cost-freshness-design.md b/docs/superpowers/specs/2026-07-12-token-cost-freshness-design.md index 0f5887429e..5c00680ef1 100644 --- a/docs/superpowers/specs/2026-07-12-token-cost-freshness-design.md +++ b/docs/superpowers/specs/2026-07-12-token-cost-freshness-design.md @@ -74,4 +74,3 @@ This ports behavior, not Swift structure: but keeps it for timed-out scans; - upstream `ModelsDevPricing` coordinates refreshes by cache path, limits repeated attempts, and retries an unknown-model lookup once. -