Skip to content

[1133] feature: local-usage-stats (3/4) - #52

Closed
myk1yt wants to merge 31 commits into
mainfrom
pr/b15-usage-capture-v2
Closed

[1133] feature: local-usage-stats (3/4)#52
myk1yt wants to merge 31 commits into
mainfrom
pr/b15-usage-capture-v2

Conversation

@myk1yt

@myk1yt myk1yt commented Aug 10, 2026

Copy link
Copy Markdown
Owner

Stack Position

Description

https://www.youtube.com/shorts/UHnnOCM1_f0

Full Feature Description

  • Feature Branch: feature/local-usage-stats
  • Feature Name: Local Usage Statistics
  • Purpose: Resolves the problem where users cannot locally view token usage, cache effects, cost, and period-based trends by provider, and where differing usage formats across providers make consistent aggregation difficult. Provides a privacy-preserving dashboard that collects only numeric usage and non-secret identifiers locally, without collecting prompts, responses, or credentials.
  • Full Change Description: B13 adds data-minimized event/query contracts and an append-only NDJSON event store. B14 adds aggregation by date, provider, model, and mode, cache ratio, and provider-aware cost recalculation. B15 records final usage exactly once from the API attempt completion path, including success/error/cancel/retry. B16 adds transactional SQLite projection, idempotent migration, local-day rollup, query/stream IPC, stale epoch prevention, and dashboard summary/session/heatmap UI.
  • Impact Scope: Affects usage-stats.ts, src/services/stats, the provider/task capture paths Task.ts, the stats IPC usageStatsMessageHandler.ts, and the UI DashboardView.tsx and useDashboardStatsStream.ts.
  • Errors and Edge Cases: Raw events are append-only and derived rollups must be reconstructable. Duplicate idempotency keys are not re-recorded. Corrupt tails preserve the valid prefix and leave only a hash in the quarantine report instead of the original text. Migrations must be transactional/idempotent. Local day and DST boundaries are calculated per-timestamp by offset. Previous subscription epochs must not overwrite new range results. The store must not contain prompts, responses, API keys, endpoint credentials, or workspace paths.
  • Testing Method: Run contract/store, aggregation/cost, exactly-once capture, database/migration/projection/stream, IPC, dashboard reducer/component, performance, locale, and visual tests step by step. Manually create complete/cancel/retry attempts, verify event counts, then rapidly switch ranges in two dashboard windows and add events, verifying convergence without stale loading or duplicate totals. Inspect stored files to confirm no sensitive fields are present.

Why Split Into 17 PRs

Instead of submitting this feature as a single unified PR, it was split into individual PRs because as code size grows, safely reviewing a PR becomes very difficult. The feature was broken into mutually exclusive individual PRs so that each can be reviewed independently.

What This PR Specifically Changes

Records provider usage delta exactly once from task API attempt finalization. Handles success, error, cancel, retry, incremental usage, and duplicate finalization. Does not include query/UI.

Included Files

  • src/services/stats/UsageRecorder.ts
  • src/core/task/Task.ts
  • src/api/providers/openai.ts
  • src/api/providers/openai-codex.ts
  • Direct task/provider usage tests

Exclusion Scope

  • Database projection/migration
  • Stats IPC/stream/dashboard UI
  • Provider changes unrelated to usage calculation
  • Session report and repair script
  • All items in the common removal rules

Summary by CodeRabbit

  • New Features
    • Added automatic usage statistics tracking across API calls and sessions.
    • Added time-range, grouping, filtering, timezone-aware summaries, and cancellation-aware reporting.
    • Added JSON and CSV export options.
    • Added safeguards for clearing statistics and handling incomplete or unavailable data.
  • Bug Fixes
    • OpenAI and OpenAI Codex usage now report calculated costs.
    • Qwen model pricing now reflects current nonzero rates.
  • Tests
    • Expanded coverage for usage capture, validation, aggregation, exports, costs, and provider reporting.

Upstream PR: Zoo-Code-Org#1133

k1yt and others added 30 commits August 2, 2026 08:27
…cit-any

Add new test file to eslint-suppressions.json with count of 26
no-explicit-any suppressions. These are standard test patterns
(mock objects, private property access via 'as any') consistent
with other test files in the suppressions list.

Fixes CI lint failure in PR #25 compile (lint) job.
…cing

- Remove UTF-8 BOM (U+FEFF) from costRecalculation.ts and costRecalculation.spec.ts
- Fix qwenCodeModels pricing: qwen3-coder-plus inputPrice 0->1.0, outputPrice 0->5.0
- Fix qwenCodeModels pricing: qwen3-coder-flash inputPrice 0->0.3, outputPrice 0->1.5

Fixes invisible-chars CI check and 3 failing costRecalculation tests
…exactly-once recorder

- UsageRecorder: per-task exactly-once usage event recording with endpoint domain extraction
- costRecalculation: compute effective cost from token deltas and model pricing
- Provider usage deltas: moonshot, openai, openai-codex, vscode-lm yield cumulative usage; Task diffs and records
- Task finalization: flush pending usage events on abort/complete
- ClineProvider: initialize UsageStatsService, expose getUsageStatsService, forward usageStatsChanged to webview
- types: add usage-stats schemas and usageStatsChanged ExtensionMessage type
…proper types, fix run->start renames, add UsageEventStore import
The B15 usage-capture cherry-pick was authored against an older base and
reverted newer upstream/base behavior in several files, causing e2e-mock
subtask timeouts (7 tests) and unit-test failures.

Restore clobbered base behavior while keeping B15's genuine usage/cost
capture additions:
- Task.ts: restore run() + _runPromise/_isHistoryTask, safeEnsureModelFetched
  (def + 3 call sites), abort-aware ask wait, resume_completed_task via
  initialStatus, and t() i18n in sayAndCreateMissingParamError.
- ClineProvider.ts: scheduler gates on task.run() (completion promise)
  instead of fire-and-forget task.start(). This is the root cause of the
  subtask/resume e2e timeouts.
- openai-codex.ts: restore service-tier feature alongside cost capture.
- moonshot.ts, vscode-lm.ts, vscode-lm-format.ts, eslint-suppressions.json:
  revert to base (pure clobber, no genuine B15 content).
- task-run-dispatch.spec.ts: bind run() (not start()).
- openai-usage-tracking.spec.ts: assert totalCost from cost capture.
…rn (PR Zoo-Code-Org#1133)

CI failures:
1. Code QA Roo Code run 31231476937 (platform-unit-test): 8 unhandled
   EnvironmentTeardownError rejections from core/task/__tests__/grace-retry-errors.spec.ts.
   The Task constructor's usage-recorder init called the mocked provider's
   getUsageStatsService(); the catch block's console.warn fired during the test
   and tripped vitest's 'Closing rpc while onUserConsoleLog was pending'.
   Silenced the best-effort init catch (recorder stays disabled).
2. E2E Tests (Mocked) run 31231476961: usage-capture.test.ts startNewTask calls
   (USAGE_CAPTURE_SMOKE, USAGE_CAPTURE_HOOK_2) had no aimock fixture
   ('404 No fixture matched').

Fix:
- Add fixtures/usage-capture.json with attempt_completion fixtures for both tags.
Runs: https://github.com/Zoo-Code-Org/Zoo-Code/actions/runs/31231476937
      https://github.com/Zoo-Code-Org/Zoo-Code/actions/runs/31231476961
…path lookup to prevent I/O event loop congestion
getStatsDirs() computed repoRoot as 3 levels up from __dirname which
resolves to apps/ instead of project root on Linux CI. Use
vscode.extensions.getExtension() to get the authoritative
globalStorageUri at runtime, matching where ClineProvider writes data.
@myk1yt myk1yt closed this Aug 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants