Skip to content

feature: split inference token usage into per-kind counters #824

Description

@evaline-ju

Feature Description

Today InferenceExtension carries a coarse TotalTokens / PromptTokens / CompletionTokens triple, plus a partial cache split (CacheReadTokens, CacheWriteTokens). This obscures real cost structure and blocks per-kind policy.

Proposed Solution

  • Add a provider-neutral split to InferenceExtension
  • Extend session-budget to leverage this

Want to contribute?

  • I would like to work on this issue.

Additional Context

No response

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions