Feature Description
Today InferenceExtension carries a coarse TotalTokens / PromptTokens / CompletionTokens triple, plus a partial cache split (CacheReadTokens, CacheWriteTokens). This obscures real cost structure and blocks per-kind policy.
Proposed Solution
- Add a provider-neutral split to InferenceExtension
- Extend session-budget to leverage this
Want to contribute?
Additional Context
No response
Feature Description
Today
InferenceExtensioncarries a coarse TotalTokens / PromptTokens / CompletionTokens triple, plus a partial cache split (CacheReadTokens, CacheWriteTokens). This obscures real cost structure and blocks per-kind policy.Proposed Solution
Want to contribute?
Additional Context
No response