feat: add context_budget_cap and context window to provider defaults - #10
Open
Salil Das (sadlilas) wants to merge 1 commit into
Open
feat: add context_budget_cap and context window to provider defaults#10Salil Das (sadlilas) wants to merge 1 commit into
Salil Das (sadlilas) wants to merge 1 commit into
Conversation
Gemini charges 2x input / 1.5x output above 200k tokens, matching Anthropic's pricing cliff. Add context_budget_cap: 200,000 so context-simple can cap its working budget at the cost-optimal threshold. Also adds context_window (1,048,576) and max_output_tokens (65,536) to ProviderInfo.defaults -- these were missing, causing context-simple to fall through to its 200k fallback instead of calculating a dynamic budget from the actual model limits. Related: microsoft-amplifier/amplifier-support#57 🤖 Generated with [Amplifier](https://github.com/microsoft/amplifier) Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
context_budget_cap: 200_000,context_window: 1_048_576, andmax_output_tokens: 65_536to Gemini'sProviderInfo.defaults.Why
Budget cap: Gemini charges 2x input / 1.5x output above 200k tokens, matching Anthropic's pricing cliff. The context module (
context-simple) needs a signal from the provider to cap its working budget at the cost-optimal threshold.Context window / max output: These were missing from
get_info().defaults, causingcontext-simpleto fall through to its 200k fallback instead of calculating a dynamic budget from Gemini's actual 1M context window. While the fallback happens to land near the right number, the budget should be calculated correctly and then capped -- not accidentally correct via a fallback path.What changes
Design
Part of the provider-driven budget cap approach. See amplifier-support#57 analysis for the full rationale.
Unlike Anthropic (which conditionally sets the cap only when 1M is enabled), Gemini always sets it because all Gemini models report 1M+ context windows and all have the pricing cliff.
Related PRs
context_budget_cap: 200_000(conditional on 1M enabled)context_budget_capfrom provider defaults and appliesbudget = min(raw_budget, cap - safety_margin)Related issues
🤖 Generated with Amplifier