Skip to content

feat: add context_budget_cap and context window to provider defaults - #10

Open
Salil Das (sadlilas) wants to merge 1 commit into
microsoft:mainfrom
sadlilas:feat/context-budget-cap
Open

feat: add context_budget_cap and context window to provider defaults#10
Salil Das (sadlilas) wants to merge 1 commit into
microsoft:mainfrom
sadlilas:feat/context-budget-cap

Conversation

@sadlilas

Copy link
Copy Markdown
Collaborator

Summary

Adds context_budget_cap: 200_000, context_window: 1_048_576, and max_output_tokens: 65_536 to Gemini's ProviderInfo.defaults.

Why

Budget cap: Gemini charges 2x input / 1.5x output above 200k tokens, matching Anthropic's pricing cliff. The context module (context-simple) needs a signal from the provider to cap its working budget at the cost-optimal threshold.

Context window / max output: These were missing from get_info().defaults, causing context-simple to fall through to its 200k fallback instead of calculating a dynamic budget from Gemini's actual 1M context window. While the fallback happens to land near the right number, the budget should be calculated correctly and then capped -- not accidentally correct via a fallback path.

What changes

defaults={
    "model": "gemini-2.5-flash",
    "max_tokens": 8192,
    "temperature": 0.7,
    "timeout": 600.0,
    "context_window": 1_048_576,       # NEW
    "max_output_tokens": 65_536,       # NEW
    "context_budget_cap": 200_000,     # NEW
},

Design

Part of the provider-driven budget cap approach. See amplifier-support#57 analysis for the full rationale.

Unlike Anthropic (which conditionally sets the cap only when 1M is enabled), Gemini always sets it because all Gemini models report 1M+ context windows and all have the pricing cliff.

Related PRs

  • provider-anthropic#27: Same context_budget_cap: 200_000 (conditional on 1M enabled)
  • context-simple: Reads context_budget_cap from provider defaults and applies budget = min(raw_budget, cap - safety_margin)

Related issues

  • microsoft-amplifier/amplifier-support#57

🤖 Generated with Amplifier

Gemini charges 2x input / 1.5x output above 200k tokens, matching
Anthropic's pricing cliff. Add context_budget_cap: 200,000 so
context-simple can cap its working budget at the cost-optimal threshold.

Also adds context_window (1,048,576) and max_output_tokens (65,536)
to ProviderInfo.defaults -- these were missing, causing context-simple
to fall through to its 200k fallback instead of calculating a dynamic
budget from the actual model limits.

Related: microsoft-amplifier/amplifier-support#57

🤖 Generated with [Amplifier](https://github.com/microsoft/amplifier)

Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant