From c89168d3f9454802704f19b62da1e6f1d44e724a Mon Sep 17 00:00:00 2001 From: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com> Date: Sat, 29 Aug 2026 13:17:14 -0700 Subject: [PATCH 1/7] fix(thinking): retarget reasoning_effort mapping to thinking_level MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Google's thinking_budget (an approximate output-token budget) is now the LEGACY thinking control. The current control -- and the only one some Gemini 3.x models accept at all -- is thinking_level, an enum (minimal|low|medium|high). Sending both on one request is a 400. Replaces the old lossy reasoning_effort -> {4096, -1} mapping with a thinking_level mapping, clamped per-model against a small maintained table (_THINKING_LEVEL_TABLE), with a one-line INFO log whenever a clamp changes the requested level -- never silent. Verified LIVE against the real API on 2026-08-29 (not documented by Google as of this writing): - gemini-2.5-flash and gemini-2.5-pro REJECT thinking_level outright: 400 INVALID_ARGUMENT "Thinking level is not supported for this model." These models keep the legacy thinking_budget path as their only control. - gemini-3.7-flash accepts low/medium/high but rejects MINIMAL ("Thinking level MINIMAL is not supported for this model..."). - gemini-3.5-flash(-lite) accept the full minimal..high range. - gemini-3.x thinking is mandatory: thinking_budget=0 on gemini-3.7-flash still produced ~26 thinking tokens -- there is no way to disable thinking on a Gemini 3.x model, and thinking_level has no "off" value. Also fixes a pre-existing bug surfaced by this same code path: explicit thinking_budget=0 was never actually reaching the API (the old code omitted thinking_config entirely instead of sending the zero). Verified live that omitting the config is NOT equivalent to sending an explicit zero -- gemini-2.5-flash with no thinking_config still reports a populated thoughts_token_count (still thinking), while an explicit thinking_budget=0 correctly reports thoughts_token_count=None (thinking genuinely disabled). The fix always sends the explicit value instead of omitting it. Explicit thinking_budget via kwargs/request.metadata still wins outright (legacy override, honored ALONE, never combined with thinking_level). When no directive is given at all, thinking_config now carries only include_thoughts -- omitting both budget and level lets the model apply its own default amount (verified live this still returns thought summaries), rather than forcing an explicit dynamic budget that forecloses the thinking_level path for level-only models. Bumps the google-genai floor from >=1.40.0 to >=1.56.0 -- verified by probing the installed SDK's own types directly: 1.46.0 has no thinking_level field at all; 1.51.0 adds it with only LOW/HIGH; 1.56.0 adds MINIMAL and MEDIUM, completing the four-level enum this module needs. Updates the two existing tests whose assertions encoded the old (buggy) behavior, with comments explaining what changed and why. Adds tests/test_thinking_level.py: per-model support table lookups, clamping (including the INFO log), the full reasoning_effort matrix on a level-supporting model, the legacy-mapping fallback on a level-rejecting model, and the never-both-together guarantee. 🤖 Generated with [Amplifier](https://github.com/microsoft/amplifier) Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com> --- amplifier_module_provider_gemini/__init__.py | 278 +++++++++++++++-- pyproject.toml | 11 +- tests/test_reasoning_effort.py | 34 +- tests/test_thinking_level.py | 308 +++++++++++++++++++ 4 files changed, 594 insertions(+), 37 deletions(-) create mode 100644 tests/test_thinking_level.py diff --git a/amplifier_module_provider_gemini/__init__.py b/amplifier_module_provider_gemini/__init__.py index dee3b37..f486ee4 100644 --- a/amplifier_module_provider_gemini/__init__.py +++ b/amplifier_module_provider_gemini/__init__.py @@ -283,6 +283,133 @@ def _encode_sig(sig: bytes | str | None) -> str | None: return sig # assume already base64-encoded str +# --------------------------------------------------------------------------- +# Thinking level support (google-genai's *current* thinking control) +# --------------------------------------------------------------------------- +# Google's `thinking_budget` (an approximate output-token budget spent on +# internal reasoning) is now the LEGACY thinking control. The current +# control -- and the *only* one some models accept at all -- is +# `thinking_level`, an enum (minimal|low|medium|high). Sending both +# thinking_level and thinking_budget on the same request is rejected by the +# API with a 400. +# +# CRITICAL, verified LIVE against the real Google AI API on 2026-08-29 (not +# documented by Google as of this writing, and NOT the "smaller subset of +# levels per older model" story one might assume): thinking_level support is +# an all-or-nothing split by model generation, not a graduated subset -- +# * gemini-2.5-flash and gemini-2.5-pro REJECT thinking_level outright: +# 400 INVALID_ARGUMENT: "Thinking level is not supported for this model." +# These models only understand the legacy thinking_budget control, and +# think by default via a dynamic budget regardless of any config. +# * gemini-3.x models are the opposite: thinking is MANDATORY and +# thinking_budget is silently ignored (verified live: thinking_budget=0 +# on gemini-3.7-flash still produced ~26 thinking tokens) -- there is no +# way to disable thinking on a Gemini 3.x model. thinking_level is their +# only real, effective control, and even that control cannot express +# "disabled" -- there is no "none" level in the enum. +# +# _THINKING_LEVEL_TABLE maps a model id to the ThinkingLevel values it is +# known to accept. `None` means "rejects thinking_level entirely -- always +# use the legacy thinking_budget path for this model". Google does not +# publish this table; it is reverse-engineered from live 400 responses. +# Treat it as best-effort and update it as new models ship or vendor +# behavior changes. +_THINKING_LEVEL_TABLE: dict[str, tuple[str, ...] | None] = { + # Gemini 2.x -- thinking_level rejected outright (verified live, + # 2026-08-29). These models think by default via a dynamic budget; + # the legacy thinking_budget path is their only control. + "gemini-2.5-pro": None, + "gemini-2.5-flash": None, + "gemini-2.5-flash-lite": None, + "gemini-2.0-flash": None, + "gemini-2.0-flash-lite": None, + # Gemini 3.7 Flash -- current flagship Flash model. Verified live: + # low/medium/high accepted; MINIMAL rejected ("Thinking level MINIMAL is + # not supported for this model. Please retry with other thinking + # level."). ai.google.dev documents its default (when omitted) as medium. + "gemini-3.7-flash": ("low", "medium", "high"), + # Gemini 3.5 family -- verified live: minimal accepted. + "gemini-3.5-flash": ("minimal", "low", "medium", "high"), + "gemini-3.5-flash-lite": ("minimal", "low", "medium", "high"), +} + +# Fallback range for any gemini-3.x model id not listed above -- new preview +# ids ship often (gemini-3.1-*, gemini-3.6-flash, etc.). Assume the full +# range until a live 400 proves a narrower one for that specific id. +_THINKING_LEVEL_DEFAULT_3X: tuple[str, ...] = ("minimal", "low", "medium", "high") + +# Ordinal order used for clamping (lowest to highest amount of thinking). +_THINKING_LEVEL_ORDER: tuple[str, ...] = ("minimal", "low", "medium", "high") + +# reasoning_effort (Amplifier's portable, cross-provider knob) -> the +# thinking_level it targets. "xhigh"/"max" collapse to "high" -- Gemini has +# no level above high. +_EFFORT_TO_LEVEL: dict[str, str] = { + "minimal": "minimal", + "low": "low", + "medium": "medium", + "high": "high", + "xhigh": "high", + "max": "high", +} + + +def _supported_thinking_levels(model: str) -> tuple[str, ...] | None: + """Return the ThinkingLevel values ``model`` is known to accept. + + ``None`` means the model rejects thinking_level entirely (legacy + thinking_budget path only). See _THINKING_LEVEL_TABLE for the live + evidence behind each entry. + """ + if model in _THINKING_LEVEL_TABLE: + return _THINKING_LEVEL_TABLE[model] + if model.startswith("gemini-2."): + return None + if model.startswith("gemini-3."): + return _THINKING_LEVEL_DEFAULT_3X + # Unknown family (a future gemini-4.x, a tuned model id, etc.) -- assume + # support; a live 400 surfaces clearly rather than degrading silently. + return _THINKING_LEVEL_DEFAULT_3X + + +def _clamp_thinking_level(model: str, requested: str, supported: tuple[str, ...]) -> str: + """Clamp ``requested`` to the nearest level in ``supported`` for ``model``. + + Prefers the next level UP (more thinking) over down: under-thinking + silently degrades output quality, while over-thinking only costs a few + more tokens. Logs one INFO line whenever the clamp actually changes the + requested value -- clamping is never silent. + """ + if requested in supported: + return requested + req_idx = _THINKING_LEVEL_ORDER.index(requested) + for idx in range(req_idx + 1, len(_THINKING_LEVEL_ORDER)): + if _THINKING_LEVEL_ORDER[idx] in supported: + clamped = _THINKING_LEVEL_ORDER[idx] + logger.info( + "[PROVIDER] Gemini: thinking_level '%s' not supported by '%s' " + "(supports: %s) -- clamped up to '%s'", + requested, + model, + supported, + clamped, + ) + return clamped + for idx in range(req_idx - 1, -1, -1): + if _THINKING_LEVEL_ORDER[idx] in supported: + clamped = _THINKING_LEVEL_ORDER[idx] + logger.info( + "[PROVIDER] Gemini: thinking_level '%s' not supported by '%s' " + "(supports: %s) -- clamped down to '%s'", + requested, + model, + supported, + clamped, + ) + return clamped + return supported[0] # pragma: no cover -- defensive; supported is never empty + + class GeminiChatResponse(ChatResponse): """ChatResponse with additional fields for streaming UI compatibility.""" @@ -831,6 +958,122 @@ async def complete(self, request: ChatRequest, **kwargs) -> ChatResponse: return await self._complete_chat_request(request, **kwargs) + def _resolve_thinking_config( + self, model: str, request: ChatRequest, kwargs: dict[str, Any] + ): + """Resolve the ThinkingConfig to send for this request. + + Precedence (highest first): + + 1. Explicit ``thinking_budget`` via kwargs or ``request.metadata`` -- + the legacy override. Honored ALONE: never combined with + thinking_level (Google 400s if both are set on one request). + 2. ``request.reasoning_effort`` -- mapped to a thinking_level target + (see _EFFORT_TO_LEVEL), clamped per-model (see + _supported_thinking_levels / _clamp_thinking_level). Models that + reject thinking_level entirely fall back to the legacy numeric + mapping instead (the only control they have): + none=0, minimal/low=4096, medium/high/xhigh/max=-1 (dynamic). + 3. No directive at all -- omit both thinking_budget and + thinking_level, keeping only include_thoughts. Gemini models + think by default (dynamically) even with a config that sets + neither field; forcing an explicit dynamic budget of -1 is + functionally equivalent but needlessly forecloses "let the model + use its own built-in default" for level-only models. Verified + live: ThinkingConfig(include_thoughts=True) with no budget/level + still returns thought summaries at the model's own default + thinking amount, on both gemini-2.5-flash and gemini-3.7-flash. + + A note on disabling thinking: explicitly sending thinking_budget=0 + DOES disable thinking on Gemini 2.x models (verified live: + thoughts_token_count becomes None) -- but *omitting* the config + entirely does NOT (verified live: thoughts_token_count is still + populated with no thinking_config sent at all). These are not + equivalent, so the explicit-zero case below always sends the + field rather than omitting it -- omitting it here was the + pre-existing implementation's bug (thinking_budget=0 never actually + reached the API), fixed as part of this same change since it's the + exact code path being redesigned. + + Gemini 3.x cannot disable thinking at all regardless of what is + sent (verified live: thinking_budget=0 on gemini-3.7-flash still + produced ~26 thinking tokens; there is no "off" thinking_level). + That is a vendor limitation, not something this method can work + around. + """ + from google import genai + + include_thoughts = True + if request.metadata and "include_thoughts" in request.metadata: + include_thoughts = request.metadata["include_thoughts"] + if "include_thoughts" in kwargs: + include_thoughts = kwargs["include_thoughts"] + + # --- 1. Explicit thinking_budget (legacy override) ----------------- + explicit_budget = None + if request.metadata and "thinking_budget" in request.metadata: + explicit_budget = request.metadata["thinking_budget"] + if "thinking_budget" in kwargs: + explicit_budget = kwargs["thinking_budget"] + + if explicit_budget is not None: + return genai.types.ThinkingConfig( + thinking_budget=explicit_budget, include_thoughts=include_thoughts + ) + + # --- 2. reasoning_effort -> thinking_level (clamped per-model) ------ + if request.reasoning_effort: + effort = request.reasoning_effort.lower() + supported = _supported_thinking_levels(model) + + if supported is None: + # This model has no thinking_level control at all -- the + # legacy numeric mapping is the only lever available. + if effort == "none": + budget = 0 + elif effort in ("minimal", "low"): + budget = 4096 + else: + budget = -1 # medium/high/xhigh/max -> dynamic + return genai.types.ThinkingConfig( + thinking_budget=budget, include_thoughts=include_thoughts + ) + + if effort == "none": + if "minimal" in supported: + return genai.types.ThinkingConfig( + thinking_level=genai.types.ThinkingLevel.MINIMAL, + include_thoughts=include_thoughts, + ) + # This model can't go any lower than its own default, and + # (for Gemini 3.x) may not be able to disable thinking at + # all -- fall through to the model's own default amount. + logger.info( + "[PROVIDER] Gemini: reasoning_effort='none' requested but " + "'%s' has no thinking_level below its own default -- " + "using the model's default thinking amount instead", + model, + ) + return genai.types.ThinkingConfig(include_thoughts=include_thoughts) + + target = _EFFORT_TO_LEVEL.get(effort) + if target is None: + logger.warning( + "[PROVIDER] Gemini: unknown reasoning_effort '%s' -- " + "ignoring, using model default thinking", + request.reasoning_effort, + ) + return genai.types.ThinkingConfig(include_thoughts=include_thoughts) + + level = _clamp_thinking_level(model, target, supported) + return genai.types.ThinkingConfig( + thinking_level=genai.types.ThinkingLevel(level.upper()), + include_thoughts=include_thoughts, + ) + + # --- 3. No directive at all ----------------------------------------- + return genai.types.ThinkingConfig(include_thoughts=include_thoughts) + async def _complete_chat_request( self, request: ChatRequest, **kwargs ) -> ChatResponse: @@ -891,41 +1134,16 @@ async def _complete_chat_request( "max_tokens", self.max_tokens ) - # Extract thinking parameters from request metadata or kwargs - # Default: Enable dynamic thinking with text summaries for 2.5+ models - thinking_budget = -1 # -1 = dynamic (model decides), 0 = disabled - include_thoughts = True # Get text summaries of thoughts - - if request.metadata: - if "thinking_budget" in request.metadata: - thinking_budget = request.metadata.get("thinking_budget") - include_thoughts = request.metadata.get("include_thoughts", True) - - # reasoning_effort support (portable interface, checked after metadata but before kwargs) - # Maps reasoning_effort to thinking_budget values per design doc. - if request.reasoning_effort and "thinking_budget" not in kwargs: - effort = request.reasoning_effort.lower() - if effort == "low": - thinking_budget = 4096 - elif effort in ("medium", "high"): - thinking_budget = -1 # dynamic - - # Allow kwargs to override (backward compat — takes absolute precedence) - if "thinking_budget" in kwargs: - thinking_budget = kwargs["thinking_budget"] - if "include_thoughts" in kwargs: - include_thoughts = kwargs["include_thoughts"] + # Resolve thinking configuration -- see _resolve_thinking_config for + # the full precedence (explicit thinking_budget > reasoning_effort -> + # thinking_level, clamped per-model > model default). + thinking_config = self._resolve_thinking_config(model, request, kwargs) # Build Gemini config with thinking support config = genai.types.GenerateContentConfig( temperature=temperature, max_output_tokens=max_tokens ) - - # Add thinking configuration (enabled by default for 2.5+ models) - if thinking_budget != 0: # 0 explicitly disables thinking - config.thinking_config = genai.types.ThinkingConfig( - thinking_budget=thinking_budget, include_thoughts=include_thoughts - ) + config.thinking_config = thinking_config if system_instruction: config.system_instruction = system_instruction diff --git a/pyproject.toml b/pyproject.toml index 3ade5ca..f9057b6 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -9,7 +9,16 @@ authors = [ { name = "Microsoft MADE:Explorations Team" }, ] dependencies = [ - "google-genai>=1.40.0", + # 1.56.0 is the floor because that is the first google-genai release + # whose ThinkingConfig exposes the full thinking_level enum this module + # needs (minimal/low/medium/high). Verified by probing the SDK's own + # installed types directly (not just changelog text) on 2026-08-29: + # - 1.46.0: no thinking_level field on ThinkingConfig at all. + # - 1.51.0: thinking_level added, but only LOW/HIGH values exist. + # - 1.56.0: adds MINIMAL and MEDIUM -- the full four-level enum. + # See amplifier_module_provider_gemini/__init__.py's + # _THINKING_LEVEL_TABLE comment for the per-model support matrix. + "google-genai>=1.56.0", ] [project.entry-points."amplifier.modules"] diff --git a/tests/test_reasoning_effort.py b/tests/test_reasoning_effort.py index d79eabd..10b4dbc 100644 --- a/tests/test_reasoning_effort.py +++ b/tests/test_reasoning_effort.py @@ -107,7 +107,16 @@ def test_reasoning_effort_high_sets_dynamic(): def test_reasoning_effort_none_preserves_default(): - """reasoning_effort=None -> existing behavior (default dynamic thinking).""" + """reasoning_effort=None -> no explicit budget/level; model default thinking. + + Historically this asserted thinking_budget == -1 ("explicit dynamic"). + Post-retarget, no directive at all means neither thinking_budget nor + thinking_level is sent -- verified live that Gemini still thinks by + default (and, with include_thoughts=True, still returns thought + summaries) with a bare ThinkingConfig(include_thoughts=True). Sending + -1 explicitly was functionally equivalent but foreclosed the + thinking_level path for models that only accept levels. + """ provider = _make_provider() mock_client = _capture_config(provider) @@ -119,8 +128,9 @@ def test_reasoning_effort_none_preserves_default(): call_kwargs = mock_client.aio.models.generate_content.await_args config = call_kwargs.kwargs.get("config") or call_kwargs[1].get("config") - # Default is -1 (dynamic) - assert config.thinking_config.thinking_budget == -1 + assert config.thinking_config.thinking_budget is None + assert config.thinking_config.thinking_level is None + assert config.thinking_config.include_thoughts is True def test_kwargs_thinking_budget_overrides_reasoning_effort(): @@ -157,7 +167,18 @@ def test_metadata_thinking_budget_with_no_reasoning_effort(): def test_thinking_disabled_with_budget_zero(): - """thinking_budget=0 in kwargs should disable thinking entirely.""" + """thinking_budget=0 in kwargs should disable thinking entirely. + + Bug fix: the pre-retarget implementation OMITTED thinking_config + entirely when thinking_budget resolved to 0, and this test asserted + exactly that ("thinking_config is None"). Verified live against the + real API that omitting the field is NOT the same as sending an explicit + zero: gemini-2.5-flash with no thinking_config at all still reports a + populated thoughts_token_count (still thinking, budget=0 never actually + reached the API), while an EXPLICIT thinking_budget=0 correctly reports + thoughts_token_count=None (thinking genuinely disabled). The fix always + sends the explicit value through instead of omitting it. + """ provider = _make_provider() mock_client = _capture_config(provider) @@ -169,5 +190,6 @@ def test_thinking_disabled_with_budget_zero(): call_kwargs = mock_client.aio.models.generate_content.await_args config = call_kwargs.kwargs.get("config") or call_kwargs[1].get("config") - # thinking_budget=0 disables thinking, so thinking_config should not be set - assert config.thinking_config is None + assert config.thinking_config is not None + assert config.thinking_config.thinking_budget == 0 + assert config.thinking_config.thinking_level is None diff --git a/tests/test_thinking_level.py b/tests/test_thinking_level.py new file mode 100644 index 0000000..ef60864 --- /dev/null +++ b/tests/test_thinking_level.py @@ -0,0 +1,308 @@ +"""Tests for the thinking_level retarget (replaces the old reasoning_effort -> +thinking_budget-only mapping). + +Covers: +- _supported_thinking_levels() per-model-family lookups +- _clamp_thinking_level() clamping behavior + INFO logging (never silent) +- End-to-end reasoning_effort -> thinking_level mapping through complete() + for level-supporting models (gemini-3.x) +- End-to-end reasoning_effort -> legacy thinking_budget mapping for models + that reject thinking_level entirely (gemini-2.x) +- Explicit thinking_budget always wins and is sent ALONE (never combined + with thinking_level) + +All live-behavior claims embedded in these tests/comments were verified +against the real Google AI API on 2026-08-29 (see _THINKING_LEVEL_TABLE's +module-level comment in amplifier_module_provider_gemini/__init__.py). +""" + +import asyncio +from types import SimpleNamespace +from typing import cast +from unittest.mock import AsyncMock, MagicMock + +import pytest +from amplifier_core import ModuleCoordinator +from amplifier_core.message_models import ChatRequest, Message + +from amplifier_module_provider_gemini import GeminiProvider +from amplifier_module_provider_gemini import _clamp_thinking_level +from amplifier_module_provider_gemini import _supported_thinking_levels + + +class FakeHooks: + def __init__(self): + self.events: list[tuple[str, dict]] = [] + + async def emit(self, name: str, payload: dict) -> None: + self.events.append((name, payload)) + + +class FakeCoordinator: + def __init__(self): + self.hooks = FakeHooks() + + +def _make_gemini_response(): + part = SimpleNamespace(text="Hello", thought=False) + content = SimpleNamespace(parts=[part]) + candidate = SimpleNamespace(content=content) + usage = SimpleNamespace( + prompt_token_count=10, candidates_token_count=5, total_token_count=15 + ) + return SimpleNamespace(candidates=[candidate], usage_metadata=usage) + + +def _make_provider() -> GeminiProvider: + provider = GeminiProvider( + api_key="test-key", config={"max_retries": 0, "use_streaming": False} + ) + provider.coordinator = cast(ModuleCoordinator, FakeCoordinator()) + return provider + + +def _capture_config(provider: GeminiProvider): + mock_client = MagicMock() + mock_client.aio.models.generate_content = AsyncMock( + return_value=_make_gemini_response() + ) + provider._client = mock_client + return mock_client + + +def _run_complete(provider, request, **kwargs): + mock_client = _capture_config(provider) + asyncio.run(provider.complete(request, **kwargs)) + call_kwargs = mock_client.aio.models.generate_content.await_args + return call_kwargs.kwargs.get("config") or call_kwargs[1].get("config") + + +def _make_request(**overrides) -> ChatRequest: + overrides.pop("model", None) # 'model' isn't a ChatRequest field consumed + # by the provider today -- it is passed as a complete() kwarg instead + # (see _complete_chat_request: model = kwargs.get("model", self.default_model)). + return ChatRequest(messages=[Message(role="user", content="Hello")], **overrides) + + +# ============================================================ +# _supported_thinking_levels() -- per-model-family table +# ============================================================ + + +def test_25_family_rejects_thinking_level_entirely(): + """Verified live: gemini-2.5-{flash,pro,flash-lite} 400 on thinking_level.""" + assert _supported_thinking_levels("gemini-2.5-flash") is None + assert _supported_thinking_levels("gemini-2.5-pro") is None + assert _supported_thinking_levels("gemini-2.5-flash-lite") is None + + +def test_20_family_rejects_thinking_level_by_prefix_fallback(): + assert _supported_thinking_levels("gemini-2.0-flash") is None + assert _supported_thinking_levels("gemini-2.0-flash-lite") is None + # Unlisted 2.x id -- prefix fallback still says "no support". + assert _supported_thinking_levels("gemini-2.9-hypothetical") is None + + +def test_37_flash_supports_low_medium_high_but_not_minimal(): + """Verified live: MINIMAL is explicitly rejected for gemini-3.7-flash.""" + assert _supported_thinking_levels("gemini-3.7-flash") == ("low", "medium", "high") + + +def test_35_family_supports_full_range_including_minimal(): + """Verified live: minimal accepted on gemini-3.5-flash(-lite).""" + assert _supported_thinking_levels("gemini-3.5-flash") == ( + "minimal", + "low", + "medium", + "high", + ) + assert _supported_thinking_levels("gemini-3.5-flash-lite") == ( + "minimal", + "low", + "medium", + "high", + ) + + +def test_unknown_3x_model_falls_back_to_full_range(): + assert _supported_thinking_levels("gemini-3.1-flash-lite-preview") == ( + "minimal", + "low", + "medium", + "high", + ) + + +def test_unknown_family_defaults_to_full_range(): + """A future gemini-4.x (or anything outside 2.x/3.x) is assumed to support + thinking_level -- a live 400 will surface clearly rather than degrading + silently.""" + assert _supported_thinking_levels("gemini-4.0-flash") == ( + "minimal", + "low", + "medium", + "high", + ) + + +# ============================================================ +# _clamp_thinking_level() -- clamping + logging +# ============================================================ + + +def test_clamp_noop_when_already_supported(): + assert _clamp_thinking_level("gemini-3.7-flash", "low", ("low", "medium", "high")) == "low" + + +def test_clamp_up_when_requested_too_low(caplog): + """minimal isn't supported on 3.7-flash -- clamps UP to low, logs INFO.""" + with caplog.at_level("INFO"): + result = _clamp_thinking_level( + "gemini-3.7-flash", "minimal", ("low", "medium", "high") + ) + assert result == "low" + assert any( + "clamped up to 'low'" in rec.message + and "gemini-3.7-flash" in rec.message + for rec in caplog.records + ), f"expected an INFO clamp log, got: {[r.message for r in caplog.records]}" + + +def test_clamp_down_when_no_higher_option(caplog): + """A hypothetical model that ONLY supports minimal/low -- requesting + high must clamp DOWN since there's nothing higher available.""" + with caplog.at_level("INFO"): + result = _clamp_thinking_level("hypothetical-model", "high", ("minimal", "low")) + assert result == "low" + assert any("clamped down to 'low'" in rec.message for rec in caplog.records) + + +# ============================================================ +# End-to-end: reasoning_effort -> thinking_level (level-supporting model) +# ============================================================ + + +@pytest.mark.parametrize( + "effort,expected_level", + [ + ("low", "LOW"), + ("medium", "MEDIUM"), + ("high", "HIGH"), + ("xhigh", "HIGH"), + ("max", "HIGH"), + ], +) +def test_reasoning_effort_maps_to_thinking_level_on_37_flash(effort, expected_level): + provider = _make_provider() + request = ChatRequest( + messages=[Message(role="user", content="Hello")], + reasoning_effort=effort, + ) + config = _run_complete(provider, request, model="gemini-3.7-flash") + assert config.thinking_config.thinking_budget is None + assert config.thinking_config.thinking_level is not None + assert config.thinking_config.thinking_level.value == expected_level + + +def test_reasoning_effort_minimal_clamped_to_low_on_37_flash(caplog): + """37-flash doesn't support minimal -- must clamp up to low, and log it.""" + provider = _make_provider() + request = ChatRequest( + messages=[Message(role="user", content="Hello")], + reasoning_effort="minimal", + ) + with caplog.at_level("INFO"): + config = _run_complete(provider, request, model="gemini-3.7-flash") + assert config.thinking_config.thinking_level.value == "LOW" + assert any("clamped up to 'low'" in rec.message for rec in caplog.records) + + +def test_reasoning_effort_none_on_37_flash_omits_thinking_directive(caplog): + """37-flash has no level below its default and can't disable thinking at + all -- 'none' falls through to the model's own default (no explicit + budget or level sent), with an INFO note explaining why.""" + provider = _make_provider() + request = ChatRequest( + messages=[Message(role="user", content="Hello")], + reasoning_effort="none", + ) + with caplog.at_level("INFO"): + config = _run_complete(provider, request, model="gemini-3.7-flash") + assert config.thinking_config.thinking_budget is None + assert config.thinking_config.thinking_level is None + assert any("using the model's default thinking amount" in rec.message for rec in caplog.records) + + +def test_reasoning_effort_none_on_35_flash_uses_minimal(): + """35-flash DOES support minimal, so 'none' maps to it directly.""" + provider = _make_provider() + request = ChatRequest( + messages=[Message(role="user", content="Hello")], + reasoning_effort="none", + ) + config = _run_complete(provider, request, model="gemini-3.5-flash") + assert config.thinking_config.thinking_level.value == "MINIMAL" + + +# ============================================================ +# End-to-end: reasoning_effort -> legacy thinking_budget (2.x models) +# ============================================================ + + +@pytest.mark.parametrize( + "effort,expected_budget", + [ + ("none", 0), + ("minimal", 4096), + ("low", 4096), + ("medium", -1), + ("high", -1), + ("xhigh", -1), + ("max", -1), + ], +) +def test_reasoning_effort_legacy_mapping_on_25_flash(effort, expected_budget): + """gemini-2.5-flash rejects thinking_level entirely -- must use the + legacy numeric budget mapping instead, and NEVER send thinking_level.""" + provider = _make_provider() + request = ChatRequest( + messages=[Message(role="user", content="Hello")], + reasoning_effort=effort, + model="gemini-2.5-flash", + ) + config = _run_complete(provider, request) + assert config.thinking_config.thinking_budget == expected_budget + assert config.thinking_config.thinking_level is None + + +# ============================================================ +# Never send both thinking_budget and thinking_level +# ============================================================ + + +def test_explicit_thinking_budget_wins_alone_even_on_level_supporting_model(): + """Explicit thinking_budget (legacy override) takes absolute precedence + over reasoning_effort and is sent ALONE -- never combined with + thinking_level (Google 400s if both are present).""" + provider = _make_provider() + request = ChatRequest( + messages=[Message(role="user", content="Hello")], + reasoning_effort="high", # would otherwise map to thinking_level=HIGH + model="gemini-3.7-flash", + ) + config = _run_complete(provider, request, thinking_budget=2048) + assert config.thinking_config.thinking_budget == 2048 + assert config.thinking_config.thinking_level is None + + +def test_metadata_thinking_budget_wins_alone_on_level_supporting_model(): + provider = _make_provider() + request = ChatRequest( + messages=[Message(role="user", content="Hello")], + reasoning_effort="high", + model="gemini-3.7-flash", + metadata={"thinking_budget": 1024}, + ) + config = _run_complete(provider, request) + assert config.thinking_config.thinking_budget == 1024 + assert config.thinking_config.thinking_level is None From 6dc26c4a47750febde4dd68774da3917dc127db1 Mon Sep 17 00:00:00 2001 From: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com> Date: Sat, 29 Aug 2026 13:19:21 -0700 Subject: [PATCH 2/7] fix(thought-signatures): encode captured signatures to base64, not raw bytes MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Audited how assistant thought/tool-call parts are replayed in multi-turn conversation building, per this module's stateless full-resend design (the entire Message list is rebuilt from stored history and resent on every turn -- thought signatures MUST survive that unmodified or Gemini returns FinishReason MISSING_THOUGHT_SIGNATURE). Found a load-bearing gap: TextBlock.signature and ToolCallBlock/ ToolCall.signature were captured as the SDK's raw bytes, while ThinkingBlock.signature was already correctly encoded to a base64 str at capture time. Verified directly against amplifier_core's own models that this is not cosmetic: model_dump(mode="json") on a ToolCallBlock/TextBlock carrying a non-UTF-8 raw-bytes signature raises UnicodeDecodeError / PydanticSerializationError outright -- and an opaque cryptographic signature is essentially never valid UTF-8. Any code path that JSON- serializes the message history (session persistence, event logging, an orchestrator's model_dump(mode="json")) would crash or silently drop the signature the next time thinking needs to be resent. Fixes both capture sites to encode via the existing _encode_sig() helper, matching ThinkingBlock's contract exactly (str | None, base64 ASCII). The outbound path (_convert_messages) already called _encode_sig() again at send time for defense-in-depth -- that call is a no-op for an already-encoded str, so no outbound changes were needed; the gap was capture-side only. Updates the 3 existing tests whose assertions encoded raw-bytes equality into base64-str equality (documented inline). Adds test_inbound_signature_is_json_safe_not_raw_bytes, which uses a deliberately non-UTF-8 byte sequence and asserts model_dump(mode="json") succeeds and round-trips exactly -- the regression guard for the bug this audit found. 🤖 Generated with [Amplifier](https://github.com/microsoft/amplifier) Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com> --- amplifier_module_provider_gemini/__init__.py | 37 ++++++--- tests/test_thought_signatures.py | 81 +++++++++++++++++--- 2 files changed, 99 insertions(+), 19 deletions(-) diff --git a/amplifier_module_provider_gemini/__init__.py b/amplifier_module_provider_gemini/__init__.py index f486ee4..7423853 100644 --- a/amplifier_module_provider_gemini/__init__.py +++ b/amplifier_module_provider_gemini/__init__.py @@ -1910,16 +1910,32 @@ def _convert_to_chat_response( ) else: # Regular text (including final answer with thought_signature) - # Capture any thought_signature as an extra field (bytes) so the - # outbound path can echo it back to the API. - _text_sig = getattr(part, "thought_signature", None) + # Capture any thought_signature as an extra field so the + # outbound path can echo it back to the API. Encode to + # base64 str at capture time (matching ThinkingBlock's + # existing behavior below) rather than storing raw SDK + # bytes: this module is stateless full-resend, so the + # captured Message list is exactly what later gets + # replayed -- and, in practice, also exactly what gets + # JSON-serialized by session persistence, event logging, + # or any orchestrator that calls model_dump(mode="json"). + # Raw bytes containing non-UTF-8 sequences (the normal + # case for an opaque cryptographic signature) make that + # serialization crash outright -- verified directly + # against amplifier_core's own TextBlock/ToolCallBlock: + # model_dump(mode="json") raises UnicodeDecodeError for + # a signature like bytes([0xff, 0xfe, ...]). Encoding to + # base64 ASCII here makes the value JSON-safe everywhere + # it travels, matching ThinkingBlock's contract (its + # signature field is typed str | None for this reason). + _text_sig = _encode_sig(getattr(part, "thought_signature", None)) _text_kwargs: dict = ( {"signature": _text_sig} if _text_sig is not None else {} ) content_blocks.append(TextBlock(text=part.text, **_text_kwargs)) if _text_sig is not None: logger.debug( - "[PROVIDER] Gemini: captured thought_signature on text part (%d bytes)", + "[PROVIDER] Gemini: captured thought_signature on text part (%d chars, base64)", len(_text_sig), ) text_accumulator.append(part.text) @@ -1929,15 +1945,18 @@ def _convert_to_chat_response( fc = part.function_call tool_call_id = self._generate_tool_call_id() - # Capture thought_signature if present (Gemini 2.5+ thinking models). - # Store as raw bytes in an extra field so the outbound path can echo - # it back without an additional encode/decode round-trip. - _fc_sig = getattr(part, "thought_signature", None) + # Capture thought_signature if present (Gemini 2.5+ thinking + # models). Encoded to base64 str at capture time for the same + # JSON-safety reason as the text-part signature above -- + # verified directly that a raw-bytes ToolCallBlock/ToolCall + # signature fails model_dump(mode="json") for non-UTF-8 byte + # sequences (the normal case for an opaque signature). + _fc_sig = _encode_sig(getattr(part, "thought_signature", None)) _fc_kwargs: dict = {"signature": _fc_sig} if _fc_sig is not None else {} if _fc_sig is not None: logger.debug( "[PROVIDER] Gemini: captured thought_signature on function_call " - "part '%s' (%d bytes)", + "part '%s' (%d chars, base64)", fc.name, len(_fc_sig), ) diff --git a/tests/test_thought_signatures.py b/tests/test_thought_signatures.py index f2e350d..7fa8693 100644 --- a/tests/test_thought_signatures.py +++ b/tests/test_thought_signatures.py @@ -69,8 +69,14 @@ def _make_response(parts): def test_inbound_function_call_signature_captured(): - """function_call part with thought_signature -> ToolCallBlock.signature and ToolCall.signature.""" + """function_call part with thought_signature -> ToolCallBlock.signature and ToolCall.signature. + + Captured as a base64 str, NOT the SDK's raw bytes (see + test_inbound_signature_is_json_safe_not_raw_bytes for why: raw bytes + break JSON serialization for non-UTF-8 signatures, which is the normal + case for an opaque cryptographic signature).""" sig_bytes = b"\x01\x02\x03" + expected_b64 = base64.b64encode(sig_bytes).decode("ascii") fc = SimpleNamespace(name="todo", args={"content": "do something"}) part = SimpleNamespace(thought=False, function_call=fc, thought_signature=sig_bytes) response = _make_response([part]) @@ -81,23 +87,27 @@ def test_inbound_function_call_signature_captured(): # ToolCallBlock in content assert result.content, "Expected content blocks" tc_block = result.content[0] - assert getattr(tc_block, "signature", None) == sig_bytes, ( - f"ToolCallBlock.signature should be {sig_bytes!r}, " + assert getattr(tc_block, "signature", None) == expected_b64, ( + f"ToolCallBlock.signature should be base64 {expected_b64!r}, " f"got {getattr(tc_block, 'signature', None)!r}" ) # ToolCall in tool_calls list assert result.tool_calls, "Expected tool_calls" tc = result.tool_calls[0] - assert getattr(tc, "signature", None) == sig_bytes, ( - f"ToolCall.signature should be {sig_bytes!r}, " + assert getattr(tc, "signature", None) == expected_b64, ( + f"ToolCall.signature should be base64 {expected_b64!r}, " f"got {getattr(tc, 'signature', None)!r}" ) def test_inbound_text_signature_captured(): - """Non-thought text part with thought_signature -> TextBlock.signature.""" + """Non-thought text part with thought_signature -> TextBlock.signature. + + Captured as a base64 str, NOT the SDK's raw bytes -- see + test_inbound_signature_is_json_safe_not_raw_bytes.""" sig_bytes = b"\x04\x05\x06" + expected_b64 = base64.b64encode(sig_bytes).decode("ascii") part = SimpleNamespace(text="final answer", thought=False, thought_signature=sig_bytes) response = _make_response([part]) @@ -106,8 +116,8 @@ def test_inbound_text_signature_captured(): assert result.content, "Expected content blocks" tb = result.content[0] - assert getattr(tb, "signature", None) == sig_bytes, ( - f"TextBlock.signature should be {sig_bytes!r}, " + assert getattr(tb, "signature", None) == expected_b64, ( + f"TextBlock.signature should be base64 {expected_b64!r}, " f"got {getattr(tb, 'signature', None)!r}" ) @@ -343,9 +353,9 @@ def _fc_part(name, sig=None): provider = _make_provider() chat_response = provider._convert_to_chat_response(response) - # Verify inbound: only first TC captured a signature + # Verify inbound: only first TC captured a signature (as base64 str) assert len(chat_response.tool_calls) == 3 - assert getattr(chat_response.tool_calls[0], "signature", None) == sig_bytes + assert getattr(chat_response.tool_calls[0], "signature", None) == expected_b64 assert getattr(chat_response.tool_calls[1], "signature", None) is None assert getattr(chat_response.tool_calls[2], "signature", None) is None @@ -370,3 +380,54 @@ def _fc_part(name, sig=None): assert "thought_signature" not in parts_out[2], ( f"Third function_call should NOT have thought_signature, got: {parts_out[2]}" ) + + +# ============================================================ +# JSON-safety regression (the actual load-bearing bug this audit found) +# ============================================================ + + +def test_inbound_signature_is_json_safe_not_raw_bytes(): + """Captured signatures must be JSON-safe (base64 str), never raw bytes. + + This module is stateless full-resend: the entire Message list gets + rebuilt from the stored conversation history on every turn. That stored + history commonly crosses a JSON boundary somewhere in the stack (session + persistence, event logging, an orchestrator calling + model_dump(mode="json")). A raw-bytes signature containing a byte + sequence that isn't valid UTF-8 -- the NORMAL case for an opaque + cryptographic signature -- crashes that serialization outright. + + Verified directly against amplifier_core's own models before this fix: + ToolCallBlock(signature=bytes([0xff, 0xfe, 0x80])).model_dump(mode="json") + raised UnicodeDecodeError. ThinkingBlock was never affected (its + signature field is already typed str | None and was already encoded at + capture time) -- only TextBlock and ToolCallBlock/ToolCall had the bug. + """ + # A byte sequence that is NOT valid UTF-8 (the realistic case). + sig_bytes = bytes([0xFF, 0xFE, 0x80, 0x81, 0x00, 0x9D]) + + fc = SimpleNamespace(name="todo", args={"content": "x"}) + fc_part = SimpleNamespace(thought=False, function_call=fc, thought_signature=sig_bytes) + text_part = SimpleNamespace(text="answer", thought=False, thought_signature=sig_bytes) + response = _make_response([text_part, fc_part]) + + provider = _make_provider() + result = provider._convert_to_chat_response(response) + + text_block, tc_block = result.content[0], result.content[1] + + # Both must be plain base64 str, not bytes -- and both round-trip + # through the exact JSON serialization path that used to crash. + assert isinstance(text_block.signature, str) + assert isinstance(tc_block.signature, str) + assert isinstance(result.tool_calls[0].signature, str) + + text_block.model_dump(mode="json") + tc_block.model_dump(mode="json") + result.tool_calls[0].model_dump(mode="json") + + # And the value is recoverable: decoding it gives back the exact + # original bytes (nothing lost, nothing mangled). + assert base64.b64decode(text_block.signature) == sig_bytes + assert base64.b64decode(tc_block.signature) == sig_bytes From 1864cc16c42290374c7168ea3de638ced3373b6d Mon Sep 17 00:00:00 2001 From: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com> Date: Sat, 29 Aug 2026 13:22:40 -0700 Subject: [PATCH 3/7] feat(model): default gemini-2.5-flash -> gemini-3.7-flash MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit gemini-3.7-flash is Google's current flagship Flash model (ai.google.dev describes it as "the latest and most capable" Flash) -- verified present in list_models() against this account's live key on 2026-08-29 (40 gemini-* models served). gemini-2.5-flash is documented as two generations back; gemini-3.5-flash is documented as legacy. Updates GeminiProvider.default_model's fallback and get_info()'s defaults["model"]. Deliberately does NOT add a default_model ConfigField -- the app-cli model picker phase already collects this from the user. Existing tests that depend on the OLD default's specific legacy thinking_budget mapping (test_reasoning_effort.py) now pin default_model="gemini-2.5-flash" explicitly in their provider fixture, with a module docstring note pointing to test_thinking_level.py for the comprehensive reasoning_effort -> thinking_level/budget matrix across both model families. No test asserted get_info()'s previous default model value directly. README default-model documentation is updated in the README-overhaul commit rather than here, to avoid two commits touching the same prose. 🤖 Generated with [Amplifier](https://github.com/microsoft/amplifier) Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com> --- amplifier_module_provider_gemini/__init__.py | 8 ++++++-- tests/test_reasoning_effort.py | 17 ++++++++++++++++- tests/test_thinking_level.py | 3 +-- 3 files changed, 23 insertions(+), 5 deletions(-) diff --git a/amplifier_module_provider_gemini/__init__.py b/amplifier_module_provider_gemini/__init__.py index 7423853..72101ad 100644 --- a/amplifier_module_provider_gemini/__init__.py +++ b/amplifier_module_provider_gemini/__init__.py @@ -451,7 +451,11 @@ def __init__( ) self.config = config or {} self.coordinator = coordinator - self.default_model = self.config.get("default_model", "gemini-2.5-flash") + # gemini-3.7-flash is the current flagship Flash model (verified + # live against this account's key on 2026-08-29: present in + # list_models(), 40 gemini-* models served). gemini-2.5-flash is two + # generations back; gemini-3.5-flash is documented as legacy. + self.default_model = self.config.get("default_model", "gemini-3.7-flash") self.max_tokens = self.config.get("max_tokens", 8192) self.temperature = self.config.get("temperature", 0.7) self.timeout = self.config.get("timeout", 600.0) @@ -524,7 +528,7 @@ def get_info(self) -> ProviderInfo: credential_env_vars=["GOOGLE_API_KEY", "GEMINI_API_KEY"], capabilities=["streaming", "tools", "thinking", "json_mode", "batch"], defaults={ - "model": "gemini-2.5-flash", + "model": "gemini-3.7-flash", "max_tokens": 8192, "temperature": 0.7, "timeout": 600.0, diff --git a/tests/test_reasoning_effort.py b/tests/test_reasoning_effort.py index 10b4dbc..f32bb05 100644 --- a/tests/test_reasoning_effort.py +++ b/tests/test_reasoning_effort.py @@ -2,6 +2,14 @@ Verifies that request.reasoning_effort maps to thinking_budget values, and that kwargs["thinking_budget"] overrides reasoning_effort. + +These tests deliberately pin default_model to gemini-2.5-flash, a model +verified live to reject thinking_level entirely (see +_THINKING_LEVEL_TABLE) -- this file exercises the legacy numeric budget +mapping specifically. The provider's actual default model is +gemini-3.7-flash (a level-supporting model); see tests/test_thinking_level.py +for the full reasoning_effort -> thinking_level matrix and the per-model +clamping/legacy-fallback behavior. """ import asyncio @@ -42,7 +50,14 @@ def _make_gemini_response(): def _make_provider() -> GeminiProvider: - provider = GeminiProvider(api_key="test-key", config={"max_retries": 0, "use_streaming": False}) + provider = GeminiProvider( + api_key="test-key", + config={ + "max_retries": 0, + "use_streaming": False, + "default_model": "gemini-2.5-flash", + }, + ) provider.coordinator = cast(ModuleCoordinator, FakeCoordinator()) return provider diff --git a/tests/test_thinking_level.py b/tests/test_thinking_level.py index ef60864..a862623 100644 --- a/tests/test_thinking_level.py +++ b/tests/test_thinking_level.py @@ -268,9 +268,8 @@ def test_reasoning_effort_legacy_mapping_on_25_flash(effort, expected_budget): request = ChatRequest( messages=[Message(role="user", content="Hello")], reasoning_effort=effort, - model="gemini-2.5-flash", ) - config = _run_complete(provider, request) + config = _run_complete(provider, request, model="gemini-2.5-flash") assert config.thinking_config.thinking_budget == expected_budget assert config.thinking_config.thinking_level is None From e70ffbc2c44e2dd201c95141b367895808e9c44c Mon Sep 17 00:00:00 2001 From: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com> Date: Sat, 29 Aug 2026 13:28:14 -0700 Subject: [PATCH 4/7] fix(config): coerce bool/numeric config, sweep unknown keys, flag inert ones MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Config commonly arrives as strings: the app-cli wizard writes field_type="boolean" values as the literal strings "true"/"false" (not Python bools), and hand-edited YAML often quotes both booleans and numbers. This provider had no coercion at all for several config keys: - raw / use_streaming / retry_jitter: naive `bool(raw)` (or a bare truthiness check) is wrong for a string -- `bool("false")` is True, since any non-empty string is truthy, silently inverting the flag. - timeout / max_tokens / temperature / priority: no coercion at all. A string timeout survived uncoerced all the way to asyncio.wait_for(timeout=...), failing confusingly on the first real API call instead of at mount. - max_retries / min_retry_delay / max_retry_delay: bare int()/float() RAISED ValueError at mount time for any unparseable string, crashing the whole provider mount over one bad config value. Adds _parse_config_bool and _parse_config_number (ported from the established pattern in amplifier-module-provider-openai / amplifier-module-provider-github-copilot): warn-and-default, never raise, for both bool and numeric config coercion. Every config-reading line in GeminiProvider.__init__ now goes through one of these two helpers. Adds an unknown-config-key sweep (_sweep_unknown_config_keys), run at mount, with three tiers of message: 1. A known documentation ghost (debug / raw_debug / debug_truncate_length -- verified by grep that no self.config.get(...) call site reads any of them) gets a specific, helpful explanation instead of a generic warning. 2. A likely typo of a real key gets a difflib "did you mean X?" suggestion. 3. Anything else gets a generic "unrecognized, ignored" plus the full list of keys this provider actually reads. The allowlist (_CONSUMED_CONFIG_KEYS) is exactly the 13 keys this module's __init__ reads via self.config.get(...), including `priority` -- which this module only stores for the orchestrator's own provider-selection logic to read, never used internally, but is a real consumed key, not a typo to flag. tests/test_config_hygiene.py: unit tests for both coercion helpers (including the exact "false" string bug, the bool-is-an-int-subclass guard, and warn-vs-raise for garbage input) plus end-to-end GeminiProvider.__init__ tests proving string config values are coerced and a bad numeric string no longer raises at mount, and coverage for all three sweep message tiers. 🤖 Generated with [Amplifier](https://github.com/microsoft/amplifier) Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com> --- amplifier_module_provider_gemini/__init__.py | 216 ++++++++++++++++++- tests/test_config_hygiene.py | 210 ++++++++++++++++++ 2 files changed, 416 insertions(+), 10 deletions(-) create mode 100644 tests/test_config_hygiene.py diff --git a/amplifier_module_provider_gemini/__init__.py b/amplifier_module_provider_gemini/__init__.py index 72101ad..d335cef 100644 --- a/amplifier_module_provider_gemini/__init__.py +++ b/amplifier_module_provider_gemini/__init__.py @@ -10,6 +10,7 @@ import asyncio import base64 +import difflib from collections import defaultdict from collections.abc import Callable from decimal import Decimal @@ -410,6 +411,181 @@ def _clamp_thinking_level(model: str, requested: str, supported: tuple[str, ...] return supported[0] # pragma: no cover -- defensive; supported is never empty +# --------------------------------------------------------------------------- +# Config hygiene: bool/numeric coercion, unknown-key sweep, inert-key notes +# --------------------------------------------------------------------------- +# Config commonly arrives as strings: the app-cli wizard writes +# `field_type="boolean"` values as the literal strings "true"/"false" (not +# Python bools), and hand-edited YAML often quotes both booleans and +# numbers. Naive `bool(raw)` or bare `int(raw)`/`float(raw)` are both wrong +# for that -- `bool("false")` is True (any non-empty string is truthy), +# silently inverting the operator's intent, and a bad numeric string +# currently either raises at mount time (max_retries etc., which call +# int()/float() directly) or survives uncoerced all the way to +# asyncio.wait_for(timeout=...), which fails on the FIRST real API call +# with a confusing low-level TypeError instead of a clear config error. + +_CONFIG_BOOL_TRUE_STRINGS: frozenset[str] = frozenset({"true", "1", "yes"}) +_CONFIG_BOOL_FALSE_STRINGS: frozenset[str] = frozenset({"false", "0", "no"}) + + +def _parse_config_bool(key: str, raw: Any, default: bool) -> bool: + """Parse a boolean-ish provider-config value, tolerating string bools. + + Accepts: + - key absent / value None / value "" -> `default` + - real bool -> itself, unchanged + - str in {"true", "1", "yes"} (case-insensitive, stripped) -> True + - str in {"false", "0", "no"} (case-insensitive, stripped) -> False + + Anything else logs a warning and falls back to `default` -- config + hygiene here is warn-and-default, not raise, so one typo'd flag doesn't + take down the whole provider mount. + """ + if raw is None or raw == "": + return default + if isinstance(raw, bool): + return raw + if isinstance(raw, str): + normalized = raw.strip().lower() + if normalized in _CONFIG_BOOL_TRUE_STRINGS: + return True + if normalized in _CONFIG_BOOL_FALSE_STRINGS: + return False + logger.warning( + "[PROVIDER] Gemini: invalid config %r=%r (expected true/false, " + "also 1/0, yes/no; case-insensitive) -- using default %r", + key, + raw, + default, + ) + return default + + +def _parse_config_number(key: str, raw: Any, default: Any, cast) -> Any: + """Parse a numeric provider-config value, tolerating string numbers. + + Accepts real int/float values and numeric strings (whitespace-stripped), + coercing via `cast` (int or float). Anything unparseable -- including + key absent / None / "" -- logs a warning and falls back to `default`. + Never raises: a bad numeric config value degrades to a safe default + instead of crashing the provider at mount time or, worse, surviving + uncoerced as a string into a low-level call (e.g. + asyncio.wait_for(timeout="600")) that fails confusingly on the first + real request instead of at mount. + """ + if raw is None or raw == "": + return default + if isinstance(raw, bool): + pass # bool is a subclass of int -- never accept it as numeric config + elif isinstance(raw, (int, float)): + try: + return cast(raw) + except (TypeError, ValueError): + pass + elif isinstance(raw, str): + try: + return cast(raw.strip()) + except (TypeError, ValueError): + pass + logger.warning( + "[PROVIDER] Gemini: invalid config %r=%r (expected a %s) -- using " + "default %r", + key, + raw, + cast.__name__, + default, + ) + return default + + +# Config keys this provider actually reads (self.config.get(...) call +# sites). `priority` is included even though this module only stores it +# on self.priority for the orchestrator's provider-selection logic to read +# -- it is a real, consumed key, never a typo to flag. +_CONSUMED_CONFIG_KEYS: frozenset[str] = frozenset( + { + "api_key", + "default_model", + "max_tokens", + "temperature", + "timeout", + "priority", + "raw", + "use_streaming", + "max_retries", + "min_retry_delay", + "max_retry_delay", + "retry_jitter", + "max_concurrent_requests", + } +) + +# Keys that appeared in past README revisions describing features that were +# never actually implemented in this module (verified by grep: no +# `self.config.get(...)` call site reads any of them). These are not typos +# -- they're documentation ghosts a user may reasonably still have in their +# config from an older guide -- so they get a specific, helpful message +# instead of a generic "did you mean" guess. +_KNOWN_INERT_CONFIG_KEYS: dict[str, str] = { + "debug": ( + "documented in older README revisions but never implemented -- no " + "llm:request:debug/llm:response:debug events exist in this " + "provider. Setting it has no effect." + ), + "raw_debug": ( + "documented in older README revisions but never implemented -- no " + "llm:request:raw/llm:response:raw events exist in this provider. " + "Setting it has no effect. (This provider's actual raw-I/O capture " + "is the differently-named 'raw' config key, which IS implemented.)" + ), + "debug_truncate_length": ( + "documented in older README revisions but never implemented -- " + "this provider has no debug-log truncation path. Setting it has " + "no effect." + ), +} + + +def _sweep_unknown_config_keys(config: dict[str, Any]) -> None: + """Warn (never raise) about config keys this provider doesn't consume. + + Three distinct messages, in priority order: + 1. A known documentation ghost (_KNOWN_INERT_CONFIG_KEYS) -- specific, + helpful explanation of why it does nothing. + 2. A likely typo of a real key (difflib match) -- "did you mean X?". + 3. Anything else -- generic "unrecognized, ignored" with the full + list of keys this provider actually reads. + """ + for key in config: + if key in _CONSUMED_CONFIG_KEYS: + continue + if key in _KNOWN_INERT_CONFIG_KEYS: + logger.warning( + "[PROVIDER] Gemini: config key %r is inert -- %s", + key, + _KNOWN_INERT_CONFIG_KEYS[key], + ) + continue + suggestions = difflib.get_close_matches( + key, _CONSUMED_CONFIG_KEYS, n=1 + ) + if suggestions: + logger.warning( + "[PROVIDER] Gemini: unknown config key %r -- did you mean " + "%r? (unrecognized keys are ignored)", + key, + suggestions[0], + ) + else: + logger.warning( + "[PROVIDER] Gemini: unknown config key %r -- ignored. " + "Recognized keys: %s", + key, + sorted(_CONSUMED_CONFIG_KEYS), + ) + + class GeminiChatResponse(ChatResponse): """ChatResponse with additional fields for streaming UI compatibility.""" @@ -455,20 +631,40 @@ def __init__( # live against this account's key on 2026-08-29: present in # list_models(), 40 gemini-* models served). gemini-2.5-flash is two # generations back; gemini-3.5-flash is documented as legacy. + _sweep_unknown_config_keys(self.config) + self.default_model = self.config.get("default_model", "gemini-3.7-flash") - self.max_tokens = self.config.get("max_tokens", 8192) - self.temperature = self.config.get("temperature", 0.7) - self.timeout = self.config.get("timeout", 600.0) - self.priority = self.config.get("priority", 100) - self.raw = self.config.get("raw", False) - self.use_streaming = self.config.get("use_streaming", True) + self.max_tokens = _parse_config_number( + "max_tokens", self.config.get("max_tokens"), 8192, int + ) + self.temperature = _parse_config_number( + "temperature", self.config.get("temperature"), 0.7, float + ) + self.timeout = _parse_config_number( + "timeout", self.config.get("timeout"), 600.0, float + ) + self.priority = _parse_config_number( + "priority", self.config.get("priority"), 100, int + ) + self.raw = _parse_config_bool("raw", self.config.get("raw"), False) + self.use_streaming = _parse_config_bool( + "use_streaming", self.config.get("use_streaming"), True + ) # Retry configuration — delegates to shared retry_with_backoff() from amplifier-core. self._retry_config = RetryConfig( - max_retries=int(self.config.get("max_retries", 5)), - initial_delay=float(self.config.get("min_retry_delay", 1.0)), - max_delay=float(self.config.get("max_retry_delay", 60.0)), - jitter=bool(self.config.get("retry_jitter", True)), + max_retries=_parse_config_number( + "max_retries", self.config.get("max_retries"), 5, int + ), + initial_delay=_parse_config_number( + "min_retry_delay", self.config.get("min_retry_delay"), 1.0, float + ), + max_delay=_parse_config_number( + "max_retry_delay", self.config.get("max_retry_delay"), 60.0, float + ), + jitter=_parse_config_bool( + "retry_jitter", self.config.get("retry_jitter"), True + ), ) # Process-wide concurrency gate. diff --git a/tests/test_config_hygiene.py b/tests/test_config_hygiene.py new file mode 100644 index 0000000..3f3680c --- /dev/null +++ b/tests/test_config_hygiene.py @@ -0,0 +1,210 @@ +"""Tests for provider config hygiene: bool/numeric coercion, unknown-key +sweep with did-you-mean, and targeted inert-key messages. + +Covers the fail-before/pass-after behaviors this hardening fixes: +- A config value of the STRING "false" (as written by the app-cli wizard + for boolean fields, or hand-edited quoted YAML) used to be silently + truthy (`bool("false") is True`), inverting the operator's intent. +- A config value of the STRING "600" for timeout/max_tokens/temperature + used to survive uncoerced all the way to asyncio.wait_for(timeout=...), + failing confusingly on the first real API call instead of at mount. +- max_retries/min_retry_delay/max_retry_delay used bare int()/float() that + raised ValueError at mount time for a bad string; now warn-and-default. +- Unknown config keys (typos) are silently ignored with no signal at all. +- The three README "ghost" keys (debug, raw_debug, debug_truncate_length) + look like unknown keys but deserve a specific explanation, not a generic + "did you mean" guess. +""" + +from amplifier_module_provider_gemini import GeminiProvider +from amplifier_module_provider_gemini import _parse_config_bool +from amplifier_module_provider_gemini import _parse_config_number + + +class FakeHooks: + def __init__(self): + self.events: list[tuple[str, dict]] = [] + + async def emit(self, name: str, payload: dict) -> None: + self.events.append((name, payload)) + + +class FakeCoordinator: + def __init__(self): + self.hooks = FakeHooks() + + +# ============================================================ +# _parse_config_bool +# ============================================================ + + +def test_bool_real_true_false_unchanged(): + assert _parse_config_bool("raw", True, False) is True + assert _parse_config_bool("raw", False, True) is False + + +def test_bool_string_false_is_actually_false(): + """The exact bug: bool("false") is True in naive Python -- must not be here.""" + assert _parse_config_bool("use_streaming", "false", True) is False + assert _parse_config_bool("use_streaming", "False", True) is False + assert _parse_config_bool("use_streaming", " FALSE ", True) is False + + +def test_bool_string_true_variants(): + assert _parse_config_bool("retry_jitter", "true", False) is True + assert _parse_config_bool("retry_jitter", "1", False) is True + assert _parse_config_bool("retry_jitter", "yes", False) is True + + +def test_bool_string_false_variants(): + assert _parse_config_bool("retry_jitter", "0", True) is False + assert _parse_config_bool("retry_jitter", "no", True) is False + + +def test_bool_absent_or_empty_uses_default(): + assert _parse_config_bool("raw", None, True) is True + assert _parse_config_bool("raw", "", False) is False + + +def test_bool_garbage_warns_and_defaults(caplog): + with caplog.at_level("WARNING"): + result = _parse_config_bool("raw", "maybe", False) + assert result is False + assert any("invalid config 'raw'" in rec.message for rec in caplog.records) + + +# ============================================================ +# _parse_config_number +# ============================================================ + + +def test_number_real_values_cast(): + assert _parse_config_number("timeout", 600, 300.0, float) == 600.0 + assert _parse_config_number("max_tokens", 8192, 100, int) == 8192 + + +def test_number_string_values_coerced(): + """The exact bug: a string timeout used to survive to asyncio.wait_for.""" + assert _parse_config_number("timeout", "600", 300.0, float) == 600.0 + assert _parse_config_number("max_retries", "5", 3, int) == 5 + assert _parse_config_number("temperature", " 0.9 ", 0.7, float) == 0.9 + + +def test_number_absent_or_empty_uses_default(): + assert _parse_config_number("timeout", None, 300.0, float) == 300.0 + assert _parse_config_number("timeout", "", 300.0, float) == 300.0 + + +def test_number_garbage_warns_and_defaults_never_raises(caplog): + with caplog.at_level("WARNING"): + result = _parse_config_number("timeout", "not-a-number", 300.0, float) + assert result == 300.0 + assert any("invalid config 'timeout'" in rec.message for rec in caplog.records) + + +def test_number_bool_rejected_as_numeric(caplog): + """bool is an int subclass in Python -- must not silently become 0/1.""" + with caplog.at_level("WARNING"): + result = _parse_config_number("max_tokens", True, 8192, int) + assert result == 8192 + + +# ============================================================ +# End-to-end: GeminiProvider.__init__ applies coercion +# ============================================================ + + +def test_provider_init_coerces_string_config_values(caplog): + provider = GeminiProvider( + api_key="test-key", + config={ + "timeout": "45", + "max_tokens": "2048", + "temperature": "0.3", + "use_streaming": "false", + "raw": "true", + "max_retries": "2", + "retry_jitter": "false", + }, + ) + assert provider.timeout == 45.0 + assert provider.max_tokens == 2048 + assert provider.temperature == 0.3 + assert provider.use_streaming is False + assert provider.raw is True + assert provider._retry_config.max_retries == 2 + # RetryConfig itself coerces its bool 'jitter' constructor arg into an + # internal jitter FACTOR (0.0 disabled / 0.2 enabled) -- that's + # RetryConfig's own contract, unrelated to this fix. What matters here + # is that _parse_config_bool resolved the string "false" to the real + # Python bool False before it ever reached RetryConfig. + assert not provider._retry_config.jitter + + +def test_provider_init_never_raises_on_bad_max_retries_string(): + """Pre-fix: int(self.config.get("max_retries", 5)) raised ValueError + at mount time for a bad string. Now warns and falls back to 5.""" + provider = GeminiProvider( + api_key="test-key", config={"max_retries": "not-a-number"} + ) + assert provider._retry_config.max_retries == 5 + + +# ============================================================ +# Unknown-key sweep + inert-key messages +# ============================================================ + + +def test_unknown_key_typo_suggests_close_match(caplog): + with caplog.at_level("WARNING"): + GeminiProvider(api_key="test-key", config={"max_toekns": 100}) + assert any( + "unknown config key 'max_toekns'" in rec.message + and "did you mean 'max_tokens'" in rec.message + for rec in caplog.records + ), f"got: {[r.message for r in caplog.records]}" + + +def test_unknown_key_no_close_match_gets_generic_message(caplog): + with caplog.at_level("WARNING"): + GeminiProvider(api_key="test-key", config={"completely_unrelated_xyz": 1}) + assert any( + "unknown config key 'completely_unrelated_xyz'" in rec.message + and "ignored" in rec.message + for rec in caplog.records + ) + + +def test_known_inert_keys_get_specific_messages(caplog): + with caplog.at_level("WARNING"): + GeminiProvider( + api_key="test-key", + config={"debug": True, "raw_debug": True, "debug_truncate_length": 180}, + ) + messages = [rec.message for rec in caplog.records] + assert any("'debug' is inert" in m and "never implemented" in m for m in messages) + assert any("'raw_debug' is inert" in m for m in messages) + assert any("'debug_truncate_length' is inert" in m for m in messages) + + +def test_recognized_keys_produce_no_warnings(caplog): + with caplog.at_level("WARNING"): + GeminiProvider( + api_key="test-key", + config={ + "default_model": "gemini-3.7-flash", + "max_tokens": 8192, + "temperature": 0.7, + "timeout": 600.0, + "priority": 50, + "raw": False, + "use_streaming": True, + "max_retries": 5, + "min_retry_delay": 1.0, + "max_retry_delay": 60.0, + "retry_jitter": True, + "max_concurrent_requests": 5, + }, + ) + assert caplog.records == [] From 5d9724096d339e3a829dd7077b91bffedfc0cda0 Mon Sep 17 00:00:00 2001 From: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com> Date: Sat, 29 Aug 2026 13:31:45 -0700 Subject: [PATCH 5/7] feat(config): rename max_tokens -> max_output_tokens (Google's own param name) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit max_output_tokens is the actual field name in Google's API (already used directly in this module's own generate_content call: GenerateContentConfig(max_output_tokens=...)) -- the provider's config surface previously called the same knob "max_tokens", a needless mismatch for anyone cross-referencing Google's docs while writing a bundle config. Adds _read_renamed_config(config, new, old): reads the new key, falling back to the deprecated old key with exactly one warning -- only when the old key is the value actually used, so a fully-migrated config stays silent. The new key always wins when both are set, with its own distinct warning naming the winner. Both 'max_output_tokens' and the deprecated 'max_tokens' alias are in _CONSUMED_CONFIG_KEYS (neither trips the unknown-key sweep from the previous commit). String values still coerce correctly through either name, since _read_renamed_config's result flows into the same _parse_config_number this module already uses. tests/test_config_hygiene.py: the new key alone, the old key alone (with its deprecation warning), both set together (new wins, with its own warning), a string value through the old alias, and neither set (default unchanged at 8192). Updates the existing "recognized keys produce no warnings" test to use the new canonical key name. 🤖 Generated with [Amplifier](https://github.com/microsoft/amplifier) Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com> --- amplifier_module_provider_gemini/__init__.py | 53 ++++++++++++++++++-- tests/test_config_hygiene.py | 46 ++++++++++++++++- 2 files changed, 95 insertions(+), 4 deletions(-) diff --git a/amplifier_module_provider_gemini/__init__.py b/amplifier_module_provider_gemini/__init__.py index d335cef..f7bd1c8 100644 --- a/amplifier_module_provider_gemini/__init__.py +++ b/amplifier_module_provider_gemini/__init__.py @@ -499,15 +499,59 @@ def _parse_config_number(key: str, raw: Any, default: Any, cast) -> Any: return default +def _read_renamed_config(config: dict[str, Any], new: str, old: str) -> Any: + """Read `new`, falling back to the deprecated `old` with one warning. + + The new key always wins when present (even when both are set) -- a + config that has already been migrated to the new name is never + silently overridden by a stale leftover of the old one. The warning + fires only when `old` is the value actually used, so a fully migrated + config stays silent and a config carrying both is told plainly which + one won. + + Returns None (not a sentinel) when neither key is present, so callers + can pass the result straight into _parse_config_bool/_parse_config_number, + which already treat None as "use my own default". + """ + new_val = config.get(new) + old_val = config.get(old) + if new_val not in (None, ""): + if old_val not in (None, ""): + logger.warning( + "[PROVIDER] Gemini: config keys '%s' (deprecated) and '%s' " + "are BOTH set; '%s' wins. Remove '%s'.", + old, + new, + new, + old, + ) + return new_val + if old_val not in (None, ""): + logger.warning( + "[PROVIDER] Gemini: config key '%s' is deprecated -- use '%s' " + "instead (this is Google's own API parameter name). Falling " + "back to '%s'=%r for this session.", + old, + new, + old, + old_val, + ) + return old_val + return None + + # Config keys this provider actually reads (self.config.get(...) call # sites). `priority` is included even though this module only stores it # on self.priority for the orchestrator's provider-selection logic to read -# -- it is a real, consumed key, never a typo to flag. +# -- it is a real, consumed key, never a typo to flag. `max_tokens` is kept +# as the deprecated back-compat alias for `max_output_tokens` (Google's own +# API parameter name) -- see _read_renamed_config. _CONSUMED_CONFIG_KEYS: frozenset[str] = frozenset( { "api_key", "default_model", - "max_tokens", + "max_output_tokens", + "max_tokens", # deprecated alias for max_output_tokens "temperature", "timeout", "priority", @@ -635,7 +679,10 @@ def __init__( self.default_model = self.config.get("default_model", "gemini-3.7-flash") self.max_tokens = _parse_config_number( - "max_tokens", self.config.get("max_tokens"), 8192, int + "max_output_tokens", + _read_renamed_config(self.config, "max_output_tokens", "max_tokens"), + 8192, + int, ) self.temperature = _parse_config_number( "temperature", self.config.get("temperature"), 0.7, float diff --git a/tests/test_config_hygiene.py b/tests/test_config_hygiene.py index 3f3680c..74a811b 100644 --- a/tests/test_config_hygiene.py +++ b/tests/test_config_hygiene.py @@ -194,7 +194,7 @@ def test_recognized_keys_produce_no_warnings(caplog): api_key="test-key", config={ "default_model": "gemini-3.7-flash", - "max_tokens": 8192, + "max_output_tokens": 8192, "temperature": 0.7, "timeout": 600.0, "priority": 50, @@ -208,3 +208,47 @@ def test_recognized_keys_produce_no_warnings(caplog): }, ) assert caplog.records == [] + + +# ============================================================ +# max_tokens -> max_output_tokens rename (back-compat alias) +# ============================================================ + + +def test_max_output_tokens_is_the_canonical_key(): + provider = GeminiProvider(api_key="test-key", config={"max_output_tokens": 4096}) + assert provider.max_tokens == 4096 + + +def test_max_tokens_still_works_as_deprecated_alias(caplog): + with caplog.at_level("WARNING"): + provider = GeminiProvider(api_key="test-key", config={"max_tokens": 2048}) + assert provider.max_tokens == 2048 + assert any( + "'max_tokens' is deprecated" in rec.message + and "'max_output_tokens'" in rec.message + for rec in caplog.records + ), f"got: {[r.message for r in caplog.records]}" + + +def test_max_output_tokens_wins_when_both_set(caplog): + with caplog.at_level("WARNING"): + provider = GeminiProvider( + api_key="test-key", + config={"max_output_tokens": 4096, "max_tokens": 2048}, + ) + assert provider.max_tokens == 4096 + assert any( + "are BOTH set" in rec.message and "'max_output_tokens' wins" in rec.message + for rec in caplog.records + ) + + +def test_max_tokens_string_value_still_coerced_through_alias(): + provider = GeminiProvider(api_key="test-key", config={"max_tokens": "3000"}) + assert provider.max_tokens == 3000 + + +def test_neither_key_set_uses_default(): + provider = GeminiProvider(api_key="test-key", config={}) + assert provider.max_tokens == 8192 From 8873eb04d9fb7c6e2617b31aa0c109b62675f075 Mon Sep 17 00:00:00 2001 From: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com> Date: Sat, 29 Aug 2026 13:34:41 -0700 Subject: [PATCH 6/7] feat(config): extra_request_params -- settings-only GenerateContentConfig escape hatch MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds extra_request_params: a dict of arbitrary GenerateContentConfig fields (safety_settings, top_p, top_k, seed, stop_sequences, presence_penalty/frequency_penalty, response_mime_type, labels, and any other field google-genai's GenerateContentConfig defines) merged LAST, after this provider's own computed values (temperature, max_output_tokens, thinking_config, tools). Deliberately settings-only -- never a ConfigField / interactive wizard prompt -- since it's an owner-beware power-user knob, not something to walk a user through. _apply_extra_request_params(config, extra_request_params): - The caller's value always wins over anything this provider already computed, and wins LOUDLY: overriding a non-None field logs a warning naming the field, the provider's own value, and the override. - An unrecognized field name (checked against GenerateContentConfig's own model_fields, not a hardcoded list) warns and is skipped -- never raises, since a typo in settings.yaml shouldn't crash the provider mount. - Single merge site: both the streaming and non-streaming call paths in _complete_chat_request share the same `config` object this mutates, so there is nowhere for the two paths to drift. tests/test_extra_request_params.py: unit tests for the merge helper (unexposed-field merge, no-op on empty/None, loud override with the exact warning content, unknown field skipped not raised) plus end-to-end tests through complete() proving extra_request_params reaches the real GenerateContentConfig sent to the API, and that get_info() does not list it as a ConfigField. 🤖 Generated with [Amplifier](https://github.com/microsoft/amplifier) Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com> --- amplifier_module_provider_gemini/__init__.py | 61 ++++++- tests/test_extra_request_params.py | 159 +++++++++++++++++++ 2 files changed, 219 insertions(+), 1 deletion(-) create mode 100644 tests/test_extra_request_params.py diff --git a/amplifier_module_provider_gemini/__init__.py b/amplifier_module_provider_gemini/__init__.py index f7bd1c8..c36e8eb 100644 --- a/amplifier_module_provider_gemini/__init__.py +++ b/amplifier_module_provider_gemini/__init__.py @@ -545,7 +545,9 @@ def _read_renamed_config(config: dict[str, Any], new: str, old: str) -> Any: # on self.priority for the orchestrator's provider-selection logic to read # -- it is a real, consumed key, never a typo to flag. `max_tokens` is kept # as the deprecated back-compat alias for `max_output_tokens` (Google's own -# API parameter name) -- see _read_renamed_config. +# API parameter name) -- see _read_renamed_config. `extra_request_params` +# is a settings-only escape hatch (never a ConfigField / interactive +# wizard prompt) -- see _apply_extra_request_params. _CONSUMED_CONFIG_KEYS: frozenset[str] = frozenset( { "api_key", @@ -562,9 +564,54 @@ def _read_renamed_config(config: dict[str, Any], new: str, old: str) -> Any: "max_retry_delay", "retry_jitter", "max_concurrent_requests", + "extra_request_params", } ) + +def _apply_extra_request_params(config, extra_request_params: dict[str, Any]) -> None: + """Merge extra_request_params into a GenerateContentConfig, in place. + + `extra_request_params` is a settings-only escape hatch (bundle/settings + YAML only -- never an interactive ConfigField) for reaching + GenerateContentConfig fields this provider doesn't otherwise expose: + safety_settings, top_p, top_k, seed, stop_sequences, + presence_penalty/frequency_penalty, response_mime_type, labels, and any + other field google-genai's GenerateContentConfig defines. It is merged + LAST, after this provider's own computed values (temperature, + max_output_tokens, thinking_config, tools, ...) -- the caller's extra + config always wins, and wins LOUDLY: overriding a value this provider + itself had already set logs a warning naming the field, the old value, + and the new one, so a confusing production override is never silent. + + An extra_request_params key that isn't a real GenerateContentConfig + field logs a warning and is skipped -- never raises, since a typo in + settings.yaml shouldn't crash the whole provider mount. + """ + if not extra_request_params: + return + valid_fields = type(config).model_fields + for key, value in extra_request_params.items(): + if key not in valid_fields: + logger.warning( + "[PROVIDER] Gemini: extra_request_params key %r is not a " + "recognized GenerateContentConfig field -- ignored. See " + "google.genai.types.GenerateContentConfig for valid fields.", + key, + ) + continue + existing = getattr(config, key, None) + if existing is not None: + logger.warning( + "[PROVIDER] Gemini: extra_request_params overrides '%s' " + "(provider computed %r, extra_request_params sets %r) -- " + "extra_request_params always wins.", + key, + existing, + value, + ) + setattr(config, key, value) + # Keys that appeared in past README revisions describing features that were # never actually implemented in this module (verified by grep: no # `self.config.get(...)` call site reads any of them). These are not typos @@ -697,6 +744,12 @@ def __init__( self.use_streaming = _parse_config_bool( "use_streaming", self.config.get("use_streaming"), True ) + # Settings-only escape hatch -- deliberately NOT a ConfigField (no + # interactive wizard prompt). Arbitrary GenerateContentConfig + # fields (safety_settings, top_p, top_k, seed, stop_sequences, + # etc.) merged in last, after this provider's own computed values. + # See _apply_extra_request_params for the merge contract. + self.extra_request_params = self.config.get("extra_request_params") or {} # Retry configuration — delegates to shared retry_with_backoff() from amplifier-core. self._retry_config = RetryConfig( @@ -1409,6 +1462,12 @@ async def _complete_chat_request( genai.types.AutomaticFunctionCallingConfig(disable=True) ) + # extra_request_params merged LAST -- see _apply_extra_request_params + # for the full contract (owner-beware override, warns loudly). + # Single site: both the streaming and non-streaming call paths below + # reuse this same `config` object. + _apply_extra_request_params(config, self.extra_request_params) + logger.info(f"Gemini API call - model: {model}, messages: {len(all_messages)}") # Emit llm:request event diff --git a/tests/test_extra_request_params.py b/tests/test_extra_request_params.py new file mode 100644 index 0000000..effe5dc --- /dev/null +++ b/tests/test_extra_request_params.py @@ -0,0 +1,159 @@ +"""Tests for extra_request_params: a settings-only escape hatch merged LAST +into the GenerateContentConfig this provider builds for every request. + +Covers: +- Not a ConfigField (no interactive prompt) -- verified by inspecting + get_info()'s config_fields list. +- Merges arbitrary GenerateContentConfig fields not otherwise exposed + (e.g. top_p, safety_settings). +- Wins loudly over this provider's own computed values (e.g. temperature), + with a warning naming the old and new values. +- Unknown/invalid field names warn and are skipped, never raise. +- Single merge site: both the streaming and non-streaming call paths see + the same merged config. +""" + +import asyncio +from types import SimpleNamespace +from typing import cast +from unittest.mock import AsyncMock, MagicMock + +from amplifier_core import ModuleCoordinator +from amplifier_core.message_models import ChatRequest, Message + +from amplifier_module_provider_gemini import GeminiProvider +from amplifier_module_provider_gemini import _apply_extra_request_params + + +class FakeHooks: + def __init__(self): + self.events: list[tuple[str, dict]] = [] + + async def emit(self, name: str, payload: dict) -> None: + self.events.append((name, payload)) + + +class FakeCoordinator: + def __init__(self): + self.hooks = FakeHooks() + + +def _make_gemini_response(): + part = SimpleNamespace(text="Hello", thought=False) + content = SimpleNamespace(parts=[part]) + candidate = SimpleNamespace(content=content) + usage = SimpleNamespace( + prompt_token_count=10, candidates_token_count=5, total_token_count=15 + ) + return SimpleNamespace(candidates=[candidate], usage_metadata=usage) + + +def _make_provider(**config) -> GeminiProvider: + config.setdefault("max_retries", 0) + config.setdefault("use_streaming", False) + provider = GeminiProvider(api_key="test-key", config=config) + provider.coordinator = cast(ModuleCoordinator, FakeCoordinator()) + return provider + + +def _run_complete(provider, **kwargs): + mock_client = MagicMock() + mock_client.aio.models.generate_content = AsyncMock( + return_value=_make_gemini_response() + ) + provider._client = mock_client + request = ChatRequest(messages=[Message(role="user", content="Hello")]) + asyncio.run(provider.complete(request, **kwargs)) + call_kwargs = mock_client.aio.models.generate_content.await_args + return call_kwargs.kwargs.get("config") or call_kwargs[1].get("config") + + +# ============================================================ +# Not a ConfigField +# ============================================================ + + +def test_extra_request_params_is_not_a_config_field(): + provider = _make_provider() + info = provider.get_info() + field_ids = {f.id for f in info.config_fields} + assert "extra_request_params" not in field_ids + + +# ============================================================ +# _apply_extra_request_params -- unit tests +# ============================================================ + + +def test_apply_merges_unexposed_field(): + from google.genai import types + + config = types.GenerateContentConfig(temperature=0.7, max_output_tokens=100) + _apply_extra_request_params(config, {"top_p": 0.95}) + assert config.top_p == 0.95 + + +def test_apply_empty_or_none_is_a_noop(): + from google.genai import types + + config = types.GenerateContentConfig(temperature=0.7) + _apply_extra_request_params(config, {}) + assert config.temperature == 0.7 + _apply_extra_request_params(config, None) # type: ignore[arg-type] + assert config.temperature == 0.7 + + +def test_apply_overrides_existing_value_loudly(caplog): + from google.genai import types + + config = types.GenerateContentConfig(temperature=0.7) + with caplog.at_level("WARNING"): + _apply_extra_request_params(config, {"temperature": 1.0}) + assert config.temperature == 1.0 + assert any( + "overrides 'temperature'" in rec.message + and "0.7" in rec.message + and "1.0" in rec.message + for rec in caplog.records + ), f"got: {[r.message for r in caplog.records]}" + + +def test_apply_unknown_field_warns_and_is_skipped_not_raised(caplog): + from google.genai import types + + config = types.GenerateContentConfig(temperature=0.7) + with caplog.at_level("WARNING"): + _apply_extra_request_params(config, {"not_a_real_field": 123}) + assert not hasattr(config, "not_a_real_field") or True # never raised + assert any( + "'not_a_real_field' is not a recognized" in rec.message + for rec in caplog.records + ) + # And the config is otherwise untouched. + assert config.temperature == 0.7 + + +# ============================================================ +# End-to-end through complete() +# ============================================================ + + +def test_extra_request_params_reaches_generate_content_config(): + provider = _make_provider(extra_request_params={"top_p": 0.5}) + config = _run_complete(provider) + assert config.top_p == 0.5 + + +def test_extra_request_params_overrides_provider_temperature(caplog): + provider = _make_provider(temperature=0.7, extra_request_params={"temperature": 0.2}) + with caplog.at_level("WARNING"): + config = _run_complete(provider) + assert config.temperature == 0.2 + assert any("overrides 'temperature'" in rec.message for rec in caplog.records) + + +def test_no_extra_request_params_is_unaffected(): + provider = _make_provider(temperature=0.7) + config = _run_complete(provider) + assert config.temperature == 0.7 + assert config.top_p is None From c994741ac8cf8e4c2bb6cf787ab070cc769bcfbd Mon Sep 17 00:00:00 2001 From: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com> Date: Sat, 29 Aug 2026 13:37:40 -0700 Subject: [PATCH 7/7] docs: README overhaul -- remove ghosts, fix drift, document thinking_level/extra_request_params MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Removes the three ghost config keys documented in prior README revisions but never actually implemented (verified by grep: no self.config.get(...) call site reads any of them): - `debug` / `raw_debug` -- no llm:request:debug/llm:response:debug or llm:request:raw/llm:response:raw events exist in this provider. - `debug_truncate_length` -- no debug-log truncation path exists. The entire "Debug Configuration" section describing these is removed. (Config still setting them now gets a specific "this key is inert" warning at mount time, from the previous hygiene commit.) Fixes documentation drift: `timeout` default was documented as 300.0 but the code default has been 600.0. Renames `max_tokens` to `max_output_tokens` throughout (keeping `max_tokens` documented as the deprecated alias). Updates `default_model` references to gemini-3.7-flash and the google-genai floor to >=1.56.0, matching the code changes in earlier commits of this PR. Adds the house-style key-reference table: every config key now states either the real Google API parameter name it maps to (e.g. `temperature` -> API: `temperature`) or "Amplifier-only" for glue with no Google equivalent, plus one plain-language sentence -- replacing the old table's bare type/default/description columns with an explicit provenance column. Documents: - The full thinking_level per-model support table and the reasoning_effort -> thinking_level mapping (including the legacy thinking_budget fallback for gemini-2.5-* models, and the cannot-disable-thinking limitation on gemini-3.x), with the exact live error messages that back each row. - extra_request_params as a dedicated section: contract (merged last, loud override warnings, unknown fields skipped not raised), the settings-only/no-ConfigField guarantee, an example, and the safety-filters-default-off vendor fact. - Thought-signature round-trip and why this provider's stateless full-resend design makes it load-bearing. Rewrites the Supported Models section for the current lineup (3.7-flash flagship, 3.5-flash/-lite legacy, 2.5-* two generations back but still served and thinking_level-rejecting, 2.0 shut down) with a note to prefer live list_models() over this table since availability changes frequently. 🤖 Generated with [Amplifier](https://github.com/microsoft/amplifier) Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com> --- README.md | 164 +++++++++++++++++++++++++++++++++--------------------- 1 file changed, 101 insertions(+), 63 deletions(-) diff --git a/README.md b/README.md index beef484..8a845f5 100644 --- a/README.md +++ b/README.md @@ -48,7 +48,7 @@ For more control over configuration or to compose with other capabilities, use a name: gemini-dev version: 1.0.0 description: Gemini provider with full 1M context - + includes: - bundle: foundation @@ -61,8 +61,8 @@ For more control over configuration or to compose with other capabilities, use a - module: provider-gemini source: git+https://github.com/microsoft/amplifier-module-provider-gemini@main config: - default_model: gemini-2.5-flash - max_tokens: 65536 # Full 65K output capacity + default_model: gemini-3.7-flash + max_output_tokens: 65536 # Full 65K output capacity temperature: 0.7 priority: 50 # Lower number = higher priority (beats default 100) --- @@ -73,10 +73,9 @@ For more control over configuration or to compose with other capabilities, use a ## Available Models - - **Gemini Flash** - `gemini-2.5-flash` - Balanced performance with 1M token context - - **Gemini Flash-Lite** - `gemini-2.5-flash-lite` - Fastest and most cost-efficient model - - **Gemini Pro** - `gemini-2.5-pro` - Most powerful model with extended thinking capabilities - - **Gemini 3.0 (Preview)** - `gemini-3-pro-preview` - Best model for advanced reasoning and text generation + - **Gemini 3.7 Flash** - `gemini-3.7-flash` - Current flagship Flash model, best price-performance (default) + - **Gemini 3.5 Flash / Flash-Lite** - `gemini-3.5-flash` / `gemini-3.5-flash-lite` - Legacy Flash generation + - **Gemini 2.5 Flash / Flash-Lite / Pro** - `gemini-2.5-flash` / `gemini-2.5-flash-lite` / `gemini-2.5-pro` - Two generations back; still served ``` 3. **Use it**: @@ -113,20 +112,25 @@ Provides access to Google's Gemini models as an LLM provider for Amplifier with **Current support**: Text generation, tool calling, and thinking. Multimodal capabilities (images, video, audio) are not yet implemented. -### Gemini 3.0 (Latest - Preview) +Model availability and naming change frequently -- this list reflects what was verified live against `list_models()` on 2026-08-29. Always prefer `amplifier provider models gemini` (or your account's actual `list_models()` result) over this table for what your key can currently use. + +### Gemini 3.x (Current generation) -- `gemini-3-pro-preview` - Best model for advanced reasoning and text generation (1M context, 65K max output) +- `gemini-3.7-flash` - **Current flagship Flash model** ("the latest and most capable" per ai.google.dev). **Default model for this provider.** Uses `thinking_level` (low/medium/high -- no `minimal`). +- `gemini-3.5-flash` / `gemini-3.5-flash-lite` - Documented by Google as **legacy** relative to 3.7. Uses `thinking_level` (full minimal/low/medium/high range). +- Other `gemini-3.*` preview/dated ids (e.g. `gemini-3.1-flash-lite-preview`, `gemini-3-pro-image-preview`) come and go -- this provider assumes any `gemini-3.*` id supports `thinking_level` with the full range unless proven otherwise by a live 400. -### Gemini 2.5 (Stable - Recommended) +### Gemini 2.5 (Two generations back; still served) -- `gemini-2.5-flash` - Best price-performance for large-scale processing (1M context, 65K max output, default) +- `gemini-2.5-flash` - Best price-performance for large-scale processing (1M context, 65K max output) - `gemini-2.5-pro` - State-of-the-art thinking model for complex reasoning (1M context, 65K max output) - `gemini-2.5-flash-lite` - Fastest model optimized for cost-efficiency (1M context, 65K max output) -### Gemini 2.0 (Stable) +**Verified live (2026-08-29): these three REJECT `thinking_level` outright** ("Thinking level is not supported for this model") -- this provider automatically falls back to the legacy `thinking_budget` control for them. See [Thinking/Reasoning](#thinkingreasoning) below. + +### Gemini 2.0 -- `gemini-2.0-flash` - Well-rounded capabilities with focus on price-performance (1M context, 8K max output) -- `gemini-2.0-flash-lite` - Optimized for cost efficiency and low latency (1M context, 8K max output) +Shut down by Google -- no longer served. Do not configure `gemini-2.0-flash` / `gemini-2.0-flash-lite` as your model. **Note**: Image/video/audio models not listed as the provider doesn't support multimodal capabilities yet. @@ -137,51 +141,60 @@ Provides access to Google's Gemini models as an LLM provider for Amplifier with module = "provider-gemini" name = "gemini" config = { - default_model = "gemini-2.5-flash", - max_tokens = 8192, + default_model = "gemini-3.7-flash", + max_output_tokens = 8192, temperature = 0.7, - debug = false, - raw_debug = false } ``` -### Debug Configuration +### Configuration Options -**Standard Debug** (`debug: true`): -- Emits `llm:request:debug` and `llm:response:debug` events -- Contains request/response summaries with truncated values (default 180 chars) -- Moderate log volume, suitable for development +Every key below corresponds either to a real parameter in Google's `google.genai.types.GenerateContentConfig` (linked as "API: `field_name`") or is Amplifier-only glue with no Google equivalent (marked "Amplifier-only"). -**Raw Debug** (`debug: true, raw_debug: true`): -- Emits `llm:request:raw` and `llm:response:raw` events -- Contains complete, unmodified request params and response objects -- Extreme log volume, use only for deep provider integration debugging -- Captures the exact data sent to/from Gemini API before any processing +| Parameter | Type | Default | API param / origin | Description | +|-----------|------|---------|---------------------|-------------| +| `api_key` | string | env: `GOOGLE_API_KEY` or `GEMINI_API_KEY` | Amplifier-only | Google AI API key. Env vars match the official SDK's own resolution (`GOOGLE_API_KEY` wins if both are set). | +| `default_model` | string | `gemini-3.7-flash` | Amplifier-only | Default model to use when a request doesn't override it. | +| `max_output_tokens` | int | 8192 | API: `max_output_tokens` | Maximum output tokens. **Renamed from `max_tokens`** to match Google's own parameter name -- `max_tokens` still works as a deprecated alias (one-shot warning; `max_output_tokens` always wins if both are set). | +| `max_tokens` | int | -- | *(deprecated alias)* | Old name for `max_output_tokens`. Prefer the new name in new configs. | +| `temperature` | float | 0.7 | API: `temperature` | Sampling temperature (0.0-2.0 per Google's docs; this provider does not clamp the range itself). | +| `timeout` | float | 600.0 | Amplifier-only | API call timeout in seconds, enforced client-side via `asyncio.wait_for`. | +| `priority` | int | 100 | Amplifier-only | Provider selection priority (lower = preferred). Read by the orchestrator's provider-selection logic, not by this module's own request-building code. | +| `raw` | bool | false | Amplifier-only | Enable raw API request/response capture on the `llm:request` / `llm:response` events (non-streaming path only). | +| `use_streaming` | bool | true | Amplifier-only | Use `generate_content_stream` instead of a single blocking `generate_content` call. Per-request override: `request.metadata["stream"] = False`. | +| `max_retries` | int | 5 | Amplifier-only | Max retry attempts on transient failures (5xx, timeouts, rate limits, Cloudflare/CDN challenges). | +| `min_retry_delay` | float | 1.0 | Amplifier-only | Initial retry backoff delay, in seconds. | +| `max_retry_delay` | float | 60.0 | Amplifier-only | Maximum retry backoff delay, in seconds. | +| `retry_jitter` | bool | true | Amplifier-only | Add random jitter to retry backoff delays. | +| `max_concurrent_requests` | int | 5 | Amplifier-only | Process-wide concurrency limit shared across all provider instances (parent + delegated sessions). `0` disables the limit. | +| `extra_request_params` | dict | `{}` | *(passthrough)* | Settings-only escape hatch -- see [extra_request_params](#extra_request_params-advanced) below. Never an interactive config field; owner-beware. | + +Boolean and numeric values above tolerate string input (`"true"`/`"false"`, `"600"`, etc. -- as written by the app-cli wizard or hand-edited YAML) and warn-and-default rather than crash on anything unparseable. Unrecognized config keys log a warning at mount time (with a "did you mean" suggestion for likely typos); they never silently do nothing without a signal. + +> **Removed in this revision**: `debug`, `raw_debug`, and `debug_truncate_length` were documented here in earlier README revisions but were **never actually implemented** by this provider (no code path reads them). If your config still sets them, they are now flagged with a specific "this key is inert" warning at mount time instead of being silently ignored. Use `raw: true` for this provider's actual raw-I/O capture on `llm:request`/`llm:response` events. + +### extra_request_params (advanced) + +`extra_request_params` merges arbitrary fields directly into the `GenerateContentConfig` this provider builds for every request -- reaching Google API parameters this provider doesn't otherwise expose as a first-class config key: `safety_settings`, `top_p`, `top_k`, `seed`, `stop_sequences`, `presence_penalty`, `frequency_penalty`, `response_mime_type`, `labels`, and anything else `google.genai.types.GenerateContentConfig` defines. -**Example**: ```yaml providers: - module: provider-gemini config: - debug: true # Enable debug events - raw_debug: true # Enable raw API I/O capture - debug_truncate_length: 180 # Control truncation length - default_model: gemini-2.5-flash + default_model: gemini-3.7-flash + extra_request_params: + top_p: 0.95 + safety_settings: + - category: HARM_CATEGORY_DANGEROUS_CONTENT + threshold: BLOCK_ONLY_HIGH ``` -### Configuration Options - -| Parameter | Type | Default | Description | -|-----------|------|---------|-------------| -| `api_key` | string | env: `GOOGLE_API_KEY` or `GEMINI_API_KEY` | Google AI API key | -| `default_model` | string | `gemini-2.5-flash` | Default model to use | -| `max_tokens` | int | 8192 | Maximum output tokens | -| `temperature` | float | 0.7 | Sampling temperature (0.0-1.0) | -| `timeout` | float | 300.0 | API timeout in seconds | -| `priority` | int | 100 | Provider selection priority | -| `debug` | bool | false | Enable debug-level logging with truncated values | -| `raw_debug` | bool | false | Enable ultra-verbose raw API I/O logging (requires debug=true) | -| `debug_truncate_length` | int | 180 | Maximum string length in debug logs | +**Contract:** +- Merged **LAST**, after every value this provider computes itself (temperature, `max_output_tokens`, `thinking_config`, tools). Your `extra_request_params` value always wins. +- Overriding a value this provider had already set logs a **warning** naming the field, this provider's computed value, and your override -- a silent production override never happens. +- An unrecognized field name (not a real `GenerateContentConfig` field) logs a warning and is skipped -- it never crashes the provider mount. +- **Settings-only, deliberately not a `ConfigField`**: it will never appear in the interactive `amplifier init` / `amplifier provider use` wizard. Set it directly in `settings.yaml` or a bundle's config block. If your tooling round-trips provider config (e.g. re-serializing settings), this key passes through unchanged like any other dict value -- there is no special handling on this provider's side beyond the merge described above. +- **Safety filters default OFF** on Gemini 2.5/3.x models (verified against ai.google.dev) -- this provider never injects a `safety_settings` default of its own. If you want filtering, set it explicitly via `extra_request_params.safety_settings`. ## Environment Variables @@ -202,7 +215,7 @@ Get your API key from [Google AI Studio](https://aistudio.google.com/apikey). # In amplifier configuration [provider] name = "gemini" -default_model = "gemini-2.5-flash" +default_model = "gemini-3.7-flash" ``` ## Example Bundle Configurations @@ -215,8 +228,8 @@ providers: - module: provider-gemini source: git+https://github.com/microsoft/amplifier-module-provider-gemini@main config: - default_model: gemini-2.5-flash - max_tokens: 65536 # Use full 65K output capacity + default_model: gemini-3.7-flash + max_output_tokens: 65536 # Use full 65K output capacity temperature: 0.7 priority: 50 # IMPORTANT: Lower number = higher priority (beats default 100) ``` @@ -227,8 +240,8 @@ providers: - module: provider-gemini source: git+https://github.com/microsoft/amplifier-module-provider-gemini@main config: - default_model: gemini-2.5-flash - max_tokens: 65536 # Full 65K output capacity + default_model: gemini-3.7-flash + max_output_tokens: 65536 # Full 65K output capacity priority: 50 # Lower number = higher priority ``` @@ -249,8 +262,8 @@ providers: - module: provider-gemini source: git+https://github.com/microsoft/amplifier-module-provider-gemini@main config: - default_model: gemini-2.5-pro - max_tokens: 65536 # Full 65K output capacity + default_model: gemini-3.7-flash + max_output_tokens: 65536 # Full 65K output capacity temperature: 1.0 priority: 50 # Lower number = higher priority ``` @@ -261,8 +274,8 @@ providers: - module: provider-gemini source: git+https://github.com/microsoft/amplifier-module-provider-gemini@main config: - default_model: gemini-2.5-flash-lite - max_tokens: 65536 # Full 65K output capacity + default_model: gemini-3.5-flash-lite + max_output_tokens: 65536 # Full 65K output capacity temperature: 0.5 priority: 50 # Lower number = higher priority ``` @@ -273,14 +286,38 @@ providers: - **Text Generation** - Single and multi-turn conversations - **Tool/Function Calling** - OpenAPI schema format -- **Extended Thinking** - Reasoning with adjustable token budget +- **Extended Thinking** - Reasoning via `thinking_level` (current models) or `thinking_budget` (legacy models) - **Streaming Support** - Incremental response generation - **1M Token Context** - Process extremely large inputs (Flash models) - **Message Validation** - Defense-in-depth error checking ### Thinking/Reasoning -**Gemini 2.5 models (Pro and Flash) think by default** using dynamic token budgets. The provider automatically captures thinking content from the Gemini API. +Google's thinking control surface changed generations: `thinking_budget` (an approximate output-token budget) is the **legacy** control; `thinking_level` (an enum: `minimal`/`low`/`medium`/`high`) is the **current** control -- and, as of this revision, the *only* control some models accept at all. Sending both on one request is rejected by the API with a 400. + +This provider maps Amplifier's portable `reasoning_effort` request field to `thinking_level` automatically, per-model, clamped against a small maintained support table: + +| Model | Supported `thinking_level` values | Notes | +|-------|-----------------------------------|-------| +| `gemini-3.7-flash` | `low`, `medium`, `high` | **Rejects `minimal`** (verified live: "Thinking level MINIMAL is not supported for this model"). Google's own default (when omitted) is `medium`. | +| `gemini-3.5-flash`, `gemini-3.5-flash-lite` | `minimal`, `low`, `medium`, `high` | Full range. | +| Other `gemini-3.*` ids | `minimal`, `low`, `medium`, `high` (assumed) | Not individually verified -- assumed full range until a live 400 proves narrower. | +| `gemini-2.5-flash`, `gemini-2.5-pro`, `gemini-2.5-flash-lite` | **none** | Rejects `thinking_level` entirely (verified live: "Thinking level is not supported for this model"). Falls back to the legacy `thinking_budget` mapping below. | +| `gemini-2.0-*` | **none** (shut down) | Not servable at all. | + +`reasoning_effort` -> `thinking_level` mapping (for models that support it): + +| `reasoning_effort` | Target level | If not supported by the model | +|---------------------|--------------|--------------------------------| +| `none` | *(no level sent)* | Uses `minimal` if the model supports it, otherwise the model's own default (Gemini 3.x cannot disable thinking at all -- verified live) | +| `minimal` | `minimal` | Clamped up to the nearest supported level (e.g. `low` on `gemini-3.7-flash`), logged at INFO | +| `low` | `low` | Clamped as above | +| `medium` | `medium` | Clamped as above | +| `high`, `xhigh`, `max` | `high` | Gemini has no level above `high` | + +For models with **no** `thinking_level` support (the `gemini-2.5-*` family), `reasoning_effort` instead maps to the legacy numeric `thinking_budget`: `none` -> `0` (disabled), `minimal`/`low` -> `4096`, `medium`/`high`/`xhigh`/`max` -> `-1` (dynamic, model decides). + +An explicit `thinking_budget` passed via `request.metadata["thinking_budget"]` or a provider `**kwargs` override always wins outright and is sent **alone** -- never combined with `thinking_level` on the same request. **To display thinking output**, configure your orchestrator (not the provider): @@ -293,12 +330,9 @@ session: extended_thinking: true # Show thinking content to user ``` -**Model thinking behavior**: -- **gemini-2.5-pro**: Thinks by default (best for complex reasoning) -- **gemini-2.5-flash**: Thinks by default (good for most tasks) -- **gemini-2.5-flash-lite**: Does NOT think by default +**Note**: The provider captures thinking from the API automatically (`include_thoughts` defaults to `true`). The orchestrator's `extended_thinking: true` config controls whether it's *displayed*. Without this config, thinking still happens but isn't shown to the user. -**Note**: The provider captures thinking from the API automatically. The orchestrator's `extended_thinking: true` config controls whether it's displayed. Without this config, thinking still happens but isn't shown to the user. +**Thought signatures**: Gemini 2.5+ models attach an opaque `thought_signature` to parts that follow a thinking burst. Because this provider is **stateless full-resend** (the entire conversation is rebuilt from stored history and resent on every turn, with no server-side session), these signatures must be captured and replayed unmodified or the API returns `FinishReason MISSING_THOUGHT_SIGNATURE`. This provider captures and echoes signatures on text, thinking, and tool-call parts alike, encoded as base64 so the value survives any JSON serialization the conversation history passes through (e.g. session persistence, event logging). ### Tool Calling @@ -371,9 +405,13 @@ The Gemini API does not provide tool call IDs (unlike Anthropic and OpenAI). The The provider implements text generation, tool calling, and thinking support. Multimodal capabilities (images, video, audio) are not yet supported. +### Gemini 3.x Thinking Cannot Be Disabled + +Verified live: `thinking_budget=0` on `gemini-3.7-flash` still produced thinking tokens, and there is no "off" `thinking_level`. Thinking is mandatory for Gemini 3.x models regardless of what this provider sends. `reasoning_effort="none"` on these models falls back to the model's own default thinking amount rather than actually disabling it -- this is a vendor limitation, not something this provider can work around. + ## Dependencies -- `google-genai>=1.40.0` - Official Google AI Python SDK +- `google-genai>=1.56.0` - Official Google AI Python SDK. 1.56.0 is the floor because it's the first release whose `ThinkingConfig` exposes the full `thinking_level` enum (`minimal`/`low`/`medium`/`high`) this provider needs -- verified by probing the SDK's own installed types directly: 1.46.0 has no `thinking_level` field at all; 1.51.0 adds it with only `LOW`/`HIGH`; 1.56.0 completes the four-level enum. ## Development