Skip to content

fix: get_info() reports runtime-effective model, not packaged default - #81

Merged
Salil Das (sadlilas) merged 1 commit into
mainfrom
fix/get-info-reports-effective-model
Aug 31, 2026
Merged

fix: get_info() reports runtime-effective model, not packaged default#81
Salil Das (sadlilas) merged 1 commit into
mainfrom
fix/get-info-reports-effective-model

Conversation

@sadlilas

Copy link
Copy Markdown
Collaborator

Summary

get_info() reports the packaged static model name from config/_models.py instead of the runtime-configured effective model, violating the module's own documented model selection priority and creating an internal inconsistency within the function itself.

The bug

contracts/behaviors.md (Model Selection Policy, MUST:1) states:

MUST respect model selection priority: request.model > config["default_model"] > YAML

get_info() already applies this priority correctly for defaults["context_window"] and defaults["max_output_tokens"] — both are resolved against self._effective_default_model (which itself correctly implements the priority: self.config.get("default_model") or self._provider_config.defaults["model"]). But defaults["model"] was left untouched, so it always reported the packaged YAML default (claude-opus-4.5) regardless of what model was actually configured at runtime — even while the window fields, computed a few lines later in the same function, were correctly reporting values for the real effective model.

Reproduction (with config={"default_model": "mai-code-1.1-flash"}):

provider._effective_default_model  == "mai-code-1.1-flash"   # correct
provider.get_info().defaults["model"] == "claude-opus-4.5"   # WRONG

This is unconditional — it reproduces identically whether the model-info cache is warm or cold, since the model field was never wired to the effective-model resolution at all.

User-visible impact: Consumers that read get_info().defaults["model"] to determine the active model (e.g. to emit a provider:resolve-style event driving a live model display) show the wrong model whenever a caller overrides default_model at runtime (for example via a routing/config layer that assigns different models per sub-agent).

The fix

One line, plus an explanatory comment, in get_info():

defaults["model"] = self._effective_default_model

Placed unconditionally — before the model-info cache lookup, not inside the if info is not None: branch that corrects the window fields — so the reported model is correct even when the model cache is cold. The cold-cache fallback behavior for context_window / max_output_tokens is intentionally unchanged.

This brings the provider in line with sibling Amplifier providers (Anthropic, Chat Completions), which already report the runtime-resolved model from get_info() rather than a static default.

Tests

Added to the existing TestRuntimeConfigOverride class in tests/test_behaviors.py (the established home for Model Selection Policy tests):

  1. test_get_info_reports_runtime_default_modelconfig={"default_model": ...} is reflected in get_info().defaults["model"].
  2. test_get_info_reports_yaml_default_without_runtime_config — no default_model configured still reports the YAML default (no regression).
  3. test_get_info_reports_runtime_model_on_cold_cache — with a cold model-info cache (lookup returns None), the model field is still correct, proving the fix applies outside the if info is not None: branch, while window-field cold-cache fallback stays unchanged.
  4. test_get_info_does_not_mutate_shared_cached_defaults — a second provider instance with a different config is unaffected by a prior instance's get_info() call, since cfg.defaults is a process-wide @lru_cached singleton.

No existing test asserted the previous (incorrect) behavior.

Quality gates

  • pytest tests/ -m "not live" — 1558 passed, 0 failed
  • ruff check amplifier_module_provider_github_copilot/ tests/ — clean
  • pyright . — 0 errors, 0 warnings

Scope

Out of scope (unchanged, deliberately): the cold-cache fallback behavior for context_window / max_output_tokens, get_model_info(), config/_models.py, FALLBACKS, resolve_effective_window, and the model cache implementation.

Generated with Amplifier

Co-Authored-By: Amplifier 240397093+microsoft-amplifier@users.noreply.github.com

get_info() reported the packaged static model name from
config/_models.py regardless of the runtime-configured model. The
context_window and max_output_tokens fields in the same defaults dict
were already correctly resolved against the effective model, but the
model key itself was never corrected -- an internal inconsistency
within the function.

contracts/behaviors.md's Model Selection Policy requires respecting
request.model > config["default_model"] > YAML priority everywhere;
get_info() ignored that priority for defaults["model"] while honoring
it for the window fields.

Fix: set defaults["model"] = self._effective_default_model
unconditionally, so the reported model is correct even on a cold
model-info cache (matching how the window fields already handle the
cold-cache case, just with the static fallback instead).

Adds 4 tests to the existing TestRuntimeConfigOverride class in
tests/test_behaviors.py covering: runtime override, YAML fallback
(no regression), cold-cache correctness, and non-mutation of the
shared lru_cached provider config.

Generated with [Amplifier](https://github.com/microsoft/amplifier)

Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>
@sadlilas
Salil Das (sadlilas) merged commit bc8d6a9 into main Aug 31, 2026
3 checks passed
@sadlilas
Salil Das (sadlilas) deleted the fix/get-info-reports-effective-model branch August 31, 2026 20:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant