Skip to content

fix(openai): record provider-resolved model provenance - #1682

Draft
adity982 wants to merge 1 commit into
simonw:mainfrom
adity982:fix/openai-resolved-model
Draft

adity982 wants to merge 1 commit into
simonw:mainfrom
adity982:fix/openai-resolved-model

Conversation

@adity982

Copy link
Copy Markdown

Summary

Fixes #1681.

Record the provider-reported model through the existing response.set_resolved_model() API when it is a nonempty string different from the configured model ID. This uses the response already received; it does not make another API request or change model selection.

The shared helper covers Chat, AsyncChat, Responses, AsyncResponses and legacy Completion, including streaming responses and the Responses chat_completions override. Custom model_name configurations retain the public configured ID while recording the served provider ID. Missing, empty and unchanged model values preserve the existing null provenance field.

Tests use the real OpenAI SDK with mock HTTP transports, verify exactly one request and unchanged request model, and check persisted turns.resolved_model plus async-to-sync conversion.

Validation

  • 52 new cases against unchanged upstream production code: 22 failed, 30 passed, reproducing missing snapshot/custom-provider provenance across all execution paths.
  • Focused provenance, Responses and logging suites: 202 passed, 2 xfailed.
  • Full suite on Windows / Python 3.12: 1,153 passed, 1 failed, 1 skipped, 6 xfailed, 15 xpassed. The sole failure, test_extract_fenced_code_crlf, also fails in a clean worktree at upstream 1df47ddcac20d58726a993949da8ef84f4081085, using the same environment and verified baseline module path. It concerns pre-existing Windows CRLF output, not these model changes. No snapshot, newline behavior or unrelated test was changed.
  • Black, Ruff and mypy: pass. Distribution sdist/wheel build and git diff --check: pass.
  • Tests ran with external network blocked, localhost loopback allowed for Windows asyncio, and recording disabled. No live provider calls, paid evaluations, secret lookup, or cassette updates.

Draft for review. The other supported OS/Python CI combinations and plugin-load shell check have not been exercised on this host.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Populate resolved_model in the default OpenAI plugin from the response model field

1 participant