build: upgrade openai to 2.48.0 - #70
Merged
Merged
Conversation
Owner
Author
|
@codex review for deterministic/RagaliQ ownership, dependency reproducibility, Python 3.14 compatibility, public-clone portability, paid-call safety, golden-fixture integrity, marker correctness, public API compatibility, and unintended behaviour changes |
|
Codex Review: Didn't find any major issues. Another round soon, please! Reviewed commit: 鈩癸笍 About Codex in GitHubCodex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 馃憤. When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback". |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Upgrades
openaifrom 2.45.0 to 2.48.0. Lock-only change;pyproject.tomlis untouched becauseopenai>=2.45.0,<3already admits 2.48.0.This is the largest runtime delta in the audit and the one the free suite cannot detect anything about. That is stated up front rather than buried.
Version delta
openai>=2.45.0,<3(pyproject.toml:11), unchangedIntervening stable releases: 2.46.0, 2.47.0, 2.48.0. No patch releases exist between them. All three release notes and
CHANGELOG.mdatv2.48.0were read; none carries a "BREAKING CHANGES" heading or a migration link.The
>=2.45.0floor is deliberately left alone. A floor is not a pin, and raising it would be an unrequested constraint change.Lock regenerated with the first form of
.agents/skills/upgrade-dependencies/SKILL.mdstep 5, targeting only this package:No transitive churn.
openai's core dependency list is byte-identical across the range (anyio,distro,httpx,jiter,pydantic,sniffio,tqdm,typing-extensions), and itsrequires-pythonis still>= 3.9:The exact SDK surface this repository binds
Anchored so the filtering below can be checked rather than trusted:
pipeline.py:28from openai import OpenAIpipeline.py:716-718OpenAI(max_retries=0, timeout=120.0)pipeline.py:130-133client.embeddings.create(model=, input=, encoding_format="float")pipeline.py:135-140response.data,item.index,item.embeddingpipeline.py:184-187client.chat.completions.create(model=, temperature=, max_completion_tokens=300, messages=)pipeline.py:158-168response.choices,choice.finish_reason,choice.message.content,getattr(choice.message, "refusal", None)That is the whole surface. Nine call sites, all in
pipeline.py.Breaking changes, and which touch this repository
None of them. And the basis for that is file identity, not the absence of a changelog mention. The five SDK files that define this repository's entire read surface are blob-SHA identical between the two tags:
None of the five appears in the
v2.45.0...v2.48.0changed-file list at all.The announced changes, classified:
prompt_cache_keyandsafety_identifierwiden fromstrtoOptional[str]on all fourchat.completions.createoverloadsspend_limitresourcesresources/admin/organization/; the admin surface is never imported.openai.__all__gainsDefaultHttpx2Client,DefaultAsyncHttpx2Client__init__.pychanges are additive only;OpenAIis still re-exported from._client._httpx2.pynever importshttpx2at module scope - the real import sits inside_require_httpx2(), and detection usessys.modules.get. Withhttpx2absent,normalize_httpx_timeoutis the identity on a plain float andtimeout_exceptions()/status_exceptions()/stream_consumed_exceptions()return single-element tuples of the samehttpxclasses as before.aiohttpextra now requiresaiohttp>=3.14.1underpython_version >= '3.10';_DefaultAioHttpClientgated behindsys.version_info >= (3, 10)openaiextras are installed, and the sub-3.10 branch is unreachable on 3.14._response.py/_legacy_response.pyisinstancebroadening on the response-construction pathValueErrorfires only when a caller passes anhttpx.Responsesubclass ascast_to, which never happens here.api_keyssubresource, moving type exports;audit_logsparams/response widened; elevenusage_*_responsetypes widenedresources/admin/organization/**ortypes/admin/organization/**. This repository touches two endpoints and never importsclient.admin.types/beta/beta_response_input_*modules, then re-added in the same releaseopenai.types.webhooksandclient.webhooks.unwrapare never imported.stlcconfigurable CI runner and private-production-repo support.github/workflowsandscripts/; nothing ships in the wheel.Two precision notes, because "byte-identical" was worth checking rather than repeating:
src/openai/_client.pyis not byte-identical (66d03b23dd4evsaeea9907a873). What is byte-identical is the two parameter declarations this repository binds:timeout: float | Timeout | None | NotGiven = not_givenandmax_retries: int = DEFAULT_MAX_RETRIES. The one differing statement in the constructor region is theWorkloadIdentityAuthconstruction moving aftersuper().__init__(), and it sits behindelif workload_identity is not None- a branch this repository never enters, since it passes onlymax_retriesandtimeout.base_urlnow passes throughnormalize_httpx_url(this repository never passesbase_url); thehttp_clientTypeErrortext now reads "httpx.Client or httpx2.Client"; thecast_toValueErrortext changed; averbositydocstring gained "The default ismedium".The environment condition that keeps every 2.47.0 path inert, verified in the installed venv rather than assumed:
Prediction recorded before running anything
A clean, behaviourally neutral upgrade, with the confidence resting on file identity rather than on the changelog being quiet. Specifically:
test_real_client_uses_exact_bounded_policymonkeypatchespipeline_module.OpenAIwith a kwargs-capturing fake, and the adapter tests injectSimpleNamespaceendpoints. The realOpenAIclass is never instantiated under the default selection..venvis in Ruff's default exclusion set and both hooks additionally pass--force-exclude, so the newsrc/openai/_httpx2.pyis never linted. The only SDK-typed expressions in the mypy scope are the-> OpenAIreturn annotation atpipeline.py:714and the constructor call at716-718, whose signature is unchanged.What actually happened
Exactly the prediction.
uv lock --checkpylock.tomluv pip checkpytest eval/ -qpre-commit validate-configpre-commit run --all-filesruff format --check .ruff check .agent-policy-symbolsbenchmark_retrieval.py(importspipeline, henceopenai)Free validation command and result:
Substitute evidence, since no test can supply it
Because the suite cannot check the SDK surface, it was checked by static introspection of the installed 2.48.0. No client was constructed, no network operation occurred, and no provider was contacted:
Every symbol at every anchored call site is present with a compatible type.
max_retries: intaccepts 0;timeout: float | ...accepts 120.0;encoding_format: Literal['float','base64'] | Omitaccepts"float". This is not equivalent to executing the calls, and it is not claimed to be - it is a static check standing in for a runtime check that policy forbids.The mutation question
Which behaviours of this dependency does the suite exercise? Precisely one: that
openaiimports and that the nameOpenAIexists at module level. That ispipeline.py:28, executed at collection time. Nothing else.Every other SDK behaviour is stood in for by fakes. Anchored:
_FakeEmbeddingsEndpoint.create(**kwargs)eval/test_verdigrise.py:2745-2759client.embeddings.createSimpleNamespace._FakeCompletionsEndpoint.create(**kwargs)eval/test_verdigrise.py:2868-2894client.chat.completions.createbuild_client(**kwargs)replacingpipeline_module.OpenAIeval/test_verdigrise.py:2690-2707OpenAIconstructor{"max_retries": 0, "timeout": 120.0}. Not that the SDK accepts them.Compounding this,
OpenAIEmbeddingProvider.__init__,OpenAIAnswerGenerator.__init__andparse_generation_responseall annotate their SDK argument asAny. Somypy --strictalso cannot check this surface. Neither the dynamic gate nor the static gate sees it.For each breaking change that touches a real call site, which test would have failed?
The honest answer is that the table is empty on the left, and that is itself the result:
encoding_formatrenamed or itsLiteralnarrowed_FakeEmbeddingsEndpoint.create(**kwargs)accepts anything.response.data[].embeddingrenamedSimpleNamespace(index=..., embedding=...)regardless of the real model.choice.message.refusalremovedgetattr(..., "refusal", None)is already tolerant, and the fake supplies the attribute.max_retriesortimeoutrenamed or retypedtest_real_client_uses_exact_bounded_policyreplaces the constructor entirely; it would still pass while the real SDK rejected the call.chat.completions.createdroppingmax_completion_tokens_FakeCompletionsEndpoint.create(**kwargs)accepts anything.from openai import OpenAIfailingThe finding, stated plainly and as the headline of this pull request. The
openaiSDK is a runtime dependency whose real behaviour is verified only by theopenai-marked paid tier, which the audit brief forbids running and which CI is designed never to select. Between the provider fakes and theAnyannotations, a green free suite on this branch carries almost no information about whether 2.48.0 is compatible - it establishes that the package imports, and nothing more. The confidence in this upgrade comes from blob-level file identity across the three releases and from static introspection of the installed wheel, both documented above. It does not come from the suite.This is a structural property of the design, not a defect introduced here: AGENTS.md and README:258 both state that the free tier uses provider fakes, and that separation is intentional. But it means every
openaibump lands on evidence gathered outside the test suite, and that cost should be explicit each time rather than assumed away by a green checkmark. Per the audit brief, no test was added to close this gap in this pass.Lock byte-comparison
pylock.tomlinvariants re-checked after the export:Clean-clone transcript
Isolated temp directory, both provider key variables unset and confirmed absent by presence check only, no sibling
../RagaliQreachable, README install commands verbatim with bareuvas documented.The clone-to-green figure is a warm-cache number: the uv cache was already populated (17G), so 5s measures command execution, not first contact. A cold-cache clone would additionally download 54 wheels including the new
openai2.48.0 artifact.Hidden costs
openaitier, which needs separate per-invocation approval and real spend.Anyannotation on the SDK client in both adapters is what makes the static gate blind here as well as the dynamic one. Narrowing it to aProtocolwould letmypy --strictcatch signature drift at zero marginal cost. That is a design change, not an upgrade, and is deliberately not attempted in this pull request.httpx2extra. This bump deliberately does not install it. Leavinghttpx2absent is exactly what keeps every new 2.47.0 code path inert, so adding the extra later is a behaviour change requiring its own evidence, not a free performance option.