ci: block a publish when the tag disagrees with pyproject.toml - #5
Merged
Conversation
Publishing is irreversible -- PyPI does not permit re-uploading a version -- and this repo has already had a near-miss of exactly this class: 0.2.0 was written into pyproject.toml, never tagged, and never released, so the version in the tree disagreed with reality for three weeks. Port the guard fixtura already runs: compare the pushed tag against the version in pyproject.toml and fail before the build if they differ. Bumps the build-time interpreter 3.10 -> 3.11 for tomllib. This does not narrow what the release supports: the wheel is pure Python (py3-none-any) and requires-python is declared in pyproject.toml, independent of the interpreter that builds it. Also mark the 0.2.0 CHANGELOG section as never released, so nobody reads it as a version they can install. Verified locally: a v0.2.1 tag passes against pyproject 0.2.1, and a mismatched tag is correctly rejected. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This was referenced Jul 29, 2026
Closed
yash161004
added a commit
that referenced
this pull request
Jul 29, 2026
The `test` job installs .[dev], which pulls openeval-core[fixtura,langchain,openai], so every optional dependency is always present and the job cannot detect an adapter that has started requiring one. That is how the eager `from .langchain import ...` in openeval/adapters/__init__.py shipped: the suite stayed green the whole time while `pip install openeval-core` could not import any adapter at all. The fix in #5 removed the bug but left nothing to stop it coming back. Add a bare-install job that reproduces what a real user gets and asserts: - adapters with no third-party dependency import (fixtura, openai -- the openai adapter parses dicts and never imports the openai package) - the LangChain adapter fails only on access, and cleanly - an unknown attribute still raises AttributeError, so the lazy __getattr__ cannot silently swallow typos - the optional deps really are absent, so the job cannot pass vacuously if one ever becomes a base dependency Wired into all-tests-passed, which is the required check, so it actually gates. Verified against a clean venv: all assertions pass on current main, and reinstating the eager import makes the import assertion fail as intended. Closes #6 Co-authored-by: yash161004 <yash161004@github.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Publishing is irreversible — PyPI does not permit re-uploading a version — and this repo has already had a near-miss of exactly this class: 0.2.0 was written into
pyproject.toml, never tagged, and never released, so the tree disagreed with reality for three weeks. That's what madeopeneval-core>=0.1.2look like a valid pin when it shipped no fixtura adapter at all.fixtura already runs this guard (yash161004/fixtura-core#10). Porting it back.
Changes
1. Tag-vs-version check before the build:
2. Build interpreter 3.10 → 3.11, which
tomllibrequires. This does not narrow what the release supports — the wheel is pure Python (py3-none-any, confirmed on the published 0.2.1 artifact) andrequires-python = ">=3.10"is declared inpyproject.toml, independent of the interpreter that builds it.3.
CHANGELOG.md: marked the0.2.0section— NEVER RELEASED, with a note thatpip install openeval-core==0.2.0does not resolve and never will. Previously it read like a shipped release.Verified
v0.2.1vs pyproject0.2.1v9.9.9vs pyproject0.2.1Workflow YAML parses;
permissionsunchanged (id-token: write,contents: read).No version change here — 0.2.1 is already published, so this is CI and docs only.
🤖 Generated with Claude Code