fix(ci): conda publish — anaconda PATH + meta.yaml hardcoded version#428
Closed
lukemartinlogan wants to merge 2 commits into
Closed
fix(ci): conda publish — anaconda PATH + meta.yaml hardcoded version#428lukemartinlogan wants to merge 2 commits into
lukemartinlogan wants to merge 2 commits into
Conversation
The conda publish has been failing on every tag since v1.5.1 with
`exit 127: anaconda: command not found`. Investigating the failed
v1.5.4 run also uncovered a latent bug — every tag was building
`iowarp-core-1.0.0-*.conda` regardless of the tag version.
Bug 1 — anaconda not on PATH (the immediate exit 127):
setup-miniconda@v3 activates the "test" env, but anaconda-client is
installed into "base" (see "Install conda-build" step). With
`bash -l {0}` the login shell activates "test", so the `anaconda`
standalone binary is not on PATH. (The build step gets away with bare
`conda build` because that's a `conda` subcommand; `anaconda` is a
separate entry-point binary.) Fix: invoke via
`conda run -n base --no-capture-output anaconda ...` so it runs from
the env where anaconda-client lives. Mirrors the workflow's existing
"conda-build lives in base" convention.
Bug 2 — meta.yaml hardcoded "1.0.0":
`installers/conda/meta.yaml` had `{% set version = "1.0.0" %}` so the
recipe rendered `iowarp-core-1.0.0-release_*.conda` for every tag.
With `--force` on upload (once bug 1 is fixed) Anaconda would just
keep overwriting the same 1.0.0 package — there'd be no actual
version progression. The pip workflow has a check-version guard for
exactly this case; conda didn't. Fix in three parts:
- meta.yaml: read version from PKG_VERSION env (fallback "1.0.0"
preserves local `conda build installers/conda/` runs).
- Workflow: new "Derive package version from CMakeLists.txt" step
extracts the version via the same regex pip uses and exports it
via $GITHUB_ENV so conda-build's jinja sees it.
- Workflow: new "Verify tag matches CMakeLists.txt version" step
(tag-only) mirroring build-pip.yml's check-version, so a stale
CMakeLists vs tag fails fast instead of silent-overwriting.
Bumps CMakeLists.txt project VERSION 1.5.4 -> 1.5.5 so a follow-up
v1.5.5 tag is the first release that publishes to Anaconda with both
the PATH fix and the correct version.
Supersedes #419 (which only had bug 1's fix and a stale 1.5.2 bump).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The previous commit on this branch (16e4954) drove the conda package version via `environ.get('PKG_VERSION', '1.0.0')` in meta.yaml. That broke install.sh's `conda build` path because conda-build itself *reserves* the name PKG_VERSION inside the meta.yaml `environ` jinja namespace: # conda_build/environ.py:595 (v26.3.0) d["PKG_VERSION"] = meta.version() `meta.version()` is called *while rendering* the recipe, before the version field has been substituted — so it returns Python None. That None is written into the `environ` dict and *overrides* any value os.environ had for PKG_VERSION. environ.get then finds the key present (value None) and returns None instead of the "1.0.0" default (dict.get only uses the default when the key is missing). Result: recipe renders as `iowarp-core-None-release_*.conda`, and the dependency solver later throws libmambapy.bindings.specs.ParseError: Found invalid version predicate in "None" …which is what was killing build-and-test (x86 + arm), all three sanitizer jobs, and Build CUDA conda package on this PR. Fix: - Rename PKG_VERSION -> IOWARP_PKG_VERSION in meta.yaml + the workflow's derive + check-version + build steps. IOWARP_PKG_VERSION is not reserved by conda-build, so the env value flows through. - Add an `or '1.0.0'` fallback in the jinja: defense-in-depth against None/empty values regardless of how environ behaves. Confirmed by local jinja2 simulation across unset / set / None / "" cases. Conda-build reserves more colliding names worth flagging for future: PKG_NAME, PKG_VERSION, PKG_BUILDNUM, PKG_BUILD_STRING, PKG_HASH, RECIPE_DIR, GIT_*, HG_*, SHLIB_EXT, PATH. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Conda publish has failed on every tag since v1.5.1 (v1.5.1, v1.5.3, v1.5.4) with the same
exit 127: anaconda: command not found. Investigating the v1.5.4 failure also surfaced a latent bug: every tag was buildingiowarp-core-1.0.0-*.condaregardless of the tag version, so even after fixing the PATH the publish would--force-overwrite1.0.0indefinitely.Bug 1 —
anaconda: command not found(the immediate exit 127)Cause.
setup-miniconda@v3activates thetestenv, butanaconda-clientis installed intobase(see "Install conda-build" step). Withbash -l {0}the login shell activatestest, so theanacondastandalone binary is not on PATH. The build step gets away with bareconda buildonly because that's acondasubcommand;anacondais a separate entry-point binary.Confirmed verbatim by the v1.5.4 log (run 26148237710):
Fix. Invoke the uploader via
conda run -n base --no-capture-output anaconda ...so it runs explicitly from the env whereanaconda-clientlives. No activation-state churn; mirrors the workflow's existing "conda-build lives in base" convention.Bug 2 —
meta.yamlhardcodedversion = 1.0.0Cause.
installers/conda/meta.yaml:1had{% set version = "1.0.0" %}, so the recipe renderediowarp-core-1.0.0-release_h2bc3f7f.condaregardless of git tag. Visible in the same v1.5.4 build log:The pip workflow has a
check-versionguard for exactly this case; the conda workflow had none.Fix (three parts):
meta.yaml: read version fromPKG_VERSIONenv (environ.get('PKG_VERSION', '1.0.0')). The1.0.0fallback preserves localconda build installers/conda/without setup.grep -oP 'project\(iowarp-core VERSION \K[\d.]+') and exports it via$GITHUB_ENVso conda-build's jinja sees it.build-pip.yml'scheck-version, so a stale CMakeLists vs tag fails fast instead of silent-overwriting1.0.0on Anaconda.Version bump
Bumps
CMakeLists.txtproject VERSION1.5.4→1.5.5so a follow-upv1.5.5tag is the first release that publishes to Anaconda with both the PATH fix and the correct version.Supersedes
Closes #419 — that PR only carried bug 1's PATH fix and a now-stale 1.5.2 bump.
Caveat
Exit 127 means we never reached the auth handshake on past runs, so whether
secrets.ANACONDA_TOKENis set/valid remains untested. After this PATH fix lands andv1.5.5is tagged, a missing/invalid token would surface as an auth-fail in the upload step (different error class). One thing at a time.Test plan
v1.5.5tag: "Verify tag matches CMakeLists.txt version" passes; "Build conda package" producesiowarp-core-1.5.5-…conda; "Upload to Anaconda.org" actually reaches the auth handshake.🤖 Generated with Claude Code