Land the build/CI overhaul's shared workflows and actions - #187
Merged
Conversation
Lets callers opt into publishing to Test PyPI instead of the real PyPI, independent of dry_run.
…ypi through deploy_nwx_pypi Fork-test-only commit: makes this fork's reusable workflows/actions self-reference the fork instead of calling back to the real NWChemEx/.github, and threads a use_test_pypi input through deploy_nwx_pypi.yaml so downstream repos can opt into Test PyPI on their merge workflow.
…-trampoline actions/runner#2473 bug in nested composite actions)
…i.yaml (broken with OIDC trusted publishing)
…0's bundled twine predates Metadata-Version 2.4)
… was reusable-workflow nesting, not action version)
…action-pypi-publish nesting limitation, confirmed via fork testing)
…confirmed via fork testing)
The bare-runner path has no pre-baked Python environment (unlike the old nwx_buildenv container), so repos whose Python tests need extra packages (e.g. numpy) failed with ModuleNotFoundError. Mirrors the old test_nwx_library.yaml's conditional requirements.txt install. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…PI-interdependent packages Packages that depend on other nwchemex-* packages only published to TestPyPI (not real PyPI) need pip to search both indexes during the post-build install sanity check, or verification fails with "no matching distribution found" for the not-yet-on-real-PyPI dependency. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
pypa/gh-action-pypi-publish only runs on Linux runners, so a per-matrix-leg publish step fails outright on the macOS leg. Splits build (every release matrix leg, artifact-uploaded) from publish (one ubuntu-latest job that downloads and merges all dist/ artifacts, then publishes once). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…reusable workflows Confirmed via a live failed run: pypa/gh-action-pypi-publish rejects OIDC claims whose workflow_ref points at a called reusable workflow rather than the actual calling repo's own workflow file (job_workflow_ref vs workflow_ref mismatch) -- publishing must happen in a job defined directly in each repo's own merge.yaml, not delegated. This is the same lesson already encoded in this repo's history for build_pypi_dist/deploy_to_pypi (see f17152f, b27520a) -- platform_matrix.yaml and build_pypi_dist remain fine to call since neither requests an id-token. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…el containers The manylinux/macOS cibuildwheel build environments are minimal and don't have MPI, unlike the ambient dev matrix. Verified locally against the exact manylinux2014_x86_64 image: openmpi3-devel (via EPEL, already baked into the image) provides mpicc/mpicxx under /usr/lib64/openmpi3/bin, which isn't on PATH by default -- CMake's find_package(MPI) only succeeds once that's added to PATH. macOS just needs `brew install open-mpi`. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Temporary(ish) diagnostic aid -- surfaces the CMake install log (rpath, destinations) inside cibuildwheel's own isolated build, which was otherwise invisible when a build/repair step failed. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Homebrew's open-mpi (and its own deps: libpmix, libhwloc, libevent) are built against the actual macos-14 runner OS, so their dylibs declare a minimum target of 14.0 -- higher than cibuildwheel's default arm64 target of 11.0. delocate then refuses to tag the wheel as 11.0-compatible, since the bundled libraries require 14.0. Raises the declared target to match reality whenever MPI is being bundled. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…a shell step Boost was never actually installed anywhere in the manylinux/macOS cibuildwheel containers -- only the CMake Module-vs-Config policy was fixed (in NWXCMake), but nothing yum/brew-installs boost-devel/boost itself. Moved the CIBW_BEFORE_ALL_*/CIBW_ENVIRONMENT_* composition from inline ternary expressions (which can't cleanly combine two independent needs_* flags without one clobbering the other) into a shell step that builds each value up conditionally and exports via GITHUB_ENV. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…anylinux manylinux2014's plain boost-devel (1.53.0) lacks headers some repos need (e.g. boost/container_hash/hash.hpp, confirmed missing when building TensorWrapper -- pluginplay's Boost usage happened to predate that gap). EPEL's boost169-devel is the newest available via yum there; it installs under a versioned include path (/usr/include/boost169), so BOOST_INCLUDEDIR must point CMake's FindBoost at it explicitly. Verified locally (docker manylinux2014_x86_64): find_package(Boost) resolves 1.69.0 with BOOST_INCLUDEDIR=/usr/include/boost169 set. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…v: install BLAS/LAPACK GauXC (SCF's dependency) unconditionally find_package(BLAS)s and find_package(OpenMP)s, and neither the manylinux/macOS cibuildwheel containers nor the ambient dev-matrix runners have them by default. For the ambient dev matrix (setup_nwx_dev_env, ubuntu/macOS apt/brew), install openblas/lapack unconditionally alongside MPI/Boost, matching the existing pattern there -- gcc-14/llvm@18 already ship their own OpenMP runtime, so no extra provisioning is needed for that. For cibuildwheel (build_pypi_dist), add needs_blas (openblas-devel + lapack-devel via EPEL on manylinux, openblas via Homebrew on macOS) and needs_openmp (libomp via Homebrew on macOS only -- AppleClang has no built-in OpenMP support and CMake's FindOpenMP can't locate Homebrew's keg-only libomp without explicit SKBUILD_CMAKE_ARGS hints; Linux's gcc ships libgomp already).
Some repos' CMake build invokes a Python codegen script at configure or build time (e.g. GauXC's gau2grid dependency imports numpy), using whichever python3 is on PATH by default in this container. Install into that same interpreter before CMake ever runs, rather than only in the module-docs/Sphinx steps later (which activate the separate /pyenv environment instead).
…v_env libint2's own CMakeLists does its own find_package(Eigen3), independent of and unsatisfied by nwxcmake's FetchContent'd Eigen target. Neither the cibuildwheel containers nor the ambient dev-matrix runners have an installed Eigen3 CMake config by default.
…s.txt install The container's system python3 is PEP 668 externally-managed and refuses a plain pip install; --break-system-packages is safe here since the whole container is ephemeral CI state.
Unlike gcc (which bundles libgomp), apt's clang-18 package has no built-in OpenMP runtime, so GauXC's unconditional find_package(OpenMP) failed on the ambient Linux+clang-18 dev-matrix leg.
llvm@18's own clang has no bundled OpenMP runtime either; CPATH/LIBRARY_PATH let find_package(OpenMP)'s compiler check and library search succeed without threading explicit -I/-L flags through every cmake_opts caller.
Some C++ tests embed a Python interpreter that imports this repo's own built pybind11 modules (or a dependency's, e.g. parallelzone) -- ctest had no PYTHONPATH at all, only the separate pytest step did.
The plain (non-cibw) path's own wheel is version-agnostic, but a pure- Python package's runtime dependencies can include compiled nwx-ecosystem packages only published for cp312 (per platform_matrix.yaml). Verifying under whatever 'Set up Python's "3.x" resolves to today (3.14) found no matching wheel for those and fell back to building from source, which fails on the bare verification venv (no MPI/Boost/etc). Surfaced by FriendZone's first real Merge Workflow run.
…ng Ubuntu's apt version EPEL7's openmpi3-devel (3.1.3) is ABI-incompatible with the OpenMPI 4.1.6 that Ubuntu 24.04's apt package provides (and that setup_nwx_dev_env's ambient Linux install uses) -- OpenMPI does not guarantee ABI compatibility across major versions. A wheel built against 3.1.3 segfaults (no Python traceback, a native-level crash) when actually imported somewhere with only 4.x installed, which is exactly what happens the first time a prebuilt MPI-linked wheel (e.g. nwchemex-simde) gets imported and run outside its own release build -- previously unexercised, since every ambient dev-matrix test builds its own MPI-linked code from source against whatever MPI is locally installed, always matching itself. EPEL7 (manylinux2014's package repo) has no openmpi4 package at all, so this can't be a yum install -- build the exact same version Ubuntu ships from source instead. Only costs build time on release/merge builds (once per push), not every PR leg the way fixing this in setup_nwx_dev_env instead would have. macOS already had no such mismatch: both build_pypi_dist's needs_mpi and setup_nwx_dev_env's ambient install already use the identical 'brew install open-mpi' formula on both sides.
The version-matching fix alone (previous commit) didn't resolve the crash -- diagnostics showed the wheel's bundled libmpi resolves fine via ldd (auditwheel does bundle the .so), but MPI_Init itself fails at runtime: 'An error occurred in MPI_Init on a NULL communicator', with OpenMPI's own help text revealing it can't find /usr/local/openmpi/share/openmpi/help-*.txt -- runtime data/plugin files hardcoded to the build container's compile-time prefix, which auditwheel has no way to bundle (it only relocates ELF DT_NEEDED dependencies, not arbitrary data directories). --enable-mpirun-prefix-by-default makes OpenMPI compute that path relative to the loaded library at runtime instead of a hardcoded absolute path -- the standard fix for exactly this relocatable-install scenario.
The mpirun-prefix flag (previous commit) didn't fix it either -- pointing OPAL_PREFIX at a copy of OpenMPI's own help text (previously missing, itself masking the real error) revealed the actual failure: 'opal_shmem_base_select failed'. OpenMPI's MCA components (shmem, btl, etc.) are separate plugin .so files under $prefix/lib/openmpi/, loaded via runtime dlopen() rather than standard ELF linking -- auditwheel only bundles a wheel's DT_NEEDED dependencies (found via ldd), so it has no way to even discover these, let alone bundle them. --disable-dlopen statically links every MCA component directly into libmpi.so/libopen-pal.so instead, so there's no runtime plugin directory to find in the first place -- the standard fix for exactly this portable/relocatable-build scenario (used by e.g. conda-forge's openmpi packaging).
… OpenMPI Diagnostics on FriendZone PR #1 confirmed --disable-dlopen fixed MCA shmem component selection, but the vendored wheel still crashes one step later: OpenMPI's singleton-mode ORTE startup can't find the orted daemon executable, since auditwheel/delocate only ever bundle shared libraries, never the accompanying bin/ executables. Vendoring MPI into a portable wheel is a dead end regardless of build flags, so exclude libmpi/libopen-rte/libopen-pal from the Linux repair step and require a real system OpenMPI at runtime instead (already provided ambient by setup_nwx_dev_env's libopenmpi-dev/openmpi-bin, and documented as an end-user runtime requirement). Keeps the from-source OpenMPI 4.1.6 build, still needed to compile against a matching version. macOS is left vendoring via delocate for now, since it previously passed test_python with brew's bundled OpenMPI.
…lobs The Linux MPI-exclusion fix landed but silently didn't work: the published wheels still bundled libmpi/libopen-rte/libopen-pal (confirmed by downloading nwchemex-parallelzone from TestPyPI and finding the hash-suffixed grafted libs still present), so FriendZone's test_python(ubuntu-latest) still failed after excluding vendored MPI, just with a different symptom (silent crash with no traceback instead of the old orted-missing error). Root cause, confirmed by reproducing the exact repair step locally via Docker against the pinned manylinux2014 image (quay.io/pypa/manylinux2014_x86_64:2024.11.16-1, matching cibuildwheel v2.22.0's default): that image bakes in auditwheel 6.1.0, whose --exclude implementation does a plain `soname in exclude` set-membership check, not fnmatch -- glob support for --exclude was only added in a later auditwheel release. 'libmpi*' matched nothing and silently bundled the library anyway; the exact soname 'libmpi.so.40' correctly excluded it. Excluding libmpi.so.40 alone is sufficient since libopen-rte/libopen-pal are only reachable transitively through it in this tree, but keep the explicit excludes for those two as well in case some other extension links them directly.
FriendZone's and NWChemEx's integration tests wrap the external NWChem program (nwx2molssi/nwx2ase, compute_energy) but are silently gated off by friends.is_nwchem_enabled() since no CI job installs it. Add an opt-in flag that installs a real nwchem binary: brew on macOS, a standalone micromamba + conda-forge env on Linux (no apt/pip package ships the actual binary). Off by default -- only the two repos whose tests need it should set it.
FriendZone's real-NWChem CI hung indefinitely on test_ccsd_t (11+ min, stuck orted process) right after test_ccsd failed cleanly with "bas_tag_lib: failed opening basis file". Root-caused by reproducing locally in Docker: conda-forge's nwchem binary has a hardcoded default NWCHEM_BASIS_LIBRARY pointing at the feedstock's own build directory, which doesn't exist outside their build container. The package's own etc/conda/activate.d/nwchem_env.sh sets the correct paths, but only on `conda activate`, which this action deliberately avoids (just PATH, to not shadow the job's Python venv). Confirmed via direct repro that exporting these two vars makes an identical CCSD(T) H2/sto-3g calculation that previously would have hung complete cleanly in ~1.4s. Initially suspected an MPI 4.1.6-vs-5.0.10 ABI mismatch (system apt MPI vs conda-forge's bundled one); ruled that out empirically -- the same PATH/orted mismatch caused no issue once the basis library paths were set.
SCF's integration tests need a pinned cp312 interpreter (ecosystem wheels are cp312-only) and need nwchemex pip-installed into the ambient Python before ctest runs (CTest-driven Python tests use the ambient Python_EXECUTABLE, whose sys.path always includes site-packages regardless of CTest's PYTHONPATH override). Both inputs default to current behavior, so every other caller is unaffected.
Without an explicit Python_EXECUTABLE, CMake's own Python search can resolve a different ambient interpreter than the one setup-python put on PATH (and that the "Install Python Requirements" step just pip installed into) -- e.g. a macOS Homebrew framework Python. Surfaced by SCF's new python_version: "3.12" pin: gau2grid's build-time codegen script embeds whatever interpreter CMake found, which didn't have the numpy that was installed into the pinned 3.12.
The reusable workflows/actions in this repo call back into themselves (platform_matrix.yaml, setup_nwx_dev_env, pip_build, cmake_build) using an absolute owner/repo@ref, which still pointed at the personal fork these were developed in. Now that this branch is landing in the NWChemEx org, those self-references need to resolve here instead.
jwaldrop107
reviewed
Aug 5, 2026
Comment on lines
+24
to
+27
| # This is Claude/Cursor directory | ||
| .claude | ||
| .cursor | ||
|
|
Contributor
|
🚀 [bumpr] Bumped! |
2 tasks
ryanmrichard
added a commit
to NWChemEx/Utilities
that referenced
this pull request
Aug 5, 2026
Repoints this repo's workflows at NWChemEx/.github (landed in NWChemEx/.github#187) instead of the ryanmrichard fork they were developed against, and switches merge.yaml's and pull_request.yaml's branch triggers from build_overhaul to master now that master is the ecosystem's working branch. Drops the test.pypi.org repository-url override on the merge workflow's publish step, so a merge to master publishes to real PyPI -- nwchemex-utilities is free there and a trusted publisher has been registered for it. get_nwx_cmake.cmake's FetchContent fallback and pyproject.toml's runtime dependency both repoint at NWChemEx/NWXCMake; the latter also switches from a git+ dependency to a published floor (nwchemex-nwxcmake>=0.1.0), now that NWXCMake has an actual PyPI release to pin against.
2 tasks
ryanmrichard
added a commit
to NWChemEx/ParallelZone
that referenced
this pull request
Aug 5, 2026
Repoints this repo's workflows at NWChemEx/.github (landed in NWChemEx/.github#187) instead of the ryanmrichard fork they were developed against, and switches merge.yaml's and pull_request.yaml's branch triggers from build_overhaul to master now that master is the ecosystem's working branch. Drops the test.pypi.org extra_index_url and repository-url overrides -- this repo's runtime dependencies (nwxcmake, utilities) are both now on real PyPI, so a merge to master can publish nwchemex-parallelzone there directly (trusted publisher registered). get_nwx_cmake.cmake's FetchContent fallback and pyproject.toml's runtime dependencies both repoint at NWChemEx/NWXCMake and NWChemEx/Utilities, switching the latter two from git+ dependencies to published floors (nwchemex-nwxcmake>=0.1.0, nwchemex-utilities>=0.1.45).
2 tasks
ryanmrichard
added a commit
to NWChemEx/ParallelZone
that referenced
this pull request
Aug 5, 2026
* refactors build system * disable MPI tests for now * update distribution name * pass includes to catch2_tests directly * add dependency * code factorization * Retarget NWChemEx/ -> ryanmrichard/ for fork testing; adopt test_nwx_cmake_build Fork-test-only commit: points merge.yaml/pull_request.yaml at the ryanmrichard/.github fork's reusable workflows instead of the real NWChemEx/.github, triggers off build_overhaul (this fork's working branch), and swaps the older test_nwx_library job for test_nwx_cmake_build with Python tests enabled. Also retargets the NWXCMake FetchContent fallback to ryanmrichard/NWXCMake, since the nwx_set_version refactor this branch needs only exists there. Adds tests/python/conftest.py with a session-scoped RuntimeView fixture so the reusable action's standalone `pytest -v` step works: today these tests only run safely through a hand-rolled aggregator script (test_parallelzone.py) that holds one RuntimeView alive across the whole unittest run; without an equivalent for plain pytest, MPI would be finalized after the first test module and every module after it would fail to re-initialize it. * Trigger CI (NWXCMake PIC fix landed on fork) * isort: reorder conftest.py imports (parallelzone before pytest) * Use steady_clock instead of high_resolution_clock in CPU::profile_it high_resolution_clock isn't guaranteed to actually be high-resolution (it's often just an alias for system_clock), so wall_time.count() > 0 failed deterministically on the macos-14/gcc-14 CI leg for a near-instant lambda call. steady_clock is the portable choice for measuring elapsed durations. * Relax CPU::profile_it wall_time assertion to >= 0 Switching to steady_clock didn't fix the macos-14/gcc-14 CI failure: the timed lambda call is fast enough to measure as 0 ticks even on a nanosecond-typed clock, since actual OS timer resolution can be coarser than the type suggests. Non-negativity is the only portable invariant for a near-instant operation's elapsed time. * Add TestPyPI publish job to merge.yaml; retarget NWXCMake build dep to fork Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * Use shared deploy_to_pypi.yaml (fixes macOS-leg publish failure) Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * Inline deploy_to_pypi (reusable workflows unsupported by PyPI trusted publishing) Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * deploy_to_pypi: install MPI inside the cibuildwheel manylinux/macOS containers Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * Retrigger CI: nwxcmake now installs transitive shared deps for wheel repair * Retrigger CI: fixed wheel rpath for shared transitive deps * Retrigger CI: debug verbosity bump for wheel build * Retrigger CI: force CMAKE_INSTALL_LIBDIR=lib (fixes manylinux2014 lib64 split) * deploy_to_pypi: add extra_index_url (parallelzone depends on nwchemex-utilities) Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * Re-trigger CI to rebuild wheel with OpenMPI 4.1.6 fix (matches Ubuntu's ambient MPI ABI) * Re-trigger CI to rebuild wheel with OpenMPI --enable-mpirun-prefix-by-default fix * Re-trigger CI to rebuild wheel with OpenMPI --disable-dlopen fix * Republish: pick up build_pypi_dist fix (stop vendoring MPI, require system MPI on Linux) * Republish: fix build_pypi_dist to use exact sonames for auditwheel --exclude (glob patterns don't work on cibuildwheel v2.22.0's pinned auditwheel 6.1.0) * use fork * update pyproject.toml * add pre-commit to gitignore * update pyproject.toml * adds python build * move nwxcmake/pybind11 to project * update nightly * Migrate CI and dependencies from ryanmrichard to NWChemEx Repoints this repo's workflows at NWChemEx/.github (landed in NWChemEx/.github#187) instead of the ryanmrichard fork they were developed against, and switches merge.yaml's and pull_request.yaml's branch triggers from build_overhaul to master now that master is the ecosystem's working branch. Drops the test.pypi.org extra_index_url and repository-url overrides -- this repo's runtime dependencies (nwxcmake, utilities) are both now on real PyPI, so a merge to master can publish nwchemex-parallelzone there directly (trusted publisher registered). get_nwx_cmake.cmake's FetchContent fallback and pyproject.toml's runtime dependencies both repoint at NWChemEx/NWXCMake and NWChemEx/Utilities, switching the latter two from git+ dependencies to published floors (nwchemex-nwxcmake>=0.1.0, nwchemex-utilities>=0.1.45). --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
ryanmrichard
added a commit
to NWChemEx/TensorWrapper
that referenced
this pull request
Aug 5, 2026
Repoints this repo's workflows at NWChemEx/.github (landed in NWChemEx/.github#187) instead of the ryanmrichard fork they were developed against, and switches merge.yaml's and pull_request.yaml's branch triggers from build_overhaul to master now that master is the ecosystem's working branch. Drops the test.pypi.org extra_index_url and repository-url overrides -- this repo's runtime dependencies (nwxcmake, utilities, parallelzone) are all now on real PyPI, so a merge to master can publish nwchemex-tensorwrapper there directly (trusted publisher registered). get_nwx_cmake.cmake's FetchContent fallback and pyproject.toml's runtime dependencies repoint at NWChemEx/NWXCMake, switching all three Python dependencies from git+ URLs to published floors (nwchemex-nwxcmake>=0.1.0, nwchemex-utilities>=0.1.45, nwchemex-parallelzone>=0.1.43).
ryanmrichard
added a commit
to NWChemEx/PluginPlay
that referenced
this pull request
Aug 5, 2026
Repoints this repo's workflows at NWChemEx/.github (landed in NWChemEx/.github#187) instead of the ryanmrichard fork they were developed against, and switches merge.yaml's and pull_request.yaml's branch triggers from build_overhaul to master now that master is the ecosystem's working branch. Drops the test.pypi.org extra_index_url and repository-url overrides -- this repo's runtime dependencies (nwxcmake, utilities, parallelzone) are all now on real PyPI, so a merge to master can publish nwchemex-pluginplay there directly (trusted publisher registered). get_nwx_cmake.cmake's FetchContent fallback and pyproject.toml's runtime dependencies repoint at NWChemEx/NWXCMake, switching all three Python dependencies from git+ URLs to published floors (nwchemex-nwxcmake>=0.1.0, nwchemex-utilities>=0.1.45, nwchemex-parallelzone>=0.1.43).
This was referenced Aug 5, 2026
ryanmrichard
added a commit
to NWChemEx/PluginPlay
that referenced
this pull request
Aug 5, 2026
Repoints this repo's workflows at NWChemEx/.github (landed in NWChemEx/.github#187) instead of the ryanmrichard fork they were developed against, and switches merge.yaml's and pull_request.yaml's branch triggers from build_overhaul to master now that master is the ecosystem's working branch. Drops the test.pypi.org extra_index_url and repository-url overrides -- this repo's runtime dependencies (nwxcmake, utilities, parallelzone) are all now on real PyPI, so a merge to master can publish nwchemex-pluginplay there directly (trusted publisher registered). get_nwx_cmake.cmake's FetchContent fallback and pyproject.toml's runtime dependencies repoint at NWChemEx/NWXCMake, switching all three Python dependencies from git+ URLs to published floors (nwchemex-nwxcmake>=0.1.0, nwchemex-utilities>=0.1.45, nwchemex-parallelzone>=0.1.43).
ryanmrichard
added a commit
to NWChemEx/PluginPlay
that referenced
this pull request
Aug 6, 2026
* switched over to new build system
* refactor CMake
* adds upstream dependencies
* code cleanup
* add mac files to gitignore
* Retarget NWChemEx/ -> ryanmrichard/ for fork testing; adopt test_nwx_cmake_build
Fork-test-only commit: points merge.yaml/pull_request.yaml at the
ryanmrichard/.github fork's reusable workflows instead of the real
NWChemEx/.github, triggers off build_overhaul (this fork's working
branch), and swaps the older test_nwx_library job for
test_nwx_cmake_build with Python tests enabled. Also retargets the
NWXCMake FetchContent fallback to ryanmrichard/NWXCMake, since the
nwx_set_version refactor this branch needs (and NWXCMake's retargeted
parallelzone/utilities dependency fetches) only exist there.
Adds tests/python/conftest.py with a session-scoped RuntimeView
fixture, same as ParallelZone, so the reusable action's standalone
`pytest -v` step works safely alongside the existing unittest
aggregator script that CTest already runs.
* Apply pre-commit formatting fixes (end-of-file, clang-format include order)
* Fix isort import ordering; link Python::Python into unit_test_pluginplay
isort: reorder imports so pluginplay/parallelzone/py_test_pluginplay
group with no blank line, matching how CI's isort classifies them
(they don't split cleanly into separate first/third-party sections).
CMakeLists.txt: unit_test_pluginplay directly exercises the CPython
C API (tests/cxx/unit_tests/pluginplay/python/python_wrapper.cpp),
unlike the .so extension (built via pybind11_add_module, which
intentionally links only Python::Module, not libpython, so the module
can be loaded by any Python providing those symbols at runtime). A
standalone executable embedding CPython calls needs libpython itself,
via Python::Python (Development.Embed component), or linking fails
with undefined references to _Py_Dealloc and friends.
* Link Python::Python on the library itself, not just the unit test target
The undefined-reference failures weren't limited to unit_test_pluginplay:
libpluginplay.so itself carries unresolved CPython symbols (python_wrapper.hpp
is a public header compiled into the library whenever BUILD_PYBIND11 is set),
so test_regression_pluginplay hit the same failure once it also linked the
library into an executable. Moving the Python::Python link onto
${PROJECT_NAME} itself (PUBLIC) fixes it for all three Catch2 test binaries
instead of patching them one at a time.
* Build py_test_pluginplay pybind11 test-helper module
Ported from the old build system's nwx_add_pybind11_module(py_test_pluginplay
SOURCE_DIR tests/python/unit_tests ...) call, which build_overhaul's
CMakeLists.txt never got an equivalent for. The .cpp sources
(test_pluginplay.cpp and friends, defining PYBIND11_MODULE(py_test_pluginplay,
m)) were still present in tests/python/unit_tests/ but nothing built them,
so `import py_test_pluginplay` in the Python test suite failed with
ModuleNotFoundError.
* Build pluginplay_examples pybind11 module
Same gap as py_test_pluginplay: the old build system built this via
nwx_add_pybind11_module(${PROJECT_NAME}_examples SOURCE_DIR
tests/python/doc_snippets DEPENDS parallelzone ${PROJECT_NAME}
${PROJECT_NAME}_examples), where ${PROJECT_NAME}_examples was a small
library built from tests/cxx/doc_snippets. build_overhaul's CMakeLists.txt
never got an equivalent, so `import pluginplay_examples` in the Python
doc-snippet tests failed with ModuleNotFoundError.
* Reuse nwx_python_module for py_test_pluginplay/pluginplay_examples
Renames the 10 test-helper .cpp files under tests/python/unit_tests/ to
export_*.cpp so they match nwx_python_module's existing source filter
(all 10 compile into the same module: test_pluginplay.cpp's single
PYBIND11_MODULE calls out to a def_submodule()-registering function in
each of the others), then replaces the bespoke pybind11_add_module
blocks for both py_test_pluginplay and pluginplay_examples with calls
to the now-generalized nwx_python_module (NO_INSTALL, and DEPENDS for
pluginplay_examples' extra link libraries).
* Add TestPyPI publish job to merge.yaml; retarget NWXCMake build dep to fork
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* Use shared deploy_to_pypi.yaml (fixes macOS-leg publish failure)
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* Inline deploy_to_pypi (reusable workflows unsupported by PyPI trusted publishing)
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* deploy_to_pypi: install MPI inside the cibuildwheel manylinux/macOS containers
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* Retrigger CI: pick up nwxcmake rpath/libdir fixes and macOS deployment target bump
* Only require Development.Embed/link libpython when BUILD_TESTING
manylinux wheel-build containers deliberately don't ship libpython.so at
all (portable wheels aren't supposed to assume one), so requiring
Development.Embed unconditionally broke the wheel-build path there. The
full libpython link (Python::Python) is only actually needed by the
Catch2 test executables (an executable can't defer symbol resolution to
runtime the way a shared library/extension can); the library itself only
needs Python.h to compile, which Python::Module (Development.Module,
always available) already provides without linking libpython.
Verified end-to-end (docker quay.io/pypa/manylinux2014_x86_64): configures,
builds, and `auditwheel repair` succeeds with BUILD_TESTING=OFF (the
default for a non-editable wheel build).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* deploy_to_pypi: add needs_boost (Boost was never actually installed in the container)
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* Re-trigger CI to rebuild wheel with OpenMPI 4.1.6 fix (matches Ubuntu's ambient MPI ABI)
* Re-trigger CI to rebuild wheel with OpenMPI --enable-mpirun-prefix-by-default fix
* Re-trigger CI to rebuild wheel with OpenMPI --disable-dlopen fix
* Republish: pick up build_pypi_dist fix (stop vendoring MPI, require system MPI on Linux)
* Republish: fix build_pypi_dist to use exact sonames for auditwheel --exclude (glob patterns don't work on cibuildwheel v2.22.0's pinned auditwheel 6.1.0)
* updates build
* use fork
* fix rtti issue and update tests for new build
* adds python interpreter to pluginplay
* adds python build
* move nwxcmake/pybind11 to project
* update nightly
* Migrate CI and dependencies from ryanmrichard to NWChemEx
Repoints this repo's workflows at NWChemEx/.github (landed in
NWChemEx/.github#187) instead of the ryanmrichard fork they were
developed against, and switches merge.yaml's and pull_request.yaml's
branch triggers from build_overhaul to master now that master is the
ecosystem's working branch.
Drops the test.pypi.org extra_index_url and repository-url overrides
-- this repo's runtime dependencies (nwxcmake, utilities, parallelzone)
are all now on real PyPI, so a merge to master can publish
nwchemex-pluginplay there directly (trusted publisher registered).
get_nwx_cmake.cmake's FetchContent fallback and pyproject.toml's
runtime dependencies repoint at NWChemEx/NWXCMake, switching all three
Python dependencies from git+ URLs to published floors
(nwchemex-nwxcmake>=0.1.0, nwchemex-utilities>=0.1.45,
nwchemex-parallelzone>=0.1.43).
* add blank line
---------
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
ryanmrichard
added a commit
to NWChemEx/TensorWrapper
that referenced
this pull request
Aug 6, 2026
* build overhaul * adds upstream dependencies * code factorization * adds mac files to gitignore * Build py_test_tensorwrapper via nwx_python_module The old build system built this test-helper pybind11 module via nwx_add_pybind11_module(py_test_tensorwrapper SOURCE_DIR tests/python/unit_tests ...), which build_overhaul's CMakeLists.txt never got an equivalent for -- test_tensorwrapper.cpp (still defining PYBIND11_MODULE(py_test_tensorwrapper, m)) was left in the tree but nothing built it, so tests/python/unit_tests/tensor/test_tensor.py's `import py_test_tensorwrapper.testing` would fail with ModuleNotFoundError. Renamed to export_test_tensorwrapper.cpp so it matches nwx_python_module's source filter, then wired up via nwx_python_module(py_test_tensorwrapper "tests/python/unit_tests" NO_INSTALL) -- reusing the same macro extended (NO_INSTALL/DEPENDS) for PluginPlay's analogous py_test_pluginplay/pluginplay_examples gap. No ryanmrichard/TensorWrapper fork exists yet (only Utilities, ParallelZone, and PluginPlay were fork-tested this pass), so this is a local-only commit pending future CI verification. * Retarget NWChemEx/ -> ryanmrichard/ for fork testing; adopt test_nwx_cmake_build Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * Retrigger CI now that Actions are enabled on fork * Add requirements.txt (numpy) for Python tests; fix .gitignore missing trailing newline Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * Add session-scoped RuntimeView pytest fixture; fix isort ordering for Python 3.14 Without a session-scoped RuntimeView, MPI is finalized after the first Tensor construction in TestTensor.setUp, so the second construction aborts with "MPI_Comm_rank() called after MPI_FINALIZE". Mirrors the fixture already added to ParallelZone/PluginPlay for the same reason. Also fixes an isort import-grouping difference between Python 3.13 and 3.14 that only reproduces under 3.14 (confirmed via docker run python:3.14). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * Fix Sigma toolchain cmake_opts: github.workspace is empty in the calling (uses:-only) job The pull_request.yaml job that delegates to test_nwx_cmake_build.yaml never runs on a runner itself, so github.workspace resolves empty there, producing "-DCMAKE_TOOLCHAIN_FILE=/.github/enable_sigma.cmake" (root-relative, missing file). The actual cmake invocation always runs from the checked-out repo root, so a plain relative path works and was verified locally (docker ubuntu:24.04 + gcc-14) to configure and build cleanly. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * Retrigger CI: workflow write permissions now enabled * deploy_to_pypi: add needs_boost (Boost was never actually installed in the container) Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * Retarget NWXCMake build dependency to fork Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * Retrigger CI: use boost169-devel with BOOST_INCLUDEDIR * Simplify Sigma CI job to a plain cmake_opts, drop stale ENABLE_SIGMA TODO ENABLE_SIGMA is already a real CMake option defined globally by nwxcmake's set_default_nwx_options.cmake, so no toolchain file is needed to set it for CI, and no placeholder TODO is needed in CMakeLists.txt. * Re-trigger CI to rebuild wheel with OpenMPI 4.1.6 fix (matches Ubuntu's ambient MPI ABI) * Re-trigger CI to rebuild wheel with OpenMPI --enable-mpirun-prefix-by-default fix * Re-trigger CI to rebuild wheel with OpenMPI --disable-dlopen fix * Republish: pick up build_pypi_dist fix (stop vendoring MPI, require system MPI on Linux) * Republish: fix build_pypi_dist to use exact sonames for auditwheel --exclude (glob patterns don't work on cibuildwheel v2.22.0's pinned auditwheel 6.1.0) * use fork * fix typo * add pre-commit to gitignore * update pyproject.toml * works in dev mode * adds python build * move nwxcmake/pybind11 to project * update nightly * Migrate CI and dependencies from ryanmrichard to NWChemEx Repoints this repo's workflows at NWChemEx/.github (landed in NWChemEx/.github#187) instead of the ryanmrichard fork they were developed against, and switches merge.yaml's and pull_request.yaml's branch triggers from build_overhaul to master now that master is the ecosystem's working branch. Drops the test.pypi.org extra_index_url and repository-url overrides -- this repo's runtime dependencies (nwxcmake, utilities, parallelzone) are all now on real PyPI, so a merge to master can publish nwchemex-tensorwrapper there directly (trusted publisher registered). get_nwx_cmake.cmake's FetchContent fallback and pyproject.toml's runtime dependencies repoint at NWChemEx/NWXCMake, switching all three Python dependencies from git+ URLs to published floors (nwchemex-nwxcmake>=0.1.0, nwchemex-utilities>=0.1.45, nwchemex-parallelzone>=0.1.43). --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ryanmrichardfork (.github/actions/{build_pypi_dist,cmake_build,pip_build,setup_nwx_dev_env}, plus updates totest_nwx_cmake_build.yaml,test_nwx_pip_build.yaml,platform_matrix.yaml,tag.yaml) that the ecosystem's per-repopull_request.yaml/merge.yaml/nightly.yamlworkflows are about to be repointed at.platform_matrix.yaml,setup_nwx_dev_env,pip_build,cmake_build) fromryanmrichard/.githubtoNWChemEx/.github, now that this is landing in the org.ryanmrichardaccount and ontoNWChemEx/real PyPI; every downstream repo's workflow files will call back into this branch once merged.Test plan
new-actions's own PR workflow (check_formatting.yaml, self-referenced via a local path) passesNWChemEx/.githubmerge tomasterexposesbuild_pypi_dist,cmake_build,pip_build,setup_nwx_dev_envunder.github/actions/