From 924064ae9d5e78139516454f5f6a2126fca24e0b Mon Sep 17 00:00:00 2001 From: snkmcb <88454117+snkmcb@users.noreply.github.com> Date: Thu, 24 Sep 2026 11:48:46 +0900 Subject: [PATCH 1/3] Resolve each dependency where it is linked, and prove plain CMake without ost The root CMakeLists.txt resolved all five usd-motion-plugins packages up front, so every configure required every package -- motionRecording included, which nothing here includes. Each member now resolves what it links through usdvrm_consume_package() (cmake/UsdVrmConsumedPackage.cmake), which promotes the targets it creates to IMPORTED_GLOBAL so a later member and a root-registered test reuse one definition. The root orchestrates and resolves nothing on a member's behalf. EDGES THAT WERE NEVER USED ARE GONE. execVrm linked, and its descriptor pinned, motionSampling and motionRecording since MIG-1 split motionRuntime, with no source including either. motion_retarget included motionCore/ headers through motionRetarget's link line and now links it by name. Measured: every member configures and builds standalone against a prefix holding only its own packages; usdVrmFileFormat, usdVrmPackageResolver, vrmSchema and vrmContainer need no motion package at all, and main's execVrm fails the same test on motionSampling. usdvrm_baseline asked whether motionCore's target was visible as a stand-in for "is this the root build", which would have silently unregistered the gate once the root stopped resolving packages. It asks USDVRM_COMPOSED_BUILD now. A CMAKE DEPENDENCY AUDIT, scripts/check_cmake_boundaries.py (workspace_cmake_boundaries + selftest): no source tree of another repository, no usd-motion-plugins identity built here, the root resolving nothing, each member's reachable packages within WORKSPACE.md section 2 (the VRM importer none; the .vrma importer neither vrmRig nor motionRetarget), resolved = linked = included per member, and descriptor = CMake. Given an installed motion include/ it also fails on code there that knows VRM. On main it reports 17 violations. A PLAIN-CMAKE LANE, .github/workflows/plain-cmake.yml: OpenUSD install + cmake --install of usd-motion-plugins -> cmake -> build -> ctest on Linux, refusing to run with ost on PATH. It copies no pin: scripts/plain_cmake_inputs.py fetches the Linux workspace cell's runtime archive and execVrm's execMotion bundle by digest, and builds usd-motion-plugins from the commit the pinned packages' SLSA provenance names. ITS FIRST RUN FOUND A DEFECT that ost's activation hid. The published libExecMotion.so carries the producer's CI checkout as its RUNPATH, so it opens only when LD_LIBRARY_PATH names OpenUSD; the suites only prepended PATH. Outside `ost test`, execVrm_diagnostics and workspace_exec_driver failed on Linux with the plugin never loaded. USDVRM_EXEC_MOTION_ENV (cmake/UsdVrmExecMotion.cmake) now carries the platform loader path, used by every suite that composes the bundle. Local: plain CMake 40/40 on Windows (VS 18, no ost) and 40/40 in an Ubuntu 24.04 container mirroring the workflow; `ost build` + `ost test` 40/40 on 0.23.6; graph valid, 13 library edges; package-consumer lane 3/3. Co-Authored-By: Claude Opus 5.5 --- .github/workflows/package-consumer.yml | 2 +- .github/workflows/plain-cmake.yml | 135 ++++ CHANGELOG.md | 64 ++ CMakeLists.txt | 84 ++- CMakePresets.json | 7 +- CONTRIBUTING.md | 10 +- README.md | 55 +- cmake/UsdVrmConsumedPackage.cmake | 67 ++ cmake/UsdVrmExecMotion.cmake | 39 ++ docs/architecture/WORKSPACE.md | 29 +- docs/reference/SUPPORTED_CONFIGURATIONS.md | 49 +- libs/vrmRig/CMakeLists.txt | 7 +- plugins/execVrm/CMakeLists.txt | 24 +- plugins/execVrm/README.md | 5 +- plugins/execVrm/openstrata.plugin.yaml | 29 +- plugins/execVrm/tests/CMakeLists.txt | 14 +- plugins/execVrm/tests/check_boundaries.py | 10 +- plugins/usdVrmFileFormat/tests/CMakeLists.txt | 19 +- plugins/usdVrmaFileFormat/CMakeLists.txt | 9 +- .../usdVrmaFileFormat/tests/CMakeLists.txt | 3 +- scripts/check_cmake_boundaries.py | 602 ++++++++++++++++++ scripts/plain_cmake_inputs.py | 293 +++++++++ tests/parity/CMakeLists.txt | 10 +- tools/motionRetarget/CMakeLists.txt | 29 +- 24 files changed, 1456 insertions(+), 139 deletions(-) create mode 100644 .github/workflows/plain-cmake.yml create mode 100644 cmake/UsdVrmConsumedPackage.cmake create mode 100644 scripts/check_cmake_boundaries.py create mode 100644 scripts/plain_cmake_inputs.py diff --git a/.github/workflows/package-consumer.yml b/.github/workflows/package-consumer.yml index e36ab292..f9ec7d64 100644 --- a/.github/workflows/package-consumer.yml +++ b/.github/workflows/package-consumer.yml @@ -163,7 +163,7 @@ jobs: - name: Materialize the packages this workspace consumes shell: bash # The libraries this repository no longer builds -- usd-motion-plugins' - # motionCore, motionSampling and motionRecording -- are declared by + # motionCore, motionRetarget, motionSampling and motionUsd -- are declared by # digest in the descriptors and resolved like OpenUSD: as a prefix the # consumer is handed. Pulled here rather than by the driver, because a # lane that reaches the network from inside a consumer fixture would be diff --git a/.github/workflows/plain-cmake.yml b/.github/workflows/plain-cmake.yml new file mode 100644 index 00000000..41d02f2c --- /dev/null +++ b/.github/workflows/plain-cmake.yml @@ -0,0 +1,135 @@ +# Hand-authored (not generated by `ost ci generate`), and it has to be: the one +# thing this lane exists to prove is that the build needs no `ost` at all. +# +# OpenUSD 26.08 install + `cmake --install` of usd-motion-plugins +# -> cmake -> cmake --build -> ctest +# +# is this repository's plain-CMake contract (docs/reference/ +# SUPPORTED_CONFIGURATIONS.md, "Plain CMake"), and every other lane here runs +# after `ost` has materialized the dependencies and activated the runtime. That +# activation can hide a real defect, and did: the published execMotion library +# opens only when the platform loader path names OpenUSD, which `ost test` sets +# and the suites' own environment did not (cmake/UsdVrmExecMotion.cmake). This +# lane found it on its first local run. +# +# WHAT IT COPIES: NOTHING. Every input is read from the file that already pins it +# (scripts/plain_cmake_inputs.py): the OpenUSD install is the Linux workspace +# cell's `runtime_artifact` in openstrata.ci.yaml, fetched by digest over HTTPS +# and extracted -- an ordinary OpenUSD prefix; the usd-motion-plugins commit is +# the one the pinned packages' SLSA provenance names, and it is built from +# source and installed with plain CMake; execMotion is execVrm's pinned bundle. +# When the pins move, this lane moves with them. +# +# One platform on purpose. What is proved is the dependency contract, not +# platform coverage -- the ost lanes carry three platforms. +# +# Fork-PR safety: this workflow never publishes, never promotes, and uses no +# secrets. Every artifact it reads is public. +name: plain cmake + +on: + pull_request: + workflow_dispatch: + +permissions: + contents: read + +env: + # The ost cell whose runtime this lane mirrors. + CELL: workspace-pr-linux + +jobs: + plain-cmake: + name: plain CMake, no ost (linux) + runs-on: ubuntu-24.04 + steps: + - name: Check out the repository + uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0 + + - name: Refuse to run with ost on PATH + shell: bash + run: | + if command -v ost; then + echo "::error title=plain cmake::ost is on PATH; this lane must not be able to call it" + exit 1 + fi + + - name: Check the CMake dependency boundary + # Static, before anything is fetched: a member reaching a package it + # may not, or the root resolving one for it, fails in seconds. + shell: bash + run: | + python3 scripts/check_cmake_boundaries.py --selftest + python3 scripts/check_cmake_boundaries.py + + - name: Fetch the OpenUSD install the CI contract pins + id: openusd + shell: bash + run: python3 scripts/plain_cmake_inputs.py openusd --cell "$CELL" --out "$RUNNER_TEMP/openusd" | tee -a "$GITHUB_OUTPUT" + + - name: Set up the Python the runtime's bindings are built for + id: python + uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6.3.0 + with: + python-version: ${{ steps.openusd.outputs.python }} + + - name: Install the host packages the runtime needs to be consumed + # MaterialX 1.39.5 (OpenUSD 26.08) exports find_dependency(X11) on + # Linux; the list is the cell's `host_packages`. + shell: bash + env: + HOST_PACKAGES_APT: ${{ steps.openusd.outputs.apt }} + run: | + sudo apt-get update + sudo apt-get install -y --no-install-recommends ninja-build $HOST_PACKAGES_APT + + - name: Resolve the usd-motion-plugins commit the pinned packages were built from + id: motion + shell: bash + run: python3 scripts/plain_cmake_inputs.py motion-source --target "${{ steps.openusd.outputs.target }}" | tee -a "$GITHUB_OUTPUT" + + - name: Check out usd-motion-plugins at that commit + uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0 + with: + repository: ${{ steps.motion.outputs.repository }} + ref: ${{ steps.motion.outputs.revision }} + path: _deps/usd-motion-plugins + + - name: Build and install usd-motion-plugins with plain CMake + shell: bash + run: | + set -euo pipefail + cmake -S _deps/usd-motion-plugins -B "$RUNNER_TEMP/build-motion" -G Ninja \ + -DCMAKE_BUILD_TYPE=Release \ + -DCMAKE_PREFIX_PATH="$RUNNER_TEMP/openusd" \ + -DCMAKE_INSTALL_PREFIX="$RUNNER_TEMP/motion" \ + -DUSDMOTION_BUILD_TESTS=OFF \ + -DUSDMOTION_BUILD_EXEC_MOTION=OFF + cmake --build "$RUNNER_TEMP/build-motion" --parallel + cmake --install "$RUNNER_TEMP/build-motion" + + - name: Check the installed motion packages know no VRM + shell: bash + run: python3 scripts/check_cmake_boundaries.py --motion-include "$RUNNER_TEMP/motion/include" + + - name: Fetch the execMotion bundle execVrm pins + id: exec-motion + shell: bash + run: python3 scripts/plain_cmake_inputs.py bundle --id execMotion --target "${{ steps.openusd.outputs.target }}" --out "$RUNNER_TEMP/execMotion" | tee -a "$GITHUB_OUTPUT" + + - name: Configure usd-vrm-plugins + shell: bash + run: | + cmake -S . -B build-plain -G Ninja \ + -DCMAKE_BUILD_TYPE=Release \ + "-DCMAKE_PREFIX_PATH=$RUNNER_TEMP/openusd;$RUNNER_TEMP/motion" \ + -DPython3_EXECUTABLE="${{ steps.python.outputs.python-path }}" \ + -DUSDVRM_EXEC_MOTION_ROOT="${{ steps.exec-motion.outputs.root }}" + + - name: Build + shell: bash + run: cmake --build build-plain --parallel + + - name: Test + shell: bash + run: ctest --test-dir build-plain --output-on-failure --parallel 4 diff --git a/CHANGELOG.md b/CHANGELOG.md index e65b091f..a715f7e7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -13,8 +13,62 @@ Current schema contract version: **1**. ## [Unreleased] +### Added + +- **A plain-CMake lane with no `ost` in it** — + [`.github/workflows/plain-cmake.yml`](.github/workflows/plain-cmake.yml). + On Linux it builds and tests the workspace from an OpenUSD 26.08 install + and a `cmake --install` of `usd-motion-plugins`, and refuses to run if + `ost` is on `PATH`. It copies no pin. + [`scripts/plain_cmake_inputs.py`](scripts/plain_cmake_inputs.py) reads the + OpenUSD archive from the Linux workspace cell in `openstrata.ci.yaml` and + the `execMotion` bundle from `execVrm`'s descriptor, and fetches both by + digest. It builds `usd-motion-plugins` from the commit that the pinned + packages' SLSA provenance names, so the lane moves when the pins move. + Measured locally in an Ubuntu 24.04 container: 40/40, the same suite that + `ost test` runs. +- **A CMake dependency-boundary audit**: + [`scripts/check_cmake_boundaries.py`](scripts/check_cmake_boundaries.py), + registered as `workspace_cmake_boundaries` plus a self-test and run first in + the plain lane. It checks the build graph and needs no build. It fails on: + - a source tree of another repository (`add_subdirectory` out of the repo, + `FetchContent`, `ExternalProject`); + - a `usd-motion-plugins` identity built here; + - the root resolving a consumed package; + - a member reaching a package that WORKSPACE.md §2 does not allow it. The + VRM importer may reach no motion package, and the `.vrma` importer may + reach neither `vrmRig` nor `motionRetarget`; + - a member whose resolved, linked and included packages are not the same + set; + - a descriptor whose `requires.libraries` disagrees with its CMake. + + Given an installed `usd-motion-plugins` `include/`, it also fails on any + code there that knows VRM. Against `main` before this change it reports 17 + violations. + ### Changed +- **The root `CMakeLists.txt` resolves no consumed package.** Each member + resolves what it links through `usdvrm_consume_package()` + ([`cmake/UsdVrmConsumedPackage.cmake`](cmake/UsdVrmConsumedPackage.cmake)). + The root used to `find_package` all five motion packages up front. That + made every configure require every package, including `motionRecording`, + which nothing here includes. Each member now configures and builds on its + own against a prefix holding only its own packages. `usdVrmFileFormat`, + `usdVrmPackageResolver`, `vrmSchema` and `vrmContainer` need no motion + package at all. +- **`execVrm` no longer links `motionSampling` or `motionRecording`**, and its + descriptor no longer pins them. Both were carried from MIG-1, when + `motionRuntime` became two packages, but no source here includes either. + `motionRecording` is no longer consumed by this workspace. +- **`motion_retarget` links `motionCore` by name**, because it includes a + `motionCore/` header. It used to reach that package only through + `motionRetarget`'s link line. +- **`usdvrm_baseline` asks whether it is in the root build** + (`USDVRM_COMPOSED_BUILD`). It used to ask whether `motionCore`'s target was + visible. That check stood in for the real question and would have silently + dropped the gate once the root stopped resolving packages. + - **Every `usd-motion-plugins` pin is v0.5.1**, the release that pushed `execMotion` and the CLIs to its registry as well as the libraries. The five library pins in `vrmRig`, `execVrm`, `usdVrmaFileFormat` and @@ -327,6 +381,16 @@ Current schema contract version: **1**. ### Fixed +- **Suites that load the consumed `execMotion` now set the loader path** + (`USDVRM_EXEC_MOTION_ENV`, + [`cmake/UsdVrmExecMotion.cmake`](cmake/UsdVrmExecMotion.cmake)). The + published library's `RUNPATH` points at the producer's CI checkout. It + therefore opens only when `LD_LIBRARY_PATH` names OpenUSD, which `ost test` + sets by activating the runtime and a plain CTest run did not. On Linux + outside `ost`, `execVrm_diagnostics` and `workspace_exec_driver` failed: + the plugin never loaded, and the computations came back empty. The plain + lane found this on its first run. + - **The release lane could not package the product since MIG-4.** `openstrata.toml`'s `release_exclude` still named `mocopi_record`, `vmc_record` and `vrchat_osc_record`, the three adapter CLIs that left for diff --git a/CMakeLists.txt b/CMakeLists.txt index 4c82b9f9..4d973165 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -3,17 +3,25 @@ # Top-level project for usd-vrm-plugins. # # This repo is dual-mode by design: -# * OpenStrata-native: `ost plugin build|test plugins/usdVrmFileFormat` drives the bundle +# * OpenStrata-native: `ost build` / `ost test` drive this file, and +# `ost plugin build|test plugins/usdVrmFileFormat` drives one bundle's # CMakeLists.txt directly against the activated runtime's toolchain. -# * Plain CMake: anyone without OpenStrata can configure THIS file, point -# CMAKE_PREFIX_PATH at an OpenUSD install, and build every bundle. Use the +# * Plain CMake: anyone without OpenStrata can configure THIS file with an +# OpenUSD 26.08 install and a `cmake --install` of usd-motion-plugins on +# CMAKE_PREFIX_PATH, and build and test every member. Use the # CMakePresets.json here, or: -# cmake -S . -B build -DCMAKE_PREFIX_PATH= +# cmake -S . -B build "-DCMAKE_PREFIX_PATH=;" # cmake --build build --config Release +# ctest --test-dir build -C Release +# .github/workflows/plain-cmake.yml runs exactly that, with no `ost`. # -# Each plugin under plugins/ is a self-contained CMake project (so `ost -# plugin build` can target it standalone) that is also add_subdirectory()-able -# from here. +# This file orchestrates and nothing else: the project and its version, the +# OpenUSD contract, the member list, and the tests that span members. A +# dependency belongs to the member that links it (cmake/UsdVrmConsumedPackage.cmake). +# +# Each member under libs/, plugins/ and tools/ is a self-contained CMake project +# (so `ost plugin build` / `ost library build` can target it standalone) that is +# also add_subdirectory()-able from here. cmake_minimum_required(VERSION 3.22) # Single source of truth for the release version: the repo-root VERSION file. @@ -91,24 +99,25 @@ add_subdirectory("libs/vrmContainer") find_package(pxr REQUIRED CONFIG) include("${CMAKE_CURRENT_SOURCE_DIR}/cmake/UsdVrmOpenUsd.cmake") -# The generic motion libraries are consumed, not built (MIG-1..MIG-3). -# `motionCore`, `motionSampling`, `motionRecording`, `motionRetarget` and -# `motionUsd` are published `usd-motion-plugins` artifacts, pinned by digest per target in each -# descriptor that names one; `ost` materializes them and puts -# their prefixes on CMAKE_PREFIX_PATH before this file is configured, so every -# `find_package` below resolves against a package rather than a sibling -# directory. A plain-CMake build resolves them from CMAKE_PREFIX_PATH the same -# way it resolves OpenUSD. +# The generic motion libraries are consumed, not built (MIG-1..MIG-3), and this +# file resolves none of them. `motionCore`, `motionRetarget`, `motionSampling` +# and `motionUsd` are usd-motion-plugins' installed packages: `ost` +# materializes each digest-pinned artifact and puts its prefix on +# CMAKE_PREFIX_PATH, and a plain-CMake build names a `cmake --install` of that +# repository there, the same way it names OpenUSD. Never a sibling source tree. # -# Resolved once here, after pxr, so the composed build never asks any of their -# configs to define OpenUSD's imported targets a second time. Each member still -# guards its own `find_package` with `if(NOT TARGET ...)`, which is what keeps a -# standalone configure of that member working. -foreach(_consumed IN ITEMS motionCore motionSampling motionRecording motionRetarget motionUsd) - if(NOT TARGET ${_consumed}::${_consumed}) - find_package(${_consumed} REQUIRED CONFIG) - endif() -endforeach() +# Each member resolves what it links, through usdvrm_consume_package() +# (cmake/UsdVrmConsumedPackage.cmake), so this build requires exactly the union +# of what its members use -- and a standalone configure of one member requires +# only that member's own. OpenUSD is resolved above first, so no package config +# is ever the one that defines OpenUSD's targets. +# +# Set for the members' workspace-wide tests, which need bundles a standalone +# member configure does not have. A variable rather than a target probe: a +# probe answers "which package happens to be visible here", which is not the +# question and stopped being the same answer when this file stopped resolving +# packages for its members. +set(USDVRM_COMPOSED_BUILD ON) # What is still built here from this layer: what a VRM rig adds to the # consumed retarget (WORKSPACE.md §2, §9.5). @@ -254,7 +263,10 @@ if(USDVRM_BUILD_TESTS AND USDVRM_EXEC_MOTION_BUNDLE AND TARGET ExecVrm elseif(USDVRM_BUILD_TESTS) message(STATUS "usd-vrm-plugins: no consumed execMotion or no execVrm in this build; " - "skipping the OpenExec parity cases") + "skipping the OpenExec parity cases. A plain-CMake build names an " + "extracted execMotion bundle with -DUSDVRM_EXEC_MOTION_ROOT=; " + "without one, workspace_ctest_labels reports the labels these cases " + "carry as missing") endif() # The live half left with MIG-4. `liveTransport`, `osc`, `motionTracking` and @@ -330,6 +342,28 @@ if(USDVRM_BUILD_TESTS AND USDVRM_TEST_PYTHON) COMMAND "${USDVRM_TEST_PYTHON}" "${CMAKE_CURRENT_SOURCE_DIR}/scripts/check_ctest_labels.py" --selftest) + + # The CMake dependency boundary (WORKSPACE.md §2) as the build states it: + # what each member resolves, links and includes, against what it may; the + # root resolving nothing on a member's behalf; no source tree of another + # repository. And the consumed packages this build actually resolved, + # read for any code that knows VRM -- whichever prefix they came from, the + # artifact `ost` pulled or a plain `cmake --install`. + set(_boundary_includes) + foreach(_consumed IN ITEMS motionCore motionRetarget motionSampling motionUsd) + if(TARGET ${_consumed}::${_consumed}) + list(APPEND _boundary_includes --motion-include + "$") + endif() + endforeach() + add_test(NAME workspace_cmake_boundaries + COMMAND "${USDVRM_TEST_PYTHON}" + "${CMAKE_CURRENT_SOURCE_DIR}/scripts/check_cmake_boundaries.py" + ${_boundary_includes}) + add_test(NAME workspace_cmake_boundaries_selftest + COMMAND "${USDVRM_TEST_PYTHON}" + "${CMAKE_CURRENT_SOURCE_DIR}/scripts/check_cmake_boundaries.py" + --selftest) endif() install(FILES diff --git a/CMakePresets.json b/CMakePresets.json index 19a5264d..7139b4c9 100644 --- a/CMakePresets.json +++ b/CMakePresets.json @@ -18,12 +18,12 @@ { "name": "windows-msvc", "displayName": "Windows / Visual Studio 2022 (plain CMake)", - "description": "Set USD_INSTALL_ROOT to your OpenUSD install (e.g. C:/dev/build/usd2605).", + "description": "Set USD_INSTALL_ROOT to your OpenUSD 26.08 install and USD_MOTION_PREFIX to a cmake --install of usd-motion-plugins.", "inherits": "base", "generator": "Visual Studio 17 2022", "architecture": "x64", "cacheVariables": { - "CMAKE_PREFIX_PATH": "$env{USD_INSTALL_ROOT}" + "CMAKE_PREFIX_PATH": "$env{USD_INSTALL_ROOT};$env{USD_MOTION_PREFIX}" }, "condition": { "type": "equals", @@ -34,10 +34,11 @@ { "name": "linux-ninja", "displayName": "Linux / Ninja (plain CMake)", + "description": "Set USD_INSTALL_ROOT to your OpenUSD 26.08 install and USD_MOTION_PREFIX to a cmake --install of usd-motion-plugins.", "inherits": "base", "generator": "Ninja Multi-Config", "cacheVariables": { - "CMAKE_PREFIX_PATH": "$env{USD_INSTALL_ROOT}" + "CMAKE_PREFIX_PATH": "$env{USD_INSTALL_ROOT};$env{USD_MOTION_PREFIX}" }, "condition": { "type": "equals", diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 0d0cfa34..3c4455e8 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -19,14 +19,20 @@ The main development path uses OpenStrata: ost plugin test --workspace ``` -Plain CMake is also supported when an OpenUSD installation is available: +Plain CMake is also supported, with an OpenUSD 26.08 install and a +`cmake --install` of `usd-motion-plugins` on the prefix path (see +[Plain CMake](docs/reference/SUPPORTED_CONFIGURATIONS.md#plain-cmake)): ```sh -cmake -S . -B build -DCMAKE_PREFIX_PATH=/path/to/openusd-install +cmake -S . -B build "-DCMAKE_PREFIX_PATH=/path/to/openusd-install;/path/to/motion-install" cmake --build build --config Release ctest --test-dir build -C Release ``` +A member resolves the packages it links itself (`usdvrm_consume_package()`), +never the root on its behalf. `python scripts/check_cmake_boundaries.py` +checks that in a second, with no build. + The supported toolchain is documented in [SUPPORTED_CONFIGURATIONS.md](docs/reference/SUPPORTED_CONFIGURATIONS.md). diff --git a/README.md b/README.md index a082316f..c5ad80cd 100644 --- a/README.md +++ b/README.md @@ -30,8 +30,9 @@ project's central design decision, and it is described below. > how this workspace is built, tested, packaged, and released. The record of > adopting it — every version from pre-0.3 to 0.22.2, including what broke — is > published in [docs/reports/ost/](docs/reports/ost/). The repo is -> **dual-mode**: everything also builds with plain CMake against any OpenUSD -> install, with no `ost` involved. +> **dual-mode**: everything also builds and tests with plain CMake against an +> OpenUSD 26.08 install and installed `usd-motion-plugins` packages, with no +> `ost` involved — and [a CI lane](.github/workflows/plain-cmake.yml) proves it. ## Workspace components @@ -146,8 +147,8 @@ usdVrmPackageResolver ──> vrmContainer usdVrmaFileFormat ──────> vrmContainer, motionCore vrmRig ─────────────────> motionCore -motion_retarget (CLI) ──> motionRetarget, vrmRig, motionSampling + OpenUSD - stage APIs +motion_retarget (CLI) ──> motionRetarget, vrmRig, motionCore, motionSampling, + motionUsd + OpenUSD stage APIs vrmAdapterVmc ──────────> motionCore, motionRuntime, liveTransport, osc vrmAdapterMocopi ───────> motionCore, motionRuntime, liveTransport @@ -161,7 +162,7 @@ motionTracking ─────────> nothing — the same again, and for (planned) execMotion ─────────────> motionCore, motionSampling, motionRecording -execVrm ────────────────> vrmSchema, motionRetarget, vrmRig +execVrm ────────────────> vrmSchema, motionCore, motionRetarget, vrmRig ``` Five rules keep those edges honest: @@ -291,24 +292,52 @@ ost plugin view plugins/usdVrmFileFormat avatar.vrm \ ### With plain CMake (no OpenStrata) -The workspace root composes every bundle: +Two installed prefixes, and nothing else: an OpenUSD 26.08 install, and a +`cmake --install` of [`usd-motion-plugins`](https://github.com/animu-sphere/usd-motion-plugins) +— which is consumed as a package, never as a sibling source tree. ```sh -cmake -S . -B build -DCMAKE_PREFIX_PATH=/path/to/openusd-install +# usd-motion-plugins, once +cmake -S usd-motion-plugins -B build-motion -DCMAKE_BUILD_TYPE=Release \ + -DCMAKE_PREFIX_PATH= -DCMAKE_INSTALL_PREFIX= +cmake --build build-motion --config Release +cmake --install build-motion --config Release + +# this workspace +cmake -S . -B build "-DCMAKE_PREFIX_PATH=;" \ + -DPython3_EXECUTABLE= cmake --build build --config Release ctest --test-dir build -C Release ``` +Two things `ost` supplies that a plain build states itself: + +- **`Python3_EXECUTABLE`**, on any host with more than one Python: CMake + otherwise picks the newest, and the Python-driven suites then fail to import + `pxr`. +- **`-DUSDVRM_EXEC_MOTION_ROOT=`**, an extracted + [`execMotion`](https://github.com/animu-sphere/usd-motion-plugins/tree/main/plugins/execMotion) + bundle, for the suites that compose it (the `execVrm` retarget cases and + the OpenExec parity rows). Without it they are not registered, and + `workspace_ctest_labels` says which labels that leaves empty. + +[`plain-cmake.yml`](.github/workflows/plain-cmake.yml) runs exactly this on +Linux, taking the usd-motion-plugins commit its pinned packages were built from +([`scripts/plain_cmake_inputs.py`](scripts/plain_cmake_inputs.py)). + The motion layer's suites carry CTest labels, so one layer runs on its own: -`motion.core`, `motion.runtime`, `motion.retarget`, `motion.cli`, -`motion.integration`, `motion.openexec` and `motion.real-corpus` -(`ctest --test-dir build -C Release -L motion.openexec`). +`motion.retarget`, `motion.cli`, `motion.integration`, `motion.openexec` and +`motion.real-corpus` (`ctest --test-dir build -C Release -L motion.openexec`). [The OpenExec plan](docs/roadmap/openexec-foundation.md) (P0-2) says what each names. -Each bundle also builds standalone against *installed* sibling packages -(`find_package(vrmSchema CONFIG REQUIRED)`), which is what CI proves; a bundle -never reaches sideways into a sibling's source tree. +Each member also builds standalone against *installed* packages +(`find_package(vrmSchema CONFIG REQUIRED)`), resolving only what it links: the +root resolves no dependency on a member's behalf, so `usdVrmFileFormat` configures +with no motion package on the prefix path at all. A member never reaches +sideways into a sibling's source tree, and +[`scripts/check_cmake_boundaries.py`](scripts/check_cmake_boundaries.py) holds +every member's edges to [WORKSPACE.md §2](docs/architecture/WORKSPACE.md). The built `libUsdVrmFileFormat.{dll,so,dylib}` lands in `plugins/usdVrmFileFormat/lib/`; add diff --git a/cmake/UsdVrmConsumedPackage.cmake b/cmake/UsdVrmConsumedPackage.cmake new file mode 100644 index 00000000..abf77b89 --- /dev/null +++ b/cmake/UsdVrmConsumedPackage.cmake @@ -0,0 +1,67 @@ +# SPDX-License-Identifier: Apache-2.0 +# +# UsdVrmConsumedPackage.cmake — how a member resolves a package this repository +# consumes rather than builds. +# +# `motionCore`, `motionSampling`, `motionRecording`, `motionRetarget` and +# `motionUsd` are usd-motion-plugins' installed packages. This repository never +# reaches them as a source tree -- no add_subdirectory(), no FetchContent -- only +# as a prefix on CMAKE_PREFIX_PATH: `ost` materializes the digest-pinned artifact +# and puts it there, and a plain-CMake build names a `cmake --install` of that +# repository the same way it names an OpenUSD install +# (docs/reference/SUPPORTED_CONFIGURATIONS.md, "Plain CMake"). +# +# The rule is that **the member that links a package resolves it**. The root +# project does not resolve any of them on a member's behalf: a root that finds +# the whole motion stack up front makes every member's configure require every +# package, which is how `motionRecording` stayed REQUIRED for a workspace in +# which nothing included one of its headers. +# +# usdvrm_consume_package( [...]) +# +# For each whose `::` target is not visible yet, runs +# `find_package( REQUIRED CONFIG)` and promotes every imported target +# that call created to IMPORTED_GLOBAL. So in the composed root build the first +# member to link a package resolves it and every later member -- and a +# workspace-wide test registered from the root -- sees that one definition, +# while a standalone configure of any single member resolves exactly what that +# member names and nothing else. +# +# A function, not a macro, on purpose: a package's config runs +# find_dependency(pxr) when OpenUSD is not resolved yet, and pxrConfig.cmake +# re-finds Python3 with only the Development components, which leaves +# `Python3_Interpreter_FOUND` FALSE in whatever scope it ran in (see the root +# CMakeLists.txt, USDVRM_TEST_PYTHON). Imported targets are directory-scoped, so +# they survive the function; the variables it would clobber do not escape it. +# +# scripts/check_cmake_boundaries.py reads the calls to this function as the +# member's package edges and holds them to docs/architecture/WORKSPACE.md §2. +include_guard(GLOBAL) + +function(usdvrm_consume_package) + foreach(_usdvrm_pkg IN LISTS ARGN) + if(TARGET ${_usdvrm_pkg}::${_usdvrm_pkg}) + continue() + endif() + + get_directory_property(_usdvrm_before IMPORTED_TARGETS) + find_package(${_usdvrm_pkg} REQUIRED CONFIG) + if(NOT TARGET ${_usdvrm_pkg}::${_usdvrm_pkg}) + message(FATAL_ERROR + "${_usdvrm_pkg}Config.cmake (${${_usdvrm_pkg}_DIR}) was found but " + "defines no ${_usdvrm_pkg}::${_usdvrm_pkg} target.") + endif() + + get_directory_property(_usdvrm_created IMPORTED_TARGETS) + if(_usdvrm_before) + list(REMOVE_ITEM _usdvrm_created ${_usdvrm_before}) + endif() + foreach(_usdvrm_target IN LISTS _usdvrm_created) + get_target_property(_usdvrm_global ${_usdvrm_target} IMPORTED_GLOBAL) + if(NOT _usdvrm_global) + set_target_properties(${_usdvrm_target} PROPERTIES + IMPORTED_GLOBAL TRUE) + endif() + endforeach() + endforeach() +endfunction() diff --git a/cmake/UsdVrmExecMotion.cmake b/cmake/UsdVrmExecMotion.cmake index 2759e009..c58cf45b 100644 --- a/cmake/UsdVrmExecMotion.cmake +++ b/cmake/UsdVrmExecMotion.cmake @@ -17,6 +17,20 @@ # Sets: # USDVRM_EXEC_MOTION_BUNDLE - the bundle root, or empty # USDVRM_EXEC_MOTION_RESOURCES - its plugInfo.json directory, or empty +# USDVRM_EXEC_MOTION_ENV - the ENVIRONMENT_MODIFICATION entries a test +# needs to load it, or empty +# +# **The loader path is part of loading it.** The published library carries no +# path to OpenUSD that holds on this host: its RUNPATH is the producer's CI +# checkout, and its activation contract (openstrata.activation.json) names the +# platform loader variable instead. So OpenUSD's libraries that this process +# has not loaded by the time the plugin is opened -- usdSkel, exec, the Python +# bindings -- are found only through that variable. `ost test` sets it by +# activating the runtime, which is why this was invisible there; a plain CTest +# run has only what the test says. Without it the plugin fails to open, and +# what a suite sees is a computation that returns nothing (measured on Linux: +# execVrm_diagnostics and workspace_exec_driver). On Windows PATH is the +# loader variable, as it always was here. # # Empty means no suite that composes the bundle is registered, and every caller # says so with a status message rather than registering a test that cannot @@ -53,3 +67,28 @@ if(_usdvrm_exec_motion_root) "plugin/resources/execMotion/plugInfo.json") endif() endif() + +set(USDVRM_EXEC_MOTION_ENV "") +if(USDVRM_EXEC_MOTION_BUNDLE) + set(_usdvrm_exec_motion_usd_lib "") + if(pxr_DIR AND EXISTS "${pxr_DIR}/bin") + set(_usdvrm_exec_motion_usd_lib "${pxr_DIR}/lib") + elseif(pxr_DIR) + get_filename_component(_usdvrm_exec_motion_usd_lib "${pxr_DIR}/../../../lib" ABSOLUTE) + endif() + if(WIN32) + set(_usdvrm_loader PATH) + elseif(APPLE) + set(_usdvrm_loader DYLD_LIBRARY_PATH) + else() + set(_usdvrm_loader LD_LIBRARY_PATH) + endif() + list(APPEND USDVRM_EXEC_MOTION_ENV + "${_usdvrm_loader}=path_list_prepend:${USDVRM_EXEC_MOTION_BUNDLE}/lib") + if(_usdvrm_exec_motion_usd_lib) + list(APPEND USDVRM_EXEC_MOTION_ENV + "${_usdvrm_loader}=path_list_prepend:${_usdvrm_exec_motion_usd_lib}") + endif() + list(APPEND USDVRM_EXEC_MOTION_ENV + "PXR_PLUGINPATH_NAME=path_list_prepend:${USDVRM_EXEC_MOTION_RESOURCES}") +endif() diff --git a/docs/architecture/WORKSPACE.md b/docs/architecture/WORKSPACE.md index b5d18e62..5f2522fd 100644 --- a/docs/architecture/WORKSPACE.md +++ b/docs/architecture/WORKSPACE.md @@ -291,8 +291,7 @@ motion_retarget -> motionRetarget, vrmRig, motionSampling, motionUsd, execMotion -> motionCore, motionRuntime execMotion =: UsdSkelAnimation (the OpenExec schema it declares) execVrm -> vrmSchema -execVrm -> motionCore, motionSampling, motionRecording, - motionRetarget, vrmRig +execVrm -> motionCore, motionRetarget, vrmRig execVrm -> execMotion (runtime only: `vrm.computeBoundPose` reads `motion.sampleAnimation` by name; nothing is linked, and the reverse edge is not allowed) @@ -621,6 +620,18 @@ no sibling adapter, no `vrmRetarget`, and no plugin bundle — which is what covers the sibling rule and the prohibitions above the line in any case, since nothing declares an edge it is forbidden to have. +**The consumed-package edges above are also checked as the build states +them**, by `scripts/check_cmake_boundaries.py` (`workspace_cmake_boundaries`), +statically and on every lane. The graph gate reads what a descriptor declares; +this reads what each member's CMake resolves and links and what its sources +include. It fails when those three disagree with each other, with the +descriptor, or with the allowed set here. `execVrm` carried a link to +`motionSampling` and `motionRecording` through a release after the last +include of either, and the graph gate could not see that: the descriptor +declared those same two packages as well. The `ALLOWED` table in that script +restates this section for consumed packages. A new member gets a row there +by hand, or the check refuses to run. + ## 3. Schema contract versioning - `vrmSchema` carries two independent versions: `plugin.version` (semantic @@ -658,6 +669,20 @@ must also build standalone against installed packages (`find_package(vrmSchema CONFIG REQUIRED)` etc.); sibling `add_subdirectory(../otherBundle)` from inside a bundle is forbidden. +**The root resolves no dependency on a member's behalf.** A package this +workspace consumes from another repository — every `usd-motion-plugins` +library — is resolved by the member that links it, through +`usdvrm_consume_package()` (`cmake/UsdVrmConsumedPackage.cmake`), which also +makes the resolved targets global so a later member and a root-registered test +reuse the one definition. Until 2026-09-24 the root `find_package`'d all five +motion packages up front, which made every configure require every package — +`motionRecording` included, which nothing here includes. That package is +reached only as an installed prefix on `CMAKE_PREFIX_PATH`: never +`add_subdirectory` of its source, never `FetchContent`. `ost` puts the +digest-pinned artifact there; a plain-CMake build puts a `cmake --install` +there, and `.github/workflows/plain-cmake.yml` proves that path with no `ost` +at all. + ## 5. Artifact naming and versioning > **What a consumer writes to use one of these is diff --git a/docs/reference/SUPPORTED_CONFIGURATIONS.md b/docs/reference/SUPPORTED_CONFIGURATIONS.md index d6271e29..b518c2a9 100644 --- a/docs/reference/SUPPORTED_CONFIGURATIONS.md +++ b/docs/reference/SUPPORTED_CONFIGURATIONS.md @@ -109,6 +109,51 @@ three. typed schema sources; the generated C++ and `generatedSchema.usda` are committed as the plain-CMake fallback, so a normal build does not run it. +## Plain CMake + +OpenStrata is how this workspace is built, packaged and released, but it is +not needed for a source build. A source build needs two installed prefixes: + +| Input | What it is | How `ost` supplies it | How a plain build supplies it | +| --- | --- | --- | --- | +| OpenUSD 26.08 | an OpenUSD install prefix (`pxrConfig.cmake` at its root), with OpenExec | the pinned runtime artifact, materialized and activated | any OpenUSD 26.08 install; the pinned runtime's archive *is* one | +| `usd-motion-plugins` ≥ 0.5, < 0.6 | the `motionCore`, `motionRetarget`, `motionSampling` and `motionUsd` CMake packages | each descriptor's digest-pinned artifact | `cmake --install` of that repository | + +```sh +cmake -S . -B build "-DCMAKE_PREFIX_PATH=;" +cmake --build build --config Release +ctest --test-dir build -C Release +``` + +That this passes is the definition of plain-CMake support. +[`plain-cmake.yml`](../../.github/workflows/plain-cmake.yml) checks it on every +PR on `ubuntu-24.04`, with no `ost` on the runner. The lane copies no pin. Its +OpenUSD is the Linux workspace cell's pinned runtime archive, fetched by +digest. Its `usd-motion-plugins` is built from the commit that the pinned +packages' provenance names. The lane runs on one platform because what it +proves is the dependency contract; the `ost` lanes provide the platform +coverage. + +`usd-motion-plugins` is consumed only as an installed package. Nothing in this +repository adds its source tree, and `scripts/check_cmake_boundaries.py` fails +if something does. Each member resolves only what it links. A standalone +configure of `usdVrmFileFormat`, `usdVrmPackageResolver`, `vrmSchema` or +`vrmContainer` needs no motion package on the prefix path. + +A plain build must state three things itself, because `ost` supplies them +through its toolchain or its session: + +- **`-DPython3_EXECUTABLE=`** the Python that OpenUSD's bindings are built + for, when the host has more than one. Otherwise CMake picks the newest, and + the Python-driven suites fail to import `pxr`. +- **`-DUSDVRM_EXEC_MOTION_ROOT=`** an extracted `execMotion` bundle, for the + suites that compose it. Without it, those suites are not registered, and + `workspace_ctest_labels` reports the labels they carry as missing. +- **The loader path**, for running a bundle outside CTest. The published + `execMotion` library finds OpenUSD only through `LD_LIBRARY_PATH` / + `DYLD_LIBRARY_PATH` / `PATH`. The suites set that path themselves + (`USDVRM_EXEC_MOTION_ENV`); a session you compose yourself has to set it. + ## Platforms & architectures (CI-verified) These match the per-PR CI matrix in `.github/workflows/ost-source-ci.yml` @@ -150,8 +195,8 @@ The file-format plugins are **shared** libraries (`libUsdVrmFileFormat.{dll,so,dylib}` and `libUsdVrmaFileFormat.{dll,so,dylib}`) — USD loads them dynamically. There is no supported static-plugin build. `vrmRig`, and the `usd-motion-plugins` -packages this product consumes (`motionCore`, `motionSampling`, -`motionRecording`, `motionRetarget`), are intentionally static and are linked +packages this product consumes (`motionCore`, `motionRetarget`, +`motionSampling`, `motionUsd`), are intentionally static and are linked into their consumers; `motion_retarget` and the BVH CLIs are ordinary executables and register nothing with OpenUSD. diff --git a/libs/vrmRig/CMakeLists.txt b/libs/vrmRig/CMakeLists.txt index 4cc1e2d2..3127811e 100644 --- a/libs/vrmRig/CMakeLists.txt +++ b/libs/vrmRig/CMakeLists.txt @@ -40,9 +40,10 @@ if(NOT pxr_FOUND) endif() include("${CMAKE_CURRENT_SOURCE_DIR}/../../cmake/UsdVrmOpenUsd.cmake") -if(NOT TARGET motionCore::motionCore) - find_package(motionCore REQUIRED CONFIG) -endif() +# The joint vocabulary a humanoid binds, and nothing else of the motion stack: +# none of what is here samples, records or retargets. +include("${CMAKE_CURRENT_SOURCE_DIR}/../../cmake/UsdVrmConsumedPackage.cmake") +usdvrm_consume_package(motionCore) add_library(vrmRig STATIC src/ExpressionResolver.cpp diff --git a/plugins/execVrm/CMakeLists.txt b/plugins/execVrm/CMakeLists.txt index dddaf4ff..f8c0d8f7 100644 --- a/plugins/execVrm/CMakeLists.txt +++ b/plugins/execVrm/CMakeLists.txt @@ -60,20 +60,14 @@ include("${CMAKE_CURRENT_SOURCE_DIR}/../../cmake/UsdVrmOpenUsd.cmake") # every node here is a thin wrapper over a `motionRetarget` call, and the # values that cross the boundary are its own (WORKSPACE.md §2, motion policy # §11.4). `vrmRig` is what a VRM rig adds -- here, the required bones. -if(NOT TARGET motionCore::motionCore) - find_package(motionCore CONFIG REQUIRED) -endif() -if(NOT TARGET motionRetarget::motionRetarget) - find_package(motionRetarget CONFIG REQUIRED) -endif() -if(NOT TARGET motionSampling::motionSampling) - find_package(motionSampling CONFIG REQUIRED) -endif() -# The other half of what was one `motionRuntime`: the live-capture intake and -# the capture trace are `motionRecording`'s. -if(NOT TARGET motionRecording::motionRecording) - find_package(motionRecording CONFIG REQUIRED) -endif() +# +# Not `motionSampling` and not `motionRecording`. Both were linked from MIG-1, +# when one `motionRuntime` became the two and this bundle took both halves, but +# no source here includes either: the pose a retarget reads is sampled by +# execMotion's `motion.sampleAnimation`, read by name across the stage, and +# nothing here reads or writes a capture trace. +include("${CMAKE_CURRENT_SOURCE_DIR}/../../cmake/UsdVrmConsumedPackage.cmake") +usdvrm_consume_package(motionCore motionRetarget) if(NOT TARGET vrmRig::vrmRig) find_package(vrmRig CONFIG REQUIRED) endif() @@ -98,8 +92,6 @@ target_include_directories(${PLUGIN_NAME} PRIVATE target_link_libraries(${PLUGIN_NAME} PRIVATE motionCore::motionCore motionRetarget::motionRetarget - motionSampling::motionSampling - motionRecording::motionRecording vrmRig::vrmRig) if(MSVC) diff --git a/plugins/execVrm/README.md b/plugins/execVrm/README.md index 523dbddb..cbceb67c 100644 --- a/plugins/execVrm/README.md +++ b/plugins/execVrm/README.md @@ -408,8 +408,9 @@ pose nothing, and the retarget refuses after it. ([its README](https://github.com/animu-sphere/usd-motion-plugins/blob/main/plugins/execMotion/README.md#how-the-rules-are-checked)). It uses that check's snapshot-rule tables, imported along the one edge the two bundles may have. What it adds is this bundle's own: the links it may have are `motionCore`, -`motionSampling`, `motionRecording`, `motionRetarget` and `vrmRig`, and nothing -of `vrmSchema`. No GLB parser or +`motionRetarget` and `vrmRig`, and nothing of `vrmSchema` — nor +`motionSampling` or `motionRecording`, which were linked from MIG-1 without a +source here including either. No GLB parser or importer may be named in the source. The binary imports neither `vrmSchema` nor `execMotion`. No schema may be declared that `execMotion`'s `plugInfo.json` declares. diff --git a/plugins/execVrm/openstrata.plugin.yaml b/plugins/execVrm/openstrata.plugin.yaml index dc01b5ca..4e59a7f0 100644 --- a/plugins/execVrm/openstrata.plugin.yaml +++ b/plugins/execVrm/openstrata.plugin.yaml @@ -69,32 +69,9 @@ requires: cy2026-windows-x86_64-py313-usd: digest: sha256:18203f389c0b045056bcfe5b462c570c799fcff112e1fa6a9c1da6879835c225 source: oci://ghcr.io/animu-sphere/usd-motion-plugins@sha256:21651895936f6985dfb6a27452f035295a66b922d74b7934da2104f3dbb2ba89 - - id: motionSampling - version: ">=0.5,<0.6" - artifact: - targets: - cy2026-linux-x86_64-py313-usd: - digest: sha256:4889f3f0aec786d64f49dcc58fc82631f8d97bb6d6d8ec6722ad9fd236608886 - source: oci://ghcr.io/animu-sphere/usd-motion-plugins@sha256:5631945c39d2b7c39d98813ca7523d5094e2032bbd0b04e6a1f038036d9fbca6 - cy2026-macos-arm64-py313-usd: - digest: sha256:3b3aa30b6cfeff2a8a7f993779a860e26b3e43856d3c8ca5f6c62030ce589f35 - source: oci://ghcr.io/animu-sphere/usd-motion-plugins@sha256:63759d3ffda0982c9ec0131dca267fd3d9030542771c5df324219548c1c8f7d9 - cy2026-windows-x86_64-py313-usd: - digest: sha256:bc7d75601e541110d4ca9d3d9a6683e1b34ddce21514fa6907e32315d6c86e45 - source: oci://ghcr.io/animu-sphere/usd-motion-plugins@sha256:a02b1a5b962e0f93d929d4588d70d19a91504ac112de9f078c8907b5efe5a9d8 - - id: motionRecording - version: ">=0.5,<0.6" - artifact: - targets: - cy2026-linux-x86_64-py313-usd: - digest: sha256:d3447732b63df0d8e247dc8316755c68ea9fa583c93b34d370b0e57d42503104 - source: oci://ghcr.io/animu-sphere/usd-motion-plugins@sha256:6337da6c33650e61872154d780677ccfc6aaf36da4c6786547a0f4a73e88fffc - cy2026-macos-arm64-py313-usd: - digest: sha256:0e7758821dd31c3cf04dc84f642163718912d658ee488e31d9b3ffaaf5f29e7d - source: oci://ghcr.io/animu-sphere/usd-motion-plugins@sha256:272c3f40271f8b00a3a53ff2bb5937f8522b45c23c464c23cd9a311a39971f2e - cy2026-windows-x86_64-py313-usd: - digest: sha256:1daf034651f86b6d1e5e2bfdf038437649fe0a6caacba6ddcc7970b8495bd6b3 - source: oci://ghcr.io/animu-sphere/usd-motion-plugins@sha256:d84cd5b53af7faa9e611c6442fe2951f3c989cb67f9669afd357a1f6c30c15cd + # Not motionSampling or motionRecording, which were pinned here from MIG-1 + # until no source was found to include either (the CMake edge went with + # them): the pose a retarget reads is sampled by execMotion, by name. # The typed Vrm*API schemas. Not linked: exec resolves `UsdVrmHumanoidAPI` # by TfType name when it reads this bundle's Exec block, so what this bundle # needs is vrmSchema's plugInfo registered in the session. Without it the diff --git a/plugins/execVrm/tests/CMakeLists.txt b/plugins/execVrm/tests/CMakeLists.txt index 85af25fa..6f807785 100644 --- a/plugins/execVrm/tests/CMakeLists.txt +++ b/plugins/execVrm/tests/CMakeLists.txt @@ -150,9 +150,10 @@ if(USDVRM_EXEC_MOTION_BUNDLE) COMMAND execVrm_diagnostics_tests "${_exec_vrm_retarget_fixture}") else() message(STATUS - "execVrm: no consumed execMotion in this build (a standalone configure); " - "execVrm_retarget, _joint_transforms, _diagnostics and _boundaries are " - "built and not registered -- the root build registers them") + "execVrm: no consumed execMotion in this build; execVrm_retarget, " + "_joint_transforms, _diagnostics and _boundaries are built and not " + "registered. `ost build` materializes the bundle; a plain-CMake build " + "names an extracted one with -DUSDVRM_EXEC_MOTION_ROOT=") endif() # The same executable, in a session with vrmSchema and no execMotion: what @@ -217,9 +218,8 @@ set(_plug_resources "${_bundle_dir}/plugin/resources/execVrm") set(_vrm_schema_bundle "${_bundle_dir}/../vrmSchema") set(_vrm_schema_resources "${_vrm_schema_bundle}/plugin/resources/vrmSchema") -# And execMotion, which the retarget reads a computation of. -set(_exec_motion_bundle "${USDVRM_EXEC_MOTION_BUNDLE}") -set(_exec_motion_resources "${USDVRM_EXEC_MOTION_RESOURCES}") +# And execMotion, which the retarget reads a computation of, through +# USDVRM_EXEC_MOTION_ENV (cmake/UsdVrmExecMotion.cmake). set(_exec_vrm_path_env "PATH=path_list_prepend:${_usd_root}/bin" @@ -234,7 +234,7 @@ if(USDVRM_EXEC_MOTION_BUNDLE) set_tests_properties(execVrm_retarget execVrm_joint_transforms execVrm_diagnostics PROPERTIES ENVIRONMENT_MODIFICATION - "${_exec_vrm_path_env};PATH=path_list_prepend:${_vrm_schema_bundle}/lib;PATH=path_list_prepend:${_exec_motion_bundle}/lib;PXR_PLUGINPATH_NAME=path_list_prepend:${_plug_resources};PXR_PLUGINPATH_NAME=path_list_prepend:${_vrm_schema_resources};PXR_PLUGINPATH_NAME=path_list_prepend:${_exec_motion_resources}") + "${_exec_vrm_path_env};PATH=path_list_prepend:${_vrm_schema_bundle}/lib;PXR_PLUGINPATH_NAME=path_list_prepend:${_plug_resources};PXR_PLUGINPATH_NAME=path_list_prepend:${_vrm_schema_resources};${USDVRM_EXEC_MOTION_ENV}") endif() # `set`, not a prepend, for the reason execVrm_humanoid_without_schema gives: diff --git a/plugins/execVrm/tests/check_boundaries.py b/plugins/execVrm/tests/check_boundaries.py index 3d2d0ce6..f329681c 100644 --- a/plugins/execVrm/tests/check_boundaries.py +++ b/plugins/execVrm/tests/check_boundaries.py @@ -9,11 +9,12 @@ What is execVrm's own: - links motionCore, motionSampling, motionRecording, motionRetarget and - vrmRig, and nothing of vrmSchema: + links motionCore, motionRetarget and vrmRig, and nothing of vrmSchema: the schema is a bundle edge exec resolves by type name, and linking it would add a runtime dependency on a library the bundle needs - nothing from + nothing from. Nor motionSampling or motionRecording: the pose is + sampled by execMotion's computation, read by name, and no capture + trace is read or written here source no GLB parser, no importer model, no reparse of the source .vrm or .vrma bytes -- `execVrm`'s only input contract is what is on the stage binary neither vrmSchema nor vrmContainer nor any importer, and not @@ -51,8 +52,7 @@ def main() -> int: errors += rules.purity_import_errors(library) errors += rules.link_errors( "execVrm", sys.argv[3], - {"motionCore::motionCore", "motionSampling::motionSampling", - "motionRecording::motionRecording", "motionRetarget::motionRetarget", + {"motionCore::motionCore", "motionRetarget::motionRetarget", "vrmRig::vrmRig"}) declared, schema_errors = rules.schema_errors("execVrm", source) diff --git a/plugins/usdVrmFileFormat/tests/CMakeLists.txt b/plugins/usdVrmFileFormat/tests/CMakeLists.txt index 5e0b976e..10fc841f 100644 --- a/plugins/usdVrmFileFormat/tests/CMakeLists.txt +++ b/plugins/usdVrmFileFormat/tests/CMakeLists.txt @@ -162,11 +162,14 @@ get_filename_component(_repo_root "${_bundle_dir}/../.." ABSOLUTE) # of this bundle, and its absence makes the generator abort rather than silently # freeze a short union. # -# So it is registered only from the root workspace build. A standalone bundle -# configure (`ost plugin build plugins/usdVrmFileFormat`) has neither that -# bundle nor motionCore in scope, and a test that cannot pass there is worse -# than no test. -if(TARGET motionCore::motionCore) +# So it is registered only from the root workspace build, which says so with +# USDVRM_COMPOSED_BUILD. A standalone bundle configure (`ost plugin build +# plugins/usdVrmFileFormat`) does not have that bundle, and a test that cannot +# pass there is worse than no test. This used to ask whether motionCore's target +# was visible, which was only ever a stand-in for the question and stopped +# answering it when the root stopped resolving packages for its members; and +# the importer tested here links no motion package at all. +if(USDVRM_COMPOSED_BUILD) set(_vrma_bundle "${_bundle_dir}/../usdVrmaFileFormat") set(_vrma_resources "${_vrma_bundle}/plugin/resources/usdVrmaFileFormat") # execMotion joins for the same reason usdVrmaFileFormat did: the generator @@ -176,9 +179,9 @@ if(TARGET motionCore::motionCore) # changes what the gate can verify, not what discovery.json contains. # It is the consumed bundle now (cmake/UsdVrmExecMotion.cmake), which the # root build this gate runs in always has. + # USDVRM_EXEC_MOTION_ENV carries its library, its plugInfo.json and the + # loader path the published library needs to open. include("${_bundle_dir}/../../cmake/UsdVrmExecMotion.cmake") - set(_exec_motion_bundle "${USDVRM_EXEC_MOTION_BUNDLE}") - set(_exec_motion_resources "${USDVRM_EXEC_MOTION_RESOURCES}") # execVrm joins for execMotion's reason, and like it registers no USD type. # Its Exec block names `UsdVrmHumanoidAPI`, a type vrmSchema declares -- # which this session already carries, so the name resolves here as it @@ -186,7 +189,7 @@ if(TARGET motionCore::motionCore) set(_exec_vrm_bundle "${_bundle_dir}/../execVrm") set(_exec_vrm_resources "${_exec_vrm_bundle}/plugin/resources/execVrm") set(_baseline_env - "${_smoke_env};PATH=path_list_prepend:${_vrma_bundle}/lib;PATH=path_list_prepend:${_exec_motion_bundle}/lib;PATH=path_list_prepend:${_exec_vrm_bundle}/lib;PATH=path_list_prepend:$;PXR_PLUGINPATH_NAME=path_list_prepend:${_vrma_resources};PXR_PLUGINPATH_NAME=path_list_prepend:${_exec_motion_resources};PXR_PLUGINPATH_NAME=path_list_prepend:${_exec_vrm_resources}") + "${_smoke_env};PATH=path_list_prepend:${_vrma_bundle}/lib;PATH=path_list_prepend:${_exec_vrm_bundle}/lib;PXR_PLUGINPATH_NAME=path_list_prepend:${_vrma_resources};PXR_PLUGINPATH_NAME=path_list_prepend:${_exec_vrm_resources};${USDVRM_EXEC_MOTION_ENV}") add_test( NAME usdvrm_baseline diff --git a/plugins/usdVrmaFileFormat/CMakeLists.txt b/plugins/usdVrmaFileFormat/CMakeLists.txt index 56ba4c6d..bb99be74 100644 --- a/plugins/usdVrmaFileFormat/CMakeLists.txt +++ b/plugins/usdVrmaFileFormat/CMakeLists.txt @@ -50,9 +50,12 @@ include("${CMAKE_CURRENT_SOURCE_DIR}/cmake/Dependencies.cmake") if(NOT TARGET vrmContainer::vrmContainer) find_package(vrmContainer CONFIG REQUIRED) endif() -if(NOT TARGET motionCore::motionCore) - find_package(motionCore CONFIG REQUIRED) -endif() +# The joint vocabulary the canonical document is stated in. Only that: a .vrma +# is read without knowing which avatar it will drive, so nothing that maps, +# retargets or knows a VRM rig -- `vrmRig`, `motionRetarget` -- is linked here +# (scripts/check_cmake_boundaries.py holds it to that). +include("${CMAKE_CURRENT_SOURCE_DIR}/../../cmake/UsdVrmConsumedPackage.cmake") +usdvrm_consume_package(motionCore) set(PLUGIN_NAME UsdVrmaFileFormat) add_library(${PLUGIN_NAME} SHARED diff --git a/plugins/usdVrmaFileFormat/tests/CMakeLists.txt b/plugins/usdVrmaFileFormat/tests/CMakeLists.txt index 4ef54ad5..16ae2928 100644 --- a/plugins/usdVrmaFileFormat/tests/CMakeLists.txt +++ b/plugins/usdVrmaFileFormat/tests/CMakeLists.txt @@ -16,7 +16,6 @@ endif() set(_bundle_dir "${CMAKE_CURRENT_SOURCE_DIR}/..") set(_lib_dir "${_bundle_dir}/lib") set(_vrm_container_dir "$") -set(_motion_core_dir "$") set(USDVRMA_TEST_LIBRARYPATH "${_lib_dir}/lib${PLUGIN_NAME}${CMAKE_SHARED_LIBRARY_SUFFIX}") set(_plug_resources "${CMAKE_CURRENT_BINARY_DIR}/resources/usdVrmaFileFormat") @@ -26,7 +25,7 @@ configure_file( @ONLY) set(_test_env - "PATH=path_list_prepend:${_usd_root}/bin;PATH=path_list_prepend:${_usd_root}/lib;PATH=path_list_prepend:${_lib_dir};PATH=path_list_prepend:${_vrm_container_dir};PATH=path_list_prepend:${_motion_core_dir};PYTHONPATH=path_list_prepend:${_usd_root}/lib/python;PYTHONPATH=path_list_prepend:${_usd_root}/lib/site-packages;PXR_PLUGINPATH_NAME=path_list_prepend:${_plug_resources}") + "PATH=path_list_prepend:${_usd_root}/bin;PATH=path_list_prepend:${_usd_root}/lib;PATH=path_list_prepend:${_lib_dir};PATH=path_list_prepend:${_vrm_container_dir};PYTHONPATH=path_list_prepend:${_usd_root}/lib/python;PYTHONPATH=path_list_prepend:${_usd_root}/lib/site-packages;PXR_PLUGINPATH_NAME=path_list_prepend:${_plug_resources}") add_test( NAME usdvrma_python_smoke diff --git a/scripts/check_cmake_boundaries.py b/scripts/check_cmake_boundaries.py new file mode 100644 index 00000000..e1134aff --- /dev/null +++ b/scripts/check_cmake_boundaries.py @@ -0,0 +1,602 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: Apache-2.0 +"""Hold the CMake build graph to the workspace's dependency boundary. + +`ost graph validate` checks the topology the descriptors declare. This checks +what the build actually does, statically, with no configure and no build, so it +runs on every lane and in a few seconds: + +* **The repository boundary.** usd-motion-plugins is reached as an installed + package and never as a source tree: no `add_subdirectory()` out of this + repository, no `FetchContent` or `ExternalProject`, no `include()` of another + repository's file. No member here builds a library under one of that + repository's identities either -- a second `motionRetarget` in this tree is + the copy MIG-1..MIG-4 deleted, coming back. +* **The root orchestrates.** The root CMakeLists.txt resolves no consumed + package: each member resolves what it links (cmake/UsdVrmConsumedPackage.cmake), + so the root does not make every member's configure require every package. +* **Each member's edges are allowed.** `ALLOWED` below is WORKSPACE.md §2 for + consumed packages: the VRM importer links no motion package at all, the + `.vrma` importer does not know a target rig (no `vrmRig`, no + `motionRetarget`), `vrmRig` builds on `motionCore` alone. +* **Each member's edges are real.** What a member resolves, what it links and + what its sources include are one set. A package resolved and not linked is a + configure-time requirement for nothing; linked and not included is the edge + `execVrm` carried on `motionRecording` for a release after its last use; and + included but not linked is borrowed from another member's link line, which + holds in the composed build and fails the standalone one. +* **The descriptor says the same.** A member's `requires.libraries` names + exactly the consumed packages its CMake resolves, so `ost` materializes what + the build uses and nothing else. + +With `--motion-include DIR` (repeatable), it also reads an installed +usd-motion-plugins `include/` tree and fails on any code in it that knows VRM: +a VRM header, a `Vrm*`/`UsdVrm*` type, a `vrmRig`/`vrmSchema` name. The +plain-CMake lane passes the prefix it just installed. + + python scripts/check_cmake_boundaries.py + python scripts/check_cmake_boundaries.py --motion-include /include + python scripts/check_cmake_boundaries.py --selftest + +Exit 0 when every rule holds, 1 on a violation, 2 when the tree is not one this +script can read (a member it has no row for is that, on purpose: a new member +is placed in `ALLOWED` by a person, not by a default). +""" + +from __future__ import annotations + +import argparse +import re +import sys +import tempfile +from pathlib import Path + +REPO_ROOT = Path(__file__).resolve().parent.parent + +# usd-motion-plugins' library identities -- every package that repository +# installs a `Config.cmake` for -- and its bundle. None may be built here. +CONSUMED = frozenset({ + "motionCore", "motionSampling", "motionRecording", "motionRetarget", + "motionUsd", "motionSource", "motionBvh", +}) +FOREIGN_IDENTITIES = CONSUMED | {"execMotion"} + +# The consumed packages each member may link (WORKSPACE.md §2). A member missing +# from this table is an error: its row is a decision, not a default. +ALLOWED: dict[str, frozenset[str]] = { + "libs/vrmContainer": frozenset(), + "libs/vrmRig": frozenset({"motionCore"}), + "plugins/vrmSchema": frozenset(), + "plugins/usdVrmFileFormat": frozenset(), + "plugins/usdVrmPackageResolver": frozenset(), + "plugins/usdVrmaFileFormat": frozenset({"motionCore", "motionUsd"}), + "plugins/execVrm": frozenset({"motionCore", "motionRetarget"}), + "tools/motionRetarget": frozenset( + {"motionCore", "motionRetarget", "motionSampling", "motionUsd"}), + "tests/parity": frozenset({"motionCore", "motionRetarget", "motionSampling"}), +} + +# Workspace libraries a member may not reach: the importers read a file without +# knowing which avatar it will drive, so VRM rig semantics are not theirs. +FORBIDDEN_WORKSPACE: dict[str, frozenset[str]] = { + "plugins/usdVrmFileFormat": frozenset({"vrmRig"}), + "plugins/usdVrmaFileFormat": frozenset({"vrmRig"}), +} + +MEMBER_GLOBS = ("libs/*/CMakeLists.txt", "plugins/*/CMakeLists.txt", + "tools/*/CMakeLists.txt") +EXTRA_MEMBERS = ("tests/parity",) +DESCRIPTOR_GLOBS = ("openstrata.plugin.yaml", "openstrata.library.yaml", + "openstrata.tool.yaml") +SOURCE_SUFFIXES = {".h", ".hh", ".hpp", ".inl", ".c", ".cc", ".cpp", ".cxx"} +# Build products and materialized artifacts live inside member directories; +# none of it is source. +SKIP_PARTS = {"build", ".strata", "__pycache__", "lib", "bin", "dist", + "third_party", "corpus"} + +_CMAKE_BRACKET_COMMENT = re.compile(r"#\[(=*)\[.*?\]\1\]", re.DOTALL) +_CPP_BLOCK_COMMENT = re.compile(r"/\*.*?\*/", re.DOTALL) +_CPP_LINE_COMMENT = re.compile(r"//[^\n]*") + + +def cmake_code(text: str) -> str: + """CMake with comments removed; a `#` inside a quoted argument is kept.""" + text = _CMAKE_BRACKET_COMMENT.sub("", text) + lines = [] + for line in text.splitlines(): + in_quote = False + escaped = False + cut = len(line) + for i, ch in enumerate(line): + if escaped: + escaped = False + elif ch == "\\": + escaped = True + elif ch == '"': + in_quote = not in_quote + elif ch == "#" and not in_quote: + cut = i + break + lines.append(line[:cut]) + return "\n".join(lines) + + +def cpp_code(text: str) -> str: + return _CPP_LINE_COMMENT.sub("", _CPP_BLOCK_COMMENT.sub("", text)) + + +def _files(root: Path, keep) -> list[Path]: + out = [] + for path in root.rglob("*"): + rel = path.relative_to(root).parts + if any(part in SKIP_PARTS for part in rel[:-1]): + continue + if path.is_file() and keep(path): + out.append(path) + return sorted(out) + + +def cmake_files(root: Path) -> list[Path]: + return _files(root, lambda p: p.name == "CMakeLists.txt" or p.suffix == ".cmake") + + +def source_files(root: Path) -> list[Path]: + return _files(root, lambda p: p.suffix.lower() in SOURCE_SUFFIXES) + + +def read(path: Path) -> str: + return path.read_text(encoding="utf-8", errors="replace") + + +_CALL = r"\b{name}\s*\(([^)]*)\)" + + +def call_args(code: str, name: str) -> list[list[str]]: + return [a.split() for a in re.findall(_CALL.format(name=name), code, re.IGNORECASE)] + + +def _foreach_items(code: str) -> dict[str, set[str]]: + """`foreach(var IN ITEMS a b)` / `foreach(var a b)`: what `${var}` can be.""" + items: dict[str, set[str]] = {} + for args in call_args(code, "foreach"): + if len(args) < 2: + continue + var, rest = args[0], args[1:] + if rest[:2] == ["IN", "ITEMS"]: + rest = rest[2:] + elif rest[0] in ("IN", "RANGE"): + continue + items.setdefault(var, set()).update(r.strip('"') for r in rest) + return items + + +def resolved_packages(code: str) -> set[str]: + found = set() + loops = _foreach_items(code) + for args in call_args(code, "usdvrm_consume_package"): + found.update(a.strip('"') for a in args) + for args in call_args(code, "find_package"): + if not args: + continue + name = args[0].strip('"') + var = re.fullmatch(r"\$\{(\w+)\}", name) + if var: + # A package named through a loop variable is every item the loop + # can hand it -- which is how the root once resolved all five. + found.update(loops.get(var.group(1), set())) + else: + found.add(name) + return found + + +def linked_packages(code: str) -> set[str]: + return {m for m in re.findall(r"\b(\w+)::\1\b", code)} + + +def included_packages(text: str) -> set[str]: + return set(re.findall(r"#\s*include\s*[<\"](\w+)/", cpp_code(text))) + + +def descriptor_libraries(path: Path) -> set[str]: + """`requires.libraries[].id` of one descriptor, without a YAML parser.""" + ids: set[str] = set() + section = None + for line in read(path).splitlines(): + stripped = line.split("#", 1)[0].rstrip() + if not stripped.strip(): + continue + key = stripped.strip() + if key.endswith(":") and not key.startswith("-") and " " not in key: + if key in ("libraries:", "bundles:", "tools:", "capabilities:"): + section = key[:-1] + continue + if len(line) - len(line.lstrip()) <= 2: + section = None + if section == "libraries" and key.startswith("- id:"): + ids.add(key.split(":", 1)[1].strip()) + return ids + + +class Report: + def __init__(self) -> None: + self.errors: list[str] = [] + self.setup: list[str] = [] + + def error(self, msg: str) -> None: + self.errors.append(msg) + + +def members(root: Path) -> list[str]: + found = [] + for pattern in MEMBER_GLOBS: + for cml in root.glob(pattern): + found.append(cml.parent.relative_to(root).as_posix()) + for extra in EXTRA_MEMBERS: + if (root / extra / "CMakeLists.txt").exists(): + found.append(extra) + return sorted(found) + + +# This repository's own CMake: the root file and the trees it adds. Anything +# else under the checkout -- a build tree, or the usd-motion-plugins checkout the +# plain-CMake lane builds its packages from -- is not this repository's build. +SOURCE_ROOTS = ("cmake", "libs", "plugins", "tools", "tests") + + +def repository_cmake_files(root: Path) -> list[Path]: + files = [root / "CMakeLists.txt"] if (root / "CMakeLists.txt").exists() else [] + for name in SOURCE_ROOTS: + if (root / name).is_dir(): + files += cmake_files(root / name) + return files + + +def check_repository_boundary(root: Path, report: Report) -> None: + for path in repository_cmake_files(root): + rel = path.relative_to(root).as_posix() + code = cmake_code(read(path)) + for word in ("FetchContent_Declare", "FetchContent_MakeAvailable", + "FetchContent_Populate", "ExternalProject_Add"): + if re.search(rf"\b{word}\s*\(", code, re.IGNORECASE): + report.error( + f"{rel}: {word}() -- a dependency is an installed package " + f"found on CMAKE_PREFIX_PATH, never a fetched source tree") + for args in call_args(code, "add_subdirectory"): + if not args: + continue + target = args[0].strip('"') + if "${" in target and not target.startswith("${CMAKE_CURRENT_SOURCE_DIR}"): + # The root's bundle loop: "plugins/${_bundle}". Resolved below. + target = target.split("${", 1)[0] + target = target.replace("${CMAKE_CURRENT_SOURCE_DIR}", ".") + resolved = (path.parent / target).resolve() + if not resolved.is_relative_to(root.resolve()): + report.error( + f"{rel}: add_subdirectory({args[0]}) leaves the repository; " + f"another repository is consumed as its install, not its source") + for args in call_args(code, "include"): + if args and "usd-motion-plugins" in args[0]: + report.error(f"{rel}: include({args[0]}) reads another repository's file") + for args in call_args(code, "add_library") + call_args(code, "add_executable"): + if args and args[0] in FOREIGN_IDENTITIES: + report.error( + f"{rel}: builds '{args[0]}', an identity usd-motion-plugins " + f"publishes -- one library, one repository") + for kind in ("libs", "plugins", "tools"): + for name in FOREIGN_IDENTITIES: + if (root / kind / name / "CMakeLists.txt").exists() and not ( + kind == "tools" and name == "motionRetarget"): + report.error( + f"{kind}/{name}: a member under usd-motion-plugins' identity '{name}'") + + +def check_root(root: Path, report: Report) -> None: + cml = root / "CMakeLists.txt" + if not cml.exists(): + report.setup.append("no root CMakeLists.txt") + return + resolved = resolved_packages(cmake_code(read(cml))) & CONSUMED + for pkg in sorted(resolved): + report.error( + f"CMakeLists.txt: the root resolves '{pkg}'; the member that links it " + f"resolves it (cmake/UsdVrmConsumedPackage.cmake)") + + +def check_member(root: Path, member: str, report: Report) -> None: + base = root / member + if member not in ALLOWED: + report.setup.append( + f"{member}: no row in check_cmake_boundaries.py's ALLOWED -- a new " + f"member's consumed packages are placed by hand (WORKSPACE.md §2)") + return + allowed = ALLOWED[member] + + code = "\n".join(cmake_code(read(p)) for p in cmake_files(base)) + resolved = resolved_packages(code) & CONSUMED + linked = linked_packages(code) & CONSUMED + sources = [read(p) for p in source_files(base)] + included_all: set[str] = set() + for text in sources: + included_all |= included_packages(text) + included = included_all & CONSUMED + + for pkg in sorted((resolved | linked | included) - allowed): + report.error( + f"{member}: reaches '{pkg}', which WORKSPACE.md section 2 does not " + f"allow it (allowed: {', '.join(sorted(allowed)) or 'no consumed package'})") + for pkg in sorted(resolved - linked): + report.error(f"{member}: resolves '{pkg}' and links nothing from it") + for pkg in sorted(linked - resolved): + report.error( + f"{member}: links '{pkg}' without resolving it -- the composed build " + f"lends it, and a standalone configure of {member} fails") + for pkg in sorted(linked - included): + report.error(f"{member}: links '{pkg}' and no source includes a {pkg}/ header") + for pkg in sorted(included - linked): + report.error( + f"{member}: includes {pkg}/ headers without linking '{pkg}' -- the " + f"edge is borrowed from another target's link line") + + forbidden = FORBIDDEN_WORKSPACE.get(member, frozenset()) + reached = (resolved_packages(code) | linked_packages(code) | included_all) + for name in sorted(forbidden & reached): + report.error( + f"{member}: reaches the workspace library '{name}' -- this member " + f"reads its input without knowing the target rig") + + for descriptor in DESCRIPTOR_GLOBS: + path = base / descriptor + if path.exists(): + declared = descriptor_libraries(path) & CONSUMED + for pkg in sorted(declared - resolved): + report.error( + f"{member}/{descriptor}: pins '{pkg}', which its CMake never " + f"resolves -- `ost` materializes a package nothing uses") + for pkg in sorted(resolved - declared): + report.error( + f"{member}/{descriptor}: its CMake resolves '{pkg}', which " + f"requires.libraries does not declare") + + +_VRM_KNOWLEDGE = re.compile( + r"#\s*include\s*[<\"](?:vrm\w*|usdVrm\w*|execVrm)/|" + r"\bUsdVrm\w*|\bVrm[A-Z]\w*|\bvrm(?:Rig|Schema|Container)\b") + + +def check_motion_include(include_root: Path, report: Report) -> int: + """Read one include/ tree's consumed-package headers; answer how many. + + Only the `/` directories of consumed packages are read. A target's + INTERFACE_INCLUDE_DIRECTORIES, read through `$`, is + transitive, so the root build hands over OpenUSD's and Python's include + directories beside the package's own -- not ours to hold to this rule. + """ + if not include_root.is_dir(): + report.setup.append(f"--motion-include {include_root}: not a directory") + return 0 + headers = [p for p in include_root.rglob("*") + if p.is_file() and p.suffix.lower() in SOURCE_SUFFIXES + and p.relative_to(include_root).parts[0] in CONSUMED] + for path in headers: + for n, line in enumerate(cpp_code(read(path)).splitlines(), 1): + if _VRM_KNOWLEDGE.search(line): + report.error( + f"{path}:{n}: a consumed usd-motion-plugins header knows VRM " + f"({line.strip()})") + return len(headers) + + +def run(root: Path, motion_includes: list[Path]) -> Report: + report = Report() + check_repository_boundary(root, report) + check_root(root, report) + for member in members(root): + check_member(root, member, report) + if motion_includes: + check_motion_includes(motion_includes, report) + return report + + +def check_motion_includes(include_roots: list[Path], report: Report) -> None: + if not sum(check_motion_include(d, report) for d in include_roots): + report.setup.append( + "--motion-include: no header of any consumed package under " + f"{', '.join(str(d) for d in include_roots)}, so nothing was checked") + + +# --------------------------------------------------------------------------- +# Self-test: every rule above, made to fire by a tree built for it. A rule that +# can no longer fail is found here rather than by the next edge it lets in. +# --------------------------------------------------------------------------- + +_CONSUME = 'include("${{CMAKE_CURRENT_SOURCE_DIR}}/../../cmake/UsdVrmConsumedPackage.cmake")\nusdvrm_consume_package({pkgs})\n' + + +def _write(root: Path, rel: str, text: str) -> None: + path = root / rel + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(text, encoding="utf-8") + + +def _good_tree(root: Path) -> None: + _write(root, "CMakeLists.txt", + 'add_subdirectory("libs/vrmRig")\n' + '# find_package(motionRecording REQUIRED CONFIG) in a comment is fine\n') + for member in ALLOWED: + _write(root, f"{member}/CMakeLists.txt", "project(x)\n") + _write(root, "libs/vrmRig/CMakeLists.txt", + _CONSUME.format(pkgs="motionCore") + + "target_link_libraries(vrmRig PUBLIC motionCore::motionCore)\n") + _write(root, "libs/vrmRig/include/vrmRig/A.h", "#include \n") + _write(root, "libs/vrmRig/openstrata.library.yaml", + "requires:\n libraries:\n - id: motionCore\n version: x\n") + _write(root, "plugins/usdVrmaFileFormat/CMakeLists.txt", + _CONSUME.format(pkgs="motionCore") + + "target_link_libraries(P PRIVATE vrmContainer::vrmContainer motionCore::motionCore)\n") + _write(root, "plugins/usdVrmaFileFormat/src/D.h", + "// #include is only a comment\n" + "#include \n") + _write(root, "plugins/usdVrmaFileFormat/openstrata.plugin.yaml", + "requires:\n libraries:\n - id: motionCore\n bundles:\n" + " - id: motionRecording\n") + + +CASES: list[tuple[str, dict[str, str], str]] = [ + ("the good tree passes", {}, ""), + ("root resolves a consumed package", + {"CMakeLists.txt": "find_package(motionRecording REQUIRED CONFIG)\n"}, + "the root resolves 'motionRecording'"), + ("root resolves the stack through a loop", + {"CMakeLists.txt": + "foreach(_c IN ITEMS motionCore motionUsd)\n" + " if(NOT TARGET ${_c}::${_c})\n" + " find_package(${_c} REQUIRED CONFIG)\n" + " endif()\nendforeach()\n"}, + "the root resolves 'motionUsd'"), + ("a fetched source tree", + {"libs/vrmRig/cmake/x.cmake": "FetchContent_Declare(m GIT_REPOSITORY u)\n"}, + "FetchContent_Declare()"), + ("a sibling source tree", + {"CMakeLists.txt": 'add_subdirectory("../usd-motion-plugins" m)\n'}, + "leaves the repository"), + ("a copied identity", + {"libs/motionCore/CMakeLists.txt": "add_library(motionCore STATIC a.cpp)\n"}, + "builds 'motionCore'"), + ("the VRM importer links a motion package", + {"plugins/usdVrmFileFormat/CMakeLists.txt": + _CONSUME.format(pkgs="motionRecording") + + "target_link_libraries(P PRIVATE motionRecording::motionRecording)\n", + "plugins/usdVrmFileFormat/src/a.cpp": "#include \n"}, + "plugins/usdVrmFileFormat: reaches 'motionRecording'"), + ("the .vrma importer knows the target rig", + {"plugins/usdVrmaFileFormat/src/r.cpp": "#include \n"}, + "reaches the workspace library 'vrmRig'"), + ("the .vrma importer reaches the retarget", + {"plugins/usdVrmaFileFormat/src/r.cpp": "#include \n"}, + "plugins/usdVrmaFileFormat: reaches 'motionRetarget'"), + ("a linked package no source includes", + {"plugins/execVrm/CMakeLists.txt": + _CONSUME.format(pkgs="motionCore") + + "target_link_libraries(E PRIVATE motionCore::motionCore)\n"}, + "links 'motionCore' and no source includes"), + ("a resolved package nothing links", + {"libs/vrmRig/CMakeLists.txt": + _CONSUME.format(pkgs="motionCore") + "add_library(vrmRig STATIC a.cpp)\n"}, + "resolves 'motionCore' and links nothing"), + ("a link borrowed from another member", + {"libs/vrmRig/CMakeLists.txt": + "target_link_libraries(vrmRig PUBLIC motionCore::motionCore)\n"}, + "links 'motionCore' without resolving it"), + ("an include borrowed from another link line", + {"libs/vrmRig/CMakeLists.txt": "add_library(vrmRig STATIC a.cpp)\n", + "libs/vrmRig/openstrata.library.yaml": "requires:\n libraries: []\n"}, + "includes motionCore/ headers without linking"), + ("a pin the build never resolves", + {"libs/vrmRig/openstrata.library.yaml": + "requires:\n libraries:\n - id: motionCore\n - id: motionSampling\n"}, + "pins 'motionSampling'"), + ("a resolved package the descriptor omits", + {"libs/vrmRig/openstrata.library.yaml": "requires:\n libraries: []\n"}, + "requires.libraries does not declare"), + ("a dependency checkout beside the source is not ours", + {"_deps/usd-motion-plugins/libs/motionCore/CMakeLists.txt": + "add_library(motionCore STATIC a.cpp)\n"}, + ""), + ("a member with no row", + {"plugins/newBundle/CMakeLists.txt": "project(n)\n"}, + "SETUP plugins/newBundle: no row"), +] + + +def selftest() -> int: + failures = 0 + for name, overrides, expect in CASES: + with tempfile.TemporaryDirectory() as tmp: + root = Path(tmp) + _good_tree(root) + for rel, text in overrides.items(): + _write(root, rel, text) + report = run(root, []) + lines = report.errors + [f"SETUP {s}" for s in report.setup] + if not expect: + ok = not lines + else: + ok = any(expect in line for line in lines) + if not ok: + failures += 1 + wanted = repr(expect) if expect else "no finding" + print(f"FAIL {name}: expected {wanted}, got:") + for line in lines or ["(nothing)"]: + print(f" {line}") + else: + print(f"ok {name}") + + with tempfile.TemporaryDirectory() as tmp: + inc = Path(tmp) + _write(inc, "motionRetarget/R.h", + "// the VRM 1.0 humanoid is where the list comes from\n" + "struct R { int a; };\n") + clean = run_include_only(inc) + _write(inc, "motionRetarget/Bad.h", "#include \n") + dirty = run_include_only(inc) + if clean or not any("knows VRM" in e for e in dirty): + failures += 1 + print(f"FAIL motion headers: clean={clean} dirty={dirty}") + else: + print("ok a consumed header that knows VRM") + + print(f"{len(CASES) + 1 - failures}/{len(CASES) + 1} self-test cases passed") + return 1 if failures else 0 + + +def run_include_only(include_root: Path) -> list[str]: + report = Report() + with tempfile.TemporaryDirectory() as foreign: + # Beside a directory holding no consumed package, as the root build + # hands them over. + check_motion_includes([include_root, Path(foreign)], report) + return report.errors + report.setup + + +def main() -> int: + parser = argparse.ArgumentParser(description=__doc__.split("\n\n")[0]) + parser.add_argument("--root", type=Path, default=REPO_ROOT) + # `extend` with nargs="+": CTest may hand a `;`-list over already split + # into several arguments, or as one; both arrive here as a flat list. + parser.add_argument("--motion-include", action="extend", nargs="+", default=[], + help="an installed usd-motion-plugins include/ directory") + parser.add_argument("--selftest", action="store_true") + args = parser.parse_args() + if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(errors="replace") + if args.selftest: + return selftest() + + # CMake hands a target's include directories over as one `;`-list, the + # same directory as often as not twice. + includes: list[Path] = [] + for value in args.motion_include: + for part in str(value).split(";"): + if part and Path(part) not in includes: + includes.append(Path(part)) + args.motion_include = includes + + report = run(args.root.resolve(), args.motion_include) + for line in report.setup: + print(f"SETUP: {line}") + for line in report.errors: + print(f"error: {line}") + if report.setup: + return 2 + if report.errors: + print(f"{len(report.errors)} CMake dependency boundary violation(s)") + return 1 + checked = len(members(args.root.resolve())) + extra = (f", and the consumed headers under {len(args.motion_include)} include dir(s)" + if args.motion_include else "") + print(f"CMake dependency boundary holds: {checked} members{extra}") + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/scripts/plain_cmake_inputs.py b/scripts/plain_cmake_inputs.py new file mode 100644 index 00000000..a52506f7 --- /dev/null +++ b/scripts/plain_cmake_inputs.py @@ -0,0 +1,293 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: Apache-2.0 +"""The inputs of the plain-CMake lane, obtained without `ost`. + +`.github/workflows/plain-cmake.yml` builds this repository the way someone +without OpenStrata would: an OpenUSD 26.08 install and a `cmake --install` of +usd-motion-plugins on CMAKE_PREFIX_PATH, then `cmake`, `cmake --build`, `ctest` +(docs/reference/SUPPORTED_CONFIGURATIONS.md, "Plain CMake"). What it must not do +is invent its own versions of those inputs -- a hand-authored workflow that +copies pins is how release.yml once built against a runtime every other lane +had moved off (ci_pins.py). So each input is read from the file that already +pins it, and fetched by that digest over plain HTTPS: + + openusd --cell --out DIR + The OpenUSD install of one `openstrata.ci.yaml` cell: its + `runtime_artifact` archive, found in its `runtime_remote` manifest, + checked against that digest and extracted. The archive is an ordinary + OpenUSD install prefix (pxrConfig.cmake at its root). Prints + `target=`, `apt=` (the cell's host packages) and `python=` (the + interpreter the runtime's bindings are built for) for the workflow. + + motion-source --target + Which usd-motion-plugins commit to build. Every `requires.libraries` + artifact pin for is read from the descriptors, each artifact's + SLSA provenance is fetched, its subject is checked against the pinned + digest, and the source revisions must all be one commit. Prints + `repository=` and `revision=`. So the lane builds from source exactly + what the pinned packages were built from, and moves when they move. + + bundle --id --target --out DIR + A consumed bundle (`requires.bundles` with an `artifact:` block -- + `execMotion`), fetched by its pinned digest and extracted, for + USDVRM_EXEC_MOTION_ROOT. + +Output lines are `key=value`, for "$GITHUB_OUTPUT". Registry access is +anonymous: every artifact here is public on GHCR. +""" + +from __future__ import annotations + +import argparse +import hashlib +import json +import re +import shutil +import subprocess +import sys +import urllib.request +from pathlib import Path + +REPO_ROOT = Path(__file__).resolve().parent.parent +CI_MANIFEST = REPO_ROOT / "openstrata.ci.yaml" +DESCRIPTOR_GLOBS = ("libs/*/openstrata.library.yaml", + "plugins/*/openstrata.plugin.yaml", + "tools/*/openstrata.tool.yaml") +ARCHIVE_MEDIA_TYPE = "application/vnd.openstrata.artifact.archive.v1+tar+zstd" + + +def fail(msg: str) -> "NoReturn": + print(f"error: {msg}", file=sys.stderr) + raise SystemExit(1) + + +# --------------------------------------------------------------------------- +# Pins, read without a YAML parser -- like ci_pins.py and check_docs.py, so the +# lane needs nothing installed before it can read what to install. +# --------------------------------------------------------------------------- + +def cell(name: str) -> dict: + text = CI_MANIFEST.read_text(encoding="utf-8") + match = re.search(rf"^ - name: {re.escape(name)}\n((?: .*\n|\s*\n| #.*\n)*)", + text, re.M) + if not match: + fail(f"{CI_MANIFEST.name} has no cell named {name!r}") + body = match.group(1) + + def field(pattern: str) -> str: + found = re.search(pattern, body, re.M) + if not found: + fail(f"cell {name!r} has no {pattern!r}") + return found.group(1) + + apt = re.search(r"^\s+apt:\s*\[([^\]]*)\]", body, re.M) + python = re.search(r"^\s+host_python:\s*\"([^\"]+)\"", body, re.M) + return { + "python": python.group(1) if python else "", + "runtime_artifact": field(r"^\s+runtime_artifact:\s*(sha256:[0-9a-f]{64})"), + "uri": field(r"^\s+uri:\s*(oci://\S+)"), + "apt": [p.strip() for p in apt.group(1).split(",")] if apt else [], + } + + +def artifact_pins(section: str, target: str) -> dict[str, dict]: + """`requires.
[]` entries carrying an artifact pin for .""" + pins: dict[str, dict] = {} + for pattern in DESCRIPTOR_GLOBS: + for path in sorted(REPO_ROOT.glob(pattern)): + current_section = current_id = current_target = None + for raw in path.read_text(encoding="utf-8").splitlines(): + line = raw.split("#", 1)[0].rstrip() + if not line.strip(): + continue + indent = len(line) - len(line.lstrip()) + key = line.strip() + if indent <= 2 and key.endswith(":") and not key.startswith("-"): + current_section = key[:-1] + current_id = current_target = None + elif key.startswith("- id:"): + current_id = key.split(":", 1)[1].strip() + current_target = None + elif key.endswith(":"): + # `artifact:`, `targets:`, or a target key -- only this + # one's `digest:`/`source:` lines are ours. + current_target = target if key[:-1] == target else None + elif current_target and current_section == section and current_id: + field, _, value = key.partition(":") + if field in ("digest", "source"): + entry = pins.setdefault(current_id, {}) + value = value.strip() + if entry.get(field) not in (None, value): + fail(f"{current_id} is pinned to two {field}s for " + f"{target}: {entry[field]} and {value} ({path})") + entry[field] = value + return pins + + +# --------------------------------------------------------------------------- +# Anonymous OCI pull +# --------------------------------------------------------------------------- + +class Registry: + def __init__(self, uri: str) -> None: + match = re.fullmatch(r"oci://([^/]+)/([^@]+)@(sha256:[0-9a-f]{64})", uri) + if not match: + fail(f"not a digest-pinned OCI reference: {uri}") + self.host, self.repository, self.manifest_digest = match.groups() + token_url = (f"https://{self.host}/token?" + f"scope=repository:{self.repository}:pull") + with urllib.request.urlopen(token_url, timeout=60) as response: + self.token = json.load(response)["token"] + + def _get(self, path: str, accept: str | None = None): + request = urllib.request.Request( + f"https://{self.host}/v2/{self.repository}/{path}", + headers={"Authorization": f"Bearer {self.token}", + **({"Accept": accept} if accept else {})}) + return urllib.request.urlopen(request, timeout=600) + + def manifest(self) -> dict: + with self._get(f"manifests/{self.manifest_digest}", + "application/vnd.oci.image.manifest.v1+json") as response: + body = response.read() + actual = "sha256:" + hashlib.sha256(body).hexdigest() + if actual != self.manifest_digest: + fail(f"manifest {self.manifest_digest} hashes to {actual}") + return json.loads(body) + + def blob_bytes(self, digest: str) -> bytes: + with self._get(f"blobs/{digest}") as response: + body = response.read() + verify(digest, hashlib.sha256(body).hexdigest()) + return body + + def blob_to(self, digest: str, dest: Path) -> None: + sha = hashlib.sha256() + with self._get(f"blobs/{digest}") as response, dest.open("wb") as out: + while chunk := response.read(1 << 20): + sha.update(chunk) + out.write(chunk) + verify(digest, sha.hexdigest()) + + +def verify(expected: str, actual_hex: str) -> None: + if expected != f"sha256:{actual_hex}": + fail(f"blob {expected} hashes to sha256:{actual_hex}") + + +def layer(manifest: dict, digest: str, uri: str) -> dict: + for entry in manifest.get("layers", []): + if entry.get("digest") == digest: + return entry + fail(f"{uri} carries no layer {digest}") + + +def layer_titled(manifest: dict, suffix: str, uri: str) -> dict: + for entry in manifest.get("layers", []): + title = entry.get("annotations", {}).get("org.opencontainers.image.title", "") + if title.endswith(suffix): + return entry + fail(f"{uri} carries no *{suffix} layer") + + +def fetch_archive(uri: str, digest: str, out: Path) -> str: + """Download the archive layer `digest` of `uri`, verify, extract into out.""" + registry = Registry(uri) + entry = layer(registry.manifest(), digest, uri) + if entry.get("mediaType") != ARCHIVE_MEDIA_TYPE: + fail(f"{uri} layer {digest} is {entry.get('mediaType')}, not an archive") + title = entry.get("annotations", {}).get("org.opencontainers.image.title", "") + out.mkdir(parents=True, exist_ok=True) + archive = out.parent / f".{out.name}.tar.zst" + registry.blob_to(digest, archive) + tar = shutil.which("tar") + if not tar: + fail("no tar on PATH") + subprocess.run([tar, "--zstd", "-xf", str(archive), "-C", str(out)], + check=True) + archive.unlink() + return title + + +# --------------------------------------------------------------------------- +# Subcommands +# --------------------------------------------------------------------------- + +def cmd_openusd(args) -> int: + found = cell(args.cell) + title = fetch_archive(found["uri"], found["runtime_artifact"], args.out) + if not (args.out / "pxrConfig.cmake").exists(): + fail(f"{title} extracted into {args.out} with no pxrConfig.cmake at its root") + target = re.fullmatch(r"openstrata-(.+)\.tar\.zst", title) + if not target: + fail(f"cannot read a target out of the archive name {title!r}") + print(f"target={target.group(1)}") + print(f"apt={' '.join(found['apt'])}") + print(f"python={found['python']}") + return 0 + + +def cmd_motion_source(args) -> int: + pins = artifact_pins("libraries", args.target) + if not pins: + fail(f"no requires.libraries artifact pin names {args.target}") + sources: dict[tuple[str, str], list[str]] = {} + for package, pin in sorted(pins.items()): + if "digest" not in pin or "source" not in pin: + fail(f"{package}'s {args.target} pin lacks a digest or a source") + registry = Registry(pin["source"]) + manifest = registry.manifest() + layer(manifest, pin["digest"], pin["source"]) + statement = json.loads(registry.blob_bytes( + layer_titled(manifest, "provenance.intoto.jsonl", pin["source"])["digest"])) + subjects = {"sha256:" + s["digest"]["sha256"] for s in statement["subject"]} + if pin["digest"] not in subjects: + fail(f"{package}'s provenance does not name the pinned {pin['digest']}") + source = (statement["predicate"]["buildDefinition"] + ["externalParameters"]["source"]) + sources.setdefault((source["repository"], source["revision"]), []).append(package) + if len(sources) != 1: + described = "; ".join(f"{', '.join(p)} from {r}@{c}" + for (r, c), p in sorted(sources.items())) + fail(f"the pinned packages were not built from one commit: {described}") + (repository, revision), packages = next(iter(sources.items())) + print(f"repository={repository}") + print(f"revision={revision}") + print(f"packages={' '.join(packages)}") + return 0 + + +def cmd_bundle(args) -> int: + pins = artifact_pins("bundles", args.target) + pin = pins.get(args.id) + if not pin or "digest" not in pin or "source" not in pin: + fail(f"no requires.bundles artifact pin for {args.id} on {args.target}") + fetch_archive(pin["source"], pin["digest"], args.out) + plug_info = args.out / "plugin" / "resources" / args.id / "plugInfo.json" + if not plug_info.exists(): + fail(f"{args.id}'s archive has no {plug_info.relative_to(args.out)}") + print(f"root={args.out.resolve().as_posix()}") + return 0 + + +def main() -> int: + parser = argparse.ArgumentParser(description=__doc__.split("\n\n")[0]) + sub = parser.add_subparsers(dest="command", required=True) + p = sub.add_parser("openusd") + p.add_argument("--cell", required=True) + p.add_argument("--out", type=Path, required=True) + p.set_defaults(func=cmd_openusd) + p = sub.add_parser("motion-source") + p.add_argument("--target", required=True) + p.set_defaults(func=cmd_motion_source) + p = sub.add_parser("bundle") + p.add_argument("--id", required=True) + p.add_argument("--target", required=True) + p.add_argument("--out", type=Path, required=True) + p.set_defaults(func=cmd_bundle) + args = parser.parse_args() + return args.func(args) + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/tests/parity/CMakeLists.txt b/tests/parity/CMakeLists.txt index 6648c8cd..002e71b0 100644 --- a/tests/parity/CMakeLists.txt +++ b/tests/parity/CMakeLists.txt @@ -10,6 +10,11 @@ # all of them. The root CMakeLists.txt adds this directory after every tool, so # each target named below either exists or the guard there skipped it. +# What the two harnesses link, declared here rather than borrowed from the +# members that happened to resolve it first. +include("${CMAKE_CURRENT_SOURCE_DIR}/../../cmake/UsdVrmConsumedPackage.cmake") +usdvrm_consume_package(motionCore motionRetarget motionSampling) + # NDEBUG undefined, as in every suite here. The harness reports through its # exit code rather than `assert()`, but a Release build silently dropping a # check is the one failure a parity run must never have. @@ -54,7 +59,7 @@ if(MSVC) endif() # The driver's own suite: each VRM_OPENEXEC_* code raised by the failure it -# names, against execMotion alone. motionRuntime for the history sampler's +# names, against execMotion alone. motionSampling for the history sampler's # result type, which is one of the keys it declares. add_executable(exec_driver_contract test_exec_driver.cpp) target_link_libraries(exec_driver_contract PRIVATE @@ -86,10 +91,9 @@ set(_plugins "${_repo_root}/plugins") set(_parity_env "PATH=path_list_prepend:${_usd_root}/bin" "PATH=path_list_prepend:${_usd_root}/lib" - "PATH=path_list_prepend:${USDVRM_EXEC_MOTION_BUNDLE}/lib" + ${USDVRM_EXEC_MOTION_ENV} "PATH=path_list_prepend:${_plugins}/execVrm/lib" "PATH=path_list_prepend:${_plugins}/vrmSchema/lib" - "PXR_PLUGINPATH_NAME=path_list_prepend:${USDVRM_EXEC_MOTION_RESOURCES}" "PXR_PLUGINPATH_NAME=path_list_prepend:${_plugins}/execVrm/plugin/resources/execVrm" "PXR_PLUGINPATH_NAME=path_list_prepend:${_plugins}/vrmSchema/plugin/resources/vrmSchema") diff --git a/tools/motionRetarget/CMakeLists.txt b/tools/motionRetarget/CMakeLists.txt index fd99a1b6..2966aef9 100644 --- a/tools/motionRetarget/CMakeLists.txt +++ b/tools/motionRetarget/CMakeLists.txt @@ -42,25 +42,21 @@ if(NOT pxr_FOUND) endif() include("${CMAKE_CURRENT_SOURCE_DIR}/../../cmake/UsdVrmOpenUsd.cmake") include("${CMAKE_CURRENT_SOURCE_DIR}/../../cmake/UsdVrmUtf8CodePage.cmake") -if(NOT TARGET motionRetarget::motionRetarget) - find_package(motionRetarget REQUIRED CONFIG) -endif() +# This tool is where the generic retarget and the VRM semantics compose: the +# algorithm is `motionRetarget`'s and knows no VRM, `vrmRig` is what a VRM rig +# adds to it, and neither library is linked to the other. +# +# `motionSampling` by name rather than through a PUBLIC re-export: the tool +# resamples the clip itself, before either half of the bake expands it, so it +# has a real edge on it and declaring it leaves nothing to be inferred from a +# transitive dependency that could stop being transitive. The clip is read +# through `motionUsd`, the library home of the clip -> pose rule `execMotion` +# reads by as well; the `vrm:` tracks it refuses are read here, in StageIo.cpp. +include("${CMAKE_CURRENT_SOURCE_DIR}/../../cmake/UsdVrmConsumedPackage.cmake") +usdvrm_consume_package(motionCore motionRetarget motionSampling motionUsd) if(NOT TARGET vrmRig::vrmRig) find_package(vrmRig REQUIRED CONFIG) endif() -# By name rather than through a PUBLIC re-export: the tool resamples the clip -# itself, before either half of the bake expands it, so it has a real edge on -# `motionSampling` and declaring it leaves nothing to be inferred from a -# transitive dependency that could stop being transitive. -if(NOT TARGET motionSampling::motionSampling) - find_package(motionSampling REQUIRED CONFIG) -endif() -# The clip is read through `motionUsd`, the library home of the clip -> pose -# rule `execMotion` reads by as well; the `vrm:` tracks it refuses are read -# here, in StageIo.cpp. -if(NOT TARGET motionUsd::motionUsd) - find_package(motionUsd REQUIRED CONFIG) -endif() # `--build-info`'s stamp. Like usdVrmFileFormat's buildInfo.json it carries no # timestamp, so an unchanged build packages to the same digest. @@ -99,6 +95,7 @@ target_include_directories(motion_retarget PRIVATE "${CMAKE_CURRENT_BINARY_DIR}/ target_compile_features(motion_retarget PRIVATE cxx_std_17) target_link_libraries(motion_retarget PRIVATE + motionCore::motionCore motionRetarget::motionRetarget vrmRig::vrmRig motionSampling::motionSampling From ff5c6e748f4c38bd2cdeaad89e1c9b88f36d3905 Mon Sep 17 00:00:00 2001 From: snkmcb <88454117+snkmcb@users.noreply.github.com> Date: Thu, 24 Sep 2026 11:55:05 +0900 Subject: [PATCH 2/3] Hand the plain lane's Python to the motion build as well pxrConfig.cmake find_dependency's Python3 Development, and on the hosted runner FindPython3 looked for the headers and libpython under /usr instead of setup-python's interpreter, failing the usd-motion-plugins configure. The local container had one Python 3.13 and never saw it. Co-Authored-By: Claude Opus 5.5 --- .github/workflows/plain-cmake.yml | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/.github/workflows/plain-cmake.yml b/.github/workflows/plain-cmake.yml index 41d02f2c..9dd635fe 100644 --- a/.github/workflows/plain-cmake.yml +++ b/.github/workflows/plain-cmake.yml @@ -97,11 +97,16 @@ jobs: - name: Build and install usd-motion-plugins with plain CMake shell: bash + # Python3_EXECUTABLE here too: pxrConfig.cmake find_dependency's + # Python3 Development, and on a runner with a system Python beside + # setup-python's, FindPython3 otherwise looks for the headers and + # libpython under /usr and fails the configure. run: | set -euo pipefail cmake -S _deps/usd-motion-plugins -B "$RUNNER_TEMP/build-motion" -G Ninja \ -DCMAKE_BUILD_TYPE=Release \ -DCMAKE_PREFIX_PATH="$RUNNER_TEMP/openusd" \ + -DPython3_EXECUTABLE="${{ steps.python.outputs.python-path }}" \ -DCMAKE_INSTALL_PREFIX="$RUNNER_TEMP/motion" \ -DUSDMOTION_BUILD_TESTS=OFF \ -DUSDMOTION_BUILD_EXEC_MOTION=OFF From 2ae40e1dcf049e026fbe0892bc2f6e5171ab44b5 Mon Sep 17 00:00:00 2001 From: snkmcb <88454117+snkmcb@users.noreply.github.com> Date: Thu, 24 Sep 2026 12:07:24 +0900 Subject: [PATCH 3/3] Give the plain lane the Python its OpenUSD was built against The second hosted run failed where the first did, with Python3_EXECUTABLE passed: an OpenUSD install's CMake package names its build Python by absolute path. pxrConfig.cmake defaults Python3_LIBRARY/_INCLUDE_DIR to it, and pxrTargets.cmake bakes /usr/include/python3.13 into gf's, usd's and others' INTERFACE_INCLUDE_DIRECTORIES, which no Python3_* hint overrides -- the configure then fails at generate time. `ost` rewrites these paths when it materializes the runtime; the local container passed only because the builder image has that Python at those paths. So the lane installs deadsnakes' python3.13-dev on the runner instead of setup-python -- the Python the pinned runtime was built against, where it expects it, as someone who built OpenUSD on their own host has it. Measured in a clean ubuntu:24.04 container: 40/40. SUPPORTED_CONFIGURATIONS and the README state the constraint. Co-Authored-By: Claude Opus 5.5 --- .github/workflows/plain-cmake.yml | 35 ++++++++++++++-------- CHANGELOG.md | 9 ++++-- README.md | 10 +++++-- docs/reference/SUPPORTED_CONFIGURATIONS.md | 27 +++++++++++++---- 4 files changed, 59 insertions(+), 22 deletions(-) diff --git a/.github/workflows/plain-cmake.yml b/.github/workflows/plain-cmake.yml index 9dd635fe..90384bb1 100644 --- a/.github/workflows/plain-cmake.yml +++ b/.github/workflows/plain-cmake.yml @@ -67,21 +67,34 @@ jobs: shell: bash run: python3 scripts/plain_cmake_inputs.py openusd --cell "$CELL" --out "$RUNNER_TEMP/openusd" | tee -a "$GITHUB_OUTPUT" - - name: Set up the Python the runtime's bindings are built for + - name: Install the Python this OpenUSD was built against, and the host packages id: python - uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6.3.0 - with: - python-version: ${{ steps.openusd.outputs.python }} - - - name: Install the host packages the runtime needs to be consumed + # AN OPENUSD INSTALL IS NOT RELOCATABLE ACROSS PYTHONS. Its CMake + # package names the Python it was built against by absolute path: + # pxrConfig.cmake defaults Python3_LIBRARY/_INCLUDE_DIR to it, and + # pxrTargets.cmake bakes its include directory into the + # INTERFACE_INCLUDE_DIRECTORIES of gf, usd and the rest -- here + # /usr/include/python3.13, the Ubuntu 24.04 + deadsnakes Python the + # runtime's producer used. `ost` rewrites those paths when it + # materializes the runtime; this lane does not, so it provides that + # Python where the install expects it, as someone who built OpenUSD on + # their own host has it. (setup-python's interpreter lives elsewhere, + # and the configure then fails at generate time on a non-existent + # include directory even with every Python3_* hint defined.) + # # MaterialX 1.39.5 (OpenUSD 26.08) exports find_dependency(X11) on - # Linux; the list is the cell's `host_packages`. + # Linux; that list is the cell's `host_packages`. shell: bash env: + PYTHON_VERSION: ${{ steps.openusd.outputs.python }} HOST_PACKAGES_APT: ${{ steps.openusd.outputs.apt }} run: | + set -euo pipefail + sudo add-apt-repository -y ppa:deadsnakes/ppa sudo apt-get update - sudo apt-get install -y --no-install-recommends ninja-build $HOST_PACKAGES_APT + sudo apt-get install -y --no-install-recommends ninja-build \ + "python${PYTHON_VERSION}" "python${PYTHON_VERSION}-dev" $HOST_PACKAGES_APT + echo "python-path=/usr/bin/python${PYTHON_VERSION}" | tee -a "$GITHUB_OUTPUT" - name: Resolve the usd-motion-plugins commit the pinned packages were built from id: motion @@ -97,10 +110,8 @@ jobs: - name: Build and install usd-motion-plugins with plain CMake shell: bash - # Python3_EXECUTABLE here too: pxrConfig.cmake find_dependency's - # Python3 Development, and on a runner with a system Python beside - # setup-python's, FindPython3 otherwise looks for the headers and - # libpython under /usr and fails the configure. + # Python3_EXECUTABLE: pxrConfig.cmake would otherwise default it to the + # producer's own interpreter path, which this host does not have. run: | set -euo pipefail cmake -S _deps/usd-motion-plugins -B "$RUNNER_TEMP/build-motion" -G Ninja \ diff --git a/CHANGELOG.md b/CHANGELOG.md index a715f7e7..5d87d8a0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -25,8 +25,13 @@ Current schema contract version: **1**. the `execMotion` bundle from `execVrm`'s descriptor, and fetches both by digest. It builds `usd-motion-plugins` from the commit that the pinned packages' SLSA provenance names, so the lane moves when the pins move. - Measured locally in an Ubuntu 24.04 container: 40/40, the same suite that - `ost test` runs. + Measured locally in a clean `ubuntu:24.04` container: 40/40, the same suite + that `ost test` runs. The lane installs deadsnakes' `python3.13-dev`, not + setup-python, because an OpenUSD install's CMake package names its build + Python by absolute path, in `pxrConfig.cmake` and in `pxrTargets.cmake`'s + include directories. `ost` rewrites those paths when it materializes a + runtime; a plain build has to supply that Python where the install expects + it (docs/reference/SUPPORTED_CONFIGURATIONS.md, "Plain CMake"). - **A CMake dependency-boundary audit**: [`scripts/check_cmake_boundaries.py`](scripts/check_cmake_boundaries.py), registered as `workspace_cmake_boundaries` plus a self-test and run first in diff --git a/README.md b/README.md index c5ad80cd..51bc07b5 100644 --- a/README.md +++ b/README.md @@ -312,9 +312,13 @@ ctest --test-dir build -C Release Two things `ost` supplies that a plain build states itself: -- **`Python3_EXECUTABLE`**, on any host with more than one Python: CMake - otherwise picks the newest, and the Python-driven suites then fail to import - `pxr`. +- **The Python OpenUSD was built against, where it was built against it.** + An OpenUSD install's CMake package names that Python's headers by absolute + path, so pass its interpreter as `Python3_EXECUTABLE`, and have its headers + and `libpython` where the install expects them. For the pinned Linux + runtime, that is deadsnakes' `python3.13-dev` on Ubuntu 24.04. + [Plain CMake](docs/reference/SUPPORTED_CONFIGURATIONS.md#plain-cmake) + explains why no `Python3_*` hint can point it elsewhere. - **`-DUSDVRM_EXEC_MOTION_ROOT=`**, an extracted [`execMotion`](https://github.com/animu-sphere/usd-motion-plugins/tree/main/plugins/execMotion) bundle, for the suites that compose it (the `execVrm` retarget cases and diff --git a/docs/reference/SUPPORTED_CONFIGURATIONS.md b/docs/reference/SUPPORTED_CONFIGURATIONS.md index b518c2a9..23e1cb50 100644 --- a/docs/reference/SUPPORTED_CONFIGURATIONS.md +++ b/docs/reference/SUPPORTED_CONFIGURATIONS.md @@ -116,7 +116,7 @@ not needed for a source build. A source build needs two installed prefixes: | Input | What it is | How `ost` supplies it | How a plain build supplies it | | --- | --- | --- | --- | -| OpenUSD 26.08 | an OpenUSD install prefix (`pxrConfig.cmake` at its root), with OpenExec | the pinned runtime artifact, materialized and activated | any OpenUSD 26.08 install; the pinned runtime's archive *is* one | +| OpenUSD 26.08 | an OpenUSD install prefix (`pxrConfig.cmake` at its root), with OpenExec, **and the Python it was built against** | the pinned runtime artifact, materialized and activated | any OpenUSD 26.08 install on the host it was built for; the pinned runtime's archive is one, for an Ubuntu 24.04 host with deadsnakes' Python 3.13 | | `usd-motion-plugins` ≥ 0.5, < 0.6 | the `motionCore`, `motionRetarget`, `motionSampling` and `motionUsd` CMake packages | each descriptor's digest-pinned artifact | `cmake --install` of that repository | ```sh @@ -140,12 +140,29 @@ if something does. Each member resolves only what it links. A standalone configure of `usdVrmFileFormat`, `usdVrmPackageResolver`, `vrmSchema` or `vrmContainer` needs no motion package on the prefix path. -A plain build must state three things itself, because `ost` supplies them +**An OpenUSD install is not relocatable across Pythons.** Its CMake package +names the Python it was built against by absolute path in two places: + +- `pxrConfig.cmake` defaults `Python3_LIBRARY` and `Python3_INCLUDE_DIR` to + it. +- `pxrTargets.cmake` bakes the include directory into the + `INTERFACE_INCLUDE_DIRECTORIES` of `gf`, `usd` and others. Defining every + `Python3_*` hint does not override this, and the configure then fails at + generate time on a non-existent directory. + +`ost` rewrites those paths when it materializes a runtime; a plain build does +not. So a plain build needs that Python at those paths. For the pinned Linux +archive, that is `python3.13-dev` from deadsnakes on Ubuntu 24.04, which is +what `plain-cmake.yml` installs. Someone who built OpenUSD on their own host +already has it. + +A plain build must also state three things itself, because `ost` supplies them through its toolchain or its session: -- **`-DPython3_EXECUTABLE=`** the Python that OpenUSD's bindings are built - for, when the host has more than one. Otherwise CMake picks the newest, and - the Python-driven suites fail to import `pxr`. +- **`-DPython3_EXECUTABLE=`** that Python's interpreter. Otherwise + `pxrConfig.cmake` defaults to the producer's interpreter path, or CMake + picks the newest Python on a host with several, and the Python-driven + suites fail to import `pxr`. - **`-DUSDVRM_EXEC_MOTION_ROOT=`** an extracted `execMotion` bundle, for the suites that compose it. Without it, those suites are not registered, and `workspace_ctest_labels` reports the labels they carry as missing.