diff --git a/docs/uv-patching.md b/docs/uv-patching.md index 12b3743d8..f47d77260 100644 --- a/docs/uv-patching.md +++ b/docs/uv-patching.md @@ -14,7 +14,9 @@ Additionally, `extra_deps` and `extra_data` allow adding dependencies or data files to the generated `py_library` target for a package. `console_scripts` overrides the complete script map for a wheel built from an sdist when its egg-info metadata is absent or unsuitable. An explicit -empty map suppresses all detected scripts. +empty map suppresses all detected scripts. For a native +extension built from an sdist, `cc_deps` wires Bazel `cc_library` targets +(their headers and static archives) into the build. ## Prerequisites @@ -126,6 +128,54 @@ uv.override_package( ) ``` +### Linking native C/C++ dependencies + +A package with a native extension often needs a C/C++ library: its headers to +compile against and its static archive to link. `cc_deps` wires a Bazel target +that provides `CcInfo` (a `cc_library`, `cc_import`, or similar) directly into +the sdist build: the dependency's transitive headers, include paths, defines, +and Apple framework search paths become compile flags (appended to `CPPFLAGS`), +and its static archives are placed in the linker's post-object slot. The +transitive closure and link order come from `CcInfo`, so you name only the +top-level target. + +`cc_deps` is the declarative counterpart to the `env` / `toolchains` escape hatch +(see [the constraints below](#constraints)): reach for `cc_deps` to _declare_ the +native dependency, and keep `env` for _tweaking_ the build: package-specific +defines, exotic linker flags, or anything `cc_deps` cannot model. The two +compose; `cc_deps` flags are appended after any you set in `env`. + +For example, building a package's native extension against an in-repo C +library. Before, with the raw `env` / `toolchains` escape hatch, the include +and archive paths are anchored by hand with `$(EXECROOT)` and fed through +make-variables a toolchain exports: + +```starlark +uv.override_package( + name = "native-package", + toolchains = ["//third_party/mylib:make_vars"], # exports $(MYLIB_INC), $(MYLIB_LIB_A) + env = { + "CPPFLAGS": "-I$(EXECROOT)/$(MYLIB_INC)", + "LDFLAGS": "$(EXECROOT)/$(MYLIB_LIB_A)", + }, +) +``` + +After, with `cc_deps`, the include path, the archive, and their transitive +closure are read from the target's `CcInfo`, and no path anchoring is needed: + +```starlark +uv.override_package( + name = "native-package", + cc_deps = ["//third_party/mylib"], # a cc_library / cc_import +) +``` + +The dependency target must be visible to the generated build repository, so mark +it `//visibility:public` (or grant that repository's package visibility). See the +[constraints below](#constraints) for the supported-library and setuptools +requirements. + ### Reserving wheel build resources Native sdist builds can be memory-hungry. Without a hint, Bazel assumes the @@ -217,11 +267,11 @@ uv.override_package( removes NumPy's bundled tests without retaining their compiled bytecode. Removing the complete `.dist-info` directory, `METADATA`, or `RECORD` is unsupported. -- `pre_build_patches`, `toolchains`, `env`, `monitor_memory`, and non-default - `resource_set` values require a source distribution. An override that applies - them to a wheel-only lock record is rejected. -- Generated pure-Python builds reject `toolchains` and `env`; those attributes - augment the native build toolchain and environment. +- `pre_build_patches`, `toolchains`, `env`, `cc_deps`, `monitor_memory`, and + non-default `resource_set` values require a source distribution. An override + that applies them to a wheel-only lock record is rejected. +- Generated pure-Python builds reject `toolchains`, `env`, and `cc_deps`; those + attributes augment the native build toolchain, environment, and link inputs. - Native build `env` values can use `$(EXECROOT)/` to anchor paths supplied by a toolchain, for example `CPPFLAGS = "-I$(EXECROOT)/$(DEP_INC)"` and `LDFLAGS = "$(EXECROOT)/$(DEP_LIB_A)"`. The anchor remains valid after the @@ -229,6 +279,66 @@ uv.override_package( - Native builds select the configured C++ compiler, archiver, linker, and strip tools by default. Explicit `CC`, `CXX`, `AR`, `LD`, and `STRIP` values in `env` override those selections. +- `cc_deps` applies only to sdists built by the setuptools backend. A package + that declares any other `[build-system].build-backend` is rejected when the + wheel is built (`cc_deps is only supported with the setuptools build backend`) + rather than having its inputs silently dropped. +- The build environment's setuptools must be `>= 65.4.0`, the release that + added `DIST_EXTRA_CONFIG`, the channel `cc_deps` routes the link inputs + through. An older or missing setuptools fails the build with + `cc_deps requires setuptools >= 65.4.0`; bump it in the lock that supplies your + build dependencies (`uv.lock` / `default_build_dependencies`). +- Only static (or PIC-static) archives are linked. A dependency that provides + only a shared/dynamic library fails at analysis time, as does an `alwayslink` + (whole-archive) library; neither is supported. +- The linked archives must contain position-independent (PIC) objects, because + they are folded into the extension's shared object. A toolchain that emits + non-PIC objects into its static archives (some GCC configurations) fails the + final link with relocation errors such as `relocation R_X86_64_32 against ... +can not be used when making a shared object; recompile with -fPIC`. Remedies: + use a toolchain that compiles PIC objects (the default on macOS, and clang/LLVM + on Linux), build with `--force_pic`, or add `copts = ["-fPIC"]` to the + `cc_library`. +- Each `cc_deps` label is referenced from the generated external build + repository, so the target must be visible to it: use `//visibility:public` or + grant that repository's package visibility. +- Link flags that reference a file the dependency declares via + `additional_linker_inputs` (for example a `-Wl,--version-script,...` linker + script) are path-anchored automatically so they survive the backend changing + directory. A relative path written directly into `linkopts` without declaring + the file there is not anchored and will not resolve after the change. +- `cc_deps` flattens a dependency's link inputs into setuptools' two link slots: + full-path static archives go to the post-object `[build_ext] link_objects` + slot in topological order; bare `-l` entries go to the post-object + `[build_ext] libraries` slot preserving their relative order; and every other + link flag is appended to `LDFLAGS` ahead of the objects. Because the archives + and `-l` entries land in separate slots, the order between an `-l` entry + and a non-`-l` flag cannot be preserved, so only flags whose effect does not + depend on that relative order are passed through. `cc_deps` accepts a fixed set + of link-flag shapes: `-L` search paths; `-pthread`; and `-Wl,` tokens + built from these directives: the rpath family (`-rpath`, `-rpath=`, and + `-rpath-link`), `--version-script` (comma and `=` argument forms), `-z` with + one of the reviewed keywords `relro`, `now`, `noexecstack`, or `origin` (each + a global link mode; the wider `-z` namespace includes position-sensitive + keywords, so others are rejected), and `--enable-new-dtags`. A comma-joined + `-Wl,` token is validated directive by directive, so an accepted leading + directive cannot smuggle a rejected one behind it (`-Wl,-rpath,/x,--as-needed` + fails, naming `--as-needed`), while benign compounds such as + `-Wl,-z,relro,-z,now` and `-Wl,-rpath,$ORIGIN,--enable-new-dtags` pass. Any + other link flag, including grouping and linker-state toggles such as + `--start-group`/`--end-group` or `--as-needed`, is rejected at analysis time + rather than silently reordered. The split `-L ` form is rejected too; + write the glued `-L`. Apple `-framework` linking is not supported in v1: + ld64 resolves frameworks in command-line order alongside `-l` entries, so the + two-slot split cannot hold one; set the framework in the override's `env` + `LDFLAGS` or patch it in with `pre_build_patches`. Three escape hatches cover + what the allowlist does not: to resolve an archive cycle that would otherwise + need `--start-group`, repeat the library name (for example `-la -lb -la`), + since `-l` order is preserved within the libraries slot; to apply a global + toggle such as `--as-needed` to the whole link, set it in the override's + `env` `LDFLAGS`, which lands ahead of the objects; and for anything else, + patch it in with `pre_build_patches` on the sdist. The accepted set can be + extended upstream on request. - Post-install patches to prebuilt wheels must preserve every retained original path used for collision and regular-package merge planning, including its file-or-directory kind and package classification. Ordinary added paths are diff --git a/e2e/cases/BUILD.bazel b/e2e/cases/BUILD.bazel index 314ebc619..c6766676c 100644 --- a/e2e/cases/BUILD.bazel +++ b/e2e/cases/BUILD.bazel @@ -168,6 +168,17 @@ write_source_files( # pep517_native_whl(...) call so the reservation reaches the action. "snapshots/sdist_build.uv_sdist_native_build.python_geohash.BUILD.bazel": "@sdist_build__uv_sdist_native_build__python_geohash__0_8_5//:BUILD.bazel", + # @sdist_build for python-geohash with uv.override_package(cc_deps) + # ---------------------------------------------------------------------- + # Covers the override_package -> repo-rule -> BUILD-template plumbing + # for `cc_deps`. Reuses python-geohash's locked sdist under a distinct + # project/hub (//uv-sdist-cc-deps) so the three sibling sdist_build + # snapshots stay byte-stable. Pins that the CcInfo label threads onto + # the generated pep517_native_whl(...) `cc_deps` list. The executable + # end-to-end regression lives in-repo at + # //uv/private/pep517_whl/tests/cc_deps. + "snapshots/sdist_build.uv_sdist_cc_deps.python_geohash.BUILD.bazel": "@sdist_build__uv_sdist_cc_deps__python_geohash__0_8_5//:BUILD.bazel", + # Venv site-packages .pth files. Pin the line shapes that # assemble_venv emits, so any change to `.pth` emission # surfaces here. Wheel site-packages dirs are referenced by diff --git a/e2e/cases/MODULE.bazel b/e2e/cases/MODULE.bazel index 2d0137ca4..c78acd7c8 100644 --- a/e2e/cases/MODULE.bazel +++ b/e2e/cases/MODULE.bazel @@ -153,6 +153,7 @@ include("//uv-platform-filter-844:setup.MODULE.bazel") include("//uv-plus-version:setup.MODULE.bazel") include("//uv-pyproject-cases:setup.MODULE.bazel") include("//uv-requirements-bzl:setup.MODULE.bazel") +include("//uv-sdist-cc-deps:setup.MODULE.bazel") include("//uv-sdist-fallback:setup.MODULE.bazel") include("//uv-sdist-jdk-build:setup.MODULE.bazel") include("//uv-sdist-mpicc:setup.MODULE.bazel") diff --git a/e2e/cases/snapshots/sdist_build.uv_sdist_cc_deps.python_geohash.BUILD.bazel b/e2e/cases/snapshots/sdist_build.uv_sdist_cc_deps.python_geohash.BUILD.bazel new file mode 100644 index 000000000..c38aa38f9 --- /dev/null +++ b/e2e/cases/snapshots/sdist_build.uv_sdist_cc_deps.python_geohash.BUILD.bazel @@ -0,0 +1,26 @@ + +load("@aspect_rules_py//uv/private/pep517_whl:rule.bzl", "pep517_native_whl") +load("@aspect_rules_py//py:defs.bzl", "py_binary") + +py_binary( + name = "build_tool", + main = "@aspect_rules_py//uv/private/pep517_whl:build_helper.py", + srcs = ["@aspect_rules_py//uv/private/pep517_whl:build_helper.py"], + deps = ["@@aspect_rules_py++uv+project__uv_sdist_cc_deps//:build", "@@aspect_rules_py++uv+project__uv_sdist_cc_deps//:setuptools", "@whl_install__uv_sdist_cc_deps__setuptools__75_8_2//:install"], +) + +pep517_native_whl( + name = "whl", + src = "@@aspect_rules_py++uv+sdist__python_geohash__05a21fcf4eda1a5e//file:file", + tool = ":build_tool", + version = "0.8.5", + cc_deps = [ + "@@//uv-sdist-cc-deps:probe", + ], + visibility = ["//visibility:public"], +) + +exports_files( + ["BUILD.bazel"], + visibility = ["//visibility:public"], +) diff --git a/e2e/cases/uv-sdist-cc-deps/BUILD.bazel b/e2e/cases/uv-sdist-cc-deps/BUILD.bazel new file mode 100644 index 000000000..1ffeb856a --- /dev/null +++ b/e2e/cases/uv-sdist-cc-deps/BUILD.bazel @@ -0,0 +1,25 @@ +# Snapshot regression for `uv.override_package(cc_deps = [...])`. +# +# Reuses python-geohash's locked sdist (already fetched by the +# uv-sdist-native-build case) under a distinct project name/hub so the +# generated sdist_build repo is independent. The override attaches `:probe` +# via `cc_deps`; the snapshot in //:snapshots pins that the label threads +# onto the generated pep517_native_whl(...) `cc_deps` list. +# +# Snapshot-only: the wheel is not built here. The executable cc_deps +# regression (real setuptools extension linking a cc_library) lives in-repo +# at //uv/private/pep517_whl/tests/cc_deps. + +load("@rules_cc//cc:defs.bzl", "cc_library") + +cc_library( + name = "probe", + srcs = ["probe.cc"], + hdrs = ["probe.h"], + includes = ["."], + # The generated @sdist_build__uv_sdist_cc_deps__python_geohash__0_8_5 + # repo references this label from its pep517_native_whl(cc_deps = [...]) + # call, so it must be visible outside this package. Public matches how + # sibling cases expose files to generated repos (exports_files default). + visibility = ["//visibility:public"], +) diff --git a/e2e/cases/uv-sdist-cc-deps/probe.cc b/e2e/cases/uv-sdist-cc-deps/probe.cc new file mode 100644 index 000000000..8b24dd046 --- /dev/null +++ b/e2e/cases/uv-sdist-cc-deps/probe.cc @@ -0,0 +1,3 @@ +#include "probe.h" + +int rules_py_cc_deps_probe(void) { return 42; } diff --git a/e2e/cases/uv-sdist-cc-deps/probe.h b/e2e/cases/uv-sdist-cc-deps/probe.h new file mode 100644 index 000000000..4c35667e2 --- /dev/null +++ b/e2e/cases/uv-sdist-cc-deps/probe.h @@ -0,0 +1,16 @@ +#ifndef UV_SDIST_CC_DEPS_PROBE_H_ +#define UV_SDIST_CC_DEPS_PROBE_H_ + +#ifdef __cplusplus +extern "C" { +#endif + +// Returns a constant probe value. Present only so the `cc_deps` override has a +// real CcInfo target to reference; the snapshot never builds this library. +int rules_py_cc_deps_probe(void); + +#ifdef __cplusplus +} +#endif + +#endif // UV_SDIST_CC_DEPS_PROBE_H_ diff --git a/e2e/cases/uv-sdist-cc-deps/pyproject.toml b/e2e/cases/uv-sdist-cc-deps/pyproject.toml new file mode 100644 index 000000000..de45e6310 --- /dev/null +++ b/e2e/cases/uv-sdist-cc-deps/pyproject.toml @@ -0,0 +1,7 @@ +[project] +name = "uv-sdist-cc-deps" +version = "0.0.0" +requires-python = ">=3.11" +dependencies = [ + "python-geohash", +] diff --git a/e2e/cases/uv-sdist-cc-deps/setup.MODULE.bazel b/e2e/cases/uv-sdist-cc-deps/setup.MODULE.bazel new file mode 100644 index 000000000..8d65e909e --- /dev/null +++ b/e2e/cases/uv-sdist-cc-deps/setup.MODULE.bazel @@ -0,0 +1,37 @@ +"""Regression for the per-package `cc_deps` plumbing on +`uv.override_package`. Reuses python-geohash's locked sdist (already +fetched by the uv-sdist-native-build case) under a distinct project +name/hub, so the generated sdist_build repo is independent and the three +existing sdist_build snapshots stay byte-stable. + +The override attaches an in-repo cc_library via `cc_deps`; the snapshot +pins that the label threads onto the generated pep517_native_whl(...) +`cc_deps` list. Snapshot-only: the wheel is not built here; the +executable cc_deps regression lives in-repo under +//uv/private/pep517_whl/tests/cc_deps. +""" + +uv = use_extension("@aspect_rules_py//uv:extensions.bzl", "uv") +uv.declare_hub(hub_name = "pypi_uv_sdist_cc_deps") +uv.project( + default_build_dependencies = [ + "build", + "setuptools", + ], + hub_name = "pypi_uv_sdist_cc_deps", + lock = "//uv-sdist-cc-deps:uv.lock", + pyproject = "//uv-sdist-cc-deps:pyproject.toml", +) + +# Attach an in-repo CcInfo target via `cc_deps`. This is additive on top of +# whatever sdist_build's BUILD template emits by default. +uv.override_package( + name = "python-geohash", + cc_deps = ["//uv-sdist-cc-deps:probe"], + lock = "//uv-sdist-cc-deps:uv.lock", +) + +# Exposed so e2e/BUILD.bazel can snapshot the generated pep517_native_whl +# call: pins that `cc_deps` threads onto the build rule. The hub repo is +# deliberately not imported; no target here consumes it. +use_repo(uv, "sdist_build__uv_sdist_cc_deps__python_geohash__0_8_5") diff --git a/e2e/cases/uv-sdist-cc-deps/uv.lock b/e2e/cases/uv-sdist-cc-deps/uv.lock new file mode 100644 index 000000000..25e8ef77e --- /dev/null +++ b/e2e/cases/uv-sdist-cc-deps/uv.lock @@ -0,0 +1,70 @@ +version = 1 +revision = 3 +requires-python = ">=3.11" + +[[package]] +name = "build" +version = "1.4.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "colorama", marker = "os_name == 'nt'" }, + { name = "packaging" }, + { name = "pyproject-hooks" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/42/18/94eaffda7b329535d91f00fe605ab1f1e5cd68b2074d03f255c7d250687d/build-1.4.0.tar.gz", hash = "sha256:f1b91b925aa322be454f8330c6fb48b465da993d1e7e7e6fa35027ec49f3c936", size = 50054, upload-time = "2026-01-08T16:41:47.696Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/c5/0d/84a4380f930db0010168e0aa7b7a8fed9ba1835a8fbb1472bc6d0201d529/build-1.4.0-py3-none-any.whl", hash = "sha256:6a07c1b8eb6f2b311b96fcbdbce5dab5fe637ffda0fd83c9cac622e927501596", size = 24141, upload-time = "2026-01-08T16:41:46.453Z" }, +] + +[[package]] +name = "colorama" +version = "0.4.6" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/d8/53/6f443c9a4a8358a93a6792e2acffb9d9d5cb0a5cfd8802644b7b1c9a02e4/colorama-0.4.6.tar.gz", hash = "sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44", size = 27697, upload-time = "2022-10-25T02:36:22.414Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/d1/d6/3965ed04c63042e047cb6a3e6ed1a63a35087b6a609aa3a15ed8ac56c221/colorama-0.4.6-py2.py3-none-any.whl", hash = "sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6", size = 25335, upload-time = "2022-10-25T02:36:20.889Z" }, +] + +[[package]] +name = "packaging" +version = "26.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/65/ee/299d360cdc32edc7d2cf530f3accf79c4fca01e96ffc950d8a52213bd8e4/packaging-26.0.tar.gz", hash = "sha256:00243ae351a257117b6a241061796684b084ed1c516a08c48a3f7e147a9d80b4", size = 143416, upload-time = "2026-01-21T20:50:39.064Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/b7/b9/c538f279a4e237a006a2c98387d081e9eb060d203d8ed34467cc0f0b9b53/packaging-26.0-py3-none-any.whl", hash = "sha256:b36f1fef9334a5588b4166f8bcd26a14e521f2b55e6b9de3aaa80d3ff7a37529", size = 74366, upload-time = "2026-01-21T20:50:37.788Z" }, +] + +[[package]] +name = "pyproject-hooks" +version = "1.2.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/e7/82/28175b2414effca1cdac8dc99f76d660e7a4fb0ceefa4b4ab8f5f6742925/pyproject_hooks-1.2.0.tar.gz", hash = "sha256:1e859bd5c40fae9448642dd871adf459e5e2084186e8d2c2a79a824c970da1f8", size = 19228, upload-time = "2024-09-29T09:24:13.293Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/bd/24/12818598c362d7f300f18e74db45963dbcb85150324092410c8b49405e42/pyproject_hooks-1.2.0-py3-none-any.whl", hash = "sha256:9e5c6bfa8dcc30091c74b0cf803c81fdd29d94f01992a7707bc97babb1141913", size = 10216, upload-time = "2024-09-29T09:24:11.978Z" }, +] + +[[package]] +name = "python-geohash" +version = "0.8.5" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/9c/e2/1a3507af7c8f91f8a4975d651d4aeb6a846dfdf74713954186ade4205850/python-geohash-0.8.5.tar.gz", hash = "sha256:05a21fcf4eda1a5eddbd291890ade23fc5ddaa6bb98f2ee23d2d384ed14f086d", size = 17636, upload-time = "2014-08-13T07:09:57.318Z" } + +[[package]] +name = "setuptools" +version = "75.8.2" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/d1/53/43d99d7687e8cdef5ab5f9ec5eaf2c0423c2b35133a2b7e7bc276fc32b21/setuptools-75.8.2.tar.gz", hash = "sha256:4880473a969e5f23f2a2be3646b2dfd84af9028716d398e46192f84bc36900d2", size = 1344083, upload-time = "2025-02-26T20:45:19.103Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/a9/38/7d7362e031bd6dc121e5081d8cb6aa6f6fedf2b67bf889962134c6da4705/setuptools-75.8.2-py3-none-any.whl", hash = "sha256:558e47c15f1811c1fa7adbd0096669bf76c1d3f433f58324df69f3f5ecac4e8f", size = 1229385, upload-time = "2025-02-26T20:45:17.259Z" }, +] + +[[package]] +name = "uv-sdist-cc-deps" +version = "0.0.0" +source = { virtual = "." } +dependencies = [ + { name = "python-geohash" }, +] + +[package.metadata] +requires-dist = [{ name = "python-geohash" }] diff --git a/uv/private/extension/defs.bzl b/uv/private/extension/defs.bzl index a7140664d..f5ad83b23 100644 --- a/uv/private/extension/defs.bzl +++ b/uv/private/extension/defs.bzl @@ -264,13 +264,14 @@ def _parse_projects(module_ctx, hub_specs): override.extra_data or override.toolchains or override.env or + override.cc_deps or override.monitor_memory or override.resource_set != "default" ) if has_target and has_modifications: fail("uv.override_package() for '{}': `target` is mutually exclusive with modification attributes. Use `target` for full replacement OR build, patch, and data attributes for modifications, not both.".format(override.name)) if not has_target and not has_modifications: - fail("uv.override_package() for '{}': must specify either `target` for full replacement or at least one modification attribute (console_scripts, pre_build_patches, post_install_patches, exclude_glob, extra_deps, extra_data, toolchains, env, monitor_memory, resource_set).".format(override.name)) + fail("uv.override_package() for '{}': must specify either `target` for full replacement or at least one modification attribute (console_scripts, pre_build_patches, post_install_patches, exclude_glob, extra_deps, extra_data, toolchains, env, cc_deps, monitor_memory, resource_set).".format(override.name)) unscoped_matches = {i: 0 for i, override in enumerate(mod.tags.override_package) if override.lock == None} @@ -522,6 +523,7 @@ def _parse_projects(module_ctx, hub_specs): pre_build_patch_strip = pkg_override.pre_build_patch_strip, supported = [], toolchains = pkg_override.toolchains, + cc_deps = pkg_override.cc_deps, ) if sdist: # HACK: Note that we resolve these LAZILY so that @@ -563,16 +565,19 @@ def _parse_projects(module_ctx, hub_specs): pre_build_patches = [str(p) for p in pkg_override.pre_build_patches] pre_build_patch_strip = pkg_override.pre_build_patch_strip - # `toolchains` / `env` on `uv.override_package` augment - # the defaults baked into sdist_build's BUILD template — - # they don't replace them. Empty == no augmentation. + # `toolchains` / `env` / `cc_deps` on `uv.override_package` + # augment the defaults baked into sdist_build's BUILD + # template; they don't replace them. Empty == no + # augmentation. extra_toolchains = [] extra_env = {} + extra_cc_deps = [] monitor_memory = False resource_set = "default" if pkg_override: extra_toolchains = [str(t) for t in pkg_override.toolchains] extra_env = pkg_override.env + extra_cc_deps = [str(d) for d in pkg_override.cc_deps] monitor_memory = pkg_override.monitor_memory resource_set = pkg_override.resource_set @@ -586,6 +591,7 @@ def _parse_projects(module_ctx, hub_specs): available_deps = project_available_deps, extra_toolchains = extra_toolchains, extra_env = extra_env, + extra_cc_deps = extra_cc_deps, monitor_memory = monitor_memory, resource_set = resource_set, ) @@ -816,6 +822,8 @@ def _uv_impl(module_ctx): sbuild_kwargs["extra_toolchains"] = sbuild_cfg.extra_toolchains if sbuild_cfg.extra_env: sbuild_kwargs["extra_env"] = sbuild_cfg.extra_env + if sbuild_cfg.extra_cc_deps: + sbuild_kwargs["extra_cc_deps"] = sbuild_cfg.extra_cc_deps if sbuild_cfg.monitor_memory: sbuild_kwargs["monitor_memory"] = True if sbuild_cfg.resource_set != "default": @@ -952,6 +960,10 @@ _override_package_tag = tag_class( default = {}, doc = "Extra environment variables merged into the build action's `env` dict. Values may reference $(VAR) make-variables sourced from extra `toolchains` listed above. Prefix an execroot-relative path with `$(EXECROOT)/` so it remains valid after the backend changes into the unpacked source tree. Omit CC/CXX/AR/LD/STRIP to use the configured C++ action tools.", ), + "cc_deps": attr.label_list( + default = [], + doc = "CcInfo targets whose transitive headers, defines, include paths, and static archives are wired into the native sdist build (compile via CPPFLAGS, archives into the link post-object). Setuptools-backend sdists only. Each label is referenced from the generated external build repository, so the target must be visible to it (e.g. `//visibility:public`).", + ), "pre_build_patches": attr.label_list( default = [], allow_files = [".patch", ".diff"], diff --git a/uv/private/pep517_whl/BUILD.bazel b/uv/private/pep517_whl/BUILD.bazel index 423988ed3..a50bd0835 100644 --- a/uv/private/pep517_whl/BUILD.bazel +++ b/uv/private/pep517_whl/BUILD.bazel @@ -69,6 +69,21 @@ py_binary( deps = ["@pypi//build"], ) +# Same helper source, but with setuptools in the build venv. The production +# tool above deliberately ships only @pypi//build; the executable cc_deps +# regression under tests/cc_deps/executable needs a real setuptools.build_meta +# backend to run under --no-isolation, so it uses this variant. +py_binary( + name = "__build_helper_setuptools", + testonly = True, + srcs = ["build_helper.py"], + main = "build_helper.py", + deps = [ + "@pypi//build", + "@pypi//setuptools", + ], +) + _PYTHON_ENV_BACKEND_SRCS = glob(["tests/python_env_backend/*"]) mtree_spec( @@ -97,7 +112,6 @@ bzl_library( srcs = ["test.bzl"], visibility = ["//uv:__subpackages__"], deps = [ - ":rule", "//uv/private:source_built_wheel", "@bazel_skylib//lib:unittest", ], diff --git a/uv/private/pep517_whl/build_helper.py b/uv/private/pep517_whl/build_helper.py index cd5725150..026f894d2 100644 --- a/uv/private/pep517_whl/build_helper.py +++ b/uv/private/pep517_whl/build_helper.py @@ -18,7 +18,7 @@ from os import chmod, defpath, listdir, makedirs, path, pathsep from subprocess import CalledProcessError, check_call, check_output, STDOUT, run from tempfile import TemporaryFile -from typing import Dict, Optional +from typing import Dict, List, Optional try: tomllib = importlib.import_module("tomllib") @@ -270,6 +270,311 @@ def _legacy_metadata_conflicts_with_pyproject(worktree: str) -> bool: ) ) + +def _load_cc_deps_info(info_path: str, execroot_marker: Optional[str]) -> Dict[str, List[str]]: + """Load the cc_deps params file, re-anchoring its execroot-relative paths. + + The rule emits every path prefixed with `execroot_marker` because only + execroot-relative paths exist at analysis time. build_helper still runs at + the execroot here (the backend chdir happens in the child process), so + `os.getcwd()` is the execroot the marker must expand to. + """ + import json + + if not execroot_marker: + # Defensive: the rule always pairs the two flags; without the marker the + # paths below cannot be anchored. + print( + "Error: --cc-deps-info requires --execroot-marker to anchor its paths.", + file=sys.stderr, + ) + exit(1) + + with open(info_path, encoding="utf-8") as f: + raw = json.load(f) + + execroot = os.getcwd() + info: Dict[str, List[str]] = {} + for key in ("compile_flags", "link_objects", "link_libraries", "link_flags"): + values: List[str] = [] + for value in raw.get(key, []): + resolved = value.replace(execroot_marker, execroot) + if execroot_marker in resolved: + print( + "Error: execroot marker survived substitution in cc_deps " + "{}: {!r}".format(key, resolved), + file=sys.stderr, + ) + exit(1) + values.append(resolved) + info[key] = values + return info + + +def _effective_build_backend(worktree: str) -> Optional[str]: + """Return the declared PEP 517 backend, or None for the setuptools default. + + A missing pyproject.toml, `[build-system]` table, or `build-backend` key all + mean the legacy setuptools path (None is in `_SETUPTOOLS_BACKENDS`), which + still honors DIST_EXTRA_CONFIG. + """ + pyproject_data = _load_pyproject_data(worktree) + if not pyproject_data: + return None + build_system = pyproject_data.get("build-system", {}) + if not isinstance(build_system, dict): + return None + backend = build_system.get("build-backend") + return backend if isinstance(backend, str) else None + + +def _require_setuptools_floor() -> None: + """Fail unless the build venv's setuptools understands DIST_EXTRA_CONFIG.""" + from importlib.metadata import PackageNotFoundError, version + + # DIST_EXTRA_CONFIG landed in setuptools 65.4.0; older setuptools silently + # ignores the [build_ext] config we hand it, dropping the linked archives. + try: + found = version("setuptools") + except PackageNotFoundError: + print( + "Error: cc_deps requires setuptools >= 65.4.0 in the build " + "environment, but setuptools is not installed. Add setuptools to your " + "uv.lock / default_build_dependencies.", + file=sys.stderr, + ) + exit(1) + + parts = found.split(".") + try: + version_tuple = (int(parts[0]), int(parts[1]) if len(parts) > 1 else 0) + except (IndexError, ValueError): + print( + "Error: cc_deps requires setuptools >= 65.4.0, but could not parse the " + "installed setuptools version {!r}.".format(found), + file=sys.stderr, + ) + exit(1) + + if version_tuple < (65, 4): + print( + "Error: cc_deps requires setuptools >= 65.4.0 (for DIST_EXTRA_CONFIG), " + "but the build environment has {}. Update setuptools in your uv.lock / " + "default_build_dependencies.".format(found), + file=sys.stderr, + ) + exit(1) + + +def _has_whitespace(value: str) -> bool: + return any(character.isspace() for character in value) + + +def _cc_deps_include_path(flag: str) -> Optional[str]: + """Return the directory carried by an -I/-iquote/-isystem/-F flag, else None.""" + for prefix in ("-iquote", "-isystem", "-I", "-F"): + if flag.startswith(prefix): + return flag[len(prefix):] + return None + + +def _reject_unsplittable_cc_deps(info: Dict[str, List[str]]) -> None: + """Reject values setuptools/CPPFLAGS/LDFLAGS would split, naming the offender. + + setuptools splits `[build_ext] link_objects` and `libraries` on whitespace + or comma, and CPPFLAGS/LDFLAGS are word-split downstream, so a path + containing either survives neither. The execroot itself can carry a space + (a user home directory), so this is exactly the case the guard catches. + """ + # os.pathsep does not appear in setuptools' link_objects split rule + # (whitespace/comma only), but the guard keeps it as an over-conservative + # fail-safe: a colon-bearing archive path has no legitimate source here. + for link_object in info["link_objects"]: + if _has_whitespace(link_object) or "," in link_object or pathsep in link_object: + print( + "Error: cc_deps link object path {!r} contains whitespace, a " + "comma, or {!r}; setuptools splits [build_ext] link_objects on " + "whitespace/comma and would mangle it. This usually means the " + "Bazel output base (execroot) contains a space; choose an " + "--output_base without spaces.".format(link_object, pathsep), + file=sys.stderr, + ) + exit(1) + for library in info["link_libraries"]: + if _has_whitespace(library) or "," in library: + print( + "Error: cc_deps library name {!r} contains whitespace or a " + "comma; setuptools splits [build_ext] libraries on those and " + "would mangle it.".format(library), + file=sys.stderr, + ) + exit(1) + # -D defines are deliberately unguarded (binding decision): they carry + # symbols, not paths, and CPPFLAGS is the only channel that can express K=V. + for flag in info["compile_flags"]: + include = _cc_deps_include_path(flag) + if include is not None and _has_whitespace(include): + print( + "Error: cc_deps compile search path {!r} contains whitespace; " + "CPPFLAGS is word-split downstream and would mangle it. This " + "usually means the Bazel output base (execroot) contains a space; " + "choose an --output_base without spaces.".format(include), + file=sys.stderr, + ) + exit(1) + # LDFLAGS is word-split downstream, so a whitespace-bearing token is + # unrepresentable, whether it's an execroot-anchored path + # (e.g. -Wl,--version-script,/.../vs.lds) or a user linkopt that + # happens to carry a space. Guard every link flag, not just the + # execroot-anchored ones. + for flag in info["link_flags"]: + if _has_whitespace(flag): + print( + "Error: cc_deps link flag {!r} contains whitespace; it is " + "appended to LDFLAGS, which is word-split downstream, so no " + "single flag can carry a space. This usually means the Bazel " + "output base (execroot) contains a space; choose an " + "--output_base without spaces.".format(flag), + file=sys.stderr, + ) + exit(1) + + +def _package_build_ext_option(worktree: str, option: str) -> Optional[str]: + """Return a `[build_ext]` option value from the package's setup.cfg, if any. + + DIST_EXTRA_CONFIG REPLACES (never merges) same-named options, so the + package's own value has to be folded back in explicitly. Tolerant of a + missing setup.cfg; warns but continues if an existing one will not parse. + """ + import configparser + + setup_cfg = path.join(worktree, "setup.cfg") + if not path.exists(setup_cfg): + return None + + parser = configparser.ConfigParser(interpolation=None) + try: + parser.read(setup_cfg, encoding="utf-8") + except configparser.Error: + print( + "Warning: ignoring [build_ext] settings in {} (could not parse it).".format(setup_cfg), + file=sys.stderr, + ) + return None + + if parser.has_option("build_ext", option): + return parser.get("build_ext", option) + return None + + +def _merged_build_ext_value(worktree: str, option: str, values: List[str]) -> str: + """Join our `values` for a [build_ext] option behind the package's own value.""" + ours = " ".join(values) + package_value = _package_build_ext_option(worktree, option) + if not package_value: + return ours + # configparser may return a multi-line value; collapse it to one physical + # line so our generated cfg stays well-formed. The option is whitespace/comma + # split downstream, so token semantics are preserved, and the package's + # entries stay worktree-relative (they resolve because the backend's cwd is + # the worktree). + package_value = " ".join(package_value.split()) + return "{} {}".format(package_value, ours) if ours else package_value + + +def _build_extra_config_text(info: Dict[str, List[str]], worktree: str) -> str: + """Render the DIST_EXTRA_CONFIG `[build_ext]` file for the link inputs. + + include_dirs and defines are deliberately omitted: they ride CPPFLAGS + instead (a cfg `define` cannot express K=V macros, and CPPFLAGS reaches + custom build commands too). + """ + lines = ["[build_ext]"] + if info["link_objects"]: + lines.append("link_objects = {}".format( + _merged_build_ext_value(worktree, "link_objects", info["link_objects"]), + )) + if info["link_libraries"]: + lines.append("libraries = {}".format( + _merged_build_ext_value(worktree, "libraries", info["link_libraries"]), + )) + return "\n".join(lines) + "\n" + + +def _append_cc_deps_env(env: Dict[str, str], key: str, additions: List[str]) -> None: + """Append flags to an env var, preserving any user value as the prefix.""" + if not additions: + return + addition = " ".join(additions) + existing = env.get(key) + env[key] = "{} {}".format(existing, addition) if existing else addition + + +def _apply_cc_deps( + env: Dict[str, str], + info_path: str, + execroot_marker: Optional[str], + worktree: str, + tmp_root: str, +) -> None: + """Route cc_deps compile/link inputs into the setuptools build. + + Mutates `env` in place: appends compile flags to CPPFLAGS and exotic link + flags to LDFLAGS, and points DIST_EXTRA_CONFIG at a `[build_ext]` file (in + tmp_root, never the worktree) carrying the static archives and `-l` + libraries. Only runs when `--cc-deps-info` was passed, so the no-cc_deps + path is unchanged. + """ + info = _load_cc_deps_info(info_path, execroot_marker) + + # cc_deps is a setuptools-only feature in v1; refuse other backends loudly + # rather than silently dropping the inputs. + backend = _effective_build_backend(worktree) + if backend not in _SETUPTOOLS_BACKENDS: + print( + "Error: cc_deps is only supported with the setuptools build backend, " + "but this package declares build-backend {!r}. cc_deps injects " + "setuptools [build_ext] settings, which other backends ignore.".format(backend), + file=sys.stderr, + ) + exit(1) + + _require_setuptools_floor() + + # Never merge with a user-provided DIST_EXTRA_CONFIG (v1 owns it). + if "DIST_EXTRA_CONFIG" in env: + print( + "Error: cc_deps needs DIST_EXTRA_CONFIG, but it is already set in the " + "build environment. Remove it from `env` to use cc_deps.", + file=sys.stderr, + ) + exit(1) + + # DIST_EXTRA_CONFIG lives only in setuptools' local distutils. + if env.get("SETUPTOOLS_USE_DISTUTILS") == "stdlib": + print( + "Error: cc_deps requires setuptools' local distutils, but " + "SETUPTOOLS_USE_DISTUTILS=stdlib is set. Unset it (or set it to " + "\"local\") to use cc_deps.", + file=sys.stderr, + ) + exit(1) + env.setdefault("SETUPTOOLS_USE_DISTUTILS", "local") + + _reject_unsplittable_cc_deps(info) + + # The config file lives in tmp_root, never the worktree. + config_path = path.join(tmp_root, "cc_deps_extra.cfg") + with open(config_path, "w", encoding="utf-8") as f: + f.write(_build_extra_config_text(info, worktree)) + env["DIST_EXTRA_CONFIG"] = config_path + + # Compose additively; any user CPPFLAGS/LDFLAGS stay in front. + _append_cc_deps_env(env, "CPPFLAGS", info["compile_flags"]) + _append_cc_deps_env(env, "LDFLAGS", info["link_flags"]) + + PARSER = ArgumentParser() PARSER.add_argument("srcarchive") PARSER.add_argument("outdir") @@ -278,6 +583,7 @@ def _legacy_metadata_conflicts_with_pyproject(worktree: str) -> bool: PARSER.add_argument("--patch-strip", type=int, default=0, help="Strip count for patch (-p)") PARSER.add_argument("--patch", action="append", default=[], dest="patches", help="Patch file to apply (repeatable)") PARSER.add_argument("--execroot-marker", help="Token in env values to replace with the absolute execroot") +PARSER.add_argument("--cc-deps-info", help="Path to the cc_deps compile/link params file to inject") opts, _ = PARSER.parse_known_args() tmp_root = path.abspath(opts.outdir) + ".tmp" @@ -324,6 +630,11 @@ def _legacy_metadata_conflicts_with_pyproject(worktree: str) -> bool: # change into the worktree. build_env = _compiler_env(tmp_root, opts.execroot_marker) +# cc_deps rides the same execroot cwd as _compiler_env: substitute the marker +# and inject the setuptools [build_ext] config before the backend chdirs. +if opts.cc_deps_info: + _apply_cc_deps(build_env, opts.cc_deps_info, opts.execroot_marker, t, tmp_root) + if _legacy_metadata_conflicts_with_pyproject(t): print( "Warning: falling back to setup.py because pyproject.toml omits dynamic dependency metadata " diff --git a/uv/private/pep517_whl/rule.bzl b/uv/private/pep517_whl/rule.bzl index 24db06c4a..db25ccd2c 100644 --- a/uv/private/pep517_whl/rule.bzl +++ b/uv/private/pep517_whl/rule.bzl @@ -8,6 +8,7 @@ build backend the sdist declares in its `[build-system]` table. load("@bazel_lib//lib:resource_sets.bzl", "resource_set", "resource_set_attr") load("@rules_cc//cc:action_names.bzl", "ACTION_NAMES") load("@rules_cc//cc/common:cc_common.bzl", "cc_common") +load("@rules_cc//cc/common:cc_info.bzl", "CcInfo") load("//py/private/toolchain:types.bzl", "NATIVE_BUILD_TOOLCHAIN", "PY_TOOLCHAIN") load("//uv/private:source_built_wheel.bzl", "SourceBuiltWheelInfo") @@ -156,6 +157,264 @@ def _cc_toolchain_inputs_and_tools(ctx): infer_cxx = infer_cxx or tools.get("CXX") == tools.get("CC") return files, {key: value for key, value in tools.items() if value}, infer_cxx +def _library_display_name(lib): + for artifact in (lib.static_library, lib.pic_static_library, lib.dynamic_library, lib.interface_library): + if artifact: + return artifact.basename + return "" + +# cc_deps flattens a dependency's link inputs into setuptools' two slots: static +# archives and `-l` entries go to the post-object [build_ext] slots, and +# every other link flag rides along in pre-object LDFLAGS. That split cannot +# preserve the relative order between an -l entry and a neighboring flag, so only +# flags whose effect does not depend on that order are safe to pass through. +# +# This is an allowlist of flag SHAPES, not a denylist of known-bad flags. A +# denylist against the open set of linker flags fails open: any order-sensitive +# flag we did not think to enumerate would be reordered silently into a stream +# that no longer links what the user wrote. The denylist this replaced had to +# grow twice under review for exactly that reason. An allowlist fails closed: an +# unrecognized flag is rejected at analysis, and the set can be widened later +# without breaking anyone, whereas a denylist can never be tightened. +# +# Each entry is (kind, token): +# "exact" the whole flag must equal token (e.g. -pthread) +# "prefix" the flag must start with token and carry a glued argument +# (e.g. -Ldir) +# "wl_arg" a -Wl, directive that takes an argument, either as the next +# comma segment (-Wl,-rpath,) or glued with = +# (-Wl,-rpath=) +# "wl_keyword" a -Wl, directive whose argument (next comma segment) must be +# one of the reviewed keywords in ALLOWED_Z_KEYWORDS +# (-Wl,-z,relro) +# "wl_exact" a -Wl, directive that stands alone (-Wl,--enable-new-dtags) +# +# -Wl, hands the linker a comma-joined list of directives, so a token whose +# leading directive is allowed could otherwise smuggle an order-sensitive +# directive behind it (-Wl,-rpath,/x,--as-needed). -Wl, tokens are therefore +# walked directive by directive and every directive must be allowed, which also +# keeps benign compounds like -Wl,-z,relro,-z,now working. +# +# -framework is deliberately absent: ld64 resolves frameworks in command-line +# order alongside -l entries (a framework is a library input, not a flag), so +# neither setuptools slot can hold one without reordering library resolution. +# +# Exported so the cc_deps test package pins it against a golden copy; a shape +# dropped here would otherwise drop its acceptance coverage silently. +ALLOWED_LINK_FLAG_SHAPES = ( + ("exact", "-pthread"), + ("prefix", "-L"), + ("wl_arg", "-rpath"), + ("wl_arg", "-rpath-link"), + ("wl_arg", "--version-script"), + ("wl_keyword", "-z"), + # --enable-new-dtags selects the global ELF dtags mode (DT_RUNPATH over + # DT_RPATH); it is position-insensitive and commonly comma-joined after an + # $ORIGIN rpath. + ("wl_exact", "--enable-new-dtags"), +) + +# Accepted -z keywords, each a global link mode. The -z namespace is open and +# contains position-sensitive keywords on some linkers (Solaris -z allextract +# toggles whole-archive extraction for the archives that follow it), so +# accepting the whole class would reintroduce the fail-open shape this +# allowlist exists to close. Keywords are reviewed individually; the set can +# be extended upstream on request. Golden-pinned by the cc_deps test package +# alongside the shapes. +ALLOWED_Z_KEYWORDS = ( + "relro", + "now", + "noexecstack", + "origin", +) + +_WL_ARG_DIRECTIVES = {token: True for kind, token in ALLOWED_LINK_FLAG_SHAPES if kind == "wl_arg"} +_WL_KEYWORD_DIRECTIVES = {token: True for kind, token in ALLOWED_LINK_FLAG_SHAPES if kind == "wl_keyword"} +_WL_EXACT_DIRECTIVES = {token: True for kind, token in ALLOWED_LINK_FLAG_SHAPES if kind == "wl_exact"} +_ALLOWED_Z_KEYWORDS = {keyword: True for keyword in ALLOWED_Z_KEYWORDS} + +def _link_flag_allowed(flag): + """Whether a bare (non-`-l`, non-`-Wl,`) flag matches an allowed shape.""" + for kind, token in ALLOWED_LINK_FLAG_SHAPES: + if kind == "exact" and flag == token: + return True + if kind == "prefix" and flag.startswith(token) and len(flag) > len(token): + return True + return False + +def _check_wl_link_flag(owner, flag): + """Validate every directive in a comma-joined `-Wl,` token, or fail. + + Walks the comma segments as directive/argument pairs: an allowed argument + directive consumes the next segment (or embeds its argument after `=`), a + keyword directive consumes the next segment and checks it against the + reviewed keyword set, and every directive in the token must be allowed. + Validating only the leading directive would let it smuggle an + order-sensitive directive behind it. + """ + segments = flag.split(",")[1:] + expect_arg = False + keyword_directive = None + last_directive = None + for segment in segments: + if expect_arg: + expect_arg = False + if keyword_directive != None and segment not in _ALLOWED_Z_KEYWORDS: + _reject_link_flag(owner, flag, "{} {}".format(keyword_directive, segment)) + keyword_directive = None + continue + last_directive = segment + if segment in _WL_EXACT_DIRECTIVES: + continue + if segment in _WL_KEYWORD_DIRECTIVES: + expect_arg = True + keyword_directive = segment + continue + if segment in _WL_ARG_DIRECTIVES: + expect_arg = True + continue + eq = segment.find("=") + if eq > 0 and eq < len(segment) - 1 and segment[:eq] in _WL_ARG_DIRECTIVES: + continue + _reject_link_flag(owner, flag, segment) + if expect_arg: + # A trailing argument directive with nothing to consume is malformed. + _reject_link_flag(owner, flag, last_directive) + +def _reject_link_flag(owner, flag, directive = None): + if directive == None: + offense = "passes the link flag {}, which is not one of the link-flag shapes cc_deps accepts".format(flag) + else: + offense = "passes the link flag {}; its directive {} is not one of the link-flag shapes cc_deps accepts".format(flag, directive) + fail(("cc_deps dependency {} {}. cc_deps splits a dependency's link " + + "inputs into pre-object LDFLAGS and the post-object [build_ext] libraries " + + "slot, so only position-insensitive flags can ride along. For an archive " + + "cycle, repeat the library name instead (e.g. -la -lb -la); order among " + + "-l entries is preserved. For a global toggle such as --as-needed, set it " + + "in the override's env LDFLAGS. For anything else, apply it with " + + "pre_build_patches on the sdist. The accepted shapes can be extended " + + "upstream on request.").format(owner, offense)) + +def _anchor_declared_paths(flag, declared_paths): + """Marker-anchor any declared additional_input path appearing in `flag`. + + Substitution goes through per-path placeholders, longest path first, so a + declared path that contains another declared path is never anchored twice. + """ + for index, path in enumerate(declared_paths): + flag = flag.replace(path, "\v{}\v".format(index)) + for index, path in enumerate(declared_paths): + flag = flag.replace("\v{}\v".format(index), "{}/{}".format(_EXECROOT_MARKER, path)) + return flag + +def _cc_deps_args_and_inputs(ctx): + """Flatten `cc_deps` CcInfo into a compile/link params file for the backend. + + Returns `(args, direct_inputs, transitive_inputs)`. When `cc_deps` is empty + all three are empty, so the action stays byte-identical to the no-cc_deps + path. Every emitted path is prefixed with `_EXECROOT_MARKER`: only + execroot-relative paths exist at analysis time, and build_helper substitutes + the real execroot before the backend changes into the unpacked source tree. + Slot information (post-object archives vs `-l` libraries vs order-insensitive + flags) is kept distinct in the JSON schema because flattening it into `env` + would lose it. + """ + if not ctx.attr.cc_deps: + return [], [], [] + + cc_info = cc_common.merge_cc_infos( + cc_infos = [dep[CcInfo] for dep in ctx.attr.cc_deps], + ) + compilation_context = cc_info.compilation_context + + compile_flags = [] + for include in compilation_context.includes.to_list(): + compile_flags.append("-I{}/{}".format(_EXECROOT_MARKER, include)) + for include in compilation_context.quote_includes.to_list(): + compile_flags.append("-iquote{}/{}".format(_EXECROOT_MARKER, include)) + for include in compilation_context.system_includes.to_list(): + compile_flags.append("-isystem{}/{}".format(_EXECROOT_MARKER, include)) + + # external_includes carries `-isystem` semantics; older Bazel lacks the field. + external_includes = getattr(compilation_context, "external_includes", None) + if external_includes: + for include in external_includes.to_list(): + compile_flags.append("-isystem{}/{}".format(_EXECROOT_MARKER, include)) + + # framework_includes carries Apple `-F` search paths; older Bazel lacks the field. + framework_includes = getattr(compilation_context, "framework_includes", None) + if framework_includes: + for framework in framework_includes.to_list(): + compile_flags.append("-F{}/{}".format(_EXECROOT_MARKER, framework)) + for define in compilation_context.defines.to_list(): + compile_flags.append("-D{}".format(define)) + + link_objects = [] + link_libraries = [] + link_flags = [] + archives = [] + additional_inputs = [] + for linker_input in cc_info.linking_context.linker_inputs.to_list(): + for lib in linker_input.libraries: + if lib.alwayslink: + fail(("cc_deps dependency {} contains alwayslink library {}; " + + "whole-archive linking is not supported.").format( + linker_input.owner, + _library_display_name(lib), + )) + archive = lib.pic_static_library or lib.static_library + if not archive: + fail(("cc_deps dependency {} provides no static archive (only a " + + "shared/dynamic library); the PEP 517 native build links " + + "static archives only. Provide a static or PIC-static " + + "library.").format(linker_input.owner)) + link_objects.append("{}/{}".format(_EXECROOT_MARKER, archive.path)) + archives.append(archive) + + # `-l` from the dep's own linkopts routes to setuptools' libraries + # slot; every other link flag must match an allowed shape (see + # ALLOWED_LINK_FLAG_SHAPES) or the build is rejected at analysis. Paths of + # files this linker_input declares via additional_inputs (e.g. + # `-Wl,--version-script,$(location ...)`) are marker-anchored so they + # survive the backend chdir. + declared_paths = sorted( + [f.path for f in linker_input.additional_inputs], + key = len, + reverse = True, + ) + for flag in linker_input.user_link_flags: + # The name carries no declared path, so -l routing precedes anchoring. + if flag.startswith("-l") and len(flag) > 2: + link_libraries.append(flag[len("-l"):]) + continue + + # -Wl, tokens are validated per directive so an allowed leading + # directive cannot smuggle an order-sensitive one behind it. + if flag.startswith("-Wl,"): + _check_wl_link_flag(linker_input.owner, flag) + elif not _link_flag_allowed(flag): + _reject_link_flag(linker_input.owner, flag) + + link_flags.append(_anchor_declared_paths(flag, declared_paths)) + additional_inputs.extend(linker_input.additional_inputs) + + params = ctx.actions.declare_file("cc_deps_info.json") + ctx.actions.write( + output = params, + content = json.encode({ + "compile_flags": compile_flags, + "link_objects": link_objects, + "link_libraries": link_libraries, + "link_flags": link_flags, + }), + ) + + return ( + ["--cc-deps-info", params.path], + [params] + additional_inputs, + [compilation_context.headers, depset(archives)], + ) + def _pep517_whl(ctx): archive = ctx.file.src wheel_dir = ctx.actions.declare_directory("whl") @@ -213,20 +472,22 @@ def _pep517_native_whl(ctx): if "CXX" not in ctx.attr.env and cc_tools.get("CXX") and infer_cxx: env[_INFER_CXX_COMPANION] = "1" + cc_deps_args, cc_deps_direct, cc_deps_transitive = _cc_deps_args_and_inputs(ctx) + ctx.actions.run( mnemonic = "PySdistNativeBuild", progress_message = "Native source compiling {} to a whl".format(archive.basename), executable = ctx.executable.tool, toolchain = None, - arguments = ctx.attr.args + patch_args + _memory_args(ctx) + [ + arguments = ctx.attr.args + patch_args + _memory_args(ctx) + cc_deps_args + [ "--execroot-marker", _EXECROOT_MARKER, archive.path, wheel_dir.path, ], inputs = depset( - [archive] + patch_inputs, - transitive = extra_inputs, + [archive] + patch_inputs + cc_deps_direct, + transitive = extra_inputs + cc_deps_transitive, ), tools = [ctx.attr.tool[DefaultInfo].files_to_run], outputs = [wheel_dir], @@ -305,6 +566,25 @@ constraints of the target platform. """, attrs = _pep517_whl_attrs | { "args": attr.string_list(), + "cc_deps": attr.label_list( + providers = [[CcInfo]], + doc = "C++ libraries whose headers and static archives are made " + + "available to the build backend. Each target's `CcInfo` " + + "compilation context (include paths and defines) and linking " + + "context (static archives and link flags) are flattened into a " + + "params file consumed by the backend; execroot-relative paths " + + "are anchored with an internal execroot marker so they survive " + + "the backend changing into the unpacked source tree. Link flags " + + "referencing files a dependency declares via " + + "`additional_linker_inputs` are anchored the same way; a " + + "relative path not declared there passes through verbatim and " + + "will not resolve after the backend changes directory. Composes " + + "additively with `env`. Only static (or PIC-static) archives " + + "are supported; shared/dynamic and alwayslink libraries fail at " + + "analysis time. Only the setuptools build backend is supported; " + + "an sdist declaring any other build-backend is rejected when the " + + "wheel is built.", + ), "env": attr.string_dict( doc = "Environment variables to set on the build action. Values may " + "contain `$(VAR)` references to the configured C++ action tools " + diff --git a/uv/private/pep517_whl/tests/cc_deps/BUILD.bazel b/uv/private/pep517_whl/tests/cc_deps/BUILD.bazel new file mode 100644 index 000000000..aef8f4dd9 --- /dev/null +++ b/uv/private/pep517_whl/tests/cc_deps/BUILD.bazel @@ -0,0 +1,422 @@ +load("@bazel_lib//:bzl_library.bzl", "bzl_library") +load("@bazel_skylib//rules:write_file.bzl", "write_file") +load("@rules_cc//cc:defs.bzl", "cc_import", "cc_library") +load("//uv/private/pep517_whl:rule.bzl", "pep517_native_whl") +load( + ":test.bzl", + "assert_allowlist_matches_golden", + "cc_framework_shim", + "pep517_native_whl_cc_deps_content_test", + "pep517_native_whl_cc_deps_failure_test", + "pep517_native_whl_cc_deps_test", +) + +# Fails the package at load time if rule.bzl's allowed link-flag shapes or -z +# keywords drift from the golden copies in test.bzl, so an entry removed +# upstream cannot quietly drop the acceptance coverage below. +assert_allowlist_matches_golden() + +bzl_library( + name = "test", + testonly = True, + srcs = ["test.bzl"], + visibility = ["//uv:__subpackages__"], + deps = [ + "//uv/private/pep517_whl:rule", + "@bazel_skylib//lib:unittest", + "@rules_cc//cc/common", + ], +) + +# cc_deps analysis fixtures live in their own package because each successful +# pep517_native_whl target declares a `whl` output directory; two successful +# fixtures in one package would collide (see native_dep for the same pattern). +# The archives/headers here are never compiled or linked; the tests inspect +# the PySdistNativeBuild action at analysis time only. + +write_file( + name = "cc_dep_hdr", + out = "cc_dep.h", + content = ["int cc_dep_value(void);"], + tags = ["manual"], +) + +write_file( + name = "cc_dep_src", + out = "cc_dep.c", + content = [ + "#include \"cc_dep.h\"", + "int cc_dep_value(void) { return 42; }", + ], + tags = ["manual"], +) + +cc_library( + name = "cc_dep", + testonly = True, + srcs = ["cc_dep.c"], + hdrs = ["cc_dep.h"], + includes = ["."], + tags = ["manual"], +) + +# A two-level chain (chain_dep -> chain_dep2) whose direct library carries +# user_link_flags: a `-l`, and a verbatim flag referencing a file +# declared via additional_linker_inputs. Together with :cc_dep it exercises +# merge coverage, topological archive order, the -l partition, and the +# marker-anchoring of declared paths inside link flags. +write_file( + name = "chain_dep2_hdr", + out = "chain_dep2.h", + content = ["int chain_dep2_value(void);"], + tags = ["manual"], +) + +write_file( + name = "chain_dep2_src", + out = "chain_dep2.c", + content = [ + "#include \"chain_dep2.h\"", + "int chain_dep2_value(void) { return 7; }", + ], + tags = ["manual"], +) + +# A define on the transitive leaf proves -D propagation reaches the merged +# compilation context (and lands in compile_flags as a bare -D, marker-free). +cc_library( + name = "chain_dep2", + testonly = True, + srcs = ["chain_dep2.c"], + hdrs = ["chain_dep2.h"], + defines = ["CHAIN_BONUS=3"], + tags = ["manual"], +) + +write_file( + name = "chain_dep_hdr", + out = "chain_dep.h", + content = ["int chain_dep_value(void);"], + tags = ["manual"], +) + +write_file( + name = "chain_dep_src", + out = "chain_dep.c", + content = [ + "#include \"chain_dep.h\"", + "#include \"chain_dep2.h\"", + "int chain_dep_value(void) { return chain_dep2_value(); }", + ], + tags = ["manual"], +) + +write_file( + name = "version_script", + out = "vs.lds", + content = ["{ global: *; local: *; };"], + tags = ["manual"], +) + +# The linkopts carry one exemplar per allowed link-flag shape, directive form, +# and reviewed -z keyword (the content test asserts each reaches link_flags +# verbatim and in order): a `-l` for the libraries slot, a declared-path +# version script in its comma and = spellings, an -L search path, the rpath +# directives in comma and = spellings plus -rpath-link, each reviewed -z +# keyword, the benign multi-directive compound -z,relro,-z,now, +# --enable-new-dtags comma-joined after an $ORIGIN rpath ($$ escapes the +# literal $), and -pthread. The -L/rpath dummy paths are fixture-relative and +# never linked (analysis-only fixture). +cc_library( + name = "chain_dep", + testonly = True, + srcs = ["chain_dep.c"], + hdrs = ["chain_dep.h"], + additional_linker_inputs = [":version_script"], + linkopts = [ + "-lchainfoo", + "-Wl,--version-script,$(location :version_script)", + "-Wl,--version-script=$(location :version_script)", + "-Lvendor/dummy", + "-Wl,-rpath,vendor/rpath", + "-Wl,-rpath=vendor/rpath_eq", + "-Wl,-rpath-link,vendor/rpath_link", + "-Wl,-z,now", + "-Wl,-z,relro,-z,now", + "-Wl,-z,noexecstack", + "-Wl,-z,origin", + "-Wl,-rpath,$$ORIGIN,--enable-new-dtags", + "-pthread", + ], + tags = ["manual"], + deps = [":chain_dep2"], +) + +# Synthesizes a CcInfo carrying only an Apple `-F` framework search path. A plain +# cc_library cannot populate framework_includes on Linux, so the content test +# relies on this to pin the -F emission and marker-anchoring. It contributes no +# headers or archives, so it does not perturb the header/archive assertions. +cc_framework_shim( + name = "cc_framework_dep", + testonly = True, + framework_includes = ["vendor/Frameworks"], + tags = ["manual"], +) + +pep517_native_whl( + name = "cc_deps_fixture", + testonly = True, + src = "//uv/private/pep517_whl:__stub_sdist", + cc_deps = [ + ":cc_dep", + ":cc_framework_dep", + ":chain_dep", + ], + tags = ["manual"], + tool = "//uv/private/pep517_whl:__stub_tool", + version = "0.0.1", +) + +pep517_native_whl_cc_deps_test( + name = "cc_deps_test", + target_under_test = ":cc_deps_fixture", +) + +pep517_native_whl_cc_deps_content_test( + name = "cc_deps_content_test", + target_under_test = ":cc_deps_fixture", +) + +# A shared-library-only dep must fail at analysis: static archives only. +write_file( + name = "dynamic_only_so", + out = "libcc_dynamic_only.so", + content = [""], + tags = ["manual"], +) + +cc_import( + name = "cc_dynamic_only_dep", + testonly = True, + shared_library = "libcc_dynamic_only.so", + tags = ["manual"], +) + +pep517_native_whl( + name = "cc_deps_dynamic_only_fixture", + testonly = True, + src = "//uv/private/pep517_whl:__stub_sdist", + cc_deps = [":cc_dynamic_only_dep"], + tags = ["manual"], + tool = "//uv/private/pep517_whl:__stub_tool", + version = "0.0.1", +) + +pep517_native_whl_cc_deps_failure_test( + name = "cc_deps_dynamic_only_test", + expected_message = "provides no static archive", + target_under_test = ":cc_deps_dynamic_only_fixture", +) + +# An alwayslink (whole-archive) dep must fail at analysis: v1 cut. +cc_library( + name = "cc_alwayslink_dep", + testonly = True, + srcs = ["cc_dep.c"], + hdrs = ["cc_dep.h"], + tags = ["manual"], + alwayslink = True, +) + +pep517_native_whl( + name = "cc_deps_alwayslink_fixture", + testonly = True, + src = "//uv/private/pep517_whl:__stub_sdist", + cc_deps = [":cc_alwayslink_dep"], + tags = ["manual"], + tool = "//uv/private/pep517_whl:__stub_tool", + version = "0.0.1", +) + +pep517_native_whl_cc_deps_failure_test( + name = "cc_deps_alwayslink_test", + expected_message = "whole-archive linking is not supported", + target_under_test = ":cc_deps_alwayslink_fixture", +) + +# A link flag outside the allowlist must fail at analysis rather than get +# silently reordered. Six cases cover the contract: (a) a flag the predecessor +# denylist named explicitly, proving those are still rejected; (b) an arbitrary +# flag no list ever enumerated, proving rejection is default-deny rather than +# list-membership; (c) a bare unwrapped token, pinning that a flag with no -Wl, +# wrapper is handled the same way; (d) a comma-joined -Wl, token whose leading +# directive is allowed but which smuggles a rejected directive behind it, +# proving the per-directive walk; (e) -framework, a library input the two-slot +# split cannot hold; (f) a -z keyword outside the reviewed set, proving -z is +# not an open class. Each test pins the exact offending flag (and directive) as +# the rule prints it. Failure fixtures may share the package: analysis fails +# before the whl output directory gets a generating action. +cc_library( + name = "cc_reject_group_dep", + testonly = True, + srcs = ["cc_dep.c"], + hdrs = ["cc_dep.h"], + linkopts = ["-Wl,--start-group"], + tags = ["manual"], +) + +pep517_native_whl( + name = "cc_deps_reject_group_fixture", + testonly = True, + src = "//uv/private/pep517_whl:__stub_sdist", + cc_deps = [":cc_reject_group_dep"], + tags = ["manual"], + tool = "//uv/private/pep517_whl:__stub_tool", + version = "0.0.1", +) + +pep517_native_whl_cc_deps_failure_test( + name = "cc_deps_reject_group_test", + expected_message = "passes the link flag -Wl,--start-group; its directive --start-group is not", + target_under_test = ":cc_deps_reject_group_fixture", +) + +cc_library( + name = "cc_reject_unknown_dep", + testonly = True, + srcs = ["cc_dep.c"], + hdrs = ["cc_dep.h"], + linkopts = ["-Wl,--definitely-not-a-real-flag"], + tags = ["manual"], +) + +pep517_native_whl( + name = "cc_deps_reject_unknown_fixture", + testonly = True, + src = "//uv/private/pep517_whl:__stub_sdist", + cc_deps = [":cc_reject_unknown_dep"], + tags = ["manual"], + tool = "//uv/private/pep517_whl:__stub_tool", + version = "0.0.1", +) + +pep517_native_whl_cc_deps_failure_test( + name = "cc_deps_reject_unknown_test", + expected_message = "passes the link flag -Wl,--definitely-not-a-real-flag; its directive --definitely-not-a-real-flag is not", + target_under_test = ":cc_deps_reject_unknown_fixture", +) + +# A bare token with no -Wl, wrapper. The compiler driver forwards such flags to +# the linker unwrapped, so this is the realistic bare form. +cc_library( + name = "cc_reject_bare_dep", + testonly = True, + srcs = ["cc_dep.c"], + hdrs = ["cc_dep.h"], + linkopts = ["--whole-archive"], + tags = ["manual"], +) + +pep517_native_whl( + name = "cc_deps_reject_bare_fixture", + testonly = True, + src = "//uv/private/pep517_whl:__stub_sdist", + cc_deps = [":cc_reject_bare_dep"], + tags = ["manual"], + tool = "//uv/private/pep517_whl:__stub_tool", + version = "0.0.1", +) + +pep517_native_whl_cc_deps_failure_test( + name = "cc_deps_reject_bare_test", + expected_message = "passes the link flag --whole-archive, which is not", + target_under_test = ":cc_deps_reject_bare_fixture", +) + +# An allowed leading directive (-rpath) must not smuggle a rejected directive +# behind it in the same comma-joined token; the walk validates every directive +# and the message names the offender. +cc_library( + name = "cc_reject_smuggle_dep", + testonly = True, + srcs = ["cc_dep.c"], + hdrs = ["cc_dep.h"], + linkopts = ["-Wl,-rpath,/x,--as-needed"], + tags = ["manual"], +) + +pep517_native_whl( + name = "cc_deps_reject_smuggle_fixture", + testonly = True, + src = "//uv/private/pep517_whl:__stub_sdist", + cc_deps = [":cc_reject_smuggle_dep"], + tags = ["manual"], + tool = "//uv/private/pep517_whl:__stub_tool", + version = "0.0.1", +) + +pep517_native_whl_cc_deps_failure_test( + name = "cc_deps_reject_smuggle_test", + expected_message = "passes the link flag -Wl,-rpath,/x,--as-needed; its directive --as-needed is not", + target_under_test = ":cc_deps_reject_smuggle_fixture", +) + +# -framework is a library input: ld64 resolves frameworks in command-line order +# alongside -l entries, so relocating the -l entries to the post-object slot +# while -framework rides pre-object LDFLAGS would reorder library resolution. +# Rejected; the head token is named before its name token is ever read. +cc_library( + name = "cc_reject_framework_dep", + testonly = True, + srcs = ["cc_dep.c"], + hdrs = ["cc_dep.h"], + linkopts = [ + "-framework", + "CoreFoundation", + ], + tags = ["manual"], +) + +pep517_native_whl( + name = "cc_deps_reject_framework_fixture", + testonly = True, + src = "//uv/private/pep517_whl:__stub_sdist", + cc_deps = [":cc_reject_framework_dep"], + tags = ["manual"], + tool = "//uv/private/pep517_whl:__stub_tool", + version = "0.0.1", +) + +pep517_native_whl_cc_deps_failure_test( + name = "cc_deps_reject_framework_test", + expected_message = "passes the link flag -framework, which is not", + target_under_test = ":cc_deps_reject_framework_fixture", +) + +# A -z keyword outside the reviewed set must be rejected: the -z namespace is +# open and contains position-sensitive keywords (Solaris -z allextract toggles +# whole-archive extraction for the archives that follow it). +cc_library( + name = "cc_reject_z_keyword_dep", + testonly = True, + srcs = ["cc_dep.c"], + hdrs = ["cc_dep.h"], + linkopts = ["-Wl,-z,allextract"], + tags = ["manual"], +) + +pep517_native_whl( + name = "cc_deps_reject_z_keyword_fixture", + testonly = True, + src = "//uv/private/pep517_whl:__stub_sdist", + cc_deps = [":cc_reject_z_keyword_dep"], + tags = ["manual"], + tool = "//uv/private/pep517_whl:__stub_tool", + version = "0.0.1", +) + +pep517_native_whl_cc_deps_failure_test( + name = "cc_deps_reject_z_keyword_test", + expected_message = "passes the link flag -Wl,-z,allextract; its directive -z allextract is not", + target_under_test = ":cc_deps_reject_z_keyword_fixture", +) diff --git a/uv/private/pep517_whl/tests/cc_deps/executable/BUILD.bazel b/uv/private/pep517_whl/tests/cc_deps/executable/BUILD.bazel new file mode 100644 index 000000000..0ee8a85ae --- /dev/null +++ b/uv/private/pep517_whl/tests/cc_deps/executable/BUILD.bazel @@ -0,0 +1,140 @@ +load("@rules_cc//cc:defs.bzl", "cc_library") +load("@tar.bzl//tar:mtree.bzl", "mtree_mutate", "mtree_spec") +load("@tar.bzl//tar:tar.bzl", "tar_rule") +load("//py:defs.bzl", "py_test") +load("//uv/private/pep517_whl:rule.bzl", "pep517_native_whl") + +# The executable cc_deps regression lives in its own package: each successful +# pep517_native_whl target declares a `whl` output directory, so it cannot share +# a package with the analysis-only fixtures in the parent cc_deps package (they +# own one already). Unlike those fixtures, these libraries are really compiled +# and linked into a wheel that is imported and called through. + +# A two-level chain: dep -> dep2. dep_value(x) returns dep2_value() plus +# extra_value(x) (compiled by the sdist's own build_clib into libextra.a), and +# mod.c adds MOD_BONUS. The sdist also builds libgroup_a.a / libgroup_b.a with a +# deliberate one-pass archive cycle; the -lgroup_a, -lgroup_b, -lgroup_a stream +# below (the documented repeat-the-library workaround for cycles) is required to +# resolve it, and proves -l order is preserved through the libraries slot. +cc_library( + name = "dep2", + testonly = True, + srcs = ["dep2.c"], + hdrs = ["dep2.h"], + includes = ["."], +) + +cc_library( + name = "dep", + testonly = True, + srcs = ["dep.c"], + hdrs = ["dep.h"], + # defines propagate transitively to the backend's compile of mod.c through + # cc_deps' CPPFLAGS routing (the -D path). The libraries resolve against + # archives the sdist's build_clib compiles hermetically in-build; setup.py + # suppresses distutils' automatic linking, so this exact linkopts stream is + # solely responsible for naming them. libgroup_a.a and libgroup_b.a form a + # cycle: group_a_entry.o -> group_b.o -> group_a_tail.o, so under single-pass + # GNU ld the linker must rescan libgroup_a.a after libgroup_b.a. Order- + # sensitive --start-group/--end-group cannot survive cc_deps' two-slot + # flattening (rejected at analysis), so the cycle is resolved with the + # documented workaround instead: repeat -lgroup_a after -lgroup_b. This also + # proves -l order is preserved through the [build_ext] libraries slot. + defines = ["MOD_BONUS=2"], + includes = ["."], + linkopts = [ + "-lextra", + "-lgroup_a", + "-lgroup_b", + "-lgroup_a", + ], + deps = [":dep2"], +) + +# A real setuptools sdist. dep.h is intentionally NOT part of the sources: the +# extension resolves it only through the cc_deps include path, proving the +# header injection rather than an accidental in-tree copy. +_SDIST_SRCS = glob( + ["sdist/**"], + exclude = [ + "sdist/**/__pycache__/**", + "sdist/**/*.pyc", + ], +) + +mtree_spec( + name = "sdist_mtree_raw", + testonly = True, + srcs = _SDIST_SRCS, + include_runfiles = False, +) + +mtree_mutate( + name = "sdist_mtree", + testonly = True, + mtree = ":sdist_mtree_raw", + strip_prefix = package_name(), +) + +tar_rule( + name = "sdist", + testonly = True, + srcs = _SDIST_SRCS, + mtree = ":sdist_mtree", +) + +# Build the wheel with the setuptools-equipped helper (the production +# __build_helper ships only @pypi//build; a real setuptools.build_meta backend +# under --no-isolation needs setuptools importable). +pep517_native_whl( + name = "whl", + testonly = True, + src = ":sdist", + cc_deps = [":dep"], + tags = ["manual"], + tool = "//uv/private/pep517_whl:__build_helper_setuptools", + version = "0.0.1", +) + +py_test( + name = "import_test", + srcs = ["import_test.py"], + data = [":whl"], + main = "import_test.py", + # The wheel's extension .so is ABI-tagged for the build interpreter; both the + # build and this test resolve the same rules_py Python toolchain. Static + # archives are linked into a shared object, so gate on Linux like the geohash + # native-import regression. + target_compatible_with = ["@platforms//os:linux"], + deps = ["@pypi//bazel_runfiles"], +) + +# build_helper.py prefers stdlib tomllib and only falls back to tomli below +# Python 3.11; the tomli dep is dead on a >=3.11 toolchain and lives only for the +# cp39 rules_py default toolchain these helper subprocesses run under. +_HELPER_SUBPROCESS_DEPS = [ + "@pypi//bazel_runfiles", + "@pypi//tomli", +] + +# Negative: build_helper.py must reject cc_deps against a non-setuptools backend +# rather than silently drop the inputs. Runs the helper as a subprocess; the +# sdist tar and cc-deps params file are synthesized in the test. +py_test( + name = "backend_gate_test", + srcs = ["backend_gate_test.py"], + data = ["//uv/private/pep517_whl:build_helper.py"], + main = "backend_gate_test.py", + deps = _HELPER_SUBPROCESS_DEPS, +) + +# The rest of the runtime gate matrix + the setup.cfg [build_ext] REPLACE-merge, +# all driven by running build_helper.py as a subprocess against synthetic sdists +# (same technique as backend_gate_test). No compiler needed, so ungated. +py_test( + name = "gates_test", + srcs = ["gates_test.py"], + data = ["//uv/private/pep517_whl:build_helper.py"], + main = "gates_test.py", + deps = _HELPER_SUBPROCESS_DEPS, +) diff --git a/uv/private/pep517_whl/tests/cc_deps/executable/backend_gate_test.py b/uv/private/pep517_whl/tests/cc_deps/executable/backend_gate_test.py new file mode 100644 index 000000000..177d49404 --- /dev/null +++ b/uv/private/pep517_whl/tests/cc_deps/executable/backend_gate_test.py @@ -0,0 +1,93 @@ +"""Runtime gate: build_helper.py must refuse cc_deps on a non-setuptools backend. + +cc_deps injects setuptools [build_ext] settings that other PEP 517 backends +ignore, so silently dropping the inputs would produce a wheel that fails to +link at import time. build_helper must instead fail fast with an actionable +message. This runs build_helper.py directly as a subprocess against a synthetic +sdist whose declared backend is not setuptools, plus a crafted cc-deps params +file, and asserts exit 1 with the backend-gate message. +""" + +import json +import os +import subprocess +import sys +import tarfile +import tempfile + +import runfiles + +_MARKER = "__ASPECT_RULES_PY_EXECROOT__" +_HELPER = "_main/uv/private/pep517_whl/build_helper.py" + + +def main() -> None: + r = runfiles.Create() + helper = r.Rlocation(_HELPER) + assert helper and os.path.isfile(helper), "build_helper.py missing: {}".format(helper) + + work = tempfile.mkdtemp(dir=os.environ["TEST_TMPDIR"]) + + # A synthetic sdist whose backend is deliberately not setuptools. + src_dir = os.path.join(work, "pkg") + os.makedirs(src_dir) + with open(os.path.join(src_dir, "pyproject.toml"), "w") as f: + f.write( + "[build-system]\n" + "requires = []\n" + 'build-backend = "not_setuptools.api"\n' + 'backend-path = ["."]\n' + ) + sdist = os.path.join(work, "sdist.tar.gz") + with tarfile.open(sdist, "w:gz") as tar: + tar.add(src_dir, arcname="pkg") + + # A minimal but non-empty cc-deps params file: cc_deps is present, so the + # gate must fire rather than no-op. + info = os.path.join(work, "cc_deps_info.json") + with open(info, "w") as f: + json.dump( + { + "compile_flags": ["-I{}/some/include".format(_MARKER)], + "link_objects": ["{}/some/libdep.a".format(_MARKER)], + "link_libraries": [], + "link_flags": [], + }, + f, + ) + + outdir = os.path.join(work, "out") + + # Deterministic by construction: a minimal env built from scratch (not a + # filtered copy of os.environ), so an ambient DIST_EXTRA_CONFIG or + # SETUPTOOLS_USE_DISTUTILS on the host cannot perturb the helper. The gate + # path needs no host tools; build_helper falls back to os.defpath itself. + subprocess_env = {"PATH": os.defpath} + + proc = subprocess.run( + [ + sys.executable, + helper, + "--cc-deps-info", info, + "--execroot-marker", _MARKER, + sdist, + outdir, + ], + cwd=work, + env=subprocess_env, + capture_output=True, + text=True, + ) + + assert proc.returncode == 1, \ + "expected exit 1, got {}; stderr:\n{}".format(proc.returncode, proc.stderr) + assert "cc_deps is only supported with the setuptools build backend" in proc.stderr, \ + "missing backend-gate error; stderr:\n{}".format(proc.stderr) + assert "not_setuptools.api" in proc.stderr, \ + "error should name the offending backend; stderr:\n{}".format(proc.stderr) + + print("ok: backend gate fired with exit 1") + + +if __name__ == "__main__": + main() diff --git a/uv/private/pep517_whl/tests/cc_deps/executable/dep.c b/uv/private/pep517_whl/tests/cc_deps/executable/dep.c new file mode 100644 index 000000000..1f86c8b23 --- /dev/null +++ b/uv/private/pep517_whl/tests/cc_deps/executable/dep.c @@ -0,0 +1,20 @@ +#include "dep.h" + +#include "dep2.h" + +/* extra_value() is defined in extra.c INSIDE the sdist, where setuptools' + * build_clib compiles it into libextra.a during the wheel build (hermetic: no + * host library involved). Only a forward declaration here: the `dep` cc_library + * carries linkopts = ["-lextra"], which cc_deps routes through the setuptools + * [build_ext] libraries slot; the sdist's build_clib subclass suppresses + * distutils' auto -l for clib libraries, so dropping that linkopt leaves + * extra_value undefined at import. The [build_ext] libraries slot is what + * carries the link. */ +extern int extra_value(int x); + +/* dep_value() calls through the transitive leaf dep2_value() and the + * build_clib-built extra_value() on an argument-derived value (extra_value is + * external, so the call cannot be folded): a wheel that imports and returns the + * expected value proves both archives linked in order AND that -lextra reached + * the link through the [build_ext] libraries slot. */ +int dep_value(int x) { return dep2_value() + extra_value(x); } diff --git a/uv/private/pep517_whl/tests/cc_deps/executable/dep.h b/uv/private/pep517_whl/tests/cc_deps/executable/dep.h new file mode 100644 index 000000000..743315c88 --- /dev/null +++ b/uv/private/pep517_whl/tests/cc_deps/executable/dep.h @@ -0,0 +1,6 @@ +#ifndef ASPECT_RULES_PY_TEST_CC_DEPS_DEP_H +#define ASPECT_RULES_PY_TEST_CC_DEPS_DEP_H + +int dep_value(int x); + +#endif diff --git a/uv/private/pep517_whl/tests/cc_deps/executable/dep2.c b/uv/private/pep517_whl/tests/cc_deps/executable/dep2.c new file mode 100644 index 000000000..cfa2a7153 --- /dev/null +++ b/uv/private/pep517_whl/tests/cc_deps/executable/dep2.c @@ -0,0 +1,3 @@ +#include "dep2.h" + +int dep2_value(void) { return 40; } diff --git a/uv/private/pep517_whl/tests/cc_deps/executable/dep2.h b/uv/private/pep517_whl/tests/cc_deps/executable/dep2.h new file mode 100644 index 000000000..81e17a8da --- /dev/null +++ b/uv/private/pep517_whl/tests/cc_deps/executable/dep2.h @@ -0,0 +1,6 @@ +#ifndef ASPECT_RULES_PY_TEST_CC_DEPS_DEP2_H +#define ASPECT_RULES_PY_TEST_CC_DEPS_DEP2_H + +int dep2_value(void); + +#endif diff --git a/uv/private/pep517_whl/tests/cc_deps/executable/gates_test.py b/uv/private/pep517_whl/tests/cc_deps/executable/gates_test.py new file mode 100644 index 000000000..fa64651f1 --- /dev/null +++ b/uv/private/pep517_whl/tests/cc_deps/executable/gates_test.py @@ -0,0 +1,307 @@ +"""Runtime gate matrix + setup.cfg [build_ext] REPLACE-merge coverage. + +Extends backend_gate_test.py's technique (run build_helper.py as a subprocess +against a synthetic sdist + crafted --cc-deps-info JSON) to the cc_deps runtime +gates that were previously untested, and to the DIST_EXTRA_CONFIG merge: + + * setup.cfg [build_ext] REPLACE-merge: the package's own link_objects / + libraries values survive, PRECEDE ours, and include_dirs/define never leak + into our generated cfg (the scariest silent failure would be dropped + archives), + * the DIST_EXTRA_CONFIG-preexists, SETUPTOOLS_USE_DISTUTILS=stdlib, whitespace, + execroot-marker-survival, and setuptools-floor gates. + +Each case asserts the gate's OWN message, so an unrelated exit 1 fails the test. + +The subprocess env is built from scratch (PATH + a PYTHONPATH that only prepends +a controlled fake setuptools dist-info), so an ambient DIST_EXTRA_CONFIG / +SETUPTOOLS_USE_DISTUTILS on the host cannot perturb the helper. tomli still +resolves from this test's own venv (the runfiles interpreter), exactly as it +does for backend_gate_test. +""" + +from __future__ import annotations + +import json +import os +import subprocess +import sys +import tarfile +import tempfile + +import runfiles + +_MARKER = "__ASPECT_RULES_PY_EXECROOT__" +_HELPER = "_main/uv/private/pep517_whl/build_helper.py" + + +def _helper_path() -> str: + r = runfiles.Create() + helper = r.Rlocation(_HELPER) + assert helper and os.path.isfile(helper), "build_helper.py missing: {}".format(helper) + return helper + + +def _work(name: str) -> str: + return tempfile.mkdtemp(prefix=name + "_", dir=os.environ["TEST_TMPDIR"]) + + +def _setuptools_sdist(work: str, setup_cfg: str | None = None) -> str: + """Write a minimal setuptools-backend sdist tar (single top-level `pkg/`).""" + src = os.path.join(work, "src") + os.makedirs(src) + with open(os.path.join(src, "pyproject.toml"), "w") as f: + f.write( + "[build-system]\n" + 'requires = ["setuptools"]\n' + 'build-backend = "setuptools.build_meta"\n' + ) + if setup_cfg is not None: + with open(os.path.join(src, "setup.cfg"), "w") as f: + f.write(setup_cfg) + sdist = os.path.join(work, "sdist.tar") + with tarfile.open(sdist, "w") as tar: + tar.add(src, arcname="pkg") + return sdist + + +def _fake_setuptools(work: str, version: str) -> str: + """Create a bare `setuptools-.dist-info` dir; return the parent path. + + importlib.metadata.version() only reads METADATA, so a standalone dist-info + on sys.path is enough to drive the floor check to an exact version. Prepended + to PYTHONPATH, it wins over any setuptools the interpreter might ship. + """ + parent = os.path.join(work, "fake_sp") + info = os.path.join(parent, "setuptools-{}.dist-info".format(version)) + os.makedirs(info) + with open(os.path.join(info, "METADATA"), "w") as f: + f.write("Metadata-Version: 2.1\nName: setuptools\nVersion: {}\n".format(version)) + return parent + + +def _info(work: str, **kwargs: list[str]) -> str: + payload = {"compile_flags": [], "link_objects": [], "link_libraries": [], "link_flags": []} + payload.update(kwargs) + info = os.path.join(work, "cc_deps_info.json") + with open(info, "w") as f: + json.dump(payload, f) + return info + + +def _env(fake_setuptools_dir: str | None = None, **extra: str) -> dict[str, str]: + env = {"PATH": os.defpath} + if fake_setuptools_dir: + env["PYTHONPATH"] = fake_setuptools_dir + env.update(extra) + return env + + +def _run( + helper: str, + sdist: str, + info: str, + outdir: str, + cwd: str, + env: dict[str, str], +) -> subprocess.CompletedProcess[str]: + return subprocess.run( + [ + sys.executable, + helper, + "--cc-deps-info", info, + "--execroot-marker", _MARKER, + sdist, + outdir, + ], + cwd=cwd, + env=env, + capture_output=True, + text=True, + ) + + +def _assert_gate(proc: subprocess.CompletedProcess[str], needle: str) -> None: + assert proc.returncode == 1, \ + "expected exit 1, got {}; stderr:\n{}".format(proc.returncode, proc.stderr) + assert needle in proc.stderr, \ + "expected gate message {!r}; stderr:\n{}".format(needle, proc.stderr) + + +def check_setup_cfg_replace_merge(helper: str) -> None: + """G1: our generated [build_ext] cfg folds the package's values in front and + never carries include_dirs/define. Read tmp_root's cfg after the (expected) + build failure: build_helper writes it during env prep, before invoking + `python -m build`, and leaves tmp_root behind when the build fails.""" + work = _work("g1") + fake = _fake_setuptools(work, "70.0.0") + setup_cfg = ( + "[build_ext]\n" + "link_objects = pkg/vendor/libpkg.a\n" + "libraries = pkgfoo\n" + "include_dirs = pkg/include\n" + "define = PKG_FLAG\n" + ) + sdist = _setuptools_sdist(work, setup_cfg=setup_cfg) + info = _info( + work, + compile_flags=["-I{}/gen/include".format(_MARKER), "-DOURDEF=1"], + link_objects=["{}/gen/libdep.a".format(_MARKER)], + link_libraries=["ourlib"], + ) + outdir = os.path.join(work, "out") + proc = _run(helper, sdist, info, outdir, cwd=work, env=_env(fake)) + + # No `build` module in this bare env, so `python -m build` fails AFTER the + # cfg is written; that non-zero exit is expected and confirms the write ran. + assert proc.returncode == 1, \ + "expected the (post-cfg) build to fail; stderr:\n{}".format(proc.stderr) + + # DIST_EXTRA_CONFIG is set only in the child's build_env (not printed), so it + # is not observable from here; assert the cfg exists at the documented + # tmp_root path it would point at instead. + cfg_path = os.path.join(os.path.abspath(outdir) + ".tmp", "cc_deps_extra.cfg") + assert os.path.isfile(cfg_path), \ + "cc_deps_extra.cfg not written at {}; stderr:\n{}".format(cfg_path, proc.stderr) + with open(cfg_path) as f: + cfg = f.read() + + lines = cfg.splitlines() + lo = next(line for line in lines if line.startswith("link_objects")) + lib = next(line for line in lines if line.startswith("libraries")) + + # cwd == execroot here, so the marker expands to `work`. + our_obj = "{}/gen/libdep.a".format(work) + + # (a) the package's own values are present and PRECEDE ours ... + assert "pkg/vendor/libpkg.a" in lo and our_obj in lo, \ + "link_objects should carry both the package archive and ours; got {!r}".format(lo) + assert lo.index("pkg/vendor/libpkg.a") < lo.index(our_obj), \ + "package link_objects should precede ours; got {!r}".format(lo) + assert "pkgfoo" in lib and "ourlib" in lib, \ + "libraries should carry both the package lib and ours; got {!r}".format(lib) + assert lib.index("pkgfoo") < lib.index("ourlib"), \ + "package libraries should precede ours; got {!r}".format(lib) + + # (b/c) include_dirs and define ride CPPFLAGS, never the cfg, so the package's + # own include_dirs/define must NOT be echoed into our generated cfg. + assert "include_dirs" not in cfg, "cfg must not carry include_dirs; got:\n{}".format(cfg) + assert "define" not in cfg, "cfg must not carry define; got:\n{}".format(cfg) + + print("ok: setup.cfg [build_ext] REPLACE-merge folds package values in front") + + +def check_dist_extra_config_preexists(helper: str) -> None: + """G4(i): a DIST_EXTRA_CONFIG already in the env is a hard error (v1 owns it).""" + work = _work("g4_dec") + fake = _fake_setuptools(work, "70.0.0") + sdist = _setuptools_sdist(work) + info = _info(work, link_objects=["{}/gen/libdep.a".format(_MARKER)]) + env = _env(fake, DIST_EXTRA_CONFIG=os.path.join(work, "user.cfg")) + proc = _run(helper, sdist, info, os.path.join(work, "out"), cwd=work, env=env) + _assert_gate(proc, "already set in the build environment") + print("ok: pre-existing DIST_EXTRA_CONFIG rejected") + + +def check_use_distutils_stdlib(helper: str) -> None: + """G4(ii): SETUPTOOLS_USE_DISTUTILS=stdlib disables the DIST_EXTRA_CONFIG path.""" + work = _work("g4_ud") + fake = _fake_setuptools(work, "70.0.0") + sdist = _setuptools_sdist(work) + info = _info(work, link_objects=["{}/gen/libdep.a".format(_MARKER)]) + env = _env(fake, SETUPTOOLS_USE_DISTUTILS="stdlib") + proc = _run(helper, sdist, info, os.path.join(work, "out"), cwd=work, env=env) + _assert_gate(proc, "SETUPTOOLS_USE_DISTUTILS=stdlib is set") + print("ok: SETUPTOOLS_USE_DISTUTILS=stdlib rejected") + + +def check_whitespace_guard(helper: str) -> None: + """G4(iii): a whitespace-bearing link object is rejected, naming the path.""" + work = _work("g4_ws") + fake = _fake_setuptools(work, "70.0.0") + sdist = _setuptools_sdist(work) + info = _info(work, link_objects=["{}/gen dir/libdep.a".format(_MARKER)]) + proc = _run(helper, sdist, info, os.path.join(work, "out"), cwd=work, env=_env(fake)) + _assert_gate(proc, "contains whitespace") + assert "gen dir/libdep.a" in proc.stderr, \ + "whitespace error should name the offending path; stderr:\n{}".format(proc.stderr) + print("ok: whitespace-bearing link object rejected, path named") + + +def check_framework_whitespace_guard(helper: str) -> None: + """A whitespace-bearing framework (-F) search path is rejected, naming it. + + Apple `-F` paths ride CPPFLAGS, which is word-split downstream, so a spaced + -F path is unrepresentable; it is caught by the same compile-path guard that + covers -I/-iquote/-isystem.""" + work = _work("g4_fw") + fake = _fake_setuptools(work, "70.0.0") + sdist = _setuptools_sdist(work) + info = _info(work, compile_flags=["-F{}/gen dir/Frameworks".format(_MARKER)]) + proc = _run(helper, sdist, info, os.path.join(work, "out"), cwd=work, env=_env(fake)) + _assert_gate(proc, "contains whitespace") + assert "gen dir/Frameworks" in proc.stderr, \ + "framework whitespace error should name the offending path; stderr:\n{}".format(proc.stderr) + print("ok: whitespace-bearing framework search path rejected, path named") + + +def check_link_flag_whitespace_guard(helper: str) -> None: + """A whitespace-bearing link flag is rejected even when it is not an + execroot-anchored path. LDFLAGS is word-split downstream, so no single flag + can carry a space, regardless of where the space came from.""" + work = _work("g4_lf") + fake = _fake_setuptools(work, "70.0.0") + sdist = _setuptools_sdist(work) + info = _info(work, link_flags=["-Wl,-rpath,/opt/my libs"]) + proc = _run(helper, sdist, info, os.path.join(work, "out"), cwd=work, env=_env(fake)) + _assert_gate(proc, "contains whitespace") + assert "/opt/my libs" in proc.stderr, \ + "link-flag whitespace error should name the offending flag; stderr:\n{}".format(proc.stderr) + print("ok: whitespace-bearing link flag rejected, flag named") + + +def check_marker_survival(helper: str) -> None: + """G4(iv): if the execroot itself contains the marker token, substitution + cannot complete and build_helper must fail rather than emit a corrupt path. + Triggered honestly by running with a cwd whose path contains the marker.""" + parent = _work("g4_surv") + work = os.path.join(parent, "dir_{}_here".format(_MARKER)) + os.makedirs(work) + # No fake setuptools needed: this gate fires in _load_cc_deps_info, before + # the backend/floor checks. + sdist = _setuptools_sdist(work) + info = _info(work, link_objects=["{}/gen/libdep.a".format(_MARKER)]) + proc = _run(helper, sdist, info, os.path.join(work, "out"), cwd=work, env=_env()) + _assert_gate(proc, "execroot marker survived substitution") + print("ok: surviving execroot marker rejected") + + +def check_setuptools_floor(helper: str) -> None: + """G4 floor gate: setuptools below the DIST_EXTRA_CONFIG floor (65.4.0) is a + hard error. Driven by a fake old dist-info prepended to PYTHONPATH.""" + work = _work("g4_floor") + fake = _fake_setuptools(work, "60.0.0") + sdist = _setuptools_sdist(work) + info = _info(work, link_objects=["{}/gen/libdep.a".format(_MARKER)]) + proc = _run(helper, sdist, info, os.path.join(work, "out"), cwd=work, env=_env(fake)) + _assert_gate(proc, "requires setuptools >= 65.4.0") + assert "60.0.0" in proc.stderr, \ + "floor error should report the offending version; stderr:\n{}".format(proc.stderr) + print("ok: sub-floor setuptools rejected") + + +def main() -> None: + helper = _helper_path() + check_setup_cfg_replace_merge(helper) + check_dist_extra_config_preexists(helper) + check_use_distutils_stdlib(helper) + check_whitespace_guard(helper) + check_framework_whitespace_guard(helper) + check_link_flag_whitespace_guard(helper) + check_marker_survival(helper) + check_setuptools_floor(helper) + print("ok: all cc_deps gate/merge cases fired") + + +if __name__ == "__main__": + main() diff --git a/uv/private/pep517_whl/tests/cc_deps/executable/import_test.py b/uv/private/pep517_whl/tests/cc_deps/executable/import_test.py new file mode 100644 index 000000000..7a1ee575f --- /dev/null +++ b/uv/private/pep517_whl/tests/cc_deps/executable/import_test.py @@ -0,0 +1,67 @@ +"""Import a real setuptools extension built via pep517_native_whl(cc_deps=...). + +End-to-end proof that, after the setuptools backend changes into the unpacked +sdist: + * resolved through the cc_deps include path (CPPFLAGS) even though the + header is deliberately absent from the sdist, + * dep_value() linked through the post-object cc_deps archive slot, + * the transitive leaf dep2_value() linked too, + * -lextra reached the link through the [build_ext] libraries slot, resolving + against the libextra.a the sdist's own build_clib compiled in-build, and + * -lgroup_a, -lgroup_b, -lgroup_a retained their relative order through the + [build_ext] libraries slot, resolving a deliberate one-pass archive cycle + via the documented repeat-the-library workaround, and + * MOD_BONUS arrived via a transitively-propagated cc_deps -D define. + +The module is loaded from the freshly extracted wheel (asserted via __file__), +so a stale ambient build cannot satisfy the test. +""" + +import glob +import os +import sys +import zipfile + +import runfiles + +_WHL_DIR = "_main/uv/private/pep517_whl/tests/cc_deps/executable/whl" + + +def main() -> None: + r = runfiles.Create() + assert r is not None, "runfiles unavailable" + + whl_dir = r.Rlocation(_WHL_DIR) + assert whl_dir and os.path.isdir(whl_dir), "wheel output dir missing: {}".format(whl_dir) + + wheels = glob.glob(os.path.join(whl_dir, "*.whl")) + assert len(wheels) == 1, "expected exactly one wheel, got {}".format(wheels) + + extract_dir = os.path.join(os.environ["TEST_TMPDIR"], "wheel_extract") + with zipfile.ZipFile(wheels[0]) as archive: + archive.extractall(extract_dir) + + sys.path.insert(0, extract_dir) + import cc_deps_ext + + # The extension .so must come from the wheel we just extracted, not from any + # ambient copy that happened to be importable. + module_file = os.path.realpath(cc_deps_ext.__file__) + assert module_file.startswith(os.path.realpath(extract_dir) + os.sep), \ + "module loaded from unexpected location: {}".format(module_file) + + result = cc_deps_ext.value() + # dep_value(7) = dep2_value() 40 + extra_value(7) 17 (7 + 10, via -lextra); + # MOD_BONUS contributes 2; and the ordered cyclic archives contribute 125 + # (group_entry 100 + group_b 20 + group_a_tail 5). Each contribution is + # distinct: 40 + 17 + 2 + 125 == 184. + expected = 40 + (7 + 10) + 2 + 125 + assert result == expected, \ + "expected cc_deps_ext.value() == {} (dep2 40 + extra 17 + MOD_BONUS 2 + group 125), got {}".format( + expected, result) + + print("ok: cc_deps_ext.value() == {} loaded from {}".format(result, module_file)) + + +if __name__ == "__main__": + main() diff --git a/uv/private/pep517_whl/tests/cc_deps/executable/sdist/extra.c b/uv/private/pep517_whl/tests/cc_deps/executable/sdist/extra.c new file mode 100644 index 000000000..d932e3451 --- /dev/null +++ b/uv/private/pep517_whl/tests/cc_deps/executable/sdist/extra.c @@ -0,0 +1,6 @@ +/* Compiled by setuptools' build_clib (declared via setup(libraries=...)) into + * libextra.a inside the wheel build, before build_ext runs (hermetically, by + * the same configured toolchain). dep.c (outside the sdist, linked via cc_deps) + * forward-declares and calls extra_value(); the -lextra that resolves it is + * injected through the cc_deps [build_ext] libraries slot. */ +int extra_value(int x) { return x + 10; } diff --git a/uv/private/pep517_whl/tests/cc_deps/executable/sdist/group_a_entry.c b/uv/private/pep517_whl/tests/cc_deps/executable/sdist/group_a_entry.c new file mode 100644 index 000000000..3581cf050 --- /dev/null +++ b/uv/private/pep517_whl/tests/cc_deps/executable/sdist/group_a_entry.c @@ -0,0 +1,3 @@ +long group_b_value(void); + +long group_entry(void) { return 100 + group_b_value(); } diff --git a/uv/private/pep517_whl/tests/cc_deps/executable/sdist/group_a_tail.c b/uv/private/pep517_whl/tests/cc_deps/executable/sdist/group_a_tail.c new file mode 100644 index 000000000..014ba6534 --- /dev/null +++ b/uv/private/pep517_whl/tests/cc_deps/executable/sdist/group_a_tail.c @@ -0,0 +1 @@ +long group_a_tail_value(void) { return 5; } diff --git a/uv/private/pep517_whl/tests/cc_deps/executable/sdist/group_b.c b/uv/private/pep517_whl/tests/cc_deps/executable/sdist/group_b.c new file mode 100644 index 000000000..d72a9d708 --- /dev/null +++ b/uv/private/pep517_whl/tests/cc_deps/executable/sdist/group_b.c @@ -0,0 +1,3 @@ +long group_a_tail_value(void); + +long group_b_value(void) { return 20 + group_a_tail_value(); } diff --git a/uv/private/pep517_whl/tests/cc_deps/executable/sdist/mod.c b/uv/private/pep517_whl/tests/cc_deps/executable/sdist/mod.c new file mode 100644 index 000000000..72a34f12d --- /dev/null +++ b/uv/private/pep517_whl/tests/cc_deps/executable/sdist/mod.c @@ -0,0 +1,44 @@ +/* A hand-rolled CPython extension module. + * + * is resolved via the cc_deps include path, not from inside the sdist; + * dep_value() links through the cc_deps static archives (dep -> dep2) plus the + * [build_ext] libraries slot (-lextra against the build_clib-built libextra.a); + * and MOD_BONUS arrives as a transitively-propagated cc_deps -D define. + * group_entry() additionally proves that the -l entries keep their relative + * order through setuptools' [build_ext] libraries slot: the group archives form + * a one-pass cycle resolved by repeating -lgroup_a after -lgroup_b. + * Expected value: dep2 40 + extra 17 + MOD_BONUS 2 + group 125 == 184. */ +#include + +#include + +long group_entry(void); + +/* MOD_BONUS is injected by the `dep` cc_library's defines = ["MOD_BONUS=2"], + * which propagate transitively into this backend compile via cc_deps CPPFLAGS. + * Its absence means the -D define propagation broke, so fail the compile loudly + * rather than silently linking a wheel with the wrong value. */ +#ifndef MOD_BONUS +#error "MOD_BONUS not defined: cc_deps -D define propagation is broken" +#endif + +static PyObject *cc_deps_ext_value(PyObject *self, PyObject *args) { + (void)self; + (void)args; + return PyLong_FromLong(dep_value(7) + MOD_BONUS + group_entry()); +} + +static PyMethodDef cc_deps_ext_methods[] = { + {"value", cc_deps_ext_value, METH_NOARGS, + "Return dep_value() from the linked cc_library chain."}, + {NULL, NULL, 0, NULL}, +}; + +static struct PyModuleDef cc_deps_ext_module = { + PyModuleDef_HEAD_INIT, "cc_deps_ext", NULL, -1, cc_deps_ext_methods, + NULL, NULL, NULL, NULL, +}; + +PyMODINIT_FUNC PyInit_cc_deps_ext(void) { + return PyModule_Create(&cc_deps_ext_module); +} diff --git a/uv/private/pep517_whl/tests/cc_deps/executable/sdist/pyproject.toml b/uv/private/pep517_whl/tests/cc_deps/executable/sdist/pyproject.toml new file mode 100644 index 000000000..717276bfe --- /dev/null +++ b/uv/private/pep517_whl/tests/cc_deps/executable/sdist/pyproject.toml @@ -0,0 +1,7 @@ +[build-system] +requires = ["setuptools"] +build-backend = "setuptools.build_meta" + +[project] +name = "cc_deps_ext" +version = "0.0.1" diff --git a/uv/private/pep517_whl/tests/cc_deps/executable/sdist/setup.py b/uv/private/pep517_whl/tests/cc_deps/executable/sdist/setup.py new file mode 100644 index 000000000..e337b4fc2 --- /dev/null +++ b/uv/private/pep517_whl/tests/cc_deps/executable/sdist/setup.py @@ -0,0 +1,45 @@ +"""A minimal real-setuptools extension linking an out-of-tree cc_library. + +The extension source #includes , which is deliberately absent from this +sdist: it is resolved only through the cc_deps include path (CPPFLAGS), and the +dep_value symbol it calls is resolved only through the cc_deps static archives +that setuptools places in the post-object link slot ([build_ext] link_objects). +Neither Extension include_dirs nor Extension libraries are set on purpose. + +libextra.a, libgroup_a.a, and libgroup_b.a are built hermetically by build_clib +(with the same configured compiler, before build_ext runs). The group archives +contain a deliberate one-pass cycle, so they link only when -lgroup_a is repeated +after -lgroup_b (the documented repeat-the-library workaround), which exercises +-l order preservation through the [build_ext] libraries slot.""" + +from setuptools import Extension, setup +from setuptools.command.build_clib import build_clib + + +class quiet_build_clib(build_clib): + """Build the test archives but hide their names from build_ext's auto-link. + + distutils' build_ext.run() extends its libraries with + build_clib.get_library_names(), which would link these archives even without + the cc_deps user_link_flags under test. Returning no names keeps the archives + and library_dirs entry while leaving all -l naming and ordering exclusively + to the injected CcInfo stream.""" + + def get_library_names(self) -> list: + return [] + + +setup( + cmdclass={"build_clib": quiet_build_clib}, + libraries=[ + ("extra", {"sources": ["extra.c"]}), + ("group_a", {"sources": ["group_a_entry.c", "group_a_tail.c"]}), + ("group_b", {"sources": ["group_b.c"]}), + ], + ext_modules=[ + Extension( + name="cc_deps_ext", + sources=["mod.c"], + ), + ], +) diff --git a/uv/private/pep517_whl/tests/cc_deps/test.bzl b/uv/private/pep517_whl/tests/cc_deps/test.bzl new file mode 100644 index 000000000..ca07f19df --- /dev/null +++ b/uv/private/pep517_whl/tests/cc_deps/test.bzl @@ -0,0 +1,301 @@ +"""Analysis-test helpers for pep517_native_whl(cc_deps = ...).""" + +load("@bazel_skylib//lib:unittest.bzl", "analysistest", "asserts") +load("@rules_cc//cc/common:cc_common.bzl", "cc_common") +load("@rules_cc//cc/common:cc_info.bzl", "CcInfo") +load("//uv/private/pep517_whl:rule.bzl", "ALLOWED_LINK_FLAG_SHAPES", "ALLOWED_Z_KEYWORDS") + +# Golden copies of the accepted link-flag shapes and -z keywords. Deliberately +# duplicated: the acceptance assertions in the content test below cover each +# shape and keyword by exemplar, so an entry dropped from a rule.bzl tuple could +# quietly drop its coverage instead of turning a test red. Comparing the +# exported tuples against these copies at load time fails the package (naming +# both copies) the moment they diverge. +_ALLOWED_LINK_FLAG_SHAPES_GOLDEN = ( + ("exact", "-pthread"), + ("prefix", "-L"), + ("wl_arg", "-rpath"), + ("wl_arg", "-rpath-link"), + ("wl_arg", "--version-script"), + ("wl_keyword", "-z"), + ("wl_exact", "--enable-new-dtags"), +) + +_ALLOWED_Z_KEYWORDS_GOLDEN = ( + "relro", + "now", + "noexecstack", + "origin", +) + +def assert_allowlist_matches_golden(): + """Fail at load if rule.bzl's allowlist drifts from the golden copies here.""" + if tuple(ALLOWED_LINK_FLAG_SHAPES) != _ALLOWED_LINK_FLAG_SHAPES_GOLDEN: + fail( + "ALLOWED_LINK_FLAG_SHAPES in rule.bzl no longer matches the golden " + + "copy in test.bzl. Update both, and the acceptance exemplars below. " + + "rule.bzl has: {}".format(list(ALLOWED_LINK_FLAG_SHAPES)), + ) + if tuple(ALLOWED_Z_KEYWORDS) != _ALLOWED_Z_KEYWORDS_GOLDEN: + fail( + "ALLOWED_Z_KEYWORDS in rule.bzl no longer matches the golden copy " + + "in test.bzl. Update both, and the acceptance exemplars below. " + + "rule.bzl has: {}".format(list(ALLOWED_Z_KEYWORDS)), + ) + +# A plain cc_library cannot populate compilation_context.framework_includes on +# Linux (Apple `-F` search paths), so this shim synthesizes a CcInfo carrying +# only framework_includes. It lets the analysis content test pin the `-F` +# emission and marker-anchoring without a macOS toolchain. +def _cc_framework_shim_impl(ctx): + compilation_context = cc_common.create_compilation_context( + framework_includes = depset(ctx.attr.framework_includes), + ) + return [CcInfo(compilation_context = compilation_context)] + +cc_framework_shim = rule( + implementation = _cc_framework_shim_impl, + attrs = {"framework_includes": attr.string_list()}, +) + +def _cc_deps_test_impl(ctx): + env = analysistest.begin(ctx) + target = analysistest.target_under_test(env) + + build_actions = [a for a in target.actions if a.mnemonic == "PySdistNativeBuild"] + asserts.equals( + env, + 1, + len(build_actions), + "expected exactly one PySdistNativeBuild action", + ) + + action = build_actions[0] + args = action.argv + asserts.true( + env, + "--cc-deps-info" in args, + "action should carry --cc-deps-info; got: {}".format(args), + ) + + inputs = action.inputs.to_list() + info_path = args[args.index("--cc-deps-info") + 1] + input_paths = [f.path for f in inputs] + asserts.true( + env, + info_path in input_paths, + "cc-deps-info params file should be an action input", + ) + + # Every header and archive in the merged two-dep closure must reach the + # action inputs BY NAME, so a mutant that drops one of the three libraries + # fails rather than passing on the survivors (the previous "some .h / some + # .a" probes could not tell the closure apart from a single dep). + input_basenames = [f.basename for f in inputs] + for header in ("cc_dep.h", "chain_dep.h", "chain_dep2.h"): + asserts.true( + env, + header in input_basenames, + "cc_deps header {} should be an action input; got {}".format(header, input_basenames), + ) + + # Archives are matched by their lib. prefix (not exact basename) so the + # toolchain's pic vs non-pic archive choice does not matter; each of the + # three must be present exactly once. + for lib in ("libcc_dep.", "libchain_dep.", "libchain_dep2."): + matches = [b for b in input_basenames if b.startswith(lib) and b.endswith(".a")] + asserts.equals( + env, + 1, + len(matches), + "expected exactly one {}a archive among inputs; got {}".format(lib, input_basenames), + ) + + return analysistest.end(env) + +pep517_native_whl_cc_deps_test = analysistest.make(_cc_deps_test_impl) + +def _cc_deps_content_test_impl(ctx): + env = analysistest.begin(ctx) + target = analysistest.target_under_test(env) + + build_actions = [a for a in target.actions if a.mnemonic == "PySdistNativeBuild"] + asserts.equals( + env, + 1, + len(build_actions), + "expected exactly one PySdistNativeBuild action", + ) + args = build_actions[0].argv + marker = args[args.index("--execroot-marker") + 1] + + write_actions = [ + a + for a in target.actions + if a.mnemonic == "FileWrite" and a.outputs.to_list()[0].basename == "cc_deps_info.json" + ] + asserts.equals( + env, + 1, + len(write_actions), + "expected exactly one cc_deps_info.json write action", + ) + info = json.decode(write_actions[0].content) + + # Every include (and Apple framework) search path must be anchored so it + # survives the backend chdir. + for flag in info["compile_flags"]: + for prefix in ("-isystem", "-iquote", "-I", "-F"): + if flag.startswith(prefix): + asserts.true( + env, + flag[len(prefix):].startswith(marker + "/"), + "compile flag path should be marker-anchored: {}".format(flag), + ) + break + + # Positive: the cc_dep(includes = ["."]) source dir must be emitted as a + # marker-anchored include. Asserting a specific expected entry is PRESENT + # (not merely that any emitted entry is well-formed) makes a mutant that + # drops include emission fail rather than pass vacuously. Bazel 8 surfaces + # cc_library(includes) via system_includes (-isystem); Bazel 9 surfaces the + # same attribute via includes (-I). Accept either spelling of the same dir. + expected_system = "-isystem{}/{}".format(marker, ctx.label.package) + expected_plain = "-I{}/{}".format(marker, ctx.label.package) + asserts.true( + env, + expected_system in info["compile_flags"] or expected_plain in info["compile_flags"], + "cc_dep(includes=['.']) should emit {} or {}; got {}".format( + expected_system, + expected_plain, + info["compile_flags"], + ), + ) + + # Positive: chain_dep2's transitive define must reach compile_flags as a bare + # -D (no marker: defines carry symbols, not paths). Asserting the specific + # entry makes a mutant that drops -D emission fail rather than pass vacuously. + asserts.true( + env, + "-DCHAIN_BONUS=3" in info["compile_flags"], + "transitive define should emit -DCHAIN_BONUS=3; got {}".format(info["compile_flags"]), + ) + + # Positive: the framework shim's Apple `-F` search path must be emitted as a + # marker-anchored -F entry. A plain cc_library cannot produce + # framework_includes on Linux, so this is the only coverage that the -F + # emission and anchoring exist; behavioral macOS linking is out of reach on + # Linux CI. + expected_framework = "-F{}/vendor/Frameworks".format(marker) + asserts.true( + env, + expected_framework in info["compile_flags"], + "framework_includes should emit {}; got {}".format(expected_framework, info["compile_flags"]), + ) + + link_objects = info["link_objects"] + for obj in link_objects: + asserts.true( + env, + obj.startswith(marker + "/"), + "link object should be marker-anchored: {}".format(obj), + ) + + # The two-level chain must contribute both archives, dependent before + # dependency (topological order), alongside the second merged dep. + chain = [i for i, obj in enumerate(link_objects) if obj.split("/")[-1].startswith("libchain_dep.")] + chain2 = [i for i, obj in enumerate(link_objects) if obj.split("/")[-1].startswith("libchain_dep2.")] + merged = [i for i, obj in enumerate(link_objects) if obj.split("/")[-1].startswith("libcc_dep.")] + asserts.equals(env, 1, len(chain), "expected the direct chain archive; got {}".format(link_objects)) + asserts.equals(env, 1, len(chain2), "expected the transitive chain archive; got {}".format(link_objects)) + asserts.equals(env, 1, len(merged), "expected the second dep's archive; got {}".format(link_objects)) + asserts.true( + env, + chain[0] < chain2[0], + "dependent archive should precede its dependency; got {}".format(link_objects), + ) + + asserts.true( + env, + "chainfoo" in info["link_libraries"], + "-lchainfoo should land in link_libraries as a bare name; got {}".format(info["link_libraries"]), + ) + + # A declared additional_linker_inputs path inside a verbatim flag must be + # marker-anchored, keyed on the file's declared execroot-relative path. + # Covered in both the comma and the = spelling of --version-script. + input_paths = [f.path for f in build_actions[0].inputs.to_list()] + vs_paths = [p for p in input_paths if p.endswith("/vs.lds")] + asserts.equals(env, 1, len(vs_paths), "version script should be an action input") + for version_script in ( + "-Wl,--version-script,{}/{}".format(marker, vs_paths[0]), + "-Wl,--version-script={}/{}".format(marker, vs_paths[0]), + ): + asserts.true( + env, + version_script in info["link_flags"], + "declared version-script path should be marker-anchored in link_flags as {}; got {}".format(version_script, info["link_flags"]), + ) + + # Acceptance: one exemplar per allowed link-flag shape and directive form, + # added to chain_dep's linkopts, must reach link_flags verbatim. The -L and + # rpath dummy paths are fixture-relative and never linked (this fixture is + # analysis-only). -pthread is the exact shape; -Lvendor/dummy is the glued + # prefix; the -Wl, entries cover each argument directive in its comma and = + # spelling, the reviewed -z keywords, the benign multi-directive compound, + # and --enable-new-dtags comma-joined after an $ORIGIN rpath. + link_flags = info["link_flags"] + for exemplar in ( + "-Lvendor/dummy", + "-Wl,-rpath,vendor/rpath", + "-Wl,-rpath=vendor/rpath_eq", + "-Wl,-rpath-link,vendor/rpath_link", + "-Wl,-z,now", + "-Wl,-z,relro,-z,now", + "-Wl,-z,noexecstack", + "-Wl,-z,origin", + "-Wl,-rpath,$ORIGIN,--enable-new-dtags", + "-pthread", + ): + asserts.true( + env, + exemplar in link_flags, + "allowed link flag {} should land verbatim in link_flags; got {}".format(exemplar, link_flags), + ) + + # Relative order among the allowed flags follows linkopts order. + expected_order = [ + "-Wl,--version-script,{}/{}".format(marker, vs_paths[0]), + "-Wl,--version-script={}/{}".format(marker, vs_paths[0]), + "-Lvendor/dummy", + "-Wl,-rpath,vendor/rpath", + "-Wl,-rpath=vendor/rpath_eq", + "-Wl,-rpath-link,vendor/rpath_link", + "-Wl,-z,now", + "-Wl,-z,relro,-z,now", + "-Wl,-z,noexecstack", + "-Wl,-z,origin", + "-Wl,-rpath,$ORIGIN,--enable-new-dtags", + "-pthread", + ] + asserts.equals( + env, + expected_order, + [flag for flag in link_flags if flag in expected_order], + "allowed link flags should appear in linkopts order; got {}".format(link_flags), + ) + + return analysistest.end(env) + +pep517_native_whl_cc_deps_content_test = analysistest.make(_cc_deps_content_test_impl) + +def _cc_deps_failure_test_impl(ctx): + env = analysistest.begin(ctx) + asserts.expect_failure(env, ctx.attr.expected_message) + return analysistest.end(env) + +pep517_native_whl_cc_deps_failure_test = analysistest.make( + _cc_deps_failure_test_impl, + expect_failure = True, + attrs = {"expected_message": attr.string(mandatory = True)}, +) diff --git a/uv/private/sdist_build/attrs.bzl b/uv/private/sdist_build/attrs.bzl index a876d8033..fab160f81 100644 --- a/uv/private/sdist_build/attrs.bzl +++ b/uv/private/sdist_build/attrs.bzl @@ -9,6 +9,7 @@ def validate_build_attrs( pre_build_patch_strip, supported, toolchains, + cc_deps, error): """Fails when a configured source-build attribute is unsupported. @@ -21,6 +22,7 @@ def validate_build_attrs( pre_build_patch_strip: Strip count for pre-build patches. supported: Names of attributes consumed by the selected build path. toolchains: Toolchains used by the wheel-build action. + cc_deps: CcInfo targets whose headers and archives feed the native build. error: Failure message with one `{}` slot for unsupported names. """ active = [] @@ -38,6 +40,8 @@ def validate_build_attrs( active.append("pre_build_patch_strip") if toolchains: active.append("toolchains") + if cc_deps: + active.append("cc_deps") unsupported = [name for name in active if name not in supported] if unsupported: fail(error.format(", ".join(unsupported))) diff --git a/uv/private/sdist_build/repository.bzl b/uv/private/sdist_build/repository.bzl index 8257f6689..0a2dc336f 100644 --- a/uv/private/sdist_build/repository.bzl +++ b/uv/private/sdist_build/repository.bzl @@ -194,6 +194,7 @@ def _sdist_build_impl(repository_ctx): "resource_set", ], toolchains = repository_ctx.attr.extra_toolchains, + cc_deps = repository_ctx.attr.extra_cc_deps, ) # Resolve additional deps discovered by the configure tool @@ -227,9 +228,11 @@ def _sdist_build_impl(repository_ctx): # AR, LD, and STRIP make variables can be synthetic. Only forward explicit # toolchains/env for JDK, Rust, and other package-specific overrides. toolchain_attrs = "" + cc_deps_attr = "" if is_native: toolchains = repository_ctx.attr.extra_toolchains extra_env = repository_ctx.attr.extra_env + cc_deps = repository_ctx.attr.extra_cc_deps env_attr = "" if extra_env: env_attr = """ @@ -246,6 +249,13 @@ def _sdist_build_impl(repository_ctx): toolchains = "\n".join([" \"{}\",".format(t) for t in toolchains]), ) toolchain_attrs += env_attr + if cc_deps: + cc_deps_attr = """ + cc_deps = [ +{cc_deps} + ],""".format( + cc_deps = "\n".join([" \"{}\",".format(d) for d in cc_deps]), + ) resource_set_attr = "" if repository_ctx.attr.resource_set != "default": @@ -272,7 +282,7 @@ py_binary( name = "whl", src = "{src}", tool = ":build_tool", - version = "{version}",{console_scripts_attr}{monitor_memory_attr}{resource_set_attr}{patch_attrs}{toolchain_attrs} + version = "{version}",{console_scripts_attr}{monitor_memory_attr}{resource_set_attr}{patch_attrs}{toolchain_attrs}{cc_deps_attr} visibility = ["//visibility:public"], ) @@ -290,6 +300,7 @@ exports_files( resource_set_attr = resource_set_attr, patch_attrs = patch_attrs, toolchain_attrs = toolchain_attrs, + cc_deps_attr = cc_deps_attr, )) sdist_build = repository_rule( @@ -331,5 +342,9 @@ sdist_build = repository_rule( default = {}, doc = "Environment variables forwarded to the generated pep517_native_whl(...) `env` dict. Values may reference $(VAR) make-variables from extra toolchains. Prefix an execroot-relative path with `$(EXECROOT)/` so it remains valid after the backend changes into the unpacked source tree. Set via `uv.override_package(env = {...})`.", ), + "extra_cc_deps": attr.string_list( + default = [], + doc = "CcInfo target labels forwarded to the generated pep517_native_whl(...) `cc_deps` list, wiring their transitive headers and static archives into the native sdist build. Set via `uv.override_package(cc_deps = [...])`.", + ), }, )