Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
122 changes: 116 additions & 6 deletions docs/uv-patching.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,9 @@ Additionally, `extra_deps` and `extra_data` allow adding dependencies or data
files to the generated `py_library` target for a package.
`console_scripts` overrides the complete script map for a wheel built from an
sdist when its egg-info metadata is absent or unsuitable. An explicit
empty map suppresses all detected scripts.
empty map suppresses all detected scripts. For a native
extension built from an sdist, `cc_deps` wires Bazel `cc_library` targets
(their headers and static archives) into the build.

## Prerequisites

Expand Down Expand Up @@ -126,6 +128,54 @@ uv.override_package(
)
```

### Linking native C/C++ dependencies

A package with a native extension often needs a C/C++ library: its headers to
compile against and its static archive to link. `cc_deps` wires a Bazel target
that provides `CcInfo` (a `cc_library`, `cc_import`, or similar) directly into
the sdist build: the dependency's transitive headers, include paths, defines,
and Apple framework search paths become compile flags (appended to `CPPFLAGS`),
and its static archives are placed in the linker's post-object slot. The
transitive closure and link order come from `CcInfo`, so you name only the
top-level target.

`cc_deps` is the declarative counterpart to the `env` / `toolchains` escape hatch
(see [the constraints below](#constraints)): reach for `cc_deps` to _declare_ the
native dependency, and keep `env` for _tweaking_ the build: package-specific
defines, exotic linker flags, or anything `cc_deps` cannot model. The two
compose; `cc_deps` flags are appended after any you set in `env`.

For example, building a package's native extension against an in-repo C
library. Before, with the raw `env` / `toolchains` escape hatch, the include
and archive paths are anchored by hand with `$(EXECROOT)` and fed through
make-variables a toolchain exports:

```starlark
uv.override_package(
name = "native-package",
toolchains = ["//third_party/mylib:make_vars"], # exports $(MYLIB_INC), $(MYLIB_LIB_A)
env = {
"CPPFLAGS": "-I$(EXECROOT)/$(MYLIB_INC)",
"LDFLAGS": "$(EXECROOT)/$(MYLIB_LIB_A)",
},
)
```

After, with `cc_deps`, the include path, the archive, and their transitive
closure are read from the target's `CcInfo`, and no path anchoring is needed:

```starlark
uv.override_package(
name = "native-package",
cc_deps = ["//third_party/mylib"], # a cc_library / cc_import
)
```

The dependency target must be visible to the generated build repository, so mark
it `//visibility:public` (or grant that repository's package visibility). See the
[constraints below](#constraints) for the supported-library and setuptools
requirements.

### Reserving wheel build resources

Native sdist builds can be memory-hungry. Without a hint, Bazel assumes the
Expand Down Expand Up @@ -217,18 +267,78 @@ uv.override_package(
removes NumPy's bundled tests without retaining their compiled bytecode.
Removing the complete `.dist-info` directory, `METADATA`, or `RECORD` is
unsupported.
- `pre_build_patches`, `toolchains`, `env`, `monitor_memory`, and non-default
`resource_set` values require a source distribution. An override that applies
them to a wheel-only lock record is rejected.
- Generated pure-Python builds reject `toolchains` and `env`; those attributes
augment the native build toolchain and environment.
- `pre_build_patches`, `toolchains`, `env`, `cc_deps`, `monitor_memory`, and
non-default `resource_set` values require a source distribution. An override
that applies them to a wheel-only lock record is rejected.
- Generated pure-Python builds reject `toolchains`, `env`, and `cc_deps`; those
attributes augment the native build toolchain, environment, and link inputs.
- Native build `env` values can use `$(EXECROOT)/` to anchor paths supplied by
a toolchain, for example `CPPFLAGS = "-I$(EXECROOT)/$(DEP_INC)"` and
`LDFLAGS = "$(EXECROOT)/$(DEP_LIB_A)"`. The anchor remains valid after the
PEP 517 backend changes into the unpacked source tree.
- Native builds select the configured C++ compiler, archiver, linker, and strip
tools by default. Explicit `CC`, `CXX`, `AR`, `LD`, and `STRIP` values in
`env` override those selections.
- `cc_deps` applies only to sdists built by the setuptools backend. A package
that declares any other `[build-system].build-backend` is rejected when the
wheel is built (`cc_deps is only supported with the setuptools build backend`)
rather than having its inputs silently dropped.
- The build environment's setuptools must be `>= 65.4.0`, the release that
added `DIST_EXTRA_CONFIG`, the channel `cc_deps` routes the link inputs
through. An older or missing setuptools fails the build with
`cc_deps requires setuptools >= 65.4.0`; bump it in the lock that supplies your
build dependencies (`uv.lock` / `default_build_dependencies`).
- Only static (or PIC-static) archives are linked. A dependency that provides
only a shared/dynamic library fails at analysis time, as does an `alwayslink`
(whole-archive) library; neither is supported.
- The linked archives must contain position-independent (PIC) objects, because
they are folded into the extension's shared object. A toolchain that emits
non-PIC objects into its static archives (some GCC configurations) fails the
final link with relocation errors such as `relocation R_X86_64_32 against ...
can not be used when making a shared object; recompile with -fPIC`. Remedies:
use a toolchain that compiles PIC objects (the default on macOS, and clang/LLVM
on Linux), build with `--force_pic`, or add `copts = ["-fPIC"]` to the
`cc_library`.
- Each `cc_deps` label is referenced from the generated external build
repository, so the target must be visible to it: use `//visibility:public` or
grant that repository's package visibility.
- Link flags that reference a file the dependency declares via
`additional_linker_inputs` (for example a `-Wl,--version-script,...` linker
script) are path-anchored automatically so they survive the backend changing
directory. A relative path written directly into `linkopts` without declaring
the file there is not anchored and will not resolve after the change.
- `cc_deps` flattens a dependency's link inputs into setuptools' two link slots:
full-path static archives go to the post-object `[build_ext] link_objects`
slot in topological order; bare `-l<name>` entries go to the post-object
`[build_ext] libraries` slot preserving their relative order; and every other
link flag is appended to `LDFLAGS` ahead of the objects. Because the archives
and `-l` entries land in separate slots, the order between an `-l<name>` entry
and a non-`-l` flag cannot be preserved, so only flags whose effect does not
depend on that relative order are passed through. `cc_deps` accepts a fixed set
of link-flag shapes: `-L<dir>` search paths; `-pthread`; and `-Wl,` tokens
built from these directives: the rpath family (`-rpath`, `-rpath=`, and
`-rpath-link`), `--version-script` (comma and `=` argument forms), `-z` with
one of the reviewed keywords `relro`, `now`, `noexecstack`, or `origin` (each
a global link mode; the wider `-z` namespace includes position-sensitive
keywords, so others are rejected), and `--enable-new-dtags`. A comma-joined
`-Wl,` token is validated directive by directive, so an accepted leading
directive cannot smuggle a rejected one behind it (`-Wl,-rpath,/x,--as-needed`
fails, naming `--as-needed`), while benign compounds such as
`-Wl,-z,relro,-z,now` and `-Wl,-rpath,$ORIGIN,--enable-new-dtags` pass. Any
other link flag, including grouping and linker-state toggles such as
`--start-group`/`--end-group` or `--as-needed`, is rejected at analysis time
rather than silently reordered. The split `-L <dir>` form is rejected too;
write the glued `-L<dir>`. Apple `-framework` linking is not supported in v1:
ld64 resolves frameworks in command-line order alongside `-l` entries, so the
two-slot split cannot hold one; set the framework in the override's `env`
`LDFLAGS` or patch it in with `pre_build_patches`. Three escape hatches cover
what the allowlist does not: to resolve an archive cycle that would otherwise
need `--start-group`, repeat the library name (for example `-la -lb -la`),
since `-l` order is preserved within the libraries slot; to apply a global
toggle such as `--as-needed` to the whole link, set it in the override's
`env` `LDFLAGS`, which lands ahead of the objects; and for anything else,
patch it in with `pre_build_patches` on the sdist. The accepted set can be
extended upstream on request.
- Post-install patches to prebuilt wheels must preserve every retained original
path used for collision and regular-package merge planning, including its
file-or-directory kind and package classification. Ordinary added paths are
Expand Down
11 changes: 11 additions & 0 deletions e2e/cases/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -168,6 +168,17 @@ write_source_files(
# pep517_native_whl(...) call so the reservation reaches the action.
"snapshots/sdist_build.uv_sdist_native_build.python_geohash.BUILD.bazel": "@sdist_build__uv_sdist_native_build__python_geohash__0_8_5//:BUILD.bazel",

# @sdist_build for python-geohash with uv.override_package(cc_deps)
# ----------------------------------------------------------------------
# Covers the override_package -> repo-rule -> BUILD-template plumbing
# for `cc_deps`. Reuses python-geohash's locked sdist under a distinct
# project/hub (//uv-sdist-cc-deps) so the three sibling sdist_build
# snapshots stay byte-stable. Pins that the CcInfo label threads onto
# the generated pep517_native_whl(...) `cc_deps` list. The executable
# end-to-end regression lives in-repo at
# //uv/private/pep517_whl/tests/cc_deps.
"snapshots/sdist_build.uv_sdist_cc_deps.python_geohash.BUILD.bazel": "@sdist_build__uv_sdist_cc_deps__python_geohash__0_8_5//:BUILD.bazel",

# Venv site-packages .pth files. Pin the line shapes that
# assemble_venv emits, so any change to `.pth` emission
# surfaces here. Wheel site-packages dirs are referenced by
Expand Down
1 change: 1 addition & 0 deletions e2e/cases/MODULE.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -153,6 +153,7 @@ include("//uv-platform-filter-844:setup.MODULE.bazel")
include("//uv-plus-version:setup.MODULE.bazel")
include("//uv-pyproject-cases:setup.MODULE.bazel")
include("//uv-requirements-bzl:setup.MODULE.bazel")
include("//uv-sdist-cc-deps:setup.MODULE.bazel")
include("//uv-sdist-fallback:setup.MODULE.bazel")
include("//uv-sdist-jdk-build:setup.MODULE.bazel")
include("//uv-sdist-mpicc:setup.MODULE.bazel")
Expand Down

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

25 changes: 25 additions & 0 deletions e2e/cases/uv-sdist-cc-deps/BUILD.bazel
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# Snapshot regression for `uv.override_package(cc_deps = [...])`.
#
# Reuses python-geohash's locked sdist (already fetched by the
# uv-sdist-native-build case) under a distinct project name/hub so the
# generated sdist_build repo is independent. The override attaches `:probe`
# via `cc_deps`; the snapshot in //:snapshots pins that the label threads
# onto the generated pep517_native_whl(...) `cc_deps` list.
#
# Snapshot-only: the wheel is not built here. The executable cc_deps
# regression (real setuptools extension linking a cc_library) lives in-repo
# at //uv/private/pep517_whl/tests/cc_deps.

load("@rules_cc//cc:defs.bzl", "cc_library")

cc_library(
name = "probe",
srcs = ["probe.cc"],
hdrs = ["probe.h"],
includes = ["."],
# The generated @sdist_build__uv_sdist_cc_deps__python_geohash__0_8_5
# repo references this label from its pep517_native_whl(cc_deps = [...])
# call, so it must be visible outside this package. Public matches how
# sibling cases expose files to generated repos (exports_files default).
visibility = ["//visibility:public"],
)
3 changes: 3 additions & 0 deletions e2e/cases/uv-sdist-cc-deps/probe.cc
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
#include "probe.h"

int rules_py_cc_deps_probe(void) { return 42; }
16 changes: 16 additions & 0 deletions e2e/cases/uv-sdist-cc-deps/probe.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
#ifndef UV_SDIST_CC_DEPS_PROBE_H_
#define UV_SDIST_CC_DEPS_PROBE_H_

#ifdef __cplusplus
extern "C" {
#endif

// Returns a constant probe value. Present only so the `cc_deps` override has a
// real CcInfo target to reference; the snapshot never builds this library.
int rules_py_cc_deps_probe(void);

#ifdef __cplusplus
}
#endif

#endif // UV_SDIST_CC_DEPS_PROBE_H_
7 changes: 7 additions & 0 deletions e2e/cases/uv-sdist-cc-deps/pyproject.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
[project]
name = "uv-sdist-cc-deps"
version = "0.0.0"
requires-python = ">=3.11"
dependencies = [
"python-geohash",
]
37 changes: 37 additions & 0 deletions e2e/cases/uv-sdist-cc-deps/setup.MODULE.bazel
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
"""Regression for the per-package `cc_deps` plumbing on
`uv.override_package`. Reuses python-geohash's locked sdist (already
fetched by the uv-sdist-native-build case) under a distinct project
name/hub, so the generated sdist_build repo is independent and the three
existing sdist_build snapshots stay byte-stable.

The override attaches an in-repo cc_library via `cc_deps`; the snapshot
pins that the label threads onto the generated pep517_native_whl(...)
`cc_deps` list. Snapshot-only: the wheel is not built here; the
executable cc_deps regression lives in-repo under
//uv/private/pep517_whl/tests/cc_deps.
"""

uv = use_extension("@aspect_rules_py//uv:extensions.bzl", "uv")
uv.declare_hub(hub_name = "pypi_uv_sdist_cc_deps")
uv.project(
default_build_dependencies = [
"build",
"setuptools",
],
hub_name = "pypi_uv_sdist_cc_deps",
lock = "//uv-sdist-cc-deps:uv.lock",
pyproject = "//uv-sdist-cc-deps:pyproject.toml",
)

# Attach an in-repo CcInfo target via `cc_deps`. This is additive on top of
# whatever sdist_build's BUILD template emits by default.
uv.override_package(
name = "python-geohash",
cc_deps = ["//uv-sdist-cc-deps:probe"],
lock = "//uv-sdist-cc-deps:uv.lock",
)

# Exposed so e2e/BUILD.bazel can snapshot the generated pep517_native_whl
# call: pins that `cc_deps` threads onto the build rule. The hub repo is
# deliberately not imported; no target here consumes it.
use_repo(uv, "sdist_build__uv_sdist_cc_deps__python_geohash__0_8_5")
70 changes: 70 additions & 0 deletions e2e/cases/uv-sdist-cc-deps/uv.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading