From 166e89de2c575909ec1a54aa47f8d571e9609d25 Mon Sep 17 00:00:00 2001 From: Connor McEntee Date: Mon, 27 Jul 2026 20:13:02 +0000 Subject: [PATCH 1/5] feat(uv): flatten cc_deps CcInfo into pep517_native_whl build inputs Add a `cc_deps` label_list(providers=[[CcInfo]]) to pep517_native_whl that merges the deps' CcInfo and writes a structured params file (compile_flags, link_objects, link_libraries, link_flags) handed to the build helper via --cc-deps-info. Compile flags derive -I/-iquote/-isystem from the compilation context includes (with -isystem for external_includes and -F for framework_includes where Bazel provides those fields) and -D from its defines; link objects take each linker input's pic_static_library or static_library in topological order. Every emitted path is anchored with the execroot marker, and paths of files a linker input declares via additional_inputs (version scripts and similar) are re-anchored inside its link flags, so both survive the backend changing into the unpacked source tree. Dynamic-only and alwayslink libraries fail at analysis with an actionable message. When cc_deps is empty the action is byte-identical to before, so existing native builds are unchanged. Link flags are fail-closed. cc_deps splits a dependency's link inputs into pre-object LDFLAGS and the post-object [build_ext] slots, which cannot preserve the order between an -l entry and a neighboring flag, so only flags whose effect does not depend on that order may ride along. A denylist against the open set of linker flags fails open (any flag we did not enumerate would be reordered silently), so the contract is an allowlist of flag shapes (ALLOWED_LINK_FLAG_SHAPES). Bare tokens match by exact token (-pthread) or glued prefix (-L). -Wl, tokens are walked directive by directive, consuming comma or = arguments, so an allowed leading directive cannot smuggle an order-sensitive one behind it (-Wl,-rpath,/x,--as-needed is rejected, naming --as-needed) while benign compounds like -Wl,-z,relro,-z,now keep working. Allowed directives: the rpath family, --version-script, -z with a reviewed keyword (relro, now, noexecstack, origin; the -z namespace is open and holds position-sensitive keywords like Solaris -z allextract, so keywords are reviewed individually), and --enable-new-dtags. -framework is rejected: ld64 resolves frameworks in command-line order alongside -l entries, so neither slot can hold one faithfully. Everything else is rejected at analysis with one message that names the dep, the flag, and the offending directive, and points at the three escape hatches: repeat the library name for archive cycles, env LDFLAGS for global toggles, pre_build_patches for anything else. Default-reject can be widened later without breaking anyone; default-accept could never be tightened. Add analysis tests under tests/cc_deps: a merged two-level chain plus a second direct dep is accepted, its headers, archives, and --cc-deps-info reach the action, and the decoded params content proves marker anchoring, topological archive order, the -l partition, transitive -D emission, and declared-path anchoring in link flags; a shared-only cc_import and an alwayslink cc_library each fail with the expected message. The link-flag contract is pinned by acceptance exemplars (every shape, directive form, and -z keyword, asserted verbatim and in order) and rejection tests (an unknown flag, a bare unwrapped token, a comma-smuggled directive, -framework, and an unreviewed -z keyword), with a golden copy of the allowlist and keyword set that fails the test package at load time if either drifts. --- uv/private/pep517_whl/BUILD.bazel | 1 - uv/private/pep517_whl/rule.bzl | 286 +++++++++++- .../pep517_whl/tests/cc_deps/BUILD.bazel | 422 ++++++++++++++++++ uv/private/pep517_whl/tests/cc_deps/test.bzl | 301 +++++++++++++ 4 files changed, 1006 insertions(+), 4 deletions(-) create mode 100644 uv/private/pep517_whl/tests/cc_deps/BUILD.bazel create mode 100644 uv/private/pep517_whl/tests/cc_deps/test.bzl diff --git a/uv/private/pep517_whl/BUILD.bazel b/uv/private/pep517_whl/BUILD.bazel index 423988ed3..22b0a68de 100644 --- a/uv/private/pep517_whl/BUILD.bazel +++ b/uv/private/pep517_whl/BUILD.bazel @@ -97,7 +97,6 @@ bzl_library( srcs = ["test.bzl"], visibility = ["//uv:__subpackages__"], deps = [ - ":rule", "//uv/private:source_built_wheel", "@bazel_skylib//lib:unittest", ], diff --git a/uv/private/pep517_whl/rule.bzl b/uv/private/pep517_whl/rule.bzl index 24db06c4a..db25ccd2c 100644 --- a/uv/private/pep517_whl/rule.bzl +++ b/uv/private/pep517_whl/rule.bzl @@ -8,6 +8,7 @@ build backend the sdist declares in its `[build-system]` table. load("@bazel_lib//lib:resource_sets.bzl", "resource_set", "resource_set_attr") load("@rules_cc//cc:action_names.bzl", "ACTION_NAMES") load("@rules_cc//cc/common:cc_common.bzl", "cc_common") +load("@rules_cc//cc/common:cc_info.bzl", "CcInfo") load("//py/private/toolchain:types.bzl", "NATIVE_BUILD_TOOLCHAIN", "PY_TOOLCHAIN") load("//uv/private:source_built_wheel.bzl", "SourceBuiltWheelInfo") @@ -156,6 +157,264 @@ def _cc_toolchain_inputs_and_tools(ctx): infer_cxx = infer_cxx or tools.get("CXX") == tools.get("CC") return files, {key: value for key, value in tools.items() if value}, infer_cxx +def _library_display_name(lib): + for artifact in (lib.static_library, lib.pic_static_library, lib.dynamic_library, lib.interface_library): + if artifact: + return artifact.basename + return "" + +# cc_deps flattens a dependency's link inputs into setuptools' two slots: static +# archives and `-l` entries go to the post-object [build_ext] slots, and +# every other link flag rides along in pre-object LDFLAGS. That split cannot +# preserve the relative order between an -l entry and a neighboring flag, so only +# flags whose effect does not depend on that order are safe to pass through. +# +# This is an allowlist of flag SHAPES, not a denylist of known-bad flags. A +# denylist against the open set of linker flags fails open: any order-sensitive +# flag we did not think to enumerate would be reordered silently into a stream +# that no longer links what the user wrote. The denylist this replaced had to +# grow twice under review for exactly that reason. An allowlist fails closed: an +# unrecognized flag is rejected at analysis, and the set can be widened later +# without breaking anyone, whereas a denylist can never be tightened. +# +# Each entry is (kind, token): +# "exact" the whole flag must equal token (e.g. -pthread) +# "prefix" the flag must start with token and carry a glued argument +# (e.g. -Ldir) +# "wl_arg" a -Wl, directive that takes an argument, either as the next +# comma segment (-Wl,-rpath,) or glued with = +# (-Wl,-rpath=) +# "wl_keyword" a -Wl, directive whose argument (next comma segment) must be +# one of the reviewed keywords in ALLOWED_Z_KEYWORDS +# (-Wl,-z,relro) +# "wl_exact" a -Wl, directive that stands alone (-Wl,--enable-new-dtags) +# +# -Wl, hands the linker a comma-joined list of directives, so a token whose +# leading directive is allowed could otherwise smuggle an order-sensitive +# directive behind it (-Wl,-rpath,/x,--as-needed). -Wl, tokens are therefore +# walked directive by directive and every directive must be allowed, which also +# keeps benign compounds like -Wl,-z,relro,-z,now working. +# +# -framework is deliberately absent: ld64 resolves frameworks in command-line +# order alongside -l entries (a framework is a library input, not a flag), so +# neither setuptools slot can hold one without reordering library resolution. +# +# Exported so the cc_deps test package pins it against a golden copy; a shape +# dropped here would otherwise drop its acceptance coverage silently. +ALLOWED_LINK_FLAG_SHAPES = ( + ("exact", "-pthread"), + ("prefix", "-L"), + ("wl_arg", "-rpath"), + ("wl_arg", "-rpath-link"), + ("wl_arg", "--version-script"), + ("wl_keyword", "-z"), + # --enable-new-dtags selects the global ELF dtags mode (DT_RUNPATH over + # DT_RPATH); it is position-insensitive and commonly comma-joined after an + # $ORIGIN rpath. + ("wl_exact", "--enable-new-dtags"), +) + +# Accepted -z keywords, each a global link mode. The -z namespace is open and +# contains position-sensitive keywords on some linkers (Solaris -z allextract +# toggles whole-archive extraction for the archives that follow it), so +# accepting the whole class would reintroduce the fail-open shape this +# allowlist exists to close. Keywords are reviewed individually; the set can +# be extended upstream on request. Golden-pinned by the cc_deps test package +# alongside the shapes. +ALLOWED_Z_KEYWORDS = ( + "relro", + "now", + "noexecstack", + "origin", +) + +_WL_ARG_DIRECTIVES = {token: True for kind, token in ALLOWED_LINK_FLAG_SHAPES if kind == "wl_arg"} +_WL_KEYWORD_DIRECTIVES = {token: True for kind, token in ALLOWED_LINK_FLAG_SHAPES if kind == "wl_keyword"} +_WL_EXACT_DIRECTIVES = {token: True for kind, token in ALLOWED_LINK_FLAG_SHAPES if kind == "wl_exact"} +_ALLOWED_Z_KEYWORDS = {keyword: True for keyword in ALLOWED_Z_KEYWORDS} + +def _link_flag_allowed(flag): + """Whether a bare (non-`-l`, non-`-Wl,`) flag matches an allowed shape.""" + for kind, token in ALLOWED_LINK_FLAG_SHAPES: + if kind == "exact" and flag == token: + return True + if kind == "prefix" and flag.startswith(token) and len(flag) > len(token): + return True + return False + +def _check_wl_link_flag(owner, flag): + """Validate every directive in a comma-joined `-Wl,` token, or fail. + + Walks the comma segments as directive/argument pairs: an allowed argument + directive consumes the next segment (or embeds its argument after `=`), a + keyword directive consumes the next segment and checks it against the + reviewed keyword set, and every directive in the token must be allowed. + Validating only the leading directive would let it smuggle an + order-sensitive directive behind it. + """ + segments = flag.split(",")[1:] + expect_arg = False + keyword_directive = None + last_directive = None + for segment in segments: + if expect_arg: + expect_arg = False + if keyword_directive != None and segment not in _ALLOWED_Z_KEYWORDS: + _reject_link_flag(owner, flag, "{} {}".format(keyword_directive, segment)) + keyword_directive = None + continue + last_directive = segment + if segment in _WL_EXACT_DIRECTIVES: + continue + if segment in _WL_KEYWORD_DIRECTIVES: + expect_arg = True + keyword_directive = segment + continue + if segment in _WL_ARG_DIRECTIVES: + expect_arg = True + continue + eq = segment.find("=") + if eq > 0 and eq < len(segment) - 1 and segment[:eq] in _WL_ARG_DIRECTIVES: + continue + _reject_link_flag(owner, flag, segment) + if expect_arg: + # A trailing argument directive with nothing to consume is malformed. + _reject_link_flag(owner, flag, last_directive) + +def _reject_link_flag(owner, flag, directive = None): + if directive == None: + offense = "passes the link flag {}, which is not one of the link-flag shapes cc_deps accepts".format(flag) + else: + offense = "passes the link flag {}; its directive {} is not one of the link-flag shapes cc_deps accepts".format(flag, directive) + fail(("cc_deps dependency {} {}. cc_deps splits a dependency's link " + + "inputs into pre-object LDFLAGS and the post-object [build_ext] libraries " + + "slot, so only position-insensitive flags can ride along. For an archive " + + "cycle, repeat the library name instead (e.g. -la -lb -la); order among " + + "-l entries is preserved. For a global toggle such as --as-needed, set it " + + "in the override's env LDFLAGS. For anything else, apply it with " + + "pre_build_patches on the sdist. The accepted shapes can be extended " + + "upstream on request.").format(owner, offense)) + +def _anchor_declared_paths(flag, declared_paths): + """Marker-anchor any declared additional_input path appearing in `flag`. + + Substitution goes through per-path placeholders, longest path first, so a + declared path that contains another declared path is never anchored twice. + """ + for index, path in enumerate(declared_paths): + flag = flag.replace(path, "\v{}\v".format(index)) + for index, path in enumerate(declared_paths): + flag = flag.replace("\v{}\v".format(index), "{}/{}".format(_EXECROOT_MARKER, path)) + return flag + +def _cc_deps_args_and_inputs(ctx): + """Flatten `cc_deps` CcInfo into a compile/link params file for the backend. + + Returns `(args, direct_inputs, transitive_inputs)`. When `cc_deps` is empty + all three are empty, so the action stays byte-identical to the no-cc_deps + path. Every emitted path is prefixed with `_EXECROOT_MARKER`: only + execroot-relative paths exist at analysis time, and build_helper substitutes + the real execroot before the backend changes into the unpacked source tree. + Slot information (post-object archives vs `-l` libraries vs order-insensitive + flags) is kept distinct in the JSON schema because flattening it into `env` + would lose it. + """ + if not ctx.attr.cc_deps: + return [], [], [] + + cc_info = cc_common.merge_cc_infos( + cc_infos = [dep[CcInfo] for dep in ctx.attr.cc_deps], + ) + compilation_context = cc_info.compilation_context + + compile_flags = [] + for include in compilation_context.includes.to_list(): + compile_flags.append("-I{}/{}".format(_EXECROOT_MARKER, include)) + for include in compilation_context.quote_includes.to_list(): + compile_flags.append("-iquote{}/{}".format(_EXECROOT_MARKER, include)) + for include in compilation_context.system_includes.to_list(): + compile_flags.append("-isystem{}/{}".format(_EXECROOT_MARKER, include)) + + # external_includes carries `-isystem` semantics; older Bazel lacks the field. + external_includes = getattr(compilation_context, "external_includes", None) + if external_includes: + for include in external_includes.to_list(): + compile_flags.append("-isystem{}/{}".format(_EXECROOT_MARKER, include)) + + # framework_includes carries Apple `-F` search paths; older Bazel lacks the field. + framework_includes = getattr(compilation_context, "framework_includes", None) + if framework_includes: + for framework in framework_includes.to_list(): + compile_flags.append("-F{}/{}".format(_EXECROOT_MARKER, framework)) + for define in compilation_context.defines.to_list(): + compile_flags.append("-D{}".format(define)) + + link_objects = [] + link_libraries = [] + link_flags = [] + archives = [] + additional_inputs = [] + for linker_input in cc_info.linking_context.linker_inputs.to_list(): + for lib in linker_input.libraries: + if lib.alwayslink: + fail(("cc_deps dependency {} contains alwayslink library {}; " + + "whole-archive linking is not supported.").format( + linker_input.owner, + _library_display_name(lib), + )) + archive = lib.pic_static_library or lib.static_library + if not archive: + fail(("cc_deps dependency {} provides no static archive (only a " + + "shared/dynamic library); the PEP 517 native build links " + + "static archives only. Provide a static or PIC-static " + + "library.").format(linker_input.owner)) + link_objects.append("{}/{}".format(_EXECROOT_MARKER, archive.path)) + archives.append(archive) + + # `-l` from the dep's own linkopts routes to setuptools' libraries + # slot; every other link flag must match an allowed shape (see + # ALLOWED_LINK_FLAG_SHAPES) or the build is rejected at analysis. Paths of + # files this linker_input declares via additional_inputs (e.g. + # `-Wl,--version-script,$(location ...)`) are marker-anchored so they + # survive the backend chdir. + declared_paths = sorted( + [f.path for f in linker_input.additional_inputs], + key = len, + reverse = True, + ) + for flag in linker_input.user_link_flags: + # The name carries no declared path, so -l routing precedes anchoring. + if flag.startswith("-l") and len(flag) > 2: + link_libraries.append(flag[len("-l"):]) + continue + + # -Wl, tokens are validated per directive so an allowed leading + # directive cannot smuggle an order-sensitive one behind it. + if flag.startswith("-Wl,"): + _check_wl_link_flag(linker_input.owner, flag) + elif not _link_flag_allowed(flag): + _reject_link_flag(linker_input.owner, flag) + + link_flags.append(_anchor_declared_paths(flag, declared_paths)) + additional_inputs.extend(linker_input.additional_inputs) + + params = ctx.actions.declare_file("cc_deps_info.json") + ctx.actions.write( + output = params, + content = json.encode({ + "compile_flags": compile_flags, + "link_objects": link_objects, + "link_libraries": link_libraries, + "link_flags": link_flags, + }), + ) + + return ( + ["--cc-deps-info", params.path], + [params] + additional_inputs, + [compilation_context.headers, depset(archives)], + ) + def _pep517_whl(ctx): archive = ctx.file.src wheel_dir = ctx.actions.declare_directory("whl") @@ -213,20 +472,22 @@ def _pep517_native_whl(ctx): if "CXX" not in ctx.attr.env and cc_tools.get("CXX") and infer_cxx: env[_INFER_CXX_COMPANION] = "1" + cc_deps_args, cc_deps_direct, cc_deps_transitive = _cc_deps_args_and_inputs(ctx) + ctx.actions.run( mnemonic = "PySdistNativeBuild", progress_message = "Native source compiling {} to a whl".format(archive.basename), executable = ctx.executable.tool, toolchain = None, - arguments = ctx.attr.args + patch_args + _memory_args(ctx) + [ + arguments = ctx.attr.args + patch_args + _memory_args(ctx) + cc_deps_args + [ "--execroot-marker", _EXECROOT_MARKER, archive.path, wheel_dir.path, ], inputs = depset( - [archive] + patch_inputs, - transitive = extra_inputs, + [archive] + patch_inputs + cc_deps_direct, + transitive = extra_inputs + cc_deps_transitive, ), tools = [ctx.attr.tool[DefaultInfo].files_to_run], outputs = [wheel_dir], @@ -305,6 +566,25 @@ constraints of the target platform. """, attrs = _pep517_whl_attrs | { "args": attr.string_list(), + "cc_deps": attr.label_list( + providers = [[CcInfo]], + doc = "C++ libraries whose headers and static archives are made " + + "available to the build backend. Each target's `CcInfo` " + + "compilation context (include paths and defines) and linking " + + "context (static archives and link flags) are flattened into a " + + "params file consumed by the backend; execroot-relative paths " + + "are anchored with an internal execroot marker so they survive " + + "the backend changing into the unpacked source tree. Link flags " + + "referencing files a dependency declares via " + + "`additional_linker_inputs` are anchored the same way; a " + + "relative path not declared there passes through verbatim and " + + "will not resolve after the backend changes directory. Composes " + + "additively with `env`. Only static (or PIC-static) archives " + + "are supported; shared/dynamic and alwayslink libraries fail at " + + "analysis time. Only the setuptools build backend is supported; " + + "an sdist declaring any other build-backend is rejected when the " + + "wheel is built.", + ), "env": attr.string_dict( doc = "Environment variables to set on the build action. Values may " + "contain `$(VAR)` references to the configured C++ action tools " + diff --git a/uv/private/pep517_whl/tests/cc_deps/BUILD.bazel b/uv/private/pep517_whl/tests/cc_deps/BUILD.bazel new file mode 100644 index 000000000..aef8f4dd9 --- /dev/null +++ b/uv/private/pep517_whl/tests/cc_deps/BUILD.bazel @@ -0,0 +1,422 @@ +load("@bazel_lib//:bzl_library.bzl", "bzl_library") +load("@bazel_skylib//rules:write_file.bzl", "write_file") +load("@rules_cc//cc:defs.bzl", "cc_import", "cc_library") +load("//uv/private/pep517_whl:rule.bzl", "pep517_native_whl") +load( + ":test.bzl", + "assert_allowlist_matches_golden", + "cc_framework_shim", + "pep517_native_whl_cc_deps_content_test", + "pep517_native_whl_cc_deps_failure_test", + "pep517_native_whl_cc_deps_test", +) + +# Fails the package at load time if rule.bzl's allowed link-flag shapes or -z +# keywords drift from the golden copies in test.bzl, so an entry removed +# upstream cannot quietly drop the acceptance coverage below. +assert_allowlist_matches_golden() + +bzl_library( + name = "test", + testonly = True, + srcs = ["test.bzl"], + visibility = ["//uv:__subpackages__"], + deps = [ + "//uv/private/pep517_whl:rule", + "@bazel_skylib//lib:unittest", + "@rules_cc//cc/common", + ], +) + +# cc_deps analysis fixtures live in their own package because each successful +# pep517_native_whl target declares a `whl` output directory; two successful +# fixtures in one package would collide (see native_dep for the same pattern). +# The archives/headers here are never compiled or linked; the tests inspect +# the PySdistNativeBuild action at analysis time only. + +write_file( + name = "cc_dep_hdr", + out = "cc_dep.h", + content = ["int cc_dep_value(void);"], + tags = ["manual"], +) + +write_file( + name = "cc_dep_src", + out = "cc_dep.c", + content = [ + "#include \"cc_dep.h\"", + "int cc_dep_value(void) { return 42; }", + ], + tags = ["manual"], +) + +cc_library( + name = "cc_dep", + testonly = True, + srcs = ["cc_dep.c"], + hdrs = ["cc_dep.h"], + includes = ["."], + tags = ["manual"], +) + +# A two-level chain (chain_dep -> chain_dep2) whose direct library carries +# user_link_flags: a `-l`, and a verbatim flag referencing a file +# declared via additional_linker_inputs. Together with :cc_dep it exercises +# merge coverage, topological archive order, the -l partition, and the +# marker-anchoring of declared paths inside link flags. +write_file( + name = "chain_dep2_hdr", + out = "chain_dep2.h", + content = ["int chain_dep2_value(void);"], + tags = ["manual"], +) + +write_file( + name = "chain_dep2_src", + out = "chain_dep2.c", + content = [ + "#include \"chain_dep2.h\"", + "int chain_dep2_value(void) { return 7; }", + ], + tags = ["manual"], +) + +# A define on the transitive leaf proves -D propagation reaches the merged +# compilation context (and lands in compile_flags as a bare -D, marker-free). +cc_library( + name = "chain_dep2", + testonly = True, + srcs = ["chain_dep2.c"], + hdrs = ["chain_dep2.h"], + defines = ["CHAIN_BONUS=3"], + tags = ["manual"], +) + +write_file( + name = "chain_dep_hdr", + out = "chain_dep.h", + content = ["int chain_dep_value(void);"], + tags = ["manual"], +) + +write_file( + name = "chain_dep_src", + out = "chain_dep.c", + content = [ + "#include \"chain_dep.h\"", + "#include \"chain_dep2.h\"", + "int chain_dep_value(void) { return chain_dep2_value(); }", + ], + tags = ["manual"], +) + +write_file( + name = "version_script", + out = "vs.lds", + content = ["{ global: *; local: *; };"], + tags = ["manual"], +) + +# The linkopts carry one exemplar per allowed link-flag shape, directive form, +# and reviewed -z keyword (the content test asserts each reaches link_flags +# verbatim and in order): a `-l` for the libraries slot, a declared-path +# version script in its comma and = spellings, an -L search path, the rpath +# directives in comma and = spellings plus -rpath-link, each reviewed -z +# keyword, the benign multi-directive compound -z,relro,-z,now, +# --enable-new-dtags comma-joined after an $ORIGIN rpath ($$ escapes the +# literal $), and -pthread. The -L/rpath dummy paths are fixture-relative and +# never linked (analysis-only fixture). +cc_library( + name = "chain_dep", + testonly = True, + srcs = ["chain_dep.c"], + hdrs = ["chain_dep.h"], + additional_linker_inputs = [":version_script"], + linkopts = [ + "-lchainfoo", + "-Wl,--version-script,$(location :version_script)", + "-Wl,--version-script=$(location :version_script)", + "-Lvendor/dummy", + "-Wl,-rpath,vendor/rpath", + "-Wl,-rpath=vendor/rpath_eq", + "-Wl,-rpath-link,vendor/rpath_link", + "-Wl,-z,now", + "-Wl,-z,relro,-z,now", + "-Wl,-z,noexecstack", + "-Wl,-z,origin", + "-Wl,-rpath,$$ORIGIN,--enable-new-dtags", + "-pthread", + ], + tags = ["manual"], + deps = [":chain_dep2"], +) + +# Synthesizes a CcInfo carrying only an Apple `-F` framework search path. A plain +# cc_library cannot populate framework_includes on Linux, so the content test +# relies on this to pin the -F emission and marker-anchoring. It contributes no +# headers or archives, so it does not perturb the header/archive assertions. +cc_framework_shim( + name = "cc_framework_dep", + testonly = True, + framework_includes = ["vendor/Frameworks"], + tags = ["manual"], +) + +pep517_native_whl( + name = "cc_deps_fixture", + testonly = True, + src = "//uv/private/pep517_whl:__stub_sdist", + cc_deps = [ + ":cc_dep", + ":cc_framework_dep", + ":chain_dep", + ], + tags = ["manual"], + tool = "//uv/private/pep517_whl:__stub_tool", + version = "0.0.1", +) + +pep517_native_whl_cc_deps_test( + name = "cc_deps_test", + target_under_test = ":cc_deps_fixture", +) + +pep517_native_whl_cc_deps_content_test( + name = "cc_deps_content_test", + target_under_test = ":cc_deps_fixture", +) + +# A shared-library-only dep must fail at analysis: static archives only. +write_file( + name = "dynamic_only_so", + out = "libcc_dynamic_only.so", + content = [""], + tags = ["manual"], +) + +cc_import( + name = "cc_dynamic_only_dep", + testonly = True, + shared_library = "libcc_dynamic_only.so", + tags = ["manual"], +) + +pep517_native_whl( + name = "cc_deps_dynamic_only_fixture", + testonly = True, + src = "//uv/private/pep517_whl:__stub_sdist", + cc_deps = [":cc_dynamic_only_dep"], + tags = ["manual"], + tool = "//uv/private/pep517_whl:__stub_tool", + version = "0.0.1", +) + +pep517_native_whl_cc_deps_failure_test( + name = "cc_deps_dynamic_only_test", + expected_message = "provides no static archive", + target_under_test = ":cc_deps_dynamic_only_fixture", +) + +# An alwayslink (whole-archive) dep must fail at analysis: v1 cut. +cc_library( + name = "cc_alwayslink_dep", + testonly = True, + srcs = ["cc_dep.c"], + hdrs = ["cc_dep.h"], + tags = ["manual"], + alwayslink = True, +) + +pep517_native_whl( + name = "cc_deps_alwayslink_fixture", + testonly = True, + src = "//uv/private/pep517_whl:__stub_sdist", + cc_deps = [":cc_alwayslink_dep"], + tags = ["manual"], + tool = "//uv/private/pep517_whl:__stub_tool", + version = "0.0.1", +) + +pep517_native_whl_cc_deps_failure_test( + name = "cc_deps_alwayslink_test", + expected_message = "whole-archive linking is not supported", + target_under_test = ":cc_deps_alwayslink_fixture", +) + +# A link flag outside the allowlist must fail at analysis rather than get +# silently reordered. Six cases cover the contract: (a) a flag the predecessor +# denylist named explicitly, proving those are still rejected; (b) an arbitrary +# flag no list ever enumerated, proving rejection is default-deny rather than +# list-membership; (c) a bare unwrapped token, pinning that a flag with no -Wl, +# wrapper is handled the same way; (d) a comma-joined -Wl, token whose leading +# directive is allowed but which smuggles a rejected directive behind it, +# proving the per-directive walk; (e) -framework, a library input the two-slot +# split cannot hold; (f) a -z keyword outside the reviewed set, proving -z is +# not an open class. Each test pins the exact offending flag (and directive) as +# the rule prints it. Failure fixtures may share the package: analysis fails +# before the whl output directory gets a generating action. +cc_library( + name = "cc_reject_group_dep", + testonly = True, + srcs = ["cc_dep.c"], + hdrs = ["cc_dep.h"], + linkopts = ["-Wl,--start-group"], + tags = ["manual"], +) + +pep517_native_whl( + name = "cc_deps_reject_group_fixture", + testonly = True, + src = "//uv/private/pep517_whl:__stub_sdist", + cc_deps = [":cc_reject_group_dep"], + tags = ["manual"], + tool = "//uv/private/pep517_whl:__stub_tool", + version = "0.0.1", +) + +pep517_native_whl_cc_deps_failure_test( + name = "cc_deps_reject_group_test", + expected_message = "passes the link flag -Wl,--start-group; its directive --start-group is not", + target_under_test = ":cc_deps_reject_group_fixture", +) + +cc_library( + name = "cc_reject_unknown_dep", + testonly = True, + srcs = ["cc_dep.c"], + hdrs = ["cc_dep.h"], + linkopts = ["-Wl,--definitely-not-a-real-flag"], + tags = ["manual"], +) + +pep517_native_whl( + name = "cc_deps_reject_unknown_fixture", + testonly = True, + src = "//uv/private/pep517_whl:__stub_sdist", + cc_deps = [":cc_reject_unknown_dep"], + tags = ["manual"], + tool = "//uv/private/pep517_whl:__stub_tool", + version = "0.0.1", +) + +pep517_native_whl_cc_deps_failure_test( + name = "cc_deps_reject_unknown_test", + expected_message = "passes the link flag -Wl,--definitely-not-a-real-flag; its directive --definitely-not-a-real-flag is not", + target_under_test = ":cc_deps_reject_unknown_fixture", +) + +# A bare token with no -Wl, wrapper. The compiler driver forwards such flags to +# the linker unwrapped, so this is the realistic bare form. +cc_library( + name = "cc_reject_bare_dep", + testonly = True, + srcs = ["cc_dep.c"], + hdrs = ["cc_dep.h"], + linkopts = ["--whole-archive"], + tags = ["manual"], +) + +pep517_native_whl( + name = "cc_deps_reject_bare_fixture", + testonly = True, + src = "//uv/private/pep517_whl:__stub_sdist", + cc_deps = [":cc_reject_bare_dep"], + tags = ["manual"], + tool = "//uv/private/pep517_whl:__stub_tool", + version = "0.0.1", +) + +pep517_native_whl_cc_deps_failure_test( + name = "cc_deps_reject_bare_test", + expected_message = "passes the link flag --whole-archive, which is not", + target_under_test = ":cc_deps_reject_bare_fixture", +) + +# An allowed leading directive (-rpath) must not smuggle a rejected directive +# behind it in the same comma-joined token; the walk validates every directive +# and the message names the offender. +cc_library( + name = "cc_reject_smuggle_dep", + testonly = True, + srcs = ["cc_dep.c"], + hdrs = ["cc_dep.h"], + linkopts = ["-Wl,-rpath,/x,--as-needed"], + tags = ["manual"], +) + +pep517_native_whl( + name = "cc_deps_reject_smuggle_fixture", + testonly = True, + src = "//uv/private/pep517_whl:__stub_sdist", + cc_deps = [":cc_reject_smuggle_dep"], + tags = ["manual"], + tool = "//uv/private/pep517_whl:__stub_tool", + version = "0.0.1", +) + +pep517_native_whl_cc_deps_failure_test( + name = "cc_deps_reject_smuggle_test", + expected_message = "passes the link flag -Wl,-rpath,/x,--as-needed; its directive --as-needed is not", + target_under_test = ":cc_deps_reject_smuggle_fixture", +) + +# -framework is a library input: ld64 resolves frameworks in command-line order +# alongside -l entries, so relocating the -l entries to the post-object slot +# while -framework rides pre-object LDFLAGS would reorder library resolution. +# Rejected; the head token is named before its name token is ever read. +cc_library( + name = "cc_reject_framework_dep", + testonly = True, + srcs = ["cc_dep.c"], + hdrs = ["cc_dep.h"], + linkopts = [ + "-framework", + "CoreFoundation", + ], + tags = ["manual"], +) + +pep517_native_whl( + name = "cc_deps_reject_framework_fixture", + testonly = True, + src = "//uv/private/pep517_whl:__stub_sdist", + cc_deps = [":cc_reject_framework_dep"], + tags = ["manual"], + tool = "//uv/private/pep517_whl:__stub_tool", + version = "0.0.1", +) + +pep517_native_whl_cc_deps_failure_test( + name = "cc_deps_reject_framework_test", + expected_message = "passes the link flag -framework, which is not", + target_under_test = ":cc_deps_reject_framework_fixture", +) + +# A -z keyword outside the reviewed set must be rejected: the -z namespace is +# open and contains position-sensitive keywords (Solaris -z allextract toggles +# whole-archive extraction for the archives that follow it). +cc_library( + name = "cc_reject_z_keyword_dep", + testonly = True, + srcs = ["cc_dep.c"], + hdrs = ["cc_dep.h"], + linkopts = ["-Wl,-z,allextract"], + tags = ["manual"], +) + +pep517_native_whl( + name = "cc_deps_reject_z_keyword_fixture", + testonly = True, + src = "//uv/private/pep517_whl:__stub_sdist", + cc_deps = [":cc_reject_z_keyword_dep"], + tags = ["manual"], + tool = "//uv/private/pep517_whl:__stub_tool", + version = "0.0.1", +) + +pep517_native_whl_cc_deps_failure_test( + name = "cc_deps_reject_z_keyword_test", + expected_message = "passes the link flag -Wl,-z,allextract; its directive -z allextract is not", + target_under_test = ":cc_deps_reject_z_keyword_fixture", +) diff --git a/uv/private/pep517_whl/tests/cc_deps/test.bzl b/uv/private/pep517_whl/tests/cc_deps/test.bzl new file mode 100644 index 000000000..ca07f19df --- /dev/null +++ b/uv/private/pep517_whl/tests/cc_deps/test.bzl @@ -0,0 +1,301 @@ +"""Analysis-test helpers for pep517_native_whl(cc_deps = ...).""" + +load("@bazel_skylib//lib:unittest.bzl", "analysistest", "asserts") +load("@rules_cc//cc/common:cc_common.bzl", "cc_common") +load("@rules_cc//cc/common:cc_info.bzl", "CcInfo") +load("//uv/private/pep517_whl:rule.bzl", "ALLOWED_LINK_FLAG_SHAPES", "ALLOWED_Z_KEYWORDS") + +# Golden copies of the accepted link-flag shapes and -z keywords. Deliberately +# duplicated: the acceptance assertions in the content test below cover each +# shape and keyword by exemplar, so an entry dropped from a rule.bzl tuple could +# quietly drop its coverage instead of turning a test red. Comparing the +# exported tuples against these copies at load time fails the package (naming +# both copies) the moment they diverge. +_ALLOWED_LINK_FLAG_SHAPES_GOLDEN = ( + ("exact", "-pthread"), + ("prefix", "-L"), + ("wl_arg", "-rpath"), + ("wl_arg", "-rpath-link"), + ("wl_arg", "--version-script"), + ("wl_keyword", "-z"), + ("wl_exact", "--enable-new-dtags"), +) + +_ALLOWED_Z_KEYWORDS_GOLDEN = ( + "relro", + "now", + "noexecstack", + "origin", +) + +def assert_allowlist_matches_golden(): + """Fail at load if rule.bzl's allowlist drifts from the golden copies here.""" + if tuple(ALLOWED_LINK_FLAG_SHAPES) != _ALLOWED_LINK_FLAG_SHAPES_GOLDEN: + fail( + "ALLOWED_LINK_FLAG_SHAPES in rule.bzl no longer matches the golden " + + "copy in test.bzl. Update both, and the acceptance exemplars below. " + + "rule.bzl has: {}".format(list(ALLOWED_LINK_FLAG_SHAPES)), + ) + if tuple(ALLOWED_Z_KEYWORDS) != _ALLOWED_Z_KEYWORDS_GOLDEN: + fail( + "ALLOWED_Z_KEYWORDS in rule.bzl no longer matches the golden copy " + + "in test.bzl. Update both, and the acceptance exemplars below. " + + "rule.bzl has: {}".format(list(ALLOWED_Z_KEYWORDS)), + ) + +# A plain cc_library cannot populate compilation_context.framework_includes on +# Linux (Apple `-F` search paths), so this shim synthesizes a CcInfo carrying +# only framework_includes. It lets the analysis content test pin the `-F` +# emission and marker-anchoring without a macOS toolchain. +def _cc_framework_shim_impl(ctx): + compilation_context = cc_common.create_compilation_context( + framework_includes = depset(ctx.attr.framework_includes), + ) + return [CcInfo(compilation_context = compilation_context)] + +cc_framework_shim = rule( + implementation = _cc_framework_shim_impl, + attrs = {"framework_includes": attr.string_list()}, +) + +def _cc_deps_test_impl(ctx): + env = analysistest.begin(ctx) + target = analysistest.target_under_test(env) + + build_actions = [a for a in target.actions if a.mnemonic == "PySdistNativeBuild"] + asserts.equals( + env, + 1, + len(build_actions), + "expected exactly one PySdistNativeBuild action", + ) + + action = build_actions[0] + args = action.argv + asserts.true( + env, + "--cc-deps-info" in args, + "action should carry --cc-deps-info; got: {}".format(args), + ) + + inputs = action.inputs.to_list() + info_path = args[args.index("--cc-deps-info") + 1] + input_paths = [f.path for f in inputs] + asserts.true( + env, + info_path in input_paths, + "cc-deps-info params file should be an action input", + ) + + # Every header and archive in the merged two-dep closure must reach the + # action inputs BY NAME, so a mutant that drops one of the three libraries + # fails rather than passing on the survivors (the previous "some .h / some + # .a" probes could not tell the closure apart from a single dep). + input_basenames = [f.basename for f in inputs] + for header in ("cc_dep.h", "chain_dep.h", "chain_dep2.h"): + asserts.true( + env, + header in input_basenames, + "cc_deps header {} should be an action input; got {}".format(header, input_basenames), + ) + + # Archives are matched by their lib. prefix (not exact basename) so the + # toolchain's pic vs non-pic archive choice does not matter; each of the + # three must be present exactly once. + for lib in ("libcc_dep.", "libchain_dep.", "libchain_dep2."): + matches = [b for b in input_basenames if b.startswith(lib) and b.endswith(".a")] + asserts.equals( + env, + 1, + len(matches), + "expected exactly one {}a archive among inputs; got {}".format(lib, input_basenames), + ) + + return analysistest.end(env) + +pep517_native_whl_cc_deps_test = analysistest.make(_cc_deps_test_impl) + +def _cc_deps_content_test_impl(ctx): + env = analysistest.begin(ctx) + target = analysistest.target_under_test(env) + + build_actions = [a for a in target.actions if a.mnemonic == "PySdistNativeBuild"] + asserts.equals( + env, + 1, + len(build_actions), + "expected exactly one PySdistNativeBuild action", + ) + args = build_actions[0].argv + marker = args[args.index("--execroot-marker") + 1] + + write_actions = [ + a + for a in target.actions + if a.mnemonic == "FileWrite" and a.outputs.to_list()[0].basename == "cc_deps_info.json" + ] + asserts.equals( + env, + 1, + len(write_actions), + "expected exactly one cc_deps_info.json write action", + ) + info = json.decode(write_actions[0].content) + + # Every include (and Apple framework) search path must be anchored so it + # survives the backend chdir. + for flag in info["compile_flags"]: + for prefix in ("-isystem", "-iquote", "-I", "-F"): + if flag.startswith(prefix): + asserts.true( + env, + flag[len(prefix):].startswith(marker + "/"), + "compile flag path should be marker-anchored: {}".format(flag), + ) + break + + # Positive: the cc_dep(includes = ["."]) source dir must be emitted as a + # marker-anchored include. Asserting a specific expected entry is PRESENT + # (not merely that any emitted entry is well-formed) makes a mutant that + # drops include emission fail rather than pass vacuously. Bazel 8 surfaces + # cc_library(includes) via system_includes (-isystem); Bazel 9 surfaces the + # same attribute via includes (-I). Accept either spelling of the same dir. + expected_system = "-isystem{}/{}".format(marker, ctx.label.package) + expected_plain = "-I{}/{}".format(marker, ctx.label.package) + asserts.true( + env, + expected_system in info["compile_flags"] or expected_plain in info["compile_flags"], + "cc_dep(includes=['.']) should emit {} or {}; got {}".format( + expected_system, + expected_plain, + info["compile_flags"], + ), + ) + + # Positive: chain_dep2's transitive define must reach compile_flags as a bare + # -D (no marker: defines carry symbols, not paths). Asserting the specific + # entry makes a mutant that drops -D emission fail rather than pass vacuously. + asserts.true( + env, + "-DCHAIN_BONUS=3" in info["compile_flags"], + "transitive define should emit -DCHAIN_BONUS=3; got {}".format(info["compile_flags"]), + ) + + # Positive: the framework shim's Apple `-F` search path must be emitted as a + # marker-anchored -F entry. A plain cc_library cannot produce + # framework_includes on Linux, so this is the only coverage that the -F + # emission and anchoring exist; behavioral macOS linking is out of reach on + # Linux CI. + expected_framework = "-F{}/vendor/Frameworks".format(marker) + asserts.true( + env, + expected_framework in info["compile_flags"], + "framework_includes should emit {}; got {}".format(expected_framework, info["compile_flags"]), + ) + + link_objects = info["link_objects"] + for obj in link_objects: + asserts.true( + env, + obj.startswith(marker + "/"), + "link object should be marker-anchored: {}".format(obj), + ) + + # The two-level chain must contribute both archives, dependent before + # dependency (topological order), alongside the second merged dep. + chain = [i for i, obj in enumerate(link_objects) if obj.split("/")[-1].startswith("libchain_dep.")] + chain2 = [i for i, obj in enumerate(link_objects) if obj.split("/")[-1].startswith("libchain_dep2.")] + merged = [i for i, obj in enumerate(link_objects) if obj.split("/")[-1].startswith("libcc_dep.")] + asserts.equals(env, 1, len(chain), "expected the direct chain archive; got {}".format(link_objects)) + asserts.equals(env, 1, len(chain2), "expected the transitive chain archive; got {}".format(link_objects)) + asserts.equals(env, 1, len(merged), "expected the second dep's archive; got {}".format(link_objects)) + asserts.true( + env, + chain[0] < chain2[0], + "dependent archive should precede its dependency; got {}".format(link_objects), + ) + + asserts.true( + env, + "chainfoo" in info["link_libraries"], + "-lchainfoo should land in link_libraries as a bare name; got {}".format(info["link_libraries"]), + ) + + # A declared additional_linker_inputs path inside a verbatim flag must be + # marker-anchored, keyed on the file's declared execroot-relative path. + # Covered in both the comma and the = spelling of --version-script. + input_paths = [f.path for f in build_actions[0].inputs.to_list()] + vs_paths = [p for p in input_paths if p.endswith("/vs.lds")] + asserts.equals(env, 1, len(vs_paths), "version script should be an action input") + for version_script in ( + "-Wl,--version-script,{}/{}".format(marker, vs_paths[0]), + "-Wl,--version-script={}/{}".format(marker, vs_paths[0]), + ): + asserts.true( + env, + version_script in info["link_flags"], + "declared version-script path should be marker-anchored in link_flags as {}; got {}".format(version_script, info["link_flags"]), + ) + + # Acceptance: one exemplar per allowed link-flag shape and directive form, + # added to chain_dep's linkopts, must reach link_flags verbatim. The -L and + # rpath dummy paths are fixture-relative and never linked (this fixture is + # analysis-only). -pthread is the exact shape; -Lvendor/dummy is the glued + # prefix; the -Wl, entries cover each argument directive in its comma and = + # spelling, the reviewed -z keywords, the benign multi-directive compound, + # and --enable-new-dtags comma-joined after an $ORIGIN rpath. + link_flags = info["link_flags"] + for exemplar in ( + "-Lvendor/dummy", + "-Wl,-rpath,vendor/rpath", + "-Wl,-rpath=vendor/rpath_eq", + "-Wl,-rpath-link,vendor/rpath_link", + "-Wl,-z,now", + "-Wl,-z,relro,-z,now", + "-Wl,-z,noexecstack", + "-Wl,-z,origin", + "-Wl,-rpath,$ORIGIN,--enable-new-dtags", + "-pthread", + ): + asserts.true( + env, + exemplar in link_flags, + "allowed link flag {} should land verbatim in link_flags; got {}".format(exemplar, link_flags), + ) + + # Relative order among the allowed flags follows linkopts order. + expected_order = [ + "-Wl,--version-script,{}/{}".format(marker, vs_paths[0]), + "-Wl,--version-script={}/{}".format(marker, vs_paths[0]), + "-Lvendor/dummy", + "-Wl,-rpath,vendor/rpath", + "-Wl,-rpath=vendor/rpath_eq", + "-Wl,-rpath-link,vendor/rpath_link", + "-Wl,-z,now", + "-Wl,-z,relro,-z,now", + "-Wl,-z,noexecstack", + "-Wl,-z,origin", + "-Wl,-rpath,$ORIGIN,--enable-new-dtags", + "-pthread", + ] + asserts.equals( + env, + expected_order, + [flag for flag in link_flags if flag in expected_order], + "allowed link flags should appear in linkopts order; got {}".format(link_flags), + ) + + return analysistest.end(env) + +pep517_native_whl_cc_deps_content_test = analysistest.make(_cc_deps_content_test_impl) + +def _cc_deps_failure_test_impl(ctx): + env = analysistest.begin(ctx) + asserts.expect_failure(env, ctx.attr.expected_message) + return analysistest.end(env) + +pep517_native_whl_cc_deps_failure_test = analysistest.make( + _cc_deps_failure_test_impl, + expect_failure = True, + attrs = {"expected_message": attr.string(mandatory = True)}, +) From 2c79a2191fc927373cc0155c348b792fa9772692 Mon Sep 17 00:00:00 2001 From: Connor McEntee Date: Mon, 27 Jul 2026 20:13:54 +0000 Subject: [PATCH 2/5] feat(uv): apply cc_deps inputs via setuptools DIST_EXTRA_CONFIG Implement the build_helper runtime half of cc_deps. When --cc-deps-info is passed, load the params file the rule emits, expand the execroot marker to the current directory (build_helper still runs at the execroot, before the backend changes into the unpacked worktree), and inject the flattened compile and link inputs into the setuptools build. Compile flags append to CPPFLAGS and exotic link flags append to LDFLAGS, each preserving any user value as the prefix; the dependencies' static archives and -l libraries become [build_ext] link_objects and libraries in a config file written to tmp_root (never the worktree) and pointed at by DIST_EXTRA_CONFIG. Because DIST_EXTRA_CONFIG replaces same-named options, any [build_ext] link_objects/libraries the package sets in its own setup.cfg are folded back in ahead of ours. include_dirs and defines ride CPPFLAGS instead, so the package's setup.cfg include_dirs keep applying. Gate the whole path on setuptools: a non-setuptools build-backend fails with an actionable message, as do setuptools older than 65.4.0 (the DIST_EXTRA_CONFIG floor), a pre-existing DIST_EXTRA_CONFIG, and SETUPTOOLS_USE_DISTUTILS=stdlib. Values that setuptools' option splitting or CPPFLAGS/LDFLAGS word-splitting would mangle fail early naming the offender; guarded are link_objects paths (whitespace/comma/os.pathsep), library names (whitespace/comma), search paths carried by -I/-iquote/-isystem/-F (whitespace), and every link flag (whitespace), since LDFLAGS is word-split downstream whether the flag carries an execroot-anchored path or a user linkopt with a space. -D defines are deliberately unguarded: they carry symbols, not paths, and CPPFLAGS is the only channel that can express K=V macros. When --cc-deps-info is absent the build is byte-for-byte unchanged. --- uv/private/pep517_whl/build_helper.py | 313 +++++++++++++++++++++++++- 1 file changed, 312 insertions(+), 1 deletion(-) diff --git a/uv/private/pep517_whl/build_helper.py b/uv/private/pep517_whl/build_helper.py index cd5725150..026f894d2 100644 --- a/uv/private/pep517_whl/build_helper.py +++ b/uv/private/pep517_whl/build_helper.py @@ -18,7 +18,7 @@ from os import chmod, defpath, listdir, makedirs, path, pathsep from subprocess import CalledProcessError, check_call, check_output, STDOUT, run from tempfile import TemporaryFile -from typing import Dict, Optional +from typing import Dict, List, Optional try: tomllib = importlib.import_module("tomllib") @@ -270,6 +270,311 @@ def _legacy_metadata_conflicts_with_pyproject(worktree: str) -> bool: ) ) + +def _load_cc_deps_info(info_path: str, execroot_marker: Optional[str]) -> Dict[str, List[str]]: + """Load the cc_deps params file, re-anchoring its execroot-relative paths. + + The rule emits every path prefixed with `execroot_marker` because only + execroot-relative paths exist at analysis time. build_helper still runs at + the execroot here (the backend chdir happens in the child process), so + `os.getcwd()` is the execroot the marker must expand to. + """ + import json + + if not execroot_marker: + # Defensive: the rule always pairs the two flags; without the marker the + # paths below cannot be anchored. + print( + "Error: --cc-deps-info requires --execroot-marker to anchor its paths.", + file=sys.stderr, + ) + exit(1) + + with open(info_path, encoding="utf-8") as f: + raw = json.load(f) + + execroot = os.getcwd() + info: Dict[str, List[str]] = {} + for key in ("compile_flags", "link_objects", "link_libraries", "link_flags"): + values: List[str] = [] + for value in raw.get(key, []): + resolved = value.replace(execroot_marker, execroot) + if execroot_marker in resolved: + print( + "Error: execroot marker survived substitution in cc_deps " + "{}: {!r}".format(key, resolved), + file=sys.stderr, + ) + exit(1) + values.append(resolved) + info[key] = values + return info + + +def _effective_build_backend(worktree: str) -> Optional[str]: + """Return the declared PEP 517 backend, or None for the setuptools default. + + A missing pyproject.toml, `[build-system]` table, or `build-backend` key all + mean the legacy setuptools path (None is in `_SETUPTOOLS_BACKENDS`), which + still honors DIST_EXTRA_CONFIG. + """ + pyproject_data = _load_pyproject_data(worktree) + if not pyproject_data: + return None + build_system = pyproject_data.get("build-system", {}) + if not isinstance(build_system, dict): + return None + backend = build_system.get("build-backend") + return backend if isinstance(backend, str) else None + + +def _require_setuptools_floor() -> None: + """Fail unless the build venv's setuptools understands DIST_EXTRA_CONFIG.""" + from importlib.metadata import PackageNotFoundError, version + + # DIST_EXTRA_CONFIG landed in setuptools 65.4.0; older setuptools silently + # ignores the [build_ext] config we hand it, dropping the linked archives. + try: + found = version("setuptools") + except PackageNotFoundError: + print( + "Error: cc_deps requires setuptools >= 65.4.0 in the build " + "environment, but setuptools is not installed. Add setuptools to your " + "uv.lock / default_build_dependencies.", + file=sys.stderr, + ) + exit(1) + + parts = found.split(".") + try: + version_tuple = (int(parts[0]), int(parts[1]) if len(parts) > 1 else 0) + except (IndexError, ValueError): + print( + "Error: cc_deps requires setuptools >= 65.4.0, but could not parse the " + "installed setuptools version {!r}.".format(found), + file=sys.stderr, + ) + exit(1) + + if version_tuple < (65, 4): + print( + "Error: cc_deps requires setuptools >= 65.4.0 (for DIST_EXTRA_CONFIG), " + "but the build environment has {}. Update setuptools in your uv.lock / " + "default_build_dependencies.".format(found), + file=sys.stderr, + ) + exit(1) + + +def _has_whitespace(value: str) -> bool: + return any(character.isspace() for character in value) + + +def _cc_deps_include_path(flag: str) -> Optional[str]: + """Return the directory carried by an -I/-iquote/-isystem/-F flag, else None.""" + for prefix in ("-iquote", "-isystem", "-I", "-F"): + if flag.startswith(prefix): + return flag[len(prefix):] + return None + + +def _reject_unsplittable_cc_deps(info: Dict[str, List[str]]) -> None: + """Reject values setuptools/CPPFLAGS/LDFLAGS would split, naming the offender. + + setuptools splits `[build_ext] link_objects` and `libraries` on whitespace + or comma, and CPPFLAGS/LDFLAGS are word-split downstream, so a path + containing either survives neither. The execroot itself can carry a space + (a user home directory), so this is exactly the case the guard catches. + """ + # os.pathsep does not appear in setuptools' link_objects split rule + # (whitespace/comma only), but the guard keeps it as an over-conservative + # fail-safe: a colon-bearing archive path has no legitimate source here. + for link_object in info["link_objects"]: + if _has_whitespace(link_object) or "," in link_object or pathsep in link_object: + print( + "Error: cc_deps link object path {!r} contains whitespace, a " + "comma, or {!r}; setuptools splits [build_ext] link_objects on " + "whitespace/comma and would mangle it. This usually means the " + "Bazel output base (execroot) contains a space; choose an " + "--output_base without spaces.".format(link_object, pathsep), + file=sys.stderr, + ) + exit(1) + for library in info["link_libraries"]: + if _has_whitespace(library) or "," in library: + print( + "Error: cc_deps library name {!r} contains whitespace or a " + "comma; setuptools splits [build_ext] libraries on those and " + "would mangle it.".format(library), + file=sys.stderr, + ) + exit(1) + # -D defines are deliberately unguarded (binding decision): they carry + # symbols, not paths, and CPPFLAGS is the only channel that can express K=V. + for flag in info["compile_flags"]: + include = _cc_deps_include_path(flag) + if include is not None and _has_whitespace(include): + print( + "Error: cc_deps compile search path {!r} contains whitespace; " + "CPPFLAGS is word-split downstream and would mangle it. This " + "usually means the Bazel output base (execroot) contains a space; " + "choose an --output_base without spaces.".format(include), + file=sys.stderr, + ) + exit(1) + # LDFLAGS is word-split downstream, so a whitespace-bearing token is + # unrepresentable, whether it's an execroot-anchored path + # (e.g. -Wl,--version-script,/.../vs.lds) or a user linkopt that + # happens to carry a space. Guard every link flag, not just the + # execroot-anchored ones. + for flag in info["link_flags"]: + if _has_whitespace(flag): + print( + "Error: cc_deps link flag {!r} contains whitespace; it is " + "appended to LDFLAGS, which is word-split downstream, so no " + "single flag can carry a space. This usually means the Bazel " + "output base (execroot) contains a space; choose an " + "--output_base without spaces.".format(flag), + file=sys.stderr, + ) + exit(1) + + +def _package_build_ext_option(worktree: str, option: str) -> Optional[str]: + """Return a `[build_ext]` option value from the package's setup.cfg, if any. + + DIST_EXTRA_CONFIG REPLACES (never merges) same-named options, so the + package's own value has to be folded back in explicitly. Tolerant of a + missing setup.cfg; warns but continues if an existing one will not parse. + """ + import configparser + + setup_cfg = path.join(worktree, "setup.cfg") + if not path.exists(setup_cfg): + return None + + parser = configparser.ConfigParser(interpolation=None) + try: + parser.read(setup_cfg, encoding="utf-8") + except configparser.Error: + print( + "Warning: ignoring [build_ext] settings in {} (could not parse it).".format(setup_cfg), + file=sys.stderr, + ) + return None + + if parser.has_option("build_ext", option): + return parser.get("build_ext", option) + return None + + +def _merged_build_ext_value(worktree: str, option: str, values: List[str]) -> str: + """Join our `values` for a [build_ext] option behind the package's own value.""" + ours = " ".join(values) + package_value = _package_build_ext_option(worktree, option) + if not package_value: + return ours + # configparser may return a multi-line value; collapse it to one physical + # line so our generated cfg stays well-formed. The option is whitespace/comma + # split downstream, so token semantics are preserved, and the package's + # entries stay worktree-relative (they resolve because the backend's cwd is + # the worktree). + package_value = " ".join(package_value.split()) + return "{} {}".format(package_value, ours) if ours else package_value + + +def _build_extra_config_text(info: Dict[str, List[str]], worktree: str) -> str: + """Render the DIST_EXTRA_CONFIG `[build_ext]` file for the link inputs. + + include_dirs and defines are deliberately omitted: they ride CPPFLAGS + instead (a cfg `define` cannot express K=V macros, and CPPFLAGS reaches + custom build commands too). + """ + lines = ["[build_ext]"] + if info["link_objects"]: + lines.append("link_objects = {}".format( + _merged_build_ext_value(worktree, "link_objects", info["link_objects"]), + )) + if info["link_libraries"]: + lines.append("libraries = {}".format( + _merged_build_ext_value(worktree, "libraries", info["link_libraries"]), + )) + return "\n".join(lines) + "\n" + + +def _append_cc_deps_env(env: Dict[str, str], key: str, additions: List[str]) -> None: + """Append flags to an env var, preserving any user value as the prefix.""" + if not additions: + return + addition = " ".join(additions) + existing = env.get(key) + env[key] = "{} {}".format(existing, addition) if existing else addition + + +def _apply_cc_deps( + env: Dict[str, str], + info_path: str, + execroot_marker: Optional[str], + worktree: str, + tmp_root: str, +) -> None: + """Route cc_deps compile/link inputs into the setuptools build. + + Mutates `env` in place: appends compile flags to CPPFLAGS and exotic link + flags to LDFLAGS, and points DIST_EXTRA_CONFIG at a `[build_ext]` file (in + tmp_root, never the worktree) carrying the static archives and `-l` + libraries. Only runs when `--cc-deps-info` was passed, so the no-cc_deps + path is unchanged. + """ + info = _load_cc_deps_info(info_path, execroot_marker) + + # cc_deps is a setuptools-only feature in v1; refuse other backends loudly + # rather than silently dropping the inputs. + backend = _effective_build_backend(worktree) + if backend not in _SETUPTOOLS_BACKENDS: + print( + "Error: cc_deps is only supported with the setuptools build backend, " + "but this package declares build-backend {!r}. cc_deps injects " + "setuptools [build_ext] settings, which other backends ignore.".format(backend), + file=sys.stderr, + ) + exit(1) + + _require_setuptools_floor() + + # Never merge with a user-provided DIST_EXTRA_CONFIG (v1 owns it). + if "DIST_EXTRA_CONFIG" in env: + print( + "Error: cc_deps needs DIST_EXTRA_CONFIG, but it is already set in the " + "build environment. Remove it from `env` to use cc_deps.", + file=sys.stderr, + ) + exit(1) + + # DIST_EXTRA_CONFIG lives only in setuptools' local distutils. + if env.get("SETUPTOOLS_USE_DISTUTILS") == "stdlib": + print( + "Error: cc_deps requires setuptools' local distutils, but " + "SETUPTOOLS_USE_DISTUTILS=stdlib is set. Unset it (or set it to " + "\"local\") to use cc_deps.", + file=sys.stderr, + ) + exit(1) + env.setdefault("SETUPTOOLS_USE_DISTUTILS", "local") + + _reject_unsplittable_cc_deps(info) + + # The config file lives in tmp_root, never the worktree. + config_path = path.join(tmp_root, "cc_deps_extra.cfg") + with open(config_path, "w", encoding="utf-8") as f: + f.write(_build_extra_config_text(info, worktree)) + env["DIST_EXTRA_CONFIG"] = config_path + + # Compose additively; any user CPPFLAGS/LDFLAGS stay in front. + _append_cc_deps_env(env, "CPPFLAGS", info["compile_flags"]) + _append_cc_deps_env(env, "LDFLAGS", info["link_flags"]) + + PARSER = ArgumentParser() PARSER.add_argument("srcarchive") PARSER.add_argument("outdir") @@ -278,6 +583,7 @@ def _legacy_metadata_conflicts_with_pyproject(worktree: str) -> bool: PARSER.add_argument("--patch-strip", type=int, default=0, help="Strip count for patch (-p)") PARSER.add_argument("--patch", action="append", default=[], dest="patches", help="Patch file to apply (repeatable)") PARSER.add_argument("--execroot-marker", help="Token in env values to replace with the absolute execroot") +PARSER.add_argument("--cc-deps-info", help="Path to the cc_deps compile/link params file to inject") opts, _ = PARSER.parse_known_args() tmp_root = path.abspath(opts.outdir) + ".tmp" @@ -324,6 +630,11 @@ def _legacy_metadata_conflicts_with_pyproject(worktree: str) -> bool: # change into the worktree. build_env = _compiler_env(tmp_root, opts.execroot_marker) +# cc_deps rides the same execroot cwd as _compiler_env: substitute the marker +# and inject the setuptools [build_ext] config before the backend chdirs. +if opts.cc_deps_info: + _apply_cc_deps(build_env, opts.cc_deps_info, opts.execroot_marker, t, tmp_root) + if _legacy_metadata_conflicts_with_pyproject(t): print( "Warning: falling back to setup.py because pyproject.toml omits dynamic dependency metadata " From 4784acdbccf1e85440d4d4a82d47e3294c55c908 Mon Sep 17 00:00:00 2001 From: Connor McEntee Date: Mon, 27 Jul 2026 20:14:04 +0000 Subject: [PATCH 3/5] feat(uv): plumb cc_deps from uv.override_package to sdist builds Thread the resolved cc_deps CcInfo labels from uv.override_package all the way to the generated pep517_native_whl(cc_deps = [...]) call, mirroring the existing toolchains/env rails end to end. On the extension side, add a cc_deps label_list tag attribute, OR it into has_modifications (so a cc_deps-only override is accepted) and name it in the must-specify-one diagnostic, stringify it with str() alongside extra_toolchains, and forward it through the sbuild struct and sbuild_kwargs as extra_cc_deps. validate_build_attrs grows a cc_deps parameter and treats it as native-only: like toolchains/env it is absent from the wheels-only, build_file_content, and pure-Python allowlists, so a cc_deps override on any non-native build path fails fast with the existing actionable message rather than silently dropping the dependency. On the sdist_build repo rule, add an extra_cc_deps string_list attribute and emit a cc_deps = [...] block onto the generated native rule using the same quoting as the toolchains list. Emission is guarded by both is_native and a non-empty extra_cc_deps, so configs without cc_deps stay byte-for-byte unchanged and the three existing sdist_build snapshots do not move. Cover the plumbing with a snapshot: a new uv-sdist-cc-deps e2e case reuses python-geohash's locked sdist under a distinct project/hub and attaches an in-repo cc_library via cc_deps, pinning that the canonical CcInfo label lands on the generated pep517_native_whl(...) call. --- e2e/cases/BUILD.bazel | 11 +++ e2e/cases/MODULE.bazel | 1 + ...v_sdist_cc_deps.python_geohash.BUILD.bazel | 26 +++++++ e2e/cases/uv-sdist-cc-deps/BUILD.bazel | 25 +++++++ e2e/cases/uv-sdist-cc-deps/probe.cc | 3 + e2e/cases/uv-sdist-cc-deps/probe.h | 16 +++++ e2e/cases/uv-sdist-cc-deps/pyproject.toml | 7 ++ e2e/cases/uv-sdist-cc-deps/setup.MODULE.bazel | 37 ++++++++++ e2e/cases/uv-sdist-cc-deps/uv.lock | 70 +++++++++++++++++++ uv/private/extension/defs.bzl | 20 ++++-- uv/private/sdist_build/attrs.bzl | 4 ++ uv/private/sdist_build/repository.bzl | 17 ++++- 12 files changed, 232 insertions(+), 5 deletions(-) create mode 100644 e2e/cases/snapshots/sdist_build.uv_sdist_cc_deps.python_geohash.BUILD.bazel create mode 100644 e2e/cases/uv-sdist-cc-deps/BUILD.bazel create mode 100644 e2e/cases/uv-sdist-cc-deps/probe.cc create mode 100644 e2e/cases/uv-sdist-cc-deps/probe.h create mode 100644 e2e/cases/uv-sdist-cc-deps/pyproject.toml create mode 100644 e2e/cases/uv-sdist-cc-deps/setup.MODULE.bazel create mode 100644 e2e/cases/uv-sdist-cc-deps/uv.lock diff --git a/e2e/cases/BUILD.bazel b/e2e/cases/BUILD.bazel index 314ebc619..c6766676c 100644 --- a/e2e/cases/BUILD.bazel +++ b/e2e/cases/BUILD.bazel @@ -168,6 +168,17 @@ write_source_files( # pep517_native_whl(...) call so the reservation reaches the action. "snapshots/sdist_build.uv_sdist_native_build.python_geohash.BUILD.bazel": "@sdist_build__uv_sdist_native_build__python_geohash__0_8_5//:BUILD.bazel", + # @sdist_build for python-geohash with uv.override_package(cc_deps) + # ---------------------------------------------------------------------- + # Covers the override_package -> repo-rule -> BUILD-template plumbing + # for `cc_deps`. Reuses python-geohash's locked sdist under a distinct + # project/hub (//uv-sdist-cc-deps) so the three sibling sdist_build + # snapshots stay byte-stable. Pins that the CcInfo label threads onto + # the generated pep517_native_whl(...) `cc_deps` list. The executable + # end-to-end regression lives in-repo at + # //uv/private/pep517_whl/tests/cc_deps. + "snapshots/sdist_build.uv_sdist_cc_deps.python_geohash.BUILD.bazel": "@sdist_build__uv_sdist_cc_deps__python_geohash__0_8_5//:BUILD.bazel", + # Venv site-packages .pth files. Pin the line shapes that # assemble_venv emits, so any change to `.pth` emission # surfaces here. Wheel site-packages dirs are referenced by diff --git a/e2e/cases/MODULE.bazel b/e2e/cases/MODULE.bazel index 2d0137ca4..c78acd7c8 100644 --- a/e2e/cases/MODULE.bazel +++ b/e2e/cases/MODULE.bazel @@ -153,6 +153,7 @@ include("//uv-platform-filter-844:setup.MODULE.bazel") include("//uv-plus-version:setup.MODULE.bazel") include("//uv-pyproject-cases:setup.MODULE.bazel") include("//uv-requirements-bzl:setup.MODULE.bazel") +include("//uv-sdist-cc-deps:setup.MODULE.bazel") include("//uv-sdist-fallback:setup.MODULE.bazel") include("//uv-sdist-jdk-build:setup.MODULE.bazel") include("//uv-sdist-mpicc:setup.MODULE.bazel") diff --git a/e2e/cases/snapshots/sdist_build.uv_sdist_cc_deps.python_geohash.BUILD.bazel b/e2e/cases/snapshots/sdist_build.uv_sdist_cc_deps.python_geohash.BUILD.bazel new file mode 100644 index 000000000..c38aa38f9 --- /dev/null +++ b/e2e/cases/snapshots/sdist_build.uv_sdist_cc_deps.python_geohash.BUILD.bazel @@ -0,0 +1,26 @@ + +load("@aspect_rules_py//uv/private/pep517_whl:rule.bzl", "pep517_native_whl") +load("@aspect_rules_py//py:defs.bzl", "py_binary") + +py_binary( + name = "build_tool", + main = "@aspect_rules_py//uv/private/pep517_whl:build_helper.py", + srcs = ["@aspect_rules_py//uv/private/pep517_whl:build_helper.py"], + deps = ["@@aspect_rules_py++uv+project__uv_sdist_cc_deps//:build", "@@aspect_rules_py++uv+project__uv_sdist_cc_deps//:setuptools", "@whl_install__uv_sdist_cc_deps__setuptools__75_8_2//:install"], +) + +pep517_native_whl( + name = "whl", + src = "@@aspect_rules_py++uv+sdist__python_geohash__05a21fcf4eda1a5e//file:file", + tool = ":build_tool", + version = "0.8.5", + cc_deps = [ + "@@//uv-sdist-cc-deps:probe", + ], + visibility = ["//visibility:public"], +) + +exports_files( + ["BUILD.bazel"], + visibility = ["//visibility:public"], +) diff --git a/e2e/cases/uv-sdist-cc-deps/BUILD.bazel b/e2e/cases/uv-sdist-cc-deps/BUILD.bazel new file mode 100644 index 000000000..1ffeb856a --- /dev/null +++ b/e2e/cases/uv-sdist-cc-deps/BUILD.bazel @@ -0,0 +1,25 @@ +# Snapshot regression for `uv.override_package(cc_deps = [...])`. +# +# Reuses python-geohash's locked sdist (already fetched by the +# uv-sdist-native-build case) under a distinct project name/hub so the +# generated sdist_build repo is independent. The override attaches `:probe` +# via `cc_deps`; the snapshot in //:snapshots pins that the label threads +# onto the generated pep517_native_whl(...) `cc_deps` list. +# +# Snapshot-only: the wheel is not built here. The executable cc_deps +# regression (real setuptools extension linking a cc_library) lives in-repo +# at //uv/private/pep517_whl/tests/cc_deps. + +load("@rules_cc//cc:defs.bzl", "cc_library") + +cc_library( + name = "probe", + srcs = ["probe.cc"], + hdrs = ["probe.h"], + includes = ["."], + # The generated @sdist_build__uv_sdist_cc_deps__python_geohash__0_8_5 + # repo references this label from its pep517_native_whl(cc_deps = [...]) + # call, so it must be visible outside this package. Public matches how + # sibling cases expose files to generated repos (exports_files default). + visibility = ["//visibility:public"], +) diff --git a/e2e/cases/uv-sdist-cc-deps/probe.cc b/e2e/cases/uv-sdist-cc-deps/probe.cc new file mode 100644 index 000000000..8b24dd046 --- /dev/null +++ b/e2e/cases/uv-sdist-cc-deps/probe.cc @@ -0,0 +1,3 @@ +#include "probe.h" + +int rules_py_cc_deps_probe(void) { return 42; } diff --git a/e2e/cases/uv-sdist-cc-deps/probe.h b/e2e/cases/uv-sdist-cc-deps/probe.h new file mode 100644 index 000000000..4c35667e2 --- /dev/null +++ b/e2e/cases/uv-sdist-cc-deps/probe.h @@ -0,0 +1,16 @@ +#ifndef UV_SDIST_CC_DEPS_PROBE_H_ +#define UV_SDIST_CC_DEPS_PROBE_H_ + +#ifdef __cplusplus +extern "C" { +#endif + +// Returns a constant probe value. Present only so the `cc_deps` override has a +// real CcInfo target to reference; the snapshot never builds this library. +int rules_py_cc_deps_probe(void); + +#ifdef __cplusplus +} +#endif + +#endif // UV_SDIST_CC_DEPS_PROBE_H_ diff --git a/e2e/cases/uv-sdist-cc-deps/pyproject.toml b/e2e/cases/uv-sdist-cc-deps/pyproject.toml new file mode 100644 index 000000000..de45e6310 --- /dev/null +++ b/e2e/cases/uv-sdist-cc-deps/pyproject.toml @@ -0,0 +1,7 @@ +[project] +name = "uv-sdist-cc-deps" +version = "0.0.0" +requires-python = ">=3.11" +dependencies = [ + "python-geohash", +] diff --git a/e2e/cases/uv-sdist-cc-deps/setup.MODULE.bazel b/e2e/cases/uv-sdist-cc-deps/setup.MODULE.bazel new file mode 100644 index 000000000..8d65e909e --- /dev/null +++ b/e2e/cases/uv-sdist-cc-deps/setup.MODULE.bazel @@ -0,0 +1,37 @@ +"""Regression for the per-package `cc_deps` plumbing on +`uv.override_package`. Reuses python-geohash's locked sdist (already +fetched by the uv-sdist-native-build case) under a distinct project +name/hub, so the generated sdist_build repo is independent and the three +existing sdist_build snapshots stay byte-stable. + +The override attaches an in-repo cc_library via `cc_deps`; the snapshot +pins that the label threads onto the generated pep517_native_whl(...) +`cc_deps` list. Snapshot-only: the wheel is not built here; the +executable cc_deps regression lives in-repo under +//uv/private/pep517_whl/tests/cc_deps. +""" + +uv = use_extension("@aspect_rules_py//uv:extensions.bzl", "uv") +uv.declare_hub(hub_name = "pypi_uv_sdist_cc_deps") +uv.project( + default_build_dependencies = [ + "build", + "setuptools", + ], + hub_name = "pypi_uv_sdist_cc_deps", + lock = "//uv-sdist-cc-deps:uv.lock", + pyproject = "//uv-sdist-cc-deps:pyproject.toml", +) + +# Attach an in-repo CcInfo target via `cc_deps`. This is additive on top of +# whatever sdist_build's BUILD template emits by default. +uv.override_package( + name = "python-geohash", + cc_deps = ["//uv-sdist-cc-deps:probe"], + lock = "//uv-sdist-cc-deps:uv.lock", +) + +# Exposed so e2e/BUILD.bazel can snapshot the generated pep517_native_whl +# call: pins that `cc_deps` threads onto the build rule. The hub repo is +# deliberately not imported; no target here consumes it. +use_repo(uv, "sdist_build__uv_sdist_cc_deps__python_geohash__0_8_5") diff --git a/e2e/cases/uv-sdist-cc-deps/uv.lock b/e2e/cases/uv-sdist-cc-deps/uv.lock new file mode 100644 index 000000000..25e8ef77e --- /dev/null +++ b/e2e/cases/uv-sdist-cc-deps/uv.lock @@ -0,0 +1,70 @@ +version = 1 +revision = 3 +requires-python = ">=3.11" + +[[package]] +name = "build" +version = "1.4.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "colorama", marker = "os_name == 'nt'" }, + { name = "packaging" }, + { name = "pyproject-hooks" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/42/18/94eaffda7b329535d91f00fe605ab1f1e5cd68b2074d03f255c7d250687d/build-1.4.0.tar.gz", hash = "sha256:f1b91b925aa322be454f8330c6fb48b465da993d1e7e7e6fa35027ec49f3c936", size = 50054, upload-time = "2026-01-08T16:41:47.696Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/c5/0d/84a4380f930db0010168e0aa7b7a8fed9ba1835a8fbb1472bc6d0201d529/build-1.4.0-py3-none-any.whl", hash = "sha256:6a07c1b8eb6f2b311b96fcbdbce5dab5fe637ffda0fd83c9cac622e927501596", size = 24141, upload-time = "2026-01-08T16:41:46.453Z" }, +] + +[[package]] +name = "colorama" +version = "0.4.6" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/d8/53/6f443c9a4a8358a93a6792e2acffb9d9d5cb0a5cfd8802644b7b1c9a02e4/colorama-0.4.6.tar.gz", hash = "sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44", size = 27697, upload-time = "2022-10-25T02:36:22.414Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/d1/d6/3965ed04c63042e047cb6a3e6ed1a63a35087b6a609aa3a15ed8ac56c221/colorama-0.4.6-py2.py3-none-any.whl", hash = "sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6", size = 25335, upload-time = "2022-10-25T02:36:20.889Z" }, +] + +[[package]] +name = "packaging" +version = "26.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/65/ee/299d360cdc32edc7d2cf530f3accf79c4fca01e96ffc950d8a52213bd8e4/packaging-26.0.tar.gz", hash = "sha256:00243ae351a257117b6a241061796684b084ed1c516a08c48a3f7e147a9d80b4", size = 143416, upload-time = "2026-01-21T20:50:39.064Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/b7/b9/c538f279a4e237a006a2c98387d081e9eb060d203d8ed34467cc0f0b9b53/packaging-26.0-py3-none-any.whl", hash = "sha256:b36f1fef9334a5588b4166f8bcd26a14e521f2b55e6b9de3aaa80d3ff7a37529", size = 74366, upload-time = "2026-01-21T20:50:37.788Z" }, +] + +[[package]] +name = "pyproject-hooks" +version = "1.2.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/e7/82/28175b2414effca1cdac8dc99f76d660e7a4fb0ceefa4b4ab8f5f6742925/pyproject_hooks-1.2.0.tar.gz", hash = "sha256:1e859bd5c40fae9448642dd871adf459e5e2084186e8d2c2a79a824c970da1f8", size = 19228, upload-time = "2024-09-29T09:24:13.293Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/bd/24/12818598c362d7f300f18e74db45963dbcb85150324092410c8b49405e42/pyproject_hooks-1.2.0-py3-none-any.whl", hash = "sha256:9e5c6bfa8dcc30091c74b0cf803c81fdd29d94f01992a7707bc97babb1141913", size = 10216, upload-time = "2024-09-29T09:24:11.978Z" }, +] + +[[package]] +name = "python-geohash" +version = "0.8.5" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/9c/e2/1a3507af7c8f91f8a4975d651d4aeb6a846dfdf74713954186ade4205850/python-geohash-0.8.5.tar.gz", hash = "sha256:05a21fcf4eda1a5eddbd291890ade23fc5ddaa6bb98f2ee23d2d384ed14f086d", size = 17636, upload-time = "2014-08-13T07:09:57.318Z" } + +[[package]] +name = "setuptools" +version = "75.8.2" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/d1/53/43d99d7687e8cdef5ab5f9ec5eaf2c0423c2b35133a2b7e7bc276fc32b21/setuptools-75.8.2.tar.gz", hash = "sha256:4880473a969e5f23f2a2be3646b2dfd84af9028716d398e46192f84bc36900d2", size = 1344083, upload-time = "2025-02-26T20:45:19.103Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/a9/38/7d7362e031bd6dc121e5081d8cb6aa6f6fedf2b67bf889962134c6da4705/setuptools-75.8.2-py3-none-any.whl", hash = "sha256:558e47c15f1811c1fa7adbd0096669bf76c1d3f433f58324df69f3f5ecac4e8f", size = 1229385, upload-time = "2025-02-26T20:45:17.259Z" }, +] + +[[package]] +name = "uv-sdist-cc-deps" +version = "0.0.0" +source = { virtual = "." } +dependencies = [ + { name = "python-geohash" }, +] + +[package.metadata] +requires-dist = [{ name = "python-geohash" }] diff --git a/uv/private/extension/defs.bzl b/uv/private/extension/defs.bzl index a7140664d..f5ad83b23 100644 --- a/uv/private/extension/defs.bzl +++ b/uv/private/extension/defs.bzl @@ -264,13 +264,14 @@ def _parse_projects(module_ctx, hub_specs): override.extra_data or override.toolchains or override.env or + override.cc_deps or override.monitor_memory or override.resource_set != "default" ) if has_target and has_modifications: fail("uv.override_package() for '{}': `target` is mutually exclusive with modification attributes. Use `target` for full replacement OR build, patch, and data attributes for modifications, not both.".format(override.name)) if not has_target and not has_modifications: - fail("uv.override_package() for '{}': must specify either `target` for full replacement or at least one modification attribute (console_scripts, pre_build_patches, post_install_patches, exclude_glob, extra_deps, extra_data, toolchains, env, monitor_memory, resource_set).".format(override.name)) + fail("uv.override_package() for '{}': must specify either `target` for full replacement or at least one modification attribute (console_scripts, pre_build_patches, post_install_patches, exclude_glob, extra_deps, extra_data, toolchains, env, cc_deps, monitor_memory, resource_set).".format(override.name)) unscoped_matches = {i: 0 for i, override in enumerate(mod.tags.override_package) if override.lock == None} @@ -522,6 +523,7 @@ def _parse_projects(module_ctx, hub_specs): pre_build_patch_strip = pkg_override.pre_build_patch_strip, supported = [], toolchains = pkg_override.toolchains, + cc_deps = pkg_override.cc_deps, ) if sdist: # HACK: Note that we resolve these LAZILY so that @@ -563,16 +565,19 @@ def _parse_projects(module_ctx, hub_specs): pre_build_patches = [str(p) for p in pkg_override.pre_build_patches] pre_build_patch_strip = pkg_override.pre_build_patch_strip - # `toolchains` / `env` on `uv.override_package` augment - # the defaults baked into sdist_build's BUILD template — - # they don't replace them. Empty == no augmentation. + # `toolchains` / `env` / `cc_deps` on `uv.override_package` + # augment the defaults baked into sdist_build's BUILD + # template; they don't replace them. Empty == no + # augmentation. extra_toolchains = [] extra_env = {} + extra_cc_deps = [] monitor_memory = False resource_set = "default" if pkg_override: extra_toolchains = [str(t) for t in pkg_override.toolchains] extra_env = pkg_override.env + extra_cc_deps = [str(d) for d in pkg_override.cc_deps] monitor_memory = pkg_override.monitor_memory resource_set = pkg_override.resource_set @@ -586,6 +591,7 @@ def _parse_projects(module_ctx, hub_specs): available_deps = project_available_deps, extra_toolchains = extra_toolchains, extra_env = extra_env, + extra_cc_deps = extra_cc_deps, monitor_memory = monitor_memory, resource_set = resource_set, ) @@ -816,6 +822,8 @@ def _uv_impl(module_ctx): sbuild_kwargs["extra_toolchains"] = sbuild_cfg.extra_toolchains if sbuild_cfg.extra_env: sbuild_kwargs["extra_env"] = sbuild_cfg.extra_env + if sbuild_cfg.extra_cc_deps: + sbuild_kwargs["extra_cc_deps"] = sbuild_cfg.extra_cc_deps if sbuild_cfg.monitor_memory: sbuild_kwargs["monitor_memory"] = True if sbuild_cfg.resource_set != "default": @@ -952,6 +960,10 @@ _override_package_tag = tag_class( default = {}, doc = "Extra environment variables merged into the build action's `env` dict. Values may reference $(VAR) make-variables sourced from extra `toolchains` listed above. Prefix an execroot-relative path with `$(EXECROOT)/` so it remains valid after the backend changes into the unpacked source tree. Omit CC/CXX/AR/LD/STRIP to use the configured C++ action tools.", ), + "cc_deps": attr.label_list( + default = [], + doc = "CcInfo targets whose transitive headers, defines, include paths, and static archives are wired into the native sdist build (compile via CPPFLAGS, archives into the link post-object). Setuptools-backend sdists only. Each label is referenced from the generated external build repository, so the target must be visible to it (e.g. `//visibility:public`).", + ), "pre_build_patches": attr.label_list( default = [], allow_files = [".patch", ".diff"], diff --git a/uv/private/sdist_build/attrs.bzl b/uv/private/sdist_build/attrs.bzl index a876d8033..fab160f81 100644 --- a/uv/private/sdist_build/attrs.bzl +++ b/uv/private/sdist_build/attrs.bzl @@ -9,6 +9,7 @@ def validate_build_attrs( pre_build_patch_strip, supported, toolchains, + cc_deps, error): """Fails when a configured source-build attribute is unsupported. @@ -21,6 +22,7 @@ def validate_build_attrs( pre_build_patch_strip: Strip count for pre-build patches. supported: Names of attributes consumed by the selected build path. toolchains: Toolchains used by the wheel-build action. + cc_deps: CcInfo targets whose headers and archives feed the native build. error: Failure message with one `{}` slot for unsupported names. """ active = [] @@ -38,6 +40,8 @@ def validate_build_attrs( active.append("pre_build_patch_strip") if toolchains: active.append("toolchains") + if cc_deps: + active.append("cc_deps") unsupported = [name for name in active if name not in supported] if unsupported: fail(error.format(", ".join(unsupported))) diff --git a/uv/private/sdist_build/repository.bzl b/uv/private/sdist_build/repository.bzl index 8257f6689..0a2dc336f 100644 --- a/uv/private/sdist_build/repository.bzl +++ b/uv/private/sdist_build/repository.bzl @@ -194,6 +194,7 @@ def _sdist_build_impl(repository_ctx): "resource_set", ], toolchains = repository_ctx.attr.extra_toolchains, + cc_deps = repository_ctx.attr.extra_cc_deps, ) # Resolve additional deps discovered by the configure tool @@ -227,9 +228,11 @@ def _sdist_build_impl(repository_ctx): # AR, LD, and STRIP make variables can be synthetic. Only forward explicit # toolchains/env for JDK, Rust, and other package-specific overrides. toolchain_attrs = "" + cc_deps_attr = "" if is_native: toolchains = repository_ctx.attr.extra_toolchains extra_env = repository_ctx.attr.extra_env + cc_deps = repository_ctx.attr.extra_cc_deps env_attr = "" if extra_env: env_attr = """ @@ -246,6 +249,13 @@ def _sdist_build_impl(repository_ctx): toolchains = "\n".join([" \"{}\",".format(t) for t in toolchains]), ) toolchain_attrs += env_attr + if cc_deps: + cc_deps_attr = """ + cc_deps = [ +{cc_deps} + ],""".format( + cc_deps = "\n".join([" \"{}\",".format(d) for d in cc_deps]), + ) resource_set_attr = "" if repository_ctx.attr.resource_set != "default": @@ -272,7 +282,7 @@ py_binary( name = "whl", src = "{src}", tool = ":build_tool", - version = "{version}",{console_scripts_attr}{monitor_memory_attr}{resource_set_attr}{patch_attrs}{toolchain_attrs} + version = "{version}",{console_scripts_attr}{monitor_memory_attr}{resource_set_attr}{patch_attrs}{toolchain_attrs}{cc_deps_attr} visibility = ["//visibility:public"], ) @@ -290,6 +300,7 @@ exports_files( resource_set_attr = resource_set_attr, patch_attrs = patch_attrs, toolchain_attrs = toolchain_attrs, + cc_deps_attr = cc_deps_attr, )) sdist_build = repository_rule( @@ -331,5 +342,9 @@ sdist_build = repository_rule( default = {}, doc = "Environment variables forwarded to the generated pep517_native_whl(...) `env` dict. Values may reference $(VAR) make-variables from extra toolchains. Prefix an execroot-relative path with `$(EXECROOT)/` so it remains valid after the backend changes into the unpacked source tree. Set via `uv.override_package(env = {...})`.", ), + "extra_cc_deps": attr.string_list( + default = [], + doc = "CcInfo target labels forwarded to the generated pep517_native_whl(...) `cc_deps` list, wiring their transitive headers and static archives into the native sdist build. Set via `uv.override_package(cc_deps = [...])`.", + ), }, ) From a5ace259614ae25c85df4223f62d4d678ffb7009 Mon Sep 17 00:00:00 2001 From: Connor McEntee Date: Mon, 27 Jul 2026 20:14:29 +0000 Subject: [PATCH 4/5] test(uv): prove cc_deps end-to-end with a real setuptools extension Add an executable regression that actually compiles, links, imports, and calls through a real setuptools.build_meta extension built with pep517_native_whl(cc_deps = [...]). This is the executable proof of the feature: it exercises the analysis params file and the build_helper runtime adapter together, past the setuptools backend chdir. The fixture is a two-level cc_library chain (dep -> dep2) whose headers are deliberately absent from the sdist: the extension's #include resolves only through the cc_deps include path, and dep_value/dep2_value link only through the post-object cc_deps archive slot. The producer-side-only slots are covered red-capably and hermetically: dep carries linkopts = ["-lextra"], resolved against a libextra.a that the sdist's own setuptools build_clib compiles from extra.c inside the wheel build (no host library: -lm cannot go red because CPython pre-loads libm, and -lz would require host zlib1g-dev). Because distutils' build_ext auto-links every build_clib library name into every extension (verified: with that auto-link, dropping the linkopt kept the test green), the fixture's setup.py suppresses build_clib.get_library_names(), so naming -lextra on the link is carried exclusively by the injected [build_ext] libraries slot. dep also carries defines = ["MOD_BONUS=2"], which propagate transitively into the backend's compile of mod.c via cc_deps CPPFLAGS, guarded by #ifndef MOD_BONUS #error. Two more sdist-built archives, libgroup_a.a and libgroup_b.a, form a deliberate one-pass cycle resolved by the documented repeat-the-library workaround (-lgroup_a -lgroup_b -lgroup_a), proving -l entries keep their relative order through setuptools' libraries slot. import_test extracts the built wheel, imports the module from it (asserted via __file__), and checks the composed value: dep2 40 + extra_value(7) 17 + MOD_BONUS 2 + group 125 == 184. backend_gate_test runs build_helper.py as a subprocess against a non-setuptools backend and asserts it fails fast. gates_test covers the remaining runtime gates the same way, each against its own message so an unrelated exit 1 fails: pre-existing DIST_EXTRA_CONFIG, SETUPTOOLS_USE_DISTUTILS=stdlib, whitespace-in-link-object, execroot-marker survival, and the setuptools >= 65.4.0 floor (driven to an exact version by a controlled fake dist-info on PYTHONPATH). The highest value case is the setup.cfg [build_ext] REPLACE-merge: a crafted sdist whose setup.cfg already sets link_objects/libraries, run so the expected build failure leaves tmp_root behind, then cc_deps_extra.cfg is read and asserted to keep the package's own values PRECEDING ours while include_dirs/define never leak into the cfg (the scariest silent failure would be dropped archives). Because the wheel build needs a real setuptools.build_meta backend under --no-isolation, it uses a testonly __build_helper_setuptools variant of the helper (setuptools added from the dev lock); the production __build_helper is untouched. import_test is Linux-gated because the archives are folded into a shared object. --- uv/private/pep517_whl/BUILD.bazel | 15 + .../tests/cc_deps/executable/BUILD.bazel | 140 ++++++++ .../cc_deps/executable/backend_gate_test.py | 93 ++++++ .../pep517_whl/tests/cc_deps/executable/dep.c | 20 ++ .../pep517_whl/tests/cc_deps/executable/dep.h | 6 + .../tests/cc_deps/executable/dep2.c | 3 + .../tests/cc_deps/executable/dep2.h | 6 + .../tests/cc_deps/executable/gates_test.py | 307 ++++++++++++++++++ .../tests/cc_deps/executable/import_test.py | 67 ++++ .../tests/cc_deps/executable/sdist/extra.c | 6 + .../cc_deps/executable/sdist/group_a_entry.c | 3 + .../cc_deps/executable/sdist/group_a_tail.c | 1 + .../tests/cc_deps/executable/sdist/group_b.c | 3 + .../tests/cc_deps/executable/sdist/mod.c | 44 +++ .../cc_deps/executable/sdist/pyproject.toml | 7 + .../tests/cc_deps/executable/sdist/setup.py | 45 +++ 16 files changed, 766 insertions(+) create mode 100644 uv/private/pep517_whl/tests/cc_deps/executable/BUILD.bazel create mode 100644 uv/private/pep517_whl/tests/cc_deps/executable/backend_gate_test.py create mode 100644 uv/private/pep517_whl/tests/cc_deps/executable/dep.c create mode 100644 uv/private/pep517_whl/tests/cc_deps/executable/dep.h create mode 100644 uv/private/pep517_whl/tests/cc_deps/executable/dep2.c create mode 100644 uv/private/pep517_whl/tests/cc_deps/executable/dep2.h create mode 100644 uv/private/pep517_whl/tests/cc_deps/executable/gates_test.py create mode 100644 uv/private/pep517_whl/tests/cc_deps/executable/import_test.py create mode 100644 uv/private/pep517_whl/tests/cc_deps/executable/sdist/extra.c create mode 100644 uv/private/pep517_whl/tests/cc_deps/executable/sdist/group_a_entry.c create mode 100644 uv/private/pep517_whl/tests/cc_deps/executable/sdist/group_a_tail.c create mode 100644 uv/private/pep517_whl/tests/cc_deps/executable/sdist/group_b.c create mode 100644 uv/private/pep517_whl/tests/cc_deps/executable/sdist/mod.c create mode 100644 uv/private/pep517_whl/tests/cc_deps/executable/sdist/pyproject.toml create mode 100644 uv/private/pep517_whl/tests/cc_deps/executable/sdist/setup.py diff --git a/uv/private/pep517_whl/BUILD.bazel b/uv/private/pep517_whl/BUILD.bazel index 22b0a68de..a50bd0835 100644 --- a/uv/private/pep517_whl/BUILD.bazel +++ b/uv/private/pep517_whl/BUILD.bazel @@ -69,6 +69,21 @@ py_binary( deps = ["@pypi//build"], ) +# Same helper source, but with setuptools in the build venv. The production +# tool above deliberately ships only @pypi//build; the executable cc_deps +# regression under tests/cc_deps/executable needs a real setuptools.build_meta +# backend to run under --no-isolation, so it uses this variant. +py_binary( + name = "__build_helper_setuptools", + testonly = True, + srcs = ["build_helper.py"], + main = "build_helper.py", + deps = [ + "@pypi//build", + "@pypi//setuptools", + ], +) + _PYTHON_ENV_BACKEND_SRCS = glob(["tests/python_env_backend/*"]) mtree_spec( diff --git a/uv/private/pep517_whl/tests/cc_deps/executable/BUILD.bazel b/uv/private/pep517_whl/tests/cc_deps/executable/BUILD.bazel new file mode 100644 index 000000000..0ee8a85ae --- /dev/null +++ b/uv/private/pep517_whl/tests/cc_deps/executable/BUILD.bazel @@ -0,0 +1,140 @@ +load("@rules_cc//cc:defs.bzl", "cc_library") +load("@tar.bzl//tar:mtree.bzl", "mtree_mutate", "mtree_spec") +load("@tar.bzl//tar:tar.bzl", "tar_rule") +load("//py:defs.bzl", "py_test") +load("//uv/private/pep517_whl:rule.bzl", "pep517_native_whl") + +# The executable cc_deps regression lives in its own package: each successful +# pep517_native_whl target declares a `whl` output directory, so it cannot share +# a package with the analysis-only fixtures in the parent cc_deps package (they +# own one already). Unlike those fixtures, these libraries are really compiled +# and linked into a wheel that is imported and called through. + +# A two-level chain: dep -> dep2. dep_value(x) returns dep2_value() plus +# extra_value(x) (compiled by the sdist's own build_clib into libextra.a), and +# mod.c adds MOD_BONUS. The sdist also builds libgroup_a.a / libgroup_b.a with a +# deliberate one-pass archive cycle; the -lgroup_a, -lgroup_b, -lgroup_a stream +# below (the documented repeat-the-library workaround for cycles) is required to +# resolve it, and proves -l order is preserved through the libraries slot. +cc_library( + name = "dep2", + testonly = True, + srcs = ["dep2.c"], + hdrs = ["dep2.h"], + includes = ["."], +) + +cc_library( + name = "dep", + testonly = True, + srcs = ["dep.c"], + hdrs = ["dep.h"], + # defines propagate transitively to the backend's compile of mod.c through + # cc_deps' CPPFLAGS routing (the -D path). The libraries resolve against + # archives the sdist's build_clib compiles hermetically in-build; setup.py + # suppresses distutils' automatic linking, so this exact linkopts stream is + # solely responsible for naming them. libgroup_a.a and libgroup_b.a form a + # cycle: group_a_entry.o -> group_b.o -> group_a_tail.o, so under single-pass + # GNU ld the linker must rescan libgroup_a.a after libgroup_b.a. Order- + # sensitive --start-group/--end-group cannot survive cc_deps' two-slot + # flattening (rejected at analysis), so the cycle is resolved with the + # documented workaround instead: repeat -lgroup_a after -lgroup_b. This also + # proves -l order is preserved through the [build_ext] libraries slot. + defines = ["MOD_BONUS=2"], + includes = ["."], + linkopts = [ + "-lextra", + "-lgroup_a", + "-lgroup_b", + "-lgroup_a", + ], + deps = [":dep2"], +) + +# A real setuptools sdist. dep.h is intentionally NOT part of the sources: the +# extension resolves it only through the cc_deps include path, proving the +# header injection rather than an accidental in-tree copy. +_SDIST_SRCS = glob( + ["sdist/**"], + exclude = [ + "sdist/**/__pycache__/**", + "sdist/**/*.pyc", + ], +) + +mtree_spec( + name = "sdist_mtree_raw", + testonly = True, + srcs = _SDIST_SRCS, + include_runfiles = False, +) + +mtree_mutate( + name = "sdist_mtree", + testonly = True, + mtree = ":sdist_mtree_raw", + strip_prefix = package_name(), +) + +tar_rule( + name = "sdist", + testonly = True, + srcs = _SDIST_SRCS, + mtree = ":sdist_mtree", +) + +# Build the wheel with the setuptools-equipped helper (the production +# __build_helper ships only @pypi//build; a real setuptools.build_meta backend +# under --no-isolation needs setuptools importable). +pep517_native_whl( + name = "whl", + testonly = True, + src = ":sdist", + cc_deps = [":dep"], + tags = ["manual"], + tool = "//uv/private/pep517_whl:__build_helper_setuptools", + version = "0.0.1", +) + +py_test( + name = "import_test", + srcs = ["import_test.py"], + data = [":whl"], + main = "import_test.py", + # The wheel's extension .so is ABI-tagged for the build interpreter; both the + # build and this test resolve the same rules_py Python toolchain. Static + # archives are linked into a shared object, so gate on Linux like the geohash + # native-import regression. + target_compatible_with = ["@platforms//os:linux"], + deps = ["@pypi//bazel_runfiles"], +) + +# build_helper.py prefers stdlib tomllib and only falls back to tomli below +# Python 3.11; the tomli dep is dead on a >=3.11 toolchain and lives only for the +# cp39 rules_py default toolchain these helper subprocesses run under. +_HELPER_SUBPROCESS_DEPS = [ + "@pypi//bazel_runfiles", + "@pypi//tomli", +] + +# Negative: build_helper.py must reject cc_deps against a non-setuptools backend +# rather than silently drop the inputs. Runs the helper as a subprocess; the +# sdist tar and cc-deps params file are synthesized in the test. +py_test( + name = "backend_gate_test", + srcs = ["backend_gate_test.py"], + data = ["//uv/private/pep517_whl:build_helper.py"], + main = "backend_gate_test.py", + deps = _HELPER_SUBPROCESS_DEPS, +) + +# The rest of the runtime gate matrix + the setup.cfg [build_ext] REPLACE-merge, +# all driven by running build_helper.py as a subprocess against synthetic sdists +# (same technique as backend_gate_test). No compiler needed, so ungated. +py_test( + name = "gates_test", + srcs = ["gates_test.py"], + data = ["//uv/private/pep517_whl:build_helper.py"], + main = "gates_test.py", + deps = _HELPER_SUBPROCESS_DEPS, +) diff --git a/uv/private/pep517_whl/tests/cc_deps/executable/backend_gate_test.py b/uv/private/pep517_whl/tests/cc_deps/executable/backend_gate_test.py new file mode 100644 index 000000000..177d49404 --- /dev/null +++ b/uv/private/pep517_whl/tests/cc_deps/executable/backend_gate_test.py @@ -0,0 +1,93 @@ +"""Runtime gate: build_helper.py must refuse cc_deps on a non-setuptools backend. + +cc_deps injects setuptools [build_ext] settings that other PEP 517 backends +ignore, so silently dropping the inputs would produce a wheel that fails to +link at import time. build_helper must instead fail fast with an actionable +message. This runs build_helper.py directly as a subprocess against a synthetic +sdist whose declared backend is not setuptools, plus a crafted cc-deps params +file, and asserts exit 1 with the backend-gate message. +""" + +import json +import os +import subprocess +import sys +import tarfile +import tempfile + +import runfiles + +_MARKER = "__ASPECT_RULES_PY_EXECROOT__" +_HELPER = "_main/uv/private/pep517_whl/build_helper.py" + + +def main() -> None: + r = runfiles.Create() + helper = r.Rlocation(_HELPER) + assert helper and os.path.isfile(helper), "build_helper.py missing: {}".format(helper) + + work = tempfile.mkdtemp(dir=os.environ["TEST_TMPDIR"]) + + # A synthetic sdist whose backend is deliberately not setuptools. + src_dir = os.path.join(work, "pkg") + os.makedirs(src_dir) + with open(os.path.join(src_dir, "pyproject.toml"), "w") as f: + f.write( + "[build-system]\n" + "requires = []\n" + 'build-backend = "not_setuptools.api"\n' + 'backend-path = ["."]\n' + ) + sdist = os.path.join(work, "sdist.tar.gz") + with tarfile.open(sdist, "w:gz") as tar: + tar.add(src_dir, arcname="pkg") + + # A minimal but non-empty cc-deps params file: cc_deps is present, so the + # gate must fire rather than no-op. + info = os.path.join(work, "cc_deps_info.json") + with open(info, "w") as f: + json.dump( + { + "compile_flags": ["-I{}/some/include".format(_MARKER)], + "link_objects": ["{}/some/libdep.a".format(_MARKER)], + "link_libraries": [], + "link_flags": [], + }, + f, + ) + + outdir = os.path.join(work, "out") + + # Deterministic by construction: a minimal env built from scratch (not a + # filtered copy of os.environ), so an ambient DIST_EXTRA_CONFIG or + # SETUPTOOLS_USE_DISTUTILS on the host cannot perturb the helper. The gate + # path needs no host tools; build_helper falls back to os.defpath itself. + subprocess_env = {"PATH": os.defpath} + + proc = subprocess.run( + [ + sys.executable, + helper, + "--cc-deps-info", info, + "--execroot-marker", _MARKER, + sdist, + outdir, + ], + cwd=work, + env=subprocess_env, + capture_output=True, + text=True, + ) + + assert proc.returncode == 1, \ + "expected exit 1, got {}; stderr:\n{}".format(proc.returncode, proc.stderr) + assert "cc_deps is only supported with the setuptools build backend" in proc.stderr, \ + "missing backend-gate error; stderr:\n{}".format(proc.stderr) + assert "not_setuptools.api" in proc.stderr, \ + "error should name the offending backend; stderr:\n{}".format(proc.stderr) + + print("ok: backend gate fired with exit 1") + + +if __name__ == "__main__": + main() diff --git a/uv/private/pep517_whl/tests/cc_deps/executable/dep.c b/uv/private/pep517_whl/tests/cc_deps/executable/dep.c new file mode 100644 index 000000000..1f86c8b23 --- /dev/null +++ b/uv/private/pep517_whl/tests/cc_deps/executable/dep.c @@ -0,0 +1,20 @@ +#include "dep.h" + +#include "dep2.h" + +/* extra_value() is defined in extra.c INSIDE the sdist, where setuptools' + * build_clib compiles it into libextra.a during the wheel build (hermetic: no + * host library involved). Only a forward declaration here: the `dep` cc_library + * carries linkopts = ["-lextra"], which cc_deps routes through the setuptools + * [build_ext] libraries slot; the sdist's build_clib subclass suppresses + * distutils' auto -l for clib libraries, so dropping that linkopt leaves + * extra_value undefined at import. The [build_ext] libraries slot is what + * carries the link. */ +extern int extra_value(int x); + +/* dep_value() calls through the transitive leaf dep2_value() and the + * build_clib-built extra_value() on an argument-derived value (extra_value is + * external, so the call cannot be folded): a wheel that imports and returns the + * expected value proves both archives linked in order AND that -lextra reached + * the link through the [build_ext] libraries slot. */ +int dep_value(int x) { return dep2_value() + extra_value(x); } diff --git a/uv/private/pep517_whl/tests/cc_deps/executable/dep.h b/uv/private/pep517_whl/tests/cc_deps/executable/dep.h new file mode 100644 index 000000000..743315c88 --- /dev/null +++ b/uv/private/pep517_whl/tests/cc_deps/executable/dep.h @@ -0,0 +1,6 @@ +#ifndef ASPECT_RULES_PY_TEST_CC_DEPS_DEP_H +#define ASPECT_RULES_PY_TEST_CC_DEPS_DEP_H + +int dep_value(int x); + +#endif diff --git a/uv/private/pep517_whl/tests/cc_deps/executable/dep2.c b/uv/private/pep517_whl/tests/cc_deps/executable/dep2.c new file mode 100644 index 000000000..cfa2a7153 --- /dev/null +++ b/uv/private/pep517_whl/tests/cc_deps/executable/dep2.c @@ -0,0 +1,3 @@ +#include "dep2.h" + +int dep2_value(void) { return 40; } diff --git a/uv/private/pep517_whl/tests/cc_deps/executable/dep2.h b/uv/private/pep517_whl/tests/cc_deps/executable/dep2.h new file mode 100644 index 000000000..81e17a8da --- /dev/null +++ b/uv/private/pep517_whl/tests/cc_deps/executable/dep2.h @@ -0,0 +1,6 @@ +#ifndef ASPECT_RULES_PY_TEST_CC_DEPS_DEP2_H +#define ASPECT_RULES_PY_TEST_CC_DEPS_DEP2_H + +int dep2_value(void); + +#endif diff --git a/uv/private/pep517_whl/tests/cc_deps/executable/gates_test.py b/uv/private/pep517_whl/tests/cc_deps/executable/gates_test.py new file mode 100644 index 000000000..fa64651f1 --- /dev/null +++ b/uv/private/pep517_whl/tests/cc_deps/executable/gates_test.py @@ -0,0 +1,307 @@ +"""Runtime gate matrix + setup.cfg [build_ext] REPLACE-merge coverage. + +Extends backend_gate_test.py's technique (run build_helper.py as a subprocess +against a synthetic sdist + crafted --cc-deps-info JSON) to the cc_deps runtime +gates that were previously untested, and to the DIST_EXTRA_CONFIG merge: + + * setup.cfg [build_ext] REPLACE-merge: the package's own link_objects / + libraries values survive, PRECEDE ours, and include_dirs/define never leak + into our generated cfg (the scariest silent failure would be dropped + archives), + * the DIST_EXTRA_CONFIG-preexists, SETUPTOOLS_USE_DISTUTILS=stdlib, whitespace, + execroot-marker-survival, and setuptools-floor gates. + +Each case asserts the gate's OWN message, so an unrelated exit 1 fails the test. + +The subprocess env is built from scratch (PATH + a PYTHONPATH that only prepends +a controlled fake setuptools dist-info), so an ambient DIST_EXTRA_CONFIG / +SETUPTOOLS_USE_DISTUTILS on the host cannot perturb the helper. tomli still +resolves from this test's own venv (the runfiles interpreter), exactly as it +does for backend_gate_test. +""" + +from __future__ import annotations + +import json +import os +import subprocess +import sys +import tarfile +import tempfile + +import runfiles + +_MARKER = "__ASPECT_RULES_PY_EXECROOT__" +_HELPER = "_main/uv/private/pep517_whl/build_helper.py" + + +def _helper_path() -> str: + r = runfiles.Create() + helper = r.Rlocation(_HELPER) + assert helper and os.path.isfile(helper), "build_helper.py missing: {}".format(helper) + return helper + + +def _work(name: str) -> str: + return tempfile.mkdtemp(prefix=name + "_", dir=os.environ["TEST_TMPDIR"]) + + +def _setuptools_sdist(work: str, setup_cfg: str | None = None) -> str: + """Write a minimal setuptools-backend sdist tar (single top-level `pkg/`).""" + src = os.path.join(work, "src") + os.makedirs(src) + with open(os.path.join(src, "pyproject.toml"), "w") as f: + f.write( + "[build-system]\n" + 'requires = ["setuptools"]\n' + 'build-backend = "setuptools.build_meta"\n' + ) + if setup_cfg is not None: + with open(os.path.join(src, "setup.cfg"), "w") as f: + f.write(setup_cfg) + sdist = os.path.join(work, "sdist.tar") + with tarfile.open(sdist, "w") as tar: + tar.add(src, arcname="pkg") + return sdist + + +def _fake_setuptools(work: str, version: str) -> str: + """Create a bare `setuptools-.dist-info` dir; return the parent path. + + importlib.metadata.version() only reads METADATA, so a standalone dist-info + on sys.path is enough to drive the floor check to an exact version. Prepended + to PYTHONPATH, it wins over any setuptools the interpreter might ship. + """ + parent = os.path.join(work, "fake_sp") + info = os.path.join(parent, "setuptools-{}.dist-info".format(version)) + os.makedirs(info) + with open(os.path.join(info, "METADATA"), "w") as f: + f.write("Metadata-Version: 2.1\nName: setuptools\nVersion: {}\n".format(version)) + return parent + + +def _info(work: str, **kwargs: list[str]) -> str: + payload = {"compile_flags": [], "link_objects": [], "link_libraries": [], "link_flags": []} + payload.update(kwargs) + info = os.path.join(work, "cc_deps_info.json") + with open(info, "w") as f: + json.dump(payload, f) + return info + + +def _env(fake_setuptools_dir: str | None = None, **extra: str) -> dict[str, str]: + env = {"PATH": os.defpath} + if fake_setuptools_dir: + env["PYTHONPATH"] = fake_setuptools_dir + env.update(extra) + return env + + +def _run( + helper: str, + sdist: str, + info: str, + outdir: str, + cwd: str, + env: dict[str, str], +) -> subprocess.CompletedProcess[str]: + return subprocess.run( + [ + sys.executable, + helper, + "--cc-deps-info", info, + "--execroot-marker", _MARKER, + sdist, + outdir, + ], + cwd=cwd, + env=env, + capture_output=True, + text=True, + ) + + +def _assert_gate(proc: subprocess.CompletedProcess[str], needle: str) -> None: + assert proc.returncode == 1, \ + "expected exit 1, got {}; stderr:\n{}".format(proc.returncode, proc.stderr) + assert needle in proc.stderr, \ + "expected gate message {!r}; stderr:\n{}".format(needle, proc.stderr) + + +def check_setup_cfg_replace_merge(helper: str) -> None: + """G1: our generated [build_ext] cfg folds the package's values in front and + never carries include_dirs/define. Read tmp_root's cfg after the (expected) + build failure: build_helper writes it during env prep, before invoking + `python -m build`, and leaves tmp_root behind when the build fails.""" + work = _work("g1") + fake = _fake_setuptools(work, "70.0.0") + setup_cfg = ( + "[build_ext]\n" + "link_objects = pkg/vendor/libpkg.a\n" + "libraries = pkgfoo\n" + "include_dirs = pkg/include\n" + "define = PKG_FLAG\n" + ) + sdist = _setuptools_sdist(work, setup_cfg=setup_cfg) + info = _info( + work, + compile_flags=["-I{}/gen/include".format(_MARKER), "-DOURDEF=1"], + link_objects=["{}/gen/libdep.a".format(_MARKER)], + link_libraries=["ourlib"], + ) + outdir = os.path.join(work, "out") + proc = _run(helper, sdist, info, outdir, cwd=work, env=_env(fake)) + + # No `build` module in this bare env, so `python -m build` fails AFTER the + # cfg is written; that non-zero exit is expected and confirms the write ran. + assert proc.returncode == 1, \ + "expected the (post-cfg) build to fail; stderr:\n{}".format(proc.stderr) + + # DIST_EXTRA_CONFIG is set only in the child's build_env (not printed), so it + # is not observable from here; assert the cfg exists at the documented + # tmp_root path it would point at instead. + cfg_path = os.path.join(os.path.abspath(outdir) + ".tmp", "cc_deps_extra.cfg") + assert os.path.isfile(cfg_path), \ + "cc_deps_extra.cfg not written at {}; stderr:\n{}".format(cfg_path, proc.stderr) + with open(cfg_path) as f: + cfg = f.read() + + lines = cfg.splitlines() + lo = next(line for line in lines if line.startswith("link_objects")) + lib = next(line for line in lines if line.startswith("libraries")) + + # cwd == execroot here, so the marker expands to `work`. + our_obj = "{}/gen/libdep.a".format(work) + + # (a) the package's own values are present and PRECEDE ours ... + assert "pkg/vendor/libpkg.a" in lo and our_obj in lo, \ + "link_objects should carry both the package archive and ours; got {!r}".format(lo) + assert lo.index("pkg/vendor/libpkg.a") < lo.index(our_obj), \ + "package link_objects should precede ours; got {!r}".format(lo) + assert "pkgfoo" in lib and "ourlib" in lib, \ + "libraries should carry both the package lib and ours; got {!r}".format(lib) + assert lib.index("pkgfoo") < lib.index("ourlib"), \ + "package libraries should precede ours; got {!r}".format(lib) + + # (b/c) include_dirs and define ride CPPFLAGS, never the cfg, so the package's + # own include_dirs/define must NOT be echoed into our generated cfg. + assert "include_dirs" not in cfg, "cfg must not carry include_dirs; got:\n{}".format(cfg) + assert "define" not in cfg, "cfg must not carry define; got:\n{}".format(cfg) + + print("ok: setup.cfg [build_ext] REPLACE-merge folds package values in front") + + +def check_dist_extra_config_preexists(helper: str) -> None: + """G4(i): a DIST_EXTRA_CONFIG already in the env is a hard error (v1 owns it).""" + work = _work("g4_dec") + fake = _fake_setuptools(work, "70.0.0") + sdist = _setuptools_sdist(work) + info = _info(work, link_objects=["{}/gen/libdep.a".format(_MARKER)]) + env = _env(fake, DIST_EXTRA_CONFIG=os.path.join(work, "user.cfg")) + proc = _run(helper, sdist, info, os.path.join(work, "out"), cwd=work, env=env) + _assert_gate(proc, "already set in the build environment") + print("ok: pre-existing DIST_EXTRA_CONFIG rejected") + + +def check_use_distutils_stdlib(helper: str) -> None: + """G4(ii): SETUPTOOLS_USE_DISTUTILS=stdlib disables the DIST_EXTRA_CONFIG path.""" + work = _work("g4_ud") + fake = _fake_setuptools(work, "70.0.0") + sdist = _setuptools_sdist(work) + info = _info(work, link_objects=["{}/gen/libdep.a".format(_MARKER)]) + env = _env(fake, SETUPTOOLS_USE_DISTUTILS="stdlib") + proc = _run(helper, sdist, info, os.path.join(work, "out"), cwd=work, env=env) + _assert_gate(proc, "SETUPTOOLS_USE_DISTUTILS=stdlib is set") + print("ok: SETUPTOOLS_USE_DISTUTILS=stdlib rejected") + + +def check_whitespace_guard(helper: str) -> None: + """G4(iii): a whitespace-bearing link object is rejected, naming the path.""" + work = _work("g4_ws") + fake = _fake_setuptools(work, "70.0.0") + sdist = _setuptools_sdist(work) + info = _info(work, link_objects=["{}/gen dir/libdep.a".format(_MARKER)]) + proc = _run(helper, sdist, info, os.path.join(work, "out"), cwd=work, env=_env(fake)) + _assert_gate(proc, "contains whitespace") + assert "gen dir/libdep.a" in proc.stderr, \ + "whitespace error should name the offending path; stderr:\n{}".format(proc.stderr) + print("ok: whitespace-bearing link object rejected, path named") + + +def check_framework_whitespace_guard(helper: str) -> None: + """A whitespace-bearing framework (-F) search path is rejected, naming it. + + Apple `-F` paths ride CPPFLAGS, which is word-split downstream, so a spaced + -F path is unrepresentable; it is caught by the same compile-path guard that + covers -I/-iquote/-isystem.""" + work = _work("g4_fw") + fake = _fake_setuptools(work, "70.0.0") + sdist = _setuptools_sdist(work) + info = _info(work, compile_flags=["-F{}/gen dir/Frameworks".format(_MARKER)]) + proc = _run(helper, sdist, info, os.path.join(work, "out"), cwd=work, env=_env(fake)) + _assert_gate(proc, "contains whitespace") + assert "gen dir/Frameworks" in proc.stderr, \ + "framework whitespace error should name the offending path; stderr:\n{}".format(proc.stderr) + print("ok: whitespace-bearing framework search path rejected, path named") + + +def check_link_flag_whitespace_guard(helper: str) -> None: + """A whitespace-bearing link flag is rejected even when it is not an + execroot-anchored path. LDFLAGS is word-split downstream, so no single flag + can carry a space, regardless of where the space came from.""" + work = _work("g4_lf") + fake = _fake_setuptools(work, "70.0.0") + sdist = _setuptools_sdist(work) + info = _info(work, link_flags=["-Wl,-rpath,/opt/my libs"]) + proc = _run(helper, sdist, info, os.path.join(work, "out"), cwd=work, env=_env(fake)) + _assert_gate(proc, "contains whitespace") + assert "/opt/my libs" in proc.stderr, \ + "link-flag whitespace error should name the offending flag; stderr:\n{}".format(proc.stderr) + print("ok: whitespace-bearing link flag rejected, flag named") + + +def check_marker_survival(helper: str) -> None: + """G4(iv): if the execroot itself contains the marker token, substitution + cannot complete and build_helper must fail rather than emit a corrupt path. + Triggered honestly by running with a cwd whose path contains the marker.""" + parent = _work("g4_surv") + work = os.path.join(parent, "dir_{}_here".format(_MARKER)) + os.makedirs(work) + # No fake setuptools needed: this gate fires in _load_cc_deps_info, before + # the backend/floor checks. + sdist = _setuptools_sdist(work) + info = _info(work, link_objects=["{}/gen/libdep.a".format(_MARKER)]) + proc = _run(helper, sdist, info, os.path.join(work, "out"), cwd=work, env=_env()) + _assert_gate(proc, "execroot marker survived substitution") + print("ok: surviving execroot marker rejected") + + +def check_setuptools_floor(helper: str) -> None: + """G4 floor gate: setuptools below the DIST_EXTRA_CONFIG floor (65.4.0) is a + hard error. Driven by a fake old dist-info prepended to PYTHONPATH.""" + work = _work("g4_floor") + fake = _fake_setuptools(work, "60.0.0") + sdist = _setuptools_sdist(work) + info = _info(work, link_objects=["{}/gen/libdep.a".format(_MARKER)]) + proc = _run(helper, sdist, info, os.path.join(work, "out"), cwd=work, env=_env(fake)) + _assert_gate(proc, "requires setuptools >= 65.4.0") + assert "60.0.0" in proc.stderr, \ + "floor error should report the offending version; stderr:\n{}".format(proc.stderr) + print("ok: sub-floor setuptools rejected") + + +def main() -> None: + helper = _helper_path() + check_setup_cfg_replace_merge(helper) + check_dist_extra_config_preexists(helper) + check_use_distutils_stdlib(helper) + check_whitespace_guard(helper) + check_framework_whitespace_guard(helper) + check_link_flag_whitespace_guard(helper) + check_marker_survival(helper) + check_setuptools_floor(helper) + print("ok: all cc_deps gate/merge cases fired") + + +if __name__ == "__main__": + main() diff --git a/uv/private/pep517_whl/tests/cc_deps/executable/import_test.py b/uv/private/pep517_whl/tests/cc_deps/executable/import_test.py new file mode 100644 index 000000000..7a1ee575f --- /dev/null +++ b/uv/private/pep517_whl/tests/cc_deps/executable/import_test.py @@ -0,0 +1,67 @@ +"""Import a real setuptools extension built via pep517_native_whl(cc_deps=...). + +End-to-end proof that, after the setuptools backend changes into the unpacked +sdist: + * resolved through the cc_deps include path (CPPFLAGS) even though the + header is deliberately absent from the sdist, + * dep_value() linked through the post-object cc_deps archive slot, + * the transitive leaf dep2_value() linked too, + * -lextra reached the link through the [build_ext] libraries slot, resolving + against the libextra.a the sdist's own build_clib compiled in-build, and + * -lgroup_a, -lgroup_b, -lgroup_a retained their relative order through the + [build_ext] libraries slot, resolving a deliberate one-pass archive cycle + via the documented repeat-the-library workaround, and + * MOD_BONUS arrived via a transitively-propagated cc_deps -D define. + +The module is loaded from the freshly extracted wheel (asserted via __file__), +so a stale ambient build cannot satisfy the test. +""" + +import glob +import os +import sys +import zipfile + +import runfiles + +_WHL_DIR = "_main/uv/private/pep517_whl/tests/cc_deps/executable/whl" + + +def main() -> None: + r = runfiles.Create() + assert r is not None, "runfiles unavailable" + + whl_dir = r.Rlocation(_WHL_DIR) + assert whl_dir and os.path.isdir(whl_dir), "wheel output dir missing: {}".format(whl_dir) + + wheels = glob.glob(os.path.join(whl_dir, "*.whl")) + assert len(wheels) == 1, "expected exactly one wheel, got {}".format(wheels) + + extract_dir = os.path.join(os.environ["TEST_TMPDIR"], "wheel_extract") + with zipfile.ZipFile(wheels[0]) as archive: + archive.extractall(extract_dir) + + sys.path.insert(0, extract_dir) + import cc_deps_ext + + # The extension .so must come from the wheel we just extracted, not from any + # ambient copy that happened to be importable. + module_file = os.path.realpath(cc_deps_ext.__file__) + assert module_file.startswith(os.path.realpath(extract_dir) + os.sep), \ + "module loaded from unexpected location: {}".format(module_file) + + result = cc_deps_ext.value() + # dep_value(7) = dep2_value() 40 + extra_value(7) 17 (7 + 10, via -lextra); + # MOD_BONUS contributes 2; and the ordered cyclic archives contribute 125 + # (group_entry 100 + group_b 20 + group_a_tail 5). Each contribution is + # distinct: 40 + 17 + 2 + 125 == 184. + expected = 40 + (7 + 10) + 2 + 125 + assert result == expected, \ + "expected cc_deps_ext.value() == {} (dep2 40 + extra 17 + MOD_BONUS 2 + group 125), got {}".format( + expected, result) + + print("ok: cc_deps_ext.value() == {} loaded from {}".format(result, module_file)) + + +if __name__ == "__main__": + main() diff --git a/uv/private/pep517_whl/tests/cc_deps/executable/sdist/extra.c b/uv/private/pep517_whl/tests/cc_deps/executable/sdist/extra.c new file mode 100644 index 000000000..d932e3451 --- /dev/null +++ b/uv/private/pep517_whl/tests/cc_deps/executable/sdist/extra.c @@ -0,0 +1,6 @@ +/* Compiled by setuptools' build_clib (declared via setup(libraries=...)) into + * libextra.a inside the wheel build, before build_ext runs (hermetically, by + * the same configured toolchain). dep.c (outside the sdist, linked via cc_deps) + * forward-declares and calls extra_value(); the -lextra that resolves it is + * injected through the cc_deps [build_ext] libraries slot. */ +int extra_value(int x) { return x + 10; } diff --git a/uv/private/pep517_whl/tests/cc_deps/executable/sdist/group_a_entry.c b/uv/private/pep517_whl/tests/cc_deps/executable/sdist/group_a_entry.c new file mode 100644 index 000000000..3581cf050 --- /dev/null +++ b/uv/private/pep517_whl/tests/cc_deps/executable/sdist/group_a_entry.c @@ -0,0 +1,3 @@ +long group_b_value(void); + +long group_entry(void) { return 100 + group_b_value(); } diff --git a/uv/private/pep517_whl/tests/cc_deps/executable/sdist/group_a_tail.c b/uv/private/pep517_whl/tests/cc_deps/executable/sdist/group_a_tail.c new file mode 100644 index 000000000..014ba6534 --- /dev/null +++ b/uv/private/pep517_whl/tests/cc_deps/executable/sdist/group_a_tail.c @@ -0,0 +1 @@ +long group_a_tail_value(void) { return 5; } diff --git a/uv/private/pep517_whl/tests/cc_deps/executable/sdist/group_b.c b/uv/private/pep517_whl/tests/cc_deps/executable/sdist/group_b.c new file mode 100644 index 000000000..d72a9d708 --- /dev/null +++ b/uv/private/pep517_whl/tests/cc_deps/executable/sdist/group_b.c @@ -0,0 +1,3 @@ +long group_a_tail_value(void); + +long group_b_value(void) { return 20 + group_a_tail_value(); } diff --git a/uv/private/pep517_whl/tests/cc_deps/executable/sdist/mod.c b/uv/private/pep517_whl/tests/cc_deps/executable/sdist/mod.c new file mode 100644 index 000000000..72a34f12d --- /dev/null +++ b/uv/private/pep517_whl/tests/cc_deps/executable/sdist/mod.c @@ -0,0 +1,44 @@ +/* A hand-rolled CPython extension module. + * + * is resolved via the cc_deps include path, not from inside the sdist; + * dep_value() links through the cc_deps static archives (dep -> dep2) plus the + * [build_ext] libraries slot (-lextra against the build_clib-built libextra.a); + * and MOD_BONUS arrives as a transitively-propagated cc_deps -D define. + * group_entry() additionally proves that the -l entries keep their relative + * order through setuptools' [build_ext] libraries slot: the group archives form + * a one-pass cycle resolved by repeating -lgroup_a after -lgroup_b. + * Expected value: dep2 40 + extra 17 + MOD_BONUS 2 + group 125 == 184. */ +#include + +#include + +long group_entry(void); + +/* MOD_BONUS is injected by the `dep` cc_library's defines = ["MOD_BONUS=2"], + * which propagate transitively into this backend compile via cc_deps CPPFLAGS. + * Its absence means the -D define propagation broke, so fail the compile loudly + * rather than silently linking a wheel with the wrong value. */ +#ifndef MOD_BONUS +#error "MOD_BONUS not defined: cc_deps -D define propagation is broken" +#endif + +static PyObject *cc_deps_ext_value(PyObject *self, PyObject *args) { + (void)self; + (void)args; + return PyLong_FromLong(dep_value(7) + MOD_BONUS + group_entry()); +} + +static PyMethodDef cc_deps_ext_methods[] = { + {"value", cc_deps_ext_value, METH_NOARGS, + "Return dep_value() from the linked cc_library chain."}, + {NULL, NULL, 0, NULL}, +}; + +static struct PyModuleDef cc_deps_ext_module = { + PyModuleDef_HEAD_INIT, "cc_deps_ext", NULL, -1, cc_deps_ext_methods, + NULL, NULL, NULL, NULL, +}; + +PyMODINIT_FUNC PyInit_cc_deps_ext(void) { + return PyModule_Create(&cc_deps_ext_module); +} diff --git a/uv/private/pep517_whl/tests/cc_deps/executable/sdist/pyproject.toml b/uv/private/pep517_whl/tests/cc_deps/executable/sdist/pyproject.toml new file mode 100644 index 000000000..717276bfe --- /dev/null +++ b/uv/private/pep517_whl/tests/cc_deps/executable/sdist/pyproject.toml @@ -0,0 +1,7 @@ +[build-system] +requires = ["setuptools"] +build-backend = "setuptools.build_meta" + +[project] +name = "cc_deps_ext" +version = "0.0.1" diff --git a/uv/private/pep517_whl/tests/cc_deps/executable/sdist/setup.py b/uv/private/pep517_whl/tests/cc_deps/executable/sdist/setup.py new file mode 100644 index 000000000..e337b4fc2 --- /dev/null +++ b/uv/private/pep517_whl/tests/cc_deps/executable/sdist/setup.py @@ -0,0 +1,45 @@ +"""A minimal real-setuptools extension linking an out-of-tree cc_library. + +The extension source #includes , which is deliberately absent from this +sdist: it is resolved only through the cc_deps include path (CPPFLAGS), and the +dep_value symbol it calls is resolved only through the cc_deps static archives +that setuptools places in the post-object link slot ([build_ext] link_objects). +Neither Extension include_dirs nor Extension libraries are set on purpose. + +libextra.a, libgroup_a.a, and libgroup_b.a are built hermetically by build_clib +(with the same configured compiler, before build_ext runs). The group archives +contain a deliberate one-pass cycle, so they link only when -lgroup_a is repeated +after -lgroup_b (the documented repeat-the-library workaround), which exercises +-l order preservation through the [build_ext] libraries slot.""" + +from setuptools import Extension, setup +from setuptools.command.build_clib import build_clib + + +class quiet_build_clib(build_clib): + """Build the test archives but hide their names from build_ext's auto-link. + + distutils' build_ext.run() extends its libraries with + build_clib.get_library_names(), which would link these archives even without + the cc_deps user_link_flags under test. Returning no names keeps the archives + and library_dirs entry while leaving all -l naming and ordering exclusively + to the injected CcInfo stream.""" + + def get_library_names(self) -> list: + return [] + + +setup( + cmdclass={"build_clib": quiet_build_clib}, + libraries=[ + ("extra", {"sources": ["extra.c"]}), + ("group_a", {"sources": ["group_a_entry.c", "group_a_tail.c"]}), + ("group_b", {"sources": ["group_b.c"]}), + ], + ext_modules=[ + Extension( + name="cc_deps_ext", + sources=["mod.c"], + ), + ], +) From 9c59ccde2a776af73838c38b64815b9eeb5e57fb Mon Sep 17 00:00:00 2001 From: Connor McEntee Date: Mon, 27 Jul 2026 20:14:37 +0000 Subject: [PATCH 5/5] docs(uv): document cc_deps native dependency wiring Add a "Linking native C/C++ dependencies" section to uv-patching.md: declaring cc_deps on uv.override_package, how transitive headers, include paths, and defines reach the compile step and static archives and -l libraries reach the link, and how it composes additively with env. Spell out the requirements and limitations: setuptools backend only (other backends are rejected, not silently dropped), setuptools >= 65.4.0 for DIST_EXTRA_CONFIG, static or PIC-static archives only with PIC objects expected, visibility from the generated build repository, and the fail-closed link-flag allowlist with its accepted shapes, reviewed -z keywords, and the three escape hatches (repeated library names for archive cycles, env LDFLAGS for global toggles, pre_build_patches otherwise). Also note cc_deps in the requires-an-sdist and pure-Python-rejection bullets, and that declared additional_linker_inputs paths are anchored across the backend chdir while undeclared relative paths are not. --- docs/uv-patching.md | 122 +++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 116 insertions(+), 6 deletions(-) diff --git a/docs/uv-patching.md b/docs/uv-patching.md index 12b3743d8..f47d77260 100644 --- a/docs/uv-patching.md +++ b/docs/uv-patching.md @@ -14,7 +14,9 @@ Additionally, `extra_deps` and `extra_data` allow adding dependencies or data files to the generated `py_library` target for a package. `console_scripts` overrides the complete script map for a wheel built from an sdist when its egg-info metadata is absent or unsuitable. An explicit -empty map suppresses all detected scripts. +empty map suppresses all detected scripts. For a native +extension built from an sdist, `cc_deps` wires Bazel `cc_library` targets +(their headers and static archives) into the build. ## Prerequisites @@ -126,6 +128,54 @@ uv.override_package( ) ``` +### Linking native C/C++ dependencies + +A package with a native extension often needs a C/C++ library: its headers to +compile against and its static archive to link. `cc_deps` wires a Bazel target +that provides `CcInfo` (a `cc_library`, `cc_import`, or similar) directly into +the sdist build: the dependency's transitive headers, include paths, defines, +and Apple framework search paths become compile flags (appended to `CPPFLAGS`), +and its static archives are placed in the linker's post-object slot. The +transitive closure and link order come from `CcInfo`, so you name only the +top-level target. + +`cc_deps` is the declarative counterpart to the `env` / `toolchains` escape hatch +(see [the constraints below](#constraints)): reach for `cc_deps` to _declare_ the +native dependency, and keep `env` for _tweaking_ the build: package-specific +defines, exotic linker flags, or anything `cc_deps` cannot model. The two +compose; `cc_deps` flags are appended after any you set in `env`. + +For example, building a package's native extension against an in-repo C +library. Before, with the raw `env` / `toolchains` escape hatch, the include +and archive paths are anchored by hand with `$(EXECROOT)` and fed through +make-variables a toolchain exports: + +```starlark +uv.override_package( + name = "native-package", + toolchains = ["//third_party/mylib:make_vars"], # exports $(MYLIB_INC), $(MYLIB_LIB_A) + env = { + "CPPFLAGS": "-I$(EXECROOT)/$(MYLIB_INC)", + "LDFLAGS": "$(EXECROOT)/$(MYLIB_LIB_A)", + }, +) +``` + +After, with `cc_deps`, the include path, the archive, and their transitive +closure are read from the target's `CcInfo`, and no path anchoring is needed: + +```starlark +uv.override_package( + name = "native-package", + cc_deps = ["//third_party/mylib"], # a cc_library / cc_import +) +``` + +The dependency target must be visible to the generated build repository, so mark +it `//visibility:public` (or grant that repository's package visibility). See the +[constraints below](#constraints) for the supported-library and setuptools +requirements. + ### Reserving wheel build resources Native sdist builds can be memory-hungry. Without a hint, Bazel assumes the @@ -217,11 +267,11 @@ uv.override_package( removes NumPy's bundled tests without retaining their compiled bytecode. Removing the complete `.dist-info` directory, `METADATA`, or `RECORD` is unsupported. -- `pre_build_patches`, `toolchains`, `env`, `monitor_memory`, and non-default - `resource_set` values require a source distribution. An override that applies - them to a wheel-only lock record is rejected. -- Generated pure-Python builds reject `toolchains` and `env`; those attributes - augment the native build toolchain and environment. +- `pre_build_patches`, `toolchains`, `env`, `cc_deps`, `monitor_memory`, and + non-default `resource_set` values require a source distribution. An override + that applies them to a wheel-only lock record is rejected. +- Generated pure-Python builds reject `toolchains`, `env`, and `cc_deps`; those + attributes augment the native build toolchain, environment, and link inputs. - Native build `env` values can use `$(EXECROOT)/` to anchor paths supplied by a toolchain, for example `CPPFLAGS = "-I$(EXECROOT)/$(DEP_INC)"` and `LDFLAGS = "$(EXECROOT)/$(DEP_LIB_A)"`. The anchor remains valid after the @@ -229,6 +279,66 @@ uv.override_package( - Native builds select the configured C++ compiler, archiver, linker, and strip tools by default. Explicit `CC`, `CXX`, `AR`, `LD`, and `STRIP` values in `env` override those selections. +- `cc_deps` applies only to sdists built by the setuptools backend. A package + that declares any other `[build-system].build-backend` is rejected when the + wheel is built (`cc_deps is only supported with the setuptools build backend`) + rather than having its inputs silently dropped. +- The build environment's setuptools must be `>= 65.4.0`, the release that + added `DIST_EXTRA_CONFIG`, the channel `cc_deps` routes the link inputs + through. An older or missing setuptools fails the build with + `cc_deps requires setuptools >= 65.4.0`; bump it in the lock that supplies your + build dependencies (`uv.lock` / `default_build_dependencies`). +- Only static (or PIC-static) archives are linked. A dependency that provides + only a shared/dynamic library fails at analysis time, as does an `alwayslink` + (whole-archive) library; neither is supported. +- The linked archives must contain position-independent (PIC) objects, because + they are folded into the extension's shared object. A toolchain that emits + non-PIC objects into its static archives (some GCC configurations) fails the + final link with relocation errors such as `relocation R_X86_64_32 against ... +can not be used when making a shared object; recompile with -fPIC`. Remedies: + use a toolchain that compiles PIC objects (the default on macOS, and clang/LLVM + on Linux), build with `--force_pic`, or add `copts = ["-fPIC"]` to the + `cc_library`. +- Each `cc_deps` label is referenced from the generated external build + repository, so the target must be visible to it: use `//visibility:public` or + grant that repository's package visibility. +- Link flags that reference a file the dependency declares via + `additional_linker_inputs` (for example a `-Wl,--version-script,...` linker + script) are path-anchored automatically so they survive the backend changing + directory. A relative path written directly into `linkopts` without declaring + the file there is not anchored and will not resolve after the change. +- `cc_deps` flattens a dependency's link inputs into setuptools' two link slots: + full-path static archives go to the post-object `[build_ext] link_objects` + slot in topological order; bare `-l` entries go to the post-object + `[build_ext] libraries` slot preserving their relative order; and every other + link flag is appended to `LDFLAGS` ahead of the objects. Because the archives + and `-l` entries land in separate slots, the order between an `-l` entry + and a non-`-l` flag cannot be preserved, so only flags whose effect does not + depend on that relative order are passed through. `cc_deps` accepts a fixed set + of link-flag shapes: `-L` search paths; `-pthread`; and `-Wl,` tokens + built from these directives: the rpath family (`-rpath`, `-rpath=`, and + `-rpath-link`), `--version-script` (comma and `=` argument forms), `-z` with + one of the reviewed keywords `relro`, `now`, `noexecstack`, or `origin` (each + a global link mode; the wider `-z` namespace includes position-sensitive + keywords, so others are rejected), and `--enable-new-dtags`. A comma-joined + `-Wl,` token is validated directive by directive, so an accepted leading + directive cannot smuggle a rejected one behind it (`-Wl,-rpath,/x,--as-needed` + fails, naming `--as-needed`), while benign compounds such as + `-Wl,-z,relro,-z,now` and `-Wl,-rpath,$ORIGIN,--enable-new-dtags` pass. Any + other link flag, including grouping and linker-state toggles such as + `--start-group`/`--end-group` or `--as-needed`, is rejected at analysis time + rather than silently reordered. The split `-L ` form is rejected too; + write the glued `-L`. Apple `-framework` linking is not supported in v1: + ld64 resolves frameworks in command-line order alongside `-l` entries, so the + two-slot split cannot hold one; set the framework in the override's `env` + `LDFLAGS` or patch it in with `pre_build_patches`. Three escape hatches cover + what the allowlist does not: to resolve an archive cycle that would otherwise + need `--start-group`, repeat the library name (for example `-la -lb -la`), + since `-l` order is preserved within the libraries slot; to apply a global + toggle such as `--as-needed` to the whole link, set it in the override's + `env` `LDFLAGS`, which lands ahead of the objects; and for anything else, + patch it in with `pre_build_patches` on the sdist. The accepted set can be + extended upstream on request. - Post-install patches to prebuilt wheels must preserve every retained original path used for collision and regular-package merge planning, including its file-or-directory kind and package classification. Ordinary added paths are