Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion docs/api/py.md
Original file line number Diff line number Diff line change
Expand Up @@ -158,7 +158,7 @@ Must not be testonly. `py_image_layer` transitions the `//py:layer_tier` flag to
<pre>
load("@aspect_rules_py//py:defs.bzl", "py_library")

py_library(<a href="#py_library-name">name</a>, <a href="#py_library-deps">deps</a>, <a href="#py_library-srcs">srcs</a>, <a href="#py_library-data">data</a>, <a href="#py_library-imports">imports</a>, <a href="#py_library-resolutions">resolutions</a>, <a href="#py_library-virtual_deps">virtual_deps</a>)
py_library(<a href="#py_library-name">name</a>, <a href="#py_library-deps">deps</a>, <a href="#py_library-srcs">srcs</a>, <a href="#py_library-data">data</a>, <a href="#py_library-dep_group">dep_group</a>, <a href="#py_library-imports">imports</a>, <a href="#py_library-resolutions">resolutions</a>, <a href="#py_library-virtual_deps">virtual_deps</a>)
</pre>


Expand All @@ -172,6 +172,7 @@ py_library(<a href="#py_library-name">name</a>, <a href="#py_library-deps">deps<
| <a id="py_library-deps"></a>deps | Targets that produce Python code, commonly `py_library` rules. | <a href="https://bazel.build/concepts/labels">List of labels</a> | optional | `[]` |
| <a id="py_library-srcs"></a>srcs | Python source files. | <a href="https://bazel.build/concepts/labels">List of labels</a> | optional | `[]` |
| <a id="py_library-data"></a>data | Runtime dependencies of the program.<br><br>The transitive closure of the `data` dependencies will be available in the `.runfiles` folder for this binary/test. The program may optionally use the Runfiles lookup library to locate the data files, see https://pypi.org/project/bazel-runfiles/. Data is analyzed in the inherited caller configuration. Put artifacts that must match the terminal's Python environment in `deps`. | <a href="https://bazel.build/concepts/labels">List of labels</a> | optional | `[]` |
| <a id="py_library-dep_group"></a>dep_group | Resolve `deps` within the named dependency group of the uv hub.<br><br>Only the `deps` edge changes configuration: this library, its `srcs` and `data`, and every consumer stay in the caller's configuration. Listing hub packages in a library with `dep_group` set keeps shared native dependencies out of per-group configurations, unlike `dep_group` on `py_binary` / `py_test`, which transitions the whole subtree. Empty inherits the caller's group. | String | optional | `""` |
| <a id="py_library-imports"></a>imports | List of import directories to be added to the PYTHONPATH. | List of strings | optional | `[]` |
| <a id="py_library-resolutions"></a>resolutions | Satisfy a virtual_dep with a mapping from external package name to the label of an installed package that provides it. See virtual_deps. | Dictionary: String -> Label | optional | `{}` |
| <a id="py_library-virtual_deps"></a>virtual_deps | - | List of strings | optional | `[]` |
Expand Down
24 changes: 24 additions & 0 deletions docs/uv.md
Original file line number Diff line number Diff line change
Expand Up @@ -132,6 +132,30 @@ py_binary(
)
```

`dep_group` on a `py_binary` / `py_test` is an incoming transition: every
target beneath it, including shared native libraries, is analyzed again per
group. To keep the group on the hub edge alone, set `dep_group` on the
`py_library` that lists the hub packages. Only that library's `deps` resolve
under the group; the library itself, its consumers and their other dependencies
share the caller's configuration.

```starlark
py_library(
name = "vendored_pip",
dep_group = "vendored_say",
deps = ["@pypi//cowsay"],
)

py_binary(
name = "say_vendored",
srcs = ["__main__.py_"],
deps = [
":vendored_pip",
"//lib:shared_native", # analyzed once, whatever group the pip deps use
],
)
```

Targets that need every dependency in one dependency group can use the
group-specific lists generated in `defs.bzl`. The `group_deps()` helper follows
the consuming target's `dep_group`, so the group name is never repeated:
Expand Down
18 changes: 17 additions & 1 deletion e2e/cases/multi-project-hub/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
# e2e snapshots. Three sub-projects (alpha, beta, gamma) all depend on
# cowsay, exercising the N>1 select-arm and dep_group config_setting
# emission that the single-project @pypi_single snapshot cannot cover.
load("@aspect_rules_py//py:defs.bzl", "py_test")
load("@aspect_rules_py//py:defs.bzl", "py_library", "py_test")

# Alpha defines `[dependency-groups]` (`dev` + hyphenated `unit-tests`)
# instead of an implicit project-name group, so the snapshot covers both
Expand Down Expand Up @@ -50,3 +50,19 @@ py_test(
"@pypi_multi//cowsay",
],
)

# `dep_group` on a py_library scopes the group to that library's deps; the
# consuming test sets no group and stays in the caller's configuration.
py_library(
name = "cowsay_beta",
dep_group = "beta",
deps = ["@pypi_multi//cowsay"],
)

py_test(
name = "beta_scoped",
srcs = ["__test__.py"],
main = "__test__.py",
python_version = "3.11",
deps = [":cowsay_beta"],
)
19 changes: 19 additions & 0 deletions e2e/rules-python-interop/reset-data-edges/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -170,4 +170,23 @@ root(
],
)

# py_library's dep_group reaches only its deps edge: the library and its data
# stay in the caller's group.
py_library(
name = "scoped_library",
data = [":probe"],
dep_group = "scoped",
tags = ["manual"],
deps = [":probe"],
)

root(
name = "scoped_root",
tags = ["manual"],
deps = [
":probe",
":scoped_library",
],
)

reset_data_edges_test_suite()
48 changes: 44 additions & 4 deletions e2e/rules-python-interop/reset-data-edges/tests.bzl
Original file line number Diff line number Diff line change
Expand Up @@ -12,14 +12,23 @@ _FREETHREADED_FLAG = "@aspect_rules_py//py/private/interpreter:freethreaded"
_RPY_FREETHREADED_FLAG = "@rules_python//python/config_settings:py_freethreaded"

_ProbeInfo = provider(fields = ["file"])
_ProbeFilesInfo = provider(fields = ["files", "modes", "bin_dirs"])
_ProbeFilesInfo = provider(fields = ["files", "modes", "bin_dirs", "groups"])

def _probe_impl(ctx):
out = ctx.actions.declare_file(ctx.label.name + ".txt")
ctx.actions.write(out, "probe")
return [_ProbeInfo(
file = out,
)]
return [
_ProbeInfo(
file = out,
),
# Lets py_library list the probe in `deps`.
PyInfo(
imports = depset(),
transitive_sources = depset(),
virtual_dependencies = depset(),
virtual_resolutions = depset(),
),
]

probe = rule(
implementation = _probe_impl,
Expand All @@ -35,6 +44,7 @@ def _probe_aspect_impl(target, ctx):
transitive = []
transitive_modes = []
transitive_bin_dirs = []
transitive_groups = []
deps = []
for attr_name in ["data", "deps"]:
deps.extend(getattr(ctx.rule.attr, attr_name, []))
Expand All @@ -49,6 +59,7 @@ def _probe_aspect_impl(target, ctx):
transitive.append(dep[_ProbeFilesInfo].files)
transitive_modes.append(dep[_ProbeFilesInfo].modes)
transitive_bin_dirs.append(dep[_ProbeFilesInfo].bin_dirs)
transitive_groups.append(dep[_ProbeFilesInfo].groups)

# Record every visited target; the test impl filters to the names it
# asserts on, keeping the fixture-name coupling in one place.
Expand All @@ -61,16 +72,19 @@ def _probe_aspect_impl(target, ctx):
# bin_dir carries the configuration's output segment, so equal paths mean
# one configuration.
bin_dirs = [(ctx.label.name, ctx.bin_dir.path)]
groups = [(ctx.label.name, ctx.attr._dep_group[BuildSettingInfo].value)]
return [_ProbeFilesInfo(
files = depset(direct = direct, transitive = transitive),
modes = depset(direct = modes, transitive = transitive_modes),
bin_dirs = depset(direct = bin_dirs, transitive = transitive_bin_dirs),
groups = depset(direct = groups, transitive = transitive_groups),
)]

_probe_aspect = aspect(
implementation = _probe_aspect_impl,
attr_aspects = ["data", "deps", "venv"],
attrs = {
"_dep_group": attr.label(default = _DEP_GROUP_FLAG),
"_freethreaded": attr.label(default = _FREETHREADED_FLAG),
"_rpy_freethreaded": attr.label(default = _RPY_FREETHREADED_FLAG),
},
Expand Down Expand Up @@ -113,6 +127,7 @@ def _root_impl(ctx):
files = depset(transitive = transitive),
modes = depset(transitive = [dep[_ProbeFilesInfo].modes for dep in ctx.attr.deps]),
bin_dirs = depset(transitive = [dep[_ProbeFilesInfo].bin_dirs for dep in ctx.attr.deps]),
groups = depset(transitive = [dep[_ProbeFilesInfo].groups for dep in ctx.attr.deps]),
)]

def _baseline_transition_impl(settings, attr):
Expand Down Expand Up @@ -234,6 +249,27 @@ def _shared_binaries_test_impl(ctx):

_shared_binaries_test = analysistest.make(_shared_binaries_test_impl)

def _scoped_library_test_impl(ctx):
env = analysistest.begin(ctx)
under_test = analysistest.target_under_test(env)[_ProbeFilesInfo]
asserts.equals(
env,
2,
len(under_test.files.to_list()),
"the probe should analyze once in the caller's group and once under the library's deps",
)
expected = [
("probe", "baseline"),
("probe", "scoped"),
("scoped_library", "baseline"),
]
tracked = {name: True for name, _ in expected}
groups = [group for group in under_test.groups.to_list() if group[0] in tracked]
asserts.equals(env, sorted(expected), sorted(groups))
return analysistest.end(env)

_scoped_library_test = analysistest.make(_scoped_library_test_impl)

def reset_data_edges_test_suite():
_reset_data_edges_test(
name = "reset_data_edges_test",
Expand All @@ -250,3 +286,7 @@ def reset_data_edges_test_suite():
tags = ["manual"],
target_under_test = ":binaries_root",
)
_scoped_library_test(
name = "scoped_library_test",
target_under_test = ":scoped_root",
)
42 changes: 31 additions & 11 deletions py/private/py_library.bzl
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ load("//py/private:providers.bzl", "PyWheelsInfo")
load("//py/private:pth.bzl", "make_imports_depset")
load("//py/private:py_info.bzl", "PyInfo")
load("//py/private:py_info_interop.bzl", "RulesPythonPyInfo", "get_py_info", "has_py_info")
load("//py/private:transitions.bzl", "reset_python_flags_transition")
load("//py/private:transitions.bzl", "dep_group_transition", "reset_python_flags_transition")

def _make_instrumented_files_info(ctx):
return coverage_common.instrumented_files_info(
Expand Down Expand Up @@ -189,12 +189,8 @@ def _py_library_impl(ctx):

return providers

_attrs = dict({
"srcs": attr.label_list(
doc = "Python source files.",
allow_files = True,
),
"deps": attr.label_list(
def _deps_attr(**kwargs):
return attr.label_list(
doc = "Targets that produce Python code, commonly `py_library` rules.",
# This attribute — shared by py_library, py_binary and py_test — is the
# public surface that supports rules_python interop: a dep may carry
Expand All @@ -203,7 +199,15 @@ _attrs = dict({
# rules_py emits @rules_python providers only under the
# migration-only //py:emit_rules_python_providers flag.
providers = [[PyInfo], [RulesPythonPyInfo], [CcInfo]],
**kwargs
)

_attrs = dict({
"srcs": attr.label_list(
doc = "Python source files.",
allow_files = True,
),
"deps": _deps_attr(),
"data": attr.label_list(
doc = """Runtime dependencies of the program.

Expand Down Expand Up @@ -247,9 +251,25 @@ py_library_utils = struct(

py_library = rule(
implementation = py_library_utils.implementation,
attrs = dict({
"virtual_deps": attr.string_list(allow_empty = True, default = []),
"_emit_rules_python_providers": attr.label(default = "//py/private:emit_rules_python_providers"),
}, **py_library_utils.attrs),
attrs = dict(
py_library_utils.attrs,
deps = _deps_attr(cfg = dep_group_transition),
dep_group = attr.string(
default = "",
doc = """Resolve `deps` within the named dependency group of the uv hub.

Only the `deps` edge changes configuration: this library, its `srcs` and `data`,
and every consumer stay in the caller's configuration. Listing hub packages in a
library with `dep_group` set keeps shared native dependencies out of per-group
configurations, unlike `dep_group` on `py_binary` / `py_test`, which transitions
the whole subtree. Empty inherits the caller's group.
""",
),
virtual_deps = attr.string_list(allow_empty = True, default = []),
_allowlist_function_transition = attr.label(
default = "@bazel_tools//tools/allowlists/function_transition_allowlist",
),
_emit_rules_python_providers = attr.label(default = "//py/private:emit_rules_python_providers"),
),
provides = py_library_utils.py_library_providers,
)
16 changes: 16 additions & 0 deletions py/private/transitions.bzl
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,22 @@ python_transition = transition(
outputs = _ALL_FLAGS,
)

# The `py_library.deps` edge. Resolving hub packages under a dependency group
# here keeps the library, its consumers and their non-Python dependencies in
# the caller's configuration.
def _dep_group_transition_impl(settings, attr):
dep_group = attr.dep_group or settings[_DEP_GROUP_FLAG]
return {
_DEP_GROUP_FLAG: dep_group,
_DEP_GROUP_BASELINE_FLAG: _capture_baseline(settings, _DEP_GROUP_FLAG, _DEP_GROUP_BASELINE_FLAG, dep_group),
}

dep_group_transition = transition(
implementation = _dep_group_transition_impl,
inputs = [_DEP_GROUP_FLAG, _DEP_GROUP_BASELINE_FLAG],
outputs = [_DEP_GROUP_FLAG, _DEP_GROUP_BASELINE_FLAG],
)

# The launcher -> venv edge. Validation never runs here: the venv's own rule
# transition always applies next, may override either half of a version/GIL
# combination, and is the sole authority for rejecting the final configuration.
Expand Down
Loading