Skip to content

test(e2e): port rules_pycross sdist-build cases - #1413

Merged
xangcastle merged 3 commits into
mainfrom
xangcastle/e2e-pycross-cases
Aug 11, 2026
Merged

xangcastle merged 3 commits into
mainfrom
xangcastle/e2e-pycross-cases

Conversation

@xangcastle

@xangcastle xangcastle commented Aug 11, 2026

Copy link
Copy Markdown
Member

Ports four cases from rules_pycross's e2e suite, exercising the existing sdist→wheel pipeline (host builds only — no cross-compilation involved). Split out of #1363, where these cases don't depend on the cross feature; every test passes against current main unchanged.

Cases

  • pycross-setuptools (build_setuptools): three setuptools C-extension sdists — PyYAML (Cython-generated extension, with an observable PYYAML_FORCE_LIBYAML=0 env override), setproctitle (plain C, pre_build_patches), zstandard (vendored libzstd, resource_set).
  • pycross-patches (patches_and_hooks): pre-build and post-install patch phases stacked on setproctitle — the post-install hunk carries the pre-build patch's output as context, so an ordering regression fails the build itself.
  • pycross-pure-python (build_pure_python): hatchling and flit-core backends, runtime imports, site-packages placement hygiene, plus a collect_wheels matrix asserting anyarch wheels stay -none-any under non-host platform transitions (exec-platform resolution of the build tooling, no native toolchain needed).
  • pycross-distutils-probe: build-action env hygiene — a fresh child interpreter spawned by the backend must resolve distutils on Python 3.12+, guarding rule.bzl's _INHERITED_PYTHON_ENV filter. Fails only in the child, so it's invisible to a plain "does it build" check.

Shared tools/: collect_wheels macro (adds the wheel-tag assertion the rules_pycross original lacks) and check_wheel_tags.py.

Not included

pycross-setuptools' cross matrix (native_wheels*, check_wheel_native.py): building a C extension for a non-host platform needs native_build_toolchain_type resolution for that platform, which lands with #1363. The second commit documents that scope cut.


Changes are visible to end-users: no

Test plan

  • New test cases added

jbedard and others added 2 commits August 10, 2026 18:37
Port of rules_pycross's e2e suite (build_setuptools, patches_and_hooks,
build_pure_python and the distutils probe) to rules_py's uv extension
and pep517 rules.

(cherry picked from commit 4869b33,
adapted to keep main's //tools:linux_x86_64 platform)
Refinements over the original port so the suite runs green against the
existing rules:

- drop pycross-setuptools' cross matrix (native_wheels targets, per-case
  platforms, check_wheel_native.py): building a C extension for a
  non-host platform needs native_build_toolchain_type resolution for
  that platform, which lands with the cross-compilation branch. The
  pure-python collect_wheels matrix stays — anyarch builds exercise the
  exec-platform resolution without a native toolchain.
- keep //tools:linux_x86_64, referenced by venv-namespace-collapse docs
- require PYCROSS_TEST_PACKAGES in test_top_level_packages instead of
  falling back to packages this workspace doesn't ship
- move design rationale from inline comments into module docstrings;
  strip the prose preamble from the post-install patch
@aspect-workflows

aspect-workflows Bot commented Aug 11, 2026

Copy link
Copy Markdown

✨ Aspect Workflows Tasks

📅 Tue Aug 11 03:10:12 UTC 2026

✅ 41 successful tasks

  • ✅ buildifier · ⏱ 20s · 🐙 GitHub Actions · ☑️ Check
    💬 Format complete (clean)
  • ✅ gazelle · ⏱ 16.3s · 🐙 GitHub Actions · ☑️ Check
    💬 Gazelle complete (clean)
  • ✅ test-e2e-bazel-8 [test] · ⏱ 5m 11s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (274/274 passed)
  • ✅ test-e2e-bazel-9 [test] · ⏱ 4m 58s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (268/268 passed)
  • ✅ test-e2e-interpreter-build-config-bazel-8 [test] · ⏱ 18.5s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (1/1 passed)
  • ✅ test-e2e-interpreter-build-config-bazel-9 [test] · ⏱ 55.4s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (1/1 passed)
  • ✅ test-e2e-interpreter-input-validation-bazel-8 [test] · ⏱ 17.2s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (1/1 passed)
  • ✅ test-e2e-interpreter-input-validation-bazel-9 [test] · ⏱ 1m 2s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (1/1 passed)
  • ✅ test-e2e-interpreter-runtime-metadata-bazel-8 [test] · ⏱ 26.5s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (2/2 passed)
  • ✅ test-e2e-interpreter-runtime-metadata-bazel-9 [test] · ⏱ 43.8s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (2/2 passed)
  • ✅ test-e2e-interpreter-toolchain-settings-bazel-9 [test] · ⏱ 31.6s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (1/1 passed)
  • ✅ test-e2e-rules-proto-grpc-python-bazel-8 [test] · ⏱ 1m 53s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (1/1 passed)
  • ✅ test-e2e-rules-proto-grpc-python-bazel-9 [test] · ⏱ 1m 45s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (1/1 passed)
  • ✅ test-e2e-rules-python-interop-bazel-8 [test] · ⏱ 35.5s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (11/11 passed)
  • ✅ test-e2e-rules-python-interop-bazel-9 [test] · ⏱ 59.7s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (11/11 passed)
  • ✅ test-e2e-rules-python-provider-compat-bazel-8 [test] · ⏱ 27s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (3/3 passed)
  • ✅ test-e2e-rules-python-provider-compat-bazel-9 [test] · ⏱ 44.2s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (3/3 passed)
  • ✅ test-examples-debugger-bazel-8 [test] · ⏱ 22.5s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (1/1 passed)
  • ✅ test-examples-debugger-bazel-9 [test] · ⏱ 36.7s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (1/1 passed)
  • ✅ test-examples-dev_deps-bazel-8 [test] · ⏱ 26.7s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (1/1 passed)
  • ✅ test-examples-dev_deps-bazel-9 [test] · ⏱ 39.6s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (1/1 passed)
  • ✅ test-examples-django-bazel-8 [test] · ⏱ 26.5s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (1/1 passed)
  • ✅ test-examples-django-bazel-9 [test] · ⏱ 46.6s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (1/1 passed · 1 cached)
  • ✅ test-examples-multi_version-bazel-8 [test] · ⏱ 25s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (2/2 passed)
  • ✅ test-examples-multi_version-bazel-9 [test] · ⏱ 34.8s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (2/2 passed)
  • ✅ test-examples-protobuf-bazel-8 [test] · ⏱ 1m 30s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (1/1 passed)
  • ✅ test-examples-protobuf-bazel-9 [test] · ⏱ 1m 51s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (1/1 passed)
  • ✅ test-examples-py_binary-bazel-8 [test] · ⏱ 27.3s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (1/1 passed)
  • ✅ test-examples-py_binary-bazel-9 [test] · ⏱ 31.8s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (1/1 passed · 1 cached)
  • ✅ test-examples-py_pex_binary-bazel-8 [test] · ⏱ 19.7s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (1/1 passed)
  • ✅ test-examples-py_pex_binary-bazel-9 [test] · ⏱ 38.9s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (1/1 passed · 1 cached)
  • ✅ test-examples-py_venv-bazel-8 [test] · ⏱ 23.7s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (3/3 passed)
  • ✅ test-examples-py_venv-bazel-9 [test] · ⏱ 38.3s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (3/3 passed)
  • ✅ test-examples-pytest-bazel-8 [test] · ⏱ 42.9s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (11/11 passed)
  • ✅ test-examples-pytest-bazel-9 [test] · ⏱ 49.4s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (11/11 passed)
  • ✅ test-examples-uv_pip_compile-bazel-8 [test] · ⏱ 26.4s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (1/1 passed)
  • ✅ test-examples-uv_pip_compile-bazel-9 [test] · ⏱ 52.3s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (1/1 passed)
  • ✅ test-examples-virtual_deps-bazel-8 [test] · ⏱ 28.9s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (1/1 passed)
  • ✅ test-examples-virtual_deps-bazel-9 [test] · ⏱ 52.5s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (1/1 passed)
  • ✅ test-root-bazel-8 [test] · ⏱ 3m 9s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (297/297 passed)
  • ✅ test-root-bazel-9 [test] · ⏱ 3m 38s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (296/296 passed)

⏱ Last updated Tue Aug 11 03:16:44 UTC 2026 · 📊 GitHub API quota 938/15,000 (6% used, resets in 41m)
🚀 Powered by Aspect CLI (v2026.28.2)  |  Aspect Build · X · LinkedIn · YouTube

@github-actions

github-actions Bot commented Aug 11, 2026

Copy link
Copy Markdown

py_binary startup benchmark

Version Mean (ms) Median (ms) ± stddev vs BCR vs main Build (s)
BCR 1.11.7 (baseline) 185.468 184.034 ±5.095 50.85
HEAD main 60.506 60.006 ±2.961 -67.4% 9.53
This PR 60.241 59.662 ±3.164 -67.5% -0.4% 6.92

Measured with hyperfine --warmup 5 --runs 50 on Linux
Gate: PR vs HEAD main (threshold: 10%). BCR is shown only as a historical baseline.
Build time: cold bazel build //:bench with isolated output base, no disk cache.

sys.path quality

Version sys.path entries distinct site-packages roots duplicate realpaths
BCR 1.11.7 (baseline) 6 1 0
HEAD main 7 2 0
This PR 7 2 0

sys.path quality measured by bench_syspath inside the assembled venv. Duplicate realpaths indicate symlink redundancy; many distinct site-packages roots suggest an inefficient venv layout.

Bazel analysis benchmark

Version Mean (ms) Median (ms) ± stddev vs BCR vs main Targets Actions
BCR 2.0.0-alpha.5 (baseline) 10261.517 10265.849 ±169.495 301 13672
HEAD main 9137.367 9083.243 ±177.281 -11.0% 301 13740
This PR 9007.863 9101.555 ±200.021 -12.2% -1.4% 301 13740

Measured with hyperfine --warmup 1 --runs 10 on Linux
Gate: PR vs HEAD main (threshold: 10%). BCR is shown only as a historical baseline.
Command: cold bazel build --nobuild //workspace/... with isolated output base, no disk cache.

@xangcastle
xangcastle requested a review from jbedard August 11, 2026 01:23
Comment thread e2e/cases/tools/collect_wheels.bzl Outdated
expected_tags is now required: every caller asserts tags, and a
collect_wheels without the tags test would reintroduce exactly the
silent wrong-platform pass the macro exists to prevent. The unused
**kwargs forwarding goes with it.

collect_wheels.py keeps only the tree-artifact branch: wheels always
arrive as pep517_whl/pep517_native_whl directory outputs, so the
plain-file handling inherited from rules_pycross never ran.
@xangcastle
xangcastle force-pushed the xangcastle/e2e-pycross-cases branch from 373e69d to 86e0368 Compare August 11, 2026 03:09
@xangcastle
xangcastle merged commit 05504ac into main Aug 11, 2026
91 checks passed
@xangcastle
xangcastle deleted the xangcastle/e2e-pycross-cases branch August 11, 2026 03:19
xangcastle pushed a commit that referenced this pull request Aug 11, 2026
Port of rules_pycross's e2e suite (build_setuptools, patches_and_hooks,
build_pure_python and the distutils probe), relocated into the
crossbuild workspace.

(originally commit 4869b33, merged to
main squashed in #1413 / 05504ac; re-homed here with the content of
the original port)
xangcastle added a commit that referenced this pull request Aug 11, 2026
The four rules_pycross ports moved from e2e/cases into the crossbuild
workspace, where the cross toolchains they exercise actually live:

- restore the pycross-setuptools native_wheels matrix deferred out of
  #1413 (zstandard rebuilt for amd64/arm64 with ELF and wheel-tag
  assertions) — under this workspace's registered cross CC toolchain it
  runs where it couldn't on main
- carry the reviewed refinements from #1413 over the original port
- pin setproctitle to -std=gnu17: gcc_toolchain's GCC defaults to C23,
  where `bool` is a keyword and 1.3.2's `typedef char bool` breaks —
  coverage the e2e/cases host toolchain never gave us
- drop the cases, their MODULE includes and the collect_wheels tooling
  from e2e/cases
xangcastle pushed a commit that referenced this pull request Aug 13, 2026
Port of rules_pycross's e2e suite (build_setuptools, patches_and_hooks,
build_pure_python and the distutils probe), relocated into the
crossbuild workspace.

(originally commit 4869b33, merged to
main squashed in #1413 / 05504ac; re-homed here with the content of
the original port)
xangcastle added a commit that referenced this pull request Aug 13, 2026
The four rules_pycross ports moved from e2e/cases into the crossbuild
workspace, where the cross toolchains they exercise actually live:

- restore the pycross-setuptools native_wheels matrix deferred out of
  #1413 (zstandard rebuilt for amd64/arm64 with ELF and wheel-tag
  assertions) — under this workspace's registered cross CC toolchain it
  runs where it couldn't on main
- carry the reviewed refinements from #1413 over the original port
- pin setproctitle to -std=gnu17: gcc_toolchain's GCC defaults to C23,
  where `bool` is a keyword and 1.3.2's `typedef char bool` breaks —
  coverage the e2e/cases host toolchain never gave us
- drop the cases, their MODULE includes and the collect_wheels tooling
  from e2e/cases
xangcastle pushed a commit that referenced this pull request Aug 14, 2026
Port of rules_pycross's e2e suite (build_setuptools, patches_and_hooks,
build_pure_python and the distutils probe), relocated into the
crossbuild workspace.

(originally commit 4869b33, merged to
main squashed in #1413 / 05504ac; re-homed here with the content of
the original port)
xangcastle added a commit that referenced this pull request Aug 14, 2026
The four rules_pycross ports moved from e2e/cases into the crossbuild
workspace, where the cross toolchains they exercise actually live:

- restore the pycross-setuptools native_wheels matrix deferred out of
  #1413 (zstandard rebuilt for amd64/arm64 with ELF and wheel-tag
  assertions) — under this workspace's registered cross CC toolchain it
  runs where it couldn't on main
- carry the reviewed refinements from #1413 over the original port
- pin setproctitle to -std=gnu17: gcc_toolchain's GCC defaults to C23,
  where `bool` is a keyword and 1.3.2's `typedef char bool` breaks —
  coverage the e2e/cases host toolchain never gave us
- drop the cases, their MODULE includes and the collect_wheels tooling
  from e2e/cases
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants