feat(uv): reproducible rustc wrapper and cc-rs environment for Rust sdists - #1537
xangcastle wants to merge 2 commits into
Conversation
…dists The rustc wrapper only injected --sysroot, so a Rust wheel carried the sandbox and execroot paths of the host that built it (debug info, panic locations) and LLVM's per-host module ids in symbol names: the wheel action itself hits the remote cache, but its bytes differ per host, so every downstream action (whl_install, venvs) misses. The wrapper now remaps the sandbox root and the execroot away and compiles target crates as a single codegen unit; build scripts and proc-macros, which never reach the wheel, keep cargo's codegen. Native builds treat every crate as target. Crates with C or C++ sources (ring, zstd-sys) build them through cc-rs, which looks up CC_<triple>, CXX_<triple>, AR_<triple> and RANLIB_<triple> before falling back to the PATH; only CARGO_TARGET_<TRIPLE>_LINKER was set, so those objects came from whatever compiler the runner had. Cross builds now export the wired C toolchain under both spellings, with a ranlib wrapper over `ar s` for toolchains that ship none. The linker stays the C driver: linking with c++ would add an implicit libstdc++ dependency to every Rust extension, not only the ones with C++ sources. Tests run the generated wrapper around an argv-echoing rustc and check the cc-rs variables end to end through the cross env.
The CycloneDX SBOM maturin writes into the wheel records every crate as path+file:///<sandbox>/..., and the sandbox id differs per action, so two builds of one sdist never produced the same wheel even with the rustc wrapper remapping paths. maturin 1.15 has no flag or variable for it, only the [tool.maturin.sbom] table, appended to the extracted pyproject.toml unless the sdist configures it itself.
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Advanced Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
py_binary startup benchmark
sys.path quality
Bazel analysis benchmark
py_image_layer benchmark
|
Stacked on #1526. Helper-only: the rustc wrapper and the cargo environment that PR introduces.
Reproducible rustc wrapper. The wrapper only injected
--sysroot, so a Rust wheel carried the sandbox and execroot paths of the host that built it (debug info, panic locations) and LLVM's per-host module ids in symbol names. The wheel action itself hits the remote cache, but its bytes differ per host, so every downstream action (whl_install, venvs) misses. The wrapper now passes--remap-path-prefixfor the sandbox root and the execroot and-C codegen-units=1for target crates; build scripts and proc-macros, which never reach the wheel, keep cargo's codegen. Native builds treat every crate as target. maturin's CycloneDX SBOM is turned off in the extractedpyproject.tomlunless the sdist configures[tool.maturin.sbom]itself: it records every crate aspath+file:///<sandbox>/..., and the sandbox id differs per action, so no two builds agreed while it was on (maturin 1.15 has no flag or variable for it). Rewriting cargo's-C metadata=hash is left for a follow-up: it needs a host-independent toolchain identity, and the changes here already make two checkouts produce identical wheels (see test plan).cc-rs environment. Crates with C or C++ sources (
ring,zstd-sys) build them through cc-rs, which looks upCC_<triple>,CXX_<triple>,AR_<triple>andRANLIB_<triple>(dashed and underscored spellings) before falling back to the PATH; onlyCARGO_TARGET_<TRIPLE>_LINKERwas set. Cross builds now export the wired C toolchain under those names, with a ranlib wrapper overar sfor toolchains that ship none (shared with the CMake toolchain file). The cargo linker stays the C driver: linking withc++would give every Rust extension an implicit libstdc++ dependency, not only those with C++ sources.Tests: the generated wrapper is run around an argv-echoing fake rustc (remap flags, codegen-units gating on
--target, native mode, cargo's arguments untouched); the cc-rs variables are checked directly and through the cross env.Changes are visible to end-users: no
Test plan
New test cases added (see above).
cd e2e/crossbuild && bazel test //pycross-rust/... //pycross-rust-rs/... //pycross-tiktoken/...passes offline (native, linux/amd64 and linux/arm64 from a macOS host).Reproducibility: the same commit checked out at two paths, hence two output bases and sandbox ids, builds byte-identical cross pydantic-core wheels:
pydantic_core-2.47.0-cp312-cp312-linux_x86_64.whl0c5d3027e6be866c0005247c277263560450fe6a57c2d01aa0665d718217a723pydantic_core-2.47.0-cp312-cp312-linux_aarch64.whlb9c9af30a4faae3ebc6781a304307722320627cb1c84de9b554ae9c94df20729Before this PR the
.sodiffered by sandbox paths and, with the wrapper alone, only the SBOM and itsRECORDline still differed.