Skip to content

test(interpreter): make the stray-bytecode e2e portable and cache-isolated - #1544

Merged
xangcastle merged 1 commit into
mainfrom
xangcastle/pycache-features-portability
Sep 17, 2026
Merged

xangcastle merged 1 commit into
mainfrom
xangcastle/pycache-features-portability

Conversation

@xangcastle

@xangcastle xangcastle commented Sep 17, 2026

Copy link
Copy Markdown
Member

The interpreter-toolchain-settings e2e added in #1540 plants bytecode inside a fetched interpreter repo and cleans up on exit.

  • The cleanup used rmdir --ignore-fail-on-non-empty, a GNU-only flag that BSD rmdir rejects, so on macOS the planted __pycache__ directories were left behind. Each directory is now removed individually, tolerating a failure.
  • With the repo contents cache (default since Bazel 9) external/<repo> is a symlink into a cache shared by every workspace of the user, so the planted files were visible to unrelated builds during the run and would survive a killed shell. The workspace .bazelrc now disables that cache so the repo is fetched into its own output base. The flag exists on Bazel 8 as well, where the cache is already off by default.

Changes are visible to end-users: no

Test plan

  • e2e/interpreter-toolchain-settings/test.sh on macOS with a fresh output base under Bazel 8.6 and 9.2: the interpreter repo is a plain directory, the script passes, and no planted file remains in the output base or the shared cache.

@coderabbitai

coderabbitai Bot commented Sep 17, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Advanced

Run ID: 0f9840aa-f67a-4063-8497-fb646bc9f269

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@xangcastle
xangcastle force-pushed the xangcastle/pycache-features-portability branch from 9af4d14 to 0334d5d Compare September 17, 2026 04:21
@aspect-workflows

aspect-workflows Bot commented Sep 17, 2026

Copy link
Copy Markdown

✨ Aspect Workflows Tasks

📅 Thu Sep 17 04:55:14 UTC 2026

✅ 44 successful tasks

  • ✅ buildifier · ⏱ 18.4s · 🐙 GitHub Actions · ☑️ Check
    💬 Format complete (clean)
  • ✅ gazelle · ⏱ 23.9s · 🐙 GitHub Actions · ☑️ Check
    💬 Gazelle complete (clean)
  • ✅ test-e2e-bazel-8 [test] · ⏱ 53.7s · ✨ Aspect · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (324/324 passed · 322 cached)
  • ✅ test-e2e-bazel-9 [test] · ⏱ 46.6s · ✨ Aspect · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (230/230 passed · 228 cached)
  • ✅ test-e2e-crossbuild-bazel-8 [test] · ⏱ 42.5s · ✨ Aspect · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (39/39 passed · 39 cached)
  • ✅ test-e2e-crossbuild-bazel-9 [test] · ⏱ 43.2s · ✨ Aspect · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (39/39 passed · 39 cached)
  • ✅ test-e2e-interpreter-build-config-bazel-8 [test] · ⏱ 18s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (1/1 passed)
  • ✅ test-e2e-interpreter-build-config-bazel-9 [test] · ⏱ 36.6s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (1/1 passed)
  • ✅ test-e2e-interpreter-input-validation-bazel-8 [test] · ⏱ 18.9s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (1/1 passed)
  • ✅ test-e2e-interpreter-input-validation-bazel-9 [test] · ⏱ 31.1s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (1/1 passed)
  • ✅ test-e2e-interpreter-runtime-metadata-bazel-8 [test] · ⏱ 31.6s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (2/2 passed)
  • ✅ test-e2e-interpreter-runtime-metadata-bazel-9 [test] · ⏱ 40.4s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (2/2 passed)
  • ✅ test-e2e-interpreter-toolchain-settings-bazel-8 [test] · ⏱ 21.7s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (1/1 passed)
  • ✅ test-e2e-interpreter-toolchain-settings-bazel-9 [test] · ⏱ 48.7s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (1/1 passed)
  • ✅ test-e2e-rules-python-interop-bazel-8 [test] · ⏱ 51.5s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (16/16 passed)
  • ✅ test-e2e-rules-python-interop-bazel-9 [test] · ⏱ 1m 52s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (16/16 passed)
  • ✅ test-e2e-rules-python-protobuf-bazel-8 [test] · ⏱ 36.4s · ✨ Aspect · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (1/1 passed · 1 cached)
  • ✅ test-e2e-rules-python-protobuf-bazel-9 [test] · ⏱ 36.5s · ✨ Aspect · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (1/1 passed · 1 cached)
  • ✅ test-e2e-rules-python-provider-compat-bazel-8 [test] · ⏱ 21.5s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (4/4 passed)
  • ✅ test-e2e-rules-python-provider-compat-bazel-9 [test] · ⏱ 1m 42s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (4/4 passed)
  • ✅ test-examples-debugger-bazel-8 [test] · ⏱ 23.3s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (1/1 passed)
  • ✅ test-examples-debugger-bazel-9 [test] · ⏱ 1m · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (1/1 passed)
  • ✅ test-examples-dev_deps-bazel-8 [test] · ⏱ 27.5s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (1/1 passed)
  • ✅ test-examples-dev_deps-bazel-9 [test] · ⏱ 50.5s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (1/1 passed)
  • ✅ test-examples-django-bazel-8 [test] · ⏱ 25s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (1/1 passed · 1 cached)
  • ✅ test-examples-django-bazel-9 [test] · ⏱ 59.9s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (1/1 passed · 1 cached)
  • ✅ test-examples-multi_version-bazel-8 [test] · ⏱ 32.4s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (2/2 passed)
  • ✅ test-examples-multi_version-bazel-9 [test] · ⏱ 42s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (2/2 passed)
  • ✅ test-examples-protobuf-bazel-8 [test] · ⏱ 19.2s · ✨ Aspect · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (1/1 passed · 1 cached)
  • ✅ test-examples-protobuf-bazel-9 [test] · ⏱ 21.1s · ✨ Aspect · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (1/1 passed · 1 cached)
  • ✅ test-examples-py_binary-bazel-8 [test] · ⏱ 17.3s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (1/1 passed · 1 cached)
  • ✅ test-examples-py_binary-bazel-9 [test] · ⏱ 39.1s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (1/1 passed · 1 cached)
  • ✅ test-examples-py_pex_binary-bazel-8 [test] · ⏱ 21.2s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (1/1 passed · 1 cached)
  • ✅ test-examples-py_pex_binary-bazel-9 [test] · ⏱ 39.1s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (1/1 passed · 1 cached)
  • ✅ test-examples-py_venv-bazel-8 [test] · ⏱ 26.8s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (3/3 passed)
  • ✅ test-examples-py_venv-bazel-9 [test] · ⏱ 33.8s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (3/3 passed)
  • ✅ test-examples-pytest-bazel-8 [test] · ⏱ 38.7s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (11/11 passed)
  • ✅ test-examples-pytest-bazel-9 [test] · ⏱ 1m 2s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (11/11 passed)
  • ✅ test-examples-uv_pip_compile-bazel-8 [test] · ⏱ 22.8s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (1/1 passed)
  • ✅ test-examples-uv_pip_compile-bazel-9 [test] · ⏱ 44.2s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (1/1 passed)
  • ✅ test-examples-virtual_deps-bazel-8 [test] · ⏱ 30.2s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (1/1 passed)
  • ✅ test-examples-virtual_deps-bazel-9 [test] · ⏱ 1m 1s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (1/1 passed)
  • ✅ test-root-bazel-8 [test] · ⏱ 11m 21s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (374/374 passed)
  • ✅ test-root-bazel-9 [test] · ⏱ 17m 9s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (373/373 passed)

⏱ Last updated Thu Sep 17 05:12:22 UTC 2026 · 📊 GitHub API quota 2,180/7,700 (28% used, resets in 24m)
🚀 Powered by Aspect CLI (v2026.28.2)  |  Aspect Build · X · LinkedIn · YouTube

@xangcastle
xangcastle force-pushed the xangcastle/pycache-features-portability branch from 0334d5d to 9754307 Compare September 17, 2026 04:24
@github-actions

github-actions Bot commented Sep 17, 2026

Copy link
Copy Markdown

py_binary startup benchmark

Version Time (ms) vs BCR vs main Build (s)
BCR 1.11.7 163.9/163.3 ±2.4 1.69
main 56.9/56.7 ±0.9 -65.3% 1.65
PR 57.7/57.4 ±1.5 -64.9% +1.2% 1.54

Time = mean/median ±stddev.
Measured with hyperfine --warmup 5 --runs 50 --shell=none on Linux
Gate: PR vs HEAD main median (threshold: 10%, and must exceed the 2×SE noise floor, here 0.9%). BCR is shown only as a historical baseline.
Build time: cold bazel build //:bench with isolated output base, no disk cache; external repos prefetched so network is excluded.

sys.path quality

Version entries sp roots dupes
BCR 1.11.7 6 1 0
main 7 2 0
PR 7 2 0

sys.path quality measured by bench_syspath inside the assembled venv: sys.path entries, distinct site-packages roots, duplicate realpaths. Duplicates indicate symlink redundancy; many roots suggest an inefficient venv layout.

Bazel analysis benchmark

Version Time (ms) vs BCR vs main Targets Actions Configured (ext) Cfg/target Actions/cfg
BCR 2.0.0-alpha.6 1214/1169 ±188 404 20613 7380 (6675) 18.3 2.79
main 1204/1151 ±178 -1.6% 404 18363 7379 (6674) 18.3 2.49
PR 1226/1148 ±183 -1.8% -0.3% 404 18363 7379 (6674) 18.3 2.49

Measured with hyperfine --warmup 1 --runs 10 on Linux
Time = mean/median ±stddev. Cfg = configured targets; (ext) = the count in external repos (the @pypi hub machinery).
Gate: PR vs HEAD main median (threshold: 10%, and must exceed the 2×SE noise floor, here 14.0%). BCR is shown only as a historical baseline.
Command: warm-server bazel build --nobuild //workspace/..., analysis cache discarded each run via a fresh --action_env value; no disk cache.
Workload: 100 py_test targets — 2 test files/package in file generation mode (one venv per test file); a single dep_group.

py_image_layer benchmark

Scenario Version Time (s) Actions vs BCR vs main
analysis BCR 2.0.0-alpha.6 0.76/0.73 ±0.12 747
analysis main 0.75/0.73 ±0.14 668 -0.5%
analysis PR 0.75/0.70 ±0.14 668 -4.6% -4.1%
1p source BCR 2.0.0-alpha.6 0.79/0.80 ±0.06 4
1p source main 0.79/0.77 ±0.07 5 -3.5%
1p source PR 0.78/0.76 ±0.07 5 -5.1% -1.7%
3p wheel BCR 2.0.0-alpha.6 0.93/0.93 ±0.05 5
3p wheel main 0.96/0.97 ±0.04 6 +3.9%
3p wheel PR 0.95/0.92 ±0.06 6 -1.1% -4.8%

Time = mean/median ±stddev.
Measured with hyperfine on Linux, building //workspace:image_layers (10 binaries, ~30-wheel dep pool, grouped first-party/pip/interpreter tier) with isolated output base, no disk cache.
Scenarios: analysis = warm-server bazel build --nobuild, re-analyzed each run via a fresh --action_env value; incrementals run against a built state with warm analysis: source = append to the last package's lib.py, wheel = rewrite click post_install_patches content.
Actions: for Analysis, the total action count behind the image target from aquery deps(...); for incrementals, actions re-executed for the mutation, from a single instrumented run's BEP build metrics (deterministic; per-mnemonic breakdown in the *-actions.json artifacts). Informational only, not gated.
Gate: PR vs HEAD main median per scenario (threshold: 10%, and must exceed the 2×SE noise floor). BCR is shown only as a historical baseline.

@xangcastle
xangcastle requested a review from jbedard September 17, 2026 04:47
@xangcastle
xangcastle marked this pull request as ready for review September 17, 2026 04:47
@xangcastle

Copy link
Copy Markdown
Member Author

@codex

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9754307bb6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread e2e/interpreter-toolchain-settings/test.sh Outdated
…lated

The cleanup used `rmdir --ignore-fail-on-non-empty`, a GNU-only flag that
BSD rmdir rejects, so on macOS the planted `__pycache__` directories were
left behind. Remove each planted file and directory individually,
tolerating a failed rmdir, with no dependence on xargs or rmdir flags.

The test writes into a fetched interpreter repo. With the repo contents
cache, on by default since Bazel 9, `external/<repo>` is a symlink into a
cache shared by every workspace of the user, so the planted files were
visible to unrelated builds while the test ran and would outlive it if
the shell were killed. Disable the cache for this workspace so the repo
is fetched into its own output base. The flag exists on Bazel 8 too,
where the cache is off by default, so the setting is a no-op there.

Verified on macOS with a fresh output base under Bazel 8.6 and 9.2: the
repo is a plain directory, the script passes, and no planted file
remains in either the output base or the shared cache.
@xangcastle
xangcastle force-pushed the xangcastle/pycache-features-portability branch from 9754307 to 82defc4 Compare September 17, 2026 04:54
@xangcastle
xangcastle merged commit e2fbcc3 into main Sep 17, 2026
103 checks passed
@xangcastle
xangcastle deleted the xangcastle/pycache-features-portability branch September 17, 2026 18:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants