What's the problem?
py_image_layer ships a ready-to-run venv plus a compiled launcher at /app. The launcher sets VIRTUAL_ENV (and effectively puts the venv on PATH) for the process it spawns. But there's no stable, documented way for other processes in the same container — ones that don't go through the /app launcher — to locate or activate that venv.
Real cases where this bites:
- KubeRay: runs the head/worker
ray start command through bash -lc (a login shell, whose /etc/profile resets PATH), and injects an autoscaler sidecar with its own bash -c command. Neither goes through the app launcher, so neither inherits VIRTUAL_ENV/PATH — yet both must resolve the venv's ray console script.
- k8s health/readiness probes exec a binary directly in the container (e.g.
wget … | grep), independent of the launcher.
- Generally: any sidecar, cron-in-container, or login shell that needs the venv's console scripts /
python.
Today the only handle is the venv directory's internal name — ._<safe_name>.venv/bin (py/private/py_venv/py_venv.bzl, venv_name = "." + safe_name) — which consumers must reverse-engineer by globbing:
VENV_BIN=$(find /app.runfiles -type d -path '*/._*.venv/bin' | head -1)
export PATH="$VENV_BIN:$PATH"
That's undocumented and version-coupled: the ._<name>.venv naming is an implementation detail, and this is an alpha. We hit exactly this (an image entrypoint + a wget probe shim both globbing that path); it's the kind of thing a rules_py bump can silently break.
Proposed
Give py_image_layer a stable, documented way to locate/activate the venv for non-launcher processes. Any one of:
- Image-level env — have
py_image_layer set VIRTUAL_ENV and prepend $VIRTUAL_ENV/bin to PATH in the image config Env, so every process in the container inherits it (the OCI analog of what the launcher already does per-process). Most ergonomic.
- Stable symlink — a documented, stable path (e.g.
<root>/.venv) → the internal ._<name>.venv, so consumers reference a stable name instead of globbing.
- At minimum, document the in-image venv-location contract.
Notes
What's the problem?
py_image_layerships a ready-to-run venv plus a compiled launcher at/app. The launcher setsVIRTUAL_ENV(and effectively puts the venv onPATH) for the process it spawns. But there's no stable, documented way for other processes in the same container — ones that don't go through the/applauncher — to locate or activate that venv.Real cases where this bites:
ray startcommand throughbash -lc(a login shell, whose/etc/profileresetsPATH), and injects an autoscaler sidecar with its ownbash -ccommand. Neither goes through the app launcher, so neither inheritsVIRTUAL_ENV/PATH— yet both must resolve the venv'srayconsole script.wget … | grep), independent of the launcher.python.Today the only handle is the venv directory's internal name —
._<safe_name>.venv/bin(py/private/py_venv/py_venv.bzl,venv_name = "." + safe_name) — which consumers must reverse-engineer by globbing:That's undocumented and version-coupled: the
._<name>.venvnaming is an implementation detail, and this is an alpha. We hit exactly this (an image entrypoint + awgetprobe shim both globbing that path); it's the kind of thing a rules_py bump can silently break.Proposed
Give
py_image_layera stable, documented way to locate/activate the venv for non-launcher processes. Any one of:py_image_layersetVIRTUAL_ENVand prepend$VIRTUAL_ENV/bintoPATHin the image configEnv, so every process in the container inherits it (the OCI analog of what the launcher already does per-process). Most ergonomic.<root>/.venv) → the internal._<name>.venv, so consumers reference a stable name instead of globbing.Notes
share/data_files) and docs: clarify nested virtualenv migration #1369 (guidance not to hardcodepy_venv_link's venv path).._<name>.venvis currently the only handle.aspect_rules_py2.0.0-alpha.5.