Skip to content

feat: render env file variables: as image ENV (silence gcloud NumPy warning) - #64

Merged
johnyaku merged 3 commits into
mainfrom
feat/env-variables-to-dockerfile-env
Jun 18, 2026
Merged

johnyaku merged 3 commits into
mainfrom
feat/env-variables-to-dockerfile-env

Conversation

@johnyaku

@johnyaku johnyaku commented Jun 18, 2026 •

Copy link
Copy Markdown
Contributor

Why the NumPy warning persists

To increase the performance of the tunnel, consider installing NumPy. ...

This was supposedly fixed in #53 ("NumPy in tool image") — but that change went into the root Dockerfile, which isn't what gets deployed. deploy.yml builds the image by dogfooding:

absconda publish --file examples/absconda-env.yaml --repository ghcr.io/swarbricklab/absconda ...

i.e. a conda image built from examples/absconda-env.yaml. That env already contains numpy and google-cloud-sdk — so NumPy is installed. The real problem: gcloud runs its Python with -S (no site-packages) unless CLOUDSDK_PYTHON_SITEPACKAGES=1, so it never imports the NumPy that's sitting right there.

That env var must be present at runtime in the image. But absconda images run binaries directly via the singularity wrapper — they never conda activate — so conda's own variables: mechanism (which only applies on activate) doesn't fire, and there was no other way to bake an env var in.

Changes

  1. Render a conda-style variables: section into image ENV lines (general, reusable — any env can now set image env vars; aligns with conda's existing environment.yaml schema):

    • Emitted in the export block, so it lands in the final image for all build modes (single-stage, multi-stage runtime, conda-on-base).
    • Values are quoted (ENV FOO="a b") so spaces survive.
    • The variables: section is stripped from the conda env file the solver reads (it's an image directive, not a solve input).
  2. Set it in the self-env to silence the warning:

    # examples/absconda-env.yaml
    variables:
      CLOUDSDK_PYTHON_SITEPACKAGES: "1"
  3. Delete the unused root Dockerfile. It's where feat: opt-in build context, NumPy in tool image, config-based GHCR auth #53's ineffective NumPy/CLOUDSDK_* lines lived, but nothing builds from it (not CI, MANIFEST.in, pyproject.toml, docs, or tests — only temp test files and the generated-Dockerfile concept reference the name). Removing it leaves a single source of truth for the absconda image.

Testing

  • pytest: 117 passed, 2 skipped. ruff + format clean.
  • Rendered the self-env Dockerfile and confirmed ENV CLOUDSDK_PYTHON_SITEPACKAGES="1" appears in the runtime stage and variables: is absent from the conda env.yaml block.
  • End-to-end silencing is verified once this is deployed (the warning comes from the deployed image's gcloud, so it only goes quiet after merge → Deploy-to-NCI rebuilds the image).

🤖 Generated with Claude Code

johnyaku and others added 3 commits June 18, 2026 16:28
… warning

absconda deploys itself by building a conda image from
examples/absconda-env.yaml (not the root Dockerfile), so the NumPy fix
added to the Dockerfile in #53 never reached the deployed image. NumPy is
already in the conda env, but gcloud runs its Python with `-S` and so
can't import it, emitting "consider installing NumPy" on every IAP tunnel.

The missing piece is CLOUDSDK_PYTHON_SITEPACKAGES=1, which must be a
runtime env var in the image. absconda images run binaries directly (no
`conda activate`), so conda's own `variables:` handling never fires.

Render a conda-style `variables:` section in the environment file into
Dockerfile `ENV` lines (in the export block, so it lands in the final
image for every build mode), and strip it from the conda env file the
solver reads. Then set CLOUDSDK_PYTHON_SITEPACKAGES=1 in the self-env.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The deployed absconda image is built by dogfooding
(`absconda publish --file examples/absconda-env.yaml` in deploy.yml), not
from this Dockerfile. It was the artifact #53 added the NumPy/CLOUDSDK
fix to, which never reached the deployed conda image — now handled via the
env file's `variables:` section. Nothing (CI, MANIFEST.in, docs, tests)
references it, so drop it to leave a single source of truth for the image.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@johnyaku
johnyaku force-pushed the feat/env-variables-to-dockerfile-env branch from 6cf3d1b to fe864cd Compare June 18, 2026 06:29
@johnyaku
johnyaku merged commit 7798a65 into main Jun 18, 2026
9 checks passed
@johnyaku
johnyaku deleted the feat/env-variables-to-dockerfile-env branch June 18, 2026 06:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant