feat: install pip deps constrained to conda's resolved versions

[johnyaku]

Problem

When an environment.yaml mixes conda dependencies with a pip: section, micromamba create -f env.yaml hands the pip subsection to a plain pip install with no constraints. pip doesn't know conda's pins exist, so a pip package (e.g. torch requiring numpy>=2) will silently upgrade the conda-installed numpy past its pin — leaving an environment that violates the conda spec, sometimes with ABI breakage.

This is inherent to composing two independent resolvers over one prefix: conda records its solve in conda-meta/, pip in *.dist-info, and neither treats the other's record as a constraint. conda's own docs flag pip-after-conda as a known hazard. Docker didn't cause it (same thing happens with a plain conda env create), but our build is the one place that sees both halves — so that's where the coordination belongs.

Fix

Split the pip: section out of the conda solve and install it in a second phase constrained to the versions conda just resolved:

RUN micromamba create -y -n env -f /tmp/env.yaml &&     micromamba run -n env python -m pip list --format=freeze > /tmp/conda.constraints.txt &&     micromamba run -n env python -m pip install --no-cache-dir         --constraint /tmp/conda.constraints.txt --requirement /tmp/requirements.txt &&     micromamba run -n env python -m pip check &&     ...

• pip cannot move a conda-managed package off its resolved version.
• A genuine conflict (pip needs a version beyond a conda pin) now fails the build with a pip resolution error, instead of silently clobbering the pin — per the "fail loudly" decision. The fix is to relax the conda pin or pin the pip package compatibly.
• A trailing pip check confirms the final dependency tree is consistent.

Details

• templates.py: _split_conda_pip() separates conda vs pip deps, injects pip into the conda deps when the user didn't list it (so phase 2 can run), and preserves pip option lines (--extra-index-url, git+ URLs). New template context: conda_env_yaml, pip_requirements, has_pip. env_yaml (full) is retained for custom templates.
• Fragments conda_on_base.j2, builder_stage.j2, single_stage.j2: two-phase install (pip step inserted before conda-pack in the packed paths).
• No behavior change for conda-only envs — when there's no pip: section, the rendered Dockerfile is identical to before (verified by test).

Testing

• pytest: 116 passed, 2 skipped (Docker smoke tests). ruff check + format clean.
• New tests: _split_conda_pip (separation, pip injection, no-duplicate, no-pip passthrough) and rendered output (constrained two-phase ordering; no-pip omits the pip steps).
• Rendered the flash_scope --base Dockerfile by hand to confirm output.

Scope / follow-up

• conda + pip only. R (conda + renv) has the same class of problem via a different mechanism (renv shadows conda's R library via .libPaths(); renv is exact-pinned so the conflict is lockfile-vs-conda, not a runaway resolver). The constraints trick doesn't transfer — the renv analog is conflict detection plus "keep R single-managed". Intentionally deferred; docs note this.

🤖 Generated with Claude Code