Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 36 additions & 8 deletions .github/workflows/build-wheels.yml
Original file line number Diff line number Diff line change
Expand Up @@ -51,29 +51,51 @@ jobs:
if-no-files-found: error
retention-days: 7

smoke-test:
name: Smoke Test Built Artifacts
verify-release-artifacts:
name: Verify Release Artifacts
runs-on: ubuntu-latest
timeout-minutes: 15
needs: [build]

steps:
- name: Checkout
uses: actions/checkout@v4

- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: "3.11"

- name: Install verification tooling
run: |
python -m pip install --upgrade pip
python -m pip install build twine

- name: Download artifacts
uses: actions/download-artifact@v4
with:
name: dist-ubuntu-latest-py3.11
path: artifacts

- name: Verify artifact presence
shell: bash
run: |
wheel_count="$(find artifacts -maxdepth 1 -type f -name '*.whl' | wc -l)"
sdist_count="$(find artifacts -maxdepth 1 -type f -name '*.tar.gz' | wc -l)"
if [ "$wheel_count" -eq 0 ] || [ "$sdist_count" -eq 0 ]; then
echo "Expected at least one wheel and one sdist before publish."
exit 1
fi
ls -l artifacts

- name: Validate distribution metadata
run: python -m twine check artifacts/*

- name: Install wheel and run smoke test
shell: bash
run: |
python -m venv .venv
source .venv/bin/activate
python -m venv .wheel-venv
source .wheel-venv/bin/activate
python -m pip install --upgrade pip

wheel_path="$(find artifacts -type f -name '*.whl' | head -n 1)"
Expand All @@ -82,22 +104,28 @@ jobs:
exit 1
fi
python -m pip install "$wheel_path"
python -c "import pyisolate; print(pyisolate.__version__)"

- name: Install sdist and run smoke test
shell: bash
run: |
python -m venv .sdist-venv
source .sdist-venv/bin/activate
python -m pip install --upgrade pip

sdist_path="$(find artifacts -type f -name '*.tar.gz' | head -n 1)"
if [ -z "$sdist_path" ]; then
echo "No sdist artifact found."
exit 1
fi

temp_dir="$(mktemp -d)"
cd "$temp_dir"
python -m pip install "$sdist_path"
python -c "import pyisolate; print(pyisolate.__version__)"

publish:
name: Publish To PyPI (Trusted Publishing)
runs-on: ubuntu-latest
timeout-minutes: 10
needs: [build, smoke-test]
needs: [build, verify-release-artifacts]
if: >-
github.repository == 'Comfy-Org/pyisolate' &&
github.event_name == 'release' &&
Expand Down
13 changes: 11 additions & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -119,13 +119,22 @@ jobs:
source .venv/bin/activate
uv pip install -e ".[dev,test]"

- name: Run ruff
- name: Run ruff check
run: |
source .venv/bin/activate
ruff check pyisolate tests

- name: Run ruff format check
run: |
source .venv/bin/activate
ruff format --check pyisolate tests

- name: Run mypy
run: |
source .venv/bin/activate
mypy pyisolate
mypy pyisolate tests

- name: Run pytest
run: |
source .venv/bin/activate
pytest -q
Comment on lines +137 to +140
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Verify whether network/integration markers exist and might be included by plain `pytest -q`.

set -euo pipefail

echo "== Network-marked tests =="
rg -n --type=py '@pytest\.mark\.network' tests || true

echo
echo "== Tests that look like real venv/network installers (heuristic) =="
rg -n --type=py 'venv|pip install|uv pip|http://|https://|subprocess' tests || true

echo
echo "If network tests are present, confirm whether lint-and-type should exclude them."

Repository: Comfy-Org/pyisolate

Length of output: 31645


Exclude network-marked tests from the lint job to prevent CI flakiness.

The lint-and-type job runs pytest -q without filtering. Network-marked integration tests exist (@pytest.mark.network in tests/test_cuda_wheels.py, tests/test_conda_integration.py), which spawn real venvs and external network calls (~30s each)—not a testament to quick linting! Use pytest -q -m "not network" to keep lint jobs speedy and reliable.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In @.github/workflows/ci.yml around lines 137 - 140, The lint-and-type CI step
named "Run pytest" currently runs pytest without filtering (the command string
containing "pytest -q"); update that step to exclude network-marked tests by
changing the pytest invocation to include the marker filter (e.g., use pytest -q
-m "not network") while keeping the virtualenv activation (source
.venv/bin/activate) intact so lint jobs skip slow network integration tests.

2 changes: 1 addition & 1 deletion pyisolate/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@
if TYPE_CHECKING:
from .interfaces import IsolationAdapter

__version__ = "0.10.1"
__version__ = "0.10.2"

__all__ = [
"ExtensionBase",
Expand Down
184 changes: 141 additions & 43 deletions pyisolate/_internal/environment.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,8 @@
from typing import Any
from urllib.parse import urlparse

from packaging.utils import parse_wheel_filename

from ..config import ExtensionConfig
from ..path_helpers import serialize_host_snapshot
from .cuda_wheels import (
Expand Down Expand Up @@ -90,12 +92,13 @@ def validate_backend_config(config: ExtensionConfig) -> None:
"Specify at least one channel (e.g. ['conda-forge'])."
)

# conda requires pixi on PATH
if not shutil.which("pixi"):
raise ValueError(
"pixi is required for conda backend but not found. "
"Install: curl -fsSL https://pixi.sh/install.sh | bash"
)
# conda requires pixi — auto-provision if not on PATH
from pyisolate._internal.pixi_provisioner import ensure_pixi

try:
ensure_pixi()
except Exception as e:
raise ValueError(f"pixi is required for conda backend but could not be provisioned: {e}") from e


logger = logging.getLogger(__name__)
Expand Down Expand Up @@ -390,12 +393,12 @@ def install_dependencies(venv_path: Path, config: ExtensionConfig, name: str) ->
if not safe_deps:
return

from packaging.requirements import InvalidRequirement, Requirement
from packaging.utils import canonicalize_name

cuda_wheels_config = config.get("cuda_wheels")
cuda_wheel_runtime: dict[str, object] | None = None
if cuda_wheels_config:
from packaging.requirements import InvalidRequirement, Requirement
from packaging.utils import canonicalize_name

cuda_pkg_names = {canonicalize_name(p) for p in cuda_wheels_config.get("packages", [])}
needs_cuda_probe = False
for dep in safe_deps:
Expand Down Expand Up @@ -436,9 +439,13 @@ def install_dependencies(venv_path: Path, config: ExtensionConfig, name: str) ->
torch_spec = f"torch=={torch_version}"
safe_deps.insert(0, torch_spec)

for extra_url in config.get("extra_index_urls", []):
common_args += ["--extra-index-url", extra_url]

descriptor = {
"dependencies": safe_deps,
"share_torch": config["share_torch"],
"share_torch_no_deps": config.get("share_torch_no_deps", []),
"torch_spec": torch_spec,
"cuda_wheels": cuda_wheels_config,
"cuda_wheel_runtime": cuda_wheel_runtime,
Expand Down Expand Up @@ -497,42 +504,133 @@ def install_dependencies(venv_path: Path, config: ExtensionConfig, name: str) ->
install_targets.append(dep)
i += 1

if cuda_wheels_config:
redacted_targets = [
f"{urlparse(t).netloc}/{Path(urlparse(t).path).name}" if "://" in t else t
for t in install_targets
]
logger.info(
"][ CUDA_WHEEL_INSTALL ext=%s targets=%s",
name,
redacted_targets,
def _run_uv_install(cmd: list[str], *, log_cuda_wheels: bool) -> None:
with subprocess.Popen( # noqa: S603 # Trusted: validated pip/uv install cmd
cmd,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
text=True,
bufsize=1,
) as proc:
assert proc.stdout is not None
output_lines: list[str] = []
for line in proc.stdout:
clean = line.rstrip()
# Filter out pyisolate install messages to avoid polluting logs
# with internal dependency resolution noise that isn't actionable
# for users debugging their own extension dependencies.
if "pyisolate==" not in clean and "pyisolate @" not in clean:
output_lines.append(clean)
if log_cuda_wheels and clean:
logger.info("][ CUDA_WHEEL_UV ext=%s %s", name, clean)
return_code = proc.wait()

if return_code != 0:
detail = "\n".join(output_lines) or "(no output)"
raise RuntimeError(f"Install failed for {name}: {detail}")

share_torch_no_deps = config.get("share_torch_no_deps", [])
if not isinstance(share_torch_no_deps, list):
raise TypeError(
"share_torch_no_deps must be a list of dependency names, "
f"got {type(share_torch_no_deps).__name__}: {share_torch_no_deps!r}"
)

cmd = cmd_prefix + install_targets + common_args

with subprocess.Popen( # noqa: S603 # Trusted: validated pip/uv install cmd
cmd,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
text=True,
bufsize=1,
) as proc:
assert proc.stdout is not None
output_lines: list[str] = []
for line in proc.stdout:
clean = line.rstrip()
# Filter out pyisolate install messages to avoid polluting logs
# with internal dependency resolution noise that isn't actionable
# for users debugging their own extension dependencies.
if "pyisolate==" not in clean and "pyisolate @" not in clean:
output_lines.append(clean)
if cuda_wheels_config and clean:
logger.info("][ CUDA_WHEEL_UV ext=%s %s", name, clean)
return_code = proc.wait()

if return_code != 0:
detail = "\n".join(output_lines) or "(no output)"
raise RuntimeError(f"Install failed for {name}: {detail}")
share_torch_no_deps_names = {canonicalize_name(dep_name) for dep_name in share_torch_no_deps}
regular_targets: list[str] = []
share_torch_no_deps_targets: list[str] = []
cuda_wheel_targets: list[str] = []
cuda_package_dirs: set[str] = set()

if cuda_wheels_config:
package_map = cuda_wheels_config.get("package_map", {})
for package_name in cuda_wheels_config.get("packages", []):
for candidate in (
package_name,
package_name.replace("-", "_"),
package_name.replace("_", "-"),
):
if candidate:
cuda_package_dirs.add(candidate)
mapped_name = package_map.get(package_name)
if mapped_name:
for candidate in (
mapped_name,
mapped_name.replace("-", "_"),
mapped_name.replace("_", "-"),
):
if candidate:
cuda_package_dirs.add(candidate)

for target in install_targets:
if "://" not in target:
if config["share_torch"]:
try:
target_name: str = canonicalize_name(Requirement(target).name)
except InvalidRequirement:
target_name = ""
if target_name and target_name in share_torch_no_deps_names:
share_torch_no_deps_targets.append(target)
continue
regular_targets.append(target)
continue

parsed = urlparse(target)
wheel_name = Path(parsed.path).name
distribution_name = ""
try:
distribution_name, _, _, _ = parse_wheel_filename(wheel_name)
except Exception:
distribution_name = ""
normalized_distribution = distribution_name.replace("-", "_")
if cuda_wheels_config and normalized_distribution in cuda_package_dirs:
cuda_wheel_targets.append(target)
else:
regular_targets.append(target)

if cuda_wheels_config:
if cuda_wheel_targets:
redacted_targets = [
f"{urlparse(t).netloc}/{Path(urlparse(t).path).name}" for t in cuda_wheel_targets
]
logger.info(
"][ CUDA_WHEEL_INSTALL ext=%s targets=%s",
name,
redacted_targets,
)

if share_torch_no_deps_targets:
logger.info(
"][ TORCH_SHARE_NO_DEPS_INSTALL ext=%s targets=%s",
name,
share_torch_no_deps_targets,
)

if regular_targets:
_run_uv_install(cmd_prefix + regular_targets + common_args, log_cuda_wheels=False)
if share_torch_no_deps_targets:
_run_uv_install(
cmd_prefix + ["--no-deps"] + share_torch_no_deps_targets + common_args,
log_cuda_wheels=False,
)
if cuda_wheel_targets:
_run_uv_install(
cmd_prefix + ["--no-deps"] + cuda_wheel_targets + common_args,
log_cuda_wheels=True,
)
else:
if regular_targets:
_run_uv_install(cmd_prefix + regular_targets + common_args, log_cuda_wheels=False)
if share_torch_no_deps_targets:
logger.info(
"][ TORCH_SHARE_NO_DEPS_INSTALL ext=%s targets=%s",
name,
share_torch_no_deps_targets,
)
_run_uv_install(
cmd_prefix + ["--no-deps"] + share_torch_no_deps_targets + common_args,
log_cuda_wheels=False,
)

lock_path.write_text(
json.dumps({"fingerprint": fingerprint, "descriptor": descriptor}, indent=2),
Expand Down
20 changes: 13 additions & 7 deletions pyisolate/_internal/environment_conda.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
from __future__ import annotations

import hashlib
import importlib.metadata
import json
import logging
import os
Expand Down Expand Up @@ -154,7 +155,15 @@ def _generate_pixi_toml(config: ExtensionConfig) -> str:
lines.append("")

lines.append("[pypi-dependencies]")
lines.append(f'pyisolate = {{ path = "{_toml_path_string(_pyisolate_source_path())}" }}')
source_path = _pyisolate_source_path()
if (source_path / "pyproject.toml").exists():
lines.append(f'pyisolate = {{ path = "{_toml_path_string(source_path)}" }}')
else:
try:
version = importlib.metadata.version("pyisolate")
except importlib.metadata.PackageNotFoundError:
version = "0.0.0"
lines.append(f'pyisolate = "=={version}"')
Comment on lines +162 to +166
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Fail fast when pyisolate metadata is unavailable (don’t pin ==0.0.0).

At Line 165, the fallback to "0.0.0" turns a config/runtime problem into a later resolver error (pixi plays hide-and-seek with the real cause). Raise immediately with a clear message instead.

Proposed fix
         else:
             try:
                 version = importlib.metadata.version("pyisolate")
             except importlib.metadata.PackageNotFoundError:
-                version = "0.0.0"
+                raise RuntimeError(
+                    "Unable to resolve pyisolate version for pixi manifest generation. "
+                    "Install pyisolate in the host environment or provide a source tree with pyproject.toml."
+                )
             lines.append(f'pyisolate = "=={version}"')

As per coding guidelines "Fail loud — surface failures immediately. Do not implement silent degradation."

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@pyisolate/_internal/environment_conda.py` around lines 162 - 166, The current
except block swallows PackageNotFoundError and pins pyisolate to "0.0.0" which
hides the real failure; change the except for
importlib.metadata.version("pyisolate") to raise a clear, immediate error (e.g.,
RuntimeError or re-raise PackageNotFoundError with a descriptive message)
instead of setting version = "0.0.0", so the call that builds
lines.append(f'pyisolate = "=={version}"') never silently uses a fake version;
update the exception handling around importlib.metadata.version to include the
descriptive message referencing "pyisolate" metadata retrieval failure.

for dep in pip_deps:
name_part, sep, version_part, extras, marker = _parse_dep(dep)
if cuda_wheel_packages and canonicalize_name(name_part) in cuda_wheel_packages:
Expand Down Expand Up @@ -266,12 +275,9 @@ def create_conda_env(env_path: Path, config: ExtensionConfig, name: str) -> None
"""
env_path.mkdir(parents=True, exist_ok=True)

pixi_path = shutil.which("pixi")
if not pixi_path:
raise RuntimeError(
"pixi is required for conda backend but not found on PATH. "
"Install: curl -fsSL https://pixi.sh/install.sh | bash"
)
from pyisolate._internal.pixi_provisioner import ensure_pixi

pixi_path = ensure_pixi()

cuda_wheels_config = config.get("cuda_wheels")

Expand Down
Loading
Loading