From c32a8d58fb0b0ae891adf213a7ba857af04278e5 Mon Sep 17 00:00:00 2001 From: Simon Adorf Date: Wed, 29 Apr 2026 21:04:39 +0000 Subject: [PATCH 01/10] Add build-cuml agent skill --- .agents/build-cuml/SKILL.md | 351 ++++++++++++++++++++++++++++++++++++ AGENTS.md | 39 ++++ 2 files changed, 390 insertions(+) create mode 100644 .agents/build-cuml/SKILL.md create mode 100644 AGENTS.md diff --git a/.agents/build-cuml/SKILL.md b/.agents/build-cuml/SKILL.md new file mode 100644 index 0000000000..d0de38fabe --- /dev/null +++ b/.agents/build-cuml/SKILL.md @@ -0,0 +1,351 @@ +--- +name: build-cuml +description: Build cuML (libcuml C++ library and cuml Python package) from source in a conda dev environment, using the repo's build.sh script. Use whenever the user asks to build, compile, install, or rebuild cuML from source, set up a cuML development environment, install local cuML changes, or before testing local edits to cuML C++/CUDA/Cython/Python code. +--- + +# Building cuML + +This skill teaches the agent how to build cuML from source in this repository. The canonical reference is [BUILD.md](../../BUILD.md) at the repo root; this skill captures the high-frequency workflows so the agent can act without re-reading the full doc each time. + +## Quick start (TL;DR) + +For an already-set-up dev env, build and install everything for the local GPU arch: + +```bash +source "$HOME/miniforge3/etc/profile.d/conda.sh" +conda activate "$(git rev-parse --show-toplevel)/.conda-env" +./build.sh --ccache +``` + +This builds and installs `libcuml` (C++), `cuml` (Python), and `prims` (tests) into `$CONDA_PREFIX`. + +> **Note on env layout:** This skill uses an **in-worktree prefix env** at `/.conda-env`. Activation is unambiguous (`conda activate ./.conda-env` always points at the right env), and the env is tied to the worktree — deleting the worktree removes the env. See [In-worktree prefix env](#in-worktree-prefix-env) for details. + +## When to apply this skill + +- The user asks to build / compile / install / rebuild cuML. +- The user wants to set up a cuML development environment. +- The agent has edited C++/CUDA/Cython/Python code in this repo and needs to rebuild before testing. +- A test run fails with `ImportError`, missing `libcuml.so`, stale `.so`, or "module not found" symptoms after a code change — the local install is likely stale and needs a rebuild. + +## 1. Activate the conda dev environment + +cuML development happens inside a conda environment that contains all build and runtime dependencies. **Always activate it before invoking `build.sh`, `cmake`, `pip`, or `pytest`.** + +### In-worktree prefix env + +Use **one conda env per cuML worktree/clone**, stored at `/.conda-env`. Properties: + +- **Deterministic activation**: `conda activate "$(git rev-parse --show-toplevel)/.conda-env"` always activates the env that belongs to the current worktree. No name-derivation logic, no collision handling. +- **Worktree-bound**: deleting the worktree (`rm -rf` or `git worktree remove`) removes the env. No orphan envs accumulating in `~/miniforge3/envs/`. +- **Parallel-agent safe**: two agents in two worktrees can never accidentally activate each other's env. + +The `.conda-env/` directory is large (~5–10 GB; conda hardlinks packages from its global cache, so real disk cost is much smaller). IDEs/editors must be told not to index it — see [Exclude the env from git and the editor](#step-3-exclude-the-env-from-git-and-the-editor). Some tooling (e.g. `conda env list`) won't show prefix envs unless they're active. + +### Initialize conda in a fresh shell + +Conda activation requires conda to be initialized in the shell first: + +```bash +source "$HOME/miniforge3/etc/profile.d/conda.sh" +``` + +(Use the path to the user's conda install — `miniforge3`, `mambaforge`, or `miniconda3`.) + +### Activate the worktree's dev environment + +```bash +conda activate "$(git rev-parse --show-toplevel)/.conda-env" +``` + +If activation fails with "EnvironmentLocationNotFound" or similar, the env hasn't been created yet — see [Create a fresh dev environment](#create-a-fresh-dev-environment) below. + +Verify the env is the right one: + +```bash +echo "$CONDA_PREFIX" # should be /.conda-env +which python # should resolve to inside $CONDA_PREFIX +python -c "import cuml; print(cuml.__file__)" # should be inside this worktree, not another clone +``` + +### Create a fresh dev environment + +#### Step 1: Pick the right env file + +Env files are named `all_cuda-_arch-.yaml`. Two axes: + +```bash +ls conda/environments/all_*.yaml +# all_cuda-129_arch-aarch64.yaml +# all_cuda-129_arch-x86_64.yaml +# all_cuda-131_arch-aarch64.yaml +# all_cuda-131_arch-x86_64.yaml +``` + +**Architecture (`arch-`)** — must match the host CPU architecture. Always use: + +```bash +uname -m # → x86_64 or aarch64 +``` + +**CUDA version (`cuda-`)** — this is the version of the CUDA toolkit and CUDA runtime that conda will install into the env (cuML does not require a system CUDA install). Choose based on the host's NVIDIA driver and GPU compute capability: + +| Env file | Conda CUDA | Min host driver | Min GPU compute capability | +| --- | --- | --- | --- | +| `cuda-131` (recommended default) | 13.1 | R580+ | 7.5 (Turing or newer) | +| `cuda-129` | 12.9 | R525+ | 7.0 (Volta or newer) | + +**Decision rule:** + +1. **Default to `cuda-131`** unless one of the conditions below applies. +2. Use `cuda-129` if the host has a **Volta (sm_70) GPU** (e.g. V100) — CUDA 13 dropped Volta support. +3. Use `cuda-129` if the host's NVIDIA driver is older than R580 — `nvidia-smi` will show the max CUDA version supported. + +**Detect the right file automatically:** + +```bash +ARCH=$(uname -m) + +# Read GPU compute capability (e.g. "7.0", "7.5", "8.0", "9.0") +CC=$(nvidia-smi --query-gpu=compute_cap --format=csv,noheader,nounits | head -n1) + +# Read the max CUDA version the installed driver supports (e.g. "13.0", "12.9") +DRIVER_CUDA=$(nvidia-smi --query-gpu=driver_version --format=csv,noheader | head -n1) +echo "Driver: $DRIVER_CUDA GPU compute cap: $CC Arch: $ARCH" + +# Use cuda-129 if any Volta GPU is in the system; otherwise prefer cuda-131 +if awk "BEGIN {exit !($CC < 7.5)}"; then + CUDA_TAG=129 +else + CUDA_TAG=131 +fi +ENV_FILE="conda/environments/all_cuda-${CUDA_TAG}_arch-${ARCH}.yaml" +echo "Using $ENV_FILE" +``` + +If unsure, on multi-GPU hosts query each GPU and pick the lowest compute capability. If the host has no GPU (CPU-only build), `cuda-131` is fine. + +#### Step 2: Configure exclusions (do this BEFORE creating the env) + +> **Critical ordering:** add the git and editor exclusions *before* running `conda create`. Otherwise git will momentarily see ~20k+ untracked files inside `.conda-env/`, the VS Code/Cursor Git extension will warn `"too many active changes, only a subset of Git features will be enabled"`, and editor indexing/file-watching will spike CPU before the exclusions kick in. + +The `.conda-env/` directory must never be committed and must be excluded from editor indexing/search. Two git-side options (pick one): + +**Option A — worktree-local exclude (recommended for agents):** doesn't modify any tracked file. + +```bash +grep -qxF '/.conda-env/' .git/info/exclude || echo '/.conda-env/' >> .git/info/exclude +``` + +**Option B — repo-wide `.gitignore`:** if the convention is repo-wide, add `/.conda-env/` to `.gitignore` and commit it. + +For VS Code / Cursor, also write `.vscode/settings.json` (worktree-local, not committed) to keep the editor responsive **and** point the Python extension at the env so terminals, debug, tests, and IntelliSense all use it automatically. Target settings: + +```json +{ + "python.defaultInterpreterPath": "${workspaceFolder}/.conda-env/bin/python", + "python.terminal.activateEnvironment": true, + "files.watcherExclude": { "**/.conda-env/**": true }, + "files.exclude": { "**/.conda-env": true }, + "search.exclude": { "**/.conda-env": true } +} +``` + +Run the script below from the worktree root to merge these into any existing `.vscode/settings.json` without clobbering it (uses the system `python3`, no conda env required): + +```bash +mkdir -p .vscode +python3 - <<'PY' +import json, pathlib +p = pathlib.Path(".vscode/settings.json") +data = json.loads(p.read_text()) if p.exists() else {} +data["python.defaultInterpreterPath"] = "${workspaceFolder}/.conda-env/bin/python" +data["python.terminal.activateEnvironment"] = True +data.setdefault("files.watcherExclude", {})["**/.conda-env/**"] = True +data.setdefault("files.exclude", {})["**/.conda-env"] = True +data.setdefault("search.exclude", {})["**/.conda-env"] = True +p.write_text(json.dumps(data, indent=2) + "\n") +PY +``` + +Notes: + +- `python.defaultInterpreterPath` only takes effect on first workspace open or after running `Python: Clear Workspace Interpreter Setting`. If the workspace was already open with a different interpreter, run that command (or pick the new interpreter via `Python: Select Interpreter`) once to switch. +- `python.terminal.activateEnvironment` is `true` by default; listing it makes the intent explicit and survives users who toggled it off globally. + +#### Step 3: Create and populate the env + +Match the Python version to what the YAML supports (currently `>=3.11,<=3.14`). Create the env at `/.conda-env`: + +```bash +PREFIX_ENV="$(git rev-parse --show-toplevel)/.conda-env" +conda create -y --prefix "$PREFIX_ENV" python=3.14 +conda env update --prefix "$PREFIX_ENV" --file="$ENV_FILE" +conda activate "$PREFIX_ENV" +``` + +After populating, sanity-check that git doesn't see env files: + +```bash +git status -s | wc -l # should be 0 (or just your real changes); not thousands +``` + +### Update an existing dev environment + +After pulling new commits that change `dependencies.yaml` or the env files, refresh the env using **the same env file the env was originally created from** (don't switch CUDA versions on an existing env — recreate instead): + +```bash +PREFIX_ENV="$(git rev-parse --show-toplevel)/.conda-env" +conda env update --prefix "$PREFIX_ENV" --file="$ENV_FILE" +``` + +### Removing an env + +```bash +conda deactivate 2>/dev/null || true +rm -rf "$(git rev-parse --show-toplevel)/.conda-env" +``` + +(Or `conda env remove --prefix "$(git rev-parse --show-toplevel)/.conda-env"`. The `rm -rf` is faster and works even if conda isn't initialized.) + +## 2. Build with `build.sh` + +`build.sh` lives at the repo root and is the recommended entry point. Targets are space-separated; flags can be mixed in. With no targets it builds and installs `libcuml`, `cuml`, and `prims` for the detected GPU arch. + +### Common targets + +```bash +./build.sh # libcuml + cuml + prims (default) +./build.sh libcuml # C++ library only +./build.sh cuml # Python package only (assumes libcuml installed) +./build.sh libcuml cuml # both, explicit +./build.sh clean # wipe all build artifacts (run first if state is corrupt) +./build.sh prims bench # ml-prims tests + C++ benchmark +``` + +### Most useful flags + +| Flag | Effect | +| --- | --- | +| `--ccache` | Cache compilations via ccache/sccache. **Strongly recommended for any iterative work.** | +| `-g` | Build with debug info (`RelWithDebInfo`). | +| `-v` | Verbose build output. | +| `-n` | Build but don't install. | +| `--allgpuarch` | Build for all RAPIDS-supported archs (slow; default is `NATIVE` = local GPU only). | +| `--singlegpu` | Drop multi-GPU/MNMG components from `libcuml` and `cuml`. Faster, smaller. | +| `--nolibcumltest` | Skip C++ test binaries (faster libcuml builds). | +| `--configure-only` | Run cmake configure but don't compile (e.g. for clang-tidy DB generation). | + +### Environment variables + +- `PARALLEL_LEVEL=N` — limit ninja parallelism. **Use this on shared machines or when ninja OOMs the system.** Default is `nproc`. +- `INSTALL_PREFIX=/path` — install location. Default is `$CONDA_PREFIX` when a conda env is active. +- `CMAKE_GENERATOR='Unix Makefiles'` — switch from Ninja to make. +- `CUML_EXTRA_CMAKE_ARGS="..."` — append extra `-D...` flags to cmake. +- `CUML_EXTRA_PYTHON_ARGS="..."` — append extra args to the `pip install` step. + +### Recommended fast-iteration command + +For day-to-day editing (any code: C++, CUDA, Cython, Python): + +```bash +PARALLEL_LEVEL=$(nproc) ./build.sh --ccache +``` + +For C++-only edits, skip the Python rebuild: + +```bash +./build.sh libcuml --ccache +``` + +For Python-only edits (no C++ touched), skip the C++ rebuild: + +```bash +./build.sh cuml +``` + +## 3. ccache / sccache for fast rebuilds + +Branch switching, debug↔release toggles, and CI-style rebuilds become much cheaper with a compile cache. cuML supports both `ccache` and `sccache` (sccache is preferred in CI). + +### Enable + +Pass `--ccache` to `build.sh`. This sets `-DUSE_CCACHE=ON` for the cmake configure step. + +### Verify a cache is installed and used + +```bash +which ccache sccache 2>/dev/null + +# After a build, inspect stats: +ccache -s # if using ccache +sccache --show-stats # if using sccache +``` + +If neither is on `PATH`, install one into the active dev env: + +```bash +conda install -y -c conda-forge ccache +# or +conda install -y -c conda-forge sccache +``` + +### Cache hit reporting in builds + +Add `--build-metrics --incl-cache-stats` to record cache hit rate and produce an HTML build report at `cpp/build/ninja_log.html`: + +```bash +./build.sh libcuml --ccache --build-metrics --incl-cache-stats +``` + +### When to clear the cache + +- Compiler version changed (`gcc`/`nvcc` upgrade) — invalidate to avoid stale objects: `ccache -C` or `sccache --zero-stats` then re-run. +- After CUDA toolkit upgrade. +- Otherwise: leave it alone; clearing the cache defeats the purpose. + +## 4. Verify the build + +After the install step finishes, confirm libcuml and cuml are importable from the active env: + +```bash +test -f "$CONDA_PREFIX/lib/libcuml.so" && echo "libcuml.so installed" +python -c "import cuml; print(cuml.__version__, cuml.__file__)" +``` + +The `cuml.__file__` path should be inside the active conda env (or the editable source tree). + +## 5. Common gotchas + +- **`ImportError` after pulling new commits**: rebuild — the C++ ABI or Cython-generated code likely changed. +- **`libcuml.so` not found at runtime**: `INSTALL_PREFIX` mismatched the active env. Re-run `build.sh` with the correct env activated. +- **`cuml.__file__` points to a different worktree than the one you're editing**: another worktree's env is active. Run `conda activate "$(git rev-parse --show-toplevel)/.conda-env"` and rebuild — see [In-worktree prefix env](#in-worktree-prefix-env). +- **Out-of-memory or thermal throttling during build**: lower `PARALLEL_LEVEL` (e.g. `PARALLEL_LEVEL=8`). +- **Stale build state after a failed build**: run `./build.sh clean` then rebuild from scratch. +- **Building without a GPU**: the build itself works on CPU-only hosts; only running cuML at test time requires a GPU. +- **`--singlegpu` + multi-GPU tests**: skip MNMG tests (e.g. `pytest --ignore=cuml/tests/dask --ignore=cuml/tests/test_nccl.py`). + +## 6. Manual cmake / pip path + +`build.sh` is a thin wrapper around `cmake` + `pip install`. If the user explicitly wants the manual flow, or `build.sh` is unavailable, see the **"Manual Process"** section of [BUILD.md](../../BUILD.md). The two key invocations are: + +```bash +# C++ library +cd cpp && mkdir -p build && cd build +cmake -DCMAKE_INSTALL_PREFIX="$CONDA_PREFIX" \ + -DCMAKE_CUDA_ARCHITECTURES=NATIVE \ + -DUSE_CCACHE=ON .. +cmake --build . -j"$(nproc)" --target install + +# Python package (from repo root) +python -m pip install --no-build-isolation --no-deps \ + --config-settings rapidsai.disable-cuda=true \ + python/cuml +``` + +## Additional resources + +- Full build docs and all cmake flags: [BUILD.md](../../BUILD.md) +- Contributing workflow (pre-commit, clang-tidy, branch naming): [CONTRIBUTING.md](../../CONTRIBUTING.md) +- Conda environment files: `conda/environments/all_cuda-*_arch-*.yaml` +- Build script source (the source of truth for flag behavior): `build.sh` diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000000..2fb837bd5d --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,39 @@ +# Agent Guide + +Onboarding notes for AI coding agents working in the cuML repository. + +## Repository overview + +This is [cuML](https://github.com/rapidsai/cuml), the RAPIDS GPU-accelerated machine learning library. The repo contains: + +- `cpp/` — `libcuml` C++/CUDA library and ML primitives. +- `python/cuml/` — `cuml` Python package (Cython + Python, scikit-learn-compatible API). +- `conda/environments/` — pinned conda env files for development and CI. +- `ci/` — CI scripts. +- `docs/` — Sphinx documentation. + +## Building cuML + +Before testing any local code change that touches C++/CUDA/Cython, the local install must be rebuilt. Follow the dedicated build skill: + +- [.agents/build-cuml/SKILL.md](.agents/build-cuml/SKILL.md) — conda env setup, `build.sh` usage, ccache, and common gotchas. + +The full build reference is [BUILD.md](BUILD.md). + +## Contributing workflow + +For pre-commit hooks, clang-tidy, branch naming, and the PR process, see [CONTRIBUTING.md](CONTRIBUTING.md). + +Key conventions: + +- Linter errors are auto-fixed by pre-commit hooks — don't fix them manually unless asked. +- Always activate the conda dev environment before running `build.sh`, `pytest`, or `pip`. + +## Code review guidelines + +When reviewing changes, agents should follow the layer-specific review rubrics: + +- C++/CUDA changes: [cpp/agents.md](cpp/agents.md) +- Python changes: [python/agents.md](python/agents.md) + +Both files focus on CRITICAL and HIGH issues only and target a sub-3% false-positive rate. From cf5a392264ea301947adcf65ada7fd44d0b443f5 Mon Sep 17 00:00:00 2001 From: Simon Adorf Date: Wed, 29 Apr 2026 21:07:45 +0000 Subject: [PATCH 02/10] Add test-cuml agent skill. --- .agents/test-cuml/SKILL.md | 240 +++++++++++++++++++++++++++++++++++++ AGENTS.md | 8 ++ 2 files changed, 248 insertions(+) create mode 100644 .agents/test-cuml/SKILL.md diff --git a/.agents/test-cuml/SKILL.md b/.agents/test-cuml/SKILL.md new file mode 100644 index 0000000000..a0a456cb55 --- /dev/null +++ b/.agents/test-cuml/SKILL.md @@ -0,0 +1,240 @@ +--- +name: test-cuml +description: Run cuML test suites — C++ gtests, standard Python tests, dask tests, cuml.accel tests, upstream library tests (sklearn/umap/hdbscan) under cuml.accel, and integration tests. Use when the user asks to run, invoke, or debug cuML tests, pytest, ctest, run sklearn tests under cuml.accel, run dask tests, run integration tests, or wants to verify a code change works. +--- + +# Running cuML Tests + +This skill covers all cuML test suites. The canonical source of truth for each suite is the corresponding `ci/run_*` and `ci/test_*` script — when in doubt, read those first. + +## 0. Prerequisites + +**Activate the worktree env before running any tests:** + +```bash +source "$HOME/miniforge3/etc/profile.d/conda.sh" +conda activate "$(git rev-parse --show-toplevel)/.conda-env" +``` + +- After editing C++/CUDA/Cython code, rebuild before testing. See [.agents/build-cuml/SKILL.md](../build-cuml/SKILL.md). +- All Python tests require a GPU. Most suites support `pytest-xdist` parallelism (`-n auto` or `--numprocesses=N --dist=worksteal`). + +--- + +## 1. C++ gtests (libcuml) + +Test binaries are built by the default `./build.sh` targets and installed to `$CONDA_PREFIX/bin/gtests/libcuml/`. + +**Canonical entry point:** [`ci/run_ctests.sh`](../../ci/run_ctests.sh) — changes into the install dir (or falls back to `cpp/build/latest`) and runs `ctest --output-on-failure --no-tests=error`. + +```bash +# Run all gtests in parallel (matches CI) +./ci/run_ctests.sh -j9 + +# Run a single test suite by regex +./ci/run_ctests.sh -R SG_DBSCAN_TEST + +# Run only prims tests +./ci/run_ctests.sh -R PRIMS_ +``` + +**Direct binary invocation** (faster iteration on one algorithm): + +```bash +cd "$CONDA_PREFIX/bin/gtests/libcuml" +./SG_DBSCAN_TEST --gtest_filter='*' +./SG_RF_TEST --gtest_filter='RandomForestClassifierTest*' +./SG_DBSCAN_TEST --gtest_list_tests # list available test cases +``` + +Test binary names come from `cpp/tests/CMakeLists.txt` (`SG_*` for single-GPU, `PRIMS_*` for ml-prims, `MG_*` for multi-GPU). Prims tests install to `$CONDA_PREFIX/bin/gtests/libcuml_prims/`. + +If binaries are missing, rebuild: `./build.sh` (default targets include the test build). + +--- + +## 2. Standard Python tests (single-GPU, no dask) + +**Path:** `python/cuml/tests/` (ignoring the `dask/` subdir). +**Canonical entry point:** [`ci/run_cuml_singlegpu_pytests.sh`](../../ci/run_cuml_singlegpu_pytests.sh) + +```bash +# Run the full suite (CI-style, parallel) +./ci/run_cuml_singlegpu_pytests.sh --numprocesses=8 --dist=worksteal + +# Quick targeted run +./ci/run_cuml_singlegpu_pytests.sh -k test_kmeans + +# Single file, verbose +./ci/run_cuml_singlegpu_pytests.sh -v python/cuml/tests/test_kmeans.py + +# Alternatively, run pytest directly from the tests dir +cd python/cuml/tests +pytest --ignore=dask -k test_linear_regression -x +``` + +**Test categories** (from [`python/cuml/tests/conftest.py`](../../python/cuml/tests/conftest.py)): + +| Flag | Runs | Default | +| --- | --- | --- | +| `--run_unit` | `@pytest.mark.unit` tests | Yes (implied when no `--run_*` flag given) | +| `--run_quality` | `@pytest.mark.quality` tests | No | +| `--run_stress` | `@pytest.mark.stress` tests | No | +| `--run_memleak` | `@pytest.mark.memleak` tests | No | + +`HYPOTHESIS_ENABLED=true` enables full Hypothesis search (CI nightly behavior; default in CI runs quality/stress profiles). + +--- + +## 3. Dask tests (multi-GPU) + +**Path:** `python/cuml/tests/dask/` — each test spins up a `LocalCUDACluster` (TCP by default, UCXX optional). +**Canonical entry point:** [`ci/run_cuml_dask_pytests.sh`](../../ci/run_cuml_dask_pytests.sh) + +```bash +# Standard TCP run (default) +./ci/run_cuml_dask_pytests.sh -k test_dask_kmeans + +# Full parallel CI run +./ci/run_cuml_dask_pytests.sh --numprocesses=8 --dist=worksteal + +# UCXX-only tests (skips all non-UCX tests) +./ci/run_cuml_dask_pytests.sh --run_ucx +``` + +CI runs both TCP and UCXX passes — see [`ci/test_python_dask.sh`](../../ci/test_python_dask.sh). + +**Note on the env:** The single-GPU CI job intentionally errors if `dask` is importable. Use a dask-enabled env (one that has `cuml[dask,test-dask]`: `dask-cudf`, `raft-dask`, `dask-cuda`, `dask-ml`) when running the dask suite. + +--- + +## 4. cuml.accel tests (proxy estimator suite) + +**Path:** `python/cuml/cuml_accel_tests/` — `conftest.py` calls `cuml.accel.install()` so all collected tests run with the proxy enabled. The `upstream/` subdir is in `collect_ignore` and must be run separately (see §5). +**Canonical entry point:** [`ci/run_cuml_singlegpu_accel_pytests.sh`](../../ci/run_cuml_singlegpu_accel_pytests.sh) + +```bash +# Full suite (CI-style, parallel) +./ci/run_cuml_singlegpu_accel_pytests.sh --numprocesses=8 --dist=worksteal + +# Targeted run +./ci/run_cuml_singlegpu_accel_pytests.sh -k test_estimator_proxy + +# Direct pytest +cd python/cuml/cuml_accel_tests +pytest test_estimator_proxy.py -k LogisticRegression -x -v +``` + +The `integration/` subdir inside `cuml_accel_tests/` belongs to the integration suite (§6), not here. + +--- + +## 5. Upstream tests (scikit-learn / umap / hdbscan under cuml.accel) + +Each upstream library has a `run-tests.sh` in `python/cuml/cuml_accel_tests/upstream//`. The script runs the library's own test suite with `-p cuml.accel` and an `xfail-list.yaml`. + +Full workflow reference: [`python/cuml/cuml_accel_tests/upstream/README.md`](../../python/cuml/cuml_accel_tests/upstream/README.md). + +### scikit-learn + +[`upstream/scikit-learn/run-tests.sh`](../../python/cuml/cuml_accel_tests/upstream/scikit-learn/run-tests.sh) — uses `--pyargs sklearn`. + +```bash +# Run all sklearn tests under cuml.accel +./python/cuml/cuml_accel_tests/upstream/scikit-learn/run-tests.sh + +# Targeted +./python/cuml/cuml_accel_tests/upstream/scikit-learn/run-tests.sh -k test_kmeans + +# Parallel + XML report +./python/cuml/cuml_accel_tests/upstream/scikit-learn/run-tests.sh \ + -n auto --dist worksteal --junitxml=report.xml + +# Run known-xfailed tests to see actual results +./python/cuml/cuml_accel_tests/upstream/scikit-learn/run-tests.sh --runxfail +``` + +### umap + +[`upstream/umap/run-tests.sh`](../../python/cuml/cuml_accel_tests/upstream/umap/run-tests.sh) — clones the umap repo at the tag matching the installed `umap.__version__` into `umap-upstream/`. + +```bash +./python/cuml/cuml_accel_tests/upstream/umap/run-tests.sh +./python/cuml/cuml_accel_tests/upstream/umap/run-tests.sh -k test_umap_transform +``` + +### hdbscan + +[`upstream/hdbscan/run-tests.sh`](../../python/cuml/cuml_accel_tests/upstream/hdbscan/run-tests.sh) — uses `--pyargs hdbscan.tests`. + +```bash +./python/cuml/cuml_accel_tests/upstream/hdbscan/run-tests.sh +./python/cuml/cuml_accel_tests/upstream/hdbscan/run-tests.sh -k test_hdbscan +``` + +### Analyzing results + +```bash +# Summary (pass/fail counts) +./python/cuml/cuml_accel_tests/upstream/summarize-results.py report.xml + +# Verbose — show failure details +./python/cuml/cuml_accel_tests/upstream/summarize-results.py -v report.xml + +# Enforce a minimum pass rate (e.g. 80%) +./python/cuml/cuml_accel_tests/upstream/summarize-results.py -f 80 report.xml + +# Tracebacks for specific failures +./python/cuml/cuml_accel_tests/upstream/summarize-results.py \ + --format=traceback -k logistic report.xml + +# Generate a new xfail list from results +./python/cuml/cuml_accel_tests/upstream/summarize-results.py \ + --format=xfail_list report.xml > new-xfail-list.yaml +``` + +For editing the xfail list (adding/updating reasons, markers, conditions), use `xfail_manager.py` — see the [README](../../python/cuml/cuml_accel_tests/upstream/README.md). + +--- + +## 6. Integration tests (cudf.pandas + cuml.accel) + +Runs the standard `python/cuml/tests/` suite (not dask) with `-p cudf.pandas` so pandas operations route through the cudf-pandas wrapper, plus `--quick_run` to keep runtime bounded. +**Canonical entry point:** [`ci/run_cuml_integration_pytests.sh`](../../ci/run_cuml_integration_pytests.sh) + +```bash +# Full CI run +./ci/run_cuml_integration_pytests.sh --numprocesses=8 --dist=worksteal + +# Targeted +./ci/run_cuml_integration_pytests.sh -k test_linear_regression + +# Direct pytest equivalent +cd python/cuml/tests +pytest -p cudf.pandas --cache-clear --ignore=dask --quick_run . +``` + +Tests marked `@pytest.mark.cudf_pandas` are skipped unless `-p cudf.pandas` is loaded (enforced in `tests/conftest.py`). + +CI script: [`ci/test_python_integration.sh`](../../ci/test_python_integration.sh). + +--- + +## 7. Common gotchas + +- **`ImportError` or wrong `cuml.__file__`**: another worktree's env is active. Run `conda activate "$(git rev-parse --show-toplevel)/.conda-env"` and rebuild. See the [build skill](../build-cuml/SKILL.md). +- **C++ test binaries not found**: the install dir `$CONDA_PREFIX/bin/gtests/libcuml/` is absent. Run `./build.sh` (default builds and installs the tests). +- **Dask import error in single-GPU env**: the single-GPU CI script (`test_python_singlegpu.sh`) intentionally fails if `dask` is installed. Use a dask-capable env for §3 tests. +- **`UnmatchedXfailTests` warning in upstream tests**: the xfail list references a test that no longer exists in the installed version of sklearn/umap/hdbscan. See the upstream [README](../../python/cuml/cuml_accel_tests/upstream/README.md) for how to triage and update the list. +- **xdist worker crash after CUDA error**: this is intentional — `tests/conftest.py` calls `os._exit(1)` on sticky CUDA errors so xdist spawns a clean worker. Don't attempt to suppress it. + +--- + +## 8. Additional resources + +- Build skill: [.agents/build-cuml/SKILL.md](../build-cuml/SKILL.md) +- Full build doc and manual cmake/test paths: [BUILD.md](../../BUILD.md) +- Upstream test workflow and xfail management: [python/cuml/cuml_accel_tests/upstream/README.md](../../python/cuml/cuml_accel_tests/upstream/README.md) +- Pytest markers and filterwarnings: [`python/cuml/pyproject.toml`](../../python/cuml/pyproject.toml) `[tool.pytest.ini_options]` +- Test suite conftest (markers, xdist hooks, Hypothesis profiles): [`python/cuml/tests/conftest.py`](../../python/cuml/tests/conftest.py) +- Dask conftest (cluster fixtures, `--run_ucx` option): [`python/cuml/tests/dask/conftest.py`](../../python/cuml/tests/dask/conftest.py) diff --git a/AGENTS.md b/AGENTS.md index 2fb837bd5d..c4d05e372c 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -20,6 +20,14 @@ Before testing any local code change that touches C++/CUDA/Cython, the local ins The full build reference is [BUILD.md](BUILD.md). +## Running tests + +For running the C++ gtests, the standard pytest suite, dask tests, the cuml.accel suite, upstream-library tests under cuml.accel, and the integration tests, follow the dedicated test skill: + +- [.agents/test-cuml/SKILL.md](.agents/test-cuml/SKILL.md) — env activation, `ci/run_*` entry points, common pytest args, and per-suite gotchas. + +The CI scripts under [ci/](ci/) (`test_cpp.sh`, `test_python_singlegpu.sh`, `test_python_dask.sh`, `test_python_integration.sh`, `test_python_scikit_learn_tests.sh`, `test_python_cuml_accel_upstream.sh`) are the source of truth for how each suite is invoked. + ## Contributing workflow For pre-commit hooks, clang-tidy, branch naming, and the PR process, see [CONTRIBUTING.md](CONTRIBUTING.md). From bf87fc2b4d775bdc7903b468776f22f8e95107df Mon Sep 17 00:00:00 2001 From: Simon Adorf Date: Wed, 29 Apr 2026 21:18:42 +0000 Subject: [PATCH 03/10] Add note on work-around for building on sm_121 --- .agents/build-cuml/SKILL.md | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/.agents/build-cuml/SKILL.md b/.agents/build-cuml/SKILL.md index d0de38fabe..6b8697c11c 100644 --- a/.agents/build-cuml/SKILL.md +++ b/.agents/build-cuml/SKILL.md @@ -19,6 +19,16 @@ conda activate "$(git rev-parse --show-toplevel)/.conda-env" This builds and installs `libcuml` (C++), `cuml` (Python), and `prims` (tests) into `$CONDA_PREFIX`. +> **Important (sm_121 / new arch + conda RAPIDS libs):** there is a known bug in the default `NATIVE` build path (documented in [rapidsai/cuml#8021](https://github.com/rapidsai/cuml/issues/8021)). Until fixed upstream, use one of these workarounds when mixing with conda-installed RAPIDS shared libraries: +> +> ```bash +> ./build.sh --allgpuarch +> # or +> CUML_EXTRA_CMAKE_ARGS="-DCMAKE_CUDA_ARCHITECTURES=120-real" ./build.sh +> ``` +> +> Bug details: [rapidsai/cuml#8021](https://github.com/rapidsai/cuml/issues/8021). + > **Note on env layout:** This skill uses an **in-worktree prefix env** at `/.conda-env`. Activation is unambiguous (`conda activate ./.conda-env` always points at the right env), and the env is tied to the worktree — deleting the worktree removes the env. See [In-worktree prefix env](#in-worktree-prefix-env) for details. ## When to apply this skill From 5f905b8f847e2aa1369a2d0ea948e76a246ec7d7 Mon Sep 17 00:00:00 2001 From: Simon Adorf Date: Wed, 29 Apr 2026 21:26:19 +0000 Subject: [PATCH 04/10] minor improvement to test-cuml skill --- .agents/test-cuml/SKILL.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/.agents/test-cuml/SKILL.md b/.agents/test-cuml/SKILL.md index a0a456cb55..0800de00ff 100644 --- a/.agents/test-cuml/SKILL.md +++ b/.agents/test-cuml/SKILL.md @@ -17,7 +17,9 @@ conda activate "$(git rev-parse --show-toplevel)/.conda-env" ``` - After editing C++/CUDA/Cython code, rebuild before testing. See [.agents/build-cuml/SKILL.md](../build-cuml/SKILL.md). +- If you see `No module named pytest` (or `python` resolves to base Miniforge instead of the worktree env), the dev environment is not active — the worktree env includes `pytest` and the rest of the test stack. - All Python tests require a GPU. Most suites support `pytest-xdist` parallelism (`-n auto` or `--numprocesses=N --dist=worksteal`). +- CI entry scripts such as `ci/run_cuml_singlegpu_pytests.sh` call `python` on your `PATH`; they assume an appropriate environment is already activated --- @@ -154,6 +156,8 @@ Full workflow reference: [`python/cuml/cuml_accel_tests/upstream/README.md`](../ ./python/cuml/cuml_accel_tests/upstream/scikit-learn/run-tests.sh --runxfail ``` +**`run-tests.sh` and the xfail list:** the script always passes `--pyargs sklearn` and [`--xfail-list=.../xfail-list.yaml`](../../python/cuml/cuml_accel_tests/upstream/scikit-learn/xfail-list.yaml). The `cuml.accel` pytest plugin marks xfails and **requires every condition-matched id in the YAML to match a collected test**; if you instead run e.g. `pytest -p cuml.accel --pyargs sklearn.neighbors.tests.test_kde --xfail-list=...`, most xfail entries have no corresponding collected test and you get `UnmatchedXfailTests` (treated as an error by [`python/cuml/cuml_accel_tests/upstream/pytest.ini`](../../python/cuml/cuml_accel_tests/upstream/pytest.ini)). For targeted runs **with** the official xfail list, keep using `run-tests.sh` and filter with `-k` (full sklearn collection, then deselect; still usually seconds of overhead). For a **fast local loop** on one module only, you may run `python -m pytest -p cuml.accel --pyargs sklearn. ...` **without** `--xfail-list` — but tests listed as xfail in the YAML may then appear as real failures. + ### umap [`upstream/umap/run-tests.sh`](../../python/cuml/cuml_accel_tests/upstream/umap/run-tests.sh) — clones the umap repo at the tag matching the installed `umap.__version__` into `umap-upstream/`. @@ -225,7 +229,7 @@ CI script: [`ci/test_python_integration.sh`](../../ci/test_python_integration.sh - **`ImportError` or wrong `cuml.__file__`**: another worktree's env is active. Run `conda activate "$(git rev-parse --show-toplevel)/.conda-env"` and rebuild. See the [build skill](../build-cuml/SKILL.md). - **C++ test binaries not found**: the install dir `$CONDA_PREFIX/bin/gtests/libcuml/` is absent. Run `./build.sh` (default builds and installs the tests). - **Dask import error in single-GPU env**: the single-GPU CI script (`test_python_singlegpu.sh`) intentionally fails if `dask` is installed. Use a dask-capable env for §3 tests. -- **`UnmatchedXfailTests` warning in upstream tests**: the xfail list references a test that no longer exists in the installed version of sklearn/umap/hdbscan. See the upstream [README](../../python/cuml/cuml_accel_tests/upstream/README.md) for how to triage and update the list. +- **`UnmatchedXfailTests` in upstream sklearn tests**: (1) the xfail list references a test id that no longer exists in the installed library version — triage and update the list (see the upstream [README](../../python/cuml/cuml_accel_tests/upstream/README.md)). (2) You passed `--xfail-list` while collecting only a **subset** of the suite (e.g. `--pyargs sklearn.neighbors.tests.test_kde`); use [`run-tests.sh`](../../python/cuml/cuml_accel_tests/upstream/scikit-learn/run-tests.sh) with `-k` instead, or drop `--xfail-list` for a quick narrow run. - **xdist worker crash after CUDA error**: this is intentional — `tests/conftest.py` calls `os._exit(1)` on sticky CUDA errors so xdist spawns a clean worker. Don't attempt to suppress it. --- From a1436bf6948a59d7066f021846636c34eba499d2 Mon Sep 17 00:00:00 2001 From: Simon Adorf Date: Wed, 29 Apr 2026 21:31:37 +0000 Subject: [PATCH 05/10] Update CODEOWNERS --- .github/CODEOWNERS | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index a38c72cbcc..7d30438ae8 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -20,6 +20,10 @@ CMakeLists.txt @rapidsai/cuml-cmake-codeowners *.cmake @rapidsai/cuml-cmake-codeowners **/cmake/ @rapidsai/cuml-cmake-codeowners +# agent instruction owners +/AGENTS.md @rapidsai/cuml-python-codeowners @rapidsai/cuml-cpp-codeowners +/.agents/ @rapidsai/cuml-python-codeowners @rapidsai/cuml-cpp-codeowners + #CI code owners /.github/ @rapidsai/ci-codeowners /ci/ @rapidsai/ci-codeowners From 50d799c339fadb76165bd9c2087e26d905f33957 Mon Sep 17 00:00:00 2001 From: Simon Adorf Date: Thu, 30 Apr 2026 20:07:01 +0000 Subject: [PATCH 06/10] Split conda dev environment creation into separate skill. --- .agents/build-cuml/SKILL.md | 181 +---------------------- .agents/setup-dev-environment/SKILL.md | 190 +++++++++++++++++++++++++ .agents/test-cuml/SKILL.md | 4 +- 3 files changed, 200 insertions(+), 175 deletions(-) create mode 100644 .agents/setup-dev-environment/SKILL.md diff --git a/.agents/build-cuml/SKILL.md b/.agents/build-cuml/SKILL.md index 6b8697c11c..f0513f9000 100644 --- a/.agents/build-cuml/SKILL.md +++ b/.agents/build-cuml/SKILL.md @@ -1,6 +1,6 @@ --- name: build-cuml -description: Build cuML (libcuml C++ library and cuml Python package) from source in a conda dev environment, using the repo's build.sh script. Use whenever the user asks to build, compile, install, or rebuild cuML from source, set up a cuML development environment, install local cuML changes, or before testing local edits to cuML C++/CUDA/Cython/Python code. +description: Build cuML (libcuml C++ library and cuml Python package) from source in a conda dev environment, using the repo's build.sh script. Use whenever the user asks to build, compile, install, or rebuild cuML from source, install local cuML changes, or before testing local edits to cuML C++/CUDA/Cython/Python code. --- # Building cuML @@ -17,6 +17,8 @@ conda activate "$(git rev-parse --show-toplevel)/.conda-env" ./build.sh --ccache ``` +If the `.conda-env` doesn't exist yet, set it up first — see [.agents/setup-dev-environment/SKILL.md](../setup-dev-environment/SKILL.md). + This builds and installs `libcuml` (C++), `cuml` (Python), and `prims` (tests) into `$CONDA_PREFIX`. > **Important (sm_121 / new arch + conda RAPIDS libs):** there is a known bug in the default `NATIVE` build path (documented in [rapidsai/cuml#8021](https://github.com/rapidsai/cuml/issues/8021)). Until fixed upstream, use one of these workarounds when mixing with conda-installed RAPIDS shared libraries: @@ -29,12 +31,11 @@ This builds and installs `libcuml` (C++), `cuml` (Python), and `prims` (tests) i > > Bug details: [rapidsai/cuml#8021](https://github.com/rapidsai/cuml/issues/8021). -> **Note on env layout:** This skill uses an **in-worktree prefix env** at `/.conda-env`. Activation is unambiguous (`conda activate ./.conda-env` always points at the right env), and the env is tied to the worktree — deleting the worktree removes the env. See [In-worktree prefix env](#in-worktree-prefix-env) for details. +> **Note on env layout:** This skill uses an **in-worktree prefix env** at `/.conda-env`. Activation is unambiguous (`conda activate ./.conda-env` always points at the right env), and the env is tied to the worktree — deleting the worktree removes the env. See [setup-dev-environment skill](../setup-dev-environment/SKILL.md) for details. ## When to apply this skill - The user asks to build / compile / install / rebuild cuML. -- The user wants to set up a cuML development environment. - The agent has edited C++/CUDA/Cython/Python code in this repo and needs to rebuild before testing. - A test run fails with `ImportError`, missing `libcuml.so`, stale `.so`, or "module not found" symptoms after a code change — the local install is likely stale and needs a rebuild. @@ -42,181 +43,12 @@ This builds and installs `libcuml` (C++), `cuml` (Python), and `prims` (tests) i cuML development happens inside a conda environment that contains all build and runtime dependencies. **Always activate it before invoking `build.sh`, `cmake`, `pip`, or `pytest`.** -### In-worktree prefix env - -Use **one conda env per cuML worktree/clone**, stored at `/.conda-env`. Properties: - -- **Deterministic activation**: `conda activate "$(git rev-parse --show-toplevel)/.conda-env"` always activates the env that belongs to the current worktree. No name-derivation logic, no collision handling. -- **Worktree-bound**: deleting the worktree (`rm -rf` or `git worktree remove`) removes the env. No orphan envs accumulating in `~/miniforge3/envs/`. -- **Parallel-agent safe**: two agents in two worktrees can never accidentally activate each other's env. - -The `.conda-env/` directory is large (~5–10 GB; conda hardlinks packages from its global cache, so real disk cost is much smaller). IDEs/editors must be told not to index it — see [Exclude the env from git and the editor](#step-3-exclude-the-env-from-git-and-the-editor). Some tooling (e.g. `conda env list`) won't show prefix envs unless they're active. - -### Initialize conda in a fresh shell - -Conda activation requires conda to be initialized in the shell first: - ```bash source "$HOME/miniforge3/etc/profile.d/conda.sh" -``` - -(Use the path to the user's conda install — `miniforge3`, `mambaforge`, or `miniconda3`.) - -### Activate the worktree's dev environment - -```bash conda activate "$(git rev-parse --show-toplevel)/.conda-env" ``` -If activation fails with "EnvironmentLocationNotFound" or similar, the env hasn't been created yet — see [Create a fresh dev environment](#create-a-fresh-dev-environment) below. - -Verify the env is the right one: - -```bash -echo "$CONDA_PREFIX" # should be /.conda-env -which python # should resolve to inside $CONDA_PREFIX -python -c "import cuml; print(cuml.__file__)" # should be inside this worktree, not another clone -``` - -### Create a fresh dev environment - -#### Step 1: Pick the right env file - -Env files are named `all_cuda-_arch-.yaml`. Two axes: - -```bash -ls conda/environments/all_*.yaml -# all_cuda-129_arch-aarch64.yaml -# all_cuda-129_arch-x86_64.yaml -# all_cuda-131_arch-aarch64.yaml -# all_cuda-131_arch-x86_64.yaml -``` - -**Architecture (`arch-`)** — must match the host CPU architecture. Always use: - -```bash -uname -m # → x86_64 or aarch64 -``` - -**CUDA version (`cuda-`)** — this is the version of the CUDA toolkit and CUDA runtime that conda will install into the env (cuML does not require a system CUDA install). Choose based on the host's NVIDIA driver and GPU compute capability: - -| Env file | Conda CUDA | Min host driver | Min GPU compute capability | -| --- | --- | --- | --- | -| `cuda-131` (recommended default) | 13.1 | R580+ | 7.5 (Turing or newer) | -| `cuda-129` | 12.9 | R525+ | 7.0 (Volta or newer) | - -**Decision rule:** - -1. **Default to `cuda-131`** unless one of the conditions below applies. -2. Use `cuda-129` if the host has a **Volta (sm_70) GPU** (e.g. V100) — CUDA 13 dropped Volta support. -3. Use `cuda-129` if the host's NVIDIA driver is older than R580 — `nvidia-smi` will show the max CUDA version supported. - -**Detect the right file automatically:** - -```bash -ARCH=$(uname -m) - -# Read GPU compute capability (e.g. "7.0", "7.5", "8.0", "9.0") -CC=$(nvidia-smi --query-gpu=compute_cap --format=csv,noheader,nounits | head -n1) - -# Read the max CUDA version the installed driver supports (e.g. "13.0", "12.9") -DRIVER_CUDA=$(nvidia-smi --query-gpu=driver_version --format=csv,noheader | head -n1) -echo "Driver: $DRIVER_CUDA GPU compute cap: $CC Arch: $ARCH" - -# Use cuda-129 if any Volta GPU is in the system; otherwise prefer cuda-131 -if awk "BEGIN {exit !($CC < 7.5)}"; then - CUDA_TAG=129 -else - CUDA_TAG=131 -fi -ENV_FILE="conda/environments/all_cuda-${CUDA_TAG}_arch-${ARCH}.yaml" -echo "Using $ENV_FILE" -``` - -If unsure, on multi-GPU hosts query each GPU and pick the lowest compute capability. If the host has no GPU (CPU-only build), `cuda-131` is fine. - -#### Step 2: Configure exclusions (do this BEFORE creating the env) - -> **Critical ordering:** add the git and editor exclusions *before* running `conda create`. Otherwise git will momentarily see ~20k+ untracked files inside `.conda-env/`, the VS Code/Cursor Git extension will warn `"too many active changes, only a subset of Git features will be enabled"`, and editor indexing/file-watching will spike CPU before the exclusions kick in. - -The `.conda-env/` directory must never be committed and must be excluded from editor indexing/search. Two git-side options (pick one): - -**Option A — worktree-local exclude (recommended for agents):** doesn't modify any tracked file. - -```bash -grep -qxF '/.conda-env/' .git/info/exclude || echo '/.conda-env/' >> .git/info/exclude -``` - -**Option B — repo-wide `.gitignore`:** if the convention is repo-wide, add `/.conda-env/` to `.gitignore` and commit it. - -For VS Code / Cursor, also write `.vscode/settings.json` (worktree-local, not committed) to keep the editor responsive **and** point the Python extension at the env so terminals, debug, tests, and IntelliSense all use it automatically. Target settings: - -```json -{ - "python.defaultInterpreterPath": "${workspaceFolder}/.conda-env/bin/python", - "python.terminal.activateEnvironment": true, - "files.watcherExclude": { "**/.conda-env/**": true }, - "files.exclude": { "**/.conda-env": true }, - "search.exclude": { "**/.conda-env": true } -} -``` - -Run the script below from the worktree root to merge these into any existing `.vscode/settings.json` without clobbering it (uses the system `python3`, no conda env required): - -```bash -mkdir -p .vscode -python3 - <<'PY' -import json, pathlib -p = pathlib.Path(".vscode/settings.json") -data = json.loads(p.read_text()) if p.exists() else {} -data["python.defaultInterpreterPath"] = "${workspaceFolder}/.conda-env/bin/python" -data["python.terminal.activateEnvironment"] = True -data.setdefault("files.watcherExclude", {})["**/.conda-env/**"] = True -data.setdefault("files.exclude", {})["**/.conda-env"] = True -data.setdefault("search.exclude", {})["**/.conda-env"] = True -p.write_text(json.dumps(data, indent=2) + "\n") -PY -``` - -Notes: - -- `python.defaultInterpreterPath` only takes effect on first workspace open or after running `Python: Clear Workspace Interpreter Setting`. If the workspace was already open with a different interpreter, run that command (or pick the new interpreter via `Python: Select Interpreter`) once to switch. -- `python.terminal.activateEnvironment` is `true` by default; listing it makes the intent explicit and survives users who toggled it off globally. - -#### Step 3: Create and populate the env - -Match the Python version to what the YAML supports (currently `>=3.11,<=3.14`). Create the env at `/.conda-env`: - -```bash -PREFIX_ENV="$(git rev-parse --show-toplevel)/.conda-env" -conda create -y --prefix "$PREFIX_ENV" python=3.14 -conda env update --prefix "$PREFIX_ENV" --file="$ENV_FILE" -conda activate "$PREFIX_ENV" -``` - -After populating, sanity-check that git doesn't see env files: - -```bash -git status -s | wc -l # should be 0 (or just your real changes); not thousands -``` - -### Update an existing dev environment - -After pulling new commits that change `dependencies.yaml` or the env files, refresh the env using **the same env file the env was originally created from** (don't switch CUDA versions on an existing env — recreate instead): - -```bash -PREFIX_ENV="$(git rev-parse --show-toplevel)/.conda-env" -conda env update --prefix "$PREFIX_ENV" --file="$ENV_FILE" -``` - -### Removing an env - -```bash -conda deactivate 2>/dev/null || true -rm -rf "$(git rev-parse --show-toplevel)/.conda-env" -``` - -(Or `conda env remove --prefix "$(git rev-parse --show-toplevel)/.conda-env"`. The `rm -rf` is faster and works even if conda isn't initialized.) +If activation fails with "EnvironmentLocationNotFound", the env doesn't exist yet. To create, update, or remove the env, see [.agents/setup-dev-environment/SKILL.md](../setup-dev-environment/SKILL.md). ## 2. Build with `build.sh` @@ -329,7 +161,7 @@ The `cuml.__file__` path should be inside the active conda env (or the editable - **`ImportError` after pulling new commits**: rebuild — the C++ ABI or Cython-generated code likely changed. - **`libcuml.so` not found at runtime**: `INSTALL_PREFIX` mismatched the active env. Re-run `build.sh` with the correct env activated. -- **`cuml.__file__` points to a different worktree than the one you're editing**: another worktree's env is active. Run `conda activate "$(git rev-parse --show-toplevel)/.conda-env"` and rebuild — see [In-worktree prefix env](#in-worktree-prefix-env). +- **`cuml.__file__` points to a different worktree than the one you're editing**: another worktree's env is active. Run `conda activate "$(git rev-parse --show-toplevel)/.conda-env"` and rebuild — see [setup-dev-environment skill](../setup-dev-environment/SKILL.md). - **Out-of-memory or thermal throttling during build**: lower `PARALLEL_LEVEL` (e.g. `PARALLEL_LEVEL=8`). - **Stale build state after a failed build**: run `./build.sh clean` then rebuild from scratch. - **Building without a GPU**: the build itself works on CPU-only hosts; only running cuML at test time requires a GPU. @@ -355,6 +187,7 @@ python -m pip install --no-build-isolation --no-deps \ ## Additional resources +- Dev environment setup (create, update, remove): [.agents/setup-dev-environment/SKILL.md](../setup-dev-environment/SKILL.md) - Full build docs and all cmake flags: [BUILD.md](../../BUILD.md) - Contributing workflow (pre-commit, clang-tidy, branch naming): [CONTRIBUTING.md](../../CONTRIBUTING.md) - Conda environment files: `conda/environments/all_cuda-*_arch-*.yaml` diff --git a/.agents/setup-dev-environment/SKILL.md b/.agents/setup-dev-environment/SKILL.md new file mode 100644 index 0000000000..76042e9d8e --- /dev/null +++ b/.agents/setup-dev-environment/SKILL.md @@ -0,0 +1,190 @@ +--- +name: setup-dev-environment +description: Set up, create, update, or recreate a cuML conda dev environment in a worktree. Use whenever the user asks to set up a cuML development environment, create or recreate the conda env, configure editor/git exclusions for the env, update the env after pulling new commits, or remove the env. +--- + +# Setting Up a cuML Dev Environment + +This skill covers creating, activating, updating, and removing the conda development environment for a cuML worktree. The canonical reference is [BUILD.md](../../BUILD.md); this skill captures the high-frequency workflows. + +## Overview: in-worktree prefix env + +Use **one conda env per cuML worktree/clone**, stored at `/.conda-env`. Properties: + +- **Deterministic activation**: `conda activate "$(git rev-parse --show-toplevel)/.conda-env"` always activates the env that belongs to the current worktree. No name-derivation logic, no collision handling. +- **Worktree-bound**: deleting the worktree (`rm -rf` or `git worktree remove`) removes the env. No orphan envs accumulating in `~/miniforge3/envs/`. +- **Parallel-agent safe**: two agents in two worktrees can never accidentally activate each other's env. + +The `.conda-env/` directory is large (~5–10 GB; conda hardlinks packages from its global cache, so real disk cost is much smaller). + +## Initialize conda in a fresh shell + +Conda activation requires conda to be initialized in the shell first: + +```bash +source "$HOME/miniforge3/etc/profile.d/conda.sh" +``` + +(Use the path to the user's conda install — `miniforge3`, `mambaforge`, or `miniconda3`.) + +## Activate the worktree's dev environment + +```bash +conda activate "$(git rev-parse --show-toplevel)/.conda-env" +``` + +If activation fails with "EnvironmentLocationNotFound" or similar, the env hasn't been created yet — see [Create a fresh dev environment](#create-a-fresh-dev-environment) below. + +Verify the env is the right one: + +```bash +echo "$CONDA_PREFIX" # should be /.conda-env +which python # should resolve to inside $CONDA_PREFIX +python -c "import cuml; print(cuml.__file__)" # should be inside this worktree, not another clone +``` + +## Create a fresh dev environment + +### Step 1: Pick the right env file + +Env files are named `all_cuda-_arch-.yaml`. Two axes: + +```bash +ls conda/environments/all_*.yaml +# all_cuda-129_arch-aarch64.yaml +# all_cuda-129_arch-x86_64.yaml +# all_cuda-131_arch-aarch64.yaml +# all_cuda-131_arch-x86_64.yaml +``` + +**Architecture (`arch-`)** — must match the host CPU architecture. Always use: + +```bash +uname -m # → x86_64 or aarch64 +``` + +**CUDA version (`cuda-`)** — this is the version of the CUDA toolkit and CUDA runtime that conda will install into the env (cuML does not require a system CUDA install). Choose based on the host's NVIDIA driver and GPU compute capability: + +| Env file | Conda CUDA | Min host driver | Min GPU compute capability | +| --- | --- | --- | --- | +| `cuda-131` (recommended default) | 13.1 | R580+ | 7.5 (Turing or newer) | +| `cuda-129` | 12.9 | R525+ | 7.0 (Volta or newer) | + +**Decision rule:** + +1. **Default to `cuda-131`** unless one of the conditions below applies. +2. Use `cuda-129` if the host has a **Volta (sm_70) GPU** (e.g. V100) — CUDA 13 dropped Volta support. +3. Use `cuda-129` if the host's NVIDIA driver is older than R580 — `nvidia-smi` will show the max CUDA version supported. + +**Detect the right file automatically:** + +```bash +ARCH=$(uname -m) + +# Read GPU compute capability (e.g. "7.0", "7.5", "8.0", "9.0") +CC=$(nvidia-smi --query-gpu=compute_cap --format=csv,noheader,nounits | head -n1) + +# Read the max CUDA version the installed driver supports (e.g. "13.0", "12.9") +DRIVER_CUDA=$(nvidia-smi --query-gpu=driver_version --format=csv,noheader | head -n1) +echo "Driver: $DRIVER_CUDA GPU compute cap: $CC Arch: $ARCH" + +# Use cuda-129 if any Volta GPU is in the system; otherwise prefer cuda-131 +if awk "BEGIN {exit !($CC < 7.5)}"; then + CUDA_TAG=129 +else + CUDA_TAG=131 +fi +ENV_FILE="conda/environments/all_cuda-${CUDA_TAG}_arch-${ARCH}.yaml" +echo "Using $ENV_FILE" +``` + +If unsure, on multi-GPU hosts query each GPU and pick the lowest compute capability. If the host has no GPU (CPU-only build), `cuda-131` is fine. + +### Step 2: Configure exclusions (do this BEFORE creating the env) + +> **Critical ordering:** add the git and editor exclusions *before* running `conda create`. Otherwise git will momentarily see ~20k+ untracked files inside `.conda-env/`, the VS Code/Cursor Git extension will warn `"too many active changes, only a subset of Git features will be enabled"`, and editor indexing/file-watching will spike CPU before the exclusions kick in. + +The `.conda-env/` directory must never be committed and must be excluded from editor indexing/search. Two git-side options (pick one): + +**Option A — worktree-local exclude (recommended for agents):** doesn't modify any tracked file. + +```bash +grep -qxF '/.conda-env/' .git/info/exclude || echo '/.conda-env/' >> .git/info/exclude +``` + +**Option B — repo-wide `.gitignore`:** if the convention is repo-wide, add `/.conda-env/` to `.gitignore` and commit it. + +For VS Code / Cursor, also write `.vscode/settings.json` (worktree-local, not committed) to keep the editor responsive **and** point the Python extension at the env so terminals, debug, tests, and IntelliSense all use it automatically. Target settings: + +```json +{ + "python.defaultInterpreterPath": "${workspaceFolder}/.conda-env/bin/python", + "python.terminal.activateEnvironment": true, + "files.watcherExclude": { "**/.conda-env/**": true }, + "files.exclude": { "**/.conda-env": true }, + "search.exclude": { "**/.conda-env": true } +} +``` + +Run the script below from the worktree root to merge these into any existing `.vscode/settings.json` without clobbering it (uses the system `python3`, no conda env required): + +```bash +mkdir -p .vscode +python3 - <<'PY' +import json, pathlib +p = pathlib.Path(".vscode/settings.json") +data = json.loads(p.read_text()) if p.exists() else {} +data["python.defaultInterpreterPath"] = "${workspaceFolder}/.conda-env/bin/python" +data["python.terminal.activateEnvironment"] = True +data.setdefault("files.watcherExclude", {})["**/.conda-env/**"] = True +data.setdefault("files.exclude", {})["**/.conda-env"] = True +data.setdefault("search.exclude", {})["**/.conda-env"] = True +p.write_text(json.dumps(data, indent=2) + "\n") +PY +``` + +Notes: + +- `python.defaultInterpreterPath` only takes effect on first workspace open or after running `Python: Clear Workspace Interpreter Setting`. If the workspace was already open with a different interpreter, run that command (or pick the new interpreter via `Python: Select Interpreter`) once to switch. +- `python.terminal.activateEnvironment` is `true` by default; listing it makes the intent explicit and survives users who toggled it off globally. + +### Step 3: Create and populate the env + +Match the Python version to what the YAML supports (currently `>=3.11,<=3.14`). Create the env at `/.conda-env`: + +```bash +PREFIX_ENV="$(git rev-parse --show-toplevel)/.conda-env" +conda create -y --prefix "$PREFIX_ENV" python=3.14 +conda env update --prefix "$PREFIX_ENV" --file="$ENV_FILE" +conda activate "$PREFIX_ENV" +``` + +After populating, sanity-check that git doesn't see env files: + +```bash +git status -s | wc -l # should be 0 (or just your real changes); not thousands +``` + +## Update an existing dev environment + +After pulling new commits that change `dependencies.yaml` or the env files, refresh the env using **the same env file the env was originally created from** (don't switch CUDA versions on an existing env — recreate instead): + +```bash +PREFIX_ENV="$(git rev-parse --show-toplevel)/.conda-env" +conda env update --prefix "$PREFIX_ENV" --file="$ENV_FILE" +``` + +## Remove an env + +```bash +conda deactivate 2>/dev/null || true +rm -rf "$(git rev-parse --show-toplevel)/.conda-env" +``` + +(Or `conda env remove --prefix "$(git rev-parse --show-toplevel)/.conda-env"`. The `rm -rf` is faster and works even if conda isn't initialized.) + +## Additional resources + +- Full build docs: [BUILD.md](../../BUILD.md) +- Conda environment files: `conda/environments/all_cuda-*_arch-*.yaml` +- Build skill (for building after the env is ready): [.agents/build-cuml/SKILL.md](../build-cuml/SKILL.md) diff --git a/.agents/test-cuml/SKILL.md b/.agents/test-cuml/SKILL.md index 0800de00ff..16466bfda3 100644 --- a/.agents/test-cuml/SKILL.md +++ b/.agents/test-cuml/SKILL.md @@ -17,6 +17,7 @@ conda activate "$(git rev-parse --show-toplevel)/.conda-env" ``` - After editing C++/CUDA/Cython code, rebuild before testing. See [.agents/build-cuml/SKILL.md](../build-cuml/SKILL.md). +- If activation fails with "EnvironmentLocationNotFound", the env doesn't exist yet — see [.agents/setup-dev-environment/SKILL.md](../setup-dev-environment/SKILL.md). - If you see `No module named pytest` (or `python` resolves to base Miniforge instead of the worktree env), the dev environment is not active — the worktree env includes `pytest` and the rest of the test stack. - All Python tests require a GPU. Most suites support `pytest-xdist` parallelism (`-n auto` or `--numprocesses=N --dist=worksteal`). - CI entry scripts such as `ci/run_cuml_singlegpu_pytests.sh` call `python` on your `PATH`; they assume an appropriate environment is already activated @@ -226,7 +227,7 @@ CI script: [`ci/test_python_integration.sh`](../../ci/test_python_integration.sh ## 7. Common gotchas -- **`ImportError` or wrong `cuml.__file__`**: another worktree's env is active. Run `conda activate "$(git rev-parse --show-toplevel)/.conda-env"` and rebuild. See the [build skill](../build-cuml/SKILL.md). +- **`ImportError` or wrong `cuml.__file__`**: another worktree's env is active. Run `conda activate "$(git rev-parse --show-toplevel)/.conda-env"` and rebuild. See the [build skill](../build-cuml/SKILL.md) and [setup-dev-environment skill](../setup-dev-environment/SKILL.md). - **C++ test binaries not found**: the install dir `$CONDA_PREFIX/bin/gtests/libcuml/` is absent. Run `./build.sh` (default builds and installs the tests). - **Dask import error in single-GPU env**: the single-GPU CI script (`test_python_singlegpu.sh`) intentionally fails if `dask` is installed. Use a dask-capable env for §3 tests. - **`UnmatchedXfailTests` in upstream sklearn tests**: (1) the xfail list references a test id that no longer exists in the installed library version — triage and update the list (see the upstream [README](../../python/cuml/cuml_accel_tests/upstream/README.md)). (2) You passed `--xfail-list` while collecting only a **subset** of the suite (e.g. `--pyargs sklearn.neighbors.tests.test_kde`); use [`run-tests.sh`](../../python/cuml/cuml_accel_tests/upstream/scikit-learn/run-tests.sh) with `-k` instead, or drop `--xfail-list` for a quick narrow run. @@ -236,6 +237,7 @@ CI script: [`ci/test_python_integration.sh`](../../ci/test_python_integration.sh ## 8. Additional resources +- Dev environment setup: [.agents/setup-dev-environment/SKILL.md](../setup-dev-environment/SKILL.md) - Build skill: [.agents/build-cuml/SKILL.md](../build-cuml/SKILL.md) - Full build doc and manual cmake/test paths: [BUILD.md](../../BUILD.md) - Upstream test workflow and xfail management: [python/cuml/cuml_accel_tests/upstream/README.md](../../python/cuml/cuml_accel_tests/upstream/README.md) From e82330fa0c9f3f6957372ebf4924141f3584b931 Mon Sep 17 00:00:00 2001 From: Simon Adorf Date: Thu, 30 Apr 2026 20:25:49 +0000 Subject: [PATCH 07/10] Demote conda env convention to suggestion --- .agents/build-cuml/SKILL.md | 19 ++++----------- .../in-worktree-prefix-env}/SKILL.md | 24 +++++++------------ .agents/test-cuml/SKILL.md | 16 ++++--------- AGENTS.md | 2 +- 4 files changed, 19 insertions(+), 42 deletions(-) rename .agents/{setup-dev-environment => examples/in-worktree-prefix-env}/SKILL.md (83%) diff --git a/.agents/build-cuml/SKILL.md b/.agents/build-cuml/SKILL.md index f0513f9000..b25cec8b16 100644 --- a/.agents/build-cuml/SKILL.md +++ b/.agents/build-cuml/SKILL.md @@ -9,16 +9,12 @@ This skill teaches the agent how to build cuML from source in this repository. T ## Quick start (TL;DR) -For an already-set-up dev env, build and install everything for the local GPU arch: +With a cuML dev env already active, build and install everything for the local GPU arch: ```bash -source "$HOME/miniforge3/etc/profile.d/conda.sh" -conda activate "$(git rev-parse --show-toplevel)/.conda-env" ./build.sh --ccache ``` -If the `.conda-env` doesn't exist yet, set it up first — see [.agents/setup-dev-environment/SKILL.md](../setup-dev-environment/SKILL.md). - This builds and installs `libcuml` (C++), `cuml` (Python), and `prims` (tests) into `$CONDA_PREFIX`. > **Important (sm_121 / new arch + conda RAPIDS libs):** there is a known bug in the default `NATIVE` build path (documented in [rapidsai/cuml#8021](https://github.com/rapidsai/cuml/issues/8021)). Until fixed upstream, use one of these workarounds when mixing with conda-installed RAPIDS shared libraries: @@ -31,8 +27,6 @@ This builds and installs `libcuml` (C++), `cuml` (Python), and `prims` (tests) i > > Bug details: [rapidsai/cuml#8021](https://github.com/rapidsai/cuml/issues/8021). -> **Note on env layout:** This skill uses an **in-worktree prefix env** at `/.conda-env`. Activation is unambiguous (`conda activate ./.conda-env` always points at the right env), and the env is tied to the worktree — deleting the worktree removes the env. See [setup-dev-environment skill](../setup-dev-environment/SKILL.md) for details. - ## When to apply this skill - The user asks to build / compile / install / rebuild cuML. @@ -43,12 +37,7 @@ This builds and installs `libcuml` (C++), `cuml` (Python), and `prims` (tests) i cuML development happens inside a conda environment that contains all build and runtime dependencies. **Always activate it before invoking `build.sh`, `cmake`, `pip`, or `pytest`.** -```bash -source "$HOME/miniforge3/etc/profile.d/conda.sh" -conda activate "$(git rev-parse --show-toplevel)/.conda-env" -``` - -If activation fails with "EnvironmentLocationNotFound", the env doesn't exist yet. To create, update, or remove the env, see [.agents/setup-dev-environment/SKILL.md](../setup-dev-environment/SKILL.md). +This skill does not manage conda env creation or activation — env naming and layout vary by developer. Activate whichever cuML dev env you use before proceeding. If you don't have one yet and want an example workflow, see [.agents/examples/in-worktree-prefix-env/SKILL.md](../examples/in-worktree-prefix-env/SKILL.md). ## 2. Build with `build.sh` @@ -161,7 +150,7 @@ The `cuml.__file__` path should be inside the active conda env (or the editable - **`ImportError` after pulling new commits**: rebuild — the C++ ABI or Cython-generated code likely changed. - **`libcuml.so` not found at runtime**: `INSTALL_PREFIX` mismatched the active env. Re-run `build.sh` with the correct env activated. -- **`cuml.__file__` points to a different worktree than the one you're editing**: another worktree's env is active. Run `conda activate "$(git rev-parse --show-toplevel)/.conda-env"` and rebuild — see [setup-dev-environment skill](../setup-dev-environment/SKILL.md). +- **`cuml.__file__` points to a different worktree than the one you're editing**: the wrong env is active. Activate the env that belongs to the worktree you're editing and rebuild. - **Out-of-memory or thermal throttling during build**: lower `PARALLEL_LEVEL` (e.g. `PARALLEL_LEVEL=8`). - **Stale build state after a failed build**: run `./build.sh clean` then rebuild from scratch. - **Building without a GPU**: the build itself works on CPU-only hosts; only running cuML at test time requires a GPU. @@ -187,7 +176,7 @@ python -m pip install --no-build-isolation --no-deps \ ## Additional resources -- Dev environment setup (create, update, remove): [.agents/setup-dev-environment/SKILL.md](../setup-dev-environment/SKILL.md) +- Example dev environment setup (one optional workflow): [.agents/examples/in-worktree-prefix-env/SKILL.md](../examples/in-worktree-prefix-env/SKILL.md) - Full build docs and all cmake flags: [BUILD.md](../../BUILD.md) - Contributing workflow (pre-commit, clang-tidy, branch naming): [CONTRIBUTING.md](../../CONTRIBUTING.md) - Conda environment files: `conda/environments/all_cuda-*_arch-*.yaml` diff --git a/.agents/setup-dev-environment/SKILL.md b/.agents/examples/in-worktree-prefix-env/SKILL.md similarity index 83% rename from .agents/setup-dev-environment/SKILL.md rename to .agents/examples/in-worktree-prefix-env/SKILL.md index 76042e9d8e..a044e318e7 100644 --- a/.agents/setup-dev-environment/SKILL.md +++ b/.agents/examples/in-worktree-prefix-env/SKILL.md @@ -1,21 +1,15 @@ --- -name: setup-dev-environment -description: Set up, create, update, or recreate a cuML conda dev environment in a worktree. Use whenever the user asks to set up a cuML development environment, create or recreate the conda env, configure editor/git exclusions for the env, update the env after pulling new commits, or remove the env. +name: in-worktree-prefix-env +description: Example workflow for creating a cuML conda dev environment stored inside the worktree at /.conda-env. This is ONE optional convention — many cuML developers use their own naming/layout. Only apply this skill when the user explicitly asks to create or set up a dev environment using the in-worktree prefix layout, or when no personal convention has been configured. --- -# Setting Up a cuML Dev Environment +# Example: In-Worktree Prefix Dev Environment -This skill covers creating, activating, updating, and removing the conda development environment for a cuML worktree. The canonical reference is [BUILD.md](../../BUILD.md); this skill captures the high-frequency workflows. +> **This is one example workflow.** cuML developers use many different conda environment conventions — date-stamped envs (`cuml-YYYYMMDD`), named envs (`cuml-dev`), custom prefixes, etc. **Only follow these steps if the user has explicitly asked for this layout, or if no personal convention is documented in your user-level config.** +> +> If you have personal setup instructions (e.g. in `~/.cursor/rules/`, `~/.claude/CLAUDE.md`, or similar), those take precedence over this file. -## Overview: in-worktree prefix env - -Use **one conda env per cuML worktree/clone**, stored at `/.conda-env`. Properties: - -- **Deterministic activation**: `conda activate "$(git rev-parse --show-toplevel)/.conda-env"` always activates the env that belongs to the current worktree. No name-derivation logic, no collision handling. -- **Worktree-bound**: deleting the worktree (`rm -rf` or `git worktree remove`) removes the env. No orphan envs accumulating in `~/miniforge3/envs/`. -- **Parallel-agent safe**: two agents in two worktrees can never accidentally activate each other's env. - -The `.conda-env/` directory is large (~5–10 GB; conda hardlinks packages from its global cache, so real disk cost is much smaller). +This workflow stores the conda env at `/.conda-env` — one env per worktree, tied to the worktree lifetime. ## Initialize conda in a fresh shell @@ -185,6 +179,6 @@ rm -rf "$(git rev-parse --show-toplevel)/.conda-env" ## Additional resources -- Full build docs: [BUILD.md](../../BUILD.md) +- Full build docs: [BUILD.md](../../../BUILD.md) - Conda environment files: `conda/environments/all_cuda-*_arch-*.yaml` -- Build skill (for building after the env is ready): [.agents/build-cuml/SKILL.md](../build-cuml/SKILL.md) +- Build skill (for building after the env is ready): [.agents/build-cuml/SKILL.md](../../build-cuml/SKILL.md) diff --git a/.agents/test-cuml/SKILL.md b/.agents/test-cuml/SKILL.md index 16466bfda3..fadd5b1d2e 100644 --- a/.agents/test-cuml/SKILL.md +++ b/.agents/test-cuml/SKILL.md @@ -9,18 +9,12 @@ This skill covers all cuML test suites. The canonical source of truth for each s ## 0. Prerequisites -**Activate the worktree env before running any tests:** - -```bash -source "$HOME/miniforge3/etc/profile.d/conda.sh" -conda activate "$(git rev-parse --show-toplevel)/.conda-env" -``` +**A cuML conda dev env must be active before running any tests.** This skill does not manage env creation or activation — env naming and layout vary by developer. Activate whichever cuML dev env you use. If you don't have one yet and want an example setup workflow, see [.agents/examples/in-worktree-prefix-env/SKILL.md](../examples/in-worktree-prefix-env/SKILL.md). - After editing C++/CUDA/Cython code, rebuild before testing. See [.agents/build-cuml/SKILL.md](../build-cuml/SKILL.md). -- If activation fails with "EnvironmentLocationNotFound", the env doesn't exist yet — see [.agents/setup-dev-environment/SKILL.md](../setup-dev-environment/SKILL.md). -- If you see `No module named pytest` (or `python` resolves to base Miniforge instead of the worktree env), the dev environment is not active — the worktree env includes `pytest` and the rest of the test stack. +- If you see `No module named pytest` (or `python` resolves to the base conda install instead of the dev env), the dev environment is not active — the cuML dev env includes `pytest` and the rest of the test stack. - All Python tests require a GPU. Most suites support `pytest-xdist` parallelism (`-n auto` or `--numprocesses=N --dist=worksteal`). -- CI entry scripts such as `ci/run_cuml_singlegpu_pytests.sh` call `python` on your `PATH`; they assume an appropriate environment is already activated +- CI entry scripts such as `ci/run_cuml_singlegpu_pytests.sh` call `python` on your `PATH`; they assume an appropriate environment is already activated. --- @@ -227,7 +221,7 @@ CI script: [`ci/test_python_integration.sh`](../../ci/test_python_integration.sh ## 7. Common gotchas -- **`ImportError` or wrong `cuml.__file__`**: another worktree's env is active. Run `conda activate "$(git rev-parse --show-toplevel)/.conda-env"` and rebuild. See the [build skill](../build-cuml/SKILL.md) and [setup-dev-environment skill](../setup-dev-environment/SKILL.md). +- **`ImportError` or wrong `cuml.__file__`**: the wrong env is active. Activate the env that belongs to the worktree you're editing and rebuild. See the [build skill](../build-cuml/SKILL.md). - **C++ test binaries not found**: the install dir `$CONDA_PREFIX/bin/gtests/libcuml/` is absent. Run `./build.sh` (default builds and installs the tests). - **Dask import error in single-GPU env**: the single-GPU CI script (`test_python_singlegpu.sh`) intentionally fails if `dask` is installed. Use a dask-capable env for §3 tests. - **`UnmatchedXfailTests` in upstream sklearn tests**: (1) the xfail list references a test id that no longer exists in the installed library version — triage and update the list (see the upstream [README](../../python/cuml/cuml_accel_tests/upstream/README.md)). (2) You passed `--xfail-list` while collecting only a **subset** of the suite (e.g. `--pyargs sklearn.neighbors.tests.test_kde`); use [`run-tests.sh`](../../python/cuml/cuml_accel_tests/upstream/scikit-learn/run-tests.sh) with `-k` instead, or drop `--xfail-list` for a quick narrow run. @@ -237,7 +231,7 @@ CI script: [`ci/test_python_integration.sh`](../../ci/test_python_integration.sh ## 8. Additional resources -- Dev environment setup: [.agents/setup-dev-environment/SKILL.md](../setup-dev-environment/SKILL.md) +- Example dev environment setup (one optional workflow): [.agents/examples/in-worktree-prefix-env/SKILL.md](../examples/in-worktree-prefix-env/SKILL.md) - Build skill: [.agents/build-cuml/SKILL.md](../build-cuml/SKILL.md) - Full build doc and manual cmake/test paths: [BUILD.md](../../BUILD.md) - Upstream test workflow and xfail management: [python/cuml/cuml_accel_tests/upstream/README.md](../../python/cuml/cuml_accel_tests/upstream/README.md) diff --git a/AGENTS.md b/AGENTS.md index c4d05e372c..e1612387f7 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -35,7 +35,7 @@ For pre-commit hooks, clang-tidy, branch naming, and the PR process, see [CONTRI Key conventions: - Linter errors are auto-fixed by pre-commit hooks — don't fix them manually unless asked. -- Always activate the conda dev environment before running `build.sh`, `pytest`, or `pip`. +- Always activate your cuML conda dev environment before running `build.sh`, `pytest`, or `pip`. Env naming and layout vary by developer — use whichever env contains the cuML you're working on. ## Code review guidelines From 776b98691913fbc2af5409fb861a2ccaf45493c6 Mon Sep 17 00:00:00 2001 From: Simon Adorf Date: Thu, 30 Apr 2026 21:05:54 +0000 Subject: [PATCH 08/10] Provide better guidance on when to create a conda environment. --- .agents/build-cuml/SKILL.md | 16 +- .../examples/in-worktree-prefix-env/SKILL.md | 184 -------------- .agents/setup-dev-environment/SKILL.md | 236 ++++++++++++++++++ .agents/test-cuml/SKILL.md | 14 +- AGENTS.md | 9 +- 5 files changed, 268 insertions(+), 191 deletions(-) delete mode 100644 .agents/examples/in-worktree-prefix-env/SKILL.md create mode 100644 .agents/setup-dev-environment/SKILL.md diff --git a/.agents/build-cuml/SKILL.md b/.agents/build-cuml/SKILL.md index b25cec8b16..294a3f78da 100644 --- a/.agents/build-cuml/SKILL.md +++ b/.agents/build-cuml/SKILL.md @@ -33,11 +33,20 @@ This builds and installs `libcuml` (C++), `cuml` (Python), and `prims` (tests) i - The agent has edited C++/CUDA/Cython/Python code in this repo and needs to rebuild before testing. - A test run fails with `ImportError`, missing `libcuml.so`, stale `.so`, or "module not found" symptoms after a code change — the local install is likely stale and needs a rebuild. -## 1. Activate the conda dev environment +## 1. Set up the development environment -cuML development happens inside a conda environment that contains all build and runtime dependencies. **Always activate it before invoking `build.sh`, `cmake`, `pip`, or `pytest`.** +Building cuML requires an **active** development environment containing all build and runtime dependencies. Building with the wrong env active (or none) silently installs into — and tests against — the wrong worktree. -This skill does not manage conda env creation or activation — env naming and layout vary by developer. Activate whichever cuML dev env you use before proceeding. If you don't have one yet and want an example workflow, see [.agents/examples/in-worktree-prefix-env/SKILL.md](../examples/in-worktree-prefix-env/SKILL.md). +### Quick check + +```bash +echo "CONDA_PREFIX=${CONDA_PREFIX:-unset} VIRTUAL_ENV=${VIRTUAL_ENV:-unset}" +which python +python -c "import cuml; print(cuml.__file__)" 2>/dev/null \ + || echo "cuml not yet installed (fine before first build)" +``` + +`cuml.__file__` should resolve inside this worktree (`python/cuml/`) or inside the active env's `site-packages`. If it resolves into a different worktree or env, the wrong env is active — see [setup-dev-environment §2](../setup-dev-environment/SKILL.md#2-environment-selection-algorithm-canonical) for the selection algorithm. ## 2. Build with `build.sh` @@ -176,7 +185,6 @@ python -m pip install --no-build-isolation --no-deps \ ## Additional resources -- Example dev environment setup (one optional workflow): [.agents/examples/in-worktree-prefix-env/SKILL.md](../examples/in-worktree-prefix-env/SKILL.md) - Full build docs and all cmake flags: [BUILD.md](../../BUILD.md) - Contributing workflow (pre-commit, clang-tidy, branch naming): [CONTRIBUTING.md](../../CONTRIBUTING.md) - Conda environment files: `conda/environments/all_cuda-*_arch-*.yaml` diff --git a/.agents/examples/in-worktree-prefix-env/SKILL.md b/.agents/examples/in-worktree-prefix-env/SKILL.md deleted file mode 100644 index a044e318e7..0000000000 --- a/.agents/examples/in-worktree-prefix-env/SKILL.md +++ /dev/null @@ -1,184 +0,0 @@ ---- -name: in-worktree-prefix-env -description: Example workflow for creating a cuML conda dev environment stored inside the worktree at /.conda-env. This is ONE optional convention — many cuML developers use their own naming/layout. Only apply this skill when the user explicitly asks to create or set up a dev environment using the in-worktree prefix layout, or when no personal convention has been configured. ---- - -# Example: In-Worktree Prefix Dev Environment - -> **This is one example workflow.** cuML developers use many different conda environment conventions — date-stamped envs (`cuml-YYYYMMDD`), named envs (`cuml-dev`), custom prefixes, etc. **Only follow these steps if the user has explicitly asked for this layout, or if no personal convention is documented in your user-level config.** -> -> If you have personal setup instructions (e.g. in `~/.cursor/rules/`, `~/.claude/CLAUDE.md`, or similar), those take precedence over this file. - -This workflow stores the conda env at `/.conda-env` — one env per worktree, tied to the worktree lifetime. - -## Initialize conda in a fresh shell - -Conda activation requires conda to be initialized in the shell first: - -```bash -source "$HOME/miniforge3/etc/profile.d/conda.sh" -``` - -(Use the path to the user's conda install — `miniforge3`, `mambaforge`, or `miniconda3`.) - -## Activate the worktree's dev environment - -```bash -conda activate "$(git rev-parse --show-toplevel)/.conda-env" -``` - -If activation fails with "EnvironmentLocationNotFound" or similar, the env hasn't been created yet — see [Create a fresh dev environment](#create-a-fresh-dev-environment) below. - -Verify the env is the right one: - -```bash -echo "$CONDA_PREFIX" # should be /.conda-env -which python # should resolve to inside $CONDA_PREFIX -python -c "import cuml; print(cuml.__file__)" # should be inside this worktree, not another clone -``` - -## Create a fresh dev environment - -### Step 1: Pick the right env file - -Env files are named `all_cuda-_arch-.yaml`. Two axes: - -```bash -ls conda/environments/all_*.yaml -# all_cuda-129_arch-aarch64.yaml -# all_cuda-129_arch-x86_64.yaml -# all_cuda-131_arch-aarch64.yaml -# all_cuda-131_arch-x86_64.yaml -``` - -**Architecture (`arch-`)** — must match the host CPU architecture. Always use: - -```bash -uname -m # → x86_64 or aarch64 -``` - -**CUDA version (`cuda-`)** — this is the version of the CUDA toolkit and CUDA runtime that conda will install into the env (cuML does not require a system CUDA install). Choose based on the host's NVIDIA driver and GPU compute capability: - -| Env file | Conda CUDA | Min host driver | Min GPU compute capability | -| --- | --- | --- | --- | -| `cuda-131` (recommended default) | 13.1 | R580+ | 7.5 (Turing or newer) | -| `cuda-129` | 12.9 | R525+ | 7.0 (Volta or newer) | - -**Decision rule:** - -1. **Default to `cuda-131`** unless one of the conditions below applies. -2. Use `cuda-129` if the host has a **Volta (sm_70) GPU** (e.g. V100) — CUDA 13 dropped Volta support. -3. Use `cuda-129` if the host's NVIDIA driver is older than R580 — `nvidia-smi` will show the max CUDA version supported. - -**Detect the right file automatically:** - -```bash -ARCH=$(uname -m) - -# Read GPU compute capability (e.g. "7.0", "7.5", "8.0", "9.0") -CC=$(nvidia-smi --query-gpu=compute_cap --format=csv,noheader,nounits | head -n1) - -# Read the max CUDA version the installed driver supports (e.g. "13.0", "12.9") -DRIVER_CUDA=$(nvidia-smi --query-gpu=driver_version --format=csv,noheader | head -n1) -echo "Driver: $DRIVER_CUDA GPU compute cap: $CC Arch: $ARCH" - -# Use cuda-129 if any Volta GPU is in the system; otherwise prefer cuda-131 -if awk "BEGIN {exit !($CC < 7.5)}"; then - CUDA_TAG=129 -else - CUDA_TAG=131 -fi -ENV_FILE="conda/environments/all_cuda-${CUDA_TAG}_arch-${ARCH}.yaml" -echo "Using $ENV_FILE" -``` - -If unsure, on multi-GPU hosts query each GPU and pick the lowest compute capability. If the host has no GPU (CPU-only build), `cuda-131` is fine. - -### Step 2: Configure exclusions (do this BEFORE creating the env) - -> **Critical ordering:** add the git and editor exclusions *before* running `conda create`. Otherwise git will momentarily see ~20k+ untracked files inside `.conda-env/`, the VS Code/Cursor Git extension will warn `"too many active changes, only a subset of Git features will be enabled"`, and editor indexing/file-watching will spike CPU before the exclusions kick in. - -The `.conda-env/` directory must never be committed and must be excluded from editor indexing/search. Two git-side options (pick one): - -**Option A — worktree-local exclude (recommended for agents):** doesn't modify any tracked file. - -```bash -grep -qxF '/.conda-env/' .git/info/exclude || echo '/.conda-env/' >> .git/info/exclude -``` - -**Option B — repo-wide `.gitignore`:** if the convention is repo-wide, add `/.conda-env/` to `.gitignore` and commit it. - -For VS Code / Cursor, also write `.vscode/settings.json` (worktree-local, not committed) to keep the editor responsive **and** point the Python extension at the env so terminals, debug, tests, and IntelliSense all use it automatically. Target settings: - -```json -{ - "python.defaultInterpreterPath": "${workspaceFolder}/.conda-env/bin/python", - "python.terminal.activateEnvironment": true, - "files.watcherExclude": { "**/.conda-env/**": true }, - "files.exclude": { "**/.conda-env": true }, - "search.exclude": { "**/.conda-env": true } -} -``` - -Run the script below from the worktree root to merge these into any existing `.vscode/settings.json` without clobbering it (uses the system `python3`, no conda env required): - -```bash -mkdir -p .vscode -python3 - <<'PY' -import json, pathlib -p = pathlib.Path(".vscode/settings.json") -data = json.loads(p.read_text()) if p.exists() else {} -data["python.defaultInterpreterPath"] = "${workspaceFolder}/.conda-env/bin/python" -data["python.terminal.activateEnvironment"] = True -data.setdefault("files.watcherExclude", {})["**/.conda-env/**"] = True -data.setdefault("files.exclude", {})["**/.conda-env"] = True -data.setdefault("search.exclude", {})["**/.conda-env"] = True -p.write_text(json.dumps(data, indent=2) + "\n") -PY -``` - -Notes: - -- `python.defaultInterpreterPath` only takes effect on first workspace open or after running `Python: Clear Workspace Interpreter Setting`. If the workspace was already open with a different interpreter, run that command (or pick the new interpreter via `Python: Select Interpreter`) once to switch. -- `python.terminal.activateEnvironment` is `true` by default; listing it makes the intent explicit and survives users who toggled it off globally. - -### Step 3: Create and populate the env - -Match the Python version to what the YAML supports (currently `>=3.11,<=3.14`). Create the env at `/.conda-env`: - -```bash -PREFIX_ENV="$(git rev-parse --show-toplevel)/.conda-env" -conda create -y --prefix "$PREFIX_ENV" python=3.14 -conda env update --prefix "$PREFIX_ENV" --file="$ENV_FILE" -conda activate "$PREFIX_ENV" -``` - -After populating, sanity-check that git doesn't see env files: - -```bash -git status -s | wc -l # should be 0 (or just your real changes); not thousands -``` - -## Update an existing dev environment - -After pulling new commits that change `dependencies.yaml` or the env files, refresh the env using **the same env file the env was originally created from** (don't switch CUDA versions on an existing env — recreate instead): - -```bash -PREFIX_ENV="$(git rev-parse --show-toplevel)/.conda-env" -conda env update --prefix "$PREFIX_ENV" --file="$ENV_FILE" -``` - -## Remove an env - -```bash -conda deactivate 2>/dev/null || true -rm -rf "$(git rev-parse --show-toplevel)/.conda-env" -``` - -(Or `conda env remove --prefix "$(git rev-parse --show-toplevel)/.conda-env"`. The `rm -rf` is faster and works even if conda isn't initialized.) - -## Additional resources - -- Full build docs: [BUILD.md](../../../BUILD.md) -- Conda environment files: `conda/environments/all_cuda-*_arch-*.yaml` -- Build skill (for building after the env is ready): [.agents/build-cuml/SKILL.md](../../build-cuml/SKILL.md) diff --git a/.agents/setup-dev-environment/SKILL.md b/.agents/setup-dev-environment/SKILL.md new file mode 100644 index 0000000000..c61467ce69 --- /dev/null +++ b/.agents/setup-dev-environment/SKILL.md @@ -0,0 +1,236 @@ +--- +name: setup-dev-environment +description: Set up, create, update, or recreate a cuML conda dev environment for a worktree. Use whenever the user asks to set up a cuML development environment, create or recreate the conda env, configure editor/git exclusions for the env, update the env after pulling new commits, or remove the env. Also use when an agent needs to build or test cuML and there is no usable env to activate yet. +--- + +# Setting Up a cuML Dev Environment + +This skill covers selecting, activating, creating, updating, and removing the development environment for a cuML worktree. The canonical reference is [BUILD.md](../../BUILD.md); this skill captures the high-frequency workflows. + +## 1. When this skill applies + +- The agent needs to decide which env to use before building or testing. +- The user asks to set up / create / recreate a cuML dev env. +- The user wants to refresh an env after a pull, or remove an env entirely. + +## 2. Environment selection algorithm (canonical) + +**Before running the algorithm, read every personal rule, skill, or memory that may govern conda or dev-environment selection.** Sources include — but are not limited to — `~/.cursor/rules/`, `~/.cursor/skills/`, `~/.agents/`, `~/.agents/skills/`, `~/.claude/CLAUDE.md`, `~/AGENTS.md`, and any "available skills" / "rules" listing supplied by your agent runtime (e.g. Cursor's `` system block, Claude's `` block). The paths above are illustrative, not exhaustive — also follow whatever the runtime advertises. These rules establish the *selection policy* (e.g. "use the latest `cuml-YYYYMMDD` env") that the algorithm applies; if you skip this prelude you will likely create an unnecessary fresh env when a usable one already exists. + +Work through these steps in order. Stop at the first that matches. + +**Step 1 — Already-active env.** If an env is already active and `cuml` resolves inside this worktree or the active env's `site-packages`, use it and stop: + +```bash +# Quick check — run these three lines +echo "CONDA_PREFIX=${CONDA_PREFIX:-unset} VIRTUAL_ENV=${VIRTUAL_ENV:-unset}" +which python +python -c "import cuml; print(cuml.__file__)" 2>/dev/null \ + || echo "cuml not yet installed (fine before first build)" +``` + +If `cuml.__file__` lives inside this worktree (`python/cuml/`) or inside the active env's `site-packages`, **you're done** — proceed to build or test. + +**Step 2 — Editor-configured interpreter.** If `/.vscode/settings.json` sets `python.defaultInterpreterPath` *and the interpreter exists on disk*, activate that env (or use that Python directly) — this is the user's explicit choice for the worktree. If the configured path doesn't exist (e.g. it points at `${workspaceFolder}/.conda-env/bin/python` but `.conda-env/` was never created), treat this step as inapplicable and fall through to Step 3 — do **not** treat it as a directive to create that env. + +```bash +python3 -c " +import json, pathlib +s = pathlib.Path('.vscode/settings.json') +if s.exists(): + d = json.loads(s.read_text()) + print(d.get('python.defaultInterpreterPath', '(not set)')) +" +``` + +**Step 3 — Apply the personal-rule selection policy.** If the rules you loaded in the prelude define an env selection policy (e.g. "use the `cuml-YYYYMMDD` env with the latest date suffix", or "always use env `foo`"), apply it now and stop on first match. This step **overrides Steps 4–5** (the worktree-prefix default and the fresh-env fallback) — do not create a new env when an env mandated by personal rules already exists. + +*Worked example.* If a personal rule says "use the `cuml-YYYYMMDD` env with the largest date suffix": + +```bash +LATEST=$(conda env list | awk '/cuml-[0-9]{8}( |$)/ {print $1}' | sort | tail -1) +echo "Picking $LATEST" +conda activate "$LATEST" +``` + +Then verify with §3 below. If the env doesn't yet have `cuml` installed, that's fine — proceed to build. + +**Step 4 — Worktree prefix env.** If `/.conda-env` exists, activate it: + +```bash +source "$HOME/miniforge3/etc/profile.d/conda.sh" # or the user's conda init path +conda activate "$(git rev-parse --show-toplevel)/.conda-env" +``` + +**Step 5 — Nothing exists.** No usable env was found. Create one from scratch — see [§4 Create a fresh env](#4-create-a-fresh-env-fallback). + +> **Do NOT** pick an arbitrary env from `conda env list` by guessing. Names like `cuml-work0`, `rapids-26.04`, `nvforest-work0` typically belong to other worktrees / cuML versions, and building into one silently installs a `cuml` whose ABI may not match this worktree's source. The exception is Step 3: when personal rules mandate a specific naming convention (e.g. `cuml-YYYYMMDD`), `conda env list` is the right tool to find the matching env — that's not "guessing", it's applying a documented policy. + +## 3. Sanity check after activation + +Regardless of how the env was selected, verify it's the right one before building or testing: + +```bash +echo "CONDA_PREFIX=${CONDA_PREFIX:-unset} VIRTUAL_ENV=${VIRTUAL_ENV:-unset}" +which python # should resolve inside the env +python -c "import cuml; print(cuml.__file__)" # should be inside this worktree, not another clone +``` + +The env is good when `cuml.__file__` lives inside this worktree (`python/cuml/`) or the active env's `site-packages`. If it points into a different worktree or env, the wrong env is active — revisit step 2 above. + +## 4. Create a fresh env (fallback) + +Use this only when steps 1–4 above found nothing to activate. The **recommended default** is an in-worktree prefix env at `/.conda-env` — one env per worktree, tied to the worktree lifetime. If the user prefers a different location or naming convention, follow their preference. + +Properties of the in-worktree prefix env: + +- **Deterministic activation**: `conda activate "$(git rev-parse --show-toplevel)/.conda-env"` always activates the env that belongs to the current worktree. No name-derivation logic, no collision handling. +- **Worktree-bound**: deleting the worktree (`rm -rf` or `git worktree remove`) removes the env. No orphan envs accumulating in `~/miniforge3/envs/`. +- **Parallel-agent safe**: two agents in two worktrees can never accidentally activate each other's env. + +The `.conda-env/` directory is large (~5–10 GB on disk; conda hardlinks packages from its global cache, so real disk cost is much smaller). + +### Step 1: Pick the right env file + +Env files are named `all_cuda-_arch-.yaml`. Two axes: + +```bash +ls conda/environments/all_*.yaml +# all_cuda-129_arch-aarch64.yaml +# all_cuda-129_arch-x86_64.yaml +# all_cuda-131_arch-aarch64.yaml +# all_cuda-131_arch-x86_64.yaml +``` + +**Architecture (`arch-`)** — must match the host CPU architecture. Always use: + +```bash +uname -m # → x86_64 or aarch64 +``` + +**CUDA version (`cuda-`)** — this is the version of the CUDA toolkit and CUDA runtime that conda will install into the env (cuML does not require a system CUDA install). Choose based on the host's NVIDIA driver and GPU compute capability: + +| Env file | Conda CUDA | Min host driver | Min GPU compute capability | +| --- | --- | --- | --- | +| `cuda-131` (recommended default) | 13.1 | R580+ | 7.5 (Turing or newer) | +| `cuda-129` | 12.9 | R525+ | 7.0 (Volta or newer) | + +**Decision rule:** + +1. **Default to `cuda-131`** unless one of the conditions below applies. +2. Use `cuda-129` if the host has a **Volta (sm_70) GPU** (e.g. V100) — CUDA 13 dropped Volta support. +3. Use `cuda-129` if the host's NVIDIA driver is older than R580 — `nvidia-smi` will show the max CUDA version supported. + +**Detect the right file automatically:** + +```bash +ARCH=$(uname -m) + +# Read GPU compute capability (e.g. "7.0", "7.5", "8.0", "9.0") +CC=$(nvidia-smi --query-gpu=compute_cap --format=csv,noheader,nounits | head -n1) + +# Read the max CUDA version the installed driver supports (e.g. "13.0", "12.9") +DRIVER_CUDA=$(nvidia-smi --query-gpu=driver_version --format=csv,noheader | head -n1) +echo "Driver: $DRIVER_CUDA GPU compute cap: $CC Arch: $ARCH" + +# Use cuda-129 if any Volta GPU is in the system; otherwise prefer cuda-131 +if awk "BEGIN {exit !($CC < 7.5)}"; then + CUDA_TAG=129 +else + CUDA_TAG=131 +fi +ENV_FILE="conda/environments/all_cuda-${CUDA_TAG}_arch-${ARCH}.yaml" +echo "Using $ENV_FILE" +``` + +If unsure, on multi-GPU hosts query each GPU and pick the lowest compute capability. If the host has no GPU (CPU-only build), `cuda-131` is fine. + +### Step 2: Configure exclusions (do this BEFORE creating the env) + +> **Critical ordering:** add the git and editor exclusions *before* running `conda create`. Otherwise git will momentarily see ~20k+ untracked files inside `.conda-env/`, the VS Code/Cursor Git extension will warn `"too many active changes, only a subset of Git features will be enabled"`, and editor indexing/file-watching will spike CPU before the exclusions kick in. + +The `.conda-env/` directory must never be committed and must be excluded from editor indexing/search. Two git-side options (pick one): + +**Option A — worktree-local exclude (recommended for agents):** doesn't modify any tracked file. + +```bash +grep -qxF '/.conda-env/' .git/info/exclude || echo '/.conda-env/' >> .git/info/exclude +``` + +**Option B — repo-wide `.gitignore`:** if the convention is repo-wide, add `/.conda-env/` to `.gitignore` and commit it. + +For VS Code / Cursor, also write `.vscode/settings.json` (worktree-local, not committed) to keep the editor responsive **and** point the Python extension at the env so terminals, debug, tests, and IntelliSense all use it automatically. This `python.defaultInterpreterPath` setting is also what later agent runs check (per §2 step 2) to avoid creating a duplicate env. Target settings: + +```json +{ + "python.defaultInterpreterPath": "${workspaceFolder}/.conda-env/bin/python", + "python.terminal.activateEnvironment": true, + "files.watcherExclude": { "**/.conda-env/**": true }, + "files.exclude": { "**/.conda-env": true }, + "search.exclude": { "**/.conda-env": true } +} +``` + +Run the script below from the worktree root to merge these into any existing `.vscode/settings.json` without clobbering it (uses the system `python3`, no conda env required): + +```bash +mkdir -p .vscode +python3 - <<'PY' +import json, pathlib +p = pathlib.Path(".vscode/settings.json") +data = json.loads(p.read_text()) if p.exists() else {} +data["python.defaultInterpreterPath"] = "${workspaceFolder}/.conda-env/bin/python" +data["python.terminal.activateEnvironment"] = True +data.setdefault("files.watcherExclude", {})["**/.conda-env/**"] = True +data.setdefault("files.exclude", {})["**/.conda-env"] = True +data.setdefault("search.exclude", {})["**/.conda-env"] = True +p.write_text(json.dumps(data, indent=2) + "\n") +PY +``` + +Notes: + +- `python.defaultInterpreterPath` only takes effect on first workspace open or after running `Python: Clear Workspace Interpreter Setting`. If the workspace was already open with a different interpreter, run that command (or pick the new interpreter via `Python: Select Interpreter`) once to switch. +- `python.terminal.activateEnvironment` is `true` by default; listing it makes the intent explicit and survives users who toggled it off globally. + +### Step 3: Create and populate the env + +Match the Python version to what the YAML supports (currently `>=3.11,<=3.14`). Create the env at `/.conda-env`: + +```bash +PREFIX_ENV="$(git rev-parse --show-toplevel)/.conda-env" +conda create -y --prefix "$PREFIX_ENV" python=3.14 +conda env update --prefix "$PREFIX_ENV" --file="$ENV_FILE" +conda activate "$PREFIX_ENV" +``` + +After populating, sanity-check that git doesn't see env files: + +```bash +git status -s | wc -l # should be 0 (or just your real changes); not thousands +``` + +## 5. Update an existing dev environment + +After pulling new commits that change `dependencies.yaml` or the env files, refresh the env using **the same env file the env was originally created from** (don't switch CUDA versions on an existing env — recreate instead): + +```bash +PREFIX_ENV="$(git rev-parse --show-toplevel)/.conda-env" +conda env update --prefix "$PREFIX_ENV" --file="$ENV_FILE" +``` + +## 6. Remove an env + +```bash +conda deactivate 2>/dev/null || true +rm -rf "$(git rev-parse --show-toplevel)/.conda-env" +``` + +(Or `conda env remove --prefix "$(git rev-parse --show-toplevel)/.conda-env"`. The `rm -rf` is faster and works even if conda isn't initialized.) + +## Additional resources + +- Full build docs: [BUILD.md](../../BUILD.md) +- Conda environment files: `conda/environments/all_cuda-*_arch-*.yaml` +- Build skill (for building after the env is ready): [.agents/build-cuml/SKILL.md](../build-cuml/SKILL.md) +- Test skill: [.agents/test-cuml/SKILL.md](../test-cuml/SKILL.md) diff --git a/.agents/test-cuml/SKILL.md b/.agents/test-cuml/SKILL.md index fadd5b1d2e..ac428c3174 100644 --- a/.agents/test-cuml/SKILL.md +++ b/.agents/test-cuml/SKILL.md @@ -9,7 +9,18 @@ This skill covers all cuML test suites. The canonical source of truth for each s ## 0. Prerequisites -**A cuML conda dev env must be active before running any tests.** This skill does not manage env creation or activation — env naming and layout vary by developer. Activate whichever cuML dev env you use. If you don't have one yet and want an example setup workflow, see [.agents/examples/in-worktree-prefix-env/SKILL.md](../examples/in-worktree-prefix-env/SKILL.md). +**A cuML dev env must be active before running any tests.** If you don't have one active yet, follow the [setup-dev-environment selection algorithm](../setup-dev-environment/SKILL.md#2-environment-selection-algorithm-canonical). + +Quick sanity check before invoking any test command: + +```bash +echo "CONDA_PREFIX=${CONDA_PREFIX:-unset} VIRTUAL_ENV=${VIRTUAL_ENV:-unset}" +which python +python -c "import cuml; print(cuml.__file__)" 2>/dev/null \ + || echo "cuml not yet installed — rebuild first" +``` + +Additional prerequisites: - After editing C++/CUDA/Cython code, rebuild before testing. See [.agents/build-cuml/SKILL.md](../build-cuml/SKILL.md). - If you see `No module named pytest` (or `python` resolves to the base conda install instead of the dev env), the dev environment is not active — the cuML dev env includes `pytest` and the rest of the test stack. @@ -231,7 +242,6 @@ CI script: [`ci/test_python_integration.sh`](../../ci/test_python_integration.sh ## 8. Additional resources -- Example dev environment setup (one optional workflow): [.agents/examples/in-worktree-prefix-env/SKILL.md](../examples/in-worktree-prefix-env/SKILL.md) - Build skill: [.agents/build-cuml/SKILL.md](../build-cuml/SKILL.md) - Full build doc and manual cmake/test paths: [BUILD.md](../../BUILD.md) - Upstream test workflow and xfail management: [python/cuml/cuml_accel_tests/upstream/README.md](../../python/cuml/cuml_accel_tests/upstream/README.md) diff --git a/AGENTS.md b/AGENTS.md index e1612387f7..ecbaddf4f4 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -35,7 +35,14 @@ For pre-commit hooks, clang-tidy, branch naming, and the PR process, see [CONTRI Key conventions: - Linter errors are auto-fixed by pre-commit hooks — don't fix them manually unless asked. -- Always activate your cuML conda dev environment before running `build.sh`, `pytest`, or `pip`. Env naming and layout vary by developer — use whichever env contains the cuML you're working on. +- For environment policy, see the section below. + +## Environment policy for agents + +- **Prefer the already-active env.** If `$CONDA_PREFIX` or `$VIRTUAL_ENV` is set and `cuml` is importable from the current worktree, use it — no setup needed. +- **Otherwise respect the configured interpreter.** Check `/.vscode/settings.json` (`python.defaultInterpreterPath`), then user-level personal rules (e.g. `~/AGENTS.md`, `~/.cursor/rules/`), then `/.conda-env` if it exists. +- **Only create a new env as a last resort.** Follow the full algorithm in [.agents/setup-dev-environment/SKILL.md §2](.agents/setup-dev-environment/SKILL.md#2-environment-selection-algorithm-canonical). +- **Never pick an env by name-guessing** from `conda env list`. Naming conventions belong to the user; the agent follows them, not the other way around. ## Code review guidelines From 3cbed24dbdbf9991d6b02e59e589fc9e368e0bd2 Mon Sep 17 00:00:00 2001 From: Simon Adorf Date: Fri, 1 May 2026 20:37:04 +0000 Subject: [PATCH 09/10] Widen codeowner and changed_files exclusions. --- .github/CODEOWNERS | 4 ++-- .github/workflows/pr.yaml | 15 ++++++++++----- 2 files changed, 12 insertions(+), 7 deletions(-) diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index 7d30438ae8..4a9fd6fa69 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -21,8 +21,8 @@ CMakeLists.txt @rapidsai/cuml-cmake-codeowners **/cmake/ @rapidsai/cuml-cmake-codeowners # agent instruction owners -/AGENTS.md @rapidsai/cuml-python-codeowners @rapidsai/cuml-cpp-codeowners -/.agents/ @rapidsai/cuml-python-codeowners @rapidsai/cuml-cpp-codeowners +**/AGENTS.md @rapidsai/cuml-python-codeowners @rapidsai/cuml-cpp-codeowners +**/.agents/ @rapidsai/cuml-python-codeowners @rapidsai/cuml-cpp-codeowners #CI code owners /.github/ @rapidsai/ci-codeowners diff --git a/.github/workflows/pr.yaml b/.github/workflows/pr.yaml index 3351be575d..0c0a01dc19 100644 --- a/.github/workflows/pr.yaml +++ b/.github/workflows/pr.yaml @@ -89,7 +89,8 @@ jobs: files_yaml: | build_docs: - '**' - - '!**/*/agents.md' + - '!**/AGENTS.md' + - '!**/.agents/**' - '!.coderabbit.yaml' - '!.devcontainer/**' - '!.git-blame-ignore-revs' @@ -113,7 +114,8 @@ jobs: - '!codecov.yml' test_cpp: - '**' - - '!**/*/agents.md' + - '!**/AGENTS.md' + - '!**/.agents/**' - '!.coderabbit.yaml' - '!.devcontainer/**' - '!.git-blame-ignore-revs' @@ -148,7 +150,8 @@ jobs: - '!wiki/**' test_notebooks: - '**' - - '!**/*/agents.md' + - '!**/AGENTS.md' + - '!**/.agents/**' - '!.coderabbit.yaml' - '!.devcontainer/**' - '!.git-blame-ignore-revs' @@ -180,7 +183,8 @@ jobs: - '!wiki/**' test_python_conda: - '**' - - '!**/*/agents.md' + - '!**/AGENTS.md' + - '!**/.agents/**' - '!.coderabbit.yaml' - '!.devcontainer/**' - '!.git-blame-ignore-revs' @@ -217,7 +221,8 @@ jobs: - '!wiki/**' test_python_wheels: - '**' - - '!**/*/agents.md' + - '!**/AGENTS.md' + - '!**/.agents/**' - '!.coderabbit.yaml' - '!.devcontainer/**' - '!.git-blame-ignore-revs' From 62957a43db3e42244e39022c6d316369629de4bb Mon Sep 17 00:00:00 2001 From: Simon Adorf Date: Fri, 8 May 2026 19:52:53 +0000 Subject: [PATCH 10/10] Remove agent env management skill --- .agents/build-cuml/SKILL.md | 8 +- .agents/setup-dev-environment/SKILL.md | 236 ------------------------- .agents/test-cuml/SKILL.md | 4 +- AGENTS.md | 14 +- 4 files changed, 9 insertions(+), 253 deletions(-) delete mode 100644 .agents/setup-dev-environment/SKILL.md diff --git a/.agents/build-cuml/SKILL.md b/.agents/build-cuml/SKILL.md index 294a3f78da..4023e44138 100644 --- a/.agents/build-cuml/SKILL.md +++ b/.agents/build-cuml/SKILL.md @@ -33,9 +33,9 @@ This builds and installs `libcuml` (C++), `cuml` (Python), and `prims` (tests) i - The agent has edited C++/CUDA/Cython/Python code in this repo and needs to rebuild before testing. - A test run fails with `ImportError`, missing `libcuml.so`, stale `.so`, or "module not found" symptoms after a code change — the local install is likely stale and needs a rebuild. -## 1. Set up the development environment +## 1. Confirm the active development environment -Building cuML requires an **active** development environment containing all build and runtime dependencies. Building with the wrong env active (or none) silently installs into — and tests against — the wrong worktree. +Building cuML requires an **active** development environment containing all build and runtime dependencies. Building with the wrong env active (or none) can install into — and test against — the wrong prefix. If the active environment is missing or ambiguous, and the current agent context does not provide clear local instructions for choosing one, stop and ask the user which development environment to use. ### Quick check @@ -46,7 +46,7 @@ python -c "import cuml; print(cuml.__file__)" 2>/dev/null \ || echo "cuml not yet installed (fine before first build)" ``` -`cuml.__file__` should resolve inside this worktree (`python/cuml/`) or inside the active env's `site-packages`. If it resolves into a different worktree or env, the wrong env is active — see [setup-dev-environment §2](../setup-dev-environment/SKILL.md#2-environment-selection-algorithm-canonical) for the selection algorithm. +`cuml.__file__` should resolve inside this worktree (`python/cuml/`) or inside the active env's `site-packages`. If it resolves into a different worktree or env, stop and ask for the local environment instructions before building. ## 2. Build with `build.sh` @@ -159,7 +159,7 @@ The `cuml.__file__` path should be inside the active conda env (or the editable - **`ImportError` after pulling new commits**: rebuild — the C++ ABI or Cython-generated code likely changed. - **`libcuml.so` not found at runtime**: `INSTALL_PREFIX` mismatched the active env. Re-run `build.sh` with the correct env activated. -- **`cuml.__file__` points to a different worktree than the one you're editing**: the wrong env is active. Activate the env that belongs to the worktree you're editing and rebuild. +- **`cuml.__file__` points to a different checkout than the one you're editing**: the wrong env is active. Activate the intended dev environment for this checkout and rebuild. - **Out-of-memory or thermal throttling during build**: lower `PARALLEL_LEVEL` (e.g. `PARALLEL_LEVEL=8`). - **Stale build state after a failed build**: run `./build.sh clean` then rebuild from scratch. - **Building without a GPU**: the build itself works on CPU-only hosts; only running cuML at test time requires a GPU. diff --git a/.agents/setup-dev-environment/SKILL.md b/.agents/setup-dev-environment/SKILL.md deleted file mode 100644 index c61467ce69..0000000000 --- a/.agents/setup-dev-environment/SKILL.md +++ /dev/null @@ -1,236 +0,0 @@ ---- -name: setup-dev-environment -description: Set up, create, update, or recreate a cuML conda dev environment for a worktree. Use whenever the user asks to set up a cuML development environment, create or recreate the conda env, configure editor/git exclusions for the env, update the env after pulling new commits, or remove the env. Also use when an agent needs to build or test cuML and there is no usable env to activate yet. ---- - -# Setting Up a cuML Dev Environment - -This skill covers selecting, activating, creating, updating, and removing the development environment for a cuML worktree. The canonical reference is [BUILD.md](../../BUILD.md); this skill captures the high-frequency workflows. - -## 1. When this skill applies - -- The agent needs to decide which env to use before building or testing. -- The user asks to set up / create / recreate a cuML dev env. -- The user wants to refresh an env after a pull, or remove an env entirely. - -## 2. Environment selection algorithm (canonical) - -**Before running the algorithm, read every personal rule, skill, or memory that may govern conda or dev-environment selection.** Sources include — but are not limited to — `~/.cursor/rules/`, `~/.cursor/skills/`, `~/.agents/`, `~/.agents/skills/`, `~/.claude/CLAUDE.md`, `~/AGENTS.md`, and any "available skills" / "rules" listing supplied by your agent runtime (e.g. Cursor's `` system block, Claude's `` block). The paths above are illustrative, not exhaustive — also follow whatever the runtime advertises. These rules establish the *selection policy* (e.g. "use the latest `cuml-YYYYMMDD` env") that the algorithm applies; if you skip this prelude you will likely create an unnecessary fresh env when a usable one already exists. - -Work through these steps in order. Stop at the first that matches. - -**Step 1 — Already-active env.** If an env is already active and `cuml` resolves inside this worktree or the active env's `site-packages`, use it and stop: - -```bash -# Quick check — run these three lines -echo "CONDA_PREFIX=${CONDA_PREFIX:-unset} VIRTUAL_ENV=${VIRTUAL_ENV:-unset}" -which python -python -c "import cuml; print(cuml.__file__)" 2>/dev/null \ - || echo "cuml not yet installed (fine before first build)" -``` - -If `cuml.__file__` lives inside this worktree (`python/cuml/`) or inside the active env's `site-packages`, **you're done** — proceed to build or test. - -**Step 2 — Editor-configured interpreter.** If `/.vscode/settings.json` sets `python.defaultInterpreterPath` *and the interpreter exists on disk*, activate that env (or use that Python directly) — this is the user's explicit choice for the worktree. If the configured path doesn't exist (e.g. it points at `${workspaceFolder}/.conda-env/bin/python` but `.conda-env/` was never created), treat this step as inapplicable and fall through to Step 3 — do **not** treat it as a directive to create that env. - -```bash -python3 -c " -import json, pathlib -s = pathlib.Path('.vscode/settings.json') -if s.exists(): - d = json.loads(s.read_text()) - print(d.get('python.defaultInterpreterPath', '(not set)')) -" -``` - -**Step 3 — Apply the personal-rule selection policy.** If the rules you loaded in the prelude define an env selection policy (e.g. "use the `cuml-YYYYMMDD` env with the latest date suffix", or "always use env `foo`"), apply it now and stop on first match. This step **overrides Steps 4–5** (the worktree-prefix default and the fresh-env fallback) — do not create a new env when an env mandated by personal rules already exists. - -*Worked example.* If a personal rule says "use the `cuml-YYYYMMDD` env with the largest date suffix": - -```bash -LATEST=$(conda env list | awk '/cuml-[0-9]{8}( |$)/ {print $1}' | sort | tail -1) -echo "Picking $LATEST" -conda activate "$LATEST" -``` - -Then verify with §3 below. If the env doesn't yet have `cuml` installed, that's fine — proceed to build. - -**Step 4 — Worktree prefix env.** If `/.conda-env` exists, activate it: - -```bash -source "$HOME/miniforge3/etc/profile.d/conda.sh" # or the user's conda init path -conda activate "$(git rev-parse --show-toplevel)/.conda-env" -``` - -**Step 5 — Nothing exists.** No usable env was found. Create one from scratch — see [§4 Create a fresh env](#4-create-a-fresh-env-fallback). - -> **Do NOT** pick an arbitrary env from `conda env list` by guessing. Names like `cuml-work0`, `rapids-26.04`, `nvforest-work0` typically belong to other worktrees / cuML versions, and building into one silently installs a `cuml` whose ABI may not match this worktree's source. The exception is Step 3: when personal rules mandate a specific naming convention (e.g. `cuml-YYYYMMDD`), `conda env list` is the right tool to find the matching env — that's not "guessing", it's applying a documented policy. - -## 3. Sanity check after activation - -Regardless of how the env was selected, verify it's the right one before building or testing: - -```bash -echo "CONDA_PREFIX=${CONDA_PREFIX:-unset} VIRTUAL_ENV=${VIRTUAL_ENV:-unset}" -which python # should resolve inside the env -python -c "import cuml; print(cuml.__file__)" # should be inside this worktree, not another clone -``` - -The env is good when `cuml.__file__` lives inside this worktree (`python/cuml/`) or the active env's `site-packages`. If it points into a different worktree or env, the wrong env is active — revisit step 2 above. - -## 4. Create a fresh env (fallback) - -Use this only when steps 1–4 above found nothing to activate. The **recommended default** is an in-worktree prefix env at `/.conda-env` — one env per worktree, tied to the worktree lifetime. If the user prefers a different location or naming convention, follow their preference. - -Properties of the in-worktree prefix env: - -- **Deterministic activation**: `conda activate "$(git rev-parse --show-toplevel)/.conda-env"` always activates the env that belongs to the current worktree. No name-derivation logic, no collision handling. -- **Worktree-bound**: deleting the worktree (`rm -rf` or `git worktree remove`) removes the env. No orphan envs accumulating in `~/miniforge3/envs/`. -- **Parallel-agent safe**: two agents in two worktrees can never accidentally activate each other's env. - -The `.conda-env/` directory is large (~5–10 GB on disk; conda hardlinks packages from its global cache, so real disk cost is much smaller). - -### Step 1: Pick the right env file - -Env files are named `all_cuda-_arch-.yaml`. Two axes: - -```bash -ls conda/environments/all_*.yaml -# all_cuda-129_arch-aarch64.yaml -# all_cuda-129_arch-x86_64.yaml -# all_cuda-131_arch-aarch64.yaml -# all_cuda-131_arch-x86_64.yaml -``` - -**Architecture (`arch-`)** — must match the host CPU architecture. Always use: - -```bash -uname -m # → x86_64 or aarch64 -``` - -**CUDA version (`cuda-`)** — this is the version of the CUDA toolkit and CUDA runtime that conda will install into the env (cuML does not require a system CUDA install). Choose based on the host's NVIDIA driver and GPU compute capability: - -| Env file | Conda CUDA | Min host driver | Min GPU compute capability | -| --- | --- | --- | --- | -| `cuda-131` (recommended default) | 13.1 | R580+ | 7.5 (Turing or newer) | -| `cuda-129` | 12.9 | R525+ | 7.0 (Volta or newer) | - -**Decision rule:** - -1. **Default to `cuda-131`** unless one of the conditions below applies. -2. Use `cuda-129` if the host has a **Volta (sm_70) GPU** (e.g. V100) — CUDA 13 dropped Volta support. -3. Use `cuda-129` if the host's NVIDIA driver is older than R580 — `nvidia-smi` will show the max CUDA version supported. - -**Detect the right file automatically:** - -```bash -ARCH=$(uname -m) - -# Read GPU compute capability (e.g. "7.0", "7.5", "8.0", "9.0") -CC=$(nvidia-smi --query-gpu=compute_cap --format=csv,noheader,nounits | head -n1) - -# Read the max CUDA version the installed driver supports (e.g. "13.0", "12.9") -DRIVER_CUDA=$(nvidia-smi --query-gpu=driver_version --format=csv,noheader | head -n1) -echo "Driver: $DRIVER_CUDA GPU compute cap: $CC Arch: $ARCH" - -# Use cuda-129 if any Volta GPU is in the system; otherwise prefer cuda-131 -if awk "BEGIN {exit !($CC < 7.5)}"; then - CUDA_TAG=129 -else - CUDA_TAG=131 -fi -ENV_FILE="conda/environments/all_cuda-${CUDA_TAG}_arch-${ARCH}.yaml" -echo "Using $ENV_FILE" -``` - -If unsure, on multi-GPU hosts query each GPU and pick the lowest compute capability. If the host has no GPU (CPU-only build), `cuda-131` is fine. - -### Step 2: Configure exclusions (do this BEFORE creating the env) - -> **Critical ordering:** add the git and editor exclusions *before* running `conda create`. Otherwise git will momentarily see ~20k+ untracked files inside `.conda-env/`, the VS Code/Cursor Git extension will warn `"too many active changes, only a subset of Git features will be enabled"`, and editor indexing/file-watching will spike CPU before the exclusions kick in. - -The `.conda-env/` directory must never be committed and must be excluded from editor indexing/search. Two git-side options (pick one): - -**Option A — worktree-local exclude (recommended for agents):** doesn't modify any tracked file. - -```bash -grep -qxF '/.conda-env/' .git/info/exclude || echo '/.conda-env/' >> .git/info/exclude -``` - -**Option B — repo-wide `.gitignore`:** if the convention is repo-wide, add `/.conda-env/` to `.gitignore` and commit it. - -For VS Code / Cursor, also write `.vscode/settings.json` (worktree-local, not committed) to keep the editor responsive **and** point the Python extension at the env so terminals, debug, tests, and IntelliSense all use it automatically. This `python.defaultInterpreterPath` setting is also what later agent runs check (per §2 step 2) to avoid creating a duplicate env. Target settings: - -```json -{ - "python.defaultInterpreterPath": "${workspaceFolder}/.conda-env/bin/python", - "python.terminal.activateEnvironment": true, - "files.watcherExclude": { "**/.conda-env/**": true }, - "files.exclude": { "**/.conda-env": true }, - "search.exclude": { "**/.conda-env": true } -} -``` - -Run the script below from the worktree root to merge these into any existing `.vscode/settings.json` without clobbering it (uses the system `python3`, no conda env required): - -```bash -mkdir -p .vscode -python3 - <<'PY' -import json, pathlib -p = pathlib.Path(".vscode/settings.json") -data = json.loads(p.read_text()) if p.exists() else {} -data["python.defaultInterpreterPath"] = "${workspaceFolder}/.conda-env/bin/python" -data["python.terminal.activateEnvironment"] = True -data.setdefault("files.watcherExclude", {})["**/.conda-env/**"] = True -data.setdefault("files.exclude", {})["**/.conda-env"] = True -data.setdefault("search.exclude", {})["**/.conda-env"] = True -p.write_text(json.dumps(data, indent=2) + "\n") -PY -``` - -Notes: - -- `python.defaultInterpreterPath` only takes effect on first workspace open or after running `Python: Clear Workspace Interpreter Setting`. If the workspace was already open with a different interpreter, run that command (or pick the new interpreter via `Python: Select Interpreter`) once to switch. -- `python.terminal.activateEnvironment` is `true` by default; listing it makes the intent explicit and survives users who toggled it off globally. - -### Step 3: Create and populate the env - -Match the Python version to what the YAML supports (currently `>=3.11,<=3.14`). Create the env at `/.conda-env`: - -```bash -PREFIX_ENV="$(git rev-parse --show-toplevel)/.conda-env" -conda create -y --prefix "$PREFIX_ENV" python=3.14 -conda env update --prefix "$PREFIX_ENV" --file="$ENV_FILE" -conda activate "$PREFIX_ENV" -``` - -After populating, sanity-check that git doesn't see env files: - -```bash -git status -s | wc -l # should be 0 (or just your real changes); not thousands -``` - -## 5. Update an existing dev environment - -After pulling new commits that change `dependencies.yaml` or the env files, refresh the env using **the same env file the env was originally created from** (don't switch CUDA versions on an existing env — recreate instead): - -```bash -PREFIX_ENV="$(git rev-parse --show-toplevel)/.conda-env" -conda env update --prefix "$PREFIX_ENV" --file="$ENV_FILE" -``` - -## 6. Remove an env - -```bash -conda deactivate 2>/dev/null || true -rm -rf "$(git rev-parse --show-toplevel)/.conda-env" -``` - -(Or `conda env remove --prefix "$(git rev-parse --show-toplevel)/.conda-env"`. The `rm -rf` is faster and works even if conda isn't initialized.) - -## Additional resources - -- Full build docs: [BUILD.md](../../BUILD.md) -- Conda environment files: `conda/environments/all_cuda-*_arch-*.yaml` -- Build skill (for building after the env is ready): [.agents/build-cuml/SKILL.md](../build-cuml/SKILL.md) -- Test skill: [.agents/test-cuml/SKILL.md](../test-cuml/SKILL.md) diff --git a/.agents/test-cuml/SKILL.md b/.agents/test-cuml/SKILL.md index ac428c3174..65f646a329 100644 --- a/.agents/test-cuml/SKILL.md +++ b/.agents/test-cuml/SKILL.md @@ -9,7 +9,7 @@ This skill covers all cuML test suites. The canonical source of truth for each s ## 0. Prerequisites -**A cuML dev env must be active before running any tests.** If you don't have one active yet, follow the [setup-dev-environment selection algorithm](../setup-dev-environment/SKILL.md#2-environment-selection-algorithm-canonical). +**A cuML dev env must be active before running any tests.** If the active environment is missing or ambiguous, and the current agent context does not provide clear local instructions for choosing one, stop and ask the user which development environment to use before running tests. Quick sanity check before invoking any test command: @@ -232,7 +232,7 @@ CI script: [`ci/test_python_integration.sh`](../../ci/test_python_integration.sh ## 7. Common gotchas -- **`ImportError` or wrong `cuml.__file__`**: the wrong env is active. Activate the env that belongs to the worktree you're editing and rebuild. See the [build skill](../build-cuml/SKILL.md). +- **`ImportError` or wrong `cuml.__file__`**: the wrong env is active. Activate the intended dev environment for this checkout and rebuild. See the [build skill](../build-cuml/SKILL.md). - **C++ test binaries not found**: the install dir `$CONDA_PREFIX/bin/gtests/libcuml/` is absent. Run `./build.sh` (default builds and installs the tests). - **Dask import error in single-GPU env**: the single-GPU CI script (`test_python_singlegpu.sh`) intentionally fails if `dask` is installed. Use a dask-capable env for §3 tests. - **`UnmatchedXfailTests` in upstream sklearn tests**: (1) the xfail list references a test id that no longer exists in the installed library version — triage and update the list (see the upstream [README](../../python/cuml/cuml_accel_tests/upstream/README.md)). (2) You passed `--xfail-list` while collecting only a **subset** of the suite (e.g. `--pyargs sklearn.neighbors.tests.test_kde`); use [`run-tests.sh`](../../python/cuml/cuml_accel_tests/upstream/scikit-learn/run-tests.sh) with `-k` instead, or drop `--xfail-list` for a quick narrow run. diff --git a/AGENTS.md b/AGENTS.md index ecbaddf4f4..e55d6678cb 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -14,9 +14,9 @@ This is [cuML](https://github.com/rapidsai/cuml), the RAPIDS GPU-accelerated mac ## Building cuML -Before testing any local code change that touches C++/CUDA/Cython, the local install must be rebuilt. Follow the dedicated build skill: +Before testing any local code change that touches C++/CUDA/Cython, the local install must be rebuilt in an active cuML development environment. Follow the dedicated build skill: -- [.agents/build-cuml/SKILL.md](.agents/build-cuml/SKILL.md) — conda env setup, `build.sh` usage, ccache, and common gotchas. +- [.agents/build-cuml/SKILL.md](.agents/build-cuml/SKILL.md) — `build.sh` usage, ccache, and common gotchas. The full build reference is [BUILD.md](BUILD.md). @@ -24,7 +24,7 @@ The full build reference is [BUILD.md](BUILD.md). For running the C++ gtests, the standard pytest suite, dask tests, the cuml.accel suite, upstream-library tests under cuml.accel, and the integration tests, follow the dedicated test skill: -- [.agents/test-cuml/SKILL.md](.agents/test-cuml/SKILL.md) — env activation, `ci/run_*` entry points, common pytest args, and per-suite gotchas. +- [.agents/test-cuml/SKILL.md](.agents/test-cuml/SKILL.md) — active environment checks, `ci/run_*` entry points, common pytest args, and per-suite gotchas. The CI scripts under [ci/](ci/) (`test_cpp.sh`, `test_python_singlegpu.sh`, `test_python_dask.sh`, `test_python_integration.sh`, `test_python_scikit_learn_tests.sh`, `test_python_cuml_accel_upstream.sh`) are the source of truth for how each suite is invoked. @@ -35,14 +35,6 @@ For pre-commit hooks, clang-tidy, branch naming, and the PR process, see [CONTRI Key conventions: - Linter errors are auto-fixed by pre-commit hooks — don't fix them manually unless asked. -- For environment policy, see the section below. - -## Environment policy for agents - -- **Prefer the already-active env.** If `$CONDA_PREFIX` or `$VIRTUAL_ENV` is set and `cuml` is importable from the current worktree, use it — no setup needed. -- **Otherwise respect the configured interpreter.** Check `/.vscode/settings.json` (`python.defaultInterpreterPath`), then user-level personal rules (e.g. `~/AGENTS.md`, `~/.cursor/rules/`), then `/.conda-env` if it exists. -- **Only create a new env as a last resort.** Follow the full algorithm in [.agents/setup-dev-environment/SKILL.md §2](.agents/setup-dev-environment/SKILL.md#2-environment-selection-algorithm-canonical). -- **Never pick an env by name-guessing** from `conda env list`. Naming conventions belong to the user; the agent follows them, not the other way around. ## Code review guidelines