Installation failure: flash-attn cannot build in HPC environments without CUDA

[seansica]

Environment

• OS: Linux x86_64 (HPC login node)
• Python: 3.10
• PyTorch: 2.7.0+cu126
• uv version: 0.7.8
• HPC Environment: Login nodes without direct GPU/CUDA access (jobs dispatched to GPU nodes via Slurm)

Problem Description

When running uv sync or uv sync --no-build-isolation, the installation fails because flash-attn attempts to build from source but cannot find required CUDA dependencies on HPC login nodes.

$ uv sync --no-build-isolation
Using CPython 3.10.12 interpreter at: /usr/bin/python3.10
Creating virtual environment at: .venv
Resolved 106 packages in 3ms
  × Failed to build `flash-attn==2.7.4.post1`
  ├─▶ The build backend returned an error
  ╰─▶ Call to `setuptools.build_meta:__legacy__.build_wheel` failed (exit status: 1)

      [stderr]
      Traceback (most recent call last):
        File "<string>", line 8, in <module>
      ModuleNotFoundError: No module named 'setuptools'

      hint: This usually indicates a problem with the package or the build environment.
  help: `flash-attn` (v2.7.4.post1) was included because `activault` (v0.1.0) depends on `flash-attn`

The output hinted that I should try uv pip install torch first before running uv sync. That worked for installing torch, but failed when trying to build/install flash-attn:

Resolved 106 packages in 3ms
  × Failed to build `flash-attn==2.7.4.post1`
  ├─▶ The build backend returned an error
  ╰─▶ Call to `setuptools.build_meta:__legacy__.build_wheel` failed (exit status: 1)

      [stdout]


      torch.__version__  = 2.7.0+cu126



      [stderr]
      /home/ssica/activault/.venv/lib/python3.10/site-packages/torch/_subclasses/functional_tensor.py:276: UserWarning: Failed to initialize NumPy: No module named 'numpy' (Triggered internally at /pytorch/torch/csrc/utils/tensor_numpy.cpp:81.)
        cpu = _conversion_method_template(device=torch.device("cpu"))
      <string>:106: UserWarning: flash_attn was requested, but nvcc was not found.  Are you sure your environment has nvcc available?  If you're installing within a container from https://hub.docker.com/r/pytorch/pytorch, only images whose names contain 'devel' will provide nvcc.
      Traceback (most recent call last):
        File "<string>", line 11, in <module>
        File "/home/ssica/activault/.venv/lib/python3.10/site-packages/setuptools/build_meta.py", line 432, in build_wheel
          return _build(['bdist_wheel'])
        File "/home/ssica/activault/.venv/lib/python3.10/site-packages/setuptools/build_meta.py", line 423, in _build
          return self._build_with_temp_dir(
        File "/home/ssica/activault/.venv/lib/python3.10/site-packages/setuptools/build_meta.py", line 404, in _build_with_temp_dir
          self.run_setup()
        File "/home/ssica/activault/.venv/lib/python3.10/site-packages/setuptools/build_meta.py", line 512, in run_setup
          super().run_setup(setup_script=setup_script)
        File "/home/ssica/activault/.venv/lib/python3.10/site-packages/setuptools/build_meta.py", line 317, in run_setup
          exec(code, locals())
        File "<string>", line 198, in <module>
        File "/home/ssica/activault/.venv/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1279, in CUDAExtension
          library_dirs += library_paths(device_type="cuda")
        File "/home/ssica/activault/.venv/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1490, in library_paths
          if (not os.path.exists(_join_cuda_home(lib_dir)) and
        File "/home/ssica/activault/.venv/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 2889, in _join_cuda_home
          raise OSError('CUDA_HOME environment variable is not set. '
      OSError: CUDA_HOME environment variable is not set. Please set it to your CUDA install root.

      hint: This usually indicates a problem with the package or the build environment.
  help: `flash-attn` (v2.7.4.post1) was included because `activault` (v0.1.0) depends on `flash-attn`

Root Cause

HPC environments typically:

1. Have login nodes without GPU access or CUDA toolkit
2. Dispatch GPU jobs to separate compute nodes via job schedulers (Slurm, PBS, etc.)
3. Don't have CUDA development tools available by default on login nodes

This prevents flash-attn from building during dependency resolution, even though the final code will run on GPU nodes.

Workaround

(This is what worked for me).

Install pre-compiled flash-attn wheel manually before running uv sync:

1. Identify which version of flash-attn to install for the environment:

   # In your virtual environment
   python -c "import sys, platform; print(f'Python: {sys.version_info.major}.{sys.version_info.minor}'); print(f'Platform: {platform.machine()}')"
   python -c "import torch; print(f'PyTorch: {torch.__version__}'); print(f'CXX11 ABI: {torch._C._GLIBCXX_USE_CXX11_ABI}')"
   
   # EXAMPLE OUTPUT:
   # PyTorch: 2.7.0+cu126
   # CUDA available: False
   # CUDA version: N/A
   # CXX11 ABI: True

   Prebuilt flash-attn packages for the latest release can be found here: https://github.com/Dao-AILab/flash-attention/releases/tag/v2.7.4.post1

2. Install compatible pre-built wheel:

   # Example for Python 3.10, PyTorch 2.6/2.7, CUDA 12, CXX11 ABI True
   uv pip install https://github.com/Dao-AILab/flash-attention/releases/download/v2.7.4.post1/flash_attn-2.7.4.post1+cu12torch2.6cxx11abiTRUE-cp310-cp310-linux_x86_64.whl

3. uv sync --no-build-isolation should work now.

Another Approach

Make flash-attn optional for GPU environments

Another option could be to make flash-attn optional for GPU environments—this would require user's run the command in their Slurm jobs before activault runs. Once the job gets dispatched to a compute node, the script should have CUDA access, which should make it possible to install flash-attn from source.

Notably, I didn't actually test this solution.

[project.optional-dependencies]
gpu = [
    "flash-attn>=2.7.0.post2",
]
dev = [
    "black",
    "isort", 
    "mypy",
    "pytest",
    "pytest-cov",
]

Usage:

• Development (login nodes): uv sync --extra dev
• GPU jobs: uv sync --extra gpu (run within Slurm job on compute nodes with CUDA)

Impact

This affects users in:

• HPC clusters in which login nodes lack CUDA access (CUDA only available on dispatch)
• CI/CD pipelines without GPU access

[dhruvbpai]

Thanks for raising this! Does uv add setuptools fix it? It is my understanding flash-attn only needs nvcc (not actual GPUs) to compile, so it should be feasible on a login node. If this does not work, making flash-attn optional is likely the best strategy...

[seansica]

That was actually the first thing I tried! It did not work (though I didn't dig much past that).

Here was the output of that run:

$ uv add setuptools
Resolved 106 packages in 3ms
      Built activault @ file:///home/ssica/activault
    Updated https://github.com/huggingface/transformers (46350f5eae87ac1d168ddfdc57a0b39b64b9a029)
  × Failed to build `flash-attn==2.7.4.post1`
  ├─▶ The build backend returned an error
  ╰─▶ Call to `setuptools.build_meta:__legacy__.build_wheel` failed (exit status: 1)

      [stderr]
      Traceback (most recent call last):
        File "<string>", line 14, in <module>
        File "/home/ssica/.cache/uv/builds-v0/.tmpJTpjfO/lib/python3.10/site-packages/setuptools/build_meta.py", line 331, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=[])
        File "/home/ssica/.cache/uv/builds-v0/.tmpJTpjfO/lib/python3.10/site-packages/setuptools/build_meta.py", line 301, in _get_build_requires
          self.run_setup()
        File "/home/ssica/.cache/uv/builds-v0/.tmpJTpjfO/lib/python3.10/site-packages/setuptools/build_meta.py", line 512, in run_setup
          super().run_setup(setup_script=setup_script)
        File "/home/ssica/.cache/uv/builds-v0/.tmpJTpjfO/lib/python3.10/site-packages/setuptools/build_meta.py", line 317, in run_setup
          exec(code, locals())
        File "<string>", line 22, in <module>
      ModuleNotFoundError: No module named 'torch'

      hint: This error likely indicates that `flash-attn@2.7.4.post1` depends on `torch`, but doesn't declare it as a build dependency. If `flash-attn` is a first-party package, consider adding `torch` to its `build-system.requires`. Otherwise, `uv pip install torch` into the
      environment and re-run with `--no-build-isolation`.
  help: If you want to add the package regardless of the failed resolution, provide the `--frozen` flag to skip locking and syncing.

[dhruvbpai]

This issue seems separate - it indicates that flash_attn was already installed and you are trying to modify a dependency, causing the package to re-build. The error here is not from setuptools installation but the order of ops, since if you are rebuilding flash_attn you need to pass the--no-build-isolation flag.

[seansica]

No, the error is indicating that it can't install flash-attn because torch is not available; torch is not available because it is not declared as a dependency of flash-attn (which we can validate by looking at flash-attn's pyproject.toml). Essentially, uv has no way of knowing that it needs to install flash-attn before torch, since flash-attn doesn't list torch as a dependency.

I think uv sync attempts to resolve dependencies in order of occurrence in pyproject.toml. A few GH Issues on the uv project seems to suggest this (example). I suggest moving torch up so that it appears above flash-attn in pyproject.toml.

Notably, it's trying to install 2.7.4.post1 instead of the listed 2.7.0.post1, it's because of the >= operator. (At the time of this writing, 2.7.4.post1 is the latest release).

This issue arose from a clean environment. I started without uv or a virtualenv. I ran git diff pyproject.toml to verify that the project file exactly matches origin main. The logs below show the exact commands I ran and their outputs.

$ pip install uv
Defaulting to user installation because normal site-packages is not writeable
Collecting uv
  Downloading uv-0.7.8-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (11 kB)
Downloading uv-0.7.8-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.5 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 17.5/17.5 MB 19.7 MB/s eta 0:00:00
Installing collected packages: uv
Successfully installed uv-0.7.8

[notice] A new release of pip is available: 25.0.1 -> 25.1.1
[notice] To update, run: pip install --upgrade pip

$ uv sync --no-build-isolation
Using CPython 3.10.12 interpreter at: /usr/bin/python3.10
Creating virtual environment at: .venv
Resolved 106 packages in 3ms
   Updating https://github.com/huggingface/transformers (v4.49.0-Gemma-3)
   Building activault @ file:///home/user/project/activault
⠙ Preparing packages... (0/89)
× Failed to build `activault @ file:///home/user/project/activault`
  ├─▶ The build backend returned an error
  ╰─▶ Call to `setuptools.build_meta.build_editable` failed (exit status: 1)

      [stderr]
      Traceback (most recent call last):
        File "<string>", line 8, in <module>
      ModuleNotFoundError: No module named 'setuptools'

      hint: This usually indicates a problem with the package or the build environment.

$ pip install setuptools
Defaulting to user installation because normal site-packages is not writeable
Collecting setuptools
  Downloading setuptools-80.9.0-py3-none-any.whl.metadata (6.6 kB)
Downloading setuptools-80.9.0-py3-none-any.whl (1.2 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 8.1 MB/s eta 0:00:00
Installing collected packages: setuptools
Successfully installed setuptools-80.9.0

[notice] A new release of pip is available: 25.0.1 -> 25.1.1
[notice] To update, run: pip install --upgrade pip

$ uv sync --no-build-isolation
Resolved 106 packages in 3ms
   Building activault @ file:///home/user/project/activault
   Updating https://github.com/huggingface/transformers (v4.49.0-Gemma-3)
⠙ Preparing packages... (0/89)
× Failed to build `activault @ file:///home/user/project/activault`
  ├─▶ The build backend returned an error
  ╰─▶ Call to `setuptools.build_meta.build_editable` failed (exit status: 1)

      [stderr]
      Traceback (most recent call last):
        File "<string>", line 8, in <module>
      ModuleNotFoundError: No module named 'setuptools'

      hint: This usually indicates a problem with the package or the build environment.

$ uv list
error: unrecognized subcommand 'list'

  tip: a similar subcommand exists: 'uv pip list'

Usage: uv [OPTIONS] <COMMAND>

For more information, try '--help'.

$ uv pip list

$ uv add setuptools
Resolved 106 packages in 3ms
      Built activault @ file:///home/user/project/activault
    Updated https://github.com/huggingface/transformers (46350f5eae87ac1d168ddfdc57a0b39b64b9a029)
  × Failed to build `flash-attn==2.7.4.post1`
  ├─▶ The build backend returned an error
  ╰─▶ Call to `setuptools.build_meta:__legacy__.build_wheel` failed (exit status: 1)

      [stderr]
      Traceback (most recent call last):
        File "<string>", line 14, in <module>
        File "/home/user/.cache/uv/builds-v0/.tmpJTpjfO/lib/python3.10/site-packages/setuptools/build_meta.py", line 331, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=[])
        File "/home/user/.cache/uv/builds-v0/.tmpJTpjfO/lib/python3.10/site-packages/setuptools/build_meta.py", line 301, in _get_build_requires
          self.run_setup()
        File "/home/user/.cache/uv/builds-v0/.tmpJTpjfO/lib/python3.10/site-packages/setuptools/build_meta.py", line 512, in run_setup
          super().run_setup(setup_script=setup_script)
        File "/home/user/.cache/uv/builds-v0/.tmpJTpjfO/lib/python3.10/site-packages/setuptools/build_meta.py", line 317, in run_setup
          exec(code, locals())
        File "<string>", line 22, in <module>
      ModuleNotFoundError: No module named 'torch'

      hint: This error likely indicates that `flash-attn@2.7.4.post1` depends on `torch`, but doesn't declare it as a build dependency. If `flash-attn` is a first-party package, consider adding `torch` to its `build-system.requires`. Otherwise, `uv pip install torch` into the
      environment and re-run with `--no-build-isolation`.
  help: If you want to add the package regardless of the failed resolution, provide the `--frozen` flag to skip locking and syncing.