Skip to content

CUDA 12.8 install of dflash[sglang] fails #123

@Sylvan820

Description

@Sylvan820

I am trying to set up the SGLang backend in a CUDA 12.8 environment, but the standard install path fails:

uv pip install -e ".[sglang]"

Environment:

CUDA: 12.8
Python: 3.11
Install command: uv pip install -e ".[sglang]"
Backend: sglang

From pyproject.toml, the SGLang extra currently depends on:

sglang[all] @ git+https://github.com/sgl-project/sglang.git@refs/pull/23000/head#subdirectory=python

However, when installing in a CUDA 12.8 environment, many dependencies appear to resolve toward CUDA 13.0 / newer SGLang-compatible packages, and the installation fails. My guess is that the latest SGLang PR dependency may currently assume or prefer CUDA 13.0-compatible packages, while CUDA 12.8 is still common in cloud environments.

Would it be possible to provide one of the following?

  1. A recommended installation command for CUDA 12.8.
  2. A known-good constraints file for uv pip install -e ".[sglang]".
  3. A pinned SGLang commit/PR that works with CUDA 12.8.
  4. An official Docker image for the SGLang backend.
  5. Documentation clarifying which CUDA / PyTorch / SGLang versions are expected to work.

For example, a CUDA 12.8 setup guide could specify the exact compatible versions of:

torch
torchvision
torchaudio
sglang
flashinfer
cuda-python
transformers

This would make it much easier to reproduce the SGLang backend environment reliably.

Thanks!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions