Skip to content

docs: add flash-attn import troubleshooting - #2133

Open
zcxGGmu wants to merge 2 commits into
kvcache-ai:mainfrom
zcxGGmu:docs/flash-attn-import-troubleshooting
Open

docs: add flash-attn import troubleshooting#2133
zcxGGmu wants to merge 2 commits into
kvcache-ai:mainfrom
zcxGGmu:docs/flash-attn-import-troubleshooting

Conversation

@zcxGGmu

@zcxGGmu zcxGGmu commented Aug 4, 2026

Copy link
Copy Markdown

Summary

  • Add troubleshooting for NameError: name 'flash_attn_func' is not defined in the install guide.
  • Show a direct import check for the active environment and clarify that the flash-attn wheel must match Python / PyTorch / CUDA / C++ ABI.
  • Point balance_serve users to the patched FlashInfer note instead of replacing that package.

Refs #944

Test Plan

  • python documentation guard check for the new flash-attn troubleshooting text and link target
  • git diff --check HEAD~1..HEAD

@greptile-apps

greptile-apps Bot commented Aug 4, 2026

Copy link
Copy Markdown

Greptile Summary

This PR adds a flash-attn import troubleshooting block to the installation guide, addressing issue #944. The addition covers the NameError: name 'flash_attn_func' is not defined failure, provides a one-liner import check, and points balance_serve users to the correct FlashInfer note.

  • Adds a diagnostic python -c command to verify the active environment has flash-attn installed correctly.
  • Clarifies wheel compatibility requirements (Python, CUDA, PyTorch, C++ ABI) and links to the balance-serve installation guide for users of that backend.

Confidence Score: 5/5

Documentation-only change with no runtime impact; the link target and diagnostic command are both correct.

The change adds a self-contained troubleshooting block to a Markdown file. The ./balance-serve.md#installation-guide anchor resolves to the correct heading in that file, and the python -c import check is syntactically valid. No code is modified.

Files Needing Attention: No files require special attention.

Important Files Changed

Filename Overview
doc/en/install.md Adds 12-line troubleshooting block for flash-attn import errors; link target and diagnostic command are both valid.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[flash_attn_func NameError at runtime] --> B{Run import check}
    B --> |"python -c 'from flash_attn import flash_attn_func'"| C{Import succeeds?}
    C --> |Yes| D[Reinstall KTransformers / check sys.path]
    C --> |No| E{Using balance_serve backend?}
    E --> |Yes| F[Follow balance-serve FlashInfer note\nDo NOT replace patched FlashInfer]
    E --> |No| G[Download matching flash-attn wheel\nPython + CUDA + PyTorch + C++ ABI]
    G --> H[Reinstall wheel & retry]
Loading

Reviews (2): Last reviewed commit: "[docs]: refine flash-attn troubleshootin..." | Re-trigger Greptile

Comment thread doc/en/install.md Outdated
@zcxGGmu

zcxGGmu commented Aug 5, 2026

Copy link
Copy Markdown
Author

Thanks for the review — I pushed 39aef7d0130c addressing the feedback.

Changes made:

  • Removed duplicate PyTorch wording from the flash-attn troubleshooting guidance.

Validation:

  • git diff --check — passed.
  • Content guard — passed: flash-attn troubleshooting now focuses on wheel compatibility.
  • Diff scan — passed: no secret/debug-marker patterns found.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants