Conversation
Toolchains before CUDA 13 cap statically allocated shared memory at 48 KiB per block. W8 and NVFP4 GEMM kernels whose staged storage (activations, weight codes, scales) exceeds the cap now stage it in dynamic shared memory, sized through the matching dynamic-bytes helpers, and every launch site raises cudaFuncAttributeMaxDynamicSharedMemorySize when the storage exceeds the cap. Kernels whose storage fits the cap keep their static allocation.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 527e71443c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| endif() | ||
|
|
||
| if(CMAKE_CUDA_COMPILER_VERSION VERSION_LESS 13.1) | ||
| if(CMAKE_CUDA_COMPILER_VERSION VERSION_LESS 12.9) |
There was a problem hiding this comment.
When users consult the supported build prerequisites, README.md:25 and every current model card still require CUDA Toolkit 13.1 or newer, while this guard now accepts 12.9. Update those active references with the implementation so the newly supported toolchain is not contradicted or undiscoverable.
AGENTS.md reference: AGENTS.md:L19-L21
Useful? React with 👍 / 👎.
2 participants
What
*_dynamic_byteshelpers, and every launch site raisescudaFuncAttributeMaxDynamicSharedMemorySizewhen the storage exceeds the cap. Kernels whose storage fits the cap keep their static allocation.Files
CMakeLists.txt— version guard 13.1 -> 12.9.src/ops/linear/w8/— new shared-storage struct and dynamic-bytes helpers; the row-split MMA and small-T MMA kernels select static vs dynamic smem at compile time.attn_input_proj,gdn_input_proj,linear,linear_add,linear_pair,linear_swiglu, anddynamic_grouped_conv— attribute setup and the dynamic smem size argument.Verification
git patch-id).