Hi gitcommit90,
thank you for your work, it works for me almost. I get following Error.
GB10/SM 12.1) — crashes entire vLLM server after single chat completion
Summary
After loading GLM-5.3-Flash EXL3 2.05 bpw with DFlash2 speculative decoding on a single NVIDIA DGX Spark (GB10, SM 12.1, ARM64), the first POST /v1/chat/completions request succeeds partially but then crashes the EngineCore process with:
RuntimeError: CUDA driver error (...deepgemm-src/.../jit/handle.hpp:154):
700 (CUDA_ERROR_ILLEGAL_ADDRESS, an illegal memory access was encountered)
The crash occurs in vllm.utils.deep_gemm.tf32_hc_prenorm_gemm, called from the GLM-5.3 HC pre-normalization fused MoE layer (mhc_pre_tilelang). The EngineCore process terminates and the API server shuts down entirely — every subsequent request returns 500 until restart.
I try it in opencode and cline. The error appears after reading md files.
Hi gitcommit90,
thank you for your work, it works for me almost. I get following Error.
GB10/SM 12.1) — crashes entire vLLM server after single chat completion
Summary
After loading GLM-5.3-Flash EXL3 2.05 bpw with DFlash2 speculative decoding on a single NVIDIA DGX Spark (GB10, SM 12.1, ARM64), the first
POST /v1/chat/completionsrequest succeeds partially but then crashes the EngineCore process with:The crash occurs in
vllm.utils.deep_gemm.tf32_hc_prenorm_gemm, called from the GLM-5.3 HC pre-normalization fused MoE layer (mhc_pre_tilelang). The EngineCore process terminates and the API server shuts down entirely — every subsequent request returns 500 until restart.I try it in opencode and cline. The error appears after reading md files.