Does sonic-moe plan to support mxfp4 w4a16?

[yuan-luo]

DSV4 support mxfp4 moe weights, does sonic-moe plan to follow up?

[tridao]

Eventually, but we'll do mxfp8 first. For now if you want MXFP4, deepgemm is the way to go.

[yuan-luo]

Eventually, but we'll do mxfp8 first. For now if you want MXFP4, deepgemm is the way to go.

Thanks @tridao , I forgot to mention we are using hopper, deepgemm doesn't support mxfp4 in hopper, it only supports sm100 fp8_fp4 gemm.

Function	SM90 (Hopper)	SM100 (Blackwell)
FP8×FP8 GEMM	✅ sm90_fp8_gemm_1d1d/1d2d	✅ sm100_fp8_gemm_1d1d
FP8×FP4 GEMM	❌	✅ sm100_fp8_fp4_gemm_1d1d
FP4 MQA/Attention	❌	✅ sm100_fp4_mqa_logits
Mega MoE (fused EP)	❌	✅ sm100_fp8_fp4_mega_moe
Grouped GEMM (MoE)	✅ FP8 only (MGrouped)	✅ FP8 + FP4

[tridao]

i don't think mxfp4 is supported on hopper hardware

[yuan-luo]

i don't think mxfp4 is supported on hopper hardware

Hopper doesn't have tcgen05.mma, it doesn't support the native mxfp4 processing as on blackwell. The library like humming supports mxfp4 dequant to bf16, providing w4a16 moe mechanism on hopper.

[tridao]

Is the dequant done in a separate kernel or inside the moe kernel?