Update Minimax2.5 H100#484
Conversation
Signed-off-by: Faradawn Yang <73060648+faradawn@users.noreply.github.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request updates the MiniMax-M2.5 model configuration by adopting the minimax_m2_append_think reasoning parser, removing the cudagraph_mode: PIECEWISE parameter, and increasing the tensor parallel size for H100 configurations. Review feedback highlights several inconsistencies where these updates were not applied to all relevant examples in the guide, specifically the NVIDIA H200 and AMD ROCm sections. Additionally, a correction was suggested for the H100 heading to align with the established naming convention.
| - "--trust-remote-code" | ||
| - "--compilation-config" | ||
| - '{"mode":3,"cudagraph_mode":"PIECEWISE","pass_config":{"fuse_minimax_qk_norm":true}}' | ||
| - '{"mode":3,"pass_config":{"fuse_minimax_qk_norm":true}}' |
There was a problem hiding this comment.
The removal of cudagraph_mode: PIECEWISE should be applied consistently across all examples in the guide section. While it has been removed here and in the Docker example (line 155), the NVIDIA H200 example at line 179 still includes this parameter.
| args: | ||
| - "--reasoning-parser" | ||
| - "minimax_m2" | ||
| - "minimax_m2_append_think" |
There was a problem hiding this comment.
The update to the minimax_m2_append_think reasoning parser should be applied consistently across all examples in the guide section. Currently, the NVIDIA H200 example (line 178) and the AMD ROCm example (line 204) still reference the old minimax_m2 parser.
| Pure TP8 is not supported. For >4 GPUs use DP+EP or TP+EP. | ||
|
|
||
| ### TP4+EP (recommended for H100) | ||
| ### TEP=8 (recommended for H100) |
There was a problem hiding this comment.
The heading TEP=8 is inconsistent with the previous heading style (TP4+EP) and the terminology used in the preceding text (line 184: TP+EP). Consider using TP8+EP to maintain consistency with the rest of the document.
### TP8+EP (recommended for H100)Signed-off-by: Faradawn Yang <73060648+faradawn@users.noreply.github.com>
|
NVFP4 H100 configs from https://github.com/SemiAnalysisAI/InferenceX/pull/1517/changes |
|
Could you clarify why we need to change the reasoning parser? |
|
I think its to add support for thinking. |
2 participants
Update the MiniMax-M2.5 recipe to match the new H100 FP8 launch flags validated in SemiAnalysisAI/InferenceX#1516: drop the explicit cudagraph_mode from the compilation-config, switch the reasoning parser to minimax_m2_append_think, and document TEP=8 (tensor-parallel-size 8 with --enable-expert-parallel) as the recommended single-node H100 strategy in place of TP4+EP.