feat(parallel): add topology-aware DLO planning - #52
Open
0z5a wants to merge 1 commit into
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
nvidia-smi topo -mand a topology-aware DP/SP device-order planner for distributed layerwise offloadLOCAL_RANK; DDP consumes the resolvedLOCAL_DEVICEValidation
631 passed, 103 skippedin the complete AstrAI test suite53 passed8x RTX 5060 Ti NCCL benchmark
The workload reconstructs a 256 MiB finalized one-byte INT8/FP8 DLO weight and runs a 64 MiB-per-rank SP all-to-all. Each result uses five warmups, 20 iterations, seven trials, and the slowest rank.
The DP4 result is an important negative control: minimizing DLO link labels alone can hurt SP traffic, so the planner keeps natural order when the measured combined cost is lower.
MiniMax-H3 end-to-end
Validated current vLLM-Omni with online FP8, DP1 x SP8, text TP8, VAE tile8, and DLO AllGather:
InfraSWE diagnostic
The diagnostic score is intentionally non-official. The checked-in score card contains every subcomponent input and rationale. Official scoring remains blocked on Draft seal, hidden probes, and a verified evidence manifest.