Skip to content

feat(parallel): add topology-aware DLO planning - #52

Open
0z5a wants to merge 1 commit into
ViperEkura:mainfrom
0z5a:codex/topology-aware-dlo-dpsp
Open

feat(parallel): add topology-aware DLO planning#52
0z5a wants to merge 1 commit into
ViperEkura:mainfrom
0z5a:codex/topology-aware-dlo-dpsp

Conversation

@0z5a

@0z5a 0z5a commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Summary

  • add a fail-closed parser for nvidia-smi topo -m and a topology-aware DP/SP device-order planner for distributed layerwise offload
  • select DLO groups with vLLM-Omni-compatible semantics while excluding TP, and prefer measured collective results over topology-label heuristics
  • map logical torchrun ranks to physical local devices without changing LOCAL_RANK; DDP consumes the resolved LOCAL_DEVICE
  • add planning and real NCCL benchmark CLIs, regression tests, launch documentation, machine-readable benchmark evidence, and an AstrAI-targeted InfraSWE Draft

Validation

  • 631 passed, 103 skipped in the complete AstrAI test suite
  • Ruff formatting and import-order checks pass
  • InfraSWE Draft resolution: explicit local AstrAI target, no audit flags
  • InfraSWE Draft/system-path engine tests: 53 passed

8x RTX 5060 Ti NCCL benchmark

The workload reconstructs a 256 MiB finalized one-byte INT8/FP8 DLO weight and runs a 64 MiB-per-rank SP all-to-all. Each result uses five warmups, 20 iterations, seven trials, and the slowest rank.

Shape Mapping Combined median Result
DP2 x SP4 natural 75.8654 ms baseline mapping
DP2 x SP4 PHB-paired topology 71.4831 ms 5.78% faster, DLO p99 67.73 -> 52.83 ms
DP4 x SP2 natural 55.2465 ms 9.67% faster than label-only choice
DP4 x SP2 label optimum 60.5890 ms rejected by measured planner

The DP4 result is an important negative control: minimizing DLO link labels alone can hurt SP traffic, so the planner keeps natural order when the measured combined cost is lower.

MiniMax-H3 end-to-end

Validated current vLLM-Omni with online FP8, DP1 x SP8, text TP8, VAE tile8, and DLO AllGather:

  • first valid 2-step request: 17.317 s client E2E
  • warm 2-step request: 13.714 s client E2E
  • 10-step stability request: 71.719 s client E2E
  • all eight GPUs reached 100% utilization; peak memory was 14,206 MiB of 16,311 MiB
  • output was verified as H.264 832x480@24 FPS plus stereo AAC 32 kHz

InfraSWE diagnostic

  • diagnostic ProjectFit: 86.73/100
  • BenchmarkTrust: 93.06/100
  • official ProjectFit: unresolved

The diagnostic score is intentionally non-official. The checked-in score card contains every subcomponent input and rationale. Official scoring remains blocked on Draft seal, hidden probes, and a verified evidence manifest.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant