fix(_workload_envs): assert profiler=torch alongside the profile bounds injected for EXTRA_VLLM_ARGS - #1565
Open
BaoYunkai wants to merge 1 commit into
Open
fix(_workload_envs): assert profiler=torch alongside the profile bounds injected for EXTRA_VLLM_ARGS#1565BaoYunkai wants to merge 1 commit into
BaoYunkai wants to merge 1 commit into
Conversation
…ds vLLM injects for EXTRA_VLLM_ARGS Magpie's vllm_mi355x.sh sets --profiler-config.profiler torch itself, but only in the flags it appends after EXTRA_VLLM_ARGS at actual launch. The argv preflight probe (now fixed to call parse_args, see previous commit) only sees EXTRA_VLLM_ARGS, so it correctly parses the delay_iterations/max_iterations/capture_torch_profiler/ detailed_trace_annotation flags this layer injects, then vLLM's own ProfilerConfig validator rejects capture_torch_profiler without profiler=='torch' present in that same fragment -- an argv that is valid once Magpie's flags are appended, rejected as VALUE_REJECTED (terminal, no repair) because the checked fragment was incomplete. Assert --profiler-config.profiler torch in this same injection so the fragment Hyperloom controls is self-consistent on its own. A duplicate from Magpie's script is a harmless last-wins repeat under vLLM's dotted-flag merge, not a conflict; an operator-set profiler flag is left untouched.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Magpie's
vllm_mi355x.shsets--profiler-config.profiler torchitself, butonly in the flags it appends after
EXTRA_VLLM_ARGSat actual launch. Theargv preflight probe only sees
EXTRA_VLLM_ARGS, so it parses thedelay_iterations/max_iterations/capture_torch_profiler/detailed_trace_annotationflags this layer injects, and then vLLM's ownProfilerConfigvalidator rejectscapture_torch_profilerwithoutprofiler == 'torch'present in that same fragment.The result is an argv that is valid once Magpie's flags are appended being
rejected as
VALUE_REJECTED(terminal, no repair), because the checkedfragment was incomplete.
Fix
Assert
--profiler-config.profiler torchin this same injection, so thefragment Hyperloom controls is self-consistent on its own. A duplicate from
Magpie's script is a harmless last-wins repeat under vLLM's dotted-flag merge,
not a conflict; an operator-set profiler flag is left untouched.
Test plan
test_profile_and_kernel_handlers.py(175 passed; 1 failure,test_trace_certificate_stays_out_of_the_resolver_namespace, also failson unmodified main with
ModuleNotFoundError)ruff check .+ruff format --check .