[rl] Register customized config parser to vllm + less vllm config dependency by wwwjn · Pull Request #3242 · pytorch/torchtitan

wwwjn · 2026-05-06T19:07:58Z

Stack from ghstack (oldest at bottom):

vllm has this customized config parser registry support so we can plug in TorchTitan's config parser. Why we need this:

get rid of dependency on a HF format checkpoint folder when initializing. Don't implicitly depend on config.json as config source of truth

Another changes in this PR:

remove the round-trip translation from torchtitan config -> vllm config -> torchtitan config. Using closure to bypass.

[ghstack-poisoned]

vllm has this customized config parser registry support so we can plug in TorchTitan's config parser. It serves as 2 purpose: 1. get rid of dependency on a HF format checkpoint folder when initializing 2. Passing customized args to VLLMModelWrapper, eg CompileConfig, skip_init_load_weights [ghstack-poisoned]

…vllm" vllm has this customized config parser registry support so we can plug in TorchTitan's config parser. It serves as 2 purpose: 1. get rid of dependency on a HF format checkpoint folder when initializing 2. Passing customized args to VLLMModelWrapper, eg CompileConfig, skip_init_load_weights [ghstack-poisoned]

vllm has this customized config parser registry support so we can plug in TorchTitan's config parser. It serves as 2 purpose: 1. get rid of dependency on a HF format checkpoint folder when initializing 2. Passing customized args to VLLMModelWrapper, eg CompileConfig, skip_init_load_weights [ghstack-poisoned]

…vllm" vllm has this customized config parser registry support so we can plug in TorchTitan's config parser. It serves as 2 purpose: 1. get rid of dependency on a HF format checkpoint folder when initializing 2. Passing customized args to VLLMModelWrapper, eg CompileConfig, skip_init_load_weights [ghstack-poisoned]

vllm has this customized config parser registry support so we can plug in TorchTitan's config parser. It serves as 2 purpose: 1. get rid of dependency on a HF format checkpoint folder when initializing 2. Passing customized args to VLLMModelWrapper, eg CompileConfig, skip_init_load_weights [ghstack-poisoned]

…vllm + less vllm config dependency" vllm has this customized config parser registry support so we can plug in TorchTitan's config parser. Why we need this: - get rid of dependency on a HF format checkpoint folder when initializing. Don't implicitly depend on `config.json` as config source of truth Another changes in this PR: - remove the round-trip translation from torchtitan config -> vllm config -> torchtitan config. Using closure to bypass. [ghstack-poisoned]

… config dependency" vllm has this customized config parser registry support so we can plug in TorchTitan's config parser. Why we need this: - get rid of dependency on a HF format checkpoint folder when initializing. Don't implicitly depend on `config.json` as config source of truth Another changes in this PR: - remove the round-trip translation from torchtitan config -> vllm config -> torchtitan config. Using closure to bypass. [ghstack-poisoned]

wwwjn · 2026-05-08T23:01:25Z

 from torchtitan.components.optimizer import OptimizersContainer
-from torchtitan.config import CommConfig, Configurable, TORCH_DTYPE_MAP
-from torchtitan.config.configs import (
+from torchtitan.config import (


This change is just consolidate the import path

tianyu-l · 2026-05-08T23:03:56Z

@@ -199,14 +214,17 @@ def __init__(
        engine_kwargs = dict(
            model=model_path,


what is this path for?

Now it serves 2 purpose:

Loading tokenizer. This can be removed by passing tokenizer=tokenizer_path to EngineArgs.
2.Initial_weight_loading: Will sort out the weight loading part for both trainer and generator in next PR.

After lifting both, we can pass some fake path, say "torchtitan", to vllm

tianyu-l · 2026-05-08T23:05:25Z


        assert vllm_config is not None, "vllm_config is required"

+        # PP and CP are not supported on this inference path


this "raise ValueError" may better happen at grpo trainer post_init, to be consistent

here we only need assert

tianyu-l · 2026-05-08T23:28:35Z

+            **kwargs,
+        ):
+            config_dict = model_spec_to_hf_config_dict(model_spec)
+            return config_dict, PretrainedConfig.from_dict(config_dict)


It's actually very weird that the contract is both a config_dict and a cls(config_dict), sounds redundant to me

…vllm + less vllm config dependency" vllm has this customized config parser registry support so we can plug in TorchTitan's config parser. Why we need this: - get rid of dependency on a HF format checkpoint folder when initializing. Don't implicitly depend on `config.json` as config source of truth Another changes in this PR: - remove the round-trip translation from torchtitan config -> vllm config -> torchtitan config. Using closure to bypass. [ghstack-poisoned]

… config dependency" vllm has this customized config parser registry support so we can plug in TorchTitan's config parser. Why we need this: - get rid of dependency on a HF format checkpoint folder when initializing. Don't implicitly depend on `config.json` as config source of truth Another changes in this PR: - remove the round-trip translation from torchtitan config -> vllm config -> torchtitan config. Using closure to bypass. [ghstack-poisoned]

…vllm + less vllm config dependency" vllm has this customized config parser registry support so we can plug in TorchTitan's config parser. Why we need this: - get rid of dependency on a HF format checkpoint folder when initializing. Don't implicitly depend on `config.json` as config source of truth Another changes in this PR: - remove the round-trip translation from torchtitan config -> vllm config -> torchtitan config. Using closure to bypass. [ghstack-poisoned]

… config dependency" vllm has this customized config parser registry support so we can plug in TorchTitan's config parser. Why we need this: - get rid of dependency on a HF format checkpoint folder when initializing. Don't implicitly depend on `config.json` as config source of truth Another changes in this PR: - remove the round-trip translation from torchtitan config -> vllm config -> torchtitan config. Using closure to bypass. [ghstack-poisoned]

config parser

45c13fb

[ghstack-poisoned]

wwwjn requested review from fegin, tianyu-l and wconstab as code owners May 6, 2026 19:07

pytorch-bot Bot added the ciflow/8gpu label May 6, 2026

meta-cla Bot added the CLA Signed This label is managed by the Meta Open Source bot. label May 6, 2026

This was referenced May 6, 2026

[rl] Enable TP2EP for unified MoE model in vLLM wrapper #3142

Open

[WIP] Enable DP+EP for MoE inference in vLLM wrapper #3236

Open

wwwjn changed the title ~~config parser~~ [rl] Register customized config parser to vllm May 6, 2026

wwwjn commented May 6, 2026

View reviewed changes

Comment thread torchtitan/experiments/rl/actors/generator.py Outdated

Comment thread torchtitan/experiments/rl/models/vllm_registry.py Outdated

Comment thread torchtitan/hf_datasets/text_datasets.py

wwwjn commented May 6, 2026

View reviewed changes

Comment thread torchtitan/experiments/rl/models/vllm_config_parser.py Outdated

Comment thread torchtitan/experiments/rl/models/vllm_registry.py Outdated

Comment thread torchtitan/experiments/rl/models/vllm_registry.py Outdated

wwwjn commented May 6, 2026

View reviewed changes

Comment thread torchtitan/experiments/rl/actors/generator.py Outdated

tianyu-l reviewed May 6, 2026

View reviewed changes

Comment thread torchtitan/experiments/rl/models/vllm_wrapper.py Outdated

Comment thread torchtitan/experiments/rl/actors/generator.py Outdated

Comment thread torchtitan/experiments/rl/models/vllm_registry.py Outdated

Comment thread torchtitan/experiments/rl/models/vllm_registry.py

wwwjn added 4 commits May 6, 2026 13:33

pytorch-bot Bot added the ciflow/rl label May 7, 2026

wwwjn changed the title ~~[rl] Register customized config parser to vllm~~ [rl] Register customized config parser to vllm + less vllm config dependency May 7, 2026

wwwjn added 2 commits May 8, 2026 15:55

wwwjn commented May 8, 2026

View reviewed changes

Comment thread torchtitan/distributed/utils.py

wwwjn commented May 8, 2026

View reviewed changes

tianyu-l reviewed May 8, 2026

View reviewed changes

wwwjn added 2 commits May 9, 2026 20:43

tianyu-l approved these changes May 10, 2026

View reviewed changes

Comment thread torchtitan/experiments/rl/models/vllm_registry.py Outdated

Comment thread torchtitan/experiments/rl/grpo.py Outdated

Comment thread torchtitan/experiments/rl/models/vllm_wrapper.py Outdated

wwwjn added 2 commits May 11, 2026 08:27

wwwjn changed the base branch from gh/wwwjn/20/base to main May 11, 2026 15:49

wwwjn merged commit ca4c7f2 into main May 11, 2026
10 of 11 checks passed

		@@ -199,14 +214,17 @@ def __init__(
		engine_kwargs = dict(
		model=model_path,


		assert vllm_config is not None, "vllm_config is required"

		# PP and CP are not supported on this inference path

Conversation

wwwjn commented May 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

wwwjn May 8, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

tianyu-l May 8, 2026

Choose a reason for hiding this comment

Uh oh!

wwwjn May 10, 2026

Choose a reason for hiding this comment

Uh oh!

tianyu-l May 8, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

tianyu-l May 8, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

wwwjn commented May 6, 2026 •

edited

Loading