Glm4MoeLiteForCausalLM computes its MoE router in bf16; upstream Glm4MoE's gate is unconditionally fp32

[localai-org-maint-bot]

Row: MODEL-TEXT-GLM4-MOE-LITE-ROUTER-F32

GLM-4.7-Flash (Glm4MoeLiteForCausalLM) computes its MoE router logits in
bf16. Upstream computes them in fp32, unconditionally, and it does so
because GLM's MoE block is a different class from DeepSeek-V2's.

Upstream, read at the pin the goldens were captured on (5559679229)

Glm4MoeLiteForCausalLM's MoE block is Glm4MoeLite, which is a bare subclass
of Glm4MoE:

• vllm/model_executor/models/glm4_moe_lite.py:86-87 — class Glm4MoeLite(Glm4MoE): pass
• vllm/model_executor/models/glm4_moe_lite.py:161-165 — the decoder layer instantiates it

and Glm4MoE's gate is an fp32 nn.Linear fed fp32 activations:

• vllm/model_executor/models/glm4_moe.py:141-146 — self.gate = nn.Linear(hidden_size, n_routed_experts, bias=False, dtype=torch.float32)
• vllm/model_executor/models/glm4_moe.py:218 — router_logits = self.gate(hidden_states.to(dtype=torch.float32))
• vllm/model_executor/models/glm4_moe.py:205 — router_logits_dtype=torch.float32

There is no config key in that path. The dtype is a property of the class.

DeepseekV2MoE resolves the same dtype from the config instead:

• vllm/model_executor/models/deepseek_v2.py:308-314 — self.router_dtype = _get_moe_router_dtype(config), GateLinear(..., out_dtype=self.router_dtype)
• vllm/model_executor/models/deepseek_v2.py:123-133 — fp32 only for model_type == "glm_moe_dsa" or an explicit moe_router_dtype: "float32"

Ours

src/vllm/model_executor/models/glm4_moe_lite_registry.cpp composes the whole
DeepSeek-V2 forward and loader, so GLM's router dtype is resolved by
ParseDeepseekV2Params, which mirrors _get_moe_router_dtype:

• src/vllm/model_executor/models/deepseek_v2_weights.cpp:332 — p.router_dtype_is_f32 = RawString(doc, "moe_router_dtype", "") == "float32";
• src/vllm/model_executor/models/deepseek_v2.cpp:363 — DBuf dlog(d, p.router_dtype_is_f32 ? DType::kF32 : DType::kBF16, {T, E});

The published zai-org/GLM-4.7-Flash config.json declares no
moe_router_dtype key, and its model_type is glm4_moe_lite, so the flag
resolves false and the router logits are stored bf16 before the top-k.

That parser is right for the architecture it serves; it must not read
model_type (deepseek_v2_weights.cpp:321-330 says so). The GLM registration
is what has to supply GLM's answer, exactly as GLM-5.3 already does at
src/vllm/model_executor/models/glm_moe_dsa.cpp:353.

Why it matters here rather than on DeepSeek-V2

GLM-4.7-Flash routes top-4 of 64 experts with topk_method: noaux_tc
(sigmoid scores + e_score_correction_bias for selection),
norm_topk_prob: true and routed_scaling_factor: 1.8. Rounding the logits to
bf16 before the top-k perturbs a discrete selection: an 8-bit mantissa is
~4e-3 relative, and any two experts whose logits sit inside that band can swap
across the rank-4 boundary. The error is bimodal, not a tolerance.
DeepseekV2ForCausalLM's own gate vehicle (DeepSeek-V2-Lite) is top-2 of 4 with
softmax/greedy and no bias, which is why nothing caught this.

The comment at include/vllm/model_executor/models/deepseek_v2.h:164-168 states
that "a token gate cannot see this either way". That is true of a store that is
merely too wide. It is not true of one that is too narrow in front of a top-k.

Consequence for the SACRED gate

Glm4MoeLiteForCausalLM matches the pinned oracle at 69 of 128 positions and 1
of 8 prompts (#2839, PR #2906). This is a divergence on the exact block the
router feeds, found by reading upstream at the goldens' own pin.

It is not established that this is the whole 59-position gap. No
zai-org/GLM-4.7-Flash snapshot exists on this box or on the NAS, so the
repaired forward cannot be run against the oracle here, and the committed
our_ids.npy is a frozen capture that no forward change can move.