fix(MODEL-TEXT-GLM4-MOE-LITE-GATE-2839): the near-tie predicate cannot fire, so apply the bar the oracle capture licenses -- and it FAILS 69/128 - #2906
Open
localai-org-maint-bot wants to merge 3 commits into
Conversation
…te asserts a bar its own oracle capture does not license Three defects, each measured at `3e246b34f`. The only vs-vLLM assertion reads `neartie_gap_mnats.npy`, which is `min 0 max 0` at all 128 positions, against a 500-mnat band, so it has no failure mode. The gate computes vLLM's own self-determinism, prints DETERMINISTIC, and then applies the distributional band that `CLAUDE.md` §Gates licenses only on a NON-deterministic oracle; under the bar that is licensed the committed ids match 69 of 128 positions and 1 of 8 prompts. And the case returns early when the checkpoint is absent, which doctest reports as a passed case with zero assertions, so on every host without a 31.2B snapshot the SACRED gate reported a pass having measured nothing. The spec also records, without gating on it, why the gap artifact is probably not a measurement at all: at a first divergence our prefix equals the oracle's, so vLLM's teacher-forced argmax there is the golden token by construction and the gap must be positive. It is 0 at all seven first divergences, with none of the script's outside-top-20 sentinels present. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [claude-code]
…pture licenses, and gate the committed artifacts with no checkpoint THIS CHANGE MAKES A GATE FAIL. That is the result, not a regression. Do not soften the new case; the two admissible exits are named in the spec. Three repairs. The checkpoint predicate moves into `doctest::skip`, so an absent snapshot is reported as skipped instead of as a passed case with zero assertions, and the two golden-absence early returns become REQUIREs because those artifacts are committed. `greedy_dist.npy` becomes required, and its verdict now selects the bar in code rather than in a MESSAGE: zero multi-valued cells means STRICT token-exact, and only a non-zero count admits the near-tie band, which now also applies the second conjunct this file's own header states and nothing implemented -- our token inside vLLM's top-K. The third repair is what makes any of it observable. Everything the SACRED case compares is committed: the oracle's ids, the oracle's K-run distribution, and this engine's own ids, which the SACRED case already REQUIREs the live engine to reproduce token for token before comparing anything. So a new case applies the licensed bar to those artifacts with no 31.2B checkpoint and no GPU, and runs wherever the suite runs. Before, on this host: 1 test case, 1 PASSED, 0 failed, 0 skipped, and 0 assertions. After: 1 skipped, 1 failed, 10 assertions with 2 failed -- `positions 69/128, prompts exact 1/8, per-prompt mismatch [2,6,6,12,12,11,10,0]`, and `committed teacher-forced gap max = 0 mnats against a 500 mnat band`. Two artifact mutations, both restored and verified by sha256. Replacing `our_ids.npy` with `greedy_ids.npy` turns the case green at 10 of 10, so the failure is data-driven rather than an unconditional assertion. Perturbing one cell of `greedy_dist.npy` makes the bar-selection REQUIRE fire and abort before the comparison, so the bar really is chosen by the oracle's own capture. The engine case itself is UNRUN: no `zai-org/GLM-4.7-Flash` snapshot exists on this host and none is on the NAS. It compiles; nothing here can execute it. Closes #2839. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [claude-code]
mudler
force-pushed
the
row/MODEL-TEXT-GLM4-MOE-LITE-GATE-2839
branch
from
September 4, 2026 21:16
86b9729 to
f3f8f71
Compare
…o-assertion skip that hid this defect This row's D3 is one instance of a shape that occurs at 228 sites in 72 files: `MESSAGE(...SKIP...); return;` inside a TEST_CASE, which doctest counts as a passed case with zero assertions. Measured at `3f12c617d`, `test_deepseek_v2_paged_engine` -- the SACRED DeepSeek-V2 gate -- reports `1 passed | 0 failed`, `assertions: 0`, `Status: SUCCESS!` on a host with no checkpoints. Repairing 228 sites is a campaign rather than an in-flow fix, so it is filed as #2918 and listed here under `## Owed` O4, which is the ownership `CLAUDE.md` requires for an issue a change references without closing. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [claude-code]
mudler
added a commit
that referenced
this pull request
Sep 5, 2026
…r is fp32 because the class says so `Glm4MoeLite` IS `Glm4MoE` (`glm4_moe_lite.py:86-87`), whose gate is `nn.Linear(..., dtype=torch.float32)` fed `hidden_states.to(torch.float32)` and declared `router_logits_dtype=torch.float32` (`glm4_moe.py:141-146`, `:218`, `:205`). fp32 is a property of the CLASS, so no config key can withhold it — and the published GLM-4.7-Flash `config.json` ships no `moe_router_dtype`, which is exactly the value `ParseDeepseekV2Params` correctly reads for the architecture IT serves. We were inheriting DeepSeek-V2's answer for a GLM question. WHY THIS IS A CORRECTNESS FIX AND NOT A PRECISION PREFERENCE. Rounding the router logits to bf16 in front of a top-4-of-64 `noaux_tc` selection changes WHICH EXPERTS RUN. A selection gate has bimodal error, not a tolerance: the top-k either picks the same set or a different one, and when it flips, the output is not slightly wrong. The measured effect is exactly that shape — the dtype is observable in the GLM forward at 6 of 16 fixed seeds, worst |f32 - bf16| logit 0.00100262, which is small as a number and decisive as a tie-break. The reachability is proven from production rather than from a constructed object: `ParseGlm4MoeLiteParams` sets `p.router_dtype_is_f32 = true` (`glm4_moe_lite_registry.cpp:75`), it is consumed at `deepseek_v2.cpp:363` on the logit store before `vt::MoeRouterTopK`, and it is reached from `LoadGlm4MoeLiteForCausalLM` (`:126`) behind `REGISTER_VLLM_MODEL(glm4_moe_lite, ...)`. The mutations are the part that makes the tests worth having. M1 removes the one line and the gate goes RED on three cases (`assertions: 23 | 20 passed | 3 failed`) — so the flag is load-bearing. M3 and M4 attack the tests' own vacuity from the other side: M3 removes the bf16 round trip T3 rests on and the SET-difference count falls to `0/4096`, and M4 makes T2's second arm f32 as well and `0/16` seeds differ at `worst 0`. A comparison that cannot fail is the failure this row's sibling (#2839) is about, so proving these CAN fail is not ceremony. WHAT THIS DOES NOT DO. It does not clear #2906's red. That gate fails 69 of 128 token positions against a golden captured on a tree with the bf16 router, so the golden itself is downstream of the defect this fixes. #2906 needs a fresh `our_ids.npy` capture on a GPU against a tree carrying THIS change, which is why it lands after this one and not before: capturing first would freeze an artifact this merge immediately invalidates and cost a second GPU capture. The red lanes on the PR are the standing `main`-side baseline plus `agent-record`, which fails on `test_oracle_pin.test_metadata_and_runtime_strings_differ_on_the_pin` — a case `dd68e0650` DELETED on `main` when the pin was corrected to its source-build value. The branch was 98 commits behind and still carried it; this merge brings it forward, so the case is gone rather than silenced. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code]
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
THIS PULL REQUEST MAKES A GATE FAIL, AND THAT IS THE RESULT.
Glm4MoeLiteForCausalLMmatches the pinned vLLM 0.25.0 oracle at 69 of 128 positions and on 1 of 8
prompts. Nothing asserted that before, because the only vs-vLLM assertion in the
tree had no failure mode and the case that carried it reported a pass with zero
assertions on every host without a 31.2B checkpoint. Do not soften the new case
to merge this; the two admissible exits are named below.
What was true at
3e246b34fD1 — the only vs-vLLM assertion cannot fail. The gate tests
mn > kNearTieMnatswithkNearTieMnats = 500againsttests/parity/goldens/glm4_moe_lite_greedy/neartie_gap_mnats.npy, which is(8,16) int32,min 0 max 0.0 > 500is false at every position, soprompt_okis unconditionally true andCHECK_MESSAGE(prompt_ok, ...)is apositive control that cannot trip.
D2 — the licensed bar is STRICT, and nothing asserted it. The gate derives
vLLM's own self-determinism from
greedy_dist.npyover K=5, CHECKs that it iszero, prints
DETERMINISTIC -> the STRICT bar is well-posed— and then appliesthe near-tie band anyway.
CLAUDE.md§Gates permits a distributional gate "onlywhen the oracle's greedy decode is non-deterministic".
D3 — a skip wearing a pass. With no snapshot the body emitted
MESSAGE("SKIP ...")and returned. Measured on this host, before:A SACRED gate reporting SUCCESS off zero assertions. The same shape hid two more
early returns, and both of their conditions are impossible: all five artifacts
are committed.
The change
The checkpoint predicate is a
doctest::skip, so absence reads as skipped. Thetwo golden-absence returns are REQUIREs, because a committed artifact going
missing is a broken checkout and not an environment.
greedy_dist.npyisrequired, and its verdict selects the bar in code: zero multi-valued cells means
STRICT, and only a non-zero count admits the band — which now also applies the
second conjunct this file's header states and nothing implemented, our token
inside vLLM's top-K.
The third piece is what makes any of it observable here. Everything the SACRED
case compares is committed, including this engine's own ids, which that case
REQUIREs the live engine to reproduce token for token before comparing anything.
So a new case applies the licensed bar to those artifacts with no checkpoint and
no GPU, and runs wherever the suite runs.
GREEN-to-RED, both counts
First measured on
3e246b34f, then re-measured after rebasing onto3f12c617d,31 commits later: identical counts, identical numbers, so nothing here was
falsified by the merge.
Mutations
Both are artifact mutations on the goldens the case reads, each restored and each
restore verified by sha256 against
d9b2e31a…/77da140b…/65e3c580…/adfc42ab….greedy_dist.npyso the oracle reads non-deterministicour_ids.npywithgreedy_ids.npy(positive control)G2 is the one that matters: it proves the 69/128 is being read out of the
artifacts rather than asserted.
A finding recorded but deliberately NOT gated
The gap artifact is very likely not a measurement. At a prompt's FIRST divergent
position our prefix is identical to the oracle's, so vLLM's teacher-forced argmax
there IS
greedy_ids[i,j]by construction and our token is a different one, somax(0, arg_lp - our_lp)must be strictly positive. It is0at all seven firstdivergences (prompts 0..6 at positions 14, 6, 6, 4, 4, 4, 6), and the generating
script's
99_999_000outside-top-20 sentinel appears nowhere. Every divergenttoken of ours would have to be inside vLLM's top-20 AND within 0.0005 nats of its
argmax, seven times out of seven at the one position where the prefixes provably
agree.
This is not gated, because a "gap at a first divergence is positive" assertion
could in principle be defeated by genuine sub-millinat ties, and the STRICT bar
settles the same question without that risk.
What is NOT verified
The engine case is UNRUN. No
zai-org/GLM-4.7-Flashsnapshot exists on this hostand none is on the NAS, so nothing here can execute a 31.2B forward. It compiles
clean; it has not run.
docs/USAGE.mdalso carries no checkpoint row for thisarchitecture — no revision, no sha256 — which the spec records as owed rather
than inventing values off an artifact this host does not have.
The two admissible exits
Either a forward repair that makes the new case 128/128, or a re-capture of
neartie_gap_mnats.npyon the checkpoint plus an explicit ratification argumentfor a distributional bar over a deterministic oracle. Widening the assertion is
neither, and
docs/FEATURES.mdalready says so:the shipped 8/8 is not a measurement.The shape is repository-wide, and is now owned
D3 is one instance of
MESSAGE(...SKIP...); return;, which occurs at 228 sitesin 72 files under
tests/. Measured at3f12c617d,test_deepseek_v2_paged_engine-- the SACRED DeepSeek-V2 gate -- reports1 passed | 0 failed,assertions: 0,Status: SUCCESS!on this host. That isthe same defect, unfixed, on a different row. Repairing 228 sites is a campaign,
so it is filed as #2918 and
listed under this spec's
## OwedO4. This pull request repairs the one instanceit touches.
Closes #2839.
FOLLOWING_AGENTS_PROTOCOL
Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [claude-code]