Skip to content

fix(MODEL-TEXT-GLM4-MOE-LITE-GATE-2839): the near-tie predicate cannot fire, so apply the bar the oracle capture licenses -- and it FAILS 69/128 - #2906

Open
localai-org-maint-bot wants to merge 3 commits into
mainfrom
row/MODEL-TEXT-GLM4-MOE-LITE-GATE-2839
Open

fix(MODEL-TEXT-GLM4-MOE-LITE-GATE-2839): the near-tie predicate cannot fire, so apply the bar the oracle capture licenses -- and it FAILS 69/128#2906
localai-org-maint-bot wants to merge 3 commits into
mainfrom
row/MODEL-TEXT-GLM4-MOE-LITE-GATE-2839

Conversation

@localai-org-maint-bot

@localai-org-maint-bot localai-org-maint-bot commented Sep 4, 2026

Copy link
Copy Markdown
Collaborator

THIS PULL REQUEST MAKES A GATE FAIL, AND THAT IS THE RESULT. Glm4MoeLiteForCausalLM
matches the pinned vLLM 0.25.0 oracle at 69 of 128 positions and on 1 of 8
prompts. Nothing asserted that before, because the only vs-vLLM assertion in the
tree had no failure mode and the case that carried it reported a pass with zero
assertions on every host without a 31.2B checkpoint. Do not soften the new case
to merge this; the two admissible exits are named below.

What was true at 3e246b34f

D1 — the only vs-vLLM assertion cannot fail. The gate tests
mn > kNearTieMnats with kNearTieMnats = 500 against
tests/parity/goldens/glm4_moe_lite_greedy/neartie_gap_mnats.npy, which is
(8,16) int32, min 0 max 0. 0 > 500 is false at every position, so
prompt_ok is unconditionally true and CHECK_MESSAGE(prompt_ok, ...) is a
positive control that cannot trip.

D2 — the licensed bar is STRICT, and nothing asserted it. The gate derives
vLLM's own self-determinism from greedy_dist.npy over K=5, CHECKs that it is
zero, prints DETERMINISTIC -> the STRICT bar is well-posed — and then applies
the near-tie band anyway. CLAUDE.md §Gates permits a distributional gate "only
when the oracle's greedy decode is non-deterministic".

D3 — a skip wearing a pass. With no snapshot the body emitted
MESSAGE("SKIP ...") and returned. Measured on this host, before:

[doctest] test cases: 1 | 1 passed | 0 failed | 0 skipped
[doctest] assertions: 0 | 0 passed | 0 failed
[doctest] Status: SUCCESS!

A SACRED gate reporting SUCCESS off zero assertions. The same shape hid two more
early returns, and both of their conditions are impossible: all five artifacts
are committed.

The change

The checkpoint predicate is a doctest::skip, so absence reads as skipped. The
two golden-absence returns are REQUIREs, because a committed artifact going
missing is a broken checkout and not an environment. greedy_dist.npy is
required, and its verdict selects the bar in code: zero multi-valued cells means
STRICT, and only a non-zero count admits the band — which now also applies the
second conjunct this file's header states and nothing implemented, our token
inside vLLM's top-K.

The third piece is what makes any of it observable here. Everything the SACRED
case compares is committed, including this engine's own ids, which that case
REQUIREs the live engine to reproduce token for token before comparing anything.
So a new case applies the licensed bar to those artifacts with no checkpoint and
no GPU, and runs wherever the suite runs.

GREEN-to-RED, both counts

First measured on 3e246b34f, then re-measured after rebasing onto 3f12c617d,
31 commits later: identical counts, identical numbers, so nothing here was
falsified by the merge.

[doctest] test cases:  1 | 0 passed | 1 failed | 1 skipped
[doctest] assertions: 10 | 8 passed | 2 failed
MESSAGE: vLLM self-determinism over K=5 batch=1 runs — multi-valued cells = 0
MESSAGE: committed teacher-forced gap max = 0 mnats against a 500 mnat band,
         so `gap > band` has no failure mode on this artifact
MESSAGE: STRICT vs vLLM 0.25.0: positions 69/128, prompts exact 1/8,
         per-prompt mismatch [2,6,6,12,12,11,10,0]
ERROR:   CHECK( exact_positions == N * T ) -> CHECK( 69 == 128 )
ERROR:   CHECK( exact_prompts   == N )     -> CHECK( 1 == 8 )

Mutations

Both are artifact mutations on the goldens the case reads, each restored and each
restore verified by sha256 against
d9b2e31a…/77da140b…/65e3c580…/adfc42ab….

# Mutation Result
G1 perturb one cell of greedy_dist.npy so the oracle reads non-deterministic the bar-selection REQUIRE fires and aborts before the comparison: 1 failed, 3 assertions, 1 failed
G2 replace our_ids.npy with greedy_ids.npy (positive control) the case goes GREEN, 1 passed, 10 of 10 assertions — so the failure is data-driven, not an unconditional assertion

G2 is the one that matters: it proves the 69/128 is being read out of the
artifacts rather than asserted.

A finding recorded but deliberately NOT gated

The gap artifact is very likely not a measurement. At a prompt's FIRST divergent
position our prefix is identical to the oracle's, so vLLM's teacher-forced argmax
there IS greedy_ids[i,j] by construction and our token is a different one, so
max(0, arg_lp - our_lp) must be strictly positive. It is 0 at all seven first
divergences (prompts 0..6 at positions 14, 6, 6, 4, 4, 4, 6), and the generating
script's 99_999_000 outside-top-20 sentinel appears nowhere. Every divergent
token of ours would have to be inside vLLM's top-20 AND within 0.0005 nats of its
argmax, seven times out of seven at the one position where the prefixes provably
agree.

This is not gated, because a "gap at a first divergence is positive" assertion
could in principle be defeated by genuine sub-millinat ties, and the STRICT bar
settles the same question without that risk.

What is NOT verified

The engine case is UNRUN. No zai-org/GLM-4.7-Flash snapshot exists on this host
and none is on the NAS, so nothing here can execute a 31.2B forward. It compiles
clean; it has not run. docs/USAGE.md also carries no checkpoint row for this
architecture — no revision, no sha256 — which the spec records as owed rather
than inventing values off an artifact this host does not have.

The two admissible exits

Either a forward repair that makes the new case 128/128, or a re-capture of
neartie_gap_mnats.npy on the checkpoint plus an explicit ratification argument
for a distributional bar over a deterministic oracle. Widening the assertion is
neither, and docs/FEATURES.md already says so: the shipped 8/8 is not a measurement.

The shape is repository-wide, and is now owned

D3 is one instance of MESSAGE(...SKIP...); return;, which occurs at 228 sites
in 72 files
under tests/. Measured at 3f12c617d,
test_deepseek_v2_paged_engine -- the SACRED DeepSeek-V2 gate -- reports
1 passed | 0 failed, assertions: 0, Status: SUCCESS! on this host. That is
the same defect, unfixed, on a different row. Repairing 228 sites is a campaign,
so it is filed as #2918 and
listed under this spec's ## Owed O4. This pull request repairs the one instance
it touches.

Closes #2839.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [claude-code]

…te asserts a bar its own oracle capture does not license

Three defects, each measured at `3e246b34f`. The only vs-vLLM assertion reads
`neartie_gap_mnats.npy`, which is `min 0 max 0` at all 128 positions, against a
500-mnat band, so it has no failure mode. The gate computes vLLM's own
self-determinism, prints DETERMINISTIC, and then applies the distributional band
that `CLAUDE.md` §Gates licenses only on a NON-deterministic oracle; under the
bar that is licensed the committed ids match 69 of 128 positions and 1 of 8
prompts. And the case returns early when the checkpoint is absent, which doctest
reports as a passed case with zero assertions, so on every host without a 31.2B
snapshot the SACRED gate reported a pass having measured nothing.

The spec also records, without gating on it, why the gap artifact is probably not
a measurement at all: at a first divergence our prefix equals the oracle's, so
vLLM's teacher-forced argmax there is the golden token by construction and the
gap must be positive. It is 0 at all seven first divergences, with none of the
script's outside-top-20 sentinels present.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [claude-code]
…pture licenses, and gate the committed artifacts with no checkpoint

THIS CHANGE MAKES A GATE FAIL. That is the result, not a regression. Do not
soften the new case; the two admissible exits are named in the spec.

Three repairs. The checkpoint predicate moves into `doctest::skip`, so an absent
snapshot is reported as skipped instead of as a passed case with zero
assertions, and the two golden-absence early returns become REQUIREs because
those artifacts are committed. `greedy_dist.npy` becomes required, and its
verdict now selects the bar in code rather than in a MESSAGE: zero multi-valued
cells means STRICT token-exact, and only a non-zero count admits the near-tie
band, which now also applies the second conjunct this file's own header states
and nothing implemented -- our token inside vLLM's top-K.

The third repair is what makes any of it observable. Everything the SACRED case
compares is committed: the oracle's ids, the oracle's K-run distribution, and
this engine's own ids, which the SACRED case already REQUIREs the live engine to
reproduce token for token before comparing anything. So a new case applies the
licensed bar to those artifacts with no 31.2B checkpoint and no GPU, and runs
wherever the suite runs.

Before, on this host: 1 test case, 1 PASSED, 0 failed, 0 skipped, and 0
assertions. After: 1 skipped, 1 failed, 10 assertions with 2 failed --
`positions 69/128, prompts exact 1/8, per-prompt mismatch [2,6,6,12,12,11,10,0]`,
and `committed teacher-forced gap max = 0 mnats against a 500 mnat band`.

Two artifact mutations, both restored and verified by sha256. Replacing
`our_ids.npy` with `greedy_ids.npy` turns the case green at 10 of 10, so the
failure is data-driven rather than an unconditional assertion. Perturbing one
cell of `greedy_dist.npy` makes the bar-selection REQUIRE fire and abort before
the comparison, so the bar really is chosen by the oracle's own capture.

The engine case itself is UNRUN: no `zai-org/GLM-4.7-Flash` snapshot exists on
this host and none is on the NAS. It compiles; nothing here can execute it.

Closes #2839.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [claude-code]
@mudler
mudler force-pushed the row/MODEL-TEXT-GLM4-MOE-LITE-GATE-2839 branch from 86b9729 to f3f8f71 Compare September 4, 2026 21:16
…o-assertion skip that hid this defect

This row's D3 is one instance of a shape that occurs at 228 sites in 72 files:
`MESSAGE(...SKIP...); return;` inside a TEST_CASE, which doctest counts as a
passed case with zero assertions. Measured at `3f12c617d`,
`test_deepseek_v2_paged_engine` -- the SACRED DeepSeek-V2 gate -- reports
`1 passed | 0 failed`, `assertions: 0`, `Status: SUCCESS!` on a host with no
checkpoints.

Repairing 228 sites is a campaign rather than an in-flow fix, so it is filed as
#2918 and listed here under `## Owed` O4, which is the ownership `CLAUDE.md`
requires for an issue a change references without closing.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [claude-code]
mudler added a commit that referenced this pull request Sep 5, 2026
…r is fp32 because the class says so

`Glm4MoeLite` IS `Glm4MoE` (`glm4_moe_lite.py:86-87`), whose gate is
`nn.Linear(..., dtype=torch.float32)` fed `hidden_states.to(torch.float32)` and
declared `router_logits_dtype=torch.float32` (`glm4_moe.py:141-146`, `:218`,
`:205`). fp32 is a property of the CLASS, so no config key can withhold it — and
the published GLM-4.7-Flash `config.json` ships no `moe_router_dtype`, which is
exactly the value `ParseDeepseekV2Params` correctly reads for the architecture
IT serves. We were inheriting DeepSeek-V2's answer for a GLM question.

WHY THIS IS A CORRECTNESS FIX AND NOT A PRECISION PREFERENCE. Rounding the
router logits to bf16 in front of a top-4-of-64 `noaux_tc` selection changes
WHICH EXPERTS RUN. A selection gate has bimodal error, not a tolerance: the
top-k either picks the same set or a different one, and when it flips, the
output is not slightly wrong. The measured effect is exactly that shape — the
dtype is observable in the GLM forward at 6 of 16 fixed seeds, worst
|f32 - bf16| logit 0.00100262, which is small as a number and decisive as a
tie-break.

The reachability is proven from production rather than from a constructed
object: `ParseGlm4MoeLiteParams` sets `p.router_dtype_is_f32 = true`
(`glm4_moe_lite_registry.cpp:75`), it is consumed at `deepseek_v2.cpp:363` on
the logit store before `vt::MoeRouterTopK`, and it is reached from
`LoadGlm4MoeLiteForCausalLM` (`:126`) behind `REGISTER_VLLM_MODEL(glm4_moe_lite, ...)`.

The mutations are the part that makes the tests worth having. M1 removes the
one line and the gate goes RED on three cases (`assertions: 23 | 20 passed |
3 failed`) — so the flag is load-bearing. M3 and M4 attack the tests' own
vacuity from the other side: M3 removes the bf16 round trip T3 rests on and the
SET-difference count falls to `0/4096`, and M4 makes T2's second arm f32 as well
and `0/16` seeds differ at `worst 0`. A comparison that cannot fail is the
failure this row's sibling (#2839) is about, so proving these CAN fail is not
ceremony.

WHAT THIS DOES NOT DO. It does not clear #2906's red. That gate fails 69 of 128
token positions against a golden captured on a tree with the bf16 router, so the
golden itself is downstream of the defect this fixes. #2906 needs a fresh
`our_ids.npy` capture on a GPU against a tree carrying THIS change, which is why
it lands after this one and not before: capturing first would freeze an artifact
this merge immediately invalidates and cost a second GPU capture.

The red lanes on the PR are the standing `main`-side baseline plus `agent-record`,
which fails on `test_oracle_pin.test_metadata_and_runtime_strings_differ_on_the_pin`
— a case `dd68e0650` DELETED on `main` when the pin was corrected to its
source-build value. The branch was 98 commits behind and still carried it; this
merge brings it forward, so the case is gone rather than silenced.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5-1m [claude-code]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Glm4MoeLite's only vs-vLLM assertion cannot fail: the gap artifact is identically zero and the goldens match 69/128

2 participants