test(gemma4): the checkpoint gates can only be produced for a checkpoint that fits beside its own reference on one card - #1050
Conversation
…environment Signed-off-by: Feathbow <feathbow@gmail.com>
Signed-off-by: Feathbow <feathbow@gmail.com>
Signed-off-by: Feathbow <feathbow@gmail.com>
Signed-off-by: Feathbow <feathbow@gmail.com>
Signed-off-by: Feathbow <feathbow@gmail.com>
FeathBow
changed the title
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ddf7f6514a
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| # The fixture set is named by the checkpoint it was dumped from; the | ||
| # committed set is 12b. Another tag selects fixtures dumped for another | ||
| # checkpoint under the same names, and the gates are told where they are. | ||
| FIXTURE_TAG=${PEGAINFER_GEMMA4_FIXTURE_TAG:-12b} |
There was a problem hiding this comment.
When PEGAINFER_GEMMA4_FIXTURE_TAG selects a non-12B set, the maintained docs still state that the gates are pinned to 12B, provide only 12B fixture names, and omit the multi-GPU --device auto dumping procedure (docs/models/gemma4/hf-golden.md and docs/models/gemma4/tokenizer.md). This leaves the newly supported workflow undocumented and directly contradicts the tokenizer guide's claim that other sizes cannot be gated; update those model docs with the tagged naming, sharded dump, and tagged chat-reference procedure.
AGENTS.md reference: AGENTS.md:L188-L190
Useful? React with 👍 / 👎.
|
@codex review please |
Signed-off-by: Feathbow <feathbow@gmail.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ddf7f6514a
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| case_cuts = cuts if probed else [] | ||
| hidden, logits = run_case(model, text_model, case_cuts, tokens, args.device) | ||
| hidden, logits = run_case( | ||
| model, text_model, case_cuts, tokens, input_device(model, args.device) |
There was a problem hiding this comment.
When --device auto actually distributes the language layers across multiple GPUs, the selected first/last-layer hooks retain their activations on those different CUDA devices. run_case() then passes them directly to the single torch.stack() at lines 210–212, which raises a cross-device tensor error before the base golden can be written. Move each captured activation to CPU (or another common device) before stacking so the newly supported sharded workflow works for checkpoints that cannot fit on one card.
Useful? React with 👍 / 👎.
Signed-off-by: Feathbow <feathbow@gmail.com>
|
@codex review thx |
|
Codex Review: Didn't find any major issues. Can't wait for the next one! Reviewed commit: ℹ️ About Codex in GitHubCodex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback". |
1 participant
Description
Closes #1049
The suite was 12B by construction. The checkpoint gates opened their four HF fixtures and the chat reference at fixed paths, and the dumpers loaded the reference tower on one device. Another checkpoint's fixtures had nowhere to live, and a tower that does not fit beside the long-context prompts on one card could not be dumped.
Paths from the environment, a set by tag. Each gate reads its fixture path from
PEGAINFER_GEMMA4_{GOLDEN,WINDOW_GOLDEN,LONGCTX_GOLDEN,GENERATE,CHAT_GOLDEN}, falling back to the committed 12B files.scripts/gemma4_gates.shtakesPEGAINFER_GEMMA4_FIXTURE_TAG(default12b; the committed chat reference keeps its untagged name), exports the five paths for that tag, and holds the set against the checkpoint's digests before the first load, as before.Dumped sharded when the tower needs it. The dumpers accept a device map that shards the reference tower over several GPUs; the golden dumper fingerprints a sharded checkpoint by its index and shard headers the way it fingerprints a single file; the chat parity reference follows the tag.
Test Env
Verification
Bound to the tip of this range.
google/gemma-4-31B-itat revision842da3794eaa(window and long-context cases with the tower sharded over two GH200s) and the whole suite run through the runner withPEGAINFER_GEMMA4_FIXTURE_TAG=31b: the fixture preflight agrees on the revision, and the suite runs to its end; the numeric results are retained in the M3 snapshot rather than here.cargo fmt --check,clippy -D warningson the gemma4 and frontend crates including tests.