Fix gemma3 token vocab mismatch #128

Open

Saibo-creator wants to merge 1 commit into

mainfrom

Commits on Apr 9, 2025

Fix gemma3 token vocab mismatch by using tokenizer.vocab_size when available
Saibo-creator
committed