Because the tokenizer in the metadata is wrong. My solution to this was to convert from safetensors myself using the latest llama.cpp, but the docs should be fixed. There are models that do work out of the box on HF.
Issue is it has to be a recent conversion since GGUF format changes every 5 minutes. I will probably upload my conversion but with 30mbps cable internet it's going to be a while.
Because the tokenizer in the metadata is wrong. My solution to this was to convert from safetensors myself using the latest llama.cpp, but the docs should be fixed. There are models that do work out of the box on HF.
Issue is it has to be a recent conversion since GGUF format changes every 5 minutes. I will probably upload my conversion but with 30mbps cable internet it's going to be a while.