Please add gguf‑quantized support for "Ideogram-4"

[makisekurisu-jp]

diffusion_models:
https://huggingface.co/rectangleworm/ideogram-4-gguf/tree/main/diffusion/cond
https://huggingface.co/rectangleworm/ideogram-4-gguf/tree/main/diffusion/uncond

text_encoders:
https://huggingface.co/unsloth/Qwen3-VL-8B-Instruct-GGUF

[juanml82]

I got a gguf working, but I think there is a problem with the mmproj file (which isn't needed for ideogram 4)
I'm using this one
https://huggingface.co/mradermacher/Qwen-3-VL-8B-Instruct-heretic-GGUF With these names:
Qwen-3-VL-8B-Instruct-heretic.Q8_0.gguf
Qwen-3-VL-8B-Instruct-heretic.Q8_0.mmproj-f16.gguf

If I try to use comfyui's native "Generate text" node, vision doesn't work (but it does in llama.cpp). OTOH, that may be an issue with comfyui, as the vision on the model comfyui team uploaded to huggingface doesn't work either

[fxd0h]

For anyone arriving here trying to run Ideogram-4 via GGUF: the diffusion-model side, plus an MPS-specific dtype problem that breaks generation, are addressed in #454 (PRs #455 and #456). A public fork with both branches is available for immediate use: https://github.com/fxd0h/ComfyUI-GGUF

That covers the UNet. For the text encoder this issue is about, the official qwen3vl_8b_fp8_scaled safetensors (Comfy-Org) loads through the standard CLIPLoader and is enough to run Ideogram-4 end to end, so a GGUF text encoder is a convenience here rather than a hard requirement. The GGUF text-encoder path this issue requests remains open.

Verified end to end on Apple Silicon (M-series, MPS, torch 2.12).

Developed with AI assistance; verified on-device before posting.

[frankyifei]

For anyone arriving here trying to run Ideogram-4 via GGUF: the diffusion-model side, plus an MPS-specific dtype problem that breaks generation, are addressed in #454 (PRs #455 and #456). A public fork with both branches is available for immediate use: https://github.com/fxd0h/ComfyUI-GGUF

That covers the UNet. For the text encoder this issue is about, the official qwen3vl_8b_fp8_scaled safetensors (Comfy-Org) loads through the standard CLIPLoader and is enough to run Ideogram-4 end to end, so a GGUF text encoder is a convenience here rather than a hard requirement. The GGUF text-encoder path this issue requests remains open.

Verified end to end on Apple Silicon (M-series, MPS, torch 2.12).

Developed with AI assistance; verified on-device before posting.

loading qwen3vl_8b as gguf would end up with a model type of Flux2TEModel_ , using qwen3vl_8b_fp8_scaled shows BooguTEModel_ in log, so the problem may not be in GGUF node side, because I tried to use https://github.com/calcuis/gguf and it gave a same result

[m8rr]

@frankyifei Add to the very bottom of loader.py.
Although vision functionality won't work, it will at least properly display as BooguTEModel and run without issues.
m8rr@a963418

# add qwen3vl part

        if arch == "qwen2vl":
            vsd = gguf_mmproj_loader(path)
            sd.update(vsd)
        if arch == "qwen3vl":
            sd["model.visual.deepstack_merger_list.0.norm.weight"] = torch.zeros(4608)
            sd["model.visual.merger.linear_fc2.weight"] = torch.zeros(4096)

[m8rr]

m8rr@249624c
Add Qwen3-VL vision support; use with mmproj file.

Tested using the GGUF files below.
https://huggingface.co/mradermacher/Qwen3-VL-8B-Instruct-GGUF/blob/main/Qwen3-VL-8B-Instruct.mmproj-Q8_0.gguf
https://huggingface.co/unsloth/Qwen3-VL-8B-Instruct-GGUF/blob/main/Qwen3-VL-8B-Instruct-Q5_K_M.gguf

[m8rr]

m8rr@c7fb703
Added Qwen3VL-4B

[makisekurisu-jp]

From my repeated testing, a machine with 12GB VRAM and 16GB RAM can run the FP8 diffusion model provided by Comfy Org together with the Q4_K_M GGUF text‑encoder model. However, if the diffusion model is replaced with a GGUF‑quantized version, the image quality drops no matter what.

[m8rr]

https://huggingface.co/rectangleworm/ideogram-4-gguf/blob/main/diffusion/cond/ideogram4-Q4_K.gguf
I used Q4K for this. The results are in order: INT8, FP8, Q4K. They’re slightly different, and the GGUF version feels a bit darker, but there isn't a huge difference overall. Q4K actually got the gloves right in PIC1. As for FP8 in PIC3, the text is a mess. I haven't used ideogram4 much, so I don't know well...

INT8 / FP8 / Q4K

[makisekurisu-jp]

https://huggingface.co/rectangleworm/ideogram-4-gguf/blob/main/diffusion/cond/ideogram4-Q4_K.gguf I used Q4K for this. The results are in order: INT8, FP8, Q4K. They’re slightly different, and the GGUF version feels a bit darker, but there isn't a huge difference overall. Q4K actually got the gloves right in PIC1. As for FP8 in PIC3, the text is a mess. I haven't used ideogram4 much, so I don't know well...

INT8 / FP8 / Q4K

Sorry — my conclusion was referring specifically to the unconditional model. During testing I was always using the ideogram4_fp8_scaled main model and did not switch to any other quantized models. So I need to clarify that point.