RTX30xx FP8 feather matmul triton kernel?

[phazei]

I saw this was released https://github.com/SuriyaaMM/feather not too long ago.

I was wondering, is that something that would ideally be implemented here? Would it basically speed up all fp8 operations presuming the operation was completed long before the memory was copied from VRAM to the registers? Seems like it's in line with the kernels here.

[Ph0rk0z]

That kernel doesn't work but it would be REALLY REALLY nice is comfy were to let us old gpu peasants have a fallback path that allows casting to FP16 or patched triton for compiles. Maybe even borrow from https://github.com/silveroxides/ComfyUI-QuantOps and support int8 weights.

[silveroxides]

@Ph0rk0z Thanks for the mention. That int8 implementation is actually from a PR on ComfyUI repo. Comfy-Org/ComfyUI#10864

Just thought I would mention that. Have you tried any int8 models with my custom node? How was the inference speed?

[Ph0rk0z]

Tried chroma, self quatized. Speed in triton mode is decent but it fails to compile on 2080 and compiles then crashes second gen on 3090. I do need torch.compile to work though, because that pushes me into it/s territory from s/it.

I also did T5XXL and int8 is good. I am using it over FP8. Unfortunately in triton mode (where there is noticeable speedup) produces NaN in the embedding causing sprinkled static image.

Honestly would prefer to be able to change all my models from FP8 -> int8. The fidelity is like GGUF without the lora slowdown. In torch 2.9, I can only compile plain FP8 E4 so miss out on the better scaled quantizations.The turning card ends up being the image-gen GPU while 3090s mainly run some LLM and have had to try everything under the sun to make it fast.

[woct0rdho]

I've tried to make it run fully on GPU and add it as ComfyUI ops, see https://github.com/woct0rdho/ComfyUI-FeatherOps , and the discussion in SuriyaaMM/feather#1 . It surely speedups GEMV, but in my tests it cannot speedup large GEMM in Z-Image and larger models.

[monnky]

I use RTX 3060 Nvidia Ampere //
why triton always disabled ?? how to enable ?? @comfyanonymous

Found comfy_kitchen backend triton: {
'available': True,
'disabled': True,
'unavailable_reason': None,

[silveroxides]

I use RTX 3060 Nvidia Ampere // why triton always diabled ?? how to enable ?? @comfyanonymous

Found comfy_kitchen backend triton: { 'available': True, 'disabled': True, 'unavailable_reason': None,

Merge this PR to your local installation and set the launch argument --enable-triton-backend

But you only get any use from it if you use a comfy-kitchen that actually has triton backend implementation for supported matmul.

Comfy-Org/ComfyUI#12730