Multi-GPU for nvidia

[vacekj]

Any interest in implementing multi-gpu CUDA setups? I have a 4x3090 + 128GB RAM I would love to use with these quants, since other quants are currently lacking in accuracy or just straight up don't exist.

[antirez]

Hi, general interest, yes. Ability to help, no because of lack of the hardware. Interested in trying this with Metal: yes, once new hardware arrives (a couple of weeks hopefully). So I believe would be great to know the results you get and decide if we can merge it. Thanks.

[d3y4n]

Possibly related #108, it's practically unusable (or with small context for 1 user) on RTX6K, any ideas?