Hosting the quantized models somewhere publicly accessible would be helpful for new users getting started quickly or for users with less powerful hardware. If you already have quantized models to upload that would be great, otherwise I can do the quants myself and then look into putting them on HuggingFace.
Hosting the quantized models somewhere publicly accessible would be helpful for new users getting started quickly or for users with less powerful hardware. If you already have quantized models to upload that would be great, otherwise I can do the quants myself and then look into putting them on HuggingFace.