Dear author, I have a question about the datatype saving.
In your released model, the weights are fp32, for example "Model.layers.7.feed_forward.w1.Plora_A.weight", data type is torch.float32, and the shape is ([256, 4096]). After it is loaded by BitsAndBytesConfig in 4bit, it becomes torch.uint8 with shape ([524288, 1]). And I think it is trained on the latter format.
The question is , after training, how did you recover the model weights into fp32 with original shape?
Dear author, I have a question about the datatype saving.
In your released model, the weights are fp32, for example "Model.layers.7.feed_forward.w1.Plora_A.weight", data type is torch.float32, and the shape is ([256, 4096]). After it is loaded by BitsAndBytesConfig in 4bit, it becomes torch.uint8 with shape ([524288, 1]). And I think it is trained on the latter format.
The question is , after training, how did you recover the model weights into fp32 with original shape?