How many resources are needed to train the model

[ysngfo]

Hi, thanks for your great job. And I wonder how many resources are needed to train it? Specifically, for various different sizes, approximately how many GPU * hour are needed to train a model

[MTDoven]

This depends on multiple factors such as the number of parameters to generate and whether it is close-set or open-set task. For example, learning a set of ViT-Tiny parameters (close-set) requires approximately 7 GPU·hours; learning a set of ViT-Base parameters (close-set) requires 30 GPU·hours; and learning a set of ViT-Tiny parameters (open-set) requires approximately 40 GPU·hours. (All results are obtained from A100 with bfloat16)