Would be great to get expected pretraining results (e.g., loss, next-sentence-prediction accuracy, etc.) and learning curves using the 64 * V100 training.
I found the fine-tuning task results on the blog but would be nice to compare the pretraining results as a "sanity check."
I don't have access to V100 GPUs for now so I'm using P40 GPUs to replicate/validate the pretraining pipeline.
@maxluk @aashnamsft any pretraining details that you guys can share? Even a screenshot of the AzureML metrics page would be helpful to compare and double-check.
Would be great to get expected pretraining results (e.g., loss, next-sentence-prediction accuracy, etc.) and learning curves using the 64 * V100 training.
I found the fine-tuning task results on the blog but would be nice to compare the pretraining results as a "sanity check."
I don't have access to V100 GPUs for now so I'm using P40 GPUs to replicate/validate the pretraining pipeline.
@maxluk @aashnamsft any pretraining details that you guys can share? Even a screenshot of the AzureML metrics page would be helpful to compare and double-check.