Pre-trained models for Bert Objective?

[skull3r7]

I have tried to use the pre-trained models (s-GPT) with the Bert objective. However, this only generates noise.

Are there extra pre-trained models that were trained on the bert task?

I could not find anything in the download.py, bert is not mentioned there.

[chenmarkGH]

I just updated download.py to allow downloading bert models: c6af2eb. Please remember to set the hparams.bert flag!