About learning rate

[GewelsJI]

Hi Jiwei!

Why you try a little bit more bigger learning rate in your training phase?

Or

You try some bigger lr like lr = 3e-4 or lr = 1e-6, can you suggest some useful experiential value?

[jiwei0921]

Hello,

Actually, I haven't done some experiments about learning rate. But I suggest you can set the lower lr at the beginning, because the higher lr may lead to less convergence.

However, choosing a suitable lr requires a lot of experiments, which is very time-consuming. So I think you can refer to some great baseline, such as ResNet, VGGNet.

I'm sorry I don't help you much.