[Question] the learning rate scheduler may not reach zero, right?

[soskek]

The implementation doesn't seem to follow the description in the paper:

Learning rate decays linearly from initial value to zero

When running the LT_glue.py command shown in README, the script sets --num_train_epochs as 30 and it means training each pruned subnetwork takes 3 epochs (and the [0%, 10%, ...90%]-pruned subnets take 30 epochs in total).
However, the learning rate scheduler depends on the --num_train_epochs (30) rather than 3:

t_total = len(train_dataloader) // args.gradient_accumulation_steps * args.num_train_epochs
...
scheduler = get_linear_schedule_with_warmup(
        optimizer, num_warmup_steps=args.warmup_steps, num_training_steps=t_total
)

So, the last learning rate after 3 epochs is 1.8e-5 (= 2e-5 * (1 - 3/30)) (rather than zero) if we start from 2e-5.
Is my understanding correct?

Sure this minor difference doesn't hurt the great contributions of the work. I just wanna confirm it and know the detail for reproduction and my own further research. Thanks!

[soskek]

@Tianlong-Chen when you have time, could you please take a look?