Maybe use hyperparameter_utils.get_lr as default learning rate?

[Butanium]

Currently tinker finetuning uses a hardcoded 2e-4 lr:

llmcomp/llmcomp/finetuning/params.py

Line 56 in 33d906d

learning_rate: float = 2e-4

Tinker cookbook provides hyperparam_utils.get_lr(model_id, is_lora: bool), maybe we should use that as the default? @HarryMayne told me this is what they've been using in their current expeirments

[Butanium]

although it might throw NotImplementedError for some model so we could fallbnack to 2e-4 in this case wsitgh a warning

[johny-b]

Sounds great!!

Can you make a PR?