On our qusi-Newton methods like SR1 and BFGS, we start with an initial Hessian approximation of the identity. Most of the time, this works ok, but on badly scaled problems, I'm unhappy with the performance. Basically, if the gradient is really, really small the truncated Krylov method will take a full steepest descent step, which is also really, really small. That means we make no progress. Now, it may be that the next step performs better because now we have Hessian information, but I need to check this and then determine whether or not we'd be better off just scaling the initial identity from the start.
On our qusi-Newton methods like SR1 and BFGS, we start with an initial Hessian approximation of the identity. Most of the time, this works ok, but on badly scaled problems, I'm unhappy with the performance. Basically, if the gradient is really, really small the truncated Krylov method will take a full steepest descent step, which is also really, really small. That means we make no progress. Now, it may be that the next step performs better because now we have Hessian information, but I need to check this and then determine whether or not we'd be better off just scaling the initial identity from the start.