Choosing the best reconstruction

[Obi-Wan]

Description

Currently, the returned reconstructions from the Solver classes are the latest iterates. This is not ideal, because reconstructions might diverge, and we might want to select reconstructions through other metrics, e.g., reconstruction residual or cross-validation. This PR changes the way that solvers return the best reconstruction, and track the best iterates with respect to reconstruction loss and cross-validation loss. This behavior is modified through a switch called criterion, which is added to the algorithm initialization.

TODO

• Implement new feature:
  • Add support in the solver classes for selection criterion
  • Add better metrics tracking in the SolutionInfo class
  • Move parts of the code that might depend on returned solutions to consider the newer metrics
• Add or update unittests.
• Add or update docstrings.
• Add or update documentation.

Notes

This feature will be used by the denoise method to provide a better reconstruction out of the box.

[Obi-Wan]

This seems to not cause any regression, and the returned reconstructions seem to be correct. We'll close and merge now.