Training Script for ViT-S/Tiny/TinyPlus Depth Estimation Distillation from ViT-L

[mrpositron]

What has changed and why?

This PR adds a depth estimation training path to LightlyTrain that fine-tunes Depth Anything V3 student models by distilling depth and sky pseudo-labels from a larger V3 ViT-L teacher into smaller, faster backbones. It exposes a new public entrypoint lightly_train.train_depth_estimation(...) and supports both relative-depth and metric-depth models:

• Relative depth — e.g. dinov3/dav3-relative-tiny, dinov3/dav3-relative-tiny-plus, dinov2/dav3-relative-small.
• Metric depth — e.g. dinov3/dav3-metric-tiny, dinov3/dav3-metric-tiny-plus, dinov2/dav3-metric-small. A metric model is typically fine-tuned from a trained relative checkpoint (matching architectures, so weights transfer cleanly), with the pseudo-labels stored in the canonical-camera depth space.

What's included:

• New public API: train_depth_estimation added to lightly_train.__init__ and wired through train_task.py / train_task_helpers.py.
• Distillation criterion (criterion.py):
  • SILogLoss — scale-invariant log depth loss (the depth term).
  • GradientMatchingLoss — multi-scale gradient matching (L_grad).
  • SkyDistillLoss — BCE on the sigmoid sky head against the teacher's soft sky map (L_sky).
  • RelativeL1Loss — scale-aware AbsRel term used only for metric depth to pin the absolute scale to the teacher (it does not vanish under global-scale error, unlike the scale-invariant term).
  • FeatureAlignmentLoss — feature-space distillation of the teacher's DPT-input patch tokens via per-stage learnable projections + mean cosine distance (training-only, discarded at export).
• Data pipeline — DepthEstimationDataArgs / dataset (depth_estimation_dataset.py) reading RGB images plus .npy depth and sky pseudo-labels matched by filename stem (depth <= 0 treated as invalid), and depth-specific transforms.
• Metrics (task_metric.py) — AbsRel, RMSE, and delta1 over valid pixels, computed after per-image least-squares scale-and-shift alignment (the standard MiDaS/DAv3 relative-depth protocol). Default watch_metric is val_metric/abs_rel.
• Visualization (depth_estimation.py) — logs predicted depth and sky-mask overlays to TensorBoard.
• Train model (train_model.py) — orchestrates the online teacher, loss weighting, and student updates.

How has it been tested?

• Added unit tests covering every new component:
  • Criterion: test_criterion.py
  • Train model & task model: test_train_model.py, test_task_model.py
  • Data & transforms: test_depth_estimation_dataset.py, test_depth_estimation_transform.py
  • Metrics: test_task_metric.py
  • Visualization: test_depth_estimation.py
  • Command wiring: test_train_task.py

Did you update CHANGELOG.md?

• Yes
• Not needed (internal change)

Did you update the documentation?

• Yes
• Not needed (internal change without effects for user)