Add Gaussian Processes as a Model

[thomasATbayer]

Feature Request: Gaussian Process Support via GPyTorch and skorch

Summary

Add native Gaussian Process (GP) support to MotherML by integrating GPyTorch through its skorch wrapper. This would give users access to principled uncertainty quantification, scalable GP inference, and a rich set of kernel choices — all within the existing Mother pipeline and optimizer framework.

---

Motivation

Gaussian Processes are a principled and well-established approach to uncertainty quantification in regression and classification. They are particularly relevant in the domains MotherML targets (chemistry, biology, drug discovery) where:

• Calibrated uncertainty estimates matter more than raw predictive accuracy alone.
• Small-to-medium dataset sizes are common, where GPs are competitive with or outperform deep ensembles and conformal methods.
• Active learning and Bayesian optimisation workflows depend on well-calibrated posterior distributions, not just point predictions with heuristic confidence intervals.
• Distribution-level outputs (mean + variance, or full posterior samples) are needed for downstream decision-making.

Currently, MotherML supports uncertainty quantification through conformal prediction and ensemble-based approaches. GPs would add a complementary method grounded in Bayesian theory, with exact or approximate posterior inference rather than post-hoc calibration.

---

Proposed Integration

GPyTorch via skorch

The recommended integration path is through skorch's GPyTorch wrapper (skorch.probabilistic), which provides:

• A scikit-learn–compatible fit / predict interface over GPyTorch models, making GPs drop-in compatible with the existing Mother pipeline.
• Support for exact GP inference (small datasets) and scalable approximate inference via inducing points (larger datasets).
• Access to GPyTorch's full kernel library (RBF, Matérn, spectral mixture, deep kernels, etc.) and likelihood types (Gaussian, Bernoulli, etc.).
• Native PyTorch training loop, meaning early stopping, learning rate scheduling, and GPU acceleration work out of the box.

Because skorch GPs follow the sklearn estimator interface, they would slot into MotherPipeline as any other model, with minimal adapter code needed on the Mother side.

---

Scope

Models to support initially

• Exact GP regressor — for small-to-medium datasets with a Gaussian likelihood.
• Approximate / sparse GP regressor — for larger datasets using inducing point methods (e.g. SVGP).
• GP classifier — binary and potentially multiclass, using a Bernoulli or Dirichlet likelihood.

Mother integration points

• Register GP models in the Mother model registry so they are discoverable via the standard MotherSettings / config interface.
• Implement get_hyperparameter_space() for GP models so MotherTuner can optimise kernel parameters, noise levels, inducing point counts, and learning rate automatically.
• Expose predict_distribution() (or equivalent) so GP posterior outputs are accessible through Mother's uncertainty interface, consistent with other uncertainty-aware models.
• Ensure compatibility with the existing CV and tuning infrastructure — GP training via skorch is a standard PyTorch training loop, so the custom objective / CV scorer hooks proposed in the companion feature request would be directly applicable here.

Out of scope (for now)

• Deep kernel learning (DKL) — combining neural networks with GP outputs. This is a natural follow-on but adds significant complexity.
• Multi-output / multi-task GPs.
• Custom inducing point initialisation strategies.

---

Dependencies

Package	Role
gpytorch	GP inference engine, kernels, likelihoods
skorch	sklearn-compatible wrapper for GPyTorch models
torch	Already a soft dependency for existing CatBoost early stopping support

Both gpytorch and skorch are well-maintained, widely used in academic and industry settings, and have compatible licenses (MIT).

---

Affected Areas

Area	Change
src/mother/ml/models/	Add GP model classes (exact GP, sparse GP, GP classifier) wrapping skorch
src/mother/ml/core.py	Register new GP models in the model registry
pyproject.toml	Add gpytorch and skorch as optional dependencies under an extras group (e.g. [deep] or [gp])
src/mother/settings.py	Expose GP model options through the settings surface
examples/	Add a notebook demonstrating GP regression with uncertainty estimates
mkdocs/docs/	Document GP models, their parameters, and when to use them

---

Related

• GPyTorch documentation: https://gpytorch.ai
• skorch probabilistic module: https://skorch.readthedocs.io/en/stable/probabilistic.html
• Companion feature request: Custom CV Scorer & Custom Optuna Objective (the custom objective hook is directly useful for GP training with early stopping)