Enable Ranking Loss Function Tuning

[thomasATbayer]

Feature Request: Enable Ranking Loss Function Tuning in CatboostRankerMother

Summary

Extend CatboostRankerMother to include the ranking loss function as part of the Optuna hyperparameter search space, mirroring the loss-function tuning already available for regression and classification CatBoost models. This also requires exposing additional ranker-specific controls (tune_pairwise_type, top, max_pairs) and adding proper pickling, predict, and uncertainty estimation support.

---

Motivation

The existing CatboostRankerMother has a minimal default_parameters() implementation and no suggested_params_loss() method. The loss function is fixed at initialisation and never varied during Optuna optimisation. This is inconsistent with how the regression model works — where Optuna already searches over RMSE, MAE, LogCosh, and Tweedie — and it leaves significant performance on the table, because the optimal ranking loss is highly dataset-dependent:

• YetiRank / YetiRankPairwise — listwise, gradient-based approximation of ranking metrics; works well when the full ranked list quality matters.
• PairLogit / PairLogitPairwise — pairwise loss; often better when relative order between pairs is the primary signal.
• QueryRMSE — regression-style loss over relevance scores within a query group; good when targets are continuous relevance labels.
• QuerySoftMax — softmax classification within a query; suited for binary relevance (click/no-click style) targets.

Additionally, ranking loss functions in CatBoost are composite strings that encode several sub-options (mode, DCG denominator type, DCG exponent type, top-k cutoff). These cannot be expressed as a single categorical hyperparameter and require dedicated logic to compose and decompose them correctly during tuning.

---

Proposed Changes

1. New constructor parameters

Add the following to CatboostRankerMother.__init__:

• tune_pairwise_type (bool, default False) — whether to include pairwise loss variants (YetiRankPairwise, PairLogitPairwise) in the search space. Pairwise losses require SymmetricTree + Plain boosting and are therefore only valid when tune_tree_structure_type and tune_boosting_type are both False. The constructor should enforce this constraint and emit a clear warning if the combination is invalid.
• top (int, default 0) — when set, restricts the ranking metric to top-k evaluation. A non-zero value narrows the available loss functions to those that support the top parameter (primarily YetiRank).
• max_pairs (int | None, default None) — maximum number of pairs per query for pairwise losses. Passed directly into the composite loss string when applicable.

2. suggested_params_loss() method

Implement a suggested_params_loss() override that:

• Composes a valid CatBoost ranking loss string from individually sampled building blocks (base_loss, mode, dcg_denominator, dcg_type).
• Adapts the candidate loss list based on runtime context: whether targets are binary, whether the current trial's grow_policy and boosting_type are pairwise-compatible, and whether top is set.
• Removes the building-block parameters from suggested_params before returning, so only the final loss_function string is passed to CatBoost (consistent with how the Focal loss is handled for classifiers).
• Returns suggested_params unchanged for multi-target scenarios.

3. Updated default_parameters()

Update default_parameters() to include the loss building-block keys (base_loss, mode, dcg_denominator, dcg_type) used by the tuning logic to enqueue the first trial with sensible defaults, consistent with how other CatBoost models enqueue their defaults. When top is set, default to NDCG mode with standard DCG settings; otherwise default to Classic mode.

4. Pickling support (set_params, __getstate__, __setstate__)

The new constructor parameters (tune_pairwise_type, top, max_pairs) are custom attributes not managed by CatBoost's own serialisation. Add overrides for set_params, __getstate__, and __setstate__ to ensure these attributes survive cloning (used heavily by Optuna's trial loop) and pickling (used for parallel CV with n_jobs > 1).

5. predict() with optional rank output

Add a ranks flag to predict() that converts raw CatBoost scores into 1-based integer ranks within a query group. Rank 1 corresponds to the highest-scoring (best) document. This makes the model's output directly interpretable without requiring users to manually invert scores.

6. predict_uncertainty() for rank stability estimation

Add a predict_uncertainty() method that uses staged predictions (iterating over the boosting rounds) to estimate rank stability. For each stage it converts scores to ranks and then summarises the distribution across stages as: mean rank, standard deviation, geometric mean rank, min rank, max rank, and IQR. This provides a lightweight uncertainty estimate for ranking without requiring ensembles or conformal methods.

---

Constraints and Edge Cases to Handle

• Pairwise losses (YetiRankPairwise, PairLogitPairwise) require SymmetricTree grow policy and Plain boosting type. The implementation must check the actual trial values at runtime, not just the tuning flags, because even when tune_tree_structure_type=False, the default grow policy may or may not be SymmetricTree.
• When top > 0, only YetiRank (and conditionally YetiRankPairwise) support the top parameter. All other loss functions must be excluded from the search space in this case.
• The max_pairs parameter is only valid for PairLogit and PairLogitPairwise losses and must only be appended to the loss string when those losses are selected.
• Binary targets (labels are exclusively 0 and 1) unlock QuerySoftMax and the MAP mode for YetiRank; these should not appear in the search space for continuous relevance targets.

---

Affected Areas

Area	Change
src/mother/ml/models/m_catboost.py	Extend CatboostRankerMother with new parameters, suggested_params_loss, updated default_parameters, pickling support, and predict methods
test/unit/test_catboost_reg_uncertainty.py or new test file	Add tests for ranking loss tuning, pickling round-trip, rank prediction, and uncertainty output
examples/	Extend or add a ranking notebook demonstrating loss tuning
mkdocs/docs/	Document the new parameters and uncertainty output

---

Related

• Existing suggested_params_loss for regression: _CatboostHyperParams in m_catboost.py
• CatBoost ranking loss documentation: https://catboost.ai/en/docs/references/loss-functions-for-ranking
• Issue Enable Ranking Loss Function Tuning #33 (original): no description, context provided via code comment

[thomasATbayer]

CatBoostRankerMother — Uncertainty & Ranking Improvements

Summary

This PR extends CatboostRankerMother and _CatboostHyperParams in src/mother/ml/models/m_catboost.py with proper uncertainty estimation support and more flexible hyperparameter tuning, aligning the ranker with the other CatBoost Mother model wrappers.

---

Changes

_CatboostHyperParams — new tune_loss_function parameter

def __init__(self, ..., tune_loss_function: bool = True)

• When False, the loss function set on the model at construction time is kept fixed throughout tuning — suggested_params_loss is not called.
• Useful when the desired loss is already known and only tree structure / boosting parameters should be optimised.
• Default is True — no change to existing behaviour.

---

New helper: scores_to_ranks

Added a module-level utility that converts raw CatBoost scores to 1-based integer ranks using descending order (rank 1 = highest score), consistent with CatBoost's ranking convention.

scores_to_ranks(scores: np.ndarray) -> np.ndarray

---

predict — new normalize_by_group_size parameter

def predict(self, X, ..., ranks: bool = False, normalize_by_group_size: bool = False)

• When ranks=True and normalize_by_group_size=True, ranks are divided by len(X), scaling values to (0, 1].
• Makes rank values comparable across groups of different sizes.
• Default is False — no change to existing behaviour.

---

predict_uncertainty — full rewrite

Aligned with the standard Mother uncertainty schema used by CatboostRegressorMother and CatboostClassifierMother.

Signature:

def predict_uncertainty(
    self,
    X: pd.DataFrame,
    n_ensembles: int = 10,
    n_threads: int = 1,
    uncertainty_for_opt: bool = False,
    normalize_by_group_size: bool = False,
) -> pd.DataFrame

Approach: divides the tree sequence into n_ensembles equally-spaced snapshots via staged_predict, converts each snapshot's raw scores to 1-based ranks, then summarises per-sample rank variability across snapshots.

Output columns:

Column	Description
pred	1-based rank from the full model
mean_predictions	Mean rank across snapshots (rounded), optionally normalised
knowledge_uncertainty	IQR of ranks across snapshots (Q3 − Q1), optionally normalised
data_uncertainty	None — not available for ranking
total_uncertainty	Equal to knowledge_uncertainty (no aleatoric component)

• normalize_by_group_size=True divides mean_predictions and knowledge_uncertainty / total_uncertainty by len(X) for cross-group comparability.
• uncertainty_for_opt=True returns only {"knowledge_uncertainty": ...}, consistent with other Mother wrappers.

---

Bug fixes

• Added Any, Optional, Union to typing imports (resolved 13 F821 ruff errors).

Terminology

• Replaced "documents" → "samples" throughout ranker docstrings (the model is used on chemical structures, not text).