Add optional feature type auto detection to Catboost and also auto sorting of ranking groups

[thomasATbayer]

Summary: CatBoost Feature Type Auto-Detection with Manual Configuration Override plus some additional fixes

This will also enable feature selection on embedding, text and categorial features when using the Mother Carboost with freature type detection for the feature selectors

This PR adds flexible feature type handling to CatBoost models by combining automatic dtype-based detection with explicit manual configuration through the Mother config system.

The callback logic was improved to automatically skip Optuna early stopping when hold-out validation or any cross-validation with fewer than 2 splits is used, preventing errors and ensuring correct behavior. This allows the use of hold out sets.

The default boostrap for catboost optimization using Optuna is now MVS, was not clear from the documentation if Bayesian or MVS, but generated some catboost models to check.

The catboost ranker can now handle non integer query groups i.e. dtype category and if the data is not sorted according to groups will be auto sorted.

🎯 What This Adds

Two complementary approaches for specifying feature types:

1. Automatic Detection (opt-in via flags):

   • auto_class_categorical=True - Auto-detects columns with category dtype
   • auto_parse_text=True - Auto-detects object dtype columns containing strings
   • auto_parse_embedding=True - Auto-detects object dtype columns containing vectors (lists, arrays)

2. Manual Configuration (via Mother config):

   • categorical_features: [] - Explicitly specify categorical feature column names
   • text_features: [] - Explicitly specify text feature column names
   • embedding_features: [] - Explicitly specify embedding feature column names

3. Why 'category' dtype is required instead of 'object':

   • The ambiguity problem with object dtype (can be categorical strings, text, or vectors)
   • The design philosophy of being explicit and intentional
   • The practical benefit of better pandas and catboost performance

🔄 How Old and New Approaches Combine

Smart Precedence System:

• Manual configuration ALWAYS takes precedence over auto-detection
• Auto-detection only activates when:
  1. The feature type is NOT explicitly specified in config, AND
  2. The corresponding auto-detection flag is enabled, AND
  3. Input is a pandas DataFrame (auto-detection requires column names)

Cross-type exclusion:

• Features manually specified in any type list are automatically excluded from auto-detection in other types
• Prevents a single feature from appearing in multiple type lists

Example workflow:

# Config specifies some features manually
categorical_features: ["feature_A"]
text_features: ["feature_B"]

# At runtime with auto_parse_embedding=True:
# - "feature_A" used as categorical (from config)
# - "feature_B" used as text (from config)  
# - Embedding features auto-detected from remaining object dtype vector columns
# - "feature_A" and "feature_B" excluded from embedding auto-detection

🚀 Extension Points

Architecture improvements:

1. Reusable detection utilities (mother/ml/models/utils.py):
   • detect_categorical_features() - Finds 'category' dtype columns
   • detect_text_features() - Finds object dtype string columns
   • detect_embedding_features() - Finds object dtype vector columns
   • Can be reused by future model implementations beyond CatBoost

[thomasATbayer]

def detect_categorical_features(X: pd.DataFrame, exclude_features: Optional[List[str]] = None) -> Optional[List[str]]:
"""
Auto-detect categorical features from DataFrame dtypes.

Detects ONLY columns with explicit 'category' dtype.
Object dtype columns are NOT treated as categorical - they are considered text or embeddings.

Why 'category' dtype instead of 'object'?
- Object dtype is ambiguous: it can contain categorical strings, free-form text, or vector embeddings
- Category dtype explicitly signals categorical intent and prevents confusion with text/embedding features
- This design forces users to be intentional about their feature types via X['col'].astype('category')
- Category dtype is also more memory-efficient and enables better pandas optimizations

Args:
    X : pd.DataFrame
        Training data.
    exclude_features : Optional[List[str]]
        Features to exclude from detection (manually specified in other types).

Returns:
    Optional[List[str]]: List of categorical feature names, or None if no categorical features found.
"""
if not isinstance(X, pd.DataFrame):
    module_logger.warning(
        "Categorical feature auto-detection only works with pandas DataFrames. "
        f"Received {type(X).__name__} instead. Returning None. "
        "To use auto-detection, convert your data to a DataFrame or specify categorical features manually."
    )
    return NoneExpand commentComment on lines R46 to R52Resolved

# Only 'category' dtype columns
category_cols: List[str] = X.select_dtypes(include=["category"]).columns.tolist()

# Exclude manually specified features from other types
if exclude_features:
    category_cols = [col for col in category_cols if col not in exclude_features]

return category_cols if category_cols else None

def detect_text_features(X: pd.DataFrame, exclude_features: Optional[List[str]] = None) -> Optional[List[str]]:
"""
Auto-detect text features from DataFrame dtypes.

Detects object dtype columns containing strings (not vectors).
Checks up to 10 non-null samples per column to ensure robustness
against inconsistent data types. Null values are automatically skipped.

Args:
    X : pd.DataFrame
        Training data.
    exclude_features : Optional[List[str]]
        Features to exclude from detection (manually specified in other types).

Returns:
    Optional[List[str]]: List of text feature names, or None if no text features found.
"""
if not isinstance(X, pd.DataFrame):
    module_logger.warning(
        "Text feature auto-detection only works with pandas DataFrames. "
        f"Received {type(X).__name__} instead. Returning None. "
        "To use auto-detection, convert your data to a DataFrame or specify text features manually."
    )
    return None

text_features: List[str] = []

if len(X) > 0:
    object_cols: List[str] = X.select_dtypes(include=["object"]).columns.tolist()
    for col in object_cols:
        # Skip if manually specified in another type
        if exclude_features and col in exclude_features:
            continue
        # Check multiple rows (up to 10 samples) to determine if column contains text
        # Sample at most 10 non-null values to check for consistency
        non_null_values: pd.Series = X[col].dropna()
        if len(non_null_values) == 0:
            continue

        sample_size: int = min(10, len(non_null_values))
        sample_values: pd.Series = non_null_values.iloc[:sample_size]

        # Check if all sampled values are NOT vectors (i.e., they are strings)
        is_text: bool = all(not isinstance(val, (list, np.ndarray, pd.Series)) for val in sample_values)
        if is_text:
            text_features.append(col)

return text_features if text_features else None

def detect_embedding_features(X: pd.DataFrame, exclude_features: Optional[List[str]] = None) -> Optional[List[str]]:
"""
Auto-detect embedding features from DataFrame dtypes.

Detects object dtype columns containing vector types (list, np.ndarray, pd.Series).
Checks up to 10 non-null samples per column to ensure robustness
against inconsistent data types. Null values are automatically skipped.

Args:
    X : pd.DataFrame
        Training data.
    exclude_features : Optional[List[str]]
        Features to exclude from detection (manually specified in other types).

Returns:
    Optional[List[str]]: List of embedding feature names, or None if no embedding features found.
"""
if not isinstance(X, pd.DataFrame):
    module_logger.warning(
        "Embedding feature auto-detection only works with pandas DataFrames. "
        f"Received {type(X).__name__} instead. Returning None. "
        "To use auto-detection, convert your data to a DataFrame or specify embedding features manually."
    )
    return None

embedding_features: List[str] = []

if len(X) > 0:
    object_cols: List[str] = X.select_dtypes(include=["object"]).columns.tolist()
    for col in object_cols:
        # Skip if manually specified in another type
        if exclude_features and col in exclude_features:
            continue
        # Check multiple rows (up to 10 samples) to determine if column contains embeddings
        # Sample at most 10 non-null values to check for consistency
        non_null_values: pd.Series = X[col].dropna()
        if len(non_null_values) == 0:
            continue

        sample_size: int = min(10, len(non_null_values))
        sample_values: pd.Series = non_null_values.iloc[:sample_size]

        # Check if all sampled values are vectors
        is_embedding: bool = all(isinstance(val, (list, np.ndarray, pd.Series)) for val in sample_values)
        if is_embedding:
            embedding_features.append(col)

return embedding_features if embedding_features else None

[thomasATbayer]

class _CatboostFeatureTypes:
"""
A utility mixin for automatic feature type detection in CatBoost models.

This class provides methods to auto-detect categorical, text, and embedding features
from pandas DataFrame dtypes when auto-detection is enabled.

**Important limitations**:
- Auto-detection ONLY works with pandas DataFrames (not numpy arrays or other array-like objects)
- Auto-detected features are referred to by column name (not by index)
- If you need to use column indices, you must specify features manually

Auto-detection behavior (when enabled via auto_class_categorical, auto_parse_text, and/or auto_parse_embedding):
- Categorical features: Detected ONLY from 'category' dtype columns (must explicitly use .astype('category'))
  * Why not 'object' dtype? Object is ambiguous and could be text or embeddings. Using 'category' dtype
    forces explicit intent and prevents confusion between categorical, text, and embedding features.
- Text features: Detected from object dtype columns containing strings
- Embedding features: Detected from object dtype columns containing vector types (list, np.ndarray, pd.Series)

When all auto-detection flags are False (default):
- ALL feature types (cat_features, text_features, embedding_features) must be explicitly
  specified during initialization. No automatic detection is performed.

Methods
-------

_detect_feature_types(X) -> tuple
    Detects or retrieves feature types (categorical, embedding, text) from data.
"""

def _detect_feature_types(
    self, X: Union[pd.DataFrame, np.ndarray]
) -> Tuple[
    Optional[Union[List[str], List[int]]],
    Optional[Union[List[str], List[int]]],
    Optional[Union[List[str], List[int]]],
]:
    """
    Detect categorical, text, and embedding features from DataFrame dtypes.

    This method auto-detects feature types based on pandas dtypes if the corresponding
    auto-detection flags are enabled, or uses only explicitly set feature types.

    **Important limitations**:
    - Auto-detection ONLY works with pandas DataFrames (not numpy arrays or other array-like objects)
    - Auto-detected features are referred to by column name (not by index)
    - If X is not a DataFrame or you need column indices, features must be specified manually

    **Precedence**: Manually defined feature types (set during initialization via cat_features,
    embedding_features, text_features parameters) ALWAYS take precedence over auto-detection.
    Auto-detection only occurs when the corresponding feature type is not explicitly specified
    AND the corresponding auto-detection flag is enabled.

    **Feature exclusion**: Features manually specified in any feature type list (cat_features,
    text_features, or embedding_features) are automatically excluded from auto-detection in
    other types to prevent a feature from appearing in multiple type lists.

    **Explicit dtype requirements for auto-detection**:

    When auto_class_categorical=True:
    - Categorical features: Auto-detected ONLY from 'category' dtype columns
    - Columns must explicitly use .astype('category')
    - Only applies if cat_features was not explicitly set

    When auto_parse_text=True:
    - Text features: Auto-detected from object dtype columns containing strings
    - Object dtype columns (non-vectors) are treated as text
    - Only applies if text_features was not explicitly set

    When auto_parse_embedding=True:
    - Embedding features: Auto-detected from object dtype columns containing vectors
      (list, np.ndarray, pd.Series)
    - Only applies if embedding_features was not explicitly set

    When all flags are False (default):
    - ALL feature types must be explicitly specified during initialization
    - No automatic detection is performed

    Args:
        X : Union[pd.DataFrame, np.ndarray]
            Training data. Auto-detection only works with pandas DataFrames.

    Returns:
        Tuple[Optional[Union[List[str], List[int]]], \
              Optional[Union[List[str], List[int]]], \
              Optional[Union[List[str], List[int]]]]
            Tuple of (cat_features, embedding_features, text_features).
            Each can be None, list of column names, or list of column indices.
    """
    category_features: Optional[Union[List[str], List[int]]] = None
    embedding_features: Optional[Union[List[str], List[int]]] = None
    text_features: Optional[Union[List[str], List[int]]] = None

    # Check if feature types were explicitly set during initialization
    params: dict = self.get_params(deep=False)
    cat_features_from_init: Optional[Union[List[str], List[int], str, int]] = params.get("cat_features", None)
    embedding_features_from_init: Optional[Union[List[str], List[int], str, int]] = params.get(
        "embedding_features", None
    )
    text_features_from_init: Optional[Union[List[str], List[int], str, int]] = params.get("text_features", None)

    # Check if auto-detection is enabled for each type
    auto_detect_cat: bool = getattr(self, "auto_class_categorical", False)
    auto_detect_text: bool = getattr(self, "auto_parse_text", False)
    auto_detect_emb: bool = getattr(self, "auto_parse_embedding", False)

    if isinstance(X, pd.DataFrame):
        # Collect all manually specified features to exclude from auto-detection
        manually_specified: set = set()
        if cat_features_from_init is not None:
            manually_specified.update(
                cat_features_from_init if isinstance(cat_features_from_init, list) else [cat_features_from_init]
            )
        if text_features_from_init is not None:
            manually_specified.update(
                text_features_from_init if isinstance(text_features_from_init, list) else [text_features_from_init]
            )
        if embedding_features_from_init is not None:
            manually_specified.update(
                embedding_features_from_init
                if isinstance(embedding_features_from_init, list)
                else [embedding_features_from_init]
            )

        # Handle categorical features (only 'category' dtype)
        if cat_features_from_init is not None:
            # Explicitly set - use those
            category_features = cat_features_from_init
        elif auto_detect_cat:
            # Auto-detect, excluding manually specified features from OTHER types
            category_features = models_utils.detect_categorical_features(
                X, exclude_features=list(manually_specified)
            )

        # Handle text features (object dtype strings)
        if text_features_from_init is not None:
            # Explicitly set - use those
            text_features = text_features_from_init
        elif auto_detect_text:
            # Auto-detect, excluding manually specified features from OTHER types
            text_features = models_utils.detect_text_features(X, exclude_features=list(manually_specified))

        # Handle embedding features (object dtype vectors)
        if embedding_features_from_init is not None:
            # Explicitly set - use those
            embedding_features = embedding_features_from_init
        elif auto_detect_emb:
            # Auto-detect, excluding manually specified features from OTHER types
            embedding_features = models_utils.detect_embedding_features(
                X, exclude_features=list(manually_specified)
            )
    else:
        # If not a DataFrame, only use explicitly set features
        category_features = cat_features_from_init
        embedding_features = embedding_features_from_init
        text_features = text_features_from_init

    return category_features, embedding_features, text_features

[thomasATbayer]

def fit(self, X, y, **kwargs):
    """
    Fit the CatBoost regressor model.

    Feature type handling depends on auto_class_categorical, auto_parse_text, and auto_parse_embedding settings:

    When auto_class_categorical=True:
    - Categorical features: Auto-detected ONLY from explicit 'category' dtype columns
    - Columns must use .astype('category') to be detected

    When auto_parse_text=True:
    - Text features: Auto-detected from object dtype columns containing strings
    - Object columns with plain strings are treated as text features

    When auto_parse_embedding=True:
    - Embedding features: Auto-detected from object dtype columns containing vectors
    - Object columns with list, np.ndarray, or pd.Series are treated as embeddings

    When all flags are False (default):
    - ALL feature types (cat_features, text_features, embedding_features) must be explicitly
      specified during initialization. No automatic detection is performed.

    Manually specified features always take precedence over auto-detection.

    Args:
        X : array-like, pandas.DataFrame, catboost.Pool
            Training data. Auto-detection of feature types only works with pandas DataFrame.
        y : array-like, pandas.Series, pandas.DataFrame
            Target values.
        **kwargs
            Additional keyword arguments passed to CatBoost's fit method (e.g., sample_weight, baseline, etc.).

    Returns:
        self : CatboostRegressorMother
            The fitted estimator.

    Notes:
        **For auto-detection** - Enable auto_class_categorical, auto_parse_text, and/or auto_parse_embedding:
        - Categorical: Use .astype('category') for explicit categorical dtype
        - Text: Use object dtype with strings
        - Embedding: Use object dtype with vectors (lists/numpy arrays)

        **Without auto-detection (default)** - All flags False:
        - ALL feature types must be explicitly specified via cat_features, text_features,
          and embedding_features parameters during initialization
    """
    # Auto-detect or use explicitly set feature types
    category_features, embedding_features, text_features = self._detect_feature_types(X)

    # Pass X, y first, then feature specifications, then remaining kwargs
    return super().fit(
        X,
        y,
        cat_features=category_features,
        embedding_features=embedding_features,
        text_features=text_features,Expand commentComment on lines R693 to R699Resolved
        **kwargs,
    )