Skip to content

feat: introduce warm start - #150

Draft
Mateusz-Switala wants to merge 10 commits into
IBM:mainfrom
Mateusz-Switala:feat-gam-opt-add-warm-start
Draft

Mateusz-Switala wants to merge 10 commits into
IBM:mainfrom
Mateusz-Switala:feat-gam-opt-add-warm-start

Conversation

@Mateusz-Switala

@Mateusz-Switala Mateusz-Switala commented Sep 10, 2026

Copy link
Copy Markdown
Collaborator

Description

Improves the warm-start strategies for the GAM-based HPO optimizer (GAMOptimizer) so greedy and balanced
starts provide meaningful discrete-parameter coverage before GAM-guided search begins.

Motivation

The previous warm-start selection could under-cover discrete parameter values, causing early trials to
cluster and leaving the fitted GAM with sparse categorical observations.

Changes

  • Coverage-aware warm starts: Greedy and balanced strategies prioritize combinations that cover
    previously unseen discrete values.
  • Balanced strategy: Uses round-robin selection across fields_to_balance value tuples while also
    prioritizing coverage of non-balanced discrete fields.
  • Automatic warm-start sizing: For greedy and balanced strategies, the effective warm-start target
    expands to the minimum required for coverage rather than failing or silently under-covering when
    n_random_nodes is too small.
  • Evaluation allocation: max_evals is optional; when specified, output allocation accounts for the
    effective warm-start target and reserves remaining capacity for GAM-guided evaluations.
  • Model-value handling: Normalizes model objects, dictionaries, and strings for coverage tracking;
    serializes model-valued DataFrame columns to model_id before GAM fitting.
  • Balanced-selection fix: Corrected the ordering/fallback behavior so round-robin balancing is
    preserved.
  • Tests: Expanded test_gam_opt.py for coverage, round-robin behavior, model serialization, auto-
    adjusted warm starts, and post-warm-start GAM iterations.

Testing

  • tests/unit/ai4rag/core/hpo/test_gam_opt.py

Checklist

  • Tests added/updated
  • Documentation updated
  • Code follows style guide
  • All checks passing

@Mateusz-Switala Mateusz-Switala changed the title introduce warm start feat: introduce warm start Sep 10, 2026
Signed-off-by: Mateusz Switala <mswitala@redhat.com>
Assisted-by: Claude Code
Signed-off-by: Mateusz Switala <mswitala@redhat.com>
Assisted-by: OpenAI Codex
Signed-off-by: Mateusz Switala <mswitala@redhat.com>
Assisted-by: OpenAI Codex
Signed-off-by: Mateusz Switala <mswitala@redhat.com>
Assisted-by: OpenAI Codex
Signed-off-by: Mateusz Switala <mswitala@redhat.com>
Assisted-by: OpenAI Codex
Signed-off-by: Mateusz Switala <mswitala@redhat.com>
Assisted-by: OpenAI Codex
Signed-off-by: Mateusz Switala <mswitala@redhat.com>
Assisted-by: OpenAI Codex
Signed-off-by: Mateusz Switala <mswitala@redhat.com>
Assisted-by: OpenAI Codex
@Mateusz-Switala
Mateusz-Switala force-pushed the feat-gam-opt-add-warm-start branch from 928b831 to 3078d22 Compare September 13, 2026 19:09
 warm start balanced fallback

Signed-off-by: Mateusz Switala <mswitala@redhat.com>
Assisted-by: OpenAI Codex
Signed-off-by: Mateusz Switala <mswitala@redhat.com>
Assisted-by: OpenAI Codex
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant