Hull Tactical Market Prediction

Kaggle Competition 2025 - Beat the market by predicting S&P 500 returns and building optimal trading strategies.

This repository contains our team's solution for the Hull Tactical Market Prediction competition, where participants must develop machine learning models to predict forward market returns and convert them into actionable trading signals.

Competition Overview

The goal is to predict daily forward excess returns of the S&P 500 and generate trading signals that optimize a custom Sharpe ratio metric. The evaluation metric penalizes strategies that:

• Have higher volatility than the market
• Underperform the market returns

Trading Signal Logic

• Signal = 0: Fully invested in risk-free assets (bonds/T-bills)
• Signal = 1: Market-neutral position
• Signal = 2: Fully invested in S&P 500
• Signal ∈ [0,2]: Linear interpolation between positions

Repository Structure

├── src/                          # Main ML pipeline
│   ├── data_class.py            # Configuration dataclasses
│   ├── data_loading.py          # Data loading & preprocessing
│   ├── model.py                 # ML models & cross-validation
│   ├── evaluator.py             # Evaluation pipeline
│   ├── score.py                 # Competition metric
│   ├── utils.py                 # Utility functions
│   ├── main.py                  # Simple training script
│   └── starter_nb.py            # Original starter notebook
├── hull-tactical-market-prediction/  # Competition data & evaluation
│   ├── train.csv               # Training data
│   ├── test.csv                # Test data (10 samples)
│   └── kaggle_evaluation/      # Kaggle evaluation infrastructure
├── results/                     # Experiment outputs
└── pyproject.toml              # Dependencies

ML Pipeline Architecture

1. Data Pipeline (data_loading.py)

Data Loading:

• Loads training data (train.csv) and test data (test.csv)
• Handles column renaming: forward_returns → target for train, lagged_forward_returns → target for test
• Casts all numeric columns to Float64

Feature Engineering:

• Selects curated feature set: ["S2", "E2", "E3", "P9", "S1", "S5", "I2", "P8", "P10", "P12", "P13", "U1", "U2"]
• Creates engineered features:
  • U1 = I2 - I1 (momentum indicator)
  • U2 = M11 / ((I2 + I9 + I7) / 3) (technical ratio)
• Handles missing values with exponential moving average imputation

Data Splitting:

• Scales features using StandardScaler (fitted on train, applied to test)
• Returns structured DatasetOutput with X_train, X_test, y_train, y_test

2. Model System (model.py)

Supported Models:

• ElasticNet: Regularized linear regression with L1/L2 penalties
• GradientBoostingRegressor: Tree-based ensemble method

Cross-Validation Strategy:

• Uses TimeSeriesSplit to prevent lookahead bias
• Expanding window approach: each fold uses all previous data for training
• Test size: 252 trading days (~1 year) per fold
• Supports Grid Search, Random Search, and Bayesian Search (planned)
• Optimizes hyperparameters for each model type
• Global Sharpe Calculation: Uses "Ratio of Averages" approach - concatenates all out-of-sample predictions from all CV folds before computing the adjusted Sharpe ratio (avoids statistical bias from averaging non-linear metrics)

Figure: Time series cross-validation with expanding window. Each fold trains on all previous data (blue) and tests on the next 252 days (orange). This prevents lookahead bias while maximizing training data usage.

Stable Hyperparameter Search: A robust workflow to find configurations that perform consistently across different random seeds:

1. Step 1 - Initial Search: Run hyperparameter search once with a fixed seed, collect top K configurations
2. Step 2 - Stability Check: Evaluate each top K config across multiple seeds (e.g., [0,1,2,3,4])
3. Step 3 - Selection: Choose based on:
   • Best Mean Score: Maximize expected performance
   • Best Mean/Std Trade-off: Robust/safer configuration

Features:

• Parallelized seed evaluation for ~4-5x speedup
• Tracks mean, std, min, max scores across seeds
• Outputs stability metrics and recommendations

3. Signal Generation (utils.py)

Return to Signal Conversion:

def convert_ret_to_signal(ret_arr, params):
    return np.clip(
        ret_arr * params.signal_multiplier + 1,
        params.min_signal,
        params.max_signal
    )

• Multiplies predicted returns by SIGNAL_MULTIPLIER (400.0)
• Adds 1.0 to center around neutral position (1.0)
• Clips to [0, 2] range

4. Evaluation System (evaluator.py)

Walk-Forward Validation:

• Implements expanding-window cross-validation
• Prevents overfitting by ensuring validation data is always after training data
• Minimum training window: 252 trading days (~1 year)

Parallel Evaluation:

• Runs multiple evaluation seeds in parallel using joblib
• Aggregates results across different random seeds
• Saves results to timestamped CSV files

5. Competition Metric (score.py)

Hull Metric Calculation: The official metric computes an adjusted Sharpe ratio with penalties:

def hull_metric_from_arrays(forward_returns, risk_free_rate, positions):
    # Calculate strategy returns
    strategy_returns = rf_rate * (1 - positions) + positions * forward_returns

    # Compute Sharpe ratio
    strategy_excess_returns = strategy_returns - risk_free_rate
    sharpe = mean_excess_return / strategy_std * sqrt(252)

    # Volatility penalty (if strategy vol > 1.2x market vol)
    vol_penalty = 1 + max(0, strategy_vol/market_vol - 1.2)

    # Return gap penalty (if strategy underperforms market)
    return_gap = max(0, market_mean_return - strategy_mean_return)
    return_penalty = 1 + (return_gap²) / 100

    # Final score
    adjusted_sharpe = sharpe / (vol_penalty * return_penalty)
    return min(adjusted_sharpe, 1_000_000.0)

Usage

Training a Model

# Simple training (main.py)
python -m src.main

# Standard evaluation with cross-validation
python -m src.evaluator --model_type GradientBoosting --cv_search Random --n_runs 50 --n_splits 10

# Stable hyperparameter search (recommended for finding robust configs)
python -m src.evaluator --stable_search --model_type GradientBoosting --cv_search Random \
    --n_splits 10 --search_seed 42 --eval_seeds 0 1 2 3 4 --top_k 5

# Or use the convenience script
bash run_stable_search.sh

Evaluation Modes

Standard Evaluation (--n_runs):

• Runs hyperparameter search multiple times with different seeds
• Good for exploring hyperparameter space
• Results show variation across random initializations

Stable Search (--stable_search):

• Finds top K configurations from initial search
• Evaluates each config across multiple seeds
• Identifies stable, robust configurations
• Parallelized for speed (use --n_jobs to control cores)

# Standard: 100 runs with different seeds
python -m src.evaluator --model_type GradientBoosting --n_runs 100

# Stable: Find top 5 configs, test across 5 seeds
python -m src.evaluator --stable_search --top_k 5 --eval_seeds 0 1 2 3 4

# Stable with custom parallelism (use 4 cores)
python -m src.evaluator --stable_search --n_jobs 4

Key Parameters

Signal Conversion:

SIGNAL_MULTIPLIER = 400.0    # Return scaling factor
MIN_SIGNAL = 0.0             # Minimum position
MAX_SIGNAL = 2.0             # Maximum position

Model Hyperparameters (examples):

ALPHAS = np.logspace(-4, 2, 100)  # ElasticNet regularization
L1_RATIO = 0.5                       # L1/L2 mixing
MAX_ITER = 1000000                   # Maximum iterations

Evaluator CLI Arguments:

Common:

• --model_type: ElasticNet or GradientBoosting
• --cv_search: Grid, Random, or None
• --n_splits: Number of CV splits (default: 10)
• --keep_all_features: Keep all features (no selection)
• --data_cleaning: Apply data cleaning

Standard Evaluation:

• --n_runs: Number of runs with different seeds (default: 100)

Stable Search:

• --stable_search: Enable stable search mode
• --search_seed: Seed for initial search (default: 42)
• --eval_seeds: Seeds for stability check (default: 0 1 2 3 4)
• --top_k: Number of top configs to evaluate (default: 5)
• --n_jobs: Parallel jobs for seed evaluation (default: -1 = all cores)

📊 Results & Experiments

Results are stored in the results/ directory with timestamped filenames:

Standard Evaluation:

• average_score_avg_YYYY-MM-DD HH:MM:SS_ModelType_hyperparametersCV_SearchType.csv: Cross-validation results across multiple runs

Stable Search:

• average_score_avg_YYYY-MM-DD HH:MM:SS_ModelType_stable_search.csv: Stability analysis with mean/std scores across seeds

Other:

• best_params.json: Best hyperparameters found

Current Best Results

Based on recent experiments:

• ElasticNet: ~0.45 average Hull score
• GradientBoosting: ~0.42 average Hull score

Interpreting Stable Search Results

The stable search output includes:

• mean_score: Average performance across seeds (higher is better)
• std_score: Consistency across seeds (lower is better)
• mean_std_ratio: Stability metric (higher = more stable)
• individual_scores: Scores for each seed

Recommendation strategies:

1. Best Mean Score: Choose config with highest mean (maximize expected performance)
2. Best Mean/Std Trade-off: Choose config with highest mean/std ratio (robust configuration)

Key Implementation Details

Time Series Cross-Validation

The pipeline uses an expanding window approach to prevent lookahead bias:

• Each fold trains on all historical data up to that point
• Test set is always the next 252 trading days (~1 year)
• This mimics real-world deployment where you only have past data

To visualize the CV splits:

bash visualize_cv.sh
# Generates figures/cv_folds_with_data.png

Global Sharpe Ratio (Ratio of Averages)

The evaluation uses a statistically sound approach for computing the adjusted Sharpe ratio:

• ❌ Wrong: Calculate Sharpe for each CV fold, then average (Average of Ratios - biased)
• ✅ Correct: Concatenate all out-of-sample predictions from all folds, then calculate Sharpe once (Ratio of Averages - unbiased)

This is critical because the adjusted Sharpe ratio is a non-linear metric. Averaging ratios introduces statistical bias.

Parallelization

• Seed-level parallelization: Multiple seeds evaluated in parallel (~4-5x speedup with 5 seeds)
• Controlled via --n_jobs parameter (default: -1 for all cores)
• Each seed evaluation runs CV folds sequentially to avoid nested parallelism

Development Roadmap

Immediate Next Steps

1. Feature Engineering: Add more technical indicators and market microstructure features
2. Model Ensemble: Combine predictions from multiple models
3. Hyperparameter Optimization: Implement Bayesian optimization
4. Feature Selection: Use recursive feature elimination or LASSO-based selection

Dependencies

• polars: High-performance DataFrame operations
• scikit-learn: ML algorithms and preprocessing
• numpy: Numerical computations
• pandas: Data manipulation
• joblib: Parallel processing

Notes

• The test set contains only 10 samples, so evaluation relies heavily on cross-validation
• Time series cross-validation is critical to avoid data leakage
• The metric heavily penalizes high-volatility underperforming strategies
• Signal clipping at [0,2] bounds is enforced in evaluation

Contributing

1. Create feature branches for new experiments
2. Run full cross-validation before merging
3. Document new approaches in experiment logs
4. Update this README with significant changes

---

Competition Link: Hull Tactical Market Prediction