Feat/pipeline optimizations

[LeFi8]

No description provided.

[gemini-code-assist]

Code Review

This pull request introduces a cheap in-sample pre-screen to skip expensive statistical evaluations for poor candidates, and an Optuna-based numeric parameter optimization feature to tune strategy parameters on in-sample data. It also updates LLM prompts to display unified diffs of failed attempts and adds database schema updates and tests. The review feedback focuses on improving the performance of the Optuna search by pre-computing walk-forward windows and asset returns, securing YAML serialization with safe_dump, robustly handling NaN values in metric comparisons, and simplifying file cleanup with unlink(missing_ok=True).

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[gemini-code-assist]

To significantly improve the efficiency of the Optuna parameter search, we can avoid re-computing the walk-forward windows and asset returns on every single trial. Since prices and settings do not change during the optimization process, these values can be pre-computed once and passed into _in_sample_objective as optional arguments.

def _in_sample_objective(
    generate_signals_fn: Any,
    flat_config: dict[str, Any],
    prices: pd.DataFrame,
    _asset_returns: pd.DataFrame | None = None,
    _wf_windows: list[Any] | None = None,
) -> tuple[float, float, pd.Series]:

[gemini-code-assist]

Use the optional pre-computed _wf_windows and _asset_returns if provided, to bypass the expensive window generation and pct_change() calculations during iterative optimization trials.

    if _wf_windows is None:
        in_sample_idx, _ = partition_holdout_data(prices.index, holdout_years=settings.default_holdout_years)
        wf_windows = generate_walk_forward_windows(in_sample_idx, train_years=5, test_years=1)
    else:
        wf_windows = _wf_windows

    if not wf_windows:
        return 0.0, 0.0, pd.Series(dtype=float)

    wf_weights = _generate_wf_weights(prices, generate_signals_fn, flat_config, tickers, wf_windows)
    if _asset_returns is None:
        _asset_returns = prices.pct_change().fillna(0.0)

[gemini-code-assist]

Pre-compute the walk-forward windows and asset returns once before starting the Optuna study, and pass them to _in_sample_objective to avoid redundant computations in every trial.

Suggested change

	original_sharpe, _, _ = _in_sample_objective(generate_signals_fn, flat_config, prices)
	if pd.isna(original_sharpe):
	original_sharpe = 0.0
	from autobacktest.config import settings
	from autobacktest.evaluator.evaluate import partition_holdout_data, generate_walk_forward_windows
	in_sample_idx, _ = partition_holdout_data(prices.index, holdout_years=settings.default_holdout_years)
	wf_windows = generate_walk_forward_windows(in_sample_idx, train_years=5, test_years=1)
	asset_returns = prices.pct_change().fillna(0.0)
	original_sharpe, _, _ = _in_sample_objective(
	generate_signals_fn, flat_config, prices, _asset_returns=asset_returns, _wf_windows=wf_windows
	)
	if pd.isna(original_sharpe):
	original_sharpe = 0.0

[gemini-code-assist]

Pass the pre-computed asset_returns and wf_windows to _in_sample_objective inside the Optuna objective function to prevent recalculating them on every trial.

Suggested change

	sh, _, _ = _in_sample_objective(generate_signals_fn, trial_config, prices)
	if pd.isna(sh):
	return -float("inf")
	return float(sh)
	sh, _, _ = _in_sample_objective(
	generate_signals_fn, trial_config, prices, _asset_returns=asset_returns, _wf_windows=wf_windows
	)
	if pd.isna(sh):
	return -float("inf")
	return float(sh)

[gemini-code-assist]

Use yaml.safe_dump instead of yaml.dump to ensure secure and standard serialization of the configuration dictionary, avoiding potential execution of arbitrary Python objects.

Suggested change

	return yaml.dump(yaml_dict, default_flow_style=False, sort_keys=False)
	return yaml.safe_dump(yaml_dict, default_flow_style=False, sort_keys=False)

[gemini-code-assist]

Use p.unlink(missing_ok=True) to simplify the cleanup logic and avoid potential race conditions where the file is deleted between the exists() check and the unlink() call.

            for p in [temp_py, temp_yaml]:
                p.unlink(missing_ok=True)

[gemini-code-assist]

If either opt_metric or orig_metric is NaN, direct comparison using >= will evaluate to False. To make the comparison robust against NaN values, default them to -inf before comparing.

Suggested change

	if sel.accepted and cnf.accepted and opt_metric >= orig_metric:
	orig_val = orig_metric if not pd.isna(orig_metric) else -float("inf")
	opt_val = opt_metric if not pd.isna(opt_metric) else -float("inf")
	if sel.accepted and cnf.accepted and opt_val >= orig_val: