Remaining Bugs in ML result parsing, pipeline control flow, and DuckDB connection handling

[eboyer221]

Bugs identified on main. Checked against the other open PRs - #17 already replaces two buggy functions in feature_rescoring.R, so those aren't included here.

PR 1) filename parsing

• Stratified result files get parsed wrong because parse_ml_filename() misreads year/country-stratified filenames (picks up "year" as the drug name, loses the seed), with no error. Proposed Fix: correct the token-splitting logic for the stratification token, add a regression test.

PR 2) pipeline control flow

• Fresh pipeline runs crash. Before doing any work, runMLmodels() is supposed to check "did I actually find any files to process?" and stop with a clear message if not. That check got deleted. Now, running it before any files exist just crashes with a confusing error instead of telling you what's wrong. Fix: add the check back in, runMDRmodels() still has it, right after it looks for input files:

if (nrow(files) == 0) {
  message("No files found.")
  return(invisible(NULL))
}

• Running the full pipeline produces nothing, but says it worked: There's a function (runModelingPipelineIntense()) meant to run the entire modeling process from start to finish: first build the feature matrices from the data, then train all the models on them. Right now, the "build the matrices" step is switched off (commented out), but every step after it assumes the matrices already exist. So when someone runs the whole thing, each later step fails because it can't find its input, and the function still prints "Pipeline Complete" at the end so it looks successful but actually produced nothing. Fix: turn the matrix-building step back on, or make the later steps stop with a clear error if the matrices are missing instead of failing.

• One function hands back the wrong list of files: createMLinputList() figures out which files each model should train on. For one specific combination of options (leave-one-out testing plus cross-testing together), it correctly builds the right list, but never hands that list back. The function just keeps running and returns a different, wrong list meant for another situation. Anyone using this combination gets the wrong training files with no error. Fix: add the missing line that returns the correct list at the right spot.

• There used to be a check that stopped right away with a clear error if someone tried LOO without year or country to group by. That check has been switched off. The code is still running, just returns back empty. Fix: turn the check back on so this mistake is caught immediately with a clear message.

PR 3) warning downgrade

• .calculateLog2APOP()'s balanced/imbalanced-class notice was changed from warning() to message(), which breaks suppressWarnings() in an existing test and any other caller relying on catching it as a warning. Fix: change it back to warning().

PR D — DuckDB connection cleanup

• DuckDB connections aren't being closed properly : two functions' cleanup step checks the wrong variable name, so the real connection never gets closed, and errors during setup get masked by a confusing secondary error. Fix: point the cleanup step at the correct variable in both functions.

[AbhirupaGhosh]

runModelingPipelineIntense() can have a PR of its own. It is a wrapper function to run everything from matrix creation to building every possible models.

[eboyer221]

runModelingPipelineIntense() can have a PR of its own. It is a wrapper function to run everything from matrix creation to building every possible models.

PR Created here: #37