summarise_scores() accepts metric/score columns in by and silently returns a broken, unsummarised table.
Reproduction
library(scoringutils)
scores <- score(as_forecast_quantile(example_quantile))
nrow(scores)
#> [1] 887
out <- summarise_scores(scores, by = c("model", "wis"))
nrow(out)
#> [1] 887 # same as input -- nothing was summarised
colnames(out)
#> [1] "model" "wis" "wis" "overprediction" "underprediction" "dispersion"
#> [7] "bias" "interval_coverage_50" "interval_coverage_90" "ae_median"
The result has a duplicated wis column (once as a grouping column, once as a "summarised" score), and because the near-unique numeric wis values are used as groups, nothing is actually aggregated. No warning or error is emitted, and the duplicate column names break downstream data.table operations.
Cause
In R/summarise_scores.R:
- Line 59:
assert_subset(by, names(scores)) allows metric columns in by.
- Lines 77-80: data.table groups
by = c(by) (emitting e.g. wis as a grouping column) while .SDcols = metric_cols also includes wis, producing the duplicate column and per-value "groups".
This is the sibling of #1179 (fixed in 0e15277), which only addressed the empty-metrics duplicate-column case.
Intended fix
Per maintainer decision: error (via cli_abort()), not warn-and-drop, when by contains any metric column, consistent with the #1179 precedent, naming the offending column(s). Update the @param by docs and add a NEWS entry.
Part of the bug audit in #1189.
summarise_scores()accepts metric/score columns inbyand silently returns a broken, unsummarised table.Reproduction
The result has a duplicated
wiscolumn (once as a grouping column, once as a "summarised" score), and because the near-unique numericwisvalues are used as groups, nothing is actually aggregated. No warning or error is emitted, and the duplicate column names break downstream data.table operations.Cause
In
R/summarise_scores.R:assert_subset(by, names(scores))allows metric columns inby.by = c(by)(emitting e.g.wisas a grouping column) while.SDcols = metric_colsalso includeswis, producing the duplicate column and per-value "groups".This is the sibling of #1179 (fixed in 0e15277), which only addressed the empty-metrics duplicate-column case.
Intended fix
Per maintainer decision: error (via
cli_abort()), not warn-and-drop, whenbycontains any metric column, consistent with the #1179 precedent, naming the offending column(s). Update the@param bydocs and add a NEWS entry.Part of the bug audit in #1189.