Skip to content

summarise_scores() accepts metric columns in by, returning an unsummarised table #1204

Description

@nikosbosse

summarise_scores() accepts metric/score columns in by and silently returns a broken, unsummarised table.

Reproduction

library(scoringutils)
scores <- score(as_forecast_quantile(example_quantile))
nrow(scores)
#> [1] 887
out <- summarise_scores(scores, by = c("model", "wis"))
nrow(out)
#> [1] 887   # same as input -- nothing was summarised
colnames(out)
#> [1] "model" "wis" "wis" "overprediction" "underprediction" "dispersion"
#> [7] "bias" "interval_coverage_50" "interval_coverage_90" "ae_median"

The result has a duplicated wis column (once as a grouping column, once as a "summarised" score), and because the near-unique numeric wis values are used as groups, nothing is actually aggregated. No warning or error is emitted, and the duplicate column names break downstream data.table operations.

Cause

In R/summarise_scores.R:

  • Line 59: assert_subset(by, names(scores)) allows metric columns in by.
  • Lines 77-80: data.table groups by = c(by) (emitting e.g. wis as a grouping column) while .SDcols = metric_cols also includes wis, producing the duplicate column and per-value "groups".

This is the sibling of #1179 (fixed in 0e15277), which only addressed the empty-metrics duplicate-column case.

Intended fix

Per maintainer decision: error (via cli_abort()), not warn-and-drop, when by contains any metric column, consistent with the #1179 precedent, naming the offending column(s). Update the @param by docs and add a NEWS entry.

Part of the bug audit in #1189.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions