get_pairwise_comparisons() (and therefore add_relative_skill()) aborts entirely when any subgroup defined by by has fewer than two comparators, even if other subgroups could be compared just fine. The error message also does not say which subgroup caused the problem.
Reproduction
library(scoringutils)
scores <- score(example_quantile)
# keep only one model for Deaths; Cases still has 3 models
scores_sub <- scores[!(target_type == "Deaths" & model != "EuroCOVIDhub-ensemble")]
scores_sub[, .(n_models = length(unique(model))), by = target_type]
#> target_type n_models
#> 1: Cases 3
#> 2: Deaths 1
get_pairwise_comparisons(scores_sub, by = "target_type")
#> Error in pairwise_comparison_one_group(scores = scores, metric = metric, :
#> ! There are not enough comparators to do any comparison
The Cases group alone would return 9 perfectly valid rows:
nrow(get_pairwise_comparisons(scores[target_type == "Cases"], by = "target_type"))
#> [1] 9
Cause
pairwise_comparison_one_group() calls cli_abort() when a group has fewer than two comparators (R/pairwise-comparisons.R:273-277). get_pairwise_comparisons() lapply()s it over split(scores, by = by) (R/pairwise-comparisons.R:221-234) with no per-group handling, so a single small subgroup aborts the whole computation.
Intended fix
- In
get_pairwise_comparisons(), subgroups with fewer than two comparators are skipped with a warning that names the offending subgroup(s) (formatted as col=value, e.g. "target_type=Deaths"); results are returned for the remaining groups.
add_relative_skill() consequently fills NA for the skipped groups (its merge already uses all.x = TRUE).
- If all subgroups have fewer than two comparators (including
by = NULL with a single model), the existing error is kept unchanged.
- The safeguard inside
pairwise_comparison_one_group() stays untouched for direct calls.
Part of the bug audit in #1189.
get_pairwise_comparisons()(and thereforeadd_relative_skill()) aborts entirely when any subgroup defined bybyhas fewer than two comparators, even if other subgroups could be compared just fine. The error message also does not say which subgroup caused the problem.Reproduction
The Cases group alone would return 9 perfectly valid rows:
Cause
pairwise_comparison_one_group()callscli_abort()when a group has fewer than two comparators (R/pairwise-comparisons.R:273-277).get_pairwise_comparisons()lapply()s it oversplit(scores, by = by)(R/pairwise-comparisons.R:221-234) with no per-group handling, so a single small subgroup aborts the whole computation.Intended fix
get_pairwise_comparisons(), subgroups with fewer than two comparators are skipped with a warning that names the offending subgroup(s) (formatted ascol=value, e.g."target_type=Deaths"); results are returned for the remaining groups.add_relative_skill()consequently fillsNAfor the skipped groups (its merge already usesall.x = TRUE).by = NULLwith a single model), the existing error is kept unchanged.pairwise_comparison_one_group()stays untouched for direct calls.Part of the bug audit in #1189.