Two related bugs in bias_quantile() / bias_quantile_single_vector() (R/metrics-quantile.R):
Bug 1: Wrong results with unsorted quantile levels
The same forecast gives different bias values depending on the order in which quantile levels are passed:
bias_quantile(observed = 1.5, c(1, 2, 3), c(0.25, 0.5, 0.75))
#> [1] 0.5
bias_quantile(observed = 1.5, c(3, 1, 2), c(0.75, 0.25, 0.5))
#> [1] -0.5
Cause (R/metrics-quantile.R:480-481): in bias_quantile_single_vector(),
order <- order(quantile_level)
predicted <- predicted[order]
sorts predicted only — quantile_level stays unsorted. interpolate_median() then picks predicted[quantile_level == 0.5] from the reordered predictions (here 3 instead of the true median 2), and max(quantile_level[predicted <= observed]) pairs sorted predictions with unsorted levels. Some permutations happen to error with "Predictions must not be decreasing"; others silently return a wrong value.
Bug 2: Crash when na.rm = TRUE removes all quantile levels on one side of the median
bias_quantile(observed = 2, c(NA, NA, 3), c(0.25, 0.5, 0.75), na.rm = TRUE)
#> Error in if (observed == median_prediction) { : argument is of length zero
#> In addition: Warning message:
#> In max(quantile_level[quantile_level < 0.5]) :
#> no non-missing arguments to max; returning -Inf
Cause: after NA removal only quantile_level = 0.75 survives. In interpolate_median() (R/metrics-quantile.R:528), max(quantile_level[quantile_level < 0.5]) is max(numeric(0)), which warns and returns -Inf, so median_prediction has length 0 and the comparison at R/metrics-quantile.R:493 crashes. The top-level asserts at lines 425-426 only check the pre-NA-removal quantile levels, so they do not protect this path.
Intended fix
- Sort
quantile_level together with predicted (add quantile_level <- quantile_level[order]), so predictions and levels stay paired.
- After NA removal, return
NA_real_ silently when no quantile level <= 0.5 or none >= 0.5 remains (median non-interpolable) — consistent with na.rm = FALSE, which returns NA_real_ silently.
Part of the bug audit in #1189.
Two related bugs in
bias_quantile()/bias_quantile_single_vector()(R/metrics-quantile.R):Bug 1: Wrong results with unsorted quantile levels
The same forecast gives different bias values depending on the order in which quantile levels are passed:
Cause (R/metrics-quantile.R:480-481): in
bias_quantile_single_vector(),sorts
predictedonly —quantile_levelstays unsorted.interpolate_median()then pickspredicted[quantile_level == 0.5]from the reordered predictions (here 3 instead of the true median 2), andmax(quantile_level[predicted <= observed])pairs sorted predictions with unsorted levels. Some permutations happen to error with "Predictions must not be decreasing"; others silently return a wrong value.Bug 2: Crash when
na.rm = TRUEremoves all quantile levels on one side of the medianCause: after NA removal only
quantile_level = 0.75survives. Ininterpolate_median()(R/metrics-quantile.R:528),max(quantile_level[quantile_level < 0.5])ismax(numeric(0)), which warns and returns-Inf, somedian_predictionhas length 0 and the comparison at R/metrics-quantile.R:493 crashes. The top-level asserts at lines 425-426 only check the pre-NA-removal quantile levels, so they do not protect this path.Intended fix
quantile_leveltogether withpredicted(addquantile_level <- quantile_level[order]), so predictions and levels stay paired.NA_real_silently when no quantile level<= 0.5or none>= 0.5remains (median non-interpolable) — consistent withna.rm = FALSE, which returnsNA_real_silently.Part of the bug audit in #1189.