Two related edge-case bugs in the quantile-based metrics.
Bug 1: interval_coverage() fails on seq()-generated quantile levels (floating point)
interval_coverage() checks for the required quantile levels with an exact floating-point comparison (R/metrics-quantile.R, line 338: !all(necessary_quantiles %in% quantile_level)). Quantile levels generated with seq() are not exactly representable, so the check fails even though the required quantiles are present:
ql <- seq(0.05, 0.95, 0.05)
0.35 %in% ql
#> [1] FALSE # element is 0.35000000000000003
interval_coverage(observed, predicted, ql, interval_range = 30)
#> Error in `interval_coverage()`:
#> ! To compute the interval coverage for an interval range of "30%", the
#> 0.35 and 0.65 quantiles are required
interval_coverage(observed, predicted, ql, interval_range = 50)
#> Error in `interval_coverage()`:
#> ! To compute the interval coverage for an interval range of "50%", the
#> 0.25 and 0.75 quantiles are required
The downstream row filter (line 351) is already safe because get_range_from_quantile() rounds ranges to 10 decimal places (R/helper-quantile-interval-range.R).
Fix: round both sides to 10 decimal places before matching, consistent with the package's existing convention (round(x, 10) in R/class-forecast-quantile.R and R/class-forecast-sample.R).
Bug 2: wis() / interval_score() / quantile_score(weigh = FALSE) return NaN for quantile levels 0 and 1
Quantile levels 0 and 1 are explicitly permitted by assert_input_quantile() (bounds lower = 0, upper = 1) and occur in real forecast hub data ("min"/"max" quantiles). But the 0/1 pair forms a 100% interval with alpha = 0, and interval_score() (R/metrics-interval-range.R, lines 168-171) computes 2 / alpha * (lower - observed) * indicator, which is Inf * 0 = NaN even when the observation lies inside the interval:
wis(5, matrix(c(1, 3, 5, 7, 9), nrow = 1), c(0, 0.25, 0.5, 0.75, 1))
#> [1] NaN
quantile_score(5, matrix(c(1, 3, 5, 7, 9), nrow = 1), c(0, 0.25, 0.5, 0.75, 1))
#> [1] 0.4
This breaks the documented identity WIS = mean of quantile scores. quantile_score(weigh = FALSE) has the same alpha = 0 division (R/metrics-quantile.R, line 692, 2 * score / alpha -> 0/0 = NaN).
Fix (per maintainer decision): apply the indicator before the 2/alpha factor and cancel 2/alpha * alpha/2 analytically in the weighted case, so quantile levels 0/1 give finite, correct scores when the observation is inside the interval. Unweighted interval_score(weigh = FALSE) returns Inf when the observation falls outside a 100% interval, which is mathematically correct. The sibling quantile_score(weigh = FALSE) NaN is fixed in the same way (0-valued weighted scores stay 0 instead of becoming 0/0). Verified numerically that outputs for all other interval ranges are unchanged and that the WIS = mean quantile score identity is restored to within 1e-14 with levels 0/1 included.
Part of the bug audit in #1189.
Two related edge-case bugs in the quantile-based metrics.
Bug 1:
interval_coverage()fails onseq()-generated quantile levels (floating point)interval_coverage()checks for the required quantile levels with an exact floating-point comparison (R/metrics-quantile.R, line 338:!all(necessary_quantiles %in% quantile_level)). Quantile levels generated withseq()are not exactly representable, so the check fails even though the required quantiles are present:The downstream row filter (line 351) is already safe because
get_range_from_quantile()rounds ranges to 10 decimal places (R/helper-quantile-interval-range.R).Fix: round both sides to 10 decimal places before matching, consistent with the package's existing convention (
round(x, 10)inR/class-forecast-quantile.RandR/class-forecast-sample.R).Bug 2:
wis()/interval_score()/quantile_score(weigh = FALSE)returnNaNfor quantile levels 0 and 1Quantile levels 0 and 1 are explicitly permitted by
assert_input_quantile()(boundslower = 0, upper = 1) and occur in real forecast hub data ("min"/"max" quantiles). But the 0/1 pair forms a 100% interval withalpha = 0, andinterval_score()(R/metrics-interval-range.R, lines 168-171) computes2 / alpha * (lower - observed) * indicator, which isInf * 0 = NaNeven when the observation lies inside the interval:This breaks the documented identity WIS = mean of quantile scores.
quantile_score(weigh = FALSE)has the samealpha = 0division (R/metrics-quantile.R, line 692,2 * score / alpha->0/0 = NaN).Fix (per maintainer decision): apply the indicator before the
2/alphafactor and cancel2/alpha * alpha/2analytically in the weighted case, so quantile levels 0/1 give finite, correct scores when the observation is inside the interval. Unweightedinterval_score(weigh = FALSE)returnsInfwhen the observation falls outside a 100% interval, which is mathematically correct. The siblingquantile_score(weigh = FALSE)NaN is fixed in the same way (0-valued weighted scores stay 0 instead of becoming0/0). Verified numerically that outputs for all other interval ranges are unchanged and that the WIS = mean quantile score identity is restored to within 1e-14 with levels 0/1 included.Part of the bug audit in #1189.