rps_ordinal() and logs_categorical() mishandle permuted predicted_label; missing ncol check for n > 1

[nikosbosse]

Three related bugs in the handling of predicted_label and matrix dimensions for categorical (nominal/ordinal) metrics. Note: the score() pipeline sorts predictions and labels into level order before calling the metrics (R/class-forecast-ordinal.R:151-157), so only direct calls to the exported metric functions are affected.

1. rps_ordinal() applies the wrong permutation when predicted_label is not in level order

R/metrics-ordinal.R:75-76 uses the forward permutation predicted[, as.numeric(predicted_label)] instead of the inverse predicted[, order(as.numeric(predicted_label))], so the score is not invariant to how the columns of predicted are labelled:

factor_levels <- c("one", "two", "three")
predicted_label <- factor(factor_levels, levels = factor_levels, ordered = TRUE)
observed <- factor(c("three", "three", "two"), levels = factor_levels, ordered = TRUE)
predicted <- matrix(
  c(0.8, 0.1, 0.1,
    0.1, 0.2, 0.7,
    0.4, 0.4, 0.2),
  nrow = 3, byrow = TRUE
)
rps_ordinal(observed, predicted, predicted_label)
#> [1] 1.45 0.10 0.20   # correct (matches manual CDF calculation)

# same forecast, columns permuted together with their labels:
perm <- c(2, 3, 1)
rps_ordinal(observed, predicted[, perm],
            factor(factor_levels[perm], levels = factor_levels, ordered = TRUE))
#> [1] 0.82 1.13 0.20   # should be identical to the above

The existing test at tests/testthat/test-metrics-ordinal.R:122-127 encodes the same wrong permutation and needs correcting (predicted[, c(3, 1, 2)] should be predicted[, order(c(3, 1, 2))] = predicted[, c(2, 3, 1)]).

2. logs_categorical() ignores predicted_label entirely

R/metrics-nominal.R:133-134 indexes predicted[cbind(1:n, as.numeric(observed))] without ever consulting predicted_label, so columns are assumed to be in factor-level order:

predicted_label <- factor(c("one", "two", "three"), levels = factor_levels)
observed <- factor(c("one", "three", "two"), levels = factor_levels)
logs_categorical(observed, predicted, predicted_label)
#> [1] 0.2231436 0.3566749 0.9162907   # correct (row 2 = -log(0.7))

perm <- c(2, 3, 1)
logs_categorical(observed, predicted[, perm],
                 factor(factor_levels[perm], levels = factor_levels))
#> [1] 2.302585 2.302585 1.609438   # row 2 = -log(0.1), probability of the wrong outcome

3. assert_input_categorical() misses the column-count check for n > 1

R/metrics-nominal.R:72 (n > 1 branch) is assert_matrix(predicted, nrows = n) with no ncols check, while the n == 1 branch (lines 64-69) enforces N columns. A 2x4 matrix with rows summing to 1 is silently accepted and scored meaninglessly:

logs_categorical(
  factor(c("one", "two"), levels = factor_levels),
  matrix(c(0.2, 0.1, 0.4, 0.3, 0.1, 0.25, 0.25, 0.4), nrow = 2, byrow = TRUE),
  predicted_label
)
#> [1] 0.9162907 1.3862944   # no error, but rows are not distributions over the 3 outcomes

The identical 1x4 input with n == 1 correctly errors ("Must have exactly 3 cols, but has 4 cols").

Intended fix

• rps_ordinal(): reorder with the inverse permutation, predicted[, order(as.numeric(predicted_label)), drop = FALSE].
• logs_categorical(): apply the same reordering before indexing so the label mapping is respected (NA-observed and single-observation behaviour preserved).
• assert_input_categorical(): add ncols = N to the n > 1 assert_matrix() call, matching the n == 1 branch. This is a new error on previously (silently, wrongly) accepted input.
• Correct the wrong expectation in tests/testthat/test-metrics-ordinal.R and add permutation-invariance tests.

Part of the bug audit in #1189.