1211: Fix sample/quantile validation type checks and PIT histogram/rounding messages#1216
Draft
nikosbosse wants to merge 1 commit into
Draft
1211: Fix sample/quantile validation type checks and PIT histogram/rounding messages#1216nikosbosse wants to merge 1 commit into
nikosbosse wants to merge 1 commit into
Conversation
- assert_forecast() for sample, quantile and multivariate sample forecasts now errors when observed or predicted are not numeric, instead of the failure only surfacing later as per-metric warnings inside score() - the rounding warning in as_forecast_quantile() now correctly says round(x, digits = 9), matching the code, and closes its parenthesis - get_pit_histogram.forecast_quantile() now emits both sentences of its warning (the second previously landed in the condition class) and actually falls back to the quantiles present in the forecast: requested quantiles that are present are kept; if none besides 0 and 1 remain, all present quantiles are used. Previously the function returned an empty data.table (num_bins) or silently recycled wrong densities (breaks). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1216 +/- ##
==========================================
+ Coverage 98.15% 98.38% +0.23%
==========================================
Files 41 41
Lines 2225 2235 +10
==========================================
+ Hits 2184 2199 +15
+ Misses 41 36 -5 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Collaborator
Author
|
Automated review (Claude Fable, directed by @nikosbosse): Verified:
Minor observations (pre-existing, not introduced by this PR, no action required):
Verdict: approve. |
nikosbosse
marked this pull request as draft
July 18, 2026 13:53
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR closes #1211.
Three related validation/messaging bugs, all part of the bug audit in #1189:
as_forecast_sample(),as_forecast_quantile()andas_forecast_multivariate_sample()accepted non-numericobserved/predictedcolumns (e.g. character), with the failure only surfacing later insidescore()as one warning per metric and an effectively empty scores table. The correspondingassert_forecast()methods now checkcheckmate::assert_numeric()on both columns, matching the behaviour of the binary/point (and multivariate point) methods, which already errored at validation time.as_forecast_quantile()claimedround(x, digits = 10)while the code executesround(x, digits = 9). The message now says 9 digits (the code is intentionally unchanged: 10-digit rounding would not collapse diffs of exactly 1e-10, the trigger threshold) and the dangling parenthesis is closed.get_pit_histogram.forecast_quantile(): thecli_warn()call passed the second sentence as an unnamed second argument, which rlang interprets as the condition class, so users never saw "The PIT histogram will be based on the quantiles present in the forecast." — and that fallback never actually happened:num_bins = 7returned a 0-row table, and partially-presentbreaksproduced recycling warnings with silently wrong densities. The warning now shows both lines, and the fallback is implemented: requested quantiles present in the forecast are kept; if none besides 0 and 1 remain, all present quantiles are used.diffsis computed after this adjustment so densities, bins and mids stay aligned. The default (num_bins = NULL) path is unchanged.Failing-before evidence (new tests run against unfixed code):
After the fix, the affected test files and the full suite pass (0 failed, 922 passed).
Dev note: bug identified by an LLM audit (#1189); fix and tests implemented with LLM support, directed by @nikosbosse.
Checklist
@importFromtags were touched and the required imports already exist in NAMESPACE)lintr::lint_package()to check for style issues introduced by my changes.