hts fixes with weight values

.claude/rules/r-package.md

@@ -15,7 +15,8 @@ paths:
 - Add a package dependency only for a concrete feature when existing dependencies and a small maintainable implementation are inadequate, unsafe, or would recreate substantial mature functionality. Do not add dependencies for bug fixes, documentation, formatting, developer convenience, or trivial helpers, and do not vendor third-party source.
 - A dependency-changing pull request must explain alternatives, necessity, maintenance/security/license implications, placement in `Imports` or `Suggests`, and version constraints. Validate it with a clean-library installation and `devtools::check()`.
 - Update roxygen comments for public API changes, then run `R -q -e 'devtools::document()'`. Never hand-edit `NAMESPACE` or `man/*.Rd`.
-- Add a `NEWS.md` entry and bump `DESCRIPTION` only for a release or user-visible behavior change. Internal agent-guidance maintenance alone does not require either change.
+- Add a `NEWS.md` entry for user-visible behavior changes; the `DESCRIPTION` version may advance for a release or user-visible change. Internal agent-guidance maintenance alone requires neither a NEWS entry nor a version change.
+- Keep exactly one 9000-series development-version section in `NEWS.md`, aligned with `DESCRIPTION`. When advancing the development version (for example, `.9005` to `.9006`), rename the existing development heading and merge new notes into it; never retain separate sections for successive unreleased development versions. Preserve all accumulated unreleased notes and released-version history.
 - Use short `# description` comments only where the code is not self-explanatory. Do not add decorative comment separators.
 - Keep errors actionable and consistent. Use `stop()` for user-facing errors, `warning()` for recoverable conditions, and `message()` for informational output.
 - Avoid unnecessary copies in tight loops; vectorize when it improves clarity or performance.

AGENTS.md

@@ -14,7 +14,7 @@ Make changes that are testable, documented, backward-compatible where practical,
 
 Detailed rules load conditionally from `.claude/rules/`, a format shared by Claude Code and VS Code. Agents that do not load that directory automatically must read the matching rule before editing:
 
-- `.claude/rules/r-package.md` for R source, package metadata, roxygen, vignettes, and release notes.
+- `.claude/rules/r-package.md` for R source, package metadata, roxygen, vignettes, and release notes, including the single-development-version policy.
 - `.claude/rules/testing.md` for tests, CRAN profiles, credentials, runtime budgets, and PSOCK behavior.
 - `.claude/rules/agent-runtime.md` for Agent graphs, iteration selection and its accuracy signals, LLM sessions, reasoning retries, history, artifacts, combo identity, and EDA prompts.
 - `.claude/rules/multistep.md` for multistep adapters, lag generation, routing, and prediction.

CLAUDE.md

@@ -6,5 +6,6 @@
 
 - Start nontrivial work with a concise plan naming the files likely to change.
 - Load and follow matching path-scoped files under `.claude/rules/`; use `/context` when diagnosing whether instructions were loaded.
+- Follow `.claude/rules/r-package.md` for development-version and release-note changes.
 - Keep project-specific instructions version-controlled. Put personal or machine-specific preferences in an ignored `CLAUDE.local.md`, never in this file.
 - Keep this file limited to Claude-specific behavior. Shared package guidance belongs in `AGENTS.md` or a scoped rule.

NEWS.md

@@ -1,11 +1,12 @@
-# finnts 0.7.0.9006 (DEVELOPMENT VERSION)
+# finnts 0.7.0.9007 (DEVELOPMENT VERSION)
 
 ## Improvements
 
 - Improved model selection to balance backtest accuracy with forecast plausibility, preserve supported growth and seasonal patterns, and reject invalid predictions, with more reliable averaging, forecast updates, and run recovery.
 
 ## Bug Fixes
 
+-   Nonnegative hierarchical reconciliation now limits the ratio between the largest and smallest inverse-error weights to `1e15`, reducing numerical slow-zone failures caused by near-perfect back-test fits. The same rule applies to standard, grouped, and Agent reconciliation without changing the HTS dependency or adding runtime timeouts. Extreme-weight cases may produce different forecasts, particularly for small series; valid weights within the limit and reconciliation that allows negative forecasts retain their previous calculation. Invalid residual variances now produce an actionable reconciliation error.
 -   Local and ADLS-mounted workflows now read known series inputs, recipes, EDA results, model outputs, and completion artifacts by exact path. Necessary directory listings are reused during model preparation, EDA aggregation, and condensed forecast reads. Standalone forecast getters discover unknown condensed batches once and preserve their precedence even when the first batch is absent. Exact-file validation is not repeated before reading, and local CSV read-time metadata and I/O failures propagate instead of becoming empty or partial results. Valid empty CSV files and optional missing artifacts retain their supported behavior. Default legacy reads, remote-provider downloads, and Spark data-frame routing are unchanged.
 -   `update_forecast()` now excludes predecessor time series that are absent from the current input before global or local update routing. Removed series no longer produce empty-schema or missing-artifact fallback errors, while current-only series continue to receive default local forecasts.
 -   Global Agent iterations now use hierarchy choices that match the outer optimization scope. Bottom-level runs may compare `bottoms_up` with the exact standard or grouped hierarchy detected by EDA after reconciliation to bottom-level series. Runs whose input was already expanded to hierarchy-level `ID` series use `bottoms_up` for every inner global and local iteration before one final outer reconciliation.

R/agent_update_forecast.R

@@ -2727,15 +2727,17 @@ reconcile <- function(initial_fcst,
         dplyr::select(tidyselect::all_of(hts_combo_list)) %>%
         as.matrix()
 
+      weights <- reconciliation_weights(residuals_tbl, negative_forecast)
+
       if (forecast_approach == "standard_hierarchy") {
         ts_combined <- data.frame(hts::combinef(ts,
-          nodes = hts_nodes, weights = (1 / colMeans(residuals_tbl^2, na.rm = TRUE)),
+          nodes = hts_nodes, weights = weights,
           keep = "bottom", nonnegative = !negative_forecast
         ))
         colnames(ts_combined) <- original_combo_list
       } else if (forecast_approach == "grouped_hierarchy") {
         ts_combined <- data.frame(hts::combinef(ts,
-          groups = hts_nodes, weights = (1 / colMeans(residuals_tbl^2, na.rm = TRUE)),
+          groups = hts_nodes, weights = weights,
           keep = "bottom", nonnegative = !negative_forecast
         ))
         colnames(ts_combined) <- original_combo_list

R/hierarchy.R

@@ -473,6 +473,41 @@ get_hts_nodes <- function(hts_object,
   }
 }
 
+#' Calculate finite reconciliation weights with bounded nonnegative confidence
+#'
+#' @param residuals Numeric matrix with one column per hierarchy node.
+#' @param negative_forecast Whether negative forecasts are allowed.
+#' @return Named inverse-MSE weights in the original column order.
+#' @noRd
+reconciliation_weights <- function(residuals, negative_forecast) {
+  if (!is.matrix(residuals) || !is.numeric(residuals) || any(dim(residuals) == 0L)) {
+    stop("Reconciliation requires a nonempty numeric residual matrix.", call. = FALSE)
+  }
+
+  mse <- colMeans(residuals^2, na.rm = TRUE)
+  invalid <- which(!is.finite(mse) | mse <= 0)
+  if (length(invalid) > 0L) {
+    nodes <- if (is.null(names(mse))) invalid else names(mse)[invalid]
+    stop(paste0(
+      "Reconciliation requires finite positive residual MSE for every node. Check back-test residuals for: ",
+      paste(nodes, collapse = ", "), "."
+    ), call. = FALSE)
+  }
+
+  if (!negative_forecast) {
+    max_weight_ratio <- 1e15
+    mse <- pmax(mse, max(mse) / max_weight_ratio)
+  }
+  weights <- 1 / mse
+  if (any(!is.finite(weights) | weights <= 0)) {
+    stop("Reconciliation requires finite positive weights. Check the scale of the back-test residuals.",
+      call. = FALSE
+    )
+  }
+
+  weights
+}
+
 #' Reconcile hierarchical forecasts down to lowest bottoms up level
 #'
 #' @param run_info run info
@@ -683,15 +718,17 @@ reconcile_hierarchical_data <- function(run_info,
               dplyr::select(tidyselect::all_of(hts_combo_list)) %>%
               as.matrix()
 
+            weights <- reconciliation_weights(residuals_tbl, negative_forecast)
+
             if (forecast_approach == "standard_hierarchy") {
               ts_combined <- data.frame(hts::combinef(ts,
-                nodes = hts_nodes, weights = (1 / colMeans(residuals_tbl^2, na.rm = TRUE)),
+                nodes = hts_nodes, weights = weights,
                 keep = "bottom", nonnegative = !negative_forecast
               ))
               colnames(ts_combined) <- original_combo_list
             } else if (forecast_approach == "grouped_hierarchy") {
               ts_combined <- data.frame(hts::combinef(ts,
-                groups = hts_nodes, weights = (1 / colMeans(residuals_tbl^2, na.rm = TRUE)),
+                groups = hts_nodes, weights = weights,
                 keep = "bottom", nonnegative = !negative_forecast
               ))
               colnames(ts_combined) <- original_combo_list
@@ -908,15 +945,17 @@ reconcile_hierarchical_data <- function(run_info,
               dplyr::select(tidyselect::all_of(hts_combo_list)) %>%
               as.matrix()
 
+            weights <- reconciliation_weights(residuals_tbl, negative_forecast)
+
             if (forecast_approach == "standard_hierarchy") {
               ts_combined <- data.frame(hts::combinef(ts,
-                nodes = hts_nodes, weights = (1 / colMeans(residuals_tbl^2, na.rm = TRUE)),
+                nodes = hts_nodes, weights = weights,
                 keep = "bottom", nonnegative = !negative_forecast
               ))
               colnames(ts_combined) <- original_combo_list
             } else if (forecast_approach == "grouped_hierarchy") {
               ts_combined <- data.frame(hts::combinef(ts,
-                groups = hts_nodes, weights = (1 / colMeans(residuals_tbl^2, na.rm = TRUE)),
+                groups = hts_nodes, weights = weights,
                 keep = "bottom", nonnegative = !negative_forecast
               ))
               colnames(ts_combined) <- original_combo_list