stableIHW

stableIHW provides solver-free independent hypothesis weighting for R. It learns bounded, covariate-dependent weights by deterministic cross-fitting and then applies weighted Benjamini-Hochberg (BH) or weighted Bonferroni adjustment.

The package is intended for analyses where stability is more important than reproducing the linear-programming solution used by the Bioconductor IHW package. It does not depend on IHW or an external mathematical-programming solver.

Important

stableIHW is under active development. Its interface follows common IHW conventions where practical, but its weight-learning algorithm is different. Weighted BH is the default for interface compatibility, but remains experimental. Calling ihw() with weighted BH does not emit a runtime warning; use of the method therefore requires explicitly accepting this limitation. Cross-fitting alone does not establish finite-sample FDR control for the global weighted BH procedure implemented here. Simulation tests provide regression evidence in selected scenarios, not a mathematical proof, and users should validate error-rate control under a data-generating process representative of their application.

Installation

Install the development version from GitHub:

install.packages("remotes")
remotes::install_github("thorohde/stableIHW")

Quick start

library(stableIHW)

set.seed(1)
number_hypotheses <- 5000L
covariate <- runif(number_hypotheses)
pvalue <- runif(number_hypotheses)

# Add signal that is more frequent at high covariate values.
signal <- covariate > 0.8
pvalue[signal] <- rbeta(sum(signal), shape1 = 0.25, shape2 = 1)

result <- ihw(
  pvalues = pvalue,
  covariates = covariate,
  alpha = 0.1,
  nfolds = 5
)

result
rejections(result)
head(as.data.frame(result))
plot(result, what = "weights")

All diagnostic plots are ggplot2 objects. In addition to the learned weights and decision boundaries, the package provides stratified p-value histograms, empirical distribution functions, covariate-strength plots, and fold-balance plots:

plot(result, what = "decisionboundary")
plot(result, what = "pvalue_histogram")
plot(result, what = "pvalue_ecdf")
plot(result, what = "covariate")
plot(result, what = "folds")

Formula input is also supported:

hypotheses <- data.frame(pvalue = pvalue, covariate = covariate)
result <- ihw(pvalue ~ covariate, data = hypotheses, alpha = 0.1)

Use weighted Bonferroni adjustment when family-wise error rate control is the target:

result <- ihw(
  pvalue,
  covariate,
  alpha = 0.05,
  adjustment_type = "bonferroni"
)

Method

The current implementation:

1. divides numeric covariates into quantile groups or uses categorical levels;
2. assigns hypotheses deterministically to folds within each group;
3. estimates group enrichment from the other folds using the proportion of p-values at or below alpha;
4. shrinks group estimates toward the global estimate with a pseudo-count;
5. bounds and normalizes the learned weights; and
6. applies weighted BH or weighted Bonferroni adjustment.

No random fold assignment or external solver is used, so identical inputs and arguments produce identical results.

Important arguments include:

• nbins: maximum number of groups for a numeric covariate; "auto" uses up to 40 groups based on the number of observed p-values;
• nfolds: number of cross-fitting folds; nfolds = 1 deliberately falls back to equal weights;
• regularization_term: pseudo-count controlling shrinkage toward the global enrichment estimate; and
• weight_range: lower and upper bounds for unnormalized learned weights.

Assumptions and interpretation

Independent hypothesis weighting requires a covariate that is informative about power while remaining independent of each null p-value. Examples can include sample size, measurement precision, or an external annotation when their construction does not use the tested null outcome.

Do not use a covariate selected from, or otherwise dependent on, the same null test statistic. Cross-fitting prevents a hypothesis's own p-value from being used to learn its weight, but it cannot repair an invalid covariate or arbitrary dependence among hypotheses.

Weighted BH additionally requires assumptions on dependence among the weighted null p-values, such as independence or an appropriate positive-dependence condition. Weighted Bonferroni does not require independence among hypotheses, but each null p-value must still be valid and independent of its learned cross-fitted weight. Discrete p-values, dependent tests, and small or highly imbalanced covariate groups should be validated for the intended application.

The reported adjusted p-values should be interpreted according to adjustment_type:

• "BH": targets FDR under the assumptions required by weighted BH;
• "bonferroni": targets family-wise error rate.

Statistical validation

The automated test suite includes fixed-seed Monte Carlo checks for:

• weighted BH FDR with independent continuous and discrete null p-values;
• weighted BH FDR under a positively dependent Gaussian-factor null model;
• weighted BH FDR and power with informative, imbalanced categorical groups;
• weighted Bonferroni FWER with continuous and discrete null p-values; and
• weighted Bonferroni FWER under positive dependence.

The complete-null checks use the fact that FDR equals the probability of at least one rejection when every hypothesis is null. Acceptance limits include a three-standard-error Monte Carlo tolerance to avoid treating simulation noise as a deterministic failure.

These checks are deliberately regression tests, not a universal validation or a mathematical proof. Before using the package for consequential analyses, validate it under a data-generating process that reflects the intended application. The automated suite also checks numerical stability for tied covariates, discrete p-values, singleton and imbalanced groups, groups smaller than the number of folds, and many categorical levels.

Compatibility with Bioconductor IHW

stableIHW provides an ihw() entry point, formula input, a stableIHWResult S4 class, and familiar accessors such as pvalues(), adj_pvalues(), weights(), rejections(), and thresholds().

The result class deliberately has a different name from Bioconductor IHW's ihwResult. This prevents S4 class-registration and validity conflicts when both packages are installed or loaded in the same R environment. Because both packages export several functions with the same names, use qualified calls such as stableIHW::ihw() and IHW::ihw() when both are attached.

It is not a drop-in numerical reproduction of Bioconductor IHW:

• the weight learner is solver-free and therefore different;
• LP-specific diagnostics and regularization paths are not reproduced; and
• unsupported original-IHW arguments fail instead of being silently ignored.

Do not expect identical weights, adjusted p-values, or rejection sets between the two packages.

License

MIT © Thomas Rohde