This library contains the Python implementation of Venn-ABERS calibration for binary and multiclass classification problems.
pip install venn-abers
The method can be applied on top of an underlying scikit-learn algorithm.
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
from sklearn.naive_bayes import GaussianNB
from venn_abers import VennAbersCalibrator
X, y = make_classification(n_samples=1000, n_classes=3, n_informative=10)
X_train, X_test, y_train, y_test = train_test_split(X, y)
clf = GaussianNB()
# Define Venn-ABERS calibrator
va = VennAbersCalibrator(estimator=clf, inductive=True, cal_size=0.2, random_state=101)
# Fit on the training set
va.fit(X_train, y_train)
# Generate probabilities and class predictions on the test set
p_prime = va.predict_proba(X_test)
y_pred = va.predict(X_test)The venn-abers library is fully compatible natively with the scikit-learn ecosystem! It seamlessly inherits from BaseEstimator allowing it to be dropped directly into workflows like Pipeline, GridSearchCV, and cross_val_score.
Furthermore, this compatibility is 100% backwards-compatible with all previous implementations of the library, so no existing code will break.
You can also pass **kwargs directly into the .fit() method to route parameters dynamically to the underlying estimator (e.g. passing eval_set to XGBoost):
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from xgboost import XGBClassifier
from venn_abers import VennAbersCalibrator
# Passing kwargs through fit explicitly
va = VennAbersCalibrator(estimator=XGBClassifier(), inductive=True, cal_size=0.2)
va.fit(X_train, y_train, eval_set=[(X_val, y_val)], early_stopping_rounds=10)
# Scikit-learn Pipeline Integration
pipeline = Pipeline([
('scaler', StandardScaler()),
('va', VennAbersCalibrator(estimator=GaussianNB(), inductive=True, cal_size=0.2))
])
pipeline.fit(X_train, y_train)
p_prime_pipeline = pipeline.predict_proba(X_test)Further examples can be found in the github repository https://github.com/ip200/venn-abers in the examples folder:
- simple_classification.ipynb for a simple example in the binary classification setting.
- multiclass_classification.ipynb for the multiclass setting.
- comparison_with_existing.ipynb for the comparison with a previous github implementation.
- ivar_example.ipynb for an example of Inductive Venn-ABERS for regression.
- scikit_learn_integration.ipynb for an example of the scikit-learn native integration.
The venn-abers library has been used or referenced in several academic works related to probability calibration, uncertainty estimation, and conformal prediction.
-
Kazantsev, D. (2025)
Adaptive Set‑Mass Calibration with Conformal Prediction.
arXiv:2505.15437 -
Rabenseifner, M. (2025)
Calibration Strategies for Robust Causal Estimation: Theoretical and Empirical Insights on Propensity Score Based Estimators.
arXiv:2503.17290 -
Löfström et al. (2024)
Calibrated Explanations: With Uncertainty Information and Counterfactuals.
Expert Systems with Applications.
These papers apply Venn‑Abers, IVAP, or CVAP calibration methods.
-
Manokhin and Grønhaug (2025)
Classifier Calibration at Scale: An Empirical Study of Model-Agnostic Post-Hoc Methods. -
Boger et al. (2025)
Functional protein mining with conformal guarantees. -
van der Laan & Alaa (2025)
Generalized Venn and Venn‑Abers Calibration with Applications in Conformal Prediction. -
Johansson, Löfström & Sönströd (2023)
Well‑Calibrated Probabilistic Predictive Maintenance using Venn‑Abers. -
Pereira et al. (2020)
Targeting the Uncertainty of Predictions at Patient Level.
Journal of Biomedical Informatics.
For a more comprehensive list please see Google Scholar
- MAPIE – Model Agnostic Prediction Interval Estimator
MAPIE includes calibration utilities referencing Venn‑Abers predictors.
https://github.com/scikit-learn-contrib/MAPIE
- Vovk & Petej (2014) — Venn‑Abers Predictors.
- Vovk, Petej & Fedorova (2015) — Large‑scale probabilistic predictors with validity guarantees.
- Manokhin (2017) — Multi‑class probabilistic classification using inductive and cross Venn‑Abers predictors.
If you use this library in academic work, please cite:
@software{petej_venn_abers,
author = {Petej, Ivan},
title = {venn-abers: Venn-Abers calibration for probabilistic classifiers},
year = {2024},
url = {https://github.com/ip200/venn-abers}
}You may also cite the foundational paper:
@inproceedings{vovk2014venn,
title={Venn-Abers predictors},
author={Vovk, Vladimir and Petej, Ivan},
booktitle={Proceedings of the Thirtieth Conference on Uncertainty in Artificial Intelligence},
pages={829--838},
year={2014}
}Contributions and pull requests are welcome.
If you are aware of additional research using this library, please open a PR to add it to the Academic Usage section.
MIT License