CRE-Router 0.1.0

Initial public release of CRE-Router, the reference implementation of the paper Cluster, Route, Escalate: Cascaded Framework for Cost-Aware LLM Serving.

Highlights

• Stage 1, clustering-based routing. k-means over query embeddings, Silhouette k-selection, cost-aware routing (Error + lambda * Cost), Pareto pruning, and budgeted lambda* selection.
• Stage 2, quality-estimation cascade. A ModernBERT accept/escalate classifier with a multi-model escalation ladder derived from the routing arithmetic.
• Live router service. A LiteLLM-backed cascade over vLLM backends, with routing-decision headers, /stats, and an optional decision log.
• Full paper reproduction guide.

Install

pip install "cre-router[full]"

Links

• PyPI: https://pypi.org/project/cre-router/0.1.0/
• Paper: https://arxiv.org/abs/2606.27457
• Reproducing the paper: https://github.com/ymoslem/CRE-Router/blob/main/REPRODUCE.md