Quantitative Assessment of Functional Impact
Not "is this variant pathogenic?" but "how much function is left?"
Most variant-effect tools return a binary pathogenic/benign call. Real variants live on a continuum — partial loss of function, wild-type-like, occasionally gain of function — and a yes/no label throws that away. QAFI predicts the deep-mutational-scanning score itself, on a normalized scale where 1 = wild-type-like, < 1 = impaired, > 1 = possible gain of function: the same axis a wet-lab multiplexed assay would produce.
Experimental DMS data exists for only a few hundred of ~20,000 human proteins. QAFI learns transferable sequence, structure and energetic regularities from 60 such assays and transfers them to proteins with no experimental data at all.
| Blind competition | CAGI7 rank 1 of all submissions on the LPL challenge (all four metrics) and rank 1 on ATP7B missense |
| vs foundation models | Median Pearson 0.480 on 393 Beltran domains — matching EVE (0.480) and ESM-1v (0.487), far ahead of AlphaMissense (0.308) |
| Compute | CPU-only gradient boosting on 28 interpretable features. No GPU, no pretrained LLM |
| Clinical transfer | ClinVar AUROC 0.914; adding QAFI to a 5-tool meta-predictor lifts it +0.010 AUC |
| Scale | 292,500 training variants across 60 proteins and 55 DMS references |
A binary classifier tells you a variant is damaging. It cannot tell you whether the protein retains 80 % of its activity or 5 % — a distinction that matters for dosage-sensitive genes, for modifier interpretation, and for anything resembling a genotype-phenotype curve.
QAFI targets the normalized DMS score directly (Gray et al. 2018 normalization: centre so wild-type = 0, orient so higher means more functional, rescale by the 1st-percentile median). The output is a continuous functional-activity estimate, not a probability.
The engineering problem this creates: a model trained on protein A's assay must predict protein B, whose assay uses a different readout, dynamic range and cell system. QAFI's answer is a two-level ensemble — train one model per reference protein, then aggregate across reference proteins — with everything validated leave-one-protein-out.
The repository ships two families. Pick by whether your protein already has experimental data:
| Strategy | Use it when | How it works | Methods |
|---|---|---|---|
| PSP — Per-protein Score Predictor | Your protein has its own DMS data | Trains a regression model on that single protein, validated leave-one-position-out | 7 variants: psp2_mlr, psp2_xgb, psp2_rfr, pspsplit1, pspsplit2, pspsplit2_obs, pspsplit2_fusion |
| QAFI — cross-protein ensemble | Your protein has no DMS data | Trains one model per reference protein, then aggregates across them by median or protein-similarity weighting | 4 variants: qafi2, qafisplit1, qafisplit2, qafisplit3 |
Both share the same 28-feature representation, so a protein can be scored either way and the results are directly comparable. qafisplit3 is the deployed default for unseen proteins.
flowchart TB
subgraph S0["① Data assembly — scripts/data/"]
A["ProteinGym<br/>53 DMS datasets"] --> D["Gray normalization<br/>WT = 1 · impaired < 1"]
B["Gray · Reeb · Dunham<br/>Riesselman · Frazer<br/>30 proteins"] --> D
C["PISCES 11,178 chains<br/>→ 4,783 PDB↔UniProt maps"] --> E["Residue position maps"]
D --> F["DB60 · 292,500 variants<br/>60 proteins · 55 references"]
end
subgraph S1["② Features — src/qafi/feature/"]
F & E --> G["23-block dependency-resolved<br/>feature registry"]
G --> H1["Evolutionary ×4<br/>PSSM · Shannon entropy · BLOSUM62"]
G --> H2["Structural ×2<br/>AlphaFold pLDDT"]
G --> H3["Neighborhood / energetic ×13<br/>NCE-NR · colasi · MJ potential<br/>neco · lpar · laar"]
G --> H4["PDFF ×9<br/>GMM density log-ratios"]
H1 & H2 & H3 & H4 --> I["28 features per variant<br/>10 of them position-level"]
end
subgraph S2["③ PSP — per-protein baseline"]
I --> J["Leave-one-position-out CV<br/>within a single protein"]
J --> K["MLR · XGBoost · RF<br/>+ two-stage split variants"]
end
subgraph S3["④ QAFI — cross-protein ensemble"]
I --> L["For each of 30 reference proteins:"]
L --> M["Stage 1: LinearRegression<br/>10 position features<br/>→ median_pos_predicted"]
M --> N["Stage 2: XGBoost on 28 features<br/>predicts score or residual"]
N --> O["Aggregate across proteins<br/>median · or similarity-weighted"]
O --> P["Functional score<br/>for an unseen protein"]
end
subgraph S4["⑤ Benchmarks"]
P --> Q["Beltran DMS · 402 proteins<br/>ClinVar · 42,177 variants<br/>Test30 hold-out · 30 proteins<br/>HGMD GOF/LOF"]
end
style S0 fill:#e8f4f8,stroke:#4a90a4
style S1 fill:#f0f4e8,stroke:#7a9a4a
style S2 fill:#eeeeee,stroke:#888888
style S3 fill:#f8e8f0,stroke:#a44a7a
style S4 fill:#fff4e0,stroke:#c49a3a
The insight behind the split model family is that position tolerance and substitution severity are separable problems. Some residues cannot take any substitution; others tolerate everything. Predicting that first, then predicting the residual, works better than predicting the variant score in one shot.
flowchart LR
A["Variant<br/>28 features"] --> B["Stage 1<br/>LinearRegression<br/>10 position-level features"]
B --> C["median_pos_predicted<br/>= how tolerant is this residue?"]
A --> D["Stage 2<br/>XGBoost"]
C --> D
D --> E1["qafisplit1<br/>position pred as extra feature"]
D --> E2["qafisplit2<br/>predicts the RESIDUAL<br/>final = pos_pred + residual"]
D --> E3["qafisplit3<br/>trains on ALL proteins<br/>fuses by similarity weights"]
style B fill:#e8f0f8,stroke:#4a7aa4
style E3 fill:#f0f8e8,stroke:#6a9a4a
Stage 1 alone — pure position-level tolerance — reaches median Pearson 0.580 across the 60-protein database (max 0.840), and 0.7–0.9 on Beltran positional medians.
qafisplit3 is the deployed variant. Rather than taking a plain median across per-protein models, it computes pairwise Pearson similarity between each model's prediction vector, converts average similarity to weights, and fuses accordingly — so reference proteins that behave like the target count for more.
28 features, all interpretable, no embeddings. Five families were introduced by this work:
| Feature | What it captures |
|---|---|
| NCE-NR | Normalized Contact Energy of the Native Residue — Miyazawa–Jernigan pair energies summed over side-chain-centroid contacts within 6.5 Å, divided by a per-amino-acid maximum from a curated PDB set |
| LEAR / LpAR | Likelihood of the energy state / pLDDT bin given an amino-acid replacement, from binarized 20×20 matrices |
| neco2 / neco3 / neco12 | Neighbor-compatibility log-ratios over ±2, ±3 and ±1±2 sequence windows |
| PDFF ×9 | Probability Density Function Features — r(x) = log[p_{A→M}(x) / p_A(x)], fitted with Gaussian mixtures (component count by AIC/BIC), zero-inflated GMM for entropy, reversed GB2 for pLDDT, validated on 80/20 splits of 380 sample files |
| colasi, fanc, fbnc | Conservation and contact statistics of the 3D neighborhood |
Ten of the 28 are position-level (PSSM, Shannon entropy, pLDDT, colasi, conserved-3D-neighbor fraction, fanc, fbnc, NCE-NR and two derived) and feed Stage 1.
Three nested levels, each stricter than the last:
- Leave-one-position-out within a protein — all variants at a held-out position are removed together, so the model cannot memorise a residue.
- Leave-one-protein-out across proteins — the cross-protein ensemble never sees the target protein.
- A 30-protein hold-out never used for model selection, plus three fully external benchmarks.
An ablation on training-set size found 30 reference proteins optimal (10, 15, 30, 45 and 60 were compared).
Submitted as team CLINTRABI. This is third-party assessed, not self-reported.
| Challenge | Result |
|---|---|
| LPL | Rank 1 of all submissions on Pearson, Spearman, Kendall's τ and MSE. Assessors' slide: "Top model: CLINTRABI_1 = QAFIMeta 2.0 v3." ThermoMPNN placed 61st (ρ = 0.409), FoldX 66th (0.396) |
| ATP7B | Rank 1 across all metrics. On the missense-only subset (~90 % of the data): Pearson 0.754, Spearman 0.741, AUC 0.902, MCC 0.632, mean rank 1.44 |
| TSC2 | 4th of 5 (Spearman 0.569, Pearson 0.619, AUROC 0.887) |
| ARSA | Pearson 2nd, Spearman 5th; overall average rank 12.5 of 109 evaluated methods |
Per-protein Pearson correlation:
| Method | Median | Mean | SD |
|---|---|---|---|
| qafisplit3 | 0.468 | 0.447 | 0.133 |
| qafisplit2 | 0.467 | 0.452 | 0.135 |
| qafisplit1 | 0.460 | 0.447 | 0.138 |
| qafi2 | 0.458 | 0.440 | 0.137 |
Head-to-head against 12 external predictors on the 393 domains where all methods have coverage:
| Method | Median Pearson |
|---|---|
| ESM-1v (domain) | 0.487 |
| EVE (domain) | 0.480 |
| QAFIsplit3 | 0.480 |
| popEVE | 0.446 |
| ESM-1v (full-length) | 0.443 |
| Tranception | 0.420 |
| ddmut | 0.404 |
| RaSP | 0.388 |
| FoldX | 0.375 |
| EVE | 0.344 |
| AlphaMissense | 0.308 |
QAFI ranks first among non-leaking baselines, level with EVE and just behind ESM-1v — using CPU-only gradient boosting on 28 hand-built features, against methods that require large pretrained models or MSA inference. thermoMPNN was excluded by the authors' own decision because its training set overlaps the benchmark; leaving it in would have flattered nobody honestly.
| Method | AUROC |
|---|---|
| BayesDel | 0.963 |
| ClinPred | 0.959 |
| QAFImeta | 0.959 |
| REVEL | 0.949 |
| QAFImeta baseline (QAFI removed) | 0.949 |
| VEST4 | 0.942 |
| AlphaMissense | 0.922 |
| QAFIsplit3 | 0.914 |
| QAFI2 | 0.906 |
| EVE | 0.899 |
| CADD-PHRED | 0.881 |
| SIFT | 0.834 |
The informative comparison is the two meta rows: a 5-tool meta-predictor scores 0.949, and adding QAFIsplit3 lifts it to 0.959. QAFI contributes +0.010 AUC of genuinely orthogonal signal to an already-strong ensemble — which is the real test of whether a new method adds anything.
Test30 — 30 unseen proteins, 129,708 variants: Pearson mean 0.443, Spearman mean 0.432 (per-protein range 0.225–0.655).
HGMD gain/loss of function vs gnomAD: LOF AUC 0.771 (n = 2,803), GOF AUC 0.756 (n = 545).
Known limitation, studied rather than hidden
Tree models cannot predict outside the training target range, which flattens the tails of the predicted distribution. experiment/xgboost_tail_extrapolation/ is a self-contained study comparing six remedies under leave-one-position-out on the 60-protein set (mlr, xgb, xgb+lr_yhat_tail, xgb+lr_blend, xgb+lr_feat_oor, xgb_clip_feat), with per-protein metrics reported.
There is also no probability calibration. The quantitative scale comes from the Gray normalization of the training target plus a post-hoc min-max rescale per benchmark. QAFI 1.0's binary threshold was 0.82, chosen to maximise median MCC.
QAFI_CODE_NEW/
├── src/qafi/ Core library (~8,500 LOC)
│ ├── feature/ 23-block dependency-resolved feature registry
│ ├── data/intermediates/ Loaders for cached intermediates (colasi, lear, nce_nr, pdff)
│ ├── model/psp/ Per-protein predictors — base + two-stage split variants
│ ├── model/qafi/ Cross-protein ensembles — qafi2, qafisplit1/2/3
│ └── validation/ Benchmark scoring + publication figure style
├── scripts/
│ ├── data/ 3-module retrieval pipeline (60 proteins / more proteins / PDB maps)
│ ├── features/ build_all_features.py + heavy preprocessing (neco, nce_nr, colasi, lear, pdff)
│ ├── models/ run_psp.py, run_qafi.py
│ └── benchmark/ beltran_predict.py, clinvar_predict.py, test30.py
├── notebook/ 7 showcase notebooks, all with executed outputs
├── data/ Dataset_60proteins_features.csv.gz (33 MB) + Q9Y375 demo protein
└── tests/ run_full_audit.py — exercises every module, script and notebook
git clone https://github.com/pipi0616/QAFI.git
cd QAFI
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txtPython 3.9+ (tested through 3.13). CPU only — XGBoost runs with tree_method="hist"; there is no GPU code anywhere.
Parallelism is controlled by environment variables: QAFI_PSP_N_JOBS, QAFI_PSP_SPLIT_N_JOBS, QAFI_XGB_N_JOBS, QAFI_RFR_N_JOBS.
# ① Build the 23 feature blocks for a protein
python scripts/features/build_all_features.py \
--input-csv data/proteins/Q9Y375/Q9Y375_features5.csv \
--output-csv outputs/features/Q9Y375_all_features.csv
# ② Per-protein baselines (leave-one-position-out)
python scripts/models/run_psp.py --list
python scripts/models/run_psp.py --method psp2_xgb
# ③ Cross-protein prediction on an unseen protein
python scripts/models/run_qafi.py --list
python scripts/models/run_qafi.py --method qafisplit3 --uniprot Q9Y375
python scripts/models/run_qafi.py --all --uniprot Q9Y375 --sim-metric pearson
# ④ External benchmarks
python scripts/benchmark/beltran_predict.py --method all
python scripts/benchmark/clinvar_predict.py --method all
python scripts/benchmark/test30.py --method allAvailable PSP methods: psp2_mlr, psp2_xgb, psp2_rfr, pspsplit1, pspsplit2, pspsplit2_obs, pspsplit2_fusion.
Available QAFI methods: qafi2, qafisplit1, qafisplit2, qafisplit3.
Every run writes run_config.json and model_info.json alongside its predictions, so any output can be traced back to the exact configuration that produced it.
Note. The benchmark scripts expect external benchmark data that is not distributed with this repository (see
scripts/benchmark/README.md). The feature pipeline additionally requires the cached PDFF density models. Both are being prepared for release — until then, the showcase notebooks innotebook/are the fastest way to see each stage working, since they ship with executed outputs.
| Notebook | What it shows |
|---|---|
feature_generation_showcase.ipynb |
All 23 feature blocks, one by one, with KDE plots |
psp_model_showcase.ipynb |
All 7 PSP methods with prediction previews |
qafi_model_showcase.ipynb |
All 4 QAFI methods with prediction previews |
beltran_benchmark_showcase.ipynb |
402-protein DMS benchmark |
clinvar_benchmark_showcase.ipynb |
42,177-variant clinical benchmark |
test30_benchmark_showcase.ipynb |
30-protein hold-out |
goflof_benchmark_showcase.ipynb |
HGMD gain/loss-of-function analysis |
Notebooks are generated from Python source so they cannot drift from the library.
| Dataset | Size | Included |
|---|---|---|
| 60-protein training DB | 292,500 variants · 60 proteins · 55 DMS references | ✅ data/Dataset_60proteins_features.csv.gz |
| Curated reference-protein sets | 30 proteins per ensemble method | ✅ data/QAFI_build_proteins.csv |
| Demo protein | Q9Y375, 6,213 predicted variants | ✅ data/proteins/Q9Y375/ |
| Beltran / ClinVar / Test30 / HGMD benchmarks | — | ❌ obtain from primary sources |
Target column is score_log_normalized (mean 0.871, SD 0.547, range −8.16 to 5.64).
The published QAFI 1.0 method:
Ozkan S., Padilla N., de la Cruz X. QAFI: a novel method for quantitative estimation of missense variant impact using protein-specific predictors and ensemble learning. Human Genetics 2025;144(2-3):191–208. doi:10.1007/s00439-024-02692-z
@article{ozkan2025qafi,
title = {QAFI: a novel method for quantitative estimation of missense
variant impact using protein-specific predictors and ensemble learning},
author = {Ozkan, Selen and Padilla, Nat{\`a}lia and de la Cruz, Xavier},
journal = {Human Genetics},
volume = {144}, number = {2-3}, pages = {191--208}, year = {2025},
doi = {10.1007/s00439-024-02692-z}
}The QAFI 2.0 / QAFIsplit methods in this repository are described in a manuscript in preparation.
Research Unit in Clinical and Translational Bioinformatics, Vall d'Hebron Institute of Research (VHIR), Universitat Autònoma de Barcelona, Spain.
Shaopei Ye · Selen Ozkan · Aitana Diaz · Natàlia Padilla · Xavier de la Cruz (VHIR / ICREA)
China Scholarship Council (CSC). CAGI is supported by NIH U24 HG007346.
- SeizeVar — mechanism-aware epilepsy variant interpretation
- HEARS-LM — hearing-loss-specialized protein language model
- OtoVCE — LLM evidence layer for hereditary hearing loss
大多数变异效应预测工具只给出致病/良性的二分类判定。而真实的变异分布在一个连续谱上——部分功能缺失、接近野生型、偶尔功能获得——二分类标签把这些信息全部丢弃了。QAFI 直接预测深度突变扫描（DMS）分数本身，采用归一化尺度：1 = 接近野生型，< 1 = 功能受损，> 1 = 可能功能获得，与湿实验多重化检测所输出的是同一个坐标轴。
人类约 2 万个蛋白中，只有几百个有实验 DMS 数据。QAFI 从 60 个此类实验中学习可迁移的序列、结构和能量学规律，并将其迁移到完全没有实验数据的蛋白上。
| 盲法竞赛 | CAGI7 LPL 挑战赛全部提交中排名第 1（四项指标全部第一），ATP7B 错义子集同样第 1 |
| 对比基础模型 | 在 393 个 Beltran 结构域上 Pearson 中位数 0.480——与 EVE（0.480）持平、接近 ESM-1v（0.487），远超 AlphaMissense（0.308） |
| 算力 | 纯 CPU 梯度提升 + 28 个可解释特征。无 GPU，无预训练大模型 |
| 临床迁移 | ClinVar AUROC 0.914；将 QAFI 加入 5 工具元预测器可提升 +0.010 AUC |
| 规模 | 60 个蛋白、55 篇 DMS 文献、292,500 个训练变异 |
二分类器只能告诉你某个变异有害，却无法告诉你这个蛋白还保留 80% 的活性还是只剩 5%——而这个区别对剂量敏感基因、对修饰基因解读、对任何类似基因型-表型曲线的分析都至关重要。
QAFI 直接以归一化 DMS 分数为预测目标（Gray 等 2018 归一化：以野生型为 0 居中，调整方向使数值越高功能越好，再按第 1 百分位中位数缩放）。输出是连续的功能活性估计值，而非概率。
由此带来的工程难题是：在蛋白 A 的实验数据上训练的模型，需要预测蛋白 B——而后者的实验采用不同的读出方式、动态范围和细胞体系。QAFI 的解法是两级集成——为每个参考蛋白各训练一个模型，再跨参考蛋白聚合——并全程采用留一蛋白法验证。
本仓库提供两个模型族。选择依据是：你的目标蛋白是否已有实验数据。
| 策略 | 适用场景 | 工作方式 | 可用方法 |
|---|---|---|---|
| PSP——单蛋白分数预测器 | 目标蛋白已有自己的 DMS 数据 | 在该单个蛋白上训练回归模型，用留一位点法验证 | 7 种：psp2_mlr、psp2_xgb、psp2_rfr、pspsplit1、pspsplit2、pspsplit2_obs、pspsplit2_fusion |
| QAFI——跨蛋白集成 | 目标蛋白没有 DMS 数据 | 为每个参考蛋白各训练一个模型，再按中位数或蛋白相似度加权聚合 | 4 种：qafi2、qafisplit1、qafisplit2、qafisplit3 |
两者共用同一套 28 维特征表示，因此同一个蛋白可以用两种方式打分，结果可直接比较。对于未见过的蛋白，qafisplit3 是默认部署版本。
flowchart TB
subgraph S0["① 数据构建 — scripts/data/"]
A["ProteinGym<br/>53 个 DMS 数据集"] --> D["Gray 归一化<br/>野生型 = 1 · 受损 < 1"]
B["Gray · Reeb · Dunham<br/>Riesselman · Frazer<br/>30 个蛋白"] --> D
C["PISCES 11,178 条链<br/>→ 4,783 个 PDB↔UniProt 映射"] --> E["残基位置映射表"]
D --> F["DB60 · 292,500 个变异<br/>60 个蛋白 · 55 篇文献"]
end
subgraph S1["② 特征 — src/qafi/feature/"]
F & E --> G["23 个特征块<br/>依赖关系自动解析的注册表"]
G --> H1["进化保守性 ×4<br/>PSSM · 香农熵 · BLOSUM62"]
G --> H2["结构 ×2<br/>AlphaFold pLDDT"]
G --> H3["邻域 / 能量学 ×13<br/>NCE-NR · colasi · MJ 势能<br/>neco · lpar · laar"]
G --> H4["PDFF ×9<br/>高斯混合密度对数比"]
H1 & H2 & H3 & H4 --> I["每个变异 28 个特征<br/>其中 10 个为位点级"]
end
subgraph S2["③ PSP — 单蛋白基线"]
I --> J["蛋白内部<br/>留一位点交叉验证"]
J --> K["多元线性回归 · XGBoost · 随机森林<br/>+ 两阶段拆分变体"]
end
subgraph S3["④ QAFI — 跨蛋白集成"]
I --> L["对 30 个参考蛋白逐一："]
L --> M["第一阶段：线性回归<br/>10 个位点级特征<br/>→ median_pos_predicted"]
M --> N["第二阶段：XGBoost 用 28 个特征<br/>预测分数或残差"]
N --> O["跨蛋白聚合<br/>中位数 · 或相似度加权"]
O --> P["对未见蛋白的<br/>功能分数预测"]
end
subgraph S4["⑤ 基准测试"]
P --> Q["Beltran DMS · 402 个蛋白<br/>ClinVar · 42,177 个变异<br/>Test30 留出集 · 30 个蛋白<br/>HGMD 功能获得/缺失"]
end
style S0 fill:#e8f4f8,stroke:#4a90a4
style S1 fill:#f0f4e8,stroke:#7a9a4a
style S2 fill:#eeeeee,stroke:#888888
style S3 fill:#f8e8f0,stroke:#a44a7a
style S4 fill:#fff4e0,stroke:#c49a3a
split 系列模型背后的洞见是：位点耐受性与替换严重程度是两个可以分离的问题。有些残基不能承受任何替换，有些则什么都能接受。先预测前者、再预测残差，效果好于一步到位地预测变异分数。
flowchart LR
A["变异<br/>28 个特征"] --> B["第一阶段<br/>线性回归<br/>10 个位点级特征"]
B --> C["median_pos_predicted<br/>= 这个残基有多耐受？"]
A --> D["第二阶段<br/>XGBoost"]
C --> D
D --> E1["qafisplit1<br/>位点预测作为额外特征"]
D --> E2["qafisplit2<br/>预测残差<br/>最终值 = 位点预测 + 残差"]
D --> E3["qafisplit3<br/>在全部蛋白上训练<br/>按相似度加权融合"]
style B fill:#e8f0f8,stroke:#4a7aa4
style E3 fill:#f0f8e8,stroke:#6a9a4a
仅第一阶段——纯粹的位点级耐受性——在 60 蛋白数据库上就达到 Pearson 中位数 0.580（最高 0.840），在 Beltran 位点级中位数上达到 0.7–0.9。
qafisplit3 是实际部署的版本。它不再对各蛋白模型取简单中位数，而是计算各模型预测向量之间的两两 Pearson 相似度，把平均相似度转换为权重后加权融合——从而让行为更接近目标蛋白的参考蛋白获得更大话语权。
28 个特征，全部可解释，不使用任何嵌入向量。其中五个特征族由本工作首次提出：
| 特征 | 刻画的内容 |
|---|---|
| NCE-NR | 天然残基归一化接触能——对侧链质心 6.5 Å 内的接触求 Miyazawa–Jernigan 配对势能之和，再除以由精选 PDB 集得出的该氨基酸类型最大值 |
| LEAR / LpAR | 给定氨基酸替换后，能量状态 / pLDDT 分箱的似然，来自二值化的 20×20 矩阵 |
| neco2 / neco3 / neco12 | 在 ±2、±3 和 ±1±2 序列窗口上的邻域相容性对数比 |
| PDFF ×9 | 概率密度函数特征——r(x) = log[p_{A→M}(x) / p_A(x)]，用高斯混合模型拟合（成分数由 AIC/BIC 确定），熵用零膨胀 GMM，pLDDT 用反向 GB2 分布，并在 380 个样本文件的 80/20 划分上验证 |
| colasi, fanc, fbnc | 三维邻域的保守性与接触统计量 |
28 个特征中有 10 个是位点级的（PSSM、香农熵、pLDDT、colasi、三维邻域保守比例、fanc、fbnc、NCE-NR 及两个衍生量），它们构成第一阶段的输入。
三个层层递进、逐级更严格的层次：
- 留一位点法（蛋白内部）——留出位点上的所有变异一并移除，使模型无法记住某个残基。
- 留一蛋白法（跨蛋白）——跨蛋白集成模型从不接触目标蛋白。
- 30 个蛋白的留出集，全程不参与模型选择，另加三个完全外部的基准集。
一项关于训练集规模的消融实验发现 30 个参考蛋白为最优（对比了 10、15、30、45 和 60）。
以 CLINTRABI 队名提交。结果由第三方评估，非自评。
| 挑战赛 | 结果 |
|---|---|
| LPL | 全部提交中排名第 1，Pearson、Spearman、Kendall's τ 和 MSE 四项指标全部第一。评估方幻灯片原文："Top model: CLINTRABI_1 = QAFIMeta 2.0 v3"。ThermoMPNN 排名第 61（ρ = 0.409），FoldX 第 66（0.396） |
| ATP7B | 所有指标均排名第 1。在错义子集（约占数据 90%）上：Pearson 0.754，Spearman 0.741，AUC 0.902，MCC 0.632，平均排名 1.44 |
| TSC2 | 5 支队伍中第 4（Spearman 0.569，Pearson 0.619，AUROC 0.887） |
| ARSA | Pearson 第 2，Spearman 第 5；在 109 个受评方法中总体平均排名 12.5 |
逐蛋白 Pearson 相关系数：
| 方法 | 中位数 | 均值 | 标准差 |
|---|---|---|---|
| qafisplit3 | 0.468 | 0.447 | 0.133 |
| qafisplit2 | 0.467 | 0.452 | 0.135 |
| qafisplit1 | 0.460 | 0.447 | 0.138 |
| qafi2 | 0.458 | 0.440 | 0.137 |
在全部方法都有覆盖的 393 个结构域上，与 12 个外部预测器正面对比：
| 方法 | Pearson 中位数 |
|---|---|
| ESM-1v（结构域） | 0.487 |
| EVE（结构域） | 0.480 |
| QAFIsplit3 | 0.480 |
| popEVE | 0.446 |
| ESM-1v（全长） | 0.443 |
| Tranception | 0.420 |
| ddmut | 0.404 |
| RaSP | 0.388 |
| FoldX | 0.375 |
| EVE | 0.344 |
| AlphaMissense | 0.308 |
在不存在数据泄漏的基线中 QAFI 排名第一，与 EVE 持平、略低于 ESM-1v——而它用的只是纯 CPU 的梯度提升和 28 个人工构建的特征，对手则需要大型预训练模型或多序列比对推断。thermoMPNN 是作者主动剔除的，因为其训练集与本基准集重叠；保留它对任何人都不构成诚实的对比。
| 方法 | AUROC |
|---|---|
| BayesDel | 0.963 |
| ClinPred | 0.959 |
| QAFImeta | 0.959 |
| REVEL | 0.949 |
| QAFImeta 基线（移除 QAFI） | 0.949 |
| VEST4 | 0.942 |
| AlphaMissense | 0.922 |
| QAFIsplit3 | 0.914 |
| QAFI2 | 0.906 |
| EVE | 0.899 |
| CADD-PHRED | 0.881 |
| SIFT | 0.834 |
真正有信息量的是两行元预测器结果：5 工具元预测器为 0.949，加入 QAFIsplit3 后升至 0.959。QAFI 为一个本已很强的集成贡献了 +0.010 AUC 的正交信号——这才是检验一个新方法是否真有增量价值的标准。
Test30——30 个未见蛋白、129,708 个变异：Pearson 均值 0.443，Spearman 均值 0.432（逐蛋白范围 0.225–0.655）。
HGMD 功能获得/缺失 vs gnomAD：功能缺失 AUC 0.771（n = 2,803），功能获得 AUC 0.756（n = 545）。
树模型无法预测训练目标范围之外的取值，这会压平预测分布的两端。experiment/xgboost_tail_extrapolation/ 是一项自包含研究，在 60 蛋白数据集上以留一位点法对比了六种补救策略（mlr、xgb、xgb+lr_yhat_tail、xgb+lr_blend、xgb+lr_feat_oor、xgb_clip_feat），并报告逐蛋白指标。
此外没有做概率校准。定量尺度来自训练目标的 Gray 归一化，加上每个基准集上的事后 min-max 重缩放。QAFI 1.0 的二分类阈值为 0.82，由最大化 MCC 中位数确定。
QAFI_CODE_NEW/
├── src/qafi/ 核心库（约 8,500 行）
│ ├── feature/ 23 个特征块，依赖关系自动解析的注册表
│ ├── data/intermediates/ 中间结果缓存加载器（colasi、lear、nce_nr、pdff）
│ ├── model/psp/ 单蛋白预测器——基础版 + 两阶段拆分变体
│ ├── model/qafi/ 跨蛋白集成——qafi2、qafisplit1/2/3
│ └── validation/ 基准评分 + 出版级图表样式
├── scripts/
│ ├── data/ 三模块数据获取流程（60 蛋白 / 更多蛋白 / PDB 映射）
│ ├── features/ build_all_features.py + 重预处理（neco、nce_nr、colasi、lear、pdff）
│ ├── models/ run_psp.py、run_qafi.py
│ └── benchmark/ beltran_predict.py、clinvar_predict.py、test30.py
├── notebook/ 7 个演示 notebook，均含已执行输出
├── data/ Dataset_60proteins_features.csv.gz（33 MB）+ Q9Y375 演示蛋白
└── tests/ run_full_audit.py——遍历所有模块、脚本与 notebook
git clone https://github.com/pipi0616/QAFI.git
cd QAFI
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt需要 Python 3.9+（已测试至 3.13）。纯 CPU ——XGBoost 使用 tree_method="hist"，代码中不含任何 GPU 逻辑。
并行度通过环境变量控制：QAFI_PSP_N_JOBS、QAFI_PSP_SPLIT_N_JOBS、QAFI_XGB_N_JOBS、QAFI_RFR_N_JOBS。
# ① 为某个蛋白构建 23 个特征块
python scripts/features/build_all_features.py \
--input-csv data/proteins/Q9Y375/Q9Y375_features5.csv \
--output-csv outputs/features/Q9Y375_all_features.csv
# ② 单蛋白基线（留一位点法）
python scripts/models/run_psp.py --list
python scripts/models/run_psp.py --method psp2_xgb
# ③ 对未见蛋白做跨蛋白预测
python scripts/models/run_qafi.py --list
python scripts/models/run_qafi.py --method qafisplit3 --uniprot Q9Y375
python scripts/models/run_qafi.py --all --uniprot Q9Y375 --sim-metric pearson
# ④ 外部基准测试
python scripts/benchmark/beltran_predict.py --method all
python scripts/benchmark/clinvar_predict.py --method all
python scripts/benchmark/test30.py --method all可用的 PSP 方法：psp2_mlr、psp2_xgb、psp2_rfr、pspsplit1、pspsplit2、pspsplit2_obs、pspsplit2_fusion。
可用的 QAFI 方法：qafi2、qafisplit1、qafisplit2、qafisplit3。
每次运行都会在预测结果旁写出 run_config.json 和 model_info.json，因此任何输出都能追溯到产生它的确切配置。
说明。 基准测试脚本需要本仓库未随附的外部基准数据（见
scripts/benchmark/README.md）。特征流程还需要缓存的 PDFF 密度模型。两者正在准备发布——在此之前，notebook/中的演示 notebook 是查看各阶段实际运行效果的最快方式，它们都带有已执行的输出。
| Notebook | 展示内容 |
|---|---|
feature_generation_showcase.ipynb |
逐个展示全部 23 个特征块，附核密度图 |
psp_model_showcase.ipynb |
全部 7 种 PSP 方法及预测预览 |
qafi_model_showcase.ipynb |
全部 4 种 QAFI 方法及预测预览 |
beltran_benchmark_showcase.ipynb |
402 蛋白 DMS 基准 |
clinvar_benchmark_showcase.ipynb |
42,177 变异临床基准 |
test30_benchmark_showcase.ipynb |
30 蛋白留出集 |
goflof_benchmark_showcase.ipynb |
HGMD 功能获得/缺失分析 |
Notebook 由 Python 源码自动生成，因此不会与库代码脱节。
| 数据集 | 规模 | 是否随仓库提供 |
|---|---|---|
| 60 蛋白训练库 | 292,500 变异 · 60 蛋白 · 55 篇 DMS 文献 | ✅ data/Dataset_60proteins_features.csv.gz |
| 精选参考蛋白集 | 每种集成方法 30 个蛋白 | ✅ data/QAFI_build_proteins.csv |
| 演示蛋白 | Q9Y375，6,213 个预测变异 | ✅ data/proteins/Q9Y375/ |
| Beltran / ClinVar / Test30 / HGMD 基准 | — | ❌ 需从原始来源获取 |
目标列为 score_log_normalized（均值 0.871，标准差 0.547，取值范围 −8.16 至 5.64）。
已发表的 QAFI 1.0 方法：
Ozkan S., Padilla N., de la Cruz X. QAFI: a novel method for quantitative estimation of missense variant impact using protein-specific predictors and ensemble learning. Human Genetics 2025;144(2-3):191–208. doi:10.1007/s00439-024-02692-z
@article{ozkan2025qafi,
title = {QAFI: a novel method for quantitative estimation of missense
variant impact using protein-specific predictors and ensemble learning},
author = {Ozkan, Selen and Padilla, Nat{\`a}lia and de la Cruz, Xavier},
journal = {Human Genetics},
volume = {144}, number = {2-3}, pages = {191--208}, year = {2025},
doi = {10.1007/s00439-024-02692-z}
}本仓库中的 QAFI 2.0 / QAFIsplit 方法将在一篇正在撰写中的论文中描述。
Vall d'Hebron 研究院（VHIR） 临床与转化生物信息学研究组，巴塞罗那自治大学，西班牙。
叶少培（Shaopei Ye）· Selen Ozkan · Aitana Diaz · Natàlia Padilla · Xavier de la Cruz（VHIR / ICREA）
国家留学基金委（CSC）。CAGI 由 NIH U24 HG007346 资助。