Breaking Bad Molecules: Are MLLMs Ready for Structure-Level Molecular Detoxification?

Fei Lin^1*, Ziyang Gong^2,5*, Cong Wang^3*, Tengchao Zhang¹, Yonglin Tian³, Yining Jiang⁴, Ji Dai⁶, Chao Guo³, Xiaotong Yu⁷, Xue Yang⁸, Gen Luo⁵, Fei-Yue Wang³

¹ Department of Engineering Science, Macau University of Science and Technology, Macau, China
² School of Computer Science, Shanghai Jiao Tong University, Shanghai, China
³ Institute of Automation, Chinese Academy of Sciences, Beijing, China
⁴ School of Pharmacy, Macau University of Science and Technology, Macau, China
⁵ Shanghai AI Laboratory, Shanghai, China
⁶ Faculty of Electrical Engineering and Computer Science, Ningbo University, Ningbo, China
⁷ State Key Laboratory of Biopharmaceutical Preparation and Delivery, Institute of Process Engineering, Chinese Academy of Sciences, Beijing, China
⁸ School of Automation and Intelligent Sensing, Shanghai Jiao Tong University, Shanghai, China

^* Equal contribution

---

This work investigates whether general Multimodal Large Language Models (MLLMs) can perform structure-level molecular refinement for toxicity repair. We introduce ToxiMol, a benchmark for molecular detoxification, together with ToxiEval, an automated evaluation framework for assessing whether generated molecules are safer, valid, drug-like, synthesizable, and structurally close to the original molecules.

---

🔥 News

• 🏆 [2026/05/17] ToxiMol has been accepted as an Oral presentation at the 32nd SIGKDD Conference on Knowledge Discovery and Data Mining (KDD 2026), AI for Sciences Track.
• 📚 [2025/06/13] The paper of ToxiMol is available on arXiv.
• 📊 [2025/06/09] We released the ToxiMol dataset on Hugging Face.

---

📚 Table of Contents

• What is ToxiMol?
• Quick Reproduction
• Citation

---

What is ToxiMol?

ToxiMol is a benchmark for evaluating whether MLLMs can repair toxic molecules at the molecular structure level. Given a toxic molecule and a task-specific detoxification prompt, the model is expected to generate a structurally valid molecule with reduced toxicity while preserving key molecular properties.

ToxiMol contains:

• 660 toxic molecules
• 11 toxicity repair tasks
• both classification and regression toxicity endpoints
• approximately 30 small-molecule toxicity mechanisms
• evaluation results for 43 state-of-the-art MLLMs

The 11 task types include:

AMES, Carcinogens, ClinTox, DILI, hERG, hERG_Central, hERG_Karim,
LD50_Zhu, Skin Reaction, Tox21, ToxCast

The dataset is available on Hugging Face:

https://huggingface.co/datasets/HydroSophyTech/ToxiMol-benchmark

ToxiEval

We evaluate generated molecules with ToxiEval. A molecule is considered successfully detoxified only when it satisfies all five criteria:

• Safety Score: toxicity is reduced
• QED: drug-likeness is acceptable
• SAS: synthetic accessibility is acceptable
• RO5: Lipinski's Rule of Five violations are limited
• Structural Similarity: the repaired molecule remains structurally close to the original molecule

---

Quick Reproduction

1. Install

git clone https://github.com/HydroSophy/ToxiMol.git --recursive
cd ToxiMol
pip install -r requirements.txt

For DeepSeek-VL V2, please first set up the environment following DeepSeek-VL2, then run:

cd experiments/opensource/DeepSeek
pip install -e .

2. Load Dataset

from datasets import load_dataset

dataset = load_dataset(
    "HydroSophyTech/ToxiMol-benchmark",
    data_dir="ames",
    split="train",
    trust_remote_code=True
)

Available tasks:

ames, carcinogens_lagunin, clintox, dili, herg, herg_central, herg_karim,
ld50_zhu, skin_reaction, tox21, toxcast

3. Run Experiments

GPT Models

# Run one task
python experiments/gpt/run_toxicity_repair.py \
    --task ames \
    --model gpt-4.1 \
    --api-key YOUR_OPENAI_API_KEY

# Run all tasks
python experiments/gpt/run_toxicity_repair.py \
    --task all \
    --model gpt-4.1 \
    --api-key YOUR_OPENAI_API_KEY

Open-Source MLLMs

# InternVL3
python experiments/opensource/run_opensource.py \
    --task ames \
    --model internvl3 \
    --model_path OpenGVLab/InternVL3-8B

# DeepSeek-VL V2
python experiments/opensource/run_opensource.py \
    --task all \
    --model deepseekvl2 \
    --model_path deepseek-ai/deepseek-vl2-small

# LLaVA-OneVision
python experiments/opensource/run_opensource.py \
    --task clintox \
    --model llava-onevision \
    --model_path lmms-lab/llava-onevision-qwen2-7b-ov

# Qwen2.5-VL
python experiments/opensource/run_opensource.py \
    --task herg \
    --model qwen2.5vl \
    --model_path Qwen/Qwen2.5-VL-7B-Instruct

Supported open-source model names:

internvl3, deepseekvl2, llava-onevision, qwen2.5vl

4. Evaluate with ToxiEval

python evaluation/run_evaluation.py \
    --results-dir experiments/gpt/results \
    --model gpt-4.1 \
    --full

Evaluate a specific task:

python evaluation/run_evaluation.py \
    --results-dir experiments/gpt/results \
    --model gpt-4.1 \
    --task ames \
    --full

Evaluate open-source model results:

python evaluation/run_evaluation.py \
    --results-dir experiments/opensource/results \
    --model InternVL3-8B \
    --full

If SMILES extraction fails, run:

python evaluation/extract_smiles.py \
    --results-dir experiments/opensource/results/model_type

If you encounter:

TypeError: process_vision_info() got an unexpected keyword argument

try:

pip install qwen-vl-utils==0.0.10

---

Citation

If you use this benchmark, please cite:

@misc{lin2026breakingbadmoleculesmllms,
      title={Breaking Bad Molecules: Are MLLMs Ready for Structure-Level Molecular Detoxification?}, 
      author={Fei Lin and Ziyang Gong and Cong Wang and Tengchao Zhang and Yonglin Tian and Yining Jiang and Ji Dai and Chao Guo and Xiaotong Yu and Xue Yang and Gen Luo and Fei-Yue Wang},
      year={2026},
      eprint={2506.10912},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2506.10912}, 
}