🚀 YOLO-TensorRT-Toolkit: Optimized Inference Pipeline for YOLOv10/v11/v12 with ONNX & TensorRT

Reproducible optimization and benchmarking of YOLOv10, YOLOv11, and YOLOv12 models via ONNX export and TensorRT (.engine) conversion with GPU acceleration. This project serves as a technical reference, professional pipeline, and deployable real-time detection base.

---

Original model	Optimized model

🎯 Objective

• Export YOLOv10/v11/v12 models to ONNX
• Convert ONNX models to TensorRT (FP16 mode)
• Automatically validate conversions
• Perform inference on images and webcam using .engine
• Compare performance (.pt vs .onnx vs .engine)
• Provide a clean, reproducible CLI for all stages

---

📊 Performance Graph

For exact latency/FPS values, see:
benchmarks/yolo10s.csv
benchmarks/yolo11s.csv
benchmarks/yolo12s.csv

---

🛠️ Base Image and Environment

📦 Base Image

nvcr.io/nvidia/pytorch:24.03-py3

Includes:

• CUDA 12.4
• PyTorch 2.3.0
• TensorRT 8.6.3
• ONNX 1.15.0rc2
• torch-tensorrt
• Polygraphy, GraphSurgeon, Netron, etc.

🔧 Benchmarking Hardware

The following hardware was used for all benchmarks:

• GPU: NVIDIA GeForce RTX 4050 Laptop GPU (6GB VRAM)
• CPU: 13th Gen Intel(R) Core(TM) i7-13700H
• OS: Ubuntu 22.04 inside Docker

---

📂 Project Structure

YOLO-TENSORRT-OPTIMIZATION/
├── benchmarks/                  ← Benchmarking scripts/results
├── data/
│   ├── images/                   ← Test images
│   ├── videos/                   ← Test videos
│   ├── synthetic_data/           ← Test data for benchmark
│   └── classes.txt             ← Labels (COCO - 80 classes)
├── models/                     ← .pt, .onnx, .engine models
├── outputs/                    ← Inference outputs
├── notebooks/                  ← Optional analysis
├── src/
│   ├── cli.py                  ← Main CLI
│   └── yolo_trt/
│       ├── export_to_onnx.py   ← .pt → .onnx
│       ├── convert_to_trt.py   ← .onnx → .engine
│       ├── validate_onnx.py    ← ONNX verification
│       ├── infer_trt.py        ← Inference with TensorRT
│       ├── benchmark.py        ← Performance comparison
├── run_pipeline.sh             ← End-to-end automation
├── requirements-extra.txt      ← Additional dependencies
├── Dockerfile                  ← Reproducible image
└── README.md                   ← This document

---

📦 requirements-extra.txt

ultralytics
netron
onnx-simplifier
pycuda==2023.1
onnxruntime-gpu==1.20.2
opencv-fixer==0.2.5


### To use opencv-fixer:
python -c "from opencv_fixer import AutoFix; AutoFix()"

Installs packages not included by default in the NVIDIA image.

---

💻 CLI Usage

Note: All the examples below can be replaced with any model from yolov10 to yolov12

🎯 Export .pt to .onnx

python src/cli.py export --model models/yolo11s.pt

With options:

python src/cli.py export --model models/yolo11s.pt --simplify --dynamic --validate

---

✅ Validate .onnx

python src/cli.py validate --onnx models/yolo11s.onnx

---

⚙️ Convert to .engine

#FP16 (default)
python src/cli.py convert --onnx models/yolo11s.onnx
#FP32
python src/cli.py convert --onnx models/yolo11s.onnx --fp32

---

🧪 Inference (Compatible for .pt and .engine models)

Image:

python src/cli.py infer --model models/yolo11s.pt --source data/images/example_1.jpg 

# Output:
outputs/inference_example_1_yolo11s_pt.jpg

Video:

python src/cli.py infer --model models/yolo11s_fp16.engine --source data/videos/sample.mp4
# Output:
outputs/inference_sample_yolo11s_engine.mp4

python src/cli.py infer --model models/yolo11s.pt --source data/video/sample.mp4
outputs/inference_sample_yolo11s_pt.mp4

Webcam:

python src/cli.py infer --model models/yolo11s_fp32.engine --source http://192.168.1.100:8080/video
# or to use your integrated webcam:
# python src/cli.py infer --model models/yolo11s_fp32.engine --source cam

---

📊 Benchmark (.pt, .onnx, .engine)

python src/yolo_trt/benchmark.py --model yolo12s --runs 100 --batch 1

---

📊 Plot Benchmark

python src/cli.py plot
# → Plot by latency

python src/cli.py plot --metric fps
# → Plot by FPS

Includes:

• Export to ONNX
• Validation
• TensorRT conversion
• Benchmark (100 iterations)

---

📜 License

MIT License.

---

📨 Contact

Rolando Cortez García
Email: rolscg@gmail.com
GitHub: rolandocortez
LinkedIn: rolando-cortez

---

📬 Citation

If you use this project or the underlying models, please consider citing the original authors:

YOLOv10

YOLOv10: Real‑Time End‑to‑End Object Detection

@article{wang2024yolov10,
  title={YOLOv10: Real-Time End-to-End Object Detection},
  author={Wang, Ao and Chen, Hui and Liu, Lihao and Chen, Kai and Lin, Zijia and Han, Jungong and Ding, Guiguang},
  journal={arXiv preprint arXiv:2405.14458},
  year={2024}
}

YOLOv11

YOLOv11: An Overview of the Key Architectural Enhancements

@article{khanam2024yolov11,
  title={YOLOv11: An Overview of the Key Architectural Enhancements},
  author={Khanam, Rahima and Hussain, Muhammad},
  journal={arXiv preprint arXiv:2410.17725},
  year={2024}
}

YOLOv12

YOLOv12: Attention‑Centric Real‑Time Object Detectors

@article{tian2025yolov12,
  title={YOLOv12: Attention-Centric Real-Time Object Detectors},
  author={Tian, Yunjie and Ye, Qixiang and Doermann, David},
  journal={arXiv preprint arXiv:2502.12524},
  year={2025}
}

---

🛠️ Tools Used

• Ultralytics (YOLO framework used):
  https://github.com/ultralytics/ultralytics

• ONNX Runtime:
  https://onnxruntime.ai/

• NVIDIA TensorRT:
  https://developer.nvidia.com/tensorrt

---

If you use any YOLO version other than v10 (e.g., v8, v11, v12), please cite their respective papers accordingly.

---