Computer Vision & AI Dataset Preprocessing Toolkit

An enterprise-ready Python preprocessing pipeline designed to optimize, convert, and run automated quality control passes on machine learning datasets. This toolkit bridges the gap between raw data annotations and production-ready training loops, specializing in coordinate normalization, annotation format translation, and dataset structural verification.

Key Features

• Format Transformation Engine: Seamlessly translates complex, nested dataset formats (e.g., COCO JSON object instances to normalized YOLO single-line text arrays) while preserving structural integrity.
• Automated Quality Control (QC): Programmatically flags human labeling issues like coordinate drift, zero-area bounding boxes, boundary overflows, and format anomalies.
• Detailed Dataset Health Reporting: Generates instant, markdown-ready auditing logs detailing the exact error types, specific annotation IDs, and structural violations.
• Zero-Dependency Parsing: Leverages Python’s native standard library for high-speed file operations and mathematical scaling, minimizing container footprint.

---

Architecture Overview

cv-dataset-toolkit/
│
├── src/
│   ├── __init__.py
│   ├── converter.py      # Core geometry translations & JSON parsing
│   └── validator.py      # Automated QC scripts & bounding box limits
│
├── data/
│   ├── sample_coco.json  # Mock nested source annotations
│   └── sample_yolo/      # Programmatically generated outputs
│
├── README.md             # Documentation
└── requirements.txt      # Project constraints

---

Installation & Usage

Setup

Ensure you have a modern Python runtime environment configured:

git clone https://github.com
cd cv-dataset-toolkit

Execution Example

Initialize the preprocessing script inside an active Python shell or automation workflow:

from src.converter import DatasetConverter
from src.validator import DatasetValidator

# Initialize modules
converter = DatasetConverter()
validator = DatasetValidator()

# 1. Run Automated QC Passes
is_valid = validator.validate_bounding_box(
    bbox=, 
    img_width=480, 
    img_height=640, 
    annotation_id=1042
)

# 2. Extract Data & Generate the QC Markdown Report
print(validator.generate_qc_report())

# 3. Perform the format migration
converter.process_coco_json("data/sample_coco.json", "data/sample_yolo")

---

Quality Control Logic & Edge-Case Handling

The engine contains strict validation rules preventing dirty data from polluting downstream AI training models:

Error Type	Core Validation Guardrail
Invalid Dimensions	Intercepts annotations where width or height is ≤ 0 pixels.
Boundary Overflow	Detects if a label box edge stretches past the maximum actual image width/height boundaries.
Coordinate Drift	Catches progressive formatting shifts or frame-by-frame rounding anomalies.