Motion Detection & Object Tracking with OpenCV, CUDA and YOLO11

A computer-vision project combining classical optical-flow motion detection, experimental CUDA/OpenGL processing, and YOLO11 object detection and tracking. Three separate implementations explore motion analysis, GPU image processing, and learned perception—engineering work relevant to robotics software.

C++17 · CUDA · OpenCV · OpenGL · Python · YOLO11

Implementations

Component	Input	Processing and output
Classical CPU	Local video	Farnebäck flow, temporal filtering, motion masks, bounding-box-center markers
CUDA/OpenGL	Generated moving-circle scene	OpenCV CUDA flow, experimental component kernels, markers on a rendered texture
YOLO11	Local video and supplied weights	Learned object detections and optional library-provided tracking

Optical flow provides motion cues without restricting detection to learned classes. YOLO recognizes specific categories. The project explores their different failure modes and the integration of rendered imagery with GPU processing.

My Contributions

My work focused on implementation, integration, CUDA experimentation, and evaluation:

• Classical optical-flow processing, temporal motion filtering, and threshold experiments.
• CUDA/OpenGL interoperability and integration of OpenCV CUDA operations.
• Custom CUDA component-processing and marker-generation kernels.
• YOLO11n fine-tuning configuration and video inference/tracking integration.
• Experimental comparison of classical motion cues and learned detections.

Third-party foundations: Farnebäck optical flow, OpenCV, YOLO11 architecture and pretrained weights, Ultralytics training/inference, and BoT-SORT tracking. These established algorithms and frameworks underpin the project; the origin of adapted CUDA labeling code still needs confirmation. See attribution.

Architecture

• CPU: grayscale and Gaussian blur precede dense optical flow. A 10-frame average and spatial-median thresholds produce masks for contour-based region extraction.
• CUDA: the texture is copied into a GpuMat, processed, and copied back. Host orchestration and scalar reductions remain; component-label correctness requires GPU validation.
• YOLO: frames pass through the detector and, in tracking mode, Ultralytics' botsort.yaml configuration. The recorded model covers six object classes.

Architecture details document parameters, memory boundaries, and outputs. Supporting tools include NumPy, PyTorch, GLFW, GLEW, and CMake.

Recorded Validation Results

Recorded validation results from the original experiment. These are not independently reproduced benchmark results.

Precision	Recall	mAP@0.50	mAP@0.50:0.95
0.599	0.462	0.496	0.327

YOLO11n was fine-tuned for 150 epochs at image size 640 and batch size 64. The saved run used two Tesla T4 GPUs and evaluated 737 validation images.

Performance varied substantially: car mAP@0.50 was 0.918, while bicycle mAP@0.50 was 0.106. See evaluation for all classes, evidence limits, and a proposed controlled comparison; training settings are supplied separately.

Quick Start

From the repository root, using Python 3.10 as the starting environment:

python -m venv .venv
source .venv/bin/activate
python -m pip install -r requirements.txt
python src/classical_cpu/motion.py --video recordings/input.mp4

Supply your own local video. For YOLO, install the hardware-appropriate PyTorch build and requirements-yolo.txt as described in setup, then supply local weights:

python src/yolo/infer.py --video recordings/input.mp4 --weights models/detector.pt --device cpu
python src/yolo/train.py --data datasets/local/data.yaml --device 0,1

The training example requires two suitable GPUs and a local dataset YAML. Training may download pretrained weights and writes outputs under runs/.

With the native CUDA/OpenCV/OpenGL dependencies installed:

cmake -S . -B build -DCMAKE_CUDA_ARCHITECTURES=native
cmake --build build --config Release
./build/motion_cuda

See setup for platform-specific commands, native dependencies, device selection, and alternate usage. The CUDA build and runtime have not been verified in the current environment.

Limitations and Validation

• Classical motion analysis is sensitive to camera movement and heuristic thresholds.
• CPU/CUDA implementations differ in inputs and algorithms; their historical FPS observations are not directly comparable. No CUDA speedup is established.
• CUDA component labeling remains experimental, with concurrency and convergence questions requiring target-system validation.

Ten numerical/mocked-boundary tests passed during preparation; they do not validate actual OpenCV, YOLO, or CUDA execution. See validation status and limitations.

The project focuses on image perception. Live sensor, ROS, localization, navigation, and control integration are outside the supplied implementation.

Documentation

Reference	Contents
Architecture	Three processing paths and CPU/GPU boundaries
Setup	Dependency versions, build requirements, usage, and tests
Evaluation	Recorded metrics, class-level results, benchmark methodology
Limitations · Validation	Known issues and checks actually performed
Technical changes	Preparation-time corrections, separate from original experiment results
Data and model provenance	Model origin and unresolved dataset/media provenance

Weights, datasets, and demo media are excluded pending redistribution-rights verification. The CUDA application generates its own simple scene when run.

Licensing and third-party attribution details are documented in licensing status and THIRD_PARTY_NOTICES.md.