My father sent me a YouTube Shorts clip of a drone being tracked in real time and asked if I could do the same with his. I said sure, how hard could it be.
Harder than expected. A quadcopter doing aerobatics is roughly 10–30 px wide, moves fast enough to leave IoU at zero between frames, and blends into the background without preprocessing. What I thought would be a weekend script turned into a full tracking pipeline — morphological detection, Kalman filtering, YOLO fine-tuning, and a proper evaluation framework against annotated ground truth. The scope grew serious enough to become a portfolio project.
Three pipelines are compared: a classical CPU-only morphology + Kalman filter, a fine-tuned YOLOv8n + Kalman, and ByteTrack as a baseline. The key finding: IoU-based association fails catastrophically on fast small targets — switching to centroid distance recovers 2× the recall with the same detector.
Track a small quadcopter (~10–30 px) performing aerobatic maneuvers over 960 frames (32 s). The target is:
- Small - often under 15 px, making IoU unreliable during fast motion
- Fast - large frame-to-frame displacement breaks standard MOT association
- Low contrast - requires morphological pre-processing to isolate the blob
Ground truth: 960-frame annotated clip in MOT format (annotation/quadcopter_chunk0/gt/gt.txt).
tracking_IR.py·detection_IR.py
New Top-Hat morphological transform isolates small bright blobs on a dark background. Detections feed a constant-velocity Kalman filter with a three-state machine: DETECTING → TRACKING → COASTING.
No GPU required. Designed for embedded deployment constraints.
python tracking_IR.py --mode display --tracker kalman
python tracking_IR.py --mode display --tracker particle # SIR particle filter variant
python tracking_IR.py --mode config # live trackbar tuning
| Argument | Values | Default |
|---|---|---|
--mode |
display / config |
config |
--tracker |
kalman / particle |
kalman |
--video |
path to .mp4 |
quadcopter_aerobatics/chunk_00.mp4 |
--output |
path to .txt |
quadcopter_chunk0/predictions_IR.txt |
config mode opens a live parameter panel (Dear PyGui) to tune detection and Kalman parameters interactively, alongside a 4-panel pipeline debug view. display mode loads config.json and runs clean.
| Parameter | What it controls |
|---|---|
sigma_q (Process Noise) |
How much the Kalman trusts its motion model |
sigma_r (Measurement Noise) |
How much it trusts the detector |
assoc_dist |
Max centroid distance (px) to accept a detection as match |
max_coast |
Frames to coast without a detection before dropping to DETECTING |
Config mode
Display mode
Tracking in action (IR colormap, Kalman filter, 6 s clip):
kaggle_tracking_SOT.ipynb
YOLOv8n fine-tuned on Anti-UAV300 (18 k images, 50 epochs, Kaggle T4 GPU). Detections associated frame-to-frame using centroid distance rather than IoU.
Infrastructure note: Training was constrained to Kaggle's free T4 GPU tier (session time limits, no persistent storage). With access to the full Anti-UAV300 dataset, longer training runs, and proper augmentation pipelines, detection recall could realistically reach 0.95+. The current F1=0.887 reflects what is achievable under amateur infrastructure constraints, not the ceiling of the approach.
kaggle_tracking_SOT.ipynb
Standard ByteTrack MOT pipeline using IoU association. Included as a baseline to quantify the failure mode of IoU on fast small targets.
Evaluated on chunk_00.mp4 - 960 frames, 32 s. Metric: detection within 50 px of ground truth centroid.
| Pipeline | Recall | Precision | F1 | Centroid err. | ID sw. |
|---|---|---|---|---|---|
| IR morpho + Kalman (σ_q=50) | 0.575 | 0.783 | 0.663 | 8.6 px | 16 |
| IR morpho + Kalman (σ_q=10) | 0.688 | 0.807 | 0.742 | 8.5 px | 11 |
| Detection only (conf=0.1) | 0.629 | 0.992 | 0.770 | 3.2 px | x* |
| YOLO + centroid (conf=0.3) | 0.686 | 0.997 | 0.813 | 5.2 px | 12 |
| YOLO + centroid (conf=0.1) | 0.808 | 0.966 | 0.880 | 4.9 px | 14 |
| YOLO + centroid (conf=0.1, σ_q=3) | 0.817 | 0.974 | 0.887 | 4.9 px | 13 |
| ByteTrack IoU (conf=0.1) | 0.391 | 1.000 | 0.562 | 3.4 px | 37# |
* Detection-only has no temporal state; ID switches are not meaningful.
# ByteTrack ID switches counted as unique track IDs minus 1.
Full pipeline comparison - Top-Hat Morphology + Kalman vs YOLO + Kalman vs ByteTrack (960 frames, ground truth in green): (Click on the image bellow to access to a full demonstration video)
ByteTrack's IoU association assumes the predicted and detected bounding boxes overlap between frames. On a fast small drone, frame-to-frame displacement routinely exceeds the box width; IoU collapses to zero and the tracker loses the target. Switching to centroid distance doubles recall (0.391 -> 0.817) with the same detector.
Frames 700–960 see a recall drop in Pipeline A. Root cause: the drone scales down significantly as it flies away, falling below the morphological detector's minimum blob area. This is a detector sensitivity problem, not a Kalman tuning problem; the Kalman coasts correctly when detections drop, but can't recover once the blob is below threshold.
Predictions are exported in MOT format (one .txt per pipeline). Local evaluation against ground truth:
python metrics.py --pred quadcopter_chunk0/predictions_IR.txt \
--gt annotation/quadcopter_chunk0/gt/gt.txt
Metrics computed: Recall, Precision, F1, mean centroid error (threshold 50 px), ID switches.
.
├── tracking_IR.py # Pipeline A — Top-Hat + Kalman/Particle filter
├── detection_IR.py # New Top-Hat blob detector (standalone tuning)
├── hud.py # HUD overlay (FPS, state, centroid, trace)
├── metrics.py # MOT evaluation — Recall / Precision / F1
├── config.json # Tuned detection + Kalman parameters
├── show_vid.py # Utility — display a video file
├── kaggle_tracking_SOT.ipynb # Pipeline B+C — YOLO training + ByteTrack
└── annotation/
└── quadcopter_chunk0/
└── gt/gt.txt # Ground truth (MOT format, 960 frames)
Video files (.mp4) and model weights (.pt) are not committed, they are too large.
Download the test clip separately and place it at quadcopter_aerobatics/chunk_00.mp4.