Skip to content
BranchesTags

Repository files navigation

FootballVision: AI-Powered Football Tactical Analysis & 2D Radar Mapping

Python PyTorch YOLOv8 OpenCV Streamlit License

An end-to-end computer vision and deep learning system for broadcast and tactical football footage analysis, featuring automated player/ball detection, pitch landmark homography projection, jersey color clustering, ball trajectory tracking, and an interactive Streamlit dashboard.

Table of Contents

Overview

FootballVision bridges the gap between raw tactical broadcast video footage and actionable tactical spatial intelligence. Analyzing sports broadcast video is inherently difficult due to perspective distortion, camera panning, dynamic lighting, player occlusions, and varying jersey styles.

FootballVision overcomes these challenges using a multi-stage computer vision pipeline:

  1. Detects players, referees, and the ball using a fine-tuned YOLOv8 Large model.
  2. Identifies 28 distinct pitch landmarks (corners, penalty box lines, arcs, center circle) using a fine-tuned YOLOv8 Medium model.
  3. Projects 3D perspective camera coordinates into a 2D top-down bird's-eye tactical radar map in real time via Planar Homography with temporal displacement smoothing.
  4. Predicts player team identities (outfield players and goalkeepers) using torso isolation, Web Palette color reduction, and CIE $L^a^b^$ Delta-E ($\Delta E^_{ab}$) majority voting.
  5. Tracks the ball path across successive frames with spatial gating and resets.
  6. Delivers an interactive GUI via Streamlit with interactive point-and-click color sampling, real-time visual playback, and MP4 video rendering.

Key Features

  • Dual-Stage YOLOv8 Architecture: Specialized, separate neural networks for actor detection (players, referees, ball) and pitch landmark geometric detection.
  • Planar Homography & Perspective Transformation: Converts broadcast camera coordinates to exact 2D coordinates on a standardized football pitch template.
  • Temporal Matrix Stabilization: Uses Mean Squared Error (MSE) displacement checks between frames to prevent tactical map jitter and flicker during smooth camera motion.
  • Perceptual Color Space Classification: Maps player kit colors into CIE $L^a^b^$ space and classifies teams via CIE76 $\Delta E^_{ab}$ distance voting, accounting for lighting changes and skin tones.
  • Click-to-Sample Color Calibration: Interactive Streamlit interface allowing users to click directly on detected player chips from any frame to calibrate team kit swatches.
  • Ball Tracking with Spatial Gating: Robust ball history tracking that filters out false-positive teleports and renders continuous trajectory polylines.
  • Full Hyperparameter Control: Live adjustments for confidence thresholds, keypoint tolerances, palette extraction depths, and track lengths.
  • Exportable Video Analysis: Automatically writes side-by-side composite videos (annotated broadcast + tactical 2D radar) to disk in MP4 format.

System Architecture & Methodology

flowchart TD
    A[Broadcast / Tactical Video Feed] --> B[Frame Extraction]
    
    subgraph Detection Pipeline
        B --> C[YOLOv8L Player Detector]
        B --> D[YOLOv8M Pitch Keypoint Detector]
        C --> C1[Bounding Boxes: Players, Referees, Ball]
        D --> D1[28 Field Landmark Classes]
    end
    
    subgraph Homography & Radar Projection
        D1 --> E[Extract Keypoint Centers]
        E --> F{Keypoints > 3?}
        F -- Yes --> G[MSE Temporal Stability Check]
        G --> H[Compute Homography Matrix H]
        F -- No --> I[Reuse Previous Homography H]
        C1 --> J[Extract Player Feet Ground Contact Points]
        C1 --> K[Extract Ball Center Position]
        H & J --> L[Transform Player Coords to 2D Pitch]
        H & K --> M[Transform Ball Coords to 2D Pitch]
    end
    
    subgraph Team Color Classification
        C1 --> N[Crop Player Bounding Boxes]
        N --> O[Apply Torso Center Filter]
        O --> P[Quantize to 216-Color Web Palette]
        P --> Q[Extract Top K Dominant RGB Colors]
        Q --> R[Convert to CIE L*a*b* Space]
        R --> S[Compute Delta-E Distance vs Team Kits]
        S --> T[Plurality Voting for Team Assignment]
    end
    
    subgraph Ball Tracking
        M --> U[Spatial Distance Gating]
        U --> V[Update Ball Track History Buffer]
        V --> W[Render Trajectory Polylines]
    end
    
    subgraph Composite Visualizer
        L & T --> X[Plot Color-Coded Players on Tactical Map]
        W --> X
        C1 & T --> Y[Annotate Video Frame with Bounding Boxes]
        X & Y --> Z[Side-by-Side Canvas + FPS Counter]
        Z --> OUT[Streamlit Display / MP4 Video Output]
    end
Loading

1. Dual YOLOv8 Deep Learning Detection

The system uses two separate fine-tuned YOLOv8 models optimized for different visual tasks:

  • Actor Detector (Yolo8L Players): Fine-tuned on high-resolution football match datasets to detect:
    • 0: player
    • 1: referee
    • 2: ball
  • Field Keypoint Detector (Yolo8M Field Keypoints): Fine-tuned to detect 28 geometric intersection landmarks across the pitch lines, penalty boxes, arcs, and center circle.

2. Pitch Landmark Detection & 2D Homography Projection

To project players from perspective camera view to a 2D top-down tactical map, we solve for the Planar Homography Matrix $H \in \mathbb{R}^{3 \times 3}$:

$$\begin{bmatrix} x' \ y' \ w' \end{bmatrix} = H \begin{bmatrix} x_{\text{src}} \ y_{\text{src}} \ 1 \end{bmatrix}, \quad \text{where } x_{\text{dst}} = \frac{x'}{w'}, \quad y_{\text{dst}} = \frac{y'}{w'}$$

Ground Contact Point Estimation

Instead of using bounding box centers (which map to player midsections in 3D space), player pitch coordinates are derived from their feet contact point on the ground: $$P_{\text{ground}} = \left( x_{\text{center}}, y_{\text{bottom}} \right) = \left( x_{\text{center}}, y_{\text{center}} + \frac{h}{2} \right)$$

Temporal Matrix Smoothing & Displacement Tolerance

Computing homography independently on every frame introduces visual jitter due to slight bounding box fluctuations. FootballVision implements a temporal error-gating algorithm:

  1. Identifies common keypoint labels between frame $t-1$ and frame $t$: $\mathcal{K}{\text{common}} = \mathcal{K}{t-1} \cap \mathcal{K}_t$.
  2. If $|\mathcal{K}{\text{common}}| \ge 4$, computes the Mean Squared Error (MSE) of coordinate displacement: $$\text{MSE} = \frac{1}{|\mathcal{K}{\text{common}}|} \sum_{i \in \mathcal{K}{\text{common}}} | p{t}(i) - p_{t-1}(i) |^2$$
  3. If $\text{MSE} \le \tau_{\text{tol}}$ (default 7-10 px), the existing matrix $H_{t-1}$ is preserved. If $\text{MSE} > \tau_{\text{tol}}$, $H_t$ is recomputed using OpenCV cv2.findHomography(src_pts, dst_pts).

3. Torso Extraction & Palette Quantization

Player bounding boxes often contain noise (green pitch grass, skin, shorts, socks, shoes). To isolate the pure jersey fabric:

  1. Torso Center Filter: An adaptive sub-region is cropped centered on the upper-chest: $$X_{\text{crop}} \in \left[ x_{\text{mid}} - 0.2w, ; x_{\text{mid}} + 0.2w \right]$$ $$Y_{\text{crop}} \in \left[ \frac{h}{3} - 0.2h, ; \frac{h}{3} + 0.2h \right]$$
  2. Web Palette Color Quantization: The cropped torso is converted to PIL's standard 216-color Web Palette (Image.Palette.WEB) to group similar RGB shades and eliminate pixel noise.
  3. Dominant Frequency Extraction: Colors are sorted by pixel count, and the top $K$ dominant colors (default $K=3$ to $5$) are extracted.

4. Perceptual Color Distance & Team Classification

Standard RGB Euclidean distance fails under stadium shadow and highlight conditions. FootballVision maps all colors into the *CIE $L^a^b^$ color space (where $L^$ represents lightness, and $a^, b^*$ represent color-opponent dimensions).

CIE76 Color Distance Formula ($\Delta E^*_{ab}$)

For each extracted player palette color $C_p$ and candidate team kit color $C_{\text{team}}$ (supporting both outfield and goalkeeper kits for both teams):

$$\Delta E^*_{ab} = \sqrt{(L_p - L_{\text{team}})^2 + (a_p - a_{\text{team}})^2 + (b_p - b_{\text{team}})^2}$$

Majority Voting Strategy

  • Each dominant color in the player's palette casts a vote for the team whose kit yields the minimum $\Delta E^*_{ab}$.
  • The final team label is assigned using plurality voting: $$\text{Team} = \operatorname{mode}(\text{Votes})$$

5. Ball Trajectory Gating & Historical Tracking

Because the ball is small and moves rapidly:

  • Spatial Proximity Gating: A new ball detection is appended to the track history only if its distance from the last known detection is below a threshold (ball_track_dist_thresh, default 100 px). If a sudden coordinate jump occurs, the track resets to eliminate false positives.
  • Track Expiry Buffer: If the ball is not detected for $> N$ consecutive frames (nbr_frames_no_ball_thresh, default 30 frames), the buffer is cleared.
  • Trajectory Polyline: Historical points up to max_track_length (default 35 detections) are mapped to 2D pitch coordinates and rendered as a smooth fading trail using cv2.polylines.

Pitch Landmark Keypoint Reference

The field keypoint detector classifies 28 standard pitch landmarks, mapped to fixed coordinates defined in pitch map labels position.json:

Keypoint Label Description Pitch Location
TLC / TRC Top-Left / Top-Right Corner Corner Flags (Top)
BLC / BRC Bottom-Left / Bottom-Right Corner Corner Flags (Bottom)
TL6MC / TR6MC Top-Left / Top-Right 6-Meter Center Goal Area Line Intersection
BL6MC / BR6MC Bottom-Left / Bottom-Right 6-Meter Center Goal Area Line Intersection
TL6ML / TR6ML Top-Left / Top-Right 6-Meter Line Goal Area Touchline Boundary
BL6ML / BR6ML Bottom-Left / Bottom-Right 6-Meter Line Goal Area Touchline Boundary
TL18MC / TR18MC Top-Left / Top-Right 18-Meter Center Penalty Box Center Intersection
BL18MC / BR18MC Bottom-Left / Bottom-Right 18-Meter Center Penalty Box Center Intersection
TL18ML / TR18ML Top-Left / Top-Right 18-Meter Line Penalty Box Touchline Boundary
BL18ML / BR18ML Bottom-Left / Bottom-Right 18-Meter Line Penalty Box Touchline Boundary
TLArc / TRArc Top-Left / Top-Right Penalty Arc Penalty D-Arc Intersections
BLArc / BRArc Bottom-Left / Bottom-Right Penalty Arc Penalty D-Arc Intersections
LML / RML Left / Right Midfield Line Halfway Line Touchline Junctions
LMC / RMC Left / Right Midfield Center Center Circle & Halfway Line Junctions

Model Performance & Evaluation

Both models were trained using Ultralytics YOLOv8 with extensive data augmentations (Mosaic, Horizontal Flips, HSV Color Jitter, Translation, Scaling).

YOLOv8L (Players, Referees & Ball)

  • Architecture: YOLOv8 Large (yolov8l.pt)
  • Dataset: Custom annotated tactical match frames (config players dataset.yaml)
  • Training Epochs: 30 (Batch Size: 8, Image Size: 640)
Metric Score
Precision (B) 91.62% (0.9162)
Recall (B) 69.33% (0.6933)
mAP@50 75.53% (0.7553)
mAP@50-95 51.68% (0.5168)

YOLOv8M (28 Field Landmark Keypoints)

  • Architecture: YOLOv8 Medium (yolov8m.pt)
  • Dataset: 28 pitch landmark intersections (config pitch dataset.yaml)
  • Training Epochs: 20 (Batch Size: 8, Image Size: 640)
Metric Score
Precision (B) 96.52% (0.9652)
Recall (B) 89.43% (0.8943)
mAP@50 94.51% (0.9451)
mAP@50-95 70.87% (0.7087)

Streamlit Web Application

The interactive web dashboard is located in Streamlit web app/ and contains three intuitive workflows:

┌────────────────────────────────────────────────────────────────────────┐
│                        FootballVision Dashboard                        │
├───────────────────┬────────────────────────────────────────────────────┤
│   Sidebar         │ Tab 1: How to use? (Walkthrough guide)             │
│   - Video Upload  │ Tab 2: Team Colors (Interactive Color Calibration) │
│   - Demo Selector │ Tab 3: Hyperparameters & Live Detection            │
│   - Team Names    │                                                    │
└───────────────────┴────────────────────────────────────────────────────┘
  1. How to use?: Complete user guide and workflow instructions.
  2. Team Colors:
    • Frame selection slider to inspect detections across any point in the video.
    • Clickable player grid powered by streamlit-image-coordinates to sample jersey and goalkeeper colors with a single click.
    • Color picker widgets for manual Hex/RGB adjustments.
  3. Model Hyperparameters & Detection:
    • Confidence threshold sliders for player and keypoint models.
    • Homography RMSE tolerance slider (keypoints_displacement_mean_tol).
    • Ball tracking thresholds (reset interval, distance gate, max path length).
    • Real-time side-by-side video rendering with live FPS counter.
    • Optional MP4 export to Streamlit web app/outputs/.

Repository Structure

FootballVision/
├── FootballVision.ipynb             # Research, pipeline prototyping & validation notebook
├── README.md                        # Master project documentation
├── requirements.txt                 # Core Python dependencies
├── environment.yml                  # Full Conda virtual environment export
├── config pitch dataset.yaml        # Dataset definition for 28 field keypoint classes
├── config players dataset.yaml      # Dataset definition for player, referee, and ball classes
├── pitch map labels position.json   # Ground-truth 2D coordinates for pitch landmarks
├── tactical map.jpg                 # 2D standardized tactical pitch template image
├── test vid.mp4                     # Sample tactical match test clip
├── Streamlit web app/
│   ├── main.py                      # Streamlit application UI and state management
│   ├── detection.py                 # Core analytical pipeline & computer vision logic
│   ├── demo_vid_1.mp4               # Demo 1 clip (France vs Switzerland)
│   ├── demo_vid_2.mp4               # Demo 2 clip (Chelsea vs Manchester City)
│   └── outputs/                     # Directory for rendered MP4 analysis videos
└── models/
    ├── Yolo8L Players/              # Fine-tuned YOLOv8L actor detector
    │   ├── weights/best.pt          # PyTorch model weights
    │   ├── args.yaml, results.csv   # Training logs and configuration
    │   └── *.png, *.jpg             # Confusion matrices, PR curves, validation batches
    └── Yolo8M Field Keypoints/      # Fine-tuned YOLOv8M pitch landmark detector
        ├── weights/best.pt          # PyTorch model weights
        ├── args.yaml, results.csv   # Training logs and configuration
        └── *.png, *.jpg             # Confusion matrices, PR curves, validation batches

Installation & Setup

Prerequisites

  • Python: 3.10 recommended
  • CUDA Toolkit & cuDNN: CUDA 11.8 or higher (for GPU acceleration)
  • Git

Option 1: Conda Environment (Recommended)

# 1. Clone the repository
git clone https://github.com/Dheerajvarma1/FootballVision.git
cd FootballVision

# 2. Create the conda environment
conda env create -f environment.yml

# 3. Activate the environment
conda activate FootballApp

Option 2: Pip Virtual Environment

# 1. Clone the repository
git clone https://github.com/Dheerajvarma1/FootballVision.git
cd FootballVision

# 2. Create and activate a virtual environment
python -m venv venv

# On Windows:
venv\Scripts\activate
# On Linux/macOS:
source venv/bin/activate

# 3. Install dependencies
pip install --upgrade pip
pip install -r requirements.txt

Note

Ensure your PyTorch installation matches your system's CUDA version. If needed, install PyTorch manually from pytorch.org.

Usage Guide

Running the Streamlit Web Application

# Navigate to the Streamlit app directory
cd "Streamlit web app"

# Launch the Streamlit server
streamlit run main.py

The web dashboard will open automatically in your browser at http://localhost:8501.

Application Workflow:

  1. Choose a Demo or Upload a Video: Select Demo 1 (France vs Switzerland) or Demo 2 (Chelsea vs Man City), or upload any match footage (.mp4, .mov, .avi).
  2. Set Team Names: Enter the names for Team 1 and Team 2 in the sidebar.
  3. Calibrate Team Colors:
    • Go to the Team Colors tab.
    • Use the slider to pick a clear frame showing players and goalkeepers.
    • Select a target role (e.g., Team 1 P color, Team 1 GK color), then click on the corresponding player in the cropped image grid.
  4. Configure Hyperparameters & Run:
    • Go to the Model Hyperparameters & Detection tab.
    • Adjust detection thresholds, toggle visual annotations, and check Save output if desired.
    • Click Start Detection to begin real-time analysis!

Running the Research & Prototyping Notebook

To experiment with individual stages of the pipeline or inspect raw transformations:

jupyter notebook FootballVision.ipynb

Hyperparameters & Tuning Guide

Parameter Recommended Default Description
player_model_conf_thresh 0.60 Minimum confidence score for player, referee, and ball detections.
keypoints_model_conf_thresh 0.70 Minimum confidence score for field landmark keypoint detections.
keypoints_displacement_mean_tol 7 - 10 px RMSE tolerance for keypoint displacement between frames. Set to -1 to recompute homography on every frame.
num_pal_colors 3 - 5 Number of dominant colors extracted from player torso crops for team voting.
nbr_frames_no_ball_thresh 30 frames Consecutive frames without ball detection before resetting the trajectory history.
ball_track_dist_thresh 100 px Maximum allowed pixel jump between consecutive ball detections (filters teleport noise).
max_track_length 35 detections Maximum historical ball coordinates stored and rendered on the tactical map.

Tech Stack

  • Deep Learning: PyTorch, Ultralytics YOLOv8
  • Computer Vision: OpenCV (cv2), Scikit-Image (skimage.color, Delta-E CIE76)
  • Web Application: Streamlit, streamlit-image-coordinates
  • Data & Math: NumPy, Pandas, Scikit-Learn
  • Image Processing: Pillow (PIL)
  • Configuration: PyYAML, JSON

About

www.youtube.com/watch?v=7vjQsNwXDew

Resources

Readme
Activity

Stars

2 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

Packages

Contributors

Languages