✈️ Aviation Demand Forecasting

An end-to-end machine-learning project that forecasts next-day scheduled flight demand by route. It processes the 2015 US Flight Delays dataset, builds leakage-safe time-series features, compares multiple model families using chronological validation, and serves the selected model in Streamlit.

Project layout

app/                 Streamlit dashboard
assets/              Dashboard and model-comparison images
data/raw/            Source CSV files (flights.csv is not committed)
data/processed/      Reproducible pipeline outputs
models/              Selected model, metadata, and experiment artifacts
notebooks/           EDA, feature engineering, and modeling narratives
src/                 Reusable pipeline and inference code

Forecast definition

• Target: next-day number of scheduled flights for each route.
• Scope: 100 busiest routes with valid three-letter IATA origin and destination codes.
• Cancelled flights: included because they represent scheduled demand.
• Diverted flights: counted on their originally scheduled routes.
• Missing route-days: filled with zero.
• Numeric airport identifiers: converted to IATA codes using the included BTS airport-ID lookup; still-unmapped identifiers are excluded.

Models compared

• Previous-day naive baseline
• Seven-day seasonal naive baseline
• Ridge regression
• Random Forest
• Histogram Gradient Boosting
• LightGBM
• XGBoost
• CatBoost

All candidates use the same features and three expanding-window validation folds. The primary selection metric is mean absolute error (MAE); RMSE, WAPE, variability across folds, and training time are also reported.

Setup

Python 3.12 is recommended. On macOS, LightGBM and XGBoost may also require the OpenMP runtime (brew install libomp).

python3 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install -r requirements.txt

Download the US Flight Delays Dataset (2015) and place it at data/raw/flights.csv. The smaller airlines.csv and airports.csv files are included.

data/raw/airport_id_mapping.csv provides the BTS airport ID/IATA mapping needed for a source-format change in the final quarter. Its airport geographic records originate from the US Department of Transportation/Bureau of Transportation Statistics.

Run the pipeline

Run commands from the repository root:

python src/preprocessing.py
python src/feature_engineering.py
python src/train_model.py
python src/predict.py
streamlit run app/streamlit_app.py

Generated outputs include:

• data/processed/route_demand.csv
• data/processed/featured_data.csv
• data/processed/model_results.csv
• models/experiments/*.pkl
• models/model.pkl
• models/model_metadata.json
• assets/model_comparison.png

Notebooks

The notebooks document the analysis and call the same reusable logic as src/. Pipeline transformations live in Python modules rather than being hidden only in notebook cells.

Limitations

The source contains only 2015 observations. The dashboard is therefore a reproducible historical ML demonstration, not a live aviation-demand service. Operational forecasting would require current data, monitoring, retraining, and external drivers such as holidays, events, fares, and weather forecasts.

Existing result

The original repository reported a LightGBM MAE near 0.37 on one route. The enhanced pipeline intentionally replaces that result with a multi-route, leakage-safe, time-validated comparison, so its metrics are not directly comparable to the original experiment.