Ranks Toronto Pearson (YYZ) flights by how much of their lateness they hand to the rest of the day, instead of by how late they are.
Built for the I4 Hackathon at Waterloo, where it took 1st place.
A 40 minute arrival delay on an aircraft with six more legs that day is a different problem from a 40 minute delay on an aircraft that is done flying. Airports triage by minutes late. This ranks by transmitted minutes.
Borrowing the term from finance, each flight gets a delay beta:
delay_beta = predicted_arrival_delay * (1 + downstream_legs_remaining)
downstream_legs_remaining is the number of later flights the same tail number still has on the schedule. An XGBoost regressor supplies predicted_arrival_delay from seven features: prior leg delay, departure hour, peak-hour flag, encoded carrier, encoded origin, widebody flag, and departure runway delay.
Everything below comes from the CSVs committed in this repo. Re-derive any of it with the commands in the next section.
Raw flight movements (data/raw/yyz_flights.csv) |
1,571 rows |
| Time span covered | 2025-12-19 09:30 to 2025-12-20 05:30 UTC, 20 hours |
Flights scored (data/processed/beta_scores.csv) |
528 |
| Distinct aircraft registrations | 220 |
| Distinct marketing airline codes | 57 |
| Flights with at least one downstream leg | 308 of 528 |
| Longest downstream chain | 11 legs |
| Mean downstream legs, per flight predicted >15 min late | 1.77 |
Top of the leaderboard is ACA743 (Air Canada), predicted 324.9 minutes late with 6 legs still to fly, normalized beta 100. It did in fact arrive 399 minutes late.
The gate stage takes the 50 highest-beta flights and reassigns them across the 112 gates observed in the data, using a PuLP/CBC integer program with a no-overlap constraint on gate occupancy windows padded by predicted delay. Overlapping same-gate pairs go from 86 to 0. With 112 gates for 50 flights the problem is not tight, so read that as a feasibility demo rather than an efficiency result.
Do not use the delay predictions. I checked them properly after the hackathon and they fail.
Sort the 528 scored flights by scheduled departure and hold out the last 20% (106 flights):
| MAE on held-out flights | |
|---|---|
| The XGBoost model | 83.4 min |
| Predicting zero for every flight | 11.8 min |
| The model, scored on its own training window | 3.4 min |
The model is about 7x worse than doing nothing.
The cause is the dataset, not the code. All 528 flights fit inside a single 20 hour window, so an 80/20 time split does not split like for like. It trains on the daytime congestion peak and tests on the overnight lull:
| Train slice (422 flights) | Test slice (106 flights) | |
|---|---|---|
| Mean actual arrival delay | 43.7 min | 11.8 min |
| Share more than 15 min late | 58% | 20% |
| Mean predicted delay | 43.7 min | 80.4 min |
The model learned "flights at YYZ run about 44 minutes late" and applied it to red-eyes that mostly ran on time. Training MAE of 3.4 minutes against test MAE of 83.4 minutes is the signature.
Fixing this needs more data, not more tuning: several weeks of movements so a time split holds the time-of-day mix constant, or a split by aircraft rather than by clock. The delay beta ranking, the graph, and the gate solver all work off whatever PREDICTED_DELAY column they are handed, so a better model drops straight in. The ranking machinery is the reusable part. The predictions are not.
Python 3.12 or newer. Verified end to end on CPython 3.14.2, macOS arm64, from a fresh git clone.
pip install -r requirements.txtpython scripts/2_compute_beta.py && python scripts/3_optimize_gates.py && python scripts/embed_data.pypython server.py # then open http://localhost:8080That runs the beta scoring, the gate solver, and the contagion graph off data/processed/delays.csv, which is committed. The three scripts reproduce the committed beta_scores.csv and data.js byte for byte, so git status stays clean afterwards.
Streamlit view of the same outputs, if you prefer it:
streamlit run dashboard.pyscripts/0_merge_data.py and scripts/1_predict_delays.py are the merge and train steps. They need a BTS / Kaggle US flight delay extract at data/raw/flights_sample_3m.csv, which is too large to commit and is not redistributable here. Without it script 0 exits with a message telling you what is missing. Everything downstream of it runs from the committed delays.csv, which is why the quickstart starts at script 2.
data/raw/yyz_flights.csv data/raw/flights_sample_3m.csv
(committed, 1,571 rows) (you supply this)
| |
+----------------+-----------------+
|
scripts/0_merge_data.py
|
data/processed/merged_flights.csv
|
scripts/1_predict_delays.py XGBoost, 7 features
|
data/processed/delays.csv <- committed, 528 rows
|
scripts/2_compute_beta.py beta = pred x (1 + downstream legs)
|
data/processed/beta_scores.csv
|
+------------------+------------------+
| | |
3_optimize_gates.py embed_data.py dashboard.py
PuLP/CBC MILP -> data.js Streamlit
| |
gate_assignments.csv index.html
D3 force graph
index.html renders the contagion graph as a D3 force layout: one node per flight in the top 40 by beta, sized and coloured by beta, with an edge joining consecutive legs of the same tail. Click a node and inject a delay to walk the cascade down that aircraft's remaining day. D3 and the web font load from a CDN, so the page needs a network connection. The Streamlit dashboard draws the same relationships with pyvis instead.
server.py also proxies the OpenSky REST API at /api/live-flights for live YYZ arrivals. Copy .env.example to .env and add OAuth2 credentials to enable it; without them that one endpoint returns 503 and the rest of the app works normally.
data/raw/ yyz_flights.csv (committed); your BTS extract goes here
data/processed/ delays.csv and beta_scores.csv (committed); other outputs ignored
data/figures/ generated plots, ignored
docs/ the two figures shown above
scripts/ 0_merge_data, 1_predict_delays, 2_compute_beta,
3_optimize_gates, embed_data
dashboard.py Streamlit dashboard
server.py Flask app plus OpenSky proxy
index.html contagion graph web app, reads data.js
lib/ vis-network and tom-select, emitted by pyvis for the dashboard
Beyond the validation failure above, four things I would fix before anyone leaned on this:
Beta ranks predictions, not reality. DELAY_BETA keys off PREDICTED_DELAY. Because the predictions are miscalibrated, 4 of the current top 5 flights by beta had an actual ARR_DELAY of 0. The ranking inherits every error the model makes.
Codeshares are counted as separate flights. One physical departure appears once per marketing carrier. Callsign ACA472 on tail C-GJXE shows up as six rows in beta_scores.csv, filed under Air Canada, Avianca, TAP Air Portugal, LOT, Cathay Pacific and Austrian, and takes three of the top ten leaderboard slots on its own. yyz_flights.csv already carries a codeshareStatus column (971 of 1,571 rows are IsCodeshared); the pipeline just never reads it. Dropping non-operator rows is the fix.
The gate solver's objective is degenerate. It minimizes sum(beta_i * x_ig) while constraining every flight to exactly one gate, which makes the objective a constant. CBC is therefore only finding a feasible conflict-free assignment, not a preferred one. A real objective would price gate changes against the original assignment, or weight buffer slack by beta.
Coverage is thin. Only 547 of 1,571 raw rows carry an aircraft registration, and the aircraft-chaining logic needs one, which is how 1,571 rows become 528. The largest feature in the importance chart above, DEP_RUNWAY_DELAY, is present on 108 of the 528 scored rows and median-filled on the other 420.
MIT. See LICENSE.