ufc-fight-model

A UFC fight-outcome model scored against the closing betting line instead of against a coin flip.

This began as a GeeseHacks Waterloo 2025 team project. The original is at Imran-k8/geesehacks, and the dataset comes from there. This repo is a from-scratch rewrite of the modelling. See Credits.

Results

6,528 fights, March 2010 to December 2024. Train on the 5,524 fights before 2023-01-01. Test on the 1,004 fights from 2023-01-01 onward, scored on the 786 that carry a two-sided closing line, because those are the only fights where the market can be scored on the same rows.

model	accuracy	AUC	log loss	Brier
market (closing line)	0.6959	0.7540	0.5885	0.2013
always-Red baseline	0.5573	0.5000	0.6881	0.2475
logistic regression	0.6768	0.7471	0.5928	0.2040
gradient boosting	0.6819	0.7586	0.5870	0.2006
gradient boosting, no market columns	0.6336	0.6951	0.6269	0.2192

Read the accuracy column first. The market gets 69.6% on these fights. A model that reports 66% accuracy is not a good model, it is worse than backing every favourite and going home. Accuracy figures published without the line next to them are unfalsifiable.

Now read the rest of the row. The boosted model beats the market on log loss, Brier score and AUC, and loses to it on accuracy. Four metrics, two different winners, one test set. That is the whole argument for not shipping a single number.

Is the edge real

No.

Bootstrap over per-fight log loss, 20,000 resamples of the 786 test fights, seed 0. The statistic is market log loss minus model log loss, so positive favours the model.

point estimate     +0.00151
95% CI             [-0.01481, +0.01771]
P(model better)    0.584

The interval crosses zero. The AUC gap does the same: point estimate +0.0047, 95% CI [-0.0124, +0.0223]. On 786 fights the model is indistinguishable from the closing line. It does not beat it, and nothing in this repo claims otherwise.

The last table row is the other half of the story. Strip every market-derived column and the model still reaches AUC 0.6951 from public fighter statistics alone, against the market's 0.7540. That recovers about three quarters of the market's discrimination above chance without ever seeing a price.

Quickstart

Python 3.11 or newer. Runs offline from the committed CSV.

git clone https://github.com/utosrad/ufc-fight-model.git
cd ufc-fight-model && python3 -m venv .venv && .venv/bin/pip install -e .
.venv/bin/ufc-model evaluate

That prints the table above plus both bootstrap intervals in a few seconds. ufc-model train fits the models and prints the table without the bootstrap, and takes --save DIR to write the fitted pipelines. ufc-model describe prints dataset facts.

To run the suite:

.venv/bin/pip install pytest && .venv/bin/python -m pytest -q

79 tests, a few seconds.

How it works

src/ufc_model/
  data.py        loading, date sort, temporal split
  market.py      American odds to de-vigged probability
  features.py    feature selection and the leakage guard
  models.py      the estimators
  evaluate.py    metrics and the bootstrap
  experiment.py  the one experiment, wired together
  cli.py         train | evaluate | describe

Market probability. American money lines convert as p = -o/(-o+100) when o < 0 and p = 100/(o+100) when o > 0. The two sides sum to more than 1 by the bookmaker's margin, mean 1.0346 across this dataset. Dividing each side by that sum strips the vig proportionally. That gives a calibrated forecast that can be scored with the same metrics as any model.

Split. By date, never at random. A random split lets a fighter's 2024 results inform a prediction about their 2019 fight. test_data.py asserts the last training date is strictly before the first test date.

Feature set. All 101 numeric and boolean columns that are knowable before the opening bell. Career records, streaks, physical measurements, ranks, and the closing odds. No text encoding, no fighter identity columns.

Models. HistGradientBoostingClassifier(max_depth=4, learning_rate=0.05, max_iter=300, random_state=0) behind a median imputer. Logistic regression adds a standard scaler and uses C=0.1, max_iter=2000. Neither is exotic. The interesting part is the benchmark, not the architecture, and a shallow boosted tree is already at the ceiling these features allow.

Leakage guard. features.BANNED_COLUMNS lists every column that only exists after the fight ends. build_matrix raises LeakageError if any of them reaches the feature matrix, and tests/test_leakage.py fails the build rather than quietly producing a good-looking number.

A leak that looks like a fix

The no-market row is worth spelling out, because the obvious way to build it is wrong.

Dropping every column whose name contains Odds leaves RedExpectedValue and BlueExpectedValue in place. Those are profit on a 100 unit stake, which is a deterministic transform of the money line: 99.8% of rows match 10000/-o for favourites and o for underdogs to within 0.01, at correlation 0.99998. A model built that way scores AUC 0.7490 and looks like proof that fighter statistics alone rival the market. They do not. Drop the ExpectedValue pair too and AUC falls to 0.6951. Both numbers are asserted in tests/test_results.py.

Name-matching is not a leakage audit.

What the rewrite fixed

The hackathon original placed 4th and shipped in a weekend, which is the job at a hackathon. The modelling underneath it had defects worth naming, since they are the reason this rewrite exists.

• Three models were described as an ensemble. All three were plain LogisticRegression.
• There was no training code in the repository, only pre-built .joblib files. Nothing could be reproduced, retrained, or checked.
• df["TitleBout"] == "TRUE" matched 0 of 6,528 rows, because pandas parses that column as bool before the comparison runs. The title-bout flag was always zero. test_data.py pins the correct count at 302 and keeps the broken comparison as a regression test.
• TotalFightTimeSecs was a feature for predicting FinishRound. Those two correlate 0.978. The model was reading the clock, not forecasting. That column is on the banned list, and test_leakage.py asserts the 0.978 correlation so the reason stays documented.
• All three classification_report calls passed the same model object, so two of the three models were never actually evaluated.
• The Flask API used @app.routes(...). That is not a Flask decorator and raises AttributeError on import. There is no web layer here at all, because the result is a table and a confidence interval.

None of the original numbers were measured against the line, which is the defect that motivated the rest.

What would actually be needed to beat the line

Nothing in this dataset supports a genuine edge, and it is worth being specific about what is missing rather than vaguely promising more feature engineering.

• Line movement over time. This dataset has one closing price per fighter per fight. The information is in the path from open to close: which way the money moved, how fast, and whether it moved against public sentiment. A single closing number is the market's answer with the working erased.
• Injury and camp news. Withdrawals, short-notice replacements, weight-cut trouble, gym changes and reported injuries move real prices and appear nowhere in these 118 columns. The market prices them within hours. A model reading only career statistics cannot.
• Per-fight odds history across books. Beating a line requires finding books that disagree, and timing the entry. That needs a time series per fight per sportsbook. This dataset has a single consensus snapshot.

Without those three, the honest ceiling is what the table shows: match the close, do not beat it.

Two further cautions on this dataset. 302 of 6,528 fights are title bouts, 4.6%, so accuracy claims restricted to title fights sit on a sample far too small to mean anything, and this repo makes none. And the Red corner wins 58.0% of all fights, so any headline accuracy under about 58% is losing to a one-line heuristic.

Data

data/ufcmaster.csv, 6,528 fights and 118 columns, 2010-03-21 to 2024-12-07. 6,290 fights carry a two-sided closing line. Committed to the repo so every result here reproduces offline.

Credits

This project began as a team entry at GeeseHacks Waterloo 2025, which placed 4th. The original repository is Imran-k8/geesehacks and the dataset in data/ comes from it. Credit for the original project and its dataset belongs to that team.

The code in this repository is a from-scratch rewrite of the modelling: the pipeline, the market benchmark, the leakage guard, the evaluation and the test suite. It shares no code with the original.

License

MIT. See LICENSE.