Temperature Forecast

Temperature Forecast is a time series forecasting task aimed at predicting the daily temperatures using 10 years worth of recorded daily temperatures. The dataset used for this project contains two columns:

1. every day from 1981-01-01 to 1990-12-31

2. temperature for respective day.

These are time series column and target column respectively.

A preview of the dataset:

	ds	y
0	1981-01-01	20.7
1	1981-01-02	17.9
2	1981-01-03	18.8
3	1981-01-04	14.6
4	1981-01-05	15.8

For this project different tools and time series forecast models were used.

1) MICROSOFT EXCEL'S FORECAST SHEET

Microsoft Excel has this feature called Forecast Sheet. This forecast sheet uses the AAA version of the Exponential Smoothing algorithm (placing more weight on last values in dataset to make predictions). This algorithm, often referred to as the Holt Winters method, effectively handles additive trends, seasonality and smoothing of data to provide forecasts based on historical patterns. A confidence interval of 80% was used. The resulting forecast and its statistics are:

This was a good starting place. Next, Auto Regressive Intergrated Moving Average (ARIMA), Seasonal Auto Regressive Intergrated Moving Average (SARIMA) and Prophet from Meta models over at Python

2) ARIMA and SARIMA

ARIMA (Autoregressive Integrated Moving Average) models are statistical tools for time series forecasting that analyze past values, but they are best for non-seasonal data. SARIMA (Seasonal Autoregressive Integrated Moving Average) models extend ARIMA by incorporating an additional seasonal component to account for repeating patterns like monthly or yearly cycles, making them suitable for data with seasonality, such as hotel occupancy rates or monthly temperature data. For this dataset, it woould be best to use SARIMA due the daily seasonality being present but to highlight differences both models will be utilized. This dataset contains only two columns hence there's no need to worry about exogenous factors or correlation of variables.

Before introducing ARIMA and SARIMA, it would be a good idea to see how our baseline models perform. These models include:

a) Naive

b) SeasonalNaive

c) Historical Average

d) Moving Average

A simple plot reveals:

The MAE (mean absolute error) of these models are:

	metric	Naive	HistoricAverage	WindowAverage	SeasonalNaive
0	mae	3.9	2.727477	0.746939	1.628571

The SeasonalNaive baseline model has a lower mean absolute error than Excel's forecast meaning its more accurate. This is great as another benchmark to use for our next model.

Now onto to ARIMA and SARIMA.

The ARIMA and SARIMA (by adding seasonality to the ARIMA model) is gotten from the StatsForecast library.

The mean absolute error would be the only metric used to measure the model's accuracy.

	metric	ARIMA	SARIMA	Naive	HistoricAverage	WindowAverage	SeasonalNaive
0	mae	2.284992	2.10385	3.9	2.727477	0.746939	1.628571

So the SARIMA model has the same MAE score as the Excel Forecast Sheet model.

Cross Validation will now be applied to further ensure these models work on newer data.

Cross Validation

Cross-validation is a statistical technique used to assess a model's performance and its ability to generalize to new, unseen data. It works by repeatedly splitting the dataset into training and testing sets, allowing the model to be trained on one part and validated on another, a process that is iterated to provide a more reliable performance estimate. The primary goal of cross-validation is to prevent overfitting, ensuring the model doesn't just "memorize" the training data but can adapt to real-world data. In time series forecasting, cross validation can be achieved introducing a cutoff on the time series and data before this cutoff is used an the training section and after the cut off is what the model is validated on. The cutoff will be set on 1990-11-05. The result of the cross validation technique is:

	unique_id	ds	cutoff	y	SeasonalNaive	ARIMA	SARIMA
0	temp_series	1990-11-06	1990-11-05	18.3	14.9	12.798342	13.048451
1	temp_series	1990-11-07	1990-11-05	19.2	14.8	12.347443	12.453626
2	temp_series	1990-11-08	1990-11-05	15.4	15.4	12.191058	12.319272
3	temp_series	1990-11-09	1990-11-05	13.1	11.8	11.990521	11.966940
4	temp_series	1990-11-10	1990-11-05	11.5	13.0	12.031094	11.730668

The cross validation plot results to:

The Mean Absolute Error (MAE), Mean Absolute Percentage Error (MAPE), Mean Squared Error (MSE), Root Mean Squared Error (RMSE) and Symmetric Mean Absolute Percentage Error (SMAPE) will be the metrics used to measure the performance of the cross validation:

	metric	SeasonalNaive	ARIMA	SARIMA
0	mae	2.789286	2.235611	2.183218
1	mape	0.209902	0.160493	0.157526
2	mse	12.917857	7.764852	7.333071
3	rmse	3.594142	2.786548	2.707964
4	smape	0.101092	0.084093	0.081926

3) PROPHET

The Prophet time series model, from Meta, decomposes a time series into trend, yearly, weekly, and daily seasonality, and holiday effects, and is particularly useful for data with strong seasonal patterns, missing data, and outliers. When future values for a year are predicted, we get a plot like so:

Cross Validation

When cross validation is then applied using the same cutoff used earlier, the resulting plot is like so:

The perfomance metrics of the first five rows of this cross validation are:

	horizon	mse	rmse	mae	mape	mdape	smape	coverage
0	5 days	17.844728	4.224302	3.426022	0.199002	0.192404	0.229538	0.6
1	6 days	13.879526	3.725524	3.017434	0.223897	0.192404	0.225933	0.6
2	7 days	4.965177	2.228268	1.685708	0.154633	0.085757	0.142033	0.8
3	8 days	3.433108	1.852865	1.304682	0.131484	0.076658	0.115399	0.8
4	9 days	3.974670	1.993657	1.530976	0.145994	0.085757	0.131222	0.8

The last five rows are:

	horizon	mse	rmse	mae	mape	mdape	smape	coverage
47	52 days	4.674090	2.161964	1.452697	0.131337	0.044491	0.113133	0.8
48	53 days	4.875450	2.208042	1.591150	0.141638	0.078920	0.122909	0.8
49	54 days	1.178939	1.085790	0.940395	0.070272	0.078920	0.067081	1.0
50	55 days	0.810778	0.900432	0.804597	0.056935	0.064717	0.055798	1.0
51	56 days	1.529449	1.236709	1.164728	0.084802	0.078920	0.081682	1.0

The Mean Absolute Squared Error will also be included:

	mase
0	0.403013
1	0.429223
2	0.429314
3	0.438160
4	0.501666

	mase
47	1.827655
48	1.974522
49	2.056712
50	2.392583
51	2.483410

Plotting this metric results in: