From Baselines to a Forecaster
Introduction
Project introduction
From Baselines to a Forecaster
You have written the pieces of a forecaster, which are the seasonal naive forecast, MASE, lag rows without leakage, walk-forward splits and a recursive forecast that feeds its own predictions back in.
In the notebook they run on three years of real hourly demand from 316 customers in Portugal. 2012 and 2013 are the history, and every hour of 2014 is a forecast of the next 24 hours, 8,737 forecasts in all. The pieces you wrote in the coding problems come already written there and explained. Six TODO cells ask for the work around them. You let the data pick the season, find the days a forecast fails, choose how much history the model reads on a validation year, run a walk-forward test, catch a model peeking at the future and put a range around the forecast.
What you will find out
Copying yesterday is hard to beat. It reaches a MASE of about 0.92 on 2014, and copying last week does worse because this data has only a weak weekly pattern. A linear model on the past week of hours plus the hour and day reaches about 0.80, roughly 13 percent less error, and a small neural network does not beat it. A forecast one hour ahead misses by about 3 MW, and from about 10 hours ahead the error levels off near 6.3 MW.
Then a nearest-neighbour model that forecasts from similar days scores about 0.79 under shuffled cross-validation, as good as the linear model. Tested walk-forward it scores about 0.89, barely better than copying yesterday. You will measure where its fake advantage came from. Hiding the hours within a day of each forecast removes a small part of it, and hiding the weeks around each forecast removes the rest, including weeks after the forecast that a real forecaster could never see.
Forecast the next 24 hours
June 2014, hourlyHow the notebook is organized
01
Baselines
The daily and weekly cycles, seasonal naive forecasts, MAE and MASE on every hour of 2014, and the days copying yesterday fails.
02
Models
Lag rows with calendar columns, a linear model whose window is checked on 2013, a small neural network, and recursive against direct forecasts.
03
Validation
Walk-forward folds, shuffled folds, a check that catches a model peeking at the future, and a forecast range that holds on 2014.
Forecast a year of hourly demand
Run the whole build in the browser, about 45 minutes.