To combine or not to combine: selecting among forecasts and their combinations

Much research shows that combining forecasts improves accuracy relative to individual forecasts. In this paper we present experiments, using the 3003 series of the M3-competition, that challenge this belief: on average across the series, the best individual forecasts, based on post-sample performance, perform as well as the best combinations. However, this finding lacks practical value since it requires that we identify the best individual forecast or combination using post sample data. So we propose a simple model-selection criterion to select among forecasts, and we show that, using this criterion, the accuracy of the selected combinations is significantly better and less variable than that of the selected individual forecasts. These results indicate that the advantage of combining forecasts is not that the best possible combinations perform better than the best possible individual forecasts, but that it is less risky in practice to combine forecasts than to select an individual forecasting method.