Medium-Range Model Performance

There's been some chatter among meteorologists in recent days about unusually poor forecast performance by the U.S. Global Forecast System, a model that predicts the global circulation for weather forecasts out to 15 days.  Summer is a more challenging time for weather forecasts in general, because relatively unpredictable small-scale features are more important than in winter.  But even compared to normal for the time of year, the recent performance has been poor.

The pair of maps below shows a 7-day forecast of 500mb height anomaly (departure from normal) on the left and the ensuing verification on the right; the forecast was issued on May 14.  While some of the features were correctly anticipated, the forecast was badly wrong from easternmost Siberia to Alaska and also in western Europe.  The spatial correlation of anomalies ("anomaly correlation") was only 0.26.

Here's the same pair of maps but for the 7-day forecast from ECMWF.  It's widely known than ECMWF's data assimilation and modeling methods produce generally superior weather forecasts compared to GFS, but in this case the difference was dramatic.

To put this event in context, I calculated the daily anomaly correlations over the Northern Hemisphere north of 20°N for 7-day forecasts back to early 2016, and I also looked at Environment Canada's global model forecasts (labeled as "CMC" in the charts below).  The first chart shows the dramatic drop-off in skill in recent days for both GFS and CMC; both models have suffered the same fate, but ECMWF has remained relatively unscathed.  Note, however, that ECMWF is not without fault - it had a remarkable failure last summer.

If we plot the differences between the models, we see that the ECMWF forecasts have outperformed more dramatically in recent days than at any other time in the past year or so.

Zooming in on the North Pacific region (see below), there is of course more noise in the statistics, but similar patterns are evident.  The CMC model has not fared quite as badly as GFS in recent days, but again the ECMWF has performed much better than the other two models.

In conclusion - let's hope the current episode proves to be instructive for the model developers so that the science can continue to advance.  NOAA scientists are paying attention to these forecast skill "dropouts" - see for example the article beginning on page 5 of the following newsletter:

https://www.jcsda.noaa.gov/documents/newsletters/2017_02JCSDAQuarterly.pdf

The bigger picture from the results I've shown here is that modern global circulation models now have the ability to predict weather patterns out to 7 days or more with fairly good skill on average, and I find that to be a remarkable achievement.