Hindcasts Skill Assessment

This website is a tool for the discusion within U.S. CLIVAR Working Group on Decadal Predictability of verification metrics towards the development of a verification framework for decadal hindcasts. If you have any questions or comments, please contact Lisa Goddard.

The different sections present results from the verification assessment of a few of the decadal hindcast experiments, mainly of CMIP5. For further details of the models and the forecast approach taken by each of the centers, please visit the CMIP5 data page: http://cmip-pcmdi.llnl.gov/cmip5/.

The verification metrics are chosen to answer specific questions regarding the quality of the forecast information. For example, they can identify where errors or biases exist in the forecasts to guide more effective use of them. The proposed questions address the accuracy in the forecast information and the representativeness of the forecast ensembles to indicate forecast uncertainty. Specifically, these questions are:

  1. Do the initial conditions in the hindcasts lead to more accurate predictions of the climate?
  2. Is the model's ensemble spread an appropriate representation of forecast uncertainty on average?

Overview

Select the parameters for the image:

Start Dates Model Metric Variables Spatial Scale Year Range Season  

Overview

Is the model's ensemble spread an appropriate representation of forecast uncertainty on average?

Select the parameters for the image:

Start Dates ModelMetricVariables Spatial Scale Year Range Season  

Observational Datasets - Data Library

Indexes - Data Library

Sample MATLAB® Code for Data Library

  1. Get observations from the Data Library and store in MATLAB®:
  2. Interpolate model simulations to fit the observation's grid (and more):
  3. Spatially smooth hindcasts and observations :
  4. Bias correction and dealing with a single uninitialized run :
  5. Local Mean Square Skill Score (MSSS) :
  6. Mean Square Skill Score (MSSS) of spatially smooth anomalies :
  7. Local correlation between hindcasts and observations :
  8. Correlation between spatially smooth hindcasts and observations :
  9. Local continuous ranked probability score (CRPS) :
  10. Continuous ranked probability score (CRPS) using smooth hindcasts and observations :
Version: dpwg-1.4.6 (2014-07-10 12:54:03 -0400)