The predictive accuracy of credit ratings: Measurement and statistical inference

Credit ratings are ordinal predictions of the default risk of an obligor. The most commonly used measure for evaluating their predictive accuracy is the Accuracy Ratio, or equivalently, the area under the ROC curve. The disadvantages of these measures are that they treat default as a binary variable, thus neglecting the timing of default events, and they fail to use all of the information available from censored observations. We present an alternative measure which is related to the Accuracy Ratio but does not suffer from these drawbacks. As a second contribution, we study statistical inference for the Accuracy Ratio and the proposed measure in the case of multiple cohorts of obligors with overlapping lifetimes. We derive methods which use more sample information and lead to tests which are more powerful than alternatives which filter just the independent part of the dataset. All procedures are illustrated in the empirical section using a dataset of S&P Credit Ratings.