ABSTRACT
This chapter obtains that a risk prediction model has been built which predicts event probabilities. The aim is now to assess how well the model predicts for new patients. The mainstay of model performance visualization is the calibration plot. It shows the expected outcome proportion across the spectrum of predicted probabilities and where those predicted probabilities tend to lie. There are two fundamental metrics that are always calculated for assessing prediction model performance. One is the area under the receiver operating characteristic curve. The other metric the authors advocate is the Brier score, and derived from it the Index of Prediction Accuracy. To illustrate the computation of the Brier score in the case of uncensored binary outcome data we consider a logistic regression model fitted to the training dataset of the in vitro fertilization study. As in the uncensored case, it is illustrative to visualize the distribution of the predicted risks conditional on the outcome at the prediction time horizon.
