ABSTRACT
This chapter provides methods for measuring the performance of a probabilistic model. It discusses the rather popular rank-based approaches for two-state problems, approaches more closely related to our utility-based approach: the likelihood, performance measurement via a loss function and model performance measurement in the author’s utility-based framework. Receiver operator characteristic (ROC) methods arise naturally from basic concepts in statistical hypothesis testing. ROC plots display information on the false positive rates and true positive rates for a collection of tests generated by a model. The likelihood of a model is the probability of observing a given set of data under the assumption that the model is correct. Measuring model performance by means of the likelihood is closely related to the likelihood principle, which states that the information provided be an observation about a model is entirely contained in the likelihood of the model. The Bayesian framework provides an alternative to the classical approach to estimating parameters of a probabilistic model.
