ABSTRACT
This chapter describes, at a simple level, a widely used tool in statistics. Hypothesis testing is the process of dichotomizing the possible outcomes of a statistical study and using probabilistic arguments to choose one option over the other. The specific application considered in this chapter is how to decide whether an estimate of AUC is consistent with a prespecified value. The concepts of null and alternative hypotheses, and Type-I and Type-II errors are introduced and illustrated with R examples. The role of prespecified values of alpha and beta as controls on Type-I and Type-II errors, respectively, is explained, as are the ubiquitous terms p-value and statistical power. The plot of empirical power (ordinate) vs. empirical alpha, termed by Metz the ROC within and ROC is explained. The reasons behind the conventional choices of alpha = 5% and beta = 20% are discussed. Very different values are adopted in other fields such as particle physics and cosmology. While comparing a single reader's performance to a specified value is not a clinically interesting problem, and the next two chapters describe methods for analyzing interpretations by a group of readers of a common set of cases in typically two modalities, the basic concepts remain the same.
