ABSTRACT
This chapter deals with the general classification problem. We wish to find a model, using a training set of data, which we can use to predict the (categorical) response of future observations given just their covariates. Using classical generalized linear model (GLM) theory for categorical data (Chapter 5, McCullagh and Neider, 1989) is restrictive and the resulting model can be difficult to interpret. Due to these problems interest in performing classification using tree-based models has grown, especially by applied statisticians. The ease of interpretability and simple output of the model, especially its rule-based nature, is particularly appealing. This leads to definite classes being assigned to each datapoint rather than possible class probabilities. While this may not seem to incorporate the uncertainty in class assignments its definiteness is seen as important to decision makers (e.g. doctors, insurance salesmen) even though this may not appear completely reasonable to more theoretical statisticians.
