ABSTRACT
An important part of any statistical analysis is the work done to check that the ˜tted model is ˜t for purpose, which often provides clues on how to improve the model. Skill in building statistical models is a key attribute of an experienced statistician. When there are numerous potential predictor variables that should be considered for inclusion, this task becomes particularly challenging. Ironically, it is in these cases when we have an abundance of potentially useful predictors that we are beset by some of the more dif˜cult statistical issues. It is well known that indiscriminately including numerous variables in a model leads to over-˜tting of the data, which results in a model with very poor predictive ability.* Hence, many statisticians have devoted large parts of their lives developing methods to help determine which variables to include in our models. Still the most common general approach is based on the concept of the p-value, in which the statistical signi˜cance of each variable is assessed, and only included if it passes some nominal threshold, typically less than 5%. Software packages such as SAS, Genstat, and R will routinely provide t and F tests for this purpose as part of their output after ˜tting linear models. The stepwise regression procedures are based on this approach (see Draper and Smith, 1998). A crude Bayesian version of this approach would be to simply inspect the posterior credible interval for a parameter, and decide based on whether zero was a credible value. Other uniquely Bayesian model checking functions, based on predictive distributions, also exist (see Section 2.9 for information on those provided by BugsXLA). Global measures of goodnessof-˜t, such as the deviance, or information criteria such as the AIC (see Brown and Prescott, 2006), as well as the Bayesian DIC, have all been used to choose between models. Irrespective of which approach is used to identify the ˜nal model, if all the inferences are based solely on this model, then a potentially major source of uncertainty will be ignored, namely, uncertainty about the model itself. As a consequence, it is likely that uncertainty about quantities of interest will be underestimated. A vast amount of research into this topic exists, with Draper (1995) providing good coverage of the major issues and the Bayesian methods for dealing with them.
