ABSTRACT
As themany examples in this book illustrate,Markov chainMonte Carlo (MCMC)methods have revolutionized Bayesian statistical analyses. Rather than using off-the-shelf models and methods, we can use MCMC to fit application-specific models that are designed to account for the particular complexities of a problem. These complex multilevel models are becoming more prevalent throughout the natural, social, and engineering sciences largely because of the ease of using standard MCMC methods such as the Gibbs and MetropolisHastings (MH) samplers. Indeed, the ability to easily fit statistical models that directly represent the complexity of a data-generationmechanismhas arguably lead to the increased popularity of Bayesian methods in many scientific disciplines. Although simple standard methods work surprisingly well in many problems, neither
the Gibbs nor theMH sampler can directly handle problemswith very high posterior correlations among theparameters. Themarginal distribution of a givenparameter ismuchmore variable than the corresponding full conditional distribution in this case, causing the Gibbs sampler to take small steps. With MH a proposal distribution that does not account for the posterior correlation either has far toomuchmass in regions of low posterior probability or has such smallmarginal variances that only small steps are proposed, causing high rejection rates and/or high autocorrelations in the resultingMarkov chains. Unfortunately, accounting for the posterior correlation requires more information about the posterior distribution than is typically available when the proposal distribution is constructed. Much work has been devoted to developing computational methods that extend the
usefulness of these standard tools in the presence of high correlations. For Gibbs sampling, for example, it is nowwell known that blocking or grouping steps (Liu et al., 1994), nesting steps (van Dyk, 2000), collapsing or marginalizing parameters (Liu et al., 1994; Meng and vanDyk, 1999), incorporatingauxiliaryvariables (BesagandGreen, 1993), certainparameter transformations (Gelfand et al., 1995; Yu and Meng, 2011), and parameter expansion (Liu andWu,1999) canall beused to improve the convergenceof certain samplers.Byembedding an MH sampler within the Gibbs sampler and updating one parameter at a time (i.e. the well-known Metropolis-within-Gibbs sampler in the terminology of Gilks et al., 1995), the same strategies can be used to improve MH samplers.
