This chapter introduces the first widely used class of schemes for stochastic simulation using Markov chains. It is generically known as Gibbs sampling because it originated in the context of image processing. In this context, the posterior of interest for sampling is a Gibbs distribution. Borrowing concepts from Mechanical Statistics, the density of the Gibbs distribution can be written as

f(x1, . . . , xd) ∝ exp [ − 1

kT E(x1, . . . , xd)

] (5.1)

where k is a positive constant, T is the temperature of the system, E is the energy of the system, a positive function, and xi is the characteristic of interest for the ith component of the system, i = 1, . . . d. In Mechanical Statistics, xi is the position or perhaps the velocity and position of the ith particle and in image processing it is (an indicator of) the colour of the ith pixel of an image.