Markov chain Monte Carlo (MCMC) is a family of algorithms used to produce approximate random samples from a probability distribution too difficult to sample directly. The method produces a Markov chain that whose equilibrium distribution matches that of the desired probability distribution. Since Markov chains are typically not independent, the theory of MCMC is more complex than that of simple Monte Carlo sampling.
MCMC was invented in 1953 but was not widely applied until statisticians became aware of it around 1990. Then problems that had previously been intractable suddenly became tractable. MCMC worked like magic. (We will return to the comparison with magic below.)
One of the most popular books on MCMC to date has been Markov Chain Monte Carlo in Practice. This book was edited by Gilks, Richardson, and Spiegelhalter and published by Chapman & Hall in 1995, just five years after the use of MCMC became wide-spread among statisticians. Now Chapman & Hall has published a new book Handbook of Markov Chain Monte Carlo, edited by Brooks, Gelman, Jones, and Ming. The Handbook is in some sense an update to MCMC in Practice reflecting 16 years of theoretical development and experience using MCMC.
After two decades of widespread use, MCMC remains magical, both in the sense of being powerful and in the sense of being mysterious. As Charles Geyer explains in the book’s opening chapter,
There is a great deal of theory about the convergence of Markov chains. Unfortunately, none of it can be applied to get useful convergence information for most MCMC applications.
Sometimes the magic can backfire in the form of pseudo-convergence. A Markov chain can appear to have converged when in fact it has not. Again Geyer explains.
People who have used MCMC in complicated problems can tell stories about samples that appeared to be converging until, after weeks of running, they discovered a new part of the state space and the distribution changed radically.
Despite the element of mystery in MCMC, no practical computational alternative exists for many problems. Fortunately, MCMC often works better in practice than we have reason to expect by theory.
Because there is no definitive theory to settle some of the practical issues in applying MCMC, there is much room for differing preferences. The Handbook has 42 contributors and four editors; differing approaches are inevitable. For example, Andrew Gelman and Kenneth Shirley advocate monitoring convergence using multiple Markov chains in one chapter while Charles Geyer argues against this approach in another chapter.
The Handbook consists of 24 chapters, arranged into two parts, the first entitled “Foundations, Methodology, and Algorithms.” and the second “Applications and Case Studies.” The line between algorithms and applications is somewhat blurry and so some of the chapters in the first part lean more toward application than the title might lead one to expect. The first part also contains valuable historical information. This history helps identify which contemporary practices are based more on tradition than demonstrated advantage. The second part gives examples of MCMC algorithms applied to a wide variety of areas: ecology, medical imaging, social science, etc.
Handbook of Markov Chain Monte Carlo is a valuable resource for those new to MCMC as well as to experienced practitioners. It is not a handbook analogous to, say, the Boy Scout Handbook. The field of MCMC is not settled enough for such a handbook to be possible. Instead, it is a collection of valuable information regarding a powerful computational approach to evaluating complex statistical models.
John D. Cook is a research statistician at M. D. Anderson Cancer Center and blogs daily at The Endeavour.