Markov chain Monte Carlo (MCMC) is a family of algorithms used to produce approximate random samples from a probability distribution too difficult to sample directly. The method produces a Markov chain that whose equilibrium distribution matches that of the desired probability distribution. Since Markov chains are typically not independent, the theory of MCMC is more complex than that of simple Monte Carlo sampling.
MCMC was invented in 1953 but was not widely applied until statisticians became aware of it around 1990. Then problems that had previously been intractable suddenly became tractable. MCMC worked like magic. (We will return to the comparison with magic below.)
One of the most popular books on MCMC to date has been Markov Chain Monte Carlo in Practice. This book was edited by Gilks, Richardson, and Spiegelhalter and published by Chapman & Hall in 1995, just five years after the use of MCMC became wide-spread among statisticians. Now Chapman & Hall has published a new book Handbook of Markov Chain Monte Carlo, edited by Brooks, Gelman, Jones, and Ming. The Handbook is in some sense an update to MCMC in Practice reflecting 16 years of theoretical development and experience using MCMC.
After two decades of widespread use, MCMC remains magical, both in the sense of being powerful and in the sense of being mysterious. As Charles Geyer explains in the book’s opening chapter,
There is a great deal of theory about the convergence of Markov chains. Unfortunately, none of it can be applied to get useful convergence information for most MCMC applications.
Sometimes the magic can backfire in the form of pseudo-convergence. A Markov chain can appear to have converged when in fact it has not. Again Geyer explains.
People who have used MCMC in complicated problems can tell stories about samples that appeared to be converging until, after weeks of running, they discovered a new part of the state space and the distribution changed radically.
Despite the element of mystery in MCMC, no practical computational alternative exists for many problems. Fortunately, MCMC often works better in practice than we have reason to expect by theory.
Because there is no definitive theory to settle some of the practical issues in applying MCMC, there is much room for differing preferences. The Handbook has 42 contributors and four editors; differing approaches are inevitable. For example, Andrew Gelman and Kenneth Shirley advocate monitoring convergence using multiple Markov chains in one chapter while Charles Geyer argues against this approach in another chapter.
The Handbook consists of 24 chapters, arranged into two parts, the first entitled “Foundations, Methodology, and Algorithms.” and the second “Applications and Case Studies.” The line between algorithms and applications is somewhat blurry and so some of the chapters in the first part lean more toward application than the title might lead one to expect. The first part also contains valuable historical information. This history helps identify which contemporary practices are based more on tradition than demonstrated advantage. The second part gives examples of MCMC algorithms applied to a wide variety of areas: ecology, medical imaging, social science, etc.
Handbook of Markov Chain Monte Carlo is a valuable resource for those new to MCMC as well as to experienced practitioners. It is not a handbook analogous to, say, the Boy Scout Handbook. The field of MCMC is not settled enough for such a handbook to be possible. Instead, it is a collection of valuable information regarding a powerful computational approach to evaluating complex statistical models.
John D. Cook is a research statistician at M. D. Anderson Cancer Center and blogs daily at The Endeavour.
Foreword Stephen P. Brooks, Andrew Gelman, Galin LJones and Xiao-Li Meng
Introduction to MCMC, Charles J. Geyer
A short history of Markov chain Monte Carlo: Subjective recollections from in-complete data, Christian Robert and George Casella
Reversible jump Markov chain Monte Carlo, Yanan Fan and Scott A. Sisson
Optimal proposal distributions and adaptive MCMC, Jeffrey S. Rosenthal
MCMC using Hamiltonian dynamics, Radford M. Neal
Inference and Monitoring Convergence, Andrew Gelman and Kenneth Shirley
Implementing MCMC: Estimating with confidence, James M. Flegal and Galin L. Jones
Perfection within reach: Exact MCMC sampling, Radu V. Craiu and Xiao-Li Meng
Spatial point processes, Mark Huber
The data augmentation algorithm: Theory and methodology, James P. Hobert
Importance sampling, simulated tempering and umbrella sampling, Charles J.Geyer
Likelihood-free Markov chain Monte Carlo, Scott A. Sisson and Yanan Fan
MCMC in the analysis of genetic data on related individuals, Elizabeth Thompson
A Markov chain Monte Carlo based analysis of a multilevel model for functional MRI data, Brian Caffo, DuBois Bowman, Lynn Eberly and Susan Spear Bassett
Partially collapsed Gibbs sampling & path-adaptive Metropolis-Hastings in high-energy astrophysics, David van Dyk and Taeyoung Park
Posterior exploration for computationally intensive forward models, Dave Higdon, C. Shane Reese, J. David Moulton, Jasper A. Vrugt and Colin Fox
Statistical ecology, Ruth King
Gaussian random field models for spatial data, Murali Haran
Modeling preference changes via a hidden Markov item response theory model, Jong Hee Park
Parallel Bayesian MCMC imputation for multiple distributed lag models: A case study in environmental epidemiology, Brian Caffo, Roger Peng, Francesca Dominici, Thomas A. Louis and Scott Zeger
MCMC for state space models, Paul Fearnhead
MCMC in educational research, Roy Levy, Robert J. Mislevy and John T. Behrens
Applications of MCMC in fisheries science, Russell B. Millar
Model comparison and simulation for hierarchical models: analyzing rural-urban migration in Thailand, Filiz Garip and Bruce Western