Bayesians Versus Frequentists

This book argues for the superiority of bayesianism (the author prefers lower-case) over frequentism, collecting nearly two dozen articles organized into a few sections with an index of persons. The sections include the origins of numerical and statistical thinking, related philosophical questions, and a contrasting of bayesian and frequentist approaches. The initial, historical section takes a broad view, encompassing the cognitive abilities of animals, numerosity (“the ability to appreciate and understand numbers”), and the formation of a foundation for statistics. The middle section presents a brisk overview of statistics from the invention of dice in India to nineteenth century formalizations by Peirce, Venn, de Morgan, etc. These two sections prepare the ground for a final culmination in the work of Bayes and the role his approach has had and, the author argues, should have. These sections are good standalone treatments in aspects of the history of mathematics.

The Rev. Bayes is properly introduced, for this is a story of a science, as a figure that plays the role of a hero and even savior to a dedicated following. Like any good story, there is dramatic tension. Here that is provided by the opposing team of frequentists. Pearson is the MVP on this team, with Fisher a curmudgeonly captain. Somewhat facetious characterizations are not out of place here, since the author includes some witty characterizations: “I will take a metaphor from biological tradition: there is not a clear winner, just the most adapted and fittest to the challenges of contemporary science, and it is … Bayesianism.”

Contemporary science is the focus of the final section, from data mining to epidemiology. Epidemiology is key to the survey here, as “frequentist tools were useful at the beginning of epidemiology” and the research area became a stronghold of frequentists. Being fit in an evolutionary sense implies a mingling of genetic material and surviving offspring; the final assessment of the contrasting schools and techniques finds applicability and validity of ideas placed along a three-axis spectrum (bayesian, frequentist, and “Fisherian”) summed up elegantly by Efron and Bradley ("R. A. Fisher in the 21st Century."Statistical Science 13.2 (1998): 95-114).

Of value to philosophers, historians of science, statisticians, and students in related fields, this study from a confirmed Bayesianist requires no deep statistical background. The author explains and introduces the key basic concepts as they would be introduced in a first-year college course, with emphasis on the philosophy over the mechanics:

We can see, then, that our posterior belief P(A|B) is calculated by multiplying our prior belief P(A) by the likelihood P(B|A) that B will occur if A is true… The prior probability of A represents our best estimate of the probability of the fact we are considering prior to attending to a new piece of evidence. Therefore, in the Bayesian paradigm, current knowledge about model parameters is expressed by placing a probability distribution on the parameters, called the “prior distribution.” When new data become available, the information they contain regarding the model parameters is expressed in the “likelihood,” which is proportional to the distribution of the observed data given the model parameters. This information is then combined with the prior to produce an updated probability distribution called the “posterior distribution,” … (pg. 44)

Later, the author places the central idea of frequentism in this “nutshell”:

…to obtain a large number of trials from which emerged the relative frequency of an event. Now, you can understand the main difference… no presence in frequentism of personal prior beliefs in the process. (pg. 50)

Anyone interested in the history of statistics, let alone the subject debate, and in how our choice of statistical foundation affects our view of the world will find this work entertaining in enlightening.

Tom Schulte shares with mathematics students of Oakland Community College is confirmed views on active versus passive learning.