There are numerous ideas of what it means for a probability distribution to be “heavy-tailed.” Most people would call the normal distribution thin-tailed and the Cauchy distribution heavy-tailed. But between these extremes, common usage of the terms is fuzzy and depends on context.
The book An Introduction to Heavy-Tailed and Subexponential Distributions by Foss, Korshunov, and Zachary presents numerous ideas of “heaviness” for probability distribution tails. It gives precise definitions to common terms such as heavy-tailed and long-tailed, as well as more esoteric terms such as subexponential or h-insensitive. For each idea of heaviness the book explores questions such whether such distribution classes are closed under convolution and how they relate to other classes.
Most mathematicians are more familiar with light-tailed distributions. We are introduced to these first. Heavy-tailed distribution come later and are often presented as pathologies. This is unfortunate since many phenomena are best modeled by heavy-tailed distributions. Since we build our intuition on light-tailed distributions, we can find the behavior of long-tailed distributions surprising.
One example of unexpected behavior is what the authors call the principle of a single big jump. Roughly speaking, if a sum of heavy-tailed distributions takes on a large value, most likely one of the terms is responsible alone for the large value. More precisely, as x goes to infinity,
Pr(X1 + X2 + … + Xn > x ) ~ Pr( max(X1, X2, … , Xn) > x )
You can find more on the single big jump principle here.
This book is small, self-contained monograph. There are some examples and applications, but the emphasis is on basic properties and classification.
John D. Cook is a research statistician at M. D. Anderson Cancer Center and blogs daily at The Endeavour.