You are here

Problems in Probability

Albert N. Shiryaev
Publication Date: 
Number of Pages: 
Problem Books in Mathematics
Problem Book
[Reviewed by
Darren Glass
, on

Albert Shiryaev published the first edition of his graduate-level textbook Probability in 1980 and it has been a mainstay of the Springer GTM series ever since. Over the following three decades he has continued to revise and expand this book, and while working on the third volume he chose to pull out all of the exercises from all of the volumes and publish them, along with some new exercises, in the book under review, aptly titled Problems in Probability. The book includes a wide variety of problems that Shiryaev has written himself and collected from other “textbooks, lecture notes, exercise manuals, monographs, research papers, private communications, and such.”

What remains is certainly not a textbook but is instead a collection of problems. Lots and lots of problems. They range from simple computational exercises to deep theoretical problems. Some of them are problems that a bright high school student could attack while others require several years of graduate-level analysis in order to understand what they are even asking. Yes, there are some problems involving balls and urns but there are also problems involving Riesz decompositions and measure theory. Some of the problems can be stated in a single sentence while others involving multiple paragraphs of notation and set up. To give a small sense of the type of problems included, I include the following sample (which is admittedly biased towards problems that are briefly stated:

  • 1.1.1 — Verify that \(A \cup A = A\). Verify that \(A \cup B = B \cup A\). Verify that \(A \cup (B \cup C) = (A \cup B) \cup C\).
  • 1.1.22 — A box contains N balls labelled 1 through N. A ball is sampled n times randomly and with repetition. Prove a formula for the probability that the largest label among the sampled balls is k.
  • 1.4.22 — Give an example of two independent random variables \(\zeta\) and \(\eta\) such that \(\zeta^2\) and \(\eta^2\) are dependent.
  • 1.12.5 — Let P and Q be stochastic matrices. Prove that PQ is also stochastic.
  • 2.1.10 — Give an example showing that a measure that can take the value \(\infty\) could be countably additive but still not continuous at 0.
  • 2.4.1 — Prove that the random variable \(\zeta\) has a continuous distribution of and only if \(P(\zeta = x) = 0\) for all real \(x\).
  • 2.6.79 — Consider a lottery with tickets numbered \(000000,\dots ,9999999\) and suppose one ticket is chosen at random. Use generating functions to find the probability that the sum of the six digits on this ticket equals 21.
  • 2.8.87 — Suppose that X is a random variable with one of the distributions: binomial, Poisson, geometric, negative-binomial, or Pareto. Find the probability of the event {X is even}.
  • 2.13.2 — Let \((\zeta,\eta_{1},\dots ,\eta_{k})\) be a Gaussian system. Describe the structure of the conditional expectations \(E(\zeta^n \mid \eta_{1},\dots ,\eta_{k})\) as functions of the variables \(\eta_{1},\dots ,\eta_{k}\).
  • 3.1.2 — Prove that in \(\mathbb{R}^m\) the class of elementary sets is a convergence defining class.
  • 3.6.12 — Give examples of random variables that are not infinitely divisible and yet their characteristic functions never vanish.
  • 3.8.12 — Let \( F(x,y) = \max(x+y-1,0)\) be the bivariate distribution function with \(0 \leq x,y \leq 1\). Prove that the associated marginal distribution functions give the uniform distribution on \([0,1]\).
  • 4.5.3 — Assuming that the random variable \(\zeta\) is non-degenerate, prove that the function H(a) is differentiable on the entire real line and also convex.
  • 5.3.12 — By providing examples, prove that the Poincaré reversibility theorem may not hold for measurable spaces with infinite measures.
  • 6.1.2 — Prove that if all zeroes of the polynomial Q(z) defined in [P, 6.1] happen to be outside of the unit disk then the auto-regresion equation admits unique stationary solution, which can be written in the form of one-sided moving average.
  • 6.3.8 — By considering sequences of the form \(\zeta_{n} = A \cos(\lambda n + \theta)\) prove that a stationary in broad sense sequence may have periodic sample paths and non-periodic covariance function.
  • 7.2.15 — Consider the stopping time \(\tau\) defined in [P, 7.2] and prove that \(E(\tau^p) < \infty\) for every \(p \geq 1\).
  • 7.4.16 — Give an example of a martingale \((X_{n})\) for which one has \(X_{n} \to -\infty\) with probability 1.
  • 8.3.3 — Give an example of a Markov chain that has a non-ergodic stationary distribution.
  • 8.4.6 — Explain whether it may be possible for all states of a given Markov chain to be inessential if the state space is finite.

As is clear from this sampling, the problems cover a wide range of topics under the heading of “Probability” and also are at a variety of levels of depth. As is also clear, many of the problems assume that you have Shiryaev’s other books close at hand, and in fact a good number of the problems are of the form “Prove Theorem X.Y of [P]” referring to his own book Probability Many of the problems have hints to help guide the reader to a solution and many others have remarks that give brief discussions expanding on the result of the problem.

Shiryaev does include several appendices that summarize a number of the most important concepts from combinatorics and probability theory. In particular, in less than 50 pages he introduces permutations and combinations, Bell numbers, distribution functions, analytic “tricks” involving Lebesgue integrals, Laplace transforms, generating functions, Stirling numbers, Appell relations, stationary random sequences, martingales, Markov chains, potential theory, Dirichlet problems, and a number of other topics. Needless to say, the coverage of these topics is neither in depth nor exhaustive, but will serve as a good reference for readers who have already seen the topics. Additionally, Shiryaev gives an extensive bibliography which serves both to give as much credit as possible to the original authors of the problems, and also will serve the reader as a good resource if they wish to learn more about the topics covered.

However one feels about the buzzwords related to “Inquiry Based Learning,” we are now at a point where most mathematicians agree that an essential part of learning a subject is to do problems in the area. Lots and lots of problems. Often it seems that graduate level texts are not designed in this way, however, and do not contain exercises for readers to attempt to solidify their understanding of the material. Problems in Probability is exactly the opposite, and is almost exclusively dedicated to exercises. Shiryaev’s book provides an excellent source of problems and will be a valuable resource to students who wish to learn probability at the graduate level.

Darren Glass is an Associate Professor of Mathematics at Gettysburg College whose mathematical interests include number theory, Galois theory, algebraic geometry, and cryptography. He can be reached at

Preface.- 1. Elementary Probability Theory.- 2. Mathematical Foundations of Probability Theory.- 3. Convergence of Probability Measures.- 4. Independent Random Variables.- 5. Stationary Random Sequences in Strict Sense.- 6. Stationary Random Sequences in Broad Sense.- 7. Martingales.- 8. Markov Chains.- Appendix.- References.