An Introduction to Markov Processes came to life from a series of lecture notes written by D. W. Stroock during his teaching of stochastic processes at MIT. The book is technical but very suitable for graduate students in the sciences.
To prepare the reader better Stroock gives an overview of measure theory and probability theory in chapter 6. This “appendix” titled “Mild Measure Theory” gives the main theorems and results, from the concept of a σ-algebra to conditional probabilities and expectations.
Usually, books on stochastic processes start with some preliminary concepts of probability theory and trivial examples so that the reader can develop intuitive appreciation for the subject. Stroock starts with a chapter titled “Random Walks — A Good Place to Begin”, and indeed it is. The first chapter uses random walks to develop (or remind the reader) some basic concepts such as recurrence and transience. The thrill of the first chapter, at least it should be for students, is the presentation of some basic calculations of random walks on Zd. This enables a careful reader to really connect and remind oneself of the basic structure of Markov processes.
The heart of the book is the theory of ergodic properties of Markov processes, which is present in several chapters. The first part of Chapter 2 discusses basic results of discrete Markov chains where the second part gives results of the Doeblin theorem and discusses elementary ergodic theory of Markov chains. Stroock continues to discuss ergodicity in chapter 3. Chapter 4 goes on to discuss Markov processes in continuous time, starting with well known Poisson processes. Reversible Markov processes are explained in chapter 5.
This is not the cookbook of Markov processes that students usually desire, but rather a book that presents the theory of Markov processes in a very clear and informative way. It is designed perfectly for students as it starts very lightly and also gives a whole chapter on measure theory. It gives enough time to adjust to the subject. Proofs given in the book are detailed and easy to follow. The author does indeed make an effort to explain the material effectively and slowly; the information is not packed but rather it flows through the pages. At the beginning of each chapter/sub-chapter he develops an intuition of why he is explaining this material by discussing the importance of certain definitions and gives leads as to where we are going with certain definitions and theorems. Furthermore, at the end of each chapter there are numerous exercises, which should enable students to really get a grasp on the subject. For students who already have had some prior experience in stochastic processes in their undergraduate studies this book will bring new light of understanding.
Ita Cirovic Donev is a PhD candidate at the University of Zagreb. She hold a Masters degree in statistics from Rice University. Her main research areas are in mathematical finance more precisely statistical mehods of credit and market risk. Apart from the academic work she works closely in consulting for financial institutions.