Chapter content: Markov chains in discrete time
A diagram illustrating the topics you will need to understand for this chapter together with the connections between topics is given below. Written notes on these various topics can be found be clicking on the links in the right panel.
| Introductory notes on Markov chains and the Markov property. |
| topic: The Markov Property |
| author: J. F. McCann |
| Some introductory notes on random walks. |
| topic: Gamblers ruin |
| author: J. F. McCann |
| Some notes on how the expected number of games is worked out in the gamblers ruin problem. |
| topic: Gamblers ruin |
| author: J. F. McCann |
| Some notes that introduce the limiting behavior of Markov chains and the definitions of transient and recurrent states. |
| topic: Transient and recurrent states in Markov chains |
| author: J. F. McCann |
| Notes on the ergodic theorem and the stationary distribution of Markov chains in general |
| topic: Ergodic Markov chains |
| author: J. F. McCann |
| More notes on the ergodic distribution and the stationary distribution of Markov chains. |
| topic: Ergodic Markov chains |
| author: J. F. McCann |