Understanding the content: Markov chains in discrete time
The panel on the right of this page contains links to videos on the topics we are studying as part of this chapter. On the pages for each of these videos you will find a link to a worksheet containing comprehension questions. You should print out the comprehension questions, watch the video and try to answer the comprehension questions based on the ideas that are explained in the video. If you are struggling to answer any question please come and ask.
| Introduction to the Markov property |
| topic: The Markov Property |
| author: G. Tribello |
| An explanation of how the Chapman-Kolmogorov relation is derived |
| topic: The Chapman Kolmogorov Relation |
| author: G. Tribello |
| A video explaining the gamblers ruin problem and the derivation of the difference equations. |
| topic: Gamblers ruin |
| author: G. Tribello |
| An explanation of the limiting behavior of Markov chains and the meanings of the terms transient and recurrent. |
| topic: Transient and recurrent states in Markov chains |
| author: G. Tribello |
| A video explaining the derivation of the ergodic theorem |
| topic: Ergodic Markov chains |
| author: G. Tribello |
| A video explaining why the limiting stationary distribution of an ergodic markov chain can be found by finding the principle left eigenvector of the transition matrix. |
| topic: Ergodic Markov chains |
| author: G. Tribello |
| A video explaining how the hitting times and hitting probabilities are calculated |
| topic: Hitting time and hitting probability |
| author: G. Tribello |