Overview: Markov chains in discrete time
- You should be able to state what it means when we say that a time series of random variables has the Markov property
- You should be able to interpret transition graphs and transition probability matrices and you should also be able to use the Chapman-Kolmogorov relation to calculate the $n$-step transition probability matrix from the 1-step transition probability matrix.
- You should be able to discuss the limiting behavior of Markov chains. This means you should be able to calculate limiting stationary distributions, hitting times and hitting probabilities.
- You should be able to write a computer program to simulate a random walk in one dimension.