Transient and recurrent states in Markov chains
Transient and recurrent states in Markov chains
A state in a Markov chain is said to be transient if: $$ P(R_i \le \infty ) < 1 $$ and recurrent if $$ P(R_i \le \infty ) = 1 $$ Here $R_i$ is the return time to state $i$, which is random as this quantity measures the number of steps in a loop that starts and end in state $i$.
Syllabus Aims
- You should be able to write out and explain what makes states in a Markov chain recurrent and what makes them transient.
- You should be able to classify the states in a Markov chain as transient or recurrent.
- You should be able to describe the limiting behavior of a Markov chain by examining the transition graph.
Description and link | Module | Author | ||
| Some notes that introduce the limiting behavior of Markov chains and the definitions of transient and recurrent states. | SOR3012 | J. F. McCann |
Description and link | Module | Author | ||
| An explanation of the limiting behavior of Markov chains and the meanings of the terms transient and recurrent. | SOR3012 | G. Tribello |
Description and link | Module | Author | ||
| Problems on drawing transition graphs for Markov chains and on identifying transient and recurrent states. | SOR3012 | G. Tribello |
Contact Details
School of Mathematics and Physics,
Queen's University Belfast,
Belfast,
BT7 1NN
Email: g.tribello@qub.ac.uk
Website: mywebsite