Transient and Recurrent state
: Exercises
Introduction
In the following exercises you will have practise in drawing transition graphs
for Markov chains from the transition matrix and in classifying states as transient
or recurrent. Remember that a state is transient if:
$$
P(R_i \le \infty ) \lt 1
$$
and that is recurrent if:
$$
P(R_i \le \infty ) = 1
$$
Here $R_i$ is the return time to state $i$, which is random as this quantity measures the number of steps in a loop that starts and
end in state $i$.
Example problems
Click on the problems to reveal the solution
Problem 1
| Problem |
State 1 |
State 2 |
State 3 |
State 4 |
| (i) |
Transient |
Absorbing (recurrent) |
Recurrent |
Recurrent |
| (ii) |
Recurrent |
Recurrent |
Recurrent |
|
| (iii) |
Transient |
Transient |
Transient |
Absorbing (recurrent) |
Problems for you to try
Draw a transition graph for a Markov chain that has three recurrent
states and two transient states. Draw a second transition graph. This time your Markov chain should have five states all
of which have period 3. Does this Markov chain have a limiting stationary distribution?
