Markov chains in continuous time

Markov chains in continuous time

The central equation in the theory of continuous time Markov chains is the Kolmogorov equation: $$ \frac{\textrm{d} \mathbf{P}(t)}{\textrm{d}t} = \mathbf{P(t)}\mathbf{Q} $$ In this equation $\mathbf{Q}$ is the so called jump rate matrix, which is the following matrix of limits. $$ \lim_{\delta t \rightarrow 0} \frac{P(\delta t) - \mathbf{I}}{\delta t} $$ The first of these two equations has the differential of a matrix of probabilities on the left hand side. A continuous time Markov chain thus has a limiting stationary distribution $\pi$ if there is any non-zero vector that satisfies: $$ \pi \mathbf{Q} = 0 $$ This equation comes about by setting $\frac{\textrm{d} \mathbf{P}(t)}{\textrm{d}t}=0$ in the first equation.

Syllabus Aims