Overview: Markov chains in continuous time

The final part of the course takes what you have learnt about discrete time Markov chains and extends it so that it also works in continuous time. Doing so ensures that we can model processes where there are discrete states and where the times between transitions between these states are random. We can thus use the theory of Markov chains to model processes such as queues and the progression of diseases. A discussion of Markov chains in continous time also allows us to move beyond the Markovian assumption and to thus introduce more complex time-dependent random processes.


  • You should be able to write out and solve the Kolmogorov forward equation.
  • You should be able to derive the equations of the Poisson process and you should also be able to write programs to simulate poisson processes.
  • You should be able to work with Markov models that describe queues. This includes being able to write programs to simulate queues.
  • You should be able how models of random processes that relax the assumption of Markovianity can be developed.

Contact Details

School of Mathematics and Physics,
Queen's University Belfast,

Email: g.tribello@qub.ac.uk
Website: mywebsite