Markov Chains in Continuous time : Introductory video
Before watching the video read the questions below. As you watch the video try to answer them
Questions
- BEFORE WATCHING THE VIDEO: Write out the definition of the derivative as a limit
- BEFORE WATCHING THE VIDEO: Write out the McLaurin expansion for the exponential function
- Explain in your own words how continuous time Markov chains differ from the Markov chains that we have been studying thus far in this course.
- How do we take a limit of a matrix of functions and how then do we differentiate a matrix of functions
- Explain why the following matrix of limits cannot be calculated using the method you have just described $\lim_{\delta t \rightarrow 0} \frac{P(t+\delta t) - P(t)}{\delta t}$
- How is the Jump rate matrix for a continuous time Markov chain, $\mathbf{Q}$, defined?
- Explain how the exponential of a matrix is calculated. Hint: You do not take the exponential of each of the elements in turn.
- Show by substitution that $\mathbf{P}(t) = \exp(\mathbf{Q}t)$ is a solution of the Kolmogorov equation.
- How do you determine whether or not a continuous time Markov chain has a limiting stationary distribution and how do you determine the elements of this distribution
- BEFORE WATCHING THE VIDEO: Write out the definition of the derivative as a limit
- BEFORE WATCHING THE VIDEO: Write out the McLaurin expansion for the exponential function
- Explain in your own words how continuous time Markov chains differ from the Markov chains that we have been studying thus far in this course.
- How do we take a limit of a matrix of functions and how then do we differentiate a matrix of functions
- Explain why the following matrix of limits cannot be calculated using the method you have just described $\lim_{\delta t \rightarrow 0} \frac{P(t+\delta t) - P(t)}{\delta t}$
- How is the Jump rate matrix for a continuous time Markov chain, $\mathbf{Q}$, defined?
- Explain how the exponential of a matrix is calculated. Hint: You do not take the exponential of each of the elements in turn.
- Show by substitution that $\mathbf{P}(t) = \exp(\mathbf{Q}t)$ is a solution of the Kolmogorov equation.
- How do you determine whether or not a continuous time Markov chain has a limiting stationary distribution and how do you determine the elements of this distribution