The information contained in uniform distributions : Introductory video
Before watching the video read the questions below. As you watch the video try to answer them
Questions
- Explain the meaning of the term functional
- State Khitchine’s four axioms for the information
- Explain what we mean when we state that a function or functional is monotonically decreasing.
- Now explain why the information in a uniform distribution decreases monotonically with the size of the sample space.
- If the uniform probability distribution $p$ has a sample space with $m$ outcomes and the probability distribution $q$ has a sample space with $n$ outcomes. How many outcomes are there in the sample space for the joint probability distribution $p \otimes q$. N.B. $p$ and $q$ are independent.
- Explain why the information in a uniform probablity distribution must be a function of $\frac{1}{n}$ where $n$ is the number of
- Hence, explain why the information in a uniform distribution, $p$, is given by: $I(p) = -k \ln n$.
- Explain the meaning of the term functional
- State Khitchine’s four axioms for the information
- Explain what we mean when we state that a function or functional is monotonically decreasing.
- Now explain why the information in a uniform distribution decreases monotonically with the size of the sample space.
- If the uniform probability distribution $p$ has a sample space with $m$ outcomes and the probability distribution $q$ has a sample space with $n$ outcomes. How many outcomes are there in the sample space for the joint probability distribution $p \otimes q$. N.B. $p$ and $q$ are independent.
- Explain why the information in a uniform probablity distribution must be a function of $\frac{1}{n}$ where $n$ is the number of
- Hence, explain why the information in a uniform distribution, $p$, is given by: $I(p) = -k \ln n$.