The information contained in non-uniform distributions : Introductory video
Before watching the video read the questions below. As you watch the video try to answer them
Questions
- Complete the following sentence: To find the information content of a non-uniform distribution, $p$, we generate an extended sample space in which ...
- Explain in your own words how conditional probabilities are used in constructing this extended sample space.
- Explain in your own words the steps in the derivation presented in the video that were taken in going from $I(p \otimes q) = I(p) + I(q)$ to $I(p \otimes q) = I(p) + \sum_i p_i I(w_i)$
- Explain what the symbol $w_i$ represents in the formula in the previous question.
- State the formula for the information contained in a non-uniform probability distribution, $p$ and explain the final steps involved in going from the formula $I(p \otimes q) = I(p) + \sum_i p_i I(w_i)$ to this result.
- A three sider spinner has edges labeled 1, 2 and 3 and an equal probability of landing on each of its three sides. Calculate the information contained in the associated probability distribution.
- Now suppose that the spinner has a probabilities of $\frac{1}{7}$, $\frac{2}{7}$ and $\frac{4}{7}$ of coming up with a 1, 2 or a 3 respectively. Calculate the information contained in the associated probability distribution.
- Now suppose that 1 never comes up and that the probabilities of 2 and 3 are equal to $\frac{4}{7}$ and $\frac{3}{7}$ how much information is contained in this distribution.
- Complete the following sentence: To find the information content of a non-uniform distribution, $p$, we generate an extended sample space in which ...
- Explain in your own words how conditional probabilities are used in constructing this extended sample space.
- Explain in your own words the steps in the derivation presented in the video that were taken in going from $I(p \otimes q) = I(p) + I(q)$ to $I(p \otimes q) = I(p) + \sum_i p_i I(w_i)$
- Explain what the symbol $w_i$ represents in the formula in the previous question.
- State the formula for the information contained in a non-uniform probability distribution, $p$ and explain the final steps involved in going from the formula $I(p \otimes q) = I(p) + \sum_i p_i I(w_i)$ to this result.
- A three sider spinner has edges labeled 1, 2 and 3 and an equal probability of landing on each of its three sides. Calculate the information contained in the associated probability distribution.
- Now suppose that the spinner has a probabilities of $\frac{1}{7}$, $\frac{2}{7}$ and $\frac{4}{7}$ of coming up with a 1, 2 or a 3 respectively. Calculate the information contained in the associated probability distribution.
- Now suppose that 1 never comes up and that the probabilities of 2 and 3 are equal to $\frac{4}{7}$ and $\frac{3}{7}$ how much information is contained in this distribution.