Lagrange multipliers : Exercises

Introduction

The method of Lagrange multipliers is used to find the optimum of a function $f(x,y)$ subject to the constraint that $g(x,y)=c$. The exercises in the first section that follows allow you to practise using this method to find constrained optimums. To solve the problems in the second section you will need to set up the problem so that you can apply Lagrange's method.

Example problems

Click on the problems to reveal the solution

Problem 1

Find the optimum of $f(x,y)=x^2+y^2$ subject to the constraint $g(x,y)=2x+y=10$

$$ x=4 \qquad y=2 \qquad f(x,y) = 20 $$

Find the optimum of $f(x,y)=x^2+y^2$ subject to the constraint $g(x,y)=x+2y=5$

$$ x=1 \qquad y=2 \qquad f(x,y) = 5 $$

Find the largest value $f(x,y) = 81x^2 + y^2$ can take subject to the constraint $g(x,y)=4x^2 +y^2=9$

The extended function here is: $$ F(x,y,\lambda) = 81x^2 + y^2 + \lambda( 4x^2 + y^2 - 9 ) $$ When this is differentiated with respect to $y$ you get: $$ \frac{\partial F}{\partial y} = 2y + 2y\lambda = 0 \qquad \rightarrow \qquad 2y(1 + \lambda) = 0 \qquad \therefore y=0 \quad \textrm{or} \quad \lambda=-1 $$ You have to insert both these solutions into the other equations you get by calculating $\frac{\partial F}{\partial x}$ and $\frac{\partial F}{\partial \lambda}$. Through this process you will get 4 local optima. You will find that the largest values are at: $$ x=0 \qquad y=\pm 3 \qquad f(x,y) = 9 $$

Find the minimum value that $f(x,y)=8x^2 - 2y$ can take subject to the constraint $g(x,y)=x^2+y^2 = 1$.

$$ x=0 \qquad y= 1 \qquad f(x,y) = -2 $$

Find the smallest value $f(x,y,z) = y^2 - 10z$ can take subject to the constraint $x^2 + y^2 + z^2 = 36$. Hint to do this you need to remember $\lambda \ne 0$.

$$ x=0 \qquad y=0 \qquad z=6 \qquad f(x,y,z) = -60 $$

Find the largest value $f(x,y,z)=3x^2 + y$ can take subject to the constraints $4x - 3y = 9$ and $x^2 + z^2 = 9$

$$ x=3 \qquad y=1 \qquad z=0 \qquad f(x,y,z) = 28 $$

Problem 2

Consider a three-sided spinner, which has edges labelled $0$, $1$ and $2$. Calculate the most probable set of probabilities $\{p_j\}$, for getting 0, 1 and 2 given that the average value of $j$ is $\langle j \rangle =10/7$. N.B. The most probable set of probabilities will have the minimum information subject to the constraints implied in the question.

The information is given by: \[ I(p) = \sum_{j=0}^2 p_j \log(p_j) \] We want to minimise this subject to the constraints $p_0+p_1+p_2=1$ (normalisation) and $p_1+2p_2=\frac{10}{7}$ (average value). The extended function to minimise is thus: \[ I'(\{p_j\}) = \sum_{j=0}^2 p_j \log(p_j) + \lambda_1 \left( \sum_{j=0}^2 j p_j - \frac{10}{7} \right) + \lambda_2 \left( \sum_{j=0}^2 p_j -1 \right) \] At the minimum of this function we have: \[ \begin{aligned} \frac{\partial I'}{\partial p_j} & = \log(p_j) + 1 + j \lambda_1 + \lambda_2 = 0 \qquad \rightarrow \qquad p_j = e^{-(1+\lambda_2) - j\lambda_1} \qquad \rightarrow \qquad p_j = \frac{x^j }{Q} \\ \frac{\partial I'}{\partial \lambda_1} & = \sum_{j=0}^2 jp_j - \frac{10}{7} =0 \\ \frac{\partial I'}{\partial \lambda_2} & = \sum_{j=0}^2 p_j -1 = 0 \end{aligned} \] Notice we have defined $Q=e^{1+\lambda_2}$ in the above and $x=e^{-\lambda_1}$. These constraints tell us that: \[ \begin{aligned} 1&=p_0+p_1+p_2 = \frac{1+ x+x^2}{Q} \qquad \rightarrow \qquad 1+x+x^2 = Q \\ \frac{10}{7} & = p_1 + 2p_2 = \frac{x+2x^2}{Q} \qquad \rightarrow \qquad x+2x^2 = \frac{10}{7} Q \end{aligned} \] Equating these two expressions we get: \[ \frac{10}{7} (1 + x + x^2 ) = x + 2x^2 \qquad \rightarrow \qquad 4x^2 - 3x - 10 = 0 \qquad \rightarrow \qquad x=2 \] and hence $Q=1+x+x^2 = 1 + 2 + 4 = 7$. We thus have the following probability distribution: \[ p_0 = \frac{1}{7} \qquad p_1 = \frac{2}{7} \qquad p_2 = \frac{4}{7} \]

Calculate the most probable set of probabilities $\{p_j\}$, given that the average value of $j$ is $\langle j \rangle$ and that the face painted 0 never occurs. (Hint: notice that we have three constraints here and that there are three unknowns).

Three equations and three unknowns means that we can solve this problem using simultaneous equations. We don't need to do Lagrange multipliers again. These three simultaneous equations are: \[ \begin{aligned} p_0 & = 0 \\ p_0 + p_1 + p_2 &= 1 \qquad \rightarrow \qquad p_1 + p_2 = 1 \\ p_1 + 2p_2 & = \frac{10}{7} \end{aligned} \] From this we can deduce that: \[ p_1 = 1 - p_2 \qquad \rightarrow \qquad 1-p_2 + 2p_2 = \frac{10}{7} \qquad \rightarrow \qquad p_2 = \frac{3}{7} \qquad p_1 = \frac{4}{7} \]

Now suppose that all three sides are equally probable. Is the information in this distribution greater than or less than the information in the distribution you calculated in the first part of this problem?

The information is lower as the uniform distribution has the minimum amount of information.

Lagrange multipliers : Exercises

Introduction

Example problems

Problem 1

Problem 2

Contact Details