Norm Minimization Examples 1

# Norm Minimization Examples 1

Recall from the Norm Minimization page that if $V$ is an inner product space and $U$ is a finite-dimensional vector space of $V$ where $V = U \oplus U^{\perp}$, and if we let $v \in V$ then for every vector $u \in U$ we have that:

(1)
\begin{align} \quad \| v - P_U(v) \| ≤ \| v - u \| \end{align}

Furthermore, evaluating holds if and only if we take $u = P_U(v)$.

We will now identify how to use the theorem above in order to solve various minimization problems.

Step 1 We must first identify a vector space $V$ and define an inner product space on $V$. We will want to also identity the subspace $U$ of $V$ for which our solution lies in. We must note that $U$ must be a finite-dimensional vector space. We must then find a basis of $U$. Using the Gram-Schmidt procedure, we can then convert this basis to an orthonormal basis, $\{ e_1, e_2, ..., e_n \}$ of $U$ with respect to the inner product defined on $V$. In applying Theorem 1 from above, we will have that the normed difference, $\| v - P_U(v) \|$ will be less than or equal to the normed difference of $\| v - u \|$ for every vector $v \in V$, that is, $\| v - P_U(v) \| ≤ \| v - u \|$. We choose $u = P_U(v)$ to minimize the difference between every vector in $V$. The vector $P_U(v) = e_1 + e_2 + ... + e_n$ (which is in $U$ since it is a linear combination of the basis vectors of $U$) is our solution.

Let's look at an example.

## Example 1

Find the polynomial $p(x) \in \wp_3 (\mathbb{R})$ that minimizes $\int_0^1 \mid 2 + 3x - p(x) \mid^2 \: dx$ such that $p(x) \in \wp_3 ( \mathbb{R})$ and $p(0) = 0$ and $p'(0) = 0$.

We first identity that the larger vector space we're looking at is $V = \wp_3 ( \mathbb{R} )$. Let $q(x) = 2 + 3x$. We define an inner product of $V$ by:

(2)
\begin{align} \quad <p(x), q(x)> = \int_0^1 p(x) q(x) \: dx \end{align}

We're looking for a polynomial in $\wp_3 (\mathbb{R})$ such that $p(0) = 0$ and $p'(0) = 0$. The set of polynomials of degree less than or equal to $3$ that satisfy these conditions are a subspace $U$ of $V$ defined by:

(3)
\begin{align} \quad U = \{ p(x) \in \wp_3 (\mathbb{R}) : p(0) = 0, p'(0) = 0 \end{align}

We will now find a basis of $U$. Let $p(x) = a_0 + a_1x + a_2x^2 + a_3x^3$. Note that $p(0) = a_0$, but we must have that $p(0) = 0$. Therefore $a_0 = 0$. Differentiate $p$ to get that $p'(x) = a_1 + 2a_2x^2 + 3a_3x^2$. Note that $p'(0) = a_1$, but we must have that $p'(0) = 0$, which implies that $a_1 = 0$.

Therefore, $p(x) = x_2a^2 + x_3a^3$, and so:

(4)
\begin{align} \quad U = \{ a_2x^2 + a_3x^3 : a_2, a_3 \in \mathbb{R} \} \end{align}

We can clearly see that $U = \mathrm{span} (x^2, x^3)$, so $\mathrm{dim} (U) = 2$. Furthermore, $\{ x^2, x^3 \}$ are linearly independent in $V$, so take $\{ x^2, x^3 \}$ to be our start up basis.

We must now use the Gram-Scmidt procedure in order to convert this basis to an orthonormal basis. Let $v_1 = x^2$ and $v_2 = x^3$. Let $\{ e_1, e_2 \}$ be our orthonormal basis. Then we have that:

(5)
\begin{align} \quad e_1 = \frac{v_1}{\| v_1 \|} = \frac{x^2}{\| x^2 \|} = \frac{x^2}{\sqrt{<x^2, x^2>}} = x^2 \frac{1}{\sqrt{\int_0^1 x^4 \: dx}} = x^2 \frac{1}{\sqrt{\frac{1}{5}}} = \sqrt{5}x^2 \end{align}
(6)
\begin{align} \quad e_2 = \frac{v_2 - <v_2, e_1>e_1}{\| v_2 - <v_2, e_1>e_1 \|} = \frac{x^3 - <x_3, \sqrt{5}x^2>\sqrt{5}x^2}{\| x^3 - <x_3, \sqrt{5}x^2>\sqrt{5}x^2 \|} = \left ( x^3 - \left ( \sqrt{5} \int_0^1 x^5 \: dx \right ) \sqrt{5}x^2 \right ) \frac{1}{\biggr \| \left ( x^3 - \left ( \sqrt{5} \int_0^1 x^5 \: dx \right ) \sqrt{5}x^2 \right ) \biggr \|} \\ \quad = \left (x^3 - \frac{5}{6} x^2 \right ) \frac{1}{\| x^3 - \frac{5}{6} x^2 \|} = \left (x^3 - \frac{5}{6}x^2 \right ) \frac{1}{\sqrt{\int_0^1 (x^3 - \frac{5}{6} x^2)(x^3 - \frac{5}{6} x^2) \: dx}} = \left (x^3 - \frac{5}{6} x^2 \right ) \frac{1}{\sqrt{\int_0^1 (x^6 - \frac{10}{6}x^5 + \frac{25}{36}x^4 ) \: dx }} \\ \quad = \left (x^3 - \frac{5}{6}x^2 \right ) \frac{1}{\sqrt{\frac{1}{7} - \frac{10}{36} + \frac{5}{36}}} = \left (x^3 - \frac{5}{6} x^2 \right ) \frac{1}{\sqrt{1/252}} = \sqrt{7} (6x^3 - 5x^2) \end{align}

Therefore $\{ \sqrt{5}x^2, \sqrt{7} (6x^3 - 5x^2) \}$ is an orthonormal basis of $U$.

We now let $p(x) = P_U(q(x)) = P_U(2 + 3x)$, which is given by the formula:

(7)
\begin{align} \quad p(x) = P_U(q(x)) = <q(x), \sqrt{5} x^2> \sqrt{5}x^2 + <q(x), \sqrt{7} (6x^3 - 5x^2)>\sqrt{7} (6x^3 - 5x^2) \\ \quad = <2 + 3x, \sqrt{5} x^2> \sqrt{5}x^2 + <2 + 3x, \sqrt{7} (6x^3 - 5x^2)>\sqrt{7} (6x^3 - 5x^2) = 24x^2 - \frac{203}{10}x^3 \end{align}