The Best Approximation Theorem for Hilbert Spaces

The Best Approximation Theorem for Hilbert Spaces

Theorem 1 (The Best Approximation Theorem for Hilbert Spaces): Let $H$ be a Hilbert space and let $K \subset H$ be a nonempty, closed, and convex subset of $H$. If $h_0 \in H \setminus K$ then there exists a unique $k_* \in K$ such that $\displaystyle{\| k_* - h_0 \| = \inf_{k \in K} \| k - h_0 \|}$.
Screen%20Shot%202017-04-03%20at%207.26.38%20PM.png

Recall that $\inf_{k \in K} \| k - h_0 \| = \mathrm{dist} (h_0, K)$. The theorem above tells us that if $K$ is a nonempty closed and convex subset of a Hilbert space $H$ then for every $h_0 \in H \setminus K$ we can find a unique $k_* \in K$ for which the distance from $h_0$ to $k_*$ is the distance from $h_0$ to the set $K$.

  • Proof: Let $(k_n)_{n=1}^{\infty}$ be a sequence of points in $K$ such that:
(1)
\begin{align} \quad \lim_{n \to \infty} \| k_n - h_0 \| = \inf_{k \in K} \| k - h_0 \| \end{align}
  • Since $K$ is a convex subset of $H$, for each $m, n \in \mathbb{N}$ we have that:
(2)
\begin{align} \quad \frac{1}{2}k_m + \frac{1}{2}k_n \in K \end{align}
  • Therefore:
(3)
\begin{align} \quad \biggr \| \left ( \frac{1}{2}k_m + \frac{1}{2}k_n \right ) - h_0 \biggr \| \geq \inf_{k \in K} \| k - h_0 \| \quad (*) \end{align}
(4)
\begin{align} \quad \| x + y \|^2 + \| x - y \|^2 &= 2 \| x \|^2 + 2 \| y \|^2 \\ \quad \biggr \| \left ( \frac{1}{2}(k_m - h_0) \right ) + \left ( \frac{1}{2}(k_n - h_0) \right ) \biggr \|^2 + \biggr \| \left ( \frac{1}{2}(k_m - h_0) \right ) - \left ( \frac{1}{2}(k_n - h_0) \right ) \biggr \|^2 &= 2 \biggr \| \frac{1}{2}(k_m - h_0) \biggr \|^2 + 2 \biggr \| \frac{1}{2}(k_n - h_0) \biggr \|^2 \end{align}
  • Simplifying the equality above gives us:
(5)
\begin{align} \biggr \| \frac{1}{2}k_m + \frac{1}{2}k_n - h_0 \biggr \|^2 + \biggr \| \frac{1}{2}k_m - \frac{1}{2}k_n \biggr \|^2 &= 2 \biggr \| \frac{1}{2} (k_m - h_0) \biggr \|^2 + 2 \biggr \| \frac{1}{2}(k_n - h_0) \biggr \|^2 \\ \biggr \| \frac{1}{2}k_m + \frac{1}{2}k_n - h_0 \biggr \|^2 + \frac{1}{4} \| k_m - k_n \|^2 &= \frac{1}{2} \| k_m - h_0 \|^2 + \frac{1}{2} \| k_n - h_0 \|^2 \end{align}
  • From $(*)$ we get that:
(6)
\begin{align} \quad \frac{1}{4} \| k_m - k_n \| &\leq \frac{1}{2} \| k_m - h_0 \|^2 + \frac{1}{2} \| k_n - h_0 \|^2 - \inf_{k \in K} \| k - h_0 \|^2 \end{align}
  • And taking the limit as $m, n \to \infty$ gives us that:
(7)
\begin{align} \quad \lim_{m, n \to \infty} \| k_m - k_n \|^2 \leq \lim_{m, n \to \infty} \left [ \frac{1}{2} \| k_m - h_0 \|^2 + \frac{1}{2} \| k_n - h_0 \|^2 - \inf_{k \in K} \| k - h_0 \|^2 \right ] = 0 \end{align}
  • Therefore $(k_n)_{n=1}^{\infty}$ is a Cauchy sequence. Since $H$ is a Hilbert space, $H$ is complete, and so the sequence $(k_n)_{n=1}^{\infty}$ converges to some $k_* \in H$, and since $K$ is closed, $k_* \in K$. Now since $\| \cdot \|$ is continuous, we have that:
(8)
\begin{align} \quad \| k_* - h_0 \| = \biggr \| \lim_{n \to \infty} k_n - h_0 \biggr \| = \lim_{n \to \infty} \| k_n - h_0 \| = \inf_{k \in K} \| k - h_0 \| \end{align}
  • Lastly, suppose that $k' \in K$ also satisfies the conditions of the theorem. By the convexity of $K$ we have that:
(9)
\begin{align} \quad \frac{1}{2}k_* + \frac{1}{2}k' \in K \end{align}
  • Therefore we have that:
(10)
\begin{align} \quad \biggr \| \frac{1}{2}k_* + \frac{1}{2}k' - h_0 \biggr \| \geq \inf_{k \in K} \| k - h_0 \| \quad (**) \end{align}
  • Set $x = \frac{1}{2}(k_* - h_0)$ and $y = \frac{1}{2}(k' - h_0)$. Then by the Parallelogram identity we have that:
(11)
\begin{align} \quad \biggr \| \frac{1}{2}k_* + \frac{1}{2}k' - h_0 \biggr \|^2 + \biggr \| \frac{1}{2}k_* - \frac{1}{2}k' \biggr \|^2 = 2 \biggr \| \frac{1}{2} (k_* - h_0) \biggr\|^2 + 2 \biggr \| \frac{1}{2} (k' - h_0) \biggr \|^2 \end{align}
  • And so from $(**)$:
(12)
\begin{align} \quad \frac{1}{4} \|k_* - k' \|^2 \leq \frac{1}{2} \| k_* - h_0 \|^2 + \frac{1}{2} \| k' - h_0 \|^2 - \inf_{k \in K} \| k - h_0 \|^2 = 0 \end{align}
  • Hence $\| k_* - k' \| = 0$, and so $k_* = k'$. $\blacksquare$

It is important to note that all of the hypotheses in the best approximation theorem are necessary. In particular, if $K$ is not convex then there may be many points in $K$ for which the distance between $h$ and those points equals the distance between $h$ and $K$ as illustrated below:

Screen%20Shot%202018-03-25%20at%2010.43.25%20PM.png
Unless otherwise stated, the content of this page is licensed under Creative Commons Attribution-ShareAlike 3.0 License