The Best Approximation Theorem for Hilbert Spaces

Theorem 1 (The Best Approximation Theorem for Hilbert Spaces): Let $H$ be a Hilbert space and let $K \subset H$ be a nonempty, closed, and convex subset of $H$. If $h_0 \in H \setminus K$ then there exists a unique $k_* \in K$ such that $\displaystyle{\| k_* - h_0 \| = \inf_{k \in K} \| k - h_0 \|}$.

Recall that $\inf_{k \in K} \| k - h_0 \| = \mathrm{dist} (h_0, K)$. The theorem above tells us that if $K$ is a nonempty closed and convex subset of a Hilbert space $H$ then for every $h_0 \in H \setminus K$ we can find a unique $k_* \in K$ for which the distance from $h_0$ to $k_*$ is the distance from $h_0$ to the set $K$.

• Proof: Let $(k_n)_{n=1}^{\infty}$ be a sequence of points in $K$ such that:

(1)

\begin{align} \quad \lim_{n \to \infty} \| k_n - h_0 \| = \inf_{k \in K} \| k - h_0 \| \end{align}

• Since $K$ is a convex subset of $H$, for each $m, n \in \mathbb{N}$ we have that:

(2)

\begin{align} \quad \frac{1}{2}k_m + \frac{1}{2}k_n \in K \end{align}

• Therefore:

(3)

\begin{align} \quad \biggr \| \left ( \frac{1}{2}k_m + \frac{1}{2}k_n \right ) - h_0 \biggr \| \geq \inf_{k \in K} \| k - h_0 \| \quad (*) \end{align}

• Furthermore, by setting $\displaystyle{x = \frac{1}{2}(k_m - h_0)}$ and $\displaystyle{y = \frac{1}{2}(k_n - h_0)}$ we have by the The Parallelogram Identity for the Norm Induced by an Inner Product that:

(4)

\begin{align} \quad \| x + y \|^2 + \| x - y \|^2 &= 2 \| x \|^2 + 2 \| y \|^2 \\ \quad \biggr \| \left ( \frac{1}{2}(k_m - h_0) \right ) + \left ( \frac{1}{2}(k_n - h_0) \right ) \biggr \|^2 + \biggr \| \left ( \frac{1}{2}(k_m - h_0) \right ) - \left ( \frac{1}{2}(k_n - h_0) \right ) \biggr \|^2 &= 2 \biggr \| \frac{1}{2}(k_m - h_0) \biggr \|^2 + 2 \biggr \| \frac{1}{2}(k_n - h_0) \biggr \|^2 \end{align}

• Simplifying the equality above gives us:

(5)

\begin{align} \biggr \| \frac{1}{2}k_m + \frac{1}{2}k_n - h_0 \biggr \|^2 + \biggr \| \frac{1}{2}k_m - \frac{1}{2}k_n \biggr \|^2 &= 2 \biggr \| \frac{1}{2} (k_m - h_0) \biggr \|^2 + 2 \biggr \| \frac{1}{2}(k_n - h_0) \biggr \|^2 \\ \biggr \| \frac{1}{2}k_m + \frac{1}{2}k_n - h_0 \biggr \|^2 + \frac{1}{4} \| k_m - k_n \|^2 &= \frac{1}{2} \| k_m - h_0 \|^2 + \frac{1}{2} \| k_n - h_0 \|^2 \end{align}

• From $(*)$ we get that:

(6)

\begin{align} \quad \frac{1}{4} \| k_m - k_n \| &\leq \frac{1}{2} \| k_m - h_0 \|^2 + \frac{1}{2} \| k_n - h_0 \|^2 - \inf_{k \in K} \| k - h_0 \|^2 \end{align}

• And taking the limit as $m, n \to \infty$ gives us that:

(7)

\begin{align} \quad \lim_{m, n \to \infty} \| k_m - k_n \|^2 \leq \lim_{m, n \to \infty} \left [ \frac{1}{2} \| k_m - h_0 \|^2 + \frac{1}{2} \| k_n - h_0 \|^2 - \inf_{k \in K} \| k - h_0 \|^2 \right ] = 0 \end{align}

• Therefore $(k_n)_{n=1}^{\infty}$ is a Cauchy sequence. Since $H$ is a Hilbert space, $H$ is complete, and so the sequence $(k_n)_{n=1}^{\infty}$ converges to some $k_* \in H$, and since $K$ is closed, $k_* \in K$. Now since $\| \cdot \|$ is continuous, we have that:

(8)

\begin{align} \quad \| k_* - h_0 \| = \biggr \| \lim_{n \to \infty} k_n - h_0 \biggr \| = \lim_{n \to \infty} \| k_n - h_0 \| = \inf_{k \in K} \| k - h_0 \| \end{align}

• Lastly, suppose that $k' \in K$ also satisfies the conditions of the theorem. By the convexity of $K$ we have that:

(9)

\begin{align} \quad \frac{1}{2}k_* + \frac{1}{2}k' \in K \end{align}

• Therefore we have that:

(10)

\begin{align} \quad \biggr \| \frac{1}{2}k_* + \frac{1}{2}k' - h_0 \biggr \| \geq \inf_{k \in K} \| k - h_0 \| \quad (**) \end{align}

• Set $x = \frac{1}{2}(k_* - h_0)$ and $y = \frac{1}{2}(k' - h_0)$. Then by the Parallelogram identity we have that:

(11)

\begin{align} \quad \biggr \| \frac{1}{2}k_* + \frac{1}{2}k' - h_0 \biggr \|^2 + \biggr \| \frac{1}{2}k_* - \frac{1}{2}k' \biggr \|^2 = 2 \biggr \| \frac{1}{2} (k_* - h_0) \biggr\|^2 + 2 \biggr \| \frac{1}{2} (k' - h_0) \biggr \|^2 \end{align}

• And so from $(**)$:

(12)

\begin{align} \quad \frac{1}{4} \|k_* - k' \|^2 \leq \frac{1}{2} \| k_* - h_0 \|^2 + \frac{1}{2} \| k' - h_0 \|^2 - \inf_{k \in K} \| k - h_0 \|^2 = 0 \end{align}

• Hence $\| k_* - k' \| = 0$, and so $k_* = k'$. $\blacksquare$

It is important to note that all of the hypotheses in the best approximation theorem are necessary. In particular, if $K$ is not convex then there may be many points in $K$ for which the distance between $h$ and those points equals the distance between $h$ and $K$ as illustrated below: