Orthogonal Projection Operators

Recall from the Orthogonal Complements page that if $U$ is a subset of an inner product space $V$, then the orthogonal complement of $U$ denoted $U^{\perp}$ is the set of vectors $v \in V$ such that $v$ is orthogonal to every vector $u \in U$, that is $U^{\perp} = \{ v \in V : <v, u> = 0 \: \forall u \in U \}$. Also recall that if $U$ is more than just a subset of $V$, that is, if $U$ is a finite-dimensional subspace of $V$, then we have that $V = U \oplus U^{\perp}$.

In such cases, for all vectors $v \in V$ we can write $v$ uniquely as the sum of a vector $u \in U$ and a vector $w \in U^{\perp}$:

(1)

\begin{align} \quad v = \underbrace{u}_{\in U} + \underbrace{w}_{\in U^{\perp}} \end{align}

Now consider the linear operator $P_U \in \mathcal L(V)$ defined such that $P_U(v) = u$ for all $v \in V$. Then $P_U$ is a Projection Operator which we could alternatively denote as $P_U = P_{U, U^{\perp}}$. More specifically, $P_U$ is an orthogonal projection operator.

Definition: Let $V$ be an inner product space and let $U$ be a subspace of $V$ such that $V = U \oplus U^{\perp}$. Then for all $v \in V$ we have that $v = u + w$ where $u \in U$ and $w \in U^{\perp}$. The the Orthogonal Projection Operator of $V$ onto $U$ is the linear operator $P_U \in \mathcal L (V)$ defined such that $P_U(v) = u$ for all $v \in V$.

The following proposition outlines some of the important properties of orthogonal projection operators.

Proposition 1: Let $V$ be an inner product space and let $U$ be a subspace of $V$ such that $V = U \oplus U^{\perp}$. Then for the projection operator $P_U \in \mathcal L(V)$ we have that: a) $\mathrm{range} (P_U) = U$. b) $\mathrm{null} (P_U) = U^{\perp}$. c) $(v - P_U(v)) \in U^{\perp}$ for all $v \in V$. d) $P_U^2 = P_U$. e) $\| P_U(v) \| ≤ \| v \|$ for all $v \in V$.

• Proof of a) We have that $P_U = P_{U, U^{\perp}}$ and it follows immediately from the proof on the Projection Operators page that $\mathrm{range} (P_U) = U$.

• Proof of b) We have that $P_U = P_{U, U^{\perp}}$ and it follows immediately from the proof on the page mentioned above that $\mathrm{null} (P_U) = U^{\perp}$.

• Proof of c) Let $v \in V$ be written as $v = u + w$ where $u \in U$ and $w \in U^{\perp}$. Then $w = v - u = v - P_U(v)$, so clearly $(v - P_U(v)) \in U^{\perp}$.

• Proof of d) Let $v \in V$ be written as $v = u + w$. Then $P_u(v) = u$. Note that $u \in V$ and so $u = u + 0$ where $u \in U$ and $0 \in U^{\perp}$. This is the only way to write $u$ as the sum of vectors from $U$ and $U^{\perp}$ since $V = U \oplus U^{\perp}$. So applying the operator again and we have that $P_U^2 (v) = P_U(u) = u = P_U(v)$. Thus $P_U^2 = P_U$.

• Proof of e) Let $v \in V$ be written as $v = u + w$. Noting that $u$ is orthogonal to $w$, we can apply the Pythagorean Theorem by taking the norm squared of both sides and we have that:

(2)

\begin{align} \quad \| v \|^2 = \| u + w \|^2 = \| u \|^2 + \| w \|^2 ≥ \| u \|^2 = \| P_U(v) \|^2 \end{align}

• If we square root both sides we get that $\| P_U(v) \| ≤ \| v \|$ as desired.