Inner Product Spaces

Inner Product Spaces

We are now going to look at a type of vector space that is associated with a function known as an inner product which we define below.

Definition: Let $V$ be a vector space over the field $\mathbb{F}$ ($\mathbb{R}$ or $\mathbb{C}$). An Inner Product on $V$ is a function which takes each pair of vectors $u, v \in V$ and assigns a number $<u, v> \in \mathbb{F}$ with the following properties:
1) $<u, u> ≥ 0$ for all vectors $u \in V$ (Positivity Property).
2) $<u, u> = 0$ if and only if $u = 0$ (Definiteness Property).
3) $<u + v, w> = <u, w> + <v, w>$ for all vectors $u, v, w \in V$ (Additivity in The First Slot Property).
4) $<au, v> = a<u, v>$ for any $a \in \mathbb{F}$ and for all vectors $u, v \in V$ (Homogeneity in The First Slot Property).
5) $<u, v> = \overline{<v, u>}$ for all vectors $u, v \in V$ (Conjugate Symmetry Property).

Note that if $V$ is a vector space over $\mathbb{R}$, then the complex conjugate of a real number is equal to itself, and thus property 5 is $<u, v> = <v, u>$ for all vectors $u, v \in V$.

One type of inner product space that we have already seen is the typical dot product. Consider the vector space $\mathbb{R}^n$. For any vectors $u, v \in \mathbb{R}^n$ we have that $u = (u_1, u_2, …, u_n)$ and $v = (v_1, v_2, …, v_n)$. Then the dot product between $u$ and $v$ is denoted $u \cdot v = <u, v>$ and is defined as:

(1)
\begin{align} \quad <u, v> = u_1v_1 + u_2v_2 + … + u_nv_n \end{align}

Let's verify that the dot product is indeed an inner product by verifying all five of the properties listed above. We first show that the positivity property holds, we note that:

(2)
\begin{align} \quad <u, u> = u_1^2 + u_2^2 + … + u_n^2 \end{align}

So $<u, u>$ is the sum of squared real numbers and is hence nonnegative, that is $<u, u> ≥ 0$. Now let's show that the definiteness property holds. Suppose that $<u, u> = 0$. Then we have that:

(3)
\begin{align} \quad <u, u> = u_1^2 + u_2^2 + … + u_n^2 = 0 \end{align}

Each $u_j^2 = 0$ though, since $u_j^2 ≥ 0$ for $j = 1, 2, …, n$ (if some $u_j > 0$ then we would need a $u_i < 0$ to make $<u, u> = 0$). Thus $u_j = 0$ for each $j = 1, 2, …, n$ and so $u = (u_1, u_2, …, u_n) = (0, 0, …, 0) = 0$. Now suppose that $u = 0$. Then $u = (0, 0, …, 0)$ and $<u, u> = 0^2 + 0^2 + … + 0^2 = 0$ so the definiteness property holds.

Now let's show that the dot product has additivity in the first slot. Let $u, v, w \in V$. Then:

(4)
\begin{align} \quad <u + v, w> = (u_1 + v_1, u_2 + v_2, …, u_n + v_n) \cdot (w_1, w_2, …, w_n) \\ \quad <u + v, w> = (u_1 + v_1)w_1 + (u_2 + v_2)w_2 + … + (u_n + v_n)w_n \\ \quad <u + v, w> = [u_1w_1 + u_2w_2 + … + u_nw_n] + [v_1w_1 + v_2w_2 + … + v_nw_n] \\ \quad <u + v, w> = <u, w> + <v, w> \end{align}

Now let's show that the homogeneity property holds. Let $a \in \mathbb{R}$. Then:

(5)
\begin{align} \quad <au, v> = (au_1, au_2, …, au_n) \cdot (v_1, v_2, …, v_n) \\ \quad <au, v> = au_1v_1 + au_2v_2 + … + au_nv_n \\ \quad <au, v> = a(u_1v_1 + u_2v_2 + … + u_nv_n) \\ \quad <au, v> = a<u, v> \end{align}

We note that the our vector space $\mathbb{R}^n$ is over the field of real numbers, and so clearly $<u, v> = \overline{<v, u>} = <v, u>$ since the dot product results in real outputs and the conjugate of a real number is itself. Therefore we have verified that the dot product is indeed an inner product space.

Of course, there are many other types of inner products that can be formed on more abstract vector spaces. Such a vector space with an inner product is known as an inner product space which we define below.

Definition: A vector space $V$ over the field $\mathbb{R}$ or $\mathbb{C}$ is with an inner product is called an Inner Product Space.

It is nice to have a relatively short list of properties to verify that a function on a vector space is an inner product. Notice though that we only specified additivity and homogeneity in the first slot. In fact, as the following proposition shows, an inner product also has additivity and conjugate homogeneity in the second slot (which can be easily derived from the five properties of inner products already listed).

Proposition 1: If $V$ is an inner product space over the field $\mathbb{F}$ ($\mathbb{R}$ or $\mathbb{C}$) then:
a) $<u, v + w> = <u, v> + <u, w>$ for all vectors $u, v, w \in V$ (Additivity in The Second Slot).
b) $<u, av> = \overline{a} <u, v>$ for $a \in \mathbb{F}$ and for all vectors $u, v \in V$ (Conjugate Homogeneity in The Second Slot).

The proof of Proposition 1 is relatively straightforward and uses only the five properties listed in the inner product space.

  • Proof of a):
(6)
\begin{align} \quad <u, v+w> = \overline{<v+w, u>} = \overline{<v, u>} + \overline{<w, u>} = <u, v> + <u, w> \end{align}
  • Proof of b):
(7)
\begin{align} \quad <u, av> = \overline{<av, u>} = \overline{a<v, u>} = \overline{a} \overline{<v, u>} = \overline{a} <u, v> \end{align}
  • Thus our proof is complete. $\blacksquare$

The following definition for orthogonality is a generalization of the dot product which we defined two vectors in $\mathbb{R}^n$ to be perpendicular (or orthogonal) if $u \cdot v = 0$.

Definition: If $V$ is an inner product space then a vector $u \in V$ is said to be Orthogonal to $v \in V$ if $<u, v> = 0$.

Note that from the definition of an inner product space we have that $<u, u> = 0$ if and only if $u = 0$. Thus $<0, 0> = 0$, and so $0$ is the only vector that is orthogonal to itself. Furthermore, it is not hard to see that any vector is orthogonal to $0$.

Example 1

Consider the vector space $V$ of $2 \times 2$ that has zeroes everywhere except having real entries on the main diagonal. Determine whether or not the function $f(A, B) = \mid \mathrm{tr}(A + B) \mid$ for $A, B \in M_{22}$ is an inner product space.

Let $A$ be defined as $A = \begin{bmatrix} a & 0\\ 0 & -a \end{bmatrix}$ for $a \in \mathbb{R}$ and $a \neq 0$. Then we have that:

(8)
\begin{align} \quad f(A, A) = \mid \mathrm{tr} (A + A) \mid = \mid \mathrm{tr} (2A) \mid = \mid 2(3 + (-3)) \mid = 0 \end{align}

However $A$ is not the $2 \times 2$ zero matrix, and so $f$ does not define an inner product space.

Unless otherwise stated, the content of this page is licensed under Creative Commons Attribution-ShareAlike 3.0 License