Vector and Matrix Norms

# Vector and Matrix Norms

Definition: Let $X$ be a vector space over $\mathbb{R}$ or $\mathbb{C}$. A Norm on $X$ is a function $|\cdot| : X \to [0, \infty)$ with the following properties:1) $|x| \geq 0$ for all $x \in X$ and $|x| = 0$ if and only if $x = 0$.2) For all $\alpha \in \mathbb{R}$ (or $\mathbb{C}$), $|\alpha x| = |\alpha| |x|$.3) For all $x, y \in X$, $|x + y| \leq |x| + |y|$. |

There are many common vector norms. These vector norms are defined below for a vector $x = (x_1, x_2, ..., x_n) \in \mathbb{R}^n$:

(1)\begin{align} \quad |x|_1 &= \sum_{k=1}^{n} |x_k| \\ \\ \\ \quad |x|_2 &= \left ( \sum_{k=1}^{n} |x_k|^2 \right )^{1/2}\\ \\ \\ \quad |x|_p &= \left ( \sum_{k=1}^{n} |x_k|^p \right )^{1/p} \quad 1 \leq p < \infty\\ \\ \\ \quad |x|_{\infty} &= \max \{ |x_1|, |x_2|, ..., |x_n| \} \end{align}

# Matrix Norms

We now define a particular matrix norm.

Definition: Let $A$ be an $m \times n$ matrix. Then the Norm of $A$ is defined as $|A| = \sup \{ |Ax| : x \in \mathbb{R}^n \: \mathrm{and} \: |x| = 1 \}$. |

*In the definition above, if $A$ is an $m \times n$ matrix then $x$ is an $n \times 1$ matrix and the product $Ax$ is an $m \times 1$ matrix. So the norm of $A$ is the supremum of $m \times 1$ vector norms $|Ax|$ where $x$ are unit vectors.*

We now state some important properties of this matrix norm.

Proposition 1: Let $A$ and $B$ be $m \times n$ matrices, let $x$ be an $n \times 1$ vector, and let $\alpha \in \mathbb{R}$. Then:a) $|A + B| \leq |A| + |B|$.b) $|Ax| \leq |A| |x|$.c) $|\alpha A| = |\alpha| |A|$.d) $|A| \geq 0$.e) $|A| = 0$ if and only if $A = 0$.f) $|A| \leq \sum_{i=1}^{m} \max_{1 \leq j \leq n} |a_{i,j}| \leq \sum_{i=1}^{m} \sum_{j=1}^{n} |a_{i,j}|$. |