Vector and Matrix Norms

Definition: Let $X$ be a vector space over $\mathbb{R}$ or $\mathbb{C}$. A Norm on $X$ is a function $|\cdot| : X \to [0, \infty)$ with the following properties: 1) $|x| \geq 0$ for all $x \in X$ and $|x| = 0$ if and only if $x = 0$. 2) For all $\alpha \in \mathbb{R}$ (or $\mathbb{C}$), $|\alpha x| = |\alpha| |x|$. 3) For all $x, y \in X$, $|x + y| \leq |x| + |y|$.

There are many common vector norms. These vector norms are defined below for a vector $x = (x_1, x_2, ..., x_n) \in \mathbb{R}^n$:

(1)

\begin{align} \quad |x|_1 &= \sum_{k=1}^{n} |x_k| \\ \\ \\ \quad |x|_2 &= \left ( \sum_{k=1}^{n} |x_k|^2 \right )^{1/2}\\ \\ \\ \quad |x|_p &= \left ( \sum_{k=1}^{n} |x_k|^p \right )^{1/p} \quad 1 \leq p < \infty\\ \\ \\ \quad |x|_{\infty} &= \max \{ |x_1|, |x_2|, ..., |x_n| \} \end{align}

Matrix Norms

We now define a particular matrix norm.

Definition: Let $A$ be an $m \times n$ matrix. Then the Norm of $A$ is defined as $|A| = \sup \{ |Ax| : x \in \mathbb{R}^n \: \mathrm{and} \: |x| = 1 \}$.

In the definition above, if $A$ is an $m \times n$ matrix then $x$ is an $n \times 1$ matrix and the product $Ax$ is an $m \times 1$ matrix. So the norm of $A$ is the supremum of $m \times 1$ vector norms $|Ax|$ where $x$ are unit vectors.

We now state some important properties of this matrix norm.

Proposition 1: Let $A$ and $B$ be $m \times n$ matrices, let $x$ be an $n \times 1$ vector, and let $\alpha \in \mathbb{R}$. Then: a) $|A + B| \leq |A| + |B|$. b) $|Ax| \leq |A| |x|$. c) $|\alpha A| = |\alpha| |A|$. d) $|A| \geq 0$. e) $|A| = 0$ if and only if $A = 0$. f) $|A| \leq \sum_{i=1}^{m} \max_{1 \leq j \leq n} |a_{i,j}| \leq \sum_{i=1}^{m} \sum_{j=1}^{n} |a_{i,j}|$.