Matrix Arithmetic

This page is intended to be a part of the Numerical Analysis section of Math Online. Similar topics can also be found in the Linear Algebra section of the site.

# Matrix Arithmetic

We are about to look at various numerical methods to solve systems of linear equations. In doing so, we will be talking a lot about matrices as they can be used to represent such systems. Thus we will look at what exactly a matrix is and some important arithmetic operations and properties of matrices on this page. We begin by defining a matrix.

 Definition: An Matrix is an array of numbers in the form $\begin{bmatrix} a_{11} & a_{12} & \cdots & a_{1n}\\ a_{21} & a_{22} & \cdots & a_{2n}\\ \vdots & \vdots & \ddots & \vdots\\ a_{m1} & a_{m2} & \cdots & a_{mn} \end{bmatrix}$. The Entries are the values of $a_{ij}$ for $i = 1, 2, ..., m$ and $j = 1, 2, ..., n$. The Size of the matrix is written as $m \times n$ where $m$ represents the number of rows and $n$ represents the number of columns in the matrix. A matrix is said to be Square if it has the same number of rows and columns, that is $m = n$.

In some cases, writing $a_{i,j}$ (separating the row number and the column number of each entry) instead of $a_{ij}$ is neater in denoting the entries of a matrix.

Often times we use capital letters to denote matrices. For example, $A = \begin{bmatrix} 1 & -2 & 8 \\ 0 & 5 & 2 \end{bmatrix}$ is a $2 \times 3$ matrix with real number entries. Another example is $B = \begin{bmatrix} a & b \\ c & d \end{bmatrix}$ which is a $2 \times 2$ (square) matrix and whose entries are $a, b, c, d$ for whatever they may represent (usually numbers). Furthermore, it should not be difficult to see that a matrix of size $m \times n$ has $mn$ entries.

Some various arithmetic operations can be defined on matrices. For example, if $A$ and $B$ are two matrices that has the same size, $m \times n$, then we can define the sum $A + B$ to be the $m \times n$ matrix whose entries are the sums of the corresponding entries from $A$ and $B$, that is:

(1)
\begin{align} \quad A + B = \begin{bmatrix} a_{11} & a_{12} & \cdots & a_{1n}\\ a_{21} & a_{22} & \cdots & a_{2n}\\ \vdots & \vdots & \ddots & \vdots\\ a_{m1} & a_{m2} & \cdots & a_{mn} \end{bmatrix} + \begin{bmatrix} b_{11} & b_{12} & \cdots & b_{1n}\\ b_{21} & b_{22} & \cdots & b_{2n}\\ \vdots & \vdots & \ddots & \vdots\\ b_{m1} & b_{m2} & \cdots & b_{mn} \end{bmatrix} = \begin{bmatrix} a_{11} + b_{11} & a_{12} + b_{12} & \cdots & a_{1n} + b_{1n} \\ a_{21} + b_{21} & a_{22} + b_{22} & \cdots & a_{2n} + b_{2n} \\ \vdots & \vdots & \ddots & \vdots\\ a_{m1} + b_{m1} & a_{m2} + b_{m2} & \cdots & a_{mn} + b_{mn} \end{bmatrix} \end{align}

Similarly, subtraction of two matrices $A$ and $B$ that have the same size, $m \times n$ can be defined analogously. If $A$ and $B$ do not have the same size, then we say that the sum/difference of $A$ and $B$ is undefined.

Now if $k$ is a scalar, then we can define the product $k$ multiplied by the $m \times n$ matrix $A$ as the $m \times n$ matrix $kA$ whose entries are obtained by taking the corresponding entries of $A$ and multiplying them by $k$, that is:

(2)
\begin{align} \quad kA = \begin{bmatrix} ka_{11} & ka_{12} & \cdots & ka_{1n}\\ ka_{21} & ka_{22} & \cdots & ka_{2n}\\ \vdots & \vdots & \ddots & \vdots\\ ka_{m1} & ka_{m2} & \cdots & ka_{mn} \end{bmatrix} \end{align}

Now defining matrix multiplication with another matrix is more complex. The obvious definition of matrix multiplication yields little when it comes to mathematical importance. Instead we say that if $A$ is an $m \times n$ matrix and $B$ is an $n \times r$ matrix, then the product of $A$ and $B$ denoted $AB = C$ is the resulting $m \times r$ matrix whose entries for $i = 1, 2, ..., m$ and $j = 1, 2, ..., r$ can be computed as:

(3)
\begin{align} \quad c_{ij} = a_{i1}b_{1j} + a_{i2}b_{2j} + ... + a_{in}b_{nj} = \sum_{k=1}^{n} a_{ik}b_{kj} \end{align}

The size of the matrix product $AB$ is equal to the number of rows of $A$ by the number of columns of $B$. Furthermore, if $A$ is an $m \times n$ matrix and $B$ is a $p \times r$ matrix where $n \neq p$, that is, the number of columns of $A$ does not equal the number of rows of $B$, then the matrix product $AB$ is said to be undefined. It is also important to note that $AB$ need not equal $BA$. In fact, $AB$ may be defined while $BA$ might not such as when $A$ and $B$ are not square matrices.

We will now look at some important types of matrices and definitions.

 Definition: If $A$ is an $m \times n$ matrix, then the Transpose of $A$ denoted $A^T$ is the $n \times m$ matrix that is obtained by interchanging the rows of $A$ with the columns of $A$.

For example, a general matrix transpose of an $m \times n$ matrix $A$ has the form:

(4)
\begin{align} \quad A^T = \begin{bmatrix} a_{11} & a_{21} & \cdots & a_{n1}\\ a_{12} & a_{22} & \cdots & a_{n2}\\ \vdots & \vdots & \ddots & \vdots\\ a_{1m} & a_{2m} & \cdots & a_{nm} \end{bmatrix} \end{align}
 Definition: If $A$ is a square $n \times n$ matrix, then the Main Diagonal of $A$ are the entries $a_{ii}$ for $i = 1, 2, ..., n$.

Simply put, the entries of the main diagonal of a square matrix are the entries whose column and row number are equal. For a less general example, if $A = \begin{bmatrix} 0 & 3 & 2 \\ 1 & -1 & 2 \end{bmatrix}$ then $A^T = \begin{bmatrix} 0 & 1 \\ 3 & -1 \\ 2 & 2 \end{bmatrix}$. Furthermore, if $A$ is a square matrix, then the entries along the main diagonal of $A$ are the same as the entries along the main diagonal of $A^T$ as you should verify.

 Definition: If $A$ is a square $n \times n$ matrix, then the Trace of $A$ denoted $\mathrm{tr}(A)$ is the sum of the elements on the main diagonal of $A$.

From the definition above, we see that the trace of an $n \times n$ matrix $A$ is simply:

(5)
\begin{align} \quad \mathrm{tr} (A) = \sum_{i=1}^{n} a_{ii} \end{align}
 Definition: The $n \times n$ Identity Matrix denoted $I_n$ is the square matrix whose entries are zero everywhere except on the main diagonal where they are ones.

For example, the $I_2 = \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix}$ is the $2 \times 2$ identity matrix.

 Definition: If $A$ is a square $n \times n$ matrix, then $A$ is said to be Invertible if there exists an $n \times n$ matrix $B$ such that $AB = I_n = BA$. In such case we say that $B = A^{-1}$ is the Inverse of $A$.

It is important to note that not all square matrices are invertible. Furthermore, it is also important to note that the inverse of a matrix is unique which can easily be proven by assuming two inverses exist and then showing that they're equal.