The Kronecker Product of Two Matrices

Before we look deeper into Hadamard matrices, we will need to define a special type of product between two matrices $A$ and $B$ known as their Kronecker product.

Definition: Let $A$ be an $m \times n$ matrix and let $B$ be an $s \times t$ matrix. The Kronecker Product of these matrices is the matrix $A \otimes B = \begin{bmatrix} a_{1,1} B & a_{1,2} B & \cdots & a_{1,n} B \\ a_{2,1} B & a_{2,2} B & \cdots & a_{2,n} B\\ \vdots & \vdots & \ddots & \vdots \\ a_{m,1} B & a_{m,2} B & \cdots & a_{m,n} B \end{bmatrix}$.

The notation $a_{i,j} B$ denotes the $s \times t$ matrix given as $a_{i,j} B = \begin{bmatrix} a_{i,j}b_{1,1} & a_{i,j}b_{1,2} & \cdots & a_{i,j}b_{1,t} \\ a_{i,j}b_{2,1} & a_{i,j}b_{2,2} & \cdots & a_{i,j}b_{2,t} \\ \vdots & \vdots & \ddots & \vdots \\ a_{i,j}b_{s,1} & a_{i,j}b_{s,2} & \cdots & a_{i,j}b_{s,t} \end{bmatrix}$. Thus, if $A$ is an $m \times n$ matrix and $B$ is an $s \times t$ matrix then $A \otimes B$ is an $ms \times nt$ matrix.

Also note that if $A$ is a $1 \times 1$ matrix, i.e., $A = \begin{bmatrix} a \end{bmatrix}$ is a scalar, then $A \otimes B = aB$.

For example, consider the following matrices:

(1)

\begin{align} \quad A_{2 \times 3} = \begin{bmatrix} 1 & 2 & 3 \\ 4 & 5 & 6 \end{bmatrix} \quad , \quad B_{4 \times 1} = \begin{bmatrix} a \\ b \\ c \\ d \end{bmatrix} \end{align}

The corresponding Kronecker product $A \otimes B$ is given as:

(2)

\begin{align} \quad A \otimes B_{8 \times 3} = \begin{bmatrix} 1 \begin{bmatrix} a \\ b \\ c \\ d \end{bmatrix} & 2 \begin{bmatrix} a \\ b \\ c \\ d \end{bmatrix}& 3 \begin{bmatrix} a \\ b \\ c \\ d \end{bmatrix} \\ 4 \begin{bmatrix} a \\ b \\ c \\ d \end{bmatrix} & 5 \begin{bmatrix} a \\ b \\ c \\ d \end{bmatrix} & 6 \begin{bmatrix} a \\ b \\ c \\ d \end{bmatrix} \end{bmatrix} = \begin{bmatrix} a & 2a & 3a \\ b & 2b & 3b \\ c & 2c & 3c \\ d & 2d & 3d \\ 4a & 5a & 6a \\ 4b & 5b & 6b \\ 4c & 5c & 6c \\ 4d & 5d & 6d \end{bmatrix} \end{align}

The special operation $\otimes$ has many important properties. We outline some of these without proof in the following theorem.

Theorem 1: Let $A$, $B$, and $C$ be matrices. Then: a) $A \otimes (B \otimes C) = (A \otimes B) \otimes C$. b) $A \otimes (B + C) = A \otimes B + A \otimes C$ whenever $B$ and $C$ are matrices of the same size. c) If $k$ is a scalar then $k(A \otimes B) = (kA) \otimes B = A \otimes (kB)$.

Note that in general we cannot expect that $A \otimes B$ will equal $B \otimes A$.