Hessian Matrices

We are about to look at a method of finding extreme values for multivariable functions. We will first need to define what is known as the Hessian Matrix (sometimes simply referred to as just the "Hessian") of a multivariable function.

Definition: Let $\mathbf{x} = (x_1, x_2, ..., x_n)$ and let $z = f(x_1, x_2, ..., x_n) = f(\mathbf{x})$ be an $n$ variable real-valued function whose second partial derivatives exist. Then the Hessian Matrix of $f$ is the $n \times n$ matrix of second partial derivatives of $f$ denoted $\mathcal H (\mathbf{x}) = \begin{bmatrix} f_{11} (\mathbf{x}) & f_{12} (\mathbf{x}) & \cdots & f_{1n} (\mathbf{x})\\ f_{21} (\mathbf{x}) & f_{22} (\mathbf{x}) & \cdots & f_{2n} (\mathbf{x})\\ \vdots & \vdots & \ddots & \vdots \\ f_{n1} (\mathbf{x}) & f_{n2} (\mathbf{x}) & \cdots & f_{nn} (\mathbf{x}) \end{bmatrix}$.

For a two variable function $z = f(x, y)$ we have that the Hessian Matrix of $f$ is:

(1)

\begin{align} \quad \mathcal H (x, y) = \begin{bmatrix} f_{11} (x, y) & f_{12} (x, y)\\ f_{21} (x, y) & f_{22} (x, y) \end{bmatrix} = \begin{bmatrix} \frac{\partial ^2 f}{\partial x^2} & \frac{\partial ^2 f}{\partial y \partial x}\\ \frac{\partial ^2 f}{\partial x \partial y} & \frac{\partial ^2 f}{\partial y^2} \end{bmatrix} \end{align}

For a three variable function $w = f(x, y, z)$ we have that the Hessian Matrix of $f$ is:

(2)

\begin{align} \quad \mathcal H (x, y, z) = \begin{bmatrix} f_{11} (x, y, z) & f_{12} (x, y, z) & f_{13} (x, y, z) \\ f_{21} (x, y, z) & f_{22} (x, y, z) & f_{23} (x, y, z)\\ f_{31} (x, y, z) & f_{32} (x, y, z) & f_{33} (x, y, z) \end{bmatrix} = \begin{bmatrix} \frac{\partial^2 f}{\partial x^2} & \frac{\partial f}{\partial y \partial x} & \frac{\partial^2 f}{\partial z \partial x} \\ \frac{\partial^2 f}{\partial x \partial y} & \frac{\partial ^2 f}{\partial y^2} & \frac{\partial ^2 f}{\partial z \partial y}\\ \frac{\partial^2 f}{\partial x \partial z} & \frac{\partial^2 f}{\partial y \partial z} & \frac{\partial ^2 f}{\partial z^2}\end{bmatrix} \end{align}

Recall from the Clairaut's Theorem on Higher Order Partial Derivatives page that if the second mixed partial derivatives of $f$ are continuous on some neighbourhood of $f$, then these mixed partial derivatives are equal on this neighbourhood. That is for $\mathbf{x} = (x_1, x_2, ..., x_n)$ and $z = f(x_1, x_2, ..., x_n)$ we have that $f_{ij} (\mathbf{x}) = f_{ji} (\mathbf{x})$ for $i = 1, 2, ...,n$ and $j = 1, 2, ..., n$. So the continuity of all of the second mixed partial derivatives of $f$ imply that the Hessian $H(\mathbf{x})$ is symmetric.

Example 1

Find the Hessian Matrix of the function $f(x, y) = x^2y + xy^3$.

We need to first find the first partial derivatives of $f$. We have that:

(3)

\begin{align} \quad \frac{\partial f}{\partial x} = 2xy + y^3 \quad , \quad \frac{\partial f}{\partial y} = x^2 + 3xy^2 \end{align}

We then calculate the second partial derivatives of $f$:

(4)

\begin{align} \quad \frac{\partial^2 f}{\partial x^2} = 2y \quad , \quad \frac{\partial^2 f}{\partial y \partial x} 2x + 3y^2 \quad , \quad \frac{\partial^2 f}{\partial x \partial y} = 2x + 3y^2 \quad , \quad \frac{\partial^2 f}{\partial y^2} = 6xy \end{align}

Therefore the Hessian Matrix of $f$ is:

(5)

\begin{align} \quad \mathcal H (x, y) = \begin{bmatrix} 2y & 2x + 3y^2 \\ 2x + 3y^2 & 6xy \end{bmatrix} \end{align}