The Gradient of a Differentiable Function from Rn to R

Definition: Let $S \subseteq \mathbb{R}^n$ be open, $\mathbb{c} \in S$, and $f : S \to \mathbb{R}$. If all of the partial derivatives of $f$ exist at $\mathbf{c}$, i.e., if $D_1 f(\mathbf{c})$, $D_2 f(\mathbf{c})$, …, $D_n f(\mathbf{c})$ all exist, then the Gradient of $f$ at $\mathbf{c}$ is defined as the vector $\nabla f(\mathbf{c}) = (D_1f(\mathbf{c}), D_2f(\mathbf{c}), ..., D_nf(\mathbf{c})$.

If $f$ is differentiable at $\mathbf{c}$ then the gradient of $f$ at $\mathbf{c}$ is defined since the differentiability of a function at a point implies the existence of all directional derivatives of the function at that point (and hence the existence of the partial derivatives at that point).

For example, consider the following function $f : \mathbb{R}^3 \to \mathbb{R}$ defined for all $(x, y, z) \in \mathbb{R}^3$ by:

(1)

\begin{align} \quad f(x, y, z) = xe^y \sin z \end{align}

Then the partial derivatives of $f$ at an arbitrary point $(x, y, z) \in \mathbb{R}^3$ are:

(2)

\begin{align} \quad D_1 f(x, y, z) = e^y \sin z \quad , \quad D_2 f(x, y, z) = xe^y \sin z \quad , \quad D_3 f(x, y, z) = xe^y \cos z \end{align}

Therefore the gradient of $f$ at $(x, y, z)$ is:

(3)

\begin{align} \quad \nabla f(x, y, z) = (e^y \sin z, xe^y \sin z, xe^y \cos z) \end{align}

Now suppose that $f$ is differentiable at $\mathbf{c}$. Then all of the directional derivatives of $f$ at $\mathbf{c}$ exist and we will soon see that in fact for any vector $\mathbf{u} \in \mathbb{R}^n$ we have that:

(4)

\begin{align} \quad f'(\mathbf{c}, \mathbf{u}) = \nabla f(\mathbf{c}) \cdot \mathbf{u} \end{align}

We will use this property in proving the following theorem which tells us that the maximum rate of change at a point in a differentiable multivariable real-valued function will always be in the direction of the gradient vector at that point.

Theorem 1: Let $S \subseteq \mathbb{R}^n$ be open, $\mathbb{c} \in S$, and $f : S \to \mathbb{R}$. If $f$ is differentiable at $\mathbf{c}$ and $\| \nabla f(\mathbf{c}) \| \neq 0$ then there exists a unit vector $\mathbf{u} \in \mathbb{R}^n$ such that $\mid f'(\mathbf{c}, \mathbf{u}) \mid = \| \nabla f(\mathbf{c}) \|$.

• Proof: Let $f : S \to \mathbb{R}$ and suppose that $\| \nabla f(\mathbf{c}) \| \neq 0$. Then for each unit vector $\mathbf{u} \in \mathbb{R}^n$, as discussed on the The Jacobian Matrix of Differentiable Functions from Rn to Rm page, the directional derivative of $f$ at $\mathbf{c}$ in the direction of $\mathbf{u}$ is given by the formula:

(5)

\begin{align} \quad f'(\mathbf{c}, \mathbf{u}) = \nabla f(\mathbf{c}) \cdot \mathbf{u} \end{align}

• Take the absolute value of both sides of the equation above and apply the Cauchy-Schwarz inequality:

(6)

\begin{align} \quad \mid f'(\mathbf{c}, \mathbf{u}) \mid = \mid \nabla f(\mathbf{c}) \cdot \mathbf{u} \mid \leq \| \nabla f(\mathbf{c}) \| \| \mathbf{u} \| = \| \nabla f(\mathbf{c}) \| \cdot 1 = \| \nabla f(\mathbf{c}) \| \end{align}

• This shows us that for each $\mathbf{u} \in \mathbb{R}^n$ that $\mid \mathbf{f}'(\mathbf{c}, \mathbf{u}) \mid$ is bounded above by $\| \nabla f(\mathbf{c}) \|$. We wish to show that a maximum for $\mid f'(\mathbf{c}, \mathbf{u}) \mid$ is obtained at some $\mathbf{u} \in \mathbb{R}^n$ and that it equals $\| \nabla f(\mathbf{c}) \|$.

• Now note by the dot product we have that $\mid \nabla f(\mathbf{c}) \cdot \mathbf{u} \mid = \| \nabla f(\mathbf{c}) \| \| \mathbf{u} \| \mid \cos \theta \mid = \| \nabla f(\mathbf{c}) \| \mid \cos \theta \mid$ where $\theta$ is the angle between the vectors $\nabla f(\mathbf{c})$ and $\mathbf{u}$. So we see that $\mid \nabla f(\mathbf{c}) \cdot \mathbf{u} \mid = \| \nabla f(\mathbf{c}) \|$ when $\theta = 0$, i.e., $\mid f'(\mathbf{c}, \mathbf{u}) \mid$ is maximized when $\mathbf{u}$ is parallel to $\nabla f(\mathbf{c})$ and $\mid f'(\mathbf{c}, \mathbf{u}) \mid = \| \nabla f(\mathbf{c}) \|$. $\blacksquare$