# The Gradient of a Differentiable Function from Rn to R

Definition: Let $S \subseteq \mathbb{R}^n$ be open, $\mathbb{c} \in S$, and $f : S \to \mathbb{R}$. If all of the partial derivatives of $f$ exist at $\mathbf{c}$, i.e., if $D_1 f(\mathbf{c})$, $D_2 f(\mathbf{c})$, …, $D_n f(\mathbf{c})$ all exist, then the Gradient of $f$ at $\mathbf{c}$ is defined as the vector $\nabla f(\mathbf{c}) = (D_1f(\mathbf{c}), D_2f(\mathbf{c}), ..., D_nf(\mathbf{c})$. |

*If $f$ is differentiable at $\mathbf{c}$ then the gradient of $f$ at $\mathbf{c}$ is defined since the differentiability of a function at a point implies the existence of all directional derivatives of the function at that point (and hence the existence of the partial derivatives at that point).*

For example, consider the following function $f : \mathbb{R}^3 \to \mathbb{R}$ defined for all $(x, y, z) \in \mathbb{R}^3$ by:

(1)Then the partial derivatives of $f$ at an arbitrary point $(x, y, z) \in \mathbb{R}^3$ are:

(2)Therefore the gradient of $f$ at $(x, y, z)$ is:

(3)Now suppose that $f$ is differentiable at $\mathbf{c}$. Then all of the directional derivatives of $f$ at $\mathbf{c}$ exist and we will soon see that in fact for any vector $\mathbf{u} \in \mathbb{R}^n$ we have that:

(4)We will use this property in proving the following theorem which tells us that the maximum rate of change at a point in a differentiable multivariable real-valued function will always be in the direction of the gradient vector at that point.

Theorem 1: Let $S \subseteq \mathbb{R}^n$ be open, $\mathbb{c} \in S$, and $f : S \to \mathbb{R}$. If $f$ is differentiable at $\mathbf{c}$ and $\| \nabla f(\mathbf{c}) \| \neq 0$ then there exists a unit vector $\mathbf{u} \in \mathbb{R}^n$ such that $\mid f'(\mathbf{c}, \mathbf{u}) \mid = \| \nabla f(\mathbf{c}) \|$. |

**Proof:**Let $f : S \to \mathbb{R}$ and suppose that $\| \nabla f(\mathbf{c}) \| \neq 0$. Then for each unit vector $\mathbf{u} \in \mathbb{R}^n$, as discussed on the The Jacobian Matrix of Differentiable Functions from Rn to Rm page, the directional derivative of $f$ at $\mathbf{c}$ in the direction of $\mathbf{u}$ is given by the formula:

- Take the absolute value of both sides of the equation above and apply the Cauchy-Schwarz inequality:

- This shows us that for each $\mathbf{u} \in \mathbb{R}^n$ that $\mid \mathbf{f}'(\mathbf{c}, \mathbf{u}) \mid$ is bounded above by $\| \nabla f(\mathbf{c}) \|$. We wish to show that a maximum for $\mid f'(\mathbf{c}, \mathbf{u}) \mid$ is obtained at some $\mathbf{u} \in \mathbb{R}^n$ and that it equals $\| \nabla f(\mathbf{c}) \|$.

- Now note by the dot product we have that $\mid \nabla f(\mathbf{c}) \cdot \mathbf{u} \mid = \| \nabla f(\mathbf{c}) \| \| \mathbf{u} \| \mid \cos \theta \mid = \| \nabla f(\mathbf{c}) \| \mid \cos \theta \mid$ where $\theta$ is the angle between the vectors $\nabla f(\mathbf{c})$ and $\mathbf{u}$. So we see that $\mid \nabla f(\mathbf{c}) \cdot \mathbf{u} \mid = \| \nabla f(\mathbf{c}) \|$ when $\theta = 0$, i.e., $\mid f'(\mathbf{c}, \mathbf{u}) \mid$ is maximized when $\mathbf{u}$ is parallel to $\nabla f(\mathbf{c})$ and $\mid f'(\mathbf{c}, \mathbf{u}) \mid = \| \nabla f(\mathbf{c}) \|$. $\blacksquare$