Newton's Method for Solving Systems of Many Nonlinear Equations

We will now extend Newton's Method further to systems of many nonlinear equations. Consider the general system of $n$ linear equations in $n$ unknowns:

(1)

\begin{align} f_1(x_1, x_2, …, x_n) = 0 \\ f_2(x_1, x_2, …, x_n) = 0 \\ \quad \quad \quad \vdots \quad \quad \quad \\ f_n(x_1, x_2, …, x_n) = 0 \end{align}

To simplify explaining Newton's method, we will introduce some shorthand notation. Let $\alpha = \begin{bmatrix} \alpha_1 \\ \alpha_2 \\ \vdots \\ \alpha_n \end{bmatrix}$ be the solution of interest to this system. Let $x^{(0)} = \begin{bmatrix} x_1^{(0)} \\ x_2^{(0)} \\ \vdots \\ x_n^{(0)} \end{bmatrix}$ be an initial approximation to the solution $\alpha$, and let $x^{(k)} = \begin{bmatrix} x_1^{(k)} \\ x_2^{(k)} \\ \vdots \\ x_n^{(k)} \end{bmatrix}$ be the $k^{\mathrm{th}}$ approximation to the solution $\alpha$ from applying Newton's Method. Furthermore, let $\delta^{(k)} = \begin{bmatrix} x_1 - x_1^{(k)} \\ x_2 - x_2^{(k)} \\ \vdots \\ x_n - x_n^{(k)} \end{bmatrix}$.

Define the matrix $F$ applied approximation $x^{(k)}$ to be:

(2)

\begin{align} \quad F(x^{(k)}) = \begin{bmatrix} f_1(x_1^{(k)}, x_2^{(k)}, …, x_n^{(k)} \\ f_2(x_1^{(k)}, x_2^{(k)}, …, x_n^{(k)} \\ \vdots \\ f_n(x_1^{(k)}, x_2^{(k)}, …, x_n^{(k)} \end{bmatrix} \end{align}

And we will define $F'$ applied to our approximation $x^{(k)}$ to be the matrix of partial derivatives evaluated at the point $x^{(k)}$:

(3)

\begin{align} \quad F'(x^{(k)}) = \begin{bmatrix} \frac{\partial}{\partial x_1} f_1(x^{(k)}) & \frac{\partial}{\partial x_2} f_1(x^{(k)}) & \cdots & \frac{\partial}{\partial x_n} f_1(x^{(k)}) \\ \frac{\partial}{\partial x_1} f_2(x^{(k)}) & \frac{\partial}{\partial x_2} f_2(x^{(k)}) & \cdots & \frac{\partial}{\partial x_n} f_2(x^{(k)}) \\ \vdots & \vdots & \ddots & \vdots \\ \frac{\partial}{\partial x_1} f_n(x^{(k)}) & \frac{\partial}{\partial x_2} f_n(x^{(k)}) & \cdots & \frac{\partial}{\partial x_n} f_n(x^{(k)}) \end{bmatrix} \end{align}

The general iteration of Newton's Method is thus given by:

(4)

\begin{align} \quad F'(x^{(k)})\delta^{(k)} = -F(x^{(k)}) \end{align}

Furthermore, each successive approximation of the root $\alpha$ is given by:

(5)

\begin{align} \quad x^{(k+1)} = x^{(k)} + \delta^{(k)} \end{align}

An equivalent form for each successive approximative $x^{(k+1)}$ of $\alpha$ can be obtained by substituting $\delta^{(k)}$ for $\left [ F'(x^{(k)} \right ]^{-1}$ so then $x^{(k+1)} = x^{(k)} - \left [ F'(x^{(k)} \right ]^{-1} F(x^{(k)}$