Newton's Method for Solving Systems of Many Nonlinear Equations

Newton's Method for Solving Systems of Many Nonlinear Equations

We will now extend Newton's Method further to systems of many nonlinear equations. Consider the general system of $n$ linear equations in $n$ unknowns:

(1)
\begin{align} f_1(x_1, x_2, …, x_n) = 0 \\ f_2(x_1, x_2, …, x_n) = 0 \\ \quad \quad \quad \vdots \quad \quad \quad \\ f_n(x_1, x_2, …, x_n) = 0 \end{align}

To simplify explaining Newton's method, we will introduce some shorthand notation. Let $\alpha = \begin{bmatrix} \alpha_1 \\ \alpha_2 \\ \vdots \\ \alpha_n \end{bmatrix}$ be the solution of interest to this system. Let $x^{(0)} = \begin{bmatrix} x_1^{(0)} \\ x_2^{(0)} \\ \vdots \\ x_n^{(0)} \end{bmatrix}$ be an initial approximation to the solution $\alpha$, and let $x^{(k)} = \begin{bmatrix} x_1^{(k)} \\ x_2^{(k)} \\ \vdots \\ x_n^{(k)} \end{bmatrix}$ be the $k^{\mathrm{th}}$ approximation to the solution $\alpha$ from applying Newton's Method. Furthermore, let $\delta^{(k)} = \begin{bmatrix} x_1 - x_1^{(k)} \\ x_2 - x_2^{(k)} \\ \vdots \\ x_n - x_n^{(k)} \end{bmatrix}$.

Define the matrix $F$ applied approximation $x^{(k)}$ to be:

(2)
\begin{align} \quad F(x^{(k)}) = \begin{bmatrix} f_1(x_1^{(k)}, x_2^{(k)}, …, x_n^{(k)} \\ f_2(x_1^{(k)}, x_2^{(k)}, …, x_n^{(k)} \\ \vdots \\ f_n(x_1^{(k)}, x_2^{(k)}, …, x_n^{(k)} \end{bmatrix} \end{align}

And we will define $F'$ applied to our approximation $x^{(k)}$ to be the matrix of partial derivatives evaluated at the point $x^{(k)}$:

(3)
\begin{align} \quad F'(x^{(k)}) = \begin{bmatrix} \frac{\partial}{\partial x_1} f_1(x^{(k)}) & \frac{\partial}{\partial x_2} f_1(x^{(k)}) & \cdots & \frac{\partial}{\partial x_n} f_1(x^{(k)}) \\ \frac{\partial}{\partial x_1} f_2(x^{(k)}) & \frac{\partial}{\partial x_2} f_2(x^{(k)}) & \cdots & \frac{\partial}{\partial x_n} f_2(x^{(k)}) \\ \vdots & \vdots & \ddots & \vdots \\ \frac{\partial}{\partial x_1} f_n(x^{(k)}) & \frac{\partial}{\partial x_2} f_n(x^{(k)}) & \cdots & \frac{\partial}{\partial x_n} f_n(x^{(k)}) \end{bmatrix} \end{align}

The general iteration of Newton's Method is thus given by:

(4)
\begin{align} \quad F'(x^{(k)})\delta^{(k)} = -F(x^{(k)}) \end{align}

Furthermore, each successive approximation of the root $\alpha$ is given by:

(5)
\begin{align} \quad x^{(k+1)} = x^{(k)} + \delta^{(k)} \end{align}

An equivalent form for each successive approximative $x^{(k+1)}$ of $\alpha$ can be obtained by substituting $\delta^{(k)}$ for $\left [ F'(x^{(k)} \right ]^{-1}$ so then $x^{(k+1)} = x^{(k)} - \left [ F'(x^{(k)} \right ]^{-1} F(x^{(k)}$