### Newton's Method of Nonlinear Minimization

Newton's method [162],[166, p. 143] finds the minimum of a nonlinear (scalar) function of several variables by locally approximating the function by a quadratic surface, and then stepping to the bottom of that bowl'', which generally requires a matrix inversion. Newton's method therefore requires the function to be close to quadratic'', and its effectiveness is directly tied to the accuracy of that assumption. For smooth functions, Newton's method gives very rapid quadratic convergence in the last stages of iteration. Quadratic convergence implies, for example, that the number of significant digits in the minimizer approximately doubles each iteration.

Newton's method may be derived as follows: Suppose we wish to minimize the real, positive function with respect to . The vector Taylor expansion [546] of about gives

for some , where . It is now necessary to assume that . Differentiating with respect to , where is presumed to be minimized, this becomes

Solving for yields

 (8.13)

Applying Eq.(7.13) iteratively, we obtain Newton's method:

 (8.14)

where is given as an initial condition.

When the is any quadratic form in , then , and Newton's method produces in one iteration; that is, for every .

Next Section:
Semi-Implicit Methods
Previous Section:
Trapezoidal Rule