One of the most frequently occurring problems in scientific work is to find the roots of equations of the form
(3.1) i.e., zeros of the function f(x). The function f(x) may be given explicitly, as, for example, a polynomial in x or as a transcendental function.
Frequently, however, f(x) may be known only implicitly; i.e., a rule for evaluating f(x) for any argument may be known, but its explicit form is unknown. Thus f(x) may represent the value which the solution of a differential equation assumes at a specified point, while x may represent an initial condition of the differential equation. In rare cases it may be possible to obtain the exact roots of (3.1), an illustration of this being a factorable polynomial. In general, however, we can hope to obtain only approximate solutions, relying on some computational technique to pro-duce the approximation. Depending on the context, “approximate solu-tion” may then mean either a point x*, for which (3.1) is “approximately satisfied,” i.e., for which |f(x*)| is “small,” or a point x* which is “close to” a solution of (3.1). Unfortunately the concept of an “approximate solution” is rather fuzzy. An approximate solution obtained on a computer will almost always be in error due to roundoff or instability or to the particular arithmetic used. Indeed there may be many “approximate solu-tions” which are equally valid even though the required solution is unique.
72
THE SOLUTION OF NONLINEAR EQUATIONS 73 To illustrate the uncertainties in root finding we exhibit below in Fig. 3.1 a graph of the function
This function has of course the single zero x = 1. A FORTRAN program was written to evaluate p6(x) in its expanded form. This program was used to evaluate p6(x) at a large number of points x1 < x2 < · · · < xN near x = 1 on a CDC 6500 computer. A Calcomp plotter was then used to produce the piecewise straight-line graph presented in Fig. 3.1. From the graph we see that p6(x) has many apparent zeros since it has many sign changes. These apparent zeros range from 0.994 to 1.006. Thus use of the expanded form of p6(x) to estimate the zero at x = 1 leads to apparently acceptable estimates which are correct to only 2 decimal digits, even though the CDC 6500 works in 14-digit floating-point arithmetic. The reason for this behavior can be traced to round-off error and significant-digit cancellation in the FORTRAN calculation of P6(x). This example illustrates some of the dangers in root finding.
Figure 3.1
In the remainder of this chapter we shall consider various iterative methods for finding approximations to simple roots of (3.1). Special attention will be given to polynomial equations because of their impor-tance in engineering applications.
3.1 A SURVEY OF ITERATIVE METHODS
In this section, we introduce some elementary iterative methods for finding a solution of the equation
(3-1) and illustrate their use by applying them to the simple polynomial equation (3.2) for which f(x) = x3 - x - 1.
For this example, one finds that
(3.3) Hence, since f(x) is continuous, f(x) must vanish somewhere in the interval [1,2], by the intermediate-value theorem for continuous functions (see Sec.
1.7). If f(x) were to vanish at two or more points in [1,2], then, by Rolle’s theorem (see Sec. 1.7), f’(x) would have to vanish somewhere in [1,2].
Hence, since f’(x) = 3x2 - 1 is positive on [1,2], f(x) has exactly one zero in the interval [1,2]. If we call this zero then
To find out more about this zero, we evaluate f(x) at the midpoint 1.5 of the interval [1,2] and get
Hence we now know that the zero lies in the smaller interval [1, 1.5]; i.e.,
Checking again at the midpoint 1.25, we find
and know therefore that lies in the yet smaller interval [1.25, 1.5]; i.e.,
This procedure of locating a solution of the equation f(x) = 0 in a sequence of intervals of decreasing size is known as the bisection method.
3.1 A SURVEY OF ITERATIVE METHODS 75 Algorithm 3.1: Bisection method Given a function f(x) continuous on the interval [a0, b0] and such that f(a0)f(b0) < 0.
We shall frequently state algorithms in the above concise form. For students familiar with the ALGOL language, this notation will appear quite natural. Further, we have used here the phrase “until satisfied” in order to stress that this description of the algorithm is incomplete. A user of the algorithm must specify precise termination criteria. These will depend in part on the specific problem to be solved by the algorithm. Some of the many possible termination criteria are discussed in the next section.
At each step of the bisection algorithm 3.1, the length of the interval known to contain a zero of f(x) is reduced by a factor of 2. Hence each step produces one more correct binary digit of the root of f(x) = 0. After 20 steps of this algorithm applied to our example and starting as we did with a, = 1, b0 = 2, one gets
Clearly, with enough effort, one can always locate a root to any desired accuracy with this algorithm. But compared with other methods to be discussed, the bisection method converges rather slowly.
One can hope to get to the root faster by using more fully the information about f(x) available at each step. In our example (3.2), we started with the information
Since |f(l)| is closer to zero than is |f(2)| the root is likely to be closer to 1 than to 2 [at least if f(x) is “nearly” linear]. Hence, rather than check the midpoint, or average value, 1.5 of 1 and 2, we now check f(x) at the weighted average
(3.4) Note that since f(1) and f(2) have opposite sign, we can write (3.4) more simply as
(3.5)
This gives for our example
. . . .
and
Hence lies in [1.166666 · · · , 2]. Repeating the process for this interval, we get
Consequently, f(x) has a zero in the interval [1.253112 · · · , 2]. This algorithm is known as the regula falsi, or false-position, method.
Algorithm 3.2: Regula falsi Given a function f(x) continuous on the interval [a0, b0] and such that f(a0)f(b0) < 0.
After 16 steps of this algorithm applied to our example and starting as we did with a0 = 1, b0, = 2, one gets
Hence, although the regula falsi produces a point at which |f(x)| is “small”
somewhat faster than does the bisection method, it fails completely to give a “small” interval in which a zero is known to lie.
A glance at Fig. 3.2 shows the reason for this. As one verifies easily, the weighted average
is the point at which the straight line through the points {an, f(an)} and {bn, f(bn)} intersects the x axis. Such a straight line is a secant to f(x), and in our example, f(x) is concave upward and increasing (in the interval [1,2] of interest); hence the secant is always above (the graph of) f(x).
Consequently, w always lies to the left of the zero (in our example). If f(x) were concave downward and increasing, w would always lie to the right of the zero.
3.1 A SURVEY OF ITERATIVE METHODS 77
Figure 3.2 Regula falsi.
The regula falsi algorithm can be improved in several ways, two of which we now discuss. The first one, called modified regula falsi, replaces secants by straight lines of ever-smaller slope until w falls to the opposite side of the root. This is shown graphically in Fig. 3.3.
Algorithm 3.3: Modified regula falsi Given f(x) continuous on [a0, b0] and such that f(a0)f(b0) < 0.
If the modified regula falsi is applied to our example with a0 = 1, b0 = 2, then after six steps, one gets
which shows an impressive improvement over the bisection method.
Figure 3.3 Modified rcgula falsi.
A second, very popular modification of the regula falsi, called the secant method, retains the use of secants throughout, but may give up the bracketing of the root.
Algorithm 3.4: Secant method Given a function f(x) and two points x-1, x0.
If the second method is applied to our example with x-1 = 1, x0 = 2, then after six steps one gets
Apparently, the secant method locates quite rapidly a point at which |f(x)|
is “small,” but gives, in general, no feeling for how far away from a zero of f(x) this point might be. Also, f(xn) and f(xn-1) need not be of opposite sign, so that the expression
(3.6) is prone to round-off-error effects. In an extreme situation, we might even have f(xn) = f(xn-1), making the calculation of xn+1 impossible. Although this does not cure the trouble, it is better to calculate xn+1 from the
3.1 A SURVEY OF ITERATIVE METHODS 79 equivalent expression
in which xn+1 is obtained from xn by adding the “correction term”
(3.7)
(3.8) The student will recognize the ratio [f(xn) - f(xn-1)]/(xn - xn-1) as a first divided difference of f(x) and from (2.10) as the slope of the secant to f(x) through the points {xn-1, f(xn-1)} and {xn, f(xn)}. Furthermore from (2.17) we see that this ratio is equal to the slope of f(x) at some point between xn-1 and xn if f(x) is differentiable. It would be reasonable therefore to replace this ratio by the value of f’(x) at some point “near” xn and xn-1, given that f’(x) can be calculated.
If f(x) is differentiable, then on replacing in (3.7) the slope of the secant by the slope of the tangent at xn, one gets the iteration formula
(3.9) of Newton’s method.
Algorithm 3.5: Newton’s method Given f(x) continuously differentiable and a point x0.
If this algorithm is applied to our example with x0 = 1, then after four steps, one gets
Finally, we mention fixed-point iteration, of which Newton’s method is a special example. If we set
(3.10) then the iteration formula (3.9) for Newton’s method takes on the simple form
(3.11) If the sequence x1, x2, · · ·
g(x) is continuous, then
so generated converges to some point and (3.12)
o r that is, is then a fixed point of g(x). Clearly, if is a fixed point of the iteration function g(x) for Newton’s method, then is a solution of the equation f(x) = 0. Now, for a given equation f(x) = 0, it is possible to choose various iteration functions g(x), each having the prop-erty that a fixed point of g(x) is a zero of f(x). For each such choice, one may then calculate the sequence x1, x2, . . . by
and hope that it converges. If it does, then its limit is a solution of the equation f(x) = 0. We discuss fixed-point iteration in more detail in Secs.
3.3 and 3.4.
Example 3.1 The function f(x) = x - 0.2 sin x - 0.5 has exactly one zero between x0 - 0.5 and xl - 1.0, since f(0.5)f(l.0) < 0, while f’(x) does not vanish on [0.5, 1].
Locate the zero correct to six significant figures using Algorithms 3.1, 3.3, 3.4, and 3.5.
The following calculations were performed on an IBM 7094 computer in single-precision 27-binary-bit floating-point arithmetic.
In Algorithms 3.1 and 3.3, x, is the midpoint between the lower and the upper bounds, an and bn, after n iterations, while the gives the corresponding bound on the error in xn provided by the algorithm. Note the rapid and systematic convergence of Algorithms 3.4 and 3.5. The bisection method converges very slowly but steadily, while the modified regula falsi method seems to converge “in jumps,” although it does obtain the correct zero rather quickly.
EXERCISES
3.1-1 Find an interval containing the real positive zero of the function f(x) = x2 - 2x - 2.
Use Algorithms 3.1 and 3.2 to compute this zero correct to two significant figures. Can you estimate how many steps each method would require to produce six significant figures?
3.1-2 For the example (Algorithm 3.3).
3.2
given in the text, carry out two steps of the modified regula falsi FORTRAN PROGRAMS FOR SOME ITERATIVE METHODS 81
3.1-3 The polynomial x3 - 2x - 1 has a zero between 1 and 2. Using the secant method (Algorithm 3.4), find this zero correct to three significant figures.
3.1-4 In Algorithm 3.1 let M denote the length of the initial interval [a0, b0]. Let {x0, x1, x2, . . . } represent the successive midpoints generated by the bisection method. Show that
Also show that the number an accuracy is given by
I of iterations required to guarantee an approximation to a root to
3.1-5 The bisection method can be applied whenever f(a)f(b) < 0. If f(x) has more than one zero in (a, b), which zero does Algorithm 3.1 usually locate?
3.1-6 With a = 0, b = 1, each of the following functions changes sign in (a, b), that is, f(a)f(b) < 0. What point does the bisection Algorithm 3.1 locate? Is this point a zero of f(x)?
3.1-7 The function f(x) = e2x - ex - 2 has a zero on the interval [0,1]. Find this zero correct to four significant digits using Newton’s method (Algorithm 3.5).
3.1-8 The function f(x) = 4 sin x - ex has a zero on the interval [0, 0.5]. Find this zero correct to four significant digits using the secant method (Algorithm 3.4).
3.1-9 Using the bisection algorithm locate the smallest positive zero of the polynomial p(x) = 2x3 - 3x - 4 correct to three significant digits.
3.2 FORTRAN PROGRAMS FOR SOME ITERATIVE METHODS
When the algorithms introduced in the preceding section are used in calculations, the vague phrase “until satisfied” has to be replaced by precise termination criteria. In this section, we discuss some of the many possible ways of terminating iteration in a reasonable way and give translations of Algorithms 3.1 and 3.3, into FORTRAN.