High order algorithms to find square roots and inverses

1  Introduction

Inverting a number of size n can be done in time O(n2) with the classical school division. This time cost can be reduced to O(nlog(n)) when using a fast multiplication, thanks to Newton's iteration. The same type of techniques apply to the n-th root computation of a number. For those operations we also describe some very efficient algorithms with high order of convergence (that is quartic, quintic, sextic, ... algorithms). Computing square roots and inverses to a great accuracy is also useful to compute many usual mathematical functions like logarithms and trigonometric functions (see Brent's remarkable paper [2]) or to compute constants like p,log(2),...([2], [6]).

2  High precision inversion

In this section we want to compute the inverse 1/A of the number A to a great accuracy.

2.1  Newton's iteration

Starting from a few digits approximation x0 of 1/A, we apply Newton's iteration to the function f(x) = 1/x-A, which gives the approximating sequence

xn+1 = xn- f(xn)
f(xn)
= 2xn-Axn2.     (*)
It's a property of Newton iteration to converge quadratically near the root, that is the number of right digits doubles at each iteration. Thus, a number of about log(n) iterations suffices to have n digits of 1/A.

The iteration (*) have a nice feature : the only needed operations are the product by 2, the difference and the product of large numbers. The Newton convergence being quadratic, the precision needed to compute iteration (*) is not the full final precision but only with the precision wanted at step n .

As an example, we compute the inverse of p, starting from the approximation x0 = 0.31831 (5 digits). The first three iterations become

x1
=
2x0-px02 = 0.3183098861 (operations done with 10 digits of precision)
x2
=
2x1-px12 = 0.31830988618379067151 (operations done with 20 digits of precision)
x3
=
2x2-px22 = 0.3183098861837906715377675267450287240665 (operations done with 40 digits)

The first 38 digits of x3 agree with those of 1/p.

The cost of this process is of the order of the multiplication cost : since each iteration involves two multiplications on high precision numbers (that is xn×xn and A×xn2), to find 1/A we need to compute 2 multiplications of size n (last step), 2 multiplications of size n/2 (previous step), 2 multiplications of size n/4, etc. Using for example an FFT multiplication, with cost O(nlogn), the final cost is of order

2nlog(n)+2 n
2
log

n
2


+2 n
4
log

n
4


+ 2n

1+ 1
2
+ 1
4
+

log(n) = 4nlog(n).

Note : a better iteration is obtained by writing (*) in the form

hn
=
1-Axn
xn+1
=
xn+xnhn.
Since cancellations occurs in hn = 1-Axn, the second multiplication xnhn can be done by taking into account only the non zero part of hn. This has also another advantage : the number of vanishing terms in front of hn gives the approximation precision of xn.

A division B/A is performed by multiplying B by the inverse of A.

A practical example of inverse of large numbers from FFT multiplication can be found as an easy C sample code at  Easy programs for constants computation.

2.2  A cubic iteration

From the general Householder's iteration ([3] and [1] p. 216)

xn+1 = xn- f(xn)
f(xn)


1+ f(xn)f(xn)
2f(xn)2


,

applied on f(x) = 1/x-A, we find

hn
=
1-Axn
xn+1
=
xn+xn(hn+hn2)

Starting with a good initial point x0, the convergence is cubical, that is, the number of good digits is multiplied by 3 at each step of the process (A similar form of this iteration is given in [1] p. 216 and [4] p. 279). Since hn tends to zero, the new term hn2 can be computed at a lower cost.

As an example, we compute again, the inverse of p, starting from the approximation x0 = 0.31831 (5 digits). The first two iterations are

x1
=
0.3183098861837906715(523...), (19 good digits)
x2
=
0.3183098861837906715377675267450287240689...,(58 good digits)

and x3 has 174 good digits...

2.3  High order iterations


   Quartic iteration  

It's possible to find a quartic iteration, just apply twice Newton's iteration on f(x) = 1/x-A, we find

hn
=
1-Axn
xn+1
=
xn+xn( hn+hn2+hn3) .

If we take a look at our example (inverse of p), starting as usual with x0 = 0.31831, we find that x1 has 26 good digits, x2 is 103 digits exact, x3 has 413 good digits ... The sequence of correct digits is then for this example

26,103,413,1650,6601,26405,...


   Quintic iteration  

A quintic iteration founded by the application of a general quintic modified iteration (here, we omit the details of the demonstration which is deduced from our essay on Newton's methods) is given by

hn
=
1-Axn
xn+1
=
xn+xn( hn+hn2+hn3+hn4) = xn+xn( 1+hn2) ( hn+hn2) ,

the rate of convergence is impressive, again on the previous example, x1 has 32 correct digits and x2 has 161 good digits, x3 has 806 good digits ... The second factorization of this iteration requires only 4 multiplications (A×xn,hn×hn,(1+hn2)×(hn+hn2),xn×(1+hn2)(hn+hn2)) at each step!


   Higher order  

The inverse of a number can be computed with an iteration at any desired order, it's possible to show that the general algorithm of order r is given by

hn
=
1-Axn
xn+1
=
xn+xnPr-1(hn),

where Pm(u) is a polynomial of degree m, given by

Pm(u)
=
m

k = 1 
akuk
ak
=
1.

For example, for m = 3,m = 4, m = 5, m = 6 and m = 7 (respectively quartic, quintic, sextic, septic and octic iterations)

P3(u)
=
u+u2+u3,
P4(u)
=
u+u2+u3+u4 = (1+u2)(u+u2),
P5(u)
=
u+u2+u3+u4+u5 = u+(1+u)(u2+u4),
P6(u)
=
u+u2+u3+u4+u5+u6 = (1+u2+u4)(u+u2),
P7(u)
=
u+u2+u3+u4+u5+u6+u7 = u+(1+u3)(u2+u3+u4)
...

In practical cases the number hn tends to zero, therefore the evaluation of the powers hnk is more and more efficient when k increases. Observe that some of the given factorizations may reduce the number of required multiplications. A careful implementation of those high order iterations may be very efficient.

3  Square roots

The same technique as for the inverses applies for square roots.

3.1  Newton's iteration

Using the function f(x) = x2-A, Newton iteration gives

xn+1 = xn- f(xn)
f(xn)
= xn
2
+ A
2xn
.
(In other words, the iteration is the mean value of xn and A/xn).

This iteration has one disadvantage : one should compute the division A/xn. When one wants to compute 1/A, a better iteration is obtained from the function f(x) = 1/x2-A, yielding the iteration

xn+1 = 3
2
xn- 1
2
Axn3.     (**)
This formula only requires multiplications ! An easier way of computing a square root A is then to compute B = 1/A from this latest process and then to perform the product A×B = A.

Note : as for the inverse, the best way to compute the iteration (**) is to write it in the form

hn
=
1-Axn2
xn+1
=
xn+ xn
2
hn,
since cancellations occur in hn = 1-Axn2.

3.2  A cubic iteration

Again, from the Householder's iteration, applied on the function f(x) = 1/x2-A, we find after some easy manipulations

xn+1 = xn
8
(15-10Axn2+3A2xn4).

This process converges cubically to the inverse of the square root of the number A (This form of the iteration is also given in [1] p. 217). Just like for the quadratic iteration, a more efficient way is to write it

hn
=
1-Axn2
xn+1
=
xn+xn

1
2
hn+ 3
8
hn2

= xn+ xn
8
( 4hn+3hn2) .

3.3  High order iterations

Again like for the inverse, it's possible to find a quartic iteration, just apply twice Newton's iteration on f(x) = 1/x2-A :

xn+1 = xn- xn
16
(1-Axn2)(-20+19Axn2-8A2xn4+A3xn6).

This algorithm converges quartically to 1/A.

It is interesting to compare this result to the direct application of the general quartic modified iteration to our function (see the essay on Newton's methods)

xn+1 = xn- xn
16
(1-Axn2)(-19+16Axn2-5A2xn4).

This simpler relation suggests that it may be more efficient to use the quartic iteration than twice the Newton iteration. The best form of this quartic algorithm is

hn
=
1-Axn2
xn+1
=
xn+ xn
16
( 8hn+6hn2+5hn3) .

As an application of the modified iterations, we provide the general expression of some high order iterations to compute square roots. The general pattern of a high order iteration of order r is given by

hn
=
1-Axn2
xn+1
=
xn+xnPr-1(hn)

with Pm(u) being a polynomial of degree m with rational coefficients. The first polynomials are given with factorization to save some multiplications, the best choice depends on your implementation.


   Quintic iteration  

P4(u)
=
1
128
( 64u+48u2+40u3+35u4)
=
1
128
( 64u+u2( 48+40u+35u2) ) .


   Sextic iteration  

P5(u)
=
1
256
( 128u+96u2+80u3+70u4+63u5)
=
1
256
( 128u+96u2+u3( 80+70u+63u2) )


   Septic iteration  

P6(u)
=
1
1024
(512u+384u2+320u3+280u4+252u5+231u6)
=
1
1024
( 512u+384u2+320u3+u3(280u+252u2+231u3) )


   Octic iteration  

P7(u)
=
1
2048
(1024u+768u2+640u3+560u4+504u5+462u6+429u7)
=
1
2048
(1024u+768u2+640u3+560u4+u4(504u+462u2+429u3))

For example to evaluate P7(u), you may compute in this order

u×u,u×u2,u×u3,u4(504u+462u2+429u3)

that is, four large multiplications and don't forget that, in our cases, u is small so that those multiplications are even faster to perform.


   Expression of the polynomials Pm(u)  

The coefficients of the polynomials Pm(u) are in fact given by the series expansion of

1
  ___
1-u
-1
=


k = 1 
akuk
ak
=
2(2k-1)!
k!(k-1)!4k

this allow to define easily any order algorithm to compute square roots.

4  m-th roots

The previous techniques may be used in order to compute the m-th root (m being an integer) of the number A. Therefore, to find with a great accuracy the number A-1/m, we use Newton's method on the function f(x) = 1/xm-A and this will produce the process

hn
=
1-Axnm
xn+1
=
xn+xn hn
m
which converges quadratically to A-1/m. To find A1/m just compute at the end of the N previous iterations
yN = AxNm-1.

Householder's iteration also becomes

hn
=
1-Axnm
xn+1
=
xn+xn

1
m
hn+ 1+m
2m2
hn2

which converges cubically to A-1/m.

4.1  High order iteration

The general pattern for any order iteration is given by

hn
=
1-Axnm
xn+1
=
xn+xnP(hn)

and to find P(u) you need to compute the following series expansion

(1-u)-1/m-1 = 1
m
u+ 1+m
2m2
u2+ (1+m)(1+2m)
3!m3
u3+...

The truncated series at order k will provide an algorithm of order k+1. Observe that for the value m = 1, we retrieve the polynomials used to compute the inverse of a number and for m = 2, it gives back the algorithms to find square roots.

4.1.1  Example

For m = 4 the following algorithm converges quartically to A-1/4

hn
=
1-Axn4
xn+1
=
xn+xn

1
4
hn+ 5
32
hn2+ 15
128
hn3

and A1/4 is given by

Axn3.

References

[1]
J.M. Borwein and P.B. Borwein, ''Pi and the AGM - A study in Analytic Number Theory and Computational Complexity'', A Wiley-Interscience Publication, New York, (1987)

[2]
R.P. Brent, Fast multiple-Precision evaluation of elementary functions, J. Assoc. Comput. Mach., (1976), vol. 23, p. 242-251

[3]
A. S. Householder, The Numerical Treatment of a Single Nonlinear Equation, McGraw-Hill, New York, (1970)

[4]
D.E. Knuth, The Art of Computer Programming, Vol. II, Seminumerical Algorithms, Addison Wesley, (1998).

[5]
I. Newton, Methodus fluxionum et serierum infinitarum, (1664-1671)

[6]
E. Salamin, Computation of p Using Arithmetic-Geometric Mean, Mathematics of Computation, (1976), vol. 30, p. 565-570 This is why it's important to find such algorithms.


File translated from TEX by TTH, version 2.32.
On 5 Feb 2001, 19:06.