Tight interval inclusions with compensated algorithms

(1)

Tight interval inclusions with compensated algorithms

Stef Graillat & Fabienne Jézéquel

Sorbonne Université, Laboratoire d’Informatique de Paris 6 (LIP6), France

18th international symposium on Scientific Computing, Computer Arithmetic and Validated Numerics (SCAN 2018)

Tokyo, Japan, 10-15 September 2018

(2)

Introduction

Exascale barrier broken in June 2018: 1.8 10¹⁸ floating-point operations per second. (Oak Ridge National Laboratory, analysis of genomic data)

Increasing power of current computers

→ GPU accelerators, Intel Xeon Phi processors, etc.

Enable to solve more complex problems

→ Quantum field theory, supernova simulation, etc.

A high number of floating-point operations performed

→ Each of them can lead to a rounding error

⇒Need for accuracy and validation

(3)

Introduction

Exascale barrier broken in June 2018: 1.8 10¹⁸ floating-point operations per second. (Oak Ridge National Laboratory, analysis of genomic data)

Increasing power of current computers

→ GPU accelerators, Intel Xeon Phi processors, etc.

Enable to solve more complex problems

→ Quantum field theory, supernova simulation, etc.

A high number of floating-point operations performed

→ Each of them can lead to a rounding error

⇒Need for accuracy and validation

(4)

Key tools for accurate computation

fixed length expansions libraries: double-double (Briggs, Bailey, Hida, Li), quad-double (Bailey, Hida, Li)

arbitrary length expansions libraries: Priest, Shewchuk, Joldes-Muller-Popescu

arbitrary precision libraries: ARPREC, MPFR, MPIR

compensated algorithms(Kahan, Priest, Ogita-Rump-Oishi,...) based on EFTs (Error Free Transformations)

EFTs: properties and algorithms to compute the generated elementary rounding errors

Leta, b∈F, for the basic operation ◦ ∈ {+,−,×}, with rounding to nearest,

a◦b= fl(a◦b) +e with e∈F

(5)

Key tools for accurate computation

fixed length expansions libraries: double-double (Briggs, Bailey, Hida, Li), quad-double (Bailey, Hida, Li)

arbitrary length expansions libraries: Priest, Shewchuk, Joldes-Muller-Popescu

arbitrary precision libraries: ARPREC, MPFR, MPIR

compensated algorithms(Kahan, Priest, Ogita-Rump-Oishi,...) based on EFTs (Error Free Transformations)

EFTs: properties and algorithms to compute the generated elementary rounding errors

Leta, b∈F, for the basic operation ◦ ∈ {+,−,×}, with rounding to nearest,

a◦b= fl(a◦b) +e with e∈F

(6)

Numerical validation with interval arithmetic

Principle: replace numbers by intervals and compute.

Fundamental theorem of interval arithmetic: the exact result belongs to the computed interval.

No result is lost, the computed interval is guaranteed to contain every possible result.

How to compute tight interval inclusions with compensated algorithms?

Assume floating-point arithmetic adhering to IEEE 754 with rounding unit u (no underflow nor overflow).

(7)

Numerical validation with interval arithmetic

(8)

Numerical validation with interval arithmetic

(9)

Outline

1 Error-free transformations (EFT) with rounding to nearest

2 Error-free transformations (EFT) with directed rounding

3 Compensated algorithm for summation with directed rounding

4 Compensated dot product with directed rounding

5 Compensated Horner scheme with directed rounding

(10)

Outline

(11)

EFT for addition

x=a⊕b ⇒ a+b =x+y with y∈F Algorithm of Dekker (1971) and Knuth (1974)

Algorithm (EFT of the sum of 2 floating-point numbers with |a| ≥ |b|)

function[x, y] =FastTwoSum(a, b) x=a⊕b

y= (a x)⊕b

Algorithm (EFT of the sum of 2 floating-point numbers)

function[x, y] =TwoSum(a, b) x=a⊕b

z =x a

y= (a (x z))⊕(b z)

(12)

EFT for the product (1/3)

x=a⊗b ⇒ a×b =x+y with y∈F AlgorithmTwoProduct by Veltkamp and Dekker (1971)

a=x+y and x and y non overlapping with |y| ≤ |x|.

Algorithm (Error-free split of a floating-point number into two parts)

function[x, y] =Split(a)

factor = 2^s+ 1 %u = 2^−p , s=dp/2e c=factor⊗a

x=c (c a) y=a x

(13)

EFT for the product (2/3)

Algorithm (EFT of the product of 2 floating-point numbers)

function[x, y] =TwoProduct(a, b) x=a⊗b

[a₁, a₂] = Split(a) [b₁, b₂] =Split(b)

y = a₂⊗b₂ (((x a₁⊗b₁) a₂⊗b₁) a₁⊗b₂)

(14)

EFT for the product (3/3)

x=a⊗b ⇒ a×b =x+y with y∈F Givena, b, c∈F,

FMA(a, b, c) is the nearest floating-point number to a×b+c

function[x, y] =TwoProdFMA(a, b) x=a⊗b

y=FMA(a, b,−x)

FMAis available for example on PowerPC, Itanium, Cell, Xeon Phi, AMD and Nvidia GPU, Intel (Haswell), AMD (Bulldozer)

processors.

(15)

Outline

(16)

EFT for addition with directed rounding

x= fl_*(a+b) ⇒ a+b=x+e but possibly e /∈F

Algorithm (EFT of the sum of 2 floating-point numbers with |a| ≥ |b|)

function[x, y] =FastTwoSum(a, b) x= fl_*(a+b)

y= fl_*((a−x) +b)

Proposition

We have y= fl_*(e) and so |e−y| ≤2u|e|. It yields |e−y| ≤4u²|x|

and |e−y| ≤4u²|a+b|. Moreover if ∗= ∆, e≤y

if ∗=∇, y≤e

(17)

EFT for addition with directed rounding

x= fl_*(a+b) ⇒ a+b=x+e but possibly e /∈F

Algorithm (EFT of the sum of 2 floating-point numbers)

function[x, y] =TwoSum(a, b) x= fl_*(a+b)

z = fl_*(x−a)

y= fl_*((a−(x−z)) + (b−z))

Proposition

We have |e−y| ≤4u²|a+b| and |e−y| ≤4u²|x|. Moreover if ∗= ∆, e≤y

if ∗=∇, y≤e

(18)

EFT for the product with directed rounding

x= fl_*(a×b) ⇒ a×b=x+y with y∈F Givena, b, c∈F,

FMA(a, b, c) is the nearest floating-point number to a×b+c

function[x, y] =TwoProdFMA(a, b) x= fl_*(a×b)

y=FMA(a, b,−x)

(19)

EFT for the product with directed rounding

a =x+y and xand y non overlapping with |y| ≤ |x|

Algorithm (Error-free split of a floating-point number into two parts)

function[x, y] =Split(a)

factor = 2^s+ 1 %u = 2^−p , s=dp/2e c= fl_*(factor×a)

x= fl_*(c−(c−a)) y= fl_*(a−x)

Proposition

We have a=x+y. Moreover,

the significand of x fits in p−s bits;

the significand of y fits in s bits.

(20)

EFT for the product with directed rounding

x= fl_*(a×b) ⇒ a×b=x+e with e∈F

function[x, y] =TwoProduct(a, b) x= fl_*(a×b)

[a₁, a₂] = Split(a) ;[b₁, b₂] =Split(b)

y = fl_*(a₂×b₂−(((x−a₁×b₁)−a₂×b₁)−a₁×b₂))

Proposition

We have |e−y| ≤8u²|a×b| and |e−y| ≤8u²|x|. Moreover if ∗= ∆, e≤y

if ∗=∇, y≤e

(21)

Outline

(22)

Compensated algorithm for summation

Letp={pi} be a vector of n floating-point numbers.

Algorithm (Ogita, Rump, Oishi (2005))

functionres =CompSum(p) π1 =p1 ;σ1 = 0

for i= 2 :n

[π_i, q_i] =TwoSum(πi−1, p_i) σi = fl_*(σi−1+qi)

res= fl_*(π_n+σ_n)

Proposition

Let us suppose CompSum is applied, with directed rounding, to p_i ∈F, 1≤i≤n. Let s:=P

p_i and S :=P

|p_i|. If nu< ¹₂, then

|res−s| ≤2u|s|+ 2(1 + 2u)γ_n²(2u)S with γ_n(u) = nu 1−nu.

(23)

Compensated algorithm for summation

Algorithm (Tight inclusion using INTLAB)

s e t r o u n d ( -1)

Sinf = C o m p S u m p ( p ) s e t r o u n d (1)

Ssup = C o m p S u m p ( p )

Proposition

Letp= {p_i} be a vector of n floating-point numbers. Then we have

Sinf≤

n

X

i=1

p_i ≤Ssup.

(24)

Numerical experiments

10⁰ 10⁵ 10¹⁰ 10¹⁵ 10²⁰ 10²⁵ 10³⁰ 10³⁵ 10⁴⁰

Condition number 10^-20

10^-15 10^-10 10^-5 10⁰ 10⁵

Radius/midpoint

Conditioning and radius/midpoint

classic summation compensated summation

(25)

Outline

(26)

Compensated dot product

Algorithm (Ogita, Rump and Oishi 2005)

functionres=CompDot(x, y) [p, s] =TwoProduct(x₁, y₁) for i= 2 :n

[h, r] =TwoProduct(xi, yi) [p, q] =TwoSum(p, h) s= fl_*(s+ (q+r)) end

res= fl_*(p+s)

Proposition

Let x_i, y_i ∈F (1≤i≤n) and res the result computed by CompDot with directed rounding. If (n+ 1)u< ¹₂, then,

|res−x^Ty| ≤2u|x^Ty|+ 2γ_n+1² (2u)|x^T||y|.

(27)

Compensated dot product

Algorithm (Tight inclusion using INTLAB)

s e t r o u n d ( -1)

Dinf = C o m p D o t ( x , y ) s e t r o u n d (1)

Dsup = C o m p D o t ( x , y )

Proposition

Let x_i, y_i ∈F (1≤i≤n) be given. Then we have Dinf≤x^Ty≤Dsup.

(28)

Numerical experiments

10⁰ 10⁵ 10¹⁰ 10¹⁵ 10²⁰ 10²⁵ 10³⁰ 10³⁵ 10⁴⁰

10^-14 10^-12 10^-10 10^-8 10^-6 10^-4 10^-2 10⁰ 10²

Radius/midpoint

classic dot product compensated dot product

(29)

Outline

(30)

Compensated Horner scheme

Letp(x) =

n

X

i=0

aixⁱ with x, ai ∈F

Algorithm (Graillat, Langlois, Louvet, 2009)

functionres=CompHorner(p, x) sn =an

r_n= 0

for i=n−1 :−1 : 0

[pi, πi] =TwoProduct(si+1, x) [s_i, σ_i] =TwoSum(p_i, a_i) r_i = fl_*(r_i+1×x+ (π_i +σ_i)) end

res= fl_*(s₀+r₀)

(31)

Compensated Horner scheme

Theorem

Consider a polynomial p of degree n with floating-point coefficients, and a floating-point value x. With directed rounding, the forward error in the compensated Horner algorithm is such that

|CompHorner(p, x)−p(x)| ≤2u|p(x)|+ 2γ_2n+1(2u)²p(|x|),e with ep(x) = Pn

i=0|a_i|xⁱ.

(32)

Compensated Horner scheme

Algorithm (x ≥ 0, Tight inclusion using INTLAB)

s e t r o u n d ( -1)

Einf = C o m p H o r n e r ( p , x ) s e t r o u n d (1)

Esup = C o m p H o r n e r ( p , x )

Ifx≤0, CompHorner(¯p,−x) is computed with p(x) =¯ Pn

i=0a_i(−1)ⁱxⁱ.

Proposition

Consider a polynomial p of degree n with floating-point coefficients, and a floating-point value x.

Einf≤p(x)≤Esup.

(33)

Numerical experiments

10⁰ 10⁵ 10¹⁰ 10¹⁵ 10²⁰ 10²⁵ 10³⁰ 10³⁵ 10⁴⁰

10^-15 10^-10 10^-5 10⁰ 10⁵

Radius/midpoint

Classic Horner Compensated Horner

(34)

Conclusion and future work

Conclusion

Compensated methodsare a fast way to get accurate results They can be used efficiently with interval arithmetic to obtain certified results with finite precision

Future work

Interval computations with K-fold compensated algorithms using Priest’s EFT and FMA

(35)

References I

[1] S. Boldo, S. Graillat, and J.-M. Muller.

On the robustness of the 2Sum and Fast2Sum algorithms.

ACM Trans. Math. Softw., 44(1):4:1–4:14, July 2017.

[2] S. Graillat, F. Jézéquel, and R. Picot.

Numerical validation of compensated summation algorithms with stochastic arithmetic.

Electronic Notes in Theoretical Computer Science, 317:55–69, 2015.

[3] S. Graillat, F. Jézéquel, and R. Picot.

Numerical validation of compensated algorithms with stochastic arithmetic.

Applied Mathematics and Computation, 329:339 – 363, 2018.

(36)

References II

[4] S. Graillat, Ph. Langlois, and N. Louvet.

Algorithms for accurate, validated and fast polynomial evaluation.

Japan J. Indust. Appl. Math., 2-3(26):191–214, 2009.

[5] T. Ogita, S. M. Rump, and S. Oishi.

Accurate sum and dot product.

SIAM Journal on Scientific Computing, 26(6):1955–1988, 2005.

[6] D.M. Priest.

On Properties of Floating Point Arithmetics: Numerical Stability and the Cost of Accurate Computations.

PhD thesis, Mathematics Department, University of California, Berkeley, CA, USA, November 1992.

(37)

Tight interval inclusions with compensated algorithms