An explicit formulation of the multiplicative Schwarz preconditionner

(1)

HAL Id: inria-00000266

https://hal.inria.fr/inria-00000266

Submitted on 20 Sep 2005

HAL is a multi-disciplinary open access

archive for the deposit and dissemination of

sci-entific research documents, whether they are

pub-L’archive ouverte pluridisciplinaire HAL, est

destinée au dépôt et à la diffusion de documents

scientifiques de niveau recherche, publiés ou non,

preconditionner

Guy Atenekeng Kahou, Emmanuel Kamgnia, Bernard Philippe

To cite this version:

Guy Atenekeng Kahou, Emmanuel Kamgnia, Bernard Philippe. An explicit formulation of the

mul-tiplicative Schwarz preconditionner. [Research Report] PI 1738, 2005, pp.25. �inria-00000266�

(2)

I

R

I

S

A

I

N

S

T

IT

U

T

D

E

R

E

C

H

E

R

C

H

E

N

I

N

F

O

R

M

A

T

I

Q

U

E

T

S

YS

T

È

M

ES

A

LÉ

A

T

O

IRE

S

P

U B L I C A T I O N

I N T E R N E

N

o

1738

AN EXPLICIT FORMULATION OF THE MULTIPLICATIVE

SCHWARZ PRECONDITIONNER

GUY ATENEKENG KAHOU , EMMANUEL KAMGNIA ,

BERNARD PHILIPPE

(3)

(4)

Tél. : (33) 02 99 84 71 00 – Fax : (33) 02 99 84 71 71

http://www.irisa.fr

An expli it formulation of the multipli ative S hwarz

pre onditionner

GuyAtenekeng Kahou

* , EmmanuelKamgnia ** ,Bernard Philippe *** Theme | ProjetSAGE

Publi ationinternen1738|Juillet2005|25pages

Abstra t: Weprovideanexpli itformulationofthesplittingasso iatedwiththe Multi-pli ativeS hwarziteration. Weshowtheadvantageof onsideringtheexpli itformulation, whentheiterationisusedasapre onditionerofaKrylovmethod.

Key-words: Domain de omposition, Multipli ative S hwarz, pre onditionner, Krylov methods,Red-Bla k oloring,iterativemethods.

(Resume: tsvp) ThisworkwassupportedbyAUF(Agen eUniversitairedelaFran ophonie)forvisitingLAMSIN/ENIT atTunisandbytheFren hministryofForeignAairsunderSARIMAprogramforvisitingINRIA/IRISA atRennes. * gaatenekirisa.fr ** kamgniauy d .uninet. m *** philippeirisa.fr

(5)

Resume: Apartird'uneexpressionexpli itedusplittingdeniparl'iteration multipli a-tivedeS hwarz,nousetudionssonutilisation ommepre ondtionnementd'unemethodede Krylov.

Mots les : De omposition de domaine, S hwarz multipli atif, pre onditionnement, methodesdeKrylov, oloriageRouge-Noir,methodesiteratives

(6)

oat

1 Introdu tion

Domain de ompositionprovidesa lassofdivide-and- onquermethods suitablefor the so-lution oflinear ornonlinear systemsofequations arisingfrom thedis retization of partial dierentialequations. Forlinearsystems,domainde omposition methods anbeviewedas pre onditionersforKrylovsubspa ete hniques.

As mentionedin [8℄,theterm domainde omposition hasslightlydierent meaningsto spe ialistsdependingontheirdis ipline: inparallel omputing,itoftenmeansthepro essof distributingdatafroma omputationalmodelamongthepro essorsinadistributedmemory omputer. Innumeri alanalysis,itmeanstheseparationofthephysi aldomainintoregions that anbemodeledwithdierentequations,withinterfa esbetweenthedomainshandled by various onditions. In pre onditioning methods, whi h is our interest in this arti le, domain de omposition refers to the pro ess of subdividing the solution of a large linear systemintosmaller problemswhose solutions anbeused toprodu eapre onditioner(or solver) for the system of equations that results from dis retizing the PDE on the entire domainormoregenerallyfromanysparsematrix. Inourwork,we onsiderthelatterand wesupposethatthedomainde omposition iswithoverlapping.

Traditionally, there are three lasses of iterative methods whi h derive from domain de omposition: AdditiveandMultipli ativeS hwarzforoverlappingssubdomainsandS hur omplementmethods for non-overlappingsubdomains. Whenusing theS hwarz methods assolvers, the onvergen e rates are veryslow and the onvergen e is mainly guarantied forsymmetri positivedenitematri esandM-matri es[3℄. Forthatreason,theparti ular interestofS hwarzmethodsisaspre onditionerofKrylovsubspa emethodssin ethey an beeÆ ientevenwhentheywould not onvergeasafullmethod.

Whenusedaspre onditioners,oneisinterestedinderivinganexpli itanduseful expres-sionofthepre onditioner. FortheAdditiveS hwarzmethodsu hanexpressionexists. To ourknowledge,noexpli itexpressionofthepre onditionnerisknownfortheMultipli ative S hwarzmethod. Inthis paperwederivesu hanexpression.

In se tion 2wesuppose that onegraph partitioner is applied resulting in subdomains withoverlaps. If thedomain de omposition were withoutoverlap,it isknown[4℄that the multipli ativeS hwarzalgorithmwould beequivalenttoaBlo kGauss Seideliteration. In se tion3,wederiveanexpli itformulationoftheMultipli ativeS hwarzpre onditioner. In se tion4,wedis usstheuseofsu hapre onditionnerwithaKrylovmethodandinse tion 5,weillustrate thebehaviourofthepre onditionner onsomenumeri altests.

(7)

2 Domain de omposition of a sparse matrix and nota-tions

Letus onsiderasparsematrixA2R nn

. ThepatternofAistheset P=f(k;l)ja k ;l

6=0g whi h istheset oftheedges ofthegraphG=(W;P) whereW =f1;:::;ng=[1:n℄isthe setofverti es.

Denition2.1 A domain de omposition of matrix A into p subdomains is dened by a olle tion ofsets ofintegers W

i W =[1:n℄,i=1;:::;psu hthat : ji jj>1=)W i \W j =;; P S p i=1 (W i W i ):

Followingthis denition, a domain de omposition anbe onsidered asresulting from a graph partitioner but with potential overlap between domains. It an be noti ed that su hade ompositiondoesnotne essarilyexist(e.gwhenAisadensematrixinwhi h ase thereisonlyonesubdomain). Fortherestofourdis ussion,weshallsupposethatagraph partitionerhasbeenappliedand hasresultedinpsets W

i

whoseunionis W,W =[1:n℄. Thesubmatrix ofA orrespondingtoW

i W i isdenotedbyA i We shall denote by L i = j2Wi (e j

) the ve tor spa e of R n

of all the ve tors with zero omponents for every index j 62 W

i . Let m i be the dimension of L i . The orthogonal proje tor onto L i

is dened by the sub-identity matrix I i

of order nn whose diagonal elementsaresetto oneifthe orrespondingnodebelongsto W

i

andtozerootherwise. We alsodenotebyA

i

theextensionofblo kA i

to thewhole spa e,therefore: A i = I i AI i : (1)

Finally,wedenethe omplementsub-identitymatrix I

i

=I I

i

andthematrix, A i = A i + I i : (2)

We assume thereafter that all the matri es A i

, for i = 1;;p are non singular. The generalizedinverseA + i ofA i satisesA + i =I i A 1 i = A 1 i I i .

Proposition 2.1 Forany domainde ompositionasdenedinDenition 2.1the following property istrue. 8 i;j2f1;:::;pg, ji jj>2)I i AI j =0: Proof. Let(k;l)2W i W j su h thata k ;l

6=0. Sin e(k;l)2P,there existsm2f1:::ng su h that k2W m andl2W m ; thereforeW i \W m 6=;andW j \W m 6=;. Consequently,

fromDenition 2.1,ji mj1andjj mj1,whi himpliesji jj2.

Letusintrodu easpe ial situationwhi hisoftensatisedandwhi h bringssome sim-pli ationinthesequel.

(8)

Denition2.2 The domain de omposition is with weak overlap if and only if, for any i; j2f1;:::;pgthe followingis true

ji jj>1)I i

AI j

=0:

Theset ofunknownswhi hrepresentstheoverlapisdenedby theset ofintegersJ i = W i \W i+1 ;i=1;:::;p 1,andlets i

bethedimensionof theoverlap. Similarlyto (1)and (2),wedene C i =O i AO i ; (3) and C i =C i + O i ; (4) where O i 2 R nn

is sub-identity matrix whose diagonal elements are set to one if the orrespondingnodebelongstoJ

i

andtozerootherwise,and O i =I O i .

Example2.1 Figure1displays anexampleofadomainde ompositionfor amatrixin the asewhereallW

i

areintervals of integers(A ij

denotesablo k).

Figure1: Amatrixdomain de ompositionwithblo koverlaps. W 1 =w 1 [w 2 [w 3 ,W 2 = w 3 [w 4 [w 5 ,W 3 =w 5 [w 6 [w 7 ,W 4 =w 7 [w 8 [w 9 ,where,fori=1;;9,w i istheset oftherowindi esofA

ii withC 1 =A 33 ,C 2 =A 55 andC 3 =A 77 .

There isa lose onne tionbetweenablo ktridiagonalstru tureandadomain de om-position. For instan e, in the previousexample, if, in order to transform A into a blo k

(9)

tridiagonalmatrix, we assumethat all theblo ksA i;i+2

and A

i+2;i

are zeros,the domain de omposition is obtained bydening the domains with three onse utive blo ksand the overlaps orrespondto onlyoneblo k. One aneasilyverifythat su hanoverlapisaweak overlap. Ifthedomainsweredenedwith onlytwoblo ksand thedomain overlapsonone blo k,theoverlapwould notbeweak.

In the denitions, the set of integers dening the subdomains are not ne essarily in-tervalsalthough this isoften the situation asin the example. However,when onsidering othersituationslikeared-bla kblo kordering, itisimportanttoin lude thegeneral ase. Nevertheless,itisalwayspossibletore overthespe ial asebyrenumberingtheunknowns.

(10)

3 Multipli ative S hwarz

ThegoalofMultipli ativeS hwarzmethodsistoiterativelysolvealinearsystem

Ax=b (5)

wherematrixAisde omposedintooverlappingsubdomainsasdes ribedintheprevious se -tion. Theiteration onsistsofsolvingtheoriginalequationinsequen eonea hsubdomain. Thisis awell-known method ;for moredetails,seefor instan e[3, 4, 6,7,8, 10℄). Inthis se tion, wepresentthemain propertiesof theiterationand deriveanexpli it formulation ofthe orrespondingmatrixsplitting.

3.1 Classi al formulation

Letx k

bethe urrent iterate and r k

=b Ax

k

the orresponding residual. The lassi al formulationofmultipli ativeS hwarzpro eeds asfollow.

Algorithm1 : One iteration of the Multipli ative S hwarz Pre onditioner builds p sub-iteratesandtheir orresponding residualsby the following re ursion:

input : x:=x k ; r:=r k ; fori=1:p x:=x+A + i r; r:=r AA + i r ; end output : x k +1 :=x ; r k +1 :=r; Itfollowsthat: r k +1 =(I AA + p ):::(I AA + 1 )r k : (6)

This method orrespondsto a relaxationiteration dened by somesplitting A = M N su h thattheiterationmatri esfortheresidualand theerrorarerespe tively

NM 1 = (I AA + p )(I AA + 1 )and (7) M 1 N = A 1 NM 1 A=(I A + p A)(I A + 1 A): (8)

The onvergen eofthisiterationisprovenforM-matri esands.p.d. matri es(eg. see[3℄). Now,letussuppose thatthegoalisto onsider anotheriterativemethodbut pre ondi-tionedbyonestepoftheMultipli ativeS hwarzmethod. Forthatpurpose,it isne essary todene y=M 1 Axory=AM 1 x

(11)

dependingonthesideofthepre onditionning,foranyve torx andwhereM isthematrix hara terizedbytheprevioussplitting. Fromtheexpression ofM

1 N andNM 1 we an deriveanexpressionofM 1 AorAM 1 asfollows: M 1 A = I (I A + p A)(I A + 1 A); (9) or AM 1 = I (I AA + p )(I AA + 1 ): (10)

3.2 Embedding in a system of larger dimension

If the subdomains do not overlap,it anbe shown [4℄ that the Multipli ative S hwarz is equivalenttoaBlo kGauss-Seidelmethodapplied onanextended system. Inthisse tion, following [9℄, we presentan extendedsystem whi h embeds theoriginal system (5)into a largeronewithnooverlappingbetweensubdomains.

Forthatpurpose,wedene theprolongationmappingandtherestri tionmapping. We assumeforthewhole se tionthat theset ofindi esdening thedomains areintervals. As mentionedbefore, thisdoesnotlimitthes opeof thestudy sin eapreliminarysymmetri permutation of thematrix, orresponding to the samerenumberingof the unknowns and theequations, analwaysendupwithsu hasystem.

Denition3.1 The prolongation mapping whi h inje ts R n into a spa e R m where m = P p i=1 m i =n+ P p 1 i=1 s i isdenedasfollows : D: R n ! R m x 7! ex;

where ex isobtained from ve tor x by dupli ating all the blo ks of entries orresponding to overlappingblo ks.

The restri tionmapping onsistsof proje ting ave torex2R m

ontoR n

,whi h onsists ofdeleting the subve tors orresponding tothe rstappearen eof ea hoverlapping blo ks

P : R m ! R n e x 7! x:

Embeddingtheoriginalsystemin alargeroneisdoneforinstan ein[4,9℄. Wepresent here a spe ial ase. In order to avoid a tedious formal presentation of the augmented system,wepresentits onstru tion onan examplewhi h isgeneri enoughto understand the denition of

e

A . In Figure 2, is displayed anexample with four domains. MappingD buildsx~bydupli ating someentries in ve torx : mappingx !ex=Dx expandsve torx to in lude subve torsy 3 , y 5 and y 7 . Theequalities y 3 =x 3 , y 5 =x 5 andy 7 =x 7 dene a subspa eJ of R m

. This subspa eis therange of mappingD. These equalities ombined with the denition of matrix

e

A show that J is an invariant subspa e of e A :

e

AJ J.

Therefore solving system Ax = b is equivalent to solving system e Aex = e b where e b = Db. OperatorP deletesentriesx

3 ,x 5 andx 7 fromve torx.~

(12)

Figure2: Denition of e

A(ExtensionofMatrixdisplayedinFig1).

Remark3.1 The following properties are straightforward onsequen es of the previous denitions: 1. Ax=P e ADx, 2. J =R(D)R m isaninvariant subspa eof e A, 3. PD=I n

andDP isaproje tion ontoJ, 4. 8x; y2R

n

,(y=Ax,Dy = e ADx). This an beillustratedby diagram(11) :

R n A ! R n D # " P R m e A ! R m : (11) One iteration of the Multipli ative S hwarz method on the original system (5) orre-spondsto oneBlo k-Seideliterationontheenhan edsystem

e

Aex = Db; (12)

where the diagonal blo ks are the blo ks dened by the p subdomains. More pre isely, denotingby

f

M theblo k lower triangularpart of e

(13)

anbeexpressedasfollows: 8 > > < > > : e x k =Dx k ; e r k =Dr k ; e x k +1 =xe k + f M 1 e r k ; x k +1 =Pxe k +1 : (13) Toproveit, letuspartition

e A= f M e N,where e

N isthestri tlyupperblo ktriangular part of ( e A). Matri es f M and e

N are partitioned by blo ks a ordingly to the domain denition. OneiterationoftheBlo kGauss-Seidelmethod anthenbeexpressedby

e x k +1 =ex k + f M 1 e r k :

The resulting blo k triangular system is solved su essively for ea h diagonal blo k. To derivetheiteration,wepartition ex

k andex

k +1

a ordingly. Attherststep,weobtain e x k +1 1 =ex k 1 +A 1 1 e r k 1 ; (14)

whi h isidenti al totherststepoftheMultipli ativeS hwarz x k +1=p =x k +A + 1 r k . The i-thstep(i=2;;p) e x k +1 i =ex k i +A 1 i e b i f M i;1:i 1 e x k +1 1:i 1 A i e x k i + e N i;i+1:p e x k i+1:p ; (15)

isequivalenttoits ounterpartx

k +(i+1)=p =x k +i=p +A + i r k +i=p

intheMultipli ativeS hwarz algorithm.

Therefore,wehavethefollowingdiagram R n M 1 ! R n D # " P R m f M 1 ! R m andwe on ludethat M

1 =P f M 1 D:

We must remark that there is an abuse in the notation, in the sense that the matrix denotedbyM

1

anbesingularevenwhen f M

1

isnonsingular.

Weshall provein thefollowingtheorem that this happens whenoneoverlappingblo k issingular. Nevertheless,wekeepthenotationforitsmeaninginthegeneral ase.

3.3 Expli it formulation of the Multipli ative S hwarz

pre ondi-tionner

Theorem3.1 Let A 2 R

nn

be de omposed into p subdomains as des ribed in se tion 2 su hthatallthe matri es

A i

,for i=1;;p,andall thematri esC i

(14)

arenon singular. The Multipli ative S hwarzpre onditionnermatrixM 1 anbe expli itly expressedby: M 1 = A p 1 C p 1 A 1 p 1 C p 2 A 2 1 C 1 A 1 1 (16) wherematri es A i and C i

aredenedinse tion 2.

Proof. TheRi hardsoniteration orrespondingto the Multipli ative S hwarz pre ondi-tionnerisexpressedbytherelationx

k +1 =x k +M 1 r k

. Whenweinje titintheaugmented dimensionwehavethefollowingiteration:

e x k +1 =ex k + f M 1 e r k , where ex k =Dx k , er k =Dr k and f M 1

represents theBlo k Gauss-Seidel pre onditionner built from

e A. Let us set f M 1 e r k = e t k ; then xe k +1 = xe k + e t k and therefore,x k +1 =Pxe k +1 =x k +P e t k : In order to ompute P e t k

, we eliminate, within the system er k = f M e t k , the unknowns whi h mustbedis ardedbythe proje tionP. Forthat purpose, wepartitionthediagonal blo k(A i )of ~ Aas follows: A i = B i F i E i C i

and a ordingly the ve tors e t i k = f i k g i k and er i k = u i k v i k for i = 1;:::;p 1and ~ t k p =f p k ,r~ k p =u p k

. We anobservefromthisrepresentationthat r k = 2 6 6 6 4 u 1 k . . . u p 1 k u p k 3 7 7 7 5 and t k = 2 6 6 6 4 f 1 k . . . f p 1 k f p k 3 7 7 7 5 : Wenowformtheredu edsystembyonlykeepingthe omponentsoft

k : (B 1 F 1 C 1 1 E 1 )f 1 k = u 1 k F 1 C 1 1 v 1 k ; (17) E i 1 f i 1 k 0 +(B i F i C 1 i E i )f i k = u i k F i C 1 i v i k ; i=2;:::;p 1 (18) A p e t p k + E p 1 f p 1 k 0 = er p k : (19)

Notethat,sin eer k =Dr k ,wehavev i k =R i+1 u i+1 k whereR i+1 u i+1 k

onsistsofsele tingthe rsts

i

omponentsofblo kve toru i+1

. Rememberthats i

isthedimensionoftheoverlap C i . LetS i =(B i F i C 1 i E i

)fori=1;:::;p 1bethelo alS hur omplementsasso iated tovariablef i k andS p =A p

. Theredu edsystembe omesBt k

=z k

wherethestru tureof B isdenedin Figure3.

Therighthandsideoftheredu edsystemz k anbewritten asz k =Tr k where:

(15)

S1

S3

A4

S2

E1

E2

E3

0

0 B=

Figure3: Matrixofredu edsystem(forp=4) z k = 0 B B B B B u 1 k F 1 C 1 1 R 2 u 2 k u 2 k F 2 C 1 2 R 3 u 3 k . . . u p 1 k F p 1 C 1 p 1 R p e r p k e r p k 1 C C C C C A andthereforez k =Tr k where Tr k = 0 B B I 1 F 1 C 1 1 0 0 0 0 I 2 F 2 C 1 2 0 0 0 0 I 3 F 3 C 1 3 0 0 0 0 I 4 1 C C A 0 B B u 1 k u 2 k u 3 k e r 4 k 1 C C A (20) (displayedforp=4). MatrixM,denedbytheMultipli ativeS hwarzsplitting,istherefore hara terizedbytherelation

(16)

Wenow provethat M = A 1 C 1 1 ::: C p 1 1 A p

satises that relation. Werst express the stru tureofthefollowingmatrixwiththeblo kstru turedenedbyB andT:

A i C i 1 = 0 B B B B B B B B B B B B B B 0 B I . . . I 1 C A 0 B i G i E i 0 I 1 A 0 B I . . . I 1 C A 1 C C C C C C C C C C C C C C A i th rowblo k (i+1) th rowblo k fori=1;;p 1. Itiseasytoprovebyindu tion thatfori=1;:::;p 2

i = T( A 1 C 1 1 )( A 2 C 2 1 ):::( A 1 C 1 1 ) = 0 B B B B B B B B B B B B B B B B B B S 1 E 1 0 S 2 . . . . . . . . . S i E i 0 I G i+1 0 . . . . . . . . . G p 1 0 I 1 C C C C C C C C C C C C C C C C C C A andtherefore p 1 = T( A 1 C 1 1 )( A 2 C 2 1 ):::( A p 1 C p 1 1 ); = 0 B B B B B B B B B S 1 E 1 0 S 2 . . . . . . S p 1 E p 1 0 I 0 0 I 1 C C C C C C C C C A :

(17)

Clearlywehave p 1 A p =B

whi h endstheproof.

Remark3.2 Let us des ribe more pre isely the situation when one of the overlapping blo ks C

i

issingular. Withthis assumption,matrix

S = A p 1 C p 1 A 1 p 1 C p 2 A 2 1 C 1 A 1 1

issingular and therefore it annot be onsidered asbeingtheinverseofamatrixM. Inthatsituation,S annotbeusedaspre onditionnerto solvesystemAx=b: aleftappli ationofthepre onditionnerwouldleadtosolveSAx=SB whi hdoesnothaveauniquesolutionandarightappli ationisimpossible. It anbenoti ed that the singularity of one of the blo ks C

i

implies the singularity of the mapping e A sin e thespe trumof

e

Aisequal totheunionofthe spe trumofAandthespe traofallthe blo ks C

i

(i=1;;p 1)[9 ℄.

Proposition 3.1 ThematrixN denedbythe multipli ativeS hwarzsplittingA=M N anbeexpressedasfollows: 8 < : N ij =G i G j 1 B j ; whenj >i+1 N ii+1 =G i B i+1 [F i 0℄; N ij =0otherwise; (21) whereG i =(F i C 1 i 0) fori ; j=1;:::;p 1.

Whenthe domain de omposition iswithweak overlap, expression(21)be omes: N ii+1 =G i B i+1 [F i 0℄; fori=1;:::;p 1; N ij =0otherwise: (22) Proof. Itfollowsfromequation20thattheinverseofmatrixT =I U is:

T 1 = p 1 X i=0 U i where: U = 0 B B B 0 G 1 0 . . . . . . G p 1 0 1 C C C A : Therefore,thematrixM anbeexpressedby:

8 > > > > > > < > > > > > M ii =B i ; M ii+1 =G i B i+1 ; M i+1i = E i 0 ; fori ; j = 1 p 1 M ij =G i G j 1 B j ; if j>i+1

(18)

By expressing matrix A in the same stru ture as the stru ture of M, we dedu e that, N =M Asatisesrelation(21). Whenthede ompositionisofweakoverlap,relation(22) followsfromthefa tthat G

i G

i+1

=0.

Corollary3.1 In the splitting A = M N asso iated with the Multipli ative S hwarz method, matrixN isofrankr

P p 1 i=1 s i .

Proof. Theproofisobviouswhenthede ompositioniswithweakoverlap. Forthegeneral ase,wehavetoprovethattherankofrowblo kiofmatrixN islessthans

i

. Thestru ture ofrowblo kN

i

,fori=1;:::;p 1,ofmatrixN is: N i = 0 ; 0 ; [F i 0℄ 0 C 1 i B i+1 (1;2) 0 I ; G i G i+1 B i+2 ; ; G i G p 1 B p : Therefore,therankofrowblo kofN

i

islimitedbytherankoffa tor[F i 0℄whi h annot ex eeds i . Thisimpliesr P p 1 i=1 s i . 3.4 Symmetrization of M

Evenwhen matrix A is symmetri , the pre onditioned onjugategradient method annot beuseddire tlysin etheMultipli ativeS hwarzpre onditionerisnotsymmetri . However, it aneasily besymmetrized by in ludingase ondsweep orrespondingto applyM

T to urrentresidualr

k +1

. It anbewrittenasfollows: x k +1 =x k +M 1 r k ; x k +2 =x k +1 +M T r k +1 : Usingthatr k +1 =NM 1 r k ,wehave x k +2 = x k +M 1 r k +M T NM 1 r k ; = x k +M T (M T +N)M 1 r k ; = x k +M T (M T +M A)M 1 r k :

Wededu e that thenew pre onditioningmatrix(the one orrespondingto twosweeps up anddown)issu hthat:

M 1 =M T (M T +N)M 1 =M T (M T +M A)M 1 : It anbeshown[3℄thatwhenAiss.p.d., thepre onditionner Miss.p.d. aswell.

3.5 Red-Bla k ordering (M and N)

TheMultipli ativeS hwarzmethod,asdes ribedinAlgorithm1, learlyisnotsuitablefor parallelismsin ethere ursionbetweenblo kspreventsindependent al ulations. The las-si alwaytooverpassthedrawba kistorelaxpartofthere ursionbyaRed-Bla k oloring.

(19)

Re allingourassumptionthatoverlaponlyo ursbetween onse utiveblo ks,likeinFigure 1,alltheblo ksofoddnumbers anbe usedin paralleland thentheupdate is performed with all the blo ks of even numbers. If we onsider a reordering of the unknowns whi h labelsrst the omponents orresponding tothe oddsubdomainsand then theeven ones, it aneasilybeshownthatthenewpre onditionerisstillaMultipli ativeS hwarzmethod but with onlytwosubdomains ; themethod be omes anAlternativeS hwarz method. In su h asituation theoverlap is theunion of all the elementary overlapsand thereforethe totaldimensionoftheoverlapdoesnot hange.

(20)

4 Multipli ative S hwarz as pre onditionner of Krylov methods

4.1 Early termination

Whenpre onditioningaKrylovmethod,we onsiderinthisse tiontheadvantageto onsider asplittingA=M N in whi hN isrankde ientwhi histhe asefortheMultipli ative S hwarzpre onditioner.

ForsolvingtheoriginalsystemAx=b,wedeneaKrylovmethod,asbeinganiterative methodwhi hbuilds,fromaninitialguessx

0 ,asequen eofiteratesx k =x 0 +y k su hthat y k 2K k (A;r 0 )where K k (A;r 0

)istheKrylovsubspa eof degreek,builtfromtheresidual r

0

oftheinitial guess: K k (A;r 0 )=P k 1 (A)r 0 where P k 1

(R) istheset of polynomialsof degreek 1orless. Theve tory

k

isobtainedbya hara teristi propertywhi hdepends on the method ; this property may be minimizing the error x

k

x for agiven norm, or proje tingthe initialerror onto thesubspa eK

k (A;r

0

) in agivendire tion. Nevertheless, we onsider that, for a given k, when the property x 2 x

0 +K k (A;r 0 ) holds, it implies that x k

= x. This last property is satised when the Krylovsubspa e sequen e be omes stationary: (K k (A;r 0 )=K k +1 (A;r 0 )))(x2K k (A;r 0 )):

WhenaKrylovmethodisleftpre onditioned bytheoperatorM,theKrylovsubspa es to onsiderare : K k (M 1 A;M 1 r 0

). Forarightpre onditioningwiththe sameoperator, thesubspa eofinterestbe omes: K

k (AM 1 ;r 0 ).

Proposition 4.1 When rank(N) = r < n, then any Krylov method rea hes the exa t solutionin atmostr+1iterations.

Proof. In M 1 A = I M 1 N the matrixM 1

N is of rank r. For any degree k, the following in lusionK k (M 1 A;M 1 r 0 )(r 0 )+R(M 1

N)guaranteesthat thedimension ofK k (M 1 A;M 1 r 0

)isatmostr+1. Therefore,themethodisstationnaryfromk=r+1

atthelatest. Theproofisidenti alfortherightpre onditioning.

Fora general non singular matrix, this result is appli able to the methods BiCG and QMR,pre onditionedbytheMultipli ativeS hwarzmethod. Inexa tarithmeti ,the num-berofiterations annotex eed thetotal dimensions of theoverlapbymorethan 1. The sameresultappliestoGMRES(m)whenmisgreaterthans.

Fora symmetri positive denite matrix, therelevant method is PCG and it requires a s.p.d pre onditionner. It is therefore ne essary to symmetrize the basi multipli ative S hwarzpre onditionerasdoneinse tion3.4. When,thematrixissymmetri nondenite, the symmetri pre onditioner might also be non denite, and the method to onsider is SQMR. However in these situations, the rank de ien y property is mu h less attra tive thanwiththenonsymmetri basi ase.

Thepreviousremarksholdinexa tarithmeti ,but,asweshallseeinthenumeri altests, roundo errorsdarkenthe pi ture espe ially for methods whi h rely on non orthonormal basisandforill onditionedmatri es.

(21)

4.2 Advantages of the expli it formulation

Inthe lassi alexpressionoftheMultipli ativeS hwarziteration(Algorithm1)the ompu-tationofthetwosteps

x k +1 = x k +M 1 r k andr k +1 =b Ax k +1 ;

is arried outre ursivelythrough thedomains whereas the expli itformulation de ouples thetwo omputations. The omputationoftheresidualisthereforemoreeasilyparallelized sin eitiswithdrawnfromthere ursion. Anotheradvantageoftheexpli itexpressionarises whenitis usedasapre onditionerofamethod already oded inalibrary. Insu h a ase, the user is supposed to provide a ode for the pro edure x ! M

1

x. Sin e the method omputestheresidual,the lassi alalgorithmimpliesadouble al ulationoftheresidual.

Wenowshowthatthenumberofoperationsinvolvedinbothapproa hesremainsroughly thesame,althoughwithaslightadvantagetotheexpli itformulation.

LetusdenotebyC(p) la

the ostofoneiterationofAlgorithm1. One anverifythat C(p) la = p X i=1 (t i +p i ); (23) where t i

= # of opsforpro essingonsubdomain i: x:=x+A + i

r; p

i

= # of opsforpro essingonsubdomain i: r:=r A(A + i

r): LetusalsodenotebyC(p)

exp

thenumberofoperationsfor omputingx!M 1

x,byusing theexpli itform oftheMultipli ativeS hwarzpre onditionner:

C(p) exp = p 1 X i=1 (t i + i )+t p ; (24) where i

isthenumberof opsformultiplyingave torinsubdomainibyC i

. Thenumberof operationsC(A)involvedinthe omputationofaresidual,whi hinvolvesthemultipli ation by matrix A, satises the relation C(A) =

P p i=1 q i P p 1 i=1 i where q i is the number of operationsinvolvedin themultipli ationbyblo kA

i

. Sin eq i

<p i

(they areusually lose numbers),weobtainthat

C(p) la

>C(p) exp

(22)

5 Numeri al experiments

Inthisse tionweillustratethenumeri albehaviouroftheMultipli ativeS hwarz pre ondi-tionnerforKrylovsubspa emethods. Thetestmatri esaretakenfrom theMatrixMarket suite [1℄. The tests were arried out in MATLAB, and they were hosen to illustrate the propertyofearlytermination.

We denode by MS the Multipli ative S hwarz pre onditioner and by SMS its Sym-metrized Multipli ative S hwarz ounterpart. For ea h method, the right-hand side is a xedrandomve tor,theinitialguess isthenulve torandthetoleran efor onvergen eis setto10

8 .

Test1: Matrix: S3RMT3M3shiftedasA:=A+10 3

kAkI

Symmetri ReverseCuthillMa keereorderingisappliedonthematrixtoredu ethe band-width.

Sour eofS3RMT3M3 : FiniteelementanalysisofCylindri alShells Order: 5357

Type: RealSymmetri positivedenite Conditionnumber: 60.99

Domainde omposition

Blo knumber 1 2 3 4 5 6

1strowindex 1 875 1830 2810 3860 4700

lastrowindex 1000 2000 3000 4000 4800 5357 rank(N)

P (rank(F

i

))=92+144+164+131+89=620 Spe tralradiusoftheiterationmatrix:(M

1

N)=0:4583

Table1 and Figure 4showa ni e onvergen e of methods sin ethe spe tral radius of theiterationmatrixM

1

N issmall.

Test2: Matrix: BCCSTK20

Sour eofBCCSTK20 : Stru turalEngineering. Order: 485

Type: RealSymmetri indenite Conditionnumber: 7:510

12

Domainde omposition

Blo knumber: 1 2 3 4

1strowindex 1 45 285 390

lastrowindex 50 300 400 485 rank(N)

P (rank(F

i

))=2+14+8=24 Spe tralradiusoftheiterationmatrix:(M

1 N)=1

Thatmatrixis losetobeingsingularandthisisadiÆ ultproblem(see onvergen ein Table 2and gure 5) when ompared to the rst one. Only GMRESsu eeds. In BICG andQMR,anear-breakdowno urs[2℄.

(23)

Symmetri ReverseCuthillMa keereorderingonthematrixisappliedtoredu ethe band-width.

Sour eofSHERMAN5: OilReservoirsimulation hallengematri es. Order: 3312

Type: RealUnsymmetri Conditionnumber: 3:910

5

Domainde omposition

Blo knumber 1 2 3 4

1strowindex 1 450 900 2495

P (rank(F

i

))=31+61+0=92 Spe tralradiusoftheiterationmatrix:(M

1

N)=0:8769.

Thisis anunsymmetri problemandtherefore,weonly onsiderthemethodsGMRES, QMRandBiCG. Table3andFigure 6showagood onvergen eforallthethree methods. QMRandBiCGperformsidenti ally,whi histhe asewhenBiCGworkswell.

Test4: Matrix: GRE1107

Sour eofGRE1107 : SimulationofComputerSystem. Order: 1107

Type: RealUnsymmetri Conditionnumber: 9:710

7

Domainde omposition

Blo knumber 1 2 3 4

1strowindex 1 130 400 875

P (rank(F

i

))=38+73+49+0=160 Spe tralradiusoftheiterationmatrix:(M

1

N)=5:302810 4

.

This is another diÆ ult problem. The matrix is almost singular. Table 4 and Figure 5 show the onvergen e of GMRES. It is interesting to note that the MS method would divergesin e(M

1

N)>1. Thelastpointin theresidualofthegraphreportedin Figure 5is surprizingsin ethe2-normoftheresidualshould deneanonin reasingsequen e. It aneasilybeexplainedbythefa tthatduringtheinneriterations,thenormoftheresidual is omputedbyaformulawhi hisnotrobustwithrespe ttothelossoforthogonalityofthe basiswhereastheresidualisee tively omputedat thebasisrestart. InBICGand QMR, nearbreakdowno urs whi h prevents onvergen e.

We an on ludefrom thesequen eof teststhat early terminationpropertyisnot suf- ient for obtaining onvergen e in oating point arithmeti . However GMRES, whi h appearstobemu h morerobust,is learlysuperior. Inmostofthe ases, onvergen ewas obtainedevenmu hearlierthanwhat ouldbeexpe ted. However,GMRESsuers forthe limitation on the size of the basis sin e atoolarge basis would imply atoo high level of

(24)

storageand a too largenumber of operations. Moreover,the lossof orthogonality within thebasismayendupwithasingularHessenbergmatrixwhi h provokesarestart.

Pre onditionner NumberofIteration kr k k=kr 0 k GMRES MS 7 2.9095e-09 PCG SMS 10 1.0679e-09 MINRES SMS 11 9.8125e-09 QMR MS 12 1.7088e-09 BICG MS 9 1.0584e-09

Table1: Convergen eoftheiterativemethodsonmatrixS3RMT3Mshiftedand permuted Pre onditionner NumberofIteration kr

k k=kr 0 k GMRES MS 8 9.3112e-09 QMR MS no onvergen e 1 BICG MS no onvergen e 1

Table2: Convergen eoftheiterativemethodsonmatrixBCCSTK20 Pre onditionner NumberofIteration kr

k k=kr 0 k GMRES MS 8 9.3112e-09 QMR MS 12 1.1845e-09 BICG MS 12 01.1910e-09

Table3: Convergen eoftheiterativemethodsonmatrixSHERMAN5 permuted Pre onditionner NumberofIteration kr

k k=kr 0 k GMRES MS 166 6.2627e-07 QMR MS no onvergen e 1 BICG MS no onvergen e 1

(25)

0

5

10

15

10 −8

10 −6

10 −4

10 −2

10

0 GMRES

0

5

10

15

10 −10

10 −5

10

0

10

5 PCG

0

5

10

15

10 −10

10 −5

10

0

10

5 QMR

0

5

10

15

10 −10

10 −5

10

0

10

5 BICG

0

5

10

15

10 −10

10 −5

10

0

10

5 MINRES

Figure4: Convergen eoftheiterativemethodsonmatrixS3RMT3Mshiftedandpermuted

6 Con lusion

The Multipli ative S hwarz is a very eÆ ient pre onditionnerespe ially for Krylov methods. We have establishedits earlytermination propertywhi h an redu e the numberof iterations, dependingon the amountofoverlap.

Inthisworkwehaveexhibitedanexpli itformulationoftheMultipli ativeS hwarzpre onditionner.By de ouplingtheappli ationofthepre onditionerandthe omputationoftheresidual,weexpe ttobeable toparallelizesu essiveiterations.Su hanapproa hispresentlybeingdeveloppedontheGMRESmethod. Arstbasi parallelversionofthe odesisstudiedin[5℄. AlthoughtheAdditiveS hwarzpre onditioner isoftenpreferedforitsabilitytobeparallelized,ithasaslower onvergen erate. AneÆ ientparallelized multipli ativeversionmight hange on lusion.

(26)

0

5

10

15

20

10 −8

10 −7

10 −6

10 −5

GMRES on BCSSTK20

0

50

100

150

200

10 −5

10

0

10

5

10

15 GMRES GRE_1107

Figure5: Convergen eoftheiterativemethodsonmatrixGRE1107

0

5

10

15

10 −10

10 −5

10

0

10

5

0

5

10

15

10 −10

10 −5

10

0

10

5

0

5

10

15

10 −10

10 −5

10

0

10

5

Figure6: Convergen eoftheiterativemethodsonmatrixSherman5permuted

7 Appendix: Alternative proof of Theorem 3.1

Notation: For 1ij p, I i:j

isthe identityonthe unionof the domainsW k

,(i kj)and

(27)

Lemma7.1 Foranyi2f1;;p 1g, A i+1 I i +I i+1 A i I i+1 AI i = C i I i:i+1 ; andforanyi2f1;;pg,

A + i = A 1 i I i =I i A 1 i : Straightforward(Seegure7)

I

I+1

C

i

Figure 7: SubdomainsofA Proofoftheorem3.1:Letusprovebyindu tionthat:

M 1 = A p 1 C p 1 ::: C 1 A 1 1 Sin ex k +1 =x k +M 1 r k andsin ex k +1

isobtainedasresultofsu essivesteps,fori=1:::p:x i+1 =x i + A + i r i wherex i denotesx k +i=p andr i denotesr k +i=p

,weshallprovebyreverseindu tiononi=p 1;;0 that: xp=xi+ Ap 1 Cp 1 ::: Ci+1 A 1 i+1 Ii+1:pri (26)

Fori=p 1,therelationx p =x p 1 + A 1 p I p r p isobviouslytrue. Letusassumethat(26)isvalidforiandletusproveitfori 1.

x p = x i 1 +A + i r i 1 +A 1 p +:::+ C i+1 A 1 i+1 I i+1:p (I AA + i )r i 1 = x i 1 + A 1 p ::: C i+1 (A + i +A + i+1 I i+1:p A 1 i+1 I i+1:p AA + i )r i 1 : Thelasttransformationwaspossiblesin ethesupportsofAp;Cp 1;Cp 1;:::;C

i+1

aredisjointfromdomain i.Letustransformthematrixexpression:

B = A + i +A + i+1 I i+1:p A 1 i+1 I i+1:p AA + i = A 1 i+1 (A + i+1 I i +I i+1:p A i I i+1:p AI i ) A 1 i

(28)

Lemma7.1andelementary al ulationsimplythat: B = A 1 i+1 ( C i I i:i+1 +I i+2:p O i+1 ) A 1 i = A 1 i+1 C i A 1 i I i:p