Transparent usage of grids for data parallelism

(1)

HAL Id: hal-01343942

https://hal.archives-ouvertes.fr/hal-01343942

Submitted on 11 Jul 2016

HAL is a multi-disciplinary open access

archive for the deposit and dissemination of

sci-entific research documents, whether they are

pub-lished or not. The documents may come from

teaching and research institutions in France or

abroad, or from public or private research centers.

L’archive ouverte pluridisciplinaire HAL, est

destinée au dépôt et à la diffusion de documents

scientifiques de niveau recherche, publiés ou non,

émanant des établissements d’enseignement et de

recherche français ou étrangers, des laboratoires

publics ou privés.

Transparent usage of grids for data parallelism

Daniel Millot, Christian Parrot, Eric Renault

To cite this version:

Daniel Millot, Christian Parrot, Eric Renault. Transparent usage of grids for data parallelism. PDCS

2005: 18th ISCA International Conference on Parallel and Distributed Computing Systems, Nov 2005,

Las-Vegas, United States. pp.241 - 246. �hal-01343942�

(2)

! " #$ %& '() * +,)

-.& /$$ .- #01 2 3$

456789:7

;<=>?@?ABCA>DB=EA> @ >FGHI@BA?BFCJDH=FK D<@=KCACL =D@HAC HF AKCLJ>AB>I<=D<MN=OAK @ >APJAKH=@Q ?BFNB@R HF EA @??Q=AC HF @ >AH FG =KC=O=CJ@Q C@H@M AK@EQA> AGL SD=AKH ?@B@QQAQ ?BFDA>>=KN FG <JNA DFQQADH=FK> FG C@H@ FK @ NB=CT ;<=> >U>HAR BAPJ=BA> KF =KOFQOARAKH FGH<A J>AB @KC ?BFCJDA> G@=BQU NFFC BA>JQH>T

V W

X78YZ[:7\Y X

] BAPJAKHQU H<A >@RA DFR?JH@H=FK => R@CA =KCA?AKL CAKHQU FOABH<A AQARAKH>FG @ N=OAK DFQQADH=FK FG C@H@M =KCJD=KN C@H@ ?@B@QQAQ=>R ^__ `T a> NB=C> ^bM c` @BANAHL H=KN RFBA@KCRFBADFRRFKM=H=>H<ABAGFBA=R?FBH@KH HF EA @EQA HF BJK @??Q=D@H=FK> =KOFQO=KN C@H@ ?@B@QL QAQ=>R FK>JD< H@BNAH?Q@HGFBR>T dKFBCABHF HBU @KC CF >FMFKA <@>H<A D<F=DAEAHIAAK A=H<AB CA>=NK @ ?@B@QQAQ @??Q=D@H=FKMI<=D< R@UEA@<@BCH@>eBA>ABOACHF>?AL D=@Q=>H>fJKQA>> J>ABLGB=AKCQUHFFQ> @BA J>ACf FB >=R?QU IB=HA@ >APJAKH=@Q?BFNB@RMBAQU=KNFK @JHFR@H=DHFFQ> HF ?BFCJDA H<A DFBBA>?FKC=KN ?@B@QQAQ DFCAT gFIAOABM H<=> HAD<K=PJA=> R@=KQU CAC=D@HAC HF >URRAHB=D RJQL H=?BFDA>>FB>FBOADHFB?BFDA>>FB>M@KCD@KFKQUEAJ>AC FK DQJ>HAB>H<BFJN< >FGHI@BALE@>AC C=>HB=EJHAC ><@BAC RARFBU >U>HAR> ^ hM _c`T a> >F R@KU DFR?JH=KN BAL >FJBDA> @??A@B HF EA @O@=Q@EQA EU RA@K> FG NB=C>M IA GFDJ>FKA@>=KNH<A=BJH=Q=i@H=FKB@H<ABH<@KFK R@e=KN GJQQ J>AFGH<A @O@=Q@EQADFR?JH=KN ?FIABT

jK

AD@K DFK>=CAB@KU>FGHI@BADFR?FKAKH@>DFKL >JR=KN C@H@GBFR @C@H@>HBA@RM@KC?BFCJD=KNBA>JQH> =KHF @ BA>JQH> >HBA@RT ;<ABAGFBAM=K H<ACA>=NK FG @ >AL PJAKH=@Q =R?QARAKH@H=FK FG >JD< @ DFR?FKAKHM H<A>A HIF >HBA@R> RJ>H EA BA>?ADH=OAQU >Q=DAC =KHF @KC NAKL AB@HAC GBFR C@H@ >HBJDHJBA> kK@RAQU lmnm @KC opqrsntT ;<J>M H<A C@H@HU?A> lmnm @KC opqrsn <@OA KADA>>AB@QU HFEA>?AD=SACI<AKIB=H=KNH<A>APJAKH=@QDFBBA>?FKCL =KN DFCAT a> IA CF KFH KAAC @KUH<=KN AQ>AMIA CF KFH BAPJ=BA @KU AuHB@ AvFBH GBFR H<A J>ABT

KAKH I<=D< DFKH=KJFJ>QU BA@C> lmnmM D@QQ> >FRA GJKDL H=FK yzo{| }~ lmnm zrn opq

rsn

FK A@D<=HAR @KC IB=HA> H<A DFBBA>?FKC=KN opqrsn @> ><FIK =K ]=NT _T ;<A

Worker

data stream

result stream

]=NJBA _ AKAB=D>APJAKH=@Q >FGHI@BA DFR?FKAKHT

C@H@ >HBA@R => GAC EU RA@K>FG@ C@H@ >FJBDA kSQA >U>L HARM C@H@E@>AM >AK>FB KAHIFBeM KAHIFBe DFKKADH=FKM AHDtM@KCBA>JQH>@BADFK>JRACEU@ DFK>JRAB?BFDA>>T wJD<

@ >FGHI@BADFR?FKAKH DFJQC EA?@BHFG @?=?AQ=KA =KOFQO=KN @ I<FQAD<@=K FG>JDDA>>=OA?BFDA>>=KN >HA?>T dGFKAKAAC> HF =KDBA@>A H<AFI H<BFJN<H<A?=?AQ=KAM H<=> DFR?FKAKH => @ ?FHAKH=@Q EFHHQAKADe H<J>M =H => @ NFFC D@KC=C@HA GFB ?@B@QQAQ AuADJH=FKT JA HF C@H@ ?@B@QQAQ=>RMFKA D@K Au?ADH @ KA@B Q=KA@B >?AACLJ?>F =KDBA@>=KN H<A KJREAB FG ?BFDA>>FB>DFJQC @QQFI HF =KL DBA@>AH<AFIM?F>>=EQUQA@C=KN HF @KFH<ABEFHHQAKADe J?>HBA@R FB CFIK>HBA@RT aCC=H=FKK@QQUM =KHBFCJD=KN >FRA EJvAB=KN EAHIAAK ?=?AQ=KA >H@NA> D@K <AQ? @EL >FBEA FI JDHJ@H=FK> @KC R@eA DFRRJK=D@H=FK @KC DFR?JH@H=FK FOABQ@?T JH =K FBCAB HF EA J>AGJQ HF AKCLJ>AB>M H<A ?@B@QQAQ AuADJH=FK ><FJQC EA FEH@=KAC EU RA@K>FG @>A@RQA>>BA?Q@DARAKHFG>APJAKH=@Q >FGHL I@BA DFR?FKAKH> EU @JHFR@H=D@QQULNAKAB@HAC APJ=O@L QAKH ?@B@QQAQ DFR?FKAKH>T

dK H<=> ?@?ABM IA ?BF?F>A @ >FGHI@BA ?BFCJDH=FK D<@=K ad kJHFR@H=D KHANB@H=FK GFB @B@QQAQ uADJH=FKtHF CA@Q I=H< @Q@BNAO@B=AHUFG@??Q=D@H=FK>T ad AK@EQA>AKCLJ>AB>HF NAH?@B@QQAQAuADJH=FKFG@K @??Q=D@H=FK FK NB=C> I=H<FJH IB=H=KN @KUH<=KN RFBA H<@K =H> >APJAKH=@Q DFCAT

jJB RAH<FC <@> EAAK >JDDA>>GJQQU J>AC HF @JHFL R@HAH<A ?BFDA>>=KN FG @ <JNADFQQADH=FK FG >?ADHB@ GFB =CAKH=SD@H=FK FG >AR=LKJDQAFH=CA ?FQURFB?<=>R> =K @ C@H@E@>A FG a >@R?QA>T dH D@K EA J>AC A=H<AB GFB R@>> ?BFCJDH=FK FG BA>JQH> FB GFB B@?=C ?BFHFHU?=KNT dKCAACM =H @QQFI> H<A CAOAQF?AB HF DFKDAKHB@HA FK CAL

(3)

F?AB D@K NAH PJ=De GAACE@De I<AK RFC=GU=KN @K @QNFL B=H<RM @> <A D@K @D<=AOA >=NK=SD@KH BA>JQH> EU DFK>=CL AB=KN =R?FBH@KH >@R?QA> I=H<FJH =KDJB=KN C=>>J@>=OA CAQ@U>T ;<ABAGFBAH<A CAOAQF?RAKH?<@>A FGH<A@??Q=L D@H=FK D@K @Q>F EAKASH GBFR ?@B@QQAQ=>RMI=H< KF >?AL D=SD ?BFNB@RR=KN AvFBHM@> IA?BFO=CA @ HB@K>?@BAKH DFR?FKAKH >JE>H=HJH=FKMKFH @ ?BFNB@RR=KN HFFQT

;<=>?@?AB=> FBN@K=iAC@>GFQQFI>T ;<A KAuH >ADL H=FK N=OA> @K FOABO=AI FG AKO=BFKRAKH> <AQ?=KN J>AB> <@OA H<A=B @??Q=D@H=FK> AuADJHA FK @ NB=CT wADH=FK ?BA>AKH> H<A >FGHI@BA @BD<=HADHJBA IA CA>=NKAC HF @QL QFI AD=AKH ?@B@QQAQ AuADJH=FKT wADH=FK c GFDJ>A> FK <FI ad R@eA> =H> J>@NA @QRF>H HB@K>?@BAKH HF @K AKCLJ>ABT

wADH=F

K ?BA>AKH>?ABGFBR@KDA SN

JBA>T wADL H=FK DFR?@BA>FJBKAI @??BF@D<HF BAQ@HACBA>A@BD< IFBe>MFJHQ=KA>GJHJBA IFBe @KC DFKDQJCA> H<A ?@?ABT

97

Z Y86

aRFKN H<A AKO=BFKRAKH> I<=D< <@OA EAAK CAOAQF?AC =K FBCAB HF CF ?@BH FG H<A FE GFB H<A AKCLJ>AB I<F KAAC>HFCA>=NK?@B@QQAQ@??Q=D@H=FK>M>FRA@BACAOFHAC HF?@B@RAHAB>IAA?@??Q=D@H=FK>M>JD<@>=RBFCL ^_ ` FBd@E^_`MI<ABA@>FH<AB>DFJQCEAJ>AC=KRFBANAKL AB@Q >AHH=KN> k a??Aw ^`M ;jL ^ ` FB =KGL ^_h`tT @B@RAHAB >IAA? D@K =KCAAC EA DFK>=CABAC @ ?@BH=DL JQ@B GFBR FG C@H@ ?@B@QQAQ=>RM @> =H =HAB@HA> =CAKH=D@Q DFR?JH@H=FK>FK @QQ H<AAQARAKH> FG@ >AHFGKJRAB=D@Q ?@B@RAHAB>M @KC A@D< FG H<A =KC=O=CJ@Q DFR?JH@H=FK> FG@ ?@B@RAHAB >IAA? @??Q=D@H=FK D@K EABJK =KCA?AKL CAKHQU FG H<AFH<AB>M@> @ C=vABAKH ?BFDA>>T ;<ABAGFBAM @ J>AB D@K DBA@HA@ DFBBA>?FKC=KN >AH FG@C<FD>DB=?H> @KC >D<ACJQA H<A=B AuADJH=FK FK FKA FB RFBA ?BFDA>L >FB>T

=RBFCL ^_` => @ NB=C OAB>=FK FG @ HFFQ CAL >=NKACGFBC=>HB=EJHAC ?@B@RAHB=DRFCAQ=KNMI<=D< <@> EAAK AK<@KDAC GBFR DQJ>HAB> HF NB=C>T J=QH FOAB H<A QFEJ> ^`NB=CR=CCQAI@BAM=H=>CAC=D@HACHF?@B@RAHL B=D >HJC=A> @KC ?BFO=CA> @ NB@?<=D@Q J>AB =KHABG@DA HF <AQ? H<AJ>AB>?AD=GUH<A?@B@RAHAB>?@DA>FG<=>@??Q=L D@H=FK@KCH<ADFR?JH@H=FKHFEACFKAT =RBFCLD@K H<AK?BA?@BAH<ADFBBA>?FKC=KN=KCA?AKCAKHFE>MDFKL HBFQ H<A=B AuADJH=FK @KC R@K@NA H<A BA>JQH>T jK J>AB BAPJA>HM=RBFCLPJAB=A>H<A w ^_`DFR?FKAKHFG QFEJ> HF QFD@HA BA>FJBDA>MH<AK >AQADH> H<AR @DDFBCL =KNHF >FRADB=HAB=@=KDQJC=KNEFH<H@>e>CA@CQ=KA>@KC DF>HJ>@NACA?AKC=KNFK>FRAADFKFR=DRFCAQT ;FFJB eKFIQACNAM KF FH<AB HU?A FG @??Q=D@H=FK> D@K EAKASH GBFR =RBFCLT

dK H<A >@RA I@U @> =RBFCLM d@E ^_` => CAC=L

H<A DBA@H=FK FG =K?JH SQA> A@>=ABM @KC EJ=QC> @JHFR@HL =D@QQU @ >DB=?H GFB A@D< BJKT =vABAKH RAH<FC> @QQFI H<AAuADJH=FKFGH<A>A>DB=?H>MA=H<ABO=@BD? @KCB>< DFRR@KC>FB H<BFJN< @FE >D<ACJQABT

;<A a??Aw ^ ` kQ=D@H=FK AOAQ D<ACJQ=KNt AKO=BFKRAKHM @> =H> K@RA >JNNA>H>MGFDJ>A> R@=KQU FK >D<ACJQ=KNT <AK CA@Q=KN I=H< @K @??Q=D@H=FKM @K a??Aw @NAKH NFA> H<BFJN< H<A GFQQFI=KN >HA?> BAL >FJBDA C=>DFOABUM BA>FJBDA >AQADH=FKM >D<ACJQA NAKAB@L H=FKM>D<ACJQA>AQADH=FKM@??Q=D@H=FK AuADJH=FK I=H< =HL AB@H=OA>D<ACJQA@C@?H@H=FK=KH<AD@>AFGQFKNLBJKK=KN @??Q=D@H=FK>T ;<A R@=K CB@IE@De=> H<@H H<A FB=N=K@Q @??Q=D@H=FK <@> HF EA @ ?@B@QQAQ FB @ C=>HB=EJHAC FKAM @KC =H RJ>H EA RF

C=S

AC =K FBCAB HF EA >D<ACJQAC EU @Ka??

Aw

@NAKHT ;<ABAGFBAMHAR?Q@HA><@OAEAAK?BFL ?F>AC =K FBCAB HF R@eA H<=KN> A@>=ABGFBH<A AKCLJ>ABM @H QA@>H GFB H<A HIF DQ@>>=D@Q ?@B@C=NR> FG ?@B@RAHAB >IAA? @KC @>HABLFBeAB J>=KN >JD< HAR?Q@HA>M H<A J>AB RABAQU <@>HF ?BFO=CA@K CA>DB=?H=FK FG <=> H@>e>@KC a??Aw@JHFR@H=D@QQUNAKAB@HA>@K@??Q=D@L H=FK=HD@K>D<ACJQAT <=QAa??AwI@>AuHAKCACGBFR KAHIFBe>FG IFBe>H@H=FK>HF NB=C>M=H> >D<ACJQ=KN HAD<L K=PJA> <@OA EAAK ?BFNBA>>=OAQU @C@?HAC HF CA@Q I=H< H<A <AHABFNAKA=HUFGNB=C>T

;jL ^` k@>eB=AKHAC@B@QQAQ t<@> EAAK CA>=NKAC HF <AQ? H<A AKCLJ>AB ?@B@QQAQ=iA @ >AL PJAKH=@Q @??Q=D@H=FKT dH J>A> @ @>HABLFBeAB ?BFL NB@RR=KN RFCAQ @KC ?BFO=CA> @ <=N< QAOAQ ad I<=D< <@> EAAK =R?QARAKHAC FK O@B=FJ> @BD<=HADHJBA>M GBFR ><@BAC RARFBU RFCAQ HF NB=C> E@>AC FK H<A QFEJ> ?BFHFDFQ>T ;<A =KHAKH => HF eAA? H<A KJREAB FG Q=KA> H<@H<@OAHF EARF

C=SAC

=KH<A>FJBDADFCA@>>R@QQ@> ?F>>=EQAT aQH<FJN< H<A@O@=Q@EQA ?B=R=H=OA> @BA <=N<AB QAOAQ H<@K d FKA>M H<A J>AB => BAPJ=BAC HF IB=HA @ ?@B@QQAQ @??Q=D@H=FKT

=KGL ^_h`M ?BFO=CA> @ BJKK=KN AKO=BFKRAKH GFB B=C M @ NB=C =R?QARAKH@H=FK FG H<A ?BFHFL DFQT aRFKN H<A @??Q=D@H=FK> GFB I<=D< B=C <@> ?BFOAKHFEAAvADH=OAMFKA=>?@B@RAHAB>IAA?I<=D<=> IAQQL>J=HACGFBAuADJH=FK EURJQH=?QA>ABOAB>FK @NB=CT gFIAOABM=H => FEO=FJ> H<@H =K FBCABHF J>A =KGLM @K @??Q=D@H=FK <@> HF EA BALIB=HHAK J>=KN H<A B=C adT

jK H<A FKA <@KCM H<A>A AKO=BFKRAKH> R@eA H<A J>A FG NB=C> A@>=ABT ;<AU >FRAH=RA> @QQFI KFK HB=O=@Q ?@B@QQAQ=>RM I<ABA@> H<A HFFQ ?BA>AKHAC =K H<=> ?@?AB CFA> KFHM@KC H<A=B R@=K GFDJ> => FK >D<ACJQ=KNM >F @> HF @D<=AOAH<AEA>H J>@NAFG BA>FJBDA>T

jK H<A FH<AB <@KCM H<AU A=H<AB BAPJ=BA @ >=NK=SL D@KHAu?ABH=>A=K?@B@QQAQ@KCC=>HB=EJHAC?BFNB@RR=KN kFB @H QA@>H >FRA =KOFQOARAKH FG H<A J>AB HF CA>DB=EA H<A DFKC=H=FK> GFB ?@B@QQAQ AuADJH=FKtM FB H<AU @BA AuL

(4)

D@KKFH CA@Q I=H< RFBA >F?<=>H=D@HAC C@H@HU?A>T ]JBL H<ABRFBAM H<A C=vABAKH BJK> AKH@=Q H<A BA?A@HAC DBAL @H=FK FG ?BFDA>>A>M HFNAH<AB I=H< BA?A@HAC HB@K>GAB FG AuADJH@EQA DFCA SQA> fA=H<AB H<BFJN< ]w ><@B=KN ^ ` FB H<BFJN< H<A J>A FG >FRA NB=C DFR?FKAKH ^_h` >JD< @>H<A aww ^` DFR?FKAKH FGQFEJ>f =K FBCABHF NAH @ DF?U FG H<A AuADJH@EQA DFCA ?B=FB AuADJH=FK FG A@D< =KC=O=CJ@Q H@>eT ;<A>A HB@K>GAB> D@K BA>JQH =K DF>HQU >FGHI@BA =KHAB@DH=FK> I<=D< R=N<H EA @K FOAB<A@C HF EA DFK>=CABACM A>?AD=@QQU GFB ><FBHLBJKK=KN =KC=O=CJ@Q Au?AB=RAKH>T

jK

H<A DFKHB@BUM@>=>><FIK =K H<A KAuH >ADH=FKMIA>AHJ?@ >FGHI@BA@BD<=HADHJBAGFBH<AI<FQA CJB@H=FKFGH<AAuADJH=FK@KCCFKFHDFK>=CABBA?A@HAC S

QADF?UFBHB@K>GABT a>H<=>>FGHI@BA@BD<=HADHJBAJ>A> KA=H<AB ] w FB dw M=H => IAQQ >J=HAC GFB H<ANB=CT Y7 98 98:\7:7[8

A E@>AC FJB >FGHI@BA @BD<=HADHJBA FK H<A @>HABL FBeAB ?@B@C=NRM I<=D< R@eA> >D<ACJQ=KN A@>=AB H<@Ke> HF DAKHB@Q DFKHBFQT ;<A FBeAB => @??Q=D@H=FK >?

AD=S

DMI<ABA@> H<A @>HAB => eA?H @>NAKAB=D@>?F>L >=EQAT a> ><FIK =K ]=NT hM H<A @>HAB => >?Q=H =KHF

Worker

data stream

result stream

Worker

. . .

Distributor

Master

ResultPrinter

DataScanner

]=NJBA h wFGHI@BA@BD<=HADHJBAT

H<BAA ?BFDA>>A> ?BFDA>>A> @H@wD@KKAB @KC A>JQHL B=KHAB AK>JBA H<A =KHABG@DA I=H< EFH< >HBA@R>M J>=KN BA>?ADH=OAQU lmnm @KC opqrsnGFBR@H> DFBBA>?FKC=KN HF J>ABLCASKACC@H@HU?A>MI<ABA@>H<AH<=BC?BFDA>>M=>L HB=EJHFBM=KHAB@DH> I=H< H<A?FFQ FGFBeAB>AuADJH=KN H<AGJKDH=FK yzo{|lmnm

opqrsn

T BAD=>AQUMH<A=>HB=EL JHFB ?BFDA>> R@=KH@=K>@ Q=>H FGGBAAFBeAB>T <AK @ C@H@ =HAR => @O@=Q@EQAMFKA FG H<A GBAAFBeAB> => D<FL >AK@DDFBC=KNHFH<ADJBBAKH>D<ACJQ=KN?FQ=DUT ;<AKM@ RA>>@NADFKH@=K=KNH<AC@H@HF EA?BFDA>>AC =>>AKHHF H<A D<F>AK FBeABT =vABAKH >D<ACJQ=KN ?FQ=D=A> @BA ?F>>=EQAM A@D< @=R=KN @H @ ?@BH=DJQ@B DB=HAB=FK @EFJH BA>FJBDA J>@NAFBAuADJH=FK H=RAT <AK @ BA>JQHRA>L >@NA => @O@=Q@EQAM=>HB=EJHFB DFRRJK=D@HA> H<A BA>JQH HF A>JQHB=KHAB @KC ?JH> H<A DFBBA>?FKC=KN FBeAB

H<A DFBBA>?FKC=KN IFBeFK @ C=vABAKH FBeABMR@e=KN H<A >FQJH=FK RFBA G@JQHLHFQAB@KHT

dK FBCAB HF @E>FBE >?AAC O@B=@H=FK> =K EFH< @H@

wD@KK

AB @KC A>JQHB=KHABM EJvAB> @BA J>AC HF >HFBA C@H@ @KC BA>JQH =HAR> H<=> <AQ?> eAA? FBeAB> EJ>UT wF @> HF @OF=C AuHB@ DF?=A> =K>=CA H<A @>L HABM @QQ H<BAA ?@BH> FG H<A @>HAB ?BFDA>> @BA BJK FK H<A >@RA <F>H kKFHA H<@H H<=> @Q>F @QQFI> DFRRJK=L D@H=FK H<BFJN< ><@BAC RARFBUM @> ><FIK =K ]=NT tT ;<ABAGFBAM I<AK @ C@H@ =HAR => BA@C FK H<A C@H@

host

n

1

2

0 Worker

data stream

result stream

DataScanner

Distributor

Worker

. . .

ResultPrinter

]=NJBA @B@QQAQ =R?QARAKH@H=FKT

>HBA@RM =H => IB=HHAK FKQU FKDA =K J>AB >?@DA EAGFBA =H => >AKH HF @ FBeAB >=R=Q@BQU BA@>FK=KN @??Q=A> HF BA>JQH>T A>=CA> @OF=C=KN J>AQA>> DF?=A>M H<=> R@eA> >UKD<BFK=i@H=FKEAHIAAK=>HB=EJHFB @KCH<A =KHABG@DA ?BFDA>>A> >Q=N<HQU QFF>AB @> H<AU J>H DFR?AHA ><FBHQU HF J?C@HA =KCAuA> HF ><@BAC RARFBU >QFH>T a NBA@HAB KJREAB FG >QFH> N=OA> RFBA Q@u=HU I<=D< => J>AGJQ =K D@>A FGQ@BNA JDHJ@H=FK> =K H<A C@H@ >HBA@R FB =K H<A ?BFDA>>=KN H=RAFG=HAR>T

=H< KF =KGFBR@H=FK @EFJH JDHJ@H=FK> R@NK=L HJCAM H<A KJREAB FG >QFH> => @>>JRAC HF EA Q@BNAB AKFJN< >F @> HF @OF=C FBeAB> GBFR I@=H=KNT JA HF ?F>>=EQA O@B=@H=FK> =K H<A=B ?BFDA>>=KN H=RAM H<A FBCAB FG =KDFR=KN C@H@ R=N<H EA C=vABAKH GBFR H<A FBCAB FG H<A DFBBA>?FKC=KN BA>JQH> CAQ=OABAC EU H<A ?@B@QQAQ DFR?FKAKHT ;<ABAGFBAM?BFDA>> @H@

wD@K KAB @HH@D<A> = CAKH=SAB> HF C@H@ =HAR> H<BFJN<FJHH<A @>HAB=GH<A FBCAB=>>=N K=S

D@KHGFBH<ABA>HFGH<A@??Q=D@H=FKMH<A>A =

CAKH=SAB>

DFJQC ?F>>=EQU EA CAQ=OABAC HFNAH<AB I=H< H<A BA>JQH> EU >Q=N<HQU @C@?H=KN H<A BA>JQH >HBA@RT dK D@>AH<A @>HABEADFRA>@EFHHQAKADeI<AK =KDBA@>=KN H<AKJREABFGFBeAB>M>AOAB@Q @>HAB>DFJQCEAJ>ACM @> >JNNA>HAC EU ^`T

a>H<A >FGHI@BA@BD<=HADHJBAFG ad =>E@>ACFK H<A @>HABLFBeAB RFCAQM =H => KFH >AK>=H=OA HF R@L D<=KA <AHABFNAKA=HUT FBeAB> @BA Q@JKD<AC FKQU FKDAM N=O=KN @ D<@KDA HF @E>FBE H<A >H@BH J? FOAB<A@CT jK H<A DFKHB@BUMRF>H Au=>H=KN AKO=BFKRAKH>DFK>=CABH<A

(5)

GFBA H<A ?AK@QHU =KDJBAC GFB SKAB NB@=K ><FJQC EA QA>> =R?FBH@KH I<AK J>=KN FJB >U>HART

FRRJK=D@H=FK EAHIAAK ?BFDA>> =>HB=EJHFB@KC FBeAB> BAQU FK H<A d BADFNK=iAC >H@KC@BC I<=D< @QQFI> } pl}mnp qp~lq @KC opp}pq H<=> @OF=C> J>AL QA>> I@=H=KN>M A>?AD=@QQU GFB =>HB=EJHFBT ]FB =R?QAL RAKH@H=FK FOAB @ NB=CM IA D<F>A HF J>A QFEJ> I<=D< FvAB>@QQ H<ABAPJ=BAC >ABO=DA>M>JD< @>@JH<AKH=D@H=FKM =KGFBR@H=FK >ABO=DAN=O=KN DB=HAB=@ @EFJH BA>FJBDA>GFB >D<ACJQ=KN?FQ=D=A>@KC>FFKMHFNAH<ABI=H<H<AR?=D<L h =R?QARAKH@H=FK FG dT FBAFOABM d@QQFI>H<A CASK=H=FK FG CUK@R=D C@H@HU?A>MI<=D< <AQ?> eAA?H<A @>HABNAKAB=DT dKCAACMNAKAB=DRA>>@NA>EAHIAAK=>L HB=EJHFB @KC FBeAB> @BA ?F>>=EQA I=H<FJH @KU J>AB @>>=>H@KDAM @> QFKN @> H<A >FGHI@BA ?BFCJDH=FK D<@=K =>@EQAHF NAKAB@HA@JHFR@H=D@QQUH<A@??BF?B=@HA d GJKDH=FK>M

CAS

K=KN dC@H@HU?A>GBFR EFH< lmnm@KC opqrsn C@H@HU?A> ?BFO=CAC EU H<A J>ABT a> d ?BFL NB@RR=KN >HUQA BAPJ=BA> eKFIQACNA @KC Au?ABH=>AM IA BAQ=AOAH<AAKCLJ>ABGBFR H<AEJBCAKFGIB=H=KN @KC CAL EJNN=KN @KU d DFCAM@> ><FIK =K H<AKAuH >ADH=FKT

89 X 698 X7 \X7 897\Y X

;<A =>HB=EJHFB @KC FBeAB> <@OA HF AuD<@KNA d RA>>@NA> GFB EFH< C@H@ =HAR> BA@C GBFR H<A C@H@ >HBA@R @KC BA>JQH>HF EA IB=HHAK HF H<ABA>JQH >HBA@RT ;<J>M H<A NAKAB=D ?BFHFHU?A> GFB d RA>>@NA> <@OA HF EA @C@?HAC HF =KDQJCA J>ABLCASKAC C@H@ @KC BAL >JQH> I<=D< CA?AKC J?FK H<A @??Q=D@H=FKT AN@BC=KN H<A FBN@K=i@H=FK FG adM H<BAA ?=ADA> FG=KGFBR@H=FK RJ>H EA?BFO=CACEU H<AAKCLJ>AB C@H@HU?A>GFBEFH< H<A C@H@ >HBA@R @KC H<A BA>JQH >HBA@RM HFNAH<AB I=H< H<A GJKDH=FK HF AuADJHA FK H<AFBeABT ;<=> GJKDH=FK H@eA>HIF @BNJRAKH>I<=D<HU?A>@BAH<F>AFGH<A C@H@ >HBA@R @KC H<A BA>JQH >HBA@R BA>?ADH=OAQUT

a>IA=KHAKC FJB?=ADAFG>FGHI@BAHF EAFGHB@K>L ?@BAKH J>A GFB H<A AKCLJ>ABM =H => =R?FBH@KH H<@H @KU e=KCFGC@H@ DFJQC EA HB@K>GABBACGBFR H<A=>HB=EJHFB HF FBeAB>T FBAFOABM IA CF KFH I@KH H<A AKCLJ>AB HF EA DFKDABKAC I=H< FJB C=vABAKH >HBJDHJBA>T dK FBL CABHF CF >FM@ >?

AD=S

DDFR?=QAB <@>EAAKCAOAQF?ACHF DFKOABH C@H@HU?A>HF d C@H@HU?A> @JHFR@H=D@QQUT FBe> <@OA EAAK CFKA ^_` @QBA@CU HF ?BFCJDA d C@H@HU?A>GBFR C@H@HU?A>T gFIAOABMH<A>A >FL QJH=FK>Au?ADHH<ACAOAQF?ABHF =KHBFCJDA>?AD=SDDFRL RAKH> =K?BFNB@R >FJBDA>>F @>HF=KC=D@HAI<=D< HU?A HF NAKAB@HAT dK H<A ?BA>AKH D@>AM IA Au?ADH H<A J>AB HF N=OA @ >?AD=SD K@RA HF >HBJDHJBA> J>AC HF HB@K>GAB C@H@ GBFR H<A C=>HB=EJHFB HF FBeAB> @KC O=DALOAB>@T

EA AuADJHAC J>=KN FJB >FQJH=FKT dK H<=> D@>AMH<A J>AB FKQU KAAC> HF ?BFO=CA @K @Q=@> HF C@H@HU?A> J>AC GFB EFH< C@H@ @KC BA>JQH >HBA@R>T

aQRF>H AOABU C@H@HU?A D@K EAHB@K>Q@HAC GBFR HF d@JHFR@H=D@QQUT aH ?BA>AKHMHIF D@>A> k?F=KHAB> @KC JK=FK>t QA@C HF ?BFEQAR> @> H<AU <@OA KF APJ=OL @QAKDA =K dT ] FB H<A SB>H D@>AM BAN@BC=KN H<A G@DH H<@H @ ?F=KHAB => H<A @CCBA>> FG @KFH<AB O@B=@EQA =KL >=CA H<A O=BHJ@Q @CCBA>> >?@DA FG H<A ?BFDA>> =K I<=D< =H <@> EAAK CADQ@BACM ?F=KHAB> ><@QQ KFH EA HB@K>GABBAC GBFR FKA ?BFDA>> HF @KFH<AB FKA @> H<A=B RA@K=KN => Q=R=HAC HF H<A ?BFDA>> H<AU EAQFKN HFT ;<J>M H<A J>A FG ?F=KHAB> =K C@H@ >HBJDHJBA> HF EA HB@K>Q@HAC BA>JQH> =K @K ABBFBT dGM GFB @KU BA@>FKM H<=> IFJQC EA =KHABL A>H=KN GFB@K @??Q=D@H=FKM?F=KHAB> DFJQC EAHB@K>Q@HAC A=H<ABHF @K FB @K @BB@UFG T] FBH<A >ADFKCD@>AMFKA <@>HF BAR=KC H<@H@ =K BA?L BA>AKH> O@B=FJ> I@U> HF @??BA<AKC H<A >@RA RARFBU @BA@T FK>=CAB=KN H<@H H<A NAKAB@HAC d ?BFNB@R R@U BJK FK @K <AHABFNAKAFJ>?Q@HGFBRM=H => KFH ?F>L >=EQAHF CAHABR=K @H DFR?=Q@H=FK I<=D< BA?BA>AKH@H=FK ><FJQC EA J>AC GFB DFRRJK=D@H=FKT ;<J>M IA QAGH H<A HB@K>GFBR@H=FK FG >HBJDHJBA> GFB H<A GJHJBA @KC H<A=B DFR?=Q@H=FK BA>JQH> =K @K ABBFBT

;<A@JHFR@H=DNAKAB@H=FKFG dC@H@HU?A>GBFR C@H@HU?A><@>EAAKR@CA?F>>=EQAEU=KHBFCJD=KNAuL HB@ >HA?> =K H<A J>J@Q DFR?=Q@H=FK D<@=K @> ><FIK =K ]=NT c FK H<A KAuH ?@NAT ;U?=D@QQUM =K FBCAB HF R@eA >JBAH<ACASK=H=FK FG@N=OAKHU?A=>DFR?QAHAMH<ASB>H >HA? =K H<A DFR?=Q@H=FK D<@=K k GFB H<A ?BA?BFDA>L >FBt => AuADJHACT a> H<A NAKAB@HAC

SQA

=> HF EA DFRL ?=QACEU H<A AvADH=OA DFR?=QAB ktM=H RJ>H DFKH@=K @QQ H<A =KGFBR@H=FK BAPJ=BAC EU @KU HU?A=K H<A

S QA k=G KFHM H<=> SQA DFJQC KFH EA DFR?=QAC ?BF?ABQU EU tT ;<J>M =K>HA@C FG GAAC=KN I=H< H<A =KHABRAC=@HA SQA NAKAB@HACEU MH<=> =KHABRAC=@HASQA=>HB@K>GFBRAC EU k@>?AD=SD?@B>ABIAIBFHAtHF ?BFCJDA@ KAI SQA =K I<=D< H<A CASK=H=FK FG d C@H@HU?A> <@OA EAAK @JHFR@H=D@QQU@??AKCAC HF H<AFB=N=K@QSQAT ;<=> KAI >FJBDASQA=> H<AK ?BFO=CACHF T ;<A =KHABRAL C=@HA SQA I<=D< BA>JQH> => H<AK ?BFDA>>AC EU H<A J>J@Q DFR?=Q@H=FK D<@=KT

;<A Q=>H FG C@H@HU?A> GBFR H<A FB=N=K@Q SQAM HF EA HB@K>GFBRAC HF d C@H@HU?A>M => ?BFO=CAC FK H<A DFRR@KC Q=KA I<AK =KOFe=KN T aH ?BA>AKHMH<=> Q=>H => <@BC DFCAC @KC R@HD<A> H<A BAPJ=BARAKH> FG FJB >FGHI@BA >J=HAT gFIAOABM F?H=FK> @BA >D<ACJQAC HF @QQFI RFBA Au=E=Q=HU @KC BAJ>A EU FH<AB ?=ADA> FG >FGHI@BAT

;<A AKCLJ>AB R@U KFH EA @I@BA FG @QQ H<=> @> =H =><=CCAK EU H<A DFR?=QAB >J=HA?BFO=CAC GFB adT dK G@DHM =K FBCAB HF DFR?=QA <=> @??Q=D@H=FKM H<A CAOAQL

(6)

.c

cpp

.i

cc

.s

as

.o

ld

.a

ar

.c

cpp

.i

.h

a.out

Usual compilation chain

extra stage

mpipp

AIPE

]=NJBA c ;<A DFR?=Q@H=FK D<@=K I=H< adT

F?AB >=R?QUJ>A> H<A DFR?=QABI<=D< => RABAQU @ IB@??AB @BFJKC H<A DFR?=QAB =K I<=D< HIF AuHB@ >H@NA><@OAEAAK @CCAC GFBH<A @JHFR@H=DHB@K>L GFBR@H=FK FG C@H@HU?A>HF d C@H@HU?A>T

8Y89 X : 96[8 X 76

;<A>APJAKH=@QDFR?FKAKHD<F>AKGFBH<AAO@QJ@H=FKFG H<A >FGHI@BA ?BFCJDH=FK D<@=K =HAB@HA> H<A Au?FKAKH=L @H=FK FG >PJ@BAR@HB=DA> FG=KHANAB> BA@C FK @ >HBA@RT wF H<A=K=H=@Q DFR?FKAKH=KDQJCA>H<BAAGJKDH=FK> H<A GJKDH=FK yzo{| }~ lmnm zrn

opqrsn

I<=D< R@eA> H<A DFR?JH@H=FKM H<AGJKDH=FK H<@H BA@C> @ R@HB=u FK H<A >H@KC@BC =K?JH @KC >HFBA> =H =K @ lmnm >HBJDHJBAM @KC H<A GJKDH=FK H<@HIB=HA>H<A DFKHAKH> FG @ opqrsn>HBJDL HJBA FK H<A >H@KC@BC FJH?JHT ;<A lmnm C@H@HU?A =>

0

500 1000

1500

2000

2500

3000

3500

4000

0 1000 2000 3000 4000 5000 6000 7000 8000

Ellapsed time (secondes)

Matrix Exponent

sequential

2 processors

4 processors

8 processors

]=NJBA QQ@?>AC H=RAT

@ >HBJDHJBA =KDQJC=KN H<A DFKHAKH> FG H<A R@HB=u @KC H<A Au?FKAKH O@QJAM I<ABA@> H<A opqrsn C@H@HU?A => @ >HBJDHJBA=KDQJC=KN FKQUH<A DFKHAKH>FGH<ABA>JQHR@L

;<A HAD<K=PJA <@> EAAK >JDDA>>GJQQU @??Q=AC HF O@B=L FJ>Au@R?QA@??Q=D@H=FK>=KOFQO=KN RFBA>F?<=>H=D@HAC C@H@HU?A>T

;<A?Q@HGFBR J>ACGFB?ABGFBR@KDARA@>JBARAKH> => DFR?F>AC FG A=N<H AKH=JR d BJKK=KN @H hT gi I=H< _h FG a RARFBUT ;<ABA => KF ]w SQA><@B=KN @RFKN H<A>A DFR?JHAB>M @KC KF dw J>AB> C@H@E@>AT QFEJ> hTcT => J>AC HF R@? H<A J>AB FK @ QFD@Q @DDFJKH FK A@D< DFR?JHABT

0

1

2

3

4

5

6

7

8

0 1000 2000 3000 4000 5000 6000 7000 8000

SpeedUp

Matrix Exponent

8 processors

4 processors

2 processors

]=NJBA w?AACLJ? T

dK FBCAB HF @OF=C <@BCI@BA F?H=R=i@H=FK AvADH> =K H<A DFR?JH@H=FKM FCC =KHANAB> @BA J>AC GFB R@HB=u AQARAKH> @KC H<A R@HB=u >=iAM@> H<A J>A FGAOAK =KHAL NAB> IFJQC QA@C HF iABFA> H<BFJN< FOABFI> GFB Q@BNA KJREAB>FG=HAB@H=FK>MI<AK=KDBA@>=KNH<AO@QJAFGH<A Au?FKAKHT ;<A >=iAD<F>AKGFBH<AR@HB=DA>=> __M@KC O@QJA> GFB Au?FKAKH> @BA H@eAK =K H<A B@KNA GBFR h HF T ]=NT ><FI> H<@H H<A AQQ@?>AC H=RA O@B=A> Q=KA@BQUI<AK H<AAu?FKAKHO@QJA=KDBA@>A>T dKG@DHM=K H<A?BFNB@R IA<@OAD<F>AK@>@K Au@R?QAMH<A<=N<AB H<A Au?FKAKH FG H<A R@HB=uM H<A Q@BNAB H<A @RFJKH FG DFR?JH@H=FK <FIAOABMH<A @RFJKH FG DFRRJK=D@H=FK BAR@=K> H<A >@RA @> H<A >=iA FG H<A BA>JQH R@HB=u => DFK>H@KHT ;<J>MH<=>DJBOA><FI>H<@HFJBRAH<FC D@K C=>HB=EJHA DFR?JH@H=FK> AD=AKHQU FOAB H<A KAHIFBeT ]=NT ?BA>AKH>H<A >?AAC J? OAB>J> H<AAu?FKAKHJ>AC HF AO@QJ@HAH<ABA>JQHR@HB=DA>T dH <=N<Q=N<H>H<@HOABU NFFC >?AACLJ? D@K EA ?BFO=CAC k @K AD=AKDU FG J? HF =>@D<=AOACGFBA=N<H?BFDA>>FB>tAOAKI<AKH<A @RFJKHFGDFR?JH@H=FK=>B@H<ABQFI kATNT @KAD=AKDU FG =>@D<=AOACFK A=N<H?BFDA>>FB>GFB@K Au?FKAKH FGh=KI<=D<D@>AH<ADFR?JH@H=FKH=RAGFB@>=KNQA

(7)

X X

dK H<=> ?@?ABMIA <@OA?BA>AKHAC H<A >FGHI@BA?BFCJDL H=FK D<@=K adT =OAK @KU >APJAKH=@Q >FGI@BA DFRL ?FKAKHM =H AK@EQA> H<A AKCLJ>AB HF FEH@=K @ ?@B@QQAQ AuADJH=FK FK @ NB=C @KC Au?QF=H H<A C@H@ ?@B@QQAQ=>R =KOFQOAC =K <=> @??Q=D@H=FKT >=KN adMH<AAKCLJ>AB CFA>KFH<@OAHF R@eA@KU?@B@QQAQ?BFNB@RR=KNAvFBH HF @C@?H H<A @??Q=D@H=FK =K FBCAB HF EAKASH GBFR H<A >U>HART u?AB=RAKH> <@OADFKSBRACH<@H H<A?@B@QQAQ AuADJH=FK D@K EA AD=AKHT

AHM J? HF KFIM H<A D<F=DA FG H<A GBAA FBeAB EU H<A =>HB=EJHFB ?BFDA>> <@> EAAK eA?H OABU >=RL ?QAM RFBA >F?<=>H=D@HAC >D<ACJQ=KN RJ>H EA DFK>=CL ABAC =K H<A GJHJBAT ;<A DAKHB@Q=iAC DFKHBFQ FG H<A @>HABLFBeAB?@B@C=NR R@eA>H<A=KHANB@H=FKFG@KU >D<ACJQ=KN >HB@HANU =K =>HB=EJHFB PJ=HA >HB@=N<HGFBL I@BCT ;<AJ>ABR=N<HAOAKEAN=OAKH<AF??FBHJK=HUHF CAD=CAH<AHB@CAFvEAHIAAK?ABGFBR@KDA@KCBA>FJBDA J>@NAH<@H EA>H >J=H> <=> KAAC>T

a?@BH GBFR >D<ACJQ=KNM FH<AB IFBe> @BA =K ?BFNBA>> HF =R?BFOAH<A >D@Q@E=Q=HU FGH<A >FGHI@BA@BL D<=HADHJBAT ] FB =K>H@KDA FK H<A ?ABGFBR@KDA >=CAM =K D@>AH<A @>HABEADFRA>@EFHHQAKADeI<AK =KDBA@>=KN H<AKJREABFGFBeAB>M>AOAB@Q @>HAB>DFJQCEAJ>ACT ]@JQHLHFQAB@KDA=>@ C=BADH=FK GFBGJHJBAIFBeMHFNAH<AB I=H<?FBH@E=Q=HUH<BFJN<H<AJ>AFG@ O=BHJ@QR@D<=KAT

4: X Y Z X76

a DFQQ@EFB@H=FKEAHIAAKH<A AKHBA@H=FK@Q CAKFL HU?@NA k t@KCH<AdK>H=HJH @H=FK@QCA> ;QDFRL RJK=D@H=FK>kd;t>H@KC>@HH<A=KDA?H=FKFGH<Aad ?BFADHT ;<ABAGFBAMIAIFJQC Q=eA HF @DeKFIQACNAH<A >H@v FG H<A E=F=KGFBR@H=D> NBFJ?T >?AD=@QQUM IAI=>< HF H<@Ke =KF @BNAH=DMFR=K=PJA@K AK BFADe @KC ] ABK@KCF F> w@KHF> GFB H<A=B >J??FBHT

8 X :6 ! " " # $ %& " ' ()*+(), -. ./ 0)***12")*** ) 3 4 32 / #5 . 6 6 7 8 97 8 :# ; 5& / " 7 #5 #5! <=== >?@ABCDE)FG)H&,+),I7 FFJ K 3 L / " ! #5 " 7 . 6 0*M 85 ! !! "N" " 7 M IL I2 ; OEPQ >?@R ABCPSTU VWXPST CYD OZ?[WZ <S\EW]CEB^CBED W _DWZPC` 81" )**K ( I L 6 85 3a 8 3 1 I / I7 b / " / / / $/ / 7 :a a" ! 7 7 / <=== cEWS]W^CP?S] ?S dWEWZZDZ WSQ eP]CEP[BCD Q >?@ABCPST MGMH&KJF+ K,) )**K

J L

- I 3 ;" /

;7"5& a ""a" #" 7 /285 -%b . 7 / 1 ( FFF f 3 3a;84 g ;b.!3 1.-3 ! " " 7 *F+J 373 7 )-hhh% 31-! / 7 G336-0*)H L ! 1 )!)M )**) , - I 3 7& " ! 7 7"7 5 <SCZ ij kBADE^?@ABCDE lAAZP^WCP?S]G)H&(+), FFf F - I 3 cYD OEPQU m

ZBDAEPSC\?E WnDo >?@ABCPST <S\EW]CEB^CBED 1! 7 .7 FFF * 7 p 1 1" . a 71 75& ; " 7" " 2 "!7"7 7 1 .- - . 3/75nDCo?EXR[W]D Q dWER WZZDZ >?@ABCPSTj >?@@BSP^WCP?Sq lE^YPCD ^CBEDq WSQ lAAZP^WCP?S]a7 KJ) rD ^CBED n?CD] PS >?@R ABCDE k^PDS^DF,+*Fs$s7 FF, / !s # : t7 k^WZW[ZD dWEWZZDZ >?@ABCPSTU cD ^YS?Z?T`q lE^YPCD ^CBEDq dE?TEW@@PST 1"#! -"$# 95$9 g / FF, ) 9 ;5 $5 / /57" ; /747 7 / 175$ !& " 6.3! # " ! 7 i?BESWZ ?\ OEPQ >?@ABCPST &M +()**K K 19# 1 1"36L# 6Is 8p a" 7 " " " 2 7 ""N" #- O_<eMJ+(f )*** M 897 32 a% / 1& ! 7" 7 >?S^BEEDS^`U dEW^CP^D u=vR ADEPDS^D FGH FFf ( t : / " . " 7 5 "a a" 1/) . " - 85 1# . " G1. )**MH )K-. "3 7 3 ! 7"85 G-.333 )**MH )**M