HAL Id: inria-00070693
https://hal.inria.fr/inria-00070693
Submitted on 19 May 2006
HAL is a multi-disciplinary open access archive for the deposit and dissemination of sci- entific research documents, whether they are pub- lished or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.
L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.
Portable Runtime Support and Exploitation of Nested Parallelism in OpenMP
Panagiotis Hadjidoukas, Laurent Amsaleg
To cite this version:
Panagiotis Hadjidoukas, Laurent Amsaleg. Portable Runtime Support and Exploitation of Nested Parallelism in OpenMP. [Research Report] RR-5307, INRIA. 2004, pp.12. �inria-00070693�
ISSN 0249-6399 ISRN INRIA/RR--5307--FR+ENG
a p p o r t
d e r e c h e r c h e
Thème SYM
INSTITUT NATIONAL DE RECHERCHE EN INFORMATIQUE ET EN AUTOMATIQUE
Portable Runtime Support and Exploitation of Nested Parallelism in OpenMP
Panagiotis E. Hadjidoukas — Laurent Amsaleg
N° 5307
Septembre 2004
Unité de recherche INRIA Rennes
IRISA, Campus universitaire de Beaulieu, 35042 Rennes Cedex (France)
Téléphone : +33 2 99 84 71 00 — Télécopie : +33 2 99 84 71 71
*+-,.- /0&1 - ,& 3245-768
9;:=<>:=?=@BA=CD@BEFGIH:KJMLN@BJOA=POQM:=ERIST:=POUNVW<.CXYZE[:=\]VW?
^`_Mabdc1eWfhgjikemlonqpraNbscDnTnrlWbtKuwvyx{zW|}cNn
~Iucp-^cobdc
TMKuwqpToccN%_}cN%_Mc=>wWikemcNoprcNb1t}c1wiDMcNn
&N} T¡ 1pr_}x{nOMKcNN¢£¤c¥}cNnrcWp>svyprcMprx§¦cIx§bd}vycNbdcWpprx§u1u¨}pr_Mc`©;ª«©'¬'e¬;=cMg~
|}Wprx§bdcv§xyt}%l¯®]©-pr_K°±x§tK²-p_Mp«pcpn-=urpt}x§vyxypl7Mc³´x§cWp;nr|}}=urp«u¨Ob|}vyprx§}vycvycN¦cNv§n-u¨
K%v§vycNvyx{nqbµ;¶cs_M[¦cxybd}v§cbdcNwpcNpr_}csr|Mwpxybdcvyx§t}%xycDn«u¨¥[¦[xyv{tMvycuwKcNo·¸nquw|}cs¬;=cMg~
ubd}x§vycNnOuw1pudu¨.©-p_M°±x§t±¢wrcDo|Mx§}hpr_m|Mn>pr_}cNxyIu¦wc_}cNw}n&Ms}u¦mx§ox§}hpr_Mcb¹£-xypr_dx§}_}cNrcNwp
nr|}}=urpT¨ºuw«}cDnpcNMvyv§cv§x{nqbµ
¡
¯Mox»pxyuw±¢=£¼cMrcDnqcNwp«co=cxybdcNwp%vxybd}v§cbdcWp%prx§uu¨
p_}c`£¼uwr½ozW|}cN|}xyM«bduoocv}Mp_}c`M%v§v§cv§xy¾Dprx§u1u¨K'}phv§|MnpcxyMhvywux»p_}b|Mnrxy}¬;=cMg~
}xycNprx§¦cNnNµ>^`_}cnrlmbsbdcprl´M7}u}·¿ocprcNrbdx§}x§nrbÀu¨Ápr_Mx§nTvywux»p_}bÂ}cDcNnnrx»p%prc;pr_Mcco}v§uxyp·
pxyuwdu¨.x»p%nI}cDnpcNsv§uWuwo·¿vycN¦cvMM%v§vycNvyx{nrbµ&^`_}cTcm=cx§bscNWpvMrcDnq|}vypnIueog~Znrc¦cNO£-x»p_d¨ºu|}
MruocDnrnru%n¤ocbduKnpprc'u|}`c³xycNWph¬;KcNMg~$r|Mwpxybdcnr|}}=urpNµ
ÃÄWÅÇÆÈ1É Ê& ¬;=cKg~O¢M|}Wprx§bsc'v§xytMrlw¢oM%v§vycNvyx{nrb¢}eog~
ËKÌTÍ%ÎÐÏwÍ%Ñ[Ñ[ÒÔÓqÕ]ÕBÓÕrÖר}Í%ÙÍ%ÚÛÎÐÜ%Ý]ÎÐÕrÞßÍ%Ñ%àÔÎÐÑ[ÜÛá[â%Í%ÕqãäoÍ%á[ÒÔÓÙÝqÞå&Ì-ÕBÍ%ÏÐÓqÚNæ%ç&ÎÐÒÔÎÐÕ¿ÍDÞèyÒ
%M-#"-(2Â-68
©Tuw|Mn}NnrcWpuMn}Mncpqpx§vyc|}Mcxybd}vbdcNwp%pxyuwvyprcMprx§¦cocNn´t}xytMvyx§u·
p_}aNzW|}cDnIcN|oprx§u©;ª«©h¬'e´¬;=cKg~®Ô©Tpr_M°ÁxytK²µ¼cprprcTx§bd}vNbdcWpprx§u'Kuw|}IuwtocNprxy¨.cN
wruwx»prc¤v§h=urpt}xyv§xyp-cp¥}c¤MruwKuWnqcN>|}nq|}MKuwqp>c³´Nc`=u|}¥vyc`Mvyvv§x§nrbdc`b1|}vyprxy·BMxy¦wcN|}ǵ
©«u|Mn-[¦wuMn¼x§bsMvNbscNWpv§cNn¤vyx§t}%x§rx§cNn`ox{nrKuw}xytMvycDn`cÛuwKcNo·¿nru|Mc`=u|}-v§cNn-ubd}x§v§prcN|}n`oc
¬;=cKg~ |7}cNnnq|Mn-}cNn-©-p_M°±x§t±¢}N}|}x§nWp-x§Mnrx.vycN|}n`uop%n`cp-cvycDn-oup%WpToc'v§s=uwnnqx§t}x§vyxyp
u=xy-o|Mvyvv§x{nqbdc«x§b1tMrx{zW| µ"!;c'}vy|Kn¢}Mu|Mn¤}NnrcWpruwMn¤|}}c'x§bsMvNbscNWppxyuw7co#Nrx§bdco·
p%v§c|} bduooav§c7}c$Mv§cNnpqpcWprc7cps|Mnnqx¼|}}c´x§bd}vNbdcWpprx§u Mvyv§av§c´|}Zvywux»p_}bdc
}cv{nnrx%$KNpxyuww&wc('v)xyMnrcrprx§u}cox§rcDÛpxy¦wcNn;¬;=cMg~&µ.°ÁctKcDnquwxyoc1Mvyvv§x{nqbdcb1|Mv»px»·
Mxy¦wcN|}ZcNnqp´Mrprx{|Mvyx§acbdcWp DcNnnx§rc}Mndvycn}ccprprc¯}}v§x§Npxyuw u+* v)u uwMnp%pc
|M}c¨ºurprcwnrnrlmb,prxycocDndpr%xyprcNbdcWpncp´|}cNqp%x§ x§MprcNrbdx§}x§nrbdc¯}MndvycDn´v§|}v{nµ0°ÁcNn
¦vyx{}pxyuwMnco#Nrx§bdcWpvycDn1nr|}|MZnqcNr¦wc|}seog~-'¯¯}uocDnrnrc|}%nNbduWprcWp1v)c³´x»p7oc
v)xybd}vbdcWp%prx§u}u=uwncwµ
.¹É D Æ / 7 ¬;=cMg~O¢}cN|}prxy¨¢}M%v§vv§x§nrbdc¢}eog~
< =
ÁO ¥
ª«v»p_}u|M_}cDnpcNsM%v§v§cv§x§nrb¹x{n¥vyv§u£¼cD1tml´¬;KcNMg~?>§@¿¢WbsuWnp¤¬;=cKg~ x§bsMvycNbscNWppxyuwMnIou
Mup¼Mru¦mx{oc`pr_}c;}cNcNnnrrlsnq|M}Kuwqp`Mpr_m|Mn¼}cNnqprcDdM%v§vycNvMcwxyuwMnIc«vy£`[lon>cocN|oprcDtmls
nrx§}v§c`pr_}cNw.µO^`_}c ¡Wprcv MA}uwqp¬;=cKg~ uwbsMxyv§c%n>Msp_}cAM|x»p%nq|~¼ ¡ gCBI~¤¬T¶?B¥
AMurpr%´ubd}x§vycN¥}u¦mx§ocTpr_}x{n¼¨º|}MÛpxyuwMv§xyplµ>^`_}c;}c¨Ô|Mv»p«¬;bd}x=ubd}x§vycND>§N@Çnr|}}=urpn¤v§xybs·
xyprcD5¨ºub u¨-}cNnqprcD M%v§vycNvyx{nrb¢&_}u£¤c¦wc¬;bd}xFEeo^G>yN@¿¢O co=cxybdcWp%vI¦wc%nqx§u u¨;¬;bd}x
cDzW|}xyMKcD £-xypr_ pr_}ceWp%%½o^`_}cNMn1Eg~vyx§t}%l¢>x§bd}vycNbdcWpndc³xycNWprv§l }cNnqprcD x§rcw|}v§1Mq·
vyv§cv§x{nqbµ^`_}c©«ª«©'¬'e uwbd}xyv§cH>y@¤v{nru¯nq|M}Kuwqp%n}cDnpcN cwxyuwMnK ¬;KcNMg~ cmprcKnqx§uMn
¨ºuwT}uocNnnru`u|}Knµ¥^`_Mcr|Mwpxybdcnr|}}=urp«u¨&pr_Mc1©«ª;©h¬'e ¼ubd}x§v§cTx{n-}u¦mx§ocDtWlc³d·
xycNWp¼|Mnrcr·Bv§c¦wcvop_}cNMnIv§xyt}%l7®]©-p_M°±x§tK²Û¢o£-_Mx§%_´£`n¥xyMx»px§vyv§loc¦wcv§u=cNdx§pr_}chuwwpcmp¼u¨.pr_}c
©;ª«©'¬'eMrucNÛpI>JK@¿µ
^`_}cbxyuwwprx§t}|opxyuwMn`u¨&pr_Mx§n`MKcN«c;pr_}c'¨ºuwvyv§u£-xyM L
M
ªTvyprcNrMprx§¦cTx§bd}v§cbdcWpprx§u7u¨±p_}c©«ª«©'¬'e|}Wprx§bdc;v§xytMrlsx§n¤}cNnrcWpcN.µ>ªTvypr_Mu|}w_
x»pOnq_KcNn=pr_Mc¥nrbdcI%%_}x»pcNpr|}c>£-x»p_'pr_}c¥Mpxy¦wc>xybd}v§cbdcWp%prx§uÁ¢Dxyp±¨ºuwvyv§u£Tn±nrubdcIox=cNrcNwp
ocNnrx§¯ocDx{nqx§uKn-prwcpxy}Kuwqp%t}x§vyxyplM=cr¨ºubMcprx{%½mn«rcNv§prcDpru|MMnpr|KÛpr|MrcD
b1|}vyprx§v§c¦cNv±Mvyv§cv§x§nrbµ
M
^`_}c|}Wprx§bscnr|}}=urpu¨`[¦[xyv{tMvycdu=c}·¿nru|}%c´¬;=cMg~ubd}x§v§prx§u5cm¦mxyu}bdcNwp%nhx§n
xybd}v§cbdcNwpcN5u5pu5u¨-©-pr_K°±x§t5£-xypr_Mu|opbduoox$Kprx§uMn'pupr_}c´ubd}x§vycNnMp_}c´uooc
pr_}cNl}ruoo|Kcµ`^`_}cco}vyuwx»p%pxyuwu¨>|Mnrcr·Bv§c¦wcv.b1|Mv»px»p_}rcDox§}Mnrubdc}}x»pxyuwMv.¨ºcD·
pr|}cNnTxyWpruoo|McN7xy©-p_M°±x§trcDo|Mc;p_}c¬;=cMg~$|}Wprx§bdcu¦c_}cD}nNµ
M
~OurptMvycr|Mwpxybdcnr|}}=urps¨ºuw}cNnqprcDZK%v§vycNvyx{nqb x{ns}u¦Wx{ocD tmlZ uwbsbduwZcNnrcN%%_
}v{pq¨ºuwrbÀpru7¬;=cMg~ ubd}x§vycNnNµ
M
ªT cm=cx§bscNWpv>x§bd}v§cbdcWpprx§u u¨¼p_}c7¬;KcNMg~£¤u½ozw|Mc|}x§}bsuoocNv>§D@¼u5pruw5u¨
©-p_M°±x§tx§n-v§nrud}cNnrcWprcD.¢}n`d}umu¨u¨&uMcop-£-xypr_cNnrKcDÛp`prud|}Wprx§bscnr|}}=urpNµ
M
A&xyMvyv§l¢Wp_}c;K%v§vycNvyx§¾Nprx§u±¢w|Mnqx§}¬;KcNMg~ }xycNprx§¦cNnN¢Wu¨Á_}xycN%_Mx§NvM}pv§|MnpcxyM
v§uwrxypr_}bÀpr_MpT}cNcNnnqxypprcNn¼p_}cco}v§uxypprx§uu¨b|}vyprx§vycN¦cvÁM%v§vycNvyx{nrb¢mx{nTocDnrrx§tKcD.µ
^`_}chcNnqp¼u¨.p_}x{n¼K=c¤x§n¤uw}x§¾cNdwnI¨ºuwvyv§u£TnL>emcDÛprx§unr|}bdbdrx§¾cDnIp_}c'ocNnrxywocDx{nqx§uMn
M¯¨ºcNp|}cNn'u¨¤u|}x§bd}v§cbdcWpprx§u5u¨`©-pr_K°±x§t±µemcNprx§u ocNnxyt=cNnhp_}cx§bsMvycNbscNWppxyuwu¨
[¦xyv{t}v§c1uwKcNo·¸nquw|}cd¬;=cMg~ v§x§t}rx§cNn;upruwu¨¤©Tpr_M°Áxyt±µemcDÛprx§u5}rcDnqcNWpnhu|}'Mvyv§cv
Mpv§|MnpcxyM´v§uxypr_}bµ!B>o=cxybdcWp%vÁcNnr|}vypn;cxyMvy|KocNx§emcDÛpxyuw¯oµA&xyKv§vylw¢ÇemcDÛpxyuw
N ox{n|MnnqcDn`u|}`uw}uwxy}s£¤u½=µ
O
QP Û&SR
^`_Mc1©«ª;©h¬'e|}Wprx§bsc1vyx§t}%l®Ô©Tpr_M°Áxyt=²I>§NK@O}u¦Wx{ocDn¤p£¼u´oxÇccWp«}xybdxyprx§¦cDn-pru7nqM[£-Mq·
vyv§cv§x{nqb¢K}c=cMox§}7upr_Mc1_}x§c%%%_ml´vycN¦cNvÁx§£-_}x{%_pr_}cs}}v§x{pxyuwx§n«£¼uwr½mx§} L¥pr_}csoccNKcDnp
v§c¦wcvmx{nwc}cNprcD£-x»p_£¤u½'ocNnxy}pru%nN¢«}phnpr|Mpr|}c¼pr_MpOuWpxyMn&=ux§Wprc%nÁpru-¨º|}KÛprx§u
T8T ÙU VWX'Y
Mx»p%n|}bdcNwp%n'MrcDnq|Mv»p%n'xy5£¼uwr½cocN|}prx§upr_}u|}w_57¨º|}KÛprx§u Nv§v>£-x»p_}u|opvyv§u£-xyM
¨º|Mqp_}cnrM[£-}x§}¯u¨«M%v§v§cv§x§nrbµTx§_}cNv§c¦wcv{nsrc7cN}c%pcN |Mnrx§}K}upr_}cNw}n¢>5ox=cNq·
cNWpxyWprcNq¨Ôwcd£-_}x{%_ }ru¦mx{ocNnhp_}cNMn'£-x»p_ nqpw%½Çµ©«}up_}rcD}nh}u¦mx§}cd }ocNnnnrMc
¨ºuw;Mrx§¦prc¦x§t}vycDn«Mt=c|MnrcNp'v§vv§c¦wcv{nµ'^`_}cx§'ocNnxy}pru;x§n'cmprcNMocD¦cNnrx§u
u¨¥£¤u½ocDnrrx§opu'M¯x§n'v§£`[lmn;v§vyuoNprcDx§p_}cnp%%½u¨Ip_}cdnnquox{pcN|KnqcNq·¿vycN¦cNv±pr_MrcD.¢
|Knqx§}7nrxyMv§cbdcbduwrlv§v§umNpxyuw¯vyvOx§5Nnrc1p_}cx§'rcDlov§xyM7zW|}c|Mcsx{nhcbdoplwµ1©«}up_}rcD}n
rcnr|}t}bdxypqpcN¨ºuw«cmcD|opxyuwxy¯rcDolzw|Mc|}cDn¢=£-_}x§vyc£¼uwr½ocNnxy}pru%nTrcx§MnrcrprcNx§¯nrKcDx{v
v§uoprx§uMn¤x§nr_MrcDbdcbduwrlwµ
^`_}c-v»pcMpxy¦wc¼x§bd}vycNbdcWpprx§usu¨K©Tpr_M°Áxytd}cNnrcWprcDx§1p_}x{nOM=cInrc¦cDnwn&'nr|}}}v§cbdcNwp
u¨«p_}cMpxy¦wcuw}cµ$^`_}cbx§Z¨ºcDp|}rcDnspr_Mp´ox%Çcx§ u|}´Nnrcrc7pr_}cnrx§}x$KWprv§l _}xyw_}c
=urpt}x§vyxyplM´pr_}c}uopxyuw´u¨v{¾lnqpw%½dvyv§uoprx§u´=uv§x§lµ AMu¼pr_}c'cNnqp¼u¨Ápr_}x{n-ouo|}bdcNwp
p_}cpcb©Tpr_M°Áxyt0£-xyv§v«c¨ºcNcxypr_Mc´pu pr_Mx§n7vyprcMprx§¦cx§bsMvycNbscNWppxyuw upruZnrubdccN}c%v
¨ºcDp|}rcDn`}cNnrcWpTp-tKupr_wnqcDn¢}Mml´c¨ºccKc;pruMprx§¦c©-p_M°±x§tx§n-bdcNwpxyuw}cN7co}v§x{xyprv§lµ
^`_}c_Mxyw_=urpt}xyv§xypl¯npcbnh¨ºub p_}c|Mnrcsu¨¥p_}c~¤¬'e ¡ nqpM}%.µ ¡ MnqprcNw¯u¨¤|Knqx§}M·
pxy¦wc´½cNrMcvy·]p_}rcD}nN¢.p_}c¦mx§qp|MvIMruocDnrnru%nc´nrlmnqprcNbs·¿nuwKcd~¤¬'e ¡ pr_}cNw}nNµ^`_}x{n¨ºcDp|}rc
x§bd}u¦cNn¥©-pr_K°±x§tn¤x§Wprcu=c%tMxyv§x»pl£-x»p_´p_}xy%m·¿Mrplsvyx§t}%xycDnIM¨Ôx§v§x»p%prcDnIp_}c;u·¿cox{npcMc
u¨¤¬;KcNMg~ M~¤¬'e ¡ ^`_}rcD}n;x§pr_Mcsnbdc1Mruw%bµ ¡u|}hnrc¢Ç|Mnrcr·¿vycN¦cvÁpr_}cNw}nhc
Mru¦mx{ocN1pru1©Tpr_M°Áxyttml1pr_}c«MocNrv§lmxy}^`_MrcD}nI°ÁxytMrl®-p_M°±x§tK²Û¢m£-_}x{%_drcN}v{cDn&pr_}c;nnqcNb·
tMvyltMnrcN;|}x{%½o^`_}rcD}nOK%½wcµ ¼uwWprcmpr·¿nr£-x»p%%_}x§}'x§n>x§bd}v§cbdcWprcDd|Mnrxy} "!$#%'&)(*,+ -!$#
Nv§v§n`uw/.10(*, 23du=c%pxyuwMnNµ&ª«|MMocv§lWx§}spr_MrcD7x§n-wÛpr|Kv§vyldnqpw%½´£-_}cchpr_}c£¤u½Ç¢mx]µcµ
u|}prx§}c¢wu¨±'MMupr_MrcDdocNnx§opruw>x§n¥cocN|oprcD.µIemxyKcTpr_M°Áxytnr|}}=urpn>pr_}cNwu|}prx§}cNn>£-xypr_
dnrxyMv§c|}bdcWpD¢oM}upr_}cNwu|}prx§}cNn-rc'cocD|opcN7pr_}u|M_}Mruw}rx{pc;Mru[olu|opxy}c
p_MpTcNcx§¦cDn¤wn-|}bdcNwp¤pr_McucNnrKuwMox§}ocDnrrx§opuDµ
ª«}up_}cTx§bdKuwqp%Wp«ox=cNrcNMc1x§n`p_}c}uopxyuwu¨Odv{¾Nl7nqpw%½v§vyuoNprx§u=uv§x§l L>pr_}cMp
nqpr|Mpr|}cNn&¨ºuw>M}upr_}cNw}nOMs£¤u½ocNnx§opruwn>rc¤x§}cWprx{vMMp_}cTnp%%½¨ºuwIhM}upr_}cNw
x{n`v§vyuoNprcD|Mnqp-tKc¨ºuc;xypn`cocN|}prx§u±µ ¡}xyprx{vÇbdupxy¦pxyuw7u¨±pr_Mx§n-ocDnqx§ocNx§nrx§u£`n¼xypn`bdxyMx»·
bvWbdcNbsuwrluMnr|}bdoprx§usMpr_}c`cDnrxycNÁbMcNbscNWp&u¨MM%v§v§cv§x§nrb useou¨p£`c !«x{nqprxyt}|}prcN
eo_McNgcbdul®]e!'eog¯²&v§|MnqprcNn >N ¢mD@¿µOeoxyMc¤nr_MrcDbdcbdul'u=c%prx§uKnÁc¥¦wcl'coKcNMnrxy¦wc
uwv§|Mnqprc%nN¢K£¤u½7ocDnrrx§opu%nTcxyKnqcNqpcNx§pr_}cr|Mwpxybdcszw|Mc|}cDnµ ¡ uw}c«MupTpu_Mrbpr_}c
bduoo|}v{xyplu¨Iu|}hx§bsMvycNbscNWppxyuw±¢Ç£¼c1¨ºuv§vyu£ p_}cdnrbscs}MruW%_u5nq_MrcDm·¿bscNbdulb1|Mv»px»·
MruocDnrnru%nNµÁ°Á¾ldnp%%½vyv§uoprx§ud_Mn¼v§vyu£¤cNs|MnIprux§Wpruoo|Mc 0145769(" 98-¢ouwoprx§bdxy¾Dpxyuw
p_MpvybduWnpcNzW|Mvyx§¾cDnspr_}c|}Wprx§bdcu¦cNr_}cD}nt=cp£¤cc$M}up_}cNMnsM £¼uwr½ }cNnxyopu%nµ
ª-Nuwox§}dpu=ccr·Bpru·¿=cc;nr%_}cDo|}v§xyMM¢Ç $M}x{nq_McNM}up_}cNMx§%½on-p_}c1McmphocDnrrx§opuD¢Kc·
prcDnsZ|}Kocvylmx§}pr_}cNwEnqpw%½¯¨ºuwp_Mp´ocDnrrx§opudM nq£-xyp%_}cDnprupr_MpNµ ©«ubv§vylw¢>pr_}c
|MMocv§lWx§}p_}rcD5x{nrcDlov§cN¨ºub+ }Mruw}rx{pczw|Mc|}cwµ:;nqx§}nqp%½_MKou O¢& $K}x§nr_}cD
K}upr_}cNwnqx§bd}v§lrc·Bx§}xyprx{v§xy¾NcNn«pr_}cs|}MocNrv§lmxyMp_}rcDtmlc}v{xyMxypn'ocNnx§opruw;£-xypr_pr_}c
Mc£uw}chM1|}bdMn-ox§rcDÛprv§ldprupr_MchK}upr_}cNwn¼cocD|opxyuw}u[olsu|}prx§}cµ<;¤l´v§vyuoNprx§}pr_}c
}cNnxyopu%n±¨ºub pr_}c`MrcNwp nOnqpw%½=¢[p_}c-prx§¦[prx§usu¨Kpr_}c`cNlovyx§}hbscD%_MMx§nrb ¨ºuIocDnrrx§opu%n
x{n-[¦uwx§}cNpumuMµ
ªrxyprx{v}=cr¨ºubdMc`x{nrnr|}c-x§b1|}vyprxypr_MrcDox§}cm¦mx§ruw}bdcWpn&p_Mp¼nqprx§v§vMv{%½onOKuwqp%t}x§v§x»plx§n
p_Mphu¨InrlWK%_}ruw}x§¾Npxyuw L¥uw|}«x§bd}v§cbdcWpprx§u|KnqcDn-pr_Mcnr}xy¯vyuo%½on«Mpr_}cspubdx§}rx§bdx»pxy¦wcNn
[¦xyv{t}v§c´x§ Mprx§¦c©Tpr_M°Áxyt±µ ^`_}cDnqcvyuo%½on1N t=cv»pcMpxy¦wcv§l¯bMKcD pru~¤¬'e ¡ ^`_}rcD}n
=?>ST=å
b|oprcocNn;u;nr}x§}v§um%½onN¢=Mp_}clc1v§nru´|MnrcNpu}upcNp«pr_Mc1cN}lzW|}cN|}cNnNµ;^`_}x{nhx§nhv{nrupr_}c
}c¨Ô|Mv»p«NnrcxyMprx§¦c©-p_M°±x§t±¢=£-_}x§%_uwoprx§uMvyv§lnr|}}=urpn`v§uo%½w·B¨ºcczW|}c|}cDnµ Ax§Mvyv§l¢}£¤c_M[¦wc
x§Wpruoo|McD¯7_mlWtMrx{¯x§bd}v§cbdcWpprx§uu¨¼tKrx§c%n«t=cp£¤ccN¯M}up_}cNMnTp_Mpuwb1t}x§}cDn;tM|MnqlW·
£`xyprx§}´Mt}v§um%½mx§}M¢=Nu%ox§}sprup_}c1m|}btKcNTu¨Op_}rcD}n;Mp_}cv§c¦wcvÁu¨>M%v§vycNvyx{nrbÀpr_Mcl
t=cv§uMpuMµI^`_}x{n`x{nrnr|}c'x§n-¨º|}qp_}cT}x§n|MnnrcN7xypr_Mc}cmp«nrcNÛpxyuw±µ
2Â-68 7N 775Á
!"$#&%(')*+
guwnqpu=co·¸nquw|}%csrcDnqcD%%_ ¬;=cMg~ ubd}x§vycNnhpMnrv§prc uwEM A}uwqpMruw%bn«£-xypr_
¬;=cKg~¹ox§rcDÛpxy¦wcNnx§Wpruuoocnr|}x»p%t}v§c¨ºuwuwbd}xyv{pxyuw £-x»p_ZMprx§¦cubd}x§vycN1v§xyM½cN £-xypr_
p_}cx§`¬;=cKg~ r|MwpxybdcTv§x§t}rlw¢£-_}x{%_|Mnr|Mv§vyldnr|}}=urpn>uw}vylsnrxy}wvyc·Bv§c¦wcvou¨ÇK%v§vycNvyx{nqbµ&¶c
_K[¦cbMcD¯pu}u¦mx§ocdp_}c´cNzW|}x§rcD5r|}Wpxybdcnr|}}=urp1tmlx§bd}v§cbdcWprx§}¯¯ucNnrKuwMox§}
nru¨p£`c1v{[lcN'upru5u¨`©-p_M°±x§t±µ´ªTvypr_Mu|}w_p_}x{n'v§[lwcuw|}v§t=csx§bd}v§cbdcWprcD¨ºruwbjnrp%_±¢
nqpqpxyM¨ºruwbp_}cubd}x§vycNq·¿}uoo|McDuoocw¢Mp_}cnru|}%cuooc[¦xyv{t}x§v§x»plv§vyu£¤cN|Mn«pru7}}v§l
dnqpr%x§_Wpq¨ºuwr£`%d=urprx§}du¨&pr_}cDnqc¬;=cMg~ |}Wprx§bdc'vyx§t}%xycDn¼uwpruu¨>©-p_M°±x§t±µ
¡ }xyprx{v§v§l¢W£¼cT}}v§xycDdu|}¥}}uww%_1pupr_}c«r|}WpxybdcTv§xytMrlu¨Çpr_}ch¬;bd}x.¬;KcNMg~Zubd}x§vycNNµ
^`_Mcnr|MNcNnnq¨º|}v-xyWpc%pxyuw u¨'©-p_M°±x§t$xy pr_Mx§nubd}x§vycNsbduprx§¦[prcDZ|Mnpru uWpxym|}c£-xypr_ pr_}c
|}Wprx§bdc-vyx§t}%xycDnOu¨=pr_}c'¬hg~Ix >(@¿¢m¬h >K@ÇM ¡Wpruw}c;¬;=cMg~ ubd}x§vycNn >@¿µ ¡dpr_}x{n>MKcNN¢
£¤c¨ºuo|Mnhu}v§lup_}c $Mnqp«p£¤u7ubd}x§vycNns®B¬;bd}xB¢¬hg~Ix²Ûµ^`_Mcx§bd}vycNbdcWpprx§u¯u¨>pr_}c¬h
M ¡ wpu}c«¬;=cMg~ r|}Wpxybdc`vyx§t}%xycDn&£¤wn>npxyw_wpr¨ºu£¤nrx§Mc`p_}c¤¨ºubscN>x§n¥nqx§bdxyv{&pru'p_Mp
u¨¼¬;bsMxn1®Ô¦c%nrxyuwwµm²Û¢M£-_Mxyv§cpr_Mc1v{pqpcT£`nTxybd}v§cbdcWpcNx§Mpxy¦wc1©-p_M°±x§t±¢=xyp_}c1uWprcWp
u¨Opr_}c ¡ Wpu}c}ucDÛpDµ¤©Tpr_M°Áxyt£`nT|MnqcDpru}u¦mx§}cr|}Wpxybdc1nr|}}=urpN¢=nqx§}v§c·¿v§c¦cNvÇpr_}uw|}_Á¢}pru
sbduoox$McN¦cNnrx§uu¨Áp_}c©«ª;©h¬'euwbsMxyv§cTM7pruspr_}c¬h}xyMg~ 0uwbd}xyv§cDµ
¬;bd}x&nq|}MKuwqp%nTt=upr_¬;=cKg~ M AMurpÇsK7xypn«v§prcDnp«¦wc%nqx§u ®qµN ²¤}u¦mx§ocDn`p£¤u
v»pcMpxy¦wc1x§bd}vycNbdcWpprx§uMnhu¨¼pr_}c¬;=cMg~ |}Wprx§bdcsv§x§t}rl¨ºuw'nr_MrcDbdcbdulb1|Mv»pxy}u·
cNnnquwnNµ^`_}c7}c¨Ô|Mv»psvyx§t}%l ®ºv§x§tKuwbsÇN²1x{ntMwnqcD u ~¤¬'e ¡ ^`_}cNw}n£-_}x§v§cpr_}cnqcDuwM u}c
®Ôvyx§t=ubdMnqp%²|KnqcDn'pr_}ceWpw%½m^`_MrcD}n'v§xytMrlwµ7^`_}c¨ºuwrbdcNx§npr_}cbsuWnp£-x§ocNvyl|MnrcN o|}c´pruxypn
=urpt}x§vyxypl £-_}x§vycp_}cv§pqprcN´nq|M}Kuwqp%nd}cNnqprcDZK%v§vycNvyx{nqb tM|op´|}o¨ºuwqp|}Mpcv§l v{%½onsKuwqp%t}x§v»·
xyplZM bduoo|}v{x»plwµ ^`_Mcx§bd}v§cbdcWpprx§u u¨¬;bsMxnsr|}Wpxybdcvyx§t}%l uZpru u¨'©-p_M°±x§t$x§n
cDzW|}xy¦vycNwph¨ºu't=up_¦wc%nqx§uMn;M5uwb1t}x§}cDnTp_}cx§bx§¨ºcNpr|}cNn L«=urpt}x§vyxypl¢.|Mnrcr·¿vycN¦cv&b1|Mv»·
px»p_}cN}xy}7M}cDnpcNMvyv§cv§x{nqbµ«¬;bd}xnTv§uo%½on-_M[¦ct=ccNb}=cNpru©Tpr_M°Áxytn;vyuo%½onTK
xypnhp_}cN=uWuwvyx§}_Mwn'tKcNc5rcN}v§wcD¯tmlp_}c¦mxyrpr|MvO}ruocNnnquwn;u¨`©-pr_K°±x§t±¢£-_Mxyv§cdcMc%prcD
£¤u½¯x{nrcN}rcDnqcNWprcN5£-xypr_ M}upr_}cNw}nNµ©-pr_K°±x§tnsocDnrrx§opruwn_M[¦c´t=ccN cmpcMocD x§ u%oc
puxyKv§|Mocx§o¨ºubpxyuw|KnqcDxy¬;bd}x)nTvyx§t}%lLI¨ºuTx§MnqpKc¢Mpr_}cocNnxy}pru-u¨IdM}upr_}cNw
p_Mp¥nrM[£-MnOK%v§vycNvyx{nqb uWp%x§Mn>v§vmpr_}c«}cNcNnnrrlMrcNWpvox§o¨ºuwrbpxyuw1p_}cTnrM[£-}cNp_}rcD}n
MccN NcNnn1pruMµ ªT}upr_}cNx{nrnr|}crcNv§prcD pru5|Mnrcr·Bv§c¦wcv¥pr_}cNwoxyM5M }cDnpcNZM%v§vycNvyx{nrb+£`n
p_}c'}}v§x{tMxyv§x»pldu¨Á}|}c;tM|MnqlW·¿£¤x»pxyM¨ºu`nrlWK%_}ruw}x§¾Npxyuw±µ ¡p¤£`n¼rcDnquwvy¦wcNdtmlcmpcMox§}pr_}c
tKrx§c%nt=cp£¤ccNZM}up_}rcD}nnqu n1pru5u=c%pc£-xypr_ rtMx»prl5m|}b1t=cdu¨«pr_}cNw}n1x§
pcNbjM¯b1|}vyprx§}v§cvycN¦cNv§nhu¨¤M%v§v§cv§x§nrbµ1^`_}x{nh¨º|}MÛpxyuwMv§xyplx§nv§nrux§bdKuwqp%Wp£-_}c 7v§rwc
T8T ÙU VWX'Y
N
m|}btKcN«u¨>pr_}cNw}n«x{nhnqM[£-McN.µ«©;pxy¦wc1©-p_M°±x§twnrnr|}bdcNn;v§£¤[lon`p_MpolmMbdx{Mvyv§cv§x§nrb x§n
cNMtMvycDMdp_m|Mn>p_}c«m|}btKcN¼u¨.nrM[£-}cDspr_}cNw}nIMc¦cN¥coccD}n>p_Mp¥u¨Á[¦x§v§t}vyc-MruocDnrnru%nNµ
¬hg~¥x&x{n'´cNcWp;cm=cx§bscNWpv" ubd}x§vycNT¨ºu¬;=cMg~ K}u¦Wx{ocDn-pr_Mc $M%nqph}|Mt}vyx{v§l
[¦xyv{t}v§c«x§bd}v§cbdcWpprx§u7u¨±p_}c'¦c%nrxyuw7oµ1u¨±pr_Mc'nqpM}.µ A}uw¤pr_}x{n¤r|}Wpxybdc'vyx§t}%l¢W£-_}x{%_
b½cNn1cmprcNMnrxy¦wc´|Mnrcu¨«pr_}c7~¤¬'e ¡ ^`_}rcD}nªT~ ¡ ¢O£¤c´¨ºuwvyv§u£¼cD c}prv§l5pr_}cnrbsc}MruW%_
wn`¨ºu¬;bd}x]µ-ª-}}x»pxyuwMv§vylw¢M£¤cx§bsMvycNbscNWprcN}}u}x{prc~`¬'e ¡ ·¿vyx§½ctKrx§c%n-MuM}x»pxyuw
¦rx{t}v§cNn¤x§©-pr_K°±x§t±µ A&x§Mv§vylw¢o£¼cx§Wpruoo|McD7nrv§xyw_wp-bduoox$Kprx§uxyWpup_}cuwbd}xyv§c-x§u%oc
pu´cN}c%pcuooc'p_Mph|MnqcDn`pr_MctKnpwÛp«}pdplm=c¨ºu«©Tpr_M°Áxyt n;vyuo%½on-x§MnqprcNwu¨>_Mm·¸uoocD
~`¬'e ¡ ^`_}cNMn¼b|oprcocDv{%pxyuwMnµ
! % #& %( " !
!;|}cpuxypnvyx§_Wp£¤cx§_Wp|MnqcNq·¿v§c¦cNv>pr_MrcD}nM5pr_Mc´v§¾l np%%½5v§vyuoprx§u =uv§x§l¢>©-p_M°±x§t
nr|}}=urp«c³xycNWprv§ldv{cm|}btKcN«u¨Op_}rcD}n«Mb|}v»pxyMvycvycN¦cv{n-u¨OK%v§vycNvyx{nqbµT¬;|}«x§}xyprx{v
}}uww%_¨ºuI_KMov§x§}'}cNnqprcDK%v§vycNvyx{nqb£-xypr_Mxyspr_Mc-xybd}v§cbdcWpcN¬;=cMg~ |}Wprx§bdc¤v§x§t}rx§cNn
x{n-s¦x§prx§uu¨Áp_}cv§v»·Bpru·¿vyv.n%_}cbdcwµ>eoKcDx$Kvyv§l¢}£¤c'ox§nqprx§t}|oprc'p_}cocDnrrx§opruwn`nrM[£-}cDp
p_}c $Mnqp¼v§c¦wcv=u¨±Mvyv§cv§x{nqb prudv§vKMruocDnrnru%nN¢xyKnqcNqpxy}1pr_}cNbÀp¼pr_}chcMu¨.p_}c;cNwoldzw|Mc|}cDnµ
gucu¦cNN¢>£¤cxyKnqcNqpp_}cocNnx§opruwnsu¨'x§}}cv§c¦wcv{npdpr_Mc¨ºruwWpdu¨«p_}cvyuoNv-cN}lZzW|}c|}c
p_Mp;tKcNvyuw}wn`pupr_Mc1}uocNnnru¤pr_}cNl£¼cNrcrcDprcDu±µ ¡ u|MTxybd}v§cbdcNwp%pxyuw±¢=x§ov§c1¦mx§qp|Mv
MruocDnrnru'nqprx§vyv>cWpwÛp%nh}cNnxyopu%n;¨ºruwb pr_}cd¨ºuWp'u¨¼xypnvyuovOrcDolzW|}cN|}cdt}|opN¢xy uwWpr%nqp
£-xypr_ Mprx§¦c©-p_M°±x§t±¢OnpcNv§nh¨ºub p_}ccKu¨¤cbduprczW|}c|}cDn¢Á¨Ô[¦uwrx§}7¦mx§qp|MvIMruocDnrnru;pru
co}vyuwx»pÁpr_}c¼KuprcWpx§vW}p-v§uovyxypl;u¨oxy}Mc&vycN¦cv{n±u¨oM%v§vycNvyx{nrbµªTvypr_}uw|}_s¬;bd}xFEe}^5vyv§u£Tn±
x{ov§c'}ruocNnnquw¥pusx{nnq|}ch%MouwbÀcNzW|}cNnqpn¼¨ºuw¤£¤u½´nqprcNvyx§}K¢WxypTcNzW|}x§rcDnIp_}cxyWpc¦cWpxyuwu¨pr_}c
cbduprc1¦mx§qp|Mv&}uocNnnruD¢M£-_}x{%_x§wpruoo|KcNn;|}Wprx§bscu¦cNr_}cDMcNo|McNn;Kuwqp%t}x§v§x»plwµh^`_}c
v{prprcN;¨ºcDp|}rcdx{nh¨º|}rpr_}cNrcDo|McD}|}csprupr_}cocNKcNMocNMcu¨-¬;bd}xFEeo^¹up_}ceWp%%½o^`_}cNMn
ubd}x§vycNN¢[TKp%_McN'¦cNnrx§u'u¨'© ;µ¬;pr_}c¤uwwprlw¢£¤u½;nqprcDv§xy}«xy1©Tpr_M°Áxytx§n=cr¨ºubscD
wnqlmM%_Mruw}u|Mnrv§lMpr_}cscNnn-u¨>p_}c1¦mx§qp|Mv&}uocNnnru%n¤pru´p_}c1cbduprc1zW|}cN|}cNn;x§n;=cr¨ºubscD
x§1|}Mx»¨ºuwrb£`[lµOgucu¦wcD¢uw|}-nqu¨p£¤rchu $M|}%pxyuw´v§vyu£Tn¥pr_Mchx§WprcN%prx§uu¨&mlMpxy¦wc
ubd}x§vycNNµ
"
^`_Mcd¬;=cMg~ £¼uwr½ozW|}cN|}xyM´bdum}cv >y[@>x§nhMcoxyt}v§c1bdcD%_M}x{nrb¨ºuwhnr=cNxy¨ºlmxyM|}}xypnhu¨I£¼uwr½
p_Mpcs}up'}c·¸ubd}|}prcNphpr_}cnqpqp'u¨Ip_}cd£¼uwr½onr_Mx§}´uwMnpr|KÛpNµ^ÁuocbduwMnpprcpr_}c
x§}_}cNrcNWpr|Mwpxybdc¯nr|}}=urp´u¨1©Tpr_M°Áxyt$pu pr_Mx§n7cocN|oprx§u$bduoocvB¢T£¤c_M[¦c}c¦cNvyuwKcD $cm·
=cx§bscNWpv¼nquw|}c·Bpru·¿nru|}%c1pr%Mnrv{pruwhpr_Mp1bMn'p_}c#,&3&),& $M $}%bwn
puNv§v§nu¨-p_}c©;ª«©'¬'e ªT~
¡
µIªTvypr_}uw|}_ p_}c7pr%Mnrv§pruw1x{n1v§xybdxyprcD pru5}uwbdn£-_}cc´p%nr½mn
rcnqx§}wvyc¨º|}MÛpxyuw Nv§v§nsMZ}uv§|MnrcNnsc|MnrcN £-xypr_Zpr_}cox§cNÛpxy¦wcNnN¢Ox»px{nwocNzW|Mprc´¨ºuws
£-x{oc'¦xycpldu¨&}}v§x§NpxyuwMnµ>ª-Nu%ox§}pu;>§D@B¢3#,&)&),& dcNprcNn¤snqx§}v§czW|}c|}c'K
nrM[£-Mn1£¼uwr½wc'p_}rcD}n1pr_KpcocN|}prc´p_}c7[¦xyv{t}v§cp%nr½mnNµemx§Mc7©-p_M°±x§tZbxyWpxyKnxypnsu£-
zW|}cN|}cNndM £¤u½cNnN¢Opr_}x{nd}%wbdnqx§bd}v§l xy}xyprx{v§x§¾cNnsp_}ccocN|oprx§u cm¦mxyuMbscNWptml }}xy}
uw}cdoc=cM}cMcpup_}c|}cWp;M}up_}cN±µ A}u'cD%_p%nr½Ç¢Kp_}coc=cKocMxycDnc1x§McNwnqcD
tml u}cK K}upr_}cNw ocDnrrx§opux{n´cNprcN K nr|}t}bdxypqpcN ¨ºucocN|oprx§u$x§ p_}cv§uov
=?>ST=å
!#" $%& '()" +*-,
.% /0 )1-2%/03 $416587
9 :<;-1>=?7@;>AB 7C;22*D,
EF GHIKJL;MBNOP( +*>Q'?7
R GL )14QS/T' U< C;**VAW/0 3*
X ,Y/0S3 )1- 7Z/0 $)1B;>[?7
\ [
] " $41Y/0S3 $%7^" )1W/03 7
_`[
=
a'T 3bcI(dF^>I+8*>,
.% :D1>I(d2+87WDADeF7@223*D,
9 GHIKJHMBNOP( +*>Q'?7
SEF GHIKJHMB -3)I(d6>I+8*
R 3f< &ghJHMFH $%!ghJHMFi +*7
X (#jHI+kA>*
\ GL >1)QS/T' HI+KY***VAVhJLMFi +*
] ,lJHMF< $)1>I+K7`mJHMFi #1- 7![
_ [
.=`[
A&x§|}cdL>~¥nrc|M}umuooch¨ºu`pr_Mc|}=MprcM_Mnrc'u¨O~ /« !B
cNwolzw|Mc|}cwµA&x§Mv§vylw¢.p«p_}cscMu¨#3&3&),& $=¢}pr_Mcs|}rcWp«K}upr_}cNwnq|Knq=cMMn«xypn
cocN|}prx§u±¢=£¤x»pxyMpr_}cscNprcNp%nr½on`pru $M}x{nq_±µ'¬;|}«}u=uwnv±¨ºu;pr_}cbdMwcbdcWpTu¨I}cNnqprcD
K%v§vycNvyx{nqbÂx{nTv§nrus}}v§x§Nt}v§chprud}cDnpcNM%v§v§cv=pwnq½zW|}cN|}cNnNµ
n O - #o ' p¹ ,.&`&Wqsr qO%mt
!hp%¯vy|MnqprcNrx§}x§nu}c7u¨Tp_}c´¨º|MM}bdcNwp%v¤prcD%_}}x{zW|}cNnxy nrxycNWprx$K}pMvylonrx§nM Mp
bdx§}x§}MµO^`_}cT}utMvycNb u¨±v§|MnqprcNrx§}'x§nIpruMqpx»pxyuwspr_}cT}pnrcp¥x§WprunrcwbscNWpn-®ÔNv§vycDdv§|Mnpc%n²
nru7pr_Mphx§Wpr%·¸v§|Mnpc'}p%7rcsnqx§bdxyv{Mx§Wprcr·¸v§|Mnpc'}p%7rcdox{nrnrx§bsx§v{Dµ ¡ ¯p_}x§n'nrcNprx§u±¢
£¤c´}cNnrcWpdM%v§v§cv¥x§bd}v§cbdcWpprx§u u¨ /« !B®)¼v§|MnpcxyM «nrx§} !B¥}rcDnqcNWppxy¦wcNn%² >Ð@B¢I
£¤cv§vy·B½m}u£-Kc³´x§cWp«_}x§c%%%_}x{v±}p%´v§|Mnqprcx§}´v§uwrxypr_}b¢}|Knqx§}¬;=cMg~&µ !;cNnr}x»pc1xypn
c³´x§cMl¢op_}c£¼uwnqpq·¸wnqc;prx§bdcuwbsMvycmxypl´u¨"/; !B x§nh¬s®
n2
v§uwK²Û¢M£-_}cc'x{n`p_}cm|}b1t=cTu¨
=ux§Wpn«pru7tKcsv§|MnpccN.µh^`_}c1K%v§vycNvyx§¾Nprx§uu¨>p_}cvywux»p_}b xybnhphcMt}vyx§}´xypnhox§rcDÛph|Mnqc
uw7¦wcl´v{c'}p%snrcp%nTK.¢oprusp_}ctKcDnpTu¨&uw|}-½m}u£-v§cNowc¢oxypTx§n¤p_}c $M%np`pxybdc'pr_Kph¬;=cMg~
_Kn`t=cc}}v§xycD´pu´nq|M%_M}vyx{prx§u±µ
~ /; !B4®º~>%v§vycNv /; !B¼²;|MnrcNn«¯%[lu¨IcNu%}nN¢M_}uwv§ox§}7xyo¨ºuwrbprx§u¯tKuw|op«p_}cnrxy¾Nc¢
p_}ccWpruwx§ M pr_}crcN}cNnrcWpprx§¦c´=ux§Wpnsu¨«cD%_ v§|MnpcDµ ª-u%oxyMprupr_Mc7_}x§c%%%_}x{v
vywux»p_}b¢wcN¦cNrld}p%=ux§Wp¤x{n¤xy}xyprx{v§v§lduMnrx{occN´wn¼nqcNM%pc«v§|MnqprcN¤£-xypr_7u}c;rcN}cNnrcWp·
pxy¦wc¢mp_}c'KuwxyWp-x»p%nqcNv»¨µIª¤p`p_}c'xy}xyprx{v§x§¾Npxyuw}_Knrc¢mpr_Mc'vywux»p_}bÀuwbd}|oprcDn¼pr_}cv§uwnrcNnqp-v§|Mnpc
¨ºuwdcN%_ v§|MnpcDµ ©«cmpN¢¼x»p´nqprpnp_}cvy|Knpcxy}K¢Ibdcx§}pr_}cv§uwnrcNnqpKx§su¨hvy|MnqprcNns|}Wpxyv
uw}v§l vy|Knpc%n1cbx§±µ5¶ _}cZp£¼u5v§|Mnqprc%nsc´bdccN±¢>£¼cnpuc´pr_Mc7x§o¨ºubpxyuw ¨ºupr_}c
T8T ÙU VWX'Y
#pragma omp parallel
0 50 100 150 200 250 300 350 400
2 4 8
Threads
Overhead (µȢ)
Omni Omni-Nth OMPi OMPi-Nth
#pragma omp for
0 50 100 150 200 250
2 4 8
Threads
Overhead (µȢ)
Omni Omni-Nth OMPi OMPi-Nth
A&xyw|}rc9LI¬;=cKg~$u¦c_}cD}nh®ºMvyv§cvB¢m¨ºuÛ²
bdcNrwcN¯vy|Knpc'xypr_McscNWprlu¨¥p_}c,$M%npv§|MnqprcN'M5nqx§bd}vylx§m¦v§x§}prcspr_McdnrcNuM¯u}cwµ^`_}c
vywux»p_}bv§nrudbdxyWp%x§Mn-=cr·¿vy|MnqprcNTx§o¨ºubprx§ut=u|}pTpr_}c1xyM}c7u¨Opr_}cvyuWnqcDnp«vy|MnqprcN«K
p_}cTbdx§}xyb|}bÂox{np%Mc-pruxypNµ>^u[¦uwx§do|}Mvyx{prx§uu¨Çp_}x§n¥xy}¨ºubdprx§udp_}c«v§uwrxypr_Mb nrcN%%_}cDn
¨ºuw`pr_}cvyuWnqcDnp-}cNxyw_mtKuw¤u¨Oswxy¦wc7vy|KnpcTuw}vylxycWprx§cNn¤£-xypr_nqbvyv§c`x§Moc.µ
A&x§|}c7s}cNnrcWpn;x§Mnrc|Kououoocpr_}csbduwnqpubd}|op%pxyuwocNbdMox§}ruw|oprx§}c1u¨¥p_}csvy|Kn·
pcxyM'vywux»p_}b¢£-_}x{%_ucNnrKuwM}npru'p_}cT|}Ç}pcTu¨=pr_Mc-}cNrcDnp>}cNxyw_Wt=u%nNµ&em=cNx§v}¨ºcNpr|}cNn
u¨Áp_}x§n«v§uxypr_}brc'xypn-wnqlmbdbdcprlMMuo·¸ocpcbdxy}x{nrb¢Ko|}c'prusp_}c£¤[l´pr_}c}cDcNnqp`vy|Kn·
pc%ndcubd}|opcN M pr_}c%}|Mv-ocDcNnrcu¨;¦v§x{ cWprx§cNn®Ôvy|Knpc%n²µ ^`_}cc³´xycNMl u¨
p_}cdM%v§v§cvOxybd}v§cbdcNwp%pxyuw5nqpru}wvyl}c=cM}n'uw¯p_}cdc¦wc ox§nqprx§t}|oprx§u5u¨`uwbsM|opprx§uMn;pru
p_}c}ruocNnnquwnNµ ^`_}x§nN¢I_}u£¤c¦wcD¢Ox§nd5%_Mv§vycN}x§}¯pnr½ o|}cprupr_Mc7=cN|}vyx{xyprx§cNnsu¨Tp_}cv§u·
xypr_}bµ¥¬;|}¥coKcNrx§bdcWpn¼u $MrbdcDdpr_Mp¥p_}c«|}Ç}prc«M_Mnrc«}}up¤nv§cTc³xycNWprv§lsxy¨±uMvylsu}c
v§c¦wcv=u¨&M%v§vycNvyx{nrb x{n`co}v§uxyprcD.¢mrcNw%ov§cNnnIu¨±pr_}c'n%_}cNo|Mvyx§}sKuwvyx{l´Mp_}ch%_m|}M½´nqx§¾ch|MnrcN.µ
^`_Mcc¨ºuc¢m£¤c'_M[¦cv{nruMvyv§cv§x§¾cN´pr_}cv§umux§p_}c *'*$ $ '*$"+ ( u|}prx§}cµ
!$#`O- -,Á %,
¶5c=cr¨ºubdcNu|}«coKcNrx§bdcWpnTu ¡ WprcNvÁ~OcWpxy|}b ¡q¡r¡ nqlonqprcNb£-xypr_¨ºu|M«}uocNnnru%n¤p;
g T¾¢¥£-x»p_ o[7; w%_}c´bdcbdul M0 ; u¨Tbxy bscNbdul¢>|}}}x§}°Áxym|o0®BoµN µN ²'£-xypr_
p_}c1©;prx§¦c1~&uWnqxy^`_MrcD}nT°ÁxytMrl5®Ô©T~¼^-°²ÛµÇª«n'¬;KcNMg~0ubd}x§vycNnT£¼c_K[¦c|MnrcN¯¬;bd}x¼µN
®Ôvyx§t=ubd=D²-K¬hg~¥xÁ}µ µ§hMn`Mprx§¦cuwbsMxyv§c'© w}µMµ}µ
A&x§|}c¯}rcDnqcNWpndp_}cnqlmM%_Mruw}xy¾DpxyuwZu¦cNr_McNMn1¨ºuwdpr_}c #,&3&,&¯M,( ¬;=cMg~
uMnqpr|MpnN¢obscDnr|}cN|Mnrxy}dp_}cIBI~ bdx{ut=cM%_Mbdr½on>Ð@Bµ`^`_}cDnqchp£¤u´uKnpr|Mpn-rcco·
cNnrcWp%prx§¦c1u¨Ip_}c¬;=cKg~ u¦wc_}cNw}nTcv{pcNpup_}cN}xy}Kµ'¶csutMnrc¦c1pr_Mp;p_}csu¦c_}cD
u¨#,&3&),&sx§n;v§u£¼cN-¨ºu;tKupr_ubd}x§vycNn«£-_}cpr_}cr|}Wpxybdcnr|}}=urp;x{n;}ru¦mx{ocNtml©-p_M°±x§t±µ
¬;sp_}cTup_}c¼_MK.¢£-_}cNdpr_}c«W|Mb1t=c¥u¨Kp_}cNMn>x§n¥vycDnrnIuIcDzw|Kvopupr_}c«W|Mb1t=c¥u¨=MruocDnrnru%nN¢
p_}c,(5uMnqpr|Mpco_}x§t}x»p%n1_Mxyw_}cu¦wc_}cNw¨ºuMprx§¦c¬hg~Ix]¢I}|}cprupr_}c7t}vyuo%½mx§}~¤¬'e ¡
tKrx§cDµ «u£¼cN¦cNN¢W£-_}c7pr_McW|Mb1t=c`u¨pr_}cNw}n`xyKcNnrcNnN¢W£-_}x§%_v§nrusumN|}-£-_Mc}cNnqprcD
K%v§vycNvyx{nqbx§nco}v§uxyprcD.¢`pr_}cu¦cNr_}cD u¨Mprx§¦c5¬;bd}x)n´t}|MnrlW·B£`xyp´tMrxycN´x{nnrxyw}x%$=Wpvyl
_Mxyw_}cDµ
=?>ST=å
Omni-NthLib
0,0 0,5 1,0 1,5 2,0 2,5 3,0 3,5 4,0
CG FT LU MG
Applications (OMP Library)
Speedups
OMP1 OMP2 OMP4
OMPi-NthLib
0,0 0,5 1,0 1,5 2,0 2,5 3,0 3,5 4,0
CG FT LU MG
Applications (OMP Library)
Speedups
OMP1 OMP2 OMP4
A&xyw|}rc'9LI©;ª;e7t=cM%_}br½on
0 1 2 3 4
Bl ock-LU Radi osi ty FFT Queens
A ppl i c a t i ons
Speedup
OMP1 OMP2 OMP4
A&x§|}c' L>~Ocr¨ºubMc;u¨&¶5u½ozW|}c|}cNxyM
A&x§|}c¼T}c}x{Ûp%n±p_}c¥=cr¨ºubMc¥nq=ccDo|}Mnu¦cN.pr_Mc¼nrcNzW|}cWpx§v¦cNnrx§uu¨o¨ºuw|}Át=cK%_}b½mn
®s¢&A&^'¢>° ¢>g ²1u¨Tp_}c©;ª;e ~Ovyv§cv ;¤cM%_Mbdr½onnq|}xyprc > @B¢¥KuwqpcN pu ¬;=cMg~ tml
p_}c ¬;bsMxhwruw|}±µ !«cDnqMx»pc¯p_}cx§}x%ÇccWpr|}Wpxybdc5u¦c_}cD}nN¢-v§v'u$K|}%pxyuwMncNnr|}v»px§
¦wclnqx§bdxyv{'cmcD|opxyuw¯pxybdcDnµ^`_}cNrc¨ºuc¢.cNnr|}v»p%nh|Mnqx§}pr_McMpxy¦wc|}Wprx§bdcsv§x§t}rx§cNn'rcs}up
x§Mvy|M}cN.µ A&x§|}c;nr_}u£Tn¥pr_}c=cr¨ºubMc;cNnr|}vypn¤u¨¨ºu|}`t=cM%_}br½on¤M%v§v§cv§xy¾NcN´|Knqx§}spr_}c
MruwKuWnqcD £¤u½mzW|}cN|}x§}¯bduoocvBµ ;¤vyuo%½o° M T[lWpr%cubdc¨ºruwbpr_}c¯eo}v§wnq_ t=cM%_}br½
nr|}xyprc>§ N @¿¢o£-_}x§vyc A AO^ M ;|}ccNMn-chp£¼udu¨Ápr_Mc£¼uwr½ozW|}c|Mxy}st=cM%_}br½on¤|MnrcNxy+>§D@¿µ
AMuuw|}v§wnpcm=cx§bscNWpN¢Á£¤c|Mnrcd~ /« !B ntKcNM%_}b½x§ u%ochpuocbduKnpprcdu|}
|}Wprx§bdcnq|}MKuwqpu¨h}cNnqprcDZK%v§vycNvyx{nqb MZpru5}ru¦mx{oc5ubdMrx{nquw £-xypr_$up_}c¬;=cMg~
cNm¦Wx§u}bdcWp%nµSA}u¼tKupr_´v§umuMn¥u¨Çpr_Mc«|}Ç}pc«}_KnrcT£¼c«|MnqcTpr_}chnp%prx{-=uv§x§ld£-x»p_%_m|}}½1cDzW|Mv
pu NMµ7¶5c´|MnrcN nrbv§vIm|}b1t=cu¨`rcDuw}n®Bww²¢u¨`}xybdcKnqx§uZM¢¨ºruwb }pnrcpp_Mp
uWpxyKn'xybwc´ocNnx§opruwnNµ7emx§Mc´pr_}cDnqc´rcuwrW}x§¾cDx§ nq}_Mcx§Nv¥vy|MnqprcNnN¢Á£¤c|Mnrcu}v§l
nrx§}v§c7cMrcDnqcNwp%pxy¦wc´KcNv§|MnpcDµZ^`_}cv§|Mnqprcx§}5nqpruwMnd¨pcdwwnqprcNMnµ ¡7A&xyw|}c}¢I£¼c
MrcDnqcNwp-p_}c=cr¨ºubMc'nr=ccN}|}MnTuwtopxyMcN7¨ºu-p_}c|}Ç}prc}_Knrchu¨>~ /« !B«µ=^`_}c $KnqpTp£¤u
uv§|}bdMn>uwrcNnr=uM1pru'p_}c-|Mnrc-u¨=pr_}c;¬;bd}xMM´¬hg~IxMubd}x§vycNnO£-xypr_dp_}cx§I|}Wprx§bdc¤v§x§t}rx§cNn
tM|}xyvyp¥uwpudu¨±©Tpr_M°Áxyt±µ>^`_}cTpr_}x§duv§|}bd®º¶ '²&x§n¼dcNzW|}x§¦v§cWpIxybd}v§cbdcNwp%pxyuwsu¨.~ /« !B
T8T ÙU VWX'Y
U p d at e Phase
0 1 2 3 4
Omni-Nth OMPi-Nth WQ Intel Omni
OpenMP Compiler
Speedup
OM P 1 OM P 2 OM P 4
`
Ax§|MrcLO~Ocr¨ºubMc;u¨&~ /« !B
tKnrcNupr_Mc£¼uwr½ozW|}c|Mcx§}dbsuoocNv]µ A&xyMvyv§l´p_}cv{nqp-p£¼uuv§|}bdMn-x§vyv§|Mnqpr%prc'p_}c=cr¨ºubMc
nr=ccN}|}Mn'£-_}cNp_}c ¡ WprcNv«®º¦wc%nqx§u µw²hM ¬;bd}xIubd}x§vycNnrcs|MnrcN.µ7¬;bd}xI|KnqcDn'x»p%n~Ouwnrx»
^`_MrcD}ntMnrcNvyx§t}%l'Mp_}c`nqx§¾c¤u¨}p_}c¼pr_}cNw=uWuwv.® m²Mx{n&}u|}t}v§c¼pr_}c¤m|}b1t=c
u¨cNzW|}cDnpcNp_}cNMn®v§c¦cNv§n¤u¨Mvyv§cv§x§nrb´²ÛµO¶5cutMnrc¦c«pr_Mp¤p_}cM}vyx{prx§unrNv§cNn¤u}v§l
£-_Mcp_}c¼|}Wprx§bdc¼nr|}}=urpx{nÁMru¦mx{ocN'tml©-p_M°±x§t±µ>^`_}c¥£¤u½ozW|}c|}x§}-¦wc%nqx§u=cr¨ºubnnrvyx§_Wpvyl
t=cprprcN¥t=cN|MnrcTx»p¤[¦ux{}nInrubdc-cmMvyx{xyp¥cocD|opxyuwtMrx§c%nNµÁ¶5c«v{nquuwtMnrc¦c¤pr_Mp¥p_}c ¡ WprcNvKK
¬;bd}x}ubd}x§vycNn&u¦mx§oc¤t=cpqpcIcNnr|}vypn&£-_}cN«p_}rcD}nOrc¤|MnrcN.µ ¡spr_}x{n>wnqcw¢pr_}c¤pupvom|}b1t=c
u¨Çpr_}cNw}n>p_Mp¤cocN|oprc«pr_}c«p£¼uMcNnqprcN´vyumuKn¥cTcDzw|Kv}pupr_Mc«m|}b1t=c¤u¨±M_Wlonrx§Nv}MruocDnrnru%nNµ
AMu>'p_}rcD}nIKcNIvyumuw±¢p_}cTnq=ccDo|}´ouKnObsuWnpvylso|}c`pupr_}cTuWprcNWprx§udu¨=pr_}c pup%v}£¤u½cN
p_}cNMn-Mpr_}cx§MrcDnrcN´u¦wc_}cNw}n¼u¨&pr_}c|}Wprx§bdchv§x§t}rx§cNnNµ
2À q1 q
¶5c7rc|}cWprv§l£¤u½mxyMuZZvyprcNrMprx§¦cM =urpt}vycxybd}v§cbdcWp%prx§u u¨-p_}c7©;ª«©h¬'e
¼~ g¯McDµ^`_Mx§n¤£-x§vyv.v§v§u£ |Mn¼prud}}v§lspr_}c'£-_}uwvyc;r|Mwpxybdchxy}¨ºwnpr|Mpr|}c«u¨±pr_Mch©;ª«©h¬'e
MrucNÛppuup_}cs¬;=cKg~¹uwbd}xyv{pxyuw nqlonqprcbnNµ7 -cNwox§}}pvy|MnqprcNrx§}K¢Áu|MuWv>x{npru
vy|Knpc-¦cNrlv{c'}pdnqcpn-tml´|}}Mxy}d~ /; !B uv§|MnqprcNn`u¨>eog~¥nµ
r À - Wq1-,
g¯mlpr_M}½onTpuvyv&pr_McMqp}c%nTx§p_}c©«ª;©h¬'eM~¤¬;~ }ucDÛpnNµh¶cd£¤u|}v{v{nruv§xy½wcpru
p_MM½ !«Dµ¼v{[l;¤rcDnq_}cD%nI¨ºuw-}u¦Wx{ox§}s|Mn`pr_Mcnquw|}chuooc'u¨&pr_}c£¤u½ozw|Mc|}x§}sc}bd}v§cNnNµ
=?>ST=å
-)`O-¤-,
>y@B«µªOlm|Mwoc¢7g µ huw}¾NvycN¾¢ µ1grpruwrcNvyvB¢Kµ1°ÁtMrpM¢´©µ1©;[¦ruK¢´MKµs¬;vyx§¦cNNµ
©;}uWn ¼uwbd}xyv§cLIem|}}=urprx§}A&vycoxyt}v§cg|}vyprx§v§c¦cNv±~>%v§vycNvyx{nrbÀxy¯¬;=cMg~&µ&(14214
2441(" #&%'9*' 41µÇemKcDx{vx§nnq|Mcu¯¬;KcNMg~O¢M¦wuvBµ±D}¢}}uKµ±D}µ}}±µ§Dw ÇD} ¢}¬hÛ·
put=c;w}µ
>@Kµwg µ;¤|}vyvBµwgcNnr|}xyMeolWK%_}ruw}x§¾NpxyuwsMeo%_}cDo|}v§xy}¬;¦wc_}cNw}n&xy7¬;=cKg~Oµ ¡ ; 2-4-
-,?69!# 2191( 0 6 , 6 9:;*# 6 :;¢m°±|}K.¢meo£¼cDoc±¢memc}prcbtKcNT J JJMµ
>(@hfsµ¼_Mc±¢Mµ=°±xB¢=eǵK¶5}M¢=M !µK¶ }Kµ=¬h ¥·¬;=cKg~LKª«¬;KcNMg~ ¼uwbd}xyv§c ;¤wnqcD
uw ¬h ;µ ¡ 2-4 , ?69' &( , , ;4'*141S[¢ '%½wu£¢O~&uwv§M.¢
w|}}cw¢KwKµ
>@! µ! 1µ !«x§b½wu=u|}v§uwnN¢/BTµ°ÁcuWpx§wox§nN¢1M sµ'^Á¾Nu|}bwnµ'ª =urpt}v§c uwbd}xyv§c¯¨ºu
¬;=cKg~" h}µMµ ¡ $ 2 4-, 6 # 2 1# 0 6 (C6 9:;$*# 6 :;%&'(¢ª«%_Mc±¢
hcNrbWlw¢Memc}prcbtKcN;wMµ
>@e.µ h|}_K}¢m µw Twnpuwx]¢WM dµmem_}x§bµ/« !BLWª« B>³´x§cWp¼v§|MnqprcNrx§}ªTv§uwrxypr_Mb¹¨ºuw¼°Árwc
!hp%;`nrcNnNµ ¡ 2-4 , ?69 )S: *,+ : 6.-/' 0 ( ,'2' 41 +: ' D-,1-$¢
JJ µ
>N @'~OµBTµ3«x§ouw|}½nN¢BTµ!µ~Ouv§lo%_}ruw}u=u|MvyuWn¢K^'µeǵ~OMp_}cuoouwruw|±µw¬;=cMg~ -|Mwpxybdc
eo|}}=urp¨ºuw ¼v§|Mnqprc%nu¨«g|}vyprx§}uocNnnru%nµ ¡ 2-4 ,;?6922$( K06 68 ' :H
41*90 "3 045 6 :;63%&'(¢K^uuWpruK¢¤M}M¢7|M}cw}µ
>ÐK@ ¡©;^«¬h©BL ¡ }}u¦prx§¦c ^Áumuwv§n ¨ºuwZ©Tuw B>oKcNqp%n¢ ¡eo^EASB¥^ }ucDÛp0®¡eo^¤·JJ J·www²Û¢
,3)#98;:<:=<=<=>50#>.3# 0?>9 :-*) (*,*:}µ
> @71µ7x§±¢og µA}|}bd½mxyÁ¢}MKµWf¥±µo^`_}c¬;=cMg~ x§bsMvycNbscNWppxyuwu¨©«ªheMvyv§cv=tKcNM%_o·
br½onsM xypn=cr¨ºubdMcwµ©;ª;eoª ªTbdcDn -cDnqcD%%_ ¼cWprcN^cN%_}Mx§Nv` -c=urpN¢I -cNKuwqp
©;ª;eW·JJ·¿Mw¢=JJ J}µ
>J(@©;ª«©'¬'e L!B ÇcNÛpxy¦wc ¡wpcwprx§u u¨$Ax§}c·Bwxy~>vyv§cv§x§nrb B>o}v§uxyppxyuw Mg|}vyprx§}u·
wbdbsx§}K¢B¼em~¥ ¡ ^ ~IucDÛp«©TuKµMoDJWo¢ ,3)#98;:<: $' 10 > 0?>.3#10@> :"*'* ($:Mµ
>§N(@'©;ª«©'¬'e ¼uMnrurprx§|}bµ-g¯oµ!h9L o·¸ &^-° ¡bd}v§cbdcNwp%pxyuw±µT©«ª;©h¬'e }ucNpocv§xy¦wc%t}v§c¢
ª¦x§v§t}v§c'p ,))#98A:<: $ 10 9>$0@>.# 0?>$:'**$(,:}¢ªT}x§vI J J µ
>§@¬;=cKg~ ª«%_Mx»pcNÛp|}c -cN¦mxycN£ ;¤uw.µ ¬;=cMg~ em=cNx%$KNpxyuwMnµ ª¦x§v§t}vyc p
,3)#98;:<:=<=<=>(#$*'!$#> ( 3+=µ
>§D@h~`¬;~L ~&cNq¨ºuwrbMcÀ~Ouwqp%t}x§vyxyplu¨ ¬;=cMg~&¢ ¡ eo^E(ASB¥^ }ucDÛp®¡eo^¤·¸M·¸mm[²Û¢
,3)#98;:<:=<=<=>50#>.3# 0?>9 : #$( #.µ
>§N(@'g µ>e}pruK¢¼eǵ>eopu_±¢ dµ '|MnMuM¢IK fµ>^&M½[Mµ!;cNnrx§ u¨'¬;KcNMg~ ¼ubd}x§vycN1¨ºuws
e}g~ ¼v§|MnqprcNNµ ¡+ 2 4- ,$?69/# (B$( K0 691?(?6 9:;F#4 6 :HC¢°Á|}M.¢
eo£¼cDoc±¢=emcopcbtKcN J JJ}µ
T8T ÙU VWX'Y