Ïðèìåíåíèå àðõèòåêòóðû multiGPU+CPU äëÿ çàäà÷
ïðÿìîãî ÷èñëåííîãî ìîäåëèðîâàíèÿ ëàìèíàðíî - òóðáóëåíòíîãî ïåðåõîäà ïðè ðàññìîòðåíèè çàäà÷ â
êà÷åñòâå íåëèíåéíûõ äèíàìè÷åñêèõ ñèñòåì
∗Åâñòèãíååâ Í.Ì., Ðÿáêîâ Î.È.
Ôåäåðàëüíûé èññëåäîâàòåëüñêèé öåíòð Èíôîðìàòèêà è óïðàâëåíèå Ðîññèéñêîé Àêàäåìèè íàóê, Èíñòèòóò Ñèñòåìíîãî Àíàëèçà.
 ðàáîòå îáîáùàþòñÿ äàííûå î ïðèìåíåíèè ðàçëè÷íûõ ïàðàëëåëüíûõ âû÷èñëè- òåëüíûõ àðõèòåêòóð ïðè ÷èñëåííîì ìîäåëèðîâàíèè çàäà÷ ëàìèíàðíî - òóðáó- ëåíòíîãî ïåðåõîäà (ËÒÏ). Îáû÷íî àíàëèç ËÒÏ îñíîâàí íà ðàññìîòðåíèè ñòà- òèñòè÷åñêèõ ïàðàìåòðîâ - êîððåëÿöèé ïóëüñàöèé ñêîðîñòè, ýíåðãåòè÷åñêèõ ñïåê- òðîâ è ò.ä. Àíàëèç ËÒÏ êàê íåëèíåéíîé äèíàìè÷åñêîé ñèñòåìû â äîïîëíåíèå ê óæå óêàçàííîìó àíàëèçó îñíîâàí íà àíàëèçå ñîáñòâåííûõ çíà÷åíèé ÿêîáèàíà, âèäà àòòðàêòîðîâ ñèñòåì â ôàçîâîì ïðîñòðàíñòâå è ñîáñòâåííûõ çíà÷åíèé ìàòðè- öû ìîíîäðîìèè.  ðåçóëüòàòå ñòðîÿòñÿ áèôóðêàöèîííûå ñöåíàðèè è äèàãðàììû.
Ýòî äàåò âîçìîæíîñòü ïðîñëåäèòü ìåõàíèçì óñëîæíåíèÿ äëÿ ðàññìàòðèâàåìûõ çàäà÷ ïðè ËÒÏ ïðè èçìåíåíèè âûáðàííûõ ïàðàìåòðîâ (÷èñåë Ðåéíîëüäñà, Ìàõà, Ôðóäà è ò.ä.). Ðàññìîòðåíèå ïðîöåññà ËÒÏ ñ òî÷êè çðåíèÿ íåëèíåéíûõ äèíàìè÷å- ñêèõ ñèñòåì íàêëàäûâàåò òðåáîâàíèÿ òî÷íîñòè è áûñòðîäåéñòâèÿ èñïîëüçóåìûõ àëãîðèòìîâ ðåøåíèÿ çàäà÷. Íà÷èíàÿ ñ 2008 ãîäà, â íàøèõ ðàáîòàõ èñïîëüçóþòñÿ GPU è multiGPU àðõèòåêòóðû ñîâìåñòíî ñ CPU. Çà ýòî âðåìÿ áûëî ðàññìîòðåíî âîñåìü ïîñòàíîâîê çàäà÷ ËÒÏ. Äëÿ ÷èñëåííîãî ìîäåëèðîâàíèÿ ïðèìåíÿëèñü ðàç- ëè÷íûå ìåòîäû âûñîêîãî ïîðÿäêà.  äàííîé ðàáîòå äëÿ êàæäîãî êëàññà ìåòîäîâ ðàññìàòðèâàþòñÿ õàðàêòåðíûå âû÷èñëèòåëüíûå îïåðàöèè, ïðèâîäÿòñÿ èñïîëüçî- âàííûå áèáëèîòåêè è âûïîëíÿåòñÿ ñðàâíåíèå ýôôåêòèâíîñòè ðàçðàáîòàííûõ àë- ãîðèòìîâ è ïðèìåíåííûõ áèáëèîòåê ñ CPU-âåðñèÿìè êîäà à òàêæå ìåæäó ñîáîé.
Ïîêàçàíî,÷òî â ñðåäíåì íà îäèí GPU ïî ñðàâíåíèþ ñ CPU óñêîðåíèå âàðüèðó- åòñÿ îò 5 äî 35 ðàç. Â ñâÿçè ñî ñëîæíîñòüþ àëãîðèòìîâ, êàê ïðè MPI CPU, òàê è ïðè multiGPU ïîäõîäå, óñêîðåíèå ðåäêî áûâàåò ëèíåéíûì è ïðîïîðöèîíàëüíî ñòåïåííîé ôóíêöèè ñ ïîêàçàòåëåì 0.78-0.81. Äëÿ multiGPU àíàëèçà àëãîðèòìû òåñòèðîâàëèñü íà ïÿòü GPU. Ïîêàçàíû ðåçóëüòàòû ïðè ãèáðèäíîì ïðèìåíåíèè CPU+multiGPU äëÿ îäíîé èç çàäà÷.
Êëþ÷åâûå ñëîâà: multiGPU, ãèáðèäíàÿ àðõèòåêòóðà GPU è CPU, Ïðÿìîå ÷èñëåí- íîå ìîäåëèðîâàíèå, Ëàìèíàðíî - òóðáóëåíòíûé ïåðåõîä, Äèíàìè÷åñêèå ñèñòåìû,
÷èñëåííûå ìåòîäû âûñîêîãî ïîðÿäêà.
1. Ââåäåíèå
Ðàáîòà ÿâëÿåòñÿ îáúåäèíåíèåì ðåçóëüòàòîâ óñêîðåíèé ïàðàëëåëüíûõ âû÷èñëåíèé äëÿ çàäà÷ Ëàìèíàðíî-Òóðáóëåíòíîãî Ïåðåõîäà (ËÒÏ). Âî ìíîãèõ ñòàòüÿõ [622] íå èìåëàñü âîçìîæíîñòü óäåëèòü äàííîìó âîïðîñó äîñòàòî÷íî âíèìàíèÿ.  ðàáîòå ðàññìàòðèâàþòñÿ ðàçëè÷íûå ìåòîäû ÷èñëåííîãî ìîäåëèðîâàíèÿ è îñîáåííîñòè, âîçíèêàþùèå ïðè ýòîì ñ òî÷- êè çðåíèÿ òåõíîëîãèè ðåàëèçàöèè è ïðîãðàììèðîâàíèÿ ïàðàëëåëüíûõ âû÷èñëåíèé. Îñíîâ- íîå âíèìàíèå óäåëÿåòñÿ multiGPU ïîäõîäó.
 îáùåì ñëó÷àå àíàëèç ËÒÏ îñíîâàí íà ïîñòðîåíèè àòòðàêòîðîâ ÷èñëåííûõ àïïðîê- ñèìàöèé ÍÊÇ è èõ ñå÷åíèé ñå÷åíèÿìè Ïóàíêàðå. Ïóñòü M ôàçîâîå ïðîñòðàíñòâî, à φ ôóíêöèÿ äåéñòâèÿ ôàçîâîãî ïîòîêà èçó÷àåìîé äèíàìè÷åñêîé ñèñòåìû íà ôàçîâîå ïðîñòðàí- ñòâî, ïîðîæäàþùàÿ òðàåêòîðèþ γ ∈M×R.Ñå÷åíèå Ïóàíêàðå S ñå÷åíèåM â íåêîòîðîé
∗Ðàáîòà ïîääåðæàíà Ðîññèéñêèì Ôîíäîì Ôóíäàìåíòàëüíûõ Èññëåäîâàíèé (ãðàíò 14-07-00123).
òî÷êå m∈M òàêîå, ÷òîS òðàíñâåðñàëüíî òðàåêòîðèèγ.Òàêèì îáðàçîì, èçó÷àåòñÿ ìíîæå- ñòâî òî÷åê ïåðåñå÷åíèÿ A =γ ∩S. Òåõíè÷åñêè äëÿ ðåàëèçàöèè äàííîãî ìåòîäà ðàñ÷åòíûå äàííûå çàïèñûâàþòñÿ èç íåêîòîðîé òî÷êè ôèçè÷åñêîãî èëè ôóíêöèîíàëüíîãî ïðîñòðàí- ñòâà çà âåñü ïåðèîä ðàñ÷åòà. Ïîñëå ÷åãî ïî èìåþùèìñÿ äàííûì ñòðîèòñÿ ñå÷åíèå Ïóàíêàðå è ìíîæåñòâî òî÷åê ïåðåñå÷åíèÿ íàíîñèòñÿ íà ïëîñêîñòü. Áîëåå äåòàëüíî, ñ.ì. [6].
Ðàáîòà ðàçäåëåíà íà ñåìü ÷àñòåé.  ïåðâîé ÷àñòè â îáùèõ òåðìèíàõ ñòàâÿòñÿ çàäà÷è, ðàññìîòðåííûå â ðàçëè÷íûõ ðàáîòàõ. Äàëåå èçëàãàþòñÿ àðõèòåêòóðû, íà êîòîðûõ ïðîèç- âîäÿòñÿ èññëåäîâàíèÿ.  ïîñëåäóþùèõ ÷åòûðåõ ÷àñòÿõ ðàññìàòðèâàþòñÿ ýëåìåíòû àëãî- ðèòìîâ è ïîëíûå ÷èñëåííûå ðåøåíèÿ ðàçëè÷íûìè ìåòîäàìè íà multiGPU àðõèòåêòóðàõ.  ïîñëåäíåé ÷àñòè äåëàåòñÿ îáùèé âûâîä ïî ïðèìåíèìîñòè multiGPU è ãèáðèäíûõ àëãîðèò- ìîâ â çàäà÷àõ ËÒÏ.
2. Èñõîäíûå íà÷àëüíî-êðàåâûå çàäà÷è
Ðàññìîòðèì çàäà÷è ËÒÏ, êîòîðûå ðåøàëèñü ñ ïðèìåíåíèåì (multi)GPU è ãèáðèäíûõ àðõèòåêòóð.  îáùåì ñëó÷àå Íà÷àëüíî-Êðàåâûå Çàäà÷è (ÍÊÇ) ìîæíî ðàçäåëèòü íà çàäà÷è äëÿ íåñæèìàåìûõ è ñæèìàåìûõ òå÷åíèé. Ïðèíöèïèàëüíîå ðàçëè÷èå çàêëþ÷àåòñÿ â ïðîöå- äóðå ðàñ÷åòà ïðè ÷èñëåííîì ìîäåëèðîâàíèè.  ïåðâîì ñëó÷àå òðåáóåòñÿ ëèáî ïðîåêöèÿ â ïðîñòðàíñòâî ñîëåíîèäàëüíûõ âåêòîð-ôóíêöèé, ëèáî ðàçëîæåíèå ðåøåíèÿ íà ïîòåíöèàëü- íîå è ñîëåíîèäàëüíîå. Âî âòîðîì ñëó÷àå òðåáóåòñÿ ðåøåíèå çàäà÷è ðàñïàäà ïðîèçâîëüíîãî ðàçðûâà íà ýëåìåíòàõ êîíå÷íîìåðíîãî ïðèáëèæåíèÿ [1]. Äëÿ ýêîíîìèè ìåñòà ñôîðìóëèðó- åì òèïè÷íóþ ÍÊÇ äëÿ äàííûõ ñëó÷àåâ.
Ïóñòü çàäàíà îáëàñòü Ω ⊂ Rk, k = 2,3, êðàåâûå è íà÷àëüíûå óñëîâèÿ. Äëÿ êàæäîãî çíà÷åíèÿ áèôóðêàöèîííîãî ïàðàìåòðà (íàïðèìåð ÷èñëà Ðåéíîëüäñà - R) èç çàäàííîãî îò- ðåçêà íàéòè u: Ω×(0, T]→Rk, p: Ω×(0, T]→R òàêèå, ÷òî íàΩâûïîëíÿåòñÿ ñèñòåìà óðàâíåíèé Íàâüå-Ñòîêñà äëÿ âÿçêîé íåñæèìàåìîé æèäêîñòè:
∂u
∂t + (u· ∇)u=−∇p+ 1
R∆u+f (1)
∇ ·u= 0. (2)
Çàäà÷à äëÿ òå÷åíèÿ âÿçêîãî ãàçà çàïèñûâàåòñÿ êàê:
Çàäàíà Ω ⊂ R3, êðàåâûå è íà÷àëüíûå óñëîâèÿ. Îïðåäåëèì ñêàëÿðíûå ôóíêöèè f êàê f : Ω×(0, T]→R è âåêòîð-ôóíêöèè f êàêf : Ω×(0, T]→R3. Òîãäà äëÿ êàæäîãî çíà÷åíèÿ áèôóðêàöèîííîãî ïàðàìåòðà èç çàäàííîãî îòðåçêà íàéòè òàêèåu,ρ,E, ÷òî âûïîëíÿåòñÿ ñèñòåìà óðàâíåíèé:
∂ρ
∂t + ∂
∂xj [ρuj] = 0; ∂
∂t(ρui) + ∂
∂xj [ρuiuj+pδij−τji] =gi, i= 1,2,3;
∂
∂t(ρE) + ∂
∂xj
[ρujE+ujp−uiτij] = 0;E = 1
2ρu2+ρe;
p= (γ−1)(E−1/2ρu2);τij = 2νSij; Sij = 1
2 (∂ui
∂xj +∂uj
∂xi )
+1 3δij∂uk
∂xk.
(3)
Çäåñü:E- ñêàëÿðíàÿ ôóíêöèÿ ïîëíîé ýíåðãèè ãàçà;e- ñêàëÿðíàÿ ôóíêöèÿ âíóòðåííåé ýíåð- ãèè ãàçà; γ - ïîêàçàòåëü àäèàáàòû ãàçà (1.4 äëÿ âîçäóõà);p- ñêàëÿðíàÿ ôóíêöèÿ äàâëåíèÿ;
u- âåêòîð-ôóíêöèÿ ñêîðîñòè ãàçà;ρ- ñêàëÿðíàÿ ôóíêöèÿ ïëîòíîñòè ãàçà;g- âíåøíÿÿ ñèëà;
ν - äèíàìè÷åñêàÿ âÿçêîñòü ãàçà. Ïîäðàçóìåâàåòñÿ ñóììèðîâàíèå ïî îäèíàêîâûì èíäåêñàì.
Ðàçìåðíîñòü êîíå÷íîìåðíîãî àïïðîêñèìèðóþùåãî ïðîñòðàíñòâà âûáèðàåòñÿ èç îöåíêè íà ðàçìåðíîñòü àòòðàêòîðà äëÿ êàæäîé çàäà÷è ïðè âåðõíåì çíà÷åíèè áèôóðêàöèîííîãî ïàðàìåòðà, ñîîòâåòñòâóþùåìó íàèáîëüøåé îöåíêè ðàçìåðíîñòè, ñ.ì. [6,22].  êàæäîì ïîä- ðàçäåëå ïðèâåäåí ìàêñèìàëüíûé ìàñøòàá äèñêðåòèçàöèè.  áîëüøèíñòâå ñëó÷àåâ ðàñ÷åòû
ïðîâîäèëèñü íà áîëåå ãðóáûõ àïïðîêñèìàöèÿõ, êîãäà ïðåäâàðèòåëüíûå ðåçóëüòàòû ñîâïàäà- ëè. Ýòî òàêæå äàëî âîçìîæíîñòü ïðîâåñòè ñîïîñòàâëåíèå ýôôåêòèâíîñòè âû÷èñëèòåëüíûõ ìåòîäîâ ïðè ðàçíûõ ðàçìåðàõ çàäà÷.
2.1. Çàäà÷à òå÷åíèÿ âÿçêîé íåñæèìàåìîé æèäêîñòè ñ ïåðèîäè÷åñêîé âîç- ìóùàþùåé ñèëîé (çàäà÷à À.Í.Êîëìîãîðîâà) íà Tk, k = 2,3.
Äàííûå çàäà÷è ðàññìàòðèâàëèñü â ðàáîòàõ [20,21] à òàêæå íà íåêîòîðûõ êîíôåðåíöèÿõ è â äîêëàäàõ. Îñîáåííîñòè ÍÊÇ: îáëàñòüΩ =Tk(α),0< α≤1,α- ïàðàìåòð óäëèíåíèÿ òî- ðà, äëÿk= 3:Ω = [0,2π/α]×[0,2π]×[0,2π],R=
√A ν
(LyLz
4π2 )34
,f = (sin (ny) sin (mz) ; 0; 0)T; äëÿ k = 2: Ω = [0,2π/α]×[0,2π], R =
√A ν
(Ly 2π
)32
,f = (sin (ny) ; 0)T; m,n - ïàðàìåòðû ñè- ëîâîãî âîçäåéñòâèÿ; A - àìïëèòóäà ñèëîâîãî âîçäåéñòâèÿ; Lj - ðàçìåðû ôèçè÷åñêîãî ïðî- ñòðàíñòâà;ν - êèíåìàòè÷åñêàÿ âÿçêîñòü æèäêîñòè.
 ñâÿçè ñ ïåðèîäè÷íîñòüþ çàäà÷è, äëÿ ðåøåíèÿ (1) ïðèìåíÿþòñÿ äâà ìåòîäà - ìå- òîä Ãàëåðêèíà è ñïåêòðàëüíûé ìåòîä. Äëÿ äâóõìåðíîé çàäà÷è ðåøåíèå èùåòñÿ â òåðìèíàõ ω −Ψ (çàâèõðåííîñòü - ôóíêöèÿ òîêà, áîëåå äåòàëüíî ñ.ì. [20]) è ïðèâîäèòñÿ ê êîíå÷- íîìåðíîé ñèñòåìå ÎÄÓ ìåòîäîì Áóáíîâà-Ãàëåðêèíà.  êà÷åñòâå êîíå÷íîìåðíîãî ïðîñòðàí- ñòâà âûáðàíà ëèíåéíàÿ îáîëî÷êà, íàòÿíóòàÿ íà ïðîñòðàíñòâî ñïåöèàëüíûõ ñïëàéí-ôóíêöèé (S-ñïëàéíû, ñ.ì. [4]).  ðåçóëüòàòå ïîëó÷àåòñÿ ñèñòåìà ÎÄÓ, äëÿ ðåøåíèÿ êîòîðîé ïðè- ìåíÿåòñÿ n ñòàäèéíûé ìåòîä Ðóíãå-Êóòòû. Äëÿ òðåõìåðíîé çàäà÷è ïðèìåíÿåòñÿ ìåòîä Ïåòðîâà-Ãàëåðêèíà è ñïåêòðàëüíûé ìåòîä ñ ïðèìåíåíèåì Äèñêðåòíîãî Ïðåîáðàçîâàíèÿ Ôó- ðüå (ÄÏÔ, áîëåå äåòàëüíî ñ.ì. [21]). Äëÿ ìåòîäà Ïåòðîâà-Ãàëåðêèíà èñïîëüçóþòñÿ ïðîáíûå ôóíêöèè èç ïðîñòðàíñòâà S-ñïëàéí-ôóíêöèé, à ïîâåðî÷íûå ôóíêöèè - ïîëèíîìû ßêîáè.
Âî âñåõ ñèñòåìàõ òåíçîð âàëåíòíîñòè òðè (ñâåðòêà çà ñ÷åò íåëèíåéíîñòè) çàìåíÿåò- ñÿ íà îïåðàöèþ ïðîèçâåäåíèÿ â ôèçè÷åñêîì ïðîñòðàíñòâå äëÿ óñêîðåíèÿ ðàñ÷åòà. Äàííàÿ îïåðàöèÿ âûïîëíÿåòñÿ ñ ïðèìåíåíèåì ëèáî êâàäðàòóð íàèâûñøåé àëãåáðàè÷åñêîé òî÷íî- ñòè (äëÿ ìåòîäîâ Ãàëåðêèíà), ëèáî ñ ïðèìåíåíèåì áûñòðîãî ïðåîáðàçîâàíèÿ Ôóðüå. Ðàç- ìåðíîñòü ïðîñòðàíñòâ - äëÿ ñïåêòðàëüíîãî ìåòîäà: 512×128 = 65536 äëÿ 2D çàäà÷è è 512×128×128 = 8388608äëÿ 3D çàäà÷è (ñ ó÷åòîì äåéñòâèòåëüíîñòè ôóíêöèé â ñïåêòðàëü- íîì ìåòîäå) è äëÿ ìåòîäà Ãàëåðêèíà:240×60 = 14400äëÿ 2D çàäà÷è è40×40×81 = 129600 äëÿ 3D çàäà÷è.
Îñíîâíûìè îïåðàöèÿìè äëÿ äàííûõ çàäà÷ ÿâëÿþòñÿ ïðèìåíåíèå ìàòðèöû ê âåêòîðó è äèñêðåòíîå ïðåîáðàçîâàíèå Ôóðüå.
2.2. Çàäà÷à òå÷åíèÿ âÿçêîé æèäêîñòè â êàíàëå ñ ðàñøèðåíèåì.
Çäåñü ðàññìàòðèâàþòñÿ äâå çàäà÷è - 3D çàäà÷à òå÷åíèÿ ñ óñòóïà è 2D çàäà÷à òå÷åíèÿ ïðîâîäÿùåé æèäêîñòè â êàíàëå ñ ðàñøèðåíèåì. ÎáëàñòüΩ = Ω1\(Ω2∩Ω3),Ω1 =L×H(×W)∗, Ω2 = l×h(×W)∗, Ω3 = (L−l)×h, ãäå L > l,H > h. Âûðàæåíèå â ñêîáêàõ - ()∗ ïðèñóò- ñòâóåò òîëüêî äëÿ 3D çàäà÷è (ò.å. ãëóáèíà), à òàêæå ñòàâèòñÿ íåñèììåòðè÷íîå ðàñøèðåíèå, ò.å. Ω3 =∅. Íà ãðàíèöàõ Ω ñòàâÿòñÿ ãðàíè÷íûå óñëîâèÿ - ñòåíêè(óñëîâèÿ íåïðîòåêàíèÿ è ïðèëèïàíèÿ), ïåðèîäè÷åñêèå, âõîäíûå è âûõîäíûå. Áîëåå äåòàëüíî ïîñòàíîâêè çàäà÷ îïè- ñàíû â ðàáîòàõ [8, 16].  çàäà÷å òå÷åíèÿ ïðîâîäÿùåé æèäêîñòè â êàíàëå, ÍÊÇ ñòàâèòñÿ àíàëîãè÷íî, äîïîëíÿÿñü ñèñòåìîé óðàâíåíèé Ìàêñâåëëà è ñîîòâåòñòâóþùèìè ãðàíè÷íûìè óñëîâèÿìè.
Äëÿ çàäà÷è òå÷åíèÿ ñ óñòóïà ïðèìåíÿåòñÿ ìåòîä ðàñùåïëåíèÿ ñ ïðîåêòîðîì Ãåëüìãîëüöà- Õàäæà. Ýòî ïðèâîäèò ê íåîáõîäèìîñòè ðåøåíèÿ óðàâíåíèÿ Ïóàññîíà íà êàæäîì øàãå ìå- òîäà èíòåãðèðîâàíèÿ ñèñòåìû ÎÄÓ (íàïðèìåð, äëÿ ìåòîäà Ðóíãå-Êóòòû k-îãî ïîðÿäêà, òðåáóåòñÿ ðåøàòü óðàâíåíèå Ïóàññîíà kðàç äëÿ îäíîãî ïîëíîãî øàãà ïî âðåìåíè).
 ñâÿçè ñî ñëîæíîñòüþ ãåîìåòðèè äëÿ äàííûõ çàäà÷ èñïîëüçîâàëèñü ãèáðèäíûå êîíå÷íî-
îáúåìíûå è êîíå÷íî-ýëåìåíòíûå ìåòîäû. Äëÿ àïïðîêñèìàöèè íåëèíåéíûõ ÷àñòåé óðàâíå- íèé ïðèìåíÿëèñü ñõåìû WENO 9 è 11 ïîðÿäêîâ (åùå íå îïóáëèêîâàíû), à äëÿ àïïðîêñè- ìàöèè ëèíåéíûõ ÷ëåíîâ - ìåòîä êîíå÷íûõ ýëåìåíòîâ ñ ïðèìåíåíèåì ïîëèíîìîâ Ëàãðàíæà øåñòîãî ïîðÿäêà. Ðàçìåðíîñòü êîíå÷íîìåðíîãî ïðîñòðàíñòâà äëÿ 3D çàäà÷è òå÷åíèÿ ñ óñòó- ïà ñîñòàâëÿåò512×256×128 = 16777216äëÿ ïåðèîäè÷åñêîãî ïî íàïðàâëåíèþzãðàíè÷íîãî óñëîâèÿ è 512×256×256 = 33554432 äëÿ ãðàíè÷íîãî óñëîâèÿ ñòåíêè ñ îäíîé ñòîðîíû.
Äëÿ 2D çàäà÷è òå÷åíèÿ ïðîâîäÿùåé æèäêîñòè - 1024×256 = 262144. Îñíîâíûå îïåðàöèè - ðàñ÷åò êîíâåêòèâíûõ ÷ëåíîâ óðàâíåíèÿ WENO - ñõåìîé, ðåøåíèå óðàâíåíèÿ Ïóàññîíà è ïðèìåíåíèå ìàòðèöû ê âåêòîðó äëÿ ÌÊÝ.
2.3. Çàäà÷à êîíâåêöèè Ðåëåÿ-Áåíàðà.
Êëàññè÷åñêàÿ çàäà÷à ñâîáîäíîé êîíâåêöèè ðàññìàòðèâàëàñü â ðàáîòàõ [9,11] äëÿ ñèñòå- ìû ïðèáëèæåíèÿ Îáåðáåêà-Áóññèíåñêà. Ðàññìîòðåíû äâå ïîñòàíîâêè çàäà÷è: Ω1 =h×T2 â öèëèíäðè÷åñêîé ãåîìåòðèè è Ω2 =L×H×W - êóáè÷åñêàÿ îáëàñòü, âçÿòàÿ, ïîñêîëüêó â êíèãå [3] èìååòñÿ ýêñïåðèìåíòàëüíîå èññëåäîâàíèå íà÷àëüíûõ áèôóðêàöèé. Óðàâíåíèÿ (1), (2) äîïîëíÿþòñÿ óðàâíåíèåì ïåðåíîñà òåìïåðàòóðû Tt+ (u,∇)T = ∆T, è ñèëîâûì âîçäåéñòâèåì f =RaP r(T−T0)(0; 0;−1)T. Áèôóðêàöèîííûìè ïàðàìåòðàìè ÿâëÿþòñÿ ÷èñ- ëà Ïðàíäòëÿ - P r = ν
χ è Ðýëåÿ - Ra = βgh3∆T
νχ , ïîñòðîåííûå ïî âðåìåííîìó ìàñøòàáó, ñâÿçàííîìó ñ ìàñøòàáîì äèôôóçèè. Çäåñü χ - òåìïåðàòóðîïðîâîäíîñòü æèäêîñòè; β - êî- ýôôèöèåíò òåìïåðàòóðíîãî ðàñøèðåíèÿ; h - âûñîòà ñëîÿ; ∆T - ïåðåïàä òåìïåðàòóðû íà ñëîå; g - óñêîðåíèå ñâîáîäíîãî ïàäåíèÿ. Äëÿ Ω1 ïðèìåíÿëñÿ ñïåêòðàëüíûé ìåòîä íà òðè- ãîíîìåòðè÷åñêèõ ïîëèíîìàõ ñ ââåäåíèåì ôèêòèâíîãî óñëîâèÿ íà âåðõíèõ ãðàíèöàõ, ñ.ì.
[9]. Äëÿ èçáåæàíèÿ ñâåðòêè ïðèìåíÿëñÿ ïåðåõîä â ôèçè÷åñêîå ïðîñòðàíñòâî ñ ïðèìåíåíèåì ÄÏÔ. Äëÿ Ω2 ñ ãðàíè÷íûìè óñëîâèÿìè íà ñòåíêå ïðèìåíÿëñÿ êîìáèíèðîâàííûé êîíå÷íî- îáúåìíûé + êîíå÷íî-ýëåìåíòíûé ìåòîä.
Ðàçìåðíîñòü êîíå÷íîìåðíîãî àïïðîêñèìèðóþùåãî ïðîñòðàíñòâà äëÿΩ1ñîñòàâëÿåò256× 256×256 = 16777216ýëåìåíòîâ, à äëÿ Ω2 - 292×292×292 = 24897088.
Îñíîâíûå îïåðàöèè - ÄÏÔ, ðàñ÷åò êîíâåêòèâíûõ ÷ëåíîâ óðàâíåíèÿ WENO - ñõåìîé, ðåøåíèå óðàâíåíèÿ Ïóàññîíà è ïðèìåíåíèå ìàòðèöû ê âåêòîðó äëÿ ÌÊÝ.
2.4. Êîíâåêòèâíàÿ ñâåðõçâóêîâàÿ íåóñòîé÷èâîñòü òå÷åíèÿ ãàçà.
Çàäà÷à ñòàâèòñÿ êàê îáòåêàíèå òâåðäîãî òåëà ïîòîêîì âÿçêîãî ãàçà ïðè M∞ = 2 äëÿ ñèñòåìû óðàâíåíèé (3). Îñîáåííîñòüþ çàäà÷è ÿâëÿåòñÿ îáðàçîâàíèå ñâåðõçâóêîâûõ è äîçâó- êîâûõ çîí êàê â ñàìîé îáëàñòè ðàñ÷åòà, òàê è íà ãðàíèöå. Ýòî ïðèâîäèò ê íåîáõîäèìîñòè ðàññìîòðåíèÿ ãðàíè÷íûõ óñëîâèé íåîòðàæàþùåãî òèïà. Áèôóðêàöèîííûå ïàðàìåòðû íå âûáèðàþòñÿ à ðàññìàòðèâàåòñÿ ïîâåäåíèå ôàçîâîãî ïðîñòðàíñòâà â çàâèñèìîñòè îò òî÷êè â ôèçè÷åñêîé îáëàñòè ðàñ÷åòà, ò.å. ïàðàìåòðîì âûñòóïàåò êîîðäèíàòà êîíôèãóðàöèîííîãî ïðîñòðàíñòâà. Äëÿ àïïðîêñèìàöèè âÿçêèõ ÷ëåíîâ ïðèìåíÿåòñÿ ðàçðûâíîé ìåòîä Ãàëåðêèíà íà ïîëèíîìàõ Ëåæàíäðà, äëÿ àïïðîêñèìàöèè íåâÿçêèõ ÷ëåíîâ ïðè ðåøåíèè çàäà÷è Ðèìà- íà - ìåòîä WENO 9 è 11 ïîðÿäêîâ äëÿ ðåêîíñòðóêöèè ïîòîêîâ [17]. Ðàçìåðíîñòü çàäà÷è 800×265×256 = 52428800. Îñíîâíûå îïåðàöèè - ðåøåíèå çàäà÷è ðàñïàäà ïðîèçâîëüíîãî ðàçðûâà, ðàñ÷åò ïîòîêîâ ñõåìàìè WENO, ïðèìåíåíèå ìàòðèöû ê âåêòîðó äëÿ ðàçðûâíî- ãî ìåòîäà Ãàëåðêèíà.
2.5. Íåóñòîé÷èâîñòü Êåëüâèíà - Ãåëüìãîëüöà òå÷åíèÿ âÿçêîãî ãàçà.
Çàäà÷à ñòàâèòñÿ â êëàññè÷åñêîì âèäå ñëîÿ ñìåøåíèÿ äâóõ ðàçíîïëîòíîñòíûõ òå÷åíèé, äâèãàþùèõñÿ ñ ðàçëè÷íîé ñêîðîñòüþ. Êðèòåðèåì óñòîé÷èâîñòè òå÷åíèÿ, ÿâëÿåòñÿ ÷èñëî Ðè÷àðäñîíà, îïðåäåëÿþùååñÿ êàê Ri = ∆ρgh
ρA∆U2, ãäå g - óñêîðåíèå ñâîáîäíîãî ïàäåíèÿ,
h - âûñîòà ñëîÿ ñìåøåíèÿ, ∆U - ðàçíèöà ñêîðîñòåé ñäâèãîâîãî ñëîÿ, ρA - ñðåäíÿÿ ïëîò- íîñòü æèäêîñòè (ãàçà), ∆ρ - ðàçíèöà ïëîòíîñòåé ñäâèãîâûõ ñëîåâ. Íà îñíîâå ëèíåéíîãî àíàëèçà óñòîé÷èâîñòè îòìå÷àåòñÿ, ÷òî òóðáóëåíòíîå ïåðåìåøèâàíèå íàñòóïàåò ïðèRi < 14. Îáëàñòü Ω - ïðÿìîóãîëüíàÿ, ðàçìåðîì X = 2.8, Y = 0.3 Z = 0.7, ñèëà ãðàâèòàöèè íà- ïðàâëåíà ïî íàïðàâëåíèþ îñè −Y. Ãðàíè÷íûå óñëîâèÿ: óñëîâèÿ ñèììåòðèè ïî îñè Y ïðè y= 0 èy= 0.3, óñëîâèÿ ïåðèîäè÷íîñòè ïî îñèZ, óñëîâèÿ ñâîáîäíîãî èñòå÷åíèÿ èç îáëàñòè ïî îñè X ïðè x = 2.8. Ïðè x = 0.0 çàäàþòñÿ íà÷àëüíîå âîçìóùåíèå ïîòîêà ñ àìïëèòó- äîé δ ≤1·10−5.  äàííîé çàäà÷å òàêæå òðåáóåòñÿ ïðèìåíåíèå íåîòðàæàþùèõ ãðàíè÷íûõ óñëîâèé, ñ.ì. [19]. Ñèëà ãðàâèòàöèè ïîäáèðàåòñÿ òàê, ÷òîáû îáåñïå÷èòü íåóñòîé÷èâîñòü ïî êðèòåðèþ Ðè÷àðäñîíà. Äàâëåíèå çàäàåòñÿ òàê, ÷òîáû ìàêñèìàëüíîå ÷èñëî Ìàõà ñîñòàâëÿ- ëî 0.8. Ðàñ÷åò ÷èñëà Ðè÷àðäñîíà âûïîëíÿåòñÿ ïî óêàçàííîìó âûðàæåíèþ. Çäåñü h = 0.15,
∆ρ = 0.4,ρA= 0.8. Äëÿ ó÷åòà âÿçêîñòè ââîäèòñÿ åäèíñòâåííîå ÷èñëî Ðåéíîëüäñà, îïðåäå- ëÿåìîå êàê R= ∆U hρA
ν . Ïîëó÷àåòñÿ äâóõïàðàìåòðè÷åñêàÿ çàäà÷à, ãäå áèôóðêàöèîííûìè ïàðàìåòðàìè ÿâëÿþòñÿRi è R.
Êàê è â çàäà÷å âûøå, äëÿ àïïðîêñèìàöèè âÿçêèõ ÷ëåíîâ ïðèìåíÿåòñÿ ðàçðûâíîé ìåòîä Ãàëåðêèíà íà ïîëèíîìàõ Ëåæàíäðà, äëÿ àïïðîêñèìàöèè íåâÿçêèõ ÷ëåíîâ ïðè ðåøåíèè çà- äà÷è Ðèìàíà - ìåòîä WENO 9 è 11 ïîðÿäêîâ äëÿ ðåêîíñòðóêöèè ïîòîêîâ [17]. Ðàçìåðíîñòü êîíå÷íîìåðíîãî ïðîñòðàíñòâà - 800×350×350 = 98000000. Àíàëîãè÷íî ïðåäûäóùåé çà- äà÷å, îñíîâíûå îïåðàöèè - ðåøåíèå çàäà÷è ðàñïàäà ïðîèçâîëüíîãî ðàçðûâà, ðàñ÷åò ïîòîêîâ ñõåìàìè WENO, ïðèìåíåíèå ìàòðèöû ê âåêòîðó äëÿ ðàçðûâíîãî ìåòîäà Ãàëåðêèíà.
3. Ìèíè multi-GPU êëàñòåð
Çà äîëãîå âðåìÿ ïðîâåäåíèÿ ðàáîò ñìåíèëîñü ìíîãî ïîêîëåíèé ãðàôè÷åñêèõ ñîïðîöåñ- ñîðîâ è öåíòðàëüíûõ ïðîöåññîðîâ. Èñòîðè÷åñêè, âñå ðàáîòû âûïîëíåíû íà GPU êîìïàíèè NVIDIA, öåíòðàëüíûõ ïðîöåññîðàõ êîìïàíèè INTEL. Ðàñ÷åòû â ðàáîòàõ ñ 2007 äî 2009 ãîäà ïðîâîäèëèñü ñ èñïîëüçîâàíèåì äâóõ GTX 8800, ñ 2009 ïî 2010 - Tesla C1060, ñ 2010 ïî 2013ã.
- ÷åòûðåõ k20, ñ 2013ã ïî íàñòîÿùåå âðåìÿ - ïÿòè GK110-B1 íà áàçå GeForce. Öåíòðàëüíûå ïðîöåññû - Intel Xeon, íà÷èíàÿ ñ L7445, 4 ÿäðà X4, äî E5 Ivy Bridge, 16 ÿäåð X2. Äëÿ ñîâðå- ìåííîãî ñðàâíåíèÿ âñå ìåòîäû áûëè ïîâòîðíî ïåðåìåòðèðîâàííû íà êîíôèãóðàöèþ ñîáðàí- íîãî multi-GPU êëàñòåðà: Intel Xeon E5 Ivy Bridge 16 ÿäåð, ïÿòü GPU NVIDIA GTX TITAN BLACK (GK110-B1), êîíôèãóðàöèÿ ñîáðàíà ñ îáùåé ïàìÿòüþ (64Gb). Òàêàÿ êîíôèãóðà- öèÿ ÿâëÿåòñÿ îòíîñèòåëüíî äåøåâîé è îáëàäàåò äîñòàòî÷íîé âû÷èñëèòåëüíîé ìîùíîñòüþ äëÿ ïðîâåäåíèÿ ïðÿìîãî ÷èñëåííîãî ìîäåëèðîâàíèÿ çàäà÷ ËÒÏ íà íà÷àëüíûõ ñòàäèÿõ. Âñå ìåòðèêè âûïîëíåíû íà äàííîé ìàøèíå, åñëè íå îãîâîðåíî îáðàòíîå. Îòäåëüíî ïðèâîäÿò- ñÿ äàííûå ïî ïðîèçâîäèòåëüíîñòè â êîëè÷åñòâàõ îïåðàöèé ñ ïëàâàþùåé òî÷êîé â ñåêóí- äó (FLOPS). Îäíàêî ïî îïûòó ðàñ÷åòîâ, àâòîðû ñ÷èòàþò îöåíêó FLOPS íåäîñòàòî÷íîé è ïðèâîäèòñÿ ñîïîñòàâëåíèå ðàçëè÷íûõ ìåòîäîâ ïî âðåìåíè âûïîëíåíèÿ. Âñå âû÷èñëåíèÿ ïðèâîäÿòñÿ â àðèôìåòèêå ñ ïëàâàþùåé òî÷êîé ñ äâîéíîé òî÷íîñòüþ.
4. Äèñêðåòíîå Ïðåîáðàçîâàíèå Ôóðüå
Äëÿ ðàñ÷åòà ÄÏÔ èñïîëüçóþòñÿ äâå ðåàëèçàöèè àëãîðèòìà áûñòðîãî ïðåîáðàçîâàíèÿ Ôóðüå (ÁÏÔ): CUFFT (CUDA v.6.5) è FFTW(3.3.4). Äëÿ óñêîðåíèÿ ðàñ÷åòà CUFFT èñ- ïîëüçîâàëñÿ â ðåæèìå äâóõ GPU, FFTW â îäíîïîòî÷íîì è ìíîãîïîòî÷íîì (16 ïîòîêîâ) ðåæèìå.
Íà ðèñ. 1 ïîêàçàíî ñîïîñòàâëåíèå âðåìåíè ðàñ÷åòà îäíîãî ïîëíîãî ïðåîáðàçîâàíèÿ R3 → C3 è C3 → R3 íà ðåàëüíûõ äàííûõ ñîîòâåòñòâóþùèõ çàäà÷ ïðè ïðèìåíåíèè îä- íîé è äâóõ GPU ïðè ðàçìåðàõ îáëàñòè âèäà 2p ×2q×2n,{p, q, n} ∈ N. Åñëè ñîïîñòàâèòü ðåçóëüòàòû áîëåå ðàííåãî òåñòà ïðè âåðñèÿõ CUFFT (CUDA 1.1/2.0) è FFTW(3.2) [5] âèäíî,
÷òî áèáëèîòåêà CUFFT óëó÷øåíà ïî ñðàâíåíèþ ñ ïðåäûäóùèìè âåðñèÿìè. Ìàêñèìàëüíîå
(a) Ïðîèçâîäèòåëüíîñòü îòíîñèòåëüíî ðàçìåðà
çàäà÷è. (b) Âðåìÿ âûïîëíåíèÿ îòíîñèòåëüíî ðàçìåðà çà-
äà÷è.
Ðèñ. 1. Ñîïîñòàâëåíèå ïðîèçâîäèòåëüíîñòè áèáëèîòåê FFTW è CUFFT íà 3D çàäà÷å À.Í.Êîëìîãîðîâà.
óñêîðåíèå ñîñòàâèëî 17 ðàç, ïî ñðàâíåíèþ ñ îäíîïîòî÷íûì FFTW è â 3.6 ðàçà ïî ñðàâíåíèþ ñ 16 ïîòî÷íûì FFTW. Ïðè ýòîì, äëÿ çàäà÷ ìàëîé ðàçìåðíîñòè (2D çàäà÷è, òåñòîâûå 3D çà- äà÷è), èñïîëüçîâàíèå CUFFT íå ÿâëÿåòñÿ îïðàâäàííûì. Ðàáîòà ïðè äâóõ GPU äëÿ CUFFT îïðàâäàíà, íà÷èíàÿ ñ ðàçìåðíîñòè çàäà÷è 1283 è âûøå. Ïðè ýòîì äëÿ ðàññìîòðåííûõ âû- øå çàäà÷ âñå ðåçóëüòàòû áûëè ïîëó÷åíû íà îäíîì GPU ââèäó óìåíüøåíèÿ êîíå÷íîìåðíîãî ïðîñòðàíñòâà ïðè ñðàâíåíèè ðåçóëüòàòîâ. Åñëè ëèíåéíûå ðàçìåðû îáëàñòè íå ÿâëÿþòñÿ ñòå- ïåíüþ äâîéêè, òî ýôôåêòèâíîñòü CUFFT ïàäàåò, è óñêîðåíèå ïî ñðàâíåíèþ ñ 16 ïîòî÷íîé FFTW ñîñòàâëÿåò 2.4 ðàçà.
 öåëîì, äëÿ ñïåêòðàëüíûõ (ïñåâäîñïåêòðàëüíûõ) ìåòîäîâ ïðèìåíåíèå CUFFT è CUDA äëÿ îñòàëüíîé ÷àñòè êîäà ïðèâîäèò ê óñêîðåíèþ â ïÿòü-øåñòü ðàç, ïî ñðàâíåíèþ ñ 16 ïî- òî÷íîé OpenMP âåðñèåé êîäà.
5. Ïðèìåíåíèå ìàòðèöû ê âåêòîðó
Äàííàÿ îïåðàöèÿ ïðèìåíÿåòñÿ, â îñíîâíîì, â ìåòîäàõ Ãàëåðêèíà è ÌÊÝ, ïðè ýòîì ìàò- ðèöû ìîãóò áûòü êàê ïëîòíûå, òàê è ðàçðÿæåííûå (ÌÊÝ). Äëÿ ïëîòíûõ ìàòðèö ïðèìåíÿ- þòñÿ áèáëèîòåêè CUBLAS è MAGMA. Ñõîæèå ïî ñòðóêòóðå, îíè èìåþò ïðèíöèïèàëüíóþ ðàçíèöó, çàêëþ÷àþùóþñÿ â òîì, ÷òî MAGMA ìîæåò áûòü ñêîìïèëèðîâàíà ïîä êîíêðåò- íóþ àðõèòåêòóðó ñ àâòîìàòè÷åñêèì ó÷åòîì êîëè÷åñòâà GPU, óñòàíîâëåííûõ íà îäíîé íî- äå êëàñòåðà. Ïîýòîìó óêàçàííàÿ âûøå êîíôèãóðàöèÿ ìèíè êëàñòåðà ÿâëÿåòñÿ äîñòàòî÷íî óñïåøíîé.
Íà ðèñ. 2 ïîêàçàíî ñðàâíåíèå ïðîèçâîäèòåëüíîñòè è ÷èñòîãî âðåìåíè äëÿ îïåðàöèé magmablas_dsymv_mgpu ècublasDsymv.
Òàêèì îáðàçîì, äîñòèãàåòñÿ ïðîèçâîäèòåëüíîñòü 85 GFLOPS íà îäíîé GPU è 435 GFLOPS íà ïÿòè GPU. Êàê âèäíî, ïðèìåíåíèå MAGMA îïðàâäûâàåòñÿ ïðè ðàñ÷åòå çàäà÷ ËÒÏ è ïîçâîëÿåò çíà÷èòåëüíî ñîêðàòèòü òðóäîçàòðàòû ïðè íàïèñàíèè êîäà.
6. Ñõåìà WENO è îáìåí äàííûõ íà multiGPU
Âîïðîñ î êîììóíèêàöèÿõ äëÿ multiGPU íå ïîäíèìàëñÿ ïðè èñïîëüçîâàíèè áèáëèîòåê.
Ïðè íàïèñàíèè ñîáñòâåííûõ ÷àñòåé êîäà äàííûé âîïðîñ âîçíèêàåò. Âî âñåõ ðåàëèçàöèÿõ èñ- ïîëüçóåòñÿ MPI îáìåí äàííûìè, â êîòîðîì âûäåëÿåòñÿ áóôåð â îïåðàòèâíîé ïàìÿòè, îòíî- ñÿùåéñÿ ê öåíòðàëüíîìó ïðîöåññîðó. Àëüòåðíàòèâîé òàêîìó ïîäõîäó ÿâëÿåòñÿ DirectGPU, ïîääåðæèâàåìûé CUDA íà÷èíàÿ ñ âåðñèè 5.5. Àâòîðû îñòàíîâèëèñü íà ïðèìåíåíèè ïåðâî-
(a) Ïðîèçâîäèòåëüíîñòü îòíîñèòåëüíî ðàçìåðà
çàäà÷è. (b) Âðåìÿ îäíîãî øàãà ïî âðåìåíè îòíîñèòåëüíî
ðàçìåðà çàäà÷è.
Ðèñ. 2. Ïðîèçâîäèòåëüíîñòüy=αAx+βy äëÿ MAGMA â ðåæèìå Multi-GPU è CUBLAS.
ãî ïîäõîäà, ïîñêîëüêó îí ìîæåò áûòü íàïðÿìóþ ïðèìåíåí ïðè ïåðåíîñå êîäà ñ îäíîé íîäû íà íåñêîëüêî. Ìèíóñàìè òàêîãî ïîäõîäà ÿâëÿåòñÿ çàãðóçêà öåíòðàëüíîãî ïðîöåññîðà ïðè ïåðåäà÷å äàííûõ è áîëüøåå âðåìÿ, çàòðà÷èâàåìîå íà ïåðåíîñ äàííûõ. Íà ïðàêòèêå â âû- øåóêàçàííûõ çàäà÷àõ çàòðàòû íà ïåðåäà÷ó äàííûõ ñîñòàâëÿëè 0.3-0.5% îò âñåõ çàòðàò (ïî nvprof). Ïðè ïðèìåíåíèè íåÿâíûõ ìåòîäîâ çàòðàòû íà ïåðåñûëêó âîçðàñòàþò è ñîñòàâëÿþò îò 1.5-3%. Ñíèæåíèÿ çàòðàò ìîæíî äîáèòüñÿ àñèíõðîííûì âûçîâîì ïåðåñûëêè äàííûõ.
Îòäåëüíî êðàòêî ñëåäóåò óïîìÿíóòü äîñòàòî÷íî ñëîæíóþ ïðîöåäóðó âû÷èñëåíèÿ ñõåìû WENO. Ïîñêîëüêó øàáëîí ñõåìû ÿâëÿåòñÿ äîñòàòî÷íî ãðîìîçäêèì (17×3 òî÷åê øàáëîíà äëÿ WENO-9 è 21×3 òî÷åê øàáëîíà äëÿ WENO-11) âû÷èñëåíèå âåñîâ ïîëèíîìîâ Ëàãðàí- æà âûïîëíÿëîñü â ÷åòûðåõ (øåñòè äëÿ WENO-11) îòäåëüíûõ ÿäðàõ. Ñåòêà êðàñèëàñü â öâåòà è êàæäîå ÿäðî ðàáîòàëî íà ñâîåì öâåòå, òàêèì îáðàçîì ìèíèìèçèðóåòñÿ coalescing.
Ïîñëå ñáîðêè âåñîâ, ðåçóëüòàòû îáúåäèíÿëèñü â ïðîìåæóòî÷íûõ ÿäðàõ, ïîñëå ÷åãî âûïîë- íÿëàñü îêîí÷àòåëüíàÿ ñáîðêà TVB ñõåìû ñ èñïîëüçîâàíèåì âåñîâ øàáëîíîâ. Íåñìîòðÿ íà ñëîæíîñòü, òàêîé ìåòîä îêàçàëñÿ â 4.5 ðàçà áûñòðåå, ÷åì íàèâíûé ìåòîä.
Ðåàëèçàöèÿ multiGPU+CPU áûëà âûïîëíåíà íà çàäà÷å î íåóñòîé÷èâîñòè Êåëüâèíà - Ãåëüìãîëüöà. Ñóòü â òîì, ÷òî ðàññìîòðåíèå õàðàêòåðèñòè÷åñêèõ ãðàíè÷íûõ óñëîâèé âìåñòå ñ áóôåðíûì ñëîåì ïðèâîäèò ê áîëüøîìó èñïîëüçîâàíèþ ðåãèñòðîâ äëÿ GPU, ÷òî çíà÷è- òåëüíî çàìåäëÿåò âûïîëíåíèå ñîîòâåòñòâóþùèõ ÿäåð. Ïîýòîìó ðàñ÷åò õàðàêòåðèñòè÷åñêèõ ãðàíè÷íûõ óñëîâèé âûïîëíÿåòñÿ íà CPU íà îñíîâå MPI. Ðåçóëüòàòû, ïðåäñòàâëåííûå íà ðèñ. 3, äëÿ äàííîé êîíôèãóðàöèè ïîëó÷åíû íà ìàøèíå ñ äâóìÿ CPU, 16 ÿäåð íà êàæäûé.
Îäèí öåíòðàëüíûé ïðîöåññîð îòâå÷àë çà êîììóíèêàöèþ ìåæäó GPU, äðóãîé - çà ðàñ÷åò ãðàíè÷íûõ óñëîâèé.  êîíôèãóðàöèè ñ îäíèì CPU ïðèðîñòà ïðîèçâîäèòåëüíîñòè íå áûëî îáíàðóæåíî.
Õàðàêòåðíûé ëîêàëüíûé ìèíèìóì íà ðèñ. 3b êàê ðàç ñâÿçàí ñ âûõîäíûìè ãðàíè÷- íûìè óñëîâèÿìè, íà êîòîðûõ âîçíèêàåò òðàíñçâóêîâîé ðåæèì òå÷åíèÿ. Ïðè ïðèìåíåíèè CPU+multiGPU äàííûé ìèíèìóì óìåíüøàåòñÿ.
7. Óðàâíåíèå Ïóàññîíà, ðåøåíèå ÑËÀÓ
Äëÿ ðåøåíèÿ óðàâíåíèÿ Ïóàññîíà ïðèìåíÿåòñÿ ìíîãîñåòî÷íûé ãåîìåòðè÷åñêèé ìåòîä (ÌÃÌ), ñ.ì. [10]. Îí æå ïðèìåíÿåòñÿ ïðè íåîáõîäèìîñòè ðåøåíèÿ ÑËÀÓ. Òàêàÿ íåîáõî- äèìîñòü âîçíèêàåò ïðè èñïîëüçîâàíèè íåÿâíûõ ìåòîäîâ öåëü êîòîðûõ - âûéòè íà êâàçè- ñòàöèîíàðíîå ðåøåíèå ïåðåä ïðèìåíåíèåì âûñîêîòî÷íûõ ÿâíûõ ìåòîäîâ (îãðàíè÷åíèå ïî óñòîé÷èâîñòè ïðè ïðÿìîì ÷èñëåííîì ìîäåëèðîâàíèè çàäà÷ ËÒÏ ìåíåå ñòðîãîå, ÷åì îãðà- íè÷åíèå ïî òî÷íîñòè).  îòëè÷èå îò ðàáîòû [10], íà âåðõíåì óðîâíå ÌÃÌ ïðèìåíÿþòñÿ ðàçëè÷íûå ìåòîäû Êðûëîâñêîãî òèïà. Èç ïðàêòèêè èñïîëüçîâàíèÿ íàèáîëåå óäà÷íûì îêà-
(a) Ïðîèçâîäèòåëüíîñòü âñåãî êîäà îòíîñèòåëü- íî êîëè÷åñòâà GPU (äëÿ ðàçëè÷íûõ ðàçìåðîâ
çàäà÷). (b) Ïðîèçâîäèòåëüíîñòü âñåãî êîäà îòíîñèòåëü-
íî ðàçìåðà çàäà÷è.
(c) Âðåìÿ îäíîãî øàãà ïî âðåìåíè îòíîñèòåëü- íî êîëè÷åñòâà GPU (äëÿ ðàçëè÷íûõ ðàçìåðîâ
çàäà÷). (d) Âðåìÿ îäíîãî øàãà ïî âðåìåíè îòíîñèòåëüíî
ðàçìåðà çàäà÷è.
Ðèñ. 3. Ïðîèçâîäèòåëüíîñòü âñåãî êîäà äëÿ ìîäåëèðîâàíèÿ òå÷åíèÿ âÿçêîãî ãàçà.
çàëñÿ ìåòîäBiCGstab. Ïðè óñëîâèè èñïîëüçîâàíèÿ ïðîëîíãàòîðîâ è ðåñòðèêòîðîâ âûñîêîãî ïîðÿäêà (ñ 4 äî 8) [10] óäàåòñÿ äîáèòüñÿ àñèìïòîòèêèO(NxNyNz)äëÿ âðåìåíè ñõîäèìîñòè ïðè ðåøåíèè óðàâíåíèÿ Ïóàññîíà è O((NxNyNz)1.15−1.2) ïðè ïðèìåíåíèè íåÿâíûõ ìåòîäîâ è ðåøåíèè íåÿâíîé ÷àñòè äëÿ ðàçðûâíûõ ìåòîäîâ Ãàëåðêèíà (ãäå íåîáõîäèìî). Íà ðèñ. 4 ïîêàçàíà ïðîèçâîäèòåëüíîñòü ðåøåíèÿ óðàâíåíèÿ Ïóàññîíà äëÿ çàäà÷è òå÷åíèÿ ñ óñòóïà.
Îïòèìàëüíûì äëÿ äàííîé çàäà÷è îêàçàëñÿW - öèêë, 5 ñåòîê ñ èçìåíåíèåì ðàçìåðà â2ðàçà, ïðîëîíãàòîðû/ðåñòðèêòîðû øåñòîãî ïîðÿäêà. Ïðè ýòîì, íà ïîñëåäíèõ âåòêàõ ïðîëîíãàòî- ðîâ ïðèìåíÿþòñÿ BiCGstab ìåòîäû â òåñòîâîì ðåæèìå, äëÿ îöåíêè ñêîðîñòè ñõîäèìîñòè è ñëåäóþùàÿ ïðîëîíãèðóþùàÿ âåòâü ïîäïðàâëÿåòñÿ ñ ó÷åòîì ýòèõ ðåçóëüòàòîâ. Êðèòåðèé ñõîäèìîñòè íà âåðõíåì óðîâíå - ïî íîðìåC,ε≤5.0·10−15.
Êàê âèäíî èç ðèñ. 4, íà äàííîé çàäà÷å óäàëîñü äîáèòüñÿ ïðàêòè÷åñêè èäåàëüíîãî ìàñ- øòàáèðîâàíèÿ, õàðàêòåðíîãî äëÿ ìíîãîñåòî÷íûõ ãåîìåòðè÷åñêèõ ìåòîäîâ. Êðîìå òîãî, ïðî- èçâîäèòåëüíîñòü îäíîãî GPU ñîñòàâèëà 620.33 GFLOPS, ÷òî ñîñòàâëÿåò 32.5% îò ìàêñè- ìàëüíîé òåîðåòè÷åñêîé ïðîèçâîäèòåëüíîñòè íà îïåðàöèÿõ ñ ïëàâàþùåé òî÷êîé äâîéíîé òî÷íîñòè.
8. Îáùèé àíàëèç ðåçóëüòàòîâ
Çà äëèòåëüíîå âðåìÿ íàïèñàíèÿ êîäîâ äëÿ ïðÿìîãî ÷èñëåííîãî ìîäåëèðîâàíèÿ ËÒÏ, àâòîðû ïðèøëè ê âûâîäó, ÷òî îòíîñèòåëüíî íåáîëüøèå ìàøèíû, ïîñòðîåííûå íà àðõèòåêòó- ðå multiGPU+CPU ÿâëÿþòñÿ îïòèìàëüíûìè ïðè òðåáîâàíèÿõ ê ñåòêàì íå áîëüøå 100 ìëí.
ýëåìåíòîâ. Ïðè÷åì áîëåå íèçêàÿ ïðîèçâîäèòåëüíîñòü, ïî ñðàâíåíèþ ñ HPC êëàñòåðàìè, ìî- æåò áûòü êîìïåíñèðîâàíà ìîíîïîëüíûì äîñòóïîì ê ìàøèíå.  äàëüíåéøåì, ïîñëå îòëàäêè âñåõ óçëîâ íà òàêîé ìàøèíå, çàäà÷è ñ áîëüøèì êîëè÷åñòâîì ýëåìåíòîâ ìîãóò áûòü ëåã-
(a) Ïðîèçâîäèòåëüíîñòü îòíîñèòåëüíî ðàçìåðà
çàäà÷è. (b) Âðåìÿ îäíîãî øàãà ïî âðåìåíè îòíîñèòåëüíî
ðàçìåðà çàäà÷è.
Ðèñ. 4. Ïðîèçâîäèòåëüíîñòü ïðè ðåøåíèè óðàâíåíèÿ Ïóàññîíà ãåîìåòðè÷åñêèì ìíîãîñåòî÷íûì ìåòîäîì.
êî ïåðåíåñåíû íà ïðîìûøëåííûé êëàñòåð. Íàèáîëåå çàòðàòíûìè äëÿ äàííîé àðõèòåêòóðû äî ñèõ ïîð îñòàþòñÿ ýëëèïòè÷åñêèå çàäà÷è, íåÿâíûå ìåòîäû è îãðàíè÷åíèå íà êîëè÷åñòâî ðåãèñòðîâ íà ÿäðî.
 öåëîì, èñïîëüçîâàíèå íàìè ÷èñëåííûõ ìåòîäîâ íà äàííîé àðõèòåêòóðå ïîçâîëèëî â 5-7 ðàç óñêîðèòü ðàñ÷åòû ïî ñðàâíåíèþ ñ áîëåå ðàííèìè êîíôèãóðàöèÿìè, óïîìÿíóòûìè âûøå. Óñêîðåíèå, ïî ñðàâíåíèþ ñ CPU âåðñèÿìè êîäîâ ñîñòàâëÿåò îò 5-6 ðàç ïðè èñïîëüçî- âàíèè áèáëèîòåê, äî 35, à â îòäåëüíûõ çàäà÷àõ è äî 50 ðàç.
Íåñìîòðÿ íà âñå åùå ñëàáóþ ïîääåðæêó äâîéíîé òî÷íîñòè â GPU, ïðîèçâîäèòåëüíîñòè óæå äîñòàòî÷íî ÷òîáû ïîëó÷àòü î÷åíü äåòàëüíîå ðàçðåøåíèå òå÷åíèÿ. Íàïðèìåð, ðåæèì òå÷åíèÿ è ïðîåêöèÿ ôàçîâîãî ïîðòðåòà âR3 (è åãî ìíîãîìåðíûå ñå÷åíèÿ) äëÿ çàäà÷è Êåëü- âèíàÃåëüìãîëüöà ïîêàçàíû íà ðèñ. 5. Âèäíî, ÷òî òî÷íîñòè äîñòàòî÷íî ÷òîáû ðàçðåøèòü òå÷åíèå, â êîòîðîì áûë îáíàðóæåí óñòîé÷èâûé ÷åòûðåõìåðíûé èíâàðèàíòíûé òîð, ïðè òîì
÷òî â äàííîé çàäà÷å íàáëþäàåòñÿ ìàñøòàáíûé ýôôåêò ìóëüòèóñòîé÷èâîñòè.
(a) Èçîïîâåðõíîñòè ïëîòíîñòè. (b) Ïðîåêöèÿ ÷åòûðåõìåðíîãî èíâàðèàíòíîãî òî- ðà.
Ðèñ. 5. Îäèí èç ðåæèìîâ ïðè ËÒÏ â çàäà÷å ÊåëüâèíàÃåëüìãîëüöà.
Ëèòåðàòóðà
1. Toro E.F. Riemann Solvers and Numerical Methods for Fluid Dynamics. Springer-Verlag, 1999.
2. Sarrisy I. E., Kassinosz S., Knaepeny B. , Carati D. Large-eddy simulations of the turbulent Hartmann ow close to the transitional regime.//Center for Turbulence Research:Proceedings of the Summer Program. 2006. P. 387 397.
3. Ôðèê Ï.Ã. Òóðáóëåíòíîñòü: ïîäõîäû è ìîäåëè. Ìîñêâà-Èæåâñê: Èíñòèòóò êîìïüþòåðíûõ èññëåäîâàíèé. 2003. 292 ñ.
4. Ñèëàåâ Ä.À., Êîðîòàåâ Ä.Î. Ðåøåíèå êðàåâûõ çàäà÷ ñ ïîìîùüþ S-ñïëàéíà.//
Êîìïüþòåðíûå èññëåäîâàíèÿ è ìîäåëèðîâàíèå. 2009. Ò.1. 2. Ñ. 161 171 5. CUFFT 1.1 / 2.0 vs FFTW 3.1.2 (x86_64) vs FFTW 3.2 (Cell) comparison. URL:
http://www.sharcnet.ca/~merz/CUDA_benchFFT/
6. Evstigneev N.M. and Magnitskii N.A. FSM Scenarios of Laminar-Turbulent Transition in Incompressible Fluids. Chapter 10 in Nonlinearity, Bifurcation and Chaos Theory and Applications. INTECH. 2013. P. 250 280.
7. Åâñòèãíååâ Í.Ì., Ìàãíèöêèé.Í.À., Ñèäîðîâ Ñ.Â. Íîâûé ïîäõîä ê îáúÿñíåíèþ ïðèðîäû òóðáóëåíòíîñòè âÿçêîé íåñæèìàåìîé æèäêîñòè. // Òðóäû ÈÑÀ ÐÀÍ. 2008.
Ò.33. C. 49 65.
8. Åâñòèãíååâ Í.Ì., Ìàãíèöêèé Í.À., Ñèäîðîâ Ñ.Â. Î ïðèðîäå òóðáóëåíòíîñòè â çàäà÷å äâèæåíèÿ æèäêîñòè çà óñòóïîì.// Äèôôåðåíöèàëüíûå óðàâíåíèÿ. 2009. Ò.45. 1. C.
69 73.
9. Åâñòèãíååâ Í.Ì., Ìàãíèöêèé Í.À., Ñèäîðîâ Ñ.Â. Î ïðèðîäå òóðáóëåíòíîñòè â êîíâåêöèè Ðåëåÿ-Áåíàðà.// Äèôôåðåíöèàëüíûå óðàâíåíèÿ. 2009. Ò.45. 6. C. 909 912.
10. Åâñòèãíååâ Í.Ì. Èíòåãðèðîâàíèå óðàâíåíèÿ Ïóàññîíà ñ èñïîëüçîâàíèåì ãðàôè÷åñêîãî ïðîöåññîðà òåõíîëîãèè CUDA. // Âû÷èñëèòåëüíûå ìåòîäû è ïðîãðàììèðîâàíèå. 2009.
Ò10, C. 268 274.
11. Åâñòèãíååâ Í.Ì., Ìàãíèöêèé Í.À. Î âîçìîæíûõ ñöåíàðèÿõ ïåðåõîäà ê òóðáóëåíòíîñòè â êîíâåêöèè Ðýëåÿ-Áåíàðà. // Äîêëàäû Àêàäåìèè Íàóê. 2010. Ò.433. 3. Ñ. 1 5 12. Åâñòèãíååâ Í.Ì., Ìàãíèöêèé Í.À. Íåëèíåéíàÿ Äèíàìèêà  Íà÷àëüíî-Êðàåâîé Çàäà÷å
Òå÷åíèÿ Æèäêîñòè Ñ Óñòóïà Äëÿ Ãèäðîäèíàìè÷åñêîãî Ïðèáëèæåíèÿ Óðàâíåíèé Áîëüöìàíà. // Äèôôåðåíöèàëüíûå Óðàâíåíèÿ. 2010. Ò. 46. 12. Ñ. 1794 1798 13. Evstigneev N.M., Magnitskii N.A., Sidorov S.V. Nonlinear dynamics of laminar-turbulent
transition in three dimensional RayleighBenard convection. //Communications in
Nonlinear Science and Numerical Simulation. Volume 15. Issue 10. October 2010. P. 2851 2859
14. Åâñòèãíååâ H.M. Ïðèìåíåíèå ãðàôè÷åñêîãî ïðîöåññîðà äëÿ óñêîðåíèÿ ÷èñëåííîãî ñåòî÷íîãî ìåòîäà Áîëüöìàíà ñ ýíòðîïèéíîé ñòàáèëèçàöèåé // Òðóäû ÈÑÀ ÐÀÍ. 2010.
Ò.53. 14. Ñ. 111 123.
15. Evstigneev N.M. and Magnitskii N.A. Nonlinear Dynamics of Laminar-Turbulent Transition in Back Facing Step Problem for Bolzmann Equations in Hydrodynamic Limit.//AIP. 2010. Volume 1281. P. 896 900.
16. Åâñòèãíååâ Í. Ì., Ìàãíèöêèé Í. À., Ðÿáêîâ Î. È. ×èñëåííîå èññëåäîâàíèå ïåðåõîäà ê òóðáóëåíòíîñòè â çàäà÷å î äâóìåðíîì òå÷åíèè âÿçêîé ñæèìàåìîé ïðîâîäÿùåé
æèäêîñòè â êàíàëå ñ ñèììåòðè÷íûì ðàñøèðåíèåì. // Òðóäû ÈÑÀ ÐÀÍ. 2012. Ò.62.
1. Ñ. 55 62.
17. Åâñòèãíååâ Í.Ì., Ìàãíèöêèé Í.À. Îñîáåííîñòè ôàçîâîãî ïðîñòðàíñòâà óðàâíåíèé äèíàìèêè ãàçà äëÿ òðàíñçâóêîâîé íà÷àëüíî-êðàåâîé çàäà÷è // Òðóäû ÈÑÀ ÐÀÍ. 2012.
Ò.62, 4. Ñ. 85 102.
18. Åâñòèãíååâ Í.Ì., Ìàãíèöêèé Í.À. Íåëèíåéíàÿ äèíàìèêà íà÷àëüíîé ñòàäèè ëàìèíàðíî-òóðáóëåíòíîãî ïåðåõîäà â çàäà÷å ðàçâèòèÿ íåóñòîé÷èâîñòè ÊåëüâèíàÃåëüìãîëüöà.// Òðóäû ÈÑÀ ÐÀÍ. 2013. Ò.63 3. ñ. 45 52.
19. Åâñòèãíååâ Í.Ì., Ìàãíèöêèé Í.À. Î ðàçâèòèè íåóñòîé÷èâîñòè
ÊåëüâèíàÃåëüìãîëüöà íà íà÷àëüíîé ñòàäèè ëàìèíàðíî-òóðáóëåíòíîãî ïåðåõîäà â âÿçêîì ãàçå.//Òðóäû ÈÑÀ ÐÀÍ. 2014. Ò.64. 3. Ñ. 41 52.
20. Åâñòèãíååâ Í.Ì., Ìàãíèöêèé Í.À., Ñèëàåâ Ä.À. Êà÷åñòâåííûé àíàëèç äèíàìèêè â çàäà÷å À.Í. Êîëìîãîðîâà òå÷åíèÿ âÿçêîé íåñæèìàåìîé
æèäêîñòè.//Äèôôåðåíöèàëüíûå Óðàâíåíèÿ. 2015. Ò.51. 10. Ñ. 1302 1314.
21. Åâñòèãíååâ Í.Ì., Ìàãíèöêèé Í.À. ×èñëåííîå èññëåäîâàíèå íåëèíåéíîé äèíàìèêè â îáîáùåííîé çàäà÷å òå÷åíèÿ À.Í.Êîëìîãîðîâà. // Ñèñòåìíûé àíàëèç è
èíôîðìàöèîííûå òåõíîëîãèè (ÑÀÈÒ'2015) Òðóäû øåñòîé ìåæäóíàðîäíîé
êîíôåðåíöèè (Ñâåòëîãîðñê, 10-20 èþíÿ, 2015ã.)Ñâåòëîãîðñê: Èçäàòåëüñêèé öåíòð ÁÔÓ èì.È.Êàíòà, 2015. Ñ. 49 55.
22. Åâñòèãíååâ Í.Ì. Îá îäíîì ñïîñîáå îñðåäåíèÿ óðàâíåíèé äèíàìèêè ñæèìàåìîé è íåñæèìàåìîé æèäêîñòè.// Âåñòíèê ÌÃÎÓ, ñåðèÿ ÔÌ. 2010. 2. Ñ. 47 52.
Application of multiGPU+CPU architecture for DNS of laminar - turbulent transition problems which are
considered as nonlinear dynamical systems
∗Evstigneev N.M., Ryabkov O.I.
Federal Research Center Computer Science and Control of Russian Academy of Science, Institute for System Analysis.
In the paper we summarize the use of parallel computational architectures for the direct numerical simulation of laminar - turbulent transition problems (LTTPs).
Usually, DNS results are analyzed by a set of statistical parameters, namely, pulsation velocities correlations, energy spectra etc. When a dynamical system analysis approach is applied to DNS results, it is necessary to evaluate additional parameters: phase space attractors and Poincare sections, eigenvalues of Monodromy matrix etc. This allows one to present results as bifurcation scenarios and diagrams and bring up more details concerning LTTP scenario as functions of bifurcation parameters (e.g.
Reynolds, Mach, Froude numbers). This task is computationally expensive and algorithms are complex. This brings up more demands on hardware eciency and software algorithm optimization. We've being using GPU and multiGPU together with CPU architectures since 2008 in this kind of DNS. We've considered eight separate LTTPs since then. Dierent high order methods were applied. In the paper we show common computational problems for each problem class. We illustrate the application of libraries and stand alone algorithms, perform eciency benchmarking across GPUs and with CPU versions. It is shown that in general one GPU is 5 to 35 times faster than CPU. The acceleration is worse than linear and scales up as a power function with exponent in 0.78-0.81. This is, presumably, due to MPI communication bandwidth and complexity of algorithms in multiCPU and multiGPU approach. We use ve GPUs for multiGPU and show CPU+multiGPU eciency for one of the considered problems.
Keywords: multiGPU, hybrid GPU and CPU architecture, Direct Numerical Simulation, Laminar - Turbulent Transition, Dynamical Systems, High Order Numerical methods.
References
1. Toro E.F. Riemann Solvers and Numerical Methods for Fluid Dynamics. Springer-Verlag, 1999.
2. Sarrisy I. E., Kassinosz S., Knaepeny B. , Carati D. Large-eddy simulations of the turbulent Hartmann ow close to the transitional regime.// Center for Turbulence Research: Proceedings of the Summer Program 2006. P. 387 397.
3. Frik P.G. Turbulentnost': podkhody i modeli. [Turbulence: approach and models]. - Moskva-Izhevsk: Institut komp'yuternykh issledovaniy [Moscow-Izhevsk: Institute for Computer Research]. 2003. 292 P.
4. Silaev D.A., Korotaev D.O. Reshenie kraevykh zadach s pomoshch'yu S-splayna [Numerical solution of boundary problems with S-splines].// Komp'yuternye issledovaniya i
modelirovanie [Computer Research and Modeling]. 2009. Vol.1. No.2. P. 161 171 5. CUFFT 1.1 / 2.0 vs FFTW 3.1.2 (x86_64) vs FFTW 3.2 (Cell) comparison. URL:
http://www.sharcnet.ca/~merz/CUDA_benchFFT/
∗Paper is supported by Russian Foundation for Basic Research (RFBR) grant 14-07-00123.
6. Evstigneev N.M. and Magnitskii N.A. FSM Scenarios of Laminar-Turbulent Transition in Incompressible Fluids. - Chapter 10 in Nonlinearity, Bifurcation and Chaos Theory and Applications. INTECH. 2013. P. 250 280.
7. Evstigneev N.M., Magnitskii N.A. and Sidorov S.V. Novyy podkhod k ob"yasneniyu prirody turbulentnosti vyazkoy neszhimaemoy zhidkosti [New approach for turbulence in a problem for incompressible viscous ow]. // Trudy ISA RAN [Proc. ISA RAS]. 2008.
Vol.33. P. 49 65.
8. Evstigneev N.M., Magnitskii N.A. and Sidorov S.V. On the nature of turbulence in a problem on the motion of a uid behind a ledge.// Dierential Equations. January 2009.
Vol.45. Iss.1. P. 68 72.
9. Evstigneev N.M., Magnitskii N.A. and Sidorov S.V. On the nature of turbulence in
Rayley-Benard convection.// Dierential Equations. June 2009. Vol.45. Iss.6. P. 909 912.
10. Evstigneev N.M. Numerical integration of Poisson's equation using a graphics processing unit with CUDA-technology. //Vychisl. Metody Programm. 2009. Vol.10. P. 268 274.
11. Evstigneev N.M. and Magnitskii N.A. On possible scenarios of the transition to turbulence in Rayleigh-Benard convection.// Doklady Mathematics. August 2010. Vol.82. Iss.1. P. 659 662.
12. Evstigneev N.M. and Magnitskii N.A. Nonlinear dynamics in the initial-boundary value problem on the uid ow from a ledge for the hydrodynamic approximation to the boltzmann equations. // Dierential Equations. December 2010. Vol.46. Iss.12. P. 1794 1798.
13. Evstigneev N.M., Magnitskii N.A. and Sidorov S.V. Nonlinear dynamics of laminar-turbulent transition in three dimensional RayleighBenard convection.
//Communications in Nonlinear Science and Numerical Simulation. October 2010. Vol.15.
Iss.10. P. 2851 2859.
14. Evstigneev N.M. Primenenie gracheskogo protsessora dlya uskoreniya chislennogo setochnogo metoda Bol'tsmana s entropiynoy stabilizatsiey [Application of GPU for
acceleration of Lattice - Boltzmann method with entropy x].// Trudy ISA RAN [Proc ISA RAS]. 2010. Vol.53, No.14. P. 111 123.
15. Evstigneev N.M. and Magnitskii N.A. Nonlinear Dynamics of Laminar-Turbulent Transition in Back Facing Step Problem for Bolzmann Equations in Hydrodynamic Limit.//AIP. 2010. Vol.1281. P. 896 900.
16. Evstigneev N.M., Magnitskii N.A. and Ryabkov O.I. Chislennoe issledovanie perekhoda k turbulentnosti v zadache o dvumernom techenii vyazkoy szhimaemoy provodyashchey zhidkosti v kanale s simmetrichnym rasshireniem [Simulation of laminar - turbulent transfer in 2D Hartmann ow problem with symmetrically expansing channel]. // Trudy ISA RAN [Proc ISA RAS]. 2012. Vol.62, No.1. P. 55 62.
17. Evstigneev N.M. and Magnitskii N.A. Osobennosti fazovogo prostranstva uravneniy dinamiki gaza dlya transzvukovoy nachal'no-kraevoy zadachi [On phase space peculiarities of gas dynamics equations for a supersonic initial-boundary value problem]. // Trudy ISA RAN [Proc.ISA RAS]. 2012. Vol.62. No.4, P. 85 102.
18. Evstigneev N.M. and Magnitskii N.A. Nelineynaya dinamika nachal'noy stadii laminarno-turbulentnogo perekhoda v zadache razvitiya neustoychivosti
Kel'vinaGel'mgol'tsa [Nonlinear dynamics of initial stage in Kelvin-Helmholtz
laminar-turbulent instability problem]. // Trudy ISA RAN [Proc ISA RAS]. 2013. Vol.63.
No.3. P. 45 52.
19. Evstigneev N.M. and Magnitskii N.A. O razvitii neustoychivosti Kel'vinaGel'mgol'tsa na nachal'noy stadii laminarno-turbulentnogo perekhoda v vyazkom gaze [On the initial stage of KelvinHelmholtz instability for laminar-turbulent transition in viscous gas
ow].//Trudy ISA RAN [Proc. ISA RAS]. 2014. Vol.64. No.3. P. 41 52.
20. Evstigneev N.M., Magnitskii N.A. and Silaev D.A. Qualitative Analysis of Dynamics in Kolmogorov's Problem on a Flow of a Viscous Incompressible Fluid. //Dierential Equations. October 2015. Vol.51. Iss.10. P. 1292 1305.
21. Evstigneev N.M. and Magnitskii N.A. Chislennoe issledovanie nelineynoy dinamiki v obobshchennoy zadache techeniya A.N.Kolmogorova [Numerical simulation of nonlinear dynamics in generalized Kolmogorov problem for 3D viscous incompressible uid]. //
Sistemnyy analiz i informatsionnye tekhnologii (SAIT'2015) Trudy shestoy
mezhdunarodnoy konferentsii (Svetlogorsk, 10-20 iyunya, 2015g.) Svetlogorsk: Izdatel'skiy tsentr BFU im.I.Kanta, 2015. S. 49 55. [System Analysis and Informational Technologies.
(SAIT'2015) Proceedings of the International Scientic Conference], Svetlogorsk, Published by Immanuel Kant Baltic Federal University. 2015. P. 49 55.
22. Evstigneev N.M. Ob odnom sposobe osredeniya uravneniy dinamiki szhimaemoy i
neszhimaemoy zhidkosti [About one averaging method of the equations of a compressible and incompressible uid]. // Vestnik MGOU, seriya FM [Bulletin MSRU, PhM series].
2010. No.2, P. 47 52.
