Ðåàëèçàöèÿ èòåðàöèîííûõ ìåòîäîâ ðåøåíèÿ ÑËÀÓ äëÿ ãèáðèäíûõ âû÷èñëèòåëüíûõ ñèñòåì è èõ èñïîëüçîâàíèå ïðè ìîäåëèðîâàíèè äèññèïàòèâíûõ
ïðîöåññîâ â çàäà÷àõ ôèçèêè ïëàçìû
∗Ï.À. Êó÷óãîâ1,2, Ã.À. Áàãäàñàðîâ1,3, À.Ñ. Áîëäàðåâ1,3, Î.Ã. Îëüõîâñêàÿ1, Þ.À. Ïîâåùåíêî1,3
Èíñòèòóò ïðèêëàäíîé ìàòåìàòèêè èì. Ì.Â. Êåëäûøà ÐÀÍ1, Ôèçè÷åñêèé èíñòèòóò èì. Ï.Í. Ëåáåäåâà ÐÀÍ2, Íàöèîíàëüíûé èññëåäîâàòåëüñêèé ÿäåðíûé óíèâåðñèòåò
ÌÈÔÈ3
Ìíîãèå çàäà÷è ôèçèêè, â òîì ÷èñëå ôèçèêè ïëàçìû, òðåáóþò ðåøåíèÿ ñèñòåì ëè- íåéíûõ óðàâíåíèé ñ áîëüøèìè ðàçðåæåííûìè ìàòðèöàìè. Âàæíûì â ýòîì âîïðî- ñå ÿâëÿåòñÿ âîçìîæíîñòü èñïîëüçîâàíèÿ âñåõ âû÷èñëèòåëüíûõ ðåñóðñîâ, ïðåäî- ñòàâëÿåìûõ ñîâðåìåííûìè ãèáðèäíûìè êëàñòåðàìè, äëÿ ìèíèìèçàöèè âðåìåí- íûõ çàòðàò ïðè ìîäåëèðîâàíèè ðàçëè÷íûõ äèññèïàòèâíûõ ïðîöåññîâ.  äàííîé ðàáîòå ïðåäëàãàåòñÿ âàðèàíò ðåàëèçàöèè íåêîòîðûõ èòåðàöèîííûõ ìåòîäîâ ïîä- ïðîñòðàíñòâ Êðûëîâà ñ èñïîëüçîâàíèåì òåõíîëîãèé CUDA è MPI. Ïðåäñòàâëåíû ðåçóëüòàòû çàìåðîâ ïðîèçâîäèòåëüíîñòè ðåøåíèÿ òåñòîâûõ çàäà÷, îòíîñÿùèõñÿ ê ðàçëè÷íûì ôèçè÷åñêèì ïðîöåññàì, ïðè èñïîëüçîâàíèè ñòðóêòóðèðîâàííûõ è íåñòðóêòóðèðîâàííûõ ñåòîê. Íà îñíîâå ýòèõ ýìïèðè÷åñêèõ äàííûõ ñäåëàíà îöåí- êà ÷èñëà ñòðîê ìàòðèöû, îïòèìàëüíîãî äëÿ îáðàáîòêè êîíôèãóðàöèåé, ñîñòîÿ- ùåé èç îäíîãî CPU è îäíîãî GPU.
Êëþ÷åâûå ñëîâà: Èòåðàöèîííûå ìåòîäû, ÑËÀÓ, ìàòåìàòè÷åñêîå ìîäåëèðîâàíèå, ôèçèêà ïëàçìû, CUDA, MPI, ãèáðèäíûå âû÷èñëèòåëüíûå ñèñòåìû.
1. Ââåäåíèå
Ïîòðåáíîñòü â ïðîâåäåíèè âñå áîëåå ïîäðîáíûõ ÷èñëåííûõ ðàñ÷åòîâ ïîäòàëêèâàåò èñ- ñëåäîâàòåëåé è ðàçðàáîò÷èêîâ ÷èñëåííûõ êîäîâ ê èñïîëüçîâàíèþ äîïîëíèòåëüíûõ ðåñóðñîâ, ïðåäîñòàâëÿåìûõ ñîâðåìåííûìè âû÷èñëèòåëüíûìè êëàñòåðíûìè ñèñòåìàìè, ÷òîáû ìèíè- ìèçèðîâàòü îáùèå âðåìåííûå çàòðàòû íà ïðîâåäåíèå ìîäåëèðîâàíèÿ ïîñòàâëåííûõ çàäà÷.
 ïåðâóþ î÷åðåäü ðå÷ü èäåò î ðàçíîãî ðîäà ñîïðîöåññîðàõ ê CPU, â ÷àñòíîñòè î GPU ïðè- ìåíèòåëüíî ê äàííîé ñòàòüå. GPU-óñòðîéñòâà, èçíà÷àëüíî îðèåíòèðîâàííûå íà ðåøåíèå ñïåöèôè÷åñêèõ çàäà÷, ñâÿçàííûõ ñ îáðàáîòêîé è îòîáðàæåíèåì ãðàôè÷åñêîé èíôîðìàöèè, íà ñåãîäíÿøíèé äåíü ìîãóò óñïåøíî âûïîëíÿòü îáùèå ìàññèâíî-ïàðàëëåëüíûå âû÷èñëåíèÿ (GPGPU). Íàèáîëüøåå ðàñïðîñòðàíåíèå äëÿ ýòèõ öåëåé ïîëó÷èëà òåõíîëîãèÿ CUDA [1], ïîçâîëÿþùàÿ ïîëó÷èòü äîñòóï ê GPU-óñòðîéñòâó ñ ïîìîùüþ íàáîðà ôóíêöèé, ïîçèöèîíè- ðóåìàÿ êàê ðàñøèðåíèå ÿçûêà Ñè.
Õîðîøî èçâåñòíî, ÷òî ïðè ìîäåëèðîâàíèè øèðîêîãî ñïåêòðà ðàçëè÷íûõ ôèçè÷åñêèõ è íå òîëüêî ïðîöåññîâ è ÿâëåíèé âîçíèêàåò íåîáõîäèìîñòü ðåøåíèÿ ñèñòåìû ëèíåéíûõ àëãåá- ðàè÷åñêèõ óðàâíåíèé (ÑËÀÓ) ñ áîëüøèìè ðàçðåæåííûìè ìàòðèöàìè, ÷òî çàíèìàåò áîëü- øóþ ÷àñòü âû÷èñëèòåëüíîãî âðåìåíè. Ýòîò ôàêò âûäåëÿåò ïîèñê ýôôåêòèâíûõ ñ òî÷êè çðåíèÿ ïðîèçâîäèòåëüíîñòè ìåòîäîâ ðåøåíèÿ ÑËÀÓ â îòäåëüíóþ çàäà÷ó, êîòîðîé çàíèìà- åòñÿ çíà÷èòåëüíîå ÷èñëî èññëåäîâàòåëåé ïî âñåìó ìèðó. Ïîäõîäû ê ðåøåíèþ ìîãóò áûòü ðàçëè÷íû. Îíè âêëþ÷àþò â ñåáÿ êàê ðàçðàáîòêó íîâûõ àëãîðèòìîâ ðåøåíèÿ ÑËÀÓ, îáëà- äàþùèõ áîëüøåé ñòåïåíüþ ïàðàëëåëèçìà, òàê è ïîèñê ýôôåêòèâíîé ðåàëèçàöèè â ðàìêàõ îäíîãî è òîãî æå àëãîðèòìà.  äàííîé ðàáîòå àâòîðû áóäóò ïðèäåðæèâàòüñÿ âòîðîãî ïîä-
∗Ðàáîòà âûïîëíåíà ïðè ÷àñòè÷íîé ôèíàíñîâîé ïîääåðæêå ÐÔÔÈ, ãðàíòû 16-31-60101-ìîë_à_äê è
16-31-00350-ìîë_à.
õîäà.  êà÷åñòâå èññëåäóåìîãî ìåòîäà áûë âûáðàí ÷àñòî èñïîëüçóåìûé íà òåêóùèé ìîìåíò ñòàáèëèçèðîâàííûé ìåòîä áèñîïðÿæåííûõ ãðàäèåíòîâ (BiCGStab) ñ ïðåäîáóñëàâëèâàíè- åì [2, 3], ÿâëÿþùèéñÿ ïðåäñòàâèòåëåì èòåðàöèîííûõ ðåøàòåëåé ïîäïðîñòðàíñòâ Êðûëîâà.
Íåñêîëüêî ñëîâ ñëåäóåò ñêàçàòü î âûáîðå ïðåäîáóñëàâëèâàòåëÿ. Ïðåäîáóñëàâëèâàíèå èñ- õîäíîé ìàòðèöû óëó÷øàåò åå ñïåêòðàëüíûå ñâîéñòâà è âëèÿåò íà ñêîðîñòü ñõîäèìîñòè èòå- ðàöèîííîãî ìåòîäà, îäíàêî ïðè ïàðàëëåëüíîé ðåàëèçàöèè ðåøàòåëåé ÑËÀÓ òàêæå ñëåäóåò îáðàùàòü âíèìàíèå íà ñòåïåíü ïàðàëëåëèçìà îïåðàöèè ïðåäîáóñëàâëèâàíèÿ è, âîçìîæíî, ïîñòóïèòüñÿ ñêîðîñòüþ ñõîäèìîñòè â ðàçóìíûõ ïðåäåëàõ â îáìåí íà îùóòèìîå ñíèæåíèå âðåìåíè, ïðèõîäÿùåãîñÿ íà îäíó èòåðàöèþ. Èç ýòèõ ñîîáðàæåíèé íàìè áûë âûáðàí ïîëè- íîìèàëüíûé ïðåäîáóñëàâëèâàòåëü â ñìûñëå íàèìåíüøèõ êâàäðàòîâ. Òàê, äëÿ ðàññìàòðè- âàåìûõ â äàííîé ðàáîòå çàäà÷ ïåðåõîä îò ðàíåå èñïîëüçîâàííîãî ïðåäîáóñëàâëèâàíèÿ ïî ìåòîäó ÃàóññàÇåéäåëÿ ïðèâåë ê óâåëè÷åíèþ ÷èñëà èòåðàöèé â 1.5-2 ðàçà, ÷òî â àáñîëþòíûõ çíà÷åíèÿõ ñîñòàâëÿåò 4-5 èòåðàöèé âìåñòî 2-3. Ïðåäñòàâëåííûé â ïîñëåäóþùèõ ðàçäåëàõ àíàëèç âðåìåíè âûïîëíåíèÿ îäíîé èòåðàöèè ïîçâîëÿåò ãîâîðèòü, ÷òî äàííûé ôàêò íå ïðè- âîäèò ê ñóùåñòâåííûì ïîòåðÿì â ïðîèçâîäèòåëüíîñòè.
Áåçóñëîâíî, íà ñåãîäíÿøíèé äåíü ñóùåñòâóåò äîñòàòî÷íî øèðîêèé ñïåêòð áèáëèîòåê êàê ñ îòêðûòûì êîäîì, òàê è ïðîïðèåòàðíûõ, ðåàëèçóþùèõ ðàçíîîáðàçíûå èòåðàöèîí- íûå ìåòîäû ïîäïðîñòðàíñòâ Êðûëîâà ñ ïðåäîáóñëàâëèâàíèåì è áåç, òàêèå êàê CUSP [4], CULA [5], PARALUTION [6], PETSc [7], CUB [8], gpu_sparse [9], ViennaCL [10] è äðóãèå.
Òðàäèöèîííî ñ êîììåð÷åñêèìè ðåøàòåëÿìè ñëîæíî ðàáîòàòü èç-çà íåâîçìîæíîñòè âíåñå- íèÿ äàæå ìèíèìàëüíûõ èçìåíåíèé, êîòîðûå ðàíî èëè ïîçäíî ïîòðåáóþòñÿ, à îòêðûòûå áèáëèîòåêè â ñâîåì áîëüøèíñòâå íå ðåàëèçóþò ðàáîòó ñ ðàñïðåäåëåííûìè ìàòðèöàìè ñ îä- íîâðåìåííûì èñïîëüçîâàíèåì GPU-ðåñóðñîâ äëÿ ëîêàëüíûõ, ò.å. îòíîñÿùèõñÿ ê êîíêðåò- íîìó ïðîöåññó, âû÷èñëåíèé. Òàêæå òðåáóåòñÿ àäàïòàöèÿ òèïîâ äàííûõ, èñïîëüçóåìûõ â îñíîâíîé ïðîãðàììå, ê òèïàì äàííûõ, ñ êîòîðûìè îïåðèðóåò áèáëèîòåêà, ÷òî çàíèìàåò äîïîëíèòåëüíîå âû÷èñëèòåëüíîå âðåìÿ. Òàêèì îáðàçîì, àâòîðû ïðèøëè ê âûâîäó î íåîá- õîäèìîñòè ñîçäàíèÿ ñîáñòâåííîé ðåàëèçàöèè ðåøàòåëÿ ÑËÀÓ â ðàìêàõ ïàêåòà ïðèêëàäíûõ ïðîãðàìì MARPLE3D [11], ïðåäíàçíà÷åííîãî äëÿ ìîäåëèðîâàíèÿ çàäà÷ âûñîêîòåìïåðàòóð- íîé ïëàçìû è ôèçèêè âûñîêèõ ïëîòíîñòåé ýíåðãèè.
Íàèáîëåå ðåñóðñîåìêîé è èíòåíñèâíî èñïîëüçóåìîé (îñîáåííî ñ ó÷åòîì âûáðàííîãî ïðå- äîáóñëàâëèâàòåëÿ) ïðè ðåøåíèè ÑËÀÓ ÿâëÿåòñÿ îïåðàöèÿ âû÷èñëåíèÿ ïðîèçâåäåíèÿ ðàç- ðåæåííîé ìàòðèöû íà ïëîòíûé âåêòîð (SpMV). Äëÿ GPU-óñòðîéñòâ ñèòóàöèÿ ñèëüíî óñó- ãóáëÿåòñÿ íåðåãóëÿðíûì äîñòóïîì ê ïàìÿòè, îñóùåñòâëÿåìûì â ñîîòâåòñòâèè ñ ïîðòðåòîì ðàçðåæåííîé ìàòðèöû, êîòîðûé â ñâîþ î÷åðåäü çàâèñèò îò ñïîñîáà àïïðîêñèìàöèè èñõîä- íûõ äèôôåðåíöèàëüíûõ óðàâíåíèé â ÷àñòíûõ ïðîèçâîäíûõ è îò ìåòîäà äèñêðåòíîãî ïðåä- ñòàâëåíèÿ îáëàñòè ìîäåëèðîâàíèÿ, ò.å. ðàñ÷åòíîé ñåòêè. Íà òåêóùèé ìîìåíò ïðåäëîæåíî íåñêîëüêî äîñòàòî÷íî ýôôåêòèâíûõ ñïîñîáîâ âû÷èñëåíèÿ SpMV [1216], ïîçâîëÿþùèõ áî- ëåå èëè ìåíåå íèâåëèðîâàòü ýôôåêò íåðåãóëÿðíîãî è íåêîãåðåíòíîãî äîñòóïà ê ïàìÿòè. Íà íàø âçãëÿä íà ñåãîäíÿøíèé äåíü êëþ÷åâîé ñòðàòåãèåé ðåàëèçàöèè SpMV ÿâëÿåòñÿ ïîäáîð çíà÷åíèé ðàçëè÷íûõ ïàðàìåòðîâ äëÿ òîãî èëè èíîãî âèäà ðàçðåæåííûõ ìàòðèö è êëàññîâ GPU-óñòðîéñòâ. Òàêæå ìîæíî ýêñïåðèìåíòèðîâàòü ñ ôîðìàòàìè õðàíåíèÿ ìàòðèö äëÿ îð- ãàíèçàöèè êîãåðåíòíîãî äîñòóïà ê ïàìÿòè èëè ñ ïåðåóïîðÿäî÷èâàíèåì ýëåìåíòîâ ìàòðèöû äëÿ ãðóïïèðîâêè íåíóëåâûõ ýëåìåíòîâ è óëó÷øåíèÿ ëîêàëèçîâàííîñòè äàííûõ, îäíàêî, è òîò, è äðóãîé ñïîñîá ïðèâîäÿò ê äîïîëíèòåëüíûì âðåìåííûì çàòðàòàì, ñâÿçàííûì ñ îïè- ñàííûìè îïåðàöèÿìè ïîäãîòîâêè ìàòðèöû.
2. Ïðîãðàììíàÿ ðåàëèçàöèÿ
Îáðàòèìñÿ ê îïèñàíèþ âûïîëíåíîé ðåàëèçàöèè ìåòîäà BiCGStab ñ ïîëèíîìèàëüíûì ïðåäîáóñëàâëèâàòåëåì â ðàìêàõ ïðîãðàììíîãî êîìïëåêñà MARPLE3D. Äàííûé ÷èñëåííûé êîä åñòåñòâåííûì îáðàçîì ïîääåðæèâàåò ïðîâåäåíèå ïàðàëëåëüíûõ âû÷èñëåíèé íà îñíîâå
ìåòîäà ãåîìåòðè÷åñêîãî ðàñïàðàëëåëèâàíèÿ. Äëÿ ðàçáèåíèÿ îáëàñòè íà äîìåíû èñïîëüçó- åòñÿ ïàêåò ParMetis [17]. Äëÿ ðåøåíèÿ ÑËÀÓ áåç ïðèâëå÷åíèÿ GPU-óñòðîéñòâ èñïîëüçóåòñÿ áèáëèîòåêà Aztec [18], ïîýòîìó ëîãè÷íûì ïðåäñòàâëÿåòñÿ èñïîëüçîâàíèå òîé æå ñòðóêòóðû äàííûõ äëÿ ðàáîòû ñ ðàñïðåäåëåííûìè ìàòðèöàìè, ÷òî è ðàíüøå. Òàêèì îáðàçîì, äëÿ õðà- íåíèÿ ðàçðåæåííîé ìàòðèöû èñïîëüçóåòñÿ ôîðìàò DMSR [18], â ëîêàëüíîì ïðåäñòàâëåíèè ñõîæèé ñ ôîðìàòîì CSR.  íàèáîëåå îáùåì ñëó÷àå ïðè êàæäîì âû÷èñëåíèè ïðîèçâåäåíèÿ ãëîáàëüíîé ìàòðèöû íà âåêòîð íåîáõîäèìî îáíîâëÿòü âíåøíèå ýëåìåíòû âåêòîðà, îòíî- ñÿùèåñÿ ê äðóãèì MPI-ïðîöåññàì. Îäíàêî, âûáðàííûé àëãîðèòì ïîçâîëÿåò ñîêðàòèòü äî ìèíèìóìà îáìåííûå îïåðàöèè è âûïîëíÿòü èõ îäèí ðàç çà èòåðàöèþ ïîñëå íàõîæäåíèÿ ñëåäóþùåãî ïðèáëèæåíèÿ ðåøåíèÿ.
Äëÿ ëîêàëüíîãî âû÷èñëåíèÿ SpMV íàìè áûëà èñïîëüçîâàíà ìîäèôèêàöèÿ àëãîðèò- ìà, ïðåäëîæåííîãî â [12]. Ðàçìåð áëîêà òðåäîâ è ðàçìåð ãðóïïû òðåäîâ, îáðàáàòûâàþùèõ îäíó ñòðîêó ìàòðèöû, áûëè ñäåëàíû ïåðåìåííûìè, à òàêæå ââåäåí äîïîëíèòåëüíûé ïàðà- ìåòð, îïðåäåëÿþùèé ÷èñëî ñòðîê ìàòðèöû, îáðàáàòûâàåìûõ êîíêðåòíîé ãðóïïîé òðåäîâ.
Íåñêîëüêî øàãîâ ïî âàðüèðîâàíèþ ýòèõ ïàðàìåòðîâ ïîçâîëÿåò ïîäîáðàòü îïòèìàëüíûå çíà-
÷åíèÿ äëÿ äàííîãî GPU-óñòðîéñòâà è êëàññà ìàòðèö.
Äëÿ âûïîëíåíèÿ ïðàêòè÷åñêè âñåõ îïåðàöèé ñ âåêòîðàìè, òàêèõ êàê ñêàëÿðíîå ïðî- èçâåäåíèå âåêòîðîâ, ïðîèçâåäåíèå âåêòîðà íà ñêàëÿð, ñóììà âåêòîðîâ è íåêîòîðûõ äðóãèõ áûëà èñïîëüçîâàíà áèáëèîòåêà cuBLAS [19], ÿâëÿþùàÿñÿ ÷àñòüþ CUDA Toolkit.
Äîñòàòî÷íî ðàñïðîñòðàíåííîé ñòðàòåãèåé ïðè íàïèñàíèè CUDA-ïðîãðàìì ÿâëÿåòñÿ ñòðàòåãèÿ ïî îáúåäèíåíèþ íåñêîëüêèõ âû÷èñëèòåëüíûõ ÿäåð â îäíî, ãäå ýòî âîçìîæíî,
÷òîáû ñîêðàòèòü èçäåðæêè íà çàïóñê ÿäåð. Íàìè áûëî çàìå÷åíî, ÷òî àëãîðèòì ÷àñòî èñ- ïîëüçóåò îïåðàöèþ αx+βy+γz, ïîýòîìó áûëî ðåàëèçîâàíî ñîîòâåòñòâóþùåå ÿäðî âìåñòî äâóêðàòíîãî âûçîâà ôóíêöèè y=αx+y èç áèáëèîòåêè cuBLAS.
Ïðè ðåàëèçàöèè ìåòîäà BiCGStab â äàííîé ðàáîòå òàêæå áûëà ïðèìåíåíà îïòèìèçàöèÿ, îñíîâûâàþùàÿñÿ íà èñïîëüçîâàíèè êëþ÷åâîãî ñëîâà __restrict__ äëÿ íåïåðåêðûâàþùèõ- ñÿ ìàññèâîâ, ÷òî ïîçâîëÿåò êîìïèëÿòîðó çíà÷èòåëüíî îïòèìèçèðîâàòü êîä, íå ïðîèçâîäÿ ëèøíèõ ïðîâåðîê ïðè âûïîëíåíèè ïðîãðàììû.
3. Ðåçóëüòàòû ÷èñëåííûõ ðàñ÷åòîâ
 äàííîì ðàçäåëå ïðåäñòàâëåíû ðåçóëüòàòû òåñòèðîâàíèÿ1 âûïîëíåííîé ïðîãðàììíîé ðåàëèçàöèè ìåòîäà BiCGStab, èñïîëüçóþùåé ðåñóðñû GPU-óñêîðèòåëåé, íà ïðèìåðå äâóõ çàäà÷ ðàçíîé ôèçè÷åñêîé ïðèðîäû è ðåøàåìûõ ðàçëè÷íûìè ÷èñëåííûìè ìåòîäàìè. Ýòî ïîçâîëÿåò ðàññìîòðåòü ðàçðåæåííûå ìàòðèöû, îáëàäàþùèå ðàçëè÷íîé ñòðóêòóðîé, íå àá- ñòðàãèðóÿñü îò ðåàëüíûõ ïðèëîæåíèé.
3.1. Ïîñòàíîâêè òåñòîâûõ çàäà÷
3.1.1. ÝÌÃ-âîëíà
 êà÷åñòâå ïåðâîé òåñòîâîé çàäà÷è (áóäåì îáîçíà÷àòü T1) ðàññìîòðèì çàäà÷ó î ðàñ- ïðîñòðàíåíèè ÝÌÃ-âîëíû [21]. Ñóùíîñòü ýòîé çàäà÷è çàêëþ÷àåòñÿ â óñêîðåííîé äèôôó- çèè ìàãíèòíîãî ïîëÿ â íåîäíîðîäíûõ ïî ïëîòíîñòè îáëàñòÿõ çà ñ÷åò òîêîâûõ ýëåêòðîíîâ.
Àíàëîãè÷íûé ýôôåêò âîçíèêàåò è â ñëó÷àå êðèâîëèíåéíûõ ëèíèé ìàãíèòíîãî ïîëÿ â îä- íîðîäíûõ îáëàñòÿõ. Èñïîëüçóÿ êîíôèãóðàöèþ, ïðåäñòàâëåííóþ íà ðèñ. 1, ìîæíî ïîëó÷èòü
1Âñå ïðèâåäåííûå â ñòàòüå ðåçóëüòàòû áûëè ïîëó÷åíû äëÿ äâîéíîé òî÷íîñòè ñ èñïîëüçîâàíèåì ãèáðèä- íîãî êëàñòåðà Ê-100 Èíñòèòóòà ïðèêëàäíîé ìàòåìàòèêè èì. Ì. Â. Êåëäûøà [20]. Äàííàÿ âû÷èñëèòåëüíàÿ ñèñòåìà ñîñòîèò èç 64 óçëîâ, îáúåäèíåííûõ êîììóíèêàöèîííîé ñèñòåìîé ÌÂÑ-Ýêñïðåññ, êàæäûé èç êî- òîðûõ îñíàùåí 2-ìÿ ïðîöåññîðàìè Intel Xeon X5670 (6 ÿäåð) è 3-ìÿ âèäåîêàðòàìè nVidia Fermi C2050.
Ðèñ. 1. Ïîñòàíîâêà çàäà÷è î ðàñïðîñòðàíåíèè ÝÌÃ-âîëíû.
ñëåäóþùåå äèôôåðåíöèàëüíîå óðàâíåíèå äëÿ äèôôóçèè:
∂B
∂t +κB∂B
∂z =D∂2B
∂z2,
ãäåB ≡Byÿâëÿåòñÿ êîìïîíåíòîé ìàãíèòíîãî ïîëÿ,D=c2/4πσ,κ= (cmi/4πZe)(
∂n−1/∂x) èëè cmi/4πρZe. Çäåñü c ýòî ñêîðîñòü ñâåòà, σ ïðîâîäèìîñòü, mi ìàññà èîíà, Z çàðÿä èîíà,e ýëåìåíòàðíûé çàðÿä. Äëÿ òåñòîâûõ çàïóñêîâ áûëè èñïîëüçîâàíû ñëåäóþùèå çíà÷åíèÿ ïàðàìåòðîâ: D= 1/4π,κ = 8.25·10−18/ρ, x|y|z ∈[0,3] â ïðîãðàììíûõ åäèíèöàõ MARPLE3D. Ðàñïðåäåëåíèå ïëîòíîñòè çàäàâàëîñü âûðàæåíèåì ρ = 2ρ0 äëÿ x < 1, ρ = 2ρ0/xäëÿ1< x <2èρ0 â îñòàëüíûõ ñëó÷àÿõ,ρ0= 10−18. Íà÷àëüíûå è ãðàíè÷íûå óñëîâèÿ ñëåäóþùèå:
B(zmin, t) = 1/√
4π, B(zmax, t) = 0, ∂B
∂s
∂
= 0, s=x|y.
Ïîñòðîåíèå äèñêðåòíîãî àíàëîãà èñõîäíîãî äèôôåðåíöèàëüíîãî óðàâíåíèÿ âûïîëíÿëîñü ñ èñïîëüçîâàíèåì ïðîåêöèîííîé ñõåìû [22]. Òåñòîâûå ðàñ÷åòû ïðîâîäèëèñü íà îðòîãîíàëüíîé ñåòêå, ñîñòîÿùåé èç ïðÿìîóãîëüíûõ ïàðàëëåëåïèïåäîâ.
3.1.2. Òåïëîâàÿ âîëíà
 êà÷åñòâå âòîðîé òåñòîâîé çàäà÷è (T2) ðàññìîòðèì çàäà÷ó î ðàñïðîñòðàíåíèè òåïëî- âîé âîëíû â ñðåäå ñ íåëèíåéíûì êîýôôèöèåíòîì òåïëîïðîâîäíîñòè [23]. Ñîîòâåòñòâóþùåå äèôôåðåíöèàëüíîå óðàâíåíèå èìååò âèä:
∂T
∂t = ∂
∂s (
κ0·Tα·∂T
∂s )
,
ãäå T ýòî íåèçâåñòíàÿ òåìïåðàòóðà, κ0, α ïðîèçâîëüíûå êîýôôèöèåíòû, s ≡ x|y|z. Íà÷àëüíûå è ãðàíè÷íûå óñëîâèÿ çàäàþòñÿ âûðàæåíèÿìè:
T(0, t) = [αD
κ0
(Dt+s0) ]α1
, t >0, è T(s, 0) =
[αD
κ0 (s0−s) ]α1
, s6s0,
0, s > s0, ãäå D ñêîðîñòü òåïëîâîé âîëíû,s0 ïðîèçâîëüíûé ïàðàìåòð.
Äëÿ àïïðîêñèìàöèè äàííîãî äèôôåðåíöèàëüíîãî óðàâíåíèÿ èñïîëüçîâàëñÿ ìåòîä îïîð- íûõ îïåðàòîðîâ [24, 25]. Òåñòîâûå çàïóñêè ïðîâîäèëèñü äëÿ çíà÷åíèé ïàðàìåòðîâ α = 2.0, κ0= 0.5,s0 = 0.5,D= 5.0ès∈[0,3]ñ èñïîëüçîâàíèåì ñòðóêòóðèðîâàííûõ è íåñòðóêòóðè- ðîâàííûõ ðàçíîñòíûõ ñåòîê è ïîñòîÿííîãî øàãà ïî âðåìåíèτ = 2·10−4.
3.2. Ðåçóëüòàòû
Íà ïåðâîì ýòàïå èçìåðåíèé íà ïðèìåðå çàäà÷è T1 áûë ïðîâåäåí ïîäáîð ïàðàìåòðîâ ÿäðà, âû÷èñëÿþùåãî SpMV. Áûëà èñïîëüçîâàíà ñåòêà100×100×10, ÷òî â ðåçóëüòàòå ïðè- âåëî ê ãëîáàëüíîé ìàòðèöå ñ ÷èñëîì ñòðîê N = 3·105 è ÷èñëîì íåíóëåâûõ ýëåìåíòîâ N N Z ≈2.2·107, êîòîðûå îêàçàëèñü ñãðóïïèðîâàííûìè âîêðóã íåñêîëüêèõ äèàãîíàëåé. Îò- ìåòèì, ÷òî è äëÿ çàäà÷è T2 äàæå ïðè èñïîëüçîâàíèè íåñòðóêòóðèðîâàííîé ñåòêè, ñîñòîÿùåé èç òåòðàýäðîâ, ïîðòðåò ìàòðèöû ìåíÿåòñÿ íåçíà÷èòåëüíî. Ýòî âåðíî è äëÿ äðóãèõ çàäà÷, îïèñûâàþùèõ äèññèïàòèâíûå ïðîöåññû, ÷òî ïîçâîëÿåò ðàñïðîñòðàíèòü ïîëó÷åííûå ðåçóëü- òàòû íà âåñü êëàññ çàäà÷. Èòàê, âàðüèðîâàëèñü çíà÷åíèÿ òðåõ îñíîâíûõ ïàðàìåòðîâ ðàç- ìåðà áëîêà (BLOCK_SIZE), ðàçìåðà ãðóïïû òðåäîâ, îáðàáàòûâàþùèõ îäíó ñòðîêó ìàòðèöû (GROUP_SIZE), è êîëè÷åñòâî ñòðîê, îáðàáàòûâàåìûõ êàæäîé ãðóïïîé òðåäîâ (ROWS_COUNT).
Ðèñ. 2. Çàâèñèìîñòü âðåìåíè âû÷èñëåíèÿ ïðîèçâåäåíèÿ ðàçðåæåííîé ìàòðèöû íà âåêòîð (äèàìåòð êðóãîâ) îò ðàçìåðà áëîêà è ÷èñëà òðåäîâ â ãðóïïå, ãäå GROUP_SIZE= 2n, ïðè ROWS_COUNT= 1.
Ðèñ. 3. Âðåìÿ âû÷èñëåíèÿ SpMV (ìêñ) â ðàçëè÷íûõ ðåàëèçàöèÿõ: îñíîâíîå âðåìÿ + âðåìÿ, çà- òðà÷èâàåìîå íà ïðåîáðàçîâàíèå îäíîãî ôîðìàòà õðàíåíèÿ ðàçðåæåííîé ìàòðèöû â äðóãîé. 1 CUSPARSE, 2 ñîáñòâåííàÿ ðåàëèçàöèÿ, 3 CUSP, 4 CUB.
Íà ðèñ. 2 ñõåìàòè÷åñêè ïðåäñòàâëåíà çàâèñèìîñòü âðåìåíè âû÷èñëåíèÿ SpMV îò ðàçìå- ðà áëîêà è ðàçìåðà ãðóïïû ïðè ROWS_COUNT = 1. ×åì ìåíüøå äèàìåòð êðóãà, òåì áîëåå ýôôåêòèâíî âûïîëíÿåòñÿ äàííàÿ îïåðàöèÿ íà GPU. Ïðè ýòîì âèäíî, ÷òî â øèðîêîì äèà- ïàçîíå ðàçìåðîâ áëîêà òðåäîâ ñèòóàöèÿ, êîãäà ÷èñëî òðåäîâ â ãðóïïå ðàâíî ðàçìåðó âàðïà, ò.å. GROUP_SIZE= 32, ñîîòâåòñòâóþùàÿ îñíîâîïîëàãàþùåé ðàáîòå [12], íå ÿâëÿåòñÿ íàèëó÷- øåé. Íàèìåíüøåå âðåìÿ ñîîòâåòñòâóåò GROUP_SIZE = 16. Óâåëè÷åíèå çíà÷åíèÿ ïàðàìåòðà
ROWS_COUNT ïðèâîäèò ê óìåíüøåíèþ ðàçìåðà ñåòêè áëîêîâ, ÷òî ìîæåò îòðèöàòåëüíî ñêà- çûâàòüñÿ íà çàãðóæåííîñòè GPU-óñòðîéñòâà ïðè äîñòàòî÷íî áîëüøèõ çíà÷åíèÿõ. Îäíàêî, ïðè âàðüèðîâàíèè ïàðàìåòðà îò 1 äî 32 òåñòû ïîêàçàëè, ÷òî âðåìÿ âûïîëíåíèÿ îïåðàöèè SpMV ìåíÿåòñÿ ñëàáî.
Òàêæå áûëî ïðîâåäåíî ñðàâíåíèå âðåìåíè âû÷èñëåíèÿ SpMV â íàèëó÷øåì íàøåì âàðè- àíòå ñ íåêîòîðûìè ñòîðîííèìè áèáëèîòåêàìè. Ðåçóëüòàòû ïðåäñòàâëåíû íà ðèñ. 3. Âèäíî,
÷òî ýôôåêòèâíîñòü íàøåé ðåàëèçàöèè íåçíà÷èòåëüíî óñòóïàåò CUSPARSE [26] è íåñêîëüêî ïðåâîñõîäèò CUSP è CUB, íî â ñî÷åòàíèè ñ îòñóòñòâèåì íåîáõîäèìîñòè ïðîâåäåíèÿ êàêîãî- ëèáî ïðåîáðàçîâàíèÿ ìàòðèöû èç îäíîãî ôîðìàòà â äðóãîé, ñóùåñòâåííî ëó÷øå âûãëÿäèò íà ôîíå âñåõ ðàññìîòðåííûõ ðåàëèçàöèé.
Äàëüíåéøèå èññëåäîâàíèÿ áûëè íàïðàâëåíû íà îïðåäåëåíèå óñêîðåíèÿ è ýôôåêòèâ- íîñòè ðåàëèçîâàííîãî èòåðàöèîííîãî ìåòîäà íà ïðèìåðå çàäà÷ T1 è T2. Äëÿ âû÷èñëåíèÿ óñêîðåíèÿ èñïîëüçîâàëîñü âûðàæåíèåS =tN/TN, à äëÿ ýôôåêòèâíîñòè E=T1/(N TN), ãäå tN âðåìÿ âûïîëíåíèÿ îäíîé èòåðàöèè ðåøàòåëÿ N CPU-ïðîöåññàìè, TN - òî æå, òîëüêî ñ äîïîëíèòåëüíûì èñïîëüçîâàíèåì ðåñóðñîâ îäíîãî GPU-óñòðîéñòâà äëÿ êàæäîãî CPU-ïðîöåññà. Ñîîòâåòñòâóþùèå çàâèñèìîñòè ïðåäñòàâëåíû íà ðèñ. 4 è 5.
Ðèñ. 4. Óñêîðåíèå S è ýôôåêòèâíîñòü ïðè ôèêñèðîâàííîì ðàçìåðå çàäà÷è E äëÿ çàäà÷è T1 â çàâèñèìîñòè îò ÷èñëà èñïîëüçóåìûõ GPU-óñòðîéñòâ.
(à) (á)
Ðèñ. 5. ÓñêîðåíèåS(à) è ýôôåêòèâíîñòü ïðè ôèêñèðîâàííîì ðàçìåðå çàäà÷èE(á) äëÿ çàäà÷è T2 â çàâèñèìîñòè îò ÷èñëà èñïîëüçóåìûõ GPU-óñòðîéñòâ ïðè èñïîëüçîâàíèè ðàçëè÷íûõ ñåòîê.
Âèäíî, ÷òî íàèëó÷øåå óñêîðåíèå äîñòèãàåòñÿ â ñëó÷àå, êîãäà îäíî GPU-óñòðîéñòâî îáðàáà-
òûâàåò îò 30 äî 50 òûñÿ÷ ñòðîê ãëîáàëüíîé ìàòðèöû. Ñðàâíèâàÿ ðèñ. 4 è ðèñ. 5à òàêæå ìîæ- íî îòìåòèòü, ÷òî ìû ïîëó÷àåì áîëåå âûñîêîå óñêîðåíèå â ñëó÷àå áîëüøåãî ÷èñëà íåíóëåâûõ ýëåìåíòîâ, â ñðåäíåì ïðèõîäÿùèõñÿ íà îäíó ñòðîêó ìàòðèöû. Íà íàø âçãëÿä ýòî ñâÿçàíî ñ ëó÷øåé çàãðóæåííîñòüþ GPU-óñòðîéñòâà. Íàáëþäàåìîå ñíèæåíèå ýôôåêòèâíîñòè âû- ïîëíåíèÿ îäíîé èòåðöèè ïðè óâåëè÷åíèè ÷èñëà èñïîëüçóåìûõ GPU-óñòðîéñòâ â îñíîâíîì îïðåäåëÿåòñÿ íàêëàäíûìè ðàñõîäàìè íà îáìåííûå îïåðàöèè ìåæäó MPI-ïðîöåññàìè.
3.3. Ïëàíèðóåìûå èññëåäîâàíèÿ
Íåñìîòðÿ íà òî, ÷òî ìû ïîñòàðàëèñü äî ìèíèìóìà ñîêðàòèòü êîëè÷åñòâî îáìåíîâ ìåæäó CPU è GPU, îíè âñå åùå çàíèìàþò çíà÷èòåëüíûå âðåìåííûå ðåñóðñû, îñîáåííî íà ñòàð- òå ðåøàòåëÿ íà êàæäîé íîâîé âðåìåííîé èòåðàöèè, ïðàêòè÷åñêè ñâîäÿ íà íåò äîñòèãíóòîå óñêîðåíèå. Çäåñü ìîæíî èäòè ïî ïóòè ïåðåíîñà áîëüøåãî êîëè÷åñòâà âû÷èñëåíèé íà GPU, íàïðèìåð, ïðîöåäóðû çàïîëíåíèÿ ìàòðèöû ÑËÀÓ. Ñ âûõîäîì CUDA 6.0 åùå îäíà âîçìîæ- íîñòü çàêëþ÷àåòñÿ â èñïîëüçîâàíèè òàê íàçûâàåìîé Unied Memory, êîòîðàÿ ñîçäàåò ïóë îáùåé äëÿ CPU è GPU ïàìÿòè, îáúåêòû èç êîòîðîé äîñòóïíû ïî îäíèì è òåì æå óêàçà- òåëÿì. Êëþ÷åâûì ìîìåíòîì ÿâëÿåòñÿ òî, ÷òî ñèñòåìà àâòîìàòè÷åñêè îáìåíèâàåòñÿ òàêèìè äàííûìè ìåæäó CPU è GPU.
Åùå îäíîé ïåðñïåêòèâíîé äîðàáîòêîé ÿâëÿåòñÿ ïåðåíîñ ÷àñòè âû÷èñëåíèé, îòíîñÿùèõ- ñÿ ê èòåðàöèîííîìó ðåøàòåëþ, íà CPU è îãðàíèçàöèÿ áàëàíñèðîâêè íàãðóçêè ìåæäó CPU è GPU. Ýòî ïîçâîëèò èñêëþ÷èòü ïðîñòîé CPU ïðè âûïîëíåíèè GPU âû÷èñëåíèé, à òàê- æå èñïîëüçîâàòü âñå èìåþùèåñÿ íà óçëå ÿäðà âíå çàâèñèìîñòè îò ÷èñëà äîñòóïíûõ GPU- óñòðîéñòâ.
4. Çàêëþ÷åíèå
 ðàáîòå ïðîâåäåíà îïòèìèçàöèÿ âû÷èñëåíèÿ ïðîèçâåäåíèÿ ðàçðåæåííîé ìàòðèöû íà âåêòîð íà îäíîì GPU-óñòðîéñòâå äëÿ êëàññà ìàòðèö, ïîëó÷àåìûõ ïðè ìîäåëèðîâàíèè äèñ- ñèïàòèâíûõ ïðîöåññîâ â 3D ãåîìåòðèè. Ðàññìîòðåí ïîäõîä ê ðåàëèçàöèè èòåðàöèîííûõ ìåòîäîâ ðåøåíèÿ ÑËÀÓ íà ãèáðèäíûõ âû÷èñëèòåëüíûõ ñèñòåìàõ. Ñîîòâåòñòâóþùàÿ ðå- àëèçàöèÿ ìåòîäà BiCGStab ñ ïîëèíîìèàëüíûì ïðåäîáóñëàâëèâàòåëåì ïðîäåìîíñòðèðîâàëà íà òåñòîâûõ çàäà÷àõ ñðåäíåå óñêîðåíèå äî 8 ðàç çà ñ÷åò èñïîëüçîâàíèÿ ðåñóðñîâ GPU- óñêîðèòåëåé. Ïðè ýòîì áûëî ïîêàçàíî, ÷òî ïèêîâûå çíà÷åíèÿ óñêîðåíèÿ äîñòèãàþòñÿ â ñëó-
÷àå, êîãäà íà îäíî GPU-óñòðîéñòâî ïðèõîäèòñÿ îò 30 äî 50 òûñÿ÷ ñòðîê èñõîäíîé ìàòðèöû ÑËÀÓ.
Ëèòåðàòóðà
1. NVIDIA Corp. CUDA C Programming Guide. Design Guide, PG-02829-001_v7.0. 2015.
2. Saad Y. Iterative Methods for Sparse Linear Systems. Society for Industrial and Applied Mathematics, 2003. 537 P.
3. van der Vorst H. Iterative Krylov Methods for Large Linear Systems. Cambridge University Press, 2009. 236 P.
4. Dalton S., Bell N., Olson L., Garland M. Cusp: Generic Parallel Algorithms for Sparse Matrix and Graph Computations. URL: http://cusplibrary.github.io/ (äàòà îáðàùåíèÿ:
01.02.2016).
5. Humphrey J. R., Price D. K., Spagnoli K. E., Paolini A. L., Kelmelis E. J. CULA: Hybrid GPU Accelerated Linear Algebra Routines // Modeling and Simulation for Defense Systems and Applications V, SPIE Proceedings. 2010. Vol. 7705. P. 770502.
6. PARALUTION Labs. User manual, PARALUTION Version 1.1.0. URL:
https://www.paralution.com/user-manual/ (äàòà îáðàùåíèÿ: 01.02.2016).
7. Balay S., Abhyankar S., Adams M. F., Brown J., Brune P., Buschelman K., Dalcin L., Eijkhout V., Gropp W. D., Kaushik D., Knepley M. G., McInnes L. C., Rupp K., Smith B. F., Zampini S., Zhang H. PETSc Web page. URL:
http://www.mcs.anl.gov/petsc/ (äàòà îáðàùåíèÿ: 01.02.2016).
8. CUB Web page. URL: https://nvlabs.github.io/cub/ (äàòà îáðàùåíèÿ: 01.02.2016).
9. Ãóáàéäóëëèí Ä. À., Íèêèôîðîâ À. È., Ñàäîâíèêîâ Ð. Â. Áèáëèîòåêà gpu_sparse äëÿ
÷èñëåííîãî ðåøåíèÿ çàäà÷ ìåõàíèêè ñïëîøíûõ ñðåä íà ãèáðèäíîé âû÷èñëèòåëüíîé ñèñòåìå // Âåñòíèê Íèæåãîðîäñêîãî óíèâåðñèòåòà èì. Í. È. Ëîáà÷åâñêîãî.
Èíôîðìàöèîííûå òåõíîëîãèè. 2011. 2(1). Ñ. 190196.
10. ViennaCL Web page. URL: http://viennacl.sourceforge.net/ (äàòà îáðàùåíèÿ:
01.02.2016).
11. Ãàñèëîâ B. A., Áîëäàðåâ À. Ñ., Äüÿ÷åíêî Ñ.Â., Îëüõîâñêàÿ Î. Ã., Êàðòàøåâà Å. Ë., Áîëäûðåâ Ñ. Í., Ãàñèëîâà È. Â., Áîÿðîâ Ì. Ñ., Øìûðîâ Â. À. Ïàêåò ïðèêëàäíûõ ïðîãðàìì MARPLE3D äëÿ ìîäåëèðîâàíèÿ íà âûñîêîïðîèçâîäèòåëüíûõ ÝÂÌ èìïóëüñíîé ìàãíèòîóñêîðåííîé ïëàçìû // Ìàòåìàòè÷åñêîå ìîäåëèðîâàíèå. 2012.
Ò. 24. 1. Ñ. 5587.
12. Bell N., Garland M. Ecient Sparse Matrix-Vector Multiplication on CUDA. Technical Report NVR-2008-004, NVIDIA Corporation. 2008.
13. Guo P., Wang L. Auto-Tuning CUDA Parameters for Sparse Matrix-Vector Multiplication on GPUs // In Computational and Information Sciences (ICCIS'10). 2010. P. 11541157.
14. Sørensen H. H. B., High-Performance Matrix-Vector Multiplication on the GPU //
Euro-Par 2011: Parallel Processing Workshops. 2012. Vol. 7155. P. 377386.
15. Reguly I., Giles M., Ecient sparse matrix-vector multiplication on cache-based GPUs //
Innovative Parallel Computing (InPar). 2012. P. 112.
16. Xu S., Xue W., Lin H. X. Performance modeling and optimization of sparse matrix-vector multiplication on NVIDIA CUDA platform // The Journal of Supercomputing. 2013.
No. 63(3). P. 710721.
17. Karypis G., Kumar V. Parmetis: Parallel graph partitioning and sparse matrix ordering library, version 4.0. Technical report, University of Minnesota, Department of Computer Science and Engineering. 2013.
18. Tuminaro R. S., Heroux M., Hutchinson S. A., Shadid J. N., Ocial Aztec User's Guide, SAND99-8801J, 1999.
19. NVIDIA Corp. CUBLAS Library. User guide, DU-06702-001_v7.0. 2015 20. Ãèáðèäíûé âû÷èñëèòåëüíûé êëàñòåð K-100. URL:
http://www.kiam.ru/MVS/resourses/k100.html (äàòà îáðàùåíèÿ: 01.02.2016).
21. Kingsep A. S., Mokhov Y. V., Chukbar K. V. Nonlinear skin phenomena in plasma //
Proceedings of the 2nd International Workshop on Nonlinear and Turbulent Processes in Physics (Kiev, USSR, 10-25 October 1983), 1983. Vol. 1. P. 267275.
22. Áàãäàñàðîâ Ã. À. Òðåõìåðíîå ìîäåëèðîâàíèå ìàãíèòîóñêîðåííîé èìïóëüñíîé ïëàçìû ñ ó÷åòîì ýôôåêòîâ, îáóñëîâëåííûõ îáîáùåííûì çàêîíîì Îìà. Äèññåðòàöèÿ íà
ñîèñêàíèå ñòåïåíè êàíäèäàòà ôèçèêî-ìàòåìàòè÷åñêèõ íàóê, Èíñòèòóò ïðèêëàäíîé ìàòåìàòèêè èì. Ì. Â. Êåëäûøà ÐÀÍ. 2012.
23. Ñàìàðñêèé À. À., Ñîáîëü È. Ì. Ïðèìåðû ÷èñëåííîãî ðàñ÷åòà òåìïåðàòóðíûõ âîëí //
Æóðíàë âû÷èñëèòåëüíîé ìàòåìàòèêè è ìàòåìàòè÷åñêîé ôèçèêè. 1963. Ò. 3. 4.
Ñ. 702719.
24. Ñàìàðñêèé À. À., Êîëäîáà À. Â., Ïîâåùåíêî Þ. À., Òèøêèí Â. Ô., Ôàâîðñêèé À. Ï.
Ðàçíîñòíûå ñõåìû íà íåðåãóëÿðíûõ ñåòêàõ. ÇÀÎ Êðèòåðèé, 1996. 274 Ñ.
25. Ãàñèëîâ Â. À., Ãàñèëîâà È. Â., Êëî÷êîâà Ë. Â., Ïîâåùåíêî Þ. À., Òèøêèí Â. Ô.
Ðàçíîñòíûå ñõåìû íà îñíîâå ìåòîäà îïîðíûõ îïåðàòîðîâ äëÿ çàäà÷ äèíàìèêè ôëþèäîâ â êîëëåêòîðå, ñîäåðæàùåì ãàçîãèäðàòû // Æóðíàë âû÷èñëèòåëüíîé ìàòåìàòèêè è ìàòåìàòè÷åñêîé ôèçèêè. 2015. Ò. 55. 8. Ñ. 13411355.
26. NVIDIA Corp. CUSPARSE Library, DU-06709-001_v7.0. 2015.
The implementation of iterative methods for solving linear systems on heterogeneous computing systems
and their application in modeling of dissipative processes in problems of plasma physics
P.A. Kuchugov1,2, G.A. Bagdasarov1, A.S. Boldarev1, O.G. Ol'khovskaya1, Yu.A. Poveshchenko1
Keldysh Institute of Applied Mathematics RAS1, P.N. Lebedev Physical Institute RAS2
Many problems in physics, including physics of plasmas require solving systems of linear equations with large sparse matrices. The use of all computing resources provided by modern heterogeneous clusters is important in this issue to minimize overall time spent on modeling of various dissipative processes. This paper proposes realization of some Krylov subspace iterative methods with using CUDA and MPI. The results of the performance measurements on test problems related to dierent physical processes, using structured and unstructured grids are presented. On the basis of these empirical data it has been estimated optimal number of rows to be processed by the conguration consisting of one CPU and one GPU.
Keywords: Iterative methods, linear systems, mathematical modeling, plasma physics, CUDA, MPI, heterogeneous computing systems.
References
1. NVIDIA Corp. CUDA C Programming Guide. Design Guide, PG-02829-001_v7.0. 2015.
2. Saad Y. Iterative Methods for Sparse Linear Systems. Society for Industrial and Applied Mathematics, 2003. 537 P.
3. van der Vorst H. Iterative Krylov Methods for Large Linear Systems. Cambridge University Press, 2009. 236 P.
4. Dalton S., Bell N., Olson L., Garland M. Cusp: Generic Parallel Algorithms for Sparse Matrix and Graph Computations. URL: http://cusplibrary.github.io/ (accessed:
01.02.2016).
5. Humphrey J. R., Price D. K., Spagnoli K. E., Paolini A. L., Kelmelis E. J. CULA: Hybrid GPU Accelerated Linear Algebra Routines // Modeling and Simulation for Defense Systems and Applications V, SPIE Proceedings. 2010. Vol. 7705. P. 770502.
6. PARALUTION Labs. User manual, PARALUTION Version 1.1.0. URL:
https://www.paralution.com/user-manual/ (accessed: 01.02.2016).
7. Balay S., Abhyankar S., Adams M. F., Brown J., Brune P., Buschelman K., Dalcin L., Eijkhout V., Gropp W. D., Kaushik D., Knepley M. G., McInnes L. C., Rupp K., Smith B. F., Zampini S., Zhang H. PETSc Web page. URL:
http://www.mcs.anl.gov/petsc/ (accessed: 01.02.2016).
8. CUB Web page. URL: https://nvlabs.github.io/cub/ (accessed: 01.02.2016).
9. Gubaydullin D. A., Nikiforov A. I., Sadovnikov R. V. Biblioteka gpu_sparse dlya chislennogo resheniya zadach mekhaniki sploshnykh sred na gibridnoy vychislitel'noy sisteme [gpu_sparse library for the numerical solution of continuum mechanics problems on hybrid computing system] // Vestnik Nizhegorodskogo universiteta
im. N. I. Lobachevskogo. Informatsionnye tekhnologii [Bulletin of the Lobachevsky University]. 2011. No. 2(1). P. 190196.
10. ViennaCL Web page. URL: http://viennacl.sourceforge.net/ (accessed: 01.02.2016).
11. Gasilov V. A., Boldarev A. S., D'yachenko S. V., Olkhovskaya O. G., Kartasheva E. L., Boldyrev S. N., Bagdasarov G. A., Gasilova I. V., Boyarov M. S., Shmyrov V. A. Program package MARPLE3D for simulation of pulsed magnetically driven plasma using high performance computing // Matem. Mod. 2012. Vol. 24. No. 1. P. 5587.
12. Bell N., Garland M. Ecient Sparse Matrix-Vector Multiplication on CUDA. Technical Report NVR-2008-004, NVIDIA Corporation. 2008.
13. Guo P., Wang L. Auto-Tuning CUDA Parameters for Sparse Matrix-Vector Multiplication on GPUs // In Computational and Information Sciences (ICCIS'10). 2010. P. 11541157.
14. Sørensen H. H. B., High-Performance Matrix-Vector Multiplication on the GPU //
Euro-Par 2011: Parallel Processing Workshops. 2012. Vol. 7155. P. 377386.
15. Reguly I., Giles M., Ecient sparse matrix-vector multiplication on cache-based GPUs //
Innovative Parallel Computing (InPar). 2012. P. 112.
16. Xu S., Xue W., Lin H. X. Performance modeling and optimization of sparse matrix-vector multiplication on NVIDIA CUDA platform // The Journal of Supercomputing. 2013.
No. 63(3). P. 710721.
17. Karypis G., Kumar V. Parmetis: Parallel graph partitioning and sparse matrix ordering library, version 4.0. Technical report, University of Minnesota, Department of Computer Science and Engineering. 2013.
18. Tuminaro R. S., Heroux M., Hutchinson S. A., Shadid J. N., Ocial Aztec User's Guide, SAND99-8801J, 1999.
19. NVIDIA Corp. CUBLAS Library. User guide, DU-06702-001_v7.0. 2015
20. Gibridnyy vychislitel'nyy klaster K-100 [Heterogeneous computing cluster K-100]. URL:
http://www.kiam.ru/MVS/resourses/k100.html (accessed: 01.02.2016).
21. Kingsep A. S., Mokhov Y. V., Chukbar K. V. Nonlinear skin phenomena in plasma //
Proceedings of the 2nd International Workshop on Nonlinear and Turbulent Processes in Physics (Kiev, USSR, 10-25 October 1983), 1983. Vol. 1. P. 267275.
22. Bagdasarov G. A. Tryekhmernoe modelirovanie magnitouskorennoy impul'snoy plazmy s uchyetom eektov, obuslovlennykh obobshchyennym zakonom Oma [Three-dimensional modeling of magnetically accelerated pulsed plasma, taking into account eects due to the generalized Ohm's law]. PhD thesis, Keldysh Institute of Applied Mathemtics. 2012.
23. Samarskii A. A., Sobol' I. M. Examples of numerical calculation of temperature waves //
Zh. Vychisl. Mat. Mat. Fiz. 1963. Vol. 3. No. 4. P. 702719.
24. Samarskii A. A., Koldoba A. V., Poveshenko Yu. A., Tishkin V. F., Favorskii A. P.
Raznostnye skhemy na neregulyarnykh setkakh [Dierence Schemes on Irregular Meshes].
JSC Kriteriy, 1996. 274 P.
25. Gasilov V. A., Gasilova I. V., Klochkova L. V., Poveshchenko Yu. A., Tishkin V. F.
Dierence schemes based on the support operator method for uids dynamics problems in a collector containing gas hydrates // Zh. Vychisl. Mat. Mat. Fiz. 2015. Vol. 55. No. 8.
P. 13411355.
26. NVIDIA Corp. CUSPARSE Library, DU-06709-001_v7.0. 2015.
