A retrospective trust-region method for unconstrained optimization

Texte intégral

(1)Institutional Repository - Research Portal Dépôt Institutionnel - Portail de la Recherche researchportal.unamur.be University of Namur. RESEARCH OUTPUTS / RÉSULTATS DE RECHERCHE. A retrospective trust-region method for unconstrained optimization Bastin, Fabian; Malmedy, Vincent; Toint, Philippe; Tomanos, Dimitri; Mouffe, Mélodie Published in: Mathematical Programming DOI: Author(s) - Auteur(s) : 10.1007/s10107-008-0258-1 Publication date: 2010 Document Version. Publication datealso - Date publication : Early version, knownde as pre-print Link to publication Citation for pulished version (HARVARD): Bastin, F, Malmedy, V, Toint, P, Tomanos, D & Mouffe, M 2010, 'A retrospective trust-region method for unconstrained Programming, vol. 123, no. 2, pp. 395-418. Permanent link optimization', - PermalienMathematical : https://doi.org/10.1007/s10107-008-0258-1. Rights / License - Licence de droit d’auteur :. General rights Copyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights. • Users may download and print one copy of any publication from the public portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain • You may freely distribute the URL identifying the publication in the public portal ? Take down policy If you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediately and investigate your claim.. Download date: 28. août. 2021 Moretus Plantin Bibliothèque Universitaire.

(2)

(3)

(4)

(5)

(6) !#"$%

(7) & '

(8) $()!

(9)

(10) ! * +,!-&#.) ,/0 12+3

(11) ! 4357698:<;>8S=@TS?A8B U

(12) M>CAD BV?8EF;H;HBGLXIKW)JL85 U

(13) M>IYC E$;HB8MVEN= M>OPJ C QSR abKc>OBJdH]>]e` Z J*[M>\?0]V^>_H]>` 1. 2,3. 2. f M>IK[O?gAhBi7jkAJ*Bo k*JlBA;>p>BJLF\g=mnAh?q[JL\gJ;H?ENAM>MeBBe;>?gG \Zqr;HJG =J;>\gk R W0J[;H\?IKJ*BV? f Q#s atdH` v C IKj3O;>k*AhGxkHwy8 4f ;eJ=zBV?gAh?B\g{nJu AD \gMAGh8 GOJ IKC ENM>BVMe?BV\g?J\g;>JG|;H8 k*G ; Cf ;HB;eL; WnJ*[;>\@?gI}JBV?~MHE;2? R J*IY;2?gAk= 6 0 o a C Z OJLJ :9W \gQO J*GGhJro = BC A:p>J*uzH\=]eA]>?z5] M> ;HI);HOI)\ C O:9\ JGhieAhOI s v IY;>AhGxwyp3AhBk*J*BV?8 IY;HGLIKAIKJLA?g5\{0Ax8 ?g OM>BIYL;H[-B8 Me;e=kH{08 4 J OC B[ LR [-A8Gh;>A[k>[8 4J>J 8 ?MeAhBV?{0 OBL[-8 ;>k>8 4J C 6M>BL=LJG; Jrk J*\k JjkAJ*BV?gAVOJ76 j C Z OJL v i>IKM>Z Z BV?JrC=@R Jr:;Hu\ak R ]eR ]>6])J*:GGhM2\gO J*GGJ= C :JGhie0AhOZ I 8 v 8 Z 8 68 8 8j 8 d C peJ*B3OJO=@v ?gIY;tp>;HJ AG|f wyM>IK\gAJ*f M>GMGAL= CAJ>68 I}uMeaO]VP feJt^{UkJMe\@O;eGhMek*O==8 J \ f JrLJ C 6\g;>BkJ 6M>BL=<[M>O\G;) M>\gIY;2?gAhMeB}G; Z Jk R J\gk R JL;>B=<Gx BLO=z?g\AJJ?Gx 0i>\gAk*OGh?O\gJ C Z OJL v i>IKM>BV?JrC=@Jr:;Hu\ak ]e]>6])J*:GGhM2\gO J*GGJ= C :JGhieAhOI Z R 1. 2. 3. 4. 5. 4. 2,5.

(14)

(15) "! #$&%')(+*,-/.0!'"! 123!'( 4

(16) 53673891: ! ;-<>=?A@CBEDGFH IJ<>KL?NMGOQPSR&TAH>KU<VKXWZY [\PNH>]"^\< _:WZD`FBSH>ab<_:WZOc?NFWZ@ dfeQg YFPihAjZjAk -l mAnop3qsrEo S,}u,tv u{,uxuwzySwzwzy|""{~}C|ySE2uES ,¡u,{S "uu:ySSuS~SwzyS 3{~}{5Sux{w{~,Vu}{uS2Euwzy)C|y,fu2u}~uwzy,C{,}C2|}"u 3 { }wCSy y,} ,E{"wzy,yS{",{~}u{ C"wzySS,u}~}cuCy&,{"{uwzwz2{Zu{wz}S 3 ux{{u)wzE,}y}~ 3}{>uSy|,u{\u}"}2u{S|{ C,,yJuu:¢ 3}} C{"u,{~{}wzSSCu ¬~y|fS}u2{~{uwz} C£¤wzysu ¥¢~u«,u®,}fuEu«ys}wz|w2{Cu}{y|,S{u«u2}9C£Cy) C}wz{S¯ wzu2°A ±V ²ff{³ÉSwzu2y}{EE9¦,¦|wz§u9u2{}wz£¤ wySu¨£Eu,vwz,yS 3S{ySu{,wzE{usy'} 3{{: C{,~© ~uwz{9yS u«w{u }~2 3~{Su«uE2{ S|ª\{-wCCwzyS>¦s £¤u C}~yS¥ E'y|£¤Cu«}2}uEzfwzu2uy,{wwyS{u¨wz £¤}~{uE¥ OBkMeB=z?g\g;>AhBJLIKAhBAIKA©¾r;2?gAhMeB C ?g\O=@?@u \gJ*i>AM>BIKJ? R ML= C kMeB3p>J*\gi>JBkJn? R J*M>\g5 C B3OIKJ*\gAk;HG J[µJ*\gAh¶,IK·´J*¸-BV¹¢?=*º8»&¼½ ¿ ÀZÁ-Â Ã&ÄÅ5Æ5Ç>Â È Ä"Á U \gO=@?@u \Ji>AM>BYIKJ? R ML=;H\gJ09J*GGhuGÉ3BM2BY?gJk R BAVOJr=<AhBXBM>BGABJ;>\9BM>Bk*M>B3p>J*}[\gM>ie\g;>IKI}ABi C R MV=@JlkMeBu kJ[? R ;>=}IY;2?O\JrL$M2p>J\IKM>\gJ? R ;HB ? R Ah\?z5 5>Jr;H\=ËÊ, Me\Y;HB J*3?J*B=@Ap>JkM2p>J\g;>i>J C =J*J f M>BB C M>OGL ;HBL =@UOMeI}AhBVJr? LC d>?gMK]>]>4]SJÌ8X;eL BJVO=@O;Hk ?R JAhIKB J*;2? R Î?gM\LO= =@C ?M>\BJJi>AM>k*M>BsBÏ C =A LRJ*\Ak=R ;NA=IK;MBLJ*J*AG i mR 4MeOMH\<R ? M3R M3J7L7MeMH4f

(17) ÍzJ? Rk?JAp>k*JKO\ \gOJ*BBVk?<?AAM>?gJ*B \;2 ?gRJ Axk R 8SA=U R;>=@A=u BA?gJ*J*A\i ;2R ?g4AhMeM>B O\ R M3MLYA=M>,?JBX\gJ*[\gJ=J*BV?gJLY435;)4;HGGAhB=@MeIKJnBMe\I C R MV=@J\g;eLAO= ∆ A=? R J*BO[ L;2?gJL7 \MeI ?gM7Ah?J*\;2?gAhMeB 435Fk*M>B=ALJ*\gAhBi M2 JGhG [\JrLAk?g=n? J)Me4fÍzJk?Ap>J OBk?gAhMeBp2;HGOJK;2? A?gJ*\;2?gJ x k 8) B\J*?\gMe=[kJrk+? C 1? R A=lIKAhi R ?=J*JI R OBB;H?O\;HmGy=AhBkJ}? R JKBJR \;>LAO= ∆ AGhGSLJ?J\IKABJ ? OR BJ0k\?JAi>M>ABFM>B};H\gAhM>BKO BRL Ak R ;[80MVmn==OAh4\0Gh;>5)AhI O[ALB ;2?? JrLKA=0IK[M3;HL[J*J*G \~mA=?gM[A=y\gM>J*[Me[=JrJk?;KJrL}pt;>?\MA;H[BV\g?~JLM>Ak?S? ? JR J?g\p2O;>=zGh?OuxJ0\gJ*MHieAhMe? R BJ~;>MeGhie4fM>ÍzJ\gkA? ?Ap>I J R R R ? R ;H?lLJ?J\IKABJ= ∆x ;>kkMe\gLABiY?gM R M2 JGhG m [\JrLAk?g=n? R J)p2;>GhOJ}MHy? R JKM>4fÍzJrk?gAhpeJ OBk?AM>B;2? ? R J\J4V5X=@53Bk R \gM>BA©¾AhBi)? R J\;>LAhO=<O[L;H?JAh? R ? R Jk R ;HBieJlAhBIKM3LJ*G=*8 x C U J~BJIKJ*? MLYA=SIKMH?gAhp2;H?JLK435K;H[[GhAk*;H?AM>B=yAB;>L;H[?gAhpeJ<?Jk BAVOJr=S Ak J[GM>Ah?y? J0AB M>\gIY;2u ? OAM>BB k?IYRAM>;>BFLkJMe;tIKp2;>[AhOG;H?gR;H4?GJAM>LB-O8\gU AhBR iKJr=@? JR J?JMek [R ?BAAIKVOA©¾rJr;2=?gAh?zMe53BX[A[k*\g;>MGhGk5XJr=;>=[A[B Jr;HM>\n\LAhJ*B \<? ?gR MYJR p2k;HMe\gBV5?J? 3R ?0JM>;>R Sk*k*;YOR B\;>M>kA=5757MHM>4? ÍzR JR Jk?gMeAhpe4fJÍzJ kO?BAp>kJ u ?AM>B C R J\J)BM>A=J)\gJLOk?AM>B k*;HB4JK;>k R AJ*peJL 4O?;H?;=@Ai>BAk;HBV?nk*Me=@?8 \g=@?0?g\O=@?@u \gJ*i>AM>BNIKJ*? R ML ?MA? AR IKL[53MVB=@J);>IK;YAIKk)M3;>Lk*k*J*OG\\g;>JkL5XOkA=?gAhMeLBNJr=Gk*;>\\A4ieJ*Jr\<LN? AR B;HBj3Jr=kM>?IKAM>BJIa]O8 Gs ?gAhM>[ GJf M>M> B? BR J)J?B;>M>G|A8=@JÊ|d>J]>p2]e;H],GOÌ8;2?gU JLNR J};2?nIK4;>M>AhB? R AL? JrR ;YJ)? k*R OJ*\\g\gJJ*BVA=? ;HBL k;HBLAL;H?JAh?J\g;H?Jr=*8 k J;>[J\BMeBIKM>BM>?M>BJ7;H[[\MV;>k ;>=4JJ*B LJ*p>JGhMe[JrL AhB$? JkM>BV?gJ3?MH BM>BGhABJ;>\S=z?gMk R ;>=@?Ak9[\Mei>\;HIKR IKAhBin435):<;e=z?gAhB Cf Ah\gAGhGM;HBLKU

(18) M>R ABeR ?:ÊxdH]e] sNÐ Ì C Ê=J*J~;HG=Ml:<;e=z?gAhR B Cf Ah\gAGhGM;HBL UMeAhBV? C d>]>] sNÑ ÌI}Me\J=[JkAhk;HGGh5 M>\? R JIKABAI}A¾;H?AM>B$M>~=;>I}[GhJ7;tpeJ*\;HieJ};>[[\gMt3AIY;2?gAhMeB=Ê|j R ;H[A\M C =@dHMe]eB ]>C,Ìarb>\gbJ*Gs 53ÌAh8BUiR M>JBIYEN;HMeABeB?gLJuAhPf J;H\\gJGhBMk=gJ;HIKAh[? RGAB\gi JC=[;JIKk?yJ?g? MlR M3? L7R J0;HG=@MeM"\«É3É)BMHM2 f BYM>;eB=B=gÒ;HÓIKÐ[Ô©ÕGJAu =y[?;2R ? R;2?M>A[??g;>AhIKGhGM2A¾~;2=y?gAh;lMeB'\gJÊLZ OMek?4AAM>BB u MH ? R J0IKMLJ*G =@IY;>GhGJ*\y? R ;HB? R JnBM>A=@J0GhJp>J*Gx8 B4M>? R k;>=J= C ? R Jn=A©¾J~MH? R J0IKMLJG\gJLOk?AM>BA=y? R J0IY;>AhB ?kRMeOI}=[;>M>[B[J*BVGAh?<JrL?MYM>LBJ?k*R ALJ7J4M>;>BX=A?=R MHJ<L[Jr;>=@=@A\g?JLAhB;>M>k*\gk*IKO\;H;>?kAM>5YB M>C ;2? ?R J? R M>J4f[ÍzJr\gkJ*?gp3AhApeM>JnO=lOAhB?kJ?g\gAh;HMe?B-J w C ?\gR ;HJ? R ;eJL\l;H? [R ?;HAp>?)Jl;2IK?J? Rk R J7;>k*BOA=\I \gJ*BVA=? a k. k. k. k. k+1. k+1. k+1. k+1. k+1. k. k+1.

(19) d.

(20) ! " #$%&' )(*+ ,-(.)/.0. M>BJe8)mnO\lBJ* [\Me[MV=;>G

(21) A=n? R JB IKM>?Apt;H?JrLF435 ? R J R Me[J}MHAhIK[\gM2p3AhBi7? R Jr=@J}?gJk R BAVOJ=4Jk;HO=J? R J IKMe=@?n\gJ*GJ*p2;HBV?nAB Me\IY;2?gAhMeBFM>B? R J}IKMLJ*Gx =lVO;>GhAh?z5N;2?? R JKkO\g\gJ*BV?nAh?J\g;H?J9M>OGLN4JKO=JL C AB=z?gJ;eLFMH ;2?<? U R JA[=\[J;>p3[AhMeJO\ =<JA?g[J*G\M>;2\g?gJJ>=8 ? J9? JM>\gJ?gAk;HGV[\Me[J\@?gAhJr=

(22) ;>BL[\g;ek?Ak*;>GeB3OIKJ\Ak*;>Ge[MH?gJ*BV?A;HG3MH? J9BJ*$?\gO=z?u \Ji>AM>BKR ;HGi>Me\Ah? R I 28 1 J0AhBV?R \gMLOR k*J<? R J~BJ* IKJ? R M3LKAB7j3Jrk?AM>B7d C ;>BLK=z?gOL5}A?=kMeB3p>J*\gi>JBkR J<ABK? R J0BJ3? =@kJrMekB?gkAhGMeOB-L8<Jj;HJBkL7?gAhMeJ*B ;> IK[AhB\gJJ=J*[BVJ*?g\=~=[[Jr\kJ?GhAAp>IKJrAh=B ;HM>\g\<5 B3OO?gOIK\JJ*\g\gAkJ;H=JG;>J*\gk [R JAh\BFAJ*Bj3Jrk*Jk?gM>AhMeBFB =@?g;>8 BL;H\LXBM>BGABJ;>\~[\gM>4GJ*IY=831J 4 5 2Ã 6\Â Ã&" Ä 798:6ÇVÂ È ;"6 Â Ã,Æ 7´=Â <Ã 6>"È3Ä"@ Á ?"A>"Ä"Ã&È2Â B,C 1 Jk*M>B=ALJ*\9? R JOBkMeB=@?\;HABJLXM>[?gAhIKA¾;H?AM>B[\Me4GJ*I Ê|d8ha¤Ì min f (x), DE R J*\gJ f A=; ?zAk*JukMeBe?gAhB3OMeO=Gh5$LAhP J*\gJ*BV?A;H4GJ M>4fÍzJrk?gAhpeJX OBk?gAhMeB R Ak R IY;H[=Y Z AhBV?gM Z ;HBLA= 4MeOBLJrL 4J*GM28 U

(23) \O=z?ux\gJ*ieAhMeB I}J*? R ML=Y;>\JAh?J*\;2?gAhpeJ7[\gMkJr==J= C R Ak R C ieAhpeJ*B ;=@?g;H\?ABi [MeAhBV? x C k;2Me?gAhBMe=zB ?g\Ok?;;X?z=@JrAk*VJOuJ*kBMeBek*J?gAhB3(xOMe)O=Gh5LM>AP AJ*?g\gJ*J*\BV;2?g?gAJ;>= 4RGhJM>[I}JM OLGJGG5FkM>B3Ape=0J*\gLi>J*ABBiKJr?LNM ;7=AkM>GO?AJM>B?\gMHO=z?ÊxdAhB8aE=@ÌA8LJ)9;?lJÊ,?z;e53k [R Ak*A;H?gJ*GG\5 u R R v Ok*A=9GhAL? kJCJ;>BNBNÌk4M>;>IKGhG[BO?JMHLX 43\g5;eLÊA;>O[= [∆\gMt>AhIY0;2?gk*J*J*GBV5?ÌS\gJIKL}AB;2Ah? IK? AR ¾*J~ABkmi}O\g? \gJ*JBV?yIKAM?gJ*L\J;2G ?J x AhC Bk=@;HALGGhJJrL? ? JlR J2?g\Ó OF=zGI?H9Ó \gJ*F«i>Ò JAM>KLB M8y#8S=zU ?gJ*[ J R R Ah ;>mBLM>BG5Ah R ? R J\g;H?AM B R s?\gA;>G[MeAhBV? A 9 = ? * J N B > ; k k J [ ? r J L e ; = ? J B * J 3 ~ ? A g ? * J \ 2 ; ? J R R x +s x ρ C n. x∈. n. 0. k k>0. k. k. k. k. k. k. k. k. k. k+1. def. ρk =. k. f (xk ) − f (xk + sk ) mk (xk ) − mk (xk + sk ). AMH=l0G;e;Hk \gR i>AhJJ\p>Jr? RL ;>\gB JL; O=kIY?gAhMe;HBGGS[Ê AMVB$=@Ah? ?R Ap>J7JYM>k*4M>ÍzBJk=@?g?g;>AhpeBeJY? ηOBkÊ?A?gAM>J*\BN;2Ì?gAh?MeM B [k\JrAL=lAk? ?R JrJL B \gk*J;HLGGOJkLN?AM>HOGQBPOPÊÒAhB H)H'ROA?G =Ô ÌG8YMk* ;HB G? IKR JM3LkJ*G;>G =gm=Ak*;>Ì GC AhB\g;>kIK\gJJ*;>9=JM>L)\É AhC ? ? R JJ??\g\gAO;H=zG?[ux\gM>J*AieBeAh?SMeAB =S;e\g;ek*LkJAhO[=~?JrALv=0OÊ,? [ L;2;H?S?AJ=yLA ;2?0? R JJBÌ LN;HM>B LKJrL;>Jrk Rk\gJAh?;>J=\gJ;HL)?AMHM>? B-w<J*\gAh?nAA=~=@JeG8J, ?lB}O? BkA=SR ;>kB;>=i>J JrL ? M>J\ BJ* p2;HGOJ ∆ R A=yk R MV=@JBAB? R J<AhBV?gJ*\gpt;>G R [γ ks k,ργ ks> ηk] C M>\y=M>IKJ9k*M>B=@?g;>BV?g= R 0 < γ < γ R < 1 891 C R JR B A=nBJ*iV;2?Ap>J ;7VO;eL\;2?Ak?MH ? R J)IKM3LJ*GA=0O=@JrL Ê;e=~AB M>BBNJ?;>G|8 d>]>]e] C [p>J8n\^25 `>=@SOÌ k*C k?JrMX==@L JO?G

(24) JAh\B IK? ARBJJ ρ;K?JBV?g;2?gAhpeJBJC \g;eLAhO= R Me=J[O\[Me=JA=?MXJ*B=O\gJ? R ;2?~? R f JBJ*3?nAh?J*\;2C ?gAhMeBNA=0 =@JB=Jn? R ;2? ρ > η Me\~=@MeI}J η ∈ (η , 1) 8SU R A=<p2;HGOJA=<i>Ap>JBX435 θ ∆ C R J*\gJ Êxd8dÌ (1 − η )h∇ f (x ), s i θ = . (1 − η )[f (x ) + h∇ f (x ), s i] + η m (x + s ) − f (x + s ) ?AM>B mnkO\Ê,? BR J*;2 ?~A;H=Gi>;2Me?\? Ah?R R JI 4JLi>AP ABJ*B\A=BAiKB7MH? R A;2?g?J*\? ;2R ?JAM>?B \gOk=z?+ux\g1J*ieÌAhMeMeBXB7?\R;>JLA4O;e=9=@AA==9OMH[ L? R;2?gJSJLF«Ò Ó;2FT,L?HJU¢\Ò)JP ;eÓK+k V¤R Ò=\gO;Hk*?k*AMJ=g=z OGA?gJ*\;2u k. 1. k. k+1. 0. k. 1. 1. k. 0. 1. k. k+1. 2. 2. k. 1. def. 2. def. ρ˜k+1 =. x. 2. k. k. x. k. k. k. 2. k. k. k. k. k. k. k. f (xk+1 ) − f (xk+1 − sk ) f (xk ) − f (xk + sk ) = mk+1 (xk+1 ) − mk+1 (xk+1 − sk ) mk+1 (xk ) − mk+1 (xk + sk ). MHn;ek AJ*p>JrL ?gM [\gJLAk?JLk ;>Bi>Jr= AGJ k*M>BV?AB3OABi?gM O=@J ?M LJrkALJ J? J*\}? JX?g\A;HG<Ah?J\g;H?J IY;t5NR4J;ek*kJ[?JrL87mnO\IKJR? R ML? R C J*\gJR Me\JKLA=z?gAhBi>OA= R J=? R JK?zρM \gM>GJ=[G;t5>JLR 435R ρ RAB? R JkG;>=g=Ak;HG ;HGi>Me\Ah? R I w ? R ;2? MHLJrkALABin;ek*kJ[?g;>BkJMH? R J?g\A;HGVAh?J*\;2?gJ;>BLl? R ;2?

(25) M>LJ*?J*\gIKAhBAhBi0? R J9\;>LAO=-O[L;H?Je8 ?;>G=M7J*3[GhAkAh?JGh5?g;É>Jr=~ABe?gMX;ek*kMeOBV?0? R ;2? m C BM>? m C A=nO=JLNAh? R AhB? R J)?\gO=@?n\gJ*i>AM>BNM>\;>LAO= ∆ 4357J*Ah? 8R UJ*\0R AOB= k*C \ Jr;>R =JAhBKBi}? R ? JR JAh?kJO\g;H\?\gJJ*BVR ?~;e\= ;>L\A=zO?S=<4M>J*\0JBGhJr;t;>pVkAkBJ*iK[?AhJ?nL OC B? Rk R ;H;H?BA=yieJ LXR AJ* B ρ˜ρ >>η˜η MeC \~9LJ~Jkk*Me\IKJr;>[=OJ?AJ<?0? M>R ? AR =SJ\\;>LAA=OJ>= 8 ;YBV? OR ;>AL=\G;2;>?g=@A?0kk*;>?0=J MHC Ah?~? R AJ=~IK;HiVM3;HLABJ*Gk AR =~MVO=@J=@JrBXLAB;e=0? R ;>J4M2ABVpe?JnJ?g\M7p2;HLG J*[γ?J*\gksIKAhBk,Jγ;Kks?JBek]?;28?gAhENpeJM>B\gJ*J*M2 peJ*\\ ;>C L AOR =JB RρÃk R A=BAJ*GhG

(26) iV;2IK?gAh;peÉ>J J C k. k. k+1. k. k+1. k. k. 0. k. 1. k. 1. 1. k.

(27) M>\l=@MeIKJ η˜ ∈ (˜η , 1) 8U R A=0p2;>GhOJ}A=.

(28) ! " #$%&' )(*+ ,-(.)/.0. ? R J}BJ*V?lAh?J*\;2?gAhMeBFp>J\5 =Ok*k*J=g=z OGABF? R J}=J*B=J)? R ;H? i>Ap>JBX435 θ˜ ∆ C R J\J k+1. ρ˜k+1 > η˜2. 2. 1. k. def θ˜k+1 =. Êxd8 ,Ì. −(1 − η˜2 )h∇x f (xk+1 ), sk i . (1 − η˜2 )[f (xk+1 ) − h∇x f (xk+1 ), sk i] + η˜2 mk+1 (xk ) − f (xk ). M>?U AkR JA=}? R G;HJ?;eLθ˜=?gMFO? =R JJX=9\g? JR ?gJ\MVi>=@\[;>JLkAh?JABep>?JY;2?g?\O? R=z?Jux\gBJ*Jie AhMeB [M>I}ABeJ*? ? C R \M;2L ? R LJ*J\9=g? kR \gAh;>4BJL ? R ;>J=}Me~GLXGi>M>M>B\gJAh? ;eR =9I AhBd8Ê|a d8 d,AÌB 8 R Ak R 9J GhJr;tp>J)? R JY[\JrkA=@JKLJBA?gAhMeB=M>? R JYI}MLJGxÊ;H?)jV?J[a¤Ì;HBL MH>Î@=OkAJ*BV?\gJLOk?AM>BsÏÊ;HC ?j3?J*[ SÌ~ Me\ ? R JBJ3?~=@Jrk?AM>B8 k+1. ¹¢º

(29) N½Ë¶ º¹A¼¶ N¶ º&¼ «º¶ `¹ "! ¹Nº# $ %

(30) '&()*,+ - C¶/. 1 ~BNABA?gA;>G[M>ABV? ;HBL ABAh?A;HG-?g\O=z?ux\gJ*ieAhMeBX\;>LAO= ;H\gJi>Ap>J*B8 U R½ J0%k*M> B;>=@B! ?gL ;>BV$¼ !?g= $η2¹ 3C η˜ C η˜ C γ C γ ;>8 BL MexγI}[;HO\g?JJ ;HG=Mi>Ap>;>JBB L;>=@BJ*? LF=g;2?gA=@ 8 5 0 <∆η <> 10C 0 < η˜ 6 η˜ < 1 0<γ <γ 616γ f f (x ) k=0 - C¶ 4A1 j3JGhJrk? ; ?zAk*JukMeBe?gAhB3OMeO=Gh5 LAhP J*\gJ*BV?A;H4GJ IKMLJ*G m LJBJL AB ½ B5 8 ¹» ¶ » ¶ 67% 2¹ 8 - C¶ 91 ½ ¶ º¹N ¼ V:¶ ; $#A¶ º ¼ <º¶ 2¹ cº !%» =¼ %V » ! C2¶ k = 0 C ieMn?gMjV?J[K8

(31) x = x C ? R J*BNk R M3Me=J ( A ρ ∈ [0, η ) C Êxd8 Ì γ ks k ∆ = A ρ < 0 C min[γ ks k, max[γ , θ ]∆ ] R J\J θ A=LJ*BJLABÊ|d8dÌ8 v G=@J C LJBJ Êxd8Ì f (x ) − f (x ) ρ˜ = m (x ) − m (x ) ;HBLXk R M3MV=@J  A ρ˜ > η˜ C max[γ ks k, ∆ ]    A ρ˜ ∈ [˜η , η˜ ) C  Êxd8 s Ì ∆ ∆ = A C  γ ks k ρ ˜ ∈ [0, η ˜ )   A ρ˜ < 0 C  min[γ ks k, max[γ , θ˜ ]∆ ] R J\J θ˜ A=LJ*BJLABÊ|d8 ,Ì8 - C¶ /> ½ - C¶ ?@! @ ! `¹ 8 f MeIK[O?J;K=@?J*[ ? R ;2?VÎ=@AO Yk*AhJBV?G5Y\JrLOkJr=9? R JlIKMLJ*GÏ m ;>BL s =Ok R ? R ;2? x + s ∈ B 8 - C¶ ?B ½ @¶ % $!A ¶b¹ CD $ ¶ Cº ! V2¹ E @ f MeI}[O?J f (x + s ) ;>BLLJBJ Êxd8 ^|Ì f (x ) − f (x + s ) ρ = . m (x ) − m (x + s ) ρ > η C ? R JB LJ*BJ x = x + s ;>BL k*M>IK[O?gJ ∇ f (x ) F M>? R J\A=@JLJBJ 8y Bk*\JIKJ*BV? 435a;>BL7ieM}?MjV?J[ a>8. . 0. 1. 2. 0. 1. 2. 0. 1. 1. 0. 2. 1. 2. 1. 0. k. k. k. k−1. 1. k. k−1. k−1. 1. k−1. 0. k−1. k−1. 1. k−1. k−1. k−1. k. k. k−1. 2. k. k−1. k. k. k−1. k−1. k. 1. k−1. 2. k. 1. k. k−1. 1. k. 0. k. k−1. 2. 1. k. k. k. k. k. k. k. k. k. k. xk+1 = xk. k+1. 1. k. k. k. k. k. k. k. k. k. k. k. k. x. k+1.

(32) .

(33) ! " #$%&' )(*+ ,-(.)/.0. Ä"Á ;"6Ã>"6Á5Ç36 Â B,6Ä2Ã 1 J BM2 AhB3peJ=@?Aie;2?gJ? R JFkMeBVpeJ*\gi>JBkJX[\Me[J\@?gAhJr=}MHnMeO\Y;HGi>Me\Ah? R I 8 j3AhBkJNAh?Yk;HB4JNk*M>B=ALJ*\gJL ;e= ;=@Ai>ptB;>A\Ak;H;HBVBV?K?)MH=nAhIK? R AGJN;>\4Ah?;eA=@JA=k7Ah?B \gO? =z?J*ux\gA\}J*ie[AhMe\M3B M>IK=*8NJ? R B ML M>;H ?f MeMeBGhGBM2~J= ?;H9GxJ8xÊ|d>;t]>p>]>JX]SÌ ;2C ?@?gJ*JIKJ[?gJ[L Jk?g?7MF=4AI}JXAGJ;H\K[G\AJrk*=@A?}OGM>?=YB ;H? BL J ;>=g=@OI}[?AM>B=;>BL7[\Me[J\@?gAhJr=R C 4O??MK\J* J*\<?gM f R R ;H[?gJ*\ s MHC

(34) ? R A=R \gJ J\JBkJl R JBJ*peJ*\<[MV==A4GhJe8 R mnO\~;>=g=@OI}[?AM>B=;>\JALJ*BV?Ak*;>G?M}? R Me=JO=@JrL7 Me\<? R J4;>=Akn?\gO=@?@u \Ji>AM>BXI}J*? R ML8 U R k*M>BJ =@~?g;>JBe=g? =Aκ;>B M>=

(35) O? k R R J? MeR 4f;HÍz? JC k?MeAp>\~Jl;HGOG Bxk∈?AM> Z B ∇C f A=~OBA Me\IKG574M>OBLJL C A|8 J>8? R J\JJ*A=@?g=~;K[Me=Ah?Ap>J . n. xx. k∇xx f (x)k 6 κ .. . U R J0IKMLJ*G m A=S\g=@?@u M>\LJ\kM R J*\gJ*BV?Ah? R ? R J0 OBk?gAhMeB f ;2?9J;ek R A?gJ*\;2?AM>B x C A|8 J>8? R J*A\p2;>GhOJr= ;>BLi>\;>LAJ*BV?g=<;>\JJVO;>G-;2? x Me\~;HGG k w ;>BL g = ∇ m (x ) = ∇ f (x ). m (x ) = f (x ) U =OR k J ~? J=g;H=? A;>B MeM>\~

(36) ;H? GGR JIKM LJ*G ;H∇BLm Me\~;HA=0GG OBAh M>\gI}G574M>OBLJL C A|8 J>8? R J\JJ*A=@?g=~;Yk*M>B=@?g;>BV? κ

(37) > 1 R R C Z C k. k. k. def. k. >. x∈. k. n. k. xx. k. x. k. k. x. k. k. k. k∇xx mk (x)k 6 κ

(38) − 1.. B. U [R M>J0ABVL? Jk*Ax\8 JeJr8 ;>=? J<JM>\BKJl? J*R 3J~A=@IK?g=~M3;KLJ*kG MemB=@?g;HABV=S? ;2?GJ;e=z?9;>=yI)Ok =R Ok ;>=S?; ;2\? ;>k?M>AM>\~BK;>MHGhG ? R ;2?Me4?g;>AhBJLY;2?? R J f ;>Ok R 5 R R R C kC F κ

(39) ∈ (0, 1) k. mk (xk ) − mk (xk + sk ) > κ . . kgk k , ∆k kgk k min βk. . Ah? R β = 1 + maxk∇ m (x)k 8 M>?J)? ;H?8 =[JrkAhJ=? JKBMH?gAhMeBMHÎ@=OkAJ*BV?l\JrLOk?gAhMeBsÏ)O=JLAhB jV?gJ*[ 7M>M>O\;>GhieM>\gA? I AhGJ ? R JKk R MeR Ak*JMH m ABjV?gJ*[$R a)A=GAhIKAh?JrLF435N8d;HBL8 8 1J};>G=MBM>?J? R ;2? s 6= 0 R JBJ*peR J*\ gC 6=R 0 4Jrk*;>O=@JlM>8 8 def. k. x∈Bk. xx. k. k. . k. k. !"$#%'&(*),+

(40) -.&0/2134$56+7&8+#%9;:=<>+?&(-. B ? R A==@Jrk?gAhMeB 9J [\gM2p>JX? R ;H?}? R JN\gJ?\gMe=[Jk?gAhpeJ7?g\O=z?ux\gJ*ieAhMeB ;HGi>Me\Ah? R I A=YieGhMe4;HGG5$kM>B3peJ*\gi>J*BV?)?M [\\gM3=z?LuxOMe\gkLJrL7J\43k5Y\gAh? ?ACk*J;H;HG9G[i>MeM>\ABeAh? ?= IC AhB d8?aR =gJ ;2?g=@AJ=@B=JJ= ? R ;2?)Jp>J\5 GAhIKAh?}[M>ABV? x M>~? R JX=JVOJ*BkJ7M>0Ah?J*\;2?gJ= (x ) R R k. ∗. Ah\g\gJ1=[JlJrk?\Ap>=@J?<ieMHAhpe? JR ;)J4k R M>M>OABkLJMHM>BX? ?R JJ=@J?g;>\\g\@M>?gAh\9B4i)J*[?zM>AJBVJ*? BXx? Jl;HB?gLX\OAJBAhM>?4fA;HÍzGJrk?g?\AOp>=zJn?ux \gOJ*BiekAh?MeABM>B \g;eLA;>OB= L∆A?=8 kO\g\gJ*BV?I}MLJG R8 R f 2 ; ? ? J [ g \ * J 3 p A > M O 9 = h A ? * J \ 2 ; g ? J R m x ∇x f (x∗ ) = 0 0. k. k−1. 0.

(41) .

(42) ! " #$%&' )(*+ ,-(.)/.0. &¶S¹Nº¶:

(43). )*. j3O[[Me=Jl? R ;H?~8a38 R MeGL-8SU R J*B 9J R ;tp>Jn? R ;H?. > . Ê8aEÌ. |f (xk ) − mk−1 (xk )| 6 κubh ∆2k−1. ;HBL C A Ah?J*\;2?gAhMeB. k−1. A=~=Ok*k*J=g=z OG C ? R ;2?. Ê8dÌ Ê8 ,Ì. |f (xk−1 ) − mk (xk−1 )| 6 κubh ∆2k−1. R J*\gJ. κ

(44) = max[κ7 , κ

(45) ]. def. >M¨Bº¤G¹ 5)¹ [C \ M2peJ U ÊRJ8 d,Ì48 M>:OBJrLJk*;>OÊ=@8J~aEÌ<? R LJnAh\gM>J4fkÍz?JrG57k?\gAp>J=J~O GO?B=<k ?g\gAhM>MeI B7;HU BR LKJ*M>? R\gJ*JnI IKs M38 L J*8aG ;HAB\gJ f CM>BB OJ*B?nk?;HAM>Gx8 BÊx= dHC ]>9]eJ~],ÌIY8 ;t1 5}J;>[? R [OG5 = ? R JXI}Jr;HB p2;HGOJY? R J*Me\JI MeB ? R J7Me4fÍzJk?Ap>JK OBk?AM>B f ;>BL M>B ? R JIKMLJ*G m C ;>BL M>4?;HAB \MeI ? R ;H? x =x −s Ê|8 Ì f (x ) = f (x ) − hs , ∇ f (x )i + hs , ∇ f (ξ )s i Ê|8Ì m (x ) = m (x ) − hs , g i + hs , ∇ m (ζ )s i M>\~=M>IKJ ξ C ζ AB? R J=J*ieI}JBV? [x , x ] 8 :Jrk*;>O=@JMH8d C ? R J)Me4fÍzJk?Ap>J OBk?gAhMeB f ;HBL ? R J}IKMLJ*G m R ;tp>J? R JK=;>I}Jpt;>GhOJ);HBLNi>\;>LAJ*BV? ;2? x 8SU R O= C =O4?g\g;ek?ABiÊ8 ,Ì \MeI Ê8 Ì<;HBL7?g;ÉVABiY;H4=@MeGhO?gJlpt;>GhOJ=<53AhJGL=9? R ;H? . 2. k. k−1. k. k−1. k−1. k. k. k. k−1. k−1. k. k. k. x. k−1. k−1. 1 2. k. 1 2. k. k−1. k−1. xx. xx. k. k. k. k−1. k−1. k. k. k. 1 2 |hsk−1 , ∇xx f (ξk )sk−1 i − hsk−1 , ∇xx mk (ζk )sk−1 i| 1 2 [|hsk−1 , ∇xx f (ξk )sk−1 i| + |hsk−1 , ∇xx mk (ζk )sk−1 i|] 2 1 2 (κufh + κumh − 1)ksk−1 k 2 1 2 (κufh + κumh − 1)∆k−1 ,. |f (xk−1 ) − mk (xk−1 )| = 6. |Ê 8 s Ì R J*\gJlJ=Ok*k*J=g=@Ap>JGh5O=@JrL? R Jl?\gA;>Bi>GJlAhBJVO;HGAh?z5 C ? R J f ;HOk R 5Vujk R <;H\¾~ABJVO;>GhAh?z5 C ? R JABLOk*JL IKÊ;H8d?Ì9\gAYk*GhJrB;HMe\g\GhI5 [M>\gGM>L[=*J*8 \?AJ= C 8a C 8 C ;HBL? R J0;>k?? R ;2? x ∈ B AhIK[GAJ=? R ;H? ks k 6 ∆ 8Sj3M R 2 U R O=9? R J;HB;HGM>iM>

(46) U R JM>\gJ*I s 8 8a0M> f MeBB7J*?;>G|8´ÊxdH]>]e],Ì R M>GL=9ABXMeO\k*;>=J C R J\J9Jn\gJ*[G;>k*J~? R J M>\g90;>=y\gLXMeOL\SAhPBJJ\ JB\k;2J ?AfM (x O=)J=y−? mJ\g(xJLOk?)AM>43B}57AB Ah?g=<\gJ?gAh\BMV=z=@?g[J;>JL}k?M>Ap>J? p2J;H\g\AJr;>LBeO? kf?(xAM>BKAhB ) − m (x9J;H\g)J<8 AhBV?J\Jr=z?gJL AhBN;}4M>OBLMeB7? R J*A\0LρÃhPJ\JBkJ R C R Ak R A=[\gM2p3AmLJrL435? R A=BJ3R ?\gJ=OG?r8 m C 6. 6. k. k+1. k. V¶:

(47).

(48) ! > . k−1. C. k−1. k+1. k−1. k. . k−1. k. k. k−1. k−1. k−1. j3O[[MV=@J? R ;H?)8a8 R M>GL8 U R J*B 9J R ;tp>JK? R ;2? C Me\Jp>J*\g5=Ok*k*J=g=z OGSA?gJ*\;2?gAhMeB Ê8 ^|Ì | [m (x )−m (x )] − [m (x ) − m (x )] | 6 2κ

(49) ∆ . k−1. k−1. k−1. k. k. k−1. k. 2 k−1. . k. ¨º¤¹ ¹ C o =@ABiN? R JIKMLJ*G<LAhPJ\JBe?gA;>4AGhAh?z5 C JX;H[[G5 ? R JXI}Jr;HB p2;HGOJY? R J*Me\JI MeB ? R JXI}MLJG C ;HBLXJlMe4?g;>AhBX? R ;H? m Ê|8 `,Ì ,∇ m (ψ )s i m (x ) = m (x ) + hs ,g i + hs M>\~=M>IKJ ψ AhB? R J=@Ji>IKJ*BV? [x , x ] 8 Z J*IKJ*I)4J\<? R ;H? Ê8 ,ÌABX? R Jl[\gJ*p3AM>O=9[\gM3MH

(50) ieAhpeJ=? R ;2? Ê|8 b,Ì , ∇ m (ζ )s i m (x ) = m (x ) − hs , g i + hs . k−1. k−1. k. k−1. k−1. k−1. k−1. k−1. k. k−1. k. 1 2. k−1. k−1. xx. k−1. k−1. k. k. k−1. k. 1 2. k−1. xx. k. k. k−1. k−1.

(51) s.

(52) ! " #$%&' )(*+ ,-(.)/.0. M>\n=M>IKJ ζ AhBF? R J}=@Ji>IKJ*BV? [x , x ] 8njO4=@?Ah?O?gAhBiJÊ8 `SÌ;HBL Ê8 bSÌ<AhB=@ALJ? R JGJ,?u R ;HBLF=ALJMH Ê8 ^|Ì ;>BLO=@ABiN8 ? JK?g\A;HBieGhJKABJVO;>GhAh?z5 C ? R J f ;>Ok R 5euzjk R <;H\¾AhBJVO;HGAh?z5 C ;>BL ? R JYABLOk*JL IK;H?\gCAXBM>\gI [\gM>[J*\?C AJ=<R 53AJ*GL? R ;2? k. k−1. k. | [mk−1 (xk−1 ) − mk−1 (xk )] − [mk (xk−1 ) − mk (xk )] |

(53)

(54) =

(55) − hsk−1 , gk−1 − gk i − 1 (hsk−1 , ∇xx mk−1 (ψk−1 )sk−1 i + hsk−1 , ∇xx mk (ζk )sk−1 i)

(56) 2. 6 ksk−1 k · kgk−1 − gk k + κ

(57) ksk−1 k2 .. M2 Me4=J*\gp>Jn? ;H? 4Jk;HO=JlMH8 d }I Jr;HBp2;HGOJl? R JR M>\gJ*C I MeB ∇ f ;HBLXCMekg4?g;>AhBX? −R ;2g? k = k∇ f (x k−1. k. x. k−1. Ê 8har],Ì 8 1Jl? R JBN;H[[G5Y? R J ) − ∇ f (x )k x. k. x. ∇x f (xk ) = ∇x f (xk−1 ) +. Z. Ê8haeaEÌ. 1. ∇xx f (xk−1 + αsk−1 )sk−1 dα.. U R O=9? R J f >; Ok R 5euzjk R <;H\¾~AhBJVO;HGAh?z5 C ;>BLX8halieAhpeJn? R ;2? kgk−1 − gk k 6. Z. 0. 1. k∇xx f (xk−1 + αsk−1 )k · ksk−1 k dα 6. j3O4=@?Ah?O?gAhBiY? R A=<4M>OBLABÊ8a]SÌ C 9JlM>4?;HAB7? R ;2? 0. Z. 1 0. κ7 ksk−1 k dα = κ ksk−1 k.. Ê8hatdÌ. | [mk−1 (xk−1 ) − mk−1 (xk )] − [mk (xk−1 ) − mk (xk )] | 6 (κ + κ

(58) )ksk−1 k = 2κ

(59) ∆2k−1 2. R J*\gJlJB;>GhG57O=@JÊ8 SÌ C ;>BLX? R Jl;ek?<? R ;H? x ∈ B 8 2 1 J}kM>BkGOLJ \gM>I ? A=n\Jr=@OGh?~? ;2?n? JLJ*BMeI}AB;H?M>\=ABN? JJ[\gJ=g=AhMeBM> ;>BL LAP J*\n435 ;7VO;>Be?gA?z5 R Ak R A=nMHS? R R J}=g;HIKJ)M>\R LJ*\;>R =0? R J}J*\g\Me\04J?z9J*JBNR ? R J}IKMLJ*Gy;HBL ? R ρ˜J}M>4ÍzJkρ?gAhpeJ OBk?AM>B8 \go ;e=LAhBAOiY=0? AR =A=~=O Mek4=J*AJ*\gBVpt;H?G?5FAM>=@B IYC ;>GhJGSk*;>M>\IKJB[M2;H\gJLNk*;>?[MX;H4? R GhJJKM>i> \;>= RLM2AhJBeA? BC iK;HB? R L ;2?~;HG?=@RMXJ? RAh?;2J?\g;H? ?R AM>J}B ?\gIOO=@?@=@u ?0\J4i>JAM>=BFOk*\k*;>JL=gAh=zO O= G

(60) R A;e =n? R ?MJ AhBk\gJ;>=JAhB? R A=~k;>=J>8 k. k−1. k. &¶S¹Nº¶:

(61). )*. > >. U R J*B Ah?J\g;H?AM>B. j3O[[Me=Jl? R ;H?~8a38 R MeGL-8j3O[[Me=Jl O\? R J\IKM>\gJn? R ;H? g ∆k−1. k−1. k. 6= 0. k−1. ;>BLX? R ;H?. (1 − η˜2 ) κ

(62) 6 min 1 − η1 , kgk−1 k. (3 − 2˜ η2 ) κ

(63) . A=~=OkkJ=g=@ OG-;HBL. . Ê|8a,Ì Ê|8a Ì. ∆k > ∆k−1 .. @=Öº¤k¹ kJr¹ =C =@ OG;>1 BJL}? R\g=@O?0=;H? [R [;2Gh? 5 xU R =J*Mex\JI +s 8 s8 d)MH 6=f xMeBBNJ8S?nmn;>4G|8 =J*Ê|\gdHp>]eJ]>]SBÌSM2?gM7? LR Jr;2L?O? kR JJ0? k*R M>;HB?0=@?gA?g;>J*BV\?g;2= ?gηAh˜MeB ;HkBL −κ1

(64) A= GhAJAhB? R JAhBV?gJ*\gpt;>G (0, 1) C R Ak R AI}[GhAJ=<? R ;2? ;HBL? R O= κ (1 − η˜ ) < 1. Ê8hatÌ 1 (1 − η˜ ) < <1 (3 − 2˜ η ) 2 (3 − 2˜ η ) U R Jnk*M>BLAh?AM>B=:Ê8har,Ì C Ê|8arÌ C ;>BL'Ê8 SÌ C kMeI4ABJLKAh? R ? R JLJBA?gAhMeBMH β AhBX8 AI}[Gh5}? R ;2? Ê8ha s Ì kg k kg k ∆ < < . . k. k−1. k−1. k−1. 2. 2. 2. mdc. 2. 2. k−1. k−1. k−1. κ

(65) . k−1. βk−1.

(66) ^.

(67) ! " #$%&' )(*+ ,-(.)/.0. 0=~;KkMeB=JVOJ*BkJ C 8 AIKI}JrLA;2?JGh5Xi>Ap>Jr=? R ;2?. Ê8ha2^|Ì mnB7? R JMH? R J*\ R ;>BL C 9JlIY;t57;H[[Gh57TJ*IKIY;K8d);>BLO=Jn? R Jl?\gA;HBieGhJlABJVO;>GhAh?z5Y?MKMe4?g;>AhBX? R ;H? mk−1 (xk−1 ) − mk−1 (xk ) > κ

(68) . . kgk−1 k kgk−1 k min , ∆k−1 = κ

(69) kgk−1 k∆k−1 . βk−1.

(70)

(71)

(72)

(73)

(74) mk−1 (xk−1 ) − mk−1 (xk )

(75) −

(76) mk (xk−1 ) − mk (xk )

(77)

(78)

(79) 6

(80) mk−1 (xk−1 ) − mk−1 (xk ) − mk (xk−1 ) − mk (xk )

(81) 6 2κ ∆2k−1. ;HBL7? R J\J* M>\gJ C Ah? R Ê8ha2^|Ì C ? R ;H?. |mk (xk−1 ) − mk (xk )|. > |mk−1 (xk−1 ) − mk−1 (xk )| − 2κ

(82) ∆2k−1 > κ

(83) kgk−1 k∆k−1 − 2κ

(84) ∆2k−1 .. M2UÊ|8aSÌAIK[GAhJr=<? R ;H? (3 − 2˜η )κ

(85) ∆ . 2. k−1. 6 (1 − η˜2 )κ

(86) kgk−1 k. ;HBLX? R O=9? R ;2?. (1 − η˜2 )(κ

(87) kgk−1 k − 2κ

(88) ∆k−1 ) > κ ∆k−1 > 0.. JlB;HGGh5IY;t57;>[[G57U R JM>\gJ*I 8ha;HBLXLJrLOk*Jl \MeI 8d C Ê|8dÌ C Ê8a`SÌ9;HBLbÊ8harb,Ì? R ;H? 1.

(89)

(90)

(91) f (xk−1 ) − mk (xk−1 )

(92) κ

(93) ∆k−1

(94)

(95) 6 |˜ ρk − 1| =

(96) 6 1 − η˜2 .

(97) mk (xk−1 ) − mk (xk ) κ kgk−1 k − 2κ ∆k−1. Ê8har`,Ì Ê8harb,Ì Ê8 d>],Ì. U R J*\gJ Me\J C ρ˜ > η˜ ;HBLbÊ|d8 s Ì? R JBJB=O\Jr=? R ;H? Ê8a Ì R M>GL=8 2 kO\g\g J*?BVA?9=Ah??RJ\gJ*;H\g?JJ MeA\=<J}Bi>MHO?~;>B\g;>JBV;H\?J*k*Jr\LNAh??Ak*R ;>;2Gh?lG5>? 8yR U JKR ?A\g=OA=z=<?uxJ*\gBJ*=ieOAhMe\gJBFLX\4V;>L57AO? R =lJk*;>BBJ3B?<M>? ?R J4M>J\gJ*LI Jrk8 \gJ;e=@JrLFABLJBA?gJ*G5Ah? R J )* &¶S¹Nº:¶

(98) > B j3O[[Me=J? R ;H?08a38 R MeGL-8yjO[[Me=Jl O\@? R J*\gIKM>\gJ0? R ;2?<? R J*\gJJA=z?=;)k*M>B=@?g;>Be? =Ok R ? R ;2? kg k > κ Me\~;HGG k 8SU R JBX? R J*\gJlA=0;}k*M>B=@?g;>BV? κ ? =Ok R ? R ;H? κ Ê|8daEÌ ∆ > κ ? M>\~;>GhG k 8 k. . 2. k. . k. ¨º¤¹ ¹ C U R Jl[\gM3MHA=<? R J=g;HIKJ;>=9 M>\~U R J*M>\gJ*I s 8 8 }AB f eM BB J?0;>G|8ÊxdH]>]e],ÌJkJ[?? R ;2? . κ . (1 − η˜2 ) γ1 κ

(99) κ = min 1 − η1 , . (3 − 2˜ η2 ) κ

(100) . i>GM>46;>\gGVM>k*I M>B3R p>J*J\g\JieJ*M>BB k*C JS? MHR J? [R \JM3;HM>G

(101) i>Me Me\\Ah? R? R I J)?g4M0;>=A\gk=@?@?u \gM>O\=zL?nJ*\\Jki>\gAAhM>?BAk*;H;HGV[[[M>GAAJBe=?= Ah? R R J*MeB)O?A?yk ieR J*;HBBJie\g;HJ>?8J1=

(102) M>JBGh5l\=@?nBLA?gJJ*LGO5kIKJl;>? B3R 5J =@Ok*kJr==@ OGAh?J\g;H?AM>B=8 j3O[[Me=J? ;2?8a8 M>GL8j3O[[Me=J O\@? J*\gIKM>\gJ? ;H?0? J*\gJ);>\JMeBG5XBA?gJ*G5 IY)* &;HB3¶S5¹Nº=@:¶O

(103) kkJr>==@ OGVA?gJ*\;2?AM>B=*R 8U R J*B x = xR M>\ ;HGGV=O kAJ*BV?G5R G;H\gi>J k ;HBR L x R A=

(104) \=z?uxMe\gLJ*\k*\Ah?Ak*;>G|8 2. . k. ∗. ∗.

(105) `.

(106) ! " #$%&' )(*+ ,-(.)/.0. ¨º¤¹ ¹ C U R J=g;HIKJ;H\gi>OIKJBe?0;e=ABFU R J*Me\JI s 8 8 AB M>BB J?;HGx8 Ê|d>]>]e],ÌIY;t574J);H[[GhAJL =AhBkJ ? R J<\;>LAO=

(107) O[L;2?J<A=ALJBV?Ak*;HGV?gMn? R ;2? MH? R J<4;e=@AkS?g\ O=z? \fJi>AM>BIKJ*? R ML M>\ OB=Ok*k*J=g=z OGVAh?J\g;H?AM>B=8 2 6

(108) AB;>GhG5 ? J~BJ3?y?zM\gJ=OGh?g= JB=@O\J<? Ji>GM>4;>GkMeBVpeJ*\gi>JBkJ9MH? J~;>GhieM>\gA? I ?gMl\g=@?@u M>\LJ\ k*\Ah?Ak*;>G [MeAhBV?= C 435C = R R M2ABi7AB;7\=z?=@?J*[ ? R ;2?;2R ?lGhJr;>=@?MeBJK;>k*k*OIOG;H?AM>BR [M>ABV?MHSR ? R JKA?gJ*\;2?Jr=l=@JreOJ*Bk*J)A= \=z?uxMe\gLJ\9k*\Ah?Ak*;>G|8 )* &¶S¹Nº:¶

(109) > j3O[[Me=Jl? R ;H?~8a38 R MeGL-8SU R J*B M>BJ R ;>=9? R ;2? Ê8 dedÌ lim inf k∇ f (x )k = 0. . . x. k→∞. k. ¨º¤¹ ¹ C j3JJU R J*Me\JI s 8 8AB f M>BBJ?0;>G|8ÊxdH]e]>],Ì8 2 \=z?0uxMe=)\gL M>J\)\9k*? R\AhJ?Ak*4;>;eG|=@8 AkY?g\O=@?@u \gJ*i>AM>B IKJ*? R ML C ? R A=)k;HB$4JXJ*3?J*BLJL$?M = R M2 ? R ;2?Y;HGGGAI}Ah?K[MeAhBV?g=K;>\J )* &¶S¹Nº:¶

(110) > j3O[[Me=Jl? R ;H?~8a38 R MeGL-8SU R J*B M>BJ R ;>=9? R ;2? Ê8 d>,Ì lim k∇ f (x )k = 0. . . k→∞. x. k. ¨º¤¹ ¹ C j3JJU R J*Me\JI s 8 8 s A B f M>BBJ?0;>G|8ÊxdH]e]>],Ì8 . . 2. !"$#%'&( "#% 4/.1 4 +

(111) &8+#%9;:=<>+&8-. JYBM2 ABVpeJ=@?Aie;H?J}? JY[Me=g=Ah4AGA?z5N?gMNJ[GM>Ah?=@JrkMeBL3u M>\LJ*\AhB M>\gIK;H?AM>BM>B? JYM>4fÍzJrk?Ap>J} OBk?AM>B A? R ? R J;>AhI M>JB=O\RABiKkMeBVpeJ*\gi>JBkJ0?M=JkMeBL3u M>\LJ*\<k\gAh?Ak*;HG[M>ABV?g= C A|8 J>8y[M>ABVR ?g= x =Ok R ? R ;2? C ;HBL ∇ f (x ) A=[Me=A?gAhpeJ=@JI}ALJ*BAh?J . ∇ f (x ) = 0 m0 k*M>O\=J 9JBJ*JrLX?M7k*G;>\Ah 5 ;H?0J[\JrkA=@JGh5IKJr;HBN435Î=@JrkMeBL3u M>\LJ*\<AB M>\gIY;2?AM>BfÏ8 1J? R J*\gJ Me\J AhBV?g\MLOk*JC ? R Jl MeGhGM2AhBi};eLLAh?AM>BR ;>G-;>=g=@OIK[?gAhMeB=w U R JI}MLJGA=S;>=53I}[?MH?gAk;HGGh5)=@JrkM>BL3uxMe\gLJ*\ykM R J*\gJ*BV?yA? R ? R J~Me4fÍzJk?Ap>J9 OBk?AM>BYBJr;H\ \=@?@u M>\LJ*\ k*\Ah?Ak*;>G[MeAhBV?g= C Ax8 Je8 R J*BJ*p>J\ lim kg k = 0. lim k∇ f (x ) − ∇ m (x )k = 0 U R J 0J=g=@A;HBMH<J*peJ*\g5NIKM3LJ*G A=TAh[=gk R Ah?«¾YkMeBV?ABVOM>O= C ? R ;2?A= C ? R J*\gJYJA=z?=; kM>B=z?;HBV? κ m =Ok R ? R ;H? C Me\~;HGG k C k∇ m (x) − ∇ m (y)k 6 κ kx − yk Me\~;HGG x, y ∈ B 8 ? R J=@IY;HGGJ=@?<JAhieJ*B3p2;HGOJ MH? R J 0J=g=@A;HBMH

(112) ? R JIKMLJ*G m ;2? x A=<BJie;2?gAhpeJ C ? R J*B τ 1. ∗. x. xx. ∗. ∗. . k→∞. xx. k. xx. k. k. k→∞. . k. xx. k. xx. k. k. . k. k. mk (xk ) − mk (xk + sk ) > κ 7. Me\~=@MeI}Jk*M>B=@?g;>Be? κ 7 ∈ (0, ) 8 . 1 2. k. |τk | min(τk2 , ∆2k ). k.

(113) b.

(114) ! " #$%&' )(*+ ,-(.)/.0. U R J=J;>=g=@OI}[?AM>B=;>\JlALJ*BV?Ak*;>G ?MK? R Me=JlO=@JrLAhBFj3Jrk?AM>B= s 8);>BL s 8 s M> MeBB J?~;HGx8 ÊxdH]>]e],ÌS M>\<? R J 4 ;>=AkY?\gO=@?@u \Ji>AM>B IKJ? R ML8 B ;ek? C ? R J =@JrkMeBL3u M>\LJ*\}kM>B3peJ*\gi>J*BkJY[\gM>[J*f \?AJ=)MH? R JX\gJ?g\MV=@[Jk?Ap>J ?;H\gBOLX=@?@? u \JJ*i>A\~AM>[BK\gM3IKMHJ=~? R kM3;HL7B;HJG=g=@=@MlJBe?gO?gA\;>B7GhG57M>O4?J*?JMB 44J0MeJ*\\gM2;ek?gJrGhLY5) ? \gR M>JI =;>? IKAJ0=0;>k=S;>=? J R MV=@JnAh? MH ? ? R JJnJ4;>k=JAk<[??A\gM>OB =@?@M>u \JTi>J*AM>IKBKIYIK; J? R 8M38 L C8 R R C R R s 1 J? R J\J* M>\gJBJ*JL?gMY[\gJ=J*BV?0;K[\gMVM>M>

(115) ? R ;2?n[;H\?AkOG;H\\gJ=OGh? M>\? R JBJ IKJ*? R ML8<0=0JABLAk*;H?J 4JGhM2 BFC M>;HOG\GMH;H? BR ;>J*Gh\9MeiY\JrMH=@SOGhT-?g=JIKi>JIKB;J*\s;HG8A©¾8 J~ 9Ah? JR Me;eO==?9Ok IKR J;>B? i>R Jn;2?0;HB? RLYJ9IKJ0MMeLBJG5}G

(116) \gIKJLJOBe?gkAh?MeAM>BYBN? R A=~J*I Jp>J M>Be\?gO? ;HR GJlG5=g;|=É>AhieJ0BMHAhk*k;>G;HBe\g?nAh?zAh5>B 8 ? R JX=@JB=@J? R ;H?A?}A=);2?GJ;e=z?)MH<? R C JX=;>I}J7M>\LJ*\;e=? R JJ*\g\Me\4J*?zJJ*B ? R J7IKMLJ*G9;HBL? R J7Me4fÍzJk?Ap>J OBk?AM>B-8 1Jl? R JB= R M2#? R ;2?? R J?\gO=@?@u \Ji>AM>B7\;>LAhO=94Jk*M>IKJ=;e=@53IK[?M>?Ak*;>GhG5YAh\g\gJ*GJ*p2;HBV?<A

(117) ? R J=@?J[= ?JBLX?M¾*J\M8 V:¶

(118)

(119) ! > j3O[[MV=@Jl? R ;2?8a38 C ;HBL8 R M>GL89j3O[[MV=@J;>G=M}? R ;H?~? R J*\gJJA=@?g=;Y=JVOJBkJ ;>BL ;KkM>B=z?;HBV? κ 7 > 0 =Ok R ? R ;2? (k ) Ê8 d Ì m (x ) − m (x + s ) > κ 7 ks k > 0 M>\~;>GhG i =@O k*AhJBe?gGh57G;>\ieJ>8y6AhB;>GhG5 C =O[[Me=Jl? R ;2? . . . i. ki. ki. U R J*B Ah?J\g;H?AM>B k A=~=Ok*k*J=g=z OG-;>BL. ki. ki. ki. lim kski k = 0.. i→∞. i. M>\ i =@Ok*AhJBe?gGh57G;>\ieJ>8. 2. . ki. ρ˜ki +1 > η˜2. ;HBL. Ê|8d>Ì. ∆ki +1 > ∆ki. @=Öº¤k¹ kJr¹ =C =@ OG1 Me\Jni\g==@O?<k;>[A[J*BVG5?G57T-JGI};H\gIYi>Je; 8 s 8M28 C M>k* M>f BM>=ABLBXJ*\ J?~k ;HGxM>8´BÊ|Jld>]>=O]>]Sk RÌ ?gAhM}?J*L\;2Jr?gLAhOMek*B-Jn8? U R R ;2J?<J*JrpeVJ*O\g;25K?gAhAMe?gBJ*\=2;2Ê?AM>8 B Ì<k;HBAL = Ê8ÌAIK[Gh57? R ;H? M>\~=M>IKJ ξ ;HBL ζ AhB? R J=@Ji>IKJ*BV? [x , x ] C . i. i. ki +1. ki +1. ki. ki +1.

(120)

(121)

(122)

(123) f (xki ) − mki +1 (xki )

(124) |˜ ρki +1 − 1| =

(125)

(126) mki +1 (xki ) − mki +1 (xki +1 )

(127)

(128)

(129)

(130) hsk , ∇xx f (ξki +1 )ski i − hski , ∇xx mki +1 (ζki +1 )ski i

(131)

(132) =

(133)

(134) i − hski , gki +1 i + 12 hski , ∇xx mki +1 (ζki +1 )ski i

(135). Ê8 d s Ì R J*\gJ)9JK;HG=@MO=JLN? R J f ;HOk R 5Vuzjk R 9;>\«¾ABJVO;>GhAh?z5>8}:95F=@O4=z?gA?gO?ABi g = ∇ f (x ) Ê4J*u \k*J;HO=@\JXAh?@?gMHJ*nB ;>8= d,ÌAh? R A?=}J[\gJ=g=@AM>B$ABXÊ|8a>a¤Ì C ? R J LJ*BMeI}AB;H?M>\ D M>~? R JXG;2??J*\} \;>k?AM>B k;HB 4J ks k2 · k∇xx f (ξki +1 ) − ∇xx mki +1 (ζki +1 )k

(136) ki

(137)

(138) − hski , gki +1 i + 1 hski , ∇xx mki +1 (ζki +1 )ski i

(139). 6. 2. ki +1.

(140) Z

(141) D =

(142)

(143) − ski , gki +. U R J*B C \J[G;ekABi D. 1. . ∇xx f (xki + αski )ski dα +. 1 2. x. ki +1.

(144)

(145) hski , ∇xx mki +1 (ζki +1 )ski i

(146)

(147) .. 435Ah?g=J*[\Jr==AM>B7ABÊ8 `SÌ C 9JMe4?g;>AhB. 0. − hski , gki i

(148)

(149) =

(150)

(151) mki (xki ) − mki (xki +1 ) +. 1 2. hski , ∇xx mki (ψki )ski i. Z + 21 hski , ∇xx mki +1 (ζki +1 )ski i − ski ,. 1 0.

(152)

(153) ∇xx f (xki + αski )ski dα

(154)

(155).

(156) a].

(157) ! " #$%&' )(*+ ,-(.)/.0. M>\Y=@MeIKJ ψ AhB ? R JN=J*ieI}JBV? [x , x ] 8 U R J7?g\A;HBi>GJXABJVO;>GhAh?z5 C [\gM>[J*\?AJ=}MH~? R J AhBV?gJ*i>\;HG C Ê8d Ì C ;>BL f ;>Ok R 5Vujk R <;H\¾~ABJVO;>GhAh?z5i>Ap>J? R J*\gJ Me\J? R Jl M>GGhM2ABi}GhM29J*\<4M>OBLMeB D w ki. D. ki. ki +1. > |mki (xki ) − mki (xki +1 )|

(158) Z 1

(159) 1

(160) [∇xx f (xki + αski ) − ∇xx mki (ψki )]ski dα − 2

(161) s ki , 0

(162) Z 1

(163) + s ki , [∇xx f (xki + αski ) − ∇xx mki +1 (ζki +1 )]ski dα

(164)

(165) 0. > κ. 7 . 2. kski k − 21 kski k −. > kski k2 (κ 7 − 12 i ). R J*\gJ def. i =. 1 2 kski k. Z. 1. Z. k∇xx f (xki + αski ) − ∇xx mki (ψki )k · kski k dα. 0. Z. 1. k∇xx f (xki + αski ) − ∇xx mki +1 (ζki +1 )k · kski k dα. 0. 1. k∇xx f (xki +αski )−∇xx mki (ψki )k dα+. U R Jl?\gA;>Bi>GJAhBJrVO;HGA?z57BM2 AIK[GhAJ=? R ;H? 0. 1. Z. k∇xx f (xki +αski )−∇xx mki +1 (ζki +1 )k dα.. 0. k∇xx f (xki + αski ) − ∇xx mki (ψki )k 6 k∇xx f (xki + αski ) − ∇xx f (xki )k. ;HBL C =@AIKAhG;H\gGh5 C ? R ;H?. Ê8 dV^|Ì. + k∇xx f (xki ) − ∇xx mki (xki )k + k∇xx mki (xki ) − ∇xx mki (ψki )k. k∇xx f (xki + αski ) − ∇xx mki +1 (ζki +1 )k 6 k∇xx f (xki + αski ) − ∇xx f (xki +1 )k. j3AhBkJ9JBM2 M>4=@J\peJ0? R ;2?. + k∇xx f (xki +1 ) − ∇xx mki +1 (xki +1 )k + k∇xx mki +1 (xki +1 ) − ∇xx mki +1 (ζki +1 )k.. k(xki + αski ) − xki k 6 kski k, k(xki + αski ) − xki +1 k 6 kski k,. Ê8 d>`,Ì Ê8 d>b,Ì. kψki − xki k 6 kski k, kζki +1 − xki +1 k 6 kski k,. JlIY;t57LJrLOkJl? R ;2?~4MH? R. H; BL k∇ f (x + αs ) − ∇ m (ζ )k kM>B3peJ*\gi>JS?gM2¾*J\MA? R ks k 4Jrk*;>O=@J9? R J<\=z?;>BL? R A\gL)?J\IY= M>? R J\Ai R ?u R ;>BL=ALJMHÊ8dH`SÌ;HBL Ê8dHbSÌy?gJ*BLX?gM¾J*\gM4357k*M>BV?AB3OA?z5YM>? R JM>4fÍzJrk?gAhpeJ0 OBk?AM>B =;>BL7IKMLJG| = ~Jr==A;>B= ;>BLX4Jrk*;HO=@J ?R 8JY7IK;>BALLFLU GJY?JJM>\\gI J*I AB8 ? ^3R 8lJ70\A=li ;7R ?@k*u RM>B;>B=JLVO=AJLBJYkJ MH? R J=JYABJV O;>GhJ*AhB ?AJ=A=l;>G==O M kkM>AB3J*peBVJ*?\gG5Ni>JG=n;>\?gieM5J ¾J*;H\gC BM LN4? Jrk*J;H\OJ*=@ JKM>\gMHJ kM>I)4ABAhBivR Ê|8d s Ì;>BLbÊ8de^Ì C ;>BL7O=@ABiK? R C Jl ?\g6A;>Bκi> G7Jn ABJrR eO;HGiA?z5 C JlM>4?g;HAB C R C k∇xx f (xki + αski ) − ∇xx mki (ψki )k. xx. ki. ki. xx. ki +1. ki +1. ki. i. |˜ ρki +1 − 1| 6 6. . 2 k∇xx f (ξki +1 ) − ∇xx mki +1 (ζki +1 )k κ 7 2 k∇xx f (ξki +1 ) − ∇xx f (xki +1 )k κ 7 + k∇xx f (xki +1 ) − ∇xx mki +1 (xki +1 )k + k∇xx mki +1 (xki +1 ) − ∇xx mki +1 (ζki +1 )k. . Ê8 e],Ì.

(166) a>a.

(167) ! " #$%&' )(*+ ,-(.)/.0. : 5 ? R JX=g;HIKJ\gJ;>=M>BAhBiN;e= M>\Ê8dH`SÌx´Ê8dHbSÌ C ? R JX\Ai R ?@u R ;>BL =@ALJM> Ê8 >]SÌn?JBL=?M'¾J*\gMN R J*B ieMVJr=9?MABBA?z5 ;HBL ? J\J* M>\gJl?JBL=?gMFa>8 ?nA=<? R O=0G;>\ieJ*\<? R ;HB η˜ < 1 M>\ i =@Ok*AhJBe?gGh5 i G;>\ieJ;>BLJÊ8d>,ÌyC MeGhGM2~ρ˜=8 R 2 0=AB$T-JIKIK; 88 M> M>BB J*?);HGx8VÊ|d>]>]e],Ì JYIY;t5 ;H[[Gh5? A=\gJ=OGh??M ? JJ*BV?gAh\gJ7=JVOJ*BkJYMH A?gJ*\;2?gJ=y;HBL}LJLOk*Js ? R ;2?;HGGf Ah?J\g;H?AM>B=y;>\JJp>J*BVC ?gO;HGGh5)=@OkkJr==@ OG;>R BL)? R J?g\O=z?ux\gRJ*ieAhMeB\;>LAO= 4M>OBLJL ;t9;t65}\gM> \gI M>I0J*¾*\gJ*J\gMM>8 B ? J? JM>\g5XAB M>BB J*?;HGx8 ÊxdH]>]e],Ì9ieJ*BJ\g;>GhA¾*Jr=<A? M>O?=@Ai>BAhk;HBV?0k ;HBi>J 5VAJ*GLABi R R C f ? R Jl MeGhGM2R ABi)\gJ=OC Gh?gR=8 R ki +1. 2. j3O[[Me=J? ;2?8ha38 M>GLN;HBL? ;2? A=0;=O4=JVOJBkJMH ? J)A?gJ*\;2?gJ=ieJ*Bu *J)*\ &;2?g¶SJ¹NL º¶:43

(168) 5 0> Gh ieM>\gA? R I Z U Z R k*M>B3p>J\ieAhBi7R ?gM; \=z?uxMeR \gLJ*\x k\gAh?Ak*;HG[MeAhBV? x R J\JR ? R J 0J=g=@A;HB M> ? R J}M>4ÍzJk?gAhpeJ OBk?AM>B ∇ f (x ) A=0[Me=A?gAhpeJ)LJ*BAh?Je8j3O[[MV=@J O\@? R J*\gIKM>\gJ? R ;2? s 6= 0 Me\n;HGG =@AO Yk*AhJBV?G5XG;H\gi>Je8U R JB ? R Jk*M>IK[GJ?gJ=@JrVOJ*Bk*JMHyAh?J*\;2?gJ=~k*M>B3p>J\ieJ=?M x C ;HGGAh?J\g;H?AM>B=0;H\gJ kJ*peJ*BV?O;>GhG5peJ*\g5=@OkkJr==@ OG ;HBLX? Jl?\gO=z?ux\gJ*ieAhMeB7\;>LAO= A=4M>OBLJL;t<;t5K \MeI¾*J\M8 C R ∆ ki. ∗. xx. k. ∗. ∗. k. ¨º¤¹ ¹ C j3JJU R J*Me\JI s 88AB f M>BBJ?0;>G|8ÊxdH]e]>],Ì8 2 GhJr;>=@1 ?M>JlBBJM2=#Jk[Me\gBM3L3MHu M>?\R L;2J*?<\<Ak\gAh? ?R Ak*Jl;H=@G-Jr;eVk*Ok*J*OBIk*JnOGM>;2-?gAAh?gMeJ*BX\;2[?gM>JA=BV\?JAIK=;>ieAhOB;H=\A;HBXBV?g;}J*JkL-MeI}8 [;>k?<=@J*? C ? R J*BX? R JJA=z?gJ*Bk*J0MH;2? . j3O[[MV=@JF? ;H?X8a8 ^ M>GL#;HBL ? ;2? ;HGG0Ah?J\g;H?J=\JIK;>AhB AB =M>IKJFkMeI}[;>k? @= J*&?8¶S¹NU ºR ¶:J

(169) BF? > R J*\g. JJ*3A=@?g=l;2?GhJrR ;>=@?nM>BJ)GAhIKAh?R [M>ABV? x MHyR ? R JY=@JreOJ*Bk*JMHSAh?J\g;H?Jr= x [\gMLOk*JLN435 ~Gi>Me\Ah? R I Z U Z C R Ak R A==JkMeBL3u M>\LJ*\<k\gAh?Ak*;HGx8 )*. k. ∗. ¨º¤¹ ¹ C j3JJU R J*Me\JI s 8 s 8AB f M>BBJ?0;>G|8ÊxdH]e]>],Ì8 2 :5 ÍzO=@?0=z?g\JBiH? R J*BABi}? R J\;>LAhO=<O[L;H?J\gOGJ435X\JreOAh\gAhBi? R ;2? A ρ˜ > η˜ ;>BL ∆ 6 ∆ , ? R J*B ∆ ∈ [γ ∆ , γ ∆ ] Ê8 aEÌ GhAM>IK\A=@?~Me[I}M>J ABVγ?~MH>

(170) ? γ J>=JV1O;>JBBkLNJl=MHM> IKAh?J J*∆\;2?gJ=<ie>J*B0JC\g;H9?JJL7IK43M>57\gJ*~M2peGi>J*Me\\M>Ah? 4?gI ;HABNU? R J)8 =Jk*M>BL3u M>\LJ\0k\gA?gAk;HGA?z5MH;>B35 R R Z Z . k. 4. k. 2. 3. max. k+1. k. 3. 4. k. max. j3O[[MV=@J)? ;2?8a38^ ;HBLÊ8 aEÌ M>GL;HBLFGJ? 4J};>B35NGhAIKA?[M>ABe?lMH? R J =@Jr&VO¶SJ*¹NBºk*:¶ Jl

(171) MH > A?g2J*\ ;2?Jr=*8yU R JB Rx A=;Y=@JrkMeBC L3u M>\LJ*\<k\gA?gAkR ;HG[M>ABV?8 x. )*. ∗. ∗. ¨º¤¹ ¹ C j3J*JU R JM>\gJ*I s 8 s 8 `)AB f MeBBJ?0;HGx8 ÊxdH]>]e],Ì8 2 U O=y? J0\gJ?\gMe=[Jk?gAhpeJ<?\gO=@?@u \Ji>AM>BY;HGi>Me\Ah? I = ;H\gJ=S;>GhG? JÊ ABe?gJ*\gJ=@?ABiSÌSkM>B3peJ*\gi>J*BkJ<[\gM>[J*\?AJ= MH? R JR 4;>=AR kn?g\O=@?@u \gJ*i>AM>BIKJ? R MLXOBLJ*\0? R J=g;HR IKJ;eR ==OIK[?AM>BR =*81JkMeBkGOLJl? R A=? R J*M>\g57=Jk?AM>B 435 BÊ [MHMe?gAh=gB=@Ai)4G? 5 R ;2A??gJ*? \R ;2J?AM>;H4B M2Lp>J*Jl[kJ*MeBB3Lp>J*J*BV\g?i>Ì)JBOkBJAh \gM>J\g=IKOGGh?5 =Jr;HV\gOJAh=@p2?;>AGhGJG-Bep2?};HGBALM>\gAhI MeB? JJ*\\gJJ*[43G;>5 kJr;H=GGhM2? RJ ABv iFO[kG\gAM>L4JrGh;HJB I BMe=gk*\;>I GhAB43i5;H;HBB3L5 C R [\gJkMeBLAh?AM>BABi8 .

(172) ard.

(173) ! " #$%&' )(*+ ,-(.)/.0. Ã 3È È3Á,?2Ã ÁÆ C 6ÃÈ3Ç3? AN6S8:6ÃÈ6Á5Ç36 1 JBM2k*M>B=ALJ*\ ? R J~B3OIKJ*\gAk;HG4J R ;tp3AM>O\yM>? R J~BJ;>GhieM>\gA? R I C AhBk*M>IK[;>\A=@MeBAh? R ? R J4;>=Ak9?\gO=z?u ?\RJi>J7AM>a B s ;>OGhieBM>k\gMeA? BR =zI?g\g;>:<AhBU JLZ [Ê\g=M>J*4JGJ*[IY;>i>=JM>a>a? Rs J MH f Me BB

(174) Sk*J*M>?KGGhJr;>kG|?8AM>ÊxdHB ]>]eÊ|=@],JÌ«JÌ8 M>1OJXGL ?gC Jmn=@?}\44;HMHBN? R ;HB;>LNGhieU

(175) M>\gM>AA? BVR ? IYC =dH]>Me]eB ,Ì;>8GhG<6MeMH\ ? R J[\Me4GhJIY= R Me=JLAI}JB=AhMeBIY;t5" 4J+3 k R>;> + B i>JrL C 9Jk R Me=J;K\Jr;>=M>B;>4Gh5=IY;HGG-p2;HGOJABMe\gLJ*\BM>??M M2p>J\GMe;eL? R J

(176) 9AhBV?gJ*\;>kJ}Ah? R 8KU R JY=z?;H\?ABi[MeAhBV?g=;>\J)? R JK=@?g;>BL;>\gLM>BJr=l[\M2p3ALJL 435Y? R J

(177)

(178) GAh4\;H\g5>8 6M>\7? R J 4;>=AkF;HGi>Me\Ah? R I ? R JF?\gO=@?@u \Ji>AM>B \;>LAO=O[L;H?JF<;>=7AhIK[GJ*IKJBe?gJL#435O=AhBi$? R J \OGJ [\gM>[Me=JLAB f M>BBJ?0;>G|8 Ê|dH]e]>C ]SÌ C [-8^H`>w  A ρ > η C max[γ ks k, ∆ ]    A ρ ∈ [η , η ) C ∆ ∆ = A ρ ∈ [0, η ) C  γ ks k   A ρ < 0 C  min[γ ks k, max[γ , θ ]∆ ] R J*\gJ γ A=JL ;H?]8 ] s de C γ ;H? 0.25 C γ ;H? 2.5 C η ;H? 0.05 ;>BL η ;2? 0.9 ;HBL R J\J θ A=i>Ap>J*B 435Ê|d8 d,Ì8SUM};tp>MeALY4A;e=@ABi? JlkM>IK[;>\A=M>B J ;tp>JLJk*ALJLY?gM)IY;É>Jn;e= J* ;eL;H[?;2?gAhMeB=;e=[Me=g=Ah4GJ ?OMB=? OR k*;2k*?YJ=g\=zO OGhJ G AAxB8 J>8MeO\K\J*?\gMe=[RJrk?gAhpeJXp2;H\gA;HBV?vC ÊA|8 J>8 R jV?gJ*[#d AhB#~Gi>Me\Ah? R I d8aEÌ8 U R O= C AhnAh?J\g;H?AM>B k A= ;>BL kMeB=@JrVOJ*BV?G5 9J;HG=MKLJk\gJ;e=@J? R J?\gO=@?@u \Ji>AM>BXO=@ABiK? R J ;H4M2p>J)\OGJ>8C C M>Bρ? R <JKkηMeBe?g\g;>\5 C A?gJ*\;2?AM>B k xA=l=@=OkxkJr==@ OC G C Ax8 J>8 ρ > η C ? R J)?\gO=@?@u \Ji>AM>BFA=O[ L;2?gJL dlMH-;H~BGLi>Me\Ah? R I d8ha~ R J*8

(179) \gU JJ0J0IKk R MMVLMVJ*=@GJ<9? ;eR =Jnk =g;HMVIK=@JJ~B p2;>AGhBYOJr4=M>?;e=S;Hk;>4=M2Jp>= J OM>=\ AhBγiC? RγJ~[;>\BML kJrγLOC \g;>J0BL}LJr?g=;k*É>\JA4η˜JrLK=AhB7ηj3?=J*[0.05 ?? MNJ4?JK\gO? R=z?J7ux\gJJ*ie;>AhMekB?U

(180) <;t;>53=GM>k*\rM> =lIK=[J*\gOAh?gJrJ=L ?\gJOA? Bk*J*;H\)?JrJL;>?gkMF?G=@5Jrη˜kOMe==BAhBL3iηu M>? \=LJJ*0.9 \ ENC ;HM>B\gLquzR j3? RM>\gJJ*BIK=@AhJBB AhIK;HAG¾*i>J*Me\\R AhMH? 9I ? R CJYÊ=J*IKJ7MEFLJ*M>R G\gAhqB=@;HABLL J C j3R M>\gJ*B=@JB C abe`>SÌ0C Me\;H[[\MtAIK;H?JGh5NOR =ABi? R JXj3?J*A R ;>OiHuRUMeAhBV?;HGi>Me\Ah? R I Ê=J*JjV?gJ*A RR ;HOi C arb>`> C U

(181) M>ABe? C abeàEÌ8K B ? A=k*;e=@J ? Jk*M>BÍzOie;2?gJKi>\;>LAJ*BV?A?gJ*\;2?gAhMeB=;H\gJ}=@?Me[[JLA<? R JK?\gO=@?@u \Ji>AM>B4M>OBL;H\g5FA= IKJ?~M>\;e==@M3R M>B ;>=9? R C JR IKMLJ*G=*i>\;>LAJ*BV?=;H?A=zJr=9? R Jk*M>BLA?gAhMeB 6A %C. 2. k+1. 1. k. k. 0. 1. k. k. 0. 2. k. 1. 2. 1. 1. 1. 2. 1. k. 2. k+1. k. 0. 2. k. 1. k. 1. k. k. k. 1. k. k. k. 2. 1. 2. @= IYJ<;>kGhGMeJ*B\=?ALR J;H\B JrLl10? R ;2?8F? R6J9;>AhAG?gOJ*\g\J;2?g<Ahpe;>J=[L\gJrMkkGJ;H=g\g=Jk*LM>B3Ahp>J? R\ieJJLl;HGi>M>R \gJAh?BR ? I R J Lv AL OkBGAMHL?}J;Hk*B)M>B3Bp>M>J\g\I ieJ)MH? Ah?R R J9AhBi>\;>? R LJ7AJ*BVIY?

(182) ;H43JAkI;HOIKI J B3OI4J*\M>y>]]e]>])A?gJ*\;2?gAhMeB=*8 M>\B31 OIJ4k R J*\MV=@MHJS?g MOBkkM>?gIKAhMe[B;>\J*Jnp2;>? RGhOJ;H?BVAOM>IB=48lJ*\~ BM>L

(183) JAhJ?L J*\C ;2? ?gR AhMeJ}Bk*=9Me?=@M?n[;eJ*k R\lAAhJ*?p>J*Jl\;2k?gMeAhMeB3BFp>J*A\g=~i>J? RBJYkJn=Ah;>BI}=z?gJJ;> M>L\M>4

(184) M>? ? R R J ;Hf Gi>QM>o \gAh? ?R AIKIYJ = ;HBL)? R J*5}4M>? R Jp2;HGO;2?gJ<? R J~Me4fÍzJk?Ap>J< OBk?A" M>BY+3 MeeB+ kJ [J*\A?gJ*\;2?gAhMeBK;>BLYkMeIK[O?J0M>BJie\g;eLAhJBV?y;H?J*peJ*\g5 =@Ok*kJr==@ OGAh?J\g;H?AM>B-8yENMe\JM2p>J*\ C ?AI}ABiV=AhB ;H\gJlMH,?JBNLA k*OG??gMYAhBV?gJ*\g[\J*?8 ~GG\gOB=9J*\gJl[J\@ Me\IKJLXAB E;2?G;H4 p 8V^8hae8 ]8har`>ËÊ Z a Ìj3J*\gp3Ak*J Q ;>kÉ7KMeB;Y8 dK R ¾ BV?gJ*G=AhBi>GJu [kMe\g\MHJG[J\MMHk*J? =gR =@J0Me\BVOk*M>IIK4[J*\O?gM>J*\ Ah?JA\g? ;HR ?AdM>Bl=S: MHM>? R Z J0?z9E$M8H6

(185) ;HGAi>i>MeO\\AhJ ? R IK8a=\g8 J*Q [J\Jr\@=@MeJ\BeIY?=-;H? BR kJJ<k[Me\gI}MH[G;HJ\g=yA=ieM>AhpeBJ 43C 5 M>;n\S[Jp>J\@J*\gMe5 \IYσ ;>>Bk1J C ? R J[\gM>[M>\?AM>B p(σ) MH3?gJ=@?

(186) [\gM>4GJ*IY=M>B R Ak R J;ek R kMeB=@ALJ\JrL;HGi>M>\gAh? R IKAk pt;>\A;HBV? R ;>=;~[J\@ Me\IY;HBkJ ?R AA? =XR AB i>O;\J;eC k?gM>J \ R σ;tp>MHJM>? RBJG5 4É>JrJ*=z[??7Ê|=@? JR JFJ W0[Me\gM>G;>4B GhJIK;>B=7L M>EN\M> \gq R C Ak dHR ]e]e4d MHC ? R M>\7;HG; i>MeIK\AhMe? R \JNIYk=XMekIKM>[B3GhpeJ*J*?\gJFi>JLL A=gk?gOM =g=@? AR M>J BAÌ=g8 ;HIK B J Gh%!M#%k;HG3=8(M>9GO% ?gAhMe B'%'Ê 94J~J kGO'L5Jr% L;HB <L C 3=!,#"($ %'&($ C Ì)8* + ,? %-.JF =@OC 4/0[1\Me24)G* J*I 3 C A0=Y4=@Me*Ghpe+ J5L " ;H5C [6[1 \g" Mt1 'AhIY7 C ;H%!?J*4%G5 ) 348(M>9.? : C ;HGi>Me) \Ah? R IY=C ;H1 AGJL M>B?;C > 2 +0(61 2 A@ )C 4"%$ C 1) +6!5% 1 C R ) 9' 0 "@5B C ) 9' 0 ":B};HBL? ) 9' 0 "'7B8 EFM>\gC J*M2p>J\R C U ;HAGhJrLKM>B 37 Ak <;>=y=M>Gp>JL}435K:9U 83mnB}? J0MH? J*\ ;HBL A? J0=O4[\gM>4GhJI A==M>Gp>JL JZ ;>Z k?G5 C 4MH? R ;H/Gi>0(M>1\g2Ah? ) R IY=

(187) C ;HAGhR JrLR MeB /012) 3'7;HBL}:<Z U Z ;>AhGJR L}MeB R ) R !5% 1 C C R R Ak R 9;e=y=@MeGhpeJL4V5 Z U Z 8 M>?J7;>G=M ? R J7BVOI4J*\M>A?gJ*\;2?AM>B=BJ*JrLJL?gMF\gJ;>k R kMeB3p>J*\gi>JBkJKAh? R ? R J Z U Z ;HGi>Me\Ah? R I M>B? R J RkMeAi I}R [G5GhJ*B?JlMeBB3kOMeIKBVpeJ*\gJA k;H* G9 \g6 J>'=}OG;H?=BL;H\g0 Jn(ieAh*pe+J*B5Ah")BN[0\[Me[4JGhJBIYLAh=A=<I8 Ok RXR Ai R J*\9? R ;>B7 Me\<? R J:<U Z ;HGi>M>\gAh? R I 8yU R J k∇x mk (xk + s)k 6 min[0.1, k∇x mk (xk )k1/2 ]k∇x mk (xk )k.. 1. −5.

(188) a.

(189) ! " #$%&' )(*+ ,-(.)/.0 RTR vs BTR 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2. MS−BTR MS−RTR TCG−BTR TCG−RTR. 0.1 0. 1. 1.5. 2. 2.5. 3. 3.5. 4. 4.5. 5. 6AhieO\J 8a>w Q J*\ M>\gIY;HBk*JS[\gMHGJk*M>IK[;H\gABi? R JB3OI)4J\

(190) M>A?gJ*\;2?AM>B=-MH? R J Z U Z ;HBL:9U Z ;HGi>M>\gAh? R IY= mnO\l\Jr=@OGh?g== R M2 ? R ;2?l? R JK\J*?\gMe=[Jrk?Ap>J};HGi>Me\Ah? R I [J*\ M>\gIY=l;>=JGhGS;e=n? R JYkG;>=g=Ak;HG M>BJK;HBLA= ÍzO=@?9;>=\JGhA;H4GhJ~Ah? R J~?\gO=@?@u \Ji>AM>BY=O4[\gM>4GJ*I A==M>Gp>JrL};>[[\gMtAhIY;2?gJ*G5>8~M29J*p>J\ C A? R J0[\Me4GhJI =A¾*J M>\9=z?g\Ok?O\gJ0;>GhGM2~=S;HBYJ*;>k?9=@MeGhO?gAhMeB C ? R Jn\J*?\gMe=[Jrk?Ap>J~;>GhieM>\gA? R I A=S? R JB7=@Ai>BAk;HBV?G5}I}Me\J0<J kAJ*BV? Ê,? J~AI}[\M2peJ*IKJ*BV?SA=y?z53[Ak*;HGG5MHMeBGh5}; J* Ah?J*\;2?gAhMeB= 4O?9A=Sp>J*\g5)k*M>B=A=z?gJ*BV?Ì ;>BL9ÍzO=@?9;>=S\JGhA;H4GJ>8 LJR ?;HAGhJrL);>B;HG5=@A=MHM>O\y\gJ=OGh?g= = R M2~=? R ;H? Z U Z A= AhB}i>C JBJ*\;HG=GhAi R ?G5IKM>\gJ<kM>B=@J\p2;2?gAhpeJ? R ;HBK:<U Z AhB ?BR J*;HiV?y;2?gA?Ahpe?JJB<L;t=5>8-?gM0 B}?;|[É>;HJ9\?IYAk;HO\gGi>;>A\B?;HGGJ5lG=M>R BMei>\@J*?g\ J*\=@?=zJ*?g[J*=

(191) [M>=8:<0UM2Jp>M>J,?\ JC B? R \gAJ=y=OLGM3? JrA=

(192) B}B;0MH?SG;>=@\JieJ*J*I \[?g\MlMe;>[GMe?g\@J*?g\ AhMe[BJ*\MH M>\gOIKB=;>OBk*kk*JJAh=@BK O; G A?gJ*\;2?gAhMeB=Ê ? R A=<IY;t5K4JlLJLOR kJrL \gM>I ? R J\gJ=OGh??;HZ 4GJl=@ABkJn? R JnB3OI)4J\9M>OB=OkkJ=g=@ OG A?gJ*\;2?AM>B=A= i>1 Ap>JXJBX;HG=@43M 5B? RM>?JJYL? AhRP ;2J*\g?J*?BR k*J7Jk4R J*Me?zAk*JJKJ*BMH~? ;HR B JBV;>OkkIO4\;2J*\~?gJKM>I}AhA?BJA\gIK;H?A©A¾rM>;2B?A=M>;>B BM>LX ? R JJ*<BV?gOM>IB-4 =J*\~IKM>MLieJ*\gG;eAhLB AhJ? BVR ?<JYJ?gp2\;HOG=@O?;2?g\gAhJ*Mei>BAM>=ÌB 8 ;HG=@MN;>[[Jr;H\=?MF4JkMeB=ALJ\g;>4G5FIKM>\gJY<J kAJ*BV?? R ;HB$;HB ;>[[\gMt3AIY;2?gJ}MeBJ ;2?GJ;e=z?AB ?J\IY=M>? R J B3kMeOBI=@4JrVJ*O\ J*MHBk*AJ?gJ*\? ;2?gJ~AhMe\gBJ?=\gBMeJ*=Jr[LJJkL?gAhpe M>J\yp2k;>Me\B3A;Hp>BVJ*?y\gi>A=yJBk*kGhJrJ ;HC \gAhGh\g5)\Jr;2=@?y[AJ?k=y?A4p>JJ9=@?SMH ? R J*J~B}k R? MeJ~Ak*kJMe4=@J?S?zMH9C J*J*Jp2B);HG:<OU ;H?Z AB;>iB? L J~Z MeU 4fZ ÍzJ8Hk0?A=yp>;J AhBOLBAkk?;2AM>?gJ~BY? ;HB;2L}?<C ie4\gR M>;e? LAhJ;>BVGh?yieM>L\gM>AIK? AhIYB;2=?;HJr\g=J~? RJr=;H=?J*MHBV?A? ;HR GGJ05)M2Op>BJ*\L;HA=@GG3?AAB?gJ*ie\O;2A?g= AhMe;>B-4R8

(193) GhJnLLR AJ*?gBAhMeBVO;HG;>?g=AJu =@?SJB<MH??SMe\gBJ*[;HM>[\[?R \gJrMtL R AhIYJ*\g;HJ~?A;HM>GB=@M = R R R R R Ê|j Z aNM>\Y:<6