A note on the convergence of barrier algorithms to second-order necessary points

(1)

RESEARCH OUTPUTS / RÉSULTATS DE RECHERCHE

Author(s) - Auteur(s) :

Publication date - Date de publication :

Permanent link - Permalien :

Rights / License - Licence de droit d’auteur :

Bibliothèque Universitaire Moretus Plantin

Institutional Repository - Research Portal

Dépôt Institutionnel - Portail de la Recherche

researchportal.unamur.be

University of Namur

A note on the convergence of barrier algorithms to second-order necessary points

Gould, Nick; Toint, Philippe

Published in:

Mathematical Programming Series B

DOI:

10.1007/s10107980030a

Publication date:

1999

Document Version

Early version, also known as pre-print

Link to publication

Citation for pulished version (HARVARD):

Gould, N & Toint, P 1999, 'A note on the convergence of barrier algorithms to second-order necessary points',

Mathematical Programming Series B, vol. 85, no. 2, pp. 433-438. https://doi.org/10.1007/s10107980030a

General rights

Copyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners

and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.

• Users may download and print one copy of any publication from the public portal for the purpose of private study or research.

• You may not further distribute the material or use it for any profit-making activity or commercial gain

• You may freely distribute the URL identifying the publication in the public portal ?

Take down policy

If you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediately

and investigate your claim.

(2)

optimization algorithms using the log-barrier fun tion

byA.R. Conn 1 ,N.Gould 2 andPh.L.Toint 3 Report 97/15 7 September2000 1

IBM T.J.Watson Resear h Center, P.O.Box218, YorktownHeights, NY 10598, USA Email: ar onnwatson.ibm. om

2

RutherfordAppletonLaboratory, Chilton,Oxfordshire,England Email: N.Gouldletterbox.rl.a .uk

Currentreportsavailablebyanonymous ftpfrom thedire tory \pub/reports" on joyous-gard. .rl.a .uk

3

Department of Mathemati s,Fa ultesUniversitairesND de laPaix, 61,ruede Bruxelles, B-5000 Namur, Belgium,EU

Email: phtmath.fundp.a .be

Currentreportsavailablebyanonymous ftpfrom thedire tory \pub/reports" on thales.math.fundp.a .be

(3)

algorithms using barrier fun tions

Ni holas I. M. Gould and PhilippeL. Toint

January8, 2003

Abstra t

Ithaslongbeenknownthatbarrieralgorithmsfor onstrainedoptimization anprodu ea sequen eofiterates onvergingtoa riti alpointsatisfyingweakse ond-orderne essary opti-mality onditions,whentheirinneriterationsensuresthatse ond-orderne essary onditions holdatea hbarrierminimizer. Weshowthat,despitethis,strongse ond-orderne essary on-ditionsmayfailtobeattainedatthelimit,evenifthebarrierminimizerssatisfyse ond-order suÆ ientoptimality onditions.

1 Introdu tion

We onsiderthe onstrained optimizationproblem

minimizef(x) (1.1)

subje tto

i

(x)0 forall i2I; (1.2)

wheref and the i

mapR n

into Rand I isa nitesetof indi es. We assumethatf(x)and the

i

(x) aretwi e ontinuously dierentiable on anopen set ontaining

F =fx2R n

j i

(x)0 forall i2Ig:

Our prin ipalinterestis inidentifyingnonlinearprogrammingmethodswhi h,underreasonable assumptions, are apable of ensuring onvergen e to points at whi h se ond-order ne essary optimality onditions are satised. When the problem is un onstrained, it is well known that a number of optimization te hniques (prin ipallytrust-region-, but also linesear h-, based, see More, 1983, Shultz,S hnabeland Byrd, 1985, M Cormi k, 1977, andMore and Sorensen,1979) are apableof guaranteeing onvergen e to se ond-order points. The diÆ ultywhen onstraints arepresentisthatthese ond-order onditionsarenotexpressibleina omputationally onvenient form. Indeed,evenestablishingthatthe onditionsaresatisedis,ingeneral,anNP-hardproblem (see Murtyand Kabadi,1987, and Vavasis, 1992).

Let `(x;y) bethe Lagrangianfun tion

`(x;y)=f(x) X i2I y i i (x): (1.3)

(4)

Under suitable onstraint quali ations(see Gouldand Tolle,1972, Mangasarian,1979, and the papersquotedtherein),itiswellknownthata(lo al)solutionx

of(1.1){(1.2), togetherwithan asso iatedsetofLagrangemultipliersy

,satisestherst-order(Karush-Kuhn-Tu ker)ne essary onditions r x `(x ;y ) = 0 (1.4) i (x )0 and (y ) i 0 forall i2I (1.5) and i (x )(y ) i = 0 forall i2I; (1.6)

aswellas thestrongse ond-order ne essary ondition

s T r xx `(x

;y)s0 forall s2N + ; (1.7) where N + = ( s2R n s T r x i (x

)=0 forall i2fj2A(x )j(y ) j >0g and s T r x i (x

)0 forall i2fj2A(x )j(y ) j =0g ) ; (1.8) and A(x)=fi2I j i (x)=0g

is the a tive set at x. The se ond-order ne essary onditions given here are those given by Flet her (1981 Se tion 9.3). Signi antly weaker onditions are given by, for instan e, Fia o and M Cormi k (1968 Se tion 2.2) and Gill, Murray and Wright (1981 Se tion 3.4), whi h are equivalenttorequiringthatthesolutionatthe onstrainedminimaunder onsiderationisstri tly omplementary,thatis fi2A(x )j(y ) i =0g = ;; (1.9)

and thusthat

N + =N def = n s2R n s T r x i (x

)=0 forall i2A(x

) o

: (1.10)

While su h an assumption is realisti for linear programming,|all linear programs have su h solutions (see,Wright,1997, page28), and manyinterior-point methodsndone|it frequently doesnotholdfornonlinearprograms. Ontheotherhand,theadvantageofrequiring(1.9)isthat the se ond-orderoptimality onditions redu e to he kingthat theHessian of theLagrangian is positive (semi-) denite on themanifold (1.10) ratherthan in the one(1.8). We shall all the requirement that s T r xx `(x

;y)s0 forall s2N (1.11)

aweakse ond-orderne essary ondition. That(1.11)isweakerthan(1.7)is learon eonerealizes that theweak onditionis satisedbythemaximizerof thequadrati programmingproblem

min x2R n x0 kxk 2 2 ;

while (1.4){(1.6) and the strong ondition are together both ne essary and suÆ ient for lo al optimalityof generalquadrati programs(see Contesse, 1980, andBorwein,1982).

A number of algorithms for solving (1.1){(1.2) have been shown to onverge to points at whi h the weak se ond-order ne essary onditions hold (see, for example, Gay, 1984, Bannert,

(5)

1994, BonnansandLaunay, 1995, Fa hineiand Lu idi,1996, and Vi ente, 1995). Inparti ular, Auslander (1979) has shown that that, if one tra es the traje tory of points at whi h se ond-order ne essary onditionshold forthebarrierfun tion|su hpointsmaybefoundbyapplying trust-region orline-sear hmethodsto theun onstrained barrierproblems|then thelimitpoint will satisfy the weak se ond-order onditions for (1.1){(1.2). However, to our knowledge, no algorithm has been shown to onverge to a point at whi h the strong onditions hold. In this paper,we askthenaturalquestionasto whetherinterior-point(or,spe i ally,barrier)methods might do so. It is our purpose to show that, in general, the limitof this barriertraje tory may failto satisfythestrongse ond-order ne essary onditions.

2 A simple ounter-example

We shall onsiderthelogarithmi barrierfun tion

b 0 (x;)=f(x) X i2I log i (x);

and there ipro albarrierfun tions

b (x;)=f(x)+ X i2I 1 ( i (x)) ; (2.1) for>0. (1)

Thesefun tionsdependonthebarrierparameter>0. Inatypi albarriermethod, (approximate) stationary points of the barrier fun tions are tra ed as the barrierparameter is redu edtozero,and,underreasonableassumptions,thisleadsto onvergen etoa Karush-Kuhn-Tu ker point.

The exampleweshall exhibitisa bound- onstrainedquadrati programof theform

min x2R n x0 1 2 x T Hx; (2.2)

whereH is asymmetri ,indenitennmatrix. Forfuture referen e,when (1.1){(1.2)isof the form (2.2), therstand se ond derivativesofthebarrierfun tions above aregiven by

r x b (x;)=r x f(x) X (+1) e (2.3) and r xx b (x;)=r xx f(x)+(+1)X (+2) ; forall0,whereeistheve torofallonesandwhereX=diag (x

1 ;:::;x

n

). Wealsonotethat

r xx

`(x;y)=H (2.4)

be ause of(1.3).

We now hooseasequen e f k

gofbarrierparameters onverging tozero andwedeneH to beof theform H =I (+ 3 2 ) zz T kzk 2 2 (2.5) (1)

Thes alingfa torin(2.1)is,perhaps,nonstandard,butmayeasilybeassimilatedintothebarrierparameter. Thisallowsforauniformtreatmentofbothbarrierfun tions.

(6)

where I is the identity matrix and where we have hosen z = e ne 1

; the ve tor e 1

being the rst ve tor of the anoni al basis. Wethen verifythat

z T e = e T e ne T 1 e=n n=0; (2.6) z T e 1 = e T e 1 ne T 1 e 1 =1 n (2.7) and kzk 2 2 =e T e+n 2 e T 1 e 1 2ne T 1 e=n+n 2 2n=n(n 1): (2.8)

The denition(2.5) and(2.6) together implythat

He=e: (2.9) Nowlet x k = 1 +2 k e: (2.10)

We then verifythat x k

isa minimizerof theproblem

min x2R n x0 b (x; k ) (2.11)

thatsatisesse ond-ordersuÆ ientoptimality onditionsforthisproblem. Indeed,therst-order optimality onditionholdssin e

r x b (x k ; k )=Hx k X (+1) k e= 1 +2 k e 1 +1 +2 k e= 1 +2 k (e e)=0;

wherewe used(2.3), (2.9) and(2.10), and wehave also that

r xx b (x k ; k )=H+ k (+1)X (+2) k = 1 2 I+(+ 3 2 ) I zz T kzk 2 2 !

isobviouslypositivedenitesin ethersttermofthelastright-handsideispositivedeniteand the lastterm inbra ketsis an orthogonal proje tor, whi h is thereforepositivesemidenite. As expe ted,fx

k

g onverges to zero, the only riti alpoint of problem(2.2). However, using(2.5), (2.7) and (2.8), we ndthat e T 1 r xx `(x;y)e 1 =e T 1 He 1 =1 (+ 3 2 ) (e T 1 z) 2 kzk 2 2 =1 (+ 3 2 ) (n 1) 2 n(n 1) = n (+ 3 2 )(n 1) n ;

whi h isstri tlynegative forall values ofnsatisfyingthe inequality

n> + 3 2 + 1 2 : But e 1 belong toN + =fx 2R n

jx0g, andthusthestrongse ond-orderne essary onditions do notholdat the origin.

(7)

3 Con lusion

We have shownthat thestrongse ond-order ne essary optimality onditionsfor inequality on-strainedproblemsmaynotholdatlimitpointsofasequen eofbarrierminimizers,evenifea hof these minimizers satisesthe se ond-order suÆ ient onditionsfor un onstrained minimization. This negative on lusionis validfora large lassof barrierfun tions,in ludingthepopular log-and re ipro albarriers.

Thisresult astsdoubtsonthepossibilityofobtainingstrongse ond-order onvergen e prop-erties for a numberof pra ti al interior-point methods fornonlinear programming. However, it also raises the intriging question of determining ifthere are barrier fun tions, outside the lass onsidered here, for whi h the desired strong se ond-order onvergen e properties are satised. Moregenerally,thequestionof whetherthere areee tive methodswhi hensure onvergen eto stong se ond-order pointsremainsopen.

A knowledgments

The authors are grateful to A. R. Conn for his areful omments on the manus ript and to F. Jarreforhis interest.

Referen es

A.Auslander. Penaltymethodsfor omputingpointsthatsatisfyse ond orderne essarypoints. Mathemati al Programming, 17(2), 229{238, 1979.

T. Bannert. A trustregion algorithmfor nonsmoothoptimization. Mathemati al Programming, 67(2),247{264, 1994.

J. F. Bonnans and G. Launay. Sequential quadrati -programmingwith penalization of the dis-pla ement. SIAM Journal on Optimization,5(4), 792{812, 1995.

J.M.Borwein.Ne essaryandsuÆ ient onditionsforquadrati minimality.Numeri alFun tional Analysisand Optimization, 5,127{140, 1982.

L.B.Contesse. Une ara terisation ompletedesminimalo auxenprogrammationquadratique. Numeris heMathematik, 34, 315{332, 1980.

F. Fa hinei and S. Lu idi. Convergen e to se ond order stationary points in inequality on-strainedoptimization. Te hni al Report 32-96, University of Rome "La Sapienza", Rome, 1996.

A. V. Fia o and G. P. M Cormi k. Nonlinear Programming: Sequential Un onstrained Mini-mization Te hniques. J.Wiley and Sons, Chi hester, England, 1968. Reprintedas Classi s in Applied Mathemati s4, SIAM,1990.

R. Flet her. Pra ti al Methods of Optimization: Constrained Optimization. J. Wiley and Sons, Chi hester, England,1981.

(8)

D.M. Gay. Atrustregionapproa h tolinearly onstrainedoptimization. inD. F.GriÆths,ed., `Numeri alAnalysis: Pro eedingsDundee1983', pp.72{105, Heidelberg,Berlin,New York, 1984.Springer Verlag. Le ture Notes inMathemati s 1066.

P. E. Gill, W. Murray, and M. H. Wright. Pra ti al Optimization. A ademi Press, London, 1981.

F. J. Gould and J. W. Tolle. Geometry of optimality onditions and onstraint quali ations. Mathemati al Programming, 2(1), 1{18,1972.

O. L.Mangasarian. Nonlinear Programming. M Graw-Hill, NewYork, USA,1979.

G.P.M Cormi k. Amodi ationofArmijo'sstep-sizerulefornegative urvature. Mathemati al Programming, 13(1), 111{115, 1977.

J. J. More. Re ent developments in algorithms and software for trust region methods. in A. Ba hem, M. Grots hel and B. Korte, eds, `Mathemati al Programming: The State of theArt', pp.258{287, Heidelberg,Berlin, New York, 1983.Springer Verlag.

J. J. More and D. C. Sorensen. On the use of dire tions of negative urvature in a modied Newtonmethod. Mathemati al Programming, 16(1), 1{20,1979.

K. G. Murty and S. N. Kabadi. Some NP- omplete problems in quadrati and nonlinear pro-gramming. Mathemati al Programming, 39(2), 117{129, 1987.

G. A. Shultz, R. B. S hnabel, and R. H. Byrd. A family of trust-region-based algorithms for un onstrained minimization with strong global onvergen e properties. SIAM Journal on Numeri al Analysis, 22(1), 47{67, 1985.

S. A. Vavasis. Nonlinear Optimization: Complexity Issues. International Series of Monographs onComputer S ien e.Oxford UniversityPress, Oxford,England,1992.

L.N.Vi ente. Trust-region interior-point algorithms for a lass of nonlinear programming prob-lems. PhDthesis,DepartmentofComputationalandAppliedMathemati s,Ri eUniversity, Houston,Texas,USA, 1995. Report TR96-05.