RESEARCH OUTPUTS / RÉSULTATS DE RECHERCHE
Author(s) - Auteur(s) :
Publication date - Date de publication :
Permanent link - Permalien :
Rights / License - Licence de droit d’auteur :
Bibliothèque Universitaire Moretus Plantin
Institutional Repository - Research Portal
Dépôt Institutionnel - Portail de la Recherche
researchportal.unamur.be
University of Namur
GALAHAD, a library of thread-safe fortran 90 packages for large-scale nonlinear
optimization
Orban, D.; Toint, Philippe; Gould, Nick
Published in:
ACM Transactions on Mathematical Software
DOI:
10.1145/962437.962438
Publication date:
2003
Document Version
Early version, also known as pre-print
Link to publication
Citation for pulished version (HARVARD):
Orban, D, Toint, P & Gould, N 2003, 'GALAHAD, a library of thread-safe fortran 90 packages for large-scale
nonlinear optimization', ACM Transactions on Mathematical Software, vol. 29, no. 4, pp. 353-372.
https://doi.org/10.1145/962437.962438
General rights
Copyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners
and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.
• Users may download and print one copy of any publication from the public portal for the purpose of private study or research.
• You may not further distribute the material or use it for any profit-making activity or commercial gain
• You may freely distribute the URL identifying the publication in the public portal ?
Take down policy
If you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediately
and investigate your claim.
Ni holasI. M.Gould
ComputationalS ien eandEngineeringDepartment,RutherfordAppletonLaboratory,
Chilton,Oxfordshire,OX110QX,England,EU
DominiqueOrban
ECEDepartment, NorthwesternUniversity,EvanstonIl60208,USA
and
PhilippeL. Toint
DepartmentofMathemati s,UniversityofNamur,B-5000Namur,Belgium,EU
Wedes ribethe designofversion1.0of GALAHAD,a libraryofFortran90pa kages for
large-s alenonlinearoptimization. Thelibraryparti ularlyaddressesquadrati programmingproblems,
ontainingbothinteriorpointanda tivesetalgorithms,aswellastoolsforprepro essingproblems
priorto solution. It also ontainsan updated versionof the venerablenonlinear programming
pa kage,LANCELOT.
CategoriesandSubje tDes riptors: G.4[Mathemati alSoftware℄:Algorithmdesignand
anal-ysis
GeneralTerms: Algorithms
AdditionalKey Words and Phrases: Fortran90, GALAHAD, LANCELOT, large-s ale nonlinear
optimization,large-s alequadrati programming
1. INTRODUCTION
We released our large-s ale nonlinear programming pa kage LANCELOT [Conn,
Gould and Toint, 1992℄ late in 1991. Over the intervening years, LANCELOT A
has been used bya largenumberof people, bothvia freesour e downloads from
ourWWW sites and bymeans ofthe NEOS fa ility [Czyzyk, Mesnier andMore,
1998℄ from theOptimization Te hnology Center at Argonne National Laboratory
and Northwestern University in the USA. It is fair to say that we (and others)
re ogniseditslimitationsfromtheoutset,andithaslongbeenourgoaleventually
toprovideasuitablesu essor.
LANCELOTA waswrittenin Fortran77. Despitethewidespreadavailabilityof
goodFortran77 ompilers,thelimitationsofthelanguage,parti ularlytheabsen e
ofstandardisedmemoryallo ationfa ilities, provedto beaserious limitation,
es-pe iallyforlargeproblems. Workstartedin1995onaFortran90implementation,
ThisworkwassupportedinpartbytheEPSRCgrantGR/R46641.
Permission to makedigital/hard opy of all or part of thismaterial without fee for personal
or lassroomuseprovidedthat the opiesarenotmadeordistributedforpro t or ommer ial
advantage,theACM opyright/servernoti e,thetitleofthepubli ation,anditsdateappear,and
noti eisgiventhat opyingisbypermissionoftheACM,In . To opyotherwise,torepublish,
topostonservers,ortoredistributetolistsrequirespriorspe i permissionand/orafee.
parti ularlytotakeadvantageofthenewlanguage'smemorymanipulationfeatures
and array onstru ts, and by 1997 wehad a working prototype of the improved
LANCELOTB. We had hosen to stay with Fortran rather than C, say, partially
be ausemanyofthepa kage'skey(external) omponents,mostespe iallytheHSL
[2002℄sparsematrix odes produ edbyour olleagues, areallFortranbased,and
also be ause webelieved (andstill believe) Fortran90 apable ofproviding allof
thefa ilitiesweneeded.
Regrettably,at oraroundthat time, anumberof our olleagueshadstarted to
release theresults of omparativetests of theirnew odes|for instan e SNOPT
[Gill,MurrayandSaunders,2002℄,LOQO[VanderbeiandShanno,1999℄,KNITRO
[Byrd, Hribar and No edal, 1999℄, and FilterSQP [Flet her and Ley er, 2002℄|
againstLANCELOTA,and theresultsmadefranklyrather depressingreadingfor
us[DolanandMore,2000,Benson,ShannoandVanderbei,2001,andChin,2001℄,
LANCELOT often, but far from always, being signi antly outperformed. Quite
learly,the promisealwaysheld for large-s alesequentialquadrati programming
(SQP)methods, basedonhowtheyoutperformedotherte hniquesin omparative
tests su h asthosedue to (Ho k andS hittkowski 1981),wasnowbeingrealised,
and thelimitofwhat might bea hievedbyaugmentedLagrangianmethodssu h
asLANCELOTAhadprobablybeenrea hed.
Relu tantly,weabandonedanyplanstoreleaseLANCELOTBat thattime,and
turned our attention instead to SQP methods. To our minds, there had never
really beenmu h doubt that SQP methods would be moresu essful in the long
term,buttherehadbeengeneral on ernsoverhowtosolve(approximately)their
all-important (large-s ale) quadrati programming(QP) subproblems. Thus, we
de ided that ournextgoal should beto produ e high-quality QP odesfor
even-tualin orporationinourownSQPalgorithm(s). Sin ewebelievethattheremight
be onsiderableinterestfromothersinsu h odes,wehavede idedtoreleasethese
beforewehave nalisedourSQPsolver(s). Andsin ewerealisedthatfarfrom
pro-du ingasinglepa kage,wearenowine e t buildingalibraryofindependentbut
inter-relatedpa kages,wehave hosentoreleasean(evolving)large-s alenonlinear
optimizationlibrary,GALAHAD.
In somesense GALAHAD Version 1.0 is a stop-gap, sin e, although it in ludes
the upgraded B version of LANCELOT, we doubt seriously whether LANCELOT
B is astate-of-the-artsolverfor generalnonlinearprogramming problems. What
GALAHAD V1.0 doesprovideare the quadrati programmingsolversand related
toolsthat we anti ipate will allowus to developthenext-generation SQP solvers
weintendtointrodu einVersion2ofthelibrary.
2. LIBRARYCONTENTS
Ea h pa kagein GALAHAD is written asaFortran 90module,and the odesare
allthreadsafe. Thedefaultpre isionforrealargumentsisdouble,butthisiseasily
transformed to single in a UNIX environment using provided sed s ripts. Ea h
pa kagehasa ompanyingdo umentationandatestprogram. Thelatterattempts
toexe uteas mu h ofthepa kageasrealisti allypossible. (Thefa t thatsomeof
thepa kagesare intendedfornonlinearproblems makesit diÆ ultto ensurethat
pathologi albehaviourthat annot be ruledoutin theory but neverthelessseems
never to o ur in pra ti e.) Options to pa kages may be passed both dire tly,
through subroutine arguments, and indire tly, viaoption-spe i ation les. The
se ond me hanismis parti ularly usefulwhen there isahierar hyof pa kagesfor
whi hauserwishesto hangeanoptionforoneofthedependentpa kageswithout
re ompilation.
2.1 Overview
GALAHAD omprisesthefollowingmajorpa kages:
. LANCELOT B is a sequential augmented Lagrangianmethod for minimizing a
(nonlinear)obje tivesubje ttogeneral(nonlinear) onstraintsandsimplebounds.
. QPBisaprimal-dualinterior-pointtrust-regionmethodforminimizingageneral
quadrati obje tivefun tion overapolyhedralregion.
. QPAisana tive/working-setmethodforminimizingageneralquadrati
obje -tivefun tion overapolyhedralregion.
. LSQPis an interior-point method for minimizing a linearorseparable onvex
quadrati fun tionoverapolyhedralregion.
. PRESOLVE is a method for prepro essing linear and quadrati programming
problemspriortosolutionbyotherpa kages.
. GLTRis amethod for minimizing a general quadrati obje tivefun tion over
theinterioror boundaryofa(s aled)hyper-sphere.
. SILSprovidesaninterfa etotheHSLsparse-matrixpa kageMA27thatis
fun -tionallyequivalentto themorere entHSLpa kageHSLMA57.
. SCUusesaS hur omplementupdateto ndthesolutionofasequen eoflinear
systemsforwhi hthesolutioninvolvingaleadingsub-matrixmaybefoundbyother
means.
Inaddition,GALAHAD ontainsthefollowingauxiliarypa kages:
. QPPreorderslinearandquadrati programmingproblemstoa onvenientform
priortosolutionbyotherpa kages.
. QPT provides a derived type for holding linear and quadrati programming
problems.
. SMTprovidesaderivedtypeforholdingsparsematri esinavarietyofformats.
. SORTgivesimplementationsofbothQui k-sortandHeap-sortmethods.
. RANDprovidespseudo-randomnumbers.
. SPECFILE allows users to provide options to other pa kages, using lists of
keyword-valuepairsgivenin pa kage-dependentoption-spe i ation les.
. SYMBOLSassignsvaluesto alistof ommonlyusedGALAHADvariables.
2.2 DetailsofmajorGALAHADpa kages
2.2.1 LANCELOT B. LANCELOT Ais fully des ribed in Conn et al. [1992℄,and
theresultsof omprehensivetestsaregiveninConn,GouldandToint[1996a℄. The
enlivenedLANCELOTBo ersanumberofimprovementsoveritsprede essor.New
|automati allo ationofworkspa e,
|anon-monotonedes entstrategy [seeToint, 1997,and Conn,Gouldand Toint,
2000a,x10.1℄tobeusedbydefault,
|optionaluseofMoreandToraldo[1991℄-typeproje tions[seealsoLinandMore,
1999a℄duringthesubproblemsolutionphase,
|aninterfa e to Linand More's[1999b℄ publi domainin omplete Cholesky
fa -torizationpa kageICFSforuseasapre onditioner,
|optionaluseofstru turedtrustregionstomodelstru turedproblemsbetter[see
Conn,Gould,SartenaerandToint,1996b,andConnet al.,2000a,x10.2℄,
|more exibilityoverthe hoi eofderivatives,whi hneedonlybeprovidedfora
subsetof theelementfun tions from whi h theproblem is built,theremainder
beingestimatedbydi eren esorse antapproximations.
The main reason for extending LANCELOT'slife is asa prototypefor what may
bea hievedusing Fortran90/95inpreparationforfutureGALAHADSQP solvers,
sin etheproblemdata stru tureandinterfa eisunlikelyto hangesigni antly.
Toillustratethee e tsofthenewfeatures,bothLANCELOTAandLANCELOT
B(usinganumberofnewoptions)wererunonalltheexamples(ex eptlinearand
quadrati programs)fromtheCUTErtestset[Gould,OrbanandToint,2002a℄. The
CUTErsetdi ersfromitsprede essorinCUTE[Bongartz,Conn,GouldandToint,
1995℄ both in the number of problems and by virtue of a long-overdue in rease
in default dimensions for all variable-dimensional problems. The options new to
LANCELOTBthat we onsideredwereasfollows:
|Thedefault: anon-monotonedes entstrategywithahistorylengthof1,aband
pre onditionerwithsemi-bandwidth5,andexa tse ondderivatives.
|Thedefault,ex eptthatamonotonedes entstrategyisused.
|Thedefault,ex eptthatSR1approximationstothese ondderivativesareused.
|Thedefault,ex eptthattheLinandMore's[1999b℄in ompleteCholesky
fa tor-izationpre onditioner,ICFS,with5extraworkve tors,isused.
|Thedefault, ex eptthat theMoreandToraldo[1991℄proje tedsear h, with5
restartsisused.
|Thedefault,ex eptthatastru turedtrustregion[Connetal.,1996b℄isused.
|Thedefault,ex eptthatthehistorylengthforthenon-monotonedes entstrategy
isin reasedto5.
LANCELOTAwasrunwithitsdefaults,sin eanexhaustivetestofotherLANCELOT
Aoptionshasalreadybeenperformed[Conn etal.,1996a℄.
InFigure2.1theperforman epro les[DolanandMore,2001℄fortheCPUtime
(inse onds)andthenumbersoffun tionevaluationsfromthesetestsarereported.
All experiments were performed on a single pro essor of a Compaq AlphaServer
DS20with3.5GbytesofRAM,usingtheCompaqf90 ompilerwithfull
ma hine-spe i optimization. Runs wereregardedasunsu essful(andterminated)ifthey
rea hed30minutesCPUtimeor10000fun tion evaluations. Ofthe749problems
for that reason or be ause the evaluation of problem fun tions led to
oating-pointex eptions. Ofthoserequiringmorethan30minutes/10000 alls,most ould
ultimatelybesolvedbyin reasingtheCPU anditerationlimits.
Suppose that a given algorithm i from a ompeting set A reports a statisti
s ij
0 when run on example j from our test set T, and that the smaller this
statisti thebetterthealgorithmis onsidered. Let
k(s;s ;)= 1 if ss 0 otherwise.
Then,theperforman epro leofalgorithmiisthefun tion
p i ()= P j2T k(s i;j ;s j ;) jTj with 1; wheres j =min i2A s ij . Thusp i
(1)givesthefra tionofthenumberofexamplesfor
whi halgorithmiwasthemoste e tive(a ordingtostatisti ss ij
),p i
(2)givesthe
fra tionforwhi halgorithmiiswithinafa torof2ofthebest,andlim !1
p i
()
givesthefra tionofexamplesforwhi hthealgorithmsu eeded. We onsidersu h
apro le to be averye e tive means of omparing algorithms and (in this ase)
therelativemeritsofthenewoptionsavailableinLANCELOTB.
Thebene tsof thenon-monotone strategyare apparentin termsof bothCPU
timeandfun tionevaluationredu tions. Likewise,theLin-More(forfun tionev
al-uations)andMore-Toraldo(forCPUtime)optionsbothprovetobeadvantageous.
Inaddition,wearepleasedtoseethatthebest ofthenewoptionsshowsomegain
withrespe ttoLANCELOTA,parti ularlyaswewereinitially on ernedthat
mov-ingfromFortran77to90mightgiverisetosomeperforman epenalties. Theonly
new optionthat weare disappointedwith is theuseof astru tured trust region,
and,onthebasisofthesetests,we annotreallyre ommendthisstrategy. Thefull
setofresultsareavailableasaninternal report[Gould,OrbanandToint,2002b℄.
2.2.2 QPB. ThemoduleQPBisanimplementationofaprimal-dualfeasible
interior-pointtrust-regionmethodforquadrati programming.
Tosetthes ene,thequadrati programmingproblem isto
minimize x2I R n q(x) 1 2 x T Hx+g T x (2.1)
subje ttothegenerallinear onstraints
l i a T i x u i ; i=1;:::;m; (2.2)
andthesimplebound onstraints
x l j x j x u j ; j=1;:::;n; (2.3)
for given ve tors g, a i , l , u , x l , x u
and a given symmetri (but not ne essarily
de nite)matrixH. Equality onstraintsand xedvariablesareallowedbysetting
u i = l i and x u j = x l j
asrequired, and any or all of the onstraint bounds may
bein nite. The required solutionx to (2.1){(2.3) satis es the primal optimality
onditions
1
1.5
2
2.5
3
3.5
4
4.5
5
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
history = 1
monotone
SR1
Lin−More (5)
More−Toraldo (5)
Structured TR
history = 5
LANCELOT A
1
1.5
2
2.5
3
3.5
4
4.5
5
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
history = 1
monotone
SR1
Lin−More (5)
More−Toraldo (5)
Structured TR
history = 5
LANCELOT A
Fig.2.1.Performan epro leforLANCELOToptions: CPUtimes(top)andnumberoffun tion
evaluations(bottom). Thehorizontalaxisgivestheargument,whiletheverti alaxisre ords p()forea hofthe ompetingoptions,i.
and l u ; x l xx u ; (2.5)
thedualoptimality onditions
Hx+g=A T y+z; y=y l +y u and z=z l +z u ; (2.6) and y l 0; y u 0; z l 0 and z u 0; (2.7)
andthe omplementarysla kness onditions
(Ax l ) T y l =0; (Ax u ) T y u =0; (x x l ) T z l =0 and (x x u ) T z u =0; (2.8)
where isanadditionalve torofprimalvariables,theve torsyandzareLagrange
multipliersforthegenerallinear onstraintsandthedualvariablesforthebounds,
respe tively,andwhere theve torinequalitieshold omponentwise.
Primal-dualinterior-pointmethodsiterate towardsapointthat satis esthe
op-timality onditions(2.4){(2.8)byultimatelyaimingtosatisfy(2.4),(2.6)and(2.8),
whileensuringthat(2.5)and(2.7)aresatis edasstri tinequalitiesatea hstage.
Appropriatenormsoftheamountsbywhi h(2.4),(2.6)and(2.8)failtobesatis ed
areknownastheprimalanddualinfeasibility,andtheviolationof omplementary
sla kness,respe tively. Thefa t that(2.5)and(2.7)aresatis edasstri t
inequal-itiesgivessu hmethodstheirname,interior-pointmethods.
Theproblemissolvedintwophases. Thegoalofthe rst\initialfeasiblepoint"
phaseisto ndastri tlyinteriorpointthatisprimalfeasible(satis es(2.4)). The
GALAHADpa kageLSQP(seex2.2.4)isusedforthispurpose,ando erstheoptions
ofeithera eptingthe rststri tlyfeasiblepointfound,orpreferablyofaimingfor
theso- alled\analyti enter"of thefeasibleregion. Given su h asuitableinitial
feasible point, the se ond \optimality" phase ensures that (2.4) remainssatis ed
whileiterating tosatisfydual feasibility(2.6) and omplementarysla kness(2.8).
Itpro eedsbyapproximatelyminimizingasequen eofbarrierfun tions
q(x) 2 4 m X i=1 log( i l i )+ m X i=1 log( u i i )+ n X j=1 log(x j x l j )+ n X j=1 log(x u j x j ) 3 5 ;
for an appropriate sequen e of positive barrier parameters onverging to zero,
while ensuring that (2.4) remains satis ed and that x and are stri tly interior
pointsfor(2.5). Notethattermsintheabovesummations orrespondingtoin nite
boundsareignored,andthat equality onstraintsaretreatedspe ially.
Ea h of the barrier subproblems is solved using a trust-region method. Su h
amethod generates atrial orre tionstep (Æx;Æ ) to the urrentiterate (x; ) by
repla ingthenonlinearbarrierfun tion lo allybyasuitablequadrati model,and
approximatelyminimizingthismodelintheinterse tionof(2.4)andatrustregion
k(Æx;Æ )kforsomeappropriatepositivetrust-regionradiusandnormkk.
Thestepisa epted/reje tedandtheradiusadjustedonthebasisofhowa urately
themodelreprodu esthevalueofthebarrierfun tionatthetrialstep. Ifthestep
provesto be una eptable, a linesear h is performed along the stepto obtainan
barrier fun tion is almost always less su essful than an alternative primal-dual
model, and onsequentlytheprimal-dualmodelisusuallytobepreferred.
Thetrust-regionsubproblemisapproximatelysolvedusingthe ombined
onjugate-gradient/Lan zosmethod[seeGould,HribarandNo edal,2001andGould,Lu idi,
RomaandToint,1999a℄implementedintheGALAHAD module GLTR(see x2.2.6).
Su h amethod requires asuitable pre onditioner, and in our ase, theonly
ex-ibility we have is in approximating the model of the Hessian. Although using a
xed form of pre onditioningis sometimes e e tive, wehave provided theoption
of an automati hoi e that aims to balan e the ost of applying the
pre ondi-tioner against the need for an a urate solution of the trust-region subproblem.
Thepre onditionerisappliedusingtheGALAHADfa torizationpa kageSILS(see
x2.2.7)|oroptionallyusingHSL MA57fromHSLifthisisavailable|butoptionsat
thisstagearetofa torizethepre onditionerasawhole (theso- alled\augmented
system"approa h)ortoperformablo kelimination rst(the\S hur- omplement"
approa h). Thelatterisusuallytobepreferredwhena(non-singular)diagonal
pre- onditionerisused,but maybeineÆ ientifanyofthe olumnsofAistoodense.
The theoreti al justi ation of the overall s heme, for problems with general
obje tivesand inequality onstraints, is given by Conn, Gould, Orbanand Toint
[2000b℄, in whi h wealso presentnumeri al results that suggest it is indeed able
to solve some problems of the size we had been aiming for. More re ently, we
investigated the ultimate rate of onvergen e of su h s hemes, and have shown
that,underfairlygeneral onditions,a omponentwisesuperlinearrateisa hievable
forbothquadrati andgeneralnonlinear programs[Gould, Orban,Sartenaerand
Toint,1999b℄.
Full advantageis takenofanyzeroelementsin thematrixH ortheve torsa i
.
AnolderversionofQPB(usingHSLMA57ratherthanSILS,seex2.2.7,butwithout
some of the most re ent features in QPB) is available ommer ially as HSL VE12
withinHSL.
2.2.3 QPA. ThemoduleQPAisanimplementationofase ondapproa htoquadrati
programming,thistimeofthea tive/workingsetvariety. QPAisprimarilyintended
withinGALAHADtodealwiththe asewhereagoodestimateoftheoptimala tive
set has beendetermined (so that relatively few iterationswill be required). The
methodisa tuallymoregeneralins ope,andisgearedtowardsolving` 1
-quadrati
programmingproblemsoftheform
minimize x2 IR n m(x; g ; b ) def = q(x)+ g v g (x)+ b v b (x) (2.9)
involvingthequadrati obje tiveq(x)and theinfeasibilities
v g (x)= m X i=1 max( l i a T i x;0)+ m X i=1 max(a T i x u i ;0) and v b (x)= n X j=1 max(x l j x j ;0)+ n X j=1 max(x j x u j ;0):
fun tion m(x; g ; b ) at x =x (k )
is sought. This isa hievedby rst omputinga
sear h dire tion s (k )
and thensetting x (k +1) =x (k ) + (k ) s (k )
, where thestepsize
(k )
is hosen as the rst lo al minimizer of ( ) = m(x (k ) + s (k ) ; g ; b ) as
in reases from zero. The stepsize al ulationis straightforward, and exploits the
fa t that ( )isapie ewisequadrati fun tion of .
Thesear hdire tionisde nedbyasubsetofthe\a tive"termsinv(x),i.e.,those
for whi h a T i x = l i or u i (for i =1;:::;m) orx j = x l j orx u j (for j =1;:::;n).
The \working set" W (k )
is hosen as the interse tion of subsets of indi es i and
j fromthea tiveterms,andissu hthatits membershavelinearlyindependent
gradients. The sear h dire tion s (k )
is hosen asan approximate solution of the
equality- onstrainedquadrati program
minimize s2I R n m Q (s) def = q(x (k ) +s)+ g l (k ) g (s)+ b l (k ) b (s); (2.10) subje tto a T i s=0; i2f1;:::;mg\W (k ) ; and x j =0; j2f 1;:::; ng\W (k ) ; (2.11) where l (k ) g (s)= m X i=1 a T i x< l i a T i s + m X i=1 a T i x> u i a T i s and l (k ) b (s)= n X j=1 xj<x l j s j + n X j=1 xj>x u j s j :
Theequality- onstrainedquadrati program(2.10){(2.11)is solvedbyaproje ted
pre onditioned onjugategradientmethod[Gouldetal.,2001℄.Themethod
termi-natesifthesolutionisfound,orifapre-spe i editerationlimitis rea hed,orifa
dire tionofin nitedes entislo ated,alongwhi hm Q
(s)de reaseswithoutbound
within the feasible region (2.11). Su essively morea urate approximationsare
requiredassuspe tedsolutionsof(2.9)areapproa hed.
Pre onditioningof the onjugate gradientiterationrequires thesolution of one
ormorelinearsystemsoftheform M (k ) A (k )T A (k ) 0 p u = g 0 ; (2.12) where M (k )
is a \suitable" approximation to H and the rows of A (k )
omprise
thegradientsofthetermsinthe urrentworkingset. Rather thanre omputinga
fa torizationof thepre onditioner at everyiteration,weuseaS hur omplement
method,re ognisingthefa tthatgradual hangeso urtosu essiveworkingsets.
Themain iteration is dividedinto asequen e of \major"iterations. At thestart
ofea hmajor iteration(say, theoveralliterationl),afa torizationof the urrent
\referen e"matrix M (l) A (l)T A (l) 0 (2.13)
isobtainedusing theGALAHAD fa torizationpa kageSILS(see x2.2.7)|or,on e
again,optionallyusingHSL MA57fromHSLifthisisavailable.Thisreferen ematrix
may be fa torized as a whole (the augmented system approa h). Alternatively,
systemsinvolving(2.13)maybesolvedbyperformingablo keliminationofpand
thenfa torizingA (l) M (l) 1 A (l)T
(the\S hur- omplement"approa h). Thelatteris
usuallyto bepreferredwhena(non-singular)diagonalpre onditionerisused, but
maybeineÆ ientifanyofthe olumnsofA (l)
istoodense. Subsequentiterations
withinthe urrentmajoriterationobtainsolutionsto(2.12)viathefa torsof(2.13)
andanappropriate(dense)S hur omplement,obtainedfromSCUinGALAHAD(see
x2.2.8). Themajoriterationterminateson ethespa erequiredtoholdthefa tors
ofthe(growing)S hur omplementex eedsagiventhreshold.
Theworkingset hangesby(a)theadditionofana tivetermen ounteredduring
the determination of the stepsize, or (b) the removal of a term if s = 0 solves
(2.10){(2.11). Thede isiononwhi htoremoveinthelatter aseisbaseduponthe
expe tedde reaseupontheremovalofanindividual term,andthisinformationis
available from themagnitude and sign of the omponentsof the auxiliaryve tor
u omputed in (2.12). At optimality, the omponents of u for a i
terms will all
lie between0 and g
|and those for theother termsbetween0and b
|and any
violationofthisruleindi atesfurther progressispossible.
Tosolvequadrati programsoftheform(2.1){(2.3),asequen eofproblemsofthe
form(2.9)aresolved,ea hwithalargervalueof g
and/or b
thanitsprede essor.
Therequiredsolutionhasbeenfoundon etheinfeasibilities v g
(x) andv b
(x)have
beenredu edtozeroatthesolutionof(2.9)forthegiven g
and b
.
Having proposed and implemented two very di erent quadrati programming
methods,anobviousquestionis: howdothemethods ompare? Weexaminedthis
questionin Gouldand Toint[2001℄by omparing QPAandQPBonthe CUTErQP
testset[Gould etal.,2002a℄.
Whileformodestsizedproblems,startedfrom \random"points,thetwo
meth-odsareroughly omparable,theadvantagesoftheinterior-pointapproa hbe ome
quite lear when problem dimensions in rease. For problems involving tens of
thousands of unknowns and/or onstraints, oura tive set approa h simply takes
toomanyiterations,while thenumberofiterationsrequiredbythe interiorpoint
approa hseemsrelativelyinsensitivetoproblemsize. Forgeneralproblems
involv-inghundredsofthousandsorevenmillionsofunknowns/ onstraints,thea tiveset
approa h is impra ti al, while weillustrate in Table2.1 that QPBis ableto solve
problemsofthissize. QPBalsoappearstos alewellwithdimension,as anbeseen
seein Table2.2.
Whilesu h guresmightseem to indi ate that QPBshould alwaysbepreferred
to QPA, this is not the ase. In parti ular, if a good estimate of the solution|
moreparti ularly,theoptimala tiveset|isknown,a tive-setmethodsmayexploit
this, while interior-pointmethods are( urrently)lessableto doso. Inparti ular
GouldandToint[2001℄illustratethatQPAoftenoutperformsQPBonwarm-started
problemsunlesstheproblemis( loseto)degenerateorveryill onditioned. Thus,
sin e nonlinear optimization (SQP) algorithms often solve a sequen e of related
problemsforwhi htheoptimala tivesetsarealmostora tuallyidenti al,thereis
Name n m type its time QPBAND 100000 50000 C 13 157 QPBAND 200000 100000 C 17 1138 QPBAND 400000 200000 C 17 2304 QPBAND 500000 250000 C 17 2909 QPNBAND 100000 50000 NC 12 32 QPNBAND 200000 100000 NC 13 71 QPNBAND 400000 200000 NC 14 156 QPNBAND 500000 250000 NC 13 181
Table2.1. QPBon large QPexamples. Runs performedon a Compaq AlphaServer DS20 (3.5
GbytesRAM), timeinCPU se onds. nisthe numberof unknowns,and m isthe numberof
general onstraints. C indi atesa onvex problem,while NCis a onvex one. Note that the
fa torizationforQPBAND llsin onsiderablymorethanthatforQPNBAND,andthisa ountsforthe
signi antlyhigherCPUtimes.
Name n m type its time
PORTSQP 10 1 C 11 0.02 PORTSQP 100 1 C 15 0.03 PORTSQP 1000 1 C 26 0.09 PORTSQP 10000 1 C 37 1.26 PORTSQP 100000 1 C 20 9.48 PORTSQP 1000000 1 C 11 72.31 PORTSNQP 10 2 NC 21 0.03 PORTSNQP 100 2 NC 30 0.04 PORTSNQP 1000 2 NC 39 0.17 PORTSNQP 10000 2 NC 32 1.70 PORTSNQP 100000 2 NC 107 58.69 PORTSNQP 1000000 2 NC 22 209.53
Table2.2. HowQPBs aleswithdimension.NotationasforTable2.1.
An enhan ed version of QPA (using HSLMA57 rather than SILS, see x2.2.7) is
available ommer iallyasHSLVE19within HSL.
2.2.4 LSQP. LSQPisaninterior-pointmethodforminimizingalinearor
separa-ble onvexquadrati fun tion
minimize 1 2 n X i=1 w 2 i (x i x 0 i ) 2 +g T x;
forgivenweightsw andgradientg, overthepolyhedral region(2.2){(2.3). Inthe
spe ial asewherew=0andg=0theso- alledanalyti enterofthefeasibleset
will be found, while linear programming, or onstrained least distan e, problems
may be solved by pi king w = 0, or g = 0, respe tively. The basi algorithm is
that of Zhang [1994℄. LSQP is used within GALAHAD by QPB to nd an initial
stri tly-interiorfeasiblepoint(seex2.2.2).
Note that sin e predi tor- orre torstepsare not taken, the method is unlikely
tobeaseÆ ientasstate-of-the-artinterior-pointmethodsforlinearprogramming.
2.2.5 PRESOLVE. The module PRESOLVE is intended to pre-pro ess quadrati
programmingproblemsoftheform(2.1){(2.3). Thepurposeisto exploitthe
opti-malityequations(2.4){(2.8)soastosimplify theproblemandredu etheproblem
toastandardform(thatmakessubsequentmanipulationeasier),de nedasfollows:
|Thevariablesareorderedsothattheirboundsappearintheorder
free x j non-negativity 0 x j lower x l j x j range x l j x j x u j upper x j x u j non-positivity x j 0
Fixed variables are removed. Within ea h ategory, the variables are further
ordered sothat those with non-zero diagonal Hessian entries o ur before the
remainder.
|The onstraintsareorderedsothat theirboundsappearintheorder
non-negativity 0 (Ax) i equality l i = (Ax) i lower l i (Ax) i range l i (Ax) i u i upper (Ax) i u i non-positivity (Ax) i 0
Free onstraintsareremoved.
|Inaddition, onstraintsmayberemovedorboundstightened,toredu ethesize
of the feasible region or simplify the problem if this is possible, and bounds
maybetightened onthedual variables andthe multipliers asso iated withthe
problem.
Thepresolving algorithm [Gould and Toint, 2002℄ pro eeds by applying a
(po-tentially long)series ofsimple transformationsto theproblem. Theseinvolvethe
removalofemptyandsingletonrows,theremovalofredundantandfor ingprimal
onstraints,the tightening ofprimal and dualbounds, theexploitation ofempty,
singleton, anddoubleton olums, mergingofdependentvariables,row
\sparsi a-tion" and splitting equalities. Transformations are applied in su essive passes,
ea hpassinvolvingthefollowinga tions:
(1) removeemptyandsingletonsrows,
(2) trytoeliminatevariablesthatdonotappearin thelinear onstraints,
(3) attempttoexploit thepresen eofsingleton olumns,
(4) attempttoexploit thepresen eofdoubleton olumns,
(5) ompletetheanalysis ofthedual onstraints,
(6) removeemptyandsingleton rows,
(7) possiblyremovedependentvariables,
(8) analyzetheprimal onstraints,
(10) he k the urrent status of the variables, dual variables and multipliers for
optimalityorinfeasibility.
All these transformations are applied on the stru ture of the original problem,
whi his onlypermuted tostandardform after alltransformationsare ompleted.
The redu ed problem may then be solved by aquadrati or linear programming
solver. Finally, thesolutionof thesimpli ed problem isre-translatedto the
vari-ables/ onstraintsformatoftheoriginalproblemina\restoration"phase.
Atthe overall level, the presolving pro ess followsone of thefollowingtwo
se-quen es:
initialize !
applytransformations !(solveproblem)! restore ! terminate or initialize ! read spe le ! apply transformations ! solve problem ! restore ! terminate
where thepro edure's ontrol parametermaybemodi ed byreading anexternal
\spe le",andwhere(solveproblem)indi atesthattheredu edproblemissolved.
Ea hofthe\boxed"stepsinthesesequen es orrespondsto allingaspe i routine
of the pa kage, while a bra keted subsequen e of steps means that they an be
repeatedwithproblemshavingthesamestru ture.
GouldandToint[2002℄indi atethat,when onsideringall178linearandquadrati
programming problems in the CUTE test set [Bongartz et al., 1995℄, an average
redu tion of roughly 20% in both the number of unknowns and the number of
onstraints results from applying PRESOLVE. With the GALAHAD QP solverQPB
(seex2.2.2),an overall averageredu tionof roughly10%in CPU timeresults. In
some ases,thegainisdramati . Forexample,fortheproblemsGMNCASE4,STNPQ1,
STNQP2andSOSQP1,PRESOLVEremovesallthevariablesand onstraints,andthus
revealsthe ompletesolutionto theproblemwithoutresortingtoaQPsolver.
Currently PRESOLVEisnot embedded within QPA/BorLSQPandmust be alled
separately,butweintendto orre tthisdefe tforVersion2.0.
2.2.6 GLTR. GLTRaimsto ndtheglobalsolutiontotheproblemofminimizing
thequadrati fun tion(2.1)wherethevariablesarerequiredtosatisfythe onstraint
kxk M
, where the M-norm of x is kxk M
= p
x T
Mx for some symmetri ,
positivede nite M, and where >0is agiven s alar. This problem ommonly
o urs as a trust-region subproblem in nonlinear optimization methods, and is
used within GALAHAD by QPB. The method may be suitable for large n as no
fa torization of H is required. Reverse ommuni ationis used to obtain
matrix-ve torprodu tsoftheformHzandM 1
z. Thepa kagemayalsobeusedtosolve
therelatedprobleminwhi hxisinsteadrequiredtosatisfytheequality onstraint
kxk M
=. Themethod isdes ribedin detailin Gouldetal.[1999a℄,andGLTRis
2.2.7 SILS. The module SILSprovides a Fortran 90 interfa e to the Fortran
77 HSLsparse linearequationpa kage MA27(Du and Reid 1982). Theinterfa e
and fun tionality aredesigned to be identi alto the morere ent HSLFortran90
pa kage HSLMA57(Du 2002), enabling anyonewith HSLMA57 to substitute this
easilyforSILSthroughoutGALAHAD.Thereasonthatwearefor edtorelyonMA27
ratherthanthesuperiorHSLMA57bydefaultissimplythattheformerisavailable
without hargefromtheHSLAr hive(http://hsl.rl.a .uk/hslar hive),while
thelatterisonlyavailable ommer ially. SILS(andhen eeitherMA27orHSLMA57)
isrequiredbyQPA/BandLSQP,andisusedoptionallybyLANCELOT B.
2.2.8 SCU. SCUmaybeusedto ndthesolutiontoanextendedsystemofn+m
sparsereallinearequationsinn+munknowns, A B C D x 1 x 2 = b 1 b 2 :
inthe asewherethenbynmatrixA isnonsingularandsolutionstothesystems
Ax=b and A T
y=
maybe obtainedfrom an external sour e, su h as an existing fa torization. The
subroutineusesreverse ommuni ationtoobtainthesolutiontosu h smaller
sys-tems. Themethod makesuseoftheS hur omplementmatrix
S=D CA
1 B:
TheS hur omplement is storedand updatedin fa tored form asadensematrix
(using either Cholesky or QR fa tors asappropriate) and the subroutine is thus
appropriateonlyifthere issuÆ ientstoragefor thismatrix. Spe ial advantageis
takenofsymmetryandde nitenessofthematri esA,DandS. Provisionismade
for introdu ing additional rowsand olumns to, and removing existing rows and
olumnsfrom,theextendedmatrix.
SCU is used by both LANCELOT B and QPA to ope with ore linear systems
thatariseas onstraintsandvariablesenterandleavetheira tive/workingsets. A
slightlysimpli ed versionof SCUisavailablein HSLasHSLMA69.
3. INSTALLATION
Justasforitsimmediateprede essorsCUTErandSifDe [seeGouldetal.,2002a℄,
GALAHADisdesignedtobeusedin amulti-platform,multi- ompilerenvironment
in whi h ore (sour e and s ript) les are available in a single lo ation, and
ma- hine/ ompiler/operatingsystemspe i omponents(mostespe ially ompiled
li-braries of binaries)are isolatedin uniquely identi ablesubdire tories. As before,
wehave on entratedonUNIX andLinuxplatforms, prin ipallybe ausewehave
noexperien ewithothersystems.
GALAHADisprovidedasaseriesofdire toriesand les,alllyingbeneatharoot
dire torythatweshall referto as$GALAHAD.Thedire torystru tureisillustrated
inFigure 3.1.
Beforeinstallationthesub-dire toriesobje ts,modules,makefiles,versions
andbin/syswill allbeempty. Thes riptinstall galahadpromptstheuserfor
theanswerstoaseriesofquestionsaimedatdeterminingwhatma hinetype,
$GALAHAD lanb qpb ...otherpa kages... dum makedefs sr makefiles seds obje ts versions ar h modules bin sys do man1 man examples ar hite ture ar hite ture2 ...otherar hite tures... double single double single ar hite ture ar hite ture2 ...otherar hite tures... double single double single
ar hsub-dire tory)tobuildGALAHADfor|we allthis ombinationofama hine,
operatingsystemand ompileranar hite ture. Ea har hite tureisassigneda
sim-ple(mnemoni )ar hite ture odename,sayar hite ture|forexampleaversion
fortheNAGFortran95 ompileronaPCruningLinuxis odedp .lnx.n95,while
another forthe CompaqFortran90 ompiler onan Alpha system runningTru64
Unixis alp.t64.f90. Having determinedthe ar hite ture, the installations ript
builds sub-dire tories of obje ts and modules named ar hite ture, as well as
furthersub-dire toriesdoubleandsingleofthesetoholdar hite ture-dependent
ompiledlibrariesandmodule leinformation. Inaddition,ar hite ture-dependent
make leinformationandenvironmentvariablesforexe utions riptsarepla edin
les named ar hite turein themakefiles and bin/sys sub-dire tories, and a
lere ordinghowthe odeisrelatedtothear hite tureisputin versions.
TheFortransour e odesforea hGALAHAD pa kageo urs inaseparate
sub-dire tory of the sr dire tory. The sub-dire tory ontains the pa kage sour e,
a omprehensivetest program (along with a simplerse ond test program, whi h
is used asan illustration on how to all the pa kage in the a ompanying
do u-mentation), and a make le. Sin e the order of ompilation of Fortran modules
is important,and as wehave seenthere is astronginterdependen y betweenthe
GALAHAD pa kages, the make les have to be arefully rafted. For this reason
(and in ontrast to CUTEr), we have hosen not to use variants of tools su h as
imaketo build and maintain the make les. Posts ript and PDF Do umentation
forallpa kagesis ontainedindo .
Not every omponent of GALAHAD is distributed as part of the pa kage. In
parti ular,theHSLAr hive odeMA27mustbedownloadedpriortotheinstallation
from
http://hsl.rl.a .uk/ar hive/hslar hive.html
beforeanyoftheQPpa kages anbeused(itisoptionallyusedbyLANCELOTB).
Additionally GALAHAD makesoptionaluse of theLin-Morepre onditionerICFS,
availableas partoftheMINPACK2libraryvia
http://www-unix.m s.anl.gov/~more/i fs/ ,
and theHSL odesMA57andMC61,whi h areonlyavailable ommer ially. If any
non-default ode is used, the le pa kages in the dire tory sr /makedefs must
beeditedto des ribewhere theexternal ode may be found|detailsare given in
pa kages. Default dummyversions ofall optional odesareprovidedin sr /dum
toensurethatlinking priorto exe utionworksproperly.
On e the orre t dire tory stru ture is in pla e, the installation s ript builds
a random-a ess library of the required pre ision by visiting ea h of the
sub-dire tories of sr and alling the Unix utility make. GALAHAD pa kages are all
written indoublepre ision,but ifauserprefersto usesinglepre ision,the
make- les all suitable Unixsed s ripts(stored in seds) to transform the sour eprior
to ompilation. A usermay hoose to installall ofGALAHAD, orjust theQP or
LANCELOT B omponents. There are also sed features to swit h automati ally
fromSILS(seex2.2.7)toMA57ifthelatterisavailable. The ommandmake tests
4. INTERFACESTOTHECUTERTESTSET
As well asproviding stand-alone Fortran modules, we provide interfa esbetween
LANCELOTB, QPA/B, LSQPandPRESOLVE and problems written in theStandard
Input Format (SIF) [Conn et al., 1992℄, most parti ularly the CUTEr test set
[Gouldetal.,2002a℄. Tousetheseinterfa esLANCELOTBuserswillneedtohave
installed SifDe (from http:// uter.rl.a .uk/ uter-www/sifde ),while users
wishing to use the interfa es to the QP pa kages will additionally need CUTEr
(fromhttp:// uter.rl.a .uk/ uter-www).
Torunoneofthesupportedpa kagesonanexamplestoredinEXAMPLE.SIF,say,
auserneedstoissuethe ommand
sdgal ode pa kage EXAMPLE[.SIF℄
where odeis thear hite ture ode dis ussed in x3, pa kage de nes thepa kage
tobeused|itmaybeoneoflanb,qpa,qpborpre,with a esstoLSQPprovided
viaqpb|and the suÆx[.SIF℄ is optional. This ommand translatestheSIF le
intoFortransubroutinesandrelateddatausingthede oderprovidedinSifDe ,and
then allstherequiredoptimizationpa kagetosolvetheproblem. On eaproblem
has been de oded, it may be re-used (perhaps with di erent options) using the
auxiliary ommand
gal ode pa kage
AfewSIF examplesaregivenintheexamplessub-dire tory,whilethesdgaland
gal ommandsareinthebinsub-dire tory,andhaveman-pagedes riptionsinthe
man/man1sub-dire tory.
Oneadditional feature isthat ifthe userhasa ess to theHSL automati
dif-ferentiationpa kagesHSLAD01orHSL AD02[Pry eandReid, 1998℄,these maybe
usedto generateautomati rstandse ondderivativesfortheelementand group
fun tions [Conn et al., 1992℄ from whi h the overall problem is re-assembled by
CUTEr and LANCELOT B. Of ourse it would be better to use one of the more
ommonlyo urringpa kagessu h asADIFOR[Bis hof, Carle,Corliss,Griewank
andHovland, 1992℄,andweplantodothisin thefuture.
Aswith LANCELOTA,optionsmaybe passeddire tlyto thesolversbymeans
of user-de nable option-spe i ation les. Ea h SIF interfa e has its own set of
options,butoverall ontrolismaintainedviatheGALAHADmoduleSPECFILE.
5. AVAILABILITY
GALAHAD may be downloaded from http://galahad.rl.a .uk/galahad-www .
There arerestri tionson ommer ialuse, and allusersare requiredto agree toa
numberofminor onditionsofuse.
6. CONCLUSIONS
Wehavedes ribedthes opeanddesignofthe rstreleaseofGALAHAD,alibrary
of Fortran 90pa kagesfor nonlinearoptimization. Version1.0 of thelibrary
par-ti ularlyaddressesquadrati programmingproblems,althoughthereisanupdated
versionofLANCELOTformoregeneralproblems.
In future, we intend to use the quadrati programming pa kages as the basi
developingAMPL [Fourer,Gay and Kernighan,2003℄ interfa esfor the prin ipal
pa kagessothatuserswillbeabletouseamorenaturalinterfa ethanprovidedby
SIF.Inaddition,weplanto in orporatethe prepro essingtoolsasoptionswithin
theQPsolvers,ratherthanhavingthemstand-aloneasat present.
A knowledgement
TheauthorsareextremelygratefultoMi haelSaundersforhisvery arefulreading
ofthismanus riptandforalargenumberofuseful omments.
REFERENCES
H.Benson,D.F.Shanno,andR.J.Vanderbei.A omparativestudyoflarge-s alenonlinear
optimizationalgorithms. Te hni alReportORFE01-04, OperationsResear hand Finan ial
Engineering,Prin etonUniversity,NewJersey,USA,2001.
Chr.Bis hof,A. Carle,G.Corliss, A.Griewank,and P.Hovland. ADIFOR- generating
derivative odesfromFortranprograms.S ienti Programming,1,1{29,1992.
I.Bongartz,A.R.Conn,N.I.M.Gould,andPh.L.Toint. CUTE:Constrainedand
Un on-strainedTestingEnvironment. ACMTransa tionsonMathemati alSoftware,21(1),123{160,
1995.
R. H. Byrd, M. E. Hribar, and J. No edal. An interior point algorithm for large s ale
nonlinearprogramming.SIAMJ.onOptimization,9(4),877{900,1999.
C.M.Chin. Numeri alresultsofSLPSQP, lterSQPandLANCELOTonsele tedCUTE
testproblems.Numeri alAnalysisReportNA/203,DepartmentofMathemati s,Universityof
Dundee,S otland,2001.
A.R.Conn,N.I.M.Gould,andPh.L.Toint.LANCELOT:aFortranpa kageforLarge-s ale
NonlinearOptimization(ReleaseA).SpringerSeriesinComputationalMathemati s.Springer
Verlag,Heidelberg,Berlin,NewYork,1992.
A.R.Conn,N.I.M.Gould,andPh.L.Toint.Numeri alexperimentswiththeLANCELOT
pa kage (ReleaseA) for large-s alenonlinear optimization. Mathemati alProgramming,
Se-riesA,73(1),73{110,1996a.
A.R.Conn,N.I.M.Gould,andPh.L.Toint. Trust-RegionMethods. SIAM,Philadelphia,
2000a.
A.R.Conn,N.I.M.Gould,D.Orban,andPh.L.Toint.Aprimal-dualtrust-regionalgorithm
fornon- onvexnonlinearprogramming. Mathemati alProgramming,87(2),215{249,2000b.
A.R. Conn,N. I. M.Gould, A. Sartenaer, and Ph.L.Toint. Convergen e properties of
minimizationalgorithmsfor onvex onstraints usingastru turedtrustregion. SIAMJ.on
Optimization,6(4),1059{1086,1996b.
J.Czyzyk,M.P.Mesnier,andJ.J.More. TheNEOSServer. IEEEJ.onComputational
S ien eandEngineering,5,68{75,1998.
E.D.Dolanand J.J.More. Ben hmarkingoptimizationsoftwarewithCOPS. Te hni al
ReportANL/MCS-246,ArgonneNationalLaboratory,Illinois,USA,2000.
E.D.DolanandJ.J.More.Ben hmarkingoptimizationsoftwarewithperforman epro les.
Mathemati alProgramming,91(2),201{213,2002b.
I. S.Du . MA57- a new ode for the solution ofsparse symmetri de niteand
inde -nitesystems. Te hni alReportRAL-TR-2002-024,RutherfordAppletonLaboratory,Chilton,
Oxfordshire,England,2002.
I.S.Du andJ.K.Reid. MA27:AsetofFortransubroutinesforsolvingsparsesymmetri
setsoflinearequations. ReportR-10533,AEREHarwellLaboratory,Harwell,UK,1982.
R.Flet herandS.Ley er.Nonlinearprogrammingwithoutapenaltyfun tion.Mathemati al
Programming,91(2),239{269,2002.
R.Fourer,D.M.Gay,andB.W.Kernighan.AMPL:AModelingLanguageforMathemati al
Programming,(2nd.edn.).Brooks/Cole{ThompsonLearning,Pa i Grove,California,USA,
P. E.Gill, W.Murray,and M.A. Saunders. SNOPT:An SQPalgorithmfor large-s ale
onstrainedoptimization. SIAMJ.onOptimization,12(4),979{1006,2002.
N.I.M.GouldandPh.L.Toint. Numeri almethodsforlarge-s alenon- onvexquadrati
programming.InTrendsinIndustrialandAppliedMathemati s(A.H.SiddiqiandM.Ko vara,
eds.).KluwerA ademi Publishers,Dordre ht,TheNetherlands,149{179,2002.
N. I. M. Gould and Ph. L.Toint. Prepro essing for quadrati programming. Te hni al
Report RAL-TR-2002-001,Rutherford Appleton Laboratory,Chilton,Oxfordshire,England,
2002.
N. I. M. Gould, M. E. Hribar, and J. No edal. On the solutionof equality onstrained
quadrati problemsarisinginoptimization. SIAMJ.onS ienti Computing, 23(4),1375{
1394,2001.
N.I.M.Gould,S.Lu idi,M.Roma,andPh.L.Toint. Solvingthetrust-regionsubproblem
usingtheLan zosmethod.SIAMJ.onOptimization,9(2),504{525,1999a.
N.I.M.Gould, D.Orban,andPh. L.Toint. CUTEr (andSifDe ),a onstrainedand
un- onstrained testing environment,revisited. Te hni alReportRAL-TR-2002-009, Rutherford
AppletonLaboratory,Chilton,Oxfordshire,England,2002a.
N. I. M. Gould, D. Orban, and Ph. L. Toint. Results from a numeri alevaluation of
LANCELOTB.Numeri alAnalysisGroupInternalReport2002-1,RutherfordAppleton
Lab-oratory,Chilton,Oxfordshire,England,2002b.
N. I. M. Gould, D. Orban, A. Sartenaer, and Ph. L. Toint. Superlinear onvergen e of
primal-dualinterior-pointalgorithmsfornonlinear programming. SIAM J.onOptimization,
11(4),974{1002,1999b.
W.Ho kandK.S hittkowski. TestExamplesforNonlinearProgramming Codes. Number
187In`Le tureNotesinE onomi sandMathemati alSystems'.SpringerVerlag,Heidelberg,
Berlin,NewYork,1981.
HSL. A olle tion of Fortran odes for large-s ale s ienti omputation, 2002. See
http://www. se. lr .a .uk/A tivity/HSL.
C.LinandJ.J.More. In ompleteCholeskyfa torizationswithlimitedmemory.SIAMJ.
onS ienti Computing,21(1),24{45,1999a.
C.LinandJ.J.More.Newton'smethodforlargebound- onstrainedoptimizationproblems.
SIAMJ.onOptimization,9(4),1100{1127,1999b.
J.J.MoreandG.Toraldo. Onthesolutionoflargequadrati programmingproblemswith
bound onstraints.SIAMJ.onOptimization,1(1),93{113,1991.
J.D.Pry eandJ.K.Reid.AD01,aFortran90 odeforautomati di erentiation. Te hni al
Report RAL-TR-1998-057,Rutherford Appleton Laboratory,Chilton,Oxfordshire,England,
1998.
Ph.L.Toint. Anon-monotonetrust-regionalgorithmfornonlinearoptimizationsubje tto
onvex onstraints.Mathemati alProgramming,77(1),69{94,1997.
R.J. Vanderbeiand D. F.Shanno. Aninterior pointalgorithmfor non onvex nonlinear
programming.ComputationalOptimizationandAppli ations,13,231{252,1999.
Y.Zhang. Onthe onvergen eofinfeasibleinterior-pointmethodsforthehorizontallinear
omplementarityproblem.SIAMJ.onOptimization,4(1),208{227,1994.
