Revisiting the upper bounding process in a safe Branch and Bound algorithm

HAL Id: hal-00297086

https://hal.archives-ouvertes.fr/hal-00297086

Submitted on 15 Jul 2008

HAL is a multi-disciplinary open access

archive for the deposit and dissemination of

sci-entific research documents, whether they are

pub-lished or not. The documents may come from

teaching and research institutions in France or

abroad, or from public or private research centers.

L’archive ouverte pluridisciplinaire HAL, est

destinée au dépôt et à la diffusion de documents

scientifiques de niveau recherche, publiés ou non,

émanant des établissements d’enseignement et de

recherche français ou étrangers, des laboratoires

publics ou privés.

Revisiting the upper bounding process in a safe Branch

and Bound algorithm

Alexandre Goldsztejn, Yahia Lebbah, Claude Michel, Michel Rueher

To cite this version:

Alexandre Goldsztejn, Yahia Lebbah, Claude Michel, Michel Rueher. Revisiting the upper bounding

process in a safe Branch and Bound algorithm. 14th International Conference on Principles and

Practice of Constraint Programming, Sep 2008, Sydney, Australia. pp.598-602. �hal-00297086�

Bran h and Bound algorithm

⋆

AlexandreGoldsztejn

1

,YahiaLebbah

2,3

,ClaudeMi hel

3

,andMi helRueher

3

1

CNRS/UniversitédeNantes2,ruedelaHoussinière,44322Nantes,Fran e alexandre.goldsztejnuniv-nantes .fr

2

Universitéd'OranEs-SeniaB.P.1524EL-M'Naouar,31000Oran,Algeria ylebbahgmail. om

3

UniversitédeNi e-SophiaAntipolis,I3S-CNRS,06903 SophiaAntipolis,Fran e { pjm, rueher}polyte h.uni e.fr

Abstra t. Findingfeasiblepointsforwhi htheproofsu eedsisa rit-i alissueinsafeBran handBoundalgorithmswhi hhandle ontinuous problems.Inthis paper, weintrodu eanewstrategy to omputevery a urateapproximationsof feasible points. This strategytakes advan-tageofthe Newtonmethodfor under- onstrainedsystemsofequations andinequalities.Morepre isely,itexploitstheoptimalsolutionofa lin-earrelaxation ofthe problemto omputee ientlyapromisingupper bound.FirstexperimentsontheCo onutsben hmarksdemonstratethat thisapproa hisveryee tive.

Introdu tion

Optimization problems are a hallenge forCP in nite domains; theyare also a big hallenge for CP on ontinuous domains. The point is that CP solvers are mu h slower than lassi al(non-safe) mathemati al methods on nonlinear onstraint problems as soon as we onsider optimization problems. The te h-niques introdu ed in this paper try to boost onstraints te hniques on these problems andthus, toredu e thegap betweene ientbut unsafe systemslike BARON

1

,andtheslowbut safe onstraintbasedapproa hes.We onsider here theglobaloptimizationproblem

P

tominimizeanobje tivefun tionunder non-linearequalitiesandinequalities,

minimize

f

(x)

subje tto

g

i

(x) = 0, i ∈ {1, .., k}

h

j

(x) ≤ 0, j ∈ {1, .., m}

(1) with

x

∈

x,

f

: IR

n

_{→ IR}

,

g

i

: IR

n

_{→ IR}

and

h

j

: IR

n

_{→ IR}

; Fun tions

f

,

g

i

and

h

j

arenonlinearand ontinuouslydierentiableon someve torx of intervalsof

IR

.For onvenien e,in thesequel,

g(x)

(resp.

h(x)

)will denotethe ve torof

g

i

(x)

(resp.

h

j

(x)

)fun tions.

⋆

Anextentedversionofthispaperisavailableat:

http://www.i3s.uni e.fr/%7Emh/RR/2008/RR-08.11-A.GOLDSZTEJN.pdf 1

Fun tionBB(INx,

ǫ

;OUT

S

,

[L, U ]

) %

S

:setofprovenfeasiblepoints

%

f

x

denotesthesetofpossiblevaluesfor

f

in

x

%

nbStarts

:numberofstartingpointsintherstupper-bounding

L←{x}

;

(L, U )←(−∞, +∞)

;

S←U pperBounding(x

′

_{, nbStarts)}

; while

w([L, U ]) > ǫ

do

x

′

_←x

′′

su hthat

f

x

′′

= min{f

x

′′

: x

′′

_{∈ L}}

;

L←L\x

′

;

f

x

′

←min(f

x

′

, U)

;

x

′

_{←P rune(x}

′

₎

;

f

x

′

←LowerBound(x

′

)

;

S←S ∪ U pperBounding(x

′

_,

₁₎

; if

x

′

_{6= ∅}

then

(x

′

1

, x

′

2

)←Split(x

′

)

;

L←L ∪ {x

′

1

, x

′

2

}

; if

L = ∅

then

(L, U )←(+∞, −∞)

else

(L, U )←(min{f

x

′′

: x

′′

∈ L}, min{f

x

′′

: x

′′

_{∈ S})}

endwhile

Thedi ulties insu h globaloptimization problems ome mainlyfrom the fa tthat manylo alminimizersmayexist butonlyfewofthemareglobal min-imizers [3℄. Moreover, the feasible region may be dis onne ted. Thus, nding feasible pointsis a riti alissue in safeBran hand Boundalgorithms for on-tinuous globaloptimization. Standard strategiesuse lo al sear h te hniques to provideareasonableapproximationofanupperboundandtrytoprovethata feasiblesolutiona tually existswithin the box aroundtheguessed global opti-mum.Pra ti ally,ndingaguessed pointforwhi h theproofsu eeds isoften avery ostlypro ess.

Inthispaper,weintrodu eanewstrategyto omputeverya urate approx-imationsoffeasiblepoints.ThisstrategytakesadvantageoftheNewtonmethod forunder- onstrainedsystemsofequationsandinequalities.Morepre isely,this pro edure exploits the optimal solution of a linear relaxation of the problem to ompute e ientlya promising upperbound.Firstexperiments onthe Co- onutsben hmarksdemonstrate thatthe ombination ofthispro edure witha safeBran handBoundalgorithm drasti allyimprovestheperforman es.

The Bran h and Bounds hema

Thealgorithm(seeAlgorithm1)wedes ribehereisderivedfromthewellknown Bran handBounds hemaintrodu edbyHorstandTuyforndingaglobal min-imizer. Interval analysis te hniques are used to ensure rigorousand safe om-putationswhereas onstraintprogrammingte hniques areused to improvethe redu tionofthefeasiblespa e.

Algorithm1 omputesen losersforminimizersandsafeboundsoftheglobal minimumvaluewithin aninitial boxx .Algorithm1maintainstwolists:alist

L

of boxesto be pro essed anda list

S

of provenfeasible boxes.It providesa rigorousen loser

[L, U ]

oftheglobaloptimumwithrespe tto atoleran e

ǫ

.

Algorithm1startswith

U pperBounding(x, nbStarts)

whi h omputesaset of feasible boxesby alling alo al sear h with

nbStarts

starting pointsand a

Thebox around the lo al solutionis added to

S

if it is proved to ontain a feasible point. At this stage, if the box

x

′

is empty then, either it doesnot ontain any feasible point or its lower bound

f

x

′

is greater than the urrent upper bound

U

. If

x

′

is notempty, theboxis split alongoneof thevariables 2 oftheproblem.

Inthe main loop, algorithm 1sele tsthe box with thelowest lowerbound of the obje tive fun tion. The

P rune

fun tion applies ltering te hniques to redu ethesizeofthebox

x

′

.Intheframeworkwehaveimplemented,

P rune

just uses a 2B-ltering algorithm [2℄. Then,

LowerBound(x

′

)

omputes arigorous lowerbound

f

x

′

using a linearprogrammingrelaxation of theinitial problem. A tually, fun tion

LowerBound

isbasedon thelinearizationte hniques ofthe Quad-framework[1℄.

LowerBound

omputesasafeminimizer

f

x

′

thankstothe te hniquesintrodu edbyNeumaieret al.

Algorithm1maintains thelowest lowerbound

L

of theremaining boxes

L

andthelowestupperbound

U

ofprovenfeasibleboxes.Thealgorithmterminates whenthespa ebetween

U

and

L

be omessmallerthanthegiventoleran e

ǫ

.

TheUpper-bounding step(seeAlgorithm 2)performsamultistart strategy where a set of

nbStarts

starting points are provided to a lo al optimization solver.Thesolutions omputedbythelo alsolverare thengivento afun tion

Inf lateAndP rove

whi husesanexisten eproofpro edurebasedontheBorsuk test. However,the proofpro edure mayfail toprovetheexisten eofafeasible point within box x

p

. The most ommon sour e of failure is that the guess providedbythelo alsear hliestoofarfrom thefeasibleregion.

A newupperbounding strategy

Theupperboundingpro eduredes ribedinthepreviousse tionreliesonalo al sear htoprovideaguessed feasiblepointlyingintheneighborhoodofalo al optima.However,theee tsofoatingpoint omputationontheprovidedlo al optimaare hardto predi t. Asaresult, thelo al optimamightlie outsidethe feasibleregionandtheproofpro eduremightfailtobuildaprovenboxaround thispoint.

Weproposehereanewupperboundingstrategywhi hattemptstotake ad-vantageofthesolutionofalinearouterapproximationoftheproblem.Thelower boundpro essusessu hanapproximationto omputeasafelowerboundof

P

. WhentheLPis solved,asolution

x

LP

isalways omputedand,thus,available forfree.Thissolutionbeinganoptimalsolutionofanouterapproximationof

P

, itliesoutsidethefeasibleregion.Thus,

x

LP

isnotafeasiblepoint.Nevertheless,

x

LP

maybeagood startingpointto onsiderforthefollowingreasons:

 At ea h iteration, the bran h and bound pro ess splits the domain of the variables.Thesmallertheboxis,thenearest

x

LP

isfromthea tualoptima of

P

.

 Theproofpro essinatesaboxaroundtheinitialguess.Thispro ess may ompensatetheee t ofthedistan eof

x

LP

from thefeasibleregion. 2

Algorithm2Upperboundingbuild fromtheLPoptimalsolution

x

∗

LP

Fun tionUpperBounding(INx,

x

∗

LP

,

nbStarts

;OUT

S

′

) %

S

′

:listofprovenfeasibleboxes;

nbStarts

:numberofstartingpoints %

x

∗

LP

:theoptimalsolutionoftheLPrelaxationof

P(

x

)

S

′

_{← ∅}

;

x

∗

corr

←

FeasibilityCorre tion(

x

∗

LP

); x

p

←

InateAndProve(

x

∗

corr

,x); if x

p

6= ∅

then

S

′

_←S

′

_∪

x

p

return

S

′

However, while

x

LP

onverges to a feasible point, the pro ess might be quite slow. To speed up the upper bounding pro ess, we have introdu ed a light weight,thoughe ient,pro edurewhi h omputeafeasiblepointfromapoint lyingin theneighborhood ofthefeasibleregion.Thispro edure whi his alled

F easibilityCorrection

willbedetailedinthenextsubse tion.

Algorithm2des ribeshowanupperbound maybebuild fromthesolution ofthelinearproblem usedinthelowerboundingpro edure.

Computing pseudo-feasiblepoints

Thisse tionintrodu esanadaptationoftheNewtonmethodtounder- onstrained systemsofequations andinequalitieswhi hprovidesverya urate approxima-tionsof feasiblepointsat alow omputational ost.Whenthesystemof equa-tions

g(x) = 0

is under- onstrained there is amanifold of solutions.

l(x)

, the linear approximation is still valid in this situation, but the linear system of equations

l(x) = 0

isnowunder- onstrained,andhasthereforeananespa eof solutions.Sowehaveto hooseasolution

x

(1)

ofthelinearizedequation

l(x) = 0

among the anespa e of solutions.As

x

(0)

is supposed to bean approximate solutionof

g(x) = 0

,thebest hoi eis ertainlythesolutionof

l(x) = 0

whi his the losestto

x

(0)

.Thissolution aneasilybe omputedwiththeMoore-Penrose inverse:

x

(1)

_{= x}

(0)

_{− A}

+

g

(x

(0)

)g(x

(0)

)

,where

A

+

g

∈ IR

n×m

istheMoore-Penrose inverseof

A

g

∈ IR

m×n

,thesolutionofthelinearized equationwhi hminimizes

||x

(1)

− x

(0)

||

.Applyingpreviousrelationre ursivelyleadstoasequen eof ve -tors whi h onvergesto asolution loseto the initial approximation,provided that thislatterisa urateenough.

TheMoore-Penroseinverse anbe omputedinseveralways:asingularvalue de omposition anbeused, orinthe asewhere

A

g

hasfullrowrank(whi h is the asefor

A

g

(x

(0)

₎

if

x

(0)

is non-singular)theMoore-Penroseinverse an be omputedusing

A

+

g

= A

T

g

(A

g

A

T

g

)

−

1

.

Inequality onstraints are hanged to equalities by introdu ing sla k vari-ables:

h

j

(x) ≤ 0

⇐⇒

h

j

(x) = −s

2

under-InthisSe tion,we ommenttheresultsoftheexperimentswithournewupper bounding strategy ona signi ant set of ben hmarks.Detailled results an be found in the resar h report ISRN I3S/RR-2008-11-FR

3

). All the ben hmarks ome fromthe olle tionofben hmarksof theCo onutsproje t

4

.We have se-le ted

35

ben hmarks where I os su eeds to nd the global minimum while relying on an unsafe lo al sear h. We did ompare our new upper bounding strategywiththefollowingupperbounding strategies:

S1: Thisstrategydire tlyusestheguessfromthelo alsear h,i.e.thisstrategy usesasimpliedversionofalgorithm1wheretheproofpro edurehasbeen dropped. As a onsequen e, it doesnot suer from the di ulty to prove theexisten eof a feasiblepoint. However,this strategy is unsafe and may produ ewrongresults.

S2: This strategy attempts to provethe existen e of afeasible point within a boxbuildaroundthelo alsear hguess.Here,allprovidedsolutionsaresafe andthesolvingpro essisrigorous.

S3: Ourupperboundingstrategywheretheupperboundingreliesontheoptimal solutionof the problem linear relaxationto build a box proved to hold a feasible point. A all to the orre tion pro edure attempts to ompensate theee t oftheouterapproximation.

S4: This strategy applies the orre tion pro edure to the output of the lo al sear h(toimprovetheapproximatesolutiongivenbyalo alsear h). S5: This strategy mainly diers from S3 by the fa t that it does not all the

orre tionpro edure

S3, our newupperbounding strategy is the best strategy: 31 ben hmarks are nowsolvedwithinthe30stimeout;moreover,almostallben hmarksaresolved in mu h less time and with a greater amount of proven solutions. This new strategyimprovesdrasti allytheperforman eoftheupperboundingpro edure and ompeteswellwithalo alsear h.

Current work aims at improving and generalizing this framework and its implementation.

Referen es

1. YahiaLebbah,ClaudeMi hel,Mi helRueher,DavidDaney,andJean-Pierre Mer-let. E ientandsafeglobal onstraintsforhandlingnumeri al onstraintsystems. SIAMJournalonNumeri alAnalysis,42(5):20762097, 2004.

2. Olivier Lhomme. Consisten y te hniques for numeri CSPs. In Pro eedings of IJCAI'93,pages232238,Chambéry(Fran e),1993.

3. ArnoldNeumaier.Completesear hin ontinuousglobaloptimizationand onstraint satisfa tion. A taNumeri a,2004.

3

http://www.i3s.uni e.fr/%7Emh/RR/2008/RR-08.11-A.GOLDSZTEJN.pdf 4