Approche décomposition de Dantzig-Wolfe en programmation entière:
apport de l’ optimisation convexe
Universit ´e Bordeaux 1 INRIA - Rhones-Aples Universit ´e de Gen `eve Olivier Briant Claude Lemar ´echal Claude Tadonki
Philippe Meurdesoif Jean-Philippe Vial
Krystel Monneris Cesar Beltran
Sophie Michel Nancy Perrot Franc¸ois Vanderbeck
Funded by Inria New Investigation Grant
“Convex Optimization and Dantzig-Wolfe Decomposition”
http://www.math.u-bordeaux.fr/MAB/ODW/
PART 1: Background
Dantzig-Wolfe Decomposition
BaPCod: our projet of building a generic
code
Decomposition: Why
Divide and Conquer
complexity
size
+ + ...
Decomposition: Why
Divide and Conquer
+ + ...
complexity
size
Exploit the structure
Decomposition: When
Difficult Constraints
Decomposition: When
Difficult Constraints
Decomposition: When
Difficult Constraints Ex: The Travelling Salesman Prob.
Decomposition: When
Difficult Constraints Linking Constraints
... . .. ...
Decomposition: When
Difficult Constraints Linking Constraints
Multiple Sub-Systems ( variable splitting)
Decomposition: How
difficult
nice
Lagrangian relaxation, Lagr. Dual, Subgradient Algorithm
Lagrangian:
Dual function:
Dual problem:
Decomposition: How
difficult
nice
Lagrangian relaxation, Lagr. Dual, Subgradient Algorithm
Lagrangian:
Dual function:
Dual problem:
Decomposition: How
difficult
nice
Lagrangian relaxation, Lagr. Dual, Subgradient Algorithm Reformulation (Variable Redefinition)
LP
LP
PSfrag replacements
LP
LP
LP
Decomposition: How
difficult
nice
Lagrangian relaxation, Lagr. Dual, Subgradient Algorithm Reformulation (Variable Redefinition)
Dynamic Cut Generation (Separation Sub-Problem)
LP
LP
PSfrag replacements
LP
LP
Decomposition: How
difficult
nice
Lagrangian relaxation, Lagr. Dual, Subgradient Algorithm Reformulation (Variable Redefinition)
Dynamic Cut Generation (Separation Sub-Problem)
LP
LP
PSfrag replacements
LP
LP
Decomposition: How
difficult
nice
Lagrangian relaxation, Lagr. Dual, Subgradient Algorithm Reformulation (Variable Redefinition)
Dynamic Cut Generation (Separation Sub-Problem)
LP
LP
PSfrag replacements
LP
LP
Decomposition: How
difficult
nice
Lagrangian relaxation, Lagr. Dual, Subgradient Algorithm Reformulation (Variable Redefinition)
Dynamic Cut Generation (Separation Sub-Problem)
Dynamic Column Generation (Optimization Sub-Problem)
Xlp
Decomposition: How
difficult
nice
Lagrangian relaxation, Lagr. Dual, Subgradient Algorithm Reformulation (Variable Redefinition)
Dynamic Cut Generation (Separation Sub-Problem)
Dynamic Column Generation (Optimization Sub-Problem)
Xlp
Decomposition: How
difficult
nice
Lagrangian relaxation, Lagr. Dual, Subgradient Algorithm Reformulation (Variable Redefinition)
Dynamic Cut Generation (Separation Sub-Problem)
Dynamic Column Generation (Optimization Sub-Problem)
Xlp
Decomposition: How
difficult
nice
Lagrangian relaxation, Lagr. Dual, Subgradient Algorithm Reformulation (Variable Redefinition)
Dynamic Cut Generation (Separation Sub-Problem)
Dynamic Column Generation (Optimization Sub-Problem)
Xlp
Decomposition: Output
difficult
nice
1. A solution method that exploits our hability to “treat” a subproblem
2. A Dual Bound that is typically better than LP bound At best, the Lagrangian Dual Bound:
While the LP relaxation bound is LP
PSfrag replacements
LP
PSfrag replacements
LP
Decomposition: Output
difficult
nice
1. A solution method that exploits our hability to “treat” a subproblem
2. A Dual Bound that is typically better than LP bound
At best, the Lagrangian Dual Bound:
While the LP relaxation bound is LP
LP
PSfrag replacements
LP
Decomposition: Output
difficult
nice
1. A solution method that exploits our hability to “treat” a subproblem
2. A Dual Bound that is typically better than LP bound
At best, the Lagrangian Dual Bound:
While the LP relaxation bound is LP
PSfrag replacements
LP
PSfrag replacements
LP
Dantzig-Wolfe Decomposition/ Reformulation
Apply the change of variables:
Then,
to solve by column generation combined with branch-and-bound:
i.e. by branch-and-price
Resoudre le Maitre Generer une colonne
Resoudre le Maitre Generer une colonne
Resoudre le Maitre Generer une colonne
PSfrag replacements
Dantzig-Wolfe Decomposition/ Reformulation
Apply the change of variables:
Then,
to solve by column generation combined with branch-and-bound:
i.e. by branch-and-price
Resoudre le Maitre Generer une colonne
Resoudre le Maitre Generer une colonne
Resoudre le Maitre Generer une colonne
PSfrag replacements
BaPCod : a generic Branch-And-Price Code
The Column Generation approach has proved very successful in practice
However, restricted to experts due to the difficulty of implementing it
‘tool-box” codes : MINTO, ABACUS, BCP, or MAESTRO Our aim is to develop a ‘black-box” implementation (in the same way as Cplex or Xpress provide implementation of Branch-and-Bound)
The user provides a MIP formulation of its problem
The user specifies what constraints make a subproblem The code reformulates the problem for column generation The code solves it with a default branch-and-price
implementation
BaPCod : a generic Branch-And-Price Code
The Column Generation approach has proved very successful in practice
However, restricted to experts due to the difficulty of implementing it
‘tool-box” codes : MINTO, ABACUS, BCP, or MAESTRO Our aim is to develop a ‘black-box” implementation (in the same way as Cplex or Xpress provide implementation of Branch-and-Bound)
The user provides a MIP formulation of its problem
The user specifies what constraints make a subproblem The code reformulates the problem for column generation The code solves it with a default branch-and-price
implementation
BaPCod : a generic Branch-And-Price Code
The Column Generation approach has proved very successful in practice
However, restricted to experts due to the difficulty of implementing it
‘tool-box” codes : MINTO, ABACUS, BCP, or MAESTRO Our aim is to develop a ‘black-box” implementation (in the same way as Cplex or Xpress provide implementation of Branch-and-Bound)
The user provides a MIP formulation of its problem
The user specifies what constraints make a subproblem The code reformulates the problem for column generation The code solves it with a default branch-and-price
implementation
BaPCod : motivations
Practical:
Provides a starting point to try out a column generation approach.
Allows to test different decompositions: just change the subproblem definition.
Research Wise:
Demands to generalize ad-hoc features to turn them into generic tools: initialization, preprocessing, branching, . . . Any algorithmic development is made available for all
applications
Permits to test algorithmic features across applications
Future Outcome:
Paves the way for future integration of column generation techniques in commercial MIP solvers
BaPCod : motivations
Practical:
Provides a starting point to try out a column generation approach.
Allows to test different decompositions: just change the subproblem definition.
Research Wise:
Demands to generalize ad-hoc features to turn them into generic tools: initialization, preprocessing, branching, . . . Any algorithmic development is made available for all
applications
Permits to test algorithmic features across applications
Future Outcome:
Paves the way for future integration of column generation techniques in commercial MIP solvers
BaPCod : motivations
Practical:
Provides a starting point to try out a column generation approach.
Allows to test different decompositions: just change the subproblem definition.
Research Wise:
Demands to generalize ad-hoc features to turn them into generic tools: initialization, preprocessing, branching, . . . Any algorithmic development is made available for all
applications
Permits to test algorithmic features across applications
Future Outcome:
Paves the way for future integration of column generation techniques in commercial MIP solvers
PART 2: Column Generation
Standard Algorithm
Primal/Dual point of view
Stabilization
Dantzig-Wolfe Reformulation: LP relax. and dual
Original P:
Master IP:
Relaxed P:
Master LP:
Dual Master:
Lagrangian D: with
Dantzig-Wolfe Reformulation: LP relax. and dual
Original P:
Master IP:
Relaxed P:
Master LP:
Dual Master:
Lagrangian D: with
Dantzig-Wolfe Reformulation: LP relax. and dual
Original P:
Master IP:
Relaxed P:
Master LP:
Dual Master:
Standard column generation: primal view
Master Problem
min
s.t.
Restricted Master Problem (iteration ):
min s.t.
Subproblem (Oracle) :
If , add to . Otherwise, STOP.
restricted master Lp values
Master LP value iteration
Standard column generation: primal view
Master Problem
min
s.t.
Restricted Master Problem (iteration ):
min
s.t.
Subproblem (Oracle) :
If , add to . Otherwise, STOP.
restricted master Lp values
Master LP value iteration
Standard column generation: primal view
Master Problem
min
s.t.
Restricted Master Problem (iteration ):
min
s.t.
Subproblem (Oracle) :
If , add to . Otherwise, STOP.
restricted master Lp values
Master LP value iteration
Standard column generation: primal view
Master Problem
min
s.t.
Restricted Master Problem (iteration ):
min
s.t.
Subproblem (Oracle) :
If , add to . Otherwise, STOP.
restricted master Lp values
Master LP value iteration
Standard column generation: primal view
Master Problem
min
s.t.
Restricted Master Problem (iteration ):
min
s.t.
Subproblem (Oracle) :
If
, add to
. Otherwise, STOP.
restricted master Lp values
Master LP value iteration
Standard column generation: primal view
Master Problem
min
s.t.
Restricted Master Problem (iteration ):
min
s.t.
Subproblem (Oracle) :
If
, add to
. Otherwise, STOP.
restricted master Lp values
Master LP value iteration
Standard column generation: dual view
Restricted Master Dual:
Approxim Dual Function : Subproblem (Oracle) :
Standard column generation: dual view
Restricted Master Dual:
Approxim Dual Function :
Subproblem (Oracle) :
u
PSfrag replacements
Standard column generation: dual view
Restricted Master Dual:
Approxim Dual Function :
Subproblem (Oracle) :
PSfrag replacements
Standard column generation: dual view
Restricted Master Dual:
Approxim Dual Function :
Subproblem (Oracle) :
u
PSfrag replacements
Standard column generation: dual view
Restricted Master Dual:
Approxim Dual Function :
Subproblem (Oracle) :
PSfrag replacements
Standard column generation: dual view
Restricted Master Dual:
Approxim Dual Function :
Subproblem (Oracle) :
u
Standard column generation: dual view
Restricted Master Dual:
Approxim Dual Function :
Subproblem (Oracle) :
PSfrag replacements
Standard column generation: primal/ dual view
At iteration k, one has
A Primal Bound on the master:
A Dual Bound on the master:
PSfrag replacements
restricted master Lp values
intermediate Lagrangian bounds
Master LP value iteration
Standard column generation: primal/ dual view
At iteration k, one has
A Primal Bound on the master:
A Dual Bound on the master:
restricted master Lp values
Master LP value iteration
Slow convergence of Kelley’s algorithm
u
PSfrag replacements
Slow convergence of Kelley’s algorithm
u
PSfrag replacements
Slow convergence of Kelley’s algorithm
u
PSfrag replacements
Slow convergence of Kelley’s algorithm
u
PSfrag replacements
Slow convergence of Kelley’s algorithm
u
PSfrag replacements
Slow convergence of Kelley’s algorithm
u
PSfrag replacements
Slow convergence of Kelley’s algorithm
u
PSfrag replacements
Slow convergence of Kelley’s algorithm
u
PSfrag replacements
Instability: observed behaviors
u
PSfrag replacements
restricted master Lp values
intermediate Lagrangian bounds
Master LP value iteration
PSfrag replacements
Tailing-off
Instability: observed behaviors
PSfrag replacementsu
restricted master Lp values
intermediate Lagrangian bounds
Master LP value iteration
PSfrag replacements
Tailing-off Heading-in
Instability: observed behaviors
PSfrag replacementsu
restricted master Lp values
intermediate Lagrangian bounds
Master LP value iteration
PSfrag replacements
Tailing-off Heading-in
Degeneracy
Instability: observed behaviors
u
u0 u1
restricted master Lp values
intermediate Lagrangian bounds
Master LP value iteration
PSfrag replacements
Tailing-off Heading-in
Degeneracy
Bang-Bang Behavior
Instability: observed behaviors
u
u0 u2 u1
restricted master Lp values
intermediate Lagrangian bounds
Master LP value iteration
PSfrag replacements
Tailing-off Heading-in
Degeneracy
Bang-Bang Behavior
Instability: observed behaviors
u
u0 u2 u3 u1
restricted master Lp values
intermediate Lagrangian bounds
Master LP value iteration
PSfrag replacements
Tailing-off Heading-in
Degeneracy
Bang-Bang Behavior
