Lagrangean Relaxation

Suppose we have an integer linear program max{cx : Ax ≤ b, Ax ≤ b, x integral} that becomes substantially easier to solve when omitting some of the constraints Ax ≤b. We write Q:= {x ∈Rⁿ : Ax ≤b,x integral}and assume that we can optimize linear objective functions overQ(for example if conv(Q)= {x: Ax ≤b}). Lagrangean relaxation is a technique to get rid of some troublesome constraints (in our case Ax≤ b). Instead of explicitly enforcing the constraints we modify the objective function in order to punish infeasible solutions. More precisely, instead of optimizing

max{cx :Ax≤b,x∈ Q} (5.7) we consider, for any vectorλ≥0,

L R(λ) := max{cx+λ(b−Ax):x∈Q}. (5.8)

For each λ ≥ 0, L R(λ) is an upper bound for (5.7) which is relatively easy to compute. (5.8) is called theLagrangean relaxationof (5.7), and the components of λare calledLagrange multipliers.

Lagrangean relaxation is a useful technique in nonlinear programming; but here we restrict ourselves to (integer) linear programming.

Of course one is interested in as good an upper bound as possible. Observe that L R(λ) is a convex function. The following procedure (called subgradient optimization) can be used to minimize L R(λ):

Start with an arbitrary vectorλ⁽⁰⁾≥0. In iterationi, givenλ⁽ⁱ⁾, find a vectorx⁽ⁱ⁾ maximizingcx+(λ⁽ⁱ⁾)(b−Ax)overQ(i.e. compute L R(λ⁽ⁱ⁾)). Setλ⁽ⁱ⁺¹⁾:= max{0, λ⁽ⁱ⁾ −ti(b− Ax⁽ⁱ⁾)} for some ti > 0. Polyak [1967] showed that if limi→∞ti =0 and_∞

i=0ti = ∞, then limi→∞L R(λ⁽ⁱ⁾)=min{L R(λ) :λ≥ 0}. For more results on the convergence of subgradient optimization, see (Goffin [1977]).

The problem

min{L R(λ):λ≥0}

is sometimes called the Lagrangean dual of (5.7). The question remains how good this upper bound is. Of course this depends on the structure of the original problem. In Section 21.5 we shall meet an application to the TSP, where La-grangean relaxation is very effective. The following theorem helps to estimate the quality of the upper bound:

Theorem 5.35. (Geoffrion [1974]) Let Q ⊂ Rⁿ be a finite set,c ∈ Rⁿ, A ∈ R^m×n and b ∈ R^m. Suppose that {x ∈ Q : Ax ≤ b}is nonempty. Then the optimum value of the Lagrangean dual ofmax{cx :Ax≤b,x∈ Q}is equal to max{cx: Ax≤b, x∈conv(Q)}.

Proof: By reformulating and using the LP Duality Theorem 3.16 we get min{L R(λ):λ≥0} b,x integral}where {x : Ax ≤ b} is integral, then the Lagrangean dual (when relaxing Ax ≤ b as above) yields the same upper bound as the standard LP

112 5. Integer Programming

relaxation max{cx :Ax≤b, Ax ≤b}. If{x: Ax ≤b}is not integral, the upper bound is in general stronger (but can be difficult to compute). See Exercise 20 for an example.

Lagrangean relaxation can also be used to approximate linear programs. For example, consider theJob Assignment Problem (see Section 1.3, in particular (1.1)). The problem can be rewritten equivalently as

min

Now we apply Lagrangean relaxation and consider L R(λ) := min

Because of its special structure this LP can be solved by a simple combinatorial algorithm (Exercise 22), for arbitraryλ. If we letQbe the set of vertices of P(cf.

Corollary 3.27), then we can apply Theorem 5.35 and conclude that the optimum value of the Lagrangean dual max{L R(λ):λ≥0}equals the optimum of (5.9).

. Prove that PI is not a polyhedron.

∗ 2.Prove the following integer analogue of Carath´eodory’s theorem (Exercise 10 of Chapter 3): For each pointed polyhedral coneC = {x : Ax ≤ 0}, each

4. Let A be an integral m×n-matrix, and let b and c be vectors, and y an optimum solution of max{cx : Ax ≤b,x integral}. Prove that there exists an optimum solutionzof max{cx: Ax ≤b}with||y−z||_∞≤n(A).

(Cook et al. [1986])

5. Prove that each unimodular matrix arises from an identity matrix by unimod-ular transformations.

Hint:Recall the proof of Lemma 5.9.

∗ 6.Prove that there is a polynomial-time algorithm which, given an integral matrix Aand an integral vectorb, finds an integral vectorx withAx =b or decides that none exists.

Hint:See the proofs of Lemma 5.9 and Lemma 5.10.

7. Consider the two systems They clearly define the same polyhedron. Prove that the first one is TDI but the second one is not.

8. Leta be an integral vector andβ a rational number. Prove that the inequality ax≤β is TDI if and only if the components ofa are relatively prime.

9. Let Ax ≤bbe TDI,k∈Nandα >0 rational. Show that ¹_kAx ≤αb is again TDI. Moreover, prove thatαAx ≤αb is not necessarily TDI.

10. Use Theorem 5.24 in order to prove K¨onig’s Theorem 10.2 (cf. Exercise 2 of Chapter 11):

The maximum cardinality of a matching in a bipartite graph equals the mini-mum cardinality of a vertex cover.

is integral for all integral vectorsb.

(Nemhauser and Wolsey [1988])

12. LetG be the digraph({1,2,3,4},{(1,3), (2,4), (2,1), (4,1), (4,3)}), and let F := {{1,2,4},{1,2},{2},{2,3,4},{4}}. Prove that(V(G),F) is cross-free but the one-way cut-incidence matrix ofF is not totally unimodular.

∗ 13.Let G and T be digraphs such that V(G)=V(T)and the undirected graph

Matrices arising this way are called network matrices. Show that the network matrices are precisely the two-way cut-incidence matrices.

114 5. Integer Programming

14. An interval matrix is a 0-1-matrix such that in each row the 1-entries are consecutive. Prove that interval matrices are totally unimodular.

∗ 15.Consider the following interval packing problem: Given a list of intervals [ai,bi], i = 1, . . . ,n with weights c1, . . . ,cn and a number k ∈ N, find a maximum weight subset of the intervals such that no point is contained in more thank of them.

(a) Find an LP formulation (without integrality constraints) of this problem.

(b) What combinatorial meaning has the dual LP? Show how to solve the dual LP by a simple combinatorial algorithm.

(c) Use (b) to obtain a combinatorial algorithm for the interval packing prob-lem. What running time do you obtain?

16. Let P := {(x,y) ∈ R² : y = √

19. In this exercise we apply Lagrangean relaxation to linear equation systems.

Let Q be a finite set of vectors inRⁿ,c∈Rⁿ and A∈ R^m×n andb ∈R^m. Prove that

min5

max{cx+λ(b−Ax):x∈ Q}:λ∈R^m6

= max{cy:y∈conv(Q), Ay=b}.

20. Consider the following facility location problem: Given a set ofn customers with demands d1, . . . ,dn, and m optional facilities each of which can be opened or not. For each facilityi =1, . . . ,m we have a cost fi for opening it, a capacityui and a distance ci j to each customer j =1, . . . ,n. The task is to decide which facilities should be opened and to assign each customer to an open facility. The total demand of the customers assigned to one facility must not exceed its capacity. The objective is to minimize the facility opening costs plus the sum of the distances of each customer to its facility. In terms ofInteger Programmingthe problem can be formulated as

min

ixi j =1 for all j. Which Lagrangean dual yields a tighter bound?

Note:Both Lagrangean relaxations can be dealt with: see Exercise 7 of Chapter 17.

∗ 21.Consider the Uncapacitated Facility Location Problem: given numbers distributed among the customers such that no subsetSpays more thanc(S). In other words: are there numbersp1, . . . ,pnsuch thatn i.e. if the integrality conditions can be left out.

Hint:Apply Lagrangean relaxation to the above LP. For each set of Lagrange multipliers decompose the resulting minimization problem to minimization problems over polyhedral cones. What are the vectors generating these cones?

(Goemans and Skutella [2004])

22. Describe a combinatorial algorithm (without usingLinear Programming) to solve (5.10) for arbitrary (but fixed) Lagrange multipliers λ. What running time can you achieve?

References

General Literature:

Bertsimas, D., and Weismantel, R. [2005]: Optimization Over Integers. Dynamic Ideas, Belmont 2005

Cook, W.J., Cunningham, W.H., Pulleyblank, W.R., and Schrijver, A. [1998]: Combinato-rial Optimization. Wiley, New York 1998, Chapter 6

Nemhauser, G.L., and Wolsey, L.A. [1988]: Integer and Combinatorial Optimization. Wiley, New York 1988

Schrijver, A. [1986]: Theory of Linear and Integer Programming. Wiley, Chichester 1986 Wolsey, L.A. [1998]: Integer Programming. Wiley, New York 1998

116 5. Integer Programming Cited References:

Boyd, E.A. [1997]: A fully polynomial epsilon approximation cutting plane algorithm for solving combinatorial linear programs containing a sufficiently large ball. Operations Research Letters 20 (1997), 59–63

Chv´atal, V. [1973]: Edmonds’ polytopes and a hierarchy of combinatorial problems. Dis-crete Mathematics 4 (1973), 305–337

Cook, W. [1983]: Operations that preserve total dual integrality. Operations Research Letters 2 (1983), 31–35

Cook, W., Fonlupt, J., and Schrijver, A. [1986]: An integer analogue of Carath´eodory’s theorem. Journal of Combinatorial Theory B 40 (1986), 63–70

Cook, W., Gerards, A., Schrijver, A., and Tardos, ´E. [1986]: Sensitivity theorems in integer linear programming. Mathematical Programming 34 (1986), 251–264

Dantzig, G., Fulkerson, R., and Johnson, S. [1954]: Solution of a large-scale traveling-salesman problem. Operations Research 2 (1954), 393–410

Edmonds, J., and Giles, R. [1977]: A min-max relation for submodular functions on graphs.

In: Studies in Integer Programming; Annals of Discrete Mathematics 1 (P.L. Hammer, E.L. Johnson, B.H. Korte, G.L. Nemhauser, eds.), North-Holland, Amsterdam 1977, pp.

185–204

Eisenbrand, F., and Schulz, A.S. [2003]: Bounds on the Chv´atal rank of polytopes in the 0/1-cube. Combinatorica 23 (2003), 245–261

Fulkerson, D.R. [1971]: Blocking and anti-blocking pairs of polyhedra. Mathematical Pro-gramming 1 (1971), 168–194

Geoffrion, A.M. [1974]: Lagrangean relaxation for integer programming. Mathematical Programming Study 2 (1974), 82–114

Giles, F.R., and Pulleyblank, W.R. [1979]: Total dual integrality and integer polyhedra.

Linear Algebra and Its Applications 25 (1979), 191–196

Ghouila-Houri, A. [1962]: Caract´erisation des matrices totalement unimodulaires. Comptes Rendus Hebdomadaires des S´eances de l’Acad´emie des Sciences (Paris) 254 (1962), 1192–1194

Goemans, M.X., and Skutella, M. [2004]: Cooperative facility location games. Journal of Algorithms 50 (2004), 194–214

Goffin, J.L. [1977]: On convergence rates of subgradient optimization methods. Mathemat-ical Programming 13 (1977), 329–347

Gomory, R.E. [1958]: Outline of an algorithm for integer solutions to linear programs.

Bulletin of the American Mathematical Society 64 (1958), 275–278

Gomory, R.E. [1963]: An algorithm for integer solutions of linear programs. In: Recent Advances in Mathematical Programming (R.L. Graves, P. Wolfe, eds.), McGraw-Hill, New York, 1963, pp. 269–302

Graver, J.E. [1975]: On the foundations of linear and integer programming I. Mathematical Programming 9 (1975), 207–226

Harvey, W. [1999]: Computing two-dimensional integer hulls. SIAM Journal on Computing 28 (1999), 2285–2299

Hoffman, A.J. [1974]: A generalization of max flow-min cut. Mathematical Programming 6 (1974), 352–359

Hoffman, A.J., and Kruskal, J.B. [1956]: Integral boundary points of convex polyhedra. In:

Linear Inequalities and Related Systems; Annals of Mathematical Study 38 (H.W. Kuhn, A.W. Tucker, eds.) Princeton University Press, Princeton 1956, 223–246

Lasserre, J.B. [2004]: The integer hull of a convex rational polytope. Discrete & Compu-tational Geometry 32 (2004), 129–139

Meyer, R.R. [1974]: On the existence of optimal solutions to integer and mixed-integer programming problems. Mathematical Programming 7 (1974), 223–235

Polyak, B.T. [1967]: A general method for solving extremal problems. Doklady Akademii Nauk SSSR 174 (1967), 33–36 [in Russian]. English translation: Soviet Mathematics Doklady 8 (1967), 593–597

Schrijver, A. [1980]: On cutting planes. In: Combinatorics 79; Part II; Annals of Discrete Mathematics 9 (M. Deza, I.G. Rosenberg, eds.), North-Holland, Amsterdam 1980, pp.

291–296

Schrijver, A. [1981]: On total dual integrality. Linear Algebra and its Applications 38 (1981), 27–32

Schrijver, A. [1983]: Packing and covering of crossing families of cuts. Journal of Combi-natorial Theory B 35 (1983), 104–128

Seymour, P.D. [1980]: Decomposition of regular matroids. Journal of Combinatorial Theory B 28 (1980), 305–359

Veinott, A.F., Jr., and Dantzig, G.B. [1968]. Integral extreme points. SIAM Review 10 (1968), 371–372

Wolsey, L.A. [1981]: Theb-hull of an integer program. Discrete Applied Mathematics 3 (1981), 193–201