Input: A numbern∈N,n≥2. A vector c∈Qn. A number 0< ≤1.
A polytope P ⊆ Rn given by an oracle for the Weak Separation Problemfor P,cand 2.
x0∈Qn andr,R∈Q+ such that B(x0,r)⊆ P⊆B(x0,R). Output: A vector y∗ ∈P withcy∗≥max{cx :x∈ P} −.
84 4. Linear Programming Algorithms
1 Set R:=max{R,2},r :=min{r,1}andγ :=max{||c||,1}. Set N :=5n2 ln4Rr2γ
. Set y∗:=x0.
2 Run theEllipsoid Method, withak in2 being computed as follows:
Run the oracle for theWeak Separation Problemwithy=xk. Ifit returns a y∈ P withcy≤cy+2 then:
Ifcy>cy∗ then sety∗ :=y. Setak:=c.
Ifit returns ad ∈Qn withd x<d yfor all x∈ P then:
Setak:= −d.
Theorem 4.19. TheGro¨tschel-Lovasz-Schrijver Algorithm´ correctly solves theWeak Optimization Problem. Its running time is bounded by
O
n6α2+n4αf(size(c),size(),nsize(x0)+n3α) ,
whereα=logRr2γ and f(size(c),size(),size(y)) is an upper bound of the run-ning time of the oracle for theWeak Separation Problemfor Pwith inputc, ,y.
Proof: (Gr¨otschel, Lov´asz and Schrijver [1981]) The running time in each of the N =O(n2α)iterations of theEllipsoid Methodis O(n2(n2α+size(R)+ size(x0)+q))plus one oracle call, whereq is the size of the output of the oracle.
As size(y)≤n(size(x0)+size(R)+N)by Lemma 4.13, the overall running time is O(n4α(n2α+size(x0)+ f(size(c),size(),nsize(x0)+n3α))), as stated.
By Lemma 4.14, we have
{x∈ P :cx ≥cy∗+
2} ⊆ EN.
Let z be an optimum solution of max{cx : x ∈ P}. We may assume that cz >
cy∗+2; otherwise we are done.
Consider the convex hullUofzand the(n−1)-dimensional ballB(x0,r)∩{x: cx = cx0}(see Figure 4.2). We have U ⊆ P and hence U := {x ∈ U : cx ≥ cy∗+2}is contained inEN. The volume ofU is
volume(U) = volume(U)
cz−cy∗−2 cz−cx0
n
= Vn−1rn−1cz−cx0
n||c||
cz−cy∗−2 cz−cx0
n
,
whereVn denotes the volume of then-dimensional unit ball. Since volume(U)≤ volume(EN), and Lemma 4.15 yields
volume(EN) ≤ e−5nNE0 = e−5nNVnRn, we have
{x:cx=cy∗+2} {x:cx=cx0} r
r x0
z
U
Fig. 4.2.
cz−cy∗−
2 ≤ e−5nN2R
Vn(cz−cx0)n−1n||c||
Vn−1rn−1
1n . Sincecz−cx0≤ ||c|| · ||z−x0|| ≤ ||c||Rwe obtain
cz−cy∗−
2 ≤ ||c||e−5nN2R
nVnRn−1 Vn−1rn−1
1n
< 2||c||e−5nN2 R2 r ≤
2. 2 Of course we are usually interested in the exact optimum. To achieve this, we need some assumption on the size of the vertices of the polytope.
Lemma 4.20. Letn∈N, letP ⊆Rnbe a rational polytope, and letx0∈Qnbe a point in the interior ofP. LetT ∈Nsuch thatsize(x0)≤logT andsize(x)≤logT for all verticesx ofP. Then B(x0,r)⊆ P⊆ B(x0,R), wherer := n1T−379n2 and R:=2nT.
Moreover, let K :=2T2n+1. Letc ∈ Zn, and define c := Knc+(1,K, . . . , Kn−1). Then max{cx : x ∈ P} is attained by a unique vector x∗, for all other vertices yof Pwe havec(x∗−y) >T−2n, andx∗ is also an optimum solution of max{cx :x∈P}.
Proof: For any vertexxof Pwe have||x|| ≤nT and||x0|| ≤nT, so||x−x0|| ≤ 2nT andx∈B(x0,R).
To show that B(x0,r)⊆ P, let F = {x∈ P:ax =β}be a facet of P, where by Lemma 4.5 we may assume that size(a)+size(β) <75n2logT. Suppose there is a point y∈F with||y−x0||<r. Then
|ax0−β| = |ax0−ay| ≤ ||a|| · ||y−x0|| < n2size(a)r ≤ T−304n2 But on the other hand the size ofax0−β can by estimated by
86 4. Linear Programming Algorithms
size(ax0−β) ≤ 4(size(a)+size(x0)+size(β))
≤ 300n2logT +4 logT ≤ 304n2logT.
Since ax0 = β (x0 is in the interior of P), this implies |ax0−β| ≥ T−304n2, a contradiction.
To prove the last statements, letx∗ be a vertex of P maximizingcx, and let y be another vertex of P. By the assumption on the size of the vertices of P we may writex∗−y= 1αz, whereα∈ {1,2, . . . ,T2n−1}andz is an integral vector whose components are less than K2. Then
0 ≤ c(x∗−y) = 1 indeed maximizescx over P. Moreover, sincez=0, we obtain
c(x∗−y) ≥ 1
Givenn,c,x0,T and a polynomial-time oracle for theSeparation Problem forP, a vertexx∗ofPattainingmax{cx:x∈ P}can be found in time polynomial inn,logT andsize(c).
Proof: (Gr¨otschel, Lov´asz and Schrijver [1981]) We first use the Gro¨ tschel-Lovasz-Schrijver Algorithm´ to solve the Weak Optimization Problem; we setc,rand Raccording to Lemma 4.20 and:= 4nT12n+3. (We first have to make c integral by multiplying with the product of its denominators; this increases its size by at most a factor 2n.)
TheGro¨tschel-Lovasz-Schrijver Algorithm´ returns a vector y∈ P with cy≥cx∗−, wherex∗is the optimum solution of max{cx:x ∈P}. By Theorem bound of the running time of the oracle for theSeparation Problemfor P. Since size(c)≤6n2logT+2 size(c), we have an overall running time that is polynomial
Now – using Lemma 4.20 – ≥ c(x∗−y) =
k i=1
λic x∗−xi
>
k i=1
λiT−2n = (1−λ0)T−2n, so 1−λ0< T2n. We conclude that
||y−x∗|| ≤ k
i=1
λi||xi−x∗|| ≤ (1−λ0)R < 2nT2n+1 ≤ 1 2T2. So when rounding each entry ofyto the next rational number with denominator at mostT, we obtainx∗. The rounding can be done in polynomial time by Theorem
4.8. 2
We have proved that, under certain assumptions, optimizing over a polytope can be done whenever there is a separation oracle. We close this chapter by noting that the converse is also true. We need the concept of polarity: If X ⊆Rn, we define thepolarof X to be the set
X◦ := {y∈Rn :yx≤1 for allx∈ X}.
When applied to full-dimensional polytopes, this operation has some nice proper-ties:
Theorem 4.22. Let P be a polytope inRn with0in the interior. Then:
(a) P◦ is a polytope with0in the interior;
(b) (P◦)◦ =P;
(c) xis a vertex of P if and only ifxy≤1is a facet-defining inequality of P◦. Proof: (a): Let P be the convex hull ofx1, . . . ,xk (cf. Theorem 3.26). By def-inition, P◦ = {y∈ Rn : yxi ≤1 for alli ∈ {1, . . . ,k}}, i.e. P◦ is a polyhedron and the facet-defining inequalities of P◦ are given by vertices of P. Moreover, 0 is in the interior of P◦ because 0 satisfies all of the finitely many inequalities strictly. Suppose P◦ is unbounded, i.e. there exists aw∈Rn\ {0}withαw∈ P◦ for all α > 0. Thenαwx ≤ 1 for all α >0 and all x ∈ P, so wx ≤ 0 for all x ∈P. But then 0 cannot be in the interior of P.
(b): Trivially, P ⊆(P◦)◦. To show the converse, suppose that z∈(P◦)◦\P. Then, by Theorem 3.23, there is an inequality cx ≤ δ satisfied by all x ∈ P but not by z. We have δ >0 since 0 is in the interior of P. Then 1δc∈ P◦ but
1
δcz>1, contradicting the assumption thatz∈(P◦)◦.
(c): We have already seen in (a) that the facet-defining inequalities of P◦ are given by vertices of P. Conversely, if x1, . . . ,xk are the vertices of P, then P¯ :=conv({12x1,x2, . . . ,xk})=P, and 0 is in the interior of P. Now (b) implies¯ P¯◦ = P◦. Hence {y ∈ Rn : yx1 ≤ 2, yxi ≤ 1(i =2, . . . ,k)} = ¯P◦ = P◦ = {y∈Rn : yxi ≤1(i =1, . . . ,k)}. We conclude that x1y≤1 is a facet-defining
inequality of P◦. 2
Now we can prove:
88 4. Linear Programming Algorithms
Theorem 4.23. Letn ∈Nand y ∈Qn. LetP ⊆Rn be a rational polytope, and letx0 ∈Qn be a point in the interior of P. LetT ∈Nsuch that size(x0)≤logT andsize(x)≤logT for all verticesxofP.
Givenn, y,x0,T and an oracle which for any givenc∈Qn returns a vertex x∗ of P attainingmax{cx : x ∈ P}, we can solve theSeparation Problemfor P and yin time polynomial inn,logT andsize(y). Indeed, in the case y∈/ P we can find a facet-defining inequality of P that is violated byy.
Proof: Consider Q:= {x−x0 :x ∈ P}and its polar Q◦. If x1, . . . ,xk are the vertices of P, we have
Q◦= {z∈Rn :z(xi−x0)≤1 for alli ∈ {1, . . . ,k}}.
By Theorem 4.4 we have size(z)≤4n(2nlogT+3n)≤20n2logT for all vertices z of Q◦.
Observe that theSeparation Problem for P and yis equivalent to the Sep-aration Problem for Q andy−x0. Since by Theorem 4.22
Q = (Q◦)◦ = {x:zx≤1 for allz∈ Q◦},
theSeparation ProblemforQandy−x0is equivalent to solving max{(y−x0)x: x ∈Q◦}. Since each vertex of Q◦ corresponds to a facet-defining inequality of Q (and thus ofP), it remains to show how to find a vertex attaining max{(y−x0)x: x ∈Q◦}.
To do this, we apply Theorem 4.21 to Q◦. By Theorem 4.22, Q◦ is full-dimensional with 0 in the interior. We have shown above that the size of the vertices of Q◦ is at most 20n2logT. So it remains to show that we can solve the Separation Problem for Q◦ in polynomial time. However, this reduces to the optimization problem for Qwhich can be solved using the oracle for optimizing
over P. 2
We finally mention that a new algorithm which is faster than theEllipsoid Method and also implies the equivalence of optimization and separation has been proposed by Vaidya [1996]. However, this algorithm does not seem to be of practical use either.
Exercises
1. Let A be a nonsingular rational n ×n-matrix. Prove that size(A−1) ≤ 4n2size(A).
∗ 2.Letn ≥2,c∈Rn and y1, . . . ,yk∈ {−1,0,1}n such that 0<cyi+1≤ 12cyi
fori =1, . . . ,k−1. Prove that then k≤3nlogn.
Hint: Consider the linear program max{ykx : (yi −2yi+1)x ≥ 0,ykx = 1,x≥0}.
(M. Goemans)
3. Consider the numbershi in theContinued Fraction Expansion. Prove that
Conclude that the number of iterations of theContinued Fraction Expan-sionis O(logq).
(Gr¨otschel, Lov´asz and Schrijver [1988])
4. Show that Gaussian Eliminationcan be made a strongly polynomial-time algorithm.
Hint:First assume that A is integral. Recall the proof of Theorem 4.10 and observe that we can choosed as the common denominator of the entries.
(Edmonds [1967]) 10 of Chapter 3). Show that all computations can be performed inO(n3)time.
Hint:Run Gaussian Eliminationwith the matrix A∈R(l+1)×k whosei-th facet-defining. Suppose that we know an optimum basic solutionx∗. Show how to use this to find an optimum solution to the dual LP min{yb:y A=c, y≥0} usingGaussian Elimination. What running time can you obtain?
∗ 7.LetAbe a symmetric positive definiten×n-matrix. Letv1, . . . , vnben (The eigenvectors correspond to the axes of symmetry of the ellipsoid.) Conclude that volume(E(A,0))=√
detA volume(B(0,1)).
8. LetE(A,x)⊆Rnbe an ellipsoid anda∈Rn, and letE(A,x))be as defined on page 75. Prove that{z∈ E(A,x):az≥ax} ⊆E(A,x).
9. Prove that the algorithm of Theorem 4.18 solves a linear program max{cx : Ax ≤b}inO((n+m)9(size(A)+size(b)+size(c))2)time.
10. Show that the assumption thatP is bounded can be omitted in Theorem 4.21.
One can detect if the LP is unbounded and otherwise find an optimum solution.
90 4. Linear Programming Algorithms
∗ 11.Let P ⊆ R3 be a 3-dimensional polytope with 0 in its interior. Consider again the graphG(P)whose vertices are the vertices of P and whose edges correspond to the 1-dimensional faces ofP(cf. Exercises 13 and 14 of Chapter 3). Show thatG(P◦)is the planar dual ofG(P).
Note:Steinitz [1922] proved that for every simple 3-connected planar graph G there is a 3-dimensional polytope P withG=G(P).
12. Prove that the polar of a polyhedron is always a polyhedron. For which poly-hedra P is(P◦)◦=P?
References
General Literature:
Gr¨otschel, M., Lov´asz, L., and Schrijver, A. [1988]: Geometric Algorithms and Combina-torial Optimization. Springer, Berlin 1988
Padberg, M. [1995]: Linear Optimization and Extensions. Springer, Berlin 1995
Schrijver, A. [1986]: Theory of Linear and Integer Programming. Wiley, Chichester 1986 Cited References:
Bland, R.G., Goldfarb, D., and Todd, M.J. [1981]: The ellipsoid method: a survey. Opera-tions Research 29 (1981), 1039–1091
Edmonds, J. [1967]: Systems of distinct representatives and linear algebra. Journal of Re-search of the National Bureau of Standards B 71 (1967), 241–245
Frank, A., and Tardos, ´E. [1987]: An application of simultaneous Diophantine approxima-tion in combinatorial optimizaapproxima-tion. Combinatorica 7 (1987), 49–65
G´acs, P., and Lov´asz, L. [1981]: Khachiyan’s algorithm for linear programming. Mathe-matical Programming Study 14 (1981), 61–68
Gr¨otschel, M., Lov´asz, L., and Schrijver, A. [1981]: The ellipsoid method and its conse-quences in combinatorial optimization. Combinatorica 1 (1981), 169–197
Iudin, D.B., and Nemirovskii, A.S. [1976]: Informational complexity and effective methods of solution for convex extremal problems. Ekonomika i Matematicheskie Metody 12 (1976), 357–369 [in Russian]
Karmarkar, N. [1984]: A new polynomial-time algorithm for linear programming. Combi-natorica 4 (1984), 373–395
Karp, R.M., and Papadimitriou, C.H. [1982]: On linear characterizations of combinatorial optimization problems. SIAM Journal on Computing 11 (1982), 620–632
Khachiyan, L.G. [1979]: A polynomial algorithm in linear programming [in Russian]. Dok-lady Akademii Nauk SSSR 244 (1979) 1093–1096. English translation: Soviet Mathe-matics Doklady 20 (1979), 191–194
Khintchine, A. [1956]: Kettenbr¨uche. Teubner, Leipzig 1956
Padberg, M.W., and Rao, M.R. [1981]: The Russian method for linear programming III:
Bounded integer programming. Research Report 81-39, New York University 1981 Shor, N.Z. [1977]: Cut-off method with space extension in convex programming problems.
Cybernetics 13 (1977), 94–96
Steinitz, E. [1922]: Polyeder und Raumeinteilungen. Enzyklop¨adie der Mathematischen Wissenschaften, Band 3 (1922), 1–139
Tardos, ´E. [1986]: A strongly polynomial algorithm to solve combinatorial linear programs.
Operations Research 34 (1986), 250–256
Vaidya, P.M. [1996]: A new algorithm for minimizing convex functions over convex sets.
Mathematical Programming 73 (1996), 291–341
In this chapter, we consider linear programs with integrality constraints: