FOR LINEAR PROGRAMING
H. ROUMILI
Communicated by the former editorial board
In this paper we propose a weighted-path-following interior-point algorithm for linear programming, where the introduction of the weighted vector r ensure that the initial strictly feasible solution obtained by the Karmarkar Algorithm became on the central path. We present an analysis of Newton’s method based on aproximity measure for our algorithm, which converges in most 6(l+l0)√
n iterations. Finally, some strategies are used for testing our algorithm.
AMS 2010 Subject Classification: 65K05, 90C05.
Key words: weighted-path-following method, linear programming, initial central point, the central path, convergence analysis.
1. INTRODUCTION
Interior point methods (IP M s) are among the most effective methods for solving wide classes of optimization problems. Since the seminal work of Karmarkar [2], many researchers have proposed and analyzed various IP M s for linear optimisation (LO) and a large amount of results have been reported.
An interesting fact is that almost all known polynomial time variants ofIP M s use the central path as a guideline to the optimal set, and some variant of Newton’s method is used to follow the central path approximately. Therefore, the theoretical analysis ofIP M sconsists of a great deal of analyzing Newton’s method. At present, there is still a gap between the practical behavior of the algorithms and the theoretical performance results, in favor of the practical behavior. This is especially true for the so-called barrier methods for solving (LO). The aim of this paper is to solve the problem of the starting initial point near the central path, where we propose a new weighted-path-following algorithm when the initial point is on the central path and we present an analysis of Newton’s method based on aproximity measure. Finally, some strategies are used for testing our algorithm.
MATH. REPORTS15(65),2(2013), 145–152
2. PROBLEM STATEMENT
The linear programming (LP) problem involves minimization of a linear (affine) function subject to linear constraints. Now, we consider the following (LP) in its standard form:
(P)
minimize cTx subject to Ax=b
x≥0 and its dual
(D)
maximize bTy
subject to ATy+s=c s≥0
whereA∈IRm×n withm≤n, b∈IRm andc∈IRn.
3. NOTATION AND ASSUMPTIONS
Let x∗ denote a solution to (P) and (y∗, z∗) a solution to (D). LetFp= {x ∈IRn :Ax= b, x ≥0 } and Fd ={(y, z) ∈ IRm× IRn :ATy+z = c, z≥0}be the primal and dual feasible region, ˚Fp={x∈IRn:Ax=b, x >0}
and ˚Fd={(y, z)∈IRm× IRn:ATy+z=c, z >0}denote its strict interior.
Throughout the paper, we will make the following assumptions:
(A1) The set ˚Fp is non–empty.
(A2) The set ˚Fdis non–empty.
(A3) rank (A) =m.
4. OPTIMALITY CONDITIONS AND CENTRAL PATH
Most of interior point methods are motivated by the logarithmic barrier function technique, so we apply the logarithmic barrier function to obtain the following perturbed problem. Letµ >0 be the barrier parameter, and consider the following barrier subproblem (Pµ) associated with (P):
(Pµ)
( minimize f(x, µ) =cTx−µ n
i=1riln (xi) subject to Ax=b, x >0
wherer = (r1, r2, ..., rn)∈IRn++is a weighted vector introduced to ensure that the initial point lies on the central path (define bellow), if ri = 1 ∀i then the weighted central path coincides with the classical one. Hence, this approach can be seen as a generalization of central path methods.
Theorem1. For everyµ >0,problem(Pµ)has a unique optimal solution x(µ). See [3,5].
Let X =diag (x1, x2, ..., xn)∈IRn×n and X−1 =diag(1/x1,1/x2, ..., 1/xn).TheKKT conditions associated with (Pµ) are given by:
(1)
c−µX−1r−ATy= 0 Ax=b, x >0
Note that the objective function of (Pµ) is convex and its constraints are linear. Moreover, by assumption (A1), problem (Pµ) has a strictly feasible solution. The first order optimality conditions (1) are both necessary and sufficient. Now, let µ > 0 be fixed and set z = µX−1r. It can be shown that under assumptions (A1),(A2) and (A3), there exists a unique solution (x(µ), y(µ), z(µ)) to the following system of equations (see [3]):
(2)
Ax=b, x >0 (a) ATy+z=c, z >0 (b)
Xz
r =µe (c)
The set of all solutions of the system (2) is called Central path following, we note it by:
C ={(x(µ), y(µ), z(µ)), µ >0}={(x, y, z)∈F˚p×F˚d: Xzr =µe, µ > 0}
we may notice that the central path equations resemble the optimality conditions for the primal–dual pair (P) and (D). We ensure that the initial point ¯x∈F˚p,z¯∈F˚dlies on the central path if we show that r= X¯µz¯.
The resolution of (Pµ) is equivalent with that of (P) with x∗(µ) is an optimal solution of (Pµ) then x∗ = lim
µ→0 x∗(µ) is an optimal solution of (P).
See [5].
5. NEWTON’S METHOD FOR SOLVING SUBPROBLEM(Pµ)
The special form of the derivatives of the barrier function makes Newton’s method a natural choice for solving problem (Pµ). When minimizing f(x, µ) subject tomlinear equality constraintsAx=b,the Newton step ∆xis designed to minimize a local Taylor-series quadratic model off (x0, µ) such that the next iterate x0 = ∆x+x satisfying the constraints Ax0 =b.
The vector ∆x is the solution of the quadratic program (3)
mingT∆x+12∆xTH∆x subject to A∆x= 0
where g(x, µ) = ∇f(x0, µ) = c−µX−1r and H(x, µ) =∇2f(x0, µ) =µX−2R designed the gradient and the Hessian of the barrier functionf, respectively.
Since H(x, µ) is positive definite, the objective function of (3) is convex.
Moreover, problem (3) has linear constraints and a strictly feasible solution (such as ∆x = 0). Thus, the KKT conditions associated with (3) are given by:
(4)
µX−2R∆x+c−µX−1r =ATy Ad= 0
y denotes the Lagrange multiplier vector associated with the constraints Ax=b.
Multiplying (4) by (X−2R)−1 and noting thatXe=x, we have:
(5) ∆x= 1µ(X−2R)−1(ATy−c+µX−1r) =x+µ1(X−2R)−1(ATy−c) Furthermore, multiplying (5) by A and using the relation A∆x = 0 to eliminate ∆x, we obtain:
A(X−2R)−1ATy= A(X−2R)−1c−µAXe
Since A has full rank and (X−2R)−1 is positive definite, we conclude that A(X−2R)−1AT is positive definite, so we have a unique solution y(x, µ) given by:
(6)y(x, µ) = (A(X−2R)−1AT)−1 [A(X−2R)−1c−µAXe]
Upon substituting (6) into (5), we obtain the following Newton step:
∆x= 1µ(X−2R)−1[AT(A(X−2R)−1AT)−1(A(X−2R)−1c−µAXe)−c+µX−1r]
By construction, the next Newton iteratex0 =x+α∆xsatisfiesAx0=b, for anyα∈IR. Thus, in order to guarantee thatx0>0, it suffices to setα∈ (0, mini{−xi/∆xi : ∆xi <0}).
6. CONVERGENCE ANALYSIS AND COMPLEXITY OF THE ALGORITHM
The first important element in the analysis is a suitable definition of a proximity measure that quantifies the closeness of a next iterate point x0 ∈F˚p to the central path. A natural idea is to find vectors y0 ∈IRm and z0 ∈IRn, such thatATy0+z0 =c andk Xr0z0 −µ0ek2 is minimized, whereµ0= (1−θ)µ, 0< θ <1. In other words, we are interested in solving the following problem:
(7)
minimize 12z0TR−1X02z−µ0x0Tz0 subject to ATy0+z0 =c
where R−1 =diag (1/r1,1/r2, ...,1/rn) andX02 =diag (x021, x022, ..., x02n) . The KKT conditions associated with (7), which are both necessary and sufficient for its optimality, are given by:
R−1X02z0−µ0X0e+λ= 0 Aλ= 0
ATy0+z0 =c We claim that:
(8)λ=λ(x0, µ0) =µ0∆x0, y0 =y(x0, µ0) and z0 =z(x0, µ0) =c−ATy0 where ∆x0 is given by (5) (replace x by x0) and y0 =y(x0, µ0) is given by (6), is a solution to the above system. Clearly, we haveAλ=µA∆x0 = 0.Now, we compute:
R−1X02z0−µ0X0e+λ=
R−1X02(c−ATy0)−µ0X0e+µ0(X0e+µ10(X0−2R)−1(ATy0−c)) = 0 This establishes the claim. In particular, an optimal solution to (7) can simply be obtained from an optimal solution to (3).
The above discussion motivates us to define the following proximity mea- sure δ(x, µ):
δ(x, µ) =
Xz(x,µ) rµ −e
2
Note that ifxlies on the central path, then we haveδ(x, µ) = 0. Thus, in some sense the value ofδ reflects the distance ofx from the central path.
Let us now collect some properties of the proximity measure . For x ∈ F˚p, let z =z(x, µ) be an optimal solution to (7). Define s = Xz(x,µ)rµ . Then, we have:
(9) δ(x, µ)2 = n
i=1(xµrizi
i −1)2=
i=1(si−1)2 =ks−ek22
Moreover, using (5) and ATy−c = z, we see that the search direction
∆x can be written as:
∆x=x−1µ(X−2R)−1z
It follows that the next iterate x0 of a pure Newton’s method is given by:
(10)x0 =x+ ∆x= 2x−µ1(X−2R)−1z= 2x−Xs wheres = X zrµ. In particular, we have:
(11)x0i= 2xi−xisi
The proximity measure as defined above allows us to show that Newton’s method converges quadratically if the current iterate is sufficiently close to the central path.
Theorem 2. Let µ > 0 be fixed and x ∈ F˚p be such that δ(x, µ) < 1.
Then, the next Newton iterate x0 = x+ ∆x also satisfies x0 ∈ F˚p. Moreover, we have δ(x0, µ)≤δ(x, µ)2 <1.
Proof. We haveAx0 =A(x+∆x) =Ax+A∆x=Ax=b(sinceA∆x= 0) and sinceδ(x, µ)<1,it follows from (10) that |si−1|<1,which implies that 0 < si <2, for i= 1, ..., n. We conclude thatx0 >0 and we obtain that x0 ∈ F˚p.Now, we have:
δ(x0, µ) =
X0z rµ −e
2
using the relationsz=µRX−1sandx0i = 2xi−xisi,we obtain Xrµ0z = 2s−S2e.
This implies that:
[δ(x0, µ)2 = n
i=1(2si−s2ie−1)2 = n
i=1(si−1)4 ≤
n
(
i=1
(si−1)2)2=δ(x, µ)4] we obtainδ(x0, µ)≤δ(x, µ)2 <1.
We conclude that the Newton iterates converge quadratically to the point on the central path.
Theorem 3. Let µ >0, θ ∈(0,1) and µ0 = (1−θ)µ. Then, for anyx∈ F˚p, we have:
δ(x, µ0)≤ δ(x,µ)+θ
√n 1−θ
Proof. By definition of δ and we let µ/µ0 = 1−θ1 = v, we prove the theorem.
Theorem 4. Let x∈F˚p be the current iterate. Suppose thatδ(x, µ)≤ 12, for some µ >0. Let θ= 6√1n. Then, when the next Newton iterate x0 is given by (11) and µ0 = (1−θ)µ, we have δ(x0, µ0)≤ 12.
Proof. By Theorem 3 and Theorem 2.
Corollary 1. Let x ∈ F˚p be the initial point (δ(x, µ) = 0), for some µ > 0. Let θ= 6√1n. Then, when the next Newton iterate x0 is given by (11) and µ0 = (1−θ)µ, we have δ(x0, µ0)< 15 ≤ 12.
Theorem5. Letl0= [ln(nµ0)].Then, the Algorithm will terminate in at most 6(l+l0)√
nsteps, and its output (x, y, z) will satisfy: cTx−bTy≤ 32e−l. Now, we give the algorithm:
———————————————————————————————–
Algorithm: A path following algorithm for solving (P)
———————————————————————————————–
Data: x0∈F˚p,(y0, z0)∈F˚d, µ0= (x0)nTz0 >0 an initial barrier parameter, r = Xµ00z0 and l an accuracy parameter.
Begin:
Setk= 0 and θ= 6√1n While nµk> e−l do
• computeµk+1 = (1−θ)µk
• compute xk+1 = 2xk−R−1(Xk)2zk/µk where zk(xk, µk) is an optimal solution to (8)
• k=k+ 1 End.
———————————————————————————————–
7. NUMERICAL IMPLEMENTATION
In this section, we deal with the numerical implementation of this algo- rithm applied to an example of LO. Here, we used x∗ the exact solution of the problem andIter means the iterations number produced by the algorithm.
The implementation is manipulated in Turbo-Pascal on a Pentium 4. Our tolerance is =10−6. For the update parameter we have a practical strategy which isµ0 = 0.1 and θ∈ {0.2, 0.5, 0.7, 0.9}.
We consider the following example:
A=
2 −6 2 7 3 8 1 0 0 0 0
−3 −1 4 −3 1 2 0 1 0 0 0
8 −3 5 −2 0 2 0 0 1 0 0
4 0 8 7 −1 3 0 0 0 1 0
5 2 −3 6 −2 −1 0 0 0 0 1
b= (1,−2,4,1,5)T
c= (−18,7,−12,−5,0,−8,0,0,0,0,0)T
The optimal solutions of the primal-dual problem are obtained as follows:
x∗ = (2.000009,4.000002,0,0,7.000004,0,0,1.000009,0,0,0.999999)T y∗= (−0.333333,0,−1.666666,−1.000000,0)T
z∗ = (0,0,4.999999,0.999999,0,0.999999,0.333333,0,1.666666,1.000000,0)T
Numbers of iterations for several choices of θ.
————————————————————
θ 0.2 0.5 0.7 0.9
————————————————————
iter 9 8 7 6
————————————————————
The numerical results show that the number of iterations of the Algorithm depends on the values of the parameter θ. It is quite susprising that θ = 0.9 gives the lowest iteration.
8. CONCLUSION
In this paper we have developed a new weighted-path-following algorithm for solving linear optimisation problems. Our approach is a generalization of the classical approch (r = 1) [1]. We have introduced a weighted vector to ensure that the initial point lies on the central path. We have proved that the algorithm performs no more than 6(l+l0)√
niterations. Our numerical results are acceptable, we point out that the implementation with the update parame- terθreduces significantly the number of iterations produced by this algorithm and leads this algorithm to reach their real numerical performances. In our next study we will generalize this method to the convex nonlinear programming.
REFERENCES
[1] Anthony Man–Cho So,Handout 10: Path Following Algorithms for Linear Programming, November 25, 2010.
[2] N.K. Karmarkar,A new polynomial-time algorithm for linear programming, Combinator- ica4(1984), 373–395.
[3] N. Megiddo, Pathways to the Optimal Set in Linear Programming. Progress in Mathe- matical Programming: Interior-Point and Related Methods, Springer–Verlag, New York, 1988, 131–158.
[4] M.H. Wright, Interior methods for constrained optimization. Acta Numerica1 (1992), 341–407.
[5] S.J. Wright, Primal-Dual Interior-Point Methods. Society for Industrial and Applied Mathematics, Philadelphia, Pennsylvania, 1997.
[6] Y.Ye, Interior Point Algorithms: Theory and Analysis. Wiley–Interscience Series in Discrete Mathematics and Optimization. John Wiley & Sons, Inc., New York, 1997.
Received 2 June 2011 University of Setif,
Department of Applied Mathematics, LMFN Laboratory,
Algeria, h [email protected]
