An Efficient Algorithm for Min-Max Convex Semi-Infinite Programming Problems

Full Terms & Conditions of access and use can be found at

http://www.tandfonline.com/action/journalInformation?journalCode=lnfa20

Download by: [Tsinghua University] Date: 08 December 2016, At: 01:31

Numerical Functional Analysis and Optimization

ISSN: 0163-0563 (Print) 1532-2467 (Online) Journal homepage: http://www.tandfonline.com/loi/lnfa20

An Efficient Algorithm for Min-Max Convex Semi- Infinite Programming Problems

Liping Zhang & Soon-Yi Wu

To cite this article: Liping Zhang & Soon-Yi Wu (2016) An Efficient Algorithm for Min-Max Convex Semi-Infinite Programming Problems, Numerical Functional Analysis and Optimization, 37:8, 1037-1053, DOI: 10.1080/01630563.2016.1191033

To link to this article: http://dx.doi.org/10.1080/01630563.2016.1191033

Accepted author version posted online: 26 May 2016.

Published online: 26 May 2016.

Submit your article to this journal

Article views: 40

View related articles

View Crossmark data

2016, VOL. 37, NO. 8, 1037–1053

http://dx.doi.org/10.1080/01630563.2016.1191033

An Efficient Algorithm for Min-Max Convex Semi-Infinite Programming Problems

Liping Zhang^aand Soon-Yi Wu^b

aDepartment of Mathematical Sciences, Tsinghua University, Beijing, China;^bDepartment of Mathematics, National Cheng Kung University, Tainan, Taiwan

ABSTRACT

In this article, we consider the convex min-max problem with infinite constraints. We propose an exchange method to solve the problem by using efficient inactive constraint dropping rules.

There is no need to solve the maximization problem over the metric space, as the algorithm has merely to find some points in the metric space such that a certain criterion is satisfied at each iteration. Under some mild assumptions, the proposed algorithm is shown to terminate in a finite number of iterations and to provide an approximate solution to the original problem.

Preliminary numerical results with the algorithm are promising.

To our knowledge, this article is the first one conceived to apply explicit exchange methods for solving nonlinear semi-infinite convex min-max problems.

ARTICLE HISTORY Received 3 November 2014 Revised 16 March 2016 Accepted 14 May 2016

KEYWORDS Convex programming;

exchange method;

semi-infinite programming

MATHEMATICS SUBJECT CLASSIFICATION 90C30; 49D39; 65K05; 65D15

1. Introduction

In this article, we consider the following nonlinear min-max problem with infinite constraints:

(P) min

x∈Rⁿ f(x):=max{f₁(x),f₂(x),. . .,f(x)} s.t. g(x,ω)≤0,∀ω ∈,

which is called the semi-infinite min-max problem. Throughout this study, the following assumptions about the data in (P) are made.

Assumption 1.1.

(a) is a compact and nonempty subset ofR^m;

(b) f_i : Rⁿ → R, i = 1,. . .,are convex and continuously differentiable onRⁿ and are not equal to each other;

(c) g :Rⁿ× → Ris continuous onRⁿ×, g(·,ω)is convex for allω ∈ , and∇xg(x,ω)exists and is continuous onRⁿ×;

(d) There is a Slater pointxˆ∈Rⁿsuch that g(x,ˆ ω) <0for allω ∈.

CONTACTLiping Zhang [email protected] Department of Mathematical Sciences, Tsinghua University, Beijing 100084, China.

© 2016 Taylor & Francis

We call problems in the form of (P) with Assumption1.1min-max convex semi-infinite programming problems. In the particular case that = 1, (P) is an ordinary convex semi-infinite programming (CSIP) problem. Notice that although the functionf is piecewise smooth and locally Lipschitz continuous, it is not differentiable. Hence, it is important to find an approach to solving the min-max CSIP problem (P). See [9] for more information on min-max optimization problems.

Min-max semi-infinite programming problems arise in various engineering applications. For example, in civil engineering, electronic circuit design, and optimal control in robot path planning (see, e.g., [2–4,6,7,10,12]). A lot of literature deals with solution methods for solving these types of problems. For example, Polak et al. [11,12] proposed algorithms with smoothing techniques for solving finite and semi-infinite min-max problems. Auslender et al. [1]

proposed penalty and smoothing methods for solving min-max CSIP in the form of (P) where the number offi(x)is infinite. Obviously, an ordinary CSIP problem is the special case of (P). Many solution methods were presented for solving semi-infinite programming problems (SIPs) [7,13]. The main idea used in solution methods is to replace (P) by a sequence of finite programming problems, i.e., problems with only a finite number of constraints. According to the way that finite problems are generated, there are three important types of numerical methods for solving SIPs including discretization methods, local reduction based methods, and exchange methods (see [13]). Discretization methods (see, e.g., [15, 16]) have the advantage of internally working with finite subsets ofonly. However, they are computationally costly, and the cost per iteration increases dramatically as the cardinality of the auxiliary problem grows. Globally convergent reduction based methods (see, e.g., [5,17]), on the other hand, require strong assumptions and are often conceptual methods which can merely be implemented in a rather simplified form. The exchange method (see, e.g., [8,18]) is one of the important methods beyond discretization and reduction based methods. For linear SIP, Lai and Wu [8] proposed an explicit algorithm in which they solved a linear programming with a finite feasible set _k. They drop out redundant points in_kat each iteration and only keep active points. Hence, the algorithm is very efficient in saving computational time.

Similar to the exchange method, an iterative method was recently proposed in [14] for solving the KKT system of SIP in which some redundant points were dropped at certain iterations. Recently, Zhang, Wu and López [19] proposed a new exchange method for solving convex SIP based on the algorithm in [8]. It is shown that the algorithm provides an approximate solution after a finite number of iterations under weaker conditions.

In this article, motivated by the ideas in [8,14,19], we design an exchange method for solving the min-max CSIP (P). However, our analysis techniques are quite different from the ones used in [8,14], because the objective function is of the min-max form, the infinite constraints functions are nonlinear, and

the nonnegative constraint does not exist on which the analysis techniques in [8] mainly depended. We note that our algorithm introduces a relaxed scheme which does not require solving the global maximization problem with respect toω ∈ at each iteration. Our algorithm has merely to find some ω ∈ such that a certain criterion with small scalarρ > 0 is satisfied. We prove that the algorithm terminates in a finite number of iterations and the output is an approximate optimal solution of (P). Namely, we show that the obtained solution converges to the solution of (P) asρtends to zero.

This article is organized as follows. We present the algorithm in Section 2 and give a convergence analysis in Section 3. In Section 4, we give some numerical results. We conclude the article with some remarks in Section 5.

2. Algorithm description

In this section, motivated by ideas in [8,14], we propose an exchange algorithm to solve the min-max CSIP problem (P).

Although (P) is also regarded as an ordinary CSIP problem satisfying the Slater constraint qualification, it is not easy to solve by directly using algo- rithms for CSIP becausef(x)is not continuously differentiable. Introducing an artificial variable x_n+1 ∈ R, we equivalently reformulate (P) as the following n+1-dimensional minimization problem:

P[] min x_n+1

s.t. fi(x)≤x_n+1, i=1, 2,. . .,, g(x,ω) ≤0,∀ω∈.

Obviously, problemP[]is an ordinarily smoothing CSIP problem. This fact allows us to develop iterative methods based on problem P[] to solve the original problem (P) without facing the non-differentiability off(x).

We now present an efficient exchange algorithm based on the auxiliary problem P[]. The exchange algorithm solves a finitely constrained convex programming at each iteration. Associated with each finite subsetB ⊂ , we define the finitely constrained convex program by

P[B]: min x_n+1

s.t. f_i(x) ≤x_n+1, i= 1, 2,. . .,, g(x,ω)≤0, ∀ω∈B.

We can easily establish the KKT conditions forP[B], which is written as

i=1

λ_i =1,

i=1

λ_i∇fi(x)+

ω∈B

ν(ω)∇xg(x,ω)=0, (1)

λ_i(fi(x)−x_n+1)= 0, λ_i≥0, fi(x)≤x_n+1, i=1,. . .,, ν(ω)g(x,ω)=0, ν(ω)≥0, g(x,ω) ≤0, ∀ω∈B.

Here, ν(ω), ω ∈ B, and λ_i,i = 1,. . ., can be regarded as the Lagrange multipliers of problem P[B].

Letxˆ ∈ Rⁿbe the Slater point in Assumption1.1(d). Choose a scalarxˆ_n+1 such that xˆ_n+1 > max{f1(x)ˆ ,. . .,f(x)ˆ }. Then we have fi(x) <ˆ xˆ_n+1,i = 1,. . .,. Hence, under Assumption1.1(d) there is a Slater point for problem P[]. Consequently, conditions (1) turn out to be necessary and sufficient optimality conditions for problemP[B].

Theorem 2.1. Let(x^∗,x^∗_n+1)∈Rⁿ⁺¹be a feasible solution for problem P[B]and Assumption1.1be satisfied. Then,(x^∗,x^∗_n+1)is optimal if and only if there exist Lagrange multipliers{ν^∗(ω)|ω ∈ B}and{λ^∗|i = 1,. . .,}such that conditions (1) are satisfied.

The details of the algorithm are described as follows.

Algorithm 2.1(An efficient exchange method).

Step 0. Choose finite points ₀ = {ω⁰

1,. . .,ω⁰

m0} and a sufficiently small numberρ > 0. Solve problemP[₀]to obtain its optimum(x⁰,x⁰_n+1).

Setk:=0.

Step 1. Find aω^k

new ∈such that

g(x^k,ω^k

new) > ρ. (2)

If such aω^k

newdoes not exist, then stop. Otherwise, let

¯

_k+1 :=_k∪ {ω^k

new}.

Step 2. Solve problemP[ ¯_k+1]to obtain its optimum(x^k+1,x^k_n+1⁺¹)and the cor- responding Lagrange multipliers{ν^k+1(ω)|ω ∈ ¯_k+1}and{λ^k_i⁺¹|i = 1,. . .,}.

Step 3. Let

_k+1 := {ω ∈ ¯_k+1|ν^k+1(ω) >0}. Setk:=k+1 and go to Step 1.

There are some remarks for Algorithm 2.1:

(a) Steps 1–3 are the main differences from the algorithms of [8,14].

(b) In Step 1, it is also possible to choose multiple elements satisfying (2).

Although we merely deal with the single-point exchange scheme in the fol- lowing analysis, the obtained results are also applicable to multiple exchange type algorithms.

(c) In Step 2, the optimal solution(x^k+1,x^k+1_n+1)of problemP[ ¯_k+1]also solves problemP[_k+1].

(d) In Step 3, all inactive constraints at the optimum(x^k+1,x^k+1_n+1)are removed since the KKT conditions of problemP[ ¯_k+1]implies thatν^k+1(ω)=0 for anyω∈ ¯_k+1withg(x^k+1,ω) <0.

We now define some notations for convenience in analyzing the convergence properties of Algorithm 2.1. Letv^∗denote the optimal value of problemP[], and let{(x^k,x^k_n+1)},{ν^k(ω)|ω ∈ _k}, and{λ^k_i|i = 1,. . .,}be the sequences of optimal solution and corresponding Lagrangian multipliers generated by Algorithm 2.1. For k=1, 2,. . ., we define

d^k:=x^k+1−x^k, θ_i^k := x^k_n+1−f_i(x^k), i=1,. . .,. (3) Then, by Assumption1.1(d) and KKT conditions (1) for problemP[_k], we have

i=1

λ^k_i =1,

i=1

λ^k_i∇fi(x^k)+

ω∈_k

ν^k(ω)∇^xg(x^k,ω) =0, (4) λ^k_iθ_i^k=0, λ^k_i ≥0, θ_i^k ≥0, i=1,. . .,,

ν^k(ω)g(x^k,ω)=0, ν^k(ω) ≥0, g(x^k,ω)≤0, ∀ω∈_k. We also define

D^k_i :=fi(x^k+1)−fi(x^k)− ∇fi(x^k)^Td^k, i=1,. . .,, G^k_i :=fi(x^k)−fi(x^k+1)+ ∇fi(x^k+1)^Td^k, i=1,. . .,, S^k(ω):= g(x^k+1,ω)−g(x^k,ω)− ∇xg(x^k,ω)^Td^k, (5) H^k(ω):=g(x^k,ω)−g(x^k+1,ω)+ ∇xg(x^k+1,ω)^Td^k.

Then, by using the Taylor expansion and the convexity off_iandg(·,ω)forω ∈ , we obtain

0≤D^k_i =o(d^k), 0≤G^k_i =o(d^k), i=1,. . .,, 0≤S^k(ω)=o(d^k), 0≤H^k(ω)=o(d^k). (6)

Let v(_k) denote the optimal value of problem P[_k]. From Step 1 of Algorithm 2.1, it is easy to see that

¯

_k+1⊇ _k,

which implies that the feasible region of problemP[ ¯_k+1]is contained in that of problemP[_k]. Hence, from(c)we have

v(_k+1) =v(¯_k+1)≥v(_k).

Consequently, we immediately obtain the following theorem.

Theorem 2.2. The sequence of optimal values{v(_k)}of{P[_k]}is nondecreas- ing, i.e.,

v(_k+1)≥v(_k) for k=1, 2,. . ..

The following theorem is very important for analyzing the convergence properties of Algorithm 2.1, because it evaluates the increment of the optimal valuev(_k)of problemP[_k]at each iteration.

Theorem 2.3. For k=1, 2,. . ., we have v(_k+1)−v(_k)=

i=1

λ^k_i

θ_i^k+1+D^k_i

+

ω∈_k

ν^k(ω)

S^k(ω)−g(x^k+1,ω)

=g(x^k,ω^k

new)ν^k+1(ω^k

new)−

ω∈k+1

ν^k+1(ω)H^k(ω)

−

i=1

λ^k_i⁺¹(G^k_i +θ_i^k). (7)

Proof. First, we prove that the first equality in (7) is satisfied. From (4) we have v(_k+1)−v(_k)=

i=1

λ^k_ix^k_n+1⁺¹ −

i=1

λ^k_ifi(x^k)

=

i=1

λ^k_iθ_i^k+1+

i=1

λ^k_i(D^k_i + ∇f_i(x^k)^Td^k)

=

i=1

λ^k_i(θ_i^k+1+D^k_i)−

ω∈_k

ν^k(ω)∇xg(x^k,ω)^Td^k

=

i=1

λ^k_i(θ_i^k+1+D^k_i)+

ω∈_k

ν^k(ω)

S^k(ω)−g(x^k+1,ω) , (8) where the first and the third equalities follow from (4), the second one follows from (3) and (5), and the last one holds due to (5) andg(x^k,ω)=0 forω∈_k. Hence, the first equality in (7) holds.

Next, we show the validity of the second equality in (7). Byg(x^k,ω) = 0 for ω∈_kand¯_k+1 =_k∪ {ω^k

new}, we have

ω∈ ¯_k+1

ν^k+1(ω)g(x^k,ω)=

ω∈_k

ν^k+1(ω)g(x^k,ω)+ν^k+1(ω^k

new)g(x^k,ω^k

new)

=ν^k+1(ω^k

new)g(x^k,ω^k

new). (9)

On the other hand, it follows from_k+1 = {ω ∈ ¯_k+1|ν^k+1(ω) >0}that

ω∈ ¯_k+1

ν^k+1(ω)g(x^k,ω)

=

ω∈_k+1

ν^k+1(ω)(g(x^k+1,ω)+H^k(ω)− ∇xg(x^k+1,ω)^Td^k)

=

ω∈_k+1

ν^k+1(ω)H^k(ω)+

i=1

λ^k_i⁺¹∇fi(x^k+1)^Td^k

=

ω∈_k+1

ν^k+1(ω)H^k(ω)+

i=1

λ^k_i⁺¹(G^k_i +f_i(x^k+1)−f_i(x^k))

=x^k+1_n+1−x^k_n+1+

ω∈_k+1

ν^k+1(ω)H^k(ω)+

i=1

λ^k+1_i (G^k_i +θ_i^k), (10) where the first and the third equalities follow from (5), the second one holds due to (4) andg(x^k+1,ω)= 0 forω∈_k+1, and the last one is satisfied since

i=1

λ^k+1_i (f_i(x^k+1)−f_i(x^k))=

i=1

λ^k+1_i x^k+1_n+1+

i=1

λ^k+1_i (θ_i^k−x^k_n+1)

=x^k_n⁺₊¹₁−x^k_n+1+

i=1

λ^k_i⁺¹θ_i^k

=v(_k+1)−v(_k)+

i=1

λ^k+1_i θ_i^k, where the second equality follows from

i=1λ^k_i⁺¹ =1. Thus, equalities (9) and (10) imply that the second equality in (7) holds.

3. Finite termination convergence analysis

In this section, we show that Algorithm 2.1 terminates in a finite number of iterations under some mild conditions. Furthermore, we prove that the output at the final iteration is sufficiently close to the optimal solution of (P) if the criterion valueρis sufficiently close to zero.

Lemma 3.1. For any given k∈ {1, 2,. . .}, we have v(_k+1) >v(_k), ω^k

new∈_k+1,

hold if(x^k,x^k_n+1)is the unique optimal solution to problem P[_k].

Proof. For any givenk≥1, it follows from Theorem2.2that v(_k+1)≥v(_k).

Suppose, reasoning by contradiction, that there existsk₀such that

v(_k0+1)= v(_k0). (11)

LetF_k

0andF¯_k

0+1be the feasible regions ofP[_k0]andP[ ¯_k0+1], respectively.

Then, from Step 1 of Algorithm 2.1, we have F^k0 ⊇ ¯F^k0+1,

which together with (11) implies that(x^k0+1,x^k_n⁰₊⁺₁¹)is also an optimal solution to problemP[_k0]. Since(x^k0,x^k_n⁰₊₁)is the unique optimal solution to problem P[_k0], it follows thatx^k0+1 = x^k0. Therefore, by (2) and Step 2 of Algorithm 2.1, we obtain the following contradiction:

0≥g(x^k0+1,ω^k0

new) =g(x^k0,ω^k0

new) > ρ >0.

Thus,v(_k+1) >v(_k). To prove the fact that ω^k

new ∈ _k+1, it suffices to show ν^k+1(ω^k_new) > 0.

It follows from (4) thatν^k+1(ω^k_new) ≥ 0. Suppose, reasoning by contradiction, that there exists an integerk₁ ≥0 such thatν^k1+1(ω^k_new¹ )=0. Then, the second equality in (7) andv(_k1+1) >v(_k1)imply that

−

ω∈_k1+1

ν^k1+1(ω)H^k1(ω)−

i=1

λ^k_i¹⁺¹(G^k_i¹+θ_i^k1) >0. (12) On the other hand, combining (4) and (6), we have

ν^k1+1(ω) >0, H^k1(ω)≥0, ∀ω ∈_k1+1, λ^k_i¹⁺¹ ≥0, G^k_i¹ ≥0, θ_i^k1 ≥0, i=1,. . .,, which contradicts (12). This completes the proof.

There are sufficient conditions for the assumption in Lemma3.1.

Lemma 3.2. If either fi,i=1,. . .,are strictly convex, or g(·,ω),ω ∈is strictly convex, then the sequence{P[_k]}of finite minimization subproblems generated by Algorithm 2.1 has the unique optimal solution{(x^k,x^k_n+1)}.

Proof. For any givenk≥1, let(x^k,x^k_n+1)and(xˆ^k,xˆ^k_n+1)be optimal solutions to problemP[_k]. Then,

ˆ

x^k_n+1 =x^k_n+1. (13)

Definey^k := ˆx^k−x^k,

Q^k_i :=f_i(xˆ^k)−f_i(x^k)− ∇f_i(x^k)^Ty^k, i=1,. . .,,

and

W^k(ω):= g(xˆ^k,ω)−g(x^k,ω)− ∇xg(x^k,ω)^Ty^k, ω∈_k. Then,

Q^k_i ≥0, i=1,. . .,, W_i^k(ω)≥0, ∀ω∈_k. (14) Using a similar argument as (8), we obtain

ˆ

x^k_n+1−x^k_n+1 =

i=1

λ^k_i ˆ

θ_i^k+Q^k_i

+

ω∈_k

ν^k(ω)

W^k(ω)−g(xˆ^k,ω)

. (15) Sinceλ^k_i ≥ 0,θˆ_i^k := ˆx^k_n+1−fi(xˆ^k) ≥ 0,i = 1,. . .,, andg(xˆ^k,ω) ≤ 0 for all ω∈_k, it follows from (13), (14), and (15) that

i=1

λ^k_iQ^k_i = 0,

ω∈_k

ν^k(ω)W^k(ω)= 0, which together with

i=1

λ^k_i = 1, λ^k_i ≥0, i=1,. . .,, ν^k(ω) >0, ∀ω ∈_k, implies that there existsi₀such that

Q^k_i0 =0, W^k(ω)=0, ∀ω∈_k. (16) Since either f_i0 org(·,ω) is strictly convex, it follows from (16) thaty^k = 0, i.e., x^k = ˆx^k. Namely, (x^k,x^k_n+1) is the unique optimal solution to problem P[_k].

Theorem 3.1. Suppose that either fi,i = 1,. . .,are strictly convex, or g(·,ω) is strictly convex and there exist k₀ > 0andδ > 0such thatν^k(ω) ≥ δ for all ω ∈ _k and for k ≥ k₀. If{x^k}is bounded, then Algorithm 2.1 terminates in a finite number of iterations.

Proof. Suppose, reasoning by contradiction, that Algorithm 2.1 does not finitely stop. Then, by Theorem2.2, we have

v^∗≥x^k_n+1⁺¹ ≥x^k_n+1, k=1, 2,. . ., which implies that

k→∞lim(x^k_n⁺₊¹₁−x^k_n+1)=0. (17) Since{x^k}is bounded,{ω^k

new} ⊂andis compact, and λ^k_i ≥0, i=1,. . .,,

i=1

λ^k_i =1,

we can assume, without loss of generality, that there existx¯ ∈ Rⁿ,ω¯ ∈ ,d¯ ∈ Rⁿ, and 0≤ ¯λ_i≤1,i=1,. . .,such that

k→∞lim λ^k_i = ¯λ_i,

i=1

¯

λ_i=1, (18)

klim→∞(x^k,ω^k

new) =(x,¯ ω¯), lim

k→∞d^k= ¯d.

Hence, it follows from (2) that

klim→∞g(x^k,ω^k

new)=g(x,¯ ω¯)≥ρ >0. (19) By (18), the first equality in (7), and (17), there existi₀withλ¯_i0 >0 such that

fi0(x¯+ ¯d)−fi0(x)¯ − ∇fi0(x)¯ ^Td¯ =0. (20) On the other hand, if there existsk₀ > 0 andδ >0 such thatν^k(ω) ≥δ for all ω∈_kand fork≥k0, then it follows from (7) and (17) that there exists a point

ˆ

ω∈such that

g(x¯+ ¯d,ωˆ)−g(x,¯ ωˆ)− ∇xg(x,¯ ωˆ)^Td¯ =0. (21) Therefore, either from strict convexity offi0and (20) or from strict convexity of g(·,ωˆ)and (21), we can obtain

k→∞lim d^k= ¯d=0.

Thus for any sufficiently smallε >0, there exists a large positive integerN ≥k₀ such that

x^N+1−x^N = d^N< ε². (22) Lemma3.1and Lemma3.2imply thatω^N

new ∈_N+1. This, together with (4), yieldsg(x^N+1,ω^N

new)=0. Since{x^k}is bounded and∇xg(x,ω)is continuous on Rⁿ×, there exists a constantc0 >0 such that

∇xg(x^N,ω^N

new) ≤c0. Hence, we obtain

g(x^N,ω^N

new)=g(x^N,ω^N

new)−g(x^N+1,ω^N

new)

≤ ∇xg(x^N,ω^N

new)^T(x^N −x^N+1)

≤ ∇xg(x^N,ω^N

new)x^N+1−x^N

≤c0ε² →0, asε→0, where the first inequality holds due to convexity ofg(·,ω^N

new), and the last one follows from (22). This contradicts (19). Thus Algorithm 2.1 terminates in a finite number of iterations.

For any givenk≥1, define index sets

I:= {1, 2,. . .,}, I^k₊:= {i∈I|λ^k_i >0}. Let us also define a matrixA^kof|I^k₊| ×n:

A^k :=(· · · ∇fi(x^k)^T· · ·)^T

i∈I^k₊.

We introduce the following assumption for discussing the convergence of Algorithm 2.1.

Assumption 3.1. There exists an integer k0 large enough, scalars σ₁ > 0 and σ₂ >0such thatσ₁ ≥σ₂and the following statements hold for all k≥k0. (a) {x^k}is bounded, andσ₂ ≤ν^k(ω)≤σ₁forω∈_k.

(b) λ^k_i ≥σ₂for i ∈I₊^k.

(c) µ_min((A^k)^TA^k)≥σ₂, whereµ_mindenotes the minimum eigenvalue.

(d) ω^k

new ∈_k+1.

Note that, Assumption 3.1(a)–(c) are regularity conditions. Assumption 3.1(d) is also mild because the conditions in Lemma 3.1 and Lemma 3.2 all ensure (d) is satisfied.

Theorem 3.2. Algorithm 2.1 finitely terminates if Assumption3.1holds.

Proof. Suppose, reasoning by contradiction, that Algorithm 2.1 does not finitely terminate. Then, by Theorem2.2we have

v(_k)≤v(_k+1)≤v^∗, k=1, 2,. . ., which implies that

klim→∞(v(_k+1)−v(_k))=0.

Since {x^k} is bounded and {ω^k

new} ⊂ and is compact, there exists a convergent subsequence. For the sake of convenience, we can assume, without loss of generality, that there existx^∗∈Rⁿandω∗ ∈such that

k→∞lim (x^k,ω^k

new)=(x^∗,ω^∗), which together with (2) yields

klim→∞g(x^k,ω^k

new)=g(x^∗,ω^∗)≥ρ >0. (23) For any sufficiently smallε∈(0,min{σ₂,σ₂²}), we can find a sufficiently large integerN> k₀such that

0≤x^N+1_n+1 −x^N_n+1< ε², |g(x^N,ω^N

new)−g(x^∗,ω^∗)|< ε². (24)

From Theorem2.3, the first formulation in (24) and Assumption3.1(d), we have 0<

i=1

λ^N_i (θ_i^N+1+D^N_i )+

ω∈_N

ν^N(ω)(S^N(ω)−g(x^N+1,ω)) < ε², (25) and

g(x^N,ω^N

new)≤

ε²+

ω∈N+1

ν^N+1(ω)H^N(ω)+

i=1

λ^N+1_i (G^N_i +θ_i^N) ν^N+1(ω^N

new) . (26)

Inequality (25) implies that

0≤λ^N_i θ_i^N+1 < ε², fori∈I₊^N, which together with Assumption3.1(b) yields

θ_i^N+1 < ε²/σ₂, fori∈I₊^N. (27) Sincef_i(x^N)=x^N_n+1 ≤x^N_n+⁺₁¹fori∈I₊^N, it follows from (27) that

A^N(x^N+1−x^N)+o(x^N+1−x^N)=O(ε²/σ₂). Hence, by Assumption3.1(c) we have

O(ε⁴/σ₂²)=(x^N+1−x^N)^T(A^N)^TA^N(x^N+1−x^N)

≥µ_minx^N+1−x^N2≥σ₂x^N+1−x^N2. (28) Since 0< ε < σ₂², it follows from (28) that

x^N+1−x^N =o(ε).

Hence, we have

G^N_i =o(d^N)= o(ε), H^N(ω)=o(d^N)= o(ε), ∀ω ∈_N+1, and

i=1

λ^N+1_i θ_i^N =

i=1

λ^N+1_i [(x^N_n+1−x^N+1_n+1)+(f_i(x^N+1)−f_i(x^N))]

=O(ε²)+

i=1

λ^N_i ⁺¹∇fi(x^N)^T(x^N+1−x^N)+o(ε)

=o(ε),

where the second equality follows from (24) and Taylor expansion, and the last one holds since{x^k}is bounded and∇f_i(x)is continuous onRⁿ. There exists a constantM >0 such that

∇f_i(x^N) ≤ M, ∀i∈I,

which together with

i=1λ^N+1_i =1 yields

i=1

λ^N+1_i ∇f_i(x^N)^T(x^N+1−x^N)≤Mx^N+1−x^N.

Consequently, by the second formulation in (24), (26), and Assumption3.1(a), we have

|g(x^∗,ω^∗)| ≤ |g(x^N,ω^N

new)| +ε²< ε²+ o(ε)+ε²

σ₂ . (29)

Sinceσ₁> σ₂> ε, (29) yields

|g(x^∗,ω^∗)| →0 asε→0,

which contradicts (23). Hence, Algorithm 2.1 terminates in a finite number of iterations.

Until now, we have shown the finite termination property of Algorithm 2.1.

Nevertheless, the previous theorems would be meaningless if the obtained solu- tion were far from the optimum of (P). Hence, we give the following theorem, which indicates that Algorithm 2.1 can yield an approximate optimal solution of (P) in a finite number of iterations.

Theorem 3.3. Suppose that Algorithm 2.1 terminates in a finite number of iterations, and let k^∗(ρ) be the number of iterations in which Algorithm 2.1 terminates. If there existsρ₀ >0such that the set

F_ρ := {(x,x_n+1)∈Rⁿ⁺¹|f_i(x)≤x_n+1,i=1,. . .,, g(x,ω)≤ρ, ∀ω∈} is bounded when ρ = ρ₀, then the optimal value of problem P[_k∗(ρ)] is sufficiently close to the optimal value v^∗of problem P[]ifρis tends to zero, i.e.,

ρlim→0v(_k∗(ρ)) =v^∗.

Moreover, if (P) has a unique optimal solutionx^∗, thenlimρ→0x^k∗(ρ)=x^∗. Proof. LetF be the feasible region of problemP[]. It is clear that

(x^k∗(ρ),x^k_n+1^∗(ρ))∈F_ρ, F ⊆ F_ρ. Since there existsρ₀ >0 such that the setF_ρ

0is bounded, we further have

ρlim→0dist(F,F_ρ)=0, (30)

wheredist(·,·)is the Hausdorff distance between two sets. Let(x^Pr,x^Pr_n+1)be the projection of(x^k∗(ρ),x^k_n^∗₊^(ρ)₁ )ontoF. Then, we have

0≤v^∗−v(_k∗(ρ))

=v^∗−x^Pr_n+1+x^Pr_n+1−x^k_n^∗₊^(ρ)₁

≤x^Pr_n+1−x^k_n^∗₊^(ρ)₁

≤ (x^Pr,x^Pr_n+1)−(x^k∗(ρ),x^k_n+1^∗(ρ))

≤dist(F,F_ρ), which, together with (30), imply that

ρlim→0v(_k∗(ρ)) =v^∗.

Consequently, we can easily show the second conclusion of this theorem by using the boundedness of{x^k∗(ρ)}.

Notice that in Step 1 of Algorithm 2.1 we may also simultaneously chooseq different points{ω^k

1,. . .,ω^k

q}such that g(x^k,ω^k

i) > ρ fori=1,. . .,q, and let

¯

_k+1 :=_k∪ {ω^k

1,. . .,ω^k

q}.

For such a multiple explicit exchange method, Theorem 2.3 and Theorems 3.1–3.3can be shown by using analogous techniques.

4. Implementation and numerical examples

In this section, we report some preliminary numerical results for Algorithm 2.1.

We implement Algorithm 2.1 in MATLAB 7.8.0 (R2009a) and run experiments on a personal computer with Pentium(R) CPU 1.73GHz and RAM of 512MB.

For all examples, we choose the vector of ones as the starting point. We implement Algorithm 2.1 with multiple exchange, and we apply nonlinear pro- gramming solverfminconfrom MATLAB toolbox to solve each subproblem. If = [a,b],₀and¯ are set to be

₀ = {a+i(b−a)/(N0−1)|i=0, 1,. . .,N0−1}, whereN0 =10, and

¯

= {a+i(b−a)/(N−1)|i=0, 1,. . .,N−1}, whereN =10.

In Step 1 of Algorithm 2.1, we find anω^k

new ∈such thatg(x^k,ω^k

new) > ρin the following way. We test each point in¯ to see whether there is a point satisfying g(x^k,ω^k

new) > ρ. If all points fail, then we setN = 100 and test each point in

the new¯ to find a point satisfyingg(x^k,ω^k

new) > ρ. If this fails again, then we setN = 1000 and test each point in the new¯ to seek a point satisfying g(x^k,ω^k

new) > ρ. If this fails again, then we set N = 10000, etc. In Step 3, we relax the criterionν^k+1(ω) > 0 toν^k+1(ω) > 10⁻⁶. We stop the iteration of Algorithm 2.1 whenmax{g(x^k,ω)|ω ∈ ¯} ≤ ρ, where¯ is defined with N=100000.

We implement Algorithm 2.1 on the following three problems with set- ting ρ := 10⁻⁶. The numerical results are summarized in Table 1, where EXAM denotes the experimented problems, NIT(NIFC) denotes the number of iterations and the cardinality of set _k at the final iteration, CPU(s) is the CPU time (in seconds) for solving each problem,FVALdenotes the final value of the objective function, and MAXG denotes the final value of the function max{g(x,ω)|ω∈ ¯}, where¯ is defined withN =100000.

For comparison, we also applyfseminf, that is a solver for SIP based on an implementation of the discretization SQP method in MATLAB toolbox, to solve the three examples in the formP[]. For the solverfseminf, we use all the default values.

Problem 1

f1(x)=x²₁+x⁴₂,

f2(x)=(x₁−2)²+(x₂−2)², g(x,ω)=5x²₁sin(π√

ω)/(1+ω²)−x₂ ≤0, = [0, 1].

After 4 iterations, we find the optimal solution

x₁ =0.514474445040588, x₂ =1.256745063664707.

Problem 2

f₁(x)=x²₁+x²₂+x²₃+x²₄−2x₁−5x₂−36x₃+7x₄,

f2(x)=11x²₁+11x²₂+12x²₃+11x²₄+5x1−15x2−11x3−3x4−80, f₃(x)=11x²₁+21x²₂+12x²₃+21x²₄−15x₁−5x₂−21x₃−3x₄−100, g(x,ω)=(1+ω²)²−x₁−x₂ω−x₃ω²−x₄ω³,

= [0, 1].

Table 1. Test results for Problems 1–3.

Algorithm EXAM NIT(NIFC) CPU (s) FVAL MAXG

Algorithm 2.1 Problem 1 3(1) 0.14 2.759214074824113 1.88e-007

Problem 2 6(2) 0.63 -55.468813235577016 1.11e-016

Problem 3 4(1) 0.23 -24.637013595823785 9.58e-008

fseminf Problem 1 4 1.09 2.759214129908945 4.04e-008

Problem 2 8698 217.83 -55.469542809472998 1.30e-004

Problem 3 19 1.34 -24.637012132368071 -2.12e-013

After 6 iterations, we find the optimal solution

x₁ =1.00000000, x₂= 1.1328729785, x₃ =1.5256254664, x₄ =0.3415015551.

Problem 3

f1(x)=x²₁+x²₂+2x²₃+x²₄−5x1−5x2−21x3+7x4,

f₂(x)=11x²₁+11x²₂+12x²₃+11x²₄+5x₁−15x₂−11x₃−3x₄−80, f3(x)=11x²₁+21x²₂+12x²₃+21x²₄−15x1−5x2−21x3−3x4−100, f4(x)=11x²₁+211x²₂+12x²₃+15x1−15x2−21x3−3x4−50, g(x,ω)=exp(ω)−x₁−x₂ω−x₃ω²−x₄ω³,

= [0, 1].

After 4 iterations, we find the optimal solution

x₁ =1.1808339962, x₂ =0.0756637247, x₃ =1.4436594485, x₄ =0.8178610799.

The numerical results reported inTable 1show that Algorithm 2.1 performs very well for all the tested problems and can give the optimal solutions. In particular, from the last columns of NIT, CPU (s), and MAXG inTable 1, it is easy to see that Algorithm 2.1 is effective. The solverfseminfis good for Problem 1 and Problem 3, but it is not good for Problem 2.

5. Concluding remarks

In this article, we present an exchange method for solving convex min-max problems with infinite constraints. The algorithm is given based on a sequence of auxiliary subproblems. Under reasonable assumptions we prove that the pro- posed algorithm has finite termination property and the approximate optimal solution of the original problem can be derived from the optimal solution of the subproblem at final iteration. Numerical results show that the algorithm is efficient.

Funding

This work is supported by the National Natural Science Foundation of China (Grant No.

11271221).

References

1. A. Auslender, M. A. Goberna, and M. A. López (2009). Penalty and smoothing methods for convex semi-infinite programming.Mathematics of Operations Research34:303–319.

2. J. W. Bandler, P. C. Liu, and H. Tromp (1976). A nonlinear programming approach to optimal design centering, tolerancing, and tunning. IEEE Transactions on Circuits and Systems 23:155–165.