Interior proximal methods and central paths for convex second-order cone programming

Contents lists available atScienceDirect

Nonlinear Analysis

journal homepage:www.elsevier.com/locate/na

Interior proximal methods and central paths for convex second-order cone programming

Shaohua Pan

^a

, Jein-Shan Chen

^b,∗,1

aDepartment of Mathematics, South China University of Technology, Guangzhou 510640, China

bDepartment of Mathematics, National Taiwan Normal University, Taipei 11677, Taiwan

a r t i c l e i n f o

Article history:

Received 18 November 2008 Accepted 28 June 2010

Keywords:

Convex second-order cone optimization Interior proximal methods

Proximal distances with respect to SOCs Convergence

Central path

a b s t r a c t

We make a unified analysis of interior proximal methods of solving convex second-order cone programming problems. These methods use a proximal distance with respect to second-order cones which can be produced with an appropriate closed proper univariate function in three ways. Under some mild conditions, the sequence generated is bounded with each limit point being a solution, and global rates of convergence estimates are obtained in terms of objective values. A class of regularized proximal distances is also constructed which can guarantee the global convergence of the sequence to an optimal solution. These results are illustrated with some examples. In addition, we also study the central paths associated with these distance-like functions, and for the linear SOCP we discuss their relations with the sequence generated by the interior proximal methods. From this, we obtain improved convergence results for the sequence for the interior proximal methods using a proximal distance continuous at the boundary of second-order cones.

©2010 Elsevier Ltd. All rights reserved.

1. Introduction

We consider the following convex second-order cone programming problem (CSOCP):

inff

(

^x

)

s

.

^t

.

^Ax

=

b

,

^x

_K0

,

⁽¹⁾

wheref:Rⁿ

→

_R

∪ {+∞}

is a closed proper convex function,Ais anm

×

nmatrix with full row rankm

,

^bis a vector in R^m

,

^x

_K0 meansx

∈

_K, andKis the Cartesian product of some second-order cones (SOCs), also called Lorentz cones [1].

In other words,

K

=

_Kⁿ¹

×

_Kⁿ²

× · · · ×

_K^nr (2)

wherer

,

ⁿ1

, . . . ,

ⁿr

≥

1 withn₁

+ · · · +

n_r

=

n, and Kⁿⁱ

:=

(

^x1

,

^x2

) ∈

_R

×

_Rⁿⁱ⁻¹

|

x₁

≥ k

x₂

k

with

k · k

being the Euclidean norm. Whenf reduces to a linear function, i.e.f

(

^x

) =

c^Txfor somec

∈

_Rⁿ,(1)becomes the standard SOCP. Throughout this paper, we denote byX∗the optimal set of(1), and letV

:= {

x

∈

_Rⁿ

|

Ax

=

b

}

. The

∗Corresponding author. Tel.: +886 2 29325417; fax: +886 2 29332342.

E-mail addresses:[email protected](S. Pan),[email protected](J.-S. Chen).

1 Member of the Mathematics Division, National Center for Theoretical Sciences, Taipei Office.

0362-546X/$ – see front matter©2010 Elsevier Ltd. All rights reserved.

doi:10.1016/j.na.2010.06.079

CSOCP, as an extension of the standard SOCP, has a wide range of applications from engineering, control, and finance to robust optimization and combinatorial optimization; see [2,3] and references therein.

There have proposed various methods for solving the CSOCP, which include the interior point methods [4–6], the smoothing Newton methods [7,8], the smoothing–regularization method [9], the semismooth Newton method [10], and the merit function method [11]. These methods are all developed by reformulating the KKT optimality conditions as a system of equations or an unconstrained minimization problem. This paper will focus on an iterative scheme which is proximal based and handles directly the CSOCP itself. Specifically, the proximal-type algorithm consists of generating a sequence

{

x^k

}

via

x^k

:=

argmin

λ

kf

(

^x

) +

H

(

^x

,

^xk−1

) |

x

∈

_K

∩

_V

,

^k

=

1

,

²

, . . .

⁽³⁾

where

{ λ

k

}

is a sequence of positive parameters, andH:Rⁿ

×

_Rⁿ

→

_R

∪ {+∞}

is a proximal distance with respect to intK (seeDefinition 3.1) which plays the same role as the Euclidean distance

k

x

−

y

k

²in the classical proximal algorithms (see, e.g., [12,13]), but possesses certain more desirable properties for forcing the iterates to stay inK

∩

_V, thus eliminating the constraints automatically. As will be shown in Section4, such proximal distances can be produced with an appropriate closed proper univariate function.

In this paper, under mild assumptions like those used in interior proximal methods for convex programs over nonnegative orthant cones (see, e.g., [14–20]), we show that the sequence

{

x^k

}

is bounded with all limit points, being a solution of(1), and obtain global rates of convergence in terms of objective values. But, unlike for interior proximal methods for convex programs over nonnegative orthant cones, the global convergence of

{

x^k

}

to an optimal solution can be guaranteed for the class of proximal distancesF1

(

^K

)

^orF2

(

^K

)

under a very restrictive assumption forX∗(seeTheorem 3.2(a)), or for their subclassesFb1

(

^Kn

)

^orFb2

(

^Kn

)

under mild assumptions forX∗(seeTheorem 3.2(b)), or for the smallest subclass_F

¯

2

(

^Kn

)

^. These results are illustrated with some examples.

Just like proximal point methods with generalized distances, the central paths derived from barrier functions have been the object of intensive study. Recently, the central paths for semidefinite programming were under active study (see, e.g., [21–24]). For example, da Cruz Neto et al. [21] established relations among the central paths in semidefinite programming, generalized proximal point methods, and Cauchy trajectories in Riemannian manifolds, extending the results of Iusem et al. [25] for monotone variational inequality problems. Motivated by this, we also investigate the properties of the central paths of(1)with respect to (w.r.t.) the distance-like functions used by interior proximal methods (seePropositions 5.2 and5.3). For the linear SOCP, we discuss the relations between the central paths and the sequences generated by the interior proximal methods, and show that the sequence generated by interior proximal methods will converge under the usual assumptions if the proximal distance satisfies a certain continuity at the boundary of second-order cones (seeTheorem 5.2).

Auslender and Teboulle [15] provided a unified technique for analyzing and designing interior proximal methods for convex and conic optimization. However, for the CSOCP, we notice that it seems hard to find a proximal distance example for the classF+

(

Kⁿ

)

such that global convergence results similar to those for [15, Theorem 2.2] can apply for it. In this paper, we extend their unified analysis technique to interior proximal methods using a proximal distance which can be produced with an appropriate univariate function in three ways, and establish the global convergence results for the smallest class_F

¯

₂

(

^Kn

)

^, and the classFb2

(

^Kn

)

with some mild assumptions ofX∗. The examples from the two classes of proximal distances are easy to find. In particular, for the linear SOCP, we obtain improved convergence results for these interior proximal methods, by exploring the relations between the sequence generated by the interior proximal methods and the central path associated with the corresponding proximal distances. In view of these contexts, this paper can be regarded as a refinement of [15] for the second-order cone optimization.

Throughout this paper,Idenotes an identity matrix of suitable dimension andRⁿdenotes the space ofn-dimensional real column vectors. For anyx

,

^y

∈

_Rⁿ, we writex

_Knyifx

−

y

∈

_Kⁿ; and we writex

_Knyifx

−

y

∈

intKⁿ. Given a matrix E

,

^Im

(

^E

)

means the subspace generated by the columns ofE. A function is closed if and only if it is lower semicontinuous (lsc), and a function is proper iff

(

^x

) < ∞

for at least onex

∈

_Rⁿandf

(

^x

) > −∞

for allx

∈

_Rⁿ. For a lsc proper convex functionf

:

_Rⁿ

→

_R

∪ {+∞}

, we denote its domain by domf

:= {

x

∈

_Rⁿ

|

f

(

^x

) < ∞}

and the

-subdifferential offat

¯

xby

∂

f

(

_x

¯ ) := { w ∈

_Rⁿ

|

f

(

^x

) ≥

f

( ¯

_x

) + h w,

^x

− ¯

x

i − , ∀

x

∈

_Rⁿ

}

. Iffis differentiable atx

, ∇

f

(

^x

)

means the gradient offatx. For a differentiablehonR

,

^h0andh00denote its first and second derivatives. For any closed setS

,

^intSdenotes the interior ofS.

In the rest of this paper, we focus on the case whereK

=

_Kⁿ, and all the analysis can be carried over to the case where Khas the direct product structure as in(2). Unless otherwise stated, we make the following minimal assumption for the CSOCP(1):

(A1) domf

∩ (

^V

∩

intKⁿ

) 6= ∅

andf∗

:=

inf

{

f

(

^x

) |

x

∈

_V

∩

_Kⁿ

} > −∞

. 2. Preliminaries

This section recalls some preliminary results that will be used in the subsequent sections. For anyx

= (

^x1

,

^x2

),

^y

= (

^y1

,

^y2

) ∈

_R

×

_Rⁿ⁻¹, their Jordan product [1] is defined as

x

◦

y

:= ( h

x

,

^y

i ,

^y1x₂

+

x₁y₂

).

⁽⁴⁾

It is easy to verify that the identity element under the Jordan product ise

≡ (

¹

,

⁰

, . . . ,

⁰

)

^T

∈

_Rⁿ, i.e.,e

◦

x

=

xfor allx

∈

_Rⁿ. Note that the Jordan product is not associative, but it is power associated, i.e.,x

◦ (

^x

◦

x

) = (

^x

◦

x

) ◦

xfor allx

∈

_Rⁿ. Thus, we

may without fear of ambiguity writex^mfor the product ofmcopies ofxandx^m+n

=

x^m

◦

xⁿfor all positive integersmand n. n We stipulatex⁰

=

e. For eachx

= (

^x1

,

^x2

) ∈

_R

×

_Rⁿ⁻¹, let

det

(

^x

) :=

x²₁

− k

x₂

k

² and tr

(

^x

) :=

2x₁

.

⁽⁵⁾

These are called thedeterminantand thetraceofx, respectively. A vectorxis said to beinvertibleif det

(

^x

) 6=

0. Ifx

∈

_Rⁿis invertible, there is a uniquey

∈

_Rⁿsatisfyingx

◦

y

=

y

◦

x

=

e. We call thisythe inverse ofxand denote it byx⁻¹.

We recall from [1] that eachxadmits a spectral factorization associated withKⁿ:

x

= λ

1

(

^x

)

^u(x¹⁾

+ λ

2

(

^x

)

^u(x²⁾

,

⁽⁶⁾

where

λ

i

(

^x

)

^andu(xⁱ⁾fori

=

1

,

2 are the spectral values ofx

= (

^x1

,

^x2

) ∈

_R

×

_Rⁿ⁻¹and the associated spectral vectors, defined by

λ

i

(

^x

) =

x₁

+ ( −

1

)

ⁱ

k

x₂

k ,

^u(xⁱ⁾

=

¹

2 1

, ( −

1

)

ⁱx

¯

₂

,

⁽⁷⁾

withx

¯

₂

=

^x2

kx2kifx₂

6=

0, otherwise being any vector inRⁿ⁻¹such that

k¯

x₂

k =

1. Ifx₂

6=

0, then the factorization is unique.

The following lemma is direct by formula(6).

Lemma 2.1. For any x

= (

^x1

,

^x2

),

^y

= (

^y1

,

^y2

) ∈

_R

×

_Rⁿ⁻¹, the following results hold:

(a) det

(

^x

) = λ

1

(

^x

)λ

2

(

^x

),

^tr

(

^x

) = λ

1

(

^x

) + λ

2

(

^x

)

^and

k

x

k

²

=

¹

2

(λ

1

(

^x

))

²

+ (λ

2

(

^x

))

² . (b)x

∈

_Kⁿ

⇐⇒ λ

1

(

^x

) ≥

_{0and x}

∈

_intKⁿ

⇐⇒ λ

1

(

^x

) >

^0.

(c)

λ

1

(

^x

)λ

2

(

^y

) + λ

2

(

^x

)λ

1

(

^y

) ≤

tr

(

^x

◦

y

) ≤ λ

1

(

^x

)λ

1

(

^y

) + λ

2

(

^x

)λ

2

(

^y

)

^.

With the spectral factorization above, one may define a vector-valued function using a univariate function. For any given h: I_R

→

_Rwith I_R

⊆

_R, defineh^soc:S

→

_Rⁿby

h^soc

(

^x

) :=

h

(λ

1

(

^x

)) ·

u⁽_x¹⁾

+

h

(λ

2

(

^x

)) ·

u⁽_x²⁾

, ∀

x

∈

S

.

⁽⁸⁾ The definition is unambiguous whetherx₂

6=

0 orx₂

=

0. For example, leth

(

^t

) =

t⁻¹for anyt

>

0; then using formulas (6)and(8)we can compute that

x⁻¹

:=

h^soc

(

^x

) =

¹

x²₁

− k

x₂

k

²

(

^x1

, −

x₂

) =

^tr

(

^x

)

^e

−

x

det

(

^x

)

^forx

∈

intKⁿ

.

⁽⁹⁾

Moreover, by Lemma 2.2 of [26],Sis open whenever I_Ris open, andSis closed whenever I_Ris closed. The following lemma shows that some favorable properties ofhcan be transmitted toh^soc, whose proofs were given in Proposition 5.1 of [8] and Lemma 2.2 of [27].

Lemma 2.2. Given h: I_R

→

_RwithI_R

⊆

_R, let h^soc:S

→

_Rⁿbe the vector-valued function induced by h via(8), where S

⊆

_Rⁿ. Then, the following results hold:

(a)If h is continuously differentiable onint I_R, then h^socis continuously differentiable onintS, and for any x

∈

intS with x

= (

^x1

,

^x2

) ∈

_R

×

_Rⁿ⁻¹,

∇

_h^soc

(

^x

) =











h⁰

(

^x1

)

^{I if x}2

=

0

,









b c x^T₂

k

x₂

k

c x₂

k

x₂

k

^aI

+ (

^b

−

a

) k

^xx²₂^x

k

^T22









otherwise

where a

=

^h(λ_λ^{2(x))−h(λ1(x))}

2(x)−λ1(x)

,

^b

=

^{h0(λ2(x))+h0(λ1(x))}

2

,

^c

=

^{h0(λ2(x))−h0(λ1(x))}

2 .

(b)If h is continuously differentiable onint I_R, thentr

(

^hsoc

(

^x

))

is continuously differentiable onintS with

∇

tr

(

^hsoc

(

^x

)) =

2

∇

h^soc

(

^x

)

^e

=

2

(

^h0

)

^soc

(

^x

)

^.

(c) If h is (strictly) convex onI_R, thentr

(

^hsoc

(

^x

))

is (strictly) convex on S.

Lemma 2.3. (a) The real-valued functionln

(

^det

(

^x

))

is strictly concave onintKⁿ. (b)For any x

,

^y

∈

intKⁿwith x

6=

y, it holds that

det

(α

^x

+ (

¹

− α)

^y

) > (

^det

(

^x

))

^α

(

^det

(

^y

))

^1−α

, ∀ α ∈ (

⁰

,

¹

).

Proof. Clearly, part (b) is a direct consequence of part (a). The proof of part (a) was given in [28, Prop. 2.4(a)] by computing the Hessian matrix of ln

(

^det

(

^x

))

. Here, we give a simpler proof. Let lnxbe the vector-valued function induced by lntvia (8). FromLemma 2.1(a), ln

(

^det

(

^x

)) =

_ln

(λ

1

(

^x

)) +

_ln

(λ

2

(

^x

)) =

_tr

(

^lnx

)

^{for anyx}

∈

_intKⁿ. The result is then direct by Lemma 2.2(c) and the strict concavity of lnt

(

^t

>

⁰

)

^.

To close this section, we review the definition of SOC-convexity and SOC-monotonicity. The two concepts, like matrix- convexity and the matrix-monotonicity in semidefinite programming, play an important role in the solution methods of SOCPs.

Definition 2.1 ([28]).Givenh: I_R

→

_Rwith I_R

⊆

_R. Leth^soc:S

→

_RⁿwithS

⊆

_Rⁿbe the vector-valued function induced byhvia formula(8). Then,

(a) his said to be SOC-convex of ordernon I_Rif for anyx

,

^y

∈

_{Sand 0}

≤ β ≤

_1,

h^soc

(β

^x

+ (

¹

− β)

^y

)

_Kn

β

^hsoc

(

^x

) + (

¹

− β)

^hsoc

(

^y

).

⁽¹⁰⁾ (b) his said to be SOC-monotone of ordernon I_Rif for anyx

,

^y

∈

S,

x

_Kny

H⇒

h^soc

(

^x

)

_Knh^soc

(

^y

).

We say thathis SOC-convex (respectively, SOC-monotone) on I_Rifhis SOC-convex of all ordersn(respectively, SOC- monotone of all ordersn) on I_R. A functionhis said to be SOC-concave on I_Rwhenever

−

his SOC-convex on I_R. Whenh is continuous on I_R, the condition in(10)can be replaced by a more special condition:

h^soc x

+

y

2

_Kn

1

2

(

^hsoc

(

^x

) +

h^soc

(

^y

)).

⁽¹¹⁾

Obviously, the set of SOC-monotone functions and the set of SOC-convex functions are both closed under positive linear combinations and under pointwise limits.

For the characterizations of SOC-convexity and SOC-monotonicity, the interested reader may refer to [28,29]. The following lemma collects some common SOC-concave functions whose proofs can be found in [27] or are direct by Lemma 3.2 of [27].

Lemma 2.4. (a)For any fixed u

∈

_R, the function h

(

^t

) = (

^t

+

u

)

^{r with r}

∈ [

0

,

¹

]

is SOC-concave and SOC-monotone on

[−

u

, +∞ )

^.

(b) For any fixed u

∈

_R, the function h

(

^t

) = − (

^t

+

u

)

^{−rwith r}

∈ [

0

,

¹

]

is SOC-concave and SOC-monotone on

( −

u

, +∞ )

^. (c) For any fixed

α ≥

0

,

^ln

(α +

t

)

is SOC-concave and SOC-monotone on

[−

a

, +∞ )

^.

(d)For any fixed u

≥

₀

,

_u+^tt is SOC-concave and SOC-monotone on

( −

_u

, +∞ )

^. 3. Interior proximal methods

First of all, we present the definition of a proximal distance w.r.t. the open cone intKⁿ.

Definition 3.1. An extended-valued functionH:Rⁿ

×

_Rⁿ

→

_R

∪ {+∞}

is called a proximal distance with respect to intKⁿ if it satisfies the following properties:

(P1) domH

( · , · ) =

_C1

×

_C2with intKⁿ

×

intKⁿ

⊂

_C1

×

_C2

⊆

_Kⁿ

×

_Kⁿ.

(P2) For each giveny

∈

intKⁿ

,

^H

( · ,

^y

)

is continuous and strictly convex onC1, and it is continuously differentiable on intKⁿwith dom

∇

₁H

( · ,

^y

) =

intKⁿ.

(P3) H

(

^x

,

^y

) ≥

0 for allx

,

^y

∈

_Rⁿ, andH

(

^y

,

^y

) =

0 for ally

∈

intK^n.

(P4) For each fixedy

∈

_C2, the sets

{

x

∈

_C1

:

H

(

^x

,

^y

) ≤ γ }

are bounded for all

γ ∈

_R.

Definition 3.1has a little difference from Definition 2.1 of [15] for a proximal distance w.r.t. intKⁿ, since hereH

( · ,

^y

)

^is required to be strictly convex overC1for any fixedy

∈

intKⁿ. We denote byD

(

^intKn

)

the family of functionsHsatisfying Definition 3.1. With a givenH

∈

_D

(

^intKn

)

, we have the following basic iterative algorithm for(1).

Interior Proximal Algorithm (IPA).GivenH

∈

_D

(

^intKⁿ

)

^andx0

∈

_V

∩

intK^{n, fork}

=

1

,

²

, . . .

^{, with}

λ

k

>

^{0 and}

k

≥

0, generate a sequence

{

x^k

} ⊂

_V

∩

intKⁿwithg^k

∈ ∂

kf

(

^xk

)

via the following iterative scheme:

x^k

:=

argmin

λ

kf

(

^x

) +

H

(

^x

,

^xk−1

) |

x

∈

_V (12)

such that

λ

kg^k

+ ∇

₁H

(

^xk

,

^xk−1

) =

A^Tu^k for someu^k

∈

_R^m

.

⁽¹³⁾

The following proposition implies that the IPA is well-defined, and moreover, from its proof we see that the iterative formula(12)is equivalent to the iterative scheme(3). When

k

>

^{0 for anyk}

∈

_N(the set of natural numbers), the IPA can be viewed as an approximate interior proximal method, and it becomes exact if

k

=

0 for allk

∈

_N.

Proposition 3.1. For any given H

∈

_D

(

^intKⁿ

)

^{and y}

∈

intKⁿ, consider the problem

f∗

(

^y

, τ) =

inf

{ τ

^f

(

^x

) +

H

(

^x

,

^y

) |

x

∈

_V

}

with

τ >

⁰

.

⁽¹⁴⁾

Then, for each

≥

0, there exist x

(

^y

, τ) ∈

_V

∩

intKⁿand g

∈ ∂

f

(

^x

(

^y

, τ))

^{such that}

τ

^g

+ ∇

₁H

(

^x

(

^y

, τ),

^y

) =

A^Tu (15)

for some u

∈

_R^m. Moreover, for such x

(

^y

, τ)

^{, we have}

τ

^f

(

^x

(

^y

, τ)) +

H

(

^x

(

^y

, τ),

^y

) ≤

f∗

(

^y

, τ) + .

Proof. SetF

(

^x

, τ) := τ

^f

(

^x

) +

H

(

^x

,

^y

) + δ

V∩Kⁿ

(

^x

)

^{, where}

δ

V∩Kⁿ

(

^x

)

is the indicator function defined on the setV

∩

_Kⁿ. Since domH

( · ,

^y

) =

_C1

⊂

_Kⁿ, it is clear that

f∗

(

^y

, τ) =

inf

F

(

^x

, τ) |

x

∈

_Rⁿ

.

⁽¹⁶⁾

Sincef∗

> −∞

, it is easy to verify that for any

γ ∈

_Rthe following relation holds:

x

∈

_Rⁿ

|

F

(

^x

, τ) ≤ γ ⊂

x

∈

_V

∩

_Kⁿ

|

H

(

^x

,

^y

) ≤ γ − τ

^f∗

⊂ {

_x

∈

_C1

|

_H

(

^x

,

^y

) ≤ γ − τ

^f∗

} ,

which together with (P4) implies thatF

( · , τ)

has bounded level sets. In addition, by (P1)–(P3),F

( · , τ)

is a closed proper and strictly convex function. Hence, the problem(16)has a unique solution, sayx

(

^y

, τ)

. From the optimality conditions of(16), we get

0

∈ ∂

^F

(

^x

(

^y

, τ)) = τ∂

^f

(

^x

(

^y

, τ)) + ∇

_1H

(

^x

(

^y

, τ),

^y

) + ∂δ

V∩_Kⁿ

(

^x

(

^y

, τ))

where the equality is due to Theorem 23.8 of [30] and domf

∩ (

^V

∩

intKⁿ

) 6= ∅

. Notice that dom

∇

₁H

( · ,

^y

) =

intKⁿand dom

∂δ

V∩Kⁿ

( · ) =

_V

∩

_Kⁿ. Therefore, the last equation impliesx

(

^y

, τ) ∈

_V

∩

intKⁿ, and there existsg

∈ ∂

^f

(

^x

(

^y

, τ))

^such that

− τ

^g

− ∇

₁H

(

^x

(

^y

, τ),

^y

) ∈ ∂δ

V∩_Kⁿ

(

^x

(

^y

, τ)).

On the other hand, by the definition of

δ

V∩Kⁿ

( · )

, it is not hard to derive that

∂δ

V∩Kⁿ

(

^x

) =

Im

(

^AT

) ∀

x

∈

_V

∩

intKⁿ

.

The last two equations imply that(15)holds for

=

0. When

>

^0,(15)also holds for suchx

(

^y

, τ)

^andgsince

∂

^f

(

^x

(

^y

, τ)) ⊂

∂

f

(

^x

(

^y

, τ))

. Finally, since for eachy

∈

intKⁿthe functionH

( · ,

^y

)

is strictly convex, and sinceg

∈ ∂

f

(

^x

(

^y

, τ))

^{, we have}

τ

^f

(

^x

) +

H

(

^x

,

^y

) ≥ τ

^f

(

^x

(

^y

, τ)) +

H

(

^x

(

^y

, τ),

^y

) + h τ

^g

+ ∇

₁H

(

^x

(

^y

, τ),

^y

),

^x

−

x

(

^y

, τ) i −

= τ

^f

(

^x

(

^y

, τ)) +

H

(

^x

(

^y

, τ),

^y

) + h

A^Tu

,

^x

−

x

(

^y

, τ) i −

= τ

^f

(

^x

(

^y

, τ)) +

H

(

^x

(

^y

, τ),

^y

) −

^{for allx}

∈

_V

,

where the first equality is from(15)and the last one is byx

,

^x

(

^y

, τ) ∈

_{V. Thus,f∗}

(

^y

, τ) =

_inf

{ τ

^f

(

^x

) +

_H

(

^x

,

^y

) |

_x

∈

_V

} ≥ τ

^f

(

^x

(

^y

, τ)) +

H

(

^x

(

^y

, τ),

^y

) −

^.

In the rest of this section, we focus on the convergence behaviors of the IPA withHfrom several subclasses ofD

(

^intKn

)

^, which also satisfy one of the following properties.

(P5) For anyx

,

^y

∈

intKⁿandz

∈

_C1

,

^H

(

^z

,

^y

) −

H

(

^z

,

^x

) ≥ h∇

₁H

(

^x

,

^y

),

^z

−

x

i

.

(

^P50

)

^{For anyx}

,

^y

∈

intKⁿandz

∈

_C2,H

(

^y

,

^z

) −

H

(

^x

,

^z

) ≥ h∇

₁H

(

^x

,

^y

),

^z

−

x

i

.

(P6) For eachx

∈

_C1, the level sets

{

y

∈

_C2

:

H

(

^x

,

^y

) ≤ γ }

are bounded for all

γ ∈

_R.

Specifically, we denote asF1

(

^intKn

)

^andF2

(

^intKn

)

the families of functionsH

∈

_D

(

^intKn

)

satisfying (P5) and

(

^P50

)

^, respectively. IfC1

=

_Kⁿ, we denote asF1

(

^Kn

)

the family of functionsH

∈

_D

(

^intKn

)

satisfying (P5) and (P6). IfC2

=

_Kⁿ, we writeF2

(

^intKⁿ

)

^asF

(

Kⁿ

)

. It is easy to see that the class of proximal distanceF

(

^intKⁿ

)

(respectively,F

(

Kⁿ

)

^{) in [15]}

subsumes the

(

^H

,

^H

)

^withH

∈

_F1

(

^intKn

)

(respectively,F1

(

^Kn

)

), but it does not include any

(

^H

,

^H

)

^withH

∈

_F2

(

^intKn

)

(respectively,F2

(

^Kn

)

^).

Theorem 3.1. Let

{

x^k

}

be the sequence generated by the IPA with H

∈

_F1

(

^intKn

)

^{or H}

∈

_F2

(

^intKn

)

^{. Set}

σ

ν

=

P_ν

k=1

λ

k. Then, the following results hold:

(a)f

(

^xν

) −

f

(

^x

) ≤ σ

_ν^−1H

(

^x

,

^x0

) + σ

_ν⁻¹P_ν

k=1

σ

k

kfor any x

∈

_V

∩

_C1if H

∈

_F1

(

^intKn

)

^{; f}

(

^xν

) −

f

(

^x

) ≤ σ

_ν^−1H

(

^x0

,

^x

) + σ

_ν⁻¹P_ν

k=1

σ

k

kfor any x

∈

_V

∩

_C2if H

∈

_F2

(

^intKn

)

^. (b)If

σ

ν

→ +∞

and

k

→

0, thenlim inf_ν→∞f

(

^xν

) =

f∗. (c) The sequence

{

f

(

^xk

) }

converges to f∗wheneverP∞

k=1

k

< ∞

.

(d)If X∗

6= ∅

, then

{

x^k

}

is bounded with all limit points in X∗under

(

^d1

)

^or

(

^d2

)

^: (d1) X∗is bounded andP∞

k=1

k

< ∞

; (d2) P∞

k=1

λ

k

k

< ∞

and H

∈

_F1

(

^Kn

) (

^{or H}

∈

_F2

(

^Kn

))

^.

Proof. The proofs are similar to those of [15, Theorem 4.1]. For completeness, we here takeH

∈

_F2

(

^intKn

)

for example to prove the results.

(a) Sinceg^k

∈ ∂

kf

(

^xk

)

, from the definition of the subdifferential, it follows that f

(

^x

) ≥

f

(

^xk

) + h

g^k

,

^x

−

x^k

i −

k

∀

x

∈

_Rⁿ

.

This, together with Eq.(13), implies that

λ

k

(

^f

(

^xk

) −

f

(

^x

)) ≤ h∇

₁H

(

^xk

,

^xk−1

),

^x

−

x^k

i + λ

k

k

∀

x

∈

_V

∩

_C2

.

Using

(

^P50

)

^withx

=

x^k

,

^y

=

x^k−1andz

=

x

∈

_V

∩

_C2, it then follows that

λ

k

(

^f

(

^xk

) −

f

(

^x

)) ≤

H

(

^xk−1

,

^x

) −

H

(

^xk

,

^x

) + λ

k

k

∀

x

∈

_V

∩

_C2

.

⁽¹⁷⁾ Summing overk

=

1

,

²

, . . . , ν

in this inequality yields that

− σ

νf

(

^x

) +

Xν

k=₁

λ

kf

(

^xk

) ≤

H

(

^x0

,

^x

) −

H

(

^xν

,

^x

) +

Xν

k=₁

λ

k

k

.

⁽¹⁸⁾

On the other hand, settingx

=

x^k−1in(17), we obtain f

(

^xk

) −

f

(

^xk−1

) ≤ λ

⁻k¹

H

(

^xk−1

,

^xk−1

) −

H

(

^xk

,

^xk−1

)

+

k

≤

k

.

⁽¹⁹⁾

Multiplying the inequality by

σ

k−1(with

σ

0

≡

0) and summing overk

=

1

, . . . , ν

^{, we get} Xν

k=1

σ

k−1f

(

^xk

) −

Xν

k=1

σ

k−1f

(

^xk−1

) ≤

Xν

k=1

σ

k−1

k

.

Noting that

σ

k

= λ

k

+ σ

k−1with

σ

0

≡

0, the above inequality can reduce to

σ

νf

(

^xν

) −

Xν

k=1

λ

kf

(

^xk

) ≤

Xν

k=1

σ

k−1

k

.

⁽²⁰⁾

Adding the inequalities(18)and(20)and recalling that

σ

k

= λ

k

+ σ

k−1, it follows that f

(

^xν

) −

f

(

^x

) ≤ σ

_ν⁻¹

H

(

^x0

,

^x

) −

H

(

^xν

,

^x

)

+ σ

_ν⁻¹X^ν

k=1

σ

k

k

∀

x

∈

_V

∩

_C2

,

which immediately implies the desired result due to the nonnegativity ofH

(

^xν

,

^x

)

^.

(b) If

σ

ν

→ +∞

and

k

→

0, then applying Lemma 2.2(ii) of [15] witha_k

=

kand b_ν

:= σ

_ν⁻¹P_ν

k=1

λ

k

kyields

σ

_ν⁻¹P_ν

k=1

λ

k

k

→

0. From part (a), it then follows that lim inf

ν→∞ f

(

^xν

) ≤

inf

f

(

^x

) |

x

∈

_V

∩

intKⁿ

.

This together withf

(

^xν

) ≥

inf

{

f

(

^x

) |

x

∈

_V

∩

_Kⁿ

}

implies that lim inf

ν→∞ f

(

^xν

) =

inf

f

(

^x

) |

x

∈

_V

∩

intKⁿ

=

f∗

.

(c) From(19), 0

≤

f

(

^xk

) −

f∗

≤

f

(

^xk−1

) −

f∗

+

k. Using Lemma 2.1 of [15] with

γ

k

≡

0 and

v

k

=

f

(

^xk

) −

f∗, we have that

{

f

(

^xk

) }

converges tof∗wheneverP∞

k=1

k

< ∞

.

(d) If the condition (d1) holds, then the sets

{

x

∈

_V

∩

_Kⁿ

|

f

(

^x

) ≤ γ }

are bounded for all

γ ∈

_R, sincef is closed proper convex andX∗

= {

x

∈

_V

∩

_Kⁿ

|

f

(

^x

) ≤

f∗

}

. Note that(19)implies

{

x^k

} ⊂ {

x

∈

_V

∩

_Kⁿ

|

f

(

^x

) ≤

f

(

^x0

) +

Pk

j=1

j

}

. Combining withP∞

k=1

k

< ∞

, clearly we have that

{

x^k

}

is bounded. Since

{

f

(

^xk

) }

converges tof∗andf is lsc, passing to the limit and recalling that

{

x^k

} ⊂

_V

∩

_Kⁿyields that each limit point of

{

x^k

}

is a solution of(1).

Suppose that the condition (d2) holds. IfH

∈

_F2

(

^Kn

)

, then inequality(17)holds for eachx

∈

_V

∩

_Kⁿ, and particularly forx∗

∈

X∗. Consequently,

H

(

^xk

,

^x∗

) ≤

H

(

^xk−1

,

^x∗

) + λ

k

k

∀

x∗

∈

X∗

.

⁽²¹⁾

Summing overk

=

1

,

²

, . . . , ν

for the last inequality, we obtain H

(

^xν

,

^x∗

) ≤

H

(

^x0

,

^x∗

) +

Xν

k=1

λ

k

k

.