Approximate inertial proximal methods using the enlargement of maximal monotone operators

HAL Id: hal-00779255

https://hal.univ-antilles.fr/hal-00779255

Submitted on 21 Jan 2013

HAL is a multi-disciplinary open access archive for the deposit and dissemination of sci- entific research documents, whether they are pub- lished or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

enlargement of maximal monotone operators

Abdellatif Moudafi, E. Elizabeth

To cite this version:

Abdellatif Moudafi, E. Elizabeth. Approximate inertial proximal methods using the enlargement of maximal monotone operators. International Journal of Pure and Applied Mathematics, Academic Publishing Ltd, 2003, 5 (3), pp.283-299. �hal-00779255�

using the enlargement of a maximal monotone operator

A. Moudafi and E. Elisabeth

Univeristé Antilles Guyane, DSI-GRIMAAG 97200 Schoelcher, Martinique, France.

[email protected]

Abstract:

An approximate procedure for solving the problem of nding a zero of a maximal monotone operator is proposed and its convergence is established under var- ious conditions. More precisely, it is shown that this method weakly converges under natural assumptions and strongly converges provided that either the inverse of the involved operator is Lipschitz continuous around zero or the interior of the solution set is nonempty. A particular attention is given to the convex minimization case.

AMSSubjectClassication:

Primary, 90C25; Secondary, 49M45, 65C25.

Key words.

monotone operators, elargements, proximal point algorithm, lo- cal Lipschitz continuity, approximate subdierential, convergence, convex min- imization.

1 Introduction and preliminaries

In this paper we will focus our attention on the classical problem of nding a zero of a maximal monotone operator

on a real Hilbert space

:

nd

such that

(1.1)

This is a well-known problem which includes, as special cases, optimization and min-max problems, complementarity problems, and variational inequalities.

One of the fondamental approaches to solving (1.1) is the proximal method, which generates the next iterates

by solving the subproblem

02

k

T(x)+(x?x

k

);

(1.2)

where

is the current iterate and

is a regularization parameter. The liter- ature on this subject is vast (see

for a survey).

1

Recently, an inertial proximal algorithm was proposed by Alvarez in

in the context of convex minimization. Afterwards, Attouch and Alvarez considered its extension to maximal monotone operators in

. It works as follows. Given

x

k ?1

;x

k

2 H

and two parameters

and

, nd

such that

k T(x

k +1 )+x

k +1

?x

k

?

k (x

k

?x

k ?1

)30:

(1.3)

It is well known that the proximal iteration may be interpreted as an implicit one-step discretisation method for the evolution dierential inclusion

dx

dt

(t)+T(x(t))30

a.e.

(1.4) where the parameter

is a (variable) stepsize. While the inspiration for (1.3) comes from the implicit discretization of the dierential system of the second- order in time, namely

d 2

x

dt 2

(t)+ dx

dt

(t)+T(x(t))30

a.e.

(1.5) where

is a damping or a friction parameter.

Under appropriate conditions on

and

Attouch

Alvarez proved that if the solution set

is nonempty, then for every sequence

gener- ated by (1.3), there exists an

such that

converges to

weakly in

as

.

For developing implementable computational techniques, it is of particular im- portance to treat the case when (1.3) is solved approximately. Before introduc- ing our approximate method, let us recall the following concepts which are of common use in the context of convex and nonlinear analysis. Troughout,

is a real Hilbert space,

denotes the associated scalar product and

stands for the corresponding norm. An operator is said to be monotone if

hu?v ;x?y i0

whenever

It is said to be maximalmonotone if, in addition, the graph,

T(x)g

, is not properly contained in the graph of any other monotone operator.

It is well-known that for each

and

there is a unique

such that

. The single-valued operator

is called the resolvent of

of parameter

. It is a nonexpansive mapping which is everywhere dened and satises:

, if and only if,

. Let us also recall a notion which is clearly inspired by the approximate subdierential. In [16], Iusem, Burachik and Svaiter dened

, an

-enlargement of a monotone operator

T

, as

T

"

(x):=fv2H;hu?v ;y?xi?" 8y ;u2T(y )g;

(1.6) where

. Since

is assumed to be maximal monotone,

, for any

. Furthermore, directly from the denition it follows that

0" " )T

"

1

(x)T

"

2

(x):

Thus

is an enlargement of

. The use of elements in

instead of

allows an extra degree of freedom, which is very useful in various applications. On the other hand, setting

one retrieves the original operator

, so that the classical method can be also treated. For all these reasons, we consider the following scheme: nd

such that

k T

"k

(x

k +1 )+x

k +1

?y

k

30:

(1.7)

where

are nonnegative real numbers.

We will impose the followingtolerance criteria on the term

which are standard in the literature:

+1

X

k =1

k

"

k

<+1;

(1.8)

and

p

k

"

k

k kx

k +1

?y

k

k

with

k =1

k

<+1:

(1.9)

The rst condition is typically needed to establish global convergence, while the second is required for local linear rate of convergence result under additional natural assumptions.

The remainder of the paper is organized as follows: In section 2, we present a weak convergence result for the sequence generated by (1.7) under criterion (1.8). We also consider various conditions for which the convergence is strong.

In section 3, we present an application to convex minimization and study the convergence of a perturbed version of (1.7) as well as conditions ensuring the convergence for both the values and the iterates.

2 The main results

2.1 A weak convergence result

Theorem 2.1 Let

be a sequence such that

02x

k +1

?x

k

?

k (x

k

?x

k ?1 )+

k T

"k

(x

k +1

);k=1;2;

where

is a maximal monotone operator with

, and the parameters

and

satisfy:

1.

such that

.

2.

such that

.

3.

.

If the followingconditionholds

+1

X

k =1

x

x

<

; (2.10)

then, there exists

x

S

x

x

k

Pro of

. Fix x

S

T

, since

x

x

x

x

T

x

, from denition (1.6) it follows that

h

x

x

x

x

;x

x

"

:

Dene the auxiliary real sequence '

x

x

. It is direct to check that

h

x

x

x

x

;x

x

'

'

x

x

?

x

x

;x

x

; and since

h

x

x

;x

x

x

x

;x

x

x

x

;x

x

=

'

'

x

x

x

x

;x

x

; it follows that

'

'

'

'

x

x

x

x

;x

x

+ k

2

j

x

x

"

= ?

1

2

j

x

x

x

x

+ k+

2

k

2

j

x

x

"

: Hence

'

'

'

'

2

j

v

x

x

"

: (2.11) Setting

'

'

and

x

x

"

, we obtain

; where

t

max

t;

, and consequently

[

; with

;

given by (2).

The rest of the proof follows that given in

and is presented here for com- pleteness and to convey the idea in

. The latter inequality yields

[

;

and therefore

1

X

k =1

[

]

1

1

([

]

+

k =1

k

)

which is nite thanks to (3) and (2.10). Consider the sequence dened by

t

k

:=

[

]

. Since

0 and

[

]

+

, it follows that

is bounded from below. But

t

k +1

=

[

]

i=1

[

]

+

i=1

[

]

=

so that

is nonincreasing. We thus deduce that

is convergent and so is

f'

k

g

. On the other hand, from (2.11) we obtain the estimate 1 2

+

[

]

+

Passing to the limit in the latter inequality and taking into account that

converges, [

]

and

go to zero as k tends to +

, we obtain

lim

k !+1 v

k +1

= 0

Now let

be a weak cluster point of

. There exists a subsequence

which converges weakly to

and satises

+

(

)

0. By denition (1.6), we have

h?

v

+1

?z;x

+1

?y i?"

8z2T

(

)

Passing to the limit, as

+

, we obtain

h?z;^x

0

this being true for any

(

), from maximal monotonicity of

, it follows that 0

(

), that is

. We end the proof by applying the well-known Opial's lemma [12].

Remark 2.1 1. Although condition (2.10) involves the iterates that are a priori unknown. In practice, as it was stressed by Alvarez & Attouch, it is easy to enforce it by applying an appropriate on-line rule. Further- more, condition (2.10) is automatically satised in some special cases (see proposition 2.1, [2]).

2. Under assumptions of theorem 2.1 and in view of its proof, it is clear that

fx

k

g

is bounded if, and only if, there exists at least one solution to (1.1).

2.2 Strong convergence results

First, we give a result showing that the criterion (1.9) ensures the strong con- vergence of the sequence generated by:

x

k +1

?y

k

+

(

)

0

(2.12) where

=

+

(

),

and

are positive reals.

Theorem 2.2 Let

be any sequence generated by (1.7) using criterion (1.9) with

nondecreasing (

+

). Assume that

is bounded,

9 2

[0

1[ such that

0

, and

is Lipschitz continuous around zero, i.e., problem (1.1) has the unique solution, say

, and there exist some constants

0 and

0 such that

jv j;v2T

(

)

Then, there is a real

]0

1[ and a range

such that

jx

k +1

?^xj

+ (2

+

k

)

Furthermore, if we assume that lim

= 0

then, the sequence

fx

k

g

strongly converges to

.

Proof . The rst part of the proof follows that given in [15] and is presented here for completeness. The sequence

being bounded, also satises condition 3) of theorem 2.1 for

=

, so the conclusions of theorem 2.1 are in force. Now, let ~

be the exact solution of the

-th inertial proximal method, that is

k

T^x

~

+ ~

0

(2.13) Denition of

combined with relations (2.12) and (2.13) leads to

hy

k

?x

k +1

?

(

~

)

~

which implies that

~

. The latter together with (1.9) yield

j^x

~

(2.14) Therefore

j^x

~

~

+

(1 +

)

which, using the convergence of

:=

0 (theorem 2.1), implies

that

~

0. Because

is bounded away from zero, we further

conclude that

=

(

~

)

0. Using the Lipschitz continuity of

around 0, we have, for indices

suciently large, that:

j^x

~

=

k

j^x

~

(2.15) We further obtain

jy

k

?^xj

=

~

+

~

+ 2

~

~

=

~

+

~

+ 2

~

jy

k

?^x

~

+

~

(1 + (

)

)

~

(2.16)

Hence, by setting

:=

+ 2

k

, we obtain

~

Using the latter relation and (2.14), we further obtain

jx

k +1

?^xj

~

+

~

k jx

k +1

?y

k

j

+

(2.17) Similarly,

jx

k +1

?y

k

j jx

k +1

?^x

~

+

~

k jx

k +1

?y

k

j

+

where also (2.16) was used in the last inequality.

Therefore

jx

k +1

?y

k j

1

1

Now, combining the latter relation with (2.17) and using the triangular inequal- ity, we obtain

jx

k +1

?^xj

+

where

:=

+

, from which we deduce easily, by taking into account conditions on

and

, the existence of a range

such that

jx

k +1

?^xj

+

where

]0

1[ and

:= (2

+

)

. Hence,

jx

k

?^xj

+

j=1

j

k ?j :

The result follows from Ortega and Rheinboldt [14

338], since by hypothesis

lim

= 0

Remark 2.2 1. When

= 0 , we obtain the linear convergence estimate obtained by Solodov & Svaiter [15] which is strictly better than the one for the classical proximal algorithm, namely,

~

=

( [14] , theorem 2).

2. The condition of Lipschitz continuity above holds true if

is globally Lipschitz continuous which is satised, for instance, when

is strongly monotone.

3. The condition of local Lipschitz continuity above is also satised if

is dierentiable at 0, that is,

(0) =

and

:

a continuous linear transformation such that, for

0

T

?1

(0)

+

(

)

if

where

stands for the closed unit ball.

We close this section with a special result showing that the Inertial Proximal Method can converge in nitely many iterations:

Theorem 2.3 Let

be any sequence generated by (1.8) under the criterion (1.9) with

bounded away from zero. Suppose that

is bounded and that

9^x

such that 0

i

(2.18) Then, for all

suciently large, it holds that

J T

k

(

+

(

)) =

Moreover, the sequence

strongly converges to

.

Proof . Let ~

be the exact solution of the

-th inertial proximal method, that is

k

T^x

~

+ ~

0

Hypothesis (2.18) and ([14], theorem 3) imply the existence of a positive real

such that

jxj")T

?1

x

=

But the hypothesis of the present theorem recovers those of theorem 2.1. there- fore, we know that

(

~

)

0. Since ~

(

(

~

)), assumption (2.18) implies that, for

suciently large, ~

=

. Thus the inertial proximal method in its exact form converges to

in a nite number of iterations from any starting points

and

. From which we deduce, for all

sucientely large, that

jx

k +1

?^xj

=

~

The strong convergence of the sequence

to

follows by passing to the limit

in the last inequality, since condition (1.8) implies that lim

= 0.

3 Convex Minimization

3.1 Approximate methods

An interesting case is obtained by taking

=

,

stands for the subdier- ential of a proper convex lower-semicontinuous function

:

+

. Indeed,

is well-known to be a maximalmonotone operator and problem (1.1) reduces to the one of nding a minimizer of the function

.

In [1], Alvarez proposed the following approximate inertial proximal method:

k

@

"

k

f

(

) +

(

)

0

(3.19) where

is the approximate subdierential of

. Since in the case

=

the enlargement given in (1.6) is larger than the the approxiamte subdierential, i.e.

@

"

f

(

)

, we can write

(

)

(

)

(

)

which leads to

k

(

)

(

) +

(

)

0

(3.20) which is a particular case of the method proposed in this paper with

=

. As a consequence of theorem 2.1 and theorem 2.2, we obtain the following convergence result which recover and completes a result by Alvarez [1].

Corollary 3.1 Let

be a sequence such that

0

(

) +

(

)

= 1

2

(3.21) where

is a proper closed convex function with Argmin

=

, and the param- eters

and

satisfy:

1.

0 such that

.

2.

[0

1[ such that

0

. 3.

+

.

If the following condition holds

+1

X

k =1

k jx

k

?x

k ?1 j

2

<

+

(3.22)

then

weakly converges to a minimizer of

and lim

k !+1

f

(

) = inf

x2H f

(

) . Moreover, if we replace condition 3) of theorem 2.1 by criterion (1.9) and we assume in addition that

is Lipschitz continuous around zero, the convergene is strong. The function

stands for the Fenchel conjuguate of

, namely,

f

(

) =

(

(

)) .

Remark 3.1 The formula lim

k !+1

f

(

) = inf

x2H

f

(

) is obtained from relation (3.19), denition of the approximate subdierential, lower semicontinuity of the function

and the fact that lim

k !+1 v

k +1

= 0 (see the proof of theorem 2.1).

3.2 A perturbed Inertial Proximal Method

When

is nonsmooth, subproblems (3.19) may be very hard to solve and several authors proposed to approximate

by a sequence

of more tractable convex functions (see for example [6], [7] and [11]).

In this section we consider

= 2

3

, a monotone decreasing sequence of proper closed convex function on

,

be such that

=

(inf

). Now,

x

0

;x

1

be given in

, let us consider a sequence

generated by the following perturbed method:

0

(

) +

(

)

= 2

3

(3.23) where the parameters

,

and

are nonnegative real numbers.

Theorem 3.1 Suppose that the sequence

is bounded and that the following condition holds true

lim

k !+1

"

k

= 0 and lim

k !+1

k

p

k jx

k ?1

?x

k ?2

j

= 0

(3.24) then lim

(

) = inf

and every weak cluster point of

is a mini- mizer of

.

Proof . For all

, by denition of the approximate subdierential, we have

f

k

(

)

(

)

1

k hx

k

?x

k ?1

?

k

(

)

(3.25) Setting

=

and taking into account the following inequalities

?hx

k

?x

k ?1

?

k

(

)

?

k hx

k ?1

?x

k ?2

;x

k

?x

k ?1 i

=

+

(

)

we infer that

f

k

(

) + 1 2

(

) + 2

+

where

:=

(

)

.

This combined with the fact that

decreases yields

f

k

(

) + 1 2

(

) + 2

+

(3.26)

Now let

=

(

)

(

)

.