Optimality conditions in variational form for non-linear constrained stochastic control problems

(1)

HAL Id: hal-03113195

https://hal.inria.fr/hal-03113195

Submitted on 18 Jan 2021

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are pub- lished or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Optimality conditions in variational form for non-linear constrained stochastic control problems

Laurent Pfeiffer

To cite this version:

Laurent Pfeiffer. Optimality conditions in variational form for non-linear constrained stochastic control problems. Mathematical Control & Related Fields, 2020, 10 (3), pp.493-526. �10.3934/mcrf.2020008�.

�hal-03113195�

(2)

AIMS’ Journals

VolumeX, Number0X, XX200X pp.X–XX

OPTIMALITY CONDITIONS IN VARIATIONAL FORM FOR NON-LINEAR CONSTRAINED STOCHASTIC CONTROL

PROBLEMS

Laurent Pfeiffer

Institute of Mathematics and Scientific Computing, University of Graz Heinrichstrasse 36, 8010 Graz, Austria

(Communicated by the associate editor name)

Abstract. Optimality conditions in the form of a variational inequality are proved for a class of constrained optimal control problems of stochastic differential equations. The cost function and the inequality constraints are functions of the probability distribution of the state variable at the final time. The analysis uses in an essential manner a convexity property of the set of reachable probability distributions. An augmented Lagrangian method based on the obtained optimality conditions is proposed and analyzed for solving iteratively the problem. At each iteration of the method, a standard stochastic optimal control problem is solved by dynamic programming. Two academical examples are investigated.

1. Introduction. This article is devoted to the derivation of optimality conditions for a class of control problems of stochastic differential equations (SDE). For a given adapted control process u, let (X

_t^0,Y⁰^,u

)

_t∈[0,T]

be the solution to the following SDE:

( dX

_t^0,Y⁰^,u

= b(X

_t^0,Y⁰^,u

, u

_t

) dt + σ(X

_t^0,Y⁰^,u

, u

_t

) dW

_t

, for a. e. t ∈ [0, T ], X

₀^0,Y⁰^,u

= Y

0

,

where the drift b, the volatility σ, and the random variable Y

0

are given. For all t ∈ [0, T ], we denote by m

^0,Y_t ⁰^,u

the probability distribution of X

_t^0,Y⁰^,u

. Along the article, constrained problems of the following form are considered:

u∈U

inf

₀(Y₀)

F (m

^0,Y_T ⁰^,u

), subject to: G(m

^0,Y_T ⁰^,u

) ≤ 0, (1) where the mappings F and G are given and satisfy differentiability assumptions.

The control set U

₀

(Y

₀

) is a set of adapted stochastic processes. In the article, n denotes the dimension of the state variable and N the number of constraints. A precise description of problem (1) will be given in Section 2. In this paper, we call the mappings F and G linear if they can be written in the form F (m) = R

Rⁿ

f (x) dm(x) and G(m) = R

Rⁿ

g(x) dm(x), where f : R

ⁿ

→ R and g : R

ⁿ

→ R

^N

. In this specific

2010Mathematics Subject Classification. Primary: 90C15, 93E20; Secondary: 49J53, 49K99.

Key words and phrases. stochastic optimal control, optimality conditions, augmented Lagran- gian method, mean-field-type control, relaxation.

This article is dedicated to Fr´ed´eric Bonnans in his 60th birthday.

1

(3)

case, problem (1) is equivalent to the following stochastic optimal control problem with an expectation constraint:

inf

u∈U0(Y₀)

E

f X

_T^0,Y⁰^,u

, subject to: E

g X

_T^0,Y⁰^,u

≤ 0. (2) The terminology non-linear used in the title refers to the fact that the functions F and G for which our result applies are not necessarily linear. In other words, the cost function and the constraints are not necessarily formulated as expectations of functions of X

_T^0,Y⁰^,u

. This is the specificity of the present article.

Stochastic optimal control problems with a non-linear cost function are mainly motivated by applications in economy and finance. In many situations, minimizing the expectation of a random cost may be unsatisfactory and one may prefer to take into account the “risk” induced by the dispersion of the cost. In the literature, there exist many different models of risk and some of them can be formulated as functions of the probability distribution of the state variable, as explained for example in [31, Chapter 6]. Some portfolio problems with risk-averse cost functions are studied in [26]. In [3, Section 5], a mean-variance portfolio selection problem is considered as well as in [22]. In [19], a gradient-based method is developed for minimization problems of the conditional value at risk, a popular risk-averse cost function. The risk can also be taken into account by considering a constraint of the form G(m

^0,Y_T ⁰^,u

) ≤ 0. For example, one can try to keep the probability of bankruptcy under a given threshold with a probability constraint. If G models the variance of the outcome of some industrial process, then a constraint on the variance can guarantee some uniformity in the outcome, which can be a desirable property.

Final-time constraints have several applications in finance, see for example [33], where probability constraints are used for solving a super-replication problem or [32], where the final probability distribution is fixed for solving a determination problem of no-arbitrage bounds of exotic options. Finally, let us mention that a problem of the form (1) can be seen as a simple model for an optimization problem of a multi-agent system, where a government, that is to say a centralized controller, influences a (very large) population of agents, whose behavior is described by a SDE (see for example [2] and the references therein on this topic). Let us mention however that for general multi-agent models, the coefficients of the SDE at time t depend on the current probability distribution m

^0,Y_t ⁰^,u

.

Let us describe the available results in the literature related to optimality conditions for problems similar to Problem (1). Problem (1), without constraints, is a specific case of an optimal control problem of a McKean-Vlasov process (or mean-field-type control problem). For this more general class of problems, the drift and volatility of the SDE possibly depend at any time t on the current distribution m

^0,Y_t ⁰^,u

. Optimality conditions usually take the form of a stochastic maximum principle: for a solution ¯ u, ¯ u

t

minimizes almost surely and for almost every time t a Hamiltonian involving a costate which is obtained as a solution to a backward stochastic differential equation (see for example [3], [4], [9], [13]). In a different but related approach, one can consider control processes in a feedback form, that is to say in the form u

_t

= u(t, X

_t^0,Y⁰^,u

), where the mapping u: [0, T ] × R

ⁿ

→ U has to be optimized. Under regularity assumptions, the probability distribution m

^0,Y_t ⁰^,u

has a density, say µ(t, x), which is the solution to the Fokker-Planck equation:

∂

_t

µ = 1 2 ∇

²

:

µ(t, x)a x, u(t, x)

− ∇ ·

µ(t, x)b x, u(t, x)

,

(4)

where a = σσ

^>

and where the operators ∇· and ∇

²

: are defined by

∇ · f (x) =

n

X

i=1

∂

_x_i

f

_i

(x) and ∇

²

: g(x) =

n

X

i,j=1

∂

_x²

ix_j

g

_ij

(x),

respectively. A derivation of the Fokker-Planck can be found in [12, Lemma 3.3], for example. Note that if b and σ do not depend on µ, then the Fokker-Planck equation is a linear partial differential equation. In this approach, the problem is an optimal control problem of the Fokker-Planck equation and optimality conditions take the form of a Pontryagin’s maximum principle. The adjoint equation is in this setting an HJB equation. We refer the reader to [1], [2], [4], and [15] for this approach.

In this article, optimal control problems of the following form:

u∈U

inf

₀(Y₀)

E

φ(X

_T^0,Y⁰^,u

)

(3) are called standard problems, considering the fact that they have been extensively studied in the last decades. They can be solved by dynamic programming, by computing the solution to the associated Hamilton-Jacobi-Bellman equation (see the textbooks [16], [28], [35] on this field). Since standard problems are of form (2) (without constraints), they fall into the general class of problems investigated in the paper. The optimality conditions provided in the present article can be shortly formulated as follows: if ¯ u is a solution to (1) and satisfies a qualification condition, then it is also the solution to a standard problem of the form (3), where the involved function φ is the derivative at m

^0,Y_T ⁰^,u

(in a specific sense) of the Lagrangian of the problem L := F + hλ, Gi, for some non-negative Lagrange multiplier λ satisfying a complementarity condition. Our optimality conditions therefore take the form of a variational inequality. Our analysis relies on the following technical result: the closure (for the Wasserstein distance associated with the L

¹

-distance) of the set of reachable probability distributions at time T is convex. This property is proved by constructing controls imitating the behaviour of relaxed controls. To the best of our knowledge, the optimality conditions for problem (1) are new, as well as the convexity property satisfied by the reachable set

¹

. They differ from the maximum principle mentioned above and are, to a certain extent, related to the optimality conditions obtained for mean-field-type control problems formulated with feedback laws. The presence of non-linear constraints is another novelty of the article; in the literature, most constraints of the form G(m

^0,Y_T ⁰^,u

) ≤ 0 are expectation constraints, see for example [8], [27]. The existence of a Lagrange multiplier, even in a linear setting, is not often considered, see [7, Section 5] or [23, Section 6]. Finally, we mention that we also prove the sufficiency of the optimality conditions in variational form, when F and G are convex.

It is well-known that problems of the form (1) are time-inconsistent, i.e. it is (in general) not possible to write a dynamic programming principle by parameterizing problem (1) by its initial time and initial condition, as is customary for standard problems of the form (3). A dynamic programming principle can be written if one considers the whole initial probability distribution as a state variable, see [18], [29], [30]. However, in practice, this approach does not allow, in general, to solve the problem, because the complexity of the method grows exponentially with the dimension of the (discretized) space of probability distributions. The optimality

1A proof of the convexity property as well as optimality conditions in variational form for unconstrained problems can be found in the unpublished research report [24].

(5)

conditions in variational form and the convexity property proved in this article naturally lead to iterative methods for solving problem (1), based on successive resolutions of standard problems and thus overcoming the difficulty related to time- inconsistency. We propose, analyse, and test such a method in the article. The cost function of the standard problem to be solved at each iteration is the derivative, in a certain sense, of an augmented Lagrangian.

We give a precise formulation of the problem under study in Section 2. We also discuss the notion of differentiability which is used. In Section 3, we prove the convexity of the closure of the reachable set of probability distributions. Optimality conditions in variational form are proved in Section 4. The case of convex problems is discussed. Our numerical method for solving the problem is described and analyzed in Section 5. We provide results for two academical examples. Elements on optimal transportation theory are given in the appendix.

2. Formulation of the problem and assumptions.

2.1. Notation.

• The set of probability measures on R

ⁿ

is denoted by P ( R

ⁿ

). For a function φ : R

ⁿ

→ R , its integral (if well-defined) with respect to the measure m ∈ P ( R

ⁿ

) is denoted by

Z

Rⁿ

φ(x) dm(x) or Z

Rⁿ

φ dm.

Given two measures m

₁

and m

₂

∈ P( R

ⁿ

), we denote:

Z

Rⁿ

φ(x) d(m

₂

(x) − m

₁

(x)) :=

Z

Rⁿ

φ(x) dm

₂

(x) − Z

Rⁿ

φ(x) dm

₁

(x).

• For a given random variable X with values in R

ⁿ

, we denote by σ σ σ(X ) the σ-algebra generated by X .

• For a given vector x ∈ R

^q

, we denote by |x| its Euclidean norm and by |x|

_∞

its supremum norm.

• For p ≥ 1, we denote by P

p

( R

ⁿ

) the set of probability measures having a finite p-th moment:

P

p

( R

ⁿ

) := n

m ∈ P( R

ⁿ

)

Z

Rⁿ

|x|

^p

dm(x) < +∞ o .

We recall that for 1 ≤ p ≤ q, the space P

q

( R

ⁿ

) is included into P

p

( R

ⁿ

).

We equip P

1

( R

ⁿ

) with the Wasserstein distance d

1

. The definition of the Wasserstein distance is recalled in the appendix. In the article, we only make use of the dual representation of d

1

[34, Remark 6.5]: for all m

1

, m

2

∈ P

1

( R

ⁿ

),

d

1

(m

1

, m

2

) = sup

φ∈1-Lip(Rⁿ)

Z

Rⁿ

φ d(m

2

− m

1

), (4) where 1-Lip( R

ⁿ

) :=

φ: R

ⁿ

→ R

|φ(y) − φ(x)| ≤ |y − x|, ∀(x, y) ∈ R

ⁿ

× R

ⁿ

is the set of real-valued Lipschitz continuous functions of modulus 1.

• For all R ≥ 0, we define B ¯

p

(R) := n

m ∈ P

p

( R

ⁿ

) | Z

Rⁿ

|x|

^p

dm(x) ≤ R o

. (5)

• The open (resp. closed) ball in R

ⁿ

of radius r ≥ 0 and center 0 is denoted by

B

_r

(resp. ¯ B

_r

), its complement B

_r^c

(resp. ¯ B

_r^c

), for the Euclidean norm.

(6)

• For a given p ≥ 1, we say that a function φ : R

ⁿ

→ R

^q

is dominated by |x|

^p

if for all ε > 0, there exists r > 0 such that for all x ∈ B

_r^c

,

|φ(x)| ≤ ε|x|

^p

. (6)

• The convex envelope of a set R is denoted conv(R). When R is a subset of P

1

( R

ⁿ

), its closure for the d

₁

-distance is denoted cl(R).

2.2. State equation. We fix a final time T > 0 and a Brownian motion (W

t

)

_t∈[0,T]

of dimension d. For all 0 ≤ t ≤ T, (W

s

− W

t

)

_s∈[t,T_]

is a standard Brownian motion.

For all s ∈ [t, T ], we denote by F

t,s

the σ-algebra generated by (W

θ

− W

t

)

_θ∈[t,s]

. Let U be a compact subset of R

^k

. Note that we do not make any other assumption on U : it can be non-convex, for example, or can be a discrete set. For a given random variable Y

t

independent of F

t,T

with values in R

ⁿ

, we define the set U

_t⁰

as the set of control processes (u

s

)

_s∈[t,T_]

taking values in U such that for all s ∈ [t, T ], u

s

is F

t,s

- measurable and we define the set U

t

(Y

t

) as the set of control processes (u

s

)

_s∈[t,T_]

taking values in U such that for all s ∈ [t, T ], u

s

is (σ σ σ(Y

t

) × F

t,s

)-measurable.

The drift b : R

ⁿ

× U → R

ⁿ

and the volatility σ : R

ⁿ

× U → R

^n×d

are given. For all u ∈ U

t

(Y

_t

), we denote by X

_s^t,Y^t^,u

s∈[t,T]

the solution to the SDE

dX

_s^t,Y^t^,u

= b(X

_s^t,Y^t^,u

, u

_s

) ds + σ(X

_s^t,Y^t^,u

, u

_s

) dW

_s

, ∀s ∈ [t, T ], X

_t^t,Y^t^,u

= Y

_t

. (7) The well-posedness of this SDE is ensured by Assumption A1 [21, Section 5] below.

We also denote by m

^t,Y_s ^t^,u

the probability distribution of X

_s^t,Y^t^,u

.

All along the article, we assume that the following assumption holds true. From now on, the initial condition Y

₀

and the real number p ≥ 2 introduced below are fixed.

Assumption A1. There exists K > 0 such that for all x, y ∈ R

ⁿ

, for all u, v ∈ U,

|b(x, u)| + |σ(x, u)| ≤ K(1 + |x|),

|b(x, u) − b(y, v)| + |σ(x, u) − σ(y, v)| ≤ K(|y − x| + |v − u|).

There exists p ≥ 2 such that E

|Y

0

|

^p

< ∞.

The following lemma is classical, see for example [17, Section 2.5].

Lemma 2.1. There exist three constants C

1

, C

2

, and C

3

> 0 depending on K, T, U , and p such that for all t ∈ [0, T ], for all random variables Y

_t

and Y ˜

_t

independent of F

t,T

and taking values in R

ⁿ

, for all u ∈ U

t

(Y

t

), for all 0 ≤ h ≤ T − t, for all t ≤ s ≤ T − h, the following estimates hold:

1. E

h sup

_t≤θ≤T

X

_θ^t,Y^t^,u

p

i

≤ C

₁

1 + E

|Y

t

|

^p

2. E

h

sup

_s≤θ≤s+h

|X

_θ^t,Y^t^,u

− X

_s^t,Y^t^,u

|

^p

i

≤ C

2

h

^p/2

1 + E

|Y

t

|

^p

3. E

h

sup

_t≤θ≤T

X

_θ^t,^Y^˜^t^,u

− X

_θ^t,Y^t^,u

p

i

≤ C

₃

E

| Y ˜

_t

− Y

_t

|

^p

.

Definition 2.2. We denote by R the set of reachable probability distributions at time T , given by

R = {m

^0,Y_T ⁰^,u

| u ∈ U

0

(Y

₀

)}. (8) By Lemma 2.1, there exists R > 0 such that

R ⊆ B ¯

_p

(R). (9)

(7)

By Lemma A.2 (in the appendix), ¯ B

p

(R) is compact for the d

1

-distance, thus it is bounded. It follows that R and cl(R) are bounded. We can therefore consider, for future reference, the diameter of cl(R), defined by

D := sup

m₁,m₂∈cl(R)

d

1

(m

1

, m

2

). (10) 2.3. Formulation of the problem and regularity assumptions. Consider two mappings F : P

p

( R

ⁿ

) → R and G: P

p

( R

ⁿ

) → R

^N

. We aim at studying the following problem:

inf

u∈U0(Y0)

F(m

^0,Y_T ⁰^,u

) subject to: G(m

^0,Y_T ⁰^,u

) ≤ 0. (P) Throughout the article, we assume that on top of Assumption A1, Assumptions A2 and A3 below, dealing with the continuity and the differentiability of F and G, are satisfied. The constant R used in these assumptions is given by (9).

Assumption A2. The restrictions of F and G to B ¯

_p

(R) are continuous for the d

₁

-distance.

In order to state optimality conditions, we need a notion of derivative for the mappings F and G. Denoting by M( R

ⁿ

) the set of finite signed measures on R

ⁿ

, we define:

M c

p

( R

ⁿ

) = n

m ∈ M( R

ⁿ

)

Z

Rⁿ

|x|

^p

d|m|(x) < +∞, Z

Rⁿ

1 dm(x) = 0 o .

Let A : M c

_p

( R

ⁿ

) → R

^q

be a linear mapping. We say that the function φ : R

ⁿ

→ R

^q

is a representative of A if for all m ∈ M c

p

( R

ⁿ

), the integral R

Rⁿ

φ(x) dm(x) is well- defined and equal to Am:

Am = Z

Rⁿ

φ(x) dm(x).

If φ is a representative of A, then for any constant c ∈ R

^q

, φ + c is also a representative of A, since

Z

Rⁿ

c dm(x) = 0, ∀m ∈ M c

p

( R

ⁿ

).

Conversely, if the value of a representative φ is fixed for a given point x

0

∈ R

ⁿ

, then for all x ∈ R

ⁿ

, the value of φ(x) is determined by

φ(x) = φ(x

0

) + A(δ

x

− δ

x₀

),

where δ

_x

and δ

_x₀

are the Dirac measures centered at x and x

₀

, respectively. The- refore, the representative, if it exists, is uniquely defined up to a constant.

Definition 2.3. Let f : [0, 1] × [0, 1] → R

^`

.

1. We say that f is Gˆ ateaux-differentiable at 0 if there exists Df (0) ∈ R

^`×2

such that

|f (z) − f (0) − Df(0)z| = o(|z|), as z → 0.

2. We say that f is strictly differentiable at 0 if there exists Df (0) ∈ R

^`×2

such that

|f (z

₂

) − f (z

₁

) − Df(0)(z

₂

− z

₁

)| = o(|z

2

− z

₁

|), as z

₁

→ 0 and z

₂

→ 0.

(8)

Assumption A3. 1. For all m, m

1

, and m

2

∈ B ¯

p

(R), there exists a linear form DF (m) : M c

p

( R

ⁿ

) → R such that the mapping

θ ∈ [0, 1]

²

7→ F ((1 − θ

1

− θ

2

)m + θ

1

m

1

+ θ

2

m

2

)

is Gˆ ateaux-differentiable at 0, with derivative (DF (m)(m

2

−m), DF (m)(m

1

− m)). Moreover, DF (m) possesses a continuous representative, dominated by

|x|

^p

, and denoted x ∈ R

ⁿ

7→ DF (m, x) ∈ R .

2. For all m, m

₁

, and m

₂

∈ B ¯

_p

(R), there exists a linear mapping DG(m) : M c

_p

( R

ⁿ

) → R

^N

such that the mapping

θ ∈ [0, 1]

²

7→ G((1 − θ

1

− θ

2

)m + θ

1

m

1

+ θ

2

m

2

)

is strictly differentiable at 0, with derivative (DG(m)(m

₂

− m), DG(m)(m

₁

− m)). Moreover, DG(m) possesses a continuous representative, dominated by

|x|

^p

, and denoted x ∈ R

ⁿ

7→ DG(m, x) ∈ R .

In the article, we make use of the derivative DF (m) (a linear form from M c

p

( R

ⁿ

) to R ) and its representative. The two notions can be distinguished according to the presence (or not) of the variable x.

2.4. Discussion of the notion of derivative. A general class of cost functions satisfying Assumptions A2 and A3 can be described as follows. Let K ∈ N , let Ψ : R

^K

→ R

^q

be differentiable, let φ: R

ⁿ

→ R

^K

be a continuous function dominated by |x|

^p

. We define then on P

_p

( R

ⁿ

):

H (m) = Ψ Z

Rⁿ

φ(x) dm(x)

. (11)

Note that for all control processes u ∈ U

₀

(Y

₀

), H m

^0,Y_T ⁰^,u

= Ψ E

φ X

_T^0,Y⁰^,u

. For all R ≥ 0, the continuity of H on ¯ B

_p

(R) follows from Lemma A.3 (in the appendix). One can easily check that the mapping H is differentiable in the sense of Assumption A3.1. The representative of its derivative is given by

DH(m, x) = DΨ Z

Rⁿ

φ(z) dm(z)

φ(x), (12)

up to a constant. Furthermore, if Ψ is continuously differentiable, then H is differentiable in the sense of Assumption A3.2. Note that the function φ does not need to be differentiable. Let us mention that the variance of a probability distribution m can be described with a function of the form (11), taking φ

1

(x) = x

²

, φ

2

(x) = x, and Ψ(z

₁

, z

₂

) = z

₁

− z

₂²

. Further examples are discussed in detail in [25, Section 4].

We finish this subsection with two remarks.

Remark 1. The fact that F should be defined on the whole space P

p

( R

ⁿ

) discards cost functions whose formulation is based on the density of the probability measure (since a density does not always exists, for probability distributions in P

p

( R

ⁿ

)). For example, the following problem does not fit in the proposed framework:

inf

u∈U₀(Y₀)

1 2 kf − f

_ref

k

L²(Rⁿ)

, subject to: f = PDF(X

_T^0,Y⁰^,u

),

where PDF stands for probability density function and where f

ref

is a given proba-

bility density function.

(9)

Remark 2. The notion of derivative provided in [12, Section 6] and the one introduced in Assumption A3 are of different nature, because they aim at evaluating the variation of functions from ¯ B

p

(R) to R on different kinds of paths. While our derivative is represented by a function from R

ⁿ

to R , the one of [12] is represented by a function from R

ⁿ

to R

ⁿ

(see [12, Theorem 6.5]). This difference of nature can be better understood by considering the mapping: m ∈ P

p

( R

ⁿ

) 7→ R

Rⁿ

φ dm.

This mapping is a monomial, according to the terminology given in [12, Exam- ple, page 43]. Its derivative (in the sense of [12]) is represented by the mapping: x ∈ R

ⁿ

7→ Dφ(x) ∈ R

ⁿ

(see [12, Example, page 44]). In the current framework, the derivative of m ∈ P

_p

( R

ⁿ

) 7→ R

Rⁿ

φ dm is the real-valued mapping x ∈ R

ⁿ

7→ φ(x) ∈ R , up to a constant.

2.5. Existence of a solution. Observe that problem (P) can be equivalently formulated as follows:

m∈R

inf F(m), subject to: G(m) ≤ 0, (P

⁰

) where R is defined by (8). Indeed, if ¯ u ∈ U

0

(Y

0

) is a solution to (P ), then ¯ m :=

m

^0,Y_T ⁰^,¯^u

is a solution to (P

⁰

) and conversely, if ¯ m ∈ R is a solution to (P

⁰

), then any ¯ u ∈ U

0

(Y

₀

) such that ¯ m = m

^0,Y_T ⁰^,¯^u

is a solution to (P ). The feasible set R

ad

of Problem (P

⁰

) is defined by

R

ad

= {m ∈ R | G(m) ≤ 0}. (13) By continuity of F for the d

1

-distance, we have

m∈R

inf

s.t.:G(m)≤0

F(m) = inf

m∈Rad

F (m) = inf

m∈cl(Rad)

F (m). (P

⁰⁰

) The notation (P

⁰⁰

) refers to the problem on the right. Lemma 2.1 enables us to prove the existence of a solution to problem (P

⁰⁰

).

Lemma 2.4. If R

ad

is non-empty, then problem (P

⁰⁰

) has a solution.

Proof. It is proved in Lemma A.2 that the set ¯ B

p

(R) is compact for the d

1

-distance.

By Lemma 2.1, R ⊆ B ¯

p

(R) and therefore, R

ad

⊆ B ¯

p

(R). Since ¯ B

p

(R) is closed, cl(R

ad

) ⊆ B ¯

p

(R) and therefore, cl(R

ad

) is compact, since it is a closed set of a compact set. The existence of a solution follows, since F is continuous.

Although we have an existence result for (P

⁰⁰

), it is in general difficult to prove the existence of a solution to (P ).

3. Convexity of the reachable set. This subsection is dedicated to the proof of the convexity of the closure of R (the set of reachable probability distributions at time T ). This result is an important tool for the proof of the optimality conditions in Section 4 and for the numerical method developed in Section 5. Let us explain the underlying purpose with a simple example. Consider two processes u

1

and u

2

∈ U

0

(Y

₀

) and the corresponding final probability distributions m

^0,Y_T ⁰^,u¹

and m

^0,Y_T ⁰^,u²

. We aim at building a control process u such that

m

^0,Y_T ⁰^,u

= 1

2 m

^0,Y_T ⁰^,u¹

+ m

^0,Y_T ⁰^,u²

. (14)

A very simple way of building such a control process is to define a random variable S independent of Y

0

and F

0,T

taking two different values with probability 1/2.

A control u realizing (14) can then be constructed in U

0

(S, Y

₀

)

: it suffices that

(10)

u = u

1

for one value of S and that u = u

2

for the other. The obtained controlled process can be seen as a relaxed control, since it is now measurable with respect to a larger filtration. The main idea of Lemma 3.1 is to construct control processes in U

0

(Y

0

) imitating the behaviour of the relaxed control process u.

Lemma 3.1. The closure of the set of reachable probability measures for the d

1

- distance, denoted cl(R), is convex.

Proof. Let us set R e =

₁

2

m

1

+

¹₂

m

2

| m

1

∈ R, m

2

∈ R . Our approach mainly consists in proving that

R ⊆ e cl(R). (15)

Let u

¹

and u

²

in U

0

(Y

0

). To prove (15), it suffices to prove that there exists a sequence (u

^ε

)

_ε≥0

in U

0

(Y

0

) such that

d

1

m

^0,Y_T ⁰^,u^ε

, 1 2

2

X

k=1

m

^0,Y_T ⁰^,u^k

−→

ε→0

0. (16) Let 0 < ε < T , let ˜ u

¹

and ˜ u

²

be two processes in U

ε

(Y

0

) such that for all k = 1 and k = 2, the processes (u

^k_s

)

_{s∈[0,T−ε]}

and (˜ u

^k_s

)

_s∈[ε,T_]

can be seen as the same measurable function of respectively

(Y

0

, (W

s

− W

0

)

_{s∈[0,T−ε]}

) and (Y

0

, (W

s

− W

ε

)

_s∈[ε,T]

).

In other words, we simply delay the observation of the variation of the Brownian motion of a time ε. Let us denote by A

₁

the event (W

_ε¹

−W

₀¹

) ∈ (−∞, 0) and by A

₂

the event (W

_ε¹

− W

₀¹

) ∈ (0, ∞), where W

¹

is the first coordinate of the Brownian motion. Of course, P [A

₁

] = P [A

₂

] =

¹₂

. Fixing u

⁰

∈ U , we define u

^ε

∈ U

₀

(Y

₀

) as follows:

u

^ε_t

= u

⁰

, for a. e. t ∈ (0, ε),

u

^ε_t

= ˜ u

^k_t

, for a. e. t ∈ (ε, T ), when A

_k

is realized.

For all φ ∈ 1-Lip( R

ⁿ

),

E

φ X

_T^0,Y⁰^,u^ε

= 1 2

2

X

k=1

E h

φ X

^ε,X

0,Y0,uε ε ,u^ε T

A

k

i

= 1 2

2

X

k=1

E h φ

X

^ε,X^0,Y⁰^,u

0 ε ,˜u^k T

A

_k

i

= 1 2

2

X

k=1

h a

_k

+ b

_k

+ E

φ X

_T^0,Y⁰^,u^k

i , (17)

where a

_k

and b

_k

are given by a

k

= E

h φ

X

^ε,X^0,Y⁰^,u

0 ε ,˜u_k T

− φ

X

_T^ε,Y⁰^,˜^u^k

A

k

i , b

k

= E

φ X

_T^ε,Y⁰^,˜^u^k

|A

k

− E

φ X

_T^0,Y⁰^,u^k

.

Let us first estimate a

k

. Using the Lipschitz-continuity of φ, we obtain that 1

2 |a

_k

| ≤ P A

_k

E h

X

^ε,X^0,Y⁰^,u

0 ε ,˜u_k

T

− X

_T^ε,Y⁰^,˜^u^k

A

_k

i

≤ E h

X

^ε,X^0,Y⁰^,u

0 ε ,˜u_k

T

− X

_T^ε,Y⁰^,˜^u^k

i

.

(11)

We deduce from the Cauchy-Schwarz inequality and Lemma 2.1 that 1

2 |a

k

| ≤ E h

X

^ε,X^0,Y⁰^,u

0 ε ,˜u_k

T

− X

_T^ε,Y⁰^,˜^u^k

2

i

^1/2

≤ p C

3

E

|X

_ε^0,Y⁰^,u⁰

− Y

0

|

²

1/2

≤ p

C

2

C

3

ε 1 + E

|Y

0

|

²

1/2

. (18)

Let us estimate b

_k

. Since ˜ u

_k

and Y

₀

are independent of A

_k

and using the definition of ˜ u

_k

, we obtain that

E

φ(X

_T^ε,Y⁰^,˜^u^k

) | A

k

= E

φ(X

_T^ε,Y⁰^,˜^u^k

)

= E

φ(X

_T^0,Y_−ε⁰^,u^k

) . Therefore,

b

k

= E

φ X

_T^0,Y_−ε⁰^,u^k

− φ X

_T^0,Y⁰^,u^k

.

We obtain with the Lipschitz-continuity of φ, the Cauchy-Schwarz inequality, and Lemma 2.1 that

|b

k

| ≤ E

X

_T^0,Y⁰^,u^k

− X

_T^0,Y_−ε⁰^,u^k

≤ E

X

_T^0,Y⁰^,u^k

− X

_T^0,Y_−ε⁰^,u^k

2

^1/2

≤ p

C

2

ε 1 + E

|Y

0

|

²

^1/2

. (19)

Combining (17), (18), and (19), we obtain that

E

φ X

_T^0,Y⁰^,u^ε

− 1 2

2

X

k=1

E

φ X

_T^0,Y⁰^,u^k

≤ 1

2

X

k=1

|a

k

| + |b

k

|

≤ M √ ε,

where M is a constant independent of φ and ε. Using the dual representation of d

1

(given by (4)), we deduce that d

₁

m

^0,Y_T ⁰^,u^ε

, 1 2

2

X

k=1

m

^0,Y_T ⁰^,u^k

≤ sup

φ∈1-Lip(Rⁿ)

n E

φ X

_T^0,Y⁰^,u^ε

− 1 2

2

X

k=1

E

φ X

_T^0,Y⁰^,u^k

o

= M √ ε.

This proves (16) and thus justifies (15). We can now conclude the proof. It follows from (15) that

cl( R) e ⊆ cl(R). (20)

Since R ⊆ R, we have cl(R) e ⊆ cl( R), and therefore by (20), cl(R) = cl( e R). It e remains to prove that cl( R) is convex, which is an easy task. e

On the feasibility of the problem. Proving the feasibility (or infeasibility) of the problem is in general a difficult task. Still, Lemma 3.1 can be used to prove the existence of m ∈ cl(R) such that G(m) < 0 (this ensures the feasibility of the problem), or can be used to prove that for all m ∈ cl(R), G(m) > 0 (in this case the problem is infeasible).

Given controls u

₁

, u

₂

,...,u

_K

∈ U

0

(Y

₀

), we obtain with Lemma 3.1 that the set conv({m

^0,Y_T ⁰^,u^k

}

k=1,...,K

) is included into cl(R). Thus, if there exists λ ∈ R

^K

such that λ ≥ 0, P

K

k=1

λ

_k

= 1, and G P

K

k=1

m

^0,Y_T ⁰^,u^k

< 0, then the problem is feasible.

(12)

Let us consider now the situation where G is of the form G(m) = Ψ( R

φ(x) dm(x)) with φ : R

ⁿ

→ R

^K

and Ψ : R

^K

→ R

^N

, as proposed in subsection 2.4. Let us set

Z = R

φ(x) dm(x) | m ∈ cl(R) ⊆ R

^K

.

If there exists z ∈ Z such that G(z) < 0, then the problem is feasible. If for all z ∈ Z , G(z) > 0, then the problem is infeasible. Thus, if we can describe in a convenient way the set Z, then the issue of feasibility is reduced to a feasibility problem of finite dimension and is likely to be easier.

Regarding the description of Z, a dual approach can be considered. The set Z is indeed convex, by Lemma 3.1. It is moreover compact, since cl(R) is compact and the mapping R

φ dm(x) continuous for the d

1

-distance. Given λ ∈ R

^K

, we define Z(λ) = inf

_z∈Z

hλ, zi. We have

Z(λ) = inf

m∈R

Z

hλ, φ(x)i dm(x) = inf

u∈U₀(Y₀)

E

hλ, φ(X

_T^0,Y⁰^,u

)i ,

thus the quantity Z(λ) can be computed numerically by solving a standard problem.

By the Hahn-Banach theorem, we have the following relation:

Z = {z ∈ R

^K

| hλ, zi ≥ Z (λ), ∀λ such that |λ| = 1}.

Thus an outer approximation of Z can be computed by replacing the sphere of radius 1 (in R

^K

) by a discrete subset, in the right-hand side of the above relation.

In the situation of expectation constraints, that is G(m) = Ψ( R

φ(x) dm(x)) with Ψ(z) = z, the following problem can be considered:

sup

λ∈R^K

Z(λ), subject to: |λ| = 1, λ ≥ 0. (21) If the value of problem (21) is negative, then Problem P is feasible, if the value of Problem (21) is positive, then Problem (P ) is infeasible. For proving this result, one has to prove that (21) is the dual of: inf

z∈Z

sup

_λ∈_RK

hλ, zi, subject to |λ| = 1, λ ≥ 0, and one has to observe that for all z ∈ R

^K

, z < 0 if and only if the following problem has a negative value: sup

_λ∈_RK

hλ, zi subject to |λ| = 1, λ ≥ 0.

4. Optimality conditions. We prove in this section the main result: if a control

¯

u is a solution to (P) and satisfies a qualification condition, then it is the solution to a standard problem of the form (3). Before proving our result, we recall in Subsection 4.1 some well-known properties of the value function associated with a standard problem.

4.1. Standard problems. Let φ : R

ⁿ

→ R be a continuous function dominated by |x|

^p

. Let us define:

Φ : m ∈ P

_p

( R

ⁿ

) 7→

Z

Rⁿ

φ(x) dm(x).

The mapping Φ is linear, in so far as for all m

1

and m

2

∈ P

p

( R

ⁿ

), for all θ ∈ [0, 1], Φ(θm

₁

+ (1 − θ)m

₂

) = θΦ(m

₁

) + (1 − θ)Φ(m

₂

).

It is also continuous for the d

₁

-distance, see Lemma A.3 (in the appendix). We denote by (P (φ)) the following standard problem:

inf

u∈U0(Y₀)

E

φ(X

_T^0,Y⁰^,u

)

. (P (φ))

(13)

Let ˆ u ∈ U

0

(Y

0

) and let ˆ m = m

^0,Y_T ⁰^,ˆ^u

. By continuity of Φ, the control process ˆ u is a solution to (P(φ)) if and only if

Φ( ˆ m) = inf

m∈cl(R)

Φ(m).

We recall, for future reference, some well-known results concerning the value function associated with the standard problem (P (φ)). We refer to the textbooks [16], [28], [35] on this topic. The value function V associated with (P (φ)) is defined for all t ∈ [0, T ] and for all x ∈ R

ⁿ

by

V (t, x) = inf

u∈U_t⁰

E

φ(X

_T^t,x,u

) .

It can be characterized as the unique viscosity solution to the following Hamilton- Jacobi-Bellman (HJB) equation:

− ∂

t

V (t, x) = inf

u∈U

n

H (x, u, ∂

x

V (t, x), ∂

_xx²

V (t, x)) o

, V (T, x) = φ(x), (22) where the Hamiltonian H is defined for x ∈ R

ⁿ

, u ∈ U, p ∈ R

ⁿ

, and Q ∈ R

^n×n

by

H(x, u, p, Q) = hp, b(x, u)i + 1

2 tr(σ(x, u)σ(x, u)

^>

Q).

If V is sufficiently smooth, one can prove with a verification argument that any control process u is a global solution to (P (φ)) if almost surely,

u

t

∈ arg min

v∈U

n

H(X

_t^0,Y⁰^,u

, v, ∂

x

V (t, X

_t^0,Y⁰^,u

), ∂

²_xx

V (t, X

_t^0,Y⁰^,u

)) o

, for a.e. t ∈ (0, T ).

Finally, note that the value of problem (P(φ)) is given by inf

u∈U0(Y₀)

E

φ(X

_T^0,Y⁰^,u

)

= Z

Rⁿ

V (0, x) dm(x), where m denotes the probability distribution of Y

₀

.

4.2. Main result. In this subsection, we give first-order optimality conditions in variational form for problem (P) (defined in the introduction, page 6). Along this subsection, a solution ¯ u ∈ U

0

(Y

0

) to problem (P) is fixed. We also set

¯

m = m

^0,Y_T ⁰^,¯^u

.

We first give a metric regularity result (Theorem 4.1), which is a key tool for the proof of the optimality conditions (Theorem 4.3).

Let us consider the sets A and I of active and inactive constraints at ¯ m, defined by

A = {j = 1, ..., N | G

j

( ¯ m) = 0} and I = {j = 1, ..., N | G

j

( ¯ m) < 0}.

Let N

A

be the cardinality of A. We define

G

A

(m) = (G

j

(m))

_j∈A

∈ R

^N^A

and G

I

(m) = (G

j

(m))

_j∈I

∈ R

^N^−N^A

. We have

R

ad

=

m ∈ R | G

A

(m) ≤ 0, G

I

(m) ≤ 0 , G

A

( ¯ m) = 0, and G

I

( ¯ m) < 0.

The following assumption is a qualification condition.

Assumption A4. There exists m

₀

∈ cl(R) such that DG

_A

( ¯ m)(m

₀

− m) ¯ < 0.

For all z ∈ R

^N^A

, we denote by z

+

the vector defined by (z

+

)

j

= max(z

j

, 0) for

j ∈ A.

(14)

Theorem 4.1. If Assumption A4 holds, then for all m ∈ cl(R), there exist two constants θ ¯ ∈ (0, 1] and C > 0 such that for all θ ∈ [0, θ], for all ¯ ε > 0, there exists η ∈ [0, 1] such that

η ≤ C

G

_A

(m

_θ

)

₊

∞

+ ε (23) and such that G (1 − η)m

_θ

+ ηm

₀

< 0, where m

_θ

= (1 − θ) ¯ m + θm and where m

₀

is given by Assumption A4.

The estimate (23) is basically an estimate of the distance of m

θ

to cl(R

ad

).

Indeed, by the convexity of cl(R), the probability measure (1 − η)m

θ

+ ηm

0

lies in cl(R). Since G is continuous and since G (1 − η)m

θ

+ ηm

0

< 0, the probability measure (1− η)m

_θ

+ηm

₀

lies in cl(R

ad

). It is at a distance ηd

₁

(m

_θ

, m

₀

) of m

_θ

. The real number η ≥ 0 is of same order as the quantity |G

A

(m

_θ

)

₊

|

∞

, which indicates how much the constraints are violated.

Proof of Theorem 4.1. For all θ ∈ [0, 1], we define

G

θ

: η ∈ [0, 1] 7→ G

θ

(η) = G

A

(1 − η)m

θ

+ ηm

0

∈ R

^N^A

. By Assumption A4, DG

_A

( ¯ m)(m

₀

− m) ¯ < 0. Let α > 0 be such that

DG

_A

( ¯ m)(m

₀

− m) ¯ ≤ −α1, (24) where 1 = (1, ..., 1) ∈ R

^N^A

. The above inequality (as well as all those involving vectors) must be understood coordinatewise.

Claim 1. There exist ¯ η ∈ (0, 1] and ¯ θ ∈ (0, 1] such that for all η ∈ [0, η] and for ¯ all θ ∈ [0, θ], ¯

G

θ

(η) − G

θ

(0) ≤ − αη

2 1. (25)

Let us prove this claim. The reader can check that as a consequence of the differentiability assumption A3, the mapping (θ, η) ∈ [0, 1]

²

7→ G

θ

(η) is strictly differentiable at 0 (in the sense of Definition 2.3). Moreover, the partial derivative with respect to η is given by

DG

0

(0) = DG

A

( ¯ m)(m

0

− m). ¯

As a consequence, there exist ¯ θ and ¯ η ∈ [0, 1] such that for all θ ∈ [0, θ] and for all ¯ η ∈ [0, η], ¯

|G

θ

(η) − G

θ

(0) − DG

0

(0)η| ≤ α 2 η.

Combining the above inequality with (24), we obtain that G

θ

(η) − G

θ

(0) ≤ DG

0

(0)η + α

2 η1 ≤ − α 2 1, which concludes the proof of the claim.

Now, let η

0

∈ (0, 1] and θ

0

∈ (0, 1] be sufficiently small, so that for all η ∈ [0, η

0

] and for all θ ∈ [0, θ

0

],

G

I

(1 − η)m

θ

+ ηm

0

< 0.

Let us set

γ = α

4 min(¯ η, η

0

). (26)

Recall that G

0

(0) = G

A

( ¯ m) = 0. Once again, we reduce the value of ¯ θ, if necessary, so that ¯ θ ≤ θ

0

and so that for all θ ∈ [0, θ], ¯

|G

θ

(0)|

∞

≤ γ. (27)

(15)

Claim 2. There exists C > 0 such that for all y ∈ R

^N^A

with |y|

_∞

≤ γ, for all θ ∈ [0, θ], there exists ¯ η ∈ [0, η] such that ¯

G

θ

(η) + y ≤ 0 and η ≤ C

G

θ

(0) + y

+

∞

.

Let us prove the claim. We set C =

_α²

. For a given θ ∈ [0, θ] and for a given ¯ y such that |y|

_∞

≤ γ, we set η = C|(G

θ

(0) + y)

+

|

_∞

. Using (26), (27), and the definition of C, we obtain that

|(G

_θ

(0) + y)

₊

|

_∞

≤ |G

_θ

(0) + y|

_∞

≤ |G

_θ

(0)|

_∞

+ |y|

_∞

≤ 2γ ≤ α¯ η 2 = η ¯

C . Therefore, η = C|(G

θ

(0) + y)

+

|

∞

≤ η. Using the first claim, we obtain that ¯

G

θ

(η) + y ≤ G

θ

(0) − αη

2 1 + y. (28)

It directly follows from the definitions of C and η that

− αη

2 1 = − η

C 1 ≤ −|(G

θ

(0) + y)

+

|

_∞

1 ≤ −(G

θ

(0) + y)

+

. Combined with (28), we obtain that

G

θ

(η) + y ≤ (G

θ

(0) + y) − (G

θ

(0) + y)

+

≤ 0.

This proves the second claim.

Conclusion. We can finally prove the theorem. Let θ ∈ [0, θ] and let ¯ ε > 0. We set

y = min γ, ε

C

1. Since |y|

_∞

≤ γ, there exists η ∈ [0, η] such that ¯ G

θ

(η) + y ≤ 0 and such that η ≤ C|(G

θ

(0) + y)

+

|

_∞

, by the second claim. Using the inequality

|b

+

|

_∞

− |a

+

|

_∞

≤

|b − a|

_∞

, which is easy to establish, we obtain that

η ≤ C|(G

_θ

(0) + y)

₊

|

_∞

≤ C |G

_θ

(0)

₊

|

_∞

+ |y|

_∞

. (29)

Since G

θ

(0) = G

A

(m

θ

) and |y|

∞

≤ ε/C , we obtain that η ≤ C|G

A

(m

θ

)

+

|

_∞

+ ε.

The estimate (23) is therefore satisfied. Since |G

_θ

(0)

₊

|

_∞

≤ γ and |y|

_∞

≤ γ, we deduce from (26) and from (29) that

η ≤ 2Cγ = 4γ

α ≤ η

0

. (30)

Let us set ˆ m = (1 − η)m + ηm

0

. By construction, G

θ

(η) + y = G

A

( ˆ m) + y ≤ 0 and thus, G

A

( ˆ m) < 0, since y > 0. Moreover, G

I

( ˆ m) < 0, since θ ∈ [0, θ

0

] and η ∈ [0, η

0

]. Therefore, G( ˆ m) < 0. The theorem is proved.

In the following theorem, we prove first-order optimality conditions in variational form for problem (P ).

Definition 4.2. The Lagrangian L is given by

L : (m, λ) ∈ B ¯

_p

(R) × R

^N

7→ F (m) + hλ, G(m)i.

The Lagrangian is differentiable (with respect to m) in the sense of Assumption A3.1 with

DL(m, λ) ˜ m = DF (m) ˜ m + hλ, DG(m) ˜ mi, ∀ m ˜ ∈ M c

p

( R

ⁿ

).

A representative of ˜ m 7→ DL(m, λ) ˜ m is given by

DL(m, λ, x) = DF (m, x) + hλ, DG(m, x)i,

(16)

up to a constant. Note that the mapping ˜ m ∈ B ¯

p

(R) 7→ DL(m, λ) ˜ m is continuous, by Lemma A.3.

In the sequel, we say that a non-negative Lagrange multiplier λ ∈ R

^N

satisfies the complementarity condition at ¯ m if

G

i

( ¯ m) < 0 = ⇒ λ

i

= 0, for all i = 1, ..., N . (31) Theorem 4.3. Assume that u ¯ is a solution to problem (P). Let m ¯ = m

^0,Y_T ⁰^,¯^u

. If Assumption A4 holds, then there exists a non-negative Lagrange multiplier λ ∈ R

^N

satisfying the complementarity condition (31) at m ¯ which is such that u ¯ is a solution to the standard problem (P(φ)) with

φ(x) = DL( ¯ m, λ, x).

Remark 3. We say then that the control process ¯ u satisfies the optimality conditions in variational form. The optimality of ¯ u for the standard problem with φ(x) = DL( ¯ m, x) is equivalent to the following variational inequality:

E

DL( ¯ m, λ, X

_T^0,Y⁰^,u

)] ≥ E

DL( ¯ m, λ, X

_T^0,Y⁰^,¯^u

)

, ∀u ∈ U

0

(Y

0

) and also equivalent to the following inequality:

DL( ¯ m, λ)(m − m) ¯ ≥ 0, ∀m ∈ cl(R). (32) In the sequel, we say that a probability measure ¯ m ∈ cl(R) such that G( ¯ m) ≤ 0 satisfies the optimality conditions in variational form if there exists a multiplier λ ≥ 0 satisfying the complementarity condition (31) and such that (32) holds. We mention that for a solution ¯ m to (P

⁰⁰

), not necessarily lying in R

ad

, one can extend the proof given below and demonstrate that the optimality conditions in variational form are also satisfied.

Proof of Theorem 4.3. In view of the complementarity condition, it suffices to prove the existence of λ

A

∈ R

^N^A

such that λ

A

≥ 0 and such that

inf

m∈cl(R)

D L( ¯ ˜ m, λ

A

)(m − m) = 0, ¯ where ˜ L(m, λ

A

) = F (m) + hλ

A

, G

A

(m)i. For all m ∈ cl(R),

D L( ¯ ˜ m, λ

_A

)(m − m) = ¯ DF ( ¯ m)(m − m) + ¯ hλ

A

, DG

_A

( ¯ m)(m − m)i. ¯

For all y ∈ R

^N^A

, we consider the following optimization problem, denoted (LP (y)):

V (y) = inf

m∈cl(R)

DF ( ¯ m)(m − m), ¯ subject to: DG

A

( ¯ m)(m − m) + ¯ y ≤ 0. (LP (y)) Step 1. We first prove that V (0) = 0. For y = 0, ¯ m is feasible (for problem (LP (y)) with y = 0), thus V (0) ≤ DF ( ¯ m)( ¯ m − m) = 0. Now, let ¯ m ∈ cl(R) be such that DG

_A

( ¯ m)(m − m) ¯ ≤ 0. Let ¯ θ > 0 and C > 0 be given by Theorem 4.1.

Let (θ

_k

)

_k∈_N

be a convergent sequence with limit 0 taking values in (0, θ]. For all ¯ k, we set

m

_k

= (1 − θ

_k

) ¯ m + θ

_k

m.

By Assumption A3, we have

G

A

(m

k

) = G

A

( ¯ m) + θ

k

DG

A

( ¯ m)(m − m) + ¯ o(θ

k

).

Since G

_A

( ¯ m) = 0 and DG

_A

( ¯ m)(m − m) ¯ ≤ 0, we have

|G

A

(m

_k

)

₊

|

∞

= o(θ

_k

).

(17)

By Theorem 4.1, there exists for all k ∈ N a real number η

k

∈ [0, 1] such that η

k

≤ C|G

A

(m

k

)

+

|

∞

+ θ

²_k

= o(θ

k

)

and such that G( ˆ m

_k

) < 0, where ˆ

m

k

= (1 − η

k

)m

k

+ η

k

m

0

= (1 − η

_k

)(1 − θ

_k

) ¯ m + (1 − η

_k

)θ

_k

m + η

_k

m

₀

. Thus,

ˆ

m

_k

− m ¯ = (1 − η

_k

)θ

_k

(m − m) + ¯ η

_k

(m

₀

− m). ¯ (33) Since cl(R) is convex (Lemma 3.1), ˆ m

k

∈ cl(R). Therefore, for all k, by continuity of F and G, there exists ˜ m

k

∈ R such that G( ˜ m

k

) ≤ 0 and such that F ( ˜ m

k

)−F ( ˆ m

k

) ≤ θ

²_k

. Using the differentiability assumption on F (Assumption A3), the feasibility of

˜

m

k

, the fact that η

k

= o(θ

k

) and (33), we obtain that 0 ≤ F( ˜ m

k

) − F( ¯ m)

θ

_k

≤ F( ˆ m

k

) − F( ¯ m) θ

k

+ F ( ˜ m

k

) − F ( ˆ m

k

) θ

k

= (1 − η

k

)DF ( ¯ m)(m − m) + ¯ η

k

θ

k

DF ( ¯ m)(m

0

− m) + ¯ o(1)

= DF ( ¯ m)(m − m) + ¯ o(1).

It follows that DF ( ¯ m)(m − m) ¯ ≥ 0 and finally proves that V (0) = 0.

Step 2. We compute now the Legendre-Fenchel transform (see [6, Relation 2.210]

for a definition) of V . For all λ

A

∈ R

^N^A

, we have V

^∗

(λ

A

) = sup

y∈R^NA

hλ

A

, yi − V (y)

= sup

y∈R^NA

hλ

A

, yi − inf

m∈cl(R)

DF ( ¯ m)(m − m), ¯ s.t.: DG

_A

( ¯ m)(m − m) + ¯ y ≤ 0

= sup

y∈R^NA m∈cl(R)

hλ

_A

, yi − DF ( ¯ m)(m − m), ¯ s.t.: DG

_A

( ¯ m)(m − m) + ¯ y ≤ 0 .

Using the change of variable z = DG

_A

( ¯ m)(m − m) + ¯ y, we obtain:

V

^∗

(λ

_A

) = sup

z∈R^NA m∈cl(R)

hλ

A

, z − DG

_A

( ¯ m)(m − m)i − ¯ DF ( ¯ m)(m − m), ¯ s.t.: z ≤ 0

= sup

m∈cl(R)

sup

z∈R^NA

hλ

A

, zi, s.t.: z ≤ 0

− D L( ¯ ˜ m, λ

A

)(m − m) ¯ .

Observing that

sup

z∈R^NA

hλ

A

, zi, subject to: z ≤ 0

=

( 0 if λ

A

≥ 0 +∞ otherwise, we deduce that

V

^∗

(λ

A

) =

( − inf

_m∈cl(R)

D L( ¯ ˜ m, λ

A

)(m − m) ¯ if λ

A

≥ 0

+∞ otherwise. (34)

(18)

Step 3. Using the convexity of cl(R) (Lemma 3.1), one can easily show that V is a convex function. Let α > 0 be such that (24) holds. Then, for any y ∈ R

^N^A

with |y|

∞

≤ α,

DG

A

( ¯ m)(m

0

− m) + ¯ y ≤ 0

and therefore, problem (LP (y)) is feasible and V (y) ≤ DF ( ¯ m)(m

₀

− m). It follows ¯ from [6, Proposition 2.018, Proposition 2.126] that V is continuous in the neig- hbourhood of 0 and has a non-empty subdifferential ∂V (0) at 0. Let λ

A

∈ ∂V (0), by [6, Relation 2.232], we have

V

^∗

(λ

A

) = V (0) + V

^∗

(λ) = h0, λi = 0.

Thus, by (34), λ

A

≥ 0 and

m∈cl(R)

inf D L( ¯ ˜ m, λ

A

)(m − m) = 0. ¯ The theorem is proved.

The approach which has been employed to prove Theorem 4.3 is similar to the one used in [5] for proving Pontryagin’s principle in a deterministic framework. In this reference, an auxiliary convex optimization problem (denoted by (P L), introduced at the end of Section 3, and involving the final-state variable) is introduced.

Pontryagin’s principle is obtained by analyzing the dual of that problem, which plays a similar role as Problem (LP (y)) in the proof of Theorem 4.3.

The following lemma shows that the value of the standard problem can be used to estimate the loss of optimality of a given probability measure ˆ m in R

_ad

(defined by (13)), when the mappings F , G

₁

,...,G

_N

are convex. The calculations involved in the proof are rather standard, see for example [20, Theorem 12.14].

Lemma 4.4. Denote by Val(P) the value of Problem (P ). Assume that the mappings F , G

₁

,...,G

_N

are convex. Then, for all m ˆ ∈ R

_ad

, for all non-negative λ ∈ R

^N

such that the complementarity condition holds at m, the following upper estimate ˆ holds:

F( ˆ m) − Val(P ) ≤ − inf

m∈cl(R)

DL( ˆ m, λ)(m − m). ˆ (35) Proof. Let m ∈ R

ad

. Since F is convex, we have

F(m) − F ( ˆ m) ≥ DF ( ˆ m)(m − m). ˆ (36) Denoting by A the active set at ˆ m and setting

G

_A

(m) = (G

_j

(m))

_j∈A

and λ

_A

= (λ

_j

)

_j∈A

,

we obtain, using the feasibility of m and the convexity of G

1

,...,G

m

that 0 ≥ G

_A

(m) = G

_A

(m) − G

_A

( ˆ m) ≥ DG

_A

( ˆ m)(m − m). ˆ Since λ

A

≥ 0, we deduce that

0 ≥ hλ

A

, DG

_A

( ˆ m)(m − m)i. ˆ (37) By the complementarity condition,

hλ

A

, DG

_A

( ˆ m)(m − m)i ˆ = hλ, DG( ˆ m)(m − m)i. ˆ (38) Adding (36), (37), and (38) together, we obtain that

F (m) − F( ˆ m) ≥ DF ( ˆ m)(m − m) + ˆ hλ, DG( ˆ m)(m − m)i ˆ = DL( ˆ m, λ)(m − m). ˆ