Interior penalty approximation for optimal control problems. Optimality conditions in stochastic optimal control theory.

HAL Id: pastel-00542295

https://pastel.archives-ouvertes.fr/pastel-00542295

Submitted on 2 Dec 2010

HAL is a multi-disciplinary open access

archive for the deposit and dissemination of

sci-entific research documents, whether they are

pub-lished or not. The documents may come from

teaching and research institutions in France or

abroad, or from public or private research centers.

L’archive ouverte pluridisciplinaire HAL, est

destinée au dépôt et à la diffusion de documents

scientifiques de niveau recherche, publiés ou non,

émanant des établissements d’enseignement et de

recherche français ou étrangers, des laboratoires

publics ou privés.

Interior penalty approximation for optimal control

problems. Optimality conditions in stochastic optimal

control theory.

Francisco Silva

To cite this version:

Francisco Silva. Interior penalty approximation for optimal control problems. Optimality conditions

in stochastic optimal control theory.. Optimization and Control [math.OC]. Ecole Polytechnique X,

2010. English. �pastel-00542295�

présentée par FranciscoJ. Silva

pour obtenir legrade de Docteur de l'Ecole Polytechnique

Spécialité: Mathématiques Appliquées

Interior penalty approximation for optimal control problems.

Optimality conditions in stochastic optimal control theory.

Thèse présentée le

29

novembre 2010 devant lejury composé de :

FrédéricBonnans Directeur de thèse Jean-Pierre Raymond Rapporteur

Agnès Sulem Examinatrice Nizar Touzi Examinateur Stefan Ulbrich Rapporteur Jiongmin Yong Rapporteur

Remerciements

Je tiens tout d'abord à remercier mon directeur de thèse, Frédéric Bonnans pour m'avoir accueilli dans son equipe COMMANDS de L'INRIA Saclay. Je lui en suis très reconnaissant pour la grande qualité de son encadrement durant toutes ces années. Il a toujours eté disponible pour répondre à mes questionsetilm'atoujoursencouragéàassisteràdenombreuses conférences, écoles etséminaires. Je luiexprime mon respect le plus profond.

J'exprime également toute ma reconnaissance à Jérôme Bolte, avec qui j'ai eu l'opportunité de travailleraudébut de ma thèse. Je lui remercie son soutienconstanttoutaulong demontravailetj'avouequetravailleravec lui m'apermid'enrichirconsidérablementmesconnaissancesenmathématiques. Je remercie aussi très vivement Felipe Alvarez, quim'a encouragéà pos-tuler àce projet. Lors des mes années de thèse j'ai eu la chancede fairedes séjours dans mon cher pays le Chili. Je tiens à remercier Felipe Álvarez et Alejandro Jofré pour leur agréableaccueil àl'Université du Chili.

Je souhaite exprimer toute ma gratitude à Jean-Pierre Raymond, Ste-fan Ulbrich et Jiongmin Yong pour avoir accepté la tâche de rapporteur. J'adresse un grandmerci pour leur lecture très soigneuse de ce manuscrit et leur remarques très intéressant.

JesuistrèsreconnaissantàAgnèsSulemetNizarTouzid'avoiracceptéde faire partie de mon jury. Leur travaux en commande optimalestochastique sont des références incontournables.

Mes remerciement vont également à tous le membres du CMAP. J'aipu bénécierd'uneambiancedetravailtrèsenrichissanteàtoutpointdevue. Je voudraisexprimermagratitudeàtout particulièrementàWallisFillipipour sa grandedisponibilitéet son aide permanente pendant toutes ces années.

Finalement, je remercie aussi tous mes camarades du laboratoire qui ont partagé avec moileur vie quotidienne. Un grand merci notamment aux doctorants de mon bureau pour l'atmosphère très amicale: María Soledad Aronna, Florent Barret, Zhihao Cen, Camille Coron, Khalil Dayri, Sylvie

Résumé

Cettethèseestdiviséeen deuxparties. Danslapremièrepartieons'intéresse aux problèmes de commande optimale déterministes et on étudie des ap-proximations intérieures pour deux problèmes modèles avec des contraintes de non-négativitésur la commande. Le premier modèle est un problème de commande optimaledont lafonction de coût est quadratique etdont la dy-namique est régie par une équation diérentielle ordinaire. Pour une classe générale de fonctions de pénalité intérieure, on montre comment calculer le terme principal du développement ponctuel de l'état et de l'état adjoint. Notre argument principal se fonde sur le fait suivant: si la commande op-timale pour le problème initial satisfait les conditions de complémentarité strictepour leHamiltoniensaufenun nombreni d'instants,lesestimations pour le problème de commande optimale pénalisé peuvent être obtenues à partir des estimations pour un problème stationnaire associé. Nos résultats fournissent plusieurs types de mesures de qualité de l'approximation pour la technique de pénalisation: estimations des erreurs de la commande pour les normes

L

s

(

s

dans

[1, +

∞]

),estimations des erreurs pour l'état etl'état adjoint dans les espaces de Sobolev

W

1,s

(

s

dans

[1, +

∞)

) et aussi estima-tionsde erreurspour lafonction valeur. Pour la norme

L

1

etlapénalisation logarithmique, les résultats optimaux sont donnés. Dans ce cas-là on ob-tientdes erreurs pour la trajectoire centrale du problème pénalisé de l'ordre

O(ε

| log ε|)

.

Lesecondmodèleest leproblème de commande optimaled'une équation semi-linéaire elliptique avec conditions de Dirichlet homogène au bord, la commande étant distribuée sur le domaine et positive. L'approche est la mêmequepour lepremiermodèle,c'est-à-dire quel'onconsidère unefamille de problèmes pénalisés par

ε > 0

, dont la solution dénit une trajectoire centrale qui converge vers lasolution du problème initial. De cette manière, onpeutétendrelesrésultats,obtenusdanslecadred'équationsdiérentielles, aucontrôle optimald'équations elliptiquessemi-linéaires.

Dans la deuxième partie on s'intéresse aux problèmes de commande op-timale stochastiques. Dans un premier temps, on considère un problème linéairequadratiquestochastiqueavecdescontraintesdenon-negativitésurla commandeetonétendlesestimationsd'erreur pour l'approximationpar pé-nalisationlogarithmique. Lapreuves'appuie sur leprincipede Pontriaguine stochastique etun argumentde dualité.

Ensuite, on considère un problème de commande stochastique général avec des contraintes convexes sur la commande. L'approche dite variation-nelle nous permet d'obtenir un développement au premier et au second

or-ces développements on peut montrer des conditions genérales d'optimalité de premierordreet, sous unehypothèsegéométrique sur l'ensembledes con-traintes, des conditions nécessaires du secondordre sont aussi établies.

Abstract

Thisthesis isdividedintwoparts. Inthe rst oneweconsider deterministic optimalcontrolproblemsandwestudyinteriorapproximationsfortwomodel problemswith non-negativityconstraints. The rstmodelisaquadratic op-timalcontrol problem governed by a nonautonomous ane ordinary dier-entialequation. Weprovidearst-orderexpansionforthe penalizedstatean adjoint state (around the corresponding state and adjoint state of the orig-inal problem), for a general class of penalty functions. Our main argument relies on the following fact: if the optimal control satises strict comple-mentarity conditions for its Hamiltonian,except for a set of times with null Lebesgue measure, the functionalestimates of the penalizedoptimalcontrol problem can be derived from the estimates of a related nite dimensional problem. Our results provide three types of measure to analyze the penal-izationtechnique: errorestimates ofthe controlfor

L

s

norms(

s

in

[1, +

∞]

), error estimates of the state and the adjoint state in Sobolev spaces

W

1,s

(

s

in

[1, +

∞)

) and also error estimates for the value function. The sharpest results are given for the

L

1

norm and a logarithmic penalty, establishing an error estimate for the central path of order

O(ε

| log ε|)

where

ε > 0

is the (small)penalty parameter.

Thesecondmodelwestudyistheoptimalcontrolproblemofasemilinear ellipticPDEwith aDirichletboundarycondition,wherethe controlvariable is distributed over the domain and is constrained to be non-negative. F ol-lowing the same approach as in the rst model, we consider an associated familyof penalized problems, parametrizedby

ε > 0

, whosesolutions dene acentralpathconvergingtothe solutionof the originalone. In thisfashion, weareable toextendthe resultsobtainedintheODE frameworktothecase of semilinearellipticPDE constraints.

In the second part of the thesis we consider stochastic optimal control problems. We begin with the study of a stochastic linear quadraticproblem with non-negativity control constraints and we extend the error estimates forthe approximationbylogarithmic penalization. The proofis basedisthe stochastic Pontryagin's principleand a duality argument.

Next, we deal with a general stochastic optimal control problem with convex control constraints. Using the variational approach, we are able to obtain rst and second-order expansions for the state and cost function, aroundalocalminimum. This analysis allows ustoprove generalrstorder necessary conditionand,under ageometricalassumptionoverthe constraint

Contents

Remerciements 1

I General introduction 9

0.1 Deterministicoptimalcontrol . . . 11

0.1.1 A brief review of interior point methodsfor quadratic programming . . . 12

0.1.2 Presentation of our main results . . . 14

0.1.2.1 Optimalcontrolof ODEs . . . 15

0.1.2.2 Optimalcontrolof PDEs. . . 18

0.2 Stochastic optimalcontrol . . . 21

0.2.1 A review of the global approach . . . 21

0.2.2 A review of the variationalapproach . . . 22

0.2.3 Presentation of our main results . . . 24

0.2.3.1 ErrorestimatesforapenalizedstochasticLQ problem . . . 24

0.2.3.2 Optimality conditions in stochastic optimal controltheory . . . 26

II Asymptotic expansions for interior penalty solu-tions of control constrained problems 29 1 Optimal control of a linear dierential equation 31 1.1 Introduction . . . 32

1.2 Problemstatement and preliminaryresults . . . 33

1.2.1 Mainproblem . . . 34

1.2.2 Penalized problems . . . 35

1.3 Interior penalty analysis inthe nite dimensionalsetting . . . 39 1.3.1 Convergence properties of the approximate projectors . 40 1.3.2 Straticationresultsandstrict complementarity

refor-1.4 Main results . . . 48

1.4.1 Error estimates for interior penalties . . . 49

1.4.2 Asymptoticexpansion . . . 55

1.5 Examples . . . 56

1.5.1 Decoupledcase:

R(t)

≡ I

. . . 56

1.5.1.1 Negativepowerpenalty . . . 57

1.5.1.2 Logarithmicpenalty . . . 57

1.5.2 Coupled case:

R(t)

 0

. . . 59

1.6 Conclusions . . . 62

2 Optimal control of a semilinear elliptic partial dierential equation 65 2.1 Introduction . . . 66

2.2 Problem statementand preliminary results . . . 67

2.3 Main results . . . 79

2.4 Examples . . . 87

2.4.1 Error estimates forthe central path . . . 87

2.4.1.1 Negativepowerpenalty . . . 88

2.4.1.2 Power penalty. . . 88

2.4.1.3 Entropy penalty . . . 88

2.4.1.4 Logarithmicpenalty . . . 88

2.4.2 Error estimatefor the cost function . . . 90

III Stochastic optimal control theory 93 3 Error estimates for the logarithmic barrier method in linear quadratic stochastic optimal control problems 95 3.1 Introduction . . . 96

3.2 Problem Statementand OptimalityConditions. . . 97

3.2.1 The initialproblem . . . 98

3.2.2 The penalizedproblem . . . 100

3.3 Main Result . . . 101

4 First and second order necessary conditions for stochastic optimal control problems 105 4.1 Introduction . . . 106

4.2 Notations, assumptions and problemstatement . . . 107

4.3 Expansions for the state and cost function . . . 110

4.4.2 Second order necessary conditions . . . 124 4.5 On the second order sucientcondition. . . 128

Part I

In thispart ofthe thesiswereview someelementaryconcepts of both de-terministicandstochasticoptimalcontrolproblemswithcontrolconstraints. After giving the necessary elements of the theory we will expose the main resultsobtained. Letusstartwiththestudy ofdeterministicoptimalcontrol problems.

0.1 Deterministic optimal control

An optimal control problem of ordinary dierential equations (ODE) with controlconstraintscan bewritten inthe following form:

inf

(y,u)∈V×Y

R

T

0

`(t, y(t), u(t))dt + φ(T, y(T ))

s.t.

˙y(t) = f (t, y(t), u(t))

for

t

∈ [0, T ]; y(0) = y

0

,

u

∈ U.

(

DCP)

0

In the notation above,

` : R

n

_{× R}

m

_{→ R}

represents the running cost,

φ : R

n

_{→ R}

the nal cost and

y(t)

∈ R

n

represents the state variable controlled by

u(t)

∈ R

m

through the dynamics

f : R

n

_{× R}

m

_{→ R}

m

. If

f

is anewith respect to

u

wemay takeascontrolspace

V = L

2

_{([0, T ]; R}

m

₎

and as state space

Y = W

1,2

_{([0, T ]; R}

n

₎

. Otherwise, we take

V = L

∞

_{([0, T ]; R}

m

₎

and

Y = W

1,∞

_{([0, T ]; R}

n

₎

. The control variable is constrained to belong to

U ⊆ V

. Note that this framework includes global constraints, e.g.

U :=

{v ∈ V / ||v||

2

≤ 1}

, as well as local constraints, e.g. the so-called box con-straints

U := {v ∈ V / a ≤ v(t) ≤ b,

for a.a.

t

∈ [0, T ]}

. In the rst part of thisthesiswewillfocus ourattention tothe caseofboxconstraints. Inorder tosimplifythe analysis,wewillrestrictourselvestothe caseof non negativ-ity constraints. In the second part of the thesis we willdetermine rst and second-order optimality condition for the stochastic version of

(

DCP)

0

and we willwork with a moregeneral constraintset

U

.

Thus, inwhat follows weassume that

U := {v ∈ V / v(t) ≥ 0,

for a.a.

t

∈ [0, T ]} .

(1) Since,the activeset (i.e. the set oftimeswhere theoptimalcontrolis

0

)isa priorinot known, numericaldiculties appear in the implementationofany direct algorithm. One way to tackle this problem is to extend the natural ideas of interior-point methods for nonlinear programming problems. More precisely,weconsiderafamilyofperturbed optimalcontrolproblems satisfy-ingthat theirsolutionsare strictly positive(and thusthey can be computed

the procedure. As an example, for the logarithmic-penalty case, a natural approximation of

(

DCP)

0

is the followingproblem

inf

(y,u)∈V×Y

R

T

0

[`(y(t), u(t))

− ε log u(t)] dt + φ(y(T ))

s.t.

˙y(t) = f (y(t), u(t))

for

t

∈ [0, T ]; y(0) = y

0

,

u

_{∈ U.}

(

_DCP)

ε

Theconvergence ofthesolutionsof

(

DCP)

ε

tothesolutionof

(

DCP)

0

,as

ε

↓ 0

, is shown in [22], but no error estimates are obtained. As we will see, theseestimatescan beobtainedasaby-product ofthequalitativeproperties of the central path(dened insection0.1.2.1), which are strongly relatedto their nite-dimensionalcounterparts, recalledin the next section.

0.1.1 Abrief reviewof interior point methodsforquadratic programming

Consider the followingnite dimensional optimizationproblem

Min

x∈R

n

1

2

x

>

Rx + c

>

x; Ax = b, x

≥ 0,

(

QP)

0

where

R

∈ R

n×n

is a positive-semidenite matrix,

c

∈ R

n

and

b

∈ R

p

. We say that the problem is linear if

R = 0

. If

(

QP)

0

has at least one solution

x

0

,there there exists

(s

0

, λ

0

)

∈ R

n

+

× R

p

such that

z

0

:= (x

0

, s

0

, λ

0

)

solves







x

>

_{s = 0,}

Ax = b, c + Rx + A

>

_{λ = s,}

x

≥ 0, s ≥ 0.

(2)

In viewof this property,from nowon we referto

z

0

asa solutionof

(

QP)

0

. Now, consider aparameterized family of problems that penalize the non negativity constraint of

(

QP)

0

. That is, for every

ε > 0

, dene the problem

(

_QP)

ε

as Min

x∈R

n

1

2

x

>

Rx + c

>

x

− ε

p

X

i=1

log x

i

; Ax = b.

(

QP)

ε

.

It is possible to prove that if

(

QP)

0

has a solution

x

0

then, for

ε

small enough, problem

(

QP)

ε

has a solution

x

ε

. Moreover, there exists

(s

ε

, λ

ε

)

∈

R

n

₊

_{× R}

p

such that

z

ε

:= (x

ε

, s

ε

, λ

ε

)

solves







x

>

_{s = ε,}

Ax = b, c + Rx + A

>

_{λ = s,}

x

_{≥ 0, s ≥ 0.}

(3)

Thus, we refer to

z

ε

as a solution of

(

QP)

ε

. The application

ε

→ z

ε

is called central path and it is well known that as

ε

↓ 0

, we have

z

ε

→ z

0

. Moreover, qualitative properties of the central path (error estimates of its slope)are relatedwith the followingnotion of strict complementarity:

Denition 1 Wesaythatthesolution

z

0

of

(

QP)

0

isstrictlycomplementary if

x

0

+ s

0

> 0

.

In the linear case (

R = 0

), if the set of solutions of

(

QP

0

)

is nonempty, there exists at least one strictly complementary solution and the central path converges to one solution of this kind (see [82]). In the strictly convex quadratic case (

R

 0

), the problem

(

QP)

0

has a unique solution

z

0

and

z

ε

→ z

0

. Inaddition,if

z

0

is strictly complementary,then

||z

ε

− z

0

|| = O(ε)

. Ifstrictcomplementaritydoesnot hold,

||z

ε

− z

0

|| = O(

√

ε)

-see[92]. Letus giveatrivialexamplewhereweseetheimportanceofstrictcomplementarity for the speed of convergence of the central path.

Example 1 Consider the problem

Min

x∈R

1

2

x

2

_{; x}

_{≥ 0,}

which has as unique solution

x

0

= 0

. The penalized version of the above problem is

Min

x∈R

1

2

x

2

_{− ε log x,}

which has as unique solution

x

ε

=

√

ε

, and thus

|x

ε

− x

0

| =

√

ε

. One can easily verify that

x

0

is not strictly complementary. On the other hand, the problem

Min

x∈R

1

2

x

2

_{; x}

_{≥ 1,}

has a unique solution

x

0

= 1

. Inthis case strict complementarity issatised anda simplecomputationshowsthatthe solution

x

ε

ofthe penalizedproblem satises

|x

ε

− x

0

| = O(ε)

.

Thesepropertiesofthecentralpathallowustojustifytheoreticallytheuseof several typesofinteriorpointalgorithmsfor

(

QP)

0

. Forexample,foraxed

ε

thepenalizedproblemcanbesolvedbyapplyingNewton'smethod. Then,

ε

isdecreasedand thementionedmethodisre-initializedtakingasthestarting point the approximate solution of the previous problem. Thus, a priori this pointmustbelong totheconvergence regionofthe new Newton'salgorithm. Thereare several variationsofthis generalprinciple, fordetailedexpositions

references therein. Finally, let us mention that these methods are studied for more general settings, as general convex problems with self concordant barrier functions [74], linear monotone complementarity problems [21] and semideniteprogramming [82], etc.

0.1.2 Presentation of our main results

Inthis sectionweapplythebarrier-methodideastotheoptimalcontrolofan ordinarydierentialequation(ODE)andtotheoptimalcontrolofa semilin-earellipticpartialdierentialequation(PDE).Inbothcasesaparameterized family of penalized problems is considered, for which optimality conditions are derived. The main idea is to eliminatethe controlvariable fromthe re-sulting equations and to apply a variation of the implicit function theorem tothe reduced optimality system.

The main tool willbe the following theorem and its corollary, which is a variantof the surjective mappingtheorem of Graves [49].

Theorem 2 (RestorationTheorem)Let

X

and

Y

beBanachspaces,

E

a metricspace and

F : U

⊂ X × E → Y

a continuous mappingon anonempty open set

U

. Let

(ˆ

x, ε

0

)

∈ U

be such that

F (ˆ

x, ε

0

) = 0

. Assume that there existsa surjective linear continuous mapping

A : X

→ Y

, with bounded right inverse B, and a function

c : R

+

→ R

+

with

c(β

0

₎

_{↓ 0}

when

β

0

_{↓ 0}

, such that: if

β > 0

satises

c(β)

||B|| < 1

and

ε

∈ B(ε

0

, β)

, then

kF (x

0

, ε)

_{−F (x, ε)−A(x}

0

_{−x)k ≤ c(β)kx}

0

_−xk,

for all

(x, x

0

₎

_{∈ B(ˆx, β)×B(ˆx, β).}

(4) Undertheassumptionsabove,forall

(x, ε)

closeenoughto

(ˆ

x, ε

0

)

, thereexists

¯

x

such that

F (¯

x, ε) = 0

and the followingestimate holds:

k¯x − xk ≤

||B||

1

_{− c(β)||B||}

kF (x, ε)k.

(5) Corollary 3 Suppose that the assumptions of Theorem 2 hold and denote by

B

a boundedright inverse of

A

. Then,for

ε

close to

ε

0

, there exists

x

ε

in a neighborhood of

x

ˆ

such that

F (x

ε

, ε) = 0

and

x

ε

= ˆ

x

− BF (ˆx, ε) + r(ε),

(6) where the remainder

r(ε)

satises

kr(ε)k ≤ c(β)(1 − c(β)kBk)

−1

_kBk

2

_{kF (ˆx, ε)k.}

(7) For the proof of the above results, we refer the reader to the appendix of

0.1.2.1 Optimal control of ODEs

In this sectionwe presentthe mainresults obtained inChapter 1,whichare thesubjectofreport [2]. Forthesakeof clarity,westudyasimpliedversion of the general linear quadraticproblem analyzedin Chapter 1. We consider the problem

(

DCP)

0

with

`(t, y, u) :=

1

2

|u|

2

₊

1

2

C(t)

|y − ¯y(t)|

2

,

φ(T, y) :=

1

2

M

|y − ¯y(T )|

2

,

f (t, y, u) := A(t)y + u,

(8)

and

U

given by (1) with

V = L

2

_{([0, T ]; R)}

. In the notation above,

C

∈

C

0

_{([0, T ])}

with

C(t)

≥ 0

,

M

≥ 0

,

A

∈ C

0

_{([0, T ])}

and

y

¯

∈ C

0

_{([0, T ])}

is a reference state function.

For every

ε > 0

dene

(

DCP)

ε

, the logarithmic penalized version of

(

_DCP)

0

,by inf

(y,u)∈Y×V

R

T

0

`

ε

(t, y(t), u(t))dt + φ(y(T ))

s.t.

˙y(t) = f (t, y(t), u(t))

for

t

∈ [0, T ]; y(0) = y

0

,

u

∈ U,

(

DCP)

ε

where

`

ε

(t, y, u) := `(t, y, u)

− ε log u

. Fornotational convenience wealsoset

`

0

(t, y, u) = `(t, y, u)

. Classical arguments yield that for every

ε

∈ [0, ∞)

problem

(

DCP)

ε

hasaunique solution,denoted by

(y

ε

, u

ε

)

. Moreover, itcan be shown [22] that there exists

c > 0

such that for every

ε > 0

we have that

u

ε

(t)

≥ cε

for a.a.

t

∈ [0, T ]

.

For

ε

∈ [0, ∞)

, dene the Hamiltonian

H

ε

: [0, T ]

× R × R × R → R

by

H

ε

(t, y, p, u) := `

ε

(t, y, u) + pf (t, y, u).

(9)

The Pontryagin minimum principle (cf. [77]) yields the existence of

p

ε

∈

W

1,2

_{([0, T ]; R)}

such that

˙y

ε

(t) = A(t)y

ε

(t) + u

ε

(t)

for a.a.

t

∈ [0, T ],

(10)

− ˙p

ε

(t) = A(t)p

ε

(t) + C(t)[y

ε

(t)

− ¯y(t)]

for a.a.

t

∈ [0, T ],

(11)

y

ε

(0) = y

0

,

p

ε

(T ) = M[y

ε

(T )

− ¯y(T )],

(12)

u

ε

(t) =

argmin

{H

ε

(t, y

ε

(t), p

ε

(t), v) : v

≥ 0}

for a.a.

t

∈ [0, T ].

(13) Our aim is to establish the relations between

(y

ε

, p

ε

, u

ε

)

, the so-called the central path, and

(y

0

, p

0

, u

0

)

, for

ε > 0

smallenough. The rst step is touse (13) in orderto eliminate

u

ε

in the system (10)-(12). In fact, condition(13) yieldsthat for a.a.

t

∈ [0, T ]

,

u

ε

(t) := ϕ

ε

(

−p

ε

(t))

,where

ϕ

ε

(x) :=



₁

2

x +

√

x

2

_{+ 4ε}

if

ε > 0,

max

_{{x, 0}}

if

ε = 0.

(14)

Thus, for every

ε

∈ [0, ∞)

,optimalityconditions (10)-(12)are equivalent to

˙y

ε

(t) = A(t)y

ε

(t) + ϕ

ε

(

−p

ε

(t)),

− ˙p

ε

(t) = A(t)p

ε

(t) + C(t)[y

ε

(t)

− ¯y(t)],

y

ε

(0) = y

0

,

p

ε

(T ) = M[y

ε

(T )

− ¯y(T )].

(15)

The forward backward system (15) induces the denition of the mapping:

F : W

1,1

([0, T ]; R)

×W

1,1

_{([0, T ]; R)}

_×R

+

→ L

1

([0, T ]; R)

×R×L

1

([0, T ]; R)

×R

dened by

F (y, p, ε)(

_{·) :=}









˙y(

·) − A(·)y(·) − ϕ

ε

(

−p(·))

y(0)

_{− y}

0

˙p(

_{·) + A(·)p(·) + C(·)(y(·) − ¯y(·))}

p(T )

− M[y(T ) − ¯y(T )]







 .

(16)

In order to obtain a rst order expansion of

(y

ε

, p

ε

)

around

(y

0

, p

0

)

the rst idea that comes to mind, as in the classical sensitivity analysis, is to apply theimplicitfunctiontheoremtothemapping

F

at

(y

0

, p

0

, 0)

. Unfortunately, itisshown inChapter 1thatthistheorem isnot applicablesince,ingeneral,

D

ε

F (y

0

, p

0

, 0)

does not exist. As analternativewe use the restoration theo-rem(theorem 2)anditscorollary(corollary3),toobtainthedesired asymp-totic expansionand the associated errorestimates for the centralpath. It is seen thatthe strict dierentiability hypothesis (4), which inour case iswith respect to

(y, p)

at

(y

0

, p

0

, 0)

, is strongly related with the concept of strict complementary for the solution of a nite-dimensional problem, exposed in subsection 0.1.1. In fact, letus assumethe

Strictcomplementarityassumption: Thereexistsasubset

T

sing

of

[0, T ]

with meas

(T

sing

) = 0

, such that for each

t

in

[0, T ]

\ T

sing

the point

u

0

(t)

satises the strict complementarity conditions for the minimization problem

min

{H

0

(t, y

0

(t), p

0

(t), w) : w

∈ R

+

} .

Theassumptionabovecanbereformulatedinthefollowinggeometricalform: Except for a null Lebesgue set the curve

p

0

(t)

does not intersect the x-axis, i.e. the function

t

∈ [0, T ] →

d

dt

ϕ

0

(

−p

0

(t))

is a.s. well dened.

Underthishypothesiswecanapplytheorem2andproveourmainresults. Therstoneconcernstheerrorestimatesforthecentralpath,anditsaysthat the error bounds can be calculated from the error bounds of the analogous nite dimensional problems (which, in the case of the logarithmic penalty, are of order

√

ε

).

Theorem 4 (Error estimates for interior penalty) Underthestrict com-plementarity assumption , for

ε

small enough we have that:

(i) The error estimates for

u

ε

, y

ε

and

p

ε

are givenby

||u

ε

− u

0

||

∞

+

||p

ε

− p

0

||

1,∞

+

||y

ε

− y

0

||

1,∞

= O(

√

ε)

with in addition

u

ε

→ u

0

in

W

1,1

.

(ii) In addition, let us assume that

{t ∈ [0, T ] ; p

0

(t) = 0

}

isnite and that the followingimplication holds:

p

0

(t

0

) = 0

⇒

d

dt

p

0

(t

0

)

6= 0.

(17) Then

||u

ε

− u

0

||

1

+

||p

ε

− p

0

||

1,1

+

||y

ε

− y

0

||

1,1

= O(ε

| log ε|).

(18) Now,westateoursecondmainresultwhichyieldstheasymptotic expan-sion of

(y

ε

, p

ε

)

around

(y

0

, p

0

)

in

W

1,1

_{([0, T ]; R)}

.

Theorem 5 (Asymptotic expansion) Supposethatthestrict complemen-tarity assumption (1.53) holds, then for

ε

smallenough,



y

ε

p

ε



=



y

0

p

0



− D

(y,p)

F (y

0

, p

0

, 0)

−1

F (y

0

, p

0

, ε) + r(ε),

where

r(ε) = o(

_{||F (y}

0

, p

0

, ε)

||

1

).

Moreover, the rst term of the expansion

−D

(y,p)

F (y

0

, p

0

, 0)

−1

_{F (y}

0

, p

0

, ε)

is the unique solution of























Min

1

₂

Z

T

0

|v(t)|

2

_{+ C(t)}

_|σ(t)|

2

_{dt +}

1

2

M

|σ(T )|

2

_,

s.t.

˙σ(t) = A(t)σ(t) + v(t) + [ϕ

ε

(

−p

0

(t))

− ϕ

0

(

−p

0

(t))] ,

σ(0) = 0,

v(t) = 0

if

p

0

(t)

≥ 0.

Finally,let usmention that theorems 4 and 5 are proved in Chapter 1 for a generallinearquadraticproblemandfor ageneralclass ofpenaltyfunctions. Of course, the error bounds obtained there depend on the chosen penalty function. The main technical diculty appears when the controlis coupled inthe cost function by a non diagonalmatrix

R(t)

.

0.1.2.2 Optimal control of PDEs

Thestudy presented hereis thesubject ofthe report[25], whichextendsthe resultsofthe previoussectiontothe optimalcontrolproblemofasemilinear PDE, under non negativity constraints over the control. For

u

∈ L

s

_(Ω)

(

s

∈ [2, ∞]

) denoteby

y

u

∈ W

2,s

_(Ω)

the unique solution of



−∆y(x) + φ(y(x)) = f(x) + u(x)

for

x

∈ Ω,

y(x) = 0

for

x

∈ ∂Ω,

(19)

where

Ω

isa bounded open set of

R

n

with

C

2

boundary,

f

∈ L

s

_(Ω)

and

φ

is a

C

2

Lipschitznondecreasingreal valuedfunctionover

R

. For

s > n/2

(

s = 2

if

n

≤ 3

),let usdene

J

0

: L

s

_(Ω)

_{→ R}

by

J

0

(u) :=

1

₂

Z

Ω

(y

u

(x)

− ¯y(x))

2

dx +

1

₂

N

Z

Ω

u(x)

2

dx.

(20)

We are interested inthe following optimizationproblem

Min J

0

(u)

subject to

u

∈ U

s

+

.

(

CP

s

0

)

where

U

+

s

:=

{v ∈ L

s

(Ω) / v(x)

≥ 0,

for a.a.

x

∈ Ω} .

Since

φ

can be nonlinear, problem

(

CP

s

0

)

is a non-convex one. Nevertheless, itcanbeshown (corollary51)that

(

CP

s

0

)

hasatleastonesolution. Ourmain results will depend heavily on a second-order sucient condition at a local minimum of

(

CP

s

0

)

. Lemma 6.27 in [24] yields that

J

0

: L

s

_(Ω)

_{→ R}

is

C

2

if

s > n/2

(

s = 2

if

n

≤ 3

). That is the main reason for considering

L

s

_(Ω)

as controlspace,rather than the standard space

L

2

_(Ω)

. Forevery

u

∈ L

s

_(Ω)

denethe adjoint state

p

u

∈ W

2,s

_(Ω)

, asthe unique solutionof



−∆p(x) + φ

0

_(y

u

(x))p(x) = y

u

(x)

− ¯y(x)

for

x

∈ Ω,

p(x) = 0

for

x

∈ ∂Ω.

(21) Let

u

0

∈ U

s

+

be a localsolution of

(

CP

s

0

)

and denoterespectively by

y

0

and

p

0

its associated state and adjoint state. Applying classical techniques (see [55, 67, 73]) we obtain that (recall(14))

u

0

(x) = ϕ

0

(

−p

0

(x))

for a.a.

x

∈ Ω.

Now, let us suppose that

u

0

is locally unique in the

L

s

_(Ω)

ball

B

¯

s

(u

0

, b)

and,for

ε > 0

,consider the followinglogarithmic penalizedversion of

(

CP

s

0

)

Min J

ε

(u) := J

0

(u)

− ε

Z

Ω

log(u(x))dx

s. t.

u

∈ U

s

As for

(

CP

s

0

)

, problem

(

CP

b,s

ε

)

has at least one solution. Note that the ap-plication

u

∈ L

s

_(Ω)

_{→ −}

Z

Ω

log(u(x))dx

∈ R ∪ {+∞}

is not continuous, hence not dierentiable. Thus it is not immediate to writeoptimalityconditionsfor

(

CP

b,s

ε

)

. However, usingan

L

1

_(Ω)

contraction principle (lemma 54), we get that, as

ε

↓ 0

, the solutions

u

ε

of

(

CP

b,s

ε

)

converge to

u

0

in

L

s

_(Ω)

. In addition, there exists

c, K > 0

such that for

ε

smallenough

cε

≤ u

ε

(x)

≤ K

fora.a.

x

∈ Ω.

(22) The estimates (22) imply that

u

ε

solves

Min J

ε

(u)

subject to

u

∈ U

s

+

∩ ¯

B

s

(u

0

, b

0

)

∩ L

∞

(Ω)

and the application

u

∈ L

∞

_(Ω)

_{→ −}

R

Ω

log(u(x))dx

∈ R ∪ {+∞}

is dier-entiable at

u

ε

, which allows us to write rst order optimalityconditions. In fact,denotingrespectivelyby

y

ε

and

p

ε

thestateandadjointstateassociated to

u

ε

, we have that (recall (14))

u

ε

(x) = ϕ

ε

(

−p

ε

(x))

for a.a.

x

∈ Ω.

Therefore, itis natural todene the map

F : W

1,s

_{× W}

1,s

_{× R}

+

→ L

s

(Ω)

×

L

s

_(Ω)

by

F (y, p, ε)(

_{·) :=}



∆y(

_{·) + ϕ}

ε

(

−N

−1

p(

·)) + f(·) − φ(y(·))

∆p(

_{·) + y(·) − ¯y(·) − φ}

0

_(y(

_·))p(·)



.

(23)

Letus assume the following hypothesis

(H1) For the adjoint state

p

0

, associated to any localsolution

u

0

of

(

CP

s

0

)

, it holdsthat

meas

(

{x ∈ Ω / p

0

(x) = 0

}) = 0.

(H2) Atanylocalsolution

u

0

of

(

CP

s

0

)

,thefollowingsecond-ordercondition holds

D

2

J

0

(u

0

)(h, h) > 0

for all

h

∈ C(u

0

)

\ {0}

(24) where

C(u

0

) := T

U

(u

0

)

∩ DJ(u

0

)

⊥

is the usual critical cone at

u

0

.

Asummptions(H1),(H2)implythat thehypothesisof theorem 2are satis-edat

(y

0

, p

0

, 0)

. Moreprecisely,assumption(H1)allowstoprove(4),while (H2)yields the surjectivity assumption of the operator

A

.

Theorem 6 Let

u

0

be a solution of

(

CP

s

0

)

, suppose that

φ

is

C

2

and that (H1), (H2) hold. Denote respectively by

y

0

and

p

0

the state and adjoint stateassociated to

u

0

. Thenthereare

¯b > 0

and

ε > 0

¯

suchthatfor

ε

∈ [0, ¯ε]

problem

(

CP

¯

b,s

ε

)

has a unique solution

u

ε

. Inaddition, denoting by

y

ε

and

p

ε

the associated state and adjoint state for

u

ε

, the following expansion around

(y

0

, p

0

)

holds



y

ε

p

ε



=



y

0

p

0



+ D

(y,p)

F (y

0

, p

0

, 0)

−1

F (y

0

, p

0

, ε) + r(ε),

(25)

where

r(ε) = o(

||F (y

0

, p

0

, ε)

||

s

)

. Moreover,

D

(y,p)

F (y

0

, p

0

, 0)

−1

_{F (y}

0

, p

0

, ε)

is characterized as being the unique solution of























Min

Z

Ω



₁

2

Nv

2

₊

1

2

(1

− p

0

φ

00

(y

0

)) z

2



_dx,

s.t.

−∆z(x) + φ

0

_(y

u

(x))z(x) = v + ϕ

ε

(q

0

)

− ϕ

0

(q

0

)

for

x

∈ Ω,

z(x) = 0

for

x

∈ ∂Ω,

v(x) = 0

if

u

0

(x) = 0.

Theorem 7 Suppose that the assumptions of theorem 6 hold. Let

¯b > 0

be such that

(

CP

¯

_b,s

ε

)

has a unique solution

u

ε

for

ε > 0

smallenough. Then: (i) We have

||u

ε

− u

0

||

∞

+

||p

ε

− p

0

||

2,s

+

||y

ε

− y

0

||

2,s

= O(

√

ε).

(26) (ii)If in addition

n

≤ 3

(hence

s = 2

),thereexist

m

∈ N

, positivereal num-bers

α > 0

,

0 < ¯

δ < 1

and a nite collection of closed

C

2

curves

(C

i

)

1≤i≤m

such that:

•

The singularset

{x ∈ Ω / p

0

(x) = 0

}

can be expressed as

{x ∈ Ω / p

0

(x) = 0

} =

m

[

i=1

C

i

.

(27)

•

For all

i

∈ {1, ..., m}

, dening

C

¯

δ

i

:=

{x ∈ Ω; dist(x, C

i

)

≤ ¯δ}

, it holds that:

|p

0

(x)

| ≥ α dist(x, C

i

)

for all

x

∈ C

¯

δ

i

.

(28)

Then

||u

ε

− u

0

||

2

+

||p

ε

− p

0

||

2,2

+

||y

ε

− y

0

||

2,2

= O(ε

3

4

).

(29) We concludethis sectionremarkingthat the aboveresults are generalizedin

0.2 Stochastic optimal control

Let

T > 0

and consider a ltered probability space

(Ω,

F, F, P)

, onwhich a

d

-dimensional(

d

∈ N

∗

)Brownianmotion

W (

·)

isdenedwith

F

=

{F

t

}

0≤t≤T

being its natural ltration,augmented by all

P

-nullsets in

F

. Consider the followingcontrolledstochastic dierentialequation (SDE)

dy(t) = f (y(t), u(t))dt + σ(y(t), u(t))dW (t),

for

s

∈ (t, T )

y(t) = x,

(30)

where

x

∈ R

n

and

0

≤ t < T

. In the notation above,

y(t)

represents the state variable, controlled by

u

∈ U[0, T ]

,where

U[0, T ] := {u : [0, T ] × Ω → U / u

isprog. measurable

}

for some subset

U

⊆ R

m

. We say that

u

is admissibleif

u

∈ U[0, T ]

and the SDE (30) has aunique solution

y

x

u

. The set of admissibleprocess isdenoted by

U

ad

. Foraxed

x

0

∈ R

n

,weare interested inproblem

V (0, x

0

)

dened as

V (0, x

0

) :=

Inf

u∈U

ad

E

Z

T

0

`(y

x

0

u

(t), u(t))dt + φ(y

x

u

0

(T ))



,

where

`

and

φ

aretherunningandnalcost,respectively. Standard assump-tions are supposed to hold for the functions that dene the dynamics and the cost.

0.2.1 A review of the global approach

We begin by briey reviewing the globalapproach (foradetailed exposition we refer the reader to the excellent books [45, 76, 93]). It consists in to embed the problem

V (0, x

0

)

into a family of problems, parameterized by

(t, x)

_{∈ [0, T ] × R}

n

, dened by

V (t, x) :=

Inf

u∈U

ad

E

Z

T

t

`(y

_u

x

(t), u(t))dt + φ(y

_u

x

(T ))



.

If

V

∈ C

1,2

_{([0, T ]}

_{× R}

n

₎

then, it is proved, using the dynamic programming principle, that

V

is asolution of the followingsecond-order PDE:

∂V

∂t

(t, x) +

H (x, V (t, x), DV (t, x), D

2

_{V (t, x)) = 0, (t, x)}

_{∈ [0, T ) × R}

n

where

H : R

n

_{× R}

n

_{× R}

n×n

_{→ R}

is dened by

H (x, r, p, A) :=

inf

u∈U



`(x, u) + p

>

f (x, u) +

1

₂

Tr



σσ(x, u)

>

A



.

Unfortunately, only continuity results hold a priori for

V

. Nevertheless, it can be shown that

V

is the unique solution of (31) in the weak sense of viscosity solutions (see [37]). In this thesis we will not deal with the latter approach,whichhasbeenwidelystudiedtheoreticallyandnumericallyinthe recent years. In fact, wewillanalyze the stochastic optimalcontrolproblem froma variationalpoint of view, whichwereview in the next section.

0.2.2 A review of the variational approach

Weoerhere onlyabriefreview ofthe variationalapproach. Foracomplete exposition we refer the reader to [10], [93, Chapter 3] and the references therein. In this approach we work directly with problem

V (0, x

0

)

and, for simplicity,wesuppose thattheadmissiblecontrolsbelongtoaBanachspace. Thisfactallowustousegeneraloptimizationtechniquesinordertoestablish optimality conditions. More precisely, consider the spaces

L

2

F

:=

{u : [0, T ] × Ω → R

m

/ u

is prog. measurable and

||u||

2

<

∞} ,

L

2,∞

_F

:=

_{{y : [0, T ] × Ω → R}

n

_{/ y}

is prog. measurable and

||y||

2,∞

<

∞} ,

where

||u||

2

2

:= E

Z

T

0

|u(t)|

2

_dt



,

_||y||

2

_2,∞

:= E

sup

t∈[0,T ]

|y(t)|

2

!

.

It is well known that if

f, σ

have linear growth, then for every

u

∈ L

2

F

equation(30)admits auniquesolution

y

u

∈ L

2,∞

F

and the thereexists

C > 0

suchthat

||y

u

||

2

_2,∞

≤ C |x

0

|

2

+

||f(0, u(·))||

2

2

+

||σ(0, u(·))||

2

2

.

(32)

Therefore, it isnatural toassume that

U

ad

=



u

_{∈ L}

2

_F

/ u(t, ω)

_{∈ U}

for a.a.

(t, ω)

∈ [0, T ] × Ω

.

(33)

Since

x

0

is xed, we will write

y

u

= y

x

0

u

. Thus, problem

V (0, x

0

)

can be expressed in the followingway

Inf

J(u) := E

Z

T

t

`(y

u

(t), u(t))dt + φ(y

u

(T ))



The existenceproblemfor

(

SP)

0

isadiculttask,whichhas been analyzed byseveralresearchers. Letuscitetheworks[7,38,41,44,60]andthesurvey [28]. Fromnowonweassumethatasolutionof

(

SP)

0

exists. Thevariational approachconsistsintostudythe eectsof perturbationsof alocalminimum onthecostfunction

J

. Inaverygeneralframework,rstorderconditionscan be established. The procedure isthe natural extensionof theanalysis inthe deterministiccase. In fact, let

u

¯

be asolution and set

y := y

¯

¯

u

. Consider the following backward stochastic dierential equation (BSDE), with variables

(p, q)

,

dp(t) =

₋

"

`

y

(t)

>

+ f

y

(t)

>

p(t) +

m

X

i=1

σ

i

_y

(t)

>

q

i

(t)

#

dt + q(t)dW (t),

p(T ) = D

y

φ(¯

y(T ))

>

,

(34) where

`

y

(t) := D

y

`(¯

y(t), ¯

u(t)); f

y

(t) := D

y

f (¯

y(t), ¯

u(t)).

Understandardassumptions(see[8,18]),theaboveequationadmitsaunique adapted solution

(¯

p, ¯

q)

∈ L

2,∞

F

× (L

2

F

)

d

called the adjoint state associated to

¯

u

. Moreover, thereexists

C

0

_{> 0}

such that

||¯p||

2

2,∞

+

d

X

i=1

||¯q

i

||

2

2

≤ C

0



E

_|D

y

φ(¯

y(T ))

|

2

+

_||`

y

(

·)||

2

2



.

(35)

The Hamiltonian

H

of the problemis dened as

H(y, p, q, u) := `(y, u) + p

· f(y, u) +

d

X

i=1

q

i

· σ

i

_{(y, u).}

(36)

When

σ

u

≡ 0

then by perturbing

u

¯

with the so-called needle (or spike) variations (see [77]),it can beshown that the optimalcontrol

u

¯

satisesthe followingPontryagin principle(see[8,9,15,16,18,53,61,62,63]forrelated works)

¯

u(t, ω)

∈

argmin

v∈U

H(¯

y(t, ω), ¯

p(t, ω), ¯

q(t, ω), v)

for a.a.

(t, ω).

(37)

Also, by introducing ageneralized Hamiltonianand adding a secondpair of adjoint variables, the previous condition (37) has been generalized, to the case when

σ

can depend on

u

by Peng in[75].

0.2.3 Presentation of our main results

We begin by extending the logarithmic barrier method of chapter 1 to the case of a stochastic LQ problem. Even if we do not obtain an asymptotic expansionforthestateandadjointstate,weareabletoprovetheconvergence for the central path together with some error estimates. Such estimates are the natural extensions of those obtained in chapter 1 in the deterministic framework.

Next,wedealwithageneralstochasticoptimalcontrolproblemwith con-vexconstraintsbutnotnecessarilyoflocaltype. Indeed,usingthevariational approach we are able to derive rst and second order optimality conditions for a localsolution. They are the natural extensions of well know results in the deterministiccase.

0.2.3.1 Error estimates for a penalized stochastic LQ problem Inthissectionweconsideranimportantinstance of

(

SP)

0

,whichisthecase ofacontrolconstrainedstochasticLQproblem. The analysispresented here are the subject of report [26]. In order to illustrate the result in a simple manner,weconsideraveryparticularconvexLQproblem. Foramoregeneral convex LQproblem we refer the reader to chapter 3. We suppose here that

m = n = d = 1

and that the data of

(

SP)

0

is

`(y, u) =

1

₂

(u

2

_{+ y}

2

_{) , φ(y) =}

1

2

y

2

_,

f (y, u) = y + u,

σ(y, u) = y + u,

x

0

∈ R

and

U

ad

:=



u

∈ L

2

F

/ u(t, ω)

≥ 0

fora.a

(t, ω)

∈ [0, T ] × Ω

.

Since the cost function is strongly convex and continuous, problem

(

SP)

0

admits a unique solution

u

0

. We denote respectively by

y

0

:= y

u

0

and

(p

0

, q

0

) := (p

u

0

, q

u

0

)

for the state and the adjoint state associated to

u

0

. The stochastic Pontryagin minimumprinciple (SPMP) (37) impliesthat

u

0

(t, ω) = φ

0

(

−p

0

(t, ω)

− q

0

(t, ω))

for a.a.

(t, ω)

∈ [0, T ] × Ω,

wherewe recall that

φ

0

is dened in(14).

Asinsection0.1.2.1,for

ε > 0

wedeneproblem

(

SP)

ε

by modifyingthe cost

`

of

(

SP)

0

by

`

ε

(t, y, u) = `(t, y, u)

− ε log u.

It can be checked that the new cost function is strongly convex and lower semicontinuous. Thus, problem

(

SP)

ε

admits a unique solution

u

ε

. We denote respectively by

y

ε

:= y

u

ε

and

(p

ε

, q

ε

) := (p

u

state and adjoint state. Recalling the denition of

φ

ε

in (14), the SPMP yieldsthat

u

ε

(t, ω) = φ

ε

(

−p

ε

(t, ω)

− q

ε

(t, ω))

fora.a.

(t, ω)

∈ [0, T ] × Ω.

Moreover,withthehelpoftheSPMPagainincanbeprovedthat(seechapter 3 fordetails)

Proposition 8 There exist

C

00

_{> 0}

such that

u

ε

(t, ω)

≥

C

00

_ε

1 +

_|p

ε

(t, ω)

| + |q

ε

(t, ω)

|

for a.a.

(t, ω)

∈ [0, T ] × Ω.

Propositionabove and adualityargument yieldthe following errorestimate for the cost function.

Proposition 9 For every

ε > 0

, it holdsthat

J(u

ε

)

− J(u

0

)

≤ T ε.

Sketch of proof. Consider the Lagrangian

L : L

2

F

× L

2

F

→ R

dened as

L(u, λ) := J

0

(u)

− hλ, ui

2

.

The dual function

d :

U

ad

→ R

is given by

d(λ) := inf

u∈L

2

F

L(u, λ)

. Proposi-tion 8 and estimate(35) implythat

1/u

ε

∈ U

ad

. The SPMP, inits sucient formfor the convex case (see [31, Theorem 3.2]),implies that

d



ε

1

u

ε



= J

0

(u

ε

)

− εT.

Therefore, by weak duality

J

0

(u

ε

)

− εT ≤ max

λ∈U

ad

min

u∈L

2

F

L(u, λ) ≤ min

u∈L

2

F

max

λ∈U

ad

L(u, λ) = min

u∈U

ad

J

0

(u) = J

0

(u

0

).

2

The strong convexity of

J(

·)

and estimates (32), (35), easilyyield Theorem 10 For every

ε > 0

, the followingestimates hold

||u

ε

− u

0

||

2

2

+

||y

ε

− y

0

||

2

_2,∞

= O(ε)

0.2.3.2 Optimalityconditionsinstochasticoptimal controltheory The results presented here are studied in report [27]. In this section we consider the followingstochastic optimalcontrolproblem

Min J(u) := E

hR

₀

T

`(t, y

u

(t), u(t))dt + φ(y

u

(T ))

i

subjectto

u

∈ U.

(

_SP)

Inthe notationabove

U ⊆ L

2

F

isanonemptyclosed,convexset and

y

u

isthe unique solutionof the followingSDE

dy(t) = f (t, y(t), u(t))dt + σ(t, y(t), u(t))dW (t),

y(0) = x

0

.

(38)

Precise assumptions over the data of

(

SP)

are specied in Chapter 4. Let usnotice that the structure of

(

SP)

diers slightly to that of

(

SP)

0

, in the sense that inthe former the controlvariablebelongs to a Banachspace and itisconstrained tobeina generalclosed,convex set of

L

2

F

. This framework contains inparticular the case of convex global and localconstraints.

In this work we present rst and second-order necessary conditions for a local optimum

u

¯

of

(

SP)

. The main idea is to analyze the behavior of

J

underperturbationsof

u

¯

in

L

∞

F

, dened as

L

∞

_F

:=

_{{v : [0, T ] × Ω → R}

m

/ v

isprog. measurable and

||v||

∞

<

∞} ,

where

||v||

∞

:=

ess sup

{|v(t, ω)|, (t, ω) ∈ [0, T ] × Ω} .

Thus, insome sense, the perturbations considered in this work are more regular than the solution itself. From now on we x a localsolution

u

¯

and wedenoteby

y

¯

itsassociatedstate. As before,

(¯

p, ¯

q)

isdened astheunique solutionof (34). We set (recall (36))

H

u

(t) := H

u

(t, ¯

y(t), ¯

u(t), ¯

p(t), ¯

q(t))

and dene

Υ

1

: L

∞

F

→ R

as

Υ

1

(v) := E

Z

T

0

H

u

(t) v(t)dt



.

(39)

Using a generalization of estimate (32) and some technical computations (that take intoaccount a rst order linearizationof the state), we obtain: Proposition 11 Let

v

∈ L

∞

F

. Then, the following rst order expansion of

J

around

u

¯

holds

J(¯

u + v) = J(¯

u) + Υ

1

(v) + r

1

(v)

where

Υ

1

(v) = O(

||v||

2

)

and

r

1

(v) = O(

||v||

2

∞

).

The radial and tangent cone to

U

at

u

¯

are dened respectively by

R

U

(¯

u) :=

{v ∈ L

2

F

;

∃ σ > 0

such that

[¯

u, ¯

u + σv]

⊆ U},

T

_U

(¯

u) :=

{v ∈ L

2

F

;

∃ u(σ) = ¯u + σv + o(σ) ∈ U, σ ≥ 0, ||o(σ)/σ||

2

→ 0}.

For a subset

A

⊆ L

2

F

we write adh

2

(A)

for the adherence of

A

in

L

2

F

. It is wellknown, since

U

isclosedand convex, that

T

U

(¯

u) =

adh

2

(

R

U

(¯

u))

. Letus assume that forevery

u

∈ U

T

_U

(u) =

adh

2

(

R

U

(u)

∩ L

∞

F

) .

(40)

Remark 12 Assumption (40) is satised, for example,by constraint sets

U

which are stable under some truncation processes.

Estimate

Υ

1

(v) = O(

||v||

2

)

inproposition11 impliesthat thelinear form

Υ

1

canbeextendedcontinuouslyto

L

2

F

. Henceforth,proposition11thefollowing rst order necessary condition holds

Proposition 13 Assume that (40) holds and let

u

¯

be a local solution of

(

_SP)

. Then

Υ

1

(v)

≥ 0

for all

v

∈ T

U

(¯

u).

(41)

In order to obtain second-order necessary conditions, a second-order lin-earization of the state variable, detailed in Chapter 3, is considered. In our main results we willneed that at least one of the following assumptions holds:

(A1) It holds that

σ

uu

≡ 0

and the followingmaps are Lipschitz

(u, y)

∈ R

m

_{× R}

n

_{→ `(u, y) ∈ R, y ∈ R}

n

_{→ φ(y) ∈ R.}

(A2) It holds that the followingmaps are ane

(u, y)

∈ R

m

_{× R}

n

_{→ f(u, y) ∈ R}

n

_{, (u, y)}

_{∈ R}

m

_{× R}

n

_{→ σ(u, y) ∈ R}

n×d

_.

Letusset

H

(y,u)

2

(t) = H

(y,u)

2

(t, ¯

y(t), ¯

u(t), ¯

p(t), ¯

q(t))

anddene

Υ

2

: L

∞

F

→

R

by

Υ

2

(v) := E

Z

T

0

H

(y,u)

2

(t)(v(t), y

₁

(t))

2

dt + φ

_yy

(¯

y(T ))(y

₁

(T ))

2



,

where

y

1

= y

1

(v)

is dened as the unique solution of the following SDE

dy

1

(t) = Df (t)(y

1

(t), v(t))dt + Dσ(t)(y

1

(t), v(t))dW (t),

y

1

(0) = 0.

(42)

In the aboveexpression

Df (t) := Df ((t, ¯

y(t), ¯

u(t)))

, similarlynotation hold for

Dσ

. Again, technical computations yield the following second-order ex-pansionfor

J

around

u

¯

(see corollary 97).

Proposition 14 Assume that either (A1) or (A2) holds. Then, the fol-lowingexpansion holds:

J(¯

u + v) = J(¯

u) + Υ

1

(v) +

1

₂

Υ

2

(v) + r

2

(v)

for all

v

∈ L

∞

F

.

(43) where

Υ

1

(v) = O(

||v||

2

)

,

Υ

2

(v) = O(

||v||

2

2

)

and

r

2

(v) = O(

||v||

∞

||v||

2

2

)

. Usingthisexpansion, second-order necessary conditions canbe obtained un-derageneralizationofassumption(40),tothesecond-order case,and assum-ingthat

U

is polyhedric. For a precise statement of this result, we refer the readertotheorem106 and corollary109inChapter 3. However, forthe sake of completeness let us state second-order necessary conditions in the scalar box constraint case, i.e. when

U =



v

∈ L

2

F

/ a

≤ v(t, ω) ≤ b,

for a.a.

(t, ω)

∈ [0, T ] × Ω

.

(44)

Proposition 15 Let

u

¯

bealocalsolutionof

(

SP)

where

U

isdenedin (44). Suppose that either (A1) or (A2) holds. Then, the following second-order necessary conditionshold at

u

¯

:

Υ

2

(v)

≥ 0,

for all

v

∈ C(¯u),

where

C(¯

u) =

{v ∈ T

U

(¯

u) / H

u

(t)v(t, ω) = 0,

if

u(t, ω)

∈ {a, b}} .

Finally, let us mention that proposition 14 directly implies (see proposi-tion 110) a second-order sucient condition for the unconstrained case, i.e. when

U = L

2

F

. However, for the constrained case only very partial re-sults are obtained. The main diculty lies in the fact that the application

u

_{∈ L}

2

F

→ y

u

(T )

∈ L

2

(Ω)

is not weakly continuous. Thisfact is proved with two counterexamples (even in the case when

σ

u

≡ 0

) in section 3.5. Thus, the interesting question of characterizing

Υ

2

in order to obtain a non-gap second-order sucient condition remainsopen.

Part II

Asymptotic expansions for

interior penalty solutions of

Chapter 1

Optimal control of a linear

dierential equation

Contents

1.1 Introduction . . . 32 1.2 Problem statement and preliminary results . . . . 33 1.2.1 Mainproblem . . . 34 1.2.2 Penalized problems . . . 35 1.3 Interiorpenaltyanalysisinthenitedimensional

setting . . . 39 1.3.1 Convergencepropertiesoftheapproximate

projec-tors . . . 40 1.3.2 Stratication results and strict complementarity

reformulations . . . 42 1.4 Main results. . . 48 1.4.1 Errorestimates for interiorpenalties . . . 49 1.4.2 Asymptotic expansion . . . 55 1.5 Examples . . . 56 1.5.1 Decoupledcase:

R(t)

≡ I

. . . 56 1.5.2 Coupledcase:

R(t)

 0

. . . 59