HAMILTON–JACOBI EQUATIONS FOR OPTIMAL CONTROL ON JUNCTIONS AND NETWORKS

DOI:10.1051/cocv/2014054 www.esaim-cocv.org

HAMILTON–JACOBI EQUATIONS FOR OPTIMAL CONTROL ON JUNCTIONS AND NETWORKS

Yves Achdou

, Salom´ e Oudet

and Nicoletta Tchou

Abstract. We consider continuous-state and continuous-time control problems where the admissible trajectories of the system are constrained to remain on a network. A notion of viscosity solution of Hamilton–Jacobi equations on the network has been proposed in earlier articles. Here, we propose a simple proof of a comparison principle based on arguments from the theory of optimal control. We also discuss stability of viscosity solutions.

Mathematics Subject Classification. 34H05, 49J15.

Received February 13, 2014. Revised September 26, 2014 Published online May 20, 2015.

1. Introduction

A network (or a graph) is a set of items, referred to as vertices (or nodes/crosspoints), with connections between them referred to as edges. In the recent years there has been an increasing interest in the investigation of dynamical system and differential equation on networks, in particular in connection with problem of data transmission and traffic management (see for example Garavello–Piccoli [12], Engel et al. [9]). While control problems with state constrained in closures of open sets are well studied [8,16,22,23] there is to our knowledge much fewer literature on problems on networks. The results of Frankowska and Plaskacz [10,11] do apply to some closed sets with empty interior, but not to networks with crosspoints (except in very particular cases).

The literature on continuous-state and continuous-time control on networks is recent: the first two articles were published in 2012: control problems whose dynamics is constrained to a network and related Hamilton–

Jacobi equations were studied in [1]: a Hamilton–Jacobi equation on the network was proposed, with a definition of viscosity solution, which reduces to the usual one if the network is a straight line (i.e. is composed of two parallel edges sharing an endpoint) and if the dynamics and cost are continuous; while in the interior of an edge, one can test the equation with a smooth test-function, the main difficulties arise at the vertices where the network does not have a regular differential structure. At a vertex, a notion of derivative similar to that of

Keywords and phrases.Optimal control, networks, Hamilton–Jacobi equations, viscosity solutions.

∗The authors were partially funded by the ANR project ANR-12-BS01-0008-01.

∗∗ The first author was partially funded by the ANR project ANR-12-MONU-0013.

1 University Paris Diderot, Sorbonne Paris Cit´e, Laboratoire Jacques-Louis Lions, UMR 7598, UPMC, CNRS, 75205 Paris, France.[email protected]

2 IRMAR, Universit´e de Rennes 1, Rennes, France.

3 IRMAR, Universit´e de Rennes 1, Rennes, France.[email protected]

Article published by EDP Sciences c EDP Sciences, SMAI 2015

Dini’s derivative (see for example [2]) was proposed: admissible test-functions are continuous functions whose restriction to each edge is C¹. With this definition, the intrinsic geodesic distance, fixed one argument, is an admissible test-function with respect to the other argument. The Hamiltonian at a vertex depends on all directional derivatives in the directions of the edges containing the vertex (see Sect.3.5) below. Independently, Imbert, Monneau and Zidani [14] proposed an equivalent notion of viscosity solution for studying a Hamilton–

Jacobi approach to junction problems and traffic flows. There is also the work by Schieborn and Camilli [21], in which the authors focus on eikonal equations on networks and on a less general notion of viscosity solution.

Both [1,14] contain the first comparison and uniqueness results: in [1], suitably modified geodesic distances are used in the doubling variables method for proving comparison theorems under rather strong continuity assump- tions. In [14], Imbert, Monneau and Zidani used a completely different argument based on the explicit solution of a related optimal control problem, which could be obtained because it was assumed that the Hamiltonians associated with each edge did not depend on the state variable.

A general comparison result has finally been obtained in the quite recent paper by Imbert–Monneau [13]. In the latter article, the Hamiltonians in the edges are completely independent from each other; the main assumption is that the Hamiltonian in each edge, sayH_i(x, p) for the edge indexedi, is bimonotone,i.e.non increasing (resp.

non decreasing) forpsmaller (resp. larger) than a given thresholdp⁰_i(x). Of course, convex Hamiltonian coming from optimal control theory are bimonotone. Moreover, [13] handles more general transmission conditions than the previous articles, with an additional running cost at the junctions. In [13], the proof of the comparison result is rather involved and only uses arguments from the theory of partial differential equations: in the most simple case where all the Hamiltonians related to the edges are strictly convex and reach their minima atp= 0, the idea consists of doubling the variables and using a suitable test-function; then, in the general case, perturbation arguments are used for applying the results proved in the former case.

In coincidence with these research efforts about networks, Barles, Briani and Chasseigne, see [3,4], have recently studied control problems with discontinuous dynamics and costs, obtaining comparison results for some Bellman equations arising in this context, with original and elegant arguments. Related problems were also recently addressed by Raoet al.[19,20].

The aim of the present paper is to focus on optimal control problems with independent dynamics and running costs in the edges, and to show that the arguments in [3] can be adapted to yield a simple proof of a comparison result.

Sections2 to5are devoted to the case of a junction,i.e. a network with one vertex only. Section2contains a description of the geometry and of the optimal control problem. In Section 3, a Hamilton–Jacobi equation is proposed for the value function, together with a notion of viscosity solution. It is proved that the value function is a viscosity solution of the Hamilton–Jacobi equation. Also in Section3, Lemma3.6on the structure of the Hamiltonian at the vertex will be important for obtaining the comparison principle. Some important properties of viscosity sub and supersolutions are given in Section4, and the comparison principle is proved in Section 5. In Section6 we discuss the stability of the viscosity sub and super solution under perturbations of the Hamiltonians. In Section7we show that all the results can be easily extended to the case when there is an additional cost at the junction. Finally, in Section 8, the results obtained for the junction are generalized for networks with more than one vertices.

2. The junction

2.1. The geometry

Let us focus on the model case of a junction inR^d withN semi-infinite straight edges,N >1. The edges are denoted by (J_i)_i=1,...,N. The edgeJ_i is the closed half-line R⁺e_i. The vectors e_i are two by two distinct unit vectors inR^d. The half-lines J_i are glued at the originO to form the junctionG:

G= N i=1

J_i.

The geodetic distance d(x, y) between two pointsx, y ofG is d(x, y) =

|x−y| ifx, y belong to the same edgeJ_i

|x|+|y| ifx, y belong to different branchesJ_i andJ_j.

2.2. The optimal control problem

We consider infinite horizon optimal control problems which have different dynamics and running costs in the edges. We are going to describe the assumptions on the dynamics and costs in each edge J_i. The sets of controls are denoted byA_i and the system is driven by a dynamicsf_i and the running cost is given by_i. Our main assumptions are as follows:

[H0]Ais a metric space (one can takeA=R^m). Fori= 1, . . . , N,A_iis a non empty compact subset ofAand f_i :J_i×A_i →Ris a continuous bounded function. The sets A_i are disjoint. Moreover, there exists L >0 such that for anyi,x, y inJ_i anda∈A_i,

|fi(x, a)−f_i(y, a)| ≤L|x−y|.

We will use the notationF_i(x) for the set{fi(x, a)e_i, a∈A_i}.

[H1]Fori= 1, . . . , N, the function_i:J_i×Ai→Ris a continuous and bounded function. There is a modulus of continuityω_isuch that for allx,y inJ_i and for alla∈A_i, |i(x, a)−_i(y, a)| ≤ω_i(|x−y|).

[H2]Fori= 1, . . . , N,x∈J_i, the non empty and closed set

FL_i(x)≡ {(fi(x, a)e_i, _i(x, a)), a∈A_i} is convex.

[H3]There is a real numberδ >0 such that for anyi= 1, . . . , N, [−δe_i, δe_i]⊂F_i(O).

Remark 2.1. In [H0] the assumption that the setsA_i are disjoint is not restrictive: it is made only for sim- plifying the proof of Theorem 2.3 below. Indeed, if A_i are not disjoint, then we define ˜A_i = {i} ×A_i and f˜_i(x,a) =˜ f_i(x, a), ˜_i(x,a) =˜ _i(x, a) ifx∈J_i and ˜a= (i, a) witha∈A_i.

The sets ˜A_i are disjoint compact subsets of ˜A=∪i=1,...,nA˜_i which is a (compact) metric space for the distance d((i, a),˜ (j, b)) =|i−j|+d_A(a, b), and the functions ˜f_i, ˜_i inherit the properties of f_i and_i.

The assumption [H2] is not essential: it is made in order to avoid the use of relaxed controls.

Assumption [H3] is a strong controllability condition at the vertex (which implies the coercivity of the Hamiltonian). It has already been widely used in the framework of networks (for instance, the same assumption is made in [1,3], and the coercivity of the Hamiltonian is assumed [13]). We will see that [H3] yields the continu- ity of the value function. Without any controllability condition, the value function may not be continuous and the definition of the viscosity solutions should differ from the one proposed below. There are of course milder controllabilty conditions, but with them, our techniques do not seem to apply in a straightforward manner.

Here is a general version of Filippov implicit function lemma, see [17], which will be useful to prove Theorem2.3 below.

Theorem 2.2. Let I be an interval of R and γ : I → R^d×R^d be a measurable function. Let K be a closed subset of R^d×A and Ψ :K→R^d×R^d be continuous. Assume that γ(I)⊂Ψ(K), then there is a measurable function Φ:I→K with

Ψ◦Φ(t) =γ(t) for a.a. t∈I.

Proof. See [17].

Let us denote byM the set:

M =

(x, a); x∈ G, a∈A_i ifx∈J_i\{O}, anda∈ ∪^N_i=1A_i ifx=O

. (2.1)

The setM is closed. We also define the function f onM by

∀(x, a)∈M, f(x, a) =

f_i(x, a)e_i ifx∈J_i\{O}, f_i(O, a)e_i ifx=O anda∈A_i. The functionf is continuous onM because the setsA_i are disjoint. Let ˜F(x) be defined by

F˜(x) =

F_i(x) ifxbelongs to the edgeJ_i\{O}

∪^N_i=1F_i(O) ifx=O.

Forx∈ G, the set of admissible trajectories starting from xis Y_x=

y_x∈Lip(R⁺;G) :

y˙_x(t)∈F˜(y_x(t)), for a.a.t >0, y_x(0) =x,

. (2.2)

Theorem 2.3. Assume[H0], [H1], [H2] and[H3]. Then 1. For anyx∈ G,Y_x is non empty.

2. For anyx∈ G, for each trajectory y_x in Y_x, there exists a measurable function Φ: [0,+∞)→M,Φ(t) = (ϕ₁(t), ϕ₂(t))with

(y_x(t),y˙_x(t)) = (ϕ₁(t), f(ϕ₁(t), ϕ₂(t))), for a.e.t, which means in particular thaty_x is a continuous representation ofϕ₁

3. Almost everywhere in[0,+∞),

˙ y_x(t) =

N i=1

1_{{yx(t)∈Ji\{O}}}f_i(y_x(t), ϕ₂(t))e_i. 4. Almost everywhere on{t:y_x(t) =O},f(O, ϕ₂(t)) = 0.

Proof. The proof of point 1 is easy, because 0∈F˜(O).

The proof of point 2 is a consequence of Theorem 2.2, with K =M, I = [0,+∞), γ(t) = (y_x(t),y˙_x(t)) and Ψ(x, a) = (x, f(x, a)).

From point 2, we deduce that

˙ y_x(t) =

N i=1

1_{{yx(t)∈Ji\{O}}}f_i(y_x(t), ϕ₂(t))e_i+ 1_{yx(t)=O}}f(O, ϕ₂(t)),

and from Stampacchia’s theorem,f(O, ϕ₂(t)) = 0 almost everywhere in {t: y_x(t) =O}. This yields points 3

and 4.

It is worth noticing that in Theorem2.3, a solutiony_xcan be associated with several control lawsϕ₂(·). We introduce the set of admissible controlled trajectories starting from the initial datumx:

Tx=

⎧⎨

⎩

(y_x, α)∈L^∞_Loc(R⁺;M) : y_x∈Lip(R⁺;G), y_x(t) =x+

_t

0

f(y_x(s), α(s))ds inR⁺

⎫⎬

⎭. (2.3)

Remark 2.4. If two different edges are aligned with each other, say the edgesJ₁ andJ₂, many other assump- tions can be made on the dynamics and costs:

• a trivial case in which the assumptions [H1]-[H3] are satisfied is when the dynamics and costs are continuous at the origin,i.e.A₁=A₂;f₁andf₂are respectively the restrictions toJ₁×A₁andJ₂×A₂of a continuous and bounded functionf_1,2defined inRe1×A1, which is Lipschitz continuous with respect to the first variable;

₁and₂are respectively the restrictions toJ₁×A₁ andJ₂×A₂ of a continuous and bounded function_1,2 defined inRe1×A₁.

• In this particular geometrical setting, one can allow some mixing (relaxation) at the vertex with several possible rules: More precisely, in [3,4], Barles et al. introduce several kinds of trajectories which stay at the junction: theregular trajectories are obtained by mixing outgoing dynamics from J₁ andJ₂, whereas singulartrajectories are obtained by mixing strictly ingoing dynamics from J₁ and J₂. Two different value functions are obtained whether singular mixing is permitted or not.

The cost functional. The cost associated to the trajectory (y_x, α)∈ Tx is J(x; (y_x, α)) =

_∞

0

(y_x(t), α(t))e^−λtdt, (2.4)

whereλ >0 is a real number and the Lagrangianis defined onM by

∀(x, a)∈M, (x, a) =

_i(x, a) ifx∈J_i\{O}, _i(O, a) ifx=O anda∈A_i. The value function. The value function of the infinite horizon optimal control problem is

u(x) = inf

(yx,α)∈Tx

J(x; (y_x, α)). (2.5)

Proposition 2.5. Assume[H0],[H1], [H2]and[H3]. Then the value functionvis bounded and continuous onG. Proof. The proof essentially uses Assumption [H3]. Since it is classical, we skip it.

3. The Hamilton–Jacobi equation

3.1. Test-functions

For the definition of viscosity solutions on the irregular set G, it is necessary to first define a class of the admissible test-functions

Definition 3.1. A functionϕ:G →Ris an admissible test-function if

• ϕis continuous in G

• for anyj,j= 1, . . . , N,ϕ|Jj ∈ C¹(J_j).

The set of admissible test-functions is noted R(G). If ϕ ∈ R(G) and ζ ∈ R, let Dϕ(x, ζe_i) be defined by Dϕ(x, ζe_i) =ζ_dx^dϕ

i(x) ifx∈J_i\{O}and Dϕ(O, ζe_i) =ζlim_{h→0+ dx}^dϕ

i(he_i).

Property 3.2. Ifϕ=g◦ψwith g∈ C¹ andψ∈ R(G), then ϕ∈ R(G)and Dϕ(O, ζ) =g(ψ(O))Dψ(O, ζ).

3.2. Vector fields

Fori= 1, . . . , N, we denote byF_i⁺(O) and FL⁺_i(O) the sets

F_i⁺(O) =F_i(O)∩R⁺e_i, FL⁺_i (O) = FL_i(O)∩(R⁺e_i×R),

which are non empty thanks to assumption [H3]. Note that 0∈ ∩^N_i=1F_i(O). From assumption [H2], these sets are compact and convex. Forx∈ G, the sets F(x) and FL(x) are defined by

F(x) =

F_i(x) ifxbelongs to the edgeJ_i\{O}

∪i=1,...,NF_i⁺(O) ifx=O, and

FL(x) =

FL_i(x) ifxbelongs to the edgeJ_i\{O}

∪i=1,...,NFL⁺_i (O) ifx=O.

3.3. Definition of viscosity solutions

We now introduce the definition of a viscosity solution of λu(x) + sup

(ζ,ξ)∈FL(x){−Du(x, ζ)−ξ}= 0 inG. (3.1) Definition 3.3. • An upper semi-continuous function u : G → R is a subsolution of (3.1) in G if for any

x∈ G, anyϕ∈ R(G) s.t.u−ϕhas a local maximum point atx, then λu(x) + sup

(ζ,ξ)∈FL(x){−Dϕ(x, ζ)−ξ} ≤0; (3.2)

• A lower semi-continuous functionu:G →Ris a supersolution of (3.1) if for any x∈ G, anyϕ∈ R(G) s.t.

u−ϕhas a local minimum point atx, then λu(x) + sup

(ζ,ξ)∈FL(x){−Dϕ(x, ζ)−ξ} ≥0; (3.3)

• A continuous functionu:G →Ris a viscosity solution of (3.1) inG if it is both a viscosity subsolution and a viscosity supersolution of (3.1) in G.

Remark 3.4. Atx∈J_i\{O}, the notion of sub, respectively super-solution in Definition3.3 is equivalent to the standard definition of viscosity sub, respectively super-solution of

λu(x) + sup

a∈Ai

{−f_i(x, a)·Du(x)−_i(x, a)}= 0.

3.4. Hamiltonians

We define the HamiltoniansH_i:J_i×R→Rby H_i(x, p) = max

a∈Ai(−pfi(x, a)−_i(x, a)) (3.4)

and the HamiltonianH_O :R^N →Rby H_O(p₁, . . . , p_N) = max

i=1,...,N max

a∈Ai s.t.fi(O,a)≥0(−pif_i(O, a)−_i(O, a)). (3.5) We also define what may be called the tangential Hamiltonian atOby

H_O^T =− min

i=1,...,N min

a∈Ais.t.fi(O,a)=0

_i(O, a). (3.6)

Thanks to the definitions of FL_i(x) (in particular of FL⁺_i(O)), the continuity properties of the data and the compactness ofA_i, one easily notes that the following definition is equivalent to Definition3.3.

Definition 3.5. • An upper semi-continuous function u : G → R is a subsolution of (3.1) in G if for any x∈ G, anyϕ∈ R(G) s.t.u−ϕhas a local maximum point atx, then

λu(x) +H_i

x,_dx^dϕ

i(x)

≤0 ifx∈J_i\{O}, λu(O) +H_O

dϕ

dx1(O), . . . ,_dx^dϕ

N(O)

≤0. (3.7)

• A lower semi-continuous functionu:G →Ris a supersolution of (3.1) if for any x∈ G, anyϕ∈ R(G) s.t.

u−ϕhas a local minimum point atx, then λu(x) +H_i

x,_dx^dϕ

i(x)

≥0 ifx∈J_i\{O}, λu(O) +H_O

dϕ

dx1(O), . . . ,_dx^dϕ

N(O)

≥0. (3.8)

The Hamiltonian H_i are continuous with respect to x ∈ J_i, convex with respect to p. Moreover p → H_i(O, p) is coercive,i.e.lim_|p|→+∞H_i(O, p) = +∞from the controlability assumption [H3]. Following Imbert–

Monneau [13], we introduce the nonempty compact intervalP₀ⁱ P₀ⁱ ={pⁱ₀∈Rs.t.H_i(O, pⁱ₀) = min

p∈RH_i(O, p)}. (3.9)

Lemma 3.6. Assume[H0], [H1], [H2] and[H3], then

1. pⁱ₀∈ P₀ⁱ if and only if there existsa^∗∈A_isuch thatf_i(O, a^∗) = 0andH_i(O, pⁱ₀) =−pⁱ₀f_i(O, a^∗)−i(O, a^∗) =

−i(O, a^∗) 2.

minp∈RH_i(O, p) =− min

a∈Ai s.t. fi(O,a)=0_i(O, a) (3.10) 3. For allp∈R, if p≥pⁱ₀ for somepⁱ₀∈ P₀ⁱ then

a∈Ai s.t.maxfi(O,a)≥0(−pf_i(O, a)−_i(O, a)) = min

q∈RH_i(O, q) =− min

a∈Ai s.t.fi(O,a)=0

_i(O, a).

Proof. The Hamiltonian H_i reaches its minimum at pⁱ₀ if and only if 0 ∈ ∂H_i(O, pⁱ₀). The subdifferential of H_i(O,·) atpⁱ₀ is characterized by

∂H_i(O, pⁱ₀) = co{−f_i(O, a);a∈A_i s.t.H_i(O, pⁱ₀) =−pⁱ₀f_i(O, a)−_i(O, a)}, see [24]. But from [H2],

{(fi(O, a), _i(O, a));a∈A_i s.t.H_i(O, pⁱ₀) =−pⁱ₀f_i(O, a)−_i(O, a)}

is compact and convex. Hence,

∂H_i(O, pⁱ₀) ={−fi(O, a);a∈A_i s.t.H_i(O, pⁱ₀) =−pⁱ₀f_i(O, a)−_i(O, a)}.

Therefore, 0∈∂H_i(O, pⁱ₀) if and only if there existsa^∗∈A_isuch thatf_i(O, a^∗) = 0 andH_i(O, pⁱ₀) =−_i(O, a^∗).

We have proved point 1.

Point 2 is a direct consequence of point 1.

Ifpis greater than or equal to somepⁱ₀∈ P₀ⁱ, then

a∈Ai:fmaxi(O,a)≥0(−pfi(O, a)−_i(O, a))≤ max

a∈Ai:fi(O,a)≥0(−pⁱ₀f_i(O, a)−_i(O, a)) =H_i(O, pⁱ₀) where the last identity comes from point 1.

On the other hand,

a∈Ai:fmaxi(O,a)≥0(−pfi(O, a)−_i(O, a))≥ − min

a∈Ai:fi(O,a) = 0_i(O, a).

Point 3 is obtained by combining the two previous observations and point 2.

0

−2 −1 1 2 3 4

3

− 5

0 10

2 4 6 8 12 14 16

1 3 5 7 9 11 13 15

Figure 1. The graphs of the Hamiltonian p → H_i(O, p) (with the circles) and of p → H_i⁺(O, p) ≡ max_{a∈Ai:fi(O,a)≥0}(−pfi(O, a)−_i(O, a)) (with the signs +). In the example, P₀ⁱ= [0,2].

Remark 3.7. It can also be proved thatp≤max(q∈ P₀ⁱ) if and only if H_i(O, p) = max

a∈Ai:fi(O,a)≥0(−pf_i(O, a)−_i(O, a)).

In Figure 1, we give an example for the graphs of p → H_i(O, p) and of p → H_i⁺(O, p) ≡ max_{a∈Ai:fi(O,a)≥0}(−pfi(O, a)−_i(O, a)), and the related intervalP₀ⁱ.

3.5. Existence

Theorem 3.8. Assume[H0], [H1], [H2] and[H3]. The value functionv defined in(2.5)is a bounded viscosity solution of (3.1)in G.

The proof of Theorem 3.8is made in several steps, namely Proposition 3.9and Lemmas3.10and3.11 below:

the first step consists of proving that the value function is a viscosity solution of a Hamilton–Jacobi equation with a more general definition of the Hamiltonian: for that, we introduce larger relaxed vector fields: forx∈ G,

f˜(x) =

⎧⎨

⎩η ∈R^d:

∃(y_x,n, α_n)_n∈N, (y_x,n, α_n)∈ Tx,

∃(tn)_n∈N

s.t.

t_n→0⁺and

n→∞lim 1 t_n

_tn

0

f(y_x,n(t), α_n(t))dt=η

⎫⎬

⎭ and

f(x) =

⎧⎪

⎪⎪

⎪⎨

⎪⎪

⎪⎪

⎩

(η, μ)∈R^d×R:

∃(y_x,n, α_n)_n∈N, (y_x,n, α_n)∈ Tx,

∃(tn)_n∈N

s.t.

t_n→0⁺,

n→∞lim 1 t_n

_tn

0

f(y_x,n(t), α_n(t))dt=η,

n→∞lim 1 t_n

_tn

0

(y_x,n(t), α_n(t))dt=μ

⎫⎪

⎪⎪

⎪⎬

⎪⎪

⎪⎪

⎭ .

Proposition 3.9. Assume[H0], [H1], [H2] and[H3]. The value function v defined in (2.5) is a viscosity solu- tion of

λu(x) + sup

(ζ,ξ)∈f(x)

{−Du(x, ζ)−ξ}= 0 inG, (3.11) where the definition of viscosity solution is exactly the same as Definition3.3, replacingFL(x)with f(x).

Proof. See [1].

For all ϕ∈ R(G), it is clear that if x∈J_i\{O}, thenH_i(x, Dϕ) = sup_{(ζ,ξ)∈f(x)} {−Dϕ(x, ζ)−ξ}. We are left with comparing sup(ζ,ξ)∈FL(O){−Dϕ(O, ζ)−ξ}and sup_{(ζ,ξ)∈f(O)} {−Dϕ(O, ζ)−ξ}. The two quantities are the same. This is a consequence of the following lemma

Lemma 3.10.

f(O) =

i=1,...,N

co

⎧⎨

⎩FL⁺_i(O)∪

j=i

FL_j(O)∩({0} ×R)⎫

⎬

⎭.

Proof. The proof being a bit long, we postpone it to the appendix.

Lemma 3.11. Assume[H0], [H1], [H2]and[H3]. For any function ϕinR(G), sup

(ζ,ξ)∈f(O)

{−Dϕ(O, ζ)−ξ}= max

(ζ,ξ)∈FL(O){−Dϕ(O, ζ)−ξ}. (3.12) Proof. It was proved in [1] that FL(O)⊂f(O). Hence

(ζ,ξ)∈FL(O)max {−Dϕ(O, ζ)−ξ} ≤ sup

(ζ,ξ)∈f(O)

{−Dϕ(O, ζ)−ξ}.

From the piecewise linearity of the function (ζ, μ)→ −Dϕ(O, ζ)−μ, we infer that sup

(ζ,μ)∈co

FL⁺_i(O)∪

j=i

FLj(O)∩({0}×R)

(−Dϕ(O, ζ)−μ)

= max

(ζ,μ)∈FLmax⁺_i(O)(−Dϕ(O, ζ)−μ),max

j=i max

(0,μ)∈FLj(O)−μ

≤max_j=1,...,Nmax_(ζ,μ)∈FL+

j(O)−Dϕ(O, ζ)−μ) = max(ζ,ξ)∈FL(O){−Dϕ(O, ζ)−ξ}.

We conclude by using Lemma 3.10.

4. Properties of viscosity sub and supersolutions

In this part, we study sub and supersolutions of (3.1), transposing ideas coming from Barles-Briani- Chasseigne [3,4] to the present context.

Lemma 4.1. Assume[H0], [H1], [H2]and[H3]. LetR be a positive real number such that for all i= 1, . . . , N

andx∈B(O, R)∩J_i

−δ 2e_i,δ

2e_i

⊂F_i(x).

For any bounded viscosity subsolution u of (3.1), there exists a constant C^∗ > 0 such that u is a viscosity subsolution of

|Du(x)| ≤C^∗ inB(O, R)∩ G,

i.e. for anyx∈B(O, R)∩ G andϕ∈ R(G) such thatu−ϕhas a local maximum point atx, dϕ

dx_i(x)

≤C^∗ if x∈B(O, R)∩J_i\{O} (4.1)

mini

dϕ

dx_i(O)≥ −C^∗ if x=O. (4.2)

Proof. LetM_u(respM) be an upper bound on|u|(resp._j for allj= 1, . . . , N). The viscosity inequality (3.7) yields that

H_i

x, dϕ dx_i(x)

≤λM_u ifx∈B(O, R)∩J_i\{O}, (4.3) H_O

dϕ

dx₁(O), . . . , dϕ dx_N(O)

≤λM_u ifx=O. (4.4)

From the controlability inB(O, R)∩J_i, we see thatH_iis coercive with respect to its second argument uniformly in x∈B(O, R)∩J_i, and more precisely thatH_i(x, p)≥^δ₂|p| −M.

Thus, from (4.3), there exists a constantC^∗= 2^λMu_δ^+M such that |_dx^dϕ_i(x)| ≤C^∗ ifx∈B(O, R)∩J_i\{O}.

Ifx=O, we use the fact thatH_i⁺(O, p)≥^δ₂max(0,−p)−M. The viscosity inequality (4.4) then yields that min_idx^dϕ

i(O)≥ −C^∗.

Lemma 4.2. Assume[H0], [H1], [H2] and [H3]. There exists a neighborhood ofO in G in which any bounded viscosity subsolutionuof (3.1)is Lipschitz continuous.

Proof. We adapt the proof of Ishii, see [15].

Take Ras in Lemma4.1, fix z∈B(O, R)∩ G and setr= (R− |z|)/4. Fix any y∈ G such that d(y, z)< r. It can be checked that for anyx∈ G, if d(x, y)<3rthen d(x, O)< R. Choose a functionf ∈ C¹([0,3r)) such that f(t) =t in [0,2r] andf(t)≥1 for allt∈[0,3r) and lim_t→3rf(t) = +∞. Fix anyε >0. We are going to show that

u(x)≤u(y) + (C^∗+ε)f(d(x, y)), ∀x∈ G such that d(x, y)<3r, (4.5) whereC^∗ is the constant in Lemma4.1.

Let us proceed by contradiction. Assume that (4.5) is not true. According to the properties off, the function x→u(x)−u(y)−(C^∗+ε)f(d(x, y)) admits a maximumξ∈B(y,3r)∩ G. However, from the fact (4.5) is not true, we deduce that ξ= y. Hence, it is possible to modify the function ψ :G → R, x→ (C +ε)f(d(x, y)) away from a neighborhood of ξ and obtain an admissible test function that we use in the viscosity inequality satisfied by u; from (4.1) and (4.2) in Lemma4.1and from explicit calculatiobns concerning the derivatives of d(x, y) at the pointξ, we obtain that

(C^∗+ε)f(d(ξ, y))≤C^∗, which leads to a contradiction.

If d(x, z)< rthen d(x, y)<2randf(d(x, y)) = d(x, y). In this case, (4.5) yields that u(x)≤u(y) + (C^∗+ε)d(x, y), ∀x, y∈ G s.t d(x, z)< r, d(y, z)< r.

By symmetry, we get

|u(x)−u(y)| ≤(C^∗+ε)d(x, y), ∀x, y∈ G s.t d(x, z)< r, d(y, z)< r, and by lettingεtend to zero:

|u(x)−u(y)| ≤C^∗d(x, y), ∀x, y ∈ Gs.t d(x, z)< r, d(y, z)< r. (4.6) Now, for two arbitrary pointsx, yinG ∩B(O, R), we taker=¹₄min(R−|x|, R−|y|) and choose a finite sequence (z_j)_j=1,...,M ∈ Gbelonging to the segment [x, y] ifxandy belong to someJ_i or to [O, x]∪[O, y] in the opposite case, and such thatz₁=x,z_M =y and d(z_i, z_i+1)< rfor alli= 1, . . . , M−1 and_M−1

i=1 d(z_i, z_i+1) = d(x, y).

From (4.6), we get that

|u(x)−u(y)| ≤C^∗d(x, y), ∀x, y∈ G ∩B(O, R).

Lemma 4.3. Assume[H0], [H1], [H2] and[H3]. Any bounded viscosity subsolutionuof (3.1)is such that

λu(O)≤ −H_O^T. (4.7)

Proof. Since, from Lemma 4.2, uis Lipschitz continuous in a neighborhood of O, we know that there exists a test-function ϕ in R(G) which touches u from above at O. Since u is a subsolution of (3.1), we see that λu(O) +H_O(_dx^dϕ

1(O), . . . ,_dx^dϕ

N(O))≤0, which implies thatλu(O) +H_O^T ≤0.

Remark 4.4. It is interesting to note that in [3] and [4], a condition similar to (4.7) is introduced to characterize a particular viscosity solution of the transmission problem studied there among all the possible solutions in the sense of Ishii, (this condition is not satisfied by all subsolutions).

In the present context, the fact that (4.7) is automatically satisfied by subsolutions seems to be linked to the richness of the spaceR(G): for any Lipschitz functionudefined in a neighborhood ofO, there existsϕ∈ R(G) such thatu−ϕhas a maximum atO.

The following lemma can be found in [3,4] in a different context:

Lemma 4.5. Letv:G →Rbe a viscosity supersolution of(3.1)inGandwbe a continuous viscosity subsolution of (3.1)inG. Then ifx∈J_i\{0}, we have for allt >0,

v(x)≥ inf

αi(·),θi

_t∧θi

0

_i(y_xⁱ(s), α_i(s))e^−λsds+v(y_xⁱ(t∧θ_i))e^{−λ(t∧θi)}

, (4.8)

and

w(x)≤ inf

αi(·),θi

_t∧θi

0

_i(y_xⁱ(s), α_i(s))e^−λsds+w(y_xⁱ(t∧θ_i))e^{−λ(t∧θi)}

, (4.9)

whereα_i∈L^∞(0,∞;A_i),y_xⁱ is the solution ofyⁱ_x(t) =x+_t

0f_i(y_xⁱ(s), α_i(s))ds

e_iandθ_iis such thaty_xⁱ(θ_i) = 0 andθ_i lies in[τ_i,τ¯_i], where τ_i is the exit time of yⁱ_x fromJ_i\{O} andτ¯_i is the exit time ofy_xⁱ from J_i.

Proof. See [3] for the detailed proof. We restrict ourselves to mentioning that the proof of (4.8) uses the results of Blanc [6,7] on the minimal supersolution of exit time control problems, whereas the proof of (4.9) uses the

comparison results of Barles–Perthame [5] and the continuity ofw.

The following theorem is reminiscent of Theorem 3.3 in [3]:

Theorem 4.6. Assume[H0], [H1], [H2] and[H3]. Letv:G →Rbe a viscosity supersolution of (3.1), bounded from below by −c|x| −C for two positive numbersc andC. Either[A] or[B] below is true:

[A]There existsη >0,i∈ {1, . . . , N} and a sequencex_k ∈J_i,lim_k→+∞x_k =O such that lim_k→+∞v(x_k) = v(O)and for each k, there exists a control law α^k_i such that the corresponding trajectory y_xk(s)∈J_i for all s∈[0, η] and

v(x_k)≥ _η

0

_i(y_xk(s), α^k_i(s))e^−λsds+v(y_xk(η))e^−λη (4.10) [B]

λv(O) +H_O^T ≥0. (4.11)

Proof. Let us assume that [B] does not hold.

For anyiin {1, . . . , N}, take for example

q_i= min

pⁱ₀∈P₀ⁱpⁱ₀,

andq= (q₁, . . . , q_N). From Lemma3.8,

H_O(q) =H_O^T. (4.12)

Consider the function

v(x)−q_i|x|+|x|²

ε² ifx∈J_i.

Standard arguments show that this function reaches its minimum nearO and any sequence of such minimum pointsx_εconverges toO and thatv(x_ε) converges tov(O).

It is not possible thatx_εbe O, because sincevis a viscosity supersolution of (3.1), we would have that λv(O) +H_O(q)≥0,

and thereforeλv(O) +H_O^T ≥0, which is a contradiction since [B] does not hold.

Therefore, there existsi∈ {1, . . . , N}such that, up to the extraction of a subsequence, x_ε∈J_i\{O}, for all ε. We can therefore apply Lemma4.5: for anyt >0,

v(x_ε)≥ inf

αi(·),θi

_t∧θi

0

_i(yⁱ_xε(s), α_i(s))e^−λsds+v(y_xⁱ_ε(t∧θ_i))e^{−λ(t∧θi)}

, (4.13)

whereyⁱ_xis the solution of yⁱ_x(t) =x+_t

0f_i(y_xⁱ(s), α_i(s))ds

e_i.

Take t = 1 for example. From [H0] and [H2], the minimum in (4.13) is reached for some α_i,ε and θ_i,ε >0, see [3]:

v(x_ε)≥ _1∧θi,ε

0

_i(yⁱ_xε(s), α_i,ε(s))e^−λsds+v(yⁱ_xε(1∧θ_i,ε))e^{−λ(1∧θi,ε)}. (4.14) Assume by contradiction that [A] does not hold, then lim_ε→0θ_i,ε = 0.

Sincex_εis a minimum ofv(x)−q_i|x|+^|x|_ε2², we deduce from (4.14) that 0≥

_θi,ε

0

_i(y_xⁱ_ε(s), α_i,ε(s))e^−λsds+v(yⁱ_xε(θ_i,ε))(e^−λθi,ε −1)−q_i|xε|+|xε|²

ε² , (4.15)

and therefore

0≥ _θi,ε

0

_i(yⁱ_xε(s), α_i,ε(s))e^−λsds+v(yⁱ_xε(θ_i,ε))(e^−λθi,ε−1)−q_i|xε|. (4.16) We can write (4.16) as

0≤ _θi,ε

0

−i(y_xⁱ_ε(s), α_i,ε(s))e^−λs−q_if_i(yⁱ_xε(s), α_i,ε(s)) ds−v(y_xⁱ_ε(θ_i,ε))(e^−λθi,ε−1). (4.17)

Dividing byθ_i,εand lettingεtend to 0, we obtain thatλv(O)+H_i(O, q_i)≥0. This implies thatλv(O)+H_O^T ≥0,

which is a contradiction since [B] does not hold.

5. Comparison principle and uniqueness

Theorem 5.1. Assume[H0], [H1], [H2]and[H3]. Letu:G →Rbe a bounded continuous viscosity subsolution of (3.1), andv:G →Rbe a bounded viscosity supersolution of(3.1). Thenu≤v inG.

Proof. It is a simple matter to check that there exists a positive real numberM such that the functionψ(x) =

−|x|²−M is a viscosity subsolution of (3.1). For 0< μ <1,μclose to 1, the functionu_μ=μu+ (1−μ)ψis a viscosity subsolution of (3.1), which tends to−∞as|x|tends to +∞. LetM_μ be the maximal value ofu_μ−v which is reached at some point ¯x_μ.

We want to prove thatM_μ≤0.

1. If ¯x_μ=O, then we introduce the functionu_μ(x)−v(x)−d²(x,x¯_μ), which has a strict maximum at ¯x_μ, and we double the variables,i.e.for 0< ε1, we consider

u_μ(x)−v(y)−d²(x,x¯_μ)−d²(x, y) ε² .

Classical arguments then lead to the conclusion thatu_μ(¯x_μ)−v(¯x_μ)≤0, thusM_μ≤0.

2. If ¯x_μ=O. We use Theorem4.6; we have two possible cases:

[B]λv(O)≥ −H_O^T.

From Lemma4.3,λu(O) +H_O^T ≤0. Therefore, we obtain thatu_μ(O)≤v(O), thusM_μ≤0.

[A]With the notations of Theorem 4.6, we have that v(x_k)≥

_η

0

_i(y_xk(s), α^k_i(s))e^−λsds+v(y_xk(η))e^−λη. Moreover, from Lemma4.5,

u_μ(x_k)≤ _η

0

_i(y_xk(s), α^k_i(s))e^−λsds+u_μ(y_xk(η))e^−λη. Therefore

u_μ(x_k)−v(x_k)≤(u_μ(y_xk(η))−v(y_xk(η)))e^−λη. Lettingk tend to +∞, we find thatM_μ≤M_μe^−λη, which implies thatM_μ≤0

We conclude by lettingμtend to 1.

Remark 5.2. Under the assumptions [H0], [H1], [H2] and [H3], it is possible to obtain the following more general comparison principle, see [18]:

Letu:G →Rbe a bounded viscosity subsolution of (3.1), andv:G →Rbe a bounded viscosity supersolution of (3.1). Thenu≤v in G.

The proof, given in [18] is more technical and is done two steps:

1. using Lemma 4.2, i.e. the Lipschitz continuity of any subsolution in a neighborhood of O, prove a local comparison principle

2. obtain the global comparison result by a localization argument.

Corollary 5.3. Assume[H0], [H1], [H2]and[H3]. The value functionuof the optimal control problem(2.5)is the unique bounded viscosity solution of (3.1).

6. Stability

We now study the stability of sub and super solutions with respect to the uniform convergence of the costs and dynamics.

6.1. Assumptions

We consider a family (indexed by ε ∈ [0,1]) of optimal control problems on the network whose dynamics and costs are denoted (f_i^ε, ^ε_i) for i = 1, . . . , N. As above, A is a metric space (one can take A = R^m) and