In the calculus of variations, the existence of a one-to-one correspondence between the set of minimizers starting from a point (t, x) and the set of all reachable gradients of the value functionV at (t, x) is a well-known fact. This allows, among other things, to identify the singular set of the value function – that is, the set of points at whichV is not differentiable – as the set of points (t, x) for which the given funtional admits more than one minimizer. The aim of this last section is to investigate the above property for Mayer functionals with differential inclusions. Here, the difficulty consists in the nonsmoothness of the Hamiltonian atp= 0 that will force us to study separately the case of 0 ∈ ∂∗V(t, x). When the Hamiltonian and the terminal cost are of class C1,1loc(Rn×(Rn\ {0})) and C1(Rn), respectively, there is an injective map from ∂∗V(t, x)\ {0}to the set of all optimal trajectories starting from (t, x) (see, for instance, [8], Thm. 7.3.10 where the authors study parameterized optimal control problems with smooth data). However, we assume below neither the existence of a smooth parameterization, nor such a regularity of the Hamiltonian. Our assumptions for the terminal cost function are also milder than in [8]. It is precisely the lack of regularity ofH that represents the main difficulty, since it does not guarantee the uniqueness of solutions to system (5.1) below. We shall prove that, in our case, there exists an injective set-valued map from ∂∗V(t, x)\ {0} into the set of optimal trajectories starting from (t, x).
Lemma 5.1. Assume (SH) and (H1), and suppose φ is locally Lipschitz and such that ∂+φ(z) = ∅ for all z ∈Rn. Then, given a point (t, x)∈(−∞, T)×Rn and a vector p= (pt, px)∈∂∗V(t, x)\ {0}, there exists at least one pair (y(·), p(·))that satisfies the system
y(s) =˙ ∇pH(y(s), p(s)) for alls∈[t, T],
−p(s)˙ ∈ ∂x−H(y(s), p(s)) a.e. in[t, T], (5.1) together with the initial conditions
y(t) =x,
p(t) =−px, (5.2)
such that y(·) is optimal forP(t, x).
Proof. Observe, first, that if (pt, px)∈∂∗V(t, x)\{0}, thenpx= 0. Indeed,V satisfies−Vt+H(x,−Vx(t, x)) = 0 at every point of differentiability. Therefore, taking limits, we have that−pt+H(x,−px) = 0 for every (pt, px)∈
∂∗V(t, x). SinceH(x,0) = 0, we conclude that if (pt, px)= 0, then px= 0.
Since (pt, px)∈∂∗V(t, x), we can find a sequence{(tk, xk)} such thatV is differentiable at (tk, xk) and
k→∞lim(tk, xk) = (t, x), lim
k→∞∇xV(tk, xk) =px.
P. CANNARSAET AL.
Letyk(·) be an optimal trajectory forP(tk, xk). Sincepx= 0, there existsk >0 such that∇xV(tk, xk)= 0 for allk > k. By Remark2.9, there exists an arcpk such that (yk, pk) satisfies
y˙k(s)∈∂p−H(yk(s), pk(s)) a.e. in [tk, T], yk(tk) =xk,
−p˙k(s)∈∂x−H(yk(s), pk(s)) a.e. in [tk, T], −pk(T) ∈∂+φ(yk(T)). (5.3) Moreover, by Theorem3.1, we have that−pk(tk)∈∂x+V(tk, xk). SinceV is differentiable at (tk, xk), it follows that−pk(tk) =∇xV(tk, xk). Thus, fork > kwe have thatpk(s)= 0 for alls∈[tk, T] (see Rem.2.10), and the first inclusion in (5.3) becomes
˙
yk(s) =∇pH(yk(s), pk(s)) for alls∈[tk, T]. (5.4) Now, we extend yk and pk on [t−1, tk) by setting y(s) = y(tk), p(s) = p(tk) for all s ∈ [t−1, tk). The last point of the reasoning consists of proving that the sequence (yk(·), pk(·)), after possibly passing to a sub-sequence, converges to a pair (y(·), p(·)) that verifies the conclusions of the lemma. It is easy to prove that the sequences of functions{pk}k and{yk}k are uniformly bounded and uniformly Lipschitz continuous in [t−1, T], using Gronwall’s inequality together with estimates (2.15) and (SH)(iii), respectively. Hence their derivatives are essentially bounded. Therefore, after possibly passing to subsequences, we can assume that the sequence (yk(·), pk(·)) converges uniformly in [t−1, T] to some pair of Lipschitz functions (y(·), p(·)). Moreover, ˙pk(·) converges weakly to ˙p(·) inL1([t−1, T];Rn). Now, observe that
((yk(s), pk(s)),−p˙k(s))∈Graph(M) a.e. in [tk, T],
whereM is the multifunction defined byM(x, p) :=∂x−H(x, p). The multifunctionM is upper semicontinuous;
this can be easily derived as it was done forGin the proof of Theorem4.1. Hence, from Theorem 7.2.2. in [2]
it follows that−p(s)˙ ∈∂x−H(y(s), p(s)) for a.e.s∈[t, T]. Moreover, we have that p(t) = lim
k→∞pk(tk) =− lim
k→∞∇xV(tk, xk) =−px= 0.
So, we appeal again to Remark 2.10 to deduce that p(·) never vanishes on [t, T]. Since the map (x, p) →
∇pH(x, p) is continuous forp= 0, we can pass to the limit in (5.4) and deduce that ˙y(s) =∇pH(y(s), p(s)) for all s∈[t, T]. In conclusion, (y(·), p(·)) is a solution of the system in (5.1) with initial conditionsy(t) =x, p(t) =−px. This implies of course that ˙y(s)∈F(y(s)) for alls∈[t, T]. Moreover, sinceV is continuous andyk(·) is optimal, we have
φ(y(T)) = lim
k→∞φ(yk(T)) = lim
k→∞V(tk, yk(tk)) =V(t, y(t)),
which means thaty(·) is optimal forP(t, x).
Thenondegeneracy condition ∂+φ(z)=∅for allz∈Rnthat we have assumed above is verified, for instance, whenφis differentiable or semiconcave.
Remark 5.2. From the above proof it follows that, if the cost is supposed to be of classC1(Rn), then the pair (y(·), p(·)) in Lemma5.1satisfies∇φ(y(T)) =−p(T). Moreover,pis a dual arc associated to y.
Now, fix (t, x) ∈ (−∞, T)×Rn. For any p = (pt, px) ∈ ∂∗V(t, x)\ {0}, we denote byR(p) the set of all trajectoriesy(·) that are solutions of (5.1)–(5.2), and optimal forP(t, x). The above lemma guarantees that the set-valued map Rthat associates with anyp∈∂∗V(t, x)\ {0} the setR(p) has nonempty values. Now let us prove thatRis strongly injective. We will use the “difference set”:
∂x−H(x, p)−∂x−H(x, p) :={a−b: a, b∈∂x−H(x, p)}.
Theorem 5.3. Assume(SH),(H1), and suppose that φis of classC1(Rn)and (H3) for eachz∈Rn,F(z)is not a singleton and has a C1 boundary (for n >1), (H4) R+p∩(∂−xH(z, p)−∂−xH(z, p)) =∅ ∀p= 0 for allz∈Rn.
Then for any p1, p2∈∂∗V(t, x)\ {0} withp1=p2, we have thatR(p1)∩ R(p2) =∅.
Proof. Suppose to have two elements pi = (pi,t, pi,x)∈∂∗V(t, x)\ {0}, i = 1,2, with p1 =p2. Note that the Hamilton–Jacobi equation implies thatp1,x =p2,x if and only p1 =p2. Furthermore, suppose that there exist two pairs (y(·), pi(·)), i= 1,2 that are solutions of system (5.1) withpi(t) =pi,xandy(·) is optimal forP(t, x).
Then, we get
˙
y(s) =∇pH(y(s), p1(s)) =∇pH(y(s), p2(s)) for alls∈[t, T].
This implies that
pi(s)∈NF(y(s))( ˙y(s)), i= 1,2 for alls∈[t, T].
By (H3), the normal coneNF(y(s))( ˙y(s)) is a half-line. Recalling also thatpi(i= 1,2) never vanishes, it follows that there exists λ(s)>0 such thatp2(s) =λ(s)p1(s), for everys ∈[t, T]. The functionλ(·) is differentiable a.e. on [t, T] because
λ(s) =|p2(s)|
|p1(s)|·
By (5.1), sinceβ∂x−H(x, p) =∂x−H(x, βp) for eachβ >0, it follows that
−p˙2(s) =−λ(s) ˙p1(s)−λ(s)p˙ 1(s)∈λ(s)∂x−H(y(s), p1(s)) for a.e. s∈[t, T].
Dividing byλ(s),
−λ(s)˙
λ(s)p1(s)∈∂x−H(y(s), p1(s))−∂x−H(y(s), p1(s)) for a.e.s∈[t, T]. (5.5) Since the “difference set” in (H4) is symmetric, (H4) is equivalent to the condition
(R\ {0})p∩
∂x−H(x, p)−∂x−H(x, p)
=∅ ∀p= 0. (5.6)
From (5.5) and (5.6) we obtain that ˙λ(s) = 0 a.e. in [t, T]. This gives thatλ is constant. Moreover, since φ ∈ C1(Rn), we have that ∇φ(y(T)) = −pi(T), i = 1,2, which implies that λ = 1. But this yields p1,x = p2,x, which contradicts the inequality p1 = p2. Hence, we can assert that the functions yi(·), i = 1,2, are
different.
Remark 5.4. Assumption (H4) is verified, for instance, when x → H(x, p) is differentiable, without any Lipschitz regularity of the mapx→ ∇pH(x, p).
Example 5.5. The above theorem is false whenφ /∈C1(Rn). Indeed, without such an assumption there could exist infinitely many reachable gradients at a point at which the optimal trajectory is unique. Let us consider the state equation in the one-dimensional space given by ˙x=u∈[−1,1]. Let the cost function be defined by
φ(x) =
x2sin1
x
+ 3x ifx= 0,
0 ifx= 0.
This function is differentiable, with derivative φ(x) =
−cos(1x) + 2xsin(1x) + 3 ifx= 0,
3 ifx= 0
P. CANNARSAET AL.
which is discontinuous at zero because cos(1x) oscillates asx→0. Therefore, this function is differentiable but not of class C1(R). Moreover,∂∗φ(0) = [2,4].
Note thatφis strictly increasing. Thus, for any (t, x)∈[0, T]×R, there exists a unique optimal control that is the constant one u≡ −1, and so the unique optimal trajectory is x(s) = x−s+t. The value function is V(t, x) =φ(x−T+t), and so it has the same regularity as the final costφ. Summarizing,∂∗V(t, x) ={(a, a) : a∈[2,4]} at any point (t, x) such that x=T −t, but there exists a unique optimal trajectory starting from such points.
Now let us consider the case when p= 0∈∂∗V(t, x).
Theorem 5.6. Assume (SH), (H1) and suppose that φ ∈ C1(Rn). Let (t, x) ∈ (−∞, T)×Rn be such that 0 ∈ ∂∗V(t, x). Then there exists an optimal trajectory y : [t, T] → Rn for P(t, x) such that ∇φ(y(T)) = 0.
Consequently, the corresponding dual arc is equal to zero.
Proof. Since 0∈∂∗V(t, x), we can find a sequence{(tk, xk)}such thatV is differentiable at (tk, xk) and
k→∞lim(tk, xk) = (t, x), lim
k→∞∇V(tk, xk) = 0.
Let yk(·) be an optimal trajectory for P(tk, xk) and pk(·) be a dual arc. We extendyk and pk on [t−1, T] as in the proof of Lemma 5.1. By (Thm. 7.2.2 in [2]), we can assume, after possibly passing to a subsequence, thatyk(·) converges uniformly toy(·) which is a trajectory of our system on [t, T]. Since
φ(y(T)) = lim
k→∞φ(yk(T)) = lim
k→∞V(tk, xk) =V(t, x),
it follows thaty is optimal forP(t, x). Furthermore, the sensitivity relation in Remark3.4 holds true and so, recalling thatV is differentiable at (tk, xk), we have
−pk(tk) =∇xV(tk, xk)→0. (5.7)
By (5.7), (2.15) and the Gronwall’s inequality, we get thatpk(T)→0 whenk→ ∞. We conclude that
∇φ(y(T)) = lim
k→∞∇φ(yk(T)) = lim
k→∞−pk(T) = 0.
As applications of the above results we obtain the following corollaries.
Corollary 5.7. Under the same assumptions of Theorem 5.3, for every (t, x)∈ (−∞, T]×Rn there exist at least as many optimal solutions ofP(t, x)as elements of ∂∗V(t, x).
Proof. Fix (t, x) ∈ (−∞, T]×Rn. By Lemma 5.1 and Remark 5.2, to every p = (pt, px) ∈ ∂∗V(t, x)\ {0}
corresponds a pair of arcs (y, p) satisfying (5.1) and such that−p(t) =px,yis optimal forP(t, x) and−p(T) =
∇φ(y(T)). Moreover, since the arc p never vanishes, we have that ∇φ(y(T)) = 0. Theorem 5.3 implies that optimal solutions corresponding to distinct elements of∂∗V(t, x)\ {0}are distinct. On the other hand, if 0∈
∂∗V(t, x), then Thereom5.6yields the existence of an optimal trajectoryyforP(t, x) such that 0 =∇φ(y(T)).
These facts give our claim.
Corollary 5.8. Assume (SH), (H1), φ ∈ C1(Rn) and suppose also that ∇φ(x) = 0 for all x ∈ Rn. Then 0∈∂∗V(t, x)for all(t, x)∈(−∞, T)×Rn.
Corollary 5.9. Assume(SH), (H1),(H3),(H4) and let φ∈C1(Rn)∩SC(Rn). IfV fails to be differentiable at a point (t, x)∈(−∞, T)×Rn, then there exist two or more optimal trajectories starting from(t, x).
Proof. Since V is semiconcave (see [5]), if it is not differentiable at a point (t, x) ∈ (−∞, T)×Rn, then we can find two distinct elementsp1, p2∈∂∗V(t, x). Ifp1, p2are both nonzero, we can apply Theorem5.3 to find two distinct optimal trajectories. If one of the two vectors is zero, for instancep1, then there exists at least an associated optimal trajectoryy1(·) such that ∇φ(y1(T)) = 0 by Theorem 5.6, but for any optimal trajectory y2(·) associated top2 it holds that∇φ(y2(T))= 0 by Lemma5.1and Remark2.10.
It might happen that two or more optimal trajectories actually start from a point (t, x) at which V is differentiable. However, ifH ∈C1,1loc(Rn×(Rn\ {0})), then it is well-know that such a behaviour can only occur when the gradient of V at (t, x) vanishes (see e.g., Thm. 7.3.14 and Example 7.2.10(iii) in [8]). More can be said in one space dimension as we explain below.
Example 5.10. Whenn= 1 it is easy to show that, ifV is differentiable at some point (t0, x0) withVx(t0, x0)= 0, then there exists a unique optimal trajectory starting from (t0, x0). Indeed, in this case,F(x) = [f(x), g(x)]
for suitable functionsf, g:R→R, withf ≤g, such that−f andg are locally semiconvex. So, H(x, p) =
f(x)p, p <0,
g(x)p, p≥0. (5.8)
Ifx(·) is an optimal trajectory at (t0, x0) andp(·) is a dual arc associated with x(·), then by Remark3.4we have that 0=Vx(t0, x0) =−p(t0). Therefore, 0 =p(t) for allt ∈[t0, T] by Remark 2.10. Thus, (5.8) and the maximum principle yield
x(t) =˙
f(x(t)), ifVx(t0, x0)>0,
g(x(t)), ifVx(t0, x0)<0. (5.9) Sincef andg are both locally Lipschitz,x(·) is the unique solution of (5.9) satisfyingx(t0) =x0.
Acknowledgements. The authors would like to thank the anonymous referees for their thoughtful comments which helped to improve the manuscript.
References
[1] J.P. Aubin and A. Cellina, Differential inclusions. Vol. 264 of Gr¨undlehren der Mathematischen Wissenschaften. Springer-Verlag (1984).
[2] J.P. Aubin and H. Frankowska, Set-valued analysis. Vol. 2 ofSystems & Control: Foundations & Applications. Birkh¨auser Boston Inc. (1990).
[3] P. Bettiol, H. Frankowska and R. Vinter, Improved Sensitivity Relations in State Constrained Optimal Control.Appl. Math.
Optim.(2014).
[4] P. Cannarsa and H. Frankowska, Some characterizations of optimal trajectories in control theory.SIAM J. Control Optim.
(1991) 1322–1347.
[5] P. Cannarsa and P. Wolenski, Semiconcavity of the value function for a class of differential inclusions.Discrete Contin. Dyn.
Syst.29(2011) 453–466.
[6] P. Cannarsa, H. Frankowska and C. Sinestrari, Optimality conditions and synthesis for the minimum time problem.Set-Valued Anal.8(2000) 127–148.
[7] P. Cannarsa, F. Marino and P. Wolenski, The dual arc inclusion with differential inclusions.Nonlin. Anal.79(2013) 176–189.
[8] P. Cannarsa and C. Sinestrari, Semiconcave functions, Hamilton–Jacobi equations, and optimal control. Birkh¨auser Boston Inc. (2004).
[9] F. Clarke, Optimization and nonsmooth analysis. John Wiley & Sons Inc. (1983).
[10] F. Clarke and R. Vinter, The relationship between the maximum principle and dynamic programming.SIAM J. Control Optim.25(1987) 1291–1311.
P. CANNARSAET AL.
[11] H. Frankowska and C. Olech, R-convexity of the integral of set-valued functions. Johns Hopkins University Press (1981) 117–129.
[12] H. Frankowska and M. Mazzola, On relations of the adjoint state to the value function for optimal control problems with state constraints.Nonlin. Differ. Eq. Appl.20(2013) 361–383.
[13] A.E. Mayer, Eine ¨Uberkonvexit¨at.Math. Z.(1935) 511–531.
[14] L. Pasqualini, Superconvexit´e.Bull. de Cl.XXV (1939) 18–24.
[15] N.N. Subbotina, The maximum principle and the superdifferential of the value function. Problems Control Inform. The-ory/Problemy Upravlen. Teor. Inform.18(1989) 151–160.
[16] P. Vincensini, Sur les figures superconvexes planes.Bull. Soc. Math. France64(1936) 197–208.
[17] R.B. Vinter, New results on the relationship between dynamic programming and the maximum principle.Math. Control Signals Systems1(1988) 97–105.