Deterministic characterization of viability for stochastic differential equation driven by fractional brownian motion

DOI:10.1051/cocv/2011188 www.esaim-cocv.org

DETERMINISTIC CHARACTERIZATION OF VIABILITY FOR STOCHASTIC DIFFERENTIAL EQUATION DRIVEN BY FRACTIONAL BROWNIAN MOTION^∗,∗∗

Tianyang Nie

and Aurel R˘ as ¸canu

Abstract. In this paper, using direct and inverse images for fractional stochastic tangent sets, we establish the deterministic necessary and sufficient conditions which control that the solution of a given stochastic differential equation driven by the fractional Brownian motion evolves in some particular setsK. As a consequence, a comparison theorem is obtained.

Mathematics Subject Classification. 60H10, 60H20, 60G22.

Received February 8, 2011. Revised July 17, 2011 Published online November 22, 2011.

1. Introduction

The viability (weakly invariant) problem of a setKfor a differential equation is to find necessary and sufficient conditions such that for any starting point in K, the differential equation has at least one solution evolving inK. If all solutions of the differential equation satisfy theK-viable property, we call that the subset is invariant (strong invariant) for the equation.

Nagumo gave a criterion of the viability in terms of contingent sets in 1942. The Nagumo theorem states that: if f : K = ¯K ⊂R^m → R^m is a bounded and continuous function, then K is viable for the differential equation

x(t) =f(x(t)), x(0) =x₀∈K if and only if,∀ x∈Kand∀ p, a normal vector atK in x, we have

f(x), p ≤0.

The important tools in studying the viability problem are the notions of tangent sets and contingent sets.

Aubin and Da Prato firstly used the stochastic contingent sets to characterize the viability and invariance for

Keywords and phrases.Stochastic viability, stochastic differential equation, stochastic tangent set, fractional Brownian motion.

∗Supported by Marie Curie ITN Project, “Controlled Systems”, No. 213841/2008.

∗∗ Supported by IDEI project, No. 395/2007.

1 School of Mathematics, Shandong University, Jinan, Shandong 250100, P.R. China.nietianyang@163.com;

tianyang.nie@uaic.ro

2 Faculty of Mathematics, “Alexandru Ioan Cuza” University, Carol I Blvd, No. 11, 700506 Iasi, Romania.

aurel.rascanu@uaic.ro

3 “Octav Mayer” Mathematics Institute of the Romanian Academy, Carol I Blvd, No. 8, 700506 Iasi, Romania.

Article published by EDP Sciences c EDP Sciences, SMAI 2011

the stochastic differential equations (SDEs) of a closed and convex subset in [1]. Milian generalized Aubin and Da Prato’s result to arbitrary subsets which can be time-dependent and random in [9].

Another approach was developed by Buckdahnet al.in [3–6]. The main points of their work consist in proving that the viability property for SDE and also backward SDE holds true if and only if the square of the distance to the constraint sets is a viscosity supersolution (subsolution) of the associated Hamilton-Jacobi-Bellman equation.

A general result on the existence and uniqueness of the solution for multidimensional, time dependent, SDEs driven by fBm with Hurst parameterH >1/2 was given by Nualart and Rascanu in [11], using the techniques of the classical fractional calculus. Moreover, Ciotir and Rascanu proved a type of Nagumo Theorem on viability property of closed bounded subsets for the SDE driven by fBm in [7].

Criterions for the viability and invariance of closed bounded subsets expressed by stochastic contingent sets given in [7] are general, but their conditions are very hard to check, even for the particular case. Using the ideas of [7], our paper will give easier checkable conditions for general SDEs driven by fBm for some particular setsK.

As a very important application of our results, a comparison theorem for the SDEs driven by fBm is obtained.

In fact, from [7], we give the extension of the result concerning stochastic tangent sets to direct images and inverse images which was studied by Aubin and Da Prato in [1], to the following stochastic differential equation driven by fractional Brownian motion,P-a.s. ω∈Ω,

X_s^t,x=x+ _s

t

b(r, X_r^t,x)dr+ _s

t

σ(r, X_r^t,x)dB^H_r , s∈[t, T], where

• B^H=

B_t^H, t≥0

is ak-dimensional fBm with Hurst parameter 1/2< H <1, and the integral with respect toB^H is a pathwise Riemann-Stieltjes integral;

• b(t, x) : [0, T]×R^d→R^d andσ(t, x) : [0, T]×R^d→R^d×k are continuous functions.

In the case of one-dimensional fBm, we derive explicit criteria for the solution of this equation to remain in some particular setsKi.e., under which it holds that for allt∈[0, T] and for allx∈K,

X_s^t,x∈K a.s. ω∈Ω, ∀s∈[t, T].

Similarly to [1], the characterization of viability ofKis obtained through the study of the direct and inverse images for fractional stochastic tangent sets. Using our main Theorem3.5, we get the conditions on b and σ such that some particular setsK are viable.

Now we explain how the paper is organized. In the second section, we recall some classical definitions and assumptions on the coefficients. We also recall the main results from [7], which we will use later. In Section3, we introduce the notion of direct and inverse images corresponding to SDE driven by fBm and give our main results. Section4is devoted to give the deterministic characterization of viability conditions for some particular set constraints. In Section5, as an application, a comparison theorem is proved.

2. Preliminaries

Consider the following stochastic equation onR^d X_s=X₀+

_s

0

b(r, X_r)dr+ _s

0

σ(r, X_r)dB^H_r , s∈[0, T], (2.1) where,

• B^H =

B^H_t , t≥0

is a k-dimensional fBm defined on a complete probability space (Ω,F,P), with Hurst parameter 1/2< H <1, and the integral with respect toB^H is a pathwise Riemann–Stieltjes integral;

• X₀ is ad-dimensional random variable;

• b: [0, T]×R^d→R^d, σ: [0, T]×R^d→R^d×k are continuous functions.

Remark 2.1. The fractional Brownian motion has the following properties:

For every 0 < ε < H and T > 0, there exists a positive random variable η_ε,T such that E(|ηε,T|^p)

<∞, for allp∈[1,∞) and for alls, t∈[0, T],

|B^H(t)−B^H(s)| ≤η_ε,T|t−s|^H−ε, a.s.

From Proposition 1.7.1, [2] (see also [8]), we know that for everyt₀∈[0,+∞),

P

⎧⎪

⎨

⎪⎩lim sup

t→t0,

t≥t0

B^H(t)−B^H(t₀) t−t₀

= +∞

⎫⎪

⎬

⎪⎭= 1. (2.2)

Using the same method, we can prove that

P

⎧⎪

⎨

⎪⎩lim sup

t→t0,

t≥t0

B^H(t)−B^H(t₀) t−t₀ = +∞

⎫⎪

⎬

⎪⎭=P

⎧⎨

⎩lim inf

t→t0,

t≥t0

B^H(t)−B^H(t₀) t−t₀ =−∞

⎫⎬

⎭= 1

2· (2.3)

2.1. Assumptions and notations

For the coefficients appearing in equation (2.1), we make the following assumptions (as in [7]):

(H₁) σ(t, x) is differentiable with respect tox, and there existβ, δ, M₀, 0< β, δ ≤1 and, for∀R > 0, there exists a constantM_R>0 such that for allt∈[0, T],

(H_σ) :

(i)|σ(t, x)−σ(s, y)| ≤M₀

|t−s|^β+|x−y|

, ∀x, y∈R^d, (ii)|∇xσ(t, y)− ∇xσ(s, z)| ≤M_R

|t−s|^β+|y−z|^δ

,∀ |y|,|z| ≤R, where∇_xσ(t, x) = (∇_xσ^i,j(t, x))i=1,d,j=1,k and

|∇xσ(t, x)|²= d l=1

d i=1

k j=1

|∂xlσ^i,j(t, x)|².

From (i), we can deduce that for allx∈R^d

|σ(t, x)| ≤ |σ(0,0)|+M₀(|t|^β+|x|)≤M_0,T(1 +|x|), whereM_0,T =|σ(0,0)|+M₀+M₀T.

Let

α₀= min

β, δ 1 +δ

·

(H₂) There existμ∈(1−α₀,1],L₀and, for∀R≥0, there exists a constantL_R>0 such that∀t∈[0, T], (H_b) :

(i)|b(r, x)−b(s, y)| ≤L_R(|r−s|^μ+|x−y|), ∀ |x|,|y| ≤R, (ii)|b(t, x)| ≤ L₀(1 +|x|), ∀x∈R^d.

Finally, we introduce some notations.

Given A= (a^i,j)_d×k and y= (yⁱ)_d×1, d, k∈N^∗. we denote|A|²=

i,j

|a^i,j|² and|y|=

i

|yⁱ|². Lett∈[0, T] be fixed. Denote by

• W^α,∞(t, T;R^d),0< α <1,the space of functionsf : [t, T]→R^d which are continuous and f_α,∞;[t,T]:= sup

s∈[t,T]

|f(s)|+ _s

t

|f(s)−f(r)| (s−r)^α+1 dr

<∞.

• W˜^1−α,∞(t, T;R^d),0< α <1/2,the space of continuous functionsg: [t, T]→R^d such that gW˜^1−α,∞(t,T;R^d):=|g(t)|+ sup

t<r<s<T

|g(s)−g(r)| (s−r)^1−α +

_s

r

|g(y)−g(r)| (y−r)^2−α dy

<∞.

• C^μ([t, T];R^d), 0< μ <1, the space ofμ-H¨older continuous functionsf : [t, T]→R^d, with the norm f_μ;[t,T]= sup

s∈[t,T]

|f(s)|+ sup

t≤r<s≤T

|f(s)−f(r)|

(s−r)^μ <∞.

• W^α,1(t, T;R^d) the space of measurable functionsf such that f_α,1;[t,T]:=

_T

t

|f(s)|

(s−t)^α + _s

t

|f(s)−f(y)|

(s−y)^α+1 dy

ds <∞, and we have

W^α,∞(t, T;R^d)⊂W^α,1(t, T;R^d).

2.2. Generalized Stieltjes integral Denoting

Λ_α(g; [t, T]) := 1

Γ(1−α) sup

t<r<s<T

D_s−^1−αg_s−

(r) where

Γ(α) = _∞

0

s^α−1e^−sds and

(D_s−^1−αg_s−)(r) = e^iπ(1−α) Γ(α)

g(s)−g(r)

(s−r)^1−α + (1−α) _s

r

g(r)−g(y) (y−r)^2−αdy

1_(t,s)(r), it follows that (see [10,11])

Λ_α(g; [t, T])≤ 1

Γ(1−α)Γ(α)gW˜^1−α,∞(t,T;R^d). Also it’s obvious that

Λ_α(g; [t, T])≤Λ_α(g; [0, T]) (:=Λ_α(g)).

Definition 2.2. Let 0 < α < 1/2. We assume f ∈ W^α,1(t, T;R^d×k) and g ∈ W˜^1−α,∞(t, T;R^k), then the generalized Stieltjes integral is defined by

_s

t

f(r) dg(r) := (−1)^α _s

t

D^α_t+f (r)

D_s−^1−αg_s−

(r) dr,

where

(D^α_t+f)(r) = 1 Γ(1−α)

f(r) (r−t)^α +α

_r

t

f(r)−f(y) (r−y)^α+1dy

1_(t,T)(r).

The integral _s

t

f(r)dg(r) exists for alls∈[t, T] and _T

t

f(r)dg(r)

≤ sup

t≤r<s≤T|(D_s−^1−αg_s−)(r)|

_T

t

|(D^α_t+f)(s)ds|

≤Λ_α(g; [t, T])f_α,1;[t,T].

Remark 2.3. It is known that whenH∈(1/2,1) and 1−H < α <1/2, the random variable G=Λ_α(B^H) = 1

Γ(1−α) sup

t<s<r<T|(D_r−^1−αB_r−)(s)|

has moments of all order. Therefore if u={ut, t∈[0, T]} is a stochastic process and its trajectories belong to W^α,1(t, T;R^d), with 1−H < α <1/2, the integral

_T

0

u_sdB_s^H has the meaning in the sense of Definition 2.2 and it follows

_T

0

u_sdB_s^H ≤Guα,1,

which means that the stochastic integral from (2.1) is a pathwise Riemann–Stieltjes integral.

Nualart and Rascanu proved in [11] that under the assumptions (H₁) and (H₂), with β > 1−H and δ >1/H−1, the SDE

X_s^t,ξ=ξ+ _s

t

b(r, X_r^t,ξ)dr+ _s

t

σ r, X_r^t,ξ

dB^H_r , s∈[t, T], has a unique solutionX^t,ξ∈L⁰

Ω,F,P;W^α,∞(t, T;R^d)

,for allα∈(1−H, α₀).Moreover, forP-almost all ω∈Ω,X_·^t,ξ(ω)∈C^1−α

0, T;R^d . 2.3. Fractional viability

Following [7], we recall the notion of the viability property for SDE driven by fBm. We also present the characterization theorem for the viability property.

Consider the stochastic differential equation driven by fBmB^H with Hurst parameter 1/2< H <1, X_s^t,x=x+

_s

t

b(r, X_r^t,x)dr+ _s

t

σ(r, X_r^t,x)dB^H_r , s∈[t, T]. (2.4) Definition 2.4. Let K = {K(t) : t ∈ [0, T]} be a family of subsets of R^d. We say that K is viable (weakly invariant) for equation (2.4) if, for everyt∈[0, T] and from every starting point x∈K(t), there exists at least one of its solutions{X_s^t,x:s∈[t, T]}which satisfies

X_s^t,x(ω)∈K(s) for all s∈[t, T], a.s. ω∈Ω.

Definition 2.5. The familyKis invariant (strong invariant) for equation (2.4) if, starting at every timet∈[0, T] and from every pointx∈K(t), all solutions{X_s^t,x:s∈[t, T]}of (2.4) satisfy

X_s^t,x(ω)∈K(s) for all s∈[t, T], a.s. ω∈Ω.

Remark 2.6. In the case that the equation has a unique solution, viability is equivalent to invariance.

Suppose that the mappingsb andσsatisfy (H₁) and (H₂).

Definition 2.7. Lett∈[0, T] andx∈K(t).Let 1/2<1−α < H.

The (1−α)-fractionalB^H-contingent set to K(t) inxis the set of the pairs (u, v)∈R^d×R^d×k, such that there exist a random variable ¯h= ¯h^t,x>0 and a stochastic processQ=Q^t,x:Ω×

t, t+ ¯h

→R^d, and for any R >0 with |x| ≤R, there exist two random variables H_R,H˜_R >0, depending only onR, L_R,M_0,T,M₀, L₀, T,α,β,Λ_α

B^H

) and a constantγ=γ_R(α, β)∈(0,1) such that for alls, τ ∈[t, t+ ¯h],P-a.s.

|Q(s)−Q(τ)| ≤H_R|s−τ|^1−α, |Q(s)| ≤H˜_R|s−t|^1+γ and

x+ (s−t)u+v

B_s^H−B_t^H

+Q(s)∈K(s). Definition 2.8. Lett∈[0, T] andx∈K(t).Let 1/2<1−α < H.

The (1−α)-fractionalB^H-tangent set toK(t) in x, denoted by S_K(t)(t, x), is the set of the pairs (u, v)∈ R^d×R^d×k, such that there exist a random variable ¯h= ¯h^t,x>0 and two stochastic processes

U =U^t,x:Ω×[t, T]→R^d, U(t) = 0,

V =V^t,x:Ω×[t, T]→R^d×k, V(t) = 0 (2.5) and for every R >0 with |x| ≤ R, there exist two random variablesD_R,D˜_R >0, depending only on R, L_R, M_0,T,M₀,L₀,T,α,β,Λ_α

B^H

), such that for alls, τ ∈

t, t+ ¯h ,P-a.s.

|U(s)−U(τ)| ≤D_R|s−τ|^1−α, |V (s)−V(τ)| ≤D˜_R|s−τ|^{min{β,1−α}} (2.6) and

x+ _s

t

(u+U(r))dr+ _s

t

(v+V(r))dB_r^H ∈K(s).

Remark 2.9. The definition ofS_ϕ(K(t))(t, ϕ(x)) is similar toS_K(t)(t, x), we only change the condition x+

_s

t

(u+U(r))dr+ _s

t

(v+V(r))dB_r^H ∈K(s), to the following form

ϕ(x) + _s

t

(u+U(r))dr+ _s

t

(v+V(r))dB_r^H ∈ϕ(K(s)).

Now we recall the main result of [7] concerning the stochastic viability.

Theorem 2.10. Let K = {K(t) :t∈[0, T]}, K(t) = K(t) ⊂ R^d. Assume that (H₁) and (H₂) are satisfied with 1/2< H <1, 1−H < β, δ > ^1−H_H . Letmax{1−H,1−μ}< α < α₀. Then the following assertions are equivalent:

(I) K is viable for the fractional SDE(2.4), i.e., for all t∈[0, T]and for allx∈K(t)there exists a solution X^t,x(ω,·)∈C^1−α

[t, T] ;R^d

of(2.4) such that

X_s^t,x∈K(s), ∀ s∈[t, T] ;

(II) for allt∈[0, T],x∈K(t), (b(t, x), σ(t, x))is(1−α)-fractionalB^H-contingent toK(t)inx;

(III) for allt∈[0, T],x∈K(t), (b(t, x), σ(t, x))is(1−α)-fractionalB^H-tangent toK(t)in x.

Remark 2.11. The assertion (III) is given in [7] only for the deterministic case,i.e.usingg∈W˜^1−α,∞(t, T;R^d) instead ofB^H. Using the same method as in [7] to prove (II), we can derive (III).

Moreover, it follows:

Corollary 2.12. Assume that (H₁) and (H₂) are satisfied with 1/2 < H < 1, 1−H < β, δ > ^1−H_H . Let max{1−H,1−μ}< α < α₀, if K⊂R^d is independent of t, the following assertions are equivalent:

(j) K is viable with respect to the fractional SDE(2.4);

(jj) for all t∈[0, T],x∈∂K,(b(t, x), σ(t, x))is(1−α)-fractional B^H-contingent toK in x;

(jjj) for allt∈[0, T],x∈∂K,(b(t, x), σ(t, x))is(1−α)-fractionalB^H-tangent to K inx.

Proof. SinceK is independent oft, from Theorem2.10, it’s obvious that (j)⇒(jj)⇒(jjj). It is sufficient to prove (jjj)⇒(j).

Lett∈[0, T] andx∈K\∂K be chosen arbitrary. SinceX^t,x is continuous, there exists a random variable

¯h, such that for alls∈[t, t+ ¯h],

X_s^t,x=x+ _s

t

b(r, X_r^t,x)dr+ _s

t

σ(r, X_r^t,x)dB_r^H ∈K, which can be written as

X_s^t,x=x+ _s

t

[b(t, x) +U(r)]dr+ _s

t

[σ(t, x) +V(r)]dB_r^H∈K, where

U(r) =b(r, X_r^t,x)−b(t, x), V(r) =σ(r, X_r^t,x)−σ(t, x).

Clearly (b(t, x), σ(t, x)) is (1−α)-fractionalB^H-tangent to Kin x. From (jjj), taking into account thatK is

independent oft, we obtain (III).

3. Stochastic tangent sets to direct and inverse images

Firstly, we give the following auxiliary lemma,

Lemma 3.1. Given two stochastic processes U =U^t,x, V =V^t,x satisfying (2.5), (2.6). Then for all t≤τ ≤ s≤t+ ¯h,

(a) _s

τ

U(r)dr

≤D_R(s−t)^1−α(s−τ), (b)

_s

τ

V(r)dB_r^H

≤C_R(α, β) ˜D_RΛ_α(B^H)(s−t)^{min{β,1−α}}(s−τ)^1−α, whereC_R(α, β)depends only on R,α, andβ.

Proof. (a) _s

τ

U(r)dr =

_s

τ

[U(r)−U(t)]dr ≤D_R

_s

τ

(r−t)^1−αdr

≤D_R(s−t)^1−α(s−τ),

(b)

_s

τ

V(r)dB_r^H =

_s

τ

[V(r)−V(t)]dB_r^H

≤Λ_α(B^H)V_α,1;[τ,s]

≤Λ_α(B^H) _s

τ

|V(r)−V(t)|

(r−τ)^α + _r

τ

|V(r)−V(y)|

(r−y)^α+1 dy

dr

≤D˜_RΛ_α(B^H) _s

τ

(r−t)^{min{β,1−α}}

(r−τ)^α + _r

τ

(r−y)^{min{β,1−α}}

(r−y)^α+1 dy

dr

≤D˜_RΛ_α(B^H) 1

1−α(s−t)^{min{β,1−α}}(s−τ)^1−α +

_s

τ

_r

τ

(r−y)min{β−α,1−2α}−1dydr

≤C_R(α, β) ˜D_RΛ_α(B^H)(s−t)^{min{β,1−α}}(s−τ)^1−α. Remark 3.2. From (a) and (b), we see that forτ=t,

(a) _s

t

U(r)dr

≤D_R(s−t)^2−α, (b)

_s

t

V(r)dB^H_r

≤C_R(α, β) ˜D_RΛ_α(B^H)(s−t)1+min{β−α,1−2α}.

The following theorem will give the extension to the fBM framework of the result concerning stochastic tangent sets to direct images which was studied by Aubin and Da Prato, [1].

Theorem 3.3. Assume that (H₁) and(H₂) are satisfied with 1/2< H <1, 1−H < β, δ > ^1−H_H . Let α, α₀ be such that max{1−H,1−μ}< α < α₀. Let K(t) = K(t)⊂R^d, t∈[0, T], S_K(t)(t, x)be (1−α)-fractional B^H-tangent set toK inxandϕbe aC² map fromR^d toR^m with a bounded second derivative. If

(b(t, x), σ(t, x))∈S_K(t)(t, x) then

(ϕ(x)b(t, x), ϕ(x)σ(t, x))∈S_ϕ(K(t))(t, ϕ(x)).

Proof. Since (b(t, x), σ(t, x))∈S_K(t)(t, x), there exist a random variable ¯h= ¯h^t,x >0 and, for allR > 0, two stochastic processes satisfying (2.5), (2.6) for all s∈

t, t+ ¯h

, andx∈R^d with|x| ≤R, x+

_s

t

(b(t, x) +U(r))dr+ _s

t

(σ(t, x) +V(r))dB^H(r)∈K(s).

Define

η_s=x+ _s

t

(b(t, x) +U(r))dr+ _s

t

(σ(t, x) +V(r))dB^H(r).

From Lemma3.1and the (H−ε) H¨older continuous property of fractional Brownian motion, it follows that for alls, τ ∈

t, t+ ¯h ,

|ηs−η_τ| ≤ζ(s−τ)^1−α.

According to the fractional Itˆo formula (see [10], Rem. 2.7.4), we know that for alls∈

t, t+ ¯h ϕ

x+

_s

t

(b(t, x) +U(r))dr+ _s

t

(σ(t, x) +V(r))dB^H(r)

=ϕ(x) + _s

t

[ϕ(η_r)(b(t, x) +U(r))]dr+ _s

t

[ϕ(η_r)(σ(t, x) +V(r))]dB^H(r)

=ϕ(x) + _s

t

[ϕ(x)b(t, x) +U₁(r)]dr+ _s

t

[ϕ(x)σ(t, x) +V₁(r)]dB^H(r), where

U₁(r) =ϕ(η_r)U(r) + (ϕ(η_r)−ϕ(x))b(t, x), V₁(r) =ϕ(η_r)V(r) + (ϕ(η_r)−ϕ(x))σ(t, x).

Then

ϕ(x) + _s

t

[ϕ(x)b(t, x) +U₁(r)]dr+ _s

t

[ϕ(x)σ(t, x) +V₁(r)]dB^H(r)

=ϕ

x+ _s

t

(b(t, x) +U(r))dr+ _s

t

(σ(t, x) +V(r))dB^H(r)

∈ϕ(K(s)), (3.1)

and

U₁(t) = 0, V₁(t) = 0.

For all s, τ ∈ t, t+ ¯h

and for every R > 0 and |x| ≤R, using the Lipschitz continuity of ϕ and (H₂), we obtain that

|U1(s)−U₁(τ)| ≤ |ϕ(η_τ)||U(s)−U(τ)|+ (|U(s)|+|b(t, x)|)|ϕ(η_s)−ϕ(η_τ)|

≤θ₁|s−τ|^1−α+θ₂|ηs−η_τ| ≤θ|s−τ|^1−α, similarly we can prove that

|V1(s)−V₁(τ)| ≤θ|s˜ −τ|^{min{β,1−α}},

where the H¨older constants θ, ˜θ are random variables which depend only onR, L_R, M_0,T, M₀, L₀, T, α, β, Λ_α

B^H .

We conclude from (3.1) that

(ϕ(x)b(t, x), ϕ(x)σ(t, x))∈S_ϕ(K(t))(t, ϕ(x)).

We can also give the extension of the result concerning stochastic tangent sets to inverse images to the fBM form.

We introduce a spaceHof the functionsϕ:R^d→R^mof classC², with a bounded and Lipschitz continuous second derivative and moreover there exist a_ϕ < b_ϕ and M > 0,L > 0, such that for all a_ϕ ≤ |x| ≤b_ϕ, the matrixϕ(x) = _dx^dϕ(x) =∇xϕ(x)∈R^m×dhas a right inverse denoted byϕ(x)⁺ satisfying

(1) [ϕ(x)⁺] ≤M,

(2) [ϕ(x)⁺]−[ϕ(y)⁺] ≤L|x−y|, where we use the notations [ϕ(x)⁺]= _dx^d[(_dx^dϕ(x))⁺] =∇x[(∇xϕ(x))⁺].

Lemma 3.4. Given two stochastic processesU =U^t,x, V =V^t,xsatisfying(2.5), (2.6)withd=m, andϕ∈ H, let

f(r, y) =ϕ(y)⁺[U(r)−(ϕ(y)−ϕ(x))b(t, x)], g(r, y) =ϕ(y)⁺[V(r)−(ϕ(y)−ϕ(x))σ(t, x)].

Assume that (H₁) and (H₂) are satisfied with 1/2 < H < 1, 1 −H < β, δ > ^1−H_H , then for α ∈ (1−H, α₀) and for every δ₀ > 0, there exists a random variable ¯h₁ = ¯h^t,x₁ , such that for a_ϕ + 2δ₀

≤ |x| ≤b_ϕ−2δ₀ andP-a.s. ω∈Ω, the following stochastic differential equation ξ_s=x+

_s

t

(b(t, x) +f(r, ξ_r))dr+ _s

t

(σ(t, x) +g(r, ξ_r))dB^H(r), s∈[t, t+ ¯h₁], has a unique solutionξ_·(ω)∈L⁰

Ω,F,P;W^α,∞(t, T;R^d)

. Moreoverξ_·(ω)∈C^1−α

t, t+ ¯h₁;R^d ,P-a.s.

Proof. Applying the partition of unity theorem from [12], page 61, we see that for everyδ₀ >0, there exists a functionα(x)∈C^∞(R^d) such thatα(x) = 1 fora_ϕ+δ₀≤ |x| ≤b_ϕ−δ₀ andα(x) = 0 for|x| ≥b_ϕ or|x| ≤a_ϕ, then we define

f˜(t, y) =α(y)f(t, y) =

⎧⎪

⎨

⎪⎩

f(t, y), a_ϕ+δ₀≤ |y| ≤b_ϕ−δ₀,

α(y)f(t, y) a_ϕ≤ |y| ≤a_ϕ+δ₀, orb_ϕ−δ₀≤ |y| ≤b_ϕ, 0, |y| ≥b_ϕ, or|y| ≤a_ϕ.

We define ˜g(t, y) by the same method, then we consider the following equation ξ˜_s=x+

_s

t

(b(t, x) + ˜f(r,ξ˜_r))dr+ _s

t

(σ(t, x) + ˜g(r,ξ˜_r))dB^H(r), s∈[t, t+ ¯h]. (3.2) Since ϕ∈ Hand U =U^t,x, V =V^t,x satisfy (2.5), (2.6) with d=m, we can verify that for α∈(1−H, α₀), f˜(t, y), and ˜g(t, y) satisfy the conditions of (H₁), (H₂) from [11], where M₀, M_R, L₀, L_R depend onω. Hence for allα∈(1−H, α₀), equation (3.2) has a unique solution ˜ξ_·(ω)∈L⁰

Ω,F,P;W^α,∞(t, T;R^d)

and moreover ξ˜_·(ω)∈C^1−α

t, t+ ¯h;R^d

,P-a.s. Sincea_ϕ+ 2δ₀≤ |x| ≤b_ϕ−2δ0, there exists a random variable ¯h₁= ¯h^t,x₁ such thata_ϕ+δ₀≤ |ξ| ≤˜ b_ϕ−δ₀,P-a.s. Then fors∈[t, t+ ¯h₁], (3.2) becomes to

ξ˜_s=x+ _s

t

(b(t, x) +f(r,ξ˜_r))dr+ _s

t

(σ(t, x) +g(r,ξ˜_r))dB^H(r), s∈[t, t+ ¯h₁], P−a.s.

Takingξ_s= ˜ξ_s, s∈[t, t+ ¯h₁], together with the uniqueness of ˜ξ_s, the proof is completed.

Theorem 3.5. Assume that (H₁) and (H₂) are satisfied with 1/2 < H < 1, 1 −H < β, δ > ^1−H_H . Set max{1−H,1−μ} < α < α₀. Let K(t) = K(t) ⊂ R^d, t ∈ [0, T] and ϕ ∈ H. Then for all x ∈ R^d with a_ϕ<|x|< b_ϕ,

(b(t, x), σ(t, x))∈S_ϕ−1(ϕ(K(t)))(t, x) if and only if

(ϕ(x)b(t, x), ϕ(x)σ(t, x))∈S_ϕ(K(t))(t, ϕ(x)).

Proof. It is sufficient to prove that if (ϕ(x)b(t, x), ϕ(x)σ(t, x)) ∈ S_ϕ(K(t))(t, ϕ(x)), then (b(t, x), σ(t, x)) ∈ S_ϕ−1(ϕ(K(t)))(t, x).

Since

(ϕ(x)b(t, x), ϕ(x)σ(t, x))∈S_ϕ(K(t))(t, ϕ(x)),

for x ∈ K(t), there exist a random variable ¯h = ¯h^t,x > 0 and two stochastic processes U₁ and V₁ satisfy- ing (2.5), (2.6) withd=m, and for alls∈

t, t+ ¯h

, for everyR >0 with |x| ≤R, ϕ(x) +

_s

t

(ϕ(x)b(t, x) +U₁(r))dr+ _s

t

(ϕ(x)σ(t, x) +V₁(r))dB^H(r)∈ϕ(K(s)).

Let

f(r, y) =ϕ(y)⁺[U₁(r)−(ϕ(y)−ϕ(x))b(t, x)], g(r, y) =ϕ(y)⁺[V₁(r)−(ϕ(y)−ϕ(x))σ(t, x)],

whereϕ(y)⁺is the right inverse ofϕ(y). By Lemma3.4, for everyδ₀>0 anda_ϕ+ 2δ₀≤ |x| ≤b_ϕ−2δ₀, there exists a random variable ¯h₁ such that forP-a.s.ω∈Ω, the following SDE

ξ_s=x+ _s

t

(b(t, x) +f(r, ξ_r))dr+ _s

t

(σ(t, x) +g(r, ξ_r))dB^H(r), s∈[t, t+ ¯h₁], has a unique solutionξ_·(ω). Taking

U(r) =ϕ(ξ_r)⁺[U₁(r)−(ϕ(ξ_r)−ϕ(x))b(t, x)], V(r) =ϕ(ξ_r)⁺[V₁(r)−(ϕ(ξ_r)−ϕ(x))σ(t, x)],

forr∈[t, t+ ¯h₁] andU(r) =U(t+ ¯h₁), V(r) =V(t+ ¯h₁) forr∈[t+ ¯h₁, T]. We deduce that U(t) = 0, V(t) = 0.

Sinceϕ∈ Handξ_·(ω)∈C^1−α

t, t+ ¯h₁;R^d

, from (H₁) and (H₂), it follows that

|U(s)−U(τ)| ≤θ|s−τ|^1−α, |V (s)−V(τ)| ≤θ˜|s−τ|^{min{β,1−α}}, ∀s, τ ∈[t, t+ ¯h₁].

According to the fractional Itˆo formula, we know for alls∈

t, t+ ¯h₁ that ϕ

x+

_s

t

(b(t, x) +U(r))dr+ _s

t

(σ(t, x) +V(r))dB^H(r)

=ϕ(x) + _s

t

(ϕ(x)b(t, x) +U₁(r))dr+ _s

t

(ϕ(x)σ(t, x) +V₁(r))dB^H(r)∈ϕ(K(s)).

which means that

(b(t, x), σ(t, x))∈S_ϕ−1(ϕ(K(t)))(t, x).

4. Applications to deterministic characterization of viability

Using Theorem 3.5, if K takes some particular forms and B^H is one-dimensional, we will get determinis- tic sufficient and necessary conditions for viability which can be checked more easily. Firstly we prove some preliminary results.

Lemma 4.1. Assume that (H₁) and(H₂) are satisfied withk = 1,1/2 < H <1, 1−H < β, δ > ^1−H_H . Let max{1−H,1−μ}< α < α₀ andK={x∈R^d;r≤ |x| ≤R}. Then for all x, such that |x|=R,

(b(t, x), σ(t, x))∈S_K(t, x)if and only if x, b(t, x) ≤0, x, σ(t, x)= 0, and for allx, such that|x|=r,

(b(t, x), σ(t, x))∈S_K(t, x)if and only if x, b(t, x) ≥0, x, σ(t, x)= 0.

Proof. Takingϕ(x) =|x|², forR/4≤ |x| ≤4R, we have ϕ(x)⁺= x

2|x|²·

We can verify thatϕ∈ H witha_ϕ =R/4,b_ϕ = 4R. For |x| =R, we know that a_ϕ<|x|< b_ϕ. We have also similar discussion with respect to|x|=r. By Theorem3.5, Lemma4.1is reduced to the following equivalence:

For allxsuch that|x|=R

(2x, b(t, x),2x, σ(t, x))∈S_ϕ(K)(t,|x|²)⇔ x, b(t, x) ≤0, x, σ(t, x)= 0.

For allx, such that |x|=r,

(2x, b(t, x),2x, σ(t, x))∈S_ϕ(K)(t,|x|²)⇔ x, b(t, x) ≥0, x, σ(t, x)= 0.

It’s convenient to prove only the case:|x|=R, the other one is similar.

Necessary.Ifx, b(t, x) ≤0, x, σ(t, x)= 0, takingU(r)≡0, V(r)≡0, we can choose ¯hsmall enough, such that∀s∈[t, t+ ¯h],

r²≤ |x|²+ _s

t

(2x, b(t, x)+U(y))dy+ _s

t

(2x, σ(t, x)+V(y))dB^H(y)≤R². This means that (2x, b(t, x),2x, σ(t, x))∈S_ϕ(K)(t,|x|²).

Sufficient. Since (2x, b(t, x),2x, σ(t, x)) ∈ S_ϕ(K)(t,|x|²), there exist a random variable ¯h =

¯h^t,x > 0, and two stochastic processes satisfying (2.5), (2.6) with d = 1, k = 1, and for all s ∈

t, t+ ¯h , for|x| ≤R,

r²≤ |x|²+ _s

t

(2x, b(t, x)+U(y))dy+ _s

t

(2x, σ(t, x)+V(y))dB^H(y)≤R². Since|x|=R, it follows that

2x, b(t, x)(s−t) +2x, σ(t, x)(B^H(s)−B^H(t)) + _s

t

U(y)dy+ _s

t

V(y)dB^H(y)≤0. (4.1) By (2.3), there existsΩ₀ ⊂Ω withP(Ω₀) = 1/2, such that for eachω₀ ∈Ω₀, (4.1) is satisfied and there is a sequencet≤s_n =s_n(ω₀)≤t+ ¯h(ω₀),s_nt, such that

slimnt

B_s^H_n(ω₀)−B_t^H(ω₀)

s_n−t = +∞. (4.2)

Then we conclude that

2x, σ(t, x)B^H_ω0(s_n)−B^H_ω0(t)

s_n−t +

_sn

t

U(y)dy+ _sn

t

V(y)dB^H_ω0(y) 1

s_n−t ≤ −2x, b(t, x). (4.3)

By Lemma3.1 _sn

t

U(y)dy+ _sn

t

V(y)dB_ω^H₀(y) 1

s_n−t →0, recalling (4.2), yields

x, σ(t, x) ≤0.

Similarly we can prove thatx, σ(t, x) ≥0, choosingω₀ and a sequencet≤r_n =r_n(ω₀)≤t+ ¯h(ω₀),r_n t, such that lim

rnt

B^H_rn(ω₀)−B^Ht (ω₀)

rn−t =−∞. Consequently,

x, σ(t, x)= 0.

Then from (4.3), we deduce that

2x, b(t, x)+ _s

t

U(y)dy+ _s

t

V(y)dB^H(y) 1

s−t ≤0.

Lets→t.ViaLemma3.1, it follows that

x, b(t, x) ≤0.

As particular cases, we have the following two corollaries

Corollary 4.2. Assume that(H₁) and(H₂)are satisfied withk= 1,1/2< H <1, 1−H < β, δ > ^1−H_H . Let max{1−H,1−μ}< α < α₀ andK={x∈R^d;|x|= 1} (the unit sphere). Then for allx∈K,

(b(t, x), σ(t, x))∈S_K(t, x)if and only if x, b(t, x)= 0, x, σ(t, x)= 0.

Corollary 4.3. Assume that(H₁) and(H₂)are satisfied withk= 1,1/2< H <1, 1−H < β, δ > ^1−H_H . Let max{1−H,1−μ}< α < α₀ andK={x∈R^d;|x| ≤1} (the unit ball). Then for allx, such that|x|= 1,

(b(t, x), σ(t, x))∈S_K(t, x)if and only if x, b(t, x) ≤0, x, σ(t, x)= 0.

As it is shown in Corollary2.12, if we want to get conditions for the viability ofK, we only need to consider the starting pointx∈∂K. Then, from Corollaries4.2and4.3, we have respectively the following two results.

Proposition 4.4. Let (H₁), (H₂) be satisfied with k = 1,1/2 < H < 1, 1 −H < β, δ > ^1−H_H . Let max{1−H,1−μ}< α < α₀ andK be the unit sphere. Then, the following assertions are equivalent:

(I) K is viable for the fractional SDE(2.4);

(II) for all t∈[0, T]and allx∈K,

x, b(t, x)= 0, x, σ(t, x)= 0.

Proposition 4.5. Let (H₁), (H₂) be satisfied with k = 1,1/2 < H < 1, 1 −H < β, δ > ^1−H_H . Let max{1−H,1−μ}< α < α₀ andK be the unit ball. Then the following assertions are equivalent:

(I) K is viable for the fractional SDE(2.4);

(II) for all t∈[0, T]and all|x|= 1,

x, b(t, x) ≤0, x, σ(t, x)= 0.

As a consequence of Theorem3.5, we have the following Corollary 4.6. Considering the one-dimensional SDE,

X_s=x+ _s

t

b(r, X_r)dr+ _s

t

σ(r, X_r) dB_r^H, s∈[t, T].

Suppose that (H₁) and (H₂) are satisfied with k = 1. Then, for any t ∈ [0, T] and every x ≥ 0, the above equation has a positive solution if and only if

b(t,0)≥0, σ(t,0) = 0, ∀t∈[0, T].

Proof. Taking ϕ(x) = x and K = [0,+∞). Analysis similar to that in the proof of Lemma 4.1 shows that (b(t,0), σ(t,0)) ∈ S_K(t,0) if and only if b(t,0) ≥ 0, σ(t,0) = 0. Considering Corollary 2.12, we get our

result.

Another interesting application is the comparison of the solution theorem.