Qualitative properties of certain piecewise deterministic Markov processes

www.imstat.org/aihp 2015, Vol. 51, No. 3, 1040–1075

DOI:10.1214/14-AIHP619

© Association des Publications de l’Institut Henri Poincaré, 2015

Qualitative properties of certain piecewise deterministic Markov processes

Michel Benaïm

, Stéphane Le Borgne

, Florent Malrieu

and Pierre-André Zitt

aInstitut de Mathématiques, Université de Neuchâtel, 11 rue Émile Argand, 2000 Neuchâtel, Suisse. E-mail:[email protected] bIRMAR UMR 6625, CNRS-Université de Rennes 1, Campus de Beaulieu, 35042 Rennes Cedex, France.

E-mail:[email protected]

cLaboratoire de Mathématiques et Physique Théorique (UMR CNRS 7350), Fédération Denis Poisson (FR CNRS 2964), Université François-Rabelais, Parc de Grandmont, 37200 Tours, France. E-mail:[email protected]

dLAMA UMR 8050, CNRS-Université-Paris-Est-Marné-La-Vallée, 5, boulevard Descartes, Cité Descartes, Champs-sur-Marne, 77454 Marne-la-Vallée Cedex 2, France. E-mail:[email protected]

Received 23 May 2012; revised 7 April 2014; accepted 12 April 2014

Abstract. We study a class of piecewise deterministic Markov processes with state spaceR^d×EwhereEis a finite set. The continuous component evolves according to a smooth vector field that is switched at the jump times of the discrete coordinate.

The jump rates may depend on the whole position of the process. Working under the general assumption that the process stays in a compact set, we detail a possible construction of the process and characterize its support, in terms of the solutions set of a differential inclusion. We establish results on the long time behaviour of the process, in relation to a certain set of accessible points, which is shown to be strongly linked to the support of invariant measures. Under Hörmander-type bracket conditions, we prove that there exists a unique invariant measure and that the processes converges to equilibrium in total variation. Finally we give examples where the bracket condition does not hold, and where there may be one or many invariant measures, depending on the jump rates between the flows.

Résumé. Nous étudions une classe de processus de Markov déterministes par morceaux, sur espace d’étatsR^d×EoùEest un ensemble fini. La composante continue du processus évolue suivant le flot d’un champ de vecteur, qui change lorsque la composante discrète saute. Les taux de saut peuvent dépendre des deux composantes. Sous l’hypothèse que le processus reste dans un ensemble compact, nous détaillons une construction possible et caractérisons son support en termes de solution d’une inclusion différentielle. Nous étudions ensuite le comportement en temps long, en faisant apparaître un certain ensemble de points accessibles, qui se trouve être fortement lié au support des mesures invariantes. Sous des conditions de type Hörmander sur les crochets de Lie entre les champs de vecteurs, nous montrons qu’il existe une unique mesure invariante vers laquelle le processus converge en variation totale. Nous donnons enfin des exemples où la condition d’unicité n’est pas vérifiée, et où le nombre de mesures invariantes dépend des taux de saut entre les flots.

MSC:60J99; 34A60

Keywords:Piecewise deterministic Markov process; Convergence to equilibrium; Differential inclusion; Hörmander bracket condition

1. Introduction

Piecewise deterministic Markov processes (PDMPs in short) are intensively used in many applied areas (molecular biology [27], storage modelling [6], Internet traffic [14,17,18], neuronal activity [7,25], . . . ). Roughly speaking, a Markov process is a PDMP if its randomness is only given by the jump mechanism: in particular, it admits no diffusive dynamics. This huge class of processes has been introduced by Davis [10]. See [11,19] for a general presentation.

In the present paper, we deal with an interesting subclass of the PDMPs that plays a role in molecular biology [7,27] (see also [29] for other motivations). We consider a PDMP evolving onR^d×E, whered≥1 andEis a finite set, as follows: the first coordinate moves continuously on R^d according to a smooth vector field that depends on the second coordinate, whereas the second coordinate jumps with a rate depending on the first one. Of course, most of the results in the present paper should extend to smooth manifolds. This class of Markov processes is reminiscent of the so-called iterated random functions in the discrete time setting (see [12] for a good review of this topic).

We are interested in the long time qualitative behaviour of these processes. A recent paper by Bakhtin and Hurth [2] considers the particular situation where the jump rates are constant and prove the beautiful result that, under a Hörmander-type condition, if there exists an invariant measure for the process, then it is unique and absolutely continuous with respect to the “Lebesgue” measure onR^d×E. Here we consider a more general situation and focus also on the convergence to equilibrium. We also provide a basic proof of the main result in [2].

Let us define our process more precisely. LetEbe a finite set, and for anyi∈E,Fⁱ:R^d→R^dbe a smooth vector field. We assume throughout that each Fⁱ is bounded and we denote byC_sp an upper bound for the “speed” of the deterministic dynamics:

sup

x∈R^d,i∈E

Fⁱ(x)≤C_sp<∞.

We letΦⁱ= {Φ_tⁱ}denote the flow induced byFⁱ. Recall that t→Φ_tⁱ(x)=Φⁱ(t, x)

is the solution to the Cauchy problem x˙=Fⁱ(x)with initial conditionx(0)=x. Moreover, we assume that there exists a compact setM⊂R^dthat is positively invariant under eachΦⁱ, meaning that:

∀i∈E,∀t≥0, Φ_tⁱ(M)⊂M. (1)

We consider here a continuous time Markov process(Z_t =(X_t, Y_t))living onM×Ewhose infinitesimal generator acts on functions

g:M×E→R,

(x, i)→g(x, i)=gⁱ(x), smooth¹inx, according to the formula

Lg(x, i)=

Fⁱ(x),∇gⁱ(x)

+

j∈E

λ(x, i, j )

g^j(x)−gⁱ(x)

, (2)

where

(i) x→λ(x, i, j )is continuous;

(ii) λ(x, i, j )≥0 fori=j andλ(x, i, i)=0;

(iii) for eachx∈M, the matrix(λ(x, i, j ))_ijis irreducible.

The process is explicitly constructed in Section2 and some of its basic properties (dynamics, invariant and em- pirical occupation probabilities) are established. In Section3.1we describe (Theorem3.4) the support of the law of the process in terms of the solutions set of a differential inclusion induced by the collection{Fⁱ: i∈E}. Section3.2 introduces theaccessible setwhich is a natural candidate to support invariant probabilities. We show (Proposition3.9) that this set is compact, connected, strongly positively invariant and invariant under the differential inclusion induced by{Fⁱ: i∈E}. Finally, we prove that, if the process has a unique invariant probability measure, its support is charac- terized in terms of the accessible set.

1Meaning thatgⁱis the restriction toMof a smooth function onR^d.

Section4contains the main results of the present paper. We begin by a slight improvement of the regularity results of [2]: under Hörmander-like bracket conditions, the law of the process after a large enough number of jumps or at a sufficiently large time has an absolutely continuous component with respect to the Lebesgue measure onR^d×E.

Moreover, this component may be chosen uniformly with respect to the initial distribution. The proofs of these results are postponed to Sections6and7. We use these estimates in Section4.2to establish the exponential ergodicity (in the sense of the total variation distance) of the process under study. In Section5, we show that our assumptions are sharp thanks to several examples. In particular, we stress that, when the Hörmander condition is violated, the uniqueness of the invariant measure may depend on the jump mechanism between flows, and not only on the flows themselves.

Remark 1.1 (Quantitative results). In the present paper,we essentially deal with qualitative properties of the asymp- totic behavior for a large class of PDMPs.Under more stringent assumptions, [4]gives an explicit rate of convergence in Wasserstein distance,via a coupling argument.

Remark 1.2 (Compact state space). The main results in the present paper are still valid even if the state space is no longer compact provided that the excursions out of some compact sets are suitably controlled(say with a Lyapunov function).Nevertheless,as shown in[5],the stability of Markov processes driven by an infinitesimal generator as (2)may depend on the jump rates.As a consequence,it is difficult to establish,in our general framework,sufficient conditions for the stability of the process under study without the invariance assumption(1);results in this direction may however be found in the recent[8].

2. Construction and basic properties

In this section we explain how to construct explicitly the process(Z_t)_t≥0driven by (2). Standard references for the construction and properties of more general PDMPs are the monographs [11] and [19]. In our case the compactness allows a nice construction via a discrete process whose jump times follow an homogeneous Poisson process, similar to the classical “thinning” method for simulating nonhomogeneous Poisson processes (see [22,28]).

2.1. Construction

SinceMis compact and the mapsλ(·, i, j )are continuous, there existsλ∈R₊such that

x∈maxM,i∈E

j∈E,j=i

λ(x, i, j ) < λ.

Let us fix such aλ, and let Q(x, i, j )=λ(x, i, j )

λ , fori=j and Q(x, i, i)=1−

j=i

Q(x, i, j ) >0.

Note thatQ(x)=(Q(x, i, j ))_i,j∈Eis an irreducible aperiodic Markov transition matrix and that (2) can be rewritten as

Lg=Ag+λ(Qg−g), where

Ag(x, i)=

Fⁱ(x),∇gⁱ(x)

and Qg(x, i)=

j∈E

Q(x, i, j )g^j(x). (3)

Let us first construct a discrete time Markov chain(Z˜_n)_n≥0. Let(N_t)_t≥0be a homogeneous Poisson with intensityλ;

denote by(Tn)_n≥0its jump times and(Un)_n≥1its interarrival times. LetZ˜0∈M×Ebe a random variable independent

of(N_t)_t≥0. Define(Z˜_n)_n=(X˜_n,Y˜_n)_nonM×Erecursively by:

X˜_n+1=Φ^Y˜n(U_n+1,X˜_n),

P[ ˜Yn+1=j| ˜Xn+1,Y˜n=i] =Q(X˜n+1, i, j ).

Now define(Z_t)_t≥0via interpolation by setting

∀t∈ [T_n, T_n+1), Z_t=

Φ^Y˜n(t−T_n,X˜_n),Y˜_n

. (4)

The memoryless property of exponential random variables makes(Z_t)_t≥0a continuous time càdlàg Markov process.

We let P =(P_t)_t≥0 denote the semigroup induced by (Z_t)_t≥0. Denoting by C0 (resp.C1) the set of real valued functionsf:M×E→Rthat are continuous (resp. continuously differentiable) in the first variable, we have the following result.

Proposition 2.1. The infinitesimal generator of the semigroupP =(Pt)_t≥0is the operatorLgiven by(2).Moreover, Pt isFeller,meaning that it mapsC0into itself and,forf∈C0, limt→0Ptf−f =0.

The transition operatorP˜ of the Markov chainZ˜ also mapsC0to itself,and ifK_tandK˜ are defined by

K_tg(x, i)=g

Φ_tⁱ(x), i

and Kf˜ = _∞

0

λe^−λtK_tfdt, (5)

thenP˜ can be written as:

P g(x, i)˜ =E g(Z˜₁)|Z₀=(x, i)

= _∞

0

K_tQg(x, i)λe^−λtdt

= ˜KQg(x, i). (6)

Proof. For eacht≥0,Pt acts on bounded measurable mapsg:M×E→Raccording to the formula P_tg(x, i)=E g(Z_t)|Z₀=(x, i)

. Fort≥0 letJ_t=K_tQ. It follows from (4) that

P_tg=

n≥0

E[1_{Nt=n}J_U1◦ · · · ◦J_Un◦K_t−Tng]. (7)

By Lebesgue continuity theorem and (7),P_tg∈C0wheneverg∈C0. Moreover, setting apart the first two terms in (7) leads to

P_tg=e^−λtK_tg+λe^−λt _t

0

K_uQK_t−ugdu+R(g, t ), (8)

where|R(g, t )| ≤ gP[N_t>1] = g(1−e^−λt(1+λt )). Therefore limt→0P_tg−g =0.

The infinitesimal generator of(K_t)is the operatorAdefined by (3). Thus ¹_t(K_tg−g)→Ag,therefore, by (8), P_tg−g

t −−−→

t→0 Ag−λg+λQg,

and the result on(P_t)follows. The expression (6) ofP˜ is a consequence of the definition of the chain. From (6) one

can deduce thatP˜ is also Feller.

Remark 2.2 (Discrete chains and PDMPs). The chainZ˜ records all jumps of the discrete partY of the PDMPZ, but,sinceQ(x, i, i) >0,Z˜ also contains “phantom jumps” that cannot be seen directly on the trajectories ofZ.

Other slightly different discrete chains may crop up in the study of PDMPs.The most natural one is the process observed at(true)jump times.In another direction,the chain(Θ_n)_n∈Nintroduced in[9]corresponds(in our setting)to the addition of phantom jumps at rate1.For this chain(Θ_n),the authors prove(in a more general setting)equivalence between stability properties of the discrete and continuous time processes.

Similar equivalence properties will be shown below for our chainZ˜ (Proposition2.4,Lemma2.5).Its advantage lies in the simplicity of its definition;in particular it leads to a simulation method that does not require the integration of jump rates along trajectories.

Notation. Throughout the paper we may writePx,i[·]forP[·|Z₀=(x, i)]andEx,i[·]forE[·|Z₀=(x, i)].

2.2. First properties of the invariant probability measures

LetM(M×E)(resp.M⁺(M×E)andP(M×E)) denote the set of signed (resp. positive, and probability) measures onM×E. Forμ∈M(M×E)andf ∈L¹(μ)we writeμf for

fdμ. Given a bounded operatorK:C0→C0and μ∈M(M×E)we letμK∈M(M×E)denote the measure defined by duality:

∀g∈C0, (μK)g=μ(Kg).

The mappingsμ→μPt andμ→μP˜preserve the setsM⁺(M×E)andP(M×E).

Definition 2.3 (Notation, stability). We denote byPinv(resp.P˜inv)the set of invariant probabilities for(P_t)(resp.P˜), μ∈Pinv ⇐⇒ ∀t≥0, μP_t=μ;

μ∈ ˜Pinv ⇐⇒ μP˜=μ.

We say that the processZ(orZ)˜ isstableif it has a unique invariant probability measure.

Forn∈N^∗andt >0we letΠ˜_nandΠ_t the(random)occupation measures defined by Π˜n=1

n n k=1

δ_Z˜

k and Πt=1

t _t

0

δZ_t.

By standard results for Feller chains on a compact space (see e.g. [13]), the setP˜invis nonempty, compact (for the weak-topology) and convex. Furthermore, with probability one every limit point of(Π_n)_n≥1lies inP˜inv.

The following result gives an explicit correspondence between invariant measures for the discrete and continuous processes.

Proposition 2.4 (Correspondence for invariant measures). The mappingμ→μK˜ mapsP˜invhomeomorphically ontoPinvand extremal points ofP˜inv(i.e.ergodic probabilities forP˜)onto extremal points ofPinv(ergodic probabil- ities forP_t).

The inverse homeomorphism is the mapμ→μQrestricted toPinv.

Consequently,the continuous time process(Z_t)is stable if and only if the Markov chain(Z˜_n)is stable.

Proof. For allf ∈C1, integrating by parts_∞

0 dKtf

dt e^−λtdtand using the identities ^dK_dt^tf =AK_tf =K_tAf leads to

K(λI˜ −A)f =λf=(λI−A)Kf.˜ (9)

Letμ∈P(M×E). Then, using (9) and the form ofLgives

μKLf˜ =μK(A˜ −λI )f+λμKQf˜ =λ(−μf +μP f ),˜ (10)

μL(Kf )˜ =μ(A−λI )Kf˜ +λμQKf˜ =λ(−μf +μQKf ).˜ (11)

Ifμ∈ ˜Pinv, (10) implies(μK)Lf˜ =0 for allf ∈C1and sinceC1is dense inC0this proves thatμK˜∈Pinv. Similarly, ifμ∈Pinv, (11) impliesμ=μQK. Hence˜ μQ=μQKQ˜ =μQP˜ proving thatμQ∈ ˜Pinv. Furthermore the identity μ=μQK˜ for allμ∈Pinvshows that the mapsμ→μK˜ andμ→μQare inverse homeomorphisms.

Lemma 2.5 (Comparison of empirical measures). Letf:M×E→Rbe a bounded measurable function.Then

tlim→∞Π_tf− ˜Π_NtKf˜ =0 and lim

n→∞Π_Tnf− ˜Π_nKf˜ =0 with probability one.

Proof. Decomposing the continuous time interval[0, t]along the jumps yields:

Πtf =N_t t

1 N_t

Nt−1 i=0

_Ti+1

T_i

f (Zs)ds+rt

,

wherer_t ≤ f^UNt_Nt⁺¹.

Now N_t → ∞ a.s. andP[U_n/n≥ε] =e^−λnε, so that r_t →0 a.s. Since tf is bounded and N_t/t →λ, this implies:

Πtf − λ N_t

Nt−1 i=0

_Ti+1

T_i

f (Zs)ds−−−→^a.s.

t→∞ 0.

Now, note that λ

_Ti+1

T_i

f (Z_s)ds=λ _Ui+1

0

f

φ_s^Y˜i(X˜_i),Y˜_i ds=λ

_∞

0

1s≤U_i+1K_sf (X˜_i,Y˜_i)ds.

Integrating theU_i+1yieldsKf (˜ X˜_i,Y˜_j), therefore Mn=

n−1

i=0

λ

T_i+1 Ti

f (Zs)ds− ˜Kf (X˜i,Y˜i)

is a martingale with increments bounded inL²:E[(Mn+1−Mn)²] ≤2f². Therefore, by the strong law of large numbers for martingales, limn→∞Mn

n =0 almost surely, and the result follows.

As in the discrete time framework, the setPinvis nonempty compact (for the weak-topology) and convex. Fur- thermore, with probability one, every limit point of(Π_t)_t≥0lies inPinv.

Finally, one can check that an invariant measure for the embedded chain and its associated invariant measure for the time continuous process have the same support. Givenμ∈P(M×E)let us denote by supp(μ)its support.

Lemma 2.6. Letμ∈ ˜Pinv.ThenμandμK˜ have the same support.

Proof. Let(x, i)∈supp(μ)and letU be a neighborhood ofx. Then fort₀>0 small enough and 0≤t≤t₀,Φ₋ⁱ_t(U) is also a neighborhood ofx. Thus

(μK)˜

U× {i}

= _∞

0

λe^−λtμ

Φ₋ⁱ_t(U)× {i} dt≥λ

_t0

0

e^−λtμ

Φ₋ⁱ_t(U)× {i} dt >0.

This proves that supp(μ)⊂supp(μK). Conversely, let˜ ν=μK˜ and(x, i)∈supp(ν)and letU be a neighborhood ofx. Then

μ

U× {i}

=(νQ)

U× {i}

=

j∈E

UQj i(x)ν

dx× {j}

≥

UQii(x)ν

dx× {i}

>0.

As a consequence, supp(μ)⊃supp(μK).˜ 2.3. Law of pure types

Letλ_M andλ_M×E denote the Lebesgue measures onMandM×E.

Proposition 2.7 (Law of pure types). Letμ∈ ˜Pinv(resp.Pinv)and letμ=μac+μsbe the Lebesgue decomposition ofμwithμacthe absolutely continuous(with respect toλM×E)measure andμsthe singular(with respect toλM×E) measure.Then bothμ_acandμ_sare inP˜inv(resp.Pinv),up to a multiplicative constant.In particular,ifμis ergodic, thenμis either absolutely continuous or singular.

Proof. The key point is thatK˜ andQ, henceP˜= ˜KQ, map absolutely continuous measures into absolutely contin- uous measures. Forμ∈ ˜Pinvthe result now follows from the following simple Lemma2.8applied toP˜. Lemma 2.8. Let (Ω,A, P ) be a probability space.Let M (respectively M⁺,P,Mac)denote the set of signed (positive,probability,absolutely continuous with respect toP)measures on Ω.Let K:M→M, μ→μK be a linear map that maps each of the preceding sets into itself.Then if μ∈P is a fixed point forK with Lebesgue decompositionμ=μ_ac+μ_s,bothμ_acandμ_sare fixed points forK.

Proof. WriteμK =μacK+μsK=μacK+νac+νs withμsK=νac+νs the Lebesgue decomposition ofμsK.

Then, by uniqueness of the decomposition,μac=μacK+νac. Thus,μac≥μacK. Now eitherμac=0 and there is nothing to prove or, we can normalize byμ_ac(Ω)and we get thatμ_ac=μ_acK.

3. Supports and accessibility

3.1. Support of the law of paths

In this section, we describe the shape of the support of the distribution of(X_t)_t≥0and we show that it can be linked to the set of solutions of a differential inclusion (which is a generalisation of ordinary differential equations).

Let us start with a definition that will prove useful to encode the paths of the processZ_t. Definition 3.1 (Trajectories and adapted sequences). For alln∈N^∗let

Tn=

(i,u)=

(i₀, . . . , i_n), (u₁, . . . , u_n)

∈Eⁿ⁺¹×Rⁿ₊ and

T^i,jn =

(i,u)∈Tn: i0=i, in=j .

Given(i,u)∈Tn andx∈M,define(x_k)_0≤k≤n by induction by settingx₀=x and x_k+1=Φ_u^ik_k⁻¹(x_k):these are the points obtained by followingFⁱ⁰ for a timeu₀,thenFⁱ¹ for a timeu₁,etc.

We also define a corresponding continuous trajectory.Lett=(t₀, . . . , t_n)be defined byt₀=0andt_k=t_k−1+u_k fork=1,2, . . . , nand let(ηx,i,u(t))_t≥0be the function(η(t))_t≥0given by

ηx,i,u(t)=η(t )=

⎧⎨

⎩

x ift=0,

Φ_tⁱ₋^k−1_tk−1(x_k−1) ift_k−1< t≤t_kfork=1, . . . , n, Φ_tⁱ₋ⁿ_t

n(x_n) ift > t_n.

(12)

Finally,letp(x,i,u)=Q(x₁, i₀, i₁)Q(x₂, i₁, i₂)· · ·Q(x_n, i_n−1, i_n),and Tn,ad(x)=

(i,u)∈Tn: p(x,i,u) >0 . An element ofTn,ad(x)is said to beadapted tox∈M.

Lemma 3.2. Let(i,u)∈Tn.Then,for any(x, i)∈M×E,anyT ≥0,and anyδ >0, Px,i

sup

0≤t≤T

X_t−η_x,i,u(t)≤δ

>0.

Proof. Suppose first that(i,u)∈Tnis adapted toxand thatistarts ati. By continuity, there existδ₁andδ₂such that

i=max1,...,n|s_i−u_i| ≤δ₁ ⇒

sup_0≤t≤Tη_x,i,s(t)−η_x,i,u(t) ≤δ, p(x,i,s)≥δ₂.

Let (U₁, . . . , U_n, U_n+1)ben+1 independent random variables with an exponential law of parameterλandU= (U₁, . . . , U_n). Then,

Px,i

sup

0≤t≤T

X_t−η_x,i,u(t)≤δ

≥P

l=max1,...,n|Ul−ul| ≤δ1, Un+1≥T −tn+δ1, (Y˜0, . . . ,Y˜n)=i

≥δ₂P

l=max1,...,n|U_l−u_l| ≤δ₁, U_n+1≥T −t_n+δ₁

≥δ2

_n

l=1

e^{−λ(ul−δ1)}−e^{−λ(ul+δ1)}

e^{−λ(T−tn+δ1)}>0.

In the general case,(i,u)∈Tnis not necessarily adapted and may start at an arbitraryi₀. However, for anyT >0 and δ >0, there exists(j,v)∈Tn,ad(x)(for somen≥n) such thatj₀=iand

sup

0≤t≤T

ηx,j,v(t)−ηx,i,u(t)≤δ,

sinceQ(x) is, by construction, irreducible and aperiodic (this allows to add permitted transitions fromi toi0and whereihas not permitted transitions, with times between the jumps as small as needed).

After these useful observations, we can describe the support of the law of(X_t)_t≥0in terms of a certain differential inclusion induced by the vector fields{Fⁱ: i∈E}.

For eachx∈R^d, let co(F )(x)⊂R^dbe the compact convex set defined as co(F )(x)=

i∈E

α_iFⁱ(x): α_i≥0,

i∈E

α_i =1

.

LetC(R₊,R^d)denote the set of continuous pathsη:R₊→R^d equipped with the topology of uniform convergence on compact intervals. A solution to the differential inclusion

˙

η∈co(F )(η) (13)

is an absolutely continuous functionη∈C(R₊,R^d)such thatη(t)˙ ∈co(F )(η(t))for almost allt∈R₊. We letS^x⊂ C(R₊,R^d)denote the set of solutions to (13) with initial conditionη(0)=x.

Lemma 3.3. The setS^xis a nonempty compact connected set.

Proof. Follows from standard results on differential inclusion, since the set-valued map co(F )is upper semicontinu-

ous, bounded with nonempty compact convex images; see [1] for details.

Theorem 3.4. IfX0=x∈Mthen,the support of the law of(Xt)_t≥0equalsS^x.

Proof. Obviously, any path ofXis a solution of the differential inclusion (13). Letη∈S^x, andε >0. Set Gt(x)=

v∈

Fⁱ(x), i∈E :

v− ˙η(t), x−η(t )

< ε .

Since η(t)˙ ∈co(F )(η(t)) almost surely, G_t(x)is nonempty. Furthermore, (t, x)→G_t(x)is uniformly bounded, lower semicontinuous inx, and measurable int. Hence, using a result by Papageorgiou [26], there existsξ:R→R^d absolutely continuous such thatξ(0)=xandξ (t )˙ ∈G_t(ξ(t))almost surely. In particular,

d

dtξ(t )−η(t )²=2ξ (t )˙ − ˙η(t), ξ(t)−η(t )

<2ε so that

sup

0≤t≤T

ξ(t )−η(t )²≤2εT .

Thus, without loss of generality, one can assume thatηis such that

˙

η(t)∈

i∈E

Fⁱ η(t )

for almost allt∈R₊. Set

∀i∈E, Ω_i=

t∈ [0, T]: η(t)˙ =Fⁱ η(t )

.

LetCbe the algebra consisting of finite unions of intervals in[0, T]. Since the Borelσ-field over[0, T]is generated byC, there exists, for alli∈E,J_i∈Csuch that, the set{J_i: i∈E}forms a partition of[0, T], and fori∈E

λ(Ω_iJ_i)≤ε,

whereλ stands for the Lebesgue measure over[0, T]andAB is the symmetric difference ofA andB. Hence, there exist numbers 0=t₀< t₁<· · ·< t_N+1=T and a mapi:k→i_k from{0, . . . , N}to{1, . . . , n0}, such that (t_k, t_k+1)⊂J_ik. Introduceη_x,i,ugiven by formula (12). For allt_k≤t≤t_k+1,

η_x,i,u(t)−η(t )=η_x,i,u(t_k)−η(t_k) +

t tk

F^ik

ηx,i,u(s)

−F^ik η(s)

ds+ t

tk

F^ik η(s)

− ˙η(s) ds.

Hence, by Gronwall’s lemma, we get that

η_x,i,u(t)−η(t )≤e^K(tk+1−tk)η_x,i,u(t_k)−η(t_k)+m_k , whereKis a Lipschitz constant for all the vector fields(Fⁱ)and

mk=2Cspλ

[tk, tk+1] \Ωi_k

.

It then follows that, for allk=0, . . . , Nandt_k≤t≤t_k+1, ηx,i,u(t)−η(t )≤

k l=0

e^K(tk+1−tl)ml≤e^KT N l=0

ml≤e^KT

|E|

i=1

λ(Ji\Ωi)≤e^KTε.

This, with Lemma3.2, shows thatS^xis included in the support of the law of(X_t)_t≥0and concludes the proof.

In the course of the proof, one has obtained the following result which is stated separately since it will be useful in the sequel.

Lemma 3.5. Ifη:R₊→R^dis such that

˙

η(t )∈

i∈E

Fⁱ η(t )

for almost allt∈R+,then,for anyε >0and anyT >0,there exists(i,u)∈Tn(for somen)such that η_x,i,u(t)−η(t )≤ε.

3.2. The accessible set

In this section we define and study theaccessible setof the processXas the set of points that can be “reached from everywhere” byXand show that long term behavior ofXis related to this set.

Definition 3.6 (Positive orbit and accessible set). For(i,u)∈Tn,letΦⁱ_ube the “composite flow”:

Φⁱ_u(x)=Φ_uⁱⁿ_n⁻¹◦ · · · ◦Φ_uⁱ⁰

1(x). (14)

Thepositiveorbit ofx is the set γ⁺(x)=

Φⁱ_u(x): (i,u)∈

n∈N^∗

Tn

.

Theaccessible setof(X_t)is the(possibly empty)compact setΓ ⊂Mdefined as

Γ =

x∈M

γ⁺(x).

Remark 3.7. Ify=Φⁱ_u(x)for some(i,u)thenyis the limit of points of the formΦ^jv(x)where(j,v)is adapted tox. It implies that

Γ =

x∈M

Φⁱ_u(x): (i,u)adapted tox

. (15)

Remark 3.8. The accessible setΓ is called the set ofD-approachable points and is denoted byLin[2].

3.2.1. Topological properties of the accessible set

The differential inclusion (13) induces a set-valued dynamical systemΨ = {Ψt}defined by Ψ_t(x)=Ψ (t, x)=

η(t ): η∈S^x enjoying the following properties

(i) Ψ₀(x)= {x},

(ii) Ψ_t+s(x)=Ψ_t(Ψ_s(x))for allt, s≥0, (iii) y∈Ψt(x)⇒x∈Ψ₋t(y).

For subsetsI⊂RandA⊂R^dwe set Ψ (I, A)=

(t,x)∈I×A

Ψ_t(x).

A setA⊂R^d is calledstronglypositively invariant underΨ ifΨ_t(A)⊂Afort≥0. It is calledinvariantif for all x∈Athere existsη∈S_R^x such thatη(R)⊂A. Givenx∈R^d, the(omega) limit setofxunderΨ is defined as

ω_Ψ(x)=

t≥0

Ψ_[t,∞[(x).

Lemma 3.9. The setω_Ψ(x)is compact connected invariant and strongly positively invariant underΨ.

Proof. It is not hard to deduce the first three properties from Lemma3.3. For the last one, letp∈ω_Ψ(x), s >0 and q∈Ψs(p). By Lemma3.5, for allε >0, there existsn∈Nand(i,u)∈Tnsuch thatΦⁱ_u(p)−q< ε. Continuity of Φ_uⁱ makes the set

Wε=

z∈M:Φⁱ_u(z)−q< ε

an open neighborhood ofp. HenceWε∩Ψ_[t,∞[(x)=∅for allt >0. This proves that the distance between the sets {q}andΨ_[t,∞[(x)is smaller thanε. Sinceεis arbitrary,qbelongs toΨ_[t,∞[(x).

Remark 3.10. For a general differential inclusion with an upper semicontinuous bounded right-hand side with com- pact convex values,the omega limit set of a point is not(in general)strongly positively invariant,see e.g. [3].

Proposition 3.11 (Properties of the accessible set). The setΓ satisfies the following:

(i) Γ =

x∈Mω_Ψ(x), (ii) Γ =ωΨ(p)for allp∈Γ,

(iii) Γ is compact,connected,invariant and strongly positively invariant underΨ, (iv) eitherΓ has empty interior or its interior is dense inΓ.

Proof. (i) Let x∈M and y∈Ψ_t(x). Thenγ⁺(y)⊂Ψ_[t,∞](x). HenceΓ ⊂Ψ_[t,∞[(x)for all x. This proves that

Γ ⊂

x∈MωΨ(x). Conversely letp∈

x∈MωΨ(x). Then, for allt >0 andx∈M,p∈Ψ_[t,∞[(x)⊂γ⁺(x)where the latter inclusion follows from Lemma3.5. This proves the converse inclusion.

(ii)By Lemma3.5,ωΨ(p)⊂Γ. The converse inequality follows from(i).

(iii)This follows from(ii)and Lemma3.9.

(iv)Suppose int(Γ )=∅. Then there exists an open setU⊂Γ and

t,iΦⁱ_t(U)is an open subset ofΓ dense inΓ. An equilibriumpfor the flowΦ¹is a point inMsuch thatF¹(p)=0. It is called anattracting equilibriumif there exists a neighborhoodUofpsuch that

tlim→∞Φ_t¹(x)−p=0

uniformly inx∈U. In this case, thebasin of attractionofpis the open set B(p)= x∈R^d: lim

t→∞Φ_t¹(x)−p=0

! .

Proposition 3.12 (Case of an attracting equilibrium). Suppose the flowΦ¹has an attracting equilibriumpwith basin of attractionB(p)that intersects all orbits:for allx∈M\B(p),γ⁺(x)∩B(p)=∅.Then

(i) Γ =γ⁺(p),

(ii) if furthermoreΓ ⊂B(p),thenΓ is contractible.In particular,it is simply connected.

Proof. The proof of(i)is left to the reader. To prove(ii), leth:[0,1] ×Γ →Γ be defined by h(t, x)=

Φ₋¹_{log(1−t )}(x) ift <1,

p ift=1.

It is easily seen thathis continuous. Hence the result.

3.2.2. The accessible set and recurrence properties

In this section, we link the accessibility (which is a deterministic notion) to some recurrence properties for the em- bedded chainZ˜ and the continuous time processZ.

Proposition 3.13 (Returns nearΓ – discrete case). Assume thatΓ =∅.Letp∈Γ andUbe a neighborhood ofp.

There existm∈Nandδ >0such that for alli, j∈Eandx∈M Px,i Z˜_m∈U× {j}

≥δ.

In particular,

Px,i ∃l∈N,Z˜_l∈U× {j}

=1.

Note that, in the previous proposition, the same discrete time mworks for all x ∈M. In the continuous time framework, one common timet does not suffice in general; however one can prove a similar statement if one allows a finite number of times:

Proposition 3.14 (Returns nearΓ – continuous case). Assume thatΓ =∅.Letp∈Γ andU be a neighborhood ofp.There existN∈N,a finite open coveringO¹, . . . ,O^NofM,N timest₁, . . . , t_N>0andδ >0such that,for all i, j∈Eandx∈O^k,

Px,i Z_tk∈U× {j}

≥δ.

SinceΓ is positively invariant under each flowΦⁱwe deduce from Proposition3.14the following result.

Corollary 3.15. AssumeΓ has nonempty interior.Then Px,i[∃t₀≥0,∀t≥t₀, Z_t∈Γ ×E] =1.

Propositions3.13and3.14are direct consequences of Lemma3.2and of the following technical result.

Lemma 3.16. AssumeΓ =∅.Letp∈Γ,Ube a neighborhood ofpandi, j∈E.There existm∈N^∗, ε, β >0,finite sequences(i¹,u¹), . . . , (i^N,u^N)∈T^ijmand an open coveringO¹, . . . ,O^NofM(i.e.M=O¹∪ · · · ∪O^N )such that for allx∈Mandτ∈R^m₊:

x∈O^k and τ−u^k≤ε ⇒ Φⁱ_τ^k(x)∈U and p x,i^k, τ

≥β.

Furthermore,m, εandβcan be chosen independent ofi, j∈E.

Proof. Fixiandj, and letV be a neighborhood ofpwith closureV⊂U. For allβ >0, define the open sets O(i,u, β)=

x∈M: Φⁱ_u(x)∈V, p(x,i,u) > β . By (15), one has

M=

n

β>0

(i,u)∈T^ijn

O(i,u, β)

. (16)

Now, since one can add “false” jumps (idoes not change anduequals 0) to(i,u)without changingΦⁱ_u(x),

∀(i,u)∈T^ijn,∀n≥n,∀β,∃β>0, i,u

∈T^ij_n such thatO(i,u, β)⊂O

i,u, β

(just addn−nfalse jumps at the beginning and letβ=β(infMQ(x, i, i))ⁿ⁻ⁿ). Therefore the union overnin (16) is increasing: by compactness, there existsmsuch that

M⊂

β>0

(i,u)∈T^ijm

O(i,u, β).

Note that by monotonicity,mcan be chosen uniformly overiandj. The union inβ increases asβ decreases, so by compactness again there existsβ₀(independent ofi,j) such that

M⊂

(i,u)∈T^ijm

O(i,u, β0).

A third invocation of compactness shows that for some finiteN,

M⊂ N k=1

Ok,

whereO^k=O(i^k,u^k, β₀)for some(i^k,u^k)inT^ijm. SinceV⊂U, the distance betweenV andU^c is positive (once more by compactness). Choosingεsmall enough therefore guarantees that forx∈O^k andv−u^k< ε,Φ^v_i(x)∈U.

This concludes the proof.

3.3. Support of invariant probabilities

The following proposition relatesΓ to the support of the invariant measures ofZ˜ orZ. We state and prove the result forZ˜ and rely on Lemma2.6forZ.

Proposition 3.17 (Accessible set and invariant measures).

(i) IfΓ =∅thenΓ ×E⊂supp(μ)for allμ∈ ˜Pinvand there existsμ∈ ˜Pinvsuch thatsupp(μ)=Γ ×E.

(ii) IfΓ has nonempty interior,thenΓ ×E=supp(μ)for allμ∈ ˜Pinv.

(iii) Suppose thatZ˜ is stable with invariant probabilityπ,thensupp(π )=Γ ×E.

Proof. (i)follows from Proposition3.13. Also, sinceΓ is strongly positively invariant, there are invariant measures supported byΓ ×E.(ii)follows from(i)and Corollary3.15. To prove(iii), let(p, i)∈supp(π ). LetU,V be open neighborhoods ofpwithU⊂Vcompact. Let 0≤f ≤1 be a continuous function onMwhich is 1 onUand 0 outside Vand letf (x, j )˜ =f (x)δ_j,i. SupposeZ₀=(x, j ). Then, with probability one,

lim inf

n→∞

1 n

1≤k≤n: Z˜k∈V× {i}

≥ lim

n→∞

1 n

n k=1

f (˜Z˜k)≥π

U× {i}

>0.

Hence(Z˜_n)visits infinitely oftenU× {i}. In particular,p∈γ⁺(x). This proves that supp(π )⊂Γ ×E. The converse

statement follows from(i).

Remark 3.18. The example given in Section5.4shows that the inclusionΓ ×E⊂supp(μ)may be strict whenΓ has empty interior.On the other hand,the condition thatΓ has nonempty interior is not sufficient to ensure uniqueness of the invariant probability since there exist smooth minimal flows that are not uniquely ergodic.An example of such a flow can be constructed on a3-manifold by taking the suspension of an analytic minimal nonuniquely ergodic diffeomorphism of the torus constructed by Furstenberg in[16] (see also[24]).As shown in[2] (see also Section4) a sufficient condition to ensure uniqueness of the invariant probability is that the vector fields verify a Hörmander bracket property at some point inΓ.