On the reflected random walk on $\mathbb{R}_+$.

HAL Id: hal-01167918

https://hal.archives-ouvertes.fr/hal-01167918v2

Preprint submitted on 12 Apr 2016

HAL is a multi-disciplinary open access archive for the deposit and dissemination of sci- entific research documents, whether they are pub- lished or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Distributed under a Creative CommonsAttribution - NonCommercial - NoDerivatives| 4.0 International License

On the reflected random walk on R_+.

Jean-Baptiste Boyer

To cite this version:

Jean-Baptiste Boyer. On the reflected random walk onR_+.. 2016. �hal-01167918v2�

JEAN-BAPTISTE BOYER

IMB, Universit´e de Bordeaux / MODAL’X, Universit´e Paris-Ouest Nanterre

Abstract. Letρbe a borelian probability measure onRhaving a moment of order 1 and a driftλ=R

Rydρ(y)<0.

Consider the random walk onR₊ starting atx∈R₊ and defined for anyn∈Nby X0 =x

Xn+1 =|Xⁿ+Yn+1| where (Yⁿ) is an iid sequence of lawρ.

We notePthe Markov operator associated to this random walk. This is the operator defined for any borelian and bounded functionf onR₊ and anyx∈R₊by

P f(x) = Z

R

f(|x+y|)dρ(y)

For a borelian bounded functionf onR₊, we call Poisson’s equation the equation f=g−P gwith unknown functiong.

In this paper, we prove that under a regularity condition on ρ, for any directly Riemann-integrable function, there is a solution to Poisson’s equation and using the renewal theorem, we prove that this solution has a limit at infinity.

Then, we use this result to prove the law of large numbers, the large deviation principle, the central limit theorem and the law of the iterated logarithm.

Contents

Introduction 2

1. Induced Markov chains 3

1.1. Definitions 3

1.2. Application to the study of invariant measures 7

2. Induction and the renewal theorem 9

3. Application to the relfected random walk on R₊ 14

References 21

E-mail address: jeaboyer@math.cnrs.fr. Date: April 12, 2016.

Key words and phrases. Markov Chains, Poisson’s equation, Gordin’s method, renewal theorem, random walk on the half line.

1

Introduction

Letρbe a borelian probability measure onRhaving a moment of order 1 and a drift λ=R

Rydρ(y)<0.

Consider the random walk onR₊ starting at x∈R₊ and defined for anyn∈Nby X₀ =x

X_n+1 =|Xn+Y_n+1| where (Yn) is an iid sequence of law ρ.

We noteP the Markov operator associated to this random walk. This is the operator defined for any borelian and bounded functionf on R₊ and any x∈R₊ by

P f(x) = Z

R

f(|x+y|)dρ(y)

The aim of this article is to study this Markov chain and to do so, we will use a standard technique (known as Gordin’s method) which consists in finding a solution g to the“Poisson equation” g−P g=f forf in a certain Banach space.

Using results of Glynn and Meyn, we will prove that under a regularity assumption on the measureρ, there always is a solution to this equation iff is borelian and bounded but this solution will not be bounded in general and this prevents us from studying the large deviation principle and complicates the study of the central limit theorem and of the law of the iterated logarithm.

However, we will see in section1that the solution to Poisson’s equation satisfies some equations (see proposition1.3) and using a “stopped” renewal theorem that we will state in section 2(see corollary 2.6) we will prove the following

Corollary (3.7). Let ρ be an absolutely continuous probability measure on R having a moment of order 1, a negative drift λ = R

Rydρ(y) < 0 and such that ρ(R^∗₊) > 0. Let (Xn) be the reflected random walk on R₊ defined by ρ.

Letνbe the unique stationary probability measure onR₊(it exists according to[Leg89]

or [PW06]).

Then, for any directly Riemann-integrable function f on R₊ such that R

fdν = 0, there is a bounded and a.e. continuous function g on R₊ such that

f =g−P g and lim

x→+∞g(x) = 0 This will allow us to prove the following

Proposition (3.8and3.11). Letρ be an absolutely continuous probability measure onR having a moment of order1, a negative driftλ=R

Rydρ(y)<0and such thatρ(R^∗₊)>0.

Let (Xn) be the reflected random walk on R₊ defined byρ.

For any directly Riemann-integrable functionf on R₊ such that R

fdν= 0, there are constants C₁, C₂ ∈R^∗₊ such that for any ε∈]0,1], anyx∈R₊ and any n∈N^∗,

P_x (

1 n

n−1

X

k=0

f(Xk)

>ε )!

6C₁e^−C2ε2n

In particular, for any x∈R₊, 1 n

n−1

X

k=0

f(X_k)−→0P_x-a.e. and in L¹(P_x)

Moreover, is ρ as a moment of order 2 +ε for some ε ∈ R^∗₊, then, for any directly Riemann-integrable function f on R₊ with R

fdν= 0 and any x∈R₊,

√1 n

n−1

X

k=0

f(X_k)−→ N^L 0, σ²(f) Where we noted N(0,0) the Dirac mass at 0 and

σ²(f) = Z

R₊

g²−(P g)²dν

with g the bounded function given by corollary 3.7and such that f =g−P g.

Moreover, if σ²(f)6= 0, then lim sup

Pn−1 k=0f(X_k)

p2nσ²(f) ln ln(n) = 1a.e. and lim inf

Pn−1 k=0f(X_k)

p2nσ²(f) ln ln(n) =−1a.e.

1. Induced Markov chains

The aim of this section is to study the process of induction of Markov chains by stopping times and to link the induced chain to the original one. We study it in a general case as it doesn’t use any particular property of the reflected random walk.

1.1. Definitions. Let (X_n) be a Markov chain on a standard Borel spaceX. We define a Markov operator onX setting, for a borelian functionf and x∈X,

P f(x) =E[f(X₁)|X₀=x]

Given a stopping timeτ, we can study the Markov chain (Xτⁿ)_n∈^Nwhereτⁿis defined by

τ⁰((Xn)) = 0

τ^k+1((Xn)) =τ^k((Xn)) +τ(θ^τk((Xn))(Xn)) whereθ stands for the shift on X^N.

We noteQthe sub-Markov operator associated to (X_τⁿ), that is, for a borelian func- tion g onX and x∈X,

Q(g)(x) = Z

{τ <+∞}

g(Xτ)dP_x((Xn)) If, for any x∈X,P_x({τ <+∞}) = 1, then Qis a Markov operator.

Finally, we define two other operator on Xsetting, for a borelian non negative func- tionf andx∈X,

Sf(x) = Z

{τ=1}

f(X₁)dP_x((X_n)) (1.1)

Rf(x) = Z

{τ <+∞}

f(X₀) +· · ·+f(X_τ−1)dP_x((Xn)) (1.2)

Definition 1.1 (θ−compatible stopping times).

We say that a stopping time τ is θ−compatible if for all x ∈ X, P_x({τ = 0}) = 0 and forP_x−a.e. (Xn)∈X^N,τ((Xn))>2 implies thatτ(θ(Xn)) =τ((Xn))−1.

Example 1.2. Let Y be a borelian subset of Xand τ_Y the time of first return inY:

τ_Y((Xn)) = inf{n∈N^∗; Xn∈Y} Then,τ_Y isθ−compatible.

Moreover,τ_Yⁿ as we defined it coresponds to the time ofn-th return toY.

Forx ∈X, we setu(x) =E_xτ_Y and we call Y strongly Harris-recurrent if u is finite onX. This imply in particular that for anyx inX,τ_Y isP_x−a.e. finite.

Indeed for any borelian non negative function f and anyx∈X, we have that

Qf(x) = Z

{τ <+∞}

f(Xτ)dP_x=

+∞

X

n=1

E_xf(Xn)1_{τ=n}

=

+∞

X

n=1

E_xf(Xn)1_Y^c(X₁). . .1_Y^c(X_n−1)1_Y(Xn)

=

+∞

X

n=1

(P1_Y^c)ⁿ⁻¹P(f1_Y) =

+∞

X

n=0

(P1_Y^c)ⁿP(f1_Y)

Rf(x) = Z

{τ <+∞}

f(X₀) +· · ·+f(X_τ−1)dP_x=

+∞

X

n=0

E_xf(Xn)1_{τ>n+1}

=f(x) +

+∞

X

n=1

E_xf(Xn)1_Y^c(X₁). . .1_Y^c(Xn)

= f(x) +

+∞

X

n=1

(P1_Y^c)ⁿ(f)(x)

!

=

+∞

X

n=0

(P1_Y^c)ⁿ(f)(x)

Sf(x) = Z

{τ=1}

f(X₁)dP_x = Z

f(X₁)1_Y(X₁)dP_x =P(f1_Y)

Thus, we have that (R+Q)f = (I_d+RP)f, RSf =Qf, (P −S)Qf = P(1_Y^cQf) = Qf−Sf and (P−S)Rf =P(1_Y^cRf) =Rf−f.

Note that P, Q, R, S, P −S, Q −S and R −I_d are positive operators and so the computations we made make sense for any non negative borelian functionf.

Next lemma generalizes those relations for any θ−compatible stopping time.

Proposition 1.3. Let τ be a θ−compatible stopping time such that for anyx∈X, τ is P_x−a.e. finite.

For any non negative borelian function f on X, we have : (R+Q)f = (Id+RP)f (Id+P R)f = (Id+S)Rf (Id+S)Qf = (S+P Q)f

RSf =Qf

Proof. Letf be a borelian non negative function on Xand x∈X.

Using the Markov property and τ being aθ−compatible stopping time, we have that for any n∈N^∗,

E_xf(Xn)1_{τ>n} =E_xP f(X_n−1)1_{τ>n}

And so,

(R+Q)f(x) =E_xf(X₀) +· · ·+f(Xτ)dP_x=E_x

+∞

X

n=0

f(Xn)1_{τ>n}

=f(x) +

+∞

X

n=1

E_xP f(X_n−1)1_{τ>n}=f(x) +RP f(x) Moreover, as τ is θ−compatible,

Z

{τ>2}

Rf(X₁)dP_x((X_n)) = Z

{τ>2}

f(X₁) +· · ·+f(X_τ−1)dP_x((X_n)) Thus,

f(x)+P Rf(x) =f(x) + Z

{τ=1}

Rf(X₁)dP_x((Xn)) + Z

{τ>2}

Rf(X₁)dP_x((Xn))

=f(x) +SRf(x) + Z

{τ>2}

f(X1) +· · ·+f(Xτ−1)dP_x((Xn))

=SRf(x) + Z

f(X₀) +· · ·+f(X_τ−1)dP_x((Xn)) =SRf(x) +Rf(x) Then, by definition of S,R

{τ=1}f(X₁)dP_x((Xn)) =R

{τ=1}f(Xτ)dP_x((Xn)), so, Sf(x) +P Qf(x) =

Z

1_{τ=1}(f(X₁) +Qf(X₁)) +1_{τ>2}f(X_τ)dP_x((X_n))

=SQf(x) +Qf(x) Finally, for any n∈N^∗,

E_xSf(Xn−1)1_{τ>n}= Z

{τ=n+1}

f(Xn+1)

therefore, RSf(x) =E_x

+∞

X

n=1

Sf(X_n−1)1_{τ>n}=

+∞

X

n=1

E_xSf(X_n−1)1_{τ>n}

=

∞

X

n=1

E_xf(X_n+1)1_{τ=n+1}=Qf(x)

And this finishes the proof of this proposition.

Lemma 1.4. Let (X_n) be a Markov chain on a standard borelian space X.

Letν be a finite P−invariant measure on Xandτ a θ−compatible stopping time such that forν-a.e x∈X,lim_n→+∞P_x(τ >n) = 0.

Then, for any non negative borelian functionf on X, we have Z

X

fdν= Z

X

SRfdν

Proof. According to proposition 1.3,f +P Rf =Rf+SRf. So, ifRf ∈L¹(X, ν), as ν isP−invariant, we get the lemma.

Iff 6∈L¹(X, ν), we will get the lemma by approximation.

First, we assume thatf is bounded. In general,Rf 6∈L¹(X, ν) so, we approximate it with a sequence of integrable functions.

More precisely, for n∈N^∗, we note Rn the operator defined like R but associated to the stopping time min(n, τ) (which is not θ−compatible).

That is to say, for a borelian non negative functionf and any x∈X,

Rnf(x) =E_x

min(τ,n)−1

X

k=0

f(Xk)

As{min(τ, n) = 1}={τ = 1} forn>2, the operator S associated to min(τ, n) does not depend onn forn>2.

As min(τ, n) is notθ−compatible, we can’t use proposition1.3, but we have forn>2, that

P R_nf(x) =E_x

min(τ◦θ,n)−1

X

k=0

f(X_k+1)

=SRnf(x) + Z

{τ>2}

min(τ−1,n)−1

X

k=0

f(X_k+1)dP_x

=SR_nf(x) + Z

{τ>2}

min(τ,n+1)−1

X

k=1

f(Xk)dP_x=SR_nf+R_n+1f−f

And, as f is bounded, for any x ∈ X, |R_nf(x)|6 nkfk∞ and so R_nf is integrable sinceµ is a finite measure and,

Z

SR_nf −fdν= Z

P R_nf −R_n+1fdν = Z

R_nf−R_n+1fdν

= Z

f(X_n)1_{τ>n}dP_x((X_n))dν(x)

= Z

P f(X_n−1)1_{τ>n}dP_x((Xn))dν(x) So,

Z

SR_nf−fdν

6kfk∞

Z

X

P_x({τ >n})dν(x)−→0 (by monotone convergence) and using the monotone convergence theorem, we get the expected result for borelian bounded functions.

Iff is not bounded and non-negative, we take an increasing sequence (fn) of bounded positive functions which converges to f and we get the expected result by monotone

convergence.

Example 1.5. If τ is the return time to some strongly Harris-recurrent set Y, then Sf(x) =P(f1_Y)(x). Moreover for everyP−invariant measureνand everyf ∈L¹(X, ν), such thatRf is ν−a.e. finite,R

XSRfdν=R

YRfdν.

In particular, with f = 1, we have that, R

YEτdν =ν(X). This is Kac’s lemma for dynamical systems.

1.2. Application to the study of invariant measures. In this subsection, X is a complete separable metric space endowed with it’s Borel tribe and “measure” stands for

“borelian measure”. We assume that there exist (at least) a P−invariant probability measure on X.

We also fix a θ−compatible stopping timeτ such that for anyx inX,E_xτ is finite.

Lemma 1.6. Let µ be a finite non-zero P−invariant borelian measure on X. Then, S^∗µ is a finite non-zero Q−invariant measure onX.

Moreover, R^∗S^∗µ=µ and S^∗µ is absolutely continuous with respect to µ.

Proof. First, for all non negativef ∈ B(X) and allx∈X,Sf(x)6P f(x).

So, R

Sfdµ6 R

P fdµ =R

fdµ since µ is P−invariant and f is bounded. And this proves that S^∗µ is absolutely continuous with respect to µ. So, as Fubuni’s theorem proves that it is σ−additive,S^∗µis a finite measure on X.

Moreover, we saw in lemma 1.4 that for all non negative borelian function f on X, R SRfdµ=R

fdµand this proves thatR^∗S^∗µ=µ.

Then, we need to prove thatS^∗µ(X)>0. But, for all x∈X, P^kS(1)(x) =E_xS1(Xk)>P_x({τ =k+ 1}) So, Pn−1

k=0P^kS(1)>P_x({τ 6n+ 1}). And, as µ is P−invariant, taking the integral on both sides, we get that,

nS^∗µ(X)>

Z

x∈X

P_x({τ 6n})dµ(x)

Finally, we use that forµ-a.e. x∈X, lim_nP_x(τ 6n) = 1 and the dominated convergence theorem, tells us that 0< µ(X)6limnS^∗µ(X), so S^∗µ(X)>0.

Lemma 1.7. Let ν be a non-zero Q−invariant borelian measure on X. Then, R^∗ν is a non zero P−invariant measure on X.

Moreover, S^∗R^∗ν =ν and ν is absolutely continuous with respect to R^∗ν.

Finally, if QR(1) is bounded on X, then R^∗ν is a finite measure if and only if ν is.

Remark 1.8. The technical assumption QR1 bounded onX is reasonable.

More specifically, using the same notations as in remark 1.2, we call Y linearily recurrent if sup_y∈YE_yτ_Y is finite.

In this case, R1(x) = E_xτ_Y and QR1(x) =E_xR1(Xτ_Y) 6sup_y∈YE_yτ_Y since for any x∈X,P_x(Xτ_Y ∈Y) = 1 be definition of τ_Y.

Proof. To prove that R^∗µis a measure, one just have to prove that it is σ−additive.

Let (An) be a sequence of pairwise disjoint borelian subsets ofX andn∈N. As R is a linear operator, we have thatR

R(1_∪ⁿ_k=0Ak)dν =Pn k=0

R R1Akdν, thus,R^∗ν is finitely additive. But, according to the monotone convergence theorem, the left side of this equation converges toR

R(1_∪An)dν and this finishes the proof that R^∗ν isσ−additive.

Moreover, for all non negativef ∈ B(X),f 6Rf, soν(f)6ν(Rf) andνis absolutely continuous with respect toR^∗ν and R^∗ν(X)>0.

Then, proposition 1.3 shows that for any positive borelian function f, Rf +Qf = f+RP f. Applying this tof =1_Afor some borelian setA, and taking the integral over ν, we get thatR

R1A+Q1Adν =R

1A+RP1Adν. But,ν is Q−invariant so ifν(A) is finite, we get thatR

R1_Adν =R

RP1Adν. Ifν(A) is infinite, the result still holds since in this case, R

R1_Adν = ν(A) =Q^∗ν(A) = R

RP1_Adν = +∞. Thus, for any borelian setA,R^∗ν(A) =P^∗R^∗ν(A) that is to say, R^∗ν isP−invariant.

AsRS=Q andν isQ−invariant, we directly have thatS^∗R^∗ν =ν.

For the last point, assume thatQR(1) is a bounded function onX.

IfR^∗ν is finite, then so is ν sinceν(X)6R^∗ν(X).

Assume that ν is finite. Then according to Chacon-Ornstein’s ergodic theorem (see chapter 3 theorem 3.4 in [Kre85]), there exist a Q−invariant non negative borelian functiong^∗ such thatR

g^∗dν =R

R1dν and for ν−almost every x∈X, 1

n

n−1

X

k=0

Q^kR1(x)−→g^∗(x)

And, sinceQR is bounded onXandR1(x) =E_xτ is finite, we get thatg^∗(x)6kQRk∞

for ν−a.e x ∈ X. So, g ∈ L^∞(X, ν) ⊂ L¹(X, ν) since ν(X) < +∞ and R

R1dν 6

kQRk∞ν(X)<+∞.

We saw in the previous lemmas that R ans S act on invariant measure. As they are linear operators and the set of invariant measures is convex, next proposition shows that they also preserve the ergodic measures (in some sense since they do not preserve probability measures).

Corollary 1.9. Let (X_n) be a Markov chain on a complete separable metric space X and τ a θ−compatible stopping time such that for any x ∈X, E_xτ is finite. Define P, Q, R and S as previously and assume that QR1 is bounded on X.

Then,S^∗ andR^∗ are reciproqual linear bijections between theP−invariant finite mea- sures and the Q−invariant ones which preserve ergodicity.

Proof. We already saw in lemma1.6 and 1.7 that S^∗ (resp R^∗) maps the P−invariant (resp. Q−invariant) finite non zero measures onto theQ−invariant (resp. P−invariant) ones and that they are reciproqual to each-other.

Thus, it remains to prove that the image by S^∗ or R^∗ of an ergodic measure still is ergodic. To do so, we use the linearity of S^∗ and R^∗ and that ergodic probability measures are extreme points of the set of invariant probability measures for a Markov chain in a complete separable metric space.

Letµbe aP−ergodic finite non zero measure. We assume without any loss of gener- ality that µis a probability measure. We saw in lemma 1.6 that S^∗µis a Q−invariant non zero finite measure.

Assume that S^∗µ = S^∗µ(X)(tν₁ + (1−t)ν₂) where ν₁ and ν₂ are two Q−invariant probability measures and t∈[0,1].

Then, we get that µ=R^∗S^∗µ=S^∗µ(X)(tR^∗ν₁+ (1−t)R^∗ν₂). But µ is ergodic, so

1

R^∗ν1(X)R^∗ν₁ = _R∗ν¹

2(X)R^∗ν₂. And applyting S^∗ again, we obtain that ν₁ = ν₂, hence, S^∗µ isQ−ergodic.

The same proof holds to show that ifν isQ−ergodic, then R^∗ν isP−ergodic.

2. Induction and the renewal theorem

In this section, we use the renewal theorem on Rto prove a “stopped renewal theorem” in corollary2.6.

Letρbe a borelian probability measure onRand define a random walk onRstarting at x∈R by

(2.1)

X₀ = x X_n+1 = Xn+Y_n+1 where (Yn) has lawρ^⊗N.

We assume thatρ has a moment of order 1 and a negative driftλ=R

Rydρ(y)<0.

In particular, for ρ^⊗N−a.e (Yn)∈R^N, Pn

k=0Y_k converges to−∞.

We noteP the Markov operator associated to ρ. This is the operator defined for any bounded borelian functionf on R and anyx∈Rby

P f(x) = Z

R

f(x+y)dρ(y) We note τ the time of first return to ]− ∞,0] :

τ((X_n)) = inf{n∈N^∗, X_n∈]− ∞,0]}

This is aϑ−compatible stopping time and our assumption onρimplies (see P1 in section 18 of [Spi64]) that for any x∈R,

E_xτ <+∞

In this section, we are interested in the operatorRdefined as in section1 for any non negative borelian function f on Rand any x∈Rby

Rf(x) :=E_x

τ−1

X

k=0

f(Xk)

The study of the operator is very close from renewal theory : indeed, if ρ(R₊) = 0 and f is null onR^∗₋, then for anyx∈R,

Rf(x) =

+∞

X

n=0

Pⁿf(x)

Therefore, we make the following definition that is usual in renewal theory :

Definition 2.1. Letf be a borelian function onR. We say thatf is directly Riemann- integrable if

h→0lim⁺hX

n∈Z

x∈[nh,(n+1)h]inf |f(x)|= lim

h→0⁺hX

n∈Z

sup

x∈[nh,(n+1)h]|f(x)|<+∞ In the sequel, we will use the following characterisation

Lemma 2.2 (Lebesgue’s criterion for Riemann-integrability). Let f be a bounded func- tion on R.

Then, f is directly Riemann integrable if and only if it is a.e. continuous and for someh ∈R^∗₊,

X

n∈Z

sup

x∈[nh,(n+1)h]|f(x)|<+∞

In the next three lemmas, we are going to prove that, notingR, Sthe operators defined as in section1and associated toτ, then for any directly Riemann-integrable functionf, SRf is also directly Riemann-integrable.

Lemma 2.3. Let ρ be a borelian probability measure on R having a moment of order 1 and a negative drift λ=R

Rydρ(y)<0.

Note τ the time of first return to]− ∞,0]and R the associated operator defined as in section1.

Then, for any directly Riemann-integrable functionf onR, the functionRfis bounded on R.

Proof. To prove this proposition, we are going to use the classical renewal theorem.

Indeed, for anyx∈Rwe have that

|Rf(x)|6R|f|(x) =E_x

τ−1

X

k=0

|f(Xk)|6E_x

+∞

X

n=0

|f(Xn)|=

+∞

X

n=0

Pⁿ|f|(x)

But, if the measure is non-lattice¹, according to the renewal theorem (see [Fel71]), we have that

x→−∞lim

+∞

X

n=0

Pⁿ|f|(x) = 0 and lim

x→+∞

+∞

X

n=0

Pⁿ|f|(x) = −1 λ

Z

R|f|dν

1For anyα∈R,ρ(αZ)<1