Maximum likelihood estimator consistency for recurrent random walk in a parametric random environment with finite support

HAL Id: hal-00976413

https://hal.archives-ouvertes.fr/hal-00976413

Preprint submitted on 9 Apr 2014

HAL is a multi-disciplinary open access archive for the deposit and dissemination of sci- entific research documents, whether they are pub- lished or not. The documents may come from teaching and research institutions in France or

L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires

Maximum likelihood estimator consistency for recurrent random walk in a parametric random environment with

finite support

Francis Comets, Mikael Falconnet, Oleg Loukianov, Dasha Loukianova

To cite this version:

Francis Comets, Mikael Falconnet, Oleg Loukianov, Dasha Loukianova. Maximum likelihood estima- tor consistency for recurrent random walk in a parametric random environment with finite support.

2014. �hal-00976413�

Maximum likelihood estimator consistency for recurrent random walk in a parametric random

environment with finite support

F. COMETS^∗ M. FALCONNET^† O. LOUKIANOV^†

D. LOUKIANOVA^†

April 9, 2014

Abstract

We consider a one-dimensional recurrent random walk in random envi- ronment (RWRE) when the environment is i.i.d. with a parametric, finitely supported distribution. Based on a single observation of the path, we pro- vide a maximum likelihood estimation procedure of the parameters of the environment. Unlike most of the classical maximum likelihood approach, the limit of the criterion function is in general a nondegenerate random variable and convergence does not hold in probability. Not only the lead- ing term but also the second order asymptotics is needed to fully iden- tify the unknown parameter. We present different frameworks to illustrate these facts. We also explore the numerical performance of our estimation procedure.

Key words: Recurrent regime, maximum likelihood estimation, random walk in random environment.MSC 2010: Primary 62M05, 62F12; secondary 62F12.

1 Introduction

Since the pioneer works of Chernov (1967) and Temkin (1972), random walks in random environments (RWRE) have attracted many probabilists and physicists, and the related literature in these fields has become richer and source of fine probabilistic results that the reader may find in surveys including Hughes (1996)

∗Laboratoire Probabilités et Modélisation Aléatoire, Université Paris Diderot, UMR CNRS 7599, 75205 Paris cedex 13, France. Email: comets@math.univ-paris-diderot.fr; ^† Laboratoire de Mathématiques et Modélisation d’Évry, Université d’Évry Val d’Essonne, UMR CNRS 8071, USC INRA, 23 Boulevard de France 91037 Evry cedex, France. E-mail:

mikael.falconnet@genopole.cnrs.fr, dasha.loukianova@univ-evry.fr, oleg.loukianov@u-pec.fr

and Zeitouni (2004). The literature dealing with the statistical analysis of RWRE is far from being as rich and we aim at making a fundamental contribution to the inference of parameters of the environment distribution for a one-dimensional nearest neighbour path.

Letω=(ωx)x∈Zbe an independent and identically distributed (i.i.d.) collection of (0,1)-valued random variables with a parametric distributionη_θof the form

ηθ= Xd i=1

piδa_i, (1)

with d an integer, p=(p_i)_1≤i≤d a probability vector anda=(a_i)_1≤i≤d the or- dered support. We further assume thatd≥2 is known, and the unknown pa- rameter isθ=(a,p).

Denote byP^θ=η^⊗_θ^Z the law on (0,1)^Z of the environmentωand byE^θ the ex- pectation under this law. The processωrepresents a random environment in which the random walk evolves. For fixed environmentω, letX=(Xt)t∈Z+be the Markov chain onZ+starting atX₀=0 and with transition probabilitiesP_ω(X_t+1= 1|X_t=0)=1, and forx>0

Pω(X_t+1=y|Xt=x)=





ωx ify=x+1, 1−ω_x ify=x−1,

0 otherwise.

For simplicity, we stick to the RWRE on the positive integers reflected at 0, but our results apply for the RWRE on the integer axis as well. The symbolPω de- notes the measure on the path space ofXgivenω, usually calledquenchedlaw.

The (unconditional) law ofX is given by P^θ(·)=

Z

Pω(·)dP^θ(ω),

this is the so-calledannealedlaw. We writeEω and E^θ for the corresponding quenched and annealed expectations, respectively.

Random environment is a classical paradigm for inhomogeneous media which possess some regularity at large scale. Introduced by Chernov (1967) as a model for DNA replication, RWRE was recently used by Baldazzi et al. (2006, 2007) and Andreoletti and Diel (2012) to analyse experiments on DNA unzipping, pointing the need of sound statistical procedures for these models. In (1) we restrict the model to environments with finite support, a framework which already covers many interesting applications and also reveals the main features of the estima- tion problem. Considering a general setup would increase the technical com- plexity without any further appeal.

The behaviour of the processXis related to the ratio sequence ρx=1−ωx

ω_x , x∈Z⁺, (2)

and we refer to Solomon (1975) for the classification ofX between transient or recurrent cases according to whetherE^θ(logρ0) is different or not from zero. The transient case may be further split into two sub-cases, calledballisticandsub- ballisticthat correspond to a linear and a sub-linear displacement for the walk, respectively.

Comets et al. (2014) provided a maximum likelihood estimator (MLE) of the pa- rameterθin the transientballisticcase. The estimator maximizes the annealed log-likelihood function, it only depends on the sequence of the number of left steps performed by the random walk. In theballistictransient case this nor- malized criterion function converges in probability to afinitedeterministic limit function, which identifies the true value of the parameter. Comets et al. (2014) establishes the consistency of MLE while asymptotic normality together with asymptotic efficiency (namely, that it asymptotically achieves the Cramér-Rao bound) is investigated in Falconnet et al. (2013). Falconnet et al. proved that a slight modification of the above criterion function also provides a well-designed limiting function in thesub-ballisticcase. In all these works, the results rely on the branching structure of the sequence of the number of left steps performed by the walk, which was originally observed by Kesten et al. (1975).

The salient probabilistic feature of a recurrent RWRE is the strong localization revealed by Sina˘ı (1982). Inhomogeneities in the medium create deep traps the walker falls into. Typically, on a time interval [0,n], the walker sneaks most of the time around the very bottom of the main “valley” which is at random distance of order log²n. Visiting repeatedly the medium at the bottom of the main "valley", it collects precise information there. Unfortunately the medium at the bottom is not typicalfrom the unknown distributionη_θ, but on the contrary, it is strongly biased by the implicit information that there is a deep trap right there. However, the localization mechanism can be completely analysed using the analogy be- tween nearest neighbour walks and electrical networks, as explained in the book of Doyle and Snell (1984). From Gantert et al. (2010), we know that the empirical distributions of the RWRE converge weakly to a certain limit law which describes the stationary distribution of a random walk in aninfinitevalley whose construc- tion goes back to Golosov (1984). This allows to unbias our observation of the medium, and to prove consistency of MLE. We recall at this point the moment estimator introduced in Adelman and Enriquez (2004), which relies on the ob- servation that the step when leaving a sitexfor the first time yields an unbiased estimator of the environment.

In the course of the proof, expanding the log-likelihood for a large observation timen, we prove convergence of the first order term at scalenand of the second order term at scale log²n. Though the first order term is random in general, it allows to identify the supportaof the environment, while the second one with a correct estimate ofa, allows to identify the probability vectorp. This discrepancy reflects that the amount of information gathered on the unknown parametera

is proportional to the duration of the observation, whereas the one forpis only proportional to the number of distinct visited sites. As emphasized above, the expansion of the likelihood, Lemma 2.2 below, is the key. Therefore it is natural to introduce a maximum pseudo-likelihood estimator (MPLE), defined by the above first two terms of the expansion, as a intermediate step to study MLE.

The rest of the article is organized as follows. In Section 2, we present the con- struction of our M-estimators (i.e.an estimator maximising some criterion func- tion), state the assumptions required on the model as well as our consistency results. Sections 3 and Section 4 are respectively devoted to the proofs of the consistency result for the estimators ofa, andprespectively. Section 5 presents some examples of environment distributions for which we provide an explicit expression of the limit of the criterion function, and check whether it is a non- degenerate or a constant random variable. Finally, numerical experiments are presented in Section 6, focusing on the three examples that were developed in Section 5.

2 Statistical problem, M-estimators and results

We address the following statistical problem: we assume that the processX is generated under the true parameter valueθ^⋆=(a^⋆,p^⋆), an interior point of the parameter spaceΘ, and we want to estimate the unknownθ^⋆from a single ob- servation (Xt)₀≤t≤n of the RWRE path with time lengthn. Letd≥2 an integer.

We always assume thatΘ⊂(0,1)^2dis compact and satisfies Assumptions I and II below, which ensure that the environment is recurrent and hasdatoms.

Assumption I(Recurrent environment). For anyθ=(a,p)inΘ, pi>0,for any i≤d , and

Xd i=1

pilog1−ai

ai

=0. (3)

Assumption II(Identifiability). For allθinΘ,

0<a₁<a₂<...<a_d<1. (4) Note that, sinceΘis compact, there existsε₀in (0,1/2d) such that

a1≥ε0, 1−ad≥ε0, ai+1−ai≥ε0, pi≥ε0, for anyi≥1. (5) We shorten toP^⋆ andE^⋆ (resp. P^⋆ andE^⋆) the annealed probabilityP^θ⋆ and its corresponding expectationE^θ⋆(resp. the law of the environmentP^θ⋆and its corresponding expectationE^θ⋆) under parameter valueθ^⋆.

Define for allx∈Z,

ξ(n,x) :=Xⁿ

t=01^{Xt=x}, (6)

ξ⁻(n,x) :=

n−1X

t=01^{Xt=x;Xt+1=x−1}, (7) ξ⁺(n,x) :=ⁿ⁻¹X

t=01^{Xt=x;Xt+1=x+1}, (8) which are respectively the local time of the RWRE inxand the number of left steps (resp. right steps) from sitexat timen. Note that

ξ(n−1,x)=ξ⁺(n,x)+ξ⁻(n,x) and |ξ⁻(n,x+1)−ξ⁺(n,x)| ≤1.

Denote byR_nthe range of the walk, R_n= |R_n|, R_n=©

x>0 :ξ(n−1,x)≥1ª

, (9)

with|E|the cardinality of the setE. It is straightforward to compute the quenched and annealed likelihood of a nearest neighbour pathX_[0,n]of lengthnstarting from 0

Pω(X_[0,n])= Y

x∈R_n

ω^ξ_x^+(n,x)(1−ωx)^ξ−(n,x), and

P^θ(X_[0,n])= Y

x∈R_n

Z₁

0 a^ξ+(n,x)(1−a)^ξ−(n,x)dη_θ(a).

Then, the annealed log-likelihood functionθ7→ℓn(θ) is defined for allθ=(a,p)∈ Θas

ℓ_n(θ)=logP^θ(X_[0,n])= X

x∈Rn

log£X^d

i=1

a^ξ_i^+(n,x)(1−ai)^ξ−(n,x)pi

¤. (10)

Definition 2.1. A Maximum Likelihood Estimator (MLE)θbnofθ^⋆is defined as a measurable choice

θb_n∈Argmax

θ∈Θ

ℓ_n(θ). (11)

We denotebanandbpnthe first and second projection ofθbn.

Due to an analysis of the log-likelihood function provided by Lemma 2.2 be- low, we are lead to Definition 2.3 of a Maximum Pseudo-Likelihood Estimator (MPLE). To do so, some additional notations are required. Let

ν(n,x)=ξ(n−1,x)

n , ν⁺(n,x)=ξ⁺(n,x)

n , and ν⁻(n,x)=ξ⁻(n,x) n .

LetΘ_a=©

a:∃p, (a,p)∈Θª

be the first projection of the parameter spaceΘ. In- troduce forainΘ_aandπ⁺,π⁻:Z→R⁺withP

x[π⁺(x)+π⁻(x)]=1 L(a,π⁺,π⁻)=X

x∈Z

maxi

©π⁺(x)logai+π⁻(x)log(1−ai)ª

, (12)

and

Ln(a)=L¡

a,ν⁺(n,·),ν⁻(n,·)¢

, (13)

where we setν⁺(n,x)=ν⁻(n,x)=0 for any non-positive integerx. Define the increasing sequenceβ=(βi)_0≤i≤ddepending onaas

β₀= −∞, β_i=log³ 1−ai

1−a_i+1

´.log³ai+1

a_i

´, for anyi≤d−1, and β_d= ∞.

(14) For anyxinR_n, define the random integer ˆı(a,n,x) as

ˆ

ı(a,n,x)=i, if ξ⁺(n,x)/ξ⁻(n,x)∈(β_i−1,β_i], (15) which is designed to satisfy

ˆ

ı(a,n,x)∈Argmaxn

ξ⁺(n,x)logai+ξ⁻(n,x)log(1−ai) :i≤do

, (16) whenxis inR_n. Finally, introduceK_n(θ) as

Kn(θ)= 1 R_n

X

x∈R_n

logpˆı(a,n,x)= Xd i=1

Rn(a,i)

R_n logpi, (17) whereRn(a,i) is the random integer defined by

R_n(a,i)= X

x∈R_n

1^{©ˆı(a,n,x)=iª= X}

x∈R_n

1 n

β_i−1<ξ⁺(n,x) ξ⁻(n,x)≤β_io

. (18) Lemma 2.2. We have

ℓ_n(θ)=n·Ln(a)+Rn·Kn(θ)+rn(θ), (19) where

rn(θ)= X

x∈R_n

log Ã

1+ X

i6=ˆı(a,n,x)

pi

p_ı(a,n,x)ˆ Ui(a,n,x)^ξ(n−1,x)

!

, (20)

with

Ui(a,n,x)=³ ai

a_ˆı(a,n,x)

´ξ⁺(n,x)/ξ(n−1,x)³ 1−ai

1−a_ˆı(a,n,x)

´ξ⁻(n,x)/ξ(n−1,x)

. (21) Furthermore, for anyθ∈Θ,

1 n

³

Rn·Kn(θ)+rn(θ)´

−−−−→

n→∞ 0, P^⋆-a.s., (22)

as well as

rn(θ) log²n−−−−→

n→∞ 0, inP^⋆-probability. (23)

Hence, the first term in the RHS of (19) is of ordern and depends only on a, whereas the second term is of order log²nand depends onaandp. The proofs of (22) and (23) are respectively provided in Sections 3.2 and 4.2. As claimed above, in view of Lemma 2.2, the following definition appears natural for an es- timator ofθ^⋆.

Definition 2.3. A Maximum Pseudo-Likelihood Estimator (MPLE)θn=(an,pn) ofθis a measurable choice of







a_n ∈ Argmax

a∈Θa

L_n(a), pn ∈ Argmax

p

Kn(an,p). (24)

In the beginning of Section 4, we prove that p_n=

µRn(an,i)

R_n :i=1,... ,d

¶

. (25)

Now, we can state our consistency results.

Theorem 2.4. Let Assumptions I and II hold. Both the ML estimatorba_nand the MPL estimatoranconverge inP^⋆-probability to the true parameter valuea^⋆. Theorem 2.5. Let Assumptions I and II hold. Both the ML estimatorbpnand the MPL estimatorpnconverges inP^⋆-probability to the true parameter valuep^⋆. We expect the speed of convergence for estimatingpis much slower than the one for estimatinga. This is supported by our simulation experiment provided in Section 6 but we leave this question for further research. Section 3 is devoted to the proof of the consistency ofan andban whereas Section 4 is devoted to the proof of the consistency ofpbnandpn.

Concluding remarks.Let us start to describe a naive estimation based on recur- rence. For allxinR_n, we can estimate the environment at this point by

ˆ

ω⁽ⁿ⁾_x = ξ⁺(n,x) ξ(n−1,x).

By recurrence, ˆω⁽ⁿ⁾_x converges toωx,P^⋆-a.s. With some extra work, it can be shown that,

1 Rn

X

x∈R_n

δ_ωˆ⁽ⁿ⁾

x −−−−→

n→∞ ηθ^⋆ P^⋆-a.s.,

whereR_nis defined by (9), and this leads to estimators of the parameters. This empirical estimator is then consistent. However, it gives equal weight to all vis- ited sitesx inR_n, without any notice of the number of visits there, which is certainly far from optimal. This is essentially how Adelman and Enriquez (2004) devise their estimators, and the simulations in Section 6 indicate they perform

poorly at some extend. On the other hand, Andreoletti (2011) proposes estima- tors based on local time of the walk, which indeed take care of this flaw, but which are adhoc in essence and difficult to use in an optimal manner. In con- trast, our estimate relies on first principles - maximum likelihood - and uses the full information gathered by the walk all through.

3 Proof of consistency for the MLE and MPLE of the or- dered support

In the present section, we first recall the weak convergence result established by Gantert et al. for the empirical distributions of the RWRE. Then, we identify the limit of our criterion as a functional ofaonly, and provide some information on it. Using its regularity properties, we can adapt the proof of consistency for M-estimators to our context.

3.1 Potential and infinite valley

The environmentωin which the random walk evolves is visualized as a potential landscapeV whereV={V(x) :x∈Z} is defined by

V(x)=





 P_x

y=1logρ_y ifx>0,

0 ifx=0,

−P₀

y=x+1logρy ifx<0,

(26)

withρ_y=(1−ω_y)/ω_y. An example of a realization ofV can be seen on Figure 1.

SettingC(x,x+1)=exp[−V(x)], for any integerx, the Markov chainXis an elec- tric network in the sense of Doyle and Snell (1984) or Levin et al. (2009), where C(x,x+1) is the conductance of the (unoriented) bond (x,x+1). In particular, the measureµdefined as

µ(x)=exp[−V(x−1)]+exp[−V(x)], x∈Z, (27) is a reversible and invariant measure for the Markov chainX.

Now, define the right bordercnof the “valley” with depth logn+(logn)^1/2as cn=min©

x≥0 :V(x)− min

0≤y≤xV(y)≥logn+(logn)^1/2ª

, (28)

and the bottombnof the “valley” as b_n=min©

x≥0 :V(x)= min

0≤y≤cn

V(y)ª .

On Figure 1, one can see a representation ofbnandcn. We are interested in the shape of the “valley” (0,bn,cn) whenntends to infinity and we recall the concept of infinite valley introduced by Golosov (1984).

V(x)

x bn

cn

logn+p logn

Figure 1: Example of potential derived from a random environment distributed as in Example I with parametera=0.3. Simulation withn=1000.

LetVe={Ve(x) : x∈Z} be a collection of random variables distributed asV con- ditioned to stay positive for any negativex, and non-negative for any non nega- tivex. For simplicity, we assume that without loss of generality,Veis also realised underP^⋆. For each realization ofVe, consider the corresponding Markov chain onZ, which is an electrical network with conductancesC(x,e x+1)=exp[−Ve(x)].

As usual, the measure µedefined asµ(x)e =Ce(x−1,x)+C(x,e x+1), x ∈Z, is a (reversible) and invariant measure for the Markov chain X. Furthermore, the measureµecan be normalized to get a reversible probability measureν, defined by

ν(x)=exp[−Ve(x−1)]+exp[−Ve(x)]

2P

z∈Zexp[−Ve(z)] . (29)

Note thatν(x) can be written as the sum ofν⁺(x) andν⁻(x), where for anyx∈Z ν⁺(x)= exp[−Ve(x)]

2P

z∈Zexp[−Ve(z)] and ν⁻(x)= exp[−Ve(x−1)]

2P

z∈Zexp[−Ve(z)]. (30) We haveν⁺(x)=ν⁻(x+1). Defineω(x)e ∈(0,1), for anyx∈Z, as

ω(x)e =ν⁺(x)

ν(x) = 1

1+exp[Ve(x)−Ve(x−1)]. (31) Remark 3.1. Noting that the possible values ofVe(x)−V˜(x−1)are those of V(x)− V(x−1)for any integer x, we deduce that underP^⋆,ω(x)e is equal to one of the coordinates ofa^⋆.

Gantert et al. (2010) showed that the empirical distribution of the RWRE, suit- ably centered atbnconverges to the stationary distribution of a random walk in

an infinite valley. More precisely, let

ν_n(x)=ν(n,x+b_n), ν⁺_n(x)=ν⁺(n,x+b_n), and ν⁻_n(x)=ν⁻(n,x+b_n).

Theorem 3.2(Gantert et al. (2010)). Let Assumptions I and II hold. The distri- butions of{ν_n(x) :x∈Z}converge weakly to the distribution ofν(as probability measures onℓ¹equipped with theℓ1-norm). As a consequence, for each strongly continuous functional f :ℓ¹→Rwhich is shift invariant, we have

f¡

{ν(n,x) :x∈Z}¢ law

−−−−→

n→∞ f¡

{ν(x) :x∈Z}¢ .

In Gantert et al. (2010) is mentioned that the result still holds with obvious ex- tensions for the non-reflected case, i.e., of a RWRE onZ. An inspection of the proof of Gantert et al. immediatly yields:

Corollary 3.3. Let Assumptions I and II hold. The distributions of {(ν⁺_n(x),ν⁻_n(x)) :x∈Z}

converge weakly to the distribution of{(ν⁺(x),ν⁻(x)) :x∈Z}. As a consequence, for each strongly continuous functional f :ℓ¹×ℓ¹→Rwhich is shift invariant, we have

f¡

{(ν⁺(n,x),ν⁻(n,x)) :x∈Z}¢ law

−−−−→

n→∞ f¡

{(ν⁺(x),ν⁻(x)) :x∈Z}¢ .

3.2 Identification and properties of the criterion limit

First, we start with

Proof of (22)in Lemma 2.2. Clearly, logε₀×max

0≤t≤nX_t≤ℓ_n(θ)− X

x∈R_n

maxi≤d

n

ξ⁺(n,x)loga_i+ξ⁻(n,x)log(1−a_i)o

≤0, (32) withε0from (5), and from (12) and (13)

X

x∈Rn

maxi≤d

nξ⁺(n,x)logai+ξ⁻(n,x)log(1−ai)o

=n·Ln(a).

SinceE^⋆(ρ₀)≥exp[E^⋆log(ρ₀)]=1, case (c’) in Theorem 2.1.9 in Zeitouni (2004) applies, andXn/n converges to 0,P^⋆-a.s. Hence, _n¹max0≤t≤nXt converges to 0 P^⋆-a.s and the claim is proved.

For anya∈Θ_a, denoteL∞(a) L∞(a)=L(a,ν⁺,ν⁻)=X

x∈Z

maxi

©ν⁺(x)logai+ν⁻(x) log(1−ai)ª

, (33)

where ν⁺ and ν⁻ are defined by (30). Anticipating Lemma 3.5 below, Corol- lary 3.3 and (22) immediatly yields

Ln(a)−−−−→^law

n→∞ L∞(a).

Therefore, we provide some useful information on the functionalL∞(·). To do so, some additional notations are required.

Definition 3.4(Boltzmann entropy function and Kullback-Leibler distance). De- fine the Boltzmann entropy function H(·)on(0,1)as

H(q)= −[qlogq+(1−q)log(1−q)]≥0, the Kullback-Leibler distance d_KL(·|·)on(0,1)×(0,1)as

dKL(q|q^′)=qlogq

q^′+(1−q)log 1−q 1−q^′ ≥0, and their multinomial extensions on probability vectorsqandq^′

H(q)= −X

i

qilogq_i≥0, dKL(q|q^′)=X

i

qilogqi

q_i^′ ≥0.

Withνandωedefined by (29) and (31), we have for allainΘ_athe identity L∞(a)= −X

x∈Z

ν(x)H£ ω(x)e ¤

−X

x∈Z

ν(x)min

i

¡d_KL£

ω(x)|ae i¤¢

. (34)

From Remark 3.1, we have min_i¡ d_KL£

ω(x)|ae ^⋆_i¤¢

=0, and using the fact thatd_KL(q|q^′)>

0 forq6=q^′, we deduce that

L∞(a)<L∞(a^⋆)= −X

x∈Z

ν(x)H£ ω(x)e ¤

, a6=a^⋆. (35)

More precisely, a useful bound is L∞(a)≤L∞(a^⋆)−min

k

¡ν©

x∈Z:ω(x)e =a_k^⋆ª¢

× Xd j=1

mini

¡d_KL£ a^⋆_j|ai¤¢

, (36) where the sum is deterministic and positive fora6=a^⋆, though its factor (minj(·)) is a.s. positive and does not depend ona.

Lemma 3.5. The function L defined by(12)is Lipschitz continuous:

|L(a,π⁺,π⁻)−L(a,µ⁺,µ⁻)| ≤ |logε0| ·¡

kπ⁺−µ⁺k₁+ kπ⁻−µ⁻k₁¢

, (37)

|L(a,π⁺,π⁻)−L(a^′,π⁺,π⁻)| ≤2ε⁻¹₀ ka−a^′k₂. (38)

Proof of Lemma 3.5. Recall thatR^d∋u7→max_iui∈Ris 1-lipschitz continuous for the normk · k∞. Moreover, the mapping

fi : (a,v⁺,v⁻)7→v⁺loga_i+v⁻log(1−ai)

is|logε₀|-lipschitz continuous inv⁺, resp. in v⁻, withε₀from (5). By compo- sition, (v⁺,v⁻)7→max_ifi(a,v⁺,v⁻) is|logε₀|-lipschitz continuous in thek · k₁ norm, and (37) follows by summing overx. By Cauchy-Schwarz,

|fi(a,v⁺,v⁻)−fi(a^′,v⁺,v⁻)| =¯¯¯ Z1

0 ∂_afi(a^′+t(a−a^′),v⁺,v⁻)·(a−a^′)dt¯¯¯

≤ ka−a^′k₂×sup

a" k∂afi(a",v⁺,v⁻)k2, where derivative can be bounded using

k∂_af_i(a,v⁺,v⁻)k₂=¡

a_i⁻²(v⁺)²+(1−a_i)⁻²(v⁻)²¢1/2

≤2ε⁻¹₀ (v⁺+v⁻).

Taking the maximum overiand summing overx, this yields (38).

3.3 Proof of Theorem 2.4

Recall thatanis defined by (24). Fix someε>0. We prove that for allε₁>0, there exists an integern1such that for alln≥n1,

P^⋆¡

an∈B(a^⋆,ε)¢

≥1−2ε₁, (39)

whereB(a^⋆,ε) is the open ball of radiusεcentered ata^⋆.

UnderP^⋆, the sequence (g[ω(x)] :e x∈Z) withg(u)=log(u⁻¹−1) is the sequence of the jumps of a random walkV conditioned to stay positive. The jump of un- conditionnedV takes the valueg(a^⋆_j) with positive probabilityp^⋆_j. Hence it is not difficult to check that for alljin {1,... ,d}, we can findP^⋆-a.s. some randomx such thatω(x)e =a^⋆_j. SinceνisP^⋆-a.s. supported by the wholeZ, we have

P^⋆¡ minj

©ν{x:ω(x)e =a^⋆_j}ª

>0¢

=1.

By continuity, for arbitraryε1>0 we can fixδ1>0 such that P^⋆

³min

j

©ν{x:ω(x)e =a^⋆_j}ª

≥δ₁i

≥1−ε₁. (40) By compactness ofK=Θ_a\B(a^⋆,ε),

κ=infnX^d

j=1

mini

©dKL£

a^⋆_j|ai¤ª

:a∈Ko

>0.

By (38),

a^′∈B¡

a,κδ1ε0/6¢

⇒ |L∞(a)−L∞(a^′)| ≤κδ1/3. (41) SinceKis totally bounded, we can select a finite covering∪^k_k⁰₌₁B(a^k,κδ₁ε₀/6) of Kwith balls of that radius. From (37) we can apply Corollary 3.3, and we have

¡L_n(a^k) : 0≤k≤k₀¢ law

−−−−→

n→∞

¡L∞(a^k) : 0≤k≤k₀¢ , where we have seta⁰=a^⋆. Also,

¡Ln(a^⋆)−Ln(a^k)¢

k≤k0

−−−−→law n→∞

¡L∞(a^⋆)−L∞(a^k)¢

k≤k0,

where the limits are simultaneously larger thanκδ₁on a set of large probability (larger than 1−ε₁) by (36) and (40). Then, we findn1such that forn≥n1,

P^⋆¡

Ln(a^⋆)−Ln(a^k)≥2κδ₁/3, 1≤k≤k0¢

≥1−2ε₁. Taking (41) into account, we obtain that, for alln≥n1,

P^⋆¡

Ln(a^⋆)−Ln(a)≥κδ1/3,a∈K¢

≥1−2ε₁.

This implies (39) for alln≥n1. Hencean→a^⋆inP^⋆-probability. Now, we turn to the convergence ofbantoa^⋆inP^⋆-probability. According to (32), we have

Ln(a)−un≤ℓn(θ)

n ≤Ln(a),

where u_n is non-negative and converges P^⋆-a.s. to 0 independently from θ.

Choosingn2such that for alln≥n2

P^⋆¡

un≥κδ1/3¢

≤ε1, achieves the proof of the convergenceban→a^⋆.

4 Proof of consistency of estimates for the probability vec- tor

First, note thatKn(θ) can be rewritten Kn(θ)= −H

³R_n(a,·) Rn

´

−dKL

³R_n(a,·) Rn

¯¯

¯p

´. (42)

Therefore,

Kn(an,p)≤ −H³Rn(an,·) R_n

´,

with equality if and only ifp=³_R

n(an,i)

Rn :i=1,... ,d´

. Hence, we can use the alter- native definition ofpngiven by (25) to prove Theorem 2.5.

4.1 Proof of Theorem 2.5: convergence of pn

For 0<δ<1 let G_n^δ=©

x≤bn : max

z∈[x,bn]V(z)−V(x)≥δlognª

, (43)

D_n^δ=©

x>b_n : max

z∈[bn,x]V(z)−V(b_n)≤(1−δ)lognª

, (44)

R^δ

n=G^δ

n∪D^δ

n. (45)

Note thatD^δ_nis an interval,D_n^δ=]b_n,c_n^δ] withc^δ_n≤cn.

Denoteβ^⋆andβthe sequences defined by (14) replacingabya^⋆andanrespec- tively. Noting that for anyiin {1,... ,d−1},d_KL(a_i^⋆|a^⋆_i+1)>0, we can chooseε^′>0 small enough such that for anyi∈{1,... ,d}

β^⋆_i−1≤ a^⋆_i

1−a^⋆_i −2ε^′< a^⋆_i

1−a_i^⋆+2ε^′≤β^⋆_i. (46) Letε>0. Define the eventsA^δ_n(ε^′),Bn(ε^′) andC^δ_n(ε,i) by

A^δ_n(ε^′)=

½

∀x∈R^δ

n :¯¯¯ξ+(n,x) ξ⁻(n,x) − 1

ρx

¯¯

¯≤ε^′

¾

, (47)

Bn(ε^′)=n

∀i∈{1,... ,d} :|βi−β^⋆_i| ≤ε^′o

, (48)

C^δ_n(ε,i)=





¯¯

¯ 1

|R_n∩R^δ_n| X

x∈R_n∩R^δ

n

1^{ωx=a^⋆_i}−p^⋆_i¯¯¯>ε 4



, (49) Denote^∁A^δ_n(ε^′) and^∁Bn(ε^′) the respective complementary events ofA^δ_n(ε^′) and Bn(ε^′), and define the quantitiesφ^δ_n(ε^′),ψ^δ_n(ε) andµ^δ_n(ε) by

φ^δ_n(ε^′)=P^⋆¡∁

A^δ_n(ε^′)¢ +P^⋆¡∁

B_n(ε^′)¢ ψ^δ_n(ε)=P^⋆³|R_n\R^δ

n| R_n >ε

2

´,

µ^δ_n(ε,i)=P^⋆³

C_n^δ(ε,i)´ . Then, we have

P^⋆³

|pn(i)−p^⋆_i| >ε´

≤φ^δ_n(ε^′)+ψ^δ_n(ε)+ψ^δ_n(ε/2)+µ^δ_n(ε,i). (50) Indeed, it is clear that

P^⋆³

|pn(i)−p^⋆_i| >ε´

≤P^⋆³

|pn(i)−p_i^⋆| >ε,A^δ_n(ε^′),Bn(ε^′)´

+φ^δ_n(ε^′), (51) and writing the setR_nas the union of the two disjoint sets¡

R_n∩R^δ_n¢ and¡

R_n\R^δ_n¢ in (18) and using (25) yields

P^⋆³

|p_n(i)−p_i^⋆| >ε,A^δ_n(ε^′),B_n(ε^′)´

≤P^⋆³

A^δ_n(ε^′),B_n(ε^′),D^δ_n(ε,i)´

+ψ^δ_n(ε), (52)

with

D^δ_n(ε,i)=





¯¯

¯ 1 Rn

X

x∈R_n∩R^δ_n

1 n

β_i<ξ⁺(n,x)

ξ(n,x) ≤β_i+1o

−p_i^⋆

¯¯

¯>ε 2



. Using our choice ofε^′to satisfy (46), we have for alliin {1,... ,d}

½

βi<ξ⁺(n,x)

ξ(n,x) ≤βi+1,A^δ_n(ε^′),Bn(ε^′)

¾

=n

ωx=a^⋆_i,A^δ_n(ε^′),Bn(ε^′)o . Hence,

P^⋆³

A^δ_n(ε^′),B_n(ε^′),D^δ_n(ε,i)´

≤P^⋆³¯¯¯ 1 R_n

X

x∈R_n∩R^δ

n

1^{ωx=a^⋆_i}−p^⋆_i ¯¯¯>ε 2

´, (53)

and clearly, P^⋆³¯¯¯ 1

R_n X

x∈R_n∩R^δ

n

1^{ωx=a^⋆_i}−p^⋆_i

¯¯

¯>ε 2

´

≤ψ^δ_n(ε/2)+µ^δ_n(ε,i). (54) Combining (51), (52), (53) and (54) yields (50).

Anticipating some lemmas which are proved independently in the next section, we conclude the proof. From Lemma 4.1, we have

δ→0lim limsup

n→∞ ψ^δ_n(ε)=0.

By the law of large numbers,

n→∞lim µ^δ_n(ε)=0.

In view of (50), to conclude the proof of Theorem 2.5 it suffices to prove that, for allδ>0,φ^δ_n(ε) vanishes asn→ ∞. On the one hand,anconverges toa^⋆in P^⋆-probability, and thus

P^⋆¡^∁ Bn(ε^′)¢

−→0.

On the other hand, by Lemma 4.2, P^⋆¡∁

A^δ_n(ε^′)¢

−→0.

This concludes the proof of the convergence inP^⋆-probability ofpntop^⋆. ✷ 4.2 Proof of(23)in Lemma 2.2

For allx∈R_n, one can write logUi(a,n,x)=h ξ⁺(n,x)

ξ(n−1,x)−aˆı(a,n,x)

ihlog ai

aı(a,n,x)ˆ

−log 1−ai

1−aı(a,n,x)ˆ

i (55)

−d_KL¡

a_ı(a,n,x)ˆ |a_i¢ .

Letc0be the quantity defined as c0= inf

a∈Θ_ainf

i6=jdKL(ai|aj)>0. (56) where the strict inequality comes from (5) and the continuity ofdKL(·|·). Recall ε0in (5) andε^′in (46), and letε^′′be the quantity defined as

ε^′′=minn ε^′,c₀

4

³log1−ε₀ ε₀

´−1o

. (57)

LetV(ε^′′)⊂Θ_abe the neighborhood ofa^⋆defined as V(ε^′′)=n

a∈Θ_a: max©

ka−a^⋆k,kβ−β^⋆kª

≤ε^′′o

. (58)

Fix 0<δ<1. Letabe inV(ε^′′) and assume thatA^δ_n(ε^′′) defined by (47) occurs, then for allx∈R^δ_nand anyi6=ı(a,ˆ n,x)

logU_i(a,n,x)≤ −c₀

2. (59)

Indeed, using (58) and the fact thatA^δ_n(ε^′′) occurs withε^′′≤ε^′, we have for all x∈R^δ

n

ˆ

ı(a,n,x)=ˆı(a^⋆,n,x) and ω_x=a^⋆_ı(a,n,x)ˆ =a^⋆_ˆı(a^⋆_,n,x). (60) Then, ifi6=ı(a,n,ˆ x), using (55), (56), (60), and the fact that

¯¯

¯log ai

aˆı(a,n,x)

−log 1−ai

1−aı(a,n,x)ˆ

¯¯

¯≤2log1−ε₀ ε₀ , yield

logUi(a,n,x)≤2log1−ε0

ε₀

µ¯¯¯h¡ξ⁺(n,x) ξ⁻(n,x)

¢−h¡ 1 ρx

¢¯¯¯+ ka−a^⋆k

¶

−c0

with

h(u)= u 1+u.

Using the fact that 0≤h^′(u)≤1, for anyu≥0, and the fact thatA^δ_n(ε^′′) occurs, we have

logU_i(a,n,x)≤2log1−ε₀ ε0

¡ε^′′+ ka−a^⋆k¢

−c0.

Using our choice ofaandε^′′achieves the proof of (59). Now, from (59), we de- duce that ifA^δ_n(ε^′′) occurs,

0≤rn(θ)≤

¯¯

¯R_n\R^δ

n

¯¯

¯·log³

1+d−1 ε₀

´ + X

x∈R^δ

n

log³

1+d−1

ε₀ e^{−ξ(n−1,x)c0/2}´ . (61) Assume that the eventE_n^δdefined by

E_n^δ=n

∀x∈R^δ_n :ξ(n,x)≥n^δ/2o

, (62)

occurs. Then

0≤rn(θ)≤d−1 ε₀

³¯¯¯R_n\R^δ

n

¯¯

¯+ |R^δ

n|e^−nδ/2c0/2´

. (63)

From Lemma 4.1, 4.2 and 4.3, we conclude that (23) occurs. ✷

4.3 Convergence ofbpn

From the definitions ofθbnandanrespectively given by (11) and (24), we have ℓ_n(bθ_n)−ℓ_n(θ_n)≥0 and Ln(ban)−Ln(a_n)≤0. (64) Recalling (19), (64) implies

K_n(θb_n)−K_n(θ_n)+ 1 Rn

[r_n(bθ_n)−r_n(θ_n)]≥0. (65) From Theorem 2.4, whennis large enough, bothba_n anda_n belong toV(ε^′′) in- troduced in (58) with large probability. Assuming that the eventA^δ_n(ε^′′) defined by (47) occurs, we have

ˆı(an,n,x)=ı(ˆban,n,x)=ˆı(a^⋆,n,x), for allx∈R^δ

n, and

Kn(bθn)≤log³1−ε₀ ε₀

´|R_n\R^δ

n| R_n + 1

R_n X

x∈R_n

logpb_ˆı(a^⋆_,n,x), as well as

Kn(θn)≥log³ ε₀ 1−ε₀

´|R_n\R^δ_n| R_n + 1

R_n X

x∈R_n

log ¯p_ˆı(a^⋆_,n,x), which holds for largenwhenp_nis in [ε₀,1−ε₀], and finally

Kn(θbn)−Kn(θn)≤2log³1−ε0

ε₀

´|R_n\R^δ_n| Rn

−d_KL(pn|pbn). (66) Assuming furthermore thatE^δ_n defined by (62) occurs, then (63) occurs. All in all, combining (63), (65) and (66) yields the existence of a positive constantC, depending onε0anddonly, such that

d_KL(p_n|pb_n)≤C³|R_n\R^δ

n| Rn

+|R^δ

n| Rn

e^−nδ/2c0/2´ ,

with large probability. From the fact thatpn converges inP^⋆-probability top^⋆, the continuity ofd_KL(·|·), Lemma 4.1, 4.2 and 4.3, we conclude thatbp_nconverges

top^⋆inP^⋆-probability. ✷

4.4 Intermediate lemmas

Lemma 4.1. There exists a random variable Z(δ)≥0such that

|R_n\R^δ_n| log²n

−−−−→law

n→∞ Z(δ), with (67)

Z(δ)−−−→

δ→0 0 inP^⋆-probability. (68)