A uniform Berry-Esseen theorem on M-estimators for geometrically ergodic Markov chains

HAL Id: hal-00563638

https://hal.archives-ouvertes.fr/hal-00563638

Submitted on 7 Feb 2011

HAL is a multi-disciplinary open access archive for the deposit and dissemination of sci- entific research documents, whether they are pub- lished or not. The documents may come from teaching and research institutions in France or

L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires

A uniform Berry-Esseen theorem on M-estimators for geometrically ergodic Markov chains

Loïc Hervé, James Ledoux, Valentin Patilea

To cite this version:

Loïc Hervé, James Ledoux, Valentin Patilea. A uniform Berry-Esseen theorem on M-estimators for geometrically ergodic Markov chains. Bernoulli, Bernoulli Society for Mathematical Statistics and Probability, 2012, 18 (2), pp.703-734. �10.3150/10-BEJ347�. �hal-00563638�

A uniform Berry-Esseen theorem on M −estimators for geometrically ergodic Markov chains

Loïc HERVÉ^∗, James LEDOUX^∗, Valentin PATILEA^∗ September 7, 2010

Abstract

Let {Xn}n≥0 be a V-geometrically ergodic Markov chain. Given some real-valued functional F, define Mn(α) := n⁻¹P_n

k=1F(α, Xk−1, Xk), α ∈ A ⊂ R. Consider an M−estimatorαbn, that is as a measurable function of the observations satisfyingMn(bαn)≤ minα∈AMn(α) +cn with {cn}n≥1 some sequence of real numbers going to zero. Under some standard regularity and moment assumptions, close to those of the i.i.d. case, the estimatorαbn satisfies a Berry-Esseen theorem uniformly with respect to the underlying probability distribution of the Markov chain.

AMS subject classification : 62F12, 62M05, 60F05, 60J05 Keywords : spectral method

I Introduction

Let(E,E) be a measurable space with E a countably generated σ-field, and let{X_n}_n≥0 be a Markov chain with state space E and transition kernels {Q_θ(x,·) : x ∈ E} where θ is a parameter in some general set Θ. The initial distribution of the chain, i.e. the probability distribution of X₀, is denoted by µ and may or may not depend on θ. Although {X_n}_n≥0 does not need to be the canonical version, we use the standard notation P_θ,µ to refer to the probability distribution of {X_n}_n≥0 (and E_θ,µ for the expectation w.r.t. P_θ,µ). We consider that {X_n}_n≥0 is a V-geometrically ergodic Markov chain, where V : E→[1,+∞) is some fixed unbounded function. This class of Markov chains is large enough to cover interesting applications (see [MT93, §16.4,§16.5]).

The parameter of interest is α₀ =α₀(θ)⊂ A, where α₀(·) is a function of the parameterθ and Ais an open interval of R. To estimate α₀, let us introduce the statistic

M_n(α) := 1 n

Xn k=1

F(α, X_k−1, X_k), (1)

whereF is a real-valued measurable functional onA ×E². We define anM−estimator¹ to be a random variableαb_n depending on the observations(X₀, . . . , X_n)such that

M_n(αb_n)≤min

α∈AM_n(α) +c_n,

∗Université Europénne de Bretagne, INSA-IRMAR, UMR-CNRS 6625. Institut National des Sciences Appliquées de Rennes, 20, Avenue des Buttes de Cöesmes CS 14315, 35043 Rennes Cedex, France. Loic.Herve, James.Ledoux, Valentin.Patilea@insa-rennes.fr

1This is slightly more general than the usual definition ofM−estimators or minimum contrast estimators, wherecn= 0, see [Arc98].

where{c_n}_n≥1 is a sequence of non-negative real numbers going to zero to be specified later.

Assume that for allθ∈Θ

M_θ(α) := lim

n→∞E_θ,µ[M_n(α)]

is well defined everywhere on A and does not depend on µ. In addition, assume that there exists a unique “true” valueα₀of the parameter of interest, that isM_θ(α₀)< M_θ(α),∀α6=α₀. We want to prove the followinguniform Berry-Esseen bound forαb_n

sup

θ∈Θ

sup

u∈R

¯¯

¯¯P_θ,µ

½ √ n

τ(θ)(bα_n−α₀)≤u

¾

−Γ(u)

¯¯

¯¯=O µ 1

√n

¶

, (BE)

where Γ denotes the standard normal distribution function, and τ(θ) is some positive real number defined in Theorem 3.

To derive (BE), we use Pfanzagl’s approach [Pfa71]. Besides technical assumptions, this approach relies on several ingredients. First, we need the uniform consistency condition:

(UC) ∀d >0, sup_θ∈ΘP_θ,µ©

|bα_n−α₀| ≥dª

=O(1/√ n).

Second, consider the following two convergence properties: ifS_n(α₀) =P_n

k=1ξ(α₀, X_k−1, X_k) withξ(α₀, X_k−1, X_k) centered,

(a) the sequence ©

E_θ,µ[S_n²(α₀)]/nª

n≥1 converges to a real number σ²(θ);

(b) there exists a positive constant B(ξ) such that for any n≥1 sup

θ∈Θ

sup

u∈R

¯¯

¯¯P_θ,µ

½ S_n(α₀) σ(θ)√

n ≤u

¾

−Γ(u)

¯¯

¯¯≤ B(ξ)

√n .

The properties (a) and (b) will be required for certain ξ(α₀, x, y) defined as linear combi- nations of some functionals related to F. To obtain (a) and (b) for such ξ(α₀, x, y)’s with V-geometrically ergodic Markov chains, a natural moment (or V−domination) condition is used: there exist positive constantsC_ξ andm such that

∀(x, y)∈E²,∀α∈ A, |ξ(α, x, y)|^m ≤C_ξ(V(x) +V(y)). (2) The paper is organized as follows. In Section II, an extended version of Pfanzagl’s theorem [Pfa71, Th 1] is stated for any sequence of observations, not necessarily markovian. Sec- tion III is devoted to a Berry-Esseen bound for the additive functionalP_n

k=1ξ(α₀, X_k−1, X_k) of aV-geometrically ergodic Markov chain{X_n}_n≥0 withξ satisfying Inequality (2). In Sub- section III.2, we prove that the properties (a) and (b) are fulfilled when Inequality (2) holds with the (almost expected) orderm, namely: m >2 ⇒(a), and m >3 ⇒(b). These results follow from the weak spectral method based on the theorem of Keller and Liverani [KL99].

This approach, introduced in [HH04], is fully described in [HP10] in the Markov context (see also [GL06, Gou08] and other references given in [HP10]). It is important to notice that Pfan- zagl’s method requires the precise control of the constantB(ξ)in Property (b) as a function of the size of ξ. The present operator-type approach shows that B(ξ) depends only on the constant C_ξ in Inequality (2). Thanks to these preliminary results, in Section IV we prove our main statement, that is:

(R) under some technical assumptions and the uniform consistency condition (UC), if two functionalsF⁰ andF⁰⁰ related to F (in the basic caseF⁰ andF⁰⁰ are the first and second order derivatives of F with respect to α) satisfy Inequality (2) for some m > 3 and constantsC_F⁰, C_F⁰⁰ that do not depend onα, thenαb_n satisfies property (BE).

To the best of our knowledge, the result(R)is new. It completes the central limit theorem for {bα_n}_n≥1 proved in [DY07] when Inequality (2) holds withm= 2. The domination condition (2) required by(R)is almost optimal in the sense that we imposem >3 in place of the best possible value m= 3 obtained in the i.i.d. case. In Section V, our results are applied to the AR(1) process with ARCH (AutoRegressive Conditional Heteroscedastic) of order 1 errors.

The paper ends with a conclusion section.

Let us close the introduction with a brief review of previous related work in the litera- ture. In [Pfa71], {X_n}_n∈N is a sequence of i.i.d. random variables and Pfanzagl proved a Berry-Esseen theorem for minimum contrast estimators (which are special instances of M- estimators) associated with functionals of the formF(α, X_k). In [Pfa71], the moment condi- tions onF⁰ :=∂F/∂α,F⁰⁰:=∂²F/∂α² are the expected ones since the property (b) is fulfilled under the expected third moment condition [Fel71, Chap. XVI]. Using convexity arguments, Bentkuset al. [BBG97] proposed an alternative method for deriving Berry-Esseen bounds for M−estimators with i.i.d. data. In the Markov context, the method proposed by Pfanzagl is extended, first by Rao to cover the case of uniformly ergodic Markov chains [Rao73], second in [MR89] to the case of the linear autoregressive model. However, their assumptions to get (BE) include much stronger moment conditions involving both the functional F and the Markov chain. Here, as already mentioned, the weak spectral method of [HP10] enables us to have an (almost) optimal treatment of (a) and (b), and hence an improved Berry-Esseen result (BE).

II The Pfanzagl method revisited

We state and prove a general result that allows to derive uniform Berry-Esseen bounds for M−estimators. This result is an extended version of Theorem 1 in [Pfa71] and is applied to our Markov context in Section IV.

II.1 The result

Consider a statistical model ¡

Ω,F,{P_θ, θ ∈ Θ}¢

, where Θ denotes some parameter space, and let {X_n}_n≥0 be any sequence of observations (not necessarily Markovian). Let us denote the expectation with respect toP_θ byE_θ.

For each n, letM_n(α)be a measurable functional of the observationsX₀, . . . , X_n and the parameter of interestα∈ AwhereAis some open interval ofR. Let{c_n}_n≥1be a sequence of non-negative real numbers going to zero at some rate to be specified later. AnM−estimator is a measurable function αb_n of the observations(X₀, . . . , X_n) such that

M_n(αb_n)≤min

α∈AM_n(α) +c_n. (3)

This is the usual definition of minimum contrast estimators as soon asc_n≡0.

Assumptions. Suppose that for all n ≥ 1 and α ∈ A, there exist M_n⁰(α), M_n⁰⁰(α) some measurable functions depending on X₀, X₁, ..., X_n and on the parameter of interest such that the following properties hold true:

(A1) ∀θ ∈Θ, there exists a unique α₀ =α₀(θ) ∈ A such that M_θ⁰(α₀) = 0 where M_θ⁰(α) :=

lim_n→∞E_θ[M_n⁰(α)] (the limit is assumed to be well defined for all(θ, α)∈Θ× A);

(A2) 0<inf_θ∈Θm(θ) ≤sup_θ∈Θm(θ) <∞ where m(θ) := lim_n→∞E_θ[M_n⁰⁰(α₀)] (the limit is assumed to be well defined for all θ);

(A3) for every n≥1, there existsr_n>0 independent of θ such that r_n=o(n^−1/2) and sup

θ∈Θ

P_θ©

|M_n⁰(αb_n)| ≥r_nª

=O(n^−1/2);

(A4) forj= 1,2, there exists a functionσ_j(·)such that0<inf_θ∈Θσ_j(θ)≤sup_θ∈Θσ_j(θ)<∞ and there exists a positive constant B such that for alln≥1

sup

θ∈Θ

sup

u∈R

¯¯

¯¯P_θ

½ √ n

σ₁(θ)M_n⁰(α₀)≤u

¾

−Γ(u)

¯¯

¯¯≤ B

√n, sup

θ∈Θ

sup

u∈R

¯¯

¯¯P_θ

½ √ n σ₂(θ)

¡M_n⁰⁰(α₀)−m(θ)¢

≤u

¾

−Γ(u)

¯¯

¯¯≤ B

√n; (A4’) for n≥1, |u| ≤2√

lnn, andθ∈Θ, there is a positive number σ_n,u(θ) such that

|σ_n,u(θ)−σ₁(θ)| ≤A⁰ |u|

√n,

¯¯

¯¯P_θ

½ √ n σ_n,u(θ)

µ

M_n⁰(α₀) + u σ₁(θ)

√n m(θ)

¡M_n⁰⁰(α₀)−m(θ)¢¶

≤u

¾

−Γ(u)

¯¯

¯¯≤ B⁰

√n

with some positive constantsA⁰, B⁰ independent of n, u, θ;

(A5) for any (α, α⁰)∈ A², letR_n(α, α⁰) be defined by the equation M_n⁰(α⁰) =M_n⁰(α) + [M_n⁰⁰(α) +R_n(α, α⁰)](α⁰−α).

For each n, there existω_n≥0 and a real-valued measurable functionW_n depending on X₀, . . . , X_n, both independent ofθ, such that ω_n=o(1)and

∀(α, α⁰)∈ A², |R_n(α, α⁰)| ≤©

|α−α⁰|+ω_nª W_n, and there is a constantc_W >0 such that

sup

θ∈Θ

P_θ{c_W ≤W_n}=O(n^−1/2).

(A6) αb_n is assumed to be uniformly consistent, that is there existsγ_n=o(1)such that sup

θ∈Θ

P_θ©

|bα_n−α₀| ≥dª

≤γ_n,

where d:= inf_θ∈Θm(θ)/8c_W with c_W and m(θ) defined in (A5) and (A2) respectively.

Let us comment on these assumptions. Condition (A1) identifies the true value of the parameter. In Conditions (A1) and (A2), the expectations E_θ[M_n⁰(α)] and E_θ[M_n⁰⁰(α₀)] may depend on n, like for instance in the Markovian framework considered in the sequel when the initial distribution is not the stationary distribution. Condition (A3) ensures that the estimator (approximately) satisfies a kind of first order condition. Such a condition allows to take into account the numerical errors we are faced when computingαb_n. It may also be useful when the estimator of the parameter α₀ depends on some “nuisance” parameters (see the example in the second part of Section V). Conditions (A4) and (A4’) are the uniform Berry- Esseen bounds forM_n⁰(α₀),M_n⁰⁰(α₀) and for some of their linear combinations. The identity definingR_n(α, α⁰) in Condition (A5) is guaranteed by a Taylor expansion when the criterion M_n(α) is twice differentiable with respect toα. In this caseM_n⁰ and M_n⁰⁰ are nothing else but the first and second order derivatives ofM_nwith respect to α. The reminder R_n(α, α⁰) must satisfy a Lipschitz condition. For instance, when ω_n = 0, this holds true if α 7→ M_n(α) is three times continuously differentiable with a bounded third order derivative. Condition (A6) is a standard consistency condition (e.g. see [BBG97]). General sufficient conditions for (A6) withγ_n=O(n⁻¹) have been proposed in the case of i.i.d. observations or uniformly ergodic Markov chains (see [MP71, Lemma 4] and [Rao73, Lemma 4.1] respectively). Such general arguments can easily be adapted to the geometrically ergodic Markov chain framework. In specific examples, like the one investigated in Section V, Condition (A6) can be checked by direct arguments.

The proof of Theorem 1, which adapts the arguments of [Pfa71], is given in Subsection II.2.

Theorem 1 Under Conditions (A1-A6), there exists a positive constant C such that

∀n≥1, sup

θ∈Θ

sup

u∈R

¯¯

¯¯P_θ

½√ n

τ(θ)(αb_n−α₀)≤u

¾

−Γ(u)

¯¯

¯¯≤C µ 1

√n+√

nr_n+ω_n+γ_n

¶ (4) with τ(θ) :=σ₁(θ)/m(θ).

To obtain the classical orderO(n^−1/2)of the Berry-Esseen bound, one needs γ_n=O(n^−1/2), r_n =O(n⁻¹) and ω_n =O(n^−1/2). Note that this usually requires that the sequence{c_n}_n≥1 in (3) decreases at the rate n^−3/2. This is to be compared to the rate n⁻¹ that is usually required to obtain the asymptotic normality of M−estimators (see [Arc98]).

Remark 1 A close inspection of the proof of Theorem 1 below , shows that the constant C in inequality (4) can be tracked provided that theO(·) ando(·) rates in Assumptions (A3)-(A6) are more explicit. For the sake of brevity, we only consider the case wherec_n=r_n=ω_n= 0, α(θ) = θ and (A3) is: for any n ≥ 1, |M_n⁰(bθ_n)| = 0. The constants C in the various inequalities of Assumptions (A4)-(A6) are denoted byC₁, C₂ in (A4), C₃, C₄ in (A4’),C₅ in (A5) and we choose γ_n≤C₆n^−1/2 in (A6). Then, we can obtain from Propositions 1-2 that

∀n≥1, sup

θ∈Θ

¯¯

¯¯P_θ

½ √ n

τ(θ)(bα_n−α₀)≤u

¾

−Γ(u)

¯¯

¯¯≤ C

√n where C := ¹₂ +^√1_2π + 2C₁+ 2C₂+^exp(−a_a^2/2) +C₅+C₆ when |u| ≥2√

lnn;

or C:= 2£ ₁

√2π + 2C₁+ 4C₂+ 2^exp(−a_a^2/2)+ 2C₅+C₆¤

+C₄+^{16e−1(C3+σ2cW)}

σ₁√ 2π

when|u|<2√

lnn provided thatp

n/lnn≥max¡

8c_Wσ²,4¢ /σ₁; with a:= inf_θ∈Θ¡

m(θ)/4σ₂(θ)¢

, σ:= sup_θ∈Θσ₁(θ)/m(θ),σ₁ = inf_θ∈Θσ₁(θ).

II.2 Proof of Theorem 1.

The hypotheses of Theorem 1 are assumed to hold. For the sake of brevity, the sequence {r_n}_n≥1 in (A3) is supposed to be such that r_n = o(n^−1/2), and |M_n⁰(bα_n)| ≤ r_n for every n ≥1. In the general case, it suffices to work on the event {|M_n⁰(αb_n)| ≤ r_n} and to bound the probability of the event {|M_n⁰(bα_n)|> r_n}using (A3). From Conditions (A2) and (A4), τ(θ) := σ₁(θ)

m(θ), m:= inf

θ∈Θm(θ), m:= sup

θ∈Θ

m(θ), σ_j := inf

θ∈Θσ_j(θ), σ_j := sup

θ∈Θ

σ_j(θ) (j = 1,2) are well defined. Recall that0< m≤m <∞ and 0< σ_j ≤σ_j <∞. Note that the function τ(·)is positive and bounded. In the following,C denotes a positive constant whose value may be different from line to line.

Inequality (4) is proved, first for |u| ≥ 2√

lnn, second for |u| < 2√

lnn. In fact, for

|u| ≥2√

lnn, the bound in Inequality (4) does not involve r_n and ω_n.

Proposition 1 There exists a positive constant C such that for each n ≥ 1 and all u ∈ R such that |u| ≥2√

lnn sup

θ∈Θ

¯¯

¯¯P_θ

½√ n

τ(θ)(bα_n−α₀)≤u

¾

−Γ(u)

¯¯

¯¯≤ C

√n +γ_n. (5) Proof. For |u| ≥2√

lnn, it is easily checked that

¯¯

¯¯P_θ

½√ n

τ(θ)(αb_n−α₀)≤u

¾

−Γ(u)

¯¯

¯¯≤P_θ

½√ n τ(θ)

¯¯bα_n−α₀¯

¯≥2√ lnn

¾

+ Γ(−2√ lnn).

Now,

Γ(−2√

lnn)≤ 1 2√

lnn

√1 2π

Z _+∞

2√ lnn

v e^−v22 dv= 1 2√

lnn

√1 2π

1 n². Finally, the proof is complete if there existsC >0 such that (see [MP71, Lem. 6])

∀n≥1, sup

θ∈Θ

P_θ

½√ n τ(θ)

¯¯bα_n−α₀¯

¯>2√ lnn

¾

≤ C

√n +γ_n. (6) It follows from (A5) and (A3) that|M_n⁰(α₀)|+r_n≥ |bα_n−α₀| |M_n⁰⁰(α₀)+R_n(bα_n, α₀)|. Then,

√n σ₁(θ)

¯¯bα_n−α₀¯¯>2

√lnn m(θ) =⇒

√n σ₁(θ)

¡¯¯M_n⁰(α₀)¯¯+r_n¢

>2

√lnn m(θ)

¯¯M_n⁰⁰(α₀) +R_n(bα_n, α₀)¯¯

provided that M_n⁰(αb_n) 6= M_n⁰(α₀). Next, introducing the event ©

2|M_n⁰⁰(α₀) +R_n(αb_n, α₀)|>

m(θ)ª

and its complement (which includes the event {M_n⁰(αb_n) =M_n⁰(α₀)}), we obtain P_θ

½√ n τ(θ)

¯¯bα_n−α₀¯

¯>2√ lnn

¾

≤ P_θ

½ √ n

σ₁(θ){|M_n⁰(α₀)|+r_n}>√ lnn

¾

+P_θ©

2|M_n⁰⁰(α₀) +R_n(αb_n, α₀)| ≤m(θ)ª .

It is easily checked from (A4) andr_n=o(n^−1/2) that sup

θ∈Θ

P_θ

½ √ n

σ₁(θ){|M_n⁰(α₀)|+r_n}>√ lnn

¾

=O µ 1

√n

¶ + 2 Γ

µ

−√ lnn+

√nr_n σ₁(θ)

¶

=O µ 1

√n

¶ .