On asymptotic normality of sequential LS-estimate for unstable autoregressive process AR(2).

HAL Id: hal-00271136

https://hal.archives-ouvertes.fr/hal-00271136

Preprint submitted on 8 Apr 2008

HAL is a multi-disciplinary open access archive for the deposit and dissemination of sci- entific research documents, whether they are pub- lished or not. The documents may come from teaching and research institutions in France or

L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires

On asymptotic normality of sequential LS-estimate for unstable autoregressive process AR(2).

Leonid Galtchouk, Victor Konev

To cite this version:

Leonid Galtchouk, Victor Konev. On asymptotic normality of sequential LS-estimate for unstable autoregressive process AR(2).. 2008. �hal-00271136�

On asymptotic normality of sequential LS-estimate for unstable autoregressive process AR(2).

By Leonid Galtchouk and Victor Konev

Louis Pasteur University of Strasbourg and University of Tomsk

Abstract

For estimating parameters in an unstableAR(2) model, the paper proposes a sequential least squares estimate with a special stopping time defined by the trace of the observed Fisher information matrix. It is shown that the sequential LSE is asymptotically normally distributed in the stability region and on its boundary in contrast to the usual LSE, having six different types of asymptotic distributions on the boundary depending on the values of the unknown parameters. ^{1 2}

∗The second author is partially supported by the RFFI-Grant 04-01-00855.

1AMS 2000 Subject Classification: 62L10, 62L12

2Key words: Autoregressive process, least squares estimate, sequential estimation, asymptotic normality .

1 Introduction

Consider an autoregressiveAR(2) model

x_n=θ₁x_n−1+θ₂x_n−2+ε_n, n= 1,2, . . . , (1.1) where (x_n) is the observation, (ε_n) is a sequence of independent identically distributed (i.i.d.) random variables withEε₁ = 0 and 0<Eε²₁ =σ² <∞, σ²is known,x₀ =x₋₁= 0. The process (1.1) is assumed to can be unstable, that is, both roots of the characteristic polynomial

P(z) =z²−θ₁z−θ₂ (1.2) lie on or inside the unit circle. The model (1.1) is a particular case of unstable autoregressive processesAR(p) which have been studied by many authors due to their applications in automatic control, identification and in modeling economic and financial time series (we refer the reader to Anderson (1971), Ahtola and Tiao (1987), Dickey and Fuller (1979), Chan and Wei (1988), Rao (1978) for details and futher references).

A commonly used estimate of parameter vector θ= (θ₁, θ₂)⁰ is the least squares estimate (LSE)

θ(n) = (θ₁(n), θ₂(n))⁰ =M_n⁻¹

n

X

k=1

X_k−1x_k, M_n=

n

X

k=1

X_k−1X_k⁰₋₁, (1.3) whereX_k= (x_k, x_k−1)⁰; the prime denotes the transpose;M_n⁻¹ denotes the inverse of matrix M_n if detM_n>0 and M_n⁻¹ = 0 otherwise.

It is well known that

√n(θ(n)−θ)=^L⇒ N(0, F), asn→ ∞,

for allθ ∈Λ, where Λ is the stability region of process (1.1), that is, Λ ={θ= (θ₁, θ₂)⁰ :−1 +θ₂ < θ₁<1−θ₂, |θ₂|<1}, (1.4)

F = F(θ) is a positive definite matrix (see, e.g., Anderson (1971), Th.

5.5.7),=^L⇒indicates convergence in law. Ifθbelongs to the boundary∂Λ of the stability region Λ, the limiting distribution of LSE is no longer normal.

Moreover, there is no one universal limiting distribution for allθ∈∂Λ and the corresponding set of limiting distributions numbers 6 different types depending on the values of roots z₁ and z₂ of the polynomial (1.2). Each limiting distribution of LSE on the boundary coincides with that of the ratio of certain Brownian functionals (we refer the reader to the paper of Chan and Wei (1988) for general results on the limiting distributions of the least squares estimates for unstable AR(p) processes and further details). For example, for conjugate complex roots z₁=e^iϕ, z₂ =e^−iϕ one has

n·(θ₁(n)−2 cosϕ)=^L⇒ (W₁²(1)−W₂²(1)) sinϕ+ (W₁²(1) +W₂²(1)−2) cosϕ R1

0[W₁²(s) +W₂²(s)]ds , n·(θ₂(n) + 1)=^L⇒(2−W₁²(1)−W₂²(1))/

Z 1 0

[W₁²(s) +W₂²(s)]ds , where (W₁(t),0 ≤t ≤1) and (W₂(t),0 ≤ t≤1) are independent standard Brownian motion processes; ifθ= (2,−1), then (see Theorem 3.1.2 ibid)



 n² 0

0 n



(θ(n)−θ)=^L⇒G⁻¹ξ, ξ=



 R₁

0 Z(t)dW(t) R1

0 W(t)dW(t)



, where

G=



 R₁

0 W²(t)dt R₁

0 W(t)Z(t)dt R1

0 W(t)Z(t)dt R1

0 Z²(t)dt



, Z(t) = Z t

0

W(s)ds .

It is well-known that a similar situation takes place in case ofAR(1) process

x_n=θx_n−1+ε_n, (1.5)

for which the limiting distributions of the least squares estimate are not normal at the end-points θ = ±1 of stability interval (-1,1) (see White (1958), Lai and Siegmund (1983)).

Lai and Siegmund (1983) for a first order non-explosive autoregressive process (1.5) proposed to use a sequential sampling scheme and proved that the sequential least squares estimate for θ with the stopping time based on the observed Fisher information is asymptotically normal uniformly in θ∈[−1,1] in contrast with the ordinary LSE.

In this paper we develop a sequential sampling scheme for estimating parameter vector θ = (θ1, θ2)⁰ in model (1.1). We will use the sequential least squares estimate defined by the formula

θ(τ(h)) =M_τ(h)⁻¹

τ(h)

X

k=1

X_k−1x_k, (1.6)

whereτ(h) is the stopping time for the thresholdh >0 : τ(h) = inf{n≥1 :

n

X

k=1

(x²_k−1+x²_k−2)≥hσ²}, inf{∅}= +∞. (1.7) This construction of sequential estimate is similar to that proposed in the paper of Lai and Siegmund forAR(1) which is defined as

θˆ_τ(h)=





τ(h)

X

k=1

x²_k−1





−1 τ(h)

X

k=1

x_k−1x_k, (1.8)

τ(h) = inf{n≥1 :

n

X

k=1

x²_k−1 ≥hσ²}. (1.9) It should be noted, however, that the first factor in (1.6) is a random matrix and not a random variable, as in (1.8), and this makes additional difficulties.

For AR(1) the stopping time (1.9) turns the denominator in (1.8) prac- tically into a constant hσ² and this allows to use the central limit theorem for martingales. In the case of AR(2) the stopping time (1.7) enables one to control the inverse matrix M_τ(h)⁻¹ in (1.6) only partially since it remains random. Nevertheless, we will see that such a change of time also enables one to improve the properties of the estimate (1.3).

In our paper (2006) we proved the following result.

Theorem 1.1. Let (εn)n≥1 in (1.1) be a sequence of i.i.d. random variables with Eε_n= 0, 0< Eε²_n=σ² <∞. Then, for any compact set K⊂Λ₁,

hlim→∞sup

θ∈K

sup

t∈R²|Pθ

M_τ(h)^1/2(θ(τ(h))−θ)≤t

−Φ₂(t/σ)|= 0, where Φ₂(t) = Φ(t₁)Φ(t₂), Φ is the standard normal distribution function,

Λ₁={θ= (θ₁, θ₂)⁰ : −1 +θ₂< θ₁ <1−θ₂, −1≤θ₂<1}, t= (t₁, t₂)⁰. This theorem implies, in particular, that estimate (1.6) is asymptotically normal not only inside the stability region (1.4) but also on the part of its boundary{θ = (θ₁,−1)⁰ :−2< θ₁<2}in contrast to the LSE (1.3).

The goal of this paper is to prove the asymptotic normality of the esti- mate (1.6),(1.7) in the whole region [Λ] including its boundary∂Λ.

Our main result (Theorem 3.1) claims that, as h→ ∞,

M_τ(h)^1/2(θ(τ(h))−θ)=^L⇒ N(0, σ²I), (1.10) for any θ= (θ₁, θ₂)⁰ inside the stability region Λ (1.4) and on its boundary

∂Λ, whereI is the identity matrix. Thus the sequential estimate (1.6), (1.7) has a unique normal asymptotic distribution in the closure [Λ] of the stability region (1.4). It will be observed that the normalizing factor M_τ(h)^1/2 in the limit theorem (1.10) remains the same in the whole region [Λ] in contrast to the case of the LSE (1.3), which has seven different limiting distributions in [Λ] and in order to apply the limiting distributions one needs some knowledge about the location of unknown parameters (see Chan and Wei (1988)). The convergence of the sequential estimate (1.6), (1.7) to the normal distribution in (1.10) is not uniform inθforθ∈[Λ]. It can be explained by the fact that in the case, when the polynomial (1.2) has one root inside and the other on the unit circle, the rates of information provided by sample valuesx_nabout the unknown parametersθ₁ and θ₂ may differ greatly.

Theorem 3.1 permits setting up tests of hypotheses aboutθand forming asymptotic confidence regions forθ on the basis of standard normal distri- bution. Moreover, the asymptotic normality holds in [Λ] for a broad class of the distributions of noises (ε_n).

The remainder of this paper is arranged as follows. Section 2 gives the asymptotic distribution of the stopping time (1.7) (Theorem 2.1) and some properties of the observed Fisher information matrix. In section 3 the asymptotic normality of sequential estimate (1.6) for unstableAR(2) model is established (Theorem 3.1). Section 4 proposes the sequential estimation scheme for the case of unknown variance σ² in model (1.1). The appendix contains some technical results.

2 Properties of the stopping time τ ( h ) and the ob- served Fisher information matrix M

.

In this section the attention is mainly focused on the case when the unknown parameterθ = (θ1, θ2)⁰ belongs to the boundary∂Λ of the stability region (1.4). The boundary∂Λ includes three sides:

Γ₁ ={θ:−θ₁+θ₂= 1,−2< θ₁<0},Γ₂ ={θ:θ₁+θ₂ = 1,0< θ₁ <2}, Γ₃ ={θ :−2< θ₁ <2, θ₂ =−1} (2.1) and three apexes (0,1),(−2,−1),(2,−1). Denote

A=





θ₁ θ₂

1 0



, B=



 1 0 0 0



,

W⁽ⁿ⁾(t) = 1 σ√

n

[nt]

X

i=0

ε_i, W₁⁽ⁿ⁾(t) = 1 σ√

n

[nt]

X

i=0

(−1)ⁱε_i, 0≤t≤1, (2.2)

and introduce the following functionals J1(x;t) =

Z _t

0

x²(s)ds, J2(x;t) = Z _t

0

Z s 0

x(u)du 2

ds, (2.3) J3(x;y;t) =

Z t 0

(x²(s) +y²(s))ds, J4(x;t) = Z t

0

x(s)ds 2

.

Theorem 2.1. Let (ε_n)_n≥1 in (1.1) be a sequence of i.i.d. random variables with Eεn = 0, Eε²_n = σ² and τ(h) be defined by (1.7). Denote by a and b real roots of the polynomial (1.2), −1≤a < b≤1. Then, for each θ∈Λ,

P^θ− lim

h→∞τ(h)/h= 1/trF, F −AF A⁰ =B. (2.4) Moreover, for eachθ ∈∂Λ, as h→ ∞,

τ(h) ψ(θ, h)

=L⇒



























ν₁(W₁) = inf{t≥0 :J1(W₁;t)≥1} ifθ∈Γ₁, ν₂(W) = inf{t≥0 :J1(W;t)≥1} ifθ ∈Γ₂,

ν₃(W, W₁) = inf{t≥0 :J3(W;W₁;t)≥1} ifθ ∈Γ₃∪ {(0,1)}, ν₄(W) = inf{t≥0 :J2(W;t)≥1} ifθ = (2,−1),

ν₅(W₁) = inf{t≥0 :J2(W₁;t)≥1} ifθ= (−2,−1),

(2.5) where inf{∅}=∞, Λ is defined in (1.4),

ψ(θ, h) =



























(1 +b)p

h/2 if θ∈Γ₁, (1−a)p

h/2 ifθ ∈Γ2,

√2hsinϕ ifθ= (2 cosϕ,−1)⁰ ∈Γ₃,

√2h if θ= (0,1),

(h/2)^1/4 ifθ∈ {(−2,−1),(2,−1)},

(2.6)

W(t), W₁(t) are independent standard Brownian motions.

Proof Assertion (2.4) easily follows from Lemma 3.12 in [6].

For θ ∈ ∂Λ we decompose the original process (1.1) into two processes (u_k)_k≥1 and (v_k)_k≥1 using the transformation

QX_k= (u_k, v_k)⁰, (2.7)

where Q is a non-degenerate constant matrix of size 2×2 which will be chosen later depending on the values of θ. The limiting relation (2.5) for θ∈ ∪³i=1Γ_i has been proved in [7], Th 2.2. It remains to consider the apexes (2,−1),(−2,−1),(0,1).

Forθ = (2,−1), putting in (2.7)

Q=





1 0

1 −1



 (2.8)

one obtains v_k=

k

X

j=1

ε_j, u_k=

k

X

j=1

(x_j−x_j−1) =

k

X

j=1

v_j =

k

X

j=1 j

X

i=1

ε_i,

n

X

k=1

kX_k−1k² =

n

X

k=1

u²_k−1+

n

X

k=1

u²_k−2 = 2

n

X

k=1

u²_k−1−u²_n−1. (2.9) By the definition of τ(h) in (1.7), one gets

Pθ{τ(h)≤th^1/4}=Pθ{

[th^1/4]

X

k=1

kX_k−1k²≥hσ²} (2.10)

=Pθ{ 2 hσ²

[th^1/4]

X

k=1

u²_k−1− 1

hσ²u²_[th1/4]−1 ≥1}.

Further we show (by the argument similar to that in the proof of Lemma 2.3 in the Appendix) that the sum

S_n(t) = 1 n⁴σ²

[nt]

X

k=1

u²_k−1

satisfies the relation

S_n(t) =J2(W⁽ⁿ⁾;t) +g⁽ⁿ⁾(t), whereg⁽ⁿ⁾(t) is a random process such that, for anyδ >0,

nlim→∞Pθ(|g⁽ⁿ⁾(t)|> δ) = 0.

Now we check that

nlim→∞u²_n/n⁴ = 0 Pθ −a.s.. (2.11) By the Cauchy-Schwarz-Bunyakovskii inequality and the law of iterated log- arithm we have

u²_n/n⁴≤n⁻³

n

X

k=1





k

X

j=1

ε_i





2

,

n

X

k=1

1 k³





k

X

j=1

ε_i





2

<∞ Pθ−a.s.. These inequalities, in virtue of the Kronecker Lemma, imply (2.11).

From here and (2.10), (2.11), we obtain

Pθ(τ(h)/ψ(θ, h)≤t) =Pθ(νθ⁽ⁿ⁾≤t) +β^θ(h), where

νθ⁽ⁿ⁾ = inf{t≥0 :J2(W⁽ⁿ⁾;t)≥1}, lim

h→∞β^θ(h) = 0,

W⁽ⁿ⁾(t) is given in (2.2). This, by the functional Donsker theorem (see Billingsley (1968)), leads to (2.5) for θ= (2,−1).

The case of the apexes (0,1),(−2,−1) can be considered similarly with the use of Theorem 5.14 given in the Appendix. This completes the proof of Theorem 2.1.

Now we will establish some properties of the observed Fisher information matrixM_n. Introduce the following subsets of the closed region [Λ] :

Λ_d = [Λ]\

2

[

i=1

B_i, Λ_d = Λ_d,1+ Λ_d,2, (2.12) where

Λ_d,1= Λ_d∩V_d, Λ_d,2 = Λ_d\Λ_d,1; V_d =

θ :−2 + d

√2 ≤θ₁≤0, −θ²₁ 4 +d²

8 < θ₂≤1 +θ₁

∪

θ: 0≤θ₁≤2− d

√2, −θ₁² 4 +d²

8 ≤θ₂ ≤1−θ₁

;

Bi are open balls of radiusd >0 centered at the apexes (−2,−1),(2,−1).

In view of Theorem 1.1, it suffices to study the properties of M_n only for the parametric subset Λ_d,1 and the apexes (−2,−1),(2,−1). In the case of Λ_d,1, one can use the transformation (2.7) with

Q=





1 −b 1 −a



, (2.13)

where−1≤a < b≤1. Substituting (2.7) and (2.13) inM_n (1.3) yields M_n=Q⁻¹S_n(Q⁰)⁻¹ =Q⁻¹R⁻_n¹J_nR⁻_n¹(Q⁰)⁻¹, (2.14) where

Sn=





(u, u)_n (u, v)_n (u, v)_n (v, v)_n



, Rn=





(u, u)⁻n^1/2 0 0 (v, v)⁻_n^1/2



,

Jn=RnSnRn=





1 ξ_n ξ_n 1



, (2.15)

ξn= (u, u)⁻_n^1/2(v, v)⁻_n^1/2(u, v)n, (u, v)n=

n

X

k=1

u_k−1v_k−1. (2.16) Proposition 2.2. Under conditions of Theorem 2.1, for any d >0, δ >0,

hlim→∞ sup

θ∈Λd,1

Pθ kJ_τ(h)−T(θ₁, θ₂)k> δ

= 0, (2.17)

where

T(θ₁, θ₂) =





1 r(a, b) r(a, b) 1



, r(a, b) =

√1−a²√ 1−b²

1−ab . (2.18) The proof of Proposition 2.2 is given in the Appendix.

Further we consider the asymptotic behaviour of the matrix J_n in the extreme cases when the process x_k is ”most” unstable, that is, θ coincides with one of the apexes (−2,−1),(2,−1) of the parametric region [Λ].

Forθ = (2,−1) we take the matrix Q from (2.8). This yields u_k =

k

X

j=0 j

X

i=0

ε_i, v_k=

k

X

j=0

ε_j, k≥1, u₀=v₀=ε₀ = 0. (2.19) Forθ= (−2,−1) we take

Q=



 1 0 1 1



.

This implies

u_k= (−1)^k

k

X

j=1 j

X

i=1

(−1)ⁱεi, v_k=

k

X

j=1

(−1)^jεj.

Lemma 2.3. Let ξ_n be given by (2.16) and θ∈ {(−2,−1),(2,−1)}. Then

ξ_n=^L⇒







ϕ(W) if θ= (2,−1), ϕ(W1) if θ= (−2,−1),

as n→ ∞, (2.20)

where

ϕ(W) = 2⁻¹J2^−1/2(W; 1)J1^−1/2(W; 1)J4(W; 1). (2.21) The proof of Lemma 2.3 is given in the Appendix.

3 Asymptotic normality.

It is known that the sequential least squares estimate (1.6),(1.7) is asymp- totically normal just like the ordinary LSE for any value ofθin the stability region Λ. Moreover, according to Theorem 1.1, this convergence of sequen- tial LSE to normal law is uniform in θ belonging to any compact set in Λ supplemented with the part of its boundary corresponding to complex roots of the polynomial (1.2). In this section, we will show that in contrast with the ordinary LSE (c.f. Chen and Wei (1988)), the sequential LSE is asymptotically normal also on the boundary∂Λ of the stability region Λ.

Theorem 3.1. Suppose that in AR(2) model (1.1), (εn)n≥1 is a sequence of i.i.d. random variables, Eε_n = 0 and 0 < Eε²_n = σ² < ∞. Define τ(h), θ(τ(h))and M_τ(h) as in (1.6),(1.7) and (1.3). Then for any θ∈[Λ]

hlim→∞sup

t∈R²

Pθ

M_τ(h)^1/2(θ(τ(h))−θ)≤t

−Φ₂(t/σ)

= 0, (3.1) where Φ₂(t) = Φ(t₁)Φ(t₂), t= (t₁, t₂)⁰, Φ is the standard normal distribu- tion function; [Λ] is the closure of the stability region (1.4).

Proof of Theorem 3.1 In view of Theorem 1.1, we have to show (3.1) forθ ∈ Γ₁∪Γ₂∪ {(0,1),(−2,−1),(2,−1)}. First we note that ifθ ∈ Γ₁∪ Γ₂∪ {(0,1)}, the minimal and the maximal rootsaand bof the polynomial (1.2) satisfy the inequalities −1 ≤ a < b ≤ 1. Therefore one can use the transformation (2.7),(2.13) to decompose the original process AR(2) (1.1) into two processes (u_k) and (v_k) which obey the equations

u_k=au_k−1+ε_k, v_k=bv_k−1+ε_k, u₀=v₀= 0. (3.2) Since the matrix Q in (2.13) is non-degenerate, one can represent the ob- served Fisher information matrixM_n in the form (2.14) to obtain

M_n^1/2 =Q⁻¹R⁻_n¹J_n^1/2. (3.3) Substituting this matrix in the standardized deviation of the sequential es- timate (1.6), one gets

M_τ(h)^1/2(θ(τ(h))−θ) =M_τ(h)^−1/2

τ(h)

X

k=1

X_k−1ε_k

=J_τ(h)^−1/2R_τ(h)

τ(h)

X

k=1

QX_k−1ε_k =J_τ(h)^−1/2Z_τ(h), where

Z_n=





(u, u)⁻_n^1/2Pn

k=1u_k−1ε_k (v, v)⁻n^1/2P_n

k=1v_k−1ε_k



. (3.4)

Further we note that Proposition 2.2 implies that, for anyδ >0,

hlim→∞ sup

θ∈Γ1∪Γ2∪{(0,1)}

Pθ

kJ_τ(h)^−1/2−Ik> δ

= 0. (3.5)

Therefore in order to prove (3.1) for θ ∈ Γ₁ ∪Γ₂ ∪ {(0,1)} it suffices to establish the following result.

Proposition 3.2. Let θ∈Γ₁∪Γ₂∪{(0,1)}. Then, for each constant vector λ= (λ₁, λ₂)⁰ ∈R² with kλk= 1, the random variable

Y_h =λ⁰Z_τ(h)/σ (3.6)

is asymptotically normal with mean 0and unit variance, as h→ ∞, that is,

hlim→∞sup

t∈R|Pθ(Y_h ≤t)−Φ(t)|= 0.

The main difficulty in the analysis of Y_h is that the stopping time (1.7) enables one to control the sums (u, u)_τ(h), (v, v)_τ(h) in the denominators of (3.6) only partially because one of them or both are random variables even in the asymptotics as h→ ∞.

The proof of Proposition 3.2 is given in the Appendix. The key idea of the proof is to replaceY_hby a more tractable random variable ˜Y_hequivalent to Y_h in distribution by making use of the Skorohod coupling theorem and then apply the Central Limit Theorem for martingales. The appendix con- tains also the proof of Theorem 3.1 for the case of θ ∈ {(−2,−1),(2,−1)}. This case is considered separately because the matrixJ_n in (3.3) converges, according to Lemma 2.3, only in distribution.

4 Asymptotic normaliy in the case of unknown variance.

In this section, we extend the sequential estimation scheme to model (1.1) with unknown variance. It is shown that the sequential least squares es-

timate modified to embrace this case remains asymptotically normal uni- formly inθ for any compact set in the region Λ₁ = Λ∪Γ₃ (Th. 4.1) and it is asymptotically normal in the closure of the stability region [Λ] (Th. 4.2).

Suppose that the variance σ² in (1.1) is unknown. A commonly used estimate for σ² in autoregression processes on the basis of observations (x₁, . . . , x_n) is defined as

ˆ

σ_n² =n⁻¹

n

X

k=1

(x_k−θ⁰(n)X_k−1)², (4.1) whereθ(n) is the least squares estimate ofθ defined in (1.3). Now we must modify the stopping time (1.7). At first sight, to this end one should replace σ² in (1.7) by ˆσ_n². However, we will use a different modification similar to that proposed by Lai and Siegmund for AR(1) model, which turns out to be more convenient in the theoretic studies. Define the sequential estimate as

θ(ˆτ(h)) =M_τ(h)ˆ⁻¹

ˆ τ(h)

X

k=1

X_k−1x_k, (4.2)

ˆ

τ(h) = inf{n≥3 :

n

X

k=1

(x²_k−1+x²_k−2)≥hs²_n}, (4.3) wheres²_n= ˆσ_n²∨δ_n, δ_n is a sequence of positive numbers withδ_n→0.

The main results of this section are stated in the following theorems.

Theorem 4.1. Let(εn)n≥1in (1.1) be a sequence of i.i.d. random variables, Eε_n= 0, 0< Eε²_n=σ² <∞. Then, for any compact set K ⊂Λ₁,

hlim→∞sup

θ∈K

sup

t∈R²|Pθ

M_τ(h)ˆ^1/2(θ(ˆτ(h))−θ)/ˆσ_ˆτ(h)≤t

−Φ₂(t)|= 0, (4.4) where Φ₂(t) = Φ(t₁)Φ(t₂), Φ is the standard normal distribution function,

Λ₁={θ= (θ₁, θ₂)⁰: −1 +θ₂< θ₁ <1−θ₂, −1≤θ₂<1}, t= (t₁, t₂)⁰.

Theorem 4.2. Let(εn)n≥1in (1.1) be a sequence of i.i.d. random variables, Eε_n= 0, 0< Eε²_n=σ² <∞. Then, for any θ∈[Λ],

hlim→∞sup

t∈R²|Pθ

M_τˆ^1/2_(h)(θ(ˆτ(h))−θ)/ˆσ_τ(h)ˆ ≤t

−Φ2(t)|= 0.

The proofs of Theorems 4.1- 4.2 proceed along the lines of those of The- orems 1.1 and 3.1 though they become more laborious because one needs to control the additional terms appearing as a result of the unknown variance.

We will give only the proof of Theorems 4.1.

Proof of Theorems 4.1. Substituting (1.1) in (4.2) yields

M_τ(h)ˆ^1/2(θ(ˆτ(h))−θ)/ˆσ_τ(h)ˆ =M_τ(h)ˆ^−1/2

ˆ τ(h)

X

k=1

X_k−1ε_k/ˆσ_τ(h)ˆ

=

M_ˆτ(h)ˆσ_τ(h)²_ˆ /(σ⁴h/2)₋1/2 ˆτ(h)

X

k=1

X_k−1ε_k/(σ²p

h/2). (4.5) Further we need the following results.

Lemma 4.3. Let M_n,τˆ(h) be given by (1.3), (4.3). Then, for any compact setK ⊂Λ1 andδ >0,

hlim→∞sup

θ∈K

Pθ

kM_τ(h)ˆ σˆ²_ˆτ(h)/(σ⁴h/2)−L(θ₁, θ₂)k> δ

= 0, (4.6) where

L(θ₁, θ₂) =





1 θ₁/(1−θ₂) θ₁/(1−θ₂) 1



.

Lemma 4.4. Under the assumptions of Theorem 1.1, for any compact set K⊂Λ₁ and for each constant vector λ= (λ₁, λ₂)⁰ with kλk= 1,

hlim→∞sup

θ∈K

sup

t∈R|P^θ(Y_h≤t)−Φ(t)|= 0, where

Y_h=λ⁰L^−1/2(θ₁, θ₂)

ˆ τ(h)

X

k=1

X_k−1ε_k/(σ²p h/2).

The proofs of these Lemmas are given below in this section.

Now we rewrite (4.5) as M_τ(h)ˆ^1/2(θ(ˆτ(h))−θ)/ˆσ_τ(h)ˆ =

M_τˆ(h)σˆ²_τ(h)ˆ /(σ⁴h/2)₋1/2

L^1/2(θ₁, θ₂) (4.7)

×L^−1/2(θ₁, θ₂)

ˆ τ(h)

X

k=1

X_k−1ε_k/(σ²p h/2). According to Lemma 4.3 we have for eachδ >0

hlim→∞sup

θ∈K

Pθ

k

M_τ(h)ˆ σˆ_τ(h)²_ˆ /(σ⁴h/2)₋1/2

L^1/2(θ1, θ2)−Ik> δ

= 0. From here and (4.7) by applying Lemma 4.4, we come to (4.4). This com- pletes the proof of Theorem 4.1.

In order to prove Lemmas 4.3, 4.4, we need the following result.

Proposition 4.5. Let θ(n) and σˆ_n² be given by (1.3) and (4.1). Then, for any compact set K⊂Λ₁ and δ >0,

mlim→∞sup

θ∈K

Pθ(kθ(n)−θk> δ for some n≥m) = 0, (4.8)

mlim→∞sup

θ∈K

Pθ |σˆ²_n−σ²|> δ for some n≥m

= 0. (4.9) Proof. We have

θ(n)−θ= (M_n/(x, x)_n)⁻¹(x, x)⁻_n¹

n

X

k=1

X_k−1ε_k.

By Lemma 3.3 in [7], for any δ > 0 and any compact K ⊂Λ= [Λ]^◦ \ {(0,1),(−2,−1),(2,−1)},

mlim→∞sup

θ∈K

Pθ(kMn/(x, x)n−L(θ1, θ2)k> δ for some n≥m) = 0. (4.10) Further it will be observed that, for any 0 < C < ∞ and compact set K, there exists a positive number ∆ that, for all matrices L(θ₁, θ₂) with θ= (θ₁, θ₂)⁰ ∈KandB such thatkB−L(θ₁, θ₂)k<∆, one haskB⁻¹k ≤C.

LetC, B be such a pair. Then, for each θ∈K, we have the inclusions (kθ(n)−θk> δ for some n≥m)

⊆ k(M_n/(x, x)_n)⁻¹k(x, x)⁻_n¹k

n

X

k=1

X_k−1ε_kk> δ for some n≥m

!

= (•)⊆(•,kM_n/(x, x)_n−L(θ₁, θ₂)k ≤∆ for all n≥m)

∪(•,kMn/(x, x)n−L(θ1, θ2)k>∆ for somen≥m)

⊂ C(x, x)⁻_n¹k

n

X

k=1

X_k−1ε_kk> δ for some n≥m

!

∪(kMn/(x, x)n−L(θ1, θ2)k>∆ for some n≥m) . This yields

Pθ(kθ(n)−θk> δ for some n≥m)

≤Pθ (x, x)⁻_n¹k

n

X

k=1

X_k−1ε_kk> δ⁰ for somen≥m

!

+Pθ(kM_n/(x, x)_n−L(θ₁, θ₂)k>∆ for some n≥m), δ⁰ =δ/C . By Lemmas 3.2,3.3 from [7], limiting m→ ∞, we come to (4.8).

Consider (4.9). Rewrite ˆσ_n² in (4.1) as ˆ

σ_n² =n⁻¹

n

X

k=1

(ε_k+ (θ−θ(n))⁰X_k−1)²=n⁻¹

n

X

k=1

ε²_k

+2n⁻¹(θ−θ(n))⁰

n

X

k=1

X_k−1ε_k+n⁻¹(θ−θ(n))⁰

n

X

k=1

X_k−1X_k⁰₋₁(θ−θ(n)). Substituting hereθ(n) from (1.3) yields

ˆ

σ²_n−σ² = n⁻¹

n

X

k=1

ε²_k−σ²

!

−2n⁻¹

n

X

k=1

X_k⁰₋₁ε_k

! M_n⁻¹

n

X

k=1

X_k−1ε_k

+n⁻¹

n

X

k=1

X_k⁰₋₁ε_k

!

M_n⁻¹M_nM_n⁻¹

n

X

k=1

X_k−1ε_k

= n⁻¹

n

X

k=1

ε²_k−σ²

!

−n⁻¹

n

X

k=1

X_k⁰₋₁ε_k

! M_n⁻¹

n

X

k=1

X_k−1ε_k

= n⁻¹

n

X

k=1

ε²_k−σ²

!

− 1 n(x, x)_n

n

X

k=1

X_k⁰₋₁ε_k

!

(M_n/(x, x)_n)⁻¹

n

X

k=1

X_k−1ε_k. The first term in the right-hand side of this equality converges to zero in virtue of the strong law of large numbers. Therefore, in order to prove (4.9), we have to verify that, for eachK ⊂Λ₁ and δ >0,

mlim→∞sup

θ∈K

Pθ

1 n

n

X

k=1

X_k⁰₋₁ε_k

! M_n⁻¹

n

X

k=1

X_k−1ε_k > δ for some n≥m

!

= 0 In view of Lemma 3.3 in [7], it is equivalent to the following limiting relations

mlim→∞sup

θ∈K

Pθ





n

X

k=1

x_k−1ε_k

2

> δn(x, x)n for some n≥m



= 0, (4.11)

mlim→∞sup

θ∈K

Pθ





n

X

k=1

x_k−2ε_k

2

> δn(x, x)_n for some n≥m



= 0. (4.12) To prove these relations we will make use of Lemma 2.2 from [10]. First we note that the matrix Adefined in (2.2) possesses the property (see, [7]):

sup

θ∈KkAⁿk ≤κ, n= 1,2, . . . , (4.13) whereκ is some positive number. This implies the following inequality

(x, x)_n≤κ²

n

X

k=1

(

k

X

j=1

|ε_j|)² =:U_n. (4.14) Indeed, writing down (1.1) in the vector form

X_k=AX_k−1+ξ_k, ξ_k= (ε_k,0)⁰, and using the formulaX_k=Pk

j=1 A^k−jξ_j, lead to the estimate

|x_k| ≤ kX_kk ≤κ

k

X

j=1

|ε_j|

and, hence, to (4.14). By making use of the law of iterated logarithm and the Kronecker Lemma, one can show thatU_nin (4.14) satisfies the following relation

U_n=o(n⁴) a.s. (4.15)

Now let us prove, for example, (4.11). From the inequality under the sign of probability in (4.11), it follows that

n

X

k=1

x_k−1ε_k

> δ^1/2(x, x)^5/8_n n⁴/(x, x)_n1/8

≥δ^1/2(x, x)^5/8_n n⁴/U_n1/8

. (4.16)

This enables us to obtain the following inclusions for ∆< σ²:

|

n

X

k=1

x_k−1ε_k|> δ^1/2n^1/2(x, x)^1/2_n for some n≥m

!

⊆ •, |n⁻¹

n

X

k=1

ε²_k−σ²| ≤∆ for all n≥m

!

∪ |n⁻¹

n

X

k=1

ε²_k−σ²|>∆ for some n≥m

!

⊆ •, n⁻¹

n

X

k=1

ε²_k> σ²−∆ for all n≥m

!

∪ |n⁻¹

n

X

k=1

ε²_k−σ²|>∆ for some n≥m

!

⊆ •, n⁻¹

n

X

k=1

ε²_k > σ²−∆ all n≥m, U_n/n⁴≤1 alln≥m

!

∪ |n⁻¹

n

X

k=1

ε²_k−σ²|>∆ for some n≥m

!

∪(U_n

n⁴ >1 for some n≥m). From here one gets

Pθ

n

X

k=1

x_k−1ε_k

> δ^1/2n^1/2(x, x)^1/2_n for somen≥m

!

≤Pθ

n

X

k=1

x_k−1ε_k

> δ^1/2

4⁻¹n^3/2(σ²−∆)∨(x, x)^5/8_n

for some n≥m

!

+Pθ |1 n

n

X

k=1

ε²_k−σ²|>∆ for some n≥m

! +Pθ

U_n

n⁴ >1 for somen≥m

.

In order to come to (4.11), it remains to use Lemma 2.2 from [10], the strong law of large numbers and putm→ ∞ . This completes the proof of Proposition 4.5.

Proof of Lemma 4.3. We start with the representation M_τ(h)ˆ σˆ²_τ(h)ˆ

σ⁴h/2 = M_τ(h)ˆ

(x, x)_ˆτ(h) · (x, x)_τˆ(h) 2⁻¹Pτ(h)ˆ

k=1kX_k−1k² · Pτ(h)ˆ

k=1kX_k−1k² hs²_ˆτ(h) · s²_τ(h)ˆ

σ² ·σˆ²_ˆτ(h) σ² . It suffices to show, for any δ >0, the limiting relations

hlim→∞sup

θ∈K

Pθ

kM_τ(h)ˆ /(x, x)_ˆτ(h)−L(θ₁, θ₂)k> δ

= 0, (4.17)

hlim→∞sup

θ∈K

Pθ





(x, x)_ˆτ(h)



2⁻¹

ˆ τ(h)

X

k=1

kX_k−1k²





−1

−1

> δ



= 0, (4.18)

hlim→∞sup

θ∈K

Pθ





ˆ τ(h)

X

k=1

kX_k−1k²/(hs²_τ(h)ˆ )−1

> δ



= 0, (4.19)

hlim→∞sup

θ∈K

Pθ

|s²_ˆτ(h)/σ²−1|> δ

= 0, (4.20)

hlim→∞sup

θ∈K

Pθ

|σˆ_τ(h)²_ˆ /σ²−1|> δ

= 0. (4.21)

Consider (4.17). We have Pθ

kM_ˆτ(h)/(x, x)_τ(h)ˆ −L(θ₁, θ₂)k> δ

≤Pθ(ˆτ(h)≤m) (4.22) +Pθ(kM_n/(x, x)_n−L(θ₁, θ₂)k> δ for somen≥m) .

In view of (4.10), we need to check only that, for each sufficiently largem,

hlim→∞sup

θ∈K

Pθ(ˆτ(h)≤m) = 0. (4.23) Letm₀ be a number such that, for all m≥m₀, the sequence (δ_m) satisfies the inequality δ_m ≤ σ²/2. By the definition of the stopping time ˆτ(h) in (4.3), it follows that

Pθ(ˆτ(h)≤m) =Pθ(

m

X

k=1

kX_k−1k² ≥hs²_m)

=Pθ(

m

X

k=1

kX_k−1k² ≥hδ_m, δ_m≥σˆ_m²) +Pθ(

m

X

k=1

kX_k−1k² ≥hˆσ²_m, δ_m<ˆσ²_m)

≤Pθ(ˆσ_m² ≤δ_m) +Pθ m

X

k=1

kX_k−1k²≥hˆσ_m²

!

≤Pθ(|σˆ²_m−σ²| ≥σ²/2) +Pθ m

X

k=1

kX_k−1k² ≥hˆσ_m²

!

. (4.24)

Further we have Pθ

m

X

k=1

kX_k−1k²≥hˆσ_m²

!

=Pθ(•, |σˆ²_m−σ²| ≤∆) +Pθ(•, |σˆ²_m−σ²| >∆)

≤Pθ m

X

k=1

kX_k−1k² ≥h(σ²−∆)

!

+Pθ(|σˆ_m² −σ²|>∆). (4.25) The inequalities (4.24),(4.25), in view of Proposition 4.5, imply (4.23). This leads to (4.17). To show (4.18) we use the identity

(x, x)_n= 2⁻¹

n

X

k=1

kX_k−1k²+x²_n−1/2,

(4.23) and apply Lemma 3.1 from [7]. The relations (4.19)-(4.21) can be checked in a similar way. This completes the proof of Lemma 4.3.

Proof of Lemma 4.4. We will use the argument similar to that in the proof of Proposition 2.1 in [10]. First we introduce a sequence (ˆx_n) of truncated observations (x_n) defined as

ˆ x_n=







x_nif x²_n≤δ²h , δ√

h ifx²_n> δ²h, 0< δ <1, and the set

Ωˆ_h = (x_n = ˆx_n for all n <τˆ_h).

Along the lines of the proof of Proposition 2.1, one can verify that

hlim→∞sup

θ∈K

Pθ( ˆΩ^c_h) = 0. (4.26)