A new statistical procedure for testing the presence of a significative correlation in the residuals of stable autoregressive processes

A new statistical procedure for testing the presence of a significative correlation in the residuals of stable

autoregressive processes

Frédéric Proïa

Colloque Jeunes Probabilistes et Statisticiens CIRM - Marseille

16-20 avril 2012 INRIA Bordeaux Sud-Ouest Institut de Mathématiques de Bordeaux

Plan

1 Introduction

Framework of the study Hypothesis

Stability conditions

2 On the autoregressive parameter Estimation and convergence Central limit theorem Rates of convergence

3 On the serial correlation parameter Estimation and convergence Central limit theorem Rates of convergence

4 Testing the residual autocorrelation of the AR(p)process The Durbin-Watson statistic

Test statistic

Empirical power and comparisons

5 Conclusion

Introduction

Framework of the study

• Significance of the serial correlation in the residuals.

• Autoregressive process of orderp.

• For alln≥1,

( Xn = θ₁Xn−1+. . .+θ_pXn−p+ε_n εn = ρεn−1+Vn

whereX0=ε₀are square-integrable andX−1,X−2, . . . ,X−p=0.

• Preliminary study.

• The empirical process is asymptotically stationary.

• (Xn)has an autoregressive behavior.

• Objectives.

• Sharp analysis on the least squares estimators ofθandρ.

• Statistical procedure for testingH₀: “ρ=0”vsH₁: “ρ6=0”.

• Comparison of the empirical power of the test with commonly used procedures.

• Use of the Durbin-Watson statistic.

• ArXiv : 1203.1871.

Introduction

Hypothesis

• Causality of the model.

• Compact expression, for all 1≤t≤N, A(B)Xt=εt

where the polynomialA(z) =1−β₁z−. . .−βpz^p+θpρz^p+1and β= θ₁+ρ θ₂−θ₁ρ . . . θ_p−θ_p−1ρ0.

• Ais causal :A(z)6=0 for allz∈Csuch that|z| ≤1.

• kθk1<1 and|ρ|<1 imply causality.

• OnN, causality often coincides with asymptotic stationarity.

• Autoregressive structure of order at leastp.

• θp6=0.

• Moments on the noise.

• (Vn)i.i.d. such thatE[V1] =0,E[V₁²] =σ²andE[V₁⁴] =τ⁴.

• Possible to do without i.i.d. assumption.

Introduction

Stability conditions

• Serial correlation stability.

• |ρ|<1 translates moments properties from(Vn)to(εn).

• Autoregression stability.

• kθk₁<1 translates moments properties from(εn)to(Xn).

Lemma (Stability, M. Duflo)

Assume that(Vn)is a sequence of independent and identically distributed random variables such that, for some a≥1,E[|V₁|^a]is finite. Then,

n

X

k=0

|Xk|^a=O(n) a.s. and sup

0≤k≤n

|Xk|=o(n^1/a) a.s.

• More generally.

• For allj≥0,

n

X

k=1

Xk−jXk=O(n) a.s.

Plan

1 Introduction

Framework of the study Hypothesis

Stability conditions

2 On the autoregressive parameter Estimation and convergence Central limit theorem Rates of convergence

3 On the serial correlation parameter Estimation and convergence Central limit theorem Rates of convergence

4 Testing the residual autocorrelation of the AR(p)process The Durbin-Watson statistic

Test statistic

Empirical power and comparisons

5 Conclusion

On the autoregressive parameter

Estimation and convergence

• Least squares estimator.

• For alln≥0, denote by

Φ^p_n= Xn Xn−1 . . . Xn−p+10

and

Sn=

n

X

k=0

Φ^p_kΦ^p_k⁰+S.

• In the AR(p)structureXn=θ⁰Φ^p_n−1+εn, for alln≥1,

θb_n= (Sn−1)⁻¹

n

X

k=1

Φ^p_k−1Xk.

• Sis a symmetric positive definite matrix added to ensure the invertibility ofSn.

On the autoregressive parameter

Estimation and convergence

• Convergence.

Theorem

We have the almost sure convergence

n→∞lim bθn=θ^∗ a.s.

• The limiting value is given by

θ^∗=α(Ip−θpρJp)β

whereIpis the identity matrix of orderp,Jpis the exchange matrix of orderpand α= (1−θpρ)⁻¹(1+θpρ)⁻¹.

• Strong consistency whenρ=0 under the stability conditions (Lai and Wei, 1983).

On the autoregressive parameter

Central limit theorem

• Central limit theorem.

Theorem

Assume that(Vn)has a finite moment of order 4. Then, we have the asymptotic normality

√n θbn−θ^∗

_L

n→∞−→ N(0,Σ_θ).

• The asymptotic covariance matrix is given by

Σθ=α²(Ip−θpρJp) ∆⁻¹_p (Ip−θpρJp) where∆pis the positive definite almost sure limiting matrix ofσ⁻²Sn/n.

• ∆pis the covariance matrix of the standard stationary process built from(Xn).

• Whenρ=0,Σ_θ= ∆⁻¹_p (see e.g. Brockwell and Davis, 1991).

On the autoregressive parameter

Rates of convergence

• Quadratic strong law.

Theorem

Assume that(Vn)has a finite moment of order 4. Then, we have the quadratic strong law

n→∞lim 1 logn

n

X

k=1

θbk−θ^∗ θbk−θ^∗0

= Σθ a.s.

• Law of iterated logarithm.

Theorem

Assume that(Vn)has a finite moment of order 4. Then, we have the law of iterated logarithm

lim sup

n→∞

n 2 log logn

θbn−θ^∗ bθn−θ^∗ 0

= Σθ a.s.

Plan

1 Introduction

Framework of the study Hypothesis

Stability conditions

2 On the autoregressive parameter Estimation and convergence Central limit theorem Rates of convergence

3 On the serial correlation parameter Estimation and convergence Central limit theorem Rates of convergence

4 Testing the residual autocorrelation of the AR(p)process The Durbin-Watson statistic

Test statistic

Empirical power and comparisons

5 Conclusion

On the serial correlation parameter

Estimation and convergence

• Estimated residual set.

• For all 1≤k≤n, let

εb_k=Xk−θb_n⁰Φ^p_k−1.

• Least squares estimator.

• In the AR(1)structureεn=ρεn−1+Vn, for alln≥1,

bρ_n= Pn

k=1εb_kbε_k−1 Pn

k=1bε_k−1² .

• Convergence.

Theorem

We have the almost sure convergence

n→∞lim ρbn=ρ^∗ a.s.

• The limiting value is given by

ρ^∗=θ_pρθ^∗_p whereθ_p^∗is thep−th element ofθ^∗.

• Whenρ=0,ρb_nconverges a.s. to 0.

On the serial correlation parameter

Central limit theorem

• Central limit theorem.

Theorem

Assume that(Vn)has a finite moment of order 4. Then, we have the joint asymptotic normality

√n

θbn−θ^∗ ρbn−ρ^∗

L

n→∞−→ N(0,Γ).

In particular,

√n

ρbn−ρ^∗ _L

n→∞−→ N(0, σ²_ρ) whereσ²_ρ= Γp+1,p+1is the last diagonal element ofΓ.

• The asymptotic covariance matrix is given by Γ =

Σ_θ θ_pρJpΣ_θe θpρe⁰ΣθJp σ²_ρ

where

σ_ρ²=P_L⁰∆pPL−2α⁻¹θ^∗_pΛ¹_p⁰JpPL+ α⁻¹θ^∗_p2

λ₀.

• Whenρ=0,σ²_ρ=θ²_p.

On the serial correlation parameter

Rates of convergence

• Quadratic strong law.

Theorem

Assume that(Vn)has a finite moment of order 4. Then, we have the quadratic strong law

n→∞lim 1 logn

n

X

k=1

ρbk−ρ^∗2

=σ²_ρ a.s.

• Law of iterated logarithm.

Theorem

Assume that(Vn)has a finite moment of order 4. Then, we have the law of iterated logarithm

lim sup

n→∞

n 2 log logn

ρbn−ρ^∗ 2

=σρ a.s.

Plan

1 Introduction

Framework of the study Hypothesis

Stability conditions

2 On the autoregressive parameter Estimation and convergence Central limit theorem Rates of convergence

3 On the serial correlation parameter Estimation and convergence Central limit theorem Rates of convergence

4 Testing the residual autocorrelation of the AR(p)process The Durbin-Watson statistic

Test statistic

Empirical power and comparisons

5 Conclusion

Testing the residual autocorrelation of the AR(p) process

The Durbin-Watson statistic

• Introduced by Durbin and Watson.

• Middle of last century.

• Testing residual autocorrelation in standard regression analysis.

• Biased in the dependent framework.

• For alln≥1,

bDn= Pn

k=1(bεk−bεk−1)² Pn

k=0bε_k² .

• Properties.

• Asymptotic equivalence

bDn ∼

+∞2(1−ρb_n) a.s.

• Convergence

n→∞lim Dbn=2(1−ρ^∗) =D^∗ a.s.

• Central limit theorem

√n

Dbn−D^∗ _L

n→∞−→ N(0,4σ²_ρ).

• Rate of convergence

bDn−D^∗2

=O

log logn n

a.s.

Testing the residual autocorrelation of the AR(p) process

Test statistic

• Test statistic.

• For alln≥1,

bZn= n 4bθ_p,n²

Dbn−22

wherebθ_p,n² is thep−th element ofbθn.

Theorem

Assume that(Vn)has a finite moment of order 4,θp6=0andθ^∗_p6=0. Then, under the null hypothesisH₀: “ρ=0”,

Zbn L n→∞−→ χ²

whereχ²has a Chi-square distribution with one degree of freedom. In addition, under the alternative hypothesisH₁: “ρ6=0”,

n→∞lim Zbn= +∞ a.s.

• Assumption{θ^∗_p 6=0}not restrictive :{θp6=0} ∩ {θ^∗_p =0} ⇒ {ρ6=0}.

• H₀not rejected ifZbn≤za, for a risk 0<a<1.

Testing the residual autocorrelation of the AR(p) process

Empirical power and comparisons

• Empirical power on simulated data.

• (Vn)i.i.d.N(0,1),p=3,θ= 0.1 −0.2 0.60

,n=500.

• On large samples.

• More powerful than theportmanteautests of Box-Pierce and Ljung-Box.

• Wrong variance in the asymptotic normality : theportmanteautests overestimateH0.

• Equivalent to theh-testof Durbin and to the Breusch-Godfrey test.

Testing the residual autocorrelation of the AR(p) process

Empirical power and comparisons

• Empirical power on simulated data.

• (Vn)i.i.d.N(0,1),p=3,θ= 0.1 −0.2 0.60

,n=30.

• On small-sized samples.

• More powerful than all other tests, except underH0.

• Always more sensitive to the presence of correlation in the residuals.

• Performs pretty well.

Plan

1 Introduction

Framework of the study Hypothesis

Stability conditions

2 On the autoregressive parameter Estimation and convergence Central limit theorem Rates of convergence

3 On the serial correlation parameter Estimation and convergence Central limit theorem Rates of convergence

4 Testing the residual autocorrelation of the AR(p)process The Durbin-Watson statistic

Test statistic

Empirical power and comparisons

5 Conclusion

Conclusion

Thank you for your attention !

• References.

• Bercu, B., and Proïa, F.A sharp analysis on the asymptotic behavior of the

Durbin-Watson statistic for the first-order autoregressive process.ESAIM Probab. Stat.

16 (2012).

• Bitseki Penda, V., Djellout, H., and Proïa, F.Moderate deviations for the Durbin-Watson statistic related to the first-order autoregressive process.arXiv 1201.3579. Submitted.

(2012).

• Durbin, J., and Watson, G. S.Testing for serial correlation in least squares regression.

I-II-III.Biometrika 37-38-57 (1950-51-71).

• Malinvaud, E.Estimation et prévision dans les modèles économiques autorégressifs.

Review of the International Institute of Statistics 29 (1961).

• Nerlove, M., and Wallis, K. F.Use of the Durbin-Watson statistic in inappropriate situations.Econometrica 34 (1966).

• Park, S. B.On the small-sample power of Durbin’s h test.Journal of the American Statistical Association 70 (1975).