Estimation and validation of weak FARIMA models

(1)

Estimation and validation of weak FARIMA models

Youssef ESSTAFA

University of Burgundy - Franche-Comté Joint work: Yacouba BOUBACAR MAINASSARA

Bruno SAUSSEREAU

25 June 2018

(2)

Introduction

Some definitions

Long memory processes.

Weak FARIMA models.

Asymptotic results of the least-squares estimator (LSE) Strong consistency and asymptotic normality.

Asymptotic variance matrix estimation and some simulations.

Diagnostic checking in weak FARIMA models

Asymptotic joint distribution of the LSE and the noise empirical autocovariances.

Asymptotic distribution of the residual autocorrelations.

Limiting distribution of the test statistics.

Numerical illustrations.

(3)

Defining long memory

LetX= (Xt)t∈Z be a second order stationary stochastic process, and letγ_X(.) be its autocovariance function.

Definition A

X is a long memory process if :

+∞

X

h=−∞

|γX(h)|= +∞.

Definition B

X is a long memory process if :

γX(h)∼h^2d−1`(h), ash→+∞,

wheredis the so-called long-memory parameter and`(.)is a slowly varying function.

3 Youssef ESSTAFA 5th Probability/Statistics day Besançon-Dijon

(4)

FARIMA processes

LetX= (X_t)t∈Z be a second order stationary process.

Definition 1

X is called a weak FARIMA(p, d0, q)process if there exists0< d0<1/2, a1, ..., ap,b1, ..., bq ∈Rsuch that the polynomialsa(z) = 1 +Pp

i=1aizⁱ andb(z) = 1 +Pq

i=1b_izⁱ have all their roots outside of the unit disk with no common factors, and(_t)_t∈_Za sequence of uncorrelated variables defined on some probability space(Ω,F,P)with zero mean and common varianceσ² >0such that, for allt∈Z,

a(L)(1−L)^d⁰Xt=b(L)t, (1) whereLis the back-shift operator.

The fractional difference operator(1−L)^d⁰ is given by : (1−L)^d⁰=

+∞

X

j=0

α_j(d₀)L^j, where α_j(d₀) =d0(d0−1)· · ·(d0−j+ 1)

j! (−1)^j.

(5)

Least-squares estimator (LSE)

Framework :Let Θ˜ be the compact space

Θ :={˜ θ˜= (θ1, θ2, . . . , θp+q)⁰; aθ˜(z) = 1 +θ1z+· · ·+θpz^p andbθ˜(z) = 1 +θp+1z+· · ·+θp+qz^q have all their roots outside the unit disk and have no common zero}.

Denote byΘthe cartesian productΘ˜ ×[d1, d2], where [d₁, d₂]⊂]0,1/2[and containing d₀.

The parameterθ0 := (a1, a2, . . . , ap, b1, b2, . . . , bq, d0)⁰ belongs to the parameter spaceΘ.

For allθ= θ˜⁰, d

⁰

∈Θ, let(t(θ))_t∈

Z be the second order stationary process defined as the solution of

_t(θ) =X

j≥0

α_j(d)X_t−j+

p

X

i=1

θ_iX

j≥0

α_j(d)X_t−i−j−

q

X

j=1

θ_p+j_t−j(θ).

(6)

Least-squares estimator (LSE)

Given a realization of lengthn,X₁, X₂, ..., X_n,_t(θ) can be approximated, for0< t≤n, by ˜t(θ) defined recursively by

˜ _t(θ) =

t−1

X

j=0

α_j(d)Xt−j+

p

X

i=1

θ_i

t−i−1

X

j=0

α_j(d)Xt−i−j−

q

X

j=1

θ_p+j˜t−j(θ),

with˜t(θ) =Xt= 0 ift≤0.

The random variableθb_nis called least-squares estimator if it satisfies, almost surely,

θbn= argmin

θ∈Θ

Qn(θ), where Qn(θ) = 1 n

n

X

t=1

˜ ²_t(θ).

(7)

Strong consistency

Our first two main results concern the strong consistency and the asymptotic normality of the least-squares estimator (LSE) of the weak FARIMA model parameter

θ₀ = (a₁, . . . , a_p, b₁, . . . , b_q, d₀)⁰.

The strong consistency of the LSE is proven under the following assumption :

A1.The process(t)t∈Z is strictly stationary and ergodic.

Theorem I (strong consistency)

Assume that(t)t∈Z satisfies (1) and belonging toL². Let(θbn)n∈N^∗

be a sequence of least-squares estimators. We have, under AssumptionA1,

θbn

−→a.s.

n→+∞θ0.

(8)

Asymptotic normality

Suppose that(t)t∈Z satisfies the following two additional conditions :

A2.We have E|_t|^4+2ν <∞ andP∞

h=0{α(h)}^2+ν^ν <∞ for some ν >0.

A3.AssumeP

i,j,k∈Z|cum(₀, i, j, k)|<∞ . Theorem II (asymptotic normality)

Under the hypotheses of Theorem I and AssumptionsA2andA3, the sequence of random variables

√ n

bθn−θ0

n∈N^∗

has a limiting centred normal distribution with covariance matrix Ω :=J⁻¹IJ⁻¹, where

I= lim

n→∞Var √

n∂

∂θQ_n(θ₀)

andJ = lim

n→∞

∂²

∂θ∂θ⁰Q_n(θ₀)

a.s.

(9)

Asymptotic variance matrix estimation

It would be necessary to estimate the variance matrix

Ω =J⁻¹IJ⁻¹ to obtain confidence intervals or to test significance of FARIMA model coefficients.

The matrix J can easily be estimated empirically by : Jb= 2

n

X

t=1

∂˜_t(θ)

∂θ

∂˜_t(θ)

∂θ⁰

θ=bθn

.

The matrix I can be rewritten in the form : I= lim

n→∞V ar ( 1

√n

n

X

t=1

Υ_t )

, where Υ_t= 2

_t(θ)∂_t(θ)

∂θ

θ=θ₀

.

We use the parametric estimation of the spectral density introduced by Berk[1974]. LetΦb_r(z) =I_p+q+1+Pr

i=1Φb_r,izⁱ, where Φbr,1, ...,Φbr,r be the least-squares regression coefficients ofΥbt on Υbt−1, ...,Υbt−r andΣb

ubr be the empirical variance of these residues.

(10)

Asymptotic result

The third main result is given by :

Theorem III (estimating the asymptotic variance matrix I) In addition to the assumptions of Theorem II, assume that the process(Υ_t)_tadmits an AR(∞) representation of the form Φ(L)Υt:= Υt−P∞

k=1ΦkΥt−k =ut in which the roots of det(Φ(z)) = 0are outside the unit disk,kΦ_kk= o(k⁻²), and Σ_u = Var(u_t) is non-singular. Moreover we assume that E

h

|_t|⁸i

<∞and that P+∞

k1=−∞· · ·P+∞

k7=−∞|cum(₀, k1, ..., k7)|<∞. Then, the spectral estimator ofI

Iˆ_n^SP:= ˆΦ⁻¹_r (1) ˆΣurΦˆ⁰_r⁻¹(1)→I

in probability whenr =r(n)→ ∞ andr³/n→0 asn→ ∞.

(11)

Some simulations

We first study numerically the behavior of the LSE for strong and weak FARIMA models of the form

(1−L)^d(Xt+aXt−1) =t+bt−1, (2) where(a, b, d) = (0.7,0.2,0.4).

The process (_t)_t is an iid centered Gaussian process with common variance 1 in the strong case.

In the weak case,

_t= η_t

|ηt−1|+ 1, for allt∈Z, (3) with (ηt)t is an iid centered Gaussian process with variance 1.

We simulatedN = 1,000independent trajectories of size n= 1,000of Model (2), first with the strong Gaussian noise, second with the weak noise (3).

(12)

(a) (b) (c)

−0.2−0.10.00.10.2

Estimation errors for n=1000 Strong FARIMA

(a) (b) (c)

−0.2−0.10.00.10.2

Estimation errors for n=1000 Weak FARIMA

(13)

−3 −1 1 3

−0.15−0.050.05

Normal quantiles a0−a^0

Strong case: Normal Q−Q Plot of

−3 −1 1 3

−0.20.00.10.2

Normal quantiles b0−b^ 0

−3 −1 1 3

−0.100.000.050.10

Normal quantiles d0−d^ 0

−3 −1 1 3

−0.15−0.050.05

Normal quantiles a0−a^0

Weak case: Normal Q−Q Plot of

−3 −1 1 3

−0.20.00.10.2

Normal quantiles b0−b^ 0

−3 −1 1 3

−0.100.000.050.10

Normal quantiles d0−d^ 0

(14)

Strong case

Distribution of a0−a^₀

Density

−0.15 0.00 0.10

02468

Strong case

Distribution of b0−b^

0

Density

−0.3 −0.1 0.1

012345

Strong case

Distribution of d0−d^

0

Density

−0.10 0.00 0.10

0246810

Weak case

Distribution of a0−a^₀

Density

−0.15 0.00 0.10

02468

Weak case

Distribution of b0−b^

0

Density

−0.2 0.0 0.2

012345

Weak case

Distribution of d0−d^

0

Density

−0.10 0.00 0.10

0246810

(15)

(a) (b) (c)

2468

Estimates of diag(Ω) Strong FARIMA: estimator of 2J⁻¹

(a) (b) (c)

2468

Estimates of diag(Ω) Strong FARIMA: estimator of J⁻¹IJ⁻¹

(a) (b) (c)

051015

Estimates of diag(Ω) Weak FARIMA: estimator of 2J⁻¹

(a) (b) (c)

051015

Estimates of diag(Ω) Weak FARIMA: estimator of J⁻¹IJ⁻¹

(16)

Let, fort≥1,ˆe_t= ˜_t(ˆθ_n) be the least-squares residuals. Using the expression of˜_t(.) we have ˆe_t= 0 for t≤0 andt > n. By (1), it holds that

ˆ e_t=

t−1

X

j=0

α_j( ˆd_n) ˆXt−j+

p

X

i=1

θˆ_n,i

t−i−1

X

j=0

α_j( ˆd_n) ˆXt−i−j−

p+q

X

j=p+1

θˆ_n,jˆet−j,

fort= 1, ..., n, with Xˆt= 0 fort≤0 andXˆt=Xt fort≥1.

Let, forh≥0,

γ(h) = 1 n

n

X

t=h+1

_tt−h and ρ(h) = γ(h) γ(0) denote the white noise "empirical" autocovariances and autocorrelations.

The residual autocovariances and autocorrelations are defined by ˆ

γ(h) = 1 n

n

X

t=h+1

ˆ

e_tˆet−h and ρ(h) =ˆ γ(h)ˆ ˆ γ(0).

(17)

For a fixed integerm≥1, let

γ_m= (γ(1), ..., γ(m))⁰ and γˆ_m= (ˆγ(1), ...,ˆγ(m))⁰. Denote also by

ˆ

ρm = ( ˆρ(1), ...,ρ(m))ˆ ⁰ the firstm sample autocorrelations.

Based on the residual empirical autocorrelationρ(h), Box-Pierceˆ and Ljung-Box have proposed the following respective statistics for the validation of strong univariate ARMA models :

Q_m^BP =n

m

X

h=1

ˆ

ρ²(h) and Q_m^LB =n(n+ 2)

m

X

h=1

ˆ ρ²(h) n−h.

Test hypotheses

(H0) : (X_t)t∈Z satisfies a FARIMA(p, d₀, q) representation ; against the alternative

(H1) : (Xt)t∈Z does not admit a FARIMA representation or admits a FARIMA(p⁰, d₀, q⁰) representation withp⁰ > por q⁰ > q.

(18)

Joint distribution of θ ˆ

_n

and the noise empirical autocovariances

Under the assumptions of Theorem II, the random vector

√n

θˆ_n−θ₀⁰ , γ_m⁰

⁰

has a limiting centred normal distribution with covariance matrixΞ, where

Ξ =





Σ_θ_ˆ Σ_θ,γ_ˆ

m

Σ⁰_ˆ

θ,γm Σ_γ_m



=

∞

X

h=−∞

E h

g_tg_t−h⁰ i ,

with

gt= g_1t g_2t

!

=

−2J⁻¹_t_∂θ^∂ _t(θ₀) (t−1, . . . , t−m)⁰t

! .

(19)

Asymptotic distribution of the residual autocorrelations

LetΨ_m be them×(p+q+ 1)matrix defined by Ψ_m=E

_t−1, . . . , _t−m⁰ ∂_t(θ₀)

∂θ⁰

.

The following proposition provides the limit distribution of the residual autocovariances and autocorrelations of weak FARIMA models.

Proposition

Under the assumptions of Theorem II, we have

√nˆγ_m −→^D

n→+∞N(0,Σ_ˆ_γ_m) and √

nˆρ_m −→^D

n→+∞N(0,Σ_ρ_ˆ_m), where

Σ_γ_ˆ_m= Σ_γ_m+ Ψ_mΣ_θ_ˆΨ⁰_m+ Ψ_mΣ_θ,γ_ˆ

m+ Σ⁰_ˆ

θ,γmΨ⁰_m and Σ_ρ_ˆ_m = 1 σ⁴Σ_ˆ_γ_m.

(20)

→The matrices Σ_ˆ_γ_m andΣ_ρ_ˆ_m depend on the unknown matricesΞ, Ψ_m and the scalarσ.

→The matrix Ψm and the noise varianceσ² can be estimated by its empirical counterpart :

Ψˆ_m= 1 n

n

X

t=1

(ˆe_t−1, ...,eˆ_t−m)⁰ ∂ˆe_t

∂θ⁰

and σˆ² = 1 n

n

X

t=1

ˆ e²_t.

→ By interpreting (2π)⁻¹Ξ as the spectral density of the stationary process(gt)tevaluated at frequency 0, we use Berk’s approach to estimate the spectral density of(gt)t by fitting a parametric autoregressive model. This estimation technique is based on the following expression

Ξ = ∆⁻¹(1)Σv∆⁰⁻¹(1)

when(g_t)_t satisfies anAR(∞) representation of the form

∆(L)g_t:=g_t−X

k≥1

∆_kg_t−k =v_t, (4)

where (v_t)_t is a(p+q+ 1 +m)-variate weak white noise with variance matrixΣv.

(21)

Letˆg_tbe the vector obtained by replacing _tby eˆ_t ing_t. Let

∆ˆ_r(z) =I_p+q+1+m−Pr

k=1∆ˆ_r,kz^k, where∆ˆ_r,1, ...,∆ˆ_r,r denote the coefficients of the least squares regression ofˆgton gˆt−1, ...,ˆgt−r. Letˆv_r,tbe the residuals of this regression, and letΣˆ_v_ˆ_r be the empirical variance ofvˆ_r,1, ...,vˆ_r,n.

Theorem IV (estimating the asymptotic variance matrix Ξ) In addition to the assumptions of Theorem II, assume that the process(gt)tadmits the AR(∞) representation (4) in which the roots ofdet(∆(z)) = 0are outside the unit disk,k∆_kk= o(k⁻²), andΣv = Var(vt)is non-singular. Moreover we assume that E

h|_t|⁸i

<∞and that P+∞

k1=−∞· · ·P+∞

k7=−∞|cum(₀, _k₁, ..., _k₇)|<∞. Then, the spectral estimator ofΞ

Ξˆ^SP_n := ˆ∆⁻¹_r (1) ˆΣ_v_ˆ_r∆ˆ⁰_r⁻¹(1)→Ξ

in probability whenr =r(n)→ ∞ andr³/n→0 asn→ ∞.

(22)

The exact limiting distribution of Box-Pierce and Ljung-Box statistics is given in the following theorem :

Theorem V (exact asymptotic distribution of the standard portmanteau statistics)

Under the assumptions of Theorem II and(H0), the statistics Q_m^BP andQ_m^LB converge in distribution, asn→ ∞, to

Zm(ξm) =

m

X

k=1

ξk,mZ_k²,

whereξ_m= (ξ_1,m, ..., ξ_m,m)⁰ is the vector of the eigenvalues of the matrixΣρˆm=σ⁻⁴Σγˆm andZ1, ..., Zm are independent and identically distributed (i.i.d.) random variables of the same distributionN(0,1).

(23)

→Let Σˆ_ρ_ˆ_m be the matrix obtained by replacingΞby Ξˆ andσ² by ˆ

σ² in Σρˆm.

→Denote by ξˆ_m = ( ˆξ_1,m, ...,ξˆ_m,m)⁰ the vector of the eigenvalues ofΣˆ_ρ_ˆ_m.

→At the asymptotic level α, the LBtest (respectively theBP test) consists in rejecting the null hypothesis of the weak FARIMA(p, d₀, q) model (the adequacy of the weak FARIMA(p, d₀, q) model) when

Q_m^LB > Sm(1−α) (resp. Q_m^BP > Sm(1−α)), whereS_m(1−α) is such that P(Z_m( ˆξ_m)> S_m(1−α)) =α.

→The proposed modified versions of theBP andLBstatistics are more difficult to implement because their critical values have to be computed from the data.

(24)

Numerical illustrations

Table –Empirical size (in %) of the modified and standard versions of the LB and BP tests in the case of FARIMA(1, d,1)model with

independent noise. The nominal asymptotic level of the tests isα= 5%.

The number of replications isN = 1,000. (ar = 0.9andma = 0.2)

d0 Lengthn Lagm LBw BPw LBs BPs

1 7.4 7.4 n.a. n.a.

2 5.5 5.5 n.a. n.a.

0.20 n= 1,000 3 4.7 4.7 n.a. n.a.

6 4.4 4.3 6.5 6.4 12 4.6 4.6 5.1 5.1 15 5.0 4.7 5.7 5.2 1 5.6 5.6 n.a. n.a.

2 5.1 5.2 n.a. n.a.

0.20 n= 5,000 3 5.2 5.2 n.a. n.a.

6 4.9 4.9 6.6 6.6 12 5.0 5.0 5.7 5.6 15 5.6 5.5 5.9 5.8

(25)

Table –Empirical size (in %) of the modified and standard versions of the LB and BP tests in the case of FARIMA(1, d,1)with an ARCH(1) noise (α1= 0.45). The nominal asymptotic level of the tests isα= 5%.

The number of replications isN = 1,000. (ar = 0.9andma = 0.2)

d0 Lengthn Lagm LBw BPw LBs BPs

1 6.4 6.4 n.a. n.a.

2 5.2 5.2 n.a. n.a.

0.20 n= 1,000 3 3.9 3.9 n.a. n.a.

6 3.8 3.7 10.0 9.8 12 1.9 1.8 7.3 6.7 15 1.0 0.9 6.4 6.3 1 6.3 6.3 n.a. n.a.

2 4.3 4.3 n.a. n.a.

0.20 n= 5,000 3 5.5 5.4 n.a. n.a.

6 3.8 3.8 11.6 11.6 12 2.3 2.3 8.3 8.2 15 1.5 1.5 8.5 8.5

(26)

References

[1] T. W. Anderson. The statistical analysis of time series .John Wiley & Sons, Inc., New York-London-Sydney, 1971.

[2] Kenneth N. Berk. Consistent autoregressive spectral estimates.

Ann. Statist., 2 :489-502, 1974. Collection of articles dedicated to Jerzy Neyman on his 80th birthday.

[3] G. E. P. Box and David A. Pierce. Distribution of residual autocorrelations in autoregressive-integrated moving average time series models.J. Amer. Statist. Assoc. , 65 :1509-1526, 1970.

[4] Christian Francq and Jean-Michel Zakoïan. Estimating linear representations of nonlinear processes.J. Statist. Plann. Inference , 68(1) :145-165, 1998

[5] Norbert Herrndorf. A functional central limit theorem for weakly dependent sequences of random variables.Ann. Probab. ,

12(1) :141-153, 1984.

(27)

ISome perspectives :

Estimation and validation of AR models with fractional noise.

Generalization to fractional ARMA models.

Comparison with weak FARIMA models.

(28)

Estimation and validation of weak FARIMA models