ON THREE CLASSES OF TIME SERIES INVOLVING EXPONENTIAL DISTRIBUTION

(1)

INVOLVING EXPONENTIAL DISTRIBUTION

GEORGIANA POPOVICI

The paper discusses some stochastic properties and parameter estimation for three types of stationary AR(1) processes involving the Exponential distribution in three different ways: Exponential innovations, Exponential stationary distribution, Exponential conditional distribution. These three approaches lead to time series which are not equivalent, like it was the case for AR(1) processes involving the Gaussian distribution. Parameter estimation is performed by means of the Conditional Least Squares method and/or the Conditional Maximum Likelihood method. The asymptotic behaviour of the estimation is discussed.

AMS 2000 Subject Classification: 62M10.

Key words: autoregressive processes, exponential distribution, parameter estimation.

1. INTRODUCTION

Time series data occur in a variety of disciplines including engineering, finance, sociology and economics among others. One of the most studied time series is the stationary, Gaussian AR(1) process which is a Markov process.

Among its remarkable properties we underline the fact that it can be defined/generated in three equivalent ways: starting from Gaussian innovations, or from a Gaussian stationary distribution, or from a Gaussian transition conditional distribution. This property was the starting point of our study, as we focused on three possible approaches of an AR(1) process involving Exponen- tial distributions.

The ARMA time series with Exponential distributions have been discussed by several authors, such as Li and McLeod, Gaver and Lewis, Grun- wald et all and they have been systematically analyzed in Popovici (PhD Thesis) [12]. The paper discusses the properties and the parameter estimation for three types of stationary AR(1) processes involving Exponential distributions: EIAR processes (with Exponential innovations), EAR processes (with Exponential stationary distribution) and ECLAR processes (with Exponential transition distribution). Unlike the Gaussian case, the corresponding AR(1) time series {X_t, t}are no longer equivalent.

MATH. REPORTS12(62),1 (2010), 45–57

(2)

The Conditional Least Squares (CLSE) and the Conditional Maximum Likelihood (CMLE) methods are used in order to estimate the parameters of these processes. The asymptotic behaviour of the estimators is obtained on using the general results which characterize the two methods (CLSE, CMLE).

2. DEFINITIONS AND PROPERTIES

An AR(1) time series{X_t, t∈Z}is defined through the linear equation Xt = φ1Xt−1 +εt, where {ε_t, t∈Z} is a sequence of independent, identical distributed random variables called innovations.

Definition 1. A stationary AR(1) process X_t=φ₁Xt−1+ε_t

with φ1 ∈ (0,1) and {ε_t}_t a sequence of independent, identical distributed random variables it is called EIAR(1)ifε_t have an Exponential distribution, Expo(µ),whereµ∈(0,∞) (E(ε_t) =µand V (εt) =µ²).

Proposition 1.For theEIAR(1)process{X_t, t∈Z}, the mean and the variance of the stationary distribution are

E(X_t) = µ

1−φ₁, V (X_t) = µ² 1−φ²₁ and the characteristic function satisfies the relation

ϕX(t) =ϕX(φ1t)·(1−itµ)⁻¹.

Proof of Proposition 1. From the linear AR equation we have E(Xt) =φ1E(Xt−1) +E(εt).

Since the process {X_t, t∈Z} is stationary, we obtain E(Xt) = µ

1−φ1

. Similarly,

V (X_t) =φ²₁V (Xt−1) +µ². This implies that,

V(X_t)(1−φ²₁) =µ². Hence

V(X_t) = µ² 1−φ²₁, Next,

ϕX(t) =E e^itX^t

=E e^it(φ¹^X^t−1^+ε^t⁾

=E e^itφ¹^X^t−1

·E e^itε^t ,

(3)

ϕ_X(t) =ϕ_X(φ1t)·ϕεt(t), ϕ_X(t) =ϕ_X(φ1t)·(1−itµ)⁻¹. Definition 2. A stationary AR(1) process

X_t=φ₁Xt−1+ε_t

with φ₁ ∈ (0,1) and {ε_t}_t a sequence of independent, identical distributed random variables it is called EAR(1) if the stationary distribution of the process is Expo(µ) (E(X_t) =µand V (X_t) =µ²).

Proposition 2. The characteristic function of the innovations corresponding to an EAR(1) process is

φ_ε_t(t) =

φ₁+ (1−φ₁)· 1 1 +µt

.

Proof of Proposition 2. We have ϕX(t) =

1 1 +µt

. Next,

ϕ_X(φ₁t) =

1 1 +µφ₁t

, φ_ε_t(t) = ϕ_X(t) ϕ_X(φ₁t). This implies that

φ_ε_t(t) =

1 +µφ1t 1 +µt

, φ_ε_t(t) =

φ₁+ (1−φ₁)· 1 1 +µt

.

Proposition3. Let{X_t, t∈Z},{Y_t, t∈Z}be two independentEAR(1) processes, with the same parameters φ1 ∈(0,1), µ∈(0,∞).Then, their sum

Zt=Xt+Yt, t∈Z is a Gamma AR process GAR(1) (2, φ₁, µ).

Proof of Proposition 3. A stochastic process {X_t, t∈Z} is called a Gamma Autoregressive process of order 1 and parameters (p, φ1, µ), denoted GAR(1) (p, φ₁, µ),if it satisfies the equation

Xt=φ1Xt−1+εt

(4)

with φ1 ∈(0,1),

ξ_t=











G0,t with the probability φ^p₁, G_1,t with the probability

p 1

φ^p−1₁ (1−φ₁),

. . . .

G_k,t with the probability p

k

φ^p−k₁ (1−φ1)^k,

. . . .

G_p,t with the probability (1−φ₁)^p,

whereG0is a random variable with the Dirac distributionδ0,and{G_i,t, t∈Z}

i = 1, . . . , p are independent sequences of independent, identical distributed random variables, such that Gi,t has a Gamma distributionGamma(i, µ) for every t.

Let {A_t, t∈Z} be a GAR(1) (p, φ₁, µ) process and {B_t, t∈Z} be a GAR(1) (q, φ1, µ) process with the same parametersφ1 ∈(0,1), µ∈(0,∞). The processes are independent. Then, their sum {C_t, t∈Z}

C_t=A_t+B_t is a GAR(1) (p+q, φ₁, µ) process.

Let

A_t=φ₁At−1+D₁, ϕ_D₁(t) = (φ₁+ (1−φ₁)ϕ_E₁)^p, where E₁ has an Exponential distribution, Expo(µ).

Let

Bt=φ1Bt−1+D2, ϕD2(t) = (φ1+ (1−φ1)ϕE2)^q,

where E2 has an Exponential distribution, Expo(µ). E1, E2, D1 and D2 are independent

C_t=φ₁Ct−1+D₁+D₂,

ϕ_D₁_+D₂(t) =ϕ_D₁(t)ϕ_D₂(t) = (φ₁+ (1−φ₁)ϕ_E)^p+q, where E has an Exponential distribution, Expo(µ).

It implies that{C_t, t∈Z} is a GAR(1) (p+q, φ1, µ) process.

Ifp= 1, we deal with an EAR(1) process.

Notice that the EAR(1) process with the parameters φ₁ and µ can be written as

X_t=φ₁Xt−1+ (1−I₁)E₁, Y_t=φ₁Yt−1+ (1−I₂)E₂,

where I₁ and I₂ are random variables with Bernoulli distribution B(1, φ₁), E1 and E2 are random variables with Exponential distribution Expo(µ), and

(5)

I1, I2, E1, E2 are independent. Then,

Zt=Xt+Yt=φ1(Xt−1+Yt−1) + (1−I1)E1+ (1−I2)E2. We denote by

B = (1−I1)E1+ (1−I2)E2. Then,

Zt=φ1Zt−1+B.

Let

A=







0 with the probability φ²₁,

Eft with the probability 2φ1(1−φ1), Gft with the probability (1−φ1)²,

where fEt has an Exponential distribution, Expo (µ), fGt has a Gamma distribution Gamma(2, µ) andEft ,Gft are independent.

ϕ_A(t) =E e^itA

=φ²₁eît0+ 2φ₁(1−φ₁)E eîtÊ^˜

+ (1−φ₁)²E e^it^G^˜

=

=φ²₁+ 2φ1(1−φ1)ϕE˜(t) + (1−φ1)²ϕ²_E_˜(t) = φ1+ (1−φ1)ϕE˜

2

. On the other hand, we have

ϕ_B(t) =ϕ_(1−I₁_)E₁_+(1−I₂_)E₂(t) =Eh

e^it[(1−I¹^)E¹^+(1−I²^)E²^]i

=

= Z

R⁴

e^it[(1−i¹^)e¹⁺⁽¹⁻ⁱ²^)e²^]dP◦(I₁, I₂, E₁, E₂)⁻¹ =

= Z

R⁴

e^it[(1−i¹^)e¹⁺⁽¹⁻ⁱ²^)e²^]dP ◦I₁⁻¹dP ◦I₂⁻¹dP◦E₁⁻¹dP ◦E₂⁻¹ =

= Z

R²

e^ite¹⁽¹⁻ⁱ¹⁾dP ◦I₁⁻¹dP ◦E₁⁻¹ Z

R²

e^ite²⁽¹⁻ⁱ²⁾dP◦I₂⁻¹dP ◦E₂⁻¹=

= Z

R²

e^ite¹⁽¹⁻ⁱ¹⁾dP◦I₁⁻¹dP◦E₁⁻¹ 2

=

= Z

R

Z

R

e^ite¹⁽¹⁻ⁱ¹⁾dP◦I₁⁻¹

dP ◦E₁⁻¹ 2

=

= Z

R

φ1e^it0+ (1−φ1) e^ite¹

dP◦E₁⁻¹ 2

=

φ₁+ (1−φ₁) Z

e^itEdP 2

= (φ₁+ (1−φ₁)ϕ_E)². So, ϕ_A(t) =ϕ_B(t).

Definition 3. A stationary AR(1) process Xt=φ1Xt−1+εt

(6)

with φ1 ∈ (0,1) and {ε_t}_t a sequence of independent, identical distributed random variables it is calledECLAR(1)if the transition distribution is Expo, F(· |Xt−1=x) = (β+φ1x).

Proposition 4. For the ECLAR(1) process {X_t, t∈Z} the mean and the variance of the stationary distribution are

E(Xt) = β 1−φ1

denoted

= µ, V (Xt) = µ² 1−2φ²₁. Proof of Proposition 4. From the linear AR equation we have

E(Xt) =φ1E(Xt−1) +β.

Hence

E(Xt) = β 1−φ1

. Similarly,

E X_t²

=E (β+φ1Xt−1)²+ β+φ1Xt−1

2

= 2E(β+φ1Xt−1)². It implies that

1−2φ²₁

E X_t²

= 2

β²+ 2βφ₁ β 1−φ₁

,

E X_t²

= 2β²

1+φ1

1−φ1

1−2φ²₁ , V (Xt) = 2β²

1+φ1

1−φ1

1−2φ²₁ − β² (1−φ₁)². After calculation we have

V (Xt) = β²

(1−φ₁)² 1−2φ²₁. We denote

β 1−φ1

=µ.

Then

V (X_t) = µ² 1−2φ²₁.

Proposition 5. For the ECLAR(1) process, the mean and the variance of the innovations are

E(εt) =β, V (εt) = µ² 1−φ²₁ 1−2φ²₁ . Proof of Proposition 5.

E(εt) =E(Xt)−φ1E(Xt−1).

(7)

It implies that

E(εt) = (1−φ1) β 1−φ1

=β

V (X_t) =φ²₁V (Xt−1) +V (ε_t), V (X_t) = 1−φ²₁ µ² 1−2φ²₁.

A simulation study was performed in order to establish the behaviour of these three processes near the border of the stationarity domain (Popovici, 2008) [13].

3. PARAMETER ESTIMATION

Parameter estimation has been performed by means of the CLS method and/or the CML method. The general properties of these two methods are presented in Klimko and Nelson [8] and in Basawa and Prakasa-Rao [1].

EIAR(1) process

The parameters of an EIAR(1) process can be estimated by the CLS method, using the linear form of the conditional expectation. Also, one can take advantage of the simplicity of the innovations and apply the ML method, as suggested by Li and McLeod.

• CLSE for the process EIAR(1)

a) Construction of the estimator. We use the Conditional Least Squares method for estimating ψ= (φ₁, µ),starting from the mean

E(Xt|Xt−1 =xt−1) =φ1xt−1+µ.

The associated sum of squares is

Qn(ψ) =

n

X

t=2

[Xt−(φ1xt−1+µ)]²,

∂Q_n(ψ)/∂φ₁=

n

X

t=2

2 [X_t−φ₁Xt−1−µ] [−X_t−1] = 0.

Then,

(3.1) −

n

X

t=2

X_tXt−1+φ₁

n

X

t=2

X_t−1² +µ

n

X

t=2

Xt−1 = 0.

(8)

On the other hand, we have

∂Qn(ψ)/∂µ=

n

X

t=2

2 [Xt−φ1Xt−1−µ] (−1) = 0,

−

n

X

t=2

X_t+φ₁

n

X

t=2

Xt−1+µ(n−1) = 0,

(3.2) bµ=

n

P

t=2

Xt−cφ1 n

P

t=2

Xt−1

n−1 .

We substitute (2) in (1) and we obtain

cφ₁=

(n−1)

n

P

t=2

XtXt−1−

n

P

t=2

Xt−1 n

P

t=2

Xt

(n−1)

n

P

t=2

X_t−1² − _n

P

t=2

Xt−1

2 .

Proposition 6. For the process EIAR(1) cφ₁ and µb are asymptotically independent and normally distributed

√n

cφ₁−φ₁

∼N 0,(1−φ₁)²·(1 +φ₁) 2

!

and √

n(µb−µ)∼N 0, µ² .

Proof of Proposition 6. The model fulfils the regularity conditions from Klimko and Nelson [8]. Thus, the asymptotic distribution of

√

n cφ₁−φ₁

√n µb−µ

is Gaussian, with mean 0. We calculate the covariance matrix:

Let

g(ψ) =φ₁Xt−1+µ, ∂g

∂φ₁ =Xt−1, ∂g

∂µ = 1, ∂g

∂φ₁∂µ = 0, V₁₁=E(Xt−1)² = µ²

1−φ1

1 1 +φ1

+ 1

1−φ1

, V₂₂=E r²

= 1, V₁₂= 0 =V₂₁, V ar(ε_t) = 1µ². Then,

V =

µ² 1−φ1

h 1

1+φ1 +_1−φ¹

1

i 0

0 1

!

and V⁻¹=

1

µ2 1−φ1

h 1 1+φ1+_1−φ¹

1

i 0

0 1

! .

(9)

It implies that,

C=

(1−φ₁)²·(1+φ₁)

2 0

0 µ²

!

b)Li and McLeod’s method. Let (x1, . . . , xn) be an observed trajectory. The values of the innovations are {x_t+1−φ₁x_t, t= 1, . . . , n−1}.Since the innovations are independent, identical distributed random variables with an Exponential distribution, the likelihood function is

L=

n−1

Y

t=1

1 µe⁻

(_Xt+1^−φ1Xt)

µ .

The MLE is obtained in the traditional way lnL=−(n−1) lnµ− 1

µ

n−1

X

t=1

(x_t+1−φ₁x_t). We differentiate with respect toµ and we get

∂lnL

∂µ =−(n−1) µ + 1

µ²

n−1

X

t=1

(xt+1−φ1xt). We denote

X⁽¹⁾= 1 n−1

n−1

X

t=1

X_t, X⁽²⁾ = 1 n−1

n−1

X

t=1

X_t+1. Then

µb=X⁽²⁾−φ₁X⁽¹⁾. We differentiate with respect toφ1 and we get

g(φ₁) = ∂lnL

∂φ₁ = 1 µ

n−1

X

t=1

X_t= (n−1)X⁽¹⁾ X⁽²⁾−φ1X⁽¹⁾

. We have

g(0) = (n−1)X⁽¹⁾ X⁽²⁾

, g(1) = (n−1)X⁽¹⁾ X⁽²⁾−X⁽¹⁾ . From these formulae above one can get cφ1.

EAR(1) process

We use the CLS method to estimate the parameters of an EAR(1) process.

• CLSE for the process EAR(1)

(10)

a)Construction of the estimator. The conditional sum of squares is Q_n(ψ) =

n

X

t=2

[x_t−(φ₁xt−1+µ(1−φ₁))]².

The CLSE estimators for φ₁ and µ are obtained by minimizing Q_n(ψ) with respect to ψ= (φ1, µ)

cφ1=

n

P

t=2

X_tXt−1−(n−1)⁻¹

n

P

t=2

X_t

n

P

t=2

Xt−1 n

P

t=2

X_t²−(n−1)⁻¹ _n

P

t=2

Xt−1

2

and

bµ=

n

P

t=2

Xt−cφ1 n

P

t=2

Xt−1

(n−1)(1−φc₁) .

b)Proposition 7. For the processEAR(1)cφ1 andµbare asymptotically independent and normally distributed:

√n

cφ₁−φ₁

∼N 0,1−φ²₁ and

√n(µb−µ)∼N

0,µ²(1 +φ₁) (1−φ1)

.

This result follows from the general properties of the CLS method.

ECLAR(1) process

We use the CML method to estimate the parameter of an ECLAR(1) process.

• CMLE

a) Construction of the estimator. The likelihood function corresponding to an observed trajectory x= (x1, . . . , xn) is

L(φ₁, β) =

n

Y

t=2

f(x_t|xt−1;φ₁, β). The log-likelihood is

−

n

X

t=2

ln (φ₁Xt−1+β)−

n

X

t=2

X_t φ₁Xt−1+β. We differentiate with respect toβ and φ1.

(11)

We get

f1(β, φ1) =

n

X

t=2

Xt

(φ₁Xt−1+β)² −

n

X

t=2

1

φ₁Xt−1+β = 0 and

f2(β, φ1) =

n

X

t=2

X_tXt−1

(φ1Xt−1+β)² −

n

X

t=2

Xt−1

φ1Xt−1+β = 0.

We differentiate again to get the JacobianJ J11(β, φ1) =

n

X

t=2

1

(φ₁Xt−1+β)² −2

n

X

t=2

Xt

(φ₁Xt−1+β)³, J₁₂(β, φ₁) =J₂₁(β, φ₁) =

n

X

t=2

Xt−1

(φ1Xt−1+β)² −2

n

X

t=2

X_tXt−1

(φ1Xt−1+β)³, J₂₂(β, φ₁) =

n

X

t=2

X_t−1²

(φ₁Xt−1+β)² −2

n

X

t=2

XtX_t−1² (φ₁Xt−1+β)³.

The values of the estimators are obtained by solving the linear system with matrix J and the constant termf.

b)The simulation study for the estimator. A simulation study has been performed in order to characterize the precision of the estimators.

We generate 1000 trajectories of length n = 1000 for the process ECLAR(1). For each generated trajectory, the estimation cφ₁,βb

has been obtained with the Newton-Raphson’s method.

The initial value φ⁽⁰⁾₁ , β⁽⁰⁾

: φ⁽⁰⁾₁ close to zero andβ⁽⁰⁾ = _n¹

n

P

t=1

Xt and the error equal to 10⁻³.

We considered the following situations

(φ₁ = 0.2, β= 5), (φ₁ = 0.5, β = 5), (φ₁ = 0.7, β= 5) the results for (cφ1,β) are presented in Tables 1–3.b

We use the programC++

Table 1

The statistic properties forφ1= 0.2, β= 5

Min 1st Qu Median Mean 3rd Qu Max std.

cφ1 0.1794 0.1936 0.1986 0.1997 0.2055 0.2217 0.008772516 βb 4.979 4.993 5.000 5.000 5.006 5.027 0.00999688

From Table 1 both estimators cφ1 and βbare stable. cv(b φc1) = 4.4% and cv(b βb) = 0.2%.

(12)

Table 2

φc1 0.4175 0.4840 0.4862 0.4847 0.4923 0 0.5437 0.002014333 βb 4.811 4.971 5.009 5.010 5.062 5.183 0.08310194

From Table 2 we can see thatcφ₁ is more stable thanβ.b cv(cb φ₁) = 0.42%

and cv(b β) = 1.66%.b

Table 3

φc1 0.6987 0.7125 0.7230 0.7377 0.7430 0.8111 0.04412418 βb 4.636 4.894 4.994 4.990 5.077 5.344 0.1319506

From Table 3 we can see that cφ₁ and βbare not stable comparing with the other two cases. cv(cb φ₁) = 6% andcv(b β) = 2.6%.b

REFERENCES

[1] B. Basawa and B. Prakasa-Rao,Statistical Inference for Stochastic Processes.Academic Press, London, 1980.

[2] L. Billard and F.I. Mohamed, Estimation of the parameters of an EAR(p) process.

Journal of Time Series Analysis12(1991), 179–192.

[3] P. Brockwell and R. Davis, Time Series: Theory and Methods. Springer Series in Sta- tistics, 1987.

[4] G.E.P. Box and G.M. Jenkins, Time Series Analysis: Forecasting and Control. Holden Day, San Francisco, 1970.

[5] D.P. Gaver and P.A.W Lewis, First order autoregressive gamma sequences and point processes.Advances in Applied Probability12(1980), 727–745.

[6] G.K. Grunwald, R.J. Hyndman, L. Tedesco and R.L. Tweedie, Non-Gaussian conditional linear AR(1)models.Australian New Zealand Journal of Statistics42(4)(2000), 479–495.

[7] W.K. Kim and I.K. Kim,Estimation for the Exponential ARMA model.Korean Statis- tical Journal9(1994), 239–248.

[8] L.A. Klimko and P.I. Nelson, On conditional least squares estimation for stochastic processes.Annals of Statistics6(1978), 629–64.

[9] A.J. Lawrance and P.A.W Lewis, The exponential autoregressive moving average EARMA(p, q) process.Journal of the Royal Statistical Society. Ser. B42(1980), 150–

161.

[10] A.J. Lawrance,The innovation distribution of a gamma distributed autoregressive process.Scandinavian Journal of Statistics9(1982), 234–236.

[11] W. Li and A. McLeod, ARMA Modelling with Non-Gaussian Innovations.Journal of Time Series9(1988),2, 155–168.

(13)

[12] G. Popovici, Inference statistical topics for time series. PhD Thesis, University of Bucharest, 2008.

[13] G. Popovici,On the Behaviour of the AR processes with Exponential distribution near the stationarity border: A simulation study. Scientific Bulletin, University of Pite¸sti, Nr.

14(2008), 1–12.

[14] C.H. Sim,First-order autoregressive models for gamma and exponential process.Journal of Applied Probability27(1990), 325–332.

Received 20 February 2009 University of Bucharest

Faculty of Mathematics and Computer Science Str. Academiei 14

010014 Bucharest, Romania gpopovici@fmi.unibuc.ro