LINEAR AUTOREGRESSIVE MODELS

ULRICH HERKENRATH, MARIUS IOSIFESCU and ANDREAS RUDOLPH

We study linear autoregressive models (LARM’s for short) in two different vari- ants: one is based on a doubly infinite sequence of i.i.d. innovations thus leading to a time series (Wt, t∈ Z) and the other one is based on a sequence of i.i.d.

innovations and an independent starting variable thus leading to a time series (Wt, t∈N). In both cases the resulting time series obeys the Markov property.

For both variants we give sufficient conditions for the “stabilization” of the process (Wt, t∈Zort∈N).That amounts to a strictly stationary process (Wt, t∈Z) in the first case and to weak respectively geometric ergodicity of the Markov chain (Wt, t∈N) in the second case. Special attention is paid to the case of normally distributed innovations.

AMS 2000 Subject Classification: 60J05, 60G10.

Key words: autoregressive time series, Markov chain, strictly stationary process, stabilization of a time series.

1. MODELS

We consider linear autoregressive models (LARM’s for short). They are characterized by a linear stochastic recursion of the form

(LARM) W_t=F Wt−1+GU_t

with either t∈N+ ort∈Z. Ift∈N+, a random variableW₀ has to be given as starting value.

As for the dimensions of the items in (LARM):W_t∈R^k, U_t∈R^m,F ∈ Mat(k, k),G∈Mat(k, m), where Mat(k, m) denotes the set ofk×mmatrices.

In this paper we generally assume that on a probability space (Ω,A,P) there are given i.i.d. random vectors

(Ut, t∈N+ ort∈Z)

with common distribution τ. The Euclidean space R^p is endowed with its Borel-σ-algebra B^p.In the case (U_t, t∈N+) the random variableW₀ defined on (Ω,A,P) is independent of (Ut, t ∈ N+). Often the Ut’s are called noise variables or innovations. Expectations and covariances refer to the probability measure P, unless otherwise stated.

MATH. REPORTS12(62),3 (2010), 245–259

In this case we consider the (time-homogeneous) Markov chain (MC for short) (W_t, t∈N) with initial distributionp₀, that meansW₀ ∼p₀. A LARM witht∈N+ is called a linear state space model in Meyn and Tweedie (1996).

In the case (Ut, t∈ Z),we consider the (time-homogeneous) MC (Wt, t∈Z) based on the doubly infinite sequence (U_t,t∈Z), i.e., we think that the MC (Wt, t∈Z) has started at time−∞, whereas we regard observation times t ∈ N. Restricted to observation times t∈N, (W_t, t ∈ N) is a MC with “an infinite history” subsumed in an unknown initial distributionp0, i.e.,W0 ∼p0. Under suitable assumptions on the components of a LARM, one may con- sider that the process (W_t, t∈Z) which has started at time −∞ did already

“stabilize” during its infinitely long history in such a way that p0 = π, the unique invariant probability measure (UIPM for short) of the MC induced by the LARM. Then (Wt, t ∈ N) is an ergodic strictly stationary process, the

“stabilized” process generated by a LARM. As general references for termino- logy and results on MC’s we refer to the books by Meyn and Tweedie (1996) and Hern´andez-Lerma and Lasserre (2003).

If (W_t, t∈N) starts according to an arbitrary initial distributionp₀6=π, this process is “unstable” and one may ask for conditions for its “stabilization”

when running through observation times t∈N, thus for some kind of asymp- totic stability or stationarity. Of course, that amounts to the same question as ensuring the stability of the observable process (W_t, t∈N) on the basis of an unobservable history (. . . , W−3, W−2, W−1).

Since a LARM is characterized by a stochastic recursion, it is natural to ask for solutions of this linear stochastic recursion (LSR for short). With regard to the above explanations one is interested in particular in strictly stationary solutions of LSR. A strictly stationary solution means a stochastic process which obeys LSR and is a strictly stationary process.

Next, we collect assumptions on the components of a LARM, which will be made in order to ensure stability properties. As for integrability condi- tions on a typical member U of the i.i.d. family (U_t, t ∈ Z ort ∈ N+), we assume that

(UL) E[(logkUk)⁺]<∞, (U1) U ∈L1, i.e., E[kUk]<∞, (U2) U ∈L₂, i.e., E[kUk²]<∞, and

Cov [U] =E [V V⁰] =:

P

U non-singular,

where V means “U centered”, i.e., U =V +ν withν =E[U],k · k denotes a vector norm and log the natural logarithm.

Lemma 1.1.(U2)⇒(U1)⇒(UL).

Proof.The first implication is trivial, so that only the second one needs to be proved. For U ∈L₁ we have

E[(logkUk)⁺] Z

{kUk≤1}

(logkUk)⁺dP+ Z

{kUk>1}

(logkUk)dP=

= Z

{kUk>1}

(logkUk)dP≤log Z

{kUk>1}

kUkdP<∞,

by Jensen’s inequality.

The essential condition on the matrix F to ensure the “stability” of a LARM is

(EVF) ρ(F)<1,

where ρ(F) denotes the spectral radius ofF, i.e., the maximal absolute value of the eigenvalues of F.

The “eigenvalue condition” (EVF) ensures in some sense a contracti- bility property of the linear mapping induced by F. If this property ofF does not hold, then, by iterating (LARM), the process (Wt, t ∈ N) “drifts away to infinity”.

It is sometimes useful to assume the so-called controllability condition for the pair of matrices (F, G). One introduces the so-called controllability matrix C_k as the aggregation of the matricesF^k−1G, F^k−2G, . . . , F G, G written side by side and abbreviated toC_k:= [F^k−1G|. . .|F G|G]∈Mat(k, km).If

(CMC) rank{C_k:= [F^k−1G|. . .|F G|G]}=k,

the pair of matrices (F, G) is called controllable or (CMC) is called valid.

Obviously, for G = I_k, I_k the identity matrix of order k, condition (CMC) holds. The essence of the controllability property of (F, G) is as follows (see Meyn and Tweedie (1996, p. 95). If one considers the deterministic recursion wt = F wt−1 +Gut, t ∈ N+, then the controllability of (F, G) means that for each pair of “states” w₀, w^∗ ∈ R^k there exists a sequence of “controls”

(u^∗₁, . . . , u^∗_k), u^∗_i ∈ R^m, such that w^∗ = w_k, when (u1, . . . , u_k) = (u^∗₁, . . . , u^∗_k) and the recursion starts at w₀, i.e., each state w^∗ can be reached from each starting point w0 in ksteps. The reason for that is simply the representation

w_k =F^kw₀+ [F^k−1G| · · · |F G|G]





 u1

... u_k







and the fact that the range of the linear mapping induced by [F^k−1G| · · · |F G|G]

is R^k.Therefore, each statew^∗ can be reached from each starting pointw₀ in t steps for all t≥k.

Condition (CMC) obviously represents a concept of communication be- tween states in the deterministic model generated by F and Gand is the key to irreducibility of the MC (Wt, t∈N+) under appropriate conditions on the sequence (Ut, t∈N+).

In this context we prove the following result.

Lemma 1.2. If for a LARM started at w0 ∈ R^k, (CMC) holds and U has a strictly positive λ^m-density, then Wt has a strictly positive λ^k-density for all t≥k.Therefore, the MC (W_t, t∈N)is λ^k-irreducible and aperiodic.

Proof. Consider for a fixedw0 ∈R^kthe linear transformationT :R^km → R^k defined by

T(u1, . . . , u_k) =C_k(u1| · · · |u_k)⁰=:w_k−F^kw0.

Then the matrix C_k ∈Mat(k, km) can be extended to a non-singular matrix C ∈ Mat(km, km), i.e., |detC| > 0, inducing a linear transformation Tb : R^km →R^km. Next,Tb⁻¹ exists as inverse transformation and, according to the theorem on transformation of densities,W_kinduced by (LARM) has a strictly positive λ^k-density q^k as marginal density of a strictly positive density for all starting points w₀ ∈ R^k. Here (u₁| · · · |u_k) := (u₁₁, . . . , u_1m, u₂₁, . . . , u_2m, . . . , uk1, . . . , ukm) = (u⁰₁, . . . , u⁰_k), so that (u1|. . .|u_k)⁰ ∈ Mat(km,1). It follows from the Markov property of (Wt, t ∈ N) that for all t ≥ k the random vectors Wthave a strictly positive λ^k-density, too,

P(Wk+j ∈A) = Z

W

P(Wk+j ∈A |Wj =w)P(Wj ∈dw) =

= Z

W

Q^k(w, A)P(W_j ∈dw) = Z

A

Z

W

q^k(w,dw⁰)P(W_j ∈dw)λ^k(dw⁰) for W =R^k, A∈ B^k and j ∈N. Here Q^k is thek-step transition probability function of the MC (Wt, t ∈ N). Strictly positive densities obviously imply λ^k-irreducibility.

As for aperiodicity, according to Proposition 5.2.4 in Meyn and Tweedie (1996), there exists a countable collection of so-called “small” sets C_i such that C_i are Borel sets and S∞

i=1C_i = R^k. Since there should exist a small set Cj with λ^k(Cj) >0, Proposition 4.2.2 in Hern´andez-Lerma and Lasserre (2003) ensures in turn the existence ofd≥1 disjoint Borel setsD₁, D₂, . . . , D_d such that

∀x∈D_i :Q(x, D_i+1) = 1 fori= 1, . . . , d−1,

∀x∈D_d:Q(x, D₁) = 1 and λ^k

R^k\

d

[

i=1

D_i

= 0.

{D₁, . . . , Dd} is called ad-cycle and the MC is called aperiodic ifd= 1.Now, because of the strictly positive λ^k-densities,dmust be equal to 1. Otherwise,

∀i= 1, . . . , d, ∀x∈D_i :Q(x, D_i) = 0 and, therefore, λ^k(S_d

i=1D_i) would be 0.

So, (W_t, t∈N) is aperiodic.

2. STRICTLY STATIONARY SOLUTIONS OF (LARM)

As for the process (Wt),we first consider the case (Ut, t∈Z), i.e., assume that (W_t,t ∈Z) starts at time −∞. Then it is natural to ask whether (W_t, t ∈ N), has “stabilized” during its infinitely long history (. . . ,−3,−2,−1).

The starting time −∞ can only be meant asymptotically, i.e., as a limit of finite starting times. So, for t∈Zfixed, we think of (t−k), k∈N, as starting time and then let k → ∞. Iterating (LARM) from (t−k) to t, the central term to be studied is

k−1

P

i=0

FⁱGUt−i. Next, we collect results on its convergence in terms of properties of (Ut, t∈Z).

Lemma 2.1. Let (EVF) hold, i.e.,ρ(F)<1.

(i)If (Ut, t∈Z) satisfies(U L), i.e., E[(logkUk)⁺]<∞, then

∀t∈Z ∃fWt:

k−1

X

i=0

FⁱGUt−i →fWt a.s. as k→ ∞.

(ii)If (Ut, t∈Z) satisfies (U1), i.e.,

E[kUk]<∞, U =V +ν with ν =E[U], then

∀t∈Z ∃fWt∈L1:

k−1

X

i=0

FⁱGUt−i L1

−→Wft as k→ ∞,

in addition to the a.s. convergence, and

E[fW_t] =µ= (I−F)⁻¹Gν.

(iii) If (Ut, t∈Z) satisfies(U2), i.e., E[kUk²]<∞ and Cov[U] =

P

U non-singular, then

∀t∈Z ∃fW_t∈L₂:

k−1

X

i=0

FⁱGUt−i−→L2 Wf_t as k→ ∞,

in addition to the convergence properties in (ii), and Cov[fW_t, Wf_t−h] =

∞

X

i=0

F^h+iG P

U G⁰Fⁱ⁰,

Cov[fWt] =

∞

X

i=0

FⁱG P

U G⁰Fⁱ⁰.

Here, ⁰ denotes the transposition of the corresponding matrix.

Proof.(i) We refer to Theorem 1.1 in Bougerol and Picard (1992) which essentially had already been proved by Brandt (1986). Set A_t = F and Bt = GUt for all t ∈ Z. Since for any matrix norm |k · k| one has ρ(F) =

n→∞lim|kFⁿk|^1/n [see Horn and Johnson (1988, p. 299)], we can write γ := inf

n∈N

1

nlog|kFⁿk|

≤ lim

n→∞log|kFⁿk|^1/n= logρ(F)<0 by (EVF) and, furthermore, (log|||A_t|||)⁺ = (log|||F|||)⁺<∞.Moreover,

E [(logkB_tk)⁺] =E[(logkGUk)⁺]≤(log|kGk|)⁺+E[(logkUk)⁺]<∞ for the operator norm |k · k|associated with the vector normk · k. Therefore, all assumptions of the quoted Theorem 1.1 are satisfied, so that our statement follows.

(ii) For allt∈Z, k∈N+ we define W_t^−k=F^kW_t−k+

k−1

X

i=0

FⁱGUt−i,

i.e., W_t^−kis the “state” at timet, if the LARM started at time (t−k) in state Wt−k. With regard to the assumed starting time−∞,we consider for a fixed t∈Z the limit of the sequence (W_t^−k, k∈N+), that is,

Wf_t:= lim

k→∞W_t^−k =

(∗)

∞

X

i=0

FⁱGUt−i =

∞

X

i=0

FⁱGVt−i+

∞

X

i=0

FⁱGν

(∗∗)=

∞

X

i=0

FⁱGVt−i+ (I−F)⁻¹Gν;

(∗) holds by condition (EVF) according to Theorem 5.6.12 in Horn and John- son (1988, p. 298) while (∗∗) holds according to a result on p. 301 in that book. Since

k−1

X

i=0

FⁱGVt−i

≤

∞

X

i=0

|kFⁱk| kGV_t−ik

for an appropriate matrix norm, we have E

^∞ X

i=0

|kFⁱk| kGV_t−ik

=

∞

X

i=0

|kFⁱk|E[kGV_t−ik]<∞

by the Monotone Convergence Theorem, condition (U1) and Corollary 5.6.14 in Horn and Johnson (1988, p. 299), i.e.,

∞

P

i=0

|kFⁱk| kGV_t−ik ∈L₁.In turn this implies [see, e.g., Lo`eve (1963, p. 163)]

k−1

X

i=0

FⁱGUt−i L1

−→

∞

X

i=0

FⁱGUt−i =Wft

as k→ ∞ and E

∞

X

i=0

FⁱGVt−i

= lim

k→∞E

k−1

X

i=0

FⁱGVt−i

= 0.

Therefore, E[fW_t] = (I−F)⁻¹Gν.

(iii) The proof follows from Proposition C.7 in L¨utkepohl (1991, p. 490), as, by (EVF) (Fⁱ, i∈N) is an absolutely summable sequence of matrices (see again Horn and Johnson (1988, p. 301), and E[U_t⁰Ut] ≤ C < ∞ because of (U2). Moreover, according to Proposition C.8 in L¨utkepohl (1991, p. 491), one can compute the covariances

ΓfW(h) :=E[(fWt−µ)(Wft−h−µ)⁰] lim

k→∞

k−1

X

i=0 k−1

X

j=0

FⁱGE[Vt−iV_t−h−j⁰ ]G⁰F^j0 =

= lim

k→∞

k−1

X

j=0

F^h+jG P

U G⁰F^j0 =

∞

X

i=0

F^h+iG P

U G⁰Fⁱ⁰ = Cov[fW_t,Wft−h].

This holds as E[VtV_s⁰] = 0 for s 6= t and E[VtV_t⁰] = P

U for all t ∈ Z. In particular, for h= 0 we have

ΓfW(0) =E[(fW_t−µ)(fW_t−µ)⁰] =

∞

X

i=0

FⁱG P

U G⁰Fⁱ⁰ = Cov[fW_t].

Next, we ask for properties of the stochastic process (fWt, t ∈Z) which is well defined under the assumptions of Lemma 2.1.

Lemma 2.2. Under conditions (EVF) and (UL) the stochastic process (Wft, t∈Z)is the unique strictly stationary solution of the stochastic recursion (LARM)and enjoys the Markov property. LetP denote the transition probabi- lity function of the M C (fW_t, t ∈ Z). Then the distribution m of Wf_t is an invariant probability measure of the chain, that is, m(A) =R

P(w, A)m(dw).

Proof. Although the result is contained in Theorem 1.1 in Bougerol and Picard (1992), we prove it here except for the uniqueness of the solution.

Uniqueness will be ensured by Theorem 3.1 to be proved in what follows.

On account of the representationWft=

∞

P

i=0

FⁱGUt−i,t∈Z,the stochastic recursion (LARM)

FWft−1+GUt=F ∞

X

i=0

FⁱGUt−1−i

+GUt=

∞

X

i=0

Fⁱ⁺¹GU_t−(i+1)+F⁰GUt=Wft. is satisfied. Since (fW_t, t∈Z) obeys (LARM), it enjoys the Markov property.

Then in combination with the equation Wft=

∞

X

i=0

FⁱGUt−i

=D

∞

X

i=0

FⁱGUt+h−i =Wf_t+h

for all h ∈ Z, the strict stationarity of (fW_t, t ∈ Z) follows. Here, = de-^D notes equality in distribution. The last property of the statement follows from the equation

m(A) =P(Wf_t∈A) = Z

P(fW_t∈A |Wft−1 =w)P(fWt−1∈dw)

= Z

P(w, A)m(dw).

Remarks 1.1. 1. The strictly stationary solution Wft =

∞

P

i=0

FⁱGUt−i, t ∈ Z

is a nonanticipative one according to Definition 2.2 in Bougerol and Picard (1992) since Wf_t is independent of the random vectors (U_r, r > t) for any t∈Z. Therefore, the solution (fW_t, t∈Z) is independent of the future at any given time and contains only noise terms or innovations from the past up to the present.

2. Bougerol and Picard (1992) introduce an irreducibility property for LARM: An affine subspace H of R^k is said to be invariant under (LARM) if {F w+GU, w ∈ H} ⊂ H a.s. A LARM is called irreducible if R^k is the only affine invariant subspace, i.e., there is no proper affine subspace H⊂R^k such that if LARM starts at W₀ = w ∈ H, then it remains in H all the time. They show in addition to the result of Lemma 1.3 that under the above irreducibility condition and condition (UL) the validity of (EVF) is even a necessary condition for the existence of a nonanticipative strictly stationary solution of (LARM).

3. In the time series literature, see e.g. L¨utkepohl (1991), the stochas- tic process (Wft, t ∈ Z), which is well-defined under the conditions (EVF)

and (UL), is called “stable” and correspondingly condition (EVF) for LARM

“stability condition” or “stationarity condition” [see L¨utkepohl (1991, p. 20)].

In the time series literature the notions “stability” and “stationarity”

are often used synonymously. Moreover, in some books, there is no distinction between “strict stationarity” and “second-order stationarity”. If one regards the stochastic recursion (LARM) as a time series model, i.e., as a vectorial autoregressive model of order 1, one can ask for time series, i.e., stochastic processes (Wt, t∈Zort∈N),which are compatible with (LARM).

Therefore, in the case “t ∈ Z” one asks for solutions of (LARM) while in the case “t∈ N”, given a starting variable W₀, one generates (W_t, t ∈N) by iterating (LARM). As was shown above, under suitable conditions, a time series which has started in the infinite past, i.e., at time−∞, has “stabilized”, so that it appears as “stable” in the stochastics sense during observation times t∈N. This explains the introduction of (fWt, t∈Z).

The stochastic process or time series fWt =

∞

P

i=0

FⁱGUt−i, t ∈ Z dealt with in Lemmas 2.1 and 2.2 is called the canonical moving average (MA for short) representation of the linear autoregressive model LARM. It requires the validity of (EVF) and exists under (UL) almost surely, under (U1) in the L₁-sense and under (U2) in theL₂-sense as infinite series based on the infinite past. Usually, in the time series literature one directly deals with (fW_t, t∈Z).

3. THE CASE(UT, T ∈N+): MARKOV CHAINS GENERATED BY LARM

Since we consider as observation times for LARMt∈N,respectively an induced time series indexed by t∈N, we now turn to study LARM starting with a random vectorW0.The distribution ofW0may bem=L(Wf0) resulting from Lemma 2.2 or any other probability measure p₀.

In any case, we study (Wt, t ∈ N) as a MC. If W0 ∼ m, we have a

“stable”, i.e., strictly stationary MC. If W₀ ∼ p₀, again the question arises whether (Wt, t ∈ N) “stabilizes” as t → ∞. In this context, we will take advantage on available results for MC’s.

Theorem 3.1. Assume (EVF)holds, i.e.,ρ(F)<1,and (U_t, t∈N+) satisfies (U1), i.e.,E[kUk]<∞, U =V +ν with ν=E[U].Let the random vector W₀ be independent of (U_t, t ∈ N+) and (W_t, t ∈ N) generated by LARM.Then

(i)W_t→^L W∞ as t→ ∞and W∞=

∞

X

i=0

FⁱGU_i+1 ∈L₁, E[W∞] = (I−F)⁻¹Gν=µ;

(ii)if P denotes the transition probability function of the M C (Wt,t∈ N), then (W_t, t ∈ N) is P^t-weakly ergodic, i.e., P^t(w₀,·) →^w π as t → ∞, π =L(W_∞)being the unique P-invariant probability measure, so that π =m;

(iii) (W_t,Pπ, t∈N),i.e., the M C (W_t, t∈N) for whichW₀ ∼π, is an ergodic strictly stationary process;

(iv)if, in addition,(U_t, t∈N+)satisfies (U2),i.e.,E[kUk²]<∞,then W∞∈L₂ with covariance matrix

Cov[W∞] =

∞

X

i=0

FⁱG P

U G⁰Fⁱ⁰.

Here →^L stands for convergence in distribution and→^w for weak convergence of probability measures.

Proof. (i) W_t=F^tW₀+

t−1

P

i=0

FⁱGUt−i after iterating (LARM). Moreover,

t−1

P

i=0

FⁱGUt−i

=D t−1

P

i=0

FⁱGU_i+1. Just as in the proof of Lemma 2.1 (ii), we con- clude that

∞

P

i=0

FⁱGUi+1 =W∞∈L1. Therefore,

t−1

P

i=0

FⁱGUi+1 L1

−→W∞,whence

t−1

P

i=0

FⁱGUi+1

→P W∞and then

t−1

P

i=0

FⁱGUi+1

→L W∞ast→ ∞.SinceF^tW0 →0 a.s. by (EVF), Wt

→L W∞ ast→ ∞.The representation ofE[W∞] follows as in Lemma 2.1 (ii).

(ii) IfP is the transition probability function of the MC (W_t, t∈N), then P^t(w0,·) is the distribution ofWt if the MC starts atW0 =w0. Therefore, (i) says thatP^t(w0,·)→^w π ast→ ∞,π =L(W∞) =m. Next, we show thatπ is P-invariant. This amounts to show that ifW0 ∼πthenW1 = (F W0+GU1)∼ π. Choose for W0 the representation W0 =

∞

P

i=0

FⁱGUi+2 ∼π. Then

W1 =

∞

X

i=0

FⁱGUi+1∼π.

Because of the time-homogeneity of the MC (W_t, t∈N) the result is valid for each transition from time t to time (t+ 1). Moreover, π turns out to be the unique P-invariant probability measure: P^t(w,·)→^w π means that

Z

W

f(w⁰)P^t(w,dw⁰)→ Z

W

f(w⁰)π(dw⁰)

for all bounded, continuous real-valued functions f onW =R^k, ast→ ∞. If q is anotherP-invariant probability measure then, for f as above,

Z

W

f(w⁰)q(dw⁰) = Z

W

f(w⁰) Z

W

P^t(w,dw⁰)q(dw) =

= Z

W

Z

W

f(w⁰)P^t(w,dw⁰)q(dw)→ Z

W

f(w⁰)π(dw⁰) as t→ ∞. Since R

W

f(w⁰)q(dw⁰) = R

W

f(w⁰)π(dw⁰) for all bounded continuous real-valued functions f, q should be equal toπ.

(iii) is a well-known consequence of the uniqueness of the invariant probabi- lity measure π, see Hern´andez-Lerma and Lasserre (2003, p. 35), e.g.

(iv) Since we know from (i) that

t−1

P

i=0

FⁱGUi+1 →

∞

P

i=0

FⁱGUi+1 a.s., ac- cording to the “Equivalence Theorem” in Lo`eve (1963, p. 251), we have

kW_∞k²=

t→∞lim

t−1

X

i=0

FⁱGU_i+1

2 t→∞lim

t−1

X

i=0

FⁱGU_i+1

2

≤

≤

∞

X

i=0

|kFⁱk|²kGU_i+1k² a.s.

for an appropriate matrix norm. As in the proof of Lemma 2.1 (ii), one con- cludesE[kW∞k²]<∞. The covariance matrix ofW∞can be calculated as in the proof of Lemma 2.1 (iii):

Cov[W∞] =E[(W∞−µ)(W∞−µ)⁰]

∞

X

i=0

FⁱG P

U G⁰Fⁱ⁰.

In order to get a stronger ergodicity property, we assume strictly positive densities of the innovations and, in addition, condition (CMC) which “spreads out” the probability mass.

Theorem 3.2. Consider a LARM starting with the random vector W₀, independent of (Ut, t∈N+). Moreover, assume that (EVF) and (CMC) are valid, (U_t, t∈N+) satisfies condition (U1) and U has a strictly positive λ^m- density. Then (Wt, t∈ N) is a geometrically ergodic MC, i.e., there exists a unique invariant probability measureπ with respect to the transition probability function P of (W_t, t∈N) such that

kP^t(w,·)−π(·)k ≤ρ^tR_w, t∈N,

for some 0 < ρ < 1, for all w ∈ R^k with corresponding constants R_w < ∞.

Here, k · k denotes the norm of total variation of signed measures.

Proof. By Lemma 1.2, the MC (Wt, t ∈ N) is λ^k-irreducible and ape- riodic. Moreover, it is a Feller chain. Geometric ergodicity of (W_t, t∈N) can be proved by means of a theorem of Feigin and Tweedie (1985), see also Meyn and Tweedie (1996, p. 354/5).

First, we spot a special vector norm from condition (EVF). Sinceρ(F)<

1, there exists a matrix norm k| · |k such that k|F|k<1, which yields a com- patible vector norm k · k, i.e., the inequality kF wk ≤ k|F|k kwk holds for all w ∈ R^k, see Horn and Johnson (1988, p. 297). Using that vector norm we define the test function h : R^k → [1,∞) by h(w) = 1 +kwk. With δ > 0 defined by k|F|k= 1−δ <1, one can estimate

E[h(W_t) |Wt−1=w] =E[h(F w+GU_t)] =E[1 +kF w+GU_tk]≤

≤ k|F|k kwk+ 1 +E[kGU_tk]≤(1−δ)kwk+ (1−δ) +δ+K1=

= (1−δ)[1 +kwk] +K₂ = (1−δ)h(w) +K₂

with finite constantsK₁, K₂.Now, a large enough compact setC⊂R^kcan be chosen such that K2 < ^δ₂[1 +kwk] for all w /∈C. Then for allw /∈C one gets

E[h(W_t)|Wt−1 =w]≤

1−δ 2

h(w).

As all norms on R^k are equivalent, C being compact with respect to k · k,it also is compact with respect to the Euclidean vector norm. Therefore, the proof is complete by the theorem mentioned above.

4. NORMALLY DISTRIBUTED INNOVATIONS

In the case of normally distributed innovations (U_t, t∈N+),the unique invariant probability measure π is a normal distribution.

Theorem 4.1. Consider a LARM starting with the random vector W₀ independent of (Ut, t∈N+). Moreover, assume that (EVF) and (CMC) are valid and

U ∼N(ν, P

U) with a positive definite covariance matrix

P

U. Then the M C (Wt, t ∈ N) is geometrically ergodic and, moreover,

W∞∼N(µ, P) =π with µ = (I −F)⁻¹Gν and P

=

∞

P

i=0

FⁱG P

U G⁰Fⁱ⁰. As a consequence, the M C(Wt, t∈N) is weakly asymptotically stationary.

Proof. The positive definiteness of P

U ensures the strict positivity of the λ^m-density of U on the whole of R^m, therefore the validity of Theorem 3.2.

It remains to determine the distribution of W∞. First, for U ∼ N(ν, P

U) with a positive definite

P

U one can find a positive definite matrix A such that AA⁰ =

P

U [see Horn and Johnson (1988, p. 405)] and one can write U =AV +ν with V ∼N(0, Im). Since

Wt=F^tW0+

t−1

X

i=0

FⁱGν+

t−1

X

i=0

FⁱGAVt−i

and

E[W_t] =:µ_t=F^tE[W₀] +

t−1

X

i=0

FⁱGν,

for W₀ =w₀ we get

Cov[W_t] =E[(W_t−µ_t)(W_t−µ_t)⁰]E X^t−1

i=0

FⁱGAVt−i

X^t−1

i=0

FⁱGAVt−i

0

=

=

t−1

X

i=0

FⁱG P

UG⁰Fⁱ⁰ :=

P

t

as

E[AV_tV_t−j⁰ A⁰] = Cov[U_t, Ut−j] = 0 fort, t−j∈N.

Obviously, the W_t are normally distributed for all t ∈ N+ : W_t ∼ N(µt,P

_t). If A denotes the block matrix with k matrices A on its main diagonal and zeroes outside and A⁰ the block matrix with k matrices A⁰ on its main diagonal and zeroes outside, then one can represent P

_k in the form P

_k=C_kA A⁰C_k⁰ with

C_k⁰ =











(F^k−1G)⁰ ... (F G)⁰

G⁰









 .

Since the matricesAandA⁰are non-singular, we have rank (C_kA) = rankC_k= k, rank(A⁰C_k⁰) = k, hence rank P

_k = k according to Horn and Johnson (1988, p. 13). Moreover, since by the Cayley Hamilton theorem, see Horn and Johnson (1988) e.g., one can express powers F^t for all t ≥ k as linear combinations of I, F, F², . . . , F^k−1, one concludes that

rank P

t= rank ^t−1

X

i=0

FⁱG P

UG⁰Fⁱ⁰

=k.

Finally, P :=

∞

P

i=0

FⁱG P

UG⁰Fⁱ⁰,which is well-defined under (EVF), has rank k, too. The distribution π of

W∞= (I−F)⁻¹Gν+

∞

X

i=0

FⁱGAVi+1

is determined by means of the limit of the Fourier-transformϕ_tof

t−1

P

i=0

FⁱGAVt−i

for t→ ∞: forr∈R^k one gets

t→∞lim ϕt(r) = lim

t→∞exp

−1 2r⁰

t−1

X

i=0

FⁱG P

U G⁰Fⁱ⁰r

=

= exp

−1 2r⁰

∞

X

i=0

FⁱG P

U G⁰Fⁱ⁰r

=ϕ

N(0,P )(r), where ϕ

N(0,P

) denotes the Fourier-transform of the corresponding N(0,P )- distribution.

Remarks. 1. If G = I_k then (CMC) holds obviously, U has a strictly λ^k-density and, therefore, (W_t, t∈N) isλ^k-irreducible and aperiodic.

2. If (CMC) is not satisfied, then the MC may be restricted to the range W_R⊂W =R^k of the controllability matrix:

W_R= range[F^k−1G| · · · |F G|G] = ^k−1

X

i=0

FⁱGα_i

α_i ∈R^m

which is also the range of

∞

P

i=0

FⁱGG⁰Fⁱ⁰. Ifw₀ ∈W_RthenF w₀+Gu₁ ∈W_Rfor any u₁ ∈R^m. This shows thatW_Ris absorbing, hence the LARM is restricted toWR. In turn, the controllability condition is satisfied on WR.

If the corresponding MC has an invariant probability measureπ, thenπ is concentrated on WR,hence is singular w.r.t. λ^k.

5. CONCLUDING REMARKS

Within the framework of LARM, numerous time series models of the au- toregressive type can be dealt with, see, e.g., Meyn and Tweedie (1996), Feigin and Tweedie (1985) or Bougerol and Picard (1992). Ergodicity properties and the Markov property of the times series (W_t, t ∈ Z or t ∈ N, respectively) make accessible the classical limit theorems of probability, see again Meyn

and Tweedie (1996). In particular, for the central limit theorem Jones (2004) presents an excellent collection of results.

The unified treatment of different time series models within the frame- work of LARM yields simultaneously many important results for those models (i.e., time series).

Acknowledgements. The authors gratefully acknowledge support from Deutsche Forschungsgemeinschaft under Grant 436/RUM113/21/4–1.

REFERENCES

[1] P. Bougerol and N. Picard, Strict stationarity of generalized autoregressive processes.

Ann. Probab.20(1992), 1714–1730.

[2] A. Brandt,The stochastic equationYnH=AnYn+Bn with stationary coefficients. Adv.

in Appl. Probab.18(1986), 211–220.

[3] P.D. Feigin and R.L. Tweedie, Random coefficient autoregressive processes: A Markov chain analysis of stationarity and finiteness of moments. J. Time Ser. Anal.6 (1985), 1–14.

[4] O. Hern´andez-Lerma and J.B. Lasserre, Markov Chains and Invariant Probabilities.

Birkh¨auser, Basel, 2003.

[5] G.L. Jones, On the Markov chain central limit theorem. Probability Surveys1 (2004), 299–320.

[6] R.A. Horn and C.R. Johnson,Matrix Analysis. Cambridge Univ. Press, Cambridge, 1988.

[7] M. Lo`eve,Probability Theory. Van Nostrand, Princeton, 1963.

[8] H. L¨utkepohl,Introduction to Multiple Time Series Analysis. Springer, Berlin, 1991.

[9] S.P. Meyn and R.L. Tweedie,Markov Chains and Stochastic Stability. Springer, London, 1996.

Received 21 June 2009 Universit¨at Duisburg - Essen, Campus Duisburg Revised 18 November 2009 Institut f¨ur Mathematik

D-47048 Duisburg, Deutschland herkenrath@math.uni-duisburg.de

Romanian Academy

“Gheorghe Mihoc-Caius Iacon” Institute of Mathematical Statistics and Applied Mathematics

Casa Academiei Romˆane Calea 13 Septembrie no. 13 050711 Bucharest, Romania miosifes@valhalla.racai.ro

and

Universit¨at der Bundeswehr M¨unchen Werner-Heisenberg-Weg 39 D-85577 Neubiberg, Deutschland Andreas.Rudolph@unibw-muenchen.de