Nonparametric estimation of the jump rate for non-homogeneous marked renewal processes

www.imstat.org/aihp 2013, Vol. 49, No. 4, 1204–1231

DOI:10.1214/12-AIHP503

© Association des Publications de l’Institut Henri Poincaré, 2013

Nonparametric estimation of the jump rate for non-homogeneous marked renewal processes ¹

Romain Azaïs, François Dufour and Anne Gégout-Petit

INRIA Bordeaux Sud-Ouest, team CQFD, France and Université Bordeaux, IMB, CNRS UMR 5251, 351, Cours de la Libération, 33405 Talence cedex, France. E-mail:romain.azais@inria.fr;dufour@math.u-bordeaux1.fr;anne.petit@u-bordeaux2.fr

Received 13 February 2012; revised 4 June 2012; accepted 19 June 2012

Abstract. This paper is devoted to the nonparametric estimation of the jump rate and the cumulative rate for a general class of non-homogeneous marked renewal processes, defined on a separable metric space. In our framework, the estimation needs only one observation of the process within a long time. Our approach is based on a generalization of the multiplicative intensity model, introduced by Aalen in the seventies. We provide consistent estimators of these two functions, under some assumptions related to the ergodicity of an embedded chain and the characteristics of the process. The paper is illustrated by a numerical example.

Résumé. Ce papier est consacré à l’estimation non-paramétrique du taux de saut et du taux de saut cumulé pour une classe générale de processus de renouvellement marqués non-homogènes, définis sur un espace métrique séparable. Dans notre cadre de travail, l’estimation nécessite seulement une observation du processus en temps long. Notre approche est basée sur une généralisation du modèle à intensité multiplicative introduit par Aalen dans les années soixante-dix. Nous donnons des estimateurs consistants de ces deux fonctions, sous des hypothèses portant sur l’ergodicité d’une chaîne immergée et sur les caractéristiques du processus. Le papier est illustré par un exemple numérique.

MSC:Primary 62G05; secondary 62M09

Keywords:Non-homogeneous marked renewal process; Nonparametric estimation; Jump rate estimation; Nelson–Aalen estimator; Asymptotic consistency; Ergodicity of Markov chains

1. Introduction

The purpose of this paper is to develop a nonparametric method for estimating the jump rate of a non-homogeneous marked renewal process, when only one observation of the process within a long time is available. Our estimation procedure is premised on a generalization of the well-known multiplicative intensity model, investigated by Aalen in [2,3].

We introduce a general class of non-homogeneous marked renewal processes (NHMRP’s), defined on an open subset of a separable metric space. The motion of the process depends on three characteristics namely the jump rateλ, which specifies the interarrival times, the transition kernelQ, and a functiont, which plays the role of a deterministic censorship depending on the state of the process. In this framework, the jump rateλis a function of two variables:

a spatial mark and time. Here, our aim is to propose a nonparametric method for estimating both the jump rateλ and the cumulative rate from only one observation of the process within a long time interval. In addition, the class of non-homogeneous marked renewal processes which we consider may be related to particular piecewise-deterministic

1This work was supported by ARPEGE program of the French National Agency of Research (ANR), project “FAUTOCOES,” number ANR-09- SEGI-004.

Markov processes (PDMP’s, see the book [13]), whose transition kernel does not depend on time. This paper is also a keystone of [6], in which we provide a consistent nonparametric estimator of the conditional density associated to the jump rate of a general PDMP.

Aalen suggested in the middle of the seventies the famous multiplicative intensity model (see his PhD thesis [1], or [2,3]). In this work [1–3], it is assumed that the intensity of the underlying counting processNcan be written as the product of a predictable processY and a deterministic functionλ, called jump rate or hazard rate. In this framework, the so-called Nelson–Aalen estimator provides an estimate of the cumulative rateΛ(t )=_t

0λ(s)ds. Ramlau-Hansen suggested a few years later, in [23], a nonparametric method for estimating directly the rate λ, by smoothing the Nelson–Aalen estimator by kernel methods.

A large number of estimation problems in survival analysis or in statistics of processes are related to counting processes depending on a spatial variable. This one may be seen as a mark or as a covariate. In this context, Aalen’s estimator is proved to be a very popular and powerful method, since the multiplicative assumption is satisfied in a large variety of applications (see for instance [5]). In particular, one can apply the Nelson–Aalen approach for estimating the jump rate of a marked counting process, whose state space is finite, from a large number of independent observations.

More recently in 2011, Comte et al. proposed in [10] an adaptive method for estimating the jump rate of a marker- dependent counting process, under the multiplicative assumption.

There exists an extensive literature on nonparametric and semiparametric estimation methods when the spatial mark belongs to a continuous state space. However, we do not attempt to present an exhaustive survey on this topic.

A significant list of references on this research field can be found in [4,5,16,20] and the references therein. In particular, McKeague and Utikal were interested in [21] in the estimation of the jump rate when the covariate takes its values in[0,1]. Their approach is based on smoothing a Nelson–Aalen type estimator both in spatial and time directions.

In particular, they prove the consistency of their estimator. Li and Doss chose another approach, based on a local linear fit in the spatial direction (see [19]). They extended McKeague and Utikal’s work for the multidimensional case, and proved weak convergence results. One may also refer to the papers written by Utikal [25,26] about jump rate estimation for two special classes of marked counting processes, under some continuous-time martingale assumptions.

The Euclidean structure of the covariate state space is a keystone of the papers mentioned above. At the same time, nonparametric approaches have also been considered by Beran in [8], Stute in [24] and Dabrowska in [12], but for independent observations. Semiparametric methods have also been considered by many authors, beginning with Cox in [11]. The interested reader may consult the book [5] and the references therein for a complete review of the literature on these models.

In many aspects, our approach and the results mentioned above are different and complementary. These differences may be briefly described as follows. Our paper is based on a generalization of the multiplicative intensity model, involving a discretization of the state space, and, as a consequence, an approximation of the functions of interest.

Indeed, we do not impose any conditions on the state space, such as to be Euclidean. This notably excludes the methods investigated by McKeague and Utikal [21], Li and Doss [19], or Utikal [25,26]. Furthermore, these authors consider some assumptions about both continuous-time martingale properties and the asymptotic behavior ofY. From a practical point of view, this kind of assumption is not completely satisfactory due to the fact that these conditions may be difficult to check, especially in our case. On our side, we overcome this difficulty by providing tractable conditions, directly related to the primitive data of the process, to ensure the consistency of our estimator.

The present paper is divided into two parts. In the first one, we consider that the transition kernel only charges a finite set of points. It amounts to considering the state space is a discrete one. In this context, Theorem3.1states that the multiplicative model is satisfied. In the second part, we assume that the kernel Qis diffuse, that is to say, it does not charge singletons. If theZ_i’s denote the marks of the underlying process, for anyx andi, the indicator function1_{Zi=x}is almost surely null. This rules out the method developed in the discrete case. Our procedure relies on a partition of the state space, labeled(A_k). In this context, it appears intuitive to consider the counting process

Nn(Ak, t )=

n−1

i=0

1_{Z_i∈A_k}1_{S_i+1≤t},

where theS_i’s denote the interarrival times. Although Aalen’s multiplicative model does not hold for this counting process, the stochastic intensity ofN_n(A_k, t )is almost surely equivalent to the productY_n(A_k, t)l(A_k, t )(see Propo- sition4.12), wherel(Ak, t )is an approximation of the jump rateλ(x, t ), forx∈Ak, and

Yn(Ak, t )=

n−1

i=0

1_{Z_i∈A_k}1_{S_i+1≥t}.

In this context, it is natural to introduce the following processes, L_n(A_k, t )=

_t

0

Y_n(A_k, s)⁺dNn(A_k, s),

whereY_n(A_k, t )⁺is the generalized inverse ofY_n(A_k, t ), and L^∗_n(Ak, t )=

_t

0

l(Ak, s)1_{Y_n(A_k,s)>0}ds.

In Aalen’s papers, the differenceL_n(A_k, t )−L^∗_n(A_k, t )is a continuous-time martingale, whereas on our part, it is not the case since there exists an extra-terman(t), which vanishes when ngoes to infinity. Intuitively, this means that the multiplicative model asymptotically makes sense. Referring to both Lenglart’s inequality and the asymptotic behavior of the extra-terma_n(t), we prove thatL_n(A_k, t )is a consistent estimator ofL(A_k, t )=_t

0l(A_k, s)ds(see Proposition4.16). We deduce from this a consistent estimator of the cumulative rateΛ(x, t )=_t

0λ(x, s)ds (see Theorem4.23), sinceL(A_k, t ) andΛ(x, t )are close forx∈A_k (see Lemma4.20). Next, we focus on smoothing this estimator by kernel methods in order to suggest a consistent estimator of l(Ak, t ) (see Proposition4.26) and, therefore, ofλ(x, t )(see Theorem4.27). An inherent difficulty throughout this paper is related to the presence of the deterministic censorship.

The paper is organized in the following way. We first give, in Section 2, the precise definition of the class of non-homogeneous marked renewal processes which we are interested in, and we provide an example of application in reliability. We state also some technical results about continuous-time martingales and conditional independences.

Section3is devoted to the discrete case, where we consider that the transition kernelQonly charges a finite number of points. The main contribution of the paper lies in Section4, in which we do not impose any conditions on the state space. In this part, we provide consistent estimators of the cumulative rate (see Theorem4.23) and the jump rate (see Theorem4.27). Finally in Section5, we present a numerical example to illustrate the good behavior of our estimators on finite sample size.

2. Definition and first results

In this section, we first define the class of non-homogeneous marked renewal processes under consideration. We consider a piecewise-constant continuous-time process(Xt)t≥0. For anyt≥0,Xt takes its values onE, which is an open subset of a separable metric space(E, d). The motion of(X_t)_t≥0may be given for anyt ≥0 in the following way,

∀t≥0, X_t=Z_n ifS₀+ · · · +S_n≤t < S₀+ · · · +S_n+1.

TheZ_n’s correspond to the locations of(X_t)_t≥0, while theS_n’s denote the interarrival times. We assume that(Z_n)_n≥0 is a Markov chain on(E,B(E)), defined on a probability space(Ω,A,Pν0), whose transition kernel is denoted byQ.

The distribution of the starting pointZ₀is assumed to beν₀. An equivalent formulation (see for instance Theorem 2.4.3 of [7]) is given by,

∀n≥1, Zn=ψ (Zn−1, εn−1), (1)

whereψ is a measurable function, and (εn)n≥0 is a sequence of independent and identically distributed random variables. Now, one defines the sequence(Sn)n≥0on(R₊,B(R₊))from two functions:λ:E×R₊→R₊which plays

the role of the jump rate, and the deterministic censorshipt:E→ ]0,+∞]. For each integern≥1, the distribution ofS_nsatisfies, for anyt≥0,

Pν0

S_n> t|{Z_i: i≥0}, S₀, . . . , S_n−1

=Pν0(S_n> t|Z_n−1) (2)

=exp

− t

0

λ(Z_n−1, s)ds 1_{0≤t <t(Zn−1)}.

In addition, we assume that S₀=0. Hence, there exist a functionϕ and a sequence of independent and identically distributed random variables(δn)n≥0, which is independent of the sequence(εn)n≥0, such that,

∀n≥1, S_n=ϕ(Z_n−1, δ_n−1).

One assumes that both the sequences(ε_n)_n≥0and(δ_n)_n≥0are independent ofZ₀.

We recall that we are interested here in the nonparametric estimation of the jump rateλ, from one observation of the embedded chain(Z_n, S_n)_n≥0. This class of non-homogeneous renewal models may be related to PDMP’s, for which the transition kernelQdoes not depend on time. Hence, the estimation method developed in this paper is very useful in order to estimate the conditional distribution of the interarrival times for PDMP’s (see [6]). Nevertheless, providing a method for estimating the jump rate for this class of stochastic models has an intrinsic interest. In the following, we present an example in reliability of non-homogeneous marked renewal process satisfying the model mentioned above.

Let us consider a machine, whose production configuration takes its values in an open subset ofR^d. The dynamic of the regime is assumed to be a non-homogeneous renewal process: the state is piecewise-constant until a failure spontaneously occurs. One naturally considers that the failure rate depends on the production regime. When the ma- chine breaks down, the repair occurs instantaneously, and the machine configuration is randomly changed, according to a transition kernelQdepending only on the previous working state. In addition, one may consider that there exists a deterministic period of inspection, depending on the production configuration too. The inspection is instantaneous and the next regime changes according to the kernelQ. The estimation of the failure rate from only one observation of the regime state within a long time may bring some informations about the behavior of the production machine.

The main benefit of this approach is as follows: the estimation does not need the observation of a great number of similar machines.

One may associate to the jump rateλ, the cumulative rate, the survival function and the probability density function, which are related to it. The conditional densityf satisfies,

∀ξ∈E,∀t≥0, f (ξ, t )=λ(ξ, t )exp

− _t

0

λ(ξ, s)ds . (3)

The cumulative rateΛis defined by,

∀ξ∈E,∀t≥0, Λ(ξ, t )= _t

0

λ(ξ, s)ds. (4)

Finally, the conditional survival function is denoted byG.

∀ξ∈E,∀t≥0, G(ξ, t )=exp

− _t

0

λ(ξ, s)ds . (5)

We have the straightforward relation between these functions: λ=f/G. For each n≥1, Gn denotes theσ-field generated by thenfirstZ_i’s,

Gn=σ (Z₀, . . . , Z_n−1). (6)

For each integeri, the one-jump counting processNⁱ⁺¹is given for anyt≥0, by Nⁱ⁺¹(t)=1_{S_i+1≤t},

and(Ftⁱ⁺¹)_t≥0denotes the associated filtration. In this section, we shall prove two results: the first one is related to two conditional independence properties; the second one deals with the continuous-time martingale associated to the counting processNⁱ⁺¹in a special filtration.

Proposition 2.1. Letnbe an integer and1≤i≤n.For each integerj =i,lett_j≥0andt≥0.Then,we have

j =i

Ft^j_j⊥

Gn

Ftⁱ and Ftⁱ ⊥

σ (Z_i−1)Gn.

Furthermore,we deduce from Proposition6.8of[17]this immediate corollary:for anys < t, 0≤i≤n−1,

j =i+1

Fs^j ⊥

Gn∨Fsⁱ⁺¹

Ftⁱ⁺¹ and Ftⁱ⁺¹ ⊥

σ (Z_i)∨Fsⁱ⁺¹

Gn.

Proof. The reader may find the proof in AppendixA.

We have also a continuous-time martingale property. This is the one associated to the counting processNⁱ⁺¹. Lemma 2.2. For each integeri,the processMⁱ⁺¹given by,

∀0≤t < t(Z_i), Mⁱ⁺¹(t)=Nⁱ⁺¹(t)− _t

0

λ(Z_i, u)1_{S_i+1≥u}du, (7)

is a continuous-time martingale in the filtration(σ (Z_i)∨Fsⁱ⁺¹)_0≤s<t(Zi).

Proof. The proof is deferred in AppendixA.

Proposition2.1and Lemma2.2are prominent for the next results. In addition, the reference to the underlying probability measurePν0 will be implicit in the text. For the sake of readability, we shall writePxinstead ofPδ_{x}. 3. Discrete state space

We assume here that the transition kernelQonly charges a finite set which we denote{x₁, . . . , x_M}. One may associate to eachxi the deterministic exit timet_i=t(xi). Now, one considers thatkis fixed. In this section, we shall prove in Theorem3.1that the multiplicative model is satisfied for estimating the cumulative rateΛ(x_k,·).

For each integern, let us introduce the counting processN_n(x_k,·)by,

∀t≥0, N_n(x_k, t )=

n−1

i=0

1_{S_i+1≤t}1_{Z_i=x_k}. (8)

In addition, for anyt≥0, we define

Y_n(x_k, t )=

n−1

i=0

1_{Si+1≥t}1_{Zi=xk}. (9)

Theorem 3.1. Letn≥1.The processM_n(x_k,·)defined by,

∀0≤t < t_k, M_n(x_k, t )=N_n(x_k, t )− _t

0

λ(x_k, s)Y_n(x_k, s)ds (10)

is a(Ftⁿ)_{0≤t <t}

k-continuous-time martingale underPν0,with,

∀0≤t < t_k, Ftⁿ=Gn∨

n−1 i=0

Ftⁱ⁺¹.

Proof. Let 0≤s < t < t_k. Plugging (8) and (9) in (10), we have

M_n(x_k, t )=

n−1

i=0

Nⁱ⁺¹(t)1_{Zi=xk}− _t

0

λ(x_k, u)1_{Si+1≥u}1_{Zi=xk}du

=

n−1

i=0

Mⁱ⁺¹(t)1_{Zi=xk}, (11)

by (7). Thus, we obtain

Eν₀

Mn(xk, t )|Fsⁿ

=

n−1

i=0

Eν₀

Nⁱ⁺¹(t)|Fsⁿ

−Eν₀

t 0

1_{S_i+1≥u}λ(xk, u)duFsⁿ

1_{Z_i=x_k}.

On the strength of Proposition2.1, we have

j =i+1

Fs^j ⊥

Gn∨Fsⁱ⁺¹

Ftⁱ⁺¹.

Together with Proposition 6.6 of [17], we deduce Eν0

M_n(x_k, t )|Fsⁿ

=

n−1

i=0

Eν₀

Nⁱ⁺¹(t)|Gn∨Fsⁱ⁺¹

−Eν₀

t 0

1_{S_i+1≥u}λ(xk, u)duGn∨Fsⁱ⁺¹

1_{Z_i=x_k}.

Furthermore, from Proposition2.1, we have Ftⁱ⁺¹ ⊥

σ (Z_i)∨Fsⁱ⁺¹

Gn.

Thus, in the light of Proposition 6.6 of [17] again, we have

Eν₀

Mn(xk, t )|Fsⁿ

=

n−1

i=0

Eν₀

Nⁱ⁺¹(t)|σ (Zi)∨Fsⁱ⁺¹

1_{Z_i=x_k}

−Eν₀

t 0

1_{S_i+1≥u}λ(xk, u)duσ (Zi)∨Fsⁱ⁺¹

1_{Z_i=x_k}

.

By Lemma2.2, this yields to

Eν0

M_n(x_k, t )|F_sⁿ

=

n−1

i=0

Mⁱ⁺¹(s)1_{Zi=xk}.

Finally, together with (11),Mn(xk,·)is, therefore, a martingale.

Theorem3.1states that one may estimate the cumulative rateΛ(x_k, t )with the Nelson–Aalen estimatorΛ_n(x_k, t ) given by

Λn(xk, t )=

n−1

i=0

1

Y_n(x_k, S_i+1)1_{S_i+1≤t}1_{Z_i=x_k}.

One refers the interested reader to [2,3] or [5] for a general survey on the properties of this estimator.

4. Continuous state space

The present section is divided into two parts. In the first one, we provide an estimate of the cumulative rateΛ. The second part deals with the estimation of the jump rateλby smoothing the estimator ofΛby kernel methods.

4.1. Estimation ofΛ

Let us assume that the transition kernel Q is diffuse, that is to say, Q does not charge singletons. The previous procedure is ruled out, since for any x∈E and each integer i,1_{Zi=x}=0 almost surely. As a consequence, we shall naturally approximate under regularity conditions the jump rate inx, by the jump rate given the state is in a neighborhood ofx. As mentioned in theIntroduction, we shall consider the counting processN_n(A, t)defined for A∈B(E)by

Nn(A, t)=

n−1

i=0

1_{Z_i∈A}1_{S_i+1≤t}.

We will see (23) that the stochastic intensity ofN_n(A, t)in a well-chosen filtration is

n−1

i=0

1_{Zi∈A}1_{Si+1≥t}λ(Z_i, t ).

The first results of this section deal with the ergodicity of the underlying Markov chains. Their properties are prominent to establish the asymptotic behavior of the stochastic intensity ofNn(A, t)(see Proposition 4.12). Based on these results, we will study in Proposition4.16and Theorem4.17the estimation ofL(A, t )=t

0l(A, s)ds, wherel(A, t ) is an approximation of the jump rateλ(x, t ), withx inA. We will deduce from this an estimator of the cumulative rateΛ(x, t )(see Theorem4.23). An additional difficulty is related to the invariant measure ofA, which we have to estimate.

We shall impose some assumptions about both the ergodicity of the Markov chain(Z_n)_n≥0and the characteristics of the process. In the following,ν_ndenotes the distribution ofZ_n, for each integern.

Assumption 4.1. There exists a probability measureνsuch that,for any initial distributionν0=δ_{x},x∈E,

n→+∞lim ν_n−νTV=0.

This assumption may be directly related to the transition kernel of the Markov chain(Z_n)_n≥0(existence of a Foster–

Lyapunov’s function or Doeblin’s condition for instance). We refer the interested reader to [22] for results about this kind of connection. This assumption leads to the following results.

Proposition 4.2. We have the following statements:

(1) (Z_n)_n≥0isν-irreducible.

(2) (Z_n)_n≥0is positive Harris-recurrent and aperiodic.

(3) νis the unique invariant probability measure of(Z_n)_n≥0.

Proof. The proof may be found in AppendixB.

Denote by η_n the distribution of the couple(Z_n, S_n+1), and by μ_z(·)the conditional law of S1 givenZ0=z.

By construction (2), the distribution ofS_n+1givenZ_n=zis also given byμ_z(·). The following result gives us the probability measure of(Z_n, S_n+1)according to(μ_z)_z∈E andν_n.

Lemma 4.3. For each integern,ηnsatisfies,for anyA×Γ ∈B(E)⊗B(R₊), ηn(A×Γ )=

A×Γ

μz(ds)νn(dz).

Proof. This is the disintegration of the measureη_naccording to its marginal distributionνn. We shall see that the sequence(η_n)_n≥0converges to the probability measureηgiven by,

∀A×Γ ∈B(E)⊗B(R₊), η(A×Γ )=

A×Γ

μz(ds)ν(dz).

Lemma 4.4. For any initial distributionν0=δ_{x},x∈E,

n→+∞lim ηn−ηTV=0.

Proof. The proof is given in AppendixB.

As a consequence, the Markov chain(Z_n, S_n+1)_n≥0has similar properties of the Markov chain(Z_n)_n≥0given in Proposition4.2.

Proposition 4.5. We have the following statements:

(1) (Z_n, S_n+1)_n≥0isη-irreducible.

(2) (Z_n, S_n+1)_n≥0is positive Harris-recurrent and aperiodic.

(3) ηis the unique(up to a multiple constant)invariant probability measure of the Markov chain(Z_n, S_n+1)_n≥0. Proof. One may state this result from Lemma4.4, with the arguments given in the proof of Proposition4.2.

According to the previous discussion, we shall apply the ergodic theorem to the Markov chains (Z_n)_n≥0 and (Z_n, S_n+1)_n≥0. Now, we impose some assumptions on the characteristics of the process.

Assumptions 4.6.

(1) The jump rateλis uniformly Lipschitz,that is,

∃[λ]Lip>0,∀ξ, ξ∈E,∀s≥0, λ(ξ, s)−λ

ξ, s≤ [λ]Lipd ξ, ξ

.

(2) There exists a locally integrable functionM:R₊→R₊such that,

∀ξ∈E,∀s≥0, λ(ξ, s)≤M(s).

(3) The densityf is continuous in time.

(4) f is bounded.

(5) The functiontis continuous.

Under these assumptions, one states some intermediate results aboutλ,tandG.

Lemma 4.7. LetA∈B(E)be a relatively compact set such thatA∩∂E=∅.Thus,

ξinf∈At(ξ ) >0.

In this case,one denotest(A)=infξ∈At(ξ ).Furthermore,

∀t≥0, inf

ξ∈AG(ξ, t ) >0.

Proof. The proof is deferred in AppendixB.

Let us introduce the following notation.

B⁺ν =

A∈B(E): Arelatively compact, ν(A) >0 andA∩∂E=∅ .

The following lemma states that, for anyA∈Bν⁺,λis bounded onA× [0, t(A)[.

Lemma 4.8. LetA∈B⁺_ν,ξ∈Aand0≤t < t(A).Thus, λ(ξ, t )≤ f_∞

infξ∈AG(ξ, t(A)).

Proof. Asf is bounded andG(ξ,·)is decreasing, we obtain by Lemma4.7, λ(ξ, t )≤ f∞

G(ξ, t ) ≤ f∞

G(ξ, t(A)).

This immediately yields to the expected result.

LetA∈B⁺ν. Let us consider for each integernthe continuous-time processY_n(A,·), defined by,

∀0≤t < t(A), Y_n(A, t)=

n−1

i=0

1_{Si+1≥t}1_{Zi∈A}. (12)

We shall state two lemmas about the asymptotic behavior ofY_n. Before, we focus our attention on the link between G(z,·)andμ_z(·), for anyz∈A.

Remark 4.9. LetA∈B⁺ν andz∈A.For any0≤t < t(A),we have μ_z

[t,+∞[

=Pν0(S₁≥t|Z₀=z)

=Pν₀(S₁> t|Z₀=z), becauset < t(A)≤t(z).Thus,

μ_z

[t,+∞[

=G(z, t ).

Lemma 4.10. LetA∈Bν⁺andx∈E.Thus,for any0≤t < t(A), Yn(A, t)

n −→

A

G(z, t )ν(dz) Px-a.s.asn→ +∞. Furthermore,this limit is strictly positive.

Proof. In the light of Proposition4.5, the Markov chain(Z_n, S_n+1)_n≥0 is positive Harris-recurrent and admitsηas its unique invariant probability measure. Thus, using the ergodic theorem (see for instance Theorem 17.1.7 of [22]), we have

Yn(A, t)

n →

A

μ_z

[t,+∞[

ν(dz) Px-a.s. asn→ +∞. Furthermore, for anyz∈A, ast < t(A)and according to Remark4.9,

μ_z

[t,+∞[

=G(z, t ).

It is a strictly positive number becauseν(A) >0 and infξ∈AG(ξ, t ) >0 by Lemma4.7.

LetA∈B_ν⁺and 0≤t < t(A). Let us introduce the generalized inverseY_n(A, t)⁺ofY_n(A, t)by Yn(A, t)⁺=

0 ifYn(A, t)=0,

1

Y_n(A,t ) else. (13)

Lemma 4.11. LetA∈B⁺ν, 0≤t < t(A)andx∈E.Thus,for all integersn,

Y_n(A, t)⁺≤1 Px-a.s. (14)

and,asngoes to infinity,

Yn(A, t)⁺−→0 Px-a.s., (15)

1_{Yn(A,t )=0}−→0 Px-a.s., (16)

t 0

1_{Yn(A,s)=0}ds−→0 Px-a.s.

Proof. Y_n(A, t)⁺ is almost surely bounded by 1, since Y_n(A, t) takes its values on the integers. One immedi- ately obtains the limits (15) and (16), because Yn(A, t)/n almost surely admits a strictly positive limit by virtue of Lemma4.10. Finally,

lim sup

n→+∞

_t

0

1_{Yn(A,s)=0}ds≤ _t

0

lim sup

n→+∞1_{Yn(A,s)=0}ds=0,

by (16).

In the following proposition, we shall apply the ergodic theorem in order to define, for anyA∈Bν⁺, the function l(A,·)which is an approximation of the jump rateλ(ξ,·), forξ∈A(see Lemma4.20). We also state the continuity of this function.

Proposition 4.12. LetA∈Bν⁺, 0≤t < t(A)andx∈E.Thus,whenngoes to infinity,

Y_n(A, t)⁺

n−1

i=0

λ(Z_i, t )1_{Zi∈A}1_{Si+1≥t}−→l(A, t )=

Af (z, t )ν(dz)

AG(z, t )ν(dz) Px-a.s. (17)

The functionl(A,·)is continuous on[0, t(A)[.We especially have K_t(A)= sup

0≤s≤t

l(A, s)<+∞. (18)

Proof. The ergodic theorem (Theorem 17.1.7 of [22]) applied to(Z_n, S_n+1)_n≥0leads to

n→+∞lim 1 n

n−1

i=0

λ(Z_i, t )1_{Z_i∈A}1_{S_i+1≥t}=

A

λ(z, t )μ_z

[t,+∞[

ν(dz) Px-a.s.

=

A

λ(z, t )G(z, t )ν(dz) Px-a.s., becauset < t(A). Notice thatf (z, t )=λ(z, t )G(z, t ). Thus,

n→+∞lim 1 n

n−1

i=0

λ(Z_i, t )1_{Zi∈A}1_{Si+1≥t}=

A

f (z, t )ν(dz) Px-a.s.

Furthermore, in the light of Lemma4.10,

n→+∞lim 1

nYn(A, t)=

A

G(z, t )ν(dz) Px-a.s.

Finally, doing the ratio of these two limits leads to the expected convergence. On the strength of Lebesgue’s theorem of continuity under the integral sign, l(A,·)is a continuous function, since f andG are continuous in time and

bounded.

Having established the asymptotic behavior ofY_n, we focus on continuous-time martingales. In particular, thanks to Lenglart’s inequality for continuous-time martingales, we shall estimate, for anyA∈B⁺ν, the functionsl(A,·)and L(A,·), whereL(A,·)is given by,

∀0≤t < t(A), L(A, t )= _t

0

l(A, s)ds. (19)

Theorem 4.13. LetA∈B_ν⁺.For each integern,the processM_n(A,·)defined by,

∀0≤t < t(A), M_n(A, t)=

n−1

i=0

Mⁱ⁺¹(t)1_{Zi∈A}, (20)

is a continuous-time martingale in the filtration(Gn∨_n−1

i=0Ftⁱ⁺¹)_{0≤t <t}(A). Proof. Let 0≤s < t < t(A). Then,

Eν0

M_n(A, t)Gn∨

n−1 i=0

F_sⁱ⁺¹

=

n−1

i=0

Eν0

Mⁱ⁺¹(t)1_{Zi∈A}Gn∨

n−1 j=0

Fs^j+1

.

Moreover, on the strength of Proposition2.1, we have

j =i+1

Fs^j ⊥

Gⁿ∨Fsⁱ⁺¹

Ftⁱ⁺¹ and Ftⁱ⁺¹ ⊥

σ (Z_i)∨Fsⁱ⁺¹

Gn.

Therefore, sinceσ (Z_i)is a sub-σ-field ofGn, we have by Corollary 6.8 of [17],

j =i+1

Fs^j ⊥

Gn∨Fsⁱ⁺¹

Ftⁱ⁺¹∨σ (Z_i) and σ (Z_i)∨Ftⁱ⁺¹ ⊥

σ (Zi)∨Fsⁱ⁺¹

Gn.

Thus, asMⁱ⁺¹(t)1_{Zi∈A}isσ (Z_i)∨Ftⁱ⁺¹-measurable,

Eν0

M_n(A, t)Gn∨

n−1 i=0

Fsⁱ⁺¹

=

n−1

i=0

Eν0

Mⁱ⁺¹(t)1_{Zi∈A}|Gn∨Fsⁱ⁺¹

=

n−1

i=0

Eν0

Mⁱ⁺¹(t)1_{Zi∈A}|σ (Z_i)∨Fsⁱ⁺¹

.

Furthermore, with Lemma2.2, Eν0

Mⁱ⁺¹(t)1_{Zi∈A}|σ (Z_i)∨Fsⁱ⁺¹

=Mⁱ⁺¹(s)1_{Zi∈A}.

Thus,

Eν₀

Mn(A, t)Gn∨

n−1 i=0

Fsⁱ⁺¹

=

n−1

i=0

Mⁱ⁺¹(s)1_{Z_i∈A}.

By (20), this ensures thatMn(A,·)is a martingale.

For anyA∈Bν⁺, let us introduce the counting processNn(A,·), defined for any 0≤t < t(A)by

N_n(A, t)=

n−1

i=0

1_{S_i+1≤t}1_{Z_i∈A}. (21)

Lemma 4.14. LetA∈B⁺_ν.For all integersn,the process given for all0≤s < t(A)by Mn(A, s)=

_s

0

Yn(A, u)⁺dMn(A, u), (22)

is a martingale whose predictable variation processM_n(A)satisfies for anyx∈E,

∀0≤s < t(A), Mn(A)

(s)→0 Px-a.s.asn→ +∞.

Proof. By (7), (20) and (21), for any 0≤t < t(A), one may differently writeMn(A, t),

M_n(A, t)=N_n(A, t)− _t

0 n−1

i=0

1_{S_i+1≥u}1_{Z_i∈A}λ(Z_i, u)du. (23)

In the light of Theorem4.13, this is a continuous-time martingale. As a consequence, the processAn(A,·)given by,

∀0≤s < t(A), An(A, s)= _s

0 n−1

i=0

1_{Si+1≥u}1_{Zi∈A}λ(Z_i, u)du,

is the compensator of the counting processNn(A,·). In order to prove thatMn(A,·)is a martingale, one may only state that

Eν0

t 0

Y_n(A, s)⁺2

dAn(A, s)

<+∞.