Semi-Markov processes for reliability studies

SEMI-MARKOV PROCESSES FOR RELIABILITY STUDIES

CHRISTIANE COCOZZA-THIVENT AND MICHEL ROUSSIGNOL

Abstract. We study the evolution of a multi-component system which is modeled by a semi-Markov process. We give formulas for the avaibility and the reliability of the system. In the^r-positive case, we prove that the quasi-stationary probability on the working states is the normalised left eigenvector of some computable matrix and that the asymptotic failure rate is equal to the absolute value of the convergence parameter

r.

1. Introduction

The motivation of this paper comes from reliability studies. We consider a system whose possible states form a nite set

E

. The set

E

is splitted into two subsets, ^M corresponding to the working states and ^P corresponding to the failure states. We are interested in the time evolution of the system.

There is an important literature on this subject when the evolution is mod- eled by a Markov process (cf Pages and Gondran (1980)). However there are cases where the evolution cannot be modeled by a Markov process, but can be modeled by a semi-Markov process. In this case by standard tech- niques on semi-Markov processes (cf Cinlar (1975)), it is possible to obtain formulas for the availability and the reliability. Furthermore, in order to obtain a description of this evolution up to the rst failure time, it is inter- esting to compute the quasi-stationary distribution on^M. The existence of quasi-stationary distributions for semi-Markov processes has been proved in Cheong (1970). The main goal of this paper is to give a method to compute this distribution in the context of reliability studies. In addition we prove that the convergence parameter is equal to the asymptotic failure rate.

In section 2 we recall classical properties of semi-Markov processes and we give a formula to compute the Laplace transform of the availability. In section 3 we dene a transient semi-Markov process which allows to compute the Laplace transform of the reliability. A family

A

(

s

) of matrices appears in the formulas of availability and reliability. In the Markov case these matrices are equal to the generating matrix of the process. In section 4 we recall properties of

r

-recurrent and

r

-positive semi-Markov processes where

r

is the convergence parameter. If the process is

r

-recurrent, we prove that

r

is the Perron-Frobenius eigenvalue of the matrix

A

¹(

r

), restriction to^Mof the matrix

A

(

r

). In section 5 we prove that when the process is

r

-recurrent, with additional hypotheses, the process is

r

-positive and that the quasi-stationary distribution on ^M is the normalized left eigenvector of the matrix

A

¹(

r

).

URL address of the journal: http://www.emath.fr/ps/

Received by the journal October 20, 1995. Revised September 16, 1996 and January 14, 1997. Accepted for publication February 14, 1997.

c Societe de Mathematiques Appliquees et Industrielles. Typeset by L^ATEX.

This gives a practical method to compute the quasi-stationary distribution on^M. We prove in section 6 that under natural hypotheses, the asymptotic failure rate of the system is equal to ^j

r

^j. We give in section 7 a numerical example.

2. Some properties of a semi-Markov process

The semi-Markov process (

Y

^t)^{t 0} is dened by means of a random se- quence (

X

ⁿ

T

ⁿ)^n2Nwhere (

X

ⁿ)^n2Nrepresents the dierent positions of the process and (

T

ⁿ)^n2Nthe jump times. The process (

X

ⁿ)^n2Ntakes its values in a nite set

E

and (

T

ⁿ)^n2Nis an increasing sequence of positive random variables with

T

⁰ = 0. We have:

Y

^t=^X

n 0

X

ⁿ1^fTnt<Tn+1g

The random properties of the process are characterized by the following semi-Markov property. For any Borel set

A

of^R+, any state

j

in

E

and any

n

in ^N, we have:

P(

X

ⁿ⁺¹=

jT

^n+1;

T

ⁿ²

A=X

⁰

::: X

ⁿ

T

⁰

::: T

ⁿ)

= ^P(

X

ⁿ⁺¹=

jT

^n+1;

T

ⁿ²

A=X

ⁿ)

=

Z

A

Q

(

X

ⁿ

jdu

)

where

A

^!

Q

(

ijA

) is a bounded measure on ^R+ for all

i

and

j

in

E

. We shall use results on semi-Markov processes which can be found in Cinlar (1975). The process (

X

ⁿ)^{n 0} is a Markov chain with transition probabilities

P

(

ij

) =

Q

(

ij

^R+) =^RR+

Q

(

ijdu

). Let

f

(

idu

) be the probability law of

T

^n+1;

T

ⁿ conditionally on^f

X

ⁿ=

i

^gand

h

(

it

) =^R]t+1

f

(

idu

). We have:

f

(

idu

) =^X

j2E

Q

(

ijdu

)

Let

Q

⁽ⁿ⁾(

ijdu

) be the probability law of (

X

ⁿ

T

ⁿ) conditionally on^f

X

⁰ =

i

^g:

P(

X

ⁿ=

jT

ⁿ²

A=X

⁰=

i

) =

Z

A

Q

⁽ⁿ⁾(

ijdu

)

We have

Q

⁽⁰⁾(

ij:

) =

I

(

ij

)

⁰(

:

) where

I

is the identity matrix on

E

E

and

⁰ the Dirac measure on 0,

Q

⁽¹⁾(

ij:

) =

Q

(

ij:

),

Q

⁽ⁿ⁺¹⁾(

ik:

) =

P

j2E

Q

(

ij:

)

Q

⁽ⁿ⁾(

jk:

).

We dene the renewal Markovian kernel by:

R

(

ijdu

) = ^X

n 0

Q

⁽ⁿ⁾(

ijdu

)

:

We have:

P(

Y

^t=

j=Y

⁰=

i

) =

Z

0t]

h

(

jt

^;

s

)

R

(

ijds

)

:

We shall deal with Laplace transforms

Q

(

ijs

),

f

(

is

),

Q

⁽ⁿ⁾(

ijs

),

R

(

ij

s

),

h

(

is

),

P

(

ijs

) respectively of measures

Q

(

ijdu

),

f

(

idu

),

Q

⁽ⁿ⁾(

ij

du

),

R

(

ijdu

) and of functions

t

^!

h

(

it

),

t

^{! P}(

Y

^t =

j=Y

⁰ =

i

). Let

Q

(

s

),

Q

⁽ⁿ⁾(

s

),

R

(

s

),

P

(

s

) be the associated matrices. We have:

Q

⁽ⁿ⁾(

s

) = (

Q

(

s

))ⁿ

f

(

is

) =^X

j2E

Q

(

ijs

)

s h

(

is

) = 1^;

f

(

is

)

R

(

s

) = ^X

n 0

(

Q

(

s

))ⁿ

P

(

ijs

) =

R

(

ijs

)

h

(

js

)

Since the measures

Q

(

ijdu

)are bounded, the Laplace transforms

Q

(

ijs

),

Q

⁽ⁿ⁾(

ijs

) and

f

(

is

) are nite for

s

non negative.

We get the following result.

Proposition 2.1. Let us dene the matrix

A

(

s

) on

E

E

by

A

(

ijs

) =

Q

(

ijs

)

h

(

is

)

if i

⁶=

j A

(

iis

) =^{; X}

fj:j6=ig

Q

(

ijs

)

h

(

is

)

If

s

is strictly positive, the Laplace transform of the transition probabilities matrix is given by:

P

(

s

) = (

sI

^;

A

(

s

))^;1

Proof. We know that:

P

(

kjs

) =

R

(

kjs

)

h

(

js

). Replacing

R

(

kjs

) by ^{Ph(k js)(js)} in the equation

R

=

I

+

R

Q

gives:

P

(

ijs

)

h

(

js

) =

I

(

ij

)+^X

k 2E

P

(

iks

)

h

(

ks

)

Q

(

kjs

) The relation

h

(

is

) = ^1;fs(is) can be written:

h

(1

js

) =

s

+

f

(

js

)

h

(

js

) =

s

+^X

k 2E

Q

(

jks

)

h

(

js

) So we get:

sP

(

ijs

)+

P

(

ijs

)^X

k 2E

Q

(

jks

)

h

(

js

) =

I

(

ij

)+^X

k 2E

P

(

iks

)

Q

(

kjs

)

h

(

ks

) This relation is equivalent to:

P

(

s

)(

sI

^;

A

(

s

)) =

I

which is the desired result.

In the Markovian case, the matrices

A

(

s

) are constant and equal to the generating matrix of the Markov process. The formula above is a generali- sation of a well-known formula in the Markovian case.

3. The transient process

To compute the reliability of the system, we dene a new semi-Markov process (

Z

^t)^{t 0} which behaves exactly like the process (

Y

^t)^{t 0} until the rst failure time. However, when the process reaches a failure state, it stays in failure states forever. Let ~

Q

(

ijdu

) be the measures dening this new process, then we get:

for

i

in ^M: ~

Q

(

ijdu

) =

Q

(

ijdu

)

for

i

in ^P and

j

in ^M: ~

Q

(

ijdu

) = 0

for

i

in ^P and

j

in ^P: ~

Q

(

ijdu

) = anything.

Of course the new process is not irreducible in

E

.

From now on, we suppose that ^M is a transient irreducible set for the process (

Z

^t)^{t 0}.

We denote by \tilde" letters the quantities related to the process (

Z

^t)^{t 0}. Let

Q

¹(

s

) and

R

¹(

s

) be the restrictions of the matrices ~

Q

(

s

) and ~

R

(

s

) to

MM

It is easy to check that

8

ij

^2M: ~

A

(

ijs

) =

A

(

ijs

)

( ~

Q

(

s

))ⁿ(

ij

) = (

Q

¹(

s

))ⁿ(

ij

)

R

¹(

s

) = ^X

n 0

(

Q

¹(

s

))ⁿ

R

¹(

s

) =

I

¹+

R

¹(

s

)

Q

¹(

s

)

(3.1) where

I

¹ is the identity matrix on ^MM.

3.1. Some definitions

Let ~

r

(

ij

) be the convergence abscissa of the Laplace transform

Q

¹(

ijs

)

and

r

~= sup

ij

r

~(

ij

)

:

Remark 3.1. If there exist

i

and

j

(in ^M) such that

Q

¹(

ij

~

r

) = +¹, we shall say that we are in the usual case. The reason for this terminology is that this is the case for measures with rational Laplace transforms.

It was shown in Cheong (1968) that all the convergence abscissas of the Laplace transforms

R

¹(

ijs

) are equal (since the set^Mis irreducible). We denote this number by

r

.

Definition 3.2. The common convergence abscissa of the Laplace trans- forms

R

¹(

ijs

) is called the convergence parameter.

We get:

r

~

r

0

:

Let

(

s

) be the Perron-Frobenius eigenvalue of ~

Q

¹(

s

). This function en- ables us to compute

r

.

Remark 3.3.

1. The function

s

^!

(

s

) is decreasing (Seneta (1973)) and continuous (since the coecients of the matrix

Q

¹(

s

) are continuous) on the in- terval ]~

r

+¹.

2. If

(0) exists, then, since ^M is a transient irreducible set, the Perron Frobenius theorem gives

(0)

<

1.

3. If

(

s

) tends to a limit strictly greater than 1 as

s

decreases to ~

r

(which is true in the usual case - see remark 3.1 - ) and if

(0)

<

1 then the greatest value

s

such that

(

s

) = 1 is the convergence parameter

r

by formula (3.1).

3.2. Reliability

In order to compute the reliability of the system, we must compute the transition probabilities of the new process.

Proposition 3.4. Suppose that for

s > r

,

Q

(

ijs

)

<

¹ for all

i

in ^M and

j

in ^P. If

A

¹(

s

) is the restriction of the matrix

A

(

s

) to ^MM, then we have:

8

s > r P

^~1(

s

) = (

sI

^1;

A

¹(

s

))^;1

where ~

P

¹(

s

) is the restriction to ^MM of the Laplace transforms matrix of the transition probabilities for the new process.

Proof. For

s > r

we have

f

(

is

)

<

¹. It is easy to check that

f

(

is

)

<

¹ implies

h

(

is

)

<

¹. Hence we can prove proposition 3.4 in the same way as proposition 2.1.

Remark 3.5. Note that if

Q

(

ijdu

) =

P

(

ij

)

f

(

idu

), then for

s > r

,

f

(

is

)

<

¹ for all

i

in ^M, and

Q

(

ijs

)

<

¹for every

j

. So the assump- tion in the above proposition is satised.

Proposition 3.4 enables us to calculate the Laplace transform

RE

(

is

) of the reliability for the initial process starting from the working state

i

and the associated mean time to failure

MTTF

(

i

). Given that

RE

(

it

) =

P

j2M

P(

Z

^t=

j=Z

⁰=

i

) and

MTTF

(

i

) =

RE

(

i

0), we get:

RE

(

i

) = ^X

j2M

(

sI

^1;

A

¹(

s

))^;1(

ij

)

MTTF

(

i

) =^{; X}

j2M

(

A

¹(0))^;1(

ij

)

For

i

⁶=

j

,

A

¹(

ij

0) =

P

(

ij

)

=

^E(

T

¹

=Y

⁰ =

i

) and

A

¹(

ii

0) = (

P

(

ii

)^; 1)

=

^E(

T

¹

=Y

⁰=

i

). This proves the following result: since the

MTTF

depends only on the embedded markov chain (

X

ⁿ)^{n 0} and the mean sojourn time in the states, it is equal to the mean time to failure of a Markov process with the same transition probabilities

P

(

ij

) and the same mean sojourn time in each state.

4. The r-recurrence property Before introducing

r

-recurrence, we need some material.

For any state

j

, let ~

F

(

jj:

) be the law of the rst return time to

j

of the process (

Z

^t)^{t 0} starting from

j

for any state

i

dierent from

j

, let ~

F

(

ij:

) be the law of the rst visit time to

j

of the process (

Z

^t)^{t 0} starting from

i

. These laws are dened on ^R+S+¹. Because the process is irreducible and transient on ^M, the quantities ~

F

(

jj

^R+) are strictly less than 1 for

j

in ^M. If

i

is in ^P and

j

in ^M, ~

F

(

ij

^R+) equals 0. Let ~

F

(

ijs

) be the

Laplace transfom of ~

F

(

ij:

). Given that ~

R

(

ijs

) = ~

F

(

ijs

)~

R

(

jjs

) for

i

⁶=

j

and ~

R

(

jjs

) =^{Pn 0}( ~

F

(

jjs

))ⁿ (cf Cinlar (1975) 10.2.12 and 10.2.13), then for

s > r

and

j

in ^Mwe have:

F

~(

jjs

)

<

1

R

~(

jjs

) = 1

1^;

F

~(

jjs

)

R

~(

ijs

) =

F

~(

ijs

)

1^;

F

~(

jjs

)

if i

⁶=

j

Remark 4.1.

If lim^s&r

R

~(

jjs

) = +¹, then

lim

^s&r

F

^~(

jjs

) = ~

F

(

jjr

) = 1.

If lim^s&r

R

~(

jjs

)

<

+¹, then lim^s&r

F

~(

jjs

) = ~

F

(

jjr

)

<

1 so in any case ~

F

(

jjr

)1.

Lemma 4.2. For

s > r

and

i

,

j

in ^M, we have the relations:

F

~(

ijs

) = ^X

k 2M

Q

~(

iks

)~

F

(

kjs

)+ ~

Q

(

ijs

)(1^;

F

~(

jjs

))

F

~(

ijs

) = ^X

k 2M

F

~(

iks

)~

Q

(

kjs

)) 1^;

F

~(

jjs

) 1^;

F

~(

kks

) + ~

Q

(

ijs

)(1^;

F

~(

jjs

))

Proof. If

i

is dierent from

j

, replace ~

R

(

jjs

) by 1

=

(1^;

F

~(

jjs

)) and

R

~(

ijs

) by ~

F

(

ijs

)

=

(1^;

F

~(

jjs

)) in formulas ~

R

(

s

) =

I

+ ~

Q

(

s

) ~

R

(

s

) and ~

R

(

s

) =

I

+ ~

R

(

s

) ~

Q

(

s

).

Remark 4.3. We can deduce from the preceeding lemma and the irre- ducibility assumption that for

s

r

and

i

,

j

in ^M, we have:

F

~(

ijs

)

<

¹

and Q

^~(

ijs

)

<

¹

:

The notion of

r

-recurrence for a semi-Markov process was dened in Cheong (1968). It is known that if for at least one state

j

in ^Mwe have

lim

s&r

R

¹(

jjs

) =

R

¹(

jjr

) = +¹ then this property applies for any state

j

in ^M.

Definition 4.4. The process is said to be

r

-recurrent on ^Mif for at least one state

j

in ^Mwe have

lim

s&r

R

¹(

jjs

) =

R

¹(

jjr

) = +¹

Remark 4.5. The equality

R

¹(

jjr

) = +¹ is equivalent to the equality

F

~(

jjr

) = 1 for

j

^2M(cf remark 4.1).

Remark 4.6. Note that since,

R

¹(

jjr

) = ^{Pn 0}(

Q

¹)ⁿ(

jjr

), the condi- tion

R

¹(

jjr

) = +¹ is equivalent to

(

r

)1.

Remark 4.7. We have seen that in the usual case the Perron-Frobenius eigenvalue

(

r

) of

Q

¹(

r

) is equal to 1. In that case the process is

r

-recurrent.

Remark 4.8. Since the semi-Markov process (

Z

^t)^{t 0} is transient and irre- ducible on ^M, we have for

j

in ^M, ~

F

(

jj

0)

<

1. Thus if the process is

r

-recurrent,

r

is necessarily strictly negative.

In order to nd other properties of the value

r

, we need a technical lemma.

Lemma 4.9. If the process is r-recurrent we have for

i

and

j

in ^M:

F

~(

ijr

) = ^X

k 2M

Q

~(

ikr

)~

F

(

kjr

)

F

~(

ijr

)

( ~

F

)⁰(

jjr

) =

X

k 2M

F

~(

ikr

)

( ~

F

)⁰(

kkr

)

Q

~(

kjr

)

Proof. We let

s

tend to

r

in the rst formula of lemma 4.2 and we obtain, using remark 4.1, for

ij

in ^M:

F

~(

ijr

) = ^X

k 2M

Q

~(

ikr

)~

F

(

kjr

))

:

Now multiply the second formula of lemma 4.2 by (

s

^;

r

)

=

(1^;

F

~(

jjs

)) and take the limit as

s

decreases to

r

. The limit of (1^;

F

~(

jjs

))

=

(

s

^;

r

) = ( ~

F

(

jjr

)^;

F

~(

jjs

))

=

(

s

^;

r

) always exists as

s

decreases to

r

and is equal to ^;( ~

F

)⁰(

jjr

) = ^R01

xe

^{;r x}

F

^~(

jjdx

) (use the dominated convergence theorem or Fatou's lemma). This quantity is nite or innite. In either case we obtain the last formula.

Proposition 4.10. If the process is r-recurrent, then

r

is the Perron-Frobe- nius eigenvalue of

A

¹(

r

).

Proof. For

j

and

`

in ^Mwe have:

X

i2M

A

¹(

`ir

)~

F

(

ijr

)

= ^X

fi2M=i6=`g

Q

(

`ir

)

h

(

`r

)

F

~(

ijr

)^{; X}

fi2E=i6=`g

Q

(

`ir

)

h

(

`r

)

F

~(

`jr

)

= ^X

i2M

Q

(

`ir

)

h

(

`r

)

F

~(

ijr

)^;X

i2E

Q

(

`ir

)

h

(

`r

)

F

~(

`jr

) Using lemma 4.9, we obtain:

X

i2M

A

¹(

lir

)~

F

(

ijr

) = ~

F

(

ljr

)1^;

f

(

lr

)

h

(

lr

)

=

r F

^~(

ljr

)

:

Therefore

r

is an eigenvalue of

A

¹(

r

) associated with the positive vector ( ~

F

(

ijr

)

i

^2M). By the subinvariance theorem (cf Seneta (1973) p.23),

r

is the Perron-Frobenius eigenvalue of

A

¹(

r

).

In the following sections, we shall use the renewal theorem to prove the existence of limits. The next proposition gives the version of that theorem we shall use. We say that a nite measure is spread out if, for some

n

, its

n

^;

th

convolution power has a component which has a density with respect to the Lebesgue measure (cf Asmussen (1992) p.140).

Proposition 4.11. Suppose that for any

j

in ^M the measures ~

F

(

jjdu

) are spread out and that the process is r-recurrent. If

g

is a positive function on ^R+, such that the function

x

^!

e

^{;r x}

g

(

x

) is bounded, tends to 0 as

x

tends to +¹ and is Lebesgue integrable, then for any

i

and

j

in^Mwe have:

lim

t!1

e

^{;r tZ t}

0

g

(

t

^;

s

)

R

(

ijds

) =

g

(

r

) ~

F

(

ijr

)

;( ~

F

)⁰(

jjr

) (where

g

(

s

) is the Laplace transform of the function

g

).

This theorem can be deduced from the renewal theorem given in Asmussen (1992) corollary VI.1.3.

The limit in the above theorem is strictly positive if the quantity ( ~

F

)⁰(

jj r

) is nite. It was proved in Cheong (1968) that if for at least one

j

in ^M the quantity ( ~

F

)⁰(

jjr

) is nite, then it is nite for any

j

in ^M.

Definition 4.12. The process is said to be

r

-positive if it is

r

-recurrent and if

( ~

F

)⁰(

jjr

) =^{;Z 1}

0

xe

^{;r x}

F

^~(

jjdx

) is nite for any

j

in ^M.

In the following proposition we consider a process which is not

r

-recurrent.

Proposition 4.13. Suppose that the process is not r-recurrent. If

g

is a positive function on ^R+, such that the function

x

^!

e

^{;r x}

g

(

x

) is bounded, tends to 0 as

x

tends to +¹ and is Lebesgue integrable, then for any

i

and

j

in ^M we have:

lim

t!1

e

^{;r tZ t}

0

g

(

t

^;

s

) ~

R

(

ijds

) = 0 Proof. For

i

⁶=

j

in ^Mwe have:

e

^{;r tZ t}

0

g

(

t

^;

s

) ~

R

¹(

ijds

) =

Z

t

0

e

^{;r (t;s)}( ~

F

(

ij:

)

g

(

:

))(

t

^;

s

)

e

^{;r s}

R

^~1(

jjds

)

:

Since the process is not

r

-recurrent, the measure

e

^{;r s}

R

¹(

jjds

) is nite on

R

+. The function

t

^!

e

^{;r (t;s)}( ~

F

(

ij:

)

g

(

:

))(

t

^;

s

) tends to 0 as

t

tends to +¹and is bounded uniformly in

s

. It is sucient now to use the dominated convergence theorem.

The proof for the case

i

=

j

is similar.

5. Quasi-stationary probability on working states A quasi-stationary probability on ^Mis given by :

lim

t!1

P(

Z

^t=

j=Z

⁰=

i

)

P

j2M

P(

Z

^t=

j=Z

⁰=

i

)

:

The existence of the quasi-stationary probability on^Mis proved in Cheong (1970) if the process is r-positive and if the total variations in 0

¹) of the functions

t

^!

e

^{;r t}

h

(

it

) are nite. In this section we rst prove the existence of the quasi-stationary distribution under slightly dierent conditions. Our main improvement on the results of Cheong (1970) is that we are able to identify the quasi-stationary distribution as a left eigenvalue of a computable

matrix. As we are on a nite space we also prove that under our assumptions, an

r

-recurrent process is

r

-positive.

For the next theorem, we will introduce the following assumption:

(

A

⁰)

8

<

:

1)

the process is r

^;

recurrent

2)⁸

j

^2M

F

^~(

jjdu

)

is spread out

3)⁸

i

^2M

⁸

j

^2P

Q

^~(

ijr

)

<

+¹

Remark 5.1. Since the process (

Z

^t)^{t 0} is irreducible on ^M, for all

j

in ^M there exists a path

i

¹

i

²

::: i

^k in ^Msuch that the process starting from j returns to j by this path with a strictly positive probability. If at least one of the probability laws

Q

(

i

^l;1

i

^l

du

) (1

l

k

+ 1

i

⁰ =

ji

^{k +1}=

j

) has a density with respect to the Lebesgue measure then the measure ~

F

(

jjdu

) is spread out. Then condition 2) in assumption (

A

⁰) is satised.

Remark 5.2. Suppose that

Q

(

ijdu

) is equal to

P

(

ij

)

f

(

idu

) for any

i

and

j

in

E

. We know that

Q

(

ijr

) is nite for any

i

and

j

in ^M(remark 4.3), consequently

f

(

ir

) is nite. In this case, condition 3) is always true.

Proposition 5.3. Under assumption (

A

⁰), for any

i

and

j

in ^Mwe have:

lim

t!1

e

^{;r tP}(

Z

^t=

j=Z

⁰=

i

) =

h

(

jr

) ~

F

(

ijr

)

;( ~

F

)⁰(

jjr

) Proof. We have:

P(

Z

^t=

j=Z

⁰=

i

) =

Z

t

0

h

(

jt

^;

s

) ~

R

(

ijds

)

:

We want to apply proposition 4.11 with

g

(

:

) =

h

(

j:

). Using the fact that

r

is strictly negative (remark 4.8), lemma 4.2 and the assumption (

A

⁰3), one gets :

e

^{;r u}

h

(

ju

)

Z

]u+1

e

^{;r s}

f

(

jds

) =

Z

]u+1

e

^{;r sX}

k 2E

Q

(

jkds

)

X

k 2E

Q

(

jkr

) =

f

(

jr

)

<

+¹

:

Thus the function

u

^!

e

^{;r u}

h

(

ju

) is bounded and tends to 0 as

u

tends to +¹. The Lebesgue integrability condition of the proposition 4.11 is equivalent to

h

(

jr

)

<

+¹, which is true since

f

(

jr

)

<

+¹.

Then under assumption (

A

⁰), if the process is not

r

-positive, we have for any

ij

in ^M:

lim

t!1

e

^{;r tP}(

Z

^t=

j=Z

⁰ =

i

) = 0

A direct application of proposition 4.13 implies that if the process is not

r

-recurrent we obtain the same limit. Then the limit is strictly positive only in the

r

-positive case.

Proposition 5.4. Under assumption (

A

⁰), the process is r-positive.

Proof. For

i

and

j

in ^M, let

'

^ij(

t

) =

e

^{;r tP}(

Z

^t=

j=Z

⁰ =

i

).

If the process is not

r

-positive we have for

ij

in ^M: lim

t!1

'

^ij(

t

) = 0

and hence lim

u&0

u '

^ij(

u

) = 0 (

'

^ij

Laplace transform of '

^ij)

:

Using lim^u&0

u'

^ij(

u

) = lim^s&r(

s

^;

r

) ~

P

(

ijs

) and proposition 3.4, we obtain:

lim

s&r

(

s

^;

r

)(

sI

^1;

A

¹(

s

))^;1(

ij

) = 0

:

Since

r

is a simple eigenvalue of

A

¹(

r

) by proposition 4.10, and since

s

^!

A

¹(

s

) is continuous, in a neighbourhood of

r

there exists a simple eigenvalue

(

s

) of

A

¹(

s

) such that: lim^s&r

(

s

) =

r

.

We can triangularise the matrix

A

¹(

s

) as

A

¹(

s

) = (

s

)

T

(

s

)^;1(

s

) and then (

sI

^1;

A

¹(

s

))^;1= (

s

)(

sI

^1;

T

(

s

))^;1;1(

s

). The matrix

T

(

s

) is lower triangular we can suppose that its element

T

(

s

)(1

1) is equal to

(

s

) and the other elements of its rst column are equal to 0. The rst column of the matrix (

s

) is equal to a right eigenvector

U

(

s

) of

A

¹(

s

) associated with

(

s

). The rst row of ^;1(

s

) is equal to a left eigenvector

V

(

s

) of

A

¹(

s

) associated with

(

s

). The element (

sI

^1;

T

(

s

))^;1(1

1) is equal to ^s;(s)1 , and we then get:

lim

s&r

(

s

^;

r

)(

sI

^1;

T

(

s

))^;1(1

1) = 1

:

For (

ij

)⁶= (1

1), (

sI

^1;

T

(

s

))^;1(

ij

) satisfy the following:

K

(

ij

)(

s

^;

(

s

))

Q

i(

s

^;

T

(

s

)(

ii

)) =

K

(

ij

)

Q

i6=1(

s

^;

T

(

s

)(

ii

))

:

Then for (

ij

)⁶= (1

1) we get:

lim

s&r

(

s

^;

r

)(

sI

^1;

T

(

s

))^;1(

ij

) = 0

:

On the other hand we have

lim

s&r

(

s

^;

r

)(

sI

^1;

A

¹(

s

))^;1(

ij

) =

U

(

ri

)

V

(

rj

)

:

But the quantity

U

(

ri

)

V

(

rj

) is non null for any

i

and

j

. So there is a contradiction and the process is

r

-positive.

Proposition 5.5. Under assumption (

A

⁰), for any

i

and

j

in ^Mwe have:

lim

t!1

P(

Z

^t=

j=Z

⁰=

i

)

P

j2M

P(

Z

^t=

j=Z

⁰=

i

) =

B

(

ij

)

P

j2M

B

(

ij

) with 0

< B

(

ij

) =

h

(

jr

) ~

F

(

ijr

)

;( ~

F

)⁰(

jjr

)

<

+¹

Proof. This is a direct application of proposition 5.3 and proposition 5.4.

We want to understand the structure of the matrix

B

(

ij

) on ^MM. Proposition 5.6. Under assumption (

A

⁰), for any

i

in ^M, the vector (

B

(

ij

)

j

^{2 M}) is a left eigenvector of the matrix

A

¹(

r

) associated with the eigenvalue

r

and for any

j

in ^M, the vector (

B

(

ij

)

i

^2M) is a right eigenvector of the matrix

A

¹(

r

) associated with the eigenvalue

r

.

Proof. Using lemma 4.2, we get for

l

and

j

in ^M:

X

i2M

B

(

li

)

A

¹(

ijr

) = ^X

fi2M=i6=jg

h

(

ir

)~

F

(

lir

)

Q

(

ijr

)

;( ~

F

)⁰(

iir

)

h

(

ir

)

; X

fi2E=i6=jg

Q

(

jir

)

h

(

jr

)

h

(

jr

)~

F

(

ljr

)

;( ~

F

)⁰(

jjr

)

=

F

~(

ljr

)

;( ~

F

)⁰(

jjr

)(1^;

f

(

jr

))

=

F

~(

ljr

)

;( ~

F

)⁰(

jjr

)

rh

(

jr

)

=

rB

(

lj

)

:

In the same way we get:

X

i2M

A

¹(

lir

)

B

(

ij

) = ^X

fi2M=i6=lg

Q

(

lir

)

h

(

jr

)~

F

(

ijr

)

;

h

(

lr

)(~

F

)⁰(

jjr

)

; X

fi2E=i6=lg

Q

(

lir

)

h

(

lr

)

h

(

jr

)~

F

(

ljr

)

;( ~

F

)⁰(

jjr

)

=

h

(

jr

)

;

h

(

lr

)(~

F

)⁰(

jjr

)

F

~(

ljr

)(1^;

f

(

lr

))

=

rB

(

lj

)

:

We have seen that

r

is the Perron-Frobenius eigenvalue of

A

¹(

r

) and we know that the eigenvector is unique up to scalar multiples. If

V

and

W

are left and right eigenvectors, we can immediately deduce that:

B

(

ij

) =

CW

(

i

)

V

(

j

). We then obtain the following theorem.

Theorem 5.7. Under assumption (

A

⁰), for any

i

and

j

in ^M: lim

t!1

P(

Z

^t=

j=Z

⁰ =

i

)

P

j2M

P(

Z

^t=

j=Z

⁰=

i

) =

V

(

j

)

P

j2M

V

(

j

) where

V

is a left eigenvector of

A

¹(

r

) associated with

r

.

6. Failure rate

An important quantity in reliability studies is the failure rate and espe- cially the asymptotic failure rate.

Definition 6.1. For any state

k

and any time

t

, let

(

kt

) be the failure rate of the system dened by :

(

kt

) = lim

&0

1

^P(

Z

^t+2P

=Z

^t2M

Z

⁰ =

k

) when this limit exists.

We can also write:

(

kt

) = 1

P

i2M

P(

Z

^t=

i=Z

⁰ =

k

)

X

i2M X

j2P

lim

&0

1

^P(

Z

^t+=

jZ

^t=

i=Z

⁰=

k

)

In this section, we suppose that the following assumptions are satised:

(

A

¹) :

8

>

>

>

>

<

>

>

>

>

:

1)

assumption

(

A

⁰)

2)⁸

i

^2M

⁸

j

^2P

Q

(

ijdu

) =

q

(

iju

)

du

3)⁸

i

^2M

⁸

j

^2P

u

^!

q

(

iju

)

is continuous a:e:

4)⁸

i

^2M

⁸

j

^2P

u

^!

q

(

iju

)

is bounded

5)⁸

i

^2M

⁸

j

^2P

lim^u!+1

e

^{;r u}

q

(

iju

) = 0

Proposition 6.2. Under assumption (

A

¹), we have for any

k

in^M and

t

in ^R+:

(

kt

) = 1

P

i2M

P(

Z

^t=

i=Z

⁰=

k

)

X

i2M X

j2P Z

t

0

q

(

ijt

^;

s

) ~

R

(

kids

)

:

Proof. For

i

in ^Mand

j

in ^P we get:

P(

Z

^t+=

jZ

^t=

i=Z

⁰ =

k

) =

X

n 0

P(~

T

ⁿ

t < T

^~n+1

t

+

< T

^~n+2

X

^~n=

i X

^~n+1=

j= X

^~0=

k

)

+^X

n 0 X

l

1 2E

X

l

2 2E

P(~

T

ⁿ

t < T

^~n+1

< T

^~n+2

< t

+

X

^~n=

i X

^~n+1=

l

¹

X

~ⁿ⁺²=

l

²

Z

^t+ =

j= X

^~0=

k

)

:

The last term is less than:

X

n 0 X

l

1 2E

X

l

2 2E

Z

fu

1 t<u

1 +u

2

<u

1 +u

2 +u

3

<t+ g

Q

~ⁿ(

kidu

¹) ~

Q

(

il

¹

du

²) ~

Q

(

l

¹

l

²

du

³)

X

l

1 2E

X

l

2 2E

Z

0t]

R

~(

kidu

¹)

Z

ft;u

1

<u

2 u

2 +u

3

<t;u

1 + g

Q

~(

il

¹

du

²) ~

Q

(

l

¹

l

²

du

³)

o

(

)

:

Elsewhere:

X

n 0

P(~

T

ⁿ

t < T

^~n+1

t

+

< T

^~n+2

X

^~n=

i X

^~n+1=

j= X

^~0=

k

)

= ^X

n 0 Z

(]0t]

Q

~ⁿ(

kids

)^P( ~

X

¹=

jt

^;

s < T

^~1

t

^;

s

+

< T

^~2

= X

^~0=

i

)

= ^X

n 0 Z

0t]

Q

~ⁿ(

kids

)^Z

]t;st;s+ ]

Q

~(

ijdu

¹)^X

l Z

ft;s+ <u

1 +u

2 g

Q

~(

jldu

²)

= ^X

l Z

0t]

R

~(

kids

)

Z

Q

~(

jldu

²)

Z

ft;s<u1t;s+ <u1+u2g

q

(

iju

¹)

du

¹

:

Since

q

(

iju

) is a.e. continuous, we have

lim

!0

1

Z

1ft;s<u1t;s+ <u1+u2g

q

(

iju

¹)

du

¹

= lim

!0

1

Z

1^{ft;s<u1t;s+ g}

q

(

iju

¹)

du

¹

=

q

(

ijt

^;

s

)

a:s:

We can conclude with the dominated convergence theorem.

We want to identify the asymptotic failure rate, that is the limit, if it exists, of

(

kt

) when

t

tends to +¹.

Theorem 6.3. Under assumption (

A

¹), we have:

lim

t!1

(

kt

) =^j

r

^j

:

Proof. The conditions of theorem 5.7 are fullled and we know that:

lim

t!1

e

^{;r tP}(

Z

^t=

i=Z

⁰=

k

) =

CW

(

k

)

V

(

i

)

:

Now apply proposition 4.11 with the function

g

equal to the function

u

^!

q

(

iju

) with

i

in ^Mand

j

in ^P. We then obtain for

k

in ^M:

lim

t!1

e

^{;r tZ t}

0

q

(

ijt

^;

s

) ~

R

(

kids

)) =

q

(

ijr

)~

F

(

kir

)

;( ~

F

)⁰(

iir

)

=

CW

(

k

)

V

(

i

)

A

(

ijr

)

for i

⁶=

j:

and lim

t!1

(

kt

) = 1

P

i2M

CW

(

k

)

V

(

i

)

X

i2M X

j2P

CW

(

k

)

V

(

i

)

A

(

ijr

)

= 1

P

i2M

V

(

i

)

X

i2M

V

(

i

)(^X

j2P

A

(

ijr

))

= ^; 1

P

i2M

V

(

i

)

X

i2M

V

(

i

)(^X

j2M

A

(

ijr

))

= ^; 1

P

i2M

V

(

i

)

X

j2M

rV

(

j

)

= ^j

r

^j

:

So we nd for the semi-Markov process that the same result as for the Markov process holds.

7. A numerical example: a two-unit parallel system with sequential preventive maintenance

7.1. The reliability model

A semi-Markov reliability model of a two-unit parallel system with se- quential preventive maintenance (PM) will now be considered. This model has been used as an example in Alam (1984) and Csenki (1995).

The system consists of two units, A and B. The model has nine states.

The states and the transitions are shown in Fig. 1.

Preventive maintenance is carried out o-line on A and B alternately.

The unit which is due for PM is removed from the system after

c

hours of parallel service and returned to service after a random time of maintenance.

Thus, until failure occurs, states 1, 2, 3 and 4 are visited in this order.

In states 1 and 3 the units A and B are up. In state 2 unit A is under preventive maintenance and unit B is up. In state 4 unit A is up and unit B is under preventive maintenance. This implies that from state 1 the next PM is on unit A and not B. The process makes a transition from state 1

to state 6 if unit B fails during a sojourn in state 1. In state 6, unit A is in service while unit B undergoes repair. There are two possible transitions from state 6. If unit A fails before B's repair is completed, system failure ensues by transition to state 9 where the two units are down. The other possible move from state 6 is back to state 1 which happens if A remains in service throughout B's repair. From state 1, the only transition hitherto not considered is that of state 5 which occurs if unit A fails within the projected

c

hours of parallel service with unit B still being in the up state. Since it is assumed that completed repair includes PM, the system enters state 3 (the next projected PM is on unit B) upon successful completion of the repair on A in state 5. State 8 can be entered from state 2 only. This happens if unit B fails while unit A is under PM. The system then resides in state 8, after which unit A enters service and repair is started on unit B (state 6).

2

A PM, B up

1 A up, B up

3 A up, B up

4 A up, B PM

6 B downA up

5 A downB up

9 A downB down

A PM, B down8

A down, B PM7

6

?

-

;

;

;

;

;

;

;

;

;

;

;

;

@

@

@

@

@

@

@ R

@

@

@

@

@

@

@ I A

A

A

A

A

A

A

A

A

A

A

A

A

A

A U

@

@

@

@

@

@

@ I

@

@

@

@

@

@

@ R

A A A A A A A A A A A A A A A K

6

? 6

?

PM : preventive maintenance Fig. 1

Departure from state 9 happens as soon as the repair on any one of the two units is completed. The remaining aspects of the system are obtained by interchanging the roles of A and B. In this sense, 3, 4, 5 and 7 correspond to 1, 2, 6 and 8. We assume that all the random variables corresponding