Data analysis and stochastic modeling

(1)

Data analysis and stochastic modeling

Lecture 7 – An introduction to queueing theory

Guillaume Gravier

guillaume.gravier@irisa.fr

with a lot of help from Paul Jensen’s course

http://www.me.utexas.edu/ jensen/ORMM/instruction/powerpoint/or_models_09/14_queuing.ppt

Data analysis and stochastic modeling – Queueing theory – p.

Where are we?

1. A gentle introduction to probability 2. Data analysis

3. Cluster analysis

4. Estimation theory and practice 5. Mixture models

6. Random processes 7. Queing systems

◦ discrete time Markov chains revisited

◦ continuous time processes

◦ queueing systems 8. Hypothesis testing

What are we gonna talk about today?

◦ Another look at discrete time Markov chains

◦ Continuous time processes

→ continuous time Markov chains

→ Birth and death processes

◦ Queueing theory

→ what is a queueing system?

→ M/M* queues

→ other types of queues and queue networks

Typology of random processes

A process is a collection of random variables

{X (t)|t ∈ T }

^over

an index set

T

^where

X(t)

take values in a state space

I

^.

T

^is

discrete continuous discrete discrete-time continuous-time

chain chain

I

^is ^continuous discrete-time continuous-time continuous-state continuous-state

process process

(2)

Markov property

A stochastic process

{X (t)|t ∈ T }

is a Markov process if, for any

t

0

< t

1

< . . . < t

n

< t

, the conditional distribution of

X (t)

^given

X (t

0

), . . . , X(t

n

)

depends only on

X (t

n

)

^{, i.e.}

Andreï A. Markov 1856–1922

P[X(t)≤x|X(tn)≤xn, . . . , X(t0)≤t0] =P[X(t)≤x|X(tn)≤xn]

A Markov chain is said to be homogeneous if

P[X(t)≤x|X(tn)≤xn] =P[X(t−tn)≤x|X(0)≤xn] .

Discrete time Markov chains (DTMC)

[slight change of notations from previous lecture]

◦ Markov property

P[Xn=xn|X0=x0, . . . , Xn−1=xn−1] =P[Xn=xn|Xn−1=xn−1]

◦ We want to study the following quantities(for homogeneous chains only)

⊲ state of the process at step

n

^:

p

i

(n) = P [X

n

= i]

⊲

n

-step transition probability:

p

ij

(n) = P [X

m+n

= j|X

m

= i]

◦ The process is entirely defined by

⊲ the initial distribution:

p (0) = [p

0

(0), p

1

(0), . . .]

⊲ the transition matrix:

P = [p

ij

= p

ij

(1) = P [X

n

= j|X

n−1

= i]]

n -step transitions

From the Markovian assumption, we can note that P[X_t+m+n=j|X_t=i] =p_ij(m+n) =X

k

p_ik(m)p_kj(n) ,

which gives us the following property(m←1, n←n−1)

P(n) =P·P(n−1) =Pⁿ .

The marginal distribution of

X

nis given by P[Xn=i] =pi(n) =X

k

P[X0 =k]P[Xn=i|X0=k] =X

k

pk(0)pki(n) ,

and the state of the system at step

n

is given by

[p0(n), p1(n), . . .] =p(n) =p(0)·Pⁿ .

Asymptotic behavior v _i = lim

n→∞ P [X _n = i] ?

◦

v

i= long run proportion of time spent in state

i

◦ when

n → ∞

^,

p

ij

(n)

becomes independent of

i

^and

n

⇒

all rows of

P

ⁿconverge toward a common limit

v

◦ the limit satisfies

v = v · P

under the constraint that

v

i

≥ 0, P

i

v

i

= 1

^.

◦ conditions for the limit to be defined

⊲ aperiodic→the limiting state probabilities exists

⊲ irreductible aperiodic→the limit exists and is independent of the initial probabilities

p(0)

⊲ irreductible aperiodic with recurrent states (or finite state irreductible aperiodic)→ vis the unique stationary probability vector

(3)

Continuous time Markov chains (CTMC)

P [X

t

= x|X

t0

= x

0

, . . . , X

tn

−1

= x

n−1

] = P [X

t

= x|X

tn

−1

= x

n−1

]

◦ We want to study the following quantities(for homogeneous chains only)

⊲ state of the process at time

t

^:

p

i

(t) = P [X

t

= i]

⊲

n

-step transition probability:

p

ij

(t) = P [X

t

= j|X

0

= i]

◦ The process is entirely defined by

⊲ the initial distribution:

p (0) = [p

0

(0), p

1

(0), . . .]

⊲ the transition probabilities:

p

ij

(t)

◦ Properties of the transition probabilities

⊲

p

ij

(t) ≥ 0 ∀t

⊲

P

j

p

ij

(t) = 1 ∀i, t

⊲

p

ij

(t + s) = P

k

p

ik

(t)p

ki

(s) ∀t, s

Transition probabilities and transition rates

◦ dealing with transition probabilities that are a function of time is a mess!

⇒

use transition rates rather than probabilities

Q = P

^′

(0) q

ij

= lim

t→0

p

ij

(t)

t

^and

q

i

= −q

ii

= − lim

t→0

p

ii

(t) − 1 t

◦ properties of the transition rate matrix

⊲

d

dt P (t) = QP (t) = P (t)Q

⊲

P [X

t+h

= j|X

t

= i] = q

ij

h + o(h) i 6= j

⊲

p

ii

(t, t + h) = 1 − q

ii

h + o(h)

Data analysis and stochastic modeling – Queueing theory – p. 10

Jump process and embedded Markov chain

◦

T

iis the time of the

i

^{-th jump}

◦ the time to the next jump follows an exponential model

P [T

1

> t|X

0

= i] = e

⁻^qⁱ^t

◦

Y

n

= X

Tn is a DTMC with transition probabilities

p

ij

=



 

 

 

  q

ij

q

i if

i 6= j

^and

q

i

6= 0 0

^if

i 6= j

^and

q

i

= 0 0

^if

i = j

^and

q

i

6= 0 1

^if

i = j

^and

q

i

= 0

Limiting probabilities

Under certain conditions, we have for a CTMC

X

t

lim

→∞

p

ij

(t) = v

j

= π

j

/q

j

X

k

π

k

/q

k

where

π

is the unique invariant measure associated with

Y

^.

The limiting probabilities, if they exist, are obtained by solving the equation system

vQ = 0 v 1 = 1

(4)

Homogeneous Poisson process

◦

N (t)

= number of occurences of an event (with rate

λ

^{) in}

[0, t]

◦

P [N (t + τ ) − N(t) = k ] = e

⁻^λτ

(λτ )

^k

k!

Poisson law

◦ a counting process with the following property is a Poisson process

⊲ lim

τ→∞P[N(t+τ)−N(t)>1|N(t+τ)−N(t)≥1] = 0

⊲ memoryless process

⇒

interarrival times are independent and exponentially distributed (mean

λ

⁾

Classical discrete distributions: Poisson

◦ A Poisson

P (α)

is defined as P[X =k] = λ^ke⁻^λ

k!

◦ Probability of

k

events occuring at a rate

c

^{over an}

interval of duration

t

⁽

λ = ct

⁾

◦ Approximation of the binomial for small

p

^{and large}

n

◦

E [X] = V [X] = λ

Siméon D. Poisson 1781–1840

[Illustration: Skbkekas (from wikipedia.org)]

Poisson process and CTMC

A Poisson process is a continuous time Markov chain with rate transition matrix

Q =







−λ λ 0 0 . . . 0 −λ λ 0 . . . 0 0 −λ λ . . .

... ... ...

. . .







0 1 k−1 k k+1

Birth and death process

◦

N (t)

= size of a population at time

t

◦ Birth distribution

⊲

λ

n= birth rate for a population of size

n

⊲ remaining time until the next birth

exp(λ

n

)

◦ Death distribution

⊲

µ

n= death rate for a population of size

n

⊲ remaining time until the next death

exp(µ

n

) N (t)

is a CTMC with transition rate diagram

(5)

Birth and death process equilibrium

The equilibrium

v

is given by solving the system

vQ = 0 v 1 = 1

whose solution is given by

v

0

= 1

1 + S v

i

= 1

1 + S λ

0

. . . λ

i−1

µ

1

. . . µ

i

with

S = X

i

λ

0

. . . λ

i−1

µ

1

. . . µ

i

.

Particular case: if

λ

i

= λ

^and

µ

i

= µ

^{, then}

v

i

= ρ

ⁱ

(1 − ρ)

^with

ρ = λ/µ

^.

All this is true only if

S < ∞

or, in the particular case, if

ρ < 1

^.

Expected rate in = Expected rate out

flow into 0

→ µ

1

v

1 =

λ

0

v

0

←

flow out of 0

flow into 1

→ λ

0

v

0

+ µ

2

v

2 =

(λ

1

+ µ

1

)v

1

←

flow out of 1 flow into 2

→ λ

1

v

1

+ µ

3

v

3 =

(λ

2

+ µ

2

)v

2

←

flow out of 2

... ...

flow into i

→ λ

i−1

v

i−1

+ µ

i+1

v

i+1 =

(λ

i

+ µ

i

)v

i

←

flow out of i

v

1

= λ

0

µ

1

v

0

, v

2

= λ

1

µ

2

v

1

= λ

0

λ

1

µ

2

µ

1

v

0

, . . . v

k

= λ

k−1

. . . λ

0

µ

k

. . . µ

1

v

0

What is a queueing system?

input population

service mechanism queue

entry exit

Arrival process

◦ size of the population

◦ bulk vs. individual arrivals

◦ interarrival distribution

◦ customer balking

Service process

◦ number of servers

◦ batch or single service

◦ service time distribution

Queue

◦ finite or infinite

◦ discipline (FIFO, LIFO, priorities, etc.)

What are queueing systems good for?

A queueing system is usefull to answer the following questions

◦ What is the average number of customers in the queue? in the system?

◦ What is the average time a customer spends in the queue? in the system?

◦ What is the probability of a customer to be rejected?

◦ What is the fraction of time a server is idle?

◦ What is the probability distribution of a customer’s waiting time in the queue? in the system?

In turn, answering those questions supports decision making

◦ Should we have one or several queues?

◦ What is the best queueing discipline for my application?

(6)

Kendall’s notation

A/B/N/m/s

◦ A and B = distribution of resp. the interarrival and the service time M exponential model

D deterministic time

E_k Erlang model G arbitrary iid model

◦ N = number of parallel servers

◦ m = max. number of customers in the system

◦ s = population size Examples:

M/M/1/

∞

^/

∞

^{, M/M/S/}

∞

^/

∞

^{, M/M/1/C/}

∞

, M/M/1/K/K M/G/1, M/D/1, etc.

Erlang distribution

f (x; k, λ) = λ

^k

x

^k⁻¹

e

⁻^λx

(k − 1)!

Quantities of interest

λ

n = mean arrival rate of entering customers when

n

customers are in the system

µ

n = mean service of the overall system when

n

N (t)

⁼ number of customers in the system (queue and service) at time

t

P

n

(t)

⁼ the probability that there are exactly

n

customers in the system at time

t

R

i = time spent in the system by the

i

th customer

N (t)

^and

P

n

(t)

are difficult to compute for an arbitrary time

t

⇒

study steady-state regime when

t → ∞

Quantities of interest

^(cont’d)

We are primarily interested in the following steady state quantities

◦

N

: number of clients in the system

L = E [N] = lim

t→∞

1 t

Z

t 0

N (s)ds

◦

R

: time spent in the system by a client

W = E [R] = lim

n→∞

1 n

X

i=1

nR

i

We can define in a similar way the two quantities excluding service time,

L

Q

and

W

Q, where

L = L

S

+ L

Q.

(7)

Little’s law

For any queueing system with a steady state, with an average arrival rate of

λ

^,

L = λW .

Example: if the average waiting time is 2h and customers arrive at an average rate of 3 per hour, then the average number of customers in the queue is 6.

The same equality holds for

L

Qand

W

Q.

Note: for a constant death rate

µ

^,

W = W

Q

+ 1/µ

^.

The M/G/1 model and the PASTA property

◦ Poisson arrival process with an average arrival rate of

λ

◦ FIFO discipline

◦ no assumption on service time (apart from the iid one)

⊲ memoryless property is not verified

⇒ N (t)

is not Markovian

◦ study a snapshot of the system immediately after each arrival

⊲ embedded Markov chain

X

n

= N(t

n

)

◦ Poisson Arrivals See Temporal Average (PASTA)

π

k

= lim

t→∞

P [N (t) = k] = lim

n→∞

P [X

n

= k]

[R. Wolf. Poisson arrivals see time averages. Op. Res., 20:223–231, 1982]

Death and birth queues

◦ memoryless arrival process

⊲ interarrival law is

exp(λ

n

)

^given

n

customers in the system

⊲ probability of an arrival in

[t, t + dt]

⁼

λ

n

dt + o(t)

◦ memoryless service process

⊲ service law is

exp(µ

n

)

^when

n

⊲ probability of a departure in

[t, t + dt]

⁼

µ

n

dt + o(t)

◦ Durations and transitions to/from state

i

⊲ stays in

i exp(λ

i

+ µ

i

)

⊲ transits to

i − 1

with probability

µ

i

/(λ

i

+ µ

i

)

⊲ transits to

i + 1

with probability

λ

i

/(λ

i

+ µ

i

)

Steady state results

π

1

= λ

0

µ

1

π

0

, π

2

= λ

1

µ

2

π

1

= λ

0

λ

1

µ

2

µ

1

π

0

, . . . π

k

= λ

k−1

. . . λ

0

µ

k

. . . µ

1

π

0

expected number in the system L=E[N] =

∞

X

k=1

kπk

expected number in the queue LQ =E[NQ] =

∞

X

k=s

(k−s)πk

average (effective) arrival rate λ=

∞

X

k=0

λkπk

efficienty of the system E= (L−LQ)/s

(8)

The three server example

Consider a birth-and-death queueing system with

s = 3

servers and the following characteristics:

◦ average arrival rate

λ = 5

^{per hour}

◦ average service rate

µ = 2

^{per hour}

◦ customers balk when 6 are already in the system

k =

⁰ ¹ ² ³ ⁴ ⁵ ⁶

π

k

=

^0.068 ^0.170 ^0.212 ^0.177 ^0.147 ^0.123 ^0.102

The three server example

^(cont’d)

What is the probability that ...

◦ all servers are idle?

P[N(t)≤0] = 0.068

◦ a customer will not have to wait?

P[N(t)≤2] =π0+π1+π2= 0.45

◦ a customer will have to wait?

P[N(t)≥3] = 0.55

◦ a customer balks?

P[N(t) = 6] = 0.102

The three server example

^(cont’d)

◦ Expected number in queue:

L

Q

= π

4

+ 2π

5

+ 3π

6

= 0.7

◦ Expected number in service:

L

S

= π

1

+ 2π

2

+ 3(1 − π

0

− π

1

− π

2

) = 2.224

◦ Efficiency of the system:

E = L

S

/s = 74.8 %

◦ Average arrival rate:

λ = λ(π

0

+ . . . + π

5

) = 4.488

^{per hour}

◦ Expected waiting times using Little’s law with

λ

⊲

W

S

= L

S

/λ = 0.5

^hours

⊲

W

Q

= L

Q

/λ = 0.156

^hours

⊲

W = L/λ = W

S

+ W

Q

= 0.656

^hours

The methodology

1. model the system and construct the rate diagram

2. develop the balance equation and solve for

π

i

i = 0, 1, . . .

◦ use general birth-and-death process results whenever they apply 3. use steady-state distribution to

L

^and

L

Q

4. use Little’s law to get

W

^and

W

Q

(9)

The M/M/1 model

◦ exponential interarrival time with constant parameter

λ

k

= λ ∀k

◦ exponential service time with constant parameter

µ

k

= µ ∀k

µ λ

2 µ

λ

µ λ

5 µ

λ

6 µ

λ

0 1 3 4

π

n

= ρ

ⁿ

(1 − ρ)

L =

∞

X

n=1

nπ

n

= λ

µ − λ W = L/λ = 1

µ − λ L

Q

= L − (1 − π

0

) = λ

²

µ(µ − λ) W

Q

= L

Q

/λ = λ

µ(µ − λ)

The repairman example

A maintenance worker must maintain 2 machines, where the 2 machines operate simultaneously when both are up. We make the following hypotheses:

◦ time until a machine breaks down exp with mean 10 hours

◦ time until a machine is fixed exp with mean 8 hours

◦ the repairman can only fix one machine at a time

⇒

^M/M/1/2/2

µ µ

2λ λ

The repairman example

^(cont’d)

flow into 0

→ µ

1

π

1 =

2λπ

0

←

flow out of 0 flow into 1

→ 2λπ

0

+ µπ

2 =

(λ + µ)π

1

←

flow out of 1

normalize

λπ

1 =

µπ

2

⇒ π

0

= 0.258 π

1

= 0.412 π

2

= 0.330 ⇒ λ = 0.0928

L=π1+ 2π2= 1.072 W = L

λ = 11.55hour

LQ=π2= 0.33 WQ= LQ

λ = 3.56hour Proportion of time the repairman is busy =

π

1

+ π

2

= 0.742

Proportion of time machine #1 is working =

π

0

+

¹₂

π

1

= 0.464

The telephone answering example

◦ A utility company wants to determine a staffing plan for its customer representatives. Calls arrive at an average rate of 10 per minute and the average service time is 1 minute.

◦ Determine the number of operators that would provide a “satisfactory”

service to the calling population.

◦ Parameters:

λ = 10, µ = 1, ρ = λ/sµ < 1 ⇒ s > 10

M/M/11 M/M/12 M/M/13

L

Q 6.82 2.25 0.95

W

Q 0.68 0.22 0.09

P [T

Q

= 0]

^0.32 ^0.55 ^0.71

P [T

q

> 1]

^0.25 ^0.06 ^0.01

(10)

One last for the road

◦ Packets arrive to a router at a rate of 1.5 min. The mean routing time is 30 seconds and if more than 3 packets are in the queue, an alternative routeur is seeked.

◦ Does this setting meet the following criteria: no more than 5% of the packets goes to another routeur and no more than 10% of the packets stay more than 1 min in the queue?

◦ Using the M/M/1/4 model, you should be able to demonstrate that

⊲ the balking probability

= π

4

= −.104

does not meet the 5 % criteria

⊲ the probability

P [T

Q

> 1] = 0.225

does not meet the 10 % objective

◦ However, a M/M/2/5 configuration does and so you need to buy one more routeur!

Networks of queues

Data analysis and stochastic modeling