Poisson process and Markov jump processes

(1)

Poisson process and Markov jump processes

Laurent Massouli´e

Inria

February 22, 2021

Laurent Massouli´e (Inria) Poisson process and Markov jump processes February 22, 2021 1 / 20

(2)

Outline

Poisson processes Markov jump processes

(3)

The Poisson distribution

(Sim´ eon-Denis Poisson, 1781-1840)

e^{−λ λ}_n!ⁿ _n∈_N As prevalent as Gaussian distribution

Law of rare events(a.k.a. law of small numbers) pn,i ≥0 such thatlimn→∞sup_ipn,i = 0,limn→∞P

ipn,i =λ >0 Then X_n=P

iZ_n,i with Z_n,i: independent Bernoulli(p_n,i) verifies X_n→^D Poisson(λ)

(4)

The Poisson distribution

(Sim´ eon-Denis Poisson, 1781-1840)

e^{−λ λ}_n!ⁿ _n∈_N As prevalent as Gaussian distribution Law of rare events(a.k.a. law of small numbers) pn,i ≥0 such thatlimn→∞sup_ipn,i = 0,limn→∞P

ipn,i =λ >0 Then X_n=P

iZ_n,i with Z_n,i: independent Bernoulli(p_n,i) verifies X_n→^D Poisson(λ)

(5)

Point process on R

⁺

Definition

Point process onR+:

Collection of random times {T_n}_n>0 with 0<T1 <T2. . . Alternative description

Collection {N_t}_t∈_R₊ with Nt:=P

n>0ITn∈[0,t]

Yet another description

Collection {N(C)}for all measurableC ⊂R+ where N(C) :=X

n>0

ITn∈C

(6)

Poisson process on R

⁺

Definition

Point process such that for alls0= 0<s1 <s2 < . . . <sn,

1 Increments {N_s_i −N_s_i₋₁}_1≤i≤n independent

2 Law of N_t+s−N_s only depends ont

3 for someλ >0,Nt ∼Poisson(λt)

In fact, (3) follows from (1)–(2); see lecture notes. λis called the intensity of the process

(7)

Poisson process on R

⁺

Definition

In fact, (3) follows from (1)–(2); see lecture notes.

λis called the intensity of the process

(8)

Poisson process on R

⁺

Definition

In fact, (3) follows from (1)–(2); see lecture notes.

λis called the intensity of the process

(9)

Structure of interarrival sequences

Proposition

For Poisson process {T_n}_n>0 of intensityλ, its interarrival times τ_i =T_i₊₁−T_i, whereT₀ = 0, verify

{τ_n}_n≥0 i.i.d. with common distributionExp(λ)

Density of Exp(λ): λe^−λxIx>0

Key property: Exponential random variableτ is memoryless, i.e. ∀t >0, P(τ −t ∈ ·|τ >t) =P(τ ∈ ·)

(10)

Proof:

Characterization of law of {τ_i}_i∈[n]} by Laplace transform EQ

i∈[n]e^−αⁱ^τⁱ: show that for all n≥1, αⁿ₁ ∈Rⁿ+,EQ

i∈[n]e^−αⁱ^τⁱ =Q

i∈[n] λ λ+αi. Fix h>0. Let Zn=INnh−N_(n−1)h≥1,S1 = inf{n >0 :Zn≥1}, S_i = inf{n>Si−1:Z_n= 1}, i >1.

Approximation of interarrivals: τ_i^(h):=h(S_i+1−S_i).

{S_i+1−S_i}: i.i.d. Geom(1−e^−λh), so that EQ

i∈[n]e^−αⁱ^τⁱ^(h) = (1 +O(h))Q

i∈[n] λ λ+α_i.

On event A_h:={∀i ∈[n], τ_i >h},∀i ∈[n], |τ_i−τ_i^(h)| ≤h and hence Q

i∈[n]e^−αⁱ^τⁱ = (1 +O(h))Q

i∈[n]e^−αⁱ^τⁱ^(h).

Conclude by noting thatA_h ↑ {∀i ∈[n], τi >0}hencelimh→0P(A_h) = 1.

(11)

Alternative construction

Proposition

Process with i.i.d., Exp(λ)interarrival times{τ_i}_i≥0 can be constructed on [0,t]by

1) DrawingN_t ∼Poisson(λt)

2) Putting Nt pointsU1, . . . ,U_N_t on[0,t]whereUi: i.i.d. uniform on[0,t]

Proof.

Establish identity for all n∈N,φ:Rⁿ+→R:

E[φ(τ0, τ0+τ1, . . . , τ0+. . .+τn−1)INt=n] =· · · e^−λt^(λt)_n!ⁿ ×n!R

(0,t]ⁿφ(s₁,s₂, . . . ,s_n)Is1<s2<...<sn

Qn i=11

tds_i

=P(Poisson(λt) =n)×E[φ(S₁, . . . ,S_n)]

where S₁ⁿ: sorted version of i.i.d. variables uniform on[0,t]

(12)

(13)

(14)

Alternative construction

Proposition

Process with i.i.d., Exp(λ)interarrival times{τ_i}_i≥0 can be constructed on [0,t]by

1) DrawingN_t ∼Poisson(λt)

2) Putting Nt pointsU1, . . . ,U_N_t on[0,t]whereUi: i.i.d. uniform on[0,t]

Proof.

Establish identity for all n∈N,φ:Rⁿ+→R:

E[φ(τ0, τ0+τ1, . . . , τ0+. . .+τn−1)INt=n] =· · · e^−λt^(λt)_n!ⁿ ×n!R

(0,t]ⁿφ(s₁,s₂, . . . ,s_n)Is1<s2<...<sn

Qn i=11

tds_i

=P(Poisson(λt) =n)×E[φ(S1, . . . ,Sn)]

where S₁ⁿ: sorted version of i.i.d. variables uniform on[0,t]

(15)

Laplace transform of Poisson processes

Definition

For function f :R+→R+, let

N(f) :=X

n>0

f(Tn).

The Laplace transform of point process N↔ {T_n}_n>0 is the functional whose evaluation at f :R+→R+ is

L_N(f) :=Eexp(−N(f)) =E(exp(−X

n>0

f(T_n))).

Proposition: Knowledge of L_N(f) on sufficiently rich class of functions f :R+→R+ characterizes law of point process N.

Exemples of such classes of functions:

- non-negative piecewise constant with compact support - non-negative piecewise continuous with compact support - non-negative continuous with compact support

(16)

Laplace transform of Poisson processes

Definition

For function f :R+→R+, let

N(f) :=X

n>0

f(Tn).

The Laplace transform of point process N↔ {T_n}_n>0 is the functional whose evaluation at f :R+→R+ is

L_N(f) :=Eexp(−N(f)) =E(exp(−X

n>0

f(T_n))).

Proposition: Knowledge of L_N(f) on sufficiently rich class of functions f :R+→R+ characterizes law of point process N.

Exemples of such classes of functions:

- non-negative piecewise constant with compact support - non-negative piecewise continuous with compact support - non-negative continuous with compact support

(17)

Laplace transform of Poisson processes

Proposition

Poisson process with intensityλadmits Laplace transform L_N(f) = exp(−

Z

R⁺λ(1−e^−f^(x))dx)

Proof: Previous construction yields expression forL_N(f) Corollary: For f =P

iα_iICi ⇒N(C_i)∼Poisson(λR

Cidx), with independence for disjoint C_i. Hence existence of Poisson process.

(18)

Laplace transform of Poisson processes

Proposition

Poisson process with intensityλadmits Laplace transform L_N(f) = exp(−

Z

R⁺λ(1−e^−f^(x))dx) Proof: Previous construction yields expression forL_N(f) Corollary: For f =P

iα_iICi ⇒N(C_i)∼Poisson(λR

Cidx), with independence for disjoint C_i. Hence existence of Poisson process.

(19)

Poisson process with general space and intensity

Definition

Point process on R^d: countable or finite collection {T_i}_i≥1 of distinct points of R^d

Definition

For λ:R^d →R+ locally integrable function,N↔ {T_n}_n>0 point process onR^d is Poisson with intensity functionλif and only if

for measurable, disjoint C_i ⊂R^d,i = 1, . . . ,n, N(Ci) independent, ∼Poisson(R

Ci λ(x)dx)

Proposition

Such a process exists and admits Laplace transform L_N(f) = exp

− Z

R^dλ(x)(1−e^−f^(x))dx

(20)

(21)

Poisson process with general space and intensity

Definition

Point process on R^d: countable or finite collection {T_i}_i≥1 of distinct points of R^d

Definition

For λ:R^d →R+ locally integrable function,N↔ {T_n}_n>0 point process onR^d is Poisson with intensity functionλif and only if

for measurable, disjoint C_i ⊂R^d,i = 1, . . . ,n, N(Ci) independent, ∼Poisson(R

Ci λ(x)dx)

Proposition

Such a process exists and admits Laplace transform L_N(f) = exp

− Z

R^dλ(x)(1−e^−f^(x))dx

(22)

(23)

Markov jump processes

Process {X_t}_t∈_R₊ with values inE, countable or finite, is Markovif

P(Xtn =xn|X_t_n−1 =xn−1, . . .Xt1 =x1) =P(Xtn =xn|X_t_n−1 =xn−1), t₁ⁿ ∈Rⁿ+, t₁ <· · ·<t_n, x₁ⁿ∈Eⁿ

Homogeneous ifP(Xt+s =y|X_s =x) =:pxy(t)independent of s, s,t ∈R+, x,y ∈E

⇒ Semi-group propertypxy(t+s) =P

z∈Epxz(t)pzy(s), or P(t+s) =P(t)P(s)with P(t) ={p_xy(t)}_x,y∈E Definition

{X_t}_t∈_R₊ is a pure jumpMarkov process if in addition

(i) It spends with probability 1 a strictly positive time in each state (ii) Trajectories t→X_t are right-continuous

(24)

Markov jump processes

z∈Epxz(t)pzy(s), or P(t+s) =P(t)P(s) with P(t) ={p_xy(t)}_x,y∈E

Definition

(25)

Markov jump processes

z∈Epxz(t)pzy(s), or P(t+s) =P(t)P(s) with P(t) ={p_xy(t)}_x,y∈E Definition

(26)

Markov jump processes: examples

Poisson process {N_t}_t∈_R₊: then Markov jump process with pxy(t) =P(Poisson(λt) =y−x)

Continuous-time random walk on finite, undirected graphG = (V,E): Sojourn time at node i ∈V: exponentially distributed with parameter d_i, degree of nodei,

after which: jump to neighbor ofi selected uniformly at random

(27)

Markov jump processes: examples

Poisson process {N_t}_t∈_R₊: then Markov jump process with pxy(t) =P(Poisson(λt) =y−x)

Continuous-time random walk on finite, undirected graphG = (V,E):

Sojourn time at node i ∈V: exponentially distributed with parameter d_i, degree of nodei,

after which: jump to neighbor ofi selected uniformly at random

(28)

Markov jump processes: more examples

Single-server queue, FIFO (“First-in-first-out”) discipline, arrival times: N Poisson(λ), service times: i.i.d. Exp(µ) independent ofN Xt= number of customers present at time t: Markov jump process by Memoryless property of Exponential distribution + Markov property of Poisson process

(the M/M/1/∞queue)

Infinite server queue with Poisson arrivals and Exponential service times: customer arrived atTn stays in system till Tn+σn, whereσn: service time

Xt= number of customers present at time t: Markov jump process (theM/M/∞/∞ queue)

(29)

Markov jump processes: more examples

Single-server queue, FIFO (“First-in-first-out”) discipline, arrival times: N Poisson(λ), service times: i.i.d. Exp(µ) independent ofN Xt= number of customers present at time t: Markov jump process by Memoryless property of Exponential distribution + Markov property of Poisson process

(the M/M/1/∞queue)

Infinite server queue with Poisson arrivals and Exponential service times: customer arrived atTn stays in system till Tn+σn, whereσn: service time

Xt= number of customers present at time t:

Markov jump process (theM/M/∞/∞ queue)

(30)

Structure of Markov jump processes

Infinitesimal Generator

∀x,y, y 6=x ∈E, limitsq_x := limt→01−pxx(t)

t ,q_xy = limt→0pxy(t)

t exist in R+ and satisfy P

y6=xqxy =qx

qxy: Jump rate fromx to y

Q :={q_xy}_x,y∈E whereq_xx =−q_x: Infinitesimal Generatorof process {X_t}t∈R+

Formally: Q= limh→0 1

h[P(h)−I]whereI: identity matrix

Structure of Markov jump processes

Sequence{Y_n}_n∈_N of visited states: Markov chain with transition matrix p_xy =Ix6=yqxy

qx

Conditionally on {Y_n}_n∈_N, sojourn times{τ_n}_n∈_Nin successive states Y_n: independent, with distributions Exp(qYn)

(31)

Structure of Markov jump processes

Infinitesimal Generator

∀x,y, y 6=x ∈E, limitsq_x := limt→01−pxx(t)

t ,q_xy = limt→0pxy(t)

t exist in R+ and satisfy P

y6=xqxy =qx

qxy: Jump rate fromx to y

Q :={q_xy}_x,y∈E whereq_xx =−q_x: Infinitesimal Generatorof process {X_t}t∈R+

Formally: Q= limh→0 1

h[P(h)−I]whereI: identity matrix Structure of Markov jump processes

Sequence{Y_n}_n∈_N of visited states: Markov chain with transition matrix p_xy =Ix6=yqxy

qx

Conditionally on {Y_n}_n∈_N, sojourn times{τ_n}_n∈_Nin successive states Y_n: independent, with distributions Exp(qYn)

(32)

Proof elements

0<T₁ <· · ·<T_n <· · ·: jump times; τ_i =T_i+1−T_i sojourn time Discrete time chain for h>0: Zn:=X_nh

τ₀^(h):=h inf{n>0 :Z_n6=Zn−1}.

On{τ₁>h},τ₀^(h)−h≤τ₀≤τ₀^(h)

By assumption, limh→0P(τ1 >h) = 1, hence Px(τ₀ >t) = limh→0Px(τ₀^(h)>t;τ₁ >h)

= limh→0pxx(h)^d^h^t^e

=e^lim^h→0^d^t^h^e^lnp^xx^(h).

.

Since p_xx(h)≥Px(τ₀ >h)^h→0→ 1,d_h^telnp_xx(h)∼ _h^t(p_xx(h)−1).

Hence ∃q_x := limh→01−pxx(h)

h , andPx(τ₀ >t) =e^−q^x^t. Caseq_x = +∞

ruled out (τ0 >0 a.s.); caseqx = 0: τ0 = +∞ a.s., i.e. process absorbed in x.

(33)

Assuming q_x >0:

Ex(e^−ατ⁰IY1=y) ∼Ex[e^−ατ⁰^(h)IX

τ(h) 0

=y]

=P

n≥1e^−αnhPx(Xh=· · ·=Xh(n−1) =x,Xhn=y)

=P

n≥1e^−αnh[pxx(h)]ⁿ⁻¹pxy(h)

=P

n≥1e^−αnhP

m≥nPx(τ₀^(h)=hm)p_xy(h)

=P

m≥1Px(τ₀^(h)=m)e^−αh^1−e_1−e^−αhm−αh pxy(h)

=e^−αh[1−Exe^−ατ⁰^(h)]_1−e^p^xy^(h)−αh

∼[1−_q^q^x

x+α]^p^xy_αh^(h)

= _q¹

x+α pxy(h)

h · Implies existence of limit limh→0pxy(h)

h =q_xy, independence of τ₀ andY₁ and Px(Y1 =y) = ^q_q^xy

x .

Similar arguments for joint law of Y₁ⁿ, τ₀ⁿ⁻¹.

(34)

Examples

for Poisson process(λ): only non-zero jump rate qx,x+1=λ=qx, x ∈N

For continuous-time random walk on graphG = (V,E), non-zero rates: q_i_,j =Ii∼j, henceq_i =d_i. Generator Q is opposite of so-called Laplacian matrixL(G) of graph G

For FIFO M/M/1/∞ queue, non-zero rates: q_x,x+1 =λ, qx,x−1 =µIx>0, x ∈Nhence qx =λ+µIx>0

For M/M/∞/∞ queue, non-zero rates: qx,x+1 =λ, qx,x−1 =µx, x∈Nhenceq_x =λ+µx

(35)

Examples

For continuous-time random walk on graphG = (V,E), non-zero rates: q_i,j =Ii∼j, henceq_i =d_i. Generator Q is opposite of so-called Laplacian matrixL(G) of graph G

For FIFO M/M/1/∞ queue, non-zero rates: q_x,x+1 =λ, qx,x−1 =µIx>0, x ∈Nhence qx =λ+µIx>0

(36)

Examples

For FIFO M/M/1/∞ queue, non-zero rates: q_x,x+1 =λ, qx,x−1=µIx>0, x ∈Nhence qx =λ+µIx>0

(37)

Examples

For FIFO M/M/1/∞ queue, non-zero rates: q_x,x+1 =λ, qx,x−1=µIx>0, x ∈Nhence qx =λ+µIx>0

For M/M/∞/∞ queue, non-zero rates: qx,x+1 =λ, qx,x−1=µx, x ∈Nhenceq_x =λ+µx

(38)

Structure of Markov jump processes (continued)

Let T_n:=Pn−1

k=0τ_k: time ofn-th jump.

IfT∞= +∞ almost surely: trajectory determined onR+, hence generator Q determines law of process{X_t}_t∈_R₊

Process is calledexplosiveif instead T∞<+∞ with positive probability. Then process not completely characterized by generator

Sufficient conditions for non-explosiveness: sup_x∈Eq_x <+∞

Recurrence of induced chain{Y_n}_n∈_N

For Birth and Deathprocesses (i.e. E =N, only non-zero rates: β_n=q_n,n+1, birth rate; δ_n=qn,n−1, death rate), non-explosiveness holds if

X

n>0

1

β_n+δ_n = +∞

(39)

Structure of Markov jump processes (continued)

Let T_n:=Pn−1

Process is calledexplosiveif instead T∞<+∞ with positive probability.

Then process not completely characterized by generator

Sufficient conditions for non-explosiveness: sup_x∈Eq_x <+∞

For Birth and Deathprocesses (i.e. E =N, only non-zero rates: β_n=q_n,n+1, birth rate; δ_n=qn,n−1, death rate), non-explosiveness holds if

X

n>0

1

β_n+δ_n = +∞

(40)

Structure of Markov jump processes (continued)

Let T_n:=Pn−1

Process is calledexplosiveif instead T∞<+∞ with positive probability.

Then process not completely characterized by generator Sufficient conditions for non-explosiveness:

sup_x∈Eq_x <+∞

For Birth and Deathprocesses (i.e. E =N, only non-zero rates:

β_n=q_n,n+1, birth rate; δ_n=qn,n−1, death rate), non-explosiveness holds if

X

n>0

1

β_n+δ_n = +∞

(41)

Takeaway messages

Poisson process a fundamental continuous-time process, adequate model for aggregate of infrequent independent events

Markov jump processes: generator Q characterizes distribution if not explosive