Model checking DTMC/MDP

3 Quantitative verification

As CONNECTors often work in a highly dynamic and distributed environment, they could behave in an unreliable way. For instance, a CONNECTor in the popcorn scenario may not always transmit a re-sponse message successfully to clients due to competition on available channels, interfere with other CONNECTors, etc. In many cases, unreliable behaviours can be formalised as probabilistic behaviours, such as the percentage of successful transmissions on average.

This chapter is devoted to quantitative verification for CONNECTors, i.e., the verification of their prob-abilistic behaviours, aiming to establish the probability of some event occurring, or the expected time or reward until a certain state is reached. In this chapter, we are focusing on quantitative models based on labelled transition systems, since they are common to the main CONNECTwork packages, including syn-thesis and learning. We thus assume that there is an effective translation from the high-level connector algebra formalism to a low-level transition system. For instance, Section 2.4.2 presented an approach for converting QIA to CTMCs, which will be introduced in Section 3.1.2, and Section 2.4.3 explained that PCA is indeed an MDP.

Section 3.1 gives an overview ofclassical stochastic models, temporal logic formalisms and model-checking algorithms for offline verification, which rely on identifying all possible system states to estab-lish probabilistic properties. This also includes one of our contributions in the project: new abstraction-refinement techniques devised and implemented for probabilistic real-time systems. Section 3.2 deals with non-functional characteristics of CONNECTors, usually arising from QoS requirements, that are ex-pressible withrewardsand the corresponding verification algorithms. The last two sections in this chapter describe new research results from WP2. In Section 3.3, we propose a compositional approach to handle verification of large systems which we aim to apply to CONNECTors. In Section 3.4, we discuss an online verification method (also known as run-time monitoring and adaptation), which employs offline verification using models and properties that are generated at run-time. This online method can be seen as a starting point for a model-checking approach to property verification for evolving connectors, which is relevant to dependability assurance in WP5.

for all statess∈S.

• L:S→2^AP is alabelling functionover the setAP of atomic propositions;

Each entryP(s, s⁰)in the transition probability matrix gives the probability of making a transition from statesto states⁰. If a state has no successor states, we assume it has a self loop on itself with probability 1. The matrix also indicates that DTMCs defined here are homogeneous, which means the transition probabilities are independent of the time when transitions are taken.

To model functional requirements, we label each state with a set of atomic propositions that hold in the state (done by the labelling functionL). Characterising probabilistic behaviours is specified by PCTL (Probabilistic Computational Tree Logic) [41,17], which is a probabilistic extension of the temporal logic CTL, over the setAP of atomic propositions and system executions. In the following, we give the formal description of executions (or paths) first and then the syntax and semantics of PCTL.

A pathωis a non-empty sequence of statess0s1s2. . ., wheresi∈SandP(sk, sk+1)>0for alli≥0.

If the path is finite, we denote it by ωf in. The ith state in the path is denoted by ω(i). A finite path ω_{f in} of lengthn(i.e., there are ntransition in the path) is aprefixof an infinite pathω if ω_{f in}(i) = ω(i) for 0 ≤ i ≤ n. Let P ath_s and P ath^{f in}_s be the set of infinite paths and finite paths starting in state s respectively. The probability of taking a path is computed through a probability measureP rob_soverP ath_s for each states∈S. We first define the probabilityP_s(ω_{f in})for a finite pathω_{f in}∈P ath^{f in}_s to be

Ps(ωf in)^def=

1 ifn= 0 P(ω(0), ω(1)). . .P(ω(n−1), ω(n)) otherwise

wherenis the length ofω_{f in}. LetC(ω_{f in})be the set of all infinite paths having prefixω_{f in}, i.e., C(ω_{f in})^def= {ω∈P ath_s|ω_{f in}is a prefix ofω}.

LetΣsbe the smallestσ-algebra onP athscontaining all the setC(ωf in)whereωf inranges over paths in P ath^{f in}_s . The probability measureP robsonΣsis the unique measure such that

P robs(C(ωf in)) =P(ωf in) for allωf in∈P ath^{f in}_s . Definition 3.2 The syntax of PCTL is as follows:

φ::=true|a| ¬φ|φ∧φ| P./p[ψ], ψ::=X φ|φU^≤k φ|φU φ whereais an atomic proposition, and./∈ {≤, <,≥, >},p∈[0,1],c∈R^≥0andk∈N.

In the above definition,φdenotes state formulae andψpath formulae. The former is evaluated over states and the latter over paths. The formulaP./p[ψ] holds in a statesif the probability of taking a path fromssatisfyingψis in the interval specified by./ p. Letω=s₀s₁. . .be a path, wheres_i(i≥0) is a state.

The formulaX φis true in the pathωifφis satisfied in the second state ofω, i.e.,s₁; the formulaφ₁U^≤kφ₂ is true ifφ₂ is satisfied withinktime-steps andφ₁ holds in all states before that point;φ₁ U φ₂ is similar to the previous one except there is no bound on the time-steps whenφ2holds. The formal semantics of PCTL is given as follows.

Definition 3.3 Let D = (S, s,P, L,C)be a DTMC. For any state s ∈ S, the satisfaction relation |= is defined inductively by:

• s|=truefor alls∈S,

• s|=aiffa∈L(s),

• s|=¬φiffs6|=φ,

• s|=φ₁∧φ₂iffs|=φ₁ands|=φ₂,

• s|=P./p[ψ]iffps(ψ)./ p

where

p_s(ψ)^def= P rob_s({ω∈P ath_s|ω|=ψ}) and for any pathω∈P aths:

• ω|=X φiffω(1)|=φ,

• ω|=φ₁U^≤k φ₂iff∃i≤k .(ω(i)|=φ₂∧ω(j)|=φ₁∀j < i),

• ω|=φ1U φ2iff∃k≥0.(ω|=φ1U^≤kφ2).

The PCTL model checking algorithm for DTMCs was first presented in [33, 35,41]. The algorithm takes a DTMC(S, s,P, L)and a PCTL formulaφ, and produces a setSat(φ)of states such thatSat(φ) = {s ∈ S | s |= φ}. Usually, we are only interested in whether the initial states satisfies the formulaφ, though the algorithm returns the whole set of states in whichφholds. To computeSat(φ), the algorithm recursively computes the setSat(φ⁰)of states satisfying each subformulaφ⁰as follows.

• Sat(true) =S,

• Sat(a) ={s|a∈L(s)},

• Sat(¬φ) =S\Sat(φ),

• Sat(φ₁∧φ₂) =Sat(φ₁)∩Sat(φ₂),

• Sat(P./p[ψ]) ={s∈S |ps(ψ)./ p}.

In the above procedure, checking Boolean operators such as∧and¬are straightforward. Here we only explain the detail for checking operatorP./p[ψ]. The key step is to computeps(ψ)for each states∈ S.

We need to distinguish three casesψ=X φ,φ₁U^≤kφ₂andφ₁U φ₂.

TheP./p[X φ]operator. Computingp_s(X φ)for each states∈ Sis done directly from the transition probability matrix after the setSat(φ)is obtained:

ps(X φ) = X

s⁰∈Sat(φ)

P(s, s⁰).

TheP./p[φ₁U^≤kφ₂]operator. In this case, we need to partition the setSinto three subsetsS^yes,S^no andS^?, which are defined as follows.

S^yes=Sat(φ2), S^no=S\(Sat(φ1)∪Sat(φ2)), Sat^?=S\(S^yes∪S^no).

Obviously, the setS^yesof states satisfy the formulaφ1U^≤k φ2(i.e.,ps(φ1U^≤k φ2) = 1), and the setS^noof states do not (i.e.,p_s(φ₁U^≤k φ₂) = 0). The probabilityp_sfor the setS^?of states is computed recursively as follows.

ps(φ1U^≤k φ2) =

0 ifk= 0 P

s⁰∈SP(s, s⁰)·ps⁰(φ1U^≤k−1φ2) ifk≥1

The P./p[φ1 U φ2] operator. Like the previous case, we partition S into S^yes, S^no and S^?, and ps(φ1U φ2) = 1and0forS^yesandS^norespectively. However, the procedures to determine these three sets are more complex than the “bounded until”U^≤k case.

S^no=P rob0(Sat(φ1), Sat(φ2)), S^yes=P rob1(Sat(φ1), Sat(φ2), S^no),

S^?=S\(S^yes∪S^no).

Thewhileloop inP rob0in Algorithm 1 incrementally computes the set of states from which with non-zero probability, a state satisfyingφ2can be reached without leaving states satisfyingφ1. Indeed, it implements a fix point computation. By subtracting this set from S, we obtain S^no. The while loop in P rob1 in Algorithm 2 works in a similar way. It incrementally computes the set of states that cannot satisfyφ1U φ2

with probability 1. Such a state must satisfy one of the two conditions:

Algorithm 1P rob0(Sat(φ1), Sat(φ2))

1: R:=Sat(φ₂)

2: done:=false

3: whiledone=falsedo

4: R⁰ :=R∪ {s∈Sat(φ₁)| ∃s⁰∈R .P(s, s⁰)>0}

5: ifR⁰=Rthen

6: done:=true

7: end if

8: R:=R⁰

9: end while

10: return S\R

Algorithm 2P rob1(Sat(φ₁), Sat(φ₂), S^no)

1: R:=S^no

2: done:=false

3: whiledone=falsedo

4: R⁰ :=R∪ {s∈(Sat(φ1)\Sat(φ2))| ∃s⁰∈R .P(s, s⁰)>0}

5: ifR⁰=Rthen

6: done:=true

7: end if

8: R:=R⁰

9: end while

10: return S\R

1. It is inS^no;

2. φ1holds in the state, butφ2does not. Further, it can reach a state satisfying the condition 1 or 2 in one step with non-zero probability.

After obtaining S^no, S^yes and S^?, we compute ps(φ1 U φ2) for all states in S recursively by letting xs=ps(φ1U φ2)and solving the linear equation system inxs:

xs=







0 ifs∈S^no

1 ifs∈S^yes

P

s⁰∈SP(s, s⁰)·x_s⁰ ifs∈S^?

MDPs (Markov decision processes) are a generalisation of DTMCs. They allow both probabilistic behaviour and non-deterministic behaviour. It is very useful when we model concurrency, e.g., several probabilistic transitions in parallel, or when the exact probability distribution of a transition is not known or not relevant. Let Dist(S) be the set of all probability distributions over S, i.e., the set of functions µ:S→[0,1]such that P

s∈S

µ(s) = 1. An MDP is formally defined as follows.

Definition 3.4 An MDP is a tupleM= (S, s, Act, Steps, L)where

• Sis a finite set ofstates,

• s∈Sis theinitial state,

• Actis a set ofactions,

• Steps:S →2Act×Dist(S)is theprobabilistic transition function.

• L:S→2^AP is alabelling function,

An MDP is similar to a DTMC except that the transition probability matrixPis replaced bySteps, which maps each states∈Sto a finite non-empty subset ofAct×Dist(S). For each states∈S,Stepsdefines a set of enabled actions that can be performed insand associates a probability distribution functionµto each enabled actiona. There are two steps to determine a successor state ofs: first, an action is chosen non-deterministically from the set of enabled actions; next, the probability of moving to states⁰is decided byµ, i.e.,µ(s⁰).

Apathin an MDP is a non-empty sequence of the form s0

(a1,µ1)

−−−−→s1 (a2,µ2)

−−−−→s2. . .

wheres_i ∈S,(a_i+1, µ_i+1)∈Steps(s_i)andµ_i+1(s_i+1)>0for alli≥0. We still useω(i)to denote theith state in the pathω,ω^{f in} to denote a finite path,last(ω_{f in})to denote the last state inω_{f in}, andP ath^{f in}_s (P aths) to denote the set of all finite (infinite) paths starting in state s. Moreover, theith action inω is represented bystep(ω, i).

To resolve the non-deterministic choices when we execute an MDP, we employ anadversaryto select a choice based on the history of choices made so far.

Definition 3.5 An adversaryAof an MDPMis a function mapping every finite pathωf inonto an element A(ωf in)of the setSteps(last(ωf in)). LetAdvMdenote the set of all possible adversaries of the MDP and, for any adversaryA, letP ath^A_s denote the subset ofP ath_swhich corresponds toA.

Since an adversary of an MDPM = (S, s, Act, Steps, L)removes non-deterministic behaviour, the behaviour froms∈Sforms an infinite-state DTMCD^A= (S^A, s^A,P^A, L^A)such that

• S^A=P ath^{f in}_s ,

• s^A=s,

• L^A(s^A) =L(last(s^A))for all states^A∈S^A and for two finite pathsω, ω⁰∈S^A:

• P^A(ω, ω⁰) = (

µ(s⁰) ifω⁰is of the formω−−−−→^(a,µ) s⁰ 0 otherwise.

From the above construction, we know that each states^A inD^A represents a pathω_{f in}∈ P ath^A_s ofM, and the path froms^Atos^Acan be mapped toω_{f in}uniquely. Therefore, we are able to use the probability measure over DTMCs to define a probability measureP rob^A_s over the set of pathP ath^A_s.

PCTL defined over DTMCs can be naturally extended to MDPs. The syntax is the same as before.

The major difference for MDPs is the semantics of theP_./p[ψ]operator:

• s|=P./p[ψ]iffp^A_s(ψ)./ pfor allA∈Adv_M where for any adversaryA∈Adv_M:

p^A_s(ψ)^def= P rob^A_s({ω∈P ath^A_s |ω|=ψ}).

In addition, we may like to reason about theexistenceof an adversary that satisfying a property, instead of asking all adversaries to satisfy the property. This can be done via translation to a dual property in the following way:

• p^A_s(ψ)> pfor some adversaryAis equivalent to¬P_≤p[ψ],

• p^A_s(ψ)≤pfor some adversaryAis equivalent to¬P>p[ψ].

The PCTL model checking algorithm for MDPs works in the same way as the one for DTMCs does.

It takes a model M= (S, s, Act, Steps, L)and a PCTL formula φas input, and recursively build the set Sat(φ)⊆Sof states that satisfy the formula. The Boolean operators are handled in the same way too. But we use a different procedure to deal withP./p[ψ], as we need to check whetherps(ψ)satisfies the bound

./ pfor all adversariesA. If./is≤or<, then we compute the maximum probability for all adversaries and obtain

Sat(P./p[ψ]) ={s∈S|p^max_s |(ψ)./ p}.

If./is≥or>, then we compute the minimum probability for all adversaries and obtain Sat(P_./p[ψ]) ={s∈S |p^min_s |(ψ)./ p}.

Now we present how to computep^max_s andp^min_s forψ=X φ,φ₁U^≤kφ₂andφ₁Uφ₂[23,34,36,37,38].

TheP_./p[X φ] operator. For this operator, p^max_s and p^min_s can be computed directly from function Steps.

p^max_s (X φ) = max

(a,µ)∈Steps(s)

X

s⁰∈Sat(φ)

µ(s⁰)

,

p^min_s (X φ) = min

(a,µ)∈Steps(s)

X

s⁰∈Sat(φ)

µ(s⁰)

,

TheP./p[φ1U^≤k φ2]operator. Like the DTMC case, we partitionSintoS^no,S^yesandS^?: S^no=S\(Sat(φ1)∪Sat(φ2)), S^yes=Sat(φ2), S^?=S\(S^no∪S^yes),

and computep^max_s andp^min_s recursively.S^nocontains states from which no path satisfiesφ1U^≤kφ2under any adversary, andS^yescontains states from which all paths satisfyφ1U^≤k φ2under all adversaries.

p^max_s (φ₁U^≤kφ₂) =











0 if(s∈S^no)∨(s∈S^?∧k= 0)

1 ifs∈S^yes

max

(a,µ)∈Steps(s)

X

s⁰∈Sat(φ)

µ(s⁰)·p^max_s (φ1U^≤(k−1)φ2)

ifs∈S^?∧k >0,

p^min_s (φ₁U^≤kφ₂) =











0 if(s∈S^no)∨(s∈S^?∧k= 0)

1 ifs∈S^yes

min

(a,µ)∈Steps(s)

X

s⁰∈Sat(φ)

µ(s⁰)·p^min_s (φ₁U^≤(k−1)φ₂)

ifs∈S^?∧k >0,

TheP./p[φ1 U φ2]operator. We partition S intoS^no, S^yes andS^? for this operator too. But we use different procedures to compute these three sets forp^max_s andp^min_s separately. For^max_s , we have

S^no = P rob0A(Sat(φ1), Sat(φ2)), S^yes = P rob1E(Sat(φ1), Sat(φ2)),

S^? = S\(S^yes∪S^no).

and forp^min_s , we have

S^no = P rob0E(Sat(φ1), Sat(φ2)), S^yes = Sat(φ2),

S^? = S\(S^yes∪S^no).

AlgorithmP rob0Afirst computes the set of states, each of which either satisfiesφ2, or satisfiesφ1and with non-zero probability, can reach a state inSat(φ2)without leavingφ1 under any adversary. Then it returns the complement of this set underSasS^no. For each states∈S^no, no path inP ath^A_s satisfies the formulaφ₁U φ₂with non-zero probability under any adversary.

Algorithm P rob1E computes a double fix point to return the set S^yes of states, each of which has p^A_s(φ₁U φ₂) = 1for some adversary. The outer loop identifies states from which no adversary can make p^A_s(φ₁U φ₂) = 1, and remove those states fromS. The inner loop collects states from which one cannot reach a state inSat(φ2)without passing through either a state not insat(φ1)or a state already removed fromS.

For p^min_s , states in S^no have p^A_s(φ1 U φ2) = 0 for some adversary. AlgorithmP rob0E computes a subset of Sat(φ1)such that with non-zero probability, each state can reach a state in Sat(φ2)without

Algorithm 3P rob0A(Sat(φ₁), Sat(φ₂))

1: R:=Sat(φ2)

2: done:=false

3: whiledone=falsedo

4: R⁰:=R∪ {s∈Sat(φ1)| ∃(a, µ)∈Steps(s).∃s⁰ ∈R . µ(s⁰)>0}

5: ifR⁰ =Rthen

6: done:=true

7: end if

8: R:=R⁰

9: end while

10: return S\R

Algorithm 4P rob1E(Sat(φ₁), Sat(φ₂))

1: R:=S

2: done:=false

3: whiledone=falsedo

4: R⁰:=Sat(φ2)

5: done⁰:=false

6: whiledone⁰=falsedo

7: R⁰⁰:=R⁰∪ {s∈Sat(φ1)| ∃(a, µ)∈Steps(s).(∀s⁰ ∈S . µ(s⁰)>0→s⁰∈R)∧(∃s⁰ ∈R⁰. µ(s⁰)>

0)}

8: ifR⁰⁰=R⁰ then

9: done⁰ :=true

10: end if

11: R⁰:=R⁰⁰

12: end while

13: ifR⁰ =Rthen

14: done:=true

15: end if

16: R:=R⁰

17: end while

18: return R

Algorithm 5P rob0E(Sat(φ₁), Sat(φ₂))

1: R:=Sat(φ2)

2: done:=false

3: whiledone=falsedo

4: R⁰:=R∪ {s∈Sat(φ1)| ∀(a, µ)∈Steps(s).∃s⁰ ∈R . µ(s⁰)>0}

5: ifR⁰ =Rthen

6: done:=true

7: end if

8: R:=R⁰

9: end while

10: return S\R

passing a state outside the subset. It then returns the complement of the union of the subset andSat(φ2) underS. The setS^yesshould contain all states such thatp^A_s(φ₁U φ₂) = 1for every adversary. To simplify the computation, we use the setSat(φ₂), which satisfy the condition trivially.

After we obtainS^no,S^yesandS^?, we can computep^max_s andp^min_s forS^?by solving a linear optimisation problem over the set of variables{x_s|s∈S^?}. Lettingx_sbep^max_s , we computep^max_s by minimising P

s∈S^?

x_s

under the constraints x_s≥ X

s⁰∈S^?

µ(s⁰)·x_s⁰+ X

s⁰∈S^yes

µ(s⁰) for alls∈S^?and all(a, µ)∈Steps(s).

Lettingx_sbep^min_s , we computep^min_s by maximising P

s∈S^?

x_sunder the constraints xs≤ X

s⁰∈S^?

µ(s⁰)·xs⁰+ X

s⁰∈S^yes

µ(s⁰) for alls∈S^?and all(a, µ)∈Steps(s).

Dans le document Capturing functional and non-functional connector (Page 72-79)