Algorithm 7Refine(Z,R,(l, ζ),Θlb,Θub)
1: ζlb:=valid(Θlb)
2: ζub:=valid(Θub)
3: Znew:={(l, ζlb),(l, ζub),(l, ζ∧ ¬(ζlb∨ζub))}\{∅}
4: Zref := (Z\{(l, ζ)})]Znew
5: Rref :=∅
6: for allθ= (z0, a,hz1, . . . , zni)∈Rdo
7: if(l, ζ)6∈ {z0, z1, . . . , zn}then
8: Rref :=Rref ∪ {θ}
9: else
10: Θnew:={(z00, a,hz10, . . . , zn0i)|zi0∈Znewifzi= (l, ζ)andz0i=zi otherwise}
11: for allθnew∈Θnewsuch thatvalid(θnew)6=∅do
12: Rref :=Rref ∪ {θnew}
13: end for
14: end if
15: end for
16: return (Zref,Rref)
This refinement scheme, applied in a iterative manner, provides a way of computing exact values for minimum or maximum reachability probabilities of a PTA. This algorithm, outlined in Algorithm 8, starts with the reachability graph constructed through forwards reachability and then repeatedly:
1. builds a stochastic game;
2. solves the game to obtain lower and upper bounds;
3. refines the reachability graph, based on an analysis of the game.
The iterative process terminates when the difference between the bounds falls below a given level of precision ε. In fact, as the following result states, this process is guaranteed to terminate, in a finite number of steps, with the precise answer.
Algorithm 8AbstractRefine(T, F, ?, ε)
1: (Z,R) :=BuildReachGraph(T, F)
2: G:=BuildGame(Z,R)
3: (plb,?G , pub,?G , σ1lb, σub1 :=AnalyseGame(G, F, ?)
4: whilepub,starG −plb,?ttG > εdo
5: choose(l, ζ)∈Z
6: (Z,R) :=Refine(Z,R,(l, ζ), σlb1(l, ζ), σ1ub(l, ζ))
7: G:=BuildGame(Z,R)
8: (plb,?G , pub,?G , σlb1, σub1 :=AnalyseGame(G, F, ?)
9: end while
Theorem 3.18 LetT be a PTA with targetFand?∈ {min,max}. The algorithmAbstractRefine(T, F, ?,0) (see Algorithm 8) terminates after a finite number of steps and returns[plb,?G , pub,?G ]whereplb,?G =p?T(F) = pub,?G .
Algorithm 9BuildReachGraph(T, F)
1: Z:=∅
2: Y:={tsuc(l,0)}
3: whileY6=∅do
4: choose(l, ζ)∈Y
5: Y:=Y\{(l, ζ)}
6: Z:=Z∪ {(l, ζ)}
7: for alla∈Actsuch thatenab(l, a)∧ζ6=∅do
8: for allei∈edges(l, a) =he1, . . . , enido
9: (l0i, ζi0) :=post[(l, a), ei](l, ζ
10: if(l0i, ζi0)6∈Zandl0i∈F then
11: Y:=Y∪ {(l0i, ζi)}
12: end if
13: end for
14: end for
15: end while
16: return (Z,R)
choose the one that guarantees the reliability with minimum cost. Many non-functional properties are either instantaneous or cumulative properties. The former means the expected value of the property at some time point, e.g., the expected number of message delivered by the message warning server after exactly 90 seconds; while the latter means the expected cumulated value over some period, e.g., the ex-pected time for the client registration process to terminate. Usually, non-functional properties are handled by extra labels in probabilistic models. In this section, we present the techniques to verify non-functional properties in DTMCs, MDPs and CTMCs.
The sets of labels used to handle non-functional requirements are often known ascost structures(also reward structures). Adding a cost structure to a DTMC, we obtain a labelled DTMC.
Definition 3.19 AlabelledDTMC is a tuple(D, C)where
• D= (S, s,P, L)is a DTMC;
• C:S×S→R≥0is acost structure.
The cost structure assigns every transition a real number. In Section 3.1.1, we have defined the probability measureP robsfor paths in DTMCs. Given a target setF of states, we can useP robsto define the cost cost(F)(ω)of reaching a state inF along a pathω∈P aths.
cost(F)(ω)def=
{j|ω(j)∈F}
X
i=1
C ω(i−1), ω(i)
if∃j ∈N. ω(j)∈F
∞ otherwise
LetEs(cost(F))be the expected cost fromsto states inF. It is the expectation of the cost of reaching any state inF via any path inP aths. Since the cost is defined to be∞if a path does not pass any target state, the expected cost would be∞if the states inF cannot be reached fromswith probability 1.
To verify nonfunctional requirements, PCTL has been extended to include an operator to reason about expected costE.
Definition 3.20 The syntax of extended PCTL is as follows:
φ::=true|a| ¬φ|φ∧φ| P./p[ψ]| E./c[φ], ψ::=X φ|φU≤kφ|φU φ whereais an atomic proposition, and./∈ {≤, <,≥, >},p∈[0,1],c∈R≥0andk∈N.
The formula E./c[φ] holds in state s if starting from s, the expected cost to reach a state satisfying φ satisfies the condition./ c. Formally,
s|=E./c[φ] iffes(φ)./ c
wherees(φ) def= Es cost({s0 ∈ S | s0 |= φ)
. Model checking procedure Sat(E./c[φ]) returns the set of states, each of which satisfies the condition./ cwith respect to the expected cost of reachingφstates, i.e.,
Sat(E./c[φ]) ={s∈S|es(φ)./ c}.
To compute the expected cost for each state inS, we partitionS into three setsS0,S∞andS?. States in S0have the expected cost 0, which simply means these states satisfyφ. States inS∞have the expected cost ∞, i.e., they cannot reach the φ states with probability 1. This set is computed using Algorithms P rob1andP rob0defined in Section 3.1.1.
S0 = Sat(φ), S∞ = S\P rob1
S, Sat(φ), P rob0 S, Sat(φ) , S? = S\(S0∪S∞).
Computinges(φ)for states inS?is done by lettingxs=es(φ)for alls∈S?and solving the linear equation system:
xs= X
s∈S?
P(s, s0)· C(s, s0) +xs0
.
The cost structure in MDP is slightly different from the one in DTMC. We name itcost function.
Definition 3.21 A labelled MDP is a tuple(M,c)where
• Mis an MDP,
• c:S→R≥0is the cost function.
The cost function gives each pair of state and action a cost. In this case, all transitions labelled with the same action in a probability distribution have the same cost. Given a adversaryAwith the measure P robAS, the expected cost of reaching a setF of target states,EsA cost(F)
, is computed based on the cost of pathsω∈P athAs:
cost(F)(ω)def=
min{i|ω(i)∈F}
X
i=0
c step(ω, i−1)
if∃i∈N. ω(i)∈F
∞ otherwise.
The extended PCTL for DTMC can be used to reason about cost-related properties in MDP as well.
However, the semantics ofE./c is different: E./c[φ] holds in state s if and only if the expected cost of reaching theφstates fromsis satisfied for./ cfor all adversaries. We have
s|=E./c[φ] iffeAs(φ)./ cif for allA∈AdvM, whereeAs(φ)def= EsA cost({s0∈S|s0|=φ)
for any adversaryA∈AdvM.
To model checkE./c[φ], we need to deal with{<,≤}and{>,≥}separately. For./is<or≤, Sat(E./c[φ]) ={s∈S|emaxs (φ)./ c};
For./is>or≥,
Sat(E./c[φ]) ={s∈S|emins (φ)./ c}.
The technique to computeemaxs (φ)andemins (φ)is similar to the one forP./c[φ]. We first partitionS into S0,S∞ andS?. In both cases,S0contains all states that satisfyφ, and therefore, have expected cost 0.
Foremaxs (φ), S∞contains states that havepAs(φ) <1 for some adversary A, and can be computed by the model checking procedure in Section 3.1.1 for formula¬P≥1[φ]; Foremins (φ),S∞contains states that havepAs(φ)<1for all adversariesA, and is computed by model checkingP<1[φ].
The computation of emaxs for S? = S\(S0∪s∞) can be done in two ways. One is to solve a linear optimisation problem over variables{xs|s∈S?}(emaxs =xs): Minimising P
s∈S?
xsunder the constraints xs≥c(a) + X
s0∈S?
µ(s0)·xs0
for alls∈S?and all(a, µ)∈Steps(s). The other way computesemaxs = limn→∞emax(n)s iteratively where
emax(n)s =
∞ ifs∈S∞
0 ifs∈S0
0 ifs∈S?andn= 0
max
(a,µ)∈Steps(s)
c(a) + X
s0∈S?
µ(s0)·emax(n−1)s
ifs∈S?landn >0.
Similarly, the computation ofemins forS?can be done by maximising P
s∈S?
xsunder the constraints xs≤c(a) + X
s0∈S?
µ(s0)·xs0
for alls∈S?and all(a, µ)∈Steps(s), or computingemins = limn→∞emin(n)s iteratively where
emin(n)s =
∞ ifs∈S∞
0 ifs∈S0
0 ifs∈S?andn= 0
min
(a,µ)∈Steps(s)
c(a) + X
s0∈S?
µ(s0)·emin(n−1)s
ifs∈S?landn >0.
The extended PCTL fro DTMCs/MDPs does not distinguish instantaneous and cumulative rewards because we can push cost assigned to states to transitions without weakening the power of the cost structure. For CTMCs, however, we added two different types of cost for instantaneous and cumulative rewards separately due to the continuous probability distributions. Every transition is associated with an instantaneous cost and every state has a cumulative cost.
Definition 3.22 A labelled CTMC is a tuple(C,C,c)where
• C= (S, s,R, L)is a CTMC,
• C:S×S→R≥0is aninstantaneous cost function,
• c:S→R≥0is acumulative cost function.
In the above definition,C(s, s0)is the actual cost incurred when the system moves from statesto states0; whilec(s)is the coefficient, at which cost is incurred in states, for the amount of timetspent ins, i.e., the actual cost would bec(s)·t. As for DTMCs and MDPs, we can define the expected cost of reaching a set of target statesF through pathsω∈P aths:
cost(F)(ω)def=
min{j|ω(j)∈F}
X
i=1
C ω(i−1), ω(j)
+c ω(i−1)
·time(ω, i−1) if∃j ∈N. ω(j)∈F
∞ otherwise.
We define the cost for a path that does not pass any target state to be∞too. Thus, the expected cost of reaching a state inFfrom statesis finite if all non-zero probability paths starting fromspass a state inF.
We extend CSL to include formulae for rewards now, as we did for DTMCs and MDPs.
Definition 3.23 The syntax of CSL is as follows:
φ::=true|a| ¬φ|φ∧φ| P./p[ψ]| S./p[φ]| E./c[φ], ψ::=X φ|φUIφ|φU φ whereais an atomic proposition,./∈ {≤, <,≥, >},p∈[0,1],c∈R≥0andIis an interval ofR≥0.
Similarly to theE./coperator for DTMCs and MDPs,E./c[φ]in CSL means that the expected cost to reach a state satisfyingφmeets the bound./ cexcept that the expected cost here involves instantaneous and cumulative costs.
LetEsdenote the expectation with respect to the measureP robs. The satisfaction ofE./c[φ]is defined as follow:
s|=E./c[φ]iffes(φ)./ c, wherees(φ)def= Es cost({t∈S|t|=φ})
. The corresponding model checking procedure is based on the embedded DTMC. We first identify the setsS0,S∞ andS? of states. The states inS0 havees(φ) = 0, and simply are the states that satisfyφ. The setS∞contains states from which the probability of reaching aφstate is less than 1. This set is computed in the same manner as the computation ofS∞forE./c[φ]
in PCTL. The set S? = S\(S0 ∪S∞) is computed by solving the linear equation system in variables {xs|s∈S?}:
xs=
X
s∈S?
P(s, s0)· C(s, s0) +xs0
+ 1
E(S)·c(s) and then lettinges(φ) =xsfors∈S?.