Any correspondence concerning this service should be sent
to the repository administrator:
tech-oatao@listes-diff.inp-toulouse.fr
This is an author’s version published in:
http://oatao.univ-toulouse.fr/19149
Official URL:
https://ieeexplore.ieee.org/document/7991957
DOI :
https://doi.org/10.1109/WFCS.2017.7991957
Open Archive Toulouse Archive Ouverte
OATAO is an open access repository that collects the work of Toulouse
researchers and makes it freely available over the web where possible
To cite this version:
Wang, Qi and Jaffrès-Runser, Katia and Xu,
Yongjun and Scharbarg, Jean-Luc A certifiable resource allocation
for real-time multi-hop 6TiSCH wireless networks. (2017) In: 13th
IEEE International Workshop on Factory Communication Systems
(WFCS 2017), 31 May 2017 - 2 June 2017 (Trondheim, Norway).
A certifiable resource allocation for real-time
multi-hop 6TiSCH wireless networks
Qi Wang ∗, Katia Jaffr`es-Runser†, Yongjun Xu ∗, Jean-Luc Scharbarg†
∗Institute of Computing Technology, Chinese Academy of Sciences, Beijing, CHINA Email:{wangqi08, xyj}@ict.ac.cn
†Universit´e de Toulouse, IRIT / ENSEEIHT, F-31061, Toulouse, FRANCE Email:{katia.jaffres-runser, jean-luc.scharbarg}@irit.fr
facilitating multi-hop operations, and is able to cope efficiently with external interference and multiple fading channels. The clean layering allows IEEE802.15.4e TSCH to fit under an IPv6-enabled protocol stack for low-power wireless networks, which was recently proposed by the IETF 6TiSCH working group.
The IETF 6TiSCH working group has defined the 6TiSCH operation Sublayer (6top) [4] as the next upper layer of the TSCH MAC (IEEE802.15.4e MAC-TSCH). This 6top sublayer manages the way communication resources are scheduled in time and frequency, while monitoring performance and collecting statistics at each node to further optimize routes and channel access. To allow maximum flexibility, the scheduling policy is not standardized by 6TiSCH. Scheduling can thus be implemented in a centralized or a distributed manner. Schedules can be pre-calculated or adjusted dynamically to new demands. Several scheduling policies for 6TiSCH have been designed, but only few of them offer a bounded worst case delay calculation method. Domingo-Prieto et al. [5] propose a distributed and dynamic scheduling policy where each node determines the number of cells to schedule ac-cording to its traffic demand. Duquennoy et al. propose a best-effort decentralized scheduling approach named Orchestra [6] that randomly allocates slots without requiring control messages to be exchanged by nodes. Recently, Hosni et. al. [7] have proposed a distributed algorithm to schedule the transmissions while upper bounding the end-to-end delay by the slot frame size. More specifically, their solution guarantees that the maximum end-to-end delay equals to the slot frame duration. Their scheduling policy reduces the end-to-end delay by allocation more cells to a communication but this over-provisioning increases both the number of collisions and the energy consumption.
This paper proposes a novel certifiable resource allocation algorithm for multi-hop 6TiSCH wireless networks that offers high communication reliability, low end-to-end delay and low energy consumption for safety-related industrial applications. Our scheduling policy pre-calculates the schedules using a novel multi-objective optimization framework where network reliability, end-to-end delay and energy consumption are opti-mized for each multi-hop flow. Our schedule is directly derived from the performance metrics of the flows, all extracted from a cross-layer Markov-based performance model of the multi-hop wireless network. From this cross-layer framework, we can derive the end-to-end delay distribution for each flow. This distribution can be leveraged to calculate a probabilistic
Abstract—In safety-related industrial systems, stringent
time-lines and a high degree of reliability in communications are
required. Networking protocols have to be certified using a
mathe-matical framework that ascertains the end-to-end communication
delay of flows is upper bounded. The recently created IETF
6TiSCH networking stack is a promising candidate to offer
real-time communication services as communications are scheduled
at the link layer in time and frequency. Its 6top sublayer
manages the way communication resources are scheduled and
thus messages routed. This paper proposes a novel certifiable
resource allocation algorithm for multi-hop 6TiSCH wireless
networks that simultaneously offers high reliability, low
end-to-end delay and low energy consumption for safety-related
industrial applications. The schedules are defined such as to
directly derive a bound on the worst-case end-to-end delay.
Schedules are fixed and computed by solving a multi-objective
optimization problem. Two different problems are defined. The
first one designs a deterministic schedule and the other one a
probabilistic schedule where node are assigned a probability to
forward a message in a TSCH cell. We show in our results that the
latter method offers better performance in terms of robustness
as possible losses due to fading are compensated for by cyclic
path redundancies.
Keywords—resource allocation, probabilistic worst-case
end-to-end delay, network reliability, TSCH
I. INTRODUCTION
Wireless multi-hop networks (i.e., sensor and mesh net-works) will be increasingly employed in safety-related indus-trial systems, such as factory automation, industrial surveil-lance and monitoring of nuclear plants, due to their appealing ease of deployment and scalability. In order to ensure safety, time evolution of such systems is constrained by strict dead-lines and requires a high degree of reliability in communica-tion. For wireless networks to be deployed in such systems, deterministic wireless communications have to be guaranteed.
Several wireless protocols have been designed recently to offer mainly real-time garanties: WirelessHART [1], ISA100.11a [2], and IEEE802.15.4e [3]. They are based on wireless short-range communication technologies and offer time-slotted channel hopping medium access. While Wire-lessHART and ISA100.11a define a complete protocol stack, IEEE802.15.4e only defines the physical layer and three dis-tinct Medium Access Control (MAC) layers: Low Latency De-terministic Network (LLDN), time-frequency blocked Channel Hopping (TSCH), and Deterministic and Synchronous Multi-channel Extension (DSME). Among these, TSCH is the one
worst-case end-to-end delay bound and thus, properly certify the communication delay.
This novel framework is the extension of our previous works [8], [9], [10] to the 6TiSCH scheduling problem. Unique to the present work, is the definition of the closed-form expres-sions for network reliability which measures the probability to receive one frame over a multihop path. Another addition is the derivation of the delay distribution of the first arrived frames. Indeed, multi-path communications can be defined in our framework and previous delay distributions would be computed for all received messages, including redundant copies traveling on different paths. Derivations of this paper only account for the frames arrived first, and not for subsequent copies.
This paper underlines two different implementations of our framework. The first one is fully deterministic in its essence as scheduling decisions are deterministic. The second one is stochastic as scheduling decisions are given by forwarding probabilities. Unique to this stochastic scheduling policy, re-dundancy can be adjusted such as to prove, given a realistic channel model, that maximum reliability is achieved and end-to-end delays are bounded. This redundancy is enforced naturally by the stochastic decisions of the nodes resulting in copies being carried over cyclic paths in the network. In this paper, we highlight in our results how cyclic paths can positively impact the overall performance of our scheduling under bad channel conditions.
To summarize, the main contributions of our work are the following ones: i) the definition of a wireless multi-hop (possibly multi-path) cross-layer network performance frame-work that captures reliability, end-to-end delay and energy consumption of flows, ii) a 6TiSCH offline static scheduling multi-objective (MO) algorithm that optimizes concurrently the aforementioned performance metrics andiii) a comparison of a deterministic and a probabilistic schedule implementation of our scheme for two core multi-hop networking topologies.
This paper is organized as follows. Section II introduces our system model and the node scheduling algorithm. Section III introduces our cross-layer model and the related reliability, end-to-end delay and energy performance metrics. Calculation of the end-to-end delay distribution is given as well for a given flow. Section IV formulates the MO optimization prob-lem that derives optimal forwarding probabilities for resource allocation. Section V validates the performance of algorithm against simulation for the deterministic and stochastic study cases. Section VI concludes this paper.
II. SYSTEM MODEL AND ALGORITHM A. Multihop topologies
In this work, we still investigate two different atomic topologies [9] that are at the core of many deployment scenar-ios of wireless multi-hop communications. We will concentrate first on a linear deployment scenario as shown in Fig. 1(a). In this example, relayR1can only communicate withR2in both
ways. R1 can not send frames toS and D can only receive
frames from R3. The second 2-flow / 2-relay topology has
been investigated to capture the contention effect of a set of nodes carrying multiple flows. The first flow is emitted by S1
S R1 R2 R3 D pR1R2 pR2R3
pR2R1 pR3R2
pR3D
pSR1
(a) 1-flow / 3-relay topology
S2 D2 S1 D1 R2 R1 pR1R2 pR2R1 pR2D2 pS2R1 pR2D1 pS1R1
(b) 2-flow / 2-relay topology
Fig. 1. Investigated topologies: (a) Linear topology composed of one flow and 3 relay nodes ; (b) Crossflow topology with 2 flows.
going to D1 and the second flow is emitted by S2, going to
D2. This situation is critical since the relays have to listen to
both flows and to re-emit them concurrently.
Source nodes generate strictly periodic flows of frames. As explained later in Section V, the frame generation period is set such as to have only one frame in the emission buffer of the source at all times. Thus, the only delay we are computing analytically or measuring by simulation is the MAC delay, i.e. the time for the frame to gain access to the channel. There is no delay related to queuing in this work.
B. Network model and algorithm
IEEE802.15.4e [3] is an amendment to the MAC protocol of IEEE802.15.4-2011 [11]. The Time Synchronized Chan-nel Hopping (TSCH) mode of IEEE802.15.4e yields ultra-high reliability and low-power operations. All sensor in a IEEE802.15.4e network are synchronized. Time is split into time slot, each typically 10ms long. Time slots are grouped into a slotframe, which continuously repeats over time, as shown in Fig. 2. TSCH doesn’t impose a slotframe size. Depending on the application needs, the slotframe size can range from 10’s to 1000’s of time slots. In consecutive slotframe cycles, the same channel offset translates into a different communica-tion frequency, resulting in channel hopping. The number of available channels is 16 when using radios that are compliant with IEEE 802.15.4 at 2.4GHz. The TSCH schedule can be represented by a 2-D matrix of width the number of slots in a slotframe, and height the number of available channels. Each element in the matrix is called a “cell” (we use as well the term “time-frequency block” interchangeably in this paper). A cell is scheduled to instruct each node what to do, either to transmit or receive.
As seen in Fig. 1, there are bi-directional links between relays and destinations. It is a way to represent multiple paths, by allowing cyclic paths (loop paths) with different lengths. With path diversity, a frame can travel multiple paths to the destination, giving additional redundancy to further increase the communication reliability.
Each relay node is characterized by a set of forward-ing probabilitiesxuv
ij, which govern its forwarding decisions.
These forwarding probabilities are optimized later to define a global schedule at using Algorithm 1. Assuming node j receives a frame from nodei in the current slotframe s on cell u, it decides to forward this frame on cell v of stotframe s + 1
with probability xuv
ij. As such, a frame may travel of one hop
in the duration of at most one slotframe. Nodej only re-emits at most once a frame received in slot u asP
vxuvij ≤ 1. In
caseP
vxuvij < 1, the frame may be dropped with probability
1−P
vxuvij.
We will first investigate the performance of scheduling pol-icy by optimizing binary forwarding probabilities (i.e. chosen to be 0 or 1). This optimisation step creates a deterministic schedule. Secondly, the search is performed for real forwarding probabilities that belong to the continuous set [0,1]. In this implementation, the schedule is stochastic. We show that in this context, the definition of cyclic paths can further improve reliability by adding path diversity to the multi-hop transmission.
Algorithm 1 Resource allocation of node j in cell u for a multihop 6TiSCH wireless networks.
ifmemory of cellu not empty then Send the frame;
else
if node j receives a frame p from i in time-frequency block u then
Generate a random valuex∈ [0, 1];
foreach time-frequency blockv∈ T , v 6= u do if v==1 andx≤ xu1
ij then
Store the framep into memory of time-frequency block1;
else
if xu(v−1)ij ≤ x ≤ xuvij then
Store the frame p into memory of time-frequency blockv; end if end if end for end if end if
Each node of the network is represented as well by its emission rate in a slot. The emission rate τu
i represents the
proportion of time a node i is active in a slot u, in average. For instance, a value ofτu
i = 0.5 means that node i is emitting
in slotu every two slotframes. Source nodes have an emission rate of 1 in time-frequency block 1 for both topologies.
To ensure flow conservation and capture the cross-layer effect, relay nodes have an emission rate that is proportional to the amount of frames they receive from other nodes and their respective forwarding probabilities. The amount of frames
Fig. 2. IEEE802.15.4-TSCH mode - illustration of a slotframe with 5 time slots and number of 3 channel.
they receive is a function of the number of frames other nodes have sent, and the link quality over which they have been received. The link quality is quantified by a channel probability pu
ij that represents the chances for a frame to be
properly received in time-frequency block u over link (i, j). This channel probability is derived from the average frame Error Rate (PER) on link(i, j) and from the emission rates of sending nodes. (cf. [8] for details). Only interference of nodes that share the same time-frequency block are accounted for in our model. To calculate the PER, any channel model can be fed into our calculations to retrieve realistic results.
Flow conservation is ensured if the following set of |E| equations are valid:
X (i,j)∈−N−→u j X u∈T τiupuijxuvij = τjv, ∀(j, v) ∈ E (1) whereτu
ipuij is the probability that a frame sent byi on
time-frequency blocku arrives in j.|E| is given by the number of edges times the number of time-frequency blocks. −→Nv
j is the
set of outgoing edges coming into nodej from any node i on u. We refer the reader to [9] for more details.
III. ANALYTICAL PERFORMANCE FRAMEWORK This section introduces our framework that helps us in defining performance criteria (e.g. reliability, end-to-end delay and energy consumption) and calculate the end-to-end delay distribution for any flow of the network. This framework derives a mathematical Markov-like network model that ac-counts for the forwarding probabilities (i.e. our scheduling decisions) and the wireless channel statistics associated to these forwarding probabilities. It is illustrated for the 2-flow / 2-relays topology of Fig. 1.
A. Relaying and arrival matrices
The relaying matrix Q gives the probabilities for any emission (i, u) in slotframe s to be emitted as (j, v) at the following slotframe(s + 1) by the relays of the networks. The arrival matrixA is composed of the probabilities to go from any transient state to any absorbing state, i.e. the probabilities for any emission(i, u) at slotframe s to arrive at a destination Dj in time-frequency block v at slotframe (s + 1).
1) The relaying matrixQ:
Q = 0 Q12 · · · Q1N Q21 0 · · · Q2N .. . ... QN 1 · · · QN −1N 0
0 is a |T |-by-|T | zero matrix representing the fact that node i never forwards a frame to itself. |T | is the number of time-frequency blocks.N is the number of relays. A reduced version of Q can be defined where only the relays that belong to a given path p are accounted for. This sub-matrixQsub(p) keeps
the valuesQij(i, j ∈ p) and sets other elements to 0.
The matrix Qij is a |T |-by-|T | matrix that gives the
possible combinations of time-frequency blocks: Qij= Q11 ij · · · Q 1|T | ij .. . ... Q|T |1ij · · · Q |T ||T | ij (2) where Quv
ij is the probability for a node j to retransmit on
time-frequency block v a frame that has been transmitted by node i on time-frequency block u. From our network model, it equals toQuv
ij = puijxuvij.
2) The arrival matrix A: A = A1D1 · · · A1D|D| .. . ... AN D1 · · · AN D|D|
AiDj is a|T |-by-|T | diagonal matrix whose diagonal elements Au
iDj give the probabilities for a frame transmitted by a node i in time-frequency block u to arrive at destination Dj and
Au iDj = p
u iDj.
QS and AS are the relaying and arrival matrices for the
frames sent by the sources. We have
QS= QS11 .. QS1N . . . . . . QS|O|1 .. QS|O|N AS= AS1D1 .. AS1D |D| . . . . . . AS|O|D1 .. AS|O|D|D| where QSij follows the pattern given by (2) and ASiDj is a |T |-by-|T | diagonal matrix whose diagonal elements are Au
SiDj = p
u
SiDj.|O| is the number of sources. Matrices are illustrated next for a basic schedule defined for the 2-flow / 2-relays topology.
3) Illustration for 2-relay / 2-flow topology: For this sce-nario, we present the matrices for a schedule where only |T | = 4 cells are set: sources S1andS2transmit in cell 1 and
cell 2, respectively ; relaysR1andR2transmit in cell 3 and 4,
respectively. Frames are only allowed to loop between relays R1 andR2. As such, the shortest path is made ofhmin = 3
hops. Relaying Q and arrival matrix A are given as follows (diagonal elements ofQ are directly set to 0 since relays don’t send data to themselves):
Q = 0 Q34 R1R2 Q43 R2R1 0 = 0 p3 R1R2x 34 R1R2 p4 R2R1x 43 R2R1 0 and A = A3 R1D 0 0 A4 R2D = p3 R1D 0 0 p4 R2D
The relaying and arrival matricesQS for frames sent byS are
defined as: QS = Q13 S1R1 Q 14 S1R2 Q23 S2R1 Q 24 S2R2 where Quv SiRj = p u SiRjx uv SiRj. B. Network reliability
From this stochastic network model, we can define several performance criteria [8], [9], [10]. Unique to this paper, we define a network reliability criterion that calculates the probability of a frame to arrive at a destination. It is equivalent to the success rate of a frame sent by the source. In this derivation, we account for the probability for a message to arrive on the direct path using the minimum number of hops and, in case the message doesn’t arrive on this direct path, we allow frames to loop for hmax hops within a cyclic path
to further improve communication robustness. For instance, in the 1-flow / 3-relays topology, frames can either arrive using the direct 4-hop path{(S, R1)(R1, R2)(R2, R3)(R3, D)}. We
can allow frames sent by R2 to be overheard by R1 such
as to be re-emitted by R1 in the next slotframe, providing a
redundant transmission. As such, we can combat fading on the link betweenR1 andR2.
Our reliability criterion is defined with the assumption that at most one cycle located at thekth relay may exist (it means
Rkcan overhear frames emitted byRk+1). For all other relays,
cycles can’t exist: all forwarding probabilities going back to the source are set to zero. Intuitively, it is beneficial to set this cycle at the first relay R1 to compensate for losses arriving
early in the path.
LetP be the set of all possible paths between Si andDj.
Next, we assume that relayRk+1 can overhearRk. Lethpbe
the number of hops of the direct path andhmax the maximum
number of hops allowed in the communication, withhmax ≥
hp. The frame transmission can succeed afterhphops (direct
path), after hp+2 hops if the frame loops in the cyclic path
once, afterhp+2ℓ if it loops in the cyclic pathℓ times.
The reliability for a given path p can be defined as: fR(Si, Dj) = 1−
Y
p∈P
[1− Psuccess(p)] (3)
withPsuccess(p) the probability of success on path p ending
in destinationDj which is given by:
Psuccess(p) = hmax
X
h=hp
P1st(h, p)
whereP1st(h, p) is the probability for the first frame to arrive
in h hops at Di on path p. To clarify notations, p is only
explicitly mentioned inP1stif needed. On the direct path,h =
hp, andhmaxis the maximum number of hops a frame needs at
the steady state of the Markov chain. Frames forwarded by the relays disappear after at most hmax slotframes have elapsed.
Thushmax is determined when the difference of the flow rate
is arbitrarily small: (−→FR(1)Qhmax −−→FR(1)Qhmax−1) <= δ
(e.g.δ = 10−11). −→F
R(1) is the outgoing flow of relays at the
first time frame. It equals to −→FR(1) = Sm· QS, where Sm is
the transmission rate vector of sources.
When h = hp, the first frame to arrive at the destination
uses the direct path. This success probability is given by: P1st(hp) = Sm· IS(Si)· QS· Qsub(p)hp−2· A · ID(Dj)
where IS(Si) (resp. ID(Dj)) is a selection vector of
to source Si (resp. destination Dj) are equal to 1 and the
others are equal to 0. QS,Qsub(p) are source and relaying
matrices for path p. ID(Dj) accumulates the frame arrival
rate in each time-frequency block at destination Dj. Sm is
the vector of emission rates of all sources, where Sm =
h τ1 S1 . . . τ |T | S1 . . . τ 1 S|O| . . . τ |T | S|O| i .
When the first frame to arrive at the destination loops once in the cyclic path, we have h = hp+ 2. In this case, the
transmission of the frame carried by the direct path failed between Rk+1 and Dj, but the transmission of the first copy
created in the loop succeeds. As such, the success probability is in this case :
P1st(hp+ 2) = Psuccess(hp+ 2)· Ploss
with Psuccess(hp + 2) = P1st(hp)· Qk(k+1)ukvk+1Qu(k+1)kk+1vk. The
probability Ploss to loose the first frame between Rk+1 and
Dj is given by:
Ploss= 1− Qu(k+1)(k+2)k+1vk+2 Q(k+2)(k+3)uk+2vk+3 . . . Qu(N −1)(N )N−1vN puN DNj If the first frame to arrive at the destination loopsℓ times in the cyclic path, we have h = hp+ 2ℓ.
P1st(hp+ 2ℓ) = P1st(hp)· Qukvk+1 k(k+1)Q uk+1vk (k+1)k · Ploss ℓ
In this case,ℓ− 1 first frames get lost but the ℓth one arrives.
To summarize, the end-to-end network reliability for the flow emitted atSi and arriving atDj is given by:
fR(Si, Dj) = 1−Qp∈P h 1−Pℓmax ℓ=0 P1st(hp, p)× Ploss· Quk(k+1)kvk+1Qu(k+1)kk+1vk ℓ (4)
withℓmax= (hmax− hp)/2 the maximum number of times a
frame can loop in the cycle located at relayRk.
If several sources emit a flow toDj, the network reliability
fR(Dj) at destination Dj is given by:
fR(Dj) =P|O|i=1fR(Si, Dj) (5)
1) Illustration for 2-relay / 2-flow topology: The reliability criterion is illustrated for this topology. We recall that the cyclic path is located atR1. First,fR(S1, D1) and fR(S2, D2)
are derived as:
fR(S1, D1) = Pℓℓ=0maxτS11Q 13 S1R1Q 34 R1R2p 4 R3D1× !Ploss(D1)· Q43R2R1Q 34 R1R2 ℓ (6) where Ploss(D1) = (1− p4R2D1). fR(S2, D2) = Phh=hmaxpτ 1 S2Q 23 S2R1Q 34 R1R2p 4 R3D2× !Ploss(D2)· Q43R2R1Q 34 R1R2 ℓ (7) where Ploss(D2) = (1− p4R2D2).
The overall reliability for the flows ending at D1 is of
fR(D1) = fR(S1, D1) + fR(S2, D1). A similar derivation
holds forfR(D2).
C. Delay metric and delay distribution
In our model, the delay is measured in hops. A frame may travel over several paths, each one of different length in number of hops. It takes at most one slotframe duration for the frame to travel one hop further. So all metrics of delay are expressed in hops, and can be easily converted in time units by multiplying them by M× ςslot, with M the slotframe size and ςslot the
slot duration.
The end-to-end delay distribution P [dj = h] is the
prob-ability for a transmission towards Dj to be done inh hops.
Thus, from previous derivations, we have the probability mass function (PMF) for the end-to-end delay given by
P [dj = h] = 1−Q p∈P[1− P1st(hp, p)] /fR(Si, Dj) h = hp (1−Q p∈P(1− P h−hp 2 ℓ=0 P1st(hp, p)× Ploss· Quk(k+1)kvk+1Qu(k+1)kk+1vk ℓ )/fR(Si, Dj) ∀h > hp (8) Here, redundant frame copies that successfully arrive atDjare
not accounted for. The average end-to-end delay fD(Si, Dj)
is computed by: fD(Si, Dj) = hmax X h=1 h∗ P [dj= h] (9)
1) Illustration for 2-relay / 2-flow topology: Since it is a symmetric topology, the delay distribution is given by
P [d1= h] = P [d2= h] = P1st(hp)/fR(D1) h = hp ((τ1 S1Q 13 S1R1+ τ 2 S2Q 13 S2R2)Q 34 R1R2[Ploss(D1)· Q43 R2R1Q 34 R1R2) h−(N +1) 2 p4 R2D1 i /fR(D1) ∀h > hp (10) D. Worst-case delay
A stochastic worst-case delay bound [9] for the flow ending at destinationDj is defined as the delaydw for which
the probability P [dj ≥ d] to find a delay larger than dw
is arbitrarily small (for instance smaller than δ = 10−9).
Formally, for a flow ending atDj:
dw = max d s.t. P [d
j≥ d] ≤ δ (11)
If several flows exist, it is possible to calculate the worst-case delay for all the flows, which is given by the maximum of the dw values calculated for all possible destinations.
E. Energy Consumption
We consider as a first approximation that the main energy consumption factor is due to the emission and reception of a frame. Thus, the energy criterionfE is defined as the average
number of emissions and reception operations performed by all nodes (sources and relays). This value is normalized by the number of frames sent by the sources. The complete definition is given in [9].
18 19 20 21 22 23 24 25 10 11 12 13 14 15 f rE frD Br
opt from simulation, 2 relays topology (PER=0.5)
Br
opt from analysis, 2 relays topology (PER=0.5)
(a) Pareto optimal bound for 2-flow 2-relay wireless networks 11 11.2 11.4 11.6 11.8 12 12.2 12.4 6.02 6.04 6.06 6.08 6.1 6.12 f rE frD Br
opt from simulation, 3 relays topology (loop R1-R2 and PER=0.1) Br
opt from analysis, 3 relays topology (loop R1-R2 and PER=0.1)
Smin
Smiddle Smax
(b) Pareto optimal bound for 1-flow 3-relay wireless networks 16 16.5 17 17.5 18 18.5 19 19.5 20 11.6 11.8 12 12.2 12.4 12.6 12.8 f r E frD Br
opt from simulation, 3 relays topology (loop R1-R2 and PER=0.25) Br
opt from analysis, 3 relays topology (loop R1-R2 and PER=0.25)
Smin
Smiddle
Smax
(c) Pareto optimal bound for 1-flow 3-relay wireless networks
Fig. 3. Reliability-achieving Pareto optimal bounds. x-axis (resp. y-axis) represents the average duration in milliseconds (resp. in Joules) for one frame received at the destination, where forwarding probability value chosen in set [0,1].
IV. MOOPTIMIZATION PROBLEM
The resource allocation we propose in this paper is obtained by selecting the forwarding probabilities of all relays by solv-ing the MO optimization problem introduced hereafter. This problem concurrently minimizes the Reliability-achieving end-to-end delay and Reliability-achieving energy metrics. Both metrics are introduced first.
A. Reliability-achieving criteria
Previously defined end-to-end delay and energy metrics are normalized by the reliability criterion fR. As such, we derive
metrics that measure the end-to-end delay and energy it would cost to achieve a perfectly reliable communication. Formally, they are defined for one flow arriving at destination Dj by:
Reliability achieving delay fr D:
fDr(Dj) = fD(Dj)/fR(Dj) (12)
Reliability achieving energyfr E:
fr
E(Dj) = fE(Dj)/fR(Dj) (13)
A global metric is proposed that combines the performance of all flows: fDr = max j∈|D|f r D(Dj) and fEr = 1 |D| |D| X j=1 fEr(Dj) (14)
The maximum end-to-end delay over all flows is minimized in fr
D and the average energy consumption is minimized in fEr.
B. MO Optimization problem
In our MO problem, we optimize the forwarding proba-bilities represented by X ∈ Xτ. Each solution X ∈ Xτ can
be evaluated according to fR, fD or fE. In this paper, the
following multi-objective optimization problem is solved: [min fr
D(x), min fEr(x)] T
(15) s.t. x = (τ, X)∈ Γ × Xτ
In this work, we define a Pareto-optimal reliability-achieving boundBr
optby the set of points given by
Br
opt={(fDr(x), fEr(x)) ∀x ∈ Sopt}
This bound andSoptare obtained by solving (15), withSopt
the set of Pareto-optimal solutions. Each solution is defined by
the values of the forwarding probabilities of the relay nodes. Two different implementations are studied in this work. The first one looks for binary values. In this case, the schedules obtained after MO optimization are deterministic. The second implementation looks for real-valued probabilities that belong to the [0,1] continuous set. Schedules are as such stochastic and each relay runs Algorithm 1 using a set of Pareto-optimal forwarding probabilities.
V. RESULTS
Deterministic and stochastic schedules are computed for the topologies of Fig. 1. For the deterministic schedule deriva-tion, no cycles are allowed (reverse path forwarding probabil-ities are set to 0). For the stochastic schedule derivation, one loop may be exploited by our framework. For both approaches, stochastic worst-case delay bounds are presented to exhibit how our framework can be leveraged for certification purposes. Imperfect channel conditions are mostly considered to high-light the impact of cyclic paths on the overall performance.
A. Simulation settings
1) Network settings: Results are given assuming an Ad-ditive White Gaussian Noise (AWGN) channel and a Binary Phase Shift Keying (BPSK) modulation without coding provid-ing a bit error rate ofBER(γ) = Q(√2γ) = 0.5∗ erfc(√γ), with γ the per bit signal to noise and interference ratio experienced on the link and erf c the complementary error function. For a specific value of SINRγ, the packet error rate P ER is computed with P ER(γ) = 1− [1 − BER(γ)]Nb where Nb is the number of bits of a data frame. Any channel
model can be exploited in our framework.
Interference is accounted for in our channel model if two relays or more are chosen to forward frames in the same cell by our optimization search. For instance, for the 1-flow 3-relay topology, we have|T | = 4 time-frequency blocks to allocate. In our search, source S can only transmit in cell 1, R1 may
transmit in cell 2 or 3; relayR2in cell 3 or 4; and relayR3in
cell 3 or 1. For the 2-flow topologies,|T | = 4 time-frequency block have to be allocated as well. The sourcesS1andS2only
emit in cell 1 and 2, respectively. The relayR1 may transmit
in block 3 or 4; and relayR2 in block 4 or 1.
For all topologies, the results are given for a slotframe duration of 3 slots. With 2 channels, there are hence 6 cells in
18 19 20 21 22 23 24 25 10 11 12 13 14 15 f r E frD Br
opt from simulation, 2 relays topology (PER=0.5)
Br
opt from analysis, 2 relays topology (PER=0.5)
(a) Pareto optimal bound for 2-flow 2-relay
6 8 10 12 14 16 18 20 22 2 4 6 8 10 12 14 16 f r E fr D Br
opt from simulation, 3 relays topology (PER=0.25) Br
opt from analysis, 3 relays topology (PER=0.25) Br
opt from simulation, 3 relays topology (PER=0.1) Br
opt from analysis, 3 relays topology (PER=0.1)
(b) Pareto optimal bound for 1-flow 3-relay
Fig. 4. Pareto optimal bound for all topologies for different donehop, where
forwarding probability chosen integer 0 or 1.
the schedule. Moreover, all nodes are perfectly synchronized. The time-frequency block is of ςslot = 10ms, the regular slot
duration of IEEE802.15.4e. Note that this duration is long enough to carry a frame with payload of 127 bytes using IEEE802.15.4e at 250kbps.
2) MO optimization: Sopt and Bropt are derived using
NSGA-2 for a population size of 300 solutions, 1000 genera-tions and a crossover probability of 0.9. A transmission power PT = 0.15mW and a pathloss exponent of 3 is considered.
For any time-frequency blocku∈ T , there is an interference-limited channel between any two nodesi and j of the network. For all considered topologies, relay locations are defined. An imperfect channel is considered by setting the 1-hop distance donehop to 340.75m, 324.58m or 357.8m to create a
1-hop transmission with a Packet Error Rate ofP ER = 0.25, P ER = 0.1 and P ER = 0.5, respectively. Nodes located at a two-hop distance are out of reach.
Wireless topologies and protocols are simulated using the realistic discrete event-driven network simulator WSNet1. In
all topologies, sources only generate frames and destinations only receive them. The sources emit a periodic flow of frames whose period is set differently according to the protocol in use. The end-to-end delay is the duration between the arrival of the frame in the source buffer and the arrival of the frame in its destination buffer. Frames have a size of 127 bytes of payload (i.e. the standard maximum IEEE802.15.4 frame size). Simulations are performed for the duration necessary to complete the transmission of 100 000 frames.
3) Results assessment: To measure the closeness of Pareto sets computed analytically and by simulations, two metrics are calculated : the root mean squared error (RMSE) and the
Gen-1http://wsnet.gforge.inria.fr/
TABLE I. RMSEANDGDFORPARETO OPTIMAL BOUNDS
fr D fEr GD 1-flow 3-relay 9.76 ∗ 10−5 7.30 ∗ 10−5 1.74 ∗ 10−3 2-flow 2-relay 3.47 ∗ 10−5 7.07 ∗ 10−5 8.50 ∗ 10−5 erational Distance (GD). RM SE = 1n q Pn i=1 (d(i)− ˜d(i))2 f (i)2 , with n the total number of points in the set, d(i) and ˜d(i) the analytical and simulated reliability-achieving end-to-end delay and energy values. The generational distance is defined byGD = N1optPNopt
i=1 (d p i)
1/p
withdithe Euclidian distance
between analytical and simulated bounds. We usep = 2.
B. Pareto bounds
Table I lists RMSE and GD values for the two considered topologies. Values are really small, showing a quasi-perfect match between the model and simulations. Bounds are pre-sented in the following for both deterministic and stochastic schedule derivations.
C. Deterministic schedule
For all investigated topologies, the optimal binary forward-ing probabilities and evaluation criteria are given in Table II. The binary values have been derived by solving the multi-objective optimization problem of Eq. (15). For this scenario, the bound resumes to a single trade-off value.
1) 2-flow / 2-relay topology: For the 2-flow / 2-relay sce-nario, optimal forwarding probabilities and criteria are shown in Table II. The Pareto set is represented in Fig. 4(a) for donehop = 357.8m (P ER = 0.5). For this simple scenario,
the whole search has resumed to a single Pareto-optimal point. Since it is a symmetric topology, the criteria for the flow ending at D1is the same as for the one ending at D2. Since there is
only one path fromS1toD1, the number of hops travelled by
frames to reach their destination is always equal to3. 2) 1-flow 3-relay: For the 3-relay scenario, two different channels are investigated for P ER = 0.25 et P ER = 0.1. As shown in Fig. 4(b), not surprisingly, the Pareto bound P ER = 0.1 dominates the solution with the higher PER (P ER = 0.25). Optimal forwarding probabilities and criteria as shown in Table II.
D. Stochastic schedule
The Pareto-optimal forwarding probabilities for the stochastic study are given in Table II for both topologies. For the 1-flow / 3-relay topology, only one loop is allowed in the search between R1 and R2. Forwarding probabilities
are searched in a discrete set[0, 0.01, . . . , 1− ∆] of step 0.01. 1) 2-flow / 2-relay topology: As seen in Table II, Pareto-optimal solutions have a non-zero forwarding probability When donehop = 357.8m (P ER = 0.5), the whole search
has resumed to a single Pareto-optimal point as represented in Fig. 3(a). The optimal forwarding probabilities and criteria are shown in Table II. We can see that the Pareto bounds and forwarding probability values are the same as the ones obtained for a deterministic schedule.
TABLE II. DETERMINISTIC SCHEDULE(BINARY FORWARDING PROBABILITIES)AND CRITERIA FOR ALL CONSIDERED TOPOLOGIES.
2-flow / 2-relay scenario (donehop= 357.8 (1-PER = 0.5))
x13
S1R1 x14S1R2 x23S2R1 x24S2R2 x34R1R2 fR(D1|2) fD(D1|2) fE(D1|2) fDr fEr
1 0 1 0 1 0.25 3 5.25 12 21
1-flow / 3-relay scenario (donehop= 340.75 (1-PER = 0.75))
x12SR1 x13SR2 x14SR3 x23R1R2 x24R1R3 x34R2R3 fR(D) fD(D) fE(D) fDr fEr
1 0 0 1 0 1 0.3165 4 5.2074 12.6398 16.4551
1-flow / 3-relay scenario (donehop= 324.58 (1-PER = 0.9))
x12 SR1 x 13 SR2 x 14 SR3 x 23 R1R2 x 24 R1R3 x 34 R2R3 fR(D) fD(D) fE(D) fDr fEr 1 0 0 1 0 1 0.6561 4 7.2632 6.0965 11.0700
TABLE III. STOCHASTIC SCHEDULE AND CRITERIA FOR ALL CONSIDERED TOPOLOGIES.
2-flow / 2-relay scenario (donehop= 357.8 (1-PER = 0.5))
x13 S1R1 x 14 S1R2 x 23 S2R1 x 24 S2R2 x 34 R1R2 fR(D1|2) fD(D1|2) fE(D1|2) fDr f r E Unique Solution 1 0 1 0 1 0.25 3 5.25 12 21
1-flow / 3-relay scenario (donehop= 340.75 (1-PER = 0.75) and loop between R1andR2)
x11
SR1 x23R1R2 x24R1R3 x32R2R1(loop) x34R2R3 xR3R142 x43R3R2 fR fD fE fDr fEr
Smin 1 1 0 0.59 1 0 0 0.3704 4.3411 6.9338 11.7191 18.7183
Smiddle 1 1 0 0.3 1 0 0 0.3417 4.1594 5.9094 12.1732 17.2947
Smax 1 1 0 0.01 1 0 0 0.3183 4.0049 5.227 12.6242 16.4763
1-flow / 3-relay scenario (donehop= 324.58 (1-PER = 0.9) and loop between R1andR2)
x12 SR1 x 23 R1R2 x 24 R1R3 x 32 R2R1(loop) x 34 R2R3 x 42 R3R1 x 43 R3R2 fR fD fE fDr f r E Smin 1 1 0 0.137 1 0 0 0.6702 4.0431 7.9327 6.0322 11.8354 Smiddle 1 1 0 0.08 1 0 0 0.6643 4.0249 7.6348 6.0589 11.4931 Smax 1 1 0 0.01 1 0 0 0.6571 4.0031 7.3070 6.0917 11.1195
2) 1-flow / 3-relay topology: In this scenario, only one cyclic path is allowed in the search betweenR1andR2. More
specifically, the forwarding probabilitiesxuv
21,∀u, v are part of
the optimization variables. Two different channel conditions are tested as well, namingP ER = 0.25 et P ER = 0.1. Three forwarding solutionsSmin,SmiddleandSmaxare investigated
for each case as shown in Table II, all extracted from the Pareto-optimal set of Fig. 3(b) and Fig. 3(c). The first one, Smin, exhibits the smallest average end-to-end delay (but the
highest average energy), the second one, Smax, exhibits the
largest average end-to-end delay (but the smallest average energy) and the last one,Smiddlean average delay in between
smallest and largest. As seen in Table II, optimal Pareto solutions have a non-zero loop forwarding probability (x32
R2R1). This clearly shows that best solutions benefit from this addi-tional redundancy, when both delay and energy are optimized concurrently. For P ER = 0.25, the forwarding probability is of x32
R2R1 = 0.59 in Smin, while for P ER = 0.1, the forwarding probability is of x32
R2R1 = 0.137 in Smin. For the better link quality, the forwarding probability is smaller than for the worst link quality. This loop clearly compensates for channel frame losses. Compared to binary schedules, stochastic schedules create a Pareto bound that dominates the binary Pareto sets. In other words, stochastic schedules are performing better than binary ones. This is an important feature where we highlight that cyclic paths are beneficial in this context.
E. Delay distribution and Worst-case delay bound
Delay distribution and worst case delay bounds for binary and stochastic forwarding probabilities are given next.
a) Deterministic schedule: For the 1-flow / 3-relay and 2-flow / 2-relay networks, since there is only one path from source to destination, the end-to-end delay and the worst case delay are equal to a fixed value as shown in Fig. 5(a) and Fig. 5(b). The analytical delay distribution for one source-destination pair matches well with the simulation results. As shown in Table IV, that shows the impact of the number of
0 0.2 0.4 0.6 0.8 1 1.2 0 20 40 60 80 100 120 140 160 PMF Delay (ms) DW = 90 donehop=357.8 Analytical delay PMF, donehop=357.8 Simulated delay PMF, donehop=357.8 Worst case delay bound for donehop=357.8 (δ=10
-5)
(a) 2-flow / 2-relay
0 0.2 0.4 0.6 0.8 1 1.2 1.4 0 50 100 150 200 250 300 PMF Delay (ms) DW = 120 donehop=340.75 DW = 120 donehop=324.58 Analytical delay PMF, donehop=340.75 Simulated delay PMF, donehop=340.75 Analytical delay PMF, donehop=324.58 Simulated delay PMF, donehop=324.58 Dw for donehop=340.75 (δ=10-5) Dw for donehop=324.58 (δ=10
-5)
(b) 1-flow / 3-relay
Fig. 5. PMF of delay for different P ER - slot duration of 10ms and binary Pareto-optimal schedule.
relays N on the linear topology, the worst case delay bound grows withN .
b) Stochastic schedule: As seen in Fig. 6, the analytical delay distribution for one source-destination pair compared
TABLE IV. DWFOR ALL TOPOLOGIES,WHERE FORWARDING
PROBABILITY INTEGER0OR1.
δ DW (ςslot= 10ms)
2-flow 2-relay 10−5 90 (3 hops) 1-flow 3-relay 10−5 120 (4 hops)
0 0.2 0.4 0.6 0.8 1 1.2 0 20 40 60 80 100 120 140 160 PMF Delay (ms) DW = 90 donehop=357.8 Analytical delay PMF, donehop=357.8 Simulated delay PMF, donehop=357.8 Worst case delay bound for donehop=357.8 (δ=10-5)
(a) 2-flow / 2-relay with the unique Pareto-optimal solution 0 0.2 0.4 0.6 0.8 1 0 200 400 600 800 1000 1200 PMF Delay (ms) DW = 480 (donehop=340.75, PER=0.25) DW = 300 (donehop=324.58, PER=0.1) Analytical delay PMF, donehop=340.75 and Loop R1-R2 Simulated delay PMF, donehop=340.75 and Loop R1-R2 Analytical delay PMF, donehop=324.58 and Loop R1-R2 Simulated delay PMF, donehop=324.58 and Loop R1-R2 Worst case delay bound for donehop=340.75 (δ=10-5) Worst case delay bound for donehop=324.58 (δ=10
-5)
(b) 1-flow / 3-relay with Pareto-optimal solution Smin
Fig. 6. PMF of delay for different P ER and slot duration of 10ms using stochastic schedules
with the simulation results for all considered networks match well. For the 1-flow / 3-relay TDMA networks, there are several Pareto solutions. We recall that we have picked three Pareto optimal solutions to show their delay distribution: Smin, Smiddle and Smax. Their respective distributions are
plotted in Fig. 6(b) for 1-flow 3-relay network. The tail of the distribution is very difficult to validate with simulations since such delays are very rare events, with very small probabilities. However, the overall fit is really close and the RMSE between analytical and simulated delay PMF for the 3-relay TDMA network confirms this statement: the RMSE is of 4.83∗ 10−3
forSmin,1.06∗10−4forSmiddleand of9.81∗10−4forSmax.
Again, these values are really small which confirms the validity of our model.
Looking at worst case delays for 1-flow 3-relay in Table V, the value of Dw clearly increases with the P ER. It higher
accuracies are required, the larger theDw values get.
VI. CONCLUSIONS
In this paper, we propose a certifiable resource allocation for real-time multi-hop 6TiSCH wireless network. This frame-work calculates offline a schedule that can be deterministic
TABLE V. DWFOR FORWARDING PROBABILITY CHOSEN IN SET[0,1],
GIVEN FORSminSOLUTION(UNIT:MS).
δ DW(ςslot= 10ms)
2-flow 2-relay 10−5 90 (3 hops) 1-flow 3-relay (PER=0.25 LoopR1-R2) 10−5 480 (16 hops) 10−7 660 (22 hops) 10−9 780 (26 hops) 1-flow 3-relay (PER=0.1 LoopR1-R2) 10−5 300 (10 hops) 10−7 420 (14 hops) 10−9 480 (16 hops)
(binary forwarding probabilities) or stochastic (real-valued forwarding probabilities). We further certify the applicability of our algorithm to real-time communications by calculating stochastic worst-case delay bounds for considered topologies and flow patterns. The results shows that stochastic forwarding probability improves overall performance compared to the deterministic scheduling with binary values. It is interesting and important because a node doesn’t always use the same channel and slot (superframe changes over time) - messages don’t always take same path. Introducing the loop probability can even further help by adding path diversity and thus offer higher robustness. Our offline resource allocation calculation is not flow-dependent: there is no variable per flow, we work on an aggregated view of all flows in the network (not a too fine grained calculation of emission decisions), which is better when we want to model a flow envelope and not the exact detail of how it is created.
VII. ACKNOWLEDGMENTS
This work was supported in part by the National Natu-ral Science Foundation of China (NSFC) under Grant No. 61602447.
REFERENCES
[1] “WirelessHART Specification 75, TDMA Data-Link Layer, HART Communication Foundation Standard hCF SPEC-75,” 2008.1.1. [2] S. Petersen and S. Carlsen, “WirelessHART Versus ISA100.11a: The
Format War Hits the Factory Floor,” Industrial Electronics Magazine,
IEEE, vol. 5, no. 4, pp. 23–34, Dec 2011.
[3] “IEEE Standard for Local and Metropolitan Area Networks—Part 15.4 Low-Rate Wireless Personal Area Networks (LR-WPANs) Amendment 1: MAC Sublayer,” IEEE Standard 802.15.4e-2012, Apr. 2012. [4] Q. Wang, X. Vilajosana, and T. Watteyne, “6tisch operation sublayer
(6top),” IETF Standard draft-wang-6tisch-6top-sublayer-01, Jul. 2014. [5] M. Domingo-Prieto, T. Chang, X. Vilajosana, and ThomasWatteyne, “Distributed pid-based scheduling for 6tisch networks,” IEEE
COM-MUNICATIONS LETTERS, vol. 20, no. 5, pp. 1006–1009, May 2016. [6] S. Duquennoy, B. A. Nahas, O. Landsiedel, and T.Watteyne, “Orchestra: Robust mesh networks through autonomously scheduled tsch,” in in
Proc. Embedded Netw. Sensor Syst. (SenSys), Nov. 2015, pp. 337–350. [7] I. Hosni, F. Theoleyre, and N. Hamdi, “Localized scheduling for end-to-end delay constrained low power lossy networks with 6tisch,” in IEEE
ISCC, 2016.
[8] Q. Wang, K. Jaffr`es-Runser, C. Goursaud, J. Li, Y. Sun, and J.-M. Gorce, “Deriving pareto-optimal performance bounds for 1 and 2-relay wireless networks,” in IEEE IC3N, Nassau, Bahamas, August 2013. [9] Q. Wang, K. Jaffres-Runser, Y.-J. Xu, J.-L. Scharbarg, Z.-L. An, and
C. Fraboul, “TDMA versus CSMA/CA for wireless multi-hop commu-nications: a stochastic worst-case delay analysis,” IEEE Transactions
on Industrial Informatics, vol. PP, no. 99, 2016.
[10] Q. Wang, K. Jaffr`es-Runser, Y.-J. Xu, J.-L. Scharbarg, Z.-L. An, and C. Fraboul, “TDMA versus CSMA/CA for wireless multi-hop communications: A comparison for soft real-time networking,” in IEEE
WFCS, 2016.
[11] “Part 15.4: Wireless Medium Access Control (MAC) and Physical Layer (PHY) Specifications for Low-Rate Wireless Personal Area Networks (LR-WPANs),” IEEE Standard 802.15.4, 2011.
