HAL Id: hal-00905063
https://hal.archives-ouvertes.fr/hal-00905063v2
Preprint submitted on 31 Mar 2014
HAL is a multi-disciplinary open access
archive for the deposit and dissemination of
sci-entific research documents, whether they are
pub-lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.
L’archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d’enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.
control with unknown routing rates
Jean Gregoire, Emilio Frazzoli, Arnaud de la Fortelle, Tichakorn
Wongpiromsarn
To cite this version:
Jean Gregoire, Emilio Frazzoli, Arnaud de la Fortelle, Tichakorn Wongpiromsarn. Supplementary
material to: Back-pressure traffic signal control with unknown routing rates. 2013. �hal-00905063v2�
Supplementary material to:
Back-pressure traffic signal control
with unknown routing rates
Jean Gregoire
?Emilio Frazzoli
†Arnaud de La Fortelle
?∗Tichakorn Wongpiromsarn
•Abstract—This is the supplementary material to the paper: back-pressure traffic signal control with unknown routing rates. It details some model details and proofs not included in the paper due to space limitations. The characterization of the capacity region, the optimality of BP* and the behaviour of the Lyapunov drift under BP control are proved.
I. ROUTING PROCESS ASSUMPTIONS
When a quantity of vehicles arrives at node Na ∈ I(Ji)
during slot t, exogenously and endogenously, it is split and added into queues Qab, b ∈ O(Ji), according to an exogenous
routing process R(t)ab, defined for all a, b ∈ N . The arrival process and the routing process are independent, and for all t, R(t)ab is independent from {Q(τ )}τ ≤t. R(t)ab takes an integer, returns an integer, and for X ∈ N, P
bR (t)
ab(X) ≤ X. For
all process X(t) such that for all t, R(t)ab is independent from {X(τ )}τ ≤t, there exists a rate rab≥ 0 for all a, b ∈ N such
that Rab(t)(X(t)) − rabX(t) is rate convergent with rate 0.
As a consequence of the above assumptions: X
b
rab≤ 1 (1)
II. NETWORK DYNAMICS
Consider the network under phase control p(t). Let define the following flow variables:
fab(t) = min [Qab(t), µab(p(t))] (2)
fain(t) =
X
b
fba(t) (3)
The network dynamics under control p(t) is completely described as follows:
Qab(t + 1) = Qab(t) + R (t)
ab Aa(t) + fain(t) − fab(t) (4)
? Mines ParisTech, Robotics Laboratory, Paris, France † Massachusetts Institute of Technology, Boston, USA
∗ Inria Paris - Rocquencourt, IMARA team, Le Chesnay, France • Ministry of Science and Technology, Thailand
III. CHARACTERIZATION OF THE CAPACITY REGION
A. Stability definition
A key property of queuing systems is stability, defined below:
Definition 1 (Stability). The queuing network is stable if each individual queueU satisfies:
lim sup T →+∞ E ( 1 T T −1 X t=0 1U (t)>V ) → 0 as V → +∞ (5) This definition of stability is standard and is applicable to networks with arbitrary inputs and control laws [1].
B. The capacity region
It is possible to define a capacity region which describes the set of arrivals rates vectors that can be stably handled by the network.
Definition 2 (Capacity region [1]). Given a routing matrix r, the capacity regionΛr is the closure of the set of all arrival
rate vectorsλ that can be stabilized by some control. The following theorem provides a characterization of the capacity region in our particular model:
Theorem (Capacity region characterisation). Given a routing matrix r, the capacity region Λr is the set of arrival vectors
λ such that there exists g ∈ Γ satisfying:
∀a, b ∈ N , rab(λa+ gain) ≤ gab (6)
where Γ is the set of feasible long-term endogenous service rates, defined below:
Γ = Convex_Hull{µ(p) : p ∈ P} (7) Moreover,
• λ ∈ Λr is a necessary condition for network stability,
considering all possible controls (including those that have perfect knowledge of future events)
• λ ∈ int(Λr) is a sufficient condition for the network to
be stabilized by a control that does not have knowledge of future events.
Proof. The proof is a slightly modified version of the charac-terization of the capacity region of [1]. Let ˜Λr denote the set
∀a, b ∈ N , rab(λa+ gina) ≤ gab (8)
Let prove that λ ∈ ˜Λr is a necessary condition for network
stability, considering all possible controls (including those that have perfect knowledge of future events). Suppose that the network is empty at t = 0, using the equation of the dynamics of the network, we obtain:
Qab(T ) = T −1 X t=0 R(t)ab Aa(t) + fain(t) − T −1 X t=0 fab(t) (9)
Suppose the network is stabilized and fix an arbitrary small value > 0. By the network stability necessary condition of [1], there must exist some finite value V such that at arbitrary large times T , the queues lengths are simultaneously less than V with probability at least 1/2. Hence, there exists a time T such that with probability at least 1/2, the following inequalities hold for all a, b ∈ N :
Qab(T ) ≤ V (10) V T ≤ (11) PT −1 t=0 R (t) abAa(t) T ≥ rabλa− (12) PT −1 t=0 R (t) abf in a (t) T ≥ rabf in a (t) − (13) (14) Define variables gab = PT −1t=0 fab(t)/T . Using the above
inequalities together with Equation 9 provides:
rab λa+ gina ≤ gab+ 3 (15)
Hence, with probability greater that 1/2, the flows gabcome
arbitrary close to satisfying Inequality 8. As a result, there must exist sample paths fab(t) from which flow variables gab
are defined that satisfy Inequality 15. This proves that λ is a limit point of ˜Λr. As ˜Λr is a compact, it contains its limit
points, and we finally obtain: Λr⊂ ˜Λr.
Now, assume that λ is interior to ˜Λr. Then, as proved in
Section IV, one can build a randomized control that stabilizes the network under arrival rates vector λ. Hence, λ ∈ int( ˜Λr)
is a sufficient condition for the network to be stabilized by some control. Combining the two above results proves that
˜ Λr= Λr.
IV. EXISTENCE OF A STABILIZING RANDOMIZED POLICY
The following theorem proves the existence of a stabiliz-ing stationary randomized policy for all arrival rates vectors interior to the capacity region.
Theorem (Existence of a stabilizing randomized policy). Suppose there exists > 0 such that λ + ∈ int(Λr), i.e.
λ and λ + interior to the capacity region. Then, there exists a stationary randomized controlp(t) such that:˜
E ˜µab(p(t)) − rab µ˜ina(p(t)) + λa ≥ raba (16)
Proof. For the sake of simplicity, we assume in this proof that E{Aa(t)} = λa and E{R
(t)
ab(X(t))} = rabE{X(t)} for all X(t) independent from R(t)ab. It is not true in the general case (it is the particular case of i.i.d. arrivals and routing), and the reader can refer to [1] for the principle of an extension to the general case using a K-steps Lyapunov drift.
Suppose λ is interior to the capacity region, i.e. there exists a positive vector such that λ + ∈ Λ. By Theorem III-B, there exists g ∈ Γ such that:
∀a, b ∈ N , rab(λa+ a+ gain) ≤ gab (17)
Since g ∈ Γ, it can be expressed as a weighted sum as follows:
g =X
p∈P
wpµ(p) (18)
where weights wp sum to 1. Let define the randomized
policy ˜p that selects randomly the phase to apply at every time slot according to probabilities (wp)p∈P.
It is direct that it will result in a randomized stationary service matrix µ(˜p(t)) verifying:
E{µab(˜p(t))} = gab (19)
As a result,
Eµab(˜p(t)) − rab µ˜ain(˜p(t)) + λa ≥ raba (20)
Now, assume that ˜p(t) is applied to the queuing network. Then, using the equation of the dynamics of the network:
Qab(t + 1) ≤ max (Qab(t) − µab(˜p(t)), 0) +
R(t)ab Aa(t) + µina(˜p(t))
(21) An inequality holds instead of an equality because the num-ber of vehicles transferred is less or equal to the transmission rate offered by servers.
Squaring both sides and using max2(x, 0) ≤ x2, we obtain:
Qab(t + 1)2− Qab(t)2≤ R(t)ab Aa(t) + µina(˜p(t)) 2 + µab(˜p(t))2 − 2Qab(t) µab(˜p(t)) − R (t) ab Aa(t) + µina(˜p(t)) (22) Define the Lyapunov function V(Q(t)) = V (t) = P
a,bQab(t)2. Taking expectations, summing over all a, b ∈
N , using independences and noting that E{A} ≤p E{A2}, we obtain: E{V (t + 1) − V (t)|Q(t)} ≤ B − 2X a,b Qab(t)E{µab(˜p(t)) − rab λa+ µina(˜p(t)))} (23)
Using Inequality 20, we obtain:
E{V (t + 1) − V (t)|Q(t)} ≤ B − 2 X
a,b
rabaQab(t) (24)
Let define η = 2 mina,braba> 0, we finally obtain:
E{V (t + 1) − V (t)|Q(t)} ≤ B − η X
a,b
Qab(t) (25)
The sufficient condition using Lyapunov drift proved in [1] enables to conclude stability of the queuing network.
V. OPTIMALITY OFBP*
Theorem (Back-pressure optimality). Assuming that pressure functions are linear with strictly positive slopes. Then, BP* is stability-optimal.
Proof. Again, for the sake of simplicity, we assume in this proof that E{Aa(t)} = λa and E{R
(t)
ab(X(t))} = rabE{X(t)} for all X(t) independent from R(t)ab. The reader can refer to [1] for the principle of an extension to the general case using a K-steps Lyapunov drift.
Let θab > 0 denote the slope of linear pressure
function Pab and Πab(t) = Pab(Qab(t)) the evolution
of pressures over time. Define the Lyapunov function V(Q(t)) = V (t) =P
a,bθabQab(t)2 and let p(t) denote the
control applied to the queuing network. With the same manip-ulations as for the proof of Theorem III-B, we obtain:
V (t + 1) − V (t) =X a,b Πab(t + 1)2− Πab(t)2= X a,b θab Qab(t + 1)2− Qab(t)2 ≤ X a,b θab R(t)ab Aa(t) + µina(p(t)) 2 + µab(p(t))2 − 2X a,b θabQab(t) µab(p(t)) − R (t) ab Aa(t) + µ in a(p(t)) ≤ B(t)−2X a,b θabQab(t) µab(p(t)) − R (t) ab Aa(t) + µ in a(p(t)) (26) with the upper-bound B(t) defined below:
B(t) =X a,b θab R(t)ab Aa(t) + sup p∈P µina(p) 2 + sup p∈P µab(p) 2 (27) Taking expectation and using independences, we get :
E{V (t + 1) − V (t)|Q(t)} ≤ B − 2X a,b θabQab(t)Eµab(p(t)) − rabµina(p(t))|Q(t) + 2X a,b θabQab(t)rabλa (28)
The upper-bound B is obtained using E{Aa} ≤
p E{A2a}: B =X a,b θabrab2 Amaxa + sup p∈P µina(p) 2 + θab sup p∈P µab(p) 2 (29) By simple sum manipulation, the following identity is obtained: X a,b Mab(gab− rabgina) = X a,b (Mab− X c rbcMbc)gab (30)
Using identity 30, Equation 28 becomes:
E{V (t + 1) − V (t)|Q(t)} ≤ B − 2X a,b θabQab(t) − X c rbcθbcQbc(t) ! E {fab(t)|Q(t)} + 2X a,b θabQab(t)rabλa = B − 2X a,b Πab(t) − X c rbcΠbc(t) ! E {fab(t)|Q(t)} + 2X a,b Πab(t)rabλa (31)
Now, assume that BP* control p?(t) is applied and let V?(t) denote the Lyapunov function under p?(t). It is assumed that in case of equality when selecting the phase that maximizes the weighted sum, the selected phase p?(t) satisfies µab(p?(t)) =
0 if Wab(t) = 0. As a result, we obtain: E{V?(t + 1) − V?(t)|Q(t)} ≤ B − 2X a,b Wab(t)E {µab(p?(t)) |Q(t)} + 2X a,b Πab(t)rabλa (32)
By construction of back-pressure control p?(t)
(see the arg max in the algorithm), p?(t) maximizes P
a,bWab(t)µab(p(t)) over all possible alternative controls
p(t).
Now, suppose that the arrival rates vector is interior to the capacity region Λr, i.e. there exists > 0 such that λ + ∈ Λ.
in this paper due to space limitations, there exists a stabilizing stationary randomized control ˜p(t) such that for all a, b ∈ N :
Eµab(˜p(t)) − rab µain(˜p(t)) + λa ≥ raba (33)
Combining the two above statements, taking expectations, and noting that the control ˜p(t) is stationary result in:
X a,b Wab(t)E{µab(p?(t))|Q(t)} ≥ X a,b Wab(t)E{µab(˜p(t))} ≥ X a,b Πab(t) − X c rbcΠbc(t) ! E{µab(˜p(t)))} = X a,b Πab(t)Eµab(˜p(t)) − rabµina(˜p(t)) ≥ X a,b Πab(t)(rabλa+ raba) (34)
Injecting the above result in the Lyapunov drift inequality results in:
E{V?(t + 1) − V?(t)|Q(t)} ≤ B − 2X
a,b
θabrabaQab(t)
(35) Let η = 2 mina,bθabraba> 0. We finally obtain:
E{V?(t + 1) − V?(t)|Q(t)} ≤ B − η X
a,b
Qab(t) (36)
The sufficient condition using Lyapunov drift proved in [1] enables to conclude stability of the queuing network.
VI. BEHAVIOUR OF THELYAPUNOV DRIFT IN HEAVY LOAD CONDITIONS
Theorem (Lyapunov drift under heavy load conditions). As-sumeλ + ∈ Λr, BP control is applied and the network is in
heavy load conditions, then there exists B, η > 0 such that :
E{V (t + 1) − V (t) | Q(t)} ≤ B − η X
a
Qa(t) (37)
for sufficiently large .
Proof. Again, for the sake of simplicity, we assume in this proof that E{Aa(t)} = λa and E{R
(t)
ab(X)} = rabX for all
X ∈ N.
Let Πa(t) denote the evolution of Pa(Qa(t)) over time and
p(t) the control applied to the queuing network. By simple manipulations, we get: V (t + 1) − V (t) = X a θa(Qa(t + 1) − Qa(t)) 2 + 2X a θaQa(t) (Qa(t + 1) − Qa(t)) (38)
As a result, Inequality 38 becomes:
V (t + 1) − V (t) ≤ B(t) − 2X a θaQa(t) X b fab(t) − R (t) ab Aa(t) + fain(t) (39) with the upper-bound B(t) defined below:
B(t) =X a θa X b R(t)ab Aa(t) + sup p∈P µina(p) 2 +X a θa X b sup p∈P µab(p) 2 (40) Taking expectations and using independences, we obtain :
E {V (t + 1) − V (t) | Q(t)} ≤ B+2 X a θaQa(t)λa X b rab − 2E X a,b θaQa(t) fab(t) − rabfain(t) | Q(t) (41)
Moreover, by simple sum manipulations, we get the below identity: X a,b Pa fab− rabfain = X a,b Pa− X c rbcPb ! fab (42)
Using identity 42, Equation 41 becomes:
E {V (t + 1) − V (t) | Q(t)} ≤ B + 2 X a θaQa(t)λa − 2X a,b Πa(t) − X c rbc ! Πb(t) ! E {fab(t) | Q(t)} (43) SinceP crbc≤ 1, we obtain: E {V (t + 1) − V (t) | Q(t)} ≤ B + 2 X a θaQa(t)λa − 2X a,b (Πa(t) − Πb(t)) E {fab(t) | Q(t)} (44)
Now, assume that BP control pBP(t) is applied and let VBP(t) denote the Lyapunov function under pBP(t). It is assumed that in case of equality when selecting the phase that maximizes the weighted sum, the selected phase p?(t) satisfies
µab(p?(t)) = 0 if Wab(t) = 0. Moreover, by definition of
dab(t), under infinite capacities, fab(t) = dab(t)µab(p(t)).
Hence, we obtain: EVBP(t + 1) − VBP(t) | Q(t) ≤ B + 2X a θaQa(t)λa − 2X a,b Wab(t)Eµab pBP(t) | Q(t) (45)
By construction of back-pressure control pBP(t)
(see arg max in the algorithm), pBP(t) maximizes
P
a,bWab(t)µab(p(t)) over all possible alternative controls
p(t).
Now, assume λ + ∈ Λrwhere is a positive vector. Then,
as proved in Section IV, there exists a stabilizing stationary randomized control ˜p(t) such that:
∃g ∈ Γ : ∀a, b ∈ N , E{µab(˜p(t))} = gab (46)
Combining the two above statements and taking expecta-tions results in:
X a,b Wab(t)E{µab(pBP(t)) | Q(t)} ≥ X a,b Wab(t)E{µab(˜p(t)) | Q(t)} = X a,b dab(t) (Πa(t) − Πb(t)) gab (47)
Now, assume that the network in heavy load conditions, then dab(t) = 1 and we obtain:
X a,b Wab(t)E{µab(pBP(t)) | Q(t)} ≥ X a,b (Πa(t) − Πb(t)) gab (48)
By simple manipulation, we get the following identity: X a,b (Πa− Πb) gab= X a Πa gaout− g in a (49) Hence, X a,b Wab(t)E{µab(pBP(t)) | Q(t)} ≥ X a Πa(t) gaout− g in a = X a θaQa(t) gaout− g in a (50) Moreover, by definition of the input/output flow,
gouta − gin a = X b gab− gina = X b gab− rabgain − (1 −X b rab)gina (51)
As a result, using the inequalities verified by g ∈ Γ, we obtain: X a,b Wab(t)E{µab(pBP(t)) | Q(t)} ≥ X a θaQa(t) " X b (rabλa+ raba) − 1 − X b rab ! gina # (52) Injecting the latter result in Inequality 45 provides:
EVBP(t + 1) − VBP(t) | Q(t) ≤ B − 2X a θaQa(t) " X b raba− 1 − X b rab ! gina # (53) Let define η as follows:
η = 2 min a θa X b raba− 1 − X b rab ! gina ! (54) We finally obtain: E{VBP(t + 1) − VBP(t) | Q(t)} ≤ B − η X a Qa(t) (55) If for all a ∈ N , X b raba> 1 − X b rab ! gain (56)
which can be verified for sufficiently large then η > 0 and the inequality of the sufficient condition for network stability using Lyapunov drift of [1] is verified in heavy load conditions. However, it does not imply that the network is stable under BP control since the heavy load assumption is not necessarily verified at any time.
REFERENCES
[1] M. J. Neely, Dynamic power allocation and routing for satellite and wireless networks with time varying channels. PhD thesis, LIDS, Massachusetts Institute of Technology, 2003.