DTN-Meteo for Randomized Local Search

So far, we have only considered greedy algorithms, for which the acceptance probabil-ityAxyfrom Section 4.8.3 is always0or1: a better solution is always chosen and a worse one is never chosen. This feature makes the Markov chain (Eq. (4.13)) for the problem absorbing and may also create local maxima, where greedy algorithms may get blocked.

To this end, randomized local search algorithms have been proposed. While a ran-domized local search still deterministically accepts a better (higher utility) solution, it may also move to a lower utility solution with a probabilityAxy>0. These probabili-ties (of moving to lower utility states) are calibrated so as to provably converge to the/an optimal solution. One example of a class of such randomized local search algorithms

are Markov Chain Monte Carlo (MCMC) methods [173]. While MCMC methods are often used to simulate and sample complex (and non-invertible) functions, they also provide a powerful tool for stochastic optimization. The most commonly used MCMC optimization algorithm and also our choice for this example is Metropolis-Hastings.

As transitions between two adjacent states are now possible in both directions, the DTN-Meteo Markov chain (Eq. (4.13)) for an MCMC algorithm is not absorbing.

Moreover, since we have established in Section 4.8.4 that mobility is stationary and ergodic, the DTN-Meteo chain is, as well, stationary and ergodic. It has a unique stationary distribution, which depends on both the contact probabilities (mobility) and on the acceptance probabilities (algorithm). The latter can be chosen by the algorithm designer.

Since we are interested in finding a/the global maximum solution, we naturally should try to maximize the stationary probabilities of good (high utility) solutions.

One example of such a distribution is the Gibbs distribution [173]:

π(x) = exp(

U(x)·T⁻¹)

∑

y∈Ωexp (U(y)·T⁻¹), (4.28) whereT is an algorithm parameter, the temperature. WhenTis small, the distribution is concentrated around the large values ofU(x)and thus, the algorithm converges to a

“good” solution with high probability.

Using the Metropolis-Hastings acceptance probability formula [173] and the above-defined stationary distribution, we obtain the following:

A_xy= min A Metropolis-Hastings algorithm using this acceptance probability has been proved to eventually converge to an/the optimum solution [173].

To sum up, assuming that node mobility creates a connected contact graph (i.e.

for all nodes i, there exists at least one node j such that their meeting probability p^c_ij > 0) and recalling that our mobility process is ergodic, the Markov chain of our randomized local search algorithm will be irreducible and aperiodic, withπ(x)as its unique stationary distribution. Since we have chosenπ(see Eq. (4.28)) to be highly concentrated around the statesx^⋆ ∈ GMwith the largest utility, the Markov chain, and hence, the algorithm, will eventually converge to those states with high probability (P[convergence]→1).

Remark:Sometimes, the temperature parameterT is variedduringthe algorithm’s operation, in order to speed up convergence. This is known assimulated annealing, whenT starts relatively high and gradually “cools down”. However, it is beyond the scope of our paper, as we do not focus on “tweaking” the algorithm itself.

Convergence Analysis for Randomized Local Search. Since the Markov chain is now irreducible, hitting an optimal state does not guarantee that the algorithm stays there forever. Convergence to one of the optimum statesx^⋆ ∈ GMnow corresponds to thefirst hitting/passage time(s)to (one of) the respective state(s)x^⋆ ∈ GMin an ergodic Markov chain, starting from any other statey∈Ω\ GM.

The first passage time may still correspond exactly to a performance metric of interest, for example end-to-end delay in routing (the algorithm terminates as soon as the destination node has been reached). However, in other cases, the first passage time will be a lower bound, for example, for the delay of reaching the optimum content storers in content placement. If the algorithm is designed correctly (according to the remark above), this bound can become tight.

Several methods are available for the analysis of stationary and ergodic Markov chains, from first step analysis to the electric network analogy, and including spectral properties. In the following, we will use an approach similar to the one in Section 4.9.1, based on an equivalent of thefundamental matrix for ergodic chains, derived from first step analysis:

Z= (I−P+Π)⁻¹, (4.30)

whereΠis a matrix with each of its rows being the stationary probability vectorπ (Eq. (4.28)) for transition matrixP.

We will derive a lower bound for the expected convergence delay of randomized content placement and the exact expected end-to-end delay of randomized routing, in an equivalent way to Theorem 9.2. We use the new fundamental matrix Zand the initial probability vector derived in Eq. (4.25).

Recall from Section 4.9.1 that, for each DTN problem’s state space, there may be more than one optimal (maximum utility) statex^⋆. For example, in fixed-replication routing, all of the(_N−1

L−1

)states in which one of theLcopies is at the destination noded are equally good and will form the setGM. This is less likely in our content placement example, where all of theLrelays defining a network statexcount towards that state’s final utilityU(x).

When several optimal states are present, each of their first passage times must be combined, in order to obtain the final performance metric of the randomized local search algorithm (e.g., end-to-end delivery delay in routing or convergence delay in content placement). Below, we will first show how to derive the first passage time for a unique optimal state (Theorem 9.3) and then how to combine the results to obtain the final performance metrics (Corollary 4).

Theorem 9.3(First Passage Time). Assuming aunique optimal statex^⋆, a lower bound for the expectedconvergence delayof a randomized algorithm modeled by our Markov chainP, starting from any source with equal probability is:

E[Td(x^⋆)]> ∑

y̸=x^⋆

p⁽⁰⁾_y zx^⋆x^⋆−zyx^⋆

π(x^⋆) , (4.31)

whereZ={z_xy}is the fundamental matrix for ergodic chains and|Ω|is the size of the solution space.

Proof. In [189] (p. 78), the authors prove that the matrixM, containing the mean first passage times from any statex∈ Ωto any other statey∈ Ωis given by: M= (I−Z+EZdg)D, whereE is a square matrix with all entries1,Zdg agrees with Zon the main diagonal and is0elsewhere, andDis a diagonal matrix with diagonal elementsdyy= (π(y))⁻¹.

From a given stateyto the unique optimal configurationx^⋆, the mean first passage is elementmyx^⋆ ofM, which from above can be written as:

myx^⋆ =zx^⋆x^⋆−zyx^⋆

π(x^⋆) . (4.32)

Using the initial probability distribution in Eq. (4.25), we calculate the weighted aver-age of Eq. (4.32) over all non-optimal statesyto obtain the expected first passage time starting from any source node with equal probability to the unique optimal statex^⋆as shown in Eq. (4.31).

To address the case when the several optimal states of equal utility values are present, we must also obtain the probabilities for each of those optimum states to be reached first (i.e., before any other optimum state), in addition to the first passage times. Then, using the first passage times calculated as above, we can derive, e.g., the expected end-to-end delay of randomized routing.

The probability for an optimum statex^⋆ to be reached before any other optimum state can be easily calculated by setting all statesx^⋆∈ GMas absorbing in our Markov chain of the randomized local search algorithm and using Theorem 9.1 from Sec-tion 4.9.1 to calculate eachpd(x^⋆). (Note that, in this case, it will not make sense to add up the probabilities, like in Theorem 9.1. They will simply sum to1, as there will be no other absorbing states in the chain.)

Corollary 4(Expected Delay). The exact expected end-to-enddelivery delayfor a randomized routing algorithm modeled by chainP, starting from any source with equal probability, is:

E[Td] = ∑

x^⋆∈GM

pd(x^⋆)E[Td(x^⋆)] (4.33) whereE[Td(x^⋆)]is given in Eq. (4.31). A lower bound for the expectedconvergence delayfor randomized content placement to find the best set ofLrelays, starting from any initial source with equal probability obeys the same relation.

Summarizing Section 4.9, we have shown detailed examples of how to use the DTN-Meteo model and Markovian analysis, to quantify the worst-case and expected performance of two types of algorithms (purely greedy and randomized local search), for two important DTN problems: routing and content placement. We have used generic utility functions, only requiring that the functions be time-invariant and that they can be locally calculated at each node.

In Section 4.10, we validate the accuracy of the results provided by DTN-Meteo against simulation results from both real world traces and synthetic mobility models.

We use state-of-the-art routing and content placement algorithms, whereof the utility functions obey our requirements.