Dynamic Programming for TSP

Travelling Salesman Problem.In the TRAVELLING SALESMAN problem (TSP), we are given a set of distinct cities {c₁,c₂, . . . ,c_n} and for each pair c_i 6=c_j the distance between c_i andc_j, denoted byd(c_i,c_j). The task is to construct a tour of the travelling salesman of minimum total length which visits all the cities and returns to the starting point. In other words, the task is to find a permutationπ of {1,2, . . . ,n}, such that the following sum is minimized

n−1

∑

i=1

d(c_π(i),c_π(i+1)) +d(c_π(n),c_π(1)).

How to find a tour of minimum length? The easy way is to generate all possible solutions. This requires us to verify all permutations of the cities and the number of all permutations isn!. Thus a naive approach here requires at leastn! steps. Using dynamic programming one obtains a much faster algorithm.

The dynamic programming algorithm for TSPcomputes for every pair(S,c_i), whereS is a nonempty subset of{c₂,c₃, . . . ,c_n} andc_i∈S, the value OPT[S,c_i] which is the minimum length of a tour which starts inc₁, visits all cities fromSand ends inc_i. We compute the valuesOPT[S,c_i]in order of increasing cardinality ofS.

The computation ofOPT[S,c_i]in the caseScontains only one city is trivial, because in this case,OPT[S,c_i] =d(c₁,c_i). For the case|S|>1, the value ofOPT[S,c_i]can be expressed in terms of subsets ofS:

OPT[S,c_i] =min{OPT[S\ {c_i},c_j] +d(c_j,c_i):c_j∈S\ {c_i}}. (1.1) Indeed, if in some optimal tour inSterminating inc_i, the cityc_j immediately pre-cedesci, then

1.2 Dynamic Programming for TSP 5 OPT[S,ci] =OPT[S\ {c_i},cj] +d(c_j,ci).

Thus taking the minimum over all cities that can precedec_i, we obtain (1.1). Finally, the valueOPT of the optimal solution is the minimum of

OPT[{c₂,c₃, . . . ,c_n},c_i] +d(c_i,c₁), where the minimum is taken over all indicesi∈ {2,3, . . . ,n}.

Such a recurrence can be transformed in a dynamic programming algorithm by solving subproblems in increasing sizes, which here is the number of cities inS. The corresponding algorithmtspis given in Fig.1.1.

Algorithmtsp({c1,c2, . . .cn},d).

Input: Set of cities{c1,c2, . . . ,cn}and for each pair of citiesci,cjthe distanced(ci,cj).

Output: The minimum length of a tour.

fori=2to ndo

OPT[c_i,c_i] =d(c1,c_i) forj=2to n−1do

forallS⊆ {2,3, . . . ,n}with|S|=jdo

OPT[S,c_i] =min{OPT[S\ {ci},ck] +d(c_k,c_i):ck∈S\ {ci}}

returnmin{OPT[{c2,c3, . . . ,cn},ci] +d(ci,c1):i∈ {2,3, . . . ,n}}

Fig. 1.1 Algorithmtspfor the TRAVELLINGSALESMANproblem

Before analyzing the running time of the dynamic programming algorithm let us give a word of caution. Very often in the literature the running time of algorithms is expressed in terms of basic computer primitives like arithmetic (add, subtract, mul-tiply, comparing, floor, etc.), data movement (load, store, copy, etc.), and control (branching, subroutine call, etc.) operations. For example, in the unit-cost random-access machine (RAM) model of computation, each of such steps takes constant time. The unit-cost RAM model is the most common model appearing in the liter-ature on algorithms. In this book we also adapt the unit-cost RAM model and treat these primitive operations as single computer steps. However in some parts of the book dealing with computations with huge numbers such simplifying assumptions would be too inaccurate.

The reason is that in all known realistic computational models arithmetic op-erations with two b-bit numbers require time Ω(b), which brings us to the log-cost RAM model. For even more realistic models one has to assume that twob-bit integers can be added, subtracted, and compared in O(b)time, and multiplied in O(blogblog logb)time. But this level of precision is not required for most of the results discussed in this book. Because of theO^∗-notation, we can neglect the dif-ference between log-cost and unit-cost RAM for most of the algorithms presented in this book. Therefore, normally we do not mention the model used to analyze running times of algorithms (assuming unit-cost RAM model), and specify it only when the difference between computational models becomes important.

Let us come back toTSP. The amount of steps required to compute (1.1) for a fixed setSof sizekand all verticesci∈SisO(k²). The algorithm computes (1.1) for every subset Sof cities, and thus takes time∑ⁿ⁻¹_k=1O( ⁿ_k

). Therefore, the total time to computeOPT is

n−1

∑

k=1

O(

n k

k²) =O(n²2ⁿ).

The improvement fromO(n!n)in the trivial enumeration algorithm toO^∗(2ⁿ)in the dynamic programming algorithm is quite significant.

For the analyses of theTSPalgorithm it is also important to specify which model is used. LetW be the maximum distance between the cities. The running time of the algorithm for the unit-cost RAM model is O^∗(2ⁿ). However, during the algo-rithm we have to operate with O(lognW)-bit numbers. By making use of more accurate log-cost RAM model, we estimate the running time of the algorithm as 2ⁿlogW n^O(1). SinceW can be arbitrarily large, 2ⁿlogW n^O(1)is not inO^∗(2ⁿ).

Finally, once all valuesOPT[S,c_i]have been computed, we can also construct an optimal tour (or a permutationπ) by making use of the following observation: A permutationπ, withπ(c₁) =c₁, is optimal if and only if

OPT=OPT[{c_π(2),c_π(3), . . . ,c_π(n)},c_π(n)] +d(c_π(n),c₁), and fork∈ {2,3, . . . ,n−1},

OPT[{c_π(2), . . . ,c_π(k+1)},c_π(k+1)] =OPT[{c_π(2), . . . ,c_π(k)},c_π(k)] +d(c_π(k),c_π(k+1)).

A dynamic programming algorithm computing the optimal value of the solution of a problem can typically also produce an optimal solution of the problem. This is done by adding suitable pointers such that a simple backtracing starting at an optimal value constructs an optimal solution without increasing the running time.

One of the main drawbacks of dynamic programming algorithms is that they need a lot of space. During the execution of the dynamic programming algorithm above described, for eachi∈ {2,3, . . . ,n}andj∈ {1,2, . . . ,n−1}, we have to keep all the valuesOPT[S,c_i]for all sets of size jand j+1. Hence the space needed is Ω(2ⁿ), which means that not only the running time but also the space used by the algorithm is exponential.

Dynamic Programming is one of the major techniques to design and analyse ex-act exponential time algorithms. Chapter3is dedicated to Dynamic Programming.

The relation of exponential space and polynomial space is studied in Chap.10.