Mathematically, to make a statement that f O(g) means that
Equation 4.2
where C is finite or zero. In other words, it says that at suitably large problem sizes, f(n) is growing no faster than g(n). Thus, a statement that the run time, f(n), of maximum flow is O(n3
), really means that f(n) O(n3
) and tells us that whatever the actual function describing max-flow run time is, its functional form is asymptotically no worse than n3
. This is informally called "Big Oh" notation. More generally, it means that there exist constants C and n0 such that
Equation 4.3
for all n no. f(n) is some mathematical function of n such that a positive real multiple of f(n) exceeds the amount of resources consumed by the given algorithm. n is the problem size, and n0 is the threshold value for the problem size beyond which Equation 4.3 is valid.
At the other extreme, if f W(g), it means that
Equation 4.4
In other words, W(g) is the set of all functional forms that are ultimately smaller in magnitude than the actual functional behavior. These are functions that are definitely more slowly growing than f(n). For example, in practice we might be uncertain about the actual asymptotic growth rate of some algorithm, but know from first principles, say, that it certainly must exceed that of max-flow. Then a statement that
"f(n)= (n3
)" or f(n) (n3
) would correctly state that whatever the actual function describing max-flow run time is, i.e., that its functional form is asymptotically at least as strong as n3
. This is called "Omega notation."
This leaves the case of f Q(g) which, as Figure 4-9 suggests, is the subset of functions in O(g) that grow exactly as g in the limit. (This is not-surprisingly called "theta notation.") Equation 4.2 applies again because f Q(g) implies f O(g), but f Q(g) is the special case where 0 < C < is a definitely non-zero finite constant. In practice the difference in usage is that f O(g) only states an upper bound on the asymptotic form, whereas f Q(g) is the stronger statement that f(n) grows at exactly the same rate as g(n) in the limit.
The formal differences in the definition of (g)and O(g) are subtle. The range of the limit in Equation 4.2 includes zero and the constant may be arbitrarily large but is definitely finite whereas in Equation 4.4. the limit is unbounded in size and strictly cannot be zero. Notionally near these limits (when C 0 or very large but not ) we are dealing with functions that are only barely within f W(g) or f O(g).
Sometimes we might want to make reference to functional forms that are much more solidly within the respective sets. This is where the
notations o(g) "little o of g" and w(g) "little omega of g" are used, implying the following, respectively:
Equation 4.5
Thus f(n) o (g) allows that f(n) O(g) is still a true statement but that f(n) is also no border-line case: it is solidly within O(g) and more specifically must be one of the smaller growing functions in O(g). w(g) expresses the corresponding notion about f(n)'s membership in (g).
4.2.2 P and NP
An algorithm that solves a certain problem is said to be efficient if there is any polynomial f(n), even f(n) = n100
, that upper-bounds the worst case run times as problem size increases, for a suitable constant C. The class of all problems for which polynomial algorithms exist or can exist by transformation from other existing algorithms is called P. A wider class of problems (that includes P as a subset) is the set of problems, called NP, for which a proposed solution can be checked (i.e., recognized as a solution if it is one) in polynomial time. Note that NP does not stand for non-P, it stands for the class of nondeterministic polynomial problems. These are problems for which you could recognize a solution efficiently if a guesser presented it, but for which there is no guarantee that all possibilities may not have to be tested, or, if tricks are known to reduce the number of possible solutions to test, the number to be checked may still exceed any polynomial bound.
NP thus includes P. An open question in mathematics is whether P and NP are equivalent.
A problem is said to be NP-hard if it is at least as hard as any problem in NP and is, strictly, a statement that can apply to problems both in NP and outside NP [BaGe00](p.561). (There are other classes of problems outside NP but beyond our scope.) Many optimization problems of real-world importance are NP-hard. In practice, this means only that no efficient (i.e., polynomial time) algorithm is known for them (and likely does not exist). More theoretically it means that if a solution for the related decision problem version of the problem at hand is proposed (by a "guess generator") you can at least test the solution for validity in polynomial time. There is, however, no assurance that you would not have to exhaust and test all possible solutions in this way.
A related concept is completeness. Completeness of a problem is not simply a higher ranking of difficulty per se. Rather, it is a special type of problem. A problem X is complete if you could solve any other problem of this class (even problems not yet posed) in polynomial time, given that you had a polynomial time algorithm for X. Hence complete problems are hardest in their own classes. To the practicing network planner NP-hard and NP-complete are not different in their practical implications: both imply the possibility of exponentially increasing run-times with problem size. The distinction of complete versus hard problems is more important to researchers who need to choose an NP-complete problem to work on to advance solving techniques for the entire class. So NP-hard problems are not necessarily less difficult than NP-complete. Graph coloring, Knapsack, Satisfiability, Traveling Salesman and Hamiltonian cycle problems are but a few of the classic NP-complete problems. These problems vary from being in Q(2 n
) to as bad as being in Q(n!). Satisfiability was the first provably NP-complete problem. Since then, by transformation proofs to satisfiability, other provably NP-complete problems have been inventoried.
A detailed listing of other known NP-complete problems is given in [GaJo79] which is also a valuable guide to NP complexity in general.
Note that the definition of problems in NP is formally in terms of decision problems, not explicit algorithms for synthesis of the solutions themselves. Decision problems are cast as yes/no-answerable or "decidable" problems. The engineering context of a problem may be to find the shortest Hamiltonian cycle in a graph. But the corresponding decision problem to which complexity theory formally applies would be: Does a Hamiltonian cycle of length less than X exist? As a simple conceptual guide to practitioners we offer a quick summary of the various classifications in Figure 4-10.
Figure 4-10. A practitioner's thumbnail guide to P, NP, NP-hard and NP-completeness.
Let us now look at algorithms and optimization methods for network problems. We will start with the useful basic ability to find shortest paths on a graph.
[ Team LiB ]
[ Team LiB ]
4.3 Shortest Path Algorithms
A frequent basic function is to find the shortest, least cost, or other minimum-weight route between nodes in a network. This is generically called the shortest path problem. When we are considering graphs with non-negative edge-weights, we can use the original Dijkstra's shortest-path algorithm [Dijk59]. Negative edge weights do not arise in real networks directly, but they do occur in intermediate steps in the later max-flow and min-cost disjoint path pair problems. In these cases Dijkstra's algorithm may fail to find the shortest path that exploits the negative edges to reduce its cost. For negative edge weights we can use the Fulkerson-Ford shortest path algorithm [FoFu62] or the modified Dijkstra algorithm below by Bhandari [Bhan99]. Let us first describe the basic Dijkstra algorithm in depth. An in-depth
understanding of Dijkstra's algorithm is worthwhile because it is a fundamental subproblem in many other algorithms and it also introduces the concepts of labeling and scanning as a basic algorithmic technique that can be easily modified for a variety of other search purposes.
A look at the internals of the Dijkstra algorithm also reveals that in finding the single shortest path, the shortest-path tree from source to all other nodes is actually built. This can be exploited in contexts where full sets of routing solutions are actually needed. Specific advantage of this has been taken in the fast k-shortest paths algorithm outlined later [MaGr94].
4.3.1 Concepts of Labeling and Scanning
A fundamental aspect of Dijkstra's algorithm is the concept of labeling nodes. Labeling marks a node with the minimum distance so far discovered from the source to the node, without any record of the route followed, but with a pointer back to the neighbor node through which one can arrive at the given node with this distance. The neighbor node in this context is called the predecessor node. A label thus has two attributes: Label = {distance, predecessor}. Labels are initially temporary until a certain stage in the algorithm where a given label is found to have the lowest distance over all the nodes scanned in a given iteration. It then becomes a permanent label. Scanning is the process of looking outwards from other nodes to all their adjacent nodes that are not permanently labeled and updating their (temporary) labels with new distance and predecessor information if the updated distance is shorter. Thus, scanning is as follows:
procedure ScanFromNode(i) {
for every adjacent node j not already in [P] { if Dj > Di + dij {
Label@[j] := (Di + dij, i) } else {
make no change }}}
where [P] is the (ordered) set of permanently labeled nodes, dij is the distance of the edge between nodes i and j, and Di is the distance part of node i's label.
4.3.2 The Dijkstra Algorithm
The basic procedure for finding the shortest path from source node s to target node t is:
procedure Dijkstra(sources, targett) { CurrentNode := s
initialize each node's Label, [P], and [D] to null
append node s to [P] and Ds to [D]
do while node t not in [P]{
ScanFromNode(CurrentNode) find node x not in [P] with minimum Dx append node x to [P]
append Dx to [D] (vector of Di values for nodes in [P]) CurrentNode := x }}
Upon exiting, the vectors (or ordered sets) [P] and [D] encode a shortest path tree rooted at the source node and including possibly all other nodes (but certainly the target node) as leaf nodes on the tree. From this information the route of the shortest path can be read out as well as the associated distance. If the shortest path tree to all other nodes, not just the target t is desired, the "do while node t not in [P]"
condition is changed to continue iteration until all nodes are permanently labeled.
Example
Figure 4-11 shows a graph with edges with distance weights associated with them. We will use this graph for an example illustrating Dijkstra's shortest path algorithm to find the shortest path from source A to target H. [P] will be the ordered set of permanently labeled nodes and [Dp] is the corresponding vector of final distances. Note in what follows that "+" means set addition. For example [P]:= [P]+C means node C is added to the existing contents of set [P]. To start out:
Figure 4-11. Graph for examples of Dijkstra, max-flow, and k-shortest paths algorithms.
[P]:= [A]; [Dp]:= [all infinite]
Step 1. Starting at the source, scan all neighbors: result is: d[B,C,D]:= [2,6,7]
The temporary labels assigned to nodes B,C,D are {2,A} {6,A} {7,A} respectively.
dmin = 2 (at node B) so node B becomes permanently labeled: {2, A}. The predecessor set has node B added to it [P]:= [P] + B;
i.e., now [P] = [A, B], [Dp] = [-, 2].
Step 2. Go to the last node to receive a permanent label, node B, and scan from there.
Scanning from B, nodes E, F receive their first temporary labels:
d[E]:= d[B] + 7 => label {9, B};
d[F]:= d[B] + 15 => label {17, B};
and the existing temporary label at C {6, A} is updated, since a lower distance is discovered: d[C]:= d[B]+3 = 5, hence the new label for C is {5, B}.
Next, checking all temporary labels network wide, we find dmin = 5 at node C. Hence node C becomes permanently labeled {5, B} and [P]:= [P] + C,
[Dp]:= [Dp] + 5, [P] = [A, B, C], [Dp] = [-, 2, 5].
Step 3. Go to node C and scan from there: nodes G, D are scanned. (Node A is already permanently labeled).
Node G gets its first temporary label: d[G]:= d[C] + 3 = 8; label = {8, C}.
Node D already has a temp. label {7, A} and is now re-scanned from node C. The new distance would be d[D]:= d[C] + 4 = 9 which is greater than that of the existing label, so no change is made to node D's label.
Next, checking globally for dmin, we find dmin=7, at node D hence node D gets permanent label {7, A}, [P]:= [P] + D,
[Dp]:= [Dp] + 7.
Step 4. Go to node D and we scan nodes G, F. (Nodes C, A are already permanently labeled).
d[G]:= d[D] + 4 = 11... exceeds existing temp label distance {8, C}, so no change.
d[F]:= d[D] + 9 = 16... which improves on the existing label {17, B}, so node F gets a new temporary label: {16, D}.
dmin = 8 found at node G; perm. label @ G = {8, C}, [P]:= [P] + G,
[Dp]:= [Dp] + 8.
Step 5. Go to G and scan H. (Nodes C, D are already permanently labeled).
d[H]:= d[G] + 6 = 14,
dmin = 14 (at H); perm. label @ H {14, G}, [P]:= [P] + H;
[Dp]:= [Dp] + 14.
At this point we would also note that H is the target node, and has received a permanent label, so we would have found the shortest path A to H as required, and we could exit at this point. However, continuing will show more generally how Dijkstra's algorithm can (and for any given source-target pair, may have to) find the complete tree of shortest paths rooted at the source node.
Step 6. Go to node H and scan nodes F, E:
d[F]:= d[H] + 9 = 23 -> no change from existing label {16, D},
d[E]:= d[H] + 7 = 21 -> no change from existing label {9, B}, dmin = 9 -> perm. label at E: {9, B}.
At this stage, only node F's label has not been made permanent. And there are no neighbors of node E to scan (B and H both have permanent labels). So at this point the "extended" algorithm halts and all labels can be considered permanent. Now if we look at the contents of the predecessor set and the distance vector, Figure 4-12 shows how the complete shortest path tree is encoded by them.
Thus, the answer to the original question is that the shortest path from A to H has a distance of 14 following route {A-B-C-G-H}.