The Greedy Algorithm as an Approximation MethodMethod

In this section we investigate independence systemsM = (E,S) which are not matroids. By definition, the greedy algorithm then does not (always) yield an optimal solution for the optimization problem

(P) determine max

w(A) :A∈S ,

where w:E→R⁺0 is a given weight function. Of course, we may apply the greedy algorithm nevertheless, in the hope of obtaining a reasonably good ap-proximate solution in this way. We shall examine the quality of this approach by deriving bounds for the term

f(M) = min

w(Tg)

w(T₀):w:E→R⁺0

,

where Tg runs over all solutions for (P) which may be constructed by the greedy algorithm, whereasT0 is an optimal solution.⁵ We follow Korte and Hausmann [KorHa78] in this section; similar results were also obtained by Jenkyns [Jen76].

First we introduce some useful parameters for independence systems. For any subsetAofE, thelower rank ofA is

lr(A) = min

|I|:I⊂A, I∈S, I∪ {a}∈/Sfor alla∈A\I . Similarly, we define theupper rank ofA as

ur(A) = max

|I|:I⊂A, I∈S, I∪ {a}∈/Sfor alla∈A\I .

5Note that the greedy algorithm can yield different solutionsTg for different orderings of the elements ofE(which may occur if there are distinct elements having the same weight).

Hence we indeed have to minimize over allTg.

Moreover, therank quotient ofM is

here terms ⁰₀ might occur; such terms are considered to have value 1. Note that Theorem5.2.1immediately yields the following result.

Lemma 5.4.1 An independence systemM= (E,S)is a matroid if and only ifrq(M) = 1.

As we will see, the rank quotient indicates how much M differs from a matroid. Below, we will get an interesting estimate for the rank quotient confirming this interpretation. But first, we prove the following theorem of [Jen76] and [KorHa78],⁶ which shows how the quality of the solution found by the greedy algorithm depends on the rank quotient ofM.

Theorem 5.4.2 Let M = (E,S) be an independence system with a weight function w:E→R⁺0.Moreover, letT_g be a solution of problem(P)found by the greedy algorithm, andT₀ an optimal solution. Then

rq(M)≤w(T_g) w(T0)≤1.

Proof The second inequality is trivial. To prove the first inequality, we in-troduce the following notation. Suppose the set E is ordered according to decreasing weight, say E={e₁, . . . , e_m} with w(e₁)≥w(e₂)≥ · · · ≥w(e_m).

We putw(e_m+1) = 0 and write

E_i={e₁, . . . , e_i} fori= 1, . . . , m.

Then we get the following formulae, which the reader should check in detail:

w(Tg) = definition of the greedy algorithm,T_g∩E_iis a maximal independent subset of

6This result was conjectured or even proved somewhat earlier by various other authors;

see the remarks in [KorHa78].

148 5 The Greedy Algorithm

E_i, and therefore|T_g∩E_i| ≥lr(E_i). Using these two observations, we obtain

|T_g∩E_i| ≥ |T₀∩E_i| × lr(Ei)

ur(E_i)≥ |T₀∩E_i| ×rq(M).

Using (5.2) and (5.3) yields w(Tg) =

m i=1

|Tg∩Ei|

w(ei)−w(ei+1)

≥rq(M)× m i=1

|T0∩Ei|

w(ei)−w(ei+1)

= rq(M)×w(T0).

As w and T_g were chosen arbitrarily in Theorem 5.4.2, we conclude rq(M)≤f(M). The following result shows that we actually have equality.

Theorem 5.4.3 LetM = (E,S)be an independence system.Then there exist a weight functionw:E→R⁺0 and a solutionTg for problem (P)obtained by the greedy algorithm such that

w(T_g)

w(T0)= rq(M), whereT0 denotes an optimal solution for (P).

Proof Choose a subset AofE with rq(M) = lr(A)/ur(A), and letIland Iu

be maximal independent subsets ofAsatisfying|Il|= lr(A) and|Iu|= ur(A).

Define the weight functionwby w(e) :=

1 fore∈A, 0 otherwise and order the elementse1, . . . , emofE such that

Il={e1, . . . , e_lr(A)}, A\Il={e_lr(A)+1, . . . , e_|A|}, E\A={e_|A|+1, . . . , em}. ThenIlis the solution for (P) found by the greedy algorithm with respect to this ordering of the elements ofE, whereasIu is an optimal solution. Hence

w(Il) w(I_u)= |Il|

|I_u|= lr(A)

ur(A)= rq(M).

As Theorems5.4.2and5.4.3show, the rank quotient of an independence system gives a tight bound for the weight of the greedy solution in comparison to the optimal solution of (P); thus we have obtained the desired measure for

the quality of the greedy algorithm as an approximation method. Of course, this leaves us with the nontrivial problem of determining the rank quotient for a given independence systemM. The following result provides an example where it is possible to determine this invariant explicitly.

Theorem 5.4.4 Let G= (V, E) be a graph and M = (E,S) the indepen-dence system given by the set of all matchings ofG;see Exercise5.1.5. Then rq(M) = 1 provided that each connected component ofGis isomorphic either to a complete graphK_i withi≤3or to a star.In all other cases, rq(M) =¹₂. Proof Assume first that each connected component ofGis isomorphic either to a complete graph K_i with i≤3 or to a star. Then lr(A) = ur(A) for all

LetI₁andI₂be two maximal independent subsets ofA, that is, two maximal matchings contained inA. Obviously, it suffices to show|I1| ≥¹₂|I2|. We define a mapping α:I2\I1→I1\I2 as follows. Let e be any edge in I2\I1. As I₁∪ {e} ⊂Aand asI₁is a maximal independent subset ofA,I₁∪ {e}cannot be a matching. Thus there exists an edge α(e)∈I1 which has a vertex in common with e. As I2 is a matching, we cannot have α(e)∈I₂, so that we have indeed defined a mapping α:I2\I1→I1\I2. Clearly, each edge e∈I1\I2can share a vertex with at most two edges in I2\I1, so thatehas at most two preimages underα. Therefore

|I₁\I₂| ≥α(I₂\I₁)≥1

Let us mention a further similar result without proof; the interested reader is referred to [KorHa78].

Result 5.4.5 Let G= (V, E) be the complete graph on n vertices, and let M= (E,S) be the independence system whose independent sets are the sub-sets of Hamiltonian cycles ofG.Then

1

2≤rq(M)≤1 2+ 3

2n.

150 5 The Greedy Algorithm

By definition, the maximal independent sets of an independence system as in Result5.4.5are precisely the Hamiltonian cycles ofG. Thus the greedy al-gorithm provides a good approximation to the optimal solution of the problem of finding a Hamiltonian cycle of maximal weight in Kn for a given weight function w:E→R⁺0. Note that this is the problem opposite to the TSP, where we look for a Hamiltonian cycle of minimal weight.

The above examples suggest that the greedy algorithm can be a really good approximation method. Unfortunately, this is not true in general. As the fol-lowing exercise shows, it is easy to construct an infinite class of independence systems whose rank quotient becomes arbitrarily small.

Exercise 5.4.6 Let G be the complete digraph on n vertices, and let M= (E,S) be the independence system determined by the acyclic directed subgraphs ofG, that is,

S={D⊂E:D does not contain any directed cycle}. Then rq(M)≤2/n, so that limn→∞rq(M) = 0.

Our next aim is to derive a useful estimate for the rank quotient of an independence system. We need a lemma first.

Lemma 5.4.7 Every independence systemM= (E,S)can be represented as the intersection of finitely many matroids onE.

Proof We have to show the existence of a positive integer k and matroids M1= (E,S1), . . . , Mk= (E,Sk) satisfyingS=k

i=1Si. LetC1, . . . , Ck be the minimal elements of the set {A⊂E :A /∈S}, that is, the circuits of the independence systemM. It is easily seen that

S= k i=1

S_i, whereS_i:={A⊆E:C_iA}.

Thus we want to show that all Mi= (E,Si) are matroids. So let A be an arbitrary subset of E. If Ci is not a subset of A, then A is independent in Mi, so thatAitself is the only maximal independent subset ofAinMi. Now supposeC_i⊆A. Then, by definition, the maximal independent subsets ofA in M_i are the sets of the form A\ {e} for some e∈C_i. Thus all maximal independent subsets ofAhave the same cardinality|A| −1 in this case. This shows thatMi satisfies condition (3) of Theorem5.2.1, so thatMi is indeed

a matroid.

Theorem 5.4.8 Let the independence systemM= (E,S)be the intersection ofk matroids M₁, . . . , M_k onE. Thenrq(M)≥1/k.

Proof LetAbe any subset ofE, and let I₁, I₂be any two maximal

contradicting the maximality ofI1. Hence each e∈I2\I1 can be contained in at mostk−1 of the setsIi,1\I1; this implies For each positive integerk, there exists an independence system for which the bound of Theorem5.4.8is tight. Unfortunately, equality does not hold in general; for instance, Result5.4.5provides a family of counterexamples. The interested reader is referred to [KorHa78].

Example 5.4.9 Let G= (V, E) be a strongly connected digraph, and let M be the intersection of the graphic matroid and the head-partition matroid ofG. Then the independent sets of maximal cardinality in M are precisely the spanning arborescences ofG. Note thatM may admit further maximal independent sets, as an arbitrary arborescence does not necessarily extend to a spanning arborescence: in general,M is not a matroid.

Exercise 5.4.10 Let G be a digraph. Find three matroids such that each directed Hamiltonian path inGis an independent set of maximal cardinality in the intersection of these matroids.

In the situation of Example 5.4.9, the greedy algorithm constructs an ar-borescence whose weight is at least half of the weight of a maximal arbores-cence, by Theorems 5.4.8and5.4.2. As mentioned at the end of Sect. 4.8, a maximal arborescence can be found with complexity O(|E|log|V|), using a

152 5 The Greedy Algorithm

considerably more involved method. The following result about the intersec-tion of two arbitrary matroids is interesting in this context.

Result 5.4.11 Consider an independence system M = (E,S) which is the intersection of two matroidsM1= (E,S1)andM2= (E,S2),and letw:E→ R⁺0 be a weight function onM.Assume that we may check in polynomial time whether a set is independent in eitherM1 or M2.Then it is also possible to find an independent set of maximal weight in M in polynomial time.

For a situation as described in Result5.4.11, we say that the two matroids M1 andM2are given byoracles for independence; this just means that it is somehow possible to check whether a given set is independent in polynomial time. Then Result5.4.11states that a maximal independent set inM can be found inoracle polynomial time, that is, by using both oracles a polynomial number of times; see [HauKo81] for more on oracles representing matroids and independence systems. Result5.4.11is very important in combinatorial optimization. We have decided to omit the proof because the corresponding algorithms as well as the proofs for correctness are rather difficult—even in the case without weights—and use tools from matroid theory which go beyond the limits of this book. The interested reader may consult [Law75], [Edm79], and [Cun86]; or the books [Law76] and [Whi87].

Of course, one may also consider the analogous problems for the inter-section of three or more matroids; we will just state the version without weights. Unfortunately, these problems are presumably not solvable in poly-nomial time, as the next result indicates.

Problem 5.4.12(Matroid intersection problem, MIP) Let three matroids Mi= (E,Si), i= 1,2,3, be given, and letkbe a positive integer. Does there exist a subsetAofE with|A| ≥kandA∈S1∩S2∩S3?

Theorem 5.4.13 MIP is NP-complete.

Proof Exercise5.4.10shows that the question whether a given digraph con-tains a directed Hamiltonian path is a special case of MIP. This problem (directed Hamiltonian path, DHP) is NP-complete, as the analogous prob-lem HP for the undirected case is NP-complete by Exercise 2.7.7, and as HP can be transformed polynomially to DHP by replacing a given graph by its complete orientation. Hence the more general MIP is NP-complete, too.

Theorem5.4.13indicates that the results presented in this chapter really are quite remarkable: even though the problem of determining a maximal independent set in the intersection of k≥3 matroids is NP-hard (maximal either with respect to cardinality or a more general weight function), the greedy algorithm gives a quite simple polynomial method for finding an ap-proximate solution which differs at most by a fixed ratio from the optimal

solution. This result is by no means trivial, as there are many optimization problems for which even the question whether an approximate solution with a performance guaranty exists is NP-hard; we will encounter an example for this phenomenon in Chap. 15.