Convex Combinations and Convex Hulls

The operations described in § 1.2 took convex sets and made new convex sets with them. The present section is devoted to another operation, which takes a nonconvex set and makes a convex set with it. First, let us recall the following basic facts from linear algebra.

(i) A linear combination of elements

Xlo ...

^,Xk of]Rn is an element

:Lf=l

^ajXj,

where the coefficients aj are arbitrary real numbers.

(ii) A (linear) subspace of]Rn is a set containing all its linear combinations; an intersection of subspaces is still a subspace.

(iii) To any nonempty set S C ]Rn, we can therefore associate the intersection of all subspaces containing S. This gives a subspace: the subspace generated by S (or linear hull of S), denoted lin S - other notations are vect S or span S.

(iv) For the c-relation, linS is the smallest subspace containing S; it can be con-structed directly from S, by collecting all the linear combinations of elements of S.

(v) Finally, Xl, ... ,Xk are said linearly independent if

:Lf=l

^{ajXj =} 0 implies that al

= ... =

^ak

=

O. In ]Rn. this implies k ~ n.

Now, let us be slightly more demanding for the coefficients aj, as follows:

(i') An affine combination of elements

Xlo ... ,

^Xk of]Rn is an element

:Lf=l

^ajxi,

where the coefficients ai satisfy

:Lf=l

^{ai = 1.}

As explained after Example 1.2.2, "affinity = linearity + translation"; it is therefore not surprising to realize that the development (i) - (v) can be reproduced starting from (i'):

(ii') An affine manifold in ]Rn is a set containing all its affine combinations (the equivalence with Example 1.1.2(b) will appear more clearly below in Proposi-tion 1.3.3); it is easy to see that an intersecProposi-tion of affine manifolds is still an affine manifold.

(iii') To any nonempty set S C ]Rn, we can therefore associate the intersection of all affine manifolds containing S. This gives the affine manifold generated by S, denoted aff S: the affine hull of S.

1 Generalities 95 it does not depend on the index chosen for the translation (here 0). In linear language, the required property is that the k vectors Xi - Xo, i =j:. 0 be linearly independent. Getting rid of the arbitrary index 0, this means that the system of equations

The corresponding coefficients ai are sometimes called the barycentric coordinates of x - even though such a terminology should be reserved to nonnegative aj 's. To say that a set of vectors are affinely dependent is to say that one of them (anyone) is an affine combination of the others. the extra requirement in (i'). We apply once more the same idea and we pass from affinity to convexity by requiring some more of the ai's. This gives a new definition, playing the role of (i) and (i'):

A convex combination is therefore a particular affine combination, which in turn is a particular linear combination. Note in passing that all convex combinations of given XI, ... , Xk form a convex set: it is the image of ..1k under the linear mapping

)Rk :3 (al,"" ak) t-+ alxl

+ ... +

^{akxk E )Rn .}

The sets playing the role oflinear or affine subspaces of (ii) and (ii') will now be logically called convex, but we have to make sure that this new definition is consistent with Definition 1.1.1.

Proposition 1.3.3 A set C C )Rn is convex

if

and only

if

it contains every convex combination of its elements.

PROOF. The condition is sufficient: convex combinations of two elements just make up the segment joining them. To prove necessity, take XI, ... ,Xk in C and a = (aI, ... ,ak) E ..1k. One at least of the aj's is positive, sayal > O. Then form

al a2

Y2:= XI

+

X2

at +a2 al +a2

[= al~a2

^(alxl

⁺

^{a 2X2)]}

which is in C by Definition 1.1.1 itself. Therefore,

al +a2 a3

Y3:= Y2

+

X3

at +a2 +a3 at +a2 +a3

[=)'~I

a.LI=lajXj]

·'=1 1

is in C for the same reason; and so on until al

+ ...

+ak-I ak

Yk := 1 Yk-I

+ T

^Xk

[ = t Lf=l

ajXj] .

o

The working argument of the above proof is longer to write than to understand. Its basic idea is just associativity: a convex combination x =

L

^{ai Xi} of convex combinations Xi

=

'L{3ijYij is still a convex combination X

=

L L(ai{3ij)Yij. The same associativity property will be used in the next result.

Because an intersection of convex sets is convex, we can logically define as in (iii), (iii') the convex hull co S of a nonempty set S: this is the intersection of all the convex sets containing S.

Proposition 1.3.4 The convex hull can also be described as the set of all convex combinations:

co S :=

n{c:

C is convex and contains S}

= {x E)Rn : for some k E N*, there exist Xl, ... ,Xk E Sand (1.3.2) a

=

(aJ, ... , ak) E..1k such that

2:f=1

ajXj

=

^{x} .}

PROOF. Call T the set described in the rightmost side of(1.3.2). Clearly, T :::> S. Also, if C is convex and contains S, then it contains all convex combinations of elements

1 Generalities 97 in S (Proposition 1.3.3), i.e. C ::J T. The proof will therefore be finished if we show that T is convex.

For this, take two points x and yin T, characterized respectively by (XI, al), ... , (Xk, ak) and by (Yi> .81), ... , (Ye, .8e); take also A E ]0, 1[. Then Ax

+

^{(l - A)Y is a}

certain combination of k

+

l elements of S; this combination is convex because its coefficients Aai and (l - A){Jj are nonnegative, and their sum is

k

e

ALai

+

(I - A) L hj

=

A

+

I - A

=

l. ^D

i=1 j=1

Example 1.3.5 Take a finite set {XI, ... , xm}. To obtain its convex hull, it is not necessary to list all the convex combinations obtained via a E Llk forallk = I, ... , m.

In fact, as already seen in Example 1.1.3, Llk C Llm if k ~ m, so we can restrict ourselves to k = m. Thus, we see that

cO{Xi>"" xm} = {Lj=1 ajxj : a = (ai>"" am) E Llm}.

Make this example a little more complicated, replacing the collection of points by a collection of convex sets:

S = CI U··· U Cm where each Ci is convex.

A simplification of (1.3.2) can again be exploited here. Indeed, consider a convex combination L~=I aiXj. It may happen that several of the Xj 's belong to the same Cj.

To simplify notation, suppose that Xk-I and Xk are in CI ; assume also ak > O. Then set ({Ji, Yi) := (ai, Xi), i = 1, ... , k - 2 and

.8k-1 := ak-I

+

ak, Yk-I :=

pL,

^(ak-IXk-1

⁺

^{akXk) E CI ,}

so that

Lf=1

^aixi = L~:::II ^{JiYi. Our convex combination (a, x) is useless, in the sense that it can also be found among those with k - I elements. To cut a long story short, associativity of convex combinations yields

COS={Lt=laiXi aELlm, XiECifori=I, ...

,m}.

From a geometrical point of view, the convex hull of CI U C2 (m = 2) is simply constructed by drawing segments, with endpoints in CI and C2 ; for CI U C2 U C3 , we

paste triangles, etc. D

When S is infinite, or has infinitely many convex components, k is a priori un-bounded in (1.3.2) and cannot be readily restricted as in the examples above. Yet, a bound on k exists for all S when we consider linear combinations and linear hulls -and consequently in the affine case as well; this is the whole business of dimension.

In the present case of convex combinations, the same phenomenon is conserved to some extent. For each positive integer k, call Sk the set of all convex combinations of k elements in S: we have

S = SI C S2 C ... C Sk C ...

The Sk'S are not convex but, "at the limit", their union is convex and coincides with co S (Proposition 1.3.4). The theorem below tells us thatk does not have to go to +00:

the above sequence actually stops at Sn+1 = co S.

Theorem 1.3.6 (C. Caratheodory) Any x E co S C ]Rn can be represented as a convex combination of n

+

1 elements of S.

PROOF. Take an arbitrary convex combination x = 2:7=1 aixi, with k > n

+

1. We claim that one of the Xi'S can be assigned a O-coefficient without changing x. For this, assume that all coefficients ai are positive (otherwise we are done).

The k > n

+

1 elements Xi are certainly affinely dependent: (1.3.1) tells us that

The same proof technique is commonly used in actual computations dealing with linearly constrained optimization. Geometrically, we start from a

=

(ai, ... , ak) E L1k. We compute a direction -d

=

("1, ... , 15k), which is in the subspace parallel to aff L1b so that for any stepsize t, a - td E aff L1k; and also, x is kept invariant. The particular t* is the maximal stepsize such that a - td E L1k; as a result, a - t*d is on the boundary of L1k, i.e. in L1k-l;

see Fig. 1.3.1.

1 Generalities 99

-/3

Fig. 1.3.1. CaratModory's theorem

The theorem of CaratModory does not establish the existence of a "basis" with n

+

1 elements, as is the case for linear combinations. Here, the generators Xi may depend on the particular x to be computed. In 1R2, think of the comers of a square: anyone of these 4 > 2

+

^I points may be necessary to generate a point in the square; also, the unit disk cannot be generated by finitely many points on the unit circle. By contrast, a subspace of dimension m can be generated by m (carefully selected but)fixed generators.

It is not the particular value n

+

1 which is interesting in the above theorem, but rather the fact that the cardinality of relevant convex combinations is bounded: this is particularly useful when passing to the limit in a sequence of convex combinations. This value n

+

I is not of fundamental importance, anyway, and can often be reduced - as in Example 1.3.5: the convex hull of two convex sets in lR 1 00 can be generated by 2-combinations; also, the technique of proof shows that it is the dimension of aff S that counts, not n. Along these lines, we mention without proof a result geometrically very suggestive:

Theorem 1.3.7 (W. Fenchel and L. Bunt)

If

S C IRn has no more than n connected com-ponents (in particular, if S is connected), then any x E co S can be expressed as a convex

combination oln elements oiS. 0

This result says in particular that convex and connected one-dimensional sets are the same, namely the intervals. In 1R2, the convex hull of a continuous curve can be obtained by joining all pairs of points in it. In lR3, the convex hull of three potatoes is obtained by pasting

triangles, etc.

1.4 Closed Convex Sets and Hulls

Closedness is a very important property in convex analysis and optimization. Most of the convex sets of interest to us in the subsequent chapters will be closed. It is therefore relevant to reproduce the previous section, with the word "closed" added.

As far as linearity and affinity are concerned, there is no difference; in words, equal-ities are not affected when limits are involved. But convexity is another story: when passing from (i), (i') to Definition 1.3.2, inequalities are introduced, together with their accompanying difficulty "< vs. :::;;".

To construct a convex hull co S, we followed in §1.3 the path (iii), (iii'): we took the intersection of all convex sets containing S. An intersection of closed sets is still closed, so the following definition is also natural:

Definition 1.4.1 The closed convex hull of a nonempty set S C IR.n is the intersection of all closed convex sets containing S. It will be denoted by co S. 0

Another path was also possible to construct co S, namely to take all possible convex combinations: then, we obtained co S again (Proposition 1.3.4); what about closing it? It turns out we can do that as well:

Proposition 1.4.2 The closed convex hull co S of Definition 1.4.1 is the closure cl(co S) of the convex hull ofS.

PROOF. Because cl( co S) is a closed convex set containing S, it contains co S as well.

On the other hand, take a closed convex set C containing S; being convex, C contains co S; being closed, it contains also the closure of co S. Since C was arbitrary, we

conclude

nc ::)

^{cl co S.} 0

From the very definitions, the operation "taking a hull" is monotone: if SI C S2, then aff SI C aff S2, cl SI

c

cl S2, co SI C co S2, and of course co SI C co S2' A closed convex hull does not distinguish a set from its closure, just as it does not distinguish it from its convex hull: co S

=

co(cl S)

=

co(co S) .

When computing co via Proposition 1.4.2, the closure operation is necessary (co S need not be closed) and must be performed after taking the convex hull: the operations do not commute. Consider the example of Fig. 1.4.1 :

S={(O,O)}U{(g,1) : g~O}.

It is a closed set but co S fails to be closed: it misses the half-line (lR+ , 0). Nevertheless, this phenomenon can occur only when S is unbounded, a result which comes directly from Caratheodory's theorem:

o

Fig. 1.4.1. A convex hull need not be closed

Theorem 1.4.3

If

S is bounded {resp. compact}, then co S is bounded (resp. com-pact).

PROOF. Let x =

E?,!/

^{(XiXi E} co S. If S is bounded, say by M, we can write

n+1 n+1

IIxlI~ L(Xilixill~ML(Xi =M.

i=1 i=1

I Generalities 101 Now take a sequence {xk} C co S. For each k we can choose

k k · k k k

XI' .• ·' x n+1 mS and a

=

^(al , ... , a n+l ) E Lln+1

such that xk =

:L?,!/

a7xf. Note that Lln+1 is compact. If S is compact, we can extract a subsequence as many times as necessary (not more than n

+

2 times) so that {ak} and each {xf} converge: we end up with an index set KeN such that, when k -+ +00,

{xf}kEK -+ Xi E Sand {ak}kEK -+ a E Lln+1 .

Passing to the limit for k E K, we see that {xk}kEK converges to a point X, which can be expressed as a convex combination of points of S: X E co S, whose compactness

is thus established. 0

Thus, this theorem does allow us to write:

S bounded in lRⁿ ==} co S

=

cl co S

=

co cl S .

Remark 1.4.4 Let us emphasize one point made clear by this and the previous sec-tions: a hull (linear, affine, convex or closed) can be constructed in two ways. In the inner way, combinations (linear, affine, convex, or limits) are made with points taken from inside the starting set S. The outer way takes sets (linear, affine, convex, or closed) containing S and intersects them.

Even though the first way may seem more direct and natural, it is the second which must often be preferred, at least when convexity is involved. This is especially true when taking the closed convex hull: forming all convex combinations is already a nasty task, which is not even sufficient, as one must close the result afterwards. On the other hand, the external construction of co S is more handy in a set-theoretic framework.

We will even see in §4.2(b) that it is not necessary to take in Definition 104.1 all closed convex sets containing S: only rather special such sets have to be intersected, namely

the closed half-spaces of Example 1.1.2( c). 0

To finish this section, we mention one more hull, often useful. When starting from linear combinations to obtain convex combinations in Definition 1.3 .2, we introduced two kinds of constraints on the coefficients: eTa = I and ai ~ O. The first constraint alone yielded affinity; we can take the second alone:

Definition 1.4.5 A conical combination of elements XI, ... ,Xk is an element of the form :L~=I ajXi, where the coefficients ai are nonnegative.

The set of all conical combinations from a given nonempty S C lR n is the conical

hull of S. It is denoted by cone S. 0

Note that it would be more accurate to speak of convex conical combinations and convex conical hulls. If a := Lf=1 ai is positive, we can set f3i := ai/a to realize that a conical combination of the type

k k

Dans le document Wissenschaften 305 A Series (Page 109-116)