Separation Between Convex Sets

==> I

_IIX211

^{IIxI II}

_~ ^{~ and} IIpKo(x)lI· ^IIpK(x)1I

Take two disjoint sets SI and S2: SI

n

S2

=

0. If, in addition, SI and S2 are convex, some more can be said: a simple convex set (namely an affine hyperplane) can be squeezed between S I and S2. This extremely important property follows directly from those of the projection operator onto a convex set.

Theorem 4.1.1 Let C C ]Rn be nonempty closed convex, and let x ¢ C. Then there exists s E IRn such that

(s, x) > sup (s, y) .

yEC (4.1.1)

PROOF. Set s := x - pc(x) =1= O. We write (3.1.3) as

o

^{~ (s, Y - x}

+

s) = (s, y) - (s, x)

+

lis 112 • Thus we have

(s,x}-lIsIl2~(S,y) forallYEC,

and our s is a convenient answer for (4.1.1). 0

Naturally, s could be replaced by -s in (4.1.1) and Theorem 4.1.1 could just be stated as: there exists s' E IRⁿ such that

(s',x) < inf{(s',y} : Y E C}.

Geometrically, we know that an s =1= 0 defines hyperplanes Hs,r as in Exam-ple 1.1.2(a), which are translations of each other when r describes R With s of (4.1.1) (which is certainly nonzero!), pick

r = rs := H(s, x}

+

^{sUPyec(Y, s}).}

Then

(s, x) - rs > 0 and (s, y) - rs < 0 for all Y E C,

which can be summarized in one sentence: the affine hyperplane Hs.rs separates the two convex sets C and {x}. These two sets are in the opposite (open) half-spaces limited by that hyperplane.

Remark 4.1.2 With relation to this interpretation, Theorem 4.1.1 is often called the Hahn-Banach Theorem in geometric form. On the other hand, consider the right-hand side of (4.1.1);

it suggests a function ac : lRⁿ -+ lR U {+oo}, called the supportfunction of C:

ads) := sup{(s, y) : y E C},

which will be studied thoroughly in Chap.V. If x E C, we have by definition (s, x) ~ ads) for all s E lRn ;

but this actually characterizes the elements of C: Theorem 4.1.1 tells us that the converse is true. Therefore the test "x E C?" is equivalent to the test "(., x) ~ ac?", which compares the linear function (., x) to the function ac. With this interpretation, Theorem 4.1.1 can be formulated in an equivalent analytical way, involving functions instead of hyperplanes; this

is called the Hahn-Banach Theorem in analytical form. 0

A convenient generalization of Theorem 4.1.1 is the following:

Corollary 4.1.3 (Strict Separation of Convex Sets) Let C I, C2 be two nonempty closed convex sets with C1

n

C2 = 121. IfC2 is bounded, there exists s E IRn such that

sup (s, y) < min (s, y) .

yeC1 yeC2

(4.1.2)

4 Separation and Applications 123 PROOF. The set C1 - C2 is convex (Proposition 1.2.4) and closed (because C2 is compact). To say that C] and C2 are disjoint is to say that 0

rf.

C] - C2• so we have by Theorem 4.1.1 an s E ]Rn separating {OJ from C1 - C2 :

This means:

sup {(s, y} : y E C1 - C2 } < (s,O)

= o.

o

^{> SUPYIEC}1 (s, yd

+

^SUPy,EC, (s, -Y2)

= sUPy,Ec,(s,yd-infy,Ec,(s,Y2}.

Because C2 is bounded, the last infimum (is a min and) is finite and can be moved

to the left-hand side. 0

Once again, (4.1.2) can be switched over to inf yeCI (s, y) > maxyeC, (s, y). Using the support function of Remark 4.1.2, we can also write (4.1.2) as acl(s)

+

^{ac,(-s) <} o.

Figure 4.1.1 gives the same geometric interpretation as before. Choosing r = rs strictly between aCI (s) and -ac, (-s), we obtain a hyperplane separating C] and C2 strictly: each set is in one of the corresponding open half-spaces.

Fig.4.1.1. Strict separation of two convex sets

C1

Fig. 4.1.2. Strict separation needs compactness

Fig.4.1.3. An improper separation

When C] and C2 are both unbounded, Corollary 4.1.3 may fail- even though the role of boundedness was apparently minor, but see Fig.4.1.2. As suggested by this picture, C I and

C2 can nevertheless be weakly separated, i.e. (4.1.2) can be replaced by a weak inequality.

Such a weakening is a bit exaggerated, however: Fig. 4.1.3 shows that (4.1.2) may hold as (s, y,) = (s, Y2) foraB (y" Y2) E C, x C2

if s is orthogonal to aff( C, U C2). For a convenient definition, we need to be more demanding:

we say that the two nonempty convex sets C, and C2 are properly separated by s E IRⁿ when sup (s, yd::;;; inf (s, Y2) and inf (s, y,) < sup (s, Y2).

YIEC1 Y2 EC2 YIEC1 Y2 EC2

This (weak) proper separation property is sometimes just what is needed for technical purposes. It happens to hold under fairly general assumptions on the intersection C, n C2.

We end this section with a possible result, stated without proof.

Theorem 4.1.4 (proper Separation of Convex Sets) If the two nonempty convex sets C, and C2 satisfo (ri C,)

n

^(ri C2) = 0, they can be properly separated. 0 Observe the assumption coming into play. We have already seen it in Proposition 2.1.10, and we know from (2.1.5) that it is equivalent to

o Ii

^{ri(C, - C2) .}

4.2 First Consequences of the Separation Properties

The separation properties introduced in §4.1 have many applications. To prove that some set S is contained in a closed convex set C, a possibility is often to argue by contradiction, separating from C a point in S\ C, and then exploiting the simple structure of the separating hyperplane. Here we review some of these applications, including the proofs announced in the previous sections. Note: our proofs are often fairly short (as is that of Corollary 4.1.3) or geometrical. It is a good exercise to develop more elementary proofs, or to support the geometry with detailed calculations.

(a) Existence of Supporting Hyperplanes First of all, we note that a convex set C, not equal to the whole of lR,n, does have a supporting hyperplane in the sense of Definition 2.4.1. To see it, use first Proposition 2.1.8: cl C =1= lR,n (otherwise, we would have the contradiction C :J ri C

=

^{ri cl} C

=

^ri lR,n

=

lR,n). Then take a hyperplane separating cl C from some x

Ii

cl C: it is our asserted support of C. Actually, we can prove slightly more:

Lemma 4.2.1 Let x E bd C, where C ⁼¹⁼ f2l is convex in IRn (naturally C ⁼¹⁼ lR,n).

There exists a hyperplane supporting Cat x.

PROOF. Because C, cl C and their complements have the same boundary (remember Remark 2.1.9), a sequence {Xk} can be found such that

Xk

Ii

cl C for k

=

I, 2, ... and lim Xk = x . k-++oo

4 Separation and Applications 125 For each k we have by Theorem 4.1.1 some Sk with IIsk

II =

^{I such that}

(SkoXk - y) > 0 for all y E

e c c1e.

Extract a subsequence if necessary so that Sk ~ S (note: S

i=

0) and pass to the limit to obtain

(S, x - y) ~ 0 for all y E

e ,

which is the required result (s, x) = r ~ (s, y) for all y E

e. o

Remark 4.2.2 The above procedure may well end up with a supporting hyperplane contain-ing C: (s, x - y)

=

0 for all y E C, a result of little interest; see also Fig. 4.1.3. This can happen only when C is a "flat" convex set (dim C ~ n - 1), in which case our construction should be done in aff C, as illustrated on Fig. 4.2.1. Let us detail such a "relative" construction, to demonstrate a calculation involving affine hulls.

affC

Fig.4.2.1. Nontrivial supports

Let V be the subspace parallel to affC, with U = Vi. its orthogonal subspace: by definition, (s, y - x) = 0 for all s E U and Y E C. Suppose x E rbdC (the case x EriC is hopeless) and translate C to Co := C - {x}. Then Co is a convex set in the Euclidean space V and 0 E rbd Co. We take as in 4.2.1 a sequence {Xk} C V\ cl Co tending to 0 and a corresponding unitary Sk E V separating the point Xk from Co. The limit S =1= 0 is in V, separates (not strictly) {O} and Co, i.e. {x} and C: we are done.

We will say that Hs.r is a nontrivial support (at x) if s

rt

U, i.e. if Sy =1= 0, with the decomposition s

=

Sy

+

su. Then C is not contained in Hs. r: if it were, we would have for all y E C

r

=

^{(s, y)}

=

^{(sy, y)}

+

^{(su, x).}

In other words, (sy, .) would be constant on C; by definition of the affine hull and of V, this would mean sy E U, i.e. the contradiction sy = O. To finish, note that Su may be assumed to be 0: if sy

+

su is a nontrivial support, so is sy

=

^sy

+

0 as well; it corresponds to a

hyperplane orthogonal to C. 0

In terms of Caratheodory's Theorem 1.3.6, a consequence of our existence lemma is the following:

Proposition 4.2.3 Let S c JR.n and C := co S. Any x E C n bd C can be represented as a convex combination of n elements of

s.

PROOF. Because x E bdC, there is a hyperplane Hs,r supporting C at x: for some s =1= 0 and r E JR.,

(s,x}-r=O and (s,y}-r:::;OforallYEC. (4.2.1) On the other hand, CaratModory's Theorem 1.3.6 implies the existence of points

XI, ...

,Xn+1 in S and convex multipliers ai, ... ,an+1 such that x

= L?:II

^aixi;

and each ai can be assumed positive (otherwise the proof is finished).

Setting successively y = Xi in (4.2.1), we obtain by convex combination n+1

0= (s,x) - r = Lai«S,Xi} - r):::; 0,

i=l

so each (s, Xi) -r is actuallyO. Each Xi is therefore not only in S, but also in Hs,r, a set whose dimension is n - 1. It follows that our starting x, which is in CO{Xb •.• ,xn+d, can be described as the convex hull of only n among these Xi'S. 0

(b) Outer Description of Closed Convex Sets Closing a (convex) set consists in intersecting the closed (convex) sets containing it. We mentioned in Remark 1.4.4 that convexity allowed the intersection to be restricted to a simple class of closed convex sets: the closed half-spaces. Indeed, Lemma 4.2.1 ensures that a nonempty convex set C

;t;JR.

ⁿ has at least one supporting hyperplane: if we denote by

Hs-:r := {y E IRn : (s, y) :( r}

a closed half-space defined by a given (s, r) E JR.n x JR., (s =1= 0), then the index-set lJc := {(s,r) E JR.n x JR. : C

c

Hs-:r}

=

{(s, r) : (s, y) :::; r for all y E C} (4.2.2) is nonempty. As illustrated by Fig. 4.2.2, we can therefore intersect all the half-spaces indexed in lJc:

Fig. 4.2.2. Outer construction of a closed convex set

C c C* := n(s,r)EIJcHs-:r

=

{z E Rⁿ (s, z) :( r whenever (s, y) :::; r for all y E C} .

4 Separation and Applications 127 Theorem 4.2.4 Let 0 # ^C

¥

JR.n be convex. The set C* defined above is the closure ofe.

PROOF. By construction, C* J cl C. Conversely, take x f/ cl C; we can separate x and cl C: there exists So =1= 0 such that

(so, x) > sup (so, y} =: ro.

YEC

Then (so, ro) E "EC; but x ¢ Hs-;;,ro' hence x ¢ C*.

o

The definition of C*, rather involved, can be slightly simplified: actually, IJc is redundant, as it contains much too many r's. Roughly speaking, for given S E JR.n, just take the number

r = rs := inf {r E lR : (s, r) E IJc}

that is sharp in (4.2.2). Letting s vary, (s, rs) describes a set IJ~, smaller than IJc but just as useful. With this new notation, the expression of C* = cl C reduces to

cl C

=

{z E lRn : (s, z) ~ SUPYEc(S, y}}.

We find again the support function of Remark 4.1.2 coming into play. Chapter V will follow this development more thoroughly.

The message from Theorem 4.2.4 is that a closed convex set can thus be defined as the intersection of the closed half-spaces containing it:

Corollary 4.2.5 The data (Sj, rj) E JR.n x JR. for j in an arbitrary index set J is equivalent to the data of a closed convex set C via the relation

C=n{XElRn : (sj,x}~rj}.

JEJ

PROOF. IfC is given, define {(Sj' rj)}J := Ec as in (4.2.2). If {(Sj, rj)}J is given, the intersection of the corresponding half-spaces is a closed convex set. Note here that we can define at the same time the whole of JR.n and the empty sets as two extreme

cases. o

As an important special case, we find:

Definition 4.2.6 (Polyhedral Sets) A closed convex polyhedron is an intersection of finitely many half-spaces. Take (S1, r1), ... , (sm, rm) in lRn x JR., with Si =1= 0 for i = 1, ... , m; then define

P:= {x E lRn : (Sj,x) ';;;Jj forj

=

1, ...

,m},

or in matrix notations (assuming the dot-product for (', .}), P

=

{x E lRn : Ax ~ b} ,

if A is the matrix whose rows are Sj and b E JR.m has coordinates rj.

A closed convex polyhedral cone is the special case where b = O. o

(c) Proof of Minkowski's Theorem We turn now to the inner description of a convex set and prove Theorem 2.3.4, asserting that C

=

co ext C when C is compact convex.

The result is trivially true if dim C = 0, i.e. C is a singleton, with a unique extreme point. Assume for induction that the result is true for compact convex sets of dimension less than k; let C be a compact convex set of dimension k and take x E C.

There are two possibilities:

- If x E rbd C, §4 .2( a) tells us that there exists a nontrivial hyperplane H supporting C at x. The nonempty compact convex set C

n

H has dimension at most k - I, so X E C

n

H is a convex combination of extreme points in that set, which is an exposed face of C. Using Remark 2.4.4, these extreme points are also extreme in C.

- If x EriC (= C\ rbd C), take in C a point x' =1= x; this is possible for dim C > O.

The affine line generated by x and x' cuts rbd C in at most two points y and z (see Remark 2.1.7, there are really two points because C is compact). From the first part of the proof, y and

z

are convex combinations of extreme points in C; and so is their convex combination x (associativity of convex combinations).

(d) Bipolar of a Convex Cone The definition of a polar cone was given in §3 .2, where some interesting properties were pointed out. Here we can show one more similarity with the concept of orthogonality in linear analysis.

Proposition 4.2.7 Let K be a convex cone with polar KO; then, the polar KOO of KO is the closure of K.

PROOF. We exploit Remark 4.1.2: due to its conical character (ax E K if x E K and a > 0), cl K has a very special support function:

( ) _ { (s,O)

=

^{0 if (s, x)} ~ 0 for all x E cl K , ad K s - +00 otherwise.

In other words, acl K is 0 on KO, +00 elsewhere. Thus, the characterization

becomes

xEclK

X E clK

<=>

^{(·,x) ~aclKO}

<=>{

«(s, x) ^{(s,x) ~O} arbitrary for ^foralls s

f/.

KO!) , ^{E KO}

in which we recognize the definition of KOO .

o

Of course, if K is already closed, K ⁰⁰ = K. With relation to (a) above, we observe that every supporting hyperplane of K at x E bd K also supports K at 0: when dealing with supports to a cone, it is enough to consider linear hyperplanes only.

Remark 4.2.8 Consider the index-set E K of (4.2.2), associated with a closed convex cone K: its r-part can be restricted to to}; as for its s-part, we see from Definition 3.2.1 that it becomes KO\ to}. In other words: barring the zero-vector, a closed convex cone is the set of

(linear) hyperplanes supporting its polar at O. 0

4 Separation and Applications 129 4.3 The Lemma of Minkowski-Farkas

Because of its historical importance, we devote an entire subsection to another con-sequence of the separation property, known as Farkas' Lemma. Let us first recall a classical result from linear algebra: if A is a matrix with n rows and m columns and conical hulls, and equalities by inequalities. This gives a result dating back to the end of the XIXth Century, due to 1. Farkas and also to H. Minkowski; we state it without proof, as it will be a consequence of Theorem 4.3.4 below.

Lemma 4.3.1 (Farkas I) Let b, SI, ... , Sm be given in ]Rn. The set To express the inclusion relation between the sets (4.3.1) and (4.3.2), one also says that the inequality with b is a consequence of the joint inequalities with Sj. Another way of expressing (4.3.3) is to say that the system of equations and inequations in a

has a solution.

m

b=LajSj, aj;;::Oforj=l, ... ,m

j=1

(4.3.4)

Farkas' Lemma is sometimes formulated as an alternative, i.e. a set of two state-ments such that each one is false when the other is true. More precisely, let P and Q be two logical propositions. They are said to form an alternative if one and only one of them is true:

P ===? not Q and not P ===? Q

or, just as simply:

P {=} not Q [or Q {=} not P].

This applies to Farkas' lemma:

Lemma 4.3.2 (Farkas II) Let b, SJ, ... , Sm be given in IRn. Then exactly one of the following statements is true.

P := (4.3.4) has a solution U E IRn.

I

The system of in equations

Q:= (b,x}>O, (Sj,x}~Oforj=l, ... ,m

has a solution x E IRn .

o

Still another formulation is geometric. Call K the convex cone generated by

SI, ... , Sm; as seen in Example 3.2.2, K^O is the set (4.3.l). What Farkas' Lemma says is that

bE K [i.e. (4.3.3) holds] if and only if (b, x) ~ 0 whenever x E K^O [i.e. b E KOO ] •

More simply, Farkas' Lemma is: K^OO

=

^K; but we know from §4.2(d) that this property holds under the sole condition that K is closed. The proof of Farkas , Lemma therefore reduces to proving the following result:

Lemma 4.3.3 (Farkas III) Let SI, •.. , Sm be given in IRn. Then the convex cone K := cone{sJ, ... , sml =

{L:j=1

^{UjSj : Uj}

~

Ofor j = 1, ... , m}

is closed.

PROOF. It is quite similar to that ofCaratheodory's Theorem 1.3.6. First, the proof is easy if the Sj'S are linearly independent: then, the convergence of

m

k_'" k. fi

x - ~ Uj sJ or k -+ 00 (4.3.5)

j=1

is equivalent to the convergence of each

{ujl

to some Uj, which must be nonnegative if each

uj

in (4.3.5) is nonnegative.

Suppose, on the contrary, that the system

E}=I

f3jSj = 0 has a nonzero solution f3 E IRm and assume f3j < 0 for some j (change f3 to -f3 if necessary). As in the proof of Theorem 1.3.6, write each x E K as

m m

X = LUjSj = L[Uj +t*(x)f3j]Sj = L ujSj,

j=1 j=1 Hi(x)

where

4 Separation and Applications 131

Now, if there is some i such that the generators of Ki are linearly dependent, we repeat the argument for a further decomposition of this Ki. After finitely many such operations, we end up with a decomposition of K as a finite union of polyhedral convex cones, each having linearly independent generators. All these cones are therefore closed (first part of the proof), so K is closed as well. 0

We are now in a position to state a general version of Farkas' Lemma, with non-homogeneous terms and infinitely many inequalities. Its proof uses in a direct way the separation Theorem 4.1.1.

Theorem 4.3.4 (Generalized Farkas) Let be given (b, r) and (Sj' Pj) in lRn x lR, where j varies in an (arbitrary) index set J. Suppose that the system of inequalities (Sj'x) ~ Pj for all j E J (4.3.6)

For each x satisfying (4.3.6) we can write

(b, x) ~ r - ao ~ r. (4.3.7)

sup [(s, d) - pt] < (b, d) - rt. (4.3.8)

(s,p)EK

It follows first that the left-hand supremum is a finite number K. Then the conical character of K implies K ~ 0, because aK ~ K for all a > 0; actually K = 0 because (0,0) E K. In summary, we have singled out (d, t) E lR.ⁿ x lR. such that

(*) (**)

t;::O

(Sj' d) - Pj t ~ 0 for all j E J (b, d) - rt > O.

Now consider two cases:

[take (0, 1) E K]

[take (Sj. Pj) E K]

[don't forget (4.3.8)]

- If t > 0, divide (*) and (**) by t to exhibit the point x

=

d / t violating (i).

- If t = 0, take Xo satisfying (4.3.6). Observe from (*) that, for all a > 0, the point x(a) = Xo

+

^ad satisfies (4.3.6) as well. Yet, let a ~

+00

ⁱⁿ

(b, x(a») = (b, xo)

+

^{a(b, d)}

to realize from (**) that x(a) violates (i) if a is large enough.

Thus we have proved in both cases that "not (ii)

=>

not (i)".

o

We finish with two comments relating Theorem 4.3.4 with the previous forms of Farkas' Lemma. Take first the homogeneous case, where r and the Pj'S are all zero.

Then the consistency assumption is automatically satisfied (by x

=

0) and the theorem says:

(i')[(sj, x) ~ 0 for j E J]

==>

[(b, x) ~ 0]

is equivalent to (ii') b E cone{sj : j E J}.

Second, suppose that J

=

{I, ... , m} is a finite set, so the set described by (4.3.6) becomes a closed convex polyhedron, assumed nonempty. A handy matrix notation (assuming the dot-product for (-,

.»,

^{is AT X ~ p, if A} is the matrix whose columns are the s/s, and P ^{E ]Rm} has the coordinates PI> ... , Pm. Then Theorem 4.3.4 writes:

(i")

{x

^{E]Rn :} AT

x

~ p} C

{x

^E lR.n : b ^T

x

~ r}

is equivalent to

(ii") 3a E lR.^m such that a;:: 0, Aa = b, pTa ~ r .

Indeed, it suffices to recall Lemma 4.3.3: the conical hull involved in (ii) of Theorem 4.3.4 is already closed. Beware that the last relation in (ii") is really an inequality.

Dans le document Wissenschaften 305 A Series (Page 136-147)