First Examples

(a) Indicator and Support Functions Given a nonempty subset S C jRn, the func-tion Is : jRn -+ jR U Hoo} defined by

{ o

^{if XES,}

Is(x):= +00 if not

is called the indicator function of S. We mention here that other notations commonly encountered in the literature are os,

1/Is,

or even

xs.

Clearly enough, Is is [closed and]

convex if and only if S is [closed and] convex. Indeed, epi Is = S X jR+ by definition.

More generally, if I E Conv Rⁿ and if C is a nonempty convex set, the function lP(x):=

{/(X)

if

X

E C,

+00 if not

is again convex under one condition: that dom I and C have a nonempty intersection (other-wise lP would be identically +00). Furthermore, lP is closed when so are I and C. Observe in passing that lP

=

I

+

^Ie·

Attached to a nonempty subset S, another function of interest is the support function of S, already encountered in Remark III.4.1.2:

l1S(x) := sup {(s, x) : s E S}.

It turns out to be closed and convex; this is already suggested by Proposition 1.2.8 and will be confirmed below in §2.1 (b). Actually, the importance of this function will motivate an extensive development in Chap. V. Here, we just observe that, for a > 0,

sup (s, ax)

=

^{a sup (s, x) ,}

seS seS

hence l1S(ax)

=

^al1S(x): the epigraph of a support function is not only closed and convex, but it is a cone in jRn x R Its domain is also a convex cone in jRn:

doml1s

=

^{{a E jRn : 3r such that (s, a) ~ r for all s E S}.}

(b) Piecewise Affine and Polyhedral Functions Let ^(SI, bl ), ... , (sm, b_{m )} be m elements oflRⁿ x lR and consider the function

lRⁿ3xl-+j(x):=max{(sj,x}-bj: j=I, ... ,m}. (1.3.1) Such a function is suggestively called piecewise affine: lRⁿ is divided into (at most m) regions in which

j

is affine: the

jci

^h region, possibly empty, is the closed convex polyhedron

{x E lRⁿ (sio'x) -bjo ~ (Sj,x) -bj for j = 1, ... ,m}.

This terminology is slightly ambiguous, though: a function whose graph is made up of pieces of affine hyperplanes need not be convex, while (1.3.1) can be seen to produce convex functions only Gust as with a support function, convexity and closedness of

j

will be confirmed below). It can even be seen that epi

f

is a closed convex polyhedron;

but again, (1.3.1) cannot describe all polyhedral epigraphs.

A polyhedral function will be a function whose epigraph is a closed convex polyhedron. Its most general form is given by Definition III.4.2.6:

epi

f =

^{{(x, r) E lRn x lR : (Sj, x)}

+

ay ~ bj for j E J},

where J is a finite set, the (s, a, b)j being given in JRn x JR x JR, (Sj, aj) =f.

°

^(and

JRn x lR is equipped with the scalar product of a product-space). For this set to be an epigraph, each aj must be nonpositive and, if aj < 0, we may assume without loss of generalityaj = -1. Furthermore, we may denote by {l, ... , m} the subset of J such that aj = -1, and by {m

+

^{1, ... ,m}

+

^p} the rest. With these notations, we see that

f

(x) is given by (1.3.1) whenever x satisfies the set of constraints (Sj, x) ~ bj for j = m

+

^{1, ... , m}

+

^p;

otherwise,

f

(x)

= +00.

Of course, these constraints (usually termed linear, but affine is more correct) define a closed convex polyhedron.

In a word, a polyhedral function is a function which is piecewise affine on its domain, the latter being a closed convex polyhedron. Said otherwise, it is a closed convex function of the form

j +

Ip, where

j

is piecewise affine and P is a closed convex polyhedron.

(c) Norms and Distances It is a direct consequence of the axioms that a norm is a convex function, finite on the whole space (use Definition 1.1.1). More generally, let C be a nonempty convex set in lRn and, with an arbitrary norm m .

III,

define the distance function

dc(x) := inf {b - xiii: Y E C}.

To establish its convexity, Definition 1.1.1 is again convenient. Take {Yk} and {Yk}

such that, for k ~

+00, IllYk

-xm and

b k

-x'il tend to dc(x) anddc(x') respectively.

Then form the sequence Zk := aYk

+

^{(1 - a)Yk E C with a} E]O, 1 [; pass to the limit fork ~

+00

in

154 IV. Convex Functions of Several Variables

dcCax

+

^{(1 -} a)x') :::; Illzk - ax - (1 - a)x'm :::; aWYk - x~1

+

^{(1 -} a)bk - x'lIl.

Here again dom de = ]Rn; the (lower semi-)continuity of de follows.

Clearly enough,

de

⁼

del

e so, with the help of Proposition 111.2.1.8, we see that C, el C and ri C have the same distance function (associated with the same norm m .

liD·

In particular, de is 0 on the whole of el C; the following variant is slightly more accurate, in that it distinguishes between int C and bd C:

DcCx) := { dcCx) if x E CC, -dec (x) if x E C ,

where CC is the complement of C in ]Rn • Assuming that C and CC are both nonempty, it is not particularly difficult to prove that De is convex, finite everywhere, and that

intC = {x E ]Rn

bdC = ^{XE]Rn (el C)C = {x E ]Rn

DcCx) < O}, DcCx) = O}, DcCx) > O}.

(d) Quadratic Forms Let A : ]Rn ~ ]Rn be a symmetric linear operator. Then the quadratic form

f(x) := t(Ax, x}

is a convex function - with dom

f

= ]Rn - if and only if A is positive semi-definite, i.e. its eigenvalues are all nonnegative. Call Al ~ ... ~ An ~ 0 these eigenvalues; it is well-known that a basis can be formed with the corresponding eigenvectors, and that as a result,

An

IIx 112 :::;

(Ax, x}:::;

Adlxll2

for all x E]Rn.

From the first inequality, direct but somewhat tedious calculations yield, with the notation of (1.1.2):

f(ax

+

^{(1 -} a)x') :::; af(x)

+

(1 - a)f(x') - tAna(1 - a)lIx - x'1I2 •

Thus, if A is positive definite,

f

is strongly convex with modulus An > 0 (while

f

is not even strictly convex when A is degenerate). A straightforward proof comes also from a general characterization of differentiable strongly convex functions, to be seen below in Theorem 4.1.4 or 4.3.1.

For r ~ 0, the sublevel-sets of f:

Sr(f) := {x E]Rn : t(Ax, x} :::;

r}

are concentric ellipsoids: SKr(f) = .jKSr(f). Their common "shape" is given by the eigenvalues of A. These ellipsoids may be degenerate, in that they contain the subspace Ker A (one should rather speak of elliptic cylinders ifKer A

:f:.

{O}). However, Sr(f) is a neighborhood of the origin for r > 0:

Sr(f) :J R(O, 8) whenever tA182 :::; r.

( e) Sum of Largest Eigenvalues of a Matrix Instead of our working space IRn, consider the vector space Sn (IR) of symmetric n x n matrices. Denote the eigenvalues of A E Sn(lR) by AI(A) ~ ... ~ An(A), and consider the sum 1m of the m largest such eigenvalues (m :::;; n given):

m

Sn(IR):3 A ~ Im(A):= LAj(A).

j=1

This is a function of A, finite everywhere. Equip Sn (IR) with the standard dot-product oflRnxn :

n

((A, B)) := tr AB = L AijBij.

i,j=1

The function 1m turns out to have the following representation:

1m (A) = sup {(QQT, A} : QEQ},

where Q := {Q : QT Q = 1m} is the set of matrices made up ofm orthonormal n-columns. Indeed, Q is compact and the above supremum is attained at Q formed with the (normalized) eigenvectors associated withAl, ... , Am. Keeping Proposition 1.2.8 in mind, this explains that 1m is convex, as being a supremum of linear functions on Sn(IR)·

Naturally, II (A) is the largest eigenvalue of A, while In(A) is the trace of A, a linear function of A. It follows by taking differences that In - 1m (for example the smallest eigenvalue An = In - In-I) is a concave function on Sn(IR).

(t) Volume ofEUipsoids Still in the space of symmetric matrices Sn(IR), define the function

A ~ I(A):= {IOg(detA-1) if A is positive definite,

+00

if not.

It will be seen in §3.1 that the concave finite-valued function An (.) is continuous. The domain of I, which is the set of A E Sn(lR) such that An(A) > 0, is therefore open, and even an open convex cone. It turns out that

I

is convex. To see it, start from the inequality

det[aA

+

(1 - a)A'] ~ (detA)O!(detA')I-O!,

valid for all symmetric positive definite matrices A and A' (and a E]O, 1[); take the inverse of each side; remember that the inverse of the determinant is the determinant of the inverse; finally, pass to the logarithms.

Geometrically, consider again an ellipsoid

EA := {x E IRn : x T Ax:::;;

I}

where A is a symmetric positive definite matrix. Up to a positive multiplicative constant (which is the volume of the unit ball EI,J, the volume of E A is precisely JdetA-I.

156 IV. Convex Functions of Several Variables

Because dom f is open, ri dom f = int dom f = dom f, which establishes the lower semi-continuity of f on its domain. Furthermore, suppose Ak -+ A with A not positive definite; by continuity of the concave function AnO, A is positive semi-definite and the smallest eigenvalue of Ak tends to 0: f(Ak) -+ +00. The function

f

is closed.

(g) Epigraphical Hull and Lower-Bound Function of a Convex Set Given a non-empty convex set C C JRn x JR, an interesting question is: when is C the epigraph of some function

f

^{E Conv JR}n? Let us forget for the moment the convexity issue, which is not really relevant. First, the condition f(x) > -00 for all x means that C contains no vertical downward half-line:

{r E JR : (x, r) E C} is minorized for all x E JRn . (1.3.2) A second condition is also obvious: C must be unbounded from above, more precisely (x, r) E C ===} (x, r') E C for all r' > r . (1.3.3) The story does not end here, though: C must have a "closed bottom", i.e.

[(x, r') E C and r' ..t-r]

==>

(x, r) E C . (1.3.4) This time, we are done: a nonempty set C satisfying (1.3.2) - (1.3.4) is indeed an epigraph (of a convex function if C is convex). Alternatively, if C satisfying (1.3 .2), (1.3.3) has its bottom open, i.e.

(x,r)EC

==>

(x,r-e)EC forsomee=e(x,r) >0,

then C is a strict epigraph. To cut a long story short: a [strict] epigraph is a union of closed [open] upward half-lines - knowing that we always rule out the value -00.

The next interesting point is to make an epigraph with a given set: the epigraphical hull of C C JRn x JR is the smallest epigraph containing C. Its construction involves only rather trivial operations in the ordered set JR :

(i) force (1.3.3) by stuffing in everything above C: for each (x, r) E C, add to C all (x, r') with r' > r;

(ii) force (1.3.4) by closing the bottom of C: put (x, r) in C whenever (x, r') E C with r' -+ r.

These operations (i), (ii) amount to constructing a function:

X t-+ tc(x) := inf {r E JR : (x, r) E C} , (1.3.5) the lower-bound function of C; clearly enough, epi t c is the epigraphical hull of C.

We have that tc(x) > -00 for all x if (and only if) C satisfies (1.3.2).

The construction of an epigraphical hull is illustrated on Fig. 1.3.1, in which the point A and the curve

r

are not in C; nevertheless, there holds (epis is the strict epigraph)

epis

tc

C C

+

{O} X JR+ C epitc C cl(C

+

^{{O} X JR+).} (1.3.6)

A

Fig.1.3.1. The lower-bound function

Theorem 1.3.1 Let C be a nonempty subset o/F,.n x F,. satisfying (1.3.2), and let its lower-bound/unction fc be defined by (1.3.5).

(i) IfC is convex, then fc E ConvF,.n;

(ii) If C is closed convex, then fc E Conv F,.n.

PROOF. We use the analytical definition (1.1.1). Take arbitrary ^£ > 0, a E ]0, 1 [ and (Xi, ri) E C, i

=

1,2 such that

ri ::::; fc(Xi)

+

^{£ for i = 1, 2 .}

When C is convex, (axJ

+

(1 - a)x2' arJ

+

^{(l - a)r2) E C, hence}

fc(axJ

+

(1 - a)x2) ::::; arJ

+

^{(l -} a)r2::::; afc(xJ)

+

(1 - a)fc(x2)

+

^£.

The convexity of fc follows, since 8 >

°

was arbitrary; (i) is proved.

Now take a sequence {(Xk, Pk)} C epifc converging to (x, p); we have to prove fc(x) ::::; P (cf. Proposition 1.2.2). By definition of fc(Xk), we can select, for each positive integer k, a real number rk such that (Xk' rk) E C and

fc(Xk) ::::; rk::::; fc(Xk)

+ k ::::;

^Pk

⁺ k·

^(1.3.7)

We deduce first that irk} is bounded from above. Also, when fc is convex, Proposi-tion 1.2.1 implies the existence of an affine funcProposi-tion minorizing fc: {rk! is bounded from below.

Extracting a subsequence if necessary, we may assume rk --+ r. When C is closed, (x, r) E C, hence fc(x) ::::; r; but pass to the limit in (1.3.7) to see that r ::::; P; the

proof is complete. 0

Dans le document Wissenschaften 305 A Series (Page 166-171)