epis (fl
t h) =
epis II+
epis12 .
(2.3.2)In fact, (X, r) E epis (fl
t h)
if and only if there is e > 0 such that II (XI)+ 12
(X2)=
r+
e for some Xl and X2 adding up to X .164
rv.
Convex Functions of Several Variables This is equivalent toIi
(Xj) < rj for some (XI, rl) and (X2, r2) adding up to (x , r)(set rj := Ij (Xj)
+
e /2 for j = I, 2, to show the ":::}" direction). This last property holds if and only if (x, r) e epis II+
episf2.
This proof explains why the infimal convolution is sometimes called the (strict)
epigraphic addition. 0
Similarly to (2.3.2), we have by construction
dom(fl
t h)
= dom II+
domh .
Let us mention some immediate properties of the infimal convolution:
11th
=h t
II (commutativity) (flt h) t h =
IIt (h t h)
(associativity)It
I{o} =I
(existence ofaneutral element in ConvlRn) II ~h =>
IIt
g ~h t
g (;t; preserves the order).(2.3.3) (2.3.4) (2.3.5)
With relation to (2.3.3), (2.3.4), more than two functions can of course be inf-convolved:
(fl
t··· t
Im)(x)=
inf{L:j=1
!.i(Xj) :L:1=1
Xj = x}.Remark 2.3.4 If CI and C2 are nonempty convex sets in Rn, then ICI
t
Icz = Icl+Cz •This is due to the additional nature of the inf-convolution, and can also be checked directly; but it leads us to an important observation: since the sum of two closed sets may not be closed, an infimal convolution need not be closed, even if it is constructed from two closed functions and if it is exact everywhere. 0 Example 2.3.5 Let C be a nonempty convex subset ofRn and M •• an arbitrary norm.
Then
IctW'W=dc,
which confirms the convexity of the distance function
dc.
It also shows that inf-convolving two non-closed functions (C need not be closed) may result in a closedfunction. 0
Example 2.3.6 Let
f
be an arbitrary convex function minorized by some affine function with slope s. Taking an affine function g = (s, .) - b, we obtainftg=g-c
where c is a constant: c = SUPy[{s, y) - f(y)]. Note: we have already encountered in Example 2.1.3 c = f*(s), the value at s of the conjugate of f.
Take in particular a constant function for g: assuming I bounded below,
Then
-g
= 1:=
inf y I(y)·It(-j)=o.
Do not believe, however, that the infimal convolution provides Conv IRn with the structure of a commutative group: in view of (2.3.5), the O-function is not the neutral element! 0 Example 2.3.7 We have seen (Proposition 1.2.1) that a convex function is indeed minorized by some affine function. The dilated versions
lu
=ulUu)
of a given convex functionI are minorized by some affine function with a slope independent of u > 0, and can be inf-convolved by each other. We obtain
lu t lUi
=lu+u
l ;the quickest way to prove this formula is probably to use (2.3.2), knowing that epis
lu
u epis
I.
In particular, inf-convolving m times a function with itself gives a sort of mean-value formula:fn(j
t··· tI)(x)
=I (fnx) .
Observe how a perspective-function gives a meaning to a non-integer number of
self-inf-convolutions. 0
Example 2.3.8 Consider two quadratic forms
f)(x) = ~(Ajx, x) for j = 1,2,
with AI and A2 symmetric positive definite. Expressing their infimal convolution as
~ inf [(Aly, y)
+
(A2(x - y), x - y}],y
the minimum can be explicitly worked out, to give (/1 ~ h)(x) = 1/2(A I2X, x}, where A 12:= (A -I A-I)-I I
+
2 .This formula has an interesting physical interpretation: consider an electrical circuit made up oftwo generalized resistors A I and A2 connected in parallel. A given current-vector i E IRn is distributed among the two branches (i
=
i I+
i2), in such a way that the dissipated power (A I ii, it}+
(A2i2, i2) is minimal (this is Maxwell's variational principle); see Fig. 2.3.1. In other words, if i = II+
12 is the real current distribution, we must have(Alii, It}
+
(A212, 12)= .
i~f .«(Alil, i l )+
(A2i2, i2)'1,+/2=1
The unique distribution (/1,/2) is thus characterized by the formulae
Alii
=
A2/2=
Al2i, (2.3.6)from which it follows that
166 N. Convex Functions of Several Variables i1
p
a
i2
Fig.2.3.1. Equivalent resistors
{AliI, ld
+
{A212, 12}=
{AI2i. i}.Thus, A 12 plays the role of a generalized resistor equivalent to A I and A2 connected in parallel; when n = 1, we get the more familiar relation 1/, = 1/'1
+
1/'2 between ordinary resistances 'I and '2. Note an interpretation of the optimality (or equilibrium) condition (2.3.6). The voltage between P and Q [resp. P' and Q'] on Fig. 2.3.1, namely AliI = A212 [resp. Al2i], is independent of the path chosen: either through AJ, or through A2, or by construction through A12.The above example of two convex quadratic functions can be extended to general functions, and it gives an economic interpretation ofthe infima! convolution: let II (x) [resp. hex)] be the cost of producing x by some production unit UI [resp. U2 ]. Ifwe want to distribute optimally the production of a given x between UI and U2, we have
to solve the minimization problem (2.3.1). 0
Remark 2.3.9 In Example III.1.2.6, we have seen two kinds of differences between sets, which may be applied to epigraphs. One difference, CI - C2
=
CI+
(-C2), leads nowhere:the opposite of an epigraph is not an epigraph. On the other hand, it is not too difficult to see that the star-difference of two epigraphs is again an epigraph; it therefore corresponds to an operation with convex functions, namely the deconvolution, or epigraphic star-difference:
Ulvh)(x):=sup{fl(x+y)-h(y): yEdomh},
Being a supremum of convex functions, the result is a convex function provided that [epiUI
v
h) =] epi II :!:epi h:;6 0.In the language of function-values, this means that, for some (xo, '0) E lin X lR:
II
(x) ~ hex - Xo) +'0 for all x E lin . In words: II must not be too larger thanh.
Indeed, the above operation can be seen to a great extent as the inverse operation of the inf-convolution. It goes without saying that the deconvolution is not commutative. A detail is worth mentioning, though: in contrast to the inf-convolution, II
v
h is now a supremum; by virtue of Proposition 2.1.2, it is therefore closed when II is closed. 0 2.4 Image of a Function Under a Linear MappingConsider a constrained optimization problem, formally written as
inf {cp(u) : c(u) ~ x}, (2.4.1)
UEU
where the optimization variable is u, the right-hand side x being considered as a parameter taken in some ordered set X. The optimal value in such a problem is then a function of x, characterized by the triple (U, cp, c), and taking its values in IRU ±{oo}.
In convex analysis and optimization, this is an important function, usually called the value function, or marginal function, or perturbation function, or primal function, etc.
Several variants of (2.4.1) are possible: we may encounter equality constraints, some constraints may be included in the objective via an indicator function, etc. A convenient unified formulation is the following:
Definition 2.4.1 Let A : IRm ~ IRn be linear and let g : IRm ~ IR U {+oo}. The image of g under A is the function Ag : IRn ~ IR U ±oo defined by
(Ag)(x) := inf {g(y) : Ay = x} (2.4.2)
(here as always, inf 0 = +00).
o
The terminology comes from the case of an indicator function: when g = Ic, with C nonempty in IRm, (2.4.2) writes
(A )(x)
= {
0 if x=
~y for some y E C, g+00
otherwise.In other words, Ag
=
IA(C) is the indicator function ofthe image of C under A (and we know from Proposition IlL 1.2.4 that this image is convex when C is convex).Even if U and X in (2.4.1) are Euclidean spaces, we seem to limit the generality when passing to (2.4.2), since only linear constraints are considered. Actually, (2.4.1) can be put in the form (2.4.2): with X = ]Rn and y = (u, v) E U X X = IRm, define Ay := v and g(y) := rp(u)
+
Ic(y), whereC := {y = (u, v) E IRm : c(u) ~ v}. (2.4.3) Note that conversely, (2.4.2) can be put in the form (2.4.1) via an analogous trick turning its equality constraints into inequalities.
Theorem 2.4.2 Let g of Definition 2.4.1 be in Conv IRm. Assume also that, for all x E IRn, g is bounded from below on the inverse image
-1
A (x)
=
{y E IRm : Ay=
x} .Then Ag E Conv]Rn.
PROOF. By assumption, Ag is nowhere -00; also, (Ag)(x) < +00 whenever x = Ay, with y E dom g. Now consider the extended operator
A': ]Rm x ]R 3 (y, r) 1-+ A' (y, r) := (Ay, r) E ]Rm x IR.
168 IV. Convex Functions of Several Variables
The set A' (epi g) =: C is convex in JRn x JR, let us compute its lower-bound function (1.3.5): for given x E JRn,
infr{r : (x, r) E C} = infy,r{r: Ay = x and g(y) ::;:; r}
=
infy{g(y) : Ay= xl =
(Ag) (x) , and this proves the convexity of Ag= lc.
-I
o Usually, A(x) contains several points - it is an affine manifold ofJRn - and Ag(x) selects one giving the least value of g (admitting that (2.4.2) has a solution). If A is invertible, Ag
=
goA -I; more generally, the above proof discloses the following interpretation: epi(Ag) is the epigraphical hull ofthe inverse image (A')(epi -I g) (a convex set in JRn x R).Corollary 2.4.3 Let (2.4.1) have the following form: U = JRP; q; E Conv JRP; X = JRn is equipped with the canonical basis; the mapping C has its components Cj E
Conv JRP for j = 1, ... , n. Suppose also that the optimal value is > -00 for all x E JRn , and that
domq;
n
domcln··· n
domcn#-
0. (2.4.4) Then the value functionVVl,c(X) := inf {q;(u) : Cj(u)::;:; Xj for j = 1, ... , n}
is in ConvJRn.
PROOF. Note first that we have assumed vVl,C(x) > -00 for all x. Take Uo in the set (2.4.4) and set M := maxj Cj(uo); then take Xo := (M, ... , M) E IRn , so that vg>,c(xo) ::;:; q;(uo) < +00. Knowing that vg>,c is an image-function, we just have to prove the convexity of the set (2.4.3); but this in turn comes immediately from the
convexity of each Cj. 0
Taking the image of a convex function under a linear mapping can be used as a mould to describe a number of other operations - (2.4.1) is indeed one of them.
An example is the infimal convolution of §2.3: with fl and
h
in Conv JRn, define g E Conv(JRn x JRn) byg(XI, X2) := fl(xl)
+
h(X2)and A : JRm x ]Rm ~ JRn by
A(XI' X2) := XI
+
X2.Then we have Ag
=
flt
hand (2.3.1) is put in the form (2.4.2). Incidentally, this shows that an image of a closed function need not be closed.Another example has lots of practical applications: the marginal function of g E
Conv(]Rn x ]Rm) is
]Rn 3 X 1--+ y(x) := inf {g(x, y) : y E JRm}.
This is the image of g under the linear mapping projecting each (x, y) E IRn x IRm onto x E IRn. It is therefore convex if g is bounded below on the set {x} x IRm for all x E IRn. Geometrically, a marginal function is given by Fig. 2.4.1, which explains why convexity is preserved: the strict epigraph of y is the projection onto IRn x IR of the strict epigraph of g (C IRn x IRm x IR). Therefore, epis y is also the image of a convex set under a linear mapping; see again Example III. 1.2.5.
Bpi"!
~ ~ - -
~-
...-.- .. Rm
Fig.2.4.1. The shadow of a convex epigraph
As seen in §2.1(b), supremization preserves convexity. Here, if g(., y) were concave for each y, y would therefore be concave: the convexity of y is a little bit surprising. Needless to say, it is the convexity of g with respect to the couple of variables x and y that is crucial.
2.5 Convex Hull and Closed Convex Hull of a Function
Given a (nonconvex) function g, a natural idea coming from§III.1.3 is to take the convex hull co epi g of its epigraph. This gives a convex set, which is not an epigraph, but which can be made so by "closing its bottom" via its lower-bound function (1.3.5).
As seen in § 111.1.3, there are several ways of constructing a convex hull; the next result exploits them, and uses the unit simplex of IRk:
.dk:= {(at, ... ,ak) EIRk : L:j=taj=l,
aj~Oforj=I,
... ,k}. (2.5.1) Proposition 2.5.1 Let g : IRn -+ IR U {+oo}, not identically +00, be minorized by an affine jUnction: for some (s, b) E IRn xR
g(x) ~ (s,x) -b for all x E IRn. (2.5.2) Then, the following three functions ft, hand h are convex and coincide on IRn:
ft(X) :=inf{r: (x,r) Ecoepig}, f2(X):=sup{h(x): hEConvlRn, h:::;;g}.
h(x):=inf{L:j=tajg(Xj): k=1,2, ...
a E .dk, Xj E domg, I:j=t ajXj = x} . (2.5.3)
170 IV: Convex Functions of Several Variables
PROOF. We denote by
r
the family of convex functions minorizing g. By assumption,r ¥
0; then the convexity of II results from § 1.3(g). described in (2.5.3), and likewise k/, (ail, {xi}, such thatk