Convex <span class="mathjax-formula">$\operatorname{SO}(N)\times \operatorname{SO}(n)$</span>-invariant functions and refinements of von Neumann’s inequality

ANNALES

DE LA FACULTÉ DES SCIENCES

Mathématiques

BERNARDDACOROGNA, PIERREMARÉCHAL

ConvexSO(N)×SO(n)-invariant functions and refinements of von Neumann’s inequality

Tome XVI, n^o1 (2007), p. 71-89.

<http://afst.cedram.org/item?id=AFST_2007_6_16_1_71_0>

© Université Paul Sabatier, Toulouse, 2007, tous droits réservés.

L’accès aux articles de la revue « Annales de la faculté des sci- ences de Toulouse Mathématiques » (http://afst.cedram.org/), implique l’accord avec les conditions générales d’utilisation (http://afst.cedram.

org/legal/). Toute reproduction en tout ou partie cet article sous quelque forme que ce soit pour tout usage autre que l’utilisation à fin strictement personnelle du copiste est constitutive d’une infraction pénale. Toute copie ou impression de ce fichier doit contenir la présente mention de copyright.

cedram

Article mis en ligne dans le cadre du

Centre de diffusion des revues académiques de mathématiques

pp. 71–89

Convex

-invariant functions and refinements of von Neumann’s inequality

Bernard Dacorogna⁽¹⁾, Pierre Mar´echal⁽²⁾

ABSTRACT. — A function f on M_N×n(R) which is SO(N)×SO(n)- invariant is convex ifand only ifits restriction to the subspace ofdiagonal matrices is convex. This results from Von Neumann type inequalities and appeals, in the case whereN=n, to the notion ofsigned singular value.

R^´ESUM´E.— Une fonctionfsurM_N×n(R) qui est SO(N)×SO(n)-invariante est convexe si et seulement si sa restriction au sous-espace des matrices diagonales est convexe. Ceci r´esulte de variantes de l’in´egalit´e de Von Neu- mann et fait appel, dans le cas o`uN=n, `a la notion de valeur singuli`ere sign´ee.

1. Introduction

A functionf:M_n(R)→[−∞,∞] is said to be SO(n)×SO(n)-invariant if

∀ξ∈M_n(R), ∀Q, R∈SO(n), f(QξR^t) =f(ξ).

The specification of an SO(n)×SO(n)-invariant function f is easily seen to be equivalent to that of a function g:Rⁿ → Rwhich is invariant under permutation of the components and under change of sign of an even number of components. We will be mostly concerned with following fact:

An SO(n)×SO(n)-invariant function f is convex if and only if its restriction toD_n(R), the subspace of M_n(R) of diagonal matrices, is convex.

This was established by Dacorogna and Koshigoe [4] in the casen= 2, and later by Vincent [17] in the general case, as a consequence of the convexity

(∗) Re¸cu le 24 mai 2005, accept´e le 22 juin 2005

(1) EPFL, CH-1015 Lausanne, Switzerland.

bernard.dacorogna@epfl.ch.

(2) Universit´e Paul Sabatier, Institut de math´ematiques, F-31062 Toulouse cedex 9, France.

marechal@mip.ups-tlse.fr.

theorem of Kostant [7]. An analogous statement, for convex O(n)×O(n)- invariant functions, is well known (see Dacorogna and Marcellini [3] ; see also Ball [1] and Le Dret [9]).

On the other hand, Von Neumann’s trace inequality, namely, tr(ξη^t)

n k=1

λ_k(ξ)λ_k(η), (1.1)

where λ1(ξ). . .λn(ξ) denote the increasingly ordered singular values ofξ, can be significantly refined. On denoting byµ1(ξ), . . . , µn(ξ) thesigned singular values, that is,

µ1(ξ) := sgn(detξ)λ1(ξ) and µk(ξ) :=λk(ξ) for k2, the following holds:

tr(ξη^t) n k=1

µ_k(ξ)µ_k(η). (1.2)

This inequality, which was first established by Rosakis [13], is strictly more stringent than that of Von Neumann, and contains it as an immediate con- sequence.

The purposes of this paper are the following. First, we give a variant of Rosakis’ proof of Inequality (1.2). This variant is self-contained, in the sense that it does not use Von Neumann’s inequality. Second, we establish the link between Inequality (1.2) and the above mentioned result on convex SO(n)×SO(n)-invariant functions. Our strategy relies mostly on convex duality rather than Lie theoretic arguments (as in Vincent [17]). Third, we consider analogous results for rectangular matrices. In the latter case, the notion of signed singular value does not make sense, but the notions of O(N)×O(n)-invariance and SO(N)×SO(n)-invariance coincide when N =n(see Proposition 2.2 below). A rectangular version of Von Neumann’s trace inequality then allows to establish the desired properties.

We now introduce some notation. We denote byM_N×n(R) andD_N×n(R) the space of (N×n)-matrices and the subspace of diagonal (N×n)-matrices, respectively. (A matrix M = (m_ij) ∈ M_N×n(R) is said to be diagonal if m_ij = 0 whenever i = j.) If N = n, we write M_n(R) = M_N×n(R) and D_n(R) = D_N×n(R). We denote by ·,· the standard scalar product in M_N×n(R):

M, N= N j=1

n k=1

MjkNjk= tr(M N^t) = tr(M^tN).

For allx∈Rⁿ, we denote by diag_N×n(x) the diagonal matrix inM_N×n(R) whose diagonal elements are the components ofx. In the square case (N = n), we will often write diag = diag_N×n.

For allm∈N^∗, we denote by GL(m), O(m) and SO(m) the group of all invertible (m×m)-matrices, the subgroup of all orthogonal matrices and the subgroup of all orthogonal matrices with determinant 1, respectively.

We denote by Π(m) the subgroup of O(m) which consists of the matrices having exactly one nonzero entry per line and per column which belongs to {−1,1}, by Π_e(m) the subgroup of Π(m) which consists of the matrices having an even number of entries equal to −1, and by S(m) the subgroup of Π_e(m) of all permutation matrices. Notice that Π_e(m) is the subgroup generated by the permutation matrices and diag_m×m(−1,−1,1, . . . ,1), and that

card Π_e(m) = 2^m−1m!.

Notice also that GL(m), O(m), SO(m), Π(m), Πe(m) and S(m) are stable under transposition.

2. Preliminaries

We consider functions of matrices inMN×n(R) either in the square case (N = n) or in the rectangular case (N = n). In the latter case, we will always assume thatN > n, the opposite case being entirely analogous.

Throughout, we will write, for allξ∈M_N×n(R),

λ(ξ) = (λ₁(ξ), . . . , λ_n(ξ)) and µ(ξ) = (µ₁(ξ), . . . , µ_n(ξ)).

Recall that, for all ξ ∈ MN×n(R), we can find Q∈ O(N) and R ∈ O(n) such that

ξ=QΛR^t where Λ := diag_N×n(λ1(ξ), . . . , λn(ξ))

(see [6], Theorem 7.3.5). It is clear that, in the square case (N =n), we may choose QandR in SO(n) provided thatλ1(ξ) is replaced byµ1(ξ) in Λ.

Given a subgroupGof GL(N) and a subgroupH of GL(n), we say that a functionf:M_N×n(R)→[−∞,∞] isG×H^t-invariant if

∀ξ∈M_N×n(R), ∀Q∈G, ∀R∈H, f(QξR^t) =f(ξ).

All subgroupsG, Hencountered in this paper are stable under transposition, so we will equivalently speak ofG×H-invariance. For example, a function f:MN×n(R)→[−∞,∞] is O(N)×O(n)-invariant if

∀ξ∈MN×n(R), ∀Q∈O(N), ∀R∈O(n), f(QξR^t) =f(ξ).

Given any subgroupGof GL(n), we say that a functiong:Rⁿ →[−∞,∞] isG-invariant if

∀x∈Rⁿ, ∀M ∈G, g(Mx) =g(x).

It is customary to refer to S(n)-invariant functions assymmetric functions.

The following proposition is an immediate consequence of the Singular Value Decomposition (see [6], Theorem 7.3.5, for example).

Proposition 2.1. —

(i) Letf:M_n(R)→[−∞,∞]. Thenf isSO(n)×SO(n)-invariant if and only iff satisfies

f =f◦diag◦µ,

and g := f◦diag is then the unique Π_e(n)-invariant function such thatf =g◦µ.

(ii) Letf:MN×n(R)→[−∞,∞], whereN n. Thenf isO(N)×O(n)- invariant if and only iff satisfies

f =f◦diag_N×n◦λ,

andg:=f◦diag_N×nis then the uniqueΠ(n)-invariant function such thatf =g◦λ.

It is clear that, ifN =n, the notions of O(N)×O(n), SO(N)×O(n) and O(N)×SO(n)-invariance coincide, but differ from that of SO(N)×SO(n)- invariance. However, ifN =n, all four notions do coincide:

Proposition 2.2. — Letf:M_N×n(R)→[−∞,∞], where N > n. Then the following are equivalent.

(i) f isO(N)×O(n)-invariant;

(ii) f isSO(N)×SO(n)-invariant.

Proof. — Obviously, we need only prove that (ii) implies (i). We will see that, if f is SO(N)×SO(n)-invariant, then f = f ◦diag_N×n◦λ. The conclusion will then follow from Proposition 2.1.

Letξ ∈ MN×n(R). By the Singular Value Decomposition, there exists U ∈O(N),V ∈O(n) such that

ξ=UΛV^t, where Λ := diag_N×n(λ1(ξ), . . . , λn(ξ))

For allm1, letH_m:= diag(−1,1. . . ,1) andK_m:= diag(1, . . . ,1,−1) in M_m(R).

• IfU ∈SO(N) andV ∈SO(n), then

f(ξ) =f(Λ) = (f ◦diag_N×n◦λ)(ξ). (2.1)

• If U ∈ O(N)\SO(N) and V ∈ O(n)\SO(n), we may write Λ = HNΛHn, so that UΛV^t = (U HN)Λ(V Hn)^t, where U HN ∈ SO(N) andV H_n ∈SO(n). Thus Equation (2.1) holds.

• If U ∈ O(N)\SO(N) and V ∈ SO(n), we may write Λ = K_NΛ, so that UΛV^t = (U K_N)ΛV^t, where U K_N ∈ SO(N). Thus Equa- tion (2.1) holds.

• IfU ∈SO(N) andV ∈O(n)\SO(n), we may write Λ =H_NK_NΛH_n, so thatUΛV^t= (U HNKN)Λ(V Hn)^t, where U HNKN ∈SO(N) and V Hn∈SO(n). Thus Equation (2.1) holds.

Thus we have shown thatf =f◦diag_N×n◦λ.

3. Von Neumann type inequalities

This section is devoted to Von Neumann type Inequalities (see Theo- rem 3.3 below). Our strategy is inspired by Rosakis’ paper [13]. It combines a variational argument and the resolution of some discrete optimization problem. The main advantage of our proof is that we get the classical von Neumann inequality as a by product, while Rosakis uses it in his proof. We will need the following technical results.

Lemma 3.1. —

(i) Let D ∈ Mn(R) be diagonal, with diagonal entries whose absolute values are pairwise distinct. If M ∈ M_n(R) is such that both M D andDM are symmetric, thenM is diagonal.

(ii) Let D ∈M_N×n(R) be diagonal (N > n), with nonzero diagonal en- tries whose absolute values are pairwise distinct. If M ∈M_n×N(R) is such that bothM D andDM are symmetric, thenM is diagonal.

Proof. —

(i) LetD= diag(d₁, . . . , d_n). Assuming thatM D andDM are symmet- ric, we have

M D²=DM^tD=D²M,

whereD²is diagonal and has pairwise distinct diagonal entries. Now, for alli, j∈ {1, . . . , n},

(M D²)ij =Mijd²_j and (D²M)ij =d²_iMij. Ifi=j, thend²_i =d²_j, which shows thatMij = 0.

(ii) Let us write D^t = [∆;Z], with ∆ = diag(d1, . . . , dn) and Z = 0 ∈ Mn×(N−n)(R). Assuming thatM D andDM are symmetric, we have

M DD^t=D^tM^tD^t=D^tDM.

On writingM = [M1;M2] withM1∈Mn(R) andM2∈M_n×(N−n)(R), the above equation says that

M₁∆²= ∆²M₁ and ∆²M₂= 0.

Part (i) then implies that M1 is diagonal, and since ∆² is diagonal with nonzero diagonal entries, we haveM2= 0.

The following proposition may be regarded as a primary version of In- equality (1.2), for diagonal matrices.

Proposition 3.2. — Letb1, . . . , bn∈Rsatisfy|b1|b2. . .bn. Let a₁, . . . , a_n∈R, and let τ be a permutation of{1, . . . , n}such that |a_τ(1)| . . .|a_τ(n)|.

(i) If_n

j=1a_j 0, thena₁b₁+· · ·+a_nb_n|a_τ(1)|b₁+. . .+|a_τ(n)|b_n; (ii) ifn

j=1a_j<0, thena₁b₁+· · ·+a_nb_n−|a_τ(1)|b₁+. . .+|a_τ(n)|b_n. In other words, if bbelongs to the set

Γe:=

x= (x1, . . . , xn)∈Rⁿ |x1|x2. . .xn

, then

max

M∈Πe(n)Ma,b=µ(diaga),b.

Proof. — The casen= 2 is straightforward. It says that, if|b₁|b₂and ifτ∈S(2) is such that|a_τ(1)||a_τ(2)|, then

(i’) a1a20 impliesa1b1+a2b2|aτ(1)|b1+|aτ(2)|b2, and (ii’) a1a2<0 impliesa1b1+a2b2−|aτ(1)|b1+|aτ(2)|b2.

We will use these rules to prove the result in the general case. The given permutation τ will be decomposed as a well chosen product of transposi- tions, each of them giving rise to an inequality via (i’) or (ii’). For example, assuming that|ak||ak+1| for somek, we can write, ifakak+10,

a₁b₁+· · ·+a_kb_k+a_k+1b_k+1+· · ·+a_nb_n

a₁b₁+· · ·+|a_k+1|b_k+|a_k|b_k+1+· · ·+a_nb_n (3.1) or, ifakak+1<0,

a1b1+· · ·+akbk+ak+1bk+1+· · ·+anbn

a₁b₁+· · · − |a_k+1|b_k+|a_k|b_k+1+· · ·+a_nb_n. (3.2) Since thebkwill keep the same place throughout, we will symbolize inequal- ities such as (3.1), (3.2) by

(a1, . . . , ak, ak+1, . . . , an) → (a1, . . . ,|ak+1|,|ak|, . . . , an), (3.3) (a₁, . . . , a_k, a_k+1, . . . , a_n) → (a₁, . . . ,−|a_k+1|,|a_k|, . . . , a_n), (3.4) respectively.

We first consider the case where b1 > 0. Suppose that _n

j=1aj 0.

Clearly,

(a1, . . . , an)→(|a1|, . . . ,|an|).

Now,|a_τ(n)|canmigraterightward by means of a transposition of type (3.3).

Thus

(|a₁|, . . . ,|a_n|)→(|a₁|, . . . ,|a_τ(n)−1|,|a_τ(n)+1|, . . . ,|a_n−1|,|a_τ(n)|).

Repeating this process with |aτ(n−1)|, |aτ(n−2)| and so on will give rise to the desired inequality. Suppose next that n

j=1a_j < 0. In this case, we decide to replace all but one of the negativea_j by their absolute values: for example, if a_k is negative,

(a1, . . . , an)→(|a1|, . . . ,|ak−1|,−|ak|,|ak+1|, . . . ,|an|).

Now we let|aτ(n)|migrate rightward, using either a transposition of type (3.3) or a transposition of type (3.4) according to the signs of the elements under

consideration. Each transposition leaves one negative element. Repeating this process with|a_τ(n−1)|,|a_τ(n−2)| and so on will eventually sort the|a_j| according toτ, and give rise to

(|a₁|, . . . ,|a_k−1|,−|a_k|,|a_k+1|, . . . ,|a_n|)

→ (|aτ(1)|,|aτ(2)|, . . . ,−|aτ(l)|, . . . ,|a_τ(n−1)|,|aτ(n)|).

Finally, it is clear that the minus sign is allowed to migrate leftward, since all elements are now sorted increasingly. Therefore,

(|a_τ(1)|,|a_τ(2)|, . . . ,−|a_τ(l)|, . . . ,|a_τ(n−1)|,|a_τ(n)|)

→ (−|aτ(1)|,|aτ(2)|, . . . ,|aτ(n)|) and we are done.

Finally, the case whereb1<0 is easily obtained from the above strategy by observing thata1b1+· · ·+anbn= (−a1)(−b1) +a2b2+· · ·+anbn.

We are now ready to prove the main theorem of this section.

Theorem 3.3. —

(i) Letξ, η∈M_n(R). Then

Q,Rmax∈SO(n){tr(QξR^tη^t)}= n j=1

µj(ξ)µj(η).

Consequently,tr(ξη^t)n

j=1µj(ξ)µj(η).

(ii) Letξ, η∈MN×n(R)whereN n. Then

Qmax∈O(N) R∈O(n)

{tr(QξR^tη^t)}= n j=1

λj(ξ)λj(η).

Consequently,tr(ξη^t)_n

j=1λ_j(ξ)λ_j(η).

Proof. —

(i) As already said, the beginning of our proof follows the one of Rosakis [13]. Observe first that we can assume thatη satisfies

η= diag (µ₁(η), . . . , µ_n(η)). (3.5) As a matter of fact, suppose that the result is proved in this case.

Letζ be any element of M_n(R), and letU, V ∈SO(n) be such that ζ=U M V^t, withM := diag (µ₁(ζ), . . . , µ_n(ζ)). For allQ, R∈SO(n),

tr(QξR^tζ^t) = tr(QξR^tV M U^t) = tr((U^tQ)ξ(R^tV)M).

SinceU^tSO(n) = SO(n)V = SO(n), we see that

Q,R∈SO(n)max {tr(QξR^tζ^t)} = max

Q1,R1∈SO(n){tr(Q₁ξR^t₁M)}

= n j=1

µ_j(ξ)µ_j(M)

= n j=1

µ_j(ξ)µ_j(ζ),

where the second equality results from the fact thatM satisfies Con- dition (3.5).

Notice that we can also assume, in addition to Condition (3.5), that ηsatisfies|µ1(η)|< µ2(η)< . . . < µn(η), since a continuity argument will then allow to extend the result to the case of wide inequalities.

Since SO(n)×SO(n) is compact and the function (Q, R)→tr(QξR^tη^t) is continuous, there existQ₀, R₀∈SO(n) such that

tr(Q₀ξR^t₀η^t) = max

Q,R∈SO(n){tr(QξR^tη^t)}. (3.6) We will prove thatQ₀andR₀ must be such thatQ₀ξR^t₀is diagonal.

Let A and B be skew-symmetric matrices, that is, A^t = −A and B^t=−B. For allt∈R, let

Q(t) :=e^tAQ₀ and R(t) :=e^tBR₀. Clearly,Q(t) andR(t) are in SO(n), and the function

ϕ(t) := tr(Q(t)ξR(t)^tη^t)

is differentiable. The optimality condition (3.6) implies that t = 0 maximizesϕ. Consequently,

0 =ϕ(0) = tr(AQ0ξR^t₀η^t) + tr(Q0ξR^t₀B^tη^t).

We have therefore shown that, for all skew-symmetric matrices A andB,

tr(AQ0ξR^t₀η^t) =A,(Q0ξR^t₀η^t)^t= 0, tr(η^tQ0ξR^t₀B^t) =(η^tQ0ξR^t₀), B= 0.

Recall thatMn(R) is the orthogonal direct sum ofSn(R) andAn(R), the subspaces of symmetric and skew-symmetric matrices, respec- tively. Therefore, the above conditions tell us that Q0ξR^t₀η^t and

η^tQ₀ξR^t₀must be symmetric. Lemma 3.1(i) then implies thatQ₀ξR^t₀ is diagonal. We have shown so far that

Q,R∈SO(n)max {tr(QξR^tη^t)}= tr(Q₀ξR^t₀η^t),

where Q₀, R₀ ∈ SO(n) are such that Q₀ξR^t₀ is diagonal. It remains to see thatQ₀ andR₀ are such that

Q₀ξR^t₀= diag (µ₁(ξ), . . . , µ_n(ξ)).

But this is an immediate consequence of Proposition 3.2.

(ii) The case where N = n, which results immediately from Part (i), corresponds to Von Neumann’s inequality itself. Thus, let us assume that N > n. The argument is analogous to that of Part (i), so we merely outline the main steps. We can assume thatη satisfies

η= diag_N×n(λ₁(η), . . . , λ_n(η)), (3.7) with 0 < λ₁(η) < . . . < λ_n(η), the case of wide inequalities being deduced by a passage to the limit. The compactness of O(N)×O(n) and the continuity of the function (Q, R) → tr(QξR^tη^t) imply the existence ofQ₀∈O(N) andR₀∈O(n) such that

tr(Q₀ξR^t₀η^t) = max

Q∈O(N) R∈O(n)

{tr(QξR^tη^t)}. (3.8)

The same variational argument as that of Part (i), together with Lemma 3.1(ii), shows thatQ0 and R0 must be such that Q0ξR^t₀ is diagonal. Finally, it is clear that, among all diagonal (N×n)-matrices ξ with prescribed singular valuesλ1(ξ), . . . , λn(ξ), the matrix

diag_N×n(λ1(ξ), . . . , λn(ξ)) maximizes tr(ξη^t). Thus we must have

Q₀ξR^t₀= diag_N×n(λ₁(ξ), . . . , λ_n(ξ)), and the result follows.

Observe that, in the square case,

−tr(ξη^t) = tr(−ξη^t)

j

λj(−ξ)λj(η) =

j

λj(ξ)λj(η),

so that

|tr(ξη^t)|

j

λ_j(ξ)λ_j(η)

for all ξ, η∈M_n(R). It is worth noticing that the analogous inequality for signed singular values holds as well ifnis even.

Corollary 3.4. — Letξ, η∈M_n(R). Ifn is even, then

|tr(ξη^t)|

j

µ_j(ξ)µ_j(η). (3.9)

If nis odd, Inequality(3.9)is false in general.

Proof. — If n is even, then det(−ξ) = detξ andµj(−ξ) =µj(ξ) for all j= 1, . . . , n. Since tr(−ξη^t) =−tr(ξη^t), we conclude that both tr(ξη^t) and

−tr(ξη^t) are majorized by

jµj(ξ)µj(η).

Ifnis odd, counterexamples are easy to construct. For example, ifn= 3, let ξ := diag (−1,1,1) and η := diag (1,−1,−1). Then tr(ξη^t) = −3 and

jµ_j(ξ)µ_j(η) = 1.

4. Invariance and convexity

In this section and the following, we refer to notions pertaining to convex analysis. Our reference books for these sections are those by Hiriart-Urruty and Lemar´echal [5] and by Rockafellar [14].

Recall that ifGis a subgroup of GL(n), then the setG^t:={M^t|M ∈G} is also a subgroup of GL(n).

Lemma 4.1. — Let g:Rⁿ → [−∞,∞] and let G be any subgroupof GL(n). Consider the following statements:

(i) g isG-invariant;

(ii) g isG^t-invariant.

Then(i)implies(ii), and the converse is true if g is closed proper convex.

Proof. — Suppose thatg isG-invariant, and letM ∈G. Then g(M^tξ) = sup

M^tξ,x −g(x) x∈Rⁿ

= sup

ξ, Mx −g(Mx) x∈Rⁿ

= sup

ξ,y −g(y) y∈Rⁿ

= g(ξ).

Thus g is G^t-invariant. If g is closed proper convex, the converse follows dually, sinceg=g in this case.

Lemma 4.2. — Let f:MN×n(R) → [−∞,∞], let G be a subgroupof GL(N), and let H be a subgroupof GL(n). Consider the following state- ments:

(i) f isG×H^t-invariant;

(ii) f isG^t×H-invariant.

Then(i)implies(ii), and the converse is true if f is closed proper convex.

Proof. — Suppose thatf isG×H^t-invariant, and letU ∈GandV ∈H. For allξ, X∈MN×n(R), we have

U^tξV, X= tr(U^tξV X^t) = tr(ξV X^tU^t) =ξ, U XV^t. Thus

f(U^tξV) = sup

U^tξV, X −f(X) X∈M_n(R)

= sup

ξ, U XV^t −f(U XV^t) X ∈M_n(R)

= sup

ξ, Y −f(Y) Y ∈M_n(R)

sinceX →U XV^t is bijective. Therefore,f(U^tξV) = f(ξ), so that f is G^t×H-invariant. Iff is closed proper convex, the converse follows dually, sincef=f in this case.

Theorem 4.3. —

(i) Letf:Mn(R)→(−∞,∞]beSO(n)×SO(n)-invariant, and letg:Rⁿ → (−∞,∞]be the uniqueΠ_e(n)-invariant function such thatf =g◦µ.

Then

f=g◦µ.

(ii) Let N n, let f:MN×n(R)→(−∞,∞] be O(N)×O(n)-invariant, and letg:Rⁿ→(−∞,∞]be the uniqueΠ(n)-invariant function such thatf =g◦λ. Then

f =g◦λ.

Proof. — (i) We have:

f(ξ) = sup

X∈Mn(R⁾{ξ, X −f(X)}

= sup

X∈Mn(R⁾{ξ, X −g(µ(X))}

= sup

X∈Mn(R⁾

sup

Q,R∈SO(n)

ξ,(QXR^t) −g(µ(QXR^t))

But

ξ,(QXR^t)= tr(ξ^tQXR^t) = tr(QXR^tξ^t) and µ(QXR^t) =µ(X) for allQ, R∈SO(n), so that, by Theorem 3.3(i), the inner supremum is equal to n

k=1µ_k(X)µ_k(ξ)−g(µ₁(X), . . . , µ_n(X)). Furthermore, µ(X) runs over

Γ_e=

x= (x₁, . . . , x_n)∈Rⁿ |x₁|x₂. . .x_n asX runs overM_n(R). Therefore,

f(ξ) = sup

x∈Γe

{µ(ξ),x −g(x)}. (4.1) On the other hand, lety∈Γe. Then, for allx in

Πe(n)x={Mx|M ∈Πe(n)},

g(x) =g(x) andy,xy,xby Proposition 3.2, so that g(y) := sup

x∈Rⁿ{y,x −g(x)}= sup

x∈Γe

{y,x −g(x)}. (4.2) The result follows from Equations (4.1) and (4.2).

(ii) We have:

f(ξ) = sup

X∈MN×n(R⁾{ξ, X −f(X)}

= sup

X∈M_N×n(R⁾



 sup

Q∈O(N) R∈O(n)

ξ,(QXR^t) −f(QXR^t)





= sup

X∈M_N×n(R⁾



 sup

Q∈O(N) R∈O(n)

ξ,(QXR^t)

−f(X)



.

By Theorem 3.3(ii), sup

Q∈O(N) R∈O(n)

ξ,(QXR^t)

= sup

Q∈O(N) R∈O(n)

tr(QXR^tξ^t)

= n k=1

λk(X)λk(ξ).

Furthermore,λ(X) runs over Γ =

x= (x₁, . . . , x_n)∈Rⁿ 0x₁. . .x_n asX runs over MN×n(R). Therefore,

f(ξ) = sup

x∈Γ{λ(ξ),x −g(x)} (4.3) On the other hand, lety∈Γ. Then, for allx in

Π(n)x={Mx|M ∈Π(n)}, g(x) =g(x) andy,xy,x, so that

g(y) := sup

x∈Rⁿ{y,x −g(x)}= sup

x∈Γ{y,x −g(x)}. (4.4) The result follows from Equations (4.3) and (4.4).

Remark 4.4. — The set of all transformations ξ → U ξV^t with U, V ∈ SO(n), endowed with the composition, is obviously a group which is iso- morphic to the product group SO(n)×SO(n). By abuse of notation, we may denote this group by SO(n)×SO(n). It results from Theorem 3.3 that the system (M_n(R),SO(n)×SO(n),diag ◦µ) satisfies:

(i) diag ◦µis SO(n)×SO(n)-invariant;

(ii) for all ξ ∈ Mn(R), there exists (U, V) ∈ SO(n)×SO(n) such that ξ=Udiag (µ(ξ))V^t;

(iii) for allξ, η∈M_n(R), tr(ξη^t)tr(diag (µ(ξ)) diag (µ(η))).

According to Lewis’ terminology [10], (Mn(R),SO(n)×SO(n),diag ◦µ) is anormal decomposition system. Our preceding results also show that, simi- larly, (MN×n(R),O(N)×O(n),diag_N×n◦λ) is a normal decomposition sys- tem.

We are now ready to prove the main theorem.

Theorem 4.5. —

(A) Letf:M_n(R)→(−∞,∞]beSO(n)×SO(n)-invariant, and letg:Rⁿ → (−∞,∞]be the uniqueΠ_e(n)-invariant function such thatf =g◦µ.

Then the following are equivalent:

(i) f is closed proper convex;

(ii) the restriction off toD_n(R), the subspace ofM_n(R)of diagonal matrices, is closed proper convex;

(iii) g is closed proper convex.

(B) LetN > n, letf:M_N×n(R)→(−∞,∞]beSO(N)×SO(n)-invariant or, equivalently, O(N)×O(n)-invariant, and let g:Rⁿ → (−∞,∞] be the uniqueΠ(n)-invariant function such that f =g◦λ. Then the following are equivalent:

(i) f is closed proper convex;

(ii) the restriction of f to D_N×n(R), the subspace of M_N×n(R) of diagonal matrices, is closed proper convex;

(iii) g is closed proper convex.

Proof. —

(A) The fact that (i) implies (ii) is clear. The fact that (ii) implies (iii) results immediately from the equalityg=f ◦diag. Finally, suppose that (iii) holds. Theng=g, and Theorem 4.3(i) implies that

f=g◦µ=g◦µ=f, which shows thatf is closed proper convex.

(B) The fact that (i) implies (ii) is clear. The fact that (ii) implies (iii) re- sults immediately from the equalityg=f◦diag_N×n. Finally, suppose that (iii) holds. Theorem 4.3(ii) then implies that

f=g◦λ=g◦λ=f, which shows thatf is closed proper convex.

In the case of O(n)×O(n)-invariant functions, the analogous statement can be derived in several ways from the above results.

Corollary 4.6. — Letf:M_n(R)→(−∞,∞]beO(n)×O(n)-invariant, and let g:Rⁿ →(−∞,∞] be the unique Π(n)-invariant function such that f =g◦λ. Then the following are equivalent:

(i) f is closed proper convex;

(ii) the restriction off toDn(R)is closed proper convex;

(iii) g is closed proper convex.

Remark 4.7. — As a convex Π(n)-invariant function, the functiongap- pearing in Theorem 4.5(B) or in Corollary 4.6 must be such that each partial mapping

x_k→g(x₁, . . . , x_n), k= 1, . . . , n

is increasing onR+. As a matter of fact, for allx= (x₁, . . . , x_n)∈Rⁿ with x₁0,

g(0, x2, . . . , xn)1

2g(−x1, x2, . . . , xn) +1

2g(x1, x2, . . . , xn) =g(x), and ifz >0, we see, using the above inequality, that

g(x) x1

x₁+zg(x1+z, x2, . . . , xn) + z

x₁+zg(0, x2, . . . , xn)

x1

x₁+zg(x1+z, x2, . . . , x_n) + z

x₁+zg(x1+z, x2, . . . , x_n)

= g(x1+z, x2, . . . , xn).

Thusx₁→g(x₁, . . . , x_n) is increasing onR+, and the same reasoning holds for all other partial applications.

5. Concluding comments

The assumption of SO(N)×SO(n)-invariance enables to reduce substan- tially the dimension of the objects whose convexity is studied. This appears clearly in Theorem 4.5, where the dimension is reduced fromN nton.

It is worth noticing that the computation of the convex envelope of some SO(N)×SO(n)-invariant functionf also benefits from this dimension reduction, as one should expect.

Theorem 5.1. —

(i) Let f:M_n(R)→(−∞,∞] be SO(n)×SO(n)-invariant, and let g:=

f ◦diag. Let Cf and Cg denote the convex envelopes of f and g, respectively. Assume that the relationships Cf =f and Cg =g hold, which happens notably whenf andg are finite. Then

Cf =Cg◦µ.

(ii) Let N n, and let f:MN×n(R)→(−∞,∞] be O(N)×O(n)-inva- riant, and letg:=f◦diag_N×n. Assume again that the relationships Cf =f andCg=g hold. Then

Cf =Cg◦λ.

Proof. — Immediate from Theorem 4.3.

Another noteworthy dimension reduction occurs in the computation of the inf-convolution of two convex invariant functions. Iff1 andf2 are two extended real-valued functions onMN×n(R), their inf-convolution is defined by

(f₁f₂)(ξ) = inf

η∈M_N×n(R⁾

f₁(ξ−η) +f₂(η) .

Recall that, in essence, inf-convolution and addition are dual operations.

More precisely, if f1 andf2 are proper, then (f1f2)=f₁+f₂, and consequently the formula

f₁f₂= (f₁+f₂)

holds wheneverf₁f₂= (f₁f₂), that is, wheneverf₁f₂is closed proper convex. This duality, combined with Theorem 4.3, gives rise to the following result.

Theorem 5.2. —

(i) Fori = 1,2, let f_i:M_n(R)→(−∞,∞] be closed proper convex and SO(n)×SO(n)-invariant, and let g_i := f_i◦diag. If f₁ or f₂ is inf- compact, then

f1f2= (g1g2)◦µ. (5.1)

(ii) Let N n. For i = 1,2, let f_i = g_i ◦λ:M_N×n(R) → (−∞,∞] be closed proper convex and O(N)×O(n)-invariant, and let g_i :=

f_i◦diag_N×n. Iff₁ orf₂ is inf-compact, then f1f2= (g1g2)◦λ.

Proof. — We restrict attention to the first statement, the second one being analogous. Recall that, by definition,fi is inf-compact if

f_i(ξ)→ ∞ as ξ → ∞.

The relationships fi = gi ◦µ and gi = fi◦ diag imply that fi is inf- compact if and only if gi is inf-compact. Note that the Πe(n)-invariance of gi and g_i implies that domgi, domg_i, domfi and domf_i contain the origin. We may assume that gi ≡0, i= 1,2, for otherwise Equation (5.1) holds trivially. The Π_e(n)-invariance ofg_ithen implies that int domg_i and int domf_i contain the origin, and that g_i and f_i are continuous at the origin. By [8], Theorem 6.5.7, g₁g₂ andf₁f₂ are closed proper convex.

Theorem 4.3 then implies that

f1f2 = (f₁+f₂)

= (g₁◦µ+g₂◦µ)

= ((g₁+g₂)◦µ)

= (g₁+g₂)◦µ

= (g₁g₂)◦µ.

Acknowledgements. — This work was performed while the second author was visiting professor at EPFL and we thank P. Metzener for interesting discussions. Moreover, after presenting Theorem 3.3(i) with our proof to Denis Serre, he found a new proof of the result. He will include it in the electronic version of his book [16].

Bibliography

[1] J.M. Ball, Convexity conditions and existence theorems in nonlinear elasticity, Archives For Rational Mechanics and Analysis, 63, p. 337-403 (1977).

[2] B. Dacorogna, Direct Methods in the Calculus ofVariations, Springer-Verlag, 1989.

[3] B. Dacorogna, P. Marcellini, Implicit Partial Differential Equations, Birkh¨auser, 1999.

[4] B. Dacorogna, H. Koshigoe, On the different notions ofconvexity for rotation- ally invariant functions, Annales de la Facult´e des Sciences de Toulouse, II(2), p.

163-184 (1993).

[5] J.-B. Hiriart-Urruty, C. Lemar´echal, Convex Analysis and Minimization Al- gorithms, I and II, Springer-Verlag, 1993.

[6] R. A. Horn, C. A. Johnson, Matrix Analysis, Cambridge University Press, 1985.

[7] B. Kostant, On convexity, the Weyl group and the Iwasawa decomposition, An- nales Scientifiques de l’Ecole Normale Suprieure, 6, p. 413-455 (1973).

[8] P. J. Laurent, Approximation et Optimisation, Hermann, 1972.

[9] H. Le Dret, Sur les fonctions de matrices convexes et isotropes, Comptes Rendus de l’Acad´emie des Sciences, Paris, S´erie 1, Math´ematiques, 310, p. 617-620 (1990).

[10] A. Lewis, Group invariance and convex matrix analysis, SIAM Journal ofMatrix Analysis and Applications, 17, p. 927-949 (1996).

[11] A. Lewis, Convex analysis on Cartan subspaces, Nonlinear Analysis, 42, p. 813- 820 (2000).

[12] A. Lewis, The mathematics ofeigenvalue optimization, Mathematical Program- ming, Series B 97, p. 155-176 (2003).

[13] P. Rosakis, Characterization ofconvex isotropic functions, Journal ofElasticity, 49, p. 257-267 (1997).

[14] R.T. Rockafellar, Convex Analysis, Princeton University Press, Princeton, 1970.

[15] A. Seeger, Convex analysis ofspectrally defined matrix functions, SIAM Journal on Optimization, 7(3), p. 679-696 (1997).

[16] D. Serre, Matrices: Theory and Applications, Grad. Text in Math. 216, Springer- Verlag, 2002. See also http://www.umpa.ens-lyon.fr/∼serre/publi.html.

[17] F. Vincent, Une note sur les fonctions convexes invariantes, Annales de la Facult´e des Sciences de Toulouse, p. 357-363 (1997).