A NNALES DE L ’ INSTITUT F OURIER
A NH N GUYEN H UU
Classification of connected unimodular Lie groups with discrete series
Annales de l’institut Fourier, tome 30, no1 (1980), p. 159-192
<http://www.numdam.org/item?id=AIF_1980__30_1_159_0>
© Annales de l’institut Fourier, 1980, tous droits réservés.
L’accès aux archives de la revue « Annales de l’institut Fourier » (http://annalif.ujf-grenoble.fr/) implique l’accord avec les conditions gé- nérales d’utilisation (http://www.numdam.org/legal.php). Toute utilisa- tion commerciale ou impression systématique est constitutive d’une in- fraction pénale. Toute copie ou impression de ce fichier doit conte- nir la présente mention de copyright.
Article numérisé dans le cadre du programme Numérisation de documents anciens mathématiques
http://www.numdam.org/
CLASSIFICATION
OF CONNECTED UNIMODULAR LIE GROUPS WITH DISCRETE SERIES
by NGUYEN HUU ANH
Let G be a separable locally compact group with center Z.
An irreducible representation^) of G is said to be a member of the discrete series if it has some non zero matrix coefficient which is square integrable modulo Z . In this article we introduce the notion of H-groups and prove that the discrete series exist for these groups. Moreover, apart from some technical requirements, it will be proved that a connected unimodular Lie group G with center Z such that the center of G/Rad G is finite has discrete series if and only if G may be written as G == HS, H H S = Z° , where H is a H-group with center Z° and S is a connected reductive Lie group with discrete series such that Cent(S)/Z is compact.
In particular, if G is the semidirect product of a simply connected solvable Lie group and a connected semisimple Lie group with finite center, then the subgroup S may be chosen so that it is the direct product of a vector group T such that T/T Fl Z is compact and a connected semisimple Lie group with discrete series.
Thus in view of the results of Harish-Chandra on the discrete series of semisimple Lie groups (cf. [6]), the problem of classifying connected unimodular Lie groups with discrete series is completely solved.
Finally it should be noted that the results in this article contain our previous result on algebraic groups (cf. [2]) as a special case, for H will be proved to be nilpotent if G is locally algebraic (see Lemma 2.2).
(1) Warning: "representation" means "continuous unitary representation"
in a Hilbert Space.
Mackey's machinery on group extensions plays again a very important role. The strictly ergodic case may be avoided thanks to Theorem 1.1 which is interesting in its own right. The notion of H-groups is introduced in section 2. Theorem 2.9 is essential for carrying out the theory of Pfaffian polynomial exactly as in [15].
Theorem 2.12 contains the most important results in the theory of representations of H-groups. In section 3 we treat the most general case. Theorems 3.4 and 3.6 are the main results of the article.
Finally in section 4. we give another characterization of connected unimodular Lie groups with discrete series. In particular Theorem 4.4 gives the solution of a conjecture stated in [ 15].
In the following we continue to use the notations of [1] and [2]. In particular ^ , 5 , . . . are the Lie algebras of the Lie groups H , Z , . . . ; ( 0 / 5 ) * is identified with the annihilator of 5 in ^* , and similarly (H/ZV is identified with the annihilator of Z in H . The neutral component of a Lie group G is denoted by G° , and the simply connected groups are always supposed to be connected.
1. Some preliminary results,
Recall that if G is a separable locally compact group, Z a closed normal subgroup of G , \ a character of Z , then a repre- sentation TT of G is said to be a x-representation if 7r|Z ^ mult x - If moreover TT has a non zero matrix coefficient which is square integrable modulo Z with respect to the right Haar measure of G/Z, then we say that TT is square integrable mod Z . In this case we have Schur's orthogonality relation and can define the formal degree dW of TT (cf. [ I ] , [9], [13]).
THEOREM 1 . 1 . — Let G be a separable locally compact uni- modular group. Let Z C K be closed normal abelian subgroups of G. Let x be a character of Z , and TT be a square integrable mod Z irreducible -^-representation of G. Then the quasi orbit of G/K in K defined by it \ K is transitive.
Proof. — Since K is smooth and of type I, we may assume that the system of imprimitivity for TT is the canonical one associated
to a measure ^ belonging to the quasi orbit defined by 7r|K (cf.
[8], Theorems 7.6 and 5.6). In particular the projection valued measure P belonging to TT|K is just the multiplication by charac- teristic functions on K . Moreover there is a measurable cocycle c on G x K taking values in the unitary group of a Hilbert space 9€ such that for all g in G and all functions / in L^K, d^) ® 9€
we have:
^(8)fW=c(g,\)p(g,\)l/2f(\g) j^-a.e.X (1.1) where p is a Borel function such that for each g , p ( g , . ) is the Radon-Nikodym derivative of the translate ^ of jix with respect to JLI.
Now the action of K on K is trivial. Therefore it follows from (1.1) that for all k in K and for all / we have:
^W/(X)=cO,X)/(X) a.e.X
this together with the fact that P is precisely the projection-valued measure associated to the direct integral decomposition of TT | K into the characters X show that c ( k , \) = \(k) ld^ for all A: in K , and for almost all X in K .
On the other hand by the same argument as in [ 1 ] we see that p. -viewed as a measure on the G-invariant subset Xo(K/Z)' - is absolutely continuous with respect to the translate of the Haar measure d\ of (K/Z)' by \, where \ is some fixed extension of x to K . Furthermore d\ is relatively invariant in the sense of [14]. Hence we may replace jn by the restriction to its support of the translate of d\ by \. This being done, we have
(Tr(kg)f,, ^) = F p(g , X)1/2 (c(k , X) c(g , X) f,(\g) , f^\)) rf^(X)
=W) I \(k)F (\) d\ (1.2)K
"(K/zr
where F^(X) = p^X^X)1/2 (c(g , \\) f,(\\g) , f,(\\)) if XQ X G supp fi and is 0 otherwise.
Note that Fg is integrable with respect to r f X . Therefore, if we denote by F^(^) its Fourier transform, then
f^ lOrQrtA^)!2^ f^ f^\Wg)f,,f^dkdg
== f f i^W^dkdg (1.3)
^G/K ^ K / Z ° '
where dg and dg are Haar measures of G/Z and G/K respec- tively.
Now it follows from (1.3) that F^(-) is square integrable for almost all g in G. Therefore, if the Haar measure of K/Z is pro- perly normalized, then we have by the Plancherel formula on K/Z:
f Wg)f,.f^dg = f f |F,(X)|2 d\ dg (i 4)
G/Z "G/K ^(K/Z)* ° \l••rf
= [ C p(g , X) I (c(g, \)ft(\g), /2(\)) |2 dju(X) dg.
"G/K " K
On the other hand by Schur's orthogonality relation:
fo/z ^8)fz,Wdg = dW-1 ||/,||2 H/JI2 . (1.5) Let us apply (1.4) and (1.5) for /i = y ® u, /, = ^ ® v , where
<p and )A belong to L2 (K, cf^Lt), and u, v are in 9€ such that H M | I = l l v l l = 1. We have:
f \^Wd{i(\) f p(g,\)\y(W \(c(g,\)u,v)\2 dg
K v G/K
=d(ir)-1 11^ II2 11^ I I2. In particular, we have, for /i almost all X in K:
f p(g,\)W\g)\2 [(c(g,\)u,v)\^d?=d(^^)-l ||^||2.
VJ/i\.
Substituting successively for (^ in the above equality the charac- teristic functions of a countable separating family of Borel subsets of K , we see that for all u, v belonging to a fixed orthonormal basis (B of 9€ and for almost all X in K:
f P(g^) ^) \(c(g, \)u, v)?dg= rf(7r)-1 r ^) dfJL(v) (1.6)
VJ/—- i/1^
where ^ is an arbitrary non negative measurable function on K . Assume that ^ is not concentrated in the orbit XG, where X satisfies (1.6). Then the characteristic function of XG is JLI- negligible, and hence we have according to (1.6):
P(g^)\(c(g,\)u,v)\2 = 0
for all u, v in (J3 and for almost all g in G. This is a contra- diction. QED
Remark. - Apparently the only assumption needed on TT is that all of its matrix coefficients are square integrable mod Z and satisfy Schur's orthogonality relation. However, as proved in [3], this condition is equivalent to the fact that TT is a square integrable mod Z irreducible representation.
LEMMA 1.2. - Let p be a representation of a connected Lie group G in a finite-dimensional real vector space V . Let \ be a
\-valued cocycle of G with respect to p. Assume that there is Co€Cent(G) such that p(co) = a(Co). Idy , a(Co) ^ 1. Then X is a coboundary. In fact we have:
X ( i r ) = ( p Q r ) - I d v ) x(co) ^ • a(Co) - 1
Proof. - We have:
W + P(g) ^(Co) = X(^o) = \(cQg) = X(co) 4- p(Co)X(^).
Therefore
XO?)=(pQr)-ldv)
x(co).
a(Co) - 1 QED PROPOSITION 1.3. -Let G , p , V , X be as above. Assume also that G is reductive and p is irreducible. Then either ( i ) p is trivial and X(G) = V , dim V = 1, or(ii) \ is a coboundary.
Proof. - If X = 0 , then it is a coboundary and (i) does not hold. Thus suppose that X ^ 0. First assume that p is trivial so that dim V = 1. Then X(G) is plainly a linear subspace of V because G is connected and hence X(G) = V . Note that X is not a coboundary in this case.
Now assume that p is non trivial. Put N = ker p . Then X is a homomorphism from N into the abelian group V . Therefore if X(N) =^ 0, we may find an element n^ in N H Cent(G) such that X ( ^ ) ^ 0 . We have
^(8) + P(^)X(^) = X(^o) = X(^) = X(^) + \(g) i-e. P ( ^ ) X ( ^ ) = X ( ^ ) V ^ G G .
This however contradicts the fact that p is a non trivial irre- ducible representation of G. Therefore \(N) = 0 , and X may be viewed as a cocycle with respect to the representation of G/N induced from p, i.e. we may assume that p is faithful. Recall that p is irreducible so that p(G) is the neutral component of a real algebraic subgroup of GL(V). By Lemma 1.2 it is sufficient to consider the case when this algebraic group contains no R-split torus in its center, i.e. when the center of p(G) is compact. Since every finite dimensional representation of such a group is completely reducible, X is a coboundary (cf. [12], Theoreme 2, expose 4).
QED
2. Representation theory of H-groups.
DEFINITION 2.1. - Let H be a connected Lie group, and Z a central subgroup of H. Then H is said to be a H-group if there exists a linear form 9. in &* such that the orbit of 9. in f)* via the coadjoint representation of H is the hyperplane £ + ( 0 / d ) * - Remark. — It follows immediately from the above definition that the symplectic form associated to S.: Bg({ , 17) = £([{;, T?]) is non degenerate on t ) / ^ , i.e. ^ is the Lie algebra of the centralizer Gg of £ in G. Since the orbit £ + (Wa)* is simply connected we must have Z = Gg = Cent(H). In particular Z is connected.
Moreover H is simply connected if and only if Z is simply connected.
LEMMA 2.2. - Let H be a H'group mth center Z. Then H is solvable. If we assume further that H is locally algebraic, then H is nilpotent.
Proof. — Let ^ be a maximal semisimple subalgebra of ^ so that ^ contains an ^-invariant subspace § complementing 3 . Let
£ E f ) * be such that the H-orbit of H in O* is £ 4- ( W a ) * . Then there exists £^ in this orbit such that £^ = 0 on ^ . Since the symplectic form B^ is also non degenerate on ^/^ and ^ is orthogonal to f) with respect to Bg^ we see that ^ = 0 , i.e. f) is solvable. In case H is locally algebraic we may take ^ to be
the reductive subalgebra in a fixed Levi's decomposition of I; . Then the above argument shows that § = 0, i.e. 1) is nilpotent. QED LEMMA 2.3. - Let G be a Lie group, Z a central subgroup of G. Assume that G has a normal subgroup H which is a H- group with center Z . Then there exists a closed subgroup S such that G = H S , H H S = = Z . (2.1) Moreover S may be chosen so that S° = S H G° .
Proof. - Let £ E § * be such that the H-orbit of £ in I;* is precisely S. + (^3)* . Put S = {x G G|£ o Ad^(x) = £} . Then S clearly satisfies (2.1)., Note that S 0 G° is the centralizer of S. in G° and hence SO G° is connected since the G°-orbit £ + 0/3)*
is simply connected. Hence S H G° = S° . QED LEMMA 2.4. - Let G , H , S , Z , C be as above. Assume also that G is connected and that there is an extension Jt^ of C to Q such that the S-orbit of J^| x> is C J ^ + ( ^ / a ) * . Then the G-orbit of S.^ in g* is C^ + ( g / a ) * . /^ particular G is a H-group with center Z .
Proof. - Let J^ E g* be such that £3 1 5 = CJ 5 . We have to prove that i^ belongs to the same G-orbit as J^ . By the assump- tion, there exist x^ in H and x^ in S such that
(^ o A d ^ ) i ^ = C o Adjc^ = CJI) and (J^o A d x ^ ) | 5 = (^ o A d ; ^ ) l & .
Then we have J^ = j^ o Ad(x^). QED
Example 2.5. - In view of the results in [10] and [15] we see that the class of simply connected H-groups contains that of uni- modular exponential groups with discrete series and hence that of simply connected nilpotent Lie groups with discrete series. On the other hand using Lemma 2.4 we may construct non-type I H-groups.
In particular these groups are not of exponential type. Let indeed I) be the five dimensional Heisenberg algebra with basis ^ , ^., -^
O ' = l , 2 ) such that the only non zero brackets satisfied by the
members of this basis are:
[ ^ , ] = ^ , f = 1 , 2 .
Let ^ be the three dimensional Heisenberg algebra with basis
?2 » ^3» ^3 such that [^3 ^3] = ?2 • Let ? be the semi direct product of f) and ^ such that ^ and $3 commute with the ele- ments of I) and:
h3^J = = ^ 1 . [ ^ 3 ^ l ] = - f l , [ ^ 3 ^ 2 ] =0^ ^ 3 ^ 2 ] = - - ^
hs^J-o
where 0 is a fixed irrational number. Let g be the quotient algebra of ^ by the ideal R ( ^ - ^ ) . Let G be the simply connected Lie group with Lie algebra Q . Let H , S be the analytic subgroups of G corresponding to ^ , § respectively, where f) , § are identified with Lie subalgebras of g so that ^ and ^ are identified with the same element ? in the center 3 of g . We have G = HS, H 0 S = Z, where Z is the analytic subgroup of G corresponding to 5 . Thus G is a H-group by Lemma 2.4. However G is not of type I since it is the central extension of the group G/Z which is isomorphic to the direct product of R and the five dimensional Mautner's group.
LEMMA 2.6. - Let f) be a Lie algebra and ^ an ideal of 1) . Assume that there exists £ in !)* such that the symplectic form Bg is non degenerate on I ) / 3 . Let T be an automorphism of f) leaving 3 invariant such that T splits over R , and that ^ has a r-invariant subspace ^ complementing 3 such that C ( " ^ ) = 0 . Finally assume that T\ 3 has at most two eigenvalues: a and — a . Then [det r| = |a|'7, where q = — dim l)/^ + dim 3 .
Proof. — Let {^.} be a basis of ^ consisting of eigenvectors of T corresponding to the eigenvalues {c^}. We have:
r[^, ^ ] = [T$, , T^.] = a, a, [^, ^.].
Hence |aJ |a,| = |a| if [S,,^i.
It follows that | a, | |c^| = |a| if Bg(^, $,) ^ 0 i.e. |r| Bg = |a| Bg, where | T| is the linear transformation of I) such that | r\ ^/ ^ |aJ ^., V / , and the action of |r| on Bg as a symplectic form on t) is
induced from that of |r| on ~t). Thus
d e t | r | = p f ( | r | B g ) ( P f ( B e ) = lal^1"1 l ) / )
where Pf is the Pfaffian of Bg with respect to a fixed volume on F) . Finally
I d e t r ] = d e t | r | |det(r| 3)! = |a|9 . p^p LEMMA 2.7. - Z^r I) , 3 , r &6? a5 a6oi^. Assume that there exists a ^ ± 1 ^cA rAar spec(r| 3) C { 1 , - 1, a, - a} and s p e c ( T | 3 ) n { a , - a } = ^ 0 . Then | d e t r | ^ l .
Proof. - Let |r| be as above. It is sufficient to prove that det |r | is not equal to any negative integral power of |a| . Let IT-F = ^l o g l T I be the one parameter subgroup of GL(^) generated by log | T |; then as in the proof of Lema 2.6 we have
l ^ r 1 0 , 1 ^ = 1 0 1 ^ if B e 0 , ^ , ) ^ 0 where /:„. = 0 or 1 .
Thus det I T ^ = Pfdrl^B^/PfCBg) is a polynomial in [ a ^ . Therefore by the unique factorization in the polynomial ring R[X] , det \r\t is not equal to any negative integral power of \0(.\t and hence det |r| is not equal to any negative integral power of |a| . QED PROPOSITION 2.8. - Let H be a connected Lie group. Let 3 be an ideal in f) . Assume that there exists C in ^* such that the H-crbit of K in ((* is £ + ( ^ 3 ) * . Let T be an automorphism of 0 leaving 3 invariant.
i)If specCr^)61 C {X € C : | X | = 1}, where (r^ is the complex if ication of (r| 3 ) , then \ det T| = 1 .
ii) // s p e c M a V C { X E C : | X [ = 1} U [a,-a} and specCThy H {a,-a} ^ 0 , where a G R \ { 1 ,- 1}, then Idet T | ^ 1 .
Proof. - Let T be the neutral component of the smallest real algebraic subgroup of Aut(^) containing r . Then T is the direct product of a unipotent real algebraic group U and the neutral
component S of a real algebraic toral subgroup of Aut(^) (cf. [5]).
Moreover S is the almost direct product of a connected R-split subgroup S^ and a compact connected subgroup S^ (cf. [11]).
Since the elements of U and S^ clearly have determinant 1 , and all of their eigenvalues corresponding to eigen vectors in ^c have absolute values 1 , we may assume that r belongs to S ^ . In other words we may assume that r is split over R . It follows then that b has a r-invariant subspace f) complementing 3 . Since the H- orbit of £ is £ + ( l ) / ^ ) * we may assume that C ( ^ ) = 0. Now the statements i) and ii) follow immediately from Lemmas 2.6 and 2.7, for the symplectic form Bg is non degenerate on ^ .
QED Now let us consider another characterization of H-groups which is more convenient for later applications since it only involves the Lie algebras in question.
THEOREM 2.9. — Let H be a connected solvable Lie group, and Z a closed connected central subgroup of H. For every S.
in f) * , the following statements are equivalent:
i) The H-orbit of S. in ()* is C 4- ( f ) / ^ )*
ii) H is unimodular and the associated symplectic form Bg is non degenerate on f) / 3 .
Proof. — We proceed by induction on dim H . First note that if i) or ii) hold then Z is the neutral component of the center of H . Hence we will assume that it is so in the following. Moreover by factoring out e x p ( k e r £ | 5 ) if necessary, we may assume that dim Z = 1 and C is non zero on 3 . Let N be the nilradical of H . a) First assume that N is isomorphic to a Heisenberg group with center Z . Since C l a = ^ 0 , the N-orbit of £ | n is £|n + ( n / g ) * (cf. [15]). Hence by Lemma 2.3, H = N S , N H S = Z , where S = [x G H : ( £ o A d x ) |n = £ | n } . Moreover by Proposition 2.8:
[ d e t A d (x)| = 1, \fx € S. On the other hand since N is unimo- dular and S/Z ^ H/N is abelian we see that H is already unimodular in this case. Thus we have proved i) ==> ii). Now assume that Bg is non degenerate on I ) / 3 . Then it is obvious that Bg^ is non degenerate on ^ / 3 , i.e. s is a Heisenberg algebra with center 3 . Therefore by Lemma 2.4 the H-orbit of £ in ^* is £ 4- ( f ) / a ) * .
<3) Assume that the center of N is still equal to Z but N is not isomorphic to any Heisenberg group. Then as in the proof of Proposition 4.2 of [1] there is a closed connected normal abelian subgroup K of H such that K/Z is contained in the center of N/Z and the N-orbit of C | f is C | f 4- ( f / 3-)*. Put
Ho == { x e H : ( C o A d x ) | f = £ | f } .
Then H^ is connected. Moreover since f)o is the annihilator of f with respect to E g , ! ) / ^ may be identified with the dual f / 3 via Bg if the latter is assumed to be non degenerate on f) / 3 . For all x in HQ , { in f and 77 in ^ , we have
Bg(Ad(xK,Ad(^)7?) = C o A d O c ) ([$, 77])
=C([^])
= B e O , 7 ? ) .
In particular the representation of Hg in I) /f)o induced by the adjoint representation is contragredient to that induced in f / 3 . This being set we have
det Ad x = det Ad, , ( x )
9/8
" ^ ^ / D o0 0 ^ ^ o / t W detAd^(x).
Hence
det Adjc = det Ad^ ^ (;c) V;c G H^ . (2.3) Thus let us assume i). Then it follows that the Ho-orbit of £ | ^ o in ^ is £ | f ) o + (^o/O*- Hence HQ/K is unimodular by the induc- tion hypothesis (strictly speaking we should factor out e x p k e r ( 6 | f ) before applying the induction hypothesis, and then go back to Hg via the canonical homomorphism H^ ——> H^/exp ker(£ | f ) . However this kind of argument is trivial and will be omitted in the following).
Now it follows from (2.3) that: | det Ad x \ = 1 \fx e Ho .
Moreover it is clear that H = NH() and det Ad x = 1 , Vx G N . Therefore H is unimodular.
Conversely assume ii). Then it follows from (2.3) that Ho/K is unimodular. Moreover Bg^ is plainly non degenerate on ^ o / f . Therefore the H^-orbit of £|?o in ^ is C | ^ o + (^o/O* accord- ing to the induction hypothesis. Let &i G ^ * be such that £J 5 = C 15 .
We have to prove that J^ belongs to the same H-orbit as S.. Since the H-orbit of £| f in f * is C| f 4- ( f / g )* we may assume that
£J f == £| f without loss of generality. Then there exists x^ in Ho such that C J b o == (S ° Ad^)|()o . Finally since f / ^ is con- tained in the center of n/^ and since ^ / ^ o ^ n / n H ^ o may be identified with the dual of f / g via Bg we see that
{ £ o A d x / x E K } = J Z + ( ^ / ^ ) * .
Hence there exists x^ in K such that J^ o A d ( x ^1) == £ o Ad(x^), i.e. Ci = £ o AdOcs^i).
7) Finally assume that Z is properly contained in the center of N . Then (Cent N)/Z must be a vector group. Let us choose K to be minimal among the connected normal subgroups of H lying in the center of N and properly containing Z so that the adjoint representation induces an irreducible representation p of H in f / 5 . Note that K is closed in N since N/Z is simply connected. Now ^(x) = (C o Adx) | f — £| f is a cocycle of H with respect to the contragredient representation p of p . Since both p and X are trivial on N , X may be viewed as a cocycle of the abelian group H/N, and hence by Proposition 1.3 X must be of the form \(x) == p(Jc)J?o — S.Q for some ^ ^ ( f / ^ ) * unless P is the identity representation. In the latter case dim( f / 3) = 1 and \(H) == ( f / 3 ) * , i.e. C o Ad^ (H) = £| f + ( f / . ^ )* . In any case put
Ho = { ^ E H : ( £ o A d J c ) | f = £ | f } and K = exp ker(£ | f ) .
Then HQ is a closed normal subgroup of H and Z = K/K is a closed connected central subgroup of Ho = H^ /K .
Let us assume i). Then the H-orbit of £| f in f * is C I f •+• ( f / 3 )*. In this case p and hence p must be the identity representation of H. Note that H^ = H^. Now it is easy to see that the H^-orbit of £|^ is C It^ + ( ^ o / f ) * . Moreover each A d ^ O c ) , x G H , is unipotent because p is trivial. Therefore we have, according to Proposition 2.8
IdetAd,, Oc)| == 1, V x e H i.e. H is unimodular since H/HQ is abelian.
Conversely assume that ii) holds. Then the H-orbit of £|I) in f * is clearly open in C| f + ( f / ^ ) * . Thus in case p is not the identity representation we have p(H) = R"*^ or p(H) = R"^ x T . Hence there exists XQ G H such that ( A d ^ o ) | f / 3 = a Id , a =^ ± 1 , a € R . On the other hand p(Ho) and hence p(Ho) reduces to the identity. Therefore
detAd^Qc) = 1 , V x E H o i.e. detAd^x) = 1 , V x G H o .
This together with the fact that H^ is a closed normal subgroup of the unimodular group H implies that H°/K is unimodular, i.e.
HO = H^/K is unimodular; Since the form Bg^ is plainly non degenerate on ^ o / f , it follows from the induction hypothesis that the H^-orbit of 6|^ is C l ^ o + (WO* . Recall that Ad^) has exactly two eigenvalues: 1 and a. Therefore Proposition 2.8 implies: |det Ad(, .(^o)! ^ 1' Le- | d e t A d ( X o ) | =^ 1 since det A d ^ 0 ; o ) == 1. This however contradicts the fact that H is unimodular. Thus p and hence p is trivial and the H-orbit of
£ | f is £ | f + ( f / a ) * . Note that H^ = H() in this case. Now by the same argument as above we see that the H^-orbit of £ [ ^ 0 !s C l & o + ( ^ /f) * and the K-orbit of £ in f * is C + ( W ^ ) * - These two facts together imply that the H-orbit of C in 0* is C + (W 3 )* as in the last part of case j3). OED Remark. — The notations being as above, assume that dim Z = 1. It follows immediately from the proof of Theorem 2.9 that if the nilradical N of H is not isomorphic to a Heisenberg group with center Z , then there is a closed normal abelian sub- group K of H lying in N such that K/Z is contained in the center of N/Z and the H-orbit of £ | f is £ | f + ( f / ^ )* . Moreover there is a linear subspace V of ^ dual to f / g with respect to Bg such that (x , $)»—> x exp { is a diffeomorphism of H^ x V onto H . Assume indeed Cent(N) = Z . Then H = NHo , and the N-orbit of £ | f is C | f 4- ( f / g ) * . In this case V may be chosen to be a subspace of n complementing n H ^ . On the other hand if Cent(N)=^Z then d i m f / 3 = l and Ad(H) reduces to the identity on f / g and 5 respectively. In this case f)o is an ideal of codimension 1 and V can be any one-dimensional subspace complementing ^ o .
COROLLARY 2.10. - Let G be a Lie group. H a closed normal subgroup of G which is a H-group with center Z, and S a Lie subgroup of G such that G = H S , H n s = Z . Assume that for each x in S, the eigenvalues of Adx corresponding to eigen- vectors in ^ have absolute values 1 . Then G is unimodular if and only if S is unimodular.
Proof. - It follows from Theorem 2.9 and Proposition 2.8 that:
|detAd,,(;c)| == 1 V ; c € G .
Moreover the last assumption implies that G (resp. S) is unimodular iff G/Z (resp. S/Z) is unimodular. Finally G/Z is plainly the semi direct product of H/Z and S/Z. Hence the Corollary follows from the standard results in the integration theory of Lie groups, opp.
Example 2.11. -To see the necessity of the assumption that H is unimodular in Theorem 2.9, let us take H = R"^ x R2 with multiplication:
( x , , y ^ z ^ ) ( x ^ , y ^ , z ^ ) = (x^x^,x^y^ + ^,
O ^ - D y , + ^ + ^ ) .
Then H is a three-dimensional solvable Lie group with center Z = { ( l , 0 , z ) : z e R } . Note that H is not unimodular. In fact the modular function of H is A(x , y , z) = x . Now' let { $ , 7 7 , ? } be the canonical basis of S = R3 so that {?} is a basis of 3 Let £€!)* be such that £($) = £(77) = 0 and £ ( ? ) = ! . Then we have £([$, 7?]) = £(r? + ?) = 1, i.e. B^ is non degenerate on W a . However H is not a H-group as indicated by Theorem 2.9.
Of course one may prove this directly by observing that the projec- tion on f * of the H-orbit of S. in ^ consists of those linear forms £1 on f such that C,(?) = 1 and C . ( 7 ? ) > - 1 where
t = R T ? + R ? .
We turn now to the representation theory of H-groups. It turns out that it is very much similar to that of nilpotent Lie groups with discrete series in [15]. First note that thanks to Theorem 2.9 we may define the Pfaffian polynomial on 3* exactly as in [15].
Let indeed P(t) denotes the Pfaffian of the two-form Bg asso- ciated to » e t i * with respect to some fixed volume element da
on 0/3 . More explicitely we have A^Bg = P(C)da, where n = — dim l)/3 and A^Bg is the n-th exterior power of Bg . Now we may prove as in [15] that P(£) depends only on the restriction of S. on 5 and hence P(£) may be viewed as a polynomial function on 5 * . The only facts needed are that H is unimodular and that the two statements i) and ii) in Theorem 2.9 are equivalent. This being set let % = % H c 3* be the complement of the nullset of P(6) in
3*. We have:
THEOREM 2.12. — Let H be a simply connected H-group with center Z. For every fi in % , put Xe = exp y^T S.. 77z^ H has a unique (up to a unitary equivalence) irreducible ^^-representation T6. Moreover every ^^-representation of H is equivalent to a multiple of T8. In particular ind^^Xo ^ mult T8 and hence T12
is square integrable mod Z .
Pwo/ - First note that by Proposition 1.2 of [1] (see also [10]), if ind^iH Xfi ^ mult T2 then T6 is square integrable mod Z. Hence it remains to prove the existence of an irreducible Xg-representation of H for each C G % such that every Xg-^P^sentation TT is equi- valent to a multiple of T6 . Furthermore by using the central de- composition of TT we may assume that TT is a factor. Let us proceed by induction on d i m H . As usual we suppose dim Z = 1 and £ is non zero on 5 .
a) First assume that the nilradical N of H is isomorphic to a Heisenberg group with center Z . Then it follows from the proof of Theorem 2.9 case a) that H = N S , N H S = Z where S is a Heisenberg group with center Z . Consider the semi direct product G = N (s) S . Let v : G —> H be the canonical homomorphism so that v(n, s) = ns. Let T^ and T^ be the unique members of the discrete series of N and S respectively such that T ^ I Z ^ m u l t X g and T ^ I Z ^ m u l t X g (cf. [1]). Note that S is defined to be the set of those elements of H leaving fixed an arbitrary but fixed exten- sion £1 of C to n , or equivalently the set of those elements leaving invariant a sub space TT of n complementing 3 . Thus the function Jg in [1] which is defined and continuous on S p ( n , B e ) has continuous square roots on S since S is simply connected. It follows then from Lemma 3.1 of [1] that T^ may be extended to
an irreducible representation of G denoted again by T^ . Moreover TT o v(n, s) = T^n , s) ® a(s) \f(n , 5) € G
where a is some factor representation of S such that a | Z ^ m u l t X g . But then a ^ mult T^ and hence
TT o ^ ^ mult T6 o i/ (2.4) where T2 is the irreducible Xg-representation of H defined by the irreducible representation T^, s) ® T^(^) of G by factoring out ker v . The factorization is possible since:
T f ( z , z -1) ® T^-1) = T f ( z ) ® T^(z-1) == XgO) X^-1) Id = Id i.e. T f ( w , 5) ® T^(^) is trivial on ker v . Now TT is equivalent to a multiple of T8 as indicated by (2.4).
j3) Let us consider now the case in which N is not isomorphic to a Heisenberg group with center Z . By the remark of Theorem 2.9 there is a closed connected normal abelian subgroup K of H lying in N such that the H-orbit of an arbitrary extension £^ of £ to f is EI + ( f / a ) * , i.e. the H-orbit of X^ = exp y^TCi in K is \^(K/Zy . This implies in particular that the quasi orbit defined by TT|K is concentrated in this orbit. Therefore according to Theorem 8.1 of [8], TT is induced from a factor representation a of the subgroup
Ho = {jceH^Ocfoc-1)^^) V f c G K }
= { x G H . - C i o Ad^(x) == £1}.
Note that Ho/exp ker £^ is a H-group as indicated by the proof of Theorem 2.9. Therefore it follows from the induction hypothesis that a is equivalent to a multiple of the unique irreducible repre- sentation T^ of Ho such that T^IK 9= mult \i . Finally this representation together with the orbit X^K/Z)' determine the unique irreducible representation T8 of H such that T ^ Z ^ m u l t X g - More explicity T6 ^ ind^ ^ T61 . Hence
TT ^ ind^ -i-H (mu^ T 1) ^ mult T^ . QED PROPOSITION 2.13. - Let H , Z , £ be as above. Put dim H/Z = 2p.
Assume that the Hoar measure dh of H/Z is defined by a positive invariant Ip'form (^ such that its value at the identity in H/Z, (a?)i
is the volume element on 1) / ^ used in the definition of P(£). Then the formal degree of the irreducible representation T6 is given by d(rTfl)= |P(£)|.
Proof — The notations being as in Theorem 2.12, let us assume first that case a) in the proof of that Theorem holds so that
^ov(n,s)==^(n,s)^T:^s).
By Theorem 4 of [15] the Haar measures dn and ds of N/Z and S/Z respectively may be normalized so that
d ( T ? ) = |P,(£)| ^ = 1 , 2
where Pi(C) and P^C) are the Pfaffian polynomial functions on 5 * defined by the structures of the Heisenberg groups N and S respectively. Note that H/Z is the semi direct product of N/Z and S/Z. This allows us to normalize dh so that dh==dnds. This being done we have ^(T8) = d(T^) d(T^). Let a , j 8 denote the invariant forms on N/Z, S/Z defining the Haar measures dn, ds respectively. Put dim N/Z = 2q , dim S/Z = 2r. We_have p = q + r.
Let £1 be the extension of 9. to n so that C i ( n ) == 0 as in the proof of Theorem 2.12 case a). Let J^ ^e an arbitrary extension of £ to ^ , and £3 the unique extension of £ to ^ so that it coincides with C^ and £^ on n and ^ respectively. Since n is clearly orthogonal to ^ with respect to Bg we have
A^ = ( A ^ B ^ A C A ' B ^ ) i.e. P(£) (o;)i = Pi(£) P2(C) (a\ A (^
i.e. P ( £ ) = P ^ ( C ) P 2 ( C ) . Hence d(^) = |P(£)|.
Now assume that case j3) in the proof of Theorem 2.12 holds so that T6 = ind^ m T2 1. Hence it follows from Theorem 1.3 of [1] that: ° _
dCr^riCr81). (2.5)
Recall that the Haar measure dh^ of H()/K is so normalized that _
dh = dh^dh dk (2.6) where dk is a fixed Haar measure of K/Z and dh is the quasi
invariant measure on HQ\H which is transported from the trans- late ri(X^X) of the Haar measure dX of (K/Zy . Here we have normalized the Haar measure d\ in such a way that the Plancherel formula on (K/Z), (K/ZY holds for the pair of measures dk, d\.
Let o?o, 0 be the invariant forms defining dh^, dk respectively.
Let V be the subspace appeared in the remark of Theorem 2.9.
Recall that V is dual to f / g via Bg- where £ is a fixed extension of £i to f) . Let us normalize the Lebesgue measure d^ of V in such a way that the Plancherel formula on f / g , V holds for the pair of measures (0)^ , d^ . This being set, we have according to (2.6):
(o;)i =(ojo)i A ^ A ( @ ) , . (2.7)
On the other hand since J& o is orthogonal to f with respect to B-g- we have
A^ = ± (A^,^) Ad{ A (6\ (2.8)
where dim H/Z = 2p , dim Ho/K = 2<7 . Now it follows from (2.7) and (2.8) that
P(fi)=±P(£i). (2.9)
Finally we have by the induction hypothesis that d(T 1) = |P(£i)|.
This together with (2.5) and (2.9) show that d^) = |P(£)|.
QED Remark. — The proof of Proposition 2.13 is somewhat more complicated than that of Theorem 4 of [15] since we have to treat separately case a). On the other hand it is shorter than the latter because we do not have to use the Plancherel formula on K/Z which has already been used in the proof of Theorem 1.3 of [ 1 ].
PROPOSITION 2.14. - The map 0 : £ i—^ T6 is a bijection of
% onto the set of (equivalent classes of) square integrable mod Z irreducible representations of H. Moreover this map is a homeo- morphism from the natural topology of % to the Fell topology on representations.
Proof. — The proof is essentially the same as that of Theorem 2 of [15]. The only thing remaining to be checked is the fact that <f>
is surjective. Thus let TT be a square integrable mod Z irreducible
Xg-representation of H . Then by Theorem 3.4 which will be proved in the next section independently of Proposition 2.14, H may be written as H = H ^ S , H^ H S = Z , where H^ is a H-group with center Z , S is a reductive subgroup such that Cent(S)/Z is compact.
Moreover there exists an extension C^ of C to 01 such that the H^-orbit of C^ in ^* is ^ + ( ^ 1 / 3 ) * . Since H is solvable, S is in fact abelian. Thus S/Z is a simply connected compact abelian Lie group and hence S/Z reduces to the identity, i.e. H = H^ .
Hence £ G % . QED
Remark. - In Theorem 2.12, Corollary 2.13 and Proposition 2.14 we have supposed that H is simply connected. In the case H is not simply connected, then Z is identified via the map 9. '—> exp \/— 1 £ with an additive subgroup of 3 * of the form V x L, where V is a vector subgroup of 5 * and L is a discrete finitely generated free abelian subgroup of 3 * such that dim 3 * = dim V + rank L. It is easy to see that the polynomial function P(£) is not identically zero on V x L . Hence the intersection of % = % H wlt!1 V ^ L is a non empty open set. Therefore the above results still hold for a non simply connected H-group H, where the set % is to be replaced by the intersection of itself with V x L .
3. The general case.
In this section we will consider the case of a general connected unimodular Lie group. First of all, for technical reasons, we have to consider non connected groups:
PROPOSITION 3.1. - Let G be a unimodular Lie group with radical R. Assume that G is contained as a closed normal subgroup in a connected Lie group F such that: ( i ) F/G° is abelian, (ii) G contains a closed connected normal nilpotent subgroup N of F such that F/N is a reductive Lie group, ( H i ) F/Rad F has finite center.
Let Z be a closed central subgroup of F lying in G such that Z H N = Z° and N/Z° is simply connected. Finally let x be a character of Z . Put 9. = — \/— 1 d\.
Under these conditions, if G has a square integrable mod Z irreducible x-representation TT, then there exist a closed connected normal subgroup H of F such that N C H C R , and a linear form S.^ on I) extending K such that the H-orbit of S.^ in O*
is GI + ( W ^ ) * - ^n particular H is a H-group with center Z°.
Proof. — W e will proceed by induction on d i m G . First of all by restricting F a little we may assume that F/G° is generated by the one-parameter subgroups passing by the elements of G/G°.
This being done F is clearly unimodular. On the other hand, by replacing Z by some subgroup of finite index if necessary we may assume that Z is the direct product of Z° and a discrete finitely generated free abelian group D C Rad F. As usual we suppose that d i m Z < 1, and £ ^ 0 if 3 = ^ = 0 .
a) Assume that Cent(N) = Z°. If N is isomorphic to some Heisenberg group we may take H = N . Otherwise let K^ be a closed connected normal subgroup of F such that K^/Z° is the center of N/Z°. In virtue of Proposition 2.3 of [1] the center K of K^ contains Z° strictly. Of course K is a closed connected normal abelian subgroup of F. Note that K ^ D and hence KD are closed subgroups of G since they are inverse images via the canonical projection G —^ G/D of closed subgroups of ND/D ^ N which are defined in a similar manner. Note also that the expo- nential map is a diffeomorphism of f / 5 onto K/Z° since the latter is a simply connected abelian group. Now we may prove exactly as in Proposition 4.2 of [1] that the N-orbit of an arbitrary ex- tension J of £ to f is of the form £"+ ( f / ^ ) * , i.e. the N-orbit and hence the G-orbit of x" in (KD)' is x(KD/Z°D)' where _x is the unique character of KD extending x such that d\= ^/— 1 £.
Thus TT is induced from a square integrable mod KD irreducible X-representation a of the subgroup Go = G H Fo where
Fo == { x € = F : x ? ~l) = X W , W G K D }
= { j c e F : C o A d ^ Q c ) == C} . As before the subgroups FQ , No = N H FQ are connected, and we have Rad FQ == (Rad F) H F^ , G^ =3 G° 0 Fo since the orbit £ + ( f / g ) * is simply connected. In particular FQ/G° is isomorphic to a subgroup of F/G° and is thus abelian. On the
other hand one may choose £ so that it vanishes on the subspace of f complementing 3 and left invariant by a fixed maximal connect- ed semisimple subgroup S of F. This implies that FQ contains S, and hence F^/N^ is reductive since F^/N^ ^ F^N/N C F/N . Moreover since F/Rad F is semisimple, F^/Rad F^ is actually isomorphic to F/Rad F. In particular F^/Rad F^ also has finite center. Finally KD/exp ker S. = (K/exp ker £) D clearly satisfies the requirements of the subgroup Z . Hence it follows from the induction hypothesis that there exist a closed connected normal subgroup H^ of F() such that N() C H^ C Rad G^ , and a linear form £1 on ^ extending £ such that the H^-orbit of £^ in
^ is ?i + ( ^ / f ) * . Now as in the proof of Theorem 2.12 case j8) there is an extension C^ of S.^ to t) = n 4- ^ such that theH-orbit of Ci in if is JZ^ 4- ( ^ / ^ )* where H = NH^ .
P) Now assume that the center of N contains Z° strictly.
Let us choose a closed connected normal subgroup K of F lying in the center of N such that the adjoint representation of F induces an irreducible representation p of F in f / g . Note as above that KD is closed and the exponential map is a diffeomorphism of f / a onto K/Z°. Thus Theorem 1.1 together with Theorem 1.3 of [1] imply that the quasi orbit defined by TT|KD is concentrated in an orbit which is open in 'x(KD/Z°D)^ where "x is some extension of x to K D , i.e. the G-orbit of C^ — v^T^x' in f * is an open subset of £ 4- ( f / ^ )*. Moreover TT is induced from a square integrable mod KD irreducible x-representation a of the subgroup GQ = G n FQ where
Fo = { j c E F r x O c f c c - ^ X W , V ^ G K D }
= { x € F : S . o A d ^ ( x ) == £ } . Suppose that p is non trivial. Since p may be viewed as a representation of the reductive group F/N, it follows from Lemma 1.3 that there is ^ in (f /a )* such that £"o Ad ^ (x) - I = p(x) J^ - C^, V x € F , where p is the representation of F in ( f / g ) * contra- gredient to p . Let F be the smallest real algebraic subgroup of G L ( ( f / 3 ) * ) containing ?(F) and F^ = {3?€ F:7Co = J?o}. Then
<^^
FQ and hence p(Fo) has finitely many connected components.
Moreover the projection FQ of F^ in F/Rad F has finitely many connected components and F°/Rad FQ has finite center. Therefore
F, = (Rad F) Fo has finitely many_ connected components and F^/Rad F^ has finite center. Now f o may be viewed as a sub- algebra of a fixed maximal semisimple subalgebra » of f which may be chosen to be in 9 so that dp induces an isomorphism of some nilpotent ideal n ^ of f o such that To/1 1! is reductive onto the Lie algebra u ^ of the unipotent radical U^ of F^ . In fact n ^ m a y b e chosen to be in Q ^ H ^ since n ^ is clearly contained in the maximal semisimple sub algebra dp ( ^ ) o f r f p ( f ) . Therefore if N^ denotes the analytic subgroup of G^ corresponding to n ^ then (Rad F)N^ is a closed normal subgroup of F^ . Moreover k e r p n N^ == 0 since the covering N^ —> U^ is trivial, for U^ is simply connected. In fact, we have (Rad F) n N^ = 0 since ker p H N^ is easily seen to be of finite index in (Rad F) H N^ . In particular we have N D H N ^ = 0 and hence NN^ n D = 0 . Now NN^/N is a direct factor of the connect- ed Lie group ( R a d F ) N ^ N and hence a topological direct factor.
Thus NN^/N is closed in ( R a d F ) N ^ N , i.e. NN^ is a closed normal subgroup of Go. Note that [ f i , f J C n + [ f o , f o ] C g H f o == g ^ . Hence F^/GQ is abelian. Moreover
f i / ( " + " i ) = ( R a d r ) + r o / (^+ ti^) = ( R a d f ) +To/(^ + " i ) - Therefore f i / ( n + n ^ ) is the direct product of the abelian central ideal (Rad f ) 4 - n ^ / ( n + n ^ ) ^ (Rad f ) / n and the reductive sub- algebra (to + tO/O1 + "i) ^ f o / " i • Hence f i / ( n + "i) is reductive.
Note finally that the restriction of a to GQ H F^ splits into at most finitely many irreducible components. Thus we are in a situation to apply the induction hypothesis and get a closed connected normal subgroup Hi of Go O F ? containing NN^ , and a linear form EI of ^ extending £ such that the H^-orbit of ^ in ^ is
^i + ( ^ i / O * . In particular J^ may be chosen to vanish on (Si ^T) •+• ( ^ i n ^ ) where JF is a ^ -invariant subspace of r com- plementing f . But then C^ °Ad^ (f) = 0 for all ^ in n ^ , i.e.
n ^ C f . Thus n ^ = 0 and hence u ^ = r f p ( n ^ ) = 0 , i.e. F^ is reductive. This implies that F/FQ is a real affine algebraic variety.
Therefore, according to Lemma 1 of [2], the center of F has a non trivial connected R-split subgroup. In particular there is x in Rad F such that p ( x ) = c Id with c E R \ { 1 , - 1} . Thus it follows from Proposition 2.8 that [det Ad;, (;c)| ^ 1 , i.e. |det Ad,^(x)| ^ 1 (note that Rad F/H^ is abelian). This however contradicts the fact
that F is unimodular. Thus p is trivial. Then it is easily seen that the R-orbit of £ in f * is ^ ( f / ^ ) * , and that there exists a closed one-parameter subgroup T of R such that R is the semi direct product of R H FQ_ and T. Put H = H i T , and let J^ be an arbitrary extension of ^ to 1) . Then as in the proof of Theorem 2.9 case 7), we see that the H-orbit of S.^ in f?* is S.^ 4- ( h / ^ ) * . QED COROLLARY 3.2. - // N does not reduce to the identity then neither does Z°.
Now let us apply the above Proposition for a connected non semisimple Lie group G. It is clear that the nilradical N and the center Z of G satisfy the conditions of Proposition 3.1 if we take F = G. Hence there exist a closed connected normal sub- group H such that N C H C R , and a linear form ^ on () extending S. such that the H-orbit of ^ in {}* is ^ 4- O / ^ ) * . Put S == { x G G : ^ o A d ( , O c ) = J ^ } . Then G = HS, H 0 S = Z°
according to Lemma 2.2. Note that by Corollary 3.2, Z° does not reduce to the identity since G is not semisimple. As usual we assume that dim Z = 1 and £ is non zero on 5 . Let_J) = ker J^ . Then S may be characterized as S = [x E G: Ad(x) f) C {j} .
Thus Bo is a non degenerate symplectic form on f) and Ad^-(S) may be viewed as a subgroup of S p ( f ) , B c ) so that the function Jg in [ 1 ] is defined and continuous on S . Let v : ? —> S be the at most two sheeted covering of S on which Jg has con- tinuous square roots. Let us denote again by v the covering v : H ©? —> H ® S . Note finally that TT lifts to a square integrable mod Z ° © Z irreducible representation of H © S and hence to a square integrable mod Z° (j)v~l(Z) irreducible representation T?
of H © I S . We have:
LEMMA 3.3. - The member T6 of the discrete series of H extends to an irreducible representation of H © S denoted again by T6. Moreover Tr may be written in the form
7 F ( A , 7 ) = T ^ A , ? ) ® ^ ) V ( / ! , 7 ) G H © S (3.1) where o is a square integrable mod ^(Z) irreducible ^-represen- tation of S, 5? = X ° ^ I ^ ' ^ Z ) . In particular the discrete series of S exist.
Proof. - The proof is just a copy of Theorem 4.5 of [1]. The only extra case to be considered is when N is isomorphic to a Heisenberg group with center Z°. In this case, according to the proof of Theorem 2.9, Hjnay be written as H = N H ^ , N H H^ = Z°, and the representation T6 obtained from T2 by composing the latter with the canonical homomorphism N © H i — — > NH == H is determined by a relation similar to (3.1) in which T6 is replaced by the member T^ of the discrete series of N , and a by the member T^ of the discrete series of Hi which is also a Heisenberg group with center Z°. Let Jg and Jg' be defined by the structures of the Heisenberg groups N and H^ respectively in the same manner as J g . Then Jg = J^ on S . Let v : S —> S be the at most two sheeted covering of S such that Jg and hence J^ have continuous square roots on S. Then T^ and T^ extend to irreducible repre- sentations of ( N © H i ) ® S ^ N © ( H i ©S) and H^ © S respec- tively. Let us denote these representations again by T^ and T^
respectively. Put
T| (n, h,, s) = ^(n, h,, s) ® T^h,, s)
V ( 7 2 , ^ , 5 ) ( E ( N © H i ) © S . If i>(s) = 1 we have
T i d . l . ^ T f d , 1,^^(1,5)
= = ( J , o ^ ) l / 2 ( ^ ) (J^ 0 ^ 0 ^ ) 1 / 2 (5). M
=-- ( J f i o ^ o ^ . I d ^ d and
T ^ z . z - ^ D ^ T^z.z-1,!)®^-1,!)^ T^z.z-1) == I d , Vz E Z° . Therefore T^ defmes, by passing to quotient, an irreducible representation of H © S extending T^ . Let us denote this repre- sentation again by T8. On the other hand Tf lifts to an irreducible representation 7^ of ( N © H i ) ( s ) S . Hence by Lemma 3 1 of [ I ] , TTi may be written as
Tr^n, AI , s) = T^n, AI , s) ® ^ ( A ^ , 5),
V ( ^ , / ? i , 5 ) E ( N © H i ) © S (3.2) where TT^ is an irreducible representation of H^ © S such that
^2 I zo © 1 ^ mult x ® 1 . Thus TT^ may be written as
^ (h,, s) == T^(h,, s ) ® a(s) , V(/^, s) G H^ © S (3.3) where & is an irreducible representation of S such that a\(vov)~^(Z) ^ mult X, X = X ° ^ ° ^ 10° ^^(Z).
From (3.2) and (3.3) we get
TT^ , /^ , s ) = Tf(^ , A,, s) 0 T^/^ , s) ® a(5)
= T^^ , h,, 5) ® a(i) V(^ , ^ , i) G (N © H,) © S . Since both Tr^ and T^ are trivial on ker v so is a , and hence a defines by passing to quotient an irreducible representation a of S . Moreover
?(A,7) = T^/z,?)® a(7) V ( A , 7 ) E H ® ? .
From this and the fact that Tr is square integrable mod Z° (s) ^(Z) it follows that a is square integrable mod^'^Z).
QED Let the notations be as above so that G = HS, H H S = Z° . By Lemma 3.3 there is an at most two sheeted covering v : S —^ S
'•^>
such that S has a square integrable mod ^"'(Z) irreducible repre- sentation. In particular Cent^V^'^Z) n Cent(S) is compact, and hence so is Cent(S)/Z. Now it is easily seen that ^(Z) n Cent(S) has finite index in ^(Z). Therefore the discrete series of S exist.
Suppose that S is not reductive. Then Rad S and thus Rad S are two-step nilpotent Lie groups. Therefore it follows from Proposition<^» '"^
3.1 that Rad S is a Heisenberg group with center Z^ , and hence Rad S is a Heisenberg group with center Z^ = ^ ( Z ^ ) . Let S^ be a maximal connected semisimple subgroup of S, and V a S^- invariant subspace of Rad si complementing a ^ . Then i^ == 3 4- V is plainly an ideal of 5 which is a Heisenberg algebra with center 3 . Let H^ be the analytic subgroup of S corresponding to ^ . Then it is obvious that Rad S/Z° is the direct product of H^/Z° and Z ^ / Z0. Therefore HH^ is a closed normal subgroup of G. Moreover we have C ^ ^ H H i by Lemma 2.4. Hence by replacing H by HH^
and S by Z^S^ if necessary we may assume that S is reductive.
We have thus proved:
THEOREM 3.4. - Let G be a connected unimodular Lie group
\^ith center Z . Assume that G/Rad G has finite center. Let x be
a character of Z . Put C == — ^/- 1 d\ G 3 * . // G Aa^ ^ square integrable mod Z irreducible ^-representation, then G may be written as G = HS, H H S = Z° "where H is a closed normal subgroup of G which is a H-group with center Z° such that C E % ^ , a^cf S is a connected reductive subgroup such that Cent(S)/Z is compact and that S has an at most two sheeted covering with discrete series.
COROLLARY 3.5. - Let G be the semi direct product of a simply connected solvable unimodular Lie group R and a connected semisimple Lie group S with finite center. Let Z be the center of G so that Z Fl R is the direct product of Z° and a discrete finitely generated free abelian group D. If the discrete series of G exist then:
i) S has an at most two sheeted covering with discrete series in the ordinary sense. In particular S has a compact Cartan subgroup.
ii) R is the semi direct product of a H-group H with center Z° and a vector group T containing D such that T/D is compact.
More precisely, let TT be a square integrable mod Z irreducible
^-representation of G where \ E Z . Then C = — \/— 1 d\ ^%H . Proof. — By Theorem 3.4 G may be written as G = HS^ , H ^ S^ = Z° , where H is a H-group with prescribed properties and S^ is a reductive subgroup having at most two sheeted cover- ing with discrete series such that Cent(S^)/Z is compact. Moreover by replacing S^ by some conjugate we may assume that it contains S. Let T be the connected subgroup of Cent(S^) generated by the one-parameter subgroups passing by the elements of D . Then S^ is clearly the direct product of Z° , T and S. Moreover T/D is compact since dim T = rk D. Finally S has an at most two sheeted covering with discrete series since S^ does. Note that S also has a finite covering which is acceptable in the sense of [7].
Hence S has a compact Cartan subgroup (cf. [6]). ^r?^
vj.bu For the converse of Theorem 3.4 and Corollary 3.5 we have:
THEOREM 3.6. — Let G be the semi direct product of a H- group H with center Z and a connected Lie group S. Assume
that t) has a S-invariant subspace ^ complementing 5 so that the center of G may be written as the direct product of Z and a subgroup D lying in the center of S. Then G has an at most two sheeted covering ^ : G == H (j) S —> G such that every member T6 of the discrete series of H can be extended to an irreducible repre- sentation of G denoted again by T6. Moreover let x e (Z^'^D))^
be such that d\\ 3 == \/—1 C . Then every ^-representation of G may be written as
7T(h, 7) = T6 (A , 7) ® a(7) \f(h, 7) E G (3.4) where a is a ^-representation of S, x^ = X\v~l(D).
Finally TT is square integrable modZ^'^D) if and only if a is square integrable mod ^(D).
Proof. — First note that it follows immediately from the defi- nition of H-groups that ZS is precisely the set of those elements of G leaving ^ invariant. Therefore the center of G is equal to ZD as claimed. Now as in the proof of Lemma 3.3, the existence of the subspace f) allows one to define the continuous function Jg on S. Then the covering v is chosen so that Jg o v has con- tinuous square roots on S. The rest of the proof is carried out exactly as in Lemma 3.3 and Theorem 4.5 of [1].
QED Remark. — 1) If the discrete series of G exist then S has square integrable mod ^(D) irreducible representations. Therefore by Theorem 3.4 S may be written as S == H^ , where H^ is a H- group with center D° and Si is reductive. Now HH^ = (HD°) (ZH^) is a H-group with center ZD° by Lemma 2.4. Moreover ZS^ is reductive, and hence G satisfies the statements of Theorem 3.4 as expected.
2) Suppose that S is reductive and Cent(S)/D is compact.
Then Adi/S) is a connected reductive Lie group with compact center and hence the subspace f) always exists. Furthermore if D is discrete then Theorem 3.6 may be viewed as a converse of Corollary 3.5. The members of the discrete series of G are described explicitly by (3.4).
3) Assume again that S is reductive and Cent(S)/D is compact.
Assume now that there is a (topological) isomorphism <p from Z
onto a closed subgroup of D , and that S has a square integrable mod D irreducible ^-representation a , where \^ E D is such that £ = — V ^ T d C X i ° ^ ^ H • Let Tfi be the irreducible re- presentation of H (s) S extending the member of the discrete series of H corresponding to S. Let 7 be the pull back of a to ?. Then (3.4) gives a square integrable mod Z@v~l(D) irre- ducible representation of H (s) S which is trivial on
{(z,,z^€Z®Z:^(z,)=v(z^)}
where Z = (^"^(Z))0. Hence by passing to quotient we obtain the following converse of Theorem 3.3:
COROLLARY 3.7. - Let G be a connected Lie group with center Z . Suppose that G == HS, H H S = Z° where H is a H-group with center Z° and S is a connected reductive subgroup containing Z such that Cent(S)/Z is compact. Finally assume that S has a square integrable mod Z irreducible -^-representation where xE Z is such that — \J- 1 J X ^ ^ H • Then G has an at most two sheeted covering with discrete series.
Now combining Corollary 3.5 and the remark 2) above of Theorem 3.6 we have the following "algebraic" characterization ofunimodular Lie groups with discrete series:
THEOREM 3.8. - Let G be a connected unimodular Lie group with center Z . Assume that the radical of G is simply connected and that G has a maximal connected semisimple subgroup with finite center. Then G has a finite covering with discrete series if and only if it may be written as the semi direct product of a H- group with center Z° and a connected reductive subgroup which is itself the direct product of a vector group T such that T/T H Z is compact and a connected semisimple subgroup with compact Carton subgroup.
Remark. - 1) The connected Lie groups with discrete series in which the conclusion of Theorem 3.8 does not hold are necessarily non unimodular. The "ax 4- &" group, and more generally the group given in the example of [2] are such.
2) In view of [16], every almost connected locally compact unimodular group with discrete series is an extension of a connected
unimodular Lie group with discrete series by a compact group.
However there is no satisfactory characterization of such groups at the present moment.
4. On a conjecture of Wolf and Moore.
In this section we give another characterization of connected unimodular Lie groups with discrete series which, in the case of simply connected solvable Lie groups, solves the conjecture stated in [15].
THEOREM 4.1. — Let G be the semidirect product of a simply connected unimodular solvable Lie group R and a connected semi- simple Lie group with finite center. Let Z be the center of G, and X a character of Z. If G has a square integrable mod Z irreducible X-representation then there exists a linear form S. on Q extend- ing ~ \/—1 d\ such that
(4.1) Gg/Z is compact, where Gg is the centralizer of S. in G.
Conversely, let S. be a linear form on g satisfying (4.1).
Then G has a finite covering G with discrete series. In fact members of the discrete series of G may be chosen so that their restrictions to Z° are equivalent to multiples of the character exp(-\/-l C [ 5 ) of Z°.
Proof. — First assume that G has a square integrable mod Z irreducible ^-representation. Then, according to Corollary 3.5, G may be written in the form G = H © (T x S) where H is a H- group with center Z° such that — v ^ T r f x ^ ^ H ' T is a vector group such that T/T H Z is compact and S is a connected semi- simple Lie group with compact Cartan subgroup. Note that T and S have been chosen so that ^ o Ad^(x) = J^ , Vx E T x S, where BI is a fixed extension of —\/— 1 d\ to ^ . Let ^ be a regular element of ^ such that its centralizer C in S is a compact Cartan subgroup. Let £ be any extension of J^ to g such that £ | ^ is defined by ^ via the Killing form of ^ . Then it is clear that Gg = Z°TC and hence Gg/Z is compact.
Conversely suppose that there is some £ in Q * satisfying (4.1). Then by an argument similar to that in the proofs of Pro- position 3.1 and Theorem 3.3 which is omitted here, one finds that G = HS, H H S == Z° , where H is a H-group such that the H-orbit of £^ = £ | I; in ()* is £^ + ( I ; / a ) * and S is a connect- ed reductive subgroup such that £^ o Ad^(x) = £^ , Vx CE S . In particular Sg = Gg , where £^ = £ 15 and Sg is the centralizer of £2 in S. It follows that Cent(S)/Z is compact. Hence S may be written as S = Z ° x T x S ' where T is a vector group such that T/T H Z is compact and S' is a connected semisimple Lie group with finite center. Let £3 = ^ l ^ 'r; then we have Sg = Z ° x T x S g , where Sg is the centralizer of £3 in S\
In particular Sg is compact. Let ^ E s/ be such that £3 is defined by $ via the Killing form of 5 \ If { is not semisimple then Sg contains unipotent elements so that it cannot be compact.
Hence $ must be semisimple. In particular Sg contains a Cartan subgroup of S' which is of course compact. Therefore G has a<"^/
finite covering G with discrete series according to Theorem 3.8.
In fact members of the discrete series of G with prescribed pro- perties may be chosen as indicated by (3.4). opn In the case when G is a simply connected solvable Lie group G = G, and we have the following more precise statement.
COROLLARY 4.2. — Let G be a simply connected unimodular solvable Lie group with center Z . Let x ^ Z . Then G has a square integrable mod Z irreducible ^-representation if and only ifthere is an extension £ of — ^/- 1 d\ to 9 such that
(4.2) Gg/Z is compact.
Moreover if it is the case then every extension of — \/~Td\ to g will satisfy (4.2).
Proof. — It remains to prove the last assertion. Thus assume that — \/~Td\ has an extension £ satisfying (4.2). Then as in the proof of Theorem 4.1, G may be written as G == H © T, where H is a H-group with center Z° such that £ | a ^ % H anc* T is a vector group such that T/T H Z is compact. Let £^ be any extension of — ^/— 1 d\ to g . Then £^ belongs to the same
orbit as f i ^ ^ g * such that C ^ ( f ) ) = 0, where I) is a fixed T- invariant subspace of I) complementing 3 . Note that such a sub- space exists since Ad^(T) is compact. Now it is plain that Go = Z°T and hence Gg /Z is compact. Therefore Go /Z is also compact since Go is conjugate to Go . ,.p^
Let G = H(s)T be as above. Then, according to Theorem 3.5, every irreducible x-representation of G may be written in the form
-n(h,t) --= \(t) T8^ , t) V A E H , V r G T (4.3) where S. = — \/— 1 d\ and T6 is a fixed extension of the member of the discrete series of H corresponding to £ and X is a character of T. Moreover these representations are all square integrable mod Z and they constitute all possible extensions of that member of the discrete series of H to G. Thus we have:
COROLLARY 4.3. - Let G = H ©T be as above. Then for each
£ E % ^ , the irreducible representations of G whose restrictions to Z° are equivalent to multiples of exp \/~~ 1 9. belong to the discrete series of G. Moreover they are precisely the extensions of the member T8 of the discrete series of H to G .
Thanks to Corollary 4.3, we can now state the following Theorem which gives the solution to the conjecture of Wolf and Moore stated in [ 15]:
THEOREM 4.4. — Let G be a simply connected unimodular solvable Lie group with center Z. Then the members of the dis- crete series of G if they exist are precisely the Auslander-Kostant's
"induced" representations associated to the integral orbits ©g such that Go/Z is compact.
Proof. — It follows from Corollary 4.2 that the existence of the discrete series of G is equivalent to the existence of S. in g*
such that Gg/Z is compact. Moreover if it is the case then Go is connected and hence the corresponding orbit is integral. Therefore the Theorem is trivial if the above equivalent conditions do not hold.
Now assume that the discrete series of G exist so that G may be written as G = H © T as in Corollary 4.3. Let f) be a T-invariant
subspace of f) complementing 3 . Then every S. G g * such that
£ l 3 ^ ^ H is conjugate to an C^ E Q* such that £^(^) = 0. On the other_hand ]et C ^ ^ G Q * be such that j^| 5 = ^ 1 3 and that C i ( ^ ) = C^) =^0._Assume that £3 = J^ o MX for some x € G . Then Ad(x) ^ C f) and hence ^ e Z ° T . Moreover J^ = £3 on t since T is abelian. Therefore ^ = £3 . Thus we have proved that the set S = { C E Q*: t | g e %„ and £(^) = 0} is a cross section of the set of integral orbits ©g such that Gg/Z is compact.
For each Ci E g , let T?^ be the character of G^ = Z°T such that dr?^ = v/'^TCjg^ . Let ^ be a positive strongly admissible polarization at ^ , and ^ = ind(7^ , ^ ) be the "induced" re- presentation constructed by Auslander and Kostant in [4]. Note that TT, is irreducible since J^ is integral. Let X be a character of T and let us extend it to a character of G by letting X(H) = { 1 } . put £ = - ^/~Td\ and £3 = j^ 4- £. Then it is clear that ^ E S.
In particular G^ = Z°T = Gg^ , and hence ^ is also a polarization at 63. Let 71^ = ind(7^,^) be the corresponding irreducible re- presentation of G. Let Xe^ and x^ be the characters of the group D introduced in [4] so that their restrictions to Z°T are 77^ and 17^ respectively. Then TT^ and TT^ are subrepresentations of the ordinary induced representations md^^X^ and ind^cX^
respectively. Now it is clear that ^ »—> \^ sets up a unitary equi2
valence between the representations X ind^c X^ and indite Xg . Moreover it follows immediately from the definition of TT^ ami TT^ that this map carries the subspace of the representation space of X ind^o Xfi^ corresponding to TT^ onto that corresponding to TT^ . Hence we have
T T ^ X T T i . (4.4)
Finally since Z° C G^ C D we see that the restriction of
"^DIG Xfi^ to Z° and hence the restriction of TT^ to Z° is equi- valent to a multiple of e x p y ^ T ^ i h ) . In particular 7^ is a member of the discrete series of G. This together with (4.4) and (4.3) show that the family of all representations TT^ associated to £, E § is precisely the discrete series of G.
QED Remark - In view of (4.3) and the description of the discrete series of H in section 2, our result is more precise than Theorem 3.5
of [10], at least for the unimodular case. Moreover in this case Theorem 4.4 contains that Theorem strictly since our groups are not necessary of type I (cf. example 2.5). Note also that in [17]
J.Y. Charbonnel obtained a similar result even for non unimodular solvable groups. There is however a supplementary assumption, namely the regular representation is of type I.
BIBLIOGRAPHY
[1] NGUYEN Huu ANH, Lie groups with square integrable represen- tations, ^4 ^flfao/Afflrt., 104 (1976), 431-458.
[2] NGUYEN Huu ANH, Classification of unimodular algebraic groups with square integrable representations, Acta Math. Vietnam.
(to appear).
[3] NGUYEN Huu ANH and V.M. SON, On square integrable factor representations of locally compact groups, Acta Math. Vietnam.
(to appear).
[4] L. AUSLANDER and B. KOSTANT, Polarization and unitary repre- sentations of solvable Lie groups, Invent. Math., 14 (1971), 255-354.
[5] C. CHEVALLEY, Theorie des groupes de Lie, Vol. 3, Act. Sci Ind., n° 1226, Hermann, Paris, 1955.
[6] HARISH-CHANDRA, The discrete series for semisimple Lie groups II. Explicit determination of the characters, Acta Math., 116 (1966), 1-111.
[7] HARISH-CHANDRA,. Invariant eigendistributions on a semisimple Lie group, Trans. A.M.S., 119 (1965), 457-508.
[8] G.W. MACKEY, tfnitary representations of group extensions I, Annals of Math., 99 (1958), 265-311.
[9] C.C. MOORE, The Plancherel formula for non unimodular groups, Abs. Int. Cong. on Func. Analysis, Univ. of Maryland, 1971.
[10] J. ROSENBERG, Square integrable factor representations of locally compact groups, Preprint, Univ. of Calif., Berkeley.
