Symmetric pairs and branching laws

(1)

HAL Id: hal-01818302

https://hal.archives-ouvertes.fr/hal-01818302

Preprint submitted on 18 Jun 2018

HAL is a multi-disciplinary open access archive for the deposit and dissemination of sci- entific research documents, whether they are pub- lished or not. The documents may come from teaching and research institutions in France or

L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires

Symmetric pairs and branching laws

Paul-Emile Paradan

To cite this version:

Paul-Emile Paradan. Symmetric pairs and branching laws. 2018. �hal-01818302�

(2)

Symmetric pairs and branching laws

Paul-Emile PARADAN^∗ June 18, 2018

Abstract

LetGbe a compact connected Lie group and letH be a subgroup fixed by an involution. A classical result assures that theHC-action on the flag varietyFofGadmits a finite number of orbits. In this article we propose a formula for the branching coefficients of the symmetric pairpG, Hqthat is parametrized byHCzF.

1 Introduction

Let G be a compact connected Lie group equipped with an involution θ.

Let G^θ :“ tg P G, θpgq “ gu be the subgroup fixed by the involution. We consider a subgroup H Ă G such that pG^θq⁰ Ă H Ă G^θ. The purpose of this paper is the study of the branching laws betweenG andH.

Let T be a maximal torus of G that we choose θ-invariant. Let t be the Lie algebra of T. Let Λ Ă t^˚ be the lattice of weights, and let t^˚_` be a Weyl chamber. The irreducible representations ofG are parametrized by the semi-group Λ^`_G:“ΛXt^˚_` of dominant weights.

Let λ P Λ^`_G. In order to study the restriction V_λ^G|H of the irreducible G-representationV_λ^G, we consider theH-action on the flag varietyF “G{T ofG. An important object is theH-invariant subset

Z_θ ĂF

formed of the elements xPF for which the stabilizer subgroup G_x :“ tgP G, gx“xuis stable underθ. In orther words,gT PZ_θif and only ifg^´¹θpgq belongs to the normalizer subgroupNpTq. A well-known result tells us that the group H has finitely many orbits in Z_θ, and that the finite set HzZθ

parametrizes theHC-orbits in F [7, 13, 11, 8].

LetxPZ_θ. The stabilizer subgroupG_x is a maximal torus inGwith Lie algebragx. We will also consider the abelian subgroupHx:“GxXH (that is not necessarily connected). Any weightµPΛ determines a characterC_µ

x

of the torus G_x by takingµ_x“g¨µif x“gT PF.

We denote byRxĂg^˚_xthe set of roots relative to the action of the Cartan subalgebrag_xon gbC. The mapµPRÞÑµ_xPR_x is an isomorphism, and we takeR^`_x ĂRx as the image ofR^`ĂRthrough this isomorphism.

The involutionθleaves the setRx invariant, andαPRx is animaginary root if θpαq “ α. If α is imaginary, the subspace pg bCqα is θ-stable.

There are two cases. If the action of θ on pgbCqα is trivial then α is compact imaginary. If the action of´θonpgbCqα is trivial, thenαisnon- compact imaginary. We denote respectively by R^ci_x and byR^nci_x the subsets

(4)

of compact imaginary and non-compact imaginary roots, and we introduce the followingGx-modules

E^ci_x :“ ÿ

αPRci

xXR^`x

pgbCqα, E^nci_x :“ ÿ

αPRnci

x XR^`x

pgbCqα.

The weight

δpxq:“ 1 2

ÿ

αPR` xXθpR`

xq

θpαq‰α

α

defines a character C_δ_p_x_q of the abelian groupH_x. Let mx“ 1

2|R^`_x XθpR^`_xq X tθpαq ‰αu| `dimE^nci_x .

We denote byRpHqand by RpHxqthe representations rings of the compact Lie groups H and H_x. An element E P RpHq can be represented as a finite sum E “ ř

VPHpm_VV, with m_V P Z. We denote by RppHq (resp.

RpHp xq) the space of Z-valued functions on Hp (resp. Hx_x). An element EPRppHq can be represented as aninfinitesumř

VPHpm_VV, withm_V PZ. The induction map Ind^K_H : RpHp xq Ñ RpHqp is the dual of the restriction morphismRpHq ÑRpH_xq.

The main result of this paper is the following theorem.

Theorem 1.1 Let λPΛ^`_G. We have the decomposition

(1.1) V_λ^G|H “ ÿ

HxPHzZ_θ

Q_Hxpλq

where the terms Q_Hxpλq PRppHq are defined by the following relation : QHxpλq “ p´1q^m^xInd^H_H_x´

C_λ

x`δpxqbdetpE^nci_x q bSympE^nci_x q bľ E^ci_x

¯ . Here SympE^nci_x q, which is the symmetric algebra of E^nci_x , is an admissible representation ofH_xandŹE^ci_x “Ź_`E^ci_xaŹ_´E^ci_x is a virtual representation of H_x.

We give now another formulation for decomposition (1.1) using the (right) action of the Weyl group W “ NpTq{T on the flag variety F. If x “gT P F and w PW we take xw :“ gwT. We notice immediately that Z_θ is stable under the action of W.

(5)

We associate to an elementx“gT PZ_θthe subgroup W_x^H ĂW defined by the relation w P W_x^H ðñ Hxw “ Hx. We denote by HzZθ{W the quotient of Z_θ by the action of HˆW, and by ¯x PHzZθ{W the image of xPZ_θ through the quotient map. We associate to ¯xPHzZ_θ{W the element Q¯xpλq PRpHqp defined as follows

Qx¯pλq “ ÿ

w¯PW_x^HzW

QHxwpλq.

Theorem 1.1 says then that V_λ^G|H “ ř

¯xPHzZ_θ{WQx¯pλq. Here is a new formulation of Theorem 1.1.

Theorem 1.2 We have V_λ^G|H “ ř

x¯PHzZθ{WQ¯xpλq where Qx¯pλq P RpHqp has the following description

Q¯xpλq “Ind^H_H_x´

M_xpλq bC_δ

pxqbľ E^ci_x

¯ ,

for some¹ M_xpλq PRpHp xq.

We finish this section by giving two basic examples associated to the group SUp2q. Here the flag variety of SUp2q is the 2-dimensional sphere S². For n ě0, we denote by Vn the irreducible representation of SUp2q of dimensionn`1.

Example 1. G“SUp2q and the involution θis the conjugaison by the matrix

ˆ 1 0 0 ´1

˙

. The subgroup fixed byθ is the torusT »Up1qand the critical set Z_θ Ă S² is composed by the poles S, N and the equator E, so thatTzZθ has three terms. We takeλ“ninSU{p2q »N.

ForHx“E, we haveE^nci_x “E^ci_x “ t0u,H_x»Z₂, and C_λ

x`δpxq“C_n|^Z2. The contribution ofE is then Ind^U_Z₂^p¹^qpC_n|Z2q “C_nbř

kPZC₂_k. ForHx“N, we haveH_x “T,E^nci_x “C₂,E^ci_x “ t0u, andC_λ

x`δpxq “C_n. The contribution ofN is then´C_n_`₂bSympC₂q.

For Hx “S, we have H_x “T, E^nci_x “ C_´₂, E^ci_x “ t0u, and C_λ

x`δpxq “ C_´_n. The contribution ofS is then´C_´_n_´₂bSympC_´₂q.

Finally, Relations (1.1) become V_n|T “ C_nbÿ

kPZ

C₂_k´C_´_n_´₂bSympC_´₂q ´C_n_`₂bSympC₂q

“ ÿ0 k“´n

C₂_k

`n.

1The precise expression ofM_xpλqis given in Proposition 3.8.

(6)

Example 2. G “ SUp2q ˆ SUp2q and the involution θ is the map pa, bq ÞÑ pb, aq. The subgroup fixed byθ isSUp2q embedded diagonally and the critical setZ_θ ĂS²ˆS² is equal to the union of the orbitsSUp2q¨pN, Nq and SUp2q ¨ pS, Nq. Letλ“ pn, mq PG.p

For x “ pN, Nq or x “ pS, Nq we have E^nci_x “ E^ci_x “ t0u and Hx »T. Forx“ pN, Nq we have λ_x`δpxq “m`n`2, and forx“ pS, Nqwe have λ_x`δpxq “m´n. Relations 1.1 give then

V_nbV_m“Ind^SU_T ^p²^qpC_m_´_nq ´Ind^SU_T ^p²^qpC_m_`_n_`₂q.

It is not difficult to see that the previous identities correspond to the classical Clebsch-Gordan relations (see Example 4.2).

Notations Throughout the paper :

• Gdenotes a compact connected Lie group with Lie algebra g.

• T is a maximal torus in Gwith Lie algebrat.

• ΛĂt^˚ is the weight lattice ofT : everyµPΛ defines a 1-dimensional T-representation, denoted by C_µ, where t “ exppXq acts by t^µ :“

eⁱ^x^µ,X^y.

• The coadjoint action ofgPGon ξPg^˚ is denoted by g¨ξ.

• When a Lie groupK acts on set X, the stabilizer subgroup of xPX is denoted by K_x :“ tkPK |k¨x “xu and the Lie algebra of K_x is denoted byk_x.

• When a Lie group K acts on a manifold M, we denote by X¨m :“

d

dte^tX¨m|t“0,mPM, the vector field generated byXPk.

2 Non abelian localization

Our main result is obtained by means of a non-abelian localization of the Riemann-Roch character on the flag variety F of G. For that we will use the familypΩrqr of symplectic structure parametrized by the interior of the Weyl chambert^˚_`. The symplectic structure Ω_r comes from the identication gT Ñg¨r ofF with the coadjoint orbit Gr. The moment map Φr :F Ñg^˚ associated to the action ofG on pF,Ωrq is the map gT ÞÑg¨r.

(7)

At the level of Lie algebras we haveg“h‘qwhereh“g^θ andq“g^´^θ. For anyξ Pg“h‘q, we denote by ξ^`his h-part and byξ^´ hisq-part. We use a G-invariant scalar product p´,´q on g such that the involution θ is an orthogonal map. It induces identificationsg^˚ »g,h^˚»hand q^˚ »q.

The moment map Φ^H_r :F Ñh^˚ associated to the action ofH onpF,Ωrq is the mapgT ÞÑ pg¨rq^`.

2.1 Matsuki duality

Consider the complex reductive groupsGC and HC associated to the compact Lie groupsGandH. LetLĂGC be the real form such thatHĂL is a maximal compact subgroup ofL.

Matsuki duality is the statement that a one-to-one correspondence exists between theHC-orbits and theL-orbits inF; two orbits are in duality when their intersection is a single orbit ofH.

Uzawa, and Mirkovic-Uzawa-Vilonen [14, 8] proved the Matsuki correspondence by showing that bothHC-orbits andL-orbits inFare parametrized by theH-orbits in the set of critical points of the function}Φ^Hr }² :F ÑR.

First we recall the elementary but fundamental fact that the subsetZθ

is equal to the set of critical points of the function}Φ^H_r }² [8, 3].

Lemma 2.1 Letx“gT PF andr PInteriorpt^˚_`q. The following statements are equivalent:

i) the subalgebrag_x is invariant under θ (i.e. xPZ_θ), ii) g^´¹θpgq PNpTq,

iii) x is a critical point of the function }Φ^H_r }², iv) pg¨rq^` and pg¨rq^´ commutes.

Proof. Letn_g “g^´¹θpgq and letr be a regular element oft^˚ »t. Since g_x“Adpgqt we see that

θpgxq “g_x ðñ n_g PN_GpTq ðñ rn_g¨θprq, rs “0 ðñ rθpg¨rq, g¨rs “0 ðñ rpg¨rq^`,pg¨rq^´s “0.

A small computation shows that for any X P g the derivative of the function t ÞÑ }Φ^H_r pe^tXxq}² at t “ 0 is equal to pX,rg¨r, θpg¨rqsq. Hence

(8)

x“gT is a critical point of the function}Φ^H_r }²if and only ifrg¨r, θpg¨rqs “0.

Finally we have proved that the statementsiq, iiq, iiiqandivqare equivalent.

l

Let us check the other easy fact.

Lemma 2.2 The set HzZθ is finite.

Proof. Let x “gT P Z_θ. A neighborhood of x is defined by elements of the forme^Xe^Yx where XPhand Y Pq. Now we see thate^Xe^YgT PZ_θ if and only ife^´²^g^´¹^Y PNpTq. IfY is sufficiently small the former relation is equivalent to g^´¹Y P t, and in this case e^Xe^Yx “e^Xx. We have proved that any element inHzZθ is isolated. AsHzZθ is compact, we can conclude thatHzZ_θ is finite. l

2.2 Borel-Weil-Bott theorem

We first recall the Borel-Weil-Bott theorem. The flag manifoldFis equipped with theG-invariant complex structure such that

T_eTF » ÿ

αPR^`

pgbCqα

is an identity of T-modules. Let us consider the tangent bundle TF as a complex vector bundle on F with the invariant Hermitian structure hF

induced by the invariant scalar product ong.

Any weight λPΛ defines a line bundleL_λ »GˆT C_λ on F. Definition 2.3 We associated to a weight λPΛ

‚ the spin-c bundle on F

S_λ:“ľ

C

TFbL_λ,

‚ the Riemann-Roch character RRGpF,L_λq P RpGq which is the equivariant index of the Dirac operator D_λ associated to the spin-c structure S_λ. The Borel-Weil-Bott theorem asserts that V_λ^G“RRGpF,L_λq when λis dominant. Now we consider the restriction V_λ^G|H “ RRHpF,L_λq. In the next section we will explain how we can localize theH-equivariant Riemann- Roch character RR_HpF,L_λq on the critical set of the function }Φ^H_r }² [9].

(9)

2.3 Localization of the Riemann-Roch character

In this section we explain how we perform the “Witten non-abelian localization” of the Riemann-Roch character with the help of the moment map Φ^H_r :F Ñh^˚attached to a regular elementrof the Weyl chamber [9, 5, 10].

Let us denote byX ÞÑ rXsg{tthe projectiongÑg{t. The Kirwan vector fieldκr on F is defined as follows:

κ_rpxq “ ´Φ^H_r pxq ¨xPTxF.

Through the identificationg{t»T_xF, X ÞÑ _dt^d|t“0ge^tXT, the vectorκ_rpxq P TxF is equal to rg^´¹θpgq ¨rsg{t. Hence the set Z_θ Ă F is exactly the set whereκr vanishes.

Lat D0 be the Dirac operator associated to the spin-c structure S₀ “ Ź

CTF. The principal symbol of the elliptic operatorD0 is the bundle map σpFq PΓpT^˚F,hompŹ_`

CTF,Ź_´

CTFqqdefined by the Clifford action σpFqpx, νq “cxp˜νq:ľ_`

CTxF Ñľ_´

CTxF.

whereνPT^˚_xO»ν˜PTxO is the one to one map associated to the identification g^˚»g(see [2]).

Now we will deform the elliptic symbolσpFqby means of the vector field κ_r [9, 10].

Definition 2.4 The symbolσpFq shifted by the vector fieldκr is the symbol on F defined by

σ_rpFqpx, νq “cxp˜ν´κ_rpxqq for anypx, νq PT^˚F.

Consider anH-invariant open subsetU ĂFsuch thatUXZθis compact inF. Then the restrictionσ_rpFq|U is aH-transversally elliptic symbol onU, and so its equivariant index is a well defined element inRppHq(see [1, 9, 10]).

Thus we can define the following localized equivariant indices.

Definition 2.5 Let HxĂZ_θ. We denote by

RRHpF,L_λ,Φ^H_r , Hxq P RpHqp

the equivariant index ofσ_rpFqbL_λ|^U whereU is an invariant neighbourhood of Hx so that U XZ_θ“Hx.

(10)

We proved in [9] that we have the decomposition RR_HpF,L_λq “ ÿ

HxPHzZθ

RR_HpF,L_λ,Φ^H_r , Hxq P RpHq.p

The computation of the characters RR_HpF,L_λ,Φ^H_r , Hxq will be handle in Section 3.1. To undertake these calculations we need to describe geomet- rically a neighborhood ofHx inF. This is the goal of the next section.

2.4 Local model near HxĂZ_θ

Letx“gT PZ_θ. We need to compute a symplectic model of a neighborhood ofHx inpF,Ωrq. Here we use the identification g»g^˚ given by the choice of an invariant scalar product. Let µ “ g¨r that we write µ “ µ^``µ^´ whereµ^`Phand µ^´Pq.

The tangent spaceT_xF is equipped with the symplectic two form Ωr|x: Ωr|xpX¨x, Y ¨xq “ pµ,rX, Ysq, X, Y Pg.

We need to understand the structure of the symplectic vector space pTxF,Ωr|xq. IfaĂgis a vector subspace we denote bya¨x:“ tX¨x, X Pau the corresponding subspace of T_xF. The symplectic orthogonal of a¨x is denoted bypa¨xq^K^,Ω.

Ifa,b are two subspaces, a small computation gives that (2.2) pa¨xq^K^,^ΩXb¨x»a^KX rb, µs,

wherea^K Ăg is the orthogonal ofa relatively to the scalar product.

We denote by g_µ` “ h_µ` ‘q_µ` the subspaces fixed by adpµ^`q. Notice thatg_µ“g_x is an abelian subalgebra containing µ^` sincerµ^`, µ^´s “ 0. It follows thatgx Ăg_µ`.

Lemma 2.6 g_µ` ¨x and rh, µ^`s ¨x are symplectic subspaces of T_xF. Proof. It is a direct consequence of (2.2). l

We consider now the symplectic subspace V_x Ă TxF defined by the relation

(2.3) V_x“ prh, µ^`s ¨xq^K^,^ΩX rg, µ^`s ¨x.

A small computation shows thatX¨xPVx if and only if rX, µs Ă rq, µ^`s.

We have the following important Lemma.

(11)

Lemma 2.7 • We have the following decomposition (2.4) T_xF “g_µ^`¨x‘^K “

h, µ^`‰

¨x‘^K V_x where K stands for the orthogonal relative to Ω_r|x.

• g_µ^`¨x is symplectomorphic to h_µ^`{hx‘ phµ^`{hxq^˚.

• rh, µ^`s ¨x is symplectomorphic to h{h_µ^` equipped with the symplectic structure Ω_µ^`p¯u,vq “ pµ¯ ^`,ru, vsq.

• Vx is symplectomorphic toph¨xq^K^,^Ω{ph¨xq^K^,^ΩXh¨x.

Proof. If we use the decomposition g“g_µ`‘ rg, µ^`sand the fact that the abelian subalgebragx is contained in g_µ^` we obtain

T_xF “g_µ`¨x‘ rg, µ^`s ¨x.

It is obvious to check that the subspacesrg, µ^`s¨xandg_µ`¨xare orthogonal relatively to the symplectic form Ωr|x. Since rh, µ^`s ¨x is a symplectic subspace we haverg, µ^`s ¨x“ rh, µ^`s ¨x‘^K V_x whereV_x is defined by (2.3).

The first point is proved.

The identitiesg_x“θpgxq “g_θ_p_x_qimply the decompositionsg_x“h_x‘q_x andrgµ^`, xs “ rqµ^`, xs‘rhµ^`, xs. The vector subspacerhµ^`, xsis isomorphic to h_µ`{hx, and the map v ÞÑ Ωr|xpv,´q defines an isomorphism between rq_µ^`, xsand the dual of rh_µ^`, xs. The second point is proved.

For the third point we use the isomophism j :rh, µ^`s Ñh{hµ^` induces by the projectionhÑh{hµ^`. Then the map ¯uÞÑjpuq ¨¯ x defines a symplec- tomorphism betweenph{h_µ^`,Ω_µ`q and rh, µ^`s ¨x.

Now we see that (2.4) together with the decomposition h ¨ x “ rh, µ^`s ¨x`h¨x leads to

ph¨xq^K^,^Ω “ prh, µ^`s ¨xq^K^,^ΩX phµ^` ¨xq^K^,^Ω

“ prh, µ^`s ¨xq^K^,^ΩX pg_µ^` ¨xq ‘V_x

“ ph¨xq^K^,ΩXh¨x‘V_x. The last point follows. l

We denote by Ω_V_x the restriction of Ω_r|x on the symplectic vector sub- spaceV_x. The action ofH_x onpVx,ΩVxq is Hamiltonian, with moment map ΦVx :V_xÑh^˚_x defined by the relation

xΦVxpvq, Ay “ 1

2ΩVxpv, Avq, v PV_x, APh_x.

(12)

Thanks to Lemma 2.7, we know that the H_x-symplectic vector space pTxF,Ωr|xq admits the following decomposition

TxF »h_µ^`{hx‘ phµ^`{hxq^{˚ K}‘h{hµ^`

‘K V_x

Thanks to the normal form Theorem of Marle [6] and Guillemin-Sternberg [4], we get the following result.

Corollary 2.8 An H-equivariant symplectic model of a neighborhoood of Hx in F is F_x :“HˆH_µ_` Yx where

Y_x“H_µ`ˆHx

`ph_µ^`{hxq^˚ˆV_x˘ .

The corresponding moment map on F_x is

ΦF_xprh;η, vsq “hpη`µ^``ΦVxpvqq for rh;η, vs PHˆHx

We finish this section by computing a compatible complex structure on V_x.

By definition, the map that sendsX¨xtorX, µsdefines an isomorphism i: V_x Ñ rq, µ^`s. The adjoint map adpµq defines also an automorphism of rg, µ^`s: for any X P rg, µ^`s we denote by ˜X P rg, µ^`s the unique element such thatadpµqX˜ “X.

The symplectic structure Ω_µ:“ pi^´¹q^˚Ω_V_x satisfies the relations ΩµpX, Yq “ pµ,rX,˜ Y˜sq “ pX,Y˜q “ ´pX, Y˜ q, @X, Y P rq, µ^`s.

We consider the one to one map ´adpµqadpθpµqq : rg, µ^`s Ñ rg, µ^`s and the Hx-invariant complex structure J_µ` “ adpµ^`qp´adpµ^`q²q^´¹^{² on rg, µ^`s. It restricts to a one to one map T_x : rq, µ^`s Ñ rq, µ^`s and a complex structure on rq, µ^`s(still denoted by J_µ`).

Let S_x :“ pT_x²q^´¹^{²T_x. The mapJ_V_x :“J_µ` ˝S_x defines a H_x-invariant complex structure on rq, µ^`s.

Lemma 2.9 The H_x-symplectic space pVx,Ω_V_xq is isomorphic to rq, µ^`s equipped with the symplectic formΩ¹_µpv, wq “ pJ_V_xv, wq.

Proof. We know already that pV_x,ΩVxq » prq, µ^`s,Ωµq. If one takes L“Tx˝ p´adpµ^`q²q^´¹^{⁴˝ pT_x²q^´¹^{⁴, we check easily that ΩµpLpvq, Lpwqq “ pJVxv, wq. l

(13)

3 Proof of the main theorem

We start with the following lemma.

Lemma 3.1 The quantity RRHpF,L_λ,Φ^H_r , Hxq does not depend on the choice of the regular element r in the Weyl chamber. In the following we will denote it by Q_Hxpλq PRppHq.

Proof. Let r0, r1 be two regular elements of the Weyl chamber. For tP r0,1s, we consider the regular elementrptq “tr1` p1´tqr0: the Kirwan vector fieldκ_r_p_t_q vanishes exactly onZθ for anytP r0,1s. IfU is an invariant neighbourhood of Hx so that U XZ_θ “ Hx, then t P r0,1s ÞÑ σ_r_p_t_qpFq b L_λ|Û defines an homotopy of transversally elliptic symbols. Accordingly, the equivariant index ofσr0pFq bL_λ|Û and σr1pFq bL_λ|Û are equal. l

3.1 Computation of Q_Hxpλq

The computation ofQ_Hxpλq is done in three steps.

3.1.1 Step 1: holomorphic induction

Let H_µ` Ă H be the stabilizer subgroup of µ^` :“ Φ^H_r pxq. We know that an H-equivariant symplectic model of a neighborhoood of Hx in F is the manifoldHˆH_µ_` Y_x where

Y_x“H_µ`ˆHx

The symplectic two form onY_x is built from the canonical symplectic structure onH_µ`ˆHxphµ^`{hxq^˚ »T^˚pHµ^`{Hxqand the symplectic structure on V_x. The moment map relative to the action ofH_µ` on Y_x is

ΦYxprh;η, vsq “hpη`µ^``ΦVxpvqq Ph^˚_µ`, forrh;η, vs PH_µ` ˆHx

Let κ_Y_x the Kirwan vector field on Y_x. It is immediate to check that rh;η, vs P tκYx “0uif and only ifη“0 andpµ^``ΦVxpvqq ¨v“0. The map vÞÑµ^`¨v is bijective andvÞÑΦ_V_xpvq ¨v is homogeneous of degree equal to 3. Then there existsǫą0 such that

pµ^``ΦVxpvqq ¨v “0 and }v} ďǫ ùñ v“0.

InYx, we still denote byxthe pointre,0,0s. We equipYxwith an invariant almost complex structure that is compatible with the symplectic structure,

(14)

and we denote by RRH_µ`pY_x,L_λ|Yx,ΦYx, H_µ`xq the Riemann-Roch character onYx localized on the componentH_µ`xĂ tκYx “0u.

The quotienth{h_µ^`, which is equipped with the invariant complex struc- tureJ_µ` :“adpµ^`qp´adpµ^`q²q^´¹^{², is a complex H_µ`-module.

In [9][Theorem 7.5], we proved that QHxpλq “ RRHpF,L_λ,Φ^H_r , Hxq is equal to

(3.5) Ind^H_H

µ`

´RR_H

µ`pYx,L_λ|Yx,Φ_Y_x, H_µ`xq bľ h{hµ^`

¯ .

3.1.2 Step 2: cotangent induction

The map Φxpvq:“µ^``ΦVxpvqis a moment map for the Hamiltonian action ofH_x on V_x. The moment map on theH_µ`-manifold

Y_x“H_µ`ˆHx

`ph_µ^`{hxq^˚ˆV_x˘ is Φ_Y_xprh;η, vsq “hpη`Φ_xpvqq Ph^˚_µ.

Let κ_V_xpvq “ ´Φxpvq ¨ v be the Kirwan vector field on V_x. We are interested in the connected component t0u of tκVx “ 0u. We choose a compatible almost complex structure on the symplectic vector space and we denote by RRHxpV_x,Φx,t0uq PRppH_xqthe Riemann-Roch character localized ont0u Ă tκVx “0u.

In Section 3.3 of [10] we have proved that (3.6)

RRH_µ_`pYx,L_λ|Yx,ΦYx, H_µ`xq “Ind^H_H^µ^`

x pRRHxpVx,Φx,t0uq bL_λ|xq. 3.1.3 Step 3: linear case

We write q{qµ^` for the vector space rq, µ^`s equipped with the complex structureJ_µ`. Soq{q_µ^` is aH_µ`-module and we denote by Sympq{q_µ^`qthe corresponding symmetric algebra.

We need to compare the virtual Hx-modules Ź

J_VxVx and Ź

´J_µ` Vx. The weight

δpxq:“ 1 2

ÿ

αPR` xXθpR`

xq

θpαq‰α

α

defines a character C_δ

pxq of the abelian group H_x. Recall that m_x P N corresponds to the quantity ¹₂|R^`_x XθpR^`_xq X tθpαq ‰αu| `dimE^nci_x .

The following lemma will be proved in Section 3.2.

(15)

Lemma 3.2 The following identity holds : ľ

J_Vx

Vx» p´1q^m^xC_δ

pxqbdetpE^nci_x q b ľ

´J_µ`

Vx.

On the vector space Vx, we can work with two localized Riemann-Roch characters:

• RR_H_xpVx,Φ_x,t0uqis defined with the complex structure J_V_x,

• ĄRRHxpV_x,Φx,t0uqis defined with the complex structure ´J_µ`. The previous Lemma gives that RRHxpV_x,Φx,t0uqis equal top´1q^m^xC_δ

pxqb ĄRRHxpV_x,Φx,t0uq.

Proposition 3.3 We have

(3.7) RRHxpVx,Φx,t0uq “ p´1q^m^xC_δ

pxqbdetpE^nci_x q bSympq{qµ^`q.

Proof. ForsP r0,1s, we consider the H_x-equivariant map Φ^s:V_xÑh^˚_x defined by the relations Φ^spvq “µ^``sΦVxpvq. The corresponding Kirwan vector field onV_x is κ^spvq “ ´Φ^spvq ¨v. It is not difficult to see that there existsǫą0 such thattκ^s“0u X t}v} ďǫu “ t0ufor any sP r0,1s. Then a simple deformation argument gives thatRRĄHxpV_x,Φ^s,t0uqdoes not depend onsP r0,1s. We have proved that

ĄRR_H_xpVx,Φ^s,t0uq “ĄRR_H_xpVx, µ^`,t0uq

where µ^` denotes the constant map Φ⁰. Standard computations gives ĄRRHxpVx, µ^`,t0uq “ Sympq{qµ^`q (see [9][Proposition 5.4]). Our proof is completed. l

3.1.4 Conclusion

If we use the formulas (3.5), (3.6) and (3.7) we obtain the following expression

Q_Hxpλq “ p´1q^m^xInd^H_H_x

˜ C_λ

x`δpxqbdetpE^nci_x q bSympq{qµ^`q bľ

C

h{hµ^`

¸

inRpHq. Herep C_λ

x is the character ofG_x associated to the weightλ_x “gλ.

The previous formula depends on a choice of a regular element r in the Weyl chamber. In the next section we will propose another expression for Q_Hxpλq that does not depend on this choice.

(16)

3.2 Another expression for QHxpλq

Let Rx Ă g^˚_x be the roots for the action of the torus G_x on gbC. The involution θ : t^˚ Ñ t^˚ leaves the set Rx invariant and a root α P Rx is called imaginary if θpαq “ α. We denote respectively by R^ci_x and by R^nci_x the subsets ofcompact imaginary andnon-compact imaginary roots.

We choose a generic element r Pt^˚_` such thatµ^`“ pg¨rq^` satisfies the following relation : for anyαPR_x, we have

pα, µ^`q “0ðñθpαq “ ´α.

Notice that an imaginary roots α is positive if and only if pα, µ^`q ą0.

Definition 3.4 We consider the subset Ax ĂRx

αPA_x ðñ αpµ^`q ą0, θpαq ‰α

The involutionθ defines a free action ofZ₂ on the setA_x. We denote by Ax{Z₂ its quotient. For anyα PRx, we denote by C_α the corresponding 1- dimensional representation ofGx, andC_α|Hx its restriction to the subgroup H_x. We have a natural maprαs PA_x{Z₂ ÞÝÑC_α|Hx PRpHxq.

For anyαPRx we define

˜ α“ ˘α where˘is the sign ofαpµqαpθpµqq.

We consider theH_x-modulesh{h_µ^` :“ prh, µ^`s, J_µ^`q,q{q_µ^` :“ pVx, J_µ`q andpV_x, J_V_xq.

Lemma 3.5 We have the following isomorphisms of H_x-modules h{hµ^` » à

rαsPAx{Z2

C_α|Hx‘ à

αPRci

xXR^`x

C_α|Hx rAs, q{qµ^` » à

rαsPAx{Z2

C_α|Hx‘ à

αPRnci

x XR^`x

C_α|Hx rBs, pV_x, J_V_xq » à

rαsPAx{Z2

C_α_˜|Hx‘ à

αPRnci

x XR^`x

C_α_˜|Hx rCs. Proof. Thanks to Lemma 2.9, we know that theHx-modulepVx, JVxqis isomorphic to the vector spacerq, µ^`sequipped with the complex structure J_V_x :“J_µ`˝S_x. We consider the vector spacesrq, µ^`sandrg, µ^`sequipped with the complex structure J_µ`. The projection (taking the real part) r : gbCÑg induces an isomorphism ofGx-modules

r: à

αpµ^`qą0

pgbCqα ÝÑ rg, µ^`s.

(17)

The orthogonal projections p1 : rg, µ^`s Ñ rq, µ^`s and p2 : rg, µ^`s Ñ rh, µ^`scommutes with theHx-action, so the maps

p1˝r: à

αpµ^`qą0

pgbCqα ÝÑ rq, µ^`s, p2˝r: à

αpµ^`qą0

pgbCqα ÝÑ rh, µ^`s

are surjective morphisms ofH_x-modules.

Let V_x¹pαq “ p1 ˝rppgbCqαq. We notice that dim^CV_x¹pαq P t0,1u:

V_x¹pαq “ t0u only ifα is a non-compact imaginary root andV_x¹pαq »C_α|Hx

whenV_x¹pαq ‰ t0u. We notice also that V_x¹pαq “V_x¹pθpαqq, hence q{qµ^` “ prq, µ^`s, J_µ`q » à

rαsPAx{Z2

V_x¹pαq ‘ à

αPRnci

x XR^`x

V_x¹pαq.

The identity rBsis proved.

Similarly we considerV_x²pαq “p2˝rppgbCqαq. We notice that dimCV_x²pαq P t0,1u: V_x²pαq “ t0u only if α is a compact imaginary root and V_x²pαq » C_α|Hx whenV_x²pαq ‰ t0u. We notice also that V_x²pαq “V_x²pθpαqq, hence

h{h_µ^` “ prh, µ^`s, J_µ^`q » à

rαsPAx{Z2

V_x²pαq ‘ à

αPRci

xXR^`x

V_x²pαq.

The identity rAsis proved.

Finally we check that the complex structuresJ_µ` and JVx preserve each V_x¹pαq and that pVxpαq, JVxq »C_α_˜|Hx when pα, µ^`q ą 0. The identity rCs follows. l

We consider theH_x-moduleV_x:“ř

rαsPAx{Z2C_α|Hx, and theG_x-modules E^nci

x :“ ř

αPRnci

x XR^`x

C_α and E^ci

x :“ ř

αPRci

xXR^`x

C_α. In the previous lemma we have proved thatH_x-modulesh{h_µ^` and q{q_µ^` are respectively isomorphic toV_x‘E^ci_x andV_x‘E^nci_x . If we use the fact that SympV_xq bŹV_x “1, we get the following corollary.

Corollary 3.6 We have the following identity of virtual Hx-modules:

Sympq{q_µ^`q bľ

h{h_µ^` »SympE^nci_x q bľ E^ci_x.

Let us now prove Lemma 3.2.

Let B :“ Ax{Z₂Ť

pR^nci_x XR^`_xq. We see that Ź

J_VxVx “ ś

αPBp1 ´ t^α^˜q whereas Ź

´J_µ_`V_x “ ś

αPBp1´t^´^αq. Accordingly we get Ź

J_VxV_x »

(18)

p´1q^|^B¹^|C_η bŹ

´J_µ_` V_x where B¹ “ tα PB,α˜ “αu and η “ř

αPB¹α. Now it is easy to check that an element αPB belongs to B¹ if and only if α and θpαq both belong toR^`_x. In other words

B¹“ αPR^`_x XθpR^`_xq, θpαq ‰α( {Z₂ď

R^nci_x XR^`_x. We have proved that

ľ

J_Vx

Vx» p´1q^m^xC_δ

pxqbdetpE^nci_x q b ľ

´J_µ`

Vx.

l

Finally, thanks to Lemma 3.2 and Corollary 3.6, we obtain the final formula forQ_Hxpλq (that does not depend on the choice ofr):

QHxpλq “ p´1q^m^xInd^H_H_x´ C_λ

x`δpxqbdetpE^nci_x q bSympE^nci_x q bľ E^ci_x

¯ .

3.3 Computation of the virtual module M_xpλq

According to Theorem 1.1, we have the decomposition V_λ^G|H “ ř

x¯Q¯xpλq where Q¯xpλq “ Ind^H_H_xpA_xpλqq, and A_xpλq P RppH_xq has the following description

A_xpλq “ 1

|W_x^H| ÿ

wPW

p´1q^m^xwC_λ

xw`δpxwqbdetpE^nci_xwq bSympE^nci_xwq bľ E^ci_xw.

The aim of this section is to simplify the expression of the virtual H_x- module A_xpλq. We start by comparing the G_x-modules E^nci_xw and E^nci_x . We use the decompositionE^nci_x “`

E^nci_x ˘_`

w‘` E^nci_x ˘_´

w where

`E^nci_x ˘_`

w :“ ÿ

αPR^nci_x XR^`xXR^`xw

C_α, and ` E^nci_x ˘_´

w “ ÿ

αPR^nci_x XR^`xX´R^`xw

C_α.

We have the following basic lemma (see Lemma 3.10).

Lemma 3.7 TheG_x-module |E^nci_x |w :“` E^nci_x ˘_`

w ‘ pE^nci_x q^´_w is isomorphic to E^nci_xw.

Letρ“ ¹2

ř

αPR^`α. We denote byw‚λ“wpλ`ρq ´ρthe affine action of the Weyl group on the lattice Λ.

The main result of this section is the following proposition.

Symmetric pairs and branching laws

HAL Id: hal-01818302

https://hal.archives-ouvertes.fr/hal-01818302

Symmetric pairs and branching laws

Paul-Emile Paradan

To cite this version:

Symmetric pairs and branching laws

Contents

1 Introduction

2 Non abelian localization

3 Proof of the main theorem