Compatibility of the theta correspondence with the Whittaker functors

Bulletin

SOCIÉTÉ MATHÉMATIQUE DE FRANCE

Publié avec le concours du Centre national de la recherche scientifique

de la SOCIÉTÉ MATHÉMATIQUE DE FRANCE

Tome 139 Fascicule 1

2011

pages 75-88

COMPATIBILITY OF THE THETA CORRESPONDENCE WITH THE

WHITTAKER FUNCTORS

Vincent Lafforgue & Sergey Lysenko

COMPATIBILITY OF THE THETA CORRESPONDENCE WITH THE WHITTAKER FUNCTORS

by Vincent Lafforgue & Sergey Lysenko

Abstract. — We prove that the global geometric theta-lifting functor for the dual pair (H, G) is compatible with the Whittaker functors, where (H, G) is one of the pairs(SO2n,Sp_2n),(Sp_2n,SO2n+2)or(GLn,GLn+1). That is, the composition of the theta-lifting functor fromH toGwith the Whittaker functor forGis isomorphic to the Whittaker functor forH.

Résumé(Compatibilité de la thêta-correspondence avec les foncteurs de Whittaker) Nous démontrons que le foncteur géométrique de théta-lifting pour la paire duale (H, G) est compatible avec la normalisation de Whittaker, où (H, G) est l’une des paires(SO2n,Sp_2n),(Sp_2n,SO2n+2)ou(GLn,GLn+1). Plus précisément, le composé du foncteur de théta-lifting de H vers G et du foncteur de Whittaker pour G est isomorphe au foncteur de Whittaker pourH.

We prove in this note that the global geometric theta lifting for the pair (H, G) is compatible with the Whittaker normalization, where (H, G) = (SO2n,Sp_2n), (Sp_2n,SO2n+2), or (GL_n,GL_n+1). More precisely, let k be an algebraically closed field of characteristicp >2. LetX be a smooth projective connected curve over k. For a stack S write D(S) for the derived category of étale constructible ¯

Q`-sheaves on S. For a reductive group G overk write

Texte reçu le 3 février 2009, accepté le 26 novembre 2009

Vincent Lafforgue, Laboratoire de Mathématiques - Analyse, Probabilités, Modélisation - Orléans (MAPMO), UMR CNRS 6628, Université d’Orléans, Rue de Chartres, B.P. 6759, 45067 Orléans Cedex 2, France

Sergey Lysenko, Institut Élie Cartan Nancy, Université Henri Poincaré Nancy 1, B.P. 239, F-54506 Vandœuvre-lès-Nancy Cedex, France

2000 Mathematics Subject Classification. — 11R39; 14H60.

Key words and phrases. — Geometric Langlands, Whittaker functor, theta-lifting.

BULLETIN DE LA SOCIÉTÉ MATHÉMATIQUE DE FRANCE 0037-9484/2011/75/$ 5.00

©Société Mathématique de France

BunG for the stack of G-torsors on X. The usual Whittaker distribution admits a natural geometrizationWhitG: D(BunG)→D(Speck).

We construct an isomorphism of functors betweenWhit_G◦F andWhit_H, whereF : D(BunH)→D(BunG)is the theta lifting functor (cf. Theorems 1, 2 and 3).

This result at the level of functions (on BunH(k) and BunG(k) when k is a finite field) is well known since a long time and the geometrization of the argument is straightforward. We wrote this note for the following reason.

Our proof holds also fork=Cin the setting ofD-modules. In this case for a reductive groupG, Beilinson and Drinfeld proposed a conjecture, which (in a form that should be made more precise) says that there exists an equivalenceαG

between the derived category ofD-modules onBunG and the derived category of

O

-modules onLocGˇ. HereLocGˇ is the stack ofG-local systems onˇ X, andGˇ is the Langlands dual group toG. Moreover,WhitGshould be the composition D(D-mod(BunG))^α→^GD(LocGˇ,

O

^{)RΓ→D(Spec}C).

A morphism γ : ˇH → Gˇ gives rise to the extension of scalars morphism

¯

γ: LocHˇ →LocGˇ. The functorγ¯_∗: D(LocHˇ,

O

^)→D(LocGˇ,

O

⁾should give rise to the Langlands functoriality functor

γL=α⁻¹_G ◦γ¯_∗◦αH : D(D-mod(BunH))→D(D-mod(BunG)) compatible with the action of Hecke functors.

In the cases (H, G) = (SO2n,Sp_2n), (Sp_2n,SO2n+2) or (GL_n,GL_n+1) the compatibility of the theta lifting functor F : D(D-mod(Bun_H)) → D(D-mod(BunG)) with the Hecke functors ([6]) and the compatibility of F with the Whittaker functors (proved in this paper) indicate that F should be the Langlands functoriality functor.

Notation. From now onk denotes an algebraically closed field of charac- teristicp >2, all the stacks we consider are defined overk. LetX be a smooth projective curve of genus g. Fix a prime ` 6= p and a non-trivial character ψ:Fp→Q¯<sup>∗</sup>`, and denote by

L

ψ the corresponding Artin-Schreier sheaf onA¹. Sincek is algebraically closed, we systematically ignore the Tate twists.

For a k-stack locally of finite type S write simply D(S) for the category introduced in ([3], Remark 3.21) and denotedD_c(S,¯

Q`)inloc.cit. It should be thought of as the unbounded derived category of constructible Q¯`-sheaves on S. For ∗ = +,−, b we have the full triangulated subcategory D^∗(S) ⊂ D(S) denoted D^∗_c(S,¯

Q`)in loc.cit. WriteD^∗(S)_!⊂D^∗(S)for the full subcategory of objects which are extensions by zero from some open substack of finite type.

Write D^≺(S) ⊂ D(S) for the full subcategory of complexes K ∈ D(S) such that for any open substack U ⊂S of finite type we have K|_U∈D⁻(U).

o

For any vector space (or bundle) E, we defineSym²(E)andΛ²(E)as quo- tients ofE⊗E (and denote byx.y andx∧y the images ofx⊗y) and we will use in this article the embeddings

Sym²(E)→ E⊗E and Λ²(E)→ E⊗E x.y 7→ ^x⊗y+y⊗x₂ x∧y 7→ ^{x⊗y−y⊗x}₂ (1)

1. Whittaker functors

Let Gbe a reductive group over k. We pick a maximal torus and a Borel subgroupT ⊂B ⊂Gand we denote by∆_G the set of simple roots ofG. The Whittaker functor

WhitG: D^≺(BunG)→D⁻(Speck)

is defined as follows. Write Ω for the canonical line bundle on X. Pick a T- torsor

F

T onXwith a trivial conductor, that is, for eachαˇ ∈∆Git is equipped with an isomorphism δ_αˇ :

L

^αˇ_FT ^{→ Ω. Here}

L

^αˇ_FT is the line bundle obtained from

F

T via extension of scalarsT →^αˇ Gm. LetBun_N^FT be the stack classifying a B-torsor

F

B together with an isomorphism

ζ:

F

B×_BTf→

F

T

Let : Bun_N^FT →A¹ be the evaluation map (cf. [1], 4.3.1 where it is denoted evω˜). Just recall that for each αˇ ∈ ∆G the class of the extension of

O

^{by Ω}

associated to

F

B, ζ and δαˇ givesαˇ : Bun_N^FT →A¹ and that =P

α∈∆ˇ Gαˇ. Write π: Bun_N^FT → BunG for the extension of scalars (

F

B, ζ)7→

F

B×BG.

SetP_ψ⁰=^∗

L

ψ[dN], wheredN = dim Bun_N^FT. Let dG = dim BunG. As in ([7], Definition 2) for

F

^∈D≺(BunG)set

Whit_G(

F

^{) = RΓ}c(Bun_N^FT, P_ψ⁰⊗π^∗(

F

^))[−dG] (2)

Remark 1. — The collection (

F

T,(δαˇ)α∈∆ˇ G) as above exists, because k is algebraically closed, and one can take

F

T = (√

Ω)^2ρ for some square root√ Ω of Ω. One has an exact sequence of abelian group schemes 1 → Z →T

Q_αˇ

→ G^∆m^G →1, where Z denotes the center ofG. So, two choices of the collection (

F

T,(δαˇ)α∈∆ˇ G)are related by a point ofBunZ(k)and the associated Whittaker functors are isomorphic up to the automophism of BunG given by tensoring with the correspondingZ-torsor.

Remark 2. — When

F

T is fixed, the functor Whit_G : D^≺(Bun_G) → D⁻(Speck)does not depend, up to isomorphism, on the choice of the isomor- phisms(δαˇ)α∈∆ˇ _G. That is, for any(λαˇ)α∈∆ˇ _G∈(k^∗)^∆G, the functors associated to (

F

T,(δ_αˇ)_{α∈∆ˇ G}) and (

F

T,(λ_αˇδ_αˇ)_{α∈∆ˇ G}) are isomorphic. Indeed, the two

diagrams BunG

←π Bun_N^FT → A¹ associated to (δαˇ)α∈∆ˇ G and (λαˇδαˇ)α∈∆ˇ G

are isomorphic for the following reason. Since k is algebraically closed, T(k)→(k^∗)^∆G is surjective. We pick any preimageγ∈T(k)of(λ_αˇ)_{α∈∆ˇ G} and get the automorphism(

F

B, ζ)7→(

F

B, γζ)ofBun_N^FT, which together with the idendity ofBunG andA¹intertwines the two diagrams.

1.1. Whittaker functor forGLn. — For i, j ∈ Zwith i ≤j we denote by

N

i,j

the stack classifying the extensions of Ωⁱ by Ωⁱ⁺¹ ... by Ω^j, i.e. classifying a vector bundle E_j−i+1 on X with a complete flag of vector subbundles 0 = E0 ⊂ E1 ⊂ ... ⊂ E_j−i+1 together with isomorphisms Ek+1/Ek ' Ω^j−k for k = 0, ..., j−i. Write i,j :

N

i,j → A¹ for the map given by the sum of the classes in Ext¹(

O

^,Ω)f→A¹ of the extensions 0 → Ek+1/Ek → Ek+2/Ek → E_k+2/E_k+1→0 fork= 0, . . . , j−i−1.

ForG=GLn, we consider the diagramBunn π0,n−1

←

N

0,n−1 0,n−1

→ A¹, where π_0,n−1 :

N

0,n−1 → Bun_n is (0 = E₀ ⊂ · · · ⊂ E_n) 7→ E_n. This diagram is isomorphic to the diagramBunG

←π Bun_N^FT → A¹ associated to the choice of

F

T whose image inBun_n isΩⁿ⁻¹⊕Ωⁿ⁻²⊕ · · · ⊕

O

^.

Therefore the functorWhit_GLn: D^≺(Bun_n)→D⁻(Speck)associated to the above choice of

F

T is given by

Whit_GL_n(

F

^{) = RΓ}c(

N

0,n−1, ^∗_0,n−1(

L

ψ)⊗π^∗_0,n−1(

F

^))[dim

N

0,n−1−dim Bunn].

Remark 3. — IfEis an irreducible ranknlocal system onXletAut_Ebe the corresponding automorphic sheaf onBunn (cf. [2]) normalized to be perverse.

Then AutE is equipped with a canonical isomorphism Whit_GL_n(AutE)f→Q¯`. This is our motivation for the above shift normalization in (2).

1.2. Whittaker functor forSp_2n. — Write Gn for the group scheme on X of automorphisms of

O

^{n ⊕Ωn} preserving the natural symplectic form ∧²(

O

^n⊕

Ωⁿ) → Ω. The stack BunG_n of Gn-torsors on X can be seen as the stack classifying vector bundlesM overXof rank2nequipped with a non-degenerate symplectic formΛ²M →Ω.

The diagram BunG_n π_Gn

←

N

G_{n Gn}

→ A¹ constructed in the next definition is isomorphic to the diagram BunG

←π Bun_N^FT → A¹ associated, for G=Gn, to the choice of

F

T whose image in Bun_Gn isL⊕L^∗⊗ΩwithL= Ωⁿ⊕Ωⁿ⁻¹⊕

· · · ⊕Ω (with the natural symplectic structure for which L and L^∗⊗Ω are lagrangians).

Definition 1. — Let

N

Gn be the stack classifying ((L₁, ..., L_n), E), where (0 =L0⊂L1⊂...⊂Ln)∈

N

1,n, andE is an extension of

O

X-modules

(3) 0→Sym²L_n→E→Ω→0

o

We associate to (3) an extension

(4) 0→Ln →M →L^∗_n⊗Ω→0

with M ∈BunG_n and Ln lagrangian as follows. EquipLn⊕L^∗_n⊗Ωwith the symplectic form (l, l^∗),(u, u^∗)7→ hl, u^∗i − hu, l^∗iforl, u∈L, l^∗, u^∗ ∈L^∗. Here h., .iis the canonical paring betweenL_nandL^∗_n. Using (1), we consider (3) as a torsor onX under the sheaf of symmetric morphismsL^∗_n⊗Ω→L_n. The latter sheaf acts naturally onLn⊕L^∗_n⊗Ωpreserving the symplectic form. ThenM is the twisting of Ln⊕L^∗_n⊗Ω by the above torsor. This defines a morphism π_Gn:

N

Gn→Bun_Gn.

Note that the extension of Ω by L_n⊗L_n obtained from (4) is the push- forward of (3) by the embeddingSym²L_n→L_n⊗L_n we have fixed in (1).

Let _Gn :

N

Gn → A¹ denote the sum of _1,n(L₁, ..., L_n) with the class in Ext(

O

^{,Ω) =}A¹ of the push-forward of (3) bySym²L_n →Sym²(L_n/L_n−1) = Ω².

The functor Whit_Gn : D^≺(Bun_Gn) → D⁻(Speck) associated to the above choice of

F

T is given by

WhitG_n(

F

^{) = RΓ}c(

N

G_n, ^∗_Gn(

L

ψ)⊗π_G^∗_n(

F

^))[dN(G_n)−dG_n] withd_N(Gn)= dim

N

Gn andd_Gn = dim Bun_Gn.

1.3. Whittaker functor for SO2n (first form). — Let H_n = SO2n.The stack BunH_n of Hn-torsors can be seen as the stack classifying vector bundles V over X equipped with a non-degenerate symmetric form Sym²V →

O

^{and a}

compatible trivializationdetVf→

O

^.

The diagram BunHn

π_Hn

←

N

Hn

_Hn

→ A¹ constructed in the next definition is isomorphic to the diagram Bun_G ←^π Bun_N^FT → A¹ associated, forG=H_n, to the choice of

F

T whose image inBunH_nisU⊕U^∗withU = Ωⁿ⁻¹⊕Ωⁿ⁻²⊕· · ·⊕

O

(with the natural symmetric structure for which U andU^∗ are isotropic).

Definition 2. — Let

N

H_n be the stack classifying ((U1, ..., Un), E), where (U₁, ..., U_n)∈

N

0,n−1 (i.e. we have a filtration0 =U₀ ⊂U₁ ⊂... ⊂U_n with Ui/U_i−1'Ωⁿ⁻ⁱ fori= 1, ..., n), andE is an extension of

O

X-modules

(5) 0→Λ²Un →E→

O

^→0

We associate to (5) an extension

(6) 0→Un→V →U_n^∗→0

withV ∈Bun_HnandU_nisotropic as follows. EquipU_n⊕U_n^∗with the symmetric form given by(u, u^∗),(v, v^∗)7→ hu, v^∗i+hv, u^∗iwithu, v∈Un, u^∗, v^∗∈U_n^∗. Us- ing (1), we consider (5) as a torsor under the sheaf of antisymmetric morphisms U_n^∗→U_n of

O

X-modules. This sheaf acts naturally onU_n⊕U_n^∗preserving the

symmetric form and the trivialization ofdet(Un⊕U_n^∗). Then (6) is the twisting ofUn⊕U_n^∗by the above torsor. This defines a morphismπHn:

N

Hn→BunHn. Note that the extension of

O

X byUn⊗Un obtained from (6) is the push- forward of (5) by the embeddingΛ²U_n→U_n⊗U_n fixed in (1).

For λ ∈ k^∗ let _Hn,λ :

N

Hn → A¹ be the sum of _0,n−1(U₁, ..., U_n) with λu, where u ∈ Ext(

O

^{,Ω) =} A¹ is the class of the push-forward of (5) by Λ²Un→Λ²(Un/Un−2) = Ω. SetH_n=H_n,1.

The functor Whit_Hn : D^≺(Bun_Hn)→ D⁻(Speck) associated to the above choice of

F

T sends

F

^∈D≺(BunH_n)to

(7) WhitH_n(

F

^{) = RΓ}c(

N

H_n, ^∗_Hn(

L

ψ)⊗π_H^∗_n(M))[dN(H_n)−dH_n]

withd_N(Hn)= dim

N

H_nanddH_n= dim BunH_n. By Remark2, if we replace in (7)H_n byH_n,λthen the functorWhitH_n gets replaced by an isomorphic one.

1.4. Whittaker functor forSO2n(second form)

Definition 3. — Let f

N

H_n be the stack classifying (V1 ⊂ · · · ⊂ Vn ⊂ V), where V ∈BunHn,Vn ⊂V is a subbundle,(V1, ..., Vn)∈

N

0,n−1 (i.e. we have a filtration 0 =V₀⊂V₁⊂...⊂V_n withV_i/V_i−1'Ωⁿ⁻ⁱ fori= 1, ..., n), and the composition

Sym²Vn→Sym²V →

O

coincides with Sym²Vn → Sym²(Vn/V_n−1) =

O

(in particular V_n−1 is isotropic).

The morphism eπ_Hn : f

N

Hn → Bun_Hn sends ((V₁, ..., V_n), V) to V. The morphisme_Hn :f

N

Hn→A¹is given bye_Hn((V₁, ..., V_n), V) =_0,n−1(V₁, ..., V_n).

Define a morphismκ:

N

H_n →f

N

H_n as follows. Let (U1, ..., Un), E)∈

N

H_n

and letV be as in Definition2. For i= 1, ..., n−1defineVi as the image ofUi

in V andV_2n−i as the orthogonal ofV_i inV. Then we have a filtration 0 =V0⊂V1⊂...⊂V_n−1⊂Vn+1⊂...⊂V_2n−1⊂V2n =V.

Recall that we have an identification U_n/U_n−1'

O

. The exact sequence0 → Un/U_n−1→Vn+1/V_n−1→Vn+1/Un→0admits a unique splittingssuch that the image of

O

^{= V}n+1/Un

→s Vn+1/Vn−1 is isotropic. Thus, Vn+1/Vn−1 is canonically identified with

O

^⊕

O

in such a way that the symmetric bilinear form Sym²(

O

^⊕

O

^)→

O

^becomes

(1,0).(1,0)7→0, (1,0).(0,1)7→1, (0,1).(0,1)7→0

Under this identification

O

^=Un/U_n−1→Vn+1/V_n−1=

O

^⊕

O

^{sends1to(1,0).}

Define Vn, equipped with

O

^{' V}n/V_n−1 by the property that

O

^'

Vn/Vn−1 ,→ Vn+1/Vn−1 sends 1 to (1,¹₂) ∈

O

^⊕

O

. The following is easy to check.

o

Lemma 1. — The map κ :

N

H_n → f

N

H_n is an isomorphism. There exists λ∈k^∗ such thate_Hn◦κ=_Hn,λ andeπ_Hn◦κ=π_Hn.

By Remark 2, if we replace in (7) H_n, πH_n by ˜H_n,π˜H_n then the functor WhitH_n gets replaced by an isomorphic one.

2. Main statements

WriteBun_n for the stack of ranknvector bundles onX. LetBun_Pn be the stack classifying L∈Bun_n and an exact sequence0 →Sym²L→?→Ω→0.

Remind the complex SP,ψ on BunP_n introduced in ([4], 5.2). Let

V

^{be the}

stack over Bunn whose fibre over L isHom(L,Ω). For

X

n =

V

^×BunnBunPn

letp:

X

n→Bun_Pn be the projection. Writeq:

X

n →A¹for the map sending s ∈ Hom(L,Ω) to the pairing of s⊗s ∈ Hom(Sym²L,Ω²) with the exact sequence 0→Sym²L→?→Ω→0. Letd_Xn be the “corrected” dimension of

X

n, i.e. the locally constant functiondim Bun_Pn−χ(L). Set SP,ψ =p!q^∗

L

ψ[d_Xn].

Let

A

be the line bundle onBunG_nwhose fibre atM isdet RΓ(X, M). Write BunfiG_nfor the gerb of square roots of

A

^andAutfor the theta-sheaf onBunfiG_n

([4], Definition 1). The projection νn : BunPn → BunGn lifts naturally to a map ˜ν_n: Bun_Pn →Bunfi_Gn. In what follows, we pick an isomorphism⁽¹⁾ (8) SP,ψf→ν˜_n^∗Aut[dim.rel(˜νn))]

provided by ([5], Proposition 1). Heredim.rel(˜ν_n)is the relative dimension of

˜

νn. The isomorphisms we construct below may depend on this choice.

2.1. FromSp_2n toSO2n+2. — Let F : D⁻(BunG_n)! → D^≺(BunH_n+1) be the theta lifting functor introduced in ([6], Definition 2).

Theorem 1. — The functorsWhit_Hn+1◦F andWhit_Gn fromD⁻(Bun_Gn)_! to D⁻(Speck)are isomorphic.

Let

X

be the stack classifying (M,(U₁, ..., U_n+1), E, s) with M ∈ Bun_Gn, (U1, ..., Un+1)∈

N

0,n (i.e. Uk+1/Uk = Ω^n−k for k= 0, ..., n),E an extension 0→Λ²Un+1→E→

O

^{→0, ands:U}n+1→M a morphism of

O

X-modules.

Letα_X :

X

^→BunG_n be the morphism(M,(U1, ..., Un+1), E, s)7→M. Let β_X :

X

^→A¹ be defined as follows. For(M,(U1, ..., Un+1), E, s)∈

X

^,

β_X(M,(U₁, ..., U_n+1), E, s) =_0,n(U₁, ..., U_n+1) +γ(E)− hE,Λ²si

(1)Once√

−1∈kis chosen, this isomorphism is well defined up to a sign.

where γ(E)is the pairing between the class of E in Ext(

O

^,Λ2Un+1) and the morphismΛ²Un+1→Λ²(Un+1/Un−1) = ΩandhE,Λ²siis the pairing between the class of E in Ext(

O

^,Λ2Un+1) and Λ²s : Λ²U_n+1 → Λ²M followed by Λ²M →Ω.

Let a_n =n(n+ 1)(1−g)(n− ¹₂), this is the dimension of the stack clas- sifying extension 0 → ∧²Un+1 →? →

O

^{→ 0 of}

O

X-modules for any fixed (U1, . . . , Un+1)∈

N

0,n.

Let dα_X denote the "corrected" relative dimension of α_X, that is, dα_X = an+dim

N

0,n+χ(U_n+1^∗ ⊗M)for anyk-pointsM ∈BunG_nand(U1, . . . , Un+1)∈

N

0,n. One checks that (8) yields for

F

^{∈ D−(Bun}Gn)_! an isomorphism in D⁻(Speck)

WhitH_n+1◦F(

F

⁾_f^→RΓc(

X

^{, α∗}_X⁽

F

^)⊗β∗_X⁽

L

ψ)[dα_X])

We will show later that Theorem 1is reduced to the following proposition.

Proposition 1. — There is an isomorphismα_X!(β^∗_X(

L

ψ)[2an])f→πG_n!^∗_G

n(

L

ψ) in D⁻(BunG_n)!.

The proposition is a consequence of the following lemmas. Let

Y

^{be the}

stack classifying (M,(U1, ..., Un+1), s)withM ∈BunG_n,(U1, ..., Un+1)∈

N

0,n

(i.e. Uk+1/Uk = Ω^n−k for k = 0, ..., n), and s : Un+1 → M a morphism such that the composition Λ²U_n+1 ^Λ

2s

→ Λ²M → Ω coincides with Λ²U_n+1 → Λ²(Un+1/U_n−1) = Ω.

Letα_Y :

Y

^→BunGnbe the morphism(M,(U₁, ..., U_n+1), s)7→M. Letβ_Y :

Y

^→A¹ be the map sending(M,(U1, ..., Un+1), s)∈

Y

^to0,n(U1, ..., Un+1).

Lemma 2. — There is an isomorphism α_X,!β^∗_X(

L

ψ) = α_Y,!β^∗_Y(

L

ψ)[−2an] in D⁻(Bun_Gn)_!.

For i ∈ {1, ..., n+ 1} let

Y

_i denote the open subset of

Y

given by the condition that the image of U_i by s is a subbundle of M. One has open immersions

Y

_n+1 ^⊂

Y

_n ^{⊂... ⊂}

Y

₁ ^⊂

Y

. Denote by α_Y

i :

Y

_i ^→BunG_n and β_Y

i :

Y

_i ^→A¹ the restrictions ofα_Y andβ_Y to

Y

_i^.

Lemma 3. — The natural maps α_Y

n+1,!β^∗_Y

n+1

(

L

ψ)→α_Y

n,!β^∗_Y

n

(

L

ψ)→... → α_Y

1,!β^∗_Y

1

(

L

ψ)→α_Y,!β^∗_Y(

L

ψ)are isomorphisms in D⁻(BunG_n)!.

Proof. — First, one has

Y

_n+1⁼

Y

_n−1thanks to the condition that the compo- sitionΛ²Un+1

Λ²s

→ Λ²M →Ωcoincides withΛ²Un+1→Λ²(Un+1/U_n−1) = Ω.

Write

Y

₀⁼

Y

^{. Leti}∈ {1, ..., n−1}. We are going to prove that the natural map

α_Y

i,!β^∗_Y

i(

L

ψ)→α_Y

i−1,!β^∗_Y

i−1(

L

ψ)

o

is an isomorphism. Set

Z

i =

Y

_i−1^\

Y

_i^{, let α}Zi andβ_Zi be the restrictions of α_Y

i−1 andβ_Y

i−1 to

Z

i. We must prove thatα_Zi,!β^∗_Z

i(

L

ψ) = 0.

Let

T

i be stack classifying (M,(U1, U2, ..., Ui), si) with M ∈ BunG_n, (U₁, U₂, ..., U_i)∈

N

n−i+1,n, s_i:U_i→M such that the restriction ofs_i toU_i−1 is injective and its image is a subbundle of M, but the image of s_i is not a subbundle of M of the same rank as Ui. The map α_Zi decomposes naturally as

Z

i

γ_Zi

→

T

i α_T

→i BunGn. It suffices to show that the∗-fibre ofγ_Zi,!β^∗_Zi(

L

ψ)at any closed point (M,(U₁, U₂, ..., U_i), s_i)∈

T

i vanishes.

The fiber

Q

^ofγZi over this point is the stack classifying((U1, ..., Un+1), s), where (U₁, ..., U_n+1) ∈

N

0,n extends (U₁, U₂, ..., U_i), s : U_n+1 → M extends s_i, and the composition Λ²U_n+1 ^Λ→^2s Λ²M → Ω coincides with Λ²U_n+1 → Λ²(Un+1/U_n−1) = Ω.

LetF denote the smallest subbundle ofM containings(U_i), its rank isior i−1. Let

R

be stack classifying((W1, ..., W_n+1−i), t)with(W1, ..., W_n+1−i)∈

N

0,n−i and t ∈Hom(Wn+1−i, M/F). There is a morphism ρ:

Q

^→

R

^which

sends ((U₁, ..., U_n+1), s) to ((U_i+1/U_i, ..., U_n+1/U_i),s)¯ where s¯ : U_n+1/U_i → M/F is the reduction ofs. Letβ_Q :

Q

^→A¹ be the restriction ofβ_Zi to

Q

^{. It}

suffices to show that ρ!β^∗_Q(

L

ψ) = 0.

Pick ((W1, ..., W_n+1−i), t)∈

R

^{, let}

S

be the fiber ofρover ((W1, ..., Wn+1−i), t).

Writeβ_S for the restriction ofβ_Q to

S

. We will show thatRΓ_c(

S

^{, β∗}_S⁽

L

ψ)) = 0.

If F is of rank i−1 then

S

identifies with the stack classifying extensions 0 → U_i/U_i−1 →? → U_n+1/U_i → 0 of

O

X-modules. Since β_S is a nontrivial character, we are done in this case.

If F is of rank i then

S

is a scheme with a free transitive action of Hom(Un+1/Ui, F/s(Ui)). Under the action of Hom(Un+1/Ui, F/s(Ui)), β_S changes by some character

Hom(U_n+1/U_i, F/s(U_i))→Hom(U_i+1/U_i, F/s(U_i))→^δ A¹.

If D = div(F/s(Ui)) then F/s(Ui)f→Ωⁿ⁻ⁱ⁺¹(D)/Ωⁿ⁻ⁱ⁺¹ naturally, and δ : H⁰(X,Ω(D)/Ω)→H¹(X,Ω)is the map induced by the short exact sequence 0→Ω→Ω(D)→Ω(D)/Ω→0, i.e. it is the sum of the residues. SinceD >0, δ is nontrivial, and we are done.

Lemma 4. — There is an isomorphism µ:

Y

n+1→

N

Gn such that π_Gn◦µ= α_Y

n+1 andG_n◦µ=β_Y

n+1.

It remains to show that Proposition 1 implies Theorem 1 . By the base change theorem we have

Whit_Gn(

F

⁾f→RΓ_c(

N

Gn, ^∗_G

n(

L

ψ)⊗π_G^∗

n(

F

^))[dN(G_n)−d_Gn]

f→RΓc(BunG_n, πG_n,!^∗_Gn(

L

ψ)⊗

F

^)[dN(Gn)−dG_n] and

RΓc(

X

^{, α∗}_X⁽

F

^)⊗βX(

L

ψ)[dα_X])f→RΓc(BunG_n, α_X,!(β^∗_X(

L

ψ)⊗

F

^[dα_X])).

It remains to prove dα_X −2an = d_N(Gn)−dG_n. This follows from dG_n =

−(1−g)n(2n+ 1),d_N(Gn)−dim

N

0,n= (1−g)(−n²+n(n+ 1)(n−¹₂)), and χ(U_n+1^∗ ⊗M) = (1−g)2n²(n+1)where(U₁, . . . , U_n+1)andM are closed points in

N

0,nandBunG_n.

2.2. FromSO2ntoSp_2n. — LetF : D⁻(Bun_Hn)_!→D^≺(Bun_Gn)be the Theta functor introduced in ([6], Definition 2).

Theorem 2. — The functors Whit_Gn◦F and Whit_Hn from D⁻(Bun_Hn)_! to D⁻(Speck)are isomorphic.

We use the same letters as in the last paragraph (with a different meaning), as the proof is very similar.

Let

X

be the stack classifying (V,(L₁, ..., L_n), E, s) with V ∈ Bun_Hn, (L1, ..., Ln) ∈

N

1,n (i.e. Lk+1/Lk = Ω^n−k for k = 0, ..., n−1), an extension 0→Sym²Ln→E→Ω→0 of

O

X-modules, and a sections:Ln→V ⊗Ω.

Let α_X :

X

^{→ Bun}Hn be the morphism (V,(L1, ..., Ln), E, s) 7→ V. Let β_X :

X

^→A¹ be the map sending(V,(L₁, ..., L_n), E, s)∈

X

^to

_1,n(L₁, ..., L_n) +γ(E)− hE,Sym²si,

where γ(E)is the pairing between the class ofE inExt¹(Ω,Sym²Ln)and the map Sym²L_n→Sym²(L_n/L_n−1) = Ω²;hE,Sym²siis the pairing between the class of E in Ext¹(Ω,Sym²Ln)and Sym²s: Sym²Ln →Sym²V ⊗Ω² followed bySym²V →

O

^.

Let b_n = −χ(Ω⁻¹⊗Sym²L_n) for any k-point (L₁, . . . , L_n) ∈

N

1,n. Write dα_X for the "corrected" relative dimension ofα_X, that is,

dα_X = dim

N

1,n+bn+χ(L^∗_n⊗V ⊗Ω)

for any k-points (L₁, . . . , L_n) ∈

N

1,n and V ∈ Bun_Hn. One checks that (8) yields for

F

^∈D−(BunH_n)! an isomorphism inD⁻(Speck)

WhitG_n◦F(

F

^{) = RΓ}c(

X

^{, α∗}_X⁽

F

^)⊗β∗_X⁽

L

ψ))[dα_X] We will derive Theorem2 from the following proposition.

Proposition 2. — There is an isomorphismα_X,!β^∗_X(

L

ψ)[2b_n]'eπ_Hn,!e^∗_H

n(

L

ψ) in D⁻(Bun_Hn)_!.

o

Proposition 2 is reduced to the following lemmas. Let

Y

be the stack classifying (V,(L1, ..., Ln), s) with V ∈ BunSO2n, (L1, ..., Ln) ∈

N

1,n (i.e., L_k+1/L_k = Ω^n−k for k = 0, ..., n−1) and s : L_n → V ⊗Ω a morphism such that the composition Sym²Ln

Sym²s

→ (Sym²V)⊗Ω²→Ω²coincides with Sym²Ln→Sym²(Ln/Ln−1) = Ω²

Let α_Y :

Y

^→BunHn be the map (V,(L₁, ..., L_n), s)7→V. Let β_Y :

Y

^→A¹ be the map sending (V,(L1, ..., Ln), s)∈

Y

^to 1,n(L1, ..., Ln).

Lemma 5. — There is an isomorphism α_X,!β^∗_X(

L

ψ)f→α_Y,!β^∗_Y(

L

ψ)[−2bn] in D⁻(BunH_n)!.

Fori∈ {1, ..., n}let

Y

_i^⊂

Y

be the open substack given by the condition that s(L_i)⊂V⊗Ωis a subbundle of ranki. We have inclusions

Y

_n ^⊂

Y

_n−1^⊂...⊂

Y

1⊂

Y

. Denote byα_Y

i :

Y

i →Bun_Hn and β_Y

i :

Y

i →A¹ the restrictions of α_Y andβ_Y to

Y

i.

As in Lemma3, one proves Lemma 6. — The natural maps α_Y

n,!β^∗_Y

n

(

L

ψ)→α_Y

n−1,!β^∗_Y

n−1

(

L

ψ)→... → α_Y

1,!β^∗_Y

1(

L

ψ)→α_Y,!β^∗_Y(

L

ψ)are isomorphisms in D⁻(BunH_n)!.

Lemma 7. — There is an isomorphismµ:

Y

_n^→f

N

SO2n such thatπeSO2n◦µ= α_Y

n ande_SO2n◦µ=β_Y

n.

Theorem2follows from Proposition2becausedα_X−2bn=d_N(Hn)−dH_n. Let us just indicate thatd_N(Hn)−dim

N

1,n= (1−g)n(n−1)(n−³₂),χ(L^∗_n⊗V⊗Ω) = (1−g)2n³,bn= (1−g)n(n+ 1)(n−¹₂)anddH_n=−(1−g)n(2n−1), where (L1, . . . , Ln)andV are closed points in

N

1,nandBunHn.

2.3. FromGL_ntoGL_n+1. — LetF : D⁻(Bun_n)_! →D^≺(Bun_n+1)be the com- position of the direct image by Bunn →Bunn,L7→L^∗ and the theta functor Fn,n+1 : D⁻(Bunn)! → D^≺(Bunn+1) introduced in ([6], Definition 3). It is a consequence of Theorem 5 in [6] that F is compatible with Hecke functors according to the morphism of dual groupsGLn→GLn+1,A7→ A0

0 1

! . Let us recall the definition of F. Denote

W

be the classifying stack of (L, U, s) with L ∈ Bun_n, U ∈ Bun_n+1 and s : L → U a morphism. We have (hn, hn+1) :

W

^{→ Bun}n×Bunn+1,(L, U, s) 7→ (L, U). Then for

F

^∈

D⁻(Bunn)!,

F(

F

^{) =h}n+1,!((h^∗_n

F

^{)[dim Bun}n+1+χ(L^∗⊗U)]),

whereχ(L^∗⊗U)is considered as a locally constant function onBun_n×Bun_n+1.

Theorem 3. — The functors Whit_GL_n+1◦F and Whit_GL_n from D⁻(Bunn)!

toD⁻(Speck)are isomorphic.

Let

X

be the stack classifying L ∈ Bunn, (U1, . . . , Un+1) ∈

N

0,n, and s : L → U_n+1 a morphism. We have α_X :

X

^{→ Bun}n and β_X :

X

^→ A¹ which send(L,(U1, . . . Un+1), s)toLand 0,n(U1, . . . Un+1).

We have

Whit_GL_n+1◦F(

F

^{) = RΓ}c(Bunn,

F

^⊗αX,!β^∗_X(

L

ψ)[dim

N

0,n+χ(L^∗⊗Un+1)]) and Whit_GL_n(

F

^{) = RΓ}c(Bunn,

F

^⊗(π0,n−1)!^∗_0,n−1(

L

ψ)[dim

N

0,n−1−dim Bunn]).

For i ∈ {0, . . . , n} denote by

X

i the open substack of

X

classifying (L,(U1, . . . Un+1), s) such that the composition L →^s Un+1 → Un+1/U_n+1−i is surjective. We have

X

⁼

X

0 ⊃

X

1 ⊃ · · · ⊃

X

n and we have an isomor- phism

N

0,n−1→

X

n which sends(E₁, . . . , E_n)to (E_n,(Ωⁿ,Ωⁿ⊕E₁, . . . ,Ωⁿ⊕ En),(0,Id))with(0,Id) :En→Ωⁿ⊕En the obvious inclusion.

It is easy to compute that for L = E_n with (E₁, . . . , E_n) ∈

N

0,n−1 and (U1, . . . , Un+1) ∈

N

0,n we have dim

N

0,n+χ(L^∗ ⊗Un+1) = dim

N

0,n−1 − dim Bunn.

Therefore we are reduced to the following lemma. We denote byα_Xi :

X

i→ Bun_n andβ_Xi :

X

i→A¹ the restrictions ofα_X andβ_X to

X

i.

Lemma 8. — The natural maps α_Xn,!β^∗_X

n(

L

ψ)→α_Xn−1,!β^∗_X

n−1(

L

ψ)→... → α_X1,!β^∗_X

1(

L

ψ)→α_X,!β^∗_X(

L

ψ)are isomorphisms in D⁻(Bunn)!.

Proof. — We recall that

X

⁼

X

0. Let i ∈ {1, ..., n}. We are going to prove that the natural map

α_Xi,!β^∗_X

i(

L

ψ)→α_Xi−1,!β^∗_Xi−1(

L

ψ)

is an isomorphism. Set

Z

i =

X

i−1\

X

i, let α_Zi andβ_Zi be the restrictions of α_Xi−1 andβ_Xi−1 to

Z

i. We must prove thatα_Zi,!β^∗_Zi(

L

ψ) = 0.

Let

T

i be stack classifying (L,(V1, V2, ..., Vi), t) with L ∈ Bunn, (V₁, V₂, ..., V_i) ∈

N

0,i−1, t : L → V_i such that the composition L →^t Vi → Vi/V1 is surjective but t is not surjective. The map α_Zi decom- poses naturally as

Z

i

γ_Zi

→

T

i α_T

→i Bunn where γ_Zi(L,(U1, . . . Un+1), s) = (L,(U_n+2−i/U_n+1−i, . . . , U_n+1/U_n+1−i),s)¯ and α_Ti(L,(U₁, . . . U_n+1), s) = L.

It suffices to show that the ∗-fibre of γ_Zi,!β^∗_Z

i(

L

ψ) at any closed point (L,(V1, V2, ..., Vi), t)∈

T

i vanishes.

Let us choose a closed point(L,(V1, V2, ..., Vi), t)∈

T

iand defineL⁰⁰= Kert andL⁰ the kernel of the compositionL→^t Vi→Vi/V1. ThenL⁰is a subbundle ofL of rankn+ 1−iandL⁰⁰ is a subbundle ofLof rankn+ 1−iorn−i.

o