Truncated microsupport and holomorphic solutions of <span class="mathjax-formula">$D$</span>-modules

4^esérie, t. 36, 2003, p. 583 à 599.

TRUNCATED MICROSUPPORT AND

HOLOMORPHIC SOLUTIONS OF D -MODULES

B

M

KASHIWARA, T

MONTEIRO FERNANDES

P

SCHAPIRA

ABSTRACT. – We study the truncated microsupport SSk of sheaves on a real manifold. Applying our results to the case of F =RHomD(M,O), the complex of holomorphic solutions of a coherent D-moduleM, we show thatSSk(F)is completely determined by the characteristic variety ofM. As an application, we obtain an extension theorem for the sections ofH^j(F), j < d, defined on an open subset whose boundary is non-characteristic outside of a complex analytic subvariety of codimension d.

We also give a characterization of the perversity forC-constructible sheaves in terms of their truncated microsupports.

2003 Éditions scientifiques et médicales Elsevier SAS

RÉSUMÉ. – Nous étudions le micro-support tronquéSSkdes faisceaux sur une variété réelle. Appliquant nos résultats au cas du complexe F =RHom_D(M,O) des solutions holomorphes d’un D-module cohérentM, nous montrons queSSk(F)est complètement determiné par la variété caractéristique deM. Comme application, nous obtenons un théorème d’extension pour les sections deH^j(F),j < d, definies sur un ouvert dont la frontière est non caractéristique en dehors d’un ensemble analytique complexe de codimensiond. Nous donnons aussi une caractérisation de la perversité des faisceauxC-constructibles en terme de leurs micro-supports tronqués.

2003 Éditions scientifiques et médicales Elsevier SAS

1. Introduction

The notion of microsupport of sheaves was introduced in the course of the study of the theory of linear partial differential equations (LPDE), and it is now applied in various domains of mathematics. References are made to [8].

For an object F of the derived category of abelian sheaves on a real manifold X, its microsupport SS(F) is a closed conic subset of the cotangent bundle π:T^∗X →X which describes the direction of “non-propagation” ofF. In particular, for a smooth closed submanifold Y of X, the support Supp(µY(F)) of the Sato microlocalization µY(F) of F along Y is contained inSS(F)∩T_Y^∗X, whereT_Y^∗X denotes the conormal bundle toY inX.

If X is a complex manifold andM is a system of LPDE, that is, a coherent module over the sheafDX of holomorphic differential operators, the complexFof holomorphic solutions of this system is given byRHom_DX(M,OX), and the microsupport ofFis then the characteristic varietyCh(M)ofM.

1The research of author was supported by FCT and Programa Ciência, Tecnologia e Inovação do Quadro Comunitário de Apoio.

ANNALES SCIENTIFIQUES DE L’ÉCOLE NORMALE SUPÉRIEURE

However some phenomena of propagation may happen in specific degrees, related to the principle of unique continuation for holomorphic functions, and this leads to a variant of the notion of microsupport, that of “truncated microsupport”, a notion introduced by the authors of [8] but never published.

Following a suggestion of these authors, Tonin [12] was able to regain in the language of the truncated microsupport a result of Ebenfelt, Khavinson and Shapiro [2,3]. In their papers, these authors obtained the extension of holomorphic solutions of a differential operator when the solutions are defined on an open subset with smooth boundary non-characteristic outside a smooth complex hypersurface. Note that the problem of extending holomorphic solutions across non-characteristic real hypersurfaces plays a crucial role in the theory of LPDE, and was initiated by Leray [9], followed by [13,1,5] (see also [4] and [10] for an exposition of these results).

The truncated microsupport is defined as follows. Letkbe a field, letX be a real manifold and letF∈D^b(kX)be an object of the derived category of sheaves ofk-vector spaces onX.

For an integerk∈Z, a pointp∈T^∗Xdoes not belong to the truncated microsupportSSk(F)if and only ifF is microlocally atpisomorphic to an object ofD^>k(kX). Hence{SSk(F)}k∈Z

is an increasing sequence, whileSSk(F)is an empty set fork0andSSk(F)coincides with SS(F)fork0.

In this paper, we give equivalent definitions of the truncated microsupport and we study its behavior under exterior tensor product, smooth inverse image and proper direct image.

We introduce then the closed subset SS^Y_k(F) ofπ⁻¹(Y)which describes the support of the microlocalization ofF along submanifolds ofY and prove (see Theorem 5.3):

SSk(F) = SSk(F)\π⁻¹(Y)∪SS^Y_k(F).

We then apply this result to the complexF =RHom_DX(M,OX)of holomorphic solutions of a coherentDX-moduleM on a complex manifoldX.

LetSbe a closed complex analytic subset of codimension greater than or equal todand letS be a closed complex analytic subset ofSof codimension greater thandsuch thatS0:=S\Sis a smooth submanifold of codimensiond. We first prove the estimate below (see Proposition 6.2):

SSd−1(F) = SSd−1(F)\π⁻¹(S),

SSd(F) = SSd(F)\π⁻¹(S)∪SSd(F)∩T_S^∗₀X.

In particular, if j: Ω→X is the embedding of a pseudo-convex open subset with smooth boundary∂Ω, and if∂Ωis transversal toS(i.e.T_∂Ω^∗ X∩T_S^∗₀X⊂T_X^∗X) and non-characteristic forM outside ofS, one has

Ext^j_DX

M,O_X⁺/OX

= 0 for anyj < d,

whereOX⁺/OX=j_∗j⁻¹OX/OX.

Next we calculate SSk(F) in terms of the characteristic variety ofM (see Theorem 6.7).

LettingCh(M) =

α∈AVαbe the decomposition ofCh(M)into irreducible components, one has

SSk(F) =

codimπ(V_α)<k

Vα

∪

codimπ(V_α)=k

T_π(V^∗ _α)X

. (1.1)

In particular, ifF is a perverse sheaf (i.e.,M is holonomic), then lettingSS(F) =

α∈AΛαbe the decomposition into irreducible components, one has

SSk(F) =

codimπ(Λ_α)k

Λα.

Conversely ifF∈D^b(CX)isC-constructible and if it satisfies SSk(F)∪SSk

RHom(F,CX)

⊂

codimπ(Λ_α)k

Λα for everyk,

thenF is a perverse sheaf.

2. Notations and review We will mainly follow the notations in [8].

LetX be a real analytic manifold. We denote byτ:T X→X the tangent bundle toX and byπ:T^∗X→X the cotangent bundle. We identifyX with the zero section ofT^∗X and set T˙^∗X =T^∗X\X. We denote by π˙: ˙T^∗X →X the restriction of π to T˙^∗X. For a smooth submanifoldY ofX,TYX denotes the normal bundle toY and T_Y^∗X the conormal bundle.

In particular,T_X^∗Xis identified withX.

For a submanifoldY ofX and a subsetS ofX, we denote byCY(S)the normal cone toS alongY, a closed conic subset ofTYX.

For a morphismf:X→Y of real manifolds, we denote by

fπ:X×Y T^∗Y →T^∗Y and fd:X×Y T^∗Y →T^∗X the associated morphisms.

For a subsetAofT^∗X, we denote byA^athe image ofAby the antipodal map a: (x;ξ)→(x;−ξ).

The closure ofAis denoted byA. For a coneγ⊂T X, the polar coneγ^◦toγis the convex cone inT^∗X defined by

γ^◦=

(x;ξ)∈T^∗X; x∈π(γ)andv, ξ0for any(x;v)∈γ .

Letkbe a field. We denote by D(kX)the derived category of complexes of sheaves ofk- vector spaces onX, and byD^b(kX)the full subcategory ofD(kX)consisting of complexes with bounded cohomologies.

Fork∈Z, we denote as usual byD^k(kX)(resp.D^k(kX)) the full additive subcategory of D^b(kX)consisting of objectsF satisfyingH^j(F) = 0for anyj < k(resp.H^j(F) = 0for any j > k). The categoryD^k+1(kX)is sometimes denoted byD^>k(kX).

We denote by τ^k:D(kX) → D^k(kX) the truncation functor. Recall that for F ∈ D(kX)the morphismτ^kF →F induces isomorphismsH^j(τ^kF)→^∼ H^j(F)forjk and H^j(τ^kF) = 0forj > k.

IfF is an object ofD^b(kX),SS(F)denotes its microsupport, a closedR⁺-conic involutive subset of T^∗X. For p∈T^∗X, D^b(kX;p) denotes the localization of D^b(kX) by the full triangulated subcategory consisting of objectsFsuch thatp /∈SS(F).

IfY is a submanifold,µY(F)denotes the Sato microlocalization ofF alongY. Recall that µY(F)∈D^b(kT_Y^∗X)and

H^j µY(F)

plim

−→

Z

H_Z^j(F)π(p) forp∈T_Y^∗Xandj∈Z, (2.1)

whereZruns through the family of closed subsets ofX such that CY(Z)π(p)\ {0} ⊂

v∈(TYX)π(p); v, p>0 . (2.2)

On a complex manifoldX, we consider the sheafOXof holomorphic functions and the sheafDX

of linear holomorphic differential operators of finite order. Concerning the theory ofD-modules, references are made to [6].

3. Truncated microsupport

We shall give here several equivalent definitions of the truncated microsupport.

For a closed coneγ⊂Rⁿ, one sets Zγ:=

(x;y)∈Rⁿ×Rⁿ; x−y∈γ .

Letq1, q2:Rⁿ×Rⁿ→Rⁿbe the first and the second projections.

One defines the integral transform with kernelk_Zγ,

kZ_γ◦:D^b(k_Rⁿ)→D^b(k_Rⁿ), kZ_γ◦G=Rq_1!

kZ_γ⊗q⁻₂¹G .

IfGhas compact support, one has the following formula for the stalk ofkZ_γ◦Gatx∈Rⁿ: (kZ_γ◦G)xRΓ

Rⁿ; kx+γ^a⊗G .

Recall that a closed convex coneγis called proper if0∈γandInt(γ^◦)=∅. For(x0;ξ0)∈Rⁿ×(Rⁿ)^∗andε∈Rwe set:

Hε(x0, ξ0) =

x∈Rⁿ; x−x0, ξ0>−ε .

If there is no risk of confusion, we writeHεinstead ofHε(x0, ξ0)for short. The following result is proved in [8].

LEMMA 3.1. – LetXbe an open subset ofRⁿand letG∈D^b(kX). Letp= (x0;ξ0)∈T^∗X.

Thenp /∈SS(G)if and only if there exist an open neighborhoodWofx0, a proper closed convex coneγandε >0such thatξ0∈Int(γ^◦),(W +γ^a)∩Hε⊂X and

H^j(X;k_(x+γa)∩H_ε⊗G) = 0 for anyj∈Zandx∈W.

We shall give a similar version of the above lemma for the truncated microsupport.

PROPOSITION 3.2. – LetX be a real analytic manifold and letp∈T^∗X. LetF∈D^b(kX) andk∈Z,α∈Z1∪ {∞, ω}. Then the following conditions are equivalent:

(i)k There existF∈D^>k(kX)and an isomorphismFFinD^b(kX;p).

(ii)k There exist F ∈D^>k(kX) and a morphism F →F in D^b(kX) which is an isomorphism inD^b(kX;p).

(iii)k,α There exists an open conic neighborhoodU ofpsuch that for anyx∈π(U)and for anyR-valued C^α-function ϕdefined on a neighborhood of xsuch that ϕ(x) = 0, dϕ(x)∈U, one has

H_{^j_ϕ0}(F)x= 0 for anyjk.

(3.1)

WhenX is an open subset ofRⁿandp= (x0;ξ0), the above conditions are also equivalent to (iv)k There exist a proper closed convex coneγ⊂Rⁿ,ε >0and an open neighborhoodW

ofx0withξ0∈Int(γ^◦)such that(W+γ^a)∩Hε⊂X and

H^j(X;k_(x+γa)∩Hε⊗F) = 0 for anyjkandx∈W.

(3.2)

Proof. – We may assumeX=Rⁿ. (ii)k⇒(i)kis obvious.

(i)k⇒(iv)kBy the hypothesis, there exist distinguished triangles G→F→K−→⁺¹ and G→F→K−→⁺¹

in D^b(kX) such that p /∈SS(K) and p /∈ SS(K). By Lemma 3.1, there exist an open neighborhoodW ofx0, a proper closed convex coneγsuch thatξ0∈Int(γ^◦), andε >0such that

H^j(X;k_(x+γa)∩Hε⊗K) =H^j(X;k_(x+γa)∩Hε⊗K) = 0 for anyj∈Zandx∈W. Hence one has

H^j(X;k_(x+γa)∩H_ε⊗F)H^j(X;k_(x+γa)∩H_ε⊗F).

Sincek_(x+γa)∩H_ε⊗Fbelongs toD^>k(kX), we get (3.2).

(i)k ⇒ (iii)k,1 Same proof as (i)k ⇒ (iv)k, replacing H^j(X;k_(x+γa)∩Hε ⊗ G) with H_{^j_ϕ0}(G)xwhereG=F,F,K,K.

(iii)_k,1⇒(iii)_k,ωis obvious.

(iv)k⇒(ii)kTo start with, note that (3.2) entails

(kZγ◦FHε)W ∈D^>k(kX).

(3.3)

Let∆denote the diagonal ofX×X. Then the morphismk_Zγ→k_∆induces the morphism inD^b(kX)

kZ_γ◦FH_ε→FH_ε, (3.4)

which is an isomorphism inD^b(kX;p)by [8, Theorem 7.1.2]. Therefore, the composition (kZγ◦FHε)W→(FHε)W →F

is an isomorphism inD^b(kX;p)and(kZγ◦FHε)W belongs toD^>k(kX;p).

(iii)k,ω⇒(iv)k We already know that (iv)k is equivalent to (i)k for everyk. Hence arguing by induction onk, we may assume that (i)k−1holds. Therefore we may assumeF∈D^k(kX).

Then we have

H^j(X;k_(x+γa)∩Hε⊗F) = 0 for anyjk−1

and

H^k(X;k_(x+γa)∩Hε⊗F)H⁰

X;k_(x+γa)∩Hε⊗H^k(F) , H_{^k_ϕ0}(F)Γ_{ϕ0}

H^k(F) .

We may assume that Int(γ)=∅, W ×(γ^◦ \ {0})⊂U and (W +γ^a)∩Hε ⊂W. Let s∈Γ(X;k_(x+γa)∩Hε⊗H^k(F)). Then there existsy∈Rⁿsuch that

x+γ^a⊂y+ Int(γ^a)⊂W∪(X\Hε) andsextends to a section

˜ s∈Γ

y+ Int(γ^a);kH_ε⊗H^k(F)

⊂Γ

y+ Int(γ^a);H^k(F) .

Set S= supp(˜s)⊂Hε∩(y+ Int(γ^a)). Then the following lemma asserts S=∅, and hence H^k(X;k(x+γ^a)∩H_ε⊗F) = 0. ✷

LEMMA 3.3. – Letγbe a proper closed convex cone inRⁿ. LetΩbe an open subset ofRⁿ such thatΩ +γ^a= Ω, and letSbe a closed subset ofΩsuch thatSRⁿ. Assume the following condition: for anyx∈Rⁿand any real analytic functionϕdefined onRⁿ, the three conditions S∩ϕ⁻¹(R<0) =∅,ϕ(x) = 0anddϕ(x)∈Int(γ^◦)implyx /∈S.

ThenSis an empty set.

Proof. – Ifγ={0}, then by takingϕ= 0, the lemma is trivially true. Hence we may assume that{0}γ. Let us takeξsuch thatγ\ {0} ⊂ {x; x, ξ>0}. Then there is a real numbera such thatS⊂ {x; x, ξ> a}. SetH₋={x; x, ξ< a}. By replacingΩwithΩ∪H₋, we may assume from the beginningH₋⊂Ω.

For a proper closed convex coneγsuch thatγ\ {0} ⊂Int(γ)andγ\ {0} ⊂ {x; x, ξ>0}, set Ωγ ={x∈Ω; x+γ^a∈Ω}. Since H₋⊂Ω, Ωγ is an open subset andΩ =

γΩγ. Set Sγ =S∩Ωγ. Since S =

γSγ, it is enough to show the assertion for Sγ. Since γ^◦\ {0} ⊂Int(γ^◦), by replacingΩ, S, γ with Ωγ, Sγ and γ, we may assume from the beginning

Int(γ)=∅, (3.5)

S∩ϕ⁻¹(R<0) =∅, ϕ(x) = 0 and dϕ(x)∈γ^◦\ {0} ⇒ x /∈S.

(3.6)

Let us setψ(x) = dist(x, γ^a) := inf{y−x; y∈γ^a}. It is well known thatψis a continuous function onRⁿ, andC¹onRⁿ\γ^a. More precisely for anyx∈Rⁿ\γ^a, there exists a unique y∈γ^asuch thatψ(x) =x−y. Moreoverdψ(x) =x−y⁻¹(x−y)∈γ^◦\ {0}. Furthermore Bψ(x)(y) :={z∈Rⁿ; z−y< ψ(x)}is contained in{z∈Rⁿ; ψ(z)< ψ(x)}.

For ε >0, we set γε^a={x∈Rⁿ; ψ(x)< ε}. Then γε^a is an open convex set. Moreover γε^a+γ^a=γε^a. SetΩε={x;x+γε^a⊂Ω}. ThenΩ =

ε>0Ωε. SetSε=S∩Ωε. It is enough to show thatSε=∅.

AssumingSε=∅, we shall derive a contradiction. Let us takex0∈Sε andv∈Int(γ). Set Vt=x0+γ^a_ε/2+tvfort∈R. Then one has

Vt=

t<t

Vt and Vt=

t>t

Vt=

t>t

Vt, (3.7)

x0∈Vt∩S fort0, and Vt⊂H₋fort0, (3.8)

Vt⊂Ω for anyt0.

(3.9)

Hence, for any compact set K and t ∈R such K∩Vt =∅, there exists t > t such that K∩Vt=∅.

Let us set

c= sup{t;Vt∩S=∅}.

Thenc0andVc∩S=∅. By (3.9), one hasVc∩S⊂Vc∩S. SinceSis a compact set, there existsx1∈S∩∂Vc. Here∂Vc:=Vc\Vc is the boundary ofVc. As seen before, there exists a ballBε/2(y) :={x;x−y< ε/2}such thatBε/2(y)⊂Vc,x1−y=ε/2andx1−y∈γ^◦. This is a contradiction by takingϕ(x) =x−y²−(ε/2)². ✷

DEFINITION 3.4. – (i) LetF∈D^b(kX). The closed conic subsetSSk(F)ofT^∗Xis defined by:p /∈SSk(F)if and only ifFsatisfies the equivalent conditions in Proposition 3.2.

(ii) Let p∈T^∗X and k ∈Z. Then D^>k(kX;p) denotes the full additive subcategory of D^b(kX;p) consisting of F satisfying p /∈SSk(F). We write sometimes D^k(kX;p) for D^>k−1(kX;p).

Note thatSSk(F)∩T_X^∗X=π(SSk(F)) = Supp(τ^kF).

Remark 3.5. – The truncated microsupport has the following properties, similarly to those of the microsupport.

(i) For anyF∈D^b(kX), one hasSSk(F[n]) = SSk+n(F).

(ii) IfF→F→F−→⁺¹ is a distinguished triangle, then one has SSk(F)⊂SSk(F)∪SSk(F),

SSk(F)\SSk−1(F)

∪

SSk(F)\SSk+1(F)

⊂SSk(F).

(3.10)

(iii) For anyF∈D^b(kX), one has

SSk(F) = SSk(τ^kF).

(3.11)

Indeed, one has a distinguished triangleτ^kF→F→τ^>kF−→⁺¹. Then (ii) implies SSk(F)⊂SSk(τ^kF)∪SSk(τ^>kF) and

SSk(τ^kF)\SSk−1(τ^>kF)⊂SSk(F).

Hence the assertion follows fromSSk−1(τ^>kF) = SSk(τ^>kF) =∅. Remark 3.6. – (i) IfF∈D^>k(kX), thenSSk(F) =∅.

(ii) IfF∈D^k(kX), thenSSk+dX(F) = SS(F). HeredXis the dimension ofX.

The last statement follows from the characterization (iv)_k in Proposition 3.2 and the fact that H^j(X;F)vanishes for anyF∈D^k(kX)andj > k+dX.

Remark 3.7. – It is not true that F ∈D^k(kX;p) implies the existence of a morphism F→F inD^b(kX)which is an isomorphism inD^b(kX;p)andF∈D^k(kX). For example take X =R, p= (0; 1), Z ={x∈X; x <0},F1=k_{0} and F =kZ[1]. Then there is a morphism F1→F which is an isomorphism in D(kX;p). Hence one has F∈D⁰(kX;p).

Assume that there is a morphismu:F→FinD^b(kX)which is an isomorphism inD^b(kX;p) andF∈D⁰(kX). SinceH⁰(F) = 0, the morphismH⁰(F1)→H⁰(F)vanishes, and hence the compositionF1→F−→^u Fvanishes. This is a contradiction.

Example 3.8. – (i) One has

SSk(kX) =

∅ for k <0, T_X^∗X for k0.

(ii) LetX=RandZ1={x∈X; x0},Z2={x∈X; x >0}. Then one has SSk(kZ1) =

∅ for k <0,

{(x;ξ);ξ= 0, x0} ∪ {(x;ξ); x= 0, ξ0} for k0,

SSk(kZ2) =







∅ for k <0,

{(x;ξ);ξ= 0, x0} for k= 0, {(x;ξ);ξ= 0, x0} ∪ {(x;ξ); x= 0, ξ0} for k1.

(iii) LetX be a complex manifold. Then

SSk(OX) =





∅ for k <0, T_X^∗X for k= 0, T^∗X for k1.

(iv) LetMbe a real analytic manifold,Xa complexification ofM,Ma coherentDX-module, and letBM denote the sheaf of Sato’s hyperfunctions onM. RegardingT^∗M as a subset of T^∗X, one has

SS0

RHom_DX(M,BM)

⊂Ch(M)∩T^∗M.

This follows immediately from the Holmgren theorem.

4. Functorial properties

In this section, we study in Propositions 4.1–4.4 below, the behavior ofSSk under external tensor products, proper direct image and smooth inverse image. These properties are proved similarly to the corresponding properties of the microsupport (cf. Chapter V of [8]), using Proposition 3.2. However the property of the microsupport corresponding to Proposition 4.1 was only known in a weaker form.

PROPOSITION 4.1. – Let X and Y be real analytic manifolds. Then for F ∈D^b(kX), G∈D^b(kY)andk∈Z, one has

SSk(FG) =

i+j=k

SSi(F)×SSj(G).

(4.1)

Proof. – Let us first show that(p, p)∈/

i+j=kSSi(F)×SSj(G)implies (p, p)∈/SSk(FG).

SinceSSk(FG)⊂SS(FG)⊂SS(F)×SS(G), we may assume that (p, p)∈SS(F)×SS(G).

SinceSSi(F) = SS(F)fori0andSSi(F) =∅fori0, there existsisuch thatp∈SSi(F) andp /∈SSi−1(F). Setj=k−i. Thenp∈/SSj(G). Hence there exist a morphismF →F which is an isomorphism in D^b(kX;p) with F ∈D^>i−1(kX), and G →G which is an isomorphism inD^b(kY;p)withG∈D^>j(kX). HenceFG→FGis an isomorphism inD^b(kX×Y; (p, p))andFG∈D^>i+j(kX×Y).

Next let us showSSi(F)×SSj(G)⊂SSi+j(FG). Ifi0orj0, then the assertion is trivial. By the induction oniandj, we may assume thatSSi−1(F)×SSj(G)⊂SSi+j−1(FG) andSSi(F)×SSj−1(G)⊂SSi+j−1(FG).

Let(p, p)∈SSi(F)×SSj(G). We shall prove(p, p)∈SSi+j(FG). Ifp∈SSi−1(F), then the assertion is trivial. Hence we may assumep /∈SSi−1(F). Thus, by replacingFwith a sheaf microlocally isomorphic atp, we may assumeF∈Dⁱ(kX)from the beginning. Similarly we may assume G∈D^j(kY). For any open neighborhoodU ofp, we can find x1∈X and a C¹-functionϕsuch that(x1;dϕ(x1))∈U and

H_{ⁱ_ϕ0}(F)x₁= Γ_{ϕ0}(Hⁱ(F))x₁= 0.

Similarly, for any open neighborhoodU ofp, we can findy1∈Y and aC¹-functionψ such that(y1;dψ(y1))∈U andH_{^j_ψ0}(G)y1= Γ_{ψ0}(H^j(G))y1= 0. Setz= (x1, y1)∈X×Y andη(x, y) =ϕ(x) +ψ(y). Then(z;dη(z))∈U×Uand

H_{^i+j_η0}(FG)z= Γ_{η0}

Hⁱ(F)H^j(G)

z,

as a subspace ofHⁱ(F)x1⊗H^j(G)y1, contains(Γ_{ϕ0}(Hⁱ(F))x1)⊗(Γ_{ψ0}(H^j(G))y1).

Hence H_{^i+j_η0}(FG)z= 0. Since U ×U forms a neighborhood system of(p, p), we can conclude(p, p)∈SSi+j(FG). ✷

COROLLARY 4.2. – Let X andY be real analytic manifolds. Then for F ∈D^b(kX) and G∈D^b(kY), one has

SS(FG) = SS(F)×SS(G).

(4.2)

PROPOSITION 4.3. – Let f:X →Y be a morphism of real analytic manifolds and let F∈D^b(kX)such thatf is proper on the support ofF. Then for anyk∈Z,

SSk

Rf_∗(F)

⊂fπfd−1

SSk(F) . (4.3)

The equality holds in case f is a closed embedding.

Proof. – We shall follow the method of proof of Proposition 5.4.4 of [8].

Lety∈Y and letϕbe a realC¹-function onY such thatϕ(y) = 0andd(ϕ◦f)(x)∈/SSk(F) for everyx∈f⁻¹(y). Therefore

H^jRΓ_{ϕ◦f0}(F)|f⁻¹(y)= 0 for anyjk.

We have

H^jRΓ_{ϕ0}

Rf_∗(F)

yH^jRf_∗

RΓ_{ϕ◦f0}(F)

y

H^j

f⁻¹(y);RΓ_{ϕ◦f0}(F)

= 0 for everyjk. This proves (4.3).

Let us now assume thatf is a closed embedding. Letp /∈SSk(Rf_∗F). We may assume that Y is a real vector space andXis a linear subspace ofY. Letγ⊂Y,W⊂Y,εbe chosen as in Proposition 3.2(iv)kwith respect topandRf_∗F, that is,

H^j(Y;k_(x+γa)∩Hε⊗Rf_∗F) = 0 for anyjkandx∈W.

Sincef is a closed embedding, one has

H^j(Y;k(x+γ^a)∩H_ε⊗Rf_∗F)H^j(X;k(x+γ^a)∩H_ε∩X⊗F).

Hence one has

H^j(X;k_(x+γa∩X)∩(H_ε∩X)⊗F) = 0 for anyjk, and the interior of the polar set ofγ∩Xcontainsfdf_π⁻¹(p), and therefore

SSk(F)∩fdf_π⁻¹(p) =∅. ✷

PROPOSITION 4.4. – LetXandY be real analytic manifolds and letf:X→Y be a smooth morphism. LetG∈D^b(kY). Then, for anyk∈Z,

SSk

f⁻¹G

=fdf_π⁻¹

SSk(G) . (4.4)

Proof. – The problem being local onX, we may assume thatX=Y×Z,Y andZare vector spaces andfis the projection. Then we have to show

SSk(GkZ) = SSk(G)×T_Z^∗Z.

(4.5)

This follows from Proposition 4.1. ✷

5. Estimates for the truncated microsupport

Let Y be a smooth submanifold of X. In this section we will give an estimate for SSk(F)∩π⁻¹(Y). Recall thatµY(F)denotes the microlocalization ofF along Y. Note that forF∈D^k(kX),H^k(µY(F))H⁰(µY(H^k(F)))is a subsheaf ofπ⁻¹H^k(F)|T_Y^∗Xand

H^k µY(F)

p

s∈H^k(F)π(p); CY

supp(s)

π(p)\ {0}

⊂

v∈(TYX)π(p); v, p>0 (5.1)

forp∈T_Y^∗X.

The following result is a generalization of [7, Theorem 5.7.1] toSSk.

THEOREM 5.1. – Let X be a real analytic manifold and Y a smooth submanifold. Let F∈D^b(kX). Then

SSk(F)∩T_Y^∗X=

T_Y^∗X∩SSk(F)\π⁻¹(Y)

∪Supp

τ^kµY(F) . (5.2)

Proof. – It is evident that the right hand side of (5.2) is contained in the left hand side. Let us show the converse inclusion. Assuming thatp∈T_Y^∗Xsatisfies

p /∈SSk(F)\π⁻¹(Y)∪Supp τ^k

µY(F) ,

we shall prove p /∈SSk(F). Arguing by induction on k, one has p /∈SSk−1(F) and by Proposition 3.2(ii)k−1we may assume thatF∈D^k(kX).

There exists an open conic neighborhoodU ofpinT^∗X such thatU ∩SSk(F)⊂π⁻¹(Y) andH^j(µY(F))|U = 0for anyjk. Furthermore, we may assume thatX=Rⁿ,Y is a linear subspace ofX,p= (x0;ξ0). Let us take an open neighborhoodW ofx0, a proper closed convex coneγandε >0such thatW×Int(γ^◦)⊂U,ξ0∈Int(γ^◦)and(W +γ^a)∩Hε⊂W. Hence one has

H^j(X;k_(x+γa)∩Hε⊗F)H^j−k

X;k_(x+γa)∩Hε⊗H^k(F)

for anyjk.

Thus it is enough to check that Γ

X;k_(x+γa)∩Hε⊗H^k(F)

= 0.

(5.3)

Lets∈Γ(X;k_(x+γa)∩Hε⊗H^k(F)). Then there exists an open setΩ0such thatΩ0+γ^a= Ω0, x+γ^a⊂Ω0 andsextends to a sections˜∈Γ(Ω0;kH_ε⊗H^k(F)). Moreover we may assume that (Ω0 ∩Hε)×γ^◦⊂U. Set S = supp(˜s). Then S\ (Y +γ) satisfies the condition in Lemma 3.3 withΩ = Ω0\(Y +γ). Hence we haveS\(Y +γ) =∅ and henceS⊂Y +γ.

SinceH^k(µY(F))|U= 0and

CY(Y +γ)⊂Y ×(Y +γ)⊂

v∈TYX;v, ξ0>0

∪

Y × {0} ,

the formula (5.1) impliess˜|Y = 0. One has thereforeS∩Y =∅. ThenS satisfies the condition in Lemma 3.3, and we can concludeS=∅, which impliess= 0. ✷

We shall need the following definition:

DEFINITION 5.2. – LetY be a closed submanifold ofX, letk∈Zand letF∈D^b(kX). The closed subsetSS^Yk(F)ofπ⁻¹(Y)is defined by:p /∈SS^Yk(F)if and only if there exists an open conic neighborhoodU ofpinπ⁻¹(Y)satisfying the following two conditions:

(i) τ^kµY(F)|U∩T_Y^∗X= 0,

(ii) for any smooth real analytic hypersurfaceZofY,

τ^kµZ(F)|U∩T_Z^∗X\T_Y^∗X= 0.

We remark thatSS^Y_k(F)is a conic closed set obviously contained inSSk(F)∩π⁻¹(Y).

THEOREM 5.3. – Let X be a real analytic manifold and Y a closed submanifold. Let F∈D^b(kX). Then

SSk(F) = SSk(F)\π⁻¹(Y)∪SS^Y_k(F).

(5.4)

Proof. – The left hand side obviously contains the right hand side. Let us prove the converse inclusion. Assuming thatp∈π⁻¹(Y)satisfies p /∈SSk(F)\π⁻¹(Y)∪SS^Y_k(F), let us show p /∈SSk(F).

Ifp∈T_Y^∗X, Theorem 5.1 implies the assertion. Hence we may assumep /∈T_Y^∗X. LetUbe an open conic neighborhood ofpinT^∗Xsuch thatSSk(F)∩U⊂π⁻¹(Y)andU∩SS^Y_k(F) =∅.

We may assume thatX={x= (u, v, t);u∈Rⁿ, v∈R^m, t∈R},Y ={(u, v, t)∈X; u= 0}

andp= ((0,0,0); (0,0,1)). We may assumeW ×γ^◦⊂U withγ={t

u²+v²}and an open neighborhood W of the origin. SetHε={(u, v, t); t >−ε}, and choose W and a sufficiently smallεsuch that(x+γ^a)∩Hε⊂W for anyx∈W.

By the induction onk, we may assumeF∈D^k(kX). It is enough to show for anyx0∈W Γ

x0+ Int(γ^a);k_Hε⊗H^k(F)

= 0.

Lets∈Γ(x0+ Int(γ^a);kHε ⊗H^k(F)). SetS= supp(s)⊂(x0+ Int(γ^a))∩Hε. Assuming S=∅, we shall derive a contradiction. Setx0= (u0, v0, t0)and set

ϕ(x) =u−u0²+v−v0²−(t−t0)².

Then Ω :=x0+ Int(γ^a) ={x;t < t0, ϕ(x)<0}, and ϕ(Ω∩Hε) is bounded from below.

Moreover one hasdϕ(x)∈Int(γ^◦)for any x∈Ω, anddϕ(x)∈/T_Y^∗X for anyx∈Ω∩Y. Set c= inf{ϕ(x); x∈S}<0. Sinceϕ|S:S→R<0is a proper map, one hasc∈ϕ(S). LetZcbe the closed subset{x= (0, v, t)∈Y; t < t0, ϕ(x) =c}ofY and setΩ= Ω\(Zc+γ). Since one has

Zc+γ=

(t, u, v);t−t0u −

v−v0²+u0²−c ,

ϕtakes values smaller thanc onY \(Zc+γ), and henceS:=S∩Ω does not intersectY. Therefore S satisfies the condition in Lemma 3.3. HenceS=∅, which means S⊂Zc+γ.

SinceCZc(Zc+γ)x\ {0} ⊂ {dϕ(x)>0}for anyx∈Zc,H⁰(µZcH^k(F))|U\T_Y^∗X= 0implies s|Zc= 0. ThereforeS∩Zc=∅. Since{x∈(Zc+γ)∩Ω;ϕ(x) =c} ⊂Zc, one hasc /∈ϕ(S), which is a contradiction. ✷

6. Applications toD-modules In this section,Xdenotes a complex manifold.

Before stating our main result, let us recall a classical lemma on the vanishing of the microlocalization ofOXalong submanifolds.

LEMMA 6.1. – LetY be a closed complex submanifold of codimensiondofX and letSbe a smooth real analytic hypersurface ofY. Then

H^k

µY(OX)

= 0 for anyk=d, (6.1)

H^k

µS(OX)

T_S^∗X\T_Y^∗X= 0 for anykd.

(6.2)

Proof. – The vanishing property (6.1) is proved in [11] and (by a different method) in [7, Proposition 11.3.4].

The vanishing property (6.2) follows from [7, Proposition 11.3.1]. Let us recall this statement.

Let p∈T_S^∗X. Set Ep =Tp(T^∗X), λS =Tp(T_S^∗X), λ0=Tp(π⁻¹π(p)), and denote by ν the complex line in Ep, the tangent space to the Euler vector field in T^∗X at p. Let c be the real codimension of the real submanifold S and let δ denote the complex dimension of λS∩√

−1λS∩λ0.

The result of loc. cit. asserts that if the real dimension ofλS∩νis1, then H^j

µS(OX)

p= 0 forj < c−δ.

(The result in loc. cit. is more precise, involving the signature of the Levi form.) If p∈ T_S^∗X\T_Y^∗X, the real dimension ofλS∩ν is1. Sincec= 2d+ 1andδ=d, we get the desired result. ✷

Now we are ready to prove the following proposition.