SCALAR CURVATURE RIGIDITY OF GEODESIC BALLS IN Sn Simon Brendle & Fernando C. Marques

88(2011) 379-394

SCALAR CURVATURE RIGIDITY OF GEODESIC BALLS IN Sⁿ

Simon Brendle & Fernando C. Marques

Abstract

In this paper, we prove a scalar curvature rigidity result for geodesic balls inSⁿ. This result contrasts sharply with the coun- terexamples to Min-Oo’s conjecture constructed in [7].

1. Introduction

This paper is concerned with rigidity phenomena involving the scalar curvature. These questions are motivated to a large extent by the pos- itive mass theorem in general relativity, which was proved by Schoen and Yau [18] and Witten [20]. An important corollary of this theorem is that any Riemannian metric onRⁿwhich has nonnegative scalar cur- vature and agrees with the Euclidean metric outside a compact set is necessarily flat.

It was observed by Miao [16] that the positive mass theorem implies the following rigidity result for metrics on the unit ball:

Theorem 1. Suppose thatg is a Riemannian metric on the unit ball Bⁿ⊂Rⁿ with the following properties:

• The scalar curvature of g is nonnegative.

• The induced metric on the boundary ∂Bⁿ agrees with the standard metric on ∂Bⁿ.

• The mean curvature of ∂Bⁿ with respect to g is at least n−1.

Then g is isometric to the standard metric on Bⁿ.

An important generalization of Theorem 1 was proved by Shi and Tam [19].

Motivated by the positive mass theorem, Min-Oo [17] posed the fol- lowing question:

Min-Oo’s Conjecture. Suppose that g is a Riemannian metric on the hemisphere S₊ⁿ with the following properties:

The first author was supported in part by the National Science Foundation under grant DMS-0905628. The second author was supported by CNPq-Brazil, FAPERJ, and the Stanford Department of Mathematics.

Received 07/11/2010.

379

• The scalar curvature of g is at least n(n−1).

• The induced metric on the boundary ∂S₊ⁿ agrees with the standard metric on ∂S₊ⁿ.

• The boundary ∂S₊ⁿ is totally geodesic with respect to g.

Then g is isometric to the standard metric on S₊ⁿ.

Min-Oo’s conjecture has been verified in many special cases (see e.g.

[11], [13], [14]). A related rigidity result for real projective space RP³ was established in [3] (see also [4]). A survey of these and other related results can be found in [6].

Very recently, counterexamples to Min-Oo’s conjecture were con- structed in [7].

Theorem 2 (S. Brendle, F.C. Marques, A. Neves [7]). Given any integer n≥3, there exists a smooth Riemannian metricgˆon the hemi- sphere S₊ⁿ with the following properties:

• The scalar curvature of gˆis at leastn(n−1) at each point on S₊ⁿ.

• The scalar curvature ofgˆis strictly greater than n(n−1) at some point on S₊ⁿ.

• The metricgˆagrees with the standard metric in a neighborhood of

∂S₊ⁿ.

The proof of Theorem 2 relies on a perturbation analysis.

In this paper, we study the analogous rigidity question for geodesic balls inSⁿ of radius less than ^π₂. To fix notation, letg be the standard metric onSⁿand letf :Sⁿ→Rdenotes the restriction of the coordinate functionx_n+1 to Sⁿ. We will consider a domain of the form Ω ={f ≥ c}. If c≥ ^√_n+3² , we have the following rigidity result:

Theorem 3. Consider the domain Ω = {f ≥ c}, where c ≥ ^√_n+3² . Let g be a Riemannian metric on Ωwith the following properties:

• Rg≥n(n−1) at each point in Ω.

• The metrics g and g induce the same metric on ∂Ω.

• Hg ≥Hg at each point on ∂Ω.

If g−g is sufficiently small in the C²-norm, then ϕ^∗(g) =g for some diffeomorphism ϕ: Ω→Ωwith ϕ|∂Ω = id.

We remark that the conclusion of Theorem 3 holds under the weaker assumption thatg is close to g inW^2,p-norm for p > n.

Note that Theorem 3 is false for the hemisphere{f ≥0}: by Theorem 4 in [7], there exist Riemannian metrics on the hemisphere which satisfy the assumptions of Theorem 3 and are arbitrary close to the standard metric g in theC^∞-topology, but which are not isometric tog.

The proof of Theorem 3 relies on a perturbation analysis which is similar in spirit to Bartnik’s work on the positive mass theorem (cf.

[1], Section 5). Similar techniques have been employed in the study

of the total scalar curvature functional (see e.g. [2], Section 4G) and the Yamabe flow (cf. [5]). Dai, Wang, and Wei [8],[9] have obtained interesting stability results for manifolds with parallel spinors, as well as for K¨ahler-Einstein manifolds.

2. The scalar curvature and boundary mean curvature of a perturbed metric

In this section, we consider a smooth manifold Ω with boundary∂Ω.

Letgbe a fixed Riemannian metric on Ω. Moreover, we consider another Riemannian metric g =g+h, where |h|^g ≤ ¹2 at each point in Ω. For abbreviation, we write (h²)ik =g^jlhijhkl.

Proposition 4. The scalar curvature of g satisfies the pointwise es- timate

Rg−Rg+hRicg, hi − hRicg, h²i +1

4g^ijg^klg^pqD_ih_kpD_jh_lq−1

2g^ijg^klg^pqD_ih_kpD_lh_jq +1

4g^pq∂p(trg(h))∂q(trg(h)) +Di

g^ikg^jl(Dkhjl−Dlhjk)

≤C|h| |Dh|²+C|h|³.

Here, D denotes the Levi-Civita connection with respect to g, and C is a positive constant which depends only on n.

Proof. The Levi-Civita connection with respect tog is given by DXY =DXY + Γ(X, Y),

where Γ is defined by g(Γ(X, Y), Z) = 1

2 (DXh)(Y, Z) + (DYh)(X, Z)−(DZh)(X, Y) .

In local coordinates, the tensor Γ can be written in the form Γ^m_jk= 1

2g^lm(Djhkl+Dkhjl−Dlhjk).

With this understood, the scalar curvature of g is given by R_g =g^ik(Ric_g)_ik+g^ikg^jlg_pqΓ^q_ilΓ^p_jk−g^ikg^jlg_pqΓ^q_jlΓ^p_ik

−g^ikg^jl(D²_i,kh_jl−D²_i,lh_jk)

(cf. [7], Proposition 16). This implies

R_g−R_g+hRicg, hi − hRicg, h²i

−3

4g^ijg^klg^pqD_ih_kpD_jh_lq+ 1

2g^ijg^klg^pqD_ih_kpD_lh_jq +1

4g^pq∂_p(trg(h))∂_q(trg(h))−g^ijg^pqD_ih_jp∂_q(trg(h)) +g^ijg^klg^pqDihjpDkhlq+g^ikg^jl(D²_i,khjl−D²_i,lhjk)

≤C|h| |Dh|²+C|h|³. Hence, we obtain

Rg−Rg+hRicg, hi − hRicg, h²i +1

4g^ijg^klg^pqD_ih_kpD_jh_lq−1

2g^ijg^klg^pqD_ih_kpD_lh_jq +1

4g^pq∂p(trg(h))∂q(trg(h)) +Di

g^ikg^jl(D_kh_jl−D_lh_jk)

≤C|h| |Dh|²+C|h|³,

as claimed. q.e.d.

In the next step, we estimate the mean curvature of the boundary

∂Ω with respect to the metric g. To that end, we assume thatg and g induce the same metric on the boundary∂Ω; in other words, we assume that h(X, Y) = 0 wheneverX and Y are tangent vectors to ∂Ω.

Proposition 5. Assume that gand ginduce the same metric on the boundary ∂Ω. Then the mean curvature of ∂Ωwith respect to gsatisfies

2 (Hg−H_g)−

h(ν, ν)−1

4h(ν, ν)²+

n−1

X

a=1

h(ea, ν)² H_g

+ 1−1

2h(ν, ν)

n−1

X

a=1

2 (Deah)(ea, ν)−(Dνh)(ea, e_a)

≤C|h|²|Dh|+C|h|³.

Here, {e_a : 1≤a≤n−1} is a local orthonormal frame on ∂Ω, and C is a positive constant that depends only on n.

Proof. Using the identity Hgν−Hgν=−

n−1

X

a=1

(Deaea−Deaea) =−

n−1

X

a=1

Γ(ea, ea),

we obtain

2 H_gg(ν, ν)−H_gg(ν, ν)

=−2

n−1

X

a=1

g(Γ(e_a, e_a), ν) =−

n−1

X

a=1

2 (D_eah)(e_a, ν)−(D_νh)(e_a, e_a) .

Clearly, g(ν, ν) = 1 +h(ν, ν). Moreover, it is easy to see that the vector ν−Pn−1

a=1h(e_a, ν)e_a is orthogonal to ∂Ω with respect to g. From this, we deduce that

ν−

n−1

X

a=1

h(e_a, ν)e_a=

1 +h(ν, ν)−

n−1

X

a=1

h(e_a, ν)^{2 1}₂

ν,

hence

g(ν, ν) =

1 +h(ν, ν)−

n−1

X

a=1

h(e_a, ν)^{2 1}₂

.

Substituting these identities into the previous formula for Hg, the as-

sertion follows. q.e.d.

3. Perturbations of the standard metric on Sⁿ

We now consider perturbations of the standard metric g on Sⁿ. To fix notation, let f : Sⁿ → R denote the restriction of the coordinate functionx_n+1 toSⁿ, and let Ω ={f ≥c}be a geodesic ball centered at the north pole. Here,c is a positive real number which will be specified later.

Letgbe a Riemannian metric on Ω. We will assume throughout that g and g induce the same metric on the boundary ∂Ω. Moreover, we assume that g=g+h, where|h|^g ≤ ¹₂ at each point in Ω.

Our goal in this section is to estimate the integral Z

Ω

(Rg−n(n−1))f dvolg

(see also [12]).

Proposition 6. We have

Z

Ω

(Rg−n(n−1)−(n−1)|h|²g)f dvolg+1 4

Z

Ω|Dh|²f dvolg

+1 4

Z

Ω|∇(trg(h))|²f dvolg−1 2

Z

Ω

g^ijg^klg^pqD_ih_kpD_lh_jqf dvolg

+ Z

Ω

g^ikg^jpg^lqhpq(D_kh_jl−D_lh_jk)∂if dvolg

+ Z

Ω

g^ipg^kqg^jlh_pq(Dkh_jl−D_lh_jk)∂_if dvolg

+ Z

∂Ω

g^jl(D_kh_jl−D_lh_jk)ν^kf dσg

− Z

∂Ω

g^jpg^lqh_pq(Dkh_jl−D_lh_jk)ν^kf dσ_g

− Z

∂Ω

g^kqg^jlh_pq(Dkh_jl−D_lh_jk)ν^pf dσ_g

≤C Z

Ω|h| |Dh|²dvolg+C Z

Ω|h|³dvolg+C Z

∂Ω|h|²|Dh|dσ_g, where C is a positive constant that depends only on nand c.

Proof. Using Proposition 4 and the divergence theorem, we obtain

Z

Ω

(Rg−n(n−1) + (n−1) trg(h)−(n−1)|h|²g)f dvolg

+1 4

Z

Ω|Dh|²f dvol_g−1 2

Z

Ω

g^ijg^klg^pqD_ih_kpD_lh_jqf dvol_g +1

4 Z

Ω|∇(trg(h))|²f dvolg− Z

Ω

g^ikg^jl(Dkh_jl−D_lh_jk)∂_if dvolg

+ Z

∂Ω

g^ikg^jl(Dkh_jl−D_lh_jk)g_imν^mf dσ_g

≤C Z

Ω|h| |Dh|²dvolg+C Z

Ω|h|³dvolg.

Here, ν denotes the outward-pointing unit normal vector to ∂Ω with respect to the metricg. Using the identityD²_i,kf =−f g_ik, we obtain

Z

Ω

(n−1) trg(h)f dvolg− Z

Ω

g^ikg^jl(Dkhjl−Dlhjk)∂if dvolg

=− Z

∂Ω

(trg(h)∂_νf−h(ν,∇f))dσ_g = 0.

Thus, we conclude that

Z

Ω

(Rg−n(n−1)−(n−1)|h|²g)f dvolg+1 4

Z

Ω|Dh|²f dvolg

+1 4

Z

Ω|∇(trg(h))|²f dvolg−1 2

Z

Ω

g^ijg^klg^pqD_ih_kpD_lh_jqf dvolg

− Z

Ω

(g^ikg^jl−g^ikg^jl) (D_kh_jl−D_lh_jk)∂_if dvolg

+ Z

∂Ω

g^ikg^jl(D_kh_jl−D_lh_jk)g_imν^mf dσg

≤C Z

Ω|h| |Dh|²dvolg+C Z

Ω|h|³dvolg.

From this, the assertion follows easily. q.e.d.

In the remainder of this section, we will assume thath is divergence- free in the sense thatg^ikD_ih_kl= 0.

Proposition 7. Assume that h is divergence-free. Then

Z

Ω

(R_g−n(n−1))f dvol_g+1 4

Z

Ω|Dh|²f dvol_g +1

4 Z

Ω|∇(trg(h))|²f dvolg+1 2

Z

Ω|h|²gf dvolg

+1 2

Z

Ω

trg(h)²f dvolg+1 4

Z

∂Ω

(|h|²g+ 3h(ν, ν)²)∂νf dσg

+ Z

∂Ω

g^jlD_kh_jlν^kf dσ_g−1 2

Z

∂Ω

g^klg^pqh_kpD_lh_jqν^jf dσ_g

− Z

∂Ω

g^jpg^lqhpq(Dkhjl−Dlhjk)ν^kf dσg

− Z

∂Ω

g^kqg^jlh_pqD_kh_jlν^pf dσ_g

≤C Z

Ω|h| |Dh|²dvolg+C Z

Ω|h|³dvolg+C Z

∂Ω|h|²|Dh|dσg, where C is a positive constant that depends only on nand c.

Proof. Sinceg has constant sectional curvature 1, we have D²_i,lhjq=D²_l,ihjq+h_lqg_ij −hiqg_jl+h_jlg_iq−hijg_lq. Since h is divergence-free, it follows that

g^ijD²_i,lh_jq=n h_lq−trg(h)g_lq.

This implies

− Z

Ω

g^ijg^klg^pqh_kpD_lh_jq∂_if dvolg− Z

Ω

g^ijg^klg^pqD_ih_kpD_lh_jqf dvolg

= Z

Ω

g^ijg^klg^pqh_kpD²_i,lh_jqf dvolg− Z

∂Ω

g^klg^pqh_kpD_lh_jqν^jf dσg

=n Z

Ω|h|²gf dvolg− Z

Ω

trg(h)²f dvolg− Z

∂Ω

g^klg^pqh_kpD_lh_jqν^jf dσ_g.

From this, we deduce that Z

Ω

g^ikg^jpg^lqh_pq(D_kh_jl−D_lh_jk)∂_if dvol_g

−1 2

Z

Ω

g^ijg^klg^pqD_ih_kpD_lh_jqf dvolg

= 1 2

Z

Ω

g^ik∂_k(|h|²g)∂_if dvol_g−1 2

Z

Ω

g^ikg^jpg^lqh_pqD_lh_jk∂_if dvol_g +n

2 Z

Ω|h|²gf dvolg−1 2

Z

Ω

trg(h)²f dvolg

−1 2

Z

∂Ω

g^klg^pqh_kpD_lhjqν^jf dσg.

Integration by parts gives Z

Ω

g^ikg^jpg^lqh_pq(Dkh_jl−D_lh_jk)∂_if dvolg

−1 2

Z

Ω

g^ijg^klg^pqDih_kpD_lhjqf dvolg

=−1 2

Z

Ω|h|²g∆gf dvolg+1 2

Z

Ω

g^ikg^jpg^lqh_pqh_jkD²_i,lf dvolg

+1 2

Z

∂Ω|h|²g∂νf dσg−1 2

Z

∂Ω

g^ikg^jphpqh_jk∂if ν^qdσg

+n 2

Z

Ω|h|²gf dvolg−1 2

Z

Ω

trg(h)²f dvolg

−1 2

Z

∂Ω

g^klg^pqhkpDlhjqν^jf dσg

= 2n−1 2

Z

Ω|h|²gf dvolg−1 2

Z

Ω

trg(h)²f dvolg

+1 2

Z

∂Ω|h|²g∂νf dσg−1 2

Z

∂Ω

g^ikg^jphpqhjk∂if ν^qdσg

−1 2

Z

∂Ω

g^klg^pqh_kpD_lh_jqν^jf dσ_g.

Moreover, we have Z

Ω

g^ipg^kqg^jlh_pq(Dkh_jl−D_lh_jk)∂_if dvolg

= Z

Ω

g^ipg^kqhpq∂_k(trg(h))∂if dvolg

=− Z

Ω

g^ipg^kqh_pqtrg(h)D²_i,kf dvolg+ Z

∂Ω

g^iph_pqtrg(h)∂_if ν^qdσ_g

= Z

Ω

trg(h)²f dvolg+ Z

∂Ω

g^iphpqtrg(h)∂if ν^qdσg.

Putting these facts together, we obtain Z

Ω

g^ipg^kqg^jlhpq(Dkhjl−Dlhjk)∂if dvolg

+ Z

Ω

g^ikg^jpg^lqh_pq(Dkh_jl−D_lh_jk)∂_if dvolg

−1 2

Z

Ω

g^ijg^klg^pqDihkpDlhjqf dvolg

= 2n−1 2

Z

Ω|h|²gf dvolg+1 2

Z

Ω

trg(h)²f dvolg

+1 2

Z

∂Ω|h|²g∂_νf dσ_g− 1 2

Z

∂Ω

g^ikg^jph_pqh_jk∂_if ν^qdσ_g +

Z

∂Ω

g^iph_pqtr_g(h)∂_if ν^qdσ_g−1 2

Z

∂Ω

g^klg^pqh_kpD_lh_jqν^jf dσ_g

= 2n−1 2

Z

Ω|h|²gf dvolg+1 2

Z

Ω

trg(h)²f dvolg

+1 4

Z

∂Ω

(|h|²g+ 3h(ν, ν)²)∂νf dσg−1 2

Z

∂Ω

g^klg^pqh_kpD_lhjqν^jf dσg.

Hence, the assertion follows from Proposition 6. q.e.d.

4. Analysis of the boundary terms

In this section, we analyze the boundary terms in Proposition 7. As in the previous section, we assume that gis the standard metric onSⁿ, and Ω = {f ≥c} centered at the north pole. Moreover, we consider a Riemannian metric on Ω of the form g=g+h, where|h|^g ≤ ¹₂ at each point in Ω.

Proposition 8. Assume that h is divergence-free. Then Z

∂Ω

g^jlD_kh_jlν^kdσ_g−1 2

Z

∂Ω

g^klg^pqh_kpD_lh_jqν^jdσ_g

− Z

∂Ω

g^jpg^lqh_pq(D_kh_jl−D_lh_jk)ν^kdσ_g

− Z

∂Ω

g^kqg^jlh_pqD_kh_jlν^pdσ_g

=− Z

∂Ω

(1−h(ν, ν))

n−1

X

a=1

2 (Deah)(ea, ν)−(Dνh)(ea, e_a) dσ_g

+ Z

∂Ω

1− 1

2h(ν, ν)

h(ν, ν)H_gdσ_g + 3n−2

2(n−1) Z

∂Ω n−1

X

a=1

h(e_a, ν)²H_gdσ_g.

Here, {ea : 1≤a≤n−1} is a local orthonormal frame on ∂Ω, and C is a positive constant that depends only on nand c.

Proof. Let{e_a : 1≤a≤n−1} be a local orthonormal frame on∂Ω.

Since h is divergence-free, we have g^jlD_kh_jlν^k−1

2g^klg^pqh_kpD_lh_jqν^j

−g^jpg^lqh_pq(D_kh_jl−D_lh_jk)ν^k−g^kqg^jlh_pqD_kh_jlν^p

=−(1−h(ν, ν))

n−1

X

a=1

2 (Deah)(ea, ν)−(Dνh)(ea, ea)

+ 1−1

2h(ν, ν)

n−1

X

a=1

(Deah)(ea, ν)−1 2

n−1

X

a=1

(Deah)(ν, ν)h(ea, ν)

+3 2

n−1

X

a,b=1

h(e_a, ν) (D_ebh)(e_a, e_b)−

n−1

X

a,b=1

h(e_a, ν) (D_eah)(e_b, e_b).

At this point, we define a one-form ω on∂Ω byω(ea) = (1−¹₂h(ν, ν)) h(e_a, ν). Since∂Ω is umbilic with respect to g, we have

D_eaν = 1

n−1H_ge_a,

whereH_g denotes the mean curvature of ∂Ω with respect to the metric g. Using this relation, we obtain the following formula for the divergence

of ω:

div_∂Ω(ω) = 1−1

2h(ν, ν)

n−1

X

a=1

(Deah)(ea, ν)− 1 2

n−1

X

a=1

(Deah)(ν, ν)h(ea, ν)

− 1− 1

2h(ν, ν)

h(ν, ν)H_g− 1 n−1

n−1

X

a=1

h(e_a, ν)²H_g. Moreover, we have the pointwise identities

n−1

X

b=1

(D_ebh)(e_a, e_b) = n

n−1h(e_a, ν)H_g

and n−1

X

b=1

(Deah)(eb, e_b) = 2

n−1h(ea, ν)H_g. Putting these facts together, we obtain

g^jlDkhjlν^k−1

2g^klg^pqhkpDlhjqν^j

−g^jpg^lqh_pq(Dkh_jl−D_lh_jk)ν^k−g^kqg^jlh_pqD_kh_jlν^p

=−(1−h(ν, ν))

n−1

X

a=1

2 (Deah)(ea, ν)−(Dνh)(ea, ea)

+ 1−1

2h(ν, ν)

h(ν, ν)H_g+ 3n−2 2(n−1)

n−1

X

a=1

h(e_a, ν)²H_g+ div_∂Ω(ω).

Therefore, the assertion follows from the divergence theorem. q.e.d.

Combining Proposition 8 and Proposition 5, we can draw the follow- ing conclusion:

Corollary 9. If h is divergence-free, then we have

Z

∂Ω

(2−h(ν, ν)) (Hg−H_g)dσ_g

− Z

∂Ω

g^jlD_kh_jlν^kdσ_g+1 2

Z

∂Ω

g^klg^pqh_kpD_lh_jqν^jdσ_g +

Z

∂Ω

g^jpg^lqh_pq(D_kh_jl−D_lh_jk)ν^kdσg

+ Z

∂Ω

g^kqg^jlh_pqD_kh_jlν^pdσ_g

+ 1 4

Z

∂Ω

h(ν, ν)²H_gdσ_g+ n 2(n−1)

Z

∂Ω n−1

X

a=1

h(ea, ν)²H_gdσ_g

≤C Z

∂Ω|h|²|Dh|dσ_g+C Z

∂Ω|h|³dσ_g,

where C is a positive constant that depends only on nand c.

Proof. It follows from Proposition 5 that

Z

∂Ω

(2−h(ν, ν)) (Hg−H_g)dσ_g

− Z

∂Ω

h(ν, ν)−3

4h(ν, ν)²+

n−1

X

a=1

h(e_a, ν)² H_gdσ_g

+ Z

∂Ω

(1−h(ν, ν))

n−1

X

a=1

2 (Deah)(ea, ν)−(Dνh)(ea, e_a) dσ_g

≤C Z

∂Ω|h|²|Dh|dσg+C Z

∂Ω|h|³dσg.

Moreover, we have

Z

∂Ω

g^jlD_kh_jlν^kdσ_g−1 2

Z

∂Ω

g^klg^pqh_kpD_lh_jqν^jdσ_g

− Z

∂Ω

g^jpg^lqh_pq(Dkh_jl−D_lh_jk)ν^kdσ_g

− Z

∂Ω

g^kqg^jlhpqD_kh_jlν^pdσg

=− Z

∂Ω

(1−h(ν, ν))

n−1

X

a=1

2 (D_eah)(e_a, ν)−(Dνh)(e_a, e_a) dσg

+ Z

∂Ω

1− 1

2h(ν, ν)

h(ν, ν)Hgdσg

+ 3n−2 2(n−1)

Z

∂Ω n−1

X

a=1

h(ea, ν)²Hgdσg

by Proposition 8. Putting these facts together, the assertion follows.

q.e.d.

Theorem 10. Assume that h is divergence-free. Then

Z

Ω

(Rg−n(n−1))f dvolg+ Z

∂Ω

(2−h(ν, ν)) (Hg−Hg)f dσ +1

4 Z

Ω|Dh|²f dvol_g+1 4

Z

Ω|∇(tr_g(h))|²f dvol_g +1

2 Z

Ω|h|²gf dvolg+1 2

Z

Ω

trg(h)²f dvolg

+ Z

∂Ω

h(ν, ν)²∂_νf dσ_g+1 2

Z

∂Ω n−1

X

a=1

h(ea, ν)²∂_νf dσ_g

+1 4

Z

∂Ω

h(ν, ν)²Hgf dσg+ n 2(n−1)

Z

∂Ω n−1

X

a=1

h(ea, ν)²Hgf dσg

≤C Z

Ω|h| |Dh|²dvolg+C Z

Ω|h|³dvolg

+C Z

∂Ω|h|²|Dh|dσg+C Z

∂Ω|h|³dσg.

Here, C is a positive constant that depends only on n and c.

Proof. Recall that f is constant along the boundary∂Ω. Hence, the assertion is a consequence of Proposition 7 and Corollary 9. q.e.d.

5. Proof of Theorem 3

To prove Theorem 3, we need an analogue of Ebin’s slice theorem for manifolds with boundary [10] (see also [12]). The proof is standard, and works on any compact manifold with boundary.

Proposition 11. Fix a real number p > n. If kg−gkW^2,p(Ω,g) is sufficiently small, we can find a diffeomorphism ϕ : Ω → Ω such that ϕ|∂Ω= id and h=ϕ^∗(g)−g is divergence-free. Moreover,

khkW^2,p(Ω,g)≤Nkg−gkW^2,p(Ω,g), where N is a positive constant that depends only on Ω.

Proof. LetS denote the space of symmetric two-tensors on Ω of class W^2,p, and letM denote the space of Riemannian metrics on Ω of class W^2,p. Moreover, let X denote the space of vector fields of class W^3,p that vanish along the boundary ∂Ω, and let D denote the space of all diffeomorphisms ϕ: Ω→ Ω of classW^3,p satisfyingϕ|∂Ω = id. Clearly, the tangent space to M at g can be identified with S; similarly, the tangent space toD at the identity can be identified with X.

There is a natural action

A:D×M →M, (ϕ, g) →ϕ^∗(g).

Let us consider the linearization of A around the point (id, g). This gives a map L:T_idD → T_gM. The map L sends a vector field ξ ∈X to the Lie derivative L_ξ(g) ∈ S. Standard elliptic regularity theory implies that

S ={L_ξ(g) :ξ∈X} ⊕ {h ∈S :h is divergence-free}

(compare [12], p. 523). Hence, the assertion follows from the implicit

function theorem. q.e.d.

We now complete the proof of Theorem 3. Let g be a Riemannian metric on the domain Ω ={f ≥c}with the following properties:

• R_g≥n(n−1) at each point in Ω.

• The metrics g and g induce the same metric on∂Ω.

• H_g ≥H_g at each point on∂Ω.

If kg−gkW^2,p(Ω,g) is sufficiently small, Proposition 11 implies the ex- istence of a diffeomorphism ϕ : Ω → Ω such that ϕ|∂Ω = id and h=ϕ^∗(g)−g is divergence-free.

Note that R_ϕ∗(g) ≥n(n−1) at each point in Ω andH_ϕ∗(g) ≥H_g at each point on ∂Ω. Applying Theorem 10 to the metric ϕ^∗(g) =g+h, we obtain

1 4

Z

Ω|Dh|²f dvolg+1 4

Z

Ω|∇(trg(h))|²f dvolg

+1 2

Z

Ω|h|²gf dvolg+1 2

Z

Ω

trg(h)²f dvolg

+ Z

∂Ω

h(ν, ν)²∂νf dσg+ 1 2

Z

∂Ω n−1

X

a=1

h(ea, ν)²∂νf dσg

+1 4

Z

∂Ω

h(ν, ν)²Hgf dσg+ n 2(n−1)

Z

∂Ω n−1

X

a=1

h(ea, ν)²Hgf dσg

≤C Z

Ω|h| |Dh|²dvolg+C Z

Ω|h|³dvolg

+C Z

∂Ω|h|²|Dh|dσ_g+C Z

∂Ω|h|³dσ_g. If we choose c≥ ^√_n+3² , then

1

4H_gf +∂_νf = n−1 4

f²

|∇f|− |∇f|= n−1 4

c²

√1−c² −p

1−c² ≥0

at each point on∂Ω. This implies 1

4 Z

Ω|Dh|²f dvolg+1 4

Z

Ω|∇(trg(h))|²f dvolg

+ 1 2

Z

Ω|h|²gf dvol_g+1 2

Z

Ω

tr_g(h)²f dvol_g

≤C Z

Ω|h| |Dh|²dvolg+C Z

Ω|h|³dvolg

+C Z

∂Ω|h|²|Dh|dσ_g+C Z

∂Ω|h|³dσ_g.

By the trace theorem, the error terms on the right hand side are bounded from above by CkhkC¹(Ω,g)khk²_W1,2(Ω,g). Hence, if khkC¹(Ω,g) is suffi- ciently small, thenhvanishes identically, and thereforeϕ^∗(g) =g. This completes the proof of Theorem 3.

References

[1] R. Bartnik, The mass of an asymptotically flat manifold, Comm. Pure Appl.

Math.39(1986), 661–693, MR 0849427, Zbl 0598.53045.

[2] A. Besse,Einstein manifolds,Classics in Mathematics, Springer-Verlag, Berlin, 2008, MR 2371700, Zbl 1147.53001.

[3] H. Bray, S. Brendle, M. Eichmair, & A. Neves, Area-minimizing projective planes in three-manifolds, Comm. Pure Appl. Math. 63 (2010), 1237–1247, MR 2675487, Zbl 1200.53053.

[4] H. Bray, S. Brendle, & A. Neves, Rigidity of area-minimizing two-spheres in three-manifolds,Comm. Anal. Geom.18(2010), 821–830

[5] S. Brendle,Convergence of the Yamabe flow in dimension6and higher,Invent.

Math.170(2007), 541–576, MR 2357502, Zbl 1130.53044.

[6] S. Brendle,Rigidity phenomena involving scalar curvature,Surveys in Differen- tial Geometry (to appear).

[7] S. Brendle, F.C. Marques, & A. Neves, Deformations of the hemisphere that increase scalar curvature,Invent. Math.185(2011), 175–197.

[8] X. Dai, X. Wang, & G. Wei,On the stability of Riemannian manifolds with par- allel spinors,Invent. Math.161(2005), 151–176, MR 2178660, Zbl 1075.53042.

[9] X. Dai, X. Wang, & G. Wei, On the variational stability of K¨ahler-Einstein metrics,Comm. Anal. Geom.15(2007), 669–693, MR 2395253, Zbl 1149.53027.

[10] D. Ebin,The manifold of Riemannian metrics,Proc. Sympos. Pure Math., vol.

XV (Berkeley, Calif., 1968), 11–40, Amer. Math. Soc., Providence RI, 1970, MR 0267604, Zbl 0205.53702.

[11] M. Eichmair, The size of isoperimetric surfaces in3-manifolds and a rigidity result for the upper hemisphere,Proc. Amer. Math. Soc.137(2009), 2733–2740, MR 2497486, Zbl 1187.53038.

[12] A.E. Fischer & J.E. Marsden,Deformations of the scalar curvature,Duke Math.

J.42(1975), 519–547, MR 0380907, Zbl 0336.53032.

[13] F. Hang & X. Wang, Rigidity and non-rigidity results on the sphere, Comm.

Anal. Geom.14(2006), 91–106, MR 2230571, Zbl 1119.53029.

[14] F. Hang & X. Wang, Rigidity theorems for compact manifolds with boundary and positive Ricci curvature,J. Geom. Anal.19(2009), 628–642, MR 2496569, Zbl 1175.53056.

[15] L. Huang & D. Wu, Rigidity theorems on hemispheres in non-positive space forms,Comm. Anal. Geom.18(2010), 339–363, MR 2672236, Zbl pre05853040.

[16] P. Miao, Positive mass theorem on manifolds admitting corners along a hyper- surface,Adv. Theor. Math. Phys.6(2002), 1163–1182, MR 1982695.

[17] M. Min-Oo, Scalar curvature rigidity of certain symmetric spaces, Geometry, topology, and dynamics (Montreal, 1995), 127–137, CRM Proc. Lecture Notes vol. 15, Amer. Math. Soc., Providence RI, 1998, MR 1619128, Zbl 0911.53032.

[18] R. Schoen & S.T. Yau,On the proof of the positive mass conjecture in general relativity,Comm. Math. Phys.65(1979), 45–76, MR 0526976, Zbl 0405.53045.

[19] Y. Shi & L.F. Tam,Positive mass theorem and the boundary behaviors of compact manifolds with nonnegative scalar curvature,J. Diff. Geom. 62(2002), 79–125, MR 1987378, Zbl 1071.53018.

[20] E. Witten,A new proof of the positive energy theorem,Comm. Math. Phys.80 (1981), 381–402, MR 0626707, Zbl 1051.83532.

Department of Mathematics Stanford University 450 Serra Mall, Bldg 380 Stanford, CA 94305 E-mail address: [email protected]

Instituto de Matem´atica Pura e Aplicada (IMPA) Estrada Dona Castorina 110 22460-320 Rio de Janeiro, Brazil E-mail address: [email protected]