Minimizers of Caffarelli–Kohn–Nirenberg Inequalities with the Singularity on the

Digital Object Identifier (DOI) 10.1007/s00205-009-0269-y

Minimizers of Caffarelli–Kohn–Nirenberg Inequalities with the Singularity on the

Boundary

Jann-Long Chern & Chang-Shou Lin

Communicated by P. Rabinowitz

Abstract

Let be a bounded smooth domain in R^N, N 3, and D¹a^,2() be the completion of C₀^∞()with respect to the norm:

||u||²a=

|x|^−2a|∇u|²dx.

The Caffarelli–Kohn–Nirenberg inequalities state that there is a constant C >0 such that

(

|x|^−bq|u|^qdx)^2q C

|x|^−2a|∇u|²dx (0.1) for u∈ Da^1,2()and

−∞<a< N−2

2 , 0b−a1, q = 2N

N−2+2(b−a). We prove the best constant for (0.1)

S(a,b;)= inf

u∈Da^1,2\{0}

|x|^−2a|∇u|²dx (

|x|^−bq|u|^qdx)^2q

is always achieved in Da^1,2() provided that 0 ∈ ∂ and the mean curvature H(0) <0, where a,b satisfies

(i)a<b<a+1 and N 3, or (ii)b=a>0 and N 4.

If a =0 and 1>b>0, then the result was proved by Ghoussoub and Robert [12].

Work partially supported by the National Science Council of Taiwan.

1. Introduction

The Caffarelli–Kohn–Nirenberg(CKN) inequalities state that there is a constant C >0 such that for all u∈C₀^∞(R^N), it holds

R^N|x|^−bq|u|^qdx _q²

C

R^N|x|^−2a|∇u|²dx (1.1) for N 3:

− ∞<a< N−2

2 , 0b−a1, q = 2N

N−2+2(b−a). (1.2) For a proof of the CKN inequality and its generalization, see [4] and [15]. Let be a domain in R^N, and D¹a^,2()be the completion of C₀^∞()with respect to the norm

||u||²a=

|x|^−2a|∇u|²dx. (1.3) Then inequality (1.1) can be extended to all functions in Da^1,2(). For a,b and q satisfying (1.2), the best constant of (1.1) is defined by

S(a,b;)= inf

u∈Da^1,2()\{0}Ea,b(u), (1.4) where

Ea,b(u)=

|x|^−2a|∇u|²dx (

|x|^−bq|u|^qdx)^2q.

Obviously, the extremal functions for S(a,b;)are the least-energy solution of the Euler–Lagrangian equation:

−div(|x|^−2a∇u)= |x|^−bqu^q−1, u>0 in ,

u=0 on ∂. (1.5)

In this paper, we want to study the problem whether S(a,b;)can be attained by some function in Da^1,2(). Whenis the whole space R^N, the existence or non- existence of minimizers for (1.4) have been extensively studied for the past 20 years, see [5,6,13,17,20] and references therein. The result can be briefly summarized in the following:

Theorem A. Suppose a,b and q satisfy the condition (1.2). Then minimizers exist for the best constant S(a,b;R^N)if and only if a,b satisfies

either a<b<a+1 or b=a 0. (1.6)

After TheoremA, it is natural to consider this minimization problem for a domainin R^N. By scaling, for anyλ >0 the best constant satisfies

S(a,b;λ)=S(a,b;),

where λ= {λx|x ∈ }. Thus, if 0 ∈ , then S(a,b;) = S(a,b;R^N). Fol- lowing this equality, we can conclude that S(a,b;)is never achieved if 0 ∈ and=R^N. However, if 0∈∂andis C²near 0, Ghoussoub and Robert [11,12] considered the case a=0 and b>0, and proved the following remarkable result.

Theorem B. Let a =0, 0 <b<1, and 0 ∈∂. Suppose the mean curvature H of∂at 0 is negative, then minimizers for the best constant S(0,b;)can be attained.

Throughout the paper, we always assume 0∈ ∂andis C²in a neighbor- hood of 0. Our purpose is to extend the result of Ghoussoub and Robert to cover the whole range of a and b, where a,b satisfy

either(i)a <b<a+1 and N 3, or(ii)b=a>0 and N4. (1.7) Our main result is the following theorem.

Theorem 1.1. Let a,b and q satisfy (1.2) and (1.7). Suppose 0∈∂and the mean curvature H(0) <0. Then the best constant S(a,b;)can be achieved in D¹a^,2().

Comparing Theorem1.1with TheoremAfor the case where N =3 and a = b >0, the minimization problem remains unsolved. We will discuss it in a forth- coming paper. Another remark is that minimizers do not exist in any domain= R^N when a = b 0. See [5,18]. Thus, when 0 ∈ ∂and H(0) < 0, Theo- rem1.1completely solves the existence (or non-existence) of minimizers for CKN inequalities except for N =3 and a=b>0.

Our proof of Theorem1.1relies on the transformation of u by letting

w(x)= |x|^−au(x) for x∈. (1.8) By a straightforward computation, we have for any u∈C₀^∞(),

|x|^−2a|∇u|²dx

=

|x|^−2a(a²|x|^2a−2|∇w(x)|²+2a|x|^2a−2w(x)x

·∇w(x)+ |x|^2a|∇w(x)|²)dx

=

|∇w(x)|²dx−γ

w²(x)

|x|² dx, (1.9)

where

γ =a(N−2−a). (1.10)

The last equality in (1.9) can be obtained by integration by parts:

2a

|x|^2a−2w(x)x· ∇w(x)dx

= −a(N−2+2a)

|x|^2a−2w²(x)dx. (1.11) We note that if a < ^N₂⁻², then both

|∇w|²dx and

w²

|x|² are finite, and by the Hardy inequality we have

N−2 2

2

w²

|x|² dx

|∇w|² dx.

Therefore, if a< ^N−₂², then

u∈ D_a^1,2() if and only ifw∈ H₀¹(), and furthermore,

Ea,b(u)=

|∇w|²−γ

w²

|x|²dx (

w^2∗

|x|^sdx)^2∗2 , (1.12) where s=(b−a)q and 2^∗= _N−2+^2N_2(b−a). It is easy to see s and 2^∗satisfy

0<s<2 and 2^∗= 2(N−s)

N−2 . (1.13)

when 0<b−a <1, and s =0, 2^∗ = _N^2N₋₂ when b=a. From now on, we will denote ²⁽_N^N₋⁻₂^s) by 2^∗(s)for 0s 2. Furthermore, the best constant S(a,b;) can be expressed in terms ofwby

S(a,b, )= inf

w∈H₀¹()\{0}

|∇w|²−γ

w²

|x|²dx (

w^2∗(s)

|x|^s dx)^2∗2(s) , (1.14) and the condition (1.7) can be translated into:

(i)γ < (^N−₂²)² and 0<s<2 for N3, or (ii) s=0 and 0< γ <_N−2

2

2

for N 4. (1.15)

Also by (1.8), u is a solution of Equation (1.5) if and only ifw(x)is a solution of w+γ_|x^w_|2 +^w2_|^∗_x^(s)−1_|s =0 in ,

w=0 on ∂. (1.16)

Therefore, instead of finding minimizers of (1.4), we will study the minimizing problem of (1.14), and prove the following result which is equivalent to Theorem 1.1.

Theorem 1.2. Let 0 ∈ ∂, H(0) < 0, and γ, s satisfy (1.15). Then there is w∈ H₀¹()which attains the best constant of (1.14).

To prove Theorem1.2, we should compare S(a,b;)with S(a,b;R₊^N), where R₊^N is the half-upper space{(x,xN)|xN >0}. It is easy to see that the inequality,

S(a,b;)S(a,b;R₊^N),

holds always. However, for the proof of Theorem1.2we need the strict inequality S(a,b;) <S(a,b;R₊^N). (1.17) In order to derive the strict inequality, we consider a least-energy solution on R₊^N:

w+γ_|x^w_|2 +^w2∗_|x^(s_|s⁾⁻¹ =0 in R₊^N,

w=0 on ∂R₊^N. (1.18)

Under the condition (1.7), the existence of least-energy solutions was proved by Bartsch et al. [2], Lin and Wang [18].

Letα0∈

−^N−₂²,1

be the root of

−α0(N−2+α0)+N−1=γ (1.19) ifγ ∈(0, (^N−₂²)²). Note thatα0(0)=1, α0(N−1)=0 andα0(γ )is decreasing inγ. Ifγ 0, then we simply letα0 = 1. The following theorem is important when (1.17) is concerned. See Section3.

Theorem 1.3. Supposew(x)∈ H₀¹(R₊^N)is a positive solution of (1.18). Then for any smallε >0, there exists C_ε>0 such that

w(x)C_ε|x|^{2−(N+α0)+ε}, and

|∇w(x)|C_ε|x|^{1−(N+α0)+ε} (1.20) for|x|1. Moreover, the following is true.

(i)wis axially symmetric with respect to the xNaxis, and there is a positive constant λsuch that

w(x)= λ

|x|

N−2

w λ²x

|x|²

. (1.21)

(ii)

2γ

R₊^N xN|x|²

|x|⁴ w²(x)dx<

R^N−1|x|²|_∂^∂w_xN(x,0)|²dx. (1.22)

By (1.17), Theorem1.2can be proved through blowup arguments. Although this is a standard procedure, there requires some additional care to go through all details. Usually, blowing up solutions after suitable scaling will converge to a bounded entire solution of (1.18). Indeed, the method applied in [7,8] can work out for the caseγ <0. However, whenγ >0, positive solutions of (1.18) generally are not bounded near the origin, due to the singular term _|x^w_|2. Indeed, this is the case whenγ N−1. Thus, the blowing up method in [7,8] cannot work for the caseγ >0. Hence we need a non-standard way to re-scale solutions. Fortunately, our method can work out for the caseγ >0. See Section5for the case s>0 and Section6for the case s=0. In Section6, we prove Theorem1.2for the case s=0, that is, b=a. In this case, 2^∗= _N^2N₋₂is the Sobolev exponent. Thus, S(a,a;)has to compare not only with S(a,a;R₊^N), but also with the Sobolev best constant SN. Hence as in [3], the dimension plays a crucial role. For N 4, the strict inequality

S(a,a;) <SN

also holds, see Lemma6.1. In a forthcoming paper, we will study the case N =3.

The blowing up argument for the caseγ >0 cannot work for the caseγ <0.

The case forγ <0 is discussed in Section4. In Section2, Theorem1.3is proved.

The best constant S(a,b;)is computed in Section3, where the strict inequality (1.17) is proved.

2. Decay estimate of solutions on R₊^N

In this section, we consider Equation (1.18) on the half space R₊^N: w+γ_|x^w_|2 +^w2_|^∗_x⁽_|^ss⁾⁻¹ =0 in R₊^N,

w=0 on∂R₊^N, (2.1)

whereγ < (^N−₂²)², 0s<2, and 2^∗(s)= ²⁽_N^N₋⁻₂^s). It is well-known that ifw is locally in H¹(R₊^N), thenwis smooth in R₊^N\{0}. In the following, we want to estimatewnear the origin O.

Lemma 2.1. Suppose wis a positive solution of (2.1) in B₁⁺(0) = R₊^N B1(0) andw∈ H¹(B₁⁺(0)). Then for anyδ >0, there exists C_δ>0 such that

w(x)C_δ|x|^α0−δ for |x| 1

2, (2.2)

whereα0is defined in (1.19).

Proof. By multiplyingφ²w^2β−1on Equation (2.1), we have 2β−1

β²

R₊^N

|∇(φw^β)|²dx−γ

R₊^N

φ²w^2β

|x|² dx

R₊^N

φ²w^{2∗(s)+2β−2}

|x|^s dx+O(1),

whereφis a cut-off function. Sinceγ < (^N−₂²)², by the Sobolev–Hardy inequality, we havew^β ∈ H¹(B₁⁺(0))ifβ >1 is close to 1. Thereforew^N2N−2 ∈ L^q(B₁⁺)for some q>1 by the Sobolev embedding.

Step 1. We claim there is a r0>0 such that

B_r0⁺

|∇φ|²dx−γ

B_r0⁺

φ²

|x|²dx−(2^∗(s)−1)

B_r0⁺

w^2∗(s)−2φ²

|x|^s dx0 (2.3) for anyφ∈ H₀¹(Br⁺₀).

By the Hölder inequality,

B_r0⁺

w^{(2∗(s)−2 2)N}

|x|^{s N2} dx

B_r0⁺w^{2N qN−2}dx

1−¹_q

B_r0⁺|x|^{−s N q2} dx _q¹

,

where

1

q =1−(2^∗(s)−2)^N₂ · ^N_2N⁻²

q =1−4−2s

2q < s 2. So,|x|^{−s N q2} ∈L¹(B₁⁺). Thus, for anyε >0, there is a r0>0 such that

⎛

⎝

B_r0⁺

w^{(2∗(s)−2 2)N}

|x|^{s N2} dx

⎞

⎠

2 N

ε.

Therefore, for anyφ∈H₀¹(Br⁺₀),

B_r0⁺

w^2∗(s)−2φ²

|x|^s dx

B_r0⁺(w^2∗(s)−2

|x|^s )^N2 dx ²

N

B_r0⁺φ^N2N−2 dx ^N−2

N

ε||∇φ||²_L2(B_r0⁺). Then by the Hardy inequality we have

B_r0⁺|∇φ|²dx−γ

B_r0⁺ φ²

|x|² dx−(2^∗(s)−1)

B_r0⁺ w^2∗(s)−2

|x|^s φ²dx (1−Cε)||∇φ||²_L2(B_r0⁺)−γ

B_r0⁺ φ²

|x|²dx0, provided thatε >0 is small. And (2.3) is proved.

Set

w(r)= 1 r^N−1

|x|=r wdx. Sincew∈L^N2N−2(B₁⁺), we have

lim_r→0w(r)r^N2−2 =0.

Then∀ε >0, ∃r1=r1(ε)r0such that

w(r1)εr₁^−N−22 , and

B_2r1 |∇w|²dxε.

(2.4)

Step 2. We claim

w(x)Cε|x|^−N−22 for|x| =r1, (2.5) holds for some constant C >0.

To see (2.5) holds, we set

v0(y)=w(r1y)r₁^N−22. Thenv0(y)satisfies

⎧⎨

⎩v 0(y)+γ^v_|⁰_y⁽_|^y2⁾+^v20∗(_|^s_y⁾⁻_|s^1(y) =0, |y|2,

|y|2|∇v0(y)|²dy=

|x|2r1|∇w|²(x)dxε.

Ifεis sufficiently small, then there exists a constant C1independent ofε, such that v0(y)C1for ¹₂ |y| ³₂. Then (2.5) follows from the Harnack inequality for linear elliptic PDE. This finishes the proof of Step 2.

Let z(θ)= ^x_|x^N_|, θ= _|^x_x| ∈S^N−1. Then z(θ)satisfies S^N−1 z(θ)= −(N−1)z(θ), whereS^N−1is the Laplacian operator on S^N−1. Set

U(x)= |x|^−N−22z(θ). (2.6) Then U(x)satisfies

U(x)+

N−2 2

2

+N−1

U(x)

|x|² =0.

Thus r2can be chosen small such that U+γ U

|x|²+U^2∗(s)−1

|x|^s 0 for |x|r2. (2.7) Without loss of generality, we may assume r1r2.

Step 3. We claim

w(x)|x|^−N−22z(θ) for |x|r1. (2.8) From the proof of (2.5), we have

w(x)C1εz(θ)|x|^−N−22 for |x| =r1

for some constant C1independent ofεand r1. Ifεis small, then w(x)U(x) for |x| =r1.

Set

v(x)=U(x)−w(x) for |x|r1.

Suppose⁻= {x∈ Br₁|v(x) <0} = ∅. Then by (2.7),v(x)satisfies v(x)+γv(x)

|x|² +(2^∗(s)−1)w^2∗(s)−2(x)

|x|^s v(x) <0 in ⁻, and we have

⁻|∇v(x)|²dx−γ

⁻

v²(x)

|x|² dx−(2^∗(s)−1)

⁻

w^2∗(s)−2(x)

|x|² v²(x)dx<0, a contradiction to (2.3). Thus⁻is an empty set and we have

w(x)|x|^−N−22z(θ) for |x|r1. Actually, in Step 2, we have proved a stronger result:

w(x)=o(1)|x|^−N2−2 as|x| →0. (2.9) By (2.9), we note that

w^2∗(s)−2(x)

|x|^s =o(1)|x|⁻² as|x| →0.

Thus for anyδ>0 there is r3<r1such that w(x)+(γ+δ)w(x)

|x|² 0 in|x|<r3. (2.10) For anyδ >0, we set

w_δ(x)=C|x|^α0−δz(θ). (2.11) Thenwδ(x)satisfies

w_δ(x)+(γ +δ)w_δ(x)

|x|² =0, whereδis chosen so that

γ+δ= −(α0−δ)(N−2−α0+δ)+N−1.

Choose C =C_δin (2.11) to be large so that

w(x)wδ(x) for|x| =r3. Thus by (2.10) and following the arguments of Step 2, we get

w(x)w_δ(x) for|x|r3. Therefore, Lemma2.1is proved.

Now we are in the position to prove Theorem1.3.

Proof of Theorem1.3. Letw(x)ˆ be the Kelvin transformation ofw, ˆ

w(x)= 1

|x|

2−N

w x

|x|²

. Thenwˆ satisfies (2.1) and

R₊^N

|∇ ˆw|²dx=

R^N₊

|∇w|²dx,

Hencewˆ ∈H¹(R₊^N). By Lemma2.1for anyε >0, there is C_ε >0 such that

| ˆw(x)|C_ε|x|^α0−ε for|x|1.

Going back tow, we have

|w(x)|C_ε|x|^{2−(N+α0)+ε}. (2.12) After (2.12) is proved, the estimate for|∇w(x)|follows from the standard gradient estimate.

To prove the axial symmetry ofw, it suffices to prove w(x1, . . . ,−xN−1,xN)=w(x1, . . . ,xN−1,xN).

The proof will use a variant of the well-known method of moving planes, see [9,10,14,16] and [17]. Let l_θ be the hyperplane {(x1,x2, . . . ,xN−1,xN)|xi ∈ R,i = 1,· · ·,N −2,_x^x_N^N₋₁ = sinθ}, and_θ = {x ∈ R₊^N|xN < xN−1sinθ}.

For any x ∈ θ, we denote x^θ to be the reflection of x with respect to ł_θ, and consider

vθ(x)=w(x^θ)−w(x), x∈θ. Thenvθ satisfies

vθ+γ v_θ

|x|² +c_θ(x)vθ =0, (2.13) where

c_θ(x)= 1

|x|^s

w^2∗(s)−1(x^θ)−w^2∗(s)−1(x)

w(x^θ)−w(x) . (2.14)

By (2.12), there is smallδ >0, such that w^2∗(s)−2

|x|^s =O 1

|x|^2+δ

for large |x|, and

w^2∗(s)−2

|x|^s =O 1

|x|^2−δ

for small |x|.

Choose R large and r0small so that for either|x|<r0or|x|R, the inequality, γ

|x|² +(2^∗(s)−1)w^2∗(s)−2(x)

|x|^s < γ0

1

|x|² (2.15)

holds for someγ0< (^N−₂²)². Then by fixing r0and R, andθis chosen to be small such thatv_θ(x) >0 provided that r0|x|R, we claim for suchθ,

v_θ(x) >0 for x ∈_θ. (2.16) To prove (2.16), we let _θ⁻ = {x ∈ θ|vθ(x) < 0} and suppose⁻_θ =

∅(empty set). Then _θ⁻⊂

θ

Br₀(0) θ

(R₊^N\BR(0)) .

If_θ⁻

Br₀(0)= ∅, then c_θ (2^∗(s)−1)w^2∗(s)−2(x)|x|^−s. Hence we get N−2

2 2

⁻_θ∩B_r0

v_θ²

|x|²dx

_θ⁻∩B_r0

|∇vθ|²dx

_θ⁻∩B_r0

γ

|x|² +(2^∗−1)w^2∗(s)−2(x)

|x|^s

v_θ²dx

< γ0

_θ⁻∩B_r0

v_θ²

|x|²dx, a contradiction. Hence_θ⁻

Br₀(0)= ∅. Similarly,_θ⁻

(R₊^N\BR(0))= ∅can be proved. Thus,⁻_θ = ∅and (2.16) is proved. Let

θ0=sup θ π

2|v_θ˜(x) >0 in_θ˜(x) for all 0θ˜θ .

By applying (2.15) and the process of the method of moving plane, we can show θ0 = ^π₂. Since the process is well-known, we omit the detail. Sinceθ0 = ^π₂, we have

w x^π2

w(x)∀x∈^π₂.

We can do the same process starting from the opposite direction, and show that w(x^π2)w(x)∀x∈^π₂, that is

w (x1, . . . ,xN−2,−xN−1,xN)=w(x1, . . . ,xN−2,xN−1,xN).

To prove (1.21), we letwˆ_α be the Kelvin transformation ofwwith respect to B_α(0), that is,

ˆ w_α(x)=

α

|x| 2−N

w α²x

|x|²

.

Thenwˆ_αsatisfies

ˆwα+_|x^γ_|2wˆα+^wˆα2∗_|x^(s_|s⁾⁻¹ =0 in R₊^N ˆ

wα =0 on∂R₊^N. Set

v_α(x)= ˆw_α(x)−w(x)∀x ∈B_α⁺(0):= {x|xN 0 and|x|α}.

Thenvα(x)satisfies v_α(x)+ γ

|x|²v_α(x)+C_α(x)v_α(x)=0∀x∈ B_α⁺(0),

where C_α(x)=^{wˆ2∗α(s)−}_wˆ¹_α^−w_−w^2∗(s)−1|x|^−s. Forαsmall and x∈ B_α⁺(0)wherevα(x) >

0, we have

C_α(x) ^{(2∗(s)−1)w}_|x|^2∗s ^(s)−2(x) o(1)|x|⁻² as|x| →0, and

|xγ|² +C_α(x)γ0|x|⁻² for someγ0<_N−2

2

2

. Then by using the Hardy inequality, we have, ifαis small,

v_α(x) >0∀x∈B_α⁺(0).

The Hopf lemma implies 0> ∂v_α

∂xN(0, . . . ,0, α)= −

2∂w

∂xN(0, . . . ,0, α)+N−2

α w(0, . . . ,0, α)

,

that is, x

N−2 2

N w(0, . . . ,0,xN)is increasing in xN. Set

α0=sup{α >0|v_α˜(x) >0 in B_α⁺_˜(0)for allα˜ α}.

Since x

N−22

N w(0, . . . ,0,xN)is increasing in xNif xN α0and

xNlim→+∞x

N−2 2

N w(0, . . . ,0,xN)=0,

α0should be a finite number. Argued as the method of moving plane, we can show ˆ

wα0(x)=w(x)∀x∈ B_α⁺₀(0), which is (1.21). The inequality (1.22) will be shown in Corollary2.2. Thus, Theorem1.3is proved.

Corollary 2.2. Supposew∈ H₀¹(R₊^N)is a positive solution of (2.1). Then, letting x=(x,xN)∈R^N, we have

|_∂^∂w_xN(x,0)|²|x|²∈ L¹(R^N−1),

xN|x|²

|x|⁴ w²(x) and^x_|^N_x|^|s^x+^|2²w^2∗(s)(x)∈ L¹(R₊^N). (2.17) Furthermore, the following inequality holds

2γ

R₊^N

xN|x|²

|x|⁴ w²(x)dx <

R^N−1

|x|²|∂w

∂xN|²dx. (2.18)

Proof. By (1.20) of Theorem1.3, we have ∂w

∂xN

²|x|²C_ε|x|^{4−2(N+α0)+ε},

where eitherα0=1 ifγ 0 orα0satisfies α0>−N−2

2 and−α0(N−2+α0)+N−1=γ.

ifγ ∈(0, (^N₂⁻²)²). Obviously ifα0=1, we have 4−2(N+1)= −2(N−1) <

−(N−1)and|_∂^∂w_xN|²|x|²∈L¹(R^N−1). In general, we have 4−2(N+α0)= −2(N−2+α0)

= −

N−2+ (N−2)²−4γ+4(N−1)

! . Thus,

2(N+α0)−4>N−2+"

4(N−1) >N−1. (2.19) Therefore, we have|_∂^∂w_x

N|²|x|² ∈ L¹(R^N−1). The other two terms of (2.17) can be proved in the same way. Since it is a straightforward computation, we skip the details.

To show (2.18), by scaling, we may assumew(x)satisfies w(x)= |x|^2−Nw

x

|x|²

. By direct computation, we have

R^N−1

|x|² ∂w

∂xN(x,0) ²dx=

R^N−1

∂w

∂xN

²dx, (2.20) By multiplying_∂^∂w_x

N on Equation (2.1) and by integration by parts, we have

−

R^N₊

∂w

∂xNwdx= −1 2

R₊^N

∂

∂xN

∂w

∂xN

2

dx=1 2

R^N−1

∂w

∂xN

2

(x,0)dx, and

R^N₊ ∂w

∂xN

γ w

|x|² +^w2_|^∗_x⁽_|^ss⁾⁻¹

dx=

R₊^N

γ 2|x|² ∂

∂xNw²+₂¹∗ 1

|x|^s ∂

∂xNw^2∗(s) dx

=

R₊^N

γx_N

|x|⁴w²+₂∗^s(s) xN

|x|^s+2w^2∗(s) dx

> γ

R₊^N xN

|x|⁴w²(x)dx.

Hence

R^N−1|_∂^∂w_xN|²(x,0)dx>2γ

R₊^N xN

|x|⁴w²(x)dx.

Clearly, (2.18) follows from (2.20) immediately.

3. Calculation of S(a,b;)

In this section, we will calculate S(a,b;)if a,b and q satisfy (1.2) and (1.7).

Under the assumption (1.7), the best constant S(a,b;R₊^N)can be achieved. See Bartsch et al. [2], Lin and Wang [18]. The main result of this section is the following:

Theorem 3.1. Letbe a bounded C¹domain of R^N and C²at 0∈∂. Suppose the mean curvature H(0)at 0 is negative. Then

S(a,b;) <S(a,b;R₊^N)

where a and b satisfy (1.2) and (1.7).

Proof. Without loss of generality, we may assume that in a neighborhood of 0,

∂can be represented by xN = ϕ(x),x = (x1, . . . ,xN−1), whereϕ(0) = 0,

∇ϕ(0)=

∂∂ϕx₁, . . . ,_∂x^∂ϕ_N−1

(0)=0, and the outnormal of∂at 0 is−eN. Letwbe a positive solution of

⎧⎨

⎩

w+γ_|x^w_|2 +S^w2_|^∗_x^(s_|s⁾⁻¹ =0 in R₊^N, w=0 on∂R₊^N, and

R₊^N w^2∗(s)

|x|^s dx=1, (3.1) where S=S(a,b;R₊^N). We note that if (1.7) is satisfied, then s>0, γ < (^N−₂²)² and N 3, or s =0, γ ∈(0, (^N−₂²)²)and N 4. The existence ofwis proved in [2] and [18].

Let U andU be a respective neighborhood of 0 such thatˆ (U)=Br0(0)and (U)ˆ =Br0

2(0), where x=(x,xN)and

(x)=(x,xN−ϕ(x)) for x∈∩U. We define

v_ε(x):=ε^−N−22w (x)

ε

for x ∈∩U and vˆ_ε:=η v_ε in,

whereη∈C^∞₀ (U)is a positive cut-off function withη≡1 inU . Then we haveˆ

S(a,b;)

|∇ ˆv_ε|²dx−γ

vˆ²_ε

|x|²dx #

vˆ^2∗ε ^(s)

|x|^s dx _2∗²

(s)

.