Ground-state solutions of a two-component elliptic system in R

(1)

Ground-state solutions of a two-component elliptic system in R

⁴

with the Sobolev critical exponent

^∗

Yuanze Wu

^a

& Wenming Zou

^b

aSchool of Mathematics, China University of Mining and Technology, Xuzhou 221116, P. R. China

bDepartment of Mathematical Sciences, Tsinghua University, Beijing 100084, P. R. China

Abstract:In this paper, we continue our study in [17] on the following elliptic system:







−ε²∆u1+λ1u1=α1u^p₁⁻¹+µ1u³₁+βu²₂u1 inΩ,

−ε²∆u2+λ2u2=α2u^p₂⁻¹+µ2u³₂+βu²₁u2 inΩ, u₁, u₂>0 inΩ, u₁=u₂= 0 on∂Ω,

whereΩ⊂R⁴is a bounded domain,µ1, µ2>0andβ̸= 0are constants,λ1, λ2, ε >0 are parameters and2 < p < 2^∗ = 4. By further employing some canonical rescalings, we prove by variational methods that the above elliptic system has a ground-state solution forallβ ≥ −√µ₁µ₂ ifεandmin{λ₁, λ₂}both sufficiently small. We also observe that the ground-state solution willvanishwith the rate(max{λ₁, λ₂})^p−2¹ if bothε → 0 andλ₁, λ₂ → 0. These results, together with that in [1, 17], suggest that the subcritical termsα₁u^p₁⁻¹, α₂u^p₂⁻¹ have strong effects on the structure of the ground-state solutions of the above system.

Keywords:Elliptic system; Ground-state solution; Variational method; Sobolev critical exponent; Asymptotic property.

Mathematical Subject Classification 2010: 35B09; 35B33; 35B40; 35J50.

1 Introduction

In this paper, we continue our study in [17] on the following elliptic system:







−ε²∆u1+λ1u1=α1u^p₁⁻¹+µ1u³₁+βu²₂u1 inΩ,

−ε²∆u2+λ2u2=α2u^p₂⁻¹+µ2u³₂+βu²₁u2 inΩ, u₁, u₂>0 inΩ, u₁=u₂= 0 on∂Ω,

(1.1)

∗E-mails: [email protected] (Y. Wu); [email protected] (W. Zou)

(2)

whereΩ⊂R⁴is a bounded domain,µi, αi>0andβ̸= 0are constants,λi, ε >0are parameters and2< p <2^∗= 4.

System (1.1) is critical since the cubic termsu³₁, u³₂and the coupling termsu²₂u1, u²₁u2

are all of critical growth in the sense of the Sobolev embedding. Such systems have been studied extensively in the literature, see, for example, [3–5, 11–17] and the refer- ences therein. By applying the Pohozaev identity, we can see that (1.1) hasnosolution whenΩis star-shaped forλ₁, λ₂ >0andα₁=α₂= 0. In the very recent work [17], by using variational methods, we proved the existence of one spiked solution to (1.1) forε >0sufficiently small, under one of the following three cases:

(1) β >0sufficiently large.

(2) −√µ1µ2< β <0orβ >0sufficiently small.

(3) β≤ −√µ1µ2andα1, α2>0sufficiently small.

Moreover, this solution is also a ground-state solution that has the same concentration behaviors of the ones in [9] asε→0. Our results suggest that the appearances of the subcritical termsα1u^p₁⁻¹, α2u^p₂⁻¹do affect the structure of the solutions of (1.1).

On the other hand, it is well-known that (1.1) withα1 =α2 = 0hasnosolutions in one of the following two cases:

(a) λ₁≤λ₂andµ₂≤β ≤µ₁with one strictly inequality holding.

(b) λ₂≤λ₁andµ₁≤β ≤µ₂with one strictly inequality holding.

Based on our very recent work [17], it is natural to ask thatcan the subcritical terms α1u^p₁⁻¹, α2u^p₂⁻¹affect the structure of the solutions of(1.1)strongly enough such that (1.1)withα1, α2>0has a solution in the above two cases(a)and(b)? Since to the best of our knowledge, this question has not been studied yet in the literature, the main purpose of this paper is to investigate this natural question.

Let us give some words about our strategies in studying the above question. We notice that in the recent work [10], the asymptotic properties of the unique ground- state solution of the following equation,

{ −∆u+εu=|u|^p⁻²u− |u|^q⁻²u inR^N, u∈H¹(R^N)∩L^q(R^N),

as ε → 0⁺, has been studied by variational arguments, where2 < p < q. It has been observed by employing some canonical rescalings, which are associated with the lowest order nonlinear term|u|^p⁻²u, that the asymptotic properties depends strongly on the relation between the orderpand critical Sobolev exponent2^∗. Now, motivated by [10], if we introduce the following canonical rescaling,

vi(x) =

( 1

min{λ1, λ2} )_p−2¹

ui

(

x0+ ε

√min{λ1, λ2}x )

,

(3)

to (1.1), then we will have the following system:











−∆u1+ λ1

min{λ1, λ2}u1=α1u^p₁⁻¹+ (min{λ1, λ2})^4−p^p−2(µ1u³₁+βu²₂u1) inΩ˘ε,

−∆u₂+ λ₂

min{λ1, λ2}u₂=α₂u^p₂⁻¹+ (min{λ₁, λ₂})^4−p^p⁻²(µ₂u³₂+βu²₁u₂) inΩ˘_ε, u₁, u₂>0 inΩ˘_ε, u₁=u₂= 0 on∂Ω˘_ε,

wherex₀∈Ωand

Ω˘ε={y∈R⁴| ε

√min{λ1, λ2}y+x0∈Ω}.

As in [10], by lettingmin{λ1, λ2}sufficiently small,the coupled termsβu²₂u1, βu²₂u1

and the critical termsµ1u³₁, , µ2u³₂can be regard as small perturbations, which means that the lowest order termsα1u^p₁⁻¹, α2u^p₂⁻¹are the main terms in the above system.

Thus, we can find solutions in the cases(a)and(b)under some further assumptions.

If we go further in this direction by employing another canonical rescaling,

˜ vi(x) =

( 1

max{λ₁, λ₂} )_p−2¹

ui

(

x0+ ε

√max{λ₁, λ₂}x )

,

to (1.1), then we will have the following system:











−∆u1+ λ1

max{λ1, λ2}u1=α1u^p₁⁻¹+ (max{λ1, λ2})^4−p^p−2(µ1u³₁+βu²₂u1) inΩ˜ε,

−∆u2+ λ2

max{λ₁, λ₂}u2=α2u^p₂⁻¹+ (max{λ1, λ2})^4−p^p−2(µ2u³₂+βu²₁u2) inΩ˜ε, u1, u2>0 inΩ˜ε, u1=u2= 0 on∂Ω˜ε,

wherex0∈Ωand

Ω˜ε={y∈R⁴| ε

√max{λ₁, λ₂}y+x0∈Ω}.

By letting ε → 0 first and max{λ₁, λ₂} → 0 next, we can obtain the scalar field equations







−∆u1+ ˜λ1u1=α1u^p₁⁻¹ inR^N,

−∆u2+ ˜λ2u2=α2u^p₂⁻¹ inR^N, u1, u2>0 inR^N, u1, u2→0 as|x| →+∞,

whereλ˜_i= lim_max_{_λ₁_,λ₂_}→₀_max_{^λ_λⁱ

1,λ2}. The above formal analysis yields that˜v_i(x) will converge to the unique ground-state solution of the above scalar field equation and uiwill vanish in passing to the limit. These properties seems to be quite different from those ofε→0orλ1, λ2→0, only (cf. [4, 9, 13, 17]).

(4)

Let us give some necessary notations before we state our main results. LetHλ_i,ε,Ω

be the Hilbert space ofH₀¹(Ω)equipped with the inner product

⟨u, v⟩λ_i,ε,Ω=

∫

Ω

ε²∇u∇v+λiuvdx.

Fori= 1,2, sinceλ_i >0andε >0,Hλi,ε,Ωis a Hilbert space and the corresponding norm is given by∥u∥λ_i,ε,Ω=⟨u, u⟩λ¹²_i,ε,Ω. SetHε,Ω=Hλ₁,ε,Ω× Hλ₂,ε,Ω. ThenHε,Ω

is a Hilbert space with the inner product

⟨u,v⟩ε,Ω=

∑2 i=1

⟨u_i, v_i⟩λi,ε,Ω.

The corresponding norm is given by∥u∥ε,Ω = ⟨u,u⟩_ε,Ω¹² . Here, u_i, v_i are the ith component ofu,v, respectively. Define a functional inHε,Ωas follows:

Jε,Ω(u) =

∑2 i=1

(1

2∥ui∥²λi,ε,Ω−αi

p∥ui∥^pΩ,p)−1 4

∑2 i=1

µi∥ui∥⁴Ω,4−β

2∥u²₁u²₂∥Ω,1. Here,∥ · ∥Ω,pis the standard norm inL^p(Ω)for allp≥1which is given by∥u∥Ω,p= (∫

Ω|u|^pdx)¹^p.

Definition 1.1. The vector(u1, u2) = u∈ Hε,Ωis called a nontrivial critical point ofJε,Ω(v)ifJε,Ω^′ (u) = 0inH⁻ε,Ω¹ withu1 ̸≡0andu2 ̸≡ 0. u ∈ Hε,Ωis called a semi-trivial critical point ofJε,Ω(v)ifJε,Ω^′ (u) = 0inH⁻ε,Ω¹ withu1̸≡0oru2̸≡0.

Here,Jε,Ω^′ (u)is the Fr´echet derivative ofJε,Ω(u)andH⁻ε,Ω¹ is the dual space ofHε,Ω. u∈ Hε,Ωis called a positive critical point ofJε,Ω(v)ifuis a nontrivial critical point andui>0for bothi= 1,2.

Clearly,Jε,Ω(u)is of classC²inHε,Ωand the positive critical points ofJε,Ω(u) are the solutions of (1.1).

Definition 1.2. uis called a ground-state solution of (1.1)ifuis a nontrivial solution of(1.1)andJε,Ω(u)≤ Jε,Ω(v)for any nontrivial solutionv.

Now, our main results can be stated as follows.

Theorem 1.1. Letβ ≥ −√µ₁µ₂ andλ_i, α_i, µ_i > 0. LetD₀ > 0 be an absolute constant. Then we have the following.

(1) There exists λ_∗ > 0only dependent on D0,µ1, µ2 andβ such that(1.1)has a ground-state solution(u_1,ε, u_2,ε)for0 < min{λ₁, λ₂} < λ_∗and 0 < ε <

D₀√

min{λ₁, λ₂}.

(2) Letε_n, λⁿ₁, λⁿ₂ →0⁺and(uⁿ₁, uⁿ₂)be the related solution obtained in(1). Sup- poseA= lim_n_→∞√ ^εⁿ

max{λⁿ₁,λⁿ₂} <+∞, then

uⁿ_i(xⁿ_i) =O((max{λⁿ₁, λⁿ₂})^p−2¹ )

(5)

and

vⁿ_i(x) =

( 1

max{λⁿ₁, λⁿ₂} )_p₋¹₂

uⁿ_i (

xⁿ_i +√ εn

max{λⁿ₁, λⁿ₂}x )

converges to somev_istrongly inH¹(R⁴)asn→ ∞, wherev_iis the solution of the following equation

{ −∆u+λ^∗_iu=α_i|u|^p⁻²u inΩ^∗_i,

u∈H₀¹(Ω^∗_i). (1.2)

Herexⁿ_i is, respectively, the maximum point ofuⁿ_i,Ω^∗_i is the limit of Ωi,n={y∈R⁴| ε_n

√max{λⁿ₁, λⁿ₂}y+xⁿ_i ∈Ω}

asn→ ∞andλ^∗_i = limn→∞ λⁿ_i

max{λⁿ₁,λⁿ₂}. Moreover, (i) IfA = lim_n_→∞√ ^εⁿ

max{λⁿ₁,λⁿ₂} > 0, thenΩ^∗_i are bounded and v_i are a ground-state solution of (1.2), respectively.

(ii) IfA = lim_n_→∞√ ^εⁿ

max{λⁿ₁,λⁿ₂} = 0, thenΩ^∗_i = R⁴and v_i is the unique radially ground-state solution of (1.2)forλ^∗_i >0whilevi≡0forλ^∗_i = 0.

Furthermore,

√max{λⁿ₁,λⁿ₂}|xⁿ₁−xⁿ₂|

εn →0asn→ ∞.

Remark 1.1. (a) By introducing the canonical re-scaling given by(2.2), we can re- scale system(1.1)into(2.3). In such system, the cubic nonlinearities and the coupled term can be treated as the perturbation terms forλ1small enough and ε ≪ √

λ1. Here, without loss of generality, we assumeλ1 = min{λ1, λ2}. Thus, under this idea, we can prove by the method of the Nehari manifold that system(1.1)has a ground-state solution for allβ ≥ −√µ1µ2under the condi- tions0< λ1< λ_∗and0< ε < D0

√λ1, which is stated by(1)of Theorem 1.1.

(b) (2)of Theorem 1.1 yields that the scalar field equation(1.2)is the limit equation of system(1.1)if bothε_n→0andmax{λⁿ₁, λⁿ₂} →0with

A= lim

n→∞

ε_n

√max{λⁿ₁, λⁿ₂} <+∞.

Moreover, the ground-state solutions will vanish at the rate of(max{λⁿ₁, λⁿ₂})^p−2¹ . To the best of our knowledge, this is the first result about such asymptotic prop- erty of system(1.1).

This paper is organized as follows. In Section 2, we obtain an existence result of (1.1) by applying the method of the Nehari manifold. In Section 3, we study the concentration behavior of the ground state solution asε→0⁺andλ1, λ2→0⁺.

(6)

2 The existence results

2.1 Some preliminaries

In this section,we are interesting in finding a ground state solution of (1.1) for allβ ≥

−√µ1µ2. Without loss of generality, we assume that0∈Ω,λ1≤λ2andD0= 1. Let Ωε={x∈R⁴|εx∈Ω}.

Then it is easy to see thatΩε → R⁴ as ε → 0⁺. Moreover, it is easy to see that u= (u1, u2)is a solution of (1.1) if and only ifbu= (ub1,ub2)withbui(x) =ui(εx)is a solution of the following system







−∆bu1+λ1bu1=µ1bu³₁+α1ub^p₁⁻¹+βbu²₂bu1 inΩε,

−∆bu2+λ2bu2=µ2bu³₂+α2ub^p₂⁻¹+βbu²₁bu2 inΩε, b

u1,bu2>0 inΩ, ub1=ub2= 0 on∂Ωε,

(2.1)

We also define a canonical rescaling ofuband denote it byu. That is,e e

ui(x) = (1

λ1

)_p−2¹ b ui

( x

√λ1

)

. (2.2)

Clearly, for everyu_i∈ Hλi,ε,Ω, we haveeu_i∈ Hλi

λ1,1,eΩ_ε, where Ωe_ε={x∈R⁴| ε

√λ₁x∈Ω}.

Moreover,ubis a solution of (2.1) if and only ifueis a solution of the following system











−∆eu₁+ue₁=α₁eu^p₁⁻¹+ (λ₁)^4−p^p−2(µ₁ue³₁+βue²₂ue₁) inΩe_ε,

−∆eu₂+λ₂ λ1

e

u₂=α₂eu^p₂⁻¹+ (λ₁)^4−p^p−2(µ₂ue³₂+βue²₁ue₂) inΩe_ε, e

u₁,ue₂>0 inΩe_ε, eu₁=eu₂= 0 onΩe_ε,

(2.3)

By a direct calculation, we have

∫

Ωeε

|∇eui|²dx= (λ1)¹⁻^p−2²

∫

Ωε

|∇bui|²dx= (λ1)¹⁻^p−2² ε⁻²

∫

Ω

|∇ui|²dx (2.4) and ∫

e Ω_ε

|eu_i|^rdx= (λ₁)²⁻^p−2^r

∫

Ω_ε

|bu_i|^rdx= (λ₁)²⁻^p−2^r ε⁻⁴

∫

Ω

|u_i|^rdx (2.5) for bothi= 1,2and all2≤r≤4. Let

E_Ω_e_ε(eu) =

∑2 i=1

(1

2(∥∇eui∥²_Ω_e

ε,2+λi

λ₁∥eui∥²_Ω_e

ε,2)−αi

p∥eui∥^p_Ω_e

ε,p))

−(λ1)^4−p^p−2(1 4

∑2 i=1

µi∥eui∥⁴_Ω_e

ε,4+β

2∥eu²₁eu²₂∥_Ω_e_ε_,1).

(7)

Then it is easy to see thatE_Ω_e_ε(eu)is of classC² inH_∗_,e_Ω_ε = H^λ1

λ1,1,eΩε × H^λ2 λ1,1,eΩε

and positive critical points ofE_e_Ω_ε(u)e are equivalent to solutions of (2.3). Moreover, by (2.4) and (2.5), we have

EeΩε(u) = (λe 1)¹⁻^p−2² J1,Ωε(bu) = (λ1)¹⁻^p−2² ε⁻⁴Jε,Ω(u). (2.6) Definition 2.1. ueis called a ground state solution of(2.3)ifueis a nontrivial solution of(2.3)andEeΩε(u)e ≤ EΩeε(ev)for any nontrivial solutionv.e

By (2.6) and Definition 1.2, ifwe is a ground state solution of (2.3), thenw is a ground state solution of (1.1), wherew= (w1, w2)withwi(x) =λ

1 p−2

1 wei(^√_ε^λ¹x).

2.2 The Nehari manifold

Define the Nehari manifold ofE_Ω_e_ε(u)e as follows:

NΩeε ={eu∈He_∗,eΩε | EΩ^′^∗_j,n(u)u^k= 0, k= 1,2}, whereHe_∗_,e_Ω_ε = (H^λ1

λ1,1,eΩ_ε\{0})×(H^λ2

λ1,1,eΩ_ε\{0}),u¹ = (u₁,0)andu² = (0, u₂).

Clearly, all nontrivial solutions of (2.3) are contained inN_e_Ω_ε. Thus, ifwe ∈ N_Ω_e_εattains the minimum ofE_Ω_e_ε(u)e inN_Ω_e_εandwe is also a positive critical point ofE_Ω_e_ε(u), thene

e

w is a ground state solution of (2.3). Here, we saywe is a positive critical point of E_e_Ω_ε(u), ife we is a critical point ofE_Ω_e_ε(u)e andwe = (we1,we2)withwei>0.

Lemma 2.1. Letβ≥ −√µ1µ2. ThenE_Ω_e_ε(eu)≥0onN_Ω_e_ε. Proof. Sinceβ ≥ −√µ₁µ₂, the functional

∑2 i=1

µi∥euⁿ_i∥⁴_Ω_e

ε,4+ 2β∥(euⁿ_i)²(euⁿ_i)²∥_Ω_e_ε_,1

is nonnegative definite inH_∗_,e_Ω_εby the H¨older inequality. Thus, the conclusion follows immediately from2< p <4and a standard calculation.

By Lemma 2.1,

cε= inf

NΩeε

EΩeε(eu) (2.7)

is well defined.

Lemma 2.2. Let β ≥ −√µ₁µ₂. Then for 0 < ε < √

λ₁, there exists C₀ > 0 independent ofεandλ₁such thatc_ε≤C₀.

Proof. Since √^ελ₁ ≤1, we can see thatB^C∗

2 (0)⊂Ωeε, whereC^∗ =dist(0, ∂Ω). Let Ω1 andΩ2be two bounded domains withΩ1∩Ω2 = ∅andΩi ⊂ B^C^∗

2 (0)for both i= 1,2. Then it is well known that the following equation

{−∆u+λ₂u=α_i|u|^p⁻²u inΩ_i,

u= 0 on∂Ωi

(8)

has a positive ground state solutionui. Denote the energy level ofuibyCi, thenCiis independent ofεandλ1. LetC0 =C1+C2, then byΩ1∩Ω2 =∅andλ1≤λ2, the conclusion immediately follows from (2.7).

By the Ekeland’s variational principle, there exists{eu_n} ⊂ NeΩεsuch that (1) E_Ω_e_ε(eun) =cε+on(1),

(2) EΩeε(ev)≥ EΩeε(eun)−_n¹∥ev−uen∥ε,Ωfor allev∈ NΩeε.

Moreover, by settingueⁿ_i ≡0outsideΩeε, we can regarduen= (euⁿ₁,ueⁿ₂)∈D^1,2(R⁴)× D^1,2(R⁴).

Lemma 2.3. Letβ≥ −√µ₁µ₂. Then there existsλ_∗>0such that

∥∇euⁿ_i∥²_R⁴,2+∥euⁿ_i∥²_R⁴,2≤C withnlarge enough for0< λ1 < λ_∗and0 < ε <√

λ1, whereC >0is a constant independent ofεandλ₁. Moreover,{eu_n} ⊂ NeΩεis also a(P S)sequence ofEΩeε(eu_n) at the energy levelc_ε.

Proof. By Lemma 2.2, we can see fromβ ≥ −√µ1µ2and2< p <4that C0+on(1) ≥ E_Ω_e_ε(eun)−1

pE_Ω_e^′

ε(uen)eun

≥ p−2 2p

∑2 i=1

(∥∇euⁿ_i∥²_Ω_e

ε,2+λi

λ₁∥ueⁿ_i∥²_Ω_e

ε,2).

Thus, we have{euⁿ_i}is bounded inH¹(R⁴)forn. Moreover, byλ₁≤λ₂, we also have

∥∇euⁿ_i∥²_R⁴,2+∥euⁿ_i∥²_R⁴,2≤2p(C0+ 1)

p−2 (2.8)

forn large enough, where ^2p(C_p₋⁰₂⁺¹⁾ > 0 is a constant independent ofε andλ1. It remains to show that{eun} ⊂ N_Ω_e_ε is also a(P S)sequence ofE_Ω_e_ε(eun)at the energy levelcε. Letwe ∈ H_∗_,e_Ω_ε. For everyn ∈ N, we consider the system Ψn(t, l) = 0, whereΨn(t, l) = (Ψⁿ₁(t, l),Ψⁿ₂(t, l))with

Ψⁿ_i(t, l) = ∥∇(tieuⁿ_i +lwei)∥²_Ω_e

ε,2+ λi

λ₁∥tiueⁿ_i +lwei∥²_Ω_e

ε,2−αi∥tiueⁿ_i +lwei∥^p_Ω_e

ε,p

−λ

4−pp−2

1 (µ_i∥t_ieuⁿ_i +lwe_i∥⁴_e_Ω

ε,4+β∥(t₁ueⁿ₁+lwe₁)²(t₂euⁿ₂ +lwe₂)²∥Ωeε,1).

Clearly,Ψ(t, l)is ofC¹. Moreover, since{eu_n} ⊂ N_Ω_e_ε, we also have thatΨ_n(1,0) = 0. By a direct calculation, we have

∂Ψⁿ_i(1,0)

∂ti

= 2(∥∇euⁿ_i∥²_Ω_e

ε,2+λi

λ1∥euⁿ_i∥²_Ω_e

ε,2)−pαi∥euⁿ_i∥^p_e_Ω

ε,p

−λ

4−p p−2

1 (4µ_i∥euⁿ_i∥⁴_Ω_e

ε,4+ 2β∥(euⁿ₁)²(euⁿ₂)²∥_Ω_e_ε_,1)

= −(p−2)αi∥euⁿ_i∥^p_Ω_e

ε,p−2µiλ

4−pp−2

1 ∥euⁿ_i∥⁴_Ω_e

ε,4 (2.9)

(9)

respectively fori= 1,2and

∂Ψⁿ₂(1,0)

∂t1

= ∂Ψⁿ₁(1,0)

∂t2

= −2βλ

4−pp−2

1 ∥(ueⁿ_i)²(ueⁿ_i)²∥_Ω_e_ε_,1 (2.10) Set

Θn = (θ_ijⁿ)i,j=1,2

withθⁿ_ij =^∂Ψⁿⁱ_∂t^(1,0)

j . By (2.8), we can see from the Sobolev inequality that

∥euⁿ_i∥²_R⁴,4≤2p(C₀+ 1)

(p−2)S (2.11)

for nlarge enough, where S is best embedding constant from H¹(R⁴) → L⁴(R⁴) defined by

S= inf{∥∇u∥²_R⁴,2|u∈H¹(R⁴),∥u∥²_R⁴,4= 1}. (2.11) together with{un} ⊂ Nε,Ωand the H¨older inequality, implies

∥∇euⁿ_i∥²_Ω_e

ε,2+ λi

λ₁∥euⁿ_i∥²_Ω_e

ε,2

= αi∥euⁿ_i∥^p_Ω_e

ε,p+λ

4−pp−2

1 (µi∥euⁿ_i∥⁴_Ω_e

ε,4+β∥(ueⁿ_i)²(ueⁿ_i)²∥Ωeε,1)

≤ α_i∥euⁿ_i∥^p_Ω_e

ε,p+λ

4−p p−2

1

2p(C₀+ 1)

(p−2)S (µ_i+|β|)∥euⁿ_i∥²_e_Ω

ε,4 (2.12)

fornlarge enough. Let λ_∗=

( p−2

4p(C0+ 1)(µ1+µ2+|β|) )^p−2_4−p

.

Then for0< λ₁< λ_∗, we have from (2.12) that

∥∇euⁿ_i∥²_Ω_e

ε,2+ λi

ε,2≤2αi∥euⁿ_i∥^p_Ω_e

ε,p

fornlarge enough, which together withλ₁≤λ₂, implies αi∥euⁿ_i∥^p_Ω_e

ε,p≥ (Sp

2 )_p−2^p

(2.13) forn large enough with0 < λ1 < λ_∗. Here, Sp is best embedding constant from H¹(R⁴)→L^p(R⁴)defined by

Sp= inf{∥∇u∥²_R⁴,2+∥u∥²_R⁴,2|u∈H¹(R⁴),∥u∥^p_R4,p= 1}.

(10)

By2< p <4, we have from (2.9)–(2.10) that det(Θn)≥(p−2)²α1α2

(Sp

2 )_p−2^2p

>0 (2.14)

fornlarge enough. Now, we can applying the implicit function theorem and the Tay- lor’s expansion in a standard way (cf. [2]) to show that{eun} ⊂ N_e_Ω_ε is also a(P S) sequence ofE_Ω_e_ε(uen)at the energy levelcεfor0< λ1< λ_∗.

By Lemma 2.3,{euⁿ_i}is bounded inH¹(R⁴)forn. Without loss of generality, we assume thateuⁿ_i ⇀ue^∗_i weakly inH¹(R⁴)asn→ ∞.

Proposition 2.1. Letβ ≥ −√µ1µ2,0 < λ1 < λ_∗and 0 < ε < √

λ1. Then there existsUe = (Ue1,Ue2)withUei>0inΩeεfor bothi= 1,2such thatUe is a critical point ofE_Ω_e_ε(eu)at the energy levelcε.

Proof. Clearly, eu_∗ = (ue^∗₁,ue^∗₂) is a critical point ofEΩeε(eu). Moreover, since Ωe_ε is bounded, we can see from the Sobolev embedding theorem and (2.13) thatueⁿ_i → eu^∗_i strongly inL^r(eΩ_ε)for1≤r <4asn→ ∞andeu^∗_i ̸≡0for bothi= 1,2. Thus, we must haveue_∗∈ N_Ω_e_ε. It follows that

cε+on(1) = E_Ω_e_ε(uen)−1 4E_Ω^′_e

ε(eun)uen

= 1 4

∑2 i=1

(∥∇euⁿ_i∥²_Ω_e

ε,2+ λi

ε,2)−4−p 4p

∑2 i=1

αi∥euⁿ_i∥^p_Ω_e

ε,p

≥ 1 4

∑2 i=1

∥∇eu^∗_i∥²_Ω_e

ε,2+λi

λ₁∥eu^∗_i∥²_Ω_e

ε,2−4−p 4p

∑2 i=1

αi∥eu^∗_i∥^p_Ω_e

ε,p+on(1)

= E_Ω_e_ε(ue_∗)−1 4E_e_Ω^′

ε(ue_∗)eu_∗+on(1)

≥ c_ε+o_n(1).

Hence,E_Ω_e_ε(ue_∗) = c_ε. That is,ue_∗ is a minimum point ofE_e_Ω_ε(u)e onN_Ω_e_ε. Note that Ue = (Ue1,Ue2)withUei=|eu^∗_i|is also a minimum point ofE_Ω_e_ε(u)e onN_Ω_e_ε. Now, by a similar argument as used for (2.14), we can apply the method of Lagrange multipliers in a standard way (cf. [2]) to show thatN_Ω_e_εis a natural constraint inH_∗_,e_Ω_ε. Therefore, by the maximum principle,Ue is a critical point ofE_Ω_e_ε(eu)at the energy levelcεwith Uei>0for bothi= 1,2.

3 Concentration behaviors

In this section, we are interesting in studying the concentration behavior of the ground state solution of (1.1) as ε → 0⁺ and λ = (λ₁, λ₂) → 0. For this purpose, we takeε_n → 0⁺andλ_n = (λⁿ₁, λⁿ₂) → 0as n → ∞. Without loss of generality, we