Central limit theorems for eigenvalues of deformations of Wigner matrices

www.imstat.org/aihp 2012, Vol. 48, No. 1, 107–133

DOI:10.1214/10-AIHP410

© Association des Publications de l’Institut Henri Poincaré, 2012

Central limit theorems for eigenvalues of deformations of Wigner matrices ¹

M. Capitaine

, C. Donati-Martin

and D. Féral

aCNRS, Institut de Mathématiques de Toulouse, Equipe de Statistique et Probabilités, F-31062 Toulouse Cedex 09, France.

E-mail:mireille.capitaine@math.univ-toulouse.fr

bUPMC Univ. Paris 06 and CNRS, UMR 7599, Laboratoire de Probabilités et Modèles Aléatoires, Case 188, 4 palce Jussieu, F-75252 Paris Cedex 05, France. E-mail:catherine.donati@upmc.fr

cInstitut de Mathématiques de Bordeaux, Université Bordeaux 1, 351 Cours de la Libération, F-33405 Talence Cedex, France.

E-mail:delphine.feral@math.u-bordeaux1.fr

Received 24 March 2009; revised 14 December 2010; accepted 27 December 2010

Abstract. In this paper, we study the fluctuations of the extreme eigenvalues of a spiked finite rank deformation of a Hermitian (resp. symmetric) Wigner matrix when these eigenvalues separate from the bulk. We exhibit quite general situations that will give rise to universality or non-universality of the fluctuations, according to the delocalization or localization of the eigenvectors of the perturbation. Dealing with the particular case of a spike with multiplicity one, we also establish a necessary and sufficient condition on the associated normalized eigenvector so that the fluctuations of the corresponding eigenvalue of the deformed model are universal.

Résumé. Dans ce papier, nous étudions les fluctuations des valeurs propres extrémales d’une matrice de Wigner hermitienne (resp.

symétrique) déformée par une perturbation de rang fini dont les valeurs propres non nulles sont fixées, dans le cas où ces valeurs propres extrémales se détachent du reste du spectre. Nous décrivons des situations générales d’universalité ou de non-universalité des fluctuations correspondant au caractère localisé ou délocalisé des vecteurs propres de la perturbation. Lorsque l’une des valeurs propres de la perturbation est de multiplicité un, nous établissons de plus une condition nécessaire et suffisante sur le vecteur propre associé pour que les fluctuations de la valeur propre correspondante du modèle déformé soient universelles.

MSC:60B20; 15A18; 60F05

Keywords:Random matrices; Deformed Wigner matrices; Extremal eigenvalues; Fluctuations; Localized eigenvectors; Universality

1. Introduction

Adding a finite rank perturbation to a GUE matrix, S. Péché [19] pointed out a sharp phase transition phenomenon: ac- cording to the largest eigenvalue of the perturbation, the largest eigenvalue of the perturbed matrix should either stick to the bulk and fluctuate according to the Tracy–Widom (or generalized Tracy–Widom) law or should be extracted away from the bulk and have then fluctuations of Gaussian nature. In the lineage of this work, in a previous paper [8], we have studied the limiting behavior of extremal eigenvalues of finite rank deformations of Wigner matrices. We established their almost sure convergence. The limiting values depend only on the spectrum of the deformationAN

and on the variance of the distribution of the entries of the Wigner matrix. On the contrary the fluctuations of these eigenvalues strongly depend on the eigenvectors ofA_N. Indeed, in the particular case of a rank one diagonal defor- mation whose non-null eigenvalue is large enough, we established a central limit theorem for the largest eigenvalue

1This work was partially supported by the Agence Nationale de la Recherche grant ANR-08-BLAN-0311-03.

which deviates from the rest of the spectrum and proved that the fluctuations of the largest eigenvalue vary with the particular distribution of the entries of the Wigner matrix. Thus, this fluctuations result differs from that of the full rank one deformation case investigated in [10] and [12] since this latter case exhibited universal limiting distributions.

Let us recall these results in the complex setting, having in mind that similar results hold in the real symmetric case. In the following, given an arbitrary Hermitian matrixMof sizeN, we will denote byλ₁(M)≥ · · · ≥λ_N(M)its Nordered eigenvalues; we will denote the centered Gaussian distribution with variancevbyN(0, v).

The random matrices under study are complex Hermitian matrices(MN)_Ndefined on a probability space(Ω,F,P) such that

MN=WN

√N +AN. (1.1)

AN is aN×N deterministic Hermitian matrix of fixed finite rank and whose spectrum does not depend onN. The matrixWN is aN×N Wigner Hermitian matrix such that theN²random variables (W_N)_ii,√

2e((W_N)_ij)_i<j,

√2m((W_N)_ij)_i<jare independent identically distributed with a centered distributionμof varianceσ².

As the rank of theAN’s is assumed to be finite, the Wigner Theorem is still satisfied for the Deformed Wigner model (MN)_N (cf. Lemma 2.2 of [1]): the spectral measure _N¹_N

i=1δ_λi(MN)of MN converges a.s. towards the semicircle lawμ_scwhose density is given by

dμsc

dx (x)= 1 2πσ²

4σ²−x²1_[−2σ,2σ](x). (1.2)

WhenAN≡0, it is well-known that onceμhas a finite fourth moment, the first largest (resp. last smallest) eigenvalues of the rescaled Wigner matrixWN/√

Ntend almost surely to the right (resp. left) endpoint 2σ(resp.−2σ) of the semi- circle support (cf. [1]). The corresponding fluctuations, which have been first obtained by Tracy and Widom [23] in the Gaussian case and then extended by Soshnikov [21] for any symmetric probability measureμhaving sub-Gaussian moments, are governed by the so-called Tracy–Widom distributions. Note that the exponential decay condition (with symmetry assumption) has been replaced by a finite number of moments in [16,20]. Under the subexponential decay assumption, the symmetry assumption onμin [21] was replaced in [22] by the vanishing third moment condition and very recently, Erdös, Yau and Yin [9] proved the edge universality under the subexponential decay assumption alone.

Let us describe how the asymptotic behavior of the extremal eigenvalues of the perturbed Wigner matrix may be affected by the perturbation by considering the particular case of a rank one perturbationAN with non-null eigen- valueθ. For a large class of probability measuresμ, it turns out that the largest eigenvalueλ₁(MN)still tends to the right-endpoint 2σ if θ≤σ whereasλ₁(MN)jumps above the bulk toρ_θ =θ+^σ_θ² if θ > σ. This was proved by Péché in her pionnering work [19] whenμis Gaussian, extended in [10] whenμis symmetric and has sub-Gaussian moments but in the particular case of the full rank one deformationANgiven by

(AN)_ij=θ/N for all 1≤i, j≤N (1.3)

and finally established in [8] for generalANwhenμis symmetric and satisfies a Poincaré inequality.

Moreover, considering the perturbation matrix defined by (1.3), Féral and Péché [10] proved that the fluctuations ofλ₁(MN)are the same as in the Gaussian setting investigated in [19] and in this sense are universal. Here is their result whenθ > σ:

Proposition 1.1. Ifμis symmetric and has sub-Gaussian moments

√N

λ1(MN)−ρθ

_L

−→N 0, σ_θ²

,

whereσ_θ=σ

1−^σ_θ2².

The proof of this result relies on the computations of moments ofMN of high order (depending onN) and the knowledge of the fluctuations in the Gaussian case, established by Péché [19].

On the other hand, for the strongly localized perturbation matrix of rank 1 given by AN=diag(θ,0, . . . ,0)

withθ > σ, we proved in [8] that the fluctuations ofλ₁(MN)vary with the particular distribution of the entries of the Wigner matrix so that this phenomenon can be seen as an example of a non-universal behavior:

Proposition 1.2. Letμbe symmetric and satisfy a Poincaré inequality.Define

c_θ= θ²

θ²−σ² and v_θ=1 2

m4−3σ⁴ θ²

+ σ⁴

θ²−σ², (1.4)

wherem₄:= x⁴dμ(x).Then cθ

√N

λ1(MN)−ρθ

_L

−→

μ∗N(0, vθ)

. (1.5)

In the present paper, we consider perturbationsANof higher rank of Wigner matrices associated to some symmetric probability measureμsatisfying a Poincaré inequality. The a.s. convergence of the extreme eigenvalues has already been described in [8] (see Theorem3.1below). Whenever the largest eigenvalues ofMN are extracted away from the bulk, we describe their fluctuations which depend on the localization of the eigenvectors ofAN, as already seen in the above rank 1 examples. We investigate two quite general situations for which we exhibit a phenomenon of different nature. To explain this, let us focus on the largest eigenvalue θ1 ofAN. We assume thatθ1> σ so that the largest eigenvalues ofMNconverges a.s towardsρ_θ1=θ1+^σ_θ1² >2σ.

First, when the eigenvectors associated to the largest eigenvalue θ₁ of AN are localized, we establish that the limiting distribution in the fluctuations ofλ_i(MN),1≤i≤k₁, aroundρ_θ1 is not universal and we give it explicitely in terms of these eigenvectors and of the distribution of the entries of the Wigner matrix, see Theorem3.2.

Secondly, if the eigenvectors are sufficiently delocalized, we establish the universality of the fluctuations of λ_i(MN),1≤i≤k₁, see Theorem3.3.

Actually, in the rank one case, this study allows us to exibit a necessary and sufficient condition on a normalized eigenvector of AN associated to the largest eigenvalueθ1 for the universality of the fluctuations (see Theorem3.4 below). Moreover if such an eigenvector ofAN is not localized but does not satisfy the criteria of universality, the largest eigenvalue ofMN may fluctuate according to a mixture ofμand normal distributions generalizing (1.5). We will describe some of such intermediate situations.

We detail the definition of localization/delocalization and these results in the Section3.

Our approach is close to the proof of (1.5), with more involved computations and are in the spirit of the works of [7] and [18]. It is valid in both real and complex settings. Actually, we assume that the eigenvectors associated to the largest eigenvalues ofAN belong to a subspace generated byk (=k(N ))canonical vectors ofC^N and the method requires thatN−k−→ ∞(and even√^k

N−→0). In particular, this approach does not cover the case of Proposition1.1 studied by [10] wherek=N.

The Deformed Wigner matrix model may be seen as the additive analogue of the spiked population models. These are random sample covariance matrices(S_N)_N defined byS_N=_N¹Y_N^∗Y_N whereY_N is ap×N complex (resp. real) matrix (withN andp=p_N of the same order asN → ∞) whose entries satisfy first four moments conditions; the sample column vectors are assumed to be i.i.d., centered and of covariance matrix a deterministic Hermitian (resp.

symmetric) matrixΣ_phaving all but finitely many eigenvalues equal to one. In their pioneering article on that topic [6], Baik–Ben Arous–Péché pointed out a phase transition phenomenon for the fluctuations of the largest eigenvalue ofSNaccording to the largest eigenvalue ofΣp, in the complex Gaussian setting; their results were extended in [18]

to the real case when the largest eigenvalue ofΣ_p is simple and sufficiently larger than 1 and in [17] to singular Wishart matrices. In the non-Gaussian case, the fluctuations of the extreme eigenvalues have been recently studied by Bai–Yao [5] and Féral–Péché [11].

The paper is organized as follows. In Section2, we present the matricial models under study and the notations that will be used throughout the paper. In Section3, we present the main results of this paper. We give a summary of our approach in Section4. Section5is devoted to the proofs of Theorems3.2,3.3and3.4. Finally, we recall some basic facts on matrices, a CLT for random sesquilinear forms and prove some technical results in anAppendix.

2. Model and notations

The random matrices under study are complex Hermitian (or real symmetric) matrices(MN)Ndefined on a probability space(Ω,F,P)such that

MN=WN

√N +AN, (2.1)

where the matricesWN andANare defined as follows:

(i) WN is a N ×N Wigner Hermitian (resp. symmetric) matrix such that the N² random variables (W_N)_ii,

√2e((WN)ij)i<j,√

2m((WN)ij)i<j (resp. the ^{N (N}₂⁺¹⁾ random variables √¹

2(WN)ii,(WN)ij,i < j) are indepen- dent identically distributed with a symmetric distributionμof varianceσ²and satisfying a Poincaré inequality; the latter condition means that there exists a positive constantCsuch that for anyC¹functionf:R→Csuch thatf and fare inL²(μ),

V(f )≤C f²dμ withV(f )=E(|f −E(f )|²).

Note that whenμis Gaussian,W_Nis a GU(O)E(N×N, σ²)matrix.

(ii) AN is a deterministic Hermitian (resp. symmetric) matrix of fixed finite rankrand built from a family ofJ fixed real numbersθ₁>· · ·> θ_J independent ofN with some j₀such that θ_j0 =0. We assume that the non-null eigenvaluesθ_j ofANare of fixed multiplicityk_j(with

j=j0k_j=r). LetJ_+σ be the number ofj’s such thatθ_j> σ.

We denote byk_+σ:=k₁+ · · · +k_J+σ. We introducek≥k_+σ as the minimal number of canonical vectors among the canonical basis(ei;i=1, . . . , N )ofC^N needed to express all the eigenvectors associated to the largest eigenvalues θ₁, . . . , θ_J+σ ofAN. Without loss of generality (using the invariance of the distribution of the Wigner matrixWNby conjugation by a permutation matrix), we can assume that thesek₊σ eigenvectors belong to Vect(e1, . . . , ek).

All along the paper we assume thatk√ N.

Let us now fixj such that 1≤j≤J_+σ and letU_kbe a unitary matrix of sizeksuch that diag

U_k^∗, I_N−k

ANdiag(Uk, I_N−k)=diag

θ_jI_kj, (θ_lI_kl)_l≤J+σ,l=j, Z_N−k+σ

, (2.2)

whereZ_N−k+σ is an Hermitian matrix with eigenvalues strictly smaller thanθ_J+σ.

DefineKj =Kj(N )as the minimal number of canonical vectors among (e1, . . . , ek)needed to express all the orthonormal eigenvectorsv_i^j, 1≤i≤k_j, ofAN associated toθ_j. Without loss of generality, we can assume that the v_i^j, 1≤i≤k_j, belong to Vect(e1, . . . , e_Kj). Considering now the vectorsvⁱ_jas vectors inC^Kj, we define theK_j×k_j matrix

U_Kj×kj:=

v^j₁, . . . , v_k^j

j

(2.3)

namelyUK_j×k_j is the upper left corner ofUk of sizeKj×kj. It satisfies U_K^∗

j×kjU_Kj×kj=I_kj. (2.4)

All along the paper, the parametert is such thatt =4 (resp. t=2) in the real (resp. complex) setting and we let m₄:= x⁴dμ(x).

Given an arbitrary Hermitian or symmetric matrixMof sizeN, we will denote byλ₁(M)≥ · · · ≥λ_N(M)itsN ordered eigenvalues.

3. Main results

We first recall the a.s. convergence of the extreme eigenvalues. Define ρ_θj =θ_j+σ²

θj

. (3.1)

Observe thatρ_θj >2σ (resp.<−2σ) whenθ_j> σ (resp.<−σ) (andρ_θj = ±2σifθ_j= ±σ).

For definiteness, we setk1+ · · · +kj−1:=0 ifj =1. In [8], we have established the following universal conver- gence result.

Theorem 3.1 (a.s. behaviour). LetJ_+σ (resp.J_−σ)be the number of j’s such thatθ_j> σ (resp.θ_j<−σ).

(1) ∀1≤j≤J₊σ,∀1≤i≤kj, λk₁+···+k_j−1+i(MN)−→ρθ_j a.s.

(2) λ_k1+···+kJ

+σ+1(MN)−→2σ a.s.

(3) λ_{k1+···+kJ−J}

−σ(MN)−→ −2σ a.s.

(4) ∀j≥J−J₋σ+1,∀1≤i≤kj,λk₁+···+k_j−1+i(MN)−→ρθ_j a.s.

3.1. Fluctuations aroundρ_θj

From Theorem3.1, for all 1≤i≤k_j,λ_{k1+···+kj−1+i}(MN)converges toρ_θj a.s. We shall describe their fluctuations in the extreme two cases:

Case (a): Localization of the eigenvectors associated toθ_j: The sequenceK_j(N )is bounded, sup

N

K_j(N )= ˜K_j

and the the upper left cornerU_K˜

j×kj ofU_kof sizeK˜_j×k_j converges towards some matrixU˜_K˜

j×kj whenN goes to infinity.

Case (b): Delocalization of the eigenvectors associated toθj: Kj=Kj(N )→ ∞whenN→ ∞andUksatisfies

kj

maxp=1 Kj

maxi=1

(U_k)_ip−→0 asN→ ∞. (3.2)

The main results of our paper are the following two theorems. Letcθ_j be defined by c_θj= θ_j²

θ_j²−σ². (3.3)

In Case (a) (which includes the particular setting of Proposition1.2), the fluctuations of the corresponding rescaled largest eigenvalues ofMN are not universal.

Theorem 3.2. In Case(a):thek_j-dimensional vector c_θj√

N

λ_{k1+···+kj−1+i}(MN)−ρ_θj

;i=1, . . . , kj

converges in distribution to(λ_i(V_kj×kj);i=1, . . . , kj)whereλ_i(V_kj×kj)are the ordered eigenvalues of the matrix V_kj×kjof sizek_jdefined in the following way.LetW_K˜

j be a Wigner matrix of sizeK˜_j with distribution given byμ(cf.

(i))andH_K˜

j be a centered Hermitian Gaussian matrix of sizeK˜_j independent ofW_K˜

j with independent entriesH_pl, p≤l,with variance

⎧⎪

⎨

⎪⎩

vpp=E H_pp²

=₄^t_m4−3σ⁴

θ_j²

+₂^t_θ2^σ4

j−σ², p=1, . . . ,K˜j, vpl=E

|Hpl|²

=_θ2^σ4

j−σ², 1≤p < l≤ ˜Kj.

(3.4)

Then,V_kj×kj is thek_j×k_j matrix defined by V_kj×kj= ˜U^∗_˜

K_j×k_j(W_K˜

j+H_K˜

j)U˜_K˜

j×kj. (3.5)

Case (b) exhibits universal fluctuations.

Theorem 3.3. In Case(b):thekj-dimensional vector c_θj√

N

λ_{k1+···+kj−1+i}(MN)−ρ_θj

;i=1, . . . , kj

converges in distribution to(λ_i(V_kj×kj);i=1, . . . , kj)where the matrixV_kj×kj is distributed as theGU(O)E(kj× k_j, ^θ

2 jσ² θ_j²−σ²).

Remark 3.1. Note that sinceμ is symmetric,analogue results can be deduced from Theorems3.2and3.3dealing with the lowest eigenvalues ofMNand theθj such thatθj<−σ.

Example.

A_N=diag

A_p(θ1), θ2I_k2,0N−p−k2

,

whereA_p(θ₁)is a matrix of sizepdefined byA_p(θ₁)_ij=θ₁/p,withθ₁, θ₂> σ,p√

N.Thenk=p+k₂,k₁=1, K₁=p,K₂=k₂.Forj =1,we are in Case(a)ifpis bounded and in Case(b)ifp=p(N )→ +∞.Forj=2,we are in Case(a).

3.2. Further result for a spikeθ_j> σof multiplicity1

Dealing with a spikeθ_j > σ with multiplicity 1, it turns out that Case (b) is actually the unique situation where universality holds since we establish the following.

Theorem 3.4. Ifk_j=1,θ_j> σ,then the fluctuations ofλ_{k1+···+kj−1+1}(MN)are universal,namely

√N

λ_{k1+···+kj−1+1}(MN)−ρ_{θj L}

−→N

0,t 2σ_θ²

j

, whereσ_θj =σ

1−σ² θ_j² if and only if

maxl≤Kj

(U_k)_l1→0 whenN→ ∞. (3.6)

Moreover, our approach allows us to describe the fluctuations ofλ_{k1+···+kj−1+1}(MN)for some particular situ- ations where the corresponding eigenvector ofAN is not localized but does not satisfy the criteria of universality maxl≤K_j|(Uk)l1| →0 (that is somehow for intermediate situations between Cases (a) and (b)). Letmbe a fixed inte- ger number. Assume that for anyl=1, . . . , m,(U_k)_l1is independent ofN, whereas maxm<l≤K_j|(U_k)_l1| →0 when N goes to infinity. We will prove at the end of Section 5thatcθ_j

√N (λk₁+···+k_j−1+1(MN)−ρθ_j)converges in dis- tribution towards the mixture ofμ-distributed or Gaussian random variablesm

i,l=1ailξil+N in the complex case,

1≤l≤i≤ma_ilξ_il+N in the real case, whereξ_il, (i, l)∈ {1, . . . , m}²,N are independent random variables such that

• for any(i, l)∈ {1, . . . , m}², the distribution ofξ_il isμ;

• a_il=

⎧⎪

⎪⎨

⎪⎪

⎩

√2

(U_k)_l1(U_k)_i1

ifi < l,

√t

(U_k)l1(U_k)i1

ifi > l, t

2(U_k)l1² ifi=l;

• N is a centered Gaussian variable with variance t

4

[m4−3σ⁴]m

l=1|(Uk)l1|⁴

θ_j² + t

2 σ⁴ θ_j²−σ²+t

2

1− _m

l=1

(U_k)_l1²2 σ².

4. Sketch of the approach

Before we proceed to the proof of Theorems3.2and3.3, let us give the sketch of our approach which are similar in both cases. To this aim, we define for any random variableλ,

ξ_N(λ)=c_θj√

N (λ−ρ_θj) (4.1)

withc_θj given by (3.3). We also setkˆ_j−1:=k₁+ · · · +k_j−1with the convention thatkˆ₀=0.

The reasoning made in the setting of Proposition1.2(for whichk=k_+σ =1) relies (following ideas previously developed in [7] and [18]) on the writing of the rescaled eigenvalue ξ_N(λ₁(MN)) in terms of the resolvent of an underlying non-Deformed Wigner matrix. The conclusion then essentially follows from a CLT on random sesquilinear forms established by J. Baik and J. Silverstein in the Appendix of [8] (which corresponds to the following TheoremA.2 in the scalar case). In the general case, to prove the convergence in distribution of the vector(ξN(λ_kˆ

j−1+i(MN));i= 1, . . . , kj), we will extend, as [5], the previous approach in the following sense. We will show that each of these rescaled eigenvalues is an eigenvalue of ak_j×kj random matrix which may be expressed in terms of the resolvent of aN−k×N−kDeformed Wigner matrix whose eigenvalues do not jump asymptotically outside[−2σ;2σ]; then, the matrix V_kj×kj will arise from a multidimensional CLT on random sesquilinear forms. Nevertheless, due to the multidimensional situation to be considered now, additional considerations are required. Let us give more details.

Consider an arbitrary random variableλwhich converges in probability towardsρ_θj. Then, applying factorizations of type (A.1), we prove thatλis an eigenvalue ofMN iffξN(λ)is (on some event having probability going to 1 as N→ ∞) an eigenvalue of akj×kj matrixXˇk_j,N(λ)of the form

Xˇ_kj,N(λ)=V_kj,N+R_kj,N(λ), (4.2)

whereV_kj,Nconverges in distribution towardsV_kj×kj and the remaining termR_kj,N(λ)turns out to be negligible. Now, whenk_j >1, since the matrixXˇ_kj,N(λ) (in (4.2)) depends onλ, the previous reasoning withλ=λ_kˆ

j−1+i(MN)for any 1≤i≤k_j does not allow us to readily deduce that thek_j normalized eigenvaluesξ_N(λ_kˆ

j−1+i(MN)),1≤i≤k_j, are eigenvalues of a same matrix of the formV_kj,N+o_P(1)and then that

ξ_N λ_kˆ

j−1+i(MN)

;1≤i≤k_j

=

λ_i(V_kj,N);1≤i≤k_j

+o_P(1). (4.3)

Note that the authors do not develop this difficulty in [5], pp. 464–465. Hence, in the last step of the proof (Step 4 in Section5), we detail the additional arguments which are needed to get (4.3) whenkj>1.

Our approach will cover Cases (a) and (b) and we will handle both cases once this will be possible. In fact, the main difference appears in the proof of the convergence in distribution of the matrixV_kj,N which gives rise to the

“occurrence or non-occurrence” of the distributionμin the limiting fluctuations and then justifies the non-universality (resp. universality) in Case (a) (resp. Case (b)).

The proof is organized in four steps as follows. In Steps 1 and 2, we explain how to obtain (4.2): we exhibit the matrixXˇk_j,N and bring its leading termVk_j,N to light in Step 2. We establish the convergence in distribution of the matrixVk_j,N in Step 3. Step 4 is devoted to the concluding arguments of the proof.

5. Proofs of Theorems3.2,3.3and3.4

As far as possible, we handle both the proofs of Theorem3.2and Theorem3.3. We will proceed in four steps. First, let us introduce a few notations.

For any matrixM∈MN(C), we denote by Tr (resp. trN) the classical (resp. normalized) trace. For a rectangular matrix,Mis the operator norm ofMandMHS:=(Tr(MM^∗))^1/2the Hilbert–Schmidt norm.

For an Hermitian matrix, we denote by Spect(M)the spectrum ofM. Forz∈C\Spect(M),GM(z)=(zIN−M)⁻¹ denotes the resolvent ofM(we suppress the indexMwhen there is no confusion). We have the following:

Forx > λ₁(M); G(x)≤ 1

x−λ₁(M). (5.1)

For am×q matrixB(orB) and some integers 1≤p≤mand 1≤l≤q, we denote respectively by[B]_p×l,[B]_p×l, [B]_p×land[B]_p×lthe upper left, upper right, lower left and lower right corner of sizep×lof the matrixB. Ifp=l, we will often replace the indicesp×lbypfor convenience. Moreover ifp=m, we may replaceorby→and orby←. Similarly ifl=q, we may replaceorby↑andorby↓.

For simplicity in the writing we will define thek×k, resp.N−k×N−k, resp.k×N−kmatrixW_k, resp.W_N−k, resp.Y, by setting

WN=

W_k Y Y^∗ W_N−k

. (5.2)

GivenB∈MN(C), we will denote byB˜ theN×N matrix given by B˜ :=diag

U_k^∗, I_N−k

Bdiag(Uk, I_N−k)=

B˜_k B˜_k×N−k B˜_N−k×k B˜_N−k

.

One obviously has thatB˜_N−k=B_N−k.

In this way, we define the matricesM˜N,W˜NandA˜N. In particular, we notice from (2.2) that A˜N=diag

θ_jI_kj, (θ_lI_kl)_l≤J+σ,l=j, Z_N−k+σ

=

A˜k A˜k×N−k

A˜_N−k×k A_N−k

. (5.3)

Note also that sinceA_N−kis a submatrix ofZ_N−k+σ, all its eigenvalues are strictly smaller thanσ. Let 0< δ < (ρ_θj−2σ )/2. For any random variableλ, define the events

Ω_N⁽¹⁾(λ)=

λ1

WN

√N +diag(Uk, IN−k)diag(0k_+σ, ZN−k_+σ)diag

U_k^∗, IN−k

<2σ+δ;λ > ρθ_j−δ

,

Ω_N⁽²⁾=

λ₁ WN−k

√N +A_N−k

≤2σ+δ

and

Ω_N(λ)=Ω_N⁽¹⁾(λ)∩Ω_N⁽²⁾. (5.4)

OnΩ_N(λ), neitherλnorρ_θj are eigenvalues ofM_N−k:=^W√N_N^−k +A_N−k, thus the resolventG(x)ˆ ofM_N−k is well defined atx=λandx=ρθ_j.

Note that from Theorem3.1, for any random sequenceΛNconverging towardsρθ_j in probability,

N−→ ∞lim P

Ω_N(Λ_N)

=1.

Let us now introduce onΩ_N(λ)some auxiliary matrices that will be of basic use to the proofs.

B_k,N=W_k+ 1

√N

YG(ρˆ _θj)Y^∗−(N−k)σ² θ_jI_k

, (5.5)

V_k+σ,N:=

U_k^∗B_k,NU_k

k₊σ, (5.6)

τN(λ)= 1

N(λ−ρθ_j)YG(λ)ˆ G(ρˆ θ_j)²Y^∗, (5.7)

φN= −1 N

YG(ρˆ θ_j)²Y^∗−σ²TrG(ρˆ θ_j)²Ik

, (5.8)

ψ_N= −σ²N−k N

trN−kG(ρˆ θ_j)²− 1 θ_j²−σ²

Ik, (5.9)

c_θjD_k,N(λ)=τ_N(λ)+φ_N+ψ_N, (5.10)

T_N(λ)=

U_k^∗

W_k+ 1

√NYG(λ)Yˆ ^∗

U_k

k₊σ×(k−k₊σ)

, (5.11)

Δ_N(λ)=

U_k^∗YG(λ)ˆ A˜_N−k×k

k_+σ×(k−k_+σ), (5.12)

Γ_{k+σ×k−k+σ}(λ)=T_N(λ)+Δ_N(λ), (5.13)

Qk,N(λ):= ˜M_k+ ˜M_k×N−kG(λ)ˆ M˜_N−k×k, (5.14)

Σ_k−k+σ(λ)=

Qk,N(λ)

k−k₊σ −λI_k−k+σ−1

. (5.15)

Note that we will justify thatΣ_k−k+σ(λ)is well defined in the course of the proof of Proposition5.1below. Finally, set

Xk_+σ,N(λ)=

U_k^∗Bk,NUk

k₊σ +√ Ndiag

0k_j, (θl−θj)Ik_l, l=1, . . . , J₊σ, l=j +ξ_N(λ)

U_k^∗D_k,N(λ)U_k

k₊σ

+ σ²

θ_j²−σ² ξ_N(λ)

c_θj k N − k

√N σ² θ_j

Ik_+σ

− 1

√NΓ_{k+σ×k−k+σ}(λ)Σ_k−k+σ(λ)Γ_{k+σ×k−k+σ}(λ)^∗. (5.16) Step 1: We show that an eigenvalue ofMN is an eigenvalue of a matrix of sizek_+σ. More, precisely, we have:

Proposition 5.1. For any random variableλand anyk₊σ×k₊σ random matrixΔk_+σ,onΩN(λ),λis an eigenvalue ofM˜N+diag(Δk₊σ,0)iffξ_N(λ)is an eigenvalue ofXk₊σ,N(λ)+√

N Δ_k+σ whereXk₊σ,N(λ)is defined by(5.16).

Moreover,thek−k_+σ×k−k_+σ matrixΣ_k−k+σ(λ)defined by(5.15)is such that Σk−k_+σ(λ)≤1/(ρθ_j −2σ−2δ).

Proof. Letλbe a random variable. OnΩ_N(λ), det(MN−λIN)=det(M˜N−λIN)

=det

M˜_k−λI_k M˜_k×N−k M_N−k×k M_N−k−λI_N−k

=det(MN−k−λI_N−k)detM˜_k−λI_k+ ˜M_k×N−kG(λ)ˆ M˜_N−k×k .

The last equality in the above equation follows from (A.1). Since onΩ_N(λ),λis not an eigenvalue ofM_N−k, we can deduce thatλis an eigenvalue ofM˜N if and only if it is an eigenvalue of

Qk,N(λ)= ˜Mk+ ˜Mk×N−kG(λ)ˆ M˜N−k×k.