www.imstat.org/aihp 2010, Vol. 46, No. 3, 677–707
DOI:10.1214/09-AIHP329
© Association des Publications de l’Institut Henri Poincaré, 2010
Limiting spectral distribution of XX matrices
Arup Bose
a, Sreela Gangopadhyay
aand Arnab Sen
baStatistics and Mathematics Unit, Indian Statistical Institute, 202 B. T. Road, Kolkata 700108, INDIA.
E-mail:bosearu@gmail.com;abose@isical.ac.in
bDepartment of Statistics, UC Berkeley, 367 Evans Hall Berkeley, CA 94720, USA Received 5 August 2008; revised 23 June 2009; accepted 16 July 2009
Abstract. The methods to establish the limiting spectral distribution (LSD) of large dimensional random matrices includes the well-known moment method which invokes the trace formula. Its success has been demonstrated in several types of matrices such as the Wigner matrix and the sample covariance matrix. In a recent article Bryc, Dembo and Jiang [Ann. Probab.34(2006) 1–38]
establish the LSD for random Toeplitz and Hankel matrices using the moment method. They perform the necessary counting of terms in the trace by splitting the relevant sets into equivalence classes and relating the limits of the counts to certain volume calculations. Bose and Sen [Electron. J. Probab.13(2008) 588–628] have developed this method further and have provided a general framework which deals with symmetric matrices with entries coming from an independent sequence.
In this article we enlarge the scope of the above approach to consider matrices of the formAp=1nXXwhereXis ap×nmatrix with real entries. We establish some general results on the existence of the spectral distribution of such matrices, appropriately centered and scaled, whenp→ ∞andn=n(p)→ ∞andp/n→ywith 0≤y <∞. As examples we show the existence of the spectral distribution whenXis taken to be the appropriate asymmetric Hankel, Toeplitz, circulant and reverse circulant matrices.
In particular, wheny=0, the limits for all these matrices coincide and is the same as the limit for the symmetric Toeplitz derived in Bryc, Dembo and Jiang [Ann. Probab.34(2006) 1–38]. In other cases, we obtain new limiting spectral distributions for which no closed form expressions are known. We demonstrate the nature of these limits through some simulation results.
Résumé. Une des méthodes pour obtenir la limite des distributions spectrales (LSD) des grandes matrices aléatoires est la fameuse méthode des moments, basée sur la formule des traces. Son succès a été clairement établi pour différents types de matrices telles que les matrices de Wigner et les matrices de covariance. Dans un article récent, Bryc, Dembo et Jiang [Ann. Probab.34(2006) 1–38]
ont obtenu la LSD pour des matrices de Toeplitz et de Hankel en utilisant cette méthode. Ils arrivent à estimer les traces des moments de telles matrices en séparant les différents termes par classes d’équivalence et en reliant les asymptotiques des dénombrements afférents avec les calculs de certains volumes. Bose et Sen [Electron. J. Probab.13(2008) 588–628] ont développé cette idée et ont donné un cadre général pour traiter de matrices symmétriques dont les entrées viennent d’une suite indépendante.
Dans cet article, nous généralisons cette approche pour considérer des matrices de la formAp= 1nXXoùXest une matricep× navec des entrées réelles. Nous démontrons un résultat général d’existence de la LSD de telles matrices, correctement recentrées et rééchelonnées, quandpetntendent vers l’infini de telle façon quep/ntende versy∈(0,∞). Par exemple, nous montrons l’existence de la LSD quandXest la matrice asymétrique de Hankel, de Toeplitz, circulante ou circulante inverse. En particulier, quandy=0, les limites correspondent à celles obtenues par Bryc, Dembo et Jiang [Ann. Probab.34(2006) 1–38]. Sinon, nous obtenons de nouvelles lois limites pour lesquelles aucune expression explicite n’est connue. Nous étudions ces lois par quelques simulations.
MSC:60F05; 60F15; 62E20; 60G57
Keywords:Large dimensional random matrix; Eigenvalues; Sample covariance matrix; Toeplitz matrix; Hankel matrix; Circulant matrix; Reverse circulant matrix; Spectral distribution; Bounded Lipschitz metric; Limiting spectral distribution; Moment method; Volume method; Almost sure convergence; Convergence in distribution
1. Introduction
1.1. Random matrices and spectral distribution
Random matrices were introduced in mathematical statistics by Wishart [20]. In 1950 Wigner successfully used ran- dom matrices to model the nuclear energy levels of a disordered particle system. The seminal work, Wigner [19] laid the foundation of this subject. This theory has grown into a separate field now, with application in many branches of sciences that includes areas as diverse as multivariate statistics, operator algebra, number theory, signal processing and wireless communication.
Spectral properties of random matrices is in general a rich and attractive area for mathematicians. In particular, when the dimension of the random matrix is large, the problem of studying the behavior of the eigenvalues in the bulk of the spectrum has arisen naturally in many areas of science and has received considerable attention. SupposeApis a p×prandom matrix. Letλ1, . . . , λp∈R(orC) denote its eigenvalues. When the eigenvalues are real, our convention will be to always write them in ascending order. The empirical spectral measureμpofApis the random measure onR (orC) given by
μp= 1 p
p i=1
δλi, (1)
whereδx is the Dirac delta measure atx. The random probability distribution function onR(or C) corresponding toμp, is known as theEmpirical Spectral Distribution(ESD) ofAp. We will denote it byFAp.
If{FAp}converges weakly (asp tends to infinity), either almost surely or in probability, to some (nonrandom) probability distribution, then that distribution is called theLimiting Spectral Distribution(LSD) of {Ap}. Proving existence of LSD and deriving its properties for general patterned matrices has drawn significant attention in the literature. We refer to Bai [1] and Bose and Sen [6] for information on several interesting situations where the LSD exists and can be explicitly specified.
Possibly the two most significant matrices whose LSD have been extensively studied, are the Wigner and the sample covariance matrices. For simplicity, let us assume for the time being that all random sequences under consideration are independent and uniformly bounded.
1.Wigner matrix. In its simplest form, the Wigner matrixWn(s)[18,19], of ordernis ann×nsymmetricmatrix whose entries on and above the diagonal are i.i.d. random variables with zero mean and variance one. Denoting those i.i.d. random variables by{xij: 1≤i≤j}, we can visualize the Wigner matrix as
Wn(s)=
⎡
⎢⎢
⎣
x11 x12 x13 . . . x1(n−1) x1n x12 x22 x23 . . . x2(n−1) x2n
...
x1n x2n x3n . . . xn(n−1) xnn
⎤
⎥⎥
⎦. (2)
It is well known that almost surely,
LSD ofn−1/2Wn(s)is thesemicircle lawW, (3)
with the density function pW(s)= 2π1√
4−s2 if|s| ≤2,
0 otherwise. (4)
2.Sample covariance matrix (S matrix). Suppose {xj k: j, k=1,2, . . .}is a double array of i.i.d. real random variables with mean zero and variance 1. The matrix
Ap(W )=n−1WpWp whereWp= (xij)
1≤i≤p,1≤j≤n (5)
is called a sample covariance matrix (in short anSmatrix). LetIpdenote the identity matrix of orderp. The following results are well known. See [1] or [6] for moment method and Stieltjes transform method proofs. To have the historical idea of the successive development of these results see [3] for (a) and [9,11,13,17,21,22] for (b).
(a) Ifp→ ∞andp/n−→0 then almost surely, LSD of
n p
Ap(W )−Ip
is the semicircle law given in (4) above. (6)
(b) Ifp→ ∞andp/n−→y∈(0,∞)then almost surely,
LSD ofAp(W )is the Marˇcenko–Pastur lawMPygiven in (8) below. (7) TheMarˇcenko–Pastur law,MPy, has a positive mass 1−1y at the origin ify >1. Elsewhere it has a density:
MPy(x)= 2π xy1 √
(b−x)(x−a) ifa≤x≤b,
0 otherwise, (8)
wherea=a(y)=(1− √y)2andb=b(y)=(1+ √y)2.
Supposep=n. ThenWnabove is a square matrix with i.i.d. elements, and the Wigner matrix Wn(s) is obtained fromWnby retaining the elements above and on the diagonal ofWnand letting symmetry dictate the choice of the rest of the elements ofWn(s). We loosely say thatWn is theasymmetricversion ofWn(s). It is interesting to note the following:
(i) The LSD ofn−1/2Wn(s)in (3) and that of
n
p(1nWpWp −Ip)in (6) are identical.
(ii) IfW andMare random variables obeying respectively, the semicircle law and the Marˇcenko–Pastur law with y=1, thenMD=W2in law. That is, the LSD ofn−1/2Wn(s)and that of 1nWpWp bears this squaring relationship when p/n→1.
1.2. The problem and its motivation
In view of the above discussion, it thus is natural to study the LSD of matrices of the formAp(X)=(1/n)XpXp where Xp is a p×n suitably patterned (asymmetric) random matrix. Asymmetry is used loosely. It just means thatXpis not necessarily symmetric. In particular one may ask the following questions.
(i) Supposep/n→y,0< y <∞. When does the LSD ofAp(X)=n1XpXp exist?
(ii) Supposep/n→0. When does the LSD of
n
p(Ap(X)−Ip)exist? Note that unlike theSmatrix, the mean of Ap(X)is not in general equal toIpand hence other centering/normalisations may also need to be allowed.
(iii) SupposeXpis ap×n(asymmetric) patterned matrix andAp(X)=1nXpXpfor which the limit in (ii) holds.
Call itA. Now consider the square symmetric matrixXn(s)obtained fromXpby lettingp=nand symmetrising it by retaining the elements ofXp above and on the diagonal and allowing symmetry to dictate the rest of the elements.
Suppose asn→ ∞, the LSD ofn−1/2X(s)n exists. Call itB. In general, is there any relation betweenBandA?
A further motivation to study the LSD of1nXpXpcomes from wireless communications theory. Many results on the information-theoretic limits of various wireless communication channels make substantial use of asymptotic random matrix theory, as the size of the matrix increases. An extended survey of results and works in this area may be found in [16]. Also see [15] for a review of some of the existing mathematical results that are relevant to the analysis of the properties of random matrices arising in wireless communications. The study of the asymptotic distribution of the singular valuesis seen as an important aspect in the analysis and design of wireless communication channels.
A typical wireless communication channel may be described by the linear vector memoryless channel:
y=Xpθ+ε,
whereθis then-dimensional vector of the signal input,yis thep-dimensional vector of the signal output, and thep dimensional vectorεis the additive noise.Xp, in turn, is thep×nrandom matrix, generally with complex entries.
Silverstein and Tulino [15] emphasize the asymptotic distribution of the squared singular-values of Xp under various assumptions on the joint distribution of the random matrix coefficients wherenandptend to infinity while
the aspect ratio,p/n→y,0< y <∞. In their model, generally speaking, the channel matrixXpcan be viewed as Xp=f (A1, A2, . . . , Ak)where{Ai}are some independent random (rectangular) matrices with complex entries, each having its own meaning in terms of the channel. In most of the cases they studied, someAi have all i.i.d. entries, while the LSD of the other matricesAjA∗j, j =iare assumed to exist. Then the LSD ofXpX∗pis computed in terms of the LSD ofAjA∗j, j =i.
Specifically, certain CDMA channels can be modeled (e.g. see [16], Chapter 3, and [15]) asXp=CSAwhereC is ap×prandom symmetric Toeplitz matrix,Sis a matrix with i.i.d. complex entries independent of C, andAis ann×ndeterministic diagonal matrix. One of the main theorems in [15] establishes the LSD ofXp=CSA for randomp×p matrixC, not necessarily Toeplitz, under the added assumption that the LSD of the matricesCC exists. WhenCis a random symmetric Topelitz matrix, the existence of the LSD ofCCis immediate from the recent work of Bryc, Dembo and Jiang [7] and, Hammond and Miller [10]. This also motivates us to study the LSD ofXpXp matrices in some generality whereXpis not necessarily square and symmetric.
1.3. The moment method
We shall use themethod of momentsto study this convergence. To briefly describe this method, suppose{Yp}is a sequence of random variables with distribution functions{Fp}such thatE(Yph)→βhfor every positive integerh, and {βh}satisfiesCarleman’s condition(see [8], page 224):
∞ h=1
β2h−1/2h= ∞. (9)
Then there exists a distribution functionF, such that for allh, βh=βh(F )=
xhdF (x) (10)
and{Yp}(or equivalently{Fp}) converges toF in distribution. We will often in short writeβhwhen the underlying distributionF is clear from the context.
Now suppose{Ap}is a sequence ofp×psymmetric matrices (with possibly random entries), and let, by a slight abuse of notation,βh(Ap)denote the hth moment of the ESD of Ap. Suppose there is a sequence of nonrandom {βh}∞h=1satisfying Carleman’s condition such that,
(M1) First moment condition: For everyh≥1,E[βh(Ap)] →βhand (M2) Second moment condition: For everyh≥1, Var[βh(Ap)] →0.
Then the LSD is identified by{βh}∞h=1and the convergence to LSD holds in probability. This convergence may be strengthened to almost sure convergence by strengthening (M2), for example, by replacing variance by the fourth moment and showing that
(M4) Fourth moment condition: For everyh≥1,E[βh(Ap)−E(βh(Ap))]4=O(p−2).
For any symmetricp×pmatrixAwith eigenvaluesλ1, . . . , λp, thetrace formula βh(A)=p−1
p i=1
λhi =p−1Tr Ah
is invoked to compute moments. Moment method proofs are known for the results on Wigner matrix and the sample covariance matrix given earlier. See for example, [6].
1.4. Brief overview of our results
LetZbe the set of all integers. Let{xα}α∈Z be aninput sequencewhereZ=ZorZ2and{xα}are independent with Exα=0 andEx2α=1. LetLp:{1,2, . . . , p} × {1,2, . . . , n=n(p)} →Zbe a sequence of functions which we call
link functions. For simplicity of notation, we will writeLforLp. We shall later impose suitable restriction on the link functionsLpand sufficient probabilistic assumptions on the variables. Consider the matrices
Xp=
(xLp(i,j ))
1≤i≤p,1≤j≤n and Ap=Ap(X)=(1/n)XpXp.
The functionLpdefines an appropriate pattern and we may call these matricespatternedmatrices. The assumptions on the link function restrict the number and manner in which any element of the input sequence may occur in the matrix.
We shall consider two different regimes:
Regime I. p→ ∞, p/n→ y ∈(0,∞). In this case, we consider the spectral distribution of Ap =Ap(X)= n−1XpXp.
Regime II. p→ ∞, p/n→0. In this case we consider the spectral distribution of (np)−1/2(XpXp −nIp)= n
p(Ap−Ip)whereIpis the identity matrix of orderp.
We will use the following assumptions on the input sequences in two regimes.
Assumption R1. In Regime I, {xα} are independent with mean zero and variance one, and are either uniformly bounded or identically distributed.
Assumption R2. In Regime II, {xα}are independent with mean zero and variance1. Further,λ≥1 is such that p=O(n1/λ)andsupαE(|xα|4(1+1/λ)+δ) <∞for someδ >0.
The moment method requires all moments to be finite. However, in RegimeI, our basic assumption will be that the input sequence has finite second moment. To deal with this, we first prove a truncation result which shows that in RegimeI, under appropriate assumption on the link function, without loss of generality, we may assume the input sequence to be uniformly bounded. The situation in RegimeIIis significantly more involved and there we assume existence of higher moments.
We then establish a negligibility result which implies that for nonzero contribution to the limiting moments, only summands in the trace formula which are “pair matched” matter. See next section for details on “matching.” The existence of the limit of the sum of pair matched terms and then the computation of the limit establishes the LSD.
Quite interestingly, under reasonable assumptions on the allowed pattern,if the limit of empirical moments exist, they automatically satisfy Carleman’s condition and thus ensures the existence of the LSD. However, we are unaware of any general existing method of imposing suitable restrictions on the link function to guarantee the existence of limits of moments.
As examples of our method, we letXp to be the nonsymmetric Toeplitz, Hankel, reverse circulant and circulant matrices. We show that the LSD exists in both RegimesIandII, thereby answering questions (i) and (ii) in the previous section affirmatively for these matrices. The LSD in RegimeIare all new.
However, in RegimeII, the LSD of all four matrices above are identical to the LSD obtained by Bryc, Dembo and Jiang [7] for thesymmetricToeplitz matrixTn(s). This implies that the answer to question (iii) is not straightforward and needs further investigation.
Closed form expressions for the LSD or for its moments do not seem to be easily obtainable. We provide a few simulations to demonstrate the nature of the LSD and it is an interesting problem to derive detailed properties of the LSD in any of these cases.
1.5. Examples
In this section we will consider some specific nonsymmetric matrices of interest. In particular, letXpbe the asymmet- ric versions of the four matrices Toeplitz, Hankel, circulant and reverse circulant. Then it turns out that in RegimeII, the LSD for
n
p(Ap(Xp)−Ip)exists and is identical to the LSD forn−1/2Tn(s)obtained by Bryc, Dembo and Jiang
[7] for the symmetricn×nToeplitz matrixTn(s). In RegimeI, the LSD forAp(Xp)exist but they are all different.
The proofs of the results given below are presented in Section5.
1.5.1. The asymmetric Toeplitz matrix
Then×nsymmetric Toeplitz matrix is defined asTn(s)=((x|i−j|)). It is known that if the input sequence{xi}has mean zero and variance one and is either independent and uniformly bounded or, is i.i.d., then
n−1/2Tn(s)converges weakly almost surely toLT (say).
See for example [6,7] and [10]. It may be noted that the moments ofLT may be represented as volumes of certain subsets of the unit hypercubes but are not known in any explicit form.
Let
Tp=T = (xi−j)
p×n, Ap(T )=n−1TpTp. Note thatTpis the nonsymmetric Toeplitz matrix.
Theorem 1. (i) [RegimeI]Assume R1holds.If pn →y∈(0,∞)then empirical spectral distributionFAp(T ) con- verges in distribution almost surely to a nonrandom distribution which does not depend on the distribution of{xi}.
(ii) [RegimeII]Assume R2holds.ThenF
n
p(Ap(T )−Ip)
converges in distribution toLT almost surely.
1.5.2. The asymmetric Hankel matrix
Suppose{xi, i=0,±1,±2, . . .}is an independent input sequence. LetHn(s)=((xi+j))be then×nsymmetric Hankel matrix. It is known that if{xi}has mean zero and variance one and is either uniformly bounded or i.i.d., then
Fn−1/2Hn(s)converges weakly almost surely.
See for example [6] and [7].
Now letH=Hp×nbe the asymmetric Hankel matrix where the(i, j )th entry isxi+jifi≥j andx−(i+j )ifi < j.
LetHp(s)=((xi+j))p,nbe the rectangular Hankel matrix with symmetric link function. Let Ap(H )=n−1HpHp and Ap
H(s)
=n−1Hp(s)Hp(s). We then have the following theorem.
Theorem 2. (i) [RegimeI]Assume R1holds.Ifpn →y∈(0,∞)thenFAp(H )converges almost surely to a nonrandom distribution which does not depend on the distribution of{xi}.
(ii) [RegimeII]Assume R2holds.ThenF
n
p(Ap(H )−Ip)
converges almost surely toLT.The same limit continues to hold ifAp(H )is replacedAp(H(s)).
1.5.3. The asymmetric reverse circulant matrix
ThesymmetricReverse CirculantRn(s)has the link functionL(i, j )=(i+j )modn. The LSD ofn−1/2R(s)n has been discussed in [5] and [6]. In particular it is known that if{xi}are independent with mean zero and variance 1 and are either (i) uniformly bounded or (ii) identically distributed, then the LSDRofn−1/2R(s)n exists almost surely and has the density,
fR(x)= |x|exp(−x2), −∞< x <∞ with moments
β2h+1(R)=0 and β2h(R)=h! for allh≥0.
LetRp(s)be thep×nmatrix with link functionL(i, j )=(i+j )modnandRp=Rp×nbe theasymmetricversion ofRp(s)with the link function
L(i, j )= (i+j )modn fori≥j,
−
(i+j )modn
fori < j. (11)
So, in effect, the rows ofRpare the firstprows of an (asymmetric) Reverse Circulant matrix. Let Ap(R)=n−1RpRp and Ap
R(s)
=n−1R(s)p Rp(s). We then have the following theorem.
Theorem 3. (i) [RegimeI]Assume R1holds.Ifpn→y∈(0,∞)thenFAp(R)converges almost surely to a nonrandom distribution which does not depend on the distribution of{xi}.
(ii) [RegimeII]Assume R2holds.ThenF
n
p(Ap(R)−Ip)
converges almost surely toLT.The same limit continues to hold ifAp(R)is replacedAp(R(s)).
1.6. The asymmetric circulant matrix
The square circulant matrix is well known in the literature. Its eigenvalues can be explicitly computed and are closely related to the periodogram. Its LSD is the bivariate normal distribution. See for example [5]. Its symmetric version is treated in [6] and [14]. Let
Cp=Cp×n=
(xL(i,j ))
whereL(i, j )=(n−i+j )modn, and Ap(C)=p−1CpCp.
Theorem 4. (i) [RegimeI]Assume R1holds.Ifpn→y∈(0,∞)thenFAp(C)converges almost surely to a nonrandom distribution which does not depend on the distribution of{xi}.
(ii) [RegimeII]Assume R2holds.ThenF
n
p(Ap(C)−Ip)
converges almost surely toLT. 2. Basic notation, definitions and assumptions
Examples of link functions that correspond to matrices of interest are as follows:
0. Covariance matrix:Lp(i, j )=(i, j ).
1. Asymmetric Toeplitz matrix:Lp(i, j )=i−j.
2. Asymmetric Hankel matrix:Lp(i, j )=sgn(i−j )(i+j )where sgn(l)=1 ifl≥0,
−1 ifl <0. (12)
3. Asymmetric Reverse Circulant:Lp(i, j )=sgn(i−j )(i+j )modn.
4. Asymmetric Circulant:Lp(i, j )=(n+j−i)modn.
For any setG, #Gand|G|will stand for the number of elements ofG. We shall use the following general assump- tions on the link function.
Assumptions on link functionL.
A. There exists positive integer such that for anyα∈Zandp≥1:
(i) #{i: Lp(i, j )=α} ≤ for allj∈ {1,2, . . . , n}and (ii) #{j: Lp(i, j )=α} ≤ for alli∈ {1,2, . . . , p}.
A. For anyα∈Zandp≥1, #{i: Lp(i, j )=α} ≤1for allj∈ {1,2, . . . , n}.
B. kpαp=O(np)where kp=#
α: Lp(i, j )=α,1≤i≤p,1≤j≤n
, αp=max
α∈ZL−p1(α).
Assumption A stipulates that with increasing dimensions, the number of times any fixed input variable appears in any given row or column, remains bounded. Assumption Astipulates that no input appears more than once in any column. Assumption B makes sure that no particular input appears too many times in the matrix. Clearly Aimplies A(i) but B is not related to either A or A. Consider the link functionLp(i, j )=(1,1)ifi=j andLp(i, j )=(i, j )if i =j. This link function satisfies A(ii) and Abut not B. On the other hand ifLp(i, j )=1 for all(i, j )then it satisfies B but not A. It is easy to verify that the link functions listed above satisfy these assumptions. It may also be noted that the symmetric Toeplitz link function doesnotsatisfy Assumption A.
Towards applying the moment method, we need a few notions, most of which are given in details in [6] in the context of symmetric matrices. These concepts and definitions will remain valid in RegimeIIalso, with appropriate changes.
The trace formula. LetAp=n−1XpXp.Then thehth moment of ESD ofApis given by p−1TrAhp=p−1n−h
1≤i1,i2,...,ih≤n
xLp(i1,i2)xLp(i3,i2)· · ·xLp(i2h−1,i2h)xLp(i1,i2h). (13) Circuits. Any functionπ:{0,1,2, . . . ,2h} →Z+is said to be acircuitif
(i) π(0)=π(2h),
(ii) 1≤π(2i)≤p∀0≤i≤hand (iii) 1≤π(2i−1)≤n∀1≤i≤h.
Thelengthl(π )ofπis taken to be(2h). A circuit depends onhandpbut we will suppress this dependence.
Matched circuits. Let ξπ(2i−1)=L
π(2i−2), π(2i−1)
, 1≤i≤h and ξπ(2i)=L
π(2i), π(2i−1)
, 1≤i≤h.
These will be calledL-values.
A circuitπis said to have anedge of ordere(1≤e≤2h)if it has anL-value repeated exactlyetimes. Anyπwith alle≥2 will be calledL-matched(in shortmatched). For any suchπ, given anyi, there is at least onej =isuch thatξπ(i)=ξπ(j ).
Define for any circuitπ, Xπ=
h i=1
xξπ(2i−1)xξπ(2i). (14)
Due to the mean zero and independence assumption, ifπhas at least one edge of order one thenE(Xπ)=0.
Equivalence relation on circuits. Two circuitsπ1andπ2of same length are said to be equivalent if theirLvalues agree at exactly the same pairs(i, j ). That is, iff{ξπ1(i)=ξπ1(j ) ⇔ξπ2(i)=ξπ2(j )}.This defines an equivalence relation between the circuits.
Words. Equivalence classes arising from the above equivalence relation may be identified with partitions of {1,2, . . . ,2h}– to any partition we associate aword w of length l(w)=2h of letters where the first occurrence of each letter is in alphabetical order. For example, ifh=3, then the partition{{1,3,6},{2,4},{5}}is represented by the wordababca. Let|w|denote the number of different letters that appear inw.
The notion of ordereedges, matching, nonmatching forπ, carries over to words in a natural manner. For instance, the wordababais matched. The wordabcadbaais nonmatched, has edges of order 1, 2 and 4 and the corresponding partition is{{1,4,7,8},{2,6},{3},{5}}. As pointed out, it will be enough to consider only matched words since E(Xπ)=0 for nonmatched words. The following fact is obvious.
Word count.
#{w: wis of length 2hand has only order 2 edges with|w| =h} =(2h)!
2hh!. (15)
The classΠ (w). Letw[i]denote theith entry ofw. The equivalence class corresponding towwill be denoted by Π (w)=
π: w[i] =w[j] ⇔ξπ(i)=ξπ(j ) .
Ifπ∈Π (w), then clearly the number of distinctL-values equals|w|. Notationally,
#
ξπ(i): 1≤i≤2h
= |w|.
Note that now, with the above definition, we may rewrite the trace formula as p−1TrAhp=p−1n−h
π:πcircuit
Xπ=p−1n−h
w
π∈Π (w)
Xπ,
where the first sum is taken over all words with length(2h). After taking expectation, only the matched words survive and
E
p−1TrAhp
=p−1n−h
wmatched
π∈Π (w)
EXπ.
To deal with (M4), we need multiple circuits. The following notions will be useful for dealing with multiple circuits:
Jointly matched and cross-matched circuits.kcircuitsπ1, π2, . . . , πkare said to bejointly matchedif eachL-value occurs at least twice across all circuits. They are said to becross-matched if each circuit has at least oneL-value which occurs in at least one of the other circuits.
The classΠ∗(w). Define for any (matched) wordw, Π∗(w)=
π: w[i] =w[j] ⇒ξπ(i)=ξπ(j )
. (16)
Note thatΠ∗(w)⊇Π (w). However, as we will see,Π∗(w)is equivalent toΠ (w)for asymptotic considerations, but is easier to work with.
Vertex and generating vertex. Each π(i)of a circuit π will be called avertex. Further,π(2i),0≤i≤hwill be calledeven verticesorp-verticesandπ(2i−1),1≤i≤hwill be calledodd verticesorn-vertices.
A vertexπ is said to begenerating if eitheri=0 orw[i]is the position of thefirstoccurrence of a letter. For example, ifw=abbcabthenπ(0), π(1), π(2), π(4)are generating vertices.
We will call an odd generating vertexπ(2i−1)Type I ifπ(2i)is also an even generating vertex. Otherwise we will callπ(2i−1)a Type II odd generating vertex.
Obviously, number of generating vertices inπis|w| +1. By Assumption A onLfunction given earlier, a circuit of a fixed length is completely determined,up to a finitely many choicesby its generating vertices. Hence, in RegimeI under the assumption A, we obtain the simple but crucial estimate
Π∗(w)=O n|w|+1
.
Let[a]denote the integer part ofa.For Regime II, we will further assume A. In that case, it shall be shown that Π∗(w)=O
p1+[(|w|+1)/2]n[|w|/2] .
3. RegimeI
In Regime I,p→ ∞ such that p/n→y ∈(0,∞) and we consider the spectral distribution of Ap=Ap(X)= n−1XpXp. There are two main theorems in this section. Theorem5is proved under Assumption B. We show that the
input sequence may be taken to be bounded without loss of generality. This allows us to deal with bounded sequences in all examples later. This result is proved sketchily in [6], with special reference to the sample covariance matrix. We provide a detailed proof for clarity and completeness. Theorem6is proved under Assumptions A and B. We show that the ESD of{Ap(X)}is almost surely tight and any subsequential limit is sub Gaussian. Invoking the trace formula, we show that the LSD exists iff the moments converge. Further, in the limit, only pair matched terms potentially contribute.
To prove Theorem5, we will need the following notion and result.
Thebounded Lipschitz metricdBLis defined on the space of probability measures as:
dBL(μ, ν)=sup fdμ−
fdν: f∞+ fL≤1
(17) where
f∞=sup
x
f (x), fL=sup
x =y
f (x)−f (y)/|x−y|. Recall that convergence indBLimplies weak convergence of measures.
The following inequalities provide estimate of the metric distancedBL in terms of trace. Proofs may be found in [2] or [1] and uses Lidskii’s theorem (see [4], page 69).
Lemma 1. (i)SupposeA, Baren×nsymmetric real matrices.Then
dBL2
FA, FB
≤ 1
n n
i=1
|λi(A)−λi(B)| 2
≤ 1 n
n i=1
λi(A)−λi(B)2
≤ 1
nTr(A−B)2. (18)
(ii)For any square matrixS,letFS denote its empirical spectral distribution.SupposeXandY are p×nreal matrices.LetA=XXandB=Y YThen
dBL2
FA, FB
≤
1 p
p i=1
λi(A)−λi(B)2
≤ 2
p2Tr(A+B)Tr
(X−Y )(X−Y )
. (19)
Theorem 5. Letp→ ∞,pn →y∈(0,∞)and Assumption B holds so thatαpkp=O(np).Suppose for every bounded mean zero and variance one i.i.d.input sequence,Fn−1XpXpconverges weakly to some fixed nonrandom distributionG almost surely.Then the same limit continues to hold if the input sequence is i.i.d.with mean zero and variance one.
Proof. Without loss of generality we shall assume thatZ=Zand also writeXforXp. Fort >0, denote μ(t )=E
x0I
|x0|> t
= −E x0I
|x0| ≤t and let
σ2(t)=Var x0I
|x0| ≤t
=E x02I
|x0| ≤t
−μ(t )2.
SinceE(x0)=0 andE(x20)=1, we haveμ(t )→0 andσ (t )→1 ast→ ∞andσ2(t)≤1. Define bounded random variables
xi∗=xiI(|xi| ≤t )+μ(t )
σ (t ) =xi− ¯xi
σ (t ) wherex¯i=xiI
|xi|> t
−μ(t )=xi−σ (t )xi∗. (20) It is easy to see thatE(x¯20)=1−σ2(t)−μ(t )2→0 ast tends to infinity. Further,{xi∗}are i.i.d. bounded, mean zero and variance one random variables. Let us replace the entriesxLp(i,j )of the matrixXp by the truncated version
xL∗
p(i,j )(respectivelyx¯Lp(i,j )) and denote this matrix byY (respectivelyX¯p). By triangle inequality and (19), dBL2
Fn−1XpXp, Fn−1Y Y
≤2dBL2
Fn−1XpXp, Fn−1σ (t )2Y Y
+2dBL2
Fn−1Y Y, Fn−1σ (t )2Y Y
≤ 2 p2n2Tr
XpXp +σ (t )2Y Y Tr
Xp−σ (t )Y
Xp−σ (t )Y + 2
p2n2Tr
Y Y+σ (t )2Y Y Tr
σ (t )Y−Y
σ (t )Y−Y .
To tackle the first term on the right side above, Tr
XpXp+σ (t )2Y Y
= p i=1
n k=1
xL2
p(i,k)+σ (t )2 p i=1
n k=1
xL∗2
p(i,k)
≤αpkp kp
i=1xi2 kp
+
p i=1
n k=1
(xLp(i,k)− ¯xLp(i,k))2
≤αpkp kp
i=1xi2 kp
+αnkp
kp
i=1xi2I{|xi|> t} kp
+ |μ(t )|αpkp kp
i=1|xi| kp
+αpkp
kp
i=1x¯i2 kp
.
Therefore usingαpkp=O(np)and the SLLN, we can see that(np)−1(Tr(XpXp +σ (t )2Y Y))is bounded. Now, 1
npTr
Xp−σ (t )Y
Xp−σ (t )Y= 1
npTr(X¯pX¯p)≤ 1 npαpkp
1 kp
kp
i=1
¯ x2i
,
which is bounded byCE(x¯i2)almost surely, for some constantC. Here we use the conditionαpkp=O(np)and SLLN for { ¯xi2}. SinceE(x¯i2)→0 ast→ ∞we can make the right side tend to 0 almost surely, by first lettingp tend to infinity, and then lettingttend to infinity. This takes care of the first term.
To tackle the second term, T2= 2
n2p2 Tr
σ (t )2
Y Y+Y Y Tr
σ (t )Y−Y
σ (t )Y−Y
= 2 n2p2
σ (t )2+1
σ (t )−12 Tr
Y Y2
≤ 2 n2p2
σ (t )2+1
σ (t )−12
p
i=1
n k=1
xL∗2
p(i,k)
2
≤ 2 n2p2
σ (t )2+1(σ (t)−1)2 σ (t )2
αpkp
kp
i=1xi2 kp
+αpkp
kp
i=1x2iI{|xi|> t} kp
+ |μ(t )|αpkp kp
i=1|xi| kp
+αpkp
kp
i=1x¯i2 kp
2
.
Again using the conditionαpkp=O(np)andσ (t )→1 ast→ ∞we getT2→0 almost surely. This completes
the proof of the theorem.
