Asymptotic analysis of covariance parameter estimation for Gaussian processes in the misspecied case

Supplementary material for:

Asymptotic analysis of covariance parameter estimation for Gaussian processes in the misspecied case

François Bachoc

Department of Statistics, University of Vienna December 4, 2014

Abstract

We restate some context elements of the manuscript Asymptotic analysis of covariance parameter estimation for Gaussian processes in the misspecied case. Then, we restate and prove technical lemmas that are used there.

1 Context elements

Condition 1.1. For alln∈N^∗, the observation points X1, ..., Xn are random and follow inde- pendently the uniform distribution on [0, n^1/d]^d. The three variables Y, (X1, ..., Xn)and are mutually independent.

Condition 1.2. The covariance function K0 is stationary and continuous onR^d. There exists C0<+∞so that for t∈R^d,

|K₀(t)| ≤ C₀ 1 +|t|^d+1.

Condition 1.3. For all θ∈Θ, the covariance function K_θ is stationary. For all xedt∈R^d, K_θ(t)is p+ 1 times continuously dierentiable with respect to θ. For all i₁, ..., i_p ∈N so that i₁+...+i_p≤p+ 1, there existsA_i1,...,ip <+∞so that for allt∈R^d,θ∈Θ,

∂ⁱ¹

∂θⁱ₁¹... ∂^ip

∂θⁱp^p

Kθ(t)

≤ Ai₁,...,i_p

1 +|t|^d+1.

There exists a constantCinf >0 so that, for any θ ∈Θ, δθ ≥Cinf. Furthermore, δθ isp+ 1 times continuously dierentiable with respect toθ. For alli1, ..., ip∈Nso thati1+...+ip≤p+1, there existsB_i1,...,ip<+∞so that for allθ∈Θ,

∂ⁱ¹

∂θ₁ⁱ¹... ∂^ip

∂θp^ip

δ_θ

≤B_i1,...,ip.

In all this supplementary material, it is assumed that Conditions 1.1, 1.2 and 1.3 hold.

Denition 1.4. Consider a xedθ∈Θ. Consider two functions of n: n2(n)∈N^∗ and∆(n)≥ 0, that we write n₂ and∆ for simplicity, so that, for any n∈N^∗, n₂ can be written n₂=N₂^d, withN₂∈N^∗, and so thatn=n₂∆. Let, fori= 1, ..., N₂−1,c_i= [((i−1)/N2)n^1/d,(i/N₂)n^1/d). Let c_N2 = [((N₂−1)/N₂)n^1/d, n^1/d]. Let, for x ∈ [0, n^1/d], i(x) be the unique i ∈ {1, ..., N2} so that x ∈ c_i. Let, for t = (t₁, ..., t_d)^t ∈ [0, n^1/d]^d, C(t) = Qd

j=1c_i(tj). Dene the non- stationary covariance function K˜_θ(t₁, t₂) = K_θ(t₁, t₂)1_C(t1)=C(t2). Dene R˜_θ, R˜_i,θ, ˜r_i,θ, y˜ˆ_i,θ,

CV˜ θ similarly to Rθ, Ri,θ, ri,θ, yˆi,θ, CVθ but with Kθ replaced by K˜θ. Furthermore, let us write the n₂ aforementioned sets of the form Qd

j=1c_ij, for i₁, ..., i_d ∈ {1, ..., N2}, as the sets C1, ..., Cn₂. [The specic one-to-one correspondence we use between{1, ..., N2}^d and{1, ..., n2} is of no interest. Note that this one-to-one correspondence depends onn. The sets C1, ..., Cn₂

also depend of n, but we drop this dependence in the notation for simplicity.]

Let Ni be the random number of observation points inCi and letXⁱ be the randomNi-tuple obtained fromX by keeping only the observation points that are inCiand by preserving the order of the indices in X. Let yⁱ be the column vector of sizeNi, composed by the componentsyj of y for which Xj is inCi (preserving the order of indexes). Let R¯i,θ andR¯i,0 be the covariance matrices, under(Kθ, δθ)and(K0, δ0), ofyⁱ, givenX.

Finally, for1≤i, j≤n2, letvi andwj be twoNi×1andNj×1vectors andM^ij be aNi×Nj

matrix. Then we use the convention that, whenNi= 0,|M^ij|=||M^ij||= 0,||vi||=|vi|= 0and v_i^tM^ijw_j = 0. Furthermore, if i=j and Mⁱⁱ is invertible when N_i≥1, we use the convention thatv_i^t(Mⁱⁱ)⁻¹w_i= 0whenN_i = 0. [These conventions enable to write equalities or inequalities involving matrices and vectors of size N_i,N_j orN_i×N_j, that hold regardless of whether N_i or N_j are zero or not. As can be checked along the proofs involving Denition 1.4, these relations boil down to trivial relations (e.g. 0 = 0) when N_i = 0 or N_j = 0. This way of proceeding considerably simplies the exposition in these proofs.]

2 Technical results

Lemma 2.1. Consider a xed number n of observation points. Consider a function f_θ(X, y) that is ptimes continuously dierentiable w.r.t θfor any X, y and so that, fori₁+...+i_p≤p,

sup

θ

(∂ⁱ¹/∂θⁱ₁¹)...(∂^ip/∂θ_p^ip)f_θ(X, y)

has nite mean value w.r.t X andy. Then, there exists a constantC_sup (depending only of Θ) so that

E

sup

θ∈Θ

|fθ(X, y)|

≤Csup

X

i1+...+ip≤p

Z

Θ

E

∂ⁱ¹

∂θⁱ₁¹... ∂^ip

∂θp^ip

fθ(X, y)

! dθ.

Lemma 2.2. There exists a nite constantCsup so that, for anya, b∈R^d, Z

R^d

1 1 +|a−c|^d+1

1

1 +|b−c|^d+1dc≤Csup

1 1 +|a−b|^d+1.

Lemma 2.3. Let 0 < Cinf ≤ Csup < ∞ be xed independently of n. Let sn be a function of n so that sn ∈ N^∗ and Cinfn ≤ sn ≤ Csupn. Consider sn observation points X¯1, ...,X¯s_n, independent and uniformly distributed on [0, n^1/d]^d. Let A1, ..., Ak be k sequences of sn×sn

random matrices so that, for l = 1, ..., k, (Al)i,j depends only on X¯i and X¯j and satises

|(Al)i,j| ≤1/(1 +|X¯i−X¯j|^d+1). ThenEX |A1...Ak|²

is bounded w.r.t. n.

Lemma 2.4. The supremum over n, θ and X of the eigenvalues of R⁻¹_θ , R⁻¹_1,θ, diag(R_θ⁻¹), diag(R⁻¹_1,θ),diag(R⁻¹_θ )⁻¹ anddiag(R⁻¹_1,θ)⁻¹ is smaller than a constantCsup<+∞.

Lemma 2.5. Lemma 2.4 also holds whenKθ is replaced byK˜θ of Denition 1.4.

Lemma 2.6. Lemma 2.4 also holds whenR_θ is replaced byR¯_k,θ of Denition 1.4.

Lemma 2.7. Letk∈N. LetA_1,θ, ..., A_k,θ bek sequences of symmetric random matrices (func- tions ofX andθ) so that, for anym∈N,a₁, ..., a_m ∈ {1, ..., k}, sup_θ∈ΘEX|Aa1,θ...A_am,θ|² is bounded (w.r.t n). Let B_1,θ, ..., B_k+1,θ be k+ 1 sequences of random symmetric non-negative matrices (functions ofX andθ) so thatsup_θ||B_1,θ||, ...,sup_θ||B_k+1,θ||are bounded (w.r.tnand X). Then

sup

θ∈ΘEX|B1,θA1,θB2,θ...Bk,θAk,θBk+1,θ|² is bounded w.r.t n.

Lemma 2.8. Consider a xed θ ∈ Θ. With the notation of Denition 1.4, we have, when n2=o(n),

E

(R1,θ−R˜1,θ)²

2

→n→∞0.

Lemma 2.9. Let C(t) be as in Denition 1.4. Dene, for T ≥0, f(T) =R

R^d\[−T ,T]^d1/(1 +

|t|^d+1)dt. Dene, for x ∈ [0, n^1/d]^d, D_∆(x) = inf_t∈Rd\C(x)|x−t|. Dene D_∆(x₁, ..., x_m) = min_i=1,...,mD_∆(x_i). Then, there exists a nite constantC_supso that, for anyn, for anyx₁, x₂∈ [0, n^1/d]^d,

Z

R^d

1 1 +|x1−x|^d+1

1

1 +|x2−x|^d+11_C(x)6=C(x1)1_C(x)6=C(x2)dx≤Csupf(D∆(x1, x2)) 1

1 +|x1−x2|^d+1. Lemma 2.10. Use the notationn₂,∆,C(t),f(T)andD_∆(x₁, x₂)of Denition 1.4 and Lemma 2.9. Then, whenn₂=o(n),

1 n

Z

[0,n^1/d]^d

dx1

Z

[0,n^1/d]^d

dx2

1

1 +|x₁−x₂|^d+1f(D∆(x1, x2))→_n→+∞0.

Lemma 2.11. Use the notation n2,∆ andC1, ..., Cn₂ of Denition 1.4. Let, fori= 1, ..., n2, X₁ⁱ, ..., X_Nⁱ

i be the Ni components of X that are inCi (so that the order of their indexes inX is preserved). Then

i) Fori= 1, ..., n2,Nifollows a binomialB(n,1/n2)distribution. For anyi, j= 1, ..., n2;i6=

j, conditionally to Ni=ki,Nj follows a binomial B(n−ki,1/(n2−1))distribution.

ii) Conditionally toN_i =k_i,X₁ⁱ, ..., X_kⁱ

i are independent and uniformly distributed onC_i. iii) For 1 ≤ i 6= j ≤ n2, conditionally to Ni = ki, Nj = kj, the sets of random variables

(X₁ⁱ, ..., X_kⁱ

i) and (X₁^j, ..., X_k^j

j) are independent, and their components are independent and uniformly distributed on Ci andCj respectively.

Considern2real-valued functionsf1, ..., fn2 ofXthat can be writtenfi(X) = ¯f(Ni, X₁ⁱ, ..., X_Nⁱ_i), and so that, for any t∈R^d,x₁, ..., x_N ∈R^d,f¯(N, x₁+t, ..., x_N +t) = ¯f(N, x₁, ..., x_N). Then

iv) The variables f₁(X), ..., f_n2(X) have the same distribution. The couples (f_i(X), f_j(X)), for1≤i6=j≤n₂, have the same distribution.

Lemma 2.12. Use the notation of Lemma 2.11, and considern2functionsf1, ..., fn2that satisfy the conditions of Lemma 2.11. Assume that there exist xed even natural numbersq, land a nite constant C_sup (independent of nand X) so that E f_i²(X)|Ni=k

≤C_sup(1 +k^q+k^q+l/∆^l). Then, if∆→n→∞+∞ and∆ =O(n^1/(2q+5)),

var 1 n2

n₂

X

i=1

fi(X)

!

→_n→∞0.

Lemma 2.13. LetN follows the binomial distributionB(n,1/n2), withn/n2= ∆→n→∞+∞. Then, for anyk∈N, there exists a nite constant Csup, independent ofn, so that

E N^k

≤Csup∆^k.

Lemma 2.14. Let n₂, ∆ and C₁, ..., C_n2 be as in Denition 1.4. Assume that ∆ is lower- bounded, as a function of n. Then, there exists a nite constant C_sup so that for any n, i ∈ {1, ..., n₂},

n2

X

j=1

1

1 +d(Ci, Cj)^d+1 ≤C_sup.

Lemma 2.15. Let Abe a real m1×m2 matrix andb be am2-dimensional real column vector.

Then

||Ab||²≤m1m2

maxi,j A²_i,j

||b||².

3 Proof of the technical results

Proof of Lemma 2.1. We use a version of the Sobolev embedding theorem on the space Θ, equipped with the Lebesgue measure (Theorem 4.12, Part I, Case A in Adams and Fournier (2003)). This result implies that, for any xed X, y, there exists a constantC_sup (depending only ofΘ) so that

sup

θ∈Θ

|fθ(X, y)| ≤Csup

X

i1+...+ip≤p

Z

Θ

∂ⁱ¹

∂θi1

... ∂^ip

∂θip

fθ(X, y)

dθ.

By applying the mean value w.r.tX andyto this last inequality, and by using Fubini theorem, we prove the lemma.

Proof of Lemma 2.2. LetDa ={c∈R^d;|a−c| ≤ |b−c|}andDb ={c∈R^d;|b−c|<|a−c|}. Note that, forc∈D_a,|b−c| ≥ |a−b|/2and that, for c∈D_b,|a−c| ≥ |a−b|/2. Then,

Z

R^d

1 1 +|a−c|^d+1

1

1 +|b−c|^d+1dc ≤ Z

D_a

1 1 +|a−c|^d+1

1 1 +_|a−b|

2

^d+1dc +

Z

D_b

1 1 +_|a−b|

2

^d+1

1

1 +|b−c|^d+1dc

≤ 1

1 +|a−b|^d+12^d+12 Z

R^d

1 1 +|c|^d+1dc.

The proof of Lemma 2.3 uses the two following lemmas.

Lemma 3.1. There exists a nite constantC_sup so that, for anya, b, c∈R^d, 1

1 +|a−c|^d+1

1

1 +|b−c|^d+1 ≤C_sup 1 1 +|a−b|^d+1.

Proof of Lemma 3.1. We have either|a−c| ≥ |a−b|/2 or|b−c| ≥ |a−b|/2. The two cases are symmetric. Assume for example|a−c| ≥ |a−b|/2. Then,

1 1 +|a−c|^d+1

1

1 +|b−c|^d+1 ≤ 1 1 +_|a−b|

2

d+1 ≤2^d+1 1 1 +|a−b|^d+1.

Lemma 3.2. There exists a nite constantC_sup so that, for anya, b, c∈R^d, Z

R^d

1 1 +|a−t|^d+1

2

1 1 +|b−t|^d+1

1

1 +|c−t|^d+1dt≤C_sup 1 1 +|a−b|^d+1

1 1 +|a−c|^d+1. Proof of Lemma 3.2.

Z

R^d

1 1 +|a−t|^d+1

2

1 1 +|b−t|^d+1

1

1 +|c−t|^d+1dt

≤Csup

Z

R^d

1 1 +|a−t|^d+1

1 1 +|a−b|^d+1

1

1 +|c−t|^d+1dt (Lemma 3.1)

≤Csup

1 1 +|a−b|^d+1

1

1 +|a−c|^d+1 (Lemma 2.2).

Proof of Lemma 2.3. We can consider without loss of generality that the matrices A1, ..., Ak

have non-negative coecients, since this only increases the quantity EX |A1...Ak|²

to upper bound. Then, note that it is enough to prove the lemma forsn =n. Indeed, note rst that for sn 6=n, it is sucient to prove the lemma under the following condition (since n/sn is lower and upper bounded).

(Al)i,j ≤ 1 1 + _X¯

i−X¯_j) (n/sn)^1/d

^d+1. (1) Now, ifsn6=nobservation points are independent and uniformly distributed on[0, n^1/d]^dand the condition on the matrices is (1), then the value ofEX(|A1...Ak|²)can be seen as an element of the sequence of the same expression, but withnindependent points uniformly distributed on [0, n^1/d]^d, and where the matrices satisfy the condition given in the lemma. This latter sequence is bounded, so also the set of all the values ofEX(|A1...Ak|²), for all the values ofsn, is bounded.

We can hence considersn =nin the proof of the lemma and write, for simplicity,X¯1, ...,X¯n

as thenstandard observation pointsX1, ..., Xn. Let us rst show the lemma fork= 1.

EX(|A1|²) ≤ EX



 1 n

n

X

i,j=1

1

1 +|Xi−Xj|^d+1





≤ 1

n n+n² 1 n²

Z

[0,n^1/d]^d

dx1

Z

[0,n^1/d]^d

dx2

1 1 +|x₁−x₂|^d+1

!

≤ 1 n n+

Z

[0,n^1/d]^d

dx1

Z

R^d

dx2

1 1 +|x2|^d+1

!

≤ Csup.

The proof of the lemma fork= 2is similar to but simpler than the proof fork≥3, that we now do. Thus, for the rest of the proof, we considerk≥3. We have

EX |A1...A_k|²

= 1 nEX

n

X

i,j=1

" _n X

a₁=1

(A1)i,a₁(A2...Ak)a₁,j

#2

= 1 nEX

n

X

i,j=1

" _n X

a1,a2=1

(A₁)_i,a1(A₂)_a1,a2(A₃...A_k)_a2,j

#2

= 1 nE^X

n

X

i,j=1





n

X

a1,a2,...,ak−1=1

(A1)i,a₁(A2)a₁,a₂...(Ak)a_k−1,j





2

= 1 nEX

n

X

i,j=1 n

X

a₁,a₂,...,a_k−1=1 n

X

b₁,b₂,...,bk−1=1

(A1)i,a₁(A2)a₁,a₂...(Ak)ak−1,j(A1)i,b₁(A2)b₁,b₂...(Ak)bk−1,j. (2) Dene

S_{j,ak−2,bk−2} :=

n

X

ak−1,bk−1=1

(A_k−1)_{ak−2,ak−1}(A_k)_ak−1,j(A_k−1)_{bk−2,bk−1}(A_k)_bk−1,j. (3)

Then we have EX |A1...Ak|²

= 1

nEX n

X

i,j=1 n

X

a₁,a₂,...,a_k−2=1 n

X

b1,b2,...,bk−2=1

(A1)i,a₁(A2)a₁,a₂...(Ak−2)ak−3,ak−2(A1)i,b₁(A2)b₁,b₂...(Ak−2)bk−3,bk−2Sj,ak−2,bk−2.

We writeX_{i,j,a1,...,ak−2,b1,...,bk−2} as a shorthand for the(2k−2)-tuple of random variables (X_i, X_j, X_a1, ..., X_ak−2, X_b1, ..., X_bk−2).

We now show that, to prove the lemma by induction on k, it is sucient to show that there exists a nite constant C_sup so that, for any values of the indexes i, j, a₁, ..., a_k−2, b₁, ..., b_k−2, and for any realization of the corresponding variableXi,j,a₁,...,a_k−2,b₁,...,b_k−2,

E |Sj,ak−2,bk−2|

X_{i,j,a1,...,ak−2,b1,...,bk−2}

≤C_sup 1

1 +|Xak−2−Xj|^d+1

1

1 +|Xbk−2−Xj|^d+1. (4) Indeed, we have

EX |A₁...A_k|²

= 1 nEX

n

X

i,j=1 n

X

a1,a2,...,ak−2=1 n

X

b₁,b₂,...,b_k−2=1

(A₁)_i,a1(A₂)_a1,a2...(A_k−2)_{ak−3,ak−2}(A₁)_i,b1(A₂)_b1,b2...(A_k−2)_{bk−3,bk−2}S_{j,ak−2,bk−2}

= 1 n

n

X

i,j=1 n

X

a1,a2,...,ak−2=1 n

X

b₁,b₂,...,b_k−2=1

EXi,j,a1,...,ak−2,b1,...,bk−2

(A₁)_i,a1(A₂)_a1,a2...(A_k−2)_{ak−3,ak−2}(A₁)_i,b1(A₂)_b1,b2...(A_k−2)_{bk−3,bk−2} E Sj,ak−2,bk−2

Xi,j,a1,...,ak−2,b1,...,bk−2

.

With the consideration thatA₁, ..., A_k have non-negative coecients we have, under (4), EX |A₁...A_k|²

≤C_sup1 n

n

X

i,j=1 n

X

a1,a2,...,ak−2=1 n

X

b₁,b₂,...,b_k−2=1

EXi,j,a1,...,ak−2,b1,...,bk−2

(A₁)_i,a1(A₂)_a1,a2...(A_k−2)_{ak−3,ak−2}(A₁)_i,b1(A₂)_b1,b2...(A_k−2)_{bk−3,bk−2} 1

1 +|Xa_k−2−Xj|^d+1

1

1 +|Xb_k−2−Xj|^d+1.

By deningA˜_k−1 by( ˜A_k−1)c,d= 1/(1 +|Xc−Xd|^d+1), we obtain EX |A1...Ak|²

≤ Csup

n EX n

X

i,j=1 n

X

a1,a2,...,ak−2=1 n

X

b₁,b₂,...,bk−2=1

(A1)i,a₁(A2)a₁,a₂...( ˜A_k−1)a_k−2,j(A1)i,b₁(A2)b₁,b₂...( ˜A_k−1)b_k−2,j.

=CsupEX

|A1A2...A˜k−1|²

from (2).

Thus, if (4) holds, we can prove the lemma by induction on k. Let us now prove (4). This is done by writing

Sj,a_k−2,b_k−2

=

n

X

ak−1,bk−1=1

(A_k−1)a_k−2,a_k−1(Ak)a_k−1,j(A_k−1)b_k−2,b_k−1(Ak)b_k−1,j

=

4

X

c=1

X

(ak−1,bk−1)∈Ic

(A_k−1)_{ak−2,ak−1}(A_k)_ak−1,j(A_k−1)_{bk−2,bk−1}(A_k)_bk−1,j,

where the sets of indices I1, ..., I4 are dened below and form a partition of {1, ..., n}². It is then sucient to show that for c = 1, ...,4, there exists a nite constantCsup so that for any (a, b)∈Ic,

|Ic|E

1

1 +|X_ak−2−X_a|^d+1

1

1 +|X_a−X_j|^d+1

1

1 +|X_bk−2−X_b|^d+1

1

1 +|X_b−X_j|^d+1 Xi,j,a₁,...,ak−2,b₁,...,bk−2

(5)

≤Csup

1

1 +|Xa_k−2−Xj|^d+1

1

1 +|Xb_k−2−Xj|^d+1.

We dene the set I1 as the set of the a, b ∈ {1, ..., n} that are dierent, and that do not belong to the set{i, j, a1, ..., a_k−2, b1, ..., b_k−2}. ForI1, the cardinality in (5) is less thann² and the conditional mean values in (5) are equal to

1 n²

Z

[0,n^1/d]^d

1

1 +|Xak−2−xa|^d+1

1

1 +|xa−Xj|^d+1dxa

!

Z

[0,n^1/d]^d

1

1 +|Xbk−2−xb|^d+1

1

1 +|xb−Xj|^d+1dx_b

! ,

so that (5) holds because of Lemma 2.2. We dene the set I2 as the set of thea, b∈ {1, ..., n}

that are equal, and that do not belong to the set {i, j, a1, ..., a_k−2, b1, ..., b_k−2}. For I2, the cardinality in (5) is less thannand the conditional mean values in (5) are equal to

1 n

Z

[0,n^1/d]^d

1

1 +|Xak−2−xa|^d+1

1 1 +|xa−Xj|^d+1

1

1 +|Xbk−2−xa|^d+1

1

1 +|xa−Xj|^d+1dxa, so that (5) holds because of Lemma 3.2. We dene the set I3 as the set of thea, b∈ {1, ..., n}

that are so that one of them is in the set{i, j, a1, ..., ak−2, b1, ..., bk−2} and the other one is not in the set{i, j, a1, ..., ak−2, b1, ..., bk−2}. ForI3, the cardinality in (5) is less thanCsupn. For the conditional mean values in (5), by symmetry, we can assume a∈ {i, j, a1, ..., ak−2, b1, ..., bk−2}. Then, the conditional mean values in (5) are equal to

1 n

Z

[0,n^1/d]^d

1

1 +|Xak−2−Xa|^d+1

1

1 +|Xa−Xj|^d+1

1

1 +|Xbk−2−xb|^d+1

1

1 +|xb−Xj|^d+1dx_b

≤Csup

1 n

1

1 +|Xa_k−2−Xj|^d+1 Z

[0,n^1/d]^d

1

1 +|Xb_k−2−xb|^d+1

1

1 +|xb−Xj|^d+1dxb (Lemma 3.1)

≤Csup

1 n

1

1 +|Xa_k−2−Xj|^d+1

1

1 +|Xb_k−2−Xj|^d+1 (Lemma 2.2).

Finally, we dene the set I4 as the set of the a, b ∈ {1, ..., n} that both belong to the set {i, j, a1, ..., a_k−2, b1, ..., b_k−2}. For I4, the cardinality in (5) is bounded (w.r.t n) and the conditional mean values in (5) are equal to

1

1 +|X_ak−2−X_a|^d+1

1

1 +|X_a−X_j|^d+1

1

1 +|X_bk−2−X_b|^d+1

1

1 +|X_b−X_j|^d+1,

so that (5) holds because of Lemma 3.1. Thus, (4) is proved, which completes the proof of the lemma.

Proof of Lemma 2.4. We show the lemma forR⁻¹_θ ,diag(R⁻¹_θ ) anddiag(R⁻¹_θ )⁻¹, the proof for R⁻¹_1,θ, diag(R_1,θ⁻¹) and diag(R_1,θ⁻¹)⁻¹ being similar. The matrix Rθ is the sum of a symmetric non-negative matrix and ofδ_θI_n. Thus, the eigenvalues of its inverse are smaller than 1/C_inf from Condition 1.3. Then, from Lemma D.6 in Bachoc (2014), the eigenvalues of diag(R⁻¹_θ ) are also smaller than 1/C_inf. Finally, from Proposition 0.1 in the supplementary material of Bachoc (2013),(diag(R⁻¹_θ )⁻¹)_i,i=K_θ(0) +δ_θ−r^t_1,θR⁻¹_1,θr_1,θ ≤C_sup, from Condition 1.3.

Proof of Lemma 2.5. Same as for Lemma 2.4.

Proof of Lemma 2.6. Same as for Lemma 2.4.

Proof of Lemma 2.7. We have, for anyθ∈Θ,

EX|B1,θA1,θB2,θ...Bk,θAk,θBk+1,θ|²≤CsupEX|A1,θB2,θ...Bk,θAk,θ|²

=CsupEX

1

nT r(A1,θB2,θ...Bk,θAk,θAk,θBk,θ...B2,θA1,θ)

=CsupEX

1

nT r A²_1,θB2,θ...Bk,θA²_k,θBk,θ...B2,θ

(Cauchy Schwarz:)≤

r EX

A²_1,θB_2,θ...B_k,θ

2r EX

A²_k,θB_k,θ...B_2,θ

2

≤Csup

r EX

A²_1,θB2,θ...Ak−1,θ

2r EX

A²_k,θBk,θ...A2,θ

2

. (6) Both of the square roots in (6) are applied to a term of the form

EX

B_1,θ⁰ A⁰_1,θB_2,θ⁰ ...B_k−1,θ⁰ A⁰_k−1,θB_k,θ⁰

2,

where the sequences of random matricesB_1,θ⁰ , A⁰_1,θ, B⁰_2,θ, ..., B_k−1,θ⁰ , A⁰_k−1,θ, B_k,θ⁰ satisfy the con- ditions of the lemma. Thus, we have shown that we can reduce the problem involvingkmatrices A1,θ, ..., Ak,θ to a similar problem involvingk−1matricesA1,θ, ..., Ak−1,θ. On the other hand, the result is true by assumption fork= 1. Thus, we have proved the lemma by induction onk.

Proof of Lemma 2.8. Let us write F_i,j and C_i,j as shorthands for 1/(1 +|Xi−X_j|^d+1) and 1_C(Xi)6=C(Xj). Let us also write i₁ 6= i₂ 6= ... 6= i_k when k numbers i₁, ..., i_k are two-by-two distinct. Then we have, from Condition 1.3,

E

(R1,θ−R˜1,θ)²

2

= 1 n

n

X

i,j,k,l=2

E

(R1,θ−R˜1,θ)i,k(R1,θ−R˜1,θ)k,j(R1,θ−R˜1,θ)i,l(R1,θ−R˜1,θ)l,j

≤Csup

1 n

n

X

i,j,k,l=2

E(Ci,kCk,jCi,lCl,jFi,kFk,jFi,lFl,j)

=Csup

1 n

n

X

i,j=2

X

k=2,...,n k6∈{i,j}

E C_i,k² C_k,j² F_i,k² F_k,j²

+Csup

1 n

n

X

i,j=2

X

k,l=2,...,n k6=l;k,l6∈{i,j}

E(Ci,kCk,jCi,lCl,jFi,kFk,jFi,lFl,j).

Hence, by distinguishing i=j fromi6=j in each of the two double sums in the above display, and by noting that two of the four corresponding cases are symmetric, we obtain

E

(R_1,θ−R˜_1,θ)²

2

≤C_sup1 n

X

i,k=2,...,n i6=k

E(C_i,kF_i,k) +C_sup1 n

X

i,j,k=2,...,n i6=j6=k

E(C_i,kC_k,jF_i,kF_k,j)

+C_sup1 n

X

i,j,k,l=2,...,n i6=j6=k6=l

E(C_i,kC_k,jC_i,lC_l,jF_i,kF_k,jF_i,lF_l,j)

≤C_sup1

nn²E(C_1,2F_1,2) +C_sup1

nn³E(C_1,3C_3,2F_1,3F_3,2) +C_sup1

nn⁴E(C_1,3C_3,2C_1,4C_4,2F_1,3F_3,2F_1,4F_4,2) (by symmetry).

(7) Let us callT1,T2andT3 the three terms in (7). For the termT1,

T₁ = 1 n

Z

[0,n^1/d]^d

dx₁ Z

[0,n^1/d]^d

dx₂ 1

1 +|x1−x₂|^d+11_C(x1)6=C(x2)

≤ 1 n

Z

[0,n^1/d]^d

dx1f(D∆(x1)) (notation of Lemma 2.9).

Now, for any > 0, there is a nite T so that f(T) ≤ , and by dening En = {x ∈ [0, n^1/d]^d;D∆(x)≤T}, we have|En|=o(n), as can be seen easily, and

1 n

Z

[0,n^1/d]^d

f(D∆(x1))dx1≤f(0)|En| n +, to thatT1→_n→∞0. For the termT2,

T2

= 1 n

Z

[0,n^1/d]^d

dx1

Z

[0,n^1/d]^d

dx2

Z

[0,n^1/d]^d

dx3

1 1 +|x1−x3|^d+1

1

1 +|x2−x3|^d+11C(x₁)6=C(x3)1C(x₂)6=C(x3)

≤ 1 n

Z

[0,n^1/d]^d

dx₁ Z

[0,n^1/d]^d

dx₂ 1

1 +|x1−x₂|^d+1f(D_∆(x₁, x₂)) (Lemma 2.9)

→_n→∞0 (Lemma 2.10.)

For the term T3, T3=

1 n

Z

[0,n^1/d]^d

dx₁ Z

[0,n^1/d]^d

dx₂ Z

[0,n^1/d]^d

dx₃ Z

[0,n^1/d]^d

dx₄ 1

1 +|x1−x3|^d+1

1 1 +|x2−x3|^d+1

1 1 +|x1−x4|^d+1

1 1 +|x2−x4|^d+1 1_C(x1)6=C(x3)1_C(x2)6=C(x3)1_C(x1)6=C(x4)1_C(x2)6=C(x4)

≤Csup

1 n

Z

[0,n^1/d]^d

dx1

Z

[0,n^1/d]^d

dx2

1 1 +|x₁−x₂|^d+1

²

f²(D∆(x1, x2)) (from Lemma 2.9.) Now, because f(t) →_t→+∞ 0, is continuous and positive, there exists a nite Csup so that f²≤Csupf. Thus, we can conclude from Lemma 2.10.

Proof of Lemma 2.9. Let T = D∆(x1, x2). Let Bx₁ = {x ∈ R^d,|x−x1| < T}, Bx₂ = {x ∈ R^d,|x−x2|< T},Ax1={x∈R^d;|x−x1| ≤ |x−x2|}andAx2 ={x∈R^d;|x−x2|<|x−x1|}. Observe that C(x)6=C(x₁) impliesx6∈ B_x1, that x∈A_x1 implies|x−x₂| ≥ |x1−x₂|/2 and that we have the symmetric results when interchanging the roles ofx₁andx₂. Then, we obtain

Z

R^d

1 1 +|x−x₁|^d+1

1

1 +|x−x₂|^d+11_C(x)6=C(x1)1_C(x)6=C(x2)dx

≤ Z

A_x1\Bx1

1 1 +|x−x1|^d+1

1 1 +_|x

1−x2| 2

^d+1dx+ Z

A_x2\Bx2

1 1 +_|x

1−x2| 2

^d+1

1

1 +|x−x2|^d+1dx

≤2^d+1 1

1 +|x₁−x₂|^d+12f(T).

Proof of Lemma 2.10. Let >0 and let T be a constant so thatf(T)≤. LetE_n(T) ={x∈ [0, n^1/d]^d;D_∆(x)≤T}. Then,

1 n

Z

E_n(T)

dx1

Z

[0,n^1/d]^d

dx2

1

1 +|x1−x2|^d+1f(D∆(x1, x2)) ≤ |En(T)|

n f(0) Z

R^d

1 1 +|t|^d+1dt

→n→∞ 0.

Also, 1 n

Z

[0,n^1/d]^d\En(T)

dx₁ Z

[0,n^1/d]^d

dx₂ 1

1 +|x1−x₂|^d+1f(D_∆(x₁, x₂))

≤ 1 n

Z

[0,n^1/d]^d\En(T)

dx1

Z

E_n(T)

dx2

1

1 +|x1−x2|^d+1f(0) + 1

n Z

[0,n^1/d]^d\E_n(T)

dx₁ Z

[0,n^1/d]^d\E_n(T)

dx₂ 1

1 +|x1−x2|^d+1 (by denition ofD∆(x1, x2)andEn(.))

≤ |En(T)|

n f(0) Z

R^d

1

1 +|t|^d+1dt+ Z

R^d

1

1 +|t|^d+1dt (by Fubini theorem)

=o(1)f(0) Z

R^d

1

1 +|t|^d+1dt+ Z

R^d

1 1 +|t|^d+1dt, which nishes the proof.

Proof of Lemma 2.11. BecauseX1, ..., Xnare independent and uniformly distributed on[0, n^1/d]^d, and because ^|C_n^i| = _n¹

2, the rst part of i) holds. For the second part of i), we calculate, for i6=j,

P(N_i=k_i;N_j =l_j) P(Ni=ki) =

n k_i

n−ki

l_j

1 n₂

ki

1 n₂

lj

n₂−2 n₂

n−ki−lj

n ki

1 n2

^ki_n

2−1 n2

n−ki

=

n−ki

l_j

1 n₂−1

^ljn2−2 n₂−1

^n−ki−l_j

, which proves the second part.

Forii), consideriand a measurable functiong(X₁ⁱ, ..., X_Nⁱ

i). Let, for a subsetCof{1, ..., n}, XC be the tuple built by extracting the elements ofX which indexes are inC. Also, forE⊂R^d and for ar-tuplev= (v1, ..., vr)∈(R^d)^r, we write v∈E when all the componentsv1, ..., vr are inE. Then we have

E g(X₁ⁱ, ..., X_Nⁱ

i)1N_i=k_i P(Ni =ki)

= E

P

k₁<...<k_ki∈{1,...,n}g(Xk1, ..., Xk_ki)1X_{k1,...,kk

i}∈Ci1_X{1,...,n}\{k1,...,kki}∈[0,n^1/d]^d\C_i

n

ki

1 n₂

^ki

n2−1 n₂

n−ki

= 1

∆^ki Z

C_i^ki

g(x₁, ..., x_ki)dx₁...dx_ki.

This proves ii). For iii), consider i 6= j and two measurable functions g(X₁ⁱ, ..., X_Nⁱ

i) and h(X₁^j, ..., X_N^j

j). Then, E

g(X₁ⁱ, ..., X_Nⁱ

i)h(X₁^j, ..., X_N^j

j)1_Ni=ki1_Nj=lj P(Ni=ki;Nj=lj)

= X

k1<...<k_ki∈{1,...,n}

X

l1<...<l_lj∈{1,...,n}\{k1,...,k_ki}

E

g(X_k1, ..., X_kki)h(X_l1, ..., X_llj)1_X{k

1,...,kki}∈Ci1_X{l

1,...,llj}∈Cj1_X{1,...,n}\{k1,...,kki,l1,...,llj}∈[0,n^1/d]^d\(Ci∪Cj)

n

ki

n−ki

lj

1 n2

ki

1 n2

lj

n₂−2 n2

n−ki−lj

= 1

∆^ki Z

C_i^ki

g(x1, ..., xk_i)dx1...dxk_i

1

∆^lj Z

C_j^lj

h(x1, ..., xl_j)dx1...dxl_j. This provesiii). Now,iv)is a consequence ofi),ii)andiii).

The proof of Lemma 2.12 uses the following lemma.

Lemma 3.3. Consider two functions ofn: τ(n)anda(n), that we writeτ andafor simplicity and so that τ →_n→∞ 0, nτ →_n→∞ +∞and a→_n→∞ +∞. Let N follow the binomial distri- bution B(n, τ). Then, for anyk ∈N, there exists a nite constant Csup, independent of n, so that,

E N^k1N≥anτ

≤Csup

(nτ)^k−1 a² .