Kernel estimation of density level sets

HAL Id: hal-00003898

https://hal.archives-ouvertes.fr/hal-00003898

Submitted on 14 Jan 2005

HAL is a multi-disciplinary open access archive for the deposit and dissemination of sci- entific research documents, whether they are pub- lished or not. The documents may come from teaching and research institutions in France or

L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires

Kernel estimation of density level sets

Benoît Cadre

To cite this version:

Benoît Cadre. Kernel estimation of density level sets. Journal of Multivariate Analysis, Elsevier, 2006, pp.999-1023. �hal-00003898�

ccsd-00003898, version 1 - 14 Jan 2005

KERNEL ESTIMATION OF DENSITY LEVEL SETS

Benot CADRE¹

Laboratoire de Mathmatiques, Universit Montpellier II, CC 051, Place E. Bataillon, 34095 Montpellier cedex 5, FRANCE Abstract.Letf be a multivariate density andf_nbe a kernel estimate of f drawn from then-sampleX1,· · ·, Xnof i.i.d. random variables with density f. We compute the asymptotic rate of convergence towards 0 of the volume of the symmetric difference between thet-level set {f ≥ t} and its plug-in estimator {f_n≥t}. As a corollary, we obtain the exact rate of convergence of a plug-in type estimate of the density level set corresponding to a fixed probability for the law induced byf.

Key-words :Kernel estimate, Density level sets, Hausdorff measure.

2000 Mathematics Subject Classification : 62H12, 62H30.

1. Introduction.Recent years have witnessed an increasing interest in esti- mation of density level sets and in related multivariate mappings problems.

The main reason is the recent advent of powerfull mathematical tools and computational machinery that render these problems much more tractable.

One of the most powerful application of density level sets estimation is in unsupervised cluster analysis (see Hartigan [1]), where one tries to break a complex data set into a series of piecewise similar groups or structures, each of which may then be regarded as a separate class of data, thus re- ducing overall data compexity. But there are many other fields where the knowledge of density level sets is of great interest. For example, Devroye and Wise [2], Grenander [3], Cuevas [4] and Cuevas and Fraiman [5] used density support estimation for pattern recognition and for detection of the abnormal behavior of a system.

In this paper, we consider the problem of estimating the t-level setL(t) of a multivariate probability densityf with support inIR^kfrom independent random variablesX₁,· · ·, X_nwith densityf. Recall that fort≥0, thet-level set of the density f is defined as follows :

L(t) ={x∈IR^k : f(x)≥t}.

1cadre@math.univ-montp2.fr

The question now is how to define the estimates ofL(t) from then-sample X₁,· · ·, X_n? Even in a nonparametric framework, there are many possible answers to this question, depending on the restrictions one can impose on the level set and the density under study. Mainly, there are two families of such estimators : theplug-in estimators and the estimators constructed by an excess mass approach. Assume that an estimator fn of the density f is available. Then a straightforward estimator of the level setL(t) is{f_n≥t}, the plug-in estimator. Molchanov [6, 7] and Cuevas and Fraiman [5] proved consistency of these estimators and obtained some rates of convergence. The excess mass approach suggest to first consider the empirical mapping M_n defined for every borel setL⊂IR^k by

M_n(L) = 1 n

n

X

i=1

1_{Xi∈L}−tλ(L),

whereλdenotes the Lebesgue measure onIR^k. A natural estimator ofL(t) is a maximizer ofM_n(L) over a given class of borel setsL. For different classes of level sets (mainly star-shaped or convex level sets), estimators based on the excess mass approach were studied by Hartigan [8], M¨uller [9], M¨uller and Sawitzki [10], Nolan [11] and Polonik [12], who proved consistency and found certain rates of convergence. When the level set is star-shaped, Tsybakov [13] recently proved that the excess mass approach gives estimators with optimal rates of convergence in an asymptotically minimax sense, whithin the studied classes of densities. Though this result has a great theoretical interest, assuming the level set to be convex or star-shaped appears to be somewhat unsatisfactory for the statistical applications. Indeed, such an assumption does not permit to consider the important case where the density under study is multimodal with a finite number of modes, and hence the results can not be applied to cluster analysis in particular. In comparison, the plug-in estimators do not care about the specific shape of the level set.

Moreover, another advantage of the plug-in approach is that it leads to easily computable estimators. We emphasize that, if the excess mass approach often gives estimators with optimal rates of convergence, the complexity of the computational algorithm of such an estimator is high, due to the presence of the maximizing step (see the computational algorithm proposed by Hartigan, [8]).

In this paper, we study a plug-in type estimator of the density level set L(t), using a kernel density estimate of f (Rosenblatt, [14]). Given a kernel K onIR^k(i.e., a probability density on IR^k) and a bandwidthh=h(n)>0

such thath→0 as ngrows to infinity, the kernel estimate off is given by f_n(x) = 1

nh^k

n

X

i=1

Kx−Xi

h

, x∈IR^k.

We let the plug-in estimate Ln(t) ofL(t) be defined as Ln(t) ={x∈IR^k : fn(x)≥t}.

In the whole paper, the distance between two borel sets in IR^k is a measure -in particular the volume or Lebesgue measure λ on IR^k- of the symmetric difference denoted ∆ (i.e., A∆B = (A∩B^c)∪(A^c∩B) for all sets A, B). Our main result (Theorem 2.1) deals with the limit law of

√nh^kλLn(t)∆L(t), which is proved to be degenerate.

Consider now the following statistical problem. In cluster analysis for instance, it is of interest to estimate the density level set corresponding to a fixed probabilityp ∈[0,1] for the law induced by f. The data contained in this level set can then be regarded as the most important data ifp is far enough from 0. Since f is unknown, the level t of this density level set is unknown as well. The natural estimate of the target density level set L(t) becomesLn(tn), where tn is such that

Z

Ln(tn)

f_ndλ=p.

As a consequence of our main result, we obtain in Corollary 2.1 the exact asymptotic rate of convergence of Ln(t_n) to L(t). More precisely, we prove that for someβn which only depends on the data, one has :

β_n√

nh^kλLn(t_n)∆L(t)→ s2

π Z

K²dλ

in probability.

The precise formulations of Theorem 2.1 and Corollary 2.1 are given in Section 2. Section 3 is devoted to the proof of Theorem 2.1 while the proof of Corollary 2.1 is given in Section 4. The appendix is dedicated to a

change of variables formula involving the (k-1)-dimensional Hausdorff mea- sure (Proposition A).

2. The main results.

2.1 Estimation of t-level sets. In the following, Θ ⊂ (0,∞) denotes an open interval and k.k stands for the euclidean norm over any finite dimen- sional space. Let us introduce the hypotheses on the densityf :

H1. f is twice continuously differentiable and f(x)→0 as kxk → ∞; H2. For allt∈Θ,

f⁻inf¹({t})k∇fk>0,

where, here and in the following, ∇ψ(x) denotes the gradient at x∈IR^k of the differentiable functionψ : IR^k→IR. Next, we introduce the assumptions on the kernelK :

H3. K is a continuously differentiable and compactly supported func- tion. Moreover, there exists a monotone nonincreasing function µ : IR+→IR such thatK(x) =µ(kxk) for all x∈IR^k.

The assumption on the support of K is only provided for simplicity of the proofs. As a matter of fact, one could consider a more general class of kernels, including the gaussian kernel for instance. Moreover, as we will use Pollard’s results [15],K is assumed to be of the form µ(k.k).

Throughout the paper,Hdenotes the (k-1)-dimensional Hausdorff mea- sure onIR^k(cf.Evans and Gariepy, [16]). Recall thatHagrees with ordinary

“(k-1)-dimensional surface area” on nice sets. Moreover,∂Ais the boundary of the setA⊂IR^k,

α(k) =

( 3 ifk= 1 ; k+ 4 ifk≥2.

and for any bounded borel functiong : IR^k →IR₊,λ_gstands for the measure defined for each borel setA⊂IR^k by

λ_g(A) = Z

A

g dλ.

Finally, the notation→^P denotes the convergence in probability.

It can be proved that if H1,H3hold and if λ(∂L(t)) = 0, one has : λLn(t)∆L(t)→^P 0.

The aim of Theorem 2.1 below is to obtain the exact rate of convergence.

Theorem 2.1. Let g : IR^k →IR₊ be a bounded borel function and assume that H1-H3 hold. If nh^k/(logn)¹⁶ → ∞ and nh^α(k)(logn)² → 0, then for almost every (a.e.)t∈Θ :

√nh^kλ_gLn(t)∆L(t)→^P s2t

π Z

K²dλ Z

∂L(t)

g k∇fkdH.

Remarks 2.1.• Notice that the rightmost integral is defined becauseg is bounded andL(t) is a compact set for all t >0 according to H1.

•In practice, this result is mainly interesting wheng≡1, since we then have the asymptotic behavior of the volume of the symmetric difference between the two level sets. The general case is provided for the proof of Corollary 2.1 below.

• If we only assume f to be Lipschitz instead of H1, then f is an almost everywhere continuously differentiable function by Rademacher’s theorem and Theorem 2.1 holds under the additional assumption on the bandwidth : nh^k+2(logn)² →0.

2.2 Estimation of level sets with fixed probability.In order to derive the corollary, we need an additional condition onf.

H4. For allt∈(0,sup_IRkf],λ(f⁻¹[t−ε, t+ε])→0 asε→0. Moreover, λ(f⁻¹(0, ε])→0 as ε→0.

Roughly speaking,H4means that the sets wheref is constant do not charge the Lebesgue measure onIR^k. Many densities with a finite number of local extrema satisfy H4. However, notice that if f is a continuous density such thatλ(f⁻¹(0, ε])→0 asε→0, then it is compactly supported.

Let us now denote by P the application P : [0,sup_IRkf] →[0,1]

t 7→λ_f(L(t)).

Observe that P is one-to-one if f satisfies H1,H4. Then, for all p∈[0,1], let t^(p) ∈[0,sup_IRkf] be the unique real number such that λ_f(L(t^(p))) =p.

Morevover, let t^(p)n ∈ [0,sup_IRkf_n] be such that λ_fn(Ln(t^(p)n )) = p. Notice thatt^(p)n does exists sincef_n is a density onIR^k.

The aim of Corollary 2.1 below is to obtain the exact rate of convergence of Ln(t_n) to L(t). We also introduce an estimator of the unknown integral in Theorem 2.1.

Corollary 2.1. Let k ≥ 2, (αn)n be a sequence of positive real numbers such that α_n → 0 and assume that H1-H4 hold. If nh^k+2/logn → ∞, nh^k+4(logn)² →0 and α²_nnh^k/(logn)² → ∞ then, for a.e. p∈ P(Θ) :

√nh^k β_n q

t^(p)n

λLn(t^(p)_n )∆L(t^(p))→^P s2

π Z

K²dλ,

where β_n=α_n/λ(Ln(t^(p)_n )− Ln(t^(p)_n +α_n)).

Remarks 2.2.• It is of statistical interest to mention the fact that under the assumptions of the corollary, we have for allp ∈[0,1] : t^(p)n →t^(p) with probability 1 (see Lemma 4.3).

• When k = 1, the conditions of Theorem 2.1 on the bandwidth h do not permit to derive Corollary 2.1. In practice, estimations of density level sets and their applications to cluster analysis for instance are mainly interesting in high-dimensional problems.

3. Proof of Theorem 2.1.

3.1. Auxiliary results and proof of Theorem 2.1.For allt >0, let Vn^t =f^−1ht−(logn)^β

√nh^k , tⁱ and V^tn=f^−1ht, t+ (logn)^β

√nh^k i,

whereβ >1/2 is fixed. Moreover, ˜K stands for the real number : K˜ =

Z

K²dλ.

Proposition 3.1. Let g : IR^k → IR₊ be a bounded borel function and assume that H1-H3 hold. If nh^k/(logn)^31β → ∞ and nh^α(k)(logn)^2β → 0, then for a.e.t∈Θ:

limn

√nh^k Z

V_n^t

P(f_n(x)≥t)dλ_g(x) = lim

n

√nh^k Z

V^t_n

P(f_n(x)< t)dλ_g(x)

= s

tK˜ 2π

Z

∂L(t)

g k∇fkdH.

Proposition 3.2. Let g : IR^k → IR₊ be a bounded borel function and assume that H1-H3 hold. If nh^k/(logn)^5β → ∞ and nh^α(k)(logn)^2β → 0, then for a.e.t∈Θ:

limn nh^kvar^hλ_gVn^t ∩ Ln(t)ⁱ= 0 = lim

n nh^kvar^hλ_gV^tn∩ Ln(t)^ci.

Proof of Theorem 2.1.Lett∈Θ be such that both conclusions of Propo- sitions 3.1 and 3.2 hold. According to H3 and Pollard ([15], Theorem 37 and Problem 28, Chapter II), we have almost surely (a.s.) :

sup

IR^k |f_n−Ef_n| →0.

Moreover, since both sup_nEf_n(x) andf(x) vanish askxk → ∞byH1,H3, we have :

sup

IR^k |Ef_n−f| →0.

Thus, a.s. and fornlarge enough : sup

IR^k |f_n−f| ≤ t 2.

Consequently,Ln(t)⊂ L(t/2) and since L(t)⊂ L(t/2), we get : λ_gLn(t)∆L(t)=

Z

L(t/2)1_{fn<t,f≥t}dλ_g+ Z

L(t/2)1_{fn≥t,f <t}dλ_g. (3.1) Let

A_n =ⁿ√

nh^k sup

L(t/2)|f_n−f| ≤(logn)^βo.

SinceL(t/2) is a compact set by H1, it is a classical exercise to prove that P(A_n) → 1 under the assumptions of the theorem. Hence, one only needs to prove that the result of Theorem 2.1 holds on the eventAn. But onAn, one has according to (3.1) :λ_g(Ln(t)∆L(t)) =J_n¹+J_n², where :

J_n¹ =λ_gV^tn∩ Ln(t)^cand J_n²=λ_gVn^t ∩ Ln(t). By Propositions 3.1 and 3.2, ifj= 1 or j= 2 :

√nh^kJ_n^j →^P s

tK˜ 2π

Z

∂L(t)

g

k∇fkdH, (3.2)

if the bandwidthh satisfies nh^α(k)(logn)^2β → 0 and nh^k/(logn)^31β → ∞. Lettingβ = 16/31, the theorem is proved•

3.2. Proof of Proposition 3.1.LetX be a random variable with density f,

V_n(x) = varKx−X h

andZ_n(x) = h^k√ n

pV_n(x)(f_n(x)−Ef_n(x)), for all x ∈ IR^k such that V_n(x) 6= 0. Moreover, Φ denotes the distribution function of theN(0,1) law.

In the proofs,c denotes a positive constant whose value may vary from line to line.

Lemma 3.1.Assume that H1, H3 hold and let C ⊂IR^k be a compact set such thatinfCf >0. Then, there exists c >0 such that for all n≥1, x∈ C andu∈IR :

|P(Z_n(x)≤u)−Φ(u)| ≤ c

√nh^k.

Proof.By the Berry-Essen inequality (cf.Feller, [17]), one has for alln≥1, u∈IR and x∈IR^k such thatV_n(x)6= 0 :

|P(Z_n(x)≤u)−Φ(u)| ≤ 3

pnVn(x)³EKx−X h

−EKx−X h

3.

It is a classical exercise to deduce fromH1,H3 that sup

x∈C

EKx−X h

−EKx−X h

3≤c h^k and inf

x∈CV_n(x)≥c h^k, hence the lemma•

For all borel bounded function g : IR^k → IR₊, we let Θ₀(g) to be the set oft∈Θ such that :

εց0lim 1

ελ_gf⁻¹[t−ε, t]= lim

εց0

1

ελ_gf⁻¹[t, t+ε]= Z

∂L(t)

g k∇fkdH. Lemma 3.2. Let g : IR^k → IR₊ be a borel bounded function and assume that H1, H2 hold. Then we have : Θ₀(g) = Θ a.e.

Proof.According to H1,H2, for all t∈Θ, there exists η >0 such that :

f⁻¹[t−η,t+η]inf k∇fk>0.

We deduce from Proposition A that for allt∈Θ andε >0 small enough : 1

ελ_gf⁻¹[t−ε, t]= 1 ε

Z t t−ε

Z

∂L(s)

g

k∇fkdHds.

Using the Lebesgue-Besicovitch theorem (cf.Evans and Gariepy, [16], The- orem 1, Chapter I), we then have for a.e.t∈Θ :

εց0lim 1

ελ_gf⁻¹[t−ε, t]= Z

∂L(t)

g k∇fkdH,

and the same result holds for λg(f⁻¹[t, t+ε]) instead of λg(f⁻¹[t−ε, t]), hence the lemma•

It is a straightforward consequence of Lemma 3.2 above thatλ(∂L(t)) = 0 for a.e.t∈Θ. For simplicity, we shall assume throughout that this is true for all t∈Θ. Since Θ is an open interval, we have in particular

λf⁻¹[t−ε, t+ε]=λf⁻¹(t−ε, t+ε), for all t∈Θ andε >0 small enough.

We now let fort∈Θ andx∈IR^k such thatf(x)V_n(x)6= 0 : t_n(x) =

s nh^k

Kf˜ (x)(t−f(x)) andt_n(x) = h^k√ n

pV_n(x)(t−Ef_n(x)), and finally, Φ(u) = 1−Φ(u) for allu∈IR.

Lemma 3.3. Let g : IR^k → IR₊ be a bounded borel function and assume that H1, H2 hold. If nh^k/(logn)^2β → ∞ and nh^k+4(logn)^2β →0, then for allt∈Θ₀(g) :

limn

√nh^kh Z

V_n^t P(fn(x)≥t)dλg(x)− Z

V_n^t Φ(tn(x))dλg(x)ⁱ= 0 and lim

n

√nh^kh Z

V^t_n

P(f_n(x)< t)dλ_g(x)− Z

V^t_n

Φ(t_n(x))dλ_g(x)ⁱ= 0.

Proof. We only prove the first equality. Lett ∈Θ₀(g). First note that for allx∈IR^k such that V_n(x)6= 0 :

P(f_n(x)≥t) =P(Z_n(x)≥t_n(x)).

There exists a compact setC ⊂IR^k such that inf_Cf >0 andVn^t ⊂ C for all n. Observe that by Lemma 3.1 and the above remarks,

√nh^kh Z

V_n^t

P(f_n(x)≥t)dλ_g(x)− Z

V_n^t

Φ(t_n(x))dλ_g(x)ⁱ≤c λ_g(Vn^t).

Sinceλ_g(Vn^t)→0 by Lemma 3.2, one only needs now to prove that : E_n:=√

nh^k Z

Vn^t

|Φ(t_n(x))−Φ(t_n(x))|dλ_g(x)→0.

One deduces from the Lipschitz property of Φ that E_n≤c√

nh^kλ_g(Vn^t) sup

x∈V_n^t|t_n(x)−t_n(x)|. (3.3) But, by definitions oft_n(x) and t_n(x), we have for all x∈ Vn^t :

√1

nh^k|t_n(x)−t_n(x)|

≤ |t−f(x)|

1

qKf˜ (x) − 1 qV_n(x)h^−k

+ s h^k

V_n(x)|Efn(x)−f(x)|

!

≤ (logn)^β

√nh^k v u u

t|Kf˜ (x)−V_n(x)h^−k| Kf˜ (x)Vn(x)h^−k +

s h^k

V_n(x)|Efn(x)−f(x)|

! .(3.4) It is a classical exercise to deduce from H1,H3that, since Vn^t is contained inC,

sup

x∈V_n^t|Ef_n(x)−f(x)| ≤c h², and similarly, that

sup

x∈V_n^t|Kf˜ (x)−V_n(x)h^−k| ≤c h.

One deduces from (3.4) and above that sup

x∈Vn^t

|t_n(x)−t_n(x)| ≤c(√

h(logn)^β+√

nh^k+4).

Thus, by (3.3) and sincet∈Θ₀(g), one has for alln large enough : E_n≤c(logn)^β(√

h(logn)^β +√

nh^k+4),

and the latter term vanishes by assumptions onh, hence the lemma • Proof of Proposition 3.1.By Lemma 3.2, one only needs to prove Propo- sition 3.1 for allt∈Θ₀(g). Fix t∈Θ₀(g), and let

I_n:=

Z

Vn^t

Φ(t_n(x))dλ_g(x) and I_n:=

Z

V^tn

Φ(t_n(x))dλ_g(x).

By Lemma 3.3, the task is now to prove that limn

√nh^kIn= s

tK˜ 2π

Z

∂L(t)

g

k∇fkdH= lim

n

√nh^kIn.

We only show the first equality. One has I_n= 1

p2πK˜ Z

V_n^t

Z ∞

bn(x)exp− u² 2 ˜K

du dλ_g(x), where for all x∈IR^k such that f(x)>0,b_n(x) = √

nh^k(t−f(x))/f(x)^1/2. By Fubini’s theorem :

I_n= 1 p2πK˜

Z _∞

0

exp−u² 2 ˜K

λ_gf^−1hmaxt−(logn)^β

√nh^k , χ u

√nh^k 2

, tⁱdu, where for allv ≥0,χ(v) =−v/2 + (1/2)√

v²+ 4t. It is straightforward to prove the equivalence :

u∈[0, r_n]⇔χ u

√nh^k 2

≥t−(logn)^β

√nh^k ,

where r_n = (logn)^β/^qt−(logn)^β(nh^k)^−1/2, so that one can split I_n into two terms,i.e.,I_n=I_n¹+I_n², where

I_n¹ = 1 p2πK˜

Z rn

0 exp− u² 2 ˜K

λ_gf^−1hχ u

√nh^k 2

, tⁱdu and I_n² = 1

p2πK˜ Z ∞

rn

exp− u² 2 ˜K

λg

f^−1ht−(logn)^β

√nh^k , tⁱdu.

Sincet∈Θ₀(g), one has for alln large enough :

√nh^kI_n²≤c(logn)^β Z ∞

rn

exp− u² 2 ˜K

du, (3.5)

and the rightmost term vanishes. Thus, it remains to compute the limit of

√nh^kI_n¹. Using an expansion ofχ in a neighborhood of the origin, we get

limn

√nh^kλ_gf^−1hχ u

√nh^k 2

, tⁱ=u√ t

Z

∂L(t)

g

k∇fkdH, (3.6) for all u≥0, sincet∈Θ₀(g). Moreover, one deduces from Lemma 3.2 that for all nlarge enough and for all u∈[0, rn] :

√nh^kλ_gf^−1hχ u

√nh^k 2

, tⁱ ≤ c√

nh^kt−χ u

√nh^k 2

≤ c u, (3.7) because r_n/√

nh^k → 0. Thus, according to (3.5)-(3.7) and the Lebesgue theorem :

limn

√nh^kI_n = lim

n

√nh^kI_n¹

= 1

p2πK˜ Z _∞

0

exp− u² 2 ˜K

u√ t

Z

∂L(t)

g

k∇fkdHdu

= s

tK˜ 2π

Z

∂L(t)

g k∇fkdH, hence the proposition•

3.3. Proof of Proposition 3.2. From now on, we introduce two random variablesN₁,N₂with law N(0,1) such thatN₁, N₂, X₁, X₂,· · ·are indepen- dent. We let

σ_n= 1

(logn)^2βlog logn, ∀n≥2.

(As we will see later, the random variableZ_n(x) +σ_nN₁ -for instance- has a density with respect to the Lebesgue measure.) For simplicity, we assume in the following that underH3, the support ofK is contained in the euclidean unit ball ofIR^k.

Lemma 3.4. Let g : IR^k → IR₊ be a borel bounded function and assume that H2 holds. If nh^k/(logn)^2β → ∞, then for all t ∈ Θ0(g) there exists

c >0 such that forn large enough : Z

V_n^t

PⁿZ_n(x)≥t_n(x)^o∆ⁿZ_n(x) +σ_nN₁ ≥t_n(x)^odλ_g(x)≤c w_n; and

Z

V^t_n

PⁿZ_n(x)< t_n(x)^o∆ⁿZ_n(x) +σ_nN₁ < t_n(x)^odλ_g(x)≤c w_n, where w_n= (logn)^β/(nh^k) +σ_n(logn)^β/√

nh^k.

Proof.We only prove the first inequality. Let t∈Θ₀(g) and P_n:=

Z

V_n^t PⁿZ_n(x)≥t_n(x)^o∆ⁿZ_n(x) +σ_nN₁≥t_n(x)^odλ_g(x).

By independence ofN₁ and Z_n(x), P_n is smaller than Z

V_n^t

Z

exp− z² 2

PⁿZ_n(x)≥t_n(x)^o∆ⁿZ_n(x) +σ_nz≥t_n(x)^odz dλ_g(x),

and consequently, P_n≤

Z

Vn^t

Z

exp− z² 2

P|Z_n(x)−t_n(x)| ≤σ_n|z|dz dλ_g(x).

Sincet∈Θ₀(g), one deduces from Lemma 3.1 that fornlarge enough : P_n ≤ cλ_g(Vn^t)

√nh^k + Z

V_n^t

Z

exp− z² 2

P|N₁−t_n(x)| ≤σ_n|z|dz dλ_g(x)

≤ c(logn)^β

nh^k +σ_n(logn)^β

√nh^k ,

hence the lemma•

Lemma 3.5.Fix t∈Θand assume thatH1,H3 hold. Then, there exists a polynomial functionQof degree 5 defined on IR² such that for all(u₁, u₂)∈ IR² and n large enough :

Eexpiu₁Z_n(x) +u₂Z_n(y)−Eexpiu₁Z_n(x)Eexpiu₂Z_n(y)

≤ Q(|u₁|,|u₂|)

√nh^k , if x, y∈ Vn^t ∪ V^tn are such that kx−yk ≥2h.

Proof.First of all, fixu₁, u₂ ∈IR,x, y∈ Vn^t∪ V^tnand consider the following quantities :

M₁ := u₁ pnV_n(x)

hKx−X h

−EKx−X h

i

and M₂ := u₂ pnV_n(y)

hKy−X h

−EKy−X h

i.

One deduces from the inequality |exp(iw)−1−iw+w²/2| ≤ |w| ∀w∈IR that

EexpiM₁+M₂−1 + 1

2E(M₁+M₂)²

=E^hexpiM₁+M₂−1−i(M₁+M₂) +1

2(M₁+M₂)²ⁱ≤E|M₁+M₂|³. In a similar fashion, ifj= 1 or j = 2 :

Eexp(iM_j)−1 +1

2EM_j²=E^hexp(iM_j)−1−iM_j +1

2M_j²ⁱ≤E|M_j|³. Consequently,

EexpiM₁+M₂−EexpiM₁EexpiM₂

≤ E|M₁+M₂|³+1−1

2E|M₁+M₂|²−1−1

2EM₁²1−1

2EM₂² +1−1

2EM₁²E|M₂|³+1−1

2EM₂²E|M₁|³. (3.8)

It is an easy exercice to prove that for allnlarge enough, one has infV_n(x)≥ ch^k, the infinimum being taken over allx∈ Vn^t ∪ V^tn. Consequently, ifj= 1 orj= 2 :

E|M_j|³≤c√|u_j|³ n³h^k, from which we deduce that :

E|M1+M2|³≤c|u₁√|³+|u₂|³ n³h^k .

Moreover,EM₁² =u²₁/n,EM₂² =u²₂/n and for all x, y∈ Vn^t ∪ V^tn such that kx−yk ≥2h :

E(M₁+M₂)²=EM₁²+EM₂²− u₁u₂

n^pV_n(x)V_n(y)EKx−X h

EKy−X h

,

because the support ofK is contained in the unit ball and hence EKx−X

h

Ky−X h

= 0.

One deduces from above and (3.8) that for all x, y ∈ Vn^t ∪ V^tn such that kx−yk ≥2h :

EexpiM₁+M₂−EexpiM₁EexpiM₂

≤ c|u₁|³+|u₂|³

√n³h^k +(u₁u₂)²

n² +c|u₂|³(1 +u²₁) +|u₁|³(1 +u²₂)

√n³h^k +c|u₁u₂|h^k n . By assumption,nh^3k →0 so that for nlarge enough : h^k ≤1/√

nh^k. Con- sequently,

EexpiM₁+M₂−EexpiM₁EexpiM₂≤ Q(|u1|,|u2|)

√nh^k , whereQis defined for all u₁, u₂ ∈IR by :

Q(u₁, u₂) =c(u³₁+u³₂+ (u₁u₂)²+u₁u₂+u²₂u³₁+u³₁u²₂).

Consequently, for allu₁, u₂ ∈IR andx, y∈ Vn^t∪ V^tnsuch thatkx−yk ≥2h:

Eexpiu₁Z_n(x) +u₂Z_n(y)−Eexpiu₁Z_n(x)Eexpiu₂Z_n(y)

= EexpiM₁+M₂ⁿ−EexpiM₁EexpiM₂ⁿ

≤ nEexpiM₁+M₂−EexpiM₁EexpiM₂

≤ Q(|u₁|,|u₂|)

√nh^k , hence the lemma•

In the following,uv stands for the usual scalar product ofu, v ∈IR². Lemma 3.6.Letx, y∈IR^kbe such thatV_n(x)V_n(y)6= 0. Then, the bivariate random variable

Zn(x) +σnN1

Z_n(y) +σ_nN₂

!

has a densityϕ^x,y_n defined for all u∈IR² by ϕ^x,y_n (u) = 1

4π² Z

E^hexpiv₁Z_n(x)+v₂Z_n(y)ⁱexp−i uv−1

2σ²_nkvk²dv.

Proof.By independence of X₁,· · ·, X_n, N₁ and N₂, the random variable Z_n(x)

Z_n(y)

!

+σ_n N₁ N₂

!

has a densityϕ^x,y_n defined for allu= (u₁, u₂)∈IR² by ϕ^x,y_n (u) = 1

2πσ_n²E^hexp−(u₁−Z_n(x))² 2σ_n²

exp− (u₂−Z_n(y))² 2σ²_n

i. Using the equality

1

p2πσ²_nexp− z² 2σ²_n

= 1 2π

Z

exp−izw−1

2σ²_nw²dw ∀z∈IR, we deduce from the Fubini theorem that

ϕ^x,y_n (u) = 1 4π²

Z

E^hexpiv₁Z_n(x) +v₂Z_n(y)ⁱexp−iuv−1

2σ²_nkvk²dv, hence the lemma•

Proof of Proposition 3.2.We only prove the first equality of Proposition 3.2. According to Lemma 3.2, one only needs to prove the result for each t∈Θ₀(g). Hence we fixt∈Θ₀(g) and we put :

A_n(x) =ⁿZ_n(x)≥t_n(x)^o, A^j_n(x) =ⁿZ_n(x) +σ_nN_j ≥t_n(x)^o, j= 1,2, for allx ∈IR^k such that V_n(x)6= 0. First note that since the events A_n(x) and{f_n(x)≥t} are equal, one has

var^hλ_gVn^t∩ Ln(t)ⁱ

= Z

(V_n^t)^×2

P(A_n(x)∩A_n(y))−P(A_n(x))P(A_n(y))dλ^⊗2_g (x, y). (3.9) But, by Lemma 3.4 and sincet∈Θ₀(g), one has for all nlarge enough :

nh^k Z

(Vn^t)^×2

P(A_n(x)∩A_n(y))−P(A¹_n(x)∩A²_n(y))dλ^⊗2_g (x, y)

≤ 2nh^kλ_g(Vn^t) Z

Vn^t

P(A_n(x)∆A¹_n(x))dλ_g(x)

≤ c(logn)^β√

nh^k(logn)^β

nh^k +σ_n(logn)^β

√nh^k

≤ c(logn)^2β

√nh^k +σ_n(logn)^2β,

and the latter term tends to 0 by assumption. In a similar fashion, one can prove that

nh^k Z

(V_n^t)^×2

P(A_n(x))P(A_n(y))−P(A¹_n(x))P(A²_n(y))dλ^⊗2_g (x, y)→0.

By the above results and (3.9), it remains to show that nh^k

Z

(Vn^t)^×2

P(A¹_n(x)∩A²_n(y))−P(A¹_n(x))P(A²_n(y))dλ^⊗2_g (x, y)→0. (3.10) Let T(h) = {(x, y) ∈ (IR^k)^×2 : kx −yk ≤ 2h}. According to the Fubini theorem,

nh^kλ^⊗2_g (Vn^t)^×2∩T(h) = nh^k Z

Vn^t

λ_gVn^t ∩B(x,2h)dλ_g(x)

≤ nh^k Z

Vn^t

λ_g(B(x,2h))dλ_g(x),

whereB(z, r) stands for the euclidean closed ball with center atz∈IR^kand radiusr >0. Sincet∈Θ₀(g), one deduces that

nh^kλ^⊗2_g (Vn^t)^×2∩T(h) ≤ c nh^k(logn)^β

√nh^k h^k

≤ c^qnh^3k(logn)^2β, so that, by assumption on the bandwidthh :

limn nh^kλ^⊗2_g (Vn^t)^×2∩T(h)= 0.

Let nowSn= (Vn^t)^×2∩T(h)^c. According to (3.10) and the above result, one only needs now to prove that :

nh^k Z

Sn

P(A¹_n(x)∩A²_n(y))−P(A¹_n(x))P(A²_n(y))dλ^⊗2_g (x, y)→0. (3.11) By Lemmas 3.5 and 3.6, one has for allx, y∈ Sn :

P(A¹_n(x)∩A²_n(y))−P(A¹_n(x))P(A²_n(y))

≤ Z

Eexpiu₁Z_n(x) +u₂Z_n(y)

−Eexpiu₁Z_n(x)Eexpiu₂Z_n(y)exp−1

2σ_n²kuk²du₁du₂

≤ 1

√nh^k Z

Q(|u1|,|u2|) exp−1

2σ²_nkuk²du1du2

≤ c

σ⁷_n√ nh^k,

where Q is the polynomial function defined in Lemma 3.5. Consequently, one has for all nlarge enough :

nh^k Z

Sn

P(A¹_n(x)∩A²_n(y))−P(A¹_n(x))P(A²_n(y))dλ^⊗2_g (x, y)

≤ c

√nh^k

σ_n⁷ λ^⊗2_g (Sn)

≤ c

√nh^k

σ_n⁷ λ_g(Vn^t)²

≤ c(logn)^2β σ_n⁷√

nh^k,

which tends to 0 by assumption, hence (3.11)• 4. Proof of Corollary 2.1.

Lemma 4.1.Letk≥2 and assume thatH1-H3hold. Ifnh^k+4(logn)² →0 andnh^k/(logn)¹⁶→ ∞, then for a.e. t∈Θ :

√nh^kλ_fn(L(t))−λ_fn(Ln(t))→^P 0.

Proof.Lett∈Θ be such that the conclusion of Theorem 2.1 holds both for g≡f and g≡1. Notice that

λ_fn(L(t))−λ_fn(Ln(t)) = Z

f_n1_{f≥t}−1_{fn≥t}dλ

= Z

L(t)

f_n1_{fn<t}dλ− Z

L(t)^c

f_n1_{fn≥t}dλ.

As in the proof of Theorem 2.1, we see that the result of the lemma will hold if we show that√

nh^kK_n→^P 0, where Kn :=

Z

V^t_nfn1_{fn<t}dλ− Z

V_n^t fn1_{fn≥t}dλ.

SplitK_n into four terms as follows : Kn =

Z

V^t_n(fn−f)1_{fn<t}dλ− Z

V_n^t(fn−f)1_{fn≥t}dλ +

Z

V^t_n

1_{fn<t}dλ_f − Z

V_n^t

1_{fn≥t}dλ_f. (4.1)

On one hand, it is a classical exercise to deduce from H1,H3that sup

V^tn

|f_n−f|→^P 0.

Thus, using (3.2),

√nh^k Z

V^t_n(f_n−f)1_{fn<t}dλ→^P 0.

In a similar fashion :

√nh^k Z

Vn^t

(f_n−f)1_{fn≥t}dλ→^P 0.

On the other hand, we get from (3.2) that : limn

√nh^k Z

Vn^t

1_{fn≥t}dλ_f = lim

n

√nh^k Z

V^tn

1_{fn<t}dλ_f,

where the limits are in probability. By the above results and (4.1),√ nh^kK_n tends to 0 in probability, hence the lemma•

Lemma 4.2.Letk≥2,t∈Θand assume that H1,H3hold. Ifnh^k+4 →0,

then : √

nh^kλ_f(L(t))−λ_fn(L(t))→^P 0.

Proof.Observe that

λ_f(L(t))−λ_fn(L(t)) = Z

L(t)(f−Efn)dλ+ Z

L(t)(Efn−fn)dλ.

According to H1,H3, we have : Z

L(t)|f −Ef_n|dλ≤ch², and since nh^k+4→0, we only need to prove that

√nh^k Z

L(t)

(Ef_n−f_n)dλ→^P 0.

We prove that this convergence holds in quadratic mean. We have : E√

nh^k Z

L(t)

(Ef_n−f_n)dλ² ≤ 1 h^kE

Z

L(t)

Kx−X h

dx²

≤ 1 h^k

Z

L(t)^×2EKx−X h

Ky−X h

dxdy.

Recall that we assume in Section 3.3 that the support of K is contained in the unit ball so that ifkx−yk ≥2h,

EKx−X h

Ky−X h

= 0.

Letting R(h) ={(x, y) ∈ L(t)^×2 : kx−yk ≤ 2h}, one deduces from above that

E√ nh^k

Z

L(t)(Ef_n−f_n)dλ² ≤ c h^k

Z

R(h)

Z

Kx−u h

f(u)dudxdy

≤ c Z

R(h)

Z

K(v)f(x−hv)dvdxdy

≤ c λ^⊗2(R(h))

≤ c Z

L(t)

λL(t)∩B(x,2h)dx, according to the Fubini theorem. Thus, we get :

E√ nh^k

Z

L(t)(Efn−fn)dλ² ≤ch^k, hence the lemma•

Lemma 4.3. Let p ∈ [0,1] and assume that H1, H3 and H4 hold. If nh^k/logn→ ∞, then t^(p)_n →t^(p) a.s.

Proof. Let t = t^(p) and t_n = t^(p)_n . As seen in the proof of Theorem 2.1, sup_IRk|f_n−f| →0 a.s. Hence, one can fix

ω∈ⁿsup

IR^k |f_n−f| →0^o.

For notational convenience, we omitω until the end of this proof. Sincef is bounded, one has sup_nsup_IR^kf_n<∞and consequently sup_nt_n<∞. Thus,