The law of the iterated logarithm for the multivariate kernel mode estimator

THE LAW OF THE ITERATED LOGARITHM

FOR THE MULTIVARIATE KERNEL MODE ESTIMATOR

Abdelkader Mokkadem

and Mariane Pelletier

Abstract. Letθ be the mode of a probability density and θn its kernel estimator. In the case θ is nondegenerate, we first specify the weak convergence rate of the multivariate kernel mode estimator by stating the central limit theorem for θn−θ. Then, we obtain a multivariate law of the iterated logarithm for the kernel mode estimator by proving that, with probability one, the limit set of the sequenceθn−θsuitably normalized is an ellipsoid. We also give a law of the iterated logarithm for the l^p norms,p∈[1,∞], ofθn−θ. Finally, we consider the caseθ is degenerate and give the exact weak and strong convergence rate ofθn−θin the univariate framework.

Mathematics Subject Classification. 62G05, 62G20, 60F05, 60F15.

Received March 1, 2002. Revised May 2, 2002.

1. Introduction

Let X₁, . . . , X_n be a sequence of independent and identically distributed R^d-valued random variables with unknown probability densityf. We assumef has a unique modeθ, that is, we suppose that there existsθ∈R^d such thatf(x)< f(θ) for anyx6=θ.

A natural approach to mode estimation is to locate the absolute maximum of a corresponding empirical density function; Parzen [19] gives consistency and asymptotic normality for such estimates based on kernel method. His results have been extended mainly by Chernoff [3], Nadaraya [17], Van Ryzin [28], R¨uschendorf [23], Eddy [5, 6], Romano [22] and Grund and Hall [10]. An approach based on order statistics is introduced by Grenander [9] and Venter [29], and developed by Sager [24] and Hall [12]. A recursive method is proposed by Tsybakov [27]. A question closely related to the mode estimation is the estimation of the conditional mode developed by Collombet al[4], and extended by Samanta and Thavaneswaran [26], Ould–Said [18], Quintela–

Del–Rio and Vieu [21], Berlinetet al.[2] and Louani and Ould–Said [15].

The present paper deals with the classical Parzen mode estimator. Let (h_n)_n≥1 be a sequence of positive real numbers such that lim_n→∞h_n = 0, lim_n→∞nh^d_n = ∞, and let K be a continuous function satisfying R

R^dK(x)dx = 1, lim_kxk→∞K(x) = 0 (where k.k is an arbitrary norm on R^d). The well-known Parzen–

Rosenblatt kernel estimator of the densityf is defined by f_n(x) = 1

nh^d_n Xn k=1

K

x−X_k h_n

for any x∈R^d

Keywords and phrases: Density, mode, kernel estimator, central limit theorem, law of the iterated logarithm.

1 D´epartement de Math´ematiques, Universit´e de Versailles-Saint-Quentin, 45 avenue des ´Etats-Unis, 78035 Versailles Cedex, France; e-mail: (mokkadem,pelletier)@math.uvsq.fr

c EDP Sciences, SMAI 2003

and the kernel mode estimator is any random variableθ_n satisfying f_n(θ_n) = sup

x∈R^df_n(x). (1)

Since K is continuous and vanishing at infinity, the choice of θ_n as a random variable satisfying (1) can be made with the help of an order onR^d. For example, one can consider the following lexicographic order: x≤y ifx=y or if the first nonzero coordinate ofx−y is negative. The definition

θ_n = inf

y∈R^d such that f_n(y) = sup

x∈R^df_n(x)

where the infimum is taken with respect to the lexicographic order on R^d, ensures the measurability of the kernel mode estimator.

The weak consistency of θ_n was established by Parzen [19] and Yamato [31], its strong consistency by Nadaraya [17], Van Ryzin [28], R¨uschendorf [23] and Romano [22]. In the univariate framework, that is when d = 1, Romano [22] proves the almost sure convergence of θ_n to θ under the optimal assumption on the bandwidth lim_n→∞nh_n[lnn]⁻¹ = ∞. We first give a straightforward extension of his Theorem 1.1 to the multivariate case and obtain the strong consistency ofθ_n under the condition lim_n→∞nh^d_n[lnn]⁻¹=∞, which weakens the assumptions on the bandwidth made in Van Ryzin [28] and R¨uschendorf [23].

Let us now assume θ is nondegenerate, that is, D²f(θ) (the second order differential at the point θ) is nonsingular.

The weak convergence rate ofθ_ntoθwas first studied in the univariate framework by Parzen [19] who proved that, if h_n is chosen such that lim_n→∞nh⁶_n = ∞ and lim_n→∞nh⁷_n = 0, then, under appropriate smoothness assumptions,

pnh³_n(θ_n−θ)→ N^D 0, f(θ) [f⁰⁰(θ)]²

Z

RK⁰²(x)dx

!

where→^D denotes the convergence in distribution andN the Gaussian-distribution. Eddy [5,6] and Romano [22]

then proved that this central limit theorem still holds when the condition lim_n→∞nh⁶_n =∞ is weakened to lim_n→∞nh⁵_n[lnn]⁻¹=∞.

The study of the weak convergence rate of θ_n to θ was extended by Konakov [13] and Samanta [25] to the multivariate framework. The key idea to establish the convergence rate ofθ_ntoθis to note that, as soon asD²f_n converges almost surely uniformly toD²f in a neighborhood ofθ, the asymptotic behaviour ofθ_n−θis given by that of−

D²f(θ)₋₁

∇f_n(θ), where∇f_n denotes the gradient off_n. The condition on the bandwidth required by Konakov [13] and Samanta [25] to ensure the strong uniform convergence of D²f_n is lim_n→∞nh^2d+4_n =∞.

Although this condition is equivalent to the one of Parzen [19] whend= 1, it is too strong to establish a central limit theorem as soon asd≥2. The reason is the following.

The weak convergence rate of∇f_n(θ) to zero is governed by the weak convergence rate of the variance term

∇f_n(θ)−E (∇f_n(θ)) on one hand and by the deterministic convergence rate of the bias term E (∇f_n(θ)) on the other hand. Since the variance term converges at the ratep

nh^d+2_n and the bias term at the rateh⁻²_n , the condition lim_n→∞nh^d+6_n = 0 is necessary to make the bias term negligible in front of the variance term, and thus to establish a central limit theorem for ∇fn(θ). The incompatibility of this last condition with the one required by Konakov [13] and Samanta [25] for the strong uniform convergence ofD²f_n prevents the transfer of a central limit theorem established for∇f_n(θ) to one which would hold forθ_n−θ. By weakening the condition lim_n→∞nh^2d+4_n =∞ of Konakov [13] and Samanta [25] to lim_n→∞nh^d+4_n [lnn]⁻¹=∞, we make possible the choice of a bandwidth for which D²f_n converges a.s. uniformly to D²f and for which the bias of ∇f_n(θ) is

negligible in front of its variance term. This allows us to state the following central limit theorem:

q

nh^d+2_n (θ_n−θ)→ N^D

0, f(θ)

D²f(θ)₋₁ G

D²f(θ)₋₁ whereGis thed×dmatrix defined by

G_i,j = Z

R^d

∂K

∂x_i(x)∂K

∂x_j(x)dx.

We now come to our main objective, which is to prove the law of the iterated logarithm for the multivariate kernel mode estimator.

In the univariate framework, upper bounds of the almost sure convergence rate ofθ_n are given in Eddy [5], Vieu [30] and Leclerc and Pierre–Loti–Viaud [14]. The exact strong convergence rate of the univariate kernel mode estimator is given in Mokkadem and Pelletier [16] who proved the following law of the iterated logarithm:

lim sup

n→∞

r nh³_n

2 ln lnn(θ_n−θ) =−lim inf

n→∞

r nh³_n

2 ln lnn(θ_n−θ) = q

f(θ)R

RK⁰²(x)dx

|f⁰⁰(θ)| a.s. (2) Our main result in the present paper is the following multivariate law of the iterated logarithm: with probability one, the sequence

s nh^d+2_n

2 ln lnn(θ_n−θ) is relatively compact and its limit set is the ellipsoid

ν∈R^d such that 1 f(θ) ν^t

D²f(θ) G⁻¹

D²f(θ) ν≤1

·

Note that the unidimensional version of this result can be written as: with probability one, the sequence r nh³_n

2 ln lnn(θ_n−θ) is relatively compact and its limit set is the interval



−q f(θ)R

RK⁰²(x)dx

|f⁰⁰(θ)| ; + q

f(θ)R

RK⁰²(x)dx

|f⁰⁰(θ)|





and thus extends the univariate result (2).

We also establish a law of the iterated logarithm for thel^pnorms,p∈[1,∞], of the vector (θn−θ). For sake of simplicity, we state here our results in the two striking cases, that is, forp= 2 and p=∞. For any vector x= (x₁, . . . , x_d)^t∈R^d, set kxk₂ =hP_d

i=1x²_i i_1/2

andkxk_∞= max_1≤i≤d|xi|. We prove that, with probability one, the sequence

s nh^d+2_n

2 ln lnn kθn−θk₂

is relatively compact and its limit set is the interval [0, δ₂p

f(θ)] whereδ₂is the spectral radius,i.e. the largest eigenvalue, of the matrix−G^1/2

D²f(θ)₋₁

. We also establish that, with probability one, the sequence s

nh^d+2_n

2 ln lnn kθn−θk_∞ is relatively compact and its limit set is the interval

h

0, δ_∞p f(θ)

i

where δ_∞ is the square root of the largest diagonal term of the matrix

D²f(θ)₋₁ G

D²f(θ)₋₁ .

These different versions of the law of the iterated logarithm for the multivariate kernel mode estimator are proved by relating the strong behaviour of (θ_n−θ) to the one of

D²f(θ)₋₁

∇fn(θ) and by applying a result of Arcones [1].

Let us finally consider the case θ is degenerate. To our knowledge, the only result in this framework is an upper bound of the complete (and thus almost sure) convergence rate ofθ_n−θ stated in Vieu [30] in the univariate case. We specify this by establishing the exact weak and strong convergence rate ofθ_n−θ in the cased= 1. The degenerate multivariate case seems very intricate and the convergence rate of the kernel mode estimator in this framework remains an open question.

Our assumptions and results are stated in Section 2, whereas Section 3 is devoted to the proofs.

2. Assumptions and results

Before stating our assumptions, let us first define the covering number condition. Let Q be a probability on R^d and let F ⊂ Ls(Q), s ∈ {1,2}, be a class of Q-integrable functions. The Ls-covering number (see Pollard [20]) is the smallest valueN_s(ε, Q,F) ofmfor which there existm functionsg₁, . . . , g_m∈ Ls(Q) such that

i∈{1,... ,m}min kf−g_ik_Ls(Q)≤ε ∀f ∈ F (3)

(if no suchmexists,N_s(ε, Q,F) =∞). Now, let Λ be aR-valued function defined onR^d, and letF(Λ) be the class of functions defined by

F(Λ) =

z7→Λ x−z

h

, h >0, x∈R^d

·

Λ is said to satisfy theLs-covering number condition if Λ is bounded and integrable on R^d, and if there exist A >0 andw >0 such that, for any probability QonR^d and any ε∈]0,1[,

N_s(ε, Q,F(Λ))≤Aε^−w. (4)

In the caseFis uniformly bounded by a constantM, one can consider in (3) only the approximating functionsg_i such that kgik_∞ ≤M (see Pollard [20]). In this case, simple inequalities show that the L₁-covering number condition is equivalent to theL₂-covering number condition. Since the only classes we consider areF(Λ) classes with Λ bounded, we shall only refer to the “covering number condition” without distinction.

The classes which satisfy (4) are often called VC classes. Whend= 1, the real-valued kernels with bounded variations satisfy the covering number condition (see Pollard [20]). Some examples of multivariate kernels satisfying the covering number condition are the following:

– the kernels defined as K(x) = ψ(kxk), where ψ is a real-valued function with bounded variations (see Pollard [20]);

– the kernels defined as K(x) = Q_d

i=1K_i(x_i) where the K_i, 1 ≤ i ≤ d, are real-valued functions with bounded variations (this follows from Lem. A1 in Einmahl and Mason [7]);

– the kernels satisfying the assumption (K1) of Gin´e and Guillou [8].

The assumptions we require for the strong consistency ofθ_n are the following:

(A1) i) lim_kxk→∞K(x) = 0 andR

R^dK(x)dx= 1;

ii)K is continuous onR^d;

iii)K satisfies the covering number condition.

(A2) i) lim_n→∞h_n = 0;

ii) lim_n→∞nh^d_n[lnn]⁻¹=∞.

(A3) i) there existsθ∈R^d such thatf(x)< f(θ) for allx6=θ;

ii)f is continuous in a neighbourhoodV ofθ;

iii) sup_{x /∈V}f(x)< f(θ).

Theorem 2.1 (Strong consistency). Assume (A1–A3) hold. Moreover, assume either that K is nonnegative or that f is uniformly continuous on R^d. Then,

n→∞lim θ_n=θ a.s.

Remarks.

1) In the casef is uniformly continuous on R^d, (A3) iii) is a straightforward consequence of the unicity of the mode off.

2) Theorem 2.1 is a straightforward extension of Theorem 1.1 of Romano [22] to the multivariate framework; it weakens the assumptions on the bandwidth made by Van Ryzin [28] and R¨uschendorf [23] in the cased≥2.

In order to state the central limit theorem, we need the following additional assumptions:

(A4) lim_n→∞nh^d+4_n [lnn]⁻¹=∞.

(A5) i) K is twice differentiable on R^d and, for any (i, j) ∈ {1, . . . , d}², ∂²K/∂x_i∂x_j satisfies the covering number condition;

ii) for anyi∈ {1, . . . , d},R

R^d

∂K

∂xi(x) ₂

dx <∞andR

R^d_∂x^∂K_i(x)^2+δdx <∞for some δ >0;

iii) there existsq≥2 such that for anys∈ {1, . . . , q−1}and anyj∈ {1, . . . , d}, Z

Ry^s_jK(y)dy_j = 0 and Z

R^d

y_j^qK(y)dy <∞.

(A6) i)f is twice differentiable onR^d,D²f is continuous in a neighborhood ofθ, andD²f(θ) is nonsingular;

ii)D²f is bounded onR^d;

iii) for any i∈ {1, . . . , d},∂f /∂x_i is bounded onR^d;

iv)f isq+ 1 times differentiable at the pointθ (qis defined in (A5)).

Remarks.

1) Some conditions in (A4–A6) clearly imply some other conditions already required in (A1–A3). For instance, since lim_n→∞h_n= 0, (A2) ii) is included in (A4).

2) Sinceθ is assumed to be the unique mode off, (A6) i) implies thatD²f(θ) is negative definite.

3) Note that (A6) iii) implies the uniform continuity off onR^d.

4) Let us finally mention that the condition (A6) ii) is useless as soon as the support ofK is bounded.

Let us setB_q(θ) the vector

B_q(θ) =(−1)^q+1 q!

D²f(θ)₋₁

∇



X^d

j=1

µ^q_j∂^qf

∂x^q_j (θ)



 with µ^q_j= Z

R^dy^q_jK(y)dy

where∇ denotes the gradient and recall that

G= (G_i,j)_1≤i,j≤d with G_i,j= Z

R^d

∂K

∂x_i(x)∂K

∂x_j(x)dx.

Theorem 2.2 (Central limit theorem). Let assumptions (A1–A6) hold.

i) If lim_n→∞nh^d+2q+2_n = 0, then q

nh^d+2_n (θ_n−θ)→ N^D

0 , f(θ)

D²f(θ)₋₁ G

D²f(θ)₋₁ . ii) If there exists c >0 such thatlim_n→∞nh^d+2q+2_n =c, then

q

nh^d+2_n (θ_n−θ)→ N^D √

cB_q(θ), f(θ)

D²f(θ)₋₁ G

D²f(θ)₋₁ .

iii) Iflim_n→∞nh^d+2q+2_n =∞, then

1 h^qn

(θ_n−θ)→^P B_q(θ).

As mentioned in the introduction, the weak convergence rate of the kernel mode estimator is given by that of D²f(θ)₋₁

∇fn(θ), which depends itself on the convergence rate of the variance term∇fn(θ)−E (∇fn(θ)) and of the bias term E (∇fn(θ)). Part i) of Theorem 2.2 corresponds to the case the bias term does not interfer, Part ii) holds when (h_n) is chosen such that the bias and the variance terms are balanced and Part iii) describes the case when the variance term does not interfer. Let us note that, in their study of the weak convergence rate of (θ_n−θ), Konakov [13] and Samanta [25] consider kernels of orderq= 2. Their condition on the bandwidth lim_n→∞nh^2d+4_n =∞ required to ensure the strong uniform convergence ofD²f_n implies lim_n→∞nh^d+6_n =∞ and leads to the weak convergence ofh⁻²_n (θ_n−θ) to a degenerate distribution. Let us finally mention that the higher order of weak convergence is attained forh_n ∼n−1/(d+2q+2).

The limit laws in Parts i) and ii) of Theorem 2.2 are nondegenerate. As a matter of fact, we shall prove the following proposition:

Proposition 2.3. IfK6≡0is continuously differentiable and vanishing at infinity, then the matrixGis positive definite.

In order to prove the law of the iterated logarithm for the kernel mode estimator, we require the following additional assumption:

(A7) There existsM >0 such thatK(x) = 0 ifkxk> M.

Theorem 2.4 (Law of the iterated logarithm for the full vector). Let assumptions (A1–A7) hold.

i) If lim_n→∞nh^d+2q+2_n /ln lnn= 0, then, with probability one, the sequence s

nh^d+2_n

2 ln lnn (θ_n−θ) is relatively compact and its limit set is the ellipsoid

C=

ν ∈R^d such that 1 f(θ)ν^t

D²f(θ) G⁻¹

D²f(θ) ν ≤1

·

ii) If there exists c >0 such thatlim_n→∞nh^d+2q+2_n /ln lnn=c, then, with probability one, the sequence s

nh^d+2_n

2 ln lnn (θ_n−θ) is relatively compact and its limit set is the ellipsoid

C= (

ν∈R^d such that 1 f(θ)

ν−

rc 2 B_q(θ)

t

D²f(θ) G⁻¹

D²f(θ) ν− rc

2 B_q(θ)

≤1 )

·

iii) Iflim_n→∞nh^d+2q+2_n /ln lnn=∞, then

n→∞lim 1

h^q_n(θ_n−θ) =B_q(θ) a.s.

The strong convergence rate of θ_n−θ is deduced from that of

D²f(θ)₋₁

∇fn(θ). Similarly to the study of its weak convergence rate, three cases have to be considered according to the choice of the bandwidth. Part i) of Theorem 2.4 corresponds to the case the bias term does not interfer, Part ii) to the case the bias and the variance terms are balanced and Part iii) to the case the variance term does not interfer. Let us underline that the conditions on the bandwidth which differentiate the three possible a.s. behaviours of the sequence (θ_n−θ) are slightly different from those which determine the weak convergence rate of the kernel mode estimator. So, the choice ofh_n which gives the optimal a.s. rate of convergence ofθ_n, that is,h_n ∼[ln lnn]^1/(d+2q+2)n−1/(d+2q+2), is not the choice of the bandwidth which ensures the optimal weak convergence rate ofθ_n.

For sake of simplicity, we shall state the next versions of the law of the iterated logarithm for the multivariate kernel mode estimator only in the case the bias term is negligible; the two other cases can be easily deduced.

Theorem 2.5 (Law of the iterated logarithm for the linear forms). Let assumptions (A1–A7) hold, assume that lim_n→∞nh^d+2q+2_n /ln lnn= 0, and set u∈R^d. Then, with probability one, the sequence

s nh^d+2_n

2 ln lnn u^t(θ_n−θ) is relatively compact and its limit set is

I(u) =

−q

f(θ)u^t[D²f(θ)]⁻¹G[D²f(θ)]⁻¹u; + q

f(θ)u^t[D²f(θ)]⁻¹G[D²f(θ)]⁻¹u

.

Note that the application of Theorem 2.5 to the i-th vector of the canonical basis ofR^d gives the limit set of

the sequence s

nh^d+2_n

2 ln lnn (θ_n,i−θ_i) that is, the law of the iterated logarithm for thei-th coordinate ofθ_n.

To conclude our study on the multivariate kernel mode estimator in the caseθ is nondegenerate, we finally state the law of the iterated logarithm for the l^p norms of (θ_n−θ). To this end, for any matrixA and any p∈[1,∞], we denote by|||A|||_2,p the matrix norm defined by

|||A|||_2,p= sup

||x||2≤1||Ax||p

where||x||p is thel^p vector norm: kxkp=hP_d

i=1|x_i|^pi_1/p

forp∈[1,∞[ andkxk∞= max_1≤i≤d|x_i|.

Theorem 2.6 (Law of the iterated logarithm for thel^p norms). Let assumptions (A1–A7) hold and assume that lim_n→∞nh^d+2q+2_n /ln lnn= 0. Setp∈[1,∞]; with probability one, the sequence

s nh^d+2_n

2 ln lnn kθn−θk_p is relatively compact and its limit set is the interval [0, δ_pp

f(θ)]with δ_p=G^1/2

D²f(θ)₋₁

2,p. In particular, for p= 2, δ₂ is the spectral radius of the matrix −G^1/2

D²f(θ)₋₁

and, for p=∞, δ_∞ is the square root of the largest diagonal term of the matrix

D²f(θ)₋₁ G

D²f(θ)₋₁ .

Let us finally consider the caseθmay be degenerate. For that purpose, we setd= 1 and require the following assumptions:

(A’4) i) There exists p≥2 such that f is p-times differentiable onR, f^(j)(θ) = 0 for any j ∈ {1, . . . , p−1}, andf^(p)(θ)6= 0;

ii) for anyj ∈ {1, . . . , p},f^(j) is bounded onR. (A’5) i) Kisp-times differentiable onR;

ii) for anyj ∈ {1, . . . , p},K^(j) satisfies the covering number condition;

iii) there existsq≥psuch thatR

Ry^jK(y)dy= 0 for anyj∈ {p−1, . . . , q−1} andR

R|y^qK(y)|dy <∞;

iv)f isq+ 1 times differentiable onRandf^(q+1) is bounded onR.

(A’6) i) lim_n→∞nh^2p+1_n [ln(1/h_n)]^−p=∞;

ii) (h_n) is a decreasing sequence and lim_n→∞ln(1/h_n)[ln lnn]⁻¹=∞;

iii) either (nh_n) is an increasing sequence or there exists csuch thath_n≤ch_2n. Remarks.

1) Sincef(θ) is a maximum off, the integerpdefined in (A’4) i) is even.

2) Note that the casef^(p)(θ) = 0 for allpis not covered by (A’4).

3) Any even kernel with a finite moment of orderpsatisfies the assumption (A’5) iii) with q=p.

4) Note that (A’5) iv) implies the continuity off^(p). Set

B_p,q(θ) = (−1)^q+1 (p−1)!f^(q+1)(θ) q! f^(p)(θ)

Z

Ry^qK(y)dy.

Theorem 2.7 (Weak convergence rate of the univariate kernel mode estimator in the degenerate case). Let assumptions (A1–A3) and (A’4–A’6) hold.

i) If there existsc≥0 such that lim_n→∞nh^2q+3_n =c, then nh³_n1/2(p−1)

(θ_n−θ)→^DZ

where the random variableZ^p−1 isN √

cB_p,q(θ), ^{[(p−1)!]2f(θ)} [f^(p)(θ)]²

R

RK⁰²(x)dx

– distributed.

ii) If lim_n→∞nh^2q+3_n =∞, then

1

h^q/(p−1)_n (θ_n−θ)→^P [B_p,q(θ)]^1/(p−1).

Theorem 2.8 (Strong convergence rate of the univariate kernel mode estimator in the degenerate case). Let as- sumptions (A1–A3, A’4–A’6) and (A7) hold.

i) If there existsc≥0 such that lim_n→∞nh^2q+3_n /ln lnn=c, then, with probability one, the sequence nh³_n

2 ln lnn

_1/2(p−1)

(θ_n−θ)

is relatively compact and its limit set is the interval

I=







r c

2B_p,q− (p−1)!

q f(θ)R

RK⁰²(x)dx

|f^(p)(θ)|





1/(p−1)

;



r c 2B_p,q+

(p−1)!

q f(θ)R

RK⁰²(x)dx

|f^(p)(θ)|





1/(p−1)

·

ii) If lim_n→∞nh^2q+3_n /ln lnn=∞, then

n→∞lim 1

h^q/(p−1)_n (θ_n−θ) = [B_p,q(θ)]^1/(p−1) a.s.

Remark. Part i) of Theorem 2.8 implies that





















 lim inf

n→∞

nh³_n 2 ln lnn

_1/2(p−1)

(θ_n−θ) =



r c

2B_p,q− (p−1)!

q f(θ)R

RK⁰²(x)dx

|f^(p)(θ)|





1/(p−1)

lim sup

n→∞

nh³_n 2 ln lnn

_1/2(p−1)

(θ_n−θ) =



r c 2B_p,q+

(p−1)!

q f(θ)R

RK⁰²(x)dx

|f^(p)(θ)|





1/(p−1)

·

This particular result has been obtained in Mokkadem and Pelletier [16] in the nondegenerate case (i.e. when p= 2) by applying a result of Hall [?] and without the assumption (A7) on the boundedness of the support ofK.

3. Proofs 3.1. Consistency of the mode estimator

We first note that, following the proof of Theorem 1.1 in Romano [22], the application of Theorem 37 (p. 34) in Pollard [20] gives the following lemma:

Lemma 3.1. Let Λ be a function on R^d satisfying the covering number condition. If there exists j ≥0 such that lim_n→∞nh^d+2j_n [lnn]⁻¹=∞, then

n→∞lim sup

x∈R^d

1 nh^d+j_n

Xn i=1

Λ

x−X_i h_n

−E

Λ

x−X_i h_n

= 0 a.s.

The application of Lemma 3.1 with Λ =K ensures that, under the assumption lim_n→∞nh^d_n[lnn]⁻¹=∞,

n→∞lim sup

x∈R^d|f_n(x)−E (f_n(x))|= 0 a.s. (5)

Now, setδ >0 such thatB(θ, δ) =

x∈R^d / kθ−xk< δ ⊂ V and setm_δ = sup_{x /∈B(θ,δ)}f(x); by assumption, we havem_δ< f(θ). Then, setε∈]0, f(θ)−m_δ[; fornlarge enough, we have

sup

x /∈B(θ,δ)E (f_n(x))< f(θ)−ε (6) this last inequality being either proved by following Romano [22] in the caseKis nonnegative or being deduced from the uniform convergence of E(f_n) to f in the case f is uniformly continuous onR^d. The combination of (5) and (6) then ensures that almost surely, fornlarge enough,

sup

x /∈B(θ,δ)f_n(x)< f(θ)−ε

2 · (7)

Since lim_n→∞f_n(θ) =f(θ) a.s., we have a.s., fornlarge enough,f_n(θ)> f(θ)−ε/2 and thusf_n(θ_n)> f(θ)−ε/2.

In view of (7), it follows thatθ_n∈ B(θ, δ) a.s. fornlarge enough, which proves Theorem 2.1.

3.2. Connection between the convergence rate of the mode estimator and that of the variance term of the derivative density estimator

∇fn(θ_n)− ∇fn(θ) =−∇fn(θ). (8)

For eachi∈ {1, . . . , d}, Taylor’s expansion applied to the real-valued application ^∂f_∂xⁿ_i implies the existence of ξ_n(i) = (ξ_n,1(i), . . . , ξ_n,d(i))^tsuch that







∂f_n

∂x_i (θ_n)−∂f_n

∂x_i (θ) = Xd j=1

∂²f_n

∂x_i∂x_j(ξ_n(i)) (θ_n,j−θ_j)

|ξn,j(i)−θ_j| ≤ |θn,j(i)−θ_j| ∀j ∈ {1, . . . , d}·

Define thed×dmatrixH_n= (H_n,i,j)_1≤i,j≤d by setting H_n,i,j= ∂²f_n

∂x_i∂x_j (ξ_n(i)). Equation (8) can then be rewritten as

H_n(θ_n−θ) =−∇fn(θ). (9)

Now, under the assumption lim_n→∞nh^d+4_n [lnn]⁻¹ =∞, the application of Lemma 3.1 with Λ =∂²K/∂x_i∂x_j ensures that

n→∞lim sup

x∈R^d

∂²f_n

∂x_i∂x_j(x)−E

∂²f_n

∂x_i∂x_j(x)

= 0 a.s.

Moreover, classical computations give the uniform convergence ofE ∂²f_n/∂x_i∂x_j

to∂²f /∂x_i∂x_j in a neigh- borhood ofθ. Since lim_n→∞θ_n=θ a.s., we thus obtain

n→∞lim H_n=D²f(θ) a.s.

In view of (9), it follows that the convergence rate ofθ_n−θis given by that of−

D²f(θ)₋₁

∇fn(θ); Sections 3.3 and 3.4 are devoted to the study of the weak and almost sure asymptotic behaviour of the variance term of∇fn(θ). The asymptotic behaviour of the bias term is given by the following lemma:

Lemma 3.2.

n→∞lim 1

h^q_n E(∇fn(θ)) =(−1)^q q! ∇



X^d

j=1

µ^q_j∂^qf

∂x^q_j(θ)



.

Proof of Lemma 3.2. Let us set f_i = _∂x^∂f

i (i∈ {1, . . . , d}), andD^jf_i (j ∈ {1, . . . , p}) the j-th differential of f_i. For x= (x₁, . . . , x_d)^t ∈R^d, set x^(j) = (x, . . . , x)∈ R^d_j

; with these notations, the j-linear application D^jf_i(θ) satisfies

D^jf_i(θ)

x^(j)

= X

1≤i1≤. . .≤ik≤d αi1+. . .+αik=j

∂^jf_i

∂x^α_i1¹. . . ∂x^α_i^k

k

(θ)x^α_i1¹. . . x^α_i^k

k. (10)

Fori∈ {1, . . . , d}, we have E

∂f_n

∂x_i(θ)

= 1

h^d+1_n Z

R^d

∂K

∂x_i

θ−x h_n

f(x)dx= 1 h^d_n

Z

R^dK

θ−x h_n

∂f

∂x_i(x)dx= Z

R^dK(y)f_i(θ−h_ny) dy so that

E ∂f_n

∂x_i(θ)

− ∂f

∂x_i(θ) = Z

R^dK(y) [f_i(θ−h_ny)−f_i(θ)] dy.

It follows from (10) and (A5) iii) that

E ∂f_n

∂x_i(θ)

− ∂f

∂x_i(θ) =h^q_n Z

R^dkyk^qK(y)



f_i(θ−h_ny)−f_i(θ)−P_q−1

j=1 (−1)^jh^j_n

j! D^jf_i(θ) y^(j) h^q_nkyk^q



dy. (11)

The bracketed term in the last integral is bounded for all values ofh_n andy, R

R^dkyk^q|K(y)|dy <∞and, for anyy6= 0,

n→∞lim



f_i(θ−h_ny)−f_i(θ)−P_q−1

j=1 (−1)^jh^j_n

j! D^jf_i(θ) y^(j) h^q_nkyk^q



= (−1)^q

q!kyk^qD^qf_i(θ)

y^(q)

.

Thus, we have

n→∞lim 1 h^q_n

E

∂f_n

∂x_i(θ)

− ∂f

∂x_i(θ)

=(−1)^q q!

Z

R^dD^qf_i(θ)

y^(q)

K(y)dy.

In view of (10) and (A5) iii), it comes:

n→∞lim 1 h^q_n

E

∂f_n

∂x_i(θ)

− ∂f

∂x_i(θ)

=(−1)^q q!

Xd j=1

Z

R^d

∂^qf_i

∂x^q_j (θ)y^q_jK(y)dy=(−1)^q q!

Xd j=1

µ^q_j ∂^q+1f

∂x_i∂x^q_j(θ)

=(−1)^q q!

∂

∂x_i



X^d

j=1

µ^q_j∂^qf

∂x^q_j(θ)





so that

n→∞lim 1 h^qn

[E(∇f_n(θ))− ∇f(θ)] =(−1)^q q! ∇



X^d

j=1

µ^q_j∂^qf

∂x^q_j(θ)





which concludes the proof of Lemma 3.2.

3.3. Central limit theorem for the kernel mode estimator

A straightforward application of Lyapounov’s theorem gives the following central limit theorem fulfilled by the variance term of the kernel estimator of the density derivative:

q

nh^d+2_n (∇fn(θ)−E (∇fn(θ)))→ N^D (0, f(θ)G). (12) In view of the previous section, Theorem 2.2 follows.

In order to justify that the limit laws in Parts i) and ii) of Theorem 2.2 are nondegenerate, we now prove Proposition 2.3 (which is straightforward in the cased= 1).

Proof of Proposition 2.3. Letu= (u₁, . . . , u_d)^t∈R^d; we have

u^tGu = Z

R^d

Xd i=1

u_i∂K

∂x_i(x)

!2

dx ≥0. (13)

Assume there exists u6= 0 such that u^tGu= 0. In view of (13), we then haveP_d

i=1u_i∂K/∂x_i≡0. Without loss of generality, assume that, for someh∈ {1, . . . , d},u₁6= 0, . . . , uh6= 0 andu_h+1=. . .=u_d= 0. Then,

u₁∂K

∂x₁ +. . .+u_h∂K

∂x_h ≡0.

The general solution of this first order linear partial differential equation is φ(u₂x₁−u₁x₂, . . . , u_hx₁−u₁x_h, x_h+1, . . . , x_d)

whereφis an arbitrary real-valued differentiable function on R^d−1. We thus haveK=φ◦M whereM: R^d→ R^d−1is a linear application. The rank ofM isd−1 and kerM = span{v}wherev= (u₁, u₂, . . . , u_h,0, . . . ,0)^t. Setz∈R^d−1 andx₀∈M⁻¹(z); we have

K(x₀+λv) =φ(z) ∀λ∈R.

Since lim_λ→∞K(x₀+λv) = 0, it follows thatφ(z) = 0 for allz ∈R^d−1and thus K≡0, which is impossible.

Thus,u^tGu >0 for anyu6= 0, which proves Proposition 2.3.

3.4. Law of the iterated logarithm for the kernel mode estimator

R_n=−

D²f(θ)₋₁

(∇fn(θ)−E (∇fn(θ))) V_n=

s nh^d+2_n 2 ln lnn AR_n. From now on, we set ∆ =−

D²f(θ)₋₁

. Fori∈ {1, . . . , d}, we have

R_n,i= Xd j=1

∆_i,j∂f_n

∂x_j(θ)−E



X^d

j=1

∆_i,j∂f_n

∂x_j(θ)





V_n,i= s

nh^d+2_n 2 ln lnn



X^d

j=1

Xd l=1

A_i,j∆_j,l∂f_n

∂x_l(θ)−E



X^d

j=1

Xd l=1

A_i,j∆_j,l∂f_n

∂x_l(θ)









= 1

p2nh^d_nln lnn Xn k=1







X^d

j=1

Xd l=1

A_i,j∆_j,l∂K

∂x_l



 θ−X_k

h_n

−E







X^d

j=1

Xd l=1

A_i,j∆_j,l∂K

∂x_l



 θ−X_k

h_n





· Theorem 4.1 in Arcones [1] applies and gives the following result:

Lemma 3.3. With probability one, the sequence(V_n)is relatively compact and its limit set is

C=









p f(θ)

Z

R^d



X^d

j=1

Xd l=1

A_i,j∆_j,l∂K

∂x_l(x)



 α(x) dx





i∈{1,... ,d⁰}

: Z

R^dα²(x) dx≤1





·

Note thatCis the image of the closed unit ball ofL²(R^d) by the linear continuous applicationH:L²(R^d)→R^d0 defined by

H(α) =



p f(θ)

Z

R^d



X^d

j=1

Xd l=1

A_i,j∆_j,l∂K

∂x_l(x)



 α(x) dx





i∈{1,... ,d⁰}

.

We now show how Theorems 2.4, 2.5 and 2.6 are deduced from Lemma 3.3.

3.4.1. Law of the iterated logarithm for the full vector

LetAbe the identity matrix; in view of Lemma 3.3, the limit set of the sequence



 s

nh^d+2_n

2 ln lnn ∆ [∇fn(θ)−E (∇fn(θ))]





is, with probability one,

C=









p f(θ)

Z

R^d



X^d

j=1

∆_i,j∂K

∂x_j(x)



 α(x) dx





i∈{1,... ,d}

: Z

R^dα²(x) dx≤1





· Now, let U be the vector subspace ofL²(R^d) spanned by

∂K

∂xi

i∈{1,... ,d}. For anyα ∈L² R^d

, there exists γ= (γ₁, . . . , γ_d)^tandα∈ U^⊥ such that

α(x) = Xd k=1

γ_k∂K

∂x_k(x) +α(x) andC can be rewritten as

C=









p f(θ)

Z

R^d



X^d

j=1

∆_i,j∂K

∂x_j(x)



 α(x) dx





i∈{1,... ,d}

: α∈ U and Z

R^d

α²(x) dx≤1





·

However, for anyα∈ U,α= Xd k=1

γ_k∂K

∂x_k, we have Z

R^dα²(x) dx= Xd k=1

Xd j=1

γ_kγ_j Z

R^d

∂K

∂x_k(x)∂K

∂x_j(x)dx=γ^tGγ