Upper functions for positive random functionals. Application to the empirical processes theory II

HAL Id: hal-01265255

https://hal.archives-ouvertes.fr/hal-01265255

Submitted on 31 Jan 2016

HAL is a multi-disciplinary open access archive for the deposit and dissemination of sci- entific research documents, whether they are pub- lished or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Upper functions for positive random functionals.

Application to the empirical processes theory II

Oleg Lepski

To cite this version:

Oleg Lepski. Upper functions for positive random functionals. Application to the empirical processes

theory II. Mathematical Methods of Statistics, Allerton Press, Springer (link), 2013, 22 (3), pp.193-

212. �10.3103/S1066530713030022�. �hal-01265255�

Upper functions for positive random functionals.

Application to the empirical processes theory II.

Oleg Lepski

Laboratoire d’Analyse, Topologie, Probabilit´es Universit´e Aix-Marseille

39, rue F. Joliot-Curie 13453 Marseille, France e-mail: lepski@cmi.univ-mrs.fr

Abstract: This part of the paper finalizes the research started in Lepski (2013b).

AMS 2000 subject classifications: Primary 60E15; secondary 62G07, 62G08.

Keywords and phrases: upper function, empirical processes, metric entropy.

1. Introduction

Let us remind the notations and basic assumptions used in Lepski (2013b).

Let ^¡ X , X, ν ^¢ be σ-finite space and let (Ω, A, P) be a complete probability space. Let X _i , i ≥ 1, be a the collection of X -valued independent random variables defined on (Ω, A, P) and having the densities f _i with respect to measure ν. Furthermore, P _f , f = (f ₁ , f ₂ , . . .), denotes the probability law of (X ₁ , X ₂ , . . .) and E _f is mathematical expectation with respect to P _f .

Let G : H × X → R be a given mapping, where H is a set. Put ∀n ∈ N ^∗ ξ _h (n) = n ⁻¹

X n i=1

h G ^¡ h, X _i ^¢ − E _f G(h, X _i ) ⁱ , h ∈ H. (1.1)

We will say that ξ _h (n), h ∈ H, is a generalized empirical process. Note that if h : X → R and G(h, x) = h(x), h ∈ H, x ∈ X , then ξ _h (n) is the standard empirical process parameterized by H.

We assume that for some m ≥ 1

H = H ₁ × · · · × H _m , (1.2) where H _j , j = 1, m, be given sets. We will use the following notations. For any given k = 0, m put

H ^k ₁ = H ₁ × · · · × H _k , H ^m _k+1 = H _k+1 × · · · × H _m ,

with the agreement that H ⁰ ₁ = ∅, H ^m _m+1 = ∅. The elements of H ^k ₁ and H ^m _k+1 will be denoted by h ^(k) and h _(k) respectively. Moreover we suppose that for some p ≥ 1

(X , ν) = ^¡ X ₁ × · · · × X _p , ν ₁ × · · · × ν _p ^¢ , (1.3) where (X _l , ν _l ) l = 1, p, are of measurable spaces and ν is the product measure. It will be also assumed that H _m = X ₁ .

Recall also that all consideration in Lepski (2013b) have been done under the following assump- tion. Put for any h ^(k) ∈ H ^k ₁

G _∞ ^¡ h ^{(k) ¢} = sup

h

∈H

sup

x∈X

|G(h, x)|,

1

imsart-generic ver. 2011/12/01 file: upper_function_final2-part3.tex date: February 7, 2013

and let G _∞ : H ^k ₁ → R ₊ be any mapping satisfying

G _∞ ^¡ h ^{(k) ¢} ≤ G _∞ ^¡ h ^{(k) ¢} , ∀h ^(k) ∈ H ^k ₁ . (1.4) Let {H _j (n) ⊂ H _j , n ≥ 1} , j = 1, k, be a sequence of sets and denote H ^k ₁ (n) = H ₁ (n) × · · · H _k (n).

Set for any n ≥ 1

G _n = inf

h

∈H

(n) G _∞ ^¡ h ^{(k) ¢} , G _n = sup

h

∈H

(n)

G _∞ ^¡ h ^{(k) ¢} . For any n ≥ 1, j = 1, k and any h _j ∈ H _j (n) define

G _j,n (h _j ) = sup

h

∈H

(n),...,h

∈H

(n),h

∈H

(n),...,h

∈H

(n)

G _∞ ^¡ h ^{(k) ¢} , G _j,n = inf

h

∈H

(n) G _j,n (h _j ).

Noting that ^{¯ ¯} ln (t ₁ ) −ln (t ₂ ) ^{¯ ¯} is a metric on R ₊ \{0}, we equip H ^k ₁ (n) with the following semi-metric.

For any n ≥ 1 and any ˆ h ^(k) , ¯ h ^(k) ∈ H ^k ₁ (n) set

% ^(k) _n ^³ ˆ h ^(k) , ¯ h ^{(k) ´} = max

j=1,k

¯ ¯

¯ ln ^© G _j,n (ˆ h _j ) ^ª − ln ^© G _j,n (¯ h _j ) ^ª¯¯ ¯ , where ˆ h _j , ¯ h _j , j = 1, k, are the coordinates of ˆ h ^(k) and ¯ h ^(k) respectively.

Assumption 1. (i) 0 < G _n ≤ G _n < ∞ for any n ≥ 1 and for any j = 1, k G _∞ ^¡ h ^{(k) ¢}

G _n ≥ G _j,n (h _j )

G _j,n , ∀h ^(k) = (h ₁ , . . . , h _k ) ∈ H ^k ₁ (n), ∀n ≥ 1;

(ii) There exist functions L _j : R ₊ → R ₊ , D _j : R ₊ → R ₊ , j = 0, k + 1, . . . , m, satisfying L _j non-decreasing and bounded on each bounded interval, D _j ∈ C ^{1 ¡} R ^¢ , D(0) = 0, and such that

° °

° G(h, ·) − G(h, ·)

° °

° ∞ ≤ ⁿ G _∞ ^³ h ^{(k) ´} ∨ G _∞ ^³ h ^{(k) ´o} D ₀ ⁿ % ^(k) _n ^³ h ^(k) , h ^{(k) ´o} +

X m j=k+1

L _j ⁿ G _∞ ^³ h ^{(k) ´} ∨ G _∞ ^³ h ^{(k) ´o} D _j ^³ % _j ^¡ h _j , h ⁰ _j ^¢´ , for any h, h ∈ H ^k ₁ (n) × H ^m _k+1 and n ≥ 1.

At last the following condition has been also exploited in the previous considerations.

Assumption 2. For any m ∈ N ^∗ there exists n[m] ∈ {m, m + 1, . . . , 2m} such that [

n∈{m,m+1,...,2m}

H e ^k ₁ (n) ⊆ H ^k ₁ ^¡ n[m] ^¢ .

2. Partially totally bounded case

We begin this section with the following definition used in the sequel. Let T be a set equipped with a semi-metric d and let n ∈ N ^∗ be fixed.

Definition 1. We say that {T _i ⊂ T, i ∈ I} is n-totally bounded cover of T if

• T = ∪ _i∈I T _i and I is countable;

2

• T _i is totally bounded for any i ∈ I;

• card ^³ {k ∈ I : T _i ∩ T _k 6= ∅} ^´ ≤ n for any i ∈ I.

Let us illustrate the above definition by some examples.

Let T = R ^d , d ≥ 1. Then any countable partition of R ^d consisted of bounded sets forms 1-totally bounded cover of R ^d . Note, however, that the partitions will not be suitable choice for particular problems studied later. We will be mostly interested in n-totally bounded covers satisfying the following separation property: there exists r > 0 such that for all i, k ∈ I satisfying T _i ∩ T _k = ∅

t

∈T inf

,t

∈T

d(t ₁ , t ₂ ) ≥ r. (2.1)

Let us return to R ^d that we equip with the metric generated by the supremum norm. Denote by B _r (t), t ∈ R ^d , r > 0, the closed ball in this metric with the radius r and the center t. For given r > 0 consider the collection ⁿ B

2

(ri), i ∈ Z ^{d o} , where we understand ri as coordinate-wise multiplication.

It is easy to check that this collection is 3 ^d -totally bounded cover of R ^d satisfying (2.1).

We would like to emphasize that n-totally bounded covers satisfying the separation property can be often constructed when T is a homogenous metric space endowed with the Borel measure obeying doubling condition. Some useful results for this construction can be found in the recent paper Coulhon et al. (2011), where such spaces were scrutinized.

We finish the discussion about n-totally bounded covers with the following notation: for any t ∈ T put

T(t) = ^[

i∈I: T

3t

[

k∈I: T

∩T

6=∅

T _k .

2.1. Assumptions and main result

Throughout this section we will assume that the representation (1.3) holds and the elements of X _l , l = 1, p, will be denoted by x _l .

Assumption 3. (i) Let (1.2) and (1.3) hold with X ₁ = H _m and for some n ∈ N ^∗ there exists a collection ⁿ H _m,i , i ∈ I ^o being the n-totally bounded cover of H _m satisfying for some N, R < ∞

E _H

_,%

(ς) ≤ N ^£ log ₂ ^© R/ς ^ª¤ ₊ , ∀i ∈ I, ∀ς > 0.

(ii) For any ς > 0

E _H

_,%

(ς) ≤ N ^£ log ₂ ^© R/ς ^ª¤ ₊ , ∀j = k + 1, m − 1.

Usually one can construct many n-totally bounded covers satisfying Assumption 3 (i). The con- dition below restricts this choice and relates it to properties of the mapping G(·, ·) describing generalized empirical process.

Assumption 4. For any n ≥ 1 and any h = (h ₁ , . . . , h _m ) ∈ H(n) sup

x∈X : x

∈H /

(h

)

|G(h, x)| ≤ n ⁻¹ G _∞ ^¡ h ^{(k) ¢} .

We would like to emphasize that in order to satisfy Assumption 4 in particular examples, the n-totally bounded cover ⁿ H _m,i , i ∈ I ^o should usually possess the separation property. Indeed, one of the typical examples, where Assumption 4 is fulfilled, is the following: there exist γ > 0 such that for G(x, h) = 0 for any x ∈ X , h ∈ H, satisfying ρ _m (x ₁ , h _m ) ≥ γ.

3

Result. For any i = 1, n we denote X _i = ^¡ X _1,i , . . . , X _p,i ^¢ , f _1,i (x ₁ ) =

Z

X

×···×X

f _i (x ₁ , . . . , x _p ) Y p l=2

ν _l ^¡ dx _l ^¢ . and if X = X ₁ (p = 1) then we put X _1,i = X _i and f _1,i = f _i .

Put for any n ≥ 1, v > 0 and any h _m ∈ H _m L _n,v (h _m ) = − ln

µ·

n ⁻¹ X n i=1

Z

H

(h

) f _1,i (x)ν ₁ ^¡ dx ^¢

¸

∨ n ^−v

¶ . Note that obviously 0 ≤ L _n,v (h _m ) ≤ v ln (n), ∀h _m ∈ H _m . Put for any h ∈ H

P e (h) = P ^¡ h ^{(k) ¢} + L _n,v ^¡ h _m ^¢ + 2 ln {1 + |ln (F _n

_,r (h))|};

M f _q (h) = M _q ^¡ h ^{(k) ¢} + L _n,v ^¡ h _m ^¢ + 2 ln {1 + |ln (F _n

_,r (h))|}.

Define for any h ∈ H, r ∈ N, z ≥ 0 and q > 0 V e _r ^(v,z) (n, h) = λ ₁

r

G _∞ ^¡ h ^{(k) ¢³} F _n

_,r (h)n ^{−1 ´¡} P ^{e ¡} h ^¢ + z ^¢ + λ ₂ G _∞ ^¡ h ^{(k) ¢³} n ⁻¹ ln ^β (n) ^´¡ P ^{e ¡} h ^¢ + z ^¢ ; U e _r ^(v,z,q) (n, h) = λ ₁

r

G _∞ ^¡ h ^{(k) ¢³} F _n

_,r (h)n ^{−1 ´¡} M ^f _q (h) + z ^¢ + λ ₂ G _∞ ^¡ h ^{(k) ¢³} n ⁻¹ ln ^β (n) ^´¡ M ^f _q (h) + z ^¢ . Theorem 1. Let Assumptions 1, 3 and 4 hold. If n ₁ 6= n ₂ suppose additionally that Assumption 2 holds. Then for any r ∈ N, v ≥ 1, z ≥ 1 and q ≥ 1

P _f

 

 sup

n∈ _N e sup

h∈ _H(n) e

h¯ ¯ ξ _h (n) ^{¯ ¯} − V ^e _r ^(v,z) (n, h) ⁱ ≥ 0

 

 ≤ n ^{5 n} 4838e ^−z + 4n ₁ ^{2−v o} ; E _f

 

 sup

n∈ N e sup

h∈ H(n) e

h¯ ¯ ξ _h (n) ^{¯ ¯} − U ^e _r ^(v,z,q) (n, h) ⁱ

 



q

+

≤ 2n ⁵ c _q

·q

(n ₁ ) ⁻¹ F _n

G _n ∨ ^³ (n ₁ ) ⁻¹ ln ^β (n ₂ )G _n ^´¸

q

e ^−z +2 ^q+2 n ⁵ (G _n ) ^q n ₁ ^2−v .

Although the assertions of the theorem are true whenever v ≥ 1 the presented results are obvi- ously reasonable only if v > 2. For example (as we will see later) the typical choice of this parameter for the ”moment bound” is v = q + 2.

In spite of the fact that upper functions presented in Theorem 1 are found explicitly their expressions are quite cumbersome. In particular, it is unclear how to compute the function L _n,v (·).

Of course, since L _n,v (h _m ) ≤ v ln (n), ∀h _m ∈ H _m , one can replace it by v ln (n) in the definition of P e (·) and M ^f _q (·), but the corresponding upper functions are not always sufficiently tight.

Our goal now is to simplify the expressions for upper functions given in Theorem 1. Surprisingly, that if n is fixed, i.e. n ₁ = n ₂ , it can be done without any additional assumption.

Set for any v > 0 and h ∈ H

P b _v ^¡ h ^{(k) ¢} = P ^¡ h ^{(k) ¢} + 2v ^{¯ ¯} ¯ ln ^³ 2G _∞ ^¡ h ^{(k) ¢´¯¯} ¯ , M ^c _q,v ^¡ h ^{(k) ¢} = M _q ^¡ h ^{(k) ¢} + 2v ^{¯ ¯} ¯ ln ^³ 2G _∞ ^¡ h ^{(k) ¢´¯¯} ¯ , and let F ^b _n

(h) = max[F _n

(h), n ₂ ⁻¹ ].

4

Corollary 1. Let the assumptions of Theorem 1 hold. If n ₁ 6= n ₂ suppose additionally that X _i,1 , i ≥ 1, are identically distributed.

Then, the results of Theorem 1 remain valid if one replaces V ^e _r ^(v,z) (n, h) and U ^e _r ^(v,z,q) (n, h) by V b ^(v,z) (n, h) = λ ₁

r

G _∞ ^¡ h ^{(k) ¢³ b} F _n

(h)n ^{−1 ´³} P ^b _v ^¡ h ^{(k) ¢} + 2(v + 1) ^{¯ ¯} ln ^{© b} F _n

(h) ^{ª¯ ¯} + z ^´ +λ ₂ G _∞ ^¡ h ^{(k) ¢³} n ⁻¹ ln ^β (n) ^´³ P ^b _v ^¡ h ^{(k) ¢} + 2(v + 1) ^{¯ ¯} ln ^{© b} F _n

(h) ^{ª¯ ¯} + z ^´ ; U b ^(v,z,q) (n, h) = λ ₁

r

G _∞ ^¡ h ^{(k) ¢³ b} F _n

(h)n ^{−1 ´³} M ^c _q,v ^¡ h ^{(k) ¢} + 2(v + 1) ^{¯ ¯} ln ^{© b} F _n

(h) ^{ª¯ ¯} + z ^´ +λ ₂ G _∞ ^¡ h ^{(k) ¢³} n ⁻¹ ln ^β (n) ^´³ M ^c _q,v ^¡ h ^{(k) ¢} + 2(v + 1) ^{¯ ¯} ln ^{© b} F _n

(h) ^{ª¯ ¯} + z ^´ .

We would like to emphasize that we do not require that X _i , i ≥ 1, would be identically dis- tributed. In particular, coming back to the generalized empirical process considered in Example 2, Section 1.1 in Lepski (2013b), where X _i = (Y _i , ε _i ), the design points Y _i , i ≥ 1, are often supposed to be uniformly distributed on some bounded domain of R ^d . As to the noise variables ε _i , i ≥ 1, the restriction that they are identically distributed cannot be justified in general.

2.2. Law of logarithm

Our goal here is to use the first assertion of Corollary 1 in order to establish the result referred later to the law of logarithm. Namely we show that for some Υ > 0

lim sup

n→∞ sup

h

∈H

(n,a)

√ n η _h

(n) r

G _∞ ^¡ h ^{(k) ¢h} ln ^© G _∞ ^¡ h ^{(k) ¢ª} ∨ ln ln (n) ⁱ

≤ Υ P _f − a.s. (2.2)

As previously we will first provide with the non-asymptotical version of (2.2).

As before we suppose that

c ≤ G _n ≤ G _n ≤ cn ^b , ∀n ≥ 1; (2.3)

sup

n≥1 sup

h∈ _H(n) e sup

i≥1 E _f ^{¯ ¯} G(h, X _i ) ^{¯ ¯} =: F < ∞ (2.4) and impose additionally the following condition. For some a > 0

L ^(k) (z) ≤ a ln (z), ∀z ≥ 2. (2.5)

Theorem 2. Let Assumptions 1, 2, 3 and 4 be fulfilled. Suppose also that (2.3), (2.4) and (2.5) hold and assume that X _i,1 , i ≥ 1, are identically distributed.

Then there exits Υ > 0 such that for any j ≥ 3 and any a > 4

P _f

 

 

 

 sup

n≥j

sup

h

∈H

(n,a)

√ n η _h

(n) r

G _∞ ^¡ h ^{(k) ¢h} ln ^© G _∞ ^¡ h ^{(k) ¢ª} ∨ ln ln (n) ⁱ

≥ Υ

 

 

 



≤ 4840n ⁵ ln (j) .

Some remarks are in order. The explicit expression of the constant Υ is available and the gen- eralization. Also, (2.2) is an obvious consequence of Theorem 2. At last, we note that in view of

5

(2.3) the factor ^h ln ⁿ G _∞ ^¡ h ^{(k) ¢o} ∨ ln ln (n) ⁱ can be replaced by ln(n) which is, up to a constant, its upper estimate. The corresponding result is, of course, rougher than one presented in the theorem, but its derivation does not require X _i,1 , i ≥ 1, to be identically distributed. This result is deduced directly from Theorem 1. Its proof is almost the same as the proof of Theorem 2 and based on the trivial bound L _n,v (h _m ) ≤ v ln (n), ∀h _m ∈ H _m .

3. Application to localized processes

Let ^¡ X _l , µ _l , ρ _l ^¢ , l = 1, d + 1, d ∈ N, be the collection of measurable metric spaces. Throughout this section we will suppose that (1.3) holds with p = 2,

X = X ₁ × X ₂ , ^¡ X ₁ , ν ₁ ^¢ = ^¡ X ₁ × · · · × X _d , µ ₁ × · · · × µ _d ^¢ =: ^¡ X ^d ₁ , µ ^{(d) ¢} , ^¡ X ₂ , ν ₂ ^¢ = ^¡ X _d+1 , µ _d+1 ^¢ , x _j denotes the element of X _j , j = 1, d + 1, and x ^(d) will denotes the element of X ^d ₁ . We equip the space X ^d ₁ with the semi-metric ρ ^(d) = max _l=1,d ρ _l .

Problem formulation This section is devoted to the application of Theorem 1 [Lepski (2013b)]

and Theorem 1 in the following case:

• H ^d ₁ := H ₁ × · · · × H _d = (0, 1] × · · · × (0, 1] = (0, 1] ^d , (i.e. k = d);

• H ^d+2 _d+1 = H _d+1 × H _d+2 := Z × X ¯ ^d ₁ , i.e. m = d + 2, where ¯ X ^d ₁ := ¯ X ₁ × · · · × X ¯ _d be a given subset of X ^d ₁ and (Z, d) is a given metric space.

• The function G(·, ·) obeys some structural assumption described below and for any h :=

¡ r, z, x ¯ ^{(d) ¢} ∈ (0, 1] ^d × Z × X ¯ ^d ₁ the function G(h, ·) ”decrease rapidly ” outside of the set ⁿ x ₁ ∈ X ₁ : ρ ₁ ^¡ x ₁ , x ¯ ₁ ^¢ ≤ r ₁ ^o × · · · × ⁿ x _d ∈ X _d : ρ _d ^¡ x _d , x ¯ _d ^¢ ≤ r _d ^o × X _d+1 .

Let K : R ^d → R be a given function, ^¡ γ ₁ , . . . , γ _d ^¢ ∈ R ^d ₊ be given vector and set for any r ∈ (0, 1] ^d K _r (·) = V _r ⁻¹ K (·/r ₁ , . . . , ·/r _d ) , V _r =

Y d l=1

r ^γ _l

.

where, as previously, for u, v ∈ R ^d the notation u/v denotes the coordinate-wise division. Let G(h, x) = g ^¡ z, x ^¢ K _r ^³ ~ρ ^¡ x ^(d) , x ¯ ^{(d) ¢´} , h = ^³ r, z, x ¯ ^{(d) ´} ∈ (0, 1] ^d × Z × X ¯ ^d ₁ =: H, (3.1) where g : Z × X → R is a given function those properties will be described later and

~ρ ^¡ x ^(d) , x ¯ ^{(d) ¢} = ^¡ ρ ₁ ^¡ x ₁ , x ¯ ₁ ^¢ , . . . , ρ _d ^¡ x _d , x ¯ _d ^¢¢ . The corresponding generalized empirical process is given by

ξ _h (n) = n ⁻¹ X n i=1

·

g ^¡ z, X _i ^¢ K _r ^³ ~ρ ^³ [X _i ] ^(d) , x ¯ ^{(d) ´´} − E _f ⁿ g ^¡ z, X _i ^¢ K _r ^³ ~ρ ^³ [X _i ] ^(d) , x ¯ ^{(d) ´´o ¸} . We will seek upper functions for the random field ζ _r ^¡ n, x ¯ ^{(d) ¢} := sup

z∈Z

¯ ¯ ξ _{r,z,¯ x}

(n) ^{¯ ¯} in two cases: ¯ X ^d ₁ = X ^d ₁ and ¯ X ^d ₁ = ⁿ x ¯ ^{(d) o} for a fixed ¯ x ^(d) ∈ X ^d ₁ .

6

To realize this program we will apply Theorem 1 [Lepski (2013b)] and Theorem 1 to ξ _h (n), h =

³

r, z, x ¯ ^{(d) ´} . It is worth mentioning that corresponding upper functions can be used for constructing of estimation procedures in different areas of mathematical statistics: M-estimation with locally polynomial fitting (non-parametric regression), kernel density estimation and many others.

Moreover, we apply Theorem 2 [Lepski (2013b)] for establishing a non-asymptotical version of the law of iterated logarithm for ζ _r ^¡ x ¯ ^(d) , n ^¢ in the case where ¯ X ^d ₁ = ⁿ x ¯ ^{(d) o} . We also apply Theorem 2 for deriving a non-asymptotical version of the law of logarithm for kζ _r (n)k _∞ := sup _{x ¯}

∈X

¯ ¯

¯ ζ _r ^¡ x ¯ ^(d) , n ^¢¯¯ ¯ . Our study here generalizes in several directions the existing results Einmahl and Mason (2000), Gin´e and Guillou (2002), Einmahl and Mason (2005), Dony et al. (2006), Dony et Einmahl (2009).

3.1. Assumptions and notations

Assumption 5. (i) kKk _∞ < ∞ and for some L ₁ > 0

|K(t) − K(s)| ≤ L ₁ |t − s|

1 + |t| ∧ |s| , ∀t, s ∈ R ^d , where | · | denotes supremum norm on R ^d .

(ii) kgk _∞ := sup

z∈Z, x∈X

¯ ¯ g ^¡ z, x ^{¢¯ ¯} < ∞, and for some α ∈ (0, 1], L _α > 0,

sup

x∈X

¯ ¯ g ^¡ z, x ^¢ − g ^¡ z ⁰ , x ^{¢¯ ¯} ≤ L _α ^£ d(z, z ⁰ ) ^{¤ α} , ∀z, z ⁰ ∈ Z ;

The conditions (i) and (ii) are quite standards. In particular (i) holds if K is compactly supported and lipschitz continuous. If g ^¡ z, · ^¢ = ¯ g(·), for any z ∈ Z , then (ii) is verified for any bounded ¯ g.

Let 0 < r ^(min) _l (n) ≤ r ^(max) _l (n) ≤ 1, l = 1, d, n ≥ 1, be given decreasing sequences and let H(n) = R(n) × Z × X ¯ ₁ ^d , R(n) =

Y d l=1

£ r ^(min) _l (2n), r ^(max) _l (n) ^¤ ;

H(n) = e R(n) ^e × Z × X ¯ ₁ ^d , R(n) = ^e Y d l=1

£ r ^(min) _l (n), r _l ^(max) (n) ^¤ .

We note that H(n) ^e ⊆ H(n) for any n ≥ 1 since r ^(min) _l (·), r ^(max) _l (·), l = 1, d, are decreasing, and obviously H(n) ^e ⊆ H(m) for any n ∈ {m, . . . , 2m} and any m ≥ 1.

Remark 1. Assumption 2 is fulfilled with n[m] = m.

Lemma 1. Suppose that Assumption 5 is fulfilled and let X ¯ ^d ₁ ⊆ X ^d ₁ be an arbitrary subset. Then, for arbitrary sequences 0 < r _l ^(min) (n) ≤ r _l ^(max) (n) ≤ 1, l = 1, d, n ≥ 1, Assumption 1 holds with

% ^(d) _n ^¡ r, r ^{0 ¢} = max

l=1,d

¯ ¯ γ _l ln ^¡ r _l /r _l ^{0 ¢¯ ¯} , % _d+1 = ^£ d ^{¤ α} , % _d+2 = max

l=1,d

ρ _l ; D ₀ (z) = exp {dz} − 1 + (L ₁ /kKk _∞ ) ^³ exp ⁿ γ ⁻¹ z ^o − 1 ^´ , γ = min

l=1,l γ _l ;

D _d+1 (z) = ^¡ L _α /kgk _∞ ^¢ z, D _d+2 (z) = L ₁ ^¡ kgk _∞ kKk ² _∞ ^{¢ −1} z, L _d+1 (z) = z, L _d+2 (z) = z ² . Additionally, if X ¯ ^d ₁ consists of a single point x ¯ ^(d) ∈ X ^d ₁ then L _d+2 ≡ 0.

7

The proof of the lemma is postponed to Appendix. We remark that % _d+1 is a semi-metric, since α ∈ (0, 1], and the semi-metric % ^(d) n is independent on n. In view of latter remark all quantities involved in Assumption 1 are independent on the choice of r _l ^(min) (·), r ^(max) _l (·), l = 1, d,. We want to emphasize nevertheless that the assertion of the lemma is true for an arbitrary but a priory chosen r _j ^(min) (·), r ^(max) _l (·), l = 1, d.

Thus, Lemma 1 guarantees the verification of Assumption 1, that makes possible the application of Theorem 1 [Lepski (2013b)] and Theorem 1. Hence, we have to match the notations of these theorems to the notations used in the present section.

Since k = d and H ^k ₁ = (0, 1] ^d we have h ^(k) = r and, therefore, in view of Assumption 5 G _∞ (r) := sup

(z,¯ x

)∈Z× X ¯

sup

x∈X

¯ ¯

¯ G ^³n r, z, x ¯ ^{(d) o} , x ^{´¯ ¯} ¯ ≤ V _r ⁻¹ kgk _∞ kKk _∞ =: G _∞ (r).

G _n := inf

r∈R(n) G _∞ (r) = V _r ⁻¹

_(n) kgk _∞ kKk _∞ , ∀n ≥ 1.

We remark that the function G _∞ (·) is independent of the choice of ¯ X ^d ₁ . Define f _i ^{(d) ¡} x ^{(d) ¢} =

Z

X

f _i (x)µ _d+1 ^¡ dx _d+1 ^¢ , i ≥ 1, and let 3 ≤ n ₁ ≤ n ₂ ≤ 2n ₁ be fixed. Set for any ^³ r, x ¯ ^{(d) ´} ∈ (0, 1] ^d × X ¯ ^d ₁

F _n

^³ r, x ¯ ^{(d) ´} =

 

 

 



kgk _∞ (n ₂ ) ^{−1 P n} _i=1

^R _X

¯ ¯

¯ K _r ^³ ~ρ ^¡ x ^(d) , x ¯ ^{(d) ¢´¯¯} ¯ f _i ^{(d) ¡} x ^{(d) ¢} µ ^(d) (dx ^(d) ), n ₁ = n ₂ ; kgk _∞ sup

i=1,n

Z

X

¯ ¯

¯ K _r ^³ ~ρ ^¡ x ^(d) , x ¯ ^{(d) ¢´¯¯} ¯ f _i ^{(d) ¡} x ^{(d) ¢} µ ^(d) (dx ^(d) ), n ₁ 6= n ₂ , and note that in view of Assumption 5 (ii)

F _n

(h) ≤ F _n

^³ r, x ¯ ^{(d) ´} , ∀h ∈ (0, 1] ^d × Z × X ¯ ^d ₁ .

We remark that the function F _n

(·, ·) is independent of the choice of Z. Put also F _n

:= sup

n∈ N e

¡ sup

r,¯ x

¢

∈R(n)× X ¯

F _n

^¡ r, x ¯ ^{d ¢} < ∞,

where, remind, N ^e = {n ₁ , . . . , n ₂ }. Finally for any r ∈ N set F _n

_,r (·, ·) = max [F _n

(·, ·), e ^−r ].

3.2. Pointwise results

Here we will consider the case, where ¯ X ^d ₁ = ⁿ x ¯ ^{(d) o} and ¯ x ^(d) is a fixed element of X ^d ₁ . Note that in view of Lemma 1 L _d+1 (z) = z and L _d+2 ≡ 0 that implies L ^(k) ≡ 0.

We are going to apply Theorem 1 from [Lepski (2013b)] and, therefore, we will need the following assumption from Lepski (2013b).

Assumption 6. Suppose that (1.2) holds and there exist N, R < ∞ such that for any ς > 0 and any j = k + 1, m

E _H

_,%

(ς ) ≤ N ^£ log ₂ ^© R/ς ^ª¤ ₊ ,

where, as previously, E _H

_,%

denotes the entropy of H _j measured in % _j .

8

We will suppose that Assumption 6 holds with k = d, m = d + 1 and (H _d+1 , % _d+1 ) = (Z, [d] ^α ).

It is equivalent obviously to assume that Assumption 6 holds with (H _d+1 , % _d+1 ) = (Z, d) and with the constants ˜ N = αN and ˜ R = R ^1/α .

Let β and C _N,R,m,k be the constants defined in Theorem 1 [Lepski (2013b)]. Set for any r ∈ (0, 1] ^d and q > 0

P (r) = (36dδ _∗ ⁻² + 6) ln Ã

1 + X d l=1

γ _l ln

( 2r _l ^(max) (n) r _l

)!

+ 18C _N,R,d+1,d ;

M _q (r) = ^¡ 72dδ ⁻² _∗ + 2.5q + 1.5 ^¢ X d l=1

γ _l ln

à 2r _l ^(max) (n) r _l

!

+ 36C _N,R,d+1,d . and define for r ∈ N and u > 0

V _r ^{(u) ¡} n, r, x ¯ ^{d ¢} = λ ₁ ^rh F _n

_,r ^¡ r, x ¯ ^{d ¢} (nV _r ) ^{−1 ih} P (r) + 2 ln ^© 1 + ^{¯ ¯} ln ^© F _n

_,r ^¡ r, x ¯ ^{d ¢ª¯ ¯ ª} + u ⁱ +λ ₂ ^h (nV _r ) ⁻¹ ln ^β (n) ^ih P (r) + 2 ln ⁿ 1 + ^{¯ ¯} ¯ ln ⁿ F _n

_,r ^¡ r, x ¯ ^{d ¢o¯¯} ¯

o + u ⁱ ; U _r ^{(u,q) ¡} n, r, x ¯ ^{d ¢} = λ ₁ ^rh F _n

_,r ^¡ r, x ¯ ^{d ¢} (nV _r ) ^{−1 ih} M _q (r) + 2 ln ^© 1 + ^{¯ ¯} ln ^© F _n

_,r ^¡ r, x ¯ ^{d ¢ª¯ ¯ ª} + u ⁱ

+λ ₂ ^h (nV _r ) ⁻¹ ln ^β (n) ^ih M _q (r) + 2 ln ⁿ 1 + ^{¯ ¯} ¯ ln ⁿ F _n

_,r ^¡ r, x ¯ ^{d ¢o¯¯} ¯

o + u ⁱ ,

where λ ₁ = ^p kgk _∞ kKk _∞ λ ₁ , λ ₂ = kgk _∞ kKk _∞ λ ₂ and λ ₁ , λ ₂ are defined in Theorem 1 [Lepski (2013b)].

The result below is the direct consequence of Theorem 1 [Lepski (2013b)] and Lemma 1. We remark that defined above quantities are functions of r and n since ¯ x ^d is fixed. Since they do not depend on the variable z, these quantities will be automatically upper functions for

ζ _r ^¡ n, x ¯ ^{(d) ¢} := sup

z∈Z

¯ ¯

¯ ξ _{r,z,¯ x}

¡ x ¯ ^{(d) ¢¯¯} ¯ .

Theorem 3. Let Assumption 5 be fulfilled and suppose that Assumption 6 holds with k = d, m = d + 1 and (H _d+1 , % _d+1 ) = (Z, [d] ^α ).

Then for any given decreasing sequences 0 < r ^(min) _l (n) ≤ r ^(max) _l (n) ≤ 1, l = 1, d, n ≥ 1, any

¯

x ^d ∈ X ¯ ^d ₁ any r ∈ N, b > 1 u ≥ 1 and q ≥ 1 P _f

 

 sup

n∈ N e sup

r∈ R(n) e h

ζ _r ^¡ n, x ¯ ^{(d) ¢} − V _r ^{(u) ¡} n, r, x ¯ ^{d ¢i} ≥ 0

 

 ≤ 2419 e ^−u ; E _f

 

 sup

n∈ N e sup

r∈ R(n) e h

ζ _r ^¡ n, x ¯ ^{(d) ¢} − U _r ^{(u,q) ¡} n, r, x ¯ ^{d ¢i}

 



q

+

≤ c ⁰ _q

"s

F _n

n ₁ V _r

(n

)

∨

à ln ^β (n ₂ ) V _r

(n

) n ₁

!# _q e ^−u ,

where c ⁰ _q = 2 ^(7q/2)+5 3 ^q+4 Γ(q + 1) ^³ C _D,b max ^hp kgk _∞ kKk _∞ , kgk _∞ kKk _∞ ^{i´ q} .

The explicit expression for C _D,b can be also found in Theorem 1 [Lepski (2013b)]. In the case considered here it is completely determined by (γ ₁ , . . . γ _d ), L ₁ , L _α and b.

As well as the assertions of Theorem 1 [Lepski (2013b)] the latter theorem is proved without any assumption imposed on the densities f _i , i = 1, n. The choice of r _l ^(min) (n), r ^(max) _l (n), l = 1, d, n ≥ 1,

9

is also assumption free. Additionally, Assumption 6 can be replaced by weaker condition, see [Lepski (2013b)].

Note also that if g ^¡ z, · ^¢ = ¯ g(·), for any z ∈ Z, then Assumption 6 is not needed anymore and, moreover, Assumption 5 (ii) is verified for an arbitrary bounded ¯ g. Hence, in this case the assertions of Theorem 3 are established under very mild Assumption 5 (i) imposed on the function K.

Remark 2. We note that the discussed in Introduction so-called price to pay for uniformity dis- appears if r = r ^(max) . Indeed, P ^³ r ^{(max) ´} and M _q ^³ r ^{(max) ´} are absolute constants. This property is crucial, in particular, for constructing statistical procedures used in the estimation of functions possessing inhomogeneous smoothness, see Lepski et al. (1997), Kerkyacharian et al. (2001).

Some additional assumptions and their consequences To apply Theorem 3 to specific problems one needs to find an efficient upper bound for the quantity F _n

(·, ·). Below we provide with sufficient condition allowing to solve this problem under general consideration and we will not be tending here to the maximal generality. We impose some additional restrictions on the densities f _i , i = 1, n, and on the measures µ _l of ρ _l -balls in the spaces X _l , l = 1, d. Moreover, we should precise the behavior of the function K at infinity. Then, we will use these assumptions for establishing of the law of iterated logarithm.

Introduce the following notations. For any t ∈ R ^d ₊ define K(t) = sup ˇ

|u|/ ∈Π

|K(u)|, Π _t = [0, t ₁ ] × · · · × [0, t _d ].

For any l = 1, d, x _l ∈ X _l and r > 0 set B _l ^¡ r, x _l ^¢ = ^© y ∈ X _l : ρ _l ^¡ y, x _l ^¢ ≤ r ^ª . Assumption 7. There exists L ₂ > 0 such that

sup

t∈R

·³ Y ^d

l=1

t ^1+γ _l

^´ K(t) ˇ

¸

≤ L ₂ ; (3.2)

For any l = 1, d and any x _l ∈ X _l one has X _l = ∪ _r>0 ^¡ B _l ^¡ r, x _l ^¢¢ and there exist L ^(l) > 0

µ _l ^³ B _l ^¡ r, x _l ^¢´ ≤ L ^(l) r ^γ

, ∀r > 0; (3.3) Moreover,

sup

i≥1

sup

x

∈X

f _i ^{(d) ¡} x ^{(d) ¢} =: f _∞ < ∞. (3.4) The condition (3.2) is obviously fulfilled if K is compactly supported on [0, 1] ^d . It is also satisfied in the case of Gaussian or Laplace kernel.

The condition (3.3) can be easily checked if X _l , l = 1, d are doubling metric spaces. In particular, if X _l = R and µ _l , l = 1, d, are the Lebesgue measures than (3.3) holds with L ^(l) = 1, γ _l = 1, l = 1, d.

If X _l = R ^d

, l = 1, d, then (3.3) holds with γ _l = d _l and the constants L ^(l) depending on the choice of the distances ρ _l .

As to condition (3.4) we remark that the boundedness of the entire density f _i is not required.

For example, under independence structure, i.e. f _i (x) = f _i ^{(d) ¡} x ^{(d) ¢} p _i ^¡ x _d+1 ^¢ , the densities p _i may be unbounded.

10

Lemma 2. The following bound holds under Assumption 7:

sup

n

≥1 sup

r∈(0,1]

sup

¯ x

∈X

F _n

^³ r, x ¯ ^{(d) ´} ≤ 2 ^d f _∞ kgk _∞ L ₂ Y d l=1

2 ^γ

L ^(l) .

The proof of lemma is postponed to Appendix. Our goal now is to deduce the law of iterated logarithm for ζ _r ^¡ n, x ¯ ^{(d) ¢} from Theorem 2, Lepski (2013b). Set for any n ∈ N ^∗ and a > 0

R _a (n) = ⁿ r ∈ (0, 1] ^d : V _r ≥ n ⁻¹ (ln n) ^{a o} . and choose h ^(max) = (1, . . . , 1) and h ^(min) = (1/n, . . . , 1/n).

Remark 3. 1 ⁰ . Note that R _a (n) ⊂ [n ⁻¹ , 1] ^d =: R(n) ^e for any n ≥ 3 and any a > 0 and, therefore, the assertion of Lemma 1 holds.

2 ⁰ . We have G _n = kK k _∞ kgk _∞ , G _n = kKk _∞ kgk _∞ n ^−d for any n ≥ 1 and, therefore, (2.3) is verified with c = kKk _∞ kgk _∞ and b = d.

3 ⁰ . Lemma 2 implies that the condition (2.4) holds with F ≤ 2 ^d f _∞ kgk _∞ L ₂ ^{Q d} _l=1 2 ^γ

L ^(l) .

4 ⁰ . In view of Lemma 1 L _d+1 (z) = z and L _d+2 ≡ 0, that implies L ^(k) ≡ 0. Hence, the condition (1.13) in Lepski (2013b) is fulfilled for any a > 0.

Thus, all assumptions of Theorem 2, Lepski (2013b), are checked and we come to the following statement.

Theorem 4. Let Assumptions 5 and 7 be fulfilled and suppose that Assumption 6 holds with k = d, m = d + 1 and (H _d+1 , % _d+1 ) = (Z, [d] ^α ). Then there exists Υ > 0 such that for any x ¯ ^d ∈ X ¯ ^d ₁ and any a > 2

P _f

 

 sup

n≥j sup

r: n

(ln n)

≤V

≤1

" √

nV _r ζ _r ^¡ n, x ¯ ^{(d) ¢} q

ln ^¡ 1 + ln (n) ^¢

#

≥ Υ

 

 ≤ 2419 ln(j) .

Remark 4. The inspection of the proof of Theorem 2, Lepski (2013b), together with Lemma 2 allows us to assert that the statement of Theorem 4 is uniform over the set of bounded densities.

More precisely, for any f > 0 there exists Υ(f) such that sup

f∈F

P _f

 

 sup

n≥j

sup

r: n

(ln n)

≤V

≤1

" √

nV _r ζ _r ^¡ n, x ¯ ^{(d) ¢} q

ln ^¡ 1 + ln (n) ^¢

#

≥ Υ(f)

 

 ≤ 2419

ln(j) , (3.5) where F _f = ^© (f _i , i ≥ 1) : f _∞ ≤ f ^ª . As before the explicit expression of Υ(·) is available.

The following consequence of Theorem 4 is straightforward.

lim sup

n→∞ sup

r: n

(lnn)

≤V

≤1

" √

nV _r ζ _r ^¡ n, x ¯ ^{(d) ¢} q

ln ^¡ 1 + ln (n) ^¢

#

≤ Υ P _f − a.s. (3.6)

Theorem 4 generalizes the existing results, see for example Dony et Einmahl (2009), in the following directions.

11

1. Structural assumption. The structural condition (3.1) is imposed in cited papers but with additional restriction: either g(z, x) ≡ const (”density case”) or g(z, x) = ¯ g(x) (”regression case”). It excludes, for instance, the problems appearing in robust estimation. We note that Assumption 5 (ii) is fulfilled here if ¯ g is bounded function and Assumption 6 is not needed anymore, since ¯ g is independent of z.

2. Anisotropy. All known to the author results treat the case where X _l = R, l = 1, d, and R(n) = ⁿ (r ₁ , . . . , r _d ) ∈ (0, 1] ^d : r _l = r, ∀l = 1, d, r ∈ ^h r ^(min) (n), r ^(max) (n) ^io (isotropic case).

We remark that (3.3) is automatically fulfilled with γ _l = 1, L ^(l) = 1, l = 1, d, and V _r = r ^d . Note also that we consider independent but not necessarily identically distributed random variables.

This is important, in particular, for various estimation problems arising in nonparametric regression model.

3. Kernel. We do not suppose that the function K is compactly supported. For instance, one can use the gaussian or laplace kernel. It allows, for instaince, to consider the problems where X ₁ ^d is not linear space. In particular, it can be some manifold satisfying doubling condition.

4. Non-asymptotic nature. The existing results are presented as in (3.6). Note, however, that the random field ζ _r ^¡ n, x ¯ ^{d ¢} appears in various areas of nonparametric estimation (density estimation, regression). As the consequence a.s. convergence has no much sense since there is no a unique probability measure (see, also Remark 4).

3.3. Sup-norm results

Here we consider ¯ X ^d ₁ = X ^d ₁ . We assume that there exists {X _i , i ∈ I} which is n-totally bounded cover of ^³ X ^d ₁ , ρ ^{(d) ´} satisfying Assumption 3 (i) and possessing the separation property.

Assumption 8. There exists t > 0 such that for any i, k ∈ I satisfying X _i ∩ X _k = ∅ inf

x

∈X

inf

y

∈X

ρ ^{(d) ³} x ^(d) , y ^{(d) ´} > t.

Also we suppose that Assumption 3 (ii) holds with k = d, m = d+1 and (H _d+1 , % _d+1 ) = (Z, [d] ^α ).

We remark that in the considered case this assumption coincides with Assumption 6.

Let, as previously, 0 < r ^(min) _l (n) ≤ r ^(max) _l (n) ≤ 1, l = 1, d, n ≥ 1, be given decreasing sequences, H(n) = R(n) × Z × X ₁ ^d , R(n) =

Y d l=1

£ r ^(min) _l (2n), r ^(max) _l (n) ^¤ ;

H(n) = e R(n) ^e × Z × X ₁ ^d , R(n) = ^e Y d l=1

£ r ^(min) _l (n), r _l ^(max) (n) ^¤ .

Our last condition relates the choice of the vector r ^(max) (n), n ≥ 1 and the kernel K with the parameter t appearing in Assumption 8. Let us assume that for any n ≥ 1

sup

r∈R(n)

sup

|u|/ ∈(0,t]

|K(u/r)| ≤ kKk _∞ n ⁻¹ . (3.7) Note that (3.7) holds if K is compactly supported on [−t, t] ^d and r ^(max) (n) ∈ (0, t) ^d for any n ≥ 1.

Lemma 3. Assumption 8 and (3.7) imply Assumption 4.

12