Sequential Model Selection Method for Nonparametric Autoregression

HAL Id: hal-02334926

https://hal.archives-ouvertes.fr/hal-02334926

Submitted on 27 Oct 2019

HAL is a multi-disciplinary open access archive for the deposit and dissemination of sci- entific research documents, whether they are pub- lished or not. The documents may come from teaching and research institutions in France or

L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires

Sequential Model Selection Method for Nonparametric Autoregression

Ouerdia Arkoun, Jean-Yves Brua, Serguei Pergamenchtchikov

To cite this version:

Ouerdia Arkoun, Jean-Yves Brua, Serguei Pergamenchtchikov. Sequential Model Selection Method for Nonparametric Autoregression. Sequential Analysis, Taylor & Francis, 2019. �hal-02334926�

Sequential Model Selection Method for Nonparametric Autoregression

Ouerdia Arkoun

Sup’Biotech, Laboratoire BIRL, 66 Rue Guy Moquet, 94800 Villejuif, and Normandie Universit´e, Universit´e de Rouen,

Laboratoire de Math´ematiques Rapha¨el Salem, CNRS, UMR 6085, Avenue de l’universit´e, BP 12, 76801 Saint-Etienne du Rouvray France

Jean-Yves Brua

Normandie Universit´e, Universit´e de Rouen,

Laboratoire de Math´ematiques Rapha¨el Salem, CNRS, UMR 6085, Avenue de l’universit´e, BP 12, 76801 Saint-Etienne du Rouvray France

Serguei Pergamenchtchikov Normandie Universit´e, Universit´e de Rouen,

Laboratoire de Math´ematiques Rapha¨el Salem, CNRS, UMR 6085, Avenue de l’universit´e, BP 12, 76801 Saint-Etienne du Rouvray France and

International Laboratory of Statistics of Stochastic Processes and Quantitative Finance, National Research Tomsk State University, Russian

Federation

Abstract: In this paper for the first time the nonparametric autoregression es- timation problem for the quadratic risks is considered. To this end we develop a new adaptive sequential model selection method based on the efficient sequential kernel estimators proposed by Arkoun and Pergamenshchikov (2016). Moreover, This work was supported by the RSF grant 17-11-01049 (National Re- search Tomsk State University).

Address correspondence to O. Arkoun, Laboratoire de Math´ematiques Rapha¨el Salem, CNRS, UMR 6085, Avenue de l’universit´e, BP 12, 76801 Saint-Etienne du Rouvray Cedex France,

E-mail: Ouerdia.Arkoun@supbiotech.fr

we develop a new analytical tool for general regression models to obtain the non asymptotic sharp oracle inequalities for both usual quadratic and robust quadratic risks. Then, we show that the constructed sequential model selection procedure is optimal in the sense of oracle inequalities.

Keywords: Non-parametric estimation; Non parametric autoregresion; Non-asymptotic estimation; Robust risk; Model selection; Sharp oracle inequalities.

Subject Classifications: primary 62G08, secondary 62G05

1 Introduction

One of the standard linear models in general theory of time series is the autore- gressive model (see, for example, Anderson (1994) and the references therein).

Natural extensions for such models are nonparametric autoregressive models which are defined by

y_k=S(x_k)y_k−1+ξ_k and x_k =a+k(b−a)

n , (1.1)

where S(·)∈ L₂[a, b] is unknown function, a < b are fixed known constants, 1≤ k≤n, the initial valuey₀ is a constant and the noise (ξ_k)_k≥1 is i.i.d. sequence of unobservable random variables withEξ₁= 0 andEξ²₁= 1.

The problem is to estimate the functionS(·) on the basis of the observations (y_k)_1≤k≤n under the condition that the noise distribution is unknown.

It should be noted that the varying coefficient principle is well known in the regression analysis. It permits the use of more complex forms for regression coef- ficients and, therefore, the models constructed via this method are more adequate for applications (see, for example,Fan and Zhang (2008),Luo et al. (2009)). In this paper we consider the varying coefficient autoregressive models (1.1). There is a number of papers which consider these models such as Dahlhaus (1996a), Dahlhaus (1996b) and Belitser (2000). In all these papers, the authors propose some asymptotic (asn → ∞) methods for different identification studies without considering optimal estimation issues. To our knowledge, for the first time, the minimax estimation problem for the model (1.1) has been treated inArkoun and Pergamenchtchikov (2008) and Moulines et al. (2005) in the nonadaptive case, i.e. for the known regularity of the function S. Then, in Arkoun (2011) it is proposed to use the sequential analysis method for the adaptive pointwise estima- tion problem in the case where the unknown H¨older regularity is less than one, i.e when the function S is not differentiable. Also it should be noted (see, Arkoun (2011)) that for the model (1.1), the adaptive pointwise estimation is possible only in the sequential analysis framework. That is why we study sequential estimation methods for the smooth functionS. In this paper we consider the quadratic risk

defined as

R_p(Sb_n, S) = E_p,SkSb_n−Sk², kSk²= Z b

a

S²(x)dx , (1.2) where Sb_n is an estimator of S based on observations (y_k)_1≤k≤n and E_p,S is the expectation with respect to the distribution lawP_p,Sof the process (y_k)_1≤k≤ngiven the distribution density p and the coefficient S. Moreover, taking into account that the distribution p is unknown, we use the robust nonparametric estimation approach proposed inGaltchouk and Pergamenshchikov (2006a). To this end we set the robust risk as

R^∗(Sb_n, S) = sup

p∈P

R_p(Sb_n, S), (1.3) whereP is a family of the distributions defined in Section2.

In order to estimate the functionSin model (1.6) we make use of the estimator family (Sb_λ, λ∈Λ), whereSb_λ is a weighted least square estimator with the Pinsker weights. For this family, similarly to Galtchouk and Pergamenshchikov (2009a), we construct a special selection rule, i.e. a random variablebλwith values in Λ, for which we define the selection estimator as Sb_∗ =Sb

bλ. Our goal in this paper is to show the non asymptotic sharp oracle inequality for the robust risks (1.3), i.e. to show that for any ˇ% >0 andn≥1

R^∗(Sb_∗, S) ≤ (1 + ˇ%) min

λ∈Λ R^∗(Sb_λ, S) +B_n ˇ

%n, (1.4)

whereB_n is a rest term such that for any ˇδ >0, lim

n→∞

B_n

n^δˇ = 0. (1.5)

In this case the estimatorSb_∗is called optimal in the oracle inequality sense.

In this paper, in order to obtain this inequality for model (1.1) we develop a new model selection method based on the truncated sequential procedures developed in Arkoun and Pergamenchtchikov (2016) for the pointwise efficient estimation.

Then we use the non asymptotic analysis tool proposed in Galtchouk and Perga- menshchikov (2009a) based on the non-asymptotic studies from Baron et al.

(1999) for a family of least-squares estimators and extended in Fourdrinier and Pergamenshchikov (2007) to some other estimator families. To this end we use the approach proposed in Galtchouk and Pergamenshchikov (2011), i.e. we pass to a discrete time regression model by making use of the truncated sequential pro- cedure introduced inArkoun and Pergamenchtchikov (2016). To this end, at any point (z_l)_1≤l≤d of a partition of the interval [a, b], we define a sequential procedure (τ_l, S_l^∗) with a stopping rule τ_land an estimator S_l^∗. ForY_l=S^∗_l with 1≤l≤d, we come to the regression equation on some set Γ⊆Ω:

Y_l=S(z_l) +ζ_l, 1≤l≤d . (1.6)

Here, in contrast with the classical regression model, the noise sequence (ζ_l)_1≤l≤d has a complex structure, namely,

ζ_l= ξ_l^∗+$_l, (1.7)

where (ξ^∗_l)_1≤l≤d is a ”main noise” sequence of uncorrelated random variables and ($_l)_1≤l≤n is a sequence of bounded random variables.

We will use the oracle inequality (1.4) to prove the asymptotic efficiency for the proposed procedure, using the same method as it is been used inGaltchouk and Pergamenshchikov (2009b). The asymptotic efficiency means that the procedure provides the optimal convergence rate and the asymptotically minimal rate normal- ized risk which coincides with the Pinsker constant. It should be emphasized that only sharp oracle inequalities of type (1.4) allow to synthesis efficiency property in the adaptive setting.

The paper is organized as follows: In Section 2 we state the main conditions for the model (1.1). In Section3we describe the passage to the regression scheme.

In Section4 we describe the sequential model selection procedure. In Section5 we announce the main results. In Section 6 we study the properties of the obtained regression model (1.6). In Section7 we prove all basic results. In AppendixA we give all the auxiliary and technical tools.

2 Main Conditions

As inArkoun and Pergamenchtchikov (2016) we assume that in the model (1.1) the i.i.d. random variables (ξ_k)_k≥1have a densityp(with respect to Lebesgue measure) from the functional classP defined as

P :=

( p≥0 :

Z +∞

−∞

p(x) dx= 1,

Z +∞

−∞

x p(x) dx= 0,

Z +∞

−∞

x²p(x) dx= 1 and sup

k≥1

R+∞

−∞ |x|^2kp(x) dx ς^k(2k−1)!! ≤1







, (2.1)

whereς ≥1 is some parameter, which may be a function of the number observation n, i.e. ς=ς(n), such that for any ˇδ >0

lim

n→∞

ς(n)

n^δˇ = 0. (2.2)

Note that the (0,1)-Gaussian density belongs to P. In the sequel we denote this density byp₀. It is clear that for any q >0

m^∗_q = sup

p∈P

E_p|ξ₁|^q<∞, (2.3)

where E_p is the expectation with respect to the density pfrom P. To obtain the stable (uniformly with respect to the functionS ) model (1.1), we assume that for some fixed 0< <1 andL >0 the unknown functionS belongs to theε-stability setintroduced inArkoun and Pergamenchtchikov (2016) as

Θ_,L=n

S ∈C₁([a, b],R) :|S|_∗≤1− and |S|˙ _∗≤Lo

, (2.4)

where C₁([a, b],R) is the Banach space of continuously differentiable [a, b] → R functions and|S|_∗= sup_a≤x≤b|S(x)|.

3 Passage to a discrete time regression model

We will use as a basic procedure the pointwise procedure fromArkoun and Perga- menchtchikov (2016) at the points (z_l)_1≤l≤d defined as

z_l=a+ l

d(b−a) and d= [√

n], (3.1)

where [a] is the integer part of a number a. So we propose to use the first ι_l observations for the auxiliary estimation ofS(z_l). We set

Sb_ι

l= 1

A_ι

l

ι_l

X

j=1

Q_l,jy_j−1y_j, A_ι

l=

ι_l

X

j=1

Q_l,jy²_j−1, (3.2)

where Q_l,j =Q(u_l,j) and the kernel Q(·) is the indicator function of the interval [−1; 1], i.e. Q(u) =1_[−1,1](u). The points (u_l,j) are defined as

u_l,j =x_j−z_l

h . (3.3)

Note that to estimateS(z_l) on the basis of kernel estimator with the kernel Qwe use only the observations (y_j)_k

1,l≤j≤k_2,l from theh- neighborhood of the pointz_l, i.e.

k_1,l= [nze_l−neh] + 1 and k_2,l= [nze_l+neh]∧n , (3.4) whereze_l = (z_l−a)/(b−a) and eh=h/(b−a). Note that, only for the last point z_d=b, we takek_2,d=n. We chooseι_lin (3.2) as

ι_l=k_1,l+q and q=q_n= [(neh)^µ0] (3.5) for some 0< µ₀<1. In the sequel for any 0≤k < m≤nwe set

A_k,m=

m

X

j=k+1

Q_l,jy_j−1² and A_m=A_0,m. (3.6)

Next, similarly toArkoun (2011), we use a kernel sequential procedure based on the observations (y_j)_ι

l≤j≤n. To transform the kernel estimator in a linear function of observations and we replace the number of observationsnby the following stopping time

τ_l= inf{ι_l+ 1≤k≤k_2,l : A_ι

l,k≥H_l}, (3.7) where inf{∅} = k_2,l and the positive threshold H_l will be chosen as a positive random variable which is measurable with respect to theσ-field{y₁, . . . , y_ι

l}.

Now we define the sequential estimator as

S_l^∗= 1 H_l





τ_l−1

X

j=ι_l+1

Q_l,jy_j−1y_j +κlQ_l,τ

ly_τ

l−1y_τ

l



1_Γ

l, (3.8)

where Γ_l={A_ι

l,k_2,l−1≥H_l} and the correcting coefficient 0<κl≤1 on this set is defined as

A_ι

l,τ_l−1+κ_l²Q_l,τ

ly_τ²

l−1=H_l. (3.9)

Note that, to obtain the efficient kernel estimator ofS(z_l) we need to use allk_2,l− ι_l−1 observations. Similarly to Konev and Pergamenshchikov (1984), one can show thatτ_l≈γ_lH_l asH_l→ ∞, where

γ_l= 1−S²(z_l). (3.10)

Therefore, one needs to chooseH_l as (k_2,l−ι_l−1)/γ_l. Taking into account that the coefficientsγ_lare unknown, we define the thresholdH_l as

H_l= 1−e

γe_l (k_2,l−ι_l−1) and e= 1

2 + lnn, (3.11) where eγ_l = 1−Se²_ι

l

and Se_ι

l is the projection of the estimator Sb_ι

l in the interval ]−1 +e,1−e[, i.e.

Se_ι

l= min(max(Sb_ι

l,−1 +e),1−e). (3.12) To obtain the uncorrelated stochastic terms in kernel estimator forS(z_l) we choose the bandwidthhas

h=b−a

2d . (3.13)

As to the estimatorSb_ι

l, we can show the following property.

Theorem 3.1. The convergence rate in probability of the estimator (3.12)is more rapid than any power function, i.e. for anyb>0

lim

n→∞n^b max

1≤l≤d sup

S∈Θ_,L

sup

p∈P

P_p,S

|Se_ι

l−S(z_l)|> ₀

= 0, (3.14) where₀=₀(n)→0 asn→ ∞ such thatlim_n→∞n^δˇ₀=∞for anyδ >ˇ 0.

Now we set

Y_l=S_l^∗(z_l)1_Γ and Γ =∩^d_l=1Γ_l. (3.15) Using the convergence (3.14), we study the probability properties of the set Γ in the following theorem.

Theorem 3.2. For any b>0, the probability of the set Γ satisfies the following asymptotic equality

lim

n→∞n^b sup

S∈Θ_,L

P_p,S(Γ^c) = 0. (3.16)

In view of this theorem we can shrink the set Γ^c. So, using the estimators (3.15) on the set Γ we obtain the discrete time regression model (1.6) in which

ξ^∗_l = Pτ_l−1

j=ι_l+1 Q_l,jy_j−1ξ_j+κlQ(u_l,τ

l)y_τ

l−1ξ_τ

l

H_l (3.17)

and$_l=$_1,l+$_2,l, where

$_1,l= Pτ_l−1

j=ι_l+1Q_l,jy²_j−1sˇ_l,j+κ_l²Q(u_l,τ

l)y²_τ

l−1ˇs_l,τ

l

H_l , ˇs_l,j=S(x_j)−S(z_l) and

$_2,l=

(κl−κ²_l)Q(u_l,τ

l)y_τ²

l−1S(x_τ

l)

H_l .

Note that in the model (1.6) the random variables (ξ_j^∗)_1≤j≤d are defined only on the set Γ. For technical reasons we need to define these variables on the set Γ^c as well. To this end, for anyj ≥1 we set

Qˇ_l,j =Q_l,jy_j−11_{j<k

2,l}+p

H_lQ_l,j1_{j=k

2,l} (3.18)

and ˇA_ι

l,m=Pm

j=ι_l+1 Qˇ²_l,j. Note, that for anyj≥1 andl6=m

Qˇ_l,jQˇ_m,j = 0. (3.19)

and ˇA_ι

l,k_2,l≥H_l. So now we can modify the stopping time (3.7) as ˇ

τ_l= inf{k≥ι_l+ 1 : ˇA_ι

l,k≥H_l}. (3.20)

Obviously, ˇτ_l ≤k_2,l and ˇτ_l =τ_l on the set Γ for any 1≤l ≤d. Now similarly to (3.9) we define the correction coefficient as

Aˇ_ι

l,ˇτ_l−1+ ˇκ_l²Qˇ²_l,ˇτ

l =H_l. (3.21)

It is clear that 0 < κˇl ≤ 1 and ˇκl =κl on the set Γ for 1 ≤ l ≤d. Using this coefficient we set

η_l= Pτˇ_l−1

j=ι_l+1Qˇ_l,jξ_j+ ˇκlQˇ_l,ˇτ

lξ_τˇ

l

H_l . (3.22)

Note that on the set Γ, for any 1≤l≤d, the random variablesη_l=ξ^∗_l. Moreover (see LemmaA.2), for any 1≤l≤dandp∈ P

E_p,S(η_l|G_l) = 0, E_p,S η²_l|G_l

=σ_l² and E_p,S η⁴_l |G_l

≤mσˇ _l⁴, (3.23) where σ_l =H_l^−1/2, G_l =σ{η₁, . . . , η_l−1, σ_l} and ˇm = 4(144/√

3)⁴m^∗₄. It is clear that

σ_0,∗≤ min

1≤l≤dσ_l²≤ max

1≤l≤dσ_l²≤σ_1,∗, (3.24) where

σ_0,∗= 1−²

2(1−e)nh and σ_1,∗= 1

(1−e)(2nh−q−3). Now, taking into account that|$_1,l| ≤Lh, for any S∈Θ_,L we obtain that

sup

S∈Θ_,L

E_p,S1_Γ$_l²≤

L²h²+ υˇ_n (nh)²

, (3.25)

where ˇυ_n = sup_p∈P sup_S∈Θ

,L E_p,S max_1≤j≤n y⁴_j. The behavior of this coefficient is studied in the following theorem.

Theorem 3.3. For anyb>0 the sequence(ˇυ_n)_n≥1 satisfies the following limiting equality

lim

n→∞n^−bυˇ_n= 0. (3.26)

Remark 3.1. It should be noted that the property (3.26)means that the asymptotic behavior of the upper bound (3.25)is approximately as h⁻² whenn→ ∞. We will use this in the oracle inequalities below.

Remark 3.2. It should be emphasized that to estimate the function S in (1.1) we use the approach developed inGaltchouk and Pergamenshchikov (2011) for the sequential drift estimation problem in the stochastic differential equation. On the basis of the efficient sequential kernel procedure developped inGaltchouk and Perga- menshchikov (2005),Galtchouk and Pergamenshchikov(2006b) andGaltchouk and Pergamenshchikov (2015) with the kernel-indicator, the stochastic differential equa- tion is replaced by regression model. It should be noted that to obtain the efficient estimator one needs to take the kernel-indicator estimator. By this reason, in this paper, we use the kernel-indicator in the sequential estimator (3.8). It also should be noted that the sequential estimator (3.8) which has the same form as inArkoun and Pergamenchtchikov (2016), except the last term, in which the correction coef- ficient is replaced by the square root of the coefficient used in Konev (2016). We modify this procedure to calculate the variance of the stochastic term (3.17).

4 Model selection

In this section we consider the nonparametric estimation problem in the non asymp- totic setting for the regression model (1.6) for some set Γ⊆Ω. The design points (z_l)_1≤l≤dare defined in (3.1). The functionS(·) is unknown and has to be estimated from observations the Y₁, . . . , Y_d. Moreover, we assume that the unobserved ran- dom variables (η_l)_1≤l≤dsatisfy the properties (3.23) with some nonrandom constant

ˇ

m>1 and the known random positive coefficients (σ_l)_1≤l≤d satisfy the inequality (3.24) for some nonrandom positive constantsσ_0,∗andσ_1,∗Concerning the random sequence$= ($_l)_1≤l≤n we suppose that

u^∗_d=E_p,S1_Γk$k²_d<∞. (4.1) The performance of any estimator Sb will be measured by the empirical squared error

kSb−Sk²_d = (bS−S,Sb−S)_d =b−a d

d

X

l=1

(bS(z_l)−S(z_l))².

Now we fix a basis (φ_j)_1≤j≤nwhich is orthonormal for the empirical inner product:

(φi, φ_j)_d=b−a d

d

X

l=1

φi(z_l)φ_j(z_l) =1_{i=j}. (4.2) For example, we can take the trigonometric basis (φ_j)_j≥1 inL₂[a, b] defined as

φ₁= 1, φ_j(x) = r 2

b−aTr_j(2π[j/2]l₀(x)), j≥ 2, (4.3) where the function Tr_j(x) = cos(x) for even j and Tr_j(x) = sin(x) for odd j, [x]

denotes the integer part of x. and l₀(x) = (x−a)/(b−a). Note that, using the orthonormality property (4.2) we can represent for any 1≤l≤dthe functionSas

S(z_l) =

d

X

j=1

θ_j,dφ_j(z_l) and θ_j,d= S, φ_j

d . (4.4)

So, to estimate the functionSwe have to estimate the Fourier coefficients (θ_j,d)_1≤j≤d. To this end, we replace the functionS by these observations, i.e.

θb_j,d= b−a d

d

X

l=1

Y_lφ_j(z_l). (4.5)

From (1.6) we obtain immediately the following regression scheme

θb_j,d = θ_j,d +ζ_j,d with ζ_j,d=

rb−a

d η_j,d+$_j,d, (4.6)

where

η_j,d=

rb−a d

d

X

l=1

η_lφ_j(z_l) and $_j,d=b−a d

d

X

l=1

$_lφ_j(z_l).

Note that the upper bound (3.24) and the Bounyakovskii-Cauchy-Schwarz inequal- ity imply that

|$_j,d| ≤ k$k_dkφ_jk_d=k$k_d. (4.7) We estimate the functionSon the grid (3.1) by the weighted least-squares estimator

Sb_λ(z_l) =

d

X

j=1

λ(j)θb_j,dφ_j(z_l)1_Γ, 1≤l≤d , (4.8)

where the weight vectorλ= (λ(1), . . . , λ(d))⁰belongs to some finite set Λ⊂[0,1]^d, the prime denotes the transposition. We set for anya≤t≤b

Sb_λ(t) =

d

X

l=1

Sb_λ(z_l)1_{z

l−1<t≤z_l}. (4.9)

Moreover, denotingλ²= (λ²(1), . . . , λ²(n))⁰ we define the following sets

Λ₁={λ², λ∈Λ} and Λ₂= Λ∪Λ₁. (4.10) Denote byν the cardinal number of the set Λ and

ν^∗= max

λ∈Λ d

X

j=1

1{λ(j)>0}.

In order to obtain a good estimator, we have to write a rule to choose a weight vectorλ∈Λ in (4.8). We define the empirical squared risk as

Err_d(λ) =kSb_λ−Sk²_d. Using (4.4) and (4.8) we can rewrite this risk as

Err_d(λ) =

d

X

j=1

λ²(j)bθ²_j,d −2

d

X

j=1

λ(j)bθ_j,dθ_j,d +

d

X

j=1

θ_j,d² . (4.11)

Since the coefficientθ_j,d is unknown, we need to replace the termθb_j,dθ_j,dby some of its estimators which we choose as

θe_j,d=θb²_j,d−b−a

d s_j,d with s_j,d=b−a d

d

X

l=1

σ²_lφ²_j(z_l). (4.12)

Note that from (3.24) - (4.2) it follows that

s_j,d≤σ_1,∗. (4.13)

Similarly to Galtchouk and Pergamenshchikov (2011) one needs to introduce a penalty term in the cost function to compensate such modification of the empirical risk. We choose it as

P_d(λ) =b−a d

d

X

j=1

λ²(j)s_j,d. (4.14)

Finally, we define the cost function in the following form J_d(λ) =

d

X

j=1

λ²(j)bθ²_j,d −2

d

X

j=1

λ(j)θe_j,d + δP_d(λ). (4.15)

where 0< δ <1 is some positive constant which will be chosen later. We set

λb= argmin_λ∈ΛJ_d(λ) (4.16)

and define an estimator ofS(t) of the form (4.9):

Sb_∗(t) =Sb

λb(t) for a≤t≤b . (4.17) Remark 4.1. We use the procedure(4.17)to estimate the functionS in the autore- gressive model (1.1)through the regression scheme (1.6)generated by the sequential procedures (3.15).

5 Main results

In this section we formulate all main results. First we obtain the sharp oracle inequality for the selection model procedure (4.17) for the general regression model (1.6).

Theorem 5.1. There exists some constant l^∗>0 such that for any weight vectors setΛ, anyp∈ P, anyn≥ 1 and0< δ≤1/12, the procedure (4.17), satisfies the following oracle inequality

E_p,SkSb_∗−Sk²_d≤1 + 4δ 1−6δmin

λ∈Λ E_p,SkSb_λ−Sk²_d

+l^∗νς² δ

σ_1,∗²

σ_0,∗d+u^∗_d+δ²p P_S(Γ^c)

!

. (5.1)

Using now LemmaA.7we obtain the oracle inequality for the quadratic risks (1.2).

Theorem 5.2. There exists some constant l^∗>0 such that for any weight vectors set Λ, any continuously differentiable function S, any p ∈ P, any n ≥ 1 and 0< δ≤1/12, the procedure (4.17)satisfies the following oracle inequality

R_p(Sb_∗, S) ≤ (1 + 4δ)(1 +δ)² 1−6δ min

λ∈Λ R_p(Sb_λ, S) +l^∗ς²ν

δ

kSk˙ ²

d² + σ_1,∗²

σ_0,∗d+u^∗_d+δ²p P_S(Γ^c)

!

. (5.2)

Now we assume that the cardinal ν of Λ and the parameter ς in the density family (2.1) are functions of the number observationsn, i.e. ν =ν(n) andς =ς(n) such that for any ˇδ >0

lim

n→∞

ν(n)

n^δˇ = 0. (5.3)

Using Theorems 3.2 – 3.3 and the bounds (3.24) - (3.25) we obtain the oracle inequality for the estimation problem for the model (1.1).

Theorem 5.3. Assume that the conditions (2.2) and (5.3) hold. Then for any p ∈ P, S ∈ Θ_,L, n ≥ 3 and 0 < δ ≤ 1/12, the procedure (4.17) satisfies the following oracle inequality

R_p(Sb_∗, S)≤(1 + 4δ)(1 +δ)² 1−6δ min

λ∈ΛR_p(Sbλ, S) + Bˇ_n(p)

δn , (5.4)

where the termBˇ_n(p)is such that for anyδ >ˇ 0 lim

n→∞

Bˇ_n(p) n^δˇ = 0. We obtain the same inequality for the robust risks

Theorem 5.4. Assume that the conditions (2.2) and (5.3) hold. Then for any n ≥ 3, any S ∈ Θ_,L and any 0 < δ ≤ 1/12, the procedure (4.17) satisfies the following oracle inequality

R^∗(Sb∗, S)≤ (1 + 4δ)(1 +δ)² 1−6δ min

λ∈ΛR^∗(Sbλ, S) + B^∗_n

δn , (5.5)

where the termBˇ_n is such that for anyδ >ˇ 0 lim

n→∞

B^∗_n n^δˇ = 0.

It is well known that to obtain the efficiency property we need to specify the weight coefficients (λ(j))_1≤j≤n(see, for example,Galtchouk and Pergamenshchikov

(2009b)). Consider for some fixed 0< ε <1 a numerical grid of the form

A={1, . . . , k^∗} × {ε, . . . , mε}, (5.6)