Minimax regression estimation for Poisson coprocess

HAL Id: hal-01430576

https://hal.archives-ouvertes.fr/hal-01430576

Submitted on 10 Jan 2017

HAL is a multi-disciplinary open access archive for the deposit and dissemination of sci- entific research documents, whether they are pub- lished or not. The documents may come from teaching and research institutions in France or

L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires

Minimax regression estimation for Poisson coprocess

Benoît Cadre, Nicolas Klutchnikoff, Gaspar Massiot

To cite this version:

Benoît Cadre, Nicolas Klutchnikoff, Gaspar Massiot. Minimax regression estimation for Poisson copro- cess. ESAIM: Probability and Statistics, EDP Sciences, 2017, 21, pp.138-158. �10.1051/ps/2017004�.

�hal-01430576�

Minimax regression estimation for Poisson coprocess

Benoˆıt CADRE

IRMAR, ENS Rennes, CNRS, UBL Campus de Ker Lann

Avenue Robert Schuman, 35170 Bruz, France benoit.cadre@ens-rennes.fr

Nicolas KLUTCHNIKOFFand Gaspar MASSIOT

IRMAR, Universit´e de Rennes 2, CNRS, UBL Campus Villejean

Place du recteur Henri le Moal CS 24307, 35043 Rennes cedex, France

klutchnikoff,massiot@univ-rennes2.fr

Abstract

For a Poisson point processX, Itˆo’s famous chaos expansion implies that ev- ery square integrable regression functionrwith covariateX can be decom- posed as a sum of multiple stochastic integrals called chaos. In this paper, we consider the case wherer can be decomposed as a sum ofδ chaos. In the spirit of Cadre and Truquet (2015), we introduce a semiparametric esti- mate of rbased on i.i.d. copies of the data. We investigate the asymptotic minimax properties of our estimator whenδ is known. We also propose an adaptive procedure whenδ is unknown.

Index Terms — Functional statistic, Poisson point process, regression estimate, Minimax estimation

AMS 2010 Classification — 62G08, 62H12, 62M30

1 Introduction

1.1 Regression estimation

Regression estimation is a central problem in statistics. It is widely used and studied in the litterature. Among all the methods explored to deal with the re- gression problem, nonparametric statistics have been widely investigated (see the monographies by Tsybakov, 2009 for a full introduction to nonparametric estima- tion and Gy¨orfi et al, 2002 for a clear account on nonparametric regression). A more recent challenge regarding this statistical problem is the regression onto a functional covariate (see the books by Ramsay and Silverman, 2005 and Horv´ath and Kokozska, 2012 for more precision on functional data analysis). Although very challenging, the functional regression problem in the minimax setting has little coverage up to our knowledge. In the kernel estimation setting, Mas (2012) studied the small ball probabilities over some Hilbert spaces to derive minimax lower bounds at fixed points. More recently, Chagny and Roche (2016) derived minimax lower bounds at fixed points for adaptive nonparametric estimation of the regression under some Wiener measure domination assumptions on the small ball probabilities. Based on the k-nearest neighbor approach, Biau, C´erou and Guyader (2010) used compact embedding theory to get bounds on the minimax risk. See also the references therein for a more complete overview.

1.2 Minimax regression for Poisson coprocess

In this paper, we focus on a regression problem for which the covariate is a Poisson point process. In the spirit of Cadre and Truquet (2015), we use a method based on the chaotic decomposition of Poisson functionals.

LetX be a Poisson point process on a compact domainX⊂R^dequipped with its Borel σ-algebraX. Lettingδ_x the Dirac measure on x∈X, the state space is identified to S ={s=∑^m_i=1δ_xi :m∈N^∗,x_i ∈X} equipped with the smallest σ-algebra making the mappings s7→s(B) measurable for all Borel set Bin X. We denote by PX the distribution of X whereasL²(PX)denotes the space of all measurable functionsg:S →Rsuch that

kgk²

L²(PX)=Eg(X)²<+∞.

Let P be a distribution onS ×R and (X,Y)with law P. Provided E|Y|<+∞

whereEis the expectation with respect toP, we consider the regression function r:S →Rdefined byr(s) =E(Y |X =s).

In this paper we assume thatrbelongs toL²(PX)and we aim at estimating r on the basis of an i.i.d. sample randomly drawn from the distributionPof(X,Y).

In this context, any measurable map ˜r:(S×R)ⁿ→L²(PX)is an estimator, the accuracy of which is measured by the risk

R_n(˜r,r) =Eⁿk˜r−rk²

L²(PX),

where Eⁿ denotes the expectation with respect to the distribution P^⊗n. Follow- ing the minimax approach, we define the maximal risk of ˜r over a class P of distributions for the random pair(X,Y)by

R_n(˜r,P) = sup

P∈P

R_n(˜r,r).

We are interested in finding an estimator ˆrsuch that R_n(r,ˆ P)inf

˜

r R_n(˜r,P),

where the infimum is taken over all possible estimates ofrandu_nv_nstands for 0<lim infnu_nv⁻¹_n ≤lim sup_nu_nv⁻¹_n <+∞. Such an estimate is called asymptoti- cally minimax overP.

1.3 Chaotic decomposition in the Poisson space

Roughly, Itˆo’s famous chaos expansion (see Itˆo, 1956 and Nualart and Vives, 1990 for technical details) says that every square integrable and σ(X)-measurable ran- dom variable can be decomposed as a sum of multiple stochastic integrals, called chaos.

To be more precise, we now recall some basic facts about chaos decomposition in the Poisson space. Let µ be the mean measure of the Poisson point ProcessX, defined by µ(A) =EX(A)forA∈X, wheneverX(A)is the number of points of X lying inA. Fixk≥1. Providedg∈L²(µ^⊗k), we can define thek-th chaosI_k(g) associated withg, namely

I_k(g) = Z

∆_k

gd X−µ)^⊗k, (1.1)

where ∆_k ={x∈ X^k :x_i 6=x_j for alli 6= j}. In Nualart and Vives (1990), it is proved that every square integrableσ(X)-measurable random variable can be de- composed as an infinite sum of chaos. Applied to our regression problem, this

statement writes as

r(X) =EY+

∑

k≥1

1

k!I_k(f_k), (1.2)

where equality holds in L²(PX), providedEY²<∞. In the above formula, each f_k is an element ofL²sym(µ^⊗k)–the subset of symmetric functions in L²(µ^⊗k)–, and the decomposition is defined in a unique way.

1.4 Organization of the paper

In this paper we introduce a new estimator of the regression functionr based on independent copies of (X,Y)and we study its minimax properties. Section 2 is devoted to the definition of a semiparametric model i.e. the construction of the family P of distributions of (X,Y). In particular, we assume that r is a sum of δ chaos. In Section 3, we provide a lower bound for the minimax risk over P. Whenδ is known, we prove that our estimator achieves this bound up to a logarithmic term. Finally, in Section 4, we define an adaptive procedure whenδ is unknown, the risk of which is also proved to be optimal up to a logarithmic term.

Last sections contain proofs.

2 Model

In the rest of the paper, we let Θ ⊂ R^p. For each θ ∈Θ, ϕ_θ : X→ R⁺ is a Borel function. The family {ϕ_θ}_θ∈Θ contains the constant function 1_X/λ(X), and is such that there exists three positive constantsϕ,ϕandγ₁satisfying, for all x,y∈Xandθ,θ⁰∈Θ,

ϕ≤ϕ_θ(x)≤ϕ, (2.1)

|ϕ_θ(x)−ϕ_θ⁰(x)| ≤ϕ|θ−θ⁰|, (2.2)

|ϕ_θ(x)−ϕ_θ(y)| ≤γ₁|x−y|, (2.3) where, here and in the following,| · |stands for the euclidean norm.

Let(X,Y)be a pair of random variables taking values inS×Rwith distribu- tionP, whereS is the Poisson space over the compact domainX⊂R^d. Here,X is a Poisson point process onXwith intensityϕ_θ, i.e. for all Borel setA∈X :

EX(A) = Z

A

ϕ_θdλ, (2.4)

whereλ is the Lebesgue measure andEis the expectation with respect toP. In other words, the mean measure ofX, sayµ, has a Radon-Nikod`ym derivativeϕ_θ with respect toλ. We assume that for alll≥1, there exists an estimator

θ˜_l: S ×R)^l→Θ, such that,

E^l|θ˜_l−θ|²≤ κ

l+1, (2.5)

where κ >0 is an absolute constant that does not depend on l and E^l is the ex- pectation with respect to P^⊗l. As shown in Birg´e (1983, Proposition 3.1), the above property is satisfied by a wide class of models, provided ˜θ_l is a maximum likelihood estimate.

Moreover, the real-valued random variableY satisfies, for someu,M>0, the exponential moment condition:

EY²e^u|Y|≤M, (2.6)

As seen in (1.2), the regression functionr(s) =E(Y |X =s)has a chaotic decom- position. In our model, we consider the case of a finite chaotic decomposition, i.e.

there exists a strictly positive integer δ and f₁∈L²sym(µ),· · ·,f_δ ∈L²sym(µ^⊗δ) such that

r(X) =EY+

δ

∑

k=1

1

k!I_k(f_k), (2.7)

where the I_k(f_k)’s are defined in (1.1). The coefficients f_k’s of the chaos lie in a nonparametric family, for which there exists two strictly positive constantsγ2and

f¯such that for allk=1, . . . ,δ, andx,y∈X^k

|f_k(x)− f_k(y)| ≤γ₂|x−y|, (2.8)

|f_k(x)| ≤ f¯. (2.9)

In the rest of the paper, the constantsϕ,ϕ,γ1,u,M,δ,γ2,f¯andκwill be fixed, and we shall denote by P the set of distributions P of (X,Y) such that the as- sumptions (2.1)-(2.5) are satisfied. In this setting, θ implicitly denotes the true value of the parameter, that isϕ_θ is the intensity ofX (with mean measureµ).

3 Minimax properties for known δ

3.1 Chaos estimator

Our task is to construct an estimate of the regression function which achieves fast rates overP. LetP∈P and(X,Y)∼PwhereX has mean measure µ =ϕ_θ·λ. First recall some basic facts about chaos decomposition in the Poisson space.

If g∈L²(µ^⊗k) andh∈L²(µ^⊗l)fork,l≥1, we have the so-called Itˆo Isometry Formula:

EI_k(g)I_l(f) =k!

Z

X^k

ghdµ^⊗k1_{k=l}andEI_k(g) =0, (3.1) wheregandhare the symmetrizations ofgandh, that is, for all(x₁, . . . ,x_k)∈X^k:

g(x₁, . . . ,x_k) = 1 k!

∑

σ

g(x_σ(1), . . . ,x_σ(k)), (3.2)

the sum being taken over all permutationsσ= σ(1), . . . ,σ(k)

of{1, . . . ,k}, and similarly forh.

Now letW be a strictly positive constant andW be a density onX such that sup_XW ≤W. Furthermore, let h_k =h_k(n)>0 a bandwidth to be tuned later on and denote

W_hk(·) = 1 h^d_kW

· h_k

.

One may easily deduce from relations (1.2) and (3.1) that EY I_k W_h^⊗k

k (x− ·)

= Z

X^k

f_kW_h^⊗k

k (x− ·)ϕ_θ^⊗kdλ^⊗k,

where, here and in the following, for any real-valued functiongdefined onX, the notationg^⊗kdenotes the real-valued function onX^k such that

g^⊗k(x) =

k

∏

g(x_i), x= (x₁, . . . ,x_k)∈X^k.

Thus, under the smoothness assumptions (2.1) on ϕ_θ and (2.9) on f_k, the right- hand side converges to f_k(x)ϕ^⊗k

θ (x), providedh_k→0.

Now let(X,Y),(X₁,Y₁), . . . ,(X_n,Y_n)be i.i.d. with distributionP. Based on this observation, a semiparametric estimate denoted ˆI_k,hk(X) of thek-th chaos I_k(f_k) of (1.1) may be defined as follows:

1 n

n

∑

i=1

Y_i1_|Yi|≤Tn Z

∆_k²

W_h^⊗k

k (x−y)

ϕ^⊗k_ˆ

θi (x) X_i−ϕ_ˆ

θi·λ⊗k

(dy) X−ϕ_ˆ

θi·λ⊗k

(dx), (3.3)

where T_n>0 is a truncation parameter to be tuned later on and the ˆθ_i’s are the leave-one-out estimates defined by ˆθi=θ˜n−1 (X_j)_j≤n,j6=i

(see Section 2).

3.2 Results

Based on the estimate (3.3) of the k-th chaos, we may define the following em- pirical mean type estimator of the regression function r for any strictly positive integerl

ˆ

r_l(X) =Y_n+

l

∑

k=1

1

k!Iˆ_k,hk(X), (3.4) whereY_nis the empirical mean ofY₁, . . . ,Y_n.

In this subsection, we study the performance of the estimate ˆr_δ of the regres- sion function from a minimax point of view when the number of chaosδ is known.

Theorem 3.1. Letε>0and set T_n= (lnn)^1+ε and h_k= (T_n²n⁻¹)^1/(2+dk). Then, lim sup

n→+∞

n (lnn)^2+2ε

2/(2+dδ)

sup

P∈P

R_n rˆ_δ,r)<∞.

Remark – Thus, the optimal rate of convergence over P is upper bounded by (lnn)^2+2εn⁻¹2/(2+dδ)

.Here it is noticeable that, up to a logarithmic factor, we recover the optimal rate n^−2/(2+dδ) corresponding to thedδ-dimensional regres- sion with a Lipschitz regression function (see, e.g., Theorem 1 in Kohler et al.

2009).

In our next result, we provide a lower bound for the optimal rate of conver- gence over P in order to assess the tightness of the upper bound obtained in Theorem 3.1.

Theorem 3.2. We have, lim inf

n→+∞n^2/(2+dδ)inf

˜ r sup

P∈P

R_n(r,˜ r)>0, where the infimum is taken over all estimatesr.˜

Remark– Theorem 3.2 indicates that the optimal rate of convergence overP is lower bounded byn^−2/(2+dδ)which, up to a logarithmic factor, corresponds to the upper bound found in Theorem 3.1. As a conclusion, up to a logarithmic factor, the estimate ˆr_δ is asymptotically minimax onP.

4 Adaptive properties for unknown δ

We now consider the case of an unknown number of chaosδ. Form>0, we set P(m) ={P∈P :kf_kk ≥m; k∈1, . . . ,δ},

wherek·kstands for theL²-norm relatively to the Lebesgue measure. Thus, when- everP∈P(m),

δ =min(k:kf_kk=0)−1.

Based on this observation, a natural estimate ofδ may be obtained as follows. Let we assume that the dataset is of size 2n, and let(X₁,Y₁), . . . ,(X_2n,Y_2n)be i.i.d. with distribution P∈P(m). Fork∈1, . . . ,δ, we introduce the empirical counterpart ofϕ_θf_kdefined by

ˆ

g_k(x) = 1 n

2n i=n+1

∑

Y_i Z

∆_k

W_b^⊗k

k (x−y) X_i−ϕ_θˆ·λ⊗k

(dy), (4.1) where ˆθ =θ˜_n(X_n+1, . . . ,X_2n)is defined in Section 2), and b_k =b_k(n) is a band- width to be tuned later. The estimator ˆδ ofδ is then defined by

δˆ =min(k:kgˆ_kk ≤ρ_k)−1, (4.2) whereρ_k=ρ_k(n)is a vanishing sequence of positive numbers that we choose later on. We may now define the adaptative estimator ˆrofrby

ˆ r=rˆ_ˆ

δ,

where ˆr_lis defined in (3.4) for all strictly positive integerl.

Theorem 4.1. Let ε >dδ ≥ 2, α,β >0 such that α +β < 1 and 2α+β >

1/(2+dδ), and set T_n= (lnn)^1+ε. Then, if we take for all integer k, h_k= (T_n²n⁻¹)^1/(2+dk), ρ_k= ((2k)!)²n^{(α+β−1)/2}and b_k=n^−β/(2dk), we obtain, for all m>0,

lim sup

n→+∞

n (lnn)^2+2ε

2/(2+dδ)

sup

P∈P(m)

R_n(r,ˆ r)<+∞.

Remark – Here it is noticeable that despite the estimation of the number of chaosδ and up to a logarithmic factor, we recover the optimal raten^−2/(2+dδ) of Theorems 3.1 and 3.2.

5 Proof of Theorem 3.1

In this section, we assume without loss of generality that the constantsϕ, γ₁, γ₂, f¯, λ(X) and W are greater than 1 and that ϕ is smaller than 1. Moreover, C denotes a positive number that only depends on the parameters of the model, i.e.

u,ϕ,ϕ,γ₁,γ₂,f¯,δ,θ,κ, λ(X),M andW, and whose value may change from line to line.

We letP∈Pand, for simplicity, we may denoteE=Eⁿand var stands for the variance relatively to P^⊗n. Finally, let(X,Y),(X₁,Y₁), . . . ,(X_n,Y_n) be i.i.d. with distributionP.

5.1 Technical results

Letk≥1 be fixed and denote for allx,y∈Xandi=1, . . . ,n:

g_i(x,y) =W_hk(x−y)

ϕθˆi(x) andg(x,y) =W_hk(x−y)

ϕ_θ(x) . (5.1)

We also let

d ˆX_i=dX_i−ϕ_θˆ

idλ, d ˆX_i⁰=dX−ϕ_θˆ

idλ, (5.2)

d ˜X_i=dX_i−ϕ_θdλ and d ˜X =dX−ϕ_θdλ. (5.3)

With this respect, we have (see (3.3)):

Iˆ_k,hk(X) =1 n

n i=1

∑

Y_i1_|Yi|≤Tn Z

∆_k²

g^⊗k_i (x,y)Xˆ_i^⊗k(dy)Xˆ_i^0⊗k(dx).

Furthermore, denote for allx∈X^k: Z_i,k(x) =Y_i1_|Yi|≤Tn

Z

∆k

g^⊗k(x,y)X˜_i^⊗k(dy). (5.4) Lemma 5.1. Let i=1, . . . ,n and k≤δ be fixed. Then for all x∈X^k:

var Z_i,k(x)

≤T_n²k!C^k h^dk_k , and

|EZ_i,k(x)−f_k(x)| ≤C^k

φ k!

h^dk_k 1/2

+C^kh_k, whereφ =EY²1_|Y|>Tn.

Proof. On the one hand, by the isometry formula (3.1) over the setP, var Z_i,k(x)

≤T_n²E Z

∆_k

g^⊗k(x,y)X˜^⊗k(dy)2

≤T_n²k!

Z

X^k

g^⊗k2(x,y)ϕ_θ^⊗k(y)dy

≤T_n²W^kϕ^k ϕ^2k

k!

h^dk_k ,

whereg^⊗k(x,·)is the symmetrization –see (3.2)– of the functiong^⊗k(x,·)defined in (5.1). On the other hand, denote

Z˜_i,k(x) =Y_i Z

∆k

g^⊗k(x,y)X˜_i^⊗k(dy), then by the isometry formula (3.1):

EZ˜_i,k(x) =Er(X) Z

∆k

g^⊗k(x,y)X˜^⊗k(dy) = Z

X^k

f_k(y)g^⊗k(x,y)ϕ^⊗k

θ (y)dy

= 1

ϕ_θ^⊗k(x) Z

X^k

f_k(x−h_kz)W^⊗k(z)ϕ^⊗k

θ (x−h_kz)dz.

Furthermore, by assumptions (2.1), (2.3), (2.8) and (2.9) on the model, we have for allx,y∈X^k:

|f_k(x)ϕ_θ^⊗k(x)−f_k(y)ϕ_θ^⊗k(y)| ≤ϕ^k|f_k(x)−f_k(y)|+f¯|ϕ_θ^⊗k(x)−ϕ_θ^⊗k(y)|

≤ ϕ^kγ₂+kf¯ϕ^k−1γ₁

|x−y|

≤2kf¯ϕ^kγ2γ1|x−y|.

Hence, lettingω_k=^R

X^k|z|W^⊗k(z)dz,

|EZ_i,k(x)−f_k(x)| ≤ |E Z_i,k(x)−Z˜_i,k(x)

|+|EZ˜_i,k(x)− f_k(x)|

≤

EY1_|Y|>Tn Z

∆k

g^⊗k(x,y)X˜^⊗k(dy)

+|EZ˜_i,k(x)−f_k(x)|

≤φ^1/2

E Z

∆_k

g^⊗k(x,y)X˜^⊗k(dy)21/2

+2kf¯ω_kϕ^kγ₂γ₁ ϕ^k

h_k. One last application of the isometry formula (3.1) to the first term on the right-

hand side of above gives the Proposition.

With the help of notations (5.1), (5.2) and (5.3), define Rⁱ_1k=E

^Z

∆_k²

g^⊗k_i (x,y)Xˆ_i^⊗k(dy)Xˆ_i^0⊗k(dx)−X˜^⊗k(dx)2

, (5.5)

Rⁱ_2k=E ^Z

∆_k²

g^⊗k_i (x,y)Xˆ_i^⊗k(dy)−X˜_i^⊗k(dy)X˜^⊗k(dx)2

. (5.6)

Lemma 5.2. Let i=1, . . . ,n and k≤δ be fixed. Then, for j=1or 2:

Rⁱ_jk≤C^k(k!)² nh^dk_k .

Proof.The proofs for the bounds forRⁱ_1kandRⁱ_2kbeing similar, we only prove the one forRⁱ_1k. We have

Rⁱ_1k=EE ^Z

∆_k²

g^⊗k_i (x,y)Xˆ_i^⊗k(dy)Xˆ_i^0⊗k(dx)−X˜^⊗k(dx)2

(X_l)_l≤n

, Using the independence of X and(X_l)_l≤n, we can apply Lemma 4.2 from Cadre and Truquet (2015), which entails

Rⁱ_1k≤

k−1 j=0

∑

j!

k j

2

ϕ^j Z

X^k

E

V_i^k−jZ

∆k

g^⊗k_i (x,y)Xˆ_i^⊗k(dy)2

dx, (5.7)

whereV_i=kϕ_ˆ

θi−ϕ_θk².Now letx∈X^k and j=0, . . . ,k−1 be fixed. We have EV_i^k−j

^Z

∆_k

g^⊗k_i (x,y)Xˆ_i^⊗k(dy)2

≤2EV_i^{k−j Z}

∆_k

g^⊗k_i (x,y) Xˆ_i^⊗k(dy)−X˜_i^⊗k(dy)2

+2EV_i^{k−j Z}

∆_k

g^⊗k_i (x,y)X˜_i^⊗k(dy) 2

.

We proceed to bound the two terms on the right-hand side of above. As before, we apply Lemma 4.2 from Cadre and Truquet (2015), but conditionally on(X_l)_l≤n,l6=i. For notational simplicity, and since it does change the result anymore, we do not specify the symmetrized version of the functions when using the isometry formula. By (3.1) and assumption (2.1), we then get

EV_i^{k−j Z}

∆k

g^⊗k_i (x,y)Xˆ_i^⊗k(dy)2

≤2

k−1 l=0

∑

l!

k l

2

ϕ^lEV_i^2k−l−j Z

X^k

g^⊗k_i (x,y)²dy +2k!ϕ^kEV_i^k−j

Z

X^k

g^⊗k_i (x,y)²dy.

Note that for allm≥1, by (2.2), we have V_i^m≤ ϕ²λ(X)m−1

sup

X

|ϕ_θˆ

i−ϕ_θ|2

λ(X)

≤ ϕ²λ(X)m

|θˆ_i−θ|², and

Z

X^k

g^⊗k_i (x,y)²dy≤ W^k ϕ^2kh^dk_k . Hence, sincel!

k l

≤k!, we get with (2.5):

EV_i^{k−j Z}

∆k

g^⊗k_i (x,y)Xˆ_i^⊗k(dy)2

≤2k!W^k φ^2k

κ

nh^dk_k φ²λ(X)k−j

ϕ+ϕ²λ(X)k

.

Finally, we deduce with similar arguments and inequality (5.7) that Rⁱ_1k≤2(k!)²W^k

ϕ^2k κ

nh^dk_k ϕ+ϕ²λ(X)2k

≤2(k!)²4^kW^kϕ^4k ϕ^2k

κ

nh^dk_k λ(X)^2k,

because bothϕ andλ(X)are greater than 1. The Lemma is proved.

Lemma 5.3. Letε >0be fixed and set T_n= (lnn)^1+ε. Then, for all k≤δ: E Iˆ_k,hk(X)−I_k(f_k)2

≤C^k(k!)² (lnn)^2+2ε nh^dk_k +h²_k

.

Proof. With the help of notations (5.1), (5.2), (5.3), we let J₁=1

n

n i=1

∑

Y_i1_|Yi|≤Tn Z

∆_k²

g^⊗k_i (x,y)X˜_i^⊗k(dy)X˜^⊗k(dx), J₂=1

n

n i=1

∑

Y_i1_|Yi|≤Tn Z

∆_k²

g^⊗k(x,y)X˜_i^⊗k(dy)X˜^⊗k(dx).

Then, using notations of Lemma 5.2, by Jensen’s Inequality E Iˆ_k,hk(X)−J₁2

≤2T_n² n

n

∑

i=1

(Rⁱ_1k+Rⁱ_2k), Hence, by Lemma 5.2

E Iˆ_k,hk(X)−J₁2

≤T_n²C^k(k!)²

nh^dk_k . (5.8)

Moreover, sequentially conditioning on (X_l)_l≤n, then on (X_l)_l≤n,l6=i, and using assumption (2.1), we find with two successive applications of the isometry for- mula (3.1) that

E J₁−J₂2

≤(k!)²T_n²ϕ^2kE Z

X^2k

g^⊗k₁ (x,y)−g^⊗k(x,y)2

dxdy.

Now letx,y∈X^k be fixed. We have g^⊗k₁ (x,y)−g^⊗k(x,y)

≤W_h^⊗k

k (x−y)

ϕ^⊗k

θ (x) kϕ^k−1 ϕ^k sup

X

|ϕ_ˆ

θ1−ϕ_θ|, so that (2.2) and (2.5) give

E J₁−J₂2

≤C^kT_n²(k!)²

nh^dk_k . (5.9)

Finally, using notation (5.4), by the isometry formula (3.1), we have E J₂−I_k(f_k)2

=E ^Z

∆k

1 n

n

∑

i=1

Z_i,k(x)−f_k(x)X˜^⊗k(dx)2

=k!

Z

X^k

E 1 n

n

∑

i=1

Z_i,k(x)− f_k(x)2

ϕ^⊗k

θ (x)dx

=k!

Z

X^k

1

nvar Z_1,k(x)

+ EZ_1,k(x)−f_k(x)2

ϕ_θ^⊗k(x)dx.

By Lemma 5.1, we thus get E J₂−I_k(f_k)2

≤T_n²(k!)²C^k

nh^dk_k +C^k(k!)²φ²

h^dk_k +C^kk!h²_k. Moreover, given that (2.6) givesφ ≤e^−uTnEY²e^u|Y|.Consequently,

E J₂−I_k(f_k)2

≤C^k(k!)² T_n²

nh^dk_k +e^−2uTn h^dk_k +h²_k

.

Finally, combining inequalities (5.8), (5.9) and above, we deduce that with the choiceT_n= (lnn)^1+ε:

E Iˆ_k,hk(X)−I_k(f_k)2

≤C^k(k!)²(lnn)^2+2ε nh^dk_k +h²_k

,

hence the Lemma.

5.2 Proof of Theorem 3.1

According to Jensen Inequality and Lemma 5.3, we have by (3.4) and (2.7):

E rˆ_δ(X)−r(X)2

=E

Y¯_n−EY+

δ k=1

∑

1

k! Iˆ_k,hk(X)−I_k(f_k) 2

≤2var(Y) n +δ

δ

∑

k=1

C^k

(lnn)^2+2ε nh^dk_k +h²_k

.

Settingh_k= (lnn)^2+εn⁻¹1/(2+dk)

, we deduce that, since var(Y)≤M:

E rˆ_δ(X)−r(X)2

≤ 2M

n +C(lnn)^2+2ε n

2/(2+dδ)

,

hence the theorem.

6 Proof of Theorem 3.2

In this section we assume for simplicity that X contains the hypercube X0 = [0,1]^d.

6.1 Technical results

We introduce the set F =Fδ(γ₂,f¯) of functions f :X^δ →R inL²sym(λ^⊗δ)for which conditions (2.8) and (2.9) hold, and letR =Rδ(γ₂,f¯)be the class of func- tionsr_f :S →Rwith f ∈F such that

r_f(·) = 1 δ!

Z

∆_δ

fd(· −λ)^⊗δ. (6.1)

LettingPthe distribution of the Poisson point process onXwith unit intensity, we may define the following distanceDonR by

D(r_f0,r_f1) =kr_f0−r_f1k_L2(P). (6.2) WheneverP∈P, we can associate the regression functionr. To stress the den- dendency onr, we now writePr instead ofP. Now letN>0. We define the fol- lowing three conditions for any sequence of sizeN+1 of functionsr⁽⁰⁾, . . . ,r^(N) fromS toR:

R1. r^(j)∈R, for j=0, . . . ,N;

R2. D(r⁽ⁱ⁾,r^(j))≥2n^−1/(2+dδ), for 0≤i< j≤N;

R3. 1 N

N

∑

K P^⊗n_r(j),P^⊗n_r(0)

≤ αlogN for some 0< α < 1/8 where K is the Kullback-Leibler divergence (see e.g. Tsybakov, 2009).

Lemma 6.1. Introduce f₀≡0,f₁, . . . ,f_N,N+1functions fromX^δ toRsuch that F1. f_j∈F;

F2. kf_i−f_jk ≥2n^−1/(2+dδ); F3. 1

N

N i=1

∑

n

2kf_ik²≤αlogN for some0<α <1/8.

Then, the sequence of functions r_f0, . . . ,r_fN defined by(6.1)verify conditions R1, R2 and R3.

Proof. First of all, remark that since f_j∈F for j=0, . . . ,N, by definition of the r_fj’s and the setR, we haver_fj ∈R. Hence condition R1 is satisfied by ther_fj’s.

Now remark that the Itˆo isometry (3.1) gives for any 0≤i,j≤N D(r_fj,r_fi) =kf_j−f_ik.

This ensures that ther_fj’s statisfy condition R2. Finally, for all j=0, . . . ,N K P^⊗nr_{f j},P^⊗nr_f0

=nK (Pr_{f j},Prf0)

=nErf0

logdPr_f0

dPr_{f j}

(X,Y)

=nErf0Erf0

logdPrf0

dPr_{f j}

(X,Y)|X

,

whereErf0 is the expectation underPrf0. Denote by pthe density of theN (0,1).

Then, since f₀≡0 Er_f0

logdPrf0

dPr_{f j}

(X,Y)|X

= Z

R

log

p(u) p u−r_fj(X)

p(u)du.

Simple calculus then give Er_f0

logdPrf0

dPr_{f j}

(X,Y)|X

≤ 1

2 r_fj(X)2

.

Thus, by the Itˆo Isometry,

K P^⊗nr_{f j},P^⊗nr_f0

≤ n 2kf_jk²,

hence the lemma.

6.2 Proof of Theorem 3.2

LetP0be the subset of distributionsPr of(X,Y)inP for whichX is a Poisson point process with unit intensity (recall that the unit function lies in{ϕ_θ}_θ∈Θ, see Section 2) and such that

Y =r(X) +ε, (6.3)

wherer∈Randε is independent fromXwith distributionN (0,1). SinceP0⊂ P, we have

infr˜ sup

Pr∈P0

R_n(r,˜ r)≤inf

˜ r sup

Pr∈PR_n(˜r,r).

As a result, in order to prove Theorem 3.2, we need only to prove that lim inf

n→+∞n^2/(2+dδ)inf

˜

r sup

Pr∈P0

R_n(r,r)˜ >0, which accordingly to (6.2) may be written

lim inf

n→+∞n^2/(2+dδ)inf

˜

r sup

Pr∈P0

EⁿrD²(˜r,r)>0, (6.4) whereEⁿr denotes expectation with respect toP^⊗nr . Then, according to Lemma 6.1 and Theorem 2.5 page 99 in the book by Tsybakov (2009), in order to prove (6.4), we need only to prove the existence of a sequence of functions satisfying conditions F1, F2 and F3 defined in the previous subsection. To this end, we let ψ ∈L²sym(λ^⊗δ) be a nonzero function such that Supp(ψ) =X^δ₀ and, for all x,y∈X^δ0 :

ψ(y)−ψ(x) ≤ γ₂

2|y−x|and|ψ(x)| ≤ f¯. (6.5) LetQ=

c₀n^dδ/(2+dδ)

≥8 wherec₀>0 andb·cis the integer part and leta_n= (1/Q)^1/(dδ). One may easily prove that there existst₁, . . . ,t_QinX^δ0 such that the functions

ψ_q(·) =a_nψ

· −t_q a_n

, forq=1, . . . ,Q, verify the following assumptions

(i) Supp(ψ_q)⊂X^δ₀,forq=1, . . . ,Q;

(ii) Supp(ψ_q)∩Supp(ψ_q⁰) = /0, forq6=q⁰; (iii) λ^⊗δ Supp(ψ_q)

=Q⁻¹.

Now let, for allω ∈ {0,1}^Q:

f_ω(·) =

Q

∑

q=1

ω_qψ_q(·),

According to the Varshamov-Gilbert Lemma (see the book by Tsybakov, 2009, Lemma 2.8 page 104), there exists a subsetΩ ={ω⁽⁰⁾, . . . ,ω^(N)}of{0,1}^Qsuch thatω⁽⁰⁾= (0,0,· · ·),N≥2^Q/8,and for all j6=k:

Q

∑

q=1

|ω_q^(j)−ωq^(k)| ≥Q 8. Now fix 0<α <1/8 and set

c₀= (4kψk²α⁻¹)^dδ/(2+dδ). We may now prove that functions{f

ω^(j) : j=0, . . . ,N}satisfy conditions F1, F2 and F3. First of all, let j6=kbe fixed and remark that,

kf

ω^(j)−f

ω^(k)k²≤ Z

X^δ₀

fω^(j)(x)− f

ω^(k)(x)2

dx

= Z

X^δ0

^Q

∑

q=1

ωq^(j)−ωq^(k)

ψq(x)2

dx

=

Q q=1

∑

Z

Supp(ψq)

ω_q^(j)−ω_q^(k)2

a²_nψ² x−t_q a_n

! dx,

=a²_n Q

Q

∑

q=1

ω_q^(j)−ω_q^(k) Z

X^δ₀

ψ²(x)dx.

Furthermore, by definition of the setΩ, we have Q

8 ≤

Q

∑

q=1

ω_q^(j)−ω_q^(k) ≤Q, so that

kψk²

8 a²_n≤ kf

ω^(j)−f

ω^(k)k²≤kψk²a²_n. (6.6)

Now let 0≤ j≤N. Sinceψ ∈L²sym(λ^⊗δ)it is clear that f

ω^(j) inherits that prop- erty. Then, using the first part of assumption (6.5) on ψ and assumption (ii) on the ψ_q’s, one may easily prove that f

ω^(j) is a Lipschitz function with constant γ2. Finally, using the second part of assumption (6.5) on ψ and assumption (ii) on the ψ_q’s, we get that for all x∈X^k, |f

ω^(j)(x)| ≤a_nf¯ witha_n ≤1 as soon as n≥c−(2+dδ)/(dδ)

0 . We conclude that f

ω^(j) ∈F so that condition F1 is satisfied by fω^(j). Now, takingk=0 in (6.6) gives

1 N

N

∑

n 2kf

ω^(j)k²≤ nkψk²

2 a²_n≤ kψk²

2 c−(2+dδ)/(dδ)

0 Q.

SinceN≥2^Q/8andc₀≥(4kψk²α⁻¹)^dδ/(2+dδ), we get 1

N

N j=1

∑

n 2kf

ω^(j)k²≤4kψk²c−(2+dδ)/(dδ)

0 logN≤αlogN,

so that F3 is satisfied. Finally, according to (6.6), we have kf

ω^(j)−f

ω^(k)k ≥ kψk 2√

2a_n= kψk 2√

2(2c₀)^−1/(dδ)n^−1/(2+dδ).

We can now conclude that conditions F1, F2 and F3 are satisfied so that according to Theorem 2.5 page 99 from the book by Tsybakov (2009) and Lemma 6.1, (6.4)

is verified. Theorem follows.

7 Proof of Theorem 4.1

In this section, we assume without loss of generality that the constantsϕ, γ₁, γ₂, f¯,λ(X)andW are greater than 1 and thatϕ andmare smaller than 1. Moreover, C denotes a positive number that only depends on the parameters of the model, i.e.m,u,ϕ,ϕ,δ,γ₁,γ₂,f¯,θ,κ,λ(X),MandW, and whose value may change from line to line.

We letP∈P(m), and for simplicity, we denoteE=E²ⁿthe expectation with respect toP^⊗2n. Recall that(X,Y),(X₁,Y₁), . . . ,(X_2n,Y_2n)are i.i.d. with distribu- tionP.

7.1 Technical results

Fork≥1, denote for allx∈X^k: g_k(x) =1

n

n

∑

i=1

Y_i Z

∆k

W_b^⊗k

k (x−y) X_i−ϕ_θ·λ⊗k

(dy). (7.1)

We also let

S_k= 1 n

n i=1

∑

|Y_i| X_i(X) +ϕ λ(X)k

. (7.2)

Lemma 7.1. We have, for all k:

kgˆ_k−g_kk ≤C^kS_k|θˆ−θ| b^dk/2_k

.

Proof. Let ˆµ the signed measure µˆ(dy) =1

n

n

∑

i=1

Y_ih

X_i−ϕ_ˆ

θ·λ⊗k

(dy)− X_i−ϕ_θ·λ⊗k

(dy)i

=1 n

n i=1

∑

Y_i

k−1

∑

l=0

k l

(−1)^k−l

l

∏

X_i(dy_j) h ^k

∏

j=l+1

ϕ_θˆ(y_j)dy_j−

k

∏

j=l+1

ϕ_θ(y_j)dy_j i

.

Then,

kgˆ_k−g_kk²= Z

X^k

_Z

∆k

W_b^⊗k

k (x−y)µ(dy)ˆ 2

dx

= Z

∆_k²

_Z

X^k

W_b^⊗k

k (x−y⁽¹⁾)W_b^⊗k

k (x−y⁽²⁾)dx

µˆ(dy⁽¹⁾)µˆ(dy⁽²⁾).

But,

Z

X^k

W_b^⊗k

k (x−y⁽¹⁾)W_b^⊗k

k (x−y⁽²⁾)dx≤W^k b^dk_k . Furthermore, one gets by induction that forl=0, . . . ,k−1:

k j=l+1

∏

ϕ_θˆ(y_j)−

k j=l+1

∏

ϕ_θ(y_j)

≤ϕ^k−l−1

k j=l+1

∑

|ϕ_ˆ

θ(y_j)−ϕ_θ(y_j)|,

and by assumptions (2.1) and (2.2), Z

X^k−l k j=l+1

∑

|ϕ_ˆ

θ(y_j)−ϕ_θ(y_j)|dy_l+1. . .dy_k≤(k−l)λ(X)^k−l−1 Z

X

|ϕ_ˆ

θ−ϕ_θ|dλ

≤(k−l)λ(X)^k−lϕ|θˆ−θ|.

Puting all pieces together, we get kgˆ_k−g_kk²≤W^k

b^dk_k 1

n

n

∑

i=1

|Y_i|

k−1

∑

l=0

k k−1

l

X_i(X)^lϕ^k−l−1λ(X)^k−l|θˆ−θ| 2

.

We can then conclude

kgˆ_k−g_kk²≤W^kk²λ(X)²

b^dk_k |θˆ−θ|²1 n

n i=1

∑

|Y_i| X_i(X) +ϕ λ(X)k−12

.

Lemma follows, sinceϕ λ(X)≥1.

Denote for alli,j≥0

s_i,j=E|Y|ⁱ X(X) +ϕ λ(X)j

. (7.3)

Moreover,(V_n)_nis a sequence of real numbers, bigger than 1 and tending to infin- ity, to be tuned latter.

Lemma 7.2. Let l≥0be fixed. Then for all x∈X^k, E

Y

Z

∆k

W_b^⊗k

k (x−y)X˜^⊗k(dy)

l≤s_l,lkC^lk b^ldk_k . Moreover,

E

Y Z

∆k

W_b^⊗k

k (x−y)X˜^⊗k(dy)2

≤C^k√ s_4,4k

k!V_n²

b^dk_k +e^−uVn/2 b^2dk_k

.

Proof. First observe that

Z

∆k

W_b^⊗k

k (x−y)X˜^⊗k(dy) ≤W^k

b^dk_k

k

∑

l=0

k l

ϕ^k−lλ(X)^k−lX(X)^l

≤W^k

b^dk_k X(X) +ϕ λ(X)k

.

Hence,

E Y

Z

∆k

W_b^⊗k

k (x−y)X˜^⊗k(dy)

l≤W^lks_l,lk b^ldk_k Regarding the second inequality, we observe that

E

Y Z

∆k

W_b^⊗k

k (x−y)X˜^⊗k(dy)2

≤V_n²E ^Z

∆k

W_b^⊗k

k (x−y)X˜^⊗k(dy)2

+E

Y1_|Y|>Vn Z

∆_k

W_b^⊗k

k (x−y)X˜^⊗k(dy)2

.

By (3.1),

E ^Z

∆k

W_b^⊗k

k (x−y)X˜^⊗k(dy)2

≤k!C^k b^dk_k . Moreover, by the Cauchy-Schwarz Inequality, (2.6) and above,

h E

Y1_|Y|>Vn Z

∆k

W_b^⊗k

k (x−y)X˜^⊗k(dy)2i2

≤P(|Y|>V_n) +E

Y

Z

∆_k

W_b^⊗k

k (x−y)X˜^⊗k(dy)4

≤C^ks_4,4ke^−uVn b^4dk_k .

Puting all pieces together gives the result.

Lemma 7.3. Let k>δ be fixed. Then,

P(δˆ =k)≤C^k((2k)!+s_1,k)² n(ρ_kb^dk/2_k )²

+ s_2,2k n((2k)!)² +C^k

ρ_k⁸

s_8,8kmax n

(nb^dk_k )⁸,(k!)⁴V_n⁸

(nb^dk_k )⁴ + e^−2uVn (nb^2dk_k )⁴

.

Proof. Sincek>δ, we have with notation (7.1) P(δˆ =k)≤P(kgˆ_kk>ρ_k)

≤P(kgˆ_k−g_kk+kg_kk>ρ_k)

≤P(2kgˆ_k−g_kk>ρ_k) +P(2kg_kk>ρ_k). (7.4)