General model selection estimation of a periodic regression with a Gaussian noise

HAL Id: hal-00532592

https://hal.archives-ouvertes.fr/hal-00532592

Submitted on 4 Nov 2010

HAL is a multi-disciplinary open access archive for the deposit and dissemination of sci- entific research documents, whether they are pub- lished or not. The documents may come from teaching and research institutions in France or

L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires

General model selection estimation of a periodic regression with a Gaussian noise

Victor Konev, Serguei Pergamenchtchikov

To cite this version:

Victor Konev, Serguei Pergamenchtchikov. General model selection estimation of a periodic regression

with a Gaussian noise. Annals of the Institute of Statistical Mathematics, Springer Verlag, 2010, 62,

pp.1083 - 1111. �10.1007/s10463-008-0193-1�. �hal-00532592�

General model selection estimation of a periodic regression with a Gaussian noise

Victor Konev

Serguei Pergamenchtchikov

Abstract

This paper considers the problem of estimating a periodic function in a continuous time regression model with an additive stationary gaus- sian noise having unknown correlation function. A general model se- lection procedure on the basis of arbitrary projective estimates, which does not need the knowledge of the noise correlation function, is pro- posed. A non-asymptotic upper bound for L

-risk (oracle inequality) has been derived under mild conditions on the noise. For the Ornstein- Uhlenbeck noise the risk upper bound is shown to be uniform in the nuisance parameter. In the case of gaussian white noise the constructed procedure has some advantages as compared with the procedure based on the least squares estimates (LSE). The asymptotic minimaxity of the estimates has been proved. The proposed model selection scheme is extended also to the estimation problem based on the discrete data applicably to the situation when high frequency sampling can not be provided.

Key words: model selection procedure, periodic regression, oracle inequal- ity, non-parametric regression, improved estimation

∗

Department of Applied Mathematics and Cybernetics, Tomsk State University, Lenin street 36, 634050 Tomsk, Russia. e-mail: vvkonev@mail.tsu.ru

†

Laboratoire de Math´ematiques Rapha¨el Salem, UMR 6085 CNRS, Universit´e de Rouen, Avenue de l’Universit´e, BP.12, 76801 Saint Etienne du Rouvray (France).

email: Serge.Pergamenchtchikov@univ-rouen.fr

1 Introduction

Consider a regression model in continuous time

dy

= S(t)dt + dξ

, (1)

where S(t) is an unknown 1-periodic function in the space L

[0, 1], (ξ

)

is a continuous gaussian process with zero mean and such that for each n ≥ 1 the stochastic integral R

0

f (t)dξ

is well-defined for any non-random function f from L

[0, n]. The correlation function of noise ξ

is unknown.

This process can be modeled in different ways.

Example 1. ξ

is a scalar non-explosive Ornstein-Uhlenbeck process defined by the equation

dξ

= θξ

dt + dw

, (2)

where (w

)

is a standard brownian motion and θ ≤ 0 is unknown param- eter; the initial value ξ

∼ N (0, 1/2 | θ | ) if θ < 0 and ξ

= 0 if θ = 0.

Example 2. ξ

is a stationary autoregressive process of order q ≥ 2 satis- fying the stochastic differential equation

ξ

= θ

ξ

+ . . . + θ

ξ

+ ˙ w

. (3) Here ( ˙ w

)

is a white gaussian noise and the unknown vector θ = (θ

, . . . , θ

)

belongs to stability region of the process

A = { θ ∈ R

: max

1≤i≤q

Reλ

(θ) < 0 } , (4) where (λ

(θ))

are eigenvalues of the matrix

A = A(θ) =

θ

. . . θ

I

0

; (5)

I

is the identity matrix of order q.

Models of type (1) and their discrete-time analogues have been stud-

ied by a number of authors (see, Efroimovich (1999), Liptser and Shyraev

(1974), Konev and Pergamenshchikov (2003), Nemirovskii (2000) and ref-

erences therein). The estimation problem of periodic signal S(t) in model

(1)–(2) has been thoroughly studied in the case, when (ξ

)

is a white

gaussian noise (see, for example, Ibragimov and Hasminskii (1981) for de-

tails and further references).

A discrete-time counterpart of model (1)–(2) was applied in the econo- metrical problems for modeling the consumption as a function of income Golfeld and Quandt (1972).

As is well known, the problem of nonparametric estimation of S(t) com- prises the following three statements: the function estimation at a fixed point t

, estimation in the uniform metric and in the integral metric. The first two problems are usually solved by making use of the kernel and lo- cal polynomial estimates. This paper focuses on the third setting with the quadratic metric. The estimation in the integral metric is based, as a rule, on the projective estimates which were first proposed in Chenstov (1962) for estimating the distribution density in a scheme of i.i.d. observations. The heart of this method is to approximate the unknown function with a finite Fourier series. Applying the projective estimates to the regression model (1) with a white noise leads to the optimal convergence rate in L

(0, 1) pro- vided that the smoothness of S is known (see for example Ibragimov and Hasminskii (1981)). Another adaptive approach based on the model selec- tion method (see for example, Barron et al. (1999), Baraud (2000), Birg´e and Massart (2001) and Fourdrinier and Pergamenshchikov (2007)) enables one to study this problem in the nonasymptotic setting when the smoothness of function S is unknown. It should be noted that this method can be used also for model (1) under the condition that the correlation function Eξ

ξ

is exactly known and besides the unknown function S belongs to the subspace spanned by its eigenfunctions (see, Theorem 1, p. 11 in Birg´e and Massart (2001)). In our case, when the noise correlation function is unknown, this method can not be applied. This paper develops a general model selection method for the regression scheme (1) with unknown correlation properties.

Note that the usual nonasymptotic selection model procedure proposed

in Barron et al. (1999), Baraud (2000), Birg´e and Massart (2001) is based

on the least square estimators (LSE) which, as was shown in Goloubev

(1982) and Pinsker (1981), are not efficient in the problem of nonparametric

regression. Our approach is close to the general model selection method pro-

posed in Fourdrinier and Pergamenshchikov (2007) for discrete time models

with spherically symmetric errors which allows one to use any projective

estimators in the model selection procedure including the LSE. In Section

2 we propose a general model selection procedure for a regression scheme

in continuous time (1) with unknown correlation structure of the gaussian

noise. In Theorem 1, under some loose conditions on the noise, we establish

a nonasymptotic upper bound for the quadratic risk in which the principal

term is minimal over the set of all admissible basic estimates. The inequal-

ities of this type are usually called oracle.

In the case of the Ornstein-Uhlenbeck noise (2), the risk upper bound is shown to be uniform in the nuisance parameter (Corollary 2).

The rest of the paper is organized as follows. In Section 3 we consider case of white gaussian noise ξ

and show that the possibility to choose dif- ferent projective estimators in the procedure may lead to a sharper upper bound for the mean square estimation accuracy. In Section 4 the upper bound and the lower bounds for the minimax quadratic risk are obtained under the assumption that the smoothness of S is unknown. In Section 6 we consider the estimation problem for the regression model (1) assuming that it is accessible for observations only at discrete times t

= k/p, k = 0, 1, . . ..

Such observation scheme is more appropriate in a number of applications, where one can not provide high frequency data sampling. Theorems 5 estab- lishs the nonasymptotic oracle inequalities in this case. Appendix contains some technical results.

2 Nonasymptotic estimation

In this section we consider the estimation problem for the model (1) in nonasymptotic setting, i.e. assuming that the estimator of S is based on the observations (y

)

with a fixed duration n. For this we apply the general model selection approach proposed in Fourdrinier and Pergamenshchikov (2007) for the discrete-time regression model.

First we introduce some notations. Let X be the Hilbert space of square integrable 1-periodic functions on R with the usual scalar product

(x, y) = Z

0

x(t) y(t) dt

and (φ

)

be a system of orthonormal functions in X , i.e. (φ

, φ

) = 0, if i 6 = j and k φ

k

= (φ

, φ

) = 1.

Then we impose the following additional conditions on the noise (ξ

)

in (1). Assume that

C

) For each n ≥ 1 and k ≥ 1 the vector ζ(n) = (ζ

(n), . . . , ζ

(n))

with components

ζ

(n) = 1

√ n Z

0

φ

(t) dξ

(6)

is gaussian with non-degenerate covariance matrix B

= E ζ (n)ζ

(n).

C

) The maximal eigenvalues of matrices B

satisfy the following inequal- ity

sup

k≥1

sup

n≥1

λ

(B

) ≤ λ

, where λ

is some known positive constant.

Processes (2) and (3) in Examples 1–2, as is shown in Lemmas 2–3, satisfy condition C

). Condition C

) is satisfied for process (2) with λ

= 2.

Condition C

) holds also for process (3) provided that the value of vector θ belongs to the following compact set

K

=

θ ∈ A : max

1≤i≤q

Re λ

(θ) ≤ − δ , | A(θ) | ≤ δ

, (7)

where 0 < δ < 1 is a known constant; | · | stands for the euclidean norm of matrix. Under this assumption process (3) satisfies condition C

) with

λ

= λ

(δ) = 2

δ

F

(δ) J

(δ) , (8) where

F

(δ) = q 2δ + 2q

δ

X

(2j)!

(j!)

δ

and J

(δ) = 1 δ + 2

δ

X

2

δ

.

Let N be the set of positive integer numbers, i.e. N = { 1, 2, . . . } . Denote by M some finite set of finite subsets of N and by ( D

)

a family of linear subspaces of X such that

D

= { x ∈ X : x = X

j∈m

λ

φ

, λ

∈ R } .

Let d

= dim D

be the number of elements in a subset m. Denote by S

the projection of S on D

, i.e.

S

= X

j∈m

α

φ

, α

= (S, φ

) . (9) To estimate the function S in (1) we will apply a general model selection approach. It requires first to choose some class of projective estimators S e

of S

, which may be any measurable functions of observations (y

)

taking on values in D

. For example, one can take the LSE S b

of S, which is the minimizer, with respect to x ∈ D

, of the quantity

γ

(x) = k x k

− 2 1 n

Z

x(t) dy

(10)

and has the form

S b

= X

j∈m

b

α

φ

, α b

= 1 n

Z

φ

(t) dy

. (11) Let (l

)

be a sequence of prior weights such that l

≥ 1 for all m ∈ M . We set

l

= X

m∈M

e

. (12)

Further one needs a penalty term on the set M . We take it in the form suggested in Birg´e and Massart (2001). We define the penalty term as

P

(m) = ρ l

d

n with ρ = 4λ

z

z

− 1 , (13)

where z

is the maximal root of the equation ln z = z − 2 which is approxi- mately equal to z

≈ 3, 1462.

Minimizing the penalized empirical contrast γ

( S e

)+P

(m) with respect to m ∈ M one finds

e

m = argmin

{ γ

( S e

) + P

(m) } (14) and obtains the model selection procedure S e

corresponding to a specific class of projective estimators ( S e

)

. For the LSE family ( S b

)

, this yields S b

with

b

m = argmin

{ γ

( S b

) + P

(m) } . (15) Our first result is the following.

Theorem 1. Assume that the conditions C

)–C

) are fulfilled for the noise in (1). Then for any class of projective estimators ( S e

)

the general model selection procedure S e

satisfies the following oracle inequality

E

k S e

− S k

≤ inf

m∈M

e a

(S) + λ

τ

n , (16)

where E

denotes the expectation with respect to the distribution of (1) given S ,

e a

(S) = 3E

k S e

− S k

+ 16λ

z

d

l

n and τ

= 16l

z

z

− 1 .

The proof of Theorem 1 is given in the Appendix.

Remark 1. It will be noted that the choice of the coefficient ρ in the penalty term (13), as will be shown in the proof of the theorem, provides the minimal value of the principal term e a

(S).

Now we will find the upper bound (16) for the LSE model selection procedure S b

defined by (11) and (15). To this end we have to calculate the accuracy of S b

for any m ∈ M . We have

E

k S b

− S k

= k S

− S k

+ E

k S b

− S

k

= k S

− S k

+ X

j∈m

E

( α b

− α

)

, where S

is given in (9). Moreover, the condition C

) yields

E

( α b

− α

)

= 1 n

E

Z

0

φ

(t) dξ

≤ λ

n . Therefore

E

k S b

− S k

≤ k S

− S k

+ λ

d

n . Thus, we obtain the following result.

Corollary 1. Under the conditions C

) and C

) the LSE model selection procedure S b

, defined by (11) and (15), satisfies the inequality

E

k S b

− S k

≤ inf

m∈M

b a

(S) + λ

τ

n , (17)

where

b a

(S) = 3 k S

− S k

+ τ

λ

d

l

n , τ

= 3 + 16z

.

Consider the upper bound in (17) in more detail for the model (1)-(2).

Corollary 2. For the model (1)–(2) the LSE model selection procedure S b

, defined by (11) and (15) with λ

= 2 satisfies, for any θ ≤ 0, the inequality

E

k S b

− S k

≤ inf

m∈M

b b

(S) + 2τ

n , (18)

where τ

is given in (16),

b b

(S) = 3 k S

− S k

+ 2τ

d

l

n .

Remark 2. It will be noted that for the model (1)–(2) the LSE model selec-

tion procedure satisfies the oracle inequality uniformly in the nuisance pa-

rameter θ including the boundary of the stationarity region of the Ornstein-

Uhlenbeck process, i.e. θ = 0.

3 The improvement of LSE.

In this section we consider a special case of the model (1)–(2) when θ = 0, i.e.

dy

= S(t) dt + dw

. (19)

By applying the improvement method proposed in Fourdrinier and Perga- menshchikov (2007) we will show that the upper bound in the oracle in- equality can be lessened by a proper choice of the projective estimators. Let us introduce a class of estimators of the form

S

(t) = S b

(t) + Ψ

( S b

)(t) . (20) Here Ψ

is a function from R

into D

, i.e.

Ψ

(x)(t) = X

j∈m

v

(x) φ

(t) , x ∈ R

(21)

and (v

( · ))

are R

→ R functions such that E

v

( α) b < ∞ , where b

α = ( α b

)

is the vector with the components α b

defined in (11). The functions v(x) = (v

(x))

will be specified below. Let

∆

(S) = E

k S

− S

k

− E

k S b

− S

k

. (22) It is easy to check that

∆

(S) = 2 E

(Ψ

, S b

− S

) + E

k Ψ

k

. (23) This function can be found explicitly for the model (19).

Lemma 1. Let S

be defined by (20)–(21) with continuously differentiable functions v

such that E

v

( α) b < ∞ . Then ∆

(S) = E

L( α), where b

L(x) = 2

n div v(x) + k v(x) k

. (24) Proof. From (11), (20), one has

b α

= 1

n Z

0

φ

(t) dy

= α

+ 1 n

Z

φ

(t) dw

, where α

= R

0

φ

(t)S(t) dt. Therefore the vector α b = ( α b

)

has a normal

distribution N (α, n

I

), where α = (α

)

and I

is the unit matrix of

order d. This enables one to find the explicit expression for the first term in the right-hand side of (23). Indeed,

J = E

(Ψ

, S b

− S

) = E

X

j∈m

v

( α) ( b α b

− α

)

= E

(v( α) b , α b − α) = Z

R^d

(v(u) , u − α) g( k u − α k

) du , where

g(a) = n 2π

e

. (25)

Making the spherical changes of the variables yields J =

Z

Z

S_r,d

(v(u) , u − α) ν

(du) g(r

) dr

= Z

0

Z

Sr,d

(v(u) , e(u)) ν

(du) r g(r

) dr ,

where S

= { u ∈ R

: k u − α k = r } , ν

( · ) is the superficial measure on the sphere S

and e(u) = (u − α)/ k u − α k is a normal vector to this sphere.

By applying the Ostrogradsky–Stokes divergence theorem we obtain that J =

Z

Z

B_r,d

div v(u) du r g(r

) dr

with B

= { u ∈ R

: k u − α k ≤ r } . By the Fubini theorem and the definition of g in (25) one gets

J = 1 2

Z

R^d

Z

g(a) da div v(u) du

= 1 n

Z

R^d

g( k u − α k

) div v(u) du = 1

n E

div v( α) b . This leads to the assertion of Lemma 1.

In particular for d

> 2, if one takes v(u) = − (d

− 2)u

n k u k

, then L(u) = − (d

− 2)

n

k u k

,

and hence, ∆

(S) < 0, that is, the estimate (20) outperforms the least

squares esimate (11) in the approximation of S

. This allows to improve

the model selection procedure by making use of the estimates (20) instead

of the least squares { S b

} . As a direct consequence of Theorem 1, one obtains

the following result for the improved model selection procedure S

.

Theorem 2. For the model (19) the improvement model selection procedure S

defined by (14) with S e

= S

and λ

= 2 satisfies the inequality

E

k S

− S k

≤ inf

m∈M

u

(S) + 2τ

n , (26)

where τ

is given in (16),

u

(S) = 3 E

k S

− S k

+ 32 z

d

l

n .

4 Asymptotic estimation

4.1 The risk upper bound

In this section we consider the asymptotic estimation problem for the model (1). To this end, we additionally assume that all functions in the orthonor- mal system (φ

)

are 1-periodic and the unknown function S( · ) in the model (1) belongs to the following functional class

Θ

= { S ∈ C ( R ) ∩ X : max

n≥1

n

ς

(S) ≤ r

} . (27) Here C ( R ) denotes the set of all continuous R → R functions and

ς

(S) = X

s

, (28)

where (s

)

are the Fourier coefficients for the basis (φ

)

, i.e.

s

= (S, φ

) = Z

0

S(t) φ

(t) dt ; β > 0 and r > 0 are unknown constants.

Similarly to Galtchouk and Pergamenshchikov (2006) we define now the risk for an estimator S e

(a measurable function of the observation (y

)

in (6)) as follows

R

( S e

, β) = sup

S∈Θ_β,r

sup

Q∈Pκ

E

k ω

( S e

− S) k

, (29)

where ω

= ω

(β) = n

. Here P

is some class of distributions Q (in

the space C[0, + ∞ )) of the noise process (ξ

)

satisfying conditions C

)

and C

) with λ

= λ

(Q) ≤ κ < ∞ for some known fixed parameter κ. In addition, this class is assumed to include the Wiener distribution Q

. The second index in E

denotes that the expectation is taken with respect to the distribution of the process (1) corresponding to the noise distribution Q.

Note that for the model (1)–(2) P

is the class of distributions of the processes (2) with θ ≤ 0. In this case κ = 2. For the model (1)–(3)

P

= Q

∪ { Q

} ,

where Q

is the family of distributions of the processes (3) of order q ≥ 2 with the parameters belonging to the set (7) for some 0 < δ < 1. In this case κ = max(2, λ

(δ)), where λ

(δ)) is given in (8).

We will apply the ordered variable model selection procedure (see, Bar- ron et al. (1999), p. 315), for which M = { m

, . . . , m

} with m

= { 1, . . . , i } , therefore d

= i. Then

D

= { x ∈ X

: x = X

α

φ

, α

∈ R } .

For the ordered variable model selection procedure one can take l

= 1 for all m ≥ 1 and find

l

= X

i=1

e

≤ 1 e − 1 .

In the sequel we denote by S b

the LSE model selection procedure (11), (15) replacing λ

by κ. Now we will show that the risk (29) for this procedure is finite.

Theorem 3. The estimator S b

satisfies the following asymptotic inequality lim sup

n→∞

sup

β>0

R

( S b

, β) < ∞ . (30) Proof. Taking into account (17) one gets, for any Q ∈ P

,

E

k S b

− S k

≤ inf

m∈Mn

3 k S

− S k

+ τ

κ d

n

+ κτ

n

≤ inf

1≤i≤n

3 k S

− S k

+ τ

κ i n

+ κτ

n .

Further, for any function S from Θ

, one has k S

− S k

=

X

s

≤ r

i

.

Therefore for each 1 ≤ i ≤ n sup

Q∈P_κ

E

k S b

− S k

≤

3 r

i

+ τ

κ i n

+ τ

κ

n .

Substituting i = i

= [n

] + 1 leads to (30).

4.2 The risk lower bound

Now we study the lower bound for the risk (29). We assume that the or- thogonal system (φ

)

in (27) is trigonometric, i.e.

φ

(x) ≡ 1 , and for j ≥ 2 φ

(x) = √

2 Tr

(2π[j/2]x) , (31) where Tr

(x) = cos x for even j ≥ 1 and Tr

(x) = sin x for odd j ≥ 1; [a] is the integer part of a.

Theorem 4. The lower bound of the risk (29) over all estimates is strictly positive, i.e. for any β ≥ 1

lim

inf

Sen

R

( S e

, β) > 0 . (32) Proof. In order to show (32) it suffices to check this inequality for the model (19), i.e. that for any β ≥ 1

lim

inf

Sen

sup

S∈Θ_β,r

E

0

k ω

( S e

− S) k

> 0 . (33) To this end we apply the method proposed in Fourdrinier and Pergamen- shchikov (2007) to our case. First, we construct an auxiliary parametric class of functions in the set Θ

. Let β = k + α with k = [β] and 0 ≤ α < 1. Let V ( · ) be k + 1 times continuously differentiable function such that V (u) = 0 for | u | ≥ 1 and R

−1

V

(u) du = 1. Let m = [n

] and Γ

be a cube in R

of the form

Γ

= { z = (z

, . . . , z

)

∈ R

: | z

| ≤ δ , 1 ≤ j ≤ m } ,

where δ = ν/ω

, ν > 0. Now, viewing the function V ( · ) as a kernel, one introduces a parametric class of 1-periodic functions (S

)

δ

where S

(t) =

X

z

ψ

(t) , 0 ≤ t ≤ 1; (34) (ψ

(t))

are 1-periodic functions defined on the interval [0, 1] as ψ

(t) = V

j

h

with h = 1/2m and a

= (2j − 1)/2m.

It will be observed that, for 0 ≤ i ≤ k − 1 and z ∈ Γ

, sup

0≤t≤1

| S

(t) | ≤ 2

sup

|a|≤1

| V

(a) | ν

n

→ 0 as n → ∞ . In order to check the second condition in (62), we estimate the increment of kth derivative of S

( · ). For any 0 ≤ s , t ≤ 1 and z ∈ Γ

, one has

| S

(t) − S

(s) | ≤ ν

ω

h

∆

, (35)

where

∆

= X

V

t − a

h

− V

s − a

h

.

If s and t belong to the same interval, that is, a

− h ≤ s ≤ t ≤ a

+ h, then putting V

= sup

| V

(a) | one obtains

∆

= V

t − a

h

− V

s − a

h

≤ 2 V

| t − s |

2h ≤ 2

V

| t − s |

h

(36)

for each 0 ≤ α ≤ 1. If s and t belong to different intervals, that is, a

− h ≤ s ≤ a

+ h ≤ a

− h ≤ t ≤ a

+ h , j

< j

, then setting s

= a

+ h and t

= a

− h, similarly to (36), one gets

∆

= V

t − a

h

− V

t

− a

h

+

V

s − a

h

− V

s

− a

h

≤ 2

V

1

h

( | t − t

|

+ | s − s

|

) ≤ 2

1

h

V

| t − s |

.

From here and (35)–(36) we come to the estimate

| S

(t) − S

(s) | ≤ 2

ν V

m

ω

| t − s |

.

Therefore (see Lemma 5 in Appendix 6.4) there exist ν > 0 and n

≥ 1 such that S

∈ Θ

for all z ∈ Γ

and n ≥ n

. Further, we introduce the prior distribution on Γ

with the density

π(z) = π(z

, . . . , z

) = Y

π

(z

) , π

(u) = 1 δ G u

δ .

The function G(u) = G

e

1 1−u2

for | u | ≤ 1 and G(u) = 0 for | u | ≥ 1, where G

is a positive constant such that R

−1

G(u) du = 1.

Let S e

( · ) be an estimate of S( · ) based on observations (y

)

in (1).

Then for any n ≥ n

we can estimate with below the supremum in (33) as sup

S∈Θβ,r

E

0

k S e

− S k

>

Z

Γ_δ

E

z,Q₀

k S e

− S

k

π(z) dz . Moreover, by the definition of S

, we obtain

k S e

− S

k

≥ X

( e z

− z

)

Z

0

ψ

(t) dt = h X

l=1

( z e

− z

)

, where z e

= R

0

S e

(x) ψ

(x) dx/ k ψ

k

. Therefore, sup

S∈Θ_β,r

E

0

k S e

− S k

≥ h X

Λ

, (37)

where Λ

= R

Γδ

E

z

( z e

− z

)

π(z)dz. To apply now lemma 6 we note that in this case

ζ

(z) = Z

0

ψ

(t) dy

− Z

0

S

(t) ψ

(t)dt and, therefore, A

= E

z,Q0

ζ

(z) = R

0

ψ

(t) dt = nh. Moreover, in this case

B

= Z

−δ

( ˙ π

(u))

π

(u) du = δ

I

; I

= 8 Z

0

u

(1 − u

)

G(u) du .

Thus, by the inequality (66), one obtains that sup

S∈Θβ,r

E

0

k S e

− S k

≥ 1 2m

X

1 nh + ω

ν

I

= 1

2nh + 2ω

ν

I

. This immediately implies (33). Hence Theorem 4.

5 Estimation based on discrete data.

The model selection procedure developed in Section 2 is intended for con- tinuous time observations. However, in a number of applied problems high frequency sampling can not be provided. In this section, we consider the estimation problem for model (1) on the basis of observations (y

)

of the process (y

)

at discrete times t

= j/p, where p is a given odd num- ber. To solve this problem, we will modify the model selection procedure of Section 2. Let X

be the set of all 1-periodic functions x : R → R with the scalar product

(x, z)

= 1 p

X

x(t

) z(t

) , x, z ∈ X

. (38) Let (φ

)

be an orthonormal basis in X

, i.e. (φ

, φ

)

= 0, if i 6 = j and k φ

k

= 1. One can use, for example, the trigonometric basis (31).

Assume that the noise (ξ

)

in (1) is such that

C

) The vector ζ

(n) = (ζ

(n), . . . , ζ

(n))

with components ζ

(n) = 1

√ n X

φ

(t

) ∆ξ

, ∆ξ

= ξ

− ξ

, (39) is gaussian with non-degenerate covariance matrix B

= E ζ

(n)(ζ

(n))

; C

) The maximal eigenvalues of matrices B

are uniformly bounded :

sup

n≥1

sup

p≥1

λ

(B

) ≤ λ

, where λ

is some known positive constant.

Conditions C

), C

) are satisfied for processes (2), (3) (cf. Lemmas 2–

50).

Now we denote by M

some set of subsets of { 1, . . . , p } and by ( D

)

p

a family of linear subspaces of X

such that D

= { x ∈ X

: x = X

j∈m

λ

φ

, λ

∈ R } .

Let S

denote the projection of S on D

in X

and S e

denote an estimator of S

, i.e. a measurable function of the observations (y

)

taking on values in D

. One can use, for example, the LSE S b

for S

, which is defined as the minimizer with respect to x ∈ D

of the distance

1 np

X

∆y

∆t

− x(t

)

that is , the quantity

γ

(x) = k x k

− 2 1 n

X

x(t

) ∆y

k

(40)

and has the form

S b

= X

j∈m

b

α

φ

, α b

= 1 n

X

φ

(t

) ∆y

. (41) Let the penalty term P

(m) be defined, as before, by (13). Then the model selection procedure, corresponding to a family of projective estima- tors ( S e

)

p

, is defined as S e

p,p

where e

m

= argmin

p

{ γ

( S e

) + P

(m) } . (42) In the case of the LSE family ( S b

)

p

, it will be S b

p,p

.

As a measure of accuracy of the approximation of a 1-periodic function S of continuous argument t by its values on the (t

)

, we will use the function

H

(S) = 1 p

X

h

(S) , h

(S) = 1

∆t

Z

tl−1

(S(t) − S(t

)) dt . (43) The following theorem gives the oracle inequality for a general model selec- tion procedure S e

p,p

based on the discrete time observations.

Theorem 5. Assume that the conditions C

)–C

) hold. Then the estimator S e

p,p

satisfies the oracle inequality E

k S e

p,p

− S k

≤ inf

m∈M_p

e a

(S) + 8H

(S) + 2τ

λ

n , (44)

where

e

a

(S) = 7E

k S e

− S k

+ 32λ

z

l

d

n .

Now we obtain the oracle inequality (44) for the least square model selection procedure S b

p,p

. To this end, we have to calculate the estimation accuracy of S b

for S

, which is the projection of S on D

, i.e.

S

= X

j∈m

α

φ

, α

= (S, φ

)

= 1 p

X

S(t

) φ

(t

) .

First of all, we note that in this case α b

− α

= 1

p X

φ

(t

) h

(S) + 1 n

Z

φ

(t) dξ

, where φ

= P

k=1

φ

(t

) 1

k−1,tk]

(t). In view of the condition C

), this implies that

E

( α b

− α

)

= 1 p

X

φ

(t

) h

(S)

!

+ 1 n

E

Z

0

φ

(t) dξ

≤ H

(S) + λ

n . Corollary 3. Under the conditions C

)–C

) the LSE procedure S b

sat- isfies the inequality

E

k S b

p,p

− S k

≤ inf

m∈M_p

b b

(S) + 8H

(S) + 2τ

λ

n , (45)

where

b b

(S) = 7 k S

− S k

+ 7d

H

(S) + λ

(7 + 32z

l

) d

n .

Now we consider the estimation problem for the model (1)–(2) on the basis of discrete data in the asymptotic setting. First, for any β ≥ 1, we set

R

( S e

, β) = sup

S∈Θ_β,r

sup

Q∈Pκ

E

k ω

( S e

− S) k

, (46) where the set Θ

is defined by (27) with the use of the trigonometric basis (31), ω

= ω

(β) = n

and the set P

is defined in (29). As in Section 4, in order to minimize this risk, we apply the least square model selection procedure S b

, constructed on the basis of the trigonometric system (31) with the ordered selection, that is, M

= { m

, . . . , m

} with m

= { 1, . . . , j } . In this case d

= j and l

= 1 for 1 ≤ j ≤ p.

It is shown in Appendix 6.7, that if p ≥ n

, then for any ε > 0 lim

sup

β≥1+ε

R

( S b

p,p

, β) < ∞ (47)

and if p ≥ n

, then for any β ≥ 1 lim

inf

Sen

R

( S e

, β) > 0 . (48) It means that the adaptive estimator S b

p,p

with the p ≥ n

( in particular, one can take p = 2[n

] + 1 ) is optimal in the sense of the risk (46).

6 Appendix

6.1 Properties of processes (2)–(3)

We start with the result for process (2) which shows that both conditions C

)–C

) and C

)–C

) are satisfied.

Lemma 2. Let (ξ

) be defined by (2) with θ ≤ 0, f = (f

)

be a family of linearly independent cadlag 1-periodic functions on R , I

(f ) = R

0

f (u)dξ

. Then the matrix

V

(f) = (v

(f ))

(49) with elements v

(f ) = E I

(f

)I

(f

) is positive definite for each k ≥ 1, n ≥ 1 and θ ≤ 0. Moreover, if (f

)

is orthonormal, then for any θ ≤ 0

sup

k≥1

sup

n≥1

sup

|z|=1

1

n z

V

(f )z ≤ 2 . (50)

Proof. Assume that for some n ≥ 1, k ≥ 1 and z = (z

, . . . , z

)

∈ R

z

V

(f )z = 0. Since

z

V

(f )z = E I

(g) , where g(t) = P

j=1

z

f

(t), one gets I

(g) = R

0

g(t)dξ

= 0 a.s. Taking into account that the distribution of I

(g) for model (2) is equivalent to that of the random variable R

0

g(t)dw

this implies that g(t) = 0 for all t ∈ [0, n].

Thus z

= . . . = z

= 0 and to we come the first assertion. Let us check (50). By applying Ito’s formula one obtains

E I

(g) = 2θ Z

0

g(t) EI

(g) ξ

dt + Z

0

g

(t)dt , where

EI

(g) ξ

= 1 2

Z

g(u)e

du . Therefore,

EI

(g) = θ Z

0

e

Z

v

g(t)g(t − v)dt dv + Z

0

g

(t)dt . (51) From here, it follows that for any θ ≤ 0

z

V

(f )z ≤ Z

0

g

(t)dt

1 + θ Z

0

e

dv

≤ 2n Z

0

g

(t)dt = 2n X

z

= 2n . This completes the proof of Lemma 2.

Lemma 3. Let (ξ

) be defined by (3) with θ ∈ A , f = (f

)

be a family of linearly independent cadlag 1-periodic functions on R . Then the matrix (49) is positive definite for each k ≥ 1, n ≥ 1 and θ ∈ A . Moreover, if (f

)

is orthonormal, then for any 0 < δ < 1 and θ ∈ K

sup

k≥1

sup

n≥1

sup

|z|=1

1

n z

V

(f )z ≤ λ

(δ) , (52)

where λ

(δ) is defined in (8).

Proof. Let η

be process (3) with zero initial values, i.e.

dη

=



 X

θ

η



 dt + dw

and η

= . . . = η

= 0. Then ξ

can be written as

ξ

=< e

Y >

+η

, (53) where < X >

denotes the ith component of a vector X; A is the matrix defined in (5) and Y is a gaussian vector in R

independent of (η

)

with zero mean and covariance matrix

F = Z

0

e

D

e

du , (54) where D

is q × q matrix in which all elements exept of the (1, 1) element are equal to zero ant the (1, 1) element is equal to 1. In view of (53), one has

I

(g) = ζ + Z

0

g(t)dη

, (55)

where ζ =< R

0

g(t)Ae

dtY >

. Integration by parts yields Z

0

g(t)dη

= Z

0

G

(t)dη

, where G

(t) = g(t) and G

(t) = R

t

G

(u)du for 1 ≤ j ≤ q − 1.

Now assume that for some n ≥ 1, k ≥ 1 and z = (z

, . . . , z

)

∈ R

z

V

(f )z = 0. Since

z

V

(f )z = E I

(g) = E ζ

+ E Z

0

g(t)dη

,

this implies that E

Z

0

g(t)dη

= E Z

0

G

(t)dη

= 0 .

Taking into account that the distribution of the process (η

)

in C[0, n] is equivalent to Wiener measure we have

Z

G

(t)dw

= 0 a.s.

and therefore G

(t) = 0 for all 0 ≤ t ≤ n and hence g( · ) = 0 and we obtain z

= . . . = z

= 0. This leads to the first assertion.

Let us show (52). By direct calculations we find E I

(g) = 2

Z

< Ae

F A

>

Z

0

g(u + s) g(s)ds

du . where < A >

denotes the (i, j)-th element of matrix A. By applying the Bunyakovskii-Caushy-Schwartz inequality one gets

EI

(g) ≤ 2 Z

0

g

(s)ds Z

0

| < Ae

F A

>

| du . Since

Z

g

(s)ds = n Z

0

g

(s)ds = n X

z

Z

0

f

(s)ds = n , we obtain the estimate

1

n z

V

(f )z ≤ 2 Z

0

| < Ae

F A

>

| du ≤ 2 | A |

| F | J(A) , where J(A) = R

0

| e

| du. In order to come to (52) it remains to use the following inequality for matrix exponents of order q ≥ 2 (see, for example, in Kabanov and Pergamenshchikov (2003) on p. 228)

| e

| ≤ e



1 + 2 | B |

q−1

X

j=1

1

j! (2t | B | )



 , where Λ = max

Re λ

, λ

are eigenvalues of the matrix B.

Indeed, from (54) for any A ∈ K

we find that

| F | ≤ F

(δ) and J(A) ≤ J

(δ) ,

where the functions F

(δ) and J

(δ) are defined in (8). Hence Lemma 3.

6.2 Mean forecast inequality

Lemma 4. (Galtchouk and Pergamenshchikov (2005))

Let α and ξ be two positive random variables. Let β be a positive real number and { Γ

, x ≥ 0 } be a family of events such that, for any x,

P(ξ > α + βx , Γ

) = 0.

Assume also that there exists some positive integrable on R

function M (x) dominating P(Γ

). Then

E ξ ≤ E α + βM

, where M

= R

0

M(x)dx.

Proof. We set η = (ξ − α)

. Thus Eξ ≤ Eα + Eη. Moreover, Eη =

Z

P(η > z)dz = Z

0

P(ξ > α + z)dz

= β Z

0

P(ξ > α + βx)dx ≤ β Z

0

P(Γ

)dx ≤ βM

.

6.3 Proof of Theorem 1

By making use of (1) and (10) we obtain

k z − S k

= γ

(z) + 2 F

(z) + k S k

, (56) where F

(z) = n

R

0

z(t) dξ

. Further, from the definition (14), it follows that for each m ∈ M

γ

( S e

) ≤ γ

( S e

) + P

(m) − P

( m) e . Thus

k S e

− S k

≤ k S e

− S k

+ 2F

( z) + e ̟

(m, m) e , (57) where z e = S e

− S e

and ̟

(m, m) = e P

(m) − P

( m). Now for each e x > 0, 0 < µ < 1/2λ

and set ι ∈ M we introduce the following gaussian function on D

+ D

U

(z, µ) = 2F

(z)

k z k

+ ̺

(x, µ) , z ∈ D

+ D

, (58) where

̺

(x, µ) = 4 s

c(µ) N + d

l

+ x

nµ with c(µ) = − 1

2 ln(1 − 2λ

µ)

and N = dim( D

+ D

).

Moreover, let functions φ

, . . . , φ

be the subset of (φ

)

which is a basis in D

+ D

. It should be noted that N ≤ d

+ d

. Then one can write a normalized vector z = z/ k z k for z 6 = 0 as

z = X

a

φ

with X

a

= 1 . Therefore

U

(z, µ) = 2 n

k z k k z k

+ ̺

(x, µ)

X

a

ζ

with ζ

= 1

√ n Z

0

φ

dξ

. By applying the Bunyakovskii-Cauchy-Schvartz inequality one gets

| U

(z, µ) | ≤ 1

√ n ̺

(x, µ) η

, (59) where η

= qP

j=1

ζ

. Now note that by the condition C

) the vector (ζ

, . . . , ζ

) is gaussian with zero mean and a non-generate covariance ma- trix B

. Therefore

E e

PN j=1ζ_ij²

= 1

(2π)

p det B

Z

R^N

e

dx ,

where T

= (B

− 2µI

)

and I

is the identity matrix of order N . One can easily verify that

E e

PN j=1ζ_ij²

=

s det T

det B

= 1

p det(I

− 2µ B

) . Thus, in view of the inequality

det(I

− 2µB

) ≥ (1 − 2µ λ

(B

))

and the condition C

), we obtain

E e

PN j=1ζ_ij²

≤ (1 − 2µλ

)

= e

, where c(µ) is defined in (58).

Now, by the Chebyshev inequality, for any b > 0 and 0 < µ < 1/2λ

, we obtain that

P( η

> b ) ≤ e

. (60)

Choosing in this inequality b = b

(x, ι) = 1 4

√ n ̺

(x, µ) = s

c(µ) N + d

l

+ x µ

yields

P(η

> b

(x, ι)) ≤ e

.

Now let Γ

= { sup

η

/b

(x, ι) ≤ 1 } . It is easy to see that P(Γ

(x)) ≤ X

ι∈M

P(η

> b

(x, ι)) ≤ X

ι∈M

e

= l

e

. Thus, we obtain the following upper bound on the set Γ

sup

ι∈M

sup

z∈D_ι+D_m

| U

(z, µ) | ≤ 1/4 , which implies

2 F

( e z) = U

( e z, µ)( ke z k

+ ̺

(x)) ≤ 1

2 k S e

− S k

+ 1

2 k S e

− S k

+ 4

nµ (c(µ)d

l

+ (c(µ) + 1) d

l

) + 4 nµ x .

By making use of this inequality in (57) we obtain, on the set Γ

, that k S e

− S k

≤ 1

2 k S e

− S k

+ 3

2 k S e

− S k

+ ̟

(m, m) e + 4

nµ (c(µ)d

l

+ (c(µ) + 1) d

l

) + 4 nµ x

= 1

2 k S e

− S k

+ 3

2 k S e

− S k

+ Ω(ρ, µ) + 4 nµ x , i.e.

k S e

− S k

≤ 3 k S e

− S k

+ 2Ω(ρ, µ) + 8 nµ x , where

Ω(ρ, µ) =

ρ + 4c(µ) µ

d

l

n +

4c(µ) + 4

µ − ρ

d

l

n .

It is clear that to obtain a nonrandom minimal upper bound for this term we have to resolve the following optimazation problem

ρ + 4c(µ)

µ → min subject to 4c(µ) + 4

µ − ρ ≤ 0 . (61)

One can check directly that the solution of this problem is µ = (z

− 1)/2λ

z

and the optimal value for ρ is given in (13). Thus, by choosing these pa- rameters we have on the set Γ

k S e

− S k

≤ 3 k S e

− S k

+ 16λ

z

l

d

n + 16z

λ

n(z

− 1) x . Applying now Lemma 4 with ξ = k S e

− S k

,

α = 3 k S e

− S k

+ 16λ

z

l

d

/n , β = 16z

λ

n(z

− 1) and M (x) = l

e

we obtain the inequality (16). Hence Theorem 1.

6.4 Some properties of the Fourier coefficients

Lemma 5. Let S be a function in C

[0, 1] such that S

(0) = S

(1) for all 0 ≤ j ≤ k and, such that, for some contants L

> 0, L > 0 and 0 ≤ α < 1

0≤j≤k−1

max max

0≤x≤1

| S

(x) | ≤ L

and | S

(x) − S

(y) | ≤ L | x − y |

(62) for all x , y ∈ [0, 1]. Then the Fourier coefficients (a

)

and (b

)

of the function S, defined as

S(x) = a

2 +

X

(a

cos(2π kx) + b

sin(2π kx)) , satisfy the following inequality

sup

n≥0

(n + 1)



 X

(a

+ b

)





1/2

≤ c

(L + L

) , (63)

where β = k + α (k being an integer and 0 ≤ α < 1) and c

= 1 + 2

+ π

9

R

0

u

sin

(π u) du 8 R

0

u

sin

(π u) du .

Proof of this Lemma is given in Fourdrinier and pergamenshchikov (2007)

Appendix A4.