Efficient estimation using the Characteristic Function

Texte intégral

(1)2013s-22. Efficient estimation using the Characteristic Function Marine Carrasco, Rachidi Kotchoni. Série Scientifique Scientific Series. Montréal Juillet 2013. © 2013 Marine Carrasco, Rachidi Kotchoni. Tous droits réservés. All rights reserved. Reproduction partielle permise avec citation du document source, incluant la notice ©. Short sections may be quoted without explicit permission, if full credit, including © notice, is given to the source..

(2) CIRANO Le CIRANO est un organisme sans but lucratif constitué en vertu de la Loi des compagnies du Québec. Le financement de son infrastructure et de ses activités de recherche provient des cotisations de ses organisations-membres, d’une subvention d’infrastructure du Ministère du Développement économique et régional et de la Recherche, de même que des subventions et mandats obtenus par ses équipes de recherche. CIRANO is a private non-profit organization incorporated under the Québec Companies Act. Its infrastructure and research activities are funded through fees paid by member organizations, an infrastructure grant from the Ministère du Développement économique et régional et de la Recherche, and grants and research mandates obtained by its research teams. Les partenaires du CIRANO Partenaire majeur Ministère de l'Enseignement supérieur, de la Recherche, de la Science et de la Technologie Partenaires corporatifs Autorité des marchés financiers Banque de développement du Canada Banque du Canada Banque Laurentienne du Canada Banque Nationale du Canada Banque Scotia Bell Canada BMO Groupe financier Caisse de dépôt et placement du Québec Fédération des caisses Desjardins du Québec Financière Sun Life, Québec Gaz Métro Hydro-Québec Industrie Canada Investissements PSP Ministère des Finances et de l’Économie Power Corporation du Canada Rio Tinto Alcan State Street Global Advisors Transat A.T. Ville de Montréal Partenaires universitaires École Polytechnique de Montréal École de technologie supérieure (ÉTS) HEC Montréal Institut national de la recherche scientifique (INRS) McGill University Université Concordia Université de Montréal Université de Sherbrooke Université du Québec Université du Québec à Montréal Université Laval Le CIRANO collabore avec de nombreux centres et chaires de recherche universitaires dont on peut consulter la liste sur son site web. Les cahiers de la série scientifique (CS) visent à rendre accessibles des résultats de recherche effectuée au CIRANO afin de susciter échanges et commentaires. Ces cahiers sont écrits dans le style des publications scientifiques. Les idées et les opinions émises sont sous l’unique responsabilité des auteurs et ne représentent pas nécessairement les positions du CIRANO ou de ses partenaires. This paper presents research carried out at CIRANO and aims at encouraging discussion and comment. The observations and viewpoints expressed are the sole responsibility of the authors. They do not necessarily represent positions of CIRANO or its partners.. ISSN 1198-8177. Partenaire financier.

(3) Efficient estimation using the Characteristic Function * Marine Carrasco †, Rachidi Kotchoni ‡. Résumé / Abstract The method of moments proposed by Carrasco and Florens (2000) permits to fully exploit the information contained in the characteristic function and yields an estimator which is asymptotically as efficient as the maximum likelihood estimator. However, this estimation procedure depends on a regularization or tuning parameter. that needs to be selected. The aim of the present paper is to. provide a way to optimally choose by minimizing the approximate mean square error (AMSE) of the estimator. Following an approach similar to that of Newey and Smith (2004), we derive a higherorder expansion of the estimator from which we characterize the finite sample dependence of the AMSE on . We provide a data-driven procedure for selecting the regularization parameter that relies on parametric bootstrap. We show that this procedure delivers a root T consistent estimator of Moreover, the data-driven selection of the regularization parameter preserves the consistency, asymptotic normality and efficiency of the CGMM estimator. Simulation experiments based on a CIR model show the relevance of the proposed approach. Mots clés/Keywords : Conditional moment restriction, continuum of moment conditions, generalized method of moments, mean square error, stochastic expansion, Tikhonov regularization. Codes JEL : C00, C13, C15.. *. An earlier version of this work was joint with Jean-Pierre Florens. We are grateful for his support. This paper has been presented at various conferences and seminars and we thank the participants for their comments, especially discussant Atsushi Inoue. Partial financial support from CRSH is gratefully acknowledged. † University of Montreal, CIREQ, CIRANO, email: [email protected]. ‡ Corresponding author. University of Montreal, CIREQ, email: [email protected]..

(4) 1. Introduction. There is a one-to-one relationship between the characteristic function (henceforth, CF) and the probability distribution function of a random variable, the former being the Fourier transform of the latter. This implies that an inference procedure based on the empirical CF has the potential to be as e¢ cient as another one that exploits the likelihood function. Paulson et al. (1975) used a weighted modulus of the di¤erence between the theoretical CF and its empirical counterpart to estimate the parameters of the stable law. Feuerverger and Mureika (1977) studied the convergence properties of the empirical CF and suggested that "it may be a useful tool in numerous statistical problems". Since then, many interesting applications have been proposed, including Feuerverger and McDunnough (1981b,c), Koutrouvelis (1980), Carrasco and Florens (2000), Chacko and Viceira (2003) and Carrasco, Chernov, Florens, and Ghysels (2007) (henceforth, CCFG (2007)). For a quite comprehensive review of empirical CF-based estimation methods, see Yu (2004). The CF provides a good alternative to econometricians when the likelihood function is not available in closed form. For example, some distributions in the -stable family are naturally speci…ed via their CFs while their densities are known in closed form only at isolated points of the parameter space (see e.g. Nolan, 2009). The density of the Variance-Gamma model of Madan and Seneta (1990) has an integral representation whereas its CF has a simple closed form expression. The transition density of a discretely sampled continuous time process is not available in closed form, except when its parameterization coincides with that of a square-root di¤usion (Singleton, 2001). Even in this special case, the transition density takes the form of an in…nite mixture of Gamma densities with Poisson weights. The same type of density arises in the autoregressive Gamma model studied in Gouriéroux and Jasiak (2005). Ait-Sahalia and Kimmel (2006) propose closed form approximations for the log-likelihood function of various continuous-time stochastic volatility models. But their method cannot be applied to other situations without solving a complicated Kolmogorov forward and backward equation. Interestingly, the conditional CF can be derived in closed form for all continuous-time stochastic volatility models. ei. The CF, ' ( ; ) ; of a random vector xt 2 Rp (t = 1; :::; T ) is nothing but the expectation of. 0x t. of xt ,. with respect the distribution of xt , where 2. Rp. is the parameter that characterizes the distribution. is the Fourier index and i is the imaginary number such that i2 =. candidate moment condition for the estimation of 0 ei xt. 0 E(ei xt ).. 0. 1. Hence, a. (i.e., the true value of ) is given by ht ( ; ) =. This moment condition is valid for all. 2 Rp and hence, ht ( ; ) is a moment. function or a continuum of moment conditions. Feuerverger and McDunnough (1981b) propose an estimation procedure that consists of minimizing a norm of the sample average of the moment function. Their objective function involves an optimal weighting function that depends on the true unknown likelihood function. Feuerverger and McDunnough (1981c) apply the Generalized Method of Moments (GMM) to a discrete set of moment conditions obtained by restricting the continuous index a discrete grid. 2 ( 1;. 2 ; ::: N ).. 2 Rp to. They show that the asymptotic variance of the resulting estimator. can be made arbitrarily close to the Cramer-Rao bound by selecting the grid for. su¢ ciently …ne. and extended. Similar discretization approaches are used in Singleton (2001) and Chacko and Viceira. 2.

(5) (2003). However, the number of points in the grid for. must not be larger than the sample size for. the covariance matrix of the discrete set of moment conditions to be invertible. In particular, the …rst order optimality conditions associated with the discrete GMM procedure becomes ill-posed as soon as the grid ( 1 ; fht ( i ;. )gN i=1. 2 ; ::: N ). is too re…ned or too extended. Intuitively, the discrete set of moment conditions. converges to the moment function. 7! ht ( ; ); 2 R as this grid is re…ned and extended.. As a result, it is necessary to apply operator methods in a suitable Hilbert space to be able to handle the estimation procedure at the limit. Carrasco and Florens (2000) proposed a Continuum GMM (henceforth, CGMM) that permits to e¢ ciently use the whole continuum of moment conditions. Similarly to the classical GMM, the CGMM is a two-step procedure that delivers a consistent estimator at the …rst step and an e¢ cient estimator at the second step. The ideal (unfeasible) objective function of the second step CGMM is a quadratic form in an suitably de…ned Hilbert space with metrics K. 1,. where K is the asymptotic covariance. operator associated with the moment function ht ( ; ). To obtain a feasible e¢ cient CGMM estimator, one replaces the operator K by an estimator KT obtained from a …nite sample. However, the latter empirical operator is degenerate and not invertible while its theoretical counterpart is invertible only on a dense subset of the reference space. To circumvent these di¢ culties, Carrasco and Florens (2000) resorted to a Tikhonov-type regularized inverse of KT , e.g. K identity operator and. T. = KT2 + I. 1. KT , where I is the. is a regularization parameter. The CGMM estimator is root-T consistent and. asymptotically normal for any …xed and reasonably small value of . However, asymptotic e¢ ciency is obtained only by letting T 1=2 go to in…nity and. go to zero as T goes to in…nity.. The main objective of this paper is to characterize the optimal rate of convergence for. as T goes. to in…nity. To this end, we derive a Nagar (1959) type stochastic expansion of the CGMM estimator. This type of expansion has been used in Newey and Smith (2004) to study the higher order properties of empirical likelihood estimators. We use our expansion to …nd the convergence rates of the higher order terms of the MSE of the CGMM estimator. These rates depend on both. and T . We …nd that. the higher order bias of the CGMM estimator is dominated by two higher order variance terms. By equating the rates of these dominant term, we …nd an expression of the form. T. = c ( 0) T. g( ) ,. where. c ( 0 ) does not depend on T and g ( ) inherits some properties from the covariance operator K. To implement the optimal selection of. empirically, we advocate a naive estimator of. T. obtained by. minimizing an approximate MSE of the CGMM estimator obtained by parametric bootstrap. Even though the CGMM estimator is consistent, there is a concern that its variance be in…nite in …nite sample for certain data generating processes. This concern seems unfounded for the CIR model on which our Monte Carlo simulations are based. If applicable, this di¢ culty is avoided by truncating the AMSE similarly as in Andrews (1991). The remainder of the paper is organized as follows. In Section 2, we review the properties of the CGMM estimator in IID and Markov cases. In Section 3, we derive a higher-order expansion for the MSE of the CGMM estimator and use this expansion to obtain the optimal rate of convergence for the regularization parameter. T.. In Section 4, we describe a simulation-based method to estimate. T. and. show the consistency of the resulting estimator. Section 5 presents a simulation study based on the 3.

(6) CIR term structure model and Section 6 concludes. The proofs are collected in appendix.. 2. Overview of the CGMM. This section is essentially a summary of known results about the CGMM estimator. The …rst subsection present a general framework for implementing the CF-based CGMM procedure whilst the second subsection presents the basic properties of the resulting estimator.. 2.1. The CGMM Based on Characteristic function. Let xt 2 Rp be a random vector process whose distribution is indexed by a …nite dimensional parameter with true value. 0.. When the process xt is IID, Carrasco and Florens (2000) propose to estimate. 0. based on the moment function given by: ht ( ; ) = e i where '( ; ) = E. 0x. ei. 0x. t+1. '( ; );. (1). is the CF of xt and E is the expectation operator with respect to the. t+1. data generating process indexed by . CCFG (2007) extend the scope of the CGMM procedure to Markov and weakly dependent models. In this paper, we restrict our attention to IID and Markov cases. The moment function used in CCFG (2007) for the Markov case is: 0. ht ( ; ) = eis xt+1. 0. 't (s; ) eir xt :. 0. where 't (s; ) = E (eis xt+1 jxt ) is the conditional CF of xt and ir0 xt. set of basis functions fe. (2). = (s; r) 2 R2p . In equation (2), the. g is being used as instruments. CCFG (2007) show that these instruments. are optimal given the Markovian structure of the model. Moment conditions de…ned by (1) are IID whereas equation (2) describes a martingale di¤erence sequence. Note that a standard conditional moment restriction (i.e., non CF-based) can be converted into a continuum of moment unconditional moment restrictions featuring (2). In this case, the CGMM estimator may be viewed as an alternative to the estimator proposed by Dominguez and Lobato (2004) and to the smooth minimum distance estimator of Lavergne and Patilea (2008). Subsequently, we use the generic notation ht ( ; );. 2 Rd to denote a moment function de…ned by either (1) or (2), where. d = p for (1) and d = 2p for (2). Let. be a probability distribution function on Rd and L2 ( ) be the Hilbert space of complex valued. functions that are square integrable with respect to , i.e.: 2. d. L ( ) = ff : R ! Cj. Z. f ( )f ( ) ( )d < 1g;. where f ( ) denotes the complex conjugate of f ( ). As jht (:; )j2 4. 2 for all. (3) 2. , the function ht (:; ).

(7) belongs to L2 ( ) for all product on. L2 (. ). L2 (. 2. and for any …nite measure . Hence, we consider the following scalar. ):. hf; gi =. Z. f ( )g( ) ( )d :. (4). Based on this notation, the e¢ cient CGMM estimator is given by D b = arg min K. E hT (:; ); b hT (:; ) :. 1b. where K is the asymptotic covariance operator associated with the moment conditions. K is an integral operator and satis…es: Kf ( 1 ) = where k( 1 ;. 2). Z. 1 1. k( 1 ; )f ( ) ( ) d ; for any f 2 L2 ( ) ;. (5). is the kernel given by: k( 1 ;. 2). = E ht ( 1 ; )ht ( 2 ; ) :. (6). Some basic properties of the operator K are discussed in Appendix A. 1 With a sample of size T and a consistent …rst step estimator b in hand, one estimates k( 1 ; kT ( 1 ;. 2;. 2). T X 1 1 b1 ) = 1 ht ( 1 ; b )ht ( 2 ; b ): T. by: (7). t=1. In the speci…c case of IID data, an estimator of the kernel that does not use a …rst step estimator is given by: kT ( 1 ;. 2). =. T 1X i e T. 0x 1 t. t=1. where ' bT ( 1) =. 1 T. PT. t=1 e. i. 0x 1 t. ei. ' bT ( 1). 0x 2 t. ' bT ( 2) :. (8). . Unfortunately, an empirical covariance operator KT with kernel function. given by either (7) or (8) is degenerate and not invertible. Indeed, the inversion of KT raises a problem similar to one of the Fourier inversion of an empirical characteristic function. This problem is worsened by the fact that the inverse of K which KT is aimed at estimating exists only on a dense subset of L2 ( ). Moreover, when K. 1f. = g exists for a given function f , a small perturbation in f may give. rise to a large variation in g. To circumvent these di¢ culties, we consider estimating K K where the hyperparameter exist for all f in. L2 (. 1 T. = KT2 + I. 1. 1. by:. KT ;. plays two roles. First, it is a smoothing parameter as it allows K. 1 Tf. ). Second, it is a regularization parameter as it dampens the sensitivity of K. to 1 Tf. to perturbations in the input f . For any function f in the range of K and any consistent estimator. 5.

(8) fbT of f , K. 1b T fT. converges to K 1 T. expression for K. 1f. as T goes to in…nity and. goes to zero at appropriate rate. The. uses a Tikhonov regularization, also called ridge regularization. Other forms of. regularization could have been used, see e.g. Carrasco, Florens and Renault (2007). The feasible CGMM estimator is given by:. D bT ( ; ) = K where Q. 1b T hT (:;. bT ( ) = arg minQ bT ( ; ) ;. (9). E b T ( ; ) in matrix ); b hT (:; ) . An expression of the objective function Q. form is given in CCFG (2007a, Section 3.3). An alternative expression and a numerical algorithm for the numerical evaluation of this objective function based on Gauss-Hermite quadratures is described in Appendix D.. 2.2. Consistency and Asymptotic Normality. In order to study the properties of a CGMM estimator obtained within the framework described previously, the following assumptions are posited: is strictly positive on Rd and admits all its. Assumption 1: The probability density function moments. Assumption 2: The equation E has a unique solution. 0. (ht ( ; )) = 0 for all. 2 Rd ;. almost everywhere,. which is an interior point of a compact set. 0. .. Assumption 3: ht ( ; ) is three times continuously di¤erentiable with respect to . Furthermore, the …rst two derivatives satisfy: V ar. T 1 X @ht ( ; ) p T t=1 @ j. !. < 1 and V ar. T 1 X @ 2 ht ( ; ) p T t=1 @ j @ k. !. < 1;. for all j, k and T . Assumption 4: E of E. 0. (hT (:; )) w.r.t.. where:. 0. (hT (:; )) 2. belong to. for all for all. 2. and for some. in a neighborhood of. n = f 2 L2 ( ) such that. K. 1, and the …rst two derivatives 0. and for the same. o f <1. as previously, (10). Assumption 5: The random variable xt is stationary Markov and satis…es xt = r (xt where r (xt. 1 ; 0 ; "t ). is three times continuously di¤erentiable with respect to. noise whose distribution is known and does not depend on. 0. 1 ; 0 ; "t ). and "t is a IID white. 0.. Assumption 1 and 2 are quite standard and they have been used in Carrasco and Florens (2000). The …rst part of Assumption 3 ensures some smoothness properties for bT ( ) while the second part. is always satis…ed for IID models. The largest real 6. such that f 2. in Assumption 4 may be.

(9) called the level of regularity of f with respect to K: the larger. is, the better f is approximated by a. linear combination of the eigenfunctions of K associated with the highest eigenvalues. Because Kf (:) involve a d-dimensional integration,. may be a¤ected by both the dimensionality of the index. smoothness of f . CCFG (2007) have shown that we always have. 1 if f = E. 0. and the. (ht ( ; )). Assumption. 5 implies that the data can be simulated upon knowing how to draw from the distribution of "t . It is satis…ed for all random variables that can be written as a location parameter plus a scale parameter time a standardized representative of the family of distribution. Examples include the exponential family and the stable distribution. The IID case is a special case of Assumption 5 where r (xt. 1 ; 0 ; "t ). takes the simpler form r ( 0 ; "t ). Further discussions on this type of model can be found in Gourieroux, Monfort, and Renault (1993) in the indirect inference context. Note that the function r (xt may not be available in analytical form. In particular, the relation xt = r (xt. 1 ; 0 ; "t ). 1 ; 0 ; "t ). can be the. numerical solution of a general equilibrium asset pricing model (e.g., as in Du¢ e and Singleton, 1993). We have the following results: Theorem 1 Under Assumptions 1 to 5, the CGMM estimator is consistent and satis…es:. as T and T 1=2 go to in…nity and. T 1=2 bT ( ). L. 0. ! N (0; I. goes to zero, where I. 1 0. 1 0. ):. denotes the inverse of the Fisher Information. Matrix. See Proposition 3.2 of CCFG (2007) for a more general statement of the consistency and asymptotic normality result. A nice feature about the CGMM estimator is that its asymptotic distribution does not depend on the probability density function .. 3. Stochastic expansion of the CGMM estimator. The conditions required for the asymptotic e¢ ciency result stated by Theorem 1 allow for a wide range of convergence rates for. . Indeed, any sequence of type. T. = cT. a. (with c > 0) satis…es these. conditions as soon as 0 < a < 1=2. Among the admissible convergence rates, we would like to …nd the one that minimizes the mean square error of the CGMM estimator for a given sample size T . To achieve this, we consider deriving the stochastic expansion of the CGMM estimator. The higher order properties of GMM-type estimators have been studied by Rothenberg (1983, 1984), Koenker et al. (1994), Rilstone et al. (1996) and Newey and Smith (2004). For estimators derived in the linear simultaneous equation framework, examples include Nagar (1959), Buse (1992) and Donald and Newey (2001). The approach followed here is similar to Nagar (1959) and Newey and Smith (2004), which tries to approximate the MSE of an estimator analytically based on the leading terms of its stochastic expansion. Two di¢ culties arise when analyzing the terms of the expansion of the CGMM estimator. First, when the rate of. as a function of T is unknown, it is not always possible to write the terms of the 7.

(10) expansion in decreasing order. The second di¢ culty stems from a result that dramatically di¤ers from the case with a …nite number of moment conditions. Indeed, when the number of moment conditions is …nite, the quadratic form T b hT ( 0 )0 K 1b hT ( 0 ) is Op (1) and follows asymptotically a chi-square. distribution with degrees of freedom given by the number of moment conditions. However, the analogue 2 p of the previous quadratic form, K 1=2 T b hT ( 0 ) , is not well de…ned in the presence of a continuum 2 1=2 p b of moment conditions. Its regularized version, K T hT ( 0 ) , exists but diverges as T goes to in…nity and. goes to zero. Indeed, we have. K. 1=2. p. Tb hT ( 0 ). 1=4. K2 + I | {z 1=4. The expansion that we derive for bT ( ). 0. 1=4. .. p. K 1=2 }|. 1. 1=4. = Op. K2 + I }| {z. Tb hT ( 0 ) {z }. (11). =Op (1). is of the same form for both the IID and Markov cases.. Namely:. where. 1. bT ( ). = Op T. 1=2. 0. ;. =. 2. 1. +. = Op. +. 3. min(1; 2. 2. 2. 1. + op 1. ). T. 1=2. T. 1. + op. and. 3. min(1; 2. 1 2. 1T. = Op. ). T. 1. 1=2. (12). . Appendix B provides. details about the above expansion whose validity is ensured by the consistency result of Theorem 1. In deriving the expansion above, we wish to …nd the rate of convergence of the. which minimizes the. leading terms of the MSE: M SE ( ;. 0). = T E T bT ( ). bT ( ). 0. 0. 0. (13). We have the following results on the higher order MSE matrix and on the optimal convergence rate for the regularization parameter. Theorem 2 Assume that Assumptions 1 to 5 hold. Then we have: 1T (i) The approximate MSE matrix of bT ( ) up to order O. 1=2. (henceforth, AMSE) is de-. composed as the sum of the squared bias and variance: AM SE ( ;. 0). = T Bias Bias0 + T V ar. where T Bias Bias0 = O T V ar = I as T ! 1,. 2T. ! 1 and. 2 1 0. T. 1. ; min(2; 2. +O. ! 0.. 8. 1 2. ) +O. 1. T. 1=2. :.

(11) (ii) The. that minimizes the trace of AM SE ( ; T. =O T. 0 ),. max( 16 ; 2. denoted 1 ) +1. T. T. ( 0 ), satis…es:. :. Remarks. 1. We have the usual trade-o¤ between a term that is decreasing in. and another that is increasing. in . Interestingly, the squared bias term is dominated by two higher order variance terms whose rates are equated to obtain the optimal rate for the regularization parameter. The same situation happens for the Limited Information Maximum Likelihood estimator for which the bias is also dominated by variance terms (see Donald and Newey, 2001). 2 1 2. The rate for the O min(2; 2 ) variance term does not improve for. > 2:5. This is due to a. property of Tikhonov regularization that is well documented in the literature on inverse problems, see e.g. Carrasco, Florens and Renault (2007). The use of another regularization such as spectral cut-o¤ or Landweber-Fridman would permit to improve the rate of convergence for large values of . However, this improvement comes at the cost of a greater complexity in the proofs (e.g. in the spectral cut-o¤, we lose the di¤erentiability of the estimator with respect to ). 3. Our expansion is consistent with the condition of Theorem 1, since the optimal regularization parameter. T. satis…es. 2T T. ! 1.. 4. It follows from Theorem 2 that the optimal regularization parameter. T. is necessarily of the. form: T. = c ( 0) T. g( ). ;. (14). for some positive function c ( 0 ) that does not depend on T and a positive function g ( ) that satis…es 1 1 6 ; 2 +1. max. g ( ) < 1=2. An expression of the form (14) is often used as starting point for optimal. bandwidth selection in nonparametric density estimation. Examples in the semiparametric context include Linton (2002) and Jacho-Chavez (2010).. 4. Estimation of the Optimal Regularization parameter. Our purpose is to select the regularization parameter of bT ( ) for a given sample of size T , i.e.: T. where. T. ( ;. 0). = TE. bT ( ). ( 0 ) = arg min. 2[0;1]. 2 0. .. so as to minimize the trace of the MSE matrix. T. ( ;. 0) ;. This raises at least three problems.. First, the MSE. might be in…nite in …nite samples even though bT ( ) is consistent.1 Second, the true parameter value 0 is unknown. Third, the …nite sample distribution of bT ( ) 0 is not known even T. ( ;. 0). 1 This is due to the fact that bT ( ) is a GMM-type estimator. The large sample properties of such estimators are well-known whilst their …nite sample properties can be established only a special cases.. 9.

(12) when. is known. Each of these problems is examined below. The variance of bT ( ) may be in…nite for some data generating processes. To hedge against such 0. situations, one may consider a truncated MSE of the form: T. where. 2. bT ( ). ( ; ). ( ;. 0;. ) = TE [. T. 0) j T. = Pr (. ( ;. T. ( ;. 0). < n ];. (15). > n ). A similar approach has been used in Andrews (1991, p. 826). Given that the …nite sample distribution of bT ( ) is unknown in T. and n satis…es. ( ;. 0). practice, it is convenient to …rst select the probability of truncation. (e.g.,. = 1%) and then deduce. the corresponding quantile n by simulation. To account for the possibility of the pair ( ; n ) depending on , one may consider instead: T. ( ;. 0;. ) = (1. )TE [. T. ( ;. 0) j T. ( ;. 0). < n ] + n T;. (16). which accounts for the probability mass at the truncation boundary. Note that the truncation will play no role if the MSE of bT ( ) is …nite. In this case, we simply let: T. ( ;. 0 ; 0). T. ( ;. 0). = TE [. T. ( ;. 0 )] :. (17). As bT ( ) is asymptotically normal, its second moment exists for large enough T . Hence, the truncation. disappears (i.e., each of the expressions (15) and (16) converges to (17)) if one let goes to. go to zero as T. in…nity.2. We de…ne the optimal regularization parameter as: T. where. T. ( ;. 0;. ( 0 ) = arg min. 2[0;1]. T. ( ;. 0;. );. (18). ) is given by either (15), (16) or (17).. Our strategy for estimating T ( 0 ) relies on approximating the unknown MSE by parametric boot1 strap. Let b be the CGMM estimator of 0 obtained by replacing the covariance operator with the T. 1 identity operator. This estimator is consistent and asymptotically normal albeit ine¢ cient. We use bT (j) 1 to simulate M independent samples of size T , denoted X (b ) for j = 1; 2; :::; M . It should be emphaT. sized that we have adopted a fully parametric approach from the beginning by assuming that the model of interest is fully speci…ed. Indeed, it would not be possible to obtain MLE e¢ ciency otherwise. The model can be simulated by exploiting Assumption 5, which stipulates that the data generating process satis…es xt = r (xt. 1;. (j). ; "t ). To start with, one …rst generates M T IID draws "t. (for j = 1; :::; M and. t = 1; :::; T ) from the known distribution of the errors. Next, M time-series of size T are obtained by 1 (j) (j) (j) (j) applying the recursion xt = r xt 1 ; bT ; "t , t = 1; :::; T , from M arbitrary starting values x0 . Using the simulated samples, one computes M IID copies of the CGMM estimator for any given. 2. As n ! 1 as ! 0, one might be concerned by the limiting behavior of n as However, this is not an issue as long as n is …nite for all …nite T .. 10. ! 0 when. T. ( ;. 0). is in…nite..

(13) j. 1. . We let bT ( ; bT ) denote the CGMM estimator computed from the j th sample. The truncated MSE. given by (15) is estimated by: bTM where. 1. ; bT ;. 1 j 1 ; bT ) = bT ( ; bT ). j;T (. T. =. b1. (1 2. 1. ; bT )1. j;T (. j=1. j;T (. 1. ; bT ). n b. ;. (19). is a probability selected by the econometrician and n b satis…es:. ,. T. )M. M X. M 1 X 1 M. 1. ; bT ). j;T (. j=1. n b. =1. ;. The truncated MSE based on the alternative Formula (16) is estimated by: bTM. 1 ;b ; T. M T X = M. j;T (. j=1. 1. ; bT )1. 1. ; bT ). j;T (. n b. + n b T:. (20). With no truncation, (21) and (19) are identical to the naive MSE estimator given by: M T X = M. 1 ;b. bTM. T. 1. ; bT );. j;T (. j=1. (21). which is aimed at estimating (17). Finally, we select the optimal regularization parameter according to: 1 b T M b = arg min b T M 2[0;1]. 1. ;b ;. where b T M Let. 1. ;b ;. T. and de…ne:. is either (19), (20) or (21). be the limit of b T M T. 1. ;b ;. T. b1. ;. (22). as the number of replications M goes to in…nity. b1 = arg min 2[0;1]. Note that. 1. ;b ;. 1. T. ;b ;. :. 1 is a deterministic function of a stochastic argument while b T M b. random, being a stochastic function of a stochastic argument. The estimator. 1. T. b. is doubly. is not feasible.. However, its properties are the key ingredients for establishing the consistency of its feasible counterpart 1 b T M b . To pursue, we need the following assumption: Assumption 6: The regularization parameter. T. ( ;. 0;. ) is of the form. T. ( 0) = c ( 0) T. g( ) ,. that minimizes (the possibly truncated criterion). for some continuous positive function c ( 0 ) that. does not depend on T and a positive function g ( ) that satis…es max. 1 1 6 ; 2 +1. g ( ) < 1=2.. Basically, Assumption 6 requires that the optimal rate found for the regularization parameter at (14) 11.

(14) be insensitive to the MSE truncation scheme. This assumption ensures that and is necessarily satis…ed as T goes to in…nity and. T. p. g( ). goes to zero. The following result can further be. proved. 1 Theorem 3 Let b be a. b1 = c b1 T. T consistent estimator of. 0.. Then under Assumptions 1 to 5,. converges in probability to zero as T goes to in…nity.. b1 ) ( T 0) T(. 1. 1 In Theorem 3, the function T (:) is deterministic and continuous but the argument b is stochastic. 1 As T goes to in…nity, b gets closer and closer to 0 ; but at the same time T ( 0 ) converges to zero at. some rate that depends on T . This prevents us from claiming without caution that. b1 ) ( T 0) T(. 1 = op (1). since the denominator is not bounded away from zero. The next theorem characterizes the rate of convergence of. bT M ( 0 ) . T ( 0). Theorem 4 Under assumptions 1 to 5, M goes to in…nity and T is …xed.. bT M ( 0 ) T ( 0). 1=2. 1 converges in probability to zero at rate M. In Theorem 4, b T M ( 0 ) is the minimum of the empirical MSE simulated with the true. 0.. as. In the. proof, one …rst shows that the conditions of the uniform convergence in probability of the empirical MSE are satis…ed. Next, one uses Theorem 2.1 of Newey and McFadden (1994) and the fact that T ( 0). is bounded away from zero for any …nite T to establish the consistency of. theorem, we revisit the previous results when 1 Theorem 5 Let b be a. Op (T. 1=2 ). + Op (M. 1=2 ). p. 0. bT M ( 0 ) . T ( 0) 1. is replaced by a consistent estimator b .. T consistent estimator of. 0.. Then under assumptions 1 to 5,. as M goes to in…nity …rst and T goes to in…nity second.. In the next. 1 b T M (b ) T ( 0). 1=. The result of Theorem 5 is obtained by using a sequential limit in M and T , which is needed here because Theorem 4 has been derived for …xed T . Such sequential approach is often used in panel data econometrics, see for instance Phillips and Moon (1999). It is also used implicitly in the theoretical 1 analysis of bootstrap.3 Theorem 5 implies that b T M (b ) bene…ts from an increase in both M and T .. The last theorem compares the feasible CGMM estimator based on b T M to the unfeasible estimator b ( T ), where T is de…ned in (14). 1 Theorem 6 Let b T M = b T M (b ) de…ned in (22). Then:. p. provided that M. T.. T b (b T M ). b(. T). = Op (T. g( ). );. 3 The properties of a bootstrap estimator are usually derived using its bootstrap distribution, hence letting M go to in…nity before T .. 12.

(15) p Hence, theorem 6 implies that the distribution of T b (b T M ) p g( ) . This ensures that replacing of T b ( T ) 0 to the order T such that. is the same as the distribution. 0 T. bT M. by a consistent estimator b T M. 1 = op (1) does not a¤ect the consistency, asymptotic normality and e¢ ciency of the …nal CGMM estimator b (b T M ). The proof of this theorem relies mainly on the fact that b ( ) is T. continuously di¤erentiable with respect to. whilst the optimal. T. is bounded away from zero for any. …nite T . Overall, our selection procedure for the regularization parameter is optimal and adaptive as it does not require the a priori knowledge of the regularity parameter .. 5. Monte Carlo Simulations. The aim of this simulation study is to investigate the properties of the MSE function b T M. 1. ; bT ;. as the regularization parameter ( ), the sample size (T ) and the number of replications (M ) vary. For this purpose, we consider estimating the parameters of a square-root di¤usion (also known as the CIR di¤usion) by CGMM. Below, the …rst subsection describes the simulation design whilst the second subsection presents the simulation results.. 5.1. Simulation Design. A continuous time process rt is said to follows a CIR di¤usion if it obeys the following stochastic di¤erential equation: drt = where the parameter mean and. (. rt ) dt +. p. rt dWt. (23). > 0 is the strength of the mean reversion in the process,. > 0 is the long run. > 0 controls the volatility of rt . This model has been widely used in the asset pricing. literature, see e.g Heston (1993) or Singleton (2001). It is shown in Feller (1951) that Equation (23) 2. admits a unique and positive fundamental solution if. 2. .. We assume that rt is observed at regularly spaced discrete times t1 ; t2 ; ..., tT such that ti ti The conditional distribution of rt given rt. 1. =. .. is a noncentered chi-square with possibly fractional order.. Its transition density is a Bessel function of type I, which can be represented as an in…nite mixture of Gamma densities with Poisson weights: f (rt jrt. where c =. 2 2 (1. e. ,q= ). 2 2. )=. and pj =. 1 X. pj. j=0. (ce. rt. rtj+qk 1 cj+q exp ( crt ) (j + q) j. ). exp( ce j!. rt. ). : To implement a likelihood based. inference for this model, one has to truncate the expression of f (rt jrt. ). However, the conditional. CF of rt has a simple closed form expression given by: 't (s; ). E eisrt jrt. =. 1. 13. is c. q. exp. ise 1. Vt is c. 1. !. (24).

(16) = ( ; ; )0 .. with. To start, we simulate one sample of size T from the CIR process assuming of. = 1 and the true value. is: =(. 0. 0;. 0;. 0). = (0:4; 6:0; 0:3) :. These parameter values are taken from Singleton (2001). We refer the reader to DeVroye (1986) and Zhou (2001) for details on how to simulate a CIR process. We treat this simulated sample as the actual 1 data available to the econometrician and use it to estimate the …rst step CGMM estimator b as: T. where b hT ( ; ) =. 1 T 1. b1 = arg min T. PT. ei. i=2. Z. R2. b hT ( ; )b hT ( ; )e. 't (s; ) ei. 1 rti. Next, we simulate M samples of size T using. 2 rti 1. ,. = ( 1;. b1 T. 0. 1). d ;. 2 R2 and 't (s; ) is given by (24).. as pseudo-true parameter value. Each simulated samples is used to compute the second step CGMM estimator bT;j ( ) as: bT;j ( ) = arg min. Z. K. R2. 1b T hT (. ; ) b hT ( ; )e. 0. d. (25). The objective function (25) is evaluated using a Gauss-Hermite quadrature with ten points. The regularization parameter 2 [10. is selected on a thirty points grid that lies between 10 10. ; 2:5. 10. 10. ;5. 10. 10. ; 7:5. 10. 10. ;1. 10. 9. ; :::; 1. 10. 10. 2. and 10. 2,. that is:. ]. For each in this grid, we compute the MSE using Equation (21) (i.e., no truncation of the distribution 1 2 of bT;j ( ) bT ).. 5.2. Simulations results. Table 1 shows the simulations T = 251, 501, 751 and 1001 for two di¤erent values of M . For a given sample size T , the scenarios with M = 500 and M = 1000 use common random numbers (i.e., the results for M = 500 are based on the …rst 500 replications of the scenarios with M = 1000). 1 Curiously enough, the estimate of b T M (b ) is consistently equal to 2:5 10 6 across all scenarios. except (T = 251; M = 1000) and (T = 1001; M = 500). This result might be suggesting that the grid on which 2:5. 10. 6. is selected is not re…ned enough. Indeed, the value that is immediately smaller than. on that grid is 1:0. 10. 6,. which is selected for the scenario (T = 1001; M = 500). Arguably,. the results suggest that the rate of convergence of generating process. Overall, 2:5. 10. 6. T ( 0). to zero is quite slow for this particular data. seems a reasonable choice for the regularization parameter for. all sample sizes for this data generating process. Note that our simulations results do not allow us to infer the behavior of. T ( 0). as. 0. vary in the parameter space.. Figure 1 presents the simulated MSE curves. For all eight scenarios, these curves are convex and. 14.

(17) have one minimum. The hump-shaped left tail of the MSE curves for T = 251 stems to the fact that the approximating matrix of the covariance operator (see Appendix D) is severely ill-posed. Hence, the shape of the MSE curve re‡ects the distortions in‡icted to the eigenvalues of the regularized inverse of this approximating matrix as. varies. A smaller number of quadrature points should be used for. smaller sample sizes in order to mitigate this ill-posedness and obtain perfectly convex MSE curves. Table 1: Estimation of T for di¤erent sample size. M = 500 M = 1000 1 1 b T M (b ) 1 b T M b T M (b ) 1 b T M T. T. T = 251. 2:5. 10. 6. 0:0270. 7:5. 10. 7. 0:0289. T = 501. 2:5. 10. 6. 0:0114. 2:5. 10. 6. 0:0119. 0:0066. 2:5. 10. 6. 0:0065. 0:0057. 2:5. 10. 6. 0:0053. T = 751. 2:5. 10. 6. T = 1001. 1:0. 10. 6. Figure 1: MSE curves of the CGMM estimator for di¤erent M and T . 1 PM b1 The vertical axis shows T1 b T M = M j=1 j;T ( ; T ) and the horizontal axis is scaled as Sample size: T = 251. log 10. :. Sample size: T = 501. 0.034. 0.014. M=500 reps M=1000 reps. 0.033. M=500 reps M=1000 reps. 0.0135 0.032 Mean square error. Mean square error. 0.013 0.031 0.03 0.029. 0.0125. 0.012. 0.028 0.0115 0.027 0.026 -2.5. -2. -1.5. -1. -0.5. 0.011 -2.5. 0. -2. -1.5. log(α )/10 -3. 7.8. Sample size: T = 751. x 10. -3. 6.8. M=500 reps M=1000 reps. 7.6. -0.5. 0. Sample size: T = 1001. x 10. M=500 reps M=1000 reps. 6.6 6.4 Mean square error. 7.4 Mean square error. -1 log(α )/10. 7.2. 7. 6.2 6 5.8. 6.8 5.6. 6.6. 6.4 -2.5. 5.4. -2. -1.5. -1. -0.5. 0. -2.5. log(α )/10. 6. -2. -1.5. -1. -0.5. 0. log(α )/10. Conclusion. The objective of this paper is to provide a method to optimally select the regularization parameter denoted. in the CGMM estimation. First, we derive a higher order expansion of the CGMM estimator. that sheds light on how the …nite sample MSE depends on the regularization parameter. We obtain the 15.

(18) convergence rate for the optimal regularization parameter variance terms. We …nd an expression of the form the sample size T and 0. g( ). 1=2, where. T. T. by equating the rates of two higher order. = c ( 0) T. g( ) ,. where c ( 0 ) does not depend of. is the regularity of the moment function with respect. to the covariance operator (see Assumption 4). Next, we propose an estimation procedure for. T. that relies on the minimization of an approximate. MSE criterion obtained by Monte Carlo simulations. The proposed estimator, b T M , is indexed by the sample size T and the number of Monte Carlo replications M . To hedge against situations where the MSE is not …nite, we propose to base the selection of. T. on a truncated MSE that is always …nite.. Under the assumption that the truncation scheme does not alter the rate of T. T,. b T M is consistent for. as T and M increase to in…nity. Our simulation-based selection procedure has the advantage to be. easily applicable to other estimators, for instance it could be used to select the number of polynomial. terms in the e¢ cient method of moments procedure of Gallant and Tauchen (1996). The optimal selection of the regularization parameter permits to devise a fully feasible CGMM estimator that is a real alternative to the maximum likelihood estimator.. 16.

(19) References [1] Ait-Sahalia, Y. and R. Kimmel (2007), “Maximum likelihood estimation of stochastic volatility models,” Journal of Financial Economics 83, 413–452. [2] Andews, D. W. K. (1991), "Heteroskedasticity and Autocorrelation Consistent Covariance Matrix Estimation," Econometrica, 59(3): 817-858 [3] Buse, A. (1992), “The Bias of Instrumental Variable Estimators”, Econometrica 60:1, 173–180. [4] Carrasco, M. (2012), “A regularization approach to the many instruments problem,” Journal of Econometrics, 170:2, 383–398. [5] Carrasco, M., Chernov, M., Florens J. P. and E. Ghysels (2007), “E¢ cient estimation of general dynamic models with a continuum of moment conditions,”Journal of Econometrics, 140, 529–573. [6] Carrasco, M. and J. P. Florens (2000), “Generalization of GMM to a continuum of moment conditions,” Econometric Theory, 16, 797-834. [7] Carrasco, M., J. P. Florens, and E. Renault (2007), “Linear Inverse Problems in Structural Econometrics: Estimation based on spectral decomposition and regularization,” in the Handbook of Econometrics, Vol. 6, edited by J.J. Heckman and E.E. Leamer. [8] Chacko, G. and L. Viceira (2003), “Spectral GMM estimation of continuous-time processes,”Journal of Econometrics, 116, 259-292. [9] Devroye, L. (1986), “Non-Uniform Random Variate Generation,” Spinger-Verlag. [10] Dominguez, M. A. and I. N. Lobato (2004), “Consistent Estimation of Models De…ned by Conditional Moment Restrictions,” Econometrica 72:5, 1601-1615. [11] Donald, S. and W. Newey (2001), “Choosing the number of instruments,”Econometrica, 69, 11611191. [12] Du¢ e, D. and K. Singleton (1993), “Simulated Moments Estimation of Markov Models of Asset Prices,” Econometrica, 61, 929-952. [13] Feller, W. (1951), “Two Singular Di¤usion Problems,” Annals of Mathematics, 54, 173-182. [14] Feuerverger, A. (1990), “An e¢ ciency result for the empirical characteristic function in stationary time-series models,” Canadian Journal of Statistics, 18, 155-161. [15] Feuerverger, A. and P. McDunnough (1981a), “On E¢ cient Inference in Symmetry Stable Laws and Processes,”Csorgo, M., ed. Statistics and Related Topics. New York: North Holland, 109-122. [16] Feuerverger, A. and P. McDunnough (1981b), “On some Fourier Methods for Inference,” J. R. Statist. Assoc. 76:379-387. 17.

(20) [17] Feuerverger, A. and P. McDunnough (1981c), “On the E¢ ciency of Empirical Characteristic Function Procedures,” J. R. Statist. Soc. B, 43, 20-27. [18] Feuerverger, A. and R. Mureika (1977), “The empirical characteristic function and its applications,” The annals of statistics, 5, No 1, 88-97. [19] Gallant, A.R. and G. Tauchen (1996), “Which Moments to Match?” Econometric Theory, 12, 657-681. [20] Gouriéroux, C. and A. Monfort (1996), “Simulation Based Econometric Methods,” CORE Lectures, Oxford University Press, New York. [21] Gouriéroux, C., A. Monfort and E. Renault (1993), “Indirect Inference,”Journal of Applied Econometrics 8, S85-S118 . [22] Hansen, L. (1982), “Large Sample Properties of Generalized Method of Moments Estimators,” Econometrica, 50, 1029-1054. [23] Heston, S. (1993), “A Closed-Form Solution for Options with Stochastic Volatility with Applications to Bond and Currency Options,” The Review of Financial Studies, 6:2, 327-343. [24] Jacho-Chavez, D. T. (2010), “Optimal Bandwidth Choice for Estimation of Inverse Conditional– Density–Weighted Expectations,” Econometric Theory, 26, 94-118. [25] Jiang, G. and J. Knight (2002), “Estimation of Continuous Time Processes via the Empirical Characteristic Function,” Journal of Business and Economic Statistics, 20, 198-212. [26] Koenker, R., Machado, J. A. F., Skeels, C. L. and A. H. I. Welsh (1994), “Momentary Lapses: Moment Expansions and the Robustness of Minimum Distance Estimators,”Econometric Theory, 10, 172-197. [27] Koutrouvelis, I. A. (1980), “Regression-type estimation of the parameters of stable laws,” J. Am. Statist. Assoc. 75:372, 918–928. [28] Lavergne, P., and V. Patilea (2008), “Smooth minimum distance estimation and testing with conditional estimating equations: uniform in bandwidth theory,” Working paper available at ideas.repec.org. [29] Linton, O. (2002) “Edgeworth Approximation for Semiparametric Instrumental Variable and Test Statistics,” Journal of Econometrics, 106, 325-368. [30] Liu, Q. and D. A. Pierce (1994), “A Note on Gauss-Hermite Quadrature,” Biometrika, 81:3, 624629. [31] Madan, D. B. and E. Senata (1990), “The Variance Gamma Model for Share Market Returns,” Journal of Business, 63:4, 511-524. 18.

(21) [32] Nagar, A. L. (1959): “The Bias and Moment Matrix of the General k-Class Estimators of the Parameters in Simultaneous Equations,” Econometrica, 27, 573-595. [33] Newey, W. K. and D. McFadden (1994), “Large Sample Estimation and Hypotheses Testing,” Handbook of Econometrics, Vol IV, ed. by R.F. Engle and D.L. McFadden. [34] Newey, W K. and R. J. Smith (2004), “Higher Order Properties of GMM and Generalized Empirical Likelihood Estimators,” Econometrica, 72:1, 219–255. [35] Nolan, J. P. (2009), “Stable Distributions: Models for Heavy Tailed Data,” Birkhauser, Boston,. In preparation. [36] Paulson, A. S., Holcomb W. E. and R. A. Leitch (1975), “The estimation of the parameters of the stable laws,” Biometrika, 62:1, 163-170. [37] Phillips, P. C. B. and H. R. Moon (1999), “Linear Regression Limit Theory for Nonstationary Panel Data,” Econometrica, 67:5, 1057–1112. [38] Rilstone, P., Srivastava, V. K., and A. Ullah (1996), “The Second-Order Bias and Mean-Squared Error of Nonlinear Estimators,” Journal of Econometrics, 75, 369-395. [39] Rothenberg, T. J. (1983), “Asymptotic properties of Some Estimators in Structural Models,” Studies in Econometrics, Time Series and Multivariate Statistics, edited by S. Karlin, T. Amemiya and L. A. Goodman, New York: Academic Press. [40] Rothenberg, T. J. (1984), “Approximating the Distributions of Econometric Estimators and Test Statistics,” Handbook of Econometrics, Vol. 2, ed. by Z. Griliches and M. D. Intriligator. New York: North-Holland. [41] Singleton, K. (2001), “Estimation of A¢ ne Pricing Models Using the Empirical Characteristic Function,” Journal of Econometrics, 102, 111-141. [42] Yu, J. (2004), “Empirical Characteristic Function Estimation and Its Applications,” Econometric Reviews, 23:2, 93-123. [43] Zhou, MLE. H. for. (2001), a. “Finite. Square-Root. Sample Interest. Properties Rate. of. Di¤usion. EMM,. GMM,. Model,”. Mimeo,. http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.26.8099.. 19. QMLE, available. and at.

(22) Appendix A. Some basic properties of the covariance operator. For more formal proofs of the results mentioned in this appendix, see Carrasco, Florens and Renault (2007). Let K be the covariance operator de…ned in (5) and (6), and b ht ( ; ) the moment function be the subset of L2 ( ) de…ned in Assumption 4.. de…ned in (1) and (2). Finally, let. De…nition 7 The range of K denoted R(K) is the set of functions g such that Kf = g for some f in L2 ( ). Proposition 8 R(K) is a subspace of L2 ( ). Note that the kernel functions k(s; :) and k(:; r) are elements of L2 ( ) because h i jk(s; r)j2 = E ht ( ; s)ht ( ; r). 2. 4; 8 (s; r) 2 R2p. (1). Thus for any f 2 L2 ( ), we have jKf (s)j. 2. =. Z. 2. k(s; r)f (r) (r) dr. Z. jk(s; r)f (r)j2 (r) dr. 4. Z. jf (r)j2 (r) dr < 1;. kKf k =. Z. jKf (s)j2 (s) ds < 1 ) Kf 2 L2 ( ) :. implying 2. De…nition 9 The null space of K denoted N (K) is the set of functions f in L2 ( ) such that Kf = 0. The covariance operator K associated with a moment function based on the CF is such that N (K) = f0g. See CCFG (2007). De…nition 10. is an eigenfunction of K associated with eigenvalue. Proposition 11 Suppose satis…es: (i). j. 1. ::::. 2. > 0 for all j, (ii). 1. if and only if K =. .. ::: are the eigenvalues of K. Then the sequence. j. < 1 and lim. j. j!1. j. = 0.. Remark. The covariance operator associated with the CF-based moment function is necessarily compact. Proposition 12 Every f 2 L2 ( ) can be decomposed as: f = As a consequence, Kf =. P1. j=1. f;. j. K. j. =. P1. j=1. 20. f;. j. P1. j=1. j j.. f;. j. j..

(23) Proposition 13 If 0 < We recall that exist and K. 2. K. 2. f. 1. 2;. 1. then. 2. 1. .. is the set of functions such that K f < 1. In fact, f 2 R(K 2 ) ) K P 2 2 2 < 1: Thus if f 2 R(K 2 ), we have: f; j = 1 j=1 j 2. f. =. 1 X. 2( j. 1). 2. 2. 2. 2. f;. j. 2( 1. j. 1). 2. j=1. )K. 1. on a wider subset of when f 2. for 1=2. 2. 2. j. f;. 2 j. j=1. 1f .. ) compared to K. 1=2 f; K 1=2 f. When f 2. < 1, the quadratic form K. 1, K 1=2 f; K 1=2 f. f. <1. R(K 1=2 ) so that the function K. f exist ) f 2 R(K 1 ). This means R(K) L2 (. 1 X. 2. 1=2 f. = K. is de…ned. 1 f; f. is well de…ned while K. . But. 1 f; f. is. not.. B. Expansion of the MSE and proofs of Theorems 1 and 2. B.1. Preliminary results and proof of Theorem 1 1. Lemma 14 Let K. = (K 2 + I). 1K. zero and n goes to in…nity, we have: K. and assume that f 2. 1 T. K. 1. = Op. 3=2. K. 1 T. K. 1. f. = Op. 1. K. 1. K. 1. f. = O. min(1;. K. 1. 1. K. Proof of Lemma 14. Subsequently,. f; f j; j. where kKT. |. (KT2 + {z. 1K (K 2 + I) 1 K T I) 1 (KT K) + (KT2 + I) 1 kKT Kk + (KT2 +. 1. Kk = Op T. 1=2. }|. {z =Op (T. = |. (KT2. 1. + I). 1. + I) {z. 1. 1. 1 2 2 2. ; ;. (3). ). ;. (4). 1. ) :. (5). j; j. = 1; 2:::; 1. We …rst consider (2). By the. I). 1K. } 1=2 ). K }|. (K 2 + I). 1. 2. 2. K. KT2 2. {z =Op (T. goes to. (2). I). 1. (K 2 + I) (K 2 + I). 1K 1. K ;. follows from Proposition 3.3 (i) of CCFG (2007). We have:. (KT2 + I) (KT2. 1=2. T. > 1. Then as. = 1; 2:::; 1 denote the eigenfunctions of the covariance. triangular inequality: (KT2 +. 1=2. T. min(1;. = O. operator K associated respectively with the eigenvalues (KT2 + I). for some. K. (K + I). K. 2. KT2 1=2. 1. ). }|. (K + I) {z. 21. 1=2. 1=2. (K 2 + I) }| {z 1. 1=2. K }.

(24) This proves (2). K. The di¤erence between (2) and (3) is that in (3) we exploit the fact that f 2 1f. with. > 1, hence. < 1. We can rewrite (3) as K. 1 T. K. 1. f =. 1 T. K. 1 T. K. 1. K. 1. KK. f. 1 T. K. K. 1. K. 1. K. f :. We have K. 1. K = (KT2 + I). 1. KT K. = (KT2 + I). 1. (KT. (KT2. +. + I). (K 2 + I). 1. K2. K) K. 1. (6). 2. (K + I). 1. 2. K :. (7). The term (6) can be bounded in the following manner (KT2 + I). 1. (KT. K) K. For the term (7), we use the fact that A (KT2 + I). 1. (KT2 + I). 1. (KT2 + I) | {z. 1. =. 1. This proves (3).. |. 1=2. 1. 1=2. 1=2. B. 1. =A. 1=2. T. kKT Kk kKk }| {z } =Op (T 1=2 ) :. B 1=2. A1=2 B. 1=2 :. It follows that. K2. KT2 (K 2 + I). K2. 1. 1. = Op. (K 2 + I) K2. (KT2 + I) {z. 1. K2. KT2. (K 2 + I) }| {z }| {z 1 =Op (T 1=2 ). 1. K 2 = Op }. 1. T. 1=2. :. Now we turn our attention toward equation (4). We can write 2. (K + I). 1. Kf. K. 1. f=. 1 X j=1. ". 1. j 2 j. +. j. #. f;. j. j. =. 1 X. 2 j. +. j=1. 1. 2 j. !. f;. j. j:. j. We now take the norm:. (4) =. (K 2 + I) 0. = @. 1 X j=1. 2 j. 2. 1. Kf. K 2 j. +. 2 j. 0. 1. f =@. !2. 1. 1 X. 2 j. +. j=1. f;. 2 j. 2 j. 11=2 A. 22. 1. 2 j. 0 @. !2. 1 X j=1. f;. 2 j 2 j 2. f;. j 2 j. 11=2 A. 11=2 A. sup 1 j 1. 1 j. +. 2: j.

(25) Recall that as K is a compact operator, its largest eigenvalue. 1. is bounded. We need to …nd an. equivalent to 1. sup 0. Case where 1. +. 1. 1. = sup. 2 j. 1 = +1. 1. 2 2 1. 0. 3: We apply another change of variables x =. (8) =2 1=2. = : sup x. =2 1=2. x 0. =2 1=2. equivalent to (8) is. x 1+x. 1. provided that. =2 1=2. x. x 1+x. is bounded on R+ . Note that g (x). . An x(3 )=2 1+x. is continuous and therefore bounded on any interval of (0; +1). It goes to 0 at +1 and its limit at 0 also equals 0 for 1. < 3. For. 1 1+x .. = 3, we have: g (x). Then g (x) goes to 1 at x = 0 and to 0. at +1. Case where. > 3: We rewrite the left hand side of (8) as 1 j. 2 j. +. 2 j. 3. =. j. 3 1. 2 j. + | {z }. = O( ):. 2(0;1). To summarize, we have for f 2. 1. min(1;. : (4)= O. 2. ). :. Finally, we consider (5). We have: (5) =. X. 1. =. X. 2 j. j. j. 2 j. 1. +. 2 j. 2. variables x = =. min(1;. 1. 2. 1 2. ). =. j. !. +. 1. and obtain supx. R+ . Finally: (5)= O Lemma 15. 2. 3=2, we have: sup. 2. f;. 2 j. 1. j. For. !. j. f;. j. 2 1 2. 3. 2. =O. 1 2. f;. 2 j j. 2 j. sup. 2 j. 2. 1 2. 1. :. +. < 3=2, we apply the change of. , as f (x) =. 2. x 1+x x. 1 2. such that sup kfT ( ). f ( )k = Op (T. for some. 1=2 ):. Then as. > 1, and a sequence of goes to zero, we have. 2. sup K 2. 1=2 T fT (. ). K. 1=2. f ( ) = Op (. 1. T. 1=2. )+O. min(1;. 1 2. ). .. Proof of Lemma 15. sup K 2. 1 T fT (. ). K. 1. f( ). B1 + B2 ;. with B1 = sup K 2. 1 T fT (. is bounded on. :. Suppose we have a particular function f ( ) 2. functions fT ( ) 2. +. = O ( ). For. 1 2. x. 2 j. X f;. j 2 j. !. 2 j. 1. j 2. 2+. x 0 1+x. X. ). K. 1 T f(. ). and B2 = sup 2. 23. K. 1 T. K. 1. f( ) :.

(26) We have 1 T. K. B1 |. sup kfT ( ) 2. + KT2 {z. T. 1=2. 1=2. T. = Op (. 1=2. T. T. f ( )k. 1=2. ):. T. }|. 1=2. + KT2 {z. 1. KT sup kfT ( ) }|2 {z =Op (T. f ( )k }. 1=2 ). On the other hand, Lemma 14 implies that: K. B2 =. K = Op. 1 T 1 T 1. K. 1. f( ). K. 1. f( ) +. 1=2. T. K. min(1;. +O. 1. K 1. 2. ). 1. f( ). :. Hence, B1 is negligible with respect to B2 and the result follows. Lemma 16 For all nonrandom functions (u; v) ; we have: E. D. E ED 1 b b u; hT (:; ) v; hT (:; ) = hu; Kvi T. Proof of Lemma 16. We have: E. D. u; b hT (:; ). E ED v; b hT (:; ). = E. Z. Z Z. u( )b hT ( ; ) ( ) d. Z. v ( )b hT ( ; ) ( ) d. b = E hT ( 1 ; )b hT ( 2 ; )u ( 1 ) v ( 2 ) ( 1 ) ( 2 ) d 1 d 2 Z Z h i = E b hT ( 1 ; )b hT ( 2 ; ) u ( 1 ) v ( 2 ) ( 1 ) ( 2 ) d 1 d 2 :. Because the ht s are uncorrelated, we have:. we have. i i h 1 h 1 hT ( 1 ; )b hT ( 2 ; ) = E ht ( 1 ; )ht ( 2 ; ) = k ( 1 ; E b T T ED E u; b hT (:; ) v; b hT (:; ) Z Z 1 k ( 1 ; 2 )v ( 2 ) ( 2 ) d T | {z. E =. D. Kv(. 1). 24. u ( 1) ( 1) d. 2. }. 1. =. 2) ;. 1 hu; Kvi : T.

(27) Lemma 17 Let S be a neighborhood of b, such that e D. b T (:; ) = and G. @b hT (:; ) . @. 1b e b e T GT (:; ); hT (:; ). Proof of Lemma 17. Note that S contains e. 0. =. E. 1b b b b T GT (:; ); hT (:; ). K. We have: D Im K. b = Op (T. E 0. and:. b hT (:; e) = b hT (:;. 0). b T (:; +G. 0). 0. e. b T (:; e) around Likewise, a …rst order Taylor expansion of G 0) +. q X j=1. b j;T (:; H. 0). for all e 2 S, where b solves:. for all e 2 S:. e b + b = Op T | {z } | {z }0 Op (T 1=2 ) Op (T 1=2 ). Hence, a …rst order Taylor expansion of b hT (:; e) around. b T (:; e) = G b T (:; G. =0. 1. = Op T. 1=2 ). 1=2. :. yields: 0. 0. + Op T. 1. :. yields:. ej. j;0. + Op T. 1. :. Hence, we have: D K. E. 1b e b e T GT (:; ); hT (:; ). D Note that the term K. …xed ):. D 0= K. E b T (:; 0 ); b hT (:; 0 ) K T1 G E D b T (:; 0 ); G b T (:; 0 ) e + K T1 G. D. =. E. 1b b T GT (:; 0 ); GT (:; 0 ). E. 1b b T GT (:; 0 ); hT (:; 0 ). D + K. D b T (:; Hence, the imaginary part of K T1 G D E b T (:; e); b K T1 G hT (:; e) for all e 2 S:. e. 0. + Op T. 1. :. is real. At the particular point e = b (and for. 1b b T GT (:; 0 ); GT (:; 0 ). b 0 ); hT (:;. 0. E ) is Op T 0. 1. E. b. 0. + Op T. 1. :. , and so is the imaginary part of. Proof of Theorem 1. The proof follows the same steps as that of Proposition 3.2 in CCFG. (2007). However, we now exploit the fact Er ht ( ) 2 Lemma 15 provided. T 1=2. ! 1 and. with. 1. The consistency follows from. ! 0. For the asymptotic normality to hold, we need to …nd a. 25.

(28) bound for the term B.10 of CCFG (2007). We have: D. jB:10j =. 1 Tr. K. K. ^ T ^T h. 1=2 T r. E ^ T ( 0) Th p ^ T ( 0) ^ T ( 0) K 1=2 E r h Th | {z } 1. K. ^ T ^T h. ^ T ( 0) ; E r h. p. =Op (1). 1=2. = Op. T. 1=2. min(1;. +O. 1. 2. ). :. Hence the asymptotic normality requires the same conditions as the consistency, that is, T 1=2 ! 1 and 1 T. ! 0. The asymptotic e¢ ciency follows from the fact that K. B.2. !K. 1. under the same conditions.. Stochastic expansion of the CGMM estimator: IID case. The objective function is. where b hT ( ; ) =. 1 T. PT. n b = arg min Q ei. t=1. 0x. T. D ( )= K. ); b hT (:; ). '( ; ) . The optimal b solves:. t. @Q. T. @. b. D = 2 Re K. @'( ; ) . @. where G(:; ) =. 1b T hT (:;. 1 b b b T G(:; ); hT (:; ). E. Eo. :. =0. (9). A third order expansion gives 0=. where. @Q. T. ( 0). @. lies between b and. Gj (:; ) =. @2Q T ( 0) b + @ @ 0 0.. 0. +. q X j=1. The dependence of b on. @'( ; ) ; H(:; ) = @ j. bj T. j;0. @3Q T @ j@ @. 0. b. 0. is hidden for convenience. Let us de…ne. @ 2 '( ; ) ; Hj (:; ) = @ @ 0. @ 2 '( ; ) ; Lj = @ @ j. @ 3 '( ; ) : @ j@ @ 0. and T ( 0). D = Re K. WT ( 0 ) =. K. 1 b T G(:; 0 ); hT (:; 0 ). 1 T G(:; 0 ); G(:; 0 ). E. ;. D + Re K. E. 1 b T H(:; 0 ); hT (:; 0 ). ; E Bj;T ( ) = 2 Re K T1 G(:; ); Hj (:; ) + Re K T1 Lj (:; ); b hT (:; ) + Re K. 1 T H(:;. ); Gj (:; ) :. 26. ;. D.

(29) Then we can write: 0=. b T ( 0 ) + WT ( 0 ). +. 0. q X j=1. bj. j;0. Bj;T ( ) b. 0. :. Note that the derivatives of the moment functions are deterministic in the IID case. We decompose T ( 0 ),. WT ( 0 ) and Bj;T ( ) as follows: T ( 0). =. T;0 ( 0 ). +. T;. where. ( 0 ) + e T; ( 0 );. D E 1 1=2 b ( ) = Re K G; h T;0 0 T = Op T D E 1 1 b ( ) = Re K K G; h 0 T; T = Op E D e T; ( 0 ) = Re K 1 K 1 G; b h T = Op T. 1. min(1; 1. T. 2. ). 1=2. T. 1. where the rates of convergence are obtained using the Cauchy-Schwarz inequality and the results of Lemma 14. Similarly, we decompose WT ( 0 ) into various terms with distinct rates of convergence:. where. f ( 0 ) + WT;0 ( 0 ) + W fT; ( 0 ); WT ( 0 ) = W0 ( 0 ) + W ( 0 ) + W W0 ( 0 ) = W ( 0) =. K. 1. G; G = O(1); 1. K. K. 1. G; G = O. min(1; 2. 1 2. ). ;. f ( 0) = W. 1 K T1 K 1 G; G = Op T 1=2 ; D E WT;0 ( 0 ) = Re K 1 H(:; 0 ); b hT (:; 0 ) = Op T 1=2 ; E D fT; ( 0 ) = Re K 1 K 1 H(:; 0 ); b h (:; ) = Op W 0 T T. 1. T. 1. :. We consider a simpler decomposition for Bj;T ( ):. Bj;T ( ) = Bj ( ) + Bj;T ( ). Bj ( ). where Bj ( ) = 2 Re K. 1. Bj;T ( ) = Bj ( ) + O. G(:; ); Hj (:; ) + Re K min(1;. 1 2. ). + Op. 27. 1. 1. T. H(:; ); Gj (:; ) = O(1); 1=2. :.

(30) By replacing these decompositions into the expansion of the FOC, we can solve for b b. W0 1 ( 0 ). =. 0. W0 1 ( 0 ). h. T;0 ( 0 ). ( 0) + W ( 0) b. T;. W0 1 ( 0 )WT;0 ( 0 ) b q X. bj. j=1 q X. bj. j=1. To complete the expansion, we replace b b. 0. =. 1. 0. 0. Bj ( )) b. W0 1 ( 0 )(Bj;T ( ). j;0. W0 1 ( 0 ). 0. by. +. 2. +. i. 0. W0 1 ( 0 )Bj ( ) b. j;0. 3. +. 4. T;0 ( 0 ). +. to obtain:. i. 0. h f ( 0) b W0 1 ( 0 ) e T; ( 0 ) + W. 0. 5. :. 0. in the higher order terms:. b + R;. b is a remainder that goes to zero faster than the following terms: where R 1. =. W0 1 ( 0 ). 2. =. W0 1 ( 0 ). 3. =. W0 1 ( 0 ) e T; ( 0 ). 4. T;0 ( 0 );. h. T;. W ( 0 )W0 1 ( 0 ). ; i f ( 0 )W 1 ( 0 ) T;0 ( 0 ) ; W 0. ( 0). T;0 ( 0 ). = W0 1 ( 0 )WT;0 ( 0 )W0 1 ( 0 ) T;0 ( 0 ) q X W0 1 ( 0 ) T;0 ( 0 ) j W0 1 ( 0 )Bj ( )W0 1 ( 0 ). T;0 ( 0 );. j=1. 5. q X. =. W0 1 ( 0 ). T;0 ( 0 ) j. W0 1 ( 0 )(Bj;T ( ). Bj ( ))W0 1 ( 0 ). T;0 ( 0 ):. j=1. To obtain the rates of these terms, we use the fact that jAf j 1. = Op T. 5. = O. 1=2. min(1;. ; 1 2. 2 ). min(1; 2. = Op. T. 1. 1. +. 1. + Op. 1 2. T. ). 1=2. T. 3=2. ,. kAk jf j. This yields immediately: 3. = Op. 1. T. 1. ;. 4. = Op T. 1. ,. :. To summarize, we have: b. 0. =. 2. +. 3. + op. 1. 28. T. 1. + op. min(1;. 1 2. ). T. 1=2. :. (10).

(31) B.3. Stochastic expansion of the CGMM estimator: Markov case. The objective function here is given by:. where b hT ( ; ) =. 1 T. n b = arg min Q. PT. t=1. 0. eis xt+1 @Q. T. @ bT ( ; ) = where G. 1 T. PT. t=1. T. D ( )= K 0. '(s; ; xt ) eir xt and. b. D = 2 Re K. @'(s; ;xt ) ir0 xt e . @. where. @Q. T. ( 0). @. lies between b and. +. @2Q T ( 0) b ( @ @ 0. ); b hT (:; ). E. 0) +. 0. Eo. :. = (s; r) 2 R2p . The optimal b solves. 1b b b b T GT (:; ); hT (:; ). The third order Taylor expansion of (11) around 0=. 1b T hT (:;. =0. (11). yields:. q X j=1. bj. j;0. @3Q T @ j@ @. 0. (b. 0 );. 0.. Let us de…ne:. bT ( ; ) = H. and. b j;T ( ; ) = H. T 1 X @ 2 '(s; ; xt ) ir0 xt b ; Gj;T ( ; ) = e T @ @ 0 t=1. 1 T. T X t=1. @ 2 '(s; ; xt ) ir0 xt b ; Lj;T ( ; ) = e @ j@. T 1 X @'(s; ; xt ) ir0 xt ; e T @ j. 1 T. t=1 T X t=1. @ 3 '(s; ; xt ) ir0 xt ; e @ j@ @ 0. E D b T (:; 0 ); b b T ( 0 ) = Re K 1 G h (:; ) ; 0 T T E D E D b T (:; 0 ); b b T (:; 0 ); G b T (:;0 ) + Re K 1 H cT ( 0 ) = h (:; ) ; W K T1 G 0 T T D E D E bj;T ( ) = 2 Re K 1 G b T (:; ); H b j;T (:; ) + Re K 1 H b T (:; ); G b j;T (:; ) B T T D E b j;T (:; ); b + Re K T1 L hT (:; ) :. Then the expansion of the FOC becomes:. cT ( 0 ) b 0 = b T ( 0) + W. 0. +. q X j=1. 29. bj. j;0. bj;T ( ) b B. 0. ;.

(32) Unlike in the IID case, the derivatives of the moment function are not deterministic. We thus de…ne: b T ( ; ); b T ( ; ); H( ; ) = p limH G( ; ) = p limG T !1. T !1. b j;T ( ; ): b j;T ( ; ); Hj ( ; ) = p limH Gj ( ; ) = p limG T !1. T !1. It follows from Assumption 3 and Markov’s inequality that: b T ( ; ) = Op T G. G( ; ) Gj ( ; ). b j;T ( ; ) = Op T G. 1=2. ; H( ; ). 1=2. ; Hj ( ; ). We have the following decomposition for b T ( 0 ): b T ( 0) =. By using the fact that kAf k. T;0 ( 0 ). +. T;. b T ( ; ) = Op T H. 1=2. b j;T ( ; ) = Op T H. ;. 1=2. :. b ( 0 ) + e T; ( 0 ) + b T; ( 0 ) + e T; ( 0 ):. kAk kf k, we obtain the following the rates:. D E 1 b ( ) = Re K G; h (:; ) = Op T 1=2 ; 0 0 T;0 T D E 1 K 1 K 1 G; b hT (:; 0 ) = Op min(1; 2 ) T 1=2 ; T; ( 0 ) = Re E D 1 e T; ( 0 ) = Re K 1 K 1 G; b h (:; ) = Op T 1 ; 0 T T D E 1=2 b T; ( 0 ) = Re K 1 G bT G ; b hT (:; 0 ) = Op T 1 ; D E b 3=2 e T; ( 0 ) = Re K 1 K 1 G bT G ; b h (:; ) = Op T 3=2 : 0 T T. The di¤erence between the above decomposition of b T ( 0 ) and the one in the IID case only comes from b the additional higher order terms b T; ( 0 ) and e T; ( 0 ). Hence we can write b T ( 0 ) as: 1T. 1. b T ( 0) =. T;0 ( 0 ). min(1;. 1. +. T;. ). T 1=2 . cT ( 0 ): We have a similar decomposition for W. where R = op. where. + op. 2. ( 0 ) + e T; ( 0 ) + R ;. c cT ( 0 ) = W0 ( 0 ) + W ( 0 ) + W f ( 0) + W c ( 0) + W f ( 0) W c f1; ( 0 ) + W c1; ( 0 ) + W f +W1 ( 0 ) + W1; ( 0 ) + W 1; ( 0 ); W0 ( 0 ) = K W ( 0) =. K. 1 G; G 1. K. = O(1); 1. G; G = O. 30. min(1; 2. 1 2. ). ;.

(33) 1 T 1=2 ; f ( 0 ) = K 1 K 1 G; G = Op W T D E 1=2 T 1=2 ; c ( 0) = K 1 G b T G ; G = Op W D E D E b T G = Op T 1=2 ; W1 ( 0 ) = Re K 1 H; b hT + K 1 G; G E D 3=2 T 1 : b T G ; G = Op W1; ( 0 ) = K T1 K 1 G D E D E c f 1; ( 0 ) = Re K 1 K 1 H; b b T G = O min(1; 2 1 ) T 1=2 ; W hT + K 1 K 1 G; G D E D E 1 1 G; G 1T 1 ; f1; ( 0 ) = Re K 1 K 1 H; b b W h + K K G = Op T T T T D E D E 1=2 T 1 and c1; ( 0 ) = Re K 1 H bT H ; b bT G ; G b T G = Op W hT + K 1 G D E D E 3=2 T bT H ; b bT G ; G b T G = Op RW;1 = Re K T1 K 1 H hT + K T1 K 1 G. For the purpose of …nding the optimal , it is enough to consider the shorter decomposition: cT ( 0 ) = W0 ( 0 ) + W ( 0 ) + W f ( 0) + W c ( 0 ) + W1 ( 0 ) + W1; ( 0 ) + RW; W. with RW. c f 1; ( 0 ) + W f1; ( 0 ) + W c1; ( 0 ) + RW;1 = Op W. 1. T. 1. +O. min(1;. 1 2. ). 1=2. T. :. Finally, we consider again a simpler decomposition for Bj;T ( ): Bj;T ( ) = Bj ( ) + Bj;T ( ). Bj ( ). where Bj ( ) = 2 Re K. 1. 1. G(:; ); Hj (:; ) + Re K 1. min(1;. Bj;T ( ) = Bj ( ) + O. 2. ). 1. + Op. T. H(:; ); Gj (:; ) = O(1) and 1=2. :. We replace these decompositions into the expansion of the FOC and solve for b b. 0. =. W0 1 ( 0 ). W0 1 ( 0 ). h. T;0 ( 0 ) T;. ( 0) + W ( 0) b. h f ( 0) b W0 1 ( 0 ) e T; ( 0 ) + W W0 1 ( 0 )W1 ( 0 ) b. 0. j=1. W0 1 ( 0 )W1; ( 0 ) b q X j=1. bj. j;0. q X. 0. W0 1 ( 0 )(Bj;T ( ). W0 1 ( 0 )RW b. 0. 0 0. bj. i j;0. c ( 0) b W0 1 ( 0 )W W0 1 ( 0 )Bj ( ) b. Bj ( )) b. W0 1 ( 0 )R : 31. i. 0. 0. to obtain:. 0. 0. 3=2. :.

(34) Next, we replace b. 0. W0 1 ( 0 ). by. b. where 1. =. W0 1 ( 0 ). 2. =. W0 1 ( 0 ). 3. =. 0. =. T;0 ( 0 ). 1. +. 2. 1=2. = Op T. +. 3. 1=2. = Op T. W0 1 ( 0 ) e T; ( 0 ). in the higher order terms. This yields:. b1 + R b2 + R b3 + R b4 ; +R ;. W ( 0 )W0 1 ( 0 ). ( 0). T;. h. T;0 ( 0 ). T;0 ( 0 ). min(1; 2. = Op. i f ( 0 )W 1 ( 0 ) T;0 ( 0 ) = Op W 0. b 1 = W 1 ( 0 )W c ( 0 )W 1 ( 0 ) R 0 0. T;0 ( 0 ). 1=2. = Op. 1. T. b2 = W 1 ( 0 )W1 ( 0 )W 1 ( 0 ) T;0 ( 0 ) R 0 0 q X W0 1 ( 0 ) T;0 ( 0 ) j W0 1 ( 0 )Bj ( )W0 1 ( 0 ). 1. T. 1. 1 2. ). T. 1=2. 1. ;. ;. ;. ;. T;0 ( 0 ). = Op T. j=1. and. b3 = W 1 ( 0 )W1; ( 0 )W 1 ( 0 ) R 0 0 b4 = R. T;0 ( 0 ). 3=2. = Op. T. 3=2. ;. W0 1 ( 0 )R + W0 1 ( 0 )RW W0 1 ( 0 ) T;0 ( 0 ) q X W0 1 ( 0 ) T;0 ( 0 ) j W0 1 ( 0 )(Bj;T ( ) Bj ( ))W0 1 ( 0 ). T;0 ( 0 );. j=1. 1. = op. T. 1. min(1;. + op. 1. 2. ). 1=2. T. :. In summary, we have: b. 0. =. 1. +. 2. +. 3. 1. + op. T. 1. min(1;. + op. 1 2. ). T. 1=2. ;. (12). which is of the same form as in the IID case.. B.4. Proof of Theorem 2.. Using the expansions given in (10) and (12), we obtain: b. 0. =. 1. +. 2. +. 3. + Op T. Lemma 17 ensures that all terms that are slower than Op T. 1. the Re symbol may be removed from the expression of. 2. 32. 1,. 1. :. in the expansion above are real. Hence and. 3..

(35) Asymptotic Variance The asymptotic variance of b is given by T V ar (. = T W0 1 E. 1). 0 T;0 ( 0 ). T;0 ( 0 ). D K. = T W0 1 E. 1. G; b hT. ED. K. 1. W0. 1 G; b hT. E0. W0. 1. 1. = W0. 1. K. G; G W0 1 ;. where the last equality follows from Lemma 16. Hence, T V ar (. 1). = W0. 1. 1. K. G; G W0. 1. = W0 1 :. Higher Order Bias The terms. 1. and. 2. have zero expectations. Hence, the bias comes from h E b. Bias where f W W. 3 1. 0. f W W0 1 e T; + W0 1 W 0. = T;0 .. 1. T;0 .. 0. i. As W0. =E[. 3:. 3]. is a constant matrix, we focus on e T; +. 1. We …rst consider the term e T; . By applying Cauchy-Schwarz twice, we obtain: E e T;. =. E. D. E. K. K. 1 T. K. 1 T. K. 1. 1. G; b hT. E. s. b hT. G. E. K. 1 T. 1. K. 2. G. E. b hT. 2. :. Using the fact that ht ( ; ) is a martingale di¤erence sequence and is bounded, we obtain: E. b hT. 2. b hT ( ; ) ( ) d hT ( ; ) b Z 1 E ht ( ; ) ht ( ; ) ( ) d T. = E =. Z. (13) = O(T. 1. ):. Next, using (6) and (7), we obtain: E. K. 1 T. K. 1. G. 2. E E +E. 1 T. K. 1. K. (KT2 + I). 1. (KT. K. (KT2 + I). 1. 2. 1. K. K) K. G. 2. 2. (K 2 + I). 1. K 1 2. G. kKk4 K. Hence: (KT2 + I). (14) = E 2. E kKT. 1. (KT. K) K. Kk2 kKk2 K 33. 2 1. G. K 2. 1. G. = O(. 2 2. 2. T. 1. );. (14) 1. G. 2. :. (15).

(36) where E kKT. Kk2 = O(T. 1). 1. (15), we use the fact that A. follows from Carrasco and Florens (2000, Theorem 4, p. 825). For 1. B. 1 (B. =A. (15) = E. (KT2 + I). 1. (K 2 + I). E. (KT2 + I). 1. KT2. KT2. 2. E. 2. E kKT. 2. K2. 2. sZ Z. 2. 16 E e T;. =. f W We now consider the term W 0. We have: E. W0. 1. 2 T;0. 1. = E =. where E. b hT (:;. 0). 2. E. T;0. W0. W0 1 2. = O(T. 1). 1. 1. D. 1. 2. kKk4 K. 1. 1. G. 2. 2. G. 1. G. T;0 .. 2T. 2. 1. 1. G; b hT (:;. G. 1. b1 ). 1). G. 2;. 2. :. b1 ) is bounded such that. 2. 2. 2. ( 1) ( 2) d 1d. 1. G. 1). O(T. 2. = O(. 1. = O(. 2. 2. 2. 1. T. 1. T. ):. ):. (16). Again, using Cauchy-Schwarz twice leads to:. W0. 1. 2;. ( 1) ( 2) d 1d. Kk2 kKk4 K. f W. K. 2 2 )j. b kT ( 1 ;. p O(. K. G. 4. Similarly, b kT ( 1 ;. 2 2 )j. jk( 1 ;. E kKT. Finally,. f W E W 0. (K 2 + I). 1. i Kk2 (kKT k + kKk)2 kKk4 K. Consequently: (15). 2. K2. kKk4 K. kKT k + kKk. Hence:. h E kKT. sZ Z. kKT k. 1 2. kKk4 K. From (1) in Appendix A, we know that jk( 1 ; 1 2 b kT ( 1 ; 2 ; b ) 4. Hence: kKk. to obtain:. Kk2 kKT + Kk2 kKk4 K. By the triangular inequality, kKT + Kk (15). 1. A) B. 2. E. s. T;0. 0). E. 2. b hT (:;. 0). follows from (13). Next:. 34. E. f W. E. W0. 2. 2. E. 1 2. = O(T. 1. W0. K );. 1. G. 1. 2 T;0. 2. :. b hT (:;. 0). 2.

(37) E. 2. f W. K. = E. 1 T. = kGk2 E. K 1 T. K. 1. 2. G; G 1. K. 2. G. E. K. = O(. 2. 1 T. T. 1. 1. K. G. 2. kGk2. );. where the rate follows from (14) and (15). Hence, by the Cauchy-Schwarz inequality, f W E W 0. 1. s. p O(. =. By putting (16) and (17) together, we …nd E [. 2. f W. E. T;0. 2T. 3]. E. W0. 1 )O(T 1T. = Op. T Bias:Bias0 = O. 1). 2. T. 1. 2. 1. T;0. = O(. 1. 1. T. 1. ):. (17). so that the squared bias satis…es:. :. Higher Order Variance The dominant terms in the higher order variance are Cov ( We …rst consider Cov ( Cov (. 1;. 2). 1;. 1;. 2). + V ar (. 2). + Cov (. 1;. 3) :. 2 ):. = W0 1 E. T;0. T;. ( 0 )0 W0. 1. W0 1 E. T;0. 0 T;0. W0 1 W W0 1 :. From Lemma 16, we have: E and E. h. T;0. 0 T;0. i. T;0. 0 T;. =. 1 T. K. 1. K. 1. G; G = W :. = W0 . Hence, Cov (. 1;. 2). =. Now we consider the term Cov ( Cov (. 1;. 3). = W0 1 E. 1 W 1 W W0 T 0 1;. 1 W 1 W0 W0 1 W W0 T 0. 1. 1. = 0:. 3 ):. T;0. e 0T;. W0. 35. 1. W0 1 E. T;0. 1f 0 T;0 W0 W. W0 1 :.

(38) We …rst consider E. E. T;0. h. T;0. E Also, D. E. K. We …rst consider b hT b hT. 4. T;. e 0T;. Hence we have:. 1 T. 4. i. e0. . By the Cauchy-Schwarz inequality:. s. E k. s. =. D. K. D K. E. 1. K. T;0 k. G; b hT. 1. G; b hT. E. 2. 1 G; b h. 2. E. E. K. 2. T;. T. E. 1. G. = E s. T Z X t=1. 2. K. 2. b hT. E. 1 T. K E. D. E. K. K 1 T. 1. K. 1 T. K. 1. = O(T. G 1. 2. G. b hT. 4. G; b hT. 1. E. 2. ). 2. E. b hT. 4. 0 12 Z X Z X T T 1 b ht ht ( ) d + ht hs ( ) d A hT ( ) d = 4@ = hT b T t=1 t6=s 0 12 ! 2 T Z T Z X X 1 1 = ht ht ( ) d + 4@ ht hs ( ) d A 4 T T t=1 t6=s 0 1 ! T Z T Z X X 1 @ 1 ht ht ( ) d ht hs ( ) d A +2 2 T T2 Z. 2. t6=s. Consider the …rst squared term of b hT 1 E4 4 T. 2. :. t=1. 2. 2. e0. ht ht ( ) d. !2 3. 5 =. 4. :. T E T4. = O(T. " Z 2. 2. ht ht ( ) d. ). 36. #. T (T 1) + E T4. Z. 2. ht ht ( ) d.

(39) The second squared term leads to: 2. E4 2. 0. 1 @ T4. 12 3. T Z X. ht hs ( ) d A 5. t6=s. 3 2 Z T 2 X 1 5+E4 1 = E4 4 ht hs ( ) d T T4 t6=s " Z # 2 T (T 1) = E ht hs ( ) d = O(T T4. Z. T X. t6=s;l6=j;(t;s)6=(l;j) 2. Z. ht hs ( ) d. hl hj ( ) d. 3 5. ); for t 6= s:. As the ht s are uncorrelated, the cross-term is equal to zero: 2. !0. 13 T Z X @ 1 ht hs ( ) d A5 = 0 T2. T Z X 4 1 ht ht ( ) d T2 t=1 4. b hT. In total, we obtain: E We now consider E. K. 2 ).. = O(T. 1 T. K. 1. t6=s. G. 4. . Using the same decomposition as in (14) and (15) leads. to: E. K. 1 T. 1. K. G. 4. E E. 1 T. 1. K. (KT2 + I). 1. (KT. (KT2 + I). +E. 4. K. K. 1. K. 1. G. 4. 4. K) K. (K 2 + I). K 1 4. 1. G. 4. (18). kKk8 K. 1. G. 4. ;. (19). Hence: (18) = E. (KT2 + I). For (19), we use A. 1. B. 1. 1. (KT. =A. K) K. 1 (B. A) B. (19) = E. (KT2 + I). 1. E. (KT2 + I). 1 2. 4. E. KT2. 4. E kKT. 4. K2. K 1. 1. G. 4. 4. E kKT. Kk4 kKk4 K. to obtain:. (K 2 + I). 1 4. K2. 4. kKk8 K. 1. KT2. 4. kKk8 K. (K 2 + I) G. 1. G. G. 1 2. 4. Kk4 kKT + Kk4 kKk8 K. 37. 1. 4. 4. kKk8 K. 1. G. 4. 1. G. 4.

(40) By the triangular inequality: 4. (19). 4. 256 due to kKT k. 2 and kKk. Kk4 (kKT k + kKk)4 kKk8 K. E kKT. Kk4 kKk8 K. E kKT. Z Z. 2. Kk. =. 1 T2 +. t( 1; 2). where. = kt ( 1 ;. 2;. E kKT Because E ([(20)] [(21)]) terms. We have:. b1 ). t=1 T Z X. 1 T2. t6=l. k( 1 ;. 2 ).. t( 1; 2). j t( 1;. Z. 2 )j. ( 1) ( 2) d 1d 2. ( 1) ( 2) d 1d. t( 1; 2) l( 1; 2). with. t. t ( 1 ; 2 ).. E [(21)]2. ( 1) ( 2) d 1d. 2 ( 1) ( 2) d 1d 2 )j. 2. i. ; for l 6= t:. Next:. t( 1; 2) l( 1; 2). E. (21). 2. E [(21)]2 , we only need to check the rates of the squared. 2. t6=l;n6=j;(t;l)6=(n;j). (20). 2. E [(20)]2 + 2E ([(20)] [(21)]) + E [(21)]2. r E [(20)]2. 2 ).. 2. Hence. ( 1) ( 2) d 1d. t6=l. T X. ;. 2. T. 1X T t=1 Z T X Z. Kk4. Hence E [(20)]2 = O(T. 1 + 4 T. 4. 4. Kk4 .. 2 RR E [(20)]2 = TT4 E j t ( 1 ; 2 )j2 ( 1 ) ( 2 ) d 1 d 2 h RR RR j t ( 1 ; 2 )j2 ( 1 ) ( 2 ) d 1 d 2 j l( 1; + T (TT 4 1) E. E [(21)]2 " Z Z T 1 X = E T4. G. G. 2.. The rates of (18) and (19) depend on the rate of E kKT kKT. 1. 1. Z Z. t l. ( 1) ( 2) d 1d. 2. 2. #. Z Z. n j. ( 1) ( 2) d 1d. Due to the m.d.s property, the last term has expectation zero. Hence:. T (T 1) E = T4. " Z Z. 2 t( 1; 2) l( 1; 2). 38. ( 1) ( 2) d 1d. 2. #. = O(T. 2. ):. 2.

(41) Kk4 = O(T. By putting these together, we obtain E kKT E E. D. K. 1 T. K 1 T. K. G; b hT. 1. K. 1. 4. G E. (18)+(19) = O s. 2. E. p O(. =. In total: s. e0 T;0. E. E. T;. p O(T. =. D K. 1 G; b h. 1). 4T. E. T. 2. 2T. O(. E. T;0. 1f 0 T;0 W0 W. T;0. We …rst consider E. 0 T;0. T;0. 2. 0 T;0. D. =E. For the second term, we have: E. f W0 1 W. s. 2. 2. E. 2. T. 1. K. D. 2). 1 T. K 1. and 4. G. O (T. =O. 1;. so that: 4. 2). E 2). We now check the rate of the second term of Cov (. E. 1 T. K. 2). 2. =O. K. G; b hT. 1. 3=2. T. b hT. E. 4. 2. T. E. 2. 3 ): 2. 0 T;0. T;0. 2. f W0 1 W. E. . By the Cauchy-Schwarz inequality:. ED K 1 G; b hT K. 1. G; b hT. = kW0 k. 2. E. kW0 k. 2. kGk2 E. E0. 2. 1. K. !. K. 1. G. K. O(. 2T. 1. 4. 4. b hT. E. = O(T. 2. ). 2. G; G. 1. K. 1. K. G. 2. = O(. 2. 1. T. 3=2. T;. + W0 1 W W0. T. 1. );. according to (14)-(15). Hence, E. T;0. p. 1f 0 T;0 W0 W. It now remains to …nd the rate of V ar (. 2). O(T. 2 ).. We recall that. 2. 1). = O(. =. W0. 1. ) 1. T;0 .. We have V ar (. 2). = W0 1 E. T;. 0 T;. W0 1 W W 0 1 E Replacing E. h. T;0. 0 T;. i. =. 1 TW. and E. W0 T;0. h. 1. W0 1 E 0 T;. T;0. W0 0 T;0. i. 1. =. 39. T;. 0 T;0. W0 1 W W0. + W0 1 W W 0 1 E 1 T W0 ,. T;0. 1 0 T;0. W0 1 W W0 1 :. we see immediately that the last two terms.