Validity of the parametric bootstrap for goodness-of-fit testing in semiparametric models

www.imstat.org/aihp 2008, Vol. 44, No. 6, 1096–1127

DOI: 10.1214/07-AIHP148

© Association des Publications de l’Institut Henri Poincaré, 2008

Validity of the parametric bootstrap for goodness-of-fit testing in semiparametric models

Christian Genest

and Bruno Rémillard

aDépartement de mathématiques et de statistique, Université Laval, Québec (Québec), Canada G1K 7P4.

E-mail: Christian.Genest@mat.ulaval.ca

bService d’enseignement des méthodes quantitatives de gestion, HEC Montréal, 3000, chemin de la Côte-Sainte-Catherine, Montréal (Québec), Canada H3T 2A7. E-mail: Bruno.Remillard@hec.ca

Received 24 May 2006; revised 3 July 2007; accepted 25 September 2007

Abstract. In testing that a given distributionPbelongs to a parameterized familyP, one is often led to compare a nonparametric estimateAnof some functionalAofP with an elementA_θncorresponding to an estimateθnofθ. In many cases, the asymptotic distribution of goodness-of-fit statistics derived from the processn^1/2(An−Aθ_n)depends on the unknown distributionP. It is shown here that if the sequencesAnandθnof estimators are regular in some sense, a parametric bootstrap approach yields valid approximations for theP-values of the tests. In other words ifA^∗_n andθ_n^∗ are analogs ofAn andθn computed from a sample fromP_θn, the empirical processesn^1/2(A_n−A_θn)andn^1/2(A^∗_n−A_θ∗

n)then converge jointly in distribution to independent copies of the same limit. This result is used to establish the validity of the parametric bootstrap method when testing the goodness- of-fit of families of multivariate distributions and copulas. Two types of tests are considered: certain procedures compare the empirical version of a distribution function or copula and its parametric estimation under the null hypothesis; others measure the distance between a parametric and a nonparametric estimation of the distribution associated with the classical probability integral transform. The validity of a two-level bootstrap is also proved in cases where the parametric estimate cannot be computed easily.

The methodology is illustrated using a new goodness-of-fit test statistic for copulas based on a Cramér–von Mises functional of the empirical copula process.

Résumé. Pour tester qu’une loiP donnée provient d’une famille paramétriqueP, on est souvent amené à comparer une estimation non paramétriqueA_nd’une fonctionnelleAdeP à un élémentA_θncorrespondant à une estimationθ_ndeθ. Dans bien des cas, la loi asymptotique de statistiques de tests bâties à partir du processusn^1/2(An−A_θn)dépend de la loi inconnueP. On montre ici que si les suitesA_netθ_nd’estimateurs sont régulières dans un sens précis, le recours au rééchantillonnage paramétrique conduit à des approximations valides des seuils des tests. Autrement dit siA^∗_netθ_n^∗sont des analogues deA_netθ_ndéduits d’un échantillon de loiP_θn, les processus empiriquesn^1/2(A_n−A_θn)etn^1/2(A^∗_n−A_θ∗

n)convergent alors conjointement en loi vers des copies indépendantes de la même limite. Ce résultat est employé pour valider l’approche par rééchantillonnage paramétrique dans le cadre de tests d’adéquation pour des familles de lois et de copules multivariées. Deux types de tests sont envisagés : les uns comparent la version empirique d’une loi ou d’une copule et son estimation paramétrique sous l’hypothèse nulle ; les autres mesurent la distance entre les estimations paramétrique et non paramétrique de la loi associée à la transformation intégrale de probabilité classique. La validité du rééchantillonnage à deux degrés est aussi démontrée dans les cas où l’estimation paramétrique est difficile à calculer. La méthodologie est illustrée au moyen d’un nouveau test d’adéquation de copules fondé sur une fonctionnelle de Cramér–von Mises du processus de copule empirique.

MSC:62F05; 62F40; 62H15

Keywords:Copula; Goodness-of-fit test; Monte Carlo simulation; Parametric bootstrap;P-values; Semiparametric estimation

1. Introduction

Given independent copiesX1, . . . , X_nof a random vectorXwith cumulative distribution functionF:R^d→R, sup- pose that it is desired to test

H₀: F ∈F= {F_θ: θ∈O},

the hypothesis thatF comes from a parametric family of distributions whose members are indexed by a parameterθ belonging to an open setO⊂R^p. To achieve this goal, a natural way to proceed consists of measuring the difference between the empirical distribution function, defined for allx∈R^dby

F_n(x)=1 n

n

i=1

1(Xi≤x), (1)

and a parametric estimateF_θnofF derived underH0from some consistent estimateθ_n=T_n(X1, . . . , X_n)of the true parameter valueθ₀. Here and in the sequel, inequalities between vectors are taken to hold componentwise.

Cramér–von Mises, Kolmogorov–Smirnov and many other standard goodness-of-fit procedures are based on sta- tistics expressed as continuous functionalsS_n=φ (G^F_n)of the empirical process

G^F_n =n^1/2(F_n−F_θn).

Formal tests, however, require knowledge of the asymptotic null distribution ofS_n, which often depends on the un- known value ofθ.

1.1. The parametric bootstrap

To solve this problem, Stute et al. [26] suggest the following “parametric bootstrap” procedure.

For some large integerN and everyk∈ {1, . . . , N}, repeat the steps below:

(a) Givenθ_n=T_n(X1, . . . , X_n), generatenindependent observationsX^∗_1,k,. . .,X^∗_n,kfrom distributionF_θn. (b) Computeθ_n,k^∗ =T_n(X^∗_1,k, . . . , X^∗_n,k)and for eachx∈R^d, let

F_n,k^∗ (x)=1 n

n

i=1

1

X_i,k^∗ ≤x .

(c) ComputeS_n,k^∗ =φ (G^F_n,k^∗), where G^F_n,k^∗=n^1/2

F_n,k^∗ −F_θ∗ n,k

.

With the convention that large values ofSnlead to the rejection ofH0, Stute et al. [26] show that under appropriate regularity conditions, an approximateP-value for the test is given by

1 N

N

k=1

1

S_n,k^∗ > S_n .

Henze [20] obtained a similar result in the univariate discrete case. In both papers, the validity of the paramet- ric bootstrap stems from the fact that under H₀ and as n→ ∞,(S_n, S_n,1^∗ , . . . , S_n,N^∗ )converges weakly to a vector (S, S₁^∗, . . . , S_N^∗)of mutually independent and identically distributed random variables.

1.2. Motivation for the present work

This investigation was motivated by the need to test the appropriateness of various dependence structures on the basis of a random sample

X₁=(X₁₁, . . . , X_1d), . . . , X_n=(X_n1, . . . , X_nd)

from a continuous random vectorXwith cumulative distribution functionF. Specifically, denote byF1, . . . , F_d the univariate margins ofXand letC:[0,1]^d→ [0,1]be the copula for which Sklar’s representation

F (x₁, . . . , x_d)=C

F₁(x₁), . . . , F_d(x_d)

holds for allx1, . . . , x_d∈R. In fact,Cis simply the cumulative distribution function ofU=ξ(X), whereξ:R^d→R^d is defined for allx1, . . . , xd∈Rby

ξ(x₁, . . . , x_d)=

F₁(x₁), . . . , F_d(x_d)

. (2)

Unless the margins are known, the vectorsU1=ξ(X1), . . . , U_n=ξ(X_n)cannot be observed. However, a consistent estimate ofFj is defined for allt∈Randj ∈ {1, . . . , d}by

F_{j n}(t )= 1 n+1

n

i=1

1(Xij≤t ).

This uncommon choice of normalization is used becauseFj n serves later as an argument in score functions and pseudo-likelihoods that could blow up at 1. Letting

ξn(x1, . . . , xd)=

F1n(x1), . . . , Fdn(xd)

, (3)

for allx1, . . . , xd∈R, one could thus base a test of the hypothesis

H₀: C∈C= {C_θ: θ∈O} (4)

on the pseudo-observationsUˆ1=ξ_n(X1), . . . ,Uˆ_n=ξ_n(X_n). Various options are possible; two of them are briefly described below.

Tests based on the empirical copula

Hypothesis (4) could be tested using a Cramér–von Mises or Kolmogorov–Smirnov statisticSn=φ (G^Cn)with G^C_n =n^1/2(C_n−C_θn),

whereC_θn is a parametric estimate ofC_θ derived from the estimationθ_n=T (X₁, . . . , X_n)ofθunderH₀whileC_nis the empirical copula, defined for allu∈ [0,1]^dby

C_n(u)=1 n

n

i=1

1(Uˆ_i≤u). (5)

This possibility is raised but quickly dismissed by Fermanian [10], due to the complexity of the weak limit ofG^C_n. See, e.g., [11,12,27] for derivations of the limit of the related empirical copula processn^1/2(Cn−Cθ₀).

Tests based on Kendall’s distribution

Another avenue explored by Wang and Wells [29] and Genest et al. [15] is to construct a test of hypothesis (4) on Kendall’s distribution, i.e., the distribution functionKof the probability integral transformW=F (X). Using the fact that one can also writeW=C(U ), Genest and Rivest [17] and Barbe et al. [1] show that a consistent estimate ofK is given by the empirical distributionK_nof the pseudo-observationsWˆ₁=C_n(Uˆ₁), . . . ,Wˆ_n=C_n(Uˆ_n). The latter is defined for allw∈ [0,1]by

Kn(w)=1 n

n

i=1

1(Wˆi≤w). (6)

Thus ifK_θ denotes the distribution ofW whenC=C_θ∈C, and ifK_θn is a parametric estimate ofK_θ derived from θ_n=T (X1, . . . , X_n)under the subsidiary hypothesis

H₀: K∈K= {K_θ: θ∈O}, (7)

a goodness-of-fit test could rely on a continuous functionalS_n=φ (G^K_n)of G^K_n =n^1/2(Kn−Kθ_n).

Whether hypothesis (4) is tested usingG^Cn or the subsidiary hypothesis (7) is tested usingG^Kn, the limiting dis- tribution of the test statisticS_ndoes not only depend on the unknown parameterθbut also possibly on the nuisance parametersF₁, . . . , F_d. Therefore, while the use of a parametric bootstrap may very well yield validP-values, this conclusion cannot be reached on the basis of the results reported by Stute et al. [26], because of the presence of dependence among the sets of pseudo-observationsUˆ1, . . . ,UˆnandWˆ1, . . . ,Wˆn.

1.3. Objective and outline of the paper

The purpose of this work is to establish the validity of the parametric bootstrap in situations where the hypothesis to be tested concerns the distributionP of an unobservables-variate random vectorU, viz.

H0: P ∈P= {P_θ: θ∈O},

where O is an open subset of R^p. Although U cannot be seen, it is assumed that U =ξ(X)for some function ξ:R^d→R^s of an observabled-variate random vectorX, and that a consistent estimatorξnofξ can be constructed from independent copiesX1, . . . , XnofX.

In order to encompass procedures based onG^C_n andG^K_n as special cases, suppose that a test ofH₀is to be derived from a continuous functionalS_n=φ (G^An)of an abstract empirical process of the form

G^An =n^1/2(A_n−A_θn).

Here,A_θn andA_n stand respectively for a parametric and a nonparametric estimate of an abstract quantityAthat depends onP. More specifically,Ais taken to be a function mapping a closed rectangleT ⊂ [−∞,∞]^r intoR^s, and Aθ denotes the form taken byAwhenP =Pθ for someθ∈O. Thus for the test based onG^Cn, one hasT = [0,1]^d, r=s=dandA_θ=C_θ; similarly,T = [0,1],r=s=1 andA_θ=K_θfor the test based onG^K_n.

The result to be shown here is that the parametric bootstrap yields a valid approximation to the null distribution of the empirical processG^An under appropriate conditions. The main requirements concern the large-sample behavior of the estimatorsAnofAandθnofθthat are constructed from the pseudo-observationsUˆ1=ξn(X1), . . . ,Uˆn=ξn(Xn).

In particular, the processΘ_n=n^1/2(θ_n−θ )needs to converge weakly, asn→ ∞, to a centered random variableΘ.

This is denoted symbolically

Θ_n=n^1/2(θ_n−θ )Θ. (8)

Similarly, it must be that, asn→ ∞,

An=n^1/2(An−A)A, (9)

i.e.,Anconverges weakly to a centered processAin the spaceD(T;R^s)of càdlàg processes fromT toR^s, equipped with the Skorohod topology.

Additional regularity conditions needed for the result are stated in Section 2. Although these conclusions could possibly be derived within a different framework considered by Bickel and Ren [4], the conditions given here are adapted to the current context and easier to verify than theirs. The present proofs are also different and yield interesting insights. The two-level parametric bootstrap introduced in Section 3 also appears to be novel; it is required in many applications whereAθ_ncannot be computed easily but can be approximated through a parametric bootstrap of its own.

The goodness-of-fit tests for copula models introduced above are revisited in Section 4. Also given there is a multivariate extension of a procedure designed by Durbin [9] for checking the fit of a univariate distribution. As a practical illustration, testing for a Gaussian copula structure is considered in Section 5 on the basis of the empirical copula (5). An explicit algorithm is also provided which can be adapted easily to test for other copula families via one- or two-level parametric bootstrapping. For a more extensive comparison of this procedure with alternative tests for copula models, see Genest et al. [16].

To avoid interrupting the flow of the presentation, most technical arguments are relegated to a series of appendices.

2. Validity of the one-level parametric bootstrap

LetU1, . . . , Unbe a random sample from some distributionP, and assume that it is desired to test the hypothesis H₀: P∈P= {P_θ: θ∈O},

wherePis a family of probability measures onR^dindexed by a parameterθliving in an open setO⊂R^p. The family is assumed to be identifiable, i.e.,θ =θ⇒P_θ =P_θ.

As discussed in the Introduction, letT ⊂ [−∞,∞]^r be a closed rectangle and suppose that the test ofH₀ is to be based on an abstract mappingA:T →R^s that depends on the true distributionP of U1, . . . , U_n. In particular, suppose thatA=Aθ whenP =Pθ, and writeA= {Aθ:θ∈O}. In this general context, identifiability is ensured if for everyε >0,

inf

sup

t∈T

A_θ(t )−A_θ0(t ): θ∈Oand|θ−θ0|> ε >0.

This condition is assumed throughout, as one might otherwise haveAθ=A_θ for someθ =θand problems could arise; see, e.g., [24]. Furthermore, the mappingθ→A_θis assumed to be Fréchet differentiable with derivativeA, i.e.,˙ for allθ₀∈O,

hlim→0sup

t∈T

A_θ0+h(t )−A_θ0(t )− ˙A(t )h

h =0. (10)

Finally, let θ_n =T_n(U₁, . . . , U_n) be a consistent estimate of θ and assume that the D(T;R^s)-valued process A_n=Υ_n(U1, . . . , U_n) estimates A consistently. Suppose specifically that the processes Θ_n=n^1/2(θ_n −θ ) and An=n^1/2(An−A)have centered Gaussian limits whenn→ ∞, as per (8) and (9).

The purpose of this section is to state additional regularity conditions on the familiesP,Aand on the sequences A_nandθ_nof estimators. These requirements will ensure that a parametric bootstrap algorithm approximates correctly the limiting behavior of the empirical process

G^A_n =n^1/2(A_n−A_θn).

Consequently, the parametric bootstrap will also provide a suitable approximation of the asymptotic distribution of goodness-of-fit test statistics expressed as continuous functionalsSn=φ (G^A_n).

The validity of the parametric bootstrap first depends on smoothness and integrability conditions on the parametric family of distributions.

Definition 1. A familyP= {P_θ: θ∈O}is said to belong to the classS(λ)for a given reference measureλ(indepen- dent ofθ)if:

1.1. The measureP_θis absolutely continuous with respect toλfor allθ∈O.

1.2. The densityp_θ=dPθ/dλadmits first and second order derivatives with respect to all components ofθ∈O.The gradient(row)vector with respect toθis denotedp˙_θ,and the Hessian matrix is represented byp¨_θ.

1.3. For arbitraryu∈R^d and everyθ₀∈O,the mappingsθ→ ˙p_θ(u)/p_θ(u)andθ→ ¨p_θ(u)/p_θ(u)are continuous atθ0,Pθ₀ almost surely.

1.4. For everyθ0∈O,there exist a neighborhoodN ofθ0and aλ-integrable functionh:R^d→Rsuch that for all u∈R^d, sup_θ∈N ˙pθ(u) ≤h(u).

1.5. For everyθ0∈O,there exist a neighborhoodN ofθ0andPθ₀-integrable functionsh1, h2:R^d→Rsuch that for everyu∈R^d,

sup

θ∈N

p˙_θ(u) p_θ(u)

²≤h₁(u) and sup

θ∈N

p¨_θ(u) p_θ(u)

≤h₂(u).

In the sequel,θ0represents the true (unknown) value ofθandP =P_θ0. Furthermore, p=p_θ0, p˙= ˙p_θ0, p¨= ¨p_θ0.

Remark 1. Using Condition1.4with the continuity ofp˙θ as a function ofθand Lebesgue’s dominated convergence theorem,one may conclude that

∂

∂θ

p_θ(u)g(u)λ(du)=

˙

p_θ(u)g(u)λ(du) (11)

for any bounded measurable functiong:R^d→R,not depending onθ.In particular,

˙

p(u)λ(du)=0.Furthermore, ifF_θ denotes the distribution function associated withP_θ,the mappingθ→F_θ is then Fréchet differentiable and its derivativeF˙_θ satisfies the following identity for allx∈R^d:

F˙_θ(x)=

˙

p_θ(u)1(u≤x)λ(du). (12)

Remark 2. WhenP∈S(λ),the multivariate central limit theorem implies that ifU1, . . . , Unform a random sample fromP=Pθ₀,then asn→ ∞,

WP ,n=n^−1/2 n

i=1

˙ p(Ui)

p(Ui) WP∼N(0, IP), (13)

whereE(WP)=0by Remark1andI_P is the Fisher information matrix,viz.

I_P =

p˙(u)p(u)˙

p(u) λ(du). (14)

The validity of the parametric bootstrap also relies on the following general notion ofP-regularity of estimators.

It is cast below in terms ofA_nbut it applies also to many other sequences in the sequel, e.g., in the caseA_n=θ_n. Definition 2. LetU₁, . . . , U_nbe a random sample fromP =P_θ0 and letWP ,nbe defined as in(13).A sequenceA_nis said to bePθ₀-regular forA=Aθ₀if,asn→ ∞,the process(An,WP ,n)withAn=n^1/2(An−A)converges weakly inD(T;R^s)×R^pto a centered Gaussian pair(A,WP)and the Fréchet derivativeA˙ofAdefined in(10)satisfies A(t )˙ =E{A(t )W_P}for everyt ∈T.The sequence is said to beP-regular forAif it isPθ₀-regular forAθ₀ at all θ0∈O.

Remark 3. TheP-regularity of a sequence of estimatorsθn=Tn(U1, . . . , Un)forθ∈Oimplies thatΘn=n^1/2(θn− θ )Θasn→ ∞,whereΘis a centered Gaussian random vector andE(ΘW_P)=I is the identity matrix.

Now letU₁^∗, . . . , U_n^∗be a bootstrap sample fromP_θn, and set θ_n^∗=T_n

U₁^∗, . . . , U_n^∗

, Θ_n^∗=n^1/2 θ_n^∗−θ

, A^∗_n=Υ_n

U₁^∗, . . . , U_n^∗

, A^∗_n=n^1/2

A^∗_n−A .

The following result, whose proof is given in Appendix B, gives conditions under which the weak limits of the processes

G^An =n^1/2(A_n−A_θn) and G^A_n^∗=n^1/2

A^∗_n−A_θn∗

.

are independent and identically distributed. This guarantees that a parametric bootstrap based on the processAn is valid.

Theorem 1. Assume thatP∈S(λ)and that asn→ ∞,

(An, Θ_n,WP ,n)(A, Θ,WP) (15)

inD(T;R^s)×R^p⊗2,where the limit is a centered Gaussian process.LetΓ =E(ΘW_P)and seta(t )=E{A(t )W_P} for everyt∈T.Then,asn→ ∞,

An,A^∗_n, Θ_n, Θ_n^∗

A,A, Θ, Θ

inD(T;R^s)^⊗2×R^p⊗2.In the limit,A=A^⊥+aΘ and Θ=Θ^⊥+Γ Θ are defined in terms of an independent copy(A^⊥, Θ^⊥)of(A, Θ).If in addition(An, θn)isP-regular forA×O,then

G^An,G^An^∗

G^A,G^A∗

=

A− ˙AΘ,A^⊥− ˙AΘ^⊥

inD(T;R^s)^⊗2,asn→ ∞,andG^A∗ is an independent copy ofG^A. 3. A two-level parametric bootstrap

To perform a goodness-of-fit test based on a continuous functionalS_n=φ (G^A_n)of the process G^A_n =n^1/2(A_n−A_θn),

one must computeA_θnat various points, but this is not always easily done.

For tests based on the empirical copula, for instance, one hasAθ_n=Cθ_n and many copula families are not alge- braically closed. In this case, a simple way to circumvent the problem is to generate a random sampleV₁^∗, . . . , V_m^∗ from probability measureQ_θnwith distribution functionC_θnand foru∈ [0,1]^d, to approximateC_θn(u)by

Cˇ_n^∗(u)= 1 m

m

j=1

1 V_j^∗≤u

.

It is typical to takem= γ nfor someγ∈(0,∞), but it will only be assumed here thatmis a function ofnsuch that m/n→γ∈(0,∞)asn→ ∞.

More generally, the strategy proposed here consists of replacingA_θnby an approximationAˇ^∗_n=Ψ_m(V₁^∗, . . . , V_m^∗) built from a random sampleV₁^∗, . . . , V_m^∗ fromQ_θn∈Q= {Q_θ: θ∈O}. In order for this approach to make sense, it must be assumed that ifA=A_θ0 andAˇ_n=Ψ_m(V1, . . . , V_m)for a random sampleV1, . . . , V_mfromQ=Q_θ0, then

Aˇn=n^1/2(Aˇn−A)Aˇ

inD(T;R^s), asn→ ∞(and hencem→ ∞).

Given that such a process exists, here is a natural way to circumvent the lack of a closed form forA_θn in the computation of the test statisticSn:

(a) Computeθ_n=T_n(U₁, . . . , U_n)and letA_n=Υ_n(U₁, . . . , U_n).

(b) GivenU₁, . . . , U_n, generate a random sampleV₁^∗, . . . , V_m^∗fromQ_θn.

(c) LetAˇ^∗_n=Ψ_m(V₁^∗, . . . , V_m^∗)and computeS_n=φ (G^A_n^ˇ∗), in whichG^A_n^ˇ∗=n^1/2(A_n− ˇA^∗_n).

Now in order to approximate the distribution ofSn, a second parametric bootstrap procedure is necessary. To this end, pickN large and repeat the following steps for everyk∈ {1, . . . , N}:

(a) GivenU1, . . . , Un,V₁^∗, . . . , V_m^∗, generate a random sampleU_1,k^∗ , . . . , U_n,k^∗ fromPθ_n. (b) Computeθ_n,k^∗ =Tn(U_1,k^∗ , . . . , U_n,k^∗ )and letA^∗_n,k=Υn(U_1,k^∗ , . . . , U_n,k^∗ ).

(c) GivenU1, . . . , U_n,V₁^∗, . . . , V_m^∗andU_1,k^∗ , . . . , U_n,k^∗ , generate a random sampleV_1,k^∗∗, . . . , V_m,k^∗∗ fromQ_θ∗ n,k. (d) LetAˇ^∗∗_n,k=Ψm(V_1,k^∗∗, . . . , V_m,k^∗∗)and computeS_n,k^∗ =φ (G^A_n,k^ˇ∗∗), in whichG^A_n,k^ˇ∗∗=n^1/2(A^∗_n,k− ˇA^∗∗_n,k).

With the convention that large values ofSnlead to the rejection ofH0, and under regularity conditions stated below, a valid approximation to theP-value for the test based onSn=φ (G^An^ˇ∗)is given by

1 N

N k=1

1

S_n,k^∗ > Sn

.

As for the standard parametric bootstrap, the validity of the above two-level extension is ensured, provided that one can show that, asn→ ∞,(G^An^ˇ∗,G^A_n,1^ˇ∗∗)converges weakly inD(T;R^s)^⊗2to a pair of independent and identically distributed limiting processes.

Assume thatQ∈S(ν)for some reference measureν(independent ofθ). Writeq_θfor the density ofQ_θ, letq˙_θ be the gradient (row) vector with respect toθ, and denote the Hessian matrix byq¨_θ. WhenQ=Q_θ0, write by extension

q=qθ₀, q˙= ˙qθ₀, q¨= ¨qθ₀.

Note that whenQ∈S(ν), the multivariate central limit theorem implies that ifV1, . . . , V_mform a random sample fromQ=Q_θ, then, asn→ ∞,

WQ,n=n^−1/2 m

i=1

˙ q(V_i)

q(Vi) WQ∼N(0, IQ), (16)

where in view of the fact thatm/n→γ∈(0,∞)asn→ ∞, I_Q=γ

q˙(u)q(u)˙

q(u) ν(du). (17)

Now letU1, . . . , U_n andV1, . . . , V_m be two mutually independent random samples fromP =P_θ0 ∈P andQ= Q_θ0∈Q, respectively. LetWP ,nandWQ,nbe defined as in (13) and (16), respectively. Conditionally onU₁, . . . , U_n andV₁, . . . , V_m, make the following additional assumptions:

(a) Givenθ_n=T_n(U1, . . . , U_n), the random vectorsU₁^∗, . . . , U_n^∗andV₁^∗, . . . , V_m^∗ are mutually independent random samples fromP_θn andQ_θn, respectively.

(b) GivenU₁^∗, . . . , U_n^∗ andV₁^∗, . . . , V_m^∗ andθ_n^∗=T_n(U₁^∗, . . . , U_n^∗), the random vectorsV₁^∗∗, . . . , V_m^∗∗ are a random sample fromQ_θ∗

n.

Finally, introduce the additional notations Aˇ_n=Ψ_m(V₁, . . . , V_m), Aˇ^∗_n=Ψ_m

V₁^∗, . . . , V_m^∗

, Aˇ^∗∗_n =Ψ_m

V₁^∗∗, . . . , V_m^∗∗

and

Aˇn=n^1/2(Aˇn−A), Aˇ^∗n=n^1/2Aˇ^∗_n−A

, Aˇ^∗∗n =n^1/2Aˇ^∗∗_n −A .

The following result, whose proof is given in Appendix C, gives conditions under which the weak limits of the processes

G^A_n^ˇ∗=n^1/2

A_n− ˇA^∗_n

and G^A_n^ˇ∗∗=n^1/2

A^∗_n− ˇA^∗∗_n

are independent and identically distributed. This proves the validity of a two-level parametric bootstrap based on the processAn.

Theorem 2. Assume thatP∈S(λ),Q∈S(ν)and that asn→ ∞, (An,Aˇn, Θ_n,WP ,n,WQ,n)(A,Aˇ, Θ,WP,WQ)

and that the limit is a centered Gaussian process in D(T;R^s)^m⊗2×R^p⊗3. Let Γ =E(ΘW_P) and set a(t )= E{A(t )W_P}anda(t )ˇ =E{ ˇA(t )W_Q}for everyt∈T.Then,asn→ ∞,

An,A^∗_n,Aˇn,Aˇ^∗_n,Aˇ^∗∗_n , Θ_n, Θ_n^∗

A,A,Aˇ,Aˇ,Aˇ, Θ, Θ

inD(T;R^s)^⊗5×R^p⊗2.In the limit,

A=A^⊥+aΘ, Θ=Θ^⊥+Γ Θ, Aˇ= ˇA^⊥+ ˇaΘ, Aˇ= ˇA^⊥⊥+ ˇaΘ,

where(A^⊥, Θ^⊥)is an independent copy of(A, Θ).In addition,the processesA,ˇ Aˇ^⊥andAˇ^⊥⊥are mutually indepen- dent and identically distributed,as well as independent ofA,A^⊥,ΘandΘ^⊥.Moreover if(A_n, θ_n)isP-regular for A×OandAˇ_nisQ-regular forA,then

G^An^ˇ∗,G^An^ˇ∗∗

G^Aˇ∗,G^Aˇ∗∗

=

A− ˇA^⊥− ˙AΘ,A^⊥− ˇA^⊥⊥− ˙AΘ^⊥

inD(T;R^s)^⊗2,asn→ ∞,andG^Aˇ∗ is an independent copy ofG^Aˇ∗∗. 4. Examples of application

In this section, the validity of the one- and two-level parametric bootstrap is established in four common goodness-of- fit testing contexts. The first example considers classical tests for parametric families of random vectors; it is discussed here because the conditions under which Theorems 1 and 2 are established seem easier to verify than the requirements imposed by Stute et al. [26]. The second and the third examples are about goodness-of-fit for copula models, while the last application revisits the approach of Durbin [9] for goodness-of-fit testing of parametric families of random vectors using the probability integral transformation.

4.1. Goodness-of-fit tests for parametric families

LetXbe ad-variate random vector with continuous distribution functionF. Suppose that it is desired to test the null hypothesis

H₀: F∈F= {F_θ: θ∈O},

i.e.,F =Fθ₀ for someθ0∈O. Given a random sampleX1, . . . , Xn fromF, a natural procedure is to compare the empirical distribution function (1) toFθ_n, whereθn=Tn(X1, . . . , Xn)is an estimation of the unknown parameter θ∈R^p. The test could be based, e.g., on a Cramér–von Mises or on a Kolmogorov–Smirnov functionalSn=φ (G^Fn) of the empirical process

G^Fn =n^1/2(Fn−Fθ_n).

To establish the validity of the parametric bootstrap for such statistics, one can use Theorems 1 and 2 withA_θ=F_θ andP_θ standing for the unique probability measure associated withF_θ and density f_θ. Assume thatP= {P_θ: θ∈ O} ∈S(λ), whereλis Lebesgue’s measure. Introduce the following notation:

f =f_θ0, f˙= ˙f_θ0, f¨= ¨f_θ0.

To check theP-regularity ofF, letFn=n^1/2(F_n−F )and WF,n=n^−1/2

n

i=1

f˙(X_i)

f (X_i) . (18)

Results from [5] imply that asn→ ∞,(Fn,WF,n)(F,WF)inD([−∞,∞]^d;R)×R^p, whereWF is a centered Gaussian variable with variance

I_F =

f˙(x)f (x)˙ f (x) λ(dx)

andFis anF-Brownian bridge, i.e.,Fis a continuous centered Gaussian process with covariance function cov

F(x),F(y)

=F (x∧y)−F (x)F (y),

wherex∧y=min(x, y)for allx, y∈R^d. The following result is a consequence of these observations and the fact that for allx∈R^d,

E

F(x)W_F

=

f (y)1(y˙ ≤x)λ(dy)= ˙F (x)

in view of Eq. (12).

Proposition 1. LetX₁, . . . , X_nbe a random sample from distributionF=F_θ0for someθ₀∈O.IfP∈S(λ),then the canonical empirical distribution functionF_ndefined in(1)isP-regular forF.

Next, assume thatθnis aP-regular sequence forOsuch that, asn→ ∞, (Fn, Θn,WF,n)(F, Θ,WF)

in D([−∞,∞]^d;R)×R^p⊗2. Suppose further that the limit is Gaussian, so that condition (15) is satisfied with An=Fn. It then follows that (Fn, θn)is P-regular forF×Obecause E(FW_F)= ˙F = ˙Fθ0 by Proposition 1 and E(ΘW_F)=I by the regularity hypothesis onθ_n.

Finally, all the conditions of Theorems 1 and 2 are met withA=F,An=Fn andAˇn= ˇFn, where the latter is defined for allx∈R^dby

Fˇ_n(x)= 1 m

m

i=1

1(Yi≤x)

in terms of a random sampleY₁, . . . , Y_mfromP_θ that is independent ofX₁,. . . , X_n. Therefore, the one- and two-level parametric bootstraps yield valid approximations of the distribution of any continuous functionalS_n=φ (G^F_n).

In this context, the class of estimators that areP-regular forOis broad, as shown below.

Definition 3. An estimatorθ_n=T_n(X₁, . . . , X_n)forθ∈Ois said to belong to classRifn^1/2(θ_n−θ )=Θ_n+oP(1), where

Θ_n =n^−1/2 n

i=1

J_θ(X_i) (19)

is expressed in terms of a score functionJ_θ:R^d→R^pthat is square integrable with respect toP_θ and such that for allθ∈O,one has both

Eθ

J_θ(X)

=

J_θ(x)f_θ(x)λ(dx)=0 and

J_θ(x)f˙_θ(x)λ(dx)=I. (20)

Proposition 2. Letθ_n=T_n(X₁, . . . , X_n)be an estimator ofθ∈Ofrom the classR.If P∈S(λ),then(F_n, θ_n)is P-regular forF×O.

To establish this result, first note that each component of the vector(Fn, Θ_n,WF,n)is tight and that the finite- dimensional distributions converge by the classical multivariate central limit theorem, because each term is a sum of independent and identically distributed centered random variables. In addition, observe thatE(ΘW_F)=I by Eq. (20).

Example 1. When it is uniquely defined andI_F is non-singular,the maximum likelihood estimator belongs toR.For, in that case,relation(19)holds withJθ=I_F⁻¹f˙_θ/fθ.Furthermore,this function satisfies conditions(20)because of identity(11)and from the fact that underP =P_θ0,

E ΘW_F

=I_F⁻¹

f˙(x)f (x)˙

f (x) λ(dx)=I_F⁻¹I_F =I.

Example 2. Moments estimators also belong toR.Assume that θ=g(μ) and μ=

M(x)f_θ(x)λ(dx)

for some integrable functionM:R^d→R^dthat does not depend onθ.Suppose also thatgis continuously differen- tiable and that the matrixg˙of derivatives is non-singular.Theng⁻¹exists and is continuously differentiable by the inverse function theorem.Furthermore,Slutsky’s theorem implies that for allx∈R^d,

Jθ(x)= ˙g

g⁻¹(θ )

M(x)−g⁻¹(θ ) .

This score function meets the appropriate requirements because of(11)and the fact that underP, E

ΘWF

= ˙g

g⁻¹(θ0) h(x)f (x)˙ f (x)λ(dx)

= ˙g

g⁻¹(θ₀) ∂

∂θ

M(x)f_θ(x)λ(dx)

θ=θ0

= ˙g

g⁻¹(θ₀) ∂

∂θg⁻¹(θ )

θ=θ₀

=I.

Example 3. When it is uniquely defined,the estimatorθ_nminimizing

n(θ )= Fn(x)−Fθ(x)2

dFn(x)

betweenFnandFθalso belongs toR,provided that Σ_θ=

F˙_θ(x)F˙_θ(x)f_θ(x)λ(dx)

is non-singular for everyθ∈O.In this case,representation(19)holds with J_θ(x)=Σ_θ⁻¹ 1(x≤y)−F_θ(y)F˙_θ(y)f_θ(y)λ(dy)