Estimation and tests in finite mixture models of nonparametric densities

DOI:10.1051/ps:2008014 www.esaim-ps.org

ESTIMATION AND TESTS IN FINITE MIXTURE MODELS OF NONPARAMETRIC DENSITIES

Odile PONS

Abstract. The aim is to study the asymptotic behavior of estimators and tests for the components of identifiable finite mixture models of nonparametric densities with a known number of components.

Conditions for identifiability of the mixture components and convergence of identifiable parameters are given. The consistency and weak convergence of the identifiable parameters and test statistics are presented for several models.

R´esum´e. Dans les mod`eles de m´elanges de densit´es non param´etriques, une question est de d´eterminer le comportement asymptotique d’estimateurs et de statistiques de test sur les composantes identifiables.

Des mod`eles de m´elanges non param´etriques d’un nombre connu de densit´es sont consid´er´es. Des con- ditions pour l’identifiabilit´e et pour les convergences des param`etres et fonctions identifiables sont pr´esent´ees. Le comportement des statistiques de test est d´ecrit et des estimateurs des composantes des densit´es sont d´efinis dans plusieurs cas.

Mathematics Subject Classification. 62G10, 62H10.

Received April 25, 2007. Revised January 16, 2008.

1. Introduction

Consider a real random variableX on a probability space (Ω, A, P₀) andF a family of densities with respect to a probability μ on (Ω, A), such that the density f₀ = dP₀/dμ belongs to F and the functions of F are L₂(μ)-integrable, it is a metric space with the metricd₂(f, f) =f−fL2(μ).

For a fixed numberp, the mixture ofpcomponents ofF is defined by a vector ofpunknown densitiesf_(p)= (f₁, . . . , f_p) ofF^p with unknown proportions in Sp={λ_(p)= (λ₁, . . . , λ_p)∈]0,1[^p;_p

j=1λ_j = 1}. The density of the mixture isg_f(p),λ(p) =_p

j=1λ_jf_jand we consider the functional setGp={gf(p),λ(p);f_(p)∈ F^p, λ_(p)∈ Sp}.

The densities and the proportions of a mixture g_f(p),λ(p) of Gp are identifiable if for anyf_(p) inF^p andλ_(p)in Sp, the equality _p

j=1λ_jf_j =_p

j=1λ_jf_j is satisfied μ-a.s. if there exists a permutation π of{1, . . . , p} such thatλ_j=λ_π(j) andf_j=f_π(j) μ-a.s., forj= 1, . . . , p. The identifiability condition is written as a condition on the parameters of a nonparametricF, it requires that the densities cannot be confounded and a mixture of two componentsf₁ andf₂ ofF does not belong toF.

For a parametric mixture of two componentsg=λf+ (1−λ)f₀ofG2, the likelihood ratio test (LRT) for the hypothesisH₀: g=f₀ is studied through a reparametrization because the information matrix is not positive

Keywords and phrases. Mixture models, nonparametric densities, test, weak convergence.

1 pons.odile@free.fr

Article published by EDP Sciences c EDP Sciences, SMAI 2009

definite and either λ or f may be considered as nuisance parameters under H₀. Conditions and proof of the convergence of the LRT asf₀is known (admixture or contamination model) were given in [4,5] for parametric families of two or p≥2 components, for a test of a known density ofGq againstGp,q < p. The results were generalized to an unknown densityf₀by plugging an estimator off₀in the same statistic, for identifiable families, and they are easily adapted for testing sub-models of the semi-parametric mixture g=_p

j=1λ_jf(η(· −μ)),f inF andθ= (η, μ) in a Borel compact subset ofR². The methods are extended here to nonparametric density familiesG2with identifiable components, the parameter of interest for an admixture isλd₂(f, f₀) which is zero if and only ifH₀holds, for a mixture it isλd₂(f₁, f₂), withλ≤1/2. Wald’s method [8] for the proof of consistency for the parametric case is no more sufficient and the asymptotic distribution is not Gaussian, as it was known for parametric distributions. The conditions are sufficient to remove the assumption of uniform convergence of small order terms present in most papers about expansions of the LRT in parametric models [1–4,6]. Under the alternative, direct estimators of λ and f are built and they satisfy the usual convergence. The tests are extended to finite mixtures of nonparametric densities and the estimators to mixtures of two nonparametric symmetric densities.

2. Admixture models for a density

Let (X₁, . . . , X_n) be an i.i.d. sample with density having two mixture componentsg=λf+ (1−λ)f₀inG_2,0, the subset ofG₂of the mixtures with a knownf₀. Denoted_χ2(f, f) =

(f−f)²f⁻¹dμand assume that A1: λandF are identifiables: ifλf+ (1−λ)f₀=λf+ (1−λ)f₀,μ-a.s., withλandλ in [0,1],f and

f in F, thenλ=λ andf =f,μ-a.s., A2: There exists a function ϕ₀ inL₂(P₀) s.t.

lim sup

d2(f,f0)→0d⁻¹₂ (f, f₀)f₀⁻¹(f−f₀)−d₂(f, f₀)ϕ_0L2(P0)→0.

Forf inF andλ∈]0,1[, denoteU ={d2(f, f₀);f ∈ F} ⊂R+,

α_f =λd₂(f, f₀), ϕ_f =d⁻¹₂ (f, f₀)f₀⁻¹(f−f₀) iff =f₀, (2.1) and ϕ_f =ϕ₀ defined byA2if f =f₀, DF ={ϕf;f ∈ F}, the L₂(P₀)-closure defined using A2of the set of functions ϕ_f. LetE0the expectation underP₀, further conditions are

A3: E₀sup_f∈F|logf|<∞andE₀sup_ϕ∈DFϕ(X)³<∞, A4: sup_ϕ∈DF|n^−1/2_n

i=1ϕ(X_i)−G_ϕ|tends to zero inP₀-probability, withG_ϕa centred Gaussian process with variance Σ_ϕ=E0ϕ²(X) and covariance Σ_ϕ1,ϕ2=E0ϕ₁(X)ϕ₂(X).

Remark 1. TheL₂(P₀)-norm of the convergence in A2 is a d_χ2-derivability of F at f₀. It is not symmetric with respect to its arguments f and f₀, unlike the L₂ distance with respect to μ, and it is weaker than the Hellinger-derivability ofF. It appears naturally for the convergence under the null hypothesis of the variance of the likelihood ratio after the reparametrization. The functionϕ₀is related to the direction of the approximation off₀ byf.

Remark 2. By A2–A3 and the definition of DF,Eϕ(X) = 0 and the weak convergence ofn^−1/2_n

i=1ϕ(X_i) holds for everyϕofDF, and forn^−1/2_n

i=1{sup_DFϕ(X_i)−E₀sup_DFϕ(X)}by integrability of sup_DFϕ²(X).

Condition A4 is satisfied under a stronger condition on the dimension ofDF.

2.1. Tests for the density f

Under the alternative, a density is written

g_λ,f =f₀{1 +λf₀⁻¹(f −f₀)}=f₀(1 +α_fϕ_f), α_f ≥0.

The parameters set for a test off₀against an admixture alternative is{αf ≥0;f ∈ F}and the hypothesisH₀ is equivalent to the existence of a functionf in F such thatα_f = 0, or inf_f∈Fα_f = 0. For a sample (X_i)_1≤i≤n ofX, the LRT statistic is written

T_n = 2 sup

λ∈]0,1[,sup

f∈F

1≤i≤n

{logg_λ,f(X_i)−logf₀(X_i)}

= 2 sup

α∈U, sup

ϕ∈DF

1≤i≤n

l_n(α, ϕ)

withl_n(α, ϕ) =

1≤i≤nlog{1 +αϕ(X_i)}.

Lemma 2.1. The estimatorα_n(ϕ) = arg max_α∈Ul_n(α, ϕ)converges P₀-a.s. to zero, uniformly onU.

Proof. For anyϕ∈DF, the MLEg_ϕ,n=f₀{1 +α_nϕ} of a densityg_α,ϕ=f₀{1 +αϕ}, withα≥0 andα_n ≥0 converges at the parametric rate, therefore sup_R|g_ϕ,n−f₀| ≤sup_R|g_n−f₀|for any other estimatorsg_nwithout this parametric model. For everyε >0 there existsδ_ε>0 s.t. fornlarge enough

P₀{ sup

ϕ∈DFα_n(ϕ)> ε} ≤ P₀{ sup

ϕ∈DFn⁻¹{ln(α_n(ϕ), ϕ)−l_n(0, ϕ)}> δ_ε}

≤ P₀{

f0>0 sup

ϕ∈DFlog{1 +α_n(ϕ)ϕ}dP_n> δ_ε}, for the empirical d.f. P_n ofP₀. Let ¯g_n= sup_ϕ∈DFf₀{1 +α_n(ϕ)ϕ}, then

P₀{ sup

ϕ∈DFα_n(ϕ)> ε} ≤ P₀{

f0>0logg¯_n

f₀ dP_n> δ_ε}

≤ P₀{

f0>0logg_n

f₀ dP_n> δ_ε}

≤ P₀{

f0>0logg_n

f₀ d(P_n−P₀)> δ_ε}

and this sequence tends to zero by the weak convergence of P_n.

LetY_n the process defined byY_n(ϕ) =n^−1/2_n

i=1ϕ(X_i), with variance-covariance Σ,Z_n= Σ^−1/2Y_n andZ be a centred and continuous Gaussian process onDF, with covariance Σ^−1/2ϕ1 Σ_ϕ1,ϕ2Σ^−1/2ϕ2 .

Theorem 2.2. UnderH₀,sup_ϕ∈DFn^1/2α_n(ϕ)−(Σϕ)^−1/2Z(ϕ)1{Z(ϕ)>0}converges in probability to zero and T_n converges weakly to the variable sup_ϕ∈DFZ²(ϕ)1{Z(ϕ)>0}.

Proof. LetϕinDF, ˙l_n(·, ϕ) the derivative ofl_n(·, ϕ) with respect toαandα_n(ϕ) under the constraintα_n ≥0, thenα_n(ϕ) = 0 or it is solution of

0 = n^−1/2l˙_n(ϕ,α_n(ϕ))1_{αn(ϕ)>0}

= n^−1/2

i

a_i(ϕ)

1 +α_n(ϕ)a_i(ϕ)1_{αn(ϕ)>0}. (2.2) For everyϕ∈DF, leta_i≡a_i(ϕ) =ϕ(X_i) andR_1n =n⁻¹

ia³_i(1 +α_na_i)⁻¹. Ifα_n(ϕ)>0, the score equation is simply written

0 = Y_n(ϕ)−n^1/2α_n(ϕ){n⁻¹

i

a²_i(ϕ)−α_n(ϕ)R_1n(ϕ)}. (2.3)

Under sup_ϕ∈DF|n⁻¹

i{ϕ²(X_i)−Σ_ϕ}|convergesP₀-a.s. to zero and (2.3) becomes 0 =Y_n(ϕ)−n^1/2α_n(ϕ){Σϕ+o_p(1)−α_n(ϕ)R_1n(ϕ)}. As proved in [7]:

n^1/2α_n(ϕ) = (Σ_ϕ)^−1/2Z_n(ϕ)1_{Zn(ϕ)>0}+o_p(1). (2.4) By integration of R_1n,l_n(α_n(ϕ), ϕ) =n^1/2α_n(ϕ)Y_n(ϕ)−¹₂α²_n(ϕ)

iϕ²(X_i) +R_2n(ϕ) where R_2n =

i

_αn(ϕ)

0

a³_iα²

1 +αa_idα=

i

α_n α^∗2_n a³_i 1 +α^∗_na_i

with α^∗_n(ϕ) between 0 and α_n(ϕ). As the functions α → a³_i(1 +αa_i)⁻¹ are decreasing and from the bound established for R_1n,n⁻¹|R_2n| ≤α³_nn⁻¹|

ia³_i|. Then sup_DFn⁻¹|R_2n(ϕ)|=o_p(1) and l_n(α_n(ϕ), ϕ) =n^1/2α_n(ϕ)Y_n(ϕ)−n

2α²_n(ϕ)Σ_ϕ+o_p(1)

uniformly on DF. The expression (2.4) of α_n(ϕ) implies a uniform approximation of T_n as T_n = 2 sup_ϕ∈DF

l_n(α_n(ϕ), ϕ) = sup_ϕ∈DF[Z_n²(ϕ)1_{Zn(ϕ)>0}] +o_P(1).

With p additional components in the mixture under the alternative, the metric is written d₂(f, f) = _p

k=1f_k−f_kL2(μ) for two densities f = (f₁, . . . , f_p) and f = (f₁, . . . , f_p). The density g_λ,f = (1−λ)f₀+ _p

k=1λ_kf_k becomes

g_λ,f =f₀{1 + p k=1

λ_kf₀⁻¹(f_k−f₀)}=f₀(1 +α^T_fϕ_f), where the coefficient off₀ satisfiesλ=_p

k=1λ_k, α_f = (α_{f k})_1≤k≤p andϕ_f = (ϕ_{f k})_1≤k≤p are given by

α_{f k} =λ_kd₂(f_k, f_0k), ϕ_{f k} =d⁻¹₂ (f_k, f_0k)f_0k⁻¹(f_k−f_0k) iff =f₀, (2.5) and ϕ_0k =f_0k⁻¹f_0k . The hypothesisH₀is equivalent to αf= 0 and the result of Lemma 2.1 still holds. The process Y_n(ϕ) is ap-dimensional process with a diagonal variance Σ = (Σ_ϕk)_1≤k≤p. and Theorem 2.2 adapts for each component with the same proof, using tensor product and norms of the vectors already defined when necessary. Denoting 1{Z(ϕ)>0}for the vector of indicators 1_{Zk(ϕ)>0}, we obtain

Theorem 2.3. UnderH₀,sup_ϕ∈DF|n^1/2α_n(ϕ) − (Σϕ)^−1/2Z(ϕ)1{Z(ϕ)>0}|converges in probability to zero andT_n converges weakly to the variable sup_ϕ∈DFZ(ϕ)²1{Z(ϕ)>0}.

Whenf₀is already a known mixture of functions inF, the result is extended as in [5] for mixtures of several parametric densities.

2.2. Estimation of the densities and mixture coefficients

Under the alternative, λ₀ ∈ {0,1} and the model for the mixture density is g₀ = λf+ (1−λ)f₀ where λ belongs to a close subset of ]0,1[. We consider a class of functionsFwith compact supports not all confounded with the support suppF₀ of the distribution function F₀, then both λ₀ and f are identifiable and may be estimated. If all the functions of F have the same support, further conditions are necessary for identifiability of the components ofg₀. Estimators ofλandf are obtained after the estimation of the different supports.

LetG_nthe empirical estimator of the d.f. of the variableX, with support suppG_n= [min_iX_i,max_iX_i], and G_n(x) =n⁻¹

n i=1

1_{{Xi∈suppF0}}1_{Xi≤x}−F₀(x)

a consistent estimator of (G₀−F₀)1{suppF∩suppF0} = λ(F −F₀)1_{{suppF∩suppF0}}, with support estimated by suppG_n. An estimator ofλ₀ is deduced from a classification of the observations into three intervals,

I_1n = suppG_n, I_2n = suppF₀\suppG_n, I_3n= suppG_n\suppF₀. The following identities

I2n

dG_n = (1−λ)

I2n

dF₀,

I1n

1_x≤tdG_n(x) = λ

I1n

1_x≤tdF_n(x) + (1−λ)

I1n

1_x≤tdF₀(x),

I3n

1_x≤tdG_n(x) = λ

I3n

1_x≤tdF_n(x),

imply simple expressions for the estimators ofλ₀ andF onI_1n andI_3n λ_n = 1−

I1ndG_n

I1ndF₀, (2.6)

I1n

1_x≤tdF_n(x) = λ⁻¹_n

I1n

1_x≤tdG_n(x)−

I1ndG_n

I1ndF₀

I1n

1_x≤tdF₀(x) ,

I3n

1_x≤tdF_n(x) = λ⁻¹_n

I3n

1_x≤tdG_n(x).

LetK a symmetric kernel inL₂(μ) andha window,g_n(x) =n⁻¹_n

i=1K_h(x−X_i) an estimator ofg₀, then the unknown density of the mixture is estimated by

f_n(x) = λ⁻¹_n {g_n(x)−

I1ndG_n

I1ndF₀f₀(x)}, ifx∈I_1n,

f_n(x) = λ⁻¹_n g_n(x), ifx∈I_3n. (2.7)

Assume that the support ofGis bounded andσ_1,λ² =

x²d(1−G)ⁿ(x)−(

xd(1−G)ⁿ(x))²<∞if inf suppF₀<

inf suppF, and σ_2,λ² =

x²dGⁿ(x)−(

xdGⁿ(x))² <∞ if inf suppF₀ >inf suppF, then n^1/2(max_iX_i− max SuppG) and n^1/2(min_iX_i−min SuppG) converge weakly to centered Gaussian variables with variances σ²_1,λandσ²_1,λ. It follows that:

Proposition 2.4. The estimators λ_n,F_n andf_n are uniformly P₀-a.s. consistent. The variable n^1/2(λ_n−λ) converges weakly to a centered Gaussian variable, n^1/2(F_n−F) converges weakly to a transformed Brownian bridge and if F is a subset of C₂ and h = o(n^−1/5), n^2/5(f_n−f) converges weakly to a centered Gaussian process.

3. Mixture model of nonparametric densities

Consider a mixture of two unknown distributions with densities in a family F of concave unimodal and symmetric densities onR,

g_λ,f1,f2 =λf₁+ (1−λ)f₂.

An identifiable mixture of two symmetric densities of F does not belong to F even with overlapping densities since the mixture density is not concave or not symmetric, except when they have the same mean and in that case they are unidentifiable. LetH₀the null hypothesis of a single unknown distribution ofF. The distribution f₀ under H₀ may be estimated by a symmetric estimatorf_0n: let θ₀ the center of symmetry of f₀, estimated by the median θ_n of the sample and consider the transformed sample X_i =X_i ifX_i ≥θ_n, X_i = 2θ_n−X_i if X_i<θ_n hence (X₁, . . . , X_n) is a sample with asymptotic distributiona⁻¹₀ F₀1_]−∞,θ0], witha₀=_θ0

−∞dF₀. The densityf₀ is estimated by a kernel estimatorf_0n(x) = n⁻¹_n

i=1K_h(x−X_i) on ]− ∞,θ_n] and by symmetry, f_0n(x) =f_0n(2θ_n−x) on ]θ_n,∞[, it isP₀-a.s. uniformly consistent onR.

In the following, any other constraint on the form of the set F may obviously replace the symmetry, for example a disymmetry with some proportions between both sides of the functions. The main condition aboutF is the identifiability of the components of mixtures of functions belonging toF and mixtures cannot themselves belong to F. UnderH₀, any estimatorf_0n of the single unknown density f₀ with the relevant constraint may be used.

A test of a mixtureg_λ,f1,f2 =λf₁+ (1−λ)f₂in G₂, withf₁ andf₂in F, against the alternative of a single unknown distribution ofF may be performed by replacingf₀ byf_0n in the expression ofT_n, with the statistic

S_n = 2 sup

λ∈]0,1/2],sup

f∈F

1≤i≤n

{logg_λ,f,f

0n(X_i)−logf_0n(X_i)}.

The conditions for convergence ofS_n are

A1: λand F are identifiable: if λf₁+ (1−λ)f₂=λf₁ + (1−λ)f₂, μ-a.s., withλand λ ∈]0,1/2],f₁, f₂,f₁ andf₂ ∈ F, thenλ₁=λ₂,f₁=f₁ andf₂=f₂,μ-a.s.,

A2: for allf inF, theL₂(P₀)-derivative atf,ϕ_f exists:

lim sup

d2(f,f)→0d⁻¹₂ (f, f)f⁻¹(f−f)−d₂(f, f)ϕ_fL2(P0)→0.

For all f and f in F, denote α_f,f = λd₂(f, f), ϕ_f,f = d⁻¹₂ (f, f)f⁻¹(f−f) if f = f and ϕ_f,f = ϕ_f, Uf={d₂(f, f);f ∈ F} ⊂R+;DFf ={ϕ_f,f;f ∈ F}andDF ={ϕ_f,f;f, f∈ F}.

A3: E0sup_λ∈]0,1],sup_f1,f2∈F|logλf₁+ (1−λ)f₂|<∞and there exists a sequence of neighborhoodsV_0n off₀in F containing (f_0n)_n and converging tof₀s.t.

E0 sup

f0n∈V0n,

sup

ϕn∈DFf0n

ϕn(X)³<∞,

A4: sup_f0n∈V0n,sup_ϕn∈DFf

0n|n^−1/2_n

i=1ϕ_n(X_i)−G_ϕ|converges inP₀-probability to zero, withG_ϕde- fined as in A4.

The mixture density is written g_λ,f,f = λf + (1−λ)f = f(1 +α_f,fϕ_f,f) and the parameter set for a test of H₀ is {αf,f;f, f ∈ F}. A sequence of parameters of smaller size is sufficient for the test with the statistic S_n, it is defined by {(α_f,f0n)_n;f ∈ F}. Let u_n,f = u(f,f_0n), α_n,f = λu_n,f, ϕ_n,f = ϕ(f,f_0n), and g_n,f =f_0n(1 +α_n,fϕ_n,f),Un={un,f;f ∈ F}andDFn={ϕn,f;f ∈ F}. The statisticS_n is then written

S_n= 2 sup

αn∈Un/2, sup

ϕn∈DFn

log(1 +α_nϕ_n).

Under conditions A, the proofs of Section 2 are easily extended, with the same notations.

Lemma 3.1. For every ϕ_n inDFn, the estimator

α_n(ϕ_n) = arg max

αn∈Un/2l_n(α, ϕ_n) is such that lim sup_ϕn∈DFn|α_n(ϕ_n)| converges P₀-a.s. to zero.

Theorem 3.2. Under H₀, sup_ϕn∈DFn|n^1/2α_n(ϕ_n)−(Σ_ϕ)^−1/2Z₍ϕ)1_{Z(ϕ)>0}| converges in probability to zero and the statisticS_n converges weakly tosup_ϕ∈DFZ²(ϕ)1{Z(ϕ)>0}.

As in Section 2, the convergence rate is due to the sum of n i.i.d. variables and does not depend on the convergence rate ofϕ_n.

With a vector ofpadditional components f = (f₁, . . . , f_p) to an unknown density f with true value f₀ in the mixture under the alternative,g_λ,f,f = (1−λ)f +_p

k=1λ_kf_k in Gp is written with the notations ofA₂ as g_λ,f,=f₀(1 +α^T_f,fϕ_f,f),

the hypothesis H₀ is equivalent to αf = 0 and the results of Lemma 2.1 and Theorem 2.3 hold. They are extended to a model with a mixture of identifiable components underH₀, as in Section 2.

Under the alternative of a mixture of two functions ofF with different supports, the components of the true densityg₀=λ₀f_1,0+ (1−λ₀)f_2,0may be estimated. For densities with known or estimated supports (densities with estimable center of symmetry), λ₀ is estimated as in (2.6) and both symmetric densities are estimated by an estimator similar to f_0n defined in this section. If only one center of symmetry of f_1,0 or f_2,0 may be estimated, one density is estimated by this method and the other density is defined by (2.7) wheref₀is replaced byf_0n. If both supports are unknown, a learning sample is necessary for the estimation of the support of the densities.

References

[1] J.M. Aza¨ıs, E. Gassiat and C. Mercadier, Asymptotic distribution and local power of the likelihood ratio test for mixtures:

bounded and unbounded cases.Bernoulli12(2006) 775–799.

[2] K. Fukumizu, Likelihood ratio of unidendifiable models and multilayer neural networks,Ann. Statist.31(2003) 833–851.

[3] J.K. Ghosh and P.K. Sen, On the aymptotic performance of the log-likelihood ratio statistic for the mixture model and related results. InProc. of the Berkeley conference in honor of Jerzy Neyman and Jack Kiefer, volume II, edited by L.M. Le Cam and R.A. Olshen, (1985), pp. 789–806.

[4] M. Lemdani and O. Pons, Likelihood ratio tests in mixture models.C. R. Acad. Sci. Paris, Ser. I322(1995) 399–404.

[5] M. Lemdani and O. Pons, Likelihood ratio tests in contamination models,Bernoulli5(1999) 705–719.

[6] X. Liu and Y. Shao, Asymptotics for likelihood ratio tests under loss of identifiability,Ann. Statist.31(2003) 807–832.

[7] O. Pons,Estimation et tests dans les mod`eles de m´elanges de lois et de ruptures. Herm`es-Science Lavoisier, London and Paris (2009).

[8] A. Wald, Note on the consistency of the maximum likelihood estimate,Ann. Math. Statist.20(1949) 595–601.