Limit Theorems for Stochastic Approximations Algorithms With Application to General Urn Models

HAL Id: hal-01726014

https://hal.archives-ouvertes.fr/hal-01726014v3

Preprint submitted on 7 May 2018

HAL

is a multi-disciplinary open access archive for the deposit and dissemination of sci- entific research documents, whether they are pub- lished or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L’archive ouverte pluridisciplinaire

destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Limit Theorems for Stochastic Approximations Algorithms With Application to General Urn Models

Soumaya Idriss, Nabil Lasmar

To cite this version:

Soumaya Idriss, Nabil Lasmar. Limit Theorems for Stochastic Approximations Algorithms With

Application to General Urn Models. 2018. �hal-01726014v3�

Limit Theorems for Stochastic Approximations Algorithms With Application to General Urn Models.

Soumaya Idriss

, Nabil Lasmar

May 7, 2018

Abstract

In the present work we study the multidimensional stochastic approximation algorithms where the drift function his a smooth function and where jacobian matrix is diagonalizable over Cbut assuming that all the eigenvalues of this matrix are in the the region Repzq ą0. We give results on the fluctuation of the process around the stable equilibrium point ofh. We extend the limit Theorem of the one dimensional Robin’s Monroe algorithm [MR73]. We give also application of these limit Theorem for some class of urn models proving the efficiency of this method.

1 Introduction

1.1 The Stochastic Approximation Algorithms

A stochastic approximation algorithm (SAA) is a sequencepXnqně0of anR^d´valued random vectorpdě1q, defined on some probability space`

Ω,F, P˘

by the following recursion:

X<sub>n`</sub>1“Xn`1{γ_n`1hpXnq (1)

where pγnqně1 is a real sequence and h : R^d ÝÑ R^d is called a drift function. The one dimensional (SAA) was first introduced by Robbins and Monroe [RM51] where they consideredγ_n as a sequence of positive constants such that ÿ

ně1

1{γn “ 8 and ÿ

ně1

1{γ_n² ă 8, they also viewed h as a monotonous function. Their aim was to find the zero θ of the function h which was assumed to beă 0 for xăθ and hpxq ą 0 for xą θ. Blum [Blu54] proved the almost sure convergence ofXn toθ under suitable conditions. Several studies on (SAA) were exhibited, most of them deal with the fluctuation of the stochastic approximation procedure around the equilibrium point. For instance, Chung who was the first to provide a central limit Theorem for the sequenceX_n´θ[Chu54] resorting to the method of moments, whereX_n is a real valued process. Due to some complications arising from the us of the moments’ method, Chung’s work was simplified by both Sacks [Sac58]and Fabian [Fab68] and was generalized to a multidimensional version of CLT. The standard adapted conditions state that the functionhshould be a smooth function and that all eigenvalues of∇hpθqare to be with real partsě 1

2. The case where those eigenvalues are with real partsă 1

2 was investigated by Major and R´ev´esz [MR73] in the one dimensional case, following Blums’ [Blu54]

conditions and using Sack’s ideas. In this paper, relying on Renluund’s one dimensional version of the (SAA), we exhibit a multidimensional (SAA) in order to provide thed-dimensional extensions of the limit Theorems found in [MR73] and that by relaxing some of the conditions required in [Fab68, Ren11].

Definition 1. A multivariate stochastic approximation algorithmpZ_nqně0is a stochastic process on some probability spacepΩ,F, Pqtaking values in the cube r0,1s^d and adapted to a filtration pF_nqně0 and satisfying

Z<sub>n`1</sub>“Z<sub>n</sub>`1{γ<sub>n`1</sub>`

hpZ_nq `Y<sub>n`1</sub>˘

, (2)

whereYn, γn PF_n, h:r0,1s^dÝÑR^d and the following conditions holds almost surely:

pS1qcℓ{nď1{γnďcu{n, pS2q›

›Yn

›

›₂ďKY, pS3q›

›hpZnq›

›₂ďKf, pS4q›

›E`

γ_{n`</sub>1Y<sub>n`}1

ˇ ˇF_n˘

}²ďKe{n² wherec_ℓ, c_u, K_f, K_e, K_y are positive constants and denoting be} }² the Euclidian norm inR^d.

∗Département des Mathématiques, Faculté des Sciences de Monastir, Monastir, Tunisia (maysoun-@hotmail.fr).

†Département des Mathématiques, Institut Préparatoire aux Études d’Ingénieur, Monastir, Tunisia (nabillasmar@yahoo.fr).

The processpY<sub>n`1</sub>qně0 called the noise of (SAA) and is broadly assumed to be a martingale difference sequence (meaningE`

Y_n`1ˇ ˇF_n˘

“0), that isY_n`1 is uncorrelated with the past of the process.

Among methods to find zeros of the drift functionhis the ordinary differential method (o.d.e method). This method consists to find the stable equilibrium points of the ordinary differential equation

x¹“hpxptqq

Recall that the Euler method (deterministic) for numerical approximation of the trajectory ofx¹ptq “hpxptqqwould be establishing the following discrimination

Z<sub>n`</sub>1“Zn`1{γhpZnq

withγą0. The difference with the stochastic algorithms is that the time scaleγis replaced by a time varying step size γ_{n`1</sub>, in addition, the presence of the noiseY<sub>n`1}. In the literature we found criteria of the existence of limit points ofpZ_nqně using the differential equation method. Theses methods impose conditions on h,γ_n, for example we have a version of almost sure convergence given by [Duf97, LP13b]

Theorem 1. Consider the stochastic approximation algorithm pZ_nqně0 on r0,1s^d Z<sub>n`</sub>1“Zn`1{γ_n`1

´

hpZnq `Y<sub>n`</sub>1

¯

where h : R^d ÝÑ R^d differentiable function, pY_n`1qně0 is a martingale difference with respect with the filtration pF_nqně0, pγ_nqně0 is a sequence of positive random variables pF_nq ´measurable and satisfying the constraining conditions

ÿ

ně0

1{γn“ 8and ÿ

ně0

1{γ_n² ă 8almost surely. (3)

Then the set of adherence valuesΘ⁸ of pZ_nqně0 is a connected compact, and left stable by the flow of the ordinary differential equationΘ9 “ ´hpΘq.

Furthermore ifΘ^˚ is a uniformly stable equilibrium onΘ⁸ then ZnÝÑΘ^˚ almost surely.

1.2 General urn Models

The general Pólya urn model (GPU) is a discrete time process with reinforcement defined as follow: an urn containing initially say T0 balls of different colors, fix the number of colors to be an integerdě2. The (GPU) amounts to drawing asampleofmě1 balls from the urn at each discrete epoch of time , that is amongmsampled balls, one hasξ^piq balls of colori, iP t1, . . . , du. At then^thdraw we observe the sample and then put back the balls in the urn according to a replacement rule, determined by a mappingRdefined on the simplex

Σ^pdq_m “ pv1, . . . , vdq PN^d,

d

ÿ

j“1

vj“m(

and taking values inZ^d(meaning that the balls are to be added as well as to be removed). To ensure the durability of the process, that is regardless of the substraction of balls the urn is never empty. In other terms, the process never dies, we impose conditions (generally sufficient),on the replacement rule and on the initial composition of the urn calledtenability conditions. See for example [KM16] for the concept of tenability for linear urn models.

Let us denote by ζn “ pξ_n^p1q, . . . , ξ^pdq_n qthe vector composition of the drawing balls at the stagen, each component, i say, correspond to the number of drawn balls of color i. We denote also by Mn “ pX_n^p1q, . . . , X_n^pdqq the vector composition of the urn, and byTn the total number of balls in the urn, both afterndraws. Therefore the evolution of the urn is determined by the following recursion:

M<sub>n`1</sub>“M<sub>n</sub>`Rpζ_n`1q. (4)

IfpF_nqně0designate theσ-field generated by the firstndraws then, givenpF_nq,ζ_n follows a multivariate hyperge- ometric distribution. Note that in the with-replacement case, one has

P`

ζ_n`1“ pv1, . . . , v_dqˇ ˇF_n˘

“ 1

`_Tn

m

˘

d

ź

i“1

ˆMn^piq

vi

˙ ,

while in the without-replacement case, P`

ζ_n`1“ pv1, . . . , v_dqˇ ˇF_n˘

“

ˆ m

v1, . . . , v_d

˙ d

ź

i“1

`Z_n^piq˘vi

,

whereZ_n^piq:“ Mn^piq

T_n refers to the proportion of thei-thcolor in the urn afterndraws.

Several studies on one draw urn models used the method of Athreya and Karlin [AK68] and this method consist to embed the process into a continuous time Markov branching process. This process is determined by the same data of the urn and each ball of typeilives an exponentially distributed with mean 1 and when it dies it is replaced by rij balls of typej. Janson [Jan04] developed this method for multi-type urn model when the replacement matrix is irreducible, that is every color is dominant [Jan04] in the following sense: it is possible to find balls of any type in the urn beginning with balls of an appropriate type.

1.3 A General Urn Model Viewed as a Stochastic Approximation Algorithm

Among recursive model with reinforcement we can adapt stochastic approximation algorithm to urn model by consideringZnas the normalized vector proportion of the number of balls of each color afterndraws divided by the total number of balls. For the multicolored urn with replacement mappingRthe proportion of ballsZn is solution of the following (SAA) [LMS16]

Z<sub>n`</sub>1“Zn`1{T<sub>n`</sub>1phpZnq `∆M<sub>n`</sub>1`ǫ_n`1q where the drift functionhis given by

hpxq “ ÿ

νPΣ^pdqn

ˆ m

ν1, . . . , νd

˙ d

ź

i“1

x^ν_iⁱpRpvq ´rpvqxq

with ∆Mn`1“Y<sub>n`</sub>1´E` Y<sub>n`</sub>1

ˇ ˇF_n˘

whereY_{n`</sub>1“Rpζ<sub>n`}1´rpζ_n`1qqZ_n.

The error termǫ<sub>n`1</sub> vanishes if the urn is with replacement but in any cases it has an order of O`

1{T_n`1˘ . The tenability conditions dos not always guarantee the assumption 0ăc1ďT_n{nďc2. For example the two colors urn with replacement matrixR“

ˆ ´1 2 1 ´2

˙

with initial compositionp1,0qis tenable and we haveT_n “Op1q. For that we impose additional assumptions such as lim infT_n{ną0. The last condition is unaffectedly realized if we suppose for example that}Rpvq}¹:“

d

ÿ

j“1

R_jpvq ě1 for everyvPΣ^pdq_m.

Higueras et al [HMPM03, HMPM06] showed that the urn composition can be written as a (SAA) under some extra assumptions including the balance condition. Laruelle and Pages [LP13b, LP13a]studied the response-adaptive randomized process in clinical trials based on the randomized urn model yet studied by Bai and Hu[BH99, BH05].

Noticing that such model evolves with a time-depend replacement rule but converging almost surely to a stochastic matrix. Moreover in Laruelle and Pages in [LP13a] investigated the weighted urn witch the drawing rule is no longer uniform among the balls of the urn but the conditional probability of drawing a ball at timenis an empirical ratio of a function of the proportions of each color and this function is chosen generally with regular variation. They derives the almost sure convergence and the asymptotic normality of the vector composition of ball by allying stochastic approximations corresponding Theorems [Duf97]. In order to reduce assumption in [LP13b] Zhang [Zha16] presented a new central limit theorem for (SAA) by imposing assumptions on the error term. Renlund [Ren10, Ren11] applied results of one dimensional stochastic algorithms to the two colors urn with one or two draws by reducing Fabian’s limit Theorems [Fab68] into one dimensional process.

1.4 Main Assumptions

Our basic assumptions are as follows:

(A1) hadmits an unique stable equilibrium point ΘP r0,1s^d such that}Θ}¹“1.

(A2) his smooth and all the eigenvalues of∇hpΘqare within the region Repzq ą0( .

(A3) There exist a zero Θ ofhsuch thatZ_nÝÑΘ almost surely and there exist a realγą0 such that n{γ_n“1{γ`O`

}Z_n´Θ}²`1{n˘

. (5)

We shall also impose the following additional assumptions on the martingale differencepY_nqně0

(A4) The Lindeberg’s condition: for everyǫą0

nÝÑ8lim E`

}Yn}²2¹}Yn}2ąǫ˘

“0.

(A5) There exist a deterministic matrix ΣY ‰0 such that lim

nÝÑ8E` Y_nY_n¹ˇ

ˇF_n´1˘

“ΣY.

The Assumption (A3) has been used by Renlund [Ren11] and this assumption is satisfied by manly urn models as balanced urn. Another example is the following when the replacement mapping is an invertible matrix (and in particular irreducible).

1.4.1 Example

Assume that R is a dˆdinvertible matrix. Then by the relation 1 we obtain R^´1M_{n`</sub>1´R<sup>´1</sup>Mn “ ζ<sub>n`}1. If Z_n“ M_n

T_n converges to Θ then,

d

ÿ

j“1

“R^´1Zn‰

i“ p1{Tnq

d

ÿ

j“1

“R^´1Z0‰

i`mn{Tn

wheremě1 represents the number of drawn balls. Hence mn{Tn´

d

ÿ

j“1

“R^´1Θ‰

i“

d

ÿ

j“1

“R^´1`

Zn´Θ˘‰

i`O` 1{n˘

.

The condition (A1) dismiss the case of the convergence of the process to a non equilibrium point: If the conditions of Theorem 1 are satisfied and Θ is the unique stable zero of hthen most Theorems of the limit in distribution give results on the fluctuation ofZn´Θ. For the general urn models, the set of equilibrium points, if it exists, is generally known. In the next example that, we present an urn models for which we compute the set of stable zeros of the drift functionh.

1.4.2 Example

Consider an irreducible and multicolored urn with replacement matrixR“ pr_ijq^d_i,j“1 such that fori‰j we have r_ijě0. Here irreducibility of the urn refers to the replacement matrix being irreducible [Jan04], that is every color is a dominant one in the following sense: there existsnPN^˚ such that aftern draws, all the colors are accounted for in the urn. Perron Froebenius theory states that for largeα, since the matrixR`αIdis positive,Rhas a largest real eigenvalueλ1 such that, for all λPsppRqztλ1u, we have Repλq ăλ1.We assume in the lines of this example that λ1ą0.The drift function of the associated stochastic process is given by hpXq “RX´ |RX|¹X where|X|¹ is the trace of the vectorX.

(a) Ifλ1is simple and ifV1denotes an eigenvector associated toλ1, with positive entries. It is easy to show that Θ“ V1

|V1|¹ is a zero of hand is an isolated point in the set of zeros ofh belonging in the affine hyperplane H1“ x“ px1, . . . , x_dq PR^d, x1`. . .`x_d“1u.

Regardless of the other zeros ofh, we prove that Θ is the unique stable equilibrium point ofh. In fact, if we set A“∇hpΘqthenAH“ pR´λ1IdqH´ |RH|¹Θ. In particular´λ1 is an eigenvalue ofA with associated eigenvector´Θ.Let us writeR^d“RΘ‘E_d´1whereE_d´1is stable byAand letX PE_d´1an eigenvector of A associated to an eigenvalueµ‰ ´λ1. ThenpR´ pλ1`µqIdqX “0. Hence we haveµ`λ1 PsppRqztλ1u. Therefore, Repµq ă0.Since the set of stable zeros ofhis connected, the result follows.

(b) If λ1 is multiple, the set of zeros of hlocated onH1 defined above isZphq “kerpR´λ1I_dq XH1. We show here that Zphqis exactly the set of the stable equilibrium of h: For fixed Θ “ pθ1, . . . , θ_dq PZphq we have

∇hpΘq “R´λ1I_d´QΘwhereQΘis the matrix with rank 1 with entries“ QΘ

‰

ij“θ_i|Re_j|¹wherepe1, . . . , e_dq denotes the canonical basis ofR^d. From the relation

d

ÿ

i“1

|Re_i|¹θ_i “λ1 we deduce that Θ is an eigenvector of QΘ associate to λ1. Let PΘ a transition matrix ofQΘ, then the projection ofP_Θ^´1∇hpΘqPΘ on kerpQΘq is equal toR1´λ1I_d´1 whereR1 is thepd´1q ˆ pd´1qmatrix given byP_Θ^´1RPΘ“

ˆ λ1 0 0 R1

˙ . It is clear by this transformation that all the eigenvalues of∇hpΘqare with real partă0.

1.5 Notations

For x“ px_iqi“1,...,d P R^d, }x}² denotes the canonical Euclidian norm of the vector x, }x}¹ “

d

ÿ

i“1

x_i the trace of x. We also use the notation }A}² for an algebraic norm of the matrix AP M_dpRq. sppAq defines the set of the eigenvalues ofA. For a differentiable functionh“ ph1, . . . , h_dq, we denote by∇hpΘqits jacobian matrix with entries pB{BxjhipΘqq^d_i,j“1 where Θ is a zero ofh.

The matrix Γ“1{γ∇hpΘq and its eigenvalues will play a essential role in this paper. The eigenvalues of ∇hpΘq are in the following order

Repλ1q ě. . .Repλ_r1q ěγ{2ąRepλ_r1`1q ě. . .Repλ_rq. since∇hpθqis assumed o be diagonalizable overCwe obtain the decomposition

C^d“ ‘λPspp∇hpΘqqkerp∇hpΘq ´λIdq.

Let for λ P spp∇hpΘq, spλq be the dimension of kerp∇hpΘq ´ λIdq and V1pλq, . . . , Vsλpλq the eigenvectors of

∇hpΘq generating the eigenspace kerp∇hpΘq ´λIdq. Define the corresponding real subspace Fλ as follow: Fλ “ kerp∇hpΘq ´λIdqif this latter eigenspace is real and we setv1pλq “V1pλq, . . . vsλpλq “Vsλpλq. Otherwise we let Fλthe subspace of R^d generated by the family of vectors pvipλqq^2si“1^λ as follow:

v_2ipλq “1{2`

V_ipλq `V_ipλq˘

andv_2i´1pλq “1{2i`

V_ipλq ´V_ipλq˘ .

We have a decomposition ofR^d into a direct sum

R^d“ ‘λPspp∇hpΘqqF_λ

and the projection of ∇hpΘq in the new basis is real and diagonal by blocks . Let P1 the transition matrix of

∇hpΘqcorresponding to this decomposition. Then P₁^´1ΓP1is diagonal by blocks in the following form P₁^´1ΓP1“

¨

˚

˝ Γpλ1q

. ..

Γpλrq

˛

‹

‚

We let π1 and π2 to be the matrices of the projection on ‘λ:Repλqěγ{2F_λ and ‘λ:Repλqăγ{2F_λ respectively, Γ1 “ π1P₁^´1ΓP1 and Γ2“π2P₁^´1ΓP1. ThenP₁^´1ΓP1 “

ˆ Γ1 0 0 Γ2

˙

and by this definitionsppΓ1q “ tλ1{γ, . . . , λr1{γu andsppΓ2q “ tλr1`1{γ, . . . , λr{γu.

We associate to Γ1 (respectively Γ2) the transition matrixQ1 (respectively Q2) that diagonalizes Γ1 (respectively Γ2).

For later use, we define the sets

Λ1“ pi, jq P r1, ds², Repλ_iq ąγ{2 and Repλ_jq ąγ{2 or if Repλ_iq “γ{2 andλ_j“λ_ithenrPΣYP¹sij “0u, Λ2“ pi, jq P r1, ds², Repλ_iq “γ{2 andλ_j“λ_iandrPΣYP¹sij ‰0u

and

Λ3“ pi, jq P r1, ds², Repλ_iq ăγ{2 and Repλ_jq ăγ{2u.

Further we define the matrix ΣPM_dpRqby the entries Σij “ rQ^´1₁ π1P₁^´1ΣYP₁^1´1π1Q¹₁^´1sij

γpλ_i`λ<sub>j</sub>´γq <sup>1pi,jqPΛ1</sup>` 1

γ²rQ^´1₁ π1P₁^´1ΣYP₁^1´1π1Q¹₁^´1sij¹pi,jqPΛ1

`rQ^´1₂ π2P₁^´1ΣYP₁^1´1π2Q¹₂^´1sij

γpγ´λ_i´λ_jq ^1pi,jqPΛ3

`i, jP t1, . . . , du˘ .

Thus if Λ3“ HthenQ1“P andP1“I_d. Otherwise if Λ1YΛ2“ HthenQ2“P andP1“I_d.

1.6 Aim of the Paper

In the present work we give some theoretical results concerning the stochastic approximation algorithms following the definition 1 and satisfying the assumptions (A1)-(A5). Our motivation for the central limit Theorems is Sacks Theorem for triangular arrays [Sac58]. For the refinement of the limit Theorems we are motivated by Major’s Theorems for one dimensional stochastic approximation algorithm [MR73]. These results find numerous applications in the field of randomized models with reinforcement. We improve results on pólya urns such as balanced Pólya process with irreducible replacement rule, the two colors urn with multiple drawing and refinement of limit theorems for cyclic urns.

1.7 Organization of the Paper

The paper is organized as follow: The section 2 we give the main results concerning the limit Theorems for stochastic approximation algorithms.

In section 3 deals with application to two types of urn models: the balanced Pólya process and the two colors urn with multiple drawing.

The section 4 is a preliminary section and the sections 5 and 6 are reserved to the proofs of main results.

Finally we complete some examples on urns models.

2 Main Results

The following Theorem is an adaptation of the central limit Theorem for (SAA) of Sacks [Sac58]. The adaptation consist in allowingγn to be random instead of deterministic (A{n for some positive constant A). We also relax Fabian’s central limit Theorem [Fab68] of the orthogonality of transition matrixP and we do no need the positivity of Γ.

Theorem 2. LepZ_nqně0 be a stochastic approximation algorithm according to definition 1. Assume that (A1)-(A5) holds.

1. IfRepλminq ěγ{2andΛ2“ Hthen

?n`

Z_n´Θ˘ ^D ÝÑN`

0, PΣP¹˘ .

2. IfΛ2‰ H then

c n lnpnq

`Zn´Θ˘ <sup>D</sup> ÝÑN`

0, PΣP¹˘ .

Theorem 3. Consider the stochastic approximation algorithm in definition 1. Assume that (A1),(A2) and(A3) hold. Suppose furtherΛ1YΛ2“ H.

1. If eitherhis linear orRepλminq ą1{2Repλmaxq, there exists a random variableZ₁ inR^d such that e^lnpnqΓ`

Z_n´Θ˘

ÝÑZ₁almost surely.

2. If for some j “ 1, . . . , r´1 we have 1{2Repλ<sub>j`</sub>1q ăRepλminq ď 1{2Repλjq, there exists a random variable Z<sub>j</sub> P t0<sub>d´sj</sub>u ˆR<sup>sj</sup> wheresj denotes the dimension of the subspace‘<sup>r</sup>ℓ“j`1ker`

∇hpΘq ´λℓId

˘and such that for every 0ăβ ďRepλminqwe have almost surely

S_n;j^pβq`

Z_n´Θ˘

ÝÑZ_j (6)

where S_n,j^pβqisdˆdmatrix given by the relation S_n,j^pβqP1“

¨

˚

˚

˚

˚

˚

˚

˚

˚

˝ n^β

. ..

n^β

e^{lnpnqΓpλj`1q} . ..

e^lnpnqΓpλrq

˛

‹

‹

‹

‹

‹

‹

‹

‹

‚ .

In the next Theorem we extend the results of Theorem 3 by giving a limit in distribution ofZ_n´Θ in the case when Repλminq ą1{2Repλmaxq. We put forNě1,

L^`“!

pP r1, ds: @ℓ“1, . . . , N, @i1, . . . , i_ℓ`1P r1, ds, ÿ

kě1

k<sup>λp{γ´pλi1`...`λiℓ`1q{γ</sup>`

1{γ_k`1´1{γk˘ ă 8)

. ForpPL<sup>`</sup>and anypi1, . . . , i<sub>ℓ`1</sub>q P r1, ds<sup>ℓ`1</sup> we define ϕ<sub>p</sub>pi1, . . . , i<sub>ℓ`1</sub>q “ ÿ

kě1

k<sup>λp{γ´pλi1`...`λiℓ`1q{γ</sup>`

1{γ_k`1´1{γk˘ and

Dppλi1, . . . , λiℓ`1q{γq “diag`

ϕ1pi1, . . . , i<sub>ℓ`</sub>1q<sup>1</sup>1PL<sup>`</sup>, . . . , ϕdpi1, . . . , i_ℓ`1q˘

1dPL^`

˘.

Denote by D^{`</sup> “ pλ1{γ<sup>1</sup>1PL<sup>`}, . . . , λ_d{γ¹_dPL`q and D<sup>´</sup> “ P<sup>´1</sup>ΓP ´D<sup>`</sup>. Further, we set Γ^{`</sup> “ P D<sup>`}P^´1 and Γ^´ “P D^´P^´1. For a complex number β we set, if it exists, pďq such that Repλ_pq ąRepβqor Repβq ąRepλ_qq and ifβPspp∇hpΘqqtakeβ“λ_p`1“. . .“λ_q´1.

Theorem 4. Suppose that the stochastic approximation algorithms pZnqně0 given by the definition 1 satisfies the assumptions (A1)-(A5) and such that Repλmaxq ă γ{2. Suppose further either h is linear or Repλminq ą Repλmaxq{2.There exists a pair of random variables Z₃,Z₂PR^d such that

?n´

Z_n´Θ´e^´lnpnqΓZ₃´S_npN,Γ,Z₁q¯ _D ÝÑN´

0, PΣP^1´1¯ whereN ětRepλmaxq{Repλminqu`1 andS_npN,Γ,Z₁qis given by the following sum SnpN,Γ,Z₁q “

N

ÿ

ℓ“1

1{ℓ! ÿ

i1,...,i_ℓ`1

Wi1. . . Wi_{ℓ`</sub>1PDppλi1`. . .`λi<sub>ℓ`}1q{γqP^´1DℓhpΘqpVi1, . . . , ViℓqVi_ℓ`1

`

N

ÿ

ℓ“2

1 γℓ!

d

ÿ

i1,...,iℓ“1

Wi1. . . WiℓD_nppλi1`. . .`λiℓq{γ,ΓqH_pλ

i1{γ`...`λ_iℓ{γqpΓqDℓhpΘq`

Vi1, . . . , Viℓ

˘.

HereW1, . . . , Wdare the complex components of the vectorZ₁in the basis of eigenvectorspV1, . . . , Vdq. The matrices HβpΓqandD_npβ,Γq pβ PCqare given by

P^´1H_βpΓqP “diag`

ζp1`β´λ1q, . . . , ζp1`β´λ_pq,1, . . . ,1,1{pλ_q´βq, . . . ,1{pλ_d´βq˘ and

P^´1D_npβ,ΓqP “diag`

n^´λ1, . . . , n^´λp, n^´βplnpnq `γq, . . . , n<sup>´β</sup>plnpnq `γq, n^´β, . . . , n^´β˘ . Recall that ζ denotes the Zeta Riemann function andγ is the Euler gamma constant.

In what follows we present a limit Theorem for the stochastic approximation algorithm pZnqně0 regardless of where the spectrum of Γ is located. We set σ “ maxtRepλq, λ P spp∇hpΘqq and Repλq ă γ{2u and N “ tσ{Repλminqu`1.

Theorem 5. Consider the stochastic approximation algorithm pZnqně0 mentioned in definition 1. Suppose that assumptions (A1)-(A5) are satisfied. Suppose furtherΛ2“ H. IfRepλminq ąσ{2then there exists a pair of random variables Z^p2q

1 andZ^p2q

2 Pπ2R^d such that

?n`

Zn´Θ´P1

´0, e^´lnpnqΓ2Z₂^p2q´e^´lnpnqΓ2SnpN,Γ2,Z₁^p2qq¯1

˘ ^D ÝÑN`

0, P1ΣP₁¹˘ .

3 Application to Urn Models

3.1 Multiple Color Balanced Pólya Process.

Consider a d-color urn for an integerdě2. At each discrete time step, a ball is drawn (uniformly), observe its color and then put it back in the urn together with new balls according to a replacement rule given by thedˆdmatrix R“`

r_ij˘

1ďijďd.Denoting byM_n“`

M_n^p1q, . . . , M_n^pdq˘1

the vector composition of the urn afternexperiments with M0corresponding to the initial composition.

We assume that the entriesr_ij ě0 for i‰j and we allow to diagonal entries to be negative (meaning that we can remove balls from the urn), provided that the urn in question is tenable.

This kind of urn model was investigated by many different authors with different approaches. For example, Janson [Jan04] Mailler[Mai14] and Pouyanne [Pou08] by an algebraic approach and the theory of branching processes, Pages [LP13b] by using the stochastic approximation methods. The two-color balanced urn model was studied by Flajolet [FGP05] relying on analytic methods.

In the present section we assume that the urn is irreducible and balanced with balance S ě 1. By the Perron- Frobenius theory of positive matricesR`AId, for a large positive constantA, has a large real eigenvalueλ1 and all the other eigenvalues are with real partsďλ1. Using the terminology in [Pou08, JP16] we say that the Pólya urn islargeifλ1is simple and all other eigenvalues satisfy Repλq ě1{2Repλ1q. The urn is said to besmallifλ1 is simple and for all iě2, Repλ_iq ď 1{2Repλ1q. We say that the process is critically small if it is small and there exists an eigenvalueλwith real part“λ1{2.

Theorem 6. Suppose that the urn is irreducible andR is diagonalizable overC.

1. If the urn is small butΛ2“ H, then

M_n´nSV1

?n

ÝÑD N`

0, P1ΣP1¹

˘

where

Σ“

ˆ 1{S²π1P₁^´1ΣYP₁^´1π¹₁ 0 0 Q1WQ¹₁

˙

(7) andW is the complex matrix with entries given by

rWskℓ“ rQ^´₁¹π2P₁^´1ΣYP₁^1´1π₂¹Q¹₁^´1skℓ

SpS´λk´λℓq p2ďk, ℓďdq and with ΣY “R`

1{SId´1{S²¹b¹˘

R¹ where ¹b¹is the matrix with entries equal to 1.

2. If the urn is small butΛ2‰ Hthen 1 anlnpnq

`Mn´nSV1˘ <sup>D</sup> ÝÑN`

0, P1ΣP₁¹˘

whereΣis the matrix with ranksď2CardpΛ2qwith entriesΣij“1{S²rQ^´1₁ π2P₁^´1ΣYP₁^1´1π¹₂Q¹₁^´1sij¹ppi,jqPΛ2q. 3. If the urn is large then there exist a random variableZ such thate^1{SlnpnqR`

M_n´nSV1

˘ÝÑZ almost surely.

Furthermore we have the limit in distribution

?1 n

`Mn´nSV1´e<sup>1{SlnpnqR</sup>Z˘ <sup>D</sup> ÝÑN`

0, S2ΣQ¹₂˘ where Σis the complex matrix

Σ“

¨

˚

˚

˝

π1P₁^´1ΣYP₁^1´1π₁¹ rπ1P₁^´1ΣYP₁^1´1π₃¹Q¹₂^´1sk

Sλ_k rπ1P₁^´1ΣYP₁^1´1π₃¹Q¹₃^´1sk

Sλ_k

rQ^´₂¹π3P₁^´1ΣYP₁^1´1π₃¹Q¹₂^´1sk

Spλ_k`λ_ℓ´Sq

˛

‹

‹

‚

. (8)

Noticing that the term e^´1{SlnpnqRZ is the oscillating term since ifW1, . . . , Wd are the components ofZ is the basis of eigenvectorsV1, . . . , Vd we gete^´1{SlnpnqRZ “

d

ÿ

k“1

n^´λk{SWkVk.The next Theorem is a limit Theorem for balanced Pólya process.

Theorem 7. Suppose that the urn is irreducible, balanced and Ris diagonalizable over Cand thatΛ2“ H. There exist a vector random variableZ“`

0,0,Z^p3q˘

such that

?1 n

´M_n´nSV1´P1

`0,0, e<sup>´lnpnqΓ2</sup>Z<sub>3</sub>˘¯ <sub>D</sub> ÝÑN`

0, P1ΣP1¹

˘.

3.2 Multiple drawing Non-Balanced Two-Color Urn Model

This model is defined as follow: Starting with a two-color urn, containing initiallyW0white balls andB0black balls such thatT0“W0`B0ěmwithmis an integerě1. At each stage of the process, we drawwithout replacementm balls from the urn and count the number of the sampled white balls (saykwhite balls). Then, we return the balls in the urn witha_m´k white balls andb_m´k black balls wherea_k, b_k PZ, 0ďkďm. We denote the replacement rule by

R“

¨

˚

˚

˚

˝

a0 b0

a1 b1

... ... a_m b_m

˛

‹

‹

‹

‚ .

We designate by W_n and B_n the number of white and black balls after n draws. The total number of balls of both colors will be T_n “ W_n `B<sub>n</sub>. The multiple drawing urn model has been originally introduced by Chen an Wei [CW05] with ak “ cpm´kq and bk “ ck where c is a positive integer. They proved by the martingale theory the almost sure convergence of Wn after suitable normalization to a positive random variable W<sub>8</sub> and showing the absolute convergence ofW<sub>8</sub>. Chen and Kuba [CK13] gaves the moments of the random variableW<sub>8</sub>. Kuba and Sulzbach [KS15] generalized the model of Chen and Wei to a general two-color model under the affinity (EpW<sub>n`</sub>1

ˇ

ˇF_nq “ anWn`bn) and the balance (ak `bk “ σ ě 1 0 ď k ď m) conditions. Kuba and Mahmoud [KMP13] studied the limit in distribution withak “Cpm´kqand bk “Ck where C PN^˚ and in [KM17] under the affinity condition.

In this section we improve the results of [LMS16] for two color unbalanced urn proved using Renlund’s stochastic approximation Theorems [Ren11]. The major problem is to control the rate of convergence of theT_n{nto its limit.

We state by giving the tenability conditions for the model with multiple drawingmě2 [KS15, LMS16].

Lemma 1. Consider the urn process with initial composition pW0, B0q¹ and replacement function R. Assume that m ě 2 and we denote by a (respectively b) the greatest common divisor of ta1, . . . , a_m´1u p respectively tb1, . . . , b_m´1uq. The urn is tenable if and only if for every1ďkďm´1 we have,

a_kě ´a and b_k ě ´b and the additional conditions

amPJ´m´a`1,´m˘

YJ´m,8KX!

ℓP ´N, rW0s¹P r´ℓs¹,r´ℓ`1s<sup>1</sup>, . . . ,rm`a´1s¹() b_mPJ´m´b`1,´m˘

YJ´m,8KX!

ℓP ´N, rB0s²P r´ℓs²,r´ℓ`1s<sup>2</sup>, . . . ,rm`b´1s²() where for all integerℓ, rℓs¹ prespectivelyrℓs²q denotes the remainder of the division ofℓ bya prespectivelybq. Theorem 8. Assume that the tenability conditions are satisfied and that lim infT_n{n ą 0. Suppose there exists θP r0,1ssuch that the proportion of white balls, denoted byZ_n converges almost surely toθ.

Let w“

m

ÿ

k“0

ˆm k

˙

θ^kp1´θq^m´kpa_m´k`b_m´kq, λ“ ´h¹pθq

w andσ²“ Hpθq w² where

Hpxq “

m

ÿ

k“0

ˆm k

˙

x^kp1´xq^m´kpa_m´k´xb_m´kq². and

hpxq “

m

ÿ

k“0

ˆm k

˙

`a<sub>m´k</sub>´xpa<sub>m´k</sub>`b_m´kq˘

x^kp1´xq^m´k Assuming that a_m‰0 ifθ“0 andb0‰a0 if θ“1. Then

paq if λą1{2then

?npZn´θqÝÑ^D N ˆ

0, σ² 2λ´1

˙ ,

pbq if λ“1{2then

c n

lnpnqpZn´θqÝÑ^D Np0, σ²q,

pcq if λă1{2then there exists a random variable Z such that n^λpZn´θq ÝÑZ almost surely. Also we have

?n

˜

Z_n´θ´n^´λZ´h¹pθq

m`1

ÿ

ℓ“2

h^pℓqp1q

ℓpℓ´1qn^´ℓλZ^ℓ

¸ ÝÑD N

ˆ 0, σ²

1´2λ

˙ .

4 Preliminaries

The following lemma is a simple version of Chung’s lemma [Chu54, MR73]

Lemma 2. Let b_n be a sequence of positive numbers such that the following recursion holds b_n`1ď´

1´c{n¯

bn`V{n<sup>1`p</sup>

with c, pą0 andV ą0. Ifcąpthenbn “O` 1{n^p˘

and ifp“1 with 0ăcă1then bn“O` 1{n^c˘

.

Lemma 3. Let M PM_dpRqbe a diagonalizable matrix over C.IfP_m,n denotes the product

n

ź

j“m

´I_d´1 jM¯

then

Pm,n“e^´lnpm{nqM´

Id`O`1 m

˘¯

. (9)

Proof. LetP be a transition matrix ofM withP^´1M P“diag`

λ1, . . . , λ_d˘

.Applying the relations 1´λℓ

j “e^{´λℓ{j`Op1{j2q}

n

ÿ

j“m

1 j “ln`

n{m˘

`O` 1{m˘

and

n

ÿ

j“m

1

j² “Op1{mq

we obtain

Pm,n “ Pdiag`

n

ź

j“m

´1´λ_ℓ j

¯

,1ďℓďd˘

P^´1“Pdiag`

e^{´λℓlnpm{nq}` 1`O`

p1 m

˘˘˘P^´1

“ Pdiag`

e^{´λℓlnpm{nq}˘ P^´1`

1`O` p1

m

˘˘“exp`

´lnpm{nqM˘`

1`O` 1{m˘˘

.

Lemma 4. If M P M_dpRq is an invertible matrix and diagonalizable over C. P denotes a transition matrix of M such that P^´1M P “diag`

λ1, . . . , λd˘

such that 1 ąRepλ1q ě. . . ěRepλdq. For a complex number β we let, if it exists, a pair of positive integers pďq such that Repλpq ąRepβq or Repβq ąRepλqq and ifβ PsppMq take β“λ_p`1“. . .“λ_q´1. Then

n

ÿ

j“1

1

j<sup>β`1</sup>e<sup>lnpj{nqM</sup> “D<sub>n</sub>pβqH<sub>β</sub>`

1`Op1{nq˘

“H_βpMqD_npβ, Mq`

1`Op1{nq˘

(10) whereD_npβ, MqandH_pβ, Mqare already defined in Theorem 4.

The next lemma is a version of the Bauer-Fike Theorem for matrices. This Theorem gives results on the perturbation of the spectrum of a diagonalizable complex matrices. In our current work we deal only with hermitian matrices.