A constructive mean field analysis of multi population neural networks with random synaptic weights and stochastic inputs

HAL Id: inria-00258345

https://hal.inria.fr/inria-00258345v3

Submitted on 4 Aug 2008

HAL is a multi-disciplinary open access

archive for the deposit and dissemination of

sci-entific research documents, whether they are

pub-L’archive ouverte pluridisciplinaire HAL, est

destinée au dépôt et à la diffusion de documents

scientifiques de niveau recherche, publiés ou non,

neural networks with random synaptic weights and

stochastic inputs

Olivier Faugeras, Jonathan Touboul, Bruno Cessac

To cite this version:

Olivier Faugeras, Jonathan Touboul, Bruno Cessac. A constructive mean field analysis of multi

population neural networks with random synaptic weights and stochastic inputs. [Research Report]

RR-6454, INRIA. 2008, pp.58. �inria-00258345v3�

a p p o r t

d e r e c h e r c h e

A constructive mean field analysis of multi

population neural networks with random synaptic

weights and stochastic inputs

Olivier Faugeras — Jonathan Touboul — Bruno Cessac

N° 6454

Olivier Faugeras , Jonathan Touboul , Bruno Cessac

Th`eme BIO — Syst`emes biologiques Projet Odyss´ee∗

Rapport de recherche n° 6454 — August 4, 2008 — 55 pages

Abstract: We deal with the problem of bridging the gap between two scales in neuronal modeling.

At the first (microscopic) scale, neurons are considered individually and their behavior described by stochastic differential equations that govern the time variations of their membrane potentials. They are coupled by synaptic connections acting on their resulting activity, a nonlinear function of their membrane potential. At the second (mesoscopic) scale, interacting populations of neurons are de-scribed individually by similar equations. The equations describing the dynamical and the stationary mean field behaviors are considered as functional equations on a set of stochastic processes. Using this new point of view allows us to prove that these equations are well-posed on any finite time interval and to provide, by a fixed point method, a constructive method for effectively computing their unique solution. This method is proved to converge to the unique solution and we characterize its complexity and convergence rate. We also provide partial results for the stationary problem on infinite time intervals. These results shed some new light on such neural mass models as the one of Jansen and Rit (Jansen and Rit, 1995): their dynamics appears as a coarse approximation of the much richer dynamics that emerges from our analysis. Our numerical experiments confirm that the framework we propose and the numerical methods we derive from it provide a new and powerful tool for the exploration of neural behaviors at different scales.

Key-words: Mean field analysis, stochastic processes, stochastic differential equations, stochastic

networks, stochastic functional equations, random connectivities, multi populations networks, neural mass models

R´esum´e : Nous traitons du probl`eme de combler le foss´ee entre duex niveaux de mod´elisation

neu-ronale. A la premi`ere ´echelle (microscopique), les neurones sont consid´er´es individuellement et leur comportement est d´ecrit par des ´equations diff´erentielles stochastiques qui gouvernent les va-riations temporelles de leur potentiel de membrane. Ils sont coupl´es par des connections synaptiques qui agissent sur leur activit´e, qui est une fonction nonlin´eaire de leur potentiel de membrane. A la seconde ´echelle (m´esoscopique) les populations de neurones sont d´ecrites individuellement par le mˆeme type d’´equation. Les ´equations qui d´ecrivent les comportements de champ moyen dynamique et stationnaire sont consid´er´ees comme des ´equations fonctionnelles dans un espace de processus stochastiques. Ce nouveau point de vue nous permet de d´emontrer que ces ´equations ssont bien pos´ees sur des intervalles de temps finis et de proposer, par une m´ethode de point fixe, une m´ethode constructive permettant de calculer efficacement leur unique solution. Nous d´emontrons que cette m´ethode converge et caracterisons sa complexit´e et son taux de convergence. Nous donnons aussi des r´esultats partiels pour le probl`eme stationnaire sur des intervalles de temps infinis. Ces r´esultats apportent un nouvel ´eclairage sur les mod´eles de masses neuronales tels que celui de Jansen et Rit (Jansen and Rit, 1995): leur dynamique apparaˆıt comme une approximation grossi`ere de la dyna-mique bien plus riche qui ´emerge de notre analyse. Nos simulations num´eriques confirme que le cadre math´ematiques que nous proposons et les m´ethodes numeriques qui en d´ecoulent fournissent un outil nouveau et puissant pour l’exploration de comportements neuronaux `a diff´erentes ´echelles.

Mots-cl´es : Analyse champs moyen, processus stochastiques, ´equations diff´erentielles stochas-tiques, r´eseaux stochasstochas-tiques, ´equations stochastiques fonctionnelles, connectivit´es al´eatoires, r´eseaux multi populations, mod`eles de masses neuronales

1

Introduction

Modeling neural activity at scales integrating the effect of thousands of neurons is of central impor-tance for several reasons. First, most imaging techniques are not able to measure individual neuron activity (“microscopic” scale), but are instead measuring mesoscopic effects resulting from the ac-tivity of several hundreds to several hundreds of thousands of neurons. Second, anatomical data recorded in the cortex reveal the existence of structures, such as the cortical columns, with a diame-ter of about50µm to 1mm, containing of the order of one hundred to one hundred thousand neurons

belonging to a few different species. These columns have specific functions. For example, in the visual cortex V1, they respond to preferential orientations of bar-shaped visual stimuli. In this case, information processing does not occur at the scale of individual neurons but rather corresponds to an activity integrating the collective dynamics of many interacting neurons and resulting in a meso-scopic signal. The description of this collective dynamics requires models which are different from individual neurons models. In particular, if the accurate description of one neuron requires “m”

pa-rameters (such as sodium, potassium, calcium conductances, membrane capacitance, etc...), it is not necessarily true that an accurate mesoscopic description of an assembly ofN neurons requires N m

parameters. Indeed, whenN is large enough averaging effects appear, and the collective dynamics is

well described by an effective mean field, summarizing the effect of the interactions of a neuron with the other neurons, and depending on a few effective control parameters. This vision, inherited from statistical physics requires that the space scale be large enough to include a large number of micro-scopic components (here neurons) and small enough so that the region considered is homogeneous. This is in effect the case of cortical columns.

However, obtaining the equations of evolution of the effective mean field from microscopic dy-namics is far from being evident. In simple physical models this can be achieved via the law of large numbers and the central limit theorem, provided that time correlations decrease sufficiently fast. This type of approach has been generalized to such fields as quantum field theory or non equilibrium statistical mechanics. To the best of our knowledge, the idea of applying mean field methods to neural networks dates back to Amari (Amari, 1972; Amari et al, 1977). In his approach, the author uses an assumption that he called the “local chaos hypothesis”, reminiscent of Boltzmann’s “molec-ular chaos hypothesis”, that postulates the vanishing of individual correlations between neurons, when the numberN of neurons tends to infinity. Later on, Crisanti, Sompolinsky and

cowork-ers (Sompolinsky et al, 1988) used a dynamic mean field approach to conjecture the existence of chaos in an homogeneous neural network with random independent synaptic weights. This approach was formerly developed by Sompolinsky and coworkers for spin-glasses (Crisanti and Sompolinsky, 1987a,b; Sompolinsky and Zippelius, 1982), where complex effects such as aging or coexistence of a diverging number of metastable states, renders the mean field analysis delicate in the long time limit (Houghton et al, 1983).

On the opposite, these effects do not appear in the neural network considered in (Sompolinsky et al, 1988) because the synaptic weights are independent (Cessac, 1995) (and especially non symmetric, in opposition to spin glasses). In this case, the Amari approach and the dynamic mean field approach lead to the same mean field equations. Later on, the mean field equations derived by Sompolin-sky and Zippelius (SompolinSompolin-sky and Zippelius, 1982) for spin-glasses were rigorously obtained by

Ben Arous and Guionnet (Ben-Arous and Guionnet, 1995, 1997; Guionnet, 1997). The application of their method to a discrete time version of the neural network considered in (Sompolinsky et al, 1988) and in (Molgedey et al, 1992) was done by Moynot and Samuelides (Moynot and Samuelides, 2002).

Mean field methods are often used in the neural network community but there are only a few rigorous results using the dynamic mean field method. The main advantage of dynamic mean field techniques is that they allow one to consider neural networks where synaptic weights are random (and independent). The mean field approach allows one to state general and generic results about the dynamics as a function of the statistical parameters controlling the probability distribution of the synaptic weights (Samuelides and Cessac, 2007). It does not only provide the evolution of the mean activity of the network but, because it is an equation on the law of the mean field, it also provides informations on the fluctuations around the mean and their correlations. These correlations are of crucial importance as revealed in the paper by Sompolinsky and coworkers (Sompolinsky et al, 1988). Indeed, in their work, the analysis of correlations allows them to discriminate between two distinct regimes: a dynamics with a stable fixed point and a chaotic dynamics, while the mean is identically zero in the two regimes.

However, this approach has also several drawbacks explaining why it is so seldom used. First, this method uses a generating function approach that requires heavy computations and some “art” for obtaining the mean field equations. Second, it is hard to generalize to models including several populations. Their approach consists in considering that dynamic mean field equations characterize in fine a stationary process. It is then natural to search for stationary solutions. This considerably simplifies the dynamic mean field equations by reducing them to a set of differential equations (see section 5) but the price to pay is the unavoidable occurrence in the equations of a non free parameter, the initial condition, that can only be characterized through the investigation of the non stationary case. Hence it is not clear whether such a stationary solution exists, and, if it is the case, how to characterize it. To the best of our knowledge, this difficult question has only been investigated for neural networks in one paper by Crisanti and coworkers (Crisanti et al, 1990).

Different alternative approaches have been used to get a mean field description of a given neural network and to find its solutions. In the neuroscience community, a static mean field study of multi population network activity was developed by Treves in (Treves, 1993). This author did not consider external inputs but incorporated dynamical synaptic currents and adaptation effects. His analysis was completed in (Abbott and Van Vreeswijk, 1993), where the authors considered a unique pop-ulation of nonlinear oscillators subject to a noisy input current. They proved, using a stationary Fokker-Planck formalism, the stability of an asynchronous state in the network. Later on, Gerstner in (Gerstner, 1995) built a new approach to characterize the mean field dynamics for the Spike Re-sponse Model, via the introduction of suitable kernels propagating the collective activity of a neural population in time.

Brunel and Hakim considered a network composed of integrate-and-fire neurons connected with constant synaptic weights (Brunel and Hakim, 1999). In the case of sparse connectivity, stationarity, and considering a regime where individual neurons emit spikes at low rate, they were able to study analytically the dynamics of the network and to show that the network exhibited a sharp transition between a stationary regime and a regime of fast collective oscillations weakly synchronized. Their

approach was based on a perturbative analysis of the Fokker-Planck equation. A similar formal-ism was used in (Mattia and Del Giudice, 2002) which, when complemented with self-consistency equations, resulted in the dynamical description of the mean field equations of the network, and was extended to a multi population network.

In the present paper, we investigate this question using a new and rigorous approach based on stochastic analysis.

The article is organized as follows. In section 2 we derive from first principles the equations relat-ing the membrane potential of each of a set of neurons as function of the external injected current and noise and of the shapes and intensities of the postsynaptic potentials in the case where these shapes depend only on the post-synaptic neuron (the so-called voltage-based model) and in the case where they depend only on the nature of the presynaptic neurons (the so-called activity-based model). Assuming that the shapes of the postsynaptic potentials can be described by linear (possibly time-dependent) differential equations we express the dynamics of the neurons as a set of stochastic differ-ential equations and give sufficient conditions for the equivalence of the voltage- and activity based descriptions. This allows us to obtain the mean field equations when the neurons belong toP

pop-ulations whose sizes grow to infinity and the intensities of the postsynaptic potentials are indepen-dent Gaussian random variables whose law depend on the populations of the pre- and post-synaptic neurons and not on the individual neurons themselves. These equations can be derived in several ways, either heuristically as in the work of Amari (Amari, 1972; Amari et al, 1977), Sompolinsky (Crisanti et al, 1990; Sompolinsky et al, 1988), and Cessac (Cessac, 1995; Samuelides and Cessac, 2007), or rigorously as in the work of Benarous and Guionnet (Ben-Arous and Guionnet, 1995, 1997; Guionnet, 1997). Our purpose in this article is not their derivation but to prove that they are well-posed and to provide an algorithm for computing their solution. Before we do this we provide the reader with two important examples of such mean field equations. The first example is what we call the simple model, a straightforward generalization of the case studied by Amari and Sompolinsky. The second example is a neuronal assembly model, or neural mass model, as introduced by Freeman (Freeman, 1975) and examplified in Jansen and Rit’s cortical column model (Jansen and Rit, 1995). In section 3 we consider the problem of solutions over a finite time interval[t0, T ]. We prove,

under some mild assumptions, the existence and uniqueness of a solution of the dynamic mean field equations given an initial condition at timet0. The proof consists in showing that a nonlinear

equation defined on the set of multidimensional Gaussian random processes defined on[t0, T ] has a

fixed point. We extend this proof in section 4 to the case of stationary solutions over the time interval

[_{−∞, T ] for the simple model. Both proofs are constructive and provide an algorithm for computing}

numerically the solutions of the mean field equations.

We then study in section 5 the complexity and the convergence rate of this algorithm and put it to good use: We first compare our numerical results to the theoretical results of Sompolinsky and coworkers (Crisanti et al, 1990; Sompolinsky et al, 1988). We then provide an example of numerical experiments in the case of two populations of neurons where the role of the mean field fluctuations is emphasized.

Along the paper we introduce several constants. To help the reader we have collected in table 1 the most important ones and the place where they are defined in the text.

2

Mean field equations for multi-populations neural network

models

In this section we introduce the classical neural mass models and compute the related mean field equations they satisfy in the limit of an infinite number of neurons

2.1

The general model

2.1.1 General framework

We consider a network composed ofN neurons indexed by i∈ {1, . . . , N} belonging to P

popula-tions indexed byα∈ {1, . . . , P } such as those shown in figure 1. Let Nαbe the number of neurons

in populationα. We have N =PPα=1Nα. In the following we are interested in the limitN → ∞.

We assume that the proportions of neurons in each population are non-trivial, i.e. :

lim

N →∞

Nα

N = nα ∈ (0, 1) ∀α ∈ {1, . . . , P }.

If it were not the case the corresponding population would not affect the global behavior of the system, would not contribute to the mean field equation, and could be neglected.

We introduce the functionp : {1, . . . , N} → {1, . . . , P } such that p(i) is the index of the

population which the neuroni belongs to.

The following derivation is built after Ermentrout’s review (Ermentrout, 1998). We consider that each neuroni is described by its membrane potential Vi(t) or by its instantaneous firing rate νi(t),

the relation between the two quantities being of the formνi(t) = Si(Vi(t)) (Dayan and Abbott,

2001; Gerstner and Kistler, 2002), whereSiis sigmoidal.

A single action potential from neuronj is seen as a post-synaptic potential P SPij(t− s) by neuron

i, where s is the time of the spike hitting the synapse and t the time after the spike. We neglect the

delays due to the distance travelled down the axon by the spikes.

Assuming that the post-synaptic potentials sum linearly, the average membrane potential of neu-roni is

Vi(t) =

X

j,k

P SPij(t− tk),

where the sum is taken over the arrival times of the spikes produced by the neuronsj. The number

of spikes arriving betweent and t + dt is νj(t)dt. Therefore we have

Vi(t) = X j Z t −∞ P SPij(t− s)νj(s) ds = X j Z t −∞ P SPij(t− s)Sj(Vj(s)) ds, or, equivalently νi(t) = Si  X j Z t t0 P SPij(t− s)νj(s) ds   . (1)

TheP SPijs can depend on several variables in order to account for instance for adaptation or

learn-ing.

The voltage-based model

The assumption, made in (Hopfield, 1984), is that the post-synaptic potential has the same shape no matter which presynaptic population caused it, the sign and amplitude may vary though. This leads to the relation

P SPij(t) = Jijgi(t).

gi represents the unweighted shape (called a g-shape) of the postsynaptic potentials andJij is the

strength of the postsynaptic potentials elicited by neuronj on neuron i. Thus we have Vi(t) = Z t t0 gi(t− s)  X j Jijνj(s)   ds.

So far we have only considered the synaptic inputs to the neurons. We also assume that neuroni

receives an external current densityIi(t) and some noise ni(t) so that

Vi(t) = Z t t0 gi(t− s)  X j Jijνj(s) + Ii(s) + ni(s)   ds. (2)

We assume that the external current and the g-shapes satisfyIi = Ip(i),gi = gp(i),Si = Sp(i),

i.e. they only depend upon the neuron population. The noise model is described later. Finally we assume thatgi= gα(whereα = p(i)) is the Green function of a linear differential equation of order

k, i.e. satisfies k X l=0 alα(t) dl_g α dtl (t) = δ(t). (3)

We assume that the functionsal

α(t) are continuous for l = 0,· · · , k and α = 1, · · · , P . We also

assumeak

α(t)≥ c > 0 for all t ∈R,α = 1,· · · , P .

Known examples of g-shapes, see section 2.2.3 below, aregα(t) = Ke−t/τY (t) (k = 1, a1(t) = 1 K, a0(t) = 1 K τ )orgα(t) = Kte−t/τY (t) (k = 2, a2(t) = 1 K, a1(t) = 2 K τ, a0(t) = 1 K τ), where

Y is the Heaviside function.

We noteDk

α the corresponding differential operator, Dkαgα = δ, and DkN theN -dimensional

differential operator containingNαcopies ofDkα,α = 1,· · · , P . We write (2) in vector form

V(N )_{= J}(N )_{diag(gα)∗ S}(N )_(V(N )_{) + diag(gα)∗ I}(N )_{+ diag(gα)∗ n}(N )_,

wherediag(gα) is the N -dimensional diagonal matrix containing Nαcopies ofgα,α = 1,· · · , P

and_{∗ indicates the convolution operator. S}(N )is the mappingR

N

→R

N_{such thatS}(N )_(V(N )₎ i=

Sp(i)(V(N )i ). We apply the operator DkN to both sides to obtain

Dk_NV(N )_{= J}(N )_{· S}(N )_(V(N )_{) + I}(N )

V + n (N )

which is a stochastic differential equation d _dk−1V(N ) dtk−1  =_−Dk−1_N V(N )_{+ J}(N )_{· S}(N )_(V(N )_{) + I}(N )_{dt + dn}(N )_{t ,}

where Dk−1_N is obtained from theP differential operators of order k_{− 1 D}k−1

α =Pk−2l=0 al α(t) ak α(t) dl dtl.

The activity-based model If we make the assumption that the shape of a PSP depends only on the nature of the presynaptic cell, that is

P SPij= Jijgj,

and define the activity as

Aj(t) =

Z t −∞

gj(t− s)νj(s) ds,

multiplying both sides of equation (1) bygi(t− s) and integrating with respect to s, we obtain

Ai(t) = Z t −∞ gi(t− s)Si  X j JijAj(s) + Ii(s) + ni(s)   ds,

where we have added an external current and a noise. Ifp(i) = α, this yields

Dk αAi= Si  X j JijAj(t) + Ii(t) + ni(t)   ,

and in terms of theN -dimensional vector A(N )

Dk NA(N )= S(N )(J(N )A(N )+ I (N ) A + n (N ) A ). (5)

Equivalence of the two models As a matter of fact these two equations are equivalent provided that J(N )is invertible1. Indeed, let us use the change of variable V(N )= J(N )_A(N )_{+ I}(N )

A + n (N ) A .

We have, because J(N )is not a function of time,

Dk_NV(N )_{= J}(N )Dk_NA(N )_{+ D}k_NI(N ) A + D k Nn (N ) A .

Replacing Dk_NV(N )by this value in (4) we obtain

J(N )Dk_NA(N )_{+ D}k_NI(N ) A + D k Nn (N ) A = J(N )_{· S}(N )_(J(N )A(N )_{+ I}(N ) A + n (N ) A ) + I (N ) V + n (N ) V .

1_{Note that in the cases we treat in this paper, the matrix J}(N)_{is always almost surely invertible since it has non-degenerate}

Assuming that the matrix J(N )is invertible yields Dk_NA(N )₌ S(N )(J(N )A(N )_{+ I}(N ) A + n (N ) A ) + (J(N ))−1  I(N ) V − DkNI (N ) A + n (N ) V − DkNn (N ) A  .

Given the current I(N )_V (respectively the noise n(N )_V ), we can choose the current I(N )_A (respec-tively the noise n(N )_A ) solution of the linear differential equation Dk_NI(N )

A = I (N ) V (respectively Dk Nn (N ) A = n (N )

V ). Using the Green functionsgα, α = 1,· · · , P this is equivalent to I(N )A =

diag(gα)∗ I(N )_V (respectively n(N )_A = diag(gα)∗ n(N )_V ).

The dynamics We introduce thek− 1 N-dimensional vectors V(l)_{(t) = [V}(l) 1 ,· · · , V

(l) N ]T,l =

1,· · · , k − 1 of the lth-order derivative of V(N )_{(t), and the N k-dimensional vector}

e V(N )_{(t) =}      V(N )_(t) V(N ) (1)_(t) .. . V(N ) (k−1)_(t)     .

TheN -neurons network is described by the N k-dimensional vector eV(N )_{(t). We consider the}

direct sumR N k_{= E}(0) ⊕ · · · ⊕ E(k−1)_{, where eachE}(l) ₌ R N_{,l = 0,} · · · , k − 1 and introduce

the following notation: if x is a vector ofR

N k_{, x}

l is its component inE(l),l = 0,· · · , k − 1, an

N -dimensional vector. In particular we have eV(N )

l (t) = V(N ) (l)(t) for l = 0,· · · , k − 1 with the

convention that V(N ) (0) = V(N )_.

We now write the equations governing the time variation of the firstk− 1 vectors of eV(N )_(t),

i.e. the derivatives of order 0, . . .k− 2 of V(N )_{(t). These equation in effect determine the noise}

model. We write d eV(N ) l (t) = eV (N ) l+1(t) dt + Λ (N ) l · dW (N ) t l = 0,· · · , k − 2, (6)

where Λ(N )_l is theN× N diagonal matrix diag(sl

α), where slα,α = 1,· · · , P is repeated Nαtimes,

and W(N )_t anN -dimensional standard Brownian process.

The equation governing the(k_{− 1)th differential of the membrane potential has a linear part}

determined by the differential operator Dk−1and must account for the external inputs (deterministic and stochastic) and the activity of the neighbors, see (4). Keeping the same notations as before for the inputs and denoting by L(N )theN× Nk matrix describing the action of the neurons membrane

potentials and their derivatives on the(k− 1)th derivative of V, we have:

d eV(N )

k−1(t) =



L(N )_{(t)· e}V(N )_{(t) + J}(N )_{· S}(N )_{( e}V(N )<sub>0 (t))
+ I</sub>(N )_(t)_dt

where L(N )₌ _diag(a0 α(t)) · · · diag(ak−1α (t))  . We define L(N )_{(t) =}       0 IdN · · · 0 0 0 . .. ₀ .. . ... IdN diag(a0 α(t)) diag(a1α(t)) · · · diag(ak−1α (t))      ,

whereIdN is theN× N identity matrix. We also denote by:

e U(N ) t =      0 .. . 0 J(N )_{· S}(N )_{( e}V_0(t))      and eI (N ) t =      0 .. . 0 I(N )_(t)     .

The full equation satisfied by eV(N )can be written:

d eV(N )_{(t) =}L(N )_{(t) e}V(N )_{(t) + e}U_t(N )_{+ eI}(N )_t _{dt + Λ}(N )_{(t)· dWt}(N )_, (8) where thekN× kN matrix Λ(N )(t) is equal to diag(Λ(N )0 ,· · · , Λ

(N ) k−1).

Note that thekth-order differential equation describing the time variation of the membrane

po-tential of each neuron contains a noise term which is a linear combination of various integrated Brownian processes (up to the orderk_{− 1) as shown in the following formula which is derived from}

(6) and (7). d eV(N ) k−1(t) = k−1 X l=0 L(N ) l (t) eV (N ) l ! dt+ k−2 X l=0 L(N ) l (t) k−l−2_X h=0 Z t 0 Z s1 0 · · · Z sh−1 0 Λ(N ) l+h(sh)dWshdsh−1· · · ds0 !! dt+ Λ(N ) k−1(t)dWt.

Comparing with equation (4) we see that the noise n(N )dt is a weighted sum of Brownian and

integrated Brownian processes.

2.2

The Mean Field equations

2.2.1 General derivation of the mean field equation

The connectivity weightJijare modeled as independent Gaussian random variables. Their

distribu-tion depends only on the populadistribu-tion pairα = p(i), β = p(j), and on the total number of neurons Nβof populationβ:

Jij∼ N  ¯_Jαβ Nβ, σαβ p Nβ  .

We are interested in the limit law when N _{→ ∞ of the vector V}(N ) _{under the joint law of}

the connectivities and the Brownian motions, which we call the mean field limit. This law can be described by a set ofP equations, the mean field equations. As mentioned in the introduction

these equations can be derived in several ways, either heuristically as in the work of Amari (Amari, 1972; Amari et al, 1977), Sompolinsky (Crisanti et al, 1990; Sompolinsky et al, 1988), and Cessac (Cessac, 1995; Samuelides and Cessac, 2007), or rigorously as in the work of Benarous and Guion-net (Ben-Arous and GuionGuion-net, 1995, 1997; GuionGuion-net, 1997). We derive them here in a pedestrian way, prove that they are well-posed, and provide an algorithm for computing their solution.

The effective description of the network population by population is possible because the neurons in each population are interchangeable, i.e. have the same probability distribution under the joint law of the multidimensional Brownian motion and the connectivity weights. This is the case because of the form of equation (8).

The Mean Field equations We noteC([t0, T ],R

P_{) (respectively C((}

−∞, T ],R

P_{)) the set of}

continuous functions from the real interval[t0, T ] (respectively (−∞, T ]) toR

P_{. By assigning a}

probability to subsets of such functions, a continuous stochastic processX defines a positive measure

of unit mass onC([t0, T ],R

P_{) (respectively C((}

−∞, T ],R

P_{)). This set of positive measures of unit}

mass is noted_M+₁(C([t0, T ],R

P_{)) (respectively}

M+1(C((−∞, T ],R

P_)).

We now define a process of particular importance for describing the limit process: the effective interaction process.

Definition 2.1 (Effective Interaction Process). LetX∈ M+

1(C([t0, T ],R

P_{)) (resp.M}+

1(C((−∞, T ],R

P₎₎

be a given stochastic process. The effective interaction term is the Gaussian process UX _{∈ M}+₁(C([t0, T ],R

P ×P_)),

(resp._M+₁(C((−∞, T ],R

P ×P_{)) statistically independent of the external noise (W}

t)t≥t0and of the

initial conditionXt0(whent0>−∞), defined by:

           EhUX αβ(t) i = ¯JαβmXαβ(t) where mXαβ(t) def = E[Sαβ(Xβ(t))]; Cov(UX αβ(t), UαβX(s)) = σαβ2 ∆Xαβ(t, s) where ∆X αβ(t, s) def = EhSαβ(Xβ(t))Sαβ(Xβ(s)) i ; Cov(UX αβ(t), UγδX(s)) = 0 if α6= γ or β 6= δ. (9)

ChooseP neurons i1, . . . , iP, one in each population (neuroniαbelongs to the populationα).

Then it can be shown, using either a heuristic argument or large deviations techniques (see appendix A), that the sequence of processesVe(N )_{(t) = [ ˜V}(N )

i1 (t), . . . , ˜V (N ) iP (t)] T t≥t0  N ≥1converges in law to the process eV_{(t) = [ ˜V1(t), . . . , ˜VP(t)]}T

t≥t0solution of the following mean field equation:

where eV is akP -dimensional vector containing the P -dimensional vector V and its k−1 derivatives,

and L is theP k× P k matrix

L_{(t) =}       0P ×P IdP · · · 0P ×P 0P ×P 0P ×P . .. 0P ×P .. . ... IdP L_0(t) L_{1(t) · · · Lk−1(t)}      ,

whereIdP is theP× P identity matrix and 0P ×P the nullP× P matrix. (Wt) is a kP -dimensional

standard Brownian process and:

e UV t =      0P .. . 0P UV t · 1      eI(t) =      0P .. . 0P I_(t)      Λ(t) = diag(Λ0(t),· · · , Λk−1(t)).

The matrices L0,· · · , Lk−1(respectively Λ0,· · · , Λk−1) are obtained by selecting the sameP rows

andP columns of the matrices L(N )0 ,· · · , L (N ) k−1 (respectively Λ (N ) 0 ,· · · , Λ (N ) k−1) corresponding to

P neurons in different populations, (UV

t ) is the effective interaction process associated with V, and

I_{(·) is the P -dimensional external current.}

To proceed further we formally integrate the equation using the flow, or resolvent, of the equa-tion, notedΦL(t, t0) (see appendix B), and we obtain, since we assumed L continuous, an implicit

representation of V: e V_{(t) = ΦL(t, t0) e}V_{(t0) +} Z t t0 ΦL(t, s)·  e UV s + eI(s)  ds + Z t t0 ΦL(t, s)· Λ(s)dWs (11)

We now introduce for future reference a simpler model which is quite frequently used in the description on neural networks.

2.2.2 Example: The Simple Model

In the Simple Model, each neuron membrane potential decreases exponentially to its rest value if it receives no input, with a time constantταdepending only on the population. The noise is modeled

by an independent Brownian process per neuron whose standard deviation is the same for all neurons belonging to a given population.

dVi(N )(t) = h −V (N ) i (t) τα + P X β=1 Nβ X j=1 JijSαβ Vj(N )(t)
+ Iα(t) i dt + sαdW(i)(t). (12)

This is a special case of equation (10) wherek = 1 and L = _−diag(1 τ1,· · · ,

1

τP), ΦL(t, t0) =

diag(e−(t−t0)/τ1_{,· · · , e}−(t−t0)/τP_{), and Λ = diag(s}

1,· · · , sP). The corresponding mean field

equation reads: dVα(t) =  −V_τα α (t) + P X β=1 UαβV (t) + Iα(t)  dt + sαdWα(t), ∀α ∈ {1, . . . , P }, (13)

where the processes(Wα(t))t≥0are independent standard Brownian motions, UV(t) = (UαβV (t); α, β ∈

{1, . . . , P })tis the effective interaction term.

This equation can be integrated implicitly and we obtain the following integral representation of the processVα(t): Vα(t) = e−(t−t0)/ταVα(t0) + Z t t0 e−(t−s)/τα P X β=1 UαβV (s) + Iα(s)
ds + sα Z t t0 e−(t−s)/τα_dW α(s)  (14)

wheret0is the initial time. It is an implicit equation on the probability distribution of V(t), a special

case of (11).

2.2.3 The Jansen and Rit’s model

One of the motivations of this study is to characterize the global behavior of an assembly of neurons in particular to get a better understanding of non-invasive cortical signals like EEG or MEG. One of the classical models of neural masses is Jansen and Rit’s mass model (Jansen and Rit, 1995), in short the JR model (see figure 1).

The model features a population of pyramidal neurons (central part of figure 1.a.) that receives excitatory and inhibitory feedback from local inter-neurons and an excitatory input from neighboring cortical units and sub-cortical structures such as the thalamus. The excitatory input is represented by an arbitrary average firing ratep(t) that can be stochastic (accounting for a non specific background

activity) or deterministic, accounting for some specific activity in other cortical units. The transfer functionsheandhiof figure 1 convert the average firing rate describing the input to a population

Figure 1: a. Neural mass model: a population of pyramidal cells interacts with two populations of inter-neurons: an excitatory one and an inhibitory one. b. Block representation of the model. The h boxes account for the synaptic integration between neuronal populations. S boxes simulate cell

bodies of neurons by transforming the membrane potential of a population into an output firing rate. The coefficientJαβ is the random synaptic efficiency of populationβ on population α (P is the

pyramidal population,E the excitatory and I the inhibitory ones), and the constants (Ci) model the

mean strength of the synaptic connections between populations (it is the mean of theJs).

into an average excitatory or inhibitory post-synaptic potential (EPSP or IPSP). They correspond to the synaptic integration.

In the model introduced originally by Jansen and Rit, the connectivity weights were assumed to be constant, equal to their mean value (it is the constantsCi,i = 1 . . . 4 in figure 1). Nevertheless,

there exists a variability on these coefficients, and as we will see in the sequel, the effect of the connectivity variability impacts the solution at the level of the neural mass. Statistical properties

of the connectivities have been studied in details for instance in (Braitenberg and Sch ¨uz, 1998). In our model we consider these connectivities as independent Gaussian random variables of mean and standard deviation equal to the ones found in (Braitenberg and Sch ¨uz, 1998).

We now use diagram 1 to derive the membrane potential expressions. We consider a network of

N neurons belonging to the three populations described. We denote by P (resp E, I) the pyramidal

(respectively excitatory, inhibitory) populations. We choose in populationP (respectively

popula-tionsE, I) a particular pyramidal neuron (respectively excitatory, inhibitory interneuron) indexed

byipyr(respectivelyiexc,iinh). The equations of their activity variable read:

               AN ipyr = he∗ S( X j Exc JijANj + he∗ p(·) + X j Inh JijANj ) AN iexc = he∗ S( X j P yr JijANj ) ANiinh = hi∗ S( X j P yr JijANj )

This is therefore an activity-based model. As stated before, it is equivalent via a change of variable to a voltage-based model, with the same connectivity matrix, the same intrinsic dynamics, and modified inputs (see section 2.1.1).

In the mean field limit, denoting byAP (respectivelyAE,AI) the activity of the pyramidal

neurons (resp excitatory, inhibitory interneurons), we obtain the following activity equations:

     AP = he∗ S(UP E+ he∗ p + UP I) AE = he∗ S(UEP) AI = he∗ S(UIP) (15)

where U= (Uij)_{i,j∈{P,E,I}}is the effective interaction process associated with this problem, i.e. a

Gaussian process of means:

         E [UEP] = ¯JEPE [AE] E [UIP] = ¯JIPE [AI] E [UP I] = ¯JP IE [AP] E [UP E] = ¯JP EE [AP]

and whose covariance matrix can be deduced from (9). The voltage-based model can be deduced from this activity-based description using a simple change of variable as stated previously. Note that the change of variable is possible since the activity currentIA is equal tohe∗ p and, as shown in

section 2.1.1,IAis smooth enough so that we can apply to it the suitable differential operator. p is

the corresponding voltage currentIV.

Let us now instantiate the synaptic dynamics and compare the mean field equation with Jansen’s population equations (sometimes improperly called also mean field equations).

The simplest model of synaptic integration is a first-order integration, which yields exponential post-synaptic potentials: h(t) =  αe−βt _{t≥ 0} 0 t < 0

that satisfies the following differential equations

˙h(t) = −βh(t) + αδ(t),

In these equationsβ is the time constant of the synaptic integration and α the synaptic efficiency.

The coefficients namedα and β are the same for the pyramidal and the excitatory population, and

different from the ones of the inhibitory synapse. In the pyramidal or excitatory (respectively the inhibitory) case we haveα = A, β = a (respectively α = B, β = b). Eventually, the sigmoid

functions are the same whatever the populations, and is given by

S(v) = νmax 1 + er(v0−v),

νmaxis the maximum firing rate, andv0is a voltage reference.

With this synaptic dynamics we obtain the first-order Jansen and Rit’s equation:

     dAP dt (t) =−a AP(t) + A S(UP E+ UP I+ he∗ p(t)) dAE dt (t) =−a AE(t) + A S(UEP) dAI dt (t) =−b AI(t) + B S(UIP) . (16)

while the “original” Jansen and Rit’s equation (Grimbert and Faugeras, 2006; Jansen and Rit, 1995)

reads: _     dAP dt (t) =−a AP(t) + A S(C2AE(t)− C4AI(t) + he∗ p(t)) dAE dt (t) =−a AE(t) + A S(C1AP(t)) dAI dt (t) =−b AI(t) + B S(C3AP(t)) . (17)

Hence the original JR equation amounts to computing the expectation of the activity in each population and to assume that

E [S(UP E+ UP I+ he∗ p(t))] = S(E [UP E+ UP I+ he∗ p]),

which is a quite sharp assumption given that the sigmoidal function is nonlinear.

A higher order model was introduced to better account for the synaptic integration and to better reproduce the characteristics of real EPSPs and IPSPs by van Rotterdam and colleagues (van Rotterdam et al, 1982). In this model the PSP satisfies a second order differential equation:

h(t) = 

αβte−βt _{t≥ 0}

solution of the differential equationy(t) = αβδ(t)¨ − 2β ˙y(t) − β2_{y(t). With this type of synaptic}

integration, we obtain the following mean field equations:

     d2_A P dt2 (t) = AaS(UP E+ UP I+ he∗ p(t)) − 2adA_dtP(t)− a2AP(t) d2_A E dt2 (t) = AaS(UEP)− 2adA_dtE(t)− a2AE(t) d2_A I dt2 (t) = BbS(UIP)− 2bdAdtI(t)− b2AI(t) (18)

while the original system satisfies the equations:

     d2_A P dt2 (t) = AaS(C2AE(t)− C4AI(t) + he∗ p(t)) − 2adA_dtP(t)− a2AP(t) d2_A E dt2 (t) = AaS(C1AP(t))− 2a dAE dt (t)− a2AE(t) d2_A I dt2 (t) = BbC4S(C3AP(t))− 2b d2_A I dt2 (t)− b2AI(t) (19)

Here again, going from the mean field equations (18) to the neural mass model (19) consists in studying the equation of the mean of the process given by (18) and commuting the sigmoidal function with the expectation.

Note that the introduction of higher order synaptic integrations results in richer behaviors. For instance, Grimbert and Faugeras (Grimbert and Faugeras, 2006) showed that some bifurcations can appear in the second-order JR model giving rise to epileptic like oscillations and alpha activity, that do not appear in the first order model.

3

Existence and uniqueness of solutions in finite time

The mean field equation (11) is an implicit equation of the stochastic process(V (t))_t≥t0. We prove

in this section that under some mild assumptions this implicit equation has a unique solution. This solution is a fixed point in the set_M+₁(C([t0, T ],R

kP_{)) of kP -dimensional processes. We construct}

a sequence of processes and prove that it converges in distribution toward this fixed point. We denote by_{X the set of random variables (r.v.) with values in}R

kP_{. We first recall some}

results on the convergence of random variables and stochastic processes.

3.1

Convergence of Gaussian processes

We recall the following result from (Bogachev, 1998).

Theorem 3.1. Let_{Xn}∞n=1be a sequence ofkP -dimensional Gaussian processes defined on [t0, T ]

or on an unbounded interval ofR

2_{. The sequence converges to a Gaussian processX if and only if}

the following three conditions are satisfied: • The sequence_{Xn}∞n=1is uniformly tight.

• The sequenceµn(t) of the mean functions converges for the uniform norm.

2_{In (Bogachev, 1998, Chapter 3.8), the property is stated whenever the mean and covariance are defined on a separable}

• The sequenceCnof the covariance operators converges for the uniform norm.

We now define such a sequence of Gaussian processes.

Let us fixZ0, akP -dimensional Gaussian random variable, independent of the Brownian.

Definition 3.1. LetX an element of_M+₁(C([t0, T ],R

kP_{)). Let F}

kbe the function such that

Fk(X)t= ΦL(t, t0)· Z0+ Z t t0 ΦL(t, s)· eUXs + eI(s)
ds + Z t t0 ΦL(t, s)· Λ(s)dWs where eUX

s and eI(s) are defined in section 2.

Note that, by definition, the random process(Fk(X))t∈[t0,T ],k≥ 1 is the sum of a deterministic

function (defined by the external current) and three independent random processes defined by the initial condition, the interaction between neurons, and the external noise.

LetX be a given stochastic process of_M+₁(C([t0, T ],R

kP_{)) such that X} t0 = Z0. We define the sequence of processes_{Xn}∞n=0∈ M+1(C([t0, T ],R kP_{) by:} ( X0 = X Xn+1 = Fk(Xn) = Fk(n)(X0). (20) In the remaining of this section we show that the sequence of processes_{Fk(n)(X)_}∞

n=0converges

in distribution toward the unique fixed-pointY of Fk.

3.2

Existence and uniqueness of solution for the mean field equations

The following upper and lower bounds are used in the sequel.

Lemma 3.2. We consider the Gaussian process((UX

t · 1)t)t∈[t0,T ]. U

X _{is defined in 2.1 and 1 is}

theP -dimensional vector with all coordinates equal to 1. We have EUX_{t · 1} ∞≤ µ def = max α X β | ¯Jαβ| kSαβk∞ (21)

for allt0≤ t ≤ T . The maximum eigenvalue of its covariance matrix is upperbounded by σmax2 def

= maxαP_βσαβ2 kSαβk2_∞wherekSαβk∞is the supremum of the absolute value ofSαβ.

Proof. The proof is straightforward from definition 3.1. We also noteσ2 min

def

= minα,βσαβ2 .

The proof of existence and uniqueness of solution, and of the convergence of the sequence (20) is in two main steps. We first prove that the sequence of Gaussian processes_{Fk(n)(X)_}∞

n=0, k≥ 1

is uniformly tight by proving that Kolmogorov’s criterion for tightness holds. This takes care of condition 1) in theorem 3.1. We next prove that the sequences of the mean functions and covariance operators are Cauchy sequences for the uniform norms, taking care of conditions 2) and 3).

Theorem 3.3. The sequence of processes F_k(n)(X)

n=0,k≥ 1 is uniformly tight.

Proof. We use Kolmogorov’s criterion for tightness and do the proof fork = 1, the case k > 1 is

similar. If we assume thatn_{≥ 1 and s < t we have}

F1(n)(X)t− F1(n)(X)s= (ΦL(t, t0)− ΦL(s, t0))Xt0 + Z s t0 (ΦL(t, s)− Id)ΦL(s, u)U F₁(n−1)(X) u · 1 du + Z t s ΦL(t, u)U F₁(n−1)(X) u · 1 du + Z s t0 (ΦL(t, s)− Id)ΦL(s, u)Λ(u) dWu+ Z t s ΦL(t, u)Λ(u) dWu + Z s t0 (ΦL(t, s)− Id)ΦL(s, u)I(u) du + Z t s ΦL(t, u)I(u) du

and therefore (Cauchy-Schwarz and Jensen’s inequalities):

1 7kF (n) 1 (X)t− F1(n)(X)sk2≤ kΦL(t, t0)− ΦL(s, t0)k2kXt0k 2 + (s− t0)kΦL(t, s)− Idk2 Z s t0 kΦL(s, u)k2kUF (n−1) 1 (X) u · 1k2du + (t− s) Z t s kΦ L(t, u)k2kUF (n−1) 1 (X) u · 1k2du + Z s t0 ΦL(s, u)(ΦL(t, s)− Id)Λ(u) dWu 2 + Z t s ΦL(t, u)Λ(u) dWu 2

+ (s− t0)2kΦL(t, s)− Idk2Imax2 sup u∈[t0,s] kΦL(s, u)k2 + (t− s)2_I2 max sup u∈[s,t]kΦ L(t, u)k2.

Because_kΦL(t, t0)− ΦL(s, t0)k ≤ |t − s|kLk we see that all terms in the righthand side of the

inequality but the two involving the Brownian motion are of the order of(t− s)2_{. We raise again}

both sides to the second power, use the Cauchy-Schwarz inequality again, and take the expected value:

1 73E h kF1(n)(X)t− F1(n)(X)sk4 i ≤ kΦL(t, t0)− ΦL(s, t0)k4EkXt0k 4 + (s− t0)3kΦL(t, s)− Idk4 Z s t0 kΦL(s, u)k4E  kUF1(n−1)(X) u · 1k4  du + (t_{− s)}3 Z t s kΦ L(t, u)k4E  kUF1(n−1)(X) u · 1k4  du + E " Z s t0 ΦL(s, u)(ΦL(t, s)− Id)Λ(u) dWu 4# + E " Z t s ΦL(t, u)Λ(u) dWu 4# (22) + (s− t0)4kΦL(t, s)− Idk4 sup u∈[t0,s] kΦL(s, u)k4Imax4 + (t_{− s)}4Imax4 sup u∈[s,t]kΦ L(t, u)k4. Remember that UF1(n−1)(X)

u · 1 is a P -dimensional diagonal Gaussian process, noted Yu,

there-fore: EkYuk4= X α EYα(u)4+ X α16=α2 EYα21(u)  EYα22(u)  .

The second order moments are upperbounded by some regular function ofµ and σmax(defined in

lemma 3.2) and, because of the properties of Gaussian integrals, so are the fourth order moments. Let us now evaluateE

 RabA(u) dWu 4

for someP× P matrix A. We have

E   Z b a A_{(u) dWu} 4  = E      Z b a A_{(u) dWu} 2  2   = E      P X i=1   P X j=1 Z b a Aij(u) dWuj   P X k=1 Z b a Aik(u) dWuk !  2   = X i1,i2,j1,j2,k1,k2 E "Z b a Ai1j1(u) dW j1 u Z b a Ai1k1(u) dW k1 u Z b a Ai2j2(u) dW j2 u Z b a Ai2k2(u) dW k2 u # .

Because of the properties of the Brownian process, the last term is the sum of three types of terms: X i1,i2 E   Z b a Ai1j(u) dW j u !2 Z b a Ai2j(u) dW j u !2  ≤ X i1,i2 E   Z b a Ai1j(u) dW j u !4  1/2 E   Z b a Ai2j(u) dW j u !4  1/2 , and X i1,i2,j16=j2 E   Z b a Ai1j1(u) dW j1 u !2 Z b a Ai2j2(u) dW j2 u !2  = X i1,i2,j16=j2 E   Z b a Ai1j1(u) dW j1 u !2  E   Z b a Ai2j2(u) dW j2 u !2  , and X i1,i2,j16=j2 E "Z b a Ai1j1(u) dW j1 u Z b a Ai2j1(u) dW j1 u Z b a Ai1j2(u) dW j2 u Z b a Ai2j2(u) dW j2 u # = E "Z b a Ai1j1(u) dW j1 u Z b a Ai2j1(u) dW j1 u # E "Z b a Ai1j2(u) dW j2 u Z b a Ai2j2(u) dW j2 u # ,

Because of the properties of the stochastic integral,R_abAi1j(u) dW

j u=N (0, Rb a A 2 i1j(u) du 1/2 )

hence, because of the properties of the Gaussian integrals

E   Z b a Ai1j(u) dW j u !4  = k Z b a A2i1j(u) du !2 ,

for some positive constantk. Moreover

E   Z b a Ai1j1(u) dW j1 u !2  =Z b a A2i1j1(u) du, and E "Z b a Ai1j1(u) dW j1 u Z b a Ai2j1(u) dW j1 u # = Z b a

This shows that the two termsE  Rts0(ΦL(t, s)− Id)ΦL(s, u)Λ(u) dWu 4 andE  RstΦL(t, u)Λ(u) dWu 4

in (22) are of the order of(t_{− s)}1+a_wherea

≥ 1. Therefore we have EhkF1(n)(X)t− F1(n)(X)sk4

i

≤ C|t − s|1+a_{, a}

≥ 1

for alls, t in [t0, T ], where C is a constant independent of t, s. According to Kolmogorov criterion

for tightness, the sequence of processesnF₁(n)(X)o∞

n=0is uniformly tight.

The proof forFk,k > 1 is similar.

Let us noteµn_{(t) (respectively C}n_{(t, s)) the mean (respectively the covariance matrix) function of}

Xn = Fk(Xn−1), n≥ 1. We have: Cn+1(t, s) = ΦL(t, t0)ΣZ0ΦL(s, t0)T + Z t∧s t0 ΦL(t, u)Λ(u)Λ(u)TΦL(s, u)Tdu+ Z t t0 Z s t0 ΦL(t, u)cov  e UXn u , eUXvn  ΦL(s, v)Tdu dv (23)

Note that thekP_{× kP covariance matrix cov}UeXn

u , eUXvn



has only one nonzeroP_{× P block:} covUeXn u , eUXvn  kk= cov U Xn u · 1, UXvn· 1
, We have cov UXn u · 1, UXvn· 1
= diag  X β σ2αβE [Sαβ(Xnβ(u))Sαβ(Xnβ(v))]   , and E [Sαβ(Xn(u))Sαβ(Xn(v))] = Z R 2 Sαβ q_Cn

ββ(u, u)Cββn (v, v)− Cββn (u, v)2

q Cn ββ(u, u) x + C n ββ(u, v) q Cn ββ(u, u) y + µnβ(v) ! Sαβ  yqCn ββ(u, u) + µ n β(u)  Dx Dy, (24) where Dx = √1 2πe −x2 2 dx.

Similarly we have µn(t) = ΦL(t, t0)µZ0+ Z t t0 ΦL(t, u)  EhUeXn u i + ˜I(u)du = ΦL(t, t0)µZ0+ Z t t0 ΦL(t, u)     0T P,· · · , 0TP,  X β ¯ Jαβ Z R Sαβ  xqCn ββ(u, u) + µ n β(u)   α=1,··· ,P   T + ˜I(u)    Dxdu

We require the following four lemmas.

Lemma 3.4. For allα = 1,_{· · · , P and n ≥ 1 the quantity C}n

αα(s, s)Cααn (t, t)− Cααn (t, s)2 is

lowerbounded by the positive symmetric function:

θ(s, t) =_{|t − s|λ}2 minλΣ

Z0

minλΓmin,

whereλminis the smallest singular value of the positive symmetric definite matrixΦL(t, t0)ΦL(t, t0)T

fort∈ [t0, T ], λΣ

Z0

min is the smallest eigenvalue of the positive symmetric definite covariance matrix

ΣZ0_{, andλ}Γ

minis the smallest singular value of the matrix Λ(u) for u∈ [t0, T ].

Proof. We use equation (23) which we rewrite as follows, using the group property of the resolvent

ΦL: Cn+1(t, s) = ΦL(t, t0) ΣZ0+ Z t∧s t0 ΦL(t0, u)Λ(u)Λ(u)TΦL(t0, u)Tdu+ Z t t0 Z s t0 ΦL(t0, u)cov  e UXn u , eUXvn  ΦL(t0, v)Tdu dv ! ΦL(t0, s)T.

We now assumes < t and introduce the following notations, dropping the index n for simplicity: A(s) = ΣZ0₊Rs t0ΦL(t0, u)Λ(u)Λ(u) T_Φ L(t0, u)Tdu B(s, t) = R_stΦL(t0, u)Λ(u)Λ(u)TΦL(t0, u)Tdu a(t, s) = R_tt₀R_ts₀ΦL(t0, u)cov  e UXn u , eUXvn  ΦL(t0, v)Tdu dv

Leteα,α = 1,· · · , kP , be the unit vector of the canonical basis whose coordinates are all equal to

0 except theαth one which is equal to 1. We note Eα(t) the vector ΦL(t, t0)Teα. We have

Cαα(t, s) = Eα(t)T(A(s) + a(t, s)) Eα(s)

Cαα(s, s) = Eα(s)T(A(s) + a(s, s))Eα(s)

Cαα(t, t) = Eα(t)T(A(s) + B(s, t) + a(t, t))Eα(t).

Note that the last expression does not depend ons, since A(s) + B(s, t) = A(t), which is consistent

with the first equality. The reason why we introduces in this expression is to simplify the following

The expressionCαα(s, s)Cαα(t, t)− Cαα(t, s)2is the sum of four sub-expressions:

e1(s, t) = Eα(s)TA(s)Eα(s)
Eα(t)TA(s)Eα(t)
− Eα(t)TA(s)Eα(s)
2

,

which is greater than or equal to 0 becauseA(s) is a covariance matrix,

e2(s, t) = Eα(s)Ta(s, s)Eα(s)
Eα(t)Ta(t, t)Eα(t)
− Eα(t)Ta(t, s)Eα(s)
2

,

which is also greater than or equal to 0 becausea(t, s) is a covariance matrix function, e3(s, t) = Eα(s)TA(s)Eα(s)
Eα(t)Ta(t, t)Eα(t)
+

Eα(t)TA(s)Eα(t)
Eα(s)Ta(s, s)Eα(s)
−

2 Eα(t)TA(s)Eα(s)
Eα(t)Ta(t, s)Eα(s)

Becausea(t, s) is a covariance matrix function we have

Eα(t)Ta(t, t)Eα(t) + Eα(s)Ta(s, s)Eα(s)− 2Eα(t)Ta(t, s)Eα(s)≥ 0,

and , as seen above,e2(s, t)≥ 0. Because e1(s, t)≥ 0 we also have

− q Eα(s)TA(s)Eα(s) q Eα(t)TA(s)Eα(t)≤ Eα(t)TA(s)Eα(s)≤ q Eα(s)TA(s)Eα(s) q Eα(t)TA(s)Eα(t),

and, as it can be readily verified, this impliese3(s, t)≥ 0.

Therefore we can lowerboundCαα(s, s)Cαα(t, t)− Cαα(t, s)2by the fourth subexpression:

Cαα(s, s)Cαα(t, t)− Cαα(t, s)2≥ Eα(s)TA(s)Eα(s)
Eα(t)TB(s, t)Eα(t)
+

Eα(s)Ta(s, s)Eα(s)
Eα(t)TB(s, t)Eα(t)
≥

Eα(s)TA(s)Eα(s)
Eα(t)TB(s, t)Eα(t)
,

sinceB(s, t) and a(s, s) are covariance matrixes. We next have Eα(s)TA(s)Eα(s) = Eα(s)TA(s)Eα(s) Eα(s)TEα(s) eT αΦL(s, t0)ΦL(s, t0)Teα eT αeα , by definition ofEα(s). Therefore

Eα(s)TA(s)Eα(s)≥ λA(s)minλ

ΦL(s,t0)ΦL(s,t0)T

min ≥ λ ΣZ0

minλmin,

whereλC

minis the smallest eigenvalue of the symmetric positive matrixC. Similarly we have

Let us writeΓ(u) = Λ(u)Λ(u)T_{. We have} λB(s,t)min = min kxk≤1 Z t s xT_Φ L(t0, u)Γ(u)ΦL(t0, u)Tx xT_x du = min kxk≤1 Z t s xT_Φ L(t0, u)Γ(u)ΦL(t0, u)Tx xT_Φ L(t0, u)ΦL(t0, u)x xT_Φ L(t0, u)ΦL(t0, u)x xT_x du≥ Z t s min kxk≤1  xT_Φ L(t0, u)Γ(u)ΦL(t0, u)Tx xT_Φ L(t0, u)ΦL(t0, u)x xT_Φ L(t0, u)ΦL(t0, u)x xT_x  du≥ (t_{− s)λ}minλΓmin.

Combining these results we have

Cαα(s, s)Cαα(t, t)− Cαα(t, s)2≥ |t − s|λ2minλΣ

Z0

minλΓmin

Lemma 3.5. For allt_{∈ [t}0, T ] all α = 1,· · · , P , and n ≥ 1, we have

Cααn (t, t)≥ k0> 0.

Proof. Cn

αα(t, t) is larger than (ΦL(t, t0)ΣZ0ΦL(t, t0)T)ααwhich is larger than the smallest

eigen-value of the matrixΦL(t, t0)ΣZ0ΦL(t, t0)T. This smallest eigenvalue is equal to

min x xT_Φ L(t, t0)ΣZ0ΦL(t, t0)Tx xT_x = min x xT_Φ L(t, t0)ΣZ0ΦL(t, t0)Tx xT_Φ L(t, t0)ΦL(t, t0)Tx xT_Φ L(t, t0)ΦL(t, t0)Tx xT_x ≥ min x xT_Φ L(t, t0)ΣZ0ΦL(t, t0)Tx xT_Φ L(t, t0)ΦL(t, t0)Tx minx xT_Φ L(t, t0)ΦL(t, t0)Tx xT_x .

In the last expression the first term is larger than the smallest eigenvalueλΣZ0

min of the matrixΣZ0

which is positive definite since we have assumed the Gaussian random variableZ0nondegenerate.

The second term is equal to the smallest singular valueλminof the matrixΦL(t, t0) which is also

strictly positive for allt_{∈ [t}0, T ] by hypothesis, see appendix B, equation (47).

We also use the following lemma.

Lemma 3.6. The2n-dimensional integral In = Z [t0,t∨s]2 ρ1(u1, v1) Z [t0,u1∨v1]2 · · · Z [t0,un−2∨vn−2]2 ρn−1(un−1, vn−1) Z [t0,un−1∨vn−1]2 ρn(un, vn)dundvn ! du_n−1dv_n−1 ! · · · ! du1dv1,

where the functionsρi(ui, vi), i = 1,· · · , n are either equal to 1 or to 1/pθ(ui, vi), is

upper-bounded bykn_{/(n− 1)! for some positive constant k.}

Proof. First note that the integral is well-defined because of lemma 3.4. Second, note that there exists a constantK such that K/pθ(u, v)_{≥ 1 for all (u, v) ∈ [t}0, t∨s]2, i.e.K = λmin

q λΣZ0

minλΓmin(T − t0).

Therefore the integral is upperbounded byKn

0, whereK0= max(1, K) times the integral obtained

whenρi(ui, vi) = 1/

p

|ui− vi| for all i = 1, · · · , n. Let us then consider this situation. Without

loss of generality we assumet0= 0. The cases n = 1, 2, 3 allow one to understand the process.

I1≤ K0 Z [0,t∨s]2 dudv p |u − v|. (25)

Let us rotate the axes by₋π₄ by performing the change of variables

u = U + V√ 2 , v = V_√− U

2 .

Using the symmetry of the integrand ins and t and the change of variable, the integral in the

righthand side of (25) is equal to (see figure 2):

2 1 21/4 Z t∨s_√ 2 0 Z √ 2(t∨s)−U U dV dU √ U = 2 3/4Z a/2 0 a_{− 2U} √ U dU = 2 3/4_α 1a3/2, wherea =√2(t_{∨ s) and α}1= 2 √ 2 3 .

Let us now look atI2. It is upperbounded by the factorK02(23/4)2α1times the integral

Z a/2 0 Z a−U U (√2(u∨ v))3/2 √ U dU dV.

Since in the area of integrationu∨ v = v = V −U_√

2 we are led to the product of2/5 by the

one-dimensional integral Z a/2 0 (a_{− 2U)}5/2 √ U dU dV = α2a 3_, whereα2=5 √ 2π 32 .

SimilarlyI3is upperbounded by the product ofK03(23/4)3α1α22₅2₈times the integral

Z a/2 0 (a_{− 2U)}4 √ U dU dV = α3a 9/2_, whereα3=128 √ 2

315 . One easily shows then that:

In ≤ K0nF (23/4)n2n n Y i=1 αi ! 1 Qn j=1(2 + 3(j− 1)) ! .

It can be verified by using a system for symbolic computation that0 < αi < 1 for all i ≥ 1. One

also notices that

n Y j=1 (2 + 3(j_{− 1)) ≥} 3n−1 2 (n− 1)!, therefore In ≤ K0n(23/4)n2n−13−(n−1) 1 (n_{− 1)!},

and this finishes the proof.

We now prove the following proposition.

Proposition 3.7. The sequences of covariance matrix functions Cn_{(t, s) and of mean functions}

µn_{(t), s, t in [t}

0, T ] are Cauchy sequences for the uniform norms.

Proof. We have Cn+1_{(t, s)} − Cn_{(t, s) =}Z t t0 Z s t0 ΦL(t, u)  covUeXn u , eUXvn  − covUeXn−1 u , eUXvn−1   ΦL(s, v)Tdu dv.

We take the infinite matrix norm of both sides of this equality and use the upperbounds_kΦL(t, u)k_∞≤

ekLk_∞(T −t0)_{= k} Land ΦL(t, u)T ∞≤ ek LTk

∞(T −t0)= k_LT (see appendix B) to obtain

Cn+1_{(t, s)} − Cn_{(t, s)} ∞≤ kLkLT Z t t0 Z s t0 covUeXn u , eUXvn  − covUeXn−1 u , eUXvn−1 v ∞du dv = kLkLT Z t t0 Z s t0 cov UXn u · 1, UXvn· 1
− cov UXn−1 u · 1, UXvn−1· 1
v ∞du dv. (26)

According to equations (24) we are led to consider the differenceAn− An−1, where:

An def

= Sαβ

qCn

ββ(u, u)Cββn (v, v)− Cββn (u, v)2

q Cn ββ(u, u) x+ C n ββ(u, v) q Cn ββ(u, u) y+µnβ(v)  Sαβ  yqCn ββ(u, u) + µ n β(u)  def

= Sαβfβn(u, v)x + gnβ(u, v)y + µnβ(v)



Sαβhnβ(u)y + µnβ(u)

 .

We write next:

An− An−1= Sαβfβn(u, v)x + gβn(u, v)y + µnβ(v)

  Sαβhnβ(u)y + µnβ(u)  − Sαβ h hn−1_β (u)y + µn−1_β (u)i+ Sαβ h hn−1_β (u)y + µn−1_β (u)i 

Sαβfβn(u, v)x + gβn(u, v)y + µnβ(v)

 − Sαβ

h

f_βn−1(u, v)x + g_βn−1(u, v)y + µn−1_β (v)i.

The mean value theorem yields:

| An− An−1|≤ kSαβk_∞ S′ αβ ∞  | x | | fn β(u, v)− fβn−1(u, v)| + | y | | gβn(u, v)− gn−1β (u, v)| + | µ n β(v)− µn−1β (v)| + | y | | h n β(u)− hn−1β (u)| + | µn β(u)− µn−1β (u)|  .

Using the fact thatR∞

−∞ | x | Dx = q 2 π, we obtain: Cn+1_{(t, s)} − Cn_{(t, s)} ∞≤ kLkLTkC r 2 π Z t t0 Z s t0 fn_{(u, v)} − fn−1_{(u, v)} ∞dudv+ r 2 π Z t t0 Z s t0 gn_{(u, v)} − gn−1(u, v) _∞dudv+ (t_{− t}0) Z s t0 µn_(v) − µn−1(v) ∞dv + (s− t0) Z t t0 µn_(u) − µn−1(u) ∞du+ r 2 π(s− t0) Z t t0 hn_(u) − hn−1_(u) ∞du ! , where kC= max α X β σαβ2 kSαβk_∞ S′αβ ∞. (27)

A similar process applied to the mean values yields:

µn+1_(t) − µn(t) _{∞≤ k}Lµ  Z t t0 hn_(u) − hn−1(u) _∞du+ Z t t0 µn_(u) − µn−1(u) _∞du.

We now use the mean value theorem and lemmas 3.5 and 3.4 to find upperbounds for fn(u, v)− fn−1_{(u, v)} ∞, gn_{(u, v)− g}n−1_{(u, v)} ∞and hn_{(u)− h}n−1_(u) ∞. We have |hn β(u)− hn−1β (u)| =     q Cn ββ(u, u)− q C_ββn−1(u, u)     ≤ ₂√1_k0   Cββn (u, u)− Cββn−1(u, u)    ,

wherek0is defined in lemma 3.5. Hence:

hn_(u) − hn−1_(u) ∞≤ 1 2√k0 Cn_{(u, u)} − Cn−1_{(u, u)} ∞.

Along the same lines we can show easily that:

gn_{(u, v)} − gn−1_{(u, v)} ∞≤ k  Cn_{(u, v)} − Cn−1_{(u, v)} ∞+ Cn_{(u, u)} − Cn−1_{(u, u)} ∞  , and that: fn_{(u, v)} − fn−1_{(u, v)} ∞≤ k p θ(u, v)  _Cn (u, v)− Cn−1_{(u, v)} ∞+ Cn_{(u, u)} − Cn−1_{(u, u)} ∞+ Cn_{(v, v)} − Cn−1_{(v, v)} ∞  ,

whereθ(u, v) is defined in lemma 3.4. Grouping terms together and using the fact that all integrated

functions are positive, we write:

Cn+1_{(t, s)} − Cn(t, s) ∞≤ k Z [t0,t∨s]2 1 p θ(u, v) Cn_{(u, v)} − Cn−1_{(u, v)} ∞ dudv+ Z [t0,t∨s]2 1 p θ(u, v) Cn_{(u, u)} − Cn−1_{(u, u)} ∞ dudv+ Z [t0,t∨s]2 Cn_{(u, v)} − Cn−1_{(u, v)} ∞ dudv+ Z [t0,t∨s]2 Cn_{(u, u)} − Cn−1(u, u) _∞dudv+ Z [t0,t∨s]2 µn_(u) − µn−1_(u) ∞dudv ! . (28)

Note that, because of lemma 3.5, all integrals are well-defined. Regarding the mean functions, we write: µn+1_(t) − µn(t) ∞≤ k Z [t0,t∨s]2 Cn_{(u, u)} − Cn−1(u, u) ∞dudv+ Z [t0,t∨s]2 µn_(u) − µn−1(u) _∞dudv ! . (29)

Proceeding recursively until we reachC0_andµ0_{we obtain an upperbound for C}n+1_{(t, s)}

− Cn_{(t, s)} ∞

(respectively for µn+1(t)_{− µ}n_(t)

∞) which is the sum of less than5

n _{terms each one being the}

product ofk raised to a power less than or equal to n, times 2µ or 2Σ (upperbounds for the norms

of the mean vector and the covariance matrix), times a2n-dimensional integral Ingiven by

Z [t0,t∨s]2 ρ1(u1, v1) Z [t0,u1∨v1]2 · · · Z [t0,un−2∨vn−2]2 ρn−1(un−1, vn−1) Z [t0,un−1∨vn−1]2 ρn(un, vn)dundvn ! dun−1dvn−1 ! · · · ! du1dv1,

where the functionsρi(ui, vi), i = 1,· · · , n are either equal to 1 or to 1/pθ(ui, vi). According

to lemma 3.6, this integral is of the order of some positive constant raised to the powern divided

by(n− 1)!. Hence the sum is less than some positive constant k raised to the power n divided by (n− 1)!. By taking the supremum with respect to t and s in [t0, T ] we obtain the same result for

Cn+1_{− C}n

∞ (respectively for

µn+1_{− µ}n

∞). Since the series

P

n≥1 kn

n! is convergent, this

implies that_kCn+p_{− C}n_k∞(respectively_kµn+p_{− µ}n_k∞) can be made arbitrarily small for large

n and p and the sequence Cn_{(respectivelyµ}n_{) is a Cauchy sequence.}

We can now prove the following theorem

Theorem 3.8. For any nondegeneratekP -dimensional Gaussian random variable Z0and any initial

processX such that X(t0) = Z0, the mapFk has a unique fixed point in M+1(C([t0, T ],R

kP₎₎

towards which the sequence_{Fk(n)(X)_}∞

n=0of Gaussian processes converges in law.

Proof. SinceC([t0, T ],R

kP_{) (respectively C([t}

0, T ]2,R

kP ×kP_{)) is a Banach space for the uniform}

norm, the Cauchy sequenceµn (respectivelyCn) of proposition 3.7 converges to an elementµ of C([t0, T ],R

kP_{) (respectively an element C of C([t}

0, T ]2,R

kP ×kP_{)). Therefore, according to}

the-orem 3.1, the sequence_{Fk(n)(X)_}∞

n=0of Gaussian processes converges in law toward the Gaussian

processY with mean function µ and covariance function C. This process is clearly a fixed point of Fk.

Hence we know that there there exists at least one fixed point for the mapFk. Assume there

Ci, i = 1, 2, with the same initial condition. Since for all n ≥ 1 we have Fk(n)(Yi) = Yi,

i = 1, 2, the proof of proposition 3.7 shows that_kµn

1− µn2k_∞(respectively Cn 1 − Cn2 ∞) is

upper-bounded by the product of a positive numberan (respectivelybn) withkµ1− µ2k_∞(respectively

with_kC1− C2k_∞). Sincelimn→∞an = limn→∞bn = 0 and µni = µi,i = 1, 2 (respectively

Cn

i = Ci,i = 1, 2), this shows that µ1 = µ2andC1 = C2, hence the two Gaussian processesY1

andY2are indistinguishable.

Conclusion

We have proved that for any non degenerate initial conditionZ0 there exists a unique solution of

the mean field equations. The proof of theorem 3.8 is constructive, and hence provides a way for computing the solution of the mean field equations by iterating the mapFk, starting from any initial

processX satisfying X(t0) = Z0, for instance a Gaussian process such as an Ornstein-Uhlenbeck

process. We build upon these facts in section 5.

Note that the existence and uniqueness is true whatever the initial timet0and the final timeT .

4

Existence and uniqueness of stationary solutions

So far, we have investigated the existence and uniqueness of solutions of the mean field equation for a given initial condition. We are now interested in investigating stationary solutions, which allow for some simplifications of the formalism.

A stationary solution is a solution whose probability distribution does not change under the flow of the equation. These solutions have been already investigated by several authors (see (Brunel and Hakim, 1999; Sompolinsky et al, 1988)). We propose a new framework to study and simulate these pro-cesses. Indeed we show in this section that under a certain contraction condition there exists a unique solution to the stationary problem. As in the previous section our proof is constructive and provides a way to simulate the solutions.

Remark. The long-time mean field description of a network is still a great endeavor in mathematics

and statistical physics. In this section we formally take the mean field equation we obtained and lett0→ −∞. This way we obtain an equation which is the limit of the mean field equation when

t0 → −∞. It means that we consider first the limit N → ∞ and then t0→ −∞. These two limits

do not necessarily commute and there are known examples, for instance in spin glasses, where they do not.

It is clear that in order to get stationary solutions, we have to assume that the leak matrix L(t)

does not depend upont. Therefore, the resolvent ΦL(t, s) is equal to eL(t−s). To ensure stability

of the solutions and the existence of a stationary process we also assume that the real parts of its eigenvalues are negative:

Re(λ) <_−λL λL> 0 (30)

for all eigenvaluesλ of L. This implies that we only consider first-order system since otherwise the