Linear response in neuronal networks: from neurons dynamics to collective response

(1)

HAL Id: cel-01986987

https://hal.archives-ouvertes.fr/cel-01986987

Submitted on 21 Jan 2019

HAL is a multi-disciplinary open access

archive for the deposit and dissemination of

sci-entific research documents, whether they are

pub-lished or not. The documents may come from

teaching and research institutions in France or

abroad, or from public or private research centers.

L’archive ouverte pluridisciplinaire HAL, est

destinée au dépôt et à la diffusion de documents

scientifiques de niveau recherche, publiés ou non,

émanant des établissements d’enseignement et de

recherche français ou étrangers, des laboratoires

publics ou privés.

dynamics to collective response

Bruno Cessac

To cite this version:

Bruno Cessac. Linear response in neuronal networks: from neurons dynamics to collective response.

Doctoral. Chile. 2019. �cel-01986987�

(2)

Linear response in neuronal networks:

from neurons dynamics to collective response

Bruno Cessac

Biovision Team, INRIA Sophia Antipolis,France.

14/15-01-2019

(3)

Neuron response to a stimulus

https://www.plasticitylab.com/methods/

(4)

Neuron response to a stimulus

(5)

Network response to a stimulus

(6)

(7)

From firing rate neurons dynamics to linear response. From spiking neurons dynamics to linear response. General conclusions Appendix: Linear response theory in physics vs linear response in neuronal networks

Network response to a stimulus

1

_{How does an input/ stimulation applied to a subgroup of}

neurons in a population affect the dynamics of the whole

network ?

another neuron ?

3

How does this ”effective connectivity” relates to :

(a)

Synaptic connectivity;

(b)

Pairwise correlations;

(c)

”Information” transport.

(8)

Network response to a stimulus

1

_{How does an input/ stimulation applied to a subgroup of}

neurons in a population affect the dynamics of the whole

network ?

2

How to measure the influence of a stimulated neuron on

another neuron ?

(a)

Synaptic connectivity;

(b)

Pairwise correlations;

(9)

Network response to a stimulus

1

_{How does an input/ stimulation applied to a subgroup of}

neurons in a population affect the dynamics of the whole

network ?

2

How to measure the influence of a stimulated neuron on

another neuron ?

3

How does this ”effective connectivity” relates to :

(a)

Synaptic connectivity;

(b)

Pairwise correlations;

(c)

”Information” transport.

(10)

1 From firing rate neurons dynamics to linear response.

2 From spiking neurons dynamics to linear response.

3 General conclusions

4 Appendix: Linear response theory in physics vs linear response

in neuronal networks

(11)

From firing rate neurons dynamics to linear

response.

(12)

Amari-Wilson-Cowan model

Amari, 1971; Wilson-Cowan, 1972; Cohen-Grossberg, 1983; Sompolinsky et al, 1988; ...

dV

i

dt

= −µV

i

+

N

X

j =1

J

ij

f (V

j

(t)) + S

i

(t);

i = 1 . . . N.

(1)

Ex: J

ij

∼ N

0,

J

_N

2

(Sompolinsky et al, 1988)

Ex: f (x ) =

1 ₂

( 1 + tanh(gx ) ),

f (x ) = tanh(gx ).

(13)

Amari-Wilson-Cowan model

dV

i

dt

= −µV

i

+

N

X

j =1

J

ij

f (V

j

(t)) + S

i

(t);

i = 1 . . . N.

(1)

Network

Ex: J

ij

∼ N

0,

J

_N

2

Non linearity

Ex: f (x ) =

1 ₂

( 1 + tanh(gx ) ),

f (x ) = tanh(gx ).

(14)

Amari-Wilson-Cowan model

d ~

V

dt

= −µ ~

V + J .f ( ~

V ) + ~

S (t);

i = 1 . . . N.

(1)

Network

Ex: J

ij

∼ N

0,

J

2

N

Non linearity

Ex: f (x ) =

1

2 ( 1 + tanh(gx ) ),

f (x ) = tanh(gx ).

(15)

Amari-Wilson-Cowan model

d ~

V

dt

= −µ ~

|

V + J .f ( ~

{z

V )

}

~

G ( ~

V )

+~

S (t);

i = 1 . . . N.

(1)

Network

Ex: J

ij

∼ N

0,

J

_N

2

Non linearity

Ex: f (x ) =

1 ₂

( 1 + tanh(gx ) ),

f (x ) = tanh(gx ).

(16)

Low gain g dynamics

Theorem. If g is small enough ~

G is contractive.

∀ ~

V , ~

V

0 ∈ M,

k ~

G ( ~

V

0 ) − ~

G ( ~

V )k ≤ ηk ~

V

0 − ~

V k,

0 < η < 1

⇒

(17)

Low gain g dynamics

Small perturbation of the fixed point. ~

V = ~

V

∗

+ ~

ξ.

d ~

ξ

dt

= DG

V

~

∗

.~

ξ + ~

S (t) + O

k~

ξk

2 ~

ξ(t) =

Z

t

−∞

e

DG

V ∗~

(t−s)

.~

S (s)ds

Λ = P−1.DG~V ∗.P; ~ξ = P ~ξ 0_{; ~}_{S = P ~}_S0

⇒ ~

_ξ

0 _{(t) =}

Z

t

−∞

e

Λ(t−s)

.~

S

0 (s)ds

ξ

0 _k

(t) =

Z

t

−∞

e

λ

k

(t−s)

_.S

0 k

(s)ds

(18)

Low gain g dynamics

Harmonic perturbation. S

_k

0 (t) = A

0 _k

e

i ωt

.

λ

k

= λ

k,r

+ i λ

k,i

;

ω = ω

r

+ i ω

i

.

The integral is finite if ω

i

< −λ

k,r

.

ξ

_k

0 (t) = ˆ

χ

0 _k

(ω)e

i ωt

.

Complex susceptibility matrix.

~

(19)

Low gain g dynamics

40 35 30 25 20 15 10 5 0.6 _0.8 1 _1.2 1.4 _1.6 1.8 ₂

ω

_r 0 0.2 0.4 0.6 0.8 1

ω

_ι 0 5 10 15 20 25 30 35 40 45

|χ(ω)|

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 -4 -2 0 2 4 6 λι ∆ ω |χ(ω)| ω

(20)

Low gain g dynamics

40 35 30 25 20 15 10 5 -1 -0.5 ₀ 0.5 ₁ 1.5 ₂ 2.5 ₃

ω

_r 0 0.2 0.4 0.6 0.8 1

ω

_ι 0 5 10 15 20 25 30 35 40 45

|χ (ω)|

0 1 2 3 4 5 6 7 8 9 -10 -5 0 5 10 |χ κ (ω)| ω

(21)

Low gain g dynamics

Example 1. f (x ) = tanh(gx ) ⇒

~

V

∗

= ~0;

DG

_V

~

∗

= −µI + g J

Let s

k

≡ s

k,r

+ is

k,i

eigenvalues of J .

λ

k

= −µ + g s

k,r

|

{z

}

λ

k,r

±i g s

_k,i

| {z }

λ

k,i

When J is random, J

ij

∼ N (0,

J

2

N

) the probability distribution of

eigenvalues is known.

(

Girko, V. L., Theory Probab. Appl. 29, 694-706, 1984. ).

(22)

Low gain g dynamics

Example 2. f (x ) =

1+tanh(gx )

₂

⇒

~

V

∗

≡ ~

V

∗

(J );

DG

_V

~

∗

= −µI + gD( ~

V

∗

)J

where D( ~

V

∗

) = diag (

1−tanh

2

_(gV

∗ i

)

2 ).

The eigenvalues of D( ~

V

∗

)J cannot be determined from the

eigenvalues of J . However, when J is random, J

ij

∼ N (0,

J

2

N

) the

probability distribution of eigenvalues can be determined.

(23)

Low gain g dynamics

Summary:

The linear response to a signal of weak amplitude is controlled

by the Jacobian matrix DG

_V

~

∗

.

Eigenvalues of DG

_V

~

∗

⇒ Poles of the complex susceptibility ⇒

Resonances.

What is the phenomenological/neuronal interpretation of

DG

_V

~

∗

?

(24)

Low gain g dynamics

Summary:

The linear response to a signal of weak amplitude is controlled

by the Jacobian matrix DG

_V

~

∗

.

Eigenvalues of DG

_V

~

∗

⇒ Poles of the complex susceptibility ⇒

Resonances.

What is the phenomenological/neuronal interpretation of

DG

_V

~

∗

?

DG

_V

~

∗

= − µI

|{z}

Leak

+ g

|{z}

Gain

D( ~

V

∗

)

|

{z

}

f

0

J

|{z}

Synapses

(25)

Low gain g dynamics

Summary:

The linear response to a signal of weak amplitude is controlled

by the Jacobian matrix DG

_V

~

∗

.

Eigenvalues of DG

_V

~

∗

⇒ Poles of the complex susceptibility ⇒

Resonances.

What is the phenomenological/neuronal interpretation of

DG

_V

~

∗

?

DG

_V

~

∗

= − µI

|{z}

Leak

+ g

|{z}

Gain

D( ~

V

∗

)

|

{z

}

f

0

J

|{z}

Synapses

(26)

Expansion/Contraction

(27)

Expansion/Contraction

(28)

(29)

Expansion/Contraction

(30)

(31)

Linear response in a dynamical regime

dV

i

dt

= −µV

i

+

N

X

j =1

J

ij

f (V

j

(t)) + I

i

(t);

i = 1 . . . N.

V

i

(t + dt) = V

i

(t)(1 − µdt) +

N

X

j =1

J

ij

f (V

j

(t))dt + S

i

(t)dt

V

i

(t + 1) =

N

X

j =1

J

ij

f (V

j

(t)) + S

i

(t).

(3)

(32)

Linear response in the chaotic regime

B. Cessac, J.A. Sepulchre, PRE (2004); Chaos (2006); Physica D (2006)

~

V (t + 1) = J .f ( ~

V (t))

|

{z

}

~

G ( ~

V (t))

+~

S (t)

f (x ) = tanh(g x )

(33)

Transition to chaos by quasi periodity as g increases

g

−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−→

Doyon B. et al, International Journal Of Bifurcation and Chaos, Vol. 3, Num. 2, 279-291 (1993) Cessac B. et al, Physica D, 74, 24-44 (1994)

(34)

(35)

Chaotic dynamics and strange attractors

https://upload.wikimedia.org/wikipedia/commons/a/ac/

H´

enon map

x (t + 1)

= 1 − ax

2 (t) + y (t)

y (t + 1)

= bx (t)

a = 1.4; b = 0.3

(36)

Chaotic dynamics and strange attractors

https://upload.wikimedia.org/wikipedia/commons/a/ac/

H´

enon map

x (t + 1)

= 1 − ax

2 (t) + y (t)

y (t + 1)

= bx (t)

a = 1.4; b = 0.3

(37)

Chaotic dynamics and strange attractors

http://www.demonstrations.wolfram.com/OrbitDiagramOfTheHenonMap/HTMLImages/index.en/

H´

enon map

x (t + 1)

= 1 − ax

2 (t) + y (t)

y (t + 1)

= bx (t)

a = 1.4; b = 0.3

(38)

Chaotic dynamics and strange attractors

http://www.sfu.ca/ rpyke/335/W00/

H´

enon map

x (t + 1)

= 1 − ax

2 (t) + y (t)

y (t + 1)

= bx (t)

a = 1.4; b = 0.3

(39)

Chaotic dynamics and strange attractors

(40)

Chaotic dynamics and strange attractors

Expansive Positive Lyapunov exponent

(41)

Chaotic dynamics and strange attractors

(42)

Time dependent perturbation

~

V (t + 1) = ~

G ( ~

V (t));

V

~

0 (t + 1) = ~

G ( ~

V

0 (t)) + ~

S (t)

Switch the stimulus on at time t

0 ; ~

V (t

0 ) = ~

V

0 (t

0 ).

~

_{δ(t) = ~}

_V

0 _{(t) − ~}

_{V (t) ⇒ ~}

_δ(t

0 + 1) = ~

V

0 (t

0 + 1) − ~

V (t

0 + 1) = ~

S (t

0 )

~

_δ(t

₀

_{+ 2) = ~}

_{G ( ~}

_V

0 _(t

0 + 1)) + ~

S (t

0 + 1) − ~

G ( ~

V (t

0 + 1))

~

_δ(t

0 + 2) = ~

G ( ~

V (t

0 + 1) + ~

S (t

0 )) + ~

S (t

0 + 1) − ~

G ( ~

V (t

0 + 1))

~

δ(t

0 + 2) =

h

DG

_{V (t}

~

₀

₊₁₎

.~

S (t

0 ) + ~

S (t

0 + 1)

i

+

2 ~

η(t

0 + 1)

δ(t) =

τ =t

0

DG

_~

V (t

0

+1)

.~

S (τ ) +

R(t)

(43)

Time dependent perturbation

~

V (t + 1) = ~

G ( ~

V (t));

V

~

0 (t + 1) = ~

G ( ~

V

0 (t)) + ~

S (t)

Switch the stimulus on at time t

0 ; ~

V (t

0 ) = ~

V

0 (t

0 ).

~

_{δ(t) = ~}

_V

0 _{(t) − ~}

_{V (t) ⇒ ~}

_δ(t

0 + 1) = ~

V

0 (t

0 + 1) − ~

V (t

0 + 1) = ~

S (t

0 )

~

_δ(t

₀

_{+ 2) = ~}

_{G ( ~}

_V

0 _(t

0 + 1)) + ~

S (t

0 + 1) − ~

G ( ~

V (t

0 + 1))

~

_δ(t

0 + 2) = ~

G ( ~

V (t

0 + 1) + ~

S (t

0 )) + ~

S (t

0 + 1) − ~

G ( ~

V (t

0 + 1))

~

δ(t

0 + 2) =

h

DG

_{V (t}

~

₀

₊₁₎

.~

S (t

0 ) + ~

S (t

0 + 1)

i

+

2 ~

η(t

0 + 1)

δ(t) =

τ =t

0

DG

_~

V (t

0

+1)

.~

S (τ ) +

R(t)

(44)

Time dependent perturbation

~

V (t + 1) = ~

G ( ~

V (t));

V

~

0 (t + 1) = ~

G ( ~

V

0 (t)) + ~

S (t)

Switch the stimulus on at time t

0 ; ~

V (t

0 ) = ~

V

0 (t

0 ).

~

_{δ(t) = ~}

_V

0 _{(t) − ~}

_{V (t) ⇒ ~}

_δ(t

0 + 1) = ~

V

0 (t

0 + 1) − ~

V (t

0 + 1) = ~

S (t

0 )

~

_δ(t

₀

_{+ 2) = ~}

_{G ( ~}

_V

0 _(t

0 + 1)) + ~

S (t

0 + 1) − ~

G ( ~

V (t

0 + 1))

~

_δ(t

0 + 2) = ~

G ( ~

V (t

0 + 1) + ~

S (t

0 )) + ~

S (t

0 + 1) − ~

G ( ~

V (t

0 + 1))

~

δ(t

0 + 2) =

h

DG

_{V (t}

~

₀

₊₁₎

.~

S (t

0 ) + ~

S (t

0 + 1)

i

+

2 ~

η(t

0 + 1)

δ(t) =

τ =t

0

DG

_~

V (t

0

+1)

.~

S (τ ) +

R(t)

(45)

Time dependent perturbation

~

V (t + 1) = ~

G ( ~

V (t));

V

~

0 (t + 1) = ~

G ( ~

V

0 (t)) + ~

S (t)

Switch the stimulus on at time t

0 ; ~

V (t

0 ) = ~

V

0 (t

0 ).

~

_{δ(t) = ~}

_V

0 _{(t) − ~}

_{V (t) ⇒ ~}

_δ(t

0 + 1) = ~

V

0 (t

0 + 1) − ~

V (t

0 + 1) = ~

S (t

0 )

~

_δ(t

₀

_{+ 2) = ~}

_{G ( ~}

_V

0 _(t

0 + 1)) + ~

S (t

0 + 1) − ~

G ( ~

V (t

0 + 1))

~

_δ(t

0 + 2) = ~

G ( ~

V (t

0 + 1) + ~

S (t

0 )) + ~

S (t

0 + 1) − ~

G ( ~

V (t

0 + 1))

~

δ(t

0 + 2) =

h

DG

_{V (t}

~

0

+1)

.~

S (t

0 ) + ~

S (t

0 + 1)

i

+

2 ~

η(t

0 + 1)

δ(t) =

τ =t

0

DG

_~

V (t

0

+1)

.~

S (τ ) +

R(t)

(46)

Time dependent perturbation

~

V (t + 1) = ~

G ( ~

V (t));

V

~

0 (t + 1) = ~

G ( ~

V

0 (t)) + ~

S (t)

Switch the stimulus on at time t

0 ; ~

V (t

0 ) = ~

V

0 (t

0 ).

~

_{δ(t) = ~}

_V

0 _{(t) − ~}

_{V (t) ⇒ ~}

_δ(t

0 + 1) = ~

V

0 (t

0 + 1) − ~

V (t

0 + 1) = ~

S (t

0 )

~

_δ(t

₀

_{+ 2) = ~}

_{G ( ~}

_V

0 _(t

0 + 1)) + ~

S (t

0 + 1) − ~

G ( ~

V (t

0 + 1))

~

_δ(t

0 + 2) = ~

G ( ~

V (t

0 + 1) + ~

S (t

0 )) + ~

S (t

0 + 1) − ~

G ( ~

V (t

0 + 1))

~

δ(t

0 + 2) =

h

DG

_{V (t}

~

0

+1)

.~

S (t

0 ) + ~

S (t

0 + 1)

i

+

2 ~

η(t

0 + 1)

~

_{δ(t) =}

t−1

X

τ =t

0

DG

t−τ +1

_~

V (t

0

+1)

.~

S (τ ) +

2 _R(t)

_~

(47)

Time dependent perturbation

~

δ(t) =

t−1

X

τ =t0

DF~

_V

t−τ +1

.~

S (τ )

|

{z

}

+

2

_R(t)

_~

(48)

Time dependent perturbation

~

δ(t) =

t−1

X

τ =t0

DF~

_V

t−τ +1

.~

S (τ )

|

{z

}

Controlled by the spectrum ofDF~V ∗

+

2

_R(t)

_~

Linear stability analysis

~

G ( ~

V ) = J f (g ~

V )

DF

_V

~

= g J Λ( ~

V )

(49)

Time dependent perturbation

~

δ(t) =

t−1

X

τ =t0

DF~

_V

t−τ +1

.~

S (τ )

|

{z

}

+

2

_R(t)

_~

Linear stability analysis

~

G ( ~

V ) = J f (g ~

V )

DF

_V

~

= g J Λ( ~

V )

Λ

ij

= f

0 (g u

j

) δ

ij

(50)

Time dependent perturbation

~

δ(t) =

t−1

X

τ =t0

DF~

_V

t−τ +1

.~

S (τ )

|

{z

}

+

2

_R(t)

_~

Linear stability analysis

~

G ( ~

V ) = J f (g ~

V )

DF

_V

~

= g J Λ( ~

V )

(51)

Time dependent perturbation

~

_{δ(t) =}

t−1

X

τ =t0

DF

_{V (t+1)}~t−τ +1

.~

S (τ ) +

2

_~

R(t)

~

G ( ~

V ) = J f (g ~

V ) + θ

DF

_V

~

= g J Λ( ~

V )

Λ

ij

= f

0 (g u

j

) δ

ij

(52)

Time dependent perturbation

~

_{δ(t) =}

t−1

X

τ =t0

DF

_{V (t+1)}~t−τ +1

.~

S (τ ) +

2

_~

R(t)

~

G ( ~

V ) = J f (g ~

V ) + θ

DF

_V

~

= g J Λ( ~

V )

Λ

ij

= f

0 (g u

j

) δ

ij

(53)

Time dependent perturbation

~

δ(t) =

t−1

X

τ =t0

DF

_~t−τ +1 V (t+1)

.~

S (τ )

|

{z

}

+

2

_R(t)

_~

(54)

Time dependent perturbation

~

δ(t) =

t−1

X

τ =t0

DF

_~t−τ +1 V (t+1)

.~

S (τ )

|

{z

}

Expansive Positive Lyapunov exponent

(55)

Time dependent perturbation

~

δ(t) =

t−1

X

τ =t0

DF

_~t−τ +1 V (t+1)

.~

S (τ )

|

{z

}

+

2

_R(t)

_~

(56)

Time dependent perturbation

~

δ(t) =

t−1

X

τ =t0

DF

_{V (t+1)}~t−τ +1

.~

S (τ )

|

{z

}

+

2

R(t)

~

Linear response vs chaotic

Butterfly effect

Van Kampen objection

The linear expansion provided by

the positive Lyapunov exponent

prevents linear response theory.

(57)

Time dependent perturbation

Linear response vs chaotic

(58)

The Sinai-Ruelle-Bowen measure

Time averaging is robust to perturbation.

lim

T →∞

1 T

T

X

t=1

Φ( ~

G

t

( ~

V )) = lim

T →∞

1 T

T

X

t=1

Φ( ~

G

t

( ~

V + ~

δ))

µ

L

Lebesgue measure on the phase-space.

µ

= lim

w

t→+∞

~

G

∗t

µ

L

,

SRB measure

lim

T →∞

1 T

T

X

t=1

Φ

h ~

G

t

( ~

V )

i

µ

L

=

a.s.

Z

Ω

Φ( ~

V )µ(d X)

(59)

The Sinai-Ruelle-Bowen measure

Time averaging is robust to perturbation.

lim T →∞ 1 T T X t=1 Φ( ~Gt( ~V )) = lim T →∞ 1 T T X t=1 Φ( ~Gt( ~V + ~δ))

µLLebesgue measure on the phase-space.

µ=w lim t→+∞ ~ G∗tµL, SRB measure lim T →∞ 1 T T X t=1 Φh ~Gt( ~V )iµL a.s.= Z Ω Φ( ~V )µ(d X)

Natural notion of averaging ”on” the attractor.

(60)

The Sinai-Ruelle-Bowen measure

Time averaging is robust to perturbation.

lim T →∞ 1 T T X t=1 Φ( ~Gt( ~V )) = lim T →∞ 1 T T X t=1 Φ( ~Gt( ~V + ~δ))

µLLebesgue measure on the phase-space.

µ=w lim t→+∞ ~ G∗tµL, SRB measure lim T →∞ 1 T T X t=1 Φh ~Gt( ~V ) i_{µL a.s.} = Z Ω Φ( ~V )µ(d X)

(61)

Out of equilibrium SRB state

D. Ruelle, J. Stat. Phys., 1998

µ

t

= µ + δ

t

µ =

lim

n→+∞

~

G

_t

0 . . . ~

G

_t−n

0 µ

δ

t

µ [Φ] =

t−1

X

τ =−∞

Z

µ(d ~

V )

D ~

G

t−τ −1

_~

V

~

S

τ

h ~

G

−1

( ~

V )

i

.∇

_{V (t−τ −1)}

~

Φ + NL

δ

t

µ[Φ] =

X

σ

D

κ

σ

S

~

t−σ−1

◦ ~

G

−1

|Φ

E

eq

(62)

Linear response in the firing rate neural network

B. Cessac, J.A. Sepulchre, PRE (2004); Chaos (2006); Physica D (2006)

Convolution

δ

t

ρ[u

i

] = [ χ ∗ S ]

i

(t)

=

N

X

j =1

t

X

σ=−∞

χ

i ,j

(σ)S

j

(t − σ − 1)

(63)

Linear response in the firing rate neural network

B. Cessac, J.A. Sepulchre, PRE (2004); Chaos (2006); Physica D (2006)

Convolution

δ

t

ρ[u

i

] = [ χ ∗ S ]

i

(t)

=

N

X

j =1

t

X

σ=−∞

χ

i ,j

(σ)S

j

(t − σ − 1)

χ

i ,j

(σ) =

P

γ

ij

(σ)

Q

σ

l =1

J

k

l

k

l −1

Q

σ

l =1

f

0 (u

k

l −1

(l − 1))

eq

(64)

Linear response in the firing rate neural network

B. Cessac, J.A. Sepulchre, PRE (2004); Chaos (2006); Physica D (2006)

Convolution

δ

t

ρ[u

i

] = [ χ ∗ S ]

i

(t)

=

N

X

j =1

t

X

σ=−∞

χ

i ,j

(σ)S

j

(t − σ − 1)

Linear response

χ

i ,j

(σ) =

P

γ

ij

(σ)

Q

σ

l =1

J

k

l

k

l −1

Q

σ

l =1

f

0 (u

k

l −1

(l − 1))

eq

(65)

Linear response in the firing rate neural network

B. Cessac, J.A. Sepulchre, PRE (2004); Chaos (2006); Physica D (2006)

Convolution

δ

t

ρ[u

i

] = [ χ ∗ S ]

i

(t)

=

N

X

j =1

t

X

σ=−∞

χ

i ,j

(σ)S

j

(t − σ − 1)

Linear response

χ

i ,j

(σ) =

P

γ

ij

(σ)

Q

σ

l =1

J

k

l

k

l −1

Q

σ

l =1

f

0 (u

k

l −1

(l − 1))

eq

(66)

Linear response in the firing rate neural network

B. Cessac, J.A. Sepulchre, PRE (2004); Chaos (2006); Physica D (2006)

Convolution

δ

t

ρ[u

i

] = [ χ ∗ S ]

i

(t)

=

N

X

j =1

t

X

σ=−∞

χ

i ,j

(σ)S

j

(t − σ − 1)

Linear response

χ

i ,j

(σ) =

P

γ

ij

(σ)

Q

σ

l =1

J

k

l

k

l −1

Q

σ

l =1

f

0 (u

k

l −1

(l − 1))

eq

(67)

Linear response in the firing rate neural network

B. Cessac, J.A. Sepulchre, PRE (2004); Chaos (2006); Physica D (2006)

Convolution

δ

t

ρ[u

i

] = [ χ ∗ S ]

i

(t)

=

N

X

j =1

t

X

σ=−∞

χ

i ,j

(σ)S

j

(t − σ − 1)

Linear response

χ

i ,j

(σ) =

P

γ

ij

(σ)

Q

σ

l =1

J

k

l

k

l −1

Q

σ

l =1

f

0 (u

k

l −1

(l − 1))

eq

(68)

Resonances

Power spectrum

Ruelle resonances

Ruelle-Pollicott resonances: In the

power spectrum. Absolutely

continuous part of the SRB

measure.

power spectrum. Singular part of

the SRB measure.

Predicted by D. Ruelle (J. Stat.

Phys, 1999)

Exhibited in B. Cessac, J.A.

Sepulchre, PRE, 2004.

(69)

Resonances

Complex susceptibility

Ruelle resonances

Ruelle-Pollicott resonances: In the

power spectrum. Absolutely

continuous part of the SRB

measure.

Exotic resonances. Not in the

power spectrum. Singular part of

the SRB measure.

Predicted by D. Ruelle (J. Stat.

Phys, 1999)

Exhibited in B. Cessac, J.A.

Sepulchre, PRE, 2004.

(70)

Response to a time-dependent stimulus

Connectivity matrix

(71)

Response to a time-dependent stimulus

(72)

(73)

Response to a time-dependent stimulus

(74)

Main conclusions

Linear response is possible in a chaotic neural network.

Convolution kernel depending on synaptic graph and dynamics

built on equilibrium (SRB) correlations.

The response graph is different from the synaptic weights

graph and depends on the stimulus.

(75)

From spiking neurons dynamics to linear response.

(76)

unbounded memory

B. Cessac, R.Cofr´e, J. Math. Neuro. submitted, 2018

(77)

An Integrate and Fire neural network model with chemical

and electric synapses

R.Cofr´e, B. Cessac, Chaos, Solitons and Fractals, 2013

Spikes

Voltage dynamics is

time-continuous.

events

Spike state ω

k

(n) ∈ 0, 1.

Spike pattern ω(n).

Spike block ω

n

m

.

(78)

An Integrate and Fire neural network model with chemical

and electric synapses

Spikes

Voltage dynamics is

time-continuous.

Spikes are time-discrete

events

(time resolution δ > 0).

t

_k

(l )

∈ [nδ, (n + 1)δ[

⇒

ω

k

(n) = 1

Spike pattern ω(n).

Spike block ω

n

m

.

(79)

An Integrate and Fire neural network model with chemical

and electric synapses

Spikes

Voltage dynamics is

time-continuous.

Spikes are time-discrete

events

(time resolution δ > 0).

Spike state ω

k

(n) ∈ 0, 1.

(80)

An Integrate and Fire neural network model with chemical

and electric synapses

Spikes

Voltage dynamics is

time-continuous.

Spikes are time-discrete

events

(time resolution δ > 0).

Spike state ω

k

(n) ∈ 0, 1.

(81)

and electric synapses

Spikes

Voltage dynamics is

time-continuous.

Spikes are time-discrete

events

(time resolution δ > 0).

Spike state ω

k

(n) ∈ 0, 1.

Spike pattern ω(n).

Spike block ω

n

(82)

A conductance-based Integrate and Fire model

M. Rudolph, A. Destexhe, Neural Comput. 2006, (GIF model)

Sub-threshold dynamics:

C

k

dV

k

dt

= −g

L,k

(V

k

− E

L

)

−

X

j

g

kj

(t, ω)(V

k

− E

j

)

Synapses

α

kj

(t) =

t

τ

e

−

t τkj

_H(t),

(83)

A conductance-based Integrate and Fire model

M. Rudolph, A. Destexhe, Neural Comput. 2006, (GIF model)

Sub-threshold dynamics:

C

k

dV

k

dt

= −g

L,k

(V

k

− E

L

)

−

X

j

g

kj

(t, ω)(V

k

− E

j

)

Synapses

0 0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09 0.1 0.5 1 1.5 2 2.5 3 3.5 4 PSP t t=1 t=1.2 t=1.6 t=3 g(x)

g

kj

(t) = g

kj

(t

j

) + G

kj

α

kj

(t − t

j

)

t > t

j

(84)

A conductance-based Integrate and Fire model

M. Rudolph, A. Destexhe, Neural Comput. 2006, (GIF model) R.Cofr´e, B. Cessac, Chaos, Solitons and Fractals, 2013

Sub-threshold dynamics:

C

k

dV

k

dt

= −g

L,k

(V

k

− E

L

)

−

X

j

g

kj

(t, ω)(V

k

− E

j

)

Synapses

0 0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09 0.1 0.5 1 1.5 2 2.5 3 3.5 4 PSP t g(x)

g

kj

(t) = G

kj

X

n≥0

α

kj

(t − t

_j

(n)

)

(85)

A conductance-based Integrate and Fire model

Sub-threshold dynamics:

C

k

dV

k

dt

= −g

L,k

(V

k

− E

L

)

−

X

j

g

kj

(t, ω)(V

k

− E

j

)

Synapses

0 0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09 0.1 0.5 1 1.5 2 2.5 3 3.5 4 PSP t g(x)

g

kj

(t,

ω

) = G

kj

X

n≥0

α

kj

(t−

nδ

)

ω

j

(n)

(86)

Sub-threshold dynamics:

C

k

dV

k

dt

= −g

L,k

(V

k

− E

L

)

−

X

j

g

kj

(t, ω)(V

k

− E

j

)

+S

k

(t) + σ

B

ξ

k

(t)

Synapses

0 0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09 0.1 0.5 1 1.5 2 2.5 3 3.5 4 PSP t g(x)

g

kj

(t,

ω

) = G

kj

X

n≥0

α

kj

(t−

nδ

)

ω

j

(n)

(87)

Sub-threshold dynamics:

C

k

dV

k

dt

+ g

k

( t, ω ) V

k

= i

k

(t, ω).

W

kj

def

= G

kj

E

j

α

kj

(t, ω) =

X

n≥0

α

kj

(t−nδ)ω

j

(n)

i

k

(t, ω) = g

L,k

E

L

+

X

j

W

kj

α

kj

(t, ω)+S

k

(t)+σ

B

ξ

k

(t)

(88)

Sub-threshold dynamics:

C

k

dV

k

dt

+ g

k

( t, ω ) V

k

= i

k

(t, ω).

Linear in V .

Spike history-dependent.

Dynamics integration

Γ

k

(t

1 , t, ω) = e

−

1 Ck

R

t t1∨τk (t,ω)

g

k

( u,ω ) du

(89)

Sub-threshold dynamics:

C

k

dV

k

dt

+ g

k

( t, ω ) V

k

= i

k

(t, ω).

Linear in V .

Spike history-dependent.

Dynamics integration

Γ

k

(t

1 , t, ω) = e

−

1 Ck

R

t t1∨τk (t,ω)

g

k

( u,ω ) du

(90)

Sub-threshold dynamics:

C

k

dV

k

dt

+ g

k

( t, ω ) V

k

= i

k

(t, ω).

Linear in V .

Spike history-dependent.

Dynamics integration

Γ

k

(t

1 , t, ω) = e

−

1 Ck

R

t t1∨τk (t,ω)

g

k

( u,ω ) du

Vk(t, ω) = Vk(sp)(t, ω) + V (S) k (t, ω) + V (noise) k (t, ω)

(91)

Sub-threshold dynamics:

C

k

dV

k

dt

+ g

k

( t, ω ) V

k

= i

k

(t, ω).

Linear in V .

Spike history-dependent.

Dynamics integration

Γ

k

(t

1 , t, ω) = e

−

1 Ck

R

t t1∨τk (t,ω)

g

k

( u,ω ) du

Vk(t, ω) = Vk(sp)(t, ω) + V (S) k (t, ω) | {z } V_k(det)(t,ω) +V_k(noise)(t, ω)

(92)

Sub-threshold dynamics:

C

k

dV

k

dt

+ g

k

( t, ω ) V

k

= i

k

(t, ω).

Linear in V .

Spike history-dependent.

Dynamics integration

Γ

k

(t

1 , t, ω) = e

−

1 Ck

R

t t1∨τk (t,ω)

g

k

( u,ω ) du

Vk(t, ω) = Vk(sp)(t, ω) + V (S) k (t, ω) | {z } V_k(det)(t,ω) +V_k(noise)(t, ω) V_k(sp)(t, ω) = V_k(syn)(t, ω) + V_k(L)(t, ω)

,

(93)

Sub-threshold dynamics:

C

k

dV

k

dt

+ g

k

( t, ω ) V

k

= i

k

(t, ω).

Linear in V .

Spike history-dependent.

Dynamics integration

Γ

k

(t

1 , t, ω) = e

−

1 Ck

R

t t1∨τk (t,ω)

g

k

( u,ω ) du

,

V

_k

(syn)

(t, ω) =

1 C

k

N

X

j =1

W

kj

Z

t

τ

k

(t,ω)

Γ

k

(t

1 , t, ω)α

kj

(t

1 , ω)dt

1

(94)

Sub-threshold dynamics:

C

k

dV

k

dt

+ g

k

( t, ω ) V

k

= i

k

(t, ω).

Linear in V .

Spike history-dependent.

Dynamics integration

Γ

k

(t

1 , t, ω) = e

−

1 Ck

R

t t1∨τk (t,ω)

g

k

( u,ω ) du

,

V

_k

(syn)

(t, ω) =

1 C

k

N

X

j =1

W

kj

Z

t

τ

k

(t,ω)

Γ

k

(t

1 , t, ω)α

kj

(t

1 , ω)dt

1

(95)

Sub-threshold dynamics:

C

k

dV

k

dt

+ g

k

( t, ω ) V

k

= i

k

(t, ω).

Linear in V .

Spike history-dependent.

Dynamics integration

Γ

k

(t

1 , t, ω) = e

−

1 Ck

R

t t1∨τk (t,ω)

g

k

( u,ω ) du

,

V

_k

(syn)

(t, ω) =

1 C

k

N

X

j =1

W

kj

Z

t

τ

k

(t, ω)

Γ

k

(t

1 , t, ω)α

kj

(t

1 , ω)dt

1

(96)

Sub-threshold dynamics:

C

k

dV

k

dt

+ g

k

( t, ω ) V

k

= i

k

(t, ω).

Linear in V .

Spike history-dependent.

Dynamics integration

Γ

k

(t

1 , t, ω) = e

−

1 Ck

R

t t1∨τk (t,ω)

g

k

( u,ω ) du

,

V

_k

(syn)

(t, ω) =

1 C

k

N

X

j =1

W

kj

Z

t

τ

k

(t,ω)

Γ

k

(t

1 , t, ω)α

kj

(t

1 , ω)

dt

1

(97)

Variable length Markov chain

P

n

ω(n)

ω

_−∞

n−1

≡ Π

ω(n),

V

th

− V

(det)

k

(n − 1, ω)

σ

k

(n − 1, ω)

!

(98)

Response to stimuli

How is the average of an observable f (ω, t) affected by the

stimulus ?

(99)

Response to stimuli

How is the average of an observable f (ω, t) affected by the

stimulus ?

If S is weak enough:

δµ [ f (t) ] = [ κ

f

∗ S ] ( t )

, (linear

response).

(100)

Response to stimuli

How is the average of an observable f (ω, t) affected by the

stimulus ?

If S is weak enough:

δµ [ f (t) ] = [ κ

f

∗ S ] ( t )

, (linear

response).

κ

k,f

( t − t

1 ) =

1 C

k

[ t−t

1

]

X

r =−∞

C

(sp)

"

f (t − t

1 , .),

H

(1)

_k

(r , .)

σ

k

(r − 1, .)

Γ

k

(0, r − 1, .)

#

(101)

Response to stimuli

How is the average of an observable f (ω, t) affected by the

stimulus ?

If S is weak enough:

δµ [ f (t) ] = [ κ

f

∗ S ] ( t )

, (linear

response).

κ

k,f

( t − t

1 ) =

1 C

k

[ t−t

1

]

X

r = −∞

C

sp

"

f (t − t

1 ,

.

),

H

_k

(1)

(r ,

.

)

σ

k

(r − 1,

.

)

Γ

k

(0, r − 1,

.

)

#

History dependence.

(102)

Response to stimuli

How is the average of an observable f (ω, t) affected by the

stimulus ?

If S is weak enough:

δµ [ f (t) ] = [ κ

f

∗ S ] ( t )

, (linear

response).

κ

k,f

( t − t

1 ) =

1 C

k

[ t−t

1

]

X

r =−∞

C

(sp)

"

f (t − t

1 , .)

,

H

(1)

_k

(r , .)

σ

k

(r − 1, .)

Γ

k

(0, r − 1, .)

#

(103)

Response to stimuli

How is the average of an observable f (ω, t) affected by the

stimulus ?

If S is weak enough:

δµ [ f (t) ] = [ κ

f

∗ S ] ( t )

, (linear

response).

κ

k,f

( t − t

1 ) =

1 C

k

[ t−t

1

]

X

r =−∞

C

(sp)

h

f (t − t

1 , .),

H

(1) k

(r ,.)

σ

k

(r −1,.)

Γ

k

(0, r − 1, .)

i

History dependence, observable, network dynamics

(104)

Response to stimuli

How is the average of an observable f (ω, t) affected by the

stimulus ?

If S is weak enough:

δµ [ f (t) ] = [ κ

f

∗ S ] ( t )

, (linear

response).

κ

k,f

( t − t

1 ) =

1 C

k

[ t−t

1

]

X

r =−∞

C

(sp)

"

f (t − t

1 , .),

H

_k

(1)(r ,.)

σ

k

(r − 1, .)

Γ

k

(0, r − 1, .)

#

,

History dependence, (spontaneous) correlation between observable

and network dynamics

(105)

Response to stimuli in a mean-field limit

Characteristic time

τ

d ,k

=

C

k

g

L

+

P

N

j =1

G

kj

ν

j

τ

kj

Approximations

(i)

Replace τ

k

(r − 1, .) by −∞;

(ii)

Replace Γ

k

(t

1 , r − 1, ω) = e

−

1 Ck

R

r −1 t1

g

k

( u,ω ) du

by e

−

(r −1−t1) τd,k

_.

(106)

Response to stimuli in a mean-field limit

δ

(1)

_{µ [ f (t) ] =}

−

2 σ

B

P

N

k=1

1 √

_τ

d ,k

P

n=[ t ]

r =−∞









( S

k

∗ e

d ,k

) (r − 1)

e

d ,k

(u) = e

−

u τd,k

(107)

Response to stimuli in a mean-field limit

δ

(1)

_{µ [ f (t) ] =}

−

2 σ

B

P

N

k=1

1 √

_τ

d ,k

P

n=[ t ]

r =−∞









( S

k

∗ e

d ,k

) (r − 1)

Markov approximation with memory depth 1







γ

_k

(1)

C

(sp)

_{[ f (t, ·), ω}

k

(r ) ]

+

P

N

i =1

γ

(2)

k;i

C

(sp)

_{[ f (t, ·), ω}

k

(r ) ω

i

(r − 1) ]

+

P

N

i ,j =1

γ

(3)

k;ij

C

(sp)

_{[ f (t, ·), ω}

k

(r ) ω

i

(r − 1) ω

j

(r − 1) ]

+

· · ·







(108)

Response to stimuli in a mean-field limit

Markov approximation with memory depth 1







γ

_k

(1)

C

(sp)

_{[ f (t, ·), ω}

k

(r ) ]

+

P

N

i =1

γ

(2)

k;i

C

(sp)

_{[ f (t, ·), ω}

k

(r ) ω

i

(r − 1) ]

+

P

N

i ,j =1

γ

(3)

k;ij

C

(sp)

_{[ f (t, ·), ω}

k

(r ) ω

i

(r − 1) ω

j

(r − 1) ]

+

· · ·







(109)

Response to stimuli in a mean-field limit

Markov approximation with memory depth 1







γ

_k

(1)

C

(sp)

_{[ f (t, ·), ω}

k

(r ) ]

+

P

N

i =1

γ

(2)

k;i

C

(sp)

_{[ f (t, ·), ω}

k

(r ) ω

i

(r − 1) ]

+

P

N

i ,j =1

γ

(3)

k;ij

C

(sp)

_{[ f (t, ·), ω}

k

(r ) ω

i

(r − 1) ω

j

(r − 1) ]

+

· · ·







(110)

Response to stimuli in a mean-field limit

Ex: Firing rate of neuron m

δ(1) µ [ ωm (t) ] = − 2 σB PN k=1 p1 τd,k Pn=[ t ] r =−∞        γ_k(1)C(sp) ωm(t), ωk (r) +PN i =1 γ (2) k;iC(sp) ωm(t), ωk (r) ωi (r − 1) +PN i ,j =1 γ (3) k;ijC(sp) h ωm (t), ωk (r) ωi (r − 1) ωj (r − 1) i + · · ·        Sk ∗ ed,k (r − 1)