Supplementary Material: Theory of spike timing based neural classiﬁers

(1)

Supplementary Material: Theory of spike timing based neural classifiers

Ran Rubin

Racah Institute of Physics, Hebrew University, 91904 Jerusalem, Israel and Laboratoire de Physique Théorique de l’ENS, CNRS,

Univ. Paris 6, 24 rue Lhomond, 75005 Paris, France

Rémi Monasson

Laboratoire de Physique Théorique de l’ENS, CNRS, Univ. Paris 6, 24 rue Lhomond, 75005 Paris, France and Simons Center for Systems Biology, Institute for Advanced Study,

Einstein Drive, Princeton, NJ 08540, USA

Haim Sompolinsky

Racah Institute of Physics, Hebrew University, 91904 Jerusalem, Israel Interdisciplinary Center for Neural Computation,

Hebrew University, 91904 Jerusalem, Israel and Center for Brain Science, Harvard University,

Cambridge, Massachusetts 02138, USA

(2)

EXTREME VALUE THEORY

Here we cite (without proof) the necessary theorems required to derive the distributions of Umax and Nspikes.

We consider a stationary continuous Gaussian process U(t) with zero mean and an auto- correlation function C(t) =hU(t^′)U(t^′+t)i that satisfies

C(t)≈1− 1 2

d²C(0) dt²

t² (1)

as t→0.

The average number of times U(t)crosses a given threshold level,Uth, from below in a time interval of length T is given by [1]:

ν(Uth, T) =Kexp −¹₂U_th²

2π (2)

where we define the effective duration of the time interval, K, as K =T

s

d²C(0) dt²

(3) We now consider the K → ∞ limit while keeping ν constant, that is, taking the threshold level to be

Uth(K, ν) = s

2 ln K

2πν

(4) We are interested in the probability distribution of the number of crossing of the threshold level and the distribution of the maximal value of U(t).

Given that the following condition holds

Klim→∞C(T) lnK = 0 (5)

it can be shown [2, 3] that the probability distribution of the number of crossings of the threshold level from bellow, Ns, is given by

Klim→∞P(Ns(K, ν) =m) = e⁻^νν^m

m! (6)

which is a Poisson distribution with mean ν.

(3)

To evaluate the distribution of the maximal value of U(t)we note that P

0<t<Tmax U(t)< Uth(K, ν)

=P (Ns(K, ν) = 0) =e⁻^ν (7) We now consider ν =e⁻^x and obtain

Klim→∞P

0<t<Tmax U(t)< Uth K, e⁻^x

=e⁻^e⁻^x (8)

where the right-hand side is the cumulative probability function of the Gumbel distribution.

From Eq. (4) we have

Uth K, e⁻^x2

= 2 (lnK +x−ln 2π) (9)

which in the large K limit entails Uth K, e⁻^x

≈√

2 lnK+ x−ln 2π

√2 lnK (10)

Finally, the distribution of the maximum of U(t) is given by

Klim→∞P

0<t<Tmax

√2 lnK

U(t)−√

2 lnK + ln 2π

√2 lnK

< x

=e⁻^e⁻^x (11) and therefore the random variable

X= max

0<t<T

√2 lnK

U(t)−√

2 lnK+ ln 2π

√2 lnK

(12) is asymptotically distributed according to a Gumbel distribution. Equations (10) and (11) lead to the result presented in Eq. (4) in the main text.

For further details and proofs regarding the asymptotic properties of Gaussian processes we refer the reader to [4].

SIMULATION OF A HODGKIN-HUXLEY NEURON

In Fig. 4b in the main text we present measurements of the probability of two Hodgkin- Huxley neurons to agree on the classification of a random pattern as a function of the overlap between their weight vectors. Here we give the details of the model neurons used in the simulations. We use the model neuron described in [5]. This model consists of a Hodgkin-Huxley neuron with an additional slow A-current. The membrane potential follows

Cm

dV

dt =−gL(V −EL)−g¯Nam³∞h(V −ENa)−g¯Kn⁴(V −EK)−¯gAa³∞b(V −EK) +I^syn (13)

(4)

with m^∞ = _α_m^α_+β^m_m, αm = _exp(⁻₋^0.1(V_0.1(V⁺³⁰⁾₊₃₀₎₎₋₁, βm = 4 exp −^V⁺⁵⁵₁₈

and a^∞ = ¹

exp(⁻^V⁺⁵⁰20 )⁺¹. The additional dynamical variables areh, n and b which follow the equations:

dh

dt = h^∞−h

τ_h (14)

dn

dt = n^∞−n τn

(15) db

dt =τ_A⁻¹ 1 exp ^V⁺⁸⁰₆

+ 1 −b

!

(16)

with h^∞ = _α_h^α_+β^h_h, τh = _α_h^φ_+β_h, αh = 0.07 exp ^V⁺⁴⁴₂₀

, βh = _1+exp(−0.1(V¹ +14)), n^∞ =

αⁿ

αn+βn, τn = _α_n_+β^φ _n, αn = _exp(⁻₋^0.01(V_0.1(V₊₃₄₎₎⁺³⁴⁾₋₁, βn = 0.125 exp −^V⁺⁴⁴80

.~ The additional parameters are: Cm = 1 µF/cm², φ = 0.1, g¯Na = 100 mS/cm², g¯K = 40 mS/cm², gL = 0.05 mS/cm², g¯A = 20 mS/cm², τA = 20 ms, ENa = 55 mV, EK = −80 mV, EL=−65mV .

The synaptic input to the neuron,I^syn, is modeled withN conductance based synapses with random synaptic efficacies, ωi, drawn from a Gaussian distribution with mean zero. The sign of the synaptic efficacy indicates whether the synapse is excitetory or inhibitory and the total synaptic input is given by

I^app =G^e(Ee−V) +Gⁱⁿ(Ein−V) (17) with Ee = 0 mV, Ein =−80 mV and

G^e=

N

X

i=1

|ωi|Θ (ωi)X

ti<t

e⁻

t−ti

τsynapse (18)

Gⁱⁿ =

N

X

i=1

|ωi|Θ (−ωi)X

tⁱ<t

e⁻

t−ti

τsynapse (19)

whereτsynapse = 5ms andΘ (x)is the Heaviside step function. Figure 1 presents an example of a voltage trace of the neuron driven with random synaptic inputs and the excitetory and inhibitory post synaptic potential induced by one synaptic input.

We simulated two neurons with correlated weights and measured Pequal as a function of the overlap between their weight vectors. We varied the total duration of the input patterns, T, and adjusted the standard deviation of the synaptic weights to ensure that a random

(5)

0 500 1000 1500

−80

−60

−40

−20 0 20 40 60

Time [ms]

V [mV]

⁰ ¹⁰ ²⁰

−50 0 50

Time [ms]

V [mV]

0 50 100 150

−0.5 0 0.5 1 1.5

Time [ms]

∆ V [mV]

EPSP IPSP (a) (b)

Figure 1. (a) An example of a voltage trace of the neuron driven with randomly chosen spikes of N = 500 neurons each firing with a Poisson rate of 1 Hz. The inset shows the shape of one spike generated by the neuron. (b) The excitatory post synaptic potential (EPSP) and the inhibitory post synaptic potential (IPSP) induced by a single input spike with |ω_i| = 1 measured from the membrane resting potential. The half width of the PSP is τ_1/2 ∼20 ms.

pattern is classified by each neuron ±1with equal probabilities. To produce the two curves shown in Fig. 4b of the main text we first fitted T to find the best agreement between the measurement ofPequalfor the Tempotron withK₁ = 100to the measurement ofPequalfor the Hodgkin-Huxley neuron and then multiplied thisT by 4 to produce the second curve. Notice that this curve fits very well the curve of Pequal for the Tempotron with K2 = 4K1 = 400.

[1] Y. Burak, S. Lewallen, and H. Sompolinsky, Neural Computation 21, 2269 (2009).

[2] J. Pickands, Transactions of the American Mathematical Society 145, 51 (1969).

[3] J. Pickands, Trans. Am. Math. Soc.145, 75 (1969).

[4] M. Leadbetter, G. Lindgren, and H. Rootzén, Extremes and related properties of random se- quences and processes (Springer, NY, USA, 1983).

[5] O. Shriki, D. Hansel, and H. Sompolinsky, Neural computation 15, 1809 (2003).