Spiking Neural Network Decoder for Brain‐Machine Interfaces

(1)

Spiking Neural Network Decoder for Brain-‐

Machine Interfaces

J

ULIE

D

ETHIER

UGR MEETING IN NANCY NOVEMBER 28TH 2011

University of Liège Montefiore Institute

Systems and Modeling Research Unit

(2)

Spiking neural network decoder for brain-‐machine interfaces Julie Dethier

BMIs:

→

Signal acquisiOon

and condiOoning

→

Spike sorOng

→

Neural decoding

→

AcOon:

→

ProstheOc limbs

→

Computer cursors

(3)

Key challenge:

→

Temperature

→

Power dissipaOon

Constraints

Temperature

1°C

Power -‐ 6x6mm

2

_implant

_10mW

2D Kalman-‐ﬁlter decoder

3.6GHz Core Duo Intel processor

1.82mW

(4)

Agenda

(5)

1. The proposed alterna9ve

2. Kalman ﬁlter decoder for BMI

3. Spiking neural network decoder with NEF

4. Implementa9ons

5. Conclusions and future work

(6)

The proposed

alternaOve

(7)

SoluOon:

→

Ultra-‐low-‐power neuromorphic

approach

→

Morphes neural systems into

silicon chips

→

Transistors analog to ion

channels

(8)

65 536 neurons

23M transistors

160 mm

2

0.18 um CMOS

Neurogrid:

→

1M neurons

→

6B synapses

(9)

Kalman ﬁlter

decoder for BMI

(10)

EsOmate of the current system state using:

→

Model dynamics

→

Noisy measurements

State update:

Steady-‐state assumpOon:

Neural applicaOons:

→

System state:

→

Measurements: neural spike rate of the recorded neurons

x

_t

_{= Ax}

_t−1

_{+ w}

_t

y

_t

_{= Cx}

_t

_{+ q}

_t

where

w

_t

_{ N (0,W)}

q

_t

_{ N (0,Q)}

where

x

_t

_{= vel}

!

_"

_tx

,vel

_ty

,1

#

_$

y

_t

x

_t

_{= (I − KC)Ax}

_t|t−1

_{+ Ky}

_t

_{= M}

_xDT

x

_t|t−1

_{+ M}

DT_y

y

_t

with

_{K = (I + WCQ}

−1

_C)

−1

_WC

T

_Q

−1

(11)

Fiang the model parameters:

Neuron

150 100 50 0 2000 4000 6000 8000 10000 12000 14000 20 0 20

Time (ms)

Velocity (cm/s)

0 5 10 x velocity y velocity

(a)

(b)

(12)

Spiking neural

network decoder

with NEF

(13)

Neural Engineering Framework:

→

A formal methodology for mapping control-‐theory

algorithms onto spiking neural networks

(14)

j

th

_{neuron’s
ﬁring
rate:}

a

_j

₍

x(t)

₎

= G

!

_"

α

_j

φ



x_j

· x(t) + J

bias_j

#

_$

= G

#

_$

α

_j

φ



x_j

· h(t) ∗

(

A x(t) + "

"

B y(t)

)

+ J

bias_j

%

_&

= G

_%&

$

α

_j

φ



x_j

· h(t) ∗

(

A

"

∑

_i

a

_i

( )

t

φ

_ix

+ "

B

∑

_k

b

_k

( )

t

φ

_ky

)

+ J

bias_j

'

₍₎

a

_j

₍

x(t)

₎

_{= G h(t) ∗}

_ω

_ji

a

_i

_{( )}

t

i

∑

+

∑

_k

ω

_jk

b

_k

( )

t

(

)

+ J

bias_j

#

$

%

&

ω

_ji

=

α

_j

φ



x_j

!

A

φ

_ix

with

and

ω

_jk

=

α

_j

φ



x_j

!

B

φ

_ky

Kalman ﬁlter:

y(t)

B'

x(t)

A'

b

_k

(t)

a

_j

(t)

(15)

ImplementaOons

(16)

Experimental setup:

0

12 -12

-12

Y

p

os

it

io

n (

cm

)

X position (cm)

0

12

Trials: 0018 - 0033

c

Decoder

Neural

Spikes

Cursor

Velocity

a

Hand

(17)

(18)

Spiking neural network decoder for brain-‐machine interfaces Julie Dethier 0.2 0.1 0 0.1 0.2 0.3 0.3 0.2 0.1 0 0.1 0.2 x velocity (m/s) y velocity (m/s) 2,000 neuron network 0.2 0.1 0 0.1 0.2 0.3 20,000 neuron network

Standard Kalman Filter Spiking Neural Network

3% 6% (b) (a) 200 2,000 6,000 10,000 20,000 0 0.05 0.1 RMS error (m/s) 200 2,000 6,000 10,000 20,000 0 0.5 1 1.5 2 Neuron count Error x sqrt(NC) (m/s) c _d

Open-‐loop:

(19)

b

0 X position (cm)

-12

0 Y

p

os

it

io

n (

cm

)

12

Trials: 2056- 2071

c

0 X position (cm)

-12

0 Y

p

os

it

io

n (

cm

)

12

Trials: 1700 - 1715

BMI: standard Kalman

BMI: Siking neural network

(20)

(21)

0 400 800

0 5 10

Time after Target Onset (ms)

Distance to Target (cm)

Mean distance to target

a

0 500 1000 1500 0 10 20 30 40

Target Acquire Time (ms)

Percent of Trials (%)

Target acquisition time histogram

b

0 200 400 600 800 1000 0 10 20 30 40 50

Time after Target Onset (ms)

Cursor Velocity (cm/s)

Mean cursor velocity

Standard Kalman filter Hand

Spiking Neural Network

c

(22)

Conclusions and

future work

(23)

SNN decoder

→

Virtually idenOcal to standard Kalman ﬁlter

→

99% success rates

Next step

→

Mapping with Neurogrid

→

Columnar organizaOon

→

Topographically assigned preferred direcOons vectors

(24)

Reference:

→

Kalman-‐ﬁlter based decoder

1.  V.Gilja, P.Nuyujukian, C.A.Chestek, J.P.Cunningham, J.M.Fan, B.M.Yu, S.I.Ryu, and K.V.Shenoy, 2010

Neuroscience Mee>ng Planner, San Diego, CA: Society for Neuroscience, 2010.

2.  P.Nuyujukian, V.Gilja, C.A.Chestek, J.P.Cunningham, J.M.Fan, B.M.Yu, S.I.Ryu, and K.V.Shenoy, 2010

Neuroscience Mee>ng Planner, San Diego, CA: Society for Neuroscience, 2010.

3.  V.Gilja, C.A.Chestek, I.Diester, J.M.Henderson, K.Deisseroth, and K.V.Shenoy, IEEE Trans. Biomed. Eng., 2011, in press.

4.  S.-‐P.Kim, J.D.Simeral, L.R.Hochberg, J.P.Donoghue, and M.J.Black, J. Neural Eng., vol. 5, 2008, pp 455–476. 5.  S.Kim, P.Tathireddy, R.A.Normann, and F.Solzbacher, IEEE Trans. Neural Syst. Rehabil. Eng., vol. 15, 2007, pp

493–501.

→

Neurogrid

1.  K.Boahen, Scien>ﬁc American, vol. 292(5), 2005, pp 56–63.

2.  R.Silver, K.Boahen, S.Grillner, N.Kopell, and K.L.Olsen, J. Neurosci., vol. 27(44), 2007, pp 11807–11819. 3.  J.V.Arthur and K.Boahen, IEEE Trans. Circuits Syst. I, Reg. Papers, vol. 58(5), 2011, pp 1034-‐1043.

→

Neural Engineering Framework

1.  C.Eliasmith and C.H.Anderson, MIT Press, Cambridge, MA; 2003. 2.  C.Eliasmit, Neural Computa>on, vol. 17, 2005, pp 1276–1314.

→

Present work

1.  J.Dethier, V.Gilja, P.Nuyujukian, S.A.Elassaad, K.V.Shenoy, and K.Boahen IEEE EMBS Conf. on Neural

Engineering, Cancun, Mexico, 2011, pp 396–399.

2.  J.Dethier, P.Nuyujukian, C.Eliasmith, T.Stewart, S.A.Elassaad, K.V.Shenoy, and K.Boahen, NIPS, Granada, Spain, 2011, to come.

(25)

Thank you for your

amenOon

Spiking Neural Network Decoder for Brain‐Machine Interfaces