Sounds on time: auditory feedback in motor learning, re-learning and over-learning of timing regularity.

HAL Id: tel-00915893

https://tel.archives-ouvertes.fr/tel-00915893

Submitted on 9 Dec 2013

HAL is a multi-disciplinary open access archive for the deposit and dissemination of sci-entific research documents, whether they are pub-lished or not. The documents may come from teaching and research institutions in France or

L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires

re-learning and over-learning of timing regularity.

Floris van Vugt

To cite this version:

Floris van Vugt. Sounds on time: auditory feedback in motor learning, re-learning and over-learning of timing regularity.. Neuroscience. Université Claude Bernard - Lyon I, 2013. English. �tel-00915893�

N°d’ordre232-2013

THESE DE L‘UNIVERSITE DE LYON Délivréepar

L’UNIVERSITE CLAUDE BERNARD LYON 1

ECOLE DOCTORALE NEUROSCIENCES ET COGNITION (NSCo)

DIPLOME DE DOCTORAT EN NEUROSCIENCES (arrêtédu 7août2006)

Soutenuepubliquementle27novembre2013 àLyon par FlorisTijmen VAN VUGT Directeursdehèse:Dr.BarbaraTILLMANN etProf.EckartALTENMÜLLER Composition du Jury Prof.EckartALTENMÜLLER (directeurdethèse) Dr.BarbaraTILLMANN (directeurdethèse) Prof.PeterKELLER (rapporteur) Prof.VirginiaPENHUNE (rapporteur) Dr.Fabien PERRIN (présidentdu jury)

Soundson time:auditory feedback in motorlearning,re-learning and over-learning oftiming regularity.

Auditory feedback isan auditory signalthatcontainsinformation aboutperformed movement.Musicperformanceisan excellentcandidateto study itsinluenceon motor actions,sincetheauditory resultistheexplicitgoalofthemovement.Indeed,auditory feedback can guideonlinemotoractions,butitsinluenceon motorlearning hasbeen investigated less.histhesisinvestigatestheinluenceofauditory feedback in motor

learning,focusing particularly on how welearn temporalcontrolovermovements.First,we investigatemotorlearning in non-musicians,inding thatthey beneitfrom temporal information supplied by theauditory signaland aresensitiveto distortionsofthistemporal information.Second,weturn to strokepatientsthatarere-learning motoractionsin a rehabilitation seting.Patientsimproved theirmovementcapacitiesbutdid notdepend on thetime-locking between movementsand theresulting auditory feedback.Surprisingly, they appearto beneitfrom distortionsin feedback.hird,weinvestigatemusicalexperts, who arguably haveestablished strong linksbetween movementand auditory feedback.We develop anovelanalysisframework thatallowsusto segmenttiming into systematicand non-systematicvariability.Ourinding isthattheseexpertshavebecomelargely

independentoftheauditory feedback.hemain claim defended in thisthesisisthat auditory feedback can and doesplay arolein motorlearning ofregularity,buttheway in which itisused variesqualitatively between diferentpopulations.heseindingsprovide new insightsinto auditory-motorintegration and arerelevantfordeveloping new perspectiveson theroleofmusicin training and rehabilitation setings.

Keywords:

SensorimotorIntegration,Auditory Feedback,MotorLearning,Motorregularity,Timing, Rehabilitation,Expertmusicians,SequenceProduction,SequenceLearning

Feedback auditifetrégularitémotrice:apprentissage,rehabilitation etexpertise

Lefeedback auditifsedéinitcommeun signalauditifquicontientdel'information surun mouvement.Ilaétémontréquelefeedback auditifpeutguiderlemouvementen temps réel,maisson inluencesurl'apprentissagemoteurestmoinsclair.Cetethèseapourbut d'examinerl'inluencedu feedback auditifsurl'apprentissagemoteur,en sefocalisantsurle contrôletemporeldesmouvements.Premièrement,nousétudionsl'apprentissagemoteur chezlesnon-musicienssainsetmontronsqu'ilsbénéicientdel'information temporelle contenuedanslefeedback auditifetqu'ilssontsensiblesaux distortionsdecete information temporelle.Deuxièmement,nousappliquonscesconnaissancesàla

rehabilitation depatientscérébro-lésés.Noustrouvonsquecespatientsaméliorentleurs capacitésdemouvementmaisnedépendentpasdelacorrespondancetemporelleentrele mouvementetleson.Paradoxalement,cespatientsontmêmebenéiciédesdistortions temporellesdanslefeedback.Troisièmement,nousétudionslesexpertsmusicaux,carils ontétablidesliensparticulièrementfortsentreleurmouvementetleson.Nous

développonsdenouveaux outilsd'analysequinouspermetentdeséparerlesdéviations temporellesen variation systématiqueetnon-systématique.Lerésultatprincipalestqueces expertssontdevenu largementindépendentsdu feedback auditif.Laproposition centralede cetethèseestquelefeedback auditifjoueun rôledansl'apprentissagemoteurdela

regularité,maislafaçon dontlecerveau l'utilisedépend delapopulation étudiée.Ces résultatsdonnentunenouvelleperspectivesurl'intégration audio-motriceetcontribuentau développementdenouvellesapprochespourl'apprentissagedelamusiqueetla

réhabilitation.

Motsclés:

Intégration sensorimotrice,feedback auditif,apprentissagemoteur,regularitémotrice, timing,réhabilitation,expertsmusicaux,production deséquences,apprentissagede séquences.

Institutes:

InstituteofMusicPhysiology and Musicians’Medicine, University ofMusic,Dramaand Media, Emmichplatz1,30175Hanover Germany Lyon NeuroscienceResearch Center CNRS-UMR 5292,INSERM U1028, University Lyon-1, 50av Tony Garnier,69007Lyon, France hisresearch wassupported by theEBRAMUS

hemusician may sing to you oftherhythm which isin allspace, buthecannotgiveyou theearwhich arreststherhythm northe voicethatechoesit.

Ta

bl

e

of

Cont

e

nt

s

I.Sensorimotorintegration:aconceptualmap....19 I.1.Introduction...20 1.1Aim...20 1.2Disclaimer:inding abalancebetween unity and diversity...21

I.2.Preliminaries...22 2.1hehuman motorsystem – abriefoverview...23

2.1.aMotorcontrolareas– abriefoutline...23 2.1.b ControlStrategies...26

I.3.Towardsadeinition ofsensorimotorintegration...28 3.1Sensorimotorintegration...28 3.2Aim...29

I.4.Motortheoriesofcognition...31

I.5.Motorinvolvementin perception....32 5.1Movementmaking perception possible...34

5.1.aMicrosaccades(vision)...35 5.1.b Motor-induced suppression...36

5.2Action perception...39

5.2.aMirrorneurons...39 5.2.b Mirrornetworks...43 5.2.cDirectmatching hypothesisand motorresonance...46 5.2.d Plasticity ofmirrornetworks:arbitrarinessoftheassociations...47

5.3Speech perception and comprehension...50

5.3.aPerception ofspeech sounds...50 5.3.b Syntacticprocessing...54

5.3.cSemanticprocessing...55 5.3.d Conversationalinteraction...56 5.3.eAnatomicalsubstrates...57 5.3.fNecessity and roleofmotorinvolvement...57

5.4Musicperception...58

5.4.aWhy music?...58 5.4.b Motorsystemssupporting perceptualmemory formation...59 5.4.cMusicperception in novices:afocuson rhythm perception...61 5.4.d Musicalexperts...62 5.4.eCreation ofauditory-motorassociations:musiclearning in novices...64

I.6.Sensory involvementin action...66 6.1Sensory feedback in action...70

6.1.aRepresenting action by itssensory consequences(action selection)...70 6.1.b Sensory guidanceofaction (action execution)...72 6.1.cMotorlearning through sensory feedback (action learning)...73

6.2Sensory stimulation in action...75

6.2.aSensorimotorcorticospinalfacilitation...75 6.2.b Sensation asareferenceformovement...77

6.3Speech production...79

6.3.aSensory co-activation...79 6.3.b Relianceon feedback...79

6.4Musicproduction...80

6.4.aSensory co-activation during silentperformance...81 6.4.b Relianceon feedback...82 6.4.cSoniication feedback in musicalnovices...82

I.7.Synthesis...84

I.8.Outlineofthethesis....85

II.Learning in musicalnovices...87 II.1.Introduction...88 1.1Motorregularity...88

1.2SerialReaction Time...89 1.3Aimsand Hypotheses...91

II.2.h resholdsofauditory-motorcoupling measured with asimpletask in musiciansand non-musicians:Wasthesound simultaneousto thekeystroke?.92

2.1Introduction...94 2.2Materialsand Methods...96 2.2.aParticipants...96 2.2.b Materials...99 2.2.cProcedure...100 2.3Results...103 2.3.aDelay Detection...103 2.3.b Anisochrony...104 2.3.cSynchronisation-Continuation Tapping...105 2.3.d Comparison between thetests...106

2.4Conclusion...108

II.3.Auditory feedback beneitsshortterm motorlearning....116 3.1Introduction...119 3.2Methods...125 3.2.aParticipants...125 3.2.b Materials...125 3.2.cProcedure...128 3.2.d DataAnalysis...134 3.3Results...137 3.3.aSequenceLearning...137 3.3.b Sequenceswitching...138 3.3.cMuting efects...140 3.3.d Scaletapping...141 3.3.eAnisochrony detection task...142 3.3.fDelay detection task...142 3.3.g Synchronisation-Continuation Tapping...142

3.4Discussion...144

3.4.b Sequence-speciicvs.unspeciiclearning...147 3.4.cAuditory feedback deprivation...148 3.4.d Scaleplaying transfertask...149 3.4.eDoesthesound need to betime-locked to themovement?...150 3.4.fAnisochrony,delay detection and synchronisation-continuation tapping tasks...151 3.4.g Limitationsand outlook...152

3.5References...152 3.6Acknowledgments...155

II.4.Discussion...162

III.Re-learning in strokepatients....163 III.1.Introduction....164 1.1Stroke:pathophysiology and rehabilitation...164

1.1.aCausesofmotorimpairments...164 1.1.b Mechanismsunderlying neuralrepairaterstroke...166

1.2Proprioception and motor(re)learning...167

1.2.aWhatisproprioception?...168 1.2.b Proprioception tests...168 1.2.cAlterationsin proprioception...170 1.2.d Proprioception and motorlearning...170 1.2.eProprioception aterstrokeand itsrolein rehabilitation...171

1.3Music-Supported herapy:working hypotheses...172

1.3.aMusicinterventionsin clinicalsetings...172 1.3.b Music-supported therapy...172 1.3.cProbing themechanismsofmusic-supported therapy...173

1.4Aimsand hypotheses...175

III.2.Random delay ismoreeicientthan immediateauditory feedback in ine motorrehabilitation aterstroke....178

2.1Introduction...181 2.2Methods...184

2.2.b herapy...187 2.2.cNine-holepegboard test...188 2.2.d Fingertapping measurements...188 2.2.ePatientmood measurements...190 2.2.fAuditory and sensorimotortasks...191 2.2.g Intra-therapy measurements...193 2.2.h u alitativeinterview...194 2.2.iStatisticalanalyses...194 2.2.jEthics...194 2.3Results...195 2.3.aNineholepegboard test...195 2.3.b Fingertapping measurements...195 2.3.cPatientmood measurements(proileofmood states)...201 2.3.d Facesscale...201 2.3.eAuditory and auditory-motortests...202 2.3.fWithin-therapy measure:keyboard keystrokes...203 2.3.g u alitativeinterviews...204 2.4Discussion...205 2.5References...212 2.6Conlictofintereststatement...215 2.7Acknowledgements...215

III.3.Music-supported motortraining aterstrokerevealsno superiority of synchronisation in group therapy...220

3.1Introduction...222 3.2Methods...223 3.2.aPatientgroup characteristics...224 3.2.b Musictraining...225 3.2.cNine-holepegboard test...226 3.2.d Fingertapping measurements...227 3.2.eMood test:ProileofMood States...229 3.2.fMood test:facesscale...229 3.2.g Ethics...229 3.2.h Dataanalysis...230

3.3Results...230 3.3.aNine-holepegboard test...230 3.3.b Fingertapping tests...231 3.3.cMood tests...235 3.3.d Facesscalemood ratings...236 3.4Conclusions...238 III.4.Discussion...244 4.1Auditory feedback...244 4.2A noteaboutmood improvements...245 4.3Implicationsforclinicalpractice...245

IV.Over-learning in MusicalExperts....247 IV.1.Introduction...248 1.1Choiceofexperimentalparadigm...248 1.2Piano timing measurements:methodologicalconsiderations...249 1.3Musicalperformanceand theperception-action interface...251 1.4Musicalscales...252 1.4.aDescription ofthescaleplaying movement...252 1.4.b Timing controlin scaleplaying...253 1.4.cTowardsanew modeloftiming in piano performance...256 1.5Aimsand hypotheses...258 IV.2.Fingersphrasemusicdiferently:trial-to-trialvariability in piano scale playing and auditory perception revealmotorchunking.....260

IV.3.Individuality thatisunheard of:systematictemporaldeviationsin scale playing leavean inaudiblepianisticingerprint....270

IV.4.Spatialand temporalsymmetriesofmotorprimitivesin skilled piano performanceatdiferenttempi....282

4.1Introduction...284 4.2Methods...287

4.2.aParticipants...287 4.2.b Procedure...287 4.2.cProcessing ofMIDIdata...288 4.2.d Scaleunevenness...288 4.2.eUnevennessacrosstempi...289 4.2.fIndividualnotetiming...289 4.2.g Towardsamodeloftiming deviations...289 4.2.h Fiting methods...290

4.3Results...291

4.3.aScale-leveltempo dependencies...291 4.3.b Note-by-noteirregularities:establishing thephrasaland neuromusculartemplates293 4.3.cTrade-of between thetwo templates...295 4.3.d Generalising to alloutward scales...297 4.3.eGeneralising to inward scales...297 4.3.fGeneralising to both handsand directions...299 4.3.g Individualleveland correlation with accumulated playing time...299

4.4Discussion...301 4.5References...307

IV.5.Discussion...318 5.1Outlineoftheirregularity-instability model...318 5.2Auditory feedback...320 5.3Futurestudies...322

V.GeneralDiscussion...325 V.1.Auditory feedback isimplicated in motorlearning,buthow dependson the population...326

1.1Disruptionsin auditory-motorcoupling:when ithelpsand when ithurts....327

1.1.aHealthy non-musicians:sound feedback-based learning...327 1.1.b Strokepatients:learning notto rely on sound feedback...329 1.1.cMusicalexperts:learning through feedback,performancethrough feedforward...337

2.1From aheterogeneousaccount...337 2.2. to auniied accountofsensorimotorintegration...338

VI.Referencesand indices...343

VII.Appendix:Parkinson'sDiseasepatients...371 VII.1.Efectsofdopaminergicand subthalamicstimulation on musical

performance.....372

VIII.Appendix:Fine-tuning thetiming analysisin experts...379 VIII.1.h einluenceofchronotypeon making music:Circadian luctuationsin pianists’inemotorskills...380

VIII.2.Musician’sdystoniain pianists:long-term evaluation ofretraining and othertherapies....390

IX.Manuscriptlisting...397

Preface

histhesisinvestigatesauditory sensorimotorintegration.Wedeinesensorimotor integration as(1)criticalinvolvementofmotorsystemsin sensory processing

(sensory-to-motor),and (2)criticalinvolvementofsensory systemsin motorprocesses (motor-to-sensory).A crucialquestion isto whatextentthisinvolvementisreally critical. hisquestion hasbeen addressed in somedepth butonly relating to oneofthetwo aspects ofsensorimotorintegration,namely motorinvolvementin sensory processing.

Experimentalstudiesdealing with thisquestion revealthatthemotorsystem isactivated in action perception,in away thatiscongruent(somatotopic)with theperformed actions.he motorsystem “mirrors”theperceived action.Similarly,atermusicalstrokerehabilitation, patientsshow motoractivation whilepassively listening to melodies.hisresultistaken to mean thatsomehow,thecoupling between perception and action isresponsibleforclinical improvement.However,thecriticality ofaction-perception coupling ischallenged by previousindingsthatthecongruency with theperformed action can bealtered.hese viewssuggestthatmotoractivation during perception merely relectsthatthebrain has learned to associatecertain motoractivationswith certain sensory activations.hisargues againstcriticalinvolvement.

However,thisquestion ofcriticality thatisamply investigated in motoractivation during perception ishardly asked forthesensory-to-motoraspect.hatis,itremains largely unclearwhetherthesensory systemsareactually criticalto motorprocesses.his, then,isthetopicofthecurrentthesis.Wewillfocusparticularly on thetiming ofmotor actions,and investigatewhethersensory systemscrucially contributeto itslearning.

hehypothesisunderlying thisthesisisthathow sensory systemsareinvolved,and whetherthey arecritical,may welldepend on thepopulation ofparticipantsthatwestudy. Forexample,musicians,who havebeen exposed to many thousandsofhoursofpractice, may haveso tightly linked theauditory outcometo motoractionsthatthey no longerneed itto bephysically present.Strokepatientswho suferfrom motordeicits,on thecontrary,

may perhapsusetheauditory feedback,butin adiferentway than healthy individuals.

Wewilladdressthesequestionsin thefollowing way.In partIwewillmakemore precisewhatweunderstand by sensorimotorprocessing,arguing thatscientiicprogress willbehampered ifwefailto makeitprecise.Weinvestigatethetwo aspectsof

sensorimotorintegration introduced abovein moredetail.Understanding thequestion of criticality ofmotorinvolvementduring perception willrequireusto review asubstantial portion oftheliteratureon action and speech perception,beforeturning to music,which is themain topicstudied presently.Next,weturn to threepopulationsin which westudy auditorimotorintegration.First,partIIinvestigatesthequestion whetherauditory

feedback can beused in motorlearning in healthy,musicalnovices.Based on itsresults,we investigatein partIIIhow auditory feedback isused in non-musician strokepatientswith motordeicitswho arere-learning motorskills.Finally,in partIVweask how auditory feedback isused by expertmusicalperformers.To answerthisquestion,wedevelop methodsofanalysisthatinvestigatetiming irregularities.Forthis,wetakescaleplaying as aparallelto theregularity learning used in partsIIand III.

“[M]abisognacercaredicapire,lavorando difantasia,edimenticarequelche sisain modo chel'immaginazionepossavagabondarelibera[.]”

-Alessandro Baricco,OceanoMare

I.1.Introduction

1

.

1 Ai

m

Sensorimotorintegration isaterm thatisused alot1_{butdiferentauthorsmean}

such diferentthingsby itthatitisdiicultto understand whattheterm really means.No review paperexiststhatcombineseven asubstantialportion ofthevariousfragmented subsetsofliteratureaboutsensorimotorintegration.

heaim ofthisintroduction chapteristo illthisvoid:to provideasystematic survey oftheliteratureon sensorimotorintegration.Itsgoalisto bean overview and a conceptualframework in which thevariouspartsoftheliteraturecan beplaced.Wewill also try to becritical,pointing outover-generalisationsand missing piecesofevidence. Such insightsopen theroad formany morefutureexperimentsthan could possibly be performed in theframework ofthisthesis.

Sincethesurveyed literatureisbroad,thepresentreview could neverbeexhaustive. heaim isto besynthetic,butnotsupericial.Asaresult,wemay sometimespointthe readerto morespeciicreviewsthataremoreexhaustiveforaparticulararea.

1 A search in PubMed (www.pubmed.org)on 28August2013revealed that15,371scientiicreferencesuse theword “sensorimotor”in theirtitleorabstract.hekeyword “sensorimotorintegration”yielded more than athousand publications.

1

.

2 Di

s

c

l

a

i

me

r

:

indi

ng

a

ba

l

a

nc

e

be

t

we

e

n

uni

t

y

a

nd

di

v

e

r

s

i

t

y

integration?heaim ofscienceisto identify and describeunderlying lawsthatgovern seemingly diferentphenomena.hatis,thehopeisto beableto reducetheoverwhelming multitudeofobservablesto asmallclassoflawsthatallow usto understand whatiscrucial abouttheirproperties.Forexample,an importantbreakthrough in sciencewasachieved when itwasrealised thatplanetary movementsweregoverned by exactly thesamelaws thatcaused an apple,oncereleased from atree,to drop to theground.In thisway,two very diferentphenomena(movementoftheplanetand thefalling oftheapple)could beunited by postulating asinglelaw (gravity).Considerthefollowing alternativeexample.Justater thesun sets,aparticularplanetoten becomesvisible.In theearly days,itwasthoughtto beastar,and thereforewascalled theEveningstar.Similarly,justbeforethesun rises,a pointoflightwould bevisible,thatby analogy wascalled theMorningstar.With theadvent ofmoreadvanced astronomy,itwasrealised thatboth wereactually thesamephysical object,namely theplanetVenus.Again,two distinctphenomenahad been united into a singlecause.

hescientiicintuition thatdrivesusto alwayslook forunifying causesand reducing phenomenato theminimum ofgoverning lawsiscertainly amostvalid one(see forexampleOckham'srazor).However,itcan sometimesbemisleading.Notethatin the lastexampleVenushad been referred to asastar.Wenow know thatitisnotastarbuta planet.Foralong time,planetsand starswerethoughtto bethesame.Whatconcerned scientistsoftheday wasthat“planetary stars”moverelativeto stars,whereasthestars remain ixed relativeto each other.Why would two instancesofthesamephenomena exhibitsuch diferentbehaviours?heinsightthatallowed thepiecesofthepuzzleto fall into placewasthatactually the“planetary stars”werenotstars.hey werefundamentally diferent.Againstthecurrentofscientiicthoughtthattried to unify thephenomena,itwas recognised thatplanetshad to bedistinguished from stars.A similarcasewasthatofatoms, which wereinitially thoughtto beallthesamelikerefrigeratorsfrom amass-production factory.Itwould betheirdiferentialarrangementthatgaveriseto thevariouschemical substances.However,onceitwasrecognised thattherewerefundamentally diferentatoms, thescientiicield could advance.heremay beaparallelto thisin thecurrentscientiic

view ofsensorimotorintegration.Forexample,sensory suppression (seesection 5.1.b)or feedback control(seesection 6.1.c)haveboth been called sensorimotorintegration. However,acloserlook atwhatthesephenomenaareshowsthatthey arestrikingly

diferent,both in how presentcomputationalmodelsimplementthem and in theunderlying neuralsubstrates(Hain etal.,2000).

hisreview willtry to approach sensorimotorintegration drawing inspiration from thesegreatscientiicdevelopmentsofthepast.Wewillthereforenottry to unify the

variousphenomenathatarereferred to by theterm sensorimotorintegrationatallcosts.As amateroffact,ourscientiicprogressmay behindered by considering two phenomena identicaljustbecausethey havebeen given thesamename.Instead,wewillpresentour framework and placethevariousphenomenain it,inviting thereaderto makeup hisorher own mind.In orderto facilitatethisprocess,wewillbecarefulin deining thephenomena underquestion.hereadermay feelthatthisistediousand overly scholarly.However,in doing so weprovideacloseraim forcriticsofthiswork atwhich to shoot,which isaterall how scientiicunderstanding advancesmost.Furthermore,ourhopeisthatitmay prime greaterclarity in ourthinking on themater.

I.2.Preliminaries

Somemay say thatin thebrain everything isconnected to everything and therefore oneshould notdividethebrain into subsystemssuch astheauditory orthemotorsystem. Perhaps,to someextent,thisistrue.However,thebrain connection network is

astoundingly small-world,meaning thatgenerally itismuch moreclustered than would be expected ifallconnectionswererandom.Atthesametimeany nodein thenetwork can be reached from any othernodethrough asmallnumberofsteps(Bullmoreand Sporns,2009). hismeansthatthereisalso agreatdealofstructurein thebrain networksthatwillallow usto identify moreorlessclearly demarcated networksthatareresponsibleforparticular kindsofprocessing.In particular,wewillhaveto startoutby deining moreclearly what themotorareasarein orderforusto beableto meaningfully talk aboutitsintegration with auditory areas.Furthermore,wewillintroducein thissection thevariousmethodologies

thatareused to measuremotoractivity orinterferewith motorsystem processing.

2

.

1 h e

huma

n

mot

or

s

ys

t

e

m

–

a

br

i

e

f

ov

e

r

vi

e

w

hemotorsystem isdeined asallpartsoftheorganism thatenableitto move.In whatfollows,wewillpay someparticularatention to thewaysthebrain controlsmuscles to efectmovement,butalso to thewaysin which thebrain receivesinformation aboutthe stateofthemuscles.Generally,information thatstreamsoutofthebrain into theefectors isreferred to aseferentinformation;theinformation thatiscarried back to thebrain is aferent.

2.1.a Motorcontrolareas– abriefoutline

heprimate'smotorsystem isacomplex and multifaceted system implicating nearly every structurein theorganism.However,severalkey areascan beidentiied and their rolesclariied.In thissection,wewillbriely review theseareasand pointoutparticular propertiesthatarerelevantto ourdiscussion lateron.Clearly,thissurvey can neverhope to beexhaustive,and thereforetheinterested readerisreferred to varioustextbooks (Rosenbaum,2009;Schmidtand Lee,1988).

i. Musclesand spinalcord

Limb movementsareachieved through muscleibrescontracting in responseto electricalstimulation from motorneurons.Muscleibresaregrouped into motorunits which areinnervated by asinglemotorneuron.hemusclebodiesalso contain sensory receptorsknown asmusclespindlesthatthecentralnervoussystem usesto detectthe presentmusclelength (also oten referred to asstretch receptors).Furthermore,thetendons, thetissuethatatachesthemusclesto thebones,contain receptorsoftheirown.hese so-called Golgitendon organs'neuralresponseisroughly oppositeto thatofthesensory cellsin themusclesthemselves:tendon receptorsireduring musclecontraction,whereas musclespindlesirewhen muscleslengthen.Muscleactivity can bepicked up on the surfaceofthebody through amethod called electromyography(EMG).Upon muscle

generally taken asan objectivemeasurefortheonsetand amplitudeofmuscularactivity.

ii. Cerebellum

hecerebellumisoneofthelowestlevelmotorareasin thehuman brain.Ithasbeen implicated in regulation ofmuscletone,thatis,controlling ofthestifnessoflimbs

(Mathewsand Rushworth,1958).Patientssufering from cerebellarlesionsoten show problemsproducing movementsequencessuch asalternating hand position between palm up and palm down (dysdiadochokinesis)(Holmes,1939).Similarly,thesepatientsoten exhibitproblemsin timing alternating contractionsofagonistand antagonistmuscles (Hallet etal.,1975),yielding behaviourthatissimilarto thatofhealthy individualshaving taken alcohol.Furthermore,thecerebellum appearsto play an importantrolein the

learning ofskilled movements.Forexample,iring ratesin cerebellarcellswereassociated with motoradaptation (Gilbertand hach,1977).Finally,thecerebellum hasbeen suggested to providetheneuralsubstrateforinternalmodels(which willbediscussed laterin this chapter)(Wolpertetal.,1998).

iii. BasalGanglia

hebasalganglia(orbasalnuclei)form asetofsubcorticalstructuresthathavebeen implicated in motorcontrol.hestructuresareparticularly densely connected to structures in thecortex (which willbediscussed below)and to thethalamus.Amongstother

functions,thebasalgangliahavebeen suggested to play acrucialroleasatimekeeperin a variety ofmotortasks,providing an essentialpulsethatmakesmovementpossible, especially when thismovementrequiresaregularrhythm such aswalking (Britain and Brown,2013).Parkinson'sDisease(PD)isassociated with widespread lossofbasalganglia functioning and consequently patients'movementssuferfrom “freezing”ortremor.

iv. (Primary)Motorcortex

hecorticalstructurethatismostdirectly associated with movementiscalled the primary motorcortex.When humansinitiatemovements,corticalactivation isspread acrosstheentirebrain around 1.5secondspriorto movementinitiation.However,closerto themovementitself,around 50msecpriorto EMG-detectableactivity thecorticalactivation

focuseson theprimary motorcortex (Deeckeetal.,1969).hissuggeststhattheprimary motorcortex ismoreinvolved in theimmediateexecution ofmotoractsand nottheir planning.Such interpretation isfurtherstrengthened by theobservation thatactivating the neuronsin themotorcortex through trans-cranialmagneticstimulation(TMS)directly causesmusclesto twitch.In TMS,arapidly changing magneticield iscreated justabove theparticipant'sskull.hisield givesrise,through induction,to weak electriccurrentsin thebrain tissue.In thisway itcan causeactivity in targeted brain areas.To measure

TMS-induced muscletwitchesonecould simply visually observethemovements.A more subtleway isto record electricmuscularactivity from theskin surfacethrough EMG. Recording EMG in responseto TMS stimulation ofmotorareas(typically theprimary motorcortex)yieldsmeasurableelectricactivity in musclesthatarereferred to as motor-evokedpotentials(MEPs).

v. Premotorcortex

heprimary motorcortex projectspredominantly to distalmuscles(i.e.,farfrom the torso)such astheingers.hepremotorcortex isacorticalstructurethatliesjustanterior to theprimary motorcortex,but,contrary to theprimary motorcortex,itprojectsmainly to proximalmuscles(i.e.,closeto thebody center).hishasled to thehypothesisthatthe premotorcortex isinvolved in posturalcontroland orienting ofthebody (Wiesendanger, 1981).Electrophysiologicalstudiessuggestthatpremotorcortex may bespeciically involved in action anticipation.Monkeysthatweretrained to respond to awarning light thatindicated movementdirection,butwaituntilago signaloccurred,showed premotor activity between thewarning lightand thego signal,butnotater(Weinrich and Wise, 1982).

vi. Supplementary MotorArea

hesupplementary motorarea (SMA)hasbeen implicated in theplanning ofmotor sequencesaswellasbimanualcoordination (Brinkman,1984;Kermadi,Y.Liu,A.Tempini E.M.Ro,1997).heareaislocated dorsalto thepremotorcortex.heareatypically shows widespread activity 1second beforemovementinitiation (Deeckeetal.,1969)and isactive during imagination ofamovementaswellasactualmovementexecution,contrary to the

primary motorcortex,which isactiveonly in thelatercase(Roland etal.,1980).

2.1.b ControlStrategies

Now thatwehavepresented amap ofthecoremotorareasin thebrain,onemight wonderhow theseareasareused to controlmovements.hebrain'smovementsarenot random butclearly (and oten successfully)achievesomegoal.In whatway isthiscontrol overmovementachieved?In brief,controlstrategiesthatareofinterestto ourdiscussion arefeedforwardand feedbackmodels,which wewillnow discussin moredetail.

i. Feedforward

Supposethatwewish to generateagrasping orreaching movementto aparticular target,forexampleto acup placed on thetable.Oneofthesimplestwaysto controla system (in ourcase,themotorsystem)isto pre-program whatitshould do atwhatpointin time.Forexample,ifwewantto makeourarm reach acup,themotorsystem could

pre-program which musclesto contractatexactly whattime.However,such asystem would bevery rigid:itwould notbeableto grasp cupscorrectly ifourmusclesweretired, orifourarmsgrew heavierovertime,orifwewereinjured.

ii. Feedback

Anotherway to makesurethatourarm actually reachesthetargetisto keep our eyesopen and continuously track thepositionsofthetargetand ourarm.hebrain can draw amentalarrow from ourcurrenthand position to theposition ofthecup.hisarrow isencodesthedirection and length thatourarm should travelto reach thegoal.hismental arrow isreferred to astheerrorsignal,sinceitindicateswhetherweareof thetargetand by how much (Fineand horoughman,2007;Robinson etal.,2003;Soberand Brainard, 2009;Weiand Körding,2009).Itconstitutesthefeedback and in thiscaseitisvisualin nature,butonecan imagineasimilarscenario in which weusetactileinformation instead. heproblem aboutrelying on feedback isthatthefeedback is(a)sometimesnotthere (becauseforexampleourarm may beblocking theview ofthecup forafew instants),(b) delayed (becauseby thetimethevisualinformation reachesthebrain and isunderstood, thestateoftheworld haschanged),and inally (c)noisy.

A third classofmodelscalled State-Feedback Controlmodelsaddressthese problems.

Illustration1:Feedback-basedlearning

iii. StateFeedback Control(SFC models)

StateFeedback Control(SFC)models(Shadmehrand Krakauer,2008;Jacobs,1974) solvetheproblem offeedback models'relianceon continuous,immediate(notdelayed), non-noisy sensory inputby constructing an internalmodel(state)and updating itin real-timeusing copiesofthemotorcommandsthatareissued.hemodelcan then beused to generateafeedback signalthatisused instead ofthesensory signal.Generally,one distinguishestwo kindsofsuch internalforward models(Wolpertetal.,1995):(I)models thatmakepredictionsaboutthestateofthemotorapparatus(on thebasisoftheprevious stateapparatusand themotorcommandsissued since),and (II)modelsthatmake

predictionsregarding thesensory consequences(again on thebasisofthegiven stateofthe motorsystem).

using sensory information)orfeedback.

I.3.Towardsa deinition ofsensorimotorintegration

3

.

1 S

e

ns

or

i

mot

or

i

nt

e

g

r

a

t

i

on

heterm sensorimotorintegration hasawidevariety ofmeaningsand isused by diferentresearchersin diferentways.In thissection,wewillmakeastarttowardsa deinition thatispreciseand yetbroad enough to encapsulateitsmostcommon uses.

Sensorimotorintegration(I)in itsirstsenseisdeined asreferring to the phenomenon in which theperceptualand motorsystemsareboth involved in both

observation and action.hatis,irstly,theperceptualsystem isinvolved in production,and secondly,themotorsystem isinvolved in perception.Wehavegeneralised thisfrom the deinition introduced by Hickok etal.(2011).

Firstofallitisimportantto pointoutthattheseideaspresupposethatwehavea cleardeinition ofwhattheperceptualand motorsystemsareand thatthey aredistinct.For example,why would wecallacertain brain areaan auditoryprocessing area?Wewould arriveatthisdesignation based on (a)aseriesofexperimentsin which weassume

participantsareengaging in auditory processing and in which weind thisareato be activated;and furthermore(b)an additionalseriesofexperimentsin which participantsare assumed to benotengaging in auditory processing and in which theseareasarenot

activated (seeZatorre(2007)forsimilarideas).An analogousprocedurewould allow usto establish which brain areasto considermotorprocessing areas.Herewerun into a

contradiction:inding thatcertain areasareinvolved in both perception and action means thatneitherwillfulilthecriteriaforbeing an auditory ormotorprocessing areain theirst place.A consequenceofaperceptual-motoroverlap isthat,strictly speaking,thesystemsin question can only bedesignated assensorimotor.Indeed thisistheposition some

researchershavetaken (Prinz,1997).

Sensorimotorintegration(II)in itssecond sense(distinctfrom thesensein which itisused above),isdeined astheprocessofcombining sensory with motorinformation so thattheorganism can efectively interactwith itsenvironment.In thiscase,integration refersto thecomputationalprocessing step thatcombines(possibly with several

transformations,seesection 6.1.b)thesensory and motoreferentstreams.Notethatitis trivialthatsensory and motorsystemsareinvolved in theprocess,becauseaterallthetask athand wasto combineinformation from them.Rather,sensorimotorintegration in this second sensepointsto thefactthatitisamazing thatthebrain isableto do itin theirst place.Forexample,Wolpertetal.(1995)usetheterm in thisway:how doesthebrain combineeferencecopiesofmotorcommandswith perception (proprioception)in arm movementsto createasingleestimateofthearm position?

Furthermore,noticethatsensorimotorintegration in itssecond senseisa

prerequisiteformostinstancesofsensorimotorintegration in theirstsense.hereason for thisisthatifmotorsystemsareinvolved in sensory processing,orviceversa,then there mustbesomeway in which information isexchanged between thetwo systems.he sensory information mustsomehow betransformed into motorinformation,orviceversa, orboth mustbetransformed into an intermediaterepresentation.So although thetwo phenomenaaredistinct,they arerelated,and thesecond senserefersto amorespeciic processthatisrequired fortheirst.

3

.

2 Ai

m

In thischapter,wewillreview theliteratureon sensorimotorintegration.hemain questionsarewhattheroleofthemotorsystem in perception is(section I.5)and whatthe roleoftheperceptualsystem in motorfunctioning is(section I.6).Forboth ofthese

sections,wewillinvestigatethreephenomenain thisorder:action in general(sections5.2, 6.1and 6.2),speech (sections5.3and 6.3)and music(sections5.4and 6.4).

hemain question willbeto whatextenttheinvolvementofonesystem really contributesto processing in theother.hisquestion liesattheheartofthedebatein motor theoriesofperception:to whatextentismotorinvolvementcritical,and to whatextentisit

merely associative?heconverseofthisquestion istowhatextentissensoryfeedback criticallyusedin motorlearning?Wewillshow thatthelaterhasbeen investigated much lessthan theformer;thatis,much eforthasgoneinto addressing thisquestion from the perspectiveofmotorinvolvementin perception,butvery litlehasgoneinto addressing it from theperspectiveofperceptualinvolvementin motorprocesses.hepresentthesis addressesthisgap,in particular,wewillexaminetheuseofauditory feedback in the learning ofmovementtiming.

hereason wedo notrestrictourselvesto auditory-motorinteractionsalone,(n)or theextentthatthey occurduring musicperformance,isthatwebelievethereismuch to gain from puting ourdiscussion in thiswidercontext.Alltoo oten speech perception scientistsdo nottalk to speech production scientists(Hickok etal.,2011)and

neuroscientistsdo notinteractwith psychologists.heaim ofthisreview isto show connectionsbetween thelinesofthoughtpursued in thesediferentields,and to construct acoherentpictureofwhatsensorimotorintegration isabout.

Why willwediscussspeech even though thisthesisdoesnotexperimentally investigatespeech?herearetwo importantreasons.First,speech hasbeen thefocusofa greatdealofscientiicinquiry and thereforeamuch greaterbody ofevidenceand models areavailable.Wewilltakeagreatdealofinspiration from therelectionsofspeech

scientists.Second,thereisanaturalainity between musicand speech.Both arecarried by an auditory signaland themotorimplementation in creating thisauditory signalisnot essentialto thefunction oftheauditory signal.Forexample,forasfarasthespeech content isconcerned,itdoesnotmaterwhetheronesays:“heleafisgreen”whilststanding on one'shead,chewing acookieorwhiletaking abath.In someway,werecognisethatthe speech contentissimilar(even though theacousticrepresentation ofthespeech could be severely distorted by thecookie).In otherwords,theauditory signaliswhatiscontrolled by thebrain instead oftheparticularmotorimplementation (Nieto-Castanon etal.,2005; Kelso etal.,1984).hesameappearsto hold formusic.hatbeing said,recentstudiesshow thatconcertaudiencesaregreatly inluenced by seeing themovementsofaperformer,and perhapsmorethan by thesound (Tsay,2013).

I.4.Motortheoriesofcognition

In English onecan metaphorically say thatyou graspan idea,playfully suggesting thatmotorprocessesareinvolved in thepurely mentalactofthinking.Indeed,some theoristspostulatethatmotorsystemsareinvolved in allcognition.An exampleisthe behaviouristschoolofthought.According to behaviourism in itsmostpureform,thoughtis merely asubtleform ofmovement.hatis,no thoughtcan existwithoutmovement.For example,itwasproposed that“thereisnothing in themind thathasnotbeen explained in termsofmovement”(Pillsbury,1911,p.84).A specialroleseemsto beplayed by thespeech apparatus:“thoughtprocessesarereally motorhabitsin thelarynx”(Watson,1913,p.174). Someexperimentalstudiesappeared to supportthisview.Forexample,when peoplesimply think (which happensto mostofus),EMG wasfound in laryngealmuscles(Sokolov,1972), indicating thattheindividualsmadespeech-likemovementsthatwerenotstrong enough to produceaudiblespeech,so-called innerspeech orsubvocalspeech,seealso Oppenheim and Dell(2010).

However,experimentalevidencesoon discredited thisview.In aspectacular experimentaresearcherallowed himselfto becomeentirely paralysed (Smith etal.,1947). During thisperiod,hewasableto havethoughtsand understand whatpeopleweresaying to him,ashereported when laterherecovered hismovementabilities.Whatthisshowsis thatbehaviourism in itsultimateform isuntenable.However,itisimportantto notethat theieldsofneuroscienceand cognitivepsychology subscribeto behaviourism in amore subtlerform.Behaviourism isaresponseto dualism,theposition thatthereexist“mind mater”,aposition thatwasmostardently defended by Descartes.A problem ofthis position isthatit,too,turned outto beuntenable,sincemind and matershould interact, butifthey areseparatesubstancesitisimpossibleforthem to do so (Dennet,1993).As mostofourscientiictheories,behaviourism aroseasacounter-movementagainstdualism (taking “movement”quiteseriously).In itsmostextremeform,itisnotdefended anymore. Instead,peopleoptforamoresubtleform,such astheposition thatthemotorsystem is cruciallyinvolved in cognition.

Positionsthataretoday realistically held in scientiicdebateincludeembodiment, which istheideathatthefeaturesofthephysicalbody ofan individual,(beyond justthe brain)play asigniicantrolein cognition (e.g.,homas,2013).Again,ittendsto beagreed

thatitsstrongestform isuntenable,asistheoppositeview thatthebody playsno role whatsoeverin cognition (Meteyard etal.,2012).Whatrolewould thebody play in cognition?heideaisthatperception isshaped by action.hatis,perception isnota function thatcan beseparated from an action,butratheritcomesinto being asaresultofa confrontation ofa(potential)actorwith an environment(Gibson,1977;Greeno,1994). Interestingly,somepatientpopulationsexhibitthisaction-contentofperception quite overtly.When they arepresented with objectsthey immediately grasp and usethem even when explicitly told notto orwhen itservesno purpose(Lhermiteetal.,1986;Lhermite, 1983).heseideaswerelaterincorporated in an inluentialpaper(O’Regan and Noë,2001) in which itwasargued thatperception arisesthrough detailed knowledgeofthe

relationship between sensory and motorevents(so-called sensorimotorcontingencies)(see also Buhrmann etal.,2013).In similartheoreticalaccounts,perception evolved in such a way thatitisinluenced by thecostsand beneitsofactions.Forexample,when

participantswearaheavy backpack theslopeofaramp seemshigher(Proit,2006).he reason isthatwalking up theramp becameconsiderably morecostly when participants werewearing aheavierload.Arguably itisevolutionarily beneicialforourperception to relectthisdirectly ratherthan requiring thebrain to draw thisconclusion separately.

Noticethatthereisasubtlediferencebetween theoriesofembodimentand motor theoriesofcognition,in thattheformerconsiderany featureofthebody (such asone's postureorexternalloadson thebody in formsofbackpacks)inluencescognition,whereas thelaterholdsitisonly onefeatureofthebody,namely movement.However,forour presentpurposeswewillcontentourselvesto pointoutthatthesescientiicviewsexist,and furthermoretreatthem asequivalentin so farasthey postulatemotorinvolvementin cognition.heinterested readermay consultrecentreviewson embodiment(Proit,2006), and speciically Davisetal.(2012)foradiscussion including musiccognition.

I.5.Motorinvolvementin perception

In thissection,wediscusstheirstofthetwo sidesofsensorimotorintegration, namely theideathatthemotorsystem isinvolved in sensory processes.Wepresenta

synthesisofabroad rangeofcrosstalk between human sensory and motorsystems. herefore,letusirsttakeamomentto presenttheabstractstructurethatwillbea

recurrentthemein theexperimentalparadigmswereporton.hisstructureisdepicted in illustration 1.Wehaveamotorleveland asensory (or,speciically,auditory)level,and the two arerelated in somespeciied way (auditory-motormapping).Forexample,whenever wemakeaparticulararticulatory speech gesture,asound willoccur,and thissound isnot random,butrelated to theparticularmovementwemade.Similarly,when wepressakey on apiano,wewillhearapiano tone(ifthepiano isnotbroken).Which tonewillsound dependson ourmovement.hatis,eventshappen in themotordomain (movements)and in thesensory domain (soundsorvisualevents),and theseeventsarerelated pairwise.he kindsofmovementsand thekindsofauditory eventswilldiferin thediferentsectionsof thischapter.When wetalk aboutspeech,theindividualmovementswillbethegeneration oftheindividualphonemes(orsegments)and theacousticeventswillbetheirsounds,and sequencesofthem may bewordsorphrases.When wetalk aboutmusic,theacoustic eventswillbenotes,and asequenceofthem willbeamelody.

An individualmay haveknowledgeaboutthemotorand auditory streams

individually.Forexample,shemay know how amelody continueswhen shehearstheirst few notes,orshecan guesstheending ofasentencewhen shehearsthebeginning.hisis knowledgeofthestructureon theperceptualleveland itisrepresented by horizontal arrowsin theillustration.Alternatively,aperson may haveknowledgeabouthow a sequenceofmovementscontinueswhen heknowshow itbegins,such asadancerwho may beableto continueachoreography aterseeing theirstfew moves,ormonkeys imagining seeing theexperimenterpicking up an occluded peanut.heseexamplesshow thatitispossibleto havesomeform ofknowledgeon ofthesequentialcontentofthelevel ofperception ormovementseparately.

Sensorimotorintegration relatesto theway thebrain representstheassociation between themovementand perception streams.hisintegration can happen on two levels, and itisimportantto distinguish thesetwo.Firstofall,thereisintegration on thelevelof individualsensory and motorevents.Forexample,in thecaseofapiano aperson may know thatstriking onekey willgiveonesound,and striking anotherkey willgiveanother sound.hisisakind ofmapping thatassociatesmovementsto perceptualevents(an

auditory-motormapping).Secondly,therecan beintegration ofsequencesofevents.For example,hearing aparticularmelody,aperson may beableto deducehow itwasplayed. Similarly,in thecaseofspeech,ithasbeen suggested thatshortsequencesofspeech sounds such assyllableswould bestored in an integrated,sensorimotorfashion (Eckersetal., 2013).hissecond kind ofintegration may happen withouttheirst:forexample,upon hearing amelody shehasbeen trained to play,aperson may know how itisplayed,but given any singlenoteshemightnotbeableto associateitwith akeystroke(which is,for instance,thecasein Lahav etal.,2007).

5

.

1 Mov

e

me

nt

ma

ki

ng

pe

r

c

e

pt

i

on

pos

s

i

bl

e

In orderto perceive,weneed to move.In orderto beableto smell,ourlungsneed to pump airthrough ournostrils.In orderto beableto feeltextures,ouringersneed to move (Gibson,1962).Similarly,in roboticsatempting to implementperceptivesystems,theidea thatoneneedsmovementin orderto implementperception iscalled activesensing

(Schroederetal.,2010;Saig etal.,2012).In thissection wewillpointto two phenomena thatimplicatemotoractivity in human perception,focusing on thevisualand auditory Illustration2:Elementvs.sequenceintegration.

domains.Wetaketheexampleofmicrosaccadesin vision.hen wediscussasecond exampleofmotion-perception crosstalk,which isthemechanism by which organismsare ableto distinguish between theirown movementand movementin theenvironment.

5.1.a Microsaccades(vision)

Oureyesbring objectsinto thefoveathrough fastjumping movementsreferred to assaccades.During saccadesvisualsensitivity isdrastically decreased (wearenearly blind), butduring theinter-saccadeintervalperception becomespossible.Itwaslong thoughtthat theeyesremain motionlessduring theseintervals.In fact,during “ixation”periodstheeyes slowly (a)drit,(b)undergo superimposed tremor,and,mostinterestingly,(c)perform small rapid microsaccadesattheapproximaterateofonceortwicepersecond (Martinez-Condeet al.,2013;Rolfs,2009).

Whatisthepurposeofthesesmalleyemovements?Upon theirdiscovery, equipmentwasdesigned thatwould counteractthesesaccadesby displacing thevisual imagein thesamedirection asthemicrosaccade.A remarkablediscovery wasmade:under thismanipulation,thevisualimagefadesin amaterofseconds(perceptualfading;

Ditchburn and Ginsborg,1952).Troxler,aSwissphilosopher,had proposed asearly as1804 thatobjectsin thevisualperiphery thatdo notmovedisappear(theTroxlerefect).he microsaccaderesearch in theitiesappeared to conirm thatthismay bepossibleeven for objectsappearing in thefovea.hislead to theinterpretation thatthevisualsystem is optimised to processchangesin visualstimulation atthecostoflosing from view scenes thatstay thesame.Asaresult,oncechangesin thevisualield areabsentduring ixation, perception fadesand artiicialchangesareinduced through microsaccadesto bring back visualexperience.However,laterthisinterpretation ofthemicrosaccades'function was challenged asobservationswerereported thatonly irregular,continuousmovement(thatis, drit-likemotion)counteracted perceptualfading (Gerritsand Vendrik,1970).Otherauthors suggested thatmicrosaccadesweretheoculomotorsystem's“busy work”,resulting from the factthatixating forlongerperiodsoftimeisan unnaturalthing to do (Rolfs,2009).

Nevertheless,overthelastfew decades,consensusappearsto favourthe

interpretation thatsomeform ofchangeisnecessary forsustained visualexperience.he oculomotorsystem achievesthisthrough smallmovementssuch asdrit ormicrosaccades,

oracombination thereof,aswellasmovementsofotherbody partssuch asthehead (Martinez-Condeetal.,2013;McCamy etal.,2012).

5.1.b Motor-induced suppression

When organismsperceive,they need to separatethesensory information originating from theirown movementsfrom thoseoriginating from changesin the environment.Forexample,sensitivity to tactilestimuliisreduced during activehand movementsbecausethey would overwhelm thesenses(Demairéetal.,1984).

Motor-induced suppression (MIS)(also sometimesreferred to assensory atenuation)isthe phenomenon thatthesensory consequencesofan organism'sown motoractsareperceived aslessstrong than physically identicalstimulithatdo notresultfrom theorganism's actions.Putdiferently,sensory suppression distinguishesinternalfrom externalsensory inlow.Forexample,auditory cortex neuronsin theprimatebrainsrespond lessstrongly to self-produced vocalisationsthan to thosesamevocalisationsplayed back to them

aterwards(Eliadesand Wang,2003;Müller-Preussand Ploog,1981).During eye movements,thebrain suppressesvisualinput(Bridgeman etal.,1975).Similarly,the chirping ofcricketsisso loud thatthey would deafen themselvesinstantly.However,at every chirp,thesensory system hasbeen warned and thesensory consequencesare

suppressed (Pouletand Hedwig,2003).Furthermore,auditory feedback dueto chewing in human beingsissuppressed (Rosenzweig and Leiman,1982)and thecorticalauditory responseto ourown phonation isatenuated relativeto aplayed back version (Houdeetal., 2002).No suppression occurswhen auditory feedback isreplaced by whitenoise

(Numminen and Curio,1999),suggesting thatthesuppression isnotmerely aforward processfrom themotorsystem to thesensory system,butalso modulated by sensory input itself.hebrain recogniseswhen itislistening to itselfand thereforetunesitselfdown,but doesnotsuppresssensory inputwhen itrealisesthatitislistening to something else.

How isthisatenuation possible?Clearly thesensory systemshavebeen informed ofwhentheorganism performswhatmovement.hiscommunication isachieved through an eferencecopy (Holstand Mitelstaedt,1950),which isacopy ofthemotorcommand as dispatched to themotorsystem,delivered to thesensory (orother)corticalsystems.Using theeferencecopy,thebrain isableto predictthenatureand thetiming ofthesensory

consequenceofthemotoraction in question.hispredicted sensory consequenceisusually referred to asthecorollary discharge(Sperry,1950)(forareview,seeCrapseand Sommer, 2008).heorganism can then adjustthesensory sensitivity accordingly.Aswehaveseen in section 2.1.b,atranslation ofintended motoractionsto sensory consequencesiscalled a forwardmodel.hisexplainswhy wecannottickleourselves:sinceourbrain isableto predicttheticklesensation beforeitactually happens,thesurpriseelementthatispresent when otherpeopletickleusisabsent(Blakemoreetal.,2000;Weiskrantzetal.,1971).In otherwords,self-produced ticklesaresuppressed.

An importantconsequenceofthetheory aboveisthatthesensory atenuation should besusceptibleto learning.When atirstwedo notknow whatthesensory

consequencesofan actwillbe,theprediction isthatno sensory atenuation willoccur.For example,when wearelearning to play thepiano wehaveno ideawhich movementswill generatewhich tonesand whatqualitiesofthemovement(such asthespeed ofthe keystroke)inluencethesound.Indeed,arbitrary movement-to-sound associationscan be created and resultin thecorresponding sensory suppression.Itwasshown thatsensory atenuation can occurand bemeasured asan atenuated EEG responseto self-triggered tones(Schaferand Marcus,1973).Auditory eventssuch astonestypically evokeaN1 responsethatismeasured atthescalp levelusing EEG.heamplitudeofthisresponseis found to bereduced when thetoneisself-triggered,and thishappenseven aterashort training block of60trials(Martikainen etal.,2005).hatis,oncethebrain learnsto associatetoneswith keystrokes,itstartsto suppressthetonesbutonly justater keystrokes.heatenuation isstrongerforan identicalstimuluswhen theparticipants believethey aretheagentrelativeto when they aremadeto (erroneously)believethey are not(Desantisetal.,2012).Similarly,thesuppression appearsindependentofatentional modulation (Timm etal.,2013),localin time(Chen etal.,2012b)and dependenton the predictability ofthetoneefect(Bässetal.,2008;Hughesetal.,2013).Motor-induced suppression happensin speech signalsaswell,and islesswhen thespeech feedback is distorted,such asby shiting itspitch (Heinks-Maldonado etal.,2005;Chen etal.,2013b). Sensory suppression appearsto also occurforjointly produced actions,although to alesser extentthan individually produced ones(Loehr,2013).

arecentstudy (Aliu etal.,2009)participantsrepeatedly pressed abuton atself-paced intervalsofabouttwo seconds.hekeystroketriggered atone,and theauthorsfound using MEG brain scansthattheN1responsein theauditory cortex wassuppressed relativeto a controlblock wheretoneswerepresented withouttheparticipantpressing akey.his suppression did notoccurimmediately,butonly ateratraining block.hisrelectsthefact thattheparticipantsneeded to train to beableto associatetheirmotoractwith thesensory consequence(tone)in orderto predicttheoccurrenceofthelater.Furthermore,therewas no suppression foradelayed tone.Interestingly,in adiferentexperimentparticipantswere trained in blockswherethetonealwayscamea100msecdelay atertheparticipant's

keystroke.heauthorsfound motor-induced suppression stilldid ariseand only in thiscase also generalised to zero-delay testtones.Furthermore,suppression generalised to alteration ofthemotoract(switching hands)orthesensory consequence(hearing atoneofadiferent frequency).Unfortunately,astheauthorspointout,thetwo conditions(training with 100msecdelay,training with 0msecdelay)cannotbedirectly compared sincein the 0msec-delay-experiment,ashortertraining block wasused.

Recently,thetheory thatthebrain needed to predictsensory efectsofitsactionsfor suppression to occurwaschallenged.In anotherstudy (Horváth etal.,2012),participants wereexposed to tonesoccurring atunpredictableintervalsand wereinvited to also pressa buton wheneverthey wanted.Although in thisexperimentalsetup no causality existed between thekeystrokesand thetones,neverthelesstheauthorsfound thecorticalresponse to thetonesin thetemporalvicinity ofthekeystroketo besuppressed.hisinding challengesthetheory thatsensory suppression isdueto thebrain'sprediction ofthe sensory consequencesofitsactions.

Itisinteresting to seehow sensory suppression parallelstemporalbinding,which is thephenomenon by which motoractionsand theirresultsappear(to theactor)to becloser in timethan they really are(Haggard etal.,2002).However,to ourknowledgeno studies havedirectly shown arelationship between theamountofsensory suppression and temporalbinding.A simpleand elegantstudy could measureboth quantitiesforthesame participantsand correlatethem.

5

.

2 Ac

t

i

on

pe

r

c

e

pt

i

on

First,wediscussmotorinvolvementin action perception in general.In latersections wewillturn to perception ofspeech (section 5.3)and music(section 5.4),which arespecial formsofaction observation.

5.2.a Mirrorneurons

Among themostfascinating recentdiscoveriesisthatofmirrorneurons:

double-duty neuronsthatireduring both action observation and action execution butnot in othercases.hisclassofneuronshasbeen suggested to play apivotalrolein

sensorimotorintegration,although thisideaisstilldebated.In thissection,wewillirst presentthebasicphenomenon (sectionsi-iii)and then investigatemoredeeply whatrole mirrorneuronsmightormightnotplay in sensorimotorintegration (sectionsiv-vi).Mirror neuronswereinitially discovered in visualaction observation.However,in lightofour futurediscussion ofspeech and music,wewillattheend ofthissection discussauditory mirrorneurons(section vii).

i. h ephenomenon:double-duty neurons

Galleseetal.(1996)reported inding singleneuronalcellsin themacaquemonkey's frontalcortex thatired both when observing an action aswellasperforming theaction. heneuronswerenottriggered when seeing eithertheagentortheobjectmanipulated in isolation.Onemightarguethatthephenomenon relectsthefactthatseeing an action remindsusofperforming it;thatis,theseneuronsmightsimply relectmotoractivity that becomesassociatedto observation.Galleseetal.(1996)argued againstthisinterpretation, showing no M1activity and no EMG activity on themusclesin question.Furthermore, mirrorneuronsdid notirewhen only theobjectoronly agentwaspresented norwhen the action ismimicked withoutan object.

ii. Brain areascontainingmirrorneurons

Initially,92mirrorneuronswerefound among 532non-mirrorneuronsin the macaquemonkey'sinferiorfrontalgyrusareaF5(Galleseetal.,1996).heF5areaisthe

motorareathatcontrolshand and mouth movementand hasbeen studied in considerable depth.Itcontains“canonicalneurons”thatirein responseto particularobjects(Rizzolati and Craighero,2004).Furthermore,theareaisthoughtto contain a“motorvocabulary,”that is,representationsofmotoractionsin theform ofneuronsthatirewhen themonkey performsspeciicactions(such asgrasping-with-the-mouth orgrasping-with-the-hand) (Rizzolatietal.,1988).

Subsequently,mirrorneuronswerealso identiied in theinferiorparietallobule (IPL)(Fogassietal.,2005)aswellasthemonkey'sprimary motorcortex (M1)(Tkach etal., 2007).helaterinding isinteresting,sincetheoriginalstudy (Galleseetal.,1996)had not found M1facilitation and used thisasan argumentagainstassociativemotoractivity.

hehuman areaBA44(Broca'sarea)wasinitially considered to bethehomologue (functionalcounterpart)ofthemacaqueF5.However,subsequently itwasthoughtthatthe human ventralpremotorcortex (vPMC,Brodmann Area6)isthemorelikely candidate (Morin and Grèzes,2008).hisisinteresting in lightoftheinding thatthehuman vPMcis involved in processing sequencesofabstractstimuli,regardlessofwhetherthey are

biologically meaningful(Schubotzand Cramon,2004).hismaterwillbediscussed in more detailin section 5.3.b.

iii. Precisely when aremirrorneuronsactivated?

heiring patern ofmirrorneurons(i.e.thesituationsin which they ireand the situationsin which they do notire)issurprisingly speciic.Mirrorneuronsonly irefor goal-directed (transitive)actionssuch asgrasping an object(Galleseetal.,1996)butnotfor objectlessactionssuch asmoving one'sarm.A subsetofthemirrorneuronsaretriggered even when theobjectthatisgrasped ormanipulated isnotdirectly seen butonly inferred (Umiltàetal.,2001).Mirrorneuronsalso appearspeciicto whethertheaction isperformed with atoolornot.Ferrarietal.(2005)ind neuronsin themonkey'sF5thatrespond even strongerwhen observing an action performed with atoolthan when themonkey performs theaction herselfwith thehand.However,otherstudiesfound no orreduced mirroriring during observation ofactionsusing tools(diPellegrino etal.,1992;Rizzolatietal.,1996).

iv. Action understanding

A debated question iswhatrolemirrorneuronsplay in cognition.Firstly,they have been proposed to befunctionalin action understanding (Rizzolatiand Craighero,2004). However,thedebatehashenceforth been hampered by lack ofclarity aboutwhatismeant by action understanding (Hickok,2009).Galleseetal.(1996)deined action understanding as“thecapacity to recognisethatan individualisperforming an action,to diferentiatethis action from othersanalogousto it,and to usethisinformation in orderto actappropriately” (p.606).Similarly,Rizzolatietal.(2001)proposethataction understanding is“thecapacity to achievetheinternaldescription ofan action and to useitto organiseappropriatefuture behaviour”(p.661).Moreprecisely,“[e]ach timean individualseesan action doneby anotherindividual,neuronsthatrepresentthataction areactivated in theobserver’s premotorcortex.hisautomatically induced,motorrepresentation oftheobserved action correspondsto thatwhich isspontaneously generated during activeaction and whose outcomeisknown to theacting individual.hus,themirrorsystem transformsvisual information into knowledge.”(Rizzolatiand Craighero,2004,p.172).Nelissen etal.,(2005) statethat“[a]merevisualrepresentation [ofan action],withoutinvolvementofthemotor system,providesadescription ofthevisibleaspectsofthemovementoftheagent,butdoes notgiveinformation criticalforunderstanding action semantics,i.e.,whattheaction is about,whatitsgoalis,and how itisrelated to otheractions”(p.332).Now noticethatthe conceptofthegoalofan action isequally elusiveto deinition.Isthegoalofan action (a) an "abstract"act,or(b)an object(e.g.when you grasp),or(c)ahypotheticalstateofafairs in theworld?Each oftheseseemsto insuiciently capturethenotion and beassociated with interpretativediicultiesoftheirown (Uitholetal.,2011).henotion ofaction understanding hassubsequently been generalised in humansto includespeech perception (Rizzolatiand Arbib,1998;Wilson etal.,2004).

v. Adaptivetask-dependentsensory-motorassociations

Asaresultofproblemsassociated with theaction understanding interpretation of mirrorneuron functioning,morecautiousinterpretationshavebeen advanced thatconsider themirrorneuronsto representadaptivetask-dependentsensory-motorassociations(Hickok, 2009)(seealso Mahon and Caramazza(2008)forarelated critiqueofembodied cognition).

Recallthatinitialinterpretationshad called upon theabsenceof“overt”motoractivity to arguethatmirrorneuron activity isnotsimply associative.Tkach etal.(2007)then found that,contrary to previousreports,themacaqueM1also containsmirrorneurons.his underminestheideathatno motoractivity isinvolved in action perception (Galleseetal., 1996).Instead,theinding suggeststhatthemirrorneuronsmay aterallbesimply asign of themotorsystem associating certain actionsto theaction percept(forexamplethrough Hebbian learning).hatis,during observation,themotorsystem is“geting ready”to move, withoutquitemoving yet(theso-called set-related response;Wiseand Mauritz,1985). Similarinterpretationshavebeen advanced towardsmirrorneuron'srolein social functioning (Keysersand Perret,2004).

vi. Otherproposed rolesformirrorneurons

Mirrorneuronsarealternatively thoughtto betheneuralbasisforimitation

(Iacobonietal.,1999;Rizzolatiand Craighero,2004),butthisisrendered lesscredibleifone takesinto accountthatthemacaquemonkey speciesdoesnotimitate(Hickok,2009). Additionally,Galleseand Goldman (1998)arguethatthemirrorneuron may betheneural substrateformind-reading (i.e.creating arepresentation ofpeople'sintention).Fogassiet al.(2005)similarly suggestthatmirrorneuronsserveto identify theintentionsoftheactor.

vii. Mirrorneuronsresponding to auditory observation

hemirrorneuronsoriginally discovered responded to visualobservation ofactions. Isthereanything specialaboutthevisualmodality ordo mirrorneuronsalso existforother modalities?Kohleretal.(2002)identiied visualmirrorneuronsin macaquemonkeysand found thatabout15% ofthem responded also to action-related soundssuch asripping paper orcracking apeanut.hissuggeststhatthisparticularsubsetofmirrorneuronsencodes action in amodality-unspeciicway (Keysersetal.,2003).Itremainsimportantto notethat thelargemajority ofmirrorneuronsdo nothavethisproperty butinstead arespeciicto onemodality,such asvision.

Furthermore,swamp sparrowsarefound to possesauditory mirrorneurons(Prather etal.,2008)thataresong-speciicand ireduring both listening to and singing thatsong. However,weshould notethattheserecordingsweremadein a"singing back"paradigm

and thereforethenatureofthetask may havegiven riseto theexhibited activations. Subsequently,theseneuronshavebeen referred to as“Echo neurons”(Zatorreetal.,2007). To ourknowledge,auditory mirrorneurons(i.e.singleauditory mirrorneurons,ratherthan auditory mirrornetworks)havenotbeen identiied in humans.

5.2.b Mirrornetworks

i. Distinguishing mirrorneuronsand mirrornetworks

Single-cellrecordingsin healthy humansareproblematicforobviousethical reasons.hereforethecounterpartofthemonkey studiesreported in theprevioussection could long notbeperformed in humans.However,fMRIbrain scansrevealed neural networksin thehuman cortex thathavepropertiescomparableto mirrorneurons.hatis, thesenetworksactivateduring action performanceand action observation (Gazzolaand Keysers,2009).FunctionalMRIdoesnothavesuicientresolution to pick up singleneuron activationsand thereforetheseresultsdo notshow conclusively thathumanshavemirror neurons.Ifneuronsinvolved in eitheraction observation oraction performanceare

spatially suiciently intermingled,onewillobtain thesameglobalnetwork activationsfor observation and execution withoutthereactually being single“mirror”neuronsinvolved in both (Illustration 3).In otherwords,wemay ind mirrornetworkswith single-duty

observation and single-duty execution neuronsheaped together,withoutany double-duty mirrorneurons.Notethatitwould bean interesting inding in itselfthatobservation and execution areso closely intermingled,butitdoesnotsuiceto demonstratemirrorneurons in human beings.

Forthisreason,itremained unclearwhethermirrorneuronsexistin humans.In 2009,Hickok published apassionateatack againstexisting assumptions,arguing thatno evidenceshowed mirrorneuronsto existin humans(Hickok,2009).Oneyearlater, single-cellrecordingsofactivity in thehuman medialfrontaland temporalcortex found evidenceofmirrorneuronsthatired both foraction observation and action execution (Mukameletal.,2010).Interestingly,thestudy also revealed “anti-mirror”neuronsthat wereactiveduring action performancebutinhibited during action observation.

Illustration3:Explanationofthediferencebetweenmirrornetworksandmirrorneurons.

ii. Visualaction observation

In humans,evidenceformirrornetworkshad been established long beforeevidence formirrorneurons.Fadiga,Fogassi,Pavesi,& Rizzolati(1995)applied TMS to themotor cortex ofhumansobserving motoractions.hey found theresulting MEP to besigniicantly increased relativeto acontrolcondition in which theirparticipantsdid notobservemotor actions.Thisinding isin linewith theinterpretation ofmirrorneuronsasaset-related response(seesection 5.2.a.v)which isfurtherstrengthened by theinding thatmotor activity during observation isenhanced when participantsknow they laterwillneed to reportspeciicpropertiesoftheaction (Schuch etal.,2010).

hemirrornetwork,likemirrorneurons,activatesspeciically during observation of biologicalmotion.Forexample,investigating whethertheobserved actorneeded to bea biologicalentity,Taietal.(2004)found no mirrornetwork activity when observing arobot perform an action,whereas(Gazzolaetal.,2007)did ind premotoractivation,leaving the issueofwhethertheobserved agentneedsto bebiologicalstillcontended.Related to this, Stevensetal.(2000)show participantsslide-showsofmovementsatvariousspeeds.When theslideswereshown athigh-speed,participantsconsidered themotion impossiblebecause

theexperimentalsetup impliesthelimbswould havehad to cross.Atlowerspeeds,the movementwasperceived asbiologically possible,asthereistimeforthelimbsto pass alongsideoneanother.Motoractivation wasmorepronounced during perception of

possiblemotion than during theperception ofbiologically impossiblemotion.In linewith thisinterpretation,adiferentstudy revealed thatrTMS overthehuman premotorcortex disruptsprocessing ofaction-related pictures(Urgesietal.,2007),which theauthors interpreted asan involvementofthehuman motorsystem in action processing (although thisinterpretation isdisputed;Hickok,2009).Similarly,rTMS overthevPMcimpairsvisual perception ofmovementsbutonly ofthosethatarebiologically possible(Candidietal., 2008).Similarly,Kilner,Paulignan,& Blakemore(2003)show thatparticipantsexhibitless motoractivity when observing arobotdo theaction,suggesting an inluenceof

characteristicbiologicalmovementproiles.Perhapsthisisdueto thefactthatbiological motion tendsto haveminimum jerk (Kilneretal.,2007).In linewith thisreasoning,motor activation ismorepronounced when observing slide-showsofstillsdepicting moreefortful actions(Proverbio etal.,2009).Kinematicmeasurementshavefurthermorerevealed thatthe observation-induced motoractivation can facilitatesimultaneousovertmovements

themselvesduring afairly brieftimewindow (Ménoretetal.,2013).

hemotoractivation during action perception appearsto bemoreclosely linked to processing ofthegoalthan oftheaction itself.Indeed,premotoractivity wasenhanced when participantswereasked to reportthegoalratherthan theefectorofobserved actions (Hyman etal.,2013).

Whatistheinluenceofexpertiseon thismirrornetwork activity?Severalstudies havesuggested thatexperienceperforming themotoraction in question subsequently enhancesmirrormotoractivity when observing theaction (Calvo-Merino etal.,2010,2005). Indeed,dancers'self-rated capacity to perform dancemovementscorrelated with ventral premotoractivity during passiveobservation (Crossetal.,2006).Furthermore,motor activity wasenhanced formovementsbelonging to thosedancers'repertoiresrelativeto movementsthey only observe(becausethey belong to theoppositegender)(Calvo-Merino etal.,2006).However,ithasfurthermorebeen suggested thatsimply observationallearning may explain thisefect,sincedanceloversshow motoractivation when observing other dancers,even when thelateraredoing movementsthatthey cannotdo (Jolaetal.,2012).