Interactive evolution for cochlear implants fitting

HAL Id: hal-00294838

https://hal.archives-ouvertes.fr/hal-00294838

Submitted on 29 Nov 2011

HAL is a multi-disciplinary open access archive for the deposit and dissemination of sci- entific research documents, whether they are pub- lished or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Interactive evolution for cochlear implants fitting

Pierrick Legrand, Claire Bourgeois-Republique, Vincent Pean, Esther Harboun-Cohen, Jacques Lévy Véhel, Bruno Frachet, Evelyne Lutton, Pierre

Collet

To cite this version:

Pierrick Legrand, Claire Bourgeois-Republique, Vincent Pean, Esther Harboun-Cohen, Jacques Lévy

Véhel, et al.. Interactive evolution for cochlear implants fitting. Genetic Programming and Evolvable

Machines, Springer Verlag, 2007, 8 (4), pp.319-354. �hal-00294838�

PierrikLegrand

1,2

,ClaireBourgeois-Republique

4

,VinentPéan

5

,

EstherHarbounCohen

6

,JaquesLevy-Vehel

2

,BrunoFrahet

6

,

EvelyneLutton

2

,PierreCollet

3

1

LaboratoiredeMathématiquesappliquéesdebordeaux,UMR5466 CNRS,U.F.R

SienesetModélisation, UniversitédeBordeaux2,146rueLéoSaignat,33076

Bordeauxedex,Frane

Legrandsm.u-bordeaux2.fr

2

COMPLEXTeam-INRIARoquenourt,B.P.105, 78153LeChesnayedex,

Frane

Pierrik.Legrandinria.fr,Evelyne.Luttoninria.fr,Jaques.levy-vehelinria.fr

http://fratales.inria.fr

3

Laboratoired'InformatiqueduLittoral,ULCOBP719, 62100Calaisedex,Frane

Pierre.Colletuniv-littoral.fr

http://lil.univ-littoral.fr

4

LE2I,UMR5158CNRS,9avenueA.Savary,B.P.47870,21078Dijonedex,Frane

Claire.bourgeois-Republiqueu-bourgogne.fr

5

CRTInnoteh,1PromenadeJeanRostand,93005Bobignyedex,Frane

vinent.peaninnoteh.fr

6

HpitalAvienne,ServieORL,125rtedeStalingrad,93000Bobigny,Frane

bruno.frahetav.ap-hop-paris.fr

Abstrat. Cohlear implantsare deviesthatbeome moreandmore

sophistiatedandadaptedtotheneedofpatients,butinthesametime

theybeomemoreand morediult to tune.After adeafpatienthas

beensurgiallyimplanted,aspeialisedmedialpratitionerhastospend

hoursduringmonthstopreiselyttheimplanttothepatient.Thispro-

essisaomplexoneimplyingtwointriatedtasks:thepratitionerhas

totunethe parametersofthe devie(optimisation)while thepatient's

brainneedstoadapttothenewdatahereeives(learning).Thispaper

presentsastudythatintendstomaketheimplantmoreadaptabletoen-

vironment(auditiveeology)andtosimplifytheproessoftting.Real

experimentson volunteer implanted patients are presented, that show

theeienyofinterativeevolutionforthispurpose.

1

1

Thisworkhas partiallybeenfundedbytheFrenhANR-RNTSHEVEAprojet

Cohlear Implants (CI) [Nih89℄ allow totally deaf people to hear again pro-

videdtheirauditorynerveandohleararestillfuntional:aomputerproesses

sounds piked up from a mirophone,to stimulate diretly the auditory nerve

throughseveraleletrodesinsertedinsidetheohlea(f.g.1).

As one an imagine, there are hundreds of parametersthat an be tuned,

andin thesametimethepatienthasto learntohear usingnewinformations

provided to his auditory nerve. Thetuning of suh a devie is thus extremely

omplex, and highly dependent on thepatient. This proess is urrentlydone

byhand bymedialpratitioners,andlookslikeanoptimisationproessbased

ontrialand error. This proessis so deliatethat sometimes,nosatisfatory

ttinganbefoundforsomepatients.

Hene,itseemsinterestingtouseaninterativeevolutionaryalgorithm(IEA)

to help ndingthe best valuesfor implantparameters. This is themain topi

of the HEVEA projet, whih is aollaboration between omputer sientists,

signalproessingexpertsandmedialresearhers.Theaimisatuallytwofold:to

failitatetheinitialttingofohlearimplants,andtoautomatisetheadaptation

ofohlearimplantstovarioussoundenvironments.AsimpleIEAwasdeveloped

with this in mind, and tested on a verybasi feature, the rangeof intensities

thataspeieletrodeantakewhenstimulatingtheauditorynerve.TheIEA

hasbeenimplementedonaPDAandtestshavebeenperformedonvolunteering

patientswithsatisfyingresults.

Thepaperisorganisedas follows:setion2presentsohlearimplants,and

setion3desribeshowtheyareurrentlytuned bymedialpratitioners.The

approahoftheHEVEAprojetisdevelopedinsetion4,andarstimplemen-

tation of an IEA is detailed in setion 5. Experimentson several patients are

reported insetion6,yielding goodresultsas wellas importantonlusionson

manual tting proedures.This rst validation step is important: an analysis

ofthesuessandfailuresraisesnewquestions thataredevelopedinsetion7,

related tothe well-knownuserfatigue problemof IEAs,and to thefat that

dierent sound environments havean important inuene on implants tting.

Automati adaptationof the devie to sound has been investigated, based on

asoundsignallassiationsheme,whihisdetailed in setion7.Conlusions

andperspetivesaredesribedin setion8.

2 Cohlear Implants

A ohlearimplantis asurgially implantable devie[GFM

+

98℄ that provides

hearingsensationstoindividualswithseveretoprofoundhearingloss,andan-

not benet from hearing aids. In a normal ear, sound energy is onverted to

mehanialenergybythe middleear, whih is thenonverted to eletrialim-

pulses by the inner ear (see gure 1). In order to perform this last stage, the

ohlea(part oftheinner ear) ontainsauidwhihis set into motionbythe

oval window whih is onneted to the middle ear. Within the ohlea, sen-

auditorynerve.Cohlearimplantsaredesignedtobeasubstituteforthemiddle

ear,ohlearmehanialmotion,andsensoryells,transformingdiretly sound

energy into eletrial energy that will initiate impulses in the auditory nerve

[B.C95℄,[Coh89℄thanksto adigitalsignalproessor.

Fig.1.Allimplantdevieshavethefollowingfeaturesinommon:soundisolletedby

amirophone(1)andsenttoeletroniomponentswithinaspeehproessor(2).The

speehproessor analyzes theinput signal (sound)and onverts it intoaneletroni

signal (eletrial).Thisodetravelsalongaable(3)tothetransmittingoil(4)and

is sent aross the skin viafrequeny modulated (FM) eletro-magnetiwavesto the

implant pakage(5).Basedonharateristisofthe odetransmittedtotheinternal

devie,eletrodeontats withinthe ohlea (6)provideeletrial stimulationto the

spiral ganglion ells and dendrites extending into the modiolus. Eletrial impulses

thentravelalongtheauditorynerve(7),asendingauditorypathwaystothebrain.

Cohlearimplants havebeen verysuessful in restoring partial hearing to

profoundly deaf people [ALM95℄, [Osb97℄. In2006, around 70 000deaf people

are implanted withsuh deviesaround theworld.Eienyis quite variable,

ranging from totallydeaf patientsthat havefully reoveredtheiraudition and

are apable to follow telephone onversations and enjoy musi, to others who

hear strange sounds they an't benet from, to a point where they prefer to

swithotheimplant[COM94℄,[GTBVC01℄,[BTE04℄,[Rom98℄.

For many people, it is still diult to fully take advantage of the devie

beause it is not easy to tune the parameters of digital signal proessor and

adjustthemfortheharateristisforeahpatient,sineallpatientsaredierent

(auseofdeafness,numberofyearsbetweentotaldeafnessandimplantation,age,

depthof eletrodeinsertion,...).

Researhhasbeengoingonsinenearly 50yearsagoonhowto eletrially

with one eletrode only, somelukypatientsmanaged to hear again and even

understandspeeh.Nowadays,itistehnologiallypossibletousemorethanone

eletrode,in order to stimulate moreofthethousands ofneuronsthe auditory

nerveismadeof[PCMF79℄[CFML83℄.However,themoreeletrodes,themore

parameterstotune.

The ohlea is used to interfae eletrodes and the auditory nerve. The

ohleaisabiologialdeviethatmainlyallowstomapdierentsoundfrequen-

iesontodierentneurons.Itisshapedlikeasnailshell.Onlylongwavelengths

(lowfrequeny sounds)an reah thefar endof theohlea,whileshort wave-

lengths(highfrequenysounds)arestoppedattheentraneoftheohlea.The

ideaisthenforsurgeonsto usethisfrequenydisriminatorandinsertintothe

ohleaathinsilionwire,bearingseveraleletrodes.

Stimulatinganeletrodeon thefar endof thewirewill therefore makethe

patienthearalowpithsound,whilestimulatinganeletrodeneartheentrane

oftheohleawillresultinthepatienthearingahighpith sound.

3 Cohlear Implant tting

3.1 Complexity ofthe problem

Being able to use morethan one eletrode to stimulate dierent neuron areas

isindeed agreatimprovement,butthenumberofparameterstotuneinreases

drastially.Conerningeletrodesonly,manyquestionsarise,amongwhih:

Whihfrequeniesshould bemappedtowhiheletrodes?

Whihrangeofintensitiesshouldbeappliedtowhiheletrodes?

Howmanyeletrodesshouldbestimulatedsimultaneously?

Shouldtheproessorprohibit neighboureletrodesto bestimulatedsimul-

taneouslyinordertoavoiddiaphony(rosstalkbetweennearbyeletrodes)?

Findinggood answersto thesequestionsis adiultoptimisationproblem.

Thisnotonlydueto theextremelylargesize ofthesearhspaebutto several

other reasons.First of all, thequalityof attingis atwostage proess where

subjetivityplaysalargerole:thepratitionerhastointerpretthequalityofthe

tting (rstsubjetiveproess)from the answersgivenby thepatient (seond

subjetiveproess).Thedisparityofpatientbehaviourwithrespettolanguage

andsensitivitytovariousthresholds,aswellastheharaterofthepratitioner

deeply inuenes the results. For example the well known psyhologialPyg-

malion eet biases answers of the patient, who often unonsiously tries to

satisfythepratitioner'sexpetations.

Thesoundenvironmentisanotherauseofvariabilityofresults,asthetting

session usually takes plae in a small room at hospital with the pratitioner.

Howevertheohlearimplantmustalsobeusedinreallife,andaorrettting

athospitalmayrevealveryunomfortableor unusefulwheninthestreet,orin

to test many possibleparameter sets during a singlesession, so the proess is

verylongandneedssometimes weeksto obtainasatisfyingresult.Inthesame

time,attingthatmaynotappearimmediatelyassatisfying,mayimprovewhen

testingitonalongerperiod (brainhasaplastiitythat annotbenegleted).

There are many fatorsthat make this problem highly irregular.However,

it has been proved that an aeptable or even good tting is reahable by a

manual searh onduted by an experiened pratitioner. We desribe below

thismanualttingtehnique,whih ismainlyahuman-guidedtrialanderror

proess,resemblingaloal searh.

3.2 Manualtting

Nowadays,dependingonthemanufaturer,thenumberofeletrodesvaries be-

tween8and22.Cohlearimplantttingisperformedbyanexpertpratitioner,

whoproeedsinthefollowingway:

Right after the surgial intervention, the pratitioner tries to determine

whiheletrodesarefuntional(aneletrodeisfuntionalifthepatienthears

asoundwhenurrentisapplied totheeletrode).

For eah funtional eletrode, thepratitionertries to determinetherange

ofintensitiesthatanbeused.Thelowestintensityabovewhihthepatient

pereives a sound is alled

T

time)isalled

C

Determining the

T

C

ation with a deaf patient, a young hild, or with an old patient an be

diult), and due to the inreasing number of eletrodes, some manufa-

turersnowadvise to determine

T

C

eletrodes,andextrapolatethevaluesfortheothereletrodes.See[Rou01℄,

[Hes02℄ formoreinformationsonthistopi.

Othermanufaturersevenset averagevaluesfor

T

C

responseorevenstatistis.

Then,one the

C − T

ttingbegins. The pratitioneruses his expertise to map frequeny bands

logarithmiallyontothedierentfuntionaleletrodes,andstartstotunethe

gainandsensitivitydependingonsoundfrequenies,thentunesthenumber

of simultaneously ative eletrodes,... while at the same time asking the

patientwhethertheyunderstandbetterorworse,whetherthesoundquality

is omfortable or not, a.s.o..In ertain ases, the pratitionerwill slightly

reduethe

C − T

neurologibandwidthislimited,andthattheneuronsfaingtheeletrode

aregettingsaturatedatonlymoderateauditorylevels.

Resultsarevariable,butoftengood.Usually,attingsessionstartswiththe

pratitioneraskingwhethertheurrentttingisbetterorworsethantheprevi-

ousone.Thebest ofthereentttingsistakenas abasisthatthepratitioner

tries to modify some parameters to help solving the problems. Two or three

parametersanbehangedduringa30to90minutesttingsession.Then,the

patientleaveswiththenewsettingsthathekeepsforaoupleofmonths,before

he omes bakfor another tting session. Thewhole proess is therefore very

long(severalyearsforproblematipatients).

4 Desription of the Problem

Asseenabove,ttingohlearimplantsisdonethroughaset oforrelatedpa-

rameters[LPD00℄,andpereptionandomfortthresholdsarelinkedtohistopatho-

logialfatorsspeitothepatient[KSC

+

98℄.Inmostases,thettingstrategy

simplyonsistsinmaximisingthenumberofeletrodesandmaximisingtheirdy-

namirange[BPG

+

92℄.Thisoftengivesgoodresults,butforsomepatientsthis

approah does not work.Moreover, the following observations have also been

reported:

Betterresultsmightbeobtainedbydereasingthedynamirange[FXP03℄.

Onlyusingasubsetofeletrodesmightimprovespeehreognition[ZCW97℄.

Holesinspetralrepresentationan existintonotopirepresentation(map-

ping of the sound frequenies on the eletrodes) and spetral information

redistributionaroundtheholesdoesnotinreasesresults[SGD02℄.

Moreover:

Mostofthepatientsdonotusealltheinformation givenbytheeletrodes

[Fis96℄.

All the eletrodes are not neessary to obtain maximal speeh pereption

performaneinsilent[DDML89,LWZF96,Fis96,KVR

+

00℄andnoisyenviron-

ments[FSBW01℄(partofthisouldbeduetoeletrialinterationbetween

hannels[SLM

+

06℄).

Thesepublishedobservationsshowthathoosingagoodsubsetofeletrodes

an havean inuene on speeh understanding, as well as the dynami range

on theeletrodes.Finally, taking intoaounta realsound environmentould

inreasespeehunderstandingforsomepatients.

Theworkpresentedinthispaperwill trytoaddressbothproblems.

5 Desription of the Interative Evolutionary Algorithm

Itseemsthatmanypatientswhoarenotsatisedwiththeirohlearimplantare

stuk in aloaloptimum: nomodiationproposedby theexpert would bring

anyimprovement.

Thistriggered the ideato use evolutionaryalgorithms, that are bothquite

goodatoptimising parametersandnoteasilytrappedin loal optima.Thege-

netiloopisthefollowing:theEAsuggestsasetofparametersthatarediretly

evolutionary algorithms, [Dur02,Tak01,Tak02℄, but they onern only onven-

tional hearing aids, with a relatively small number of parametersthat an be

tuned. Toourknowledge,nobodyhastriedtoapplyevolutionaryalgorithmsto

CohlearImplantstting.

5.1 Managing the runs

In an interativeevolutionary algorithm, a humanuser evaluates the dierent

individualsproposedbythealgorithm.

ThomasBäk'sresults([Bae05℄),suggestthatanevolutionaryalgorithmmay

do as well (if not better) than ahuman expert on a number of evaluations of

thesameorderthanthenumberofrealparameterstooptimise.Therefore,ifthe

problem hasaround 100 parameters to tune, performing only 100evaluations

should already allow to obtain interesting results. If it is possible to nd an

evaluationproedurethat takesaround5mn,arun wouldlast around8hours.

However, it is also important to take psyhology and human fatigue into

aount: awell tuned onvergene speed over100 evaluations ould seemdis-

ouragingfor a human patient, who may think that improvement is too slow.

Besides, sine it is not possible to have an 8 hour run in one go, an elegant

solution onsists in frationing the experimentation into several partial fast-

onvergingruns, with arestartat the end of eah run [Jan02℄. Dividing the8

hourruninto 5makesfor51h30runs,thatarequitemanageable.

Ratherthannding waysto avoidprematureonvergene,it isonthe on-

traryaveryfastonvergenethatissoughtontheseshortrunsofapproximately

20runs.Thisisquitenie,sineevolutionaryalgorithmsareknowntoonverge

quitefast,ifnoounter-measuresaretaken.

Thispoliy allowstouse averyfastonvergingalgorithm tryingtoexploit

loal minima,ratherthanaslowonvergingalgorithmtryingtowidelyexplore

thesearhspae,lookingfortheglobalminimum.Theonsequenesofpremature

onvergenearedealtwiththankstotheperiodialrestarts.Duringthelastrun,

oneanrestartthealgorithmwiththebestindividualsfoundinthe4rstruns,

soas tobenetfromtheresultspreviouslyfound.

Populationsizeandnumberofhildrenpergeneration. Foranidential

number of evaluations, twopossibilities exist:either many hildren per gener-

ation and a small number of generations, or a small number of hildren per

generationandmanygenerations.

Outofthesetwopossibilities,itisthealgorithmthatmaximisesthenumber

ofgenerationsthatwillfavourmostonvergene.ThissuggestsaSteadyStatere-

plaementpoliy,ora

(µ+λ)

λ

Thenin orderto notspend toomanyevaluationsin theinitialpopulation,one

analso redueitasisdoneinmiro-GAs[Kri89℄.

Extremely lowvaluesan beused, suhas 3to 6individuals forthe initial

4previousruns.

Thealgorithmhosenforthisspeiinterativeoptimisationwilltherefore

beamodernevolutionaryalgorithm,inthesensethatitdoesnottakeafterany

of thefour usual paradigms(EvolutionStrategies, GenetiAlgorithm, Geneti

Programming,EvolutionaryProgramming)[DJ05℄.

Aording to Bäk [Bae05℄, using an Evolution Strategie paradigm for100

evaluations should allowto optimiseup to 100real variables. In CohlearIm-

plantstting,however,oneanstartwithtryingtondthebest

T

C

foreaheletrode.WiththeMXM15eletrodesCIusedforthisexperiment,the

genomeisthereforeanarrayofonly30realvalues,meaningthatthehanesto

ndagoodttingaremuh higher.

5.2 Initialisation

Onehardonstraintneedstoberespeted:thealgorithmshouldnotgobeyond

themaximumintensityforeahoftheeletrodesforfearofdestroyingsomeof

the patient's auditory neurons. Therefore,for eah newpatient, arst session

withapratitionerisrealisedtodeterminethemaximumadmissibleintensityfor

eaheletrode,thatisalledapsyhophysial test.Inordertoreduethesearh

spae, a minimal intensity below whih the patient does not hear anything is

alsodetermined.

The initialisation of eah individual therefore simply onsists, for eah of

the15eletrodes,to pikuptworandom valueswithin the

[min, max]

determined during thepsyhophysialtest, and to takethelowervalueas a

T

threshold,andthehighervalueasa

C

5.3 Seletion of the parents

Parents seletion is dierent from the replaement stage, in that it an selet

an individual several times. Whenever a hild must be reated, two dierent

individuals are seleted among the parent's population, that an be seleted

againtoreateanotherhild.

Sinetheseletionpressureofproportionalseletiondepends onthetness

landsape of the problem to be solved(whih is unknown), astohasti tour-

nament is seleted [BT97℄, with a 90% probability, that onsists in randomly

seleting2individualsandto takethebest ofthetwowitha90%probability.

5.4 Crossover

Thegenesarerealvalues,whihouldhavesuggestedsomekindofbaryentri

rossover(suhasusedin EvolutionStrategies),whereeahgeneofthehildis

an average between thetwogenes of his parents.But sineit is intervals that

mustbeevolved,thistypeofrossoverwouldhaveledtoreduingtheintervals