HAL Id: hal-01915217
https://hal.sorbonne-universite.fr/hal-01915217
Submitted on 3 Sep 2019
HAL is a multi-disciplinary open access archive for the deposit and dissemination of sci- entific research documents, whether they are pub- lished or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.
L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.
Serious Games for Home Based Rehabilitation: Inertial Sensor Energy Consumption
Halim Tannous, Dan Istrate, Aziz Benlarbi-Delai, Julien Sarrazin, Marie-Christine Ho Ba Tho, Tien Tuan Dao
To cite this version:
Halim Tannous, Dan Istrate, Aziz Benlarbi-Delai, Julien Sarrazin, Marie-Christine Ho Ba Tho, et al.. Serious Games for Home Based Rehabilitation: Inertial Sensor Energy Consumption. In- novation and Research in BioMedical engineering, Elsevier Masson, 2018, 39 (6), pp.440-444.
�10.1016/j.irbm.2018.10.014�. �hal-01915217�
Contents lists available atScienceDirect
IRBM
www.elsevier.com/locate/irbm
JETSAN
Serious Games for Home Based Rehabilitation: Inertial Sensor Energy Consumption
H. Tannousa, D. Istratea,A. Benlarbi-Delaib, J. Sarrazinb, M.-C. Ho Ba Thoa, T.T. Daoa,∗
aSorbonneUniversité,UniversitédetechnologiedeCompiègne,CNRS,UMR7338BiomechanicsandBioengineering(BMBI),CentrederechercheRoyallieu- CS 60 319- 60203Compiègnecedex,France
bSorbonneUniversité,UPMCParis06,UR2,L2E,F-75005,Paris,France
h i g h l i g h t s g r a p h i c a l a b s t ra c t
•Batterylifestudyoftheinertialsensor.
•Currentconsumptionstudyoftheinertial sensor.
•Optimal configurationfor ahome based seriousgamesystem.
a r t i c l e i n f o a b s t ra c t
Articlehistory:
Received6October2017
Receivedinrevisedform4October2018 Accepted8October2018
Availableonlinexxxx
Keywords:
Energyconsumption Inertialsensors Seriousgames Homerehabilitation
Background: Serious games haverecently immerged as agood tool for physical rehabilitation.Thisnew technology can be used at home,to complement a traditional,clinic based, rehabilitation program. To implement a serious game athome, we need to use multiple sensors to record patients’ data. Many seriousgamesusevisualmotioncapturetechniques,liketheKinectcamera,duetotheirlowpriceandhigh portability.Ontheotherhand,someothersystemsuseinertialsensorstocollectdataatahigherdegreeof accuracy.Inpreviousworks,weshowedthataseriousgamingsystemcouldbenefitfromcombiningdata fromdifferentsensors.However,theuseofinertialsensors,inahome-basedsetting,remainsachallenge sincetheyneedtobesuppliedbyanindependentbatterysource,whichcouldinfluencetheacceptability ofsuchsystems.
Methods: Inthispaper,wepresentanenergyconsumptionstudy,performedontheinertialsensorsusedin ourseriousgamesystem.
Results: Theresultsshowthatthesensorsarerarelyaffectedbyenvironmentalfactors.Theyalsoshowthat thesensorscanfunctioncontinuouslyforabout14hourswithoutbatteryrecharge.
Conclusion: Finally,theseresultsallowedusto establishanoptimalset upconfigurationfor homebased rehabilitationusingseriousgames.
©2018AGBM.PublishedbyElsevierMassonSAS.Allrightsreserved.
* Correspondingauthor.
E-mailaddress:tien-tuan.dao@utc.fr(T.T. Dao).
1. Introduction
Physicalrehabilitationisalongprocessthat requirestheinter- ventionofaspecificteamofexperts[1].Usually,apatient under- going thisprocess will perform some sessions atthe clinic, with https://doi.org/10.1016/j.irbm.2018.10.014
1959-0318/©2018AGBM.PublishedbyElsevierMassonSAS.Allrightsreserved.
JID:IRBM AID:525 /FLA [m5G; v1.246; Prn:19/10/2018; 14:38] P.2 (1-5)
2 H. Tannous et al. / IRBM•••(••••)•••–•••
expert supervision, and will be required later to do some exer- cisesathometoremainactivebetweenclinicalsessions.However, therearenocurrentsolutionsforexpertstomonitorpatientmove- ments while performing exercises at home. In addition, patients drop home sessions due to the lack ofmotivation, and the high repetitiveness of these assigned exercises. Recently, these chal- lenges have been the center of interest for many engineers and scientists,whousedseriousgamesasacomplimentarytoolforre- habilitation.
Researchers have implemented serious games for different types of pathologies. Parkinson’s disease (PD) is one of these pathologies. Yu et al.developed a real-time Parkinson’s rehabili- tationenvironmentusingavisual motioncapturesystem[2].The systemisimplementedinaclinicandrequiresthepatienttoreach andstepindifferentdirectionsandspeeds. Avirtual avatarmim- icsthe patient’s movementson the screen. However, the system wasnevertestedonPDpatients. Paraskevopoulosetal.alsostud- iedserious gamesforPD rehabilitation[3]. Theyproposeda new design guideline forPD rehabilitation games, anddeveloped two seriousgames.TheytestedthesegamesonfivePDpatients.Stroke rehabilitation is anotherinteresting pathology for serious games.
Zannathaetal.developedaseriousgameusingKinectcameraand EMG sensors [4]. The systemhas 4 games for upper limb reha- bilitation,butwasnottestedonstrokepatients.Someresearchers were interested in general health and wellbeing of the elderly.
Lozano-Quilis et al. developed an augmented reality system for multiple sclerosis using the Kinect camera [5]. The system has 3 different exercises, and was testedon 11 patients. The results showedthatthepatientsacceptedthesystemandfeltsafewhile executingtheexercises.
Implementinganysystemathomerequiresappropriatesensors tocapturemotiondata.Twotypesofsensorsaregenerallyusedfor thispurpose,visualmotioncapturesensors(e.g.Kinect) andiner- tialmeasurement units (IMU). The Kinect camera iswidely used in serious game rehabilitation since it presents a portable solu- tion with low cost [4,5]. However, Kinect has very low accuracy whenused toestimate jointangles.Previous studieshaveshown thatKinectcouldestimatekneeanglewithanerrorofabout14.5◦ [6–8]. This error is high when compared to the valuesaccepted by medical expertsto analyzejoint data (6◦ for higher extremi- ties[9] and 5.5◦ forlower extremities[10]).These problemscan be avoided by using IMU sensors that are able to estimate an- glesat ahigher degree ofprecision using complexmathematical filters[11,12].Moreover, newstudieshavestartedinvestigatingif a combinationofdifferentsensors could lead tohigher precision injointangleestimation.Atrsaei etal.studied afusionalgorithm using unscented Kalman [13]. The results showed that the new algorithm helpedimproving the position estimationof some up- per body segments but not the angle estimation. Glonek et al.
proposed analgorithm thataverages inputsfromKinectandIMU sensorstoestimatejointangles[14].Thestudywasvalidatedwith onesubjectperformingmultipleexercises.Otherstudieshavetried tousemultipleKinectcamerato capturehumanmovementsina livinglab[15].Inapreviouswork,wealsoproposedanewfusion algorithmbetweenKinectandIMUsensors,tobetterestimatethe kneeanglewhileperformingaseriousgamerehabilitationexercise [6]. However,studies proposingthe useofmultipletypesof sen- sors havenotinvestigated howto usethistechnologyina home based environment to monitor patients or execute rehabilitation programs. The addition of wireless sensors and the necessity to rechargethembetweensessions could influencethe useraccept- abilityforthese solutions.Thatis whywe studied,inthis paper, thebatteryandcurrentconsumption fortheShimmer3 IMU [16]
thatwillbeusedwithourpreviouslydevelopedseriousgamesys- tem[17–19].
Thepaperisorganizedasfollows.Section2presentsthestudies performed, which includes the battery and current consumption study ofthe IMU sensors. Section 3 highlights the resultsof the battery consumption.Section 4discusses theresults.Finally, Sec- tion5concludesthestudy.
2. Materialsandmethods 2.1. IMUstudy
This test was performed on Shimmer3 IMU sensors [16], to identify how many sessions a patient can perform, without rechargingthesensor.Ourchoiceofsensorisbasedonourprevi- oususeofShimmer3inadatafusionstudy,betweenIMUsensors andKinectcamera[6].Inthestudy,Shimmer3provedtobeavery accurate sensorfor jointangle estimation.The sensor contains a tri-axial accelerometer,gyroscope, andmagnetometer that areal- waysswitchedonduringourtests.Allthesesignalsareneededto achieve a more accurate estimation ofjoint angle [6]. The study canbe dividedintotwo parts:batterylife studyandcurrentcon- sumptionstudy.
2.1.1. IMUbatterylife
Thebatterylifestudyincludes3tests:1)Effectofcommunica- tiondistanceandsamplingrateonbatterylife;2)Effectofmotion onbatterylife;and3) Effectofmultisensorystreamingonbattery life.
The sensors are charged until their batteries are full. Then, the sensors are connectedto a developed application (using C#) that savestheir datato afile inreal-time.The applicationallows streamingdatafromonetosevendifferentsensors,usingtheBlue- tooth communicationprotocol. The objective is to determine the effectofdifferentconditionsonbatterylife. Notethatthesetests wereperformeduntilbatterydepletion.
2.1.2. IMUaveragecurrentconsumption
The current consumption study includes 5 tests: 1) Current consumption untilbatterydepletion at51.2Hz;2) Effectofcom- munication distance andsampling rate on current consumption;
3) Effectofmotiononcurrentconsumption;4)Effectofmultisen- sory streaming on currentconsumption; and5) Effect of placing sensorbehindhumanbodyoncurrentconsumption.
Forthesetests,thesensoristakenoutofitsbox,andtheelec- tronicchip ismodified to allow theuse ofa multimeter(Fig. 1).
Three trialsweredone foreach testtoensure thereproducibility.
Thesameapplicationdescribedaboveisusedtoconnectthesen- sor to the PC. The multimeter is connectedto the PC via a USB port, andan application allows us to save multimeter data to a file. The firsttest was performedto make surethat currentcon- sumptionishomogeneous foracertainamountoftime,whichcan allowustorecord multipletrialscontinuouslywithoutrecharging thesensor.Fortheothertests,thesensor(s)streamedfor10min- utesandthecurrentwascollectedfromthemultimeter.Notethat thetestthatrequiresmovingthesensorwasdonemanually.
2.1.3. IMUreal-timecurrentconsumption
This studyaimedto providea detaileddescriptionofthe cur- rent pattern in real time, when the sensor is streaming to the PC. For this reason, we adopted the same scheme described in paragraph2.2,butwereplacedthemultimeterwithadigitaloscil- loscope(10 mV/div).The currentwas recordedfor10minutesat differentstreamingsamplingrates.Thedelaybetweenthereceived packets was also calculated to better understandthe sending/re- ceptionmechanismputinplacebytheIMUmanufacturers.
Fig. 1.Electric scheme for current measurement.
Fig. 2.Battery depletion time for different distances at different sampling rates.
3. Results 3.1.Batterylife
Firstly,theeffectofcommunicationdistanceandsamplingrate on battery life was studied. The results of this studyare shown inFig.2.Theresultsobtainedwithtwodifferentdistances(10 cm and5 m)fromthePCwerecomparedtoresultsobtainedinapre- viousstudydoneonShimmer1[20].
Thesecond studyinvestigates theeffect ofmotion onbattery life.Sinceweneededtosimulatemotionforalongperiodoftime, we attached the sensors to a small electric fan, and let it run overnight.Thetestwasperformedat51.2Hz,asitisthesampling rateused in our previous studies to estimate body joint angles.
This sampling rateproved to be sufficient enough to access the movement of the human body, which varies generally between 1–10 Hz in frequency [21]. The results show that when attach- ingthe sensorto anelectricfan overnight at51.2Hz,thesensor streamedfor14.88 h,comparedto14.9 hforastaticsensor.
Thethirdstudyshowstheeffectofmultisensory streamingon batterylife. Thetestwas performedwith7sensors versus1sen- sor,streamingcontinuouslyat51.2 Hz,untilbatterydepletion.The results show that when connecting 7 sensors at the same time at51.2 Hz, they streamed fora mean battery depletion time of 14.71 hcomparedto14.9 hwhenusingonesensor.
Fig. 3.Current dissipated during the test until battery depletion at 51.2 Hz.
Fig. 4.Average current for different distances at different sampling rates.
3.2. Averagecurrentconsumption
The first current consumption study investigated the changes intheaveragecurrentconsumed,forevery10 min,inordertofig- ureoutifthisaveragechangesovertime.Iftheaverageisconstant duringaperiodoftime,themeasuredcurrentaveragescanbere- liable, andwe will avoidthe need to rechargethe sensorbefore each currentconsumption test. The resultofthisstudyis shown inFig.3.
The effect of the distance andsampling rate on average cur- rent consumption is shown in Fig. 4. The mean average current consumedpresents themeanof 3trials of10minutes each.The averageiscomputedthroughcomputingthemeanoftheaverage 10minutecurrentconsumedover3trials.
When movingthe sensorfor10minuteswe obtaineda mean averagecurrentequalto 34.8±4.46 mA versus34.63±4.52 mA forastaticsensor.
Multisensory streaming was studied for 2 different sampling rates51.2Hz(thesamplingrateofinterest)and256Hz(thehigh- estsamplingrate).Threetrialsweretestedforeachsamplingrate.
At 51.2 Hz,when streaming 1 sensor we obtained a mean aver- agecurrentof34.63±4.52 mAvs34.48±2.41 mAfor7sensors.
At256Hz,whenstreaming1sensorweobtainedameanaverage currentof38.55±4.49 mAvs34.48±1.92 mA for7sensors.Fi- nally,whenplacingthesensorbehindahumanbodywithrespect to theremote central node, weobtained amean averagecurrent of35.14±4.62 mAvs34.8±4.52 mAforasensorplacednextto thePC.
JID:IRBM AID:525 /FLA [m5G; v1.246; Prn:19/10/2018; 14:38] P.4 (1-5)
4 H. Tannous et al. / IRBM•••(••••)•••–•••
Fig. 5.Real-time dissipated current during streaming at 51.2 Hz.
Fig. 6.Delay between samples received by the PC while streaming at 51.2 Hz.
3.3. Real-timecurrentconsumption
The real-time current dissipated during streaming at 51.2 Hz wasrecordedusingadigitaloscilloscope.Theresultsarepresented inFig.5.Thefigureshowsthatthereareperiodicallypeaksofcur- rentconsumption, andalmost a constant current during therest ofthetime.Theperiodofthesepeaksisabout40 ms.Othertests atdifferentsamplingratesshowedno differenceintheseperiods ofpeaks,andonlyaslightchangeintheaverageconstantcurrent consumed.Wealsomeasuredthedelaysbetweensamplesreceived bythePC,presentedinFig.6.
4. Discussion
Thebatterylifeandcurrentconsumptionstudyshowsthatthe usageconditionsrarelyaffectShimmer3sensorefficiency.Thefirst battery life test showed that the distance andthe sampling rate donotaffectbatterylife.Fig.2showsthat Shimmer1batterylife is lower than that of Shimmer3 but variesin the same manner.
Shimmer 1.0 does not contain a magnetometer, and its battery hasacapacityof280 mA hversus 450 mA hinShimmer3,which could explaintheseresults. Thesecond batterylife test,concern- ing sensormovement showedno significantdifference in battery lifebetweenastaticandadynamic sensor.Thesameobservation was notedwhencomparing multisensorystreaming versus single sensorstreaming.
The current consumption seemed homogeneous for the first 8 h (Fig. 3) when we performed a test until battery depletion.
Thismeansthatwecantestcurrentconsumptionwithoutrecharg- ing the sensor after each trial.The average currentseems to be slightlyaffected by distance (Fig. 4). However, when it comes to sensor movement, multisensory streaming and on body stream- ing,thecurrentconsumptiondoesnotexhibitsignificantchanges.
Theseresultsshow thatShimmer3 sensorisnot affectedbyenvi-
Table 1
Optimalconditionsforhomebasedrehabilitationusingseriousgames.
Criteria Recommendations Purposes
Distance from the system Atleast2meters Optimalconditionfor Kinectjointestimation Room dimensions Atleast4×4 meters Patientmovementin
spaciousarea Room lighting Well-litroom Optimalconditionfor
Kinectjointestimation Clothe type Shortsandt-shirts
withwell-fittingsize
Maximalcontactbetween inertialsensorandthe body
Number of sensors Amaximumof7 sensors
Maximalnumberina Bluetoothpiconet Sensor battery Rechargeafter25
sessions
Batterydepletionafter14 hoursofusage
ronmentalfactors.Theonlyconstraintofthesystemisamaximum numberof7sensors.
The third studywasto investigatethereal-time currentdissi- pated during streaming.The Fig.5showsthat thereare peaksof currentconsumptionattimes.Sincethefrequencyofcurrentcon- sumptionpeaksdidnot changewithdifferentsamplingrates,we proposedahypothesisthat thesensorsavesthesamplesrecorded ataparticularsamplingrate,andthen allocatesperiodsof20 ms tosendallthesaveddata.Thus,withhighersamplingrates,there is no increase in the number of peaks, but an increase in the meancurrentdissipated,whichcausesthebatterytodepletemuch faster.Toconfirmthishypothesis,wemeasuredthedelaybetween thereceivedsamplesbythePC,presentedinFig.6.Theresultsof this study confirmed our hypothesis, as there are some samples that are receivedwithbig delays. Thesedelayshappenwhen the sensorsends asample attheendofasending window,andthen sendsthenextoneinthenextsendingwindow.Afterthesepeaks, the delays get smaller asthe sensor sends the samples continu- ously.
Therefore based onthese results,andoptimal conditionspro- posed by the Kinect manufacturers, we can propose an optimal configurationforahomebasedseriousgamesystem(Table1).We recommendaminimaldistanceof2metersbetweentheuserand the visual sensor, to obtain an optimaljoint recognition through the Kinectbody jointestimation, thisrequiresa room ofatleast 4×4 meters, where the furniture is not allowed between the player andthe sensors. Moreover, the visual sensor worksin an optimalconditionwhentheroomiswelllit.Whenitcomestoin- ertial sensors, thetype of clothes wornby the useraffects them generally.Thatiswhywerecommend thattheuserswearsshorts andt-shirts,orclothesthatarenotlargerthantheirsize,inorder to maximize the contactbetweenthe sensorand thebody joint.
TheserecommendationsarepresentedinTable1.
5. Conclusion
In this paper, we presented a study of the energy consump- tionofShimmer3inertialsensors.Thistechnicalstudyshowsthat shimmer sensors can hold up to 14 hours when streaming con- tinuously at 51.2 Hz. The study also investigated how the cur- rent is consumed in real-time, which could help us understand howtooptimizetheuseofthesesensors.Inparticular,thisstudy showed that Shimmer3 sensors are not affected by environmen- tal factors,andthus canbeusedwithout anylimitingconditions.
The user should only charge the batteryonce after about25 re- habilitation sessions of 30mins each. Moreover, our system that uses the Kinect camera combined with inertial sensors has to take intoaccount thelimitationsofthe cameraalone.Finally,we gave some recommendationsonthe optimalroom andusercon- ditionstodeployseriousgamesathome.Infutureworks, wewill
