HAL Id: hal-01795263
https://hal.archives-ouvertes.fr/hal-01795263
Preprint submitted on 18 May 2018
HAL
is a multi-disciplinary open access archive for the deposit and dissemination of sci- entific research documents, whether they are pub- lished or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.
L’archive ouverte pluridisciplinaire
HAL, estdestinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.
On Improving Motor Imagery BCI Using A Novel Physical Feedback Technique
Mahmoud Haroun, M Salaheldeen
To cite this version:
Mahmoud Haroun, M Salaheldeen. On Improving Motor Imagery BCI Using A Novel Physical Feed-
back Technique. 2018. �hal-01795263�
On Improving Motor Imagery BCI Using A Novel Physical Feedback Technique
Haroun, M. A. SalahEldeen, M.A.*,
* Science, STEM Egypt School
Abstract- In this study, and through an understanding of neuronal system communication, A light was shed upon a novel model serves as an assistive technology for locked-in people suffering from Motor neuronal disease. MND patients, who have an intact minds, are able -till now- to make a mean of on- screen communication. Work was done upon the potential of brain wave activity patterns to be detected as electrical signals, classified and translated into commands following BCI constructing paradigm. However, The interface constructed was for the first time a device which can reconstruct this command physically. The intervention is in a software is used to classify imagined right and left limb movement into binary results, and then make use of the magnetic field produced by the amplified pulses to reconstruct the activity into ferrofluid droplets movement- these moved due rotation of a desk according to the data received.
Objectives of the project is to address the challenges of the inaccurate performance and cost of user-training which are yet the main issues preventing BCI from being upgraded into more immersive and effective technology. The design requirements addressed were the communication speed of the droplet movement and the accuracy of hitting fixed targets . Tests were performed based on software programs accelerated changing motor imagery movement of left and right limbs. The tests achieved an average speed of 0.469 cm/s and average accuracy of 81.6% for the best volume from many were tried. A conclusion to be drawn was that the promise of this other point of view on BCI systems to be more Brain-Real World Systems.
INTRODUCTION
WHO (World Health Organization), about 14% of world’s population have disabilities with 2-4% experience significant difficulties in functioning.
Correspondingly, Egypt contain about 2 million people with disabilities representing about 3.5% of the total population; a statistic made in 1997 by central Authority for public Mobilization and Statistics. The statistic revel that about 15% of people with disabilities in Egypt have Mobility Impairment.
Although there is currently no cure, it is not true to say that, ‘nothing can be done for the person with motor neuronal disease'. A great deal can be done to maintain quality of life through assistive communication technologies. Brain computer interfaces or BCIs are now the tool for MND patients. Depending on understanding of how the brain works, A BCI reads the waves produced from the brain at different locations in the human head,
Classifies the signals in program, and generates commands that can control the computer.
While the importance of subject's motor imagery skills in BCIs is well recognized, most studies have focused on the computer side and improving classification algorithms and very few have attended the human side and training paradigms that can facilitate the skill acquisition process for subjects. Powerful machine-learning algorithms have been created. On the other hand, less research has focused on investigating how BCI users may optimally adapt. This Project Focuses on the latter aspect with totally new intervention into the system. More precisely, The goal of this project is to explore the influence of novel intervention of physical feedback design on enhancement of user's performance and interaction with a BCI system.
ARCHITECTURE OF BCISYSTEM
the BCI system generally performs a six step process (Figure 1): brain activity measurement, preprocessing, feature extraction, classification, command translation, and feedback .
EEG Data Acquisition: Brain activity measurement is the step in which electrodes are used to obtain the user’s EEG at specific regions on the scalp, to form input for the BCI system. This step involves determining the number and location of the channels, amplification, analogue filtering, and A/D conversion. Channel locations are selected according to the paradigm used and mental task performed.
Most people exhibit prominent oscilation in the 8-12 Hz band of the EEG recorded ove sensomotor areas . When they are not actively enganged in motor action, sensory processing or imagination of actions. This oscilation is usually called mu rhythm and is thought to be produced by thalamocortical circuits. Analysis of mu rhythm showed that it also usually associated with 18-25 Hz beta rythms. While some of the beta rythms are harmonics of the mu rhythm, some are separated by topography or timing from mu rhythm.
In fact,BCIs use the different types of paradigms to operate like Visually Evoked Potentials, Event Related Potentials, Readiness Potentials and Motor Imagery.
Here we consider MI detection. What makes these neural signal patterns important to Human computer interfacing studies is that the mu and beta frequency drop occurs even when the planned movement is not executed—that is, the signals are still observable in paralyzed patients like those with advanced ALS .Thereby, MI BCI can provide direct communication without any limb movement or external stimulus. MI BCI uses “induced” brain activity from the cortex, rather than
“evoked” brain activity.
Figure 1 : EEG BCI paradigm steps
Figure 2 : Figure 6 : Brain Functional regions
The mu wave is found over the motor cortex, in a band approximately from ear to ear. A person suppresses mu wave patterns when he or she performs a motor action or, with practice, when he or she visualizes performing a motor action. This suppression is called desynchronization of the wave because EEG wave forms are caused by large numbers of neurons firing in synchrony. The electroencephalographic (EEG) mu rhythm is an 8-13 Hz rhythm generated by the sensorimotor cortex. Motor imagery implies a thought activity of imagining of physical movement. In other words, when imagining a hand movement or actually moving the hand, ERD (Event-Related Desynchronization) occurs around the μ- rhythm area within the sensory motor cortex (see figure 2 )
The mu wave is hypothesized to represent an “idling” rhythm of motor cortex that is interrupted when movement occurs. The free running EEG shows characteristic changes in mu activity, which are unique for the movement of different limbs.
These findings have and will -for our work- be useful in the construction of BCI systems. That is shown through (fig. 3) which shows basic phenomenon of mu/beta rhythm modulations in EEG while imagination of right hand movement.
Brain activity is picked by one of different types of electrodes. In their most simple method, they are placed on the scalp using gel as a conducting material. The placement of electrodes commonly follows the 10–20 system
(2) Preprocessing
After the signals being detected by the headset, normal BCI systems include g.USBamp, It is a bio signal acquisition system with 24-bit resolution that provides signal-conditioning functionality to amplify, filter, and convert electrode outputs into digital values. Its user-selectable, multichannel modules allow the simultaneous recording of electro -encephalogram (EEG) data.
(3) Feature Extraction and Classification
The brain patterns used in BCIs are characterized by certain features or properties.
For instance, amplitudes and frequencies are essential features of sensorimotor rhythms. The firing rate of individual neurons is an important feature
of invasive BCIs using intracortical recordings. The feature extraction algorithms of a BCI calculate (extract) these features. extraction is the process of distinguishing the pertinent signal characteristics from extraneous content and representing them in a compact and/or meaningful form, amenable to interpretation by a human or computer. Classification is a step which assigns a class label to a set of features extracted from the signal. In other words it is about assigning a command to a specific pattern of the EEG. The two process are usually integrated in one “processing program” in normal BCI systems
(5) Visualization and command representation is the step of exhibiting the command. It is the step which understood to be revealing the intention of the brain.
The step has always an on-screen visualization in normal BCI systems.
THE PHYSICAL FEEDBACK TECHNIQUE
In the seek of achieving a physical visualization system, Our system undertake a modifications on the normal system steps. Firstly, and as hinted in the step of Acquisition, we work on mu/beta rhythm while subject in motor imagery activity.
Here, we adopt two different positions to detect signals from, namely the C3 (Left hemisphere) and C4 (Right hemisphere) in the 10/20 system. Although the normal use is a 32-channel of positions, the promise of the only two positions mentioned and the differ in the following steps made our modification reliable and achieving a simplicity would contribute to the final outcome in terms of speed and accuracy.
Fig 5 shows the notable attenuation while in motor imagery .
In the following, mention our novel system to be integrated with the normal BCS systems (or with the mentioned modification. To achieve the reliable physical visualization, 3 major concepts in electronics, physics and chemistry was integrated in an intellectual unification paradigm. Those are:
A) Classification Software
This attenuation in mu rhythm which is our feature to be extracted was induced by a simulation of a software program named BCI 2000 as there were two electrodes in the C3 position in the two hemispheres. the classification process begins when this attenuation is detected and then sampled by a MATLAB program. It works as follows: within the range of attenuation the higher of the two will be sampled in a two bit sampled as 1 or 0 (this is because even if the activity is centered in one hemisphere , the other will have other – but much lower -one ) . Those 4 outcomes which established from the 2 bit sampling was used to produce the current and the then the movement based upon to achieve the simulation of the imagined movement.
B) Arduino Board and Electromagnet
The 0/1 binaries gathered from the BCI simulation system was transformed to an Arduino controlling system. The Arduino was used to induce current flowing through a solenoid or electromagnet. That was based upon Ampère's circuital law that an induced electric field could establish a magnetic field and vice versa. The electromagnet dimensions was estimated as follows: it will be like a solenoid having the length of an iron nail (10cm) with radius of 1cm. According to Elementary physics equation which describe the intensity of the magnetic field around the solenoid,
Figure 3: mu rythem attenuation
Figure 4: 10/20 positioning system
Figure 5: C3 C4 attenuation in Motor Imagery
C) Ferro Fluid Droplet Movement :
The movement of the ferrofluid movement is based – as mentioned before- upon the induced current in electromagnet. The idea is that this electromagnet is movable in a circular path based upon a motor and gear . The angle of rotation depends on the integration of the data at the Arduino. Algorithm was used to achieve the following table paradigm.
So, now if patient thought about his right hand movement, his left hemisphere gives higher attenuation and is then 1 in the binary code and left hemisphere is 0. this makes the electromagnet rotates 180 and we know that he thought to move his right hand. Similar approach, achieving 360 rotation is done for left hand imagining movement
TESTS AND RESULTS 1.Velocity
Velocity is measured by calculating the distance and dividing it by time taken by the droplet to move. Although a force affects the droplet and acceleration should be calculated, High Stroke force affect the motion decreasing the magnitude of the acceleration making it negligible. Data is Figure .. shows this relation such that velocity of the droplet is nearly negligible. For precise calculation for the time the droplet taken to move three trials for each distance was experimented and taken to average. The experiment was made for each of the three different volumes of the droplet ranging from 5 to 20 μL. The following data table no.2 show each trial corresponds to what distance and volume. There was a random error by ± 0.335 ( 1%). Measurement tool of distance was a roller and measurement tool of time was a stop watch
Expected Distance Trial 1 Trial 2 Trial 3 Average
Volume in μL
3 90.91 90.82 91.2 90.98
20
6 181.83 182.1 181.92 181.95
20
9 272.91 275.71 279.12 278.81
20
12 363.64 368.1 365.2 365.65
20
3 41.72 42.35 41.87 41.98
10
6 83.54 83.62 84.21 83.79
10
9 125.47 126.75 128.57 126.93
10
12 166.97 168.54 167.94 167.82
10
3 27.54 27.39 26.47 27.13
5
6 54.78 56.31 52.87 57.65
5
9 81.97 84.79 83.65 83.47
5
12 110.91 109.87 111.54 110.77
5
DATACOLLECTION
2.Accuracy
Accuracy is defined as the how much success does the droplet achieved. A simple ruler was used to determine the actual distance the droplet moved and then compared with the distance the droplet should move. The experiment is considered precise as even the friction is calculated in the stroke force as it defined by the viscosity of the droplet. The distance measurement error by ± 0.2 (20.5%). The measurement tool of distances was the roller. Table 4 shows the average of the many trial tests on accuracy
Binary Code Achieved
Angle Of Rotation Right
hand IM
0 From Right Hemisphere
1 From Left Hemisphere 180 Degree
Left Hand IM
1 From Right Hemisphere
0 From Left Hemisphere 360 Degree
Distance 20 μL 10 μL 5 μL 3 90.98 41.98 27.13 6 181.95 83.79 54.65 9 278.81 126.93 83.47 12 365.65 167.82 110.77
0 100 200 300 400 500
3 6 9 12
Average Time Taken (Seconds)
Distance Travelled (cm)
Velocity
Series1Series2 Series3 Linear (Series1) Linear (Series2) Linear (Series3)
0 50 100 150 200 250 300 350 400
0 5 10 15
Time in secoonds
Expected Distance in cm
Trial 1 Trial 2 Trial 3 Table 1 Figure 6: schematic of our device
Table 2
Graph 1
Graph 2 Table 3
MATERIALS AND METHOD
Formation of the Ferrofluid: (Fig 7)
.0 mL of 2M FeCl2 solution and 4.0 mL of 1M FeCl3 were added to ammonia solution with a concentration of 50 mL of 0.7 M. An exothermic reaction was observed that preaches for the formation of Magnetite, a black precipitate. At the end reaction, heat energy was added to the solution to get rid of excess ammonia.
The heating of the solution persisted for 3 hours. A clear black solution was
observed that sign for the successfulness of the chemical synthesis. The balanced equation of magnetite formation is as follows:
2FeCl3+ FeCl2 + 8NH3 + 4H2O → Fe3O4 + 8NH4Cl
The surfactant used in this synthesis is citric acid with a concentration of 2 mL (25%). The Ferrofluid was then heated again. After the second thermal accumulation, Kerosene was added to the solution as a carrier to the nanoparticles.
Constructing the controlling system (figure 8,9):
As shown in Fig 1 , The interface was designed at AutoCAD and – as shown in Fig 2- then printed by Ultimaker 2+ machine in STEM FabLab. The glass sheet was also shaped to a circular sphere. The radius of the internal circle was 6 cm. There was also another side-connected circle With radius of
7 cm.
Distance 5 μL 10 μL 20 μL
3 cm 80% 82% 69%
6 cm 81.67% 78% 59%
9 cm 81.48% 70% 62%
12 cm 83.61% 76% 55.00%
Average 0.816898 0.765278 0.613889
Time Trial 1 Trial 2 Trail 3 Average
Volume in μL
90.98 2.4 /3 2.5 2.3 2.43 5
181.95 5.1 /6 4.7 4.9 4.91 5
278.81 7.3 /9 7.6 7.1 7.33 5
365.65 9.8 /12 10.1 10.2 10.03 5
41.98 2.2 /3 2.3 2.9 2.4666667 10
83.79 4.4 /6 4.5 5.1 4.6666667 10
126.93 6.3 /9 6.7 5.9 6.3 10
167.82 8.9 /12 9.1 9.4 9.13 10
27.13 1.7 /3 2.1 2.4 2.0666667 20
57.65 3.3 /6 3.5 3.9 3.5666667 20
83.47 5.5 /9 5.1 6.2 5.60 20
110.77 6.4 /12 6.1 7.3 6.62 20
80%82% 81.67% 81.48% 83.61%
78%
70% 76%
69%
59% 62%
55.00%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
3 cm 6 cm 9 cm 12 cm
Accuracy in Percentage
Distance Travelled in cm 5 microL 10 micro L 20 micro L
Accuracy
Table 3
Graph 3
Table 4
Figure 8 : AutoCAD design
Figure 9 : Our final device
RECOMMENDATION & FURTHER RESEARCH
- One recommendation for further research in the field is to extend the use of our physical visualization system as a wider channel of communication. This can be done by shedding light upon new brain activity patterns in relation with new situations and actions.
Extending the system use can be achieved also after increasing the number of options that the interface can manage to exhibit. This can be done by adding more option for the classification and their corresponding command of different angle of rotation. Idea demonstration is shown in figure 11 .(where different colors correspond to different commands)
The tool can also be used for improvement of BCI – games as the tool is suitable to be used ,if other electromagnets were added and a study on the impact of each of which on the other, in games as it would be possible to reconstruct abstract 2D object as shown in figure 12.
This multi effect of different magnetic fields can be understood through Maxwell’s equations. In our case, we are changing magnetic fields slowly (compared to radio frequencies) thus the magneto-static equations are appropriate. These are
where B⃑ is the magnetic field [in Tesla], H⃗ is the magnetic intensity [Amperes/meter], j⃗ is the current density [A/m2], M⃗ is the material
magnetization [A/m], χ is the magnetic susceptibility, and μo = 4π × 10−7 N/A2
is the permeability of a vacuum. The force on a single superparamagnetic particle is then
When a magnetic force is applied, a single particle will accelerate in the direction of that force until it sees an equal and opposite fluid (Stokes) drag force. Since the Stokes force is
REFERENCES
[1] Probst, R., Lin, J., Komaee, A., Nacev, A., Cummins, Z., & Shapiro, B.
(2011). Planar steering of a single ferrofluid drop by optimal minimum power dynamic feedback control of four electromagnets at a distance. Journal of Magnetism and Magnetic Materials, 323(7), 885-896.
doi:10.1016/j.jmmm.2010.08.024
[2] Krahn, G. L. (2011). WHO World Report on Disability: A review. Disability and Health Journal,4(3), 141-142. doi:10.1016/j.dhjo.2011.05.001
[3] Scherer, C., & Neto, A. M. (2005). Ferrofluids: properties and applications.
Brazilian Journal of Physics, 35(3a), 718-727. doi:10.1590/s0103- 97332005000400018
[4] Reindl, M., Leschhorn, A., Lücke, M., & Odenbach, S. (2009). Flow control of magnetic fluids exposed to magnetic fields. Journal of Physics: Conference Series, 149, 012109. doi:10.1088/1742-6596/149/1/012109
[5] Proverbio, A. M., Riva, F., & Zani, A. (2009). Observation of Static Pictures of Dynamic Actions Enhances the Activity of Movement-Related Brain Areas.
PLoS ONE, 4(5). doi:10.1371/journal.pone.0005389
[6] Kaag, J. (2008). The neurological dynamics of the imagination.
Phenomenology and the Cognitive Sciences, 8(2), 183-204. doi:10.1007/s11097- 008-9106-2
[7] Malafronte, A., & Martins, M. (n.d.). Inexpensive Magnetic Field Controller. Proceedings of the 2005 Particle Accelerator Conference.
doi:10.1109/pac.2005.1591285
[8] Rajňák, M., Dolník, B., Kováč, J., Paulovičová, K., Mitróová, Z., Kurimský, J., . . . Timko, M. (2017). AC Magnetic Susceptibility of Ferrofluids Exposed to an External Electric Field. Acta Physica Polonica A, 131(4), 887-889.
doi:10.12693/aphyspola.131.887
[9] Liu, M., & Stierstadt, K. (2008). Thermodynamics, Electrodynamics, and Ferrofluid Dynamics. Lecture Notes in Physics Colloidal Magnetic Fluids, 1-74.
doi:10.1007/978-3-540-85387-9_2
[10] Conroy, D., & Matar, O. K. (2017). Dynamics and stability of three- dimensional ferrofluid films in a magnetic field. Journal of Engineering Mathematics, 107(1), 253-268. doi:10.1007/s10665-017-9938-2
[11] Moore, J., Middleton, D., Haggard, P., & Fletcher, P. (2012). Exploring implicit and explicit aspects of sense of agency. Consciousness and Cognition, 21(4), 1748-1753. doi:10.1016/j.concog.2012.10.005
[12] Kastner, S., & Scolari, M. (2012). Faculty of 1000 evaluation for Reconstructing visual experiences from brain activity evoked by natural movies. F1000 - Post-publication peer review of the biomedical literature.
doi:10.3410/f.13968963.15426068 Figure 10 : Increasing number of commands
Figure 11 : 2D possible designs
[13] Miller, E., & Buschman, T. (2012). Faculty of 1000 evaluation for A continuous semantic space describes the representation of thousands of object and action categories across the human brain. F1000 - Post-publication peer review of the biomedical literature. doi:10.3410/f.717968492.793468224 [14] Naselaris, T., Prenger, R. J., Kay, K. N., Oliver, M., & Gallant, J. L. (2009).
Bayesian Reconstruction of Natural Images from Human Brain Activity.
Neuron, 63(6), 902-915. doi:10.1016/j.neuron.2009.09.006
[15] Hyvärinen, A. (2009). Faculty of 1000 evaluation for Visual image reconstruction from human brain activity using a combination of multiscale local image decoders. F1000 - Post-publication peer review of the biomedical literature. doi:10.3410/f.1147979.605083
[16] Shapiro B. Towards Dynamic Control of Magnetic Fields to Focus Magnetic Carriers to Targets Deep Inside the Body. Journal of Magnetism and Magnetic Materials. 2009;321(10):1594–1599.
[17] Krusienski, D., Sellers, E., Mcfarland, D., Vaughan, T., & Wolpaw, J. (2008). Toward enhanced P300 speller performance. Journal of Neuroscience Methods, 167(1), 15-21. doi:10.1016/j.jneumeth.2007.07.017 [18] Liu, M., & Stierstadt, K. (2008). Thermodynamics, Electrodynamics, and Ferrofluid Dynamics. Lecture Notes in Physics Colloidal Magnetic Fluids, 1-74. doi:10.1007/978-3-540-85387-9_2
[19] Conroy, D., & Matar, O. K. (2017). Dynamics and stability of three- dimensional ferrofluid films in a magnetic field. Journal of Engineering Mathematics, 107(1), 253-268. doi:10.1007/s10665-017-9938-2
[20] Moore, J., Middleton, D., Haggard, P., & Fletcher, P. (2012).
Exploring implicit and explicit aspects of sense of agency. Consciousness and Cognition, 21(4), 1748-1753. doi:10.1016/j.concog.2012.10.005
[21] Kastner, S., & Scolari, M. (2012). Faculty of 1000 evaluation for Reconstructing visual experiences from brain activity evoked by natural movies. F1000 - Post-publication peer review of the biomedical literature.
doi:10.3410/f.13968963.15426068
[22] Miller, E., & Buschman, T. (2012). Faculty of 1000 evaluation for A continuous semantic space describes the representation of thousands of object and action categories across the human brain. F1000 - Post-publication peer review of the biomedical literature. doi:10.3410/f.717968492.793468224 [23] Naselaris, T., Prenger, R. J., Kay, K. N., Oliver, M., & Gallant, J. L.
(2009). Bayesian Reconstruction of Natural Images from Human Brain Activity. Neuron, 63(6), 902-915. doi:10.1016/j.neuron.2009.09.006 [24] Hyvärinen, A. (2009). Faculty of 1000 evaluation for Visual image reconstruction from human brain activity using a combination of multiscale local image decoders. F1000 - Post-publication peer review of the biomedical literature. doi:10.3410/f.1147979.605083
[25] Liang, L., Zhu, J., & Xuan, X. (2011). Three-dimensional diamagnetic particle deflection in ferrofluid microchannel flows. Biomicrofluidics, 5(3), 034110. doi:10.1063/1.3618737
[26] Odenbach, S. (2002). Ferrofluids: magnetically controllable fluids and their applications. Berlin: Springer.
GRUBLER, G. (2016). Brain-computer-interfaces in their ethical, social and cultural contexts. Place of publication not identified: SPRINGER.
[27] Zjajo, A. (2016). Brain-machine interface: circuits and systems.
Switzerland: Springer.
[28] Fleisch DA. A student’s guide to Maxwell’s equations. Cambridge, UK; New York: Cambridge University Press; 2008.
