Brain computer Interfaces

INTRODUCTION The goal of human-computer interaction (HCI) is to improve the computer’s understanding of the user’s needs. With the development of HCI, people can send their commands to the computer via various channels, such as keyboards and mouses. However, all these channels rely on muscle move- ments, which may neither be easy to use for disabled peo- ple, nor be able to benefit healthy users when their hands are not free for the control. Brain-computer interfaces are systems that measure specific brain activities (e.g. attention level, motor imagery) and use them to build a direct com- munication between brains and computers [4]. They bring us the possibility of using our mind to control a computer without muscle movements, which may give birth to a revo- lution of HCI [2]. The traditional application of this technique mainly focuses on personal assistance for improving the HCI

This paper describes the OpenViBE software platform which enables to design, test and use Brain-Computer Interfaces. Brain-Computer Interfaces (BCI) are communication systems that enable users to send commands to computers only by means of brain activity. BCI are gaining interest among the Virtual Reality (VR) community since they have appeared as promis- ing interaction devices for Virtual Environments (VE). The key features of the platform are 1) a high modularity, 2) embedded tools for visualization and feedback based on VR and 3D displays, 3) BCI design made available to non-programmers thanks to visual programming and 4) various tools of- fered to the different types of users. The platform features are illustrated in this paper with two entertaining VR applications based on a BCI. In the first one, users can move a virtual ball by imagining hand movements, while in the second one, they can control a virtual spaceship using real or imagined foot movements. Online experiments with these applications together with the evaluation of the platform computational performances showed its suit- ability for the design of VR applications controlled with a BCI. OpenViBE is a free software distributed under an open-source license.

EEG signals can be represented in three different domains: in time (as they are acquired by the amplifier), in frequency (spectral power) and in space (topo- graphical layout on the scalp according to the electrode positions, as in Figure 3). Time-based representations are especially useful when dealing with event- related potentials (ERPs). ERPs come with time triggers (see 2.3), which al- low to extract epochs, which are time-windowed portions of signals. The EEG measures a mixture of activities coming from many regions of the brain, and although ERPs are stereotyped deflections, they are difficult to distinguish in a single epoch. Statistical procedures are called for to extract the significant deflections from several epochs (or trials). The simplest statistical procedure is cross-trial averaging: this decreases the amplitude of the background oscil- lations, whose phase is not consistent across the trials. For Brain Computer Interfaces where the analysis must be done in real time, there are specific tech- niques for single-trial detection of ERPs (see 3.2.2).

RÉSUMÉ Les interfaces cerveau machine (BCI pour Brain Computer Interfaces) non invasives permettent à leur utilisateur de contrôler une machine par la pensée. Ce dernier doit porter un dispositif d’acquisition de signaux électroencéphalo- graphiques (EEG), lesquels sont par la suite traités et transformés en commandes. Cependant, les signaux EEG sont dotés d’un rapport signal sur bruit très faible ; à ceci s’ajoute l’importante variabilité que l’on trouve tant à travers les sessions d’utilisation qu’à travers les utilisateurs. Par conséquent, la calibration du BCI, pendant laquelle l’utilisateur est amené à effectuer une tâche prédéfinie, doit sou- vent précéder son utilisation. Le sujet de cette thèse est l’étude des sources de cette variabilité, dans le but d’explorer, concevoir, et implémenter des méthodes d’autocalibration. Nous nous intéressons en particulier aux interfaces cerveau machine qui utilisent des potentiels évoqués comme marqueur neurophysiologique (ERP-based BCI), que nous introduisons dans la première partie.

Brain Computer Interfaces (BCI) provide a way of communi- cating directly from brain activity, bypassing muscular control. We report some recent advances in a BCI communication sys- tem called the P300 speller, which is a virtual brain-operated keyboard. This system relies on electroencephalographic acti- vity time-locked to the flashing of the desired letters. It requires calibration of the system, but very little training from the user. Clinical tests are being conducted on a target population of pa- tients suffering from Amyotrophic Lateral Sclerosis, in order to confirm the usability of the P300 speller for reliable communi- cation.

Chapter 2. Brain-Computer Interfaces: Recent Advances and Open Challenges 27 reported a passive system to identify images of interest presented to the subjects by analysing EEG and pupil dilation activity. Promising applications for BCIs in general, both in medical and non-medical fields, were highlighted in previous reviews [3, 174–177]. Most of the medical applications mentioned in these reviews were the focus of some of the selected articles: assistive technologies such as spellers, navigation, user interface control, FES device control for grasping and rehabilitation. However, in the case of non-medical applications, there is still room for innovation: for instance, user-state monitoring, as made possible by passive BCIs, was mentioned as a promising application for BCIs in [3, 175] but was the focus of only one of the selected articles [170]. This type of application is better suited to non-medical users since it is not in competition with standard user interface technologies. Indeed, active and reactive BCIs cannot achieve the same communication bandwidth as a mouse or keyboard, at least as of now, and thus healthy users are less likely to adopt these as standard input devices. Passive BCIs rather provide an additional input related to a user’s affect, mental workload, fatigue or attention. Moreover, hybrid BCIs can readily benefit from additional modalities typically used in affect recognition such as SCR and PPG. Additionally, gaming and entertainment applications are highly promising, as these industries are often early adopters of new technology [3]. Hybrid BCIs are particularly relevant for this type of application: mixed hBCIs combining traditional controllers and brain-driven inputs can naturally enhance a gamer’s experience without suffering from the bandwidth bottleneck of BCIs used for direct control. Other applications that were not mentioned in the selected articles include applications in training and education, cognitive enhancement, neuroergonomics and neurofeedback [3].

Uncued brain-computer interfaces: a variational hidden markov model of mental state dynamics.. Cedric Gouy-Pailler, Jérémie Mattout, Marco Congedo, Christian Jutten.[r]

Anton Nijholt (Univ. Twente, The Netherlands, a.nijholt@utwente.nl ) Brain-Computer Interfaces (BCIs) are systems that translate a measure of a user‘s brain activity into messages or commands for an interactive application. A typical example of a BCI is a system that enables a user to move a ball on a computer screen towards the left or towards the right, by imagining left or right hand movement respectively. The very term BCI was coined in the 70’s, and since then, interest and research efforts in BCIs grew tremendously, with possibly hundreds of laboratories around the world studying this topic. This has resulted in a very large number of paradigms, methods,

Towards a user-centred methodological framework for the design and evaluation of applications combining brain-computer interfaces and virtual environments: contributions of ergonomics Résumé : Ce rapport présente un cadre méthodologique centré sur l’utilisateur visant à assister la conception et l’évaluation d’applications combinant les Interfaces Cerveau-Ordinateur (ICO) et les Environnements Virtuels (EV) afin qu’elles soient adaptées aux utilisateurs finaux. Sur la base de connaissances issues de l’ergonomie, ce cadre fournit des méthodes, des critères et des métriques permettant de réaliser les étapes du processus de conception centrée- utilisateur , étapes visant à comprendre le contexte d’utilisation, spécifier les besoins des utilisateurs et évaluer les solutions développées. En l’occurrence, plusieurs méthodes ergonomiques (e.g., entretiens, études longitudinales, évaluations avec des utilisateurs), métriques objectifs (e.g., réussite de la tâche, nomb re d’erreurs) et métriques subjectifs (e.g., note associée à un item) sont suggérés pour définir et mesurer l’utilité, l’utilisabilité, l’acceptabilité, les qualités hédoniques et affectives, l’expérience utilisateur, l’immersion et la présence associées à l’interaction avec ces applications. Les bénéfices et les contributions de notre cadre méthodologique pour la conception ergonomique des applications combinant ICO et EV sont ensuite discutés.

The probabilities of channels being selected are shown in Fig. 6 . The red areas indicate the important brain areas where the channels are often selected. We also marked out the key channels with the selection probabilities above 80 %. The similarity is shown between CSTI and CSL, although there are more key channels when using CSTI. For most subjects (except subject ‘‘av’’), the key channels are distributed over the hand representative area of the sensorimotor cortex. Motor imagery of the right hand typically elicits strong ERD in the hand representative area of the sensorimotor cortex of the left brain (see Fig. 7 ). Nevertheless, for some subjects (e.g., subjects ‘‘al’’ and ‘‘aw’’), the key electrodes are also found over the right hemisphere (see Fig. 6 ). The reason is that motor imagery can also cause an ERS in a ‘‘non-active’’ area [ 24 ]. For example, performing a foot motor imagery can generate an ERS in the hand representation area (see Fig. 7 ). The ERS can also contribute to classification [ 25 ]. Channels in central, frontal and occipital cortices are with very low selection probabilities, indicating that those areas are less important for distinguishing motor imagery of foot and hand. This result implies the possibility of using a part instead of all of the electrodes in an EEG cap to find the optimal subset of channel.

Adaptive classifiers can employ both supervised and unsupervised adaptation, i.e. with or without knowledge of the true class labels of the incoming data, respectively. With supervised a[r]

To do so, a variety of machine learning methods have been proposed, among which the most efficient ones include Riemannian Geometry-based Classifiers (RGC) and adaptive classifiers [2]. RGC represent EEG signals as covariance matrices, and can classify such matrices based on dedicated distance measures between them, known as Riemannian dis- tances, see [3], [4] for reviews. RGC have been shown to be very effective, due to their affine invariance properties, their formulation removing the need for a separate spatial filter optimization, and their ability to be calibrated with little data [4]. RGC were actually used to win several international brain signal classification competitions [2], [3].

Another system proposing to manipulate a robot was designed by Petit et al. in a similar setup where the robot would see the user and interact with her with the help of AR markers [ Petit et al. , 2014 ]. Users were wearing a Head-Mounted Display (HMD) as well as an EEG cap fitted with elec- trodes. Tasks ranged from navigation to touching a marker placed on the user’s arm. Instead of a robot, a robotic arm has been used by Martens et al., driven thanks to the SSVEP paradigm. The robotic arm was used to insert a key in a keyhole, in order to open a door [ Martens et al. , 2012 ]. Their setup also included the P300 paradigm, where users had to manipulate a robotic arm to move objects on a field. This manipulation was achieved using a P300 speller grid to select both the ob- ject to move and the target position. Finally, Blum et al. have studied the use of a BCI and SSVEP combined with a gaze-tracker and AR to pilot an X-ray device [ Blum et al. , 2012b ]. Their objective was to allow physicians to use the X-ray device without having to use their hands. The pilot study performed by Blum et al. included the BCI device, but the brain activity data was not recorded.

4 DOC Neurophysiological Assessment at FSL The stability of Event-Related Potentials (ERPs) is essential for ef ficient and effective ERP-based BCI systems, especially when BCIs is applied in a challenging clinical condition such as DOC. In this regard, there are several factors that can limit (if not prevent) the use of BCI technology in patients diagnosed with DOC such as fluctuations of vigilance, attention span and abnormal brain activity due to brain damage (Giacino et al. [ 21 ]) to name few. In a recent study conducted at Fondazione Santa Lucia (Rome), Aric ò and colleagues [ 14 ] showed a signi ficant correlation between the magnitude of the jitter in P300 latency and the performance achieved by healthy subjects in controlling a visual covert attention P300-based BCI. In particular, the higher the P300 latency jitter, the lower the BCI accuracy. We speculated that the covert attention modality increases the variability of the time needed to perceive and categorize the visual stimuli.

BCI competition II dataset IV [ 32 ]: It contains data from a single subject with two finger movements (fingers from left and right hands), recorded using 28 EEG chan- nels, which have been used in [ 110 ] as experimental data for reducing the number of electrodes. This dataset was recorded from a normal subject during a no-feedback session. The real finger movements are performed in a self-chosen order and a self- paced timing, thus it poses the challenge of algorithm design for self-paced BCI systems. Meanwhile, as finger movements unavoidably cause artifacts to EEG sig- nals, this dataset can also be used to test the robustness of a method to noise. BCI competition III dataset I [ 96 ]: This dataset contains two-class cue-driven motor imagery (left small finger v.s. tongue) data from a single subject (epileptic patient) but with 64 ECoG channels, which is an invasive BCI system. ECoG data usually have higher SNR than scalp EEG data. As the training and testing data are recorded in different days, this dataset poses the challenge of using a classifier that was trained on the first day to classify the data recorded during the following days (without retraining). It is a common but tough challenge due to at least two reasons: 1) the patient might be in a different state concerning motivation, fatigue, etc., so that his brain will show a different electrical activity, 2) the recording system might have undergone slight changes concerning electrode positions and impedances. Thus, this dataset can be used to test the robustness of a method to data evolution for a long- term-use invasive BCI system. This dataset has also served as experimental data with BCI competition II dataset IV in [ 110 ] for electrode reduction.

4.1.3 Experimental design Participants were sitting in front of a computer screen, wearing an EEG headset (see Figure 4.1). Three flickering square targets were displayed on screen. For each trial, the task consisted in selecting one target (indicated with an arrow), by simply focusing on it without looking away during 4 seconds. A fake feedback based on visual and auditory cues was provided at the end of each trial, indicating either a success or a failure. The visual feedback was a single word displayed at the center of the screen (success; failure). The auditory feedback was either a buzzer sound (failure) or a game-like reward sound (success). Additionally, a vertical progress bar was displayed on the right of the screen. This progress bar grew for each success, and stayed at the same level in case of failure. In a controlled laboratory experiment, the number of variable factors needs to be kept to a minimum, in order to avoid potential uncontrolled bias. However, this constraint makes the task seem artificial, and the participants have little incentive to success, compared to a normal use of an interface. Pre-experiments revealed that without an appropriate feedback, participants quickly lose interest in the task. Our choice of a rewarding or punishing feedback aims at maintaining the user engagement, and emulating the cost of errors encountered in real use.

One can observe that graph analysis gives noisier results. The main cause is the use of correlation measures, which thus have to be replaced by a more stable one. The method using graph analysis shows similar results in comparison to a simpler method based on the power spectrum of time series. The use of connectivity graphs has already shown to be of great interest in neuroscience to explore the role of brain regions at rest [1]. In the context of BCI, we have shown that a simple, univariate method based on the power spectrum of time series is also able to analyse signals from an experiment based on auditory attention. However, the use of a multivariate method such as graph connectivity in this context is also motivated by the possibility of improvements for further experiments like motor imagery. The connectivity graphs will allow us to select the most relevant sensors to take into account in the classification, for example in looking at category of nodes like hubs. Finally, the method using connectivity graphs will allow us to use more specialized characteristics (such as clustering, modularity. . . ) in adequation with the experiments and nature of the data. Lastly, source separation could be able to extract the different dynamical causes of an observed graph. This will be done by considering a graph as a linear mixture of statistically independent graphs.

To do so, a variety of machine learning methods have been proposed, among which the most efficient ones include Riemannian Geometry-based Classifiers (RGC) and adaptive classifiers [2]. RGC represent EEG signals as covariance matrices, and can classify such matrices based on dedicated distance measures between them, known as Riemannian dis- tances, see [3], [4] for reviews. RGC have been shown to be very effective, due to their affine invariance properties, their formulation removing the need for a separate spatial filter optimization, and their ability to be calibrated with little data [4]. RGC were actually used to win several international brain signal classification competitions [2], [3].

interface that enables the direct communication between human and computers through decoding of brain activity. As such, event-related potentials (ERPs) like the P300 can be obtained with an oddball paradigm whose targets are selected by the user. This paper deals with methods to reduce the needed set of EEG sensors in the P300 speller application. A reduced number of sensors yields more comfort for the user, decreases installation time duration, may substantially reduce the financial cost of the BCI setup and may reduce the power consumption for wireless EEG caps. Our new approach to select relevant sensors is based on backward elimination using a cost function based on the signal to signal-plus-noise ratio, after some spatial filtering. We show that this cost function select sensors subsets that provide a better accuracy in the speller recognition rate during the test sessions than selected subsets based on classification accuracy. We validate our selection strategy on data from 20 healthy subjects.

Extending Riemannian Brain-Computer Interface to Functional Connectivity Estimators Sylvain Chevallier 1 , Marie-Constance Corsi 2 , Florian Yger 3 and Camille Noˆus 4 Abstract— This abstract describes a novel approach for handling brain-computer interfaces (BCI), that could be used for robotic applications. State-of-the-art approaches rely on the classification of covariance matrices in the manifold of symmetric positive-definite matrices. Functional connectivity estimators have demonstrated their reliability and are good candidates to improve the classification accuracy of covariance- based methods. This abstract explores possible application of functional connectivity in Riemannian BCI.