state-of-the-art classification performances.
This paper introduces an online version of the minimum distance to Rie- mannian mean algorithm [ 34 ], with an application on Steady-State Visually Evoked Potential (SSVEP) recordings. In SSVEP, the subjects concentrate on stimuli blinking with fixed frequencies; depending on the focus of their attention, brain waves will arise with the same phase and frequency than the stimulus chosen by the subject. The proposed online implementation im- proves the performances in the BCI task, as the parameters of the classifier adapt to new, unlabeled samples. Such adaptation takes into account the qualitative changes in the brain responses and the acquired signals, caused respectively by internal and external factors, e.g. user fatigue and slight variations in experimental settings. The proposed online implementation is similar to the unsupervised or semi-unsupervised learning scheme proposed in [ 35 , 36 ]; that has the potential of shortening (or even removing) the cal- ibration phase. The proposed algorithm applies a similar approach to dy- namic stopping criterion used in [ 37 ] to increase the speed of the BCI system. This approach allows to dynamically determine the trial length and ensure robustness in classification results. When working with covariance matrices, a crucial point is to correctly estimate the covariance when the number of samples is small or heavily corrupted by noise. Several approaches have been proposed to build the covariance matrices, relying on normalization or regularization of the sample covariances. To assess the quality of the covari- ance matrices obtained from EEG samples, a comparative study of these estimators is conducted.
4 Cogitamus, CNRS, Paris, France
Marie-Constance Corsi 1 Florian Yger 2 Sylvain Chevallier 3
Controlling a brain-computerinterface (BCI) requires time to achieve high performance . Despite its clinical applications, one of the main drawbacks is the high inter-subject variability that could be noticed for performance. This is sometimes referred in the literature as the « BCI inefficiency » phenomenon  and affects its usability. Among the approaches adopted to tackle these issues are the search for neuromarkers, that potentially capture better the neurophysiological mechanisms underlying the BCI performance and the optimization of the classification pipelines, that could be robust enough to be applied to any subject.
Research Report n° 7954 Avrili 2012 18 pages
Abstract: Within the medical imaging community, 3D models of anatomical structures are now widely used in order to establish more accurate diagnoses than those based on 2D images. Many research works focus on an automatic process to build such 3D models. However automatic reconstruction induces many artifacts if the anatomical structure exhibits tortuous and thin parts (such as vascular networks) and the correction of these artifacts involves 3D-modeling skills and times that radiologists do not have. This article presents a semi-automatic approach to build a correct topology of vascular networks from 3D medical images. The user interface is based on sketching; user strokes both denes a command and the part of geometry where the command is applied to. Moreover the user-gesture speed is taken into account to adjust the command: a slow and precise gesture will correct a local part of the topology while a fast gesture will correct a larger part of the topology. Our system relies on an automatic segmentation that provides a initial guess that the user can interactively modify using the proposed set of commands. This allows to correct the anatomical aberrations or ambiguities that appear on the segmented model in a few strokes. Key-words: Sketching,3D Reconstruction, Model Annotation
Extending RiemannianBrain-ComputerInterface to Functional Connectivity Estimators
Sylvain Chevallier 1 , Marie-Constance Corsi 2 , Florian Yger 3 and Camille Noˆus 4
Abstract— This abstract describes a novel approach for handling brain-computer interfaces (BCI), that could be used for robotic applications. State-of-the-art approaches rely on the classification of covariance matrices in the manifold of symmetric positive-definite matrices. Functional connectivity estimators have demonstrated their reliability and are good candidates to improve the classification accuracy of covariance- based methods. This abstract explores possible application of functional connectivity in Riemannian BCI.
proposed to limit the number of necessary repetitions given a high prediction accuracy. The first way is to use a more complicated classifier than a simple average. For instance, Rakotomamonjy et al.  used as classifier an ensemble of several linear support vector machines (SVM)  with an au- tomatic channels selection, and Hoffmann et al.  proposed a boosting approach. Another way to improve the symbol prediction accuracy is to enhance the P300 evoked potentials by a spatial filtering of the channels. Several methods, based on independent component analysis (ICA) , , , were thus proposed to enhance the SNR and to remove the artifacts, e.g. , . However, the major drawback of such methods is that they are not specifically designed to separate brain waves. In most of BCI systems using ICA, after the decomposition in independent components (IC) it is necessary to select (manually or thanks to spatiotemporal prior) the ICs which mainly contained the desired evoked potentials.
weights to be applied to obtain the probability of each class.
The UDESC architecture efficiently combines the gen- eralization ability of individual Echo State Networks, the low computational load resulting from aggressive subsam- pling, the good performances of regularized logistic re- gression and the robustness of ensemble learning. UDESC was inspired by the well-known Random Forests  pro- posed by L. Breiman, which efficiently combine ensemble learning principles with discrimination based on decision trees. UDESC provides a subject-independent features construction method. Such methods might be of great interest to build subject-independent BCI systems. On the one hand it might allow a better control of the amount of data needed to calibrate a BCI. In classical methods the amount of data needed to train the system depends on the one needed to train the features construction method (e.g. linear spatial filters) and the one needed to train the classifier. The global amount of time thus equals the maximum of these two quantities. When subject- independent features construction methods are used, the amount of data needed simplifies to the amount of data needed to train the classifier. Using such algorithms might thus improves controlling the number of training trials. On the other hand the proposed approach might constitutes a first step toward the design of universal BCI systems. Subject-independent features construction methods indeed provide an invariant features representation. Future works might consider using a unique UDESC representation for each subject and combining the output linear functions as a database to be used by new subjects. Training a BCI system might then consist in selecting the adapted output vectors.
Using left-hand KMI versus right-hand KMI is very common in the MI-BCI field. Nevertheless, we can question whether these two KMI tasks are most relevant for applications in this area, especially concerning KMI-based BCI performance estimation. A KMI generates an activity over specific regions of the primary motor cortex within the contralateral hemisphere of the body part used in the process ( Pfurtscheller, 2001 ). Some BCIs are based on this contralateral activation to differentiate the cerebral activity generated by right-hand KMI from left-hand KMI. However, several studies have previously shown that some subjects have bilateral activity ( Hashimoto and Ushiba, 2013; Rimbert et al., 2017 ). For such subjects, BCI performance would remain low for a classification task between left-hand KMI and right-hand KMI. Subsequently, the good accuracy obtained for all subjects in our study, as well as the low number of subjects that could be considered as BCI-illiterate in our study (i.e., only 4 subjects), may be linked to our classification task choice (right- hand KMI vs resting state). KMI is a complex task that requires specific skills, sometimes even adapted training ( Jeunet et al., 2015, 2016 ). Moreover, performing KMI with the dominant hand is already not so easy for the subject. To include an additional KMI task involving the non-dominant hand maximizes that difficulty and could decrease BCI performance. This is not the case for the resting state, which is a more natural task. In addition, in the MIQ-RS questionnaire, the tasks to be performed by the subject are all directed toward the dominant hand. Finally, using a BCI based on right-hand and left-hand KMI to rehabilitate stroke patients is controversial, as one of the two hemispheres is often damaged.
Figure 2: The BCI session included 6 runs divided into two steps: (1) data acquisition to train the system (2 runs) and (2) user training (4 runs). After Run 2, the classifier is trained on the data acquired during the two first runs.
During each run, participants had to perform 40 trials (20 per MI-task, presented in a ran- dom order), each trial lasted 8s. At t = 0s, a cross was displayed on the screen. At t = 2s, an acoustic signal announced the appearance of a red arrow, which appeared one second later (at t = 3s) and remained displayed for 1.25s. The arrow pointed in the direction of the task to be performed, namely left or right to imagine a movement of the left or right hand. Participants are instructed to start performing the corresponding MI-task as soon as the arrow appeared, and to keep doing so until the cross disappeared. Finally, from t = 4.25s, a visual feedback was continuously provided in the shape of a blue bar, the length of which varied according to the BCI classifier output. Only positive feedback was displayed, i.e., the feedback was provided only when the instruction matched the recognized task. The feedback was provided for 3.75s and was updated at 16Hz, using a 1s sliding window. After 8 seconds, the screen turned black again until the beginning of the next trial. The participant could then rest for a few seconds. The timeline of a trial is shown in Figure 3.
4.3. Role of beta oscillations in BCI learning Prominent theories describing the neural processes that give rise to cognition and shape our behavior often involve integration of complex multimodal information using a combination of top-down pre- dictions (built from prior experience) and bottom- up, sensory-driven representations of the dynamic world around us [ 63 , 100 ]. These generalized frame- works, in turn, require the precise coordination of ensemble neural activity both within and between brain regions. Several theoretical approaches have examined how these two scales of functional activity may harmonize to produce the desired behavior [ 91 ], and empirical research has shown that there is con- sistent cross-talk between these scales [ 90 ]. Within human neuroimaging work, synchronous oscillations have been critical to understanding this complex coordination, where cortico-cortical propagation delays and membrane potentials give rise to observed oscillatory activity in the brain [ 10 , 96 ]. Here, we study the time varying connectivity within α, β, and γ bands. Much like how specialized functions arise from different brain regions, different narrowband oscilla- tions have been implicated in diverse but specialized processes, where some generalizable theories suggest a role for α in disengagement of task irrelevant areas or a lack of sensory processing [ 83 ], β in sustaining the current cognitive state [ 34 ], and γ in task-active local cortical computation [ 38 ]. Specifically in the context of motor imagery based BCIs, α and β bands have prominent signatures in motor imagery [ 74 ]. Our results show that only the β band’s functional connectivity is well suited to modulate patterns of activity that support sustained attention (not motor imagery), which is a critical process for BCI control. While our results are in line with generalized theories on the role of oscillations in cognition, the specificity of the β band in our results extends classic studies that discuss the role of this oscillation in attention [ 85 ] and in maintaining the current cognitive state [ 34 ]. Our results suggest that this maintenance, a consistent control (or attention to) internally generated activity, may play a crucial role in longterm BCI use.
the estimated trajectory smoothness [ Koyama et al., 2010b ]. The application of generative dynamic models is nevertheless more complex than the application of generic discriminative models. In particular, Kalman ﬁltering involves recurrent matrix inversions which are computationally expensive when neural features are high-dimensional. A supplementary dimensionality reduction step may thus be required for real-time application of the KF. Numerical issues due to roundoﬀ errors may additionally arise when a generic implementation of the KF is used [ Tusell, 2011 ]. The ability of generative models to handle (possibly) multi-limb asynchronous decoding was investigated in the present doctoral thesis. Switching generative models had been proﬁtably used for NC support in a simulated study [ Srinivasan et al., 2007 ]. Additionally, their application has been proposed to improve MUA/SUA decoding accuracy during reaching movements by combining trajectory models [ Yu et al., 2007 ], and to enhance kinematic reconstruction from SUA/MUA signals during IC states by using several emission models [ Wu et al., 2003b ] [ Wu et al., 2004 ]. In spite of these studies, the relevance of generative switching models on ECoG data had not yet been established, and their performance had not yet been compared with the one of their discriminative analogues. Despite the advantage conferred by explicitly modelling the kinematic dynamic during NC states, generative switching models here appeared as less eﬃcient that discriminative switching models for asynchronous ECoG decoding. This suggests that Kalman ﬁltering, which is increasingly popular for kinematic decoding, may suﬀer from diﬃculties to handle asynchronous decoding. Such diﬃculties were probably encountered in the MUA/SUA-based clinical trial completed by Hochberg and colleagues [ Hochberg et al., 2012 ], where it was reported that the KF used for kinematic decoding was re-initialized at the beginning of each trial. Kalman-based IC experts additionally yielded IC kinematic estimates which were either equivalent or less accurate than the ones output by the MSLM linear experts.
1. Addressing the aroused hope
Indeed, BCI studies often suggest real prospects for improvement among heavily handicapped patients, for whom there is no other alternative at the moment. This sole word of improvement may give rise to many expectancies, some of them being totally unrealistic to immediately benefit the patient, but rather future patients. Thus, a wide gap may exist between the expectation of the patients and what can be offered to them in return of being included in a study designed to test a paradigm, a material, a hypothesis, not aiming at improving the status of a real person (Lidz et al., n.d.; Clausen, 2009). For health professionals, the challenge posed by this gap consists of knowing how to address the hope risen by these new technologies among “expecting” patients, that is in individuals whose state is so serious that they have reached the point to expect everything from medicine, the progress of which is now considered so important that patients tend to put on practitioners the hopes that they once placed in natural or in God’s healing: Today doctors are supposed to know, they are supposed to have the power to give patients their health back. Addressing this hope becomes even more significant when the effectiveness of non-invasive BCI systems remains limited, and that a choice has to be made to resort or not to a surgery for testing a more invasive system. Although the present results obtained with invasive interfaces offer extremely promising perspectives with a growing number of degrees of freedom that can be controlled simultaneously (Hochberg et al., 2012; Ifft et al., 2013; Wodlinger et al., 2014), they nevertheless still remain modest (Dietrich et al., 2010) and great importance is still given to more conventional techniques for the compensation of handicaps using interfaces based on residual movements (Pino et al., 2003; Brunner et al., 2010; Treder and Blankertz, 2010; Takahashi et al., 2011). However, these approaches are themselves
A common approach for the brain signal processing, intended for event detection/prediction, consists in the extraction of the event-related features of the neuronal activity. Information from spatial (Rakotomamonjy et al., 2005), frequency (Schlögl et al., 2005), and temporal (Vidaurre et al., 2009) domains could be analyzed by means of the Principal Component Analysis (PCA) (Kayser et al., 2003), the Independent Component Analysis (ICA) (Makeig et al., 1999), the Linear Discriminant Analysis (LDA) (Scherer et al., 2004), the Common Spatial Patterns (CSP) (Zhao et al., 2008), the Partial Least Squares (PLS) (Chao et al., 2009), etc. Let us note that the standard methods are designed mainly for vector input variables which present either one domain of analysis or several domains unfolded in one line. Using only one domain usually does not provide satisfactory results. Combination of several domains is thus necessary. In most cases, two or three ways of analysis are applied sequentially. For example, see Galán et al., (2008), where first stable frequency components are determined and second the best electrodes are chosen. A multi-way analysis allows simultaneous treatment of several domains, by means of a tensor-based data representation. In recent years, it was applied in several BCI studies and demonstrated promising results (e.g., tensor factorization with PARAFAC (Nazarpour et al., 2006), Tucker (Zhao et al., 2009), Non-negative Tensor Factorization (Mørup et al., 2008), Multi-way Partial Least Squares (NPLS) (Bro, 1996), General Tensor Discriminant Analysis (Li et al., 2009), Regularized Tensor Discriminant Analysis (Li and Zhang, 2010)). Therefore, it is chosen for the ECoG data representation in the current study.
noticed the link between failures and frustration: “there is a bit of frustration sometimes when the system does not indicate the correct fixation, and it causes fatigue and loss of concentration".
Beyond those results, we were able to observe the importance of engagement for user experience. In pre-experiments, we observed that participants did not feel frustrated by the results, because they did not care about it. They just felt bored. Frustration likely depends on the engagement in the task. In “real life”, human-computer interaction is always directed towards a goal. However, in a strictly controlled experimental environment, the goal is artificial, and the users may not be motivated about it. In order to test more accurately user experience in real life, motivation has to be induced in the controlled experiment. One solution is to measure user experience over longer periods of time, while they accomplish a real-life task. But the complexity of a BCI setup, along with the difficulty of controlling external factors, makes this method difficult to apply. In our case, we were able to motivate participants to get good results by using a “carrot and stick” feedback, even though the success or failure was uncorrelated to the users’ actions. One participant commented: “The progress bar gives the incentive to success and the (fail) noises are very frustrating”.
Brain-Computer Interfaces (BCIs) allow end users to inter- act with a system using modulations of brain activity which is partially observable in electroencephalographic (EEG) signals . For several years, BCI progress has been focused essentially on rehabilitating people with severe neuromus- cular disorders . Nonetheless, the use of passive BCI to enhance surveillance of bioclinical signals is now emerging and could have a significant impact in the medical field . During general anesthesia, the occurrence of one particular phenomenon is feared by both patients and anesthesiologists: Accidental Awareness during General Anesthesia (AAGA) . This event can be defined as an unexpected awakening of the patient during a surgical procedure under general anesthesia . This situation occurs when the depth of anesthesia is insufficient to compensate for the surgical and environmental stimuli , . In high-risk practices, the AAGA rate is up to 1% ,  which is considered as high, and is especially difficult to detect when patients are paralyzed by the use of neuromuscular blocking agents.
Communication through spelling is still one of the main challenge in BCI applications. Writing a sim- ple message, an e-mail,... remains a difficult task to achieve for people with severe disabilities. The BCI literature has expentionaly increased in the past few years. Whereas recent BCI competitions have allowed to compare different machine learning methods, these benchmarks are limited to one aspect of a BCI. The graphical user interface should actually benefit the same attention than the signal processing part . This is particularly the case with SSVEP spellers that have constraints due to the number of available flashes on the screen. The presented spellers in this paper are only based on one type of brain activity. The next generation of BCIs will combine the detection of several brain responses, as hybrid BCIs . Such so- lution could provide faster and more robust spellers. They could solve to some extent the BCI illiteracy. This problem can be determinant for the choice of a specific BCI. Recent works have been conducted to address this problem: for P300 [21, 20], SSVEP , and sensorimotor rhythms . Further work should be carried out in the comparison of different well known and proven BCI systems with the same set of subjects. The different materials (amplifiers, caps,...) could be an obstacle for comparing and sharing BCIs. Hopefully, well established and promising BCI frame- works like BCI2000 and OpenViBE could allow a bet- ter comparison between spellers [35, 34].
This study was undertaken at the Liuhuaqiao Hospital, Guangzhou, China, between October 2012 and July 2013. Brain activity was detected only when patients were free of centrally acting sedative drugs. Eight severely brain-damaged Chinese patients participated in this experiment (four males; four VS, three MCS, and one LIS; mean age ± SD, 38 ± 19 years; see table 1 ). The study was approved by the Ethics Committee of Liuhuaqiao Hospital which complies with the Code of Ethics of the World Medical Association (Declara- tion of Helsinki). Written informed consent was obtained from each patient ʼs legal surrogates. The VS and MCS clin- ical diagnoses were based on the JFK Coma Recovery Scale- Revised (CRS-R), which comprises six subscales that address auditory, visual, motor, oromotor, communication, and arousal functions (Giacino et al 2004 ). During the experi- ment, the eight patients underwent a CRS-R assessment every two weeks (the ﬁrst CRS-R assessment happened within the week before the experiment). For each patient, the CRS-R scores presented in table 1 were based on his/her best responses of the repeated CRS-R assessments. No patient had a history of impaired visual acuity. In all eight patients, visual evoked potentials to ﬂash stimulation showed preserved bilateral cortical responses. Additionally, four healthy sub- jects (HC1, HC2, HC3 and HC4) with no history of neuro- logical disease (three males; mean age ± SD, 29 ± 2 years) were included in our experiment as control group.
the nerve fibers. The activity of the motor cortex is then analysed to decode the intention of movement. Using multielectrode arrays placed invasively in the primary motor cortex, it is possible to decode the direction of movement from the activity of a neural population  and send commands to a robotic limb or an exoskeleton to reach and grasp a target . Electrocorticography electrodes placed on the cortex are also used to obtain a good enough quality of signals, less invasively than microelectrode arrays which can cause brain tissue damage. Non-invasive signals such as EEG are also used in neuroprosthetics  although the spatial resolution is low and the signals are noisy. Thus, P300 wave and SSVEP can be used for a control task . But ERD is most commonly used to detect a desynchronisation in various parts of the cortex meaning that a specific mental task is performed. As an example, each mental task can be translated into a command for a wheelchair in a shared-control with an obstacle avoidance system [110, 111]. Motor imagery tasks are often used to control neural prosthesis or to trigger a functional electrical stimulation system [112, 113]. It should be noted, however, that designing neuroprostheses for the lower limb is still a challenge , notably due to the lack of tactile or proprioceptive feedback to the user. Therefore, current efforts are focussed on adding stimulations in the somatosensory cortex or on the skin of user to provide him/her with tactile feedback and therefore to obtain a complete action- perception loop.
The two proposed Riemannian methods are compared with the classical approach consisting in a spatial filtering by multi- class CSP ,  followed by a LDA classification  on the feature space composed by the log variance of the spatially filtered signals. The multi-class CSP used in this paper is an implementation of  that performed approximate joint diagonalization of all the class-related mean covariance matrices. This algorithm is more efficient than the standard CSP used in a one-versus-rest manner for the multi-class case. The number of CSP spatial filter is set to 8 as proposed in the reference papers , . For a more fair comparison, we also presented the results for the CSP method (denoted CSP*+LDA) whereby the optimal number of filters was se- lected for each subject according to a weighted FDR criterion that is similar to our method.
Inria / LaBRI Bordeaux, France email@example.com
Abstract—The omnipresence of non-stationarity and noise in Electroencephalogram signals restricts the ubiquitous use of Brain-Computerinterface. One of the possible ways to tackle this problem is to adapt the computational model used to detect and classify different mental states. Adapting the model will possibly help us to track the changes and thus reducing the effect of non-stationarities. In this paper, we present different adaptation strategies for state of the art Riemanniangeometrybased classifiers. The offline evaluation of our proposed methods on two different datasets showed a statistically significant improvement over baseline non-adaptive classifiers. Moreover, we also demon- strate that combining different (hybrid) adaptation strategies generally increased the performance over individual adaptation schemes. Also, the improvement in average classification accuracy for a 3-class mental imagery BCI with hybrid adaption is as high as around 17% above the baseline non-adaptive classifier.
2) In-House dataset: 18 BCI-naive subjects took part in this study, for 6 different sessions each (each on a different day). Subjects had to perform three different mental imagery (MI) tasks: 1) left-hand motor imagery, 2) mental rotation of a 3D geometric figure and 3) mental subtraction of a 2 digit number from a 3-digit number (both displayed on screen). EEG were recorded from 30 channels. Each session comprised 5 runs. During each run, subjects had to perform 45 trials (15 trials per task), each trial lasting 8s. At t=0s, a cross was displayed on screen. At t=2s, a “beep” announced the coming instruction and at t=3s, an arrow was displayed, the direction of which informed the subject which task to perform. Finally, at t=4.250s, for 4s, a visual feedback was provided in the shape of bar, whose length reflected the classifier output. More details about this dataset can be found in .