• Aucun résultat trouvé

Figure 1.1. Including emotions in the human-machine loop ... 2 Figure 1.2. Emotion assessment in human computer interfaces, adapted from the execution / evaluation model [8]. ... 4 Figure 2.1. Plutchik’s wheel of emotions. (Left) The basic emotion represented as quadrants and possible combinations of basic emotions. (Right) The same wheel with the added concept of intensity2. ... 16 Figure 2.2. Valence arousal space with associated labels as (a) points (adapted from [Russell], (adjectives have been changed to nouns and only some of the words are displayed for clarity) and (b) areas. ... 17 Figure 2.3. Self-assessments distribution obtained when eliciting emotions with images: most of the self-assessed images lie inside the U-shape. ... 18 Figure 2.4. The OCC typology (from [52]). Green and blue dotted lines correspond to the examples above. ... 20 Figure 2.5. Picture of the SAM scales (from [68]).The first line evaluates valence from positive (left) to negative (right), the second arousal from excited to calm and the third dominance from submissive to powerful. ... 26 Figure 2.6. An acquisition system for visualization and storage of physiological data. ... 27 Figure 2.7. (a) Figure of a neuron connected with two input neurons (named 1 and 2). (b) Representation of the integration of input action potentials; the neuron fires only if its membrane potential exceeds a given threshold. ... 28 Figure 2.8. Image of the brain, the brain stem and the cerebellum with the different lobes highlighted (from [69]). ... 29 Figure 2.9. Principal structures of the limbic system together with their functions. ... 32 Figure 2.10. (left) Example of a signal representing the changes of resistance of the skin, (right) the characterization of an electrodermal response. ... 35 Figure 2.11. Examples of signals obtained from a respiration belt tied across the chest and a temperature sensor placed bellow the nostrils during different type of respirations. ... 39 Figure 3.1. Hardware and software for signal acquisition. ... 51

Figure 3.2. (left) A participant wearing the EEG cap with 64 electrodes plugged. (right) Top head view with the positions and names of the 64 electrodes used for EEG recording. For a 19 electrodes configuration only the green electrodes were used. ... 53 Figure 3.3. Pictures and positions of the sensors used to monitor peripheral activity. The CMS / DRL position was used only in the case where EEG activity was not monitored simultaneously with peripheral activity. ... 54 Figure 3.4. The heart waves in a BVP signal. (Left) Three pulses of the BVP signal with the different peaks, (rigth) example of a pulse where it is difficult to identify the different peaks. .... 57 Figure 3.5. Example of the beat detection and HR computation algorithm on a 9 seconds signal.

The HR signal is represented as a staircase function with the length of a step corresponding to the duration of an IBI. ... 58 Figure 3.6. Top head view with EEG electrode locations and corresponding frequency bands (from [77]). ... 63 Figure 4.1. Validation scheme for classification, where yˆis the vector of the classes estimated by model for the test set, A is the accuracy... 74 Figure 4.2. Obtaining posterior probabilities p( i | h) from SVM outputs. a) Histograms representing the distributions of the SVM output for two classes. b) Posterior probabilities estimates from the Bayes rules applied on the histogram of a) and from the sigmoid fit proposed by Platt [135]. ... 79 Figure 4.3. Different possible distributions of a feature value for a 3 classes scenario (green, red and black classes). (left) The feature is relevant since it is usefull to distinguish the green class from the others (low p value). (right) A non relevant feature (high p value). ... 81 Figure 5.1. Description of the acquisition protocol. (left) the modified SAM used for self assessment. (right) the schedule of the protocol. ... 91 Figure 5.2. Histograms of the IAPS and self evaluations (valence and arousal) for the valence experiment. For easier comparison of IAPS evaluations and self evaluations the IAPS values have been normalized to the same range as the self evaluations. ... 94 Figure 5.3. Histograms of the IAPS and self evaluations (valence and arousal) for the arousal experiment. For easier comparison of IAPS evaluations and self evaluations the IAPS values have been normalized in the same range as the self evaluations. ... 95 Figure 5.4. LDA accuracy for classification of negative and positive stimuli. ... 98

Figure 5.5. Classifiers accuracy with 2 classes constructed from self-assessment. ... 99 Figure 5.6. Classifiers accuracy with 3 classes constructed from self-assessment. ... 100 Figure 6.1. (left) The different emotional classes represented in the valence-arousal space and their associated image. (right) schedule of the protocol and detail of a trial. ... 103 Figure 6.2. Complete process of trial acquisition, classification, fusion and rejection for a given participant. As defined in Chapter 4, ki is the confidence measure of class i after opinion fusion and reject is the rejection threshold. ... 105 Figure 6.3. Mean classifier accuracy across participants for the EEG_STFT feature set and the different classification schemes. The bars on top of each column represents the standard deviation across participants. ... 110 Figure 6.4. Mean classifier accuracy across participants for the EEG_MI feature set and the different classification schemes. The bars on top of each column represents the standard deviation across participants. ... 111 Figure 6.5. Mean classifier accuracy across participants for peripheral features and the different classification schemes. The bars on top of each column represents the standard deviation across participants. ... 111 Figure 6.6. classification accuracy using participant 1 EEG_STFT features with LDA and with SVM on the five sets of classes, with or without FCBF feature selection. The bottom horizontal axis indicates the value of the threshold FCBF, while the top horizontal axis corresponds to the number of selected features. ... 115 Figure 6.7. Accuracy of LDA and the Linear SVM classifiers for different numbers of selected features of the EEG_STFT feature set using the Fisher criterion (only for the CPN classification scheme). Only the number of features marked with a ‘+’ have been computed while the other values are linearly interpolated. ... 116 Figure 6.8. Average accuracy across participants for different modalities and their associated classifiers, as well as for fusion of the two EEG and the three physiological modalities. ... 117 Figure 6.9. Relation between the threshold value, classification accuracy and the amount of eliminated samples for the CPN classification task. ... 118 Figure 7.1. Flow chart and the suggested automatic adaptation to emotional reactions. ... 122 Figure 7.2. Screen shot of the Tetris (DotNETris) game... 123

Figure 7.3. Histogram of the skill levels of the 20 participants. ... 124 Figure 7.4. Schedule of the protocol. ... 125 Figure 7.5. Mean and standard deviation of judgments for each axis of the two component (comp.) space and the different difficulties (diff.): easy, medium (med.) and hard. ... 128 Figure 7.6. Boxplot of the EEG_W values for the three condition. The red line represent the median of the EEG_W values, the box the quartile and the whiskers the range. NS: non significant. ... 130 Figure 7.7. Accuracies of the different classifiers and feature selection metods on the peripheral features. ... 133 Figure 7.8. Histograms of the number of cross-validation iterations (over a total of 20) in which features have been selected by the FCBF, ANOVA and SFFS feature selection algorithms. The SFFS feature selection is displayed for the DQDA classification. ... 134 Figure 7.9. Accuracies of the different classifiers and feature selection metods on the EEG features. ... 135 Figure 7.10. Histograms of the number of cross-validation iterations (over a total of 14) in which features have been selected by the FCBF, ANOVA and SFFS feature selection algorithms. The SFFS feature selection is displayed for the DLDA classification. ... 136 Figure 7.11. Classification accuracy as a function of the duration of a trial for EEG and peripheral features. ... 138 Figure 7.12.Averages of the normalized GSR and HR signals for the 5 seconds following the game-over triggers. Points that are marked with a ‘+’ corresponds to the samples that were found to be significantly different (p-value < 0.1) among the two conditions. ‘**’ indicate that p-value <

0.05. ... 141