Chapter 2 State of the art

2.1 Emotion representations and models

2.1.3 Models of emotions

Sections 2.1.1 and 2.1.2 describe different representations of emotions. While they are useful for taxonomy, categorization and differentiation of emotions they do not provide much information about the process that gives rise to emotions. Several models, essentially from the cognitive view of emotions, have been proposed to answer this question [9, 10, 32, 56-58]. Most of them are based on the central concept of appraisal which has been defined by Arnold as the cognitive process that constantly evaluates the environment and elicit emotions [41]. It is generally considered as an unconscious process that is direct and non-reflective. To our view, the main differences between models of appraisal are the different evaluation criteria they propose.

Two of the most famous models are presented in the following sections: the OCC (Ortony Clore and Collins) typology [57], which has been used to define computational models of emotions, and Scherer’s SECs (Stimulus Evaluation Checks) [9, 10], which can be viewed as an attempt to unify the different emotion theories.

a. OCC typology

According to Ortony [34], “the defining feature [of an emotion] that we consider most reasonable and least contentious is that the appraisal underlying the emotion be valenced, either positively or negatively”. It is thus not surprising to find the valence concept at the heart of his model. In the OCC (Ortony, Clore and Collins) typology [57], an emotion is viewed as a valenced reaction to a stimulus. More precisely, the elicitation of an emotion relies on the evaluation of three main criteria: the type of the stimulus (agent, object, and event), the concerned entity (self or another) and finally the valence of the emotion (positive or negative). Since each of these criteria can only be discretely evaluated, it is possible to represent the model as a tree with the resulting emotions as the leaves (Figure 2.4).

Two examples are given bellow that respectively corresponds to the green and blue dotted line of Figure 2.4. As a first example, imagine that a student just sat for an exam and is rather pleased by his / her performance. In this case the type of stimuli is an action (sitting for an exam), the agent of interest is the self, and since the valence is positive the resulting emotion is pride (green dotted line in Figure 2.4). As a second example, our student receives a letter concerning the result of the exam and reads it. Here the stimulus is an event (reading of the letter) and the concerned entity is again the self. Since prospects are relevant in this case and the student has opened the letter, the elicited emotions will be either satisfaction or disappointment depending of his / her results (blue dotted line in Figure 2.4). Notice that as long as the letter wasn’t read the elicited emotion was hope (he or she was happy about his / her performance).

Figure 2.4. The OCC typology (from [57]). Green and blue dotted lines correspond to the examples above.

This model has the advantage of being computationally tractable. Consequently, it has been used for the purpose of emotion synthesis. Elliot extended it in [59] by adding few emotions, and implemented it in a computational affective reasoner. In [60], the authors converted this model in a BDI (Belief, Desire and Intensions) architecture and also demonstrated the effectiveness of this approach.

This model can also be used for the purpose of user-emotion assessment. In order for this model to be applicable in this case, it is mandatory to have a complete knowledge or control of the environment in which the user is evolving to be able to follow a path in the tree. This is an interesting fact since it strongly emphasizes the importance of context in emotion assessment.

However, even with a strong knowledge of context, valence is still a hard criterion to evaluate.

Let’s take the example of a learning application since, as explained in the introduction, there are several advantages to include emotion assessment in this type of environment. Imagine that the user is informed that he failed in a task then the valence could be guessed as being negative. Now

imagine that he just finished the task but he has not been informed about his / her performance then the valence will depend on the user interpretation of his / her success. Since this information is not directly available to the system, it is not possible to infer any emotional state. In this case, detecting the valence from the measures of user’s emotional cues can solve the issue.

b. SECs theory

In his theory, Scherer [9, 10] proposed that appraisal can be decomposed into several checks that are cognitively evaluated. They are called the Stimulus Evaluation Checks (SECs, Table 2.2).

Each of these processes is supposed to occur in a specific time span from the elicitation event with the possibility of parallel processing (time span that overlap). Those SECs can be grouped in the four appraisal objectives listed below.

- Relevance detection: the objective of this group of SECs is to determine if the stimulus requires attention and further processing by analyzing its relevance for the person; the goal is to answer the question “Are there any possible implications to me or my direct environment?”.

- Implication assessment: the next step is then to determine what the implications of the stimuli are in terms of consequences for the self. This objective regroups the central processes of appraisal and its function is mainly the protection and the progress of the organism: “is the stimulus dangerous or appealing?”, “does it correspond to my goals and needs?”. Notice that this appraisal objective is very close to what Darwin defines as the function of emotions as a whole.

- Coping potential determination: the concept of coping is well known in psychology and is defined as the cognitive mechanisms that are implemented to control and reduce the impact of stressful and emotional events. According to Lazarus [32], there are two types of coping: problem oriented (determination of an action to solve the problem that gave rise to the stressful situation) and emotion oriented (cognitive regulation of the stress for instance by reconsidering the situation). The current appraisal objective is to deal with both of those aspects.

- Normative significance evaluation: the objective of this SECs group is to evaluate whether the stimulus is compatible with one’s own principles and standards as well as with social norms and values.

The order in which the SECs are presented in Table 2.2 is significant as it represents the sequence in which the checks are supposed to be completely evaluated. The appraisal decomposition in the present SECs as well as the temporal aspects have been validated in [61, 62].

Appraisal objectives

SECs Description / Answer the question

Relevance detection

Novelty Is stimulus novel and does it require attention?

Is it familiar and/or predictable?

Intrinsic pleasantness

Will the stimulus lead to pleasure or pain?

This is a property of the stimulus and is not related to the current state of the organism (it does not depend on the current goal and objectives of the subject but could have been learned in the past).

Goal relevance check Does the stimulus have some consequences on my current goals?

Implication assessment

Causal attribution

What or who is the cause of the stimulus and why?

Often divided in two categories: the responsible agent and the motive.

Outcome probability What are the consequences of the stimulus?

What is the probability of each consequence?

Discrepancy from expectation

To which extent the stimulus is different from what was expected.

Goal / need conduciveness

Will the consequences of the stimulus help me to accomplish my goals or will they obstruct my goals?

Urgency Should the stimulus be handled quickly?

Coping potential determination

Control To which extent the stimulus can be influenced and controlled.

Power

If control is possible, check the available resources (physical, knowledge, etc.) for a potential action.

It is related to the problem oriented coping of Lazarus.

Adjustment

If it is not possible to influence the situation (because of a non controllable stimulus or lack of resources), check how well the organism can adjust with the situation.

It is related to the emotion oriented coping of Lazarus.

Normative significance

Internal standards Is the stimulus in accordance with one’s own principles, ideals and moral code?

External standards Is the stimulus in accordance with social norm and values?

Table 2.2. List and description of the different Stimulus Evaluation Checks (SECs) grouped by appraisal objective and temporally ordered.

Contrarily to the OCC model, where the different criteria are evaluated discretely, Scherer proposed that most of the SECs are evaluated on continuous scales (for instance the outcome probability SEC is evaluated for each event on a continuous scale ranging from 0 to 1).

Moreover, 14 emotions such as happiness, disgust, anxiety, fear, pride, guilt and boredom were profiled by giving for each of them the possible associated evaluation of the SECs [9] (for instance Happiness is associated to high intrinsic pleasantness, medium goal/need relevance, very high outcome probability, very low urgency, etc.). It was shown that these profiles can be used to correctly identify an emotion associated with situations described according to the SECs.

However, the emotional profiles are described with words like “low”, “high”, “medium” so that the computational implementation of this model could not be directly done with continuous evaluations of the SECs. A first step should be taken to find evaluation thresholds (for each SEC) that define the boundaries between emotions. Moreover, it still remains that this model is very complex and requires the evaluation of 13 SECs confirming the remark made for the OCC model about the difficulty of components evaluation.

In his appraisal theory, Scherer does not limit his analysis to the different components of the cognitive appraisal but also studied the relation between the systems (or components) of the organism taking part in the emotion elicitation and differentiation [10]:

- the cognitive system, that is responsible for the evaluation of the emotional event (appraisal of the stimuli);

- the autonomic system, that provides support for other components (for instance the quantity of blood in hands can increase in case of anger to prepare for action);

- the motivational system, related to action tendencies, urges and desires (i.e. will one withdraw from the stimuli or approach it);

- the motor system, to execute the (re)-action but also to show facial expressions as well as emotional gestures and behaviors;

- the monitor system, which gives rise to the subjective feeling one can experience after being confronted to an emotional situation.

This list clearly demonstrates the multimodal aspects of emotions, since emotional activation is reflected in the activity of all those systems. Notice that this activity could be monitored in order to assess emotions; actually it is even possible that monitoring the activity of ALL those systems is necessary to reliably and fully assess emotions.

This last statement goes along with the componential patterning theory. According to this theory, a given evaluation of the SECs leads to a precise activity pattern in the five organism components and this pattern can be used to predict emotion. Another important statement is that the different components are not independent but rather interrelated. Consequently, they exchange information for the evaluation of the different SECs. For instance the evaluation of a SEC (cognitive component) will give rise to activity in another component (for instance in the autonomic system); this activity could then be feedbacked for the evaluation of another SEC. Since the SECs are evaluated in sequence, this theory implies that the pattern of emotional activity changes during the appraisal process and that synchronization between the different systems is necessary.

The collaboration between different components has also been suggested in [63] while the dynamic and synchronization aspects are detailed in [64].

Dans le document Emotion assessment for affective computing based on brain and peripheral signals (Page 36-41)