Emotional expressions conveyed by the face, voice, or body provide information that can facilitate social interaction and make it more predictable and manageable. Thus, accurately
recognizing the type and intensity of others’ emotional states from their nonverbal expressions is a precondition for understanding and adequately responding to their reactions, thoughts, and
intentions. This importance for human communication has made emotion expression and emotion recognition a widely researched field in psychology over the past decades.
In the typical paradigm to investigate emotion recognition, participants are presented with pictures or recordings of emotional expressions and are asked to choose from a list of emotion terms such as anger, sadness, happiness, surprise, fear, and disgust, the one that describes best each expression. This approach has its roots in basic emotion theory which emerged from the pioneering research conducted by Tomkins (1962), Ekman and colleagues (e.g., Ekman, Friesen, & Ellsworth, 1972), and Izard (1971). Influenced by Darwin’s notion of the universality and innateness of emotion displays, these scholars proposed a small, fixed set of discrete “basic” emotions, which
consist of a pre-programmed neuromotor network (an “affect program”) and have a
biological origin. Other emotions than the basic ones (i.e., happiness, anger, disgust, fear, sadness, and surprise) are considered blends of the basic emotions. According to this view, emotional expressions signal specific emotions which perceivers decode in a categorical and immediate fashion (Ekman, 1992). The high agreement between perceivers in judging basic emotion
expressions that has been found in many studies is considered as support for this view (e.g., Ekman, 1992). Following basic emotion theory, emotion recognition thus involves linking features of an emotion expression to knowledge about the meaning of the different emotion categories and labels.
However, this perspective on emotion recognition has not been left unchallenged. During the past decades, other emotion theories gained importance which highlight different “objects” of recognition from emotional expressions. For example, dimensional theories postulate that emotions are located in a space defined by several dimensions such as valence and arousal (e.g., Russell, 2003). According to this perspective, Carroll and Russell (1996) claimed that what is recognized from expressions is valence and arousal, on the basis of which a discrete emotion label is inferred.
One reason for this claim is Russell and Bullock’s (1986) finding that the boundaries between discrete categories are fuzzy at the level of recognition and that agreement between perceivers drops considerably when they are asked to freely label emotion expressions instead of picking a label from a list of basic emotions (Russell, 1994). Further supporting the dimensional perspective on emotion recognition, Russell, Bachorowski, and Fernandez-Dols (2003) noted that perceivers agreed substantially in their judgments of valence and arousal and that the same dimensions emerged when analyzing the confusions perceivers made when judging discrete emotions.
The third dominant approach to emotion encompasses componential theories which
postulate that emotions are characterized by changes in different psychological subsystems such as cognitive appraisals, feelings, bodily sensations, and action tendencies (for an overview, see Scherer, Schorr, & Johnstone, 2001). For example, the component process model (Scherer, 1984)
posits that the differentiation of emotions is determined by the results of event evaluation processes based on a set of appraisal criteria. Support for the appraisal perspective in emotion recognition was found in different sensory modalities. For example, Scherer (1999) found that emotions are
recognized faster from verbal descriptions of emotional events when the description follows a theoretically predicted sequence of appraisal segments. Furthermore, Banse and Scherer (1996) found that vocal patterns of emotion expressions corresponded to a large extent to theoretically derived appraisal constellations and substantially predicted participants’ recognition accuracy.
Other accounts of what is being recognized from emotional expressions can be derived from Fridlund’s (1994) theory of social messages and Frijda’s work on action tendencies (e.g., Frijda, Kuipers, & ter Schure, 1989). According to Fridlund (1994), emotional expressions have the
function of signaling behavioral dispositions or intentions to other people and what is recognized by perceivers is this social message. In contrast, Frijda (see Frijda et al., 1989) proposed that emotional expressions primarily signal action tendencies. Scherer and Grandjean (2008) compared
participants’ accuracy of emotion judgments for facial expressions made on the basis of a) discrete emotion categories, b) social messages, c) appraisal results, and d) action tendencies. They found that emotion categories and appraisals were judged significantly more accurately from faces than social messages and action tendencies. However, recognition accuracy for messages and action tendencies was still substantially above chance level. The authors therefore concluded that facial expressions allow for the recognition of different emotion components individually as well as of the discrete emotion label that integrates these components.
Taken together, these studies suggest that what is actually recognized by people from emotional expressions is not restricted to a discrete emotion label as suggested by basic emotion theory. Rather, emotional expressions encompass a wide range of social signals and convey appraisals of the emotional-related event, action tendencies, social messages, or broad dimensions of valence or arousal. Which aspect or object of recognition is investigated in a particular study
depends on the underlying emotion theory. To date, the major part of studies on emotion
recognition relies on basic emotion theory. This approach has also largely influenced research on the mechanisms and processes underlying emotion recognition which are discussed in the next section.