First, we will review and discuss significant Kruskal-Walliss and U Mann Whitney tests for the basic emotions prototypes; then we will do the same for the masked emotion category.
Happiness
The only EMFACS prediction for any enjoyable emotions includes a combined innervations of the Orbicularis oculi, Pars orbitalis (AU6) and the Zygomatic major (AU12) see figure 15. The Kruskal-Wallis test confirms that the relative duration of this combination differs significantly across the five rating groups [H (4, N = 200) = 42.81 p = 0.000]. More
75 specifically, the Mann Whitney tests reveals that the relative duration of overlap between action unit 6 and 12 is significantly longer in the video records rated as conveying a sense of enjoyment than in all the other clusters (see table 23).
Figure 15 AU6+12
Anger
For angry expressions, only 2 out of 8 proposed combinations are shown to differ across clusters. These are combinations AU4+5 [H (4, N = 200) = 8.86 p = 0.039] and AU4+7 [H (4, N = 200) = 8.31 p = 0.040]. Both these configurations involve the brow and eye regions. Action unit 4 draws the brows down and together. Combined with AU4, action units 5 or 7, produce the two versions of “angry” eyes described by Ekman (Ekman and Friesen, 2003, p.83). In the AU4+7 version (figure 16), the lower eyelid is tensed narrowing the lower part of the eye. The lowering of the upper part of the eye is due to the action of unit 4 that creates the impression that the upper eyelid is lowered. In the AU4+5 version (figure 17), the brow is also lowered reducing the eye aperture but the action of unit 5 produces a wider gaze opening than in AU4+7. The AU4+5 configuration is found in both the “hostility” and
“sadness” clusters. Mann Whitney tests shows it is significantly more frequent in the
“hostility” than in the “sadness” cluster (see table). The AU4+7 combination is found only in the “embarrassment” and “enjoyment” clusters. Significantly more in the “embarrassment”
than in the “enjoyment” group (see table). The Kruskal-Wallis tests for the other predictions tested were all non significant: AU17+23 [H (4, N = 200) = 2.57 p = 0.039]; AU17+24 [H (4, N = 200) = 1.85 p = 0.039]; AU4+5+10 [H (4, N = 200) = 3.47 p = 0.482]; AU4+7+10 [H (4, N = 200) = 5.06 p = 0.280]; AU4+7+23 [H (4, N = 200) = 2.70 p = 0.600] and AU4+5+7 [H (4, N = 200) = 3.24 p = 0.518]. Prototypical “anger” expressions also include lower face
76 movements notably lips pressing against each other or open square mouth (as in screaming).
When these lower face elements are not present together with the involvement of the brows / forehead and eyes / lids, the meaning of these expressions are ambiguous (Ekman and Friesen, 2003, p.87). Aside from a slight display of anger or signs of anger control; a serious, concentrated or a determinate attitude are also presented as possible interpretative alternatives by Ekman and Friesen (2003).
Figure 17 AU4+5 Figure 16
AU4+7
Fear
Out of the five "fear" expressions investigated, only a single configuration involving three upper face actions was found to differ across the clusters: AU1+2+4 [H (4, N = 200) = 13.29 p = 0.009]. The combination of AU1+2 with AU4 (see figure 18) is considered a typical
“fear” brow in Ekman’s terminology (Ekman and Friesen, 2003, p.50). The brows are lifted as they are in surprise, but in addition to the lift they are drawn together so that the inner corner of the brows are closer together in fear than in surprise. According to Ekman, a full blown expression of fear would also include an upper eyelid raise (AU5) as well as a bilateral stretch of the mouth (AU20). When the brow is held in the fear position with the rest of the face uninvolved, Ekman suggests that worry or controlled fear might be conveyed (Ekman and Friesen, 2003, p.52). The Mann-Whitney test reported in table 23 shows that this expression is found significantly more in the embarrassment and sadness clusters than in the hostility group. The surprise and enjoyment clusters contain no instances of this combination. The non significant combinations tested were: AU1+2+5 [H (4, N = 200) = 3.60 p = 0.560];
AU1+2+20 [H (4, N = 200) = 3.95 p = 0.412]; AU1+2+4+20 [H (4, N = 200) = 0.00 p = 1.000]; AU1+2+5+20 [H (4, N = 200) = 3.60 p = 0.462].
77 Figure 18.
AU1+2+4
Surprise
None of the three proposed configurations for "surprise" expressions seems to differ in terms of frequency of common overlapping time units across the five clusters. The configurations tested were: AU1+2+5 [H (4, N = 200) = 2.94 p = 0.560]; AU1+2+26 [H (4, N
= 200) = 2.94 p = 0.908] and AU25+26 [H (4, N = 200) = 2.94 p = 0.36]. Note, that Ekman (2003) has pointed out that the evidence for surprise being a basic emotion in his sense is the weakest of all candidates because it hedonically neutral. Moreover in recognition studies prototypical displays of surprise are often not distinguishable from fear (Ekman, 2003). Apart from the possible ambiguous status of surprise as a “basic” emotion possessing a distinctive facial display, an alternative explanation can be invoked to explain the apparent lack of specificity in the distribution of the AU1+2+5 configuration across the five rating groups. In our dataset emotional as well as conversational facial signals are presented together to the judges. Because the combination of AU1 with AU2 has been documented to serve as a common conversational gesture used to emphasize (baton) or underlie (underliner) parts of speech it is possible that a large proportion of action units 1+2 in the database serve such conversational functions. If this is the case, it becomes difficult to find quantitative differences in the association of AU1+2 with other AUs that are not due to chance alone.
Sadness
From the six configurations tested for their predicted potential to convey a sad demeanor only one did come out as significantly differing across the five clusters [H (4, N = 200) = 11.82 p = 0.019]. The combination of action units AU6 (cheek raise) with AU15 (corner lip depressor) (figure) is found to be more prevalent in the enjoyment than in the
78 hostility and sadness clusters (see table). The non significant combinations tested were AU1 +14, [H (4, N = 200) = 2.51 p = 0.642]; AU1 +10, [H (4, N = 200) = 5.10 p = 0.277];
AU1+14 [H (4, N = 200) = 5.29 p = 0.258]; AU6+17 [H (4, N = 200) = 8.98 p = 0.068] and finally AU6 +15 +17 [H (4, N = 200) = 5.61 p = 0.230].
Figure 19 AU6+15
Disgust
For disgust displays, the frequency of occurrences of only 1 out of 4 possible EMFACS prototype tested is found to differ across the rating groups [H (4, N = 200) = 17.18 p = 0.0018]. Table 23 shows that in the sample files belonging to the enjoyment group the duration of overlap between AU10 and AU12 (see figure 20) is significantly longer than in any other cluster. Additionally this combination is also more characteristic of the embarrassment than the sadness cluster. Although "disgust" expressions are mainly depicted with single actions; AU9 or AU10 in emotion recognition studies, the combination of AU10 with a smiling action (AU12) appears in the dictionary predictions as a prototypical expression of disgust.
79 Figure 20
AU10+12
Interestingly, the same configuration is also listed as a possible masked expression of disgust. The authors of the dictionary seem to leave open the interpretation of this display as communicating either a frank attitude of disgust or alternatively an attempt at concealing a disgust reaction. The fact that this display is more characteristic of video files rated as conveying enjoyment rather than hostility can be interpreted in several ways. First it is possible that raters have simply disregarded subtle signs of disgust (AU10) in their assessment of video-files rated as positively valenced. Second the AU10 may have been noted but its association with AU12 may have dampened its negative message value
Contempt
For contempt displays, the only action units combination proposed in the EMFACS taxonomy involves the following actions: AU1+2+14. The other predictions are limited to single action units AU10U, AU12U and AU14U that were shown not to differ significantly in their frequency of occurrences across the five clusters. In our dataset the cumulating time units when AU1+2 and AU14 overlap is not found to be significantly larger in any of the rating clusters [Kruskal Wallis: H (4, N = 200) = 3.47 p = 0.1].
80