N EURAL CORRELATES OF FACE RECOGNITION AND LEARNING

As mentioned in section 1.1.2. of this introduction, cell recordings in monkeys showed that face processing involved dedicated brain circuits, including the superior and inferior temporal cortices (e.g., Perrett et al., 1982). Neuropsychological reports (e.g., Farah, 1996; Vuilleumier et al., 2003) and brain imaging studies in humans also highlighted specific neural correlates for face processing (e.g., Puce et al., 1995; Haxby et al., 2001). Kanwisher et al. (1997) identified a region in the fusiform gyrus which was activated while subjects viewed rapidly presented sequences of faces versus sequences of inanimate objects. They named this region the ‘fusiform face area’ (FFA).

Gobbini and Haxby (2007) proposed a model of the distributed neural system underlying face perception and recognition, dividing it into a core system and an extended system (Figure 7).

The core system includes the inferior occipital gyri (IOG), the fusiform gyri (FG) and the posterior superior temporal sulcus (STS), mediating the visual analysis of faces. In this core system, IOG and FG play a pivotal role in the processing of invariant features of faces and in identification. The extended system includes the posterior cingulate cortex, involved in episodic memory for faces, as well as subcortical and other cortical areas involved in the extraction of the emotional and semantic content associated to the face.

Numerous imaging studies have suggested that the ‘‘fusiform face area’’ (FFA) is a key structure for face detection but also for face identification (e.g., Grill-Spector et al., 2004;

Rotshtein et al., 2005). More precisely, a PET-study showed that the activity in the right fusiform gyrus during the encoding of unfamiliar faces predicted subsequent recognition performance (Kuskowski & Pardo, 1999). An fMRI study on visual imagery also showed an implication of the FFA during the recall of familiar faces (Ishai et al., 2002). Consequently, Rapcsak (2003) suggested that the FFA might be a potential anatomical substrate for the FRU and the anterior temporal lobe the location of the PIN of Bruce’s cognitive model (Bruce &

Young, 1986). In a model built on neuropsychological data, Rapcsak et al. (Rapcsak et al., 1999; Rapcsak, 2003) also included the medial temporal lobe (MTL), implicated in encoding and recognition, and the prefrontal cortex (PFC), acting as an executive module which verifies inputs from the other modules. They observed that lesion of the PFC induced extremely high false-recognition rates (Rapcsak et al., 1999).

More recently, Bartlett et al. (2009) analyzed data from several face-recognition tests including old faces, new faces and conjunction faces (formed of previously seen parts but not in the learned configuration). They extracted two main components explaining the data, one related to the hit rates (and probably originated in occipito-temporal regions), and the other related to false-alarm rates (linked to frontal areas). They integrated their findings in a new version of the Rapcsak’s (1999) model of face recognition, where the face recognition module was divided into two components, one for the configural processing of faces, and one for the part-based (featural) processing, both signaling resemblance to the MTL component. The configural resemblance signal should be higher for old faces than for conjunction faces, allowing accurate recognition of the old stimuli and accurate rejection of the distracters. The MTL is supposed to integrate information coming from these two components, and to send them to the frontal executive component (FEC), which in turn, synthesizes resemblance and contextual signals, and prevents from false recognition (Figure 8).

Figure 7: A model of the distributed human neural system for face perception. Source: Gobbini &

Haxby, 2007.

Figure 8: Model of face recognition, based on Rapcsak et al. (1999), including components for configural and part-based (featural) processing. Upward-pointing arrows represent bottom-up perceptual and retrieval processes and downward-pointing arrows represent controlled, top-down processes of (A) analysis of resemblance information, and (B) analysis of contextual/associative information (e.g., biographic information for familiar faces). The broken arrow from “resemblance information” to the frontal executive component reflects the assumption that resemblance information is weighed less heavily than contextual/associative information in bottom-up recognition. Source:

Bartlett et al., 2009.

Bartlett et al. (2009) manipulated the number of faces presentation and measured the corresponding number of hits for the previously learned faces, and the number of false alarms for the faces made of repeated parts. They observed that repetition increased the number of hits and false alarms, with a higher increase for hits. This effect was also found with inverted faces. Thus, brain activity related to both featural and configural processing as well as the activity in the frontal executive module might be modulated by experience.

This Bartlett’s model (2009) is the most recent one but also seems to be the most complete one. The only module missing might be an ‘affective’ module, which would be linked to all the modules and also to the response.

However, despite these models, the nature of the representations held in the occipito-temporal region is not clear yet. Is the FFA specifically processing configuration? Is the inferior occipital gyrus only processing features? Further investigations are needed in order to answer these questions.

In the next section, we will review what is currently known on the nature of the representations of faces in the fusiform gyrus.

Nature of the representations in the Fusiform Gyrus

Domain-general vs. domain-specific

The first question about the exact function of the fusiform gyrus is whether it is truly specific to faces or whether it involves a domain-general processing that could be used for other homogeneous stimuli. Gauthier and colleagues trained subjects with similar exemplars of a new category and found that the right fusiform was activated in discrimination tasks using these ‘objects’ (Gauthier et al., 1999; Tarr & Gauthier, 2000). They claimed that this was a strong evidence for a non-specific information treatment in the fusiform gyrus. However, this evidence can be refuted considering the nature itself of ‘Greebles’ which can be considered as highly similar to faces, because of their symmetry and arrangement of parts similar to the configuration of eyes, nose and mouth (Kanwisher & Yovel, 2006). Gauthier and colleagues also showed activation in the fusiform gyrus of birds and cars experts (Gauthier, Skudlarski et al., 2000). They interpreted these results as the expression of an expertise, involving a perception at the individual level (Gauthier, Tarr, Moylan, Skudlarski et al., 2000).

However, another study found a correlation between the activity in the fusiform gyrus and face detection and recognition, but no such results in within-category identification of other objects (including cars seen by car experts) (Grill-Spector et al., 2004). Moreover, Henke and colleagues (1998) observed that prosopagnosic patients were not impaired at identifying

visually homogeneous objects at the individual level. Thus the expertise vs. specificity controversy is not totally solved yet.

Invariance

Another important question is the degree of invariance of face representations (i.e., their degree of activation for visual inputs in varying conditions such as illumination, viewpoint, or size). Haxby et al. (2000) proposed that the fusiform gyrus holds representation of the invariant aspects of faces. FMRI-adaptation paradigms have shown contradictory evidences.

Several studies found invariant representations for faces in the fusiform gyrus, regarding size for example (Andrews & Ewbank, 2004). Other studies did not find such adaptation effects, especially using viewpoint variations (Fang et al., 2007; Davies-Thompson et al., 2009).

Intermediate results were also found, showing invariant adaptation in the FFA for familiar faces and diminished adaptation for unfamiliar faces as a function of increasing angle (Ewbank & Andrews, 2008). Finally, Pourtois and colleagues could not observe any priming effect in the FFA but found a region slightly more medial than FFA in the right fusiform which showed viewpoint-invariant adaptation (Pourtois et al., 2005b; Pourtois et al., 2009).

Configuration

In order to assess the configural nature of face representations, several authors performed neuroimaging studies using the inversion effect paradigm. They showed a decrease in brain activity in the medial fusiform gyrus (MFG) during the repetition of upright faces but no such effect, and even an increase of activity, during the repetition of inverted faces (Mazard et al., 2006; Yovel & Kanwisher, 2005). This decrease of activity in the fusiform gyrus was also correlated with the behavioral inversion effect (Yovel & Kanwisher, 2005). This was not the case in the Occipital Face Area (OFA), another face-selective region relying more on features than configuration.

A greater contribution of the right compared with the left hemisphere has been found in the configural processing of faces (Rossion et al., 2000; Schiltz & Rossion, 2006; Goffaux et al., 2009; Rhodes et al., 2009). Lesion studies also showed that patients with right fusiform face area damage show deficits in configural processing (e.g., Barton et al., 2002). Finally, Rossion et al. (2000) argued that this asymmetry towards the right-side was specific to faces.

Indeed, they found that attending to whole faces, rather than parts of faces, enhanced the right FFA response, whereas attending to whole houses rather than parts of houses did not.

Identity

Configural processing might be the basis of identity recognition. Thus it is also crucial to know whether the fusiform gyrus is modulated by different identities perception. Rotshtein et al. (2005) used morphed between famous faces to investigate this question in an fMRI study.

They found that the right fusiform gyrus was sensitive to identity rather than to physical

changes whereas the inferior occipital gyrus showed the contrary. In section 1.2.1, we mentioned a model of face representation, where each face was coded as a function of its distance from a prototype. This model was tested using fMRI, and the activity in the FFA increased as a function of the distance from the average face (Loffler et al., 2005). Loffler et al. (2005) also found adaptation along a whole identity trajectory corresponding to the adapted face. Thus it appears that the fusiform gyrus is sensitive to face configuration and identity. However, the specific role of the fusiform gyrus in identity recognition is not clear yet. Indeed, a recent study on prosopagnosic patients showed that the right ‘OFA’ seems to play a pivotal role in face identity processing (Steeves et al., 2009). The authors’ hypothesis is that the OFA may “contribute to the refinement of the face representation, following the initial categorization of a face in the “higher-order” right FFA”.

One part of this thesis work specifically addresses the question of the configuration/identity processing, which is the core of face recognition.

Experience-dependent neural changes underlying face learning

Many advances have been done in the understanding of face processing and the nature of the representations in the fusiform gyrus. In particular, it seems that the FFA is not only involved in face processing but also in face recognition, and thus, might be the location holding long-term face representations. However, still little is known about how new face representations are acquired and consolidated.

A few studies investigated the effect of familiarity on the FFA, but their results are controversial. Some authors found increased activity in the fusiform gyrus for familiar compared with unfamiliar faces (Henson et al., 2000; Leveroni et al., 2000), while some others found decreased activity for the familiar faces (Dubois et al., 1999; Rossion et al., 2001; Gobbini & Haxby, 2006). This heterogeneity in results might be due to protocol and stimuli differences. However, further investigation is needed to clarify the modulation of face representations by familiarity.

All these studies investigate the effect of learning comparing brain activity for unfamiliar vs.

familiar faces. Very few studies focused on the acquisition of familiarity. Kosaka et al. (2003) investigated this process across nine learning sessions and observed increased activity in the posterior cingulate cortex, decreased activity in the amygdala and in the left medial fusiform gyrus but nothing in the right medial fusiform gyrus. More recently, an ERP study of Tanaka and colleagues (Tanaka et al., 2006) also assessed the neurophysiological mechanisms underlying the acquisition of new face representations. During the course of the experiments, subjects saw their own face, an unfamiliar face whose identity was learned and an unfamiliar

face whose identity was not learned. In the first part of the experiment, only the subject’s own face produced an N250 response. In the second part of the experiment, the learned identity also produced this component. Another component, the P300 appeared earlier in the course of the experiment for the learned face. The P300 is thought to depend upon the hippocampus (Curran et al., 2006; Duzel et al., 2001) and the N250 upon cortical regions in inferior temporal regions, including the fusiform gyrus (George et al., 1997; Schweinberger et al., 2004; Schweinberger et al., 2002). This would be consistent with an early involvement of the hippocampus during learning and more cortical involvement trough multiple repetitions, leading to long-term memory representations (McClelland et al., 1995). These results have been replicated and extended in a very recent study including a large set of faces (Kaufmann et al., 2009).

This part on face recognition highlights the specificity of the face stimuli, and processes underlying face learning and recognition. However, the understanding of the creation of a face representation and of its evolution across several encounters or across time, is still in its early stage. Thus, further investigation is needed in particular to assess how robust a face representation is towards distracters and how it evolves across time.