Synchronic and Diachronic Content Cognitive Pene-

In this chapter and in the next one I will examine neuroscientific studies that demonstrate that there is cognitive penetration in early vision; the consequence of the penetration influences the experience. However, before starting I would like to explain the importance of neuroscientific studies in the cognitive penetrability debate.

Frequently, philosophical arguments for or against cognitive

penetrabil-ity of perception rely on psychological studies. The subjects’ verbal report or behaviour is taken as evidence to decide whether the perceptual expe-rience has or has not been cognitively penetrated. However, using solely psychological evidence is not sufficient to show any form of cognitive pen-etration of perception. First of all, verbal reports alone do not allow us to distinguish between cases of cognitive penetration of perception or post-perceptual cognitive effects. For instance, after being exposed to a series of pictures of female faces, new faces are perceived as more man look-alike (Webster et al. 2004); our knowledge of the colour of objects (e.g., ba-nanas) might affect how we see these objects (e.g., more yellow than they are) (Hansen et al. 2006; Olkkonen et al. 2008); and so on. Some philoso-phers might claim that new faces look more man-alike or objects look more coloured because of cognitive penetration of perception (memory modifies what the subject sees). However, this is not the only interpretation. The cases could be explained by appeal to the adaptation of the visual system itself to the frequent exposure of faces or objects (Fodor 1983, 193) or as a case in which the subjects judge what they think to be the case rather than what they see — this is known as the judgement problem (Deroy 2013, 97-99; Lyons 2011, 304-305; Macpherson 2012, 39-42; Pylyshyn 1984, 135;

2003, 40-44; Siegel 2012, 206; Stokes 2012, 485-486;2013, 655-656;2014, 5-6;Stokes and Bergeron ming, 7-8;Vetter and Newen 2014, 65-66;Zeimbekis 2013).¹ With regard to this observation Lyons writes:

[S]ome, maybe much, of what seems to be perceptual penetration may actually be post-perceptual. One further possibility in the yel-low banana case is [that] the experiential state is not affected, but because of their knowledge that bananas are yellow, subjects think they are being appeared to yellowly, even though they’re not. [...]

1Notice that these cases illustrate failures in the computation condition of the refor-mulated definition of cognitive penetration (FIC). In the former case the influence affects the visual system but the effect does not need to be explained by rational principles. In the former cases, the processes can be semantically explained (it actually takes place at the visual level) but the cognitive influence does not affect the visual system. See section 4.4.5.

tion is no less intractable than perceptual penetration. Once again, the protracted scientific and philosophical debate about such exam-ples indicates that subjects cannot introspectively tell whether their perception is cognitively penetrated and if so, at which locus. (Lyons 2011, 304-305)

Then, verbal or conscious reports do not help to distinguish between cognitive penetration of perception and of cognition. What might look like a case of cognitive penetration could in fact be an entirely post-perceptual phenomenon.

The second problem is that psychological evidence does not allow us to distinguish if the content of the perceptual experience changed due to cognitive influences or in virtue of other forms of penetration of perception (e.g., cross-modal). In section4.4.2.1I presented a few cases of cross-modal penetration. Namely, I examined an empirical study byWanab et al.(2015) showing visual penetration of the gustatory system (the colour of beverages influences how they taste) (see also Harrar et al. (2011);Piqueras-Fiszman and Spence(2012)). Psychological reports do not allow to rule out the pos-sibility that the content of the perceptual experience was changed by intra-modal, cross-intra-modal, motor, or any kind of non-cognitive influence rather than by cognitive penetration. Again, psychological studies alone do not help to account for the complexity of cognitive penetration of perception (a phenomenon that implies unconscious processes).

The third problem with psychological studies is the locus of the cognitive penetration of perception: does cognition affect early or late vision? If we use subjects’ conscious reports to determine whether there is or not cognitive penetration, all we can show at best is that there is cognitive penetration of the experience. That is, the kind of cognitive penetration of perceptual content which eventually affects the perceptual experience. But verbal or behavioural reports do not tell us the level of the influence in the visual system: whether cognition affects early stages of visual processing or only

late vision.² Pylyshyn writes:

At some level beyond the transducer, where what is perceived may become available to consciousness, perception is largely symbolic and knowledge-based. That is the burden of demonstrations constituting the ‘new look’ movement in perception research (for example,Bruner 1957). Thus, although transducers provide symbolic input to the complex, cognitive processes involved in perception,a psychophysical experiment cannot bypass the cognitive system and directly examine the output of the transducers. (Pylyshyn 1984, 174; my italics; see also Pylyshyn 1980, 112.)

Pylyshyn remarks that psychological studies do not help to solve the problem of cognitive penetration because we only have access to the final result of the perceptual process: perceptual experiences, actions, and the like. In the previous quotation he specifies that we cannot examine the outputs of early vision to decide if the penetrated system was late or early vision. He argues:

[T]he occurrence of visual experience in and of itself need not always indicate that the visual system is involved. In the case of visual ex-periences arising from hallucinations, dreams, and direct brain stim-ulation, it is not obvious that any visual information processing is occurring, or even that what I have called the early-vision system is directly involved. (Pylyshyn 2003, 128-129)

A great many studies suggest that unconscious states play the same role in cognition, as do conscious states (e.g., stimuli of which we have no conscious awareness appear to influence perception and attention the same way as stimuli of which we are aware; see Merikle et al.

2001). But there are also some different information-processing and

2Very roughly, higher cognitive influences on the visual processing observed in the time windows from 0 to 120 ms may count as cognitive penetration of early vision (strong CP).

Likewise, higher effects detected in between 120 and 300 ms after stimulus presentation represent cognitive penetration of late vision (weak CP). And any influence posterior to this time period is a form of cognitive penetration of cognition.

(Dehaene and Naccache 2001; Driver and Vuilleumier 2001; Kan-wisher 2001). What I have argued, here and elsewhere, is just that we are not entitled to take the content of our experience as reflecting, in any direct way, the nature of the information-processing activity (what Pessoa et al. 1998 call the “analytical isomorphism” assump-tion). In particular, the evidence does not entitle us to conclude that episodes that we experience as seeing in one’s mind’s eye involve ex-amining uninterpreted, spatially displayed depictive representations (i.e., pictures) using the early-vision system. (Pylyshyn 2003, 356)

The previous explanation shows that psychological studies neither ac-count for cognitive penetration of early vision nor explain that changes in the perceptual experience reflect cognitive modulation in early visual pro-cessing.³ In other words, they are not a sufficient method to decide whether there is cognitive penetration of the visual system. So, because one of the aims of this thesis is to show cognitive penetration of early vision, we need some method capable of providing any evidence of when and where the cog-nitive signal influences the visual system. This method is supplied by brain imaging techniques used in neurophysiological studies.

Brain imaging techniques provide both a highly detailed topographic map of brain activation (e.g., fMRI, functional magnetic resonance imag-ing) and a high time resolution of such activity (e.g., EEG, electro-encephalography, and MEG, magneto-encephalography). Both aspects are essential to assess cognitive penetration of perception at early and late visual stages. For instance, fMRI techniques supply very detailed neuroanatomical

3It is worth noticing thatPylyshyn(1999a, e.g., 344) andFodor(1988, e.g., 193-194) frequently use psychological studies involving perceptual experiences and conscious re-ports (Ames room, M¨uller-Lyer illusion, Ponzo illusion, Hering illusion) to argue against cognitive penetration of early vision. However, from cognitive penetration of the expe-rience does not follow that there is cognitive penetration of early vision and cognitive penetration of early vision does not imply cognitive penetration of the experience. That is, cognitive penetration of the experience could occur due to changes in perceptual content at the late visual level; and cognitive penetration of early vision might affect perceptual content, e.g., for action, but not the content of the perceptual experience.

SeeBullier(1999) andMacpherson(2012, 27, fn. 1) for a similar observation.

and functional brain activation maps. These methods provide very rich ev-idence to assess which brain areas have been activated (perceptual, motor, or cognitive) during a perceptual task. EEG and MEG techniques have a high time resolution capable to assess the origin and destination of electric signals in the order of 2 milliseconds. Time information is fundamental to estimate the time course of brain electric activity (whether signals affect early or late vision). Another advantage of these techniques is that they can assess abundant and clear evidence of the unconscious brain processes (motor, perceptual, semantic, emotional) which psychological studies can-not access.

This argument contrasts with Pylyshyn’s claim that brain imaging tech-niques cannot provide clear information of brain activation because they are too coarse to detect cognitive influences in early vision:

[W]e identified certain shortcomings in using signal detection mea-sures to establish the locus of cognitive effects and argued that although event-related potentials might provide timing measures that are independent of response-preparation, the stages they dis-tinguished are also too coarse to factor out such memory-accessing decision functions as those involved in recognition. So, as in so many examples in science, there is no simple and direct method — no methodological panacea — for answering the question whether a particular observed effect has its locus in [early] vision or in pre-or post-visual processes. (Pylyshyn 1999a, 364)

However, we need to take into account that Pylyshyn wrote this article before the flourishing of brain imaging techniques.⁴

Here is a possible example of the advantages of brain imaging techniques.

Certain neurons in the visual cortex are highly specialized in specific stimuli detection (e.g., orientation, size, position, colour, and the like).⁵ During a perceptual task demanding the detection of a stimulus orientation, imaging

4An analogy can be done with computers and phones. Since the beginning of the 2000s there has been an important revolution in terms of laptops and smartphones.

5Neurons in the primary visual cortex are highly specialized in location, orienta-tion, size, and monocular discrimination (Karni and Sagi 1991, 4966, 4969; Ahissar and

cortex) and the area affected (e.g., the primary visual cortex), the property trained during the task (e.g., neural networks sensitive to orientation), the time course of the influence (e.g., it happens in early or late vision), and finally at some point the behavioural consequences on the visual system (e.g., modification of the perceptual content). (See chapter 6 for a scrutiny of this and other cases.)

However, notice that neuroscientific studies alone are not sufficient either to provide an explanation of cognitive penetration of perception. Pylyshyn claims:

[O]nly a small fraction of physically discriminable states of a system are computationally, or cognitively, distinct. Furthermore, the com-putational states consist of classes of physical states that may have no natural, or even finite, physical characterization. This fact has the following far-reaching implication: The mere fact that an organ-ism can be shown to respond neurophysiologically to a certain physi-cally defined stimulus property such as wavelength does not mean this property is cognitively or computationally relevant. (Pylyshyn(1984, 172); see also Pylyshyn (1999a, 347;2003, 69-70))

It is not the visual complexity of the class to which the cell responds, nor whether the cell is modulated in a top-down manner that is at issue, but whether or not the cell responds to how a visual pattern Hochstein1993, 5718;Crist et al. 2001, 519;Schwartz et al. 2002, 17137, 17139;2007, 28;

Bear et al. 2007, 325-326). Neurons in higher stages of the visual system are sensitive to more complex stimuli. Extrastriate cortical areas (V2 and V3) appear to be sensitive to global detection (Ahissar and Hochstein 1993, 5720); area V3 seems to be specialized in processing of dynamic forms (Tov´ee 2008, 69); area V4 appears to process colour (Val-berg 2005, 309); and motion detection and stereoscopic depth recruits neurons in V5 or MT (Ahissar and Hochstein 1993, 5718; Crist et al. 2001, 519; Valberg 2005, 400-401;

Tov´ee 2008, 69; see also Jacob and Jeannerod 2003, 57-71 for a detailed explanation of the properties encoded by other brain areas). See also section 6.4.2 and figure3 in appendix B.) In addition, certain brain areas are specialized in specific functions. For example, face perception processing recruits neurons in temporal and occipital areas of the fusiform gyrus: the fusiform face area (FFA), the occipital face area (OFA), and face-selective region in the superior temporal sulcus (fSTS) (Kanwisher et al. 1997;Kanwisher 2001;Liu et al. 2010). (I discuss face perception as an example of cognitive penetration in early vision in section5.3).