The Informational Encapsulation Criterion (Refor-

In this section, my aim is to provide a definition of cognitive penetration capable of accommodating both the visual system defined as the conjunction of motor mechanisms, early and late vision and cognitive penetration char-acterised by changes in the architectural and content of content-dependent

and content-independent processes. Such a definition can be given by re-formulating the Informational Encapsulation criterion. The sufficient con-dition is defined as follows:

(1) Internal Causal Link: The visual process P of a visuo-motor system VM is in part causally dependent on a non-visual system Y via an internal causal link.

(2) Computation Condition: The influence of Y on P of VM makes the visual process intelligible due to the visual com-putations that underwrite P using information in Y as a resource.

(3) No Explanatory Defeaters: the resulting visual process P is not explained by changes in the proximal stimulus.

(FIC) If (1), (2) and (3) hold in respect of visual process P and systemY, then the visuo-motor systemVM is not informa-tionally encapsulated from Y (or Y penetrates VM).

Let’s analyse this definition and each condition separately. The first characteristic of this definition is that, ifY stands for the ‘cognitive system’, we have a sufficient condition for the cognitive penetration of the visuo-motor systemVM.

Condition (1) demands an internal and causal connection between the penetrating and the penetrated system. The causal connection must be internal to prevent the following case to count as cognitive penetration:

when a subject decides to read a book, her intention of reading the book influences her motor system by extending her arm, to reach and open the book. This action makes the visual system produce a perceptual experi-ence of the letters printed on the pages. There is a causal relation between the subject’s intention to read the book and her visual experience of the printed letters but this relation is external: the intention influences the mo-tor system which eventually has an indirect consequence on the perceptual

experience. Such a form of influence on the visual system is excluded from any account of cognitive penetration of perception.

In addition, the main characteristic of this condition is that cognitive penetration is not defined in terms of the impact on the content of the perceptual experience. Instead, it is defined as the influence on visual pro-cesses. This interpretation allows us to account for architectural and content changes without appealing to the content of experiences. That is, cognitive penetration can produce either modulations in the signal of the perceptual process by changing the content of perceptual states (content cognitive pen-etration) or changes in the channel of the perceptual process (architectural cognitive penetration). (See section 4.2.)

Furthermore, this condition is silent on the type of mechanism that pen-etrates the visual system. It only states that the visual computation that underwrites P (partially) depends on information stored in the cognitive system. This allows us to sustain (in accordance with my argument in section 4.1) that personal cognitive mental states are the triggers of the subpersonal cognitive mechanisms that eventually penetrate the visual sys-tem. Furthermore, this condition enables us take into consideration cogni-tive influences on content- and non-content-related mechanisms as forms of cognitive penetration of perception.

Condition (2) states that the visual computation that underwrites the process in the visuo-motor system VM uses information stored in the cog-nitive system. The condition involves two intimately connected aspects:

failures in informational encapsulation and the application of some seman-tic criterion. The second part of the condition holds that the visual system doesP using information from the cognitive system (failure in informational encapsulation). The first part states that the cognitive influence makes the visual process P intelligible (semantic explanation).

The second part of the computation condition determines the loci of cognitive penetration — motor processes, early or late vision — since it indicates that the process P must at some point be underwritten by the

information coming from the cognitive state.³² If higher cognitive signals influencing the visual process P enter the system in the time windows of 0 to 100-120 ms (early vision), this might count as cognitive penetration of early vision (strong CP). If the cognitive influences affect the process in the time window between 100-120 and 300 ms, it would count as cognitive penetration of late vision (weak CP).³³

The first part of the condition(2)applies the semantic criterion. To con-sider a cognitive influence on the visual processing as cognitive penetration, the computation of the signal must be semantically or rationally explained (i.e., in an intelligibly coherent way) by appeal to the organism’s goals. That means that there must not be an alternative neurophysiological explanation of the visual system’s behaviour capable of spelling out the penetrated pro-cess. (See section 4.4.1 for an explanation of the semantic criterion.) The semantic dimension blocks counterexamples that can be considered cases of cognitive penetration: e.g., a belief about an exam causes the subject a migraine which produces a mental state of pain which also influences the visual processing resulting in blurry vision (Stokes 2013, 650; Macpherson 2012, 26). This type of counterexample is similar to the heart rate case discussed byPylyshyn (1984, 138, 142) and analysed previously.

It is worth noticing that the computation condition states that two cir-cumstances have to occur: the visual process must compute information coming from higher brain areas and this computation must be rationally (semantically) explained. If only one circumstance occurs the condition is not fulfilled. Let me explain. Cognitive penetration of a visual process de-pends on the modulation of this process by higher brain signals. It could be possible that, in a specific case, cognitive signals do not affect the

vi-32Notice that informational encapsulation is a property of modular systems, such as the early visuo-motor system. Late vision is not modular (see section2.2). Wu’s definition is intimately related to modularity. Nevertheless, this definition can also be applied to late vision.

33Brain imaging techniques such as fMRI (functional magnetic resonance imaging), EEG (electro-encephalography), and MEG (magneto-encephalography) furnish clear ev-idence of top-down signals in the visual system, including their origin and time period.

This claim is supported by empirical evidence. I will scrutinize it in chapters5 and6.

sual system, but the behaviour could still be rationally explained. There could be other cases in which higher signals affect the visual process (e.g., they enhance neural activity facilitating the process) but this influence does not need a semantic explanation; this neural behaviour can be spelled out by biological principles.³⁴ None of these two previous examples would be situations of cognitive penetration of perception.

Finally, condition (3) states that the resulting visual process P is not explained by changes in the proximal stimulus. In my view, this is the only defeater we should consider. Firstly, changes in the focus of attention due to cognitive guidance do count as cognitive penetration. Taking this form of modulation as a defeater will exclude cognitive influences on motor mechanisms as cognitive penetration of perception. Second, changes in the perceptual organ do not have to be considered as defeaters. Cognitive influences might even modulate the state of the visual organ.

In sum, the reformulated informational encapsulation criterion seems to provide an encompassing definition of cognitive penetration which allows us to distinguish cognitive penetration of motor, early and late visual processes, producing changes in the content as well as in the architecture of the system (content- and non-content-related processes).

In the next two chapters, I will scrutinize content and architectural cog-nitive penetration of the visual system in the light of empirical studies. The goal will be to argue for the existence of cognitive penetration of the per-ceptual experience by cognitive influences on early vision. The experiments I will examine include two aspects present in all the debates on cognitive penetration: cognitive penetration of early vision and cognitive penetration of perceptual experience.

34As I explained in section 2.1.1 there are cases in which cognitive influences affect early vision, but according toPylyshyn(1999a, 359-360) andRaftopoulos(2009a, 79-88) they do not count as cognitive penetration: these influences only facilitate the visual processing but do not modify it. They could also be task-irrelevant cognitive influences (see section3.2).

5

Content Cognitive Penetration

Contents

5.1 Psychological Studies and Content Cognitive Penetration193

5.2 Content Cognitive Penetration of Late Vision. . . 200 5.3 Content Cognitive Penetration of Early Vision . . . 204

5.3.1 Synchronic and Diachronic Content Cognitive Penetration . . . 215

In this chapter and in the next one I will examine neuroscientific studies that demonstrate that there is cognitive penetration in early vision; the consequence of the penetration influences the experience. However, before starting I would like to explain the importance of neuroscientific studies in the cognitive penetrability debate.

Frequently, philosophical arguments for or against cognitive

penetrabil-ity of perception rely on psychological studies. The subjects’ verbal report or behaviour is taken as evidence to decide whether the perceptual expe-rience has or has not been cognitively penetrated. However, using solely psychological evidence is not sufficient to show any form of cognitive pen-etration of perception. First of all, verbal reports alone do not allow us to distinguish between cases of cognitive penetration of perception or post-perceptual cognitive effects. For instance, after being exposed to a series of pictures of female faces, new faces are perceived as more man look-alike (Webster et al. 2004); our knowledge of the colour of objects (e.g., ba-nanas) might affect how we see these objects (e.g., more yellow than they are) (Hansen et al. 2006; Olkkonen et al. 2008); and so on. Some philoso-phers might claim that new faces look more man-alike or objects look more coloured because of cognitive penetration of perception (memory modifies what the subject sees). However, this is not the only interpretation. The cases could be explained by appeal to the adaptation of the visual system itself to the frequent exposure of faces or objects (Fodor 1983, 193) or as a case in which the subjects judge what they think to be the case rather than what they see — this is known as the judgement problem (Deroy 2013, 97-99; Lyons 2011, 304-305; Macpherson 2012, 39-42; Pylyshyn 1984, 135;

2003, 40-44; Siegel 2012, 206; Stokes 2012, 485-486;2013, 655-656;2014, 5-6;Stokes and Bergeron ming, 7-8;Vetter and Newen 2014, 65-66;Zeimbekis 2013).¹ With regard to this observation Lyons writes:

[S]ome, maybe much, of what seems to be perceptual penetration may actually be post-perceptual. One further possibility in the yel-low banana case is [that] the experiential state is not affected, but because of their knowledge that bananas are yellow, subjects think they are being appeared to yellowly, even though they’re not. [...]

1Notice that these cases illustrate failures in the computation condition of the refor-mulated definition of cognitive penetration (FIC). In the former case the influence affects the visual system but the effect does not need to be explained by rational principles. In the former cases, the processes can be semantically explained (it actually takes place at the visual level) but the cognitive influence does not affect the visual system. See section 4.4.5.

tion is no less intractable than perceptual penetration. Once again, the protracted scientific and philosophical debate about such exam-ples indicates that subjects cannot introspectively tell whether their perception is cognitively penetrated and if so, at which locus. (Lyons 2011, 304-305)

Then, verbal or conscious reports do not help to distinguish between cognitive penetration of perception and of cognition. What might look like a case of cognitive penetration could in fact be an entirely post-perceptual phenomenon.

The second problem is that psychological evidence does not allow us to distinguish if the content of the perceptual experience changed due to cognitive influences or in virtue of other forms of penetration of perception (e.g., cross-modal). In section4.4.2.1I presented a few cases of cross-modal penetration. Namely, I examined an empirical study byWanab et al.(2015) showing visual penetration of the gustatory system (the colour of beverages influences how they taste) (see also Harrar et al. (2011);Piqueras-Fiszman and Spence(2012)). Psychological reports do not allow to rule out the pos-sibility that the content of the perceptual experience was changed by intra-modal, cross-intra-modal, motor, or any kind of non-cognitive influence rather than by cognitive penetration. Again, psychological studies alone do not help to account for the complexity of cognitive penetration of perception (a phenomenon that implies unconscious processes).

The third problem with psychological studies is the locus of the cognitive penetration of perception: does cognition affect early or late vision? If we use subjects’ conscious reports to determine whether there is or not cognitive penetration, all we can show at best is that there is cognitive penetration of the experience. That is, the kind of cognitive penetration of perceptual content which eventually affects the perceptual experience. But verbal or behavioural reports do not tell us the level of the influence in the visual system: whether cognition affects early stages of visual processing or only

late vision.² Pylyshyn writes:

At some level beyond the transducer, where what is perceived may become available to consciousness, perception is largely symbolic and knowledge-based. That is the burden of demonstrations constituting the ‘new look’ movement in perception research (for example,Bruner 1957). Thus, although transducers provide symbolic input to the complex, cognitive processes involved in perception,a psychophysical experiment cannot bypass the cognitive system and directly examine the output of the transducers. (Pylyshyn 1984, 174; my italics; see also Pylyshyn 1980, 112.)

Pylyshyn remarks that psychological studies do not help to solve the problem of cognitive penetration because we only have access to the final result of the perceptual process: perceptual experiences, actions, and the like. In the previous quotation he specifies that we cannot examine the outputs of early vision to decide if the penetrated system was late or early vision. He argues:

[T]he occurrence of visual experience in and of itself need not always indicate that the visual system is involved. In the case of visual ex-periences arising from hallucinations, dreams, and direct brain stim-ulation, it is not obvious that any visual information processing is occurring, or even that what I have called the early-vision system is directly involved. (Pylyshyn 2003, 128-129)

A great many studies suggest that unconscious states play the same role in cognition, as do conscious states (e.g., stimuli of which we have no conscious awareness appear to influence perception and attention the same way as stimuli of which we are aware; see Merikle et al.

2001). But there are also some different information-processing and

2Very roughly, higher cognitive influences on the visual processing observed in the time windows from 0 to 120 ms may count as cognitive penetration of early vision (strong CP).

Likewise, higher effects detected in between 120 and 300 ms after stimulus presentation represent cognitive penetration of late vision (weak CP). And any influence posterior to this time period is a form of cognitive penetration of cognition.

(Dehaene and Naccache 2001; Driver and Vuilleumier 2001; Kan-wisher 2001). What I have argued, here and elsewhere, is just that we are not entitled to take the content of our experience as reflecting, in any direct way, the nature of the information-processing activity (what Pessoa et al. 1998 call the “analytical isomorphism” assump-tion). In particular, the evidence does not entitle us to conclude that episodes that we experience as seeing in one’s mind’s eye involve ex-amining uninterpreted, spatially displayed depictive representations (i.e., pictures) using the early-vision system. (Pylyshyn 2003, 356)

The previous explanation shows that psychological studies neither ac-count for cognitive penetration of early vision nor explain that changes in the perceptual experience reflect cognitive modulation in early visual pro-cessing.³ In other words, they are not a sufficient method to decide whether there is cognitive penetration of the visual system. So, because one of the aims of this thesis is to show cognitive penetration of early vision, we need some method capable of providing any evidence of when and where the cog-nitive signal influences the visual system. This method is supplied by brain imaging techniques used in neurophysiological studies.

Brain imaging techniques provide both a highly detailed topographic map of brain activation (e.g., fMRI, functional magnetic resonance imag-ing) and a high time resolution of such activity (e.g., EEG, electro-encephalography, and MEG, magneto-encephalography). Both aspects are essential to assess cognitive penetration of perception at early and late visual stages. For instance, fMRI techniques supply very detailed neuroanatomical

3It is worth noticing thatPylyshyn(1999a, e.g., 344) andFodor(1988, e.g., 193-194) frequently use psychological studies involving perceptual experiences and conscious re-ports (Ames room, M¨uller-Lyer illusion, Ponzo illusion, Hering illusion) to argue against cognitive penetration of early vision. However, from cognitive penetration of the expe-rience does not follow that there is cognitive penetration of early vision and cognitive penetration of early vision does not imply cognitive penetration of the experience. That is, cognitive penetration of the experience could occur due to changes in perceptual content at the late visual level; and cognitive penetration of early vision might affect perceptual content, e.g., for action, but not the content of the perceptual experience.

SeeBullier(1999) andMacpherson(2012, 27, fn. 1) for a similar observation.

and functional brain activation maps. These methods provide very rich ev-idence to assess which brain areas have been activated (perceptual, motor, or cognitive) during a perceptual task. EEG and MEG techniques have a high time resolution capable to assess the origin and destination of electric signals in the order of 2 milliseconds. Time information is fundamental to estimate the time course of brain electric activity (whether signals affect early or late vision). Another advantage of these techniques is that they can assess abundant and clear evidence of the unconscious brain processes (motor, perceptual, semantic, emotional) which psychological studies can-not access.

This argument contrasts with Pylyshyn’s claim that brain imaging tech-niques cannot provide clear information of brain activation because they are too coarse to detect cognitive influences in early vision:

[W]e identified certain shortcomings in using signal detection mea-sures to establish the locus of cognitive effects and argued that although event-related potentials might provide timing measures that are independent of response-preparation, the stages they dis-tinguished are also too coarse to factor out such memory-accessing decision functions as those involved in recognition. So, as in so many examples in science, there is no simple and direct method — no methodological panacea — for answering the question whether a particular observed effect has its locus in [early] vision or in pre-or post-visual processes. (Pylyshyn 1999a, 364)

However, we need to take into account that Pylyshyn wrote this article before the flourishing of brain imaging techniques.⁴

Here is a possible example of the advantages of brain imaging techniques.

Certain neurons in the visual cortex are highly specialized in specific stimuli detection (e.g., orientation, size, position, colour, and the like).⁵ During a perceptual task demanding the detection of a stimulus orientation, imaging

4An analogy can be done with computers and phones. Since the beginning of the 2000s there has been an important revolution in terms of laptops and smartphones.

5Neurons in the primary visual cortex are highly specialized in location, orienta-tion, size, and monocular discrimination (Karni and Sagi 1991, 4966, 4969; Ahissar and

cortex) and the area affected (e.g., the primary visual cortex), the property trained during the task (e.g., neural networks sensitive to orientation), the time course of the influence (e.g., it happens in early or late vision), and finally at some point the behavioural consequences on the visual system (e.g., modification of the perceptual content). (See chapter 6 for a scrutiny of this and other cases.)

cortex) and the area affected (e.g., the primary visual cortex), the property trained during the task (e.g., neural networks sensitive to orientation), the time course of the influence (e.g., it happens in early or late vision), and finally at some point the behavioural consequences on the visual system (e.g., modification of the perceptual content). (See chapter 6 for a scrutiny of this and other cases.)