Pavlovian conditioning paradigms in humans

In this section, we provide an overview of the main methods used to investigate Pavlovian conditioning in humans. To this end, we briefly review the stimuli, procedures, conditioning phases, and measures that are typically employed in human Pavlovian conditioning research (see Lonsdorf et al., 2017, for a more detailed review on methodological considerations in human Pavlovian aversive conditioning studies), and specify the methodology used in the experiments reported in the experimental part of this thesis (see chapter 3).

Conditioned stimuli

In humans, Pavlovian conditioning studies predominantly use discrete exteroceptive cues as CSs (Lonsdorf et al., 2017; but see De Peuter, Van Diest, Vansteenwegen, Van den Bergh, & Vlaeyen, 2012, for a review on interoceptive aversive conditioning). Typical CSs consist of visual stimuli, such as colored squares (e.g., LaBar et al., 1998; Phelps et al., 2004;

Schiller et al., 2010), geometric shapes (e.g., Gottfried, O’Doherty, & Dolan, 2003), human faces (e.g., Öhman & Dimberg, 1978; Öhman & Mineka, 2001; Olsson et al., 2005; Olsson &

Phelps, 2004) or images of animals (e.g., Ho & Lipp, 2014; Öhman et al., 1976; Öhman &

Mineka, 2001; Olsson et al., 2005), though auditory, olfactory, gustatory, and tactile CSs have also been employed (Lonsdorf et al., 2017). In our studies (see chapter 3), we used visual CSs corresponding to images of threat-relevant stimuli (angry faces and snakes in Study 1, angry faces in Study 2), positive relevant stimuli (baby faces and erotic stimuli in Study 1, happy faces in Study 2), and neutral stimuli (neutral faces and colored squares in Study 1, neutral faces in Study 2, geometric figures in Study 3 and 4).

Unconditioned stimuli

In human Pavlovian aversive conditioning paradigms, the most commonly used and among the most efficient USs are electro-tactile stimulations, which consist of the delivery of mild electric currents to the skin, and auditory stimuli, such as loud noise or human screams (Lonsdorf et al., 2017). Interoceptive discomfort (e.g., Pappens, Smets, Vansteenwegen, Van den Bergh, & Van Diest, 2012) and unpleasant odors (e.g., Gottfried, O’Doherty, & Dolan, 2002; Hermann et al., 2000), as well as secondary reinforcers, such as loss of money (e.g., Delgado, Jou, & Phelps, 2011; Delgado, Labouliere, & Phelps, 2006), have also been employed as aversive USs. As electric stimulation constitutes a highly, if not the most, efficient aversive

US, it was used as US in all our experiments employing a Pavlovian aversive conditioning paradigm (see chapter 3).

Pavlovian appetitive conditioning studies in humans have both utilized primary reinforcers, such as food (e.g., Andreatta & Pauli, 2015), water (e.g., Kumar et al., 2008), tastes (e.g., Prévost, McNamee, Jessup, Bossaerts, & O’Doherty, 2013), pleasant odors (e.g., Gottfried et al., 2002, 2003), or erotic pictures (e.g., Klucken et al., 2009), and secondary reinforcers, such as money (e.g., Austin & Duka, 2010), as appetitive USs. Of note, these appetitive USs however typically (a) elicit physiological reactions that are less intense (e.g., Martin-Soelch et al., 2007), (b) are more sensitive to the organism’s psychological and physiological state (e.g., the organism generally needs to be in a hunger state for food to be rewarding; Clark, Hollon, & Phillips, 2012), and (c) are more difficult to administer (see, e.g., Andreatta & Pauli, 2015) than aversive USs, such as electric stimulations, thereby making Pavlovian appetitive conditioning processes more complex than Pavlovian aversive conditioning ones to investigate in humans. In Study 4 (see chapter 3.4), we used a pleasant odor as appetitive US that was individually selected among various pleasant odors according to liking and intensity ratings in order to ensure it was rewarding for each participant and constituted an appropriate appetitive US.

Procedures

Single-cue Pavlovian conditioning procedures and differential Pavlovian conditioning procedures are typically used to study Pavlovian conditioning in animals and humans (e.g., Lonsdorf et al., 2017). In single-cue procedures, a single CS (or a compound of CSs) is associated with the US. The CRs elicited by the CS are then compared either with the responses of a control group (e.g., receiving the same number of US administrations that are not contingent upon the CS presentation or explicitly unpaired with the CS) or with the responses in the absence of the CS (e.g., during the intertrial interval), thus involving exclusively between-subject comparisons (e.g., Lonsdorf et al., 2017; Rescorla, 1967). While these procedures are the most used in rodents, they are not very common in humans (Lonsdorf et al., 2017). Human research on Pavlovian conditioning indeed mostly employs differential procedures (Lonsdorf et al., 2017). In a differential Pavlovian conditioning procedure (see Figure 2.3), two types of CSs are used: One stimulus (the reinforced stimulus, CS+) is contingently paired with the US, whereas another stimulus (the unreinforced stimulus, CS-) is never associated with the US. The CRs are operationalized as the differential response to the CS+ versus the CS- (e.g., Lonsdorf et al., 2017; Olsson et al., 2005). Differential Pavlovian

conditioning procedures thereby allows for controlling for (a) between-subject differences in responding, (b) nonassociative processes (e.g., orienting responses, habituation, sensitization) that are thought to affect the responses to the CS+ and the CS- in a similar manner, and (c) the possible confounding role of preexisting differences in the CS categories’ emotional salience when several categories of CS are used and compared with each other in the same experiment (e.g., Lonsdorf et al., 2017; Olsson et al., 2005). In a similar vein, differential Pavlovian conditioning procedures provide more statistical power than single-cue procedures (Lonsdorf et al., 2017). It is however important to note that the CS- may not constitute a completely neutral control stimulus as it signals the absence of the US, hence possibly inducing inhibitory learning (Lonsdorf et al., 2017). Because of its many advantages over single-cue procedures, a differential Pavlovian conditioning paradigm was used in each study of the experimental part of this thesis (see chapter 3).

Conditioning phases

Differential Pavlovian conditioning paradigms in humans generally include three successive and distinctive phases: habituation, acquisition, and extinction (see Figure 2.3).

During the habituation phase, all the CSs are presented repeatedly without being associated with the US. This phase notably serves to (a) establish a baseline response rate, allowing to determine and correct for potential pre-existing differences in responding to the to-be-CS+ and the to-be-CS-, and (b) to reduce the initial reactivity of certain physiological measures that show a decline in responding over the first number of trials (e.g., orienting responses; Lonsdorf et al., 2017). The habituation phase may be preceded by a US calibration or selection procedure.

US calibration procedures are commonly implemented in Pavlovian aversive conditioning paradigms, with the aim of individually adjusting the US intensity (mostly for electric stimulations) to a pre-defined subjective level of unpleasantness or aversiveness across participants (Lonsdorf et al., 2017). US selection procedures are also sometimes used in Pavlovian appetitive conditioning studies (e.g., Stussi, Delplanque, et al., 2018) to individually select the most liked stimulus as appetitive US, thus optimizing its chances to have rewarding properties for each and every participant. In the acquisition phase, the CS+ is contingently paired with the US, while the CS- is never associated with it. In humans, delay conditioning and, to a lesser extent, trace conditioning procedures are predominantly used given their greater efficiency to produce Pavlovian conditioning. Finally, during the extinction phase, all the CSs are presented in the absence of the US, the delivery of which is discontinued. These three phases (habituation, acquisition, and extinction) were included in all the experiments that we

performed (see chapter 3). Studies investigating the phenomena of the CR recovery after extinction may contain additional phases after extinction, such as reextinction, reinstatement, or renewal for instance (e.g., Schiller et al., 2010; see Haaker, Golkar, Hermans, & Lonsdorf, 2014, for a review on human reinstatement studies).

Conditioned response measures

Table 2.1 summarizes the main dependent variables used in the existing literature to measure the CR in humans (see also Lonsdorf et al., 2017, for a review of the outcome measures used in human Pavlovian aversive conditioning research). Psychophysiological indicators are commonly applied to index Pavlovian conditioning (see, e.g., Leuchs, Schneider,

& Spoormaker, 2019). Among those, the most widely employed index of the CR is electrodermal activity (EDA), and more specifically skin conductance response (SCR). EDA is primarily under sympathetic control and originates from the eccrine sweat glands, which are most dense on the palms and soles of the feet (Dawson, Schell, & Fillion, 2016). Whereas eccrine sweat glands primary function is thermoregulation, those located on the palms have been suggested to be more responsive to emotional than thermal stimuli (Dawson et al., 2016), thereby reflecting EDA’s sensitivity to emotional processes. SCR is considered a valid indicator of autonomic arousal and corresponds to a phasic response to a stimulus that triggers an increase in skin conductance (e.g., Lykken & Venables, 1971). Accordingly, the CS+

typically elicits SCRs of larger amplitude than the CS- during Pavlovian conditioning

H

A

E

CS+

CS-US US

US US

Figure 2.3. Illustration of a differential Pavlovian conditioning procedure. During the habituation phase, the conditioned stimuli (CSs; e.g., colored squares) are presented without being reinforced. In the acquisition phase, the reinforced CS (CS+) is associated with the unconditioned stimulus (US), whereas the unreinforced CS (CS-) is never paired with the US. In the extinction phase, the US is no longer delivered.

(Lonsdorf et al., 2017). Whereas SCR has been extensively used as a reliable and sensitive psychophysiological indicator of Pavlovian aversive conditioning processes in humans, Pavlovian appetitive conditioning studies using SCR as an indicator of appetitive CRs have yielded rather mixed results, some studies reporting enhanced SCRs to the CS+ associated with the appetitive US compared with the CS- (e.g., Andreatta & Pauli, 2015; Klucken et al., 2009), while others observing no difference in SCR to the CS+ versus the CS- (e.g., Hermann et al., 2000; Stussi, Delplanque, et al., 2018). It has been argued that this asymmetry might arise from the fact that SCR may be particularly sensitive to the US intensity, hence possibly failing to detect subtle changes caused by Pavlovian appetitive conditioning due to a weaker US intensity than in Pavlovian aversive conditioning (Stussi, Delplanque, et al., 2018). As SCR is one of the most well-established psychophysiological measures of Pavlovian conditioning in humans, it was used as the main dependent variable of the CR in all the experiments that we conducted (see chapter 3).

Another widely applied psychophysiological measure of the CR in human Pavlovian conditioning research is the startle reflex, which is an automatic defensive response to a sudden, intense, and unexpected sensory event. The startle reflex is elicited with a startle probe typically consisting of a brief white noise burst (acoustic startle probe). In humans, the startle reflex is frequently indexed with the eyeblink reflex. The eyeblink reflex has been shown to be potentiated in response to a CS+ paired with an aversive US (e.g., Andreatta & Pauli, 2015;

Grillon, 2002; Grillon & Davis, 1997; Hamm, Greenwald, Bradley, & Lang, 1993), and attenuated in response to a CS+ paired with an appetitive US (Andreatta & Pauli, 2015; but see Bradley, Zlatar, & Lang, 2018; Stussi, Delplanque, et al., 2018), relative to a CS-. We used startle eyeblink reflex as a measure of appetitive CRs in Study 4 (see chapter 3.4) in order to assess its sensitivity in measuring Pavlovian appetitive learning in comparison with the postauricular reflex. Other common but less frequently used psychophysiological measures of Pavlovian conditioning include heart rate (e.g., Hamm et al., 1993), finger-pulse volume responses (e.g., Hamm et al., 1989), and pupillary response (e.g., Korn, Staib, Tzovara, Categnetti, & Bach, 2017; Leuchs et al., 2019; O’Doherty et al., 2003; Reinhard & Lachnit, 2002).

At the neural level, electroencephalography (EEG), magnetoencephalography (MEG), and functional magnetic resonance imaging (fMRI) have been used to measure Pavlovian CRs.

Research using EEG (e.g., Bacigalupo & Luck, 2018; Stolarova, Keil, & Moratti, 2006;

Wieser, Miskovic, Rausch, & Keil, 2014) and MEG (e.g., Yuan, Giménez-Fernández,

Méndez-Bértolo, & Moratti, 2018) has indicated that Pavlovian learning induces amplification of cortical sensory and attentional processing of the CS+ versus the CS- mainly in aversive (for a review, see Miskovic & Keil, 2012) but also in appetitive paradigms (e.g., Franken, Huijding, Nijs, & van Strien, 2011). Studies using fMRI have shown that the acquisition, expression, and extinction of CRs during Pavlovian aversive conditioning involve a distributed neural network, including in particular the amygdala (e.g., Büchel et al., 1998; LaBar et al., 1998; Phelps et al., 2004; Sehlmeyer et al., 2009), the anterior cingulate cortex, the insula, the hippocampus, the striatum, the ventromedial prefrontal cortex, the thalamus, and motor and sensory cortices (see Fullana et al., 2016; Sehlmeyer et al., 2009, for a recent meta-analysis and a systematic review, respectively). In comparison, effects of Pavlovian appetitive conditioning have been observed across a distributed brain network, including in particular the amygdala, the striatum, the orbitofrontal cortex, and the anterior cingulate cortex (e.g., Gottfried et al., 2002, 2003; Martin-Soelch et al., 2007; O’Doherty et al., 2003).

Behavioral measures are also sometimes used as indices of Pavlovian CRs. They essentially consist of reaction time measures for detecting the CSs (e.g., Gottfried et al., 2003;

Pool, Brosch, Delplanque, & Sander, 2015), generally resulting in faster reaction times in response to the CS+ than to the CS-. Reaction times have also been used to index avoidance and approach tendencies (e.g., Krypotos, Effting, Arnaudova, Kindt, & Beckers, 2014; Van Gucht, Vansteenwegen, Van den Bergh, & Beckers, 2008), with faster reaction times to avoid the CS+ compared with the CS- in Pavlovian aversive conditioning and faster reaction times to approach the CS+ relative to the CS- in Pavlovian appetitive conditioning. Reaction times can be measured trial-by-trial during the Pavlovian conditioning procedure (e.g., Pool et al., 2015), intermittently after each conditioning phase or group of trials, or even retrospectively after the entire conditioning procedure (e.g., Van Gucht et al., 2008).

Subjective ratings or verbal reports provide another complementing measure of Pavlovian conditioning at the behavioral level. Subjective ratings include for instance CS-US contingency or US expectancy ratings (e.g., Boddez et al., 2013), CS pleasantness (or valence or liking) ratings, CS arousal ratings, and subjective feeling ratings, such as fear ratings in Pavlovian aversive conditioning (e.g., Lonsdorf et al., 2017). CS-US contingency ratings assess the extent to which the CS+ and the CS- are deemed predictive of the US, and provide an indicator of whether participants are aware of the contingencies between the CSs and the US (but see Boddez et al., 2013; Lovibond & Shanks, 2002, for discussions on methodological considerations for these ratings). Ratings of the CS pleasantness evaluate to what extent the

Table 2.1

Overview of the main measures of the conditioned response commonly used in human Pavlovian conditioning paradigms.

Level Measure Paradigm Measurement time Typical effect of Pavlovian conditioning

Psychophysiological Skin conductance response Aversive and appetitive Continuous CS+ > CS-

Startle eyeblink reflex Aversive and appetitive Continuous Aversive: CS+ > CS-; Appetitive: CS+ < CS-

Heart rate Aversive Continuous CS+ < CS- (orienting response) or

CS+ > CS- (defensive response)

Finger-pulse volume Aversive Continuous CS+ < CS-

Pupillary response Aversive and appetitive Continuous CS+ > CS-

Neural EEG/MEG Aversive and appetitive Continuous ERPs/ERFs/ssVEPs/ssVEFs:

CS+ > CS- in sensory brain regions

fMRI Aversive and appetitive Continuous

Aversive: BOLD signal CS+ > CS- in a brain network including the amygdala, the ACC, the insula, the hippocampus

Appetitive: BOLD signal CS+ > CS- in a brain network including the amygdala, the striatum, the ACC, the OFC Behavioral Reaction times Aversive and appetitive Continuous, intermittent,

or retrospective CS+ < CS- Subjective CS-US contingency/

US expectancy Aversive and appetitive Continuous, intermittent,

or retrospective CS+ > CS- CS pleasantness Aversive and appetitive Continuous, intermittent,

or retrospective Aversive: CS+ < CS-; Appetitive: CS+ > CS- CS arousal Aversive and appetitive Continuous, intermittent,

or retrospective CS+ > CS-

Fear ratings Aversive Continuous, intermittent,

or retrospective CS+ > CS-

Note. ERPs = event-related potentials, ERFs = event-related field time averaged responses, ssVEPs = steady-state visual evoked potentials, ssVEFs = steady-state visual evoked fields, BOLD = blood-oxygen-level dependent, ACC = anterior cingulate cortex, OFC = orbitofrontal cortex.

CSs are pleasant or unpleasant, thereby reflecting the evaluative effects of Pavlovian conditioning (see, e.g., De Houwer, Thomas, & Baeyens, 2001; Hofmann, De Houwer, Perugini, Baeyens, & Crombez, 2010). CS arousal ratings assess the degree to which the CSs are deemed subjectively arousing. Fear ratings are employed to evaluate how strong participants’ subjective feelings of fear are in response to the CSs (Lonsdorf et al., 2017). All of these subjective rating measures can be assessed online trial-by-trial, in an intermittent fashion after each conditioning phase or group of trials, or in retrospect after the end of the conditioning procedure. We measured CS-US contingency ratings and CS liking or pleasantness ratings after the conditioning procedure in all the empirical studies reported in the experimental part (see chapter 3), whereas we additionally measured CS arousal ratings and CS subjective relevance ratings in Studies 2, 3, and 4, and in Studies 2 and 3, respectively.

These subjective ratings primarily served as manipulation checks with the aim of assessing participants’ awareness of the reinforcement contingencies, as well as the evaluative effects of the Pavlovian conditioning procedure.