Propositional Models

2.3.2.1 The core of propositional models Associative models have dominated learning research for more than one hundred years now, basically right from the start. As a result, for some, classical conditioning as an effect is almost synonymous to association formation as a mechanism (see De Houwer, 2018b, for an historical review). It is only by clearly separating the functional level of explanation (including classical conditioning as an effect) from the cognitive level of explanation (including association formation as a mechanism) that one can take seriously the idea that classical conditioning might be mediated by processes other than association formation.

It is only recently that a second class of mental process theories on classical conditioning has been proposed (e.g., De Houwer, 2009, 2018c; Mitchell et al., 2009; Waldmann & Holyoak, 1992). What they have in common is the assumption that the effect of stimulus pairings on behavior is mediated by the (typically nonautomatic) formation of propositions about relations in the environment. Propositions are units of information that specify assumptions about the nature of events in the world. For instance, a proposition could specify that the ringing of a bell is always followed by food. Propositions have two unique characteristics: (1) A proposition has a truth value: it is possible, at least in principle, to evaluate whether the assumptions about events in the world are right or wrong (Strack & Deutsch, 2004). (2) Propositions contain relational information, that is, information about how events are related (e.g., bell predicts food, smoking causes cancer; Lagnado, Waldmann, Hagmayer, & Sloman, 2007; Waldmann & Holyoak, 1992).

Suppose you establish that people with a certain disease always have a certain chemical in their blood and that this substance is not present in people who do not have the disease. One possible proposition about this relation is that the chemical in the blood causes the disease. Another pos-sible proposition is that the disease causes the chemical in the blood. According to both proposi-tions, there is a relation between the chemical and the disease. The propositions differ, however, with regard to the nature of the relation between the two (Waldmann & Holyoak, 1992). Note that propositions are not necessarily verbal (i.e., expressed in words). It seems fair to assume that nonverbal organisms also have knowledge about how events in the world are related. Neverthe-less, the exact nature of and the flexibility in using propositions might vary greatly depending on whether an organism is verbal or not (see De Houwer, Hughes, & Barnes- Holmes, 2016, for a discussion). We will revisit this issue in chapters 3 and 4.

Classical Conditioning 111

Associations in memory are not propositions because they do not have the two character-istics of propositions: (1) An association in memory does not specify any assumption about events in the world. It is a hypothetical state in memory that is assumed to have been created as the result of a spatiotemporal relation in the environment. It is pointless to say that an association is right or wrong. (2) An association does not encode relational information, that is, information about how events are related.⁶

What do we mean when we say that learning is the result of nonautomatic formation of propositions about relations in the environment? First of all, this means that people (and certain nonhuman animals) form hypotheses about relations in the environment and try to determine which hypothesis is correct. When doing so, they deploy all information that can be useful to discover and evaluate a relation. This is not only information about when stimuli occur, but also knowledge that people already have in memory and knowledge that they acquire through observations and instructions. They can deploy this knowledge not only when they experience the events that constitute the regularity (e.g., when a tone and a shock are paired) but also when changes in behavior are assessed (e.g., when the conditioned fear for the tone is measured). Secondly, they often do this in a nonautomatic way, that is to say (among other things) that they are aware of the propositions they form and that they have to make an effort to form those propositions. The effect that a relation in the environ-ment will have on behavior is determined by what the person consciously thinks about that relation (i.e., what proposition about the relation people perceive as being true).

How does all of this relate to classical conditioning? Let’s return to the example of Pav-lov’s dog. According to propositional models, presenting an important stimulus such as food will encourage the dog to actively (purposefully) search for predictors or causes of the food.

Because the bell is a salient stimulus, the dog will soon consider the possibility that there is a relation between the bell and the food. The fact that the bell is indeed always followed by food offers support for the hypothesis that the arrival of the food can be predicted on the basis of the bell. Behavior can be influenced by the bell- food relation only after the dog has formed a proposition about that relation. More specifically, the proposition specifying that the bell is followed by food will lead to the expectation that food will be delivered after hear-ing the bell. That expectation results in salivation when hearhear-ing the bell.

2.3.2.2 General evaluation of propositional models Can propositional models offer an explanation for available functional knowledge on classical conditioning? We will now dis-cuss a number of important findings.

– Influence of stimulus characteristics and intrinsic relations (see section 2.2.1.2) How fast one discovers a particular relation depends on the properties of the CS, US, and the intrinsic rela-tion between the two. People will be more motivated to discover relarela-tions with important

112 Chapter 2

USs. One will detect relations that contain striking CSs more quickly. A relation is also easier to detect if there are pre- existing reasons to suspect that such a relation would exist. For example, when people become nauseated, they will tend to think of food as a cause of that nausea because they know from experience that you can become nauseous from food (see Testa, 1974, for a precursor to this idea). Organisms therefore use available and past knowl-edge to discover new relations in the environment.

– Classical conditioning can also influence involuntary behavior (see section 2.2.2.1) In principle, propositions can influence all kinds of behavior. Contrary to what is sometimes thought (e.g., Shanks, 1990), behavior does not have to be a rational, logical consequence of a propo-sition about the relation between stimuli. Let us consider the fact that in autoshaping studies, pigeons move toward and peck on the illuminated key when doing so reduces their chance of contacting food. From the perspective of propositional models, it is possible that the prop-ositional belief that the illumination of the key is followed by food, creates a tendency to walk toward the key. Propositional models of learning do not in themselves say anything about why certain propositions have certain effects on behavior. That is, they do not provide a full specified theory of behavior. What they do say is that relations in the environment can have an effect on behavior only after a proposition has been formed about that relation. It is, however, not possible to predict behavior perfectly on the basis of propositional knowledge unless one has a perfect theory of behavior (see Mitchell et al., 2009).

– Contingency awareness is important (see section 2.2.2.3) In contrast to associative models, propositional models can explain why learning usually occurs only after people are aware of the relation. At least in humans, forming and evaluating propositions happens in a nonau-tomatic and therefore conscious manner. The existence of conditioning without awareness of the CS- US relations would, however, be difficult to explain on the basis of propositional models (but see De Houwer, 2018c).

– Classical conditioning is a general phenomenon that occurs in different organisms (see section 2.2.3) At first sight, this observation seems to contradict propositional models. If all learn-ing is based on propositional processes, then one would have to assume that all animal spe-cies are capable of forming and evaluating propositions in a conscious, nonautomatic way.

This criticism is correct in the sense that it is unlikely that learning in simple animal species such as snails and bees is based on propositional processes. Yet, there are indications that learning in certain nonhuman animals such as rats is indeed based on the formation and evaluation of propositions (e.g., Blaisdell, Sawa, Leising, & Waldmann, 2006; Beckers, Miller, De Houwer, & Urushihara, 2006; see also Mitchell et al., 2009).

– Secondary tasks have an important impact on classical conditioning (see section 2.2.4) This con-clusion fits well with the idea that learning is determined by propositional processes. After

Classical Conditioning 113

all, it takes a lot of effort to formulate and test hypotheses. If you have to invest energy in performing secondary tasks, there is less energy left for forming and evaluating propositions.

If, however, attention is focused on the relation between the CS and the US (see Baeyens, Eelen, & Van den Bergh, 1990), then one will more quickly form hypotheses about that rela-tion and evaluate them as being present.

– Conclusion In sum, from the above it appears that propositional models are capable of explaining many aspects of the existing functional knowledge about classical conditioning.

They therefore have a high heuristic value. Nevertheless, there are also findings that seem to challenge a propositional account of classical conditioning. Perhaps the most intriguing challenge comes from research on the so- called Perruchet effect. Perruchet (1985) presented a series of trials in which a tone could be followed by an air puff delivered to the eye of the participant. He registered eye blink responses after presentation of the tone and asked par-ticipants to rate the extent to which they expected that an air puff would be delivered after hearing the tone. As the number of consecutive trials on which the tone was followed by the air puff increased, the likelihood of an eye blink response increased whereas the expectancy of the air puff after the tone decreased. The expectancy results are in line with the so- called gambler’s fallacy, which refers to the fact that gamblers believe that a loss is more likely after a series of wins, even when the chance of winning is equally high with each gamble. This clear dissociation between conscious expectancies and conditioned eye blink responses seems to suggest that the latter are not based on propositional knowledge of the tone- air puff relation but that they might reflect the operation of a separate, nonpropositional learning system.

Although this conclusion is still being debated (e.g., Weidemann, McAndrew, Livesey, &

McLaren, 2016), the Perruchet effect is widely regarded as a problem for the idea that all conditioning effects are mediated by propositions.

The predictive value of propositional models is also high. Propositional models have led to a better understanding of the conditions under which conditional contingency is impor-tant. More specifically, these models have led to a number of important studies on blocking (see Mitchell et al., 2009, and Boddez, De Houwer, & Beckers, 2017, for an overview). As indicated earlier, blocking refers to the finding that a cue X elicits a less strong CR when AX+

trials are presented together with A+ trials than when only AX+ trials are offered. According to propositional models, this result can be the consequence of causal reasoning. Suppose A and X are seen as two possible causes of the US. The fact that the US is just the same when only A is present than when both A and X are present, suggests that X has no causal influence on the US. After all, causes normally have additive effects. The effect of two causes should therefore be stronger than the effect of one cause in itself. However, we note that A and X together have just the same effect as A alone. We can therefore conclude that X is not a cause of the US. If blocking is indeed the result of this reasoning, then it should only occur if A and

114 Chapter 2

X are presented as possible causes of the US. Waldmann and Holyoak (1992) confirmed this prediction. In human contingency learning studies in which participants had to assess the strength of the relation between X and the US, blocking was only established when A and X were described as chemical substances in the blood and the US as a disease that could be caused by A and X. However, they found no blocking if A and X were described as chemical substances in the blood and the US as a disease that can cause A and X. So blocking only occurred if A and X were possible causes of the US but not if the US was a possible cause of A and X.

Even if A and X are considered as possible causes of the US, blocking should only occur if one assumes that the effects of two causes are additive. De Houwer, Beckers, and Glautier (2002) investigated the role of this assumption by providing test subjects with information about the maximum intensity of the US. In the submaximal condition, the subjects were told that the US had an intensity of 10 out of 20 on both the A+ trials and the AX+ trials. Given this information, one can be pretty sure that X is not a cause of the US because the US is just as intense on the A+ trials as on the AX+ trials. If X was a cause, the US should have been stronger on the AX+ trials than on the A+ trials. In this condition blocking was observed: test subjects believed that there was no causal relation between X and the US. In the maximal condition, participants were told that the intensity of the US on the A+ and AX+ trials had a value of 10 out of 10. Because A alone has the maximum effect, it is no longer possible for X to strengthen this effect. The fact that the US is the same on the A+ trials than on the AX+ tri-als therefore does not say anything about the effect of X on the US. It is possible that X does have an effect on the US, but that this extra effect does not show up because A alone already has the maximum effect. In other words, there is a “ceiling effect” that makes it impossible to draw conclusions about the relation between X and the US. No blocking was found in this condition. Beckers et al. (2006) showed that blocking in rats also depends on how likely it is that the effects of different causes are additive.

On the basis of such findings, most researchers now agree that at least some of the condi-tioning effects in humans are due to propositional processes (e.g., McLaren et al., 2014). Most researchers, however, continue to assume that associations can also lead to conditioning. Such dual- process models are currently very popular because they can explain more data than single- process models. But if there is more than one learning system, the question arises as to how these different learning processes relate to each other (e.g., when do the different pro-cesses guide behavior). Attention is increasingly being paid to this difficult question by propo-nents of dual- process models (see Mitchell et al., 2009, for a critique of dual- process models).

Despite the successes of propositional models of classical conditioning, their impact on learning research has been relatively limited. In part this can be attributed to the lack of precision in formulating these models. Whereas associative models are often formulated in

Classical Conditioning 115

very precise mathematical terms (e.g., Rescorla & Wagner, 1972; Stout & Miller, 2007), propo-sitional models are often not more than a few verbally formulated ideas about the nature of the mental processes and representations that mediate learning. This lack of precision not only renders it difficult to derive precise predictions from propositional models but also to falsify those models on the basis of empirical evidence. Although these criticisms are valid, it is important to realize that propositional models as they are currently described in the litera-ture, are a class of models that share the core idea that learning is mediated by propositional knowledge. It is indeed difficult to falsify a whole class of models. What is often forgotten, however, is that it is also impossible to falsify the class of all possible associative models of learning (Miller & Escobar, 2001). For every result in the learning literature, it is possible to find an associative model that can explain that result. At the same time, it would also be pos-sible to find another (version of an) associative model that predicts the opposite result. More generally, we believe that the possibility to formalize or refute a theoretical model is not an essential criterion for the quality of the model (De Houwer, 2018c). What is more important is the ability of the model to explain existing functional knowledge (i.e., its heuristic value) and to predict new functional knowledge (i.e., its predictive value). From that perspective, we continue to believe in the value of propositional models of classical conditioning.

After reading this chapter, you should be able to:

• Indicate under which conditions operant conditioning (as an effect) will occur.

• Provide an overview of the core assumptions of the main mental process theories of oper-ant conditioning.

Introductory Task

Try to find five examples from daily life that show that behavior can be influenced by the con-sequences of that behavior. Which factors determine whether operant conditioning will occur?

What does this say about the mental processes that mediate operant conditioning effects?

3.1 Some Basic Terms and Procedures

3.1.1 Basic Terms

Operant conditioning (sometimes also referred to as instrumental conditioning) is, like classi-cal conditioning, an effect of regularities in the presence of events in the environment. How-ever, operant and classical conditioning differ with regard to the nature of events that occur.

Whereas classical conditioning refers to regularities in the presence of stimuli, operant condi-tioning refers to the effect of regularities in the presence of behavior and stimuli. By describ-ing operant conditiondescrib-ing in such broad terms from the outset, we reject the stereotype that it has to do only with the rats that press levers in Skinner boxes. It is more than that, just as classical conditioning is more than the prototypical example of Pavlov’s dog; these are just two of many possible procedures that may be used to study (operant or classic) conditioning.

Indeed, Skinner knew that by placing a rat in a noise- free room containing only a food box and a lever to push, he was creating a fairly artificial situation. He did so purposefully because

3 Operant Conditioning: Effects of Regularities in the Presence