Within-domain associations - The maintenance of cross-domain associations in working memory

Research on the maintenance of associated features was already ongoing before the introduction of the concept of the episodic buffer, especially in the context of short-term memory. In general, two opposing views were then proposed: either associated features are maintained as objects (e.g., Luck & Vogel, 1997), or they are maintained as separate features and glued together by use of attentional resources (e.g., Wheeler & Treisman, 2002). Within the multi-component model, this would correspond either to the representation of the

information as an object within the episodic buffer, or to the representation of single isolated features within domain-specific maintenance buffers, with a transition to the episodic buffer to form a coherent object with the help of the attentional resources of the central executive.

The role of these attentional processes was investigated more in detail by Allen, Baddeley and Hitch (2006) within a working memory context in order to clarify the

relationships between the different components of the multi-component model. Although the existence of the episodic buffer was taken for granted, its relationships with the other

structures within the multi-component model was to be defined. As stated earlier, the episodic buffer was initially assumed to rely exclusively on the central executive and its attentional resources for its input, with no direct access from the domain-specific maintenance buffers (Baddeley, 2000). Cutting within these attentional resources should thus have a drastic effect on the performance of the episodic buffer. Allen et al. (2006) therefore compared in three experiments the maintenance of isolated visuo-spatial features (e.g., either the color red or the shape triangle) with the binding and maintenance of associated visuo-spatial features (e.g., a red triangle, see Figure 1.3) in the presence or absence of an attention-demanding task. They did this using a change detection task. In this kind of task, participants are presented with a number of items to maintain. After a retention interval, a probe is shown for which

participants have to decide whether it is different from all items in the study array (lure) or corresponds to one of these items (target). Figure 1.3 shows a general example of the change

detection task as used by Allen et al. (2006), presenting the type of stimuli used for the maintenance items and at test.

Figure 1.3: General example of the different presentation conditions used by Allen et al.

(2006).

In each of the three experiments, the attention-demanding task had a harming effect on the overall maintenance performance as compared to a control condition without the

attention-demanding task. It was nonetheless observed that the memory performance for the associated features (colored shapes) was not more affected by the presence of an attention-demanding task than the memory performance for single features (colors or shapes). The encoding and maintenance of feature associations is hence not more dependent on attentional resources than the encoding and maintenance of single features. This suggests no particular role for the central executive in the binding process or the maintenance of associated features, a finding at odds with the initial assumptions about the relation between the episodic buffer and the central executive. One might argue that the attention-demanding task did not recruit enough of the attentional resources to allow for any interaction between this task and the maintenance items (single features or feature associations) to occur. However, the attentional demands of the concurrent task were already amplified in the three successive experiments in order to counteract this argument. In the first experiment, participants had to count backwards by one, while in the second experiment they had to maintain a six digit load and in the third experiment they had to count backwards in steps of three. None of these experiments showed a particular role for attentional resources in the encoding or maintenance of feature

associations.

Instead of continuing to increase the attentional demand of the concurrent task, a subsequent study (Karlsen, Allen, Baddeley, & Hitch, 2010) manipulated the attentional demand of the binding process. Karlsen et al. tried to increase the attentional demands of the binding process by separating the associated features in time or space (see Figure 1.4;

Unitized, Spatially separated and Temporally separated conditions). Participants knew they were going to be tested on the feature associations and were as such obliged to create the corresponding associations in order to correctly perform the task.

Figure 1.4: Examples of unitized, spatially and temporally separated feature trials as used in the study by Karlsen et al. (2010).

First of all Karlsen et al. (2010) compared a unitized feature presentation with a spatially separated feature presentation. As in the previously mentioned study by Allen et al.

(2006), they used the task of counting backwards in steps of three to evaluate the dependence on attentional resources. Despite the fact that the spatially separated feature presentation resulted in less accurate recognition than the unitized feature presentation, creating associations did not depend any more on attentional resources than just encoding the associations. Subsequently, Karlsen et al. compared a temporally separated feature presentation with two variants of a unitized presentation, an early and a late version (see Figure 1.4). Two experiments showed that when comparing the temporally separated features with the early unitized condition, the interaction between presentation mode and concurrent task was not significant. This implies thus, once again, that attentional demands are to a same extent implicated in both conditions. Yet, when comparing the temporally separated

presentation with the late unitized presentation, the interaction between presentation mode and concurrent task was significant in both experiments. The effect of backward counting was more pronounced on the temporally separated than on the late unitized presentation. Karlsen et al. reasoned that this interaction had probably been caused by the longer duration of the dual task situation in the temporally separated condition. Indeed, in the late unitized

condition, during the first 1000 ms only the backward counting task was to be performed (in combination with a screen showing three black dots) while in the temporally separated presentation, the backward counting task was already combined with encoding and

maintenance of the color features from the beginning. The supposed effect of the dual task to explain the interaction between presentation condition and concurrent task was confirmed by comparing the early with the late unitized conditions. This comparison gave rise to a

significant interaction as well; the effect of the attention-demanding task was more pronounced in the case of the early unitized presentation. Karlsen et al. considered the comparison between the temporally separated feature presentation and the early unitized presentation thus the fairest, as these two conditions were matched for dual-task duration.

From their study, it could thus once again be concluded that the creation of bindings, even when rendered particularly difficult, does not specifically depend on attention.

In the meantime, another methodology was applied to increase the attentional demands of the binding process. This was done by presenting the features to be bound in different modalities (Allen, Hitch, & Baddeley, 2009). For example participants saw a triangle on screen and heard the word “red” through a headphone, or vice versa. This condition was compared with a unitized visual presentation of the features. These three conditions were compared in the presence or absence of the attention-demanding task of counting backward by three. Despite the expected reliance on attention to create cross-modal bindings, it was once again observed that creating feature associations did not depend more on attentional resources than a unitized presentation of the features.

Due to the disappointing results about the role of executive resources in creating and maintaining visuo-spatial feature associations, another line of research was set up in parallel.

The same methodology was applied this time to the binding and maintenance of verbal information. If the creation and maintenance of verbal associations takes places in the episodic buffer, than the attentional resources of the central executive should be involved in these processes. This involvement was studied by comparing the recall of either sentences (associations) or word lists (single features) in the presence of an attention-demanding task (Baddeley, Hitch, & Allen, 2009). The results showed that recall performance for sentences was not more affected by the concurrent attention-demanding task then recall of the word lists. This confirmed hence the results obtained in the visuo-spatial domain: no particular role could be attributed to attention for the creation or maintenance of feature associations.

From the point of view of the multi-component model, three explanations were possible to account for the ensemble of the results from the studies described above. First, assuming that feature associations are maintained within the episodic buffer, it is possible that binding takes places automatically, without any recourse to attentional resources, and that its maintenance does not depend any more on attention than the maintenance of single features.

This would make the episodic buffer a passive storage buffer containing multi-modal

information (e.g., Allen et al., 2009; Baddeley, Allen, & Hitch, 2010; Baddeley et al., 2009).

A second option is that binding, as explored in the above-mentioned studies, concerns automatic binding, but that another more effortful kind of binding may as well exist (e.g., Allen et al., 2006; Allen et al., 2009; Baddeley et al., 2011; Karlsen et al., 2010). Binding colors and shapes might indeed be a too basic process for attentional resources to come into play. Effortful binding might for example consist of creating new mental images, for which attentional resources would be necessary. In that case, the episodic buffer would anyhow depend on the central executive, however making use of the central executive’s resources only when required. A third alternative would be to assume that associations are created and maintained within the visuo-spatial or verbal subsystem of working memory, and not within the episodic buffer (Allen et al., 2006; Baddeley et al., 2011). As the multi-component model does not assume the visuo-spatial sketchpad or the phonological loop to depend particularly on attentional resources, this would explain why attention is not particularly involved in the binding and maintenance of these feature associations. Research on cross-domain associations should hence be able to elucidate this last hypothesis. If it were the case that the binding and/or maintenance of cross-domain (e.g., verbal-spatial) associations did depend particularly on attentional resources, then this would suggest that cross-domain feature association are maintained in the episodic buffer while within-domain feature associations would not be represented within the episodic buffer. This would then leave the current conception of the episodic buffer as a system dependent on attentional resources nevertheless intact.

Although this last hypothesis could have been easily verified by applying a same research design to cross-domain associations, the research team behind the multi-component model continued to explore the maintenance aspects of visuo-spatial feature associations with a different paradigm, i.e. the suffix paradigm (Figure 1.5). A final experiment in the study by Allen et al. (2006) described above had shown that feature associations might be more vulnerable to feature overwriting than single features. The suffix paradigm could allow a verification of this hypothesis. This paradigm consists in the presentation of a to-be-ignored item after the presentation of the study array. It has been shown that the presentation of such a suffix interferes with to-be-maintained items, mainly the most recent items (Hitch, 1975;

Parmentier, Tremblay, & Jones, 2004).

Figure 1.5: Example of the trial design in the suffix paradigm.

Ueno, Allen, Baddeley, Hitch, and Saito (2011) used this suffix paradigm to compare the maintenance of single features and feature associations. They showed first of all that single features as well as feature associations were vulnerable to the interference created by the suffix. This was shown by using a suffix that was not part of the experimental study set (hence an implausible suffix). In this case, the suffix effect was equivalent for single features and feature associations. Secondly, they also showed that when using as a suffix features that were part of the experimental set (hence a plausible suffix), the suffix effect was more pronounced for the maintenance of feature associations than for single features. Moreover, in the case of feature associations, a suffix made of only one feature from the experimental set (e.g., the shape) and the other from outside the experimental set (e.g., the color, hence a semi-plausible suffix), was as devastating as the combination of both features from the

experimental set (plausible suffix, Ueno, Mate, Allen, Hitch, & Baddeley, 2011). These results led Ueno, Allen, et al. (2011) and Ueno, Mate, et al. (2011) to assume the maintenance of feature associations to be achieved as represented as in Figure 1.6.

Single features would be represented within the visuo-spatial sketchpad both at the feature and at the object levels. This results in a certain redundancy which makes them less prone to feature overwriting. Object information, on the other hand, is represented only at the object level within the visuo-spatial sketchpad. The lack of redundancy of this information

makes it more prone to overwriting. A general mechanism, relying on attentional resources, is assumed to function as a filter to block the suffixes from entering the visuo-spatial sketchpad.

This filtering happens on the feature level, given that semi-plausible suffixes have the same impact as plausible suffixes. The reliance on attention of this filtering mechanism explains why even implausible suffixes have a small negative effect on maintenance performance of features and feature associations. However, semi-plausible and plausible suffixes can often make their way through this attentional filter, leading to an overwriting of maintained information. This overwriting process takes place at the object level. Thus, when a suffix makes it through the filter, object information is overwritten and hence lost. However, feature information is represented both at the feature and the object levels. Even though information would be overwritten at the object level, it might still be retrieved at the feature level. This mechanism could thus account for the increased fragility of feature associations, observed on several occasions (e.g., Allen et al., 2006; Ueno, Allen, et al., 2011; Ueno, Mate, et al., 2011).

In a review study on the role of the episodic buffer within binding, Baddeley et al.

(2011) elaborated their current conception of the creation and maintenance of binding, especially within the visuo-spatial domain. Figure 1.7 represents the framework of the multi-component model corresponding to this conception. While originally information from the domain-specific buffers was supposed to pass the central executive to flow into the episodic buffer (see Figure 1.2, p. 17), this assumption was now abandoned (see Figure 1.7).

Considering the studies showing a lack of reliance on attentional resources for the creation of feature associations, this seemed indeed a dispensable assumption. Baddeley et al. (2011) assumed the creation of visuo-spatial bindings to be achieved within the visuo-spatial sketchpad (see Figure 1.6). Although not explicitly stated, we suppose hence that binding of verbal information takes places within the phonological loop. The conscious maintenance of object information was however still attributed to the episodic buffer, which is assumed to be purely object-based. Within this conception of the episodic buffer, it is considered a passive maintenance store, making object information available for consciousness. Baddeley (2012) stated that although the creation or maintenance of bindings is not particularly attention-demanding, maintaining these bindings against distraction is dependent on attentional

resources. The studies from Ueno, Allen, et al. (2011) and Ueno, Mate, et al. (2011) using the suffix paradigm support this statement. Nevertheless, these studies also showed that the attentional involvement (used for the filtering mechanism) is the same for the maintenance of single features and feature associations. It appears thus that the differential role of the episodic

buffer and/or attentional resources within the maintenance of single features and feature associations still lacks some precision.

Figure 1.7: Conception of the multi-component model according to Baddeley et al.

To summarize, the research line concentrating on the representations of within-domain feature associations appeared to be more tumultuous than expected. Although the existence of the episodic buffer has never been put into doubt, the assumptions made about it did not seem to correspond with the observations made. Hence, the creation of within-domain associations has been attributed to the visuo-spatial sketchpad instead of the episodic buffer. The

maintenance of feature associations is still assumed to be achieved within the episodic buffer.

Although this maintenance was initially supposed to rely on the attentional resources of the central executive, the role of attention has later on been limited to a filter against distraction.

This line of research on within-domain feature associations has taught us a lot on their creation and maintenance. However, at the moment the multi-component model lacks precision about the exact representations of feature associations, and this is all the more the case for cross-domain associations. Research has focused on within-domain associations, and several assumptions are not generalizable to cross-domain associations. For example, it is assumed that visuo-spatial feature associations are created within the visuo-spatial sketchpad.

There is however no equivalent structure for the creation of cross-domain associations. Where are cross-domain associations created, and do they depend on the attentional resources of the central executive anyhow? In order to refine the current version of the multi-component model, exploring cross-domain feature associations is a necessity.

Based on the framework of the multi-component model, several independent researchers have explored the creation and maintenance of cross-domain associations. The structure of the episodic buffer was less taken for granted by these researchers. A first mission

was hence to ascertain the need for the episodic buffer to exist in order to be able to maintain

Dans le document The maintenance of cross-domain associations in working memory (Page 47-55)