# Eye-tracking data

Dans le document Prediction in Interpreting (Page 191-200)

## 5 Study 3. Prediction during consecutive interpreting in noise

### 5.4.3 Eye-tracking data

The linear mixed model showed that, in the listening task, participants began looking significantly more at the target object from -500ms before word onset until the end of the time period analysed (see Figure 2). Participants also looked significantly more at the English competitor object compared to the target object from 600ms after word onset until 700ms after word onset, and again at 800ms after word onset. There was a general trend for more fixations on the English competitor than on the unrelated object for the entire period from 600 to 900ms after word onset, although this trend did not reach significance in every bin.

We also applied a linear mixed model to the data from the consecutive task (see Figure 3). In this task, participants began looking significantly more at the target object from -550ms before word onset until the end of the time period analysed.

Figure 2. Graph showing fixation proportions on target, English competitor and unrelated objects in the listening task. A linear mixed-model was used to establish when the difference in fixation proportions between target and unrelated and competitor and unrelated objects became significant. Open circles along the top represent bins during which there was a significant difference between fixation proportions on target and unrelated objects. Filled circles along the top represent bins during which there was a significant difference between fixations proportions on English competitor and unrelated objects. Transparent thick lines are error bars representing standard errors.

Figure 3. Graph showing fixation proportions on target, English competitor and unrelated objects in the consecutive task. A linear mixed-model was used to establish when the difference between fixation

proportions on target and unrelated objects, and English competitor and unrelated objects became significant.

Open circles along the top represent bins during which there was a significant difference between fixation proportions on target and unrelated objects. Filled circles along the top represent bins during which there was a significant difference between fixation proportions on English competitor and unrelated objects. Transparent thick lines are error bars representing standard errors.

We directly compared the listening and consecutive tasks using the same linear mixed model and specifying an interaction of task for each condition (listening/consecutive) (see Figure 4).

Figure 4. Graph showing fixation proportions on target, English competitor and unrelated objects with both tasks overlaid. A linear mixed-model was used to check for an interaction between condition and task. Open circles along the top represent bins during which there was a significant difference between fixations on target and unrelated objects. Filled circles along the top represent bins during which there was a significant

difference between fixations on the English competitor and unrelated objects. Filled squares show the where there was an interaction between the English competitor condition vs. unrelated condition and the task.

Transparent thick lines are error bars representing standard errors.

There was no significant interaction of task with target vs. unrelated condition at any point in the time course. There was a significant interaction of task with competitor vs.

unrelated condition in the period from 750 to 850ms. However, this interaction does not take place over a sustained period of at least three consecutive bins.

We carried out an exploratory analysis to investigate the effect of task order on predictive fixations. We compared fixations in the listening task when it took place in the first and second half of the experiment, as well as fixations in the consecutive task when it took place in the first and second half of the experiment.

We ran the same linear-mixed model on our results broken down by task (listening or consecutive), and considered whether the divergence in fixation proportions between conditions depended on task order.

First, we investigated whether carrying out the listening task before or after the consecutive task affected fixation proportions (see Figure 5)23. For the listening task, the divergence in fixation proportions between the target and unrelated objects became significant at -450ms, both when the listening task was carried out first and second. When the listening task was completed first, there was a significant difference between fixation proportions on the competitor and unrelated objects from 700ms to 850ms after word onset, whereas when the listening task took place second, there was, at no point, a

significant difference between fixation proportions on the English competitor and unrelated objects (although there was a trend towards this). However, when we compared task order in a model which included included an interaction term for condition and task order, we did not find any significant interaction between task order and condition during the listening task.

23 There was a warning message for convergence at -400ms in the listening task. We ran the model without the

Figure 5. Graph comparing task order in the listening task. On the left-hand side, results from participants who completed the listening task before the consecutive task. On the right-hand side, results from participants who had completed the consecutive task first. Fixation proportions on target, English competitor and unrelated objects are shown. A linear mixed-model was used to establish when the difference between fixation

proportions on target and unrelated objects, and English competitor and unrelated objects became significant.

Open circles along the top represent bins during which there was a significant difference between fixation proportions on target and unrelated objects. Filled circles along the top represent bins during which there was a significant difference between fixation proportions on English competitor and unrelated objects. Transparent thick lines are error bars representing standard errors.

We carried out the same analysis for the consecutive task. When participants carried out the consecutive task first, they fixated the target object significantly more than the unrelated object from -550ms. When they carried out the consecutive task second, fixation proportions on the target object significantly diverged from those on the unrelated object from -450ms until the end of the time window. There were also predictive fixations on the English competitor object in consecutive bins from -550ms to -400ms, and again from -200ms to 0ms. We added an interaction of task order to our model. We found a significant interaction between the competitor condition and the task order for four consecutive bins between -200ms and 0ms. In this period, we also recorded a significant main effect of task

order for two consecutive bins from -200ms to -100ms. This suggests that the divergence in fixation proportions between the Competitor and the unrelated object depended on task order in the period from -400ms before word onset until word onset.

Figure 6. Graph comparing task order in the consecutive task. On the left-hand side, results from participants who completed the consecutive task before the listening task. On the right-hand side, results from participants who completed the listening task first. Fixation proportions on target, English competitor and unrelated objects are shown. A linear mixed-model was used to establish when the difference between fixation

proportions on target and unrelated objects, and English competitor and unrelated objects became significant.

Open circles along the top represent bins during which there was a significant difference between fixation proportions on target and unrelated objects. Filled circles along the top represent bins during which there was a significant difference between fixation proportions on English competitor and unrelated objects. Transparent thick lines are error bars representing standard errors. The area shaded grey on the graph represents the period during which there was a significant interaction between task order and Competitor vs. unrelated condition.

We also analysed the filler items to check whether our results could be due to visual bias towards the target or competitor objects. When participants carried out the listening task (Figure 7), there were significant differences over consecutive bins between fixation proportions on the English competitor and unrelated objects. These significant differences in consecutive bins occurred at two different time points. There were significantly fewer

fixations on the competitor object compared to the unrelated object from -900ms until -750ms before word onset, and significantly more fixations on the competitor object than on the unrelated object from -450ms until -250ms. This could suggest a visual bias towards the unrelated object in the earlier time period, and for the competitor object in the later period. However, this does not pattern with the results from the listening task, as we did not observe any predictive fixations on the competitor object in the listening task. There were no significant differences between conditions for filler sentences when participants completed the consecutive task (Figure 8).

Figure 7. Graph showing fixation proportions on target, English competitor and unrelated objects in the listening task, when images were presented with the filler sentences. A linear mixed-model was used to establish whether the difference between target and unrelated and English competitor and unrelated conditions became significant. Black dots along the top represent bins during which there was a significant difference between competitor and unrelated conditions. Transparent thick lines are error bars representing standard errors.

Figure 8. Graph showing fixation proportions on target, English competitor and unrelated objects in the consecutive task, when images were presented with the filler sentences. A linear mixed-model was used to establish whether the difference between target and unrelated and competitor and unrelated conditions became significant. There were no significant differences between conditions. Transparent thick lines are error bars representing standard errors.

5.5 Discussion

We investigated the time-course of prediction in speech-shaped sound and in two different tasks, a consecutive interpreting task and a listening task, in a within-group

experiment. Participants, whose L1 was Dutch, listened to sentences in English containing a predictable word, and either answered a comprehension question or else consecutively interpreted sentences into Dutch and then answered a comprehension question. The results showed that in both tasks, participants predicted an upcoming word in noisy speech.

However, we did not find evidence that they made specific predictions about the form of that upcoming word. Predictive fixation patterns were similar in both tasks. However, in the

listening task, we found a phonological competitor effect after the onset of the predictable word, and there was an interaction between task and Competitor condition in the same time period, showing that fixations proportions on the competitor object after word onset depended on task. In an exploratory analysis, we found that when participants carried out the consecutive task after the listening task, they were more likely to make predictive fixations on the competitor object.

Dans le document Prediction in Interpreting (Page 191-200)