Impact of the experimental conditions on the final evaluation scores

In the final set of analyses, we performed two sets of multiple linear regressions: one with Originality as dependent variable (DV), another with Quality as DV. Once more, we proceeded by steps. In a first step, the control variables were included in the model; these were three dummy variables coding the effect of the text’s theme (comparing all seasons to spring), three verbal abilities variables (vocabulary, comprehension, fluency), a dummy variable coding whether French was the native language, and the variable of the creative writing habits. In a second step, we tested the main effect of the experimental conditions as well as more specific group comparisons, above and beyond the effects of the control variables.

3.3.1. Overall effect of control variables

For both DVs, as expected, there was no effect of the theme of the text (F(3,169)<1.1, p>.30). But for both DVs, other control variables had a significant effect. For Quality (Table 11, page 143), two task-related variables were important: time spent on task had a positive effect³³ (b_z=.23, p=.001), and presence of missing data in the process questionnaire had a negative effect (b_z=-.16, p=.021); these results indicate that participants who did the experiment quickly and possibly not very carefully produced texts of poorer quality.

33 The coefficients mentioned below, bz, are standardized coefficients.

Conversely, as expected, fluency (bz=.22, p=.003) and French mother tongue (bz=.19, p=.008) had a positive significant impact on Quality. All other variables had no significant effect and total R² was .25. For Originality (Table 12, page 144), the impact of the control variables was smaller. Only the creative writing habits had a significant positive impact (b_z=.19, p=.02), suggesting that experience in creative writing may facilitate the production of originality in this experimental task. The total R² was .06.

3.3.2. Effect of experimental conditions

To test the effect of experimental condition we tested first the effects of all experimental conditions, and then specific, theoretically motivated group comparisons.

Overall main effect. To test the main effect of the experimental conditions, we added 9 dummy-coded variables (i.e, every experimental condition compared to the control group) in the regression model described in the previous section. Doing this allowed us to test the main effect of the experimental conditions while controlling for other influential variables. Results indicated that the main effect of the experimental conditions led to a significant increase in R² for both Originality (ΔR²=0.097, F(9,152)=2.47, p=.033) and Quality (ΔR²=0.135, F(9,152)=2.86, p=.011). This indicates that as a whole the experimental manipulation was successful in altering the final creative product. Also, there is substantial variability among the experimental groups. We do not interpret each comparison in detail³⁴.

Complexity of instruction. The first set of planned comparisons we tested was the ones corresponding to the complexity of the instruction. For Quality (Table 11), the complexity of the instruction had no impact; the effects of both the ‘complex’ and ‘very complex’ dummy variables were not significant (respectively p=.21 and p=.43), the R² did not change significantly (ΔR²=0.017, F(2, 159)=1.83, p=.16). For Originality (Table 12), the addition of these complexity variables improved the overall prediction (ΔR²=.048, F(2, 159)=4.27, p=.015). More specifically, the ‘very complex’ instruction (C1 and C2) had a negative impact (bz=-.21, p=.012), indicating that these instructions induced a workload detrimental to the production of original ideas. On Figure 15 (page 145), which displays the group mean scores of Quality and Originality, it is indeed clear that these two groups are among the ones that produced texts of poor originality.

34 Doing multiple comparisons would have led to serious difficulties balancing the risks of type I and II errors.

To avoid such risks, given the small group sizes, we decided to run only a small set of educated comparisons.

Tests of specific hypotheses. The second set of planned comparisons we tested were the ones corresponding to the core hypotheses of this paper. Hypothesis (a) stated that low or increasing patterns of Generation should be detrimental to Originality, whereas the other experimental conditions should be more favorable. Hence we compared the supposedly detrimental conditions A3 (low flat pattern of Generation), B2, and C1 (increasing patterns of Generation) to the remaining experimental conditions. Results in Table 11 indicated that the text written by these groups were less original that those of the other groups (M=2.94, SD=0.82 vs. M=3.35, SD=0.77, t=2.71, p=.007). Figure 15 shows the poor Originality scores of these three groups. The only other group scoring as low on Originality was the C2 group, most probably because of the very complex instructions. To further refine our comprehension of the impact of Generation on creativity, we tested hypothesis (b), according to which a decreasing pattern of Generation should lead to higher Quality than a constant high flat pattern. Thus, we compared group A1 (high flat pattern of Generation) and B1 (decreasing pattern) on Quality. Results (Table 11) modestly supported the initial hypothesis: the group B1 (M=3.75, SD=0.47), compared to A1 (M=3.54, SD=0.46), had a marginally significant superior performance on Quality (t=1.91; p=0.058).

Table 11. Results of the final regression model (DV= Originality)

bz Std.Err. t(161) p-value total R² ΔR²

Step 1 (theme) 0.019

Step 2 (step 1 + other control) 0.0605 0.041 (ns)

Time spent on task (log) -0.062 0.079 -0.778 0.438 Missing data in questionnaire 0.040 0.079 0.509 0.611 Native French speaker -0.041 0.080 -0.513 0.609 Vocabulary knowledge 0.007 0.084 0.081 0.935 Verbal comprehension -0.032 0.080 -0.403 0.687

Verbal fluency 0.003 0.080 0.031 0.975

Creative writing habits 0.188 0.080 2.338 0.021

Step 3a (step 2 + complexity) 0.108 0.048*

Complex instructions 0.021 0.084 0.246 0.806

Very complex instructions -0.214 0.084 -2.555 0.012

Step 3b (1+favorability) 0.102 0.042*

(c) A3, B2, C1 vs others (except

CTRL) 0.214 0.076 2.812 0.006

(d) A1 vs B4 -0.001 0.078 -0.007 0.994

Note. bz = standardized coefficient; Std.Err. = Standard error; ns = non-significant.

Table 12. Results of the final regression model (DV=Quality)

bz Std.Err. t(161) p-value total R² ΔR²

Step 1 (theme) 0.001

Step 2 (step 1 + other control) 0.247 0.246***

Time spent on task (log) 0.231 0.071 3.257 0.001 Missing data in questionnaire -0.164 0.070 -2.332 0.021 Native French speaker 0.191 0.072 2.668 0.008 Vocabulary knowledge 0.106 0.075 1.417 0.158 Verbal comprehension -0.088 0.072 -1.229 0.221

Verbal fluency 0.216 0.072 3.001 0.003

Creative writing habits 0.037 0.072 0.512 0.609

Step 3a (step 2 + complexity) 0.264 0.017 (ns)

Complex instructions -0.096 0.076 -1.257 0.211

Very complex instructions 0.060 0.076 0.786 0.433

Step 3b (step 2 + favorability) 0.275 0.028*

(a) A2, B3, C1 vs others (except

CTRL) 0.152 0.068 2.235 0.027

(b) A1 vs B1 0.128 0.067 1.913 0.058

Note. bz = standardized coefficient; Std.Err. = Standard error; ns = non-significant.

Then we investigated the impact of Selection and tested hypothesis (c), stating that decreasing or low flat pattern of Selection should be detrimental to Quality. To do so we compared conditions A2 (low flat pattern of Selection), B3, and C1 (decreasing patterns of Selection) to the remaining experimental conditions. Results in Table 12 indicated that these three groups obtained lower Quality scores than the other groups (M=3.59, SD=0.43 vs.

M=3.72, SD=0.46, t=2.23, p=0.027). Figure 15 shows a more subtle pattern however. Group B3 was actually the worst on Quality, but A2 and C1 had intermediary performance. Hence, although the overall performance of these groups taken together met our expectations, the results are not clear-cut.

Finally we tested hypothesis (d) according to which an increasing pattern of Selection should be more favorable to Originality than a high flat pattern. To do so, we compared groups A1 (high flat pattern of Selection) to B4 (increasing pattern of Selection). These two groups were not different in terms of their Originality scores (M=3.45, SD=0.65 vs. M=3.38, SD=0.69, t=-0.007; p=0.99).

Figure 15. Quality and creativity mean scores across the conditions.

Note. CTRL = control groupe. See Figure 13 (p. 133) for the detailed patterns of Generation and Selection in each experimental condition. slope of the ‘Low Flat’ group, initially expected to have a constant low level of Generation.

This suggests that idea production in the beginning of a task is hardly repressible. For Selection, where a strong natural pattern was found in the control group (i.e., increasing slope), the manipulation of the process was achieved only in relative terms; the induced trends were systemically shifted toward the strong control group’s trend. Together, these results