As mentioned above, only test items that were covered by Quebec’s curriculum were included in the analysis, in order to control for possible differences in item difficulty. Of these 140 test items, 106 were covered by Alberta’s curriculum and 34 were not.
Reflecting Alberta’s lower scores compared to Quebec in all three content domains, the proportion of students who correctly answered questions was lower in Alberta than in Quebec in all three content domains—number, geometric shapes and measures, and data display—as shown in Table 3.
Table 3 Comparison of Alberta and Quebec scores on TIMSS 2015 test items
Content domain Number of test
* Only items included in Quebec’s curriculum are included in this analysis.
The following general observations can be made from the results of the analysis:
I. Alberta’s performance in the content domain of number does not appear to be affected adversely by curriculum coverage. In fact, Alberta students did relatively better than their Quebec peers when dealing with items not covered in Alberta’s curriculum than they did on items that are covered.
II. Alberta’s performance in the content domain of data display is also not affected adversely by curriculum
coverage. Only one item in data display was not covered in the Alberta curriculum, and there was no significant difference between Alberta’s performance on test items that were covered and the one that was not covered, when compared to Quebec’s performance.
III. Alberta students’ performance in geometric shapes and measures appears to be affected by the fact that a large proportion of TIMSS items are not in the province’s curriculum. In this content domain, the difference between the proportion of students with correct answers in Alberta and the proportion with correct answers in Quebec is more than twice as large for test items not in Alberta’s curriculum (–24 per cent) as for test items that are in Alberta’s curriculum (–10 per cent).
The following are the results for the various topic areas under the number and geometric shapes and measures content domains.
Number The content domain of number assessed three topic areas: fractions and decimals; whole numbers; and expressions, simple equations, and relationships . Although the performance of Alberta students on the overall content domain does not seem to have been affected by curriculum coverage, an analysis of their performance in the three topic areas that together make up the number domain uncovered a more nuanced picture (see Table 4).
First, curriculum coverage of fractions and decimals was broader in Alberta than in Quebec. All items that were covered in Quebec were also covered by Alberta’s curriculum. Consequently, the performance gap between students’
performance in the two provinces with respect to this topic area has nothing to do with differences in curriculum
coverage. Interestingly, this topic area in the number content domain had the largest difference in performance between Alberta and Quebec in any of the content domains.
Second, in the whole numbers topic area, Alberta students’ performance was relatively weaker than that of their Quebec peers on test items covered in Alberta’s curriculum than on items not covered. Data in the Appendix table clearly show that this is fully attributable to relatively weaker performance of students in Quebec on these items rather to an improvement of performance of students in Alberta. While we do not have an explanation for this finding, it is clear that Alberta’s performance in this topic area is not adversely affected by curriculum coverage.
Finally, expressions, simple equations, and relationships is the only topic area within the numbers domain in which curriculum coverage appears to play a role in the weaker performance of students in Alberta . In this topic area, the difference between Alberta and Quebec in the proportion of students correctly responding to an item was almost twice as large for test items not included in Alberta curriculum (–17 per cent) as for the items that were included in Alberta’
curriculum (–9 per cent). As data in the Appendix table show, students in both provinces performed better on items not in Alberta curriculum, suggesting that these items are of a lower level of difficulty. Improvement is much more pronounced in Quebec than in Alberta, which explains the increase in the performance gap.
Table 4 Comparison of scores in the number content domain Topic areas within number domain Number of test
items* Difference in % correct
Expressions, simple equations, and relationships 20 –11
In Alberta curriculum 15 –9
Not in Alberta curriculum 5 –17
Total 75 –14
*Only items included in Quebec’s curriculum are included in this analysis.
Geometric shapes and measures The content domain of geometric shapes and measures included two topic areas:
points, lines, and angles; and two- and three-dimensional shapes. On both of these topics, the performance of students in Alberta was relatively weaker than students in Quebec on items that were in the Alberta curriculum. This indicates that differences between Alberta and Quebec in terms of curriculum coverage are not the only reason for poorer performance of students in Alberta. However, in points, lines, and angles, the difference between Alberta and Quebec was larger for test items that were not included in Alberta’s curriculum (–29 per cent) than for items that were included (–7 per cent) (see Table 5). This indicates that Alberta’s performance in this topic area is associated with lower curriculum coverage. Similar to the findings for expressions, simple equations, and relationships, the larger gap is almost fully explained by the significant increase in the proportion of students in Quebec with correct answers. (There was only a small decline in the proportion of students in Alberta with correct answers.) This indicates that items not covered in Alberta curriculum are of a lower level of difficulty.
The topic area of two- and three-dimensional shapes also revealed a gap in relative performance, but it was not as pronounced as that for points, lines, and angles. The difference between Alberta and Quebec was larger for test items not included in Alberta’s curriculum (–18 per cent), compared to the items that were included in Alberta’s curriculum (–11 per cent). This indicates that Alberta’s performance in two- and three-dimensional shapes is also associated with lower curriculum coverage.
Table 5 Comparison of scores in geometric shapes and measures Topic areas within geometric shapes
and measures Number of test
items* Difference in % correct (Alberta minus Quebec)
Points, lines, and angles 20 –19
In Alberta curriculum 9 –7
Not in Alberta curriculum 11 –29
Two- and three-dimensional shapes 26 –13
In Alberta curriculum 18 –11
Not in Alberta curriculum 8 –18
Total 46 –16
*Only items included in Quebec’s curriculum are included in this analysis.
Conclusion
Alberta’s performance is lower than Quebec’s in all content domains, and Quebec’s curriculum covers more TIMSS items at Grade 4 than Alberta’s in most topic areas. However, differences in curriculum coverage seem to be linked to differences in student performance only in the geometric shapes and measures domain and in the topic area of expressions, simple equations, and relationships. There is also a topic area (whole numbers) in which Alberta students’ performance is closer to the performance of their Quebec peers on items that are not in Alberta’s curriculum than on those that are.
All of this suggests that factors other than curriculum play a more significant role in explaining why students in Quebec perform better then students in Alberta. In particular, looking at teaching and learning processes in Quebec (as opposed to examining what teachers are supposed to cover: the curriculum) seems a promising direction for further analysis of TIMSS data.
Topic areas in which the curriculum coverage is linked to student performance appear to contain items that are at a relatively lower level of difficulty. Changes to Alberta curriculum in these topic areas could offer opportunities for quick improvement in results. Specifically, Alberta’s performance in TIMSS Grade 4 math should improve if curriculum coverage were broadened in the following areas:
• points, lines, and angles
• two- and three-dimensional shapes
• expressions, simple equations, and relationships
Further analysis may shed light on whether the change in Alberta’s results between 2007 and 2015 can be linked to any changes in curriculum coverage in both Alberta and Quebec. This analysis could enhance our understanding of the reasons for the significant decline in Alberta’s scores in the geometry content domain.