stable over time
PISA 2009 is the third assessment of mathematics since PISA 2003, when the first major assessment of mathematics took place, and the second assessment of science since 2006, when the first major assessment of science took place. Since comparisons over time can only be made from the point at which a major assessment of the domain took place, comparisons in mathematics and science are more limited since there have not yet been two full assessments of either area in the nine years of PISA testing. While this section looks at changes over time, performance differences should be interpreted with caution for several reasons. First, since data is available for three points in time for mathematics and two points in time for science it is not possible to determine the extent to which observed differences are indicative of longer-term changes. Secondly, in order to allow for comparability over time some common assessment items were used in each survey. However, because there are a limited number of common items, particularly when the domain was a minor focus, an additional measurement error must be taken into account for these comparisons over time. Consequently only changes that are indicated as statistically significant should be considered.
Across OECD countries as a whole, mathematics performance remained unchanged between PISA 2003 and PISA 2009. However there were changes in performance in some of the 40 countries that participated in both PISA 2003 and 2009. In eight countries mathematics performance improved, in 22 countries mathematics performance remained unchanged and in 10 countries mathematics performance was significantly lower. In Canada, performance in mathematics did not change significantly from 532 in 2003 to 527 in PISA 2006 and 2009.
Across OECD countries as a whole, science performance remained unchanged between PISA 2006 and PISA 2009, although changes in performance were observed in some of the 57 countries that participated in both PISA 2006 and 2009. Science performance increased in 11 countries, remained stable in 40 countries and decreased in six countries. In Canada, science performance remained stable between 2006 (532 score points) and 2009 (529 score points).
In order to understand how Canada’s performance level in mathematics and science has evolved, Canada’s change in performance should be considered alongside with its overall performance. Although Canada continues to have strong performance in mathematics, and experienced no significant change over time, the number of countries who statistically outperformed Canada increased from two in 2003 to four in 200920. Finland and Hong-Kong China outperformed Canada in 2003 and continued to do so in PISA 2009. Additionally, Korea outperformed Canada in 2009 as a result of improved performance and Liechtenstein outperformed Canada because it did not have a significant change in performance between 2003 and 2009.
Canada’s change in relative performance for science followed a similar pattern. When science was first included as a major domain in PISA 2006, two countries outperformed Canada in science compared to four countries in 200921. Finland and Hong-Kong China outperformed Canada in science 2006 and continued to do so in 2009. Additionally, both Korea and Japan outperformed Canada in 2009 as a result of improved performance between 2006 and 2009.
As shown in Table 2.5 there was a significant decrease in mathematics scores between 2003 and 2009 in Newfoundland and Labrador, Prince Edward Island, New Brunswick, Manitoba, Alberta and British Columbia. Performance in the remaining four provinces was not statistically different. For science, performance decreased in Prince Edward Island and Manitoba, and remained stable in the remaining 8 provinces (Table 2.6).
Despite a decrease in performance in mathematics, Alberta and British Columbia continued to have strong performance in mathematics in 2009, performing above the OECD average. On the other hand, as a result of a decrease in performance in mathematics, Newfoundland and Labrador, New Brunswick and Manitoba went from performing above the OECD average in 2006 to performing at the OECD average in 2009. Additionally, as a result of a decrease in science performance, Manitoba went from performing above the OECD average in 2006 to performing at the OECD average in 2009. Due to its decrease in performance, Prince Edward Island went from performing at the OECD average in mathematics (in 2003) and above the OECD average in science (in 2006) to performing below the OECD average in 2009 in both minor domains. Mathematics performance in Nova Scotia, Quebec, Ontario and Saskatchewan did not change between 2003 and 2009 and remained above the OECD average.
Table 2.5
Comparisons of performance in mathematics in PISA 2003, 2006 and 2009, Canada and the provinces
PISA mathematics score
2003 2006 2009
standard standard
average standard average error with average error with
score error score linkage error score linkage error
Newfoundland and Labrador 517 (2.5) 507 (3.1) 503* (3.4)*
Prince Edward Island 500 (2.0) 501 (2.7) 487* (3.0)*
Nova Scotia 515 (2.2) 506 (2.8) 512 (3.0)
New Brunswick 512 (1.8) 506 (2.5) 504* (3.0)*
Quebec 537 (4.7) 540 (4.4) 543 (3.9)
Ontario 530 (3.6) 526 (4.0) 526 (3.8)
Manitoba 528 (3.1) 521 (3.6) 501* (4.1)*
Saskatchewan 516 (3.9) 507 (3.7) 506 (3.8)
Alberta 549 (4.3) 530* (4.0)* 529* (4.8)*
British Columbia 538 (2.4) 523* (4.7)* 523* (5.0)*
Canada 532 (1.8) 527 (2.4) 527 (2.6)
* Statistically significant differences compared to PISA 2003
Note: The linkage error is incorporated into the standard error for 2006 and 2009.
Table 2.6
Comparisons of performance in science in PISA 2006 and 2009, Canada and the provinces
PISA science score
2006 2009
standard
average standard average error with
score error score linkage error
Newfoundland and Labrador 526 (2.5) 518 (3.9)
Prince Edward Island 509 (2.7) 495* (3.5)*
Nova Scotia 520 (2.5) 523 (3.7)
New Brunswick 506 (2.3) 501 (3.5)
Quebec 531 (4.2) 524 (4.1)
Ontario 537 (4.2) 531 (4.2)
Manitoba 523 (3.2) 506* (4.8)*
Saskatchewan 517 (3.6) 513 (4.5)
Alberta 550 (3.8) 545 (4.9)
British Columbia 539 (4.7) 535 (4.8)
Canada 534 (2.0) 529 (3.0)
* Statistically significant differences.
Note: The linkage error is incorporated into the standard error for 2009.
Summary
Because mathematics and science were considered to be minor domains in PISA 2009, a smaller proportion of students were assessed in those domains compared to the reading assessment, which was the major focus of the PISA 2009. Additionally, a smaller number of items were included in each of these assessments than were included in the reading assessment. Consequently, this chapter focuses on providing an update on overall performance in these two domains.
Canada continues to perform well internationally in both mathematics and science scoring well above the OECD average and being outperformed by seven countries in mathematics and six countries in science among 65 countries that participated in 2009. Across the provinces, students in Nova Scotia, New Brunswick, Quebec, Ontario, Saskatchewan, Alberta and British Columbia performed above the OECD average in mathematics. Students in Newfoundland and Labrador and Manitoba performed at the OECD average in mathematics while students in Prince Edward Island were below the OECD average. At the Canadian level, gender differences in performance existed with males outperforming females in both mathematics and science.
In five of the seven provinces where performance was examined by school-language sector, students attending majority-language schools generally outperformed students attending minority language schools in both mathematics and science. This performance gap was less pronounced in mathematics ranging from 11 to 41 score points compared to science ranging from 34 to 64 score points.
Canadian students’ performance in mathematics and science remained stable over time. However, as a result of a lack of improvement in performance, coupled
with increased performance in other countries, more countries outperformed Canada in mathematics and science than in previous PISA assessments. Additionally, a few countries participating in PISA for the first time in 2009 outperformed Canada in mathematics and science.
Although Canada’s performance in mathematics remained stable between 2003 and 2009, performance decreased in six of the ten provinces. Two of these provinces, Alberta and British Columbia, continued to have strong performance in PISA 2009, well above the OECD average. On the other hand, as a result of a decrease in performance in mathematics, Newfoundland and Labrador, New Brunswick and Manitoba performed at the OECD average in 2009 while Prince Edward Island performed below the OECD average. In addition, Manitoba and Prince Edward Island had a significant decrease in performance in science and consequently performed at and below the OECD average respectively in 2009.
Notes
18. OECD (2009). PISA 2009 Assessment Framework: Key competencies in reading, mathematics and science. Paris.
19. OECD (2009). PISA 2009 Assessment Framework: Key competencies in reading, mathematics and science. Paris.
20. Although Shanghai-China, Chinese Taipei and Singapore outperformed Canada in mathematics in PISA 2009, they are not included in this comparison since they did not participate in PISA 2003.
21. Although Shanghai-China and Singapore outperformed Canada in science in PISA 2009, they are not included in this comparison since they did not participate in PISA 2006.
Conclusion
Skills and knowledge play a crucial role in determining the economic success of societies and individuals and its importance is evident in today’s global economic environment. Equipping young people with the necessary skills provides them with the foundation for full participation in adult life as productive members of society and the economy. As such, the skill levels of youth are an important determinant of a country’s economic success. Governments around the world recognize the importance of skills and invest heavily in their education systems. The outcomes of these investments require monitoring and analysis to ensure that these outcomes are meeting countries’ needs.
The Programme for International Student Assessment was developed to provide a picture of the extent to which youth have acquired some of the knowledge and skills that are essential for full participation in modern societies. Developed by the Organisation for Economic Co-operation and Development, PISA 2009 measures the skill levels of 15-year olds in 65 countries in three key subject areas – reading, mathematics and science.
In addition to providing information on skill levels of countries, PISA also enables countries to monitor change in their performance over time. Implemented every three years since 2000, the 2009 PISA marks the fourth time that a comprehensive set of information on skills of 15-year olds has been available. For Canada, not only does PISA provide insight on the skill levels of its 15-year olds in an international perspective, it also provides an opportunity for individual provinces to compare themselves nationally and internationally and to monitor their change in performance over time.
The 2009 PISA results revealed that Canadian 15-year olds have relatively strong sets of skills in reading, mathematics and science. That Canada’s youth is equipped with a high skill level is an encouraging sign for the country’s future economic prosperity. However,
although Canadian results remained statistically similar between 2000 and 2009, its relative ranking declined in all domains. This decline is attributable to improvements in other countries’ performance and the introduction of new countries to PISA 2009 that had high performance.
In reading, the major domain of PISA 2009, Canadian results also indicate a decrease in the proportion of high achievers between 2000 and 2009. In a global economy, this decrease may be one indication of potential loss of future competitiveness.
Although Canada’s performance over time was not significantly different, several provinces experienced significant declines in their 15-year olds’ skill levels, mostly in reading and in mathematics. In addition, over the same time period, there was not a significant increase in performance in the three domains in any province.
The results also identified gender differences in performance as well as specific groups of 15-year olds who had significantly lower skill levels. Females continued to outperform males in reading, and males outperform females in mathematics and science although the gender gap is less pronounced in these two domains.
Additionally, 15-year olds attending minority-language school systems tended to perform lower than those attending majority-language school systems in all three domains.
The results presented in this report are only a highlight of what is possible with a rich database such as PISA. A second Pan-Canadian report, with more detailed analyses of factors associated with student performance, will be published in early 2011. The array of sources of information on the skill levels of Canadians is growing. Along with results from other studies such as the Pan-Canadian Assessment Program, the Progress in International Reading Literacy Study and the forthcoming Programme for International Assessment of Adult Competencies, a more complete picture of competencies of Canadians is emerging. These data sources will help to better our understanding of the levels of skills in the country, as well as the context in which learning is taking place.