TIMSS is designed to assess mathematics and science achievement of students in their fourth and eighth year of schooling. The international target population in TIMSS is defined in terms of the number of years of formal schooling to ensure the international comparability of results. In Canada and in most other countries, the target grade of four years of schooling would be Grade 4; similarly, the target grade of eight years of schooling would be Grade 8 (or Secondary II in Quebec)15. TIMSS uses a two‑stage random sample design, with a sample of schools drawn as a first stage and at least one intact class of students selected from each of the sampled schools as a second stage. However, school‑entry age varies across countries. Therefore, in order to avoid testing very young students, the average age of fourth grade students at the time of testing must be at least 9.5 years. In Canada, Grade 4 students were on average 9.9 years old at the time of testing.
The 2019 cycle of TIMSS marked the beginning of the transition from a paper‑based to a digital assessment.
TIMSS was offered both in a digital format as well as in a paper‑and‑pencil format. More than half of the participating countries, including Canada, administered the digital version of the assessment, while the other half administered the paper version. To control for mode effects and allow data linking across modes, the administration of a bridge study was required. As part of the bridge study, countries transitioning to the digital assessment of TIMSS administered the paper version of the trend items to a separate, equivalent sample of students. In Canada, the bridge study was administered to a sample of 1,604 Grade 4 students across 83 schools. While the results of the bridge study were used to control for mode effects and to link the digital and paper versions of TIMSS, the achievement scores of students who participated in the bridge study were not included in the reporting of TIMSS achievement results. Likewise, the results of the problem solving and inquiry (PSI) items (see the introduction) are not included in this report. Consequently, the schools and students that participated in the bridge study or students who were administered PSI items are not included in the exclusion and participation rates presented in this section of the report, consistent with the TIMSS international report.
The total weighted rate of school‑level exclusions in Canada was 3.1 percent (Table A.1). These included geographically remote schools, schools having very few students, schools with a radically different grade structure or curriculum, and schools providing instruction solely to students with special needs, as determined by the provincial education authority. At the provincial level, school‑level exclusions ranged from 2.3 percent in Ontario to 6.7 percent in Manitoba.
The total weighted rate of student‑level exclusions in Canada was 3.9 percent (Table A.1). These included:
• Students with functional disabilities. This category comprised students who had permanent physical disabilities such that they could not perform in the TIMSS testing situation. Students with physical disabilities who were able to take the test had to be included.
• Students with intellectual disabilities. This category consisted of students who were considered, in the professional opinion of the school principal or other qualified staff, to have intellectual disabilities and/
or who had been psychologically tested as such. The category included students who were emotionally
15 The sample size of Grade 8/Secondary II was not large enough to obtain overall results for Canada. Therefore, this report, including this section on the exclusion and participation rates, will cover the results at the Grade 4 level only.
or mentally unable to follow even the general instructions of the test. It should be noted that students could not be excluded solely because of poor academic performance or normal disciplinary problems.
Systematic exclusion of all students with dyslexia, or other such learning disabilities, was not acceptable–
students had to be accommodated in the test situation, if possible, rather than excluded.
• Non-native language speakers. This category included students who were unable to read or speak the language of the test (English or French) and would be unable to overcome the language barrier in the test situation. Typically, a student who had received less than one year of instruction in the language of the test had to be excluded.
It was the responsibility of individual schools to determine whether a student should be included or excluded from participating in the TIMSS assessment, based on the international guidelines described above. At the provincial level, student‑level exclusions ranged from 1.2 percent in Quebec to 6.2 percent in Alberta.
The exclusion rates must be kept to a minimum so that national samples accurately represent the national target population. The overall weighted student exclusion rate (including both school‑level and within‑school exclusions) in Canada was 7.0 percent, which exceeds the maximum exclusion rate of 5 percent allowed by quality standards in TIMSS. The overall weighted student exclusion rate ranged from 4.4 percent in Quebec to 10.7 percent in Manitoba. Further steps will be required in future TIMSS cycles to address the issue of high exclusion rates for schools and students in some provinces.
Table A.1 TIMSS 2019 exclusion rates by type of exclusion School-level
exclusions (%) Student-level
exclusions (%) Overall (%) Grade 4
Newfoundland and Labrador 6.1 3.6 9.7
Quebec 3.3 1.2 4.4
Ontario 2.3 4.7 7.0
Manitoba 6.7 4.0 10.7
Alberta 3.9 6.2 10.1
Canada 3.1 3.9 7.0
Note: Non-participating provinces/territories are taken into account when calculating the exclusion rates for Canada overall.
In order to minimize the potential for non‑response bias,16 TIMSS quality standards require minimum participation rates for schools, classrooms, and students. At the national level, for a sample to be fully acceptable, it must have either:
• a minimum school participation rate of 85 percent, based on originally sampled schools, and
• a minimum classroom participation rate of 95 percent, from originally sampled schools and replacement schools, and
• a minimum student participation rate of 85 percent, from sampled schools and replacement schools, or
• a minimum combined school, classroom, and student participation rate of 75 percent, based on
originally sampled schools (although classroom and student participation rates may include replacement schools).
Tables A.2 and A.3 show school and student sample sizes, and Table A.4 shows school, class, and student participation rates. In Canada overall, the weighted school participation rate based on originally sampled
schools was 86 percent, the weighted classroom participation rate was 100 percent, and the weighted student participation rate was 95 percent. Thus, the international standards for participation in the assessment were successfully maintained in Canada. Furthermore, the overall weighted participation rate (at school, classroom, and student levels) was 86 percent in Canada. This overall weighted participation rate ranged from 78 percent in Alberta to 91 percent in Manitoba.
Table A.2 TIMSS 2019 school sample sizes Number of schools that
participated Grade 4
Newfoundland and Labrador 100 98 91 0 91
Quebec 172 172 140 8 148
Ontario 171 170 160 3 163
Manitoba 165 164 158 24 158
Alberta 180 173 120 0 144
Canada 788 777 669 35 704
† This number includes participating, not participating, and excluded schools.
†† This number includes participating and not participating schools.
Table A.3 TIMSS 2019 student sample sizes in participating schools Number of
sampled students in participating
schools†
Number of eligible students in the
sample††
Number of
students absent Number of students assessed Grade 4
Newfoundland and Labrador 1,475 1,370 124 1,246
Quebec 4,047 4,001 164 3,837
Ontario 4,251 4,073 243 3,830
Manitoba 2,611 2,462 163 2,299
Alberta 2,780 2,630 189 2,441
Canada 15,164 14,536 883 13,653
† This number includes participating, not participating, and excluded students.
†† This number includes participating and not participating students.
Table A.4 TIMSS 2019 participation rates (weighted) School participation (%) Class
participation
and Labrador 93 93 100 92 85 85
Quebec 82 86 100 96 79 83
Ontario 93 95 100 95 88 90
Manitoba 97 97 100 94 91 91
Alberta 68 83 100 93 64 78
Canada 86 90 100 95 82 86