Sharing IEA’s results and information

report that provides extensive high-quality information on students’ achievement outcomes and their educational contexts and helps countries to assess their educational systems in a comparative context.

Countries that have participated in previous cycles of the same assessment may also gain insights into their own national trends, as well as the international trends illustrated from the longitudinal data collection.

Reports are available online, allowing readers to search and download or print particular topics and information of their interest.

The IEA’s international databases allow for public access to the data collected and processed by each of its studies. All participating countries contribute by releasing their national data as part of these databases. The databases provide student achievement data, as well as background information about curricula and learning environments: students, home (in the case of PIRLS and TIMSS), teachers and schools. All data are anonymised so that scores cannot be linked to individual students or schools.

Alongside the main international report, our supporting publications help stakeholders to understand and work with these data. User guides describe the organization and content of each database. This complements the assessment framework’s theoretical overview of the study and the documentation

describing the rationale for the techniques used and the variables created in the process of data collection and compilation (technical report or methods and procedures). Each study’s encyclopedia also gives

information from participating countries on the structure of their educational systems and school curricula. In order to work effectively with any of IEA’s data, it is necessary to consult these publications to understand the characteristics of the study.

In addition to supporting researchers by providing data and documenting procedures, the IEA invests in methodological research with potential to provide new insights in the field of methodology and interpretations. To share its work and increase the impact of the research, the IEA invests in open source publishing of articles in the IEA-ETS Research Institute Journal, large-scale assessments in education and books as part of the Springer, IEA Research for Education series. It also offers workshops and training courses to support researchers working with large-scale assessment data. This commitment to making IEA’s research publicly available has been an effective strategy for increasing the reach of findings.

IEA studies are familiar to many scholars and other education specialists within the subject areas of reading, mathematics, science, civics and citizenship, information and communication technology

and teacher education. Its studies are also well-known by experts interested in methodology and statistical analysis of large-scale data on educational achievement worldwide. They are less well known by researchers in other fields, many practitioners and decisionmakers responsible for educational policy, particularly in the countries that are not represented among IEA membership. The IEA is addressing these issues through its commitment to publishing findings in open datasets and by actively sharing and promoting its results as a solid evidence base for researchers, educators and policymakers worldwide.

IEA data are recognised by UNESCO as invaluable for monitoring progress toward SDG 4, which encompasses a wide range of aspects related to education. Its longitudinal datasets include information about student achievement in core subjects (literacy, mathematics and science). This is in addition to contextual information about learning environments,

access to education and the development of cross-curricular competencies in areas such as civics and citizenship and digital literacy (see Section 3.2.1).

Supporting educators

Once a study is completed, there are many resources that can be used by teachers and other stakeholders in classrooms and schools. Teachers, teacher educators and researchers may access some of the test items used in the assessments to understand what tasks students are expected to accomplish.

They may look for items by content domains, such as algebra and geometry, or cognitive domains, such as knowing and reasoning. In addition, they can see what percentage of tested students across the participating countries answered each question correctly to gain insights into how and where students may be struggling. Background data almanac files contain weighted summary statistics for each participating country on each variable in the student, home, teacher and school context questionnaires, including the context questionnaire scales. This approach helps to identify the obstacles where students are not performing as well as expected and the potential interventions to address those challenges.

Advantages of a curriculum-based approach

The IEA “curricular model” enables practitioners, educators and researchers to review, interpret and utilise results in their national context. This often starts with the production of a national report directed by, and directly used by, the educational community and decisionmakers within a single country. In some cases, countries team up in order to analyse matters of their common interest or to focus on differences and similarities among them (see Northern Lights on TIMSS and PIRLS 2011 as an example).

Linking research to policy and practice Both international and national findings can lead to direct interventions and policy changes based on

Figure 1.1 Interim reporting of SDG 4 indicators

study results, especially by teacher educators who shape teaching practice at a national level.

For the data to have a direct impact on pedagogy in the classroom, researchers and local teachers usually need to collaborate. One example is the Oxford University’s PIRLS for Teachers project that aimed to use PIRLS data to provide teachers in England with guidance on improving their own teaching of reading in primary schools. In response to a request from the teachers for visual materials that they could display in a staff room, the project produced two posters about best teaching practices that were intended to prompt teachers to reflect on the methods they used to teach reading. The first poster encouraged oral after-reading activities – such as talking with peers and answering questions about what they have read – to foster interest in reading and motivate students who had limited exposure to books at home. The second poster summarised analyses revealing that boys were more motivated for reading and lower achieving pupils were more engaged with reading when these groups were exposed to a variety of reading resources (Hopfenbeck and Lenkeit, 2018).

While it is encouraging to see the influence of IEA study results on good practice at the classroom and school levels, findings must also reach and influence policymakers to achieve a more lasting impact. As an independent, non-political organization, the IEA does not make specific policy recommendations for individual education systems. The country-specific context and culture demand a lot of insider knowledge; from identifying the right questions to analysing, interpreting and understanding the results, while taking national influencing factors into account.

The IEA supports countries by facilitating knowledge sharing and offering inspiration for potential evidence-based pedagogical and policy interventions. The IEA Compass: Briefs in Education series – formerly known as the IEA Policy Briefs – is made up of short, accessible articles published on its website and aimed at a general audience. The goal of the series is to connect study findings to recurrent and emerging questions in educational debates at the international

and national levels to provide an evidence base for practitioners who are engaged in developing solutions for their own, national educational challenges.

Outcomes of IEA studies have influenced educational policy across its member countries. Table 8.2 provides an overview of some of these changes.

The changes can be grouped into four main areas:

curricular changes, teacher’s education, professional development and support, focusing on a specific group or a specific need of students, and material supports like textbooks, libraries, and other forms of physical mechanisms that support pedagogy.

In some cases, it is also possible to document where IEA study results led to the launch of a new agenda for an education system. For example, Germany has developed a dedicated digital agenda for education after ICILS 2013 results revealed a relatively low achievement level for students accompanied by a lack of computers in schools and adequately trained teachers (Fraillon et al., 2014).

Most changes attributed to IEA study results involve curricular amendments. This can be understood by the fact that the researchers engaged in IEA studies are often scholars active in teacher education. TIMSS and PIRLS have proved to be the most powerful agents of change, particularly because their design allows for the monitoring of trends over time. This is especially important in developing education systems that are striving to achieve universal enrolment where children from the most socioeconomically-disadvantaged communities are usually the last groups to be reached. These children need particular attention and lessons from TIMSS and PIRLS have helped countries to develop tailored pedagogical tools to engage them, as documented in the TIMSS 2015 and PIRLS 2016 Encyclopedias (Mullis et al., 2016a, 2017a and 2017c).

For example, Morocco has made significant improvements to both the equity and quality of its education system which are demonstrated in the country’s achievement scores over time (see

Table 8.2 Examples of the impact of IEA study results on national education systems

Country Action Study

Belgium Support policies for low socioeconomic status and immigrant students. ICCS

Botswana Curriculum amendments; guidelines for classroom testing. PIRLS and TIMSS Canada

(Ontario) Curriculum changes; more time for mathematics and reading instructions. PIRLS and TIMSS Chinese Taipei Results used as one of the primary resources in evaluating the efficacy of

mathematics and science education and curriculum development.

Focus on assistance for low-performing and disadvantaged students in mathematics and science.

TIMSS

Czechia Series of teacher manuals developed based on the most common misconceptions and errors of Czech students.

TIMSS Egypt Curriculum amendments; introduction of new teaching methods fostering

interaction between students and teachers.

TIMSS England Teacher training programmes to stimulate positive attitudes towards reading. PIRLS Hong Kong,

SAR of China Teacher training programmes and other initiatives to stimulate children’s

reading. PIRLS

Hungary Extending reading teaching to Grade 6. PIRLS

Indonesia Focus on second language learners. PIRLS

Jordan Revision of the mathematics and science curricula; use of released items in the development of textbooks; development of related teacher guides and trainings.

TIMSS

Latvia Lowering the school entry age from 7 to 6; new guidelines for teaching primary grades.

PIRLS Lithuania In-service training for primary grade teachers aimed at improving their teaching

methods.

PIRLS and TIMSS Malaysia Measures to address students’ lack of opportunity for application of knowledge

and to develop higher order thinking skills (HOTS), including teacher trainings, textbook reviews, and increasing HOTS items in national assessments; new curriculum since 2011.

TIMSS

Oman Improvements of curricula and revision of learning outcomes; teacher training focused on question development according to the cognitive domains and incorporating them in classroom instruction.

PIRLS TIMSS New Zealand Focus on reading achievement of Maori and Pacifica children. PIRLS Norway More focus on reading instruction, including an earlier start of reading

instruction. PIRLS

Romania Curriculum amendments; new teacher guides as well as new science text

books issues; more emphasis on reading informational texts. TIMSS and PIRLS Russian

Federation Alignment of achievement goals with the frameworks of the international

large-scale assessments. PIRLS, TIMSS

Singapore Focus on policies supporting lower-performing, lower-socioeconomic status students.

PIRLS, TIMSS South Africa Support programmes for school and classroom libraries. PIRLS

Spain Reading promotion. PIRLS

United Arab

Emirates Sharing the best practices of teaching. TIMSS and PIRLS

Note: Further details may be found in the references (Aggarwalla, 2004; Elley, 2002; Gilmore, 2005; Schwippert, 2003; Schwippert and Lenkeit, 2012).

Source: IEA.

Figure 1.1 Interim reporting of SDG 4 indicators

Table 8.3). Almost all children have access to school (net enrolment in Grade 1 is above 97%), and Morocco now administers diagnostic tests at the beginning of the school year to facilitate student grouping so that specific learning support programmes can be designed and implemented for students with similar difficulties. PIRLS findings also influenced Morocco’s Reading for Success project which encourages children to read for pleasure. This approach has proved to improve children’s motivation to read and help them advance their reading

proficiency (see Table 8.2).

8.3.3 The challenges of using IEA data