3. Learning evidence for Indicator 4.1.1
3.1 Learning evidence for Indicator 4.1.1 from regional assessments
3.1.1 Latin American Laboratory for Assessment of the Quality of Education (LLECE)
(LLECE)
LLECE is the leading quality assessment educational network in Latin America. It conducts the region’s most representative evaluation of learning outcomes
17 One regional assessment is excluded: SEA-PLM (Southeast Asian Primary Learning Metric) because it will be first administered in 2019.
Table 3.1 Summary of cross-national initiatives
Grade/age Assessments
1 EGMA, EGRA
2 EGMA, EGRA, PASEC
3 EGMA, EGRA, LLECE
4 PILNA, LANA , PIRLS, TIMSS
5 SEA-PLM
6 LANA, PASEC, PILNA,
SACMEQ, LLECE
8 TIMSS
15-year-olds
(Grade 7 or above) PISA
14- to 16-year-olds PISA-D
5- to 16-year-olds ASER, Uwezo
Source: Treviño and Ordenes, 2017.
Figure 3.1 The geographical coverage of regional assessments
in primary education, the Regional Comparative and Explanatory Study (ERCE). This pan-Latin American network is made up of national-level directors of educational assessments in Latin American and Caribbean countries (LAC) and has its seat in the Regional Office for Education in Latin America and the Caribbean in Santiago, Chile. LLECE is also an important forum for analysing new approaches to educational quality and evaluation and for the discussion of learning outcomes. Importantly, it serves as an instrument for training and professional development of national technical teams.
Access to education is not the main challenge in LAC, where 95% of children are in school. However, ensuring that children learn well in the classroom and measuring their learning outcomes is critical to improving the quality of education in the region.
Recent research and surveys in preparation for SDG 4 have shown that, despite progress in the domain of access, the quality of learning is an issue in education systems, as well as the availability and access to educational resources.
Historically, the quality of assessments in most Latin-American countries has been uneven, with little knowledge of advanced student assessment. Many have government-conducted evaluations, but results have not been publicised. In fact, there has often been high resistance to publishing evaluation results and intense diplomatic efforts have been required to seek support from countries. The only country that has published the results of its assessments is Chile.
In 1994, Chile’s system was extended to all countries in LAC and established as a regional cooperation framework for the region. In 1994, LLECE had 15
Source: UNESCO Institute for Statistics (UIS).
Figure 3.1 The geographical coverage of regional assessments
LLECE
Regional Assessment PASEC
SACMEQ SEA-PLM PILNA
founding members. Currently it includes 19 education systems in LAC.
The work of LLECE in assessing the quality of education
The three main objectives of LLECE assessments are: to promote evidence-based education policy through the generation of (empirical) data on quality education and associated factors; to develop education assessment capacities; and to serve as a forum to generate and share ideas and best practices in education.
LLECE works through regional assessments within all contributing LAC countries to assess primary education in language, mathematics and science.
So far, there have been three regional assessments:
Primer Estudio Regional Comparative y Explicativo (First Regional Comparative and Explanatory Study) (PERCE), Segundo Estudio Regional Comparative y Explicativo (Second Regional Comparative and Explanatory Study) (SERCE) and Tercer Estudio Regional Comparative y Explicativo (Third Regional Comparative and Explanatory Study) (TERCE).
LLECE published PERCE on learning achievements in reading and mathematics among students in third and fourth grades in primary education. Its second regional study, SERCE, was implemented in 2006 and published in 2008. Among its innovations, SERCE applied the assessment of writing skills as well as a third discipline – science. LLECE’s third study, TERCE, initiated in 2013, was a large-scale study of learning achievements implemented in 15 countries.18 TERCE worked with its implementation partners, the Center for Measurement Mide UC, Pontifical Catholic University of Chile (UC), the Centre of Compared Policies of University Diego Portales (Chile) and the Colombian Institute for Educational Evaluation (ICFES), to develop the research tools and training that would lead to capacity building and the correct use of data.
18 Argentina, Brazil, Chile, Colombia, Costa Rica, Dominican Republic, Ecuador, Guatemala, Honduras, Mexico, Nicaragua, Panama, Paraguay, Peru and Uruguay, as well as the Mexican state of Nuevo Leon.
The results of TERCE have been measured against those of SERCE (2006). This comparison demonstrated the changes which have occurred in the performance of the education systems of participating countries over the last seven years.
Specifically, the results allowed learning achievements to be compared between pupils in Grades 3 and 6 in mathematics and reading tests. This was applied to all countries which participated in both studies.
In addition, the natural sciences test results were compared for the eight countries for which data on both measurements were available. Participation in the science test was voluntary in SERCE and it was applied in only a few countries.
Other innovations in TERCE are the “national modules” of associated factors, which enabled countries to study in greater detail the factors which affect learning. It included a module to study the impact of the use of ICT on the quality of education and the relationship between nutrition and learning.
TERCE also integrated the expertise of world-renowned experts in educational assessments.19
For the next study, ERCE 2019, there are two main innovations: a module to study in detail which pedagogical practices affect student learning and measure their impact; and a module to assess the development of socio-emotional skills, expanding the domains of evaluation of the LLECE’s studies, according to the role of global citizenship education in the 2030 Agenda.
The national modules are very relevant for countries.
For the implementation of ERCE-2019, one-half of the participating countries have developed a module to study specific topics of national interest. Some of these topics are: inter-culturality, perception of family support, pedagogical activities in science classes, impact of armed conflict, etc.
19 The databases for these three studies are available at: http://www.unesco.
org/new/en/santiago/education/education-assessment-llece/perce-serce-databases/
Figure 3.2 Participation of Latin American countries in cross-na-tional assessments
Reviewing and strengthening educational policies
In 2018 and 2019, the region will hold its fourth survey of regional assessment. A newcomer will be Bolivia.
Cooperating with LLECE, Bolivia initiated an induction and capacity-building process for the Bolivian Ministry of Education and the Plurinational Observatory
of Educational Quality (OPCE), strengthening the technical and institutional capacities of these entities.
Within this framework, a national diagnostic was applied in order to serve as a record (baseline) for participation in ERCE 2019, which will be applied in 18 countries of the region. The ministry of education assumes responsibility for the enormous task of objectively evaluating what has been, is and will be
TERCE/ERCE PISA
TIMSS PIRLS
Figure 3.2 Participation of Latin American countries in cross-national assessments
Source: LLECE.
the Plurinational Educational System, and to rely on empirical evidence in the design of policies to improve the quality of education in LAC countries.
Bolivia’s choice to participate in the LLECE evaluation model is a testament that the LLECE evaluation mode is better suited to the needs and characteristics of the region’s countries; is based on national theoretical curricula and constructs; and makes it possible to address issues of specific relevance to Latin America.
The study includes an evaluation of the factors associated with achievements in apprenticeships through questionnaires directed at school directors, teachers, families and students, with the objective of understanding how various socioeconomic and other factors affect the results. LLECE is acknowledged as creating a unique database which will help ministers of education in Latin America to make informed decisions on education policies based on the results of these investigations and the work of LLECE.