What are the options for reporting on SDG indicator 4.1.1?

Indicator 4.1.1 refers to the “proportion of children and young people (a) in Grade 2 or 3; (b) at the end of primary education; and (c) at the end of lower secondary education achieving at least a minimum proficiency level in (i) reading and (ii) mathematics, by sex” 

There are a variety of options for countries to use to report on SDG indicator 4.1.1. The options make use of countries’ existing or planned assessments and include: 

• Cross-national assessments: these are international and regional assessments, that streamline reporting for participating countries, organized by the level of indicator that the data reports to (a: Grade 2/3, b: end of primary, c: end of lower secondary). 
• National assessments: the data of such assessments should be aligned with the global MPL in order to produce data comparable at the global level. A number of methodologies may be used to map data when a national assessment is used.

Each assessment option and mapping methodology comes with costs and benefits. Education officials are encouraged to explore and choose the path forward that works best for their unique circumstances and objectives.

Note: Though reporting on all three levels of 4.1.1 is ideal, there are options for reporting on one or two levels, if necessary. The partial data is still a valuable addition to a country’s understanding of student achievement in the global context.

Assessment options for reporting to indicator 4.1.1a

International assessments

Progress in International Reading Literacy Study (PIRLS)
PIRLS is organized by the International Association for the Evaluation of Educational Achievements (IEA). It measures reading skills at grade 4 and the data can be used to report to indicator 4.1.1a.

Trends in International Mathematics and Science Study (TIMSS -4th Grade)
TIMMS is also organized by the IEA. It measures mathematics and science knowledge at grade 4 and the data can be used to report to indicator 4.1.1a.

Regional assessments 

Regional Comparative and Explanatory Study (ERCE):
ERCE focuses on countries in Latin America and measures mathematics, reading and writing skills at grades 3 and 6. The grade 3 data can be used to report to indicator 4.1.1a.

Programme for the Analysis of Education Systems (PASEC):
PASEC is intended for francophone countries in West Africa and measures reading and mathematics at grades 2 and 6. The grade 2 data can be used to report to indicator 4.1.1a.

Pacific Islands Literacy and Numeracy Assessment (PILNA)
PILNA measures literacy and numeracy knowledge at grades 4 and 6. The grade 4 data can be used to report to indicator 4.1.1a.

Grade 2/3 (4.1.1.a): Predominance of national assessments and only 2 regional assessments

Assessment options for reporting to indicator 4.1.1b

International assessments

Progress in International Reading Literacy Study (PIRLS)
PIRLS measurement of reading skills at grade 4 and the data can be used to report to indicator 4.1.1b (as well as indicator a, noted in that section).

Trends in International Mathematics and Science Study (TIMSS -4th Grade)
The TIMMS 4th grade measurement of mathematics and science knowledge at grade 4 data can be used to report to indicator 4.1.1b (as well as indicator a, noted in that section).

Regional Assessments

Regional Comparative and Explanatory Study (ERCE):
ERCE is intended for countries in Latin America and measures mathematics, reading and writing skills at grades 3 and 6. The grade 6 data can be used to report to indicator 4.1.1b.

Programme for the Analysis of Education Systems (PASEC):
PASEC is intended for francophone countries in West Africa and measures reading and mathematics at grades 2 and 6. The grade 6 data can be used to report to indicator 4.1.1b.

Pacific Islands Literacy and Numeracy Assessment (PILNA)
PILNA measures literacy and numeracy knowledge at grades 4 and 6. The grade 6 data can be used to report to indicator 4.1.1b.

Southeast Asia Primary Learning Metrics (SEA-PLM)
SEA-PLM is an assessment conducted by UNICEF. It measures reading, writing, mathematics and global citizenship skills at grade 5 and the data can be used to report to indicator 4.1.1b.

The Southern and Eastern Africa Consortium for Monitoring Education Quality (SACMEQ):
SACMEQ measures reading and mathematics skills at grade 6 and the data can be used to report to indicator 4.1.1b.

End of primary (4.1.1.b): Multiple assessment options (international, regional and national)

Assessment options for reporting to indicator 4.1.1c

International assessments 

Programme for International Student Assessment (PISA)

The Organization for Economic Co-operation and Development (OECD) conducts PISA. It measures the reading and mathematics knowledge of 15-year olds and the data can be used to report to indicator 4.1.1c.

Trends in International Mathematics and Science Study (TIMSS -8th Grade)

TIMMS 8th grade assessment measures mathematics and science knowledge at grade 8. The data can be used to report to indicator 4.1.1c.

End of lower secondary (4.1.1.c): Two major international assessments (TIMSS and PISA) and no regional assessments

What if a country uses national assessments?

If a country uses national assessments as the primary measurement of student achievement, there are a variety of methodologies that can be used to map that data to indicator 4.1.1. The methodologies include:

Statistical linking:

These are methods that can help countries map existing assessment data to regional and international assessment data so that it can be used to report to the 4.1.1 indicator.

Examples include:

• Rosetta Stone Study: The Rosetta Stone is a pilot program designed to establish a relationship between existing national and regional assessment data and international assessment scores so that the existing data is relevant to the indicator 4.1.1.More about Rosetta Stone
• AMPL: AMPL has 3 modules (AMPL-a, -b, -c), with each measuring 4.1.1 indicators at the following three levels of education (a: Grade 2/3, b: end of primary, c: end of lower secondary). AMPL modules can be administered as standalone assessments or as sections integrated into existing assessments. AMPL-b modules, designed to measure the proportion of students meeting the Minimum Proficiency Level (MPL) in reading and mathematics at the End of Primary, were administered in 2021 alongside national or regional assessments and aligned to the Global Proficiency Framework.
  More about AMPL
  
  AMPL Brochure

Non-statistical linking: 

This is a methodology that draws educators, curriculum and subject matter experts together to determine the relationships between existing data and indicator 4.1.1 proficiency levels.

Policy Linking is an example of this method.

• Policy Linking: This entails training in-country reading and mathematics experts (mainly teachers) to use an internationally approved judgment method to set the minimum score students must achieve to meet the global definition of MPLs. More about Policy Linking

What are other alternatives to measure learning?

There are circumstances when the data collection system does not yield results equivalent to a national assessment. In these cases, countries can choose to use existing school- or community-based assessments of reading and mathematics that align with the GPF or develop assessments that align with the GPF and apply the Policy Linking method to interpret results. Alternatively, countries can administer AMPL as a stand-alone national assessment or integrate portions of the AMPL into the assessments being built. Policy Linking can then be used to map the data from the AMPL to indicator 4.1.1.

Some population-based assessments are being explored as a source of learning outcomes data, such as UNICEF’s Foundational Learning Module, PAL or citizen-led assessments, and EGRA/EGMA. Such population-based assessments can be used to target populations that may or may not be enrolled in or attend school, capturing therefore skills regardless of enrolment. They include any household surveys that have an assessment component in their data collection (e.g. UNICEF’s MICS). A particular case are citizen-led assessments which originate in non-governmental organizations or think tanks and differ in the way they gather the data. (Pal Network). The Early Grade Reading Assessment (EGRA) is a test students take that can measure their skill at both pre-reading and reading subtasks and the Early Grade Mathematics Assessment (EGMA) is an orally administered assessment of the core mathematical competencies taught in primary grades. Usually, EGRAs are given to students in kindergarten through primary school and test children’s skill at different subtasks they need to learn to be able to read fluently. The test is typically administered by a teacher one-on-one with a student, out loud. With USAID support, EGRA and, to a lesser extent, EGMA have been used to inform program design and continuous improvement in dozens of countries, but they face the challenges of comparability, quality assurance, validity, reliability, proper sampling and assessment of the technical quality.

Both population-based surveys and school-based assessments collect background information that add context to data on learning outcomes. By including children and young people in and out of school, population-based surveys provide information on families and enabling environments. School-based assessments provide system-level information on classroom and school environments and sometimes gather information about the home environment either via a parent or via child recall. Together, they help to provide a snapshot of how children and youth around the world are learning. However, the results from these different types of assessments cannot, for now, be legitimately compared internationally or internally within a country.