Evidence from the UIS Catalogue of Student Assessments Mr Georges Boade, UIS

(1)

ED/TLC/LTR/2015/08 Original: English

Evidence from the UIS Catalogue of Student Assessments Mr Georges Boade, UIS

Consultation on National Assessments and

Measuring Learning for the Post-2015 Education Agenda

UNESCO Headquarters, 26-27 February 2015

(2)

Summary

The UIS Catalogue captures key meta-data on large-scale national student assessments and public examinations in primary and lower secondary schools (complementary to the World Bank’s

SABER). The purpose is to better understand data sources and to identify potential models to share with other countries. The first wave of the Catalogue includes information on 12 countries in Sub-Saharan Africa, 10 countries in Asia, and one country in Latin America and the Caribbean. It will target 47 countries in Africa, 16 countries in Asia and 19 countries in Latin America and the Caribbean. It will be updated regularly when new data are available. The country summaries and reports provide an overall picture of the assessment system in the country and highlight the characteristics of assessments included in the catalogue.

A number of key challenges are highlighted, including:

• Different organizations or ministerial departments implementing some national assessments or examinations, implying that catalogue focal points are from different organizations. This implies a coordination mechanism among different actors to access required information, but this access is not always easy for the national focal.

• National reports are not available for most countries included in the project and there are as yet few details on country level sampling strategies, assessment frameworks,

stakeholders and data dissemination strategies.

(3)

UNESCO Institute for Statistics

Consultation on National Assessments and Measuring Learning for the Post-2015 Education Agenda.

UNESCO Headquarters, Paris 26-27 February

Session I: What do assessment systems look like?

Evidence gathered through UIS catalogue of learning outcomes survey

By Georges Boade

Email: [email protected]

(4)

Target of the UIS survey

 Student population: ISCED 1 & 2 general and technical/vocational education programs.

 Type of studies: national assessments and public examinations

 Type of data: metadata- descriptive characteristics of the existing

assessments: Scope, purpose, funding, test design, coverage, sampling and dissemination.

Note: While scores are not always comparable between assessments and across countries, their metadata are.

(5)

Target of UIS survey (cont’d)

But alternatives to study in general or technical program orientation vary from one country to another:

(1) Students follow a regular technical program only after successful

completion of the fundamental education (first nine years of studies in Mali, Mauritania, Haiti and Thailand) – age 15+

(2) Students have to choose between general and technical program after completion of primary education (ISCED 1): Cameroon, Chad, Côte d’Ivoire, etc.

3

(6)

Five categories of countries

Classification variables: existence of national assessment and participation to international assessment:

1) Countries involved both in national and international assessments in ISCED 1 or/and ISCED 2

2) Countries involved in national assessments only in ISCED 1 or/and ISCED 2

3) Countries involved in international assessments only in ISCED 1 or/and ISCED 2

4) Countries not involved in assessments but having public examinations in ISCED 1 or/and ISCED 2

5) Countries not involved in assessments and in public examination in ISCED 1 or ISCED 2.

Draft examples catalogue entries:

 http://www.uis.unesco.org/nada/en/index.php/catalogue/learning_assessments ₄

(7)

Emerging issues in national assessments

 School institutions involved in national assessments are not always

representative: some only cover public institutions where private schools also provide educational programs in the target grades: Cambodia, Malawi,

Myanmar, Pakistan, Vietnam.

 Irregularity and discontinuity in the administration of given national assessments

 When they cover the priority domains for global monitoring, learning areas assessed do not have the same definition across countries, hence distribution of test items across the main content areas and cognitive processes vary from one assessment to another even if the target population is the same.

 Proficiency levels: No clear definition of what students are able to do at each level. Most of them refer to a range of score without more explanation

 Scoring and metrics are not always comparable: standard score, percentages out of total score, percentage items correct.

 Lack of up to date assessment frameworks ₅

(8)

Questions:

6

Q1: What are the priority grades to assess students in numeracy and literacy skills if countries were to refer to your advice?

Q2: Most national and international assessments target only general education programs- Is there a strategy for a global monitoring of learning outcomes for students in technical or vocational programs, particularly for countries in point 2 above- age 12+?

Q3: Do we need a national assessment that applies both to general and vocational education programs at ISCED 2 grades equivalent?

Q4: What if existing assessments in a given country only cover students in public schools?

Q5: What if there is no standard national assessment of students for all states or provinces or regions in a Federal country?

Q6: To what extend results in public exam can be used for global monitoring of learning outcomes?

(9)

Merci!

7