PPT

Measuring Results in Education:
a State of the Field at the
International Level
M. Carton & S. Langstaff, NORRAG
Education Network, SDC, F2F
Morges, 18 September 2013
Outline of the presentation
1. Results-based approaches in education:
origins, key actors and initiatives
2. Measuring learning: limits and
challenges
Origins and development
• Public-sector management since the 19th in the
Anglo-Saxon world: emphasis on quantifiable and
easy to measure targets. Statistics for change and
control(Eyben 2013).
• In the development sector since the 2002
Monterrey conference on result-based aid :
conferences on aid effectiveness, e.g. Paris 2005,
Accra 2008, Busan 2011 (NN 42 and 47).
• From access to learning. Measuring learning.
Global policy-makers:
‘From Bono to Bill Gates, the
importance of more, better, and
transparent data is gaining attention.’
•
•
•
•
•
World Bank
DFID
Private foundations: Gates, Hewlett…
USAID
Washington Think-Tanks: Center for Universal
Education (Brookings Inst.), Center for Global
Development
• UNESCO: UIS and EFA-GMR
From assessment to measurement?
• Participation of low and middle-income
countries in international large-scale
assessments: from a handful in the 1970s to
half of them by 2010 (Lockheed 2012)
• From national evaluations to international
comparability. LMTF & PISA for Development:
towards a global framework post 2015? HLP:
statistics of quality for quality education
Large-scale learning
assessments
PASEC - Programme
By
Since
CONFEMEN 1990
d'analyse des systèmes
éducatifs de la CONFEMEN
What/When
Maths and French (or
language used as MoI)
Every year in selected
countries at the beginning and
at the end of the school year
SACMEQ -The Southern
SACMEQ
1995
and Eastern Africa
Consortium for Monitoring
Educational Quality
PISA - Programme for
Reading, mathematics
+ HIV/AIDS knowledge
Every 5-6 y.
OECD
1997
International Student
Assessment
Reading, maths and
sciences* (end of
compulsory schooling)
*The focus changes every 3 y.
TIMSS - Trends in
IEA – Int.
International Mathematics
and Science
Study
Assoc. For the
Evaluation of
Educational
Achievement
PIRLS/prePIRLS -Progress
IEA
1995
Maths and science
Every 4 y.
2001
Reading
in Reading Literacy Study
Every 5 y.
Who/Where
Primary school
learners in G2 and
G5 in Frenchspeaking countries in
WCA and Asia
Primary school
learners (G6) and
teachers in 16
countries in SEA
Randomly selected
school learners aged
15 in 70 countries
(2012) - LDCs excl.
School learners in G4
and G8 in 63
countries (2011)
Primary school
learners (G4-6) in 50
countries (2011)
Out-of-school population? Early drop-out?
NFET?
• Household surveys ? Potential and limits
• Early Grade Reading Assessment (USAID & WB,
since 2006), G1-3, 50 countries, 70 languages
• RAMAA - Measuring Learning Outcomes of Adult
Education Programmes: action research by UIL since
2010/11 for the population above 15 y.
•TVSD ?
‘What can be measured gets done‘
but…
• A NARROWING OF QUALITY through
standartised approaches. Context? Diversity?
• Measuring gender and educational quality:
parity of learning outcomes and inputs, +
alternative measures such as social learning
outcomes (e.g. beliefs of themselves and
others, changes in gender attitudes &
behaviors)?
DE la QUANTIFICATION à la MESURE
des RESULTATS via les
CONVENTIONS: le cas de l’éducation
Passage des mots aux nombres fondé sur
conventions d’équivalences:
 Comparaisons selon une échelle numérique,
négociations, compromis, traductions, procédures,
calculs puis mise en nombres
 Peut-on mettre en nombres = can we/ may we:
possibilité technique/autorisation?
 Mise en nombres fondée sur conventions:
dimensions socialement et cognitivement
créatrices: reflet+ transformation du monde et
non pas réification de la mesure
- + 60 conventions définissent l’ »Alpha » (UIL)
Variété de résultats des mesures: problème?
- « Intelligence » est-elle un concept, ou ce qui
est mesuré uniquement par le QI?
- « Opinion » est-elle une notion, ou ce qui est
mesuré uniquement par sondages?
Peut-on traduire notion qualité éd. en nombres,
c.a.d espace équivalence mots- nombres
permettant comparaisons et classements?
Post 2015: espace déjà clos? Si oui, réification
Si non: par qui, comment va être occupé
espace, et combien de temps avant fermeture
conventionnelle ? Réouverture quand, par qui?