Identifying Effective Education Interventions in Sub-Saharan Africa: A meta-analysis of rigorous impact evaluations

摘要

The aim of this dissertation is to identify effective educational interventions in Sub-Saharan African with an impact on student learning. This is the first meta-analysis in the field of education conducted for Sub-Saharan Africa. This paper takes an in-depth look at twelve different types of education interventions or programs and attempts to not only present analytics on their relative effectiveness, but to also explore why certain interventions seem to be more effective than others. After a systematic literature review, I combine 56 articles (containing 66 separate experiments, 83 treatment arms, and 420 effect size estimates), and I use random-effects meta-analytic techniques to both a.) evaluate the relative impact of different types of interventions and b.) explain variation in effect sizes within and across intervention types. When I examine the relative pooled effect sizes of all twelve intervention areas, I find that interventions in pedagogical methods (changes in instructional techniques) have a higher pooled effect size on achievement outcomes than all other eleven intervention types in the full sample (e.g., school management programs, school supplies interventions, or interventions that change the class size or composition). The pooled effect size associated with these pedagogical interventions is 0.918 standard deviations in the full sample (SE = 0.314, df = 15.1, p = 0.01), 0.566 in the sample excluding outliers and including only randomized controlled trials (SE = 0.194, df = 11, p = 0.01), and 0.228 in a sample that includes only the highest quality studies (SE = 0.078, df = 5.2, p = 0.03). These findings are robust to a number of moderating factors. Using meta-regression, I find that on average, interventions in pedagogical methods have an effect size over 0.30 standard deviations (significant at the 5% level) greater than all other intervention areas combined, even after controlling for multiple study-level and intervention-level variables. Beyond this average effect, I show that studies that employ adaptive instruction and teacher coaching techniques are particularly effective. Further, while studies that provide health treatments or school meals have on average the lowest pooled effect size, I show that if these studies are analyzed using cognitive assessments (tests of memory and attention), health treatments actually produce a relatively large pooled effect size of 0.176 standard deviations (SE = 0.028, df = 2.18); this is particularly true of studies that either prevent or treat malaria. In addition, this meta-analysis examines the state of current education impact evaluation research in Sub-Saharan Africa and highlights both research gaps as well as differences in study design, methodology, and reporting of metrics by academic field. I find that the bulk of the research in this area comes from the field of economics (62%), followed by the fields of education (23%) and public health (15%). Further, the majority of this research has been conducted in a set of six countries: Kenya, Nigeria, South Africa, Uganda, Burkina Faso, and Madagascar, while rigorous evaluations of education programs have never taken place in others. Moreover, topics currently under rigorous study are not necessarily representative of the major issues facing many Sub-Saharan African school systems today. For example, there are no impact evaluations of multi-grade or multi-shift teaching and only one evaluation of a bilingual education program. This meta-analysis thus recommends a shift in the impact evaluation research agenda to include both a broader geographic and topical focus, as well as an increased emphasis on improvements in pedagogical methods, without which other interventions may not reach their maximum potential impact.