Some children left behind: Variation in the effects of an educational intervention

Julie Buhl-Wiggers, Jason T. Kerwin, Juan Muñoz-Morales, Jeffrey Smith, Rebecca Thornton

Research output: Contribution to journalArticlepeer-review

1 Scopus citations

Abstract

We document substantial variation in the effects of a highly-effective literacy program in northern Uganda. The program increases test scores by 1.4 SDs on average, but standard statistical bounds show that the impact standard deviation exceeds 1.0 SD. This implies that the variation in effects across our students is wider than the spread of mean effects across all randomized evaluations of developing country education interventions in the literature. This very effective program does indeed leave some students behind. At the same time, we do not learn much from our analyses that attempt to determine which students benefit more or less from the program. We reject rank preservation, and the weaker assumption of stochastic increasingness leaves wide bounds on quantile-specific average treatment effects. Neither conventional nor machine-learning approaches to estimating systematic heterogeneity capture more than a small fraction of the variation in impacts given our available candidate moderators.

Original languageEnglish (US)
JournalJournal of Econometrics
DOIs
StateAccepted/In press - 2022

Bibliographical note

Funding Information:
We thank participants at the Heckman 75th Birthday Conference, seminar audiences at Aarhus, CESifo, Copenhagen Business School, and RISE, as well as Natalie Bau, Jishnu Das, Paul Glewwe, Lois Miller, Paul Niehaus, Lant Pritchett and three anonymous referees for helpful comments, Brigham Frandsen for assistance in implementing the Frandsen–Lefgren bounds, and Joseph Cummins for sharing his rank similarity test code. Deborah Amuka, Victoria Brown, and Katie Pollman of Ichuli Institute were indispensable to the data collection for this study. This project would not have been possible without the efforts of Anne Alum, Patrick Engola, Craig Esbeck, Jimmy Mwoci, James Odongo, JB Opeto, and the rest of the Mango Tree Uganda staff, who developed and carried out the NULP intervention. We also thank the students, parents, and teachers from our study schools in northern Uganda. We are grateful for funding from DFID/ESRC Raising Learning Outcomes Grant ES/M004996/2 , Wellspring, and the International Growth Centre. The original data collection for this project is registered with the AEA RCT Registry under registration number AEARCTR-0000021 . The data and code for this paper are available on the Harvard Dataverse as Buhl-Wiggers et al. (2022) . The usual disclaimer applies.

Funding Information:
We thank participants at the Heckman 75th Birthday Conference, seminar audiences at Aarhus, CESifo, Copenhagen Business School, and RISE, as well as Natalie Bau, Jishnu Das, Paul Glewwe, Lois Miller, Paul Niehaus, Lant Pritchett and three anonymous referees for helpful comments, Brigham Frandsen for assistance in implementing the Frandsen–Lefgren bounds, and Joseph Cummins for sharing his rank similarity test code. Deborah Amuka, Victoria Brown, and Katie Pollman of Ichuli Institute were indispensable to the data collection for this study. This project would not have been possible without the efforts of Anne Alum, Patrick Engola, Craig Esbeck, Jimmy Mwoci, James Odongo, JB Opeto, and the rest of the Mango Tree Uganda staff, who developed and carried out the NULP intervention. We also thank the students, parents, and teachers from our study schools in northern Uganda. We are grateful for funding from DFID/ESRC Raising Learning Outcomes Grant ES/M004996/2, Wellspring, and the International Growth Centre. The original data collection for this project is registered with the AEA RCT Registry under registration number AEARCTR-0000021. The data and code for this paper are available on the Harvard Dataverse as Buhl-Wiggers et al. (2022). The usual disclaimer applies.

Publisher Copyright:
© 2022 Elsevier B.V.

Keywords

  • Education programs
  • Machine learning
  • Treatment effect heterogeneity

Fingerprint

Dive into the research topics of 'Some children left behind: Variation in the effects of an educational intervention'. Together they form a unique fingerprint.

Cite this