Cheaper (and more effective) by the dozen: Evidence from 12 randomised A/B tests optimising tutoring for scale

Noam Angrist, Claire Cullen and Janica Magat

Abstract

Over the course of 12 rapid randomised experiments, we optimise an educational tutoring programme. Tutoring is one of the most effective educational approaches yet has remained difficult to scale due to high costs. We adaptively test and improve a technology-enabled tutoring program to enhance cost-effectiveness and scalability. Results show that seven of twelve tests led to efficiency improvements, a ‘rate of discovery’ of 58%. This compares favorably to the tech sector where 10-40 percent of tests generate improvements, demonstrating the potential for A/B testing to yield large efficiency gains in the education sector. The largest efficiency gains were driven by cost-reducing modifications that streamlined labor-intensive implementation processes and effectiveness-enhancing innovations that actively involved caregivers in their child’s education, more than doubling impact at minimal additional cost. We explicitly measure practitioner prior and posterior beliefs, and find that rigorous testing facilitates more accurate identification of ‘what works’. Our findings both reveal the returns to iterative testing in social programmes and contribute new evidence on simple, cost-effective strategies to improve learning outcomes.

Angrist, N., Cullen, C. & Magat, J. 2025. Cheaper (and more effective) by the dozen: Evidence from 12 randomised A/B tests optimising tutoring for scale. What Works Hub for Global Education Working Paper Series. 2025/001. https://doi.org/10.35489/BSG-WhatWorksHubforGlobalEducation-WP_2025/001