Iterative A/B Testing Toolkit

Organisations need fast, affordable ways to optimise programme cost-effectiveness on the path to scale. Iterative A/B testing is emerging as a practical, low-cost method for social and public sector organisations to learn, adapt and improve rigorously.

Youth Impact has been running A/B tests for nearly a decade, and this toolkit codifies what they have learned: it explains how to get started, outlines best practices, and shares tools and tips developed through years of field-based testing.

The primary audience for this toolkit is anyone in the social or public sectors – NGOs, philanthropists, bilaterals, multilaterals, and regional or national governments – looking to integrate A/B testing into their M&E systems. These tools and guidance can be adapted to a user’s specific context. While Youth Impact’s experience centres on optimising youth education programs, the toolkit applies broadly to any organisation seeking to develop iterative learning systems.

References

Angrist, N., Beatty, A., Cullen, C. & Matsheng, M. (2024). A/B testing in education: Rapid experimentation to optimise programme cost-effectiveness. What Works Hub for Global Education. https://doi.org/10.35489/BSG-WhatWorksHubforGlobalEducation-RI_2024/001 wwhge.org 

Angrist, N., Bergman, P. & Matsheng, M. (2022). Experimental evidence on learning using low-tech when school is out. Nature Human Behaviour, 6(7), 941–950. https://doi.org/10.1038/s41562-022-01381-z Nature 

Angrist, N., Cullen, C., Ainomugisha, M., Bathena, S. P., Bergman, P., Crossley, C., Letsomo, T., Matsheng, M., Panti, R. M., Sabarwal, S. & Sullivan, T. (2023). Learning curve: Progress in the replication crisis. AEA Papers and Proceedings, 113, 482–488. https://doi.org/10.1257/pandp.20231009 American Economic Association 

Angrist, N., Cullen, C. & Magat, J. (2025). Cheaper (and more effective) by the dozen: Evidence from 12 randomised A/B tests optimising tutoring for scale. What Works Hub for Global Education. https://doi.org/10.35489/BSG-WhatWorksHubforGlobalEducation-WP_2025/001 wwhge.org 

Angrist, N., Matshaba, M., Gabaitiri, L. & Anabwani, G. (2019). Revealing a safer sex option to reduce HIV risk: A cluster-randomized trial in Botswana. BMC Public Health, 19, 610. https://doi.org/10.1186/s12889-019-6844-8 Springer Link 

Dupas, P. (2011). Do teenagers respond to HIV risk information? Evidence from a field experiment in Kenya. American Economic Journal: Applied Economics, 3(1), 1–34. https://doi.org/10.1257/app.3.1.1 American Economic Association 

List, J. A. (2023). The voltage effect. Business Economics, 58(1), 3–8. https://doi.org/10.1057/s11369-022-00298-9 

Bhula, R., Mahoney, M. & Murphy, K. (n.d.). Conducting cost-effectiveness analysis (CEA). Abdul Latif Jameel Poverty Action Lab (J-PAL). https://www.povertyactionlab.org/resource/conducting-cost-effectiveness-analysis-cea?utm_source=chatgpt.com  

Doyle, M.-A. & Feeney, L. (n.d.). Quick guide to power calculations. Abdul Latif Jameel Poverty Action Lab (J-PAL). https://www.povertyactionlab.org/resource/quick-guide-power-calculations?utm_source=chatgpt.com  

Evidence in Governance and Politics (EGAP). (n.d.). EGAP App: Power Calculator. https://egap.org/resource/egap-app-power-calculator/?utm_source=chatgpt.com  

Gupta, S., & Kopper, S. (2021). Power calculations. Abdul Latif Jameel Poverty Action Lab (J-PAL). https://www.povertyactionlab.org/resource/power-calculations?utm_source=chatgpt.com  

Abdul Latif Jameel Poverty Action Lab (J-PAL). (2025). J-PAL costing guidelines.  https://www.povertyactionlab.org/sites/default/files/research-resources/costing-guidelines_5.27.25.pdf?utm_source=chatgpt.com  

Abdul Latif Jameel Poverty Action Lab (J-PAL). (n.d.). Randomization. https://www.povertyactionlab.org/resource/randomization?utm_source=chatgpt.com  

Beggs, C. H., Dintilhac, C. & Kaffenberger, M. (2024). The need for more and better cost-effectiveness data to inform government delivery of foundational learning programmes. What Works Hub for Global Education. https://doi.org/10.35489/BSG-WhatWorksHubforGlobalEducation-BL_2024/003 

Angrist, N., Evans, D. K., Filmer, D., Glennerster, R., Rogers, H. & Sabarwal, S. (2023). How to improve education outcomes most efficiently? What Works Hub for Global Education. https://doi.org/10.35489/BSG-WhatWorksHubforGlobalEducation-WP_2023/002 

IDinsight. (2022). Lesson 4: Cost-effectiveness analysis. IDinsight Technical Bootcamp. https://www.idinsight.org/topics/lesson-4-cost-effectiveness-analysis-3/?utm_source=chatgpt.com