CIES 2025: A/B testing in education has arrived!

Amanda Beatty, Vishal Sunil, Andrés Parrado, Noam Angrist, Clio Dintilhac, Kate Ross and Yue-Yi Hwa

A/B testing is becoming increasingly prominent in global education as an approach that rigorously evaluates different versions of a programme while adapting to the time pressures and iterative troubleshooting of real-world implementation. This growing visibility was clear at the Comparative and International Education Society (CIES) 2025 Conference in Chicago, where A/B testing featured in seven conference sessions.

One of these sessions, a panel titled ‘Educational technology and implementation science: Using A/B testing to iteratively improve and scale educational interventions‘, brought together presentations from some What Works Hub for Global Education partner organisations that are on the cutting edge of A/B testing. The panellists were:

• Amanda Beatty (Youth Impact)
• Vishal Sunil (Rocket Learning)
• Andrés Parrado (Innovations for Poverty Action)

The panel was chaired by Noam Angrist (What Works Hub for Global Education) and also featured Clio Dintilhac (Gates Foundation) and Kate Ross (FCDO) as discussants.

Here are some snapshots of these organisations’ latest work in A/B testing:

Youth Impact: cutting costs while raising effectiveness

A/B testing has long been part of the DNA of Youth Impact, an evidence-focused organisation delivering programmes for young people in education and health. At the CIES panel, Amanda Beatty made the case that iterative A/B testing can address urgent gaps in implementation and evidence in education in the global south. She then showed how Youth Impact is using A/B testing to address such gaps.

Notably, Youth Impact’s A/B testing work incorporates tests to determine not only whether new versions of a programme can be more effective (‘effectiveness-enhancing tests’), but also whether new, cheaper versions of the programme can be at least as effective as prior versions (‘cost-reducing tests’).

Youth Impact’s cost-reducing A/B tests on their ConnectEd phone-based tutoring programme have led to the implementation of several programme design tweaks that improve cost-effectiveness.

Beyond such technical innovations, Youth Impact has also recently developed a process for supporting other organisations to build their own A/B testing practice.

Rocket Learning: testing across the whole implementation cycle and automating for efficiency

Rocket Learning provides smartphone-based tools and networks for caregivers in India – both parents and staff at early childhood development centres. At the CIES panel, Vishal Sunil shared Rocket Learning’s journey with A/B testing to improve their impact on children’s learning.

One recent change in their approach to A/B testing is a shift from deploying A/B testing to improve user engagement – for example, through personalised psychological nudges – toward using A/B testing to optimise every part of the implementation cycle.

Rocket Learning has started using A/B testing at every step of product development and delivery.

Another recent innovation is that Rocket Learning have developed a platform for automating the deployment of A/B tests – which they intend to make open-source so that it is available to other organisations.

IPA’s Right-Fit Evidence Unit: boosting organisational learning through a shared roadmap

At the Right-Fit Evidence Unit of Innovations for Poverty Action (IPA), A/B testing is used to refine the cost-effectiveness of interventions in both the education and health sectors. At the CIES panel, Andrés Parrado​ spoke about how IPA had used A/B testing in collaborations with two Colombian ed tech platforms, Mentu and Tirando X Colombia.

Crucially, the team has developed a learning roadmap to ensure that A/B testing is used as effectively as possible. The roadmap provides guidance and a common language for aligning A/B testing and programme implementation with theory of change.

IPA’s Right-Fit Evidence Unit has developed a learning roadmap for ensuring that A/B testing takes place in an effective, well-integrated way.

This learning roadmap is centred on a common language for cost-effectiveness – which is then used as both the goal of A/B testing and a framework for identifying programme variations on which to conduct A/B tests.

A/B testing as a tool for uncertain times

During the discussion, panellists raised critical points about ways in which A/B testing can contribute to improving implementation in education.

A/B testing should be viewed as a decision-making tool, rather than an academic exercise. Moreover, it is a dynamic process, not a static question. – Noam Angrist

A/B testing can help to move the needle on some unresolved issues in the global education sector, such as how to deliver impact with fewer resources, as development funding declines. – Clio Dintilhac

One area to be explored and codified in A/B testing for education is the types of new knowledge that can be gained for A/B testing, and how these new forms of knowledge can be most useful to policymakers and practitioners. – Kate Ross

Amid shrinking development aid budgets, on top of other political and fiscal pressures facing governments in the Global South, the potential for A/B testing to facilitate real-time course correction will only become more impactful.

Beatty, A., Sunil, V., Parrado, A., Angrist, N., Dintilhac, C., Ross, K & Hwa, YY. 2025. CIES 2025: A/B testing in education has arrived! What Works Hub for Global Education. Blog. 2025/015. https://doi.org/10.35489/BSG-WhatWorksHubforGlobalEducation-BL_2025/015