Analysing and optimising PAL Network’s ‘My Village’ project

Project overview

Cost-effectiveness and impact analysis of PAL Network’s ‘My Village’ project 

In 2023, the People’s Action for Learning (PAL) Network began implementing a large-scale learning intervention project called ‘My Village’. Taking place in Kenya, Tanzania and Nepal, the ‘My Village’ project aimed to improve foundational literacy and numeracy using a pedagogy based on Pratham’s Teaching at the Right Level (TaRL) approach. The initiative covered 291 villages, 22,323 households, and 39,028 children in total. 

Now, a What Works Hub for Global Education team is aiming to analyse the impact and cost-effectiveness of this ambitious endeavour. To what extent did this initiative further foundational literacy and numeracy skills among children in Kenya, Tanzania, and Nepal? What equity considerations came into play? 

The project will pursue these sub-questions: 

1. What is the impact of the ‘My Village’ project on children’s foundational literacy and numeracy outcomes in each country, and how does this vary by country? 
2. How do the costs of the ‘My Village’ project vary across Kenya, Tanzania, and Nepal? 
3. What is the cost-effectiveness of the ‘My Village’ project in each country? 
4. How do gender and socio-economic disparities affect the impact and cost-effectiveness of the ‘My Village’ project interventions? 

To evaluate the ‘My Village’ project’s impact and cost-effectiveness, the What Works Hub for Global Education research team will leverage a quantitative approach. The team will analyse baseline and endline earning assessment data, as well as cost data based on total expenditures submitted from the implementing countries. 

A/B Testing of PAL Network’s ‘My Village’ project 

This study will build on the other one by piloting A/B testing as a means of optimising and scaling ‘My Village’ and examining how the application of A/B testing methodologies affects the impact and efficiency of the ‘My Village’ interventions.  

The research aims to inform evidence-based decision-making and broader educational policy discussions in each of the ‘My Village’ countries, as well as to inform global debates. Sub-questions include: 

1. How can the insights from rapid iterative testing be operationalised and scaled for evidence-based programming optimisation within the PAL Network? 
2. What specific elements of the ‘My Village’ project interventions show the most significant improvements in outcomes when optimised through A/B testing methodologies in the respective countries? 
3. What are the challenges and successes in operationalising and scaling A/B testing methodologies within the ‘My Village’ project across diverse educational and cultural contexts? 

The A/B testing will follow a 4-step approach of: 

1. Identifying partners, 
2. Initiating a small pilot, 
3. Identifying a ‘golden indicator’ for decision-making, and 
4. Executing the A/B test and beginning to plan the next one.