Research Brief
Evaluations

The growth of social protection programmes aimed at reducing poverty by transferring resources to the poor has been a novel feature of development policy and practice in the last decade. Combined with other policies, transfer programmes have the capacity to make a significant contribution to global reduction of poverty and vulnerability.

Rigorous impact evaluations that produce ‘hard’ evidence have been important for the spread of social protection programmes. They have played a role in the shift towards evidence-based development policy in developing countries. Government agencies responsible for antipoverty efforts have a duty to ensure programmes are effective, based on existing knowledge, and have established processes to learn lessons from their implementation. Impact evaluation studies also play a political role. Well-designed studies can provide crucial arguments to those advocating anti-poverty transfer programmes, and help to overcome the objections of those opposing them.

The importance of impact evaluations for the successful spread of anti-poverty policies is made clear by the contrasting examples of Latin-America and sub-Saharan Africa. In Latin America, the rapid spread of human development conditional transfer programmes has been facilitated by the paradigmatic case of the rigorous impactevaluation protocols implemented in Mexico’s Progresa/Oportunidades. In sub-Saharan Africa, on the other hand, the spread of anti-poverty transfer programmes has been slowed down, arguably, by the lack of strong monitoring and evaluation processes.

Evaluations of anti-poverty transfers in Latin America

In Latin America the major example of a transfer programme being evaluated effectively comes in the form of Mexico’s Progresa/Oportunidades. Rigorous evaluations carried out by a number of agencies—after the programme’s introduction in 1997—found that it had positive effects on participant households. These results allowed proponents of the scheme to overcome political opposition from competing agencies, and to encourage a nationwide adoption. Especially in the aftermath of the 1995 financial crisis, the implementation of Progresa/Oportunidades meant that the programme needed to be implemented in substitution of other schemes. The positive impact evaluation provided the technical and political justifi cation for the expansion of the programme. The findings from the evaluation of Progresa also infl uenced the efforts to expand social protection in other Latin American countries.

Not all transfer schemes in Latin America have been subject to equally rigorous evaluations. When Brazil began introducing transfer programmes such as Bolsa Escola, they did not face strong opposition within the national government. The lack of political opposition contributed to the less emphasis being placed on setting up evaluations of the different transfer programmes which eventually became Bolsa Familia. Similarly, across Latin America, transfer programmes which target the elderly and disabled, and therefore do not have implications for work incentives, have faced less opposition and received less support from donors, and have consequently been subject to less evaluation. The experience of Latin America seems to shows that social protection programmes are subject to more evaluation when they face opposition from political constituencies, competition for resources with established programmes, the presence of donor agencies, and discretion over their implementation.

Pilots in sub-Saharan Africa

In contrast to the experience from Latin America, social protection programmes in sub-Saharan Africa have developed at a much slower pace. There are several explanations for this. In particular, the incidence of poverty in the region is significantly greater than in Latin America, and the  available resources are more limited.

Consequently, while donors in sub-Saharan Africa are often enthusiastic about transfer programmes, domestic finance ministries are often reluctant, and rightly so as they know that while donors may help to cover the start-up costs of a transfer programme they are unlikely to provide the necessary funds to make the programmes sustainable in the longer term.

This standoff between donors and domestic elites should have, in principle, been resolved through the implementation of pilot projects. Pilot projects can help provide information on whether transfer programmes have been designed and implemented correctly, and also provide necessary learning and training for the relevant agencies. However, in practice pilots have seldom  incorporated strong evaluation procedures.

This can be explained by competition amongst donors which often focus on pilot projects as a tool to engage and persuade domestic political elites, rather than as a means to assess the effectiveness of this type of activities in the different contexts.

Overall, the evaluation of social protection programmes in sub-Saharan Africa has been much weaker and less successful than in Latin America, despite the fact that the political and policy environment are such that evaluation could have played an important role. Monitoring and evaluation systems can be crucial in facilitating improvements in government effectiveness—however, the extent to which this is possible is strongly determined by the political conditions within the recipient countries.