Join the network
Policy Brief
Experimental and non-experimental methods to study government performance – contributions and limits
In recent years, field experiments using randomized trials have gained increasing popularity in the field of development economics. In particular, scholars have argued strongly for their use as the best means of identifying ‘what works’ in foreign aid. Field experiments have also become increasingly popular in political science, particularly in the study of democracy and elections.
The project, that is part of the of the Research and Communication on Foreign Aid (ReCom) programme, brings together economists and political scientists to investigate the use of field experiments in the study of governance, and their limitations with regards to ‘what works’, ‘what is scalable’, and ‘what is transferable’ in foreign aid. The project focuses on interventions aimed to enhance government performance in the areas of public service delivery, and contributes to the literature in three important ways:
First, the project speaks to one of the core questions for development practitioners today – how donors can best help developing countries to improve their governance – and systematically explores the insights that can be gained on this question from randomised experimental design. While experimental methods have been increasingly used in the study of other areas of development assistance over the last decade, field experiments have covered only a limited range of issues related to governance. The project will build on the literature to provide an overview of key issues, findings, and challenges relevant to future work on governance.
Second, the project speaks to one of the core criticisms of experimental work in political science: that it cannot and does not address ‘big’ questions and ‘big’ theories, much less help us to evaluate the national-level policy interventions that these theories suggest. The project explores to what extent such criticism is valid and whether in fact experiments can be designed to address some of the ‘big’ theories about governance.
Third, in focusing on ‘what is scalable’ and ‘what is transferable’ in foreign aid, the project directly studies one of the core criticisms of experimental work more generally: that its findings are not externally valid and tell us little about ‘what works’ beyond the narrow context of the experiment.
Furthermore, field experiments are contested by their short-term window of analysis, a fact that strongly limits the examination of social phenomena that occur in the course of years or even generations. Many scholars criticize experimental work, along these lines, but few studies in the social sciences have empirically investigated these criticisms. Project contributors employ use new and existing data to analyse precisely (1) what is gained and lost through the use of randomized controlled trials versus other approaches (e.g., quasi-experimental designs, survey techniques) and (2) the possibilities of employing new techniques in the design and analysis of randomized experimental trials that can improve their ability to speak beyond the narrow context of the experiment.
Context
Main subject
This project is part of Governance and fragility and ReCom - research and communication on foreign aid
Theme: Past, 2012-13