Research Brief
A Meta-Analysis of the Literature on Aid and Growth

In 2008 Doucouliagos and Paldam published a paper, hereafter known as DP08, based on a meta-analytic approach to the aid-growth question.  Working with a database including 68 studies on the relationship between foreign aid and economic growth they arrived at a pessimistic result. This is in line with much of the fourth generation literature identified by Arndt et al (2010) in the WIDER working paper ‘Aid and Growth: Have we come Full Circle?‘. In particular, DP08 conclude that the literature on aid and growth has thus far failed to show any significant link between aid and growth, and they attribute the variation in the reported effects amongst the papers to different study characteristics.

In the WIDER Working Paper ‘Aid and Growth: What Meta-Analysis Reveals’, Mekasha and Tarp re-examine these results, and argue that when the analysis is expanded to better reflect econometric, statistical, and data challenges, the results show that the effect of aid on growth is both positive and statistically significant, and that this is not a result of publication bias.

Why reassess?

Mekasha and Tarp’s reassessment of DP08 is motivated by three underlying concerns regarding the econometric model, statistical choices and issues with the data of DP08. DP08 begins from an assumption that there exists, a single ‘true’ effect of aid which is common to all the 68 studies. This implies that any variation in the results of studies is due solely to sampling error. Mekasha and Tarp, using both statistical tests and graphical tools, show how this assumption of a single true effect is inappropriate in a meta-analysis of aid and growth literature. Moreover, they indicate that this assumption can also be rejected on theoretical grounds as the effect of aid on growth is a function of a large number of factors. Consequently a real variation in results is to be expected, and thus must be accommodated by the model.

They go on to point out that for papers that include interaction terms (aid interacted with policy, institutions or itself) to capture the non-linear effect of aid on growth, the effect of aid will be mis-measured if one ignores the coefficient of the nonlinear term(s).This issue is ignored by DP08. Mekasha and Tarp address the issue by estimating the weighted average for papers that include such variables separately from those that exclude them. Moreover, DP08 makes the assumption that studies with a larger sample size are inherently more accurate, and on the basis of this they use sample size as the key determinant of the relative weight individual studies are given. Mekasha and Tarp argue that the precision of an effect estimate cannot be fully captured by sample size and point out that using sample size as weight is not in line with best practice, which suggests using inverse-of variance of the estimates.

Furthermore, in the process of reassessing DP08, Mekasha and Tarp re-entered all the data from the 68 studies. In doing so they were able to increase the number of observations from 471 to 519 by entering the data that was originally wrongly coded as missing. These changes, while important, are not enough to render a comparison with original study invalid as the two data sets are still highly correlated.

Publication Bias

Every meta-analytic study has to take into account the potential for results to be skewed by publication bias, otherwise known as the file drawer problem. This problem arises if researchers, editors and reviewers favor statistically significant findings and as a result studies which find only small and/or not statistically significant results sit unpublished in a file drawer. It is clear that in the aid-growth literature at least some studies which yield small or insignificant results have nonetheless received considerable attention from both academics and policy makers, Mosley’s 1983 micro-macro paradox and Rajan and Subramanian’s 2008 ‘aid is insignificant’ findings being good examples.

If publication bias exists, it would tend to bias empirical effects and as such need to be investigated with the aim of disentangling any genuine empirical effect from publication bias. Thus, to examine this Mekasha and Tarp run regression based tests and show that publication bias is not a problem in the aid growth literature once the heterogeneity between the 68 studies is controlled for. Moreover, the regression based tests give a clear evidence of the presence of a positive and statistically significant effect of aid on growth that goes beyond publication bias.

Results

Having reworked the DP08 model to address the issues described above Mekasha and Tarp are able to draw conclusions which differ from those presented in the original study:

• The link between aid and growth is both positive and significant with a magnitude of 0.14.
• Publication bias is not a severe problem in the aid growth literature.

Mekasha and Tarp finish by suggesting that while these findings are significant, they do not present the full story about the relationship between aid and growth. Economic growth is just one of many aims of development aid, and for many foreign aid programmes it is poverty reduction that is the key target. Furthermore, they argue that these results should not lead to an uncritical view of aid, rather they agree with calls for improvements in the design, and implementation, of foreign aid programmes that could lead to greater aid effectiveness, and ultimately benefit the poorest people in the poorest countries.