Q & A with Esther Duflo of MIT’s Poverty Action Lab

from the International Herald Tribune Blogs

A blogger for the International Herald Tribune had an e-mail Q and A with Esther Duflo of the Massachusetts Institute of Technology Poverty Action Lab. What follows are about half of the Q and A.

Q. What do you think are the worst mistakes made by Western countries in their politics to defeat poverty? Do you think there’s a real will to defeat poverty?

Can you give me some hints about books to read to have a better knowledge of the problem? Thanks for everything, and please continue in your commitment. Our world really needs people like you.

Fabio Di Gaetano
Italy

A. I do think there is a real will to defeat poverty. Not every politician is 100 percent honest about it, for sure. But my sense is that many people really are committed.

The worst mistake we have been making over and over again (and we continue to make) is not to learn from past mistakes, and in fact not to give ourselves the chance to learn properly from experience. Successions of well-meaning academics and policy makers are looking for the magic bullet that will get rid of poverty once and for all. The magic bullet is then tried out and everybody is enthusiastic about it for a little while. Objections start to accumulate, problems creep up, and the magic bullet is abandoned for the next one. In the process, we usually fail to learn the positive aspects of that bullet before discarding it. If we are to make any progress against poverty, we have to stop thinking in these terms; we have to be ready to experiment with new ideas, on a scale modest enough so that we can afford to fail and to admit failure.

You want some readings: I would recommend Understanding Poverty, a collective volume edited by Abhijit Banerjee, Roland Benabou and Dilip Mookherjee, with many essays on poverty by different economists. Then there is Debraj Ray’s Development Economics. It is an undergraduate textbook, but it is written in an extremely engaging way, and it will give you an excellent idea of the issues that poor countries confront, both at a micro and a macro level. To get two radically divergent views of the role aid can play, you can read Jeffrey Sachs’s The End of Poverty and William Easterly’s White Man’s Burden. An intermediate view is presented in Abhijit Banerjee’s Making Aid Work, which describes promise and challenges of aid, and discusses the need for evaluation. It also has rejoinders from people who disagree with him, which will give you a good idea of the debate.

Q. How is World Bank- and International Monetary Fund-backed development economics different from mainstream economics? Are they both governed by the same set of principles to understand the incidence of poverty?

S Lakshma Reddy
India

A. It is interesting that you should phrase the question in this way. The World Bank and IMF are often criticized for being too mainstream, representing the famous “Washington consensus” (fiscal discipline, privatization, trade liberalization, etc…)! In any case, I do not think one could easily describe the World Bank has having “one economics”. The World Bank Group is a very large organization, made of several institutions, with somewhat different missions (Robert Zoellick described their roles on this blog early this year). While it has a common goal to reduce poverty and promote sustainable and inclusive development, different people in different parts of the World Bank Group have somewhat contrasting views of the best way to achieve this goal. I am not exactly sure how you define “mainstream economics”, but some in the World Bank certainly do believe that some version of the mainstream Washington consensus (augmented or not) is the best way to achieve poverty reduction, while others believe that these prescriptions have failed.

The IMF has a somewhat more narrowly defined mission (to help countries prevent and face short-term financial crises) and a more coherent ideology, largely inspired by the Washington consensus. The fact that the world now faces liquidity crises that have become too big for the IMF to handle with its resources is forcing this institution to go through some soul-searching on what its mission should now be. The new chief economist, my colleague Olivier Blanchard, has his work cut out….

Q. As someone who has dedicated your career to discovering and applying effective approaches to alleviate poverty, what do you think are the best ways to deal with the corruption and mismanagement of aid by governments? Have hard-line approaches from the United States and other nations yielded any significant benefits so far, or have such methods hurt everyone except the leaders of the world’s most corrupt governments?

Gary Henscheid
United States

A. This is an excellent question, but I am afraid I don’t know enough to answer it conclusively as yet! The American approach you refer to (and the World Bank’s approach under the past president) has been to make (part of) aid conditional on indicators showing improvements in corruption and governance. In the US, countries become eligible for aid from the Millennium Challenge Corporation when they satisfy a number of criteria, including satisfactory performance on corruption indicators. There are two rationales for this: first, sending aid to more or less well-governed countries should limit wastage of foreign aid, and working with “good” countries in principle makes it possible to give more control to the aid to the countries themselves, and thus reduce micro-management and overheads; second, knowing that large amounts of aid are conditional on “good behavior” should give incentives to countries to improve on these counts.

I think there is potentially a lot of merit in the first argument. I am a little skeptical of the second, however: if the corruption is at the very top, it implies that corrupt governments care enough about their people to want this aid to arrive, even if they cannot grab it because the country is now less corrupt. Somehow that seems somewhat circular. If the corruption is not at the very top, but somewhere in the bureaucracy, it implies that the governments of developing countries know how to control it. The bad news here is that I don’t think we know too much about that yet.

The good news is that there is a lot of exciting research going on, often in collaboration with developing country governments, to understand corruption and to figure out effective ways to fight it. For example, Ben Olken ran an experiment where he compared the effectiveness of audits versus community participation to limit corruption in community road projects in Indonesia. Based on his findings that audits were effective, the Indonesian government has generalized the use of random audits in the project. Along with other researchers at the Abdul Latif Jameel Poverty Action Lab, Ben is now engaged in several collaborative projects with the World Bank and the Indonesian Government to try to understand how to fight corruption in many different areas.

As you point out, it is also important to remember that very corrupt countries are also often also extremely poor. By not sending aid there until the corruption problem has been tackled, we are punishing innocent populations a second time for a leader they generally did not choose. So we still need to figure out some ways of getting aid there, and wasting as little as possible. One option is to focus on very well defined projects (immunizing and deworming children, distributing iron and tetanus shots to pregnant women, etc.) and to provide a lot of oversight. Even then we may waste some money.

But how much money leaks is not the relevant measure of whether a program should be undertaken. What is relevant is how useful each dollar or aid is. Usefulness is the combination of how much money actually reaches its intended beneficiary, and how useful the program is for the beneficiary. So if an intervention is extremely beneficial (which it may be for populations that have absolutely nothing), it may still be worthwhile undertaking it in a very corrupt and poor country even if we think that some of it will be lost along the way. This was my rationale for deciding, for example, to donate money to Myanmar after the floods of last spring, despite the real risk of misappropriation. Of course, this has to be balanced against the risk of inadvertently helping to keep someone in power because the leakage provides funds that help him maintain his base or pay for his army.

Q. How far can conclusions derived from randomized controlled trials be stretched when it comes to policy prescription to tackle poverty? If we find out that a certain intervention is having a positive impact on the fight against poverty, then how appropriate would it be to prioritize the intervention at a macro level and deduce national economic policy based on the result of the intervention? Besides local political and economic institutions, what other factors should we be careful of when scaling up policies which are rigorously and successfully tested at a micro level?

Chandan Sapkota
United States

A. This question, as well as Kartik’s (below), is an excellent question, and both are related. Let me start by reminding everyone what a randomized control trial is, and how they are used to evaluate poverty alleviation intervention. You can find much more information on the site of the Abdul Latif Jameel Poverty Action Lab. In particular, we describe randomized control trials and their rationale in some detail.

Generally, people or communities who benefit from an intervention are not comparable. For example, schools that receive extra textbooks may be either richer or poorer than other schools: they could be richer because only rich schools can afford books, or poorer because an NGO has decided to give textbooks to the poorest schools in the area. This makes it very difficult to evaluate the impact of extra textbooks by comparing schools with and without them. Randomized evaluations follow the lead of medicine: a sample of schools is selected, and half of them are randomly assigned to receive the textbooks (usually, the other half is also given textbooks after the experiment is concluded and the results are out). If the sample is large enough, we can now be sure that the children in schools with and without extra textbooks differ only because of the textbooks. When we compare their scores after a year, we can be sure that any difference is due to textbooks, not to something else. Michael Kremer and colleagues ran exactly such an experiment in Kenya, and found quite surprising results, which you may want to check out….

Now, let me turn to your question. When we run an experiment and we get the results, we know the effect of this program had in this particular place. This is much better than the information we have in general to decide on policy (nothing…), but is it good enough to act on and to move on to recommend a more general policy? There are several obstacles.

First, the results may not replicate across contexts; I discuss that in the next answer.

Second, a program may be implemented in very different ways in a large scale. For example, it may be done well by a non-governmental organization, but corruption problems may creep in when it is implemented by a government. These implementation issues will have to be ironed out. This is important, and scaling up challenges have to be considered, but it does not take away from the finding that we now know what the potential of the program would be if it were correctly implemented. If we find an effective program, this suggests that it is worth investing some effort in figuring out how to correctly implement it on a large scale. This can also be experimented with, by the way: some of the very exciting work in development economics these days is precisely about how to effectively implement programs (see for example Ben Olken’s work, which I discussed in response to another question).

Third, there may be market equilibrium effects. For example, if I find that by randomly offering secondary school scholarship to some kids, I increase their wage, compared to those who did not receive the scholarships, this may not tell me what the effect of doing this nationwide would be: if everybody received a secondary education, the returns to secondary school may go up or down, compared to a situation where few people received a secondary school education. There are two ways to deal with these: in some cases, it may be possible to organize experiments at the “market” level (though I think it would be hard in the example I just described). In others, we have to use a priori economic reasoning to think whether market equilibrium effects are going to be important or not. In many cases, we have no reason to think they would be large enough to undo the effect of the policy.

Abhijit Banerjee and I discuss these issues in a recent article (“The Experimental Approach to Development Economics”), which I have posted on my web page at MIT.

Q. I am currently involved in the type of rigorous evaluation research that you and your colleagues have brought to the forefront of development economics, and I admire your tremendous contributions to poverty alleviation through the use of randomized controlled trials. Nonetheless, some have concerns about the ascendance of RCTs in your field. You and your colleagues suggest that the same interventions should be implemented and evaluated (using RCTs) in a variety of settings to see how they fare in different cultural/political contexts. Others argue that as evidence from different settings accumulates, there will be solid arguments (and evidence) on both sides of a debate (for example, the debate over free vs. positive pricing of bed nets) - thus leaving our policy questions unresolved. What is your response to this concern?

Kartik Akileswaran
Peru

A. The more we experiment, the more information we have. That can never be bad! If it were the case that the pricing of bednets has different impacts on their use in different countries, then it seems to be we would want to know that, no? And it would seem that we would need very rigorous evidence to really know it. If we get several different results in different contexts, it will eventually help us understand what matters.

For example, you refer to Jessica Cohen and Pascaline Dupas’s work on the pricing of bednets: the hypothesis they want to test is whether, when bednets are distributed for free, those who get them are less likely to effectively use them (either because if it is free, everybody gets it, even those who don’t need it, or because getting something for free signals that it has no value). They found that, among pregnant women in Kenya, those who got a bednet free were as likely to use it as those who had to pay for it. When Dani Rodrik posted this on his blog, people objected that this finding might not generalize to non-pregnant women (because all pregnant women really need a bednet, and know it), or to countries other than Kenya, where PSI, a non-governmental organization specializing in social marketing, had already spent many years promoting the net’s value. Well, this is perfect: since the hypotheses are spelt out, we know what experiment we need to run: one with non-pregnant women, and one in a country other than Kenya.

It turns out that Pascaline Dupas did another project with randomly sampled individuals (men as well as women), also in Kenya. The paper is not yet public, but she found essentially the same results. Vivian Hoffman ran a similar experiment in Uganda, and found that free nets were actually more likely to be used by small children than nets that the household paid for. One could still argue that we need to run some experiments outside East Africa, so I don’t mean to say the debate is settled. But we certainly know quite a bit more on this question than before the first Cohen-Dupas study, when we had very little evidence either way on the impact of pricing on the use of bednets.

This is not to say that the findings of experiments always replicate across contexts. And sometimes, the difference in findings across context may leave us surprised. I would argue this is a good thing; it will force us (collectively, as a profession, and as concerned citizens) to apply our minds to the problem, and try to develop a theory for what about the environment should or should not matter for the success of a policy.

In the early days of randomized evaluations, there was more emphasis on evaluating an array of programs, mainly because the research was just starting, and the field was so large. Currently, there are a lot of large-scale efforts going on, running the same experiments (to evaluate the same program) in many different contexts. The organization Innovation for Poverty Action is leading several of these efforts, involving a large number of research teams. We will learn much from these exercises about how much and in what conditions randomized evaluations results generalize. For the few examples where we have a range of study evaluating similar interventions in similar settings, the extent to which results generalize is actually surprisingly high.

Q. What about mental illness in developing countries? The mentally ill are deprived of chances of getting educated because they get sick when they are teenagers or in their twenties. They actually have been left behind for a long, long time. If you defend their human rights, what is the first step to help those people?

According to the essay by Laurie Garrett from Foreign Affairs, some countries lack sufficient health-care workers because they don’t have medical schools. How do you think they can develop the medical infrastructure from nothing there? Some people say medicine and security cannot be outsourced. Do you think foreign doctors and nurses are able to complete the role instead of local medical services givers?

Toni Nathan
Japan

A. There are actually two questions here. I don’t know much about the first one, though I would agree with you that this is an important issue.

One the second one, Laurie Garrett is right that the supply of doctors and nurses is an issue in some countries, but I think the key difficulty in improving medical care in developing countries is to provide incentives for people to become doctors and do a good job as a doctor. Health is an extremely complex issue, and it is extremely difficult for the patients to assess whether or not the doctor is doing a good job. In many cases, we have the wrong idea of what a good job would be. For example, in India many people do not consider that they have been properly treated unless they have received a shot.

To deliver good health care, we need to have a system where doctors have an incentive to work hard even when the final customer cannot judge their effort, or even if they are likely to misjudge their effort. This requires effective regulation of the private sector. In the public sector, it requires monitoring and sanctions for errant doctors and nurses. Those are not in place. A multi-country study shows that personnel in public health centers have astonishingly high rates of absence. And when they are there, they don’t work too hard: A study in Delhi showed that doctors spend usually less than a few minutes examining a patient before making a diagnosis. I don’t know of a magic bullet to solve these problems… in fact the one thing we tried in partnership with a great non-governmental organization, Seva Mandir (monitoring the absence of nurses and communicating that information to the government), failed; after a few months, the nurses’ supervisors colluded with them to ensure that all their absences were reported as “excused”. A project in Uganda had much better results when they involved the community. Much more work is needed on this question.

Link to full article. May expire in future.