A Lesson about Statistics in Development from David Simon

David Simon, the creator of the greatest television show ever made, The Wire, is interviewed by Bill Moyers in Guernica magazine.  The topics vary, but one particular answer to a question about the relevance of “facts” in understanding the nature of a problem – or the progress and impact of the solution – resonated with me.

One of the themes of The Wire really was that statistics will always lie. Statistics can be made to say anything. You show me anything that depicts institutional progress in America: school test scores, crime stats, arrest reports, anything that a politician can run on, anything that somebody can get a promotion on, and as soon as you invent that statistical category, fifty people in that institution will be at work trying to figure out a way to make it look as if progress is actually occurring when actually no progress is. I mean, our entire economic structure fell behind the idea that these mortgage-backed securities were actually valuable, and they had absolutely no value. They were toxic. And yet they were being traded and being hurled about, because somebody could make some short-term profit. In the same way that a police commissioner or a deputy commissioner can get promoted, and a major can become a colonel, and an assistant school superintendent can become a school superintendent, if they make it look like the kids are learning and that they’re solving crime. That was a front-row seat for me as a reporter, getting to figure out how once they got done with them the crime stats actually didn’t represent anything.

I have not spent long in the world of development, but I have had a chance to understand the mechanics of this world.  I believe that my own project-  one that has taken an innovative market facilitation approach to agriculture economic development – is making the right moves.  But through conversations with career development workers who have been part of the system for a long time and have seen how the sausage is made at the highest levels, I have found that the same principle largely holds true in the world of aid and development.

Typically, the way it works is that governments allot a certain amount of money to be used for development in countries chosen based on the strategic interest of the donor in the country and the broader region.  Aid is often used as a carrot to gain leverage in a country.  For food aid (much is which is necessary in certain cases), developing countries serve as a repository for food surpluses from the donor country.  The other day someone told me that a certain affluent Asian country (not China) is obligated to purchase rice from other countries as part of a WTO agreement.  This country is self-sufficient in rice and is an exporter itself, and the rice farmers in that country would be very angry if imported rice from other countries was brought in and sold to consumers.  So this country buys rice from other countries, as part of its obligation, and then ships it to developing nations, flooding the local market with cheap, low-cost rice.  In doing so, the local rice farmers become uncompetitive serving the domestic market, and are forced to sell at a loss or switch to another crop.  As the cycle continues, these recipient nations become more dependent on foreign food aid, as opposed to moving in the direction of self-sufficiency.  In addition, the local consumers gain a taste for imported rice, which may or may not be readily available or even well-adapted to local growing conditions.  The end result?  Dependency.

The statistics, on the other hand, show a certain number of tons of rice going to developing nations each year, increasing food security for the country and providing its people with a stable source of food.

Or, a project invests money in a sector by purchasing tractors or providing technical assistance to farmers to increase their production.  Another project from another government is doing the same thing.  Both projects are working with the same sets of farmers.  Both projects have obligations to their governments to deliver certain metrics.  For example, the donor government wants to know how many women directly benefited from the project’s activities, and wants to quantify increases in household incomes (10% per year seems to be the norm).  If they don’t deliver those statistics, the project is deemed a failure.  So they provide those statistics, regardless of whether they reflect systemic and sustainable change that will lead to 10% increase in incomes each and every year.  Too often, the statistics tell a story of change and improvement when the systemic barriers that are keeping poor farmers poor remain unaddressed.

These donor governments demand these statistics because they are responsible to politicians to show progress and value for the taxpayer money.  Based on these statistics, the politicians may or may not decide to release additional funds for the projects.  The jobs and livelihoods of countless people within this system depend on these statistics for their daily bread.  In this kind of environment, the focus is on the numbers, while the underlying barriers remain firmly in place.

Statistics that show “verifiable” progress mask the source of the problems – mainly global trade dynamics and domestic agriculture policy.  They allow politicians to show the difference they are making in developing countries, while passing legislation that directly undermines the industries and the people that the statistics show they are helping.  This is called realpolitik“politics or diplomacy based primarily on power and on practical and material factors and considerations, rather than ideological notions or moralistic or ethical premises.” This is the reality of the world in which we live.  A government needs to protect the interests of the people that elected them into office.  Sometimes, those people are large-scale commercial farmers in the Midwest with a surplus crop and no market to sell  (remember ethanol?).  Other times, it is other people.

In closing, this post is not intended to speak ill of development work or development workers.  On the project I am currently working, we are achieving real results and the good work of the people involved has decidedly made the industries in which it is working more competitive.  My concern is around the broader way that progress is captured.  These great examples of success might end up falling through the cracks under the system of quantifying progress.  The Wire showed the fundamental corruptions within the systems that govern the fate of the urban lower-class in America.  When the Baltimore police department needs to show results to the mayor’s office and government in Annapolis, it ramps up the number of “hand-to-hands” – sending in an undercover officer to buy drugs from a low-level dealer.  That low-level dealer, who went into the drug trade because of the limited options he is faced with, is then sent to prison.  Another thug off the streets.  But he’ll be replaced by another, as long as the demand for drugs remains and the source is left free to operate.  Spending six months developing a case to take down a single drug lord, even if successful, shows exactly one bust.  Looking at the statistics, those numbers aren’t great.

So, increasing farmer incomes by a certain percentage is good.  But as long the barriers keeping those farmers poor remain in place, real progress will seldom be made.  Until the politician and the taxpayer (or the broader system as a whole) sees the value in systemic change – something that will never be captured by the numbers – I’m afraid development projects will forever be susceptible to “juking the stats.”

4 thoughts on “A Lesson about Statistics in Development from David Simon

  1. John Modra

    Your insight as a newcomer is pointed and helpful . Most of the interest and motivation in both the participants and observers in precarious economies ( like rural ) comes from a positive set of stats- that can often be misleading .
    If you want to be effective though , an objective view of economic capacity and elasticity is essential for supporting real resilience and growth . So much “stat” stuff only moves what is a wonderful opportunity to help ( western interest in these issues ) from one quickfiz to another . Lets aim to do better by aiming much better at long term stats that need no qualifications !
    An article I published recently on dealing with questions of scarcity and abundance .

  2. Ed Center

    I learned about Cash On Delivery aid from a recent Global Properity Wonkast (an informative, though as the name implies, somewhat nerdy and technical podcast by the Center for Global Development). This only exists in theory right now, and I in theory I love it. The basic premise is that a donor agrees to pay if and when a specific outcome is met by the grantee country. For example, the US agrees to give Ethiopia X million dollars if Ethiopia increases the number of girls educated in a certain amount of time. Ethiopia can decide how to meet this goal. I am infatuated by the idea of greater local control and focus on outputs (more girls pass a high school exit exam), rather than inputs (more teachers are educated, better curricula is developed).

    Yet your post here gives me pause. I wonder if COD aid will only incentivize local governments to figure out creative statistics to prove results while the numbers belie the sustained reality. If the donor country, the local grantee and a third party auditor agree on the output measurement beforehand, can this mitigate statistical storytelling? The Wire at least would seem to say no.

  3. Josh Post author

    Hi John,

    Thanks very much for reading and responding. I appreciate your feedback and insight. I couldn’t access your article though. Do you have another link? I am overall pretty bullish on agric development, especially after being on a pretty forward-thinking project. We are using a new approach called market facilitation (or reasonably new, at least for Ghana), and its focus on the market realities of agriculture supply chains and adherence to non-direct intervention makes its victories more sustainable.


  4. Josh Post author

    Hi Ed – I listen to the wonkcast as well, but, hey, its good bus listening. Yeah – I am afraid that is the case, but I could be wrong. But the thing to remember is that, when you do the right things, the numbers will come. It is just about getting the right numbers. If you focus on meeting a target of educating X number of girls, but disregard the quality and caliber of that education, than you are looking at the wrong statistics. But if you do focus on the quality of education, and couple those improvements with something like conditional cash transfers to incentivize attendance rates, you can effect real, lasting change. So the key is not to disregard the numbers, but rather to focus on the right numbers.

Leave a Reply

Your email address will not be published. Required fields are marked *