When Rabbit asked Winnie the Pooh whether he wanted honey or condensed milk with his bread Pooh excitedly answered, "Both." But then not to seem greedy, he added, "But don't bother about the bread, please."
The debate on Sustainable Development Goals (SDGs), which are to replace the Millennium Development Goals (MDGs) in 2015, is reminiscent of a Winnie-the-Pooh-approach to measuring development. Do you want your poverty measure disaggregated by region or by gender? Both, and don't bother worrying about the measurement please.
Cheryl Doss, Senior Lecturer in African Studies and Economics at Yale University, is one of the stakeholders in this debate who wants more data, and she argues that the questionnaire currently being used to measure poverty should be longer. Her favorite development target would be to collect gender-disaggregated data on the ownership of assets.
She takes issue with the Copenhagen Consensus project on the Sustainable Development Goals. Copenhagen Consensus is currently publishing background papers that evaluates the probable cost-effectiveness of a range of suggested targets, in order to give advice to the stakeholders in the process before there is a final decision on the post-2015 list.
I have contributed a paper measuring the cost of measurement. To be very specific: I have attempted to quantify how much it would cost just to measure whether kids are vaccinated, schooled, fed and have access to navigable roads and drinkable water. I am not addressing how much it would cost to vaccinate, school, feed and to bring them roads and water -- just how much it may cost to monitor the progress in reaching the targets that will be set post-2015.
Let us remind ourselves that the Millennium Development Goals had 8 goals, 18 targets and 48 indicators. The measurement agenda was too ambitious or not taken seriously enough, and as a consequence the Millennium Development Goals had more gaps and missing data than it had reliable observations. In my background paper I tried to first measure how much it would have cost if the monitoring of MDGs indicators would have been adequately funded.
What used to be 8 goals are now 17, the list of targets has ballooned from 18 to 169, with the final list of indicators has not yet been determined. My estimate suggest that just the monitoring such a list would cost about $250 billion over 15 years -- or twice what is spent annually on Official Development Assistance globally. In other words, if the list is adopted as suggested and it was to be accompanied be serious measurement, we would have to set aside all ODA to measurement in the first and the second year and then only in the third year would there be any funds to even begin to vaccinate, feed and school children. It is very likely that success and failure in the post-2015 agenda will be measured with deficient and bad data unless the list of targets is radically shortened.
So while I do not want to pick on Doss specifically, her approach is as close to a perfect example as you can get of the measurement debate. Doss's favorite development target is to collect gender-disaggregated data on the ownership of assets. There is not a clear estimate of what the cost would be of collecting such data. No information is provided as to what would be the benefit of having the data. Finally, I cannot see any specific suggestion for a target like "By 2030 45 percent of all cows in India should be owned by women."
Implicitly these data are supposedly self-evidently valuable, and so the development impact does not have to be demonstrated. My question is not whether gender matters in the division in the ownership of assets. It does. My question is how important and plausible is it that we establish it as a global goal for development?
I propose two ground rules for debating SDGs. First, for every proposed target: tell us how much it would cost to measure, and show us who is going to pay for it. Second, for every new target proposed, suggest two other targets that should be deleted. In the world of no trade-offs it is perfectly fine to say -- both please -- but in the real world you actually have to choose.
I do think that it is important to know something about the cost of measuring things, and about the trade-offs of having say, statistics on gender distribution of land versus annual employment statistics. It is about time we realized that official statistics is a public good -- if we demand too much to be publicly provided then the quality of our information will deteriorate.
I think Doss knows this too. First she suggests that it would be very easy to measure "Adding one simple question, "Who owns the parcel?" provides information on whether the land is owned individually by a man or woman, or jointly by the couple." But then she writes that you need to prod further to ask who has the legal title, and then in the next paragraph: "To be sure, understanding all of the nuances of asset ownership would require more than one or two questions in a household survey. It requires understanding of the legal frameworks and how the laws are implemented..."
This is precisely why I advised against it. Not everything that counts can be counted and not all issues have to be surveyed as part and parcel of the public information system. Doss suggests adding one, two or three more questions to a survey... but so will many other scholars and NGOs out there. Every single one of them can claim that the marginal cost of one more data query is small, but someone has to pick up the total bill.
We really have to make a choice between 'measure everything wrong all the time' or 'measure something correctly some of the time'. Taking the value of data for development seriously means being aware of its costs, its opportunity costs and gauging its trade-offs.