People lie in surveys. Which is why India needs a new methodology to gather data
A leaked National Sample Survey Office (NSSO) survey says India’s consumption fell 3.7% between 2011-12 and 2017-18, rural consumption fell 8.8%, and rural food consumption crashed by 10%. If true, poverty must have skyrocketed.
GoI refuses to release the report saying it has serious data problems. Critics declare this is one more example of the government fudging, or killing, inconvenient statistics. However, consumption declines occur only in deep recessions when GDP growth collapses. In this period, Indian GDP grew 7% annually, making it the world’s fastest-growing economy.
Lies, Damned Lies & Surveys
Our national accounts (which give GDP data) say Indian growth in recent years has been boosted mainly by consumption. How could sales taxes — or goods and services tax (GST) — have shot up in this period if consumption had really fallen? How could so many corporations say the rural market was booming for everything from motorcycles to shampoo?
Still, we must reconcile these contrary data. The government must release the flawed NSSO survey and let analysts vivisect it. Are GDP data completely wrong? Or is the consumption survey wrong? The reliability of GDP data is in question, but the survey results look too ridiculous to take seriously.
Economists should learn from the persistent failure of opinion and exit polls to predict election results. Many polls spend large sums on proper statistical sampling to ensure accuracy. And, yet, they end up highly inaccurate. Globally, exit polls are usually accurate. But in India, they go wildly wrong and differ hugely from one another, which should not happen with large samples.
Why? Because people lie to pollsters. One voter told exit pollsters, ‘What will I get out of telling you?’ It is rational for people to tailor their replies to possible personal advantage. This must happen in consumption surveys no less than opinion polls.
Devesh Kapur and fellow researchers give a telling example in a dalit survey in Uttar Pradesh. They hired a local facilitator to assist the survey. Answering questions, one dalit claimed to be badly off. But as the survey proceeded, it came to light that a recent marriage proposal for the dalit’s son came from the facilitator’s cousin. Immediately, the dalit declared he was actually well off and had lied about being poor to ensure he did not lose any freebies.
A gap has grown for decades between consumption as reported by GDP data and NSSO surveys. The ratio of survey to GDP consumption plummeted from 87% in 1972-73 to 43% in 2009-10. It may be just 35% after the latest survey. This cannot be dismissed as under-reporting by the rich alone, as has been past practice. Everybody does it.
Under-reporting is spurred by the targeting of freebies and subsidies to the poor. Back in the 1980s, the food subsidy was the same for everybody. But from the 1990s, the government began providing much cheaper grain (and other benefits) to those below the poverty line. Today, many subsidies of the central and state governments are targeted at the poor, and sometimes to the bottom two-thirds.
Those rising above these income lines will cut their own throats if they tell the truth. The incentive to lie increases with prosperity. If my reasoning is correct, the gap between the two data sets should increase with time. That has, in fact, happened.
Everything But the Truth
The world over, self-reported data — where people tell surveyors about themselves — are suspect. A 2017 study (bit.do/fhLVv) by Liana Fox, Misty L Heggeness, José Pacas and Kathryn Stevens found that in four US states, at least 40% of recipients of food stamps denied getting any. Bruce D Meyer, Wallace K C Mok and James X Sullivan in a 2015 paper, ‘Household Surveys in Crisis’ (bit.do/fhLWJ), found that welfare recipients in surveys declared barely half what they got through food stamps, welfare cash transfers and workers’ compensation. Americans, no less than Indians, under-report.
Economist Trudi Renwick says nonresponse to questions of the US Census Bureau on income rose from 28% in 1997 to 45% in 2015. Non-response was highest among the richest and poorest. Hence, survey data are unreliable for capturing poverty or plutocracy.
In the US, self-reporting consumption surveys are so problematic that economists estimating poverty are seeking alternatives. Some alternatives entail sophisticated modelling of tax, social security and wage data.
India, too, needs a change of methodology. In the 1970s and 1980s, NSSO surveys were used to find the distribution of consumption between different income levels, and that distribution was superimposed on GDP data to estimate different consumption levels as well as the poverty headcount ratio. This adjusted for the dangers of relying on self-reported data alone. However, the government later dropped the adjustment, saying that could distort reality. Alas, unadjusted data can be even worse, and today seem to represent widespread fake reporting.
We should return to the old method combining data from surveys and GDP. That will end the ridiculous gap that has appeared in the two consumption measures. If possible, that can be adjusted using other data sources for under-reporting by the rich or poor.
This will not be ideal. But it will surely be closer to reality than the ridiculous NSSO survey showing falling consumption in the face of rising tax, corporate and GDP data.
