Meanings of Means

Social scientists tend to propose and test hypotheses of the following type:

Hypothesis: As [our explanatory variable] increases, [our outcome variable] also increases, all else equal.

The way that hypotheses of this sort are tested is by compiling a large data set and examining if in a multivariate framework there exists a positive coefficient on the explanatory variable. If so, then the hypothesis is confirmed (well, the null hypothesis is rejected) and all is right with the world.

Those conducting survey or field experiments operate similarly, although there is less concern in those situations about the interaction of other potentially confounding factors as the treatment has been administered randomly to a sample that resembles the non-treated sample (i.e. it is balanced).

This is fine. Obviously there are numerous questions where we are trying to establish the direction and magnitude of causal effects of one factor on an outcome of interest, and this methodology tends to produce compelling evidence in support of these kinds of hypotheses.

What I would like to suggest, however, is that we do not limit the work of social science to the investigation of average treatment effects. Those interested in pushing randomized controlled trials as the only acceptable methodology appear less concerned on this point. A positive mean effect can result from a consistent positive treatment effect across all individuals or a bimodal treatment effect with a larger share of individuals being effected in one direction than in the other (for a ludicrously outsized example, see the Pax in Serenity).

Politics is a complicated domain (as Einstein said) with creative and independent thinking actors that can change over time and respond to similar stimuli in wildly different directions.

The singular focus on means can cause us to ignore the variety of responses inside of the distribution from which the mean emerges. For example, Dean Baker has a great post on his Beat the Press blog about the reality of a very low–but positive–inflation rate.

Expressing concern over deflation (i.e. the inflation rate turning negative) is the way in which people tell you that they have no clue about economics and just repeat what they heard others say. The inflation indexes we use are an aggregation of millions of different price changes. There is a substantial amount of dispersion around the overall inflation number. This means that when the inflation rate is near zero, there are many goods and services whose prices are already falling.

The almost total focus on the mean number ignores that in order to produce a number so close to zero implies a broad set of goods where deflation is already in progress. If deflation is terrifying, then a very low overall inflation rate implies not that deflation is not occurring but that the share of the economy in deflation is relatively large.

Focusing solely on the meaning of the mean effect can lead us astray when the real story is elsewhere.

Posted in Uncategorized | Leave a comment

Bias and Manipulation

I am on the record as having doubts about the validity of some official Chinese economic statistics. Provincial level GDP growth rates jump at moments of political turnover in excess of their electricity consumption and after controlling for other economic changes. I argue that data manipulation is the most likely cause of this discrepancy.

The WSJ has a story today that could fit into a similar narrative. Two purchasing manager indices (PMIs), one official and one unofficial, come to different snap judgments about the current state of the Chinese economy. Unsurprisingly, the official statistic shows growth and the private index shows contraction. Is this another case of data manipulation?

None of the economists asked to interpret the PMIs suggest that data manipulation is going on. However, the indices have known biases that help account for the different takes on the economy. The official state index has a poor track record of capturing trends surrounding the Chinese New Year holiday. Additionally, the purchasing managers that it polls are tilted towards the state sector compared with the real economy and the HSBC and Markit PMI. It is quite possible that state firms are performing better than private firms in the current situation, thus account for the difference.

Biases can certainly be one way to manipulate statistical results. One could imagine only sampling air quality in Yuanmingyuan in Beijing with sampling air in the more polluted center of the city. Conducting a city-wide survey by standing at bus stops or outside of churches on Sunday would yield a biased sample. The question is one of intent. If a bias is present because there is an intent to induce a bias in the data, then manipulation is occurring.

What about the PMIs? Taking snapshots of the Chinese economy is a difficult task. Perfecting seasonal adjustments takes serious efforts, and this is particularly the case for the Chinese New Year which moves around in time every year. Finding the right mix of firms that represent an unbiased sample of economic players is also hard.  To simply assume that the private PMI is better or more honest than the official data because of the possibility of manipulation seems a stretch at this point. If private figures continue to diverge–with the official data consistently painting a rosy picture, then bias likely will have stepped over the line into manipulation.


Posted in Uncategorized | Leave a comment

Urbanization Won’t Drive Chinese Democracy


Alex Lo in the South China Morning Post has a short column with the following explosive title:

Urbanisation will drive democracy

The piece begins:

The greatest driver for democracy in China will not come from its dissidents, overseas subversives or bleeding-heart expatriate busybodies. It will come from the Communist Party’s own urbanisation drive.

There is little reason to believe this is the case. As I’ve described at length (and Lo notes as well), China’s new urbanization plan is hardly anything radical. Indeed, the policies are carefully crafted to reduce the political pressures on the largest and most significant urban centers while expanding growth in smaller cities in the interior of the country.

What mechanism connects individuals living in cities with democracy then in the column? Pure modernization theory:

China reached that crossover point two years ago. Property rights, rule of law and rise of the bourgeois middle class – the basis of Western democracy – became reality for the majority as part of a vast historical process of which urbanisation across Europe was a key part. Beijing has long argued urbanisation will be the engine for economic growth. It must realise it may have even greater political implications.

It is certainly possible that China will democratize. If democratization were to happen, individuals living in China’s cities would be involved. Indeed, large cities are dangerous for dictators. Cities can undermine individual nondemocratic regimes but democracy is not guaranteed just because one nondemocratic regime is ousted. Most dictators are replaced by other dictators.

The Chinese regime continues to manage urbanization to reduce political threats that might emanate from its cities. To simply rely on the idea that modernization–the lumping together of urbanization, education, industrial specialization, etc–will inevitably cause democracy ignores the fact that history continues and did not end in 1989. This urbanization plan has been crafted to try to separate the different pieces of modernization–to urbanize and develop–without leading to pressures to democratize. Urbanization, even China’s managed urbanization, changes the economic and political situation in the country, and such changes might drive China somewhere different from where it is today. However, to assume democracy is the drive’s destination is a mistake.

Posted in Uncategorized | Leave a comment

Today’s China is Communist and Modern, Not High Modernist

The Chinese government released its long-awaited urbanization plan (国家新型城镇化规划) on 16 March. Ian Johnson, who has written extensively about China’s urbanization for the New York Times, begins his piece on the announcement of the plan in grand terms:

China has announced a sweeping plan to manage the flow of rural residents into cities, promising to promote urbanization but also to solve some of the drastic side effects of this great uprooting.

These descriptions of nondemocratic regime’s releasing “sweeping” plans to reshape their economic geography made Jay Ulfelder think of High Modernism, largely from Jim Scott’s Seeing Like a State. Scott describes significant disasters that have emerged out of failed social engineering projects. Ulfelder quotes from a review of Scott’s excellent book by Cass Sunstein:

Scott does not deny that some designs are well-motivated, and he acknowledges that plans can sometimes do a lot of good. He is concerned to show that when a government, with its “thin simplifications” of complicated systems, fails to understand how human beings organize (and disorganize) themselves, its plans are doomed from the start. Scott calls some governments practitioners of “high modernism,” a recipe for many natural and social disasters, including tyranny… Left to itself, this ideology is overconfident but benign. [High modernism] becomes authoritarian when it is conjoined to “an authoritarian state that is willing and able to use the full weight of its coercive power to bring these high-modernist designs into being.” This is especially dangerous when it is linked to “a prostrate civil society that lacks the capacity to resist these plans.” Thus the greatest calamities in Scott’s book involve a weak society that cannot adapt to a government’s plans.

In some ways, then, the summary of the plan in the NYT looks like a classical example of High Modernism. As Ulfelder writes,

China’s sweeping plans for controlled urbanization strike me as high modernism par excellence. This scheme is arguably the twenty-first century version of agricultural collectivization—the kind of “revolution from above” that Stalin promised, only now the goal is to put people into cities instead of farms, and to harness market forces instead of refuting them. ”We are here on the path to modernity,” the thinking seems to go, “and we want to be there. We are a smart and powerful state, so we will meticulously plan this transformation, and then use our might to induce it.”

Such a characterization leads Ulfelder to two predictions.

If Scott is right about these “certain schemes,” though, then two things are liable to happen. First, China’s new plan for managed urbanization will probably fail on its own terms. It will fail because human planners don’t really understand how these processes work, and even if those planners did understand, they still couldn’t control them. This prediction doesn’t imply that China won’t continue to urbanize, or even that city-dwellers’ quality of life won’t continue to improve on average. It just means that those trends will continue in spite of these grand plans instead of because of them. If the American experience in Afghanistan—or, heck, in its own urban centers—is any guide, we should expect many of the housing developments, schools, and transportation infrastructure born of this plan to go underused and eventually to decay. Or, as an economist might put it, the return on investment will probably be poor.

The second prediction of sorts I take from Scott’s book is that the Chinese Communist Party’s plans for socially engineered urbanization will probably produce a lot of conflict and suffering on their way to failure.

I disagree with the assessment of the plan as high modernism and with the causal mechanisms underlying the predictions that arise from it. It isn’t high modernist because China doesn’t “plan” like it used to and the described policies incrementally adjust the status quo. The predictions themselves are not wrong so much as they are already correct.

First, the nature of planning in China has gradually moved away from the intense micro-managing of the eponymous Planned Economy to something much more akin to policies that shape the incentive structure of local governments and individuals by allocating marginal resources more to one locale rather than another. That is, China governs like a modern state, not a high modern one. Even the words used in plans have changed, as pointed out by Philipp C.C. Huang:

If one looks to the evolution in the Chinese terms for planning, we can see that the words have changed first from jihua 计划 and zhilingxing jihua 指令性计划 or “commandist planning” to zhidaoxing jihua 指导性计划 or “guidance planning,” and, more recently, to abandoning the old term jihua completely in favor of guihua 规划, now the commonly used term for what the new National Development and Reform Commission (国家发展和改革委员会), which replaced the old National Planning Commission (国家计划委员会), undertakes.

This semantic change reflects a real reduction in the Party’s control of the day-to-day operations of the economy. This can be seen in the fact that this document is often described as “long-awaited.” It is long-awaited because it was supposed to be announced last year. As Jamil Anderlini of the FT put it,

The urbanisation plan was originally expected to be published more than a year ago, but deep divisions between government departments and dissatisfaction from Li Keqiang, the Chinese premier, who has been a strong champion of the scheme, delayed the plan’s publication until now.

I would argue that this slowly rolled out plans like this one are less likely to be sweeping than those that emerge out of nowhere. Additionally, this dissatisfaction implies that, unlike in China under Mao, local implementation of the plan is unlikely to be anything but grudging. There is a growing literature on local resistance to implementing central dictates in China (e.g., Margaret Pearson and Mei Ciqi have a nice forthcoming paper in China Journal entitled “Killing the Chicken to Scare the Monkey: Sanctions, Shared Beliefs and Local Defiance in China” that I can’t find online).

Second, the document is not a radical departure from prior policy. Johnson’s statement “the plan [is] the country’s first attempt at broadly coordinating one of the greatest migrations in history” fits awkwardly with a history of policies regulating and restricting migration that have existed since 1950s (I might have just finished writing a book about China’s management of urbanization).

The household registration (hukou) system was established when Soviet-style industrialization was initiated to control that true high modernist policy’s unintended consequences, namely blind flows of farmers into cities looking for work and escaping rural taxation. This system of effective migration restrictions has been tinkered with at the national and subnational level countless times during China’s post-Mao Reform Era (1978–). Over the past ten years, such reforms have been constantly trumpeted but implemented reality rarely measures up to the hype of policy announcement. Yet reforms have certainly taken place; Tom Miller’s great China’s Urban Billion summarizes many recent changes well.

The newly released document describes policies that are broadly similar to what we have seen time after time in recent years: continued “strict control” of population growth in the largest cities and encouragement of development of small and medium-sized cities, particularly in the country’s central and western regions. What is different here is a central commitment to assist local government’s fund the infrastructure of their cities and efforts to contain “land urbanization,” where local governments claim rural land from village collectives, pay farmers a pittance, and sell it at a huge profit to developers. The urbanization of land causes the “forced urbanization” of individuals that Ian Johnson’s reporting decries, so attempts to reduce its prevalence going forward should be welcomed.

Why does this plan sound high modernist then? Because it emanates from a Communist Party-led regime that still tends to use language more appropriate to the grand pronouncements of Marxism. It is a Communist state. The regime retains the power to manage the economy and guides it towards in desired directions but in general refrains from stating desired ends.

As for the predictions coming from classifying China as high modernist, the country already is dealing with serious problems of ghost cities where any return on investment is questionable. It is certainly possible that aiding the development of small and medium cities will turn out being wasteful economically, even if it might be savvy politically. In terms of urban instability and violence, I’m sanguine. I see this plan as continuing in a long line of policies that the regime has put forward to try to avoid urban unrest–incorporating slums, expanding access to urban social services, and slowing down land confiscations–are all reasonable levers for the center to use to tamp down the possibilities of protest in cities.

In the end, the Chinese regime speaks with archaic language–that is indeed, occasionally frightening–but acts like a modern state. Today’s CCP leadership certainly prefers to depoliticize and to quantify, to argue that it is pursuing “development,” “progress,” and “modernization” without giving the Chinese people much of a voice to prevent them from doing so. But so do other modern states. China today is far from the catastrophes of its high modern era, namely the Great Leap Forward. Let us all be thankful that this is so.

Posted in Uncategorized | 1 Comment

Seven Deadly Sins — Fresh from the Journals

Phil Schrodt’s influential paper on the “Seven deadly sins of contemporary quantitative political analysis” is now published in the March 2014 issue of the Journal of Peace Research. Here’s the abstract:

A combination of technological change, methodological drift and a certain degree of intellectual sloth, particularly with respect to philosophy of science, has allowed contemporary quantitative political analysis to accumulate a series of dysfunctional habits that have rendered much of contemporary research more or less meaningless. I identify these ‘seven deadly sins’ as: Garbage can models that ignore the effects of collinearity; Pre-scientific explanation in the absence of prediction; Excessive reanalysis of a small number of datasets; Using complex methods without understanding the underlying assumptions; Interpreting frequentist statistics as if they were Bayesian; A linear statistical monoculture that fails to consider alternative structures; Confusing statistical controls and experimental controls. The answer to these problems is not to abandon quantitative approaches, but rather engage in solid, thoughtful, original work driven by an appreciation of both theory and data. The article closes with suggestions for changes in current practice that might serve to ameliorate some of these problems.

Some papers are published and never heard from again; others resound throughout the kingdom before peer review signals its approval. This paper is certainly part of the latter category.

Posted in Uncategorized | Leave a comment

Fresh from the Journals

Kezhou Xiao and Brantly Womack have a new article, entitled “Distortion and Credibility within China’s Internal Information System” at the Journal of Contemporary China. Here’s the abstract:

Behind the problems of credibility of public official information in China lie two patterns of internal information distortion, one restricting the downward flow of sensitive general information and the other filtering the upward flow of local information. Information gathered at the center is increasingly restricted as it is transmitted down the bureaucracy. Meanwhile, the ‘facts on the ground’ are sifted by local official interests at each level of upward transmittal. Awareness of these distortions has been increased by the Internet revolution, but the structures that encourage them remain in place. An empirical survey of different levels of local cadres in Guangdong Province indicates the different perspectives produced by different positions in the internal information system. Municipal level officials, who have more general information but less diverse local information, tend to be more positive about the quality and objectivity of statistics, while their staff members, further from central sources but closer to messy local realities, are more skeptical.

Officials surveyed acknowledge that data manipulation problems exist in China, which is comforting for those of us who have written about their existence.


Posted in Uncategorized | Leave a comment

Nighttime Lights and More

Nighttime lights satellite imagery of China. NOAA/NASA imagery.

Nighttime lights satellite imagery of China. NOAA/NASA.

Joshua Keating at Slate continues his good work popularizing interesting social science research. Last year, he said some nice things about my paper on cities, redistribution, and authoritarian regime survival, writing for Foreign Policy. Today, he writes about the value of satellite imagery of nighttime lights in determining the level of poverty in a country, based on research from Maxim Pinkovskiy and Xavier Sala-i-Martin. The debate is rather technical, but that does not detract from its importance. As the authors write:

How many people are poor, and how fast are they leaving poverty? The World Bank (Chen and Ravallion 2001, 2010) says that a quarter of the people in the developing world lived on an income of less than $1.25 a day (the threshold of absolute poverty defined by the lowest poverty lines in developing countries), down from about 40% in 1992. However, another body of the development literature (to which the authors of this column belong) argues that poverty is much lower (under 5% in 2010), and has declined much faster (Bhalla 2002; Sala-i-Martin 2002, 2004, 2006; Pinkovskiy and Sala-i-Martin 2009 and Sala-i-Martin and Pinkovskiy 2010).

Both literatures use inequality data from household income and consumption surveys to compute income distributions and poverty rates in developing countries, and to conjecture at the shape of the distribution of income for countries and years with no available data. However, from this point, the literature bifurcates. The World Bank anchors estimated country distributions of income to mean income from the surveys they use to compute inequality, whereas Bhalla, Sala-i-Martin and Pinkovskiy rescale the mean of each country distribution to be equal to the GDP per capita for that country in the national accounts. The choice of the mean of the distribution matters empirically because it turns out that, for many developing countries, the survey means are much smaller than the national accounts’ GDP per capita and grow much more slowly; for example, India grows by over 100% between 1994 and 2010 in the national accounts, but only 38% in the surveys. Such differences dwarf any difference in reasonable assumptions about the evolution of inequality in the developing world.

They argue that the nighttime lights data, which is generated via a completely independent process from either the GDP or household income surveys can help in adjudicating this debate. The satellite imagery points to their previous interpretation of low poverty and selecting a mean based on national accounts rather than survey data.

My concern with this analysis is the possibility that the GDP data are manipulated for political purposes. At the subnational level in China, GDP growth figures jump in ways that are not reflected in real economic shifts and that I have described as “juking the stats.” That paper presents some preliminary evidence of systematic differences in national accounts data across regime types as well–namely that dictatorships seem to report higher levels of GDP per capita than democracies that use the same amount of electricity. I’m currently drafting another paper, to be presented at this year’s MPSA conference, that will more seriously investigate this possibility.

Posted in Uncategorized | Leave a comment