Confirmation Bias

Apologies for the light posting lately. I was in Florida and mostly internet-less for the past week. I have posts on financial reform and income inequality in the works but first I want to point an example of how statistical analysis can be used, abused, and simply be murky. Reinhart and Rogoff have recently released a paper (ungated pdf) showing a link between high debt levels and a slowdown in economic growth:

[I]t is evident that there is no obvious link between debt and growth until public debt reaches a threshold of 90 percent. The observations with debt to GDP over 90 percent have median growth roughly 1 percent lower than the lower debt burden groups and mean levels of growth almost 4 percent lower. ...

Interestingly, introducing the longer time series yields remarkably similar conclusions. Over the past two centuries, debt in excess of 90 percent has typically been associated with mean growth of 1.7 percent versus 3.7 percent when debt is low (under 30 percent of GDP), and compared with growth rates of over 3 percent for the two middle categories (debt between 30 and 90 percent of GDP).

This is for advanced economies; the effect is stronger for emerging economies. The paper does not utilize any fancy statistical techniques, but merely reports charts and tables of average growth rates at different debt levels for different countries. The association between high debt levels and lower growth rates is pretty consistent across countries, with a few exceptions (about which more in a minute). Obviously this is bad news for stimulus proponents who want to ramp up spending. The U.S. debt is now approaching the 90% of GDP threshold, and this paper could be (and has been) used by stimulus opponents as evidence in favor of austerity. This, in turn, has led stimulus proponents to attack it pretty severely. Here's Krugman, arguing that the R-R data don't count:

I think we can say that this paper has been completely discredited. I’m actually sort of shocked that R-R apparently failed even to notice that all of their high-debt observations for the US — and remember, it was their own choice to highlight US data — come from the years immediately after World War II, and to think about what that means.

There's one truth here, one partial truth, and one falsehood. It's (mostly) true that all of the high-debt observations for the U.S. come immediately after WWII -- though 2008 and 2009 come close to the 90% threshold, and 1944 and 1945 are not postwar years and do reach the threshold -- and it's true that R-R don't explicitly discuss this fact in any detail. (Although it's telling that Krugman has no such qualms about the Romer and Romer paper that finds a large fiscal multiplier by examining the WWII years.) It's partially true that R-R highlight U.S. data, but they also discuss (and disclose) data from 20 other advanced economies and a number of emerging economies in the paper as well. These cases also support the pattern. Without actually saying it, Krugman gives the impression that they ignore the rest of the world. It is untrue that the fact that the high-debt U.S. years follows WWII discredits the paper, and Krugman has to have willfully misread the paper (or not read it at all) in order to make that conclusion.

First, R-R open the paper by citing a previous study showing that debt increases are generally a consequence of financial crisis, not a cause. Indeed, this is one of the great takeaways from their book, along with the observation that debt crises usually follow financial crises. In this light, their newer article can be read as a claim that both low growth and high debt follow financial crises like the one we've recently experienced. This is probably the reading R-R prefer, since they explicitly claim that this is an empirical, not a theoretical, paper. They even say in the paper, following a discussion of the differences between war-debt and peace-debt, "Here we will not attempt to discriminate the genesis of debt buildups, and instead simply look at their connection to average and median growth and inflation outcomes."

Second, just because data has a context does not mean that correlations are "discredited". R-R can't help that the data are what they are. Maybe there are confounding circumstances that render the correlations spurious; maybe not. R-R acknowledge that more study is needed on the question. But the fact that this pattern persists across two centuries of data in 44 countries indicates that the correlation is at least fairly robust. Krugman is the one ignoring the cross-sectional comparison here, not R-R, when he only looks at the U.S. years and throws them out because they are special cases. In fact, the exceptions to the association R-R find run the opposite direction of what Krugman alleges:

Of course, there is considerable variation across the countries, with some countries such as Australia and New Zealand experiencing no growth deterioration at very high debt levels. It is noteworthy, however, that those high-growth high-debt observations are clustered in the years following World War II.

In other words, the R-R paper is not discredited. In one sense it can't be, because they offer no causal theory for why the association they observe exists, so there's nothing to discredit. They merely point out the correlation, and offer possibilities for further research. They do not do any time series analysis, which is a shame because it makes it impossible to tell whether the low growth followed from the high debt, or whether the high debt followed from the low growth. If I was Krugman, that's the weakness of the study that I'd hammer on.

But to do that he'd first have to acknowledge that the association exists, which would mean that high debt levels are something to be avoided (except during war), which would weaken the case for massive fiscal stimulus when the debt-to-GDP ratio is already approaching 90%. He doesn't want to do that, so instead he has to attack the credibility of two very good empirical economists. In the end R-R aren't the ones being discredited.