What is the real relationship between economic growth and national debt – and should highly indebted countries all adopt austerity budgets that prioritise debt reduction? Kelvyn Jones, Andrew Bell and Ron Johnston have been re-analysing the data set used in the controversial Reinhart-Rogoff paper whose findings appeared to offer evidence-based research support for such austerity budgeting, and find that the world is a more complex place than the previous analyses have suggested. There is no general trend to which all OECD countries conform – and thus, it seems, no rationale for them all adopting a particular economic policy.
The oft-cited paper on ‘Growth in a time of debt’ by Carmen Reinhart and Kenneth Rogoff has twice attracted much comment since its publication. The first set of responses reflected the important substantive and policy implications of their conclusion that across 20 OECD countries for the period 1947-2009:
‘… median growth rates for countries with public debt over roughly 90 percent of GDP are about one percent lower than otherwise; average (mean) growth rates are several percent lower.’
Since ‘Public debt has been soaring in the wake of the recent global financial maelstrom’, journalists, other commentators and policy analysts interpreted this as suggesting the existence of a ‘debt burden cliff’: once the public debt:GDP ratio exceeded 90 per cent, then the rate of growth declined substantially – and probably became negative. Austerity budgets prioritising debt reduction were supported on the basis of this finding.
The paper’s second impact came in spring 2013. A University of Massachusetts-Amherst Political Economy Research Institute graduate student and his supervisors were unable to replicate Reinhart and Rogoff’s findings. After exhaustive re-analyses they identified no evidence of a ‘debt burden cliff’: the average rate of growth declined as the debt:GDP ratio increased, but with no significant change in the relationship at or close to the supposed 90 per cent threshold (or indeed any other). They criticised three elements of Reinhart and Rogoff’s work:
- Their exclusion of data for certain years in three countries, which included several of the key years there when the debt:GDP ratio exceeded 90;
- The (accidental) exclusion of all data for five countries at the top of the alphabet; and
- Some weakly-justified averaging-cum-weighting decisions that substantially affected the outcome.
Correcting for the errors and exclusions, and removing the weighting and averaging procedures, Herndon et al concluded that:
‘… when properly calculated, the average real GDP growth for countries carrying a public-debt-to-GDP ratio of over 90 percent is actually 2.2 percent, not -0.1 percent as published by Reinhart and Rogoff [RR]. That is, contrary to RR, average GDP growth at public debt/GDP ratios over 90 percent is not dramatically different than when debt/GDP ratios are lower.’
This careful (failed) attempt at replication not surprisingly had a very substantial impact on economic commentators because the policy implication was undermined: there was no empirical rationale for austerity budgets to ensure that the debt:GDP ratio stayed below 90 per cent.
Herndon et al.’s paper carried two important messages for social scientists. The first – widely trawled in the media and through social networks – was to be sure your data are correct and any manipulations fully justified. The second – highlighted here – is that ‘… the relationship between public debt and GDP growth varies significantly by time period and country’. They don’t pursue this because their goal was to refute Reinhart and Rogoff’s findings regarding a ‘debt burden cliff’, but if there is no general relationship then there can be no policy implication with wide relevance.
There is a frequent (usually implicit) assumption in many such analyses that there is no significant variation across countries because their economies operate similarly and respond in the same way to an economic stimulus. But if countries do vary in the relationship between their debt burden and growth rate (and if that relationship also varies across time), then simple conclusions (based on a mythical ‘average country’) such as Reinhart and Rogoff’s – known, after Kaldor, as stylised facts – have no policy relevance.
This argument is illustrated by the simple example of two countries in the first graph. The first country (red) displays a positive relationship between debt burden and GDP growth whereas n second (blue) has a negative relationship: greater debt is associated with faster growth in the former, but with slower growth in the latter. If, however, you combine the two to get the average relationship – the dotted green line – the growth rate does not vary with the debt burden, which implies no need for economic policies aimed at changing the debt burden in order to either stimulate or restrain growth.
Is the world like that? Or is it ‘messy’, with different national economies responding differently to changes in their debt burdens, for a probably wide range of reasons? Answering that question demands more than Reinhart and Rogoff’s extremely simple non-parametric analysis based on four groups of country-years according to their debt:GDP ratio (<30, 30-60, 60-90, and 90<). Simple OLS regression is not a sufficient replacement, since it does not allow for cross-country variations. Multi-level modelling – which allows inter-country variations in both the intercept and the slope of the relationship (i.e. variations in each country’s average growth rate and the direction and strength of the relationship between growth and the debt burden) – is necessary to explore the issue fully.
We have been re-analysing the Reinhart-Rogoff data – after Herndon’s ‘cleansing’ and with no weighting of observations – using this technique, and the second graph illustrates our basic findings. (More detailed work is progressing and fuller analyses will be reported in due course.) It shows very substantial variations in the evaluated debt burden:growth relationship (which for these purposes we assume is linear) across the 20 OECD countries between 1947-2009 (there are some missing years early in the sequence for some countries). In some – notably Japan, Italy, and the USA (identified in Figure 2) – there is the ‘expected’ negative relationship: the greater the debt burden the slower the growth, though only in Japan does growth become negative – in none of the others has the debt:GDP ratio exceeded 90. Against that, the relationship in some countries (Norway and New Zealand stand out) has been positive: the greater the debt burden the faster the growth. Finally, some of the regression lines – particularly the UK which (like Australia) has experienced a debt:GDP ratio exceeding 90 – are much flatter: whatever the level of debt, the average rate of growth there has been largely unaffected.
Reinhart and Rogoff promoted a simple conclusion from their analysis: above a certain ratio of national debt to GDP economic growth drops, suddenly and substantially. Reworking their data, Herndon et al. drew a different simple conclusion: there is a negative relationship between debt burden and growth, but no rapid decline when that burden crosses a particular threshold (whether 90 or any other ratio). We have re-analysed Herndon et al.’s data and reached a further simpler conclusion: the relationship between growth and the debt burden varies significantly and substantially across countries. There is no ‘average country’ and therefore no ‘average relationship’ between the two variables from which a policy prescription with general applicability can be drawn. It all depends on the place and the time – geography matters.
Politicians like straightforward policy prescriptions from social scientists, based on incontrovertible research-based evidence expressed as simple, non-controversial statements, like Reinhart and Rogoff’s. Unfortunately, even where the data are excellent, it often turns out that the world is a messy place with key relationships varying both between countries during any one period and within countries at different periods. That is clearly being demonstrated by our in-depth re-analyses of their data set: there was no ‘average OECD country’ with regard to the growth:debt burden relationship between 1947 and 2009 so no universally-applicable general conclusion (or ‘stylised fact’) can be drawn and no consistent policy direction derived.
Which leads to (our) third message: analyse your data fully and you may discover that place and time matter – the world is a complex, complicated set of places, not a single, uniform system. Look at the geography.
Note: This article gives the views of the author, and not the position of the British Politics and Policy blog, nor of the London School of Economics. Please read our comments policy before posting.
Kelvyn Jones is Professor of Quantitative Human Geography at the University of Bristol (since 2001), being Head of the School of Geographical Sciences from 2005-2009.
Andrew Bell is a PhD student at the University of Bristol in Advanced Quantitative Methods.
Ron Johnston is Professor of Geography in the School of Geographical Sciences at the University of Bristol. His academic work has focused on political geography (especially electoral studies), urban geography, and the history of human geography.