Junk science and booze tax - a study in spin
Where's the beer?
Putting the price of alcohol up to a minimum of 40p a unit would keep 41,000 people a year out of hospital, save the NHS £116m a year, and avoid 12,400 cases of unemployment, a report from Sheffield University claimed last week. These appear to be remarkably precise predictions. The government used the report - widely quoted in the press - to justify higher duties and greater regulation of the sale of alcohol. Yet on close examination, the report appears to be a prime example of "policy-based evidence making".
The blockbuster report, from Sheffield University's Section of Public Health, is in two major parts: a review of evidence, and a statistical model, totalling over 500 pages. Researchers examined the effects of alcohol pricing and alcohol promotion (and advertising) on three areas: consumption, public health and crime. I won't cover the latter, because these proposals were dropped before the Queen's Speech, but it is evident from the amount of time the Sheffield researchers devoted to this, that this was a legislative priority. Academia marches in lockstep with its financial benefactor - in this case, of course, the Department of Health.
The first thing you'll notice is that for "evidence", there is no original fieldwork in this Department-commissioned report. It relies on other studies, and leans particularly heavily on what are called "meta studies" - statistical confections of existing, earlier fieldwork. In other words, it's a "meta-meta study".
Undeterred, the authors remind us in the preamble that: "It is imperative that such policy decisions are based on a sound evaluation of existing evidence." And at first sight, the list of studies included certainly looks impressive. 65 earlier reports are are examined for pricing and taxation, and 33 metastudies are examined pertaining to health consequences of alcohol.
The second thing that leaps out is that much of the evidence is actually quite old. Estimates of price elasticity for the UK are taken from studies published in 1989 to 1990, for example. This was a different era, when the population demographic was different, the demographic spread of wealth was different, and the choices of entertainment available were different - and differently priced, too.
Thirdly, very little of the evidence is drawn from the UK - or even Europe. That whittles the long list (65 and 33) down considerably. We'll see how important this cultural factor is in a moment, but keep in mind that cultural expectations and laws vary considerably from state to state in the USA, let alone between countries.
So what actually pertains to the UK? For taxation effects, the researchers admit that "none of the studies were committed [sic] in the UK". For pricing effects, only four of the older studies studied were UK-specific. And again, these are very old studies indeed. The bulk of the evidence on beer pricing and young people comes from a 1993 study (Laixuthai and Chaloupka), followed up in 1996.
Quite incredibly, the health findings relevant to the UK are even patchier. In one of the key metastudies examined, only three were from the UK. But wait. Of the the three, "the youngest age in the studies was 35 (Britton and Marmott, 2004), and only one included both women and men". Er, what?
For such apparently precise modelling, it seems little of the evidence base reflects modern Britain. Yet the cultural issue is so great that the researchers cannot fail to acknowledge its significance.
"A major limitation of the evidence base is that most studies examining the impact of such policies have been conducted in the United States," they admit sheepishly. And models have been shown to be seriously defective before.
In 1999, the case for punitive alcohol taxes received a serious blow. Thomas Dee, then an economics professor at Georgia Institute of Technology looked at some assumptions.