Useless scientific advice

I lifted the box to the left from the Wall Street Journal, a newspaper that has to be clear, concise and to the point in its communications or its busy and clever readers buy the Financial Times instead. If only the European Commission could choose where it gets its scientific advice, and the scientists involved felt some pressure to be clear, concise and to the point. Alas, I came to read the preliminary report of the European Scientific Committee on Emerging and Newly Identified Health Risks on the subject of smokeless tobacco [PDF]. This will inform European public health policy, and if it was barely competent, it would lead to the lifting of the absurd policy of banning 'oral tobacco' (smokeless tobacco) in the EU outside Sweden. However, despite hundreds of citations and pages of data, the report doggedly conceals, obfuscates and evades the most obvious and important conclusions.

My response to the Committee: Suffice to say, I have been driven to pen a response. This required two components: an on-line response to constrained questions set by the Committee (see here), and a fuller response (see here) covering the broader failings of the work.

Not stating the obvious, focussing on the obscure. Because smokeless tobacco is many times less hazardous than smoking and can substitute for smoking, there are potential large public health gains to be had (or more likely lost, if the stuff continues to be banned). Sweden has the highest rates of smokeless tobacco use and lowest rates of tobacco-related cancer, respiratory and cardiovascular disease. Instead we ban the product and prevent other countries benefiting in this way. You would have expected these insights to form the core of the assessment. I really don't know why they are avoiding this: I can only assume someone involved thinks they are duty-bound to ensure these tobacco products stay banned on the ultra-naive basis that banning something harmful must be progress. But how does distortion and evasion help protect anyone? Apparently, several of the external experts are fed up, perhaps to the point of a walk-out, with the clear bias and manipulation in the drafting of the report's conclusions. The most important part of any scientific assessment is the framing of the issues - my initial memo to the Committee in January 2006 addressed this. As I had expected, it was roundly ignored but I think this now accounts for the problems.

The report actually does quite a good job of surveying the literature, but it is marred by misinterpretation and inappropriate conclusions drawn from the evidence. Glaring and important truths are ignored or sidelined (er, the very low levels of disease in Sweden hardly features) and great effort is expended on trivial detail - others more expert than me will no doubt tear its flawed inferences apart. But I'll highlight three major failings here that I think transcend the tobacco / public health issues:

1. Communicating risk. The report discusses at great length whether the use of smokeless tobacco is hazardous and addictive. It is. Everyone knows it is. But risk is only interesting if quantified in some way and set in context. Bacon is hazardous to health and coffee addictive. What is missing in this report is some sort of spectrum of risk - with common consumer risks at one end (eating meat), medicinal nicotine, smokeless tobacco - in all its various forms, smoking, drinking hemlock etc. That way, we would know how much to worry about a few extra people using smokeless tobacco that would otherwise have remained tobacco-free, compared to how much we might hope to gain if other people used smokeless tobacco instead of smoking. In fact, the risks of smokeless tobacco use vary markedly between products - but this range is compressed into one end of the spectrum that has combustible tobacco products clustered at the other end.

2. Communicating knowledge in conditions of uncertainty. Whether lazy or manipulative, scientists are often very poor at dealing with uncertainty - saying what is known, even if it is not known beyond reasonable doubt. There is a tendency to say 'no evidence' when what they really mean is that there are no randomised controlled trials showing significant results at greater than 95% confidence. But this is just an arbitrary, if widely used, convention in medical literature. In policy work, insights based on the balance of probabilities are often more important and a good scientific assessment will help policy makers through the difficulties of understanding knowledge where there is not high certainty. In trying to find a way of putting this to the Committee I came across the Intergovernmental Panel on Climate Change (IPCC) Guidance notes to lead authors on addressing uncertainties, which I think is an excellent guide and should be required reading for anyone working at the science-policy interface.

3. Burden of proof. Who should be doing the proving and what are the hypotheses? I think there is an in-built bias in so-called evidence-based policy making that favours the status quo. The problem is that high evidential hurdles are set as a pre-condition to justifying doing something new, but the case for carrying on with the current approach may not even be scrutinised and most probably questions never asked. The right way is to assess all the options including staying with the status quo taking a balance of probabilities approach. The ban on smokeless tobacco is an extreme case, but amazingly no-one seems to think it is important to justify the partial ban on smokeless tobacco in the EU - a bizarre intervention, and utterly without precedent, to ban a much less hazardous product variant than the market leader, in this case cigarettes.

One more small reason to despair at the European Union - to me, a completely vital institution in a globalising world. But it does too much of the wrong things, does too many things incompetently that it should do well, and does not do enough of what it really needs to do. A subject I'll be returning to....!