Monday, April 09, 2007

What was the question again? ...Green polling examined

There is emerging conventional wisdom that people are concerned about the environment as never before, but are unwilling or unable to do much about it - for example, from last week's Independent... Britons unwilling to change despite climate change. And that's not unusual... but how realistic is it?

First of all, environmental concern isn't at a record high in Britain... Ipsos/Mori's tracker shows a recent spike in concern (upper chart), but this is in fact lower than the peaks in the late 1980s, which I assume were driven by the iconic images of the Antarctic ozone hole. Concern today is still well below defence, crime, the NHS and immigration. My question is whether green polling tells us much about anything. If we look at some actual polls, there are real weaknesses...

I reproduce the lower chart from the Energy Saving Trust 'Green Barometer' [release / report], which is to become a quarterly tracker of environmental attitudes and behaviours (click to enlarge). This tests concern (high), intentions (good), actions taken (mixed) and probes trade-offs, asking what "activities we’re willing to take for the sake of the environment?".

Behind the responses - the personal 'demand curve' for carbon
I think these questions provide arbitrary and difficult-to-interpret perspectives on a fundamentally far more interesting construct - namely people's underlying 'demand curve' for carbon for different purposes.

The binary yes/no framing of the question loses the subtlety in real trade-offs that people would make. For example, in their reluctance to give up plasma TVs, do people think it is a high value use of energy but doesn't use that much, compared to say driving a big car? Is flying off for foreign holidays a premium use of carbon - that people are prepared to pay a lot for? Do people accept that walking when able to do so is a fair deal, but just don't think they are able to walk that much? How much more expensive would it need to be? Surely to get to the bottom of these attitudes, people need to know the value of a sacrifice to them in term of what it means to the environment, and also to be able to express their preferences for sacrifice in a non-binary way, for example by expressing a willingness to pay as a way of expressing how important it is to them. The underlying idea is that people will pay more or less for emitting carbon depending on what it is used for and what options they have to avoid it.

Asking non-sequiturs
It's perhaps in the nature of polling, but the questions in green polls are usually highly unsatisfactory...

  • The Green Barometer finds that people rate 'reducing the amount of energy used in the home' on a par with 'charitable giving' in a Good Deeds Index... yet much home energy saving more than pays for itself and is both selfish and altruistic. Other surveys ask some odd questions:
  • ICM for Retail Week: Would you be willing to pay a lot more for goods and services that were more environmentally-friendly...? Most people say 'a bit more', indicating caution about the vagueness and ambiguity of the proposition... What would the environmental benefit be? Would everyone have to participate? Would not paying mean not having the goods and services.
  • ICM for BBC: How much extra would you be prepared to pay in road tax in order to fund research and development to cut down pollution caused by cars? But is this how R&D is done? How much of the cost of a car is already in R&D - and how much of that is on environmental performance?
  • ICM for Sunday Mirror: Q4-6 ask about support for green taxes. But there is no proposition here about what would be done with the revenues - in most green tax proposals these would typically be hypothecated to better alternatives or used to reduce taxes on employment or investment.
  • CommunicateResearch for The Sun: Q7. How many of these energy-saving changes would you be prepared to make to help save the planet...? 53% say yes to installing solar panels, but nothing is said about the cost. Q8 Do you think businesses which create pollution should be taxed more heavily? What, even if they passed the cost on to you? The CommunicateResearch poll (Q10) shows 63% prepared to sacrifice one foreign holiday a year (ie one return flight) to help save the planet by being more Green? That which compares with the Green Barometer showing 22% for a similar though not identical question about what activities you are willing to take for the sake of the environment: Flying less (flying to one holiday destination rather than two). The latter poll was 6 months later, so attitudes may have changes, but I suspect the latter sacrifice feels more serious.
  • YouGov for Daily Telegraph. Familiar results summarised in the commentary: The popular measures are those that are easy, or affect only a minority, or involve government largesse, while the unpopular measures are those that involve taxing or giving up everyday activities. This is one of the better polls, examining reasons why people might be selfish - and implicitly raising the question under what circumstances would people not be selfish. The commentary concludes: What is clear is that the most voters would be receptive to strong and effective political leadership – as long as people are persuaded that the Government plays fair on green taxes and can draw other countries into worldwide agreements that can be monitored and enforced. But the poll doesn't actually ask questions that would confirm that conjecture.
  • Ecoreports - finds that 100% of people (no less!): would like access to good quality organic food at reasonable prices; think the United States should be in a climate convention; and that it's worth making a little effort to make the World a better place for our children! Friends of the Earth polling comes up with better questions, but results show high levels of green awareness and motivation. Not surprising really, both FoE and Ecoreports use their own websites for polling and self-selecting visitors to the site as the base... so they don't count for anything much.
[Ps. please suggest others...]

Conclusion
The results of polls are highly sensitive to the way questions are asked and propositions framed, and how much supporting information is built into the question. Put another way, they can be entirely worthless if they don't allow people to express their preferences in a graduated way or if they present people with unrealistic or incomplete descriptions of the proposition - the worst offender in recent times being the No 10 web site petition on road pricing. This underlines the importance of the 'engagement' component of behaviour change strategies - see UK Sustainable Development Strategy / my posting on 'soft paternalism.

1 comment:

Simon said...

Hi Clive, I think the best (and certainly the funniest) example to show how "The results of polls are highly sensitive to the way questions are asked and propositions framed" comes from an episode of Yes (Prime) Minister dealing with conscription:

Sir Humphrey: "You know what happens: nice young lady comes up to you. Obviously you want to create a good impression, you don't want to look a fool, do you? So she starts asking you some questions: Mr. Woolley, are you worried about the number of young people without jobs?"
Bernard Woolley: "Yes"
Sir Humphrey: "Are you worried about the rise in crime among teenagers?"
Bernard Woolley: "Yes"
Sir Humphrey: "Do you think there is a lack of discipline in our Comprehensive schools?"
Bernard Woolley: "Yes"
Sir Humphrey: "Do you think young people welcome some authority and leadership in their lives?"
Bernard Woolley: "Yes"
Sir Humphrey: "Do you think they respond to a challenge?"
Bernard Woolley: "Yes"
Sir Humphrey: "Would you be in favour of reintroducing National Service?"
Bernard Woolley: "Oh...well, I suppose I might be."
Sir Humphrey: "Yes or no?"
Bernard Woolley: "Yes"
Sir Humphrey: "Of course you would, Bernard. After all you told you can't say no to that. So they don't mention the first five questions and they publish the last one."
Bernard Woolley: "Is that really what they do?"
Sir Humphrey: "Well, not the reputable ones no, but there aren't many of those. So alternatively the young lady can get the opposite result."
Bernard Woolley: "How?"
Sir Humphrey: "Mr. Woolley, are you worried about the danger of war?"
Bernard Woolley: "Yes"
Sir Humphrey: "Are you worried about the growth of armaments?"
Bernard Woolley: "Yes"
Sir Humphrey: "Do you think there is a danger in giving young people guns and teaching them how to kill?"
Bernard Woolley: "Yes"
Sir Humphrey: "Do you think it is wrong to force people to take up arms against their will?"
Bernard Woolley: "Yes"
Sir Humphrey: "Would you oppose the reintroduction of National Service?"
Bernard Woolley: "Yes"
Sir Humphrey: "There you are, you see Bernard. The perfect balanced sample."