Of lies, damned lies and polls

There's a principle in science called the Observer Effect that says the phenomenon being observed is changed by the act of observation.

If the same principle were applied to the science of polling, someone might reasonably ask if the canvassing of public opinion during elections is actually altering the outcomes.

Do polls lead or follow public opinion?

Where, for example, would the political parties be in the standings today if there were no polls? Is the so-called NDP surge real, or is it being sustained, reinforced and propelled by polling? Will it collapse because of the surge, or will it become a rising, unstoppable juggernaut?

On the flip side, did polls that suggested a Tory majority was within reach cause the party to slip?

Of course it's impossible to answer these questions, although the odds are that some of the pollsters will turn out to be roughly correct on election day.

The possibility that their prognostications might only be self-fulfilling prophecies -- that the prediction of a certain outcome produces that outcome -- cannot be measured, although in broad terms the NDP's unexpected breakthrough in Quebec seems to have been the catalyst for its growth elsewhere.

If nothing else, it is clear the electorate is volatile and that many Canadians are not thrilled with the choices in this campaign, which might explain the sharp differences in some poll results and the deep uncertainty about the final outcome.

There have been suggestions in the past that polling should be banned during elections because of their unhealthy bandwagon effect, but the idea has never received much support.

It would be like saying that we can't be told what we think, even if there's a risk that disclosing our collective thoughts might unnaturally create winners to be embraced and losers to avoid.

(It's not unusual, for example, to hear someone say they voted for Candidate X because they thought the person was going to win, and not for any other reason. Thank God, in my view, that voting is not compulsory.)

For their part, polling companies say their work is scientific and that the proof is in the pudding.

Indeed, although there are examples of polls that got it wrong, most of them achieve varying degrees of accuracy within the margin of error, 19 times out of 20.

So why, then, is one of the country's most famous pollsters so negative about political polling?

Allan Gregg, chairman of Harris-Decima, said last February in an interview with The Canadian Press, which is one of his clients, that he's appalled at the state of polling in Canada.

"The dirty little secret of the polling business," he said, "is that our ability to yield results accurately from samples that reflect the total population has probably never been worse in the 30 to 35 years that the discipline has been active in Canada."

Part of the problem, Gregg explained, is that fewer people have land lines today, and of those who do, many do not answer calls from polling firms or hang up after answering. As a result, response rates have fallen to just 15 per cent, mainly from elderly, less-educated Canadians who live in rural areas. It means the results are skewed and cannot be trusted, Gregg says.

He also blames the media for substituting quickie polls for real research, and for allowing polls to direct coverage and story lines.

And yet, assuming the polls are scientific, it is a little unfair to blame the media for telling the public where the race appears to be going, particularly when the political parties themselves are often citing their own polls as evidence of their momentum or some other untrustworthy spin.

The media are sometimes given too much credit and too much blame for how elections turn out. It's unlikely, for example, that these brief observations will have the effect of changing anyone's voting intentions, which merely shows there's an exception to every rule, most of the time.

dave.obrien@freepress.mb.ca

Republished from the Winnipeg Free Press print edition April 30, 2011 A19