Survey says: Can’t trust those polls

Hey there, time traveller!
This article was published 19/05/2013 (4612 days ago), so information in it may no longer be current.

OTTAWA — British Columbia’s stunning election upset has turned on its head one of the strongest arguments pollsters have always used against banning or restricting public opinion surveys during campaigns.

Since George Gallup pioneered political polling some 75 years ago, pollsters have maintained their surveys are vital to the health of democracy.

They expose for all to see the true state of voter opinion, the pollsters contend. To ban or limit them would be to return to the bad old days when biased, unreliable survey results were strategically leaked by political parties to advance their own partisan interests.

“Our polls speak truth to power,” Ipsos Reid senior executives Darrell Bricker and John Wright argued in a statement two years ago, responding to criticism from some of their own industry colleagues who’d challenged the credibility and accuracy of political polls.

“Not doing polls, or doing fewer of them, means that there will be no checks on bad polls (in this sense, the more polls the better), and that the political elites (including the media) will push their own distorted view of public opinion.”

Yet, in last Tuesday’s B.C. election, it was the public polls that turned out to be seriously distorted, giving the NDP anywhere from a six to 13-percentage-point lead over Premier Christy Clark’s seemingly doomed Liberals.

It was the Liberals’ internal party polls that accurately pegged the result: a healthy Clark majority.

“We had it nailed,” Don Guy, one of the Clark team’s top strategists, said in an interview.

“We couldn’t get anyone to believe us because of the public polls.”

Various explanations have been offered by pollsters whose surveys failed so spectacularly to predict the B.C. result: Voters made up their minds in the polling booths; turnout was lower than anticipated.

But Guy, a pollster by profession, believes it’s more serious than that.

“It’s a chronic problem and ‘problem’ is too weak a word to describe it,” he said.

Guy is not alone. An increasing number of pollsters are worried about a credibility crisis facing the polling industry, fuelled by methodological problems and a proliferation of fly-by-night polling firms offering free surveys to undiscerning journalists who are slavishly addicted to horse-race numbers.

Unlike many of the polls published in the media, parties pay big money to conduct sophisticated surveys with large samples.

During the B.C. campaign, Clark pollster Dimitri Pantazopoulos used old-fashioned, live telephone interviews to conduct his surveys — not the cheaper online or automated phone surveys increasingly used for media polls.

Online surveys — which use a self-selected panel of people who sign up to take part in a company’s polls — produce all kinds of anomalies, said Guy. Not everyone has Internet access; those that do tend to be younger and more inclined to support the NDP.

Media polls tend to be weighted to try to take account of such anomalies and make the sample representative of the general population. But Guy said that skews the result since not all segments of the population are equally likely to vote, a factor Pantazopoulos’s surveys did take into account.

For instance, Guy said, 18 to 34-year-olds “do not vote and, in particular, under 30s do not vote anywhere close to their incidence in the general population, whereas 55-pluses dramatically overachieve relative to their incidence in the population.”

Perhaps the most crucial difference between media polls and the B.C. Liberal party’s internal polls was the emphasis on the horse-race numbers.

“That is the number I pay the absolute least attention to,” said Guy, “because the choice that someone expresses about attachment to a political party if an election were held tomorrow is about the weakest possible measure or predictor of behaviour you can imagine.”

Rather, Guy said party polls looked at “issues, emotions, trends, personalities, characteristics, concerns, worries, fears, hopes — all those things that campaigns are founded on.”

It’s that kind of intensive polling that allows a party to fine-tune its message, micro-target constituencies, shift undecideds and motivate voters.

“If (campaigns) were founded on who are you going to vote for tomorrow, then we wouldn’t have campaigns,” Guy observed.

— The Canadian Press