You can’t both be right

Opinion

Hey there, time traveller!
This article was published 27/09/2011 (5124 days ago), so information in it may no longer be current.

There is a reason why smaller provinces do not see a plethora of mid-campaign opinion surveys.

Good surveys are expensive. They can only be done by a handful of companies. And in a province the size of Manitoba, with the media resources that now exist, it’s pretty hard to find the money to do a comprehensive survey.

The Free Press knows this. For the first time in a long while we’re producing a poll, conducted by Winnipeg’s Probe Research, without another media partner. Our Probe poll will be published Friday and Saturday of this week. Bucking the trend from past elections, the other news organizations wanted to go in another direction and declined to partner. Or they realized that once it appears in our newspaper first thing in the morning, they can report the results  essentially for free. Many times without mentioning who paid for the poll.

For those who went in another direction, we got two early surveys. CJOB struck a deal with Viewpoints Research on a small-sample poll the radio station released early Monday morning. It showed the NDP (41 per cent) way out in front of the Progressive Conservatives (32 per cent). The spread was even bigger in Winnipeg, where the Viewpoints survey found the NDP leading 46-25.

Party reaction to the Viewpoints survey was swift. Several Tories contacted Monday morning used words that rhyme with “hoarse spit” to describe the poll. NDP types were pleased but not comforted by the numbers. The NDP is doing its own rolling poll across key ridings and if they believe the race there is over, they’re not letting on.

The results were a bit surprising. Not because it wasn’t possible, but because most of the ground-level intelligence being collected by scribes such as myself indicated a closer battle. That completely unscientific observation was reflected in a second poll, this one sponsored by The Canadian Press and conducted by Environics Group. That survey showed the Tories (45 per cent) and NDP (42 per cent) running very close to each other.

The only thing we know for sure is that one of these polls is definitely wrong. Perhaps both. Call me a homer, but now I REALLY want to see the Probe numbers.

What could account for the variation? Tories were quietly fuming that CJOB partnered with Viewpoints, a pollster that is also working for the NDP and is co-owned by Ginny Devine, wife of former Manitoba premier Gary Doer. Sources, however, confirmed that the ‘OB poll was conducted separately from the party’s polling, and the questions were different. So, this is not a matter of ‘OB getting a piggyback on a partisan survey. And Viewpoints does not earn enough money off NDP work to screw around another paying customer. No conspiracy here, although it’s a horrible bit of bad luck for Viewpoints if they are the authors of an improbably positive NDP poll.

The thing that leapt out from the Viewpoints survey was the sample size – 579 respondents contacted between September 14 and 21, with only 363 in Winnipeg – which by measure is pretty small. Pollsters will tell you that a well constructed survey that collects properly weighted respondents across all demographic groups can be accurate even with a small survey. And yet, many are concerned, and with good reason, about the Winnipeg sample size.

That does not mean Environics should automatically be accepted at face value. No margin of error was reported with this survey, an odd thing to leave out during a writ period. And Environics was an online poll, which means the pollster did not keep making phone calls until they built a group that statistically represents the Manitoba electorate. Environics does do some weighting to adjust the survey results, but it’s unclear whether that really produces a representative group.

What does all this mean? If one of these polls is an outlier, it should be known by week’s end when Probe Research delivers its results. Probe has been pretty good at nailing the voting intentions and will be using a larger sample and more traditional methods to build a representative group. One thing is for sure, it’s impossible for both of these polls to be accurate.

Dan Lett
Columnist

Dan Lett is a columnist for the Free Press, providing opinion and commentary on politics in Winnipeg and beyond. Born and raised in Toronto, Dan joined the Free Press in 1986.  Read more about Dan.

Dan’s columns are built on facts and reactions, but offer his personal views through arguments and analysis. The Free Press’ editing team reviews Dan’s columns before they are posted online or published in print — part of the our tradition, since 1872, of producing reliable independent journalism. Read more about Free Press’s history and mandate, and learn how our newsroom operates.