Winnipeg Free Press - PRINT EDITION
Forum poll highlights need for skepticism
In the endless rumble over public opinion survey accuracy and elections, Lorne Bozinoff has become the latest punching bag.
Bozinoff is president of Forum Research, the Toronto polling company that failed spectacularly in its bid to predict the outcome of the federal byelection in Brandon-Souris.
In a survey last Sunday, Forum tracked Liberal Rolf Dinsdale with a 29-point lead over Tory Larry Maguire. When it was released to the media, the Brandon-Souris result seemed incredulous to just about everyone, including Liberals.
In the past 60 years, the Liberals have won that seat only once, and that win in 1993 required both a complete collapse by the Progressive Conservatives and a split of right-of-centre support with the Reform party.
On byelection day, the Liberal vote did surge, just as Forum had predicted, but so did the Tory vote. Maguire won, if only by a few hundred votes. The Forum result was so wrong Brandon-Souris is destined to be mentioned in the same breath as the worst election polling results of all time.
So, what happened?
Bozinoff said Forum is still studying the results to see where things went wrong. However, he said it's pretty clear his firm -- which employs Interactive Voice Response technology to gather its data -- did not accurately identify those people who were actually going to vote.
This is a key issue for all pollsters. You may get 100 people to say they are going to vote for a particular candidate, but the data don't mean much unless you can predict with reasonable accuracy whether they will show up to mark a ballot.
However, there are other factors Bozinoff believes were outside of Forum's control. These include an incredible effort by the Tories to get out their vote Monday. That last-ditch effort to squeeze every Tory vote out of the riding, motivated perhaps by the poll's huge Liberal lead, was not picked up in polling data, Bozinoff added.
Bozinoff is still defending his methodology despite claims from Brandon-Souris voters they were called as many as six times by Forum's automated IVR system. Bozinoff said it was possible an individual could be called in each of the four surveys conducted during the campaign. However, he said it was "impossible" for someone to receive six calls, or more than one call for any individual survey.
"I can tell you that didn't happen," Bozinoff said of the multiple-call allegation. "Look, we were the only firm that actually polled on this. People can say whatever they want but we were the only ones out there doing it."
While Bozinoff is searching for answers, his critics and competitors are sounding off. Darrell Bricker, CEO of Ipsos Public Affairs, was all over Twitter Wednesday assailing both Forum and the news organizations that published the outrageous results.
Bricker said there are best practices available to any polling company to avoid the problems Forum experienced. Failures of this magnitude cannot be explained by election-day ground game, electorate volatility or weather, Bricker said.
This was a badly constructed, badly executed poll, he added. The fact they got the other three races right does not erase the problems with Brandon-Souris. "(Forum) will tell you that they got most of the results right, but that isn't the real test here. Getting it right every once in a while is not the test."
Bricker said the truth could be revealed if Forum, like some other polling companies, published its raw and weighted data.
Bozinoff said he won't do that.
Still, the pollster was not the only target of criticism in the wake of the byelection results. News organizations that published the its poll, including the Free Press, were soundly thrashed.
For the record, the Free Press did publish the last poll with the 29-point lead but it was in a story questioning the methodology and quoting sources suggesting the result was not believable.
Bricker said it is acceptable to publish the results of a rogue poll as long as the news organization takes the time to include the proper background and context so readers know the results are suspect. Unfortunately, he said, not enough news organizations are doing that.
We know Forum's poll was flawed, but we still don't know why. And while we can't necessarily ensure bad polls won't happen in future, those of us in the news business can ensure all polls get the same treatment: healthy, respectful skepticism.
Republished from the Winnipeg Free Press print edition November 28, 2013 A4
Have you found an error, or know of something we’ve missed in one of our stories?
Please use the form below and let us know.
Having problems with the form?Contact Us Directly
(1 of 50 articles for this week)4:34 PM 0
Photo Store Gallery
- Man, 45, missing since Monday
- Break-in on top of her grieving
- RCMP charge more than dozen impaired drivers
- Synagogue serves up some Christmas meals and cheer
- Warm temperatures in Manitoba for Christmas
- Guests welcomed with open arms at Christmas Feast in the West End
- Other alleged victims for therapist charged with assault
- Openings, closings over the Christmas week
- No charges laid in fatal collision
- For some First Nations people, like Tina Fontaine’s family, grieving is part of the yuletide season
- MTS says services have been restored
- Winnipeg trucker dies in semi-truck collision
- New Transcona Boulevard offically opens today
- Remains may be decade old
- Break-in on top of her grieving
- Infant remains in locker may be decade old: Brodsky
- Police seek driver after pickup truck hits pedestrian
- No charges laid against man who killed teen mugger
- Oswald officially enters NDP leadership race; promises an "alternative vision"
- Police continue to seek info on driver; hit-and-run proves fatal
- Slain man leaves behind nine children
- Trucking firm in mourning after driver killed in crash
- Teacher pulled from classroom
- 'Fun party for two' ends in tragedy
- Photos released of suspects in coat heist
- RCMP investigate allegations of doctored invoices, improper payment
- Semi driver dies in collision at Lagimodiere, Dugald
- Twelve arrested, including former city council candidate; weapons and drugs
- Keep car windows clear or face fine: police
- Selby among victims of debit card-skimming scam
Ads by Google