Winnipeg Free Press - PRINT EDITION

Analyzing the studies

Canadian Medical Association Journal's deputy editor explains how to decipher the latest headline-making research

  • Print

Vitamin Y is good for you and can add years to your life, reads a news story reporting the latest, head-turning research to appear in a reputable medical journal.

So you head to the drugstore as fast as you can to load up on the miraculous vitamin.

A few weeks later, you come across news about another study claiming the same vitamin Y may be linked to early death.

What should you believe? Should you trash your recent purchase? Are most journalists reporting about so-called junk science? Why are you being bombarded with conflicting health news?

"I'm not concerned that the information is out there ... because that's the nature of scientific research," says Dr. Matthew Stanbrook, a Toronto-based lung specialist and researcher. "It's how you distill what's important from it."

Maybe you consider yourself news-savvy and disregard studies funded by corporations. Or perhaps you ignore studies that appear to be too small to matter.

You might be surprised to find out what makes health studies credible is not always so black and white.

Stanbrook, who is also deputy editor of the Canadian Medical Association Journal, a century-old publication that's considered one of the most reputable in the country, spoke to the Free Press about how to decipher the latest headline-making science.

Here's his advice:


Prominent journals are usually peer-reviewed, meaning every study they publish has been examined and approved by several experts in the field. "That's some sign that there has been some attempt from the scientific community to vet what's in there," says Stanbrook. He notes, however, there's a "wide spectrum" of quality even among peer-reviewed journals, so always approach studies with a skepticism.


Scientists want to see their work published in journals, where it garners the most attention and perceived credibility. So they submit their research papers to journal editors who, if they are interested in the study, then send it to experts in the field for further criticism. This back and forth process between the authors and the expert critics often goes a few rounds. "At the end, if the authors seem to satisfy all the criticisms being raised, the paper gets published, often with a lot of improvement from the initial submitted version."


"Not all scientifically valid findings turn out to be true. They can be contradicted by future studies, for example," says Stanbrook, noting that's the nature of all science. That means you probably shouldn't take mega-doses of the latest supplement to make the headlines without consulting your physician.


"A good scientist is skeptical of everything they read. And a good physician and a good member of the public should be skeptical of new things they hear about in science. That's part of science," says Stanbrook. "That doesn't mean we should dismiss studies because they're funded by companies that have an interest in selling their drug." He says that all scientists are likely to have some degree of bias; even scientist/academics who want to further their point of view. He says industry is unlikely to engage in "outright fraud" but may tend to leave information out of their studies. "It's not that they lied about what they did. It's what they don't tell you."


In general, the larger the study and the longer it is, the better. "But there are nuances around that," explains Stanbrook. He says too few patients in a study can leave questions about whether the study's outcome was accidental or not, or whether it affects men and women the same way. At the same time, studies that are too big also pose problems. (When you study population databases of 12 million people, there are many variables between the people, making coming up with a conclusion difficult.) What number makes a study too small or too large? That depends on what the study is trying to find out, says Stanbrook.


Experts consider randomized, double-blind, placebo-controlled studies among the best in the business. (That's when neither the researchers nor the patients know what treatment they are receiving. There's also a control group which is getting a placebo disguised as treatment/medicine). Problems with these types of trials? The patients in the experiment often don't represent a realistic cross section of the types of patients physicians tend to see. Also, they are expensive and take a lot of time to complete, says Stanbrook.

A randomized crossover trial (like the kind the Free Press wrote about last week) allows all groups to get the treatment before they are given a resting period to "wash out" the effects of the last treatment from their systems. The benefits? It allows researchers to conduct a fairly reliable study with smaller numbers and less money. The downside? Stanbrook says the effects of each treatment may not "wash out," thereby skewing results.

Keep in mind there are numerous scientific methods, each with pros and cons.


"People are not rats," says Stanbrook, although he notes that lots of promising research starts with animal studies. That's because "you can do the kinds of things with animals you can't do with humans" such as gene manipulation. At the same time, "the fact you can wipe out all cancers in rats with this treatment doesn't necessarily meat you'll be doing it to people the next week. It's just a hopeful development," he says.


Look for other studies that prove the same thing as the study-of-the-day, says Stanbrook, noting that repetition in science is the ultimate proof a finding is legitimate.


Rather than relying on a short television news segment for your information about the latest science, try reading the study itself, says Stanbrook, noting that his journal's website is getting more traffic than ever from the public rather than just doctors. He says that although studies aren't written in plain language, the editorial comment that goes with it often is and can give the public a good idea about details a news story may have missed.

Follow Shamona on Twitter: @ShamonaHarnett

Have an interesting idea you'd like Shamona to write about? Contact her at

Republished from the Winnipeg Free Press print edition July 23, 2012 D1

Fact Check

Fact Check

Have you found an error, or know of something we’ve missed in one of our stories?
Please use the form below and let us know.

* Required
  • Please post the headline of the story or the title of the video with the error.

  • Please post exactly what was wrong with the story.

  • Please indicate your source for the correct information.

  • Yes


  • This will only be used to contact you if we have a question about your submission, it will not be used to identify you or be published.

  • Cancel

Having problems with the form?

Contact Us Directly
  • Print

You can comment on most stories on You can also agree or disagree with other comments. All you need to do is be a Winnipeg Free Press print or e-edition subscriber to join the conversation and give your feedback.

You can comment on most stories on You can also agree or disagree with other comments. All you need to do is be a Winnipeg Free Press print or e-edition subscriber to join the conversation and give your feedback.

Have Your Say

New to commenting? Check out our Frequently Asked Questions.

Have Your Say

Comments are open to Winnipeg Free Press print or e-edition subscribers only. why?

Have Your Say

Comments are open to Winnipeg Free Press Subscribers only. why?

The Winnipeg Free Press does not necessarily endorse any of the views posted. By submitting your comment, you agree to our Terms and Conditions. These terms were revised effective April 16, 2010.


Make text: Larger | Smaller


Key of Bart: God Rest Ye Premier Selinger

View more like this

Photo Store Gallery

  • Challenges of Life- Goose Goslings jump over railway tracks to catch up to their parents at the Canadian Pacific Railway terminalon Keewatin St in Winnipeg Thursday morning. The young goslings seem to normally hatch in the truck yard a few weeks before others in town- Standup photo- ( Day 4 of Bryksa’s 30 day goose project) - Apr 30, 2012   (JOE BRYKSA / WINNIPEG FREE PRESS)
  • A red squirrel peaks out of the shade in a tree in East Fort Garry, Sunday, September 9, 2012. (TREVOR HAGAN/WINNIPEG FREE PRESS)

View More Gallery Photos


Now that former cabinet minister Theresa Oswald has entered the NDP leadership race, do you believe the "gang of five" rebel ministers were right to publicly criticize Premier Greg Selinger's leadership?

View Results

View Related Story

Ads by Google