Winnipeg Free Press - PRINT EDITION

Junk science is increasing exponentially

  • Print

A simple idea underpins science: "Trust, but verify." Results should always be subject to challenge from experiment.

That simple-but-powerful idea has generated a vast body of knowledge. Since its birth in the 17th century, modern science has changed the world beyond recognition, and overwhelmingly for the better.

Success can breed complacency, however. Modern scientists are doing too much trusting and not enough verifying, to the detriment of the whole of science and of humanity.

Too many of the findings that fill the academic ether are the results of shoddy experiments or poor analysis. A rule of thumb among biotechnology venture capitalists is half of published research cannot be replicated.

Even that may be optimistic. Last year researchers at one biotech firm, Amgen, found they could reproduce only six of 53 "landmark" studies in cancer research. Earlier, a group at Bayer, a drug company, managed to repeat only a quarter of 67 similarly important papers. A leading computer scientist frets that three-quarters of papers in his sub-field are bunk. Between 2000 and 2010, roughly 80,000 patients took part in clinical trials based on research that was later retracted because of mistakes or improprieties.

Even when flawed research does not put people's lives at risk -- and much of it is too far from the market to do so -- it squanders money and the efforts of some of the world's best minds. The opportunity costs of stymied progress are hard to quantify, but they are likely to be vast and could be rising.

One reason is the competitiveness of science. In the 1950s, when modern academic research took shape after its successes in the Second World War, it was still a rarefied pastime. The entire club of scientists numbered a few hundred thousand.

As their ranks have swelled -- to as many as seven million active researchers on the latest reckoning -- scientists have lost their taste for self-policing and quality control. The obligation to "publish or perish" has come to rule academic life. Competition for jobs is cutthroat. Full professors in America earned an average of $135,000 in 2012, more than judges did. Every year six freshly minted PhDs vie for every academic post.

Nowadays verification -- the replication of other people's results -- does little to advance a researcher's career. Without verification, dubious findings live on to mislead.

Careerism also encourages exaggeration and the cherry-picking of results. In order to safeguard their exclusivity, the leading journals impose high rejection rates -- in excess of 90 per cent of submitted manuscripts. The most striking findings have the greatest chance of making it onto the page. Little wonder that one in three researchers knows of a colleague who has pepped up a paper by, say, excluding inconvenient data from results "based on a gut feeling."

As more research teams around the world work on a given problem, the odds increase that at least one will fall prey to an honest confusion between the sweet signal of a genuine discovery and a freak of the statistical noise. Nonetheless, such spurious correlations often are reported in journals eager for startling papers. If they touch on drinking wine, going senile or letting children play video games, they may well command the front pages of newspapers, too.

Conversely, failures to prove a hypothesis are rarely even offered for publication, let alone accepted. "Negative results" now account for only 14 per cent of published papers, down from 30 per cent in 1990. Nonetheless, knowing what is false is as important to science as knowing what is true. The failure to report failures means researchers will waste money and effort exploring blind alleys already investigated by other scientists.

The hallowed process of peer review is not all it is cracked up to be, either. When a prominent medical journal ran research past other experts in the field, it found most of the reviewers failed to spot mistakes it had deliberately inserted into the papers, even after being told they were being tested.

All this makes a shaky foundation for an enterprise dedicated to discovering the truth about the world. What might be done to shore it up?

One priority should be for all disciplines to follow the example of those who have done most to tighten standards. A start would be getting to grips with statistics, especially in the growing number of fields that sift through untold amounts of data looking for patterns. Geneticists have done this, and thereby turned an early torrent of specious results from genome sequencing into a trickle of truly significant ones.

Ideally, research protocols should be registered in advance and monitored in virtual notebooks. This would curb the temptation to fiddle with the experiment's design midstream so as to make the results look more substantial than they are. It already is meant to happen in clinical trials of drugs, but compliance is patchy. Where possible, trial data also should be open for other researchers to inspect and test.

The most enlightened journals already are becoming less averse to humdrum papers. Some government funding agencies, including America's National Institutes of Health, which dishes out $30 billion in research funding each year, are working out how best to encourage replication. Growing numbers of scientists, especially young ones, understand statistics.

These trends need to go much further, however. Journals should allocate space for "uninteresting" work, and grant-givers should set aside money to pay for it. Peer review should be tightened, or perhaps dispensed with altogether in favour of post-publication evaluation in the form of appended comments. That system has worked well in recent years in physics and mathematics. Lastly, policy-makers should ensure institutions using public money also respect the rules.

Science still commands enormous, if sometimes bemused, respect. However, its privileged status is founded on the capacity to be right most of the time and to correct its mistakes when it gets things wrong. It is not as if the universe is short of genuine mysteries to keep generations of scientists hard at work. The false trails laid down by shoddy research are an unforgivable barrier to understanding.

Republished from the Winnipeg Free Press print edition October 19, 2013 A17

Fact Check

Fact Check

Have you found an error, or know of something we’ve missed in one of our stories?
Please use the form below and let us know.

* Required
  • Please post the headline of the story or the title of the video with the error.

  • Please post exactly what was wrong with the story.

  • Please indicate your source for the correct information.

  • Yes

    No

  • This will only be used to contact you if we have a question about your submission, it will not be used to identify you or be published.

  • Cancel

Having problems with the form?

Contact Us Directly
  • Print

You can comment on most stories on winnipegfreepress.com. You can also agree or disagree with other comments. All you need to do is be a Winnipeg Free Press print or e-edition subscriber to join the conversation and give your feedback.

You can comment on most stories on winnipegfreepress.com. You can also agree or disagree with other comments. All you need to do is be a Winnipeg Free Press print or e-edition subscriber to join the conversation and give your feedback.

Have Your Say

New to commenting? Check out our Frequently Asked Questions.

Have Your Say

Comments are open to Winnipeg Free Press print or e-edition subscribers only. why?

Have Your Say

Comments are open to Winnipeg Free Press Subscribers only. why?

The Winnipeg Free Press does not necessarily endorse any of the views posted. By submitting your comment, you agree to our Terms and Conditions. These terms were revised effective April 16, 2010.

letters

Make text: Larger | Smaller

LATEST VIDEO

City Beautiful trailer: How architecture shaped Winnipeg's DNA

View more like this

Photo Store Gallery

  • MIKE APORIUS/WINNIPEG FREE PRESS STANDUP - pretty sunflower in field off HWY 206 near Bird's Hill Park Thursday August 09/2007
  • Geese take cover in long grass in the Tuxedo Business Park near Route 90 Wednesday- Day 28– June 27, 2012   (JOE BRYKSA / WINNIPEG FREE PRESS)

View More Gallery Photos

Poll

Are you still on the Bombers' and Jets' bandwagons?

View Results

View Related Story

Ads by Google