Hey there, time traveller!
This article was published 2/2/2014 (1147 days ago), so information in it may no longer be current.
TORONTO — Several related things caught my eye recently, none of which provided much comfort on the subject of taking cues from our betters. Alas, it would seem that we still have to think for ourselves.
In The Wall Street Journal, Joseph Epstein reminisced about the year he spent as director of the antipoverty program for Pulaski County, Arkansas. It was the mid-1960s, and he was "27 years old, appropriately left-wing, and confident that society could be greatly improved with the help of large infusions of money and the serious thinking of people like myself."
But although the poverty Epstein saw was real, the purported solutions were ineffective. More precisely, they were "in the realm of fantasy."
Then The Observer — the venerable UK weekly that’s reputedly the world’s oldest Sunday newspaper — published an article on how a graduate student effectively debunked a widely-cited academic study on the mathematics of happiness. And this happened despite the fact that the study was peer-reviewed, and had been published in the prestigious American Psychologist.
Co-authored by Barbara Fredrickson and Marcial Losada, the study purported to have found a precise mathematical cut-off with respect to the impact of an individual’s positive-to-negative emotions. The critical number was 2.9013. If the ratio of your positive-to-negative emotions was greater than this, you’d flourish. If it was less, you’d languish.
And the study’s authors are heavy hitters. Fredrickson is an award-winning psychology professor and best-selling author, while Losada — who supplied the maths for the study — is a psychologist who consults on the development of high performance teams.
In contrast, the guy who initiated the takedown, Nick Brown, is a 53-year-old part-time graduate student. And although he insists he’s no whiz, he says that seeing the implausibility of Losada’s thesis required only a relatively straightforward understanding of mathematics. As he colourfully put it: "I can read simple calculus but I can’t solve differential equations. But then neither could Losada!"
Blogger James Joyner further stirred the dissonance pot. Drawing heavily on a recent piece in The Economist, he ventured the opinion that a healthy scepticism of most scholarly research is in order. The Economist expressed it this way: "There are errors in a lot more of the scientific papers being published, written about and acted on than anyone would normally suppose, or like to think."
Shallowness of expertise comes into play here. In Epstein’s telling, "My only qualification for directing a local antipoverty program was that a few months before I had been approached for the job I had published an article in Harper’s magazine about urban renewal. The article was roughly 6,000 words, and my total knowledge of the subject was perhaps 8,000 words."
But Epstein wasn’t a scientist writing scholarly papers, so perhaps he’s an unconvincing example of the problem. Scientists, however, have their own limitations, one of which is the lack of mathematical expertise.
Although scientific studies increasingly depend on complex statistical techniques, most scientists aren’t statisticians. Consequently, there’s a tendency to use tools that aren’t fully understood, an exposure that’s compounded by the proliferation of sophisticated software packages.
Surely, though, peer review should save us. After all, isn’t that where the wheat and the chaff are separated?
Not necessarily so, says The Economist, citing two examples where deliberately incorrect papers fooled peer reviewers. In one of the cases, the paper — described as a "dog’s dinner from a fictitious researcher at a made-up university" — was accepted for publication by 157 journals.
To biologist Michael Eisen, "peer review is a joke." The Economist is more circumspect, noting that reviewers "do not typically re-analyse the data presented from scratch, contenting themselves with a sense that the authors’ analysis is properly conceived."
So are all academics quacks, is science bunk, and are experts invariably useless? Hardly, any more than Epstein’s hapless experience in Pulaski County means that poverty is an unworthy subject.
But we do need to be tougher minded when it comes to ascertaining the true state of expert knowledge, particularly as it pertains to complex subjects. And we should be pre-emptively sceptical of "case closed" arguments that claim to speak from expert authority.
Put another way, reference to what "studies show" or "experts conclude" shouldn’t be the end of public debate on any particular issue. It should be the beginning.
Troy Media columnist Pat Murphy worked in the Canadian financial services industry for over 30 years. Originally from Ireland, he has a degree in history and economics.