Winnipeg Free Press - ONLINE EDITION

CBC’s ‘Rate my Hospital’ needs a second look

  • Print
Victoria General Hospital received an A+ ranking.

Enlarge Image

Victoria General Hospital received an A+ ranking.

The CBC’s Fifth Estate recently produced an investigation on the quality of hospitals in Canada — Rate My Hospital — which has been enormously popular and set off discussions across the country about the need to improve our hospital services. Clearly getting a better picture about how our hospitals perform is of interest to Canadians, and the wish to exceed status quo health service delivery resonates with patients, policy makers and health-care providers alike.

So what’s the problem?

Rate My Hospital is based on various pieces of evidence that were collected from patients, hospital workers and hospitals themselves (where they cooperated), along with data from the Canadian Institute for Health Information (CIHI). The general goal of the initiative is to make all of the information relevant to patients at the individual hospital level: anyone can go on the program’s web page, pick a hospital, and learn about how well it performs.

Each hospital receives a letter-grade built on five standard indicators of hospital performance: mortality after surgery; re-admission after surgery and medical treatment; and adverse events after surgery and medical treatment. CBC used indicators taken from CIHI annual hospital data, "standardized them" within four peer-groups (teaching hospitals, large, medium and small community) and then assigned each hospital a grade.

So, instead of saying that St. Joseph’s Healthcare (Hamilton) has a 5.34/1,000 mortality rate compared to eight at the national level, Rate My Hospital determines that it is one standard deviation below the average in its peer-group, thus, it receives a letter grade A.

The goal of making data accessible and in a formula most of us can understand is laudable. Unfortunately, it is also misleading.

For starters, the formula assumes that each indicator contributes equally to the overall ranking. How can we decide, however, that re-admission matters as much as mortality?

But the main concern is that the ranking system is relative, not absolute. In any system, excellent or terrible, some hospitals will be one standard deviation away from the mean, by the very definition of the standard deviation. A standard deviation is the average distance to the mean. It necessarily takes some units to be distant by more than the average to get that average distance.

Does this mean that the information in Rate My Hospital is valueless?

Certainly not, but it should be of more interest to hospital CEOs than to patients or relatives. Patients do not really need an overall score for each hospital, because individual patients are admitted for a specific diagnostic: as a result, they should be much more interested to know how a given hospital performs on a specific treatment.

And what are the untold consequences of publishing evidence on adverse events at the hospital level and having patients use that evidence to decide where to be admitted?

It really depends on the origin of adverse events: if they result from overwhelmed providers in facilities used beyond capacity, publication of adverse events may have a welcomed balancing effect. That is, patients may move to facilities with lower occupancy rates and lower adverse events rates — a good thing.

But, if they result from caseload characteristics, such as more frail patients at one hospital than another, and if better informed, potentially less frail, patients are the most likely to use that information to select their hospital, some hospitals will end up with ever more complex and frail patients whereas others will end up cherry picking the easy ones. In other words, it could lead to imbalance, and ever greater disparity in outcomes between facilities.

Other data used by Rate My Hospital are similarly problematic, such as the survey in which hospital nurses were asked whether they would recommend their hospital to relatives. The results made headlines because, worryingly, 25 per cent of nurses would not recommend their own institutions. This was not, however, really a representative survey and it is quite likely that dissatisfied nurses were more willing to answer the survey (in the negative).

There is no doubt that having more information available on hospitals is good thing — and patients should make health-care decisions based on good evidence. But patients do not need rankings based on assumptions and standardization that paint only a general portrait. Patients need more refined measurements of how a given hospital performs on a menu of relevant items specific to them.

We can’t rely on the media to provide this. Perhaps it’s time governments — and hospitals themselves — stepped forward and worked together with the media to make this happen.

 

Michel Grignon is an expert advisor with EvidenceNetwork.ca, an associate professor with the departments of economics and health, aging & society at McMaster University and director of the Centre for Health Economics and Policy Analysis.

 

—The Evidence Network

Fact Check

Fact Check

Have you found an error, or know of something we’ve missed in one of our stories?
Please use the form below and let us know.

* Required
  • Please post the headline of the story or the title of the video with the error.

  • Please post exactly what was wrong with the story.

  • Please indicate your source for the correct information.

  • Yes

    No

  • This will only be used to contact you if we have a question about your submission, it will not be used to identify you or be published.

  • Cancel

Having problems with the form?

Contact Us Directly
  • Print

You can comment on most stories on winnipegfreepress.com. You can also agree or disagree with other comments. All you need to do is be a Winnipeg Free Press print or e-edition subscriber to join the conversation and give your feedback.

You can comment on most stories on winnipegfreepress.com. You can also agree or disagree with other comments. All you need to do is be a Winnipeg Free Press print or e-edition subscriber to join the conversation and give your feedback.

Have Your Say

New to commenting? Check out our Frequently Asked Questions.

Have Your Say

Comments are open to Winnipeg Free Press print or e-edition subscribers only. why?

Have Your Say

Comments are open to Winnipeg Free Press Subscribers only. why?

The Winnipeg Free Press does not necessarily endorse any of the views posted. By submitting your comment, you agree to our Terms and Conditions. These terms were revised effective April 16, 2010.

letters

Make text: Larger | Smaller

LATEST VIDEO

Jaws of life used to free two people after two-car collision

View more like this

Photo Store Gallery

  • BORIS MINKEVICH / WINNIPEG FREE PRESS  070527 The 21st Annual Teddy Bears' Picnic at Assiniboine Park. The Orlan Ukrainian Dancers perform on stage.
  • Susan and Gary Harrisonwalk their dog Emma on a peaceful foggy morning in Assiniboine Park – Standup photo– November 27, 2011   (JOE BRYKSA / WINNIPEG FREE PRESS)

View More Gallery Photos

Poll

What should the city do with the 102-year-old Arlington Street bridge?

View Results

View Related Story

Ads by Google