Comparing rankings to reality on hospital quality

Opinion

Hey there, time traveller!
This article was published 26/02/2025 (217 days ago), so information in it may no longer be current.

Often when hearing a spokesperson tell us what we “deserve” — better service, lower taxes, stronger representation — we arguably do not. Rather, it’s a populist gimmick intended to imbue the wielder’s agenda with entitlement — the last thing we need.

With no such angle in play, then, comes the following contention: Manitoba hospitals deserve better treatment than the sentence “Winnipeg’s Health Sciences Centre has earned the dismal distinction of being the worst-performing hospital in Canada,” (HSC Canada’s worst-performing hospital, some Manitoba health care no better under NDP: nurses union report, Free Press, Feb. 12), and than the title of the Manitoba Nurses Union’s (MNU) white paper Healthcare in Manitoba is in Crisis, along with its claim of “deep systemic issues.”

That the ranking of a single measure — standardized mortality ratio, in this case — of the multi-faceted, multi-functional services of public health yields such a comprehensive indictment, is one issue.

Another is that ranks always include a first and a last, including when all entities being compared are very good at what they do.

Despite this, ranks are generally interpreted in absolute terms, ranging from functioning well to being deeply flawed or in crisis, such as the case in point, without justification for the interpretive leap.

The same happens with large-scale assessments of student academic performance. We’ll likely see it again this fall when results from the spring 2023 administration of the Pan-Canadian Assessment Program’s — PCAP — assessment of Grade 8 students’ achievement in science (the focus this time around), reading and mathematics, are expected to be released.

Ranks will be exploited to create a sense of panic and to buttress calls for more testing, more students failing, more stringent teacher training and qualifications, or insert your cause here.

Meanwhile, as with health care, exogenous factors (that is, out of scope or control) that plausibly explain the ranks receive short, if any, shrift in terms of analysis and interpretation. The considerable effort needed to do this is easily overlooked when eager to capitalize on a chance to make a splash for a favoured cause.

In the Manitoba context, be it education or health care, Manitoba’s documented high levels of socio-economic distress (see Doing the math, Free Press, Dec. 15, 2023) mean that street drugs, mental health issues, lack of affordable and safe housing, food insecurity and unlivable income are among factors that stand as compelling explanations for low ranks and signal strategies to improve outcomes.

In addition to exogenous factors, any absolute judgments based on ranks must be preceded by interpreting the measures in understandable, absolute terms. For example, crisis-mongering about how mathematics is taught based on Manitoba’s bottom rank in PCAP 2019 should be attenuated by the fact that the difference between Manitoba’s score and that of most other provinces equates to within about five percentage points on a final examination. This is despite the lowest levels of youth socio-economic status across provinces.

The burden is on the authors of reports, such as the MNU’s white paper, to undertake these analyses, with their own credibility and the valid interpretation and use of the results at stake.

Ranking is useful. An anomalous rank (considering contextual factors such as socio-economic status, for example) is intriguing and calls for interrogation leading to further insight and to more appropriate diagnoses and treatments.

Nurses (and other health-care professions) apply deft skills, in-depth knowledge, discretion, empathy, and patience, and do so under pressure and often, unfortunately, when exhausted. The MNU’s white paper recommendations are important and reflect the nature and needs of the profession. They are clear and reasonable in terms of their likely effect on nurses’ ability to perform at their best, for our best.

The paper’s claim of “deep systemic issues” and “crisis” in hospitals remains to be established, however, as real or central to what ails hospital urgent care and emergency care outcomes. The MNU’s recommendations stand well on their own as systemic antidotes, without the need for this tactic of broad disparagement of the public hospital system.

Ken Clark, retired in Winnipeg, spent most of his time while in the field of education specializing in large-scale assessments of student learning, which included interpreting results such as ranks.