Doing the math

Opinion

Hey there, time traveller!
This article was published 15/12/2023 (636 days ago), so information in it may no longer be current.

A number of interesting and unexplored, to date, issues arise from the 2022 Programme for International Student Assessment results.

PISA is an initiative of the Organization for Economic Co-operation and Development (OECD) of which Canada is one of 38 current members. Targeting students turning 16 in the calendar year of administration, PISA was first administered in 2000 and then every three years since, except for a four-year pandemic-related gap leading to PISA 2022. The focus in 2022 was mathematics.

Ranked (‘dead’) last in PISA 2018, Manitoba’s mathematics rank of sixth of the 10 provinces in PISA 2022 appears to have come as a relief. This is despite being three points – easily within the statistical noise of this study – from ninth place, and in decline since 2006, a trend seen broadly across the OECD. The Free Press headline Manitoba students math scores improve, but still ‘worrying’ (Dec. 5) along with Minister Altomare finding the results ‘remarkable’, illustrates the power of rankings to drive perception. Ironically, this perception is not far from a reasonable interpretation; Manitoba’s score is in line with the OECD average and equivalent to those of many developed nations and of five provinces. A measly three points probably delivered us from misinformed crisis mongering and management.

The Canadian report and the OECD’s Volume 1 report notify us that results for Canada (and several other nations) need to be treated with ‘caution’ due to possible score bias. (They also report the threat to be minimal for Canada.) The main issue is insufficient rates of participation of the randomly selected schools and/or students. Just three provinces are spared the asterisk – Saskatchewan, Prince Edward Island and New Brunswick. High-scoring Quebec and Alberta stand out in terms of concern; any PISA-motivated inclination to wonder what might be special about how they teach mathematics is unjustified.

If the public is to be informed by these studies, and ministries of education want to continue to use PISA for guiding policy and programming – the Bill 64 attempted overhaul of the oversight of public education was premised partly on such results – the asterisk issue needs to be addressed by ministries.

It is interesting and concerning that neither the local CBC nor the Free Press has brought perspectives from specialists in the teaching of mathematics to K-12 students. For example, explicit instruction re-emerged as a remedy for declining mathematics achievement in the Free Press Dec. 5 article. Space does not permit an explanation for why, in the research based on studies such as PISA, teaching strategy may be more properly viewed as an effect, rather than as a cause, in relation to student achievement and PISA scores. Strategies other than direct instruction might be best for some struggling students, explaining their association with lower PISA scores and the debatable conclusion that direct instruction is superior. This would benefit from the perspective of specialists in K-12 mathematics education.

As noted earlier, Manitoba’s mathematics score and trends are similar to those of many provinces and countries throughout the world. As such, score declines may be rooted in things other than how mathematics is taught. A factor well known to be related to student achievement is socio-economic status (SES). It is a focus of PISA and worth a look.

PISA 2018 and PISA 2022 results reported that participating Manitoba students had the lowest average PISA-generated indices of SES among provinces. This is consistent with the Social Planning Council of Winnipeg report Campaign 2000, End Child and Family Poverty that states that rates of child poverty are higher in Manitoba than in all other provinces.

In PISA 2022, the five provinces with the lowest average mathematics score in Canada – Manitoba, Nova Scotia, New Brunswick, Saskatchewan and Newfoundland and Labrador – also have the five lowest indices of SES.

This very cursory look – much deeper interrogation based on PISA results is possible – illustrates that SES is a critical factor in understanding PISA scores and the Manitoba context in particular. Little should be expected in terms of improved PISA scores from changes to the teaching of mathematics or other elements of ‘schooling’ given the pervasive and relatively widespread effects of poverty.

The math, at least, indicates that this merits attention as a top priority for Manitoba whether concerned with PISA scores or societal well-being.

Ken Clark writes from Winnipeg. His time in education included direct involvement with large-scale assessments, including PISA.