Online city ‘surveys’ fundamentally flawed

Hey there, time traveller!
This article was published 15/12/2020 (314 days ago), so information in it may no longer be current.

The City of Winnipeg has released its draft 2021 budget, which I anticipate supporting at the council vote in mid-December.

One aspect of the budget, however, leaves me frustrated — the City’s habit of asking for online input prior to the budget, and then announcing the results as a "survey".  Any public opinion survey should try to ensure that the sample of respondents is representative of the larger population. The City has many smart, professional people doing public engagement, but unfortunately there seems to be no effort to make clear that their online questionnaires are NOT statistically valid surveys.

This year’s online "budget survey" contains the startling revelation that Winnipegers view the police as the third-least important service delivered by the City, trailing only golf and parking in unimportance.

So, what was the population sample that provided that result?

Only 19 per cent of respondents were aged 55-plus, while a full 37 per cent of the actual adult population of Winnipeg are in that age group.  

The respondents included 42 per cent who were 18 to 34 and 39 per cent who were 35 to 54.  Both of those groups actually account for 31 per cent of the city’s adult population. Similarly, the city is comprised of 35 different postal codes, yet 10 per cent of the respondents came from one code — Wolseley (but that is a separate issue!).  

How do these results compare to two other statistically valid surveys that were recently completed by city agencies?  

The City’s June 2020 citizen survey notes approximately 75 per cent satisfaction with police services, with an increase in calls for service from 585,000 in 2017 to 648,000 in 2019.  Moreover, of the 231,000 dispatched calls there were 108,000 classified as urgent or emergency.

A 2019 survey conducted by the Winnipeg Police Board noted that "only 25 per cent of respondents under the age of 35 thought the service was underfunded, compared with 53 per cent of respondents over 55."  

Other national surveys have shown a higher level of trust in the police for the over-55 demographic. These findings indicate a much higher perceived importance level for the police among Winnipegers than the budget questionnaire suggests.

If we want a realistic picture of how Winnipegers view their police services, and funding levels for police, as part of our budget deliberations, we should be asking those questions across all parts of the city and across all age groups.

I do not in any way discount the views of young people, but we should also be hearing from those aged 55 plus.