Overcast

Winnipeg, MB

23°c Overcast

Full Forecast

Welch's Gripe Juice

Winnipeg Free Press - ONLINE EDITION

City of pointless secrecy

Posted: 04/7/2014 3:03 PM | Comments: 0

Advertisement

  • Print

Let me preface this rather long and nerdy post by saying I have rarely had a more head-explodingly frustrating experience with any government in my 15 years of journalism. To illustrate my feelings, please enjoy the short animated GIF at the end of this blog.

Also let me preface this with one important definition: Open data refers to the idea that government information ought to be, to the fullest extent possible, available to the public. Data - everything from mosquito trap counts to oil well licenses - should be downloadable for free in an easy-to-use format that allows citizens to sort and search and map. Data don't lie. Open data is how citizens hold their government to account. Open data is how entrepreneurs create cool apps like this and this.

The city of Winnipeg (and the province, for that matter) is years behind other governments, even the uber-controlling Harperites, on open data. Very, very little information is available for download.

What we do get is only available after months and months of FOIs. I've been arguing for the last seven months with the city, for example, over a bit of harmless info nearly every other municipality in Canada released years ago.

To illustrate my point, let me tell you about my Kafkaesque experience trying to get what are called boundary shapefiles for data available on the Now Winnipeg site. This site is kinda nifty. It appears to be the start of some kind of public service portal based on Winnipeg's 236 neighbourhoods. It includes an  export tool that supposedly allows the public to "download your own datasets based on your current slice-and-dice settings, made available in a variety of machine-readable formats."

Holy moly! When we read those words, my online boss, Wendy Sawatzky and I nearly raced down the street for a celebratory beer at the Lincoln. Those words are music to a data nerd's ears.

Wendy and I then immediately tried to download some data, specifically the boundary files that would allow us to recreate maps of each of the city's 236 neighbourhoods and then overlay other cool data on top.

We hoped to, for example, match the neighbourhood shapefiles to the city's regular list of vacant buildings to create a map that shows which neighbourhoods are hotspots, or which neighbourhoods improved. Or, we could map census data, to get a really clear visual portrait of poverty.

With those shapefiles, we could map so much cool stuff, and uncover so many cool stories that illustrate otherwise invisible trends.

The trouble with the Now Winnipeg site, we soon found, was twofold.

First, the data was downloadable in obscure formats (irJSON? SPARQL?) that even our smartest geeks downstairs could not parse into a format useable by our industry-standard mapping program. That breaks one of the cardinal rules of open data: stuff has to be user-friendly.

Second, the ever-skeptical eye of my boss fell on this long and unwelcoming list of rules about how the data can be used.

Basically, the rules say the data can't be used. You can download it. But you can't modify or distribute it. You can't republish it. You can't even give it to a buddy. It is entirely useless to anyone, and, again, at odds with all modern open data principals, unless you get written permission.

So, we asked the city for two things. Would it waive the conditions of use, and could we have the data in usable formats such as ArcGIS or Excel?

We thought the answer to both questions would be a quick and cheerful yes. That's because, over a year ago, the city passed a policy on open data that mandates the city do exactly what we asked:  allow the public free and unfettered access to the biggest number of databases possible in easy-to-use formats.

Here's the text of our questions and the replies from the city's top communications boss, Steve West:

Q: We want permission to export all the datasets on the Now Winnipeg site. And we want to be able to use, disseminate and display that data with no restrictions.

A. We are pleased to approve access to the data on the NOW site under its terms of use.

Q: As per the Open Data plan, we request the data in "prevailing open standards" for machine-readable formats. For databases, we request standard formats such as: tab- or comma-delimited text (.txt or .csv), or other standard spreadsheet formats (.xml, .xls, .accdb, etc) For geographic data we request standard formats such as: ArcGIS (.shp), MapInfo (.tab), Geography Markup Language (.gml) or Keyhole Markup Language (.kml).

A. As the Open Data initiative has not yet been launched, we are not in the position to provide this data. An announcement would be made when the site is available.

Okay, I said to my boss, we'll have to have another crack at deciphering those obscure Borg formats, but at least the conditions of use have been waived.

Not so fast, said Wendy. West said "we're pleased to approve access," but also "under its terms of use," and those are the very restrictions we're trying to have waived.

Jeez, that's true, I said. So, I went back to the city and asked for clarification, even though my repeated questions seemed pretty simple to me. In response I got this:

The NOW site is currently governed by the overall City of Winnipeg terms of use. If the requested data is for personal use there is no concern or restriction; if being used for other than personal use, we require an explanation of the intended use.

And that's when I did this:

rage-type

After wasting weeks on this, we still have data Steve Jobs himself wouldn't be able to decipher, and we aren't allowed to use it for anything, anyway.

This is a system entirely focused on thwarting all modern methods of government accountability and transparency.

You'll find me and Wendy at the Linc, drowning our sorrows.

Fact Check

Fact Check

Have you found an error, or know of something we’ve missed in one of our stories? Please use the form below and let us know.

* Required
  • Please post the headline of the story or the title of the video with the error.

  • Please post exactly what was wrong with the story.

  • Please indicate your source for the correct information.

  • Yes

    No

  • This will only be used to contact you if we have a question about your submission, it will not be used to identify you or be published.

  • Cancel

Having problems with the form?

Contact Us Directly
  • Print

You can comment on most stories on winnipegfreepress.com. You can also agree or disagree with other comments. All you need to do is be a Winnipeg Free Press print or e-edition subscriber to join the conversation and give your feedback.

You can comment on most stories on winnipegfreepress.com. You can also agree or disagree with other comments. All you need to do is be a Winnipeg Free Press print or e-edition subscriber to join the conversation and give your feedback.

Have Your Say

New to commenting? Check out our Frequently Asked Questions.

Have Your Say

Comments are open to Winnipeg Free Press print or e-edition subscribers only. why?

Have Your Say

Comments are open to Winnipeg Free Press Subscribers only. why?

The Winnipeg Free Press does not necessarily endorse any of the views posted. By submitting your comment, you agree to our Terms and Conditions. These terms were revised effective April 16, 2010.