Government’s open-data portal at risk of becoming a data dump

The Government's new open-data portal is not yet where it would like it to be, Minister Brendan Howlin said in a Department of Public Expenditure and Reform meeting room earlier this week.

In case expectations are too high, the word “pilot” is in italics when you visit the site in question – data.gov.ie.

Meanwhile the words "start" and "beginning" pepper the conversation with the Minister and a variety of data experts from the Insight Centre in NUI Galway who have helped create the site.

Data.gov.ie allows those in the Government, as well as interested businesses and citizens, to examine data from a variety of public bodies, opening opportunities for Government efficiencies and commercial possibilities along the way.

The main problem is that there is not much of it, and a lot of what is there can’t be utilised in a particularly useful fashion.

As director of the US Open Data Institute Waldo Jaquith told The Irish Times, with "almost no data" available in a format that's genuinely usable by app developers, businesses or interested parties, for the moment data.gov.ie represents "a haphazard collection of data".

Dermot Daly, founder of Dublin-based app developers Tapadoo, who previously used open data released from the Dublinked "datastore" project – which opened up data from the capital's four local authorities – to help build a Dublin Bus app was also concerned as he made his way through the data available thus far.

“If I look at [data.gov.ie] most of the data formats they offer are HTML or PDF,” said Daly, who explained that this means “they’re either web pages or documents and they’re things which are not easily consumed by applications”.

Not useable

What Daly is getting at is that there’s not a lot of it that’s machine readable. Developers, businesses and even other departments or local authorities need all the information to be in a data format “that’s really easy to parse and consume”, and in turn actually useful.

For the data already housed in these awkward formats on data.gov.ie, he adds that “there’s no quick way” of transferring it into a format that will help this overdue open-data project fulfil its role.

Five years on from the UK data repository launch (data.gov.uk), after the guts of two years of talking about it, the Irish open-data portal is also following the example of a number of US states as well as cities such as Helsinki and Cape Town.

However, on the plus side for Minister Howlin and his colleagues, they should be aware that, when it comes to open data, aside from the odd superb example, they don’t have much to live up to. “Most of them are quite bad,” said Jaquith.

Data.gov.ie, it is hoped though, will become a hub for data from all departments, local councils and other arms of Government. So far it contains 418 datasets – which takes a variety of documents, web pages or other formats that Daly spoke about, as well as some machine readable data.

Over 18 months since Minister Howlin announced Ireland’s intention to explore and implement open Government in his budget speech, the result thus far is a somewhat basic search engine of the data available.

With an audit to be carried out of datasets available within the public service by mid-2016, involving “all public bodies”, it has to be hoped that in certain crucial areas, data becomes more widely available. The HSE, for example, provides just one dataset of the 418, regarding discharge activity in acute public hospitals.

An Garda Síochána have supplied one batch of data on traffic fatalities and offences. The Central Bank provides five, including stats on the value of outstanding residential mortgages. National debt stats and four other documents are supplied by the Department of Finance.

Minister Howlin's own department has offered up just three datasets thus far, while the Marine Institute and the Central Statistics Office are keeping the whole thing afloat with 244 datasets between them.

‘Full of holes’

The portal project intends to create the type of intelligence that can drive department efficiencies and produce tax-raising commercial possibilities, and Minister Howlin wants engagement from almost all arms of the State.

As with many other Government projects, we could just follow the example of what Britain has already achieved. Indeed, Minister Howlin has said they are “market leaders in relation to open data” and talks of “a number of conversations with Francis Maude” in which the UK minister for the Cabinet Office told him he regards open data as “the most important issue he has undertaken”.

Surely after five years they can show how to put our best foot forward?

Not so, said Ian Makgill, managing director at Spend Network which specialises in collecting the spend statements and tender documents published by the UK government as well as other governments around Europe before publishing this data openly.

“To begin with [data.gov.uk] was exactly the same” as the current Irish version, he said, “it was full of holes, not a lot of it was machine readable, there was a lot of PDF stuff and I’m afraid to say it’s still like that”.

There is a lot more data than previously, said Makgill, but considering his business is built on government data it’s telling he says that the data.gov.uk initiative has “not been a big help to me”.

The new Irish open-data site was accompanied this week by the Open Government Partnership Ireland National Action Plan, and the Open Data Ireland Roadmap. Both of which paint an image, and an often vague one at that, of where this new interactive repository of official, non-personal data can make a difference.

The Irish open-data plans are centred on goals over those three years which, taking into account the issues experienced in the UK and the technical difficulties involved with making what’s already on data.gov.ie in any way useful, seem extraordinarily ambitious.

Certainly there's a recognition in the documents released by the Government this week of the task ahead. Research on best data practice, produced by Insight researchers Deirdre Lee, Richard Cyganiak and Stefan Decker – notes that "the aim of any open-data initiative is not simply to throw datasets onto a website, but to build a sustainable ecosystem around data that supports social, economic and political impact".

Encouraging release of information

The trio want each public body to carry out a data audit of the data they have. They then want any data that’s already out there to be machine readable, and built on an open-source platform called CKAN so that all the data involved will eventually mesh together.

As part of the plan, the Department of Public Expenditure and Reform will appoint an open-data officer, while each public body is expected to “designate a person or team” who will be responsible for open data and to “create their own strategy” in this area.

Emer Coleman, formerly former deputy director of digital engagement in the UK's Government Digital Service and now with Transport API which uses open-data feeds from Transport for London and other sources to provide nationwide timetables and service details across all modes of transport in Britain, said that some credit must be given to the Government for beginning the process of opening up public data.

“It’s very difficult to get governments to open up data in the first place,” said Coleman. From her experience in the UK, she says she’s learned that government agencies simply “don’t want to release data”.

Her recommendation for Irish Government arms weighing up the consequences of becoming part of an open Government is to “put out what you’ve put on your websites in machine-readable form” and then leave it to developers, departments and data scientists to make use of it.

“It’s not something government can do on their own. Internally it’s very difficult to say what people can make of this,” added Coleman.

Richard Stirling, international director of the UK Open Data Institute, said that ultimately the project will rise and fall on "how the Government is going to engage with the business community and the people who are going to use" open data.

Stirling talks of state employees learning about “visualisation tools” to map the data and “modern data analysis languages” being taught. Said Stirling, Government bodies should “focus their efforts on the data that’s going to have the biggest return”.

And there can be a return. Take Climate Corporation in the US, a company who utilised freely available data related to soil, weather, crop pricing and more before selling their product for almost $1 billion (€74 million).

Or how about Mastadon C in the UK, which figured out a way to save the NHS £200 million (€252 million) through generic drug use.

Irish companies such as Daly’s, as well as San Francisco-based Buildingeye, too have used publicly available data to create new products, services and business opportunities.

Civil service incentives

However, the difficulties in getting staff to adapt to the open data is typified by the news that back in the UK local authorities are now having to be incentivised to the tune of £7,000 to publish some key datasets.

In this context, it’s difficult not to agree with Makgill that talk from Minister Howlin of “reaching out” to local authorities won’t be enough to properly kick-start things.

“What chance do you think you have of persuading everyone in the civil service – let’s ignore local government and the health service for the moment – who’s [got] responsibility for publishing [data], to get them to understand a uniform data model and a uniform data publication [method]?”