The new Data.gov.uk - Behind the scenes
The new data.gov.uk site is fizzing with new features to benefit both data consumers and data providers. Here are the highlights:
- Searching for data is easier than ever, using the improved faceted search.
- Tim Berners Lee’s ‘Five Stars of Openness’ is used to ‘grade’ each dataset and if the web-link has actually broken, it is flagged up.
- Previews of spreadsheets are shown in the web-pages about each dataset, making it easier to check before you download. Knowing it has the right columns or row formats can really help when looking for similar datasets to compare, for example.
- A copy of all the data is stored by data.gov.uk, which helps secure its availability, for example when the source server may change its links and hasn’t updated them in data.gov.uk yet.
Most of this has been enabled by building the data catalogue on a brand new version of CKAN, a rapidly developing open source project spearheaded by the Open Knowledge Foundation. It has been used until now as the back-end for data.gov.uk and was only accessed via an API. Now it is used on the front-end of the site for all the pages under the ‘Data’ tab, exposing all the latest features. It’s a big step forward in lowering barriers to finding and using the data, and there are plenty more features on the horizon too we’re excited to be adding. The big challenge for any catalogue is getting really good records, so we’ve been also improving the tools for public bodies putting their data onto data.gov.uk. The web-form they use is vastly improved, with helpful nudges in the right direction - it checks their link to the data, discourages them from just putting links to intermediate web-pages, and helps them work with series of data, such as expenditure data which are updated monthly for nearly every public body. Some of these improvements will take a while to filter through, but we are dedicated to help public bodies publish their data better on data.gov.uk. Since data.gov.uk first used CKAN three years ago, it has become extremely popular for cataloguing data, used in dozens of public data sites across Europe, North and South Americas. With deep links into the open data community, it connects with projects like OpenSpending to provide tools for getting the most from the data, whether that is for transparency, building businesses or socially beneficial activities.
Lead CKAN Developer, data.gov.uk