Meta-data for data.gov.uk datasets

Openness rating:
Reason: Content of file appeared to be format "CSV" which receives openness score: 3.
★★★★★  Linked data - data URIs and linked to other data (e.g. RDF)
★★★★☆  Linkable data - served at URIs (e.g. RDF)
★★★☆☆  Structured data in open format (e.g. CSV)
★★☆☆☆  Structured data but proprietry format (e.g. Excel)
★☆☆☆☆  Unstructured data (e.g. PDF)
Score updated: 04/12/2016
Government

A dataset of all the meta-data for all of the datasets available through the data.gov.uk service. This is provided as a zipped CSV or JSON file. It is published nightly.

Updates:

From 13/10/16 we added .v2.jsonl dump, which is set to replace the .json dump (which will be discontinued after a 3 month transition). This is produced using 'ckanapi dump'. It provides an enhanced version of each dataset ('validated', or what you get from package_show in CKAN API v3 - the old json was the unvalidated version). This now includes full details of the organization the dataset is in, rather than just the owner_id. Plus it includes the results of the archival & qa for each dataset and resource, showing whether the link is broken, detected format and stars of openness. It also benefits from being json lines http://jsonlines.org/ format, so you don't need to load the whole thing into memory to parse the json - just a line at a time.

On 12/1/2015 the organizations of the CSV was changed:

  • Before this date, each dataset was one line, and resources added as numbered columns. Since a dataset may have up to 300 resources, it ends up with 1025 columns, which is wider than many versions of Excel and Libreoffice will open. And the uncompressed size of 170Mb is more than most will deal with too. It is suggested you load it into a database, ahandle it with a python or ruby script, or use tools such as Refine or Google Fusion Tables.

  • After this date, the datasets are provided in one CSV and resources in another. On occasions that you want to join them, you can join them using the (dataset) "Name" column. These are now manageable in spreadsheet software.

You can also use the standard CKAN API if you want to search or get a small section of the data. Please respect the traffic limits in the API: http://data.gov.uk/terms-and-conditions

Read More

Data Resources (5)

HTML
Directory of CSV/JSON dumps
CSV
Latest datasets CSV dump
JSON
Latest datasets JSON dump
JSON
Latest organizations JSON dump

Additional Links (4)

HTML
CKAN API documentation
HTML
CKAN API terms and conditions
HTML
Dumps tips

Additional Information

Added to data.gov.uk 09/02/2010
Theme Government
Themes (secondary) Environment, Towns & Cities, Society, Transport
Geographic coverage Overseas, Global, United Kingdom (England, Scotland, Wales, Northern Ireland)
Update frequency As needed
Temporal granularity point
Mandate No value
Temporal coverage No value
Schema/Vocabulary No value
Code list No value
Service Level No value

Login to create an issue »

Issues

0 open issues

0 closed issues