Meta-data for datasets

Openness rating:
Reason: Content of file appeared to be format "XML" which receives openness score: 3.
★★★★★  Linked data - data URIs and linked to other data (e.g. RDF)
★★★★☆  Linkable data - served at URIs (e.g. RDF)
★★★☆☆  Structured data in open format (e.g. CSV)
★★☆☆☆  Structured data but proprietry format (e.g. Excel)
★☆☆☆☆  Unstructured data (e.g. PDF)
Score updated: 18/03/2018
Towns & Cities

A dataset of all the meta-data for all of the datasets available through the service. This is provided as a zipped CSV or JSON file. It is published nightly.

Updates: 27 Sep 2017: we've moved all the previous dumps to an S3 bucket at - This link is now listed here as a data file.

From 13/10/16 we added .v2.jsonl dump, which is set to replace the .json dump (which will be discontinued after a 3 month transition). This is produced using 'ckanapi dump'. It provides an enhanced version of each dataset ('validated', or what you get from package_show in CKAN API v3 - the old json was the unvalidated version). This now includes full details of the organization the dataset is in, rather than just the owner_id. Plus it includes the results of the archival & qa for each dataset and resource, showing whether the link is broken, detected format and stars of openness. It also benefits from being json lines format, so you don't need to load the whole thing into memory to parse the json - just a line at a time.

On 12/1/2015 the organizations of the CSV was changed:

  • Before this date, each dataset was one line, and resources added as numbered columns. Since a dataset may have up to 300 resources, it ends up with 1025 columns, which is wider than many versions of Excel and Libreoffice will open. And the uncompressed size of 170Mb is more than most will deal with too. It is suggested you load it into a database, ahandle it with a python or ruby script, or use tools such as Refine or Google Fusion Tables.

  • After this date, the datasets are provided in one CSV and resources in another. On occasions that you want to join them, you can join them using the (dataset) "Name" column. These are now manageable in spreadsheet software.

You can also use the standard CKAN API if you want to search or get a small section of the data. Please respect the traffic limits in the API:

Read More

Data Resources (6)

Directory of CSV/JSON dumps
Latest datasets CSV dump
Latest datasets JSON dump
Latest organizations JSON dump
Archive of old CSV/JSON dumps

Additional Links (4)

CKAN API documentation
CKAN API terms and conditions
Dumps tips

Additional Information

Added to 09/02/2010
Theme Towns & Cities
Themes (secondary) Government, Environment, Society, Transport
Geographic coverage Overseas, Global, United Kingdom (England, Scotland, Wales, Northern Ireland)
Update frequency As needed
Temporal granularity point
Mandate No value
Temporal coverage No value
Schema/Vocabulary No value
Code list No value
Service Level No value