Finding the data is hard.
I'm finding it hard to search through the data you have available. The search works to some extent, but without some idea of what is on the site it's hard to know if I've just used the wrong search terms or if the data isn't there at all.
Some way to browse the data would be nice - for instance a page listing all the tags.
Otherwise well done on the nice site.
Cheers,
Ed
I second this, its the first thing I noticed when I came to the site 'Where's the browse data button?'.
Just to add that we are working on a browse mechanism in the background at the minute. We should have something to launch shortly to enable browsing by department & geographic area. Thanks for the feedback though.
Being able to browse the site by department and geographic area would be great :)
Also some kind of tutorial on what can be done with SPARQL would be very useful.
You can find links to SPARQL tutorials on the resources page - http://www.data.gov.uk/resources
hope this helps
I cannot get theSPARQL tutorial it flashes up with overlapping areas then disappears with "error on page" in status bar
Hi,
The link seems to be working okay now - maybe try it again?
http://jena.sourceforge.net/ARQ/Tutorial/
Thanks,
Phil
/search/node/*
CONNECT TO:-
/category/tags/*
I've been trying to use the site this morning and have noticed a couple of problems with search and browse facilities. I tried to email the site on the Contact page, but the Captcha system kept rejecting my submission! After 8 attempts to submit I decided it was a problem with Captcha and not my eyesight.
1. The search facility is consistently failing to return results when you click on the common example tags listed under the search box on the main page (education, NHS etc.). However, the search seems to work when the tags are actually typed in explicitly.
2. When you go to the list all data sets page and click on a subsequent page the system fails to find results; for example when clicking on "Next" or "2" to move to the following page.
It may be that these are OS or broswer-specific issues - I am currently using Firefox 3.5.7 on Mac OS X 10.5.8. Anyone else having similar problems?
Is there any way to work out what fields are in tables - data dictionary etc or am I missing the point? Maybe a filter by data type, dept, etc. It would be also be really useful in search to be able to filter data by it's regional nature - National, regional, council etc
Why has the site been designed with absolutely no thought for the end user?
Surely the whole point is to allow easier access to the Government-based data and statistics? As it stands it's a poorly designed site, which appears to be deliberately hiding its information.
A great idea, an abysmal execution.
I second that remark....As usual govenment retorich has been replaced by a deliberately confusing and unhelpful web site that makes accessing the real data difficult. No doubt done to put off those less determined to get to the data.
Would be helpful if the links to data were correct and accurate - some form of "link not valid" button would be useful - would save a lot of time.
http://data.gov.uk/dataset/local_authority_recycling_rates_1998-99_to_20...
Hopefully browsing will be easy, at the moment, it looks like the "Data" is being kept a secret !
Eromsnud
1. Data: There doesn't seem to be much, most of the links saying 'download' just take you to another page which you then have to sift through. (In some cases the link is broken as well!) So this site is really just an aggregator and not a genuine source.
2. Data: Put real data, i.e. numbers, in spreadsheet form. Not just a few charts.
3. US equivalent: Can't you copy www.data.gov in style? That's much, much more intuitive and easy to use. (Even though it also is rather short of yer actual 'data.')
I know this is only beta, but I read about it on the BBC website, and it has the feel of going off half-cocked, I'm afraid!
Anyway, good luck!
I have just read a number of entries in the Forum, one of which is yours.
I had this feeling and decided to create a website listing actual numbers, as published by the ONS and DWP.
Have a look at:
for data (including a few charts!) on:
Economy
Employment
Inflation
Population
Unemployment.
It is still being developed.
George
thanks
Agreed, it would be useful to have a list on one page of all the datasets (just the titles). Would save scrolling through almost 300 pages! Otherwise excellent site.
If this site encourages people to look at and appreciate raw data it will be worth the development effort.
A comment on searching terminology: perhaps an FAQ on simple searching would be helpful - eg, is a keyword the same as a tag; what exactly is a keyword.
For example, I searched for boarding (looking to see if any data sets referred to boarding kennels) and got back lots of returns about abortion and other health matters. It seems the search was picking up "Board",as in NHS health board. I don't expect it to search on individual syllables!!
Steve
The site would really benefit from a faceted search in my opinion. Being able to drill down the search results based on taxonomy information would make finding the content/data you are looking for much easier.
I think a certain amount of 'intelligent' work is needed to actually make the data accessible.
A bit more signposting, and narrative, of what and where the data sources are could go a long way.
For example, if I knew I wanted to compare NHS expenditure against healthcare outcomes, but didn't know where (or if!) the data existed, I could turn to a page on 'Data about Health' which might explain concepts around DH, SHAs, Trusts, ONS, ... and how to meaningfully make comparisons.
Just a thought.
JH
A list with a brief description of the data plus an indication of the size of the (xls, txt, ...) file would be quite useful.
Datasets are not much use without the ability for them to be properly discoverable. This means no only finding the set but finding the context of the data etc. The information that gives this context is metadata.
Without good metadata the data set loses much of its value.
Many of the complaints here would be recified with better metadata.
Bobbin
As Post #18 says a list of most used categorised data would be very useful. For example, Schools, Crime, Transport etc. Then split this up into seperate sections maybe. As it is Im a little confused...this might just be though.
Alex Askew
This whole site is a mess.
Somewhere there should be a contents list, schema etc. Otherwise we just don't know what is here. It is impossible to find stuff
Data needs categorizing and then sorting into appropriate sub sections,
Such as education, healthcare etc...
On average 54% of fields are blank. Check out my comments on metadata and data on this site on the strand "data on data on data.gov.uk" in General Discussions.
OMG has this web site been produced with completely no assumed for the end user?
Certainly the complete point will be to enable easier access towards the Government-based information and statistics? As it stands it is a poorly created site, which seems to become deliberately hiding its information.
A fantastic idea, an bad delivery.
I've also found it difficult to find data using the search function. It seems very specific to exact search terms rather than matching similar or related terms. It would be good if there was some kind of predictive/related term function so it returns similar results.
I've worked in data procurement for over 10 years and was keen to understand more about the 3236 datasets behind the headlines on data.gov.uk.
I started with the metadata itself and discovered quickly that it's pretty incomplete. To unlock the true value of the data on data.gov.uk it would be great to see an effort to increase completeness & input accuracy of the metadata. I absolutely take on board that the main function is to get the data out there first and foremost but the more reliable metadata the easier it is to find this data, thus unlocking its potential.
Here's what was found whilst digging around the metadata:
Data.gov.uk provides room for a weighty 29 attributes for each data set and with 3236 data sets [June 2010] , that’s a lot of information on the information. Of these 29 attributes on average 54% were blank with the worst offender being “last updated” with 92% blank entries [currency is fundamental when considering fitness for purpose].
I made some pretty graphs but I can't post them on here so are some key stats:
"Key" Fields - percentage of blank entries:
Last Updated: 92%
Data Type: 69%
Geographic Granularity: 47%
Update Frequency: 46%
Released: 36%
Department: 34%
Geographic coverage: 8%
Once you get past the blank fields it's interesting to see the types of data being inputted.
Here are the [crude] stats for the main fields:
Date Released - year by percentage [top 5]:
Blank entries: 36%
2009: 30%
2010: 16%
2008: 10%
Last Updated - year by percentage [top 5]:
Blank entries: 92%
2009: 4%
"Other": 3%
2010: 1%
2008: <1%
Update frequency - freq by percentage [top 5]:
Blank entries: 46%
Annual: 27%
Not updated: 8%
Quarterly: 7%
Monthly or more freq: 5%
Department: Entries by percentage [ top 5]:
Department of Health: 11%
Department for Children, Schools & Families: 8%
DEFRA: 6%
Welsh Assembly: 6%
Home Office: 5%
[Blank entries: 34%]
Geographic coverage: area by percentage [top 5]:
England: 41%
UK: 15%
England & Wales: 14%
Blank entries: 8%
Scotland: 7%
Geographic Granularity: area by percentage [4 entries- clear confusion about what this field means]:
Blank entries: 47%
GB & UK: 25%
“Other”: 17%
Local Authority: 11%
Data Type: Type by percentage [4 entries]
Blank entries: 69%
Administrative data: 22%
Survey data: 7%
Modelled: 2%
The search engine here is actually quite good. But there is a lot of data and it could be slightly better optimised for internal search.
Google search is much more sophisticated and you can use it to query the data on this website, which is indexed quite frequently by Google.
The Google query below, followed by your keywords, will return relevant results from this website - simply paste it into Google:
inurl:data.gov.uk keyword1 keyword2
what is lacking is not a browse button per se but any proper organisation of the datasets into broad themes - there are too many tags to look through really.
Other sites actually organise their data, and provide associated information on what is included under each theme for example the National Statistics Publication Hub http://www.statistics.gov.uk/hub/index.html or the Data Archive http://www.esds.ac.uk/ or ONS http://www.statistics.gov.uk/default.asp
(or Eurostat http://epp.eurostat.ec.europa.eu/portal/page/portal/eurostat/home/ or the UN Statistical Division http://unstats.un.org/unsd/databases.htm )
if people couldnt find data already via one of these routes then no doubt somebody thought that developing yet another website would be a good way to help users find data, but all they've done is waste more taxpayers money on lots more web developers through yet more duplication.
I echo the fact that the biggest problem is organisation, not necessarily the search tools themselves.
However, I will say there's definitely a good opportunity here for enterprising developers to aggregate data based on certain subjects into easy to use and browse through tools. Developer Jeff Gilfelt, for instance, used two data.gov.uk datasets on ASBOs to make the very popular ASBOrometer mobile phone app.
It would nice to see more iPhone apps for different subjects, like the NHS.
I'm trying to contact someone to discuss uploading data to the site, however the "Contact us" Word verification box is too small to read, making impossible to send messages as the audio captcha doesn't seem to correlate with the "word" in the box.
Hi, apologies for the issue with the contact form - we will investigate immediately. Please email the team directly at team@data.gov.uk Thanks, Paul
It would be useful to be able to browse date by publication date, and also to have a "new on site" list of the most recent data uploaded.
Hi Leah,
We are continueing to work on the browsing and searching options and publication date is something we are hoping to implement. In the meantime you can view the latest datasets via http://data.gov.uk/data/list or track via rss at http://data.gov.uk/data/%2A/rss.xml
Thanks, Paul
If you added a tag or an app which could capture every dataset relevant to a local government level (Postcode or LSOA is what we need easily not Regional, or just at the Local Authority level) or , which we could cut at different levels of geographical analysis this would be an extremely helpful tool and an improvement on other existing sources. Likewise you could then add apps for other client groups, then this would be a very useful site and safe a significant amount of research time - but currently I cant use it at all and am using other sources.
I agree. A little more 'transparency' with browsing the data would be helpful.
i also found th site hard to browse far from being transparent it all seems to be well hidden.
I wish the search functionality would search inside the data. But yeah, otherwise great site.
I am finding it a real struggle to wade through such as large amount of data. I am trying to get a simple list of schools matched to LEA so that I can have an AJAX driven drop down menu style login for my website (http://www.reportcommentbank.co.uk) but all I seem able to find is a whole host of complex data. If a browse system is implemented - as suggested on this forum - then a page history breadcrumb to click back to where you have been is a MUST !
Hi Ed - well it was not too bad when you first posted but getting a bit crazy now with so many more data sets coming online !
I totally agree with this: "...a page history breadcrumb to click back to where you have been..."
You must got it for easy to use first. Then, you can do somethings else.