Current landscape and scope for improvement

Summary of current environment

6.1 At the moment, government’s approach to the release of public data can be described as haphazard and in need of reform.  Some key examples include:

• It is not always clear what government department or public body from which to access data, nor the process required (e.g. Freedom of Information requests can be denied because the relevant department has not been contacted, and the permission process for licensed information – even where anonymised - can be lengthy and complicated); 
• We collect but do not publish other datasets e.g. data behind the indices of deprivation is not released, and the questions underlying the monthly Labour Force survey (without additional permissions);
• Local public services collect data but do not always make it available in accessible public formats e.g. release of location identifiers has been restricted for linking with certain datasets (including Citizenship Survey and Pupil Level School Census data) impeding more insightful analysis on migration and patterns of demand for education; and 
• Boundaries are not clear as to what public service data is. Some public service data is held by non-government or quasi government service providers and not subject to FoIA legislation.

6.2 This paper proposes to cover data relating to the provision of ‘public services’[1],  and welcomes views through this consultation.   

6.3 The current legislative, statutory and regulatory landscape is multi-faceted and its key elements, summarised in Table 1 below, include: Freedom of Information Act (FoIA); the Environmental Information Regulations (EIRs); the Data Protection Act (DPA); the Re-use of Public Sector Information Regulations (RPSI); and the INSPIRE Regulations.  The first three are regulated by the Information Commissioner’s Office (ICO) and The National Archives (TNA) has further regulatory responsibilities, including the investigation of complaints under the RPSI as well as managing Crown Copyright, and monitoring compliance against required standards under the Information Fair Trader Scheme (IFTS).  For queries and complaints under the INSPIRE Regulations, the ICO deals with restrictions of access, TNA with charging, and the UK Location Coordination Unit (DEFRA) with technical matters. Scotland will make its own arrangements. The degree to which this legislative framework applies to different public service providers varies, and this has often helped to create confusion and act as a barrier to openness.

Table 1: Overview of the current right to data landscape

Legislation and regulations Freedom of Information Act – FoIA gives a statutory right of access to all information held by over 100,000 public authorities, including government departments, local authorities, police and fire services, schools and universities, and the NHS. It applies to English, Welsh, Northern Irish and UK-wide bodies, but not to Scottish bodies, which are subject to a separate Scottish Act. There is a presumption of openness although some information may be deemed exempt from disclosure, for example in order to safeguard personal data, national security, commercial interests, and the effective conduct of public affairs. Environmental Information Regulations – The EIRs give the public access rights to environmental information held by a public authority.  Most bodies subject to the FoIA are also subject to the EIRs.  The EIRs also extend to some public authorities not subject to the FoIA.  Scotland has its own EIR legislation. Data Protection Act – The DPA (1998) provides a statutory framework for the ‘processing’, i.e. collection, use and disclosure, of personal information about living individuals. It applies to any organisation or individual within the UK who decides how to process such personal data. It also provides a number of rights to individuals, including the right to request to see any data that an organisation might hold on them.  The DPA implements the European Union's Data Protection Directive 95/46 into UK law, which is currently being reviewed by the European Commission.[1] Re-use of Public Sector Information Regulations – These regulations cover a wide range of public sector bodies in the UK, from central government departments and the devolved administrations, to parish councils, the health service and the emergency services. The RPSI encourage the re-use of public sector information by removing obstacles to re-use.  Public sector bodies are obliged to provide clear statements on their arrangements for re-use, including any licence terms and conditions and details of any charges. INSPIRE – the 2009 UK INSPIRE Regulations transposed the 2007 EU INSPIRE Directive, which aims to improve environmental policy-making in Europe. Member States are required to make available in a consistent format spatial datasets within the scope of the Directive, and create services for accessing these datasets. Doing so will enable datasets to be more easily shared and facilitate the development and monitoring of environmental policy and practice in Member States and across the European Union.  Implementation of INSPIRE forms part of the UK Location Strategy.     Bodies with regulatory powers Information Commissioner’s Office – the Information Commissioner regulates information rights legislation, specifically DPA, the Privacy and Electronic Communications Regulations, FoIA and the EIRs. The Commissioner’s remit is UK-wide except in relation to the Scottish FoIA and the EIRs, for which there is a separate Scottish Commissioner. The National Archives – under RPSI, the Office of Public Sector Information (part of TNA) has a statutory responsibility for the investigation of complaints relating to re-use. This links to the IFTS, which sets standards and principles, such as simplicity, transparency and fairness, which information providers in the public sector are required to meet as a condition of accreditation.[1] TNA also has agreements in place with key regulatory partners, the Office of Fair Trading (OFT), the ICO and the Scottish Information Commissioner.

 

Opportunities for improvement

6.4 Despite this array of legislation and guidance, potential requesters and re-users of data face a range of barriers to accessing, using and re-using data that could generate economic or social value:

1. Cost barriers resulting from historic ICT procurement and data management – information is held within government in a way that makes it costly to release, so requesters are refused on the grounds of cost.
2. Contracts agreed without consideration of Open Data principles – situations where public bodies have not considered in advance how data gathered might be made open.
3. Formats, quality or timeliness – often it will turn out that data is collected, but the manner in which it is held makes it difficult to use and re-use.
4. Incorrect application of exemptions or exceptions – which can lead to information being withheld inappropriately, requiring the use of the appeals process to challenge the decision.  
5. Charging to cover short-term costs – situations where the public sector charges for information in order to cover short term costs, perhaps not considering the opportunity costs of lost innovation or more efficient public services, which additionally would likely to be accrued elsewhere.
6. Licensing conditions and processes which are restrictive and can stifle innovation - these licences may also be delivered through slow, inflexible and cumbersome processes, where they are not released under the OGL and the wider UK Government Licensing Framework.  

6.5 In moving to the approach outlined in this document, there are a number of considerations, in particular cost and privacy.

Cost

6.6 Simply stated, the proposals outline how we might move to a position where most data held by public service providers about the provision of public services would be available for re-use under the Open Government Licence, except in very specific circumstances.  There are a number of factors that may preclude releasing data for free re-use and these would create some exceptions to this rule.

6.7 It is our intention that data already provided for free re-use should not be charged for, and there is no question of charging for data required for holding public bodies accountable or for “key data about public services, user satisfaction and the performance of all providers from all sectors. This will include data on user satisfaction, spending, performance and equality.”[2]

6.8 When considering whether or not to charge for data, a transparent business case setting out why will be made, including any cost or value for money implications. Under existing rules, where Crown bodies wish to charge for the re-use of data, they are required to justify this against the criteria for exceptions for marginal cost pricing to The National Archives, which manages Crown Copyright and database right. This justification ensures that government does not limit or restrict re-use.[3]

6.9 Any charging should follow existing rules in Managing Public Money (MPM) guidance[4]. MPM guidance states that much information about public services should be available free, or at low cost, in the public interest.  The guidance explains that where re-use of data is charged for the norm is to charge at marginal cost and for value-added data and information currently sold by trading funds the norm is to charge at full cost plus an appropriate rate of return. Other value-added data services may also be charged for, for example: services commissioned in response to particular requests; services where there are statutory powers to charge; and publications processing publicly gathered data for the convenience of the public, through editing, reclassification or other analysis.

6.10 The norm for all information supplied by trading funds is to charge, within the constraints of MPM guidance, although the Government is committed to moving to making more data freely available, within the constraints of affordability and value for money.  The implication for a Public Data Corporation (PDC) is being explored through a public consultation.

6.11 Over time, we would expect costs to lower in most cases, particularly as the ICT-related costs of providing data lower. There will also be benefits which may offset costs in the medium-term from reductions in bureaucracy through transparency of data; reduced FoIA requests for the data published; and some reduction of data where collection is considered to be unnecessary. To mitigate costs during a time when the public sector must be particularly mindful of public funds, we propose that the emphasis be placed on releasing new data rather than old, and on releasing data ‘as is’, rather than spending time and resource on improving quality immediately.

6.12 In the autumn, when the response to the consultation and full strategy is set out, we will include a full Impact Assessment of costs and benefits of Open Data, including consideration of any opportunities and burdens on public bodies and public service providers which will inform the policy development.

Privacy

6.13 With the exception of allowing individuals access to their own personal records, at no point in this document does the data we describe refer to personal data. Transparency will not be extended at the cost of privacy.  Releasing greater quantities of anonymised data brings with it complex questions about how we can ensure that personal data remains protected.  We will consider how we will, practically, ensure that personal data is anonymised, particularly when they are released alongside many other datasets, which have the potential to be merged.    

6.14 The Minister for the Cabinet Office has commissioned an independent review by Dr Kieron O’Hara of Southampton University to consider privacy and Transparency, recommend steps for ensuring that as the Open Data approach is adopted, privacy is not compromised.  We will publish this report over the summer.

6.15 In addition, where there is good reason not to publish information or data as set out in existing exemptions or exceptions in the legislation, for example because it would compromise national security – these exemptions will remain.

Footnotes

[1] In line with the recent Open Public Services White Paper http://www.cabinetoffice.gov.uk/resource-library/open-public-services-white-paper

[2] Para 3.4 of the Open Public Services White Paper: http://files.openpublicservices.cabinetoffice.gov.uk/OpenPublicServices-WhitePaper.pdf. 

[3] See http://www.nationalarchives.gov.uk/information-management/ifts/cost-pricing.htm

[4] http://www.hm-treasury.gov.uk/psr_mpm_index.htm