Description & Request Overview
- The UK Met office (“UKMO”) has an extensive historical database of raw weather observation data. There are currently c.200 UK weather stations used by the UKMO to collect synoptic data (http://www.metoffice.gov.uk/climate/uk/networks/), with an average of c.40 parameters per station including Temperature, Wind speed, Wind direction, Cloud cover (rating 0-9), Solar radiation, Relative humidity in %, Precipitation in millimetres, etc.
- In essence, historical weather observation data can be used to analyze and study the past to identify trends and patterns to help forecast weather (i.e. calibrating weather prediction models) and how weather correlates with other variables (i.e. temperature and crop yield, energy use or road temperatures, rainfall and flooding / flood defences, wind speed and wind farm power generation).
- We believe this should be made available at reasonable cost for full access to stimulate private enterprise that can provide wider benefits to the economy and society.
- Currently the UKMO does provide hourly historical weather observations data sets (combination of free and paid for) but these are all limited in some way, either by access, time / date, level of detail or pricing (higher frequency data is recorded but not made available). For example the current free service is limited to a rolling 12 month period only via the UK Met offices Datapoint service, along with less than a quarter of the full list of weather parameters in the full hourly weather report. Longer term historical observation data is available, but is based on monthly averages for a few select parameters.
- The applications cover a wide range of industries and uses (see 1.2 below), much wider than what we believe can be adequately addressed by one organization, the UKMO.
- Some of these applications link with broader government objectives such as renewable energy, climate change, growth of the private sector etc.
Data Release Rationale
- Private sector enterprises can use the raw data and add value (i.e. through tailoring, interpretation, presentation, collating with other data sources, etc) to create innovative and often highly specialized products and services for specific customer needs.
- The application of historical weather observational data can be used in many industries including:
- Transportation i.e. Route optimisation and winter maintenance for the roads, rail and aviation industries, improving safety and efficient use of resources.
- Energy i.e. site assessment, operational management and power management for renewable energy and more traditional uses such as load forecasting.
- Agriculture i.e. choices of crop/mix, land to cultivate, frost protection, animal selection etc
- Water management: i.e. planning the size and operation of reservoirs
- Insurance i.e. claims verification and risk assessment to correctly price premiums and incentivise customers to take preventive actions,
- Construction i.e. optimal planning of construction activities
- Media i.e. provision of weather information direct to consumers
- Given the wide range of applications and that new uses for weather data continue to be developed (i.e. Smart grids and energy consumption) the value to the economy in driving weather related commercial enterprise, risk assessment and resource allocation is significant.
- By opening up the full suite of raw data collected and archived regularly by the UKMO as a public service, the value of this data could be more fully leveraged by the economy / society as a whole.
Can you identify further areas where this dataset release will create opportunities for innovation and new business?
Can you help identify sectors, businesses and organisations that will benefit from the release of these data?
- The primary users of the Historical UKMO observation data would be private enterprise (companies, SMEs or individuals) allowing them to develop and/or improve products and services for the private weather market.
- The benefits provided to UK society due to increased private sector activity in weather includes:
- Increased economic activity leading to employment, profits, taxes etc;
- Improved services provided for other private enterprise / public sector organisations that can drive efficiencies and safety
- Increased range of services provided by private enterprise and innovation that competition creates – benefiting the consumer and society at large
- Quantifying the benefit of the release of UKMO historical data is difficult due to:
- Historically, the weather industry in the UK and Europe has been dominated by the national Met offices and therefore the private weather market is still relatively nascent;
- There is a lack of information on the size and revenues of the private weather market;
- Historical observation data and its uses only comprise part of the market.
- It has a wide range of uses (see 1.2 above for various sector examples)
- However, there have been estimates calculated for the total private weather market for the US and EU, which can be used as a guide for the potential impact.
- The total value of the private Weather markets (excluding the large financial weather derivative market) was estimated to be c.$1.4bn in the US and c.$0.5bn in the EU by PRIMET (1), and was broadly supported by other studies (2 and 3).
- As a % of GDP (4), the US private weather market is c.0.009% of the economy compared to c.0.003% of the EU economy. A key difference between the two markets is that in the US data produced by the National Weather Service (UKMO equivalent) is available for free (plus the additional cost of distribution), whereas the main Met offices in Europe charge / restrict data.
- For simplicity, assuming that EU weather market has the potential to grow to a similar relative size if restrictions were lifted, then this would increase the EU market to c.$1.6bn. Again, for simplicity, assuming the UK’s share is in proportion to its share of GDP, this would result in the UK weather market increasing from c.$70m to c.$225m, a $155m increase (c.£100m), providing additional employment, tax revenues, etc.
- It is important to note that the market associated with the raw observation data is only part of the market and that the data is already available on a paid for basis, therefore, the impact of this data release would be significantly smaller.
- None-the-less, the private sector has already proven it is capable of leveraging weather data, with private weather companies having grown significantly as more public weather data has become available.
- For example, the UKMO’s commercial arm currently represents c.16.6% of total turnover totalling c.£32m and over the last 4 years has grown by c.£3m, a CAGR of c.3.3% (5). This is in comparison to the leading European private weather company having grown revenues over a similar period by a CAGR of c.12.5%, generating c.£28.9m in 2011 (6).
Input on further case studies or quantifiable evidence to support the release of this dataset would be welcome
Barriers and Requirements for Release
- Higher quality data at a reasonable cost would allow private enterprise to innovate and drive revenue growth, as well as encourage new entrants to the market by reducing a key barrier to entry.
- Additional effort should be minimal as the data is already collected and used by the UKMO and its commercial operations. In addition, the UKMO already releases / distributes a significant amount of data of a similar nature.
- The current commercial operations of the UKMO would not be unfairly affected, as no IP or internal know how needs to be released, but only the raw observation data.
- In order to achieve the aims of this request the data should:
- Contain all the raw data as used and recorded by the UKMO i.e. all time stamps, all meta data descriptions, all weather station parameters and instrument data
- Data should be formatted and distributed in a usable format
- For all relevant periods available
- If charged, it should be the incremental cost of delivery / formatting.
- There are minimal risks of fraud and privacy.
Are you aware of any other barriers to this data release?
- Further research could be performed to provide more detailed analysis. However, making historical raw observation data available for free (plus reasonable distribution re-work costs) and reducing limitations (frequency / all parameters / meta-data) should be a relatively straight forward process for the UKMO.
- Therefore, given the impact of opening up weather data in other economies, and the UK so far, we believe this would lead to an increase in private sector activity.
- On this basis, we believe this should be progressed.
- PRIMET: The Association of Private Meteorological Services: Exploiting Meteorological Data (August 2011).
- The Private Sector in Meteorology : An update, Bulletin of the American Meteorological Society. David Spiegler (2007)
- Board on Earth Sciences and Resources (BESR): Fair Weather: Effective Partnerships in Weather and Climate Services (2003)
- "Nominal 2011 GDP for the world and the European Union (EU).". World Economic Outlook Database, October 2012. International Monetary Fund.
- UKMO statutory accounts
- MeteoGroup – as part of the Press Association statutory accounts