Unlocking smart meter data for research

James O’Toole explains how the UK’s Smart Energy Research Lab is enabling researcher access to smart meter data.

With over 21 million smart gas and electric meters now in Great Britain’s homes, there is a growing wealth of half-hourly and daily domestic energy use data available to those who can access it.

The Smart DCC – the national secure network for digital smart meters – is currently mainly used by energy suppliers and power network operators.

However, this data has enormous potential to enhance research that can inform policy and industry approaches and help build momentum towards net zero.

Our consortium of universities – led from University College London (UCL) and funded by UK Research and Innovation (UKRI) – has taken up the challenge of making domestic smart meter data available to the UK research community and the work has already begun to bear fruit.

The Smart Energy Research Lab consortium (SERL) is currently enabling access to half-hourly and daily domestic gas and electricity data from some 13,000 GB homes to 50 researchers working on 21 research projects across 17 different research organisations.

In addition, we are working with government departments to facilitate their evaluation of several domestic energy saving initiatives.

Becoming other users

For non-supplier/distributor/transporter organisations to gain access to smart meter data via the Smart DCC they must apply for ‘other user’ status and go through a rigorous assessment by the Smart Energy Code Administrator and Secretariat (SECAS).

This is a very complex and time-consuming process which SERL was among the first non-energy industry organisations to go through.

Via our technical partners CGI we were able to navigate – and help formulate – the process to legally allow us to access the data so that we could begin to build the infrastructure that could make it available to researchers.

By making smart meter data available via SERL we have enabled many academics to access smart meter data without needing to become ‘other users’ individually

Have you read?
Think smart; act smart; win smart
‘Software is enabling utilities to push the envelope of grid capability’: Siemens CEO

The DCC adapter

The next step on the road was to access a DCC adapter – the technical piece of the puzzle that would allow us (via CGI) – to communicate with the Smart DCC and gather data from smart meters into the SERL Research Portal housed at the University of Essex’s UK Data Archive.

Again, this was a major undertaking that took several years to build and fine tune with our partners at the University of Essex before we arrived at our current position of supplying data from over 13,000 homes to 21 research projects.

The data

As well as providing smart meter data, SERL links this with other datasets such as Energy Performance Certificates (EPC), weather data and participant survey responses with info about the household and the building.

Smart meter data is personal data and consumers are understandably wary of sharing it with third parties.

All participating households in the SERL project gave informed, opt-in consent to use their data and verified their addresses by means of a unique code sent out in the recruitment materials.

Three waves of recruitment were run from 2019 to 2021, eventually signing up over 13,000 homes.

In the longer term we would like to increase the size of the sample with further waves of recruitment, but this is dependent on the availability of further funding.

The SERL research portal also offers the potential for researchers to bring in their own datasets to add further value – following a rigorous approval process.

Accessing the data

As you may have gathered by now, smart meter data is very well protected and those accessing it must go through thorough checks to ensure that their accessing of the data is in line with the Smart Energy Code and GDPR.

All researchers must have Accredited Researcher status via the Office for National Statistics (ONS), and all applications to use SERL data must be scrutinised by an independent Data Governance Board that checks that the project is ethical and that the research is in the public interest, in line with the principles of SERL and its funders.

All the SERL data is anonymised, and nothing can be exported from the SERL researcher portal without being thoroughly checked for any risk of re-identification of the data.

Currently our users are mainly academic researchers, but we are seeing greater interest from industry and public sector colleagues. We are also exploring ways of enabling non-academics to partner with university researchers on projects to achieve the maximum value for SERL.

Listen to:
Energy Transitions Podcast: Smart water at the heart of sustainable cities

The research

SERL has its own research programme – funded as part of the overall research grant to run from 2017 to 2022. This programme consists of eight projects led from our consortium of UK universities.

Research topics addressed include:
• identifying winter-time comfort in UK homes;
• analysing Welsh social housing stock;
• assessing the impact of Covid on home energy use; and
• understanding building thermal efficiency.

At the time of writing there are another 13 external projects using the SERL data with a steady stream of new project applications coming in.

For example, one project is looking at the impact of public health measures on household energy consumption, with the nationwide introduction of lockdowns in specific periods providing useful markers for analysing the data.

Another project is using SERL to assess how weather shocks affect domestic gas and electricity use.

A third project is looking at how SERL data could be used to test models for identifying fuel poverty and thus be able to introduce measures to alleviate it.

Next year, we will see many of these projects completing their work, and case studies and findings will be made available.

Stats report

Due to the nature of SERL, the research team spent a lot of time setting up the necessary governance and technical infrastructure before beginning the actual research.

Thus, they were exceptionally pleased to begin publishing papers and reports in 2021 and 2022. In particular the ‘SERL Statistical Report: Vol 1’ – published in May 2022 - was the first time it was possible to make publicly available the fruits of all the labour that had gone into the project since 2017.

Following shortly after was a granular statistical dataset and the statistical tables that had fed into the report.

These documents were well-received by the energy research community and at the time of writing there have been 683 downloads of the statistical report from across 36 countries.

The report shows how – for the SERL observatory sample – residential energy use in Great Britain varies over time (monthly over the year and half-hourly over the course of the day), with occupant characteristics (number of occupants, tenure), property characteristics (age, size, form, and EPC), by type of heating system, presence of solar panels and electric vehicles, and by weather, region and Index of Multiple Deprivation (IMD) quintile.

Key findings included:
• Energy use reduces with improved EPC Energy Efficiency rating. D-rated dwellings (the GB average) use 27% more gas and 18% more electricity than C-rated dwellings (the Government target for as many homes as possible by 2035);
• There is a greater presence of solar panels in A&B rated dwellings, and they consequently have a very different 24-hour energy demand profile than C-rated homes;
• Homes with solar panels use 62% less electricity from the grid than those without;
• During the coldest periods, daily gas use peaks around 7am whereas electricity peaks at around 6pm, with the peak occurring later in the evening when the temperature is warmer;
• Homes with electric vehicles have a median electricity use of 2,000kWh per annum more than homes without. However, homes with EVs use 22% more gas as well as 70% more electricity and – within the SERL observatory – are larger and have more occupants, which will also contribute to their increased electricity use.

What’s next?

SERL’s current funding phase came to an end in August 2022 but a no-cost extension has ensured that the work will go on for another year at least.

The SERL team are working hard to identify other sources and models of funding to keep the resource available to UK researchers, and if at all possible, to develop and improve the SERL offering.

We continue to work with government departments on evaluating intervention programmes and the applications from researchers continue to arrive.

After taking our first steps in facilitating what we hope will be a revolution in domestic energy demand research, we are keen to build on our work to enable research that can help tackle some of the biggest societal and environmental challenges faced by the world today.

ABOUT THE AUTHOR

James O’Toole is the Smart Energy Research Lab Consortium Manager.