The how, why and benefits of synthetic smart meter data

Synthetic smart meter data has all the benefits of real data while protecting the privacy of customers, says Gareth Jones, COO of the London-based Centre for Net Zero.

Data is all-important in today’s energy sector, particularly smart meter data. It tells us how all the electricity – and similarly other utilities such as gas and water – that is produced is being used.

Long gone are the days when such data was used only for billing. Now it is opening the way for multiple other use cases from the delivery of flexibility to long term grid planning.

An individual’s data has limited value, but when aggregated – whether on the scale of a utility service area or country – it can become an essential input to the applicable planning process.

This in essence is the motivation for ‘open data’ and ‘data spaces’ where data from multiple sources can be made available for use by bona fide parties for their intended purpose.

Key in this is that all privacy requirements are met to safeguard the confidentiality of the customers.

One approach that has been used is anonymising the data, but there also is growing interest in the use of synthetic data. To find out more, Gareth Jones, COO of the Octopus Energy-powered Centre for Net Zero, which leverages data to accelerate the energy transition, gives some insights.

Have you read?
Smart meter technology advances in India
E-REDES completes 6.6 million smart meter rollout in Portugal

One of the Centre’s projects has focussed on the development of a tool for generating synthetic smart meter data, dubbed Faraday, and the creation of a UK nationally representative synthetic smart meter dataset.

Together these have been released as open source in the OpenSynth project through Linux Foundation (LF) Energy.

Let’s start with the background of the initiative

The energy system is changing dramatically, with lots of renewable energy, decentralisation of assets and changing demand profiles. So it's really important to understand more about consumer energy consumption behaviours to plan and optimise for the future and make the transition successful.

A lot of modelling and decision-making today tends to be based on data from the past, and we would like those processes to happen using the most up-to-date data as the nuances, such as bidirectional flows, are so important as we move forward.

But when we're looking at real data, we hit a number of issues, particularly around privacy. Synthetic data has huge potential for resolving some of those privacy issues, whilst getting people access to the data that they need.

So what is synthetic data and how is it created?

We start with the real smart meter data from households and combine it with relevant metadata of various different types, both on the customers and on external factors such as building efficiency, and then use all of that data to train a generative AI model.

We can then use that model to create consumption profiles. For example, we could ask for a consumption profile for a detached house with a high energy efficiency rating, solar panels on the roof and an EV charger in the driveway, and it will output a daily consumption profile with half hourly data points for a household that meets those requirements.

The first version of our Faraday model that we released was based on smart meter data from Octopus Energy customers, consisting of about 350 million data points from about 20,000 households.

The latest release is trained on 1.8 billion data points from 190,000 households and has been trained to be nationally representative, based on data from government sources that indicates what a national representation needs to look like.

How does synthetic data differ from anonymised data?

Anonymised data is real data, but with no connection to a household, or data that has been aggregated in some way.

Synthetic data is fake data that looks like real data. One can add metadata to it, such as low carbon technology ownership or EPC rating and so on.

Of course, one can do that with anonymised data but the challenge comes back to privacy again. The more metadata that is added, the more the privacy of the source data is risked.

With synthetic data, because it looks real but is entirely artificial, the original source can never be identified as its not based on only one source but on all the households in the dataset. As such it is representative.

What are the benefits of synthetic data?

For specific needs such as billing and customer support, obviously real data is needed, but for everything else synthetic data is a great substitute and actually has more benefits as it can be faster to generate without the restrictions around access.

With synthetic data the model can generate multiple, different consumption profiles for the same input, drawing on all the datasets that match those inputs and to a lesser extent those that are close to matching them.

This is important because ultimately every household has a different real profile. We need this because when doing energy planning, we can’t simply use an average but need to understand the behaviours at the individual level to model how to manage them.

Another benefit is because it is generative, one can create one’s own populations of households – and those don’t have to be just of today but what populations might look like in the future.

For example, we can take a population and add say 20% more EVs to generate that future hypothetical scenario, which is something that is hard to do with real data.

The other point is that synthetic data is scalable so we can access entire populations rather than just subsets of them.

What use cases has synthetic data been applied to?

We’ve had a lot of interest in Faraday and so far, although it’s still early days, it is currently being used by a diverse group including researchers, policymakers and system operators.

In addition to the open source release through LF Energy, we also have a variety of external users on our instance of Faraday that we are hosting.

A common use case is to model the grid impacts of different technologies, such as heat pumps and EVs, and how the consumption profiles might change in specific areas.

We have seen it being used for modelling the impact of building new homes in an existing area and how they might lead to constraints on the grid, and for modelling the impacts of smart tariffs and the weather on the grid.

Ultimately the possibilities are endless and we expect to see a diverse range of use cases emerge.

What are the next steps?

The data we have released is probably most useful to researchers who don’t normally have access to smart meter data. We hope with the OpenSynth community that more people will start generating synthetic data and contribute it back to the community and then we will end up with a really diverse data set.

It could be used by utilities, for example, to look at customer profiles that don’t form part of its customer base or in countries where it doesn’t operate.

Companies such as SSEN and UK Power Networks among others have started open data initiatives and the more data that can be made available, the better. But as I’ve mentioned, there are potential privacy challenges with real smart meter data, and it’s unlikely that it could be shared with sufficient metadata to be useful for many modelling scenarios.

And that's what we really like about synthetic data – that we can slice and dice the consumption profiles, and attach different information points.

Since we started this project, we have had a lot of interest and requests for access to the data. Now we are excited to see the benefits to research, policy and infrastructure planning, and the others uses to which the data can be applied – all in support of a successful energy transition.