Daring to share: Realising the potential of interoperable data

The merits of sharing data, for everything from net zero to delivering help to vulnerable customers, is clear to the utilities sector. However, there is still no clear pathway to embedding a culture of data sharing and differences of opinion remain on whether a centralised approach to setting standards for data exchange would accelerate or obfuscate this journey. Data and innovation leaders shared their thoughts with Utility Week.

A report released by the National Preparedness Commission in January looked at lessons from the coronavirus pandemic in relation to data sharing.

It highlighted that wider sharing of data on vulnerable customers would have enabled better delivery of services and quicker identification of those newly in need.

The report suggested that in many cases barriers to sharing data were self-imposed and that organisations across multiple sectors need to adopt philosophy of “dare to share”.

This theme was also explored in a report produced last year by Utility Week, in association with data sharing specialists IOTICS, which looked at ways of achieving a zero boundaries world for industrial data.

The report was based on in-depth interviews with innovation and digital leaders across utilities and highlighted a common appetite to collaborate with peers and other industries on data sharing as well as communicating the value of digitalisation within their own businesses.

However, barriers remain, not least the need to ensure the resilience of critical national infrastructure. The report also identified perceived risks in areas such as legislative and regulatory compliance, data privacy, intellectual property and consumer impact.

It also analysed an ongoing debate about the need for data standardisation and management and the question of who should be driving forward progress on digitalisation and giving legitimacy to data-sharing initiatives.

Opinions on the merits of this kind of framework and to what extent data-sharing standards should be centralised are mixed. Some point to the need for the potential of the market to be unleashed, while others are sceptical this will actually happen without a guiding hand.

This topic was at the heart of a follow-up roundtable with senior industry figures hosted by Utility Week and IOTICS. The following is a summary of the debate.

Do we need a digital spine?

The period between the report launch and the roundtable also saw the publication of the long-awaited Energy Digitalisation Taskforce report, which gave its own perspective on encouraging an open data culture.

Its recommendations included delivering a digital spine solution, described as enabling a minimal layer of operation critical data to be ingested, standardised and shared in near real time. The report also called for new approaches – and entities – around digital governance and urged the Department for Business, Energy & Industrial Strategy (BEIS) to appoint a chief data officer.

However, some attendees at the roundtable expressed concerns about a centralised approach and whether this would ultimately rob utilities of agency and derail innovation.

One framed it in the difference between collaboration and co-operation.

They said: “You can have a common frame of reference, so that we all agree what we will share, where we will share it and agree to a data governance piece. But where you need to move is towards a culture of co-operation – the balance between competition and collaboration. Co-operation means we all say what we are driving at has a common goal, without then saying we all need to be in it to the same level.”

The data expert stressed that utilities should be focussing on “selective sharing” rather than opening up entire datasets, which would inevitably necessitate a framework of governance.

They added: “Trying to virtualize the data and the information in such a way that we can selectively share and do that progressively is the key. When we meet in real life and we progressively share information as trust evolves, as we feel comfortable or as we see benefit to ourselves, we don’t all agree up front to a common level of what we’re going to share.”

Touching on the Taskforce’s suggestion of a digital spine, this participant said: “We need a digitalisation web. We don’t need the core, we need the flourishing of all the individual projects. We need regulation, absolutely, but we should be looking at how we can decentralise it and make it progressive.”

However, a senior figure from an energy company posed the challenge of how the electricity grid could be managed live without these underlying standards.

The said: “This is all well and good when you’re talking about encouraging innovation but when it comes to balancing the system, if things go wrong, there’s a blackout. That has criminal consequences. I don’t see how we can operate without having at least part of that agreed – having that skeleton set down.”

The selective sharing advocate explained their view of a “federated” system where some elements have standards but not everything is bound by them.

They said: “There are intranets and internets. There are pieces we use where we know, this has a standard, this has control. Selective sharing means there will be those edges, the interoperability piece. It’s why for me the [DT Hub’s] information management framework approach is right because it doesn’t say this goes all the way to the ground at the same level. It says there are standards to this point, then we use technologies like semantics to allow interoperability out of this silo and into another. It doesn’t have to be completely decentralised.”

Another participant suggested that a central spine was feasible within sub-sectors and agreed that some form of standardisation was needed when dealing with critical national infrastructure. However, they posed the problem of expanding that spine across multiple organisations.

“When you are going across organisations, market dynamics win out. If you look at the industrial control centres out there, everyone is slowly moving towards an IP infrastructure. Why is that? Because the market dynamics have won out. You need to start sharing and work out what are the market dynamics, what is the winning solution. That’s why, for me, a spine across organisations won’t win.”

Identifying an enforcer

But how does this trust in market dynamics work in heavily regulated sectors such as utilities?

One participant said: “There are three things you need – the plan; who is going to force it and how they are going to force it.

“If we don’t have those three hand in hand, it’s going to be a failure. You can have a brilliant plan but if you don’t come with an enforcer or a way to enforce it, it’s going to go nowhere. Likewise you can have a central data officer at BEIS or something but if they don’t have a plan, it’s going nowhere.”

They added: “If we agree that common standards are necessary for this to work then we must also accept that someone is going to have to impose them. It might sound draconian but this is how we work as an industry. Most of us are regulated. We’re not purely driven by the market. You need a plan, yes, but you also need a way to enforce it.”

A representative of the water sector pointed out that if regulation is going to play a prominent role in creating a world of open boundaries in data-sharing, then it needs to evolve and truly embrace cross-sector working.

“If regulated industries can’t do this then who can? Perhaps we need to look at the different approaches of regulators and if that makes sense. Joint targets across regulators is something that should be considered.”

A senior figure at an electricity network pointed out that one of the problems of debating a digital spine is that it is not clear what this means –  “is it a technology, a framework, standardisation”?

They added: “There’s no way we can have everything centralised in government because it just doesn’t work in reality. That’s because we’re not just talking about multiple organisations but multiple sectors.

“I struggle to see how you could have an over-arching body for digitalisation because it would have to cut across multiple different sectors. How would it operate and interact with different regulators and different systems? Equally we can’t be at the other end of the spectrum where absolutely everyone is doing their own thing because that interoperability doesn’t work without some standardisation. You need that commonality.”

They urged the sector to take the initiative and break down boundaries wherever possible.

“We need to come together at an individual sectoral level and then across sectors to establish where we need to align and standardise. We need the common foundational building blocks that will provide that structure, rigour and alignment that then drives us, as individual organisations and as separate sectors, to develop internally and provision externally an output that is readily useable and interoperable.”

However, they stressed that there still needed to be someone “steering the ship” – who can ultimately make the decision on where organisations align.

“No part of government or regulator seems to be driving this. The Taskforce report, though very welcome, is also rather aspirational and conceptual. Who is taking that last step to drive the change that is needed?”

Meta matters

A live example of the decarbonisation and digitalisation hand in hand is the smart meter rollout and the need to optimise the data they are producing.

Here again, regulatory hurdles were mentioned, with one participant saying: “There’s a real tension around maximizing the utility of the data versus retaining really robust data privacy processes. How can you get down to the lowest possible granularity of the data set but making sure you’re not impinging on consumer rights?”

An energy networks representative said smart meter data represents a huge opportunity but added there was a common level of disappointment among distribution network operators that they are forced to aggregate the data to the level they do at present.

“All the networks have put in place their plans for how they’re going to hold and utilise smart meter data. But the fact we have to aggregate and we have to protect that privacy aspect to the level we do means there are certain use cases that are somewhat more comprised than we would like.

“We’ve got lots of players in this eco-system, all with different perspectives and priorities. When it comes to commercialisation vs security, it’s not about right or wrong, they’re just different. And that’s what we have to navigate through with the open data piece.”

While they conceded regulation remains a blocker in many respects they noted an improved appreciation of data among regulators.

They added: “We have come a long way in understanding the what and the why, we now need to concentrate on the how. We all seem to be waiting for someone else to come up with the answer. Culture is the thing that needs to change.”

One participant pointed to the value of the meta data surrounding the smart metering system, which provides visibility of which messages go where and when.

“There’s still a huge amount of utility in that dataset in itself. This a GB wide dataset growing by the day and there are various use cases that data could potentially unlock, around identifying consumer behaviour and profiling. There’s also role it can play in energy system performance and visibility.”

They concluded that while robust and strict standards are needed around some elements of data sharing, there are “focal points” where collaboration can unlock value without the need for a centralised standard.

A data expert agreed that data and meta data are often conflated and that “often it means something very specific to someone very specific in a very specific way and in some cases that can be handled at a meta data level without needing all of the data standardisation”.

Building a digital culture

Returning to the risk of paralysis if all actors in the digital eco-system wait for someone else to make a move, there was agreement that the industry needs to be bolder.

Several utilities firms highlighted projects currently underway that were underpinned by the need to find ways of securely and responsibly sharing data, with one describing this as “carving our own route through the quagmire that is data sharing”. The point was made several times that to legitimise this approach a clear case has to be made for the end goal of data sharing, be that to further decarbonisation or to improve service for customers.

There is also a need to ensure the benefits of digitalisation and open data are fully understand within organisations.

One network operator said: “There’s a huge gap between buying a new system and shoving it into an organisation and saying we’re digital now and developing that broad business knowledge, that culture of understanding right throughout the organisation of how everyone fits in.

“It’s fair to say we’re all at different stages on the journey to a digital culture and making sure people feel empowered. There’s a huge potential for people to feel their job is being automated. I can’t say loud enough that digitalising things isn’t automating them. It’s taking away the boring stuff and adding commonality and standardisation to get them to be doing what they’re really good at.

“I’m very focused on the people before the process and that all comes down to culture and how people feel the value and the need and want to come along on that journey rather than pushing a business along to digitalising rather pulling along with it.”

Another participant agreed that digitalisation remains an abstract concept for many – whether internally or by customers and other stakeholders.

They added: “Digitalisation is seen as a  noun rather than a verb. We talk about digitalisation as a conceptual thing and that concept is very ethereal to a lot of the people that are actually at the sharp end. Sometimes there’s a lot of fear and scepticism. It’s about what is the detrimental impact, not what is the benefit to me, how does it make my life better, easier or provide me with a better service or product.

“We have got a long way to go on maturity and understanding and effective implementation of digitalisation. It comes down to technologists and those delivering digitalisation having a better understanding of the stakeholders, the customers, those impacted and making sure we’re communicating properly. We shouldn’t be talking about digitalisation itself, we should be talking about what it delivers and keeping the technical conversations at the backend.

“For me there’s a transition to be made to stop talking about IT projects as opposed to the true transformational aspects of digitalisation.”

Summing up the debate, Ali Nicholl, head of engagement at IOTICS, said a key question was “how we marshall the pull and the push”.

He added: “Sustainability and net zero are providing a very clear pull, both in this country and globally.

“But where is the push? Not just the plan but the authoritative push.

“What is encouraging is that there is a clear recognition this is inherently progressive. No one is expecting a solution to arrive fully formed tomorrow and no one is expecting one software or actor can achieve it alone.

“If we can have the pull from the value and the push from the regulatory elements along with the recognition that the plan has to be progressive and co-operative, then we can just start and make real progress and make a real difference.”