Conquering complexity: utilities’ digital challenge

Can utilities overcome the challenges posed by mounting digital complexity as they race to build the smart infrastructure of the future? Industry experts share their views ahead of appearances at Utility Week Live in May.

Digital transformation is the key to faster, more flexible and efficient utilities businesses. Access to bigger, better data can help create more resilient asset bases, capable of dynamic and efficient operations and optimise constrained resources, resulting in more sustainable operations, better services for customers, improved profitability, and new routes to market.

But tapping into intelligent infrastructure and all that rich data brings with it high levels of complexity. Recent research from Utility Week revealed that, second to the threat of a serious cyber breach, an inability to manage an increasingly intricate network of operational and information based technologies is the biggest digital/security risk likely to impact utilities in the next 5-10 years.

Established players in the energy and water sectors have extensive legacy assets, systems and processes, including fragmented IT application infrastructures, that present barriers when scaling up and planning improvements.

In parallel, new expectations and demands, such as tackling storm overflows, leakage, or mobilising for net-zero means rapidly developing capabilities and wrestling with entirely new systems and ways of working to boost network control, monitoring and visibility.

In the absence of a single controlling mind, cross-sector collaboration is required to organise and align data standards and governance, and educate and upskill the workforce to ensure a consistent approach.

“There’s no clear blueprint of what a fully synchronised intelligent energy system will look like in future so we’re on a voyage of discovery,” says Matt Webb, head of enterprise data management at UK Power Networks. “When you’ve got such a complex legacy system, and growing complexity in terms of all the people involved and the types of technologies that need to communicate and be orchestrated, it’s a real challenge.”

Discovery time

Utilities are in the early stages of digital transformation, but now recognise the imperative to upgrade networks and systems to boost productivity and respond to pressing customer, environmental and regulatory demands.

Research by professional services company Accenture found that during the pandemic utilities’ adoption of certain technologies was above the global average, with the majority of investments going into the Internet of Things (65%), streaming data (56%), edge computing (54%), bottom-up AI (54%), and trust-based architecture (53%).

The escalating roll out of IoT sensors by the water industry, is producing data at a much higher rate and a finer granularity than previously possible, which alongside the imminent roll out of tech like smart metering, will trigger a massive influx of information. Businesses are grappling with the task of how to manage that torrent and sweat it to maximise the benefits.

SES Water is running a proof of concept trial of a new data platform created to channel selected data into a ‘data lake’, where the firm can carry out analysis in combination with data from other sources.

Dilani Pararajasingam, head of data engineering at SES Water tells Utility Week: “We’re figuring out what the process is in terms of creating the data pipelines and refresh times etc. The other part is the modelling work, looking at the relationships between datasets, and what value we could add by combining them to gain more insights for specific needs and use cases.”

The flow of information and data in the energy sector has traditionally been unidirectional between a few large organisations, including the system operator and distribution network operators.

Looking ahead, the widespread expansion of local energy generation, needed to balance supply and demand and accelerate the path to net zero places a heavy emphasis on collating and managing data from multiple providers and sources including solar panels, heat pumps and electric vehicles.

Webb comments: “The active network management piece is really key, our main role in decarbonisation is to facilitate the connection of low carbon technologies, we cannot be a blocker. As all those complexities come together we have to navigate through it.”

DNOs are already transitioning to a Distribution System Operator model, which among other things will bring greater visibility and control to the low voltage end of the network and require companies to collaborate to ensure interoperability across systems, processes and data.

But potential for disconnected data grows without appropriate standards and interoperability to communicate effectively. The water industry lags behind on data standards with none yet published, “Everyone’s quite good at looking after their own data, it’s when you want to combine data that these kinds of things need to be in play,” says Pararajasingam.

Energy companies are coming together through organisations like the Energy Networks Association, in a drive to understand the common enablers, foundational components and standards needed to align on to support future investments in systems and data.

According to Paul Linnane, director of data and development at Electralink, the operator of Britain’s central energy data transfer function, an important goal is to achieve a balance between enabling technology innovation and creating standards needed to build a resilient network. “Interoperability standards will enable organisations to invest with certainty when building interoperating systems to support the network,” he tells UW. “However, producing standards too early can add significant delay and cost to innovation projects without any benefit. Balancing these elements to enable innovation in the early stages, but providing certainty in the later stages of the process, is key.”

Electralink’s Paul Linnane will speak about whole system challenges for building out digital infrastructure at utility Week Live on 17 May. Full programme details can be found here.

Data best practice guidance, written principally by the Energy Systems Catapult, and now being taken forward by Ofgem is expected to provide more direction on areas such as data licensing, metadata, standards, and data modelling. The regulator’s rules are built into licence conditions for the current price control period, ED-2, which runs to March 2028.

Ensuring data is managed appropriately and securely on networks is also contingent on gaining the requisite funding for digital infrastructure, says Linnane, who also underlines the importance of machine learning and ElectraLink’s Project FAIR, which helps collate data from many data providers and sources into a single dataset.

Clean start

The emerging fields of data science and AI have shown huge potential to generate insights but proofs of concept have often faced issues scaling up due to the high degree of manual data cleansing required before algorithms can be set to work.

“Often you have highly-skilled SMEs, such as process modellers, employed to do analysis, but they can spend over 80% of their time collecting data and data wrangling and cleansing before they even get to start on the value-add of modelling,” explains Jethro Yates head of data science and engineering at Thames Water.

According to Yates, a smarter approach means separating out roles and responsibilities and employing teams of dedicated data engineers and platform engineers to carry out data acquisition and cleansing ready for SMEs and data scientists to pick up the baton and focus on what they do best.

Thames Water’s Jethro Yates will speak about smart water networks at Utility Week Live on 17 May. Full Programme details can be found here.

“Shiny, exciting models and machine learning represent the tip of the pyramid, there’s a whole base of work that needs to happen before they can function effectively,” says Yates.

The data cleansing challenge highlights the additional skills and third party expertise needed by energy and water companies as they venture deeper into digital transformation.

A recent report by the Energy Systems Catapult found energy companies are struggling to recruit people with the skills to match a data-led future. Some 40% of companies surveyed described difficulties hiring data scientists with the requisite skills. The report points out that sectors such as FinTech and social media have been more proactive in recruiting the most skilled candidates.

“Other sectors have been ahead of the water sector on recruitment, we’re learning we need a substantial amount of new skills across the industry and new types of partnerships with third parties to bring in expertise,” says Yates.

Becoming ‘data-ready’ also demands culture change and upskilling employees with new levels of data competency so they understand and categorise data in the same way to build efficiencies.

Pararajasingam says SES Water is planning to launch data literacy workshops to get people on board with using the same terms for data across different areas of the business so that everyone’s on the same page.

Gated community?

In a world where open data is now an expectation – recognised as helping speed innovation within and between companies and acting as a vital enabler for net zero – utilities come up against the conundrum of how to manage complexity on their networks without putting constraints on certain data flows and sharing.

The very concept of open data seems to run counter to the notion of keeping a degree of control over data and putting data behind a firewall to ensure companies don’t fall foul of any issues.

Pioneering open data projects, including Project Stream for the water sector and a ‘digital spine’ across the energy system and working through the nuts and bolts of how to make it work.

UKPN is pushing out a variety of open data through a new portal launched in 2021, which Webb claims has actually managed to reduce the firm’s risk profile rather than increase it.

The portal and associated governance framework focuses on data in the public realm and exploits a data triage process used to understand the potential limitations and risks of publication and how to mitigate them.

“That improved understanding, control, and coordination, including a greater focus on the potential downsides of sharing data, enabled us to take those risks away, whilst also providing a better service and access to data by the end consumer,” says Webb.

Moving forward UKPN plans to not only respond to prevailing customer wants, needs and demands for data, but also be more proactive and exploratory, pushing out datasets there is perhaps no clear requirement or request for, but which it thinks might provide value to a certain community.

As more complex digital infrastructure blurs the boundaries between companies, so it creates new opportunities for hackers to target weaknesses in the supply chain. Given water and energy companies role operating critical infrastructure and handling sensitive customer data, it’s a ball they cannot afford to drop.

The Government’s National Cyber Security Centre (NCSC) has been working closely with utilities to advise on what cyber security best practice looks like, for systems such as internet of things (IoT) devices, and how it should be incorporated from design through to installation.

Lessons can be learnt from robust security measures implemented on the UK’s smart meter network by the Data Communications Company (DCC), an organisation set up by the Government to manage smart meter communications on behalf of suppliers.

The highly-secure digital spine wraps home energy use data using two layers of encryption and broadcasts it on a wide area network that operates entirely separately from the public internet to security levels endorsed by the NCSC. DCC, which operates and monitors the network, can’t see or hold the data, it merely transports it directly to energy suppliers.

According to Silvano Sogus, director of security architecture and assurance at DCC, the biggest challenge as digital systems become more complex is not only making sure they are protected against adversarial threats, but also against technological evolution.

“Our infrastructure needs to be flexible enough to accommodate growing demand for our existing services but also to adopt entirely new paradigms,” says Silvano. “For example, quantum-resistant cryptography [using new ciphers that can resist quantum computers’ ability to attack conventional cryptography] will be an important topic for us and we are already looking into how we will be able to adapt to new algorithms and integrate them in our systems.”

DCC’s Silvano Sogus will speak about cyber security risks in the utilities sector at Utility Week Live on 16 May. Full programme details can be found here.

Utilities operate in a rapidly evolving ecosystem, both from a technological standpoint, and in terms of changing stakeholders, market participants and consumer expectation and demand, all set in the broader context of an energy crisis and environmental breakdown. Efforts to build their capacity to manage and organise data will put them in a stronger position to  tackle these challenges in the months and years ahead.

Utility Week Live takes place at the Birmingham NEC from 16-17 May. Register to attend.