A first-ever cyberattack against U.S. wind and solar assets in 2019 opened a new front in the ongoing battle to ensure the security of the nation's power grid.
More recent incidents involving multinational energy company Enel as well as a natural gas compressor station also show a growing threat to the energy sector.
To reduce the risks, companies are examining their security culture and turning to new approaches like machine learning, while the Trump Administration is working to address threats from China, Russia and other countries.
The following trendline examines various threats, responses and challenges in the ever expanding "cat and mouse game" of cybersecurity.
IoT cyber bill clears Congress — what's next for industry players?
Long awaited legislation is seen as a springboard to widespread adoption of standards across the booming connected devices industry.
By: David Jones
A multiyear effort to raise cybersecurity standards in the IoT industry is on the horizon following passage of a comprehensive bill last month that will establish uniform standards for companies operating in the federal market.
Backers of the bill have sought to use the federal cybersecurity baseline to impact the broader enterprise and consumer markets for IoT devices, in a similar way that the EnergyStar rating has impacted wider energy efficiency standards.
"While more and more products and even household appliances today have software functionality and internet connectivity, too few incorporate even basic safeguards and protections, posing a real risk to individual and national security," Sen. Mark Warner, D-VA, said in a statement. "The IoT Cybersecurity Improvement Act — now to be law — leverages the purchasing power of the federal government to establish some minimum security standards for IoT devices."
President Donald Trump is expected to sign the bill, which was passed by the Republican-controlled Senate by unanimous consent. The bill, which has been on his desk since early last week, would take effect this week if he does not sign absent a veto.
"Key industry stakeholders have been moving in the right direction since our last attempt to pass this bill in the 115th Congress, so I think there's a general trend towards better security for these devices," Warner said. "But to ensure that the market for these devices continues to move in that direction, having the purchasing power of the federal government steering it will certainly help."
The technology behind IoT devices has been around for decades. IoT technology is widely used in major industries to control access points and manage other systems.
Industrial control systems have been around since the 1960's and are used in everything from building control systems to power generation facilities, warehouse management systems and healthcare, according to Bill Malik, vice president of infrastructure strategies at Trend Micro. More recently the technology has been used to develop automated thermostats, connected vehicles and home security devices.
The 2016 Mirai botnet attacks exposed the vulnerabilities of IoT devices, leading to massive disruption of internet service in the U.S. Closed circuit cameras lacking the most basic protections, including passwords, security controls and the ability to patch defects.
"Botnets have been targeting low hanging fruit, like consumer IoT devices and exploiting weak security features," said Gonda Lamberink, senior business development manager at UL's Identity Management & Security division. "Once compromised, IoT devices are misused to send huge volumes of data to all types of internet sites and services."
Warner and Sen. Cory Gardner, R-CO, co-founders of the Senate Cybersecurity Caucus, introduced the bill in 2017 and later pushed the legislation through Congress with backing from Reps. Will Hurd, R-TX, and Robin Kelly, D-IL, in the House.
The federal bill calls for the National Institute of Standards and Technology to issue recommendations, at minimum, on secure development, identity management, patching and configuration of IoT devices. The Office of Management and Budget would set guidelines for each agency consistent with the NIST recommendations.
NIST will also work with the Department of Homeland Security and industry experts on guidelines for vulnerability disclosure. Contractors and vendors providing information systems to the federal government would be required to create coordinated vulnerability disclosure policies.
IoT and cybersecurity experts differ on how much this legislation will influence the enterprise and consumer markets. Some experts say the legislation will help drive the private sector to harmonize standards in a way similar to the use of green building technologies.
"This can have the impact of bringing about a de facto national security baseline or standard for IoT devices that the private sector is already working towards and can be built upon further," Lamberink said.
The most established and mature companies often took their products to third-party certification labs for penetration testing and reviews of various design, manufacturing and life cycle processes, according to Sequitur Labs, a Fall City, Washington-based firm that develops security technology for IoT devices.
"At the end of the day a lot of it comes down to the risk or liability associated with the product, or loss of brand or the risk to the customer," said Philip Attfield, co-founder and CEO of Sequitur Labs.
The large system integrators already in the federal space will likely be the first to comply, and will push the new standards down the supply chain, he said. The impact on IoT products in the consumer space will be far more limited.
Despite widespread optimism, others are more cautious on the immediate impact of the legislation, as market pressures may lead some companies to parse out their product inventory to meet the new federal regulations, while maintaining a more competitively priced offering for non-government business.
"Vendors to the U.S. government will create separate SKUs for devices intended to comply [with] these rules," Malik said. Consumers and enterprises may be willing to pay for the higher prices versions if they want, but will typically buy the less secure, lower cost choice, he said.
Article top image credit: Brian Tucker/Utility Dive
Utilities one of most vulnerable industries as cybersecurity IT/OT threats merge, survey finds
Not only is operational technology connected to the internet now, cyberattacks can trickle through information technology environments.
By: Samantha Ann Schwartz
Data protection isn't often discussed in the context of operational technology. Data breaches are typically incidents ininformation technology, yet OT holds just as many trade secrets and critical information — it just looks different.
Consider a pharmaceutical plant: Within operations, a controller directs a valve attached to pipe carrying fluid. Attached to the valve is a sensor responsible for the fluid's flow rate. "That flow rate is then sent back to the controller where it is compared to the output that has been programmed by the engineers," said Brian Kime, senior analyst at Forrester, while speaking at a virtual Forrester event in September.
"If there is a difference, then the controller then changes the valve position to match the design flow rate," said Kime. If a control or sensor is manipulated through a cyberattack, the product OT helps create is at best useless, and at worst, dangerous.
This July, the Department of Homeland Security's Cybersecurity and Infrastructure Security Agencyissued a cybersecurity alert to operators of critical infrastructure. Recommendations include disconnecting any operational systems that do not need internet connectivity for safe and reliable operations, and planning for "continued manual process operations" should industrial control systems become unavailable.
OT environments were once islands, seldom connected to the internet. Though industries are still struggling to accept it, OT came online when the digital world interrupted the physical world and perceived air gaps between IT and OT closed.
There's a disconnect between assumed security policies and "operational reality," said Chris Hallenbeck, CISO for the Americas at Tanium. Policies might falsely claim IT and OT environments don't talk to each other, yet data derived from IT "has come out from the OT environment at some level to facilitate the business."
Industries are dealing with cyberattacks filtering through IT into OT. Yet "there's strikingly low visibility into those environments," said Kime, in an interview. "When you talk about municipal water systems, they're already stretched thin, and I can't imagine that they know who's in their network."
IT/OT professionals consider manufacturing, electric utilities and building management systems the most vulnerable U.S. industries, according to a Claroty survey of 1,100 respondents.
And yet, historically the security of OT systems in a manufacturing setting was relegated to manufacturing maintenance teams, according to EY. When a manufacturer can't protect its processes, it can overflow into its products.
Wam Voster, senior director analyst at Gartner, said clients come to him wanting to make changes but are strapped by their equipment manufacturer and don't know how to proceed. "That is a ridiculous situation" and one of the problems with equipment manufacturers, he said.
Cyber goes physical
OT cyber vulnerabilities were proven exploitable with Stuxnet in 2010. The computer worm made its way into an Iranian uranium enrichment facility, one of the first known cases of a virus spreading through industrial systems.
Stuxnet's code was fashioned to search for programmable logic controllers (PLC) by Siemens. Once a PLC is found, the worm instigated a series of commands that effectively caused centrifuges within the Iranian facility to "burn themselves out," according to McAfee.
Researchers and adversaries learned from Stuxnet's code. Before, for an attacker tocause an operational meltdown through malware, they would have to be on-premise, according to Atul Vashistha, vice chair of the Business Board at the Department of Defense and chairman of Supply Wisdom.
"I'm extremely, extremely concerned when we think about critical infrastructure," said Vashistha. While "I think our country's cyber capabilities are significantly enhanced than they ever used to be .... We're going to see a lot more attacks not just on corporations but actually leveraging OT."
With COVID-19 as the backdrop to increased cyberthreats, 65% of U.S. IT/OT security professionals say their IT and OT networks became more interconnected, according to Claroty. While IT and OT converge, 62% of global security professionals say the networks are not equally secure.
Ransomware counts on organizations neglecting network detection and response (NDR) sensors between East-West data flows (traffic between servers within a company's network, including between internal and OT environments), according to a report from IronNet. Instead, North-South data flows (traffic entering and leaving a network) is often "heavily defended" with updated software and firmware patches and NDR solutions.
"NDR sensors are sometimes not afforded direct visibility into internal customer environments, due to restricted placements at North-South ingress/egress boundary points," said IronNet. But in a study of high-profile ransomware — Maze, NetWalker, Ruk, REvil, WastedLocker and Snake — didn't "produce much" in North-South data pathways. "From an NDR perspective, that's a little disheartening," according to the report.
"As our ransomware variants did access the Win10 remote fileshare, there was a boatload of East-West data, almost all of it via SMB over IPv6," said IronNet. While studying Maze, IronNet found the malware running through East-West traffic, out of sight from "most intelligent firewall and network security appliances" focused on North-South flows.
Even if rules are built into IT/OT data communication, "you almost have to build your world with the assumption that despite all of the rules, it's still going to happen," where someone intentionally or unintentionally connects systems they aren't supposed to, said Hallenbeck. And because of that, additional detection solutions are needed.
Separation of powers
Interlocked IT and OT is sometimes more efficient, even with the added security risks. Traditional security is focused on integrity and confidentiality of data, typically prioritized over availability, said Kime. But in an OT environment, availability supersedes confidentiality to maintain safety.
According to the Claroty survey, 69% of IT and OT security professionals say a cyberattack on critical infrastructure can cause more damage than an enterprise data breach. Only 31% respondents say an enterprise data breach is more harmful.
"Many of the companies that are really critical infrastructure, they're not necessarily considered 'the risk,'" said Vashistha. "What you are talking about is not about theft of consumer data, you're actually talking about outages."
And unlike data breaches, "so many organizations will not tell anybody that this has happened," said Voster, referring to ransomware attacks impacting OT. About 41% of ransomware attacks in 2020 targeted organizations with OT, according to IBM's Security X-Force. And there's only a requirement for national critical infrastructure to report these incidents.
Consider thePurdue Model, or the structure of OT engineering leading all the way from basic IT-like web servers to the physical process in need of securing. Endpoint detection or protection is not necessarily for OT.
"Things like intrusion detection systems that do deep packet inspection of all the network traffic can introduce latency," said Kime. "Endpoint detection and response tools are great in IT, but they are very burdensome in OT, because when they do detect something, they take a lot of CPU cycles" and send data to the cloud for analysis.
When cyberthreats transverse the IT/OT threshold of the Purdue model, hackers can explore OT networks for vulnerabilities. "It's not as simple as finding a standard database in your IT network, and then using a commodity exploit against it. All the capabilities used in ICS [industrial control systems] attacks are heavily researched by the threat and takes them a long time to execute their plans," said Kime.
A common misconception is that IoT or ICS having IP addresses make them equals, said Kime. The reason? "Ignorance."
Consider a safety instrumented system coming from the same manufacturer as a basic security camera. If a safety instrumented system is triggered after something crosses safety thresholds, "you definitely do not want to treat an IP camera, like a safety system," said Kime. With medical or any kind of factory, "we're seeing in ransomware targeting the 'IP-ish' type of systems that sit on the OT side of the enterprise, but they don't directly control the processes."
By: Jason Christopher, Dragos Principal Cyber Risk Advisor
The electric utility industry is a valuable target for adversaries seeking to exploit industrial control systems (ICS) and operations technology (OT) for a variety of purposes. A power disruption event from a cyberattack can occur from multiple components of an electric system including disruptions of the operational systems used for situational awareness and energy trading, targeting enterprise environments to achieve an enabling attack through interconnected and interdependent IT systems, or through a direct compromise of cyber digital assets used within OT environments. Attacks on electric systems – like attacks on other critical infrastructure sectors – can further an adversary’s criminal, political, economic, or geopolitical goals. As adversaries and their sponsors invest more effort and money into obtaining effects-focused capabilities, the risk of a disruptive or destructive attack on the electric sector significantly increases.
The number of publicly known attacks impacting ICS environments around the world continues to increase, and correspondingly the potential risk due to a disruptive cyber event impacting the North American electric sector is currently assessed as high. This report highlights multiple threats and adversaries focusing on critical infrastructure and their capabilities. Dragos anticipates the threat landscape associated with the sector will remain high as the detected intrusions continue to rise.
Of the activity groups that Dragos is actively tracking, nearly two-thirds of the groups performing ICS specific targeting and disruption activities are focused on the North American electric sector. Additionally, existing threats to ICS are expanding and establishing new interest in electric utility operations in North America. For example, the Dragos tracked activity group XENOTIME – the most dangerous and capable activity group – initially focused its targeting efforts on oil and gas operations before expanding to include North American electric utilities. Dragos also identified the MAGNALLIUM activity group expanding targeting to include electric utilities in the US. This activity group expansion and shift to the electric sector coincided with increasing political and military tensions in Gulf Coast Countries (GCC).
Dragos research of the CRASHOVERRIDE attack indicates ELECTRUM targeted recovery operations. Such activity, if successful, could prolong outages following a cyberattack and cause physical damage to equipment or harm to operators. These findings suggest the group had greater ambitions than what it achieved during its 2016 attack, and represent worrying possibilities for safety and protection-focused attacks in the future.
Historically, adversaries have demonstrated the capabilities to significantly disrupt electric operations in largescale cyber events through specialized malware and deep knowledge of targets’ operations environments. Although North America has not experienced similar attacks, ICS-targeting adversaries exhibit the interest and ability to target such networks with activities that could facilitate such attacks.
The electric sector, as a whole, has been working for over a decade to address cyber threats through board level decisions, 1 preparedness exercises like GridEx, the NERC CIP standards, and direct investment in ICS specific security technologies. However, adversaries will continue to evolve and the industry must be ready to adapt
Article top image credit: Permission granted by Dragos
Utilities face growing ransomware threat as hackers improve strategy, execution
The energy industry faces an increasing threat from financially-motivated hackers. Experts say more companies are paying, and ransoms are rising.
By: Robert Walton
The utility sector, including electric, gas and water companies, has a growing ransomware problem, say security experts.
Ransomware is a type of financially-motivated malware, which steals or locks up a company's data or computing systems until the victim pays a fee to the hacker. And it is quickly becoming more common across the economy, as deploying the malware becomes easier and hackers learn to right-size their demands.
"It's a volume business. A volume, low-margin business," said Miles Keogh, executive director of the National Association of Clean Air Agencies. He advised state public utility commissions on cybersecurity issues until 2017.
While there have been a few high-profile cases where companies paid large ransoms to recover their data or systems (Garmin reportedly paid millions this summer), experts say more frequently smaller cases are resolved quietly, including some utilities.
"As you can imagine, a lot of organizations, if they pay, they keep it under cover. It's difficult to understand if a victim paid or did not pay — but we are seeing an increase in the number of victims."
Chief Information Security Officer, Cybereason
Victims of ransomware are reticent to talk. The Reading Municipal Light Department in Massachusetts acknowledged earlier this year it had been hit by ransomware. The utility declined to discuss the attack, saying in a statement “the RMLD implements best utility practices.”
The Lansing Board of Water & Light in Michigan in 2016 paid a $25,000 ransom to unlock some of its communication systems. The utility declined to comment on the incident.
"As you can imagine, a lot of organizations, if they pay, they keep it under cover. It's difficult to understand if a victim paid or did not pay — but we are seeing an increase in the number of victims," said Israel Barak, chief information security officer at Cybereason, a Boston-based security firm, referring to utilities and other companies.
Attacks are on the rise — and so are ransom demands
The Edison Electric Institute, which represents investor-owned utilities, said it has seen "an uptick in attempted attacks" in part related to the COVID-19 pandemic, but added that its members are "prepared to mitigate and manage the extra risks." The group did not address whether ransomware costs could be passed on to consumers.
The National Rural Electric Cooperative Association did not address ransomware questions but said generally that its members "remain vigilant against cyber threats and those who might perpetuate them."
Cybereason in June warned utility ransomware attacks were becoming more sophisticated. The company ran a "honeypot" operation where it created a fake industrial control network designed to look like an electricity company — and watched as hackers broke in within three days.
In particular, Barak said he sees an increase in complicated, multi-stage ransomware attacks that can paralyze entire networks and enterprises while simpler attacks focused on individuals are declining.
"We went a long time without having any power companies here attacked, even though there were cyber attacks of a nation-state origin happening between countries."
Executive Director, National Association of Clean Air Agencies.
"For the last two to three years, ransomware has been at the top of everyone's list of threats companies are facing," said Bob Parisi, U.S. cyber product leader for Marsh, a global insurance broker and risk adviser.
Parisi helps customers, including utilities, procure insurance plans that cover cybersecurity and data extortion.
"What we've seen in the last couple of years is that ransomware is more prevalent and more damaging" than other forms of hacking, said Parisi.
And, Parisi said, the average ransom payment he sees is up 30% over the last year, to about $110,000. But more ransoms, paid without the help of an insurance company, are likely much smaller.
While EEI has seen an increase in attacks, it's not clear how many utilities have been hit by ransomware. Parisi said of companies suffering a ransomware attack in 2020, as many as 10% — or as few as 2% — may have been utilities.
"Utilities are right there at the forefront of risk," said Parisi. "No one is immune, largely because it's a fairly effective exploit."
There have been relatively few hacks in the U.S. utility sector that caused significant damage — so far. But ransomware threatens to change that, said Keogh.
Nation-state actors have had these kinds of hacking capabilities — including the ability to disrupt power grids — for years, said Keogh. But because the United States generally had stronger cyber capabilities, the attacks were rarely attempted.
"If you flipped the switch here, you'd get a non-proportional response that would be really bad," he said. But with financial gain rather than disruption as the primary goal of ransomware, the number of potential bad actors has suddenly exploded.
"The people who could screw with critical infrastructure, especially the power system, were not motivated to do it because it was mostly nation-states or folks aligned with nation-states who were kind of disavowable," said Keogh. "We went a long time without having any power companies here attacked, even though there were cyber attacks of a nation-state origin happening between countries."
"The hope is that if the worst does happen and a utility ... gets ransomed, they have protections in place to recover without having to pay the bad guys."
Vice President of Security Solutions, Fortress Information Security
Ransomware, however, "aligns the means and motives," Keogh said. "It's not motivated by geopolitics ... it's literally motivated by folks who just want to get paid out."
And now, attacks on companies are happening all the time. And when a hacker is successful, the demand — depending on the victim — can be as low as a thousand or a few hundred dollars. For most companies of any size, paying the ransom is cheaper than lost productivity and output.
"Everyone says don't pay the ransom," Keogh said. But he also said that depending on the business and what type of impact the malware is having, "I can imagine the temptation to just pay the ransom. That pressure has to be enormous."
Parisi said the percentage of victims actually paying ransoms has "gone up steadily in the last two years." It was about 40% of companies hit by ransomware in 2018, rising to 45% in 2019 and now hovering around 60% in 2020.
Avoiding or mitigating a ransomware attack
Training staff in good cyber hygiene is an important way to avoid being the victim of an attack, said security experts.
"The way these folks get in is almost always phishing. It's the stupidest attack in the world. We've all known about it for decades," said Keogh.
If a utility does fall victim to an attack, a speedy and affordable recovery usually involves extensive backups of data and operating systems, according to those who help companies resume operations after ransomware.
"The hope is that if the worst does happen and a utility ... gets ransomed, they have protections in place to recover without having to pay the bad guys," said Tony Turner, vice president of security solutions at Fortress Information Security.
Those "protections" are essentially system and data backups, said Turner. They are vital because there is no guarantee that hackers will — or even can — unlock a company's data. And if they do, there's nothing that says they wont simply re-encrypt it and issue another ransom demand.
"The risk to the public of not having power or water, and the impact it can have if it happens for a sustained amount of time, means utilities should take [ransomware] more seriously — and a lot of times will be forced to pay."
Chief Cloud Officer and Chief Operating Officer, Avant
"From a best practice standpoint, utilities should be pulling ransomware into business continuity and recovery plans," he said. Much like natural disasters, including storms, flooding and wildfires, "a ransomware event is no less and perhaps even more impactful for operations."
Utilities "should be doing backups for everything required for operations," said Turner.
That said, Turner acknowledges that extensive backups and the ability to restore systems "is a more mature capability that most utilities are not yet planning for. They are more focused on natural disaster-type threats, and not as focused on the cyber threat as they could be."
Call in the FBI
Ron Hayman, chief cloud officer and chief operating officer at Avant, a communications technology company, says his firm has seen "significant spikes in ransomware," and often victims don't have the needed backups.
"They're typically going to have to pay the ransom," Hayman said. "With utilities, the risk is obviously significantly higher. The risk to the public of not having power or water, and the impact it can have if it happens for a sustained amount of time, means utilities should take it more seriously — and a lot of times will be forced to pay."
"Typically that [payment] is passed on to the customer, in terms of increased costs," said Hayman.
In the event of a ransomware attack, the first line of action is to reach out to the local Federal Bureau of Investigation (FBI) office, said Turner. "Ideally, a utility already has the local FBI office on speed dial. The FBI has a lot of tools and capabilities at their disposal," and can sometimes quickly decrypt a company's systems.
Working with the FBI or employing other decryption tools can unlock files and data, but "maybe paying the ransom is the last-ditch effort. Before paying, there should be some transition planning in place. ... It's likely the company pays, gets access to files back, and then gets ransomed again, if they got caught once and didn't do anything to keep it from happening again," said Turner.
If a utility determines to pay the ransom in return for the data, they often utilize an insurance company or other specialized third party to pay. Most ransomware operators take cryptocurrency, said Barak.
"Once they've decided it is a viable option, the question becomes how to negotiate with the attacker," said Barak. In more complex attacks, the hackers have set up a kind of "help desk" that allows for communication and "most are very open to negotiation."
"They want a good payday, but they're willing to talk about what that payday is," said Barak. In some instances, companies can negotiate to return access to only critical systems in return for a smaller ransom.
Barak also cautioned that paying the ransom is no guarantee the data will be returned. And there are also "moral aspects" to the decision, he said, of giving money to criminal enterprises whose activities may stretch beyond hacking.
"At the end of the day, it's very difficult to make those decisions," he said.
Correction: A previous version of this story misidentified the National Association of Clean Air Agencies.
Article top image credit: Getty
Trump's grid security order sows confusion in power sector — but don't expect a quick fix
The executive order limits the installation of bulk power system equipment sourced from foreign adversaries, but experts say the vague action has the industry 'freaked out.'
By: Robert Walton
In May, President Donald Trump issued an executive order blocking the installation of bulk power system (BPS) equipment sourced from adversaries of the United States. Grid security is a major concern, but experts say the White House order is vague — and the process to translate it into enforceable regulations could stretch into next year.
In the meantime, the power industry is struggling to respond. Some companies have made changes to their procurement processes, but no one knows yet what equipment is safe and what will be prohibited.
"We're already seeing utilities limiting procurement," Tobias Whitney, vice president of energy security solutions at Fortress Information Security, told Utility Dive.
Some of those actions may be premature, according to U.S. Department of Energy officials. The executive order is not an immediate "rip and replace" directive, Michael Coe, director for operational modeling and technical assistance at the agency's Office of Electricity, told stakeholders during a May 21 phone call discussing the order.
"As of today, no equipment is prohibited," Coe said. "Any immediate steps would not only be premature but may be unnecessary."
"The industry is kind of freaked out — and rightfully so."
Partner, Squire Patton Boggs
For already-installed grid equipment that is found to be problematic, DOE is considering how to allow it to remain in place. The agency "will be considering procedures for mitigation measures that may allow for the use of equipment that would otherwise be prohibited," Coe said.
Still, the industry is anxious for more information. And observers say the DOE's stakeholder outreach has so far not been specific enough to quell concerns. The order does not mention specific countries from which equipment could be prohibited — though experts say the new rules will largely impact Chinese companies.
"The industry is kind of freaked out — and rightfully so," Keith Bradley, a partner at Squire Patton Boggs, told Utility Dive.
Bradley says the power sector is aware China is a security concern, and the North American Electric Reliability Corp. (NERC) is working to boost security through Critical Infrastructure Standards and new vendor requirements. "But nobody was expecting the administration to shortcut all that," said Bradley. "It throws a wrench into what's going on right now."
DOE says it is moving quickly on the issue and the order gives 150 days — until September 28 — for the agency to finalize new rules. Experts say it will be a challenge for DOE to meet that deadline.
"There is enormous uncertainty about the terms in the executive order and to what they apply. It will be difficult to narrow down quickly," Mike Leiter, a partner at Skadden, Arps, Slate, Meagher & Flom, told Utility Dive.
In the meantime, utility companies are making their best estimates of what equipment could be covered by the order.
"A lot of people are continuing to procure Chinese solar panels on the basis they don't think they're covered or they pose little risk to the bulk power system," Skadden partner Lance Brasher said. With other equipment, such as inverters, "people are a little more worried" and in some cases have been opting for non-Chinese equipment when they have a choice.
The order "is having some procurement effect in shifting away from vendors that appear to be of greater risk," Brasher said.
What is covered by the executive order?
According to Bradley, a point of confusion is the administration's use of the term Bulk Power System.
The administration likely drew the term "Bulk Power System" from the Energy Policy Act of 2005, said Bradley, where it defines the scope of NERC's reliability jurisdiction. But it isn't well defined, he said.
The executive order says it will apply to "electric energy from generation facilities needed to maintain transmission reliability." But which are those? The order "leaves many unanswered questions," said Bradley. Some of those could be answered through additional orders issued over the coming months, he said.
Fortress' Whitney, who formerly worked at NERC and the Electric Power Research Institute, believes the bulk power system is relatively well-understood by industry. However, he said there are gray areas and greater clarity is needed to turn the executive order into enforceable regulations.
"The devil is going to be in the details, and helping utilities understand what is the scope not only in terms of large assets, but within technology and subcomponents, where to draw the line."
Vice President of Energy Security Solutions, Fortress Information Security
The big question, he said, will be in how far DOE drills down to find problematic equipment.
"There are suppliers of suppliers. The question is how far do we go," Whitney said. "All the way down to the microprocessor? The chip? The mother board? ... The devil is going to be in the details, and helping utilities understand what is the scope not only in terms of large assets, but within technology and subcomponents, where to draw the line. There needs to be much more fidelity around that conversation."
Fortress runs the Asset to Vendor Network, which helps utilities better understand their supply chains and what equipment might run afoul of the order. "There very well may be a 'do not buy from' list, based on criticality of the asset," Whitney said. "A lot of those details are yet to be determined."
The executive order also contemplates a "white list" of pre-screened vendors and equipment, but so far there has been nothing concrete from DOE.
"Reading the executive order, it sounded like the white list could come out before the regulations," said Brasher. But from DOE calls, he said it appears the background work has not been done and the white list won't be ready in the short term. "It sounds like [the white list] is not necessarily on a path different from the regulations themselves."
In a statement to Utility Dive, DOE said it is currently working on a notice of proposed rulemaking (NOPR), "which will provide an opportunity for stakeholder comment and input on the substance of the rule. It is anticipated that the issue of pre-qualification will be addressed in the NOPR."
How long to generate regulations?
Energy law experts say they doubt DOE will be able to meet the 150-day deadline to generate regulations, leaving the utility sector in a state of flux for now.
DOE has indicated it wants to move quickly, but "a rulemaking of this complexity in normal times might typically take a few years," said Bradley. "DOE doesn't do things that fast," and meeting the 150 day deadline is unlikely.
"We're also entering an election year, and that never accelerates things,"
Partner, Skadden, Arps, Slate, Meagher & Flom
The process will likely include private consultations with industry and other stakeholders, followed by proposed rules, a comment period and then final regulations, said Bradley. During that time, utilities can't just put maintenance and expansion plans on hold. "If you run a utility, you are constantly repairing and refurbishing equipment. On a regular basis, people are replacing transformers," said Bradley.
It does not appear there was much preparation going on behind the scenes before the executive order was issued, Skadden's Leiter said. So while DOE is "not quite starting from square one, they have a lot of work to do."
"We're also entering an election year, and that never accelerates things," Leiter said.
Possible challenges to the order?
While federal regulations almost always face some kind of challenge, Trump has broad authority to act on national security concerns.
"I'm not sure there are any regulations adopted without a challenge," said Leiter. "As a general matter, because these regulations are based on national security concerns, that tends to provide the executive branch with a very significant amount of discretion on both substance and process. I expect there will be challenges but I also expect the success of those challenges to be somewhat limited given the subject matter."
There will probably be no challenges directly to the executive order, said Bradley, because "challenges to Presidential orders are very hard and they mostly fail. But there will almost certainly be litigation around the content of DOE's rule, whenever that happens."
And there are political questions because of the upcoming election, said Bradley.
"There's pretty much zero prospect DOE will have the rule done before January," he said. "If the administration does change, I don't know what Biden will do with this. Presumably it gets refined in a way to be more palatable to industry."
If Trump doesn't win, said Leiter, "there could be questions about whether and how the rulemaking will continue."
How much will it cost?
A key question is how much this directive will wind up costing utilities.
"Naturally, this will lead to cost increases," Ollie Whitehouse, global chief technology officer at cybersecurity consultancy NCC Group, told Utility Dive. The countries inferred in the order "have the ability to undercut pretty much every western country on technology."
"The immediate disruption will be a slowdown in [both] capital projects and the adoption of some technologies," Whitehouse said.
But it is still too soon to estimate the cost of Trump's executive order — though Whitney said those costs may wind up being passed along to consumers if equipment must be removed from the grid.
"If there are potentially sweeping changes then costs are a huge component," Whitney said. "There could be some type of support to help offset those costs, capitalize those costs, potentially passing them on to the consumer."
Alternatively, the federal government could offer incentives to utilities to take any necessary steps.
"You can get a lot more done with carrots than sticks," said Whitney. "There is mandatory compliance, versus incentives to replace equipment, which gives utilities flexibility."
DOE says its understands the industry is concerned and plans to address the issue.
"The Department recognizes complexity of the bulk-power system and the accompanying supply chain risk management, and we intend to take a phased, thoughtful approach, working closely with industry, to avoid any unintended consequences," the agency said in a statement.
Article top image credit: Getty Images
Enel ransomware attack highlights the value of a top-down security culture
By: Robert Walton
Utilities in the United States face a growing threat from ransomware attacks, which create financial incentives for hackers beyond potential disruptions to the power grid. In response, energy companies are making security a priority — and increasingly see it tied to their own corporate culture and bottom line.
A new survey from cybersecurity firm Mimecast found more than half of energy sector respondents were impacted by ransomware in the past year, and almost three-quarters say it is "inevitable or likely" they will suffer from an email-borne attack this year.
Hackers "focus on [return on investment] just like everyone else," Matthew Gardiner, Mimecast’s cybersecurity strategist, told Utility Dive.
A June attack on multinational energy company Enel Group is evidence of the ransomware trend — as well as how companies can counter the threat.
Cybersecurity begins with the board
While the North American Reliability Corp. develops rules and regulations that set baseline security protocols, experts say a compliance-based approach does not go far enough and utilities must be incentivized to roll out best practices that can keep pace with rapidly changing threats.
Security is most effective when it begins with organizational leadership, according to the World Economic Forum (WEF).
Addressing the growing threat will require organizations to focus on security as a business continuity and resilience issue — with boards of directors ideally suited to instill a "culture of shared cyber-risk ownership," according to a June WEF report.
"Only the board of directors can instill the cultural shifts and motivate the organizational shifts that must take place to ensure cyber resilience," WEF concluded in its report.
The report lays out a host of recommendations for cyber resilience. Among them: Assign primary oversight of security to a permanent board committee and assign accountability to a senior corporate officer who "is charged with responsibility for governance and oversight of cybersecurity strategy across the IT and OT environments."
Mimecast's Gardiner sees a growing trend of security-minded executives sitting on boards, similar to audit committees whose members include financial experts.
Enel Group was highlighted by WEF as an example of a company that has established a security culture — it established a Cyber Security Risk Committee with its CEO acting as chairperson.
That didn't stop the company from suffering a cyberattack last month — but Enel officials and cyber experts say the company's approach to security may have helped it escape the incident relatively unscathed.
Enel's security culture mitigated ransomware attack, say experts
"Despite an insidious and potentially very damaging attack, the speed and effectiveness of Enel's response ensured that the impacts on business processes were irrelevant," an Enel spokesperson told Utility Dive in an email.
Enel identified the ransomware attack on June 7, following a disruption to its internal IT network. The company said it temporarily isolated its corporate network and restored it the following day. "Temporary disruptions" to customer care activities were possible during the downtime, but the company said the impact stopped there.
Enel said no critical issues concerning the remote control systems of distribution infrastructure and power plants were registered, customer data was not exposed and all internal IT services "were rapidly and efficiently restored, allowing all business activities to run smoothly."
Enel credits its response to efforts begun in 2016 when the company adopted a "structured and systemic approach on cyber security" that included establishing an organizational and procedural framework that set roles and responsibilities for all involved with security. The CEO-chaired committee "has the dual purpose of aligning cyber intervention priorities with corporate strategies," the company said.
"The recent increase in the frequency of attacks on utility sector players underscores the absolute need for this industry to make a change of pace in the active management of cyber risk," the company said.
Enel's work on cybersecurity is "a good example of embedding security strategy with business strategy," Matthew Selheimer, chief marketing officer at cybersecurity firm PAS Global, told Utility Dive. But he disagreed with the WEF report's conclusion that only the board of directors can motivate the corporate culture shifts needed to ensure security.
"Leaning solely on the board of directors to instill cultural shifts and motivate organizational shifts towards cyber resilience can create its own issues," Selheimer said. "Driving [security] solely from the board down can lead to a minimal effort to 'check the box' in order to meet the board requirements."
Pairing security compliance with financial incentives
"Cyber resilience is made much stronger when it is linked to specific business objectives," Selheimer said, including reliability of operations, profitability or the ability to drive competitive advantages through digital transformation. "To make this viewpoint pervasive, board sponsorship helps, but having senior executive sponsorship and oversight by the [Chief Information Security Officer], [Chief Information Officer] and the head of operations is key."
While Enel could not prevent the attack, Shawn Wallace, vice president of energy at IronNet Cybersecurity, told Utility Dive it appears the company's security efforts spared it from potentially-expensive losses.
Enel "has moved to a posture of resilience (as opposed to compliance or risk mitigation) and I have to believe this was a contributing factor in their ability to prevent the attack," Wallace said in an email. Having the right corporate governance and culture "will determine if a company views cybersecurity in terms of resilience. Organizations need a champion, be it on the Board of Directors or another senior leader, to carry this forward. Otherwise, they are bound to revert to just compliance."
Federal regulators have recognized this, and are considering steps to give utilities financial incentives to implement stronger security on top of what is required by critical infrastructure protection standards. Last month, the Federal Energy Regulatory Commission issued a white paper contemplating transmission incentives for utilities making voluntary cybersecurity improvements.
"Although regulation is helpful, compliance does not mean a system is secure forever," Joe Saunders, founder and CEO of RunSafe Security, told Utility Dive. "Technology changes faster than regulations adapt, so the best companies will adopt a culture that values security. ... It is also important to rethink software development and deployment practices, aligning development with operations and building security into products while not just relying on perimeter defenses."
According to Enel, the recent increase in the frequency of attacks on utility sector players "underscores the absolute need for this industry to make a change of pace in the active management of cyber risk."
Companies must organize themselves "not only internally, but also by participating in multi-stakeholder initiatives, with public and private players, in order to create an ecosystem which is secure by design on products and processes," Enel said.
Utilities say they are prepared to meet cyber threats. Are they?
Budgets are rising and power companies are focused on exceeding the baseline security standards required by energy-sector regulation.
By: Robert Walton
The power sector in the the United States faces a growing cybersecurity threat from increasingly sophisticated hackers, but a new survey from Utility Dive shows companies generally believe they are well prepared. And experts concur, pointing to rising spending levels related to security, the expansion of security standards and information sharing across the industry.
But despite industry confidence, Utility Dive's 2020 State of the Electric Utility (SEU) survey reveals there is still much work to be done and state and federal regulators are stepping up to help energy providers address the threat.
"Utilities take security very seriously," Alex Santos, CEO of Fortress Information Security, told Utility Dive. "They have the right people in place and are getting support from upper management, whether with dollars, resources or attention."
Almost two-thirds of respondents to the SEU survey indicated their organization has increased its budget for digital operations and security. But while 62% say they have qualified personnel and updated systems in place, 38% still have issues that need to be addressed.
While specific figures are hard to come by, a new report from Navigant bears out findings that security budgets are on the rise.
Navigant on Tuesday issued a report estimating the global market for energy IT and cybersecurity software and services will surpass $19 billion in 2020 and reach more than $32 billion in 2028. Only about 7% of that is security related, said Navigant research analyst Michael Kelly. But the security component is growing faster than the overall anticipated 6.6% growth rate.
The utility sector has seen an increase in security spending since 2015, when Ukraine's electric grid was hit by a cyberattack that led to a lengthy blackout for almost 250,000 people.
"The Ukraine event certainly woke the industry to the potential impacts of a large-scale event," Kelly told Utility Dive.
The Edison Electric Institute (EEI), which represents investor-owned utilities (IOUs), does not track actual cybersecurity spending. But "anecdotally, the trend line is definitely increasing," Scott Aaronson, the industry group's vice president of security and preparedness, told Utility Dive.
"And it's not just cyber, but critical protection budgets generally," Aaronson said.
Some of that spending is going towards in-house talent, along with partnerships, vendors, suppliers and contractors. "We're thinking about protection of the grid in a more holistic way," Aaronson said. "Security budgets generally are increasing."
In Michigan, regulators have developed rules to ensure close collaboration between the Public Service Commission and energy providers. An interconnected electric grid, increased asset digitalization and materials sourced internationally "really presents some risk," Chairman Sally Talberg said during a Utility Dive webinar on Wednesday to present the SEU results.
Michigan has put in place rules for small and large gas and electric utilities, Talberg said. "Our technical staff meets with them to examine their plans, their mitigation methods, their funding for these programs, to protect against cyber risk." The commission has also instituted incident reporting rules "to protect this information so we have more of a free flow discussion with regulated utilities and can support them in addressing this risk," she said.
Security experts say the utility sector is focused on keeping hackers at bay, despite growing threats.
"Overall, cybersecurity is a topic utilities take incredibly seriously. I don't think you'll find an IOU or muni that isn't paying attention, and they are always looking at what is the right level of investment," Sharon Chand, a principal with Deloitte & Touche's cyber risk services, told Utility Dive.
Chand said her research broadly shows utilities increasing both cybersecurity headcounts and budgets, but she acknowledged there are limits on their resources. "Utilities are always focused on value to shareholders," she said. "There is not an unlimited pool of dollars to pull from. Sometimes looking for efficiencies is the right answer."
Those efficiencies can come in many forms, including information sharing between utilities and Robotic Process Automation (RPA), where automated processes in the form of bots, machine learning and artificial intelligence take a lead role in protecting the electric grid.
A key area for RPA is managing network access certifications, said Chand. Machines can take on the task of monitoring access "cheaply, efficiently and, one could argue, more effectively," she said.
The growing use of RPA can also help the energy sector deal with a cybersecurity workforce shortage, said Chand, which "utilities can sometimes feel more acutely."
"It may be hard for a muni in middle of the country to compete for top talent," said Chand. "We see utility clients today with bots helping to operate controls, control monitoring, and achieve efficiencies. There is also a focus on artificial intelligence, though she sees "more opportunities in the future" as the technology matures.
Cybersecurity spending is outpacing IT spending, especially at utilities and telecommunications firms, according to James Evelyn, vice president of compliance solutions for security firm Force 5.
"The bulk of the spending seems to be in staffing. A lack of qualified cybersecurity professionals that can implement and maintain security systems is driving staffing costs," Evelyn told Utility Dive. While regulations are forcing the implementation of security technology, "utilities are struggling to correctly implement and maintain these technologies to ensure their effectiveness."
Leveraging vendors’ services helps bring expertise and resources to "complete the missing pieces of their defense programs," Evelyn said.
Aaronson said a trend in the last decade has been for utilities to hire Chief Security Officers from the law enforcement community. "It's how you approach any risk ... you look at what the priorities are, and resource them accordingly," he said.
Standards ≠ security
Another key to improving cybersecurity in the utility sector is a growing focus on standards, in particular Critical Infrastructure Protection (CIP) standards set by the North American Electric Reliability Corp. to help ensure the reliability of the bulk power system.
"Not every organization is in compliance 100% of the time, as CIP audit violations attest, but many utilities have significantly increased their annual NERC budgets in an effort to increase compliance," Force 5's Evelyn said. The six regional entities "are also coming alongside NERC to help the utilities in each region develop and maintain a culture of compliance."
But experts also warn that compliance is not synonymous with security, which Utility Dive's annual survey also reflects.
"In 2020, an overwhelming majority of participants (84%) said they believe their organization is now fully or mostly prepared to address cyber threats," SEU2020 found. But a smaller number, less than 60%, "believe their organization is in or approaching compliance with government cybersecurity mandates."
"I'm not surprised by the survey results. You need to decouple security from compliance," Shawn Wallace, vice president of energy at IronNet, told Utility Dive in an email.
Wallace views NERC's CIP standards as a compliance program mostly centered around asset identification, policies and procedures. "A utility can have a robust cybersecurity program and still be out of compliance with NERC CIP; the same is true for the opposite. You can't be trapped into believing that because you have a strong NERC CIP compliance program that your security is good to go."
"The CIP Standards are important for creating a foundational level of security, but compliance with standards is never enough to be secure," he said.
NERC does not release statistics regarding compliance with CIP standards, but does conduct regular audits of utilities through its Compliance Monitoring and Enforcement Program. If instances of noncompliance are found, a mitigation plan is developed and violations may be assessed. The violations, including fines when appropriate, are filed with the Federal Energy Regulatory Commission.
Regulation creates "a minimum acceptable standard, a floor to minimize the number of weak links," Santos said. But he added that the power sector tends to be ahead of standards.
"Utilities have become one of the more secure industries," Santos said. "They have a high security IQ, and are ahead of compliance in many ways. Regulated industries overall have better security than non-regulated, so it is important to have these safeguards."
NERC critical infrastructure standards are useful, according to Greg Conti, senior security strategist at IronNet, "but with any such program, compliance doesn’t mean an organization is fully prepared."
"The key differentiator is how companies innovate and implement cybersecurity on top of baseline standards," Conti said. Large utility companies can lack sector-wide visibility of threats, he said, while smaller municipal and cooperative power companies have limited cybersecurity budgets and fewer staff.
"I’d like to see all members of the sector defend more as a team rather than as individual enterprises to complement NERC compliance," Conti said. He pointed to the Electricity Information Sharing and Analysis Center, mutual support agreements and inter-company collaboration, as areas where the industry is making progress.
"I believe a team-based approach that complements and transcends compliance is absolutely necessary," Conti said.
While the electric sector is well-acquainted with the concept of "load balancing," Santos said the same phrase and idea also applies to risk.
"The utility industry in particular takes care of itself from the perspective of large companies trying to partner with smaller companies," said Santos."Share the risk, share the burden. The industry is extremely collaborative."
Santos said he has worked in a variety of sectors, including health care and finance. "Utilities are by far the most collaborative of any industry, bar none, whether it is a hurricane [response], spare parts or security."
Article top image credit: Ruslan Dashinsky and tampatra via Getty Images
First cyberattack on solar, wind assets revealed widespread grid weaknesses, analysts say
New details of a denial-of-service attack earlier this year show an energy sector with uneven security.
By: Robert Walton
A March 5 cyberattack of U.S. wind and solar assets is back in the news, with fresh documents helping shed light not just on the extent, but also the simplicity of the first-of-its-kind intrusion. Cybersecurity experts say it reveals a utility sector not sufficiently vigilant, and failing to employ the most simple fixes.
Owned by AES and AIMCo, sPower bills itself as the United States' largest private owner of operating solar assets. Though there was no loss of generation, the March cyberattack impacted the company's visibility into about 500 MW of wind and PV across California, Utah and Wyoming.
Security experts say the attack is a wake-up call for the electric sector and a sign that clear vulnerabilities remain.
"The news begs a bigger question about cybersecurity regulations for the energy industry," Phil Neray, vice president of security firm CyberX, said in an email. "The manner in which it was carried out was very basic — exposing some essential weaknesses in the way energy companies currently patch and monitor their network devices."
Utilities must do basic security maintenance
CyberX released a report last month that concluded utility networks and unmanaged devices are "soft targets for adversaries." Many utilities use outdated operating systems and unencrypted passwords that leave them vulnerable, the firm found.
That means in some instances utilities are not even maintaining the most basic of protection: keeping systems up to date.
"The simplicity of this attack should make generators sit up and take notice."
Chief information security officer, PAS Global
Neray said the grid is made vulnerable by network appliances like the ones that were compromised in the attack on sPower: directly exposed to the internet, unpatched and with limited malware capabilities. "We’ve seen attackers go after unpatched network devices in the past," he said.
The March 5 attack is "one more example .... that cyber risk in the industrial space is not only real, but operant," Jason Haward-Grau, chief information security officer at cyber firm PAS Global, said in an email.
"The simplicity of this attack should make generators sit up and take notice," Haward-Grau said. "This was a ‘simple’ IT attack on an unpatched firewall, which was still vulnerable, in spite of the patch being available."
The documents obtained by E&E show that in the aftermath of the attack, sPower did deploy a firmware update to its firewall and no further issues were seen. The company said there was no intrusion beyond the DoS attack.
Information about the March 5 attack was included in a Department of Energy Form OE-417 which details an "electric emergency incident and disturbance." The attack, eight months ago, was classified as one that "could potentially impact electric power system adequacy or reliability."
Response will be key with growing frequency of attacks
Cybersecurity experts say attacks on the electric sector are becoming more common, and that trend will continue along with a growing sophistication.
"The frequency of attacks are continuing to grow and digitalization and hyper-connectivity are only going to expand the risk," Haward-Grau said. "Hackers are getting more and more sophisticated about industrial operations attacks."
There is also a clear need for response and recovery planning capabilities, said Haward-Grau, despite the specifics of the March attack where a firewall patch provided quick respite. "This could have been significantly worse had the attacker understood what they were dealing with and gone further with their attack," he said.
It is "highly unlikely that attackers could take down the entire U.S. power grid."
NERC is scheduled to hold its biennial GridEx event next week, which simulates a grid attack in order to give the utility sector an opportunity to run through response protocols. While the March attack did not impact grid operations, hackers have previously disrupted the Ukraine grid and experts say it may be an inevitability for the United States.
"Impacting operations is only a matter of time," Haward-Grau said. "If a simple firewall crash can do this, imagine what a dedicated and skilled attacker can do. ... the emphasis needs to shift to not just identifying an attack but equally important responding to one."
It is "highly unlikely that attackers could take down the entire U.S. power grid," CyberX's Neray said. The bulk system has been specifically designed to eliminate single points of failure. However, he said it is not difficult to imagine other scenarios.
"It’s easy to imagine how determined nation-state attackers could target specific population centers to cause major disruption and chaos," Neray said. He pointed to the Ukrainian grid attacks of 2015 and 2016, which left hundreds of thousands without power.
Last year, the U.S. Department of Homeland Security concluded that Russian government cyber actors had targeted and compromised "government entities and multiple U.S. critical infrastructure sectors, including the energy, nuclear, commercial facilities, water, aviation, and critical manufacturing sectors."
"This is not completely theoretical," Neray said. "Organizations should be on high-alert for similar incidents."
Article top image credit:
Natural gas ransomware attack offers critical lessons for electric utilities, analysts say
By: Robert Walton
A ransomware attack shut down a natural gas compressor station for two days causing a "loss of productivity and revenue," according to an alert last week from the U.S. Cybersecurity and Infrastructure Security Agency (CISA).
The disruption represents a growing threat to the domestic energy sector, with more sophisticated attacks beginning to target the industrial control systems (ICS) which help to run electric grids and pipeline systems. The compressor station attack began on the information technology (IT) side of a pipeline company's operations, but spread to the operations technology (OT) side because of a lack of system segmentation, experts say.
Security analysts have seen a growth in malware targeting the OT side of energy operations and a new report from Dragos finds the cyber risk to ICS networks "continues to grow and remains at a high level."
IT to OT threats
Dragos's review of 2019 cyber threats found a growing number of hacker groups targeting ICS systems, and warned "ransomware and other malware infections continue to be a major issue across industrial operations."
That was the technique used by attackers to shut down operations at a gas pipeline compressor station, according to CISA. While the ransomware obtained initial access to the organization’s IT systems through a spearphishing attack, it was then able to "pivot" to the OT side, the agency said.
"Impacted assets were no longer able to read and aggregate real-time operational data," according to the government analysis of the attack. "Although the victim’s emergency response plan did not specifically consider cyberattacks, the decision was made to implement a deliberate and controlled shutdown to operations."
Security experts say the attack can serve as a warning for electric utilities — though they should have already insulated themselves from similar intrusions.
The malware spread from IT to OT "due to lack of network segmentation, which all electric utilities should already have in place," Phil Neray, vice president of industrial cybersecurity for security firm CyberX, told Utility Dive.
Electric utilities have the benefit of many years of critical infrastructure protection (CIP) standards developed by the North American Electric Reliability Corp. (NERC), Neray said. Those standards have helped to raise awareness about cyber risks and minimum best practices.
"But security professionals know that passing a compliance audit does not necessarily mean you're secure," Neray said, warning that electric utilities "should constantly be examining their security controls to ensure they've reduced the risk to an acceptable level."
Expert: Oil, gas operations may be more exposed than electric utilities
NERC does not release statistics regarding compliance with CIP standards, though it does audit utilities through its Compliance Monitoring and Enforcement Program. Utility Dive's 2020 State of the Electric Utility survey found less than 60% of respondents "believe their organization is in or approaching compliance with government cybersecurity mandates."
Natural gas systems are more automated and there can be unattended remote devices along the entire length of a pipeline, RunSafe Security CEO Joe Saunders told Utility Dive in an email.
"Most utilities don’t have IP enabled smart grid at any scale and SCADA is a little harder to attack," Saunders said. "But as they shift, they need protection in this area." Older operating and information systems need to be protected or upgraded, he added.
Oil and gas operations are likely more at-risk than the electrical generation facilities, according to Richard Henderson, head of global threat intelligence at cybersecurity firm Lastline. But he stressed that the successful compressor station attack was a failure of the organization to establish hard boundaries between its IT environment and its OT environment.
"It’s just as likely that this was a crime of opportunity by an overseas attacker than a targeted attack against a critical infrastructure player" Henderson told Utility Dive. "If you do not have very clear delineation and segregation from your regular IT infrastructure to the OT infrastructure where all your operations run, attacks like this are not an if, but a when."
Cybersecurity experts at the time said it revealed the utility sector was not sufficiently vigilant, as it struck a known vulnerability in an unpatched Cisco firewall.
Organizations need to begin immediately "getting their OT systems in a better place security-wise," Henderson said.
"It’s one thing when the HQ office PCs all go down from a commodity ransomware infection. It becomes something else altogether when things at the plant start doing things they shouldn’t," Henderson said.
'Chokepoints' can limit malware spread: Dragos security recommendations
Following news of the compressor station hack, Dragos released a series of recommendations for asset owners and operators to implement "to prevent the infection and spread of ransomware that could potentially impact ICS operations."
"Aggressively monitor outbound communications from ICS networks to identify signs of infection events within OT space," Dragos said.
The security consultant also recommended: training employees to recognize and respond to phishing campaigns; developing strong network defenses between the IT and OT networks; creating "chokepoints" to limit malware spread; and keeping anti-virus signatures up to date.
In the compressor station hack, operational impacts were likely caused by a combination of insufficient segregation of IT and ICS environments and shared Windows operating system infrastructure, Dragos said. That allowed the impacts to spread beyond the attackers' initial targets.
"Even though CISA reporting indicates only one compression facility was directly targeted, overall pipeline operations ceased for two days during restoration from backup operational data and stored configuration files," the firm warned.
"Aggressively monitor outbound communications from ICS networks to identify signs of infection events within OT space," Dragos also said.
In the 'cat and mouse game' of utility cyberattacks, AI and machine learning show promise, limits
"There is so much low hanging fruit in utility system challenges that can be addressed by advanced data analytic strategies that AI and machine learning are not the place to start."
By: Herman K. Trabish
If somebody hacked communications to grid-connected devices and interrupted a demand response (DR) event, peak demand might not be cut, capacity prices could spike and that somebody could make a lot of money.
Because of the fast-rising number of grid-connected devices in DR programs like smart thermostats and water heaters and the even faster-rising number of smart phones and other Internet technologies through which customers communicate with DR programs, market manipulations like that are possible, cybersecurity experts from the Electric Power Research Institute (EPRI) told the Demand Response World Forum October 17. It is one of many potential intrusions of communications between utilities and customers with grid connected devices and distributed energy resources (DER), they said.
To counter these threats, data analytics experts are using the laws of physics and unprecedented masses of data to find cybersecurity breaches. And their work is leading to machine learning (ML) and artificial intelligence (AI) algorithms which, though only just beginning to find actual deployment, are expected to soon advance the ability to identify patterns to the intrusions and raise the level of protection for critical power systems.
"There are two types of cybersecurity threats to utilities — those that affect system hardware and connected devices and those that affect utility internet systems,"
"There is so much low hanging fruit in utility system challenges that can be addressed by advanced data analytic strategies that AI and machine learning are not the place to start."
Cybersecurity Manager, OSIsoft
Bryan Owen told Utility Dive. "We worry about the ones that can result in physical harms, but high levels of connected devices are making them an important cybersecurity issue now, too."
Collaborations between utilities and researchers at U.S. national labs are revealing new protections but also new challenges to secure data management that must be addressed. ML and AI can help utilities protect themselves and their customers from cyberattacks, but only if utilities can allow these new allies access to the data and systems that need protection.
Connected devices. exceeded the world's 7.3 billion population in 2014, EPRI Senior Program Manager Rish Ghatikar told the DR conference. At the 31.7% growth rate of U.S. connected homes reported by McKinsey for 2017, there were "over 1 billion connected home devices" in the U.S. by the end of 2018, he estimated. Each of those homes could have as many as 10 grid-connected devices, ranging from laptops and wireless modems to smart thermostats and smart inverter-based solar plus storage systems.
Homes and businesses have their own "device ecosystems" of internet technology (IT) devices, smart thermostats, water heaters, other smart appliances, EV chargers, and smart inverter-based solar and batteries that utilities do not control, Ghatikar said. "They cannot manage what they cannot control, which makes the risk of intrusions exponentially higher and cybersecurity more relevant."
Cybersecurity for ICS is "a cat and mouse game where adversaries and defenders continue to evolve with one another."
Research Scientist, Sandia National Laboratory
EPRI is developing a framework to identify cybersecurity assets, threats, key players, and architectures on which researchers, utilities and device vendors can focus to find solutions, EPRI Engineer Scientist Alekhya Vaddiraj told the conference.
Cyberattacks can compromise reliability or resilience, impose financial losses, appropriate customer data, or threaten customers and system safety, she said. Although it is a "futuristic" scenario, the concept of false information or disrupted communications being used to manipulate power markets is also a realistic possibility, she and Ghatikar agreed.
But an estimated 1,500 outages have been caused by wildlife and only three by human attacks, according to CyberSquirrel, OSIsoft's Owen said. His company remains focused on protecting systems from the more common types of intrusions, though it continues to monitor the progress of ML and AI deployment.
"There is so much low hanging fruit in utility system challenges that can be addressed by advanced data analytic strategies that AI and machine learning are not the place to start," he said. Yet advanced work is already underway, he acknowledged.
AI is the big circle of a Venn diagram of computer operations, Sandia National Laboratory Research Scientist Adrian Chavez told Utility Dive. ML is "a subset of AI within the circle. AI is a fully automated system that can respond on its own, even to unknown attacks, and machine learning is a component that enables that response."
Cybersecurity for ICS is "a cat and mouse game where adversaries and defenders continue to evolve with one another," Chavez said. The adversaries that must be defended against have "a high level of sophistication" in "multiple fields spanning cybersecurity, physical security, and power engineering."
The "signature-based intrusion detection system" Chavez is developing monitors DER communications for "very specific strings of characters or bytes with known unique characteristics," he said. "It can automatically alert a grid operator of a potential malware intrusion if those strings are observed."
A more complex, "behavior-based" ML algorithm can use "statistics and past trends" to "analyze network communications for abnormal behavior," Chavez said. "If it recognizes abnormal patterns with a statistical probability of attack, even from an unknown intrusion, it can respond in real time or, if it does not have a mitigation strategy, alert an operator."
Utilities "are increasingly interested in how AI algorithms and deep learning can automate the protection of customer information, the optimization and balancing of the grid, and the finding of efficiencies in the details of customer usage."
A behavior-based algorithm "that autonomously identifies the attack, protects against the attack, and recovers from the attack without operator intervention, fits my definition of AI," he said. His algorithms have been "tested and verified within a 2.5 MW microgrid environment" but not "at a large utility scale."
The private sector has already deployed AI for purposes ranging from cybersecurity to marketing, but change in utilities requires "incremental steps" that demonstrate ML and AI can readily detect abnormal events without system disruption, Chavez said. Using ML and AI could take "another five to 10 years," but it could happen sooner because "a lot of people at the national labs are using Department of Energy (DOE) funding to work in this area."
Utilities want modern cybersecurity, but "legacy devices and protocols deployed decades ago are not capable of supporting it," he said. "That makes upgrades expensive and it makes upgrading hardware and software a potential interruption of operations."
"This is a whole new game for utilities," Colin Gounden, CEO of data specialist VIA agreed. "They are increasingly interested in how AI algorithms and deep learning can automate the protection of customer information, the optimization and balancing of the grid, and the finding of efficiencies in the details of customer usage," he told Utility Dive. "But AI requires access, particularly to data."
The difference between AI and ML is that "ML can identify the notes in a piece of music and AI can create music," VIA's Gounden said. "Some problems, like disaggregation of home appliances on a distribution system, are better solved with ML. AI is better for optimizing the distribution system."
Both require an enormous amount of data, but it can be protected the same way email is scanned for spam by an AI algorithm "because it is too big a dataset for a person," Gounden said.
Machines have learned by being given rules, but establishing comprehensive rules for today's complex systems is very difficult, he said. In the past five years, ML has become possible because computing power has increased exponentially, and algorithms can use it to "do more and faster unsupervised learning from the massive datasets in the internet cloud."
AI has shown its potential by advancing voice recognition and autonomous driving capabilities, but its use by utilities for cybersecurity remains limited, Gounden said.
"They have been doing a lot of data science and statistics, but they have not been able to deploy AI because many individual utilities do not have enough data for algorithms to learn from," he said. And there has been no way to securely aggregate data from multiple utilities to allow shared learning while "keeping the data safe and secure and making sure it is used properly."
VIA uses AI to curate a securely aggregated library of utility data for cybersecurity research. The Hawaiian Electric Company (HECO) utilities, New Zealand's Vector, and "many others" are participating, Gounden said. "Interest is increasing because there are so many different ways to steal data." The ultimate goal is an AI algorithm "that can allow data sharing on a system-wide basis" for applications ranging from cybersecurity and customer engagement to predictive maintenance.
Companies like Apple, Google and Amazon are accelerating their ML and AI capabilities because they have huge volumes of system and user data, he added. Utilities' "critical infrastructure standard" for reliability should not prevent them from also using algorithms to protect their systems from cyberattacks, support their human resources, optimize value from DER, and support decision-making.
Skepticism and the laws of physics
Utility members of the National Rural Electric Cooperatives Association (NRECA) are working in DOE-funded pilots to test ML and AI algorithms, NRECA Analytics Research Program Manager David Pinney told Utility Dive. The primary focus is protection of communications to smart inverter-based solar and battery storage systems from cyberattacks.
"In lab work, these devices have been hacked," Pinney said. A recent HECO software update was safely sent to "about 800,000 smart inverters." It was apparently to improve performance and reduce outages, but these pilots simulate the huge impact an update of that size infected with malware could have.
The pilots will identify smart inverter settings not readily destabilized by hackers, he said. "A big part of the solution is using machine learning to train utility networks to repeatedly find new protective settings as hackers change their attacks. The next step is helping our co-ops introduce these protections into their individual systems."
High costs, computational limits, inadequate interoperability standards, and a lack of access to data all contribute to an "appropriate level of skepticism" from co-ops toward autonomous operations, Pinney said. But skepticism raised "great questions" about safety during previous demonstration deployments of advanced technology that led to "a better understanding of the technology," he recalled. "We expect the same to happen with machine learning-based controls."
Utilities' key vulnerability is to cyberattacks on their IT systems, OSIsoft's Owen stressed. The 2010 Stuxnet attack on Iran's nuclear facility came through a thumb drive on a laptop, and Norsk Hydro's $52 million production loss earlier this year started with an intrusion through its email system.
"AI is being used to protect email," he acknowledged. "But anybody who looks at their inbox would probably say it is not working very well."
OSIsoft instead relies on data analysis and the laws of physics to identify cyberattacks, he said. "Electricity moves according to the laws of physics. Our system data on voltage, temperature and other factors can show if a system is acting normal, and if a measurement is out of line with others, it is reported for investigation, whether it is a hack or an equipment failure."
ML "can be very good at learning how a DER device operates, identifying when it is not operating properly, and blocking that operation," Owen said. "But it may encounter an event like a power outage that it has never seen before and respond in the wrong way." An example is a threat described in an August 2018 Princeton University paper, he added.
Aggregated DER could be used "to manipulate the power demand," Princeton researchers reported. Simulations demonstrated that such manipulations could cause "local power outages and in the worst cases large-scale blackouts" or "be used to increase the operating cost of the grid to benefit a few utilities in the electricity market."
The research showed the "vulnerability" of the power grid and similar systems and the need for attention to cybersecurity, the report concluded. Addressing that vulnerability is what AI and ML researchers like Sandia's Chavez hope to do.
It is a vulnerability currently being addressed largely by data analytics and the laws of physics, OSIsoft's Owen insisted. But it is a vulnerability that the growth of connected devices may increase, which is why learning what ML and AI have to offer and "incrementally adding security as we go" is necessary, he acknowledged.
Article top image credit:
NERC struggles to involve utility vendors in its biennial cyberattack simulation, and that's a big problem, analysts say
By: Robert Walton
Only three major electric industry supply chain vendors registered to participate in last year's GridEx V, the biennial simulated cyberattack run by the North American Electric Reliability Corporation (NERC) in order to test the utility sector's preparedness and response plans.
Experts say failure to fully engage the sector's supply chain is worrisome because vendors will likely be targeted by hackers and must be involved in the response to an actual cyberattack.
"Frankly, there's no awareness that we are invited," Alex Santos, CEO of Fortress Information Security, told Utility Dive. "To do a proper readiness exercise, you have to involve the vendors."
NERC officials say the two-day event in November 2019 drew 7,000 participants and achieved most of its objectives. However, the after-action report, released Tuesday, also revealed that some utilities "lacked the resources needed to coordinate responses" to the simulated attack.
GridEx V drew "unprecedented" participation, NERC officials said Tuesday in a call with reporters, but also revealed uneven response capabilities across the utility sector.
"It is a diverse industry, with 3,300-some utilities across the continent — some with tremendous capabilities, and others that are only now building from the baseline of their capabilities to fend off a cyber or major physical attack against the grid," said Matt Duncan, the manager of policy and coordination at NERC's Electricity Information Sharing and Analysis Center. "That's a gap we need to be honest about."
While NERC does not grade individual utility responses to GridEx, the simulation did yield broad recommendations including the need to enhance coordination with communications providers to support restoration and recovery. Another recommendation called for strengthening the operational industry and government coordination between the United States and Canada.
NERC set seven objectives for GridEx V, and achieved six of them including engaging critical interdependencies. The report indicates 16 natural gas utilities, 13 water utilities and three telecommunications companies participated. There were also improvements to local and regional responses: eight National Guard units, 29 field offices of the Federal Bureau of Investigation and 26 state governments participated in the event.
GridEx officials say business competition holds back vendors participation
"It is incumbent upon participating organizations to include supply chain partners in their response plans," NERC's report notes. "Some organizations chose to engage with their supply chain partners during the exercise while others did not."
According to Duncan, some of the difficulty in engaging the utility supply chain is "because it's a different type of business space, a very competitive type of business space."
"It is too critical not to include the vendor community, as these scenarios and supply chains get more complicated."
Manager of policy and coordination, NERC Electricity Information Sharing and Analysis Center
The electric industry benefits from being regulated at multiple levels, with critical infrastructure protection protocols in place and fewer competitive concerns, Duncan said. And utilities have a history of collaborating in response to major events, such as hurricane recovery.
In GridEx IV, vendors were engaged through a helpdesk but that went unused during the exercise. This time, utilities were encouraged to invite their vendors to participate alongside them.
"It's a question of bringing them into the fold and showing them the value of participating, and not making it a sales opportunity," said Duncan. "I think we're going to get there. It is too critical not to include the vendor community, as these scenarios and supply chains get more complicated."
Could a 'contractual obligation' for vendors boost security?
Security experts say involving vendors, including service providers and equipment manufacturers, is vital to preparing the utility sector to respond to attacks.
"It is going to be essential to have private sector vendors participate in responding to an attack in the future," Richard Henderson, head of global threat intelligence at security provider Lastline, told Utility Dive in an email. "Nation-state actors who may decide to target the North American grid are just as likely to target specific vendor technologies as they are to target generalized computing infrastructure."
Each vendor who provides products used in the grid should be required to have detailed response plans in place in the event of an attack causing widescale energy disruption, Henderson said. That could include storing spare equipment located in strategic locations or having highly-skilled incident response professionals on retainer.
"The smaller [utilities] are going to be depending more on the vendors because they don't have all the resources to invest that the big guys do."
CEO, Fortress Information Security
"This should be a contractual obligation going forward," Henderson said.
Having as diverse a group of participants as possible can only help GridEx, said Dave Weinstein, chief security office of industrial cybersecurity company Claroty.
While original equipment manufacturers "aren't the most critical constituency for GridEx, they would absolutely be involved in a real-world situation," Weinstein told Utility Dive. "Because a lot of this technology is proprietary to one vendor or the other, that knowledge base is critical during a response. Of course it's the proprietary nature of the technology that deters participation for competitive reasons."
"Utilities need to be able to secure and support all hardware and software involved during an incident or threat," James Evelyn, general manager of security firm Force 5 Solutions, told Utility Dive. Electric companies "need to simulate attacks happening in real life today. Without vendor participation, they have no idea what vendor support will look like during an incident."
And vendor support will likely be more critical to smaller utilities than larger ones, said Fortress' Santos.
"The larger utilities probably have more functionality," Santos said. "The smaller guys are going to be depending more on the vendors because they don't have all the resources to invest that the big guys do."